Numerical Methods Qualification Exam Problems and Solutions (University of Maryland)/January 2008

Solution 1a
We wish to show that $$ \,\! \|b-Ax(c)\| = \| \nu e_1-H_k c \|$$

$$ \begin{align} \|b- Ax(c) \|	&= \|b-A(x_0+V_kc) \| \\ &= \|b-Ax_0-AV_kc \| \\ &= \|r_0 - AV_kc \| \\ &= \|r_0-V_{k+1}H_k c \| \\ &= \|\nu v_1 -V_{k+1}H_k c \| \\ &= \|V_{k+1}\underbrace{(\nu e_1 -H_k c)}_{h_c} \| \\ &= (V_{k+1}h_c, V_{k+1}h_c)^{\frac{1}{2}} \\ &= ((V_{k+1}h_c)^TV_{k+1}h_c)^{\frac{1}{2}}\\ &= ( h_c^T V_{k+1}^T V_{k+1} h_c )^{\frac{1}{2}} \\ &= ( h_c^T h_c)^{\frac{1}{2}} \\ &= \| h_c \| \\ &= \| \nu e_1- H_k c \| \end{align} $$

Solution 1b
We would like to transform $$H_k\!\,$$, a $$(k+1)\times k\!\,$$ upper Hessenberg matrix, into QR form.

The cost is on the order of $$4k^2+\frac{1}{2}k^2=4.5k^2$$ from the cost of Given's Rotations and backsolving.

Given's Rotations Cost
We need $$k\!\,$$ Given's Rotation multiplies to zero out each of the $$k\!\,$$ subdiagonal entries and hence transform $$H_k\!\,$$ into upper triangular form $$R\!\,$$,

Each successive Given's Rotations multiply on an upper Hessenberg matrix requires four fewer multiplies because each previous subdiagonal entry has been zeroed out by a Given's Rotation multiply.

Hence the cost of $$k\!\,$$ Given's Rotations multiplies is

4k+4(k-1)+\ldots+4\cdot 1 < 4k \cdot k = 4k^2 $$

Back Solving Cost
$$R \!\,$$ is a $$(k+1)\times k\!\,$$ upper triangular matrix with last row zero. Hence, we need to backsolve $$k\!\,$$ upper triangular rows.


 * $$1+2+\ldots+k=\frac{k(k+1)}{2}= \frac{k^2}{2}+\frac{k}{2}$$

Solution 2a
The composite trapezoidal rule is


 * $$ Q_{T,n}(f)=\frac{h}{2}(f(a)+f(b))+h \sum_{k=1}^{n-1} f(x_k) \!\,$$

The error is


 * $$ I(f)-Q_{T,n}(f)=-\frac{(b-a)f^{\prime\prime}(\xi)}{12}h^2 $$

where $$ \xi \in [a,b] \!\,$$.

Derivation of Composite Trapezoidal Error
The local error, the error on one interval, is
 * $$ -\frac{h^3}{12}f^{\prime\prime}(\eta) \!\,$$.

Observe that


 * $$ n \min_{ \eta \in [a,b]} f^{\prime \prime} (\eta_i) \leq \sum_{i=1}^n f^{\prime\prime}(\eta_i) \leq n \max_{\eta \in [a,b]} f^{\prime \prime} (\eta_i) \!\,$$

which implies


 * $$ \min_{ \eta \in [a,b]} f^{\prime \prime} (\eta_i) \leq \sum_{i=1}^n \frac{f^{\prime\prime}(\eta_i)}{n} \leq \max_{\eta \in [a,b]} f^{\prime \prime} (\eta_i) \!\,$$

Hence, the Intermediate Value Theorem implies there exists a $$ \xi \in [a,b]\!\,$$ such that
 * $$ f^{\prime\prime}(\xi)=\sum_{i=1}^n \frac{f^{\prime\prime}(\eta_i)}{n} \!\,$$.

Multiplying both sides of the equation by $$n \!\,$$,
 * $$ n f^{\prime\prime}(\xi)=\sum_{i=1}^n f^{\prime\prime}(\eta_i) \!\,$$

Using this relationship, we have

$$ \begin{align} I(f)-Q_{T,n}(f)	&= \sum_{i=1}^n I_i(f)-Q_i(f) \\ &= \sum_{i=1}^n \left[-\frac{h^3}{12}f^{\prime \prime}(\eta_i) \right] \\ &= -\frac{h^3}{12} f^{\prime \prime}(\xi) n \\ &= -\frac{h^3}{12} f^{\prime \prime}(\xi) (\frac{b-a}{h}) \\ &= -\frac{h^2}{12} f^{\prime \prime}(\xi) (b-a) \end{align} $$

Derivation of Local Error
The error in polynomial interpolation can be found by using the following theorem:

Assume $$f^{n+1}\!\,$$ exists on $$[a,b]\!\,$$ and $$\{x_0,x_1,\ldots,x_n | x \in [a,b] \}. \!\,$$ $$P_n\!\,$$ interpolates $$f\!\,$$ at $$\{x_j\}_{j=0}^n\!\,$$. Then there is a $$\xi_x\!\,$$ ($$\xi\!\,$$ is dependent on $$x\!\,$$) such that $$ f(x)-p_n(x)= \frac{(x-x_0)(x-x_1)...(x-x_n)}{(n+1)!}f^{(n+1)}(\xi_x) \!\,$$ where $$\xi_x\!\,$$ lies in $$(m,M)\!\,$$, $$m=\min\{x_0,x_1,\ldots,x_n,X\}\!\,$$,  $$M=\max\{x_0,x_1,\ldots,x_n,X\}\!\,$$

Applying the theorem yields,


 * $$f(x)-p_1(x) =(x-a)(x-b)\frac{f^{\prime\prime}(\xi_x)}{2} $$

Hence,

$$ \begin{align} E(f)  &= \int_a^b f(x)-p_1(x)dx \\ &= \int_a^b\underbrace{\underbrace{(x-a)}_{>0}\underbrace{(x-b)}_{<0}}_{w(x)}\frac{f^{\prime\prime}(\xi_x)}{2}dx \end{align} $$

Since $$a\!\,$$ is the start of the interval, $$ x-a \!\,$$ is always positive. Conversely, since $$b\!\,$$ is the end of the interval, $$ x-b \!\,$$ is always negative. Hence, $$ w(x) < 0\!\,$$ is always of one sign. Hence from the mean value theorem of integration, there exists a $$ \zeta \in [a,b] \!\, $$ such that


 * $$ E(f)= \frac{f^{\prime \prime}(\zeta)}{2}\int_a^b (x-a)(x-b) dx \!\, $$

Note that $$ f^{\prime \prime}(\zeta) $$ is a constant and does not depend on $$ x \!\, $$.

Integrating $$\int_a^b (x-a)(x-b) dx \!\, $$, yields,

$$ \begin{align} E(f) &= \frac{f^{\prime \prime}(\zeta)}{2}\left(-\frac{(b-a)}{6}\right) \\ &= -\frac{f^{\prime \prime}(\zeta)(b-a)}{12}\!\, \end{align} $$

Solution 2b
STOER AND BUESCH pg 162

INTRODUCTION TO APPLIED NUMERICAL ANALYSIS by RICHARD HAMMING pg 178

The error for the composite trapezoidal rule at $$n\!\,$$ points in $$[a,b]\!\,$$ is

$$E_{n} = I(f) - Q_{T,n} = \frac{-(b-a)h^2}{12} + \mathcal{O}(h^3)\!\,$$

With $$2n\!\,$$ points, there are twice as many intervals, but the intervals are half as wide. Hence, the error for the composite trapezoidal rule at $$2n\!\,$$ points in $$[a,b]\!\,$$ is

$$E_{2n} = I(f) - Q_{T,2n} = 2\cdot\frac{-(b-a)(\frac{h}{2})^2}{12} + \mathcal{O}(h^3) = \frac{-(b-a)h^2}{24} + \mathcal{O}(h^3)\!\,$$

We can eliminate the $$h^2 \!\,$$ term by choosing using an appropriate linear combination of $$E_n\!\,$$ and $$E_{2n}\!\,$$. This gives a new error rule with $$h^3 \!\,$$error.

$$ \begin{align}

E_{n} - 2E_{2n} &= \frac{-(b-a)h^2}{12} - 2\cdot\frac{-(b-a)h^2}{24} + \mathcal{O}(h^3) \\ &= \mathcal{O}(h^3)

\end{align} $$

Substituting our equations for $$E_n\!\,$$ and $$E_{2n}\!\,$$ on the left had side gives

$$I(f) - Q_{T,n} - 2I(f) - 2Q_{T,2n} = \mathcal{O}(h^3)\!\,$$

$$I(f) = 2Q_{T,2n} - Q_{T,n} + \mathcal{O}(h^3)\!\,$$

If we call our new rule $$\hat{Q}\!\,$$ we have

$$\hat{Q} = 2Q_{T,2n} - Q_{T,n}\!\,$$ whose error is on the order of $$\mathcal{O}(h^3)\!\,$$

Solution 3a
Let $$\tilde{A}_k \!\,$$ be the $$\sigma_k \!\,$$-shifted matrix of $$A_k \!\,$$ i.e.


 * $$\tilde{A}_k=A_k-\sigma_kI= \begin{bmatrix}

a_{11}-\sigma_k     & a_{12}      \\ a_{21}     & a_{22}-\sigma_k \end{bmatrix}, $$

$$Q_k^T \!\,$$ is an orthogonal 2x2 Given's rotations matrix. $$Q_k^T \!\,$$'s entries are chosen such that when $$Q_k^T \!\,$$ is pre-multiplied against the 2x2 matrix $$\tilde{A}_k \!\,$$, $$Q_k^T\!\,$$ will zero out the (2,1) entry of $$\tilde{A}_k \!\,$$ and scale $$\tilde{A}_k \!\,$$'s remaining three entries i.e.


 * $$Q_k^T \tilde{A}_k= \begin{bmatrix}

*     &  *      \\  0      &  *      \end{bmatrix}=R_k $$

where $$*\!\,$$ denotes calculable scalar values we are not interested in and $$ R_k \!\,$$ is our desired upper triangular matrix sought by the QR algorithm.

Since $$Q_k^T \!\,$$ is orthogonal, the above equation implies

$$ \begin{align} Q_k^T \tilde{A}_k &=R_k \\ Q_kQ_k^T \tilde{A}_k &= Q_kR_k \\ \tilde{A}_k &=Q_kR_k \end{align} $$

The Given's rotation $$Q_k^T\!\,$$ is given by


 * $$ Q_k^T =\begin{bmatrix}

c & s \\ -s & c \end{bmatrix} $$

where

$$ \begin{align} c  &=  \frac{a_{11}-\sigma_k}{r} \\ s  &=  \frac{a_{21}}{r} \\ r  &=  \sqrt{(a_{11}-\sigma_k)^2+a_{21}^2} \end{align} $$

Taking the transpose of $$Q_k^T\!\,$$ yields


 * $$ Q_k =\begin{bmatrix}

c & -s \\ s & c \end{bmatrix} $$

Solution 3b
From hypothesis and part (a),

$$ \begin{align} A_{k+1} &= R_kQ_k+\sigma_kI \\ &= \begin{bmatrix} * & * \\            0 & r_{22} \end{bmatrix} \begin{bmatrix} c & -s \\ s & c           \end{bmatrix} + \begin{bmatrix} \sigma_k & 0 \\ 0       & \sigma_k \end{bmatrix} \end{align} $$

Let $$A_{k+1}(2,1) \!\,$$ be the (2,1) entry of $$A_k \!\,$$. Using the above equation, we can find $$A_{k+1}(2,1) \!\,$$ by finding the inner product of the second row of $$R_k \!\,$$ and first column $$Q_k \!\,$$ and adding the (2,1) entry of $$\sigma_k I \!\,$$ i.e.

$$ \begin{align} A_{k+1}(2,1) &= (0 \cdot c + r_{22} \cdot s) + 0 \\ &= r_{22}s \end{align} $$

We need to find the value of $$r_{22} \!\,$$ so we need to calculate $$ R_k \!\, $$.

From hypothesis and the orthogonality of $$Q_k\!\,$$, we have

$$ \begin{align} A_k-\sigma_kI          &=  Q_kR_k\\ Q_k^T(A_k-\sigma_kI)   &=  Q_k^TQ_kR_k\\ Q_k^T(A_k-\sigma_kI)   &=  R_k\\ R_k                    &=Q_k^T(A_k-\sigma_kI)   \\ &=\begin{bmatrix} c &  s \\ -s &  c                         \end{bmatrix} \begin{bmatrix} a_{11}-\sigma_k & a_{12} \\ \delta       & a_{22}-\sigma_k \end{bmatrix} \end{align} $$

From $$R_k \!\,$$, we can find its (2,2) entry $$ r_22 \!\,$$ by using inner products.


 * $$ r_{22}=-s \cdot a_{12} + c \cdot (a_{22}-\sigma_k) \!\,$$

Now that we have $$ r_{22} \!\,$$ we can calculate $$ A_{k+1}(2,1) \!\, $$ by using appropriate substitutions

$$ \begin{align} A_{k+1}(2,1)  &=  r_{22} \cdot s \\ &= (-s a_{12} + c (a_{22}-\sigma_k))s \\ &= -s^2 a_{12} + cs (a_{22}-\sigma_k) \\ &= \frac{-\delta^2a_{12}}{(a_{11}-\sigma_k)^2+\delta^2}+\frac{(a_{11}-\sigma_k)(a_{22}-\sigma_k)\delta}{(a_{11}-\sigma_k)^2+\delta^2} \\ &=\frac{-\delta^2a_{12}+\delta(a_{11}-\sigma_k)(a_{22}-\sigma_k)}{(a_{11}-\sigma_k)^2+\delta^2} \\ &\approx \frac{-\delta^2a_{12}+\delta(a_{11}-\sigma_k)(a_{22}-\sigma_k)}{(a_{11}-\sigma_k)^2} \end{align} $$

since $$\delta \!\,$$ is a small number.

Let our shift $$ \sigma_k = a_{22} \!\,$$. Then the above equation yields,

$$ \begin{align} A_{k+1}(2,1) &\approx  \frac{-\delta^2a_{12}+\delta(a_{11}-\sigma_k)(a_{22}-\sigma_k)}{(a_{11}-\sigma_k)^2} \\ &\approx \frac{-\delta^2a_{12}+\delta(a_{11}-a_{22})(a_{22}-a_{22})}{(a_{11}-a_{22})^2} \\ &=\frac{-\delta^2a_{12}}{(a_{11}-a_{22})^2} \end{align} $$

Hence,

$$

\begin{align} &\leq |\delta^2| \end{align} $$
 * A_{k+1}(2,1)| &\approx \left|\frac{\delta^2a_{12}}{(a_{11}-a_{22})^2} \right| \\

since $$ |a_{12}| \leq (a_{11}-a_{22})^2 \!\,$$

Hence we have shown that $$A_{k+1}(2,1) \!\,$$ is of order $$ \delta^2 \!\,$$.

If $$ \delta \!\,$$ is small, the QR convergence rate will be quadratic.