Numerical Methods Qualification Exam Problems and Solutions (University of Maryland)/Aug04 667

Solution 5a
$$x_{k+1}=x_k - \frac{f(x_k)}{f'(x_k)} \!\,$$

Solution 5b
$$ \begin{align} x_{k+1} &= x_k - f'(x_k)^{-1}f(x_k) \\ \underbrace{x_{k+1}-x_*}_{e_{k+1}} &= \underbrace{x_k - x_*}_{e_k} - f'(x_k)^{-1}f(x_k) \\ &=f'(x_k)^{-1}[f'(x_k)e_k - ( f(x_k) - \underbrace{f(x_*)}_0 ) ) \\   &=f'(x_k)^{-1}\left[f'(x_k)e_k - e_k\int_0^1 f'(sx_k+(1-s)x_*)ds \right] \\    &=f'(x_k)^{-1}\left[f'(x_k)e_k - f'(x_*)e_k - e_k \int_0^1 ( f'(sx_k+(1-s)x_*)-f'(x_*)) ds \right ] \\ \|e_{k+1}\| &\leq \|f'(x_k)^{-1}\| \left[ L \|e_k\|^2+\frac{L}{2}\|e_k\|^2 \right] \mbox{ where L is Lipschitz constant of } f' \\    &=\|f'(x_k)^{-1}\| \frac32 L \|e_k\|^2 \end{align} \!\,$$

Solution 5c
The typical order of local convergence is quadratic.

Consider the Newton's method as a fixed point iteration i.e.:

$$x_{k+1}=\underbrace{x_k-\frac{f(x_k)}{f'(x_k)}}_{g(x_k)} \!\,$$

Then

$$g'(x_k)=1-\frac{f'(x_k)^2-f'(x_k)f(x_k)}{f'(x_k)^2} \!\,$$

$$g'(x_*)=0 \!\,$$

Expanding $$g(x_k) \!\,$$ around $$x_* \!\,$$ gives an expression for the error

$$\underbrace{g(x_k)}_{x_{k+1}}=\underbrace{g(x_*)}_{x_*}+\underbrace{g'(x_*)}_0 e_k +\frac{g(x_*)}{2}e_k^2 + \frac{g'(\xi)}{6}e_k^3 \!\,$$

Note that if $$g''(x_*) =0 \!\,$$, then we have better than quadratic convergence.

Solution 6a
Find $$ u \in H \!\,$$ such that for all $$v \in H = \{ v \in H^1[0,1]: v(1)=0 \} \!\,$$

$$\underbrace{\int_0^1 u'v' dx + \int_0^1 uv}_{a(u,v)} = \underbrace{\int_0^1 f v - 2 v(0)}_{f(v)} \!\,$$

Solution 6b
Let $$ P=\{x_i\}_{i=0}^{N} \!\,$$ be a partition of $$[0,1] \!\,$$. Choose a an appropriate discrete subspace $$V_h \!\,$$ and basis functions $$ \{\phi_i\}_{i=0}^N\!\,$$. Then

$$u_h = \sum_{i=0}^N u_{hi} \phi_i \!\,$$

The coefficients $$u_{hi} \!\,$$ can be found by solving the following system of equations:

For $$j=0,1,\ldots,N \!\,$$

$$\sum_{i=0}^N u_{hi} \int_0^1 \phi_i' \phi_j' dx + \sum_{i=0}^N u_{hi} \int_0^1 \phi_i \phi_j dx = \int_0^1 f \phi_j - 2 \phi_j(0) \!\,$$

Solution 6c
Cea's Lemma:

$$\|u-u_h\|_1 \leq \inf_{v \in V_h} C \|u - v\|_1 \!\,$$

In particular choose $$v_I \!\,$$ to be the linear interpolant of $$ u\!\,$$.

Then,

$$\|u-u_h\| \leq C \|u''\|_\infty h \!\,$$

Alternative Solution 6c
Let $$ \{x_k\}_0^n $$ be a discrete mesh of $$ [0,1] $$ with step size $$ h $$. Consider the following integral

$$ \int_{x_k}^{x_{k+1}} |u(x)-u_h(x)|^2 dx $$.

For some $$\xi_k\in [x_k, x_{k+1}]$$, $$ u(x)-u_h(x)=u'(\xi_k)*h $$ as $$ u_h $$ is just a linear interpolation on this interval. Hence

$$ \int_{x_k}^{x_{k+1}} |u(x)-u_h(x)|^2 dx=\int_{x_k}^{x_{k+1}} h^2 u'(\xi_k)^2 dx=h^3 u'(\xi_k)^2\leq h^3 ||u'||_\infty $$.

Similarly, we can bound the $$ L_2(x_k, x_k+h) $$ norm of the error in the derivatives with $$ h^3 ||u''||_\infty $$. With $$ n=1/h $$ such intervals we have

$$\ ||u-u_h\||^2 \leq 2nh^3 \max(||u'||^2_\infty, ||u||^2_\infty)=2h^2 \max(||u'||^2_\infty, ||u||^2_\infty) .$$

Solution 6d
$$ \begin{align} \underbrace{a(u-u_h,u-u_h)}_{\|u-u_h\|_1^2}+ 2 \underbrace{a(u-u_h,u_h)}_0 &= a(u,u)-2a(u,u_h)+a(u_h,u_h)+2a(u,u_h)-2a(u_h,u_h) \\ &=a(u,u)-a(u_h,u_h) \\ &=\|u\|_1^2 - \|u_h\|_1^2 \end{align} \!\,$$

Solution 7a
$$ \begin{align} y_{n+1}   &= y_{n-1} + 2h f(t_n,y_n) \\ \hat{y}_{n+1} &= \hat{y}_{n-1} + 2h f(t_n, \hat{y}_n) \end{align} $$

Subtracting both equations, letting $$e_n \equiv y_n -\hat{y}_n \!\,$$, and applying the Lipschitz property yields,

$$ \begin{align} e_{n+1} &\leq e_{n-1} + 2hL e_n \\ &\leq (1+2hL) \max \{ e_n, e_{n-1} \} \end{align} \!\,$$

Therefore,

$$ \begin{align} e_n &\leq (1+2hL)^{n-1} \epsilon \\ &\leq (1+2hL)^n \epsilon \\ &\leq e^{2Lhn} \epsilon \\ &\leq e^{2L(t_n-t_0)} \epsilon \end{align} \!\,$$

Solution 7b
Substituting into the midpoint rule we have,

$$y_{n+1}=y_{n-1}+2h \lambda y_n \!\,$$

or

$$y_{n+1}-2h\lambda y_n - y_{n-1}=0 \!\,$$

The solution of this equation is given by

$$y_n = C_1 z_1^n + C_2 z_2^n \!\,$$

where $$z_1, z_2 \!\,$$ or the roots of the quadratic

$$z^2-2h\lambda z - 1 \!\,$$

The quadratic formula yields

$$z= h\lambda \left(1 \pm \sqrt{1+\frac{4}{4h^2 \lambda^2}} \right) \!\,$$

If $$\lambda \!\,$$ is a small negative number, than one of the roots will be greater than 1. Hence, $$y_{n} \rightarrow \infty \!\,$$ as $$ n \rightarrow \infty\!\,$$ instead of converging to zero since $$y(t)=e^{\lambda t} \!\,$$.