Numerical Methods Qualification Exam Problems and Solutions (University of Maryland)/Aug08 667

Method of Order 2
Method $$(1)\!\,$$ finds an approximation for $$ y \!\,$$ such that $$y' = f(x,y) \!\,$$.

Let $$x_n = a_0+hn \!\,$$ be the $$ n\!\,$$th point of evaluation where $$a_0 \!\,$$ is the starting point and $$ h\!\,$$ is the step size.

Taylor Expansion of y' about a_0
$$ \begin{align} y'(x_n) &= y'(a_0 +hn) \\ &= y'(a_0) + h n y(a_0)+ h^2 n^2 y'(a_0)/2 + \mathcal{O}(h^3) \\ \\ \\ y'(x_{n-1}) & =y'(a_0+h(n-1)) \\ & = y'(a_0) + h (n-1) y(a_0)+ h^2 (n-1)^2 y'(a_0)/2 + \mathcal{O}(h^3) \end{align} $$

Substituting into the second term on the right hand side of $$ (1) \!\,$$ and simplifying yields

$$ \begin{align} h[\frac{5}{2}y'(x_n) + \frac{1}{2}y'(x_{n-1})] &= \frac{5}{2}[h y'(a_0) + h^2 n y(a_0)+ \mathcal{O}(h^3)] + \frac{1}{2}[h y'(a_0) + h^2 (n-1) y(a_0)+ \mathcal{O}(h^3)] \\

&= 3h y'(a_0) + \frac{1}{2}(6n-1) y''(a_0) + \mathcal{O}(h^3)

\end{align}

\!\,$$

Taylor Expansion of y about a_0
Since $$y_n \approx y(x_n) \!\,$$, we also take Taylor Expansion of $$y \!\,$$ about $$ a_0\!\,$$

$$ \begin{align} y(x_{n+1}) &= y(a_0) + h (n+1) y'(a_0)+ h^2 (n+1)^2 y''(a_0)/2 + \mathcal{O}(h^3)\\ \\ \\ y(x_n)    &= y(a_0) + h n y'(a_0)+ h^2 n^2 y''(a_0)/2 + \mathcal{O}(h^3) \\ \\ \\ y(x_{n-1}) &= y(a_0) + h (n-1) y'(a_0)+ h^2 (n-1)^2 y''(a_0)/2 + \mathcal{O}(h^3) \end{align} $$

Substituting and simplifying yields,


 * $$ y_{n+1} + y_n - 2y_{n-1} = 3h y'(a_0) + \frac{1}{2}(6n-1) y''(a_0) + \mathcal{O}(h^3) \!\,$$

Take Difference of Taylor Expansions
Hence $$ y_{n+1} + y_n - 2y_{n-1} - h[\frac{5}{2}y'(x_n) + \frac{1}{2}y'(x_{n-1})] = \mathcal{O}(h^3) \!\,$$ shows that (1) is a method of order 2.

Does Not Satisfy Root Condition
The Characteristic equation of (1) is


 * $$\lambda^2+\lambda-2 = 0 \!\,$$

Giving the roots


 * $$\lambda_1 = 1 \!\,$$
 * $$\lambda_2 = -2 \!\,$$

$$\lambda_2 \!\,$$ clearly does not satisfy $$ |\lambda_j| \leq 1 \!\,$$

Solution 4b
Let $$f(x,y)=0, y(0)=y_0=1\!\,$$. Then $$y(t)=1 \!\,$$. We have the difference equation

$$y_{n+1}+y_n-2y_{n-1}=0 \!\,$$

which has general solution (use the roots)

$$y_n=C_1(-2)^n+C_2\cdot 1^n \!\,$$

If $$C_1 \neq 0 \!\,$$, then $$y_n \rightarrow \infty \!\,$$ as $$ n \rightarrow \infty \!\,$$

$$y_0=C_1+C_2 \!\,$$

$$y_1=-2C_1+C_2 \!\,$$

Hence, $$\frac{y_0-y_1}{3}=C_1 \!\,$$. Therefore if $$y_0 \neq y_1 \!\,$$, then $$C_1 \neq 0 \!\,$$.

Solution 5a
Let $$u_1 \!\,$$ and $$u_2 \!\,$$ be solutions. Let $$u \equiv u_1-u_2 \!\,$$.

By subtracting the two equations and their conditions we have

$$-u''+(1+x)u=0 \quad u'(0)=0, u(1)=0 \!\,$$

Multiplying by test function $$v \in V=\{v \in H^1: v(1)=0 \} \!\,$$ and integrating by parts from 0 to 1, we want to find $$u \in V \!\,$$ such that for all $$v \in V\!\,$$

$$\int_0^1 u'v'+ \int_0^1 (1+x)uv = 0 \!\,$$

Let $$v=u \!\,$$. Then, we have

$$\int_0^1 (u')^2+ \int_0^1 (1+x)u^2 = 0 \!\,$$

Since $$(u')^2 \!\,$$, $$(1+x) \!\,$$, and $$u^2 \!\,$$ are all $$ >0 \!\,$$, $$u^2 =0 \!\,$$. Hence $$u_1=u_2 \!\,$$.

Solution 5b
The three point difference formula for $$u'' \!\,$$ is given by

$$u''(x)=\frac{u(x+h)-2u(x)+u(x-h)}{h^2} \!\,$$

Equations for i=2,...,n-2
Substituting into $$(2) \!\,$$ with our difference formula we have in matrix formulation

$$ \begin{bmatrix} -\frac{1}{h^2} &&& \frac{2}{h^2} +(1+ih) &&& -\frac{1}{h^2} \end{bmatrix} \begin{bmatrix} u_{i-1} \\ u_{i} \\ u_{i+1} \end{bmatrix} =(ih)^2 $$

Equation for i=1
We can eliminate the $$u_0 \!\,$$ variable by using the approximation

$$u'(0)=\frac{u_1-u_0}{h}=1 \!\,$$

which implies

$$u_0=u_1-h \!\,$$

Using this relationship and the three difference formula, we have

$$ \begin{bmatrix} \frac{1}{h^2}+(1+h) &&& -\frac{1}{h^2} \end{bmatrix} \begin{bmatrix} u_1 \\ u_2 \end{bmatrix} =h^2-\frac1h

\!\,$$

Equations for i=n-1
Since $$u(1)=u_n=1 \!\,$$, we can eliminate the $$u_n \!\,$$ variable by substituting into the n-1 equation.

$$ \begin{bmatrix} -\frac{1}{h^2} &&& \frac{2}{h^2}+(1+(n-1)h) \end{bmatrix} \begin{bmatrix} u_{n-2} \\ u_{n-1} \end{bmatrix} =h^2-((n-1)h)^2+\frac{1}{h^2} $$

Solution 5c
Since the matrix $$A \!\,$$ is diagonally dominant, the system $$Ax=b \!\,$$ has unique solution.

Solution 5d
Let $$u \!\,$$,a solution of the boundary value problem, be represented as the sum of solutions to two different boundary value problems i.e.

$$u=w+v \!\,$$ where

$$-w''+(1+x)w=f(x)\quad w'(0)=0, w(1)=0 \!\,$$

$$-v''+(1+x)v=g(x)\quad v'(0)=1, v(1)=1 \qquad (*)\!\,$$

$$-u''+(1+x)u=x^2=f(x)+g(x) \quad u'(0)=1,u(1)=1 \!\,$$

Suppose $$v(x)=ax+b \!\,$$. Then

$$v'(0)=a=1 \!\,$$ and $$v(1)=1+b=1 \!\,$$ which implies $$ b=0\!\,$$ and hence

$$ v(x)=x\!\,$$

Substituting into $$(*) \!\,$$, we then have

$$-0+(1+x)x=g(x) \!\,$$

which implies

$$g(x)=x^2+x \!\,$$

Since $$f(x)+g(x)=x^2 \!\,$$, we have $$f(x)=-x \!\,$$

Therefore an equivalent boundary value problem with hemogenous boundary conditions is given by

$$-w''+(1+x)w=-x \quad w'(0)=0, w(1)=0\!\,$$

Variational Formulation and Sobolev space
Using the problem's notation, we want to find $$v \in H=\{w \in H^1 : w(1)=0 \} \!\,$$ such that for all $$w \in H \!\,$$, we have

$$a(v,w)=\int_0^1 v'w' + \int_0^1 (1+x)vw = -\int_0^1 x w=F(w) \!\,$$

The above comes from integrating by parts and applying the boundary conditions.

Unique Solution
To show that $$v \!\,$$ is unique, we show that the hypothesis of the Lax-Milgram Theorem are met.

a(v,w) bounded and continuous
$$ \begin{align} a(v,w) &= \int_0^1 v'w' + \int_0^1 \underbrace{(1+x)}_{\leq 2} vw \\ &\leq 2\left[ \int_0^1 v'w' + \int_0^1 vw \right] \\ &\leq 2\left[ \left[\int_0^1 v'^2 \right]^{\frac12} \left[\int_0^1 w'^2 \right]^{\frac12}+\left[\int_0^1 v^2 \right]^{\frac12}\left[\int_0^1 w^2\right]^{\frac12} \right] \quad \mbox{ Cauchy-Schwartz in L2 }\\ &\leq 2\left[ \left[\int_0^1 v'^2+ \int_0^1 v^2 \right]^{\frac12}\left[\int_0^1 w'^2+ \int_0^1 w^2\right]^{\frac12}\right] \mbox{ Cauchy-Schwartz in R2 }\\ &=2 \|w\|_1 \|v\|_1 \end{align} \!\,$$

a(v,v) coercive
$$ \begin{align} a(v,v) &= \int_0^1 v'^2 + \int_0^1 \underbrace{(1+v)}_{\geq 1}v^2 \\ &\geq \int_0^1 v'^2 + \int v^2 \\ &= \|v\|_1^2 \end{align} $$

F(v) bounded
$$ |F(v)| = |-\int_0^1 xv | \leq | \int_0^1 v | \leq \|v\|_1^2 \!\,$$

Solution 6a
$$\phi(x_n) := x_{n+1} = x_n - Af(x_n) \!\,$$ is a fixed point iteration. By the contraction mapping theorem, if $$\phi(x) \!\,$$ is a contraction in some neighborhood of $$x_* \!\,$$ then the iteration converges at least linearly.

We have to show there exists $$\gamma < 1 \!\,$$ such that $$\| \phi(x) - \phi(y) \| \leq \gamma \| x - y \| \!\,$$.

By the mean value theorem we have that $$\| \phi(x) - \phi(y) \| \leq \| D\phi(\xi) \| \| x - y \| \!\,$$, that is $$\gamma = D\phi(\xi) \!\,$$ for some $$\xi \!\,$$ in our neighborhood of $$x_* \!\,$$.

In particular, $$\|D\phi(x_*)\| = \|I - A D\phi(x_*)\| = 0 < 1 \!\,$$, implying that $$\phi(x) \!\,$$ is a contraction and that the iterative method converges at least linearly.

Calculating the Jacobian
$$D\phi = Dx - A Df \!\,$$

$$Dx = I \!\,$$

$$Df = \begin{pmatrix} 2x-2 & 1 \\ 2 & -2y \end{pmatrix} \!\,$$

$$ \begin{align} D\phi(x_*) &= I - A D\phi(x_*) \\ &= \begin{pmatrix}1 & 0 \\ 0 & 1 \end{pmatrix} - \begin{pmatrix}1 & \frac12 \\ 1 & 0 \end{pmatrix} \begin{pmatrix} 2(1)-2 & 1 \\ 2 & -2(1) \end{pmatrix} \\ &= \begin{pmatrix}1 & 0 \\ 0 & 1 \end{pmatrix} - \begin{pmatrix}1 & 0 \\ 0 & 1 \end{pmatrix} \\ &= \begin{pmatrix}0 & 0 \\ 0 & 0 \end{pmatrix} \\ \| D\phi(x_*) \| &= 0 \end{align} $$

$$ \!\,$$

Solution 6b
$$ \begin{align} e_{n+1} &= x_{n+1}-x_* \\ &= \underbrace{x_n - A f(x_n)}_{\phi(x_n)} - x_* \\ &= \phi(x_n)-x_* \\ &= \left ( \phi(x_*)  + (x_n-x_*)^T\underbrace{D_\phi(x_*)}_0+ (x_n-x_*)^T\frac{H_\phi(x_*)}{2!}(x_n-x_*) + R(x_n,x_*) \right)  - x_*  \\ &= \left(x_*+A\underbrace{f(x_*)}_0  + \frac{H_\phi(x_*)}{2!}\underbrace{(x_n-x_*)^T(x_n-x_*)}_{e_n^2} + R(x_n,x_*)\right)  -x_* \\ &\leq Ce_n^2 + R(x_n,x_*) \end{align} \!\,$$

where $$ R(x_n,x_*) \!\,$$ satisfies $$ \frac{\|R(x_n,x_*)\|}{\|x_n-x_*\|} \rightarrow 0 \!\,$$ when $$ \|x_n-x_*\|  \rightarrow 0 \!\,$$.

Then, we obtain $$ \| e_{n+1}\| \leq C \| e_{n}\|^2 \!\,$$.

Solution 6c
The Newton iteration looks like this:

$$ x_{n+1} = x_n - B(x_n)f(x_n) \!\, $$

$$ \begin{bmatrix} x_{n+1} \\  y_{n+1} \end{bmatrix} = \begin{bmatrix} x_n \\  y_n \end{bmatrix} - \underbrace{ \begin{bmatrix} 2x_n-2 & 1    \\ 2     & -2y_n \end{bmatrix}^{-1}}_{B} \begin{bmatrix} x_n^2-2x_n+y_n \\ 2x_n-y_n^2-1 \end{bmatrix} $$

Where B is the inverse of the Jacobian of f.

$$ B(x_*) = \begin{bmatrix} 2(1)-2 & 1    \\ 2      & -2(1) \end{bmatrix}^{-1} = \begin{bmatrix} 0 & 1   \\ 2 & -2 \end{bmatrix}^{-1} = \begin{bmatrix} 1 & \frac12   \\ 1 & 0 \end{bmatrix} = A $$

That is, $$ B(x_*) \!\,$$ in the Newton Iteration gives (4).