Numerical Methods Qualification Exam Problems and Solutions (University of Maryland)/Practice Problems and Solutions

Introduction
This is a compilation of problems and solutions from past numerical methods qualifying exams at the University of Maryland.

Problem 1
Consider the system $$\displaystyle Ax=b$$. The GMRES method starts with a point $$\displaystyle x_0$$ and normalizes the residual $$\displaystyle r_0=b-Ax_0$$ so that $$\textstyle v_1=\frac{r_0}{\nu}$$ has 2-norm one. It then constructs orthonormal Krylov bases $$\scriptstyle V_k=(v_1 \, v_2 \, \cdots v_m)$$ satisfying


 * $$\displaystyle AV_k=V_{k+1} H_k $$

where $$\displaystyle H_k$$ is a $$\textstyle (k+1)\times k$$ upper Hessenberg matrix. One then looks for an approximation to $$\displaystyle x$$ of the form


 * $$\displaystyle x(c)=x_0+V_k c$$

choosing $$\displaystyle c_k$$ so that $$\textstyle \| r(c) \|= \|b-Ax(c)\|$$ is minimized, where $$\textstyle\| \cdot \|$$ is the usual Euclidean norm.

Part 1a
Show that $$\displaystyle c_k$$ minimizes $$\| \nu e_1 - H_k c \|$$.

Solution 1a
We wish to show that


 * $$ \displaystyle \|b-Ax(c)\| = \| \nu e_1-H_k c \|$$

$$ \begin{align} \|b- Ax(c) \|	&= \|b-A(x_0+V_kc) \| \\ &= \|b-Ax_0-AV_kc \| \\ &= \|r_0 - AV_kc \| \\ &= \|r_0-V_{k+1}H_k c \| \\ &= \|\nu v_1 -V_{k+1}H_k c \| \\ &= \|V_{k+1}\underbrace{(\nu e_1 -H_k c)}_{h_c} \| \\ &= (V_{k+1}h_c, V_{k+1}h_c)^{\frac{1}{2}} \\ &= ((V_{k+1}h_c)^TV_{k+1}h_c)^{\frac{1}{2}}\\ &= ( h_c^T V_{k+1}^T V_{k+1} h_c )^{\frac{1}{2}} \\ &= ( h_c^T h_c)^{\frac{1}{2}} \\ &= \| h_c \| \\ &= \| \nu e_1- H_k c \| \end{align} $$