Introduction to Numerical Methods/Roots of Equations

=Root Finding= Objectives:
 * find solutions of quadratic and cubic equations
 * derive the formula and follow the algorithms for the solutions of non-linear equations using the following methods:
 * Bisection
 * Newton-Raphson
 * Secant
 * False-Position

Resources
 * book chapters on various methods
 * Root Finding Algorithms
 * Bisection Method
 * Newton's Method
 * Secant Method
 * False-Position Method

Roots (or Zeros) of a function f(x) are values of x that produces an output of 0. Roots can be real or complex numbers. Finding the root of $$f(x)-g(x)$$ is the same as solving the equation $$f(x) = g(x)$$. Solving an equation is finding the values that satisfy the condition specified by the equation.

Lower degree (quadratic, cubic, and quartic) polynomials have closed-form solutions, but numerical methods may be easier to use. To solve a quadratic equation we can use the quadratic formula:


 * $$ax^2+bx+c=0$$


 * $$x=\frac{-b\pm\sqrt{b^2-4ac\ }}{2a}$$

There are many root-find algorithms for solving equations numerically.

Bisection Method
The bisection method starts with two guesses and uses a binary search algorithm to improve the answers. If a function is continuous between the two initial guesses, the bisection method is guaranteed to converge. Here is a picture that illustrates the idea:



The advantages of bisection method include guaranteed convergence on continuous functions and the error is bounded.

The disadvantages of bisection method include relatively slow convergence and non convergence on certain functions.

Newton-Raphson Method
Newton's Method (a.k.a Newton-Raphson Method) is an open method for solving non-linear equations. Contrary to a bracketing-method (e.g. bisection method) Newton's method needs one initial guess but it doesn't guarantee to converge.

The basic idea of Newton's method is as follows:
 * Given a function f of "x" and a initial guess $$x_{0}$$ for the root of this function, a better guess $$x_{1}$$ is
 * $$x_{1} = x_0 - \frac{f(x_0)}{f'(x_0)} $$
 * This is the Newton-Raphson formula.

The following animation illustrates the method:



Lets solve the following equations using Newton's method:
 * $$f(x)=x^{2}+2$$
 * $$f(x)=sin(x)=0$$
 * $$f(x)=a-\frac{1}{x}=0$$

Division by Zero
Because a division is involved in Newton's formula when the denominator becomes zero the method won't continue correctly. The following program demonstrates this issue. Output:

Divergence at Inflection Points
When the initial guess is near a inflection point the method may diverge from the desired root.

The following code using the same implementation of Newton's method (assume the newton function is imported) demonstrates divergence at an inflection point (x=0).

Secant Method
Secant method is similar to Newton's method in that it is an open method and use a intersection to get the improved estimate of the root. Secant method avoids calculating the first derivatives by estimating the derivative values using the slope of a secant line.

The following figure illustrates the secant method with two initial guesses and two successive improved estimates:

The formula for secant method is as follows:

x_{i+1} =x_{i}-\frac{f(x_{i})(x_{i}-x_{i-1})}{f(x_{i})-f(x_{i-1})} $$

False-Position Method
The false-position method is similar to the bisection method in that it requires two initial guesses (bracketing method). Instead of using the midpoint as the improved guess, the false-position method use the root of secant line that passes both end points. The following figure illustrates the idea.

The formula for false-position method is as follows:
 * $$x_{r}=\frac{x_uf(x_l) - x_lf(x_u)}{f(x_l) - f(x_u)}$$