Newton polynomial explained

In the mathematical field of numerical analysis, a Newton polynomial, named after its inventor Isaac Newton,^[1] is an interpolation polynomial for a given set of data points. The Newton polynomial is sometimes called Newton's divided differences interpolation polynomial because the coefficients of the polynomial are calculated using Newton's divided differences method.

Definition

Given a set of k + 1 data points

(x _{0,} y _{0),\ldots,(x} _{j,} y _{j),\ldots,(x} _{k,} y _{k)}

where no two x_j are the same, the Newton interpolation polynomial is a linear combination of Newton basis polynomials

N(x) : = \sum ^{k} _{j =0} a _{j} n _{j} (x)

with the Newton basis polynomials defined as

n _{j(x)} : = \prod ^{j -1} _{i =0} (x - x _{i)}

for j > 0 and

n _{0(x)} \equiv 1 .

The coefficients are defined as

a _{j} : = [y _{0,\ldots,y} _{j]}

where

[y _{0,\ldots,y} _{j]}

is the notation for divided differences.

Thus the Newton polynomial can be written as

N(x) = [y _{0]} + [y _{0,y} _{1](x-x} _{0)} + … + [y _{0,\ldots,y} _{k](x-x} _{0)(x-x} _{1) … (x-x} _{k-1} ).

Newton forward divided difference formula

The Newton polynomial can be expressed in a simplified form when

x _{0,} x _{1,} ... , x _{k} are arranged consecutively with equal spacing.

If

x _{0,} x _{1,} ... , x _{k} are consecutively arranged and equally spaced with {x} _{i} = {x} _{0} + ih for i = 0, 1, ..., k and some variable x is expressed as {x} = {x} _{0} + sh , then the difference x - x _{i} can be written as (s - i)h . So the Newton polynomial becomes\begin{align} N(x) & = [y _{0]} + [y _{0,y} _{1]sh} + … + [y _{0,\ldots,y} _{k]} s (s -1 ) … (s - k +1 ){h} ^{k} \\ & = \sum ^{k} _{i =0} s(s -1 ) … (s - i +1 ){h} ^{i} [y _{0,\ldots,y} _{i]} \\ & = \sum ^{k} _{i =0} {s \choose i}i!{h} ^{i} [y _{0,\ldots,y} _{i]. \end{align}}

This is called the Newton forward divided difference formula.

Newton backward divided difference formula

If the nodes are reordered as

{x} _{k} ,{x} _{k-1} , ... ,{x} _{0} , the Newton polynomial becomesN(x) = [y _{k]+[{y}} _{k} , {y} _{k-1} ](x - {x} _{k} ) + … + [{y} _{k} ,\ldots,{y} _{0} ](x - {x} _{k} )(x - {x} _{k-1} ) … (x - {x} _{1} ).

If

{x} _{k} , {x} _{k-1} , ... , {x} _{0} are equally spaced with {x} _{i} = {x} _{k} - (k - i)h for i = 0, 1, ..., k and {x} = {x} _{k} + sh , then,\begin{align} N(x) & = [{y} _{k} ] + [{y} _{k} , {y} _{k-1} ]sh + … + [{y} _{k} ,\ldots,{y} _{0} ]s(s +1 ) … (s + k -1 ){h} ^{k} \\ & = \sum ^{k} _{i =0} {( -1 )} ^{i} { - s \choose i}i!{h} ^{i} [{y} _{k} ,\ldots,{y} _{k-i} ]. \end{align}

This is called the Newton backward divided difference formula.

Significance

Newton's formula is of interest because it is the straightforward and natural differences-version of Taylor's polynomial. Taylor's polynomial tells where a function will go, based on its y value, and its derivatives (its rate of change, and the rate of change of its rate of change, etc.) at one particular x value. Newton's formula is Taylor's polynomial based on finite differences instead of instantaneous rates of change.

Polynomial interpolation

See main article: Polynomial interpolation. For a polynomial

p _{n} of degree less than or equal to n, that interpolates f at the nodes x _{i} where i = 0 , 1 , 2 , 3 , … ,n . Let p _{n+1} be the polynomial of degree less than or equal to n+1 that interpolates f at the nodes x _{i} where i = 0 , 1 , 2 , 3 , … ,n, n +1 . Then p _{n+1} is given by:

$$p\_(x)\; =\; p\_n(x)\; +a\_w\_n(x)$$

where $w\_n(x)\; :=\; \backslash prod\_^n\; (x-x\_i)$ and $a\_\; :=$.

Proof:

This can be shown for the case where

i = 0 , 1 , 2 , 3 , … ,n :$$p\_(x\_i)\; =\; p\_n(x\_i)\; +a\_\backslash prod\_^n\; (x\_i-x\_j)\; =\; p\_n(x\_i)$$and when i = n +1 :

$$p\_(x\_)\; =\; p\_n(x\_)\; +\; w\_n(x\_)\; =\; f(x\_)$$

By the uniqueness of interpolated polynomials of degree less than

n +1 , $p\_(x)\; =\; p\_n(x)\; +a\_w\_n(x)$ is the required polynomial interpolation. The function can thus be expressed as:

$p\_(x)\; =\; a\_0+a\_1(x-x\_0)+a\_2(x-x\_0)(x-x\_1)+\backslash cdots\; +\; a\_n(x-x\_0)\backslash cdots(x-x\_)$

where the factors

a _{i} are divided differences. Thus, Newton polynomials are used to provide polynomial interpolation formula of n points.^[2]

Taking

y _{i=f(x} _{i)} for some unknown function in Newton divided difference formulas, if the representation of x in the previous sections was instead taken to be x = x _{j+sh} , in terms of forward differences, the Newton forward interpolation formula is expressed as:$$f(x)\; \backslash approx\; N(x)=N(x\_j+sh)\; =\; \backslash sum\_^\backslash Delta^\; f(x\_j)$$whereas for the same in terms of backward differences, the Newton backward interpolation formula is expressed as:$$f(x)\; \backslash approx\; N(x)\; =N(x\_j+sh)=\backslash sum\_^^\backslash nabla^\; f(x\_j).$$This follows since relationship between divided differences and forward differences is given as:^[3] $$[y\_j,\; y\_\{j+1\},\; \backslash ldots,\; y\_\{j+n\}]\; =\; \backslash frac\backslash Delta^y\_j,$$whereas for backward differences, it is given as:$$[\{y\}\_\{j\},\; y\_\{j-1\},\backslash ldots,\{y\}\_\{j-n\}]\; =\; \backslash frac\backslash nabla^y\_j.$$

Addition of new points

As with other difference formulas, the degree of a Newton interpolating polynomial can be increased by adding more terms and points without discarding existing ones. Newton's form has the simplicity that the new points are always added at one end: Newton's forward formula can add new points to the right, and Newton's backward formula can add new points to the left.

The accuracy of polynomial interpolation depends on how close the interpolated point is to the middle of the x values of the set of points used. Obviously, as new points are added at one end, that middle becomes farther and farther from the first data point. Therefore, if it isn't known how many points will be needed for the desired accuracy, the middle of the x-values might be far from where the interpolation is done.

Gauss, Stirling, and Bessel all developed formulae to remedy that problem.^[4]

Gauss's formula alternately adds new points at the left and right ends, thereby keeping the set of points centered near the same place (near the evaluated point). When so doing, it uses terms from Newton's formula, with data points and x values renamed in keeping with one's choice of what data point is designated as the x₀ data point.

Stirling's formula remains centered about a particular data point, for use when the evaluated point is nearer to a data point than to a middle of two data points.

Bessel's formula remains centered about a particular middle between two data points, for use when the evaluated point is nearer to a middle than to a data point.

Bessel and Stirling achieve that by sometimes using the average of two differences, and sometimes using the average of two products of binomials in x, where Newton's or Gauss's would use just one difference or product. Stirling's uses an average difference in odd-degree terms (whose difference uses an even number of data points); Bessel's uses an average difference in even-degree terms (whose difference uses an odd number of data points).

Strengths and weaknesses of various formulae

For any given finite set of data points, there is only one polynomial of least possible degree that passes through all of them. Thus, it is appropriate to speak of the "Newton form", or Lagrange form, etc., of the interpolation polynomial. However, different methods of computing this polynomial can have differing computational efficiency. There are several similar methods, such as those of Gauss, Bessel and Stirling. They can be derived from Newton's by renaming the x-values of the data points, but in practice they are important.

Bessel vs. Stirling

The choice between Bessel and Stirling depends on whether the interpolated point is closer to a data point, or closer to a middle between two data points.

A polynomial interpolation's error approaches zero, as the interpolation point approaches a data-point. Therefore, Stirling's formula brings its accuracy improvement where it is least needed and Bessel brings its accuracy improvement where it is most needed.

So, Bessel's formula could be said to be the most consistently accurate difference formula, and, in general, the most consistently accurate of the familiar polynomial interpolation formulas.

Divided-Difference Methods vs. Lagrange

Lagrange is sometimes said to require less work, and is sometimes recommended for problems in which it is known, in advance, from previous experience, how many terms are needed for sufficient accuracy.

The divided difference methods have the advantage that more data points can be added, for improved accuracy. The terms based on the previous data points can continue to be used. With the ordinary Lagrange formula, to do the problem with more data points would require re-doing the whole problem.

There is a "barycentric" version of Lagrange that avoids the need to re-do the entire calculation when adding a new data point. But it requires that the values of each term be recorded.

But the ability, of Gauss, Bessel and Stirling, to keep the data points centered close to the interpolated point gives them an advantage over Lagrange, when it isn't known, in advance, how many data points will be needed.

Additionally, suppose that one wants to find out if, for some particular type of problem, linear interpolation is sufficiently accurate. That can be determined by evaluating the quadratic term of a divided difference formula. If the quadratic term is negligible—meaning that the linear term is sufficiently accurate without adding the quadratic term—then linear interpolation is sufficiently accurate. If the problem is sufficiently important, or if the quadratic term is nearly big enough to matter, then one might want to determine whether the sum of the quadratic and cubic terms is large enough to matter in the problem.

Of course, only a divided-difference method can be used for such a determination.

For that purpose, the divided-difference formula and/or its x₀ point should be chosen so that the formula will use, for its linear term, the two data points between which the linear interpolation of interest would be done.

The divided difference formulas are more versatile, useful in more kinds of problems.

The Lagrange formula is at its best when all the interpolation will be done at one x value, with only the data points' y values varying from one problem to another, and when it is known, from past experience, how many terms are needed for sufficient accuracy.

With the Newton form of the interpolating polynomial a compact and effective algorithm exists for combining the terms to find the coefficients of the polynomial.^[5]

Accuracy

When, with Stirling's or Bessel's, the last term used includes the average of two differences, then one more point is being used than Newton's or other polynomial interpolations would use for the same polynomial degree. So, in that instance, Stirling's or Bessel's is not putting an N−1 degree polynomial through N points, but is, instead, trading equivalence with Newton's for better centering and accuracy, giving those methods sometimes potentially greater accuracy, for a given polynomial degree, than other polynomial interpolations.

General case

For the special case of x_i = i, there is a closely related set of polynomials, also called the Newton polynomials, that are simply the binomial coefficients for general argument. That is, one also has the Newton polynomials

p _{n(z)} given byp _{n(z)={z} \choose n} = (z(z -1 ) … (z - n +1 ))/(n!)

In this form, the Newton polynomials generate the Newton series. These are in turn a special case of the general difference polynomials which allow the representation of analytic functions through generalized difference equations.

Main idea

Solving an interpolation problem leads to a problem in linear algebra where we have to solve a system of linear equations. Using a standard monomial basis for our interpolation polynomial we get the very complicated Vandermonde matrix. By choosing another basis, the Newton basis, we get a system of linear equations with a much simpler lower triangular matrix which can be solved faster.

For k + 1 data points we construct the Newton basis as

n _{0(x)} : = 1 , n _{j(x)} : = \prod ^{j -1} _{i =0} (x - x _{i)} j =1 ,\ldots,k.

Using these polynomials as a basis for

\Pi _{k} we have to solve\begin{bmatrix} 1 & & \ldots & & 0 \\ 1 & x _{1-x} _{0} & & & \\ 1 & x _{2-x} _{0} & (x _{2-x} _{0)(x} _{2-x} _{1)} & & \vdots \\ \vdots & \vdots & & \ddots & \\ 1 & x _{k-x} _{0} & \ldots & \ldots & \prod ^{k -1} _{j =0} (x _{k} - x _{j) \end{bmatrix} \begin{bmatrix}} a _{0} \ \ \vdots \ \ a _{k} \end{bmatrix} = \begin{bmatrix} y _{0} \ \ \vdots \ \ y _{k} \end{bmatrix}

to solve the polynomial interpolation problem.

This system of equations can be solved iteratively by solving

\sum ^{j} _{i =0} a _{i} n _{i} (x _{j)} = y _{j} j = 0 , ... ,k.

Derivation

While the interpolation formula can be found by solving a linear system of equations, there is a loss of intuition in what the formula is showing and why Newton's interpolation formula works is not readily apparent. To begin, we will need to establish two facts first:

Fact 1. Reversing the terms of a divided difference leaves it unchanged:

[y _{0,} \ldots, y _{n]} = [y _{n,} \ldots, y _{0].} The proof of this is an easy induction: for n =1 we compute $$[y\_0,\; y\_1]\; =\; \backslash frac\; =\; \backslash frac\; =\; [y\_1,\; y\_0].$$ Induction step: Suppose the result holds for any divided difference involving at most n +1 terms. Then using the induction hypothesis in the following 2nd equality we see that for a divided difference involving n +2 terms we have $$[y\_0,\; \backslash ldots,\; y\_\{n+1\}]\; =\; \backslash frac\; =\; \backslash frac\; =\; [y\_\{n+1\},\; \backslash ldots,\; y\_0].$$

We formulate next Fact 2 which for purposes of induction and clarity we also call Statement

n (Stm _{n} ) :

Fact 2. (

Stm _{n} ) : If (x _{0,} y _{0),} \ldots, (x _{n-1} , y _{n-1} ) are any n points with distinct x -coordinates and P = P(x) is the unique polynomial of degree (at most) n -1 whose graph passes through these n points then there holds the relation$$[y\_0,\; \backslash ldots,\; y\_n](x\_n\; -\; x\_0)\backslash cdot\backslash ldots\backslash cdot(x\_n-x\_)\; =\; y\_n\; -\; P(x\_n)$$

Proof. (It will be helpful for fluent reading of the proof to have the precise statement and its subtlety in mind:

P is defined by passing through (x _{0,y} _{0),.} . . ,(x _{n-1} ,y _{n-1} ) but the formula also speaks at both sides of an additional arbitrary point (x _{n,y} _{n)} with x -coordinate distinct from the other x _{i} .)

We again prove these statements by induction.To show

Stm _{1,} let (x _{0,y} _{0)} be any one point and let P(x) be the unique polynomial of degree 0 passing through (x _{0,} y _{0)} . Then evidently P(x) = y _{0} and we can write $$[y\_0,\; y\_1](x\_1\; -\; x\_0)\; =\; \backslash frac\; (x\_1-x\_0)\; =\; y\_1\; -\; y\_0\; =\; y\_1\; -\; P(x\_1)$$ as wanted.

Proof of

Stm _{n+1} , assuming Stm _{n} already established: Let P(x) be the polynomial of degree (at most) n passing through (x _{0,} y _{0),} \ldots, (x _{n,} y _{n).}

With

Q(x) being the unique polynomial of degree (at most) n -1 passing through the points (x _{1,} y _{1),} \ldots, (x _{n,} y _{n)} , we can write the following chain of equalities, where we use in the penultimate equality that Stm_{n} applies to Q :

The induction hypothesis for

Q also applies to the second equality in the following computation, where (x _{0,y} _{0)} is added to the points defining Q :

Now look at

Q(x) + [y _{0,\ldots,y} _{n](x} - x _{1) ⋅ \ldots ⋅ (x} - x _{n).} By the definition of Q this polynomial passes through (x _{1,y} _{1),} . . ., (x _{n,y} _{n)} and, as we have just shown, it also passes through (x _{0,y} _{0).} Thus it is the unique polynomial of degree \leq n which passes through these points. Therefore this polynomial is P(x); i.e.: P(x) = Q(x) + [y _{0,\ldots,y} _{n](x} - x _{1) ⋅ \ldots ⋅ (x} - x _{n).}

Thus we can write the last line in the first chain of equalities as `

y _{n+1} - P(x _{n+1} ) ' and have thus established that$$[y\_0,\backslash ldots,y\_\{n+1\}](x\_\; -\; x\_0)\backslash cdot\backslash ldots\backslash cdot(x\_\; -\; x\_n)=y\_-P(x\_).$$ So we established Stm _{n+1} , and hence completed the proof of Fact 2.

Now look at Fact 2: It can be formulated this way: If

P is the unique polynomial of degree at most n -1 whose graph passes through the points (x _{0,} y _{0),} . . . , (x _{n-1} , y _{n-1} ), then P(x) + [y _{0,} \ldots, y _{n](x} - x _{0) ⋅ \ldots ⋅ (x-x} _{n-1} ) is the unique polynomial of degree at most n passing through points (x _{0,} y _{0),} . . . , (x _{n-1} , y _{n-1} ), (x _{n,} y _{n).} So we see Newton interpolation permits indeed to add new interpolation points without destroying what has already been computed.

Taylor polynomial

The limit of the Newton polynomial if all nodes coincide is a Taylor polynomial, because the divided differences become derivatives.

Application

As can be seen from the definition of the divided differences new data points can be added to the data set to create a new interpolation polynomial without recalculating the old coefficients. And when a data point changes we usually do not have to recalculate all coefficients. Furthermore, if the x_i are distributed equidistantly the calculation of the divided differences becomes significantly easier. Therefore, the divided-difference formulas are usually preferred over the Lagrange form for practical purposes.

Examples

The divided differences can be written in the form of a table. For example, for a function f is to be interpolated on points

x _{0,} \ldots, x _{n} . Write\begin{matrix} x _{0} & f(x _{0)} & & \\ & & {f(x _{1)-f(x} _{0)\over} x _{1} - x _{0}} & \\ x _{1} & f(x _{1)} & & {{f(x _{2)-f(x} _{1)\over} x _{2} - x _{1}-{f(x} _{1)-f(x} _{0)\over} x _{1} - x _{0}} \over x _{2} - x _{0}} \\ & & {f(x _{2)-f(x} _{1)\over} x _{2} - x _{1}} & \\ x _{2} & f(x _{2)} & & \vdots \\ & & \vdots & \\ \vdots & & & \vdots \\ & & \vdots & \\ x _{n} & f(x _{n)} & & \\ \end{matrix} Then the interpolating polynomial is formed as above using the topmost entries in each column as coefficients.

For example, suppose we are to construct the interpolating polynomial to f(x) = tan(x) using divided differences, at the points

n	x _{n}	f(x _{n)}
0	- \tfrac{ 3 }{ 2 }	-14 . 1014
1	- \tfrac{ 3 }{ 4 }	-0 . 931596
2	0	0
3	\tfrac{ 3 }{ 4 }	0 . 931596
4	\tfrac{ 3 }{ 2 }	14 . 1014

Using six digits of accuracy, we construct the table

\begin{matrix} - \tfrac{ 3 }{ 2 } & -14 . 1014 & & & &\\ & & 17 . 5597 & & &\\ - \tfrac{ 3 }{ 4 } & -0 . 931596 & & -10 . 8784 & &\\ & & 1 . 24213 & & 4 . 83484 & \\ 0 & 0 & & 0 & & 0 \\ & & 1 . 24213 & & 4 . 83484 &\\ \tfrac{ 3 }{ 4 } & 0 . 931596 & & 10 . 8784 & &\\ & & 17 . 5597 & & &\\ \tfrac{ 3 }{ 2 } & 14 . 1014 & & & &\\ \end{matrix} Thus, the interpolating polynomial is\begin{align} & -14 . 1014+17 . 5597 (x + \tfrac{ 3 }{ 2 }) -10 . 8784 (x + \tfrac{ 3 }{ 2 })(x + \tfrac{ 3 }{ 4 }) +4 . 83484 (x + \tfrac{ 3 }{ 2 })(x + \tfrac{ 3 }{ 4 })(x) +0 (x + \tfrac{ 3 }{ 2 })(x + \tfrac{ 3 }{ 4 })(x)(x - \tfrac{ 3 }{ 4 }) \\ = {}& -0 . 00005-1 . 4775 x -0 . 00001 x ^{2+4.83484x} ^{3 \end{align}} Given more digits of accuracy in the table, the first and third coefficients will be found to be zero.

Another example:

The sequence

f _{0} such that f _{0(1)} = 6 , f _{0(2)} = 9 , f _{0(3)} = 2 and f _{0(4)} = 5 , i.e., they are 6 , 9 , 2 , 5 from x _{0} = 1 to x _{3} = 4 .

You obtain the slope of order

1 in the following way:f _{1(x} _{0,} x _{1)} = (f _{0(x} _{1)} - f _{0(x} _{0)})/(x _{1} - x _{0}) = (9-6)/(2-1) = 3 f _{1(x} _{1,} x _{2)} = (f _{0(x} _{2)} - f _{0(x} _{1)})/(x _{2} - x _{1}) = (2-9)/(3-2) = -7 f _{1(x} _{2,} x _{3)} = (f _{0(x} _{3)} - f _{0(x} _{2)})/(x _{3} - x _{2}) = (5-2)/(4-3) = 3

As we have the slopes of order

1 , it is possible to obtain the next order:f _{2(x} _{0,} x _{1,} x _{2)} = (f _{1(x} _{1,} x _{2)} - f _{1(x} _{0,} x _{1)})/(x _{2} - x _{0}) = (-7-3)/(3-1) = -5 f _{2(x} _{1,} x _{2,} x _{3)} = (f _{1(x} _{2,} x _{3)} - f _{1(x} _{1,} x _{2)})/(x _{3} - x _{1}) = (3- ( -7 ))/(4-2) = 5

Finally, we define the slope of order

3 :f _{3(x} _{0,} x _{1,} x _{2,} x _{3)} = (f _{2(x} _{1,} x _{2,} x _{3)} - f _{2(x} _{0,} x _{1,} x _{2)})/(x _{3} - x _{0}) = (5- ( -5 ))/(4-1) = (10)/(3)

Once we have the slope, we can define the consequent polynomials:

p _{0(x)} = 6 .p _{1(x)} = 6 + 3 (x - 1 ) p _{2(x)} = 6 + 3 (x - 1 ) - 5 (x - 1 )(x - 2 ) .p _{3(x)} = 6 + 3 (x - 1 ) - 5 (x - 1 )(x - 2 ) + (10)/(3) (x - 1 )(x - 2 )(x - 3 )

See also

• , a work by Thomas Harriot describing similar methods for interpolation, written 50 years earlier than Newton's work but not published until 2009
• Newton series
• Neville's schema
• Polynomial interpolation
• Lagrange form of the interpolation polynomial
• Bernstein form of the interpolation polynomial
• Hermite interpolation
• Carlson's theorem
• Table of Newtonian series

External links

• Module for the Newton Polynomial by John H. Mathews

Notes and References

1. Book: Dunham . William . Journey Through Genius: The Great Theorems of Mathematics . 1990 . Kanak Agrawal, Inc . 9780140147391 . 155–183 . https://archive.org/details/journeythroughge00dunh_0/page/155 . 24 October 2019 . 7 .
2. Book: Epperson, James F. . An introduction to numerical methods and analysis . 2013 . Wiley . 978-1-118-36759-9 . 2nd . Hoboken, NJ.
3. Book: Burden . Richard L. . Numerical Analysis . Faires . J. Douglas . 2011 . 9780538733519 . 9th . 129 . Cengage Learning . limited.
4. Book: Hamming, Richard W. . Numerical methods for scientists and engineers . 1986 . Dover . 978-0-486-65241-2 . Unabridged republ. of the 2. ed. (1973) . New York.
5. Web site: Stetekluh. Jeff. Algorithm for the Newton Form of the Interpolating Polynomial.