How Newton could solve ANY equation.
Note: Please switch to Dark mode for a more immersive experience.
1.1 Introduction
You like solving equations, right? What, you don’t? Stop lying; we all do!
Right from when we started solving simple ones like 2x + 1 = 17, or moving to high school and facing the quadratic bosses.
Some of you may have even dabbled in some high-degree solving with cubic and quartic equations.
And what about degrees of 5 and higher? Anyone?
Nevertheless, today we are here to discuss a more fundamental method and — I would say — cleverer than all the methods you’ve learnt till now.
This is because not only does it give you the zeros of simple polynomial functions, but so long as your function is well-behaved enough, no matter what it is, the Newton-Raphson method will find your root.
1.2 Use Cases
Let’s discuss where this method would be used in today’s world.
We are doing this before looking at the method itself because I want you to ‘derive’ the concept by sensing the need.
The first is finding the zeros of functions involving terms that we simply cannot solve algebraically
An example of that looks like:
If you set this equal to zero, there won't be any clean way to isolate the x.
You might think this kind of situation occurs only in pure mathematics, but that is not at all the case!
For example, consider the famous Kepler’s Equation.
Here, E is an Angular quantity that is present both inside the sine function and algebraically as + E.
For this, no rearrangement can get you the value of E for given M and e.
Thus, organizations like NASA use Newton’s method to find E.
Lastly, not just transcendental equations, Newton's method also helps solve all sorts of high-degree polynomial equations that you could not solve algebraically or even by a brute-force formula (esp. for degrees 5+).
A good example of this would be when finding the rate required to achieve a certain total amount after a given time period from a given principal amount in compound interest.
The formula is:
So clearly, when n gets bigger, it gets harder and harder to find a solution rate; however, Newton's method can do it without any cap on n (it can even be fractional!)
Example of such a situation with A = 10,000 ; P = 2000 and n = 6 will end you up with:
Which is nearly impossible to solve without special methods such as this.
Alright, now that we’ve seen the basic ideas and use-cases associated with this method, let’s discuss the method itself.
And, of course, we shall first build the intuitive understanding before definitions.
However, I’d like to give you a quick historical note before we move on.
The method we will learn here is officially called the Newton-Raphson method and actually has a kind of complicated history.
Isaac Newton’s original 1669 version of this method used absolutely zero calculus.
His original method was purely algebraic.
He would make a guess, substitute a placeholder for the small “error” back into the polynomial, and ignore high-power terms because they were too small and hence get to solve a linear equation for the error. Then he would iterate this, by finding the “error” in the error and so on.
In 1690, Joseph Raphson simplified the method so you don't have to expand large polynomials every time and made it faster and optimised
The elegant, calculus version we will discuss today — using derivatives to find the tangents — was actually published later by Thomas Simpson in 1740.
So, it was finally Simpson who took Newton’s core concept of iterative error correction and used calculus to speed up the algebra.
However, to keep things simple, I’ll be using Newton's name as a placeholder for all the explanations.
1.3 Graphical Derivation
The goal is to find the zeros of a given function. Simple, right?
Like all of calculus, this method was developed by stepping back and thinking about the problem from a bird 's-eye view.
And the best way to get that view on a function is to observe its graph!
So, let's take an example function on which we are gonna do this observation:
This is a cubic whose roots we won't be able to find by any trial and error.
Then here's its graph in the section that we are worried about, i.e., the one with the root:
Right, so let’s begin thinking. Our final goal is to find the x-coordinate of the point marked Z (for zero).
Pause and think how you would approach this for a bit before I reveal the solution.
Done?
Well, here we go.
What Newton observed was that solving linear equations is way easier than any of the crazy ones he was trying to find the solutions to.
So, why not try somehow and bring linear equations into this?
That way, you can solve the simple linear functions and actually get the answer to the original hard problem.
Alright, sounds sweet, but how do we do it?
Well, the next thing he did was approach the toolkit of calculus — that he himself invented —, and specifically, the tool called tangents.
The idea is simple and very similar to how we created the Taylor approximation around chosen points ‘a ’, but we take it further with the power of iterations.
I’ll be more rigorous from now on.
The first step is to pick a reasonably close starting point to your actual root.
We normally also call this the ‘guess’ root, and it doesn't matter what it is as long as it's around the value of your actual root (which we will find).
For our case, we can see in the graph that the root is between 1 and 2, so I’ll take the guess root to be x_0 = 2.
Then the next step is to make a tangent line to the curve at that point.
Of course, this can also be called the linear approximation at x_0 = 2.
I’ve named the x-intercept of the tangent line as A1 (for Approximation 1).
These x-intercepts are the main stars of this method, and the ones that actually approach the answer later (hence the name, Approximation).
From here, we start iteration. The next step is to form the tangent line to the curve at the x-coordinate of point A1.
After the tangent is drawn, name its x-intercept as A2.
As you will see now, this gives us an A2 that is way closer to the actual root Z.
I’ll zoom in a bit so you can see the difference in Z and A2.
So, in just one iteration, we have gone from our bad guess of 2 to A1, which is 1.6363~, and now A2, which is 1.5303~, which is very close to our actual root.
But we want more accuracy, right? Let's do it one more time!
Take A2’s x-coordinate and draw a tangent to the curve at that point.
Repeating the same steps with x-intercept A3 gives us the new mind-boggling approximation in green here, compared to the pink A2 and the actual solution in blue:
I’ve shown the actual numbers here to showcase the scale; in just two steps of calculations and drawing tangents, we have already found the root correct to four decimal places!
And this method will only get closer and closer to the value. Try doing method 1, 2 more times, and see how you get near 8 decimal places by iteration 4!
Again, note that it feels the same in the visual graph to see the roots of the red function( our cubic) and the blue and black linear functions (tangents).
But with pen and paper, one of them is infinitely easier to solve than the other!
So, with this method, the scary cubics you could never find the roots of become simple problems of running the calculator!
The essence of this method can also be expressed as:
Think of it as finding the root of the linear approximation of the function, which will get us the solution of the original function when done iteratively.
Make sense?
To further our understanding, I’ll briefly mention the algebra that went behind making these tangents, as that's where the actual calculus is happening.
It's pretty simple, see, to make a tangent on a given point on the curve, we simply need two things: 1. The point on the curve itself, and 2. The value of the function's derivative at that point.
Note: The value of the function's derivative at a point is, by definition, the slope of the tangent line at that point.
Then all you have to do is write the line using the point-slope form:
So, here are the equations of the three lines we drew on the graph, respectively:
And the numbers in the x — () terms are our approximations x_0, A1, and A2.
The value of y is the value of the function at x equal to those values.
Get OreoPlushie’s stories in your inbox
Join Medium for free to get updates from this writer.
I hope you've realised the reason for applying this method. Finding the x-intercept from these equations is a complete piece of cake!
1.4 Algebraic Representation
Now, as much as I love the drawing-tangents method, it does pose a significant bottleneck in our solving.
We have to first sit and calculate the value of the function at each part, then write the line, find its x-intercept, and then repeat.
While not all, some of these steps can be cut down if we just do some algebra beforehand.
What I mean is that we are performing an algorithm with identical steps, which could be generalised into a formula!
To derive it:
Let's consider A_n to be our current iteration of Newton’s method and A_n+1 the next one.
Thus, by definition, A_n+1 is the x-intercept of the tangent line drawn to the curve — say f(x) — when the x-coordinate is A_n.
We use the same method as last time to get the tangent at the point with A_n as:
Next, we only have to find its x-intercept by putting y = 0:
And the ‘x ’ we are finding here is simply A_n+1, thus we have the final form of the formula as:
This is equivalent to doing our tangent drawing business, but we can now only focus on this one formula and computing the required values.
One more thing, this is also why this method is so widespread in use! It requires just one differentiation step, one division, and a subtraction!
This simplicity allows it to be easy not only for students in the exam, but more so for the computers doing this kind of solving every single second!
1.5 Picking the starting guess.
Okay, I think you've realised the power of this godsend of a method, let's learn the rules to properly use it, eh? After all, it is unfortunately not omnipotent.
The first question is how we pick the starting guess value.
This is important because if your guess is too far off the actual answer, the method will either take way too many iterations to reach the answer or get stuck in a loop and never get there.
The most powerful method here is, of course, looking at the graph, but algebraically we have a just as strong a tool known as the Intermediate Value Theorem.
The concept of the theorem is pretty simple:
If your continuous function has a positive value at one x and a negative value at another, it has to have taken the value zero at some point in between that interval, i.e., it must have a root in that interval.
The idea of using it is to first try a large range, like from -1 to 1, and see if you can restrict the root there. Then you must pick a dividing value inside that interval and check which half contains the root to narrow your range.
The dividing value is usually just taken to be the midpoint of the interval for simplicity.
Note: This interval “bisection” is also the building block of another numerical root-finding method called the “Bisection Method”
Alright, that's the first part. Next, we can look at some common points to make sure of in Newton’s method itself, so that it converges and gives you a meaningful value:
1. The function must be continuous and differentiable:
The curve cannot have any breaks, jumps, or missing points, and you should be able to take its derivative anywhere near your guess.
2. The derivative must not equal or approach zero:
If the slope becomes flat, the tangent line runs completely horizontal. It will never cross the x-axis, and the formula will do division by zero (the formula has f’(x) in the denominator.
Likewise, if the slope approaches zero, the fraction in the algebraic formula (aforementioned) will blow up to infinity and give bogus answers.
3. No sharp corners or cusps (derivative must not be undefined):
If the curve has a sharp point (like an absolute value) in the neighborhood of the root, the derivative becomes undefined, and the formula breaks down.
4. No vertical tangents or asymptotes (derivative must not become infinity):
If the curve becomes perfectly steep (vertical asymptote/vertical tangent), the derivative approaches infinity. This causes the formula’s fraction to shrink to zero, breaking your approximation so it never reaches the root.
All 4 of these are ONLY talking about the immediate vicinity of the root and your guess, but NOT the global behaviour of the function.
Out of these, 1 is something you understand from the function itself, and 2, 3, and 4 are from its derivative.
If you were to find any of the other rules being broken in your question, all you have to do is narrow the root down to an interval where there's no problem and pick a guess there.
1.6 Boss Fight.
It’s finally time to tackle today’s main question.
I’ve kept it simple because we are only just learning the concept, but the solution to this eqaution is actually very famous in the math community and even has a special name!
Here's the equation itself:
We will restrict x to [0, 2π]
Let's get right into it.
First, Newton’s method requires us to study a function to get its zeros, so from our equation, let's define f(x) in this way:
Now, I’ll show you the formula for Newton's method again, and then we shall find the roots of f(x)!
Here it is:
Now, clearly, we will need f’(x) for our purposes, so let's keep it handy.
It would come out to be:
Now, the next step is selecting a starting guess for the root. For this, we shall proceed with the bisection process I mentioned earlier.
Since the range for x is [0, 2π], let's keep halving it and checking the value of the function.
As a base, f(0) is 1.
Then f(π) is:
Right, so f(0) > 0 and f(π) < 0, so there's a root in [0, π]
Similarly, f(π/2) is:
Again, f(0) > 0 and f(π/2) < 0, so there’s a root in [0, π/2]
Then, halving the interval again, we have f(π/4) is:
Again, f(0) > 0 and f(π/4) < 0, so there’s a root in [0, π/4]!
See how much we have narrowed it down? From approx [0, 6.28] to about [0, 0.78]. This is the kind of pre-preparation that allows Newton’s method to reach those insane convergence numbers!
Since the value of the function at π/4 is already very close to zero, let's move on to our next step of the process while taking our original guess as π/4 itself!
Now, we have to use the formula to get better approximations of the root. This part is mostly for your calculator, so here are the final results.
This is the first iteration, now we put A_1 into the function and its derivative to get A_2, which is:
I’m taking so many decimal places to show the differences between the approximations, because if I only looked at 3 decimal places, the numbers would look identical.
For good measure, let’s do the process 2 more times.”
Then again,
Hey! Between the 3rd and the 4th iteration, the number didn’t change!
Well, no, it did, it’s just that we have gotten 10 decimal places of precision! So the numbers after that are changing, but I've truncated those away.
The point is that in only 4 iterations, we have the solution we were looking for with an accuracy of 0.0000000001 !!
This is the true power of Newton’s Method. It converges ‘quadratically’, under good conditions.
This means the number of correct places will double per iteration! ( For a sufficiently close starting guess)
Thus, numerically we have found the solution to the equation as:
Now, as I said before, this is a special number known as Dottie’s number after a famous anecdote of a French professor.
Quick Tangent (pun unintended)
Aside from being the unique solution to cosx = x, it is a transcendental, irrational number (like e and π) and also acts as a ‘fixed point’ for the cosine function.
Without getting too much into it, a fixed point of a function basically means that the function's output is the same as the input, but a cooler way of saying that is that it’s the limiting value of the iterative expression :
So, if we take Dottie's number as d, then it satisfies this relation:
No matter what input you put in!
1.7 Conclusion
So, there you have it. We started with the basic 2x + 1 = 17 problems and just beat a transcendental Boss Fight to find Dottie’s number!
The crazy part? The method’s true genius is using calculus to strip the scariest, “impossible” equations down to basic tangents.
Next time a textbook tells you a problem can’t be solved algebraically, you know exactly what to pull out of your toolkit. Just make a guess, draw a tangent, and let the iterations do the heavy lifting.
See ya’ll next week!
