Why Do We Need Limits and Infinitesimals?

Get the Math, Better Explained eBook and turn Huh? to Aha!

So many math courses jump into limits, infinitesimals and Very Small Numbers (TM) without any context. But why do we care?

Math helps us model the world. We can break a complex idea (a wiggly curve) into simpler parts (rectangles):

But, we want an accurate model. The thinner the rectangles, the more accurate the model. The simpler model, built from rectangles, is easier to analyze than dealing with the complex, amorphous blob directly.

The tricky part is making a decent model. Limits and infinitesimals help us create models that are simple to use, yet share the same properties as the original item (length, area, etc.).

The Paradox of Zero

Breaking a curve into rectangles has a problem: How do we get slices so thin we don’t notice them, but large enough to “exist”?

If the slices are too small to notice (zero width), then the model appears identical to the original shape (we don’t see any rectangles!). Now there’s no benefit — the ‘simple’ model is just as complex as the original! Additionally, adding up zero-width slices won’t get us anywhere.

If the slices are tiny but measurable, the illusion vanishes. We see that our model is a jagged approximation, and won’t be accurate. What’s a mathematician to do?

We want the best of both: slices so thin we can’t see them (for an accurate model) and slices thick enough to create a simpler, easier-to-analyze model. A dilemma is at hand!

The Solution: Zero is Relative

The notion of zero is biased by our expectations. Is “0 + i”, a purely imaginary number, the same as zero?

Well, “i” sure looks like zero when we’re on the real number line: the “real part” of i, Re(i), is indeed 0. Where else would a purely imaginary number go? (How far East is due North?)

Here’s a different brain bender: did your weight change by zero pounds while reading this sentence? Yes, by any scale you have nearby. But an atomic measurement would show some mass change due to sweat evaporation, exhalation, etc.

You see, there are two answers (so far!) to the “be zero and not zero” paradox:

  • Allow another dimension: Numbers measured to be zero in our dimension might actually be small but nonzero in another dimension (infinitesimal approach — a dimension infinitely smaller than the one we deal with)

  • Accept imperfection: Numbers measured to be zero are probably nonzero at a greater level of accuracy; saying something is “zero” really means “it’s 0 +/- our measurement error” (limit approach)

These approaches bridge the gap between “zero to us” and “nonzero at a greater level of accuracy”.

Overview of Limits & Infinitesimals

Let’s see how each approach would break a curve into rectangles:

  • Limits: “Give me your error margin (I know you have one, you limited, imperfect human!), and I’ll draw you a curve. What’s the smallest unit on your ruler? Inches? Fine, I’ll draw you a staircasey curve at the millimeter level and you’ll never know. Oh, you have a millimeter ruler, do you? I’ll draw the curve in nanometers. Whatever your accuracy, I’m better. You’ll never see the staircase.”

  • Infinitesimals: “Forget accuracy: there’s an entire infinitely small dimension where I’ll make the curve. The precision is totally beyond your reach — I’m at the sub-atomic level, and you’re a caveman who can barely walk and chew gum. It’s like getting to the imaginary plane from the real one — you just can’t do it. To you, the rectangular shape I made at the sub-atomic level is the most perfect curve you’ve ever seen.”

Limits stay in our dimension, but with ‘just enough’ accuracy to maintain the illusion of a perfect model. Infinitesimals build the model in another dimension, and it looks perfectly accurate in ours.

The trick to both approaches is that the simpler model was built beyond our level of accuracy. We might know the model is jagged, but we can’t tell the difference — any test we do shows the model and the real item as the same.

That trick doesn’t work, does it?

Oh, but it does. We’re tricked by “imperfect but useful” models all the time:

  • Audio files don’t contain all the information of the original signal. But can you tell the difference between a high-quality mp3 and a person talking in the other room?

  • Computer printouts are made from individual dots too small to see. Can you tell a handwritten note from a high-quality printout of the same?

  • Video shows still images at 24 times per second. This “imperfect” model is fast enough to trick our brain into seeing fluid motion.

On and on it goes. We resist because of our artificial need for precision. But audio and video engineers know they don’t need a perfect reproduction, just quality good enough to trick us into thinking it’s the original.

Calculus lets us make these technically imperfect but “accurate enough” models in math.

Working In Another Dimension

We need to be careful when reasoning with the simplified model. We need to “do our work” at the level of higher accuracy, and bring the final result back to our world. We’ll lose information if we don’t.

Suppose an imaginary number (i) visits the real number line. Everyone thinks he’s zero: after all, Re(i) = 0. But i does a trick! “Square me!” he says, and they do: “i * i = -1″ and the other numbers are astonished.

To the real numbers, it appeared that “0 * 0 = -1″, a giant paradox.

But their confusion arose from their perspective — they only thought it was “0 * 0 = -1″. Yes, Re(i) * Re(i) = 0, but that wasn’t the operation! We want Re(i * i), which is different entirely! We square i in its own dimension, and bring that result back to ours. We need to square i, the imaginary number, and not 0, our idea of what i was.

Beware similar mistakes in calculus: we deal with tiny numbers that look like zero to us, but we can’t do math assuming they are (just like treating i like 0). No, we need to “do the math” in the other dimension and convert the results back.

Limits and infinitesimals have different perspectives on how this conversion is done:

  • Limits: “Do the math” at a level of precision just beyond your detection (millimeters), and bring it back to numbers on your scale (inches)

  • Infinitesimals: “Do the math” in a different dimension, and bring it back to the “standard” one (just like taking the real part of a complex number; you take the “standard” part of a hyperreal number — more later)

Nobody ever told me: Calculus lets you work at a better level of accuracy, with a simpler model, and bring the results back to our world.

A Real Example: sin(x) / x

Let’s try a conceptual example. Suppose we want to know what happens to sin(x) / x at zero. Now, if we just plug in x = 0 we get a nonsensical result: sin(0) = 0, so we get 0 / 0 which could be anything.

Let’s step back: what does “x = 0″ mean in our world? Well, if we’re allowing the existence of a greater level of accuracy, we know this:

  • Things that appear to be zero may be nonzero in a different dimension (just like i might appear to be 0 to us, but isn’t)

We’re going to say that x can be really, really close to zero at this greater level of accuracy, but not “true zero”. Intuitively, you can think of x as 0.0000…00001, where the “…” is enough zeros for you to no longer detect the number.

(In limit terms, we say x = 0 + d (delta, a small change that keeps us within our error margin) and in infinitesimal terms, we say x = 0 + h, where h is a tiny hyperreal number, known as an infinitesimal)

Ok, we have x at “zero to us, but not really”. Now we need a simpler model of sin(x). Why? Well, sine is a crazy repeating curve, and it’s hard to know what’s happening. But it turns out that a straight line is a darn good model of a curve over short distances:

Just like we can break a filled shape into tiny rectangles to make it simpler, we can dissect a curve into a series of line segments. Around 0, sin(x) looks like the line “x”. So, we switch sin(x) with the line “x”. What’s the new ratio?

\displaystyle{ \frac{sin(x)}{x} \sim \frac{x}{x} = 1 }

Well, “x/x” is 1. Remember, we aren’t really dividing by zero because in this super-accurate world: x is tiny but non-zero (0 + d, or 0 + h). When we “take the limit or “take the standard part” it means we do the math (x / x = 1) and then find the closest number in our world (1 goes to 1).

So, 1 is what we get when sin(x) / x approaches zero — that is, we make x as small as possible so it becomes 0 to us. If x became pure, true zero, then the ratio would be undefined (and it is at the infinitesimal level!). But we’re never sure if we’re at perfect zero — something like 0.0000…0001 looks like zero to us.

So, “sin(x)/x” looks like “x/x = 1″ as far as we can tell. Intuitively, the result makes sense once we read about radians).

Visualizing The Process

Today’s goal isn’t to solve limit problems, it’s to understand the process of solving them. To solve this example:

  • Realize x=0 is not reachable from our accuracy; a “small but nonzero” x is always available at a greater level of accuracy
  • Replace sin(x) by a straight line as a simpler model
  • “Do the math” with the simpler model (x / x = 1)
  • Bring the result (1) back into our accuracy (stays 1)

Here’s how I see the process:

In later articles, we’ll learn the details of setting up and solving the models.

Caveats: The Trick Doesn’t Always Work

Some functions are really “jumpy” — and they might differ on an infinitesimal-by-infinitesimal level. That means we can’t reliably bring them back to our world. It looks like the function is unstable at microscopic level and doesn’t behave “smoothly”.

The rigorous part of limits is figuring out which functions behave well enough that simple yet accurate models can be made. Fortunately, most of the natural functions in the world (x, x2, sin, ex) behave nicely and can be modeled with calculus.

Limits Or Infinitesimals?

Logically, both approaches solve the problem of “zero and nonzero”. I like infinitesimals because they allow “another dimension” which seems a cleaner separation than “always just outside your reach”. Infinitesimals were the foundation of the intuition of calculus, and appear inside physics and other subjects that use it.

This isn’t an analysis class, but the math robots can be assured that infinitesimals have a rigorous foundation. I use them because they click for me.

Summary

Phew! Some of these ideas are tricky, and I feel like I’m talking from both sides of my mouth: we want to be simpler, yet still perfectly accurate?

This famous dilemma about “being zero sometimes, and non-zero others” is a famous critique of calculus. It was mostly ignored since the results worked out, but in the 1800s limits were introduced to really resolve the dilemma. We learn limits today, but without understanding the nature of the problem they were trying to solve!

Here are the key concepts:

  • Zero is relative: something can be zero to us, and non-zero somewhere else
  • Infinitesimals (“another dimension”) and limits (“beyond our accuracy”) resolve the dilemma of “zero and nonzero”
  • We create simpler models in the more accurate dimension, do the math, and bring the result to our world
  • The final result is perfectly accurate for us

My goal isn’t to do math, it’s to understand it. And a huge part of grokking calculus is realizing that simple models created beyond our accuracy can look “just fine” in our dimension. Later on we’ll learn the rules to build and use these models. Happy math.

Other Posts In This Series

  1. A Gentle Introduction To Learning Calculus
  2. How To Understand Derivatives: The Product, Power & Chain Rules
  3. How To Understand Derivatives: The Quotient Rule, Exponents, and Logarithms
  4. An Intuitive Introduction To Limits
  5. Why Do We Need Limits and Infinitesimals?
  6. Learning Calculus: Overcoming Our Artificial Need for Precision
  7. Prehistoric Calculus: Discovering Pi
  8. A Calculus Analogy: Integrals as Multiplication
  9. Calculus: Building Intuition for the Derivative
  10. Understanding Calculus With A Bank Account Metaphor
  11. A Friendly Chat About Whether 0.999... = 1
Kalid Azad loves sharing Aha! moments. BetterExplained is dedicated to learning with intuition, not memorization, and is honored to serve 250k readers monthly.

Enjoy this article? Try the site guide or join the newsletter:
Math, Better Explained is a highly-regarded Amazon bestseller. This 12-part book explains math essentials in a friendly, intuitive manner.

"If 6 stars were an option I'd give 6 stars." -- read more reviews

32 Comments

  1. Hi Kalid! =) Wonderful to have those insights as always!

    Just one very unimportant correction; you said:
    “Video shows still images at 24 times per second”
    However the correct would be to say that Film shows the frames at 24 times/second. Even when converted to digital, it is sped-up to 25 FPS (PAL video standart) or “telecined” to 29.97 FPS (NTSC standart, USA’s video)
    Note: It’s not so simple, there are other standarts and variants, I was refering to SD video (not HD, which is commonly the double FPS) and also: digital video on a computer can have any FPS, even floating numbers and variable frame-rate (a totally complex thing to handle, easier to get in than to get out of it)

    And as I’m talking about video, I’d like to suggest an idea for an article: how digital video and image works. It’s a fabulous bunch of AHA! moments when you get the concept behind all of those blocky artifacts and color schemes other than ye olde RGB.

    And I loved to think on infinitesimals in a new way! Thanks! =D

  2. Hi Kalid,

    Thanks for the wonderful post. I have totally forgotten all my math and have been thinking of re-learning it (especially from a computer science perspective).

    I found your post very useful, and I think it will also give that little push I needed to get started.

    BTW, I too share your passion for helping others learn. I have aggregated various open computer science course videos on my website.

  3. Hi….This one is as good as the previous posts! I appreciate ur enthusiasm in promoting the interest in Math among young readers! I enjoyed every bit of the article man….Thank you very much….

  4. @Camilo: Ah, thank you for the clarification! I’ll change ‘video’ to ‘film’ :).

    That’d be a really cool article — I don’t know too much about the video formats, but know that MPEG has some really neat technology to make it compress well.

    @Parag: Great, glad you enjoyed the article! Checking out your site now, thanks for collecting all those links — I’m hoping to go back and refresh a lot of my cs knowledge also :).

    @Murugesh: Thanks for the support, I’ve had a lot of fun trying to get my brain around these concepts again, but being able to ask “Wait, what does it _really_ mean to me?”. Glad it was useful for you!

  5. I found a few typos:

    Paradox of zero: “slices so thin we can’t _seem_ them” (see)
    Summary: “I feel like I’m talking from _boths ides_ of my mouth” (both sides)
    Summary: “_Here’s_ the key concepts” (Here are)

    Thanks for another great article!

  6. Hi Khalid, props for this great series which I just found out recently and reading your posts has been a daily habit for me.

    Just one confusion in this topic, can you elaborate more on this:
    ——
    Around 0, sin(x) looks like the line “x”.
    ——

    I think of x as the x-axis in the plane that was demonstrated. It could also be just the variable in the equation. But neither makes sense. I know x/x is 1, but how come sin(x) is x?

    thanks! and more power!

  7. Hello, Kalid,
    Very well-written and descriptive. Thank you for giving me a good and pleasant read on things past and nearly forgotten!

    I could only wish that more people like you were teaching in high schools and universities. Around here, the tutors are often skilled in their field, but regularly and gravely fail to convey the meaning behind the definitions, theorems and proofs they teach – only the items themselves; and the educational process plummets.

    Arbie:
    ——
    Around 0, sin(x) looks like the line “x”.
    ——
    I believe this means the line “y = x”. Thus y_1 = sin(x), y_2 = x and y1 ~= y2 for x -> 0.

  8. @Arbie: Wow, I like that functional representation of it! Yes, integrations are a general “applying” one function to another, vs. some static multiplication just to find area (area just limits our creativity/intuition I think).

    Ah, I should be more clear about that… the I meant the line “y = x”, that is, a 45 degree line extending from the origin. So the equation y = sin(x) looks very similar to y = x for very small numbers (sin(x) extends 45 degrees from the origin when it first starts off).

    Hope this helps!

  9. @mcmlxxxvi: Glad you enjoyed it, and thanks for the comment. I too wish there was more emphasis on true understanding vs. the “let’s learn enough to pass the next test” mentality. Learning the intuition may take a bit longer than memorizing in the short term, but in the long run it gives you a more flexible set of knowledge, and not to mention it’s way more fun. I sometimes see grades as a curse because rather than being an indication of knowledge, they become an end in itself vs. the learning it should represent. It’s very hard to test intuition — it’s a gutcheck you need to ask yourself. But with no grades there’s no “incentive” (carrot or stick) — I don’t know the answer, but I too wish there was another way.

  10. Smooth Infinitesimal Analysis handles infinitesimals better than Non-Standard Analysis:

    http://en.wikipedia.org/wiki/Smooth_infinitesimal_analysis

    In intuitionistic math, the law of excluded middle is rejected (i.e. not not A doesn’t imply A) so you must provide an algorithm for constructing all your objects.

    There is no general procedure for detecting whether or not 2 objects are equal. You must explicitly provide an algorithm for showing 2 objects are equal.

    The trichotomy law (a b, a = b) doesn’t hold in general.

    All functions are continuous. Piecewise functions are nonsensical.

    In other words, the continuum is unbreakable into points. Functions transform the continuum onto the continuum.

    With this as our basis, Smooth Infinitesimal Analysis introduces an object called epsilon.

    There is no algorithm to tell whether or not epsilon != 0 or epsilon = 0. This avoids the first problem entirely.

    epsilon^2 = 0 though which gives us a way to get rid of them from our formulas.

    So I view infinitesimals as the glue that makes the continuum unbreakable and there is no algorithm to decide if the expression “epsilon = 0 or epsilon != 0″ is true (see why we have to reject the law of excluded middle to make this work?).

  11. @asdf: Wow, really interesting stuff! I like that insight of infinitesimals as the “glue” that makes the continuum unbreakable. Great analogy.

  12. Hey, Kalid, I’ve just got a quick question to ask.

    If you learn calculus via the use of infinitesimals, is it possible to then make the leap over to using limits? While I doubt it would happen, I’d like to be an amateur mathematician in the vein of Fermat some time and develop proofs (more as a beauty thing, to be honest), but writing in a fashion that is contrary to the norm is rather like handing out Spanish pamphlets in an English neighborhood- they might understand, but they won’t like it.

    So, yeah, can you jump from infinitesimals over to limits? From what I can tell, limits are mainly used because they’re easily to rigorously define an to keep the constructivist camp from yelling at you.

  13. @Dave: Great question. I can’t say I’m completely comfortable with limits, but I think you can jump back and forth (the Keisler Calculus book has some examples like this I believe). I think the bigger goal is to figure out what is being said, i.e. “What does this equation equal, within some level of tolerance?”. Limits and infinitesimals are two ways to define that tolerance threshold, but infinitesimals are “easier” in that it’s built in (and you don’t need to explicitly define epsilon, delta, etc.).

  14. @werterber: Not a silly question at all! In my head, it’s saying “what’s the ratio of width [cos(x)] to distance traveled (x)”.

    As our distance traveled goes to 0 (we aren’t moving from the starting point), cos(x) tends towards 1 — we’re pretty much at the same width. So it becomes “1 / 0″ in my head.

  15. This comes down to this: we can’t possibly describe what we can’t possibly imagine. That’s why it must always be “small enough rectangles” of a sort…

    Interestingly, Brian Greene in his “Elegant Universe” gives to understand, that the “superstring theory”, along with expected resolution of some fundamental problems, must bring about radical change in mathematical modes, so that you can’t decrease the size of those “small rectangles” down to infinity, but that it must have its limit somewhere around the level Plank constant ~10^-34. After which further decrease will actually mean increase.
    Now every theory serves for some convenience. Therefore, aren’t we free to take such approximation with those rectangles, as will serve our purpose the best? And not bother any more than we can help? Cause that’s what we do anyway.

  16. Hey, Kalid … You hold a marvelous scape valve from the montains of unintuitive theorems and corolaries contained in every text-book.Outside, our memory rests in peace, and the big picture awakes our deep passions about math.Oh, precious and full of insight scape valve.

  17. Thanks Kalid,
    Your articles did help me a lot.

    By the way, what software do you use to illustrate examples in your articles (like this one)? Thanks

  18. Hey khalid plz i am getting a doubt !
    You said that infinitesimal are the values which we cannot measure ! My question is can we imgaine infinitesimal ?? According to me , humans can think only of finite values ….so whenever we try to assign a value to infinitesimal it woud be of finite digits and tat would be against the defination of infinitesimal …… So according to me due to the limited scope of human brain we can never think of what value wud be of infinitesimal …… Am i correct plz ????

  19. Hi khalid plz i hav a doubt ??
    My question is can we think about wat number would be infinitesimal ??
    According to me we humans hav a limited horizon of thinking and so we can just think of finit numbers……. So even if we assign any value to infinitesimal it would be some fiite value and a value smaller than it will still exist…….. So is the limit which we are talking about is the limit of our brains to comprehend such small amounts ??? Plz help ??

  20. @kalid
    @ kalid

    Hi kalid plz i hav a doubt ?? My question is can we think about wat number would be infinitesimal ?? According to me we humans hav a limited horizon of thinking and so we can just think of finit numbers……. So even if we assign any value to infinitesimal it would be some fiite value and a value smaller than it will still exist…….. So is the limit which we are talking about is the limit of our brains to comprehend such small amounts ??? Plz help ??

  21. @Dave
    post 17

    Regarding your question, “If you learn calculus via the use of infinitesimals, is it possible to then make the leap over to using limits?”, I suppose it is possible for I have (in a way) done it, though I never knew I was learning infinitesimals.

    I must admit that prior to reading this post I have never even heard about the defined mathematic concept of ‘infinitesimal’. I also never took a formal Calculus course. I originally learned Calc in my AP physics class in high school. Our teacher (one of the few who truly loved the craft of teaching and had a passion for what she did) had both the constraint of putting her Physics class on hold to teach Calc to those who have never seen it, and also the freedom that brevity provided; she was free to teach the idea of calculus without the strict procedural rigor that a formal class drags its pupil through. We learned the basic idea of the integral before the derivative, heresy in Calc101. Here it is 21 years later and I can still hear her voice saying ‘Taking the integral just means add up a whole bunch of things, and ‘taking a differential element of’ just means cut the thing into really teenie weenie chunks.” We learned the idea of a derivative as slope of a function without being given 2 points, just one point and an interval to the next. After seeing what happened as the interval got smaller we finally visualized ‘slope at a point’. Only afterward were we shown the ‘official’ formula with a limit in it. I saw it as a perfectly nice piece of legal-eez that made the rest of the world happy for me to have learned the ‘right way’, and I was enormously grateful our teacher taught us the intuitive way.

  22. Fascinating article Kalid!

    This is something new for me. After reading this post I started some research on infinitesimals, and quickly re-affirmed how valuable your common sense approach is by comparison to an army of equations, lemmas, and theorems.

    My great ‘a-ha’ moment was your description of infinitesimals as another dimension, similar to the way imaginary numbers are another dimension to reals. In a strange way, that may not be obvious at first, it reminded me of a conundrum I faced learning the history of physics. It seems that every time we define what an ‘element’ is -the smallest indivisible component of a thing- some clever lad comes along later and figures out way to break that ‘element’ into something smaller. This means, of course, that the old thing never was a true element, we just thought it was. But then what about this new ‘element’, how can we know it is the smallest thing?
    A revelation came when I realized that in order to be an ‘element’ we don’t really need it to be true that you can’t break it apart, it just means that if you do break it down further then it is no longer the same stuff. Thus the element is really just the smallest possible piece of a thing WHICH can still be the same thing. E.g. an ‘element’ of water (H2O) can be broken down, but it is no longer water, just hydrogen and oxygen atoms. An atom can be broken down into protons, neutrons, and electrons, but it is no longer the same stuff as the original atom. A little chunk of matter (a superstring exhibiting one class of vibration in 10 dimensions) can indeed be broken down, it is just no longer matter. It is also not exactly energy, but when the ‘stuff’ comes back together in a different pattern (the superstring having the same vibration just in a different dimension) it appears to us as a little chunk of energy.
    It seems natural to me to take a cue from the physical world to comprehend numbers. When we look at an element and it appears we’ve ‘hit the limit’ in terms of breaking it up, but we can go further it just means we have to view it in a different dimension. Why then could we not do the same with numbers? Here’s a rational number you can only break it apart but to a certain extent and no smaller. I know you may object and say ‘take that number and divide by 2, it is smaller and still rational’. But take notice of the irrationals, like sqrt(2). It does exist, sitting there staring us in the face. It is in between rationals. So how does there exist any space between rationals? How can the rationals be broken down finer than it is possible to break them down? Imagine thinking you understand that atoms are elementary particles, then this clown Rutherford comes along and experimentally identifies this object (nucleus) in the middle of an atom.
    I say the best way forward is to take as true those things that must be true and re-evaluate our preconceived notions that have pigeon-holed us into an apparent paradox. It is difficult and un-nerving. You can be guaranteed you’ll get it wrong a few times before you make some progress, but some progress is far better than the certainty of smaller minds.

  23. Thanks Eric, that’s a really thought-provoking comment.

    I think the element analogy is apt, we’re able to function at a certain level (water molecules) and while we *can* go to a deeper level (individual subatomic particles) those details presumably don’t change the measurements we’re making at the macro level. In the same way, infinitesimals can bounce around in funny ways but not effect the numbers one level up. (I.e., when we switch domains, the infinitesimal part goes to 0.)

    Trying to “fill in” the number line with rationals is another great example. We have a smooth continuum on the number line, but the rationals are so sparse they’ll never complete it! There must be another way to get to those in-between numbers, and it isn’t by dividing the ones we have into smaller bits.

Your feedback is welcome -- leave a reply!

Your email address will not be published.

LaTeX: $$e=mc^2$$