So many math courses jump into limits, infinitesimals and Very Small Numbers (TM) without any context. But why do we care?

Math helps us model the world. We can break a complex idea (a wiggly curve) into simpler parts (rectangles):

But, we want an accurate model. The thinner the rectangles, the more accurate the model. The simpler model, built from rectangles, is easier to analyze than dealing with the complex, amorphous blob directly.

The tricky part is making a decent model. Limits and infinitesimals help us create models that are simple to use, yet share the same properties as the original item (length, area, etc.).

## The Paradox of Zero

Breaking a curve into rectangles has a problem: How do we get slices so thin we don’t notice them, but large enough to “exist”?

If the slices are too small to notice (zero width), then the model appears identical to the original shape (we don’t see any rectangles!). Now there’s no benefit — the ‘simple’ model is just as complex as the original! Additionally, adding up zero-width slices won’t get us anywhere.

If the slices are tiny but measurable, the illusion vanishes. We *see* that our model is a jagged approximation, and won’t be accurate. What’s a mathematician to do?

We want the best of both: slices so thin we can’t see them (for an accurate model) and slices thick enough to create a simpler, easier-to-analyze model. A dilemma is at hand!

## The Solution: Zero is Relative

The notion of zero is biased by our expectations. Is “0 + i”, a purely imaginary number, the same as zero?

Well, “i” sure looks like zero when we’re on the real number line: the “real part” of i, Re(i), is indeed 0. Where else would a purely imaginary number go? (How far East is due North?)

Here’s a different brain bender: did your weight change by zero pounds while reading this sentence? Yes, by any scale you have nearby. But an atomic measurement would show *some* mass change due to sweat evaporation, exhalation, etc.

You see, there are two answers (so far!) to the “be zero and not zero” paradox:

**Allow another dimension**: Numbers measured to be zero in our dimension might actually be small but nonzero in another dimension (infinitesimal approach — a dimension*infinitely smaller*than the one we deal with)**Accept imperfection**: Numbers measured to be zero are probably nonzero at a greater level of accuracy; saying something is “zero” really means “it’s 0 +/- our measurement error” (limit approach)

These approaches bridge the gap between “zero to us” and “nonzero at a greater level of accuracy”.

## Overview of Limits & Infinitesimals

Let’s see how each approach would break a curve into rectangles:

**Limits:**“Give me your error margin (I know you have one, you limited, imperfect human!), and I’ll draw you a curve. What’s the smallest unit on your ruler? Inches? Fine, I’ll draw you a staircasey curve at the millimeter level and you’ll never know. Oh, you have a millimeter ruler, do you? I’ll draw the curve in nanometers. Whatever your accuracy, I’m better. You’ll never see the staircase.”**Infinitesimals:**“Forget accuracy: there’s an entire*infinitely small dimension*where I’ll make the curve. The precision is totally beyond your reach — I’m at the sub-atomic level, and you’re a caveman who can barely walk and chew gum. It’s like getting to the imaginary plane from the real one — you just can’t do it. To you, the rectangular shape I made at the sub-atomic level is the most perfect curve you’ve ever seen.”

Limits stay in our dimension, but with ‘just enough’ accuracy to maintain the illusion of a perfect model. Infinitesimals build the model in another dimension, and it looks perfectly accurate in ours.

The trick to both approaches is that the simpler model was built beyond our level of accuracy. We might *know* the model is jagged, but we can’t tell the difference — any test we do shows the model and the real item as the same.

## That trick doesn’t work, does it?

Oh, but it does. We’re tricked by “imperfect but useful” models all the time:

Audio files don’t contain all the information of the original signal. But can you tell the difference between a high-quality mp3 and a person talking in the other room?

Computer printouts are made from individual dots too small to see. Can you tell a handwritten note from a high-quality printout of the same?

Video shows still images at 24 times per second. This “imperfect” model is fast enough to trick our brain into seeing fluid motion.

On and on it goes. We resist because of our artificial need for precision. But audio and video engineers know they don’t need a perfect reproduction, just quality *good enough* to trick us into thinking it’s the original.

Calculus lets us make these technically imperfect but “accurate enough” models in math.

## Working In Another Dimension

We need to be careful when reasoning with the simplified model. We need to “do our work” at the level of higher accuracy, and bring the *final result* back to our world. We’ll lose information if we don’t.

Suppose an imaginary number (i) visits the real number line. Everyone thinks he’s zero: after all, Re(i) = 0. But i does a trick! “Square me!” he says, and they do: “i * i = -1″ and the other numbers are astonished.

To the real numbers, it appeared that “0 * 0 = -1″, a giant paradox.

But their confusion arose from their perspective — they only *thought* it was “0 * 0 = -1″. Yes, Re(i) * Re(i) = 0, but that wasn’t the operation! We want Re(i * i), which is different entirely! We square i in its own dimension, and bring *that* result back to ours. We need to square i, the imaginary number, and not 0, our *idea* of what i was.

Beware similar mistakes in calculus: we deal with tiny numbers that *look like zero* to us, but we can’t do math assuming they are (just like treating i like 0). No, we need to “do the math” in the other dimension and convert the results back.

Limits and infinitesimals have different perspectives on how this conversion is done:

**Limits:**“Do the math” at a level of precision just beyond your detection (millimeters), and bring it back to numbers on your scale (inches)**Infinitesimals:**“Do the math” in a different dimension, and bring it back to the “standard” one (just like taking the real part of a complex number; you take the “standard” part of a hyperreal number — more later)

Nobody ever told me: Calculus lets you work at a better level of accuracy, with a simpler model, and bring the results back to our world.

## A Real Example: sin(x) / x

Let’s try a conceptual example. Suppose we want to know what happens to sin(x) / x at zero. Now, if we just plug in x = 0 we get a nonsensical result: sin(0) = 0, so we get 0 / 0 which could be anything.

Let’s step back: what does “x = 0″ mean in our world? Well, if we’re allowing the existence of a greater level of accuracy, we know this:

- Things that
*appear*to be zero may be nonzero in a different dimension (just like i might appear to be 0 to us, but isn’t)

We’re going to say that x can be really, really close to zero at this greater level of accuracy, but not “true zero”. Intuitively, you can think of x as 0.0000…00001, where the “…” is enough zeros for you to no longer detect the number.

(In limit terms, we say x = 0 + d (delta, a small change that keeps us within our error margin) and in infinitesimal terms, we say x = 0 + h, where h is a tiny hyperreal number, known as an infinitesimal)

Ok, we have x at “zero to us, but not really”. Now we need a simpler model of sin(x). Why? Well, sine is a crazy repeating curve, and it’s hard to know what’s happening. But it turns out that a *straight line* is a darn good model of a curve over short distances:

Just like we can break a filled shape into tiny rectangles to make it simpler, we can dissect a curve into a series of line segments. Around 0, sin(x) looks like the line “x”. So, we switch sin(x) with the line “x”. What’s the new ratio?

Well, "x/x" is 1. Remember, we aren’t really dividing by zero because in this super-accurate world: x is tiny but non-zero (0 + d, or 0 + h). When we “take the limit or “take the standard part” it means we do the math (x / x = 1) and then find the closest number in our world (1 goes to 1).

So, 1 is what we get when sin(x) / x approaches zero — that is, we make x as small as possible so it becomes 0 to us. If x became pure, true zero, then the ratio would be undefined (and it is at the infinitesimal level!). But we’re never sure if we’re at perfect zero — something like 0.0000…0001 looks like zero to us.

So, "sin(x)/x" looks like "x/x = 1" as far as we can tell. Intuitively, the result makes sense once we read about radians).

## Visualizing The Process

Today’s goal isn’t to solve limit problems, it’s to understand the process of solving them. To solve this example:

- Realize x=0 is not reachable from our accuracy; a “small but nonzero” x is always available at a greater level of accuracy
- Replace sin(x) by a straight line as a simpler model
- “Do the math” with the simpler model (x / x = 1)
- Bring the result (1) back into our accuracy (stays 1)

Here’s how I see the process:

In later articles, we’ll learn the details of setting up and solving the models.

## Caveats: The Trick Doesn’t Always Work

Some functions are really “jumpy” — and they might differ on an infinitesimal-by-infinitesimal level. That means we can’t reliably bring them back to our world. It looks like the function is unstable at microscopic level and doesn’t behave “smoothly”.

The rigorous part of limits is figuring out which functions behave well enough that simple yet accurate models can be made. Fortunately, most of the natural functions in the world (x, x^{2}, sin, e^{x}) behave nicely and *can* be modeled with calculus.

## Limits Or Infinitesimals?

Logically, both approaches solve the problem of “zero and nonzero”. I like infinitesimals because they allow “another dimension” which seems a cleaner separation than “always just outside your reach”. Infinitesimals were the foundation of the intuition of calculus, and appear inside physics and other subjects that use it.

This isn’t an analysis class, but the math robots can be assured that infinitesimals have a rigorous foundation. I use them because they click for me.

## Summary

Phew! Some of these ideas are tricky, and I feel like I’m talking from both sides of my mouth: we want to be simpler, yet still perfectly accurate?

This famous dilemma about “being zero sometimes, and non-zero others” is a famous critique of calculus. It was mostly ignored since the results worked out, but in the 1800s limits were introduced to really resolve the dilemma. We learn limits today, but without understanding the nature of the problem they were trying to solve!

Here are the key concepts:

- Zero is relative: something can be zero to us, and non-zero somewhere else
- Infinitesimals (“another dimension”) and limits (“beyond our accuracy”) resolve the dilemma of “zero and nonzero”
- We create simpler models in the more accurate dimension, do the math, and bring the result to our world
- The final result is perfectly accurate for us

My goal isn’t to do math, it’s to understand it. And a huge part of grokking calculus is realizing that simple models created beyond our accuracy can look “just fine” in our dimension. Later on we’ll learn the rules to build and use these models. Happy math.

## Other Posts In This Series

- A Gentle Introduction To Learning Calculus
- Understanding Calculus With A Bank Account Metaphor
- Prehistoric Calculus: Discovering Pi
- A Calculus Analogy: Integrals as Multiplication
- Calculus: Building Intuition for the Derivative
- How To Understand Derivatives: The Product, Power & Chain Rules
- How To Understand Derivatives: The Quotient Rule, Exponents, and Logarithms
- An Intuitive Introduction To Limits
- Why Do We Need Limits and Infinitesimals?
- Learning Calculus: Overcoming Our Artificial Need for Precision
- A Friendly Chat About Whether 0.999... = 1
- Analogy: The Calculus Camera
- Abstraction Practice: Calculus Graphs

## Leave a Reply

44 Comments on "Why Do We Need Limits and Infinitesimals?"

Hi Kalid! =) Wonderful to have those insights as always!

Just one very unimportant correction; you said:

“Video shows still images at 24 times per second”

However the correct would be to say that

Filmshows the frames at 24 times/second. Even when converted to digital, it is sped-up to 25 FPS (PALvideostandart) or “telecined” to 29.97 FPS (NTSC standart, USA’s video)Note: It’s not so simple, there are other standarts and variants, I was refering to SD video (not HD, which is commonly the double FPS) and also: digital video on a computer can have any FPS, even floating numbers and variable frame-rate (a totally complex thing to handle, easier to get in than to get out of it)

And as I’m talking about video, I’d like to suggest an idea for an article: how digital video and image works. It’s a fabulous bunch of AHA! moments when you get the concept behind all of those blocky artifacts and color schemes other than ye olde RGB.

And I loved to think on infinitesimals in a new way! Thanks! =D

Hi Kalid,

Thanks for the wonderful post. I have totally forgotten all my math and have been thinking of re-learning it (especially from a computer science perspective).

I found your post very useful, and I think it will also give that little push I needed to get started.

BTW, I too share your passion for helping others learn. I have aggregated various open computer science course videos on my website.

Hi….This one is as good as the previous posts! I appreciate ur enthusiasm in promoting the interest in Math among young readers! I enjoyed every bit of the article man….Thank you very much….

@Camilo: Ah, thank you for the clarification! I’ll change ‘video’ to ‘film’ :).

That’d be a really cool article — I don’t know too much about the video formats, but know that MPEG has some really neat technology to make it compress well.

@Parag: Great, glad you enjoyed the article! Checking out your site now, thanks for collecting all those links — I’m hoping to go back and refresh a lot of my cs knowledge also :).

@Murugesh: Thanks for the support, I’ve had a lot of fun trying to get my brain around these concepts again, but being able to ask “Wait, what does it _really_ mean to me?”. Glad it was useful for you!

I found a few typos:

Paradox of zero: “slices so thin we can’t _seem_ them” (see)

Summary: “I feel like I’m talking from _boths ides_ of my mouth” (both sides)

Summary: “_Here’s_ the key concepts” (Here are)

Thanks for another great article!

@Anonymous: You’re welcome, and thanks for the corrections! I just made them now.

Again I started reading and did not realized that I finished a long article! It was very entertaining. Thank you very much for your efforts.

@nanoturkiye: You’re welcome! Glad you enjoyed the article.

Hi Khalid, props for this great series which I just found out recently and reading your posts has been a daily habit for me.

Just one confusion in this topic, can you elaborate more on this:

——

Around 0, sin(x) looks like the line “x”.

——

I think of x as the x-axis in the plane that was demonstrated. It could also be just the variable in the equation. But neither makes sense. I know x/x is 1, but how come sin(x) is x?

thanks! and more power!

@Arbie: Wow, I like that functional representation of it! Yes, integrations are a general “applying” one function to another, vs. some static multiplication just to find area (area just limits our creativity/intuition I think).

Ah, I should be more clear about that… the I meant the line “y = x”, that is, a 45 degree line extending from the origin. So the equation y = sin(x) looks very similar to y = x for very small numbers (sin(x) extends 45 degrees from the origin when it first starts off).

Hope this helps!

Hello, Kalid,

Very well-written and descriptive. Thank you for giving me a good and pleasant read on things past and nearly forgotten!

I could only wish that more people like you were teaching in high schools and universities. Around here, the tutors are often skilled in their field, but regularly and gravely fail to convey the meaning behind the definitions, theorems and proofs they teach – only the items themselves; and the educational process plummets.

Arbie:

——

Around 0, sin(x) looks like the line “x”.

——

I believe this means the line “y = x”. Thus y_1 = sin(x), y_2 = x and y1 ~= y2 for x -> 0.

@mcmlxxxvi: Glad you enjoyed it, and thanks for the comment. I too wish there was more emphasis on true understanding vs. the “let’s learn enough to pass the next test” mentality. Learning the intuition may take a bit longer than memorizing in the short term, but in the long run it gives you a more flexible set of knowledge, and not to mention it’s way more fun. I sometimes see grades as a curse because rather than being an indication of knowledge, they become an end in itself vs. the learning it should represent. It’s very hard to test intuition — it’s a gutcheck you need to ask yourself. But with no grades there’s no “incentive” (carrot or stick) — I don’t know the answer, but I too wish there was another way.

This paper offers similar views about mathematics education as well as a criticism of the cultural opinion of mathematics that you might like. http://maa[dot]org/devlin/devlin_03_08.html

@Anonymous: Thank you — I’ve seen the essay and really like it :).

Smooth Infinitesimal Analysis handles infinitesimals better than Non-Standard Analysis:

http://en.wikipedia.org/wiki/Smooth_infinitesimal_analysis

In intuitionistic math, the law of excluded middle is rejected (i.e. not not A doesn’t imply A) so you must provide an algorithm for constructing all your objects.

There is no general procedure for detecting whether or not 2 objects are equal. You must explicitly provide an algorithm for showing 2 objects are equal.

The trichotomy law (a

b, a = b) doesn’t hold in general.All functions are continuous. Piecewise functions are nonsensical.

In other words, the continuum is unbreakable into points. Functions transform the continuum onto the continuum.

With this as our basis, Smooth Infinitesimal Analysis introduces an object called epsilon.

There is no algorithm to tell whether or not epsilon != 0 or epsilon = 0. This avoids the first problem entirely.

epsilon^2 = 0 though which gives us a way to get rid of them from our formulas.

So I view infinitesimals as the glue that makes the continuum unbreakable and there is no algorithm to decide if the expression “epsilon = 0 or epsilon != 0” is true (see why we have to reject the law of excluded middle to make this work?).

@asdf: Wow, really interesting stuff! I like that insight of infinitesimals as the “glue” that makes the continuum unbreakable. Great analogy.

Hey, Kalid, I’ve just got a quick question to ask.

If you learn calculus via the use of infinitesimals, is it possible to then make the leap over to using limits? While I doubt it would happen, I’d like to be an amateur mathematician in the vein of Fermat some time and develop proofs (more as a beauty thing, to be honest), but writing in a fashion that is contrary to the norm is rather like handing out Spanish pamphlets in an English neighborhood- they might understand, but they won’t like it.

So, yeah, can you jump from infinitesimals over to limits? From what I can tell, limits are mainly used because they’re easily to rigorously define an to keep the constructivist camp from yelling at you.

@Dave: Great question. I can’t say I’m completely comfortable with limits, but I think you can jump back and forth (the Keisler Calculus book has some examples like this I believe). I think the bigger goal is to figure out what is being said, i.e. “What does this equation equal, within some level of tolerance?”. Limits and infinitesimals are two ways to define that tolerance threshold, but infinitesimals are “easier” in that it’s built in (and you don’t need to explicitly define epsilon, delta, etc.).

Hello, i have silly question. How intuitively explain that cos x/x is undefind?

There is graf> http://www.wolframalpha.com/input/?i=Plot%5B{cos%5Bx%5D%2C+x}%2C+{x%2C+-1.0%2C+1.0}%5D

thx

@werterber: Not a silly question at all! In my head, it’s saying “what’s the ratio of width [cos(x)] to distance traveled (x)”.

As our distance traveled goes to 0 (we aren’t moving from the starting point), cos(x) tends towards 1 — we’re pretty much at the same width. So it becomes “1 / 0” in my head.

@Dave

post 17

Regarding your question, “If you learn calculus via the use of infinitesimals, is it possible to then make the leap over to using limits?”, I suppose it is possible for I have (in a way) done it, though I never knew I was learning infinitesimals.

I must admit that prior to reading this post I have never even heard about the defined mathematic concept of ‘infinitesimal’. I also never took a formal Calculus course. I originally learned Calc in my AP physics class in high school. Our teacher (one of the few who truly loved the craft of teaching and had a passion for what she did) had both the constraint of putting her Physics class on hold to teach Calc to those who have never seen it, and also the freedom that brevity provided; she was free to teach the idea of calculus without the strict procedural rigor that a formal class drags its pupil through. We learned the basic idea of the integral before the derivative, heresy in Calc101. Here it is 21 years later and I can still hear her voice saying ‘Taking the integral just means add up a whole bunch of things, and ‘taking a differential element of’ just means cut the thing into really teenie weenie chunks.” We learned the idea of a derivative as slope of a function without being given 2 points, just one point and an interval to the next. After seeing what happened as the interval got smaller we finally visualized ‘slope at a point’. Only afterward were we shown the ‘official’ formula with a limit in it. I saw it as a perfectly nice piece of legal-eez that made the rest of the world happy for me to have learned the ‘right way’, and I was enormously grateful our teacher taught us the intuitive way.

Fascinating article Kalid!

This is something new for me. After reading this post I started some research on infinitesimals, and quickly re-affirmed how valuable your common sense approach is by comparison to an army of equations, lemmas, and theorems.

My great ‘a-ha’ moment was your description of infinitesimals as another dimension, similar to the way imaginary numbers are another dimension to reals. In a strange way, that may not be obvious at first, it reminded me of a conundrum I faced learning the history of physics. It seems that every time we define what an ‘element’ is -the smallest indivisible component of a thing- some clever lad comes along later and figures out way to break that ‘element’ into something smaller. This means, of course, that the old thing never was a true element, we just thought it was. But then what about this new ‘element’, how can we know it is the smallest thing?

A revelation came when I realized that in order to be an ‘element’ we don’t really need it to be true that you can’t break it apart, it just means that if you do break it down further then it is no longer the same stuff. Thus the element is really just the smallest possible piece of a thing WHICH can still be the same thing. E.g. an ‘element’ of water (H2O) can be broken down, but it is no longer water, just hydrogen and oxygen atoms. An atom can be broken down into protons, neutrons, and electrons, but it is no longer the same stuff as the original atom. A little chunk of matter (a superstring exhibiting one class of vibration in 10 dimensions) can indeed be broken down, it is just no longer matter. It is also not exactly energy, but when the ‘stuff’ comes back together in a different pattern (the superstring having the same vibration just in a different dimension) it appears to us as a little chunk of energy.

It seems natural to me to take a cue from the physical world to comprehend numbers. When we look at an element and it appears we’ve ‘hit the limit’ in terms of breaking it up, but we can go further it just means we have to view it in a different dimension. Why then could we not do the same with numbers? Here’s a rational number you can only break it apart but to a certain extent and no smaller. I know you may object and say ‘take that number and divide by 2, it is smaller and still rational’. But take notice of the irrationals, like sqrt(2). It does exist, sitting there staring us in the face. It is in between rationals. So how does there exist any space between rationals? How can the rationals be broken down finer than it is possible to break them down? Imagine thinking you understand that atoms are elementary particles, then this clown Rutherford comes along and experimentally identifies this object (nucleus) in the middle of an atom.

I say the best way forward is to take as true those things that must be true and re-evaluate our preconceived notions that have pigeon-holed us into an apparent paradox. It is difficult and un-nerving. You can be guaranteed you’ll get it wrong a few times before you make some progress, but some progress is far better than the certainty of smaller minds.

Thanks Eric, that’s a really thought-provoking comment.

I think the element analogy is apt, we’re able to function at a certain level (water molecules) and while we *can* go to a deeper level (individual subatomic particles) those details presumably don’t change the measurements we’re making at the macro level. In the same way, infinitesimals can bounce around in funny ways but not effect the numbers one level up. (I.e., when we switch domains, the infinitesimal part goes to 0.)

Trying to “fill in” the number line with rationals is another great example. We have a smooth continuum on the number line, but the rationals are so sparse they’ll never complete it! There must be another way to get to those in-between numbers, and it isn’t by dividing the ones we have into smaller bits.

Kalid

Leibnitz and Newton originated calculus in the 17th century, long before imaginary numbers were around. Can’t we just say that limits are paradoxical but they work and leave it at that?

i loved your paper and the method of teaching limits…i last learnt or rather didnt learn at all about limits in 1979…now my son is in the 11th standard and got switched off by his lecturer on the first day and first lecture on imaginary numbers….going through your paper makes me feel confident of holding a conversation or better still tutoring my child as he stretches his maths muscles….I appreciate the trouble you have taken to make maths and calculus enjoyable…

Very interesting but the discussion seems to assume I know what limits are. When I look at my old calculus book the limits and or x dont always equate to zero or 1.

What I thought the discussion would provide is an explanation as to why we need to know what a limit is? ie why do most calculus books have a set of questions asking us to determine the limits of functions? What purpose does it serve? Why would I need to know the limit?

While I still have those questions in my head I nevertheless enjoyed the discussion about modelling functions by breaking em down into very small bits. I’ve always got that from a practical viewpoint but I didn’t think of the total as a collected bunch of imperfect estimates.

As always, I read your article with great interest. I can see when you talk about the infinitesimals approach, you are using the non-standard calculus (with hyperreal number system) invented by Abraham Robinson. Personally, I prefer this way too although it doesn’t fit into the mainstream.

Please explain the answer in detail..

Why we study limits, functions and differentiability in Computer Science??

OR

What is an importance of limits, functions and differentiability in Computer Science??

any one answer please

It depends.

If the goal is to be a computer programmer, the answer is definite No. High school or community college education, common sense logic and a little bit smartness are good enough.

If the goal is to be a CS researcher, the answer is maybe No. CS mainly deals with discrete math, so the topics you mentioned are not necessary relevant.

If the goal is to use computer as a tool to solve engineering problems, the answer is maybe Yes. The majority problems in real world are “well-behaved” continuous functions, so the idea of limits and differentiability are very easy to grasp even if you don’t fully understand their rigorous definitions.

If the goal is to become a well-educated person, the answer is definite Yes. You learn something for the sake of learning, and do not intend to profit from it. Studying Euclidean geometry on tangent of a circle, Archimedes’ approximation of pi using the method of exhaustion, and the calculus invented by Newton and Leibniz and refined by Cauchy and others, allows you to appreciate the inner beauty of math and the intelligence of many great minds.

Although this article is very clear and understandable; it’s really discussing a very deep philosophical issues..

Great article from a passionate and deep thinker writer

I did a bit of the maths by myself before searching if other people see calculus the way i see it and i’m intrigued to see that I used similar analogies to the ones you used…i referred to the infinitesimally small triangles and small rectangles that we use as “trojan triangles” or “trojan rectangles” because they allow us to sneak into the quantum/small world/ or other dimension.