Here’s my take on derivatives:

• We have a system to analyze, our function f
• The derivative f’ (df/dx) is the moment-by-moment behavior
• It turns out f is part of a bigger system (h = f + g)
• Using the behavior of the parts, can we figure out the behavior of the whole?

Yes. Every part has a “point of view” about how much change it added. Combine every point of view to get the overall behavior. Each derivative rule is an example of merging various points of view.

And why don’t we analyze the entire system at once? For the same reason you don’t eat a hamburger in one bite: small parts are easier to wrap your head around.

Instead of memorizing separate rules, let’s see how they fit together:

The goal is to really grok the notion of “combining perspectives”. This installment covers addition, multiplication, powers and the chain rule. Onward!

Functions: Anything, Anything But Graphs

The default calculus explanation writes “f(x) = x^2″ and shoves a graph in your face. Does this really help our intuition?

Not for me. Graphs squash input and output into a single curve, and hide the machinery that turns one into the other. But the derivative rules are about the machinery, so let’s see it!

I visualize a function as the process “input(x) => f => output(y)”.

It’s not just me. Check out this incredible, mechanical targetting computer (beginning of youtube series).

The machine computes functions like addition and multiplication with gears — you can see the mechanics unfolding!

Think of function f as a machine with an input lever “x” and an output lever “y”. As we adjust x, f sets the height for y. Another analogy: x is the input signal, f receives it, does some magic, and spits out signal y. Use whatever analogy helps it click.

Wiggle Wiggle Wiggle

The derivative is the “moment-by-moment” behavior of the function. What does that mean? (And don’t mindlessly mumble “The derivative is the slope”. See any graphs around these parts, fella?)

The derivative is how much we wiggle. The lever is at x, we “wiggle” it, and see how y changes. “Oh, we moved the input lever 1mm, and the output moved 5mm. Interesting.”

The result can be written “output wiggle per input wiggle” or “dy/dx” (5mm / 1mm = 5, in our case). This is usually a formula, not a static value, because it can depend on your current input setting.

For example, when f(x) = x^2, the derivative is 2x. Yep, you’ve memorized that. What does it mean?

If our input lever is at x = 10 and we wiggle it slightly (moving it by dx=0.1 to 10.1), the output should change by dy. How much, exactly?

• We know f’(x) = dy/dx = 2 * x
• At x = 10 the “output wiggle per input wiggle” is = 2 * 10 = 20. The output moves 20 units for every unit of input movement.
• If dx = 0.1, then dy = 20 * dx = 20 * .1 = 2

And indeed, the difference between 10^2 and (10.1)^2 is about 2. The derivative estimated how far the output lever would move (a perfect, infinitely small wiggle would move 2 units; we moved 2.01).

The key to understanding the derivative rules:

• Set up your system
• Wiggle each part of the system separately, see how far the output moves
• Combine the results

The total wiggle is the sum of wiggles from each part.

Addition and Subtraction

Time for our first system:

What happens when the input (x) changes?

In my head, I think “Function h takes a single input. It feeds the same input to f and g and adds the output levers. f and g wiggle independently, and don’t even know about each other!”

Function f knows it will contribute some wiggle (df), g knows it will contribute some wiggle (dg), and we, the prowling overseers that we are, know their individual moment-by-moment behaviors are added:

Again, let’s describe each “point of view”:

• The overall system has behavior dh
• From f’s perspective, it contributes df to the whole [it doesn't know about g]
• From g’s perspective, it contributes dg to the whole [it doesn't know about f]

Every change to a system is due to some part changing (f and g). If we add the contributions from each possible variable, we’ve described the entire system.

df vs df/dx

Sometimes we use df, other times df/dx — what gives? (This confused me for a while)

• df is a general notion of “however much f changed”
• df/dx is a specific notion of “however much f changed, in terms of how much x changed”

The generic “df” helps us see the overall behavior.

An analogy: Imagine you’re driving cross-country and want to measure the fuel efficiency of your car. You’d measure the distance traveled, check your tank to see how much gas you used, and finally do the division to compute “miles per gallon”. You measured distance and gasoline separately — you didn’t jump into the gas tank to get the rate on the go!

In calculus, sometimes we want to think about the actual change, not the ratio. Working at the “df” level gives us room to think about how the function wiggles overall. We can eventually scale it down in terms of a specific input.

And we’ll do that now. The addition rule above can be written, on a “per dx” basis, as:

Multiplication (Product Rule)

Next puzzle: suppose our system multiplies parts “f” and g”. How does it behave?

Hrm, tricky — the parts are interacting more closely. But the strategy is the same: see how each part contributes from its own point of view, and combine them:

• total change in h = f’s contribution (from f’s point of view) + g’s contribution (from g’s point of view)

Check out this diagram:

What’s going on?

• We have our system: f and g are multiplied, giving h (the area of the rectangle)
• Input “x” changes by dx off in the distance. f changes by some amount df (think absolute change, not the rate!). Similarly, g changes by its own amount dg. Because f and g changed, the area of the rectangle changes too.
• What’s the area change from f’s point of view? Well, f knows he changed by df, but has no idea what happened to g. From f’s perspective, he’s the only one who moved and will add a slice of area = df * g
• Similarly, g doesn’t know how f changed, but knows he’ll add as slice of area “dg * f”

The overall change in the system (dh) is the two slices of area:

Now, like our miles per gallon example, we “divide by dx” to write this in terms of how much x changed:

(Aside: Divide by dx? Engineers will nod, mathematicians will frown. Technically, df/dx is not a fraction: it’s the entire operation of taking the derivative (with the limit and all that). But infinitesimal-wise, intuition-wise, we are “scaling by dx”. I’m a smiler.)

The key to the product rule: add two “slivers of area”, one from each point of view.

Gotcha: But isn’t there some effect from both f and g changing simultaneously (df * dg)?

Yep. However, this area is an infinitesimal * infinitesimal (a “2nd-order infinitesimal”) and invisible at the current level. It’s a tricky concept, but (df * dg) / dx vanishes compared to normal derivatives like df/dx. We vary f and g indepdendently and combine the results, and ignore results from them moving together.

The Chain Rule: It’s Not So Bad

Let’s say g depends on f, which depends on x:

The chain rule lets us “zoom into” a function and see how an initial change (x) can effect the final result down the line (g).

Interpretation 1: Convert the rates

A common interpretation is to multiply the rates:

x wiggles f. This creates a rate of change of df/dx, which wiggles g by dg/df. The entire wiggle is then:

This is similar to the “factor-label” method in chemistry class:

If your “miles per second” rate changes, multiply by the conversion factor to get the new “miles per hour”. The second doesn’t know about the hour directly — it goes through the second => minute conversion.

Similarly, g doesn’t know about x directly, only f. Function g knows it should scale its input by dg/df to get the output. The initial rate (df/dx) gets modified as it moves up the chain.

Interpretation 2: Convert the wiggle

I prefer to see the chain rule on the “per-wiggle” basis:

• x wiggles by dx, so
• f wiggles by df, so
• g wiggles by dg

Cool. But how are they actually related? Oh yeah, the derivative! (It’s the output wiggle per input wiggle):

Remember, the derivative of f (df/dx) is how much to scale the initial wiggle. And the same happens to g:

It will scale whatever wiggle comes along its input lever (f) by dg/df. If we write the df wiggle in terms of dx:

We have another version of the chain rule: dx starts the chain, which results in some final result dg. If we want the final wiggle in terms of dx, divide both sides by dx:

The chain rule isn’t just factor-label unit cancellation — it’s the propagation of a wiggle, which gets adjusted at each step.

The chain rule works for several variables (a depends on b depends on c), just propagate the wiggle as you.

Try to imagine “zooming into” different variable’s point of view. Starting from dx and looking up, you see the entire chain of transformations needed before the impulse reaches g.

Chain Rule: Example Time

Let’s say we put a “squaring machine” in front of a “cubing machine”:

input(x) => f:x^2 => g:f^3 => output(y)

f:x^2 means f squares its input. g:f^3 means g cubes its input, the value of f. For example:

input(2) => f(2) => g(4) => output:64

Start with 2, f squares it (2^2 = 4), and g cubes this (4^3 = 64). It’s a 6th power machine:

And what’s the derivative?

• f changes its input wiggle by df/dx = 2x
• g changes its input wiggle by dg/df = 3f^2

The final change is:

Chain Rule: Gotchas

Functions treat their inputs like a blob

In the example, g’s derivative (“x^3 = 3x^2″) doesn’t refer to the original “x”, just whatever the input was (foo^3 = 3*foo^2). The input was f, and it treats f as a single value. Later on, we scurry in and rewrite f in terms of x. But g has no involvement with that — it doesn’t care that f can be rewritten in terms of smaller pieces.

In many examples, the variable “x” is the “end of the line”.

Questions ask for df/dx, i.e. “Give me changes from x’s point of view”. Now, x could depend on something deeper variable, but that’s not being asked for. It’s like saying “I want miles per hour. I don’t care about miles per minute or miles per second. Just give me miles per hour”. df/dx means “stop looking at inputs once you get to x”.

How come we multiply derivatives with the chain rule, but add them for the others?

The regular rules are about combining points of view to get an overall picture. What change does f see? What change does g see? Add them up for the total.

The chain rule is about going deeper into a single part (like f) and seeing if it’s controlled by another variable. It’s like looking inside a clock and saying “Hey, the minute hand is controlled by the second hand!”. We’re staying inside the same part.

Sure, eventually this “per-second” perspective of f could be added to some perspective from g. Great. But the chain rule is about diving deeper into “f’s” root causes.

Power Rule: Oft Memorized, Seldom Understood

What’s the derivative of x^4? 4x^3? Great. You brought down the exponent and subtracted one. Now explain why!

Hrm. There’s a few approaches, but here’s my new favorite: x^4 is really x * x * x * x. It’s the multiplication of 4 “independent” variables. Each x doesn’t know about the others, it might as well be x * u * v * w.

Now think about the first x’s point of view:

• It changes from x to x + dx
• The change in the overall function is [(x + dx) - x][u * v * w] = dx[u * v * w]
• The change on a “per dx” basis is [u * v * w]

Similarly,

• From u’s point of view, it changes by du. It contributes (du/dx)*[x * v * w] on a “per dx” basis
• v contributes (dv/dx) * [x * u * w]
• w contributes (dw/dx) * [x * u * v]

The curtain is unveiled: x, u, v, and w are the same! The “point of view” conversion factor is 1 (du/dx = dv/dx = dw/dx = dx/dx = 1), and the total change is

In a sentence: the derivative of x^4 is 4x^3 because x^4 has four identical “points of view” which are being combined. Booyeah!

Take A Breather

I hope you’re seeing the derivative in a new light: we have a system of parts, we wiggle our input and see how the whole thing moves. It’s about combining perspectives: what does each part add to the whole?

In the follow-up article, we’ll look at even more powerful rules (exponents, quotients, and friends). Happy math.

Psst! The derivative is the heart of calculus, buried inside this definition:

But what does it mean?

Let's say I gave you a magic newspaper that listed the daily stock market changes for the next few years (+1% Monday, -2% Tuesday...). What could you do?

Well, you'd apply the changes one-by-one, plot out future prices, and buy low / sell high to build your empire. You could even hire away the monkeys who currently throw darts at newspapers.

Others call the derivative "the slope of a function" -- it's so bland! Like the stock list, the derivative is a total, predictive understanding of a system. You can plot the past/present/future, find minimums/maximums, and yes, staff your simian workforce.

Step away from the gnarly equation. Equations exist to convey ideas: understand the idea, not the grammar.

Derivatives create a perfect model of change from an imperfect guess.

This result came over thousands of years of thinking, from Archimedes to Newton. Let's look at the analogies behind it.

We all live in a shiny continuum

Infinity is a constant source of paradoxes ("headaches"):

• A line is made up of points? Sure.
• So there's an infinite number of points on a line? Yep.
• How do you cross a room when there's an infinite number of points to visit? (Gee, thanks Zeno).

And yet, we move. My intuition is to fight infinity with infinity. Sure, there's infinity points between 0 and 1. But I move two infinities of points per second (somehow!) and I cross the gap in half a second.

Distance has infinite points, motion is possible, therefore motion is in terms of "infinities of points per second".

Instead of thinking of differences ("How far to the next point?") we can compare rates ("How fast are you moving through this continuum?").

It's strange, but you can see 10/5 as "I need to travel 10 'infinities' in 5 segments of time. To do this, I travel 2 'infinities' for each unit of time".

Analogy: See division as a rate of motion through a continuum of points

What's after zero?

Another brain-buster: What number comes after zero? .01? .0001?

Hrm. Anything you can name, I can name smaller (I'll just halve your number... nyah!).

Even though we can't calculate the number after zero, it must be there, right? Like demons of yore, it's the "number that cannot be written, lest ye be smitten".

Call the gap to the next number "dx". I don't know exactly how big it is, but it's there!

Analogy: dx is a "jump" to the next number in the continuum.

Measurements depend on the instrument

The derivative predicts change. Ok, how do we measure speed (change in distance)?

Officer: Do you know how fast you were going?

Driver: I have no idea.

Officer: 95 miles per hour.

Driver: But I haven't been driving for an hour!

We clearly don't need a "full hour" to measure your speed. We can take a before-and-after measurement (over 1 second, let's say) and get your instantaneous speed. If you moved 140 feet in one second, you're going ~95mph. Simple, right?

Not exactly. Imagine a video camera pointed at Clark Kent (Superman's alter-ego). The camera records 24 pictures/sec (40ms per photo) and Clark seems still. On a second-by-second basis, he's not moving, and his speed is 0mph.

Wrong again! Between each photo, within that 40ms, Clark changes to Superman, solves crimes, and returns to his chair for a nice photo. We measured 0mph but he's really moving -- he goes too fast for our instruments!

Analogy: Like a camera watching Superman, the speed we measure depends on the instrument!

Running the Treadmill

We're nearing the chewy, slightly tangy center of the derivative. We need before-and-after measurements to detect change, but our measurements could be flawed.

Imagine a shirtless Santa on a treadmill (go on, I'll wait). We're going to measure his heart rate in a stress test: we attach dozens of heavy, cold electrodes and get him jogging.

Santa huffs, he puffs, and his heart rate shoots to 190 beats per minute. That must be his "under stress" heart rate, correct?

Nope. See, the very presence of stern scientists and cold electrodes increased his heart rate! We measured 190bpm, but who knows what we'd see if the electrodes weren't there! Of course, if the electrodes weren't there, we wouldn't have a measurement.

What to do? Well, look at the system:

• measurement = actual amount + measurement effect

Ah. After lots of studies, we may find "Oh, each electrode adds 10bpm to the heartrate". We make the measurement (imperfect guess of 190) and remove the effect of electrodes ("perfect estimate").

Analogy: Remove the "electrode effect" after making your measurement

By the way, the "electrode effect" shows up everywhere. Research studies have the Hawthorne Effect where people change their behavior because they are being studied. Gee, it seems everyone we scrutinize sticks to their diet!

Understanding the derivative

Armed with these insights, we can see how the derivative models change:

Start with some system to study, f(x):

1. Change by the smallest amount possible (dx)
2. Get the before-and-after difference: f(x + dx) - f(x)
3. We don't know exactly how small "dx" is, and we don't care: get the rate of motion through the continuum: [f(x + dx) - f(x)] / dx
4. This rate, however small, has some error (our cameras are too slow!). Predict what happens if the measurement were perfect, if dx wasn't there.

The magic's in the final step: how do we remove the electrodes? We have two approaches:

• Limits: what happens when dx shrinks to nothingness, beyond any error margin?
• Infinitesimals: What if dx is a tiny number, undetectable in our number system?

Both are ways to formalize the notion of "How do we throw away dx when it's not needed?".

My pet peeve: Limits are a modern formalism, they didn't exist in Newton's time. They help make dx disappear "cleanly". But teaching them before the derivative is like showing a steering wheel without a car! It's a tool to help the derivative work, not something to be studied in a vacuum.

An Example: f(x) = x^2

Let's shake loose the cobwebs with an example. How does the function f(x) = x^2 change as we move through the continuum?

Note the difference in the last 2 equations:

• One has the error built in (dx)
• The other has the "true" change, where dx = 0 (our measurements have no effect on the outcome)

Time for real numbers. Here's the values for f(x) = x^2, with intervals of dx = 1:

• 1, 4, 9, 16, 25, 36, 49, 64...

The absolute change between each result is:

• 1, 3, 5, 7, 9, 11, 13, 15...

(Here, the absolute change is the "speed" between each step, where the interval is 1)

Consider the jump from x=2 to x=3 (3^2 - 2^2 = 5). What is "5" made of?

• Measured rate = Actual Rate + Error
• 5 = 2x + dx
• 5 = 2(2) + 1

Sure, we measured a "5 units moved per second" because we went from 4 to 9 in one interval. But our instruments trick us! 4 units of speed came from the real change, and 1 unit was due to shoddy instruments (1.0 is a large jump, no?).

If we restrict ourselves to integers, 5 is the perfect speed measurement from 4 to 9. There's no "error" in assuming dx = 1 because that's the true interval between neighboring points.

But in the real world, measurements every 1.0 seconds is too slow. What if our dx was 0.1? What speed would we measure at x=2?

Well, we examine the change from x=2 to x=2.1:

• 2.1^2 - 2^2 = 0.41

Remember, 0.41 is what we changed in an interval of 0.1. Our speed-per-unit is 0.41 / .1 = 4.1. And again we have:

• Measured rate = Actual Rate + Error
• 4.1 = 2x + dx

Interesting. With dx=0.1, the measured and actual rates are close (4.1 to 4, 2.5% error). When dx=1, the rates are pretty different (5 to 4, 25% error).

Following the pattern, we see that throwing out the electrodes (letting dx=0) reveals the true rate of 2x.

In plain English: We analyzed how f(x) = x^2 changes, found an "imperfect" measurement of 2x + dx, and deduced a "perfect" model of change as 2x.

The derivative as "continuous division"

I see the integral as better multiplication, where you can apply a changing quantity to another.

The derivative is "better division", where you get the speed through the continuum at every instant. Something like 10/5 = 2 says "you have a constant speed of 2 through the continuum".

When your speed changes as you go, you need to describe your speed at each instant. That's the derivative.

If you apply this changing speed to each instant (take the integral of the derivative), you recreate the original behavior, just like applying the daily stock market changes to recreate the full price history. But this is a big topic for another day.

Gotcha: The Many meanings of "Derivative"

You'll see "derivative" in many contexts:

• "The derivative of x^2 is 2x" means "At every point, we are changing by a speed of 2x (twice the current x-position)". (General formula for change)

• "The derivative is 44" means "At our current location, our rate of change is 44." When f(x) = x^2, at x=22 we're changing at 44 (Specific rate of change).

• "The derivative is dx" may refer to the tiny, hypothetical jump to the next position. Technically, dx is the "differential" but the terms get mixed up. Sometimes people will say "derivative of x" and mean dx.

Gotcha: Our models may not be perfect

We found the "perfect" model by making a measurement and improving it. Sometimes, this isn't good enough -- we're predicting what would happen if dx wasn't there, but added dx to get our initial guess!

Some ill-behaved functions defy the prediction: there's a difference between removing dx with the limit and what actually happens at that instant. These are called "discontinuous" functions, which is essentially "cannot be modeled with limits". As you can guess, the derivative doesn't work on them because we can't actually predict their behavior.

Discontinuous functions are rare in practice, and often exist as "Gotcha!" test questions ("Oh, you tried to take the derivative of a discontinuous function, you fail"). Realize the theoretical limitation of derivatives, and then realize their practical use in measuring every natural phenomena. Nearly every function you'll see (sine, cosine, e, polynomials, etc.) is continuous.

Gotcha: Integration doesn't really exist

The relationship between derivatives, integrals and anti-derivatives is nuanced (and I got it wrong originally). Here's a metaphor. Start with a plate, your function to examine:

• Differentiation is breaking the plate into shards. There is a specific procedure: take a difference, find a rate of change, then assume dx isn't there.
• Integration is weighing the shards: your original function was "this" big. There's a procedure, cumulative addition, but it doesn't tell you what the plate looked like.
• Anti-differentiation is figuring out the original shape of the plate from the pile of shards.

There's no algorithm to find the anti-derivative; we have to guess. We make a lookup table with a bunch of known derivatives (original plate => pile of shards) and look at our existing pile to see if it's similar. "Let's find the integral of 10x. Well, it looks like 2x is the derivative of x^2. So... scribble scribble... 10x is the derivative of 5x^2.".

Finding derivatives is mechanics; finding anti-derivatives is an art. Sometimes we get stuck: we take the changes, apply them piece by piece, and mechanically reconstruct a pattern. It might not be the "real" original plate, but is good enough to work with.

Another subtlety: aren't the integral and anti-derivative the same? (That's what I originally thought)

Yes, but this isn't obvious: it's the fundamental theorem of calculus! (It's like saying "Aren't a^2 + b^2 and c^2 the same? Yes, but this isn't obvious: it's the Pythagorean theorem!"). Thanks to Joshua Zucker for helping sort me out.

Reading math

Math is a language, and I want to "read" calculus (not "recite" calculus, i.e. like we can recite medieval German hymns). I need the message behind the definitions.

My biggest aha! was realizing the transient role of dx: it makes a measurement, and is removed to make a perfect model. Limits/infinitesimals are a formalism, we can't get caught up in them. Newton seemed to do ok without them.

Armed with these analogies, other math questions become interesting:

• How do we measure different sizes of infinity? (In some sense they're all "infinite", in other senses the range (0,1) is smaller than (0,2))
• What are the real rules about making "dx go away"? (How do infinitesimals and limits really work?)
• How do we describe numbers without writing them down? "The next number after 0" is the beginnings of analysis (which I want to learn).

The fundamentals are interesting when you see why they exist. Happy math.

Normal multiplication combines growth rates: “3 x 4″ can mean “Take your 3x growth and make it 4x larger (i.e., 12x)”. Complex multiplication lets us combine rotations. Integrals let us do piece-by-piece multiplication.

A vector is “growth in a direction”. The dot product lets us apply the directional growth of one vector to another: the result is how much we went along the original path (positive progress, negative, or zero).

Today let’s build our intuition for how the dot product works.

Getting the Formula Out of the Way

You’ve seen the dot product equation everywhere:

And also the justification: “Well Billy, the Law of Cosines (you remember that, don’t you?) says the following calculations are the same, so they are.” Not good enough — it doesn’t click! Beyond the computation, what does it mean?

The goal is to apply one vector to another. Each computation examines this from a rectangular perspective (x- and y-coordinates) or a polar one (magnitudes and angles). The “blah = foo” equation above really means “Here’s two equivalent ways to ‘directionally multiply’ vectors”.

(Similarly, we can show that Euler’s formula (e^ix = cos(x) + i*sin(x)) is true because the Taylor series is the same on both sides. Accurate but unsatisfying! Instead, see how both sides can describe the same motion.)

Seeing Numbers as vectors

Let’s start simple, and see 3 x 4 as a dot product:

The number 3 is “directional growth” in a single dimension (x-axis, let’s say), and 4 is “directional growth” in that same direction. 3 x 4 = 12 means 12x growth in that single dimension. Ok.

Now, suppose each number refers to a different dimension? Suppose 3 means “triple your bananas” (sigh… or “x-axis”) and 4 means “quadruple your oranges” (y-axis). They’re not the same “type” of number: what happens when we apply growth (take the dot product) in the (bananas, oranges) universe?

• (3,0) is “Triple your bananas, destroy your oranges”
• (0,4) is “Destroy your bananas, quadruple your oranges”

Applying (0,4) to (3,0) means “Destroy your banana growth, quadruple your orange growth”. But (3, 0) had no orange growth to begin with, so the net result is 0 (“Destroy all your fruit, buddy”).

See how we’re “applying” and not adding. With addition, we sort of smush the items together: (3,0) + (0, 4) = (3, 4) [a vector which triples your oranges and quadruples your bananas].

“Application” is different. We’re mutating the original vector according to the rules in the second. And the rules are “Destroy your banana growth rate, and triple your orange growth rate“. And, sadly, this leaves us with nothing.

The final result of this process can be:

• zero: we don’t have any growth in the original direction
• positive number: we have some growth in the original direction
• negative number: we have negative (reverse) growth in the original direction

Understanding the Calculation

“Applying vectors” is still a bit abstract. I think “How much energy/push is one vector giving to the other?”. Here’s how I visualize it:

Rectangular Coordinates: Component-by-component overlap

Like multiplying complex numbers, see how each x- and y-component interacts:

We list out all four combinations (x-x, y-x, x-y, y-y). Since the x- and y-coordinates don’t affect each other (like holding a bucket sideways under a waterfall — nothing falls in), the total energy absorbtion is absorbtion(x) + absorbtion(y):

Polar coordinates: Projection

The word “projection” is so sterile: I prefer “along the path”. How much energy is actually going in our original direction?

Here’s one way to see it:

Take two vectors, a and b. Rotate our coordinates so b is horizontal: it becomes (|b|, 0), and everything is on this new x-axis. What’s the dot product now? (It shouldn’t change just because we tilted our head).

Well, vector a has new coordinates (a1, a2), and we get:

a1 is really “What is the x-coordinate of a, assuming b is the x-axis?”. That is |a|cos(θ), aka the “projection”:

Analogies for the Dot Product

The common interpretation is “geometric projection”, but it’s so sterile. Here’s some analogies that click for me:

Energy Absorbtion

One vector are solar rays, the other is where the solar panel is pointing (yes, yes, the normal vector). Larger numbers mean stronger rays or a larger panel. How much energy is absorbed?

• Energy = Overlap in direction * Strength of rays * Size of panel
• 

If you hold your panel sideways to the sun, no rays hit (cos(θ) = 0).

Photo credit

But… but… solar rays are leaving the sun, and the panel is facing the sun, and the dot product is negative when vectors are opposed! Take a deep breath, and remember the goal is to embrace the analogy (besides, physicists lose track of negative signs all the time).

Mario-Kart Speed Boost

In Mario Kart, there are “boost pads” on the ground that increase your speed (Never played? I’m sorry.)

Photo source

Imagine the red vector is your speed (x and y direction), and the blue vector is the orientation of the boost pad (x and y direction). Larger numbers are more power.

How much boost will you get? For the analogy, imagine the pad multiplies your speed:

• If you come in going 0, you’ll get nothing
• If you cross the pad perpendicularly, you’ll get 0 [just like the banana obliteration, it will give you 0x boost in the perpendicular direction]

But, if we have some overlap, our x-speed will get an x-boost, and our y-speed gets a y-boost:

Neat, eh? Another way to see it: your incoming speed is |a|, and the max boost is |b|. The amount of boost you actually get (for being lined up with it) is cos(θ), for the total |a||b|cos(θ).

Physics Physics Physics

The dot product appears all over physics: some field (electric, gravitational) is pulling on some particle. We’d love to multiply, and we could if everything were lined up. But that’s never the case, so we take the dot product to account for potential differences in direction.

It’s all a useful generalization: Integrals are “multiplication, taking changes into account” and the dot product is “multiplication, taking direction into account”.

And what if your direction is changing? Why, take the integral of the dot product, of course!

Onward and Upward

Don’t settle for “Dot product is the geometric projection, justified by the law of cosines”. Find the analogies that click for you! Happy math.

• 6 figures
• Double digits
• Order of magnitude

You’re describing numbers in terms of their powers of 10 — a logarithm. Ever mention an interest rate or rate of return? It’s the logarithm of your growth.

Surprised that logarithms are so common? Me too. Many attempts at Math In the Real World are attempts to point out logarithms in some arcane formula or pretending we’re geologists fascinated by the Richter Scale. “Scientists care about logs, and you should too. Also, can you imagine a world without zinc?”

No, no, no, no no, no no! (Mama mia!)

Math expresses concepts with notation like “ln” or “log”. Finding “math in the real world” means encountering ideas in life and seeing how they could be written with notation. Don’t look for the literal symbols! When was the last time you wrote a division sign? When was the last time you chopped up some food?

Ok, ok, we get it: what are logarithms about?

Logarithms find the cause for an effect, i.e the input for some output

A common “effect” is seeing something grow, like going from $100 to $150 in 5 years. How did this happen? We’re not sure, but the logarithm finds a possible cause: A continuous return of ln(150/100) / 5 = 8.1% would account for that change. It might not be the actual cause (did all the growth happen in the final year?), but it’s a smooth average we can compare to other changes.

By the way, the notion of “cause and effect” is nuanced. Why is 1000 bigger than 100?

• 100 is 10 which grew by itself for 2 time periods (10 * 10)
• 1000 is 10 which grew by itself for 3 time periods (10 * 10 * 10)

We can think of numbers as outputs (1000 is “1000 outputs”) and inputs (“How many times does 10 need to grow to make those outputs?”). So,

1000 outputs > 100 outputs

because

3 inputs > 2 inputs [i.e., because log(1000) > log(100)]

Why is this useful?

Logarithms put numbers on a human-friendly scale.

Large numbers break our brains. Millions and trillions are “really big” even though a million seconds is 12 days and a trillion seconds is 30,000 years. It’s the difference between an American vacation year and the entirety of human civilization.

The trick to overcoming “huge number blindness” is to write numbers in terms of “inputs” (i.e. their power base 10). This smaller scale (0 to 100) is much easier to grasp:

• power of 0 = 10^0 = 1 (single item)
• power of 1 = 10^1 = 10
• power of 3 = 10^3 = thousand
• power of 6 = 10^6 = million
• power of 9 = 10^9 = billion
• power of 12 = 10^12 = trillion
• power of 23 = 10^23 = number of molecules in a dozen grams of carbon
• power of 80 = 10^80 = number of molecules in the universe

A 0 to 80 scale took us from a single item to the number of things in the universe. Not too shabby.

Logarithms count multiplication as steps

Logarithms describe changes in terms of multiplication: in the examples above, each step is 10x bigger. With the natural log, each step is “e” (2.71828…) times more.

When dealing with a series of multiplications, logarithms help “count” them, just like addition counts for us when effects are added.

Show me the math

Time for the meat: let’s see where logarithms show up!

Six-figure salary or 2-digit expense

We’re describing numbers in terms of their digits, i.e. how many powers of 10 they have (are they in the tens, hundreds, thousands, ten-thousands, etc.). Adding a digit means “multiplying by 10″, i.e.

1 [1 digit] * 10 * 10 * 10 * 10 * 10 [5 more digits] = 10^5 = 100,000

Logarithms count the number of multiplications added on, so starting with 1 (a single digit) we add 5 more digits (10^5) and 100,000 get a 6-figure result. Talking about “6″ instead of “One hundred thousand” is the essence of logarithms. It gives a rough sense of scale without jumping into details.

Bonus question: How would you describe 500,000? Saying “6 figure” is misleading because 6-figures often implies something closer to 100,000. Would “6.5 figure” work?

Not really. In our heads, 6.5 means “halfway” between 6 and 7 figures, but that’s an adder’s mindset. With logarithms a “.5″ means halfway in terms of multiplication, i.e the square root (9^.5 means the square root of 9 — 3 is halfway in terms of multiplication because it’s 1 to 3 and 3 to 9).

Taking log(500,000) we get 5.7, add 1 for the extra digit, and we can say “500,000 is a 6.7 figure number”. Try it out here:

Order of magnitude

We geeks love this phrase. It means roughly “10x difference” but just sounds cooler than “1 digit larger”.

In computers, where everything is counted with bits (1 or 0), each bit has a doubling effect (not 10x). So going from 8 to 16 bits is “8 orders of magnitude” or 2^8 = 256 times larger. (These bit sizes refers to the amount of memory available, not the processor speed). Going from 16 to 32 bits means 16 orders of magnitude, or 2^16 ~ 65,536 times larger.

Isn’t “16 extra bits of memory” better than “65,536 times more memory?”.

Interest Rates

How do we figure out growth rates? A country doesn’t intend to grow at 8.56% per year. You look at the GDP one year and the GDP the next, and take the logarithm to find the implicit growth rate.

My two favorite interpretations of the natural logarithm (ln(x)), i.e. the natural log of 1.5:

• Assuming 100% growth, how long do you need to grow to get to 1.5? (.405, less than half the time period)
• Assuming 1 unit of time, how fast do you need to grow to get to 1.5? (40.5% per year, continuously compounded)

Logarithms are how we figure out how fast we’re growing.

Measurement Scale: Google PageRank

Google gives every page on the web a score (PageRank) which is a rough measure of authority / importance. This is a logarithmic scale, which in my head means “PageRank counts the number of digits in your score”.

So, a site with pagerank 2 (“2 digits”) is 10x more popular than a PageRank 1 site. My site is PageRank 5 and CNN has PageRank 9, so there’s a difference of 4 orders of magnitude (10^4 = 10,000).

Roughly speaking, I get about 7000 visits / day. Using my envelope math, I can guess CNN gets about 7000 * 10,000 = 70 million visits / day. (How’d I do that? In my head, I think 7k * 10k = 70 * k * k = 70 * M). They might have a few times more than that (100M, 200M) but probably not up to 700M.

Google conveys a lot of information with a very rough scale (1-10).

Measurement Scale: Richter, Decibel, etc.

Sigh. We’re at the typical “logarithms in the real world” example: Richter scale and Decibel. The idea is to put events which can vary drastically (earthquakes) on a single 1 – 10 scale. Just like PageRank, each 1-point increase is a 10x improvement in power.

Decibels are similar, though it can be negative. Sounds can go from intensely quiet (pindrop) to extremely loud (airplane) and our brains can process it all. In reality, the sound of an airplane’s engine is millions (billions, trillions) of times more powerful than a pindrop, and it’s inconvenient to have a scale that goes from 1 to a gazillion. Logs keep everything on a reasonable scale.

Logarithmic Graphs

You’ll often see items plotted on a “log scale”. In my head, this means one side is counting “number of digits” or “number of multiplications”, not the value itself. Again, this helps show wildly varying events on a single scale (going from 1 to 10, not 1 to billions).

Moore’s law is a great example: we double the number of transistors every 18 months (image courtesy Wikipedia).

The neat thing about log-scale graphs is exponential changes (processor speed) appear as a straight line. Growing 10x per year means you’re steadily marching up the “digits” scale.

Onward and upward

If a concept is well-known but not well-loved, it means we need to build our intuition. Find the analogies that work, and don’t settle for the slop a textbook will trot out. In my head:

• Logarithms find the root cause for an effect (see growth, find interest rate)
• They help count multiplications or digits, with the bonus of partial counts (500k is a 6.7 digit number)

Happy math.