Vector Calculus: Understanding the Cross Product

Taking two vectors, we can write every combination of components in a grid:

This completed grid is the outer product, which can be separated into the:

Dot product, the interactions between similar dimensions (x*x, y*y, z*z)
Cross product, the interactions between different dimensions (x*y,y*z, z*x, etc.)

The dot product ($\vec{a} \cdot \vec{b}$) measures similarity because it only accumulates interactions in matching dimensions. It’s a simple calculation with 3 components.

The cross product (written $\vec{a} \times \vec{b}$) has to measure a half-dozen “cross interactions”. The calculation looks complex but the concept is simple: accumulate 6 individual differences for the total difference.

Instead of thinking “When do I need the cross product?” think “When do I need interactions between different dimensions?”.

Area, for example, is formed by vectors pointing in different directions (the more orthogonal, the better). Indeed, the cross product measures the area spanned by two 3d vectors (source):

(The “cross product” assumes 3d vectors, but the concept extends to higher dimensions.)

Did the key intuition click? Let’s hop into the details.

Defining the Cross Product

The dot product represents the similarity between vectors as a single number:

$\displaystyle{\text{dot product} = (a_x, a_y, a_z) \cdot (b_x, b_y, b_z) = a_x b_x + a_y b_y + a_z b_z = \|\vec{a}\| \|\vec{b}\| \cos(\theta)}$

For example, we can say that North and East are 0% similar since $(0, 1) \cdot (1, 0) = 0$. Or that North and Northeast are 70% similar ($\cos(45) = .707$, remember that trig functions are percentages.) The similarity shows the amount of one vector that “shows up” in the other.

Should the cross product, the difference between vectors, be a single number too?

Let’s try. Sine is the percentage difference, so we could write:

$\displaystyle{\text{cross product candidate} = \text{amount of difference} = \|\vec{a}\| \|\vec{b}\| \sin(\theta)}$

Unfortunately, we’re missing some details. Let’s say we’re looking down the x-axis: both y and z point 100% away from us. A number like “100%” tells us there’s a big difference, but we don’t know what it is! We need extra information to tell us “the difference between $\vec{x}$ and $\vec{y}$ is this” and “the difference between $\vec{x}$ and $\vec{z}$ is that“.

So, let’s express the cross product as a vector:

The size of the cross product is the numeric “amount of difference” (with $\sin(\theta)$ as the percentage). By itself, this doesn’t distinguish $\vec{x} \times \vec{y}$ from $\vec{x} \times \vec{z}$.
The direction of the cross product is based on both inputs: it’s the direction orthogonal to both (i.e., favoring neither).

Now $\vec{x} \times \vec{y}$ and $\vec{x} \times \vec{z}$ have different results, each with a magnitude indicating they are “100%” different from $\vec{x}$.

(Should the dot product be a vector result too? Well, we’re tracking the similarity between $\vec{a}$ and $\vec{b}$. The similarity measures the overlap between the original vector directions, which we already have.)

Geometric Interpretation

Two vectors determine a plane, and the cross product points in a direction different from both (source):

Here’s the problem: there’s two perpendicular directions. By convention, we assume a “right-handed system” (source):

If you hold your first two fingers like the diagram shows, your thumb will point in the direction of the cross product. I make sure the orientation is correct by sweeping my first finger from $\vec{a}$ to $\vec{b}$. With the direction figured out, the magnitude of the cross product is $|a| |b| \sin(\theta)$, which is proportional to the magnitude of each vector and the “difference percentage” (sine).

The Cross Product For Orthogonal Vectors

To remember the right hand rule, write the xyz order twice: xyzxyz. Next, find the pattern you’re looking for:

xy => z (x cross y is z)
yz => x (y cross z is x; we looped around: y to z to x)
zx => y

Now, xy and yx have opposite signs because they are forward and backward in our xyzxyz setup.

So, without a formula, you should be able to calculate:

$\displaystyle{\vec{x} \times \vec{y} = (1, 0, 0) \times (0, 1, 0) = (0, 0, 1) = \vec{z}}$

Again, this is because x cross y is positive z in a right-handed coordinate system. I used unit vectors, but we could scale the terms:

$\displaystyle{(3, 0, 0) \times (0, 4, 0) = (0, 0, 12)}$

Calculating The Cross Product

A single vector can be decomposed into its 3 orthogonal parts:

$\displaystyle{ \vec{a} = (a_x, a_y, a_z) = (a_x, 0, 0) + (0, a_y, 0) + (0, 0, a_z)}$

$\displaystyle{ \vec{b} = (b_x, b_y, b_z) = (b_x, 0, 0) + (0, b_y, 0) + (0, 0, b_z)}$

When the vectors are crossed, each pair of orthogonal components (like $a_x \times b_y$) casts a vote for where the orthogonal vector should point. 6 components, 6 votes, and their total is the cross product. (Similar to the gradient, where each axis casts a vote for the direction of greatest increase.)

xy => z and yx => -z (assume $\vec{a}$ is first, so xy means $a_x b_y$)
yz => x and zy => -x
zx => y and xz => -y

xy and yx fight it out in the z direction. If those terms are equal, such as in $(2, 1, 0) \times (2, 1, 1)$, there is no cross product component in the z direction (2 – 2 = 0).

The final combination is:

$\displaystyle{(a_x, a_y, a_z) \times (b_x, b_y, b_z) = (a_y b_z - a_z b_y, a_z b_x - a_x b_z, a_x b_y - a_y b_x) = \|a\| \|b\| \sin(\theta) \vec{n}}$

where $\vec{n}$ is the unit vector normal to $\vec{a}$ and $\vec{b}$.

Don’t let this scare you:

There’s 6 terms, 3 positive and 3 negative
Two dimensions vote on the third (so the z term must only have y and x components)
The positive/negative order is based on the xyzxyz pattern

If you like, there is an algebraic proof, that the formula is both orthogonal and of size $|a| |b| \sin(\theta)$, but I like the “proportional voting” intuition.

Example Time

Again, we should do simple cross products in our head:

$\displaystyle{(1, 0, 0) \times (0, 1, 0) = (0, 0, 1)}$

Why? We crossed the x and y axes, giving us z (or $\vec{i} \times \vec{j} = \vec{k}$, using those unit vectors). Crossing the other way gives $-\vec{k}$.

Here’s how I walk through more complex examples:

$\displaystyle{(1, 2, 3) \times (4, 5, 6) = ?}$

Let’s do the last term, the z-component. That’s (1)(5) minus (4)(2), or 5 – 8 = -3. I did z first because it uses x and y, the first two terms. Try seeing (1)(5) as “forward” as you scan from the first vector to the second, and (4)(2) as backwards as you move from the second vector to the first.
Now the y component: (3)(4) – (6)(1) = 12 – 6 = 6
Now the x component: (2)(6) – (5)(3) = 12 – 15 = -3

So, the total is $(-3, 6, -3)$ which we can verify with Wolfram Alpha.

In short:

The cross product tracks all the “cross interactions” between dimensions
There are 6 interactions (2 in each dimension), with signs based on the xyzxyz order

Appendix

Connection with the Determinant

You can calculate the cross product using the determinant of this matrix:

$\mathbf{u\times v}=\begin{vmatrix} \mathbf{i} & \mathbf{j} & \mathbf{k}\\ u_1 & u_2 & u_3\\ v_1 & v_2 & v_3\\ \end{vmatrix}$

$\mathbf{u\times v}= \begin{vmatrix} u_2 & u_3\\ v_2 & v_3 \end{vmatrix}\mathbf{i} -\begin{vmatrix} u_1 & u_3\\ v_1 & v_3 \end{vmatrix}\mathbf{j} +\begin{vmatrix} u_1 & u_2\\ v_1 & v_2 \end{vmatrix}\mathbf{k}$

There’s a neat connection here, as the determinant (“signed area/volume”) tracks the contributions from orthogonal components.

There are theoretical reasons why the cross product (as an orthogonal vector) is only available in 0, 1, 3 or 7 dimensions. However, the cross product as a single number is essentially the determinant (a signed area, volume, or hypervolume as a scalar).

Connection with Curl

Curl measures the twisting force a vector field applies to a point, and is measured with a vector perpendicular to the surface. Whenever you hear “perpendicular vector” start thinking “cross product”.

We take the “determinant” of this matrix:

$\begin{vmatrix} \vec{i} & \vec{j} & \vec{k} \\ \\ {\frac{\partial}{\partial x}} & {\frac{\partial}{\partial y}} & {\frac{\partial}{\partial z}} \\ \\ F_x & F_y & F_z \end{vmatrix}$

$\displaystyle{\nabla \times \vec{F} = \left(\frac{\partial F_z}{\partial y} - \frac{\partial F_y}{\partial z}\right) \vec{i} + \left(\frac{\partial F_x}{\partial z} - \frac{\partial F_z}{\partial x}\right) \vec{j} + \left(\frac{\partial F_y}{\partial x} - \frac{\partial F_x}{\partial y}\right) \vec{k}}$

Instead of multiplication, the interaction is taking a partial derivative. As before, the $\vec{i}$ component of curl is based on the vectors and derivatives in the $\vec{j}$ and $\vec{k}$ directions.

Relation to the Pythagorean Theorem

The cross and dot product are like the orthogonal sides of a triangle:

$\displaystyle{a^2 + b^2 = c^2 }$

For unit vectors, where $|a| = |b| = 1 $, we have:

$\displaystyle{ \|\text{dot product}\|^2 + \|\text{cross product}\|^2 = \cos^2 + \sin^2 = 1}$

I cheated a bit in the grid diagram, as we have to track the squared magnitudes (as done in the Pythagorean Theorem).

Advanced Math

The cross product & friends get extended in Clifford Algebra and Geometric Algebra. I’m still learning these.

Cross Products of Cross Products

Sometimes you’ll have a scenario like:

$\displaystyle{\vec{a} \times \vec{b} \times \vec{c} = ? }$

First, the cross product isn’t associative: order matters.

Next, remember what the cross product is doing: finding orthogonal vectors. If any two components are parallel ($\vec{a}$ parallel to $\vec{b}$) then there are no dimensions pushing on each other, and the cross product is zero (which carries through to $0 \times \vec{c}$).

But it’s ok for $\vec{a}$ and $\vec{c}$ to be parallel, since they are never directly involved in a cross product, for example:

$\displaystyle{\vec{i} \times \vec{j} \times \vec{i} = \vec{k} \times \vec{i} = \vec{j} }$

Whoa! How’d we get back to $\vec{j}$? We asked for a direction perpendicular to both $\vec{i}$ and $\vec{j}$, and made that direction perpendicular to $\vec{i}$ again. Being “doubly perpendicular” means you’re back on the original axis.

Dot Product of Cross Products

Now if we take

$\displaystyle{\vec{a} \times \vec{b} \cdot \vec{c} = ? }$

what happens? We’re forced to do $\vec{a} \times \vec{b}$ first, because $\vec{b} \cdot \vec{c}$ returns a scalar (single number) which can’t be used in a cross product.

If $\vec{a}$ and $\vec{c}$ are parallel, what happens? Well, $\vec{a} \times \vec{b}$ is perpendicular to $\vec{a}$, which means it’s perpendicular to $\vec{c}$, so the dot product with $\vec{c}$ will be zero.

I never really memorized these rules, I have to think through the interactions.

Other Coordinate Systems

The Unity game engine is left-handed, OpenGL (and most math/physics tools) are right-handed. Why?

In a computer game, x goes horizontal, y goes vertical, and z goes “into the screen”. This results in a left-handed system. (Try it: using your right hand, you can see x cross y should point out of the screen).

Applications of the Cross Product

Find the direction perpendicular to two given vectors.
Find the signed area spanned by two vectors.
Determine if two vectors are orthogonal (checking for a dot product of 0 is likely faster though).
“Multiply” two vectors when only perpendicular cross-terms make a contribution (such as finding torque).
With the quaternions (4d complex numbers), the cross product performs the work of rotating one vector around another (another article in the works!).

Happy math.