Definition

The gradient (often abbreviated to grad) is simply a fancy word for derivative (which is a fancy enough word already). The gradient usually applies to vector functions, or functions of more than one variable. Thus, a line has a gradient, also known as its slope. Using gradient for one-variable functions is just unnecessarily confusing. The word gradient is also used to indicate a gradual change (you've probably seen it in Photoshop or the like) but here we are sticking to the math definition. Of course, you shall see the relation between the two terms.

Realizing that the gradient is just a derivative for multivariable functions, let's derive some properties.

First, the gradient of a function is a vector. A derivative gives us the rate of change of one variable. In order to specify multiple variables, we need a vector, with one component for each variable in the function. Thus, a 3-variable function has a 3-component gradient.

Like the regular derivative, the gradient points in the direction of greatest increase. Do you want to see why? [**ADD IN PAGE FOR DERIVATION***]However, with multiple directions, this direction is no longer simply "forward" or "back". If we have 2 variables, then our 2-component gradient can specify any direction on a plane. Likewise, with 3 variables, we can specify any direction in space.

A Twisted Example

Suppose we have a magical oven with coordinates written on it and special display screen. We type in any 3 coordinates, and the display gives us the gradient of the temperature at that point (it thus has 3 components). Also, we have a cheap oven: it is unstable, and the temperature varies from point to point inside the oven. However, we have a little robotic arm that can move anything inside the oven to any other point. Understand the scenario?

Now suppose we are in need of psychiatric help and put the Pillsbury Dough Boy inside the oven because we think he would taste good.  We place him in some random point in the oven and we want to cook him as fast as possible. The gradient can help!  

The gradient at any location points in the direction of greatest increase of a function; in this case our function measures temperature.  So, the gradient tells in which direction to move the doughboy to get him to cook even faster. Remember that the gradient does not give us the coordinates of where to go; it gives us the direction to move.

Thus, we would start out at any random point and check the gradient. We would follow that direction for a bit and get to a new point, which would have its own gradient (the new best direction), and so on.  Eventually we would get to the hottest part of the oven and we would stay there. Pretty soon we'd have our cookies and we'd be good to go.

Before you eat those cookies, let's make some observations. First, when we reach the top (the hottest temperature in the oven), what is the gradient? It will be zero. Why? There is no direction of greatest increase; every direction leads to a decrease in temperature. Thus, a zero gradient tells you to stay put; you are at the max.

Wait... what if there are two maximums? Ah, now we are venturing into the not-so-pretty underbelly of the gradient. Finding the maximum in regular (1 variable) functions involves finding all the places where the derivative is zero, and testing those points. The reason is that the derivative will point to local maximums and minimums. The absolute max/min must be tested.

The same applies for the gradient, which is simply a generalization of the derivative. You must find where the gradient is zero, and test those points to see which one is the absolute max (both will be local maximums). You can imagine two hills: the top of each has a zero gradient. Now that we have cleared that up, enjoy your cookie.

Mathematics

We know the definition of the gradient. The symbol used is an upside-down delta (a triangle). I can't seem to find the symbol in Maple, so we'll have to go without it for now.


Notice how the x component is the partial derivative with respect to x, and the y component is the partial with respect to y. For a one variable function, there are no y's at all, so the gradient is simply the derivative. Check out my links section for some vector calculus sites for some mathematical examples.

Obvious applications of the gradient are finding the max/min of multivariable functions. Another less obvious but related application is finding the maximum of a constrained function (a function whose x and y values have a limited domain, i.e., they are constrained to lie on a certain circle). This calls for my boy Lagrange, but all in due time, all in due time...