Multivariable Differential Calculus

A function $f:\mathbb{R}^2\to\mathbb{R}$ assigns a number to each point of the plane: think of $f(x,y)$ as the elevation at point $(x,y)$ of a landscape. The domain $D_f$ is the set of points where $f$ makes sense (e.g. $\ln(xy)$ requires $xy>0$).

Level curves. The level curve of value $c$ is the set of points where $f$ equals exactly $c$:

These are precisely the contour lines of topographic maps (points at the same elevation) or the isotherms of weather maps. Cutting the graph of $f$ with horizontal planes $z=c$ and projecting down gives the level curves.

How to read them. Where level curves are dense, the function varies rapidly (steep slope); where they are sparse, it varies slowly (flat). Closed concentric level curves signal a peak or a basin. In $\mathbb{R}^3$ the analogue is level surfaces $f(x,y,z)=c$ (e.g. equipotential surfaces in electrostatics).

The definition of limit mirrors the one-variable case, but "approaching" now happens in all directions of the plane. The Euclidean norm replaces the absolute value:

The big difference. In $\mathbb{R}$ you approach $x_0$ only from the right or the left. In $\mathbb{R}^2$ you can approach along infinitely many paths: lines, parabolas, spirals. For the limit to exist, it must equal $L$ along every path.

Proving non-existence. Just find two paths giving different limits. Classic example: $f(x,y)=\dfrac{xy}{x^2+y^2}$ at $(0,0)$. Along the $x$-axis ($y=0$): $f=0$. Along $y=x$: $f=\dfrac{x^2}{2x^2}=\dfrac12$. Different limits ⟹ the limit does not exist. (Polar coordinates $x=r\cos\theta$, $y=r\sin\theta$ are often the fastest test: if the result depends on $\theta$, the limit does not exist.)

$f$ is continuous at $\mathbf{x}_0$ if $\lim_{\mathbf{x}\to\mathbf{x}_0}f=f(\mathbf{x}_0)$.

The partial derivative $\partial_x f$ measures how $f$ changes when moving only in the $x$ direction, keeping $y$ frozen:

In practice you differentiate with respect to $x$ treating $y$ as constant. It is the slope of the graph along the slice $y=y_0$.

Warning: partials are not enough. Unlike the 1D case, the existence of partial derivatives does not even guarantee continuity! A stronger notion is needed: differentiability. $f$ is differentiable at $\mathbf{x}_0$ if it is well approximated by a linear function (the differential):

The error $o(|\mathbf{h}|)$ tends to zero faster than $|\mathbf{h}|$: near $\mathbf{x}_0$ the graph is almost a plane.

Practical criterion (total differential theorem). If the partial derivatives exist and are continuous in a neighbourhood ($f\in C^1$), then $f$ is differentiable, hence continuous. This covers almost all "elementary" functions.

The gradient gathers all partial derivatives into a vector:

The directional derivative in the direction of the unit vector $\hat v$ measures the slope of $f$ in that direction, and is the projection of the gradient:

Three fundamental properties of the gradient.

• Direction of steepest ascent. $D_{\hat v}f$ is maximal when $\hat v$ is aligned with $\nabla f$ ($\cos\theta=1$): the gradient points "uphill" the most steeply. The maximal slope is $|\nabla f|$.
• Perpendicularity to level curves. Along a level curve $f$ does not change, so $D_{\hat v}f=0$: the gradient is orthogonal to level curves. (That is why water flows perpendicular to contour lines.)
• Steepest descent in the direction $-\nabla f$: the principle behind gradient descent used to train neural networks.

The tangent plane to the graph of $f$ at $(x_0,y_0)$ is the analogue of the tangent line: the plane that best approximates the surface there.

It is exactly the linear approximation given by the differential (§3). For $(x,y)$ near $(x_0,y_0)$, $f$ is almost its tangent plane.

Second-order Taylor. For a better approximation we add the quadratic term built from the Hessian matrix $H_f$ (the second derivatives):

The linear term $\nabla f\cdot\mathbf{h}$ tilts, the quadratic term $\tfrac12\mathbf{h}^TH_f\mathbf{h}$ curves. At critical points (where $\nabla f=\mathbf{0}$) only the quadratic term survives: it is the Hessian's quadratic form that decides minimum, maximum or saddle — the topic of the next section.

To optimise $f$ we first find the critical points, where the gradient vanishes:

There the tangent plane is horizontal: candidates for maximum, minimum or saddle. To classify them we use the Hessian matrix and the sign of the quadratic form it generates. In 2D, with $\det H=f_{xx}f_{yy}-f_{xy}^2$:

• $\det H>0$ and $f_{xx}>0$: $H$ positive definite ⟹ local minimum (bowl up, $f$ grows in every direction).
• $\det H>0$ and $f_{xx}<0$: $H$ negative definite ⟹ local maximum.
• $\det H<0$: $H$ indefinite ⟹ saddle point (grows in some directions, decreases in others — like a mountain pass or a horse saddle).
• $\det H=0$: test inconclusive, higher orders needed.

Why it works. Near a critical point $f(\mathbf{x}_0+\mathbf{h})\approx f_0+\tfrac12\mathbf{h}^TH\mathbf{h}$: the behaviour is that of the quadratic form. Positive definite ⟹ bowl (minimum); indefinite ⟹ saddle. The $\det H, f_{xx}$ criterion is the Hessian eigenvalue test in disguise (same-sign positive eigenvalues ⟹ minimum, etc.).

Chain rule. If $f$ is evaluated along a curve $\mathbf{r}(t)=(x(t),y(t))$, the composite $g(t)=f(\mathbf{r}(t))$ has derivative given by the gradient dotted with the velocity:

It is the multivariable version of $(f\circ g)'=f'g'$. It confirms that the directional derivative (§4) is the derivative along a line traversed at unit speed.

Implicit function theorem (Dini). A level curve $F(x,y)=0$ is generally not the graph of a function. But locally it is, provided the tangent is not vertical. If $F(x_0,y_0)=0$, $F\in C^1$ and $F_y(x_0,y_0)\neq 0$, then there exists $y=\varphi(x)$ with $F(x,\varphi(x))=0$ near $x_0$, and

Derivation: differentiating $F(x,\varphi(x))=0$ in $x$ via the chain rule, $F_x+F_y\varphi'=0$. Example: on the circle $x^2+y^2=1$, near $(0,1)$ we have $\varphi'=-x/y$, horizontal at the top ($x=0$).

Often we want to optimise $f(x,y)$ not everywhere, but subject to a constraint $g(x,y)=0$ (e.g. maximise an area at fixed perimeter). The method of Lagrange multipliers solves the problem.

The geometric idea. At a constrained optimum, the level curve of $f$ is tangent to the constraint $g=0$. Since the gradient is perpendicular to level curves, this means $\nabla f$ and $\nabla g$ are parallel:

The number $\lambda$ is the Lagrange multiplier. Solve the three equations ($f_x=\lambda g_x$, $f_y=\lambda g_y$, $g=0$) in the unknowns $x,y,\lambda$.

Meaning of $\lambda$. It measures the sensitivity of the optimum to relaxing the constraint: in economics it is the "shadow price", the change of optimal value per unit of constraint. For several constraints, $\nabla f=\sum_i\lambda_i\nabla g_i$.