Back to the lessonPreparing the printout…

Mathematical Foundations for Data Science · Part II — Calculus and Optimization

Topic 8 · Partial Derivatives and the Gradient

[video] Lecture video — Topic 8: Partial Derivatives and the Gradient — /api/lesson-media/videos/4d3653ff-a3ca-4f63-bf8e-bb93c553e599-1794892661691-ce8b6a488aa1a892.mp4

Flashcards · Partial derivatives and the gradient

Function of several variables — F: ℝⁿ → ℝ assigns one real number to each input vector (x₁, …, xₙ).

F(b, w) = b² + 3bw + 2w².

Partial derivative — ∂F/∂xⱼ: the derivative along the slice where only xⱼ varies and all other variables are held fixed.

∂F/∂b = 2b + 3w; at (1, 2) it equals 8.

Gradient — ∇F: the column vector of all n partial derivatives, in input order.

∇F(1, 2) = (8, 11)ᵀ.

Linear approximation — F(a + h) ≈ F(a) + ∇F(a)·h for small h, if F is differentiable at a.

Δx₁ = 0.01, Δx₃ = −0.02 gives ΔF ≈ 4(0.01) + 1(−0.02) = 0.02.

Directional derivative — D_uF(a) = ∇F(a)·u: the rate of change per unit distance along a unit vector u (needs differentiability).

u = (3/5, 4/5): D_uF = 68/5.

Steepest direction — ∇F points toward steepest increase, −∇F toward steepest decrease; the rates are ±‖∇F‖. A direction orthogonal to ∇F has first-order change 0.

Partials do not imply differentiability — All partial derivatives can exist (even be 0) while the function is not differentiable. Continuous partials near a point do guarantee differentiability.

G = s²t/(s² + t²): ∇G(0, 0) = 0 but D(1,1)G(0, 0) = 1/2.

Gradient of the squared-error loss — With rᵢ = b + wxᵢ − yᵢ: ∇L_MSE = (2/m)(Σrᵢ, Σrᵢxᵢ), and ∇L_half = ½∇L_MSE. Vectorised: (1/m)Aᵀ(Aθ − y) for half-MSE.

Chain rule along a path — d/dt F(x(t)) = ∇F(x(t))·x′(t): add each coordinate's partial derivative times its speed.

F(1 + t, 2 − t) = 15 − 3t, rate −3.

Gradient of a sigmoid loss — For ℓ = ½(p − y)² with p = σ(aᵀθ): ∇ℓ = (p − y)·p(1 − p)·a. The factor p(1 − p) is small when p is near 0 or 1.

[markdown] content available only on the platform

Mathematical Foundations for Data Science · Topic 8 · Partial Derivatives and the Gradient · printed on November 20, 2026