Đại học Kinh tế Quốc dân – National Economics UniversityMICRO4ASIA
🏆 Arena
Sign inSign up
MICRO4ASIAThe six micro-credentials

Courses are delivered on the Limio learning platform. Limio

← Mathematical Foundations for Data Science

You are viewing a public preview

Sign in to save your progress, do assignments, join discussions and get feedback.

Sign in
Part II — Calculus and Optimization

Topic 8 · Partial Derivatives and the Gradient

Print / Save as PDF
Lecture video — Topic 8: Partial Derivatives and the Gradient

Functions of several variables, partial derivatives, the gradient, directional derivatives, model, parameters and loss.

Flashcards · Partial derivatives and the gradient

10 words
WordMeaningExample
  • Function of several variables

    F: ℝⁿ → ℝ assigns one real number to each input vector (x₁, …, xₙ).

    F(b, w) = b² + 3bw + 2w².

  • Partial derivative

    ∂F/∂xⱼ: the derivative along the slice where only xⱼ varies and all other variables are held fixed.

    ∂F/∂b = 2b + 3w; at (1, 2) it equals 8.

  • Gradient

    ∇F: the column vector of all n partial derivatives, in input order.

    ∇F(1, 2) = (8, 11)ᵀ.

  • Linear approximation

    F(a + h) ≈ F(a) + ∇F(a)·h for small h, if F is differentiable at a.

    Δx₁ = 0.01, Δx₃ = −0.02 gives ΔF ≈ 4(0.01) + 1(−0.02) = 0.02.

  • Directional derivative

    D_uF(a) = ∇F(a)·u: the rate of change per unit distance along a unit vector u (needs differentiability).

    u = (3/5, 4/5): D_uF = 68/5.

  • Steepest direction

    ∇F points toward steepest increase, −∇F toward steepest decrease; the rates are ±‖∇F‖. A direction orthogonal to ∇F has first-order change 0.

  • Partials do not imply differentiability

    All partial derivatives can exist (even be 0) while the function is not differentiable. Continuous partials near a point do guarantee differentiability.

    G = s²t/(s² + t²): ∇G(0, 0) = 0 but D(1,1)G(0, 0) = 1/2.

  • Gradient of the squared-error loss

    With rᵢ = b + wxᵢ − yᵢ: ∇L_MSE = (2/m)(Σrᵢ, Σrᵢxᵢ), and ∇L_half = ½∇L_MSE. Vectorised: (1/m)Aᵀ(Aθ − y) for half-MSE.

  • Chain rule along a path

    d/dt F(x(t)) = ∇F(x(t))·x′(t): add each coordinate's partial derivative times its speed.

    F(1 + t, 2 − t) = 15 − 3t, rate −3.

  • Gradient of a sigmoid loss

    For ℓ = ½(p − y)² with p = σ(aᵀθ): ∇ℓ = (p − y)·p(1 − p)·a. The factor p(1 − p) is small when p is near 0 or 1.

Practice in Python

Work through Section 3 of Lab 2 (Calculus and Optimization, MFDS_Lab2_Calculus_Optimization.ipynb), attached to the Part II materials lesson at the start of this part.