🔄 1. Backpropagation

• Backpropagation is the core technique to optimize weights (W) in neural networks.
• It works by computing gradients using the chain rule — so the network knows how to adjust each weight to reduce the loss.
• This process allows each layer to learn how it should change its parameters based on the final prediction error.

📐 2. What is the Jacobian Matrix?

• The Jacobian Matrix is the matrix of all first-order partial derivatives for a function.
• In backpropagation, it describes how small changes in inputs (like image pixels) affect outputs (like class scores).
• It’s often a diagonal matrix in simpler cases — this ensures each input feature maps correctly to its gradient.

Key Role:

• Computes how outputs change with respect to inputs or intermediate parameters.
• Essential for propagating gradients layer by layer.

🧩 3. How It Works in CNNs

• For a layer with m outputs and n inputs, the Jacobian is an m × n matrix.
• It’s used to:
  • Calculate weight updates.
  • Perform sensitivity analysis (understanding which inputs affect which outputs).

📚 4. See More: Full Backpropagation Walkthrough

The lecture covers the full details of backpropagation step-by-step.