Day 7 of self-studying Berkeley CS189 — stochastic gradient descent notes 📚📝
🔥 *Stochastic Gradient Descent (SGD)* is a powerful optimization algorithm used to minimize loss functions in machine learning. Unlike batch gradient descent, which uses the entire dataset to compute gradients, SGD updates parameters using a single training example (or a small mini-batch) at a time.
🚀 Key Benefits:
- Faster convergence on large datasets
- Escapes local minima more easily
- Suitable for online learning scenarios
📊 The Update Rule:
Where
📌 Challenges:
- High variance in updates
- Requires careful tuning of the learning rate
🧠 *Tip:* Use momentum or adaptive learning rates (like Adam) to stabilize training!
#MachineLearning #CS189 #SGD #DeepLearning #DataScience #Algorithms
✨ Join Best TG Channels https://xn--r1a.website/addlist/0f6vfFbEMdAwODBk
⭐️ Join Our WhatsApp Channel https://whatsapp.com/channel/0029VaC7Weq29753hpcggW2A
🔥 *Stochastic Gradient Descent (SGD)* is a powerful optimization algorithm used to minimize loss functions in machine learning. Unlike batch gradient descent, which uses the entire dataset to compute gradients, SGD updates parameters using a single training example (or a small mini-batch) at a time.
🚀 Key Benefits:
- Faster convergence on large datasets
- Escapes local minima more easily
- Suitable for online learning scenarios
📊 The Update Rule:
θ = θ - α * ∇J(θ; x⁽ⁱ⁾, y⁽ⁱ⁾)Where
α is the learning rate and (x⁽ⁱ⁾, y⁽ⁱ⁾) is a single training example.📌 Challenges:
- High variance in updates
- Requires careful tuning of the learning rate
🧠 *Tip:* Use momentum or adaptive learning rates (like Adam) to stabilize training!
#MachineLearning #CS189 #SGD #DeepLearning #DataScience #Algorithms
✨ Join Best TG Channels https://xn--r1a.website/addlist/0f6vfFbEMdAwODBk
⭐️ Join Our WhatsApp Channel https://whatsapp.com/channel/0029VaC7Weq29753hpcggW2A
❤6