# 📚 PyTorch Tutorial for Beginners - Part 2/6: Deep Neural Networks & Training Techniques
#PyTorch #DeepLearning #MachineLearning #NeuralNetworks #Training

Welcome to Part 2 of our comprehensive PyTorch series! This lesson dives deep into building and training neural networks, covering architectures, activation functions, optimization, and more.

---

## 🔹 Recap & Setup

import torch
import torch.nn as nn
import torch.optim as optim
import matplotlib.pyplot as plt
from torch.utils.data import DataLoader, TensorDataset

# Check GPU
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
print(f"Using device: {device}")

---

## 🔹 Deep Neural Network (DNN) Architecture
### 1. Key Components
| Component | Purpose | PyTorch Implementation |
|--------------------|-------------------------------------------------------------------------|------------------------------|
| Input Layer | Receives raw features | nn.Linear(input_dim, hidden_dim) |
| Hidden Layers | Learn hierarchical representations | Multiple nn.Linear + Activation |
| Output Layer | Produces final predictions | nn.Linear(hidden_dim, output_dim) |
| Activation | Introduces non-linearity | nn.ReLU(), nn.Sigmoid(), etc. |
| Loss Function | Measures prediction error | nn.MSELoss(), nn.CrossEntropyLoss() |
| Optimizer | Updates weights to minimize loss | optim.SGD(), optim.Adam() |

### 2. Building a DNN

class DNN(nn.Module):
    def __init__(self, input_size, hidden_sizes, output_size):
        super().__init__()
        layers = []
        
        # Hidden layers
        prev_size = input_size
        for hidden_size in hidden_sizes:
            layers.append(nn.Linear(prev_size, hidden_size))
            layers.append(nn.ReLU())
            prev_size = hidden_size
            
        # Output layer (no activation for regression)
        layers.append(nn.Linear(prev_size, output_size))
        
        self.net = nn.Sequential(*layers)
        
    def forward(self, x):
        return self.net(x)

# Example: 3-layer network (input=10, hidden=[64,32], output=1)
model = DNN(10, [64, 32], 1).to(device)
print(model)

---

## 🔹 Activation Functions
### 1. Common Choices
| Activation | Formula | Range | Use Case | PyTorch |
|-----------------|----------------------|------------|------------------------------|------------------|
| ReLU | max(0, x) | [0, ∞) | Hidden layers | nn.ReLU() |
| Leaky ReLU | max(0.01x, x) | (-∞, ∞) | Avoid dead neurons | nn.LeakyReLU() |
| Sigmoid | 1 / (1 + e^(-x)) | (0, 1) | Binary classification | nn.Sigmoid() |
| Tanh | (e^x - e^(-x)) / ... | (-1, 1) | RNNs, some hidden layers | nn.Tanh() |
| Softmax | e^x / sum(e^x) | (0, 1) | Multi-class classification | nn.Softmax() |

### 2. Visual Comparison

x = torch.linspace(-5, 5, 100)
activations = {
    "ReLU": nn.ReLU()(x),
    "LeakyReLU": nn.LeakyReLU(0.1)(x),
    "Sigmoid": nn.Sigmoid()(x),
    "Tanh": nn.Tanh()(x)
}

plt.figure(figsize=(12, 4))
for i, (name, y) in enumerate(activations.items()):
    plt.subplot(1, 4, i+1)
    plt.plot(x.numpy(), y.numpy())
    plt.title(name)
plt.tight_layout()
plt.show()

---

🌟 Vision Transformer (ViT) Tutorial – Part 1: From CNNs to Transformers – The Revolution in Computer Vision

Let's start: https://hackmd.io/@husseinsheikho/vit-1

#VisionTransformer #ViT #DeepLearning #ComputerVision #Transformers #AI #MachineLearning #NeuralNetworks #ImageClassification #AttentionIsAllYouNeed

✉️ Our Telegram channels: https://xn--r1a.website/addlist/0f6vfFbEMdAwODBk

📱 Our WhatsApp channel: https://whatsapp.com/channel/0029VaC7Weq29753hpcggW2A