97_Simple ANN Example Using PyTorch

 

 

Having previously focused on thoretical content in the deep learning category, I plan to conduct an example. In this post I will perform a simple ANN (Artificial Neural Network) example in a PyTorch environment using the MNIST dataset.

MNIST is a large-scale dataset composed of handwritten digits and is widely used in the fields of machine learning and computer vision for training and testing image processing algorithms. 

 

Because MNIST is widely used as a standard benchmark for comparing and evaluating the performance of algorithms, using a deep learning framework like PyTorch allows for the easy loading and use of the MNIST dataset.

 

First, import the necessary libraries.

 

import torch
import torchvision
import torchvision.transforms as transforms
from torch.utils.data import DataLoader, random_split
import torch.optim as optim
from tqdm import tqdm
import matplotlib.pyplot as plt

 

'torchvision' helps in loading image datasets and facilitates easy transformations on image data.

 

train_set = torchvision.datasets.MNIST(root='./data', train=True, download = True, 
                                       transform=transforms.Compose([transforms.ToTensor()]))

train_size = int(0.8*len(train_set))
validation_size = len(train_set)-train_size
train_dataset, validation_dataset = random_split(train_set, [train_size, validation_size])

train_loader = DataLoader(train_dataset, batch_size=64, shuffle = True)
validation_loader = DataLoader(validation_dataset, batch_size=64)

 

Using 'torchvision.datasets', you can fetch and save the MNIST dataset. With 'transforms.Compose', it's possible to convert images to PyTorch tensor types, and the pixel values are automatically scaled to a range of 0 to 1. The ratio for training and validation has been set to 8:2, and the batch size is configured to 64.

The loaded data is as follows.

 

 

Now, define the model.

 

class Net(torch.nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.input = torch.nn.Linear(28*28, 256) 
        self.hidden = torch.nn.Linear(256, 128)   
        self.output = torch.nn.Linear(128, 10)    

    def forward(self, x):
        x = x.view(-1, 28*28) 
        x = torch.relu(self.input(x))
        x = torch.relu(self.hidden(x))
        x = self.output(x)
        return x

 

Define the class using 'torch.nn.Module', and the model's architecture is kept very simple. It takes an input of 28*28 pixels and produces 10 outputs in the output layer. The activation function used is ReLU.

 

The neural network training is as follows.

 

def train(model, train_loader, validation_loader, epochs = 10):
    train_losses = []
    validation_losses = []
    
    criterion = torch.nn.CrossEntropyLoss()
    optimizer = optim.SGD(model.parameters(), lr=0.01, momentum=0.9)
    
    for epoch in range(epochs):
        model.train()
        running_loss = 0.0
        for inputs, labels in tqdm(train_loader, desc=f"Epoch {epoch+1}/{epochs}"):
            optimizer.zero_grad()
            outputs = model(inputs)
            loss = criterion(outputs, labels)
            loss.backward()
            optimizer.step()
            running_loss += loss.item()
        train_losses.append(running_loss / len(train_loader))

        model.eval()
        validation_loss = 0.0
        with torch.no_grad():
            for inputs, labels in validation_loader:
                outputs = model(inputs)
                loss = criterion(outputs, labels)
                validation_loss += loss.item()
        validation_losses.append(validation_loss / len(validation_loader))
        
        print(f'Epoch {epoch+1}, Train Loss: {train_losses[-1]:.4f}, Validation Loss: {validation_losses[-1]:.4f}')

    return train_losses, validation_losses

 

For the loss function, CrossEntropyLoss, which is widely used in classification tasks, was utilized, and the optimizer for updating the model's parameters was SGD to minimize the model's loss.

Training proceeded for 10 epochs, and 'tqdm' was used to monitor progress within each epoch. After each epoch, the Training loss and Validation loss were printed and saved. Once all training was completed, the loss values for each were returned.

 

model = Net()
train_losses, validation_losses = train(model, train_loader, validation_loader)

plt.plot(train_losses, label='Training loss')
plt.plot(validation_losses, label='Validation loss')
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.legend()
plt.show()

 

By executing the above code to train the model, the results are as follows.

 

 

It was observed that both Training loss and Validation loss decreased as training progressed, and, as expected, the Validation losswas higher than the Training loss.