AI ACCELERATOR SOC DESIGN

Lecture 6: Neural Network Framework

SAWERA KHURSHID
2023299011
 NEURAL NETWORK FRAMEWORK
Neural network frameworks are software libraries designed to simplify the process of building,
training, and deploying neural networks. They provide a set of pre-implemented algorithms,
optimization techniques, and tools to work with neural networks efficiently.
1. PyTorch:
PyTorch is an open-source machine learning library developed by Facebook's AI Research lab
(FAIR). It has gained significant popularity due to its flexibility, dynamic computation graph,
and easy-to-use interface.
Key features of PyTorch include:
 Dynamic computation graph: Allows for defining and modifying computational graphs on-
the-fly, which is beneficial for tasks like dynamic networks, recurrent neural networks
(RNNs), and generative adversarial networks (GANs).
 GPU acceleration: PyTorch supports GPU acceleration, enabling fast computation for
training deep neural networks on compatible hardware.
 TorchScript: PyTorch provides a Just-In-Time (JIT) compiler called TorchScript, which
allows users to convert PyTorch models into a statically-typed intermediate representation,
suitable for deployment.
 TorchVision and TorchText: PyTorch also offers domain-specific libraries like TorchVision
for computer vision tasks and TorchText for natural language processing (NLP).
 Libraries:
torch.tensor:
 Core data structure representing multi-dimensional arrays.
 Stores data, tracks gradients, and supports operations like addition and multiplication.
 Essential for building neural networks in PyTorch.
import torch
# Create a tensor
x = torch.tensor([[1, 2], [3, 4]], dtype=torch.float32)
# Print tensor properties
print("Tensor:", x)
print("Shape:", x.shape)
print("Data type:", x.dtype)
torch.autograd:
 Automatic differentiation engine in PyTorch.
 Computes gradients of tensors with respect to some scalar value (usually loss) during
backpropagation.
 Enables training neural networks by automatically computing gradients of operations.
import torch
# Create a tensor with requires_grad=True
x = torch.tensor([2.0], requires_grad=True)
y = torch.tensor([3.0], requires_grad=True)
# Perform some computation
z = x**2 + y
# Compute gradients
z.backward()
# Print gradients
print("Gradient of x:", x.grad) # Should be 4.0
print("Gradient of y:", y.grad) # Should be 1.0

torch.nn.Module:
 Base class for all neural network modules in PyTorch.
 Manages parameters, defines network architecture, and implements forward computation.
 Subclassing it allows for easy organization and management of network layers.
import torch
import torch.nn as nn
# Define a simple neural network module
class SimpleNN(nn.Module):
def __init__(self):
super(SimpleNN, self).__init__()
self.fc1 = nn.Linear(2, 3) # Fully connected layer with input size 2 and output size 3
self.fc2 = nn.Linear(3, 1) # Fully connected layer with input size 3 and output size 1
def forward(self, x):
x = torch.relu(self.fc1(x)) # Apply ReLU activation to the first layer
x = self.fc2(x) # Second layer without activation (assuming regression task)
return x
# Create an instance of the network
model = SimpleNN()
 # Example usage
input_data = torch.tensor([[1.0, 2.0], [3.0, 4.0]]) # Example input data
output = model(input_data) # Forward pass through the network
print("Output:", output)

2. TensorFlow:
TensorFlow is an open-source machine learning framework developed by Google Brain. It
provides comprehensive support for building various machine learning models, including neural
networks, deep learning models, and reinforcement learning algorithms.
Key features of TensorFlow include:
 Static computation graph: TensorFlow initially adopted a static computation graph model,
where users define the entire computational graph before running the model. However,
TensorFlow 2.x introduced eager execution, which provides dynamic graph execution similar
to PyTorch.
 TensorFlow Serving: A system for serving machine learning models in production
environments, allowing for scalable and efficient deployment of TensorFlow models.
 TensorFlow Hub: A repository of pre-trained machine learning models and model
components, facilitating transfer learning and model reuse.
 TensorFlow Lite: An optimized version of TensorFlow for mobile and embedded devices,
enabling inference on resource-constrained platforms.

3. Keras:
Keras is an open-source deep learning library that provides a high-level API for building and
training neural networks. It was originally developed as an independent project but later
integrated into TensorFlow as its official high-level API.
Key features of Keras include:
 User-friendly API: Keras offers a simple and intuitive API for defining neural network
architectures, making it suitable for both beginners and experienced practitioners.
 Modular design: Keras follows a modular design philosophy, allowing users to easily
assemble neural network layers to create complex architectures.
 Multi-backend support: While Keras is tightly integrated with TensorFlow, it also supports
other backends such as Microsoft Cognitive Toolkit (CNTK) and Theano.