Crafting Deep Learning from the Ground Up

Explore the docs »

View Demo · Report Bug · Request Feature

Melpy is a Python deep learning library built from scratch using only NumPy. It provides an accessible way to create, train, and understand fundamental neural network architectures like Feedforward Neural Networks (FNNs), Convolutional Neural Networks (CNNs), and Long Short-Term Memory (LSTMs), featuring a clear automatic differentiation engine. Ideal for learners and those seeking intuition into machine learning algorithms without high-level abstractions.

Started in 2022 during high school, this project stemmed from frustration with the lack of clarity in how complex algorithms worked in existing frameworks. Building custom implementations led to deeper understanding and evolved into Melpy.

Inspired by leading machine learning tools, Melpy simplifies creating and training common deep learning models. It includes essential tools for data preprocessing and visualization, offering a complete, albeit fundamental, neural network workflow. Melpy's key feature is simplicity, offering an accessible experience for beginners exploring deep learning. It serves as both a learning resource and a foundation for expansion, implementing core functionalities simply to provide clear intuition.

(back to top)

(back to top)

Requires an up-to-date Python environment (conda recommended for scientific use).

Dependencies (including NumPy) are installed automatically.

Melpy is available on PyPI. Install the latest version using pip:

pip3 install melpy --upgrade

(back to top)

Melpy computations rely on Tensor objects (extended NumPy arrays) and Operation types. Together, they construct computational graphs, enabling precise gradient calculation via an automatic differentiation algorithm. This simplifies implementing the backpropagation step for various layer types in your neural network.

Tensor extends NumPy arrays, adding requires_grad (bool), grad (gradient tensor), and _op (producing operation) attributes.

Class Tensor:
    Function __init__(object, requires_grad, _operation, additional arguments):
        Attributes
        - array : float64 numpy array
        - grad : melpy tensor
        - requires_grad : bool
        - _op : melpy operation

Operation adds a forward pass (builds the graph) and a backward pass (computes derivatives, updates/propagates gradients).

Class _Operation:
    Function __init__(x1, x2, additional arguments):
        Attributes
        - x1 : melpy tensor
        - x2 : melpy tensor
        - output : melpy tensor

Example: Compute y = 5 * x, find dy/dx using automatic differentiation.

x = tensor([0, 1, 2, 3, 4], requires_grad=True)
y = 5 * x

print(y)

Output:

[0., 5., 10., 15., 20.]

Propagate gradient backward:

y.backward(1)

print(x.grad)

Output shows dy/dx = 5 for each element, as expected:

[5., 5., 5., 5., 5.]

Figure 1: Computational graph of the operation (forward/backward passes).

Tensor and Operation classes form Melpy's core, simplifying complex neural network architecture implementation.

Learn more about automatic differentiation on Wikipedia.

(back to top)

Melpy supports common deep learning models built from scratch:

• Feedforward Neural Networks (FNNs): Using Dense layers for tasks like classification or regression on tabular data.
• Convolutional Neural Networks (CNNs): Using Convolution2D layers, suitable for basic image recognition tasks.
• Recurrent Neural Networks (RNNs): Specifically Long Short-Term Memory (LSTM) units for sequence analysis or time series prediction.

Build these architectures using provided layers and train using the Sequential class.

Future plans include Batch Normalization and potentially generative models like GANs.

(back to top)

Key strength: Modularity. Melpy leverages Object-Oriented Programming with specific types, making it easy to add/modify layers, loss functions, callbacks, etc.

The Tensor type combined with automatic differentiation simplifies creating new components without manual gradient calculation for simpler tasks. The Sequential class integrates components while allowing custom neural network architectures and training loops.

Result: Melpy is flexible, adaptable, and upgradable.

(back to top)

Example: Train a Feedforward Neural Network (FNN) for Iris flower classification, a classic machine learning problem. The dataset has 3 classes and 4 features. Build models for tasks ranging from simple classification (like this example) to basic image recognition (using Convolution2D) or sequence analysis (with LSTM).

Load data and split:

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split

iris_dataset = load_iris()

X_train, X_test, y_train, y_test = train_test_split(
        iris_dataset['data'], iris_dataset['target'], test_size=0.25, random_state=0)

Visualize data:

import matplotlib.pyplot as plt

plt.figure()
plt.scatter(X_train[:,0], X_train[:,1], c=y_train, alpha=0.3, cmap="coolwarm")
plt.show()

Figure 2: Iris dataset visualized, showing feature separation.

Figure 2 shows correlation between features (Sepal Length/Width) and species.

Neural Networks often require scaled input. Use StandardScaler:

The Standard Scaler centers data (removes mean) and scales by variance. More on Feature Scaling.

from melpy.preprocessing import StandardScaler

sc = StandardScaler()
X_train = sc.transform(X_train)
X_test = sc.transform(X_test)

Encode target labels using OneHotEncoder for classification:

One-hot encoding represents categories as binary vectors. More on One-hot.

from melpy.preprocessing import OneHotEncoder

ohe = OneHotEncoder()
y_train = ohe.transform(y_train)
y_test = ohe.transform(y_test)

For this multi-class tabular classification problem, we need:

• Dense layers (Fully Connected) for feature learning.
• Softmax activation for probability outputs.
• Categorical Cross-Entropy loss function for optimization.

Build the neural network model with Sequential:

Sequential models stack layers linearly; data flows sequentially.

import melpy.NeuralNetworks as nn

# Define the sequential model structure
model = nn.Sequential(X_train, y_train, X_test, y_test)

# Add layers: Input -> Hidden (ReLU) -> Output (Softmax)
model.add(nn.Dense(X_train.shape[1], 6, activation="relu"))
model.add(nn.Dense(6, y_train.shape[1], activation="softmax"))

# Compile the model with loss function and optimizer
model.compile(loss_function=nn.CategoricalCrossEntropy(), optimizer=nn.SGD(learning_rate=0.01))

We define: Training/validation data, hidden layer (6 neurons, ReLU activation), output layer (Softmax activation), loss function, and optimizer (SGD).

This defines the FNN architecture. New to deep learning? Check 3Blue1Brown's excellent video series.

View model structure:

model.summary()

Dense: (1, 6)
Dense: (1, 3)

Train the neural network model:

model.fit(epochs=5000, verbose = 1, callbacks=[nn.LiveMetrics()])
model.results()

Figure 3: Plot updated during training via LiveMetrics() callback.

Epoch [5000/5000]: 100%|██████████| 5000/5000 [00:03<00:00, 1543.94it/s, loss=0.0389, accuracy=0.988]

-------------------------------------------------------------------
| [TRAINING METRICS] train_loss: 0.03893 · train_accuracy: 0.9881 |
-------------------------------------------------------------------
| [VALIDATION METRICS] val_loss: 0.06848 · val_accuracy: 0.98246  |
-------------------------------------------------------------------

Achieves ~98% accuracy on train/test. Good result for this simple neural network. Potential for 100% with tuning.

Right plot shows model inputs colored by prediction, visually confirming good training performance.

Save trained model parameters and metrics:

model.save_params("iris_parameters")
model.save_histories("iris_metrics")

Reload parameters with load_params(path) and metrics using h5py.

See more examples (including potential CNN or LSTM use cases) in the Examples folder

(back to top)

First, clear your console to avoid issues with old variables.

Most runtime errors in neural network code involve tensor shape mismatches. Common cause: Mismatched input/output feature counts between consecutive Dense or Convolution2D layers.

Dense example:

model.add(nn.Dense(X_train.shape[1], 7, activation="relu")) # Output has 7 features
model.add(nn.Dense(6, y_train.shape[1], activation="softmax")) # Expects 6 input features

ValueError: shapes (1,7) and (6,2) not aligned: 7 (dim 1) != 6 (dim 0) 

Convolution2D example:

model.add(nn.Convolution2D(in_channels=1, out_channels=32, ...)) # Output has 32 channels
model.add(nn.Convolution2D(in_channels=12, out_channels=64, ...)) # Expects 12 input channels

ValueError: matmul: Input operand 1 has a mismatch in its core dimension 0, with gufunc signature (n?,k),(k,m?)->(n?,m?) (size 128 is different from 48)

Resolution: Ensure out_features/out_channels of a layer match the expected in_features/in_channels of the next layer in the sequence.

(back to top)

Future plans:

• Optimize computation speed (potentially using Numba, JAX and/or multiprocessing).
• Implement more deep learning architectures (e.g., GANs).
• Add traditional machine learning algorithms.
• Expand layer options.

See open issues for proposed features and known issues.

(back to top)

Contributions make the open source community thrive and are greatly appreciated. Help improve this deep learning library!

If you have suggestions, please fork the repo and create a pull request, or open an issue with the "enhancement" tag. Don't forget to star the project! Thanks!

1. Fork the Project
2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
3. Commit your Changes (git commit -m 'Add some AmazingFeature')
4. Push to the Branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

(back to top)

Distributed under the MIT License. See LICENSE.txt for details.

(back to top)

Lenny Malard - [email protected] or Linkedin

Project Link: https://github.com/lennymalard/melpy-project

(back to top)