SmartDNN is a modern C++ deep learning library designed to offer a flexible and efficient framework for developing and training deep neural networks. With a focus on providing a high-level API, SmartDNN simplifies the process of building and training various neural network architectures while maintaining the performance advantages of C++.
Creating your first neural network with SmartDNN is straightforward. Define your model's architecture using a variety of available layers, compile it with your chosen loss function and optimizer, and then train it using your dataset.
int epochs = 100;
float learningRate = 0.001f;
SmartDNN model;
model.addLayer( FullyConnectedLayer(10, 100) ); // Fully Connected Layer - 10 -> 100
model.addLayer( ActivationLayer( ReLU()) ); // Activation Function Layer - ReLU.
model.addLayer( FullyConnectedLayer(100, 100) ); // Fully Connected Layer - 100 -> 100
model.addLayer( ActivationLayer( Sigmoid() ) ); // Activation Function Layer - Sigmoid.
model.addLayer( FullyConnectedLayer(100, 10) ); // Fully Connected Layer - 100 -> 10
model.addLayer( ActivationLayer( Softmax() ) ); // Activation Function Layer - Softmax.
model.compile(MSELoss(), AdamOptimizer());
model.train(inputs, targets, epochs);// Initialize the SmartDNN MNist model
SmartDNN<float> model;
model.addLayer( Conv2DLayer(1, 32, 3) ); // Conv2D layer.
model.addLayer( BatchNormalizationLayer(32) ); // Batch normalization after Conv2D.
model.addLayer( ActivationLayer( ReLU() ) ); // ReLU activation.
model.addLayer( MaxPooling2DLayer(2, 2) ); // Added MaxPooling.
model.addLayer( DropoutLayer(0.25f) ); // Reduced dropout rate.
model.addLayer( FlattenLayer() ); // Flatten layer.
model.addLayer( FullyConnectedLayer(5408, 128) ); // Adjusted input size due to MaxPooling.
model.addLayer( BatchNormalizationLayer(128) ); // Batch normalization after FC.
model.addLayer( ActivationLayer( ReLU() ) ); // ReLU activation.
model.addLayer( DropoutLayer(0.25f) ); // Reduced dropout rate.
model.addLayer( FullyConnectedLayer(128, 10) ); // Output layer.
model.addLayer( ActivationLayer( Softmax() ) ); // Softmax activation.
AdamOptions adamOptions;
adamOptions.learningRate = learningRate;
adamOptions.beta1 = 0.9f;
adamOptions.beta2 = 0.999f;
adamOptions.epsilon = 1e-8f;
adamOptions.l1Strength = 0.0f;
adamOptions.l2Strength = 0.0f;
adamOptions.decay = 0.0f;
model.compile(CategoricalCrossEntropyLoss(), AdamOptimizer(adamOptions));
model.train(inputs, targets, epochs);- Custom Tensor Library: A robust and feature-rich tensor library with comprehensive tensor operations.
- Testing Environment: A built-in testing environment that facilitates clean and efficient development.
- Layers: Includes essential layers such as Fully Connected layer, Convolutional 2D layer, Flatten layer and Activation layers.
- Optimizers: Currently supports the Adam optimizer.
- Loss Functions: Implements Mean Squared Error (MSE) for regression tasks and Categorical Cross Entropy.
- Activation Functions: Includes popular activation functions like Softmax, Sigmoid, Tanh, ReLU, and Leaky ReLU.
- Regularisation Techniques: Batch Normalisation, Dropout, Max Pooling 2D.
- Linear Regression Classification.
- Tested on 1000 samples.
- Trained for 1000 epochs.
- Average non-templated runtime: ~17680ms
- Average optimised templated runtime: ~8325ms
- Performance gains: ~53% performance improvement!
- MNist Classification task.
- Tested on 1000 samples, batch size: 64.
- Trained for 1000 epochs.
- Average non-templated runtime: (per epoch) ~83 minutes
- Average optimised templated runtime (per epoch): ~10969ms
- Performance gains: ~99.8% performance improvement!
- Slice View: SliceView allowing for sliced data access into a Tensor without copying.
- Broadcast View: BroadcastView instead of actually broadcasting data significantly increases performance.
- Transforms: Implementated transforms across all operations running on iterators, enables much more performant compiler optimisations.
- Cleaner interface: better principles applied to retain single responsibility.
- Templates: Now you can specify what type of data you are using on any default type
- Parallel Directives: Better application of directives for expensive loops.
- Simple Linear Regression model with dataset generator - [Fully Connected Layer, ReLU, Mean Squared Error Loss, Adam Optimizer]
- Convolutional Neural Network for training MNist Dataset - [Convolutional 2D Layer, ReLU, Max Pooling 2D Layer, Flatten Layer, Softmax, Categorical Cross Entropy Loss, Adam Optimizer]
To install the library, follow these steps:
-
Clone the repository:
git clone https://github.com/A-Georgiou/SmartDNN.git -
Build the library by running the command
CMake .which will install required dependencies. -
Create a
src/main.cppfile to create your neural networks. -
Build your project from the Makefile with the command
make -
Run the program with command
./SmartDNN
To create your first model and run using docker, follow these steps:
-
Clone the repository:
git clone https://github.com/A-Georgiou/SmartDNN.git -
Create a
src/main.cppfile where your model creation code should exist, feel free to copy an example from theExamples/folder. -
Build the Docker Image by running the command
docker build -f .docker/Dockerfile -t smartdnn-app . -
Run the project with the command
docker run --rm -it smartdnn-app
- Extended Layer Support: Upcoming support for additional layers, including Convolutional, Recurrent, and more.
- Advanced Network Architectures: Flexible and customizable network architectures with user-friendly APIs.
- GPU Acceleration: Future integration of CUDA for efficient GPU-based training and inference.
- Comprehensive Documentation: Detailed documentation and examples to accelerate your development process.
Contributions are welcome! If you would like to contribute to the project, please reach out to me via my contact information below.
This project is licensed under the MIT License.
For any questions or inquiries, please contact [email protected].