Deep Learning & Related Platforms

1. TensorFlow 

TensorFlow is arguably one of the best deep learning frameworks and has been adopted by several giants such as Airbus, Twitter, IBM, and others mainly due to its highly flexible system architecture. 

The most well-known use case of TensorFlow has got to be Google Translate coupled with capabilities such as natural language processing, text classification/summarization, speech/image/handwriting recognition, forecasting, and tagging. 

TensorFlow is available on both desktop and mobile and also supports languages such as Python, C++, and R to create deep learning models along with wrapper libraries. 

TensorFlow comes with two tools that are widely used: 

TensorBoard for the effective data visualization of network modeling and performance. 

TensorFlow Serving for the rapid deployment of new algorithms/experiments while retaining the same server architecture and APIs. It also provides integration with other TensorFlow models, which is different from conventional practices and can be extended to serve other model and data types. 

If you’re taking your first steps toward deep learning, it is a no-brainer to opt for TensorFlow given that is Python-based, is supported by Google, and comes loaded with documentation and walkthroughs to guide you. 

2. Caffe 

Caffe is a deep learning framework that is supported with interfaces like C, C++, Python, and MATLAB as well as the command line interface. It is well known for its speed and transposability and its applicability in modeling convolution neural networks (CNN). The biggest benefit of using Caffe’s C++ library (comes with a Python interface) is the ability to access available networks from the deep net repository Caffe Model Zoo that are pre-trained and can be used immediately. When it comes to modeling CNNs or solving image processing issues, this should be your go-to library. 

Caffe’s biggest USP is speed. It can process over 60 million images on a daily basis with a single Nvidia K40 GPU. That’s 1 ms/image for inference and 4 ms/image for learning — and more recent library versions are faster still. 

Caffe is a popular deep learning network for visual recognition. However, Caffe does not support fine-granular network layers like those found in TensorFlow or CNTK. Given the architecture, the overall support for recurrent networks, and language modeling it’s quite poor, and establishing complex layer types has to be done in a low-level language. 

3. Microsoft Cognitive Toolkit/CNTK 

Popularly known for easy training and the combination of popular model types across servers, the Microsoft Cognitive Toolkit (previously known as CNTK) is an open-source deep learning framework to train deep learning models. It performs efficient convolution neural networks and training for image, speech, and text-based data. Similar to Caffe, it is supported by interfaces such as Python, C++, and the command line interface. 

Given its coherent use of resources, the implementation of reinforcement learning models or generative adversarial networks (GANs) can be done easily using the toolkit. It is known to provide higher performance and scalability as compared to toolkits like Theano or TensorFlow while operating on multiple machines. 

Compared to Caffe, when it comes to inventing new complex layer types, users don’t need to implement them in a low-level language due to the fine granularity of the building blocks. The Microsoft Cognitive Toolkit supports both RNN and CNN types of neural models and thus is capable of handling images, handwriting, and speech recognition problems. Currently, due to the lack of support on ARM architecture, its capabilities on mobile are fairly limited. 

4. Torch/PyTorch 

Torch is a scientific computing framework that offers wide support for machine learning algorithms. It is a Lua-based deep learning framework and is used widely amongst industry giants such as Facebook, Twitter, and Google. It employs CUDA along with C/C++ libraries for processing and was basically made to scale the production of building models and provide overall flexibility. 

As of late, PyTorch has seen a high level of adoption within the deep learning framework community and is considered to be a competitor to TensorFlow. PyTorch is basically a port to the Torch deep learning framework used for constructing deep neural networks and executing tensor computations that are high in terms of complexity. 

As opposed to Torch, PyTorch runs on Python, which means that anyone with a basic understanding of Python can get started on building their own deep learning models. 

Given PyTorch framework’s architectural style, the entire deep modeling process is far simpler as well as transparent compared to Torch. 

5. MXNet 

Designed specifically for the purpose of high efficiency, productivity, and flexibility, MXNet(pronounced as mix-net) is a deep learning framework supported by Python, R, C++, and Julia. 

The beauty of MXNet is that it gives the user the ability to code in a variety of programming languages. This means that you can train your deep learning models with whichever language you are comfortable in without having to learn something new from scratch. With the backend written in C++ and CUDA, MXNet is able to scale and work with a myriad of GPUs, which makes it indispensable to enterprises. Case in point: Amazon employed MXNet as its reference library for deep learning. 

MXNet supports long short-term memory (LTSM) networks along with both RNNs and CNNs. 

This deep learning framework is known for its capabilities in imaging, handwriting/speech recognition, forecasting, and NLP. 

6. Chainer 

Highly powerful, dynamic and intuitive, Chainer is a Python-based deep learning framework for neural networks that is designed by the run strategy. Compared to other frameworks that use the same strategy, you can modify the networks during runtime, allowing you to execute arbitrary control flow statements. 

Chainer supports both CUDA computation along with multi-GPU. This deep learning framework is utilized mainly for sentiment analysis, machine translation, speech recognition, etc. using RNNs and CNNs. 

7. Keras 

Well known for being minimalistic, the Keras neural network library (with a supporting interface of Python) supports both convolutional and recurrent networks that are capable of running on either TensorFlow or Theano. The library is written in Python and was developed keeping quick experimentation as its USP. 

Due to the fact that the TensorFlow interface is a tad bit challenging coupled with the fact that it is a low-level library that can be intricate for new users, Keras was built to provide a simplistic interface for the purpose of quick prototyping by constructing effective neural networks that can work with TensorFlow. 

Lightweight, easy to use, and really straightforward when it comes to building a deep learning model by stacking multiple layers: that is Keras in a nutshell. These are the very reasons why Keras is a part of TensorFlow’s core API. 

The primary usage of Keras is in classification, text generation and summarization, tagging, and translation, along with speech recognition and more. If you happen to be a developer with some experience in Python and wish to dive into deep learning, Keras is something you should definitely check out. 

8. Deeplearning4J (DL4J) 

Parallel training through iterative reduce, microservice architecture adaptation, and distributed CPUs and GPUs are some of the salient features of the Deeplearning4j deep learning framework. It is developed in Java as well as Scala and supports other JVM languages, too. 

Widely adopted as a commercial, industry-focused distributed deep learning platform, the biggest advantage of this deep learning framework is that you can bring together the entire Java ecosystem to execute deep learning. It can also be administered on top of Hadoop and Spark to orchestrate multiple host threads. DL4J uses MapReduce to train the network while depending on other libraries to execute large matrix operations. 

Deeplearning4j comes with a deep network support through RBM, DBN, convolution neural networks (CNNs), recurrent neural networks (RNNs), recursive neural tensor networks (RNTNs), and long short-term memory (LTSM). 

Since this deep learning framework is implemented in Java, it is much more efficient compared to Python. When it comes to image recognition tasks using multiple GPUs, it is as fast as Caffe. This framework shows matchless potential for image recognition, fraud detection, text mining, parts-of-speech tagging, and natural language processing. 

With Java as your core programming language, you should certainly opt for this deep learning framework if you’re looking for a robust and effective method of deploying your deep learning models to production. 

9. FANN 

Fast Artificial Neural Network Library is a free open source neural network library, which implements multilayer artificial neural networks in C with support for both fully connected and sparsely connected networks. Cross-platform execution in both fixed and floating point are supported. It includes a framework for easy handling of training data sets. It is easy to use, versatile, well documented, and fast. Bindings to more than 20 programming languages are available. An easy to read introduction article and a reference manual accompanies the library with examples and recommendations on how to use the library. Several graphical user interfaces are also available for the library. 

FANN Features: 

• Multilayer Artificial Neural Network Library in C 
• Backpropagation training (RPROP, Quickprop, Batch, Incremental) 
• Evolving topology training which dynamically builds and trains the ANN (Cascade2) 
• Easy to use (create, train and run an ANN with just three function calls) 

• Fast (up to 150 times faster execution than other libraries) 
• Versatile (possible to adjust many parameters and features on-the-fly) 
• Well documented (An easy to read introduction article, a thorough reference manual, and a 50+ page university report describing the implementation considerations etc.) 
• Cross-platform (configure script for linux and unix, dll files for windows, project files for MSVC++ and Borland compilers are also reported to work) 
• Several different activation functions implemented (including stepwise linear functions for that extra bit of speed) 

• Easy to save and load entire ANNs 
• Several easy to use examples 
• Can use both floating point and fixed point numbers (actually both float, double and int are available) 
• Cache optimized (for that extra bit of speed) 
• Open source, but can still be used in commercial applications (licenced under LGPL) 

• Framework for easy handling of training data sets 
• Graphical Interfaces 
• Language Bindings to a large number of different programming languages 
• Widely used (approximately 100 downloads a day) 

10. OpenCog 

OpenCog is a unique and ambitious open-source software project. Our aim is to create an open source framework for Artificial General Intelligence, intended to one day express general intelligence at the human level and beyond. 

That is: We're undertaking a serious effort to build a thinking machine. We've developed a detailed plan (subject to ongoing revision based on what we learn as we proceed!), possess the ability to execute it, and we're proceeding with the hard work step-by-step. 

More specifically: Our long-term goal is the creation of an advanced Artificial General Intelligence system, with cognitive capability at the human level and (we hope) ultimately beyond. In the short term, as well as working toward our long-term R&D goal, we aim to create code that can be used (piece by piece or as a whole) to help make practical software applications smarter 

Much of the work on the project is inspired by the AGI theories of project co-founder Dr. Ben Goertzel. But there is a lot of other stuff in the codebase too, and the community is generally welcoming to new ideas. 

OpenCog is a diverse assemblage of cognitive algorithms, each embodying their own innovations — but what makes the overall architecture powerful is its careful adherence to the principle of cognitive synergy. 

The human brain consists of a host of subsystems carrying out particular tasks — some more specialized, some more general in nature — and connected together in a manner enabling them to (usually) synergetically assist rather than work against each other. 

The OpenCog design aims to capture the spirit of the brain’s architecture and dynamics without imitating the details (which are largely unknown), via: 

integrating together a carefully selected combination of cognitive algorithms acting on different kinds of knowledge in a scalable, robust and flexible C++ software architecture in a manner specifically designed: to cooperate with “cognitive synergy” for the scope of tasks characteristic of human intelligence 

to give rise to the emergence of an effectively functioning knowledge network in the AI system’s mind, as it interacts with the world, including a self-updating hierarchical/heterarchical ontology and models of itself and others 

The detailed argument as to why we think our selection of cognitive algorithms and our integration methods will have the desired effect, is sketched on the OpenCog wiki site and various previously-published conference papers and has been presented more thoroughly (from a more particular perspective) in the 2014 books Engineering General Intelligence vol. 1 and 2, linked from the wiki site. 

In order to put this architecture to work, we have crafted a roadmap based on roughly mimicking the environment and development of young human children. A series of child-level learning tasks has been carefully laid out, which may be manifested via either virtual world agents or physical robots, and which lead from infant-level capabilities up to the grade school level. These tasks cover all the major cognitive capabilities displayed by young humans, and involve the integration of all major aspects of human intelligence, including perception, action, cognition, learning, memory, creativity, socialization, language, self-modeling, etc. 

11. H2O 

H2O is a fully open source, distributed in-memory machine learning platform with linear scalability. H2O’s supports the most widely used statistical & machine learning algorithms including gradient boosted machines, generalized linear models, deep learning and more. H2O also has an industry leading AutoML functionality that automatically runs through all the algorithms and their hyperparameters to produce a leaderboard of the best models. The H2O platform is used by over 14,000 organizations globally and is extremely popular in both the R & Python communities. 

• Algorithms developed from the ground up for distributed computing and for both supervised and unsupervised approaches including Random Forest, GLM, GBM, XGBoost, GLRM, Word2Vec and many more. 
• Use the programing language you already know like R, Python and others to build models in H2O, or use H2O Flow, a graphical notebook based interactive user interface that does not require any coding. 
• H2O’s AutoML can be used for automating the machine learning workflow, which includes automatic training and tuning of many models within a user-specified time limit. Stacked Ensembles will be automatically trained on collections of individual models to produce highly predictive ensemble models which, in most cases, will be the top performing models in the AutoML Leaderboard. 
• In-memory processing with fast serialization between nodes and clusters to support massive datasets. Distributed processing on big data delivers speeds up to 100x faster with fine-grain parallelism, enabling optimal efficiency without introducing degradation in computational accuracy. 

• Easy to deploy POJOs and MOJOs to deploy models for fast and accurate scoring in any environment, including very large models.