Principles of the Neopulse® Modeling Language

NeoPulse® AI Studio uses the NeoPulse® Modeling Language (NML) to abstract away a great deal of the complexity that goes into building AI models. NML is an AI/deep learning domain specific language that allows both machine learning experts and non-experts to efficiently and effectively build AI models.

The design goal behind NML was to create a scripting language that allows the user to first describe the machine learning task at a very high level, and second, to allow the user to exploit any prior machine learning knowledge they may have when building an AI model to solve that task. This flexible framework allows NML to help both developers without any prior machine learning experience as well as machine learning experts create state of the art AI models with a fraction of the effort.

The language comes with a standard runtime environment that removes the headache of saving models, and creating memory efficient APIs. The language also has tight integration with Google's groundbreaking Tensorboard neural network diagnostic tool.

To make NML easy to read, the compiler is completely agnostic to whitespace.

AI oracle assisted compiler

The NML compiler contains an AI oracle with a knowledge base of architectures and hyper-parameters for the most common machine learning tasks that people who are not AI experts can exploit to quickly build AI models to solve real-world business problems.

There are two specific ways to talk to the oracle:

  1. [Indirect Hints](NML-Oracle-indirect.md): Gives the oracle high level hints about the type and complexity of the problem.
  2. [Direct Hints](NML-Oracle-direct.md): Using the auto keyword, the user can narrow down the hyper-parameter space that the oracle will search.

Basic Syntax

Types:

  • Singular Data Types:
    • Int: 5, 10, -5
    • Float: 1.0, -1.2, 1E-5
    • String: "example string",'another example'
    • Boolean: True or False
  • Lists:
    • List: [5, 1.0, "example", [0, 0]]
      • The language does not force a list to contain the same types.
  • Method/Layer Calls:
    • The language does not define a way to form generic methods. Neural Layers are treated as first class functions and therefore are called the same way.
    • Method calls take form of <method name>: <argument list>.

      For example, to call a convolution layer:

      Conv2D: [nb_filter=32, nb_row=10, nb_col=10]

Constructs in an NML script.

An NML script contains three constructs that define everything necessary to construct and train an AI model.

Constructs are declared with the syntax: construct: block1, block2, ...

The required constructs are:

  • source: Identifies the data source(s) and any transformations to be applied to the data prior to training. The source construct is described in detail here.

  • architecture: Defines the shape of the input and the output, and constructs the complete model as a neuralflow from the input, through a network of layers, to the output. The architecture construct is described in detail here.

  • train: Defines functions and parameters necessary to compile and train the model. The train construct is described in detail here.

NEW IN 2.0

  • train_opt: Defines functions and parameters necessary to compile and train the model using the spectral optimization algorithm. The train_opt construct is described in detail here. NOTE: When using the train_opt construct, it is not necessary to use the source construct.

  • multi-GPU training: It is now possible to train models using multiple GPUs (if present on the machine running AI Studio). This functionality is enabled in the train or train_opt constructs.

To use the standard training algorithm with 2 GPUs:

train Ngpu 2:

To use the spectral optimization algorithm with 8 GPUs:

train_opt Ngpu 8:

NOTE: These constructs should appear in order in an NML script The compiler will throw an error if you try to train an architecture you haven't defined yet.

NOTE: Multiple block constructs can be defined in one NML script.

Block constructs may also be defined in separate NML scripts, and included using the import keyword:

import '/DM-Dash/path/to/my/library/my_block.nml'

In this way, machine learning experts can develop libraries of layer blocks that can be reused in complex architectures.

NEW in 2.0

  • function: Allows the user to define a function. The function construct is described in detail here. Multiple functions can be defined in the same NML script.

Variables

Variables in NML can be defined outside of any construct. Like any sensible language, variables must be declared before they are used.

dictionary_size = 20000

Example script

Here is an example script that can be used for the IMDB Sentiment task.

oracle("mode") = "classification"
oracle("hyperalgo") = "random"
oracle("generated") = 2
oracle("generated_strict") = True

dictionary_size = 20000

source:
    bind = "/DM-Dash/my_files/sentiment/data.csv" ;
    input:
        x ~ from "Review"
            -> text: [200]
            -> TextDataGenerator: [nb_words = dictionary_size] ;
    output:
        y ~ from "Label" -> flat: [1] -> FlatDataGenerator: [] ;
    params: ;

architecture:
    input:  x ~ text: [200] ;
    output: y ~ flat: [1] ;

    x -> auto -> y ;


train:
    compile:
        optimizer = auto,
        loss = 'binary_crossentropy',
        metrics = ['accuracy'] ;
    run:
        epochs = 8 ;

    dashboard: ;