Concepts

Concepts

PAC-1

PAC-1 is our first-gen program synthesis engine. Standing for Plastic Auto-Constructor (reflecting the flexible construction of concept libraries), it is the first prototype of a natural language interpreter that converts the symbols in which humans think (natural language) to symbols in which machines think (programs). It can learn & remember how to generate new custom programs of arbitrary length - from as few as 2 sample demonstrations.

Underneath, PAC-1 combines the iterative power of symbolic, probabilistic program search with the semantic understanding of custom language models trained on code. Today, the entire stack needs to run on our servers, but is designed from the ground-up to fit on a consumer GPU.

This brings to real-world tasks the learnings from Flatland (opens in a new tab), a toy environment, DSL & benchmark we built in early 2022, for synthesising functional programs to draw 2D line images. Progress on Flatland draws from recent breakthroughs in symbolic AI systems , which have resulted in powerful reasoning for visual[1] (opens in a new tab), spatial[2] (opens in a new tab), action[3] (opens in a new tab), and theoretical[4] (opens in a new tab), gameplay[5] (opens in a new tab) understanding, approximating functions from just a few samples.

Script

A natural language script is a self-contained list of instructions (in string form) to the machine written in natural language.

1. set {{payload}} to 29
2. add 1 to {{payload}}
3. print {{payload}}
4. if {{payload}} is less than 36
    - 4.1. go to step 3
5. else if {{payload}} is more than 36
    - 5.1. print {{payload}}

PAC-1 is designed from ground-up to interpret these natural language scripts and turn them into discrete programs, with a focus on :

  1. Interpretability : Machine-generated text and art allow for creative output and can be quickly checked for accuracy, but code requires precision and debugging skills. Providing a high-level list of steps in plain English allows anyone to read and understand the code before and after deployment. The program itself is a description of its behaviour.

  2. Composability : Condensing complex behaviour into a brief list of steps makes it possible to hold and move around the natural language program in one's head. This enables easier planning and task breakdown for both humans and machines, making complex tasks more tractable.

  3. Portability : Compressing software to a list of steps in English makes it easy to package and move it around, copy-pasting it to others over a text message, storing it in a note taking app, viewing it in a terminal, in a chatbot response, or version controlling it in a repository.

Function

A Function is a standalone object that encapsulates the script, the program graph generated from the script, auto-installed dependencies and dedicated compute required to run it. Functions have an unique property name, and can be reused using that.

main.py
function = mayalabs.Function(name="UniqueName")
function.update(script=script)
function.call({...})

Note : The Function name is the same as the name of the Worker compute runtime associated with it.

Program Graph

PAC-1 interprets a script and assembles an interpretable control flow graph, decision tree, like in the image below. The arrows here demonstrate the flow of control through an interpretable syntax tree. The program graph is linked to the script that is used to assemble it, so changes in the natural language script are also reflected in the deployed program graph.

Hello

Message Passing

In the program graph, a JSON object called msg is passed between nodes. Each node modifies this object, and passes it to the next node. Since steps are made up of nodes stitched together, the system as a whole behaves like a data-processing factory - networks of reusable components running "black-box" functions, which communicate via the msg object travelling across defined connections.

The most commonly used field in the msg object is msg.payload.

Hello

Worker

A Worker is a unit of ephemereal compute on our on-demand compute cluster, that handles execution and dependency installation for the program graph and exposes a visual flow-based editor to view/modify it. A program graph once generated is installed on the worker.

Today, each Function is associated with one Worker, mapped by name.

Module

Modules are open-source npm packages that package nodes of related, resuable blocks of Javascript code, that can be assembled. For instance, the browser (opens in a new tab) module exposes the node types like these for interacting with a browser via puppeteer and includes documentation for setting each up :

  • puppeteer-browser-launch
  • puppeteer-browser-newPage
  • puppeteer-page-goto
  • puppeteer-page-screenshot
  • puppeteer-page-content
  • puppeteer-page-click
  • puppeteer-page-keyboard-type
  • puppeteer-page-keyboard-press
  • puppeteer-page-querySelector

Hello

Check out the Maya store here (opens in a new tab) for a list of modules that can be used.

TutorialExamples