Open-sourcing AIConfig

A config-driven, source-control-friendly AI application development framework

AIConfig is a framework that makes it easy to build generative AI applications for production. It manages generative AI prompts, models and model parameters as JSON-serializable configs that can be version controlled, evaluated, monitored and opened in a notebook playground for rapid prototyping.

It allows you to store and iterate on generative AI behavior separately from your application code, offering a streamlined AI development workflow.

Today, prompts, model parameters and model-specific logic are all jumbled in with application code. This tight coupling causes several problems:

• Increased complexity — having to wrangle different model providers in core application code.

• Difficult to experiment and iterate on prompts or switch from one model to another.

• Hard to version and evaluate prompt/model performance.

AIConfig helps unwind this complexity by capturing the generative AI bits of your application as an AI artifact, which you can:

• Version control for confidence in production

• Evaluate and monitor

• Experiment and productionize in the same workflow — open the AIConfig in a playground, switch models, try different parameters, and when done, save the result back as an aiconfig

• Simplify application code — simply call config.run to invoke a prompt, no matter which model or modality.

AIConfig is open source, and we welcome contributions:
https://github.com/lastmile-ai/aiconfig

Let’s quickly demonstrate what AIConfig looks like, and how to use it with your application. We’ll return to this example later, and you can find the code for it here: https://github.com/lastmile-ai/aiconfig/tree/main/cookbooks/Getting-Started.

Let's use ChatGPT to plan a trip to New York.

{
  "name": "NYC Trip Planner",
  "description": "Intrepid explorer with ChatGPT and AIConfig",
  "schema_version": "latest",
  "metadata": {
    "models": {
      "gpt-3.5-turbo": {
        "model": "gpt-3.5-turbo",
        "top_p": 1,
        "temperature": 1
      },
      "gpt-4": {
        "model": "gpt-4",
        "max_tokens": 3000,
        "system_prompt": "You are an expert travel coordinator with exquisite taste."
      }
    },
    "default_model": "gpt-3.5-turbo"
  },
  "prompts": [
    {
      "name": "get_activities",
      "input": "Tell me 10 fun attractions to do in NYC."
    },
    {
      "name": "gen_itinerary",
      "input": "Generate an itinerary ordered by {{order_by}} for these activities: {{get_activities.output}}.",
      "metadata": {
        "model": "gpt-4",
        "parameters": {
          "order_by": "geographic location"
        }
      }
    }
  ]
}

This AIConfig travel.aiconfig.json contains a prompt chain to get a list of NYC travel activities from an LLM (GPT-3.5), and then generate an itinerary based on user preferences (using GPT-4).

It’s ok if you don’t understand every part of the AIConfig. You can use a notebook editor, called an AI Workbook, to create it visually:

Try it yourself by cloning this workbook: https://lastmileai.dev/workbooks/clooqs3p200kkpe53u6n2rhr9?source=post_page-----6fbab53a59f7--------------------------------

Finally, let’s use this AIConfig in our application. We only show Python here, but AIConfig ships with both Node.js and Python SDKs.

pip3 install python-aiconfig

import asyncio
from aiconfig import AIConfigRuntime, InferenceOptions

async def main():
  # Load the aiconfig
  config = AIConfigRuntime.load('travel.aiconfig.json')

  # Run a single prompt (with streaming)
  inference_options = InferenceOptions(stream=True)
  await config.run("get_activities", options=inference_options)

asyncio.run(main())

That’s it! Just reference the prompt you want to run, and use the same config.run API no matter what model provider is being used underneath.

We’ll return to this example in a moment to demonstrate how to pass data into a config, and chaining prompts and models together.

Unlike traditional predictive ML development undertaken largely by ML researchers, generative AI applications are being developed collaboratively by software engineers and product teams.

Separating prompt management from application development leads to a few powerful consequences:

1. Separation of concerns: You can iterate on prompts and models separately from application code — and different people could be responsible for them, making the overall development more collaborative.

2. Notebook editor for prompts: Having prompts and models in one place allows a notebook-like editor environment to iterate on the aiconfig. This greatly increases the velocity of prototyping and iteration.

3. Governance: As a source-controlled artifact, aiconfig can be used for reproducibility and provenance of the generative AI bits of your application.

We’ve specifically designed AIConfig for software developers and teams building production applications of generative AI.

AIConfig is multi-modal and model-agnostic. This enables powerful interop between different models and modalities, including chaining them together (see prompt chaining). For example, a LLaMA2 (text-to-text) prompt can be connected to a DALL-E prompt (text-to-image) to build complex AI applications, all backed by the same aiconfig serialization format.

AIConfig has 3 core pillars:

1. aiconfig file format: a standardized JSON format to store generative AI model settings, prompt inputs and outputs, and flexible multi-purpose metadata.

2. AIConfig SDK: Python and Node SDKs to use aiconfig in your application code, and extension points to customize behavior.

3. AI Workbook editor: A notebook-like playground to edit aiconfig files visually, run prompts, tweak models and model settings, and chain things together.

Extensibility is a core design principle for the framework, so anyone — including you — can create ModelParser extensions which customize which models to run, how to serialize/deserialize model-specific data into/from an aiconfig, and connect to your custom model deployments.

Let’s continue from the above example, but this time, instead of just getting a list of activities, let’s ask GPT-4 to create an itinerary for us, ordered by some criteria.

In our travel.aiconfig.json, we have a gen_itinerary prompt

{
  "name": "gen_itinerary",
  "input": "Generate an itinerary ordered by {{order_by}} for these activities: {{get_activities.output}}.",
  "metadata": {
    "model": "gpt-4",
      "parameters": {
        "order_by": "geographic location"
      }
  }
}

Observe the following:

1. The prompt depends on the output of the get_activities prompt.

2. It also depends on an order_by parameter

3. It uses GPT-4, whereas the get_activities prompt it depends on uses GPT-3.5.

Effectively, this is a prompt chain between gen_itinerary and get_activities prompts, as well as as a model chain between gpt-3.5 and gpt-4.

Let’s run this with AIConfig:

import asyncio
from aiconfig import AIConfigRuntime, InferenceOptions

async def main():
  # Load the aiconfig
  config = AIConfigRuntime.load('travel.aiconfig.json')

  # Run a single prompt (with streaming)
  inference_options = InferenceOptions(stream=True)
  await config.run(
    "gen_itinerary",
    params={"order_by": "duration"},
    options=inference_options,
    run_with_dependencies=True)

asyncio.run(main())

Notice how simple the syntax is to perform a fairly complex task — running 2 different prompts across 2 different models and chaining one’s output as part of the input of another — all with a single config.run call! We even passed data into the prompt with the params syntax.

Finally, you can use AIConfig to monitor your generative AI usage in production. We have defined callback handlers to allow you to connect to your monitoring & observability platform:

from aiconfig import CallbackManager, CallbackEvent
import pprint

async def custom_callback(event: CallbackEvent) -> None:
  """
    This is a custom callback that prints the event to stdout.

    Args:
        event (CallbackEvent): The event that triggered the callback.
  """
  print(f"Event triggered: {event.name}")
  pprint.pprint(event, width = 150)

callback_manager = CallbackManager([custom_callback])
config.set_callback_manager(callback_manager)

Over the last several months, we’ve been working with several enterprises who have voiced the same issue — they would love to build with generative AI, but they need confidence in production and a developer workflow that is integrated into their existing software engineering workflow.

We believe that we need a more methodical way of building generative AI applications, that inspires confidence in production, makes it easy to iterate, evaluate performance, and collaborate as a team.

For any such standard to succeed, it needs to be open source, and needs the developer community’s input. This is our first open source release, and we’d love the community’s feedback on the project, input on the direction, and also contributions to help drive this forward.

Specifically, you can help with:

1. Extension Contributions: We always welcome code contributions to the library. In particular, model-parsers, extensions, and cookbooks are areas that can always use contributions.

2. Cookbooks: Showcase innovative ways and use cases to use AIConfig in your code through a cookbook.

There is a lot you can do with AIConfig. We have several tutorials to help you get started:

• CLI Chatbot

• RAG with AIConfig

• Prompt routing

• OpenAI function calling

• Chain of thought

The travel.aiconfig.json showed how to use OpenAI models, but AIConfig already supports LLaMA2, Google PaLM and Hugging Face text generation. We’ll continue to add additional model support.

In addition, there’s quite a lot that we are excited to build in the coming weeks.

• Evaluation interfaces: allow aiconfig artifacts to be evaluated with user-defined eval functions.

• Local editor for aiconfig, so you can iterate on aiconfigs in a locally hosted playground.

• OpenAI Assistants API support

• Multi-modal model support (such as GPT4-V, DALL-E, Whisper and Hugging Face models)

Get started at https://github.com/lastmile-ai/aiconfig, and happy coding!