Deploying Chained OpenAI LLM Calls to Vercel with the Inngest SDK

Large-language modals (LLMs) like OpenAI GPT-4 are revolutionizing many sectors due to their ability to comprehend and generate human-like text. They are pre-trained on vast amounts of data, allowing them to generate relevant responses to any given prompt.

However, for more complex applications, single responses from an LLM might not suffice. Chaining, or executing multiple LLM calls in a series or parallel, can offer refined results.

Here are some of the benefits of chaining LLMs:

• Maintains contextual continuity in conversations.
• Allows for elaboration and expansion on a topic.
• Facilitates multi-turn interactions for a dynamic conversation.
• Aids in problem solving and planning by breaking down tasks.
• Enables error correction and feedback to improve model responses.

Inngest offers a solution to chain LLMs efficiently without infrastructure overhead. It provides:

• Automatic retries and state preservation
• Optional parallelization for performance
• Built-in observability and transparency
• Function cancellation capability

Here's an example:

import { Inngest } from "inngest";

const inngest = new Inngest({ name: "Your Application Name" });

export const chain = inngest.createFunction(
  { name: "Function Name" },
  { event: "api/function.endpoint" },
  async ({ event, step }) => {
    const llm = new OpenAI();
    
    const task1 = await step.run("Task 1 Name", async () => {
      const prompt = "...";
      return await llm.createCompletion({
        model: "...",
        prompt: `${prompt}: ${event.data.input}`,
      });
    });

    const task2 = await step.run("Task 2 Name", async () => {
      const prompt = "...";
      return await llm.createCompletion({
        model: "...",
        prompt: `${prompt}: ${task1}`,
      });
    });

    await step.run("Save to DB", async () => {
      await db.collection.create({ task1, task2, requestID: event.data.requestID });
    });

    return { task1, task2 };
  },
)

In this example, we define a chained function that is triggered by a specific event.

It consists of two tasks – each generating a completion using OpenAI's language model based on a given prompt and input.

The results of these tasks are then saved to a database.

Functions created with Inngest can be served via HTTP and hosted on any platform, such as Vercel, a popular deployment platform for front-end and serverless functions.

For example, here's the code above as used in an API endpoint on Vercel:

import { serve } from "inngest/next";

const inngest = new Inngest({ name: "Your Application Name" });

export const chain = inngest.createFunction(
   ...
) // chain code from above

export const handler = serve(inngest, [chain]);

You can then trigger this endpoint by sending a HTTP POST request:

await inngest.send({
  name: "api/function.endpoint",
  data: {
    requestID: "...",
    input: "...",
  }
});

For more information on chained LLMs, you can refer to the following resources:

• Running chained LLMs with TypeScript in production
• Inngest GitHub repository