AI providers can be slow when producing responses, but many make their responses available in chunks as they're processed. Streaming enables you to show users those chunks of data as they arrive rather than waiting for the full response, improving the perceived speed of AI-powered apps.
Vercel recommends using Vercel's AI SDK to stream responses from LLMs and AI APIs. It reduces the boilerplate necessary for streaming responses from AI providers and allows you to change AI providers with a few lines of code, rather than rewriting your entire application.
This example demonstrates a function that sends a message to one of Open AI's GPT models and streams the response:
Before you begin, ensure you're using Node.js 18 or later.
- Install the
ai
and@ai-sdk/openai
packages:
pnpm install ai openai
-
Copy an OpenAI API key in the
.env.local
file with nameOPENAI_API_KEY
. See the AI SDK docs for more information on how to do this -
Add the following code to your example
import { streamText } from 'ai';
import { openai } from '@ai-sdk/openai';
// This method must be named GET
export async function GET() {
// Make a request to OpenAI's API based on
// a placeholder prompt
const response = await streamText({
model: openai('gpt-4o-mini'),
messages: [{ role: 'user', content: 'Say this is a test.' }],
});
// Respond with the stream
return response.toTextStreamResponse({
headers: {
'Content-Type': 'text/event-stream',
},
});
}
- Build your app and visit
localhost:3000/api/chat-example
. You should see the text"This is a test."
in the browser.