Back to Knowledge
New

AI SDK + AI Gateway

You want a chatbot, a summarizer, or an "ask the docs" box in your Next.js app. The AI SDK

AI SDK

Vercel's open-source TypeScript toolkit for building AI features: `generateText` and `streamText`, tools, structured output, agents (`ToolLoopAgent`), and React hooks like `useChat`. It works with many providers through one API. The current major version is AI SDK 7, which needs Node 22+ and ESM.

"Like a universal power adapter for AI models. Same plug in your code, whichever provider is on the other end."

gives you one TypeScript API for every model, and the AI Gateway

AI Gateway

A single endpoint that sits between your app and many AI providers, handling keys, routing, fallbacks, budgets, and usage tracking. With Vercel AI Gateway you pass a plain `"provider/model"` string like `'anthropic/claude-sonnet-5'` to the AI SDK and authenticate with `AI_GATEWAY_API_KEY` (or OIDC on Vercel).

"Like a travel agent who books any airline for you. One contact, one bill, and they rebook you if a flight is cancelled."

gives you one key, automatic fallbacks, and a single place to watch spend. Here's how to wire both up in about fifteen minutes.

1What these two things actually are

Every LLM

LLM (Large Language Model)

An AI trained on massive amounts of text that can understand and generate human-like language, including code. Anthropic's Claude, OpenAI's GPT models, and Google's Gemini are all LLMs.

"Like a super-reader who's read the entire internet and can now write essays, code, and poetry on demand."

provider ships its own SDK with its own quirks: different message shapes, different streaming formats, different ways to describe tools. The AI SDK (npm package ai, currently version 7) papers over all of that. You write streamText(...) once and swap models by changing a string. The AI Gateway is Vercel's hosted proxy that sits between your code and the providers. It holds the provider keys, retries on another model when one is down, and logs every request so you can see what you're paying for.

One API

Text, streaming chat, structured output, tools, and agents with the same functions for every provider.

One key

No juggling separate Anthropic, OpenAI, and Google keys in every environment.

One dashboard

Requests, tokens, latency, and spend in one place, taggable by user and feature.

2The mental model: a universal remote and a switchboard

Think of the AI SDK as a universal TV remote: same buttons, any brand of TV. The AI Gateway is the switchboard operator behind the wall: when you press "Sonnet," it connects the call, and if that line is busy it quietly patches you through to the backup you named. Your browser never talks to a model directly. The request always goes through your own server

Server

A computer (or program) that provides data, services, or resources to other computers over a network. When you visit a website, a server sends the page to your browser.

"Like a restaurant kitchen. You (the client) order food, and the kitchen (server) prepares and delivers it to you."

code first, which is where the secrets live.

Browser: useChat()app/api/chat/route.ts: streamText()AI GatewayAnthropic / OpenAI / Google ...

Tokens stream back along the same path, so the user sees words appear as they're generated.

3Setup: install, then authenticate

Install the packages

AI SDK 7 requires Node.js

Node.js

A runtime that lets you run JavaScript outside of a web browser. It's the engine that powers most modern dev tools, including Claude Code.

"Like installing a game console. You need the console (Node) before you can play any games (run tools)."

22 or newer and is ESM only (no require()). Inside a Next.js app that just works because Next compiles your imports. If you run a standalone script, give it a .mjs extension or set "type": "module" in package.json.

Terminal

# Node 24 (Active LTS) recommended; AI SDK 7 needs Node 22 or newer
node --version

# The SDK core, React hooks for chat UIs, and zod for schemas
npm install ai @ai-sdk/react zod

Authenticate with the gateway

Two options. On Vercel, the recommended one is OIDC: Vercel injects a short-lived token automatically in deployments, and vercel env pull gives you one locally. Anywhere else, use an API key

API Key

A unique code that identifies you when using an API. It's how services know who's making requests (and who to bill).

"Like a VIP pass. It proves you're allowed in and tracks your usage."

in AI_GATEWAY_API_KEY.

Terminal / .env.local

# Option A (on Vercel): link the project and pull a short-lived OIDC token
vercel link
vercel env pull          # writes .env.local, including VERCEL_OIDC_TOKEN

# Option B (anywhere): create an AI Gateway API key in the Vercel dashboard,
# then put it in .env.local (never commit this file)
AI_GATEWAY_API_KEY=your_key_here

Local OIDC tokens expire, so if requests suddenly return auth errors after a day, run vercel env pull again. More on this in the environment variables guide.

Bonus: route your coding agent through it too

Terminal

# Point a coding agent at AI Gateway (walks you through keys and config)
npx vercel@latest ai-gateway setup

4Build a streaming chat in two files

The server half is a App Router

App Router

Next.js's routing system based on the /app directory (the default since Next.js 13). Folders become routes, layouts nest, and components are Server Components by default. In Next.js 16, `params`, `searchParams`, `cookies()`, and `headers()` must all be awaited.

"Like a GPS that automatically knows every street in your app. Create a folder = create a route."

route handler. It receives the chat history, calls streamText, and streams the answer back. Note instructions: in AI SDK 7 the system prompt

System Prompt

Instructions that set the model's role, rules, and tone for a whole conversation, separate from what the user types. In AI SDK 7 it goes in a top-level `instructions` field; CLAUDE.md plays a similar role for Claude Code. Assume users can eventually extract it, so never put secrets there.

"Like the briefing an actor gets before improv: who you are, what the scene is, what's off-limits."

goes there, and system-role messages inside messages are rejected by default. That's a small security win: a user can't smuggle in a fake "system" message.

app/api/chat/route.ts

import {
  convertToModelMessages,
  createUIMessageStreamResponse,
  streamText,
  toUIMessageStream,
  type UIMessage,
} from 'ai';

// Give slow answers room to finish streaming (seconds).
export const maxDuration = 60;

export async function POST(req: Request) {
  const { messages }: { messages: UIMessage[] } = await req.json();

  const result = streamText({
    model: 'anthropic/claude-sonnet-5', // "provider/model" string = routed via AI Gateway
    instructions: 'You are a friendly support assistant for Taco Tracker. Keep answers short.',
    messages: await convertToModelMessages(messages),
  });

  return createUIMessageStreamResponse({
    stream: toUIMessageStream({ stream: result.stream }),
  });
}

The client half is a Client Component

Client Component

A React component that runs in the browser. Required for interactivity (useState, onClick, useEffect). Add "use client" at the top of the file.

"Like the dining room. The customer sees it, interacts with it, and clicks the buttons."

using the useChat hook from @ai-sdk/react. Messages arrive as a list of parts (text, tool calls, reasoning, files), so you render each part by type.

app/chat/page.tsx

'use client';

import { useChat } from '@ai-sdk/react';
import { DefaultChatTransport } from 'ai';
import { useState } from 'react';

export default function ChatPage() {
  const [input, setInput] = useState('');
  const { messages, sendMessage, status } = useChat({
    transport: new DefaultChatTransport({ api: '/api/chat' }),
  });

  return (
    <main>
      {messages.map((message) => (
        <div key={message.id}>
          <strong>{message.role === 'user' ? 'You' : 'AI'}:</strong>{' '}
          {message.parts.map((part, i) =>
            part.type === 'text' ? <span key={i}>{part.text}</span> : null,
          )}
        </div>
      ))}

      <form
        onSubmit={(e) => {
          e.preventDefault();
          if (!input.trim()) return;
          sendMessage({ text: input });
          setInput('');
        }}
      >
        <input value={input} onChange={(e) => setInput(e.target.value)} />
        <button type="submit" disabled={status !== 'ready'}>
          Send
        </button>
      </form>
    </main>
  );
}

Run npm run dev, open /chat, and you should see tokens stream in. Want a different model? Change one string in the route. Nothing else moves.

5Model strings, fallbacks, and watching spend

When you pass a plain "provider/model" string as model, the AI SDK sends it through the AI Gateway. Watch out for one gotcha: gateway slugs use dots in version numbers, while Anthropic's own API IDs use dashes. Copy the wrong one and you get a "model not found" error.

ModelAI Gateway stringDirect Anthropic API ID
Opus 5.5anthropic/claude-opus-5.5claude-opus-5-5
Sonnet 5anthropic/claude-sonnet-5claude-sonnet-5
Fable 5.1anthropic/claude-fable-5.1claude-fable-5-1
Haiku 4.5anthropic/claude-haiku-4.5claude-haiku-4-5
OpenAI GPT-6 Astraopenai/gpt-6-astra(use OpenAI's own ID)

The live list of gateway models is at ai-gateway.vercel.sh/v1/models. Check it before hard-coding a new model.

Fallbacks and spend attribution

Gateway-specific options go under providerOptions.gateway. models is your backup list, tried in order if the primary fails. user and tags show up in usage reports, so you can answer "which feature is eating the budget?" without guessing.

lib/summarize.ts

import { generateText } from 'ai';
import type { GatewayProviderOptions } from '@ai-sdk/gateway';

const userId = 'user_123';
const { text } = await generateText({
  model: 'anthropic/claude-sonnet-5',
  prompt: 'Summarize this support ticket in one sentence: ...',
  providerOptions: {
    gateway: {
      models: ['openai/gpt-6-astra', 'anthropic/claude-haiku-4.5'], // tried in order if the primary fails
      user: userId,              // attribute spend to one of your users
      tags: ['support-summary'], // filter usage reports by feature
    } satisfies GatewayProviderOptions,
  },
});
console.log(text);

Budgets and observability: the AI Gateway section of your Vercel dashboard shows requests, token counts, and spend per model, and lets you set spending limits. Set one on day one, before a runaway loop sets it for you. The cost guide covers the rest of the money side.

6The alternative: talk to one provider directly

You don't have to use the gateway. Each provider has its own AI SDK package (for Claude: @ai-sdk/anthropic) that calls the provider with your own key. Same generateText and streamText, just a model object instead of a string. Note the model ID switches to the dashed Anthropic API form.

Terminal / .env.local

npm install @ai-sdk/anthropic
# .env.local
ANTHROPIC_API_KEY=sk-ant-...

scripts/haiku.ts

import { anthropic } from '@ai-sdk/anthropic';
import { generateText } from 'ai';

// Reads ANTHROPIC_API_KEY from the environment. No gateway involved.
const { text } = await generateText({
  model: anthropic('claude-sonnet-5'), // Anthropic API ID (dashes, not dots)
  prompt: 'Write a haiku about deploy previews.',
});
console.log(text);

Direct is a good fit when you already have negotiated pricing or credits with one provider, need a provider-only feature on day one, or can't send traffic through a third party. You give up automatic fallbacks and the unified dashboard.

7Common traps

Calling the model from the browser

Anything in a Client Component ships to users, keys included. Always call models from a route handler or Server Action.

Mixing up dots and dashes

anthropic/claude-opus-5.5 (gateway) vs claude-opus-5-5 (Anthropic API). Wrong format = model not found.

Old AI SDK tutorials

Code from v4/v5 blog posts uses renamed APIs (stepCountIs, generateObject, system in messages). Check ai-sdk.dev for v7.

Using require() or Node 20

AI SDK 7 is ESM only and needs Node 22+. Node 20 is end-of-life; use Node 24 locally and on Vercel.

No timeout headroom

Long answers can outlive a short function timeout. Set maxDuration on the route; for truly long jobs, see durable agents.

Committing .env.local

Keep AI_GATEWAY_API_KEY out of Git. Add it in the Vercel dashboard for each environment instead.

8When to use which

AI SDK + Gateway is the default when

  • You deploy on Vercel and want the least setup
  • You might switch models, or want a backup model
  • You want per-user and per-feature spend tracking
  • You're building chat UIs, tools, or agents in TypeScript

Consider something else when

  • You need a brand-new provider feature before the SDK supports it: use the provider's own SDK (e.g. @anthropic-ai/sdk)
  • Your backend is Python: use the provider SDK or the gateway's compatible API
  • Compliance rules forbid a proxy in the path: use direct provider packages

Official docs: ai-sdk.dev and vercel.com/docs/ai-gateway

Chat works. Now make it useful.

Next, get typed JSON out of the model and let it call your own functions. Then lock it down before real users arrive.