# Create a message
Source: https://docs.pipellm.ai/api-reference/anthropic/messages

POST /v1/messages
Create a message with the Anthropic Messages API.

Use `x-api-key: $PIPELLM_API_KEY` and `anthropic-version: 2023-06-01`. Native `/v1/messages` only routes to Anthropic-compatible platforms.

<RequestExample>
  ```bash cURL theme={"dark"}
  curl https://api.pipellm.ai/v1/messages \
    -H "Content-Type: application/json" \
    -H "x-api-key: $PIPELLM_API_KEY" \
    -H "anthropic-version: 2023-06-01" \
    -d '{
      "model": "claude-sonnet-4-6",
      "max_tokens": 1024,
      "messages": [{"role": "user", "content": "Hello"}]
    }'
  ```

  ```python Python theme={"dark"}
  import os
  import anthropic

  client = anthropic.Anthropic(
      api_key=os.environ["PIPELLM_API_KEY"],
      base_url="https://api.pipellm.ai",
  )
  message = client.messages.create(
      model="claude-sonnet-4-6",
      max_tokens=1024,
      messages=[{"role": "user", "content": "Hello"}],
  )
  print(message.content[0].text)
  ```
</RequestExample>

<ParamField type="string">
  Anthropic model ID from [`GET /v1/models`](/api-reference/list-models).
</ParamField>

<ParamField type="array">
  Conversation messages. Roles are `user` or `assistant`.

  <Expandable title="messages[]">
    <ParamField type="string">
      `user` or `assistant`.
    </ParamField>

    <ParamField type="string | array">
      Plain text, or an array of content blocks.

      <Expandable title="content blocks">
        <ParamField type="string">
          Usually `text`. Image blocks use `image`.
        </ParamField>

        <ParamField type="string">
          Required when `type` is `text`.
        </ParamField>

        <ParamField type="object">
          Required when `type` is `image`. Typically `{ "type": "base64", "media_type": "image/png", "data": "..." }` or a URL source.
        </ParamField>
      </Expandable>
    </ParamField>
  </Expandable>
</ParamField>

<ParamField type="integer">
  Maximum output tokens.
</ParamField>

<ParamField type="string | array">
  System prompt.
</ParamField>

<ParamField type="number">
  Sampling temperature, 0–1.
</ParamField>

<ParamField type="number">
  Nucleus sampling.
</ParamField>

<ParamField type="integer">
  Top-k sampling.
</ParamField>

<ParamField type="boolean">
  If `true`, return server-sent events.
</ParamField>

<ParamField type="string[]">
  Custom stop sequences.
</ParamField>

<ParamField type="array">
  Tool definitions. See [Tool Use](/api-reference/anthropic/tool-use).

  <Expandable title="tools[]">
    <ParamField type="string">
      Tool name.
    </ParamField>

    <ParamField type="string">
      When Claude should call this tool.
    </ParamField>

    <ParamField type="object">
      JSON Schema for the tool input.
    </ParamField>
  </Expandable>
</ParamField>

## Endpoint

```
POST https://api.pipellm.ai/v1/messages
```

<Note>
  This free route accepts Anthropic Messages requests and routes only to
  Anthropic-compatible platforms. To keep the Anthropic format while calling
  OpenAI or Gemini models, use the
  [Anthropic Format Converter](/converter/anthropic-format).
</Note>

## Install SDKs

<Tabs>
  <Tab title="Python">
    ```bash theme={"dark"}
    pip install anthropic
    ```
  </Tab>

  <Tab title="Node.js">
    ```bash theme={"dark"}
    npm install @anthropic-ai/sdk
    ```
  </Tab>

  <Tab title="Go">
    ```bash theme={"dark"}
    go get github.com/anthropics/anthropic-sdk-go
    ```
  </Tab>
</Tabs>

## Code Examples

<Tabs>
  <Tab title="cURL">
    ```bash theme={"dark"}
    curl https://api.pipellm.ai/v1/messages \
      -H "Content-Type: application/json" \
      -H "x-api-key: $PIPELLM_API_KEY" \
      -H "anthropic-version: 2023-06-01" \
      -d @- << 'EOF'
    {
      "model": "claude-sonnet-4-5-20250929",
      "max_tokens": 1024,
      "messages": [
        {"role": "user", "content": "Why is the sky blue?"}
      ]
    }
    EOF
    ```
  </Tab>

  <Tab title="Python">
    ```python theme={"dark"}
    import os
    import anthropic

    client = anthropic.Anthropic(
      api_key=os.getenv('PIPELLM_API_KEY'),
      base_url='https://api.pipellm.ai'
    )

    message = client.messages.create(
      model='claude-sonnet-4-5-20250929',
      max_tokens=1024,
      messages=[
        {'role': 'user', 'content': 'Why is the sky blue?'}
      ]
    )

    print(message.content[0].text)
    ```
  </Tab>

  <Tab title="Node.js">
    ```javascript theme={"dark"}
    import Anthropic from '@anthropic-ai/sdk';

    const client = new Anthropic({
      apiKey: process.env.PIPELLM_API_KEY,
      baseURL: 'https://api.pipellm.ai'
    });

    const message = await client.messages.create({
      model: 'claude-sonnet-4-5-20250929',
      max_tokens: 1024,
      messages: [
        { role: 'user', content: 'Why is the sky blue?' }
      ]
    });

    console.log(message.content[0].text);
    ```
  </Tab>

  <Tab title="Go">
    ```go theme={"dark"}
    package main

    import (
      "context"
      "os"
      "github.com/anthropics/anthropic-sdk-go"
      "github.com/anthropics/anthropic-sdk-go/option"
    )

    func main() {
      client := anthropic.NewClient(
        option.WithAPIKey(os.Getenv("PIPELLM_API_KEY")),
        option.WithBaseURL("https://api.pipellm.ai"),
      )

      message, _ := client.Messages.New(context.TODO(), anthropic.MessageNewParams{
        Model:     anthropic.F("claude-sonnet-4-5-20250929"),
        MaxTokens: anthropic.Int(1024),
        Messages: anthropic.F([]anthropic.MessageParam{
          anthropic.NewUserMessage(anthropic.NewTextBlock("Why is the sky blue?")),
        }),
      })
    }
    ```
  </Tab>
</Tabs>

## Response Format

```json theme={"dark"}
{
  "id": "msg_xxx",
  "type": "message",
  "role": "assistant",
  "content": [
    {"type": "text", "text": "The sky appears blue because..."}
  ],
  "model": "claude-sonnet-4-5-20250929",
  "stop_reason": "end_turn",
  "usage": {
    "input_tokens": 10,
    "output_tokens": 50
  }
}
```

<ResponseField name="id" type="string">
  Message ID.
</ResponseField>

<ResponseField name="type" type="string">
  Always `message`.
</ResponseField>

<ResponseField name="role" type="string">
  Always `assistant`.
</ResponseField>

<ResponseField name="content" type="array">
  Output blocks. Text replies use `{ "type": "text", "text": "..." }`. Tool calls use `tool_use` blocks.
</ResponseField>

<ResponseField name="stop_reason" type="string">
  `end_turn`, `max_tokens`, or `tool_use`.
</ResponseField>

<ResponseField name="usage" type="object">
  `input_tokens` and `output_tokens`.
</ResponseField>

## Errors

Same envelope as [Errors](/api-reference/errors). Native `/v1/messages` also returns `400` when the body is not Anthropic format.

<AccordionGroup>
  <Accordion title="400 invalid_request_error">
    Missing `model`, `messages`, or `max_tokens`, or a protocol mismatch.

    ```json theme={"dark"}
    {
      "error": {
        "type": "invalid_request_error",
        "code": "400",
        "message": "The 'model' field is required but was not provided in the request"
      }
    }
    ```
  </Accordion>

  <Accordion title="401 authentication_error">
    Missing or invalid `x-api-key`.

    ```json theme={"dark"}
    {
      "error": {
        "type": "authentication_error",
        "code": "401",
        "message": "Incorrect API key provided. Please visit https://console.pipellm.ai/account/api-keys to find your API key."
      }
    }
    ```
  </Accordion>

  <Accordion title="402 insufficient_balance">
    ```json theme={"dark"}
    {
      "error": {
        "type": "insufficient_balance",
        "code": "402",
        "message": "Insufficient balance. Please recharge your account at https://console.pipellm.ai/billing."
      }
    }
    ```
  </Accordion>

  <Accordion title="429 rate_limit_error">
    See [Rate Limits](/api-reference/rate-limits).

    ```json theme={"dark"}
    {
      "error": {
        "type": "rate_limit_error",
        "code": "429",
        "message": "Rate limit exceeded. Your current limit is 120 requests per minute (approximately 2.00 requests per second). Please visit https://console.pipellm.ai/billing to upgrade your plan for higher limits."
      }
    }
    ```
  </Accordion>
</AccordionGroup>

## Related Docs

<Columns>
  <Card title="Routing & Protocols" icon="route" href="/guides/routing-protocols">
    Native-route constraints and converter routes
  </Card>

  <Card title="Anthropic Overview" icon="message" href="/api-reference/anthropic/overview">
    Headers, models, and route family overview
  </Card>

  <Card title="Anthropic Format Converter" icon="shuffle" href="/converter/anthropic-format">
    Keep Anthropic format while calling other providers
  </Card>
</Columns>


# Tool Use
Source: https://docs.pipellm.ai/api-reference/anthropic/tool-use

Claude function calling and tool use

For authentication and model list, see [Overview](/api-reference/anthropic/overview).

## What is Tool Use

Tool Use (also called function calling) allows Claude to interact with external tools, APIs, and data sources. Claude doesn't execute tools directly—it generates structured requests that your application executes.

## How It Works

1. **Define tools** with names, descriptions, and input schemas
2. **Claude identifies** when a tool is needed
3. **Claude generates** a tool use request (JSON)
4. **Your app executes** the tool
5. **Return result** to Claude
6. **Claude responds** with the final answer

## Basic Example

<Tabs>
  <Tab title="Python">
    ```python theme={"dark"}
    import os
    import anthropic

    client = anthropic.Anthropic(
      api_key=os.getenv('PIPELLM_API_KEY'),
      base_url='https://api.pipellm.ai'
    )

    # Define tools
    tools = [
      {
        "name": "get_weather",
        "description": "Get the current weather in a location",
        "input_schema": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "City name, e.g., San Francisco"
            }
          },
          "required": ["location"]
        }
      }
    ]

    # First call - Claude decides to use the tool
    response = client.messages.create(
      model="claude-sonnet-4-5-20250929",
      max_tokens=1024,
      tools=tools,
      messages=[
        {"role": "user", "content": "What's the weather in Tokyo?"}
      ]
    )

    # Check if Claude wants to use a tool
    if response.stop_reason == "tool_use":
      tool_use = next(b for b in response.content if b.type == "tool_use")
      
      # Execute your tool (example)
      tool_result = {"temperature": "22°C", "condition": "Sunny"}
      
      # Return result to Claude
      final_response = client.messages.create(
        model="claude-sonnet-4-5-20250929",
        max_tokens=1024,
        tools=tools,
        messages=[
          {"role": "user", "content": "What's the weather in Tokyo?"},
          {"role": "assistant", "content": response.content},
          {
            "role": "user",
            "content": [
              {
                "type": "tool_result",
                "tool_use_id": tool_use.id,
                "content": str(tool_result)
              }
            ]
          }
        ]
      )
      
      print(final_response.content[0].text)
    ```
  </Tab>

  <Tab title="Node.js">
    ```javascript theme={"dark"}
    import Anthropic from '@anthropic-ai/sdk';

    const client = new Anthropic({
      apiKey: process.env.PIPELLM_API_KEY,
      baseURL: 'https://api.pipellm.ai'
    });

    const tools = [
      {
        name: "get_weather",
        description: "Get the current weather in a location",
        input_schema: {
          type: "object",
          properties: {
            location: {
              type: "string",
              description: "City name, e.g., San Francisco"
            }
          },
          required: ["location"]
        }
      }
    ];

    const response = await client.messages.create({
      model: 'claude-sonnet-4-5-20250929',
      max_tokens: 1024,
      tools: tools,
      messages: [
        { role: 'user', content: "What's the weather in Tokyo?" }
      ]
    });

    if (response.stop_reason === 'tool_use') {
      const toolUse = response.content.find(b => b.type === 'tool_use');
      
      // Execute your tool
      const toolResult = { temperature: "22°C", condition: "Sunny" };
      
      const finalResponse = await client.messages.create({
        model: 'claude-sonnet-4-5-20250929',
        max_tokens: 1024,
        tools: tools,
        messages: [
          { role: 'user', content: "What's the weather in Tokyo?" },
          { role: 'assistant', content: response.content },
          {
            role: 'user',
            content: [
              {
                type: 'tool_result',
                tool_use_id: toolUse.id,
                content: JSON.stringify(toolResult)
              }
            ]
          }
        ]
      });
      
      console.log(finalResponse.content[0].text);
    }
    ```
  </Tab>
</Tabs>

## More Resources

<Card title="Tool Use Documentation" icon="book" href="https://docs.anthropic.com/en/docs/build-with-claude/tool-use">
  Complete guide on defining tools and handling responses
</Card>


# Authentication
Source: https://docs.pipellm.ai/api-reference/authentication

API keys, headers, and base URLs for Relay.

All public Relay routes use the API key from [console.pipellm.ai](https://console.pipellm.ai). Never send provider keys; Relay authenticates your account and routes upstream.

## Default path

For OpenAI-compatible chat, use:

|          |                                          |
| -------- | ---------------------------------------- |
| Base URL | `https://api.pipellm.ai/v1`              |
| Header   | `Authorization: Bearer $PIPELLM_API_KEY` |

```bash theme={"dark"}
curl https://api.pipellm.ai/v1/chat/completions \
  -H "Authorization: Bearer $PIPELLM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-5","messages":[{"role":"user","content":"Hello"}]}'
```

Video uses the same Bearer header on `https://api.pipellm.ai` (no `/v1` prefix):

```bash theme={"dark"}
curl https://api.pipellm.ai/v2/videos \
  -H "Authorization: Bearer $PIPELLM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"seedance-2-0-mini","prompt":"A cat walking in the rain"}'
```

## Protocol headers

Native routes keep the header the corresponding SDK expects:

| Request format                      | Base URL                    | Header                                   |
| ----------------------------------- | --------------------------- | ---------------------------------------- |
| OpenAI Chat Completions / Responses | `https://api.pipellm.ai/v1` | `Authorization: Bearer $PIPELLM_API_KEY` |
| Anthropic Messages                  | `https://api.pipellm.ai`    | `x-api-key: $PIPELLM_API_KEY`            |
| Gemini Generate Content             | `https://api.pipellm.ai`    | `x-goog-api-key: $PIPELLM_API_KEY`       |
| Video (`/v2/videos`)                | `https://api.pipellm.ai`    | `Authorization: Bearer $PIPELLM_API_KEY` |

Anthropic requests also send `anthropic-version: 2023-06-01`.

## Converter routes

Use a converter base URL only when you want to keep one SDK format and call a model from another protocol family. See [Routing & Protocols](/guides/routing-protocols).

| Format                     | Base URL                              |
| -------------------------- | ------------------------------------- |
| OpenAI converter           | `https://api.pipellm.ai/openai/v1`    |
| OpenAI Responses converter | `https://api.pipellm.ai/responses/v1` |
| Anthropic converter        | `https://api.pipellm.ai/anthropic`    |
| Gemini converter           | `https://api.pipellm.ai/gemini`       |

## Failed authentication

Missing or invalid keys return `401`:

```json theme={"dark"}
{
  "error": {
    "type": "authentication_error",
    "code": "401",
    "message": "Incorrect API key provided. Please visit https://console.pipellm.ai/account/api-keys to find your API key."
  }
}
```

See [Errors](/api-reference/errors) for the full error envelope.


# Errors and Debugging
Source: https://docs.pipellm.ai/api-reference/errors

API error types, HTTP status codes, and retry strategies

All API errors follow a consistent JSON format:

```json theme={"dark"}
{
  "error": {
    "type": "error_type",
    "code": "http_status_code",
    "message": "Human-readable error message"
  }
}
```

## Error Types

| HTTP Code | Error Type              | Description                                           |
| --------- | ----------------------- | ----------------------------------------------------- |
| 400       | `invalid_request_error` | Request body or parameters are malformed              |
| 401       | `authentication_error`  | API key is missing, invalid, or expired               |
| 402       | `insufficient_balance`  | Account balance is insufficient                       |
| 403       | `permission_error`      | Account lacks access rights to the resource           |
| 404       | `not_found_error`       | Model or resource does not exist                      |
| 405       | `invalid_request_error` | Wrong HTTP method                                     |
| 429       | `rate_limit_error`      | Too many requests, please retry later                 |
| 500+      | `api_error`             | An error occurred while processing your request       |
| 529       | `overloaded_error`      | Service is temporarily overloaded, please retry later |

## Error Examples

### Authentication Error (401)

```json theme={"dark"}
{
  "error": {
    "type": "authentication_error",
    "code": "401",
    "message": "Incorrect API key provided. Please visit https://console.pipellm.ai/account/api-keys to find your API key."
  }
}
```

### Rate Limit Error (429)

When you exceed rate limits, the response includes your current limits:

```json theme={"dark"}
{
  "error": {
    "type": "rate_limit_error",
    "code": "429",
    "message": "Rate limit exceeded. Your current limit is 120 requests per minute (approximately 2.00 requests per second). Please visit https://console.pipellm.ai/billing to upgrade your plan for higher limits.",
    "rate_limit": {
      "rpm": 120,
      "qps": 2.00
    }
  }
}
```

### Insufficient Balance (402)

```json theme={"dark"}
{
  "error": {
    "type": "insufficient_balance",
    "code": "402",
    "message": "Insufficient balance. Please recharge your account at https://console.pipellm.ai/billing."
  }
}
```

### Internal Error (500)

```json theme={"dark"}
{
  "error": {
    "type": "api_error",
    "code": "500",
    "message": "Internal server error occurred. Our team has been notified and is working on it."
  }
}
```

## Model-Specific Errors

### Model Not Found

Unknown model IDs return `400`. List current IDs with [`GET /v1/models`](/api-reference/list-models).

```json theme={"dark"}
{
  "error": {
    "type": "invalid_request_error",
    "code": "400",
    "message": "not-a-model is not a valid model ID. You can view all the models in our documentation: https://docs.pipellm.ai"
  }
}
```

### Model Missing in Request

```json theme={"dark"}
{
  "error": {
    "type": "invalid_request_error",
    "code": "400",
    "message": "The 'model' field is required but was not provided in the request"
  }
}
```

## Retry

`429` and `529` are retryable. Back off and retry; `402` and `401` are not. Rate-limit windows are in the `x-ratelimit-*` response headers. See [Rate Limits](/api-reference/rate-limits).

## Support

<Columns>
  <Card title="Contact Support" icon="envelope" href="mailto:support@pipellm.ai">
    Email us for assistance with API issues.
  </Card>

  <Card title="Usage Dashboard" icon="chart-line" href="https://console.pipellm.ai">
    Monitor your API usage and manage your account.
  </Card>
</Columns>


# Generate content
Source: https://docs.pipellm.ai/api-reference/gemini/generate-content

POST /v1beta/models/{model}:generateContent
Generate content with the Gemini generateContent API.

Native Gemini path. Streaming uses [`POST /v1beta/models/{model}:streamGenerateContent`](/api-reference/gemini/stream-generate-content).

<RequestExample>
  ```bash cURL theme={"dark"}
  curl "https://api.pipellm.ai/v1beta/models/gemini-3-flash-preview:generateContent" \
    -H "Content-Type: application/json" \
    -H "x-goog-api-key: $PIPELLM_API_KEY" \
    -d '{
      "contents": [{"role": "user", "parts": [{"text": "Hello"}]}]
    }'
  ```

  ```python Python theme={"dark"}
  import os
  from google import genai

  client = genai.Client(
      api_key=os.environ["PIPELLM_API_KEY"],
      http_options={"base_url": "https://api.pipellm.ai"},
  )
  print(client.models.generate_content(
      model="gemini-3-flash-preview",
      contents="Hello",
  ).text)
  ```
</RequestExample>

<ParamField type="string">
  Gemini model ID, for example `gemini-3-flash-preview`.
</ParamField>

<ParamField type="array">
  Conversation turns.

  <Expandable title="contents[]">
    <ParamField type="string">
      `user` or `model`.
    </ParamField>

    <ParamField type="array">
      Content parts for this turn.

      <Expandable title="parts[]">
        <ParamField type="string">
          Text prompt or model text.
        </ParamField>

        <ParamField type="object">
          Inline media. Use `mime_type` (for example `image/png`) and `data` (base64). Image models return generated images here.
        </ParamField>
      </Expandable>
    </ParamField>
  </Expandable>
</ParamField>

<ParamField type="object">
  Optional sampling controls.

  <Expandable title="generationConfig fields">
    <ParamField type="number">
      Sampling temperature.
    </ParamField>

    <ParamField type="integer">
      Maximum output tokens.
    </ParamField>

    <ParamField type="number">
      Nucleus sampling.
    </ParamField>

    <ParamField type="integer">
      Top-k sampling.
    </ParamField>
  </Expandable>
</ParamField>

## Endpoint

```
POST https://api.pipellm.ai/v1beta/models/{model}:generateContent
```

<Note>
  This free route accepts Gemini native requests and routes only to Gemini
  platforms. To keep the Gemini format while calling OpenAI or Anthropic
  models, use the [Gemini Format Converter](/converter/gemini-format).
</Note>

## Install SDK

```bash theme={"dark"}
pip install google-genai
```

## Code Examples

<Tabs>
  <Tab title="cURL">
    ```bash theme={"dark"}
    curl "https://api.pipellm.ai/v1beta/models/gemini-3-flash-preview:generateContent" \
      -H "Content-Type: application/json" \
      -H "x-goog-api-key: $PIPELLM_API_KEY" \
      -d '{
        "contents": [
          {
            "role": "user",
            "parts": [
              {"text": "Why is the sky blue?"}
            ]
          }
        ]
      }'
    ```
  </Tab>

  <Tab title="Python">
    ```python theme={"dark"}
    import os
    from google import genai

    client = genai.Client(
      api_key=os.getenv('PIPELLM_API_KEY'),
      http_options={'base_url': 'https://api.pipellm.ai'}
    )

    response = client.models.generate_content(
      model='gemini-3-flash-preview',
      contents='Why is the sky blue?'
    )

    print(response.text)
    ```
  </Tab>

  <Tab title="Node.js">
    ```javascript theme={"dark"}
    import { GoogleGenAI } from '@google/genai';

    const ai = new GoogleGenAI({
      apiKey: process.env.PIPELLM_API_KEY,
      httpOptions: {
        baseUrl: 'https://api.pipellm.ai'
      }
    });

    const response = await ai.models.generateContent({
      model: 'gemini-3-flash-preview',
      contents: 'Why is the sky blue?'
    });

    console.log(response.text);
    ```
  </Tab>

  <Tab title="Go">
    ```go theme={"dark"}
    package main

    import (
      "context"
      "os"
      "github.com/google/generative-ai-go/genai"
      "google.golang.org/api/option"
    )

    func main() {
      ctx := context.Background()
      client, _ := genai.NewClient(ctx,
        option.WithAPIKey(os.Getenv("PIPELLM_API_KEY")),
        option.WithEndpoint("https://api.pipellm.ai"),
      )
      defer client.Close()

      model := client.GenerativeModel("gemini-3-flash-preview")
      resp, _ := model.GenerateContent(ctx,
        genai.Text("Why is the sky blue?"),
      )
    }
    ```
  </Tab>
</Tabs>

## Request Format

```json theme={"dark"}
{
  "contents": [
    {
      "role": "user",
      "parts": [
        {"text": "Your message here"}
      ]
    }
  ],
  "generationConfig": {
    "temperature": 0.7,
    "maxOutputTokens": 1024
  }
}
```

## Response Format

```json theme={"dark"}
{
  "candidates": [
    {
      "content": {
        "parts": [
          {"text": "The sky appears blue because..."}
        ],
        "role": "model"
      },
      "finishReason": "STOP"
    }
  ],
  "usageMetadata": {
    "promptTokenCount": 10,
    "candidatesTokenCount": 50,
    "totalTokenCount": 60
  }
}
```

<ResponseField name="candidates" type="array">
  Generated candidates.

  <Expandable title="candidates[]">
    <ResponseField name="content" type="object">
      `role` is `model`. `parts` holds `text` and/or `inline_data`.
    </ResponseField>

    <ResponseField name="finishReason" type="string">
      Typically `STOP`.
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="usageMetadata" type="object">
  `promptTokenCount`, `candidatesTokenCount`, `totalTokenCount`.
</ResponseField>

## Errors

Same envelope as [Errors](/api-reference/errors). Use `x-goog-api-key`. Native Gemini routes return `400` when the body is not Gemini format.

<AccordionGroup>
  <Accordion title="400 invalid_request_error">
    Missing `contents`, unknown model ID, or a protocol mismatch.

    ```json theme={"dark"}
    {
      "error": {
        "type": "invalid_request_error",
        "code": "400",
        "message": "The 'model' field is required but was not provided in the request"
      }
    }
    ```
  </Accordion>

  <Accordion title="401 authentication_error">
    Missing or invalid `x-goog-api-key`.

    ```json theme={"dark"}
    {
      "error": {
        "type": "authentication_error",
        "code": "401",
        "message": "Incorrect API key provided. Please visit https://console.pipellm.ai/account/api-keys to find your API key."
      }
    }
    ```
  </Accordion>

  <Accordion title="402 insufficient_balance">
    ```json theme={"dark"}
    {
      "error": {
        "type": "insufficient_balance",
        "code": "402",
        "message": "Insufficient balance. Please recharge your account at https://console.pipellm.ai/billing."
      }
    }
    ```
  </Accordion>

  <Accordion title="429 rate_limit_error">
    See [Rate Limits](/api-reference/rate-limits).

    ```json theme={"dark"}
    {
      "error": {
        "type": "rate_limit_error",
        "code": "429",
        "message": "Rate limit exceeded. Your current limit is 120 requests per minute (approximately 2.00 requests per second). Please visit https://console.pipellm.ai/billing to upgrade your plan for higher limits."
      }
    }
    ```
  </Accordion>
</AccordionGroup>

## Related Docs

<Columns>
  <Card title="Routing & Protocols" icon="route" href="/guides/routing-protocols">
    Native-route constraints and converter routes
  </Card>

  <Card title="Gemini Overview" icon="sparkles" href="/api-reference/gemini/overview">
    Headers, models, and route family overview
  </Card>

  <Card title="Gemini Format Converter" icon="shuffle" href="/converter/gemini-format">
    Keep Gemini format while calling other providers
  </Card>
</Columns>


# Stream generate content
Source: https://docs.pipellm.ai/api-reference/gemini/stream-generate-content

POST /v1beta/models/{model}:streamGenerateContent
Stream Gemini generateContent output as server-sent events.

Same request body as [Generate content](/api-reference/gemini/generate-content). The response is an SSE stream instead of a single JSON object.

<ParamField type="string">
  Gemini model ID, for example `gemini-3-flash-preview`.
</ParamField>

```bash theme={"dark"}
curl "https://api.pipellm.ai/v1beta/models/gemini-3-flash-preview:streamGenerateContent" \
  -H "Content-Type: application/json" \
  -H "x-goog-api-key: $PIPELLM_API_KEY" \
  -N \
  -d '{
    "contents": [
      {
        "role": "user",
        "parts": [{"text": "Count to five."}]
      }
    ]
  }'
```

Use `x-goog-api-key: $PIPELLM_API_KEY`. Native Gemini routes only go to Gemini platforms. To keep Gemini format while calling OpenAI or Anthropic models, use the [Gemini converter](/converter/gemini-format).


# Edit an image
Source: https://docs.pipellm.ai/api-reference/image/edits

POST /v1/images/edits
Edit an existing image, optionally masked, with GPT image models.

Edits take one or more source images and a prompt describing the change. With a mask, only the transparent region is repainted; without one, the model rewrites the whole frame guided by the references.

Two request encodings work, and PipeLLM passes both through:

* `multipart/form-data` with the image bytes uploaded directly. This is what the OpenAI SDK sends.
* `application/json` with an `images` array of `file_id` or `image_url` entries, up to 16 references.

<ParamField type="file | array">
  Source image, or several. Send file parts under `image[]` in multipart, or the JSON `images` array. PNG, JPEG, or WebP.
</ParamField>

<ParamField type="string">
  What the edited image should show. Up to 32,000 characters on GPT image models.
</ParamField>

<ParamField type="string">
  Image model ID, for example `gpt-image-2.5-sunburst`.
</ParamField>

<ParamField type="file">
  A PNG whose transparent pixels mark the region to repaint. Must match the source dimensions. Omit it to let the model edit the whole image.
</ParamField>

<ParamField type="integer">
  Number of images to return, 1–10.
</ParamField>

<ParamField type="string">
  `auto`, `1024x1024`, `1536x1024`, `1024x1536`, or a custom `WIDTHxHEIGHT` divisible by 16.
</ParamField>

<ParamField type="string">
  `low`, `medium`, `high`, or `auto`. The `gpt-image-2.5` variants add `xhigh` and `max`.
</ParamField>

<ParamField type="string">
  `high` preserves faces, logos, and fine texture from the source more closely, at a higher input-token cost. `low` gives the model more freedom.
</ParamField>

<ParamField type="string">
  `transparent`, `opaque`, or `auto`. Transparent requires `png` or `webp` output.
</ParamField>

<ParamField type="string">
  `png`, `jpeg`, or `webp`.
</ParamField>

<ParamField type="integer">
  Compression level 0–100, for `jpeg` and `webp` only.
</ParamField>

<ParamField type="string">
  Stable end-user identifier.
</ParamField>

<RequestExample>
  ```bash cURL theme={"dark"}
  curl https://api.pipellm.ai/v1/images/edits \
    -H "Authorization: Bearer $PIPELLM_API_KEY" \
    -F model="gpt-image-2.5-sunburst" \
    -F image="@room.png" \
    -F mask="@room-mask.png" \
    -F prompt="Replace the sofa with a green velvet one" \
    -F size="1024x1024"
  ```

  ```python Python theme={"dark"}
  import base64
  import os
  from openai import OpenAI

  client = OpenAI(
      api_key=os.environ["PIPELLM_API_KEY"],
      base_url="https://api.pipellm.ai/v1",
  )

  result = client.images.edit(
      model="gpt-image-2.5-sunburst",
      image=open("room.png", "rb"),
      mask=open("room-mask.png", "rb"),
      prompt="Replace the sofa with a green velvet one",
      size="1024x1024",
  )

  with open("room-edited.png", "wb") as f:
      f.write(base64.b64decode(result.data[0].b64_json))
  ```

  ```typescript TypeScript theme={"dark"}
  import fs from "node:fs";
  import OpenAI, { toFile } from "openai";

  const client = new OpenAI({
    apiKey: process.env.PIPELLM_API_KEY,
    baseURL: "https://api.pipellm.ai/v1",
  });

  const result = await client.images.edit({
    model: "gpt-image-2.5-sunburst",
    image: await toFile(fs.createReadStream("room.png"), "room.png"),
    mask: await toFile(fs.createReadStream("room-mask.png"), "room-mask.png"),
    prompt: "Replace the sofa with a green velvet one",
    size: "1024x1024",
  });

  fs.writeFileSync("room-edited.png", Buffer.from(result.data[0].b64_json, "base64"));
  ```
</RequestExample>

<ResponseExample>
  ```json theme={"dark"}
  {
    "created": 1726400000,
    "size": "1024x1024",
    "quality": "medium",
    "output_format": "png",
    "background": "opaque",
    "data": [
      { "b64_json": "iVBORw0KGgoAAAANSUhEUgAA..." }
    ],
    "usage": {
      "input_tokens": 342,
      "input_tokens_details": { "text_tokens": 14, "image_tokens": 328 },
      "output_tokens": 1056
    }
  }
  ```
</ResponseExample>

The response shape matches [Create an image](/api-reference/image/generations): the bytes come back base64-encoded in `data[].b64_json`, and `usage.input_tokens_details.image_tokens` covers the source images you uploaded.

## Combining references

With several source images the model composes across them — a product plus a scene, a person plus an outfit. Order matters: describe the references in the prompt in the order you send them, for example "dress the person from the first reference in the outfit from the second".

Masks apply to the first image only.

## Long requests

Edits use the same keep-alive behavior as generation: after 90 seconds of upstream silence the gateway writes a whitespace padding chunk every 30 seconds ahead of the JSON body. See [Long requests](/api-reference/image/generations#long-requests).

## Errors

Same envelope as [Errors](/api-reference/errors).

<AccordionGroup>
  <Accordion title="400 invalid_request_error">
    Mask dimensions not matching the source, an unreadable upload, or a `size` the model rejects.

    ```json theme={"dark"}
    {
      "error": {
        "type": "invalid_request_error",
        "code": "400",
        "message": "The mask must have the same dimensions as the image"
      }
    }
    ```
  </Accordion>

  <Accordion title="401 authentication_error">
    ```json theme={"dark"}
    {
      "error": {
        "type": "authentication_error",
        "code": "401",
        "message": "Incorrect API key provided. Please visit https://console.pipellm.ai/account/api-keys to find your API key."
      }
    }
    ```
  </Accordion>

  <Accordion title="413 invalid_request_error">
    The upload exceeded the accepted body size. Downscale the source before sending.
  </Accordion>

  <Accordion title="429 rate_limit_error">
    See [Rate Limits](/api-reference/rate-limits).
  </Accordion>
</AccordionGroup>


# Generate an image with Gemini
Source: https://docs.pipellm.ai/api-reference/image/gemini

POST /v1beta/models/{model}:generateContent
Generate and edit images with Gemini image models.

Gemini image models do not use `/v1/images/*`. They generate images through the same `generateContent` endpoint as text models, and return the bytes as an inline part alongside any text the model produced.

For GPT image models, use [`POST /v1/images/generations`](/api-reference/image/generations) instead.

See [Nano Banana](/use-cases/nano-banana) for worked text-to-image and image-to-image examples.

<Info>
  Model IDs change over time. List the ones available to your account with [`GET /v1/models`](/api-reference/list-models).
</Info>

## Endpoint

| Mode          | Path                                                |
| ------------- | --------------------------------------------------- |
| Non-streaming | `POST /v1beta/models/{model}:generateContent`       |
| Streaming     | `POST /v1beta/models/{model}:streamGenerateContent` |

Authenticate with `x-goog-api-key`, as on every Gemini route. See [Gemini overview](/api-reference/gemini/overview).

<ParamField type="array">
  Conversation turns. For text-to-image a single user turn with one text part is enough; for editing, add the source image as an inline part in the same turn.

  <Expandable title="contents[]">
    <ParamField type="string">
      `user` or `model`.
    </ParamField>

    <ParamField type="array">
      Mixed text and inline image parts.

      <Expandable title="parts[]">
        <ParamField type="string">
          Prompt text.
        </ParamField>

        <ParamField type="object">
          A source image, as `mimeType` (for example `image/png`) plus base64 `data`.
        </ParamField>
      </Expandable>
    </ParamField>
  </Expandable>
</ParamField>

<ParamField type="object">
  Standard Gemini generation settings. Image models also accept sizing controls here; the accepted aspect ratios and resolutions differ per model, so check the model's own card before relying on a value.
</ParamField>

<RequestExample>
  ```bash cURL theme={"dark"}
  curl https://api.pipellm.ai/v1beta/models/gemini-3-pro-image:generateContent \
    -H "x-goog-api-key: $PIPELLM_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "contents": [{
        "role": "user",
        "parts": [{"text": "A red bicycle leaning against a cafe window, morning light"}]
      }]
    }'
  ```

  ```python Python theme={"dark"}
  import os
  from google import genai

  client = genai.Client(
      api_key=os.environ["PIPELLM_API_KEY"],
      http_options={"base_url": "https://api.pipellm.ai"},
  )

  response = client.models.generate_content(
      model="gemini-3-pro-image",
      contents="A red bicycle leaning against a cafe window, morning light",
  )

  for part in response.parts:
      if part.inline_data is not None:
          part.as_image().save("bicycle.png")
  ```

  ```typescript TypeScript theme={"dark"}
  import fs from "node:fs";
  import { GoogleGenAI } from "@google/genai";

  const ai = new GoogleGenAI({
    apiKey: process.env.PIPELLM_API_KEY,
    httpOptions: { baseUrl: "https://api.pipellm.ai" },
  });

  const response = await ai.models.generateContent({
    model: "gemini-3-pro-image",
    contents: "A red bicycle leaning against a cafe window, morning light",
  });

  for (const part of response.candidates[0].content.parts) {
    if (part.inlineData) {
      fs.writeFileSync("bicycle.png", Buffer.from(part.inlineData.data, "base64"));
    }
  }
  ```
</RequestExample>

<ResponseExample>
  ```json theme={"dark"}
  {
    "candidates": [{
      "content": {
        "role": "model",
        "parts": [
          { "text": "Here is the bicycle scene you asked for." },
          {
            "inlineData": {
              "mimeType": "image/png",
              "data": "iVBORw0KGgoAAAANSUhEUgAA..."
            }
          }
        ]
      },
      "finishReason": "STOP"
    }],
    "usageMetadata": {
      "promptTokenCount": 12,
      "candidatesTokenCount": 1290,
      "totalTokenCount": 1302
    }
  }
  ```
</ResponseExample>

<ResponseField name="candidates" type="array">
  Generated responses.

  <Expandable title="candidates[]">
    <ResponseField name="content.parts" type="array">
      Interleaved output. A part carries either `text` or `inlineData`, never both. A single response can hold several of each, so iterate over all parts rather than reading `parts[0]`.
    </ResponseField>

    <ResponseField name="content.parts[].inlineData.mimeType" type="string">
      For example `image/png`.
    </ResponseField>

    <ResponseField name="content.parts[].inlineData.data" type="string">
      Base64-encoded image bytes.
    </ResponseField>

    <ResponseField name="finishReason" type="string">
      `STOP` on success. A safety stop returns no image part.
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="usageMetadata" type="object">
  Token counts. Generated images are billed as image output tokens, which dominate the cost — see [Pricing](/api-reference/model-pricing).
</ResponseField>

## Reading the response

Two habits save trouble:

* **Iterate over every part.** Ask for an illustrated explanation and you get text and image parts interleaved. Taking only the first part silently drops content.
* **Expect a text-only response sometimes.** If the prompt is refused on safety grounds the response still returns `200` with a text part and no `inlineData`. Check for the image part before decoding.

Images generated by Gemini carry a SynthID watermark.

## Errors

Gemini routes return the Gemini error shape, not the OpenAI one. See [Errors](/api-reference/errors) for the full mapping.

<AccordionGroup>
  <Accordion title="400 INVALID_ARGUMENT">
    Malformed `contents`, an unsupported `mimeType` on an inline part, or a sizing value the model does not accept.
  </Accordion>

  <Accordion title="401 UNAUTHENTICATED">
    Missing or wrong `x-goog-api-key`.
  </Accordion>

  <Accordion title="429 RESOURCE_EXHAUSTED">
    See [Rate Limits](/api-reference/rate-limits).
  </Accordion>
</AccordionGroup>


# Create an image
Source: https://docs.pipellm.ai/api-reference/image/generations

POST /v1/images/generations
Generate images from a text prompt with GPT image models.

OpenAI-compatible image generation. Point the OpenAI SDK at `https://api.pipellm.ai/v1` and the request works unchanged.

For Gemini image models, use [`generateContent`](/api-reference/image/gemini) instead — the two families take different request shapes.

<Info>
  Model IDs change over time. List the ones available to your account with [`GET /v1/models`](/api-reference/list-models).
</Info>

<ParamField type="string">
  Text description of the image. Up to 32,000 characters on GPT image models.
</ParamField>

<ParamField type="string">
  Image model ID, for example `gpt-image-2.5-sunburst`. Defaults to `dall-e-2` upstream, so set it explicitly.
</ParamField>

<ParamField type="integer">
  Number of images to generate, 1–10.
</ParamField>

<ParamField type="string">
  `auto`, `1024x1024`, `1536x1024`, or `1024x1536` on GPT image models. A custom `WIDTHxHEIGHT` is accepted when both sides divide by 16 and the aspect ratio stays between 1:3 and 3:1.
</ParamField>

<ParamField type="string">
  `low`, `medium`, `high`, or `auto`. The `gpt-image-2.5` variants add `xhigh` and `max`.
  Higher quality costs more output tokens.
</ParamField>

<ParamField type="string">
  `transparent`, `opaque`, or `auto`. Transparent requires `output_format` of `png` or `webp`.
</ParamField>

<ParamField type="string">
  `png`, `jpeg`, or `webp`.
</ParamField>

<ParamField type="integer">
  Compression level 0–100, for `jpeg` and `webp` only.
</ParamField>

<ParamField type="string">
  `low` or `auto`.
</ParamField>

<ParamField type="string">
  Stable end-user identifier, passed through for abuse monitoring.
</ParamField>

<RequestExample>
  ```bash cURL theme={"dark"}
  curl https://api.pipellm.ai/v1/images/generations \
    -H "Authorization: Bearer $PIPELLM_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "model": "gpt-image-2.5-sunburst",
      "prompt": "A red bicycle leaning against a cafe window, morning light",
      "size": "1024x1024",
      "quality": "medium",
      "n": 1
    }'
  ```

  ```python Python theme={"dark"}
  import base64
  import os
  from openai import OpenAI

  client = OpenAI(
      api_key=os.environ["PIPELLM_API_KEY"],
      base_url="https://api.pipellm.ai/v1",
  )

  result = client.images.generate(
      model="gpt-image-2.5-sunburst",
      prompt="A red bicycle leaning against a cafe window, morning light",
      size="1024x1024",
      quality="medium",
  )

  with open("bicycle.png", "wb") as f:
      f.write(base64.b64decode(result.data[0].b64_json))
  ```

  ```typescript TypeScript theme={"dark"}
  import fs from "node:fs";
  import OpenAI from "openai";

  const client = new OpenAI({
    apiKey: process.env.PIPELLM_API_KEY,
    baseURL: "https://api.pipellm.ai/v1",
  });

  const result = await client.images.generate({
    model: "gpt-image-2.5-sunburst",
    prompt: "A red bicycle leaning against a cafe window, morning light",
    size: "1024x1024",
    quality: "medium",
  });

  fs.writeFileSync("bicycle.png", Buffer.from(result.data[0].b64_json, "base64"));
  ```
</RequestExample>

<ResponseExample>
  ```json theme={"dark"}
  {
    "created": 1726400000,
    "size": "1024x1024",
    "quality": "medium",
    "output_format": "png",
    "background": "opaque",
    "data": [
      { "b64_json": "iVBORw0KGgoAAAANSUhEUgAA..." }
    ],
    "usage": {
      "input_tokens": 18,
      "input_tokens_details": { "text_tokens": 18, "image_tokens": 0 },
      "output_tokens": 1056,
      "output_tokens_details": { "image_tokens": 1056 }
    }
  }
  ```
</ResponseExample>

<ResponseField name="created" type="integer">
  Unix timestamp of the generation.
</ResponseField>

<ResponseField name="data" type="array">
  One entry per generated image.

  <Expandable title="data[]">
    <ResponseField name="b64_json" type="string">
      Base64-encoded image bytes. GPT image models always return the image this way — there is no URL to fetch.
    </ResponseField>

    <ResponseField name="revised_prompt" type="string">
      The rewritten prompt. DALL-E 3 only.
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="size" type="string">
  Resolved output size, useful when the request asked for `auto`.
</ResponseField>

<ResponseField name="quality" type="string">
  Resolved quality.
</ResponseField>

<ResponseField name="output_format" type="string">
  `png`, `jpeg`, or `webp`.
</ResponseField>

<ResponseField name="background" type="string">
  `transparent` or `opaque`.
</ResponseField>

<ResponseField name="usage" type="object">
  Token counts that back the charge. `output_tokens_details.image_tokens` is the part that scales with size and quality.
</ResponseField>

## Long requests

Image generation can run for minutes. On a non-streaming request PipeLLM holds the connection open: if the upstream has not answered after 90 seconds, the gateway writes a padding chunk every 30 seconds until the real body arrives.

The padding is whitespace before the JSON document, so any standard JSON parser handles it. Two things to watch:

* Do not assume the first bytes on the wire are `{`.
* Set a generous client read timeout. The default in most HTTP clients is shorter than a slow high-quality generation.

## Errors

Same envelope as [Errors](/api-reference/errors).

<AccordionGroup>
  <Accordion title="400 invalid_request_error">
    Missing `prompt`, a `size` the model does not accept, or `background: transparent` with `output_format: jpeg`.

    ```json theme={"dark"}
    {
      "error": {
        "type": "invalid_request_error",
        "code": "400",
        "message": "Invalid value for 'size'"
      }
    }
    ```
  </Accordion>

  <Accordion title="401 authentication_error">
    ```json theme={"dark"}
    {
      "error": {
        "type": "authentication_error",
        "code": "401",
        "message": "Incorrect API key provided. Please visit https://console.pipellm.ai/account/api-keys to find your API key."
      }
    }
    ```
  </Accordion>

  <Accordion title="402 insufficient_balance">
    ```json theme={"dark"}
    {
      "error": {
        "type": "insufficient_balance",
        "code": "402",
        "message": "Insufficient balance. Please recharge your account at https://console.pipellm.ai/billing."
      }
    }
    ```
  </Accordion>

  <Accordion title="429 rate_limit_error">
    See [Rate Limits](/api-reference/rate-limits).

    ```json theme={"dark"}
    {
      "error": {
        "type": "rate_limit_error",
        "code": "429",
        "message": "Rate limit exceeded. Your current limit is 120 requests per minute (approximately 2.00 requests per second). Please visit https://console.pipellm.ai/billing to upgrade your plan for higher limits."
      }
    }
    ```
  </Accordion>
</AccordionGroup>


# Create an image variation
Source: https://docs.pipellm.ai/api-reference/image/variations

POST /v1/images/variations
Generate variations of a source image.

<Warning>
  Upstream, this endpoint only supports `dall-e-2`. GPT image models do not implement it. To vary an image with a GPT image model, send the source to [`POST /v1/images/edits`](/api-reference/image/edits) with a prompt describing the variation you want.
</Warning>

Returns new images derived from a source image, with no prompt. Always `multipart/form-data`.

<ParamField type="file">
  Source image. Must be a square PNG under 4 MB.
</ParamField>

<ParamField type="string">
  Only `dall-e-2` implements variations.
</ParamField>

<ParamField type="integer">
  Number of variations, 1–10.
</ParamField>

<ParamField type="string">
  `256x256`, `512x512`, or `1024x1024`.
</ParamField>

<ParamField type="string">
  `url` or `b64_json`. URLs expire about 60 minutes after generation, so download promptly.
</ParamField>

<ParamField type="string">
  Stable end-user identifier.
</ParamField>

<RequestExample>
  ```bash cURL theme={"dark"}
  curl https://api.pipellm.ai/v1/images/variations \
    -H "Authorization: Bearer $PIPELLM_API_KEY" \
    -F model="dall-e-2" \
    -F image="@logo.png" \
    -F n=2 \
    -F size="1024x1024"
  ```

  ```python Python theme={"dark"}
  import os
  from openai import OpenAI

  client = OpenAI(
      api_key=os.environ["PIPELLM_API_KEY"],
      base_url="https://api.pipellm.ai/v1",
  )

  result = client.images.create_variation(
      model="dall-e-2",
      image=open("logo.png", "rb"),
      n=2,
      size="1024x1024",
  )
  print([item.url for item in result.data])
  ```
</RequestExample>

<ResponseExample>
  ```json theme={"dark"}
  {
    "created": 1726400000,
    "data": [
      { "url": "https://..." },
      { "url": "https://..." }
    ]
  }
  ```
</ResponseExample>

<ResponseField name="created" type="integer">
  Unix timestamp of the generation.
</ResponseField>

<ResponseField name="data" type="array">
  One entry per variation, carrying `url` or `b64_json` depending on `response_format`.
</ResponseField>

## Errors

Same envelope as [Errors](/api-reference/errors). A non-square or oversized source returns `400`; a model that does not implement variations returns the upstream's error unchanged.


# Overview
Source: https://docs.pipellm.ai/api-reference/introduction

Relay HTTP operations: path, headers, parameters, and response shape.

These pages document the Relay API. Default chat path is OpenAI Chat Completions at `https://api.pipellm.ai/v1`. Each page below is one HTTP operation.

## Jev / System One

<Card title="POST /v1/systemone" href="/api-reference/systemone">
  Call jev-1.13.0 for typed judgments, choices and scores.
</Card>

## Models

<Columns>
  <Card title="GET /v1/models" href="/api-reference/list-models">
    List model IDs visible to your account
  </Card>

  <Card title="GET /v1/models/pricing" href="/api-reference/model-pricing">
    Look up the authoritative price for a model
  </Card>
</Columns>

## Chat

<Columns>
  <Card title="POST /v1/chat/completions" href="/api-reference/openai/chat-completions">
    OpenAI-compatible chat completions, including streaming
  </Card>

  <Card title="POST /v1/responses" href="/api-reference/openai/responses">
    OpenAI Responses API
  </Card>

  <Card title="POST /v1/messages" href="/api-reference/anthropic/messages">
    Anthropic Messages
  </Card>

  <Card title="POST /v1beta/models/{model}:generateContent" href="/api-reference/gemini/generate-content">
    Gemini generateContent
  </Card>

  <Card title="POST /v1beta/models/{model}:streamGenerateContent" href="/api-reference/gemini/stream-generate-content">
    Gemini streaming
  </Card>
</Columns>

## Video

<Columns>
  <Card title="POST /v2/videos" href="/api-reference/video/create">
    Create a video task
  </Card>

  <Card title="GET /v2/videos/{id}" href="/api-reference/video/get">
    Poll a video task
  </Card>

  <Card title="DELETE /v2/videos/{id}" href="/api-reference/video/cancel">
    Cancel a video task
  </Card>
</Columns>

## WebSearch

<Columns>
  <Card title="GET /v1/websearch/search" href="/websearch/search-api">
    Deep search
  </Card>

  <Card title="GET /v1/websearch/simple-search" href="/websearch/simple-search">
    Fast snippets
  </Card>

  <Card title="GET /v1/websearch/reader" href="/websearch/reader">
    Fetch a URL as text
  </Card>

  <Card title="GET /v1/websearch/search-news" href="/websearch/news-search">
    News search
  </Card>
</Columns>

## Auth, errors, limits

| Topic                      | Page                                             |
| -------------------------- | ------------------------------------------------ |
| Keys and headers           | [Authentication](/api-reference/authentication)  |
| Error envelope             | [Errors](/api-reference/errors)                  |
| RPM and 402                | [Rate Limits](/api-reference/rate-limits)        |
| Native vs converter routes | [Routing & Protocols](/guides/routing-protocols) |


# List models
Source: https://docs.pipellm.ai/api-reference/list-models

GET /v1/models
Retrieve the PipeLLM unified model catalog. Does not include pricing.

Use this route to retrieve the model IDs currently visible to your account.
`/v1/models` is now a catalog-only endpoint. It returns model identity and
protocol metadata, not pricing.

For pricing, use [`/v1/models/pricing`](/api-reference/model-pricing).

## Endpoint

```text theme={"dark"}
GET https://api.pipellm.ai/v1/models
```

## Authentication

Use a standard PipeLLM API key header:

| Header          | Example                   |
| --------------- | ------------------------- |
| `Authorization` | `Bearer $PIPELLM_API_KEY` |
| `x-api-key`     | `$PIPELLM_API_KEY`        |

## Notes

* This route only supports `GET`.
* The response is PipeLLM's unified catalog schema.
* Hidden and unlisted models are filtered out before the response is returned.
* Results are sorted by model ID.
* Pricing is intentionally excluded so the catalog does not imply a single authoritative price when the same model can route across multiple mappings.

<RequestExample>
  ```bash cURL theme={"dark"}
  curl https://api.pipellm.ai/v1/models \
    -H "Authorization: Bearer $PIPELLM_API_KEY"
  ```

  ```python Python theme={"dark"}
  import os
  import requests

  response = requests.get(
      "https://api.pipellm.ai/v1/models",
      headers={"Authorization": f"Bearer {os.getenv('PIPELLM_API_KEY')}"},
  )
  payload = response.json()
  for model in payload["data"]:
      print(model["id"], model["type_target"])
  ```

  ```javascript JavaScript theme={"dark"}
  const response = await fetch("https://api.pipellm.ai/v1/models", {
    headers: { Authorization: `Bearer ${process.env.PIPELLM_API_KEY}` },
  });
  const payload = await response.json();
  for (const model of payload.data) {
    console.log(model.id, model.type_target);
  }
  ```
</RequestExample>

## Response Format

```json theme={"dark"}
{
  "object": "list",
  "data": [
    {
      "id": "gpt-5",
      "display_name": "Gpt 5",
      "type_target": "openai",
      "created_at": "2025-05-14T00:00:00Z"
    },
    {
      "id": "claude-sonnet-4-6",
      "display_name": "Claude Sonnet 4.6",
      "type_target": "anthropic",
      "created_at": "2025-03-01T00:00:00Z"
    }
  ],
  "total": 2
}
```

## Response Fields

| Field                 | Type    | Description                                                                    |
| --------------------- | ------- | ------------------------------------------------------------------------------ |
| `object`              | string  | Always `"list"`                                                                |
| `data`                | array   | Array of model catalog entries                                                 |
| `data[].id`           | string  | Model ID used in API requests                                                  |
| `data[].display_name` | string  | Human-readable label generated from the model ID                               |
| `data[].type_target`  | string  | Target protocol family for routing, such as `openai`, `anthropic`, or `gemini` |
| `data[].created_at`   | string  | RFC3339 timestamp chosen from the model metadata                               |
| `total`               | integer | Total number of returned models                                                |

## Why This Route Matters

Use `/v1/models` when you need to:

* populate a model picker in your UI
* validate whether a model is visible to the current account
* decide which protocol family a model is exposed under

Use [`/v1/models/pricing`](/api-reference/model-pricing) when you need
authoritative billing data.

## Related Docs

<Columns>
  <Card title="Model Pricing" icon="wallet" href="/api-reference/model-pricing">
    Retrieve authoritative pricing for a specific model, provider, or platform mapping
  </Card>

  <Card title="Routing & Protocols" icon="route" href="/guides/routing-protocols">
    Understand how model protocol families affect requests
  </Card>

  <Card title="Developer Tools" icon="terminal" href="/integrations/overview">
    Use these model IDs in Claude Code, OpenCode, OpenClaw, and LangChain
  </Card>
</Columns>


# Get model pricing
Source: https://docs.pipellm.ai/api-reference/model-pricing

GET /v1/models/pricing
Retrieve the authoritative price for a model.

Use this route when you need pricing, not just catalog visibility.

`/v1/models/pricing` returns a single authoritative pricing object for a model;
it does not list prices from every provider. When a model can route across
multiple providers, PipeLLM only returns a price if they resolve to one
authoritative price. Otherwise, you can select a provider with `provider`.

## How billing works

PipeLLM bills from your console balance. Insufficient balance returns `402` before the
request is processed — see [Errors](/api-reference/errors).

Rates for every model live in the [model library](https://pipellm.ai/models). This route
returns the same numbers programmatically. The docs deliberately do not restate rates, so
there is nothing here to go stale.

Video is billed in USD when you call [`POST /v2/videos`](/api-reference/video/create). The
quote is taken at create time and does not change if prices are edited later; query and
cancel do not charge.

## Endpoint

```text theme={"dark"}
GET https://api.pipellm.ai/v1/models/pricing
```

## Query Parameters

<ParamField type="string">
  Model ID to price.
</ParamField>

<ParamField type="string">
  Provider name when the same ID has more than one price.
</ParamField>

<ParamField type="integer">
  Platform mapping ID when you need an exact channel.
</ParamField>

## Notes

* This route only supports `GET`.
* `model` is required.
* If there is only one visible provider, the route returns that provider's price.
* If all visible providers produce the same effective price, the route returns one authoritative price.
* If visible providers produce different effective prices, the route returns `409 Conflict`; use `provider` to select a provider.

<RequestExample>
  ```bash cURL theme={"dark"}
  curl "https://api.pipellm.ai/v1/models/pricing?model=gpt-5" \
    -H "Authorization: Bearer $PIPELLM_API_KEY"
  ```

  ```bash cURL (provider) theme={"dark"}
  curl "https://api.pipellm.ai/v1/models/pricing?model=gpt-5&provider=openai" \
    -H "Authorization: Bearer $PIPELLM_API_KEY"
  ```
</RequestExample>

## Response Format

```json theme={"dark"}
{
  "object": "pricing",
  "data": {
    "id": "gpt-5",
    "display_name": "Gpt 5",
    "type_target": "openai",
    "provider": "openai",
    "pricing": {
      "kind": "token",
      "text": {
        "prompt": "0.000003",
        "completion": "0.000015",
        "internal_reasoning": "0.000015"
      },
      "cache": {
        "read": "0.000001",
        "write": "0.00000375",
        "write_1h": "0.0000075"
      },
      "tiers": {
        "over_200k": {
          "text": {
            "prompt": "0.0000055",
            "completion": "0.000022",
            "internal_reasoning": "0.000022"
          }
        }
      }
    }
  }
}
```

## Response Fields

| Field               | Type   | Description                                                                          |
| ------------------- | ------ | ------------------------------------------------------------------------------------ |
| `object`            | string | Always `"pricing"`                                                                   |
| `data.id`           | string | Model name used in API requests                                                      |
| `data.display_name` | string | Human-readable label generated from the model name                                   |
| `data.type_target`  | string | Target protocol family for routing                                                   |
| `data.provider`     | string | Resolved provider when the request identifies one provider or explicitly selects one |
| `data.pricing`      | object | Authoritative pricing object                                                         |

## Ambiguous Pricing

PipeLLM returns `409 Conflict` when:

* the same `model` matches multiple visible mappings
* those mappings have different prices
* and you did not select a provider with `provider`

That behavior is intentional. The API does not fabricate a representative price
when the routing layer can legitimately choose different-priced mappings.


# Create a chat completion
Source: https://docs.pipellm.ai/api-reference/openai/chat-completions

POST /v1/chat/completions
Create a model response for a chat conversation. Supports streaming.

Use `Authorization: Bearer $PIPELLM_API_KEY`. Pick `model` from [`GET /v1/models`](/api-reference/list-models).

## Endpoint

```
POST https://api.pipellm.ai/v1/chat/completions
```

<Note>
  This free route accepts OpenAI-compatible requests and routes only to
  OpenAI-compatible platforms. To keep the OpenAI format while calling
  Anthropic or Gemini models, use the
  [OpenAI Format Converter](/converter/openai-format).
</Note>

<RequestExample>
  ```bash cURL theme={"dark"}
  curl https://api.pipellm.ai/v1/chat/completions \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer $PIPELLM_API_KEY" \
    -d '{
      "model": "gpt-5",
      "messages": [{"role": "user", "content": "Hello"}]
    }'
  ```

  ```python Python theme={"dark"}
  import os
  from openai import OpenAI

  client = OpenAI(
      api_key=os.environ["PIPELLM_API_KEY"],
      base_url="https://api.pipellm.ai/v1",
  )
  print(client.chat.completions.create(
      model="gpt-5",
      messages=[{"role": "user", "content": "Hello"}],
  ).choices[0].message.content)
  ```

  ```javascript JavaScript theme={"dark"}
  import OpenAI from "openai";

  const client = new OpenAI({
    apiKey: process.env.PIPELLM_API_KEY,
    baseURL: "https://api.pipellm.ai/v1",
  });
  const response = await client.chat.completions.create({
    model: "gpt-5",
    messages: [{ role: "user", content: "Hello" }],
  });
  console.log(response.choices[0].message.content);
  ```
</RequestExample>

## Code Examples

<Tabs>
  <Tab title="cURL">
    ```bash theme={"dark"}
    curl https://api.pipellm.ai/v1/chat/completions \
      -H "Content-Type: application/json" \
      -H "Authorization: Bearer $PIPELLM_API_KEY" \
      -d '{
        "model": "gpt-5",
        "max_completion_tokens": 1024,
        "messages": [
          {
            "role": "user",
            "content": "Why is the sky blue?"
          }
        ]
      }'
    ```
  </Tab>

  <Tab title="Python">
    ```python theme={"dark"}
    import os
    from openai import OpenAI

    client = OpenAI(
      api_key=os.getenv('PIPELLM_API_KEY'),
      base_url='https://api.pipellm.ai/v1'
    )

    response = client.chat.completions.create(
      model='gpt-5',
      messages=[
        {
          'role': 'user',
          'content': 'Why is the sky blue?'
        }
      ]
    )
    ```
  </Tab>

  <Tab title="Node.js">
    ```javascript theme={"dark"}
    import OpenAI from 'openai';

    const client = new OpenAI({
      apiKey: process.env.PIPELLM_API_KEY,
      baseURL: 'https://api.pipellm.ai/v1'
    });

    const response = await client.chat.completions.create({
      model: 'gpt-5',
      messages: [
        {
          role: 'user',
          content: 'Why is the sky blue?'
        }
      ]
    });
    ```
  </Tab>

  <Tab title="Go">
    ```go theme={"dark"}
    package main

    import (
      "context"
      "os"
      openai "github.com/sashabaranov/go-openai"
    )

    func main() {
      config := openai.DefaultConfig(os.Getenv("PIPELLM_API_KEY"))
      config.BaseURL = "https://api.pipellm.ai/v1"
      client := openai.NewClientWithConfig(config)

      resp, _ := client.CreateChatCompletion(
        context.Background(),
        openai.ChatCompletionRequest{
          Model: "gpt-5",
          Messages: []openai.ChatCompletionMessage{
            {
              Role:    openai.ChatMessageRoleUser,
              Content: "Why is the sky blue?",
            },
          },
        },
      )
    }
    ```
  </Tab>
</Tabs>

## Request Parameters

Pick `model` from [`GET /v1/models`](/api-reference/list-models). Native `/v1/chat/completions` only routes to OpenAI-compatible platforms.

<ParamField type="string">
  Model ID visible to your account.
</ParamField>

<ParamField type="array">
  Conversation messages. At least one item.

  <Expandable title="messages[]">
    <ParamField type="string">
      `system`, `user`, `assistant`, or `tool`.
    </ParamField>

    <ParamField type="string | array">
      Plain text, or an array of content parts.

      <Expandable title="content parts">
        <ParamField type="string">
          `text` or `image_url`.
        </ParamField>

        <ParamField type="string">
          Required when `type` is `text`.
        </ParamField>

        <ParamField type="object">
          Required when `type` is `image_url`. Set `url` to an HTTPS URL or a data URL.
        </ParamField>
      </Expandable>
    </ParamField>

    <ParamField type="string">
      Optional speaker name.
    </ParamField>

    <ParamField type="string">
      Required when `role` is `tool`.
    </ParamField>
  </Expandable>
</ParamField>

<ParamField type="number">
  Sampling temperature, 0–2. Default `1`.
</ParamField>

<ParamField type="integer">
  Maximum tokens to generate.
</ParamField>

<ParamField type="integer">
  Preferred maximum for newer OpenAI-compatible models. Use this instead of `max_tokens` when the model requires it.
</ParamField>

<ParamField type="boolean">
  If `true`, return server-sent events. See [Streaming](/api-reference/streaming).
</ParamField>

<ParamField type="object">
  Streaming extras.

  <Expandable title="stream_options fields">
    <ParamField type="boolean">
      If `true`, the last SSE chunk includes `usage`.
    </ParamField>
  </Expandable>
</ParamField>

<ParamField type="number">
  Nucleus sampling, 0–1.
</ParamField>

<ParamField type="integer">
  Number of completions to generate. Default `1`.
</ParamField>

<ParamField type="string | string[]">
  Stop sequence or list of stop sequences.
</ParamField>

<ParamField type="number">
  Presence penalty, -2 to 2. Default `0`.
</ParamField>

<ParamField type="number">
  Frequency penalty, -2 to 2. Default `0`.
</ParamField>

<ParamField type="array">
  Function tools the model may call. See [Function Calling](#function-calling).

  <Expandable title="tools[]">
    <ParamField type="string">
      Always `function`.
    </ParamField>

    <ParamField type="object">
      Tool definition.

      <Expandable title="function fields">
        <ParamField type="string">
          Function name.
        </ParamField>

        <ParamField type="string">
          What the function does.
        </ParamField>

        <ParamField type="object">
          JSON Schema for the function arguments.
        </ParamField>
      </Expandable>
    </ParamField>
  </Expandable>
</ParamField>

<ParamField type="string | object">
  `auto`, `none`, or `{"type":"function","function":{"name":"..."}}`.
</ParamField>

<ParamField type="string">
  End-user identifier for abuse tracking.
</ParamField>

## Response Format

```json theme={"dark"}
{
  "id": "chatcmpl-xxx",
  "object": "chat.completion",
  "created": 1234567890,
  "model": "gpt-5",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "The sky appears blue because..."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 10,
    "completion_tokens": 50,
    "total_tokens": 60
  }
}
```

<ResponseField name="id" type="string">
  Completion ID.
</ResponseField>

<ResponseField name="object" type="string">
  `chat.completion`, or `chat.completion.chunk` when streaming.
</ResponseField>

<ResponseField name="model" type="string">
  Model that produced the response.
</ResponseField>

<ResponseField name="choices" type="array">
  Completion choices.

  <Expandable title="choices[]">
    <ResponseField name="index" type="integer">
      Choice index.
    </ResponseField>

    <ResponseField name="message" type="object">
      Present when `stream` is false. Contains `role`, `content`, and optional `tool_calls`.
    </ResponseField>

    <ResponseField name="delta" type="object">
      Present when streaming. Partial `role` / `content` / `tool_calls`.
    </ResponseField>

    <ResponseField name="finish_reason" type="string">
      `stop`, `length`, or `tool_calls`.
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="usage" type="object">
  Token counts: `prompt_tokens`, `completion_tokens`, `total_tokens`.
</ResponseField>

## Errors

Same envelope as [Errors](/api-reference/errors).

<AccordionGroup>
  <Accordion title="400 invalid_request_error">
    Malformed JSON, missing `model` / `messages`, or a protocol mismatch on this native route.

    ```json theme={"dark"}
    {
      "error": {
        "type": "invalid_request_error",
        "code": "400",
        "message": "The 'model' field is required but was not provided in the request"
      }
    }
    ```
  </Accordion>

  <Accordion title="401 authentication_error">
    Missing or invalid API key.

    ```json theme={"dark"}
    {
      "error": {
        "type": "authentication_error",
        "code": "401",
        "message": "Incorrect API key provided. Please visit https://console.pipellm.ai/account/api-keys to find your API key."
      }
    }
    ```
  </Accordion>

  <Accordion title="402 insufficient_balance">
    Balance check failed before the request was processed.

    ```json theme={"dark"}
    {
      "error": {
        "type": "insufficient_balance",
        "code": "402",
        "message": "Insufficient balance. Please recharge your account at https://console.pipellm.ai/billing."
      }
    }
    ```
  </Accordion>

  <Accordion title="429 rate_limit_error">
    Account RPM exceeded. See [Rate Limits](/api-reference/rate-limits).

    ```json theme={"dark"}
    {
      "error": {
        "type": "rate_limit_error",
        "code": "429",
        "message": "Rate limit exceeded. Your current limit is 120 requests per minute (approximately 2.00 requests per second). Please visit https://console.pipellm.ai/billing to upgrade your plan for higher limits."
      }
    }
    ```
  </Accordion>
</AccordionGroup>

## Function Calling

Function Calling allows models to generate structured JSON to call functions in your code.

<Tabs>
  <Tab title="Python">
    ```python theme={"dark"}
    tools = [
      {
        "type": "function",
        "function": {
          "name": "get_weather",
          "description": "Get the current weather in a location",
          "parameters": {
            "type": "object",
            "properties": {
              "location": {"type": "string", "description": "City name"}
            },
            "required": ["location"]
          }
        }
      }
    ]

    response = client.chat.completions.create(
      model="gpt-5",
      messages=[{"role": "user", "content": "What's the weather in Tokyo?"}],
      tools=tools,
      tool_choice="auto"
    )

    if response.choices[0].message.tool_calls:
      tool_call = response.choices[0].message.tool_calls[0]
      # Execute your function and return result
    ```
  </Tab>

  <Tab title="Node.js">
    ```javascript theme={"dark"}
    const tools = [
      {
        type: "function",
        function: {
          name: "get_weather",
          description: "Get the current weather in a location",
          parameters: {
            type: "object",
            properties: {
              location: { type: "string", description: "City name" }
            },
            required: ["location"]
          }
        }
      }
    ];

    const response = await client.chat.completions.create({
      model: 'gpt-5',
      messages: [{ role: 'user', content: "What's the weather in Tokyo?" }],
      tools: tools,
      tool_choice: 'auto'
    });

    if (response.choices[0].message.tool_calls) {
      const toolCall = response.choices[0].message.tool_calls[0];
      // Execute your function and return result
    }
    ```
  </Tab>
</Tabs>

<Card title="Function Calling Documentation" icon="book" href="https://platform.openai.com/docs/guides/function-calling">
  Complete guide on defining functions and handling responses
</Card>


# Create a response
Source: https://docs.pipellm.ai/api-reference/openai/responses

POST /v1/responses
Create a model response with the OpenAI Responses API.

For authentication and model list, see [Overview](/api-reference/openai/overview).

## Endpoint

```
POST https://api.pipellm.ai/v1/responses
```

<Note>
  This free route accepts OpenAI Responses requests and routes only to
  OpenAI-compatible platforms. To call Anthropic or Gemini models with the
  Responses format, use the
  [Responses Format Converter](/converter/responses-format). The converter is
  stateless and does not support `previous_response_id`, stored responses,
  hosted tools, or structured output formats.
</Note>

## Code Examples

<Tabs>
  <Tab title="cURL">
    ```bash theme={"dark"}
    curl https://api.pipellm.ai/v1/responses \
      -H "Content-Type: application/json" \
      -H "Authorization: Bearer $PIPELLM_API_KEY" \
      -d '{
        "model": "gpt-5",
        "input": "Write a one-sentence explanation of why the sky is blue."
      }'
    ```
  </Tab>

  <Tab title="Python">
    ```python theme={"dark"}
    import os
    from openai import OpenAI

    client = OpenAI(
      api_key=os.getenv("PIPELLM_API_KEY"),
      base_url="https://api.pipellm.ai/v1"
    )

    response = client.responses.create(
      model="gpt-5",
      input="Write a one-sentence explanation of why the sky is blue."
    )

    print(response.output_text)
    ```
  </Tab>

  <Tab title="TypeScript">
    ```typescript theme={"dark"}
    import OpenAI from "openai";

    const client = new OpenAI({
      apiKey: process.env.PIPELLM_API_KEY,
      baseURL: "https://api.pipellm.ai/v1",
    });

    const response = await client.responses.create({
      model: "gpt-5",
      input: "Write a one-sentence explanation of why the sky is blue.",
    });

    console.log(response.output_text);
    ```
  </Tab>
</Tabs>

## Request Parameters

<ParamField type="string">
  OpenAI-compatible model ID from [`GET /v1/models`](/api-reference/list-models).
</ParamField>

<ParamField type="string | array">
  Text or structured input items.
</ParamField>

<ParamField type="string">
  Developer or system instruction.
</ParamField>

<ParamField type="boolean">
  If `true`, stream the response.
</ParamField>

<ParamField type="array">
  Built-in or custom tools.
</ParamField>

<ParamField type="number">
  Sampling temperature.
</ParamField>

<ParamField type="integer">
  Maximum output tokens.
</ParamField>

## Response Shape

```json theme={"dark"}
{
  "id": "resp_123",
  "object": "response",
  "model": "gpt-5",
  "output": [
    {
      "type": "message",
      "role": "assistant",
      "content": [
        {
          "type": "output_text",
          "text": "The atmosphere scatters blue light more strongly than red light."
        }
      ]
    }
  ]
}
```

## Related Docs

<Columns>
  <Card title="Routing & Protocols" icon="route" href="/guides/routing-protocols">
    Understand native route constraints
  </Card>

  <Card title="OpenAI Overview" icon="bolt" href="/api-reference/openai/overview">
    OpenAI-compatible endpoints on PipeLLM
  </Card>

  <Card title="OpenAI Migration Guide" icon="book" href="https://platform.openai.com/docs/guides/responses-vs-chat-completions">
    Official guidance on Responses vs Chat Completions
  </Card>
</Columns>


# Rate Limits
Source: https://docs.pipellm.ai/api-reference/rate-limits

API rate limits by account tier

## Rate Limiting

Your API rate limit is based on your account tier. Your balance is checked separately before each request; it does not dynamically increase or decrease your RPM.

### Account Tiers

Accounts are automatically assigned to a tier based on eligible successful payments.

| Tier     | Eligibility                                                                                    | RPM |
| -------- | ---------------------------------------------------------------------------------------------- | --- |
| `tier_0` | No eligible payment                                                                            | 6   |
| `tier_1` | Eligible payments totaling at least \$5                                                        | 30  |
| `tier_2` | Eligible payments totaling at least \$50, with first eligible payment more than 7 days ago     | 120 |
| `tier_3` | Eligible payments totaling at least \$250, with first eligible payment more than 7 days ago    | 300 |
| `tier_4` | Eligible payments totaling at least \$1,000, with first eligible payment more than 14 days ago | 600 |

Eligible payments are successful customer payments. Promotional credits, corrections, complimentary credits, and manual adjustments may not count toward automatic tier upgrades unless marked as eligible by PipeLLM.

### Enterprise and Custom Limits

Enterprise customers can receive a custom RPM override. A custom override takes precedence over the automatic tier limit.

Need higher limits? Contact us at [support@pipellm.ai](mailto:support@pipellm.ai).

## Balance Protection

To prevent unexpected overuse, the platform performs a **real-time balance check** before processing each request. If your balance is insufficient, the request will be rejected with a `402` status code.

### Insufficient Balance Response (402)

```json theme={"dark"}
{
  "error": {
    "type": "insufficient_balance",
    "code": "balance_exceeded",
    "message": "Insufficient balance. Please recharge to continue."
  }
}
```

## Rate Limit Response (429)

```json theme={"dark"}
{
  "error": {
    "type": "rate_limit_error",
    "code": "rate_limit_exceeded",
    "message": "Rate limit exceeded. Your current limit is 120 requests per minute (approximately 2.00 requests per second). Please visit https://console.pipellm.ai/billing to upgrade your plan for higher limits."
  }
}
```

## Rate Limit Headers

Every API response includes rate limit information:

| Header                           | Description                       |
| -------------------------------- | --------------------------------- |
| `x-ratelimit-limit-requests`     | Your current RPM limit            |
| `x-ratelimit-remaining-requests` | Remaining requests in this window |
| `x-ratelimit-reset-requests`     | Time until the window resets      |


# Streaming
Source: https://docs.pipellm.ai/api-reference/streaming

Stream chat tokens as they are generated.

Set `stream: true` on Chat Completions. PipeLLM returns [server-sent events](https://html.spec.whatwg.org/multipage/server-sent-events.html) in the same shape as OpenAI.

<CodeGroup>
  ```python Python theme={"dark"}
  import os
  from openai import OpenAI

  client = OpenAI(
      api_key=os.environ["PIPELLM_API_KEY"],
      base_url="https://api.pipellm.ai/v1",
  )

  stream = client.chat.completions.create(
      model="gpt-5",
      stream=True,
      messages=[{"role": "user", "content": "Count to five."}],
  )

  for chunk in stream:
      delta = chunk.choices[0].delta.content
      if delta:
          print(delta, end="", flush=True)
  ```

  ```typescript TypeScript theme={"dark"}
  import OpenAI from "openai";

  const client = new OpenAI({
    apiKey: process.env.PIPELLM_API_KEY,
    baseURL: "https://api.pipellm.ai/v1",
  });

  const stream = await client.chat.completions.create({
    model: "gpt-5",
    stream: true,
    messages: [{ role: "user", content: "Count to five." }],
  });

  for await (const chunk of stream) {
    const delta = chunk.choices[0]?.delta?.content;
    if (delta) process.stdout.write(delta);
  }
  ```

  ```bash cURL theme={"dark"}
  curl https://api.pipellm.ai/v1/chat/completions \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer $PIPELLM_API_KEY" \
    -N \
    -d '{
      "model": "gpt-5",
      "stream": true,
      "messages": [{"role": "user", "content": "Count to five."}]
    }'
  ```
</CodeGroup>

Anthropic and Gemini native routes also stream when you set their protocol flags (`stream: true` on Messages, or `:streamGenerateContent` on Gemini). Video generation is asynchronous and is **not** streamed — poll [`GET /v2/videos/{id}`](/api-reference/video/get) instead.

See [Chat Completions](/api-reference/openai/chat-completions) for the non-streaming response shape.


# Jev / System One
Source: https://docs.pipellm.ai/api-reference/systemone

POST /v1/systemone
Typed judgments with Jev.

Call `jev-1.13.0` over HTTP with your PipeLLM API key. This native route returns typed judgments. It does not use chat messages or support streaming.

<ParamField type="string">
  `jev-1.13.0`
</ParamField>

<ParamField type="string | object | array">
  Text or application state to evaluate.
</ParamField>

<ParamField type="object">
  Named questions with type and instructions. Supported types: noul, choice and score. Choice and score also require criteria.
</ParamField>

<RequestExample>
  ```bash cURL theme={"dark"}
  curl https://api.pipellm.ai/v1/systemone \
    -H "Authorization: Bearer $PIPELLM_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "model": "jev-1.13.0",
      "state": "The service is reachable.",
      "questions": {
        "reachable": {
          "type": "noul",
          "instructions": "Does the state say the service is reachable?"
        }
      }
    }'
  ```
</RequestExample>

Illustrative response; probabilities and usage vary by request.

<ResponseExample>
  ```json theme={"dark"}
  {
    "model": "jev-1.13.0",
    "answers": {"reachable": {"type": "noul", "noul": 0.99}},
    "usage": {"input_tokens": 300, "output_tokens": 20}
  }
  ```
</ResponseExample>

## Response and billing

Answer keys match your question names. Noul returns a probability from 0 to 1; choice returns an option and distribution; score returns a rating. Confidence is not a guarantee of correctness.

The actual model version and `usage.input_tokens` / `usage.output_tokens` are preserved. Current pricing is **`$0.042 / million input tokens`**, with no output charge. Check the console for current rates.

Malformed requests return `400`. Authentication, balance and rate-limit failures use the [standard error envelope](/api-reference/errors). See the [TypeSafe HTTP API](https://docs.typesafe.ai/api) for detailed question schemas; use the PipeLLM URL and key shown here.


# Cancel a task
Source: https://docs.pipellm.ai/api-reference/tasks/cancel

POST /v1/tasks/{task_id}/cancel
Cancel an asynchronous task that has not finished.

Stops a task that is still `pending` or `processing`. A task that already reached a
terminal state cannot be cancelled.

<Info>
  Video created through [`POST /v2/videos`](/api-reference/video/create) is cancelled
  with `DELETE /v2/videos/{id}` instead.
</Info>

<ParamField type="string">
  Task ID, as returned when the task was created.
</ParamField>

<RequestExample>
  ```bash cURL theme={"dark"}
  curl -X POST https://api.pipellm.ai/v1/tasks/gen-1732891234-abc123xyz/cancel \
    -H "Authorization: Bearer $PIPELLM_API_KEY"
  ```
</RequestExample>

<ResponseExample>
  ```json theme={"dark"}
  {
    "id": "gen-1732891234-abc123xyz",
    "status": "cancelled",
    "type": "video_generation",
    "model": "seedance-2-0-mini",
    "created_at": "2026-09-17T08:14:02Z"
  }
  ```
</ResponseExample>

The response is the same task object returned by
[`GET /v1/tasks/{task_id}`](/api-reference/tasks/get), with `status` set to `cancelled`.

## Billing

Cancelling does not refund work the upstream provider already performed. For video, the
quote is taken when the task is created — see [Pricing](/api-reference/model-pricing).

## Errors

Same envelope as [Errors](/api-reference/errors).

<AccordionGroup>
  <Accordion title="404 not_found">
    Unknown task ID, or a task belonging to another account.
  </Accordion>

  <Accordion title="409 invalid_request_error">
    The task already reached `completed`, `failed`, or `cancelled`.
  </Accordion>
</AccordionGroup>


# Get a task
Source: https://docs.pipellm.ai/api-reference/tasks/get

GET /v1/tasks/{task_id}
Poll the status of an asynchronous image, video, or audio task.

Media generation that cannot finish inside one request becomes a task. This route
returns the current state of one task and, once it finishes, its result.

<Info>
  Video created through [`POST /v2/videos`](/api-reference/video/create) has its own
  envelope and its own poll route, [`GET /v2/videos/{id}`](/api-reference/video/get).
  Use this route for tasks created by the `/v1` media endpoints.
</Info>

<ParamField type="string">
  Task ID. PipeLLM-managed tasks start with `gen-`. The upstream provider's own task ID
  is also accepted for tasks that were created directly against a provider.
</ParamField>

<RequestExample>
  ```bash cURL theme={"dark"}
  curl https://api.pipellm.ai/v1/tasks/gen-1732891234-abc123xyz \
    -H "Authorization: Bearer $PIPELLM_API_KEY"
  ```

  ```python Python theme={"dark"}
  import os, time, requests

  headers = {"Authorization": f"Bearer {os.environ['PIPELLM_API_KEY']}"}

  while True:
      task = requests.get(
          f"https://api.pipellm.ai/v1/tasks/{task_id}", headers=headers
      ).json()

      if task["status"] in ("completed", "failed", "cancelled"):
          break
      time.sleep(3)

  print(task["status"], task.get("data"), task.get("error"))
  ```
</RequestExample>

<ResponseExample>
  ```json theme={"dark"}
  {
    "id": "gen-1732891234-abc123xyz",
    "status": "processing",
    "type": "video_generation",
    "model": "seedance-2-0-mini",
    "created_at": "2026-09-17T08:14:02Z"
  }
  ```
</ResponseExample>

<ResponseField name="id" type="string">
  Task ID. Use it on this route and on [cancel](/api-reference/tasks/cancel).
</ResponseField>

<ResponseField name="status" type="string">
  One of `pending`, `processing`, `completed`, `failed`, or `cancelled`.
</ResponseField>

<ResponseField name="type" type="string">
  `image_generation`, `image_edit`, `video_generation`, or `audio_generation`.
</ResponseField>

<ResponseField name="model" type="string">
  Model that is producing the result.
</ResponseField>

<ResponseField name="created_at" type="string">
  When the task was created.
</ResponseField>

<ResponseField name="data" type="object">
  The result. Present once `status` is `completed`; its shape follows the task `type`.
</ResponseField>

<ResponseField name="error" type="string">
  Failure reason. Present when `status` is `failed`.
</ResponseField>

## Polling

A task moves `pending` → `processing` → one of `completed` / `failed` / `cancelled`.
Only those three are terminal; stop polling when you see one.

Poll every few seconds rather than in a tight loop — polls count against your
[rate limit](/api-reference/rate-limits), and polling faster does not make the
upstream finish sooner.

## Errors

Same envelope as [Errors](/api-reference/errors). An unknown ID returns `404`; a task
that belongs to another account is not visible to you and returns the same `404`.


# Cancel a video
Source: https://docs.pipellm.ai/api-reference/video/cancel

DELETE /v2/videos/{id}
Cancel a video task on the platform that created it. This request is not billed.

Cancels an in-flight task and returns the task envelope with `status` set to `cancelled` when the cancel succeeds.

<ParamField type="string">
  Public task ID (`gen-...`).
</ParamField>

<RequestExample>
  ```bash cURL theme={"dark"}
  curl -X DELETE https://api.pipellm.ai/v2/videos/gen-xxxxxxxx \
    -H "Authorization: Bearer $PIPELLM_API_KEY"
  ```

  ```python Python theme={"dark"}
  import os
  import requests

  response = requests.delete(
      "https://api.pipellm.ai/v2/videos/gen-xxxxxxxx",
      headers={"Authorization": f"Bearer {os.environ['PIPELLM_API_KEY']}"},
  )
  print(response.json())
  ```
</RequestExample>

<ResponseExample>
  ```json theme={"dark"}
  {
    "id": "gen-xxxxxxxx",
    "object": "video",
    "model": "seedance-2-0-mini",
    "provider": "volcengine",
    "status": "cancelled",
    "created_at": 1726400000,
    "updated_at": 1726400030
  }
  ```
</ResponseExample>

A missing or foreign task ID returns `404`. This route does not charge.

Completed tasks cannot be cancelled. Poll [`GET /v2/videos/{id}`](/api-reference/video/get) if you only need the current status.


# Create a video
Source: https://docs.pipellm.ai/api-reference/video/create

POST /v2/videos
Create an asynchronous video generation task.

Creates a video task and returns the provider-neutral task envelope. Poll [`GET /v2/videos/{id}`](/api-reference/video/get) until the task completes.

See [supported models and billing](/guides/video#supported-v2-models) for Mini, standard 2.0, Fast and 2.5 limits and a full poll loop.

<ParamField type="string">
  Use `doubao-seedance-2-0-mini-260615`, `seedance-2.0`, `doubao-seedance-2-0-fast-260128`, or `doubao-seedance-2-5-260628`. Mini also accepts `seedance-2-0-mini` and `seedance-2.0-mini`.
</ParamField>

<ParamField type="string">
  Text description of the video to generate.
</ParamField>

<ParamField type="array">
  Optional media inputs. Each item needs `type` and `url`. Mini allows at most 9 images, 3 videos, and 3 audios. Combined reference-video duration must be ≤ 15s. Standard, Fast and 2.5 accept up to 9 images only.

  <Expandable title="inputs[]">
    <ParamField type="string">
      `image_url`, `video_url`, or `audio_url`.
    </ParamField>

    <ParamField type="string">
      HTTPS URL of the media.
    </ParamField>

    <ParamField type="string">
      Protocol values: `first_frame`, `last_frame`, `reference_image`, `reference_video`, `reference_audio`. Mini images require `reference_image`. Mini videos/audio use `reference_video` / `reference_audio` or omit `role`.
    </ParamField>
  </Expandable>
</ParamField>

<ParamField type="integer">
  Output seconds: 4–15 for Mini, standard and Fast; 4–30 for 2.5. Default `5`.
</ParamField>

<ParamField type="string">
  `adaptive`, `16:9`, `9:16`, `1:1`, `4:3`, `3:4`, or `21:9`. Mini defaults to `16:9`.
</ParamField>

<ParamField type="string">
  `480p`, `720p`, or `1080p`. Only 2.5 also accepts `1080p`; the other models accept `480p` and `720p`. Default `480p`.
</ParamField>

<ParamField type="boolean">
  Whether to generate audio. Mini defaults to `false`.
</ParamField>

<ParamField type="object">
  Provider-keyed native options. Not supported on these four model profiles.
</ParamField>

<RequestExample>
  ```bash cURL theme={"dark"}
  curl https://api.pipellm.ai/v2/videos \
    -H "Authorization: Bearer $PIPELLM_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "model": "seedance-2-0-mini",
      "prompt": "A cat walking across a sunlit kitchen floor",
      "duration": 4,
      "aspect_ratio": "16:9",
      "resolution": "480p",
      "audio": false
    }'
  ```

  ```python Python theme={"dark"}
  import os
  import requests

  response = requests.post(
      "https://api.pipellm.ai/v2/videos",
      headers={
          "Authorization": f"Bearer {os.environ['PIPELLM_API_KEY']}",
          "Content-Type": "application/json",
      },
      json={
          "model": "seedance-2-0-mini",
          "prompt": "A cat walking across a sunlit kitchen floor",
          "duration": 4,
          "aspect_ratio": "16:9",
          "resolution": "480p",
          "audio": False,
      },
  )
  print(response.json())
  ```
</RequestExample>

<ResponseExample>
  ```json theme={"dark"}
  {
    "id": "gen-xxxxxxxx",
    "object": "video",
    "model": "seedance-2-0-mini",
    "provider": "volcengine",
    "status": "queued",
    "progress": 0,
    "created_at": 1726400000,
    "updated_at": 1726400000
  }
  ```
</ResponseExample>

Unknown fields are rejected. Duration lives on `outputs[]` after completion, not on the task top level.

<ResponseField name="id" type="string">
  Public task ID (`gen-...`). Use this on GET and DELETE. Upstream IDs stay internal.
</ResponseField>

<ResponseField name="object" type="string">
  Always `video`.
</ResponseField>

<ResponseField name="model" type="string">
  Model name from the request.
</ResponseField>

<ResponseField name="provider" type="string">
  Upstream provider that accepted the task, for example `volcengine`.
</ResponseField>

<ResponseField name="status" type="string">
  `queued`, `in_progress`, `completed`, `failed`, `cancelled`, or `expired`.
</ResponseField>

<ResponseField name="progress" type="integer">
  Optional 0–100 progress.
</ResponseField>

<ResponseField name="outputs" type="array">
  Present after `completed`. Duration is on each output, not the task.

  <Expandable title="outputs[]">
    <ResponseField name="type" type="string">
      `video`, `image`, or `audio`.
    </ResponseField>

    <ResponseField name="role" type="string">
      `result`, `last_frame`, `cover`, or `watermarked`.
    </ResponseField>

    <ResponseField name="url" type="string">
      Download URL for the artifact.
    </ResponseField>

    <ResponseField name="mime_type" type="string">
      For example `video/mp4`.
    </ResponseField>

    <ResponseField name="duration" type="number">
      Length of this video or audio output, in seconds.
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="error" type="object">
  Present when `status` is `failed`. Contains `code` and `message`.
</ResponseField>

## Errors

Mini and standard 2.0 are billed after acceptance. Fast and 2.5 settle on completion from actual video tokens. Validation failures return `400` and are not billed. Same envelope as [Errors](/api-reference/errors). Unknown JSON fields are rejected.

<AccordionGroup>
  <Accordion title="400 invalid_request_error">
    Missing `model` / `prompt`, unsupported Mini duration or resolution, wrong input `role`, too many references, or `provider_options` on Mini.

    ```json theme={"dark"}
    {
      "error": {
        "type": "invalid_request_error",
        "code": "400",
        "message": "Mini image inputs require role=reference_image"
      }
    }
    ```
  </Accordion>

  <Accordion title="401 authentication_error">
    ```json theme={"dark"}
    {
      "error": {
        "type": "authentication_error",
        "code": "401",
        "message": "Incorrect API key provided. Please visit https://console.pipellm.ai/account/api-keys to find your API key."
      }
    }
    ```
  </Accordion>

  <Accordion title="402 insufficient_balance">
    Quote is taken at create time. Insufficient balance rejects the request before a task is created.

    ```json theme={"dark"}
    {
      "error": {
        "type": "insufficient_balance",
        "code": "402",
        "message": "Insufficient balance. Please recharge your account at https://console.pipellm.ai/billing."
      }
    }
    ```
  </Accordion>

  <Accordion title="429 rate_limit_error">
    See [Rate Limits](/api-reference/rate-limits).

    ```json theme={"dark"}
    {
      "error": {
        "type": "rate_limit_error",
        "code": "429",
        "message": "Rate limit exceeded. Your current limit is 120 requests per minute (approximately 2.00 requests per second). Please visit https://console.pipellm.ai/billing to upgrade your plan for higher limits."
      }
    }
    ```
  </Accordion>
</AccordionGroup>


# Create a video (V1)
Source: https://docs.pipellm.ai/api-reference/video/create-v1

POST /v1/videos/generations
Legacy video generation through the PipeLLM gateway.

<Warning>
  V1 is planned for retirement. A shutdown date has not been announced. For new integrations, prefer [V2](/api-reference/video/create) when your model is supported.

  Before migrating, check the [V2 model list and request schema](/guides/video). V1 models are not automatically supported by V2; changing the URL alone is not a migration. Keep using the documented V1 route for models that have not yet been migrated.
</Warning>

Use `POST https://api.pipellm.ai/v1/videos/generations` for legacy video models, including WaveSpeed Kling models. Supplier paths such as `/api/v3/kwaivgi/...` are internal routing details, not the public PipeLLM endpoint.

```bash theme={"dark"}
curl https://api.pipellm.ai/v1/videos/generations \
  -H "Authorization: Bearer $PIPELLM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"kling-o1-text-to-video","prompt":"A blue ball rolling on a table"}'
```

Include the public model ID in `model`. Other required fields, supported options, and the asynchronous response depend on the selected model. This example shows the gateway request envelope; supply any additional fields required by that model.

Seedance models listed in the [V2 guide](/guides/video#supported-v2-models) use [`POST /v2/videos`](/api-reference/video/create). Do not mix V1 model-specific request bodies with the V2 schema.


# Get a video
Source: https://docs.pipellm.ai/api-reference/video/get

GET /v2/videos/{id}
Poll a video task. This request is not billed.

Returns the same task envelope as create. Query is pinned to the platform that created the task.

<ParamField type="string">
  Public task ID returned by create (`gen-...`).
</ParamField>

<RequestExample>
  ```bash cURL theme={"dark"}
  curl https://api.pipellm.ai/v2/videos/gen-xxxxxxxx \
    -H "Authorization: Bearer $PIPELLM_API_KEY"
  ```

  ```python Python theme={"dark"}
  import os
  import requests

  response = requests.get(
      "https://api.pipellm.ai/v2/videos/gen-xxxxxxxx",
      headers={"Authorization": f"Bearer {os.environ['PIPELLM_API_KEY']}"},
  )
  print(response.json())
  ```
</RequestExample>

<ResponseExample>
  ```json theme={"dark"}
  {
    "id": "gen-xxxxxxxx",
    "object": "video",
    "model": "seedance-2-0-mini",
    "provider": "volcengine",
    "status": "completed",
    "progress": 100,
    "created_at": 1726400000,
    "updated_at": 1726400120,
    "completed_at": 1726400120,
    "outputs": [
      {
        "type": "video",
        "role": "result",
        "url": "https://example.com/output.mp4",
        "mime_type": "video/mp4",
        "duration": 4.0
      }
    ]
  }
  ```
</ResponseExample>

<ResponseField name="status" type="string">
  `queued`, `in_progress`, `completed`, `failed`, `cancelled`, or `expired`.
</ResponseField>

<ResponseField name="progress" type="integer">
  Optional 0–100 progress.
</ResponseField>

<ResponseField name="outputs" type="array">
  Present when the task completed. Duration is on each output, not the task.
</ResponseField>

<ResponseField name="error" type="object">
  Present when `status` is `failed`. Contains `code` and `message`.
</ResponseField>

<ResponseField name="usage" type="object">
  Optional provider metering (`completion_tokens`, `total_tokens`). Customer billing uses the create-time quote, not this object.
</ResponseField>

A missing or foreign task ID returns `404`. This route does not charge.


# Anthropic Format Converter
Source: https://docs.pipellm.ai/converter/anthropic-format

Use Gemini and OpenAI models with Anthropic SDK

## Overview

The Anthropic Format Converter allows you to call **Gemini** and **OpenAI** models using the Anthropic SDK and API format. This is useful when you have existing code using the Anthropic SDK but want to access other providers' models.

## Configuration

### SDK Configuration

Set your base URL to:

```
https://api.pipellm.ai/anthropic
```

<Note>
  The SDK automatically appends `/v1/messages`, so you only need to set the base
  URL to `/anthropic`.
</Note>

### cURL / Direct API

For direct API calls, use the full endpoint:

```
https://api.pipellm.ai/anthropic/v1/messages
```

## Usage Examples

<Tabs>
  <Tab title="Python SDK">
    ```python theme={"dark"}
    import anthropic

    client = anthropic.Anthropic(
        api_key="your-pipellm-api-key",
        base_url="https://api.pipellm.ai/anthropic"
    )

    message = client.messages.create(
        model="gemini-3-flash-preview",  # or "gpt-5"
        max_tokens=1024,
        messages=[
            {"role": "user", "content": "Hello, how are you?"}
        ]
    )

    print(message.content[0].text)
    ```
  </Tab>

  <Tab title="TypeScript SDK">
    ```typescript theme={"dark"}
    import Anthropic from "@anthropic-ai/sdk";

    const client = new Anthropic({
      apiKey: "your-pipellm-api-key",
      baseURL: "https://api.pipellm.ai/anthropic",
    });

    const message = await client.messages.create({
      model: "gemini-3-flash-preview", // or 'gpt-5'
      max_tokens: 1024,
      messages: [{ role: "user", content: "Hello, how are you?" }],
    });

    console.log(message.content[0].text);
    ```
  </Tab>

  <Tab title="cURL">
    ```bash theme={"dark"}
    curl https://api.pipellm.ai/anthropic/v1/messages \
      -H "Content-Type: application/json" \
      -H "x-api-key: your-pipellm-api-key" \
      -H "anthropic-version: 2023-06-01" \
      -d '{
        "model": "gemini-3-flash-preview",
        "max_tokens": 1024,
        "messages": [
          {"role": "user", "content": "Hello, how are you?"}
        ]
      }'
    ```
  </Tab>
</Tabs>

## Supported Models

You can use any Gemini or OpenAI model available on PipeLLM with the Anthropic format:

| Provider | Example Models           |
| -------- | ------------------------ |
| Gemini   | `gemini-3-flash-preview` |
| OpenAI   | `gpt-5`                  |

## Feature Support

| Feature        | Status      |
| -------------- | ----------- |
| Streaming      | ✅ Supported |
| Tool Use       | ✅ Supported |
| Vision         | ✅ Supported |
| System Prompts | ✅ Supported |
| Thinking       | ✅ Supported |

## Gemini 3 Thinking and Thought Signatures

Gemini 3 models return a `thoughtSignature` on function call parts and reject
the next request in the same turn with `400` if the signature is not sent back
on the same part. This is a native model requirement; see Google's
[Thought signatures](https://ai.google.dev/gemini-api/docs/generate-content/thought-signatures) guide.

How this converter carries signatures:

* **Non-streaming:** the signature is returned in the `signature` field of
  `thinking` blocks and in a non-standard `signature` field on `tool_use`
  blocks. It is restored when you send those blocks back unchanged. If your
  code rebuilds `tool_use` blocks from `id`, `name`, and `input` only, the
  signature is lost.
* **Streaming:** `tool_use` blocks do not carry a signature. A signature is only
  emitted as a `signature_delta` on a preceding `thinking` block, which does not
  restore it to the function call on replay.

<Warning>
  Function calling with Gemini 3 models over this format can fail with `400`
  when tool results are returned, especially with streaming. For Gemini 3 tool
  use, prefer the [native Gemini route](/api-reference/gemini/overview) or the
  [Responses format](/converter/responses-format#thinking-and-signatures).
</Warning>

For Gemini 2.5 models, returning signatures is optional.

### Thinking levels

For Gemini 3 targets, enabled `thinking.budget_tokens` is mapped to a thinking level: up to 1,024 → `LOW`, 1,025–8,192 → `MEDIUM`, and above 8,192 → `HIGH`. This is an approximation, not a Gemini 3 token budget. Gemini 2.5 uses a numeric `thinkingBudget`. The upstream model must support the mapped level.


# Gemini Format Converter
Source: https://docs.pipellm.ai/converter/gemini-format

Use OpenAI and Anthropic models with Gemini SDK

## Overview

The Gemini Format Converter allows you to call **OpenAI** and **Anthropic** models using the Gemini SDK and API format. This is useful when you have existing code using the Gemini SDK but want to access other providers' models.

## Configuration

### SDK Configuration

Set your base URL to:

```
https://api.pipellm.ai/gemini
```

<Note>
  The SDK automatically appends the model path, so you only need to set the base
  URL to `/gemini`.
</Note>

### cURL / Direct API

For direct API calls, use the full endpoint:

```
https://api.pipellm.ai/gemini/v1beta/models/{model}:generateContent
```

## Usage Examples

<Tabs>
  <Tab title="Python SDK">
    ```python theme={"dark"}
    import google.generativeai as genai
    from google.generativeai import types

    # Configure with PipeLLM
    genai.configure(
        api_key="your-pipellm-api-key",
        transport="rest",
        client_options={"api_endpoint": "api.pipellm.ai/gemini"}
    )

    # Use OpenAI or Anthropic models
    model = genai.GenerativeModel("gpt-5")  # or "claude-sonnet-4-6"

    response = model.generate_content("Hello, how are you?")
    print(response.text)
    ```
  </Tab>

  <Tab title="cURL">
    ```bash theme={"dark"}
    curl "https://api.pipellm.ai/gemini/v1beta/models/gpt-5:generateContent" \
      -H "Content-Type: application/json" \
      -H "x-goog-api-key: your-pipellm-api-key" \
      -d '{
        "contents": [
          {"role": "user", "parts": [{"text": "Hello, how are you?"}]}
        ]
      }'
    ```
  </Tab>
</Tabs>

## Supported Models

You can use any OpenAI or Anthropic model available on PipeLLM with the Gemini format:

| Provider  | Example Models      |
| --------- | ------------------- |
| OpenAI    | `gpt-5`             |
| Anthropic | `claude-sonnet-4-6` |

## Feature Support

| Feature        | Status                                                        |
| -------------- | ------------------------------------------------------------- |
| Streaming      | ⚠️ Cross-protocol output to Gemini clients is not implemented |
| Tool Use       | ✅ Supported                                                   |
| Vision         | ✅ Supported                                                   |
| System Prompts | ✅ Supported                                                   |
| Thinking       | ✅ Supported                                                   |

## Gemini 3 Models

This converter targets OpenAI and Anthropic models, so Gemini
[thought signature](https://ai.google.dev/gemini-api/docs/generate-content/thought-signatures) rules do not apply to it. To call Gemini 3 models
with the Gemini SDK, use the [native Gemini route](/api-reference/gemini/overview).

<Warning>
  Cross-protocol streaming output in Gemini format is not implemented. Use `generateContent` on this converter route; use the [native Gemini API](/api-reference/gemini/overview) for streaming Gemini models.
</Warning>


# OpenAI Format Converter
Source: https://docs.pipellm.ai/converter/openai-format

Use Gemini and Anthropic models with OpenAI SDK

## Overview

The OpenAI Format Converter allows you to call **Gemini** and **Anthropic** models using the OpenAI SDK and API format. This is useful when you have existing code using the OpenAI SDK but want to access other providers' models.

## Configuration

### SDK Configuration

Set your base URL to:

```
https://api.pipellm.ai/openai/v1
```

<Note>
  The SDK automatically appends `/chat/completions`, so you only need to set the
  base URL to `/openai/v1`.
</Note>

### cURL / Direct API

For direct API calls, use the full endpoint:

```
https://api.pipellm.ai/openai/v1/chat/completions
```

## Usage Examples

<Tabs>
  <Tab title="Python SDK">
    ```python theme={"dark"}
    from openai import OpenAI

    client = OpenAI(
        api_key="your-pipellm-api-key",
        base_url="https://api.pipellm.ai/openai/v1"
    )

    response = client.chat.completions.create(
        model="gemini-3-flash-preview",  # or "claude-sonnet-4-6"
        messages=[
            {"role": "user", "content": "Hello, how are you?"}
        ]
    )

    print(response.choices[0].message.content)
    ```
  </Tab>

  <Tab title="TypeScript SDK">
    ```typescript theme={"dark"}
    import OpenAI from "openai";

    const client = new OpenAI({
      apiKey: "your-pipellm-api-key",
      baseURL: "https://api.pipellm.ai/openai/v1",
    });

    const response = await client.chat.completions.create({
      model: "gemini-3-flash-preview", // or 'claude-sonnet-4-6'
      messages: [{ role: "user", content: "Hello, how are you?" }],
    });

    console.log(response.choices[0].message.content);
    ```
  </Tab>

  <Tab title="cURL">
    ```bash theme={"dark"}
    curl https://api.pipellm.ai/openai/v1/chat/completions \
      -H "Content-Type: application/json" \
      -H "Authorization: Bearer your-pipellm-api-key" \
      -d '{
        "model": "gemini-3-flash-preview",
        "messages": [
          {"role": "user", "content": "Hello, how are you?"}
        ]
      }'
    ```
  </Tab>
</Tabs>

## Supported Models

You can use any Gemini or Anthropic model available on PipeLLM with the OpenAI format:

| Provider  | Example Models           |
| --------- | ------------------------ |
| Gemini    | `gemini-3-flash-preview` |
| Anthropic | `claude-sonnet-4-6`      |

## Feature Support

| Feature        | Status      |
| -------------- | ----------- |
| Streaming      | ✅ Supported |
| Tool Use       | ✅ Supported |
| Vision         | ✅ Supported |
| System Prompts | ✅ Supported |
| Thinking       | ✅ Supported |

## Gemini 3 Thinking and Thought Signatures

Gemini 3 models return a `thoughtSignature` on function call parts and reject
the next request in the same turn with `400` if the signature is not sent back
on the same part. This is a native model requirement; see Google's
[Thought signatures](https://ai.google.dev/gemini-api/docs/generate-content/thought-signatures) guide.

<Warning>
  The Chat Completions format has no per-tool-call field for these signatures,
  and this converter does not replay them. A response exposes only its first
  signature as a non-standard top-level `thought_signature` field, and
  `tool_calls` you send back in `messages` reach Gemini without signatures.
  Function calling with Gemini 3 models can therefore fail when you return tool
  results. For Gemini 3 tool use, prefer the
  [native Gemini route](/api-reference/gemini/overview) or the
  [Responses format](/converter/responses-format#thinking-and-signatures),
  which replays signatures per function call.
</Warning>

Plain chat, streaming, and `reasoning_content` output with Gemini 3 are not
affected. For Gemini 2.5 models, returning signatures is optional.

### Thinking levels

For Gemini 3 targets, `reasoning_effort` values `low`, `medium`, and `high` map to `generationConfig.thinkingConfig.thinkingLevel`. The `max_reasoning_tokens` extension takes precedence when present and is mapped to a level, not an exact token budget. Gemini 2.5 uses a numeric `thinkingBudget`. The upstream model must support the requested level.


# Protocol Conversion Overview
Source: https://docs.pipellm.ai/converter/overview

Protocol conversion routes for OpenAI Chat Completions, OpenAI Responses, Anthropic, and Gemini request formats across Relay

Relay converters let developers keep their existing SDK format while calling
models from another provider family.

## When to Use a Converter

Use a converter route when:

* your application already speaks OpenAI, Anthropic, or Gemini format
* you want to switch providers without rewriting request and response handling
* the native public route does not support the protocol family you need

If your request already matches the target provider protocol, use the native
public route instead. See [Routing & Protocols](/guides/routing-protocols).

## Converter Matrix

| Request format          | Base URL                              | Main endpoint                            | Typical use                                                                                 |
| ----------------------- | ------------------------------------- | ---------------------------------------- | ------------------------------------------------------------------------------------------- |
| OpenAI format           | `https://api.pipellm.ai/openai/v1`    | `/chat/completions`                      | Keep the OpenAI SDK and call Claude or Gemini models                                        |
| OpenAI Responses format | `https://api.pipellm.ai/responses/v1` | `/responses`                             | Keep Responses clients such as the OpenAI SDK or Codex CLI and call Claude or Gemini models |
| Anthropic format        | `https://api.pipellm.ai/anthropic`    | `/v1/messages`                           | Keep the Anthropic SDK and call GPT or Gemini models                                        |
| Gemini format           | `https://api.pipellm.ai/gemini`       | `/v1beta/models/{model}:generateContent` | Keep the Gemini SDK and call GPT or Claude models                                           |

## Current Limitations

* The Responses converter is stateless and partial: no `previous_response_id`,
  server-side storage, hosted tools, or structured output formats. See
  [Responses Format](/converter/responses-format#not-supported).
* Gemini 3 models require thought signatures to be replayed on function calls.
  How well each format preserves them differs; check the Gemini 3 note on the
  format page you use. For multi-step tool use with Gemini 3, prefer the
  [native Gemini route](/api-reference/gemini/overview) or the
  [Responses format](/converter/responses-format#thinking-and-signatures).

## Pick a Format

<Columns>
  <Card title="OpenAI Format" icon="bolt" href="/converter/openai-format">
    Use OpenAI SDK format with Anthropic or Gemini models
  </Card>

  <Card title="Responses Format" icon="sparkles" href="/converter/responses-format">
    Use OpenAI Responses clients with Anthropic or Gemini models
  </Card>

  <Card title="Anthropic Format" icon="message" href="/converter/anthropic-format">
    Use Anthropic SDK format with OpenAI or Gemini models
  </Card>

  <Card title="Gemini Format" icon="sparkles" href="/converter/gemini-format">
    Use Gemini SDK format with OpenAI or Anthropic models
  </Card>
</Columns>


# Responses Format Converter
Source: https://docs.pipellm.ai/converter/responses-format

Use Claude, Gemini, and Chat Completions models with the OpenAI Responses API format

## Overview

The Responses Format Converter lets OpenAI Responses clients — the OpenAI SDK
`client.responses` API, Codex CLI, and other Responses-based tools — call
**Anthropic**, **Gemini**, and **Chat Completions** models. Models that natively
support the Responses API are passed through unchanged on the same route.

<Warning>
  This route is a **stateless, partial** implementation of the Responses API.
  It converts text, images, function calling, reasoning, and streaming. It does
  not provide server-side response storage or OpenAI hosted tools. See
  [Not Supported](#not-supported) before migrating a workflow.
</Warning>

## Configuration

### SDK Configuration

Set your base URL to:

```
https://api.pipellm.ai/responses/v1
```

### cURL / Direct API

```
https://api.pipellm.ai/responses/v1/responses
```

## Usage Examples

<Tabs>
  <Tab title="Python SDK">
    ```python theme={"dark"}
    from openai import OpenAI

    client = OpenAI(
        api_key="your-pipellm-api-key",
        base_url="https://api.pipellm.ai/responses/v1"
    )

    response = client.responses.create(
        model="claude-sonnet-4-6",  # or "gemini-3-flash-preview"
        input="Hello, how are you?",
        store=False,
    )

    print(response.output_text)
    ```
  </Tab>

  <Tab title="TypeScript SDK">
    ```typescript theme={"dark"}
    import OpenAI from "openai";

    const client = new OpenAI({
      apiKey: "your-pipellm-api-key",
      baseURL: "https://api.pipellm.ai/responses/v1",
    });

    const response = await client.responses.create({
      model: "claude-sonnet-4-6", // or "gemini-3-flash-preview"
      input: "Hello, how are you?",
      store: false,
    });

    console.log(response.output_text);
    ```
  </Tab>

  <Tab title="cURL">
    ```bash theme={"dark"}
    curl https://api.pipellm.ai/responses/v1/responses \
      -H "Content-Type: application/json" \
      -H "Authorization: Bearer your-pipellm-api-key" \
      -d '{
        "model": "claude-sonnet-4-6",
        "input": "Hello, how are you?",
        "store": false
      }'
    ```
  </Tab>
</Tabs>

## Stateless Multi-Turn

Converted models keep no server-side conversation state. Send `store: false`
(or omit `store`), and carry the history yourself: on every turn, append the
**complete** `response.output` array of the previous response to `input`,
followed by your new items.

```python theme={"dark"}
tools = [{
    "type": "function",
    "name": "get_weather",
    "description": "Get the weather for a city",
    "parameters": {
        "type": "object",
        "properties": {"city": {"type": "string"}},
        "required": ["city"],
    },
}]

history = [{"role": "user", "content": "What is the weather in Paris?"}]

first = client.responses.create(
    model="gemini-3-flash-preview",
    input=history,
    tools=tools,
    store=False,
    include=["reasoning.encrypted_content"],
)

# Replay every output item as returned: reasoning items (with
# encrypted_content), function calls, and messages.
history += [item.model_dump(exclude_none=True) for item in first.output]

for item in first.output:
    if item.type == "function_call":
        history.append({
            "type": "function_call_output",
            "call_id": item.call_id,
            "output": '{"temperature_c": 18}',
        })

final = client.responses.create(
    model="gemini-3-flash-preview",
    input=history,
    tools=tools,
    store=False,
    include=["reasoning.encrypted_content"],
)
print(final.output_text)
```

Replay rules:

* Do not filter, reorder, or rewrite output items. Reasoning items may have an
  empty `summary` and exist only to carry `encrypted_content`; they must stay
  directly before the function call they were returned with.
* Every `function_call_output` must reference a `call_id` that appears in the
  same request's `input`. Duplicate `call_id` values in one history are rejected
  with `400`.
* `encrypted_content` is opaque provider state — Anthropic thinking signatures
  or Gemini thought signatures. It is only valid for the provider family that
  issued it. Content issued by another provider is ignored rather than replayed.

## Thinking and Signatures

| Target    | What `encrypted_content` carries                                          | If it is not replayed                                                                           |
| --------- | ------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------- |
| Anthropic | Each thinking block’s `signature`, or opaque `data` for redacted thinking | The earlier thinking is left out of the upstream request; the conversation continues without it |
| Gemini    | The `thoughtSignature` of function call parts and single thought parts    | Gemini 3 models reject the function call turn with `400`                                        |

Gemini 3 models validate thought signatures on function calls in the current
turn. This is a native model requirement, not a converter rule; see Google's
[Thought signatures](https://ai.google.dev/gemini-api/docs/generate-content/thought-signatures)
guide. The converter's part is to return each signature inside a reasoning item
and put it back on the exact function call part when you replay the item.

<Warning>
  **Known limitation: Gemini 3 signatures across upstream channels.** PipeLLM
  may serve consecutive requests for the same model through different upstream
  channels, and this route has no channel affinity. In our validation with the
  official OpenAI SDK, the client replayed `encrypted_content` correctly and
  replays that stayed on the same channel succeeded, but a replay that landed on
  a different channel was rejected upstream with
  `400 Thought signature is not valid`. This limitation is not resolved yet:
  a retry may succeed but is not guaranteed, so handle this error in Gemini 3
  tool loops.
</Warning>

### Gemini thinking levels

For Gemini 3 targets, `reasoning.effort` accepts `low`, `medium`, or `high` and maps to `thinkingLevel`; Gemini 2.5 uses `thinkingBudget`. Native model levels such as `minimal` are not exposed by this conversion path. The upstream model must support the chosen level.

## Supported Features

| Feature                                                                             | Status                                                                                                                      |
| ----------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- |
| Text and image input (`input_text`, `input_image` with URL or data URL)             | ✅ Supported                                                                                                                 |
| `instructions`, `system` / `developer` messages                                     | ✅ Supported                                                                                                                 |
| Streaming (Responses SSE events)                                                    | ✅ Supported                                                                                                                 |
| Function calling, including streamed arguments                                      | ✅ Supported                                                                                                                 |
| `reasoning.effort`, `reasoning.summary`, `include: ["reasoning.encrypted_content"]` | ✅ Supported                                                                                                                 |
| `custom`, `shell` / `local_shell`, `apply_patch`, `namespace` tools                 | ⚠️ Mapped to function tools and restored in the output; custom tool grammars are passed as a description only, not enforced |
| Function tool `strict`                                                              | ⚠️ Only effective for OpenAI targets                                                                                        |
| `usage`                                                                             | ✅ `input_tokens` includes cached tokens, `output_tokens` includes reasoning tokens                                          |

A stream always ends with exactly one terminal event: `response.completed`,
`response.incomplete` (output limit or content filter), or `response.failed`
(upstream error or truncated stream).

`prompt_cache_key`, `client_metadata`, and
`stream_options.include_obfuscation` are accepted but have no effect.

## Not Supported

For converted models, anything that cannot be expressed in the target protocol
returns `400` instead of being silently dropped:

| Feature                                                                                               | Behavior |
| ----------------------------------------------------------------------------------------------------- | -------- |
| `store: true`, `previous_response_id`, `conversation`, `background: true`                             | `400`    |
| `GET` / `DELETE /responses/{id}` and other stored-response endpoints                                  | `404`    |
| Hosted tools: `web_search`, `file_search`, `code_interpreter`, `mcp`, image generation, computer use  | `400`    |
| Structured outputs (`text.format` other than plain text), `text.verbosity`                            | `400`    |
| `input_file`, image `file_id`, `item_reference`                                                       | `400`    |
| `truncation: "auto"`, non-empty `metadata`, `tool_choice` with `allowed_tools`                        | `400`    |
| Unknown top-level fields and unknown fields on tools or input items                                   | `400`    |
| Gemini targets: `parallel_tool_calls: false`, `reasoning.effort` other than `low` / `medium` / `high` | `400`    |

If you need these features with an OpenAI model, use the native
[`/v1/responses`](/api-reference/openai/responses) route.

## Codex CLI

Codex CLI works with this route using a custom provider with
`wire_api = "responses"` and the base URL above. Disable hosted web search for
converted models (`web_search = "disabled"`), because hosted tools are rejected.

## Related Docs

<Columns>
  <Card title="Converters Overview" icon="shuffle" href="/converter/overview">
    All converter routes
  </Card>

  <Card title="Native Responses" icon="sparkles" href="/api-reference/openai/responses">
    `/v1/responses` for OpenAI-compatible models
  </Card>

  <Card title="Routing & Protocols" icon="route" href="/guides/routing-protocols">
    Native routes versus converter routes
  </Card>
</Columns>


# Chat Completions
Source: https://docs.pipellm.ai/guides/chat

Send a chat completion through PipeLLM.

The default chat path is OpenAI Chat Completions:

```
POST https://api.pipellm.ai/v1/chat/completions
```

<CodeGroup>
  ```python Python theme={"dark"}
  import os
  from openai import OpenAI

  client = OpenAI(
      api_key=os.environ["PIPELLM_API_KEY"],
      base_url="https://api.pipellm.ai/v1",
  )

  response = client.chat.completions.create(
      model="gpt-5",
      messages=[
          {"role": "system", "content": "You are a helpful assistant."},
          {"role": "user", "content": "Explain rate limits in one sentence."},
      ],
  )
  print(response.choices[0].message.content)
  ```

  ```typescript TypeScript theme={"dark"}
  import OpenAI from "openai";

  const client = new OpenAI({
    apiKey: process.env.PIPELLM_API_KEY,
    baseURL: "https://api.pipellm.ai/v1",
  });

  const response = await client.chat.completions.create({
    model: "gpt-5",
    messages: [
      { role: "system", content: "You are a helpful assistant." },
      { role: "user", content: "Explain rate limits in one sentence." },
    ],
  });
  console.log(response.choices[0].message.content);
  ```

  ```bash cURL theme={"dark"}
  curl https://api.pipellm.ai/v1/chat/completions \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer $PIPELLM_API_KEY" \
    -d '{
      "model": "gpt-5",
      "messages": [
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain rate limits in one sentence."}
      ]
    }'
  ```
</CodeGroup>

Pick `model` from [`GET /v1/models`](/api-reference/list-models). Native `/v1/chat/completions` only routes to OpenAI-compatible platforms. To keep this request shape while calling Claude or Gemini, use the [OpenAI converter](/converter/openai-format).

## Related

<Columns>
  <Card title="Streaming" href="/guides/streaming">
    Set `stream: true` and read SSE chunks.
  </Card>

  <Card title="Chat Completions API" href="/api-reference/openai/chat-completions">
    Parameters, function calling, and response fields.
  </Card>

  <Card title="Responses API" href="/api-reference/openai/responses">
    OpenAI Responses on PipeLLM.
  </Card>

  <Card title="Routing" href="/guides/routing-protocols">
    Native routes versus converter routes.
  </Card>
</Columns>


# Images
Source: https://docs.pipellm.ai/guides/images

Generate and edit images with GPT and Gemini image models.

There are two ways to generate images, and which one you use depends on the model family.

| Family              | Path                                          | Reference                                                    |
| ------------------- | --------------------------------------------- | ------------------------------------------------------------ |
| GPT image models    | `POST /v1/images/generations`                 | [Create an image](/api-reference/image/generations)          |
| Gemini image models | `POST /v1beta/models/{model}:generateContent` | [Generate an image with Gemini](/api-reference/image/gemini) |

The OpenAI-compatible route is the familiar one: point the OpenAI SDK at `https://api.pipellm.ai/v1` and call it unchanged. It also covers [editing](/api-reference/image/edits) an existing image.

Gemini image models have no `/v1/images/*` route of their own. They generate through the same `generateContent` endpoint as text models and return the image as an inline part. There is no `/v2/images` route on PipeLLM.

See [Nano Banana](/use-cases/nano-banana) for the full Gemini request and response examples.

| Model                         | Typical use                       |
| ----------------------------- | --------------------------------- |
| `gemini-3.1-flash-lite-image` | Cheapest and fastest              |
| `gemini-3.1-flash-image`      | Fast image generation             |
| `gemini-3-pro-image`          | Higher-fidelity production images |

```bash theme={"dark"}
curl -X POST \
  "https://api.pipellm.ai/v1beta/models/gemini-3.1-flash-image:generateContent" \
  -H "x-goog-api-key: $PIPELLM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "contents": [{
      "role": "user",
      "parts": [{"text": "A red bicycle leaning against a cafe window"}]
    }]
  }'
```

Confirm the current image model IDs with [`GET /v1/models`](/api-reference/list-models). For video, use [`POST /v2/videos`](/guides/video).


# Prompt Caching
Source: https://docs.pipellm.ai/guides/prompt-caching

Reuse a long prefix across requests and pay less for it.

When many requests share a long prefix — a system prompt, a tool schema, a document you
keep asking about — caching lets the provider reuse that work instead of reprocessing it.
Cached input is billed at a much lower rate than fresh input.

PipeLLM passes caching through to the upstream provider and meters it separately, so the
savings land on your bill. How you turn it on depends on the protocol.

## Anthropic: explicit

Anthropic caching is opt-in. Mark the end of the prefix you want cached with
`cache_control`, and everything before that point becomes the cache key.

```json theme={"dark"}
{
  "model": "claude-sonnet-4-6",
  "system": [
    {
      "type": "text",
      "text": "<a long style guide>",
      "cache_control": { "type": "ephemeral" }
    }
  ],
  "messages": [{ "role": "user", "content": "Rewrite this paragraph: ..." }]
}
```

The first request pays a **write** price; later requests that match the prefix pay the much
cheaper **read** price. Caches are short-lived — a five-minute window by default, with a
one-hour option priced separately.

Responses report what happened:

```json theme={"dark"}
"usage": {
  "cache_creation_input_tokens": 12043,
  "cache_read_input_tokens": 0
}
```

`cache_creation_input_tokens` means you paid to write. `cache_read_input_tokens` means you
got a hit.

<Warning>
  **Bedrock and Vertex AI accept at most 4 `cache_control` blocks.** When a request carries
  more, PipeLLM keeps the last four — the ones closest to the end of the prompt, which are
  usually the most valuable — and drops the earlier ones before forwarding. The request
  still succeeds, so a request that works on Anthropic's own API can quietly cache less
  here. Keep to four breakpoints if you route across providers.
</Warning>

## OpenAI and Gemini: automatic

No request field to set. The provider detects a repeated prefix on its own and applies a
discount to the cached portion. OpenAI reports it under
`usage.prompt_tokens_details.cached_tokens`.

Because it is automatic, the only thing you control is prompt shape — see below.

## Writing cacheable prompts

Caching keys on an exact prefix match, so the rule is simple: **put what is stable first
and what varies last.**

* Static system prompt, tool definitions, and reference documents at the top.
* The user's current turn at the bottom.
* Nothing variable early on. A timestamp, a request ID, or a shuffled list near the start
  invalidates the whole prefix and you pay full price every time.

A cache only pays off when it is reused inside its lifetime. One-off requests cost slightly
*more* with an explicit cache write, so reserve `cache_control` for prefixes you will
actually hit again within a few minutes.

## Checking the price

Cache rates are per model. Look them up with
[`GET /v1/models/pricing`](/api-reference/model-pricing) — the response carries them
alongside the regular token prices:

```json theme={"dark"}
"pricing": {
  "kind": "token",
  "text": { "prompt": "0.000003", "completion": "0.000015" },
  "cache": {
    "read": "0.0000003",
    "write": "0.00000375",
    "write_1h": "0.0000075"
  }
}
```

A model with no `cache` object does not support caching on the mapping you are routed to.

## Related

* [Get model pricing](/api-reference/model-pricing) — authoritative cache rates
* [Anthropic Messages](/api-reference/anthropic/messages) — where `cache_control` goes


# Routing & Protocols
Source: https://docs.pipellm.ai/guides/routing-protocols

Choose the correct native route or converter route for each request format

Relay exposes two kinds of public routes:

* Native public routes on `https://api.pipellm.ai`
* Converter routes under `/openai`, `/responses`, `/anthropic`, and `/gemini`

Use a native route only when your request already matches the target protocol.
Use a converter route when you want to keep one SDK format and call models from
another protocol family.

## Route Matrix

| Request format                 | Native route                                   | Converter route                                       | Notes                                                                                                                                                                                               |
| ------------------------------ | ---------------------------------------------- | ----------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| OpenAI Chat Completions        | `/v1/chat/completions`                         | `/openai/v1/chat/completions`                         | Native route for OpenAI-compatible platforms. Converter route for Anthropic or Gemini models with OpenAI format.                                                                                    |
| OpenAI Responses               | `/v1/responses`                                | `/responses/v1/responses`                             | Native route for OpenAI-compatible platforms. Converter route for Anthropic or Gemini models with Responses format; stateless, with [documented limits](/converter/responses-format#not-supported). |
| Anthropic Messages             | `/v1/messages`                                 | `/anthropic/v1/messages`                              | Native route for Anthropic-compatible platforms. Converter route for OpenAI or Gemini models with Anthropic format.                                                                                 |
| Gemini Generate Content        | `/v1beta/models/{model}:generateContent`       | `/gemini/v1beta/models/{model}:generateContent`       | Native route for Gemini platforms. Converter route for OpenAI or Anthropic models with Gemini format.                                                                                               |
| Gemini Stream Generate Content | `/v1beta/models/{model}:streamGenerateContent` | `/gemini/v1beta/models/{model}:streamGenerateContent` | Native streaming is available; cross-protocol streaming output in Gemini format is not implemented.                                                                                                 |
| Jev / System One               | `/v1/systemone`                                | —                                                     | Native typed judgments for `jev-1.13.0` with `state` and `questions`. No chat conversion or streaming. [Reference](/api-reference/systemone).                                                       |
| Video                          | `/v2/videos`                                   | —                                                     | Async video tasks. Native route only; create, GET, and DELETE share one task envelope.                                                                                                              |

## Protocol Guard Rules

* `/v1/chat/completions` and `/v1/responses` accept OpenAI-compatible request
  bodies and route only to OpenAI-compatible platforms.
* `/v1/messages` accepts Anthropic Messages request bodies and routes only to
  Anthropic-compatible platforms.
* Gemini native endpoints accept Gemini request bodies and route only to Gemini
  platforms.
* `/v2/videos` accepts the provider-neutral video task body and routes only to
  video platforms.
* Converter-prefixed routes such as `/openai/...`, `/responses/...`,
  `/anthropic/...`, and `/gemini/...` are not affected by the native-route
  protocol guard.

## Common Patterns

| Goal                                                                   | Route to use                                                          |
| ---------------------------------------------------------------------- | --------------------------------------------------------------------- |
| Keep OpenAI SDK, call Claude                                           | `https://api.pipellm.ai/openai/v1/chat/completions`                   |
| Keep OpenAI SDK, call Gemini                                           | `https://api.pipellm.ai/openai/v1/chat/completions`                   |
| Keep a Responses client (OpenAI SDK, Codex CLI), call Claude or Gemini | `https://api.pipellm.ai/responses/v1/responses`                       |
| Keep Anthropic SDK, call GPT                                           | `https://api.pipellm.ai/anthropic/v1/messages`                        |
| Keep Anthropic SDK, call Gemini                                        | `https://api.pipellm.ai/anthropic/v1/messages`                        |
| Keep Gemini SDK, call GPT                                              | `https://api.pipellm.ai/gemini/v1beta/models/{model}:generateContent` |
| Keep Gemini SDK, call Claude                                           | `https://api.pipellm.ai/gemini/v1beta/models/{model}:generateContent` |
| Generate video                                                         | `https://api.pipellm.ai/v2/videos`                                    |

## Error Behavior

Protocol mismatches return a consistent JSON body:

```json theme={"dark"}
{
  "error": {
    "type": "invalid_request_error",
    "code": "400",
    "message": "..."
  }
}
```

The error message tells developers:

* which request format was detected
* why the native route is incompatible
* which converter route or native route to use instead
* which docs page to read next

## Related Docs

<Columns>
  <Card title="API Reference Overview" icon="book" href="/api-reference/introduction">
    Public routes and endpoint families
  </Card>

  <Card title="Converters Overview" icon="shuffle" href="/converter/overview">
    Keep your SDK format and route across providers
  </Card>

  <Card title="OpenAI Responses" icon="sparkles" href="/api-reference/openai/responses">
    `/v1/responses` on PipeLLM
  </Card>
</Columns>


# Video generation
Source: https://docs.pipellm.ai/guides/video

Create an asynchronous video task, poll until it completes, and download the output.

Video is asynchronous. Submit a job, poll until `status` is terminal, then read `outputs[].url`.

1. `POST /v2/videos` — create a task
2. `GET /v2/videos/{id}` — poll status
3. When `status` is `completed`, download `outputs[].url`
4. Optional: `DELETE /v2/videos/{id}` — cancel a running task

Public task IDs look like `gen-...`. Query and cancel stay on the platform that created the task.

## Supported V2 models

All four models use `POST /v2/videos`, with polling and cancellation at `/v2/videos/{id}`. Send the exact model ID below. Fast is the variant sometimes referred to as “Flash”.

| Model             | Request `model`                   | Duration     | Resolution        | Enabled inputs                          |
| ----------------- | --------------------------------- | ------------ | ----------------- | --------------------------------------- |
| Seedance 2.0 Mini | `doubao-seedance-2-0-mini-260615` | 4–15 seconds | 480p, 720p        | Text, image, video and audio references |
| Seedance 2.0      | `seedance-2.0`                    | 4–15 seconds | 480p, 720p        | Text and images                         |
| Seedance 2.0 Fast | `doubao-seedance-2-0-fast-260128` | 4–15 seconds | 480p, 720p        | Text and images                         |
| Seedance 2.5      | `doubao-seedance-2-5-260628`      | 4–30 seconds | 480p, 720p, 1080p | Text and images                         |

Defaults are `duration: 5`, `resolution: "480p"`, `aspect_ratio: "16:9"`, and `audio: false`. Standard, Fast and 2.5 accept up to 9 images; video and audio references are not enabled. None of these profiles accepts `provider_options`. These are the capabilities currently exposed by PipeLLM.

## Create a task

<CodeGroup>
  ```bash cURL theme={"dark"}
  curl https://api.pipellm.ai/v2/videos \
    -H "Authorization: Bearer $PIPELLM_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "model": "seedance-2-0-mini",
      "prompt": "A golden retriever running along a wet beach at sunrise",
      "duration": 5,
      "aspect_ratio": "16:9",
      "resolution": "480p",
      "audio": false
    }'
  ```

  ```python Python theme={"dark"}
  import os
  import time
  import requests

  headers = {
      "Authorization": f"Bearer {os.environ['PIPELLM_API_KEY']}",
      "Content-Type": "application/json",
  }

  created = requests.post(
      "https://api.pipellm.ai/v2/videos",
      headers=headers,
      json={
          "model": "seedance-2-0-mini",
          "prompt": "A golden retriever running along a wet beach at sunrise",
          "duration": 5,
          "aspect_ratio": "16:9",
          "resolution": "480p",
          "audio": False,
      },
  )
  created.raise_for_status()
  task = created.json()
  task_id = task["id"]

  while True:
      poll = requests.get(
          f"https://api.pipellm.ai/v2/videos/{task_id}",
          headers=headers,
      )
      poll.raise_for_status()
      task = poll.json()
      status = task["status"]
      print(status, task.get("progress"))
      if status in {"completed", "failed", "cancelled", "expired"}:
          break
      time.sleep(5)

  if task["status"] == "completed":
      print(task["outputs"][0]["url"])
  else:
      print(task.get("error"))
  ```

  ```typescript TypeScript theme={"dark"}
  const headers = {
    Authorization: `Bearer ${process.env.PIPELLM_API_KEY}`,
    "Content-Type": "application/json",
  };

  const created = await fetch("https://api.pipellm.ai/v2/videos", {
    method: "POST",
    headers,
    body: JSON.stringify({
      model: "seedance-2-0-mini",
      prompt: "A golden retriever running along a wet beach at sunrise",
      duration: 5,
      aspect_ratio: "16:9",
      resolution: "480p",
      audio: false,
    }),
  });
  const task = await created.json();
  let current = task;

  while (true) {
    const poll = await fetch(`https://api.pipellm.ai/v2/videos/${current.id}`, {
      headers,
    });
    current = await poll.json();
    console.log(current.status, current.progress);
    if (["completed", "failed", "cancelled", "expired"].includes(current.status)) {
      break;
    }
    await new Promise((resolve) => setTimeout(resolve, 5000));
  }

  if (current.status === "completed") {
    console.log(current.outputs[0].url);
  } else {
    console.error(current.error);
  }
  ```
</CodeGroup>

Create returns the same task envelope as poll. `outputs` is empty until the job completes.

## Seedance 2.0 Mini

|                          |                                                          |
| ------------------------ | -------------------------------------------------------- |
| Public name              | `seedance-2-0-mini`                                      |
| Aliases                  | `seedance-2.0-mini`, `doubao-seedance-2-0-mini-260615`   |
| Duration                 | 4–15 seconds (default `5`)                               |
| Resolution               | `480p`, `720p` (default `480p`)                          |
| Aspect ratio             | default `16:9`                                           |
| Audio                    | default `false`                                          |
| Image inputs             | `type=image_url` and `role=reference_image`, max 9       |
| Video inputs             | `type=video_url`, optional `role=reference_video`, max 3 |
| Audio inputs             | `type=audio_url`, optional `role=reference_audio`, max 3 |
| Reference video duration | each clip ≤ 15s; combined duration ≤ 15s                 |

`provider_options` is not supported on this model profile. Unknown JSON fields are rejected.

### Text to video

```json theme={"dark"}
{
  "model": "seedance-2-0-mini",
  "prompt": "A slow pan across a neon-lit alley in the rain",
  "duration": 5,
  "resolution": "480p"
}
```

### Reference images

```json theme={"dark"}
{
  "model": "seedance-2-0-mini",
  "prompt": "The same character walks through a forest",
  "inputs": [
    {
      "type": "image_url",
      "url": "https://example.com/character.png",
      "role": "reference_image"
    }
  ]
}
```

## Statuses

| Status        | Meaning                      |
| ------------- | ---------------------------- |
| `queued`      | Accepted and waiting         |
| `in_progress` | Generating                   |
| `completed`   | Ready; read `outputs`        |
| `failed`      | Terminal error; read `error` |
| `cancelled`   | Cancelled with `DELETE`      |
| `expired`     | No longer queryable upstream |

Poll every few seconds. Generation often takes tens of seconds to a few minutes.

## Billing

GET and DELETE do not add charges. Rates are frozen at creation. Mini and standard 2.0 are charged per second after acceptance, with background refunds for failed, cancelled or expired tasks. Fast and 2.5 settle only after completion using actual upstream `completion_tokens`; failed tasks are not charged.

Mini is priced in USD per second:

| Resolution | No video input         | With video input                   |
| ---------- | ---------------------- | ---------------------------------- |
| `480p`     | \$0.06 / output second | \$0.0375 / (input + output) second |
| `720p`     | \$0.12 / output second | \$0.075 / (input + output) second  |

A 4-second `480p` text-to-video job is `$0.24`. Confirm current rates in the [console](https://console.pipellm.ai).

Standard 2.0 retains its existing **`$0.48 / output second`** rate. Fast costs **`$4.080882 / million video tokens`**. Seedance 2.5 costs **`$10.294118 / million video tokens`** at 480p / 720p and **`$11.323529`** at 1080p. These two token rates use **USD 1 = CNY 6.8**. Video tokens are not seconds or text tokens; final charges use actual generation usage.

## API reference

<Columns>
  <Card title="Create" href="/api-reference/video/create">
    `POST /v2/videos`
  </Card>

  <Card title="Get" href="/api-reference/video/get">
    `GET /v2/videos/{id}`
  </Card>

  <Card title="Cancel" href="/api-reference/video/cancel">
    `DELETE /v2/videos/{id}`
  </Card>
</Columns>


# Overview
Source: https://docs.pipellm.ai/index

Relay is PipeLLM's Auto Router: one compatible endpoint for approved models.

Relay is PipeLLM's Auto Router. Keep the OpenAI, Anthropic, or Gemini SDK you already use. Relay authenticates the request, applies policy, and selects an approved route by protocol, provider health, and cost.

The default path is OpenAI-compatible: point the OpenAI SDK at `https://api.pipellm.ai/v1` and send a request.

<CodeGroup>
  ```bash cURL theme={"dark"}
  curl https://api.pipellm.ai/v1/chat/completions \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer $PIPELLM_API_KEY" \
    -d '{
      "model": "gpt-5",
      "messages": [{"role": "user", "content": "Hello"}]
    }'
  ```

  ```python Python theme={"dark"}
  from openai import OpenAI

  client = OpenAI(
      api_key="...",
      base_url="https://api.pipellm.ai/v1",
  )

  response = client.chat.completions.create(
      model="gpt-5",
      messages=[{"role": "user", "content": "Hello"}],
  )
  print(response.choices[0].message.content)
  ```

  ```typescript TypeScript theme={"dark"}
  import OpenAI from "openai";

  const client = new OpenAI({
    apiKey: process.env.PIPELLM_API_KEY,
    baseURL: "https://api.pipellm.ai/v1",
  });

  const response = await client.chat.completions.create({
    model: "gpt-5",
    messages: [{ role: "user", "content": "Hello" }],
  });
  console.log(response.choices[0].message.content);
  ```
</CodeGroup>

<Tip>
  Create a Relay API key in the [console](https://console.pipellm.ai). Model IDs change over time — list the ones available to your account with [`GET /v1/models`](/api-reference/list-models).
</Tip>

## Next

<Columns>
  <Card title="Quickstart" icon="play" href="/quickstart">
    Get a key, install the SDK, and send your first request.
  </Card>

  <Card title="Routing & Protocols" icon="route" href="/guides/routing-protocols">
    Native routes, converter routes, and when Relay switches protocols.
  </Card>

  <Card title="Authentication" icon="key" href="/authentication">
    API keys, headers, and base URLs.
  </Card>

  <Card title="Models" icon="layer-group" href="/models">
    How to list models and pick an ID.
  </Card>
</Columns>


# Claude Code
Source: https://docs.pipellm.ai/integrations/claude-code

Connect Claude Code to PipeLLM with Anthropic-compatible routing

For routing rules, see [Routing & Protocols](/guides/routing-protocols). For
available model IDs, see [List Models](/api-reference/list-models).

Claude Code works with Anthropic Messages-compatible gateways. PipeLLM supports
that in two ways:

| Use case                             | Base URL                           | Why                                             |
| ------------------------------------ | ---------------------------------- | ----------------------------------------------- |
| Call Claude models                   | `https://api.pipellm.ai`           | Use PipeLLM's native Anthropic-compatible route |
| Keep Claude Code, call GPT or Gemini | `https://api.pipellm.ai/anthropic` | Use the Anthropic converter route               |

## Config File

Claude Code reads configuration from `~/.claude/settings.json`.

<Warning>
  Remove conflicting shell variables before testing a new setup, especially
  `ANTHROPIC_BASE_URL`, `ANTHROPIC_API_KEY`, and `ANTHROPIC_AUTH_TOKEN`.
</Warning>

## Recommended Setup

<Tabs>
  <Tab title="Claude Models">
    ```json theme={"dark"}
    {
      "env": {
        "ANTHROPIC_BASE_URL": "https://api.pipellm.ai",
        "ANTHROPIC_AUTH_TOKEN": "your-pipellm-api-key",
        "ANTHROPIC_DEFAULT_SONNET_MODEL": "claude-sonnet-4-6",
        "ANTHROPIC_DEFAULT_HAIKU_MODEL": "claude-haiku-4-5-latest"
      }
    }
    ```
  </Tab>

  <Tab title="GPT or Gemini Models">
    ```json theme={"dark"}
    {
      "env": {
        "ANTHROPIC_BASE_URL": "https://api.pipellm.ai/anthropic",
        "ANTHROPIC_AUTH_TOKEN": "your-pipellm-api-key",
        "ANTHROPIC_DEFAULT_SONNET_MODEL": "gpt-5",
        "ANTHROPIC_DEFAULT_HAIKU_MODEL": "gemini-3-flash-preview"
      }
    }
    ```

    Swap these IDs for any model from [`GET /v1/models`](/api-reference/list-models).
  </Tab>
</Tabs>

## Run Claude Code

```bash theme={"dark"}
claude
```

If you want to keep a checked-in project config instead of editing your global
settings, keep the same `env` block in the settings file that Claude Code reads
for your environment.

## Notes

* Claude Code expects Anthropic Messages-compatible endpoints such as
  `/v1/messages`.
* PipeLLM's native route is the cleanest choice for Claude models.
* PipeLLM's `/anthropic` converter route is the safest way to keep Claude Code's
  protocol while switching to OpenAI or Gemini models.

## Related Docs

<Columns>
  <Card title="Developer Tools" icon="terminal" href="/integrations/overview">
    Compare Claude Code, OpenCode, and OpenClaw
  </Card>

  <Card title="Anthropic Converter" icon="shuffle" href="/converter/anthropic-format">
    Keep Anthropic format across providers
  </Card>

  <Card title="Claude Code Gateway Docs" icon="book" href="https://code.claude.com/docs/en/llm-gateway">
    Official Claude Code gateway requirements
  </Card>
</Columns>


# LangChain (Anthropic)
Source: https://docs.pipellm.ai/integrations/langchain-anthropic

Use LangChain's ChatAnthropic with PipeLLM's Anthropic-compatible routes

For authentication and model list, see [Anthropic Overview](/api-reference/anthropic/overview).

<Note>
  This page is a framework integration guide. For the raw HTTP endpoint
  reference, see [Messages](/api-reference/anthropic/messages).
</Note>

## Installation

```bash theme={"dark"}
pip install langchain-anthropic
```

## Code Examples

<Tabs>
  <Tab title="Python">
    ```python theme={"dark"}
    import os
    from langchain_anthropic import ChatAnthropic

    llm = ChatAnthropic(
      model='claude-sonnet-4-5-20250929',
      anthropic_api_key=os.getenv('PIPELLM_API_KEY'),
      anthropic_api_url='https://api.pipellm.ai'
    )

    response = llm.invoke([
      ('user', 'Why is the sky blue?')
    ])

    print(response.content)
    ```
  </Tab>

  <Tab title="Node.js">
    ```javascript theme={"dark"}
    import { ChatAnthropic } from '@langchain/anthropic';

    const llm = new ChatAnthropic({
      model: 'claude-sonnet-4-5-20250929',
      anthropicApiKey: process.env.PIPELLM_API_KEY,
      anthropicApiUrl: 'https://api.pipellm.ai'
    });

    const response = await llm.invoke([
      { role: 'user', content: 'Why is the sky blue?' }
    ]);

    console.log(response.content);
    ```
  </Tab>
</Tabs>

## Streaming

```python theme={"dark"}
import os
from langchain_anthropic import ChatAnthropic

llm = ChatAnthropic(
  model='claude-sonnet-4-5-20250929',
  anthropic_api_key=os.getenv('PIPELLM_API_KEY'),
  anthropic_api_url='https://api.pipellm.ai',
  streaming=True
)

for chunk in llm.stream('Tell me a story'):
  print(chunk.content, end='', flush=True)
```

## Function Calling

```python theme={"dark"}
import os
from langchain_anthropic import ChatAnthropic
from langchain_core.tools import tool

@tool
def get_weather(location: str) -> str:
  """Get the weather for a location."""
  return f"Weather in {location}: 22°C, Sunny"

llm = ChatAnthropic(
  model='claude-sonnet-4-5-20250929',
  anthropic_api_key=os.getenv('PIPELLM_API_KEY'),
  anthropic_api_url='https://api.pipellm.ai'
)

llm_with_tools = llm.bind_tools([get_weather])
response = llm_with_tools.invoke("What's the weather in Tokyo?")

print(response.tool_calls)
```

## Related Docs

<Columns>
  <Card title="Anthropic Overview" icon="message" href="/api-reference/anthropic/overview">
    Headers, models, and native routes
  </Card>

  <Card title="Messages" icon="code" href="/api-reference/anthropic/messages">
    Raw endpoint reference for `POST /v1/messages`
  </Card>

  <Card title="Developer Tools Overview" icon="terminal" href="/integrations/overview">
    More tools and framework integrations on PipeLLM
  </Card>
</Columns>


# LangChain (Gemini)
Source: https://docs.pipellm.ai/integrations/langchain-gemini

Use LangChain's ChatGoogleGenerativeAI with PipeLLM's Gemini routes

For authentication and model list, see [Gemini Overview](/api-reference/gemini/overview).

<Note>
  This page is a framework integration guide. For the raw HTTP endpoint
  reference, see [Generate Content](/api-reference/gemini/generate-content).
</Note>

## Installation

```bash theme={"dark"}
pip install langchain-google-genai
```

## Code Examples

<Tabs>
  <Tab title="Python">
    ```python theme={"dark"}
    import os
    from langchain_google_genai import ChatGoogleGenerativeAI

    llm = ChatGoogleGenerativeAI(
      model='gemini-3-flash-preview',
      google_api_key=os.getenv('PIPELLM_API_KEY'),
      transport='rest',
      client_options={'api_endpoint': 'https://api.pipellm.ai'}
    )

    response = llm.invoke([
      ('user', 'Why is the sky blue?')
    ])

    print(response.content)
    ```
  </Tab>

  <Tab title="Node.js">
    ```javascript theme={"dark"}
    import { ChatGoogleGenerativeAI } from '@langchain/google-genai';

    const llm = new ChatGoogleGenerativeAI({
      model: 'gemini-3-flash-preview',
      apiKey: process.env.PIPELLM_API_KEY,
      baseUrl: 'https://api.pipellm.ai'
    });

    const response = await llm.invoke([
      { role: 'user', content: 'Why is the sky blue?' }
    ]);

    console.log(response.content);
    ```
  </Tab>
</Tabs>

## Streaming

```python theme={"dark"}
import os
from langchain_google_genai import ChatGoogleGenerativeAI

llm = ChatGoogleGenerativeAI(
  model='gemini-3-flash-preview',
  google_api_key=os.getenv('PIPELLM_API_KEY'),
  transport='rest',
  client_options={'api_endpoint': 'https://api.pipellm.ai'},
  streaming=True
)

for chunk in llm.stream('Tell me a story'):
  print(chunk.content, end='', flush=True)
```

## Related Docs

<Columns>
  <Card title="Gemini Overview" icon="sparkles" href="/api-reference/gemini/overview">
    Headers, models, and native routes
  </Card>

  <Card title="Generate Content" icon="code" href="/api-reference/gemini/generate-content">
    Raw endpoint reference for `POST /v1beta/models/{model}:generateContent`
  </Card>

  <Card title="Developer Tools Overview" icon="terminal" href="/integrations/overview">
    More tools and framework integrations on PipeLLM
  </Card>
</Columns>


# LangChain (OpenAI)
Source: https://docs.pipellm.ai/integrations/langchain-openai

Use LangChain's ChatOpenAI with PipeLLM's OpenAI-compatible routes

For authentication and model list, see [OpenAI Overview](/api-reference/openai/overview).

<Note>
  This page is a framework integration guide. For the raw HTTP endpoint
  reference, see [Chat Completions](/api-reference/openai/chat-completions) or
  [Responses](/api-reference/openai/responses).
</Note>

## Installation

```bash theme={"dark"}
pip install langchain-openai
```

## Code Examples

<Tabs>
  <Tab title="Python">
    ```python theme={"dark"}
    import os
    from langchain_openai import ChatOpenAI

    llm = ChatOpenAI(
      model='gpt-5',
      api_key=os.getenv('PIPELLM_API_KEY'),
      base_url='https://api.pipellm.ai/v1'
    )

    response = llm.invoke([
      ('user', 'Why is the sky blue?')
    ])

    print(response.content)
    ```
  </Tab>

  <Tab title="Node.js">
    ```javascript theme={"dark"}
    import { ChatOpenAI } from '@langchain/openai';

    const llm = new ChatOpenAI({
      model: 'gpt-5',
      apiKey: process.env.PIPELLM_API_KEY,
      configuration: {
        baseURL: 'https://api.pipellm.ai/v1'
      }
    });

    const response = await llm.invoke([
      { role: 'user', content: 'Why is the sky blue?' }
    ]);

    console.log(response.content);
    ```
  </Tab>
</Tabs>

## Streaming

```python theme={"dark"}
import os
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
  model='gpt-5',
  api_key=os.getenv('PIPELLM_API_KEY'),
  base_url='https://api.pipellm.ai/v1',
  streaming=True
)

for chunk in llm.stream('Tell me a story'):
  print(chunk.content, end='', flush=True)
```

## Function Calling

```python theme={"dark"}
import os
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool

@tool
def get_weather(location: str) -> str:
  """Get the weather for a location."""
  return f"Weather in {location}: 22°C, Sunny"

llm = ChatOpenAI(
  model='gpt-5',
  api_key=os.getenv('PIPELLM_API_KEY'),
  base_url='https://api.pipellm.ai/v1'
)

llm_with_tools = llm.bind_tools([get_weather])
response = llm_with_tools.invoke("What's the weather in Tokyo?")

print(response.tool_calls)
```

## Related Docs

<Columns>
  <Card title="OpenAI Overview" icon="bolt" href="/api-reference/openai/overview">
    Headers, models, and native routes
  </Card>

  <Card title="Chat Completions" icon="comments" href="/api-reference/openai/chat-completions">
    Raw endpoint reference for `POST /v1/chat/completions`
  </Card>

  <Card title="Developer Tools Overview" icon="terminal" href="/integrations/overview">
    More tools and framework integrations on PipeLLM
  </Card>
</Columns>


# OpenClaw
Source: https://docs.pipellm.ai/integrations/openclaw

Connect OpenClaw to PipeLLM with a custom provider

For routing rules, see [Routing & Protocols](/guides/routing-protocols). For
available model IDs, see [List Models](/api-reference/list-models).

OpenClaw supports custom providers in `~/.openclaw/openclaw.json`. For PipeLLM,
the simplest broad-coverage setup is an OpenAI-compatible custom provider that
points at PipeLLM's OpenAI converter route.

## Recommended Route

```text theme={"dark"}
https://api.pipellm.ai/openai/v1
```

If you specifically want Anthropic Messages semantics, OpenClaw can also target
`https://api.pipellm.ai/anthropic` with `api: "anthropic-messages"`.

## Fastest Setup

Use the onboarding wizard:

1. Run `openclaw onboard`
2. Choose `Custom Provider`
3. Choose `OpenAI-compatible`
4. Set base URL to `https://api.pipellm.ai/openai/v1`
5. Paste your PipeLLM API key
6. Enter a model ID from [`GET /v1/models`](/api-reference/list-models), such as `claude-sonnet-4-6`, `gpt-5`, or `gemini-3-flash-preview`

## Manual Config

OpenClaw uses JSON5, so comments and trailing commas are allowed.

```json5 theme={"dark"}
{
  env: {
    PIPELLM_API_KEY: "your-pipellm-api-key",
  },
  agents: {
    defaults: {
      model: {
        primary: "pipellm/claude-sonnet-4-6",
      },
      models: {
        "pipellm/claude-sonnet-4-6": {
          alias: "Claude Sonnet 4.6",
        },
        "pipellm/gpt-5": {
          alias: "GPT-5",
        },
        "pipellm/gemini-3-flash-preview": {
          alias: "Gemini 3 Flash",
        },
      },
    },
  },
  models: {
    mode: "merge",
    providers: {
      pipellm: {
        api: "openai-completions",
        baseUrl: "https://api.pipellm.ai/openai/v1",
        apiKey: "${PIPELLM_API_KEY}",
        models: [
          { id: "claude-sonnet-4-6", name: "Claude Sonnet 4.6" },
          { id: "gpt-5", name: "GPT-5" },
          { id: "gemini-3-flash-preview", name: "Gemini 3 Flash" },
        ],
      },
    },
  },
}
```

## Notes

* `model.primary` uses the `provider/model` format in OpenClaw.
* `models.providers.<id>.api` controls the protocol adapter. For PipeLLM's
  recommended setup, use `openai-completions`.
* If you later want Anthropic-compatible routing instead, switch the provider to
  `api: "anthropic-messages"` and set `baseUrl` to
  `https://api.pipellm.ai/anthropic`.

## Related Docs

<Columns>
  <Card title="Developer Tools" icon="terminal" href="/integrations/overview">
    Compare supported coding assistants
  </Card>

  <Card title="OpenAI Converter" icon="shuffle" href="/converter/openai-format">
    Keep OpenAI-compatible format across providers
  </Card>

  <Card title="WebSearch npm Package" icon="box" href="/websearch/npm-package">
    Add PipeLLM WebSearch tools to OpenClaw
  </Card>

  <Card title="OpenClaw Config Docs" icon="book" href="https://docs.openclaw.ai/gateway/configuration-reference">
    Official OpenClaw configuration reference
  </Card>
</Columns>


# OpenCode
Source: https://docs.pipellm.ai/integrations/opencode

Connect OpenCode to PipeLLM with a custom OpenAI-compatible provider

For routing rules, see [Routing & Protocols](/guides/routing-protocols). For
available model IDs, see [List Models](/api-reference/list-models).

OpenCode supports custom providers through `opencode.json`. The cleanest PipeLLM
setup is to use PipeLLM's OpenAI converter route so one provider configuration
can access Claude, GPT, and Gemini models.

## Recommended Route

```text theme={"dark"}
https://api.pipellm.ai/openai/v1
```

Use `https://api.pipellm.ai/v1` only if you want OpenAI-compatible native
routing and do not need cross-protocol access.

## Config Locations

OpenCode loads configuration from:

* `~/.config/opencode/opencode.json` for global user config
* `opencode.json` in the project root for project-specific config

## Quick Setup

1. Export your PipeLLM key:

```bash theme={"dark"}
export PIPELLM_API_KEY="your-pipellm-api-key"
```

2. Add a custom provider to `opencode.json`:

```json theme={"dark"}
{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "pipellm": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "PipeLLM",
      "options": {
        "baseURL": "https://api.pipellm.ai/openai/v1",
        "apiKey": "{env:PIPELLM_API_KEY}"
      },
      "models": {
        "claude-sonnet-4-6": {
          "name": "Claude Sonnet 4.6"
        },
        "gpt-5": {
          "name": "GPT-5"
        },
        "gemini-3-flash-preview": {
          "name": "Gemini 3 Flash"
        }
      }
    }
  },
  "model": "pipellm/claude-sonnet-4-6"
}
```

3. Start OpenCode:

```bash theme={"dark"}
opencode
```

## Notes

* OpenCode custom providers accept any provider id. `pipellm` is just an
  example.
* `@ai-sdk/openai-compatible` is the right package when you are targeting
  Chat Completions-style endpoints.
* If a model or workflow specifically needs the OpenAI Responses API, that is a
  separate integration path and should not reuse this Chat Completions setup.

## Related Docs

<Columns>
  <Card title="Developer Tools" icon="terminal" href="/integrations/overview">
    Compare supported coding assistants
  </Card>

  <Card title="OpenAI Converter" icon="shuffle" href="/converter/openai-format">
    Keep OpenAI-compatible format across providers
  </Card>

  <Card title="OpenCode Providers Docs" icon="book" href="https://opencode.ai/docs/providers">
    Official OpenCode custom provider examples
  </Card>
</Columns>


# Agent Integrations Overview
Source: https://docs.pipellm.ai/integrations/overview

Coding assistants and agent frameworks that connect to Relay through protocol-compatible routes

This section documents agent-facing tools that can run directly on Relay by
pointing them at the correct protocol route.

## Recommended Routes

| Tool or framework   | Recommended route                                                                         | Why                                                                         |
| ------------------- | ----------------------------------------------------------------------------------------- | --------------------------------------------------------------------------- |
| Claude Code         | `https://api.pipellm.ai` for Claude, `https://api.pipellm.ai/anthropic` for GPT or Gemini | Claude Code speaks the Anthropic Messages protocol                          |
| OpenCode            | `https://api.pipellm.ai/openai/v1`                                                        | OpenCode custom providers work well with OpenAI-compatible chat completions |
| OpenClaw            | `https://api.pipellm.ai/openai/v1`                                                        | OpenClaw custom providers support OpenAI-compatible endpoints cleanly       |
| LangChain OpenAI    | `https://api.pipellm.ai/v1`                                                               | `ChatOpenAI` maps directly to OpenAI-compatible native routes               |
| LangChain Anthropic | `https://api.pipellm.ai`                                                                  | `ChatAnthropic` maps directly to Anthropic Messages                         |
| LangChain Gemini    | `https://api.pipellm.ai`                                                                  | `ChatGoogleGenerativeAI` maps directly to Gemini native routes              |

## What a Tool Guide Covers

Each tool guide answers four questions:

* which base URL to use
* which environment variables or config keys to set
* which model families make sense as starting points
* which protocol limitations or workarounds developers should know

## Coding Agents and Assistants

<Columns>
  <Card title="Claude Code" icon="terminal" href="/integrations/claude-code">
    Anthropic-native coding assistant
  </Card>

  <Card title="OpenCode" icon="code" href="/integrations/opencode">
    Open source coding agent with JSON config
  </Card>

  <Card title="OpenClaw" icon="terminal" href="/integrations/openclaw">
    Gateway-first assistant with custom provider onboarding
  </Card>
</Columns>

## Agent Framework Integrations

<Columns>
  <Card title="LangChain (OpenAI)" icon="link" href="/integrations/langchain-openai">
    Use `ChatOpenAI` with PipeLLM's OpenAI-compatible routes
  </Card>

  <Card title="LangChain (Anthropic)" icon="link" href="/integrations/langchain-anthropic">
    Use `ChatAnthropic` with PipeLLM's Anthropic-compatible routes
  </Card>

  <Card title="LangChain (Gemini)" icon="link" href="/integrations/langchain-gemini">
    Use `ChatGoogleGenerativeAI` with PipeLLM's Gemini routes
  </Card>
</Columns>

## Shared References

<Columns>
  <Card title="Routing & Protocols" icon="route" href="/guides/routing-protocols">
    Choose the right route before configuring a tool
  </Card>

  <Card title="Converters" icon="shuffle" href="/converter/overview">
    Keep the tool's native SDK format across providers
  </Card>

  <Card title="List Models" icon="list" href="/api-reference/list-models">
    Confirm the model IDs available to your account
  </Card>
</Columns>


# Models
Source: https://docs.pipellm.ai/models

List the Relay model IDs available to your account and choose one for requests.

Do not copy model lists from older docs pages. The Relay catalog for your account is [`GET /v1/models`](/api-reference/list-models).

```bash theme={"dark"}
curl https://api.pipellm.ai/v1/models \
  -H "Authorization: Bearer $PIPELLM_API_KEY"
```

Each item has:

| Field          | Meaning                                                                      |
| -------------- | ---------------------------------------------------------------------------- |
| `id`           | Value to send as `model`                                                     |
| `display_name` | Human-readable label                                                         |
| `type_target`  | Protocol family used for routing, such as `openai`, `anthropic`, or `gemini` |

Pricing is a separate call: [`GET /v1/models/pricing`](/api-reference/model-pricing).

## Chat

Use any `id` whose `type_target` matches the route you are calling.

* OpenAI-compatible models go to `POST /v1/chat/completions` or `POST /v1/responses`
* Anthropic models go to `POST /v1/messages`
* Gemini models go to `POST /v1beta/models/{model}:generateContent`

To keep one SDK and call a model from another family, use a [converter route](/guides/routing-protocols).

## Video

Video is not a chat completion. Send the public model name to [`POST /v2/videos`](/api-reference/video/create).

The first documented video model is `seedance-2-0-mini`. These aliases resolve to the same model:

* `seedance-2-0-mini` (public name)
* `seedance-2.0-mini`
* `doubao-seedance-2-0-mini-260615`

See [Video generation](/guides/video) for duration, resolution, and reference-input limits.

## Hidden models

`GET /v1/models` omits hidden and unlisted models. If a request returns `400` for an unknown model ID, list the catalog again and use a current `id`.


# Quickstart
Source: https://docs.pipellm.ai/quickstart

Get a Relay key, point the OpenAI SDK at the Auto Router, and send a request.

The fastest way to use Relay is the OpenAI SDK against `https://api.pipellm.ai/v1`. Other protocols are available later if you already use Anthropic or Gemini clients.

<Steps>
  <Step title="Create an API key">
    1. Sign in at [console.pipellm.ai](https://console.pipellm.ai)
    2. Add balance
    3. Create an API key

    Export it in your shell:

    ```bash theme={"dark"}
    export PIPELLM_API_KEY="..."
    ```
  </Step>

  <Step title="Install the OpenAI SDK">
    <CodeGroup>
      ```bash Python theme={"dark"}
      pip install openai
      ```

      ```bash Node.js theme={"dark"}
      npm install openai
      ```
    </CodeGroup>
  </Step>

  <Step title="Send a chat request">
    Replace `gpt-5` with any ID from [`GET /v1/models`](/api-reference/list-models).

    <CodeGroup>
      ```python Python theme={"dark"}
      import os
      from openai import OpenAI

      client = OpenAI(
          api_key=os.environ["PIPELLM_API_KEY"],
          base_url="https://api.pipellm.ai/v1",
      )

      response = client.chat.completions.create(
          model="gpt-5",
          messages=[{"role": "user", "content": "Hello"}],
      )
      print(response.choices[0].message.content)
      ```

      ```typescript TypeScript theme={"dark"}
      import OpenAI from "openai";

      const client = new OpenAI({
        apiKey: process.env.PIPELLM_API_KEY,
        baseURL: "https://api.pipellm.ai/v1",
      });

      const response = await client.chat.completions.create({
        model: "gpt-5",
        messages: [{ role: "user", content: "Hello" }],
      });
      console.log(response.choices[0].message.content);
      ```

      ```bash cURL theme={"dark"}
      curl https://api.pipellm.ai/v1/chat/completions \
        -H "Content-Type: application/json" \
        -H "Authorization: Bearer $PIPELLM_API_KEY" \
        -d '{
          "model": "gpt-5",
          "messages": [{"role": "user", "content": "Hello"}]
        }'
      ```
    </CodeGroup>
  </Step>
</Steps>

## What to do next

<Columns>
  <Card title="Streaming" href="/guides/streaming">
    Stream tokens as they are generated.
  </Card>

  <Card title="Images" href="/guides/images">
    Generate and edit images.
  </Card>

  <Card title="Video" href="/guides/video">
    Create an async video task and poll until it completes.
  </Card>

  <Card title="API Reference" href="/api-reference/introduction">
    Endpoint paths, parameters, and response shapes.
  </Card>
</Columns>

## Other protocols

If your app already speaks Anthropic or Gemini, keep that SDK and change the base URL. See [Routing & Protocols](/guides/routing-protocols) when you need a converter route instead of the native path.

<AccordionGroup>
  <Accordion title="Anthropic SDK">
    ```python theme={"dark"}
    import anthropic

    client = anthropic.Anthropic(
        api_key="...",
        base_url="https://api.pipellm.ai",
    )

    message = client.messages.create(
        model="claude-sonnet-4-6",
        max_tokens=1024,
        messages=[{"role": "user", "content": "Hello"}],
    )
    ```
  </Accordion>

  <Accordion title="Gemini SDK">
    ```python theme={"dark"}
    from google import genai

    client = genai.Client(
        api_key="...",
        http_options={"base_url": "https://api.pipellm.ai"},
    )

    response = client.models.generate_content(
        model="gemini-3-flash-preview",
        contents="Hello",
    )
    ```
  </Accordion>
</AccordionGroup>


# Nano Banana Image Generation
Source: https://docs.pipellm.ai/use-cases/nano-banana

Generate and edit images with Gemini Nano Banana multimodal models

For authentication, see [Overview](/api-reference/gemini/overview).

## What is Nano Banana

Nano Banana is Gemini's **multimodal image generation capability**, using the same `generateContent` endpoint as text models.

| Model                  | Model ID                      | Features                                                                                       |
| ---------------------- | ----------------------------- | ---------------------------------------------------------------------------------------------- |
| **Nano Banana 2 Lite** | `gemini-3.1-flash-lite-image` | Cheapest and fastest; not tuned for multiple references or multi-turn editing                  |
| **Nano Banana 2**      | `gemini-3.1-flash-image`      | The generalist. Balances speed with 4K generation and reliable text rendering                  |
| **Nano Banana Pro**    | `gemini-3-pro-image`          | Professional production, advanced reasoning, high-fidelity text rendering, up to 4K resolution |

## Endpoints

Same as text models:

| Type          | Endpoint                                            |
| ------------- | --------------------------------------------------- |
| Non-Streaming | `POST /v1beta/models/{model}:generateContent`       |
| Streaming     | `POST /v1beta/models/{model}:streamGenerateContent` |

## Text to Image

<Tabs>
  <Tab title="cURL">
    ```bash theme={"dark"}
    curl -s -X POST \
      "https://api.pipellm.ai/v1beta/models/gemini-3-pro-image:streamGenerateContent" \
      -H "x-goog-api-key: $PIPELLM_API_KEY" \
      -H "Content-Type: application/json" \
      -d @- << 'EOF'
    {
      "contents": [{
        "role": "user",
        "parts": [
          {"text": "Create a picture of a nano banana dish in a fancy restaurant with a Gemini theme"}
        ]
      }]
    }
    EOF
    ```
  </Tab>

  <Tab title="Python">
    ```python theme={"dark"}
    from google import genai
    from PIL import Image

    client = genai.Client(
    api_key=os.getenv('PIPELLM_API_KEY'),
    http_options={'base_url': 'https://api.pipellm.ai'}
    )

    response = client.models.generate_content(
    model="gemini-3.1-flash-image",
    contents=["Create a picture of a nano banana dish in a fancy restaurant with a Gemini theme"],
    )

    for part in response.parts:
    if part.text is not None:
    print(part.text)
    elif part.inline_data is not None:
    image = part.as_image()
    image.save("generated_image.png")

    ```
  </Tab>

  <Tab title="Node.js">
    ```javascript theme={"dark"}
    import { GoogleGenAI } from "@google/genai";
    import * as fs from "node:fs";

    const ai = new GoogleGenAI({
      apiKey: process.env.PIPELLM_API_KEY,
      httpOptions: { baseUrl: 'https://api.pipellm.ai' }
    });

    const response = await ai.models.generateContent({
      model: "gemini-3.1-flash-image",
      contents: "Create a picture of a nano banana dish in a fancy restaurant with a Gemini theme",
    });

    for (const part of response.candidates[0].content.parts) {
      if (part.text) {
        console.log(part.text);
      } else if (part.inlineData) {
        const buffer = Buffer.from(part.inlineData.data, "base64");
        fs.writeFileSync("generated_image.png", buffer);
      }
    }
    ```
  </Tab>
</Tabs>

## Image to Image (with Reference)

Upload a reference image and generate a new one with text prompts:

<Tabs>
  <Tab title="cURL">
    ```bash theme={"dark"}
    IMG_BASE64=$(base64 -w0 /path/to/cat_image.jpeg)

    curl -s -X POST \
     "https://api.pipellm.ai/v1beta/models/gemini-3-pro-image:streamGenerateContent" \
     -H "x-goog-api-key: $PIPELLM_API_KEY" \
      -H "Content-Type: application/json" \
      -d @- << EOF
    {
      "contents": [{
        "role": "user",
        "parts": [
          {"inlineData": {"mimeType": "image/jpeg", "data": "$IMG_BASE64"}},
    {"text": "Create a picture of my cat eating a nano-banana in a fancy restaurant under the Gemini constellation"}
    ]
    }],
    "generationConfig": {
    "imageConfig": {
    "aspectRatio": "16:9"
    }
    }
    }
    EOF

    ```
  </Tab>

  <Tab title="Python">
    ```python theme={"dark"}
    from google import genai
    from PIL import Image

    client = genai.Client(
      api_key=os.getenv('PIPELLM_API_KEY'),
      http_options={'base_url': 'https://api.pipellm.ai'}
    )

    image = Image.open('/path/to/cat_image.png')

    response = client.models.generate_content(
      model="gemini-3.1-flash-image",
      contents=[
        "Create a picture of my cat eating a nano-banana in a fancy restaurant under the Gemini constellation",
        image
      ],
    )

    for part in response.parts:
      if part.text is not None:
        print(part.text)
      elif part.inline_data is not None:
        image = part.as_image()
        image.save("edited_image.png")
    ```
  </Tab>

  <Tab title="Node.js">
    ```javascript theme={"dark"}
    import { GoogleGenAI } from "@google/genai";
    import * as fs from "node:fs";

    const ai = new GoogleGenAI({
    apiKey: process.env.PIPELLM_API_KEY,
    httpOptions: { baseUrl: 'https://api.pipellm.ai' }
    });

    const imageData = fs.readFileSync("cat_image.png");
    const base64Image = imageData.toString("base64");

    const response = await ai.models.generateContent({
    model: "gemini-3.1-flash-image",
    contents: [
    { text: "Create a picture of my cat eating a nano-banana in a fancy restaurant under the Gemini constellation" },
    { inlineData: { mimeType: "image/png", data: base64Image } }
    ],
    });

    for (const part of response.candidates[0].content.parts) {
    if (part.text) {
    console.log(part.text);
    } else if (part.inlineData) {
    const buffer = Buffer.from(part.inlineData.data, "base64");
    fs.writeFileSync("edited_image.png", buffer);
    }
    }

    ```
  </Tab>
</Tabs>

## Response Format

Image generation responses are multimodal, containing text and base64-encoded image data:

```json theme={"dark"}
{
  "candidates": [{
    "content": {
      "parts": [
        { "text": "Here is your generated image..." },
        {
          "inlineData": {
            "mimeType": "image/png",
            "data": "<BASE64_IMAGE_DATA>"
          }
        }
      ]
    }
  }]
}
```

## More Features

<Card title="Nano Banana Documentation" icon="book" href="https://ai.google.dev/gemini-api/docs/image-generation">
  Supports multi-turn editing, 4K resolution, multi-image composition, and more
</Card>


# Search news
Source: https://docs.pipellm.ai/websearch/news-search

GET /v1/websearch/search-news
Search recent news articles.

Use this route when you want recent news articles instead of general web
results.

## Endpoint

| Method | Endpoint                                          |
| ------ | ------------------------------------------------- |
| `GET`  | `https://api.pipellm.ai/v1/websearch/search-news` |

## Authentication

```text theme={"dark"}
Authorization: Bearer $PIPELLM_API_KEY
```

## Query Parameters

| Parameter | Type   | Required | Description |
| --------- | ------ | -------- | ----------- |
| `q`       | string | Yes      | News query  |

## Example Request

```bash theme={"dark"}
curl -X GET "https://api.pipellm.ai/v1/websearch/search-news?q=OpenAI" \
  -H "Authorization: Bearer $PIPELLM_API_KEY"
```

## Response Shape

```json theme={"dark"}
{
  "code": 200,
  "message": "ok",
  "took_ms": 4200,
  "data": {
    "organic": [
      {
        "title": "Latest OpenAI news",
        "link": "https://example.com/news",
        "snippet": "Recent product and research updates...",
        "contexts": [
          {
            "idx": 0,
            "text": "..."
          }
        ]
      }
    ]
  }
}
```

## Notes

* This route proxies to the internal `/api/search-news` handler.
* It uses the news-specific search source, then applies the same retrieval flow as deep search.
* If no results are found, this route returns `404` with `code: 404` and `message: "no search results"`.
* Successful requests cost \$0.08 each and are recorded with internal model name `websearch-news`.
* PipeLLM derives the model name from the route; clients do not send it.

## Related Docs

<Columns>
  <Card title="Deep Search" icon="magnifying-glass" href="/websearch/search-api">
    Search the general web with retrieval support
  </Card>

  <Card title="Simple Search" icon="bolt" href="/websearch/simple-search">
    Fast snippets when you do not need news-specific retrieval
  </Card>

  <Card title="WebSearch Overview" icon="globe" href="/websearch/overview">
    Product overview and route matrix
  </Card>
</Columns>


# pipellm-websearch (npm)
Source: https://docs.pipellm.ai/websearch/npm-package

Install PipeLLM WebSearch from npm as an OpenClaw plugin

`pipellm-websearch` is the npm distribution for PipeLLM WebSearch as an
OpenClaw plugin. It adds search, page-reading, and news tools to OpenClaw while
using your PipeLLM API key for authentication and billing.

## Package

| Item         | Value                                                    |
| ------------ | -------------------------------------------------------- |
| Package name | `pipellm-websearch`                                      |
| Distribution | npm                                                      |
| Primary use  | OpenClaw plugin                                          |
| Repository   | `https://github.com/PipeLlm-AI/pipellm-websearch-plugin` |

## Install

```bash theme={"dark"}
npm install pipellm-websearch
```

## What the Plugin Adds

| Tool                    | Description                                                   | Public route                  |
| ----------------------- | ------------------------------------------------------------- | ----------------------------- |
| `pipellm_web_search`    | Deep web search with content extraction and relevance ranking | `/v1/websearch/search`        |
| `pipellm_simple_search` | Faster search returning search snippets                       | `/v1/websearch/simple-search` |
| `pipellm_web_reader`    | Read a web page and return clean text                         | `/v1/websearch/reader`        |
| `pipellm_news_search`   | Search recent news with retrieval support                     | `/v1/websearch/search-news`   |

## OpenClaw Plugin Config

Add the package to your `openclaw.json`:

```json theme={"dark"}
{
  "plugins": {
    "load": {
      "paths": ["./node_modules/pipellm-websearch"]
    },
    "entries": {
      "pipellm-websearch": {
        "enabled": true,
        "config": {
          "apiKey": "pipe-your-api-key-here"
        }
      }
    }
  }
}
```

## Agent Tool Allowlist

Enable the tools for your agent:

```json theme={"dark"}
{
  "agents": {
    "list": [
      {
        "name": "my-agent",
        "tools": {
          "allow": [
            "pipellm_web_search",
            "pipellm_simple_search",
            "pipellm_web_reader",
            "pipellm_news_search"
          ]
        }
      }
    ]
  }
}
```

## Plugin Configuration

| Field         | Required | Default                  | Description                               |
| ------------- | -------- | ------------------------ | ----------------------------------------- |
| `apiKey`      | Yes      | —                        | PipeLLM API key                           |
| `apiEndpoint` | No       | `https://api.pipellm.ai` | Override only for self-hosted deployments |

## Notes

* This package is currently documented as an OpenClaw plugin, not a generic SDK.
* Use your PipeLLM API key from the Console.
* Requests made through the plugin use route-based PipeLLM WebSearch billing:
  `pipellm_web_search` records `websearch-deepsearch` at $0.08, `pipellm_simple_search` records `websearch` at $0.01,
  `pipellm_web_reader` records `websearch-reader` at $0.01, and `pipellm_news_search` records `websearch-news` at $0.08 per successful request.
* The plugin does not send billing parameters; PipeLLM assigns the internal model name from the selected route.

## Related Docs

<Columns>
  <Card title="OpenClaw" icon="terminal" href="/integrations/openclaw">
    Configure OpenClaw to use PipeLLM
  </Card>

  <Card title="Deep Search" icon="magnifying-glass" href="/websearch/search-api">
    Use the full retrieval route over HTTP
  </Card>

  <Card title="npm Package" icon="box" href="https://www.npmjs.com/package/pipellm-websearch">
    View the package on npm
  </Card>
</Columns>


# Overview
Source: https://docs.pipellm.ai/websearch/overview

PipeLLM WebSearch is a standalone search and retrieval product surface for developers and agents

PipeLLM WebSearch is a standalone product surface for real-time search,
retrieval, and page reading. It can be used directly through PipeLLM's public
API or installed as the `pipellm-websearch` npm package for OpenClaw.

## What WebSearch Includes

| Surface     | What it is for                                            |
| ----------- | --------------------------------------------------------- |
| Search API  | Call PipeLLM WebSearch over HTTP from your app or backend |
| npm package | Install PipeLLM WebSearch as an OpenClaw plugin from npm  |

## Core Capabilities

* real-time web search through PipeLLM-managed retrieval
* optional rich page context in search results
* page reading and news-search tools through the OpenClaw plugin
* per-request billing with the same PipeLLM API key used across your account

## Authentication

Use the same PipeLLM API key you use for the rest of the platform:

```text theme={"dark"}
Authorization: Bearer $PIPELLM_API_KEY
```

## Public Routes

PipeLLM Gateway currently exposes four public WebSearch routes:

| Route                             | Purpose                                     | Query parameter |
| --------------------------------- | ------------------------------------------- | --------------- |
| `GET /v1/websearch/search`        | Deep search with retrieval and reranking    | `q`             |
| `GET /v1/websearch/simple-search` | Fast search snippets without RAG enrichment | `q`             |
| `GET /v1/websearch/reader`        | Read a page and return clean text           | `url`           |
| `GET /v1/websearch/search-news`   | News search with retrieval and reranking    | `q`             |

These routes are proxy endpoints on the public gateway and map to the internal
WebSearch service allowlist.

## Pricing

Current WebSearch pricing is billed per successful request. PipeLLM assigns an
internal model name from the public route; clients do not send extra billing
parameters.

| Public route                  | Internal model name    | Price per successful request |
| ----------------------------- | ---------------------- | ---------------------------- |
| `/v1/websearch/search`        | `websearch-deepsearch` | \$0.08                       |
| `/v1/websearch/simple-search` | `websearch`            | \$0.01                       |
| `/v1/websearch/reader`        | `websearch-reader`     | \$0.01                       |
| `/v1/websearch/search-news`   | `websearch-news`       | \$0.08                       |

<Note>
  WebSearch requests share your account-level authentication and are billed as a
  separate product surface from model generation endpoints.
</Note>

## Usage and Reporting

For successful WebSearch requests, Metering calculates the fixed price from
the internal model name and persists that model name with the final cost.
Admin Report groups usage by the recorded model name. Each group includes:

* request count
* recorded usage cost
* report revenue after the Admin Report discount

<Note>
  Internal model names are used only for metering and reporting. They do not
  change the WebSearch response body and do not need to be supplied by clients.
</Note>

## Route Matrix

| Public route                  | Internal service route | Notes                                     |
| ----------------------------- | ---------------------- | ----------------------------------------- |
| `/v1/websearch/search`        | `/api/search`          | Full search + content extraction + rerank |
| `/v1/websearch/simple-search` | `/api/simple-search`   | Serper search only                        |
| `/v1/websearch/reader`        | `/api/reader`          | Read one page by URL                      |
| `/v1/websearch/search-news`   | `/api/search-news`     | News search + content extraction + rerank |

## Start Here

<Columns>
  <Card title="Deep Search" icon="magnifying-glass" href="/websearch/search-api">
    Full search with content extraction and reranking
  </Card>

  <Card title="Simple Search" icon="bolt" href="/websearch/simple-search">
    Fast search snippets without deep retrieval
  </Card>

  <Card title="Reader" icon="book" href="/websearch/reader">
    Read a page and convert it to clean text
  </Card>

  <Card title="News Search" icon="newspaper" href="/websearch/news-search">
    Search recent news with retrieval support
  </Card>

  <Card title="pipellm-websearch (npm)" icon="box" href="/websearch/npm-package">
    Install the OpenClaw plugin from npm
  </Card>

  <Card title="OpenClaw" icon="terminal" href="/integrations/openclaw">
    Connect coding agents to PipeLLM and WebSearch
  </Card>
</Columns>


# Read a page
Source: https://docs.pipellm.ai/websearch/reader

GET /v1/websearch/reader
Fetch a URL and return clean text.

Use this route when you already know the page URL and want PipeLLM WebSearch to
fetch and convert it into clean text.

## Endpoint

| Method | Endpoint                                     |
| ------ | -------------------------------------------- |
| `GET`  | `https://api.pipellm.ai/v1/websearch/reader` |

## Authentication

```text theme={"dark"}
Authorization: Bearer $PIPELLM_API_KEY
```

## Query Parameters

| Parameter | Type   | Required | Description          |
| --------- | ------ | -------- | -------------------- |
| `url`     | string | Yes      | The page URL to read |

## Example Request

```bash theme={"dark"}
curl -X GET "https://api.pipellm.ai/v1/websearch/reader?url=https://go.dev" \
  -H "Authorization: Bearer $PIPELLM_API_KEY"
```

## Response Shape

```json theme={"dark"}
{
  "code": 200,
  "message": "ok",
  "content": "# The Go Programming Language\n\nGo is an open source programming language..."
}
```

## Error Shape

```json theme={"dark"}
{
  "code": 400,
  "message": "url parameter is required"
}
```

## Notes

* This route proxies to the internal `/api/reader` handler.
* The response shape uses `content` instead of `data`.
* The reader path is also the most direct HTTP equivalent of `pipellm_web_reader`.
* Successful requests cost \$0.01 each and are recorded with internal model name `websearch-reader`.
* PipeLLM derives the model name from the route; clients do not send it.

## Related Docs

<Columns>
  <Card title="pipellm-websearch (npm)" icon="box" href="/websearch/npm-package">
    Enable page-reading as an OpenClaw tool
  </Card>

  <Card title="Deep Search" icon="magnifying-glass" href="/websearch/search-api">
    Use search first when you do not already know the target page
  </Card>

  <Card title="WebSearch Overview" icon="globe" href="/websearch/overview">
    Product overview and route matrix
  </Card>
</Columns>


# Deep search
Source: https://docs.pipellm.ai/websearch/search-api

GET /v1/websearch/search
Deep web search with content extraction and reranking.

PipeLLM exposes WebSearch through a public gateway route. Use this API when you
want web results in your own app, backend, or agent workflow without installing
the npm plugin.

## Endpoint

| Method | Endpoint                                     |
| ------ | -------------------------------------------- |
| `GET`  | `https://api.pipellm.ai/v1/websearch/search` |

## Authentication

```text theme={"dark"}
Authorization: Bearer $PIPELLM_API_KEY
```

## Query Parameters

| Parameter | Type   | Required | Description  |
| --------- | ------ | -------- | ------------ |
| `q`       | string | Yes      | Search query |

## What This Route Does

`/v1/websearch/search` is the full retrieval route. It starts with search
results, then extracts page content, runs embedding similarity, and reranks the
best contexts when available.

## Example Request

<Tabs>
  <Tab title="cURL">
    ```bash theme={"dark"}
    curl -X GET "https://api.pipellm.ai/v1/websearch/search?q=latest+AI+news" \
      -H "Authorization: Bearer $PIPELLM_API_KEY"
    ```
  </Tab>

  <Tab title="Python">
    ```python theme={"dark"}
    import os
    import requests

    response = requests.get(
        "https://api.pipellm.ai/v1/websearch/search",
        params={"q": "latest AI news"},
        headers={"Authorization": f"Bearer {os.getenv('PIPELLM_API_KEY')}"},
    )

    data = response.json()
    for result in data.get("data", {}).get("organic", []):
        print(f"- {result['title']}: {result['link']}")
    ```
  </Tab>

  <Tab title="Node.js">
    ```javascript theme={"dark"}
    const response = await fetch(
      "https://api.pipellm.ai/v1/websearch/search?q=latest+AI+news",
      {
        headers: {
          Authorization: `Bearer ${process.env.PIPELLM_API_KEY}`,
        },
      }
    );

    const data = await response.json();
    data.data.organic.forEach((result) => {
      console.log(`- ${result.title}: ${result.link}`);
    });
    ```
  </Tab>
</Tabs>

## Response Format

```json theme={"dark"}
{
  "code": 200,
  "message": "ok",
  "took_ms": 6050,
  "data": {
    "organic": [
      {
        "title": "AI News | Latest News | Insights Powering AI-Driven Business ...",
        "link": "https://www.artificialintelligence-news.com/",
        "snippet": "AI News delivers the latest updates in artificial intelligence..."
      },
      {
        "title": "The Latest AI News and AI Breakthroughs that Matter Most",
        "link": "https://www.crescendo.ai/news/latest-ai-news-and-updates",
        "snippet": "Summary: Xiaomi has announced a next-gen AI voice model...",
        "contexts": [
          {
            "idx": 0,
            "text": "December 26, 2025\n\n# The Latest AI News and AI Breakthroughs..."
          }
        ]
      }
    ]
  }
}
```

## Response Fields

| Field                            | Type    | Description                            |
| -------------------------------- | ------- | -------------------------------------- |
| `code`                           | integer | `200` for success                      |
| `message`                        | string  | Status message                         |
| `took_ms`                        | integer | Time taken in milliseconds             |
| `data.organic`                   | array   | Organic search results                 |
| `data.organic[].title`           | string  | Page title                             |
| `data.organic[].link`            | string  | Page URL                               |
| `data.organic[].snippet`         | string  | Search snippet                         |
| `data.organic[].contexts`        | array   | Extracted page contexts when available |
| `data.organic[].contexts[].idx`  | integer | Context index                          |
| `data.organic[].contexts[].text` | string  | Extracted text content                 |

## Error Response

```json theme={"dark"}
{
  "code": 400,
  "message": "Bad Request, missing query parameter"
}
```

## Pricing and Limits

| Item                | Value                                                                       |
| ------------------- | --------------------------------------------------------------------------- |
| Price               | \$0.08 per successful deep search request                                   |
| Internal model name | `websearch-deepsearch`                                                      |
| Limit model         | Shares your account-level limits                                            |
| Retry behavior      | If you receive `503 Service Unavailable` with `Retry-After`, wait and retry |

PipeLLM assigns `websearch-deepsearch` from the route and stores it with the
usage and request-audit record. It is not a client-supplied parameter.

## Example: RAG Context for an LLM

```python theme={"dark"}
import os
import requests
from openai import OpenAI

search_response = requests.get(
    "https://api.pipellm.ai/v1/websearch/search",
    params={"q": "OpenAI o3 model capabilities"},
    headers={"Authorization": f"Bearer {os.getenv('PIPELLM_API_KEY')}"},
)
search_data = search_response.json()

context = "\n".join(
    f"[{r['title']}]({r['link']}): {r['snippet']}"
    for r in search_data.get("data", {}).get("organic", [])[:5]
)

client = OpenAI(
    api_key=os.getenv("PIPELLM_API_KEY"),
    base_url="https://api.pipellm.ai/v1",
)

response = client.chat.completions.create(
    model="gpt-5",
    messages=[
        {"role": "system", "content": f"Use this context:\n{context}"},
        {"role": "user", "content": "What are the key features of OpenAI o3?"},
    ],
)

print(response.choices[0].message.content)
```

## Related Docs

<Columns>
  <Card title="WebSearch Overview" icon="globe" href="/websearch/overview">
    Product overview, pricing, and entry points
  </Card>

  <Card title="Simple Search" icon="bolt" href="/websearch/simple-search">
    Faster search route without deep retrieval
  </Card>

  <Card title="pipellm-websearch (npm)" icon="box" href="/websearch/npm-package">
    Install the OpenClaw plugin from npm
  </Card>
</Columns>


# Simple search
Source: https://docs.pipellm.ai/websearch/simple-search

GET /v1/websearch/simple-search
Fast web search snippets without page extraction.

Use this route when you want faster search results without page extraction,
embedding, or reranking.

## Endpoint

| Method | Endpoint                                            |
| ------ | --------------------------------------------------- |
| `GET`  | `https://api.pipellm.ai/v1/websearch/simple-search` |

## Authentication

```text theme={"dark"}
Authorization: Bearer $PIPELLM_API_KEY
```

## Query Parameters

| Parameter | Type   | Required | Description  |
| --------- | ------ | -------- | ------------ |
| `q`       | string | Yes      | Search query |

## Example Request

```bash theme={"dark"}
curl -X GET "https://api.pipellm.ai/v1/websearch/simple-search?q=latest+AI+news" \
  -H "Authorization: Bearer $PIPELLM_API_KEY"
```

## Response Shape

```json theme={"dark"}
{
  "code": 200,
  "message": "ok",
  "took_ms": 950,
  "data": {
    "answerBox": {
      "title": "Latest AI news",
      "snippet": "..."
    },
    "organic": [
      {
        "title": "AI News",
        "link": "https://example.com",
        "snippet": "Latest updates..."
      }
    ]
  }
}
```

## Notes

* This route proxies to the internal `/api/simple-search` handler.
* It returns search results quickly and does not add `contexts` via deep retrieval.
* Successful requests cost \$0.01 each and are recorded with internal model name `websearch`.
* PipeLLM derives the model name from the route; clients do not send it.

## Related Docs

<Columns>
  <Card title="Deep Search" icon="magnifying-glass" href="/websearch/search-api">
    Use retrieval and reranking when you need richer context
  </Card>

  <Card title="News Search" icon="newspaper" href="/websearch/news-search">
    Search recent news articles instead of general web results
  </Card>

  <Card title="WebSearch Overview" icon="globe" href="/websearch/overview">
    Product overview and full route matrix
  </Card>
</Columns>