Batch

Asynchronous inference for work that can wait. OpenAI Files and Batches, JSONL in and JSONL out, 24h or Flex window.

Create a JSONL file

One request per line with custom_id, method, url, and body. Every url in a file must match the batch's endpoint.

requests.jsonl

{"custom_id": "request-1", "method": "POST", "url": "/openai/v1/chat/completions", "body": {"model": "qwen3.8-27b-batch", "messages": [{"role": "user", "content": "Summarize this text."}]}}
{"custom_id": "request-2", "method": "POST", "url": "/openai/v1/chat/completions", "body": {"model": "qwen3.8-27b-batch", "messages": [{"role": "user", "content": "Classify this ticket."}]}}

Upload it

POST /openai/v1/files with purpose: "batch". Each line is checked on upload. A bad line returns 400 with its line number. Files up to 200 MB.

file = client.files.create(file=open("requests.jsonl", "rb"), purpose="batch")

Create the batch

completion_window is always 24h, as in the OpenAI API. Ask for the Flex tier through metadata. Endpoints: /openai/v1/chat/completions, /openai/v1/completions, /openai/v1/embeddings.

batch = client.batches.create(
    input_file_id=file.id,
    endpoint="/openai/v1/chat/completions",
    completion_window="24h",
    metadata={"tier": "flex"},
)
print(batch.id, batch.status)

Poll, then download

Read the batch until status is completed, then download output_file_id. Failed lines go to error_file_id.

batch = client.batches.retrieve(batch.id)
if batch.status == "completed":
    content = client.files.content(batch.output_file_id).text

Routes

Route	Purpose
`POST /openai/v1/files`, `GET /openai/v1/files`, `GET /openai/v1/files/{id}`, `GET /openai/v1/files/{id}/content`, `DELETE /openai/v1/files/{id}`	Input and output files.
`POST /openai/v1/batches`, `GET /openai/v1/batches`, `GET /openai/v1/batches/{id}`, `POST /openai/v1/batches/{id}/cancel`	The OpenAI Batch object.

Batch states

State	Meaning
`validating`	Input file is being checked.
`in_progress`	Requests are running.
`finalizing`	Output file is being written.
`completed`	`output_file_id` and `error_file_id` are ready. Billed.
`failed`	Validation failed. `errors` is filled.
`expired`	The completion window passed.
`cancelling`	Cancel requested. Running requests finish.
`cancelled`	Cancel finished.

Retries

Failed requests are retried. You are billed once per completed request.

Tiers and pricing

Model	Model id	24h tier per 1M tokens	Flex tier per 1M tokens
Qwen3.8 27B	`qwen3.8-27b-batch`	$0.22	$0.15

Tier	Request	Window
24h	`completion_window: "24h"`	24 hours.
Flex	`metadata: {"tier": "flex"}`	Up to 72 hours. Lowest price.