OpenAI-compatible Batch API

Model Gate implements the OpenAI Files + Batch workflow on https://api.model-gate.com/v1. This is a compatibility layer: each JSONL item is executed through the normal Model Gate inference path. Model Gate does not submit an upstream provider-native OpenAI batch.

With a valid active API key, you may list, read, download, cancel or delete your existing Files/Batch resources even when the balance is zero or a resettable spending limit is exhausted. Uploading files and creating batches require a positive balance and available spending allowance. Each item must also meet the current limits when execution starts. Stored bytes, files, active jobs and queued items have separate quotas; an upload can be rejected even when funds are available.

Supported batch endpoints in this release are:

• /v1/responses
• /v1/chat/completions
• /v1/embeddings

Every executed item is marked request_mode=batch, batch_protocol=openai, with its batch_job_public_id and custom_id.

1. Upload a JSONL input file

Each non-empty line contains custom_id, method, url, and body. The URL must equal the endpoint later supplied to /v1/batches.

Example batch.jsonl:

{"custom_id":"request-1","method":"POST","url":"/v1/responses","body":{"model":"gpt-5.4","input":"Summarize this text."}}
{"custom_id":"request-2","method":"POST","url":"/v1/responses","body":{"model":"gpt-5.4","input":"Classify this text."}}

Request

curl https://api.model-gate.com/v1/files \
  -H "Authorization: Bearer mg_live_..." \
  -F "purpose=batch" \
  -F "file=@batch.jsonl"

Response — 200

{
  "id": "file-01K...",
  "object": "file",
  "bytes": 322,
  "created_at": 1786610000,
  "filename": "batch.jsonl",
  "purpose": "batch",
  "status": "processed",
  "status_details": null
}

2. Create the batch

completion_window must be 24h.

Request

curl https://api.model-gate.com/v1/batches \
  -H "Authorization: Bearer mg_live_..." \
  -H "Content-Type: application/json" \
  -d '{
    "input_file_id":"file-01K...",
    "endpoint":"/v1/responses",
    "completion_window":"24h",
    "metadata":{"job":"nightly-evaluation"}
  }'

Response — 200

{
  "id": "batch_01K...",
  "object": "batch",
  "endpoint": "/v1/responses",
  "input_file_id": "file-01K...",
  "completion_window": "24h",
  "status": "in_progress",
  "output_file_id": null,
  "error_file_id": null,
  "request_counts": {
    "total": 2,
    "completed": 0,
    "failed": 0
  },
  "metadata": {
    "job": "nightly-evaluation"
  }
}

3. Retrieve a batch

Request

curl https://api.model-gate.com/v1/batches/batch_01K... \
  -H "Authorization: Bearer mg_live_..."

Response — completed

{
  "id": "batch_01K...",
  "object": "batch",
  "endpoint": "/v1/responses",
  "status": "completed",
  "output_file_id": "file-01KOUTPUT...",
  "error_file_id": null,
  "request_counts": {
    "total": 2,
    "completed": 2,
    "failed": 0
  },
  "usage": {
    "input_tokens": 240,
    "input_tokens_details": {"cached_tokens": 0},
    "output_tokens": 90,
    "output_tokens_details": {"reasoning_tokens": 0},
    "total_tokens": 330
  }
}

4. Download results

Request

curl https://api.model-gate.com/v1/files/file-01KOUTPUT.../content \
  -H "Authorization: Bearer mg_live_..."

Response — 200

{"id":"batch_req_01K...","custom_id":"request-1","response":{"status_code":200,"request_id":"01K...","body":{"id":"resp_...","status":"completed"}},"error":null}

Failed, canceled, or expired items are written to error_file_id as JSONL records with response:null and an error object.

5. List batches

limit defaults to 20 and must be from 1 to 100. Use after for cursor pagination.

Request

curl "https://api.model-gate.com/v1/batches?limit=20&after=batch_01K..." \
  -H "Authorization: Bearer mg_live_..."

Response — 200

{
  "object": "list",
  "data": [],
  "first_id": null,
  "last_id": null,
  "has_more": false
}

Completed batch objects may include an aggregate usage object when settled Model Gate request accounting is available.

6. Cancel a batch

Cancel prevents queued items from starting; an item already processing may finish.

Request

curl -X POST https://api.model-gate.com/v1/batches/batch_01K.../cancel \
  -H "Authorization: Bearer mg_live_..."

Response — 200

{
  "id": "batch_01K...",
  "object": "batch",
  "status": "cancelling",
  "request_counts": {
    "total": 2,
    "completed": 0,
    "failed": 0
  }
}

7. File metadata, listing, and deletion

Retrieve metadata:

curl https://api.model-gate.com/v1/files/file-01K... \
  -H "Authorization: Bearer mg_live_..."

{
  "id": "file-01K...",
  "object": "file",
  "bytes": 322,
  "filename": "batch.jsonl",
  "purpose": "batch",
  "status": "processed",
  "expires_at": 1789202000
}

List files with optional purpose, after, and order=asc|desc; limit defaults to 10000 and must be from 1 to 10000:

curl "https://api.model-gate.com/v1/files?purpose=batch&order=desc&limit=100" \
  -H "Authorization: Bearer mg_live_..."

{
  "object": "list",
  "data": [],
  "first_id": null,
  "last_id": null,
  "has_more": false
}

Delete an unreferenced/expired-compatible file:

curl -X DELETE https://api.model-gate.com/v1/files/file-01K... \
  -H "Authorization: Bearer mg_live_..."

{
  "id": "file-01K...",
  "object": "file",
  "deleted": true
}

Limits and recovery semantics

Model Gate accepts up to 50,000 JSONL items and limits the uploaded file to the configured OPENAI_BATCH_MAX_FILE_BYTES value (200 MiB by default). Each individual batch item must also fit Model Gate's normal MAX_REQUEST_BODY_BYTES limit. custom_id values must be unique. Batch items cannot use stream:true or nested async:true. Large JSONL files are persisted internally as database chunks rather than one oversized SQL value.

Execution is at-least-once, not exactly-once. An interrupted request may be attempted again under the same Model Gate request ID even if the provider already accepted it. Duplicate-billing protection does not make external tool actions exactly-once. Make your application’s tool and external side effects idempotent.

Each JSONL item must meet current account, key and group limits when execution starts. Otherwise-valid items remain queued while the balance is non-positive or a resettable spending limit is exhausted. Waiting does not consume an attempt or create an error record. A top-up, usage reset or limit increase can make the remaining items eligible. No theoretical maximum batch cost is reserved: concurrent accepted items can finish with a negative balance or a small limit overshoot; new items then wait until the account is eligible again.

Pricing and accounting

The batch adapter does not use provider-native batch execution. Every item goes through normal Model Gate model routing and settlement, then receives the pricing template's Batch request price coefficient. Default: 1.

If the coefficient is 0.5, an item whose normal Model Gate cost is 0.02 is debited as 0.01. Official reference pricing remains unchanged. The applied coefficient is snapshotted and stored on the request for audit.

Errors

{
  "error": {
    "message": "line 2 url must match batch endpoint /v1/responses",
    "type": "invalid_request_error",
    "param": null,
    "code": "invalid_batch_file"
  }
}

Images are not Batch items

/v1/images/generations and /v1/images/edits are not supported batch endpoints. Use the dedicated Images operation API, which preserves the per-intent paid-dispatch fence. A legacy queued image item returns images_dedicated_api_required without a new supplier call; this does not prove that an earlier attempt was free. Do not bulk-requeue those items.

Audio transcription is available only through the dedicated transcription API, not this queue. An audio path executed through the generic queue is rejected with audio_dedicated_api_required; do not requeue it as a new paid intent.

Dedicated Video operations are not accepted by this generic queue. Use the JSON Video API with its own Idempotency-Key and operation history; an attempted legacy Video dispatch fails with video_dedicated_api_required before a supplier request. Video API.