TypeScript SDK

Already using the OpenAI or Anthropic SDK? Keep it — point baseURL at https://api.ai.ml/v1 and use an aiml key. That is the fastest path and it is tested in CI on every commit.

Reach for aiml when you want what a compatibility dialect cannot express: the IR's typed blocks, the routing controls, dryRun, and the cost that comes back on every response.

npm install aiml

A generation, streamed

const aiml = new AIML({ baseURL: process.env.AIML_BASE_URL ?? "https://api.ai.ml" });

const request = {
  model: process.env.AIML_MODEL ?? "openai/gpt-5-mini",
  messages: [{ role: "user" as const, content: [{ type: "text" as const, text: "Write one sentence about gateways." }] }],
  generation: { max_tokens: 128 },
};

// what will this cost, and where will it go? Nothing is sent upstream.
const size = await aiml.countTokens(request as never);
const plan = await aiml.dryRun(request as never);
console.log(`input_tokens=${size.input_tokens} source=${size.source}`);
console.log(`plan=${plan.plan.map((p) => p.endpoint_id).join(",")} reserve_micro=${plan.reserve_micro}`);

// now actually send it, streaming
const stream = await aiml.generate.stream(request as never);
for await (const text of stream.textDeltas()) process.stdout.write(text);
console.log();

// the final response is assembled from the events, not fetched again
const final = (await new AIML({ baseURL: process.env.AIML_BASE_URL ?? "https://api.ai.ml" }).generate(request as never)) as unknown as {

stream.textDeltas() is the text; iterating the stream itself gives the IR events, which are the same events the gateway received from the provider, translated once. stream.finalResponse() assembles the response from those events rather than asking the server again: the event sequence is a prefix-closed rendering of the response, so the two are the same answer and one of them costs nothing.

Retries and idempotency

Every call carries an Idempotency-Key, generated per call unless you pass one, and every retry reuses it. That is what makes retrying a billed operation safe: the gateway bills the request once however many times it arrives.

Retries happen only before the first byte, and only for the cases where a retry can work — connection failures, 408, 409, 429 and 5xx. Nothing retries after the first byte, in the SDK or in the gateway: a half-delivered stream cannot be replayed without either double-billing or lying about what the model said. A stream that fails midway gives you the partial answer and an AIMLError; deciding what to do with a half-written paragraph is yours, not ours.

new AIML({ retry: { maxRetries: 4, initialDelayMs: 250 } });
await aiml.generate(req, { retry: { maxRetries: 0 } });   // per call

Errors carry the request id

import { AIMLError } from "aiml";

try {
  await aiml.generate(req);
} catch (e) {
  if (e instanceof AIMLError) console.error(e.status, e.code, e.requestId);
}

The id is on the response header even when the body is not JSON — a proxy's 502 page still tells you which request it was. Look it up on the request lookup page or with aiml.requests.get(id).

Before you send

await aiml.countTokens(req);   // { input_tokens, source: "exact" | "estimated", tokenizer }
await aiml.dryRun(req);        // the plan, the capability warnings and the reserve

dryRun runs the real router over the real catalog and calls no provider. It answers the question that makes routing changes frightening — if I send this, where does it go and what does it cost? — without sending it.

Cancellation

An AbortSignal closes the upstream connection rather than just unsubscribing, so the generation actually stops and you stop paying for it at the next checkpoint.

const ac = new AbortController();
setTimeout(() => ac.abort(), 2_000);
const s = await aiml.generate.stream(req, { signal: ac.signal });

Versioning

The SDK pins one API date version and sends it on every request, so upgrading the SDK is what changes the wire format — never a deploy on our side. An API date version is served for at least twelve months after its successor ships.