Every new model breaks something different

In a semantic anime search engine I have a reranker that orders twenty-five candidates with an LLM. It had temperature: 0 from day one, which is the obvious thing when you want reproducible ordering. I swapped the model for a reasoning one and the call started returning an error: that model rejects temperature. It doesn't ignore it, it rejects it.

The fix wasn't deleting the line. It was replacing it with reasoning_effort, pulling it out into a constant defaulting to low, and passing it through the other place that called the same model too, an ingest script that synthesizes franchise premises. Three files, a new environment variable and a document to update. For one parameter.

That's happened six times in a year and a half, and each time differently.

The four kinds of breakage

After collecting them, it's always the same four:

• The parameter that disappears. temperature on reasoning models. There's no exact equivalent: reasoning_effort isn't the same thing, it's a different lever. You don't migrate the value, you decide a new one.
• The shape of the payload. Going from Chat Completions to the Responses API for a vision parser isn't changing the method name. messages becomes input, {"type": "text"} becomes input_text, and image_url stops being a {url, detail} object and becomes a string with detail as a sibling at the same level.
• The shape of the response. In that same migration, response.choices[0].message.content became response.output_text. And the token counter stopped being one: now there's input_tokens and output_tokens separately. That forced the function signature from -> str to -> tuple[str, int, int] and accumulating them by hand when collecting the futures in parallel.
• Quality changes without warning. This is the worst one because it doesn't error. The model answers, the JSON validates, and the results are worse.

You see the first three on the first deploy. The fourth you eat for weeks.

OpenRouter and LangChain move the problem

The logical reaction is to put a layer on top. I've done it twice.

In a transcription SaaS I had a lib/claude.ts with a processWithClaude(text, prompt) calling the Anthropic SDK directly. I deleted the whole thing and replaced it with an AIService singleton against OpenRouter, with modelName and temperature as parameters travelling from the endpoint's FormData. Real gain: switching models became sending a different string in the request.

In a chat app with projects and contexts, the layer was LangChain, with @langchain/anthropic and @langchain/google-genai so I could have a dropdown grouped by provider. There I learned the other half: the layer breaks too. One of the early commits is literally called "update dependencies for @langchain/anthropic and zod to latest versions". Updating to latest on abstractions over moving APIs is trading a problem you understand for one you don't.

What these layers do is real but limited: they normalize the shape of the call. They don't normalize the capabilities. If model A accepts temperature and B doesn't, the layer lets you pass both parameters and the error reaches you all the same, only now from inside a library, with the stack trace three levels deeper.

The cost of switching mid-product

The day I changed a schedule generator's model from google/gemini-3-flash-preview to deepseek/deepseek-v3.2, the diff touched three files: app/api/generate/route.ts, app/api/adjust-schedule/route.ts and app/api/verify-checks/route.ts. The same literal, copied three times inside three different JSON.stringify calls. That's not the provider's fault, it's mine, and it's the real cost: when the model name is embedded at the call site, migrating is a find-and-replace and praying you didn't miss one.

In a document analysis service I tried to fix this by centralizing models and cost calculation in a constants file. It took me three refactors in three weeks: one on the 18th, another the next day, and one more before that. One of the commit names has the typo baked in, "reafactor constants", which says plenty about that afternoon. The constants layer didn't land first time because I was trying to fit two different things into the same structure: which model to use and how much it costs.

How I isolate the call today

No ceremony. Three rules.

The model is never a literal at the call site. Always a constant read from the environment with a default, like RERANK_MODEL = process.env.RERANK_MODEL || "gpt-5.4-mini". Changing model in production shouldn't need a deploy.

Model-specific parameters travel with the model. If one model needs reasoning_effort and another temperature, that's model config, not function config. One object per model, not a loose parameter you drag through six signatures.

Defense at the edge, always. When I added the timeout to the OpenAI calls, I also put this into the embeddings batch:

if (embeddings.length !== input.length) {
  throw new Error(
    `Embedding provider returned ${embeddings.length} vectors for ${input.length} inputs`
  );
}

Four lines. They turn a silent misalignment between vectors and texts — which leaves your database full of embeddings assigned to the wrong anime — into an immediate error. The AbortSignal.timeout(30_000) from the same commit does the same for hung calls.

What I treat as stable

Stable: that there's an HTTP endpoint, that it takes JSON, that it returns JSON. That's where it ends.

Not stable: the model name, the sampling parameters, the message shape, the response shape, where the token counter lives, the price, and whether today's model still exists in six months.

I don't design for the model I'm using. I design for the day it's gone.