Skip to content

Reliability and roster review: 2026-11-07 ​

Why some deployed models seem not to respond, what is retiring, which models are worth adding, and whether there is a better way to run a multi-model Foundry estate. The measurements come from a live read-only audit of a 44-deployment, two-account estate. The claims about Azure behaviour come from Microsoft Learn and are linked. Anything that is inference rather than documented fact is labelled, and anything not yet verified is listed under Open tests.

Prices are Azure Retail Prices API list prices in USD per million tokens, not spend.

Summary ​

  1. The models are not broken. Long reasoning is meeting short timeouts. Over a 30 day window, a reasoning model run at its default effort took a median of about 160 seconds per answer through a non-streaming client with a 210 second timeout. Close to half of its calls were abandoned at exactly 210 seconds. A non-reasoning-heavy model on the same path (DeepSeek V4 Pro) answered in a median of about 10 seconds with a 100 percent success rate.
  2. Foundry reasoning models send nothing until they finish thinking, even when streaming. Microsoft documents this and recommends client timeouts of up to 29 minutes for reasoning models (quotas and limits). App Service closes any request that is silent for about four minutes, and that limit cannot be raised (App Service timeout).
  3. The fix is configuration and plumbing, not a new model. Cap reasoning effort for interactive and non-streaming callers, stream upstream, send keep-alive bytes, retry on 429, and align the content filter.
  4. A month of drift and several retirements are due now. See Retirements.
  5. Keep a thin custom gateway; do not replace it with API Management. API Management sits behind the same four minute ceiling and costs far more than the gap it would close. See A better way to manage it.

What the telemetry shows ​

The gateway logs metadata only (no prompts or responses): requested model, status, duration, token counts, reasoning tokens, and whether the client cancelled. In 30 days of the hosted MCP path (non-streaming, 210 second client timeout):

Model familyCalls that succeededMedian durationPattern
Grok 4.7about 53 percentabout 162 sMedian reasoning tokens about 6,700 on a roughly 2,700 token prompt. Nearly all failures are client cancellations at 210 s. A handful are 429 pd_decode_pushback.
Grok 4.6about 42 percentabout 130 sSame pattern.
GPT-6 Astraabout 83 percentabout 49 sCancellations only on the longest calls.
GPT-6 Sol100 percentabout 24 sNo failures. GPT-6.1 Sol also had no failures in a smaller sample.
DeepSeek V4 Pro100 percentabout 10 sNo reasoning tokens reported.
Kimi K2.7 Code100 percentnot reportedNo failures.

Two readings follow, and only the first is established:

  • Established: the failures are the caller giving up, not Azure returning errors. Grok 4.7 defaults to reasoning_effort: high (Grok on Foundry), and a high-effort answer can exceed any timeout shorter than a few minutes.
  • Verified: DeepSeek V4 reports no reasoning tokens through this path because thinking is off unless reasoning_effort is sent. See Why DeepSeek V4 can seem not capable.

The telemetry covers only the hosted MCP consumer. Editor traffic did not appear in it, so the editor path is unmeasured here.

Why a reasoning model looks dead ​

HopLimitSource
App Service front endabout 240 s with no response bytes, fixedLearn
API Management forward-requestdefault 300 s, values over 240 s "may not be honored"Learn
Container Apps default ingress240 sLearn
Foundry guidance for reasoning modelsclient timeout up to 29 minutesLearn

Moving the proxy to another Azure host does not help, because every option sits behind roughly the same idle ceiling.

A second Grok failure was found by direct probing: Grok 4.7 rejects stop and presence_penalty with HTTP 400 decode_bad_request, and the error carries no param field, so a gateway that strips only when param is set passes the failure through. Any editor or tool that sends those fields to Grok fails every call. Grok 4.7 does accept temperature, top_p, max_tokens, max_completion_tokens and reasoning_effort (low, medium, high), and calls tools correctly. Streaming a reasoning model returned the first byte exactly when the answer started, so nothing is sent during reasoning.

On a moderate prompt Grok 4.7 answered in 11 to 17 seconds at every effort setting, so the long calls come from large agentic prompts, where reasoning scales with context. Lowering effort is a ceiling on worst-case time, not a guarantee of speed.

Three further documented behaviours compound it:

  • 429 handling. Rate limiting counts the prompt plus max_tokens at admission, checks requests per minute over 1 or 10 second windows, counts rejected requests, and can lower a Standard deployment's effective limit under load. Use retry-after-ms with backoff and jitter (429 explainer). A gateway that does not retry turns every 429 into a failed turn.
  • Guardrail mismatch. The default guardrail (Microsoft.DefaultV2) runs prompt-injection detection on prompts and protected-material detection on completions (default policies). Agentic coding prompts full of tool output are a plausible false-positive source. A blocked prompt is HTTP 400, and a proxy that treats every 400 as an unsupported parameter will strip fields and retry into a second 400. Use one tuned guardrail on every chat deployment and never retry a content_filter 400.
  • Quota per capacity unit. For Grok 4.7 one capacity unit is 1,000 tokens per minute and 1 request per minute (Grok 4.7 quotas). A 250 or 375 capacity DeepSeek Flash deployment has a 250K or 375K token-per-minute budget against a 1M token context, so one large agentic turn can exhaust a minute.

Why DeepSeek V4 can seem not capable ​

Verified by direct calls on 2026-11-07: DeepSeek V4 Pro and V4.1 Flash run with thinking off by default. The same moderately hard prompt produced about 410 output tokens and no reasoning_content by default. With reasoning_effort: "high" or "max" it produced 1,550 to 2,630 tokens and returned message.reasoning_content, in 11 to 18 seconds. A thinking body field is rejected with HTTP 400 (unrecognized_request_argument), so reasoning_effort is the switch on Foundry. A client that never sends it gets a fast, shallow model.

What works, also verified:

  • Tool calling, including two parallel tool calls in one turn, on both models.
  • Two-turn tool flows succeed with and without echoing reasoning_content back, so no gateway workaround is needed on Foundry (DeepSeek's own API differs).
  • No inline <think> tags: reasoning arrives in reasoning_content.
  • temperature, stop and max_tokens are accepted.

Remaining weak spots: the Flash deployments are small (250 and 375 capacity) against a 1M token window, so one large agentic turn can exhaust a minute of quota, and system prompts are followed less strictly by reasoning models. Set reasoning_effort high in the client or a gateway profile, and size the editor's input and output limits to the quota.

Retirements ​

Dates are the deprecation value from the Azure model catalog API on 2026-11-07, cross-checked against the retirement schedule. Sorted by urgency.

Deployed modelDateSuccessor already deployedAction
Sora 22026-11-02, pastNoneDeployment is Disabled. Delete it. No replacement is named.
MAI-Image-2.5, 2.5-Flash, 2.5-Pro2026-11-22MAI-Image-2.6 and 2.6-FlashMove references to 2.6. There is no 2.6-Pro. 2.6 allows about 1536 by 1536 pixels (2.5: 1024 by 1024) and has three times the request rate per tier.
Mistral Medium 25052026-12-09Mistral Medium 3.5Delete 2505.
DeepSeek V4.1 Flash (2026-09-10)2026-12-15, PreviewDeepSeek V4.1 Flash 1017 exists in the catalog (Preview, retires 2027-03-06)Decide between the Preview successor and the GA V4 Flash and V4 Pro (retire 2028-02-20).
Llama 4 Maverickdocs: 2026-11-19; catalog: noneNoneThe documented date and the catalog disagree. Plan removal.
Whisperdocs: 2026-12-15; catalog: 2027-02-26gpt-transcribeMove transcription to gpt-transcribe.
Mistral Document AI 2512Legacy 2026-11-06Mistral OCR 4.0Move traffic, then delete.
Codestral 25012027-02-15 (docs)None namedPlan a move to a general coding model.
Phi-4 reasoning2027-01-17 (docs)Managed Compute onlyDelete rather than migrate.
FLUX 1.1 pro and Kontext pro2027-01-05 (docs)None namedPlan.
FLUX.2 pro and flex2027-01-20 (docs)None namedPlan.
Kimi K2.7 Code2027-04-16Kimi K3.1 Pro (new)Evaluate.
Grok 4.202027-04-06Grok 4.7No action.

Catalog and docs disagree ​

Microsoft's retirement page and the catalog API disagree for Llama 4 Maverick, Whisper and the MAI-Image-2.5 family. The catalog API is what Azure enforces for a deployed version, so use it for planning, and use the earlier of the two dates for anything you cannot afford to lose. Re-query before acting.

Drift since the last audit ​

A month after the previous snapshot, three deployments differ from the record: Grok 4.7 now uses the tuned guardrail and a doubled capacity, the router's capacity is ten times higher, and Sora 2 is disabled. Grok 4.7 and two other deployments still allow automatic upgrade to a newer default version, while the other deployments are pinned. Decide per model whether you want pinning, and record it.

What is new in the catalog ​

Newer than the previous audit, or newly deployable:

ModelVersionStatusNotes
gpt-6.1-terra2026-10-26GA1.05M context, reasoning, Responses API, computer use. About half the price of GPT-5.6 Terra at $1.00 input and $5.00 output. Catalog lists it in East US only. Not in the router pool.
Kimi K3.1 Pro2026-11-04GA1M context, image input, tool calling, reasoning. Supersedes Kimi K2.7 Code. Price not yet published. Available in both estate regions in the catalog.
DeepSeek V4.1 Flash 10172026-10-17PreviewNewer than the deployed V4.1 Flash.
Claude Opus 5.5, Sonnet 5.5, Haiku 5.51 and 2GAVersion 1 is Anthropic-hosted, version 2 is Azure-hosted. Native Messages API only. Claude cannot be bought on credit-only or sponsored subscriptions (Claude on Foundry).
MAI-Voice-2.1 and 2.1-Flashn/aPreviewThe MAI-Voice page now lists only these. Voices chosen on MAI-Voice-2 may not carry over, so test each voice before a batch run.

Worth adding, in order of value: gpt-6.1-terra (replaces GPT-5.6 Terra at roughly half the cost), Claude Haiku 5.5 as a second-vendor bulk model ($0.10 input and $0.50 output up to 100K tokens), Claude Sonnet 5.5 for agentic coding ($2 and $10), and Kimi K3.1 Pro once it has a price. Skip MAI-Thinking-1 (Preview, 256K context, costs more than GPT-6.1 Sol on output-heavy work).

Model Router ​

  • No new router version exists. 2025-11-18 is updated in place and retires 2027-05-20.
  • The routing pool now includes Grok 4.6, Grok 4.1 Fast, GPT-6 Astra, DeepSeek V3.2, several Claude models and three Fireworks models. It does not include Grok 4.7, GPT-6 Luna or Sol, GPT-6.1, DeepSeek V4 or V4.1, or the 5.5 Claude tier. So the router cannot reach most of a newer roster (router concepts).
  • Documented features: automatic failover (a custom subset of at least two models doubles as the fallback set), session affinity (preview), and reasoning_effort pass-through.
  • Treat the router as one optional auto entry, not the backbone.

A better way to manage it ​

OptionFixes the four minute wallRetry and failoverPer-model parametersMonthly cost changeVerdict
Hardened thin gateway, editors connect straight to FoundryYes for editors (no proxy in the path); in the gateway via streaming and keep-aliveBuilt in codeYesabout $0 to $5Recommended
API Management AI gatewayNoBackend pools, circuit breaker, retryPolicy XMLabout +$150 or moreDefer. Worth it for several users or teams.
AI Gateway in Foundry (preview)NoAs API ManagementLimitedPricing not publishedRevisit at GA.
Foundry model routerNot applicableBuilt-in failoverNot applicableRouter fee plus modelOne optional entry.
LiteLLM on Container AppsNo (same ingress)Yesdrop_paramsComputeAlternative if you prefer off the shelf.

Target shape:

text
Editor (interactive, streaming, tools)
   -> straight to each Foundry account's /openai/v1 endpoint, one provider group per account
Hosted MCP server and scripts (programmatic)
   -> thin gateway: model-to-account routing, per-model parameter profiles
        upstream always streaming; SSE keep-alive comments every 15 s while waiting
        non-streaming callers: cap reasoning effort and total time below the caller timeout
        429 and 5xx: honour retry-after-ms, backoff with jitter, then a fallback chain
        400 content_filter: never retry
        DeepSeek: normalise <think> content, echo reasoning_content on tool turns if required
        Entra managed identity to Foundry instead of the account key
Governance: Azure Monitor alerts on 429 and 400 rate, one tuned guardrail on every chat deployment,
            pinned versions, and a small evaluation set before any model is added.

Why keep a gateway at all: a single Foundry resource is limited to 32 deployments, so a larger roster needs two accounts, and something must route a model name to the right account and shape its parameters. Neither API Management nor the router does that without equivalent custom work.

Also available at no gateway cost: Foundry's Azure Monitor metrics (ModelRequests by status code and deployment, TimeToResponse, NormalizedTimeToFirstToken, router fallback counts) and the model leaderboards for choosing between candidates. The Foundry MCP server (preview) exposes deployments, quota and deprecation checks to an editor, but has no SLA and a changing tool set, so use it as a console, not a dependency.

Deployment-level levers: a Data Zone Standard deployment of the same model draws on a separate quota pool, and Global Standard deployments of the same model share one pool across regions, so spreading across two regions alone adds no quota. Provisioned throughput is not justified at personal scale.

Open tests ​

Verified on 2026-11-07 and removed from this list: Grok 4.7 parameter acceptance (stop and presence_penalty rejected), DeepSeek tool calling, reasoning_content handling, inline <think> (absent), the DeepSeek thinking switch (reasoning_effort), and time to first byte when streaming (no bytes during reasoning).

Still unverified. Each needs one direct call to the Foundry endpoint, not through the gateway.

  1. Grok 4.7 time to answer at each effort on a large (80K to 150K token) agentic prompt, which is where the long calls occur.
  2. Requests and tokens per capacity unit for DeepSeek V4 and the other Grok deployments (read properties.rateLimits).
  3. Whether the asynchronous content filter takes effect on xAI and DeepSeek deployments.
  4. Whether Responses background: true works for Grok 4.7, which would be the clean fix for non-streaming callers.
  5. Where the editor's temperature: 0.1 comes from.

Sources ​

Microsoft Learn pages were retrieved on 2026-11-07. Live catalog, deployment and telemetry figures are from read-only Azure Resource Manager and Application Insights queries on the same day.