Homestead Foundry

Appearance

SPIKE-09: AI Foundry model workloads on Azure Local (Arc-connected, cluster-scale on-prem) ​

Role: foundry-researcher (Opus). Status: research spike complete. Read-only: no Azure resources created, read, or modified; no deployment; no spend. Date: 2026-07-22

Scope: independent, first-party verification of whether this repo's Foundry model roster (or comparable models) can run on Azure Local (Arc-enabled, cluster-scale on-premises infrastructure, formerly Azure Stack HCI), as a second hosting target beside the cloud AIServices path this repo already uses. Every factual claim is tied to a Microsoft first-party source, cited inline. Anything Microsoft has not published is marked UNKNOWN with the test that would resolve it.

Distinct-product guard: this spike is about Azure Local (cluster-scale, Arc-managed on-prem infrastructure). It is the companion to docs/research/SPIKE-08-foundry-local-on-device.md, which covers Foundry Local as an on-device runtime for a single end-user machine. The two are cross-referenced in Q6. Microsoft's own naming makes this subtle: the first-party mechanism for AI on Azure Local is branded "Foundry Local on Azure Local", the same product family as on-device Foundry Local but a different deployment target. Q1 and Q6 keep that distinction explicit so the two spikes do not blur together.

Grounding documents read first: the model roster (authoritative roster), docs/research/SPIKE-01-image-model.md, and docs/research/SPIKE-02-voice-model.md (the cloud baseline this is measured against).

Q1. What running AI / Foundry workloads on Azure Local actually means today ​

Question. Confirm Microsoft's current first-party mechanism for running Foundry-style model workloads on Azure Local, as of the research date, rather than trusting the brief's framing.

Findings.

  • The mechanism is "Foundry Local on Azure Local" (preview), an Azure Arc extension that runs on an Arc-enabled Kubernetes cluster. Microsoft: "Foundry Local on Azure Local brings AI inference to your Azure Local environment. Deploy and run AI models on an Arc-enabled Kubernetes cluster with Kubernetes-native operations." It is installed as an Azure Arc extension (extension type Microsoft.Foundry) on AKS enabled by Azure Arc, or via Helm. Source: What is Foundry Local on Azure Local?, AI workloads on Azure Local, Available extensions for Azure Arc-enabled Kubernetes clusters.

  • Control plane and API surface. It uses an operator-based control plane (a Kubernetes inference operator) that reconciles two custom resources, Model (metadata, catalog or your own registry) and ModelDeployment (runtime intent: scaling, endpoint exposure). Catalog sync pulls Foundry model-catalog metadata into the cluster for discovery. Inference is exposed through internal services or the Kubernetes Gateway API and is OpenAI-compatible: /v1/chat/completions for generative tasks and /v1/predict for predictive tasks. Source: overview (architecture summary), AI workloads on Azure Local.

  • Two inference engines, and this is the load-bearing constraint for our roster. Generative inference runs on either the default ONNX-GenAI engine (CPU or GPU) or the vLLM engine (GPU only, high-throughput). Predictive inference (classification, scoring, regression) uses a separate ONNX-Runtime predictive engine and accepts only bring-your-own ONNX models. Both are text / tensor engines. There is no diffusion / text-to-image engine and no text-to-speech engine in the supported-workloads list. Source: overview (supported workloads), Inference runtimes in Foundry Local on Azure Local, Model catalog and sourcing (image selection table).

  • Lifecycle and access. Foundry Local on Azure Local is in public preview, available by request during preview via an access form (aka.ms/FoundryLocalAzure_PreviewRequest). Preview carries no SLA and "can change or have limited capabilities before general availability." The June 2026 release (extension version 2605) added multinode Kubernetes deployment, the vLLM runtime, disconnected-environment operations, automatic GPU inference tuning, and model caching. Source: What is Foundry Local on Azure Local?, What's new in Foundry Local on Azure Local (June 2026).

  • Security and connectivity. Endpoints are protected with API keys or Microsoft Entra ID token validation (via a Microsoft identity sidecar) plus TLS through the Gateway API. It supports disconnected / air-gapped operation: model artifacts pull from a local container registry, cert-manager and trust-manager handle certificates locally, and authentication can integrate with local Active Directory. Source: overview, Available extensions.

  • Two adjacent capabilities exist on the same Arc-enabled Kubernetes substrate, both preview: (1) Agentic Retrieval in Foundry Local (an on-prem agentic RAG platform: document ingestion, embeddings, vector search, MCP server, local chat UI, Azure RBAC), and (2) plain Foundry Local model serving beneath it. Agentic Retrieval's recommended language model is gpt-oss-20b served by Foundry Local on the same cluster. Source: AI workloads on Azure Local, Choose your language model for Agentic Retrieval.

Net. The first-party Azure Local mechanism is Kubernetes-native (AKS Arc) model serving of open-weight text and predictive models through an OpenAI-compatible API, managed as an Azure Arc extension, in preview by request. It is not a lift of the cloud Foundry hosted-model catalog onto your hardware; it is a curated open-weight catalog plus bring-your-own-model, restricted to text-generation and predictive engines.

Q2. Model catalog fit for this repo's roster ​

Question. Which models on this repo's existing or planned roster (the model roster) can run on Azure Local at all, and which are cloud-only.

Findings.

  • The Foundry Local catalog is small language models and open-weight models, not the proprietary hosted models this repo runs. Microsoft's representative catalog list covers "general chat, reasoning, coding, speech, and multimodal workloads" and names Phi-4 and Phi-4-mini (Microsoft), Mistral 7B and Mixtral-8x7B (Mistral AI), DeepSeek R1 Distill Qwen 7B/14B, Qwen2.5 7B/14B and Qwen2.5 Coder (Alibaba), gpt-oss-20b (OpenAI open-weight), the Nemotron family (NVIDIA), and Whisper Large v3 Turbo (OpenAI, speech-to-text). Source: Model catalog and sourcing (example models), Generative small language models. The live, complete list is at aka.ms/FL_Models.

  • No image-generation path. The supported workloads are generative text (chat / completion) and predictive (classification, scoring). There is no diffusion or text-to-image engine. So neither MAI-Image-2.5 nor any FLUX model has a runtime here, independent of the fact that they are also proprietary hosted models not offered as open weights. Source: overview (supported workloads), Inference runtimes.

  • No text-to-speech path. The only speech model in the catalog is Whisper Large v3 Turbo, which is speech-to-text transcription, the inverse of what MAI-Voice-2 does. There is no TTS engine, so MAI-Voice-2 has no on-prem path here. Source: Model catalog and sourcing (example models).

  • The reviewer-LLM role is the one place with genuine, if inexact, fit. The planned gpt-5.6-terra (proprietary OpenAI hosted) and grok-4-1-fast-reasoning (proprietary xAI hosted) cannot run on Azure Local: they are cloud-only hosted models, not open weights. But comparable open-weight reasoning models are in the catalog: gpt-oss-20b, Phi-4 / Phi-4-mini-reasoning, DeepSeek R1 Distill Qwen, and Qwen2.5. These could serve a local "second eyes" reviewer or a RAG assistant on the existing cluster. They are substitutes, not the exact planned pairing, and their fitness for the vision part of the reviewer role (grading generated images) is unverified (see UNKNOWN). Source: Generative small language models, Choose your language model for Agentic Retrieval.

  • Sora / Sora 2 (video) has no path: no video-generation engine, proprietary hosted, already rejected for the core look in the roster. Source: roster the model roster; absence from supported workloads.

Roster fit table (source per row):

Model (roster id)KindRuns on Azure Local today?Basis
MAI-Image-2.5 (mai-image-25)image genNoProprietary hosted; also no image-gen engine on Foundry Local on Azure Local (supported workloads)
FLUX.2-pro (flux-2-pro)image genNoProprietary (Black Forest Labs) hosted; no image-gen engine (inference runtimes)
FLUX.1-Kontext-pro (flux-1-kontext-pro)image genNoAs above
FLUX-1.1-pro (flux-1-1-pro)image genNoAs above
MAI-Voice-2 (mai-voice-2)voice (TTS)NoNo TTS engine; only speech-to-text (Whisper) is in the catalog (model catalog)
gpt-5.6-terra (gpt-5.6-terra)reasoning / reviewNo (comparable open-weight exists)Proprietary OpenAI hosted; not open weights. Comparable local: gpt-oss-20b, Phi-4-reasoning, DeepSeek-R1-distill, Qwen2.5 (generative SLMs)
grok-4-1-fast-reasoning (grok-4-1-fast-reasoning)reasoning / reviewNo (comparable open-weight exists)Proprietary xAI hosted; not open weights. Comparable local reasoning models as above
Sora / Sora 2 (sora)video genNoProprietary hosted; no video-gen engine; already rejected for core look (roster)

Net. Zero of the roster's proprietary generation and hosted-reasoning models run on Azure Local today. The only real on-prem fit is a substitute open-weight reasoning model for the reviewer / RAG role, not any of the exact roster entries.

Q3. Hardware and cluster requirements ​

Question. GPU support matrix for Azure Local, minimum cluster shape for a realistic image / voice / reasoning workload, and how it compares to the always-on cluster this owner already operates.

Findings.

  • Two GPU attach models on Azure Local. Discrete Device Assignment (DDA) dedicates a whole physical GPU to one VM (native driver, full CUDA, highest performance, low density, no live migration). GPU Partitioning (GPU-P) splits one GPU into fractional partitions shared across VMs (high density, supports live migration on OS build 26100+ with NVIDIA vGPU software v18+). A physical GPU is one or the other, not both. Source: Prepare GPUs for Azure Local, Manage GPUs using partitioning, GPU partitioning (Windows Server).

  • AKS Arc, which is what Foundry Local runs on, uses DDA passthrough only. The Azure Local platform exposes physical GPUs to AKS Arc worker VMs "using discrete device assignment (DDA) with vfio-pci PCI passthrough." AKS Arc does not support GPU partitions. GPUs are supported only on Linux node pools, not Windows. Source: Use GPUs in AKS on Azure Local for multi-rack, Prepare GPUs for Azure Local (support matrix), Use GPUs for compute-intensive workloads in AKS on Azure Local.

  • AKS Arc GPU support matrix (Linux node pools), with the release that added each: NVIDIA A2 (2311.2), A16 (2402.0), T4 (2408.0), L4 / L40 / L40S (2512.0), RTX Pro 6000 (2603.0). Multi-rack deployments expose RTX Pro 6000 Blackwell SKUs: Standard_NC16_RTX6000Pro_1 (16 vCPU, 64 GiB, 1 GPU) and Standard_NC32_RTX6000Pro_1 (32 vCPU, 128 GiB, 1 GPU); the two-GPU SKUs are in the catalog but not yet GA. Source: Use GPUs in AKS on Azure Local (support matrix), multi-rack GPU node pools.

  • You cannot start an AKS Arc cluster on a GPU SKU. The control plane and system node pool must use a standard non-GPU VM size; you add a GPU-enabled Linux user node pool afterward (az aksarc nodepool add --node-vm-size Standard_NC16_RTX6000Pro_1 ...). The platform validates physical GPU capacity at command time and fails if the cluster lacks enough physical GPUs to back the requested node count. Source: multi-rack GPU node pools (step 2), Use GPUs in AKS on Azure Local.

  • VRAM sizing for a realistic reasoning workload. Per the vLLM model reference, Phi-3.5-mini needs about 8.4 GB VRAM and Phi-4-mini about 7.8 GB (A10-class, Ampere CC 8.0+). gpt-oss-20b recommends Blackwell (CC 10.0+) and about 14.8 GB by the vLLM reference, but the Agentic Retrieval prerequisites call for "40 GB+ VRAM or multi-GPU setups" for gpt-oss-20b in practice. vLLM requires Ampere CC 8.0 or higher. So a serious local reviewer model wants an L40S (48 GB) or RTX Pro 6000 (96 GB) class card; a small Phi-class model fits an A2 / A10 / T4. Source: vLLM runtime model reference, Generative SLMs (VRAM table), Create your language model endpoint (prereqs).

  • Agentic Retrieval's node shape, as a concrete worked example. Its default "combined" mode needs 2 GPU VMs (knowledge layer: text and image embedding) plus 3+ CPU VMs; "agentic" mode needs no GPU node pool (3+ CPU VMs). For the Agentic Retrieval preview, AKS Arc "supports only Nvidia A2 and A16 GPUs," a narrower list than base AKS Arc, an app-level constraint to note. Source: Prepare AKS cluster for Agentic Retrieval, Enabling GPU on AKS enabled by Azure Arc.

  • Hardware must be GPU-validated Azure Local. Supported GPU solutions are listed in the Azure Local catalog under the "AI workload" solution capability; NVIDIA validates its GPUs and server platforms separately through its vGPU software. Source: Prepare GPUs for Azure Local (supported GPU models).

  • How this maps to the owner's existing cluster. Azure Local is the current name for Azure Stack HCI, so the owner's S2D storage, Hyper-V compute, Network ATC intent-based networking, and failover clustering are the same substrate. What this workload adds on top: (1) GPU-validated hardware plus supported NVIDIA cards (none of the owner's existing capability guarantees GPUs are present), (2) an AKS enabled by Azure Arc cluster with a GPU-enabled Linux user node pool using DDA passthrough (so the whole GPU is consumed by the AKS worker VM, not GPU-P shared), and (3) the Foundry Local Arc extension (preview access request). DDA-only for AKS Arc means no live migration of the GPU node and lower GPU density than GPU-P, a real trade against the owner's usual clustered-VM HA expectations.

Q4. Identity, networking, and governance fit with this repo's CAF / WAF patterns ​

Question. Can Azure Local workloads still use this repo's existing Entra ID, Key Vault by name, and cost-governance patterns, or does Arc's hybrid model change those assumptions.

Findings.

  • Entra ID fits directly. Foundry Local on Azure Local validates Microsoft Entra ID JSON web tokens through a Microsoft identity sidecar as an alternative to API keys, and Agentic Retrieval controls access with Microsoft Entra integration and Azure RBAC. Arc-enabled Kubernetes supports Azure RBAC for Kubernetes. So the repo's Entra ID identity model carries over to the on-prem endpoint. Source: overview (Entra ID authentication), AI workloads on Azure Local (Agentic Retrieval, Azure RBAC).

  • Azure-managed governance carries over because Azure Local is Arc-managed. Azure Local instances and their AKS Arc clusters surface as Azure resources, so Azure Policy, Azure Monitor (AKS Arc platform metrics and alerts are supported), Azure RBAC, and Cost Management apply the same way they do for the cloud AIServices resource. CAF resource / resource-group naming and WAF pillar reviews apply unchanged. Source: What's new in AKS enabled by Azure Arc (metrics, alerts, Azure RBAC), Architecture best practices for Azure Local.

  • Networking and secure exposure fit WAF. Inference is exposed through the Kubernetes Gateway API with TLS; disconnected mode uses locally installed cert-manager / trust-manager and can integrate with local Active Directory. This is compatible with a WAF security-pillar posture (private endpoint, TLS in transit, token auth). Source: overview, Available extensions (disconnected).

  • Key Vault integration is not explicitly documented for the Foundry Local extension's own secrets. The docs describe API-key and Entra-ID auth and (for Agentic Retrieval) an app registration for authentication, but do not name Azure Key Vault as the secret store for the extension. The repo's "vaults by name" convention still applies to any app-side secrets, and Arc-enabled Kubernetes supports secret management patterns generally, but a first-party "store the Foundry Local API key in Key Vault" flow was not found. Marked UNKNOWN below. Source: absence in overview and Create your language model endpoint.

Net. The repo's CAF / WAF governance model (Entra ID, Azure RBAC, Azure Policy, Azure Monitor, Cost Management, CAF naming) applies to an Azure Local track with little change, because the whole thing is Arc-managed. The one open item is a documented Key Vault path for the extension's secrets.

Q5. Cost model versus the cloud AIServices path ​

Question. Azure Local's licensing / subscription cost structure versus the pure-cloud per-token AIServices path this repo uses. Cheaper, more expensive, or depends on hardware already owned.

Findings.

  • Azure Local bills a flat rate per physical processor core per month, on your Azure invoice. It is "priced per core on your on-premises servers," billed through Azure Commerce as one itemized monthly bill. The charge is on physical cores present in the system and does not vary with the number of VMs or vCPUs you run. There is a 60-day free trial after registration. Source: Azure Local billing and payment, Architecture best practices for Azure Local (cost optimization).

  • The per-core figure and add-ons. Microsoft's own GitHub Enterprise Local billing page and an Azure Q&A both describe the Azure Local Host Service fee at "$10/core/month," with a separate Windows Server guest add-on (about $23.30/core/month) that applies only if you run Windows Server guests, and note that Azure Hybrid Benefit (eligible Windows Server licenses with active Software Assurance, or a Windows Server subscription) can waive the host fee. AI services are billed separately from the Azure Local host charge. The exact current dollar figure should be read live at azure.microsoft.com/pricing/details/azure-local/ (a client-rendered pricing page); treat the per-physical-core model as first-party confirmed and the specific number as confirm-at-quote. Source: GitHub Enterprise Local billing overview, Azure Local subscriptions and OEM license Q&A, Azure Local billing.

  • There is no per-token or per-character inference charge for Foundry Local on Azure Local. Inference runs on your own cores and GPUs, so beyond the per-core host fee (and your own hardware, power, and cooling) there is no metered token cost. This is the structural opposite of the cloud path, where SPIKE-01 and SPIKE-02 confirm token-metered image billing and $22-per-1M-character voice billing. Source: overview (no consumption meter for inference); cloud baseline: docs/research/SPIKE-01-image-model.md, docs/research/SPIKE-02-voice-model.md.

  • So the answer is: it depends almost entirely on hardware already owned and on utilization. Azure Local is a fixed-cost model (per-physical-core host fee plus capex on GPU-validated hardware) that rewards high, steady utilization. The cloud AIServices path is pure consumption that rewards bursty, low-volume use. This repo's actual generation workloads are exactly the bursty, low-volume kind: a one-time back-catalog backfill plus per-story batches, well inside the owner's $100/month voice cap. For that shape, cloud is almost certainly cheaper unless the Azure Local cluster and GPUs already exist and are justified by other workloads, in which case the marginal cost of adding a local reviewer or RAG model is close to the electricity to run it.

Q6. Relationship to SPIKE-08 (Foundry Local on-device) ​

Question. Are Foundry Local (on-device) and Azure Local complementary, competing, or is one a strict subset of the other.

Findings.

  • They are the same product family, Foundry Local, with two different deployment targets, and Microsoft frames them as a deliberate pair. Microsoft: Foundry Local "is one of two options to run AI models locally." Option one, Foundry Local (on-device, SPIKE-08): "an end-to-end local AI solution for shipping applications that run entirely on the user's device," with an SDK (C#, JavaScript, Rust, Python), data staying on the device, offline operation, no per-token cost, and no Azure subscription required. Option two, Foundry Local on Azure Local (this spike): "enterprise-scale AI inference on your own infrastructure with Kubernetes-native operations and Azure Arc management," deployed to Arc-enabled Kubernetes clusters. Source: What is Foundry Local?, What is Foundry Local on Azure Local?.

  • Complementary, not competing, and neither is a strict subset. Device-scale versus cluster-scale. On-device runs on end-user hardware with no Azure subscription and offline-by-default, but no shared multi-node scale, no Azure RBAC / Arc governance, no cluster HA. Azure Local requires Azure Arc and a subscription (billed per core) and adds multinode Kubernetes scale, Entra ID / Azure RBAC, Azure Monitor, and disconnected-cluster operation, but is not something you embed in a client app. They share the OpenAI-compatible API surface and the curated open-weight catalog concept, so a model proven on one is portable to the other. Source: What is Foundry Local?, overview.

  • The same roster conclusion holds for both. Both target open-weight text-generation and predictive / STT models, so neither gives an on-prem home to the proprietary image (MAI-Image, FLUX), voice (MAI-Voice-2 TTS), or hosted-reasoning (gpt-5.6-terra, grok) models. The reviewer / RAG substitute story is the shared opportunity. Cross-check the exact device-side catalog and SDK details against SPIKE-08 once both are final.

What is still UNKNOWN ​

  1. Whether any Foundry Local catalog model is vision-capable enough for the image-grading reviewer role. The catalog concept mentions "multimodal workloads," but the representative list and reference-models pages show text-generation, coding, reasoning, and STT (Whisper), not a named vision LLM for grading generated art. Resolve by reading the live catalog at aka.ms/FL_Models for a multimodal / vision entry (for example a Phi vision or Qwen-VL variant) and testing image-input on /v1/chat/completions. This decides whether Azure Local can host the vision half of the reviewer pair or only the text half.
  2. Exact current per-physical-core price for Azure Local. The per-core billing model is first-party confirmed; the specific dollar figure ($10/core/month host fee, ~$23.30/core Windows Server guest add-on) comes from a Microsoft Q&A and the GitHub Enterprise Local billing page, not from a rendered pricing table. Resolve by reading azure.microsoft.com/pricing/details/azure-local/ live or via an account rep quote at design time.
  3. A documented Azure Key Vault path for the Foundry Local extension's own secrets (API keys). Docs describe API-key and Entra-ID auth and an app registration, but do not name Key Vault as the store. Resolve by checking whether the extension supports a Key Vault-backed secret or CSI secret store, versus the repo having to manage the API key itself.
  4. Whether the two-GPU multi-rack SKUs and the full L4 / L40 / L40S / RTX Pro 6000 matrix are GA in the region and release the owner runs. GPU support is release-gated (for example RTX Pro 6000 arrived in 2603, two-GPU SKUs are "not yet GA"). Resolve against the owner's actual Azure Local build number and the Azure Local catalog "AI workload" solution list at design time.
  5. Preview-to-GA timeline and SLA for Foundry Local on Azure Local. It is public preview, by request, no SLA. No GA date is published. Any production reliance waits on GA or an accepted preview-risk decision, the same preview-risk posture the owner already accepted for the cloud MAI models.

Recommendation ​

  1. Do not treat Azure Local as a home for this repo's image or voice generation backbone. Confirmed: zero of the proprietary generation models (MAI-Image-2.5, the three FLUX models, MAI-Voice-2, Sora) and neither hosted reasoning model (gpt-5.6-terra, grok) can run on Foundry Local on Azure Local today, because it serves only open-weight text-generation and predictive / STT models, with no image-generation and no TTS engine. The cloud AIServices path stays the backbone for scene art and narration.

  2. An Azure Local track is nonetheless worth a narrowly scoped ADR, for one reason: the owner already runs Azure Local (Stack HCI, S2D, Hyper-V, Network ATC) and has the operational expertise, so the marginal cost of a local open-weight inference capability is mostly GPU hardware plus power, not a greenfield platform build. The defensible use case is an on-prem, sovereign reviewer / RAG capability: a local open-weight reasoning model (gpt-oss-20b, Phi-4-reasoning, DeepSeek-R1-distill, or Qwen2.5), optionally fronted by Agentic Retrieval for grounding over the repo's own prompt and brand material, keeping that content on-prem. Scope the ADR to that, explicitly not to the generation pipeline.

  3. Gate the ADR on three preconditions the spike surfaced: (a) GPU-validated Azure Local hardware with a supported NVIDIA card actually present (the owner's existing capability does not imply GPUs), (b) an AKS Arc cluster with a GPU-enabled Linux user node pool using DDA passthrough (accepting no GPU live-migration and lower density than GPU-P), and (c) preview access approval for the Foundry Local Arc extension, with preview-no-SLA risk accepted the same way it was for the cloud MAI models.

  4. Before the ADR, resolve UNKNOWN #1 (is there a vision-capable local model good enough to grade generated art). If yes, Azure Local could host a genuine on-prem second-eyes reviewer; if no, its role narrows to a text-only reviewer / RAG assistant, which weakens (but does not kill) the case, since the reviewer pair's core job is grading images.

  5. Follow the standard gate. If pursued, a follow-up ADR (docs/adr/ADR-NNNN-azure-local-reviewer-track.md) is required before any deployment work: spike, then ADR, then design, then a gated deploy, the same discipline as every other model in this backlog. This spike is the research input; it authorizes no provisioning.

Net: no on-prem path for the generation backbone, a real but substitute-only fit for the reviewer / RAG role, and a genuine "worth an ADR" only because the owner's existing Azure Local investment makes the incremental cost small. The recommendation is a bounded reviewer / RAG ADR, not an Azure Local rehost of the media pipeline.

Sources ​

All first-party (Microsoft Learn or Microsoft docs), reviewed 2026-07-22: