Homestead Foundry

Appearance

ADR-0014: Track 3 deployment layers, GPU scope, and authentication for Foundry Local on Azure Local ​

  • Status: Proposed
  • Date: 2026-07-25

This ADR resolves the open decision ADR-0011 left for track 3 ("where the ARM to Kubernetes seam sits is still an open decision"), re-scopes the GPU precondition ADR-0009 imposed, and settles the authentication and secret question SPIKE-09 left open. It is grounded entirely in docs/research/SPIKE-19-foundry-local-azure-local-deployment.md; where that spike logged an UNKNOWN, this ADR carries it forward rather than resolving it.

This ADR authorizes no deployment and no spend. It authorizes one free, reversible preview access request, described in the Decision, and nothing else. It records how track 3 is automated and under what preconditions, gated the same way every other decision in this backlog is: spike, then ADR, then design, then a gated deploy.

It supersedes ADR-0011's track 3 decision 2 on the layer boundary, amends ADR-0009's precondition (a), and closes SPIKE-09's UNKNOWN #3. ADR-0011's substantive choice of declarative Bicep for the Azure-projected surface survives.

Context ​

ADR-0011 decided track 3's automation form in 2026-07, authorized no build, and explicitly deferred one question: where the boundary between ARM and Kubernetes sits. Track 3 has sat unbuilt since, gated on ADR-0009's three preconditions, the first of which is a GPU hardware requirement. SPIKE-19 re-examined the track against a first-party documentation set that has grown substantially since SPIKE-09 was written, and found that the deferred question is now answerable in a single table, and that the GPU gate is no longer justified as an absolute.

The forces this decision must reconcile, all from SPIKE-19:

  1. The deployment is not a two-layer stack. Microsoft's documented install sequence puts two Kubernetes-layer steps before the ARM extension can be created: Gateway API CRDs (v1.4.0 or later), then Gateway API Inference Extension CRDs (v1.5.0 or later), then Istio (istio-base and istiod, 1.29 or later) via Helm as the Gateway API provider. Only then can the two ARM extensions be created. ADR-0011 track 3 decision 2 states Bicep "owns the Azure-projected resources up to and including the extension install," which misses that mandatory non-ARM prerequisites sit underneath it.

  2. Ordering is load-bearing and Microsoft warns about it. The CRDs must precede Istio, because istiod registers the istio GatewayClass only once it sees them, and "installing the CRDs and Istio in the reverse order forces an istiod restart and is reported as flaky on some clusters." The Inference Extension CRDs must precede istiod so it picks up InferencePool support at startup, and istiod needs pilot.env.ENABLE_GATEWAY_API_INFERENCE_EXTENSION=true. There is a documented gate: kubectl get gatewayclass istio must show Accepted. ARM cannot express any of this.

  3. A GPU is required only for specific workloads. Foundry Local on Azure Local "supports CPU-backed and GPU-backed deployments," and "you need a GPU node pool only for GPU model variants such as *-cuda-gpu and deployments that use vLLM." compute: cpu is a first-class ModelDeployment value and the default onnx-genai runtime supports CPU. ADR-0009 precondition (a) treats GPU-validated hardware as a blanket gate before any design or deploy work, and current documentation does not support that.

  4. Where GPU is still mandatory, it is mandatory. vLLM is GPU-only. Agentic Retrieval's recommended model, GPT-OSS-20B, "requires its own GPU." NVIDIA only, via DDA passthrough, from a release-gated SKU matrix; AMD is unsupported.

  5. There is no documented Key Vault path for the extension's secrets, and Entra ID authentication makes the question moot. API keys are generated by the operator into a Kubernetes Secret, not Key Vault. Entra ID authentication validates JWTs through an identity sidecar and evaluates authorization with Azure RBAC, injecting an msi-adapter sidecar. Choosing Entra removes the key entirely.

  6. Entra ID authentication is a forward-compatibility gate, not just a preference. It must be enabled at extension install time to keep Agentic Retrieval possible later; disabling it "prevent[s] Agentic Retrieval from connecting to your deployed models." It is also unavailable on the Helm onboarding channel.

  7. The model layer is genuinely declarative and registry-friendly. Catalog models need no Model custom resource because of lazy registration; only ModelDeployment is needed. Runtime is inferred from the catalog's framework field. The spec surface a generator needs is small.

  8. ModelDeployment defaults cannot run a real model. Defaults are requests.memory: 256Mi and limits.memory: 1Gi, against a CPU catalog model of roughly 4.8 GB.

  9. Cluster defaults are a trap too. Microsoft states directly: "Don't use the az aksarc create default worker size Standard_A4_v2 (8 GiB). Use at least Standard_D4s_v3." Recommended is Standard_D8s_v3.

  10. East US is a supported region, so track 3 can match ADR-0001 and ADR-0004 with no new region decision.

  11. Every current example uses az aksarc and az k8s-extension, never Bicep. ADR-0011 asserted Bicep for the AKS Arc cluster and node pools without testing that assertion.

Decision ​

1. Track 3 has three layers, and Bicep owns the middle one ​

The deployment shape is recorded as follows, superseding ADR-0011 track 3 decision 2:

LayerContentsToolingDeclarative in ARM?
Prerequisite (Kubernetes and Helm)Gateway API CRDs, then Gateway API Inference Extension CRDs, then Istio istio-base and istiod as Gateway API providerkubectl apply --server-side, helm installNo
Platform (ARM)azure-cert-manager (extensionType Microsoft.CertManagement), then inference-operator (extensionType Microsoft.Foundry), both Microsoft.KubernetesConfiguration/extensions against connectedClustersBicepYes
Intent (Kubernetes)Generated ModelDeployment manifestskubectl applyNo

Bicep owns the platform layer only. An ordering wrapper (script or pipeline) owns the sequence across all three layers, because ARM cannot express "these CRDs must exist and this GatewayClass must be Accepted before this extension is created." The wrapper is a first-class deliverable of this track, not glue.

The prerequisite layer's internal order is mandatory and the wrapper enforces it: Gateway API CRDs, then Inference Extension CRDs, then istio-base, then istiod with ENABLE_GATEWAY_API_INFERENCE_EXTENSION=true, then verify gatewayclass istio is Accepted before proceeding to the platform layer.

The disconnected path (Azure Local Disconnected Operations 2604.3.0 or later) changes the source of the prerequisite layer, not its shape: the expansion pack bundles Istio and both CRD sets into the local edgeartifacts registry. The three-layer model holds for both connected and disconnected deployments.

2. Istio is adopted as a Gateway API provider only, not as a service mesh ​

Foundry Local routes model traffic through the Kubernetes Gateway API rather than an Ingress controller, and Microsoft is explicit that mesh features are optional: "sidecar injection, ambient mode, and mesh mTLS are optional and aren't required by the inference operator." Track 3 therefore takes the Gateway API dependency and does not adopt a service mesh. This is recorded so the operational commitment is not overstated to a future reader.

3. ADR-0009 precondition (a) is amended from a blanket GPU gate to a per-workload one ​

GPU-validated Azure Local hardware is not a precondition for track 3 generally. It is a precondition for specific workloads:

WorkloadGPU required?
Text reviewer inference on onnx-genai, CPU catalog variantNo
*-cuda-gpu catalog variantsYes
vLLM runtime (high throughput, multi-replica)Yes
Agentic Retrieval with GPT-OSS-20BYes

Where GPU is required it is NVIDIA only, via DDA passthrough, from the supported SKU matrix (Standard_NC*_A2, Standard_NC*_L4_*, Standard_NC*_L40_*, Standard_NC*_L40S_*, Standard_NC*_RTX6000Pro_*, Tesla T4 Standard_NK*), with CUDA drivers on the nodes and the NVIDIA Kubernetes device plugin configured. AMD is unsupported. SPIKE-09's finding that AKS Arc uses DDA passthrough and not GPU partitioning, with the consequent loss of GPU live migration, stands unchanged.

ADR-0009's preconditions (b) and (c) stand: an AKS Arc cluster (now with a minimum Kubernetes version of 1.29), and preview access approval. Precondition (b) is amended only in that the user node pool need not be GPU-enabled for the first increment.

This amendment is recorded here and must be reflected in ADR-0009, so that a reader consulting ADR-0009 alone does not re-impose the gate.

4. The first increment is CPU-only text reviewer inference in East US ​

Scope, committed:

  • An AKS Arc cluster on Azure Local, Kubernetes 1.29 or later, in East US (a supported region, consistent with ADR-0001 and ADR-0004).
  • A non-GPU worker node pool at Standard_D8s_v3 (8 vCPU / 32 GiB) or better, with at least 28 GiB allocatable per node. Standard_A4_v2, the az aksarc create default, is explicitly forbidden. Minimum acceptable is Standard_D4s_v3.
  • The prerequisite layer, then both ARM extensions, then one ModelDeployment for a CPU catalog model in the Phi-4-mini-instruct-generic-cpu class with explicit resource values.
  • Entra ID authentication enabled, per decision 5.
  • A TLS termination certificate from a company or well-known public CA. Self-signed is not used for anything beyond throwaway evaluation, per Microsoft's guidance.

Deliberately out of scope for the first increment, each a clean later increment: GPU node pools, the vLLM runtime, Agentic Retrieval, predictive workloads (BYO-only in preview), multi-node scale-out, and disconnected operation.

The storeModel.cacheJob.resources memory defaults (16 GiB request, 32 GiB limit) are tuned down at install time for the first increment, since it uses one small model and a 32 GiB limit against a 32 GiB node is self-defeating.

5. Entra ID authentication is the authentication mode, and this closes the secret question ​

Track 3 uses Entra ID authentication (entraAuth.enabled: true, the default), with an app registration supplying entraAuth.tenantId and entraAuth.clientId at extension install time. Consequences, all intended:

  • No API key to store. SPIKE-09's UNKNOWN #3, asking for a documented Key Vault path for the extension's API keys, is closed as answered in the negative and made moot: keys live in a Kubernetes Secret, no Key Vault integration is documented, and Entra ID removes the key from the design. This satisfies ADR-0005 better than any Key Vault workaround would.
  • Authorization resolves to Azure RBAC, via the injected Entra Auth SDK and msi-adapter sidecars, which is the governance property track 3 exists to provide and track 2 cannot.
  • The Helm onboarding channel is disqualified for anything but throwaway evaluation, because Entra ID authentication is unavailable on it. The Arc extension path is the deployment channel for this track.
  • The Agentic Retrieval option is preserved. Entra must be enabled at install to keep it possible later, and ADR-0009 contemplates RAG, so disabling it now would quietly foreclose a decision ADR-0009 wants open.

If an API key is ever used despite this decision, for example by a client that cannot present an Entra token, this repository's no-secrets-in-git rule applies unchanged: the key is referenced by name only, never by value, in any committed file, and the value lives in the tenant Key Vault. Note the operator generates the key into a Kubernetes Secret (<model>-api-keys, field primary-key) and no first-party Key Vault integration is documented, so copying it into Key Vault is a manual, deliberate act with a rotation obligation attached. That obligation is a further reason to prefer Entra and is exactly what decision 5 avoids.

6. The model layer is registry-driven, with explicit resources and never CRD defaults ​

ModelDeployment manifests are generated from the same model registry that drives track 1, mirroring track 1's registry-driven model deployments. Which models the cluster serves is data, not hand-edited resource blocks.

Generated manifests set resources explicitly for every deployment. Relying on ModelDeployment defaults is forbidden: requests.memory: 256Mi and limits.memory: 1Gi cannot run a roughly 4.8 GB model, and this is exactly the class of error a generator prevents and hand-editing produces.

Only ModelDeployment is generated. Model custom resources are not authored for catalog models, because lazy registration creates them from the catalog ConfigMap automatically. Model is authored only for bring-your-own models, which are out of scope for the first increment.

Runtime is not set manually for catalog models; the operator selects it from the catalog's framework field. spec.runtime is set only for BYO models.

7. Drift detection is split, because what-if cannot see the Kubernetes layer ​

Track 1 gets drift detection free: az deployment sub what-if compares desired against actual for every ARM resource. Two of track 3's three layers are invisible to it. ARM has no knowledge of whether the Gateway API CRDs are present, whether istiod is healthy, or whether a ModelDeployment matches the registry. This is the direct cost of the layer split in decision 1 and it has to be answered, not left implied.

Each layer gets its own detection mechanism:

LayerDetects drift withCatches
Prerequisite (Kubernetes and Helm)kubectl get crd against the pinned CRD versions, kubectl get gatewayclass istio for Accepted, and helm list -n istio-system against pinned chart versionsMissing or downgraded CRDs, an istiod that is unhealthy or was upgraded out from under the operator
Platform (ARM)az deployment sub what-if, exactly as track 1Extension removed, config changed, wrong release train
Intent (Kubernetes)kubectl get modeldeployments -A -o json diffed against the manifests generated from the registryA model deployed by hand, a model removed, replicas or resources edited in-cluster

Four rules make this workable:

  1. The generated manifests are the desired state for the intent layer, exactly as the registry is for track 1. Drift is defined as a difference between generated output and live cluster state, so a hand-edited ModelDeployment shows up as drift rather than becoming the new normal.
  2. A single read-only check script covers all three layers and returns one verdict. Splitting detection across three tools is acceptable; making an operator run three unrelated commands and correlate the results by hand is not. This mirrors what check-docs-currency does for the documentation.
  3. Pin the prerequisite versions and check the pinned value, not merely presence. The Gateway API CRDs, the Inference Extension CRDs, and the Istio charts all have minimum versions, and "a CRD exists" is not the same claim as "the right CRD version exists."
  4. The operator's own reconciliation is not drift detection. The inference operator reconciles ModelDeployment resources against cluster state on a 30-second timer, which keeps the cluster matching its CRs. It has no knowledge of the registry, so a CR that never should have existed is reconciled just as faithfully as one that should. Detecting that is this check's job.

what-if remains authoritative for the ARM layer and nothing here weakens it. What changes is the honest admission that for track 3 it covers one layer out of three.

8. Bicep for the AKS Arc cluster itself is provisional pending a what-if test ​

ADR-0011 asserted Bicep for the resource group, AKS Arc cluster, and node pools. The cluster and node pool part is not yet proven: every current first-party example uses az aksarc, and no Bicep example exists in the Foundry Local documentation set. Before the first increment is designed, a minimal Bicep template for the cluster and one node pool is authored and run through what-if against a real Azure Local custom location (a read-only operation).

If Bicep expresses it cleanly, ADR-0011's claim stands and the platform layer grows to include the cluster. If it does not, the cluster moves into the wrapper layer alongside the CRDs, and this ADR is amended to say so. Either way the two extensions remain declarative, so this is a narrowing of scope at worst, not a reversal.

9. The gate: submit the preview access request now ​

Foundry Local on Azure Local is public preview, by request, at aka.ms/FoundryLocalAzure_PreviewRequest, with no SLA and no published GA date. The access request is free, reversible, commits nothing, and gates every other step in this track.

It is submitted now, in parallel with track 2's install test, not serially after it. Waiting to request access until the design is finished is the sequencing error that would keep this track stalled for another cycle.

Preview-with-no-SLA risk is accepted for a non-production reviewer capability, on the same basis the owner already accepted preview risk for the cloud MAI models. It would not be accepted for a production dependency.

Nothing else is authorized: no cluster creation, no extension install, no spend. The what-if test in decision 7 and the access request in this decision are the only actions this ADR permits.

10. CAF naming and governance, unchanged ​

Resources follow this repo's <resource-type-abbreviation>-<workload>-<env>-<region>-<instance> pattern, consistent with ADR-0009 and ADR-0011. Governance uses Entra ID token authentication, Azure RBAC, Azure Policy, Azure Monitor, and Cost Management, all of which reach the Arc-managed endpoint. Consistent with ADR-0005 and ADR-0011, the Arc-connected Azure Local servers and cluster use their system-assigned managed identity for Azure resource access. Unlike track 2 (see ADR-0013), track 3 requires no exception to ADR-0005.

Consequences ​

Positive

  • ADR-0011's open question is closed with a table a future implementer can follow directly.
  • The track no longer requires a GPU purchase to begin. This is the single biggest unblocking effect of this ADR.
  • The secret question is resolved by removing the secret, not by inventing a Key Vault workaround Microsoft does not document.
  • Track 3 delivers what track 2 cannot: a governed AI endpoint with Entra authentication and Azure RBAC. The two tracks are now honestly differentiated rather than presented as near-equivalents.
  • The registry-driven model layer reuses track 1's proven pattern and one source of truth.
  • Two documented default-value traps (Standard_A4_v2, and ModelDeployment memory limits) are recorded before anyone hits them.
  • The preview request goes in now, so lead time runs in parallel with track 2 rather than after it.

Negative, and accepted

  • The wrapper layer is real work and cannot be avoided. A future reader hoping "it is all Bicep" will be disappointed; ARM genuinely cannot express the prerequisite layer.
  • Track 3 takes a hard Istio dependency. Scoped to Gateway API provider only, but it is still a component to install, version, and operate.
  • The track depends on a public preview with no SLA and no GA date, and on preview access approval that is outside this project's control.
  • CPU-only inference latency for a roughly 5 GB model on a Standard_D8s_v3 worker is unmeasured. The first increment may prove functional but too slow to be useful, in which case the GPU question returns as a throughput decision rather than a compatibility one.
  • ADR-0011's Bicep claim for the AKS Arc cluster is now provisional. That is a small retreat from a stated position, made deliberately because the position was never tested.
  • A real TLS certificate is a prerequisite this project has not previously had to source.

Neutral

  • East US requires no new region decision, so ADR-0001 and ADR-0004 are untouched.
  • Disconnected operation remains available later without re-deciding the layer model.
  • Predictive workloads stay BYO-only in preview, which does not affect the reviewer use case.

Alternatives considered ​

  1. Keep ADR-0011's two-layer description (Bicep up to and including the extension, Kubernetes above it). Rejected: it is factually wrong about the prerequisite layer. Following it would produce a Bicep deployment that fails at extension creation because no GatewayClass exists.
  2. Push everything into the wrapper and drop Bicep from track 3 entirely. Rejected: both extensions are genuine ARM resources with real idempotency, drift detection, and what-if support. Discarding that because the layers above and below are imperative would give up the one place declarative IaC actually helps here.
  3. Express the prerequisite layer as Bicep deployment scripts to keep one artifact. Rejected: it would wrap kubectl and helm in an opaque script resource and report only that a script ran, which is the same anti-pattern ADR-0011 correctly rejected for track 2. The ordering and the GatewayClass gate deserve to be visible.
  4. Hold the blanket GPU gate from ADR-0009 until GPU hardware is bought. Rejected: current documentation supports CPU-backed deployments as a first-class path, and this gate is the specific reason track 3 has no code. A CPU-only increment proves the whole automation pattern.
  5. Use API key authentication and store the key in Key Vault. Rejected: no Key Vault integration is documented, so this would mean hand-managing a key extracted from a Kubernetes Secret, and it would foreclose Agentic Retrieval. Entra ID is both the better-governed and the more forward-compatible choice.
  6. Use the Helm onboarding channel for a faster first look. Rejected as the track's channel because Entra ID authentication is unavailable on it, which contradicts decision 5. Permitted only for throwaway evaluation that is never promoted.
  7. Start with vLLM for throughput. Rejected for the first increment: vLLM is GPU-only, so it cannot be the increment that proves the pattern on existing hardware.
  8. Wait for GA before doing anything. Rejected: no GA date is published, the access request is free and reversible, and the same preview-risk posture was already accepted for the cloud MAI models.

Sources ​

Every claim traces to docs/research/SPIKE-19-foundry-local-azure-local-deployment.md, which cites its first-party Microsoft Learn sources inline. The load-bearing ones:

Related records: ADR-0011 (multi-target deployment automation, whose track 3 decision 2 this supersedes on the layer boundary), ADR-0009 (Azure Local reviewer track, whose precondition (a) decision 3 amends), ADR-0005 (governing identity ADR, satisfied by decision 5 with no exception needed), ADR-0001 and ADR-0004 (region, unchanged), ADR-0013 and SPIKE-18 (the track 2 counterparts).