Architecture and governance for AI

This article explains how AI workloads fit into your Azure landing zone and enterprise architecture. It identifies the additional platform capabilities that organizations should consider as AI adoption scales.

Apply the same Azure landing zone principles and best practices to AI workloads as you do to other workloads. AI workloads don't require a different Azure landing zone architecture or governance model. Platform teams should use their existing governance baseline and extend it where needed to account for the AI services and models that the organization plans to use.

Outcome: By the end of architecture and governance planning, you should have an updated governance baseline that supports the AI technologies you plan to use in Azure.

Architectures and decision tree

Platform landing zone

AI adoption should build on the existing Azure landing zone rather than introduce a separate platform structure. AI workloads use the same management group hierarchy and platform capabilities as other Azure workloads. Organizations generally don't need AI-specific management groups. Instead, platform teams should extend existing governance and connectivity capabilities to address the Azure services that AI workloads use.

Governance baseline: The existing Azure landing zone governance baseline should remain the starting point for AI workloads. Platform teams should extend that baseline when AI services introduce requirements that existing policies don't address.

• Azure Policy: Organizations should start with the baseline policies from the platform landing zone accelerator. Platform teams should then add Azure Policy definitions for the AI services that the organization approves, such as Microsoft Foundry models, Foundry Tools, Azure AI Search, and Azure Virtual Machines. Applying these policies through the existing management group hierarchy gives AI workloads consistent guardrails without requiring a separate governance model. Microsoft guidance supports applying the AI-specific policies through the landing-zone management group hierarchy so workload resources inherit them.

• Networking: Existing Azure landing zone networking patterns should also remain the foundation for AI workloads. Platform teams should extend shared connectivity and network protection where AI workloads create additional requirements, particularly for internet-facing services or access to on-premises data.

  • DDoS Protection: Internet-facing AI workloads should use Azure DDoS Protection. The platform team can provide this protection as part of the organization's shared networking capabilities rather than requiring individual workload teams to establish their own platform-level approach.

  • Hybrid connectivity: Organizations that access on-premises data from Azure AI workloads should choose connectivity based on their performance and availability requirements. Azure ExpressRoute provides private connectivity and is appropriate when organizations require predictable connectivity to Azure. Azure VPN Gateway provides encrypted connectivity over the public internet and can suit less demanding connectivity requirements. Existing Azure landing zone guidance also supports VPN Gateway and ExpressRoute as hybrid connectivity patterns.

Workload landing zones

AI workloads use the same workload landing zone principles as other Azure workloads. Each workload should have its own workload landing zone and use one or more Azure subscriptions to separate environments based on the organization's governance and isolation requirements. Azure subscriptions provide boundaries for areas such as quota, cost, governance, and security.

The platform team should place Workload landing zone subscriptions for AI workloads under the existing management group that provides the appropriate governance controls. Internet-facing AI workloads generally belong under the online management group. Internal AI workloads generally belong under the internal ("Corp") management group. Platform teams should use these existing boundaries instead of creating management groups specifically for AI. Azure Policy assignments at the management group level then provide consistent platform guardrails across AI and other workloads.

Reliability

AI workloads should use the organization's existing availability and disaster recovery standards. Platform teams should prepare the Azure landing zone to support multiple regions so workload teams can deploy AI workloads across regions when their business continuity requirements justify the added cost and complexity. Azure landing zone region guidance specifically recommends designing the platform so it can support multiregion workload landing zones.

• Data location: Data location should also be part of the regional design. Workload teams should place retrieval data near the AI services that consume it when latency and performance are important. Regional placement must also meet the organization's data residency requirements.

• Availability and capacity: AI introduces additional regional considerations because service availability, model availability, and capacity can vary by Azure region. Organizations should identify the regions that support their approved AI services and models before making those regions available for AI workloads.

• Failover: Workload teams that require regional failover should also account for AI assets that the workload depends on, such as model deployments and retrieval data. The recovery design should ensure that the secondary region has the resources and data necessary to restore the workload.

Enterprise integration

AI workloads should integrate with enterprise platforms through the same architecture principles as other workloads. The primary AI-specific consideration is access to trusted organizational data.

Organizations should establish a unified approach to data that makes governed data available to AI workloads without requiring every workload team to build separate data integration patterns. Microsoft Fabric can prepare and provide organizational data for analytics and AI scenarios. Azure AI workloads can then consume the appropriate data through approved integration patterns. For guidance on establishing this foundation, see Unified data platform.

Next steps

The architecture and governance considerations for a workload landing zone depend on the AI platforms and technologies that the workload uses. The following guidance provides deeper architecture recommendations for different AI adoption patterns.