Reliability in Azure Storage Discovery

Azure Storage Discovery is a fully managed service that provides visibility into your Azure Blob Storage and Azure Data Lake Storage estate. You create a Storage Discovery workspace to analyze costs, activity, configuration, and errors across storage accounts in multiple subscriptions, resource groups, and regions.

When you use Azure, reliability is a shared responsibility. Microsoft provides a range of capabilities to support resiliency and recovery. You're responsible for understanding how those capabilities work within all of the services you use, and selecting the capabilities you need to meet your business objectives and uptime goals.

This article describes how Azure Storage Discovery is resilient to transient faults, availability zone failures, region-wide failures, and service maintenance. It also describes backup and restore considerations.

Production deployment recommendations for reliability

For production workloads, we recommend that you:

Reliability architecture overview

This section describes some of the important aspects of how the service works that are most relevant from a reliability perspective. The section introduces the logical architecture, which includes some of the resources and features that you deploy and use. It also discusses the physical architecture, which provides details on how the service works under the covers.

Logical architecture

The resource you deploy is a Storage Discovery workspace. You configure one or more workspace roots to identify subscriptions and resource groups that contain the storage accounts you want the service to analyze. You can also create scopes that use resource tags to filter those storage accounts into groups for reporting. A workspace can analyze storage accounts across multiple subscriptions, resource groups, and regions in the same Microsoft Entra tenant.

Storage Discovery periodically analyzes the storage accounts and generates reports that contain insights about your storage estate. Insight aggregation often completes within a few hours but can take more than a day. For more information about aggregation timing and troubleshooting missing report data, see Frequently asked questions for Azure Storage Discovery. The pricing plan determines which insight types are available and how long Storage Discovery retains insights. For more information, see Azure Storage Discovery pricing.

A Storage Discovery workspace is a regional resource. The region that you select for the workspace determines where Storage Discovery stores its configuration and computed insights.

Physical architecture

Storage Discovery is a fully managed service. Microsoft manages the underlying infrastructure, replication of configuration and report data, and service capacity. You don't need to deploy or manage infrastructure for Storage Discovery. For more information about the service architecture and capabilities, see What is Azure Storage Discovery? and Azure Storage Discovery concepts.

Storage Discovery has a control plane and a data plane:

• Control plane: Manages workspace configuration.
• Data plane: Ingests information from storage accounts, generates reports, processes report queries, and stores report data.

Resilience to transient faults

Transient faults are short, intermittent failures in components. They occur frequently in a distributed environment like the cloud, and they're a normal part of operations. Transient faults correct themselves after a short period of time. It's important that your applications can handle transient faults, usually by retrying affected requests.

All cloud-hosted applications should follow the Azure transient fault handling guidance when they communicate with any cloud-hosted APIs, databases, and other components. For more information, see Recommendations for handling transient faults.

Transient faults can affect Storage Discovery in two main ways:

• Client requests: Transient faults can affect requests to deploy or modify workspaces and requests to retrieve reports. If you automate these operations, you're responsible for handling transient faults in your automation. Use Microsoft-provided SDKs where possible, and use bounded retries with an appropriate delay between attempts.

• Insight aggregation: Storage Discovery automatically retries transient faults that occur while it collects and aggregates information from storage accounts. In very rare cases, there can be permanent gaps in the report data when retries are exhausted.

Storage Discovery processes and aggregates insights asynchronously. Most reports become available within a few hours, while some advanced insights may require additional processing time and can take up to 48 hours or longer to appear. If expected insights remain unavailable after 48 hours, verify your workspace configuration and contact Azure Support. For more information, see Frequently asked questions for Azure Storage Discovery.

Resilience to availability zone failures

Availability zones are physically separate groups of datacenters within an Azure region. When one zone fails, services can fail over to one of the remaining zones.

Consider these three components when you plan for availability zone failures:

• Control plane: In regions that have availability zones, the Storage Discovery control plane is zone-redundant. The control plane remains available if a zone is unavailable, although individual management operations might experience transient interruptions. Microsoft selects the zones and manages zone redundancy. You can't select the zones, and you don't need to configure zone redundancy.
• Data plane: The Storage Discovery data plane ingests information, generates reports, processes report queries, and stores report data. Stored report data is zone-redundant, but some data-plane compute components can be affected by a failure in the zone that hosts them. Ingestion, report generation, and report queries can pause until the affected zone recovers. Microsoft manages the data plane and the zone redundancy of stored report data.
• Analyzed storage accounts: Your storage accounts aren't part of the Storage Discovery service. Their redundancy configuration determines whether they remain available during a zone failure. Storage Discovery can't collect new information from an unavailable storage account. Even when an analyzed account remains available, Storage Discovery data-plane processing can pause as described in Behavior during a zone failure.

Requirements

• Region support: Zone redundancy is available when you deploy the workspace in a region that supports both Storage Discovery and availability zones. For the current list of regions that support Storage Discovery, see Select an Azure region for your deployment. For regions that provide availability zones, see Azure regions with availability zones.

• Pricing plan support: Zone redundancy for the control plane and stored report data is available for both the Free and Standard pricing plans.

Cost

Storage Discovery doesn't provide a separate setting or billing meter for zone redundancy. Your workspace cost depends on its pricing plan and the storage resources and objects in scope. For more information, see Azure Storage Discovery pricing.

Configure availability zone support

When you create a Storage Discovery workspace in a supported region, Storage Discovery automatically enables zone redundancy for the control plane and stored report data. You can't disable zone redundancy or select the availability zones that the workspace uses. For more information about creating a workspace, see Create an Azure Storage Discovery workspace.

Behavior when all zones are healthy

This section describes what to expect when your Storage Discovery workspace is in a region that supports availability zones, the storage accounts that it analyzes use zone-redundant storage (ZRS), and all availability zones are operational.

• Cross-zone operation: The Storage Discovery data plane periodically ingests information from storage accounts, generates reports, processes report queries, and stores report data. Infrastructure in any availability zone might perform these operations.

• Cross-zone data replication: Storage Discovery stores report data redundantly across multiple availability zones.

Behavior during a zone failure

This section describes what to expect when your Storage Discovery workspace is in a region that supports availability zones, the storage accounts that it analyzes use ZRS, and there's an outage in one availability zone.

• Detection and response: Microsoft detects the availability zone failure and manages the service response. You don't need to initiate failover for the Storage Discovery workspace.

• Notification: Microsoft doesn't automatically notify you when a zone is down. However, you can use Azure Service Health to understand the overall health of the service, including any zone failures, and you can set up Service Health alerts to notify you of problems.

• Active requests: Active management operations and report queries might be interrupted during a zone failure. Retry failed requests by following the guidance in Resilience to transient faults.

  If the failed zone hosts affected data-plane compute components, ingestion, report generation, and report queries can pause until the zone recovers.

  If a storage account becomes unavailable, for example because it uses locally redundant storage (LRS), Storage Discovery can't collect new information from it until the account recovers. A storage account that uses ZRS remains available during a zone failure, but Storage Discovery might not ingest new information from it while data-plane processing is paused.

• Expected data loss: Storage Discovery isn't expected to lose previously generated reports because of an availability zone failure.

• Expected downtime:

  • Control plane: The Storage Discovery workspace control plane remains available, but individual management operations might experience transient interruptions.

  • Data plane: If the failed zone hosts affected data-plane compute components, ingestion, report generation, and report queries can pause until the zone recovers.

• Redistribution: Microsoft manages request redistribution for the Storage Discovery control plane. Storage Discovery doesn't redistribute affected data-plane processing to another zone. The storage account's redundancy configuration determines how Azure Storage responds to the zone failure.

Zone recovery

Microsoft manages zone recovery for the Storage Discovery workspace. When the affected zone recovers, Storage Discovery resumes data ingestion, report generation, and report queries.

If the affected zone is permanently unavailable, Microsoft can recover the stored report data without data loss. Recovery from permanent zone loss might take longer than recovery from a temporary outage, and data-plane operations can remain paused during the recovery process.

When an affected storage account recovers, Storage Discovery resumes aggregating its insights. Storage Discovery might not backfill metrics that it couldn't collect while the account was unavailable, especially for LRS accounts, so reports and aggregations might contain gaps for the outage period. Report updates can take up to 24 hours to appear.

Test for zone failures

You can't simulate an availability zone failure for the Storage Discovery service. Test the redundancy configuration and recovery procedures for the storage accounts that your workspace analyzes separately.

Resilience to region-wide failures

A Storage Discovery workspace is a single-region resource. Storage Discovery doesn't replicate the workspace or its retained insights to another region, and it doesn't automatically fail over the workspace. The effect of a region-wide failure depends on whether the failure affects the workspace or a storage account that it analyzes:

• Workspace region failure: If the workspace region is unavailable, you can't manage the workspace, retrieve its reports, or aggregate new insights until the region recovers. These effects apply even when the storage accounts are in healthy regions and remain available.

• Storage account region failure: If the workspace region remains healthy, you can continue to manage the workspace and retrieve its reports. Storage Discovery continues to aggregate insights from storage accounts in healthy regions, but it can't collect new information from a storage account that's unavailable. Existing insights for the unavailable account remain accessible.

  When the storage account becomes available again, Storage Discovery resumes insight aggregation. If data is permanently lost from the storage account, Storage Discovery can't generate or backfill insights that depend on the lost data.

If the same region also hosts storage accounts that the workspace analyzes, the region failure might make both the workspace and those storage accounts unavailable until the region recovers.

Custom multiregion solutions for resiliency

If you need access to Storage Discovery reports during a region-wide outage, create a second workspace in another supported region. Configure both workspaces to analyze the same storage resources. The workspaces operate independently, and each workspace performs its own analysis and retains its own insights. Each workspace is billed separately.

This approach isn't a built-in disaster recovery feature. It protects report access from an outage in either workspace region, but it doesn't replicate reports between the workspaces. It also doesn't protect insight aggregation when a region failure makes an analyzed storage account unavailable, because neither workspace can collect information from that account.

Configure the storage accounts separately to meet their regional resiliency requirements. For more information, see Reliability in Azure Blob Storage.

Backup and restore

Storage Discovery doesn't provide backup or restore operations for workspace configuration, insights, or generated reports. You can download the list of storage resources behind an individual report chart to a CSV file, but you can't use that file to restore the workspace or its insights. For more information about downloading a list of resources, see What is Azure Storage Discovery?.

When you delete a scope, you immediately lose the insights gathered for that scope. When you delete a workspace, you immediately lose the insights gathered for all its scopes. You can't recover deleted historical insights.

If you move a workspace to a different Microsoft Entra tenant, you permanently lose its insight history, even if you later move the workspace back to the original tenant. For more information, see Frequently asked questions for Azure Storage Discovery.

Define your workspace, workspace roots, scopes, and pricing plan by using infrastructure as code, and store the definitions in source control. You can use those definitions to redeploy the workspace configuration, but redeployment doesn't restore deleted insights.

If you downgrade a workspace from the Standard plan to the Free plan, Storage Discovery permanently deletes insights older than 15 days. You can't recover those insights by changing the workspace back to the Standard plan. For more information, see Frequently asked questions for Azure Storage Discovery.

Resilience to service maintenance

Microsoft regularly applies service updates and performs other maintenance. The Azure platform handles these activities automatically, ensuring that maintenance is seamless and transparent to you. No downtime is expected during maintenance events unless you've been advised through Azure Service Health planned maintenance.

Service-level agreement

Azure Storage Discovery doesn't offer an availability service-level agreement (SLA).

Related content

• Plan your Storage Discovery deployment
• Azure Storage Discovery pricing
• Frequently asked questions for Azure Storage Discovery
• Reliability in Azure Blob Storage