AI Data Residency: Why Choosing a Region Isn't Enough

I’ve been working with Microsoft Foundry, and while thinking about making it available to more people, I got stuck on a question: where does the data we send it actually go?

With a couple of test prompts and some public documents, I can afford a few shortcuts. Once I start thinking about a client’s technical documentation, files retrieved by an agent, and conversation history, those shortcuts become harder to justify.

The easy answer would be to open the portal and check the region. The resource is in Europe. Done.

Except the resource location and the place where inference runs aren’t necessarily the same. Microsoft makes that distinction in its deployment documentation.¹ And the model is only one of the services handling our data.

“The data must stay in Europe”

That sounds like a clear requirement. Try turning it into a configuration.

Do we mean geographic Europe, the European Union, or the European Economic Area? Are we talking about stored files, processing, or both? Does the requirement cover support access? Is it an internal policy or something we’ve promised a client?

I’d settle those questions before picking a model. First, separate storage location from processing boundaries. Then bring the questions about access and contractual terms to the people responsible for privacy and security. A location on a map won’t answer all of them.

One GDPR shortcut is worth avoiding here: the regulation doesn’t impose a blanket ban on transfers of personal data outside the EEA. Those transfers are allowed under specified conditions, alongside the other obligations that apply to the processing.³ Keeping everything in the EEA doesn’t make a system compliant by itself.

A technical drawing might not contain personal data at all and still be covered by confidentiality obligations. “Can we use Azure?” is too broad a question. I’d want to know which data we can process, for what purpose, and within which limits.

The document doesn’t just sit in storage

Imagine an assistant that compares a technical specification with project documents.

In a possible RAG pipeline, documents are ingested, split into chunks, and indexed. When someone asks a question, the application retrieves relevant passages and adds them to the prompt. The model receives that text too, not just the user’s question.⁴

Checking where the original file is stored already looks insufficient.

If the pipeline uses an embedding service, I’d check where vectorization happens. If PDFs first go through a text-extraction service, that’s another processing step. Azure AI Search can integrate these steps into content preparation; the dependencies to review are the ones the actual pipeline uses.⁴

Figure 1. Foundry resources, projects, and connected services. These are governance boundaries, not a map of where inference runs. Source: Microsoft Learn.⁵

Foundry distinguishes the top-level resource, projects, and connected resources. Storage, Key Vault, and Azure AI Search have their own governance boundaries. Access and networking need to be configured on those services as well.⁵

I’d add the easy-to-forget copies to the diagram: extracted text, intermediate files, conversations, and application traces. When I delete the original document, what happens to the rest? I’d rather find out during design than during the first deletion request.

Global, Data Zone, and regional deployments

For this section, I’m referring to Models sold by Azure, which include Azure OpenAI. I wouldn’t extend the same terms to every model in the catalog. With partner models, we also need to check who provides the model and which terms apply.⁶¹⁵

This is the distinction I’d keep in front of me while choosing a deployment.¹

Deployment family	Where processing can take place
Global	Across geographies where the model is available, not just the resource’s geography.
Data Zone	Within Microsoft’s defined zone, which can include multiple regions and geographies.
Standard and Regional Provisioned	Within the chosen Azure geography; the detailed documentation allows processing across regions in that geography for operational reasons.

A Global deployment doesn’t automatically mean persistent data is stored everywhere. The privacy documentation distinguishes data at rest, stored in the customer-designated geography, from the location of processing.²

But if the requirement is “inference only in Italy,” seeing italynorth in the location field isn’t enough. I’d need to verify the model, deployment type, and guarantee applicable to that combination.

Microsoft’s pages need to be read together

The sources I checked on October 3, 2026 don’t use entirely consistent wording. The deployment page connects the EU Data Zone to the Azure EU Data Boundary and mentions possible inclusion of EFTA countries such as Norway and Switzerland.¹ The privacy page describes European Data Zone processing in terms of EU member states.²

There’s a similar issue with region and geography. The deployment overview still uses “Standard (single region),” while the detailed pages describe a geography boundary.¹⁵¹²

I wouldn’t pick whichever sentence makes the design easier to approve. If a contract distinguishes the EU from other European countries, I’d ask for written clarification covering the service we’re buying and keep it with the architecture decision. An optimistic interpretation can turn into a promise to a client surprisingly quickly.

“Your data isn’t used for training” answers a different question

For Models sold by Azure, Microsoft states that prompts and responses aren’t used to train foundation models without the customer’s permission or instruction. The same documentation describes features that persist data, including uploaded files and conversation histories associated with stateful entities.²

So content can be excluded from training and still remain in the service. Those are separate checks.

I’d also include abuse monitoring. The documented process includes automated analysis and, under certain conditions, human review of flagged content. Eligible customers can apply for modified abuse monitoring, but approval is required. I wouldn’t assume it’s enabled by default, or that it removes every automated check.⁷

For confidential documents, I’d keep training use, retention, and access as separate questions. “The provider doesn’t learn from our data” doesn’t cover the other two.

Standard agent setup doesn’t mean a self-hosted model

The naming doesn’t help here.

Agent Service’s Basic setup uses platform storage for agent state. Standard setup lets you use your own Azure resources, including a private-networking variant.⁸ In that setup, Azure Storage holds files, Azure AI Search holds vector stores, and Cosmos DB holds messages, conversation history, and agent metadata.⁹

Figure 2. Resources used to persist agent data in Standard setup. This does not mean the model runs on customer-managed GPUs. Source: Microsoft Learn.⁸

Having those resources in your subscription gives you more control over persistence. The model still has its own deployment.

Standard agent setup and a Standard model deployment are separate choices. For example, the environment setup guide lists GlobalStandard as the template’s default modelSkuName.⁸ You can have customer-owned resources for agent state and Global inference in the same project.

For company data with specific retention and access requirements, I’d consider using our own resources. I want to know which controls we can apply directly and which remain the platform’s responsibility. But that also means owning permissions, capacity, and the data lifecycle. Adding resources to a resource group doesn’t magically assign someone to maintain them.

A Private Endpoint doesn’t change the deployment

A Private Endpoint concerns the access path to a service. The geographic scope of inference depends on the selected deployment. These are different controls.¹⁸

With agents, I’d pay just as much attention to outbound traffic. The documentation separates private access to the Foundry resource from isolation of communications with connected services and resources.⁸¹⁰ The fact that the initial request reaches a private address isn’t where I’d stop checking.

An agent should reach only the destinations it needs, using dedicated identities and the minimum permissions required. A private connection to a database isn’t a reason to give it access to every table.

I’d test the requests that should fail, too: an unauthorized destination, a public endpoint that should be blocked, a DNS failure. A working demo tells me the intended path exists. These tests help me check whether I’ve accidentally left other paths open.

Where I’d put API Management

When several applications share access to models, I’d consider a gateway. Azure API Management offers authentication, routing, token quotas, and observability, with availability and details depending on the feature and tier.¹¹

Figure 3. AI gateway capabilities. Actual coverage depends on which flows pass through APIM. Source: Microsoft Learn.¹¹

I’d use it to give teams a set of approved backends and consistent rules. But I’d still trace the real calls. Putting APIM in front of an agent API doesn’t, by itself, prove that every internal inference call goes through the gateway. That depends on the runtime and integration.

The case I’d look at most closely is fallback. The approved backend hits its quota. A rule forwards the request to a second deployment. That deployment is Global.

The user gets an answer. We may have just sent their data outside the agreed boundary.

With a strict location requirement, I’d rather queue or reject the request than send it to an unapproved destination. That’s a decision to agree on beforehand. It’s harder to make while users are waiting and someone is asking us to “just get it working.”

I’d also be careful about what token limits promise. The llm-token-limit policy supports consumption limits, but concurrent requests can temporarily exceed them, and counters aren’t aggregated across all gateways in an instance.¹² Useful control? Yes. A guaranteed spending cap down to the last cent? No.

Logs, replicas, and restores

After checking the model and storage, I’d open the logging configuration.

APIM can record token consumption and, when enabled, prompts and responses in Azure Monitor.¹³ If I log full request bodies, I’ve created another content store. I need to know who can read it and how long the data stays there.

For routine operations, I’d start with request IDs, deployment names, token usage, latency, and errors. I’d collect full content only for a specific need, with defined access and retention. Debug logging left on because “it might be useful” is an easy thing to forget.

Then there are replicas. In Azure Storage, LRS and ZRS keep replication within the primary region; GRS and GZRS add a secondary region.¹⁴ I wouldn’t choose weaker redundancy just to have fewer locations on a diagram. I’d look for a configuration that meets both the geographic requirement and the recovery requirement.

The same review applies to backups, exports, and restores for other components. Restoring a backup to an unapproved destination is still a problem, even if the main application never left the intended region.

What if the managed service isn’t enough?

I’d consider a model that runs on infrastructure administered by the company, whether in the cloud or on-premises. But the comparison isn’t just GPU prices versus token prices. Someone has to size capacity, update the runtime, secure the endpoint, and keep the service running.

I wouldn’t confuse that option with Foundry’s managed compute. In the documentation I checked, it’s a preview offering where Microsoft manages GPU capacity and the runtime, and the page currently lists Global deployments.¹⁵ I wouldn’t treat it as a shortcut to single-region inference.

Splitting workloads may make sense: tightly restricted content on one platform, other data on services with a broader processing boundary. That split should follow data classification and application controls, not the hope that every user will pick the right endpoint.

Even with a self-hosted model, I’d check telemetry and external tools. One tool sending documents to a third-party service is enough to bring the same questions back.

The full bill

Foundry’s cost guidance includes connected services alongside the model.¹⁶ For this scenario, I’d budget for the gateway, Search, Cosmos DB, Storage, networking, and observability where needed. For each, I’d check supported tiers, minimum capacity, and whether resources can be shared.

I’d include the time spent managing it, too. Running our own platform might give us the control we need while still consuming staff time and capacity on days with few requests. The comparison only makes sense for the same workload and requirements. A universal price for “private Foundry” wouldn’t help me decide.

Before giving other teams access

I’d take a representative request using synthetic data and follow it all the way through:

• Which identity sends it, and which documents can it retrieve?
• Which deployment receives the context, including any embedding calls?
• What remains stored after the response, in services and logs?
• Where does the request go if the primary backend is unavailable?
• What happens to copies when the original document is deleted or restored?

I’d keep those answers next to the configurations and sources that support them. Where we control the behavior, I’d add a test. Where we rely on a provider commitment, I’d keep the reference and the date it was checked. An endpoint in a log doesn’t, on its own, prove which physical datacenter processed the request.

From there, I’d decide which changes need a fresh review. Adding a fallback model can change how data is processed even when the chat interface looks exactly the same.

Before opening access to more teams, I want to be able to explain where their documents go when something breaks, too. The resource’s region is a useful place to start. By then, I still have quite a few boxes left to check.

---

Sources checked on October 3, 2026. Availability and guarantees can change; production decisions require checking the applicable terms and documentation for the service being purchased. This article discusses design considerations, not a specific company’s implementation, and is not legal advice.

Figures 1–3: Microsoft / Microsoft Learn, reproduced without modification. Original sources are linked in the captions. MicrosoftDocs repository materials are distributed under CC BY 4.0 unless otherwise noted.¹⁷¹⁸ Trademarks and logos belong to their respective owners. Their use does not imply Microsoft’s endorsement of this article.

Sources

• 1 Microsoft Learn: Deployment types for Microsoft Foundry Models.
• 2 Microsoft Learn: Data, privacy, and security for Models sold by Azure.
• 3 EDPB: International data transfers.
• 4 Microsoft Learn: RAG in Azure AI Search.
• 5 Microsoft Learn: Microsoft Foundry architecture.
• 6 Microsoft Learn: Models from partners and community.
• 7 Microsoft Learn: Abuse monitoring.
• 8 Microsoft Learn: Set up an environment for Foundry Agent Service.
• 9 Microsoft Learn: Standard agent setup.
• 10 Microsoft Learn: Private networking for Foundry Agent Service.
• 11 Microsoft Learn: AI gateway capabilities in API Management.
• 12 Microsoft Learn: LLM token limit policy.
• 13 Microsoft Learn: LLM API logging.
• 14 Microsoft Learn: Azure Storage redundancy.
• 15 Microsoft Learn: Deployment overview for Microsoft Foundry Models.
• 16 Microsoft Learn: Plan and manage costs.
• 17 MicrosoftDocs azure-ai-docs license.
• 18 MicrosoftDocs azure-docs license.