Hello once again fellow geek. In this post I will be continuing my series of NSPs (Network Security Perimeters). Over the past few posts in this series I’ve explained the problem NSPs solve, the components that make up an NSP, demonstrated NSPs through a Key Vault use case and an AI workload use case, and even shown how they can used to troubleshoot the dreaded Private Endpoints and forward web proxy problem we all run into. After reading those posts you are probably wondering why you aren’t seeing them more talked about in the Azure world. Besides the fact it’s not called “Frontier Network Agentic Security AI Perimeter”, the reason NSPs haven’t been getting the love they should is due to a technical gap that is finally in the process of being lifted.
The only way to work around this limitation is to place the delivery destination (LAW, storage account, event hub) in the same NSP. While this sounds like a simple solution, a given resource can only belong to a single NSP. If you attempt to add a resource to an NSP when it’s already associated to another NSP it will fail as seen below.
Resources can only be associated to a single NSP
If you’ve ever done logging as scale in Azure, you already see the problem with this limitation. When logging at scale in Azure, it’s very common to send logs to a centralized Log Analytics Workspace, Event Hub, or Storage Account. There are a number of benefits to doing this beyond the scope of this post which you can read about in the CAF (Cloud Adoption Framework) documentation. These resource are typically managed by a security team or a platform team and live in subscriptions controlled by those teams.
If you’re like, your first instinct is to create an outbound rule to allow the access to the Log Analytics Workspace. Good instinct, but not possible because today NSPs only support outbound rules of type FQDN and there is no FQDN for platform diagnostic logs that Microsoft provide today. A service tag would be the best bet, but sadly it is not yet supported for NSP outbound rules.
Centralized logging problem with NSPs
Up until very recently, NSPs in enforcement mode have been pretty much a non-starter for enterprise customers due to this limitation. Notice I say “enforcement”. If you aren’t using NSPs in learning (or transition mode) you are losing out on insanely valuable egress visibility you can’t get using any other means as I describe in my troubleshooting post.
The PG (Product Group) for NSPs is one of those amazing product groups who listens to the customer, wants to see their product adopted, and works their butt off. Just a few weeks ago they introduced a new feature into public preview that begins to address this technical limitation
The Solution
The solution for this technical problem is perimeter links. In the ARM (Azure Resource Manager) API, these are referred to as links and are a child resource of the Network Security Perimeter. Perimeter links allow traffic to flow between NSPs either per profile or for all profiles. This means we can slap a perimeter link between the NSP containing our resources that need to deliver our logs and the NSP containing the resource we’re delivering the logs to!
Perimeter links can be across a single profile or all profiles within an NSP
In the image below I’ve created a perimeter link between an NSP containing a storage account the NSP containing the log analytics workspace the storage account diagnostic log setting has been configured to deliver logs to. This link enables the traffic to flow freely between the two NSPs.
Perimeter link example
Once the perimeter link is created, new inbound and outbound access rules are created in each profile allowing the two NSPs to communicate with each other.
In the example above, once the perimeter link is in place and rules have been automatically created and taken affect (takes about 10-15 seconds) the spice will flow! Alright, logs will flow but I’ve been re-reading Dune so leave me alone in my nerdy world. In the screenshot below you can see the logs being blocked initially and then being allowed after the perimeter and its automatically created rules take affect.
The spice will flow!
The good and gotchas of perimeter links
So first some of the good things about perimeter links:
You can now do cross perimeter communication.
In my testing, terraform state doesn’t get botched up when the perimeter link triggers the automatic creation of the access rule. A reapply after didn’t complain and didn’t overwrite them.
Perimeter links work across subscriptions. Note there is a (as there should be) to make that work.
The biggest pain point of them right now is the maximum amount of perimeter links per NSP is 10. Obviously, this is no where near the scale needed for the centralized logging use case. However, this is still preview and I’m confident the PG will increase that limit (hopefully a shit ton) prior to GA.
Well folks, let me leave you with a few recommendations:
If you’re not using NSPs as a detective control for data exfiltration you are missing out. You need to make this a priority, if only for the visibility into egress.
This is preview, so muck around with it, but don’t go rolling this to prod.
Welcome back folks. Today I’ll be continuing my deep dive series into Entra. In my last post I went over the basics of Entra ID covering what it is at a high level and the important resources you’ll want to understand to grasp how human and non-humans identities are treated within the service. One of the features of Entra ID that I highlighted in that post was that it provides authentication services. It is capable of providing authentication of a human or non-human through older protocols like Kerberos (don’t get me started on this feature or else I’ll spend the whole post ranting) and LDAP (through Entra ID Domain Services, another service I’m no fan of), but also more modern protocols such as SAML and OIDC (OpenID Connect). Before I dive into Microsoft’s implementation of OIDC and the protocol it’s built on top of, OAuth, I figured it was a good time to do a light protocol primer (primarily for my own benefit because I can only re-read the RFC so many times before it stops being fun. Yeah I find it fun to read a good RFC, so what?).
What is OAuth?
You are probably thinking, “Why the hell are we talking about OAuth?” We need to talk about OAuth (Open Authorization) because OIDC is built on top of the OAuth protocol. If you have a basic understanding of OAuth, then OIDC makes a lot more sense. I’m not going to try to make you an expert, because to make you an expert I’d need to expert which I am very far from. Instead, I’m going to give you the basics. If you want a better/smarter explanation, start with the RFC(s) and then take a read through Vittorio Bertocci’s (an industry legend) many articles, ebook, and videos online.
There are a lot of misconceptions out there where folks will talk about OAuth authentication, which is not a real thing. OAuth itself exists as an authorization protocol to provide a framework (lots of SHOULDs/COULDs and not a ton of MUSTs in that RFC) for how applications can get limited access to a user’s data based around the user’s consent using delegation.
The protocol refers to this limited access as a scope. The assignment of a specific scope to an application gives you the ability to do delegation vs impersonation. In impersonation, the application will typically act as you with your full permission set vs with delegation you grant consent for the application to access a subset of your data with a more restricted set of permissions. A good example would be delegating the application the read permission over your email vs the reading, writing new emails, and deleting emails impersonation might give the application.
When it comes to the whole process of a user delegating a scope of access to an application, a number of different roles are involved. These roles include:
Client
Resource Owner
Authorization Server
Resource Server
Client
The client is an application that needs to access some data. Within the protocol it’s important to divide applications into a few different buckets, because the protocol supports them in different ways as I’ll cover in a bit. The standard breaks them into three buckets: web applications, browser-based applications, and native applications. I’m going to keep it simple and consolidate those three buckets into two which will be web applications and non-web applications.
Clients can be either public clients (non-web apps) or confidential clients (web apps). Confidential clients have some type of credential they use to authenticate themselves to the authorization server while public clients do not have a credential (because their code runs on the user’s machine so there is no way to secure the credential). Clients must register with the authorization server either through dynamic registration (which Entra ID does not support today) or through some other type of process. Registration, at a minimum, will include providing the authorization server with a redirectUri, which grant types it will use, and whether it’s a public or confidential client. The client is then issued a unique identifier called a client_id and optionally a client_secret if a confidential client. We’ll see an example of how Entra does it later on this series. Examples of clients could be applications you develop, third-party applications you integrate with, or Microsoft-native applications like Microsoft Teams.
Resource Owner
The resource owner is the user or organization that owns the data the client wants to access and is the entity that is capable of granting access to that data through a consent process. Consent is a major focus in OAuth since it relies on delegation of a specific scope of access to the data. Consent is the process of the resource owner approving that delegation. Resource owners in the Entra ID world are going to be the enterprise at the top layer, business units underneath that layer, and finally its employees which are represented by user objects in Entra. Consent will either be granted for all users within the tenant by an administrator or by individual users to data they have permissions over. In Entra, which consent is required is defined by the type property in the application resource’s permissionScope.
Authorization Server
The authorization server is the role that glues all other roles together. This is the server authenticates the user (OAuth doesn’t care how), gets the user’s consent for the client to access the data, and issues an access token to the client. Entra ID fulfills this role in the Microsoft cloud world.
Resource Server
The resource server hosts the resource owner’s data. It will consume the access token obtained by the client from the authorization server and allow or deny access to the data. Resource servers in the Microsoft world could be your custom built application or the Microsoft Graph API.
The RFC has a basic diagram which does a good job explaining the flow at a high level.
High level OAuth flow
You’ll notice the the term authorization grant in the above image. An authorization grant represents the resource owner’s authorization (delegation) of a specific scope of access to their data and is used by the client to get an access token which the resource server consumes and approves/denies access. In the base specification for OAuth 2.1, there are three types of grants (there are a ton of extension grants, some of which we’ll cover in this series) which include the authorization code grant, the refresh token grant, and the client credentials grant.
Before I describe the grant types, it’s worth calling out that I’m going to be talking specifically about OAuth 2.1 (which is still a draft RFC right now). OAuth 2.1 seeks to address a lot of the security issues with OAuth 2.0. In OAuth 2.0 there were a bunch more grant types including resource owner credentials flow and the implicit flow there are somewhat of security nightmares. OAuth 2.1 removes those grant types and the official spec sticks to the three I described above while adding an additional requirement for PKCE (Proof-Key for Code Exchange) for both public AND confidential clients. PKCE helps to address authorization code interception attacks. This Okta article does a great job describing the security benefit brings. I’ll demonstrate this with MSAL in a later post. Now back to the authorization grant types.
The authorization code grant type involves sending the resource owner to the authorization server to authenticate and consent to the client’s access of their data, returning an authorization code to the client, and the client exchanging that with the authorization server for an access token. This is going to be your go to grant type any delegation use case. An example of this would be an application accessing my data in a storage account that belongs to me.
Authorization Code Flow
Next up is the client credentials flow grant type. In this flow there is no user consent because the data the client is trying to access is under its control. Essentially, the client uses its own identity context to access the data because it’s already been authorized to do so. An example here would be an application pulling Entra ID sign-in logs from the MS Graph API.
Client Credentials Flow
Lastly, we have the refresh token grant. This grant type is used by the client to exchange a refresh token for a fresh access token. Access tokens must be short lived (typically around an hour). Instead of having the resource owner go through the whole authentication and consent process again, the client can exchange its longer living refresh token (if it requested one) for a new access token of the same or lesser scope.
In addition to grant types above there are extension grant types. The one that will be relevant to this series is the jwt bearer type, or more formally the JSON Web Token (JWT) profile. In the Microsoft world, you’ll see this referred to as the on-behalf-of flow. This is the flow that Microsoft will use for any multi-hop OAuth. There is also another newer grant type to be aware of which is the token exchange flow. This has a similar use case as the jwt bearer flow for multi-hop OAuth but isn’t limited to JWTs and provides additional information in the access token which can be very helpful in identifying client (actor) vs the resource owner (subject) in the access token. Entra doesn’t support this flow to my understanding, so you’ll be using jwt-bearer instead for multi-hop flows as we’ll see in a future post.
In an authorization request will look something like the below:
GET /authorize?response_type=code&client_id=s6BhdRkqt3&state=xyz
In the above example we see the client is requesting the authorization code grant type, is specifying its client id and its redirect_uri (which were established during client registration), a code challenge (for PKCE), and the scope of access it is requesting.
Access tokens come in a few flavors which you can read about in the RFC. The most common type of token is a bearer token within Entra’s implementation. The bearer token is exactly what it sounds like, whoever bears the token holds the power! Bearer tokens are typically JWTs (JSON Web Tokens). While RFC doesn’t specifically require the access token to be cryptographically signed, the ones that Entra ID issues are. The public key used to verify the signature can be obtained for Entra ID from a public metadata endpoint we’ll see later.
There are a few important endpoints the client needs to know about for the authorization server. This includes the authorization endpoint (where the resource owner is sent to authenticate and consent) and the token endpoint (where the client obtains an access token). These can be retrieved via a metadata endpoint. This is how Entra ID does it as we’ll see in a future post.
Ok, with that you should now have a high level understanding of OAuth and be aware of its role as an authorization protocol. Key in on that word, authorization. When I perform an OAuth flow I get an access token back to my app that I can use to access a resource owner’s data, but I would still need to authenticate the user to my application and get some basic profile information via another means. In comes OIDC.
What is OpenID Connect?
Like the prior section, my goal is give you a primer. If you want the gory details, take a read through the specification (another tolerable if not enjoyable read). Microsoft and Auth0 have solid one pagers if reading specifications isn’t your style.
The OIDC protocol is built on top of the OAuth (Open Authorization) protocol to provide an identity layer and authentication layer. It gives us the means get some assurance that the user is who they say they are and get some basic information about the user.
Within the OpenID protocol there are three roles that exist. These include:
End User
RP (Relying Party)
OP (OpenID Provider)
Since the protocol is built on top of the OAuth protocol, these roles will map nicely to the OAuth roles as we’ll see.
We first have the end user. The end user is the human participant that will be access our application. They are very often also the resource owner for any data we may want to access about them as we’ll see later.
Next, we have the RP. The RP is the application that is requesting end user authentication and claims (or data/attributes) about the user. This will be an OAuth client application.
Finally we have the OP. The OP is the server capable of authenticating the user and providing claims about the user’s identity. This role is fulfilled by the OAuth authorization server in all instances (I don’t believe their are exceptions, but feel free to correct me) .
The OP will issue a security token referred to as an id token with claims about the authentication of the end user, including claims about the user.
This is another instance where the specification did a great job with a high level flow diagram.
High-level OIDC Flow
The structure of the authentication request is almost identical to the structure of an authorization request in OAuth.
GET /authorize?response_type=code&client_id=s6BhdRkqt3
Notice above that we have an added state and nonce. The state helps to provide CSRF (cross-site request forgery) attacks, such as fooling the victim into accessing an attacker’s account to get them to upload data or perhaps purchase things for the attacker’s account. The nonce helps to mitigate id token replay attacks (app validates the nonce in the id token matches what it expects for the user’s session). Now the major things to pay attention to is the additional scopes. Here we see the openid and profile scopes. The openid scope tells the OP the client is looking for an id token. The profile scope is an optional scope which tells the OP to include additional claims in the id token such as the user’s name, preferred_username, and the like.
Once the RP (client / application) exchanges its authorization code (think authorization code grant) to the OP/authorization server, it returns back the an OIDC id token in addition to the OAuth access token. The application can then use the claims in the id token to identify the user, get information into how the user authenticated, get group information, and anything else you can stuff into the claims. It gives the application context about the user.
Below is an example id token’s payload. ID tokens follow the JWT standard and are cryptographically signed by a private key held by the OP. Clients will need to verify the signature using the OPs public key which is usually published in the OIDC discovery endpoint which looks something like this https://{issuer}/.well-known/openid-configuration. We’ll see an example when we break down Entra’s implementation.
At this point you should have a reasonably decent high level understanding of OAuth/OIDC. If you are already an “expert” you likely snorted milk through you nose reading my shitty explanation. What I mainly want you to take away from this is that OAuth is an authorization protocol with OIDC providing an authentication layer nicely on top. I like to think of OAuth as the cake with OIDC as the frosting. That top layer doesn’t work without the bottom layer (you freaks that eat the frosting right out of the can shall remain silent) and the bottom layer is enriched by the frosting on top. It’s 7PM and I’m craving a sweet, lay off.
In my next post I’ll walk through building out the required components in Entra ID for the frontend application. You’ll read and recognize properties that translate directly back to these two protocols. Other properties may not have the same name, but you’ll understand why they exist.
Yeah… You read that right. I’m finally biting the bullet and doing a series on Entra ID. There are some cobwebs there, but once upon a time I was a half-baked “identity guy”. Nah, I’m not returning to that place, but I have had to visit it more frequently over the past few months. With the growing use of agents, identity has one again become of those things that apparently everyone is a so called “expert in”. I aint no expert, but I have been digging into the chatter around the implications of agents into the identity world. Given my employer, that has been spending some time in the Entra ID Agent Identity space.
Digging into that demanded I go back to some of the basics of Entra ID such as the purpose of an application registration versus a service principal and the request flow when authenticating a user to an application using OIDC (OpenID Connect) or executing an on-behalf-of flow in OAuth. I had conceptual knowledge of the above, but it was too ivory tower. I needed to eat the dog food and do it. After spending a few weeks reading through the documentation, reviewing captured requests and responses, reviewing RFCs, and poorly coding some applications to exercise on the concepts I figured it was time to brain dump what I learned before I get distracted with some new shiny widget and forget 60% of it.
So yeah… that’s where this series is coming from. Hopefully some folks out there draw some value out of it beyond serving as a refresher for my neural pathways.
With that rambling out of the way, let’s get to it.
WTF is Entra ID?
Many years ago when Microsoft first introduced Azure Active Directory my peers and I scratched our heads with this question. Given the name, the initial assumption was it was the Windows Active Directory “killer” (stop trying to make that happen, it aint gonna happen). That seems to have been a pretty common assumption given what I’ve heard from customers over the years as I sat in the vendor space and is likely why the name was shifted a few years back to Entra ID. Either that or marketing needed to justify their existence with another product rename.
If you want the professional explanation as to what Entra ID is you can read the public documentation and bask in the marketing mumbo jumbo. Since you’re here, you’re going to get the less fancy and quick and dirty Matt Felton explanation. My take is that Entra ID does a lot of shit, but at its core it is:
An identity store for the identities of human and non-human security principals, their attributes, and their credentials.
An authentication service which can act as an OAuth Authorization Server, OIDC identity provider, SAML IdP / SP, and even Kerberos / LDAP provider (gross)
A data store for resources that represent applications that can be used for OAuth and OIDC
It provides these core services to Microsoft’s cloud services such as M365, Azure, Dynamics 365, and whatever other clouds Microsoft has that I’m missing. It can be extended to provide these services to other applications by integrating with it via one of the supported protocols like we’ll see in further posts in this series.
Entra ID is divided into separate tenants which represent unique identity boundaries. Services for a given customer (such as an Azure subscription) are associated to an Entra ID tenant which acts as the identity boundary for those resources. Tenants can trust other tenants to facilitate cross tenant accessing of resources through features like Entra ID External ID.
The identity and data store pieces of the Entra ID service are where users, groups, many types of service principals, applications, and device resources are stored. For us old people, similar to an LDAP, each of these resources is similar to an object in LDAP that has a schema with specific properties that are consumed by native Microsoft and third-party services.
That should be enough context to give you a very general idea of what Entra ID is. Like I mentioned above, it does a lot of shit (conditional access, privileged identity management, etc) that isn’t relevant to the point of this series and that I’m not going to discuss. If you can hammer it in your head those two core functions, it will be enough to get you through this series.
Human Resources in Entra
Within Entra ID human accounts will be represented by user resources. User resources have a selection of properties you’ll be familiar with if you’ve ever worked with an LDAP or Windows Active Directory. These include properties like givenName, surname, mail, and userPrincipalName. These user resources can be created directly in Entra to create “cloud users”, created through the Entra ID External ID, or created by a synchronization process like Entra Connect Sync or a third-party solution like Okta. Humans will use these user accounts (resources) to authenticate to Entra ID to access integrated services. A typical user account looks like the below:
Users can be organized into group resources. Like Windows Active Directory, these groups can be used to grant access to resource or used as distribution groups for email. Entra has multiple types of groups including Microsoft 365 Groups (Unified, your mail-enabled security groups for the modern age), Security Groups (for providing access to Entra-integrated resources), and mail-enabled security groups and distribution groups synchronized from Windows Active Directory. You’ll likely use a mix and there are benefits and considerations of both paths. A typical Microsoft 365 group looks like the below:
That’s all I’m going to say about humans, since the primary purpose of this series is the non-human side of the fence.
Non-Human Resources
For non-human identities our main resources are applications and service principals. There are others such as device identities, but I won’t be getting into that.
First, there is the application resource. You’ll see this referred to as the application registration or app registration in Microsoft documentation and Microsoft GUI-based experiences like the Azure Portal. The application resource (or app registration) is the globally unique representation of an application across all of Entra ID and exists only within the Entra ID tenant it was registered in. It’s easiest to think of an application resource as an OAuth client registration, because OAuth and OIDC are likely why this type of resource exists anyway (I’ll cover this a bit more in some upcoming posts). An application resource can have a credential which is used to authenticate it to Entra ID (OAuth confidential client) and this credential can be a secret, certificate (very cool and under used), or a federated credential (basis of workload identity). This resource will include properties required for OAuth flows such as the client id (appId), the audience (identifierUris), any scopes it exposes (oauth2PermissionScopes), and redirectUris (when using interactive OAuth flows). Below is an example of an application resource associated with an application that authenticates users with Entra ID.
The service principal resource is the other major non-human resource to be aware of. A service principal represents an instance of an application resource in an Entra ID tenant. While you’ll only ever have a single application resource in the registered Entra ID tenant to represent an application, that application might have multiple service principals in multiple Entra ID tenants if it was built as a multi-tenant application. When permissions are granted to an application to access a resource secured with Entra ID, it will use the permissions associated to the service principal. The Microsoft GUI-experience will call a service principal an Enterprise Application.
As you’ll see below, a service principal looks a lot like the application resource its providing an identity for. The appId property is the property used to directly map it back to the application resource. The service principal type in this scenario is Application. There are a variety of service principal types that provide a non-human identity for different use cases like for managed identities, agent identity blueprints, and agent identities.
{
"id": "44444444-4444-4444-4444-444444444444",
"deletedDateTime": null,
"accountEnabled": true,
"alternativeNames": [],
"appDisplayName": "Demo Entra ID application - Frontend",
"appDescription": "This application is the frontend for my demo Entra ID application",
"displayName": "Demo Entra ID application - Frontend",
"homepage": null,
"isDisabled": null,
"loginUrl": null,
"logoutUrl": null,
"notes": null,
"notificationEmailAddresses": [],
"preferredSingleSignOnMode": null,
"preferredTokenSigningKeyThumbprint": null,
"replyUrls": [
"http://localhost:8100/callback"
],
"servicePrincipalNames": [
"611111111-1111-1111-1111-111111111111"
],
"servicePrincipalType": "Application",
"signInAudience": "AzureADMultipleOrgs",
"tags": [],
"tokenEncryptionKeyId": null,
"samlSingleSignOnSettings": null,
"addIns": [],
"appRoles": [],
"info": {
"logoUrl": null,
"marketingUrl": null,
"privacyStatementUrl": null,
"supportUrl": null,
"termsOfServiceUrl": null
},
"keyCredentials": [],
"oauth2PermissionScopes": [],
"passwordCredentials": [],
"resourceSpecificApplicationPermissions": [],
"verifiedPublisher": {
"displayName": null,
"verifiedPublisherId": null,
"addedDateTime": null
}
}
The official documentation likes to refer to the application object as the template for the application and the service principal as the security principal. I think that’s a pretty damn good single sentence explanation.
What are we going to build?
Now that you have the bare bones basics of Entra, you likely want to understand how an application would go about using it for authentication and authorization. While you might not be doing this now and it may not seem relevant, it will become very relevant to you if you begin building agents in Microsoft’s clouds through Microsoft Foundry, CoPilot Studio, or the 18 other random services Microsoft allows agents to be built. This will also be relevant to you if you’re going to consume Microsoft resources (such as Azure) from other clouds through an application or an agent. So yeah, you should understand what this looks like to do. The whole Entra ID Agent Identity feature builds on these foundational pieces.
To see these concepts in action I’m going to walk through a very simplistic solution with a frontend website and backend API in Python. The frontend website is built using the Flask Framework and the backend API is built with fastapi.
Demo application architecture
The frontend website will use Entra ID to authenticate the user and will be issued an OIDC id token to identify the user. The frontend will get some information from the user from the id token and access token it receives from Entra and will make additional calls to the Microsoft Graph API using the client credentials flow to grab other attributes of the user’s identity. I’ll also show how to include the user’s Entra ID group information in the id or access token and how to handle nested group membership.
The frontend will have some pages that call functions within the backend API. One example is the story endpoint will pull a pre-built AI generated story about the user which has been uploaded to blob storage in an Azure Storage Account. The backend API will use the on-behalf-of flow to access the storage account as the user to pull the user’s specific story.
This will demonstrate some of the most common flows including authentication, OAuth client credentials flow, and OAuth on-behalf-of (or jwt bearer). I’ll show you how the id tokens and access tokens look in different scenarios pointing out the relevant claims and how they’re used upstream. I’ll even share some process flows so you understand what does what in a given flow.
My primary goal here is give you the basics so you can walk away with a more solid understanding of what’s happening under the hood and how Entra ID has decided to implement OIDC and OAuth. I can’t stress how helpful this will be for you once you start diving into the agent identity space (which I’ll be covering after this series).
Summing it up
Ok, so you know what you’re in for. This series is gonna be relatively deep in the weeds so bring your favorite caffeinated beverage for future posts. I’m doing everything direct with the REST APIs because I want to show you the gory details. No pretty SDKs for you. If you’ve had a “conceptual” idea of how this works without the implementation specifics (like I did before I went down this rabbit hole) this series should help to fill those gaps.
In the next post I’ll walk through setting up Entra for the frontend website, authenticating a user, and exploring the id token and access token. The post following that will walk through using the application’s identity context to get more information about the user from the Microsoft Graph API such as nested group membership, then I’ll finish up the series by walking through the on-behalf-of flow with the backend API to show you how to carry the user’s identity context through the application to the destination resource down the line.
Hello again! Today I’m going to continue my series on Microsoft Foundry’s new support for the BYO AI Gateway. In my past few posts I’ve walked through the evolution of Foundry and covered at a high level what an AI Gateway is and the problem this feature solves. In this post we’re gonna get down and dirty with the technical details on setting this up within Microsoft Foundry. I’ll do a follow-up post to focus on the APIM (API Management) configuration. Grab your coffee and put on your thinking music (for me that is some Blink and Third Eye Blind. Yeah, I’m old.).
Let’s get to it!
Current State Architecture
My customer base is primarily in the regulated industry so most of my customers are still at the experimentation state with the Foundry Agent Service. Given these customers have strict security requirements they are largely using the agent service with the standard agent configuration. In this configuration the outbound traffic (subsets of it, but that is a much larger conversation) can be tunneled through the customer virtual network for centralized logging, mediation, and facilitating access to private resources (again, with limitations today) through what the product group calls VNet injection but I’d say is more closely described as VNet integration via a delegated subnet. Threads (conversations in v2 agents) and agent metadata are stored in a Cosmos DB, vector stores created by an agent from tools such as the File Search tool are stored in AI Search, and files uploaded to the Foundry resource by users are stored in a Storage Account. These resources are all provisioned by the customer into the customer subscription and fully managed by the customer (RBAC, encryption, HA settings, etc). Private Endpoints for each resource are created within the customer’s virtual network and made accessible from the agent delegated subnet. The whole environment looks similar to what you see below.
Foundry Agent Service – Standard Agent Configuration with VNet Injection
As I covered in my last post, as of the date of this post Foundry native agents can only consume models deployed to their own Foundry resource. This creates an issue for customers wanting the governance of the models, visibility into the use of the LLMs, and improvements security posture and operational optimizations an AI Gateway can provide when it sits between the agent and the model. For now, customers are working around doing this using what I refer to as external agents. External agents run outside of Microsoft Foundry on customer-managed compute like an on-premises Kubernetes cluster or an Azure Function deployed to the customer subscription. The downfall of this direction is these external agents live on compute customers have to manage and can’t access many of the tools available to Foundry-native agents. This is the problem the BYO AI Gateway feature is attempting to fix.
No BYO Gateway vs BYO Gateway
Foundry resource architecture
Here is where the new connection type introduced in Foundry comes to the rescue. Before I dive into the details of that, I think it’s helpful to level set a bit on the resource hierarchy within Foundry. At the top is the top-level Azure resource referred to as the Foundry service which under the hood is a Cognitive Services account. The relevant resources for this discussion are below the account resource and are projects, deployments, and connections. Projects serve a few purposes with two of them being logical boundaries around connections (at the management plane) and agents (at the data plane) provisioned under the projects. Deployments of models (such as GPT-5) are children of the account and are made available to all projects within the account. The account can also have connections objects which can be shared across projects.
Relevant resource hierarchy
For the purposes of this discussion, I’m going to focus on the connection objects. Connection objects can be created at the account level and project level as discussed above. In the standard agent configuration, you’ll create a number of different connections during setup including connections to Cosmos, AI Search, and Azure Storage. Additional common connections could be to an App Insights instance for tracing or a Grounding With Bing Search resource to use with the Grounding with Bing tool. Connection objects will contain some type of pointer, like a URI and a credential. That credential is usually API Key, some Entra ID-based authentication mechanism, or general OAuth.
Connections are created at the account level when the Foundry account itself needs to access them. This could be for the usage of Content Understanding, to a Key Vault for storing connection secrets (API keys) in a customer subscription, an an App Insights instance used for tracing. From what I’ve observed, you will create connections at the account level if they need to be shared across all projects OR they’re used by the Foundry resource in general vs some type of project construct. Connections used by projects can also be created at the project level. When you provision a standard agent for example, you’ll create connection objects to the Cosmos DB, Storage Account, and AI Search resources mentioned above. The new category of connections for this post will be created at the project level. I’d had mixed behavior with how effectively connection objects at the account can be used downstream by the projects.
APIM and Model Gateway Connections
The BYO AI Gateway feature uses two new types of connection categories: ApiManagement and ModelGateway. These objects are the glue that allow the Foundry native agents to route requests for models through an AI Gateway. When we’re connecting to an APIM instance, you should ideally use the ApiManagement category and when you’re connecting to a third-party category you’ll use the ModelGateway category.
As of the date of this blog post, these connection objects have the following schema (relevant properties to this discussion only):
name: The name of the connection (needs to be less than 60 characters in my testing)
properties: {
category: ApiManagement or ModelGateway
target: The URI you want the agent to connect to
authType: For ApiManagement this can be ApiKey or ProjectManagedIdentity
credentials: This will be populated with the value of the API key if using that authType
isSharedToAll: true or false if you want this shared across all projects
# ApiManagement category with static models
metadata: {
deploymentInPath: true or false
inferenceAPIVersion: API version used for inferencing (not used if using OpenAI v1 API)
I’ll walk through each of these properties in as much detail as I’ve been able to glean from them with my testing.
The category property is self-explanatory. You either set to this to ApiManagement (if using APIM) or Model Gateway (if using a third-party AI Gateway like a Kong or LiteLLM).
The target property is the URI you want the agent to try to connect to. As an example, if I create an API on my APIM instance for the v1 OpenAPI named openai-v1 my target would look like “https://myapim.azure-api.net/openai-v1/v1”. As of the date of this blog post, you MUST use the azure-api-net FQDN for the APIM. If you try to do a custom domain you’ll get an error back telling you that it’s not supported. I have a request into the product group to lift this limitation. I’ll update this if that is done. For third-party model gateway, this property serves the same purpose but can be any valid domain.
The authType property is going to be either ApiKey or ProjectManagedIdentity for an APIM connection. ProjectManagedIdentity will authenticate to the upstream APIM using the agent’s project’s Entra ID managed identity. When using ProjectManagedIdentity you must also specify the audience property and set it to cognitive services.azure.com if connecting to a backend Foundry resource hosting models. For a model gateway connection this will either be ApiKey or OAuth. Details on the OAuth setup can be found in the samples GitHub (I haven’t mucked with it yet). If you’re using the authType of ApiKey you additional need to pass the credentials property which includes a property of key with the API key similar to what you see below.
authType: ApiKey
credentials = {
key = MYAPIKEY
}
I haven’t messed extensively with the isSharedToAll property as of yet. For my use case I set this to false so each project got its own connection object. You may be able to create this object at the account level and set the isSharedToAll property, but I haven’t tested that yet. If you have, def let me know if that works.
Ok, now on to the property that can bring the most pain. Here we have the metadata property. This property is going to the main guts that makes this whole thing work. A few considerations, if doing this with Terraform or REST (can’t speak to Bicep or ARM), each of the properties I’m going to cover are CASE SENSITIVE. If you do the wrong casing, your connection object will not work. When connecting to an APIM or model gateway you can have Foundry either enumerate the models available (called dynamic discovery) or you can provide the exact models you want to expose (called static models).
Let’s first cover static models. Here is an example of me creating a connection to an APIM instance with static models using the authType or ProjectManagedIdentity. One thing to note is in my backend object in my APIM I’m appending /v1 to the backend path vs doing it in this connection object.
Since I’m using the v1 Azure OpenAI API, I don’t need to specify an inferenceAPIVersion. If I was using the classic API I’d need to specify the version (such as 2025-04-01-preview). Notice also I have set deploymentInPath to false. When set to true the connection will add the /deployments/deployment_name to the path. For the v1 API this isn’t required. Finally you got the models property. With a static model setup I list out the models I’m exposing to the connection. If you’re using Terraform, you MUST wrap the models in the jsonecode function. If you don’t, it will not work. The static model option is pretty helpful if you want to strictly control exactly what models the project is getting access to.
Let’s now switch over to dynamic discovery. Dynamic discovery requires you define a few additional operations inside of your API. The details can be found in this GitHub repo, but the basics of is you define an operation for a GET on a specific model and a LIST to find all the models available. These operations are management plane operations at the ARM API to retrieve deployment information. Here is an example of a setup with dynamic discovery using an APIM connection.
When doing the dynamic discovery, you’ll see the deploymentAPIVersion property set to the API version for the GET and LIST deployment operations of the ARM REST API. I added these operations into the API after I imported the v1 OpenAI spec. You can see an example in Terraform I put together in my lab repo. Dynamic discovery is a great solution when you want to the developer to have access to any new deployments you may push to the Foundry resources.
I’m not going to run through the ModelGateway connection categories because they will largely emulate what you see above with some minor differences. The official Foundry samples GitHub repo has the gory details. I also have examples in Terraform available in my own repo (if you dare subject yourself to reading my code).
Ok, so now you understand the basics of setting up the connection and what you need to do on the APIM side. For more details on setting up APIM you can reference this official repo.
Summing It Up
Ok, so you now you understand the basic connection object, how to set it up, and how it works. I’m going to cut it here and continue in another post where I’ll dig into the dirty details of how it looks to use this because I don’t want to overload your brain (and mine) with a super long post.
Before I jet I will want to provide some critical resources:
My AMAZING peer Piotr Karpala has put together a repository with examples of this pattern (and some 3rd-party integrations) with Bicep. The stuff in there is gold. He was also my late night buddy helping me work through the quirks of this integration late at night. Couldn’t have gotten it done without him (or at least would have broken many keyboards).
The Product Group’s official samples and explanations of the setup are located here. I’d highly recommending referencing them because they will always have more up to date instructions than my blog.
I’ve put together some Terraform samples for my own purposes which are you welcome to reference, loot for your own means, and laugh at my pathetic coding ability. Check out this one for the Foundry portion and this one for the APIM portion.
And here are your tips for this post:
RTFM. Seriously, read the official documentation. Today, this integration is challenging to put in place. If you try to lone wolf it, let me know how many keyboards end up being thrown through your window.
If you’re coding in Terraform or making REST calls to create these connections, remember CASE SENSITIVITY matters. If you do the wrong case sensitivity, the resource will still create but it won’t work. You’ll get very frustrated trying to troubleshoot it.
If you’re coding in Terraform don’t forget to use the jsonencode function on the models property. If you skip that, the resource will create but shit will not work.
This is only supported for prompt agents today.
Don’t forget this is public preview. So test it, but expect things to change and don’t throw this into production.
In the next post I’ll walk through how you can test the integration, some of the quirks and considerations for identity and authentication, and some of the neat APIM policy you can craft given some of the new information that is sent in the request.
Journey Of The Geek 
