Force Tunneling Azure Firewall to pfSense – Part 1

Force Tunneling Azure Firewall to pfSense – Part 1

The Problem

Welcome back fellow geeks!  I hope you all are staying healthy and not going too stir crazy being stuck at home.  I’m here tonight to help break the monotony and walk you through a fun lab I recently put together.

I recently had a customer building out a sandbox environment for experimentation in Microsoft Azure.  For this environment the customer opted to setup a S2S VPN (site-to-site virtual private network) to establish connectivity between their on-premises data center and Azure.  The customer had requirements to use BGP (border gateway protocol) to exchange routes between on-premises and Azure.  Additionally, their security team required all Internet-bound traffic be piped back on-premises (force tunneling) through a set of security appliances before being egressed out to the Internet from their data center.

While I’ve setup connectivity with Azure in the past using an S2S VPN, it was with a policy-based VPN vs a route-based VPN that utilized BGP.  I’ve also worked with a lot of customers that had requirements for forced tunneling, but never got involved much in the implementation.  My customers typically use Microsoft ExpressRoute for connectivity with on-premises and a third-party NVA (network virtual appliance) like a Palo Alto or Imperva.  Since I’m not cool enough to have a lab with ExpressRoute and I’m too cheap to pay for an NVA, I’ve never had a chance to do the implementation myself.   This has meant relying on documentation and other folks within Microsoft that have had that experience.

Beyond the implementation gap in that pattern, I also have gaps in my BGP skill set.  While I’ve been lucky enough to play with a lot different technologies over the course of my career, enterprise routing was one area I never got to dive deep in.  Over my time at Microsoft and AWS, I’ve had to learn the concepts of the protocol and how to use it within the public cloud, but still have lacked any practical implementation experience.

If you know me, you know I hate not being able to implement the technologies I speak with customers about.  Hence, this blog post was born.  I’ll be walking you through the lab I built to address the gaps in my BGP and get some practical experience force tunneling traffic.  Enough with my blabbing, let’s get into it.

Lab Environment

Lab Environment

The complete lab setup I used is illustrated above.  In my home lab I’m using the 192.168.100.0/24 address range and have assigned the .1 address to the pfSense interface.  Another interface on the device has been configured for DHCP to receive a public IP address from my ISP.  Within Azure I’ve setup a single VNet (Virtual Network) assigned the address block of 10.0.0.0/16.  Within the VNet I’ve create five subnets each using a /24 block of address space (I’m terrible at subnetting).

Inside the GatewaySubnet I’ve provisioned a VPN VNG (Virtual Network Gateway) with the VpnGw2 SKU to support BGP.  The subnet named primary contains a single Windows Server 2016  VM (Virtual Machine) that I’ll be using to test the setup.  Azure Bastion sits in the Azure Bastion subnet providing me with remote access into the VM.

Finally, an Azure Firewall instance has been provisioned using the new forced tunneling feature in preview.  To support this feature, I’ve provisioned two subnets, one named AzureFirewallSubnet and one named AzureFirewallManagementSubnet  as well as two public IPs.  To route the traffic as needed, I’ve created three route tables with some user defined routes.

For this post I’m going to walk through the setup of the S2S VPN tunnel.  Anytime I can refer you to official documentation for a step-by-step process, I’ll include a hyperlink.  The steps that aren’t documented in a single place or documented at all will be the steps I’ll cover in detail.

The first thing you need to do is provision a VNet (Virtual Network).  The VNet must at least include a subnet named GatewaySubnet.  Microsoft requires this name for the subnet in order to deploy a VNG (Virtual Network Gateway).  You’ll additionally want to provision another subnet named whatever you want to hold the VM (virtual machine) to test connectivity with.  If you want to use Azure Bastion for remote access to the VM, you’ll need a third subnet which must be named AzureBastionSubnet.

While you’re twiddling your thumbs for 20 minutes waiting for the VNG, optional Bastion, and VM, you can create the local network gateway.  The local network gateway is a logical resource in Azure which represents your on-premises VPN appliance. To set this resource up you’ll need a few different items:

  • The public IP address in use by your VPN appliance
  • The BGP peer address you’ll be peering with Azure
  • The ASN (autonomous system number) you’re using on-premises

For this lab you’ll want to use a private ASN between 64512-65514 or 65521-65534.

Below is a screenshot of my configuration.  I included the entire address space I’m going to advertise, but if you’re using BGP you only need to include the addresses you’ll be using as BGP peer.

localgateway

Now that Azure is provisioning all your necessary resources, it’s a good time to bounce over to pfSense.  Note that pfSense doesn’t provide BGP support.  For that you’ll need to add the OpenBGPD package.  To do that you’ll navigate to the System drop down menu and choose Package Manger.  Search for BGP and install the OpenBGP package.  Once complete you’ll see it as an installed package as seen below.packagemanager

Once the VPN Gateway has been provisioned you can begin configuration of the connection.  The connection is also represented in Azure as a logical resource.  There isn’t much to configure when you create the connection through the Portal.  If you configure it through PowerShell, CLI, or an ARM template, you’ll have the flexibility to tweak the configuration of the tunnel.  This includes the ability to limit the encryption ciphers and hashing algorithms supported on the Azure end.  Once the connection is provisioned, open up the resource blade for it, go to the Configuration menu item in the Settings section and toggle BGP to Enabled.

connection

Before you bounce over to pfSense and configure that end, you’ll need a few pieces of information from the VPN Gateway.  Within the Portal open up the VNG resource blade.  Note the public IP address that has been assigned to the VNG.  You’ll need this for the pfSense setup.Next click the Configuration menu item in the Settings section.  Here you’ll want to check off the Configure BGP ASN check box and note the ASN (by default 65515) and the BGP peer IP address because you’ll need them later.  Click Save once you complete.  This change will take around 5 minutes.

bgp

It’s now time to hop over to pfSense.  From the main menu navigate to the VPN drop down menu and choose the IPsec option.  You’ll first need to create a IKE Phase 1 entry to establish the authentication for the tunnel.In the General Information section ensure the Key Exchange Version box is populated with IKEv2 and the Remote Gateway is populated with the public IP address of the VNG.  In the Phase 1 Proposal (Authentication) section, choose to the Mutual PSK (Pre-Shared Key) option, the My identifier is set to My IP Address and Peer identifier set to Peer IP address.  Plus in the PSK you setup in Azure.In the Phase 1 Proposal (Encryption Algorithm) section pick your preferred encryption algorithm, key length, hashing algorithm, and Diffie-Hellman Group.The Azure end supports a number of cryptographic combinations just be aware you’ll need to configure a custom IPSec Policy using the CLI, PowerShell, or ARM template if you pick a combination that isn’t offered by default.  I’m not sure what it supports by default because I couldn’t find any documentation on it.  It seems like you’ll be forced to use DHGroup2 if you create through the Azure Portal, which you really shouldn’t be using due the small key length.  If you want to nerd out a bit, take a read through this document.  I wanted to bump this up to DHGroup24, so I opted to create the custom IPSec policy with the configuration below.

ipsecpol = New-AzIpsecPolicy -IkeEncryption AES256 -IkeIntegrity SHA256 -DhGroup Dhgroup24 -IpsecEncryption GCMAES256 -IpsecIntegrity GMAES256 -PfsGroup None -SALifeTimeSeconds 28800  

Next up you need to configure a Phase 2 entry which will control how traffic is carried across the tunnel.  Expand the Phase 1 entry you created and click the Add P2 button to add a phase 2 entry.  In the General Information section you’ll want to set the Local Network option to Network with an address of 0.0.0.0/0.  This will allow us to tunnel traffic to any address through the VPN tunnel which will support our use case for the forced tunneling we’ll create later on.  In the Remote Network section, set it to the CIDR block of the VNet.In the Phase 2 proposal configure the settings to support whatever encryption setup you’re using.  For my configuration, I set it up as seen in the screenshot below.

phase2
Once the phase 2 entry is configured, navigate to the Status drop-down menu and choose IPsec.  Click the Connect button and assuming you configured everything correctly, the status shift from Disconnected, to Connecting, and will end on Established as seen below.
ipsecstatus

Hurray, you have an established VPN tunnel.  Now it’s time to configure BGP.

Since you’ve already toggled the appropriate options in Azure to support BGP, it’s now time to configure it in pfSense.  You will first need to create a firewall rule to allow the BGP traffic to flow between Azure and the pfSense box.  To do this you’ll select the Firewall drop-down menu and choose the Rules option.  Create a new rule to allow TCP port 179 from the source of the Azure BGP peer IP you noted earlier to the pfSense interface IP for the network you’re connecting to Azure.

firewallrule1

Next you have to open the Services drop-down menu and choose OpenBGPD.In this section you have a few menu options, one which allows you to modify the raw config.  Like the idiot I am, I ignored the comment at the beginning of the raw config that says not to edit it.  After editing it, I was unable to configure using the menu options.  If you’re not an idiot like me, you should be able to configure it using the menus.  My working config is illustrated below.

bgpconfigOnce you have your Config set, save it and give it a minute.  The navigate to the Status section of the OpenBGPD service.  Scroll to the bottom and check out the OpenBGPD Neighbors section.  If you’ve misconfigured anything you’ll receive an error that the log file can’t be written (useful right?)

bgpstatus

Additionally when I check the effective routes for the network interface of the VM in Azure I can see the routes propagating into the VM’s subnet.

routes

You can validate your connectivity at this point in any number of ways.  I went the lazy route and used pfSense’s Test Port capability located in the Diagnostics drop-down menu.  Make sure that you open the appropriate rules in any NSGs between you and the VM.  Also consider the VM’s host firewall if you opt to use a non-standard port or protocol like ICMP.  If you opt to test from Azure back on-premises, make sure to open the appropriate firewall rules in the pfSense firewall for the IPSec interface.

connectiontest

With that you have a working S2S VPN complete with BGP exchange of routes.  That will wrap up this post.  In the next post I’ll walk through the configuration of forced tunneling with Azure Firewall.

Continue the journey in the second post.

DNS in Microsoft Azure Part 1 – Azure-provided DNS

DNS in Microsoft Azure Part 1 – Azure-provided DNS

Updates:

  • 7/2025 – Removed bullet point highlighting lack of DNS query logging and added that this can be achieved with DNS Security Policy

This is part of my series on DNS in Microsoft Azure.

Hi everyone,

In this series of posts I’m going to talk about a technology, that while old, still provides a critical foundational service.  Yes folks, we’re going to cover Domain Naming System (DNS).  Specifically, we’re going to look at how internal DNS (non-public) works in Microsoft Azure and what the positives and negatives are of each pattern.  I’m going to go into this assuming you have a basic knowledge of DNS and understand the namespaces, different record types, forward and reverse lookup zones, recursive and iterative queries, DNS forwarding and conditional forwarding, and other core DNS concepts. If those topics are unfamiliar to you, I’d suggest reading through DNS 101 by RedHat

I’m a big fan of establishing a shared vocabulary. Below I’m going to define some terms I’ll be using throughout the series.

  • A record – Resolves a hostname to an IP address such as http://www.journeyofthegeek.com to 5.5.5.5.
  • PTR Record – Resolves an IP address to a hostname.
  • CNAME record – Alias record where you can point on (FQDNs) fully qualified domain name to another to make it a domain a human can remember and for high availability configurations.
  •  Recursive Name Resolution – A DNS query where the client asks for a definitive answer to a query and relies on the upstream DNS server to resolve it to completion.  Forwarders such as Google DNS function as recursive resolvers.
  • Iterative Name Resolution – A DNS query where a client receives back a referral to another server in place of resolving the query to completion.  Querying root hints often involves iterate name resolution.
  • Standard DNS Forwarder – Forward all queries the DNS service can’t resolve to an upstream DNS service.  These upstream servers are typically configure to perform recursive or iterative name resolution.
  • Conditional Forwarder – Forward queries for a specific DNS namespace to an upstream DNS service for resolution. This is referred to as a forward zone in BIND.
  • Split-brain / Split Horizon DNS – A DNS configuration where a DNS namespace exists authoritatively across one or more DNS implementations.  A common use case is to have a single DNS namespace defined on Internet-resolvable public facing DNS servers and also on Intranet private facing DNS servers.  This allows trusted clients to reach the service via a private IP address and untrusted clients to reach the service via a public IP address.

Now that I’ve established our vocabulary for DNS, I want to cover the 168.63.129.16 address.  If you’ve ever done anything even basic in Azure, you’ve probably run into this address or used it without knowing it.  This public IP address is owned by Microsoft and is presented as a virtual IP address serving as a communication channel to the host node  for a number of platform resources.  It provides functionality such as virtual machine (VM) agent communication of the VM’s ready state, health state, enables the VM to obtain an IP address via DHCP, and you guessed it, enables the VM to leverage Azure DNS services.  The address is static and is the same for any VNet you create in every Azure region.

Traffic is routed to and from this virtual IP address through the subnet gateway.  If you run a route print on a Windows machine, you can see this route defined in the routing table of the VM.

route
Output of route print on Azure VM

The IP address is also defined in the VirtualNetwork service tag meaning the default rules within a network security group (NSG) allow this traffic to and from the VM. DNS traffic to the IP address is not filtered by NSGs by default, but you can block it with an NSG if you wish to using the instructions outlined here. You might do this if you do not want clients using the Azure platform for DNS and instead want all of these lookups to occur through another DNS mechanism such as the Azure Private DNS Resolver or a 3rd-party DNS service you have deployed.

Now that you understand what the 168.63.129.16 virtual IP address is, let’s first cover the very basics of DNS in Azure. You can configure Azure’s DHCP service to push a custom set of DNS servers to Azure resources within the virtual network or leave the default. The default DNS Server settings for a virtual network is 168.63.129.16 IP address which provides access to the Azure-provided DNS service. DNS Server settings pushed through the Azure DHCP Service can be configured at the virtual network or virtual network interface (VNI). Best practice is to configure this at the virtual network. I’ve never come across a use case to configure it at the VNI.

Configure DNS on VNet
Configure DNS Server DHCP option on VNet

This brings us to the first option for DNS resolution in Azure, Azure-provided name resolution.  Each time you spin up a virtual network Azure assigns it a unique private DNS namespace using the format <randomly generated>.internal.cloudapp.net.  This namespace is pushed to any virtual machines with VNIs in the virtual network via DHCP Option 15. An A record for each VM deployed in the virtual network is automatically registered which allows each VM the built-in ability to resolve the names of virtual machines within the same virtual network. The platform also creates PTR records in reverse lookup zones created for each of the subnets in the virtual network where VMs have VNICs in.

Let’s look at an example with a single VNet.  I’ve created a single VNet named vnet1.  I’ve assigned the CIDR block of 10.101.0.0/16 and created a single subnet assigned the 10.101.0.0/24 block.  Two Windows Server 2016 VMs have been created named azuredns and azuredns1 with the IP addresses 10.101.0.4 and 10.101.0.5.  Azure has assigned the a namespace of r0b5mqxog0hu5nbrf150v3iuuh.bx.internal.cloudapp.net to the VNet.  Note the DHCP Server and DNS Server settings in the ipconfig output of the azuredns vm shown below.

ipconfig
IPConfig output of Azure VM

If azuredns1 is pinged from azuredns you can see the in below Wireshark capture that prior to executing the ping, azuredns performs a DNS query to the 168.63.129.16 VIP and gets back a query response with the IP address of azuredns1. Pinging the single label name of the virtual machine will work as well because the Azure-provided virtual network DNS namespace is automatically prepended to the label by the operating system due to DHCP option 15 (assuming you haven’t configured the operating system to do anything different).

wireshark
Wireshark packet capture of DNS query

An example of the resolution path is diagrammed below.

In this example, the following happens:

  1. VM1 creates a DNS query for vm2 and the FQDN configured for the virtual network is automatically added to the single label resulting in a query for vm2.random.internal.cloudapp.net. VM1 does not have a cached entry for vm2.random.internal.cloudapp.net so the query is passed on to the DNS Server configured for the VMs virtual network interface (VNIC). The DNS Server has been configured by the Azure DHCP Service to the 168.63.129.16 virtual IP which is the default configuration for virtual network DNS Server settings.
  2. The DNS query is passed through the virtual IP and on to the Azure-provided DNS Service. The Azure-provided DNS Service resolves this query against the virtual network namespaces and returns the IP address for vm2.
Azure-provided DNS resolution within a virtual network

That’s all well and good for very basic DNS resolution, but who the heck has a single VNet in anything but a test environment?  So can we expand Azure-provided DNS to multiple VNets?  The answer is yes.  Recall that each VNet has its own private DNS namespace.  The only way to resolve names contained within that namespace is for a VM in that VNet to send the query to the 168.63.129.16 address.  Yes folks, this means you would need to drop a DNS server in each VNet in order to resolve the Azure-provided DNS host names assigned to VMs within that VNet by another VMs in another VNet as illustrated in the diagram below.

In this example, the following happens:

  1. VM1 creates a DNS query for vm3.random2.internal.cloudapp.net. The fully-qualified domain name (FQDN) must be provided because each virtual network uses a different randomly generated namespace. VM1 does not have a cached entry for vm3.random2.internal.cloudapp.net so the query is passed on to the DNS Server configured for the VMs virtual network interface (VNIC). Here the virtual network DNS server settings have been configured to for 10.0.2.4 which has been passed down to VM1 through the Azure DHCP Service.
  2. The DNS query arrives at DNS Server 1. DNS Server 1 does not have a cached entry. It determines it is not authoritative for the random2.internal.cloudapp.net namespace but determines it has a conditional forwarder configured for the zone pointing to 10.100.2.4. The DNS query is recursively passed on to 10.100.2.4 over the virtual network peering.
  3. The DNS query arrives at DNS Server 2. DNS Server 2 does not have a cached entry. It determines it is not authoritative for the random2.internal.cloudapp.net namespace and it does not have a conditional forwarder configured. DNS Server 2 has been configured with a standard forwarder to the 168.63.129.16 virtual IP. The query is passed on to the virtual IP and to Azure-provided DNS which resolves the query for requested record and returns the response.
Azure-provided DNS with multiple virtual networks

You can see as the number of VNets increases the scalability of this solution quickly breaks down because who the heck wants to have a DNS Server deployed to evey virtual network. Take note that if you wanted to resolve these host names from on-premises you could use a similar conditional forwarder pattern where you would pass the query to the DNS Server in Azure and on to Azure-provided DNS.

Let’s sum up the positives and negatives of Azure-provided DNS with the default virtual network namespaces..

  • Positives
    • No need to provision your own DNS servers and worry about high availability or scalability
    • DNS service provided by Azure automatically scales
    • VMs within a VNet can resolve each other’s IP addresses out of the box
    • VMs within a VNet can perform reverse lookups to get the IP address of another VM
    • DNS Query Logging is supported with use of DNS Security Policy
  • Negatives
    • Solution doesn’t scale with multiple VNets
    • You’re stuck with the namespace assigned to the VNet
    • WINS and NetBIOS are not supported
    • Only A records and PTR records that are automatically registered by the service are supported (no manual registration of records)

As you can see from the above the negatives far outweigh the positives for using the default virtual network namespaces and you’ll likely never use them. The important thing to take away from this post is an understanding of how DNS Server settings are configured and how you can configure a DNS Server to communicate with the Azure-provided DNS service. This will be relevant for everything we talk about moving forward.

In the next post l cover Azure’s new offering in the DNS space, Azure Private DNS Zones.  I’ll walk through how it works and how we can combine it with BYO DNS to create some pretty neat patterns.

See you then!

AWS Managed Microsoft AD Deep Dive Part 1 – Overview

AWS Managed Microsoft AD Deep Dive  Part 1 – Overview

Welcome back my fellow geeks!

Earlier this year I did a deep dive into Microsoft’s managed Active Directory service, Microsoft Azure Active Directory Domain Services (AAD DS).  I found was a service in its infancy and showing some promise, but very far from being an enterprise-ready service.  I thought it would be fun to look at Amazon’s (which I’ll refer to as Amazon Web Services (AWS) for the rest of the entries in this series) take on a managed Microsoft Active Directory (or as Microsoft is referring to it these days Windows Active Directory).

Unless your organization popped up in the last year or two and went the whole serverless route you are still managing operating systems that require centralized authentication, authorization, and configuration management.  You also more than likely have a ton of legacy/classic on-premises applications that require legacy protocols such as Kerberos and LDAP.  Your organization is likely using Windows Active Directory (Windows AD) to provide these capabilities along with Windows AD’s basic domain name system (DNS) service and centralized identity data store.

It’s unrealistic to assume you’re going to shed all those legacy applications prior to beginning your journey into the public cloud.  I mean heck, shedding the ownership of data centers alone can be a huge cost driver.  Organizations are then faced with the challenge of how to do Windows AD in the public cloud.  Is it best to extend an existing on-premises forest into the public cloud?  What about creating a resource forest with a trust?  Or maybe even a completely new forest with no trust?  Each of these options have positives and negatives that need to be evaluated against organizational requirements across the business, technical, and legal arenas.

Whatever choice you make, it means additional infrastructure in the form of more domain controllers.  Anyone who has managed Windows AD in an enterprise knows how much overhead managing domain controllers can introduce.  Let me clarify that by managing Windows AD, it does not mean opening Active Directory Users and Computers (ADUC) and creating user accounts and groups.  I’m talking about examining performance monitor AD counters and LDAP Debug logs to properly size domain controllers, configuring security controls to comply with PCI and HIPAA requirements or aligning with DISA STIGS, managing updates and patches, and troubleshooting the challenges those bring which requires extensive knowledge of how Active Directory works.  In this day an age IT staff need to be less focused on overhead such as this and more focused on working closely with its business units to drive and execute upon business strategy.  That folks is where managed services shine.

AWS offers an extensive catalog of managed services and Windows AD is no exception.  Included within the AWS Directory Services offerings there is a powerful offering named Amazon Web Services Directory Service for Microsoft Active Directory, or more succinctly AWS Managed Microsoft AD.  It provides all the wonderful capabilities of Windows AD without all of the operational overhead.  An interesting fact is that the service has been around since December 2015 in comparison to Microsoft’s AAD DS which only went into public preview at in 3rd Q 2017.  This head start has done AWS a lot of favors and in this engineer’s opinion, has established AWS Managed Microsoft AD as the superior managed Windows AD service over Microsoft’s AAD DS.  We’ll see why as the series progresses.

Over the course of this series I’ll be performing a similar analysis as I did in my series on Microsoft AAD DS.  I’ll also be examining the many additional capabilities AWS Managed Microsoft AD provides and demoing some of them in action.  My goal is that by the end of this series you understand the technical limitations that come with the significant business benefits of leveraging a managed service.

See you next post!

The Evolution of AD RMS to Azure Information Protection – Part 7 – Deep Dive into cross Azure AD tenant consumption

The Evolution of AD RMS to Azure Information Protection – Part 7 – Deep Dive into cross Azure AD tenant consumption

Each time I think I’ve covered what I want to for Azure Information Protection (AIP), I think of another fun topic to explore.  In this post I’m going to look at how AIP can be used to share information with users that exist outside your tenant.  We’ll be looking at the scenario where an organization has a requirement to share protected content with another organization that has an Office 365 tenant.

Due to my requirements to test access from a second tenant, I’m going to supplement the lab I’ve been using.  I’m adding to the mix my second Azure AD tenant at journeyofthegeek.com.  Specific configuration items to note are as follows:

  • The tenant’s custom domain of journeyofthegeek.com is an Azure AD (AAD)-managed domain.
  • I’ve created two users for testing.  The first is named Homer Simpson (homer.simpson@journeyofthegeek.com) and the second is Bart Simpson (bart.simpson@journeyofthegeek.com).
  • Each user have been licensed with Office 365 E3 and Enterprise Mobility + Security E5 licenses.
  • Three mail-enabled security groups have been created.  The groups are named The Simpsons (thesimpsons@journeyofthegeek.com), JOG Accounting (jogaccounting@journeyofthegeek.com), and JOG IT (jogit@journeyofthegeek.com).
  • Homer Simpson is a member of The Simpsons and JOG Accounting while Bart Simpson is a member of The Simpsons and JOG IT.
  • Two additional AIP policies have been created in addition to the Global policy.  One policy is named JOG IT and one is named JOG Accounting.
  • The Global AIP policy has an additional label created named PII that enforces protection.  The label is configured to detect at least one occurrence of a US social security number.  The document is protection policy allows only members of the The Simpsons group to the role of viewer.
  • The JOG Accounting and JOG IT AIP policies have both been configured with an additional label of either JOG Accounting or JOG IT.  A sublabel for each label has also been created which enforces protection and restricts members of the relevant departmental group to the role of viewer.
  • I’ve repurposed the GIWCLIENT2 machine and have created two local users named Bart Simpson and Homer Simpson.

Once I had my tenant configuration up and running, I initialized Homer Simpson on GIWCLIENT2.  I already had the AIP Client installed on the machine, so upon first opening Microsoft Word, the same bootstrapping process I described in my previous post occurred for the MSIPC client and the AIP client.  Notice that the document has had the Confidential \ All Employees label applied to the document automatically as was configured in the Global AIP policy.  Notice also the Custom Permissions option which is presented to the user because I’ve enabled the appropriate setting in the relevant AIP policies.

7aip1.png

I’ll be restricting access to the document by allowing users in the geekintheweeds.com organization to hold the Viewer role.  The geekintheweeds.com domain is associated with my other Azure AD tenant that I have been using for the lab for this series of posts.  First thing I do is change the classification label from Confidential \ All Employees to General.  That label is a default label provided by Microsoft which has an RMS Template applied that restricts viewers to users within the tenant.

One interesting finding I discovered through my testing is that the user can go through the process of protecting with custom permissions using a label that has a pre-configured template and the AIP client won’t throw any errors, but the custom permissions won’t be applied.  This makes perfect sense from a security perspective, but it would be nice to inform the user with an error or warning.  I can see this creating unnecessary help desk calls with how it’s configured now.

When I attempt to change my classification label to General, I receive a prompt requiring me to justify the drop in classification.  This is yet another setting I’ve configured in my Global AIP policy.  This seems to be a standard feature in most data classification solutions from what I’ve observed in another major vendor.

7aip2.png

After successfully classifying the document with the General label protection is removed from the document. At this point I can apply my custom permissions as seen below.

7aip3.png

I repeated the process for another protected doc named jog_protected_for_Ash_Williams.docx with permissions restricted to ash.williams@geekintheweeds.com.  I packaged both files into an email and sent them to Ash Williams who is a user in the Geek In The Weeds tenant.  Keep in mind the users in the Geek In The Weeds tenant are synchronized from a Windows Active Directory domain and use federated authentication.

After opening Outlook the message email from Homer Simpson arrives in Ash William’s inbox.   At this point I copied the files to my desktop, closed Outlook, opened Microsoft Word and used the “Reset Settings” options of the AIP client, and signed out of my Office profile.

7aip4

At this point I started Fiddler and opened one of the Microsoft Word document. Microsoft Word pops-up a login prompt where I type in my username of ash.williams@geekintheweeds.com and I’m authenticated to Office 365 through the standard federated authentication flow. The document then pops open.

7aip5.png

Examining the Fiddler capture we see a lot of chatter. Let’s take a look at this in chunks, first addressing the initial calls to the AIP endpoint.

7aip6

If you have previous experience with the MSIPC client in the AD RMS world you’ll recall that it makes its calls in the following order:

  1. Searches HKLM registry hive
  2. Searches HKCU registry hive
  3. Web request to the RMS licensing pipeline for the RMS endpoint listed in the metadata attached to the protected document

In my previous deep dives into AD RMS we observed this behavior in action.  In the AIP world, it looks like the MSIPC client performs similarly.  The endpoint we see it first contacting is the Journey of the Geek which starts with 196d8e.

The client first sends an unauthenticated HTTP GET to the Server endpoint in the licensing pipeline. The response the server gives is a list of available SOAP functions which include GetLicensorCertificate and GetServerInfo as seen below.

7aip8.png

The client follows up the actions below:

  1. Now that the client knows the endpoint supports the GetServerInfo SOAP function, it sends an unauthenticated HTTP POST which includes the SOAP action of GetServerInfo.  The AIP endpoint returns a response which includes the capabilities of the AIP service and the relevant endpoints for certification and the like.
  2. It uses that information received from the previous request to send an unauthenticated HTTP POST which includes the SOAP action of ServiceDiscoveryForUser.  The service returns a 401.

At this point the client needs to obtain a bearer access token to proceed.  This process is actually pretty interesting and warrants a closer look.

7aip9

Let’s step through the conversation:

  1. We first see a connection opened to odc.officeapps.live.com and an unauthenticated HTTP GET to the /odc/emailhrd/getfederationprovider URI with query strings of geekintheweeds.com.  This is a home realm discovery process trying to the provider for the user’s email domain.

    My guess is this is MSAL In action and is allowing support for multiple IdPs like Azure AD, Microsoft Live, Google, and the like.  I’ll be testing this theory in a later post where I test consumption by a Google user.

    The server responds with a number of headers containing information about the token endpoints for Azure AD (since this is domain associated with an Azure AD tenant.)

    7aip10.png

  2. A connection is then opened to odc.officeapps.live.com and an unauthenticated HTTP GET to the /odc/emailhrd/getidp with the email address for my user ash.williams@geekintheweeds.com. The response is interesting in that I would have thought it would return the user’s tenant ID. Instead it returns a JSON response of OrgId.

    7aip11.png

    Since I’m a nosey geek, I decided to unlock the session for editing.  First I put in the email address associated with a Microsoft Live.  Instead of OrgId it returned MSA which indicates it detects it as being a Microsoft Live account.  I then plugged in a @gmail.com account to see if I would get back Google but instead I received back neither.  OrgId seems to indicate that it’s an account associated with an Azure AD tenant.  Maybe it would perform alternative steps depending on whether it’s MSA or Azure AD in future steps?  No clue.

  3. Next, a connection is made to oauth2 endpoint for the journeyofthegeek.com tenant. The machine makes an unathenticated requests an access token for the https://api.aadrm.com/ in order to impersonate Ash Williams. Now if you know your OAuth, you know the user needs to authenticate and approve the access before the access token can be issued. The response from the oauth2 endpoint is a redirect over to the AD FS server so the user can authenticate.

    7aip12.png

  4. After the user successfully authenticates, he is returned a security token and redirected back to login.microsoftonline.com where the assertion is posted and the user is successfully authenticated and is returned an authorization code.

    7aip13.png

  5. The machine then takes that authorization code and posts it to the oauth2 endpoint for my journeyofthegeek.com tenant. It receives back an Open ID Connect id token for ash.williams, a bearer access token, and a refresh token for the Azure RMS API.

    7aip14.png

    Decoding the bearer access token we come across some interesting information.  We can see the audience for the token is the Azure RMS API, the issuer of the token is the tenant id associated with journeyofthegeek.com (interesting right?), and the identity provider for the user is the tenant id for geekintheweeds.com.

    7aip15.png

  6. After the access token is obtained the machine closes out the session with login.microsoftonline.com and of course dumps a bunch of telemetry (can you see the trend here?).

    7aip16.png

  7. A connection is again made to odc.officeapps.live.com and the /odc/emailhrd/getfederationprovider URI with an unauthenticated request which includes a query string of geekintheweeds.com. The same process as before takes place.

Exhausted yet?  Well it’s about to get even more interesting if you’re an RMS nerd like myself.

7aip17.png

Let’s talk through the sessions above.

  1. A connection is opened to the geekintheweeds.com /wmcs/certification/server.asmx AIP endpoint with an unauthenticated HTTP POST and a SOAP action of GetServerInfo.  The endpoint responds as we’ve observed previously with information about the AIP instance including features and endpoints for the various pipelines.
  2. A connection is opened to the geekintheweeds.com /wmcs/oauth2/servicediscovery/servicediscovery.asmx AIP endpoint with an unauthenticated HTTP POST and a SOAP action of ServiceDiscoveryForUser.  We know from the bootstrapping process I covered in my last post, that this action requires authentication, so we see the service return a 401.
  3. A connection is opened to the geekintheweeds.com /wmcs/oauth2/certification/server.asmx AIP endpoint with an unauthenticated HTTP POST and SOAP action of GetLicensorCertificate.  The SLC and its chain is returned to the machine in the response.
  4. A connection is opened to the geekintheweeds.com /wmcs/oauth2/certification/certification.asmx AIP endpoint with an unauthenticated HTTP POST and SOAP action of Certify.  Again, we remember from my last post that this requires authentication, so the service again responds with a 401.

What we learned from the above is the bearer access token the client obtained earlier isn’t attended for the geekintheweeds.com AIP endpoint because we never see it used.  So how will the machine complete its bootstrap process?  Well let’s see.

  1. A connection is opened to the journeyofthegeek.com /wmcs/oauth2/servicediscovery/servicediscovery.asmx AIP endpoint with an unauthenticated HTTP POST and SOAP action of ServiceDiscoveryForUser.  The service returns a 401 after which the client makes the same connection and HTTP POST again, but this time including its bearer access token it retrieved earlier.  The service provides a response with the relevant pipelines for the journeyofthegeek.com AIP instance.
  2. A connection is opened to the journeyofthegeek.com /wmcs/oauth2/certification/server.asmx AIP endpoint with an authenticated (bearer access token) HTTP POST and SOAP action of GetLicensorCertificate.  The service returns the SLC and its chain.
  3. A connection is opened to the journeyofthegeek.com /wmcs/oauth2/certification/certification.asmx AIP endpoint with an authenticated (bearer access token) HTTP POST and SOAP action of Certify.  The service returns a RAC for the ash.williams@geekintheweeds.com along with relevant SLC and chain.  Wait what?  A RAC from the journeyofthegeek.com AIP instance for a user in geekintheweeds.com?   Well folks this is supported through RMS’s support for federation.  Since all Azure AD’s in a given offering (commercial, gov, etc) come pre-federated, this use case is supported.
  4. A connection is opened to the journeyofthegeek.com /wmcs/licensing/server.asmx AIP endpoint with an uauthenticated HTTP POST and SOAP action of GetServerInfo.  We’ve covered this enough to know what’s returned.
  5. A connection is opened to the journeyofthegeek.com /wmcs/licensing/publish.asmx AIP endpoint with an authenticated (bearer access token) HTTP POST and SOAP action of GetClientLicensorandUserCertificates.  The server returns the CLC and EUL to the user.

After this our protected document opens in Microsoft Word.

7aip18.png

Pretty neat right? Smart move by Microsoft to take advantage and build upon of the federated capabilities built into AD RMS. This is another example showing just how far ahead of their game the product team for AD RMS was. Heck, there are SaaS vendors that still don’t support SAML, let alone on-premises products from 10 years ago.

In the next few posts (can you tell I find RMS fascinating yet?) of this series I’ll explore how Microsoft has integrated AIP into OneDrive, SharePoint Online, and Exchange Online.

Have a great week!