Integrating Azure AD and AWS – Part 2

Posted on December 5, 2017 by mattfeltonma

Update: In November 2019 AWS introduced support for integration between Azure AD and AWS SSO. The integration offers a ton more features, including out of the box support for multiple AWS accounts. I highly recommend you go that route if you’re looking to integrate the two platforms. Check out my series on the new integration here.

Today I will continue the journey into the integration between Azure AD and Amazon Web Services. In my first entry I covered the reasons why you’d want to integrate Azure AD with AWS and provided a high-level overview of how the solution works. The remaining entries in this series will cover the steps involved in completing the integration including deep dives into the inner workings of the solution.

Let me start out by talking about the testing environment I’ll be using for this series.

The environment includes three virtual machines (VMs) running on Windows Server 2016 Hyper V on a server at my house. The virtual machines consists three servers running Windows Server 2016 with one server acting as a domain controller for the journeyofthegeek.local Active Directory (AD) forest, another server running Active Directory Federation Services (AD FS) and Azure AD Connect (AADC), and the third server running MS SQL Server and IIS. The IIS instance hosts a .NET sample federated application published by Microsoft.

In Microsoft Azure I have a single Vnet configured for connectivity to my on-premises lab through a site-to-site IPSec virtual private network (VPN) I’ve setup with pfSense. Within the Vnet exists a single VM running Windows 10 that is domain-joined to the journeyofthegeek.local AD domain. The Azure AD tenant providing the identity backend for the Microsoft Azure subscription is synchronized with the journeyofthegeek.local AD domain using Azure AD Connect and is associated with the domain journeyofthegeek.com. Authentication to the Azure AD tenant is federated using my instance of AD FS. I’m not synchronizing passwords and am using an alternate login ID with the user principal name being synchronized to Azure AD being stored in the AD attribute msDS-CloudExtensionAttribute1. The reason I’m still configured to use an alternate login ID was due to some testing I needed to do for a previous project.

I created a single test user in the journeyofthegeek.local Active Directory domain named Rick Sanchez with a user principal name (UPN) of rick.sanchez@journeyofthegeeklocal and msDS-CloudExtensionAttribute1 of rick.sanchez@journeyofthegeek.com. The only attribute to note that the user has populated is the mail attribute which has the value of rick.sanchez@journeyofthegeek.com. The user is being synchronized to Azure AD via the Azure AD Connect instance.

In AWS I have a single elastic compute cloud (EC2) instance running Windows Server 2016 within a virtual private cloud (VPC). I’ll be configuring Azure AD as an identity provider associated with the AWS account and will be associating an AWS IAM role named AzureADEC2Admins. The role will grant full admin rights over the management of the EC2 instances associated to the account via the AmazonEC2FullAccess permissions policy.

Let’s begin shall we?

The first step I’ll be taking is to log into the Azure Portal as an account that is a member of the global admins and navigate to the Azure Active Directory blade. From there I select Enterprise Applications blade and hit the New Application link.

I then search the application gallery for AWS, select the AWS application, accept the default name, and hit the Add button. Azure AD will proceed to add the application and will then jump to a quick start page. So what exactly does it mean to add an application to Azure AD? Good question, for that we’ll want to use the Azure AD cmdlets. You can reference this link.

Before we jump into running cmdlets, let’s talk very briefly about the concept of application identities in AAD. If you’ve managed Active Directory Domain Services (AD DS), you’re very familiar with the concept of service accounts. When you needed an application (let’s call it a non-human to be more in-line with industry terminology) to access AD-integrated resources directly or on-behalf of a user you would create a security principal to represent the non-human. That security principal could be a user object, managed service account object, or group managed service account object. You would then grant that security principal rights and permissions over the resource or grant it the right to impersonate a user and access the resource on the user’s behalf. The part we want to focus in on is the impersonation or delegation to access a resource on the behalf of a user. In AD DS that delegation is accomplished through the Kerberos protocol.

When we shift over to AAD the same basic concepts still exist of creating a security principal to represent the application and granting that application direct or delegated access to a resource. The difference is the protocol handling the access shifts from Kerberos to OAuth 2.0. One thing many people become confused about is thinking that OAuth handles authentication. It doesn’t. It has nothing do with authentication and everything to do with authorization, or more clearly delegation. When we add an application the AAD a service principal object and sometimes application object (in AWS instance both are created) are created in the AAD tenant to represent the application. I’m going to speak to the service principal object for the AWS integration, but you can read through this link for a good walkthrough on application and service principal objects and how they differ.

Now back to AWS. So we added the application and we now have a service principal object in our tenant representing AWS. Here is a few of the attributes for the object pulled via PowerShell.

Review of the attributes for the object provide a few pieces of interesting information. We’ll get to experiment more with what these mean when we start doing Fiddler captures, but let’s talk a bit about them now. The AppRoles attribute provides a single default role of msiam_access. Later on we’ll be adding additional roles that will map back to our AWS IAM roles.

Next up we have the KeyCredentials which contains two entries. This attribute took me a while to work out. In short, based upon the startDate, I think these two entries are referencing the self-signed certificate included in the IdP metadata that is created after the application is added to the directory. I’ll cover the IdP metadata in the next entry.

The Oauth2Permissions are a bit funky for this use case since we’re not really allowing the application to access AWS on our behalf, but rather asking it to produce a SAML assertion asserting our identity. Maybe the delegation can be thought of as delegating Azure AD the right to create logical security tokens representing our users that can be used to assert an identity to AWS.

The PasswordCredentials contains a single entry which shares the same KeyID as the KeyCredential. As best I can figure from reading the documentation is this would normally contain entries for client keys when not using certificate authentication. Given that it contains a single entry with the same KeyID as the KeyCredential for signing, I can only guess it will contain an entry even with a certificate is used to authenticate the application.

The last attributes of interest are the PreferredTokenSigningKeyThumbprint which references the certificate within the IdP metadata and the replyURLs which is the assertion consumer URI for AWS.

So yeah, that’s what happens in those 2 or 3 seconds the AWS application is registered with Azure AD. I found it interesting how the service principal object is used to represent trust between Azure AD and AWS and all the configuration information attached to the object after the application is simply added. It’s nice to have some of the configuration work done for us out of the box, but there is much more to do.

In the next entry I’ll walk through the Quick Start for the AWS application configuration and explore the metadata Azure AD creates.

The journey continues in my third entry.

Integrating Azure AD and AWS – Part 1

Posted on December 3, 2017 by mattfeltonma

Hi everyone. After being slammed with work from the real job over the past few months, I’m back with a new deep dive into the integration between Azure Active Directory (AAD) and Amazon Web Services (AWS). I enjoyed the heck out of this one because I finally got some playtime in AWS and got to integrate two of the big cloud providers together to make some cool stuff happen. There are a lot of blogs and articles out there (including Microsoft’s and Amazon’s *cough cough*) which provide the steps to accomplish this integration, but either the steps are incomplete, outdated, or wrong and none of them give a great explanation of the why or the how. I can’t complain though, what else would I blog about?

Before we jump into the technologies that power the integration, let me first answer the question as to why we’d want to do the integration in the first place.

Let’s face it, managing digital identities is hard and it’s only getting harder with the introduction of cloud technology to the mix. The tens of thousands of identities you’re managing for your users on-premises data center can quickly grow into the millions when SaaS comes into the mix. The operational overhead or supporting those millions of identities can eat up a large part of your IT budget and make your user experience miserable. Beyond the cost issue, saddling your users with hundreds of credentials means users are going to re-use passwords and store them in whatever ways are convenient for them (under the keyboard anyone?) which introduces the risk of the credentials being compromised and sensitive data getting leaked.

Now more than ever you need to put a strong focus on centralized identity management and modern authentication and authorization. Historically this was very challenging to do because of the lack of application programming interfaces (APIs) that allowed for create read update delete (CRUD) operations against the individual user records represented in an application database such as a SQL backend. Beyond the lack of good APIs, you also were stuck using complicated and limiting legacy authentication and authorization protocols such as Kerberos, NTLM, LDAP, and the like.

Thankfully the industry has made a dramatic shift towards providing robust web-based APIs and support for modern authentication and authorization such as SAML, WS-Fed, Open ID Connect, and OAuth. This presents a unique opportunity for organizations to shift towards a centralized identity management model where one authoritative store drives the lifecycle of an identity across all applications. With the introduction of the modern protocols, users aren’t required to maintain thousands of credentials and can instead rely upon a singular trusted credential service provider (CSP) to act as the primary authentication point allowing users to then assert their identities to applications. This frees the application from having to be saddled with storing and managing user credentials as well as improving the user experience, not to mention using these modern protocols is far simpler for your average developer.

Integrating AAD and AWS allow you to take advantage of centralized identity and modern authentication and authorization. AAD specifically allows you to leverage all the cool features of a modern Identity-as-a-Service (IDaaS) offering such as behavioral analytics, multifactor authentication, adaptive authentication, and contextual-based authorization. The short of it is you get a rock solid IDaaS to back the industry leading PaaS and IaaS offerings of AWS. The best of both worlds right?

Now that you understand why you’d want to integrate the two solutions, let’s look at the technology powering the solution. In this integration the vendors are leveraging the concepts of modern APIs and modern authentication and authorization I touched upon above. First up is authentication.

aws-signon

In this integration SAML, specifically the identity provider-initiated single sign-on POST binding, is being used to assert the user’s identity to the service provider (SP) after the user successfully authenticates with the identity provider (IdP). Azure AD plays the role of IdP and AWS plays the role of SP. The sequence of events plays out as follows:

The user navigates to AAD and authenticates using either a credential or an asserted identity from a federated identity store. The user then selects AWS from the listing of applications exposed through a method like the MyApps portal. AAD generates an assertion containing a claim of the user’s identity and the AWS Identity and Access Management (IAM) role(s) the user is authorized to use and redirects the user to an endpoint at AWS.
The user’s browser posts the assertion to the endpoint at AWS.
The assertion is passed to the AWS security token service (STS) which checks the assertion to ensure it is from an identity provider that has been configured to be trusted for the AWS account, verifies the roles can be granted to a federated user, and completes the authentication process granting the user access to the AWS management console.(Don’t worry, we’ll dig into this process much more deeply using Fiddler in the next post.)

For provisioning, the AWS API is used. AAD queries the AWS API using credentials for an AWS security principal that is associated with a role that has the IAMReadOnlyAccess permissions policy or greater. It queries for the IAM roles configured for the account and synchronizes those roles back to AAD. When the synchronization is complete, AAD users can then be added to the relevant roles from within AAD creating a one stop shop for doing your identity lifecycle management, authentication, and authorization. Nice right?

At a high level that is the why and the what. In my upcoming posts in this series I’ll be digging deep into the how. This will include how to do the integration, the pitfalls of the Microsoft tutorial, and of course Fiddler captures showing the conversations between the web browser, AAD, and AWS.

The journey continues in my second entry.

Deep dive into AD FS and MS WAP – User Certificate Authentication through a WAP

Posted on August 24, 2017 by mattfeltonma

Hi everyone,

Today I continue my series of posts that cover a behind the scenes look at how Active Directory Federation Service (AD FS) and the Microsoft Web Application Proxy (WAP) interact. In my first post I explained the business cases that would call for the usage of a WAP. In my second post I did a deep dive into the WAP registration process (MS refers to this as the trust establishment with AD FS and the WAP). In this post I decided to cover how user certificate authentication is achieved when AD FS server is placed behind the WAP.

AD FS offers a few different options to authenticate users to the service including Integrated Windows Authentication (IWA), forms-based authentication, and certificate authentication. Readers who work in environments with sensitive data where assurance of a user’s identity is important should be familiar with certificate authentication in the Microsoft world. If you’re unfamiliar with it I recommend you take a read through this Microsoft article.

With the recent release of the National Institute of Standards and Technology (NIST) Digital Identity Guidelines 800-63 which reworks the authenticator assurance levels (AAL) and relegates passwords to AAL1 only, organizations will be looking for other authenticator options. Given the maturity of authenticators that make use of certificates such as the traditional smart card it’s likely many organizations will look at opportunities for how the existing equipment and infrastructure can be further utilized. So all the more important we understand how AD FS certificate authentication works.

I’ll be using the lab I described in my first post. I made the following modifications/additions to the lab:

Configure Active Directory Certificate Services (AD CS) certificate authority (CA) to include certificate revocation list (CRL) distribution point (CDP). The CRLs will be served up via an IIS instance with the address crl.journeyofthegeek.com. This is the only CDP listed in the certificates. Certificates created during my original lab setup that are installed within the infrastructure do not include a CDP.
Added a non-domain-joined Windows 10 computer which be used as the endpoint the test user accesses the federation service from.

Tool-wise I used ProcMon, Fiddler, API Monitor, and WireShark.

So what did I discover?

Prior to doing any type of user interaction, I setup the tools I would be using moving forward. On the WAP I started ProcMon as an Administrator and configured my filters to capture only TCP Send and TCP Receive operations. I also setup WireShark using a filter of ip.addr==192.168.100.10 && tcp.port==80. The IP address is the IP of the web server hosting my CRLs. This would ensure I’d see the name of the process making the connection to the CDP as well as the conversation between the two nodes.

** Note that the machine will cache the CRLs after they are successfully downloaded from the CDP. It will not make any further calls until the CRLs expire. To get around this behavior while I was testing I ran the command certutil -setreg chain\ChainCacheResyncFiletime @now as outlined in this article. This forces the machine to pull the CRLs again from the CDP regardless of whether or not they are expired. I ran the command as the LOCAL SYSTEM security principal using psexec.

The final step was to start Fiddler as the NETWORK SERVICE security principal using the command psexec -i -u “NT AUTHORITY\Network Service” “C:\Program Files (x86)\Fiddler2\Fiddler.exe”. Remember that Fiddler needs the public key certificate in the appropriate file location as I outlined in my last post. Recall that the Web Application Proxy Service and the Active Directory Federation Service running on the WAP both run as that security principal.

Once all the tools were in place I logged into the non-domain joined Windows 10 box and opened up Microsoft Edge and popped the username of my test user into the username field.

After home realm discovery occurred within Azure AD, I received the forms-based login page of my AD FS instance.

Let’s take a look at what’s happened on the WAP so far.

In the initial HTTP Connect session the WAP makes to the AD FS farm, we see that the ClientHello handshake occurs where the WAP authenticates to the AD FS server to authenticate itself as described in my last post.

Once the secure session is established the WAP passes the HTTP GET request to the AD FS server. It adds a number of headers to the request which AD FS consumes to identify the client is coming from the WAP. This information is used for a number of AD FS features such as enforcing additional authentication policies for Extranet access.

The WAP also passes a number of query strings. There are a few interesting query strings here. The first is the client-request-id which is a unique identifier for the session that AD FS uses to correlate event log errors with the session. The username is obvious and shows the user’s user principal name that was inputted in the username field at the O365 login page. The wa query string shows a value of wsignin1.0 indicating the usage of WS-Federation. The wtrealm indicates the relying party identifier of the application, in this case Azure AD.

The wctx query string is quite interesting and needs to be parsed a bit on its own. Breaking down the value in the parameter we come across three unique parameters.

LoginOptions=3 indicates that the user has not selected the “Keep me signed in” option. If the user had selected that checkbox a value of 1 would have been passed and AD FS would create a persistent cookie which would exist even after the browser closes. This option is sometimes preferable for customers when opening documents from SharePoint Online so the user does not have to authenticate over and over.

The estsredirect contains the encoded and signed authentication request from O365. I stared at API monitor for a few hours going API call by API call trying to identify what this looks like once it’s decoded, but was unsuccessful. If you know how to decode it, I’d love to know. I’m very curious as to its contents.

The WAP next makes another HTTP GET to the AD FS server this time including the additional query string of pullStatus which is set equal to 0. I’m clueless as to the function on of this, I couldn’t find anything. The only other thing that changes is the referer.

My best guess on the above two sessions is the first session is where AD FS performs home realm discovery and maybe some processing on to determine if there are any special configurations for the WAP such as limited or expanded authentication options (device authN, certAuthN only). The second session is simply the AD FS server presenting the authentication methods configured for Extranet users.

The user then chooses the “Sign in with an X.509 certificate” (I’m not using SNI to host both forms and cert authN on the same port) and the WAP then performs another HTTP CONNECT to port 49443 which is the certificate authentication endpoint on the AD FS server. It again authenticates to the AD FS server with its client certificate prior to establishing the secure tunnel.

The third session we see a HTTP POST to the AD FS server with the same query parameters as our previous request but also providing a JSON object with a key of AuthMethod and the key value combination of AuthMethod=CertificateAuthentication in the body.

The next session is another HTTP POST with the same JSON object content and the key value pairs of AuthMethod=CertificateAuthentication and RetrieveCertificate=1 in the body. The AD FS server sends a 307 Temporary Redirect to the /adfs/backendproxytls/ endpoint on the AD FS server.

Prior to the redirect completing successful we see the calls to the CDP endpoint for the full and delta CRLs.

I was curious as to which process was pulling the CRLs and identified it was LSASS.EXE from the ProcMon capture.

At the /adfs/backendproxytls/ endpoint the WAP performs another HTTP POST this time posting a JSON object with a number of key value combinations.

The interesting key value types included in the JSON object are the nested JSON object for Headers which contains all the WAP headers I covered earlier. The query string JSON object which contains all the query strings I covered earlier. The SeralizedClientCertificate contains the certificate the user provided after selecting to use certificate authentication. The AD FS server then sends back a cookie to the WAP. This cookie is the cookie the representing the user’s authentication to the AD FS server as detailed in this link.

The WAP then performs a final HTTP GET back at the /adfs/ls/ endpoint including the previously described headers and query strings as well as provided the cookie it just received. The AD FS server responds by providing the assertion requested by Microsoft along with a MSISAuthenticated, MSISSignOut, and MSISLoopDetectionCookie cookies which are described in the link above.

What did we learn?

The certificate is checked at both the WAP and the AD FS server to ensure it is valid and issued from a trusted certificate authority. Remember to verify you trust the certificate chain of any user certificates on both the AD FS servers and WAPs.
CRL Revocation checking is enabled by default and is performed on both the AD FS server and the WAP. Remember to verify the locations in your CDP are available by both devices.
The AD FS servers use the LSALogonUser function in the secur32.dll library to perform standard certificate authentication to Active Directory Domain Services. I didn’t include this, but I captured this by running API monitor on the AD FS server.

In short, if you’re going to use device authentication or user certificate authentication make sure you have your PKI components in order.

See you next post!

Deep dive into AD FS and MS WAP – WAP Registration

Posted on August 8, 2017 by mattfeltonma

Hi everyone,

In today’s blog entry I’ll be doing a deep dive into how the Microsoft Web Application Proxy (WAP) established a trust with the Active Directory Federation Service (AD FS) (I’ll be referring to this as registration) in order to act as a reverse proxy for AD FS. In my first entry into this series I covered the business use cases that would call for such an integration as well as providing an overview of the lab environment I’ll be using for the series. So what does registration mean? Well, the best way to describe it is to see it in action.

Figuring out how to capture the conversation took some trial and error. This is where Sysinternals Process Explorer comes into play. I went through the process of registering the WAP with AD FS using the Remote Access Management Console configuration utility and monitored the running processes with Process Explorer. Upon reviewing the TCP/IP activity of the Remote Access Management Console process (RAMgmtUI.exe) I observed TCP connectivity to the AD FS farm.

RemoteReg

The process is running as the logged in user, in my case the administrator account I’ve configured. This meant I would need to run Fiddler using the logged in user context rather than having to do some funky with running it as SYSTEM or another security principal using PSEXEC.

I started up Fiddler and configured it to intercept HTTPS traffic as per the configuration below. Ensure that you’ve trusted the Fiddler root certificate so Fiddler can establish a man-in-the-middle (MITM) scenario.

I next ran the Remote Access Management Console and initiated the Web Application Proxy Configuration wizard. Here I ran the wizard a few different times specifying invalid credentials on the AD FS server to generate some web requests. The web conversation below popped up Fiddler.

Digging into the third session shows an HTTP POST to sts.journeyofthegeek.com/adfs/Proxy/EstablishTrust with a return code of 401 Unauthorized which we would expect given our application doesn’t know if authentication is required yet and didn’t specify an Authorization header.

estab1

Session four shows another HTTP POST to the same URL this time with an Authorization header specifying Basic authentication with our credentials Base64 encoded. We receive another 401 because we have invalid credentials which again is expected.

What’s interesting is the JSON object being posted to the URL. The JWT includes a key named SerializedTrustCertificate with a value of a Base64 encoded public-key certificate as the value.

Copy and pasting the encoded value to notepad and saving the file with a CER extension yields the certificate below of which the WAP has both the public and private key pairs. The certificate is a 2048-bit key length self-signed certificate.

At this point the WAP will attempt numerous connections to the /adfs/Proxy/GetConfiguration URL with a query string of api-version=2 as seen in the screenshot below. It will receive a 401 back because Fiddler needs a copy of the client certificate to provide to the AD FS server. At this point I let it time out and eventually the setup finished.

So what does the configuration information look like from AD FS when it’s successfully retrieved? So to see that we have to now pay attention to the Microsoft.IdentityServer.ProxyService.exe process which runs as the Active Directory Federation Services service (adfssrv).

Since the process runs as Network Service I needed to get a bit creative in how I captured the conversation with Fiddler. The first step is to export the public-key certificate for the self-signed certificate generated by the WAP, name it ClientCertificate.cer, and to store it in the Network Service profile folder in C:\Windows\ServiceProfiles\NetworkService\Documents\Fiddler2. By doing this Fiddler will use that certificate for any website requiring client certificate authentication.

The next step was to start Fiddler as the Network Service security principal. To do this I used PSEXEC with the following options:

Psexec -i -u “NT AUTHORITY\Network Service” “C:\Program Files (x86)\Fiddler2\Fiddler.exe.

I then restarted the Active Directory Federation Service on the WAP and boom there are our successful GET from the AD FS server at the /adfs/Proxy/GetConfiguration URL.

The WAP receives back a JSON object with all the configuration information for the AD FS server as seen below. Much of this is information about endpoints the AD FS server is supporting. Beyond that we get information the AD FS service configuration. The WAP uses this configuration to setup its bindings with the HTTP.SYS kernel mode driver. Yes the WAP uses HTTP.SYS in the same way AD FS uses it.

So what did we learn? When establishing the trust with the AD FS server (I’m branding this registration 🙂 ) the WAP does the following:

Generates a 2048-bit self-signed certificate
Opens an HTTPS connection with an AD FS server
Performs a POST on /adfs/Proxy/EstablishTrust providing a JSON object containing the public key certificate and authenticating to the AD FS server with the credentials provided with the wizard using Basic authentication.If the authentication is successful the AD FS server establishes the trust. (I’ll dig into this piece in the next post)
Performs a GET on /adfs/Proxy/GetConfiguration using the self-signed certificate to authenticate itself to the AD FS server.
Consumes the configuration information and configures the appropriate endpoints with calls to HTTP.SYS.

So that’s the WAP side of the fence for establishing the trust. In my next post I’ll briefly cover what goes on with the AD FS server as well as examining the LDAP calls (if any) to AD DS during the registration process.

See you next time!

Journey Of The Geek

The chronicles of a Bostonian tech geek navigating through life and technology

Category Archives: cloud

Integrating Azure AD and AWS – Part 2

Integrating Azure AD and AWS – Part 1