Deep Dive Into Resource-Based Constrained Delegation – Part 1

Hello all.

Today I wanted to take a step back from cloud and step back into the world of “legacy” authentication protocols. Many of you reading this blog should be familiar with the Kerberos authentication protocol, its usage, and its plusses and minuses. If you’re unfamiliar with the protocol there are plenty of resources online that will walk you through the basics.

The best overview I’ve seen is the Mark Minasi “Cracking Open Kerberos: Understanding How Active Directory Knows Who You Are” presentation. The challenge there is you’ll need to find a working stream of the video as the Channel 9 video seems quite dead. If you enjoy nerding out like I do you can read through the following RFCs and standards:

  • The Kerberos Network Authentication Service V5 RFC 4120
  • MS-KILE Kerberos Protocol Extensions
  • MS-SFU Kerberos Protocol Extensions for Service for User and Constrained Delegation Protocol

For this blog post I’ll be focusing on Resource-Based Kerberos Constrained Delegation (RBKCD). RBKCD provides new capabilities to the Microsoft implementation of Kerberos and was introduced in Windows Server 2012. I am not a fan of re-creating documentation when someone has already done a wonderful job putting together the information, so take a read through the links below for a wonderful overview of the different types of delegation and information as to how RBKCD works.

So what types of business problems could we solve with RBKCD? Let’s suppose two organizations wish to collaborate and leverage a single shared solution to provide data analysis and the solution must meet the following requirements:

  1. The solution will be owned by Company A and hosted using Company A’s infrastructure.
  2. The solution must be capable of querying data sources in both Company A and Company B.
  3. The solution must be accessible by users within both Company A and Company B.
  4. The solution must only return the data the user is authorized to see.
  5. The solution must make security a top priority.

In the above scenario we can satisfy requirement 3 through a standard Active Directory trust between Company A and Company B. Requirement 1, 2, and 4 could be satisfied by Kerberos Unconstrained (Open) Delegation. However, requirement 5 requires we leverage the more secure Kerberos Constrained Delegation. The problem we have with that solution is Kerberos Constrained Delegation can’t satisfy requirement 2 when combined with requirements 3 and 4. Prior to Windows Server 2012, we’d have to accept the risk and loss of features such as protocol transition when using Kerberos Unconstrained Delegation. With Windows Server 2012 and later, we can leverage RBKCD to satisfy all five requirements. There are additional benefits to RBKCD in simplifying administration, but I won’t go into those, you can read about them in the links above.

So now we know what value it can bring to the business, but how does it work and what does it look like? The links I provide above answer a bit of the first question, but not much of the second. We’ll take a look at the latter question in my next blog post.

Attribute Uniqueness in Azure Active Directory

As I dive deeper into Azure Active Directory, I am learning quickly that AAD is a very different animal than on-premises Active Directory Domain Services (AD DS). While both solutions provide identity, authentication, and authorization services, they do so in very different ways. These differences require organizations to be prepared to adjust standard processes to get the two services to work together. Today I will focus on the identity portion of the solution and how the different attribute uniqueness requirements in AAD and AD DS can introduce the need for evolution of management processes for AD DS.

The attributes I want to focus on are userPrincipalName, proxyAddresses, and mail. In AD DS userPrincipalName is a single valued attribute, proxyAddresses is a multivalued attribute, and the values included in those attributes must be unique to the object in the forest. The mail attribute (the attribute that populates the E-mail field on the General tab of Active Directory Users and Computers (ADUC)) is a single valued attribute that doesn’t have a uniqueness requirement. In AAD all three attributes retain their single value or multivalued properties, however, the uniqueness requirements change considerably.

AD DS allows these values to be duplicated across different attributes. For example, one object could have a userPrincipalName of john@contoso.com and another object could have a value in its proxyAddresses attribute of SMTP:john@contoso.com. The same goes for an object that has a mail attribute of john@contoso.com and another object has a value in its proxyAddresses of john@contoso.com.

In AAD this is no longer true. User, group, and contact objects synchronized to AAD from AD DS require the userPrincipalName, proxyAddresses, and mail (also targetAddresses if you’re using it) to be unique among all objects in the directory. This means that each of the scenarios I discussed above will create synchronization errors. You can’t have one user object with a value in the proxyAddresses of john@contoso.com and another use object with mail attribute of john@contoso.com.

What happens if you do? Well, let’s make it happen. In this scenario we have two user objects with the configuration below:

Object 1
userPrincipalName: jess.felton@journeyofthegeek.com
proxyAddresses: SMTP:felton@feltonma.com
Sync Status: Already synced to Azure AD

Object 2
userPrincipalName: matt.felton@journeyofthegeek.com
proxyAddresses:
mail: felton@feltonma.com
Sync Status: Not yet synced to Azure AD

After we force a delta synchronization of Azure AD Sync, the errors provided below pop up in Synchronization Manager and an email alert:

Screen-Shot-2016-06-05-at-8.07.18-PM.png

Screen-Shot-2016-06-05-at-8.08.24-PM

The net result of the above matt.felton@journeyofthegeek.com won’t synchronize correctly to AAD and the user will be unable to authenticate to AAD. How about two user objects with the same mail attribute? That’s a common use case, right? Nope, same issue. Take note that just because you receive an error saying the issue is with a duplicate value in the proxyaddresses attribute, it could be the userPrincipalName, mail, or targetAddress of another object in AD DS.

Small differences like this can lead to major changes in how organizations manage AD DS when they begin their journey into AAD. The key take away here is to understand that AD DS and AAD are not the same thing, the differences need to be understood, and you must be prepared to evolve existing processes if you wish to leverage the solution.

I’ll end this with a thank you to Jimmie Lightner from MS for his blog post that brought light to this issue many months ago. You can read that post here.

P.S. Take note that if you opt to an alternate login ID (separate attribute from userPrincipalName for user identifier in AAD), the uniqueness will carry over to that attribute as well.

Troubleshooting AD RMS – Server Side

For server side troubleshooting I’m going to focus on two different scenarios. In the first scenario we will assume we have a user or group of users who are unable to activate or unable to consume a protected document, but we have other users who are not having issues. In the second scenario we will assume that all users are unable to activate or consume protected content.

Scenario 1

In this scenario we have a user or group of users who are unable to activate and consume protected content. We have done our client side troubleshooting, but we haven’t been able to determine the problem. Let’s begin.

  1. Our first step should be to run a troubleshooting report within the AD RMS Management Console.** When you open the Troubleshooting report you are presented with menu below.

    RMSReportWizard.jpg

    Select the appropriate date and time for the query (these would be the dates the user attempted to active or consume protected content.) Select the appropriate user and hit Finish. Once the report has finished, it will be presented to you similar to the image below.

    RMSReportResults.jpg

    The reporting features of AD RMS in Server 2008 R2 leave a lot to be desired, but for basic troubleshooting the report can be helpful. I’m not going to go into any detail with how to read the report, as it is straightforward.

    **The reporting features of AD RMS in Server 2008 R2 require the Microsoft Report Viewer Distributable 2005 update be installed.

  2. Is the troubleshooting report not helpful enough to diagnose the issue? Maybe it’s an issue with the user object, distribution group objects, or contact object.
    • For user objects, verify the user object has an email address. Typically if a user attempts to open a protected document and does not have an email address, it will be logged to the AD RMS server Application Event log.
    • For distribution group objects, verify an email address is assigned. Also verify the user is actually a member of the distribution group.
    • For contact objects representing groups in another forest, verify the object has an email address and has msExchOriginatingForest is populated.
  3. Is the issue stemming from users in a trusted Active Directory forest which has its own implementation of AD RMS and you have a Trusted User Domain with? Well there are a few things we can check.
    • Verify the service account in the trusted forest has been granted the appropriate permissions. Namely Read on the group expansion pipeline. This is required when group expansion needs to occur.
    • Verify you have not set HKLMSoftwareMicrosoftDRMSGICURL. I’ve explained this in a previous host, but let me go into a bit more detail. When this registry entry is set, both intraforest and interforest users will be referred to whatever pipeline is referenced in the GICURL entry. This means if you’ve set it to something like https://adrms.contoso.local/_wmcs/certification, users in the fabrikam forest will receive that URL back when requesting an activation URL for their domain. The AD RMS client on the fabrikam user’s computer will then attempt to activate in the wrong forest and will fail.I can’t stress this one enough folks, play carefully with the GICURL registry entry. It’s a painful Easter egg for future administrators.

Scenario 2

Complete failure of the service could mean a lot of things, but let’s quickly run through a list of must checks.

  1. Check the GICURL registry entry as discussed above.
  2. Check to make sure the service can connect with the backend WID or SQL database.
  3. Check the IIS logs to verify the users aren’t being denied access to the pipelines. If you’re seeing a ton of 401s and users are failing to authenticate, check the ACLs on the AD RMS pipelines.
  4. If the user is being prompted for a username and password and you have verified the AD RMS website has been added to the Intranet zone, ensure that the appropriate SPNs have been assigned to the AD RMS service account.
  5. If you’re experiencing really odd behavior at the client, verify the certificates being used by your AD RMS server are valid and have reachable CDPs if you’re doing CRL checking.

Remember you can always fall back on Server Side Debug Logging. It is extremely useful when you’re stumped.

Troubleshooting AD RMS – Client Side

As our long journey through AD RMS comes to a close, I wanted to touch on troubleshooting the service. While putting together the lab for the deep dive I ended up doing a ton of troubleshooting. The goal of this post is to pass that knowledge on in hopes it will save someone some time in the future.

When troubleshooting an AD RMS activation, consumption, or creation of protected documents, it’s best to start at the client.

Troubleshooting the AD RMS Client

  1. This may seem like an obvious troubleshooting step, but it is often overlooked.  Verify that the user has an email address assigned to the identity they are using to access the document.
  2. When troubleshooting the AD RMS client, your first step should be to clear the AD RMS cache and reset the AD RMS client. These two Microsoft articles do a wonderful job of outlining the steps:
    1. Reset the AD RMS Client (Office 2010)
    2. Reset the MSIPC AD RMS Client (Office 2013)

    Resetting the client according to the articles above will clear out those cached registry entries that were created during previous activations and consumption operations. It’s possible for these entries to become corrupt or outdated for some reason and they will be properly recreated during the next client activation. The other step the articles have you taking is clearing out cached keys (CLC, RAC, etc). By clearing out all the keys the client will process through the whole activation cycle.

    If activation is still failing, test on another client after resetting that client as well. If activation works on the second client, then you know there’s an issue with the first client. Move to the next step.

  3. Next let’s verify the AD RMS server information wasn’t hardcoded within the client’s registry. Maybe it was done for some troubleshooting purpose in the past or perhaps an SCP wasn’t used previously. The registry entries below are worth checking.  Keep in mind if you’re using a 64-bit Office client, you’ll want to check the SoftwareWow6432Node node.
    1. HKLMSoftwareMicrosoftMSDRMServiceLocationActivation – This is a hardcoded location for client activation.
    2. HKLMSoftwareMicrosoftMSDRMServiceLocationEnterprise – This is the hardcoded location to obtain a CLC.
    3. HKCUSoftwareMicrosoftOffice14.0CommonDRMServiceLocation – This is where previously discovered service locations are stored for Office 2010.
    4. HKCUSoftwareMicrosoftOffice15.0CommonDRMServiceLocation – This is where previously discovered service locations are stored for Office 2013.

    If you found any of these registry, talk to your administrator to see why things are being hardcoded. If you determine it’s remnants of previous troubleshooting, clear out the entries, reset the client, and try again.

  4. Still not working? Remember when I mentioned earlier about the importance AD RMS places on CRL checking? The AD RMS client will inherit the CRL checking configuration that is set in Internet Explorer. If you determine you can’t reach or validate the certificate distribution point in the AD RMS root cluster’s cert or SSL cert, disable CRL checking in Internet Explorer. Check out this article for details. Make sure that you uncheck both “check for server certificate revocation” and “check for publisher’s certificate revocation” options.Once that is complete, reset the client and try again.
  5. Still having issues? Well my friend, you have exhausted the easier troubleshooting steps and now you need to do some client side debug tracing. You can utilize the sample debug traces I provided in my deep dive to see what a normally functionomg client debug trace looks like. If you’ve taken the time to read through the previous posts, the issue should immediately pop out to you in the debug trace.

My next blog post will explore how to troubleshoot at the server end.