Unlocking the black box that is AD RMS Part 1

So you want to understand what’s going on under the hood of Active Directory Rights Management Service (AD RMS)?  You can try searching Microsoft TechNet and MSDN, and what you’ll find is somewhat disappointing. I’m not sure why, but Microsoft hasn’t done a great job of breaking AD RMS down into the nit and grit as they have done with other applications.

My goal over the next few weeks is to fill that gap. Over the past few days, I have been working to break open that black box and I want to share what I’ve learned. I will not be focusing on the basics of AD RMS, as I feel the following posts do a good job of that:

Still curious? Well then, let’s begin. This first post will focus primarily on the tools I used to gather the information I will be presenting.

Packet Capturing Tools

You can’t properly study a tool that has a client-server model without capturing that network traffic.  Normally I stick with Wireshark, but for this process I ended up using Microsoft Network Monitor (Netmon).  The reason for Netmon over Wireshark is Netmon will report the PIDs and process names of the applications creating the traffic. This was a big help in monitoring AD RMS on the servers as we can narrow the traffic we care about down to the IIS worker process. I also find the LDAP query parser in Netmon to be far superior to Wireshark.

Web Proxy Debugging Tool

With AD RMS transactions taking place over HTTP/HTTPS, it only makes sense to utilize a tool such as Fiddler. For those unfamiliar with Fiddler, it is a web proxy debugging tool that makes analyzing web transactions (both HTTP and HTTPS) a breeze. I began using it when I was digging into Federation a year or two back. I find myself using it on a daily basis as more and more apps begin to communicate primarily over HTTP/HTTPS.

I used Fiddler to capture the web transactions between the AD RMS client and the AD RMS server. It parses the packets so nicely, you can easily dig into the XrML of the RAC/CLC/SLC/EULs going back and forth. This was a huge aid in understanding what exactly an AD RMS certificate looks like.

AD RMS Client and Server Side Tracing

Client and server side tracing was an absolute necessity. This is the stuff you’ll want to take a look at if you really want to understand AD RMS. Microsoft has some wonderful articles that describe how to set it up. Keep in mind the client tracing is specific to Office 2010.

Event Logs and Diagnostic Logging

You can’t dig into a Microsoft product without examining event logs! The Directory Services log on the domain controllers are huge here. You’ll need to enable AD diagnostic logging to capture the LDAP queries being run by the AD RMS client and server. This article is a great place to learn more about turning on AD diagnostic logging.

Procmon

Do you hate yourself? If so, procmon is the tool for you! Procmon came in very handy when I needed know what registry keys the AD RMS client and server were hitting when looking for hardcoded settings.

Lab Environment

Last and most importantly, let me describe my lab environment. For this project I setup four servers and two clients. The server/client setup is detailed below. A two-way transitive forest trust was established between contoso.local and fabrikam.local. A two-way AD RMS Trusted User Domain between the separate AD RMS clusters.

SERVER01
OS: Windows Server 2008 R2 Standard
Domain: contoso.local
Roles: Active Directory Domain Services, Active Directory Certificate Services
Applications: Microsoft Network Monitor

SERVER02
OS: Windows Server 2008 R2 Standard
Domain: fabrikam.local
Roles: Active Directory Domain Services, Active Directory Certificate Services
Applications: Microsoft Network Monitor

SERVER03
OS: Windows Server 2008 R2 Standard
Domain: contoso.local
Roles: Active Directory Rights Management Services, Windows Internal Database, SQL Management Studio
Applications: Microsoft Network Monitor, Debugview

SERVER02
OS: Windows Server 2008 R2 Standard
Domain: fabrikam.local
Roles: Active Directory Rights Management Services, Windows Internal Database, SQL Management Studio
Applications: Microsoft Network Monitor, Debugview

CLIENT01
OS: Windows 7 Professional
Domain: contoso.local
Application: Microsoft Office 2010 Professional, Microsoft Network Monitor, Debugview, Fiddler

CLIENT02
OS: Windows 7 Professional
Domain: fabrikam.local
Application: Microsoft Office 2010 Professional, Microsoft Network Monitor, Debugview, Fiddler

When working with AD RMS, it is critical to remember the importance of certificate and certificate revocation checks. In a lab environment, I would suggest you make it easy on yourself and disable CRL checking for your client and server boxes according to this article. This will save you some headaches.

Do yourself a favor and disable AD RMS caching. It will save you significant time when trying to replicate the acquisition of a fresh RAC or EUL. This article describes the process. Also, if you opt to use a WID, check out this article for connecting to the WID using SQL Management Studio.

All right folks, that will do it for this post. Next up will be a breakdown of intraforest consumption of protected content. To prep yourself for the post, I would suggest reviewing the following articles:

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s