Monthly Archives: June 2017

Helpful hints for resolving AD FS problems – Part 2

Posted on by

Welcome back to part two of my series of posts which looks at resolving problems with AD FS.  You can check out part 1 here.  In this post I’ll look another problem you may encounter while administering the service.

With the introduction of AD FS 2012 R2, Microsoft de-coupled AD FS from IIS.  AD FS running on MS versions 2012 R2 or later now use the HTTP Server API (more often referred to as HTTP.SYS).  HTTP.SYS is a kernal mode drive that was introduced in Windows Server 2003 and is used by a Windows system to listen for HTTP and HTTPS requests (check out this article for a detailed breakdown of how it works.)  Infrastructure services such as IIS and WINRM use the driver.  By integrating AD FS directly with HTTP.SYS, Microsoft was able to cut the footprint of the solution by eliminating the need for IIS.  Awesome right?  Of course it is, however, it is a bit more challenging to troubleshoot.

Issue 2: Replacing the AD FS Service Communications certificate

The service communications certificate is one of the “big three” certificates used within an AD FS implementation.  The certificate that is assigned as the service communications certificate is used to protect web communication between clients and the AD FS service (i.e. SSL/TLS).  Like any certificate, it will have a standard lifecycle and will eventually need to be replaced.  When that time comes, you can run into a very interesting problem depending on how you go about replacing that certificate.

If you’ve been managing an AD FS instance for any period of time, you’ve more than likely become quite familiar with the AD FS Management Console.  When replacing the certificate in AD FS 2012 R2 or above, you may be tempted to use the Set Service Communications Certificate action seen below.  Let’s give it a try shall we?

ADFSMMCSC

I first requested a new web certificate from the instance of AD CS through the Certificate MMC and placed it in the Computer store.  I then granted READ access to the private key for the service account AD FS is using.  After that I used the Set Service Communications Certificate action and selected the new certificate.  A quick check of the thumbprint of the certificate now being used matches the thumbprint of the new certificate (pay attention to the thumbprint, I’ll reference it again later).  Last step is to restart the AD FS service.

Screen Shot 2017-06-10 at 3.12.06 PM.png

Let’s now test the sample claim app I described in my first post.

Screen Shot 2017-06-10 at 2.58.35 PM

Uh oh.  What happened?  A check of the Application, System, and AD FS Admin logs shows no errors or warning nor does the AD FS Debug after another attempt.  Heck, even the log for the HTTP.SYS kernal driver httperr.log in C:\Windows\System32\LogFiles\HTTPERR is empty.  This is yet another instance of where the answer could not be found in any of the logs I reviewed because it’s another error related to the integration with HTTP.SYS.  What to do next?

Much of the administration of the integration with HTTP.SYS is doing using netsh.  Here we’re going to look at the certificate bindings configured for the HTTP listeners using the command http showsslcert from the netsh command prompt.

Screen Shot 2017-06-10 at 3.07.19 PM.png

Well our bindings are there, but look at the thumbprint: 12506a00b40617b096002089383015bbbb99e970.  That thumbprint does not match the thumbprint for the new certificate I set for the Service Communications certificate.  So what happened?  My best guess is when one of the HTTPS listeners are hit, the configuration in the AD FS database does not match the configuration of the HTTP.SYS listeners causing AD FS to crash.  How do we fix it?  Come to find out from this blog, there is one additional command that needs to be run to setup the listeners with the proper bindings, Set-AdfsSslCertificate.  After using the Set-AdfsSslCertificate and setting it with the new thumbprint then restarting the AD FS service, netsh http showsslcert now shows the correct thumbprint and the sample claim app is now working as expected.

Screen Shot 2017-06-10 at 3.26.51 PM

What you should take from this post is that while integrating with HTTP.SYS helps to limit the AD FS footprint, it also adds some intricacies to troubleshooting the service when it stops working.  In the next and final post in this series I will cover an issue that can pop up when a Web Application Proxy (WAP) is integrated in the mix.

See you next post!

 

Helpful hints for resolving AD FS problems – Part 1

Posted on by

Hi everyone.

Over the past week I’ve been building a lab for an upcoming deep dive into Microsoft’s Web Application Proxy.  During the course of building the lab I ran into a few interesting issues with AD FS and the Web Application Proxy that I wanted to cover.  Some were similar to issues I’ve run into in production environments and some were new to me.

These issues are interesting in that there aren’t any obvious indicators of the problem in any of the typical logs.  Two out of three required some trial and error to determine root cause, while the third drove me quite insane for a good two weeks before getting an answer from an “official” source.  Over the course of this series of blogs I’ll cover each issue in detail with the hopes that it will help others troubleshoot these issues in the future.

Issue 1: AD FS Certificate authentication fails

I’m going to start with the problem that took me the longest to resolve and eventually required getting the answer directly from an official source.

For those of you that are unfamiliar, AD FS provides the capability to offer multi-factor authentication methods both native and third-party.  Out of the box, it supports certificate-based authentication as an option for a multi-factor or “step-up” authentication mechanism.

A few months back I wanted to take advantage of the certificate authentication feature to provide a two-factor authentication solution for applications integrated with AD FS.  Like a good engineer I did my Googling, read the Microsoft articles and various blogs out there to understand how the feature worked and what the requirements were.  I built a lab in Azure, setup an AD FS server, and ensured port 49443 was open in addition to the the typical ports required by AD FS.  I created my instance of AD CS, issued a user certificate containing the user’s UPN in the subject alternate name field, and setup a sample SAML app and configured it to require Certificate authentication.

How easy it all sounds right?  I navigated to the sample application and got the screen below…

Screen Shot 2017-06-04 at 9.29.35 PM

and I waited….  and waited…. and waited…  Ummm, what went wrong?  Well surely the AD FS log will tell me what happened.

Screen Shot 2017-06-04 at 9.34.03 PM.png

Well isn’t that odd.  No errors or warnings in the AD FS Admin log.  A quick check of the Application and System logs showed no errors either.  Maybe the AD FS Debug log would show me something?  I flipped on the log and attempted another authentication.

Screen Shot 2017-06-04 at 9.38.07 PM

Nothing as well?  Maybe the server can’t query the revocation lists designated in the certificates CDP?  Nope, not that either the server can successfully contact the CDP endpoints.  At this point I began to get quite frustrated and attempted packet captures, Fiddler captures, and anything and everything I could think of.  Nothing I tried revealed the answer.

I finally gave in (which I can tell you is incredibly challenging for me) and reached out to an “official” source.  We chatted back and forth and went through much of the same steps as outlined above to ensure I didn’t miss anything.  However, we ran into another dead end.  He then reached out to some other engineers he knew and eventually we got a hit.  We were told to check to see if there were any intermediary certificates stored within the trusted root certificate authorities store.  Sounds like an odd circumstance, but sure why not.

Upon opening up the certificates MMC, opening the machine store, and exploring the trusted root certificate authorities store low and behold I see an intermediary certificate within the store.  I deleted the certificate, restarted the AD FS server and attempted another login to the sample claim application and hit the screen below.

Screen Shot 2017-06-04 at 9.50.16 PM

Boom, I’m finally receiving the certificate prompt.  Clicking the OK button brings about the successful login below.

Screen Shot 2017-06-04 at 9.51.23 PM

So what was the issue?  Apparently AD FS certificate authentication fails without generating an error in any logical location (maybe nowhere at all?) if there is an intermediary certificate in the trusted root certificate authority machine store.  I’ve verified this is an issue in both AD FS 2012 R2 and AD FS 2016.  Now why this occurs is unknown to me.  It could be the underlining HTTPS.SYS driver that pukes and doesn’t report any errors to the event logs.  I didn’t get a straight answer as to why this occurs, just that it will due to some type of integrity check on the machine certificate store.  Odd right?

That completes the rundown of the first of three problems I’ll be outlining in this series of blogs.  Hopefully this helps save someone else some time and aggravation.

See you next post!