Using Python to Pull Data from MS Graph API – Part 2

Using Python to Pull Data from MS Graph API – Part 2

Welcome back my fellow geeks!

In this series I’m walking through my experience putting together some code to integrate with the Microsoft Graph API (Application Programming Interface).  In the last post I covered the logic behind this pet project and the tools I used to get it done.  In this post I’ll be walking through the code and covering what’s happening behind the scenes.

The project consists of three files.  The awsintegration.py file contains functions for the integration with AWS Systems Manager Parameter Store and Amazon S3 using the Python boto3 SDK (Software Development Kit).  Graphapi.py contains two functions.  One function uses Microsoft’s Azure Active Directory Library for Python (ADAL) and the other function uses Python’s Requests library to make calls to the MS Graph API.  Finally, the main.py file contains the code that brings everything together. There are a few trends you’ll notice with all of the code. First off it’s very simple since I’m a long way from being able to do any fancy tricks and the other is I tried to stay away from using too many third-party modules.

Let’s first dig into the awsintegration.py module.  In the first few lines above I import the required modules which include AWS’s Boto3 library.

import json
import boto3
import logging

Python has a stellar standard logging module that makes logging to a centralized location across a package a breeze.  The line below configures modules called by the main package to inherit the logging configuration from the main package.  This way I was able to direct anything I wanted to log to the same log file.

log = logging.getLogger(__name__)

This next function uses Boto3 to call AWS Systems Manager Parameter Store to retrieve a secure string.  Be aware that if you’re using Parameter Store to store secure strings the security principal you’re using to make the call (in my case an IAM User via Cloud9) needs to have appropriate permissions to Parameter Store and the KMS CMK.  Notice I added a line here to log the call for the parameter to help debug any failures.  Using the parameter store with Boto3 is covered in detail here.

def get_parametersParameterStore(parameterName,region):
    log.info('Request %s from Parameter Store',parameterName)
    client = boto3.client('ssm', region_name=region)
    response = client.get_parameter(
        Name=parameterName,
        WithDecryption=True
    )
    return response['Parameter']['Value']

The last function in this module again uses Boto3 to upload the file to an Amazon S3 bucket with a specific prefix.  Using S3 is covered in detail here.

def put_s3(bucket,prefix,region,filename):
    s3 = boto3.client('s3', region_name=region)
    s3.upload_file(filename,bucket,prefix + "/" + filename)

Next up is the graphapi.py module.  In the first few lines I again import the necessary modules as well as the AuthenticationContext module from ADAL.  This module contains the AuthenticationContext class which is going to get the OAuth 2.0 access token needed to authenticate to the MS Graph API.

import json
import requests
import logging
from adal import AuthenticationContext

log = logging.getLogger(__name__)

In the function below an instance of the AuthenticationContext class is created and the acquire_token_with_client_credentials method is called.   It uses the OAuth 2.0 Client Credentials grant type which allows the script to access the MS Graph API without requiring a user context.  I’ve already gone ahead and provisioned and authorized the script with an identity in Azure AD and granted it the appropriate access scopes.

Behind the scenes Azure AD (authorization server in OAuth-speak) is contacted and the script (client in OAuth-speak) passes a unique client id and client secret.  The client id and client secret are used to authenticate the application to Azure AD which then looks within its directory to determine what resources the application is authorized to access (scope in OAuth-speak).  An access token is then returned from Azure AD which will be used in the next step.

def obtain_accesstoken(tenantname,clientid,clientsecret,resource):
    auth_context = AuthenticationContext('https://login.microsoftonline.com/' +
        tenantname)
    token = auth_context.acquire_token_with_client_credentials(
        resource=resource,client_id=clientid,
        client_secret=clientsecret)
    return token

A properly formatted header is created and the access token is included. The function checks to see if the q_param parameter has a value and it if does it passes it as a dictionary object to the Python Requests library which includes the key values as query strings. The request is then made to the appropriate endpoint. If the response code is anything but 200 an exception is raised, written to the log, and the script terminates.  Assuming a 200 is received the Python JSON library is used to parse the response.  The JSON content is searched for an attribute of @odata.nextLink which indicates the results have been paged.  The function handles it by looping until there are no longer any paged results.  It additionally combines the paged results into a single JSON array to make it easier to work with moving forward.

def makeapirequest(endpoint,token,q_param=None):
 
    headers = {'Content-Type':'application/json', \
    'Authorization':'Bearer {0}'.format(token['accessToken'])}

    log.info('Making request to %s...',endpoint)
        
    if q_param != None:
        response = requests.get(endpoint,headers=headers,params=q_param)
        print(response.url)
    else:
        response = requests.get(endpoint,headers=headers)    
    if response.status_code == 200:
        json_data = json.loads(response.text)
            
        if '@odata.nextLink' in json_data.keys():
            log.info('Paged result returned...')
            record = makeapirequest(json_data['@odata.nextLink'],token)
            entries = len(record['value'])
            count = 0
            while count < entries:
                json_data['value'].append(record['value'][count])
                count += 1
        return(json_data)
    else:
        raise Exception('Request failed with ',response.status_code,' - ',
            response.text)

Lastly there is main.py which stitches the script together.  The first section adds the modules we’ve already covered in addition to the argparse library which is used to handle arguments added to the execution of the script.

import json
import requests
import logging
import time
import graphapi
import awsintegration
from argparse import ArgumentParser

A simple configuration for the logging module is setup instructing it to write to the msapiquery.log using a level of INFO and applies a standard format.

logging.basicConfig(filename='msapiquery.log', level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')
logger = logging.getLogger(__name__)

This chunk of code creates an instance of the ArgumentParser class and configures two arguments.  The sourcefile argument is used to designate the JSON parameters file which contains all the necessary information.

The parameters file is then opened and processed.  Note that the S3 parameters are only pulled in if the –s3 switch was used.

parser = ArgumentParser()
parser.add_argument('sourcefile', type=str, help='JSON file with parameters')
parser.add_argument('--s3', help='Write results to S3 bucket',action='store_true')
args = parser.parse_args()

try:
    with open(args.sourcefile) as json_data:
        d = json.load(json_data)
        tenantname = d['parameters']['tenantname']
        resource = d['parameters']['resource']
        endpoint = d['parameters']['endpoint']
        filename = d['parameters']['filename']
        aws_region = d['parameters']['aws_region']
        q_param = d['parameters']['q_param']
        clientid_param = d['parameters']['clientid_param']
        clientsecret_param = d['parameters']['clientsecret_param']
        if args.s3:
            bucket = d['parameters']['bucket']
            prefix = d['parameters']['prefix']

Next up the get_parametersParameterStore function from the awsintegration module is executed twice.  Once to get the client id and once to get the client secret.  Note that the get_parameters method for Boto3 Systems Manager client could have been used to get both of the parameters in a single call, but I didn’t go that route.

    logging.info('Attempting to contact Parameter Store...')
    clientid = awsintegration.get_parametersParameterStore(clientid_param,aws_region)
    clientsecret = awsintegration.get_parametersParameterStore(clientsecret_param,aws_region)

In these next four lines the access token is obtained by calling the obtain_accesstoken function and the request to the MS Graph API is made using the makeapirequest function.

    logging.info('Attempting to obtain an access token...')
    token = graphapi.obtain_accesstoken(tenantname,clientid,clientsecret,resource)

    logging.info('Attempting to query %s ...',endpoint)
    data = graphapi.makeapirequest(endpoint,token,q_param)

This section creates a string representing the current day, month, and year and prepends the filename that was supplied in the parameters file.  The file is then opened using the with statement.  If you’re familiar with the using statement from C# the with statement is similar in that it ensures resources are cleaned up after being used.

Before the data is written to file, I remove the @odata.nextLink key if it’s present.  This is totally optional and just something I did to pretty up the results.  The data is then written to the file as raw text by using the Python JSON encoder/decoder.

    logging.info('Attempting to write results to a file...')
    timestr = time.strftime("%Y-%m-%d")
    filename = timestr + '-' + filename
    with open(filename,'w') as f:
        
        ## If the data was paged remove the @odata.nextLink key
        ## to clean up the data before writing it to a file

        if '@odata.nextLink' in data.keys():
            del data['@odata.nextLink']
        f.write(json.dumps(data))

Finally, if the s3 argument was passed when the script was run, the put_s3 method from the awsintegration module is run and the file is uploaded to S3.

    logging.info('Attempting to write results to %s S3 bucket...',bucket)
    if args.s3:
        awsintegration.put_s3(bucket,prefix,aws_region,filename)

Exceptions thrown anywhere in the script are captured here written to the log file.  I played around a lot with a few different ways of handling exceptions and everything was so interdependent that if there was a failure it was best for the script to stop altogether and inform the user.  Naftali Harris has an amazing blog that walks through the many different ways of handling exceptions in Python and the various advantages and disadvantages.  It’s a great read.

except Exception as e:
    logging.error('Exception thrown: %s',e)
    print('Error running script.  Review the log file for more details')

So that’s what the code is.  Let’s take a quick look at the parameters file below.  It’s very straight forward.  Keep in mind both the bucket and prefix parameters are only required when using the –s3 option.  Here are some details on the other options:

  • The tenantname attribute is the DNS name of the Azure AD tenant being queries.
  • The resource attribute specifies the resource the access token will be used for.  If you’re going to be hitting the MS Graph API, more than likely it will be https://graph.microsoft.com
  • The endpoint attribute specifies the endpoint the request is being made to including any query strings you plan on using
  • The clientid_param and clientsecret_param attributes are the AWS Systems Manager Parameter Store parameter names that hold the client id and client secret the script was provisioned from Azure AD
  • The q_param attribute is an array of key value pairs intended to story OData query strings
  • The aws_region attribute is the region the S3 bucket and parameter store data is stored in
  • The filename attribute is the name you want to set for the file the script will produce
{
    "parameters":{
        "tenantname": "mytenant.com",
        "resource": "https://graph.microsoft.com",
        "endpoint": "https://graph.microsoft.com/beta/auditLogs/signIns",
        "clientid_param":"myclient_id",
        "clientsecret_param":"myclient_secret",
        "q_param":{"$filter":"createdDateTime gt 2019-01-09"},
        "aws_region":"us-east-1",
        "filename":"sign_in_logs.json",
        "bucket":"mybucket",
        "prefix":"myprefix"
    }
}

Now that the script has been covered, let’s see it action.  First I’m going to demonstrate how it handles paging by querying the MS Graph API endpoint to list out the users in the directory.  I’m going to append the $select query parameter and set it to return just the user’s id to make the output more simple and set the $top query parameter to one to limit the results to one user per page.  The endpoint looks like this https://graph.microsoft.com/beta/users?$top=1&select=id.

I’ll be running the script from an instance of Cloud9.  The IAM user I’m using with AWS has appropriate permissions to the S3 bucket, KMS CMK, and parameters in the parameter store.  I’ve set each of the parameters in the parameters file to the appropriate values for the environment I’ve configured.  I’ll additionally be using the –s3 option.

 

run_script.png

Once the script is complete it’s time to look at the log file that was created.  As seen below each step in the script to aid with debugging if something were to fail.  The log also indicates the results were paged.

log

The output is nicely formatted JSON that could be further transformed or fed into something like Amazon Athena for further analysis (future post maybe?).

json.png

Cool right?  My original use case was sign-in logs so let’s take a glance that.  Here I’m going to use an endpoint of https://graph.microsoft.com/beta/auditLogs/signIns with a OData filter option of createdDateTime gt 2019-01-08 which will limit the data returned to today’s sign-ins.

In the logs we see the script was successfully executed and included the filter specified.

graphapi_log_sign.png

The output is the raw JSON of the sign-ins over the past 24 hours.  For your entertainment purposes I’ve included one of the malicious sign-ins that was captured.  I SO can’t wait to examine this stuff in a BI tool.

sign_in_json

Well that’s it folks.  It may be ugly, but it works!  This was a fun activity to undertake as a first stab at making something useful in Python.  I especially enjoyed the lack of documentation available on this integration.  It really made me dive deep and learn things I probably wouldn’t have if there were a billion of examples out there.

I’ve pushed the code to Github so feel free to muck around with it to your hearts content.

Using Python to Pull Data from MS Graph API – Part 1

Welcome to 2019 fellow geeks! I hope each of you had a wonderful holiday with friends and family.

It’s been a few months since my last post. As some of you may be aware I made a career move last September and took on a new role with a different organization. The first few months have been like drinking from multiple fire hoses at once and I’ve learned a ton. It’s been an amazing experience that I’m excited to continue in 2019.

One area I’ve been putting some focus in is learning the basics of Python. I’ve been a PowerShell guy (with a bit of C# thrown in there) for the past six years so diving into a new language was a welcome change. I picked up a few books on the language, watched a few videos, and it wasn’t clicking. At that point I decided it was time to jump into the deep end and come up with a use case to build out a script for. Thankfully I had one queued up that I had started in PowerShell.

Early last year my wife’s Office 365 account was hacked. Thankfully no real damage was done minus some spam email that was sent out. I went through the wonderful process of changing her passwords across her accounts, improving the complexity and length, getting her on-boarded with a password management service, and enabling Azure MFA (Multi-factor Authentication) on her Office 365 account and any additional services she was using that supported MFA options.  It was not fun.

Curious of what the logs would have shown, I had begun putting together a PowerShell script that was going to pull down the logs from Azure AD (Active Directory), extract the relevant data, and export it CSV (comma-separate values) where I could play around with it in whatever analytics tool I could get my hands on. Unfortunately life happened and I never had a chance to finish the script or play with the data. This would be my use case for my first Python script.

Azure AD offers a few different types of logs which Microsoft divides into a security pillar and an activity pillar. For my use case I was interested in looking at the reports in the Activity pillar, specifically the Sign-ins report. This report is available for tenants with an Azure AD Premium P1 or P2 subscription (I added P2 subscriptions to our family accounts last year).  The sign-in logs have a retention period of 30 days and are available either through the Azure Portal or programmatically through the MS Graph API (Application Programming Interface).

My primary goals were to create as much reusable code as possible and experiment with as many APIs/SDKs (Software Development Kits) as I could.  This was accomplished by breaking the code into various reusable modules and leveraging AWS (Amazon Web Services) services for secure storage of Azure AD application credentials and cloud-based storage of the exported data.  Going this route forced me to use the MS Graph API, Microsoft’s Azure Active Directory Library for Python (or ADAL for short), and Amazon’s Boto3 Python SDK.

On the AWS side I used AWS Systems Manager Parameter Store to store the Azure AD credentials as secure strings encrypted with a AWS KMS (Key Management Service) customer-managed customer master key (CMK).  For cloud storage of the log files I used Amazon S3.

Lastly I needed a development environment and source control.  For about a day I simply used Sublime Text on my Mac and saved the file to a personal cloud storage account.  This was obviously not a great idea so I decided to finally get my GitHub repository up and running.  Additionally I moved over to using AWS’s Cloud9 for my IDE (integrated development environment).   Cloud9 has the wonderful perk of being web based and has the capability of creating temporary credentials that can do most of what my AWS IAM user can do.  This made it simple to handle permissions to the various resources I was using.

Once the instance of Cloud9 was spun up I needed to set the environment up for Python 3 and add the necessary libraries.  The AMI (Amazon Machine Image) used by the Cloud9 service to provision new instances includes both Python 2.7 and Python 3.6.  This fact matters when adding the ADAL and Boto3 modules via pip because if you simply run a pip install module_name it will be installed for Python 2.7.  Instead you’ll want to execute the command python3 -m pip install module_name which ensures that the two modules are installed in the appropriate location.

In my next post I’ll walk through and demonstrate the script.

Have a great week!

 

 

 

 

Updates coming soon

Hello my fellow geeks!  It’s been a while since my last post.  I took on a new role recently and have been very busy getting up to speed.  I finally have enough breathing room where I’ll have some time to work on some new posts.

Stay tuned!

A Comparison – AWS Managed Microsoft AD and Azure Active Directory Domain Services

A Comparison – AWS Managed Microsoft AD and Azure Active Directory Domain Services

Over the past year I’ve done deep dives into both Amazon’s AWS Managed Microsoft Active Directory and Microsoft’s Azure Active Directory Domain Services.  These services represent each vendor’s offering of a managed Windows Active Directory (AD) service.  I extensively covered the benefits of a service over the course of the posts, so today I’m going to cover the key features of each service.  I’m also going to include two tables.  One table will outline the differences in general features while the other outlines the differences in security-related features.

Let’s hit on the key points first.

  • Amazon provides a legacy (Windows AD is legacy folks) managed service while Microsoft provides a modernized service (Azure AD) which has been been integrated with a legacy service.
  • Microsoft synchronizes users, passwords hashes, and groups from the Azure AD to a managed instance of Windows Active Directory.  The reliance on this synchronization means the customer has to be comfortable synchronizing both directory data and password hashes to Azure AD.  Amazon does not require any data be synchronized.
  • Amazon provides the capability to leverage the identities in the managed instance of Windows AD or in a forest that has a trust with the managed instance to be leveraged in managing AWS resources.  In this instance Amazon is taking a legacy service and enabling it for management of the modern cloud management plane.
  • The pricing model for the services differs where Amazon bills on a per domain controller basis while Microsoft bills on the number of objects in the directory.
  • Amazon’s service is eligible to be used in solutions that require PCI DSS Level 1 or HIPAA.
  • Both services use a delegated model where the customer has full control over an OU rather the directory itself.  Highly privileged roles such as Schema Admin, Enterprise Admins, and Domain Admins are maintained by the cloud provider.
  • Both services provide LDAP for legacy applications customers may be trying to lift and shift.  Microsoft limits LDAP to read operations while Amazon supports both read and write operations.
  • Both services support LDAPS.  At this time Amazon requires an instance of Active Directory Certificate Services be deployed to act as a Certificate Authority and provide certificates to the managed domain controllers.
  • Both services do not allow the customer to modify the Default Domain Policy or Default Domain Controller Policies.  This means the customer cannot modify the password or lockout policy applied to the domain.  Amazon provides a method of enforcing custom password and lockout policies through Fine Grained Password Policies.  Additionally, the customer does not have the ability to harden the OS of the domain controllers for either service so it is important to review the vendor documentation.
  • Amazon’s service supports Active Directory forest trusts and external trusts.  Microsoft’s service doesn’t support trusts at this time.

Here is a table showing the comparison of general features:

Features AWS Managed Microsoft AD Azure Active Directory Domain Services
Cost Basis Number of Domain Controllers Number of Directory Objects
Schema Extensions Yes, with limitations No
Trusts Yes, with limitations No
Domain Controller Log Access Security and DNS Server Event Logs No
DNS Management Yes Yes
Snapshots Yes No
Limit of Managed Forests 10 per account 1 per Azure AD tenant
Supports being used on-premises Yes with Direct Connect or VPN No, within VNet only
Scaled By Customer Yes No
Max number of Domain Controller 20 per directory Unknown how service is scaled

Here is a table of security capabilities:

Features AWS Managed Microsoft AD Azure Active Directory Domain Services
Requires Directory Synchronization No Yes, including password
Fine-Grained Password Policies Yes, limited to seven No
Smart Card Authentication Not native, requires RADIUS No
LDAPS Yes, with special requirements Yes, but LDAP operations are limited to read
LDAPS Protocols SSLv3, TLS 1.0, TLS 1.2 TLS 1.0, TLS 1.2
LDAPS Cipher Suites RC4, 3DES, AES128, AES256 RC4, 3DES, AES128, AES256
Kerberos Delegation Account-Based and Resource-Based Resource-Based
Kerberos Encryption RC4, AES128, AES256 RC4, AES128, AES256
NTLM Support NTLMv1, NTLMv2 NTLMv1, NTLMv2

Well folks that sums it up.  As you can see from both of the series as well as this summary post both vendors have taken very different approaches in providing the service.  It will be interesting to see how these offerings evolve over the next few years.  As much as we’d love to see Windows Active Directory go away, it will still be here for years to come.

Until next time my fellow geeks!

AWS Managed Microsoft AD Deep Dive Part 7 – Trusts and Domain Controller Event Logs

AWS Managed Microsoft AD Deep Dive  Part 7 – Trusts and Domain Controller Event Logs

Welcome back fellow geek.  Today I’m continuing my deep dive series into AWS Managed Microsoft AD.  This will represent the seventh post in the series and I’ve covered some great content over the series including:

  1. An overview of the service
  2. How to setup the service
  3. The directory structure, pre-configured security principals, group policies and the delegated security model
  4. How to configure LDAPS and the requirements that pop up due to Amazon’s delegation model
  5. Security of the service including supported secure transport protocols, ciphers, and authentication protocols
  6. How do schema extensions work and what are the limitations

Today I’m going cover three additional capabilities of AWS Managed Microsoft AD which includes the creation of trusts, access to the Domain Controller event logs, and scalability.

I’ll first cover the capabilities around Active Directory trusts.  Providing this capability opens up the possibility a number of scenarios that aren’t possible in managed Windows Active Directory (Windows AD) services that don’t support trusts such as Microsoft’s Azure Active Directory Domain Services.  Some of the scenarios that pop up in my head are resource forest, trusts with trusted partners to maintain collaboration for legacy applications (applications dependent on legacy protocols such as Kerberos/NTLM/LDAP), trusts between development, QA, and production forests, and the usage of features features such as selective authentication to mitigate the risk to on-premises infrastructure.

For many organizations, modernization of an entire application catalog isn’t feasible but those organizations still want to take advantage of the cost and security benefits of cloud services.  This is where AWS Managed Microsoft AD can really shine.  It’s capability to support Active Directory forests trusts opens up the opportunity for those organizations to extend their identity boundary to the cloud while supporting legacy infrastructure.  Existing on-premises core infrastructure services such as PKI and SIEM can continue to be used and even extended to monitor the infrastructure using the managed Windows AD.

As you can see this is an extremely powerful capability and makes the service a good for almost every Windows AD scenario.  So that’s all well and good, but if you wanted marketing material you’d be reading the official documentation right?  You came here for the deep dive, so let’s get into it.

The first thing that popped into my mind was the question as to how Amazon would be providing this capability in a managed service model.  Creating a forest trust typically requires membership in privileged groups such as Enterprise Admins and Domain Admins, which obviously isn’t possible in a manged service.  I’m sure it’s possible to delegate the creation of Active Directory trusts and DNS conditional forwarders with modifications of directory permissions and possibly user rights, but there’s a better way.  What is this better way you may be asking yourself?  Perhaps serving it up via the Directory Services console in the same way schema modifications are served up?

Let’s walk through the process of setting up an Active Directory forest trust with a customer-managed traditional implementation of Windows Active Directory and an instance of AWS Managed Microsoft AD.  For this I’ll be leveraging my home Hyper-V lab.  I’m actually in the process of rebuilding it so there isn’t much there right now.  The home lab consists of two virtual machines, one named JOG-DC running Windows Server 2016 and functions as a domain controller (AD DS) and certificate authority (AD CS) for the journeyofthegeek.com Active Directory forest.  The other virtual machine is named named JOG-CLIENT, runs Windows 10, and is joined to the journeyofthegeek.com domain.  I’ve connected my VPC with my home lab using AWS’s Managed VPN to setup a site-to-site IPSec VPN connection with my local pfSense box.

7awsadds1.png

Prior to setting up the trusts there are a few preparatory steps that need to be completed.  The steps will be familiar to those of you who have established forests trusts across firewalled network segments.  At a high level, you’ll want to perform the following tasks:

  1. Ensure the appropriate ports are opened between the two forests.
  2. Ensure DNS resolution between the two forests is established

For the first step I played it lazy since this is is a temporary configuration (please don’t do this in production).   I allowed all traffic from the VPC address range to my lab environment by modifying the firewall rules on my pfSense box.  On the AWS side I needed to adjust the traffic rules for the security group SERVER01 is in as well as the security group for the managed domain controllers.

7awsadds2.png

To establish DNS resolution between the two forests I’ll be using conditional forwarders setup within each forest.  Setting the conditional forwarders up in the journeyofthegeek.com forest means I have to locate the IP addresses of the managed domain controllers in AWS.  There are a few ways you could do it, but I went to the AWS Directory Services Console and selected the geekintheweeds.com directory.

7awsadds3

On the Directory details section of the console the DNS addresses list the IP addresses the domain controllers are using.

7awsadds4.png

After creating the conditional forwarder in the DNS Management MMC in the journeyofthegeek.com forest, DNS resolution of a domain controller from geekintheweeds.com was successful.

7awsadds5.png

I next created the trust in the journeyofthegeek.com domain ensuring to select the option to create the trust in this domain only and recording the trust password using the Active Directory Domains and Trusts.  We can’t create the trusts in both domains since we don’t have an account with the appropriate privileges in the AWS managed domain.

Next up I bounced back over to the Directory Services console and selected the geekintheweeds.com directory.  From there I selected the Network & security tab to open the menu needed to create the trust.

7awsadds6.png

From here I clicked the Add trust relationship button which brings up the Add a trust relationship menu.  Here I filled in the name of the domain I want to establish the trust with, the trust password I setup in the journeyofthegeek.com domain, select a two-way trust, and add an IP that will be used within configuration of the conditional forwarder setup by the managed service.

7awsadds7.png

After clicking the Add button the status of the trust is updated to Creating.

7awsadds8.png

The process takes a few minutes after which the status reports as verified.

7awsadds9.png

Opening up the Active Directory Users and Computers (ADUC) MMC in the journeyofthegeek.com domain and selecting the geekintheweeds.com domain successfully displays the directory structure.  Trying the opposite in the geekintheweeds.com domain works correctly as well.  So our two-way trust has been created successfully.  We would now have the ability to setup any of the scenarios I talked about earlier in the post including a resource forest or leveraging the managed domain as a primary Windows AD service for on-premises infrastructure.

The second capability I want to briefly touch on is the ability to view the Security Event Log and DNS Server logs on the managed domain controllers.  Unlike Microsoft’s managed Windows AD service, Amazon provides ongoing access to the Security Event Log and DNS Server Log.  The logs can be viewed using the Event Log MMC from a domain-joined machine or programmatically with PowerShell.  The group policy assigned to the Domain Controllers OU enforces a maximum event log size of 256MB but Amazon also archives a year’s worth of logs which can be requested in the event of an incident.  The lack of this capability was a big sore spot for me when I looked at Azure Active Directory Domain Services.  It’s great to see Amazon has identified this critical use case.

Last but definitely not least, let’s quickly cover the scalability of the service.  Follow Microsoft best practices and you can take full advantage of scaling horizontally with the click of a single button.  Be aware that the service only scales horizontally and not vertically.  If you have applications that don’t follow best practices and point to specific domain controllers or perform extremely inefficient LDAP queries (yes I’m talking to you developers who perform searches using front and rear-facing wildcards and use LDAP_MATCHING_RULE_IN_CHAIN filters) horizontal scaling isn’t going to help you.

Well folks that rounds out this entry into the series.  As we saw in the post Amazon has added key capabilities that Microsoft’s managed service is missing right now.  This makes AWS Managed Microsoft AD the more versatile of the two services and more than likely a better fit in almost any scenario where there is a reliance on Windows AD.

In my final posts of the series I’ll provide a comparison chart showing the differing capabilities of both AWS and Microsoft’s services.

See you next post!

 

 

 

AWS Managed Microsoft AD Deep Dive Part 6 – Schema Modifications

AWS Managed Microsoft AD Deep Dive  Part 6 – Schema Modifications

Yes folks, we’re at the six post for the series on AWS Managed Microsoft AD (AWS Managed AD.  I’ve covered a lot of material over the series including an overview, how to setup the service, the directory structure, pre-configured security principals, group policies, and the delegated security model, how to configure LDAPS in the service and the implications of Amazon’s design, and just a few days ago looked at the configuration of the security of the service in regards to protocols and cipher suites.  As per usual, I’d highly suggest you take a read through the prior posts in the series before starting on this one.

Today I’m going to look the capabilities within the AWS Managed AD to handle Active Directory schema modifications.  If you’ve read my series on Microsoft’s Azure Active Directory Domain Services (AAD DS) you know that the service doesn’t support the schema modifications.  This makes Amazon’s service the better offering in an environment where schema modifications to the standard Windows AD schema are a requirement.  However, like many capabilities in a managed Windows Active Directory (Windows AD) service, limitations are introduced when compared to a customer-run Windows Active Directory infrastructure.

If you’ve administered an Active Directory environment in a complex enterprise (managing users, groups, and group policies doesn’t count) you’re familiar with the butterflies that accompany the mention of a schema change.  Modifying the schema of Active Directory is similar to modifying the DNA of a living being.  Sure, you might have wonderful intentions but you may just end up causing the zombie apocalypse.  Modifications typically mean lots of application testing of the schema changes in a lower environment and a well documented and disaster recovery plan (you really don’t want to try to recover from a failed schema change or have to back one out).

Given the above, you can see the logic of why a service provider providing a managed Windows AD service wouldn’t want to allow schema changes.  However, there very legitimate business justifications for expanding the schema (outside your standard AD/Exchange/Skype upgrades) such as applications that need to store additional data about a security principal or having a business process that would be better facilitated with some additional metadata attached to an employee’s AD user account.  This is the market share Amazon is looking to capture.

So how does Amazon provide for this capability in a managed Windows AD forest?  Amazon accomplishes it through a very intelligent method of performing such a critical activity.  It’s accomplished by submitting an LDIF through the AWS Directory Service console.  That’s right folks, you (and probably more so Amazon) doesn’t have to worry about you as the customer having to hold membership in a highly privileged group such as Schema Admins or absolutely butchering a schema change by modifying something you didn’t intend to modify.

Amazon describes three steps to modifying the schema:

  1. Create the LDIF file
  2. Import the LDIF file
  3. Verify the schema extension was successful

Let’s review each of the steps.

In the first step we have to create a LDAP Data Interchange Format (LDIF) file.  Think of the LDIF file as a set of instructions to the directory which in this could would be an add or modify to an object class or attribute.  I’ll be using a sample LDIF file I grabbed from an Oracle knowledge base article.  This schema file will add the attributes of unixUserName, unixGroupName, and unixNameIinfo to the default Active Directory schema.

To complete step one I dumped the contents below into an LDIF file and saved it as schemamod.ldif.

dn: CN=unixUserName, CN=Schema, CN=Configuration, DC=example, DC=com
changetype: add
attributeID: 1.3.6.1.4.1.42.2.27.5.1.60
attributeSyntax: 2.5.5.3
isSingleValued: TRUE
searchFlags: 1
lDAPDisplayName: unixUserName
adminDescription: This attribute contains the object's UNIX username
objectClass: attributeSchema
oMSyntax: 27

dn: CN=unixGroupName, CN=Schema, CN=Configuration, DC=example, DC=com
changetype: add
attributeID: 1.3.6.1.4.1.42.2.27.5.1.61
attributeSyntax: 2.5.5.3
isSingleValued: TRUE
searchFlags: 1
lDAPDisplayName: unixGroupName
adminDescription: This attribute contains the object's UNIX groupname
objectClass: attributeSchema
oMSyntax: 27

dn:
changetype: modify
add: schemaUpdateNow
schemaUpdateNow: 1
-

dn: CN=unixNameInfo, CN=Schema, CN=Configuration, DC=example, DC=com
changetype: add
governsID: 1.3.6.1.4.1.42.2.27.5.2.15
lDAPDisplayName: unixNameInfo
adminDescription: Auxiliary class to store UNIX name info in AD
mayContain: unixUserName
mayContain: unixGroupName
objectClass: classSchema
objectClassCategory: 3
subClassOf: top

For the step two I logged into the AWS Management Console and navigated to the Directory Service Console.  Here we can see my instance AWS Managed AD with the domain name of geekintheweeds.com.

6awsadds1.png

I then clicked hyperlink on my Directory ID which takes me into the console for the geekintheweeds.com instance.  Scrolling down shows a menu where a number of operations can be performed.  For the purposes of this blog post, we’re going to focus on the Maintenance menu item.  Here we the ability to leverage AWS Simple Notification Service (AWS SNS) to create notifications for directory changes such as health changes where a managed Domain Controller goes down.  The second section is a pretty neat feature where we can snapshot the Windows AD environment to create a point-in-time copy of the directory we can restore.  We’ll see this in action in a few minutes.  Lastly, we have the schema extensions section.

6awsadds2.png

Here I clicked the Upload and update schema button and entered selected the LDIF file and added a short description.  I then clicked the Update Schema button.

6awsadds3.png

If you know me you know I love to try to break stuff.  If you look closely at the LDIF contents I pasted above you’ll notice I didn’t update the file with my domain name.  Here the error in the LDIF has been detected and the schema modification was cancelled.

6awsadds4.png

I went through made the necessary modifications to the file and tried again.  The LDIF processes through and the console updates to show the schema change has been initialized.

6awsadds5.png

Hitting refresh on the browser window updates the status to show Creating Snapshot.  Yes folks Amazon has baked into the schema update process a snapshot of the directory provide a fallback mechanism in the event of your zombie apocalypse.  The snapshot creation process will take a while.

6awsadds6.png

While the snapshot process, let’s discuss what Amazon is doing behind the scenes to process the LDIF file.  We first saw that it performs some light validation on the LDIF file, it then takes a snapshot of the directory, then applies to the changes to a single domain controller by selecting one as the schema master, removing it from directory replication, and applying the LDIF file using the our favorite old school tool LDIFDE.EXE.  Lastly, the domain controller is added back into replication to replicate the changes to the other domain controller and complete the changes.  If you’ve been administering Windows AD you’ll know this has appeared recommended best practices for schema updates over the years.

Once the process is complete the console updates to show completion of the schema installation and the creation of the snapshot.

6awsadds7.png

 

AWS Managed Microsoft AD Deep Dive Part 5 – Security

AWS Managed Microsoft AD Deep Dive  Part 5 – Security

You didn’t think I was done with AWS Managed Microsoft AD yet did you?  In this post I’m going to perform some tests to evaluate the protocols and ciphers suites available for LDAPS as well as checking out the managed Domain Controllers support for NTLMv1 and the cipher suites supported for Kerberos.  I’ll be using the same testing mechanisms I used when for my series on Microsoft Azure Active Directory Domain Services.

For those of you who are new to the series, I’ve been performing a deep dive review of AWS Managed Microsoft AD which is Amazon’s answer to a managed Windows Active Directory service.  In the first post I provided a high level overview of the service, in the second post I covered the setup of the service, the third post reviewed the directory structure, pre-configured security principals and group policies, and the delegated security model, and in the fourth entry I delved into how Amazon has managed to delegate configuration of LDAPS and the requirements that pop up due to their design choices.  I highly recommend you review those posts as well as my series on Microsoft Azure AD Domain Services if you’d like to compare the two services.

I’ve made a modification to my lab and have added another server named SERVER02 which will be running Linux.  The updated Visio looks like this.

labpart5

Server01 has been configured with the Windows Remote Server Administration Tools (RSAT) for Active Directory as well as holding the Active Directory Certificate Services (AD CS) role and being configured as a root Enterprise CA.  I’ve also done all the necessary configuration to distribute the certificates to the managed domain controllers and have successfully tested LDAPS.  Server02 will be used to test SSLv3 and NTLM.  I’ve modified the instance to use the domain controllers as DNS servers by overriding DHCP settings as outlined in this article.

The first thing I’m going to do is test to see if SSLv3 has been disabled on the managed domain controllers.  Recall that the managed Domain Controllers are running Windows Server 2012 R2 which has SSLv3 enabled by default.  It can be disabled by modifying the registry as documented here.  Believe it or not you can connect to the managed domain controllers registry via a remote registry connection.  Checking the registry location shows that the SSLv3 node hasn’t been created which is indicative of SSLv3 still being enabled.

5awsadds1.png

To be sure I checked it using the same method that I used in my Azure AD Domain Services post which is essentially compiling another version of openssh that supports SSLv3.  After the customized version was installed and I queried the Domain Controller over port 636 which you can see in the screenshot below that SSLv3 is still enabled.  Suffice to say this surprised me considering what I had seen so far in regards to the security of the service.  This will be a show stopper for some organizations in adopting the service especially since it isn’t configurable by the customer that I observed.

5awsadds2.png

So SSLv3 is enabled and presents a risk.  Have the cipher suites been hardened?  For this I’ll again use a tool developed by Thomas Pornin.   The options I’m using perform an exhaustive search across the typically offered cipher suites, space the connections out by 1 second, and start with a minimum of sslv3.

5awsadds3.png

The results are what I expected and mimic the results I saw when testing Azure AD Domain Services, minus the support for SSLv3 which Microsoft has disabled in their managed offering.  The supported cipher suites look to be the out of the box defaults for Server 2012 R2 and include RC4 and 3DES which are ciphers with known vulnerabilities.  The inability to disallow the ciphers might again be a show stopper for organizations with strict security requirements.

The Kerberos protocol is a critical component of Windows Active Directory providing the glue to hold the service together including (but in no way exhaustive) being behind the users authentication to a domain-joined machine, the single sign-on experience, and the ability to form trusts with other forests.  Given the importance of the protocol, it’s important to ensure its backed by strong ciphers.  The ciphers supported by a Windows Active Directory are configurable and can be checked by looking at the msDS-SupportedEncryptionTypes attribute of a domain controller object.

I next pulled up a domain controller object in ADUC and reviewed the attribute.  The attribute on the managed domain controllers has a value of 28, which is the default for Windows Server 2012 R2.  The value translates to support of the following cipher suites:

  • RC4_HMAC_MD5
  • AES128_CTS_HMAC_SHA1
  • AES256_CTS_HMAC_SHA1_96

These are the same cipher suites supported by Microsoft’s Azure AD Domain Services service.  In this case both vendors have left the configuration to the defaults.

Lastly, to emulate my testing Azure AD Domain Services, I tested support for NTLMv1.  By default Windows Server 2012 R2 supports NTLMv1 due to requirements for backwards compatibility. Microsoft has long recommended disabling NTLMv1 due to the documented issues with the security of the protocol. Sadly there are a large number of applications and devices in use in enterprises which still require NTLMv1.

To test the AWS managed domain controllers I’m going to use Samba’s smbclient package on SERVER02.  I’ll use the client to connect to the domain controller’s share from SERVER02 using NTLM.  I first installed the smbclient package by running:

yum install samba-client.

The client enforces the use NTLMV2 in smbclient by default so I needed to make some modifications to the global section of the smb.conf file by adding client ntlmv2 auth = no. This option disables NTLMv2 on smbclient and will force it to use NTLMv1.

5awsadds4.png

In order to see whether or not the client was using NTLMv1 when connecting to the domain controllers, I started a packet capture using tcpdump before initiating a connection with the smbclient.

5awsadds6.png

I then transferred the packet capture over to my Windows box with WinSCP, opened the capture with WireShark, and navigated to the packet containing the Session Setup Request.  In the parsed capture we don’t see an NTLMv2 Response which means NTLMv1 was used to authenticate to the domain controller indicating NTLMv1 is supported by the managed domain controllers.

 

5awsadds5

 

So what can we take from the findings of this analysis?

  1. Amazon has left the secure transport protocols to the defaults which means SSLv3 is supported.
  2. Amazon has left the cipher suites to the defaults which means both RC4 and 3DES cipher suites are supported for both LDAPS and Kerberos.

I’d really like to see Amazon address the support for SSLv3 as soon as possible.  There is no reason I can see why that shouldn’t be shut off by default.  Similar to my requests to Microsoft, I’d like to see Amazon allow the supported cipher suites to be configurable via the AWS Management Console.  These two changes would save organizations with strict security requirements, such as those in the public sector, to utilize the services without introducing significant risk (and audit headaches).

In my next post I’ll demonstrate how the service can be leveraged to provide Windows Active Directory service to on-premises machines or machines in another public cloud as well as exploring how to create a forest trust with the service.

See you next post!