Deep Dive into Azure AD and AWS SSO Integration – Part 2

Posted on December 3, 2019 by mattfeltonma

Welcome back folks.

Today I’ll be continuing my series on the new integration between Azure AD and AWS SSO. In my last post I covered the challenges with the prior integration between the two platforms, core AWS concepts needed to understand the new integration, and how the new integration addresses the challenges of the prior integration.

In this post I’m going to give some more context to the challenges covered in the first post and then provide an overview of the what the old and new patterns look like. This will help clarify the value proposition of the integration for those of you who may still not be convinced.

The two challenges I want to focus on are:

The AWS app was designed to synchronize identity data between AWS and Azure AD for a single AWS account
The SAML trust between Azure AD and an AWS account had to be established separately for each AWS account.

Challenge 1 was unique to the Azure Marketplace AWS app because they were attempting to solve the identity lifecycle management problem. Your security token service (STS) needs to pass a SAML assertion which includes the AWS IAM roles you are asserting for the user. Those roles need to be mapped to the user somewhere for your STS to tap into them. This is a problem you’re going to feel no matter what STS you use, so I give the team that put together the AWS app together credit for trying.

The folks over at AWS came up with an elegant solution requiring some transformation in the claims passed in the SAML token and another solution to store the roles in commonly unused attributes in Active Directory. However, both solutions suffered the same problem in that you’re forced to workaround that mapping, which becomes considerably difficult as you began to scale to hundreds of AWS accounts.

Challenge 2 plagues all STSs because the SAML trust needs to be created for each and every AWS account. Again, something that begins to get challenging as you scale.

AWS Past Integration

In the image above, we see an example of how some enterprises addressed these problems. We see that there is some STS in use acting as an identity provider (idP) (could be Azure AD, Okta, Ping, AD FS, whatever) that has a SAML trust with each AWS account. The user to AWS IAM role mappings are included in an attribute of the user’s Active Directory user account. When the user attempts to access AWS, the STS queries Active Directory for the information. There is a custom process (manual or automated) that queries each AWS account for a list of AWS IAM Roles that are associated with the IdP in the AWS account. These roles are then populated in the attribute for each relevant user account. Lastly, CloudFormation is used to push IAM Roles to each AWS account. This could be pushed through a manual process or a CI/CD pipeline.

Yeah this works, but who wants all that overhead? Let’s look at the new method.

Azure AD and AWS SSO Integration

In the new integration where we use Azure AD and AWS SSO together, we now only need to establish a single SAML trust with AWS SSO. Since AWS SSO is integrated with AWS Organizations it can be used as a centralized identity source for all AWS accounts within the organization. Additionally, we can now leverage Azure AD to manage the synchronization of identity data (users and groups) from Azure AD to AWS SSO. We then map our users or groups to permission sets (collections of IAM policies) in AWS SSO which are then provisioned as IAM roles in the relevant AWS accounts. If we want to add a user to a role in AWS IAM, we can add that user to the relevant group in Azure AD and wait for the synchronization process to occur. Once it’s complete, that user will have access to that IAM role in the relevant accounts. A lot less work, right?

Let’s sum up what changes here:

We can use existing processes already in place to move users in and out of groups either on-premises in Windows AD (that is syncing to Azure AD with Azure AD Connect) or directly in Azure AD (if we’re not syncing from Windows AD).
Group to role mappings are now controlled in AWS SSO
Permission sets (or IAM policies for the IAM roles) are now centralized in AWS SSO
We no longer have to provision the IAM roles individually into each AWS account, we can centrally control it in AWS SSO

Cool right?

In my few posts I’ll begin walking through the integration an demonstrating some the solution.

Thanks!

Capturing and Visualizing Office 365 Security Logs – Part 2

Posted on January 31, 2019 by mattfeltonma

Hello again my fellow geeks.

Welcome to part two of my series on visualizing Office 365 security logs. In my last post I walked through the process of getting the sign-in and security logs and provided a link to some Lambda’s I put together to automate pulling them down from Microsoft Graph. Recall that the Lambda stores the files in raw format (with a small bit of transformation on the time stamps) into Amazon S3 (Simple Storage Service). For this demonstration I modified the parameters for the Lambda to download the 30 days of the sign-in logs and to store them in an S3 bucket I use for blog demos.

When the logs are pulled from Microsoft Graph they come down in JSON (JavaScript Object Notation) format. Love JSON or hate it is the common standard for exchanging information these days. The schema for the JSON representation of the sign-in logs is fairly complex and very nested because there is a ton of great information in there. Thankfully Microsoft has done a wonderful job of documenting the schema. Now that we have the logs and the schema we can start working with the data.

When I first started this effort I had put together a Python function which transformed the files into a CSV using pipe delimiters. As soon as I finished the function I wondered if there was an alternative way to handle it. In comes Amazon Athena to the rescue with its Openx-JsonSerDe library. After reading through a few blogs (great AWS blog here), StackOverflow posts, and the official AWS documentation I was ready to put something together myself. After some trial and error I put together a working DDL (Data Definition Language) statement for the data structure. I’ve made the DDLs available on Github.

Once I had the schema defined, I created the table in Athena. The official AWS documentation does a fine job explaining the few clicks that are provided to create a table, so I won’t re-create that here. The DDLs I’ve provided you above will make it a quick and painless process for you.

Let’s review what we’ve done so far. We’ve setup a reoccurring job that is pulling the sign-in and audit logs via the API and is dumping all that juicy data into cheap object storage which we can further enforce lifecycle policies for. We’ve then defined the schema for the data and have made it available via standard SQL queries. All without provisioning a server and for pennies on the dollar. Not to shabby!

At this point you can use your analytics tool of choice whether it be QuickSight, Tableau, PowerBi, or the many other tools that have flooded the market over the past few years. Since I don’t make any revenue from these blog posts, I like to go the cheap and easy route of using Amazon QuickSight.

After completing the initial setup of QuickSight I was ready to go. The next step was to create a new data set. For that I clicked the Manage Data button and selected New Data Set.

On the Create a Data Set screen I selected the Athena option and created a name for the data source.

From there I selected the database in Athena which for me was named azuread. The tables within the database are then populated and I chose the tbl_signin_demo which points to the test S3 bucket I mentioned previously.

Due to the complexity of the data structure I opted to use a custom SQL query. There is no reason why you couldn’t create the table I’m about to create in Athena and then connect to that table instead to make it more consumable for a wider array of users. It’s really up to you and I honestly don’t know what the appropriate “big data” way of doing it is. Either way, those of you with real SQL skills may want to look away from this query lest you experience a Raiders of The Lost Ark moment.

You were warned.

SELECT records.id, records.createddatetime, records.userprincipalname, records.userDisplayName, records.userid, records.appid, records.appdisplayname, records.ipaddress, records.clientappused, records.mfadetail.authdetail AS mfadetail_authdetail, records.mfadetail.authmethod AS mfadetail_authmethod, records.correlationid, records.conditionalaccessstatus, records.appliedconditionalaccesspolicy.displayname AS cap_displayname, array_join(records.appliedconditionalaccesspolicy.enforcedgrantcontrols,' ') AS cap_enforcedgrantcontrols, array_join(records.appliedconditionalaccesspolicy.enforcedsessioncontrols,' ') AS cap_enforcedsessioncontrols, records.appliedconditionalaccesspolicy.id AS cap_id, records.appliedconditionalaccesspolicy.result AS cap_result, records.originalrequestid, records.isinteractive, records.tokenissuername, records.tokenissuertype, records.devicedetail.browser AS device_browser, records.devicedetail.deviceid AS device_id, records.devicedetail.iscompliant AS device_iscompliant, records.devicedetail.ismanaged AS device_ismanaged, records.devicedetail.operatingsystem AS device_os, records.devicedetail.trusttype AS device_trusttype,records.location.city AS location_city, records.location.countryorregion AS location_countryorregion, records.location.geocoordinates.altitude, records.location.geocoordinates.latitude, records.location.geocoordinates.longitude,records.location.state AS location_state, records.riskdetail, records.risklevelaggregated, records.risklevelduringsignin, records.riskstate, records.riskeventtypes, records.resourcedisplayname, records.resourceid, records.authenticationmethodsused, records.status.additionaldetails, records.status.errorcode, records.status.failurereason  FROM "azuread"."tbl_signin_demo" CROSS JOIN (UNNEST(value) as t(records))

This query will de-nest the data and give you a detailed (possibly extremely large depending on how much data you are storing) parsed table. I was now ready to create some data visualizations.

The first visual I made was a geospatial visual using the location data included in the logs filtered to failed logins. Not surprisingly our friends in China have shown a real interest in my and my wife’s Office 365 accounts.

Next up I was interested in seeing if there were any patterns in the frequency of the failed logins. For that I created a simple line chart showing the number of failed logins per user account in my tenant. Interestingly enough the new year meant back to work for more than just you and me.

Like I mentioned earlier Microsoft provides a ton of great detail in the sign-in logs. Beyond just location, they also provide reasons for login failures. I next created a stacked bar chat to show the different reasons for failed logs by user. I found the blocked sign-ins by malicious IPs interesting. It’s nice to know that is being tracked and taken care of.

Failed logins are great, but the other thing I was interested in is successful logins and user behavior. For this I created a vertical stacked bar chart that displayed the successful logins by user by device operating system (yet more great data captured in the logs). You can tell from the bar on the right my wife is a fan of her Mac!

As I gather more data I plan on creating some more visuals, but this was great to start. The geo-spatial one is my favorite. If you have access to a larger data set with a diverse set of users your data should prove fascinating. Definitely share any graphs or interesting data points you end up putting together if you opt to do some of this analysis yourself. I’d love some new ideas!

That will wrap up this series. As you’ve seen the modern tool sets available to you now can do some amazing things for cheap without forcing you to maintain the infrastructure behind it. Vendors are also doing a wonderful job providing a metric ton of data in their logs. If you take the initiative to understand the product and the data, you can glean some powerful information that has both security and business value. Even better, you can create some simple visuals to communicate that data to a wide variety of audiences making it that much more valuable.

Have a great weekend!

AWS Managed Microsoft AD Deep Dive Part 6 – Schema Modifications

Posted on August 29, 2018 by mattfeltonma

Yes folks, we’re at the six post for the series on AWS Managed Microsoft AD (AWS Managed AD. I’ve covered a lot of material over the series including an overview, how to setup the service, the directory structure, pre-configured security principals, group policies, and the delegated security model, how to configure LDAPS in the service and the implications of Amazon’s design, and just a few days ago looked at the configuration of the security of the service in regards to protocols and cipher suites. As per usual, I’d highly suggest you take a read through the prior posts in the series before starting on this one.

Today I’m going to look the capabilities within the AWS Managed AD to handle Active Directory schema modifications. If you’ve read my series on Microsoft’s Azure Active Directory Domain Services (AAD DS) you know that the service doesn’t support the schema modifications. This makes Amazon’s service the better offering in an environment where schema modifications to the standard Windows AD schema are a requirement. However, like many capabilities in a managed Windows Active Directory (Windows AD) service, limitations are introduced when compared to a customer-run Windows Active Directory infrastructure.

If you’ve administered an Active Directory environment in a complex enterprise (managing users, groups, and group policies doesn’t count) you’re familiar with the butterflies that accompany the mention of a schema change. Modifying the schema of Active Directory is similar to modifying the DNA of a living being. Sure, you might have wonderful intentions but you may just end up causing the zombie apocalypse. Modifications typically mean lots of application testing of the schema changes in a lower environment and a well documented and disaster recovery plan (you really don’t want to try to recover from a failed schema change or have to back one out).

Given the above, you can see the logic of why a service provider providing a managed Windows AD service wouldn’t want to allow schema changes. However, there very legitimate business justifications for expanding the schema (outside your standard AD/Exchange/Skype upgrades) such as applications that need to store additional data about a security principal or having a business process that would be better facilitated with some additional metadata attached to an employee’s AD user account. This is the market share Amazon is looking to capture.

So how does Amazon provide for this capability in a managed Windows AD forest? Amazon accomplishes it through a very intelligent method of performing such a critical activity. It’s accomplished by submitting an LDIF through the AWS Directory Service console. That’s right folks, you (and probably more so Amazon) doesn’t have to worry about you as the customer having to hold membership in a highly privileged group such as Schema Admins or absolutely butchering a schema change by modifying something you didn’t intend to modify.

Amazon describes three steps to modifying the schema:

Let’s review each of the steps.

In the first step we have to create a LDAP Data Interchange Format (LDIF) file. Think of the LDIF file as a set of instructions to the directory which in this could would be an add or modify to an object class or attribute. I’ll be using a sample LDIF file I grabbed from an Oracle knowledge base article. This schema file will add the attributes of unixUserName, unixGroupName, and unixNameIinfo to the default Active Directory schema.

To complete step one I dumped the contents below into an LDIF file and saved it as schemamod.ldif.

dn: CN=unixUserName, CN=Schema, CN=Configuration, DC=example, DC=com
changetype: add
attributeID: 1.3.6.1.4.1.42.2.27.5.1.60
attributeSyntax: 2.5.5.3
isSingleValued: TRUE
searchFlags: 1
lDAPDisplayName: unixUserName
adminDescription: This attribute contains the object's UNIX username
objectClass: attributeSchema
oMSyntax: 27

dn: CN=unixGroupName, CN=Schema, CN=Configuration, DC=example, DC=com
changetype: add
attributeID: 1.3.6.1.4.1.42.2.27.5.1.61
attributeSyntax: 2.5.5.3
isSingleValued: TRUE
searchFlags: 1
lDAPDisplayName: unixGroupName
adminDescription: This attribute contains the object's UNIX groupname
objectClass: attributeSchema
oMSyntax: 27

dn:
changetype: modify
add: schemaUpdateNow
schemaUpdateNow: 1
-

dn: CN=unixNameInfo, CN=Schema, CN=Configuration, DC=example, DC=com
changetype: add
governsID: 1.3.6.1.4.1.42.2.27.5.2.15
lDAPDisplayName: unixNameInfo
adminDescription: Auxiliary class to store UNIX name info in AD
mayContain: unixUserName
mayContain: unixGroupName
objectClass: classSchema
objectClassCategory: 3
subClassOf: top

For the step two I logged into the AWS Management Console and navigated to the Directory Service Console. Here we can see my instance AWS Managed AD with the domain name of geekintheweeds.com.

I then clicked hyperlink on my Directory ID which takes me into the console for the geekintheweeds.com instance. Scrolling down shows a menu where a number of operations can be performed. For the purposes of this blog post, we’re going to focus on the Maintenance menu item. Here we the ability to leverage AWS Simple Notification Service (AWS SNS) to create notifications for directory changes such as health changes where a managed Domain Controller goes down. The second section is a pretty neat feature where we can snapshot the Windows AD environment to create a point-in-time copy of the directory we can restore. We’ll see this in action in a few minutes. Lastly, we have the schema extensions section.

Here I clicked the Upload and update schema button and entered selected the LDIF file and added a short description. I then clicked the Update Schema button.

If you know me you know I love to try to break stuff. If you look closely at the LDIF contents I pasted above you’ll notice I didn’t update the file with my domain name. Here the error in the LDIF has been detected and the schema modification was cancelled.

I went through made the necessary modifications to the file and tried again. The LDIF processes through and the console updates to show the schema change has been initialized.

Hitting refresh on the browser window updates the status to show Creating Snapshot. Yes folks Amazon has baked into the schema update process a snapshot of the directory provide a fallback mechanism in the event of your zombie apocalypse. The snapshot creation process will take a while.

While the snapshot process, let’s discuss what Amazon is doing behind the scenes to process the LDIF file. We first saw that it performs some light validation on the LDIF file, it then takes a snapshot of the directory, then applies to the changes to a single domain controller by selecting one as the schema master, removing it from directory replication, and applying the LDIF file using the our favorite old school tool LDIFDE.EXE. Lastly, the domain controller is added back into replication to replicate the changes to the other domain controller and complete the changes. If you’ve been administering Windows AD you’ll know this has appeared recommended best practices for schema updates over the years.

Once the process is complete the console updates to show completion of the schema installation and the creation of the snapshot.

My Experience Passing AWS Certified Cloud Practitioner Exam

Posted on August 27, 2018 by mattfeltonma

Welcome back my fellow geeks!

Today I’m going to interrupt the series on AWS Managed Microsoft AD For the past few weeks, in between writing the entries for the recent deep dive series, I’ve been preparing for the AWS Cloud Practitioner exam. I thought it would be helpful to share my experience prepping for and passing the exam.

If you’re not familiar with the AWS Certificated Cloud Practitioner exam, it’s very much an introductory exam into the Amazon Web Services’ overarching architecture and products. Amazon’s intended audience for the certification are your C-levels, sales people, and technical people who are new to the AWS stack and potentially cloud in general. It’s very much an inch deep and mile wide. For those of you who have passed your CISSP, the experience studying for it similar (although greatly scaled down content-wise) in that you need to be able to navigate the shallow end of many pools.

Some of you may be asking yourselves why I invested my time in getting an introductory certificate rather than just going for the AWS Certified Solutions Architect – Associate. The reason is my personal belief that establishing a solid foundation in a technology or product is a must. I’ve encountered too many IT professionals with a decade more of experience and a hundred certificates to their name who can’t explain the basics of the OSI model or the difference in process between digitally signing something versus encrypting it. The sign of a stellar IT professional is one who can start at the business justification for an application and walk you right down through the stack to speak to the technology standards being leveraged within the application to deliver its value. This importance in foundation is one reason I recommend every new engineer start out by taking the CompTIA A+, Network+, and Security+ exams. You won’t find exams out there that better focus on foundational concepts than CompTIA exams.

The other selling point of this exam to me was the audience it’s intended for. Who wouldn’t want to know the contents and messaging in an exam intended for the C-level? Nothing is more effective influencing the C-level than speaking the language they’re familiar with and pushing the messaging you know they’ve been exposed to.

Let me step off this soapbox and get back to my experience with the exam. 🙂

As I mentioned above I spent about two weeks preparing for the exam. My experience with the AWS stack was pretty minimal prior to that restricted to experience for my prior blogs on Azure AD and AWS integration for SSO and provisioning and Microsoft Cloud App Security integration with AWS. As you can tell from the blog, I’ve done a fair amount of public cloud solutions over the past few years, just very minimally AWS. The experience in other public cloud solutions such as Microsoft Azure and Google Cloud Platform (GCP) proved hugely helpful because the core offerings are leveraging similar modern concepts (i.e. all selling computer, network, and storage). Additionally, the experiences I’ve had over my career with lots of different infrastructure gave me the core foundation I needed to get up and running. The biggest challenge for me was really learning the names of all the different offerings, their use cases, and their capabilities that set them apart from the other vendors.

For studying materials I followed most of the recommendations from Amazon which included reviews of a number of whitepapers. I had started the official Amazon Cloud Practitioner Essentials course (which is free by the way) but didn’t find the instructors engaging enough to keep my attention. I ended up purchasing a monthly subscription to courses offered by A Cloud Guru which were absolutely stellar and engaging at a very affordable monthly price (something like $29/month). In addition to the courses I read each of the recommended whitepapers (ended reading a bunch of others as well) a few times each taking notes of key concepts and terminology. While I was studying for this exam, I also was working on my AWS deep dive which helped to reinforce the concepts by actually building out the services for my own use.

I spent a lot of time diving into the rabbit whole of products I found really interesting (RedShift) as well as reading up on concepts I’m weaker on (big data analytics, modern nosql databases, etc). That rabbit hole consisted of reading blogs, Wikipedia, and standards to better understand the technical concepts. Anything I felt would be worthwhile I captured in my notes. Once I had a good 15-20 pages of notes (sorry all paper this time around), I grabbed the key concepts I wanted to focus on and created flash cards. I studying the deck of 200 or so flash cards each night as well as re-reading sections of the whitepapers I wanted to familiarize myself with.

For practice exams I used the practice questions Amazon provides as well as the quizzes from A Cloud Guru. I found the questions on the actual exam more challenging, but the practice question and quizzes were helpful to getting into the right mindset. The A Cloud Guru courses probably covered a good 85-90% of the material, but I wouldn’t recommend using it was a sole source of study, you need to read those whitepapers multiple times over. You also need to do some serious hands on because some of the questions do ask you very basic questions about how you do things in the AWS Management Console.

Overall it was a well done exam. I learned a bunch about the AWS product offerings, the capabilities that set AWS apart from the rest of the industry, and gained a ton of good insight into general cloud architecture and design from the whitepapers (which are really well done). I’d highly recommend the exam to anyone who has anything to do with the cloud, whether you’re using AWS or not. You’ll gain some great insight into cloud architecture best practices as well seeing modern technology concepts put in action.

I’ll be back with the next entry in my AWS Managed Microsoft AD series later this week. Have a great week and thanks for reading!

Journey Of The Geek

The chronicles of a Bostonian tech geek navigating through life, technology, and general geekiness.

Tag Archives: amazon

Deep Dive into Azure AD and AWS SSO Integration – Part 2

Capturing and Visualizing Office 365 Security Logs – Part 2

AWS Managed Microsoft AD Deep Dive Part 6 – Schema Modifications

My Experience Passing AWS Certified Cloud Practitioner Exam