Hello geeks!

Over the past few weeks I’ve been working with a customer who has adopted the CIS (Center for Internet Security) controls framework.  CIS publishes a set of best practices and configurations called benchmarks for commonly used systems .  As you would expect there is a set of benchmarks for Microsoft Azure.  Implementing, enforcing, and auditing for compliance with the benchmarks can be a challenge.  Thankfully, this is where Azure Policy comes to the rescue.

Azure Policy works by evaluating the properties of resources (management plane right now minus a few exceptions) created in Azure either during deployment or for resources that have already been deployed.  This means you can stop a user from deploying a non-compliant resource vs addressing it after the fact.  This feature is value added for organizations that haven’t reached that very mature level of DevOps where all infrastructure is codified and pushed through a CI/CD pipeline that performs validation tests before deployment.

Policies are created in JSON format and contain five elements.  For the purposes of this blog post, I’ll be focusing on the policy rule element.  The other elements are straightforward and described fully in the official documentation.  The policy rule contains two sub elements, a logical evaluation and effect.  The logical evaluation uses simple if-then logic.  The if block contains one or more conditions with optional logical operators.  The if block will be where you spend much of your time (and more than likely frustration).

I would liken the challenge of learning how to construct working Azure Policy to the challenge presented writing good AWS IAM Policies.  The initial learning curve is high, but once you get a hang of it, you can craft works of art.  Unfortunately, unlike AWS IAM Policy, there are some odd quirks with Azure Policy right now that are either under documented or not documented.  Additionally, given how much newer Azure Policy is, there aren’t a ton of examples to draw from online for more complicated policies.

This brings us to the purpose of this blog.  While being very very very far from an expert (more like I’m barely passable) on Azure Policy, I have learned some valuable lessons from the past few weeks that I’ve been struggling through writing custom policies.  These are the lessons I want to pass on in hopes they’ll make your journey a bit easier.

• • Just because a resource alias exists, it doesn’t mean you can use it in a policy
    When you are crafting your conditions you’ll use fields which map to properties of Azure resources and describe their state.  There are a selection of fields that are supported, but one you’ll probably use often is the property alias.   You can pull a listing of property aliases using PowerShell, CLI, or the REST API.  Be prepared to format the output because some namespaces have a ton of properties.  I threw together a Python solution to pull the namespaces into a more consumable format.If you are using an alias that is listed but your Policy fails to do what you want it to do, it could be that while the alias exists, it’s not accessible by policy during an evaluation.  If the property belongs to a namespace that contains a property that is sensitive (like a secret) it will more than likely not be accessibly by Policy and hence won’t be caught.  The general rule I follow is if the namespace’s properties aren’t accessible with the Reader Azure RBAC role, policy evaluations won’t pick them up.A good example of this is the authsettings namespace under the Microsoft.Web/sites/config.  Say for example you wanted to check to see if the Web App was using FaceBook as an identity provider, you wouldn’t be able to use policy to check whether or not facebookAppId was populated.

• • Resource Explorer, Azure ARM Template Reference, and Azure REST API Reference are your friends, use them
    When you’re putting together a new policy make sure to use Azure Resource Explorer, Azure ARM Template Reference, and Azure REST API Reference.  The ARM Template Reference is a great tool to use when you are crafting a new policy because it will give you an idea of the schema of the resource you’ll be evaluating.  The Azure REST API Reference is useful when the description of a property is less than stellar in the ARM Template Reference (happens a lot).  Finally, the Azure Resource Explorer is an absolute must when troubleshooting a policy.A peer and I ran into a quirk when authoring a policy to evaluate the runtime of an Azure Web App.  In this instance Azure Web Apps running PHP on Windows were populating the PHP runtime in the phpVersion property while Linux was populating it in the linuxFxVersion property.  This meant we had to include additional logic in the policy to detect the runtimes based on the OS.  Without using Resource Explorer we would never have figured that out.
    

• • Use on-demand evaluations when building new policies
    Azure Policy evaluations are triggered based upon the set of the events described in this link.  The short of it is unless you want to wait 30 minutes after modifying or assigning a new policy, you’ll want to trigger an on-demand evaluation.  At this time this can only be done with a call to an Azure REST API endpoint.  I’m unaware of a built-in method to do this with Azure CLI or PowerShell.Since I have a lot of love for my fellow geeks, I put together a Python solution you can use to trigger evaluation.  Evaluations take anywhere between 5-10 minutes.  It seems like this takes longer the more policies you have, but that could simply be in my head.

• • RTFM.
    Seriously, read the public documentation.  Don’t jump into this service without spending an hour reading the documentation.  You’ll waste hours and hours of time smashing your head against the keyboard.  Specifically, read through this page to understand how processing across arrays works.  When you first start playing with Azure Policy, you’ll come across policies with double-negatives that will confuse the hell out of you.  Read that link and walk through policies like this one.  You can thank me later.

• • Explore the samples and experiment with them.
    Microsoft has published a fair amount of sample policies in the Azure Policy repo, the built-in policies and initiatives included in the Azure Portal, and the policy samples in the documentation.  I’ve thrown together a few myself and am working on others, so feel free to use them as you please.

Hope the above helps some of you on your journey to learning Azure Policy.  It’s a tool with a ton of potential and will no doubt improve over time.  One of the best ways to help it evolve is to contribute.  If you have some kick ass policies, submit them to get them published to the Azure Policy repo and to give back to the wider community.

Have a great week folks!