Secure Python console apps with Azure AD

Last week during our regular stream, we looked at how to secure a Python console (daemon/service) app with Azure Active Directory and acquire a token to call a downstream/upstream API. If you want to see how we build it live on Twitch, you can watch our video on YouTube

Create the Azure AD App Registrations

Let's shortcut this whole thing and use the awesome .NET Interactive Notebook to wire up and configure our 2 App Registrations. The Notebook is checked in the GitHub repo. You'll need to setup your VS Code to run it but it should be pretty explanatory and provides you with the steps that walk you through the process. In summary, the notebook will:

Create an API app registration

Assign an Application Role

Create a Service Principal for the API App

Create a client app registration for our Python console app

Create a client secret for authenticating the console app to AAD (Client Credentials Flow)

The .NET API

For the purpose of this blog, I created an .NET 5 API that retrieves data from the OpenWeatherMap API. You can create a free account too or change the code to return any other data (the data is not the point here)

With regards to the Weather API, I find it a great service for working with real APIs and data, even for data purposes.

The API uses the latest Microsoft.Identity.Web .NET library to wire up the authentication and authorization. There are 4 main components

the AAD configuration settings

the API authentication settings in the middleware

a policy to handle scope and role-based authorization

applying the policy to the APi controller

I've added a CodeTour that walks you through the steps.

Let's go....

Open the appsettings.json and add the following settings:

"AzureAd": {
    "Instance": "https://login.microsoftonline.com/",
    "ClientId": "<your client id>",
    "Domain": "<your tenant name>.onmicrosoft.com",
    "TenantId": "<your tenant id>"
  },

Install the necessary NuGet package:

dotnet add package microsoft.identity.web

Open startup.cs and update the ConfigureServices() method with the following code:

var ScopeClaim = "http://schemas.microsoft.com/identity/claims/scope";
var ExpectedRole = "access_as_application";
services.AddMicrosoftIdentityWebApiAuthentication(Configuration);
services.AddAuthorization(options => options.AddPolicy(
    "AllowedAccess",
    policyBuilder => policyBuilder.RequireAssertion(
          context 
          => context.User.IsInRole(ExpectedRole)
          || context.User.HasClaim(ScopeClaim,"access_as_user"))
    ));
services.AddControllers();
services.AddCors(options =>
    {
        options.AddPolicy(name: MyAllowSpecificOrigins,
               builder =>
               {
                  builder
                       .AllowAnyOrigin()
                       .AllowAnyMethod()
                       .AllowAnyHeader();
               });
    });
services.AddSwaggerGen(c =>
{
    c.SwaggerDoc("v1", new OpenApiInfo { Title = "API", Version = "v1" });
});

You will need to change the following:

policy name (optional)

the ExpectedRole (needs to match the role name you defined in the .NET Notebook)

the CORS settings (right now it's WIDE OPEN - DON't be me)

In the Configure() method, ensure you add app.UseAuthentication() before the UseAuthorization() call

Finally, open the WeatherForecastController.cs and add the following action:

[HttpGet]
[Authorize(Policy="AllowedAccess")]
public async Task<string> Get(string city)
{
    var context = this.HttpContext;
    var url = $"http://api.openweathermap.org/data/2.5/weather?q={city}&appid={configuration["WeatherApiKey"]}";
    var client = new HttpClient();
    var response = await client.GetStringAsync(url);

    return response;
}

At this point we have all the authentication and authorization wired up. However, accessing the 3rd party Weather API requires a key. We have a few options:

store it in .NET Secrets (only available locally and stored in clear text under the user's profile)

store it in env variables (clear text so not secure and only available locally)

add it in appsettings.json (a big No-No as this is the least secure options)

use Azure Key Vault and lock everything down as it should be.

Adding Azure Key Vault to our API

Key Vault is by far the best way to store sensitive information that needs to be used by our application. Things like connection strings, passwords, secrets, API keys etc are perfectly suited for Key Vault Secrets.

Let's create a Key Vault. Open the Azure CLI, then type the following

az login
az group create --name "<your-resource-group-name>" -l "<desired region>"
az keyvault create --name "<your-unique-keyvault-name>" --resource-group "<your-resource-group-name>" --location "<desired region>"

To authenticate and use Azure Key Vault locally, we will create an Azure AD Service Principal. When running in production, we will be making use of Azure Managed Identities. This setup allows our code to run anywhere without making config or code changes!

In the Azure CLI, create a service principal and give it the appropriate role and scope

az ad sp create-for-rbac --role Reader --scopes <your Key Vault Resource Id>

Next, make sure to sign in with this user in the Azure CLI

az login --service-principal -u http://<your SP Name> -p <your SP password> --tenant <your Tenant Id>

Finally, we need to ensure that this Service Principal account has the right Access Policy in Key Vault. We can also use the Azure CLI for this. You need to use the following command doesn't work:

az keyvault set-policy -n <your KV name> --secret-permissions get list --object-id <your SP **app  id**>

NOTE: the docs say that you need to use the object id of the service principal, but I was unable to get it working. it worked as soon as I changed the command to use the **App ID(( of the service principal

You'll need to run this command as an Azure Contributor/Owner

If you want to do this in the portal, go to Key Vault -> Access Policy and select Add Access Policy. Select Secrets, List and Get permissions and add the SP in question. Don't forget to press the Save button as the policy won't take effect until you do so.

Integrate Azure Key Vault in the .NET API

ASP.NET 5 has a configuration extension that works directly with Azure Key Vault via the Azure SDKs. Open the terminal and add the following NuGet package

dotnet add package Azure.Extensions.AspNetCore.Configuration.Secrets

Next, open Program.cs and update the CreateHostBuilder() method with the following code:

public static IHostBuilder CreateHostBuilder(string[] args) =>
            Host.CreateDefaultBuilder(args)
                .ConfigureAppConfiguration((context, config) =>
                {
                        var builtConfig = config.Build();
                        config.AddAzureKeyVault( new Uri("https://cm-identity-kv.vault.azure.net"),
                            //new DefaultAzureCredential());
                            new ChainedTokenCredential(
                                new AzureCliCredential(),
                                new ManagedIdentityCredential()
                        )); 
                })
                .ConfigureWebHostDefaults(webBuilder =>
                {
                    webBuilder.UseStartup<Startup>();
                    webBuilder.UseUrls("http://localhost:8080");
                });

This code will look for settings in appsettings.json and try to map them to Azure Key Vault secrets. This means that we need to add an empy setting in our code. Open appsettings.json and add this: "WeatherApiKey": ""

In Azure Key Vault, we need to create a new Secret with the same name and the actual OpenWeatherMap api key as the value. As long as these two match, the code will be able to resolve the settings and populate ASP.NET Configuration object.

If the permissions or anything else is not right, the API app will throw an exception at startup and won't be able to continue until you resolve any outstanding issues.

Create a Python Console (Daemon) app to seurely call our API

The Python console app will try to access the API unattended. This means that the authentication to Azure AD will happen without user intervention. We will make use of the Client Credential flow (OAuth2) to achieve this. Microsoft Identity provides an official library for Python: MSAL and we'll make use of it to acquire an access token to call our API.

Let's get coding.

First create a requirements.txt file and add the following dependencies:

msal>=1.12.0
requests>=2.25.1
azure-identity>=1.6.0
azure-keyvault-secrets>=4.3.0

Next, create a new file to store our application settings config.json. Add the following code and populate it with the values you got from running Client App registration using the .NET Notebook.

{
    "authority": "https://login.microsoftonline.com/<your tenant id>",
    "client_id": "<your client id>",
    "scope": ["api://855dac46-661b-4463-97cf-d57a190bf2ed/.default"],
    "vault_url": "https://<your vault name>.vault.azure.net"
}

Notice the scope here. We are using the App Role configured in our App Registration but instead using the value as is, we replace the actual role name with .default. This is necessary since we are using the Client Credential flow and, therefore, there is no way to consent to permissions. You can read all about .default here.

Now we can write some code. Create a console.py file and add the following code:

import json

import requests
import msal
from azure.identity import ChainedTokenCredential, AzureCliCredential, ManagedIdentityCredential
from azure.keyvault.secrets import SecretClient

jsondata = open("config.json","r")
config = json.load(jsondata)

credential = ChainedTokenCredential(AzureCliCredential(),ManagedIdentityCredential())
secret_client = SecretClient(config["vault_url"], credential=credential)
aad_client_secret = secret_client.get_secret("AadClientSecret")

app = msal.ConfidentialClientApplication(
    client_id=config["client_id"],
    client_credential=aad_client_secret.value,
    authority=config["authority"],
)

result = None

result = app.acquire_token_silent(config["scope"], account=None)

if not result:
    result = app.acquire_token_for_client(scopes=config["scope"])

if "error" in result:
   print(result["error_description"])

if "access_token" in result:
    session = requests.sessions.Session()
    session.headers.update({'Authorization': f'Bearer {result["access_token"]}'})
    response = session.get("http://localhost:8080/weatherforecast?city=London")
    if response.status_code == 200 :
        print(response.content)
    else:
        print(f'Request failed. Response code: {response.status_code}, reason: {response.reason}')

We first use the Azure SDK to authenticate and retrieve the Azure AD App Registration client secret from Key Vault. We then instantiate an MSAL client to acquire an access token from Azure AD and call our protected API.

We can now run our code end to end. First, spin up the API with dotnet run. Then set up the Python console app and run it. Be aware that the instructions for setting up the virtual environment are different across OS'es. On Windows, open a console and type:

py -m venv .venv
.venv/scripts/activate
pip install -r requirements.txt

We can now run and call the Python code with:

py console.exe

If all's worked as expected, you should be presented with the following:

Success!

Show me the codez

You can find the full source code on GitHub

Summary

This blog post was special for many reasons. First, it's the first time we made use of App Roles. That's because console applications calling APIs securely can't make use of delegated permissions. We also had to use the special /.default scope due to the fact that that there is no user to interactively consent to the required permissions. Finally, since we made use of the Client Credential flow, we added Azure Key Vault to secure sensitive information.

At a later time, we'll see how to use Azure KeyVault to create and store a certificate instead of a credential to authenticate against Azure AD and add another layer of security to our solution.