Setting up an Azure Integration

The Azure integration uses DefaultAzureCredential for authentication, which supports multiple authentication methods. This guide focuses on setting up authentication using Azure CLI and Service Principal, which is recommended for production use.

Set up a Service Principal

Syncing from single subscription

You will need to create a service principal for the plugin to use:

1. Install the Azure CLI if you haven't already

2. Open your terminal and run:

# Login to Azure
az login

1. Create a service principal with a Reader access:

# Register the security provider
az provider register --namespace 'Microsoft.Security'

# Create a service principal and grant Reader access
az ad sp create-for-rbac --name cloudquery-sp \
          --scopes /subscriptions/{subscription-id} --role Reader

The command will output credentials in the following format:

{
  "appId": "YOUR_AZURE_CLIENT_ID",
  "displayName": "cloudquery-sp",
  "password": "YOUR_AZURE_CLIENT_SECRET",
  "tenant": "YOUR_AZURE_TENANT_ID"
}

1. Save these credentials - you'll need them in the next step.

Syncing from multiple subscriptions

There are two main approaches to sync data from multiple Azure subscriptions:

1. Management Group Level Access

The most dynamic approach is to scope the service principal at the Management Group level. This allows automatic discovery of all subscriptions under the specified Management Group(s), including any new subscriptions added later.

To create a service principal with Management Group access:

# Register the security provider
az provider register --namespace 'Microsoft.Security'

# Create service principal with Management Group access
az ad sp create-for-rbac --name cloudquery-sp-root-1 \
  --scopes /providers/Microsoft.Management/managementGroups/{management-group-name} \
  --role Reader

1. Specific Subscriptions Access

If you prefer to limit access to specific subscriptions, you can list them explicitly in the scope. This command grants access to all currently accessible subscriptions:

# Register the security provider
az provider register --namespace 'Microsoft.Security'

# Create service principal with access to specific subscriptions
az ad sp create-for-rbac --name cloudquery-sp \
  --scopes $(az account subscription list --query "[].id" -o tsv --only-show-errors | xargs) \
  --role Reader

When using the specific subscriptions approach, the service principal won't automatically get access to newly added subscriptions. You'll need to run the command again to include new subscriptions.

Setting up a Azure Integration

1. In CloudQuery Platform, go to Data Pipelines → Integrations. Click Create Integration and type Azure to find the Azure integration.

1. Copy-paste the saved credentials into their respective fields:

Azure authentication fields

1. Optionally, configure the subscriptions to sync or skip and choose the Azure cloud to sync from.

Additional Azure integration configuration

1. Click Continue to proceed with service data selection

2. Select services to sync from. Selecting a service will create and sync all tables related to the Azure service. For the list of individual tables, see Azure Integration Documentation.

1. Click Test Connection and save to verify the configuration and save this integration.

Next Steps

With your Azure integration created, you can now proceed to use it in a new sync. This will give you the opportunity to specify when your Azure sync should be run, and to which destination databases.