Provisioning Azure Cosmos DB using Powershell

Azure Cosmos database is one of the NoSQL database that is available in Microsoft Azure cloud platform. Azure Cosmos db is getting more popular compared to other NoSQL databases in the cloud market so many organisations started using Cosmos database with their organization. When we started incorporating Cosmos database into our project we provisioned it manually and continued with the development.

When we started working with many different environments and different projects it became a complex and time consuming task for us to provision Cosmos database to all the development environments and carry it through out the deployment pipeline. At this stage I thought of writing set of scripts that will provision the Cosmos database with respect to our standards. I choose to write all the scripts using powershell so that I can hook these scripts in Octopus. Octopus is a deployment application which allows to seamlessly deploy anything to multiple environments setup through a pipeline. I can hook the scripts into octopus project to enable continuous integration.

Before I start explaining how to use the scripts there are some pre-requisites , we need to ensure all these are passed before the scripts gets executed.

  • A subscription exists in Azure Cloud
  • The account or the person execute the scripts should have contributor role to the subscription

I have split the Cosmos database deployment into four different sections as below

  • Provision Resource Group
  • Provision Cosmos Database Account
  • Provision Cosmos Database
  • Provision Cosmos Database Collection

This allowed me to configure and troubleshoot the steps without having hassles of debugging one full script. I used powershell script for automation instead of ARM templates. As of today ARM templates has capability to provision resource group and cosmos db account however it doesn’t have capability to create cosmos database and collection.

How to use the scripts

  • Ensure you provide proper values for the variables. All the variables values are required, so ensure you check the name and provide the details. When you run the script it will prompt to login into azure portal so you can connect with the account that has access to create resource in the cloud.
  • You can run the script multiple times , if the resource exists the script wont provision it or doesn’t modify it.
Provision Resource Group
<###################### Update Variable values ############################>
$SubscriptionName = "Free Trial"
$ResourceGroupName = "rg-sql-articles-sin-devtest"
$Location = "South India"
<##########################################################################>
<###################### Dont update anything below ########################>
## When requested login with Azure credentials


Login-AzureRmAccount

Write-Host "SubscriptionName      : `"$SubscriptionName`""
Write-Host "ResourceGroupName     : `"$ResourceGroupName`""
Write-Host "Location              : `"$Location`""

function Set-AzureResourceGroup {

    param
    (
        [Parameter(Mandatory = $true)]
        [ValidateNotNullOrEmpty()]
        [string]$ResourceGroupName,
        [Parameter(Mandatory = $true)]
        [ValidateNotNullOrEmpty()]
        [string]$Location
    )

    $SubscriptionExists=Get-AzureRmSubscription -SubscriptionName $SubscriptionName 

    IF ($SubscriptionExists.Name -eq $SubscriptionName)
    {
        $subscription=Select-AzureRmSubscription -SubscriptionName $SubscriptionName
    }
    else
    {
        throw "You dont have access to [$SubscriptionName] subscription"
    }

    try
    {
        $CheckResourceGroupExists = Get-AzureRmResourceGroup -Name $ResourceGroupName -ErrorAction SilentlyContinue
    }
    catch
    { 
        if ($Error[0].Exception.Message -match "Provided resource group does not exist") 
        { 
            Write-Host ("Resource Group {0} not found, creating" -f $ResourceGroupName)
        }
        else
        {
            throw ("Could not determine if resource group already exists`n{0}" -f $Error[0])
        }
    }

    if ($null -eq $CheckResourceGroupExists)
    {
        New-AzureRmResourceGroup -Name $ResourceGroupName -Location $Location
    }

    else
    {
        Write-Host ("Resource Group {0} already exists" -f $ResourceGroupName)
    }
}

Set-AzureResourceGroup -ResourceGroupName $ResourceGroupName -Location $Location

Provision Cosmos Database Account
<###################### Update Variable values ############################>
$SubscriptionName = "Free Trial"
$ResourceGroupName = "rg-sql-articles-sin-devtest"
$CosmosDbAccountName = "sql-articles-cosmos-dev-account"
$Location = "South India"
$DocDbResourceGroupLocationSeconday = "Central India"
$FirewallIPAddress = "10.0.0.1"
<##########################################################################>
<###################### Dont update anything below ########################>
## When requested login with Azure credentials

Login-AzureRmAccount
    
    Write-Host "SubscriptionName                 : `"$SubscriptionName`""
    Write-Host "ResourceGroupName                : `"$ResourceGroupName`""
    Write-Host "ResourceGroupLocation            : `"$Location`""
    Write-Host "ResourceGroupLocationSecondary   : `"$DocDbResourceGroupLocationSeconday`""
    Write-Host "CosmosdbAccountName              : `"$CosmosDbAccountName`""
    Write-Host "FirewallIPAddress                : `"$FirewallIPAddress`""

    Register-AzureRmResourceProvider -ProviderNamespace Microsoft.DocumentDB

    $SubscriptionExists=Get-AzureRmSubscription -SubscriptionName $SubscriptionName 

    IF ($SubscriptionExists.Name -eq $SubscriptionName)
    {
        $subscription=Select-AzureRmSubscription -SubscriptionName $SubscriptionName
    }
    else
    {
        throw "You dont have access to [$SubscriptionName] subscription"
    }

function Test-ResourceGroupExist{
    param(
        [Parameter(Mandatory = $true)]
        [ValidateNotNullOrEmpty()]
        [string]$ResourceGroupName
    )

    try
    {
        [bool]$exists = $false
        Get-AzureRmResourceGroup -Name $ResourceGroupName    
        $exists = $true
    }
    catch
    {
        if ($Error[0].Exception.Message -eq 'Provided resource group does not exist.')
        {
            return $exists
        }
        else
        {
            throw $_
        }
    }

    return $exists

}

function Test-ValidIPAddress {
    param
    (
        [Parameter(Mandatory = $true)]
        [ValidateNotNullOrEmpty()]
        [string]$IpAddress
    )

    

    if ($IpAddress -cnotmatch '^(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)$')
    {
        throw "`"$IpAddress`" is not a valid IpAddress."
    }
}

function Test-CosmosDbAccountName {
    param
    (
        [Parameter(Mandatory = $true)]
        [ValidateNotNullOrEmpty()]
        [string]$CosmosdbAccountName
    )

    

    if ($CosmosdbAccountName -cnotmatch '^([a-z0-9]+)([-]*[a-z0-9]*)*[^-]$')
    {
        throw "Servername failed naming rules: `"$CosmosdbAccountName`" cannot be empty or null. It can only be made up of lowercase letters 'a'-'z', the numbers 0-9 and the hyphen. The hyphen may not lead or trail in the name."
    }

    if ($CosmosdbAccountName.Length -gt 63)
    {
        throw "$CosmosdbAccountName exceeds max name length of 63 chars."
    }

}

function Test-CosmosAccountExistence{

  param 
  (
    [Parameter(Mandatory = $true)]
        [ValidateNotNullOrEmpty()]
        [string]$CosmosdbAccountName,
        [Parameter(Mandatory = $true)]
        [ValidateNotNullOrEmpty()]
        [string]$ResourceGroupName

  )

  try {
      [bool]$cosmosdbexists = $false
      $cosmosdbexists = Get-AzureRmResource -ResourceName $CosmosdbAccountName -ResourceGroupName $ResourceGroupName
            return $cosmosdbexists
  } catch {

    if ($Error[0].Exception.Message -like "*NotFound*") 
    {
      Write-Host "Resource Group '$ResourceGroupName' not found."
    } 
    else 
    {
      $Error[0]
    }
  }
}

function New-CosmosAccount {

  
  param 
  (
    [Parameter(Mandatory = $true)]
        [ValidateNotNullOrEmpty()]
        [string]$CosmosdbAccountName,
        [Parameter(Mandatory = $true)]
        [ValidateNotNullOrEmpty()]
        [string]$ResourceGroupName,
        [Parameter(Mandatory = $true)]
        [ValidateNotNullOrEmpty()]
        [string]$ResourceGroupLocation,
        [Parameter(Mandatory = $true)]
        [ValidateNotNullOrEmpty()]
        [string]$ResourceGroupLocationSecondary,
        [Parameter(Mandatory = $true)]
        [ValidateNotNullOrEmpty()]
        [string]$IpAddress


  )

    $locations = @(@{"locationName"="$ResourceGroupLocation"; 
                     "failoverPriority"=0}, 
                   @{"locationName"="$ResourceGroupLocationSecondary"; 
                      "failoverPriority"=1})

    $consistencyPolicy = @{"defaultConsistencyLevel"="Session";}

    $DBProperties = @{"databaseAccountOfferType"="Standard"; 
                              "locations"=$locations; 
                              "consistencyPolicy"=$consistencyPolicy; 
                              "ipRangeFilter"=$iprangefilter}

    try {
        New-AzureRmResource -ResourceType "Microsoft.DocumentDb/databaseAccounts" `
                        -ApiVersion "2015-04-08" `
                        -ResourceGroupName $resourceGroupName `
                        -Location $resourceGroupLocation `
                        -Name $CosmosdbAccountName `
                        -PropertyObject $DBProperties `
              -Force
    }
    catch
    {
        throw $_
    }
}


function New-CreateCosmosAccount
{

  param 
  (
    [Parameter(Mandatory = $true)]
        [ValidateNotNullOrEmpty()]
        [string]$CosmosdbAccountName,
        [Parameter(Mandatory = $true)]
        [ValidateNotNullOrEmpty()]
        [string]$ResourceGroupName,
        [Parameter(Mandatory = $true)]
        [ValidateNotNullOrEmpty()]
        [string]$ResourceGroupLocation,
        [Parameter(Mandatory = $true)]
        [ValidateNotNullOrEmpty()]
        [string]$ResourceGroupLocationSecondary,
        [Parameter(Mandatory = $true)]
        [ValidateNotNullOrEmpty()]
        [string]$IpAddress

  )


    Test-ValidIPAddress -IpAddress $IpAddress


     try
    {
        $CheckResourceGroupExists = Test-ResourceGroupExist -ResourceGroupName $ResourceGroupName

    }
    catch
    { 
        throw $_
    }

    if ($CheckResourceGroupExists -eq $false)
    {
        throw ("Resource group `"$ResourceGroupName`" not found.")
    }


    try
    {
        Test-CosmosDbAccountName -CosmosdbAccountName $CosmosdbAccountName
    }
    catch
    {
        throw
    }


    try
    {
        $existingAzureRmCosmosDbAccount = Test-CosmosAccountExistence -CosmosdbAccountName $CosmosdbAccountName -ResourceGroupName $ResourceGroupName
    }
    catch
    {
        if ($Error[0].Exception.Message -match "ResourceNotFound: The Resource `'Microsoft.Sql/servers/$CosmosdbAccountName`' under resource group `'$ResourceGroupName`' was not found.") 
        { 
            Write-Host ('SqlAzure Server {0} not found.' -f $CosmosdbAccountName)
        }
        else
        {
            throw ("Could not determine if CosmosDb Account already exists`n{0}" -f $Error[0])
        }
    }


        try
    {
        if ($false -eq $existingAzureRmCosmosDbAccount)
        {
            New-CosmosAccount -CosmosdbAccountName $CosmosdbAccountName -ResourceGroupName $ResourceGroupName -ResourceGroupLocation $ResourceGroupLocation -ResourceGroupLocationSecondary $ResourceGroupLocationSecondary -IpAddress $IpAddress -
        }
        else
        {
            Write-Host 'The CosmosDb account "'$CosmosdbAccountName'" already exists, Use Set-AzureRmSqlServer cmdlet to update the settings.'
        }
    }
    catch
    {
        throw
    }

}

New-CreateCosmosAccount -ResourceGroupName $ResourceGroupName -ResourceGroupLocation $Location -CosmosdbAccountName $CosmosDbAccountName -ResourceGroupLocationSecondary $DocDbResourceGroupLocationSeconday -IpAddress $FirewallIPAddress
Provision Cosmos Database
<###################### Update Variable values ############################>
$SubscriptionName = "Free Trial"
$ResourceGroupName = "rg-sql-articles-sin-devtest"
$CosmosDbAccountName = "sql-articles-cosmos-dev-account"
$CosmosDBApiVersion = "2017-02-22"
$CosmosDbEndPoint = "https://$CosmosDbAccountName.documents.azure.com:443/"
$CosmosDbDatabaseName = "sql-articles-cosmos-db"
<##########################################################################>
<###################### Dont update anything below ########################>
## When requested login with Azure credentials

Login-AzureRmAccount

    Write-Host "ResourceGroupName    : `"$ResourceGroupName`""
    Write-Host "DocumentDBApiVersion : `"$CosmosDBApiVersion`""
    Write-Host "EndPoint             : `"$CosmosDBEndPoint`""
    Write-Host "DatabaseName         : `"$CosmosDbDatabaseName`""
    Write-Host "CosmosdbAccountName  : `"$CosmosDbAccountName`""


;Add-Type -AssemblyName System.Web

# generate authorization key
Function Generate-MasterKeyAuthorizationSignature
{
  [CmdletBinding()]
  Param
  (
    [Parameter(Mandatory=$true)][String]$verb,
    [Parameter(Mandatory=$false)][String]$resourceLink,
    [Parameter(Mandatory=$false)][String]$resourceType,
    [Parameter(Mandatory=$true)][String]$dateTime,
    [Parameter(Mandatory=$true)][String]$key,
    [Parameter(Mandatory=$true)][String]$keyType,
    [Parameter(Mandatory=$true)][String]$tokenVersion
  )

  $hmacSha256 = New-Object System.Security.Cryptography.HMACSHA256
  $hmacSha256.Key = [System.Convert]::FromBase64String($key)

  If ($resourceLink -eq $resourceType) {
    $resourceLink = ""
  }

  $payLoad = "$($verb.ToLowerInvariant())`n$($resourceType.ToLowerInvariant())`n$resourceLink`n$($dateTime.ToLowerInvariant())`n`n"
  $hashPayLoad = $hmacSha256.ComputeHash([System.Text.Encoding]::UTF8.GetBytes($payLoad))
  $signature = [System.Convert]::ToBase64String($hashPayLoad);
  [System.Web.HttpUtility]::UrlEncode("type=$keyType&ver=$tokenVersion&sig=$signature")
}

Function Get-CosmosDb
{
  [CmdletBinding()]
  Param
  (
        [Parameter(Mandatory=$true)][String]$DocumentDBApi,
    [Parameter(Mandatory=$true)][String]$EndPoint,
    [Parameter(Mandatory=$true)][String]$MasterKey
  )

  $Verb = "GET"
  $ResourceType = "dbs";
  $ResourceLink = "dbs"

  $dateTime = [DateTime]::UtcNow.ToString("r")
  $authHeader = Generate-MasterKeyAuthorizationSignature -verb $Verb -resourceLink $ResourceLink -resourceType $ResourceType -key $MasterKey -keyType "master" -tokenVersion "1.0" -dateTime $dateTime
  $header = @{authorization=$authHeader;"x-ms-documentdb-isquery"="True";"x-ms-version"=$DocumentDBApi;"x-ms-date"=$dateTime}
  $contentType= "application/query+json"
  $queryUri = "$EndPoint$ResourceLink"

  $result = Invoke-RestMethod -Method $Verb -ContentType $contentType -Uri $queryUri -Headers $header

  return $result.Databases
}


Function Create-CosmosDb
{
  [CmdletBinding()]
  Param
  (
        [Parameter(Mandatory=$true)][String]$DocumentDBApi,
    [Parameter(Mandatory=$true)][String]$EndPoint,
    [Parameter(Mandatory=$true)][String]$MasterKey,
        [Parameter(Mandatory=$true)][String]$DatabaseName
  )

  $Verb = "POST"
  $ResourceType = "dbs";
  $ResourceLink = "dbs"
    $body = '{
                "id":"'+$DatabaseName+'"
             }'

  $dateTime = [DateTime]::UtcNow.ToString("r")
  $authHeader = Generate-MasterKeyAuthorizationSignature -verb $Verb -resourceLink $ResourceLink -resourceType $ResourceType -key $MasterKey -keyType "master" -tokenVersion "1.0" -dateTime $dateTime
  $header = @{authorization=$authHeader;"x-ms-version"=$DocumentDBApi;"x-ms-date"=$dateTime}
  $contentType= "application/json"
  $queryUri = "$EndPoint$ResourceLink"

  
    try 
    {
        $result = Invoke-RestMethod -Method $Verb -ContentType $contentType -Uri $queryUri -Headers $header -Body $body
        $result 
    } 
    catch 
    {
        Write-Host "ErrorStatusCode:" $_.Exception.Response.StatusCode.value__ 
        Write-Host "ErrorStatusDescription:" $_.Exception.Response.StatusDescription
    }
}

Function Get-PrimaryKey
{

    [CmdletBinding()]
  Param
  (
        [Parameter(Mandatory=$true)][String]$DocumentDBApi,
    [Parameter(Mandatory=$true)][String]$ResourceGroupName,
        [Parameter(Mandatory=$true)][String]$CosmosdbAccountName
    )

    try
    {
  
        $keys=Invoke-AzureRmResourceAction -Action listKeys -ResourceType "Microsoft.DocumentDb/databaseAccounts" -ApiVersion $DocumentDBApi -ResourceGroupName $ResourceGroupName -Name $CosmosdbAccountName -Force
        $connectionKey=$keys[0].primaryMasterKey
        return $connectionKey
    }
    catch 
    {
        Write-Host "ErrorStatusDescription:" $_
    }
}


Function New-ProvisionCosmosDb
{

    Param
  (
        [Parameter(Mandatory=$true)][String]$DocumentDBApi,
    [Parameter(Mandatory=$true)][String]$CosmosDBEndPoint,
        [Parameter(Mandatory=$true)][String]$DatabaseName,
        [Parameter(Mandatory=$true)][String]$ResourceGroupName,
        [Parameter(Mandatory=$true)][String]$CosmosdbAccountName
  )



    try
    {
        $MasterKey = Get-PrimaryKey -DocumentDBApi "2016-03-31" -ResourceGroupName $ResourceGroupName -CosmosdbAccountName $CosmosdbAccountName 
        IF ($MasterKey -eq "" )
        {
            throw "Unable to Retrieve MasterKey data for [$CosmosdbAccountName]."
        }
    }
    catch
    {
        Write-Host ($_)
        throw $_
    }

    try
    {
        $DBExistence =  Get-CosmosDb -EndPoint $CosmosDBEndPoint -MasterKey $MasterKey -DocumentDBApi $DocumentDBApi | WHERE id -EQ $DatabaseName
       
    if ($DBExistence.id -eq $DatabaseName) 
    {
      Write-Host "Cosmos database [$DatabaseName] already exists in [$CosmosdbAccountName] account"
    } 

    
    }
    catch
    {
        Write-Host ($_)
        throw $_
    }
    
    try
    {
        if ($DBExistence.id -ne $DatabaseName) 
        {
            Create-CosmosDb -EndPoint $CosmosDBEndPoint -MasterKey $MasterKey -DatabaseName $DatabaseName -DocumentDBApi $DocumentDBApi
        }
    }
    catch
    {
        Write-Host ($_)
        throw $_
    }
}

    $SubscriptionExists=Get-AzureRmSubscription -SubscriptionName $SubscriptionName

    IF ($SubscriptionExists.Name -eq $SubscriptionName)
    {
        $subscription=Select-AzureRmSubscription -SubscriptionName $SubscriptionName
    }
    else
    {
        throw "You dont have access to [$SubscriptionName] subscription"
    }

  
New-ProvisionCosmosDb -DocumentDBApi $CosmosDBApiVersion -CosmosDBEndPoint $CosmosDbEndPoint -DatabaseName $CosmosDbDatabaseName  -ResourceGroupName $ResourceGroupName  -CosmosdbAccountName $CosmosDbAccountName

Provision Cosmos Database Collection

Make sure you read & provide variable values correctly especially for partitioning and index related details or else it will throw an exception. By default I’m geo-replicating the database to another region for reads since RU’s will be shared between primary and secondary db’s.

<###################### Update Variable values ############################>
$SubscriptionName = "Free Trial"
$ResourceGroupName = "rg-sql-articles-sin-devtest"
$CosmosDbAccountName = "sql-articles-cosmos-dev-account"
$CosmosDBApiVersion = "2017-02-22"
$CosmosDbEndPoint = "https://$CosmosDbAccountName.documents.azure.com:443/"
$CosmosDbDatabaseName = "sql-articles-cosmos-db"
$CosmosDbCollectionName = "sqlarticlesblogscollection"
$CosmosDbIsPartitioned = 0  # 1 for Paritioned collection 0 for non paritioned collection
$CosmosDbPartitionKey = ""  # Provide paritioning key value, system wont validate the key as this is NoSQL db. So ensure you provide it correctly. Example "EmployeeId"
$CosmosDbIndex = '' #Leave Blank if default index need to get created
$CosmosDbMaxRU = 400 #Min 400 & Max 10000 for single partition collection, Min 2500 & Max 100000 for partitioned collection
<##########################################################################>
<###################### Dont update anything below ########################>

## When requested login with Azure credentials

Login-AzureRmAccount

    Write-Host "ResourceGroupName    : `"$ResourceGroupName`""
    Write-Host "CosmosdbAccountName  : `"$CosmosdbAccountName`""
    Write-Host "CosmosDBEndPoint     : `"$CosmosDBEndPoint`""
    Write-Host "DatabaseName         : `"$CosmosDbDatabaseName`""
    Write-Host "CollectionName       : `"$CosmosDbCollectionName`""
    Write-Host "PartitionKey         : `"$CosmosDbPartitionKey`""
    Write-Host "CustomIndexScript    : `"$CosmosDbIndex`""
    Write-Host "DocumentDBApiVersion : `"$CosmosDBApiVersion`""
    Write-Host "MaximumRU            : `"$CosmosDbMaxRU`""

;Add-Type -AssemblyName System.Web

# generate authorization key
Function Generate-MasterKeyAuthorizationSignature
{
  [CmdletBinding()]
  Param
  (
    [Parameter(Mandatory=$true)][String]$verb,
    [Parameter(Mandatory=$false)][String]$resourceLink,
    [Parameter(Mandatory=$false)][String]$resourceType,
    [Parameter(Mandatory=$true)][String]$dateTime,
    [Parameter(Mandatory=$true)][String]$key,
    [Parameter(Mandatory=$true)][String]$keyType,
    [Parameter(Mandatory=$true)][String]$tokenVersion
  )

  $hmacSha256 = New-Object System.Security.Cryptography.HMACSHA256
  $hmacSha256.Key = [System.Convert]::FromBase64String($key)

  If ($resourceLink -eq $resourceType) {
    $resourceLink = ""
  }

  $payLoad = "$($verb.ToLowerInvariant())`n$($resourceType.ToLowerInvariant())`n$resourceLink`n$($dateTime.ToLowerInvariant())`n`n"
  $hashPayLoad = $hmacSha256.ComputeHash([System.Text.Encoding]::UTF8.GetBytes($payLoad))
  $signature = [System.Convert]::ToBase64String($hashPayLoad);
  [System.Web.HttpUtility]::UrlEncode("type=$keyType&ver=$tokenVersion&sig=$signature")
}

Function Get-CosmosCollections
{
  [CmdletBinding()]
  Param
  (
        [Parameter(Mandatory=$true)][String]$DocumentDBApi,
    [Parameter(Mandatory=$true)][String]$EndPoint,
    [Parameter(Mandatory=$true)][String]$DatabaseName,
    [Parameter(Mandatory=$true)][String]$MasterKey
  )

  $Verb = "GET"
  $ResourceType = "colls";
  $ResourceLink = "dbs/$DatabaseName"

  $dateTime = [DateTime]::UtcNow.ToString("r")
  $authHeader = Generate-MasterKeyAuthorizationSignature -verb $Verb -resourceLink $ResourceLink -resourceType $ResourceType -key $MasterKey -keyType "master" -tokenVersion "1.0" -dateTime $dateTime
  $header = @{authorization=$authHeader;"x-ms-documentdb-isquery"="True";"x-ms-version"=$DocumentDBApi;"x-ms-date"=$dateTime}
  $contentType= "application/json"
  $queryUri = "$EndPoint$ResourceLink/colls"

    $result = Invoke-RestMethod -Method $Verb -ContentType $contentType -Uri $queryUri -Headers $header

  $result.DocumentCollections

}


Function Create-CosmosCollections
{
  [CmdletBinding()]
  Param
  (
        [Parameter(Mandatory=$true)][String]$DatabaseName,
    [Parameter(Mandatory=$true)][String]$EndPoint,
        [Parameter(Mandatory=$true)][String]$CollectionName,
        [Parameter(Mandatory=$true)][String]$MaxRU,
    [Parameter(Mandatory=$true)][String]$MasterKey,
        [Parameter(Mandatory=$true)][Boolean]$IsPartitioned,
        [Parameter(Mandatory=$false)][String]$PartitionKey,
        [Parameter(Mandatory=$false)][String]$CustomIndex,
        [Parameter(Mandatory=$true)][String]$DocumentDBApi
  )

  $Verb = "POST"
  $ResourceType = "colls";
  $ResourceLink = "dbs/$DatabaseName"
    
    #Check JSON structure is valid

    try {
    $powershellRepresentation = ConvertFrom-Json $CustomIndex -ErrorAction Stop;
    $validJson = $true;
    } catch {
    $validJson = $false;
    }



        IF($validJson -eq $false)
        {
            throw "Invalid JSON index format"
            Write-Error "Invalid JSON index format"
        }

        if ($IsPartitioned -eq $true -and ($PartitionKey -eq "" -or $PartitionKey -eq $null))
        {
            throw "IsPartitioned is set to true but no parition key is provided. Update the variable and redeploy."
        }

        
    $partitionschema = IF ($IsPartitioned -eq $true) { ',"partitionKey": {  
                "paths": [  
                  "/'+$PartitionKey+'"  
                ],  
                "kind": "Hash"  
              }'} else {""}

    $indexschema = IF ($validJson -eq $true -and ($CustomIndex.Length -ne 0 )) { ',"indexingPolicy":'+$CustomIndex} else {""}

    $body = '{  
              "id": "'+$CollectionName+'"'+$indexschema+$partitionschema+'}' 
            
  $dateTime = [DateTime]::UtcNow.ToString("r")
  $authHeader = Generate-MasterKeyAuthorizationSignature -verb $Verb -resourceLink $ResourceLink -resourceType $ResourceType -key $MasterKey -keyType "master" -tokenVersion "1.0" -dateTime $dateTime
  $header = @{authorization=$authHeader;"x-ms-version"=$DocumentDBApi;"x-ms-date"=$dateTime; "Accept" =  "application/json";"x-ms-offer-throughput"=$MaxRU }
  $contentType= "application/json"
  $queryUri = "$EndPoint$ResourceLink/colls"
  $result = Invoke-RestMethod -Method $Verb -ContentType $contentType -Uri $queryUri -Headers $header  -Body $body

  $result 
}

Function Get-PrimaryKey
{

    [CmdletBinding()]
  Param
  (
        [Parameter(Mandatory=$true)][String]$DocumentDBApi,
    [Parameter(Mandatory=$true)][String]$ResourceGroupName,
        [Parameter(Mandatory=$true)][String]$CosmosdbAccountName
    )

    try
    {
  
        $keys=Invoke-AzureRmResourceAction -Action listKeys -ResourceType "Microsoft.DocumentDb/databaseAccounts" -ApiVersion $DocumentDBApi -ResourceGroupName $ResourceGroupName -Name $CosmosdbAccountName -Force
        $connectionKey=$keys[0].primaryMasterKey
        return $connectionKey
    }
    catch 
    {
        Write-Host "ErrorStatusDescription:" $_
    }
}

Function New-ProvisionCosmosCollection
{

    Param
  (
    [Parameter(Mandatory=$true)][String]$ResourceGroupName,
        [Parameter(Mandatory=$true)][String]$CosmosdbAccountName,
    [Parameter(Mandatory=$true)][String]$CosmosDBEndPoint,
    [Parameter(Mandatory=$true)][String]$DatabaseName,
        [Parameter(Mandatory=$true)][String]$CollectionName,
        [Parameter(Mandatory=$true)][Boolean]$IsPartitioned,
        [Parameter(Mandatory=$false)][String]$PartitionKey,
        [Parameter(Mandatory=$false)][String]$CustomIndex,
        [Parameter(Mandatory=$true)][String]$DocumentDBApi,
        [Parameter(Mandatory=$true)][String]$MaxRU
  )



    try
    {
        $MasterKey = Get-PrimaryKey -DocumentDBApi "2015-04-08" -ResourceGroupName $ResourceGroupName -CosmosdbAccountName $CosmosdbAccountName 
        IF ($MasterKey -eq "" -or $MasterKey -eq $null )
        {
            throw "Unable to Retrieve MasterKey data for [$CosmosdbAccountName]."
        }
    }
    catch
    {
        Write-Host ($_)
        throw $_
    }
    
  
    try
    {
        $CollectionExistence =  Get-CosmosCollections -EndPoint $CosmosDBEndPoint -MasterKey $MasterKey -DatabaseName $DatabaseName -DocumentDBApi $DocumentDBApi | WHERE id -eq $CollectionName
                
    if ($CollectionExistence.id -eq $CollectionName) 
    {
      Write-Host "Collection [$CollectionName] already exists in [$DatabaseName] database"
    } 

    
    }
    catch
    {
        Write-Host ($_)
        throw $_
    }
    
    try
    {
        if ($CollectionExistence.id -ne $CollectionName) 
        { Write-Host ($CustomIndex)
            Create-CosmosCollections -DatabaseName $DatabaseName -EndPoint $CosmosDBEndPoint -CollectionName $CollectionName -MaxRU $MaxRU -MasterKey $MasterKey -IsPartitioned $IsPartitioned -PartitionKey $PartitionKey -CustomIndex $CustomIndex -DocumentDBApi $DocumentDBApi
        }
    }
    catch
    {
        Write-Host ($_)
        throw $_
    } 
}

    $SubscriptionExists=Get-AzureRmSubscription -SubscriptionName $SubscriptionName

    IF ($SubscriptionExists.Name -eq $SubscriptionName)
    {
        $subscription=Select-AzureRmSubscription -SubscriptionName $SubscriptionName
    }
    else
    {
        throw "You dont have access to [$SubscriptionName] subscription"
    }

New-ProvisionCosmosCollection -ResourceGroupName $ResourceGroupName -CosmosdbAccountName $CosmosDbAccountName -CosmosDBEndPoint $CosmosDbEndPoint -DatabaseName $CosmosDbDatabaseName -CollectionName $CosmosDbCollectionName -IsPartitioned $CosmosDbIsPartitioned -PartitionKey $CosmosDbPartitionKey -CustomIndex $CosmosDbIndex -DocumentDBApi $CosmosDBApiVersion -MaxRU $CosmosDbMaxRU

Below is the screen shot from Azure portal listing out the complete provision. I have provisioned a single partitioned collection.  If these scripts can be called directly one by one or using it in octopus will make help in reducing the effort taken to provision cosmos database.


Posted

in

by

Comments

7 responses to “Provisioning Azure Cosmos DB using Powershell”

  1. Harshil avatar
    Harshil

    Thanks a lot for this !! You’re a life saver !!

  2. Dmytro avatar
    Dmytro

    Great article and well written code. Thank you for your effort!

  3. Leszek avatar
    Leszek

    Great article.
    Thank you.

    1. ChuckKings avatar

      Have you faced this error :

      Invoke-RestMethod : The remote server returned an error: (404) Not Found.
      At line:59 char:13
      + $result = Invoke-RestMethod -Method $Verb -ContentType $contentType …
      + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      + CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebException
      + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand

      In this line:
      $result = Invoke-RestMethod -Method $Verb -ContentType $contentType -Uri $queryUri -Headers $header

      1. VidhyaSagar avatar

        As I said earlier ARM template for Cosmos database changed quite a lot. Let me update this

  4. SY avatar
    SY

    Hi I am trying to create indexes with collections and passing below json for indexes. But i am getting 400 bad request error.
    {“indexingMode”: “consistent”,”automatic”: true,”includedPaths”: [{“path”: “/SomeId/?”,”indexes”: [{“kind”: “Range”,”dataType”: “String”,”precision”: 1}]}],”excludedPaths”: [{“path”: “/*”}]}

    If i pass any path /* like this it is working.

    Can you tell me where i have done mistake

    1. VidhyaSagar avatar

      Hi Sudheer.. Lot of things changed in Cosmos db ARM template after I published this article. Let me revisit this and update the template

Leave a Reply to ChuckKings Cancel reply

Your email address will not be published. Required fields are marked *