Category: DevOps

Azure Key Vault – List Secrets that have not been set

When working across multiple environments, with restricted access, it can by difficult tracking which Key Vault secrets have been configured and which still need values set.

On my projects I like to give the dev teams autonomy on the dev environments setting their own Key Vault Keys and Secrets (following an agreed convention).

Periodically we then copy these keys to the other environment Key Vaults (see my other post on https://blog.nicholasrogoff.com/2021/08/09/azure-key-vault-script-for-copying-secrets-from-one-to-another/ )

The following script can be run to output all the Key Vault secrets, or only show the ones that need configuration. It can also be set to output pure markdown like…

# hms-sapp-kv-qa-ne (2021-07-03 14:31:04.590)

| Secret Name                                           | Needs Configuration                    |
|-------------------------------------------------------|----------------------------------------|
| apim-api-subscription-primary-key                     | ******                                 |
| apim-tests-subscription-primary-key                   | ******                                 |
| b2c-client-id-extensions-app                          | ******                                 |
| b2c-client-id-postman                                 | ******                                 |
| saleforce-client-id                                   | Needs Configuration                    |
| salesforce-client-secret                              | Needs Configuration                    |
| sql-database-ro-password                              | ******                                 |
| sql-database-rw-password                              | ******                                 |

that can be copy and pasted into the Azure or Github Wiki too and displays like the table below when rendered.

hms-sapp-kv-qa-ne (2021-07-03 14:31:04.590)

Secret Name Needs Configuration
apim-api-subscription-primary-key ******
apim-tests-subscription-primary-key ******
b2c-client-id-extensions-app ******
b2c-client-id-postman ******
saleforce-client-id Needs Configuration
salesforce-client-secret Needs Configuration
sql-database-ro-password ******
sql-database-rw-password ******

The script will output ‘Needs Configuration’ for any Secret values that are either blank, contain ‘Needs Configuration’ or ‘TBC’.

Script below, aslo contains examples:

<#PSScriptInfo
.VERSION 1.1
.GUName 48b4b27a-b77e-41e6-8a37-b3767da5caee
.AUTHOR Nicholas Rogoff

.RELEASENOTES
Initial version.
#>
<# 
.SYNOPSIS 
Copy Key Vault Secrets from one Vault to another. If the secret name exists it will NOT overwrite the value. 
 
.DESCRIPTION 
Loops through all secrets and copies them or fills them with a 'Needs Configuration'.
Will not copy secrets of ContentType application/x-pkcs12

PRE-REQUIREMENT
---------------
Run 'Import-Module Az.Accounts'
Run 'Import-Module Az.KeyVault'
You need to be logged into Azure and have the access necessary rights to both Key Vaults.

.INPUTS
None. You cannot pipe objects to this!

.OUTPUTS
None.

.PARAMETER SrcSubscriptionName
This is the Source Subscription Name

.PARAMETER SrcKvName
The name of the Source Key Vault

.PARAMETER MarkdownOnly
Output Markdown Only

.PARAMETER NameOnly
Set to only copy across the secret name and NOT the actual secret. The secret will be populated with 'Needs Configuration'

.NOTES
  Version:        1.1
  Author:         Nicholas Rogoff
  Creation Date:  2021-08-09
  Purpose/Change: Refined for publication
   
.EXAMPLE 
PS> .\List-UnSet-KeyVault-Secrets.ps1 -SrcSubscriptionName $srcSubscriptionName -SrcKvName $srcKvName 
This will list the secrets keys in the specified key vault and diplay 'Needs Configuration' for those that are blank, contain 'Needs Configuration' or 'TBC'

.EXAMPLE 
PS> .\List-UnSet-KeyVault-Secrets.ps1 -SrcSubscriptionName $srcSubscriptionName -SrcKvName $srcKvName -MarkdownOnly
Use for updating the Wiki. This will list the secrets keys, as MarkDown ONLY, in the specified key vault and diplay 'Needs Configuration' for those that are blank, contain 'Needs Configuration' or 'TBC'


.EXAMPLE 
PS> .\List-UnSet-KeyVault-Secrets.ps1 -SrcSubscriptionName $srcSubscriptionName -SrcKvName $srcKvName -ShowSecrets
This will list the secrets keys in the specified key vault and show the secrets


#>
#---------------------------------------------------------[Script Parameters]------------------------------------------------------
[CmdletBinding()]
Param(
  [Parameter(Mandatory = $true, HelpMessage = "This is the Source Subscription Name")]
  [string] $SrcSubscriptionName,
  [Parameter(Mandatory = $true, HelpMessage = "The name of the Source Key Vault")]
  [string] $SrcKvName,
  [Parameter(Mandatory = $false, HelpMessage = "Output Markdown Only")]
  [switch] $MarkdownOnly,
  [Parameter(Mandatory = $false, HelpMessage = "Output the secrets set")]
  [switch] $ShowSecrets
)

#---------------------------------------------------------[Initialisations]--------------------------------------------------------

# Set Error Action to Silently Continue
$ErrorActionPreference = 'Continue'

#----------------------------------------------------------[Declarations]----------------------------------------------------------
# Any Global Declarations go here

$SecretsFound = @()

#----------------------------------------------------------[Functions]----------------------------------------------------------

# Inline If Function
Function IIf($If, $Right, $Wrong) { If ($If) { $Right } Else { $Wrong } }

#-----------------------------------------------------------[Execution]------------------------------------------------------------

$checked = 0
$failed = 0

if (!$MarkdownOnly) {
  Write-Host "======================================================"
  Write-Host ($(Get-Date -Format 'yyyy-MM-dd HH:mm:ss.fff') + " Starting to check secrets in " + $SrcKvName + "... ") -ForegroundColor Blue
  Write-Host "======================================================"
}
else {
  Write-Host ("# " + $SrcKvName + " (" + $(Get-Date -Format 'yyyy-MM-dd HH:mm:ss.fff') + ")") -ForegroundColor Blue
}
Write-Host ""



$sourceSecrets = Get-AzKeyVaultSecret -VaultName $SrcKvName | Where-Object { $_.ContentType -notmatch "application/x-pkcs12" }

if (!$MarkdownOnly) {
  # Headers
  Write-Host -NoNewline "|".PadRight(60, "-")
  Write-Host -NoNewline "|".PadRight(60, "-")
  Write-Host "|"
}

Write-Host -NoNewline "| Secret Name".PadRight(60)
if (!$ShowSecrets) {
  Write-Host -NoNewline "| Needs Configuration".PadRight(60)  
}
else {
  Write-Host -NoNewline "| Secret Value".PadRight(60)      
}
Write-Host "|"

Write-Host -NoNewline "|".PadRight(60, "-")
Write-Host -NoNewline "|".PadRight(60, "-")
Write-Host "|"

ForEach ($sourceSecret in $sourceSecrets) {
  $Error.clear()

  $name = $sourceSecret.Name
  $plainTxtSecret = Get-AzKeyVaultSecret -VaultName $srckvName -Name $name -AsPlainText

  if ($plainTxtSecret -eq "Needs Configuration" -or $plainTxtSecret -eq "TBC" -or !$plainTxtSecret) {
    $secretToShow = $plainTxtSecret
  }
  elseif ($ShowSecrets) {
    $secretToShow = $plainTxtSecret
  }
  else {
    $secretToShow = "******"
  }

  Write-Host -NoNewline "| $name".PadRight(60)
  if ($secretToShow -eq "Needs Configuration" -or $plainTxtSecret -eq "TBC" -or !$plainTxtSecret) {
    Write-Host -NoNewline "| $secretToShow".PadRight(60) -ForegroundColor Magenta  
  }
  else {
    Write-Host -NoNewline "| $secretToShow".PadRight(60) -ForegroundColor DarkGray      
  }
  Write-Host "|"

  if (!$Error[0]) {
    $checked += 1
  }
  else {
    $failed += 1
    Write-Error "!! Failed to get secret $name"
  }
}

if (!$MarkdownOnly) {
  Write-Host -NoNewline "|".PadRight(60, "-")
  Write-Host -NoNewline "|".PadRight(60, "-")    
  Write-Host "|"

  Write-Host ""
  Write-Host ""
  Write-Host "================================="
  Write-Host "Completed Key Vault Secrets Copy"
  Write-Host "Checked: $checked"
  Write-Host "Failed: $failed"
  Write-Host "================================="
}
else {
  Write-Host ""
  Write-Host ("**Total Secrets Listed: " + $checked + "**")
  if ($failed -gt 0) {
    Write-Host "Failed: $failed" -ForegroundColor Red
  } 
  Write-Host ""
}

Cloud Resource Naming Convention (Azure)

In any organisation it is important to get a standard naming convention in place for most things, but especially with cloud based resources.

As many types of cloud resources require globally unique names (due to platform DNS resolution), it’s important to have a strategy that will give you a good chance of achieving global uniqueness, but also as helpful as possible to human beings, as well as codefiable in DevOps CD pipelines.

Continue reading “Cloud Resource Naming Convention (Azure)”
‘*** Execution Timeout Expired’ with SqlPackage.exe on Azure DevOps Release Pipeline Fix

‘*** Execution Timeout Expired’ with SqlPackage.exe on Azure DevOps Release Pipeline Fix

A while ago I was having an issue deploying a SQL Data Tools project using the Azure DevOps pipelines. It worked fine when there was no data in the database, but once there was some data to preserve in the release the DACPAC deployment started to timeout with the following error when running SqlPackage.

Continue reading “‘*** Execution Timeout Expired’ with SqlPackage.exe on Azure DevOps Release Pipeline Fix”