Category: Development

Azure Key Vault – List Secrets that have not been set

When working across multiple environments, with restricted access, it can by difficult tracking which Key Vault secrets have been configured and which still need values set.

On my projects I like to give the dev teams autonomy on the dev environments setting their own Key Vault Keys and Secrets (following an agreed convention).

Periodically we then copy these keys to the other environment Key Vaults (see my other post on https://blog.nicholasrogoff.com/2021/08/09/azure-key-vault-script-for-copying-secrets-from-one-to-another/ )

The following script can be run to output all the Key Vault secrets, or only show the ones that need configuration. It can also be set to output pure markdown like…

# hms-sapp-kv-qa-ne (2021-07-03 14:31:04.590)

| Secret Name                                           | Needs Configuration                    |
|-------------------------------------------------------|----------------------------------------|
| apim-api-subscription-primary-key                     | ******                                 |
| apim-tests-subscription-primary-key                   | ******                                 |
| b2c-client-id-extensions-app                          | ******                                 |
| b2c-client-id-postman                                 | ******                                 |
| saleforce-client-id                                   | Needs Configuration                    |
| salesforce-client-secret                              | Needs Configuration                    |
| sql-database-ro-password                              | ******                                 |
| sql-database-rw-password                              | ******                                 |

that can be copy and pasted into the Azure or Github Wiki too and displays like the table below when rendered.

hms-sapp-kv-qa-ne (2021-07-03 14:31:04.590)

Secret Name Needs Configuration
apim-api-subscription-primary-key ******
apim-tests-subscription-primary-key ******
b2c-client-id-extensions-app ******
b2c-client-id-postman ******
saleforce-client-id Needs Configuration
salesforce-client-secret Needs Configuration
sql-database-ro-password ******
sql-database-rw-password ******

The script will output ‘Needs Configuration’ for any Secret values that are either blank, contain ‘Needs Configuration’ or ‘TBC’.

Script below, aslo contains examples:

<#PSScriptInfo
.VERSION 1.1
.GUName 48b4b27a-b77e-41e6-8a37-b3767da5caee
.AUTHOR Nicholas Rogoff

.RELEASENOTES
Initial version.
#>
<# 
.SYNOPSIS 
Copy Key Vault Secrets from one Vault to another. If the secret name exists it will NOT overwrite the value. 
 
.DESCRIPTION 
Loops through all secrets and copies them or fills them with a 'Needs Configuration'.
Will not copy secrets of ContentType application/x-pkcs12

PRE-REQUIREMENT
---------------
Run 'Import-Module Az.Accounts'
Run 'Import-Module Az.KeyVault'
You need to be logged into Azure and have the access necessary rights to both Key Vaults.

.INPUTS
None. You cannot pipe objects to this!

.OUTPUTS
None.

.PARAMETER SrcSubscriptionName
This is the Source Subscription Name

.PARAMETER SrcKvName
The name of the Source Key Vault

.PARAMETER MarkdownOnly
Output Markdown Only

.PARAMETER NameOnly
Set to only copy across the secret name and NOT the actual secret. The secret will be populated with 'Needs Configuration'

.NOTES
  Version:        1.1
  Author:         Nicholas Rogoff
  Creation Date:  2021-08-09
  Purpose/Change: Refined for publication
   
.EXAMPLE 
PS> .\List-UnSet-KeyVault-Secrets.ps1 -SrcSubscriptionName $srcSubscriptionName -SrcKvName $srcKvName 
This will list the secrets keys in the specified key vault and diplay 'Needs Configuration' for those that are blank, contain 'Needs Configuration' or 'TBC'

.EXAMPLE 
PS> .\List-UnSet-KeyVault-Secrets.ps1 -SrcSubscriptionName $srcSubscriptionName -SrcKvName $srcKvName -MarkdownOnly
Use for updating the Wiki. This will list the secrets keys, as MarkDown ONLY, in the specified key vault and diplay 'Needs Configuration' for those that are blank, contain 'Needs Configuration' or 'TBC'


.EXAMPLE 
PS> .\List-UnSet-KeyVault-Secrets.ps1 -SrcSubscriptionName $srcSubscriptionName -SrcKvName $srcKvName -ShowSecrets
This will list the secrets keys in the specified key vault and show the secrets


#>
#---------------------------------------------------------[Script Parameters]------------------------------------------------------
[CmdletBinding()]
Param(
  [Parameter(Mandatory = $true, HelpMessage = "This is the Source Subscription Name")]
  [string] $SrcSubscriptionName,
  [Parameter(Mandatory = $true, HelpMessage = "The name of the Source Key Vault")]
  [string] $SrcKvName,
  [Parameter(Mandatory = $false, HelpMessage = "Output Markdown Only")]
  [switch] $MarkdownOnly,
  [Parameter(Mandatory = $false, HelpMessage = "Output the secrets set")]
  [switch] $ShowSecrets
)

#---------------------------------------------------------[Initialisations]--------------------------------------------------------

# Set Error Action to Silently Continue
$ErrorActionPreference = 'Continue'

#----------------------------------------------------------[Declarations]----------------------------------------------------------
# Any Global Declarations go here

$SecretsFound = @()

#----------------------------------------------------------[Functions]----------------------------------------------------------

# Inline If Function
Function IIf($If, $Right, $Wrong) { If ($If) { $Right } Else { $Wrong } }

#-----------------------------------------------------------[Execution]------------------------------------------------------------

$checked = 0
$failed = 0

if (!$MarkdownOnly) {
  Write-Host "======================================================"
  Write-Host ($(Get-Date -Format 'yyyy-MM-dd HH:mm:ss.fff') + " Starting to check secrets in " + $SrcKvName + "... ") -ForegroundColor Blue
  Write-Host "======================================================"
}
else {
  Write-Host ("# " + $SrcKvName + " (" + $(Get-Date -Format 'yyyy-MM-dd HH:mm:ss.fff') + ")") -ForegroundColor Blue
}
Write-Host ""



$sourceSecrets = Get-AzKeyVaultSecret -VaultName $SrcKvName | Where-Object { $_.ContentType -notmatch "application/x-pkcs12" }

if (!$MarkdownOnly) {
  # Headers
  Write-Host -NoNewline "|".PadRight(60, "-")
  Write-Host -NoNewline "|".PadRight(60, "-")
  Write-Host "|"
}

Write-Host -NoNewline "| Secret Name".PadRight(60)
if (!$ShowSecrets) {
  Write-Host -NoNewline "| Needs Configuration".PadRight(60)  
}
else {
  Write-Host -NoNewline "| Secret Value".PadRight(60)      
}
Write-Host "|"

Write-Host -NoNewline "|".PadRight(60, "-")
Write-Host -NoNewline "|".PadRight(60, "-")
Write-Host "|"

ForEach ($sourceSecret in $sourceSecrets) {
  $Error.clear()

  $name = $sourceSecret.Name
  $plainTxtSecret = Get-AzKeyVaultSecret -VaultName $srckvName -Name $name -AsPlainText

  if ($plainTxtSecret -eq "Needs Configuration" -or $plainTxtSecret -eq "TBC" -or !$plainTxtSecret) {
    $secretToShow = $plainTxtSecret
  }
  elseif ($ShowSecrets) {
    $secretToShow = $plainTxtSecret
  }
  else {
    $secretToShow = "******"
  }

  Write-Host -NoNewline "| $name".PadRight(60)
  if ($secretToShow -eq "Needs Configuration" -or $plainTxtSecret -eq "TBC" -or !$plainTxtSecret) {
    Write-Host -NoNewline "| $secretToShow".PadRight(60) -ForegroundColor Magenta  
  }
  else {
    Write-Host -NoNewline "| $secretToShow".PadRight(60) -ForegroundColor DarkGray      
  }
  Write-Host "|"

  if (!$Error[0]) {
    $checked += 1
  }
  else {
    $failed += 1
    Write-Error "!! Failed to get secret $name"
  }
}

if (!$MarkdownOnly) {
  Write-Host -NoNewline "|".PadRight(60, "-")
  Write-Host -NoNewline "|".PadRight(60, "-")    
  Write-Host "|"

  Write-Host ""
  Write-Host ""
  Write-Host "================================="
  Write-Host "Completed Key Vault Secrets Copy"
  Write-Host "Checked: $checked"
  Write-Host "Failed: $failed"
  Write-Host "================================="
}
else {
  Write-Host ""
  Write-Host ("**Total Secrets Listed: " + $checked + "**")
  if ($failed -gt 0) {
    Write-Host "Failed: $failed" -ForegroundColor Red
  } 
  Write-Host ""
}

Azure Key Vault – Script for copying secrets from one to another

The following script can be used to copy secrets from one Key Vault to another.

You can use this to copy secrets or just the secret names (-NameOnly) from one Key Vault to another, in the same or another subscription.

It’s as simple as the following PowerShell commands below to execute:

$srcSubscriptionName="Development"
$srcKvName="my-keyvault-dev-ne"
$destSubscriptionName="Production"
$destKvName="my-keyvault-prod-ne"
.\Copy-KeyVault-Secrets.ps1 -SrcSubscriptionName $srcSubscriptionName -SrcKvName $srcKvName -DestSubscriptionName $destSubscriptionName -DestKvName $destKvName -NameOnly
<#PSScriptInfo
.VERSION 1.1
.GUName 48b4b27a-b77e-41e6-8a37-b3767da5caee
.AUTHOR Nicholas Rogoff

.RELEASENOTES
Initial version.
#>
<# 
.SYNOPSIS 
Copy Key Vault Secrets from one Vault to another. 
 
.DESCRIPTION 
Loops through all secrets and copies them or fills them with a 'Needs Configuration'.

PRE-REQUIREMENT
---------------
Run 'Import-Module Az.Accounts'
Run 'Import-Module Az.KeyVault'
You need to be logged into Azure and have the access necessary rights to both Key Vaults.

.INPUTS
None. You cannot pipe objects to this!

.OUTPUTS
None.

.PARAMETER SrcSubscriptionName
This is the Source Subscription Name

.PARAMETER SrcKvName
The name of the Source Key Vault

.PARAMETER DestSubscriptionName
This is the destination Subscription Name

.PARAMETER DestKvName
The name of the destination Key Vault

.PARAMETER NameOnly
Set to only copy across the secret name and NOT the actual secret. The secret will be populated with 'Needs Configuration'

.NOTES
  Version:        1.1
  Author:         Nicholas Rogoff
  Creation Date:  2021-08-09
  Purpose/Change: Refined for publication
   
.EXAMPLE 
PS> .\Copy-KeyVault-Secrets.ps1 -SrcSubscriptionName $srcSubscriptionName -SrcKvName $srcKvName -DestSubscriptionName $destSubscriptionName -DestKvName $destKvName -NameOnly
This will copy across only the secret names, filling the secret with 'Needs Configuration'
#>
#---------------------------------------------------------[Script Parameters]------------------------------------------------------
[CmdletBinding()]
Param(
  [Parameter(Mandatory = $true, HelpMessage = "This is the Source Subscription Name")]
  [string] $SrcSubscriptionName,
  [Parameter(Mandatory = $true, HelpMessage = "The name of the Source Key Vault")]
  [string] $SrcKvName,
  [Parameter(Mandatory = $false, HelpMessage = "This is the destination Subscription Name. If not set or blank then same subscription is assumed")]
  [string] $DestSubscriptionName,
  [Parameter(Mandatory = $true, HelpMessage = "The name of the destination Key Vault")]
  [string] $DestKvName,
  [Parameter(Mandatory = $false, HelpMessage = "Only copy across the secret name and NOT the actual secret. The secret will be populated with 'Needs Configuration'")]
  [switch] $NameOnly
)

#---------------------------------------------------------[Initialisations]--------------------------------------------------------
Write-Host ($(Get-Date -Format 'yyyy-MM-dd HH:mm:ss.fff') + " Starting copying from " + $SrcKvName + " to " + $DestKvName + "... ") -ForegroundColor Blue

# Set Error Action to Silently Continue
$ErrorActionPreference = 'Continue'

#----------------------------------------------------------[Declarations]----------------------------------------------------------
# Any Global Declarations go here

#----------------------------------------------------------[Functions]----------------------------------------------------------


#-----------------------------------------------------------[Execution]------------------------------------------------------------

$success = 0
$failed = 0

# ensure source subscription is selected
Select-AzSubscription -Subscription $SrcSubscriptionName

$Tags = @{ 'Migrated' = 'true'; 'Source Key Vault' = $SrcKvName }

$sourceSecrets = Get-AzKeyVaultSecret -VaultName $SrcKvName
if ($DestSubscriptionName) {
  #Need to switch subscriptions
  Select-AzSubscription -Subscription $DestSubscriptionName
}

ForEach ($sourceSecret in $sourceSecrets) {
  $Error.clear()

  $name = $sourceSecret.Name
  $tags = $sourceSecret.Tags
  $secret = Get-AzKeyVaultSecret -VaultName $srckvName -Name $name


  Write-Host "Adding SecretName: $name ..."
  if ($NameOnly) {
    $value = ConvertTo-SecureString 'Needs Configuration' -AsPlainText -Force
  }
  else {
    $value = $secret.SecretValue
  }
  $secret = Set-AzKeyVaultSecret -VaultName $destkvName -Name $sourceSecret.Name -SecretValue $value -ContentType $sourceSecret.ContentType -Tags $tags 
  
  if (!$Error[0]) {
    $success += 1
  }
  else {
    $failed += 1
    Write-Error "!! Failed to copy secret $name"
  }
}

Write-Output "================================="
Write-Output "Completed Key Vault Secrets Copy"
Write-Output "Succeeded: $success"
Write-Output "Failed: $failed"
Write-Output "================================="

Cloud Resource Naming Convention (Azure)

In any organisation it is important to get a standard naming convention in place for most things, but especially with cloud based resources.

As many types of cloud resources require globally unique names (due to platform DNS resolution), it’s important to have a strategy that will give you a good chance of achieving global uniqueness, but also as helpful as possible to human beings, as well as codefiable in DevOps CD pipelines.

Continue reading “Cloud Resource Naming Convention (Azure)”
How to get the Ratings from SharePoint into your PowerApp

How to get the Ratings from SharePoint into your PowerApp

The seems to be a disconnect between how ratings work in SharePoint, the actual data values in the underlying table and how to use them in a PowerApp.

I’ll explain what and how I got the ratings to show in my PowerApp.

Continue reading “How to get the Ratings from SharePoint into your PowerApp”

IActionResult Return Types and StatusCodes quick reference

I’m always forgetting which return types are available directly from ASP.Net controllers, so have created a quick listing here.

For general information on HTTP Status codes see
https://en.wikipedia.org/wiki/List_of_HTTP_status_codes

ASP.Net Core 2.2

All Status Codes:

https://docs.microsoft.com/en-gb/dotnet/api/microsoft.aspnetcore.http.statuscodes?view=aspnetcore-2.2

ControllerBase IActionResult Return Types

  • OK (200)
  • BadRequest (400)
  • Forbid (403)
  • LocalRedirect (302)
  • LocalRedirectPermanent (301)
  • LocalRedirectPermanentPreserve (308)
  • LocalRedirectPermanentPreserveMethod (307)
  • NoContent (204)
  • NotFound (404)
  • RedirectToAction (302)
  • RedirectToActionPermanent (301)
  • RedirectToActionPermanentPreserve (308)
  • RedirectToActionPermanentPreserveMethod (307)
  • RedirectToPage (302)
  • RedirectToPagePermanent (301)
  • RedirectToPagePermanentPreserve (308)
  • RedirectToPagePermanentPreserveMethod (307)
  • RedirectToPage (302)
  • RedirectToPagePermanent (301)
  • RedirectToPagePermanentPreserve (308)
  • RedirectToPagePermanentPreserveMethod (307)
  • StatusCode (set own status code)
    • Use 409 – Conflict, for updates that fail due to conflicts such as already exists etc..
  • Unauthorized (401)
  • UnprocessableEntity (422)
  • ValidationProblem (400)

ApiController MVC Compatibility Shim

https://docs.microsoft.com/en-us/dotnet/api/system.web.http.apicontroller?view=aspnetcore-2.2

  • OK (200)
  • BadRequest (400)
  • Conflict (409)
  • Created (201)
  • CreatedAtRoute (201)
  • InternalServerError (500)
  • NotFound (404)
  • Redirect (302)
  • RedirectToRoute (302)
  • StatusCode (set own status code)
    • Use 409 – Conflict, for updates that fail due to conflicts such as already exists etc..
  • Unauthorized (401)
  • UnprocessableEntity (422)
  • ValidationProblem (400)
‘*** Execution Timeout Expired’ with SqlPackage.exe on Azure DevOps Release Pipeline Fix

‘*** Execution Timeout Expired’ with SqlPackage.exe on Azure DevOps Release Pipeline Fix

A while ago I was having an issue deploying a SQL Data Tools project using the Azure DevOps pipelines. It worked fine when there was no data in the database, but once there was some data to preserve in the release the DACPAC deployment started to timeout with the following error when running SqlPackage.

Continue reading “‘*** Execution Timeout Expired’ with SqlPackage.exe on Azure DevOps Release Pipeline Fix”