Markdown generation for ARM and PowerShell
After my session at Azure LowLands with the title "You build It, You run It on the Microsoft Platform" a lot of people were interested in a script that I showed during the session.
The session itself is about developing a software package / product the DevOps way that also needs to be maintained after moving it to production. I tell a lot about how it should be done using 5 principles and my experience I had with multiple clients. During the demonstration I show Cloud Native resources in Azure that can be used in any kind of product or service.
Improvement of Daily work
One of the principles discussed is "Improvement of Daily work". This principle really makes our teams think about how they can improve the things we are doing and how that should be done. One of the thing that we did not like to do is write the documentation for our PowerShell scripts and ARM templates. For this we (Leon Boers and I) wrote some automation scripts to take care of this.
PowerShell
The script to generate the PowerShell documentation uses the "Help" that is specified within the script file.
<#
.SYNOPSIS
Script for generating Markdown documentation based on information in PowerShell script files.
.DESCRIPTION
All PowerShell script files have synopsis attached on the document. With this script markdown files are generated and saved within the target folder.
.PARAMETER ScriptFolder
The folder that contains the scripts
.PARAMETER OutputFolder
The folder were to safe the markdown files
.PARAMETER ExcludeFolders
Exclude folder for generation. This is a comma seperated list
.PARAMETER KeepStructure
Specified to keep the structure of the subfolders
.PARAMETER IncludeWikiTOC
Include the TOC from the Azure DevOps wiki to the markdown files
.NOTES
Version: 1.0.0;
Author: 3fifty | Maik van der Gaag | Leon Boers;
Creation Date: 20-04-2020;
Purpose/Change: Initial script development;
.EXAMPLE
.\New-MDPowerShellScripts.ps1 -ScriptFolder "./" -OutputFolder "docs/arm" -ExcludeFolder ".local,test-templates" -KeepStructure $true -IncludeWikiTOC $false
.EXAMPLE
.\New-MDPowerShellScripts.ps1 -ScriptFolder "./" -OutputFolder "docs/arm"
#>
The idea of using this we got from someone who was using the same principle on GitHub. We adopted the idea and created the below script.
[CmdletBinding()]
Param (
[Parameter(Mandatory = $true, Position = 0)][string]$ScriptFolder,
[Parameter(Mandatory = $true, Position = 1)][string]$OutputFolder,
[Parameter(Mandatory = $false, Position = 2)][string]$ExcludeFolders,
[Parameter(Mandatory = $false, Position = 3)][bool]$KeepStructure = $false,
[Parameter(Mandatory = $false, Position = 4)][bool]$IncludeWikiTOC = $false
)
BEGIN {
Write-Output ("ScriptFolder : $($ScriptFolder)")
Write-Output ("OutputFolder : $($OutputFolder)")
Write-Output ("ExcludeFolders : $($ExcludeFolders)")
Write-Output ("KeepStructure : $($KeepStructure)")
Write-Output ("IncludeWikiTOC : $($IncludeWikiTOC)")
$arrParameterProperties = @("DefaultValue", "ParameterValue", "PipelineInput", "Position", "Required")
$scriptNameSuffix = ".md"
$option = [System.StringSplitOptions]::RemoveEmptyEntries
$exclude = $ExcludeFolders.Split(',', $option)
}
PROCESS {
try {
Write-Information ("Starting documentation generation for folder $($ScriptFolder)")
if (!(Test-Path $OutputFolder)) {
Write-Information ("Output path does not exists creating the folder: $($OutputFolder)")
New-Item -ItemType Directory -Force -Path $OutputFolder
}
# Get the scripts from the folder
$scripts = Get-Childitem $ScriptFolder -Filter "*.ps1" -Recurse
foreach ($script in $scripts) {
if (!$exclude.Contains($script.Directory.Name)) {
Write-Information ("Documenting file: $($script.FullName)")
if ($KeepStructure) {
if ($script.DirectoryName -ne $ScriptFolder) {
$newfolder = $OutputFolder + "/" + $script.Directory.Name
if (!(Test-Path $newfolder)) {
Write-Information ("Output folder for item does not exists creating the folder: $($newfolder)")
New-Item -Path $OutputFolder -Name $script.Directory.Name -ItemType "directory"
}
}
} else {
$newfolder = $OutputFolder
}
$help = Get-Help $script.FullName -ErrorAction "SilentlyContinue" -Detailed
if ($help) {
$outputFile = ("$($newfolder)/$($script.BaseName)$($scriptNameSuffix)")
Out-File -FilePath $outputFile
if ($IncludeWikiTOC) {
("[[_TOC_]]`n") | Out-File -FilePath $outputFile
"`n" | Out-File -FilePath $outputFile -Append
}
#synopsis
if ($help.Synopsis) {
("## Synopsis") | Out-File -FilePath $outputFile -Append
("$($help.Synopsis)") | Out-File -FilePath $outputFile -Append
"`n" | Out-File -FilePath $outputFile -Append
} else {
Write-Warning -Message ("Synopsis not defined in file $($script.fullname)")
}
#syntax
if ($help.Syntax) {
("``````PowerShell`n $($help.Syntax.syntaxItem.name)`n``````") | Out-File -FilePath $outputFile -Append
"`n" | Out-File -FilePath $outputFile -Append
} else {
Write-Warning -Message ("Syntax not defined in file $($script.fullname)")
}
#notes (seperated by (name): and (value);)
if ($help.alertSet) {
("## Information") | Out-File -FilePath $outputFile -Append
$text = $help.alertSet.alert.Text.Split(';', $option)
foreach ($line in $text) {
$items = $line.Trim().Split(':', $option)
("**$($items[0]):** $($items[1])`n") | Out-File -FilePath $outputFile -Append
}
"`n" | Out-File -FilePath $outputFile -Append
} else {
Write-Warning -Message ("Notes not defined in file $($script.fullname)")
}
#description
if ($help.Description) {
"## Description" | Out-File -FilePath $outputFile -Append
$help.Description.Text | Out-File -FilePath $outputFile -Append
"`n" | Out-File -FilePath $outputFile -Append
} else {
Write-Warning -Message "Description not defined in file $($script.fullname)"
}
#examples
if ($help.Examples) {
("## Examples") | Out-File -FilePath $outputFile -Append
"`n" | Out-File -FilePath $outputFile -Append
forEach ($item in $help.Examples.Example) {
$title = $item.title.Replace("--------------------------", "").Replace("EXAMPLE", "Example")
("### $($title)") | Out-File -FilePath $outputFile -Append
if ($item.Code) {
("``````PowerShell`r`n $($item.Code) `r`n``````") | Out-File -FilePath $outputFile -Append
}
}
} else {
Write-Warning -Message "Examples not defined in file $($script.fullname)"
}
if ($help.Parameters) {
("## Parameters") | Out-File -FilePath $outputFile -Append
forEach ($item in $help.Parameters.Parameter) {
("### $($item.name)") | Out-File -FilePath $outputFile -Append
$item.description[0].text | Out-File -FilePath $outputFile -Append
("| | |") | Out-File -FilePath $outputFile -Append
("|-|-|") | Out-File -FilePath $outputFile -Append
("| Type: | $($item.Type.Name) |") | Out-File -FilePath $outputFile -Append
foreach ($arrParameterProperty in $arrParameterProperties) {
if ($item.$arrParameterProperty) {
("| $arrParameterProperty : | $($item.$arrParameterProperty)|") | Out-File -FilePath $outputFile -Append
}
}
"`n" | Out-File -FilePath $outputFile -Append
}
} else {
Write-Warning -Message "Parameters not defined in file $($script.fullname)"
}
} else {
Write-Error -Message ("Synopsis could not be found for script $($script.FullName)")
}
}
}
} catch {
Write-Error "Something went wrong while generating the output documentation: $_"
}
}
END {}
Within this script you can see that we have added a different implementation for the "Notes" section. In the notes section we wanted to place more information and also be able to format in a specific way. That is why we created this format:
- [Name]: [Value];
This why we are able to add all kind of information and keep is nicely formatted. The generated documentation looks like:
ARM Templates
When we finished the documentation generation for PowerShell script files we thought that it is also quite easy to use the same principle for ARM templates.
The only thing we had to think of was the way we wanted to add additional information for the documentation. For this we added the "Metadata" property. This property does not violate the schema validation and additional properties could be added like description, version and author.
"metadata": {
"Description": "This template deploys a standard storage account.",
"Author": "3fifty | Maik van der Gaag | Leon Boers",
"Version": "1.0.0"
}
The script looks quite the same as the PowerShell version but adopted the specific ideas for ARM.
[CmdletBinding()]
Param (
[Parameter(Mandatory = $true, Position = 0)][string]$TemplateFolder,
[Parameter(Mandatory = $true, Position = 1)][string]$OutputFolder,
[Parameter(Mandatory = $false, Position = 2)][string]$ExcludeFolders,
[Parameter(Mandatory = $false, Position = 3)][bool]$KeepStructure = $false,
[Parameter(Mandatory = $false, Position = 4)][bool]$IncludeWikiTOC = $false
)
BEGIN {
Write-Output ("TemplateFolder : $($TemplateFolder)")
Write-Output ("OutputFolder : $($OutputFolder)")
Write-Output ("ExcludeFolders : $($ExcludeFolders)")
Write-Output ("KeepStructure : $($KeepStructure)")
Write-Output ("IncludeWikiTOC : $($IncludeWikiTOC)")
$templateNameSuffix = ".md"
$option = [System.StringSplitOptions]::RemoveEmptyEntries
$exclude = $ExcludeFolders.Split(',', $option)
}
PROCESS {
try {
Write-Information ("Starting documentation generation for folder $($TemplateFolder)")
if (!(Test-Path $OutputFolder)) {
Write-Information ("Output path does not exists creating the folder: $($OutputFolder)")
New-Item -ItemType Directory -Force -Path $OutputFolder
}
# Get the scripts from the folder
$templates = Get-Childitem $TemplateFolder -Filter "*.json" -Recurse -Exclude "*parameters.json","*descriptions.json","*parameters.local.json"
foreach ($template in $templates) {
if (!$exclude.Contains($template.Directory.Name)) {
Write-Information ("Documenting file: $($template.FullName)")
if ($KeepStructure) {
if ($template.DirectoryName -ne $TemplateFolder) {
$newfolder = $OutputFolder + "/" + $template.Directory.Name
if (!(Test-Path $newfolder)) {
Write-Information ("Output folder for item does not exists creating the folder: $($newfolder)")
New-Item -Path $OutputFolder -Name $template.Directory.Name -ItemType "directory"
}
}
} else {
$newfolder = $OutputFolder
}
$templateContent = Get-Content $template.FullName -Raw -ErrorAction Stop
$templateObject = ConvertFrom-Json $templateContent -ErrorAction Stop
if (!$templateObject) {
Write-Error -Message ("ARM Template file is not a valid json, please review the template")
} else {
$outputFile = ("$($newfolder)/$($template.BaseName)$($templateNameSuffix)")
Out-File -FilePath $outputFile
if ($IncludeWikiTOC) {
("[[_TOC_]]`n") | Out-File -FilePath $outputFile
"`n" | Out-File -FilePath $outputFile -Append
}
if ((($templateObject | get-member).name) -match "metadata") {
if ((($templateObject.metadata | get-member).name) -match "Description") {
Write-Verbose ("Description found. Adding to parent page and top of the arm-template specific page")
("## Description") | Out-File -FilePath $outputFile -Append
$templateObject.metadata.Description | Out-File -FilePath $outputFile -Append
}
("## Information") | Out-File -FilePath $outputFile -Append
$metadataProperties = $templateObject.metadata | get-member | where-object MemberType -eq NoteProperty
foreach ($metadata in $metadataProperties.Name) {
switch ($metadata) {
"Description" {
Write-Verbose ("already processed the description. skipping")
}
Default {
("`n") | Out-File -FilePath $outputFile -Append
("**$($metadata):** $($templateObject.metadata.$metadata)") | Out-File -FilePath $outputFile -Append
}
}
}
}
("## Parameters") | Out-File -FilePath $outputFile -Append
# Create a Parameter List Table
$parameterHeader = "| Parameter Name | Parameter Type |Parameter Description | Parameter DefaultValue | Parameter AllowedValues |"
$parameterHeaderDivider = "| --- | --- | --- | --- | --- | "
$parameterRow = " | {0}| {1} | {2} | {3} | {4} |"
$StringBuilderParameter = @()
$StringBuilderParameter += $parameterHeader
$StringBuilderParameter += $parameterHeaderDivider
$StringBuilderParameter += $templateObject.parameters | get-member -MemberType NoteProperty | ForEach-Object { $parameterRow -f $_.Name , $templateObject.parameters.($_.Name).type , $templateObject.parameters.($_.Name).metadata.description, $templateObject.parameters.($_.Name).defaultValue , (($templateObject.parameters.($_.Name).allowedValues) -join ',' )}
$StringBuilderParameter | Out-File -FilePath $outputFile -Append
("## Resources") | Out-File -FilePath $outputFile -Append
# Create a Resource List Table
$resourceHeader = "| Resource Name | Resource Type | Resource Comment |"
$resourceHeaderDivider = "| --- | --- | --- | "
$resourceRow = " | {0}| {1} | {2} | "
$StringBuilderResource = @()
$StringBuilderResource += $resourceHeader
$StringBuilderResource += $resourceHeaderDivider
$StringBuilderResource += $templateObject.resources | ForEach-Object { $resourceRow -f $_.Name, $_.Type, $_.Comments }
$StringBuilderResource | Out-File -FilePath $outputFile -Append
if ((($templateObject | get-member).name) -match "outputs") {
write-verbose ("Output objects found.")
if (Get-Member -InputObject $templateObject.outputs -MemberType 'NoteProperty') {
("## Outputs") | Out-File -FilePath $outputFile -Append
# Create an Output List Table
$outputHeader = "| Output Name | Output Type | Output Value |"
$outputHeaderDivider = "| --- | --- | --- | "
$outputRow = " | {0}| {1} | {2} | "