Markdown generation for ARM and PowerShell

After my session at Azure LowLands with the title "You build It, You run It on the Microsoft Platform" a lot of people were interested in a script that I showed during the session.

The session itself is about developing a software package / product the DevOps way that also needs to be maintained after moving it to production. I tell a lot about how it should be done using 5 principles and my experience I had with multiple clients. During the demonstration I show Cloud Native resources in Azure that can be used in any kind of product or service.

Improvement of Daily work

One of the principles discussed is "Improvement of Daily work". This principle really makes our teams think about how they can improve the things we are doing and how that should be done. One of the thing that we did not like to do is write the documentation for our PowerShell scripts and ARM templates. For this we (Leon Boers and I) wrote some automation scripts to take care of this.

PowerShell

The script to generate the PowerShell documentation uses the "Help" that is specified within the script file.

<#
.SYNOPSIS
    Script for generating Markdown documentation based on information in PowerShell script files.

.DESCRIPTION
    All PowerShell script files have synopsis attached on the document. With this script markdown files are generated and saved within the target folder.

.PARAMETER ScriptFolder
    The folder that contains the scripts

.PARAMETER OutputFolder
    The folder were to safe the markdown files

.PARAMETER ExcludeFolders
    Exclude folder for generation. This is a comma separated list

.PARAMETER KeepStructure
    Specified to keep the structure of the subfolders

.PARAMETER IncludeWikiTOC
Include the TOC from the Azure DevOps wiki to the markdown files

.NOTES
    Version:        1.0.0;
    Author:         3fifty | Maik van der Gaag | Leon Boers;
    Creation Date:  20-04-2020;
    Purpose/Change: Initial script development;

.EXAMPLE
    .\New-MDPowerShellScripts.ps1 -ScriptFolder "./" -OutputFolder "docs/arm"  -ExcludeFolder ".local,test-templates" -KeepStructure $true -IncludeWikiTOC $false
.EXAMPLE
    .\New-MDPowerShellScripts.ps1 -ScriptFolder "./" -OutputFolder "docs/arm"
#>

The idea of using this we got from someone who was using the same principle on GitHub. We adopted the idea and created the below script.

[CmdletBinding()]

Param (
    [Parameter(Mandatory = $true, Position = 0)][string]$ScriptFolder,
    [Parameter(Mandatory = $true, Position = 1)][string]$OutputFolder,
    [Parameter(Mandatory = $false, Position = 2)][string]$ExcludeFolders,
    [Parameter(Mandatory = $false, Position = 3)][bool]$KeepStructure = $false,
    [Parameter(Mandatory = $false, Position = 4)][bool]$IncludeWikiTOC = $false
)

BEGIN {
    Write-Output ("ScriptFolder         : $($ScriptFolder)")
    Write-Output ("OutputFolder         : $($OutputFolder)")
    Write-Output ("ExcludeFolders       : $($ExcludeFolders)")
    Write-Output ("KeepStructure        : $($KeepStructure)")
    Write-Output ("IncludeWikiTOC       : $($IncludeWikiTOC)")

    $arrParameterProperties = @("DefaultValue", "ParameterValue", "PipelineInput", "Position", "Required")
    $scriptNameSuffix = ".md"
    $option = [System.StringSplitOptions]::RemoveEmptyEntries

    $exclude = $ExcludeFolders.Split(',', $option)

}
PROCESS {
    try {
        Write-Information ("Starting documentation generation for folder $($ScriptFolder)")

        if (!(Test-Path $OutputFolder)) {
            Write-Information ("Output path does not exists creating the folder: $($OutputFolder)")
            New-Item -ItemType Directory -Force -Path $OutputFolder
        }

        # Get the scripts from the folder
        $scripts = Get-Childitem $ScriptFolder -Filter "*.ps1" -Recurse

        foreach ($script in $scripts) {
            if (!$exclude.Contains($script.Directory.Name)) {
                Write-Information ("Documenting file: $($script.FullName)")


                if ($KeepStructure) {
                    if ($script.DirectoryName -ne $ScriptFolder) {
                        $newfolder = $OutputFolder + "/" + $script.Directory.Name
                        if (!(Test-Path $newfolder)) {
                            Write-Information ("Output folder for item does not exists creating the folder: $($newfolder)")
                            New-Item -Path $OutputFolder -Name $script.Directory.Name -ItemType "directory"
                        }
                    }
                } else {
                    $newfolder = $OutputFolder
                }

                $help = Get-Help $script.FullName -ErrorAction "SilentlyContinue" -Detailed

                if ($help) {
                    $outputFile = ("$($newfolder)/$($script.BaseName)$($scriptNameSuffix)")
                    Out-File -FilePath $outputFile

                    if ($IncludeWikiTOC) {
                        ("[[_TOC_]]`n") | Out-File -FilePath $outputFile
                        "`n" | Out-File -FilePath $outputFile -Append
                    }

                    #synopsis
                    if ($help.Synopsis) {
                        ("## Synopsis") | Out-File -FilePath $outputFile -Append
                        ("$($help.Synopsis)") | Out-File -FilePath $outputFile -Append
                        "`n" | Out-File -FilePath $outputFile -Append
                    } else {
                        Write-Warning -Message ("Synopsis not defined in file $($script.fullname)")
                    }

                    #syntax
                    if ($help.Syntax) {
                        ("``````PowerShell`n $($help.Syntax.syntaxItem.name)`n``````") | Out-File -FilePath $outputFile -Append
                        "`n" | Out-File -FilePath $outputFile -Append
                    } else {
                        Write-Warning -Message ("Syntax not defined in file $($script.fullname)")
                    }

                    #notes (separated by (name): and (value);)
                    if ($help.alertSet) {
                        ("## Information") | Out-File -FilePath $outputFile -Append
                        $text = $help.alertSet.alert.Text.Split(';', $option)
                        foreach ($line in $text) {
                            $items = $line.Trim().Split(':', $option)
                            ("**$($items[0]):** $($items[1])`n") | Out-File -FilePath $outputFile -Append
                        }
                        "`n" | Out-File -FilePath $outputFile -Append
                    } else {
                        Write-Warning -Message ("Notes not defined in file $($script.fullname)")
                    }

                    #description
                    if ($help.Description) {
                        "## Description" | Out-File -FilePath $outputFile -Append
                        $help.Description.Text | Out-File -FilePath $outputFile -Append
                        "`n" | Out-File -FilePath $outputFile -Append
                    } else {
                        Write-Warning -Message "Description not defined in file $($script.fullname)"
                    }

                    #examples
                    if ($help.Examples) {
                        ("## Examples") | Out-File -FilePath $outputFile -Append
                        "`n" | Out-File -FilePath $outputFile -Append
                        forEach ($item in $help.Examples.Example) {
                            $title = $item.title.Replace("--------------------------", "").Replace("EXAMPLE", "Example")
                            ("### $($title)") | Out-File -FilePath $outputFile -Append
                            if ($item.Code) {
                                ("``````PowerShell`r`n $($item.Code) `r`n``````") | Out-File -FilePath $outputFile -Append
                            }
                        }
                    } else {
                        Write-Warning -Message "Examples not defined in file $($script.fullname)"
                    }

                    if ($help.Parameters) {
                        ("## Parameters") | Out-File -FilePath $outputFile -Append
                        forEach ($item in $help.Parameters.Parameter) {
                            ("### $($item.name)") | Out-File -FilePath $outputFile -Append
                            $item.description[0].text | Out-File -FilePath $outputFile -Append
                            ("| | |") | Out-File -FilePath $outputFile -Append
                            ("|-|-|") | Out-File -FilePath $outputFile -Append
                            ("| Type: | $($item.Type.Name) |") | Out-File -FilePath $outputFile -Append
                            foreach ($arrParameterProperty in $arrParameterProperties) {
                                if ($item.$arrParameterProperty) {
                                    ("| $arrParameterProperty : | $($item.$arrParameterProperty)|") | Out-File -FilePath $outputFile -Append
                                }
                            }
                            "`n" | Out-File -FilePath $outputFile -Append
                        }
                    } else {
                        Write-Warning -Message "Parameters not defined in file $($script.fullname)"
                    }

                } else {
                    Write-Error -Message ("Synopsis could not be found for script $($script.FullName)")
                }
            }
        }
    } catch {
        Write-Error "Something went wrong while generating the output documentation: $_"
    }
}
END {}

Within this script you can see that we have added a different implementation for the "Notes" section. In the notes section we wanted to place more information and also be able to format in a specific way. That is why we created this format:

• [Name]: [Value];

This why we are able to add all kind of information and keep is nicely formatted. The generated documentation looks like:

ARM Templates

When we finished the documentation generation for PowerShell script files we thought that it is also quite easy to use the same principle for ARM templates.

The only thing we had to think of was the way we wanted to add additional information for the documentation. For this we added the "Metadata" property. This property does not violate the schema validation and additional properties could be added like description, version and author.

"metadata": {
   "Description": "This template deploys a standard storage account.",
   "Author": "3fifty | Maik van der Gaag | Leon Boers",
   "Version": "1.0.0"
}

The script looks quite the same as the PowerShell version but adopted the specific ideas for ARM.

[CmdletBinding()]

Param (
    [Parameter(Mandatory = $true, Position = 0)][string]$TemplateFolder,
    [Parameter(Mandatory = $true, Position = 1)][string]$OutputFolder,
    [Parameter(Mandatory = $false, Position = 2)][string]$ExcludeFolders,
    [Parameter(Mandatory = $false, Position = 3)][bool]$KeepStructure = $false,
    [Parameter(Mandatory = $false, Position = 4)][bool]$IncludeWikiTOC = $false
)


BEGIN {
    Write-Output ("TemplateFolder       : $($TemplateFolder)")
    Write-Output ("OutputFolder         : $($OutputFolder)")
    Write-Output ("ExcludeFolders       : $($ExcludeFolders)")
    Write-Output ("KeepStructure        : $($KeepStructure)")
    Write-Output ("IncludeWikiTOC       : $($IncludeWikiTOC)")

    $templateNameSuffix = ".md"
    $option = [System.StringSplitOptions]::RemoveEmptyEntries
    $exclude = $ExcludeFolders.Split(',', $option)
}
PROCESS {
    try {
        Write-Information ("Starting documentation generation for folder $($TemplateFolder)")

        if (!(Test-Path $OutputFolder)) {
            Write-Information ("Output path does not exists creating the folder: $($OutputFolder)")
            New-Item -ItemType Directory -Force -Path $OutputFolder
        }

        # Get the scripts from the folder
        $templates = Get-Childitem $TemplateFolder -Filter "*.json" -Recurse -Exclude "*parameters.json","*descriptions.json","*parameters.local.json"

        foreach ($template in $templates) {
            if (!$exclude.Contains($template.Directory.Name)) {
                Write-Information ("Documenting file: $($template.FullName)")

                if ($KeepStructure) {
                    if ($template.DirectoryName -ne $TemplateFolder) {
                        $newfolder = $OutputFolder + "/" + $template.Directory.Name
                        if (!(Test-Path $newfolder)) {
                            Write-Information ("Output folder for item does not exists creating the folder: $($newfolder)")
                            New-Item -Path $OutputFolder -Name $template.Directory.Name -ItemType "directory"
                        }
                    }
                } else {
                    $newfolder = $OutputFolder
                }

                $templateContent = Get-Content $template.FullName -Raw -ErrorAction Stop
                $templateObject = ConvertFrom-Json $templateContent -ErrorAction Stop

                if (!$templateObject) {
                    Write-Error -Message ("ARM Template file is not a valid json, please review the template")
                } else {
                    $outputFile = ("$($newfolder)/$($template.BaseName)$($templateNameSuffix)")
                    Out-File -FilePath $outputFile
                    if ($IncludeWikiTOC) {
                        ("[[_TOC_]]`n") | Out-File -FilePath $outputFile
                        "`n" | Out-File -FilePath $outputFile -Append
                    }

                    if ((($templateObject | get-member).name) -match "metadata") {
                        if ((($templateObject.metadata | get-member).name) -match "Description") {
                            Write-Verbose ("Description found. Adding to parent page and top of the arm-template specific page")
                            ("## Description") | Out-File -FilePath $outputFile -Append
                            $templateObject.metadata.Description | Out-File -FilePath $outputFile -Append
                        }

                        ("## Information") | Out-File -FilePath $outputFile -Append
                        $metadataProperties = $templateObject.metadata | get-member | where-object MemberType -eq NoteProperty
                        foreach ($metadata in $metadataProperties.Name) {
                            switch ($metadata) {
                                "Description" {
                                    Write-Verbose ("already processed the description. skipping")
                                }
                                Default {
                                    ("`n") | Out-File -FilePath $outputFile -Append
                                    ("**$($metadata):** $($templateObject.metadata.$metadata)") | Out-File -FilePath $outputFile -Append
                                }
                            }
                        }
                    }

                    ("## Parameters") | Out-File -FilePath $outputFile -Append
                    # Create a Parameter List Table
                    $parameterHeader = "| Parameter Name | Parameter Type |Parameter Description | Parameter DefaultValue | Parameter AllowedValues |"
                    $parameterHeaderDivider = "| --- | --- | --- | --- | --- | "
                    $parameterRow = " | {0}| {1} | {2} | {3} | {4} |"

                    $StringBuilderParameter = @()
                    $StringBuilderParameter += $parameterHeader
                    $StringBuilderParameter += $parameterHeaderDivider

                    $StringBuilderParameter += $templateObject.parameters | get-member -MemberType NoteProperty | ForEach-Object { $parameterRow -f $_.Name , $templateObject.parameters.($_.Name).type , $templateObject.parameters.($_.Name).metadata.description, $templateObject.parameters.($_.Name).defaultValue , (($templateObject.parameters.($_.Name).allowedValues) -join ',' )}
                    $StringBuilderParameter | Out-File -FilePath $outputFile -Append

                    ("## Resources") | Out-File -FilePath $outputFile -Append
                    # Create a Resource List Table
                    $resourceHeader = "| Resource Name | Resource Type | Resource Comment |"
                    $resourceHeaderDivider = "| --- | --- | --- | "
                    $resourceRow = " | {0}| {1} | {2} | "

                    $StringBuilderResource = @()
                    $StringBuilderResource += $resourceHeader
                    $StringBuilderResource += $resourceHeaderDivider

                    $StringBuilderResource += $templateObject.resources | ForEach-Object { $resourceRow -f $_.Name, $_.Type, $_.Comments }
                    $StringBuilderResource | Out-File -FilePath $outputFile -Append


                    if ((($templateObject | get-member).name) -match "outputs") {
                        write-verbose ("Output objects found.")
                        if (Get-Member -InputObject $templateObject.outputs -MemberType 'NoteProperty') {
                            ("## Outputs") | Out-File -FilePath $outputFile -Append
                            # Create an Output List Table
                            $outputHeader = "| Output Name | Output Type | Output Value |"
                            $outputHeaderDivider = "| --- | --- | --- |  "
                            $outputRow = " | {0}| {1} | {2} | "