Running Powershell on AWS Lambda

How to run a simple PowerShell function on AWS Lambda. It retrieves data from DESTATIS database via its REST API and stores the tabular data in S3.

Prerequisites

This post assumes you have

an AWS account,
an AWS role that permits executing AWS Lambda functions and accessing S3.

How to install and prepare PowerShell is explained in the next section (or in AWS Lambda Developer Guide). Although I’m going to explain how this works on Windows, note that PowerShell 7.0 runs on .NET Core 3.1 which is available for Linux and MacOS as well.

Setting Up PowerShell On Your System

Scripts for PowerShell 6.0 and higher can be run in AWS Lambda’s .NET Core 3.1 (C#/PowerShell) runtime. The latest version is available in github and for this task I have used version 7.0.2. For Windows users it will be easiest to use the MSI installer package PowerShell-7.0.2-win-x64.msi. Note that Windows users need to install this package as well, since Windows PowerShell, which is shipped with Windows 10, is not the same as PowerShell 7.0.

Furthermore, to be able to develop PowerShell apps, you need to have the .NET 3.1 SDK on your machine. You can download installers and binaries from Microsoft. Make sure to get the SDK and not just the runtime.

Once you’ve installed both packages, open a PowerShell 7 terminal (not Windows PowerShell!) and install modules for working with AWS with the following commands:

Install-Module AWSLambdaPSCore -Scope CurrentUser
Install-Module -Name AWS.Tools.Common -MinimumVersion 4.0.6.0
Install-Module -Name AWS.Tools.Lambda -MinimumVersion 4.0.6.0
Install-Module -Name AWS.Tools.S3 -MinimumVersion 4.0.6.0

For more information on these packages, see this post on the AWS Developer Blog. In short, these modules contain the minimum packages needed to create AWS Lambda functions that can access S3. AWSLambdaPSCore provides four cmdlets that come in handy, three of which we will use in the following:

Get-AWSPowerShellLambdaTemplate – lists templates
New-AWSPowerShellLambda – creates an initial PowerShell script based on a template
New-AWSPowerShellLambdaPackage – creates a Lambda package (a .zip file)

The PowerShell 7.0 Function

Create the project

Creating a function for AWS lambda is easiest if we start from a template. You can list all available templates with

Get-AWSPowerShellLambdaTemplate

but we will use the bare bone Basic template in this exercise:

New-AWSPowerShellLambda -ScriptName destatis-table-download -Template Basic -Directory aws-lambda

This creates a new directory in your working environment (aws-lambda) with a script called destatis-table-download.ps1 based on the Basic template.

Write the function

First, we need to declare all dependencies for our script. Here, we will need the Lambda and S3 as well as the Common module from the AWS toolkit:

#Requires -Modules @{ModuleName='AWS.Tools.Common';ModuleVersion='4.0.6.0'}
#Requires -Modules @{ModuleName='AWS.Tools.Lambda';ModuleVersion='4.0.6.0'}
#Requires -Modules @{ModuleName='AWS.Tools.S3';ModuleVersion='4.0.6.0'}

Since we don’t want to hard code our credentials for DESTATIS and also allow to flexibly provide other parameters via environment variables, let’s map these local variables:

$format=$env:format
$table=$env:table
$username=$env:username
$password=$env:password
$s3bucket=$env:s3bucket
$s3key=$env:s3key

The first two are needed to tell DESTATIS which table we want and in what format to provide it (see my recent post on the new REST API for details)). Furthermore, we provide our credentials and the s3 destination via environment variables:

Environment Variables in AWS Lambda UI

Next, we build the URL for the DESTATIS REST API:

$baseurl="https://www-genesis.destatis.de/genesisWS/rest/2020"
$service="data"
$method="tablefile"
$url="$($baseurl)/$($service)/$($method)?username=$username&password=$password&name=$table&format=$format"

and send the request using the Invoke-RestMethod cmdlet:

$response=Invoke-RestMethod -Method 'Get' -Uri $url

This cmdlet returns a string, which is stored in variable response.

Finally, we create a file in S3 and set the content of the file to the value of response:

Write-S3Object -BucketName $s3bucket -Key $s3key -Content $response

This cmdlet is provided by the AWS Tools for PowerShell package (see docs).

Package and Deployment

To package your script with all dependencies for AWS lambda, switch to the project folder and run the New-AWSPowerSehllLambdaPackage cmdlet:

cd aws-lambda
New-AWSPowerShellLambdaPackage -ScriptPath .\destatis-table-download.ps1 -OutputPackage destatis-table-download.zip

After a couple of seconds you should see a message that tells you how to deploy this package in AWS Lambda, most notably, how to specify the lambda handler. In our case it is

destatis-table-download::destatis_table_download.Bootstrap::ExecuteFunction

Now, upload the .zip file to your lambda function in AWS and change the lambda handler to the value above (in Basic Settings).

Test the Function

As our function does not process any information from input events, you can test it using the default Hello World test event. If it runs successfully, you will see a new/updated file in your S3 bucket containing the DESTATIS table.

Appendix

Code

# dependencies
#Requires -Modules @{ModuleName='AWS.Tools.Common';ModuleVersion='4.0.6.0'}
#Requires -Modules @{ModuleName='AWS.Tools.Lambda';ModuleVersion='4.0.6.0'}
#Requires -Modules @{ModuleName='AWS.Tools.S3';ModuleVersion='4.0.6.0'}

# variables from environment
$format=$env:format
$table=$env:table
$username=$env:username
$password=$env:password
$s3bucket=$env:s3bucket
$s3key=$env:s3key

# build the url
$baseurl="https://www-genesis.destatis.de/genesisWS/rest/2020"
$service="data"
$method="tablefile"
$url="$($baseurl)/$($service)/$($method)?username=$username&password=$password&name=$table&format=$format"

# request
$response=Invoke-RestMethod -Method 'Get' -Uri $url

# upload to S3
Write-S3Object -BucketName $s3bucket -Key $s3key -Content $response