At this point, you have all your code built and ready to be tested. We will use TaskCat, an open source tool developed by AWS Quick Start team to simplify and automate the testing of AWS CloudFormation templates. It test the CloudFormation templates by deploying it in multiple AWS Regions simultaneously and generates a report with a pass/fail result for each region. You can customize the tests through a test config file.
It’s a best practice to use virtual environment when working with python projects.
TaskCat requires python3. Therefore, we will create a virtual environment with python3 as default interpreter before installing TaskCat.
Run following commands, to create a virtual environment and use python3 as default interpreter for virtual environment.
cd ~/environment/ virtualenv -p /usr/bin/python36 vpy36 source vpy36/bin/activate
python --version, and you should see the output as below:
(vpy36) Admin:~/environment $ python --version Python 3.6.x
TaskCat can be installed with Docker or by using pip. We will use pip for this workshop.
From your terminal, run the following command.
pip install taskcat
This will install the latest TaskCat version 9.
After the installation is finished, run
taskcat --version to ensure that TaskCat version 9 is installed correctly.
TaskCat requires a project configuration file to configure details about the project and define tests.
This file is created at <PROJECT_ROOT>/.taskcat.yml.
There are 2 configuration sections in this file as shown below:
project: name: my-cfn-project regions: - us-west-2 - eu-north-1 tests: default: template: ./templates/my-template.yaml
project section contains the project specific configuration. At minimum, it should have following 2 configurations:
tests section is where you define the test related configuration for your project. At minimum, it should have the following:
For the workshop, open the qs-workshop/.taskcat.yml file and copy-paste the following contents in it, and save.
project: name: qs-workshop regions: - ap-southeast-1 - us-east-1 - us-west-2 - eu-central-1 tests: default: template: ./templates/master.template.yaml regions: - us-west-1
TaskCat has several configuration files which can be used to set behaviors in a flexible way and you can read the details in the TaskCat documentation
When testing your CloudFormation templates, you need to pass parameter values to use for creating a stack. With TaskCat you can pass these parameter values from configuration file, described above. If you don’t specify parameter values in the TaskCat configuration file, you should have default values defined in the template itself for all the parameters.
As you can see in the qs-workshop/.taskcat.yml file, there are no parameter values defined. To specify the parameter values, close the .taskcat.yml file and run the following command.
cd ~/environment/qs-workshop/ curl -s https://raw.githubusercontent.com/aws-quickstart/quickstart-workshop-labs/master/implementing/.taskcat.yml >>.taskcat.yml
Your .taskcat.yml file should look like below.
project: name: qs-workshop regions: - ap-southeast-1 - us-east-1 - us-west-2 - eu-central-1 tests: default: template: ./templates/master.template.yaml regions: - us-west-1 parameters: AvailabilityZones: "$[taskcat_getaz_2]" EmailAddress: email@example.com KeyPairName: YOUR-KEYPAIR-HERE WebserverCIDR: "0.0.0.0/0" QSS3KeyPrefix: "qs-workshop/" QSS3BucketName: "$[taskcat_autobucket]" QSS3BucketRegion: "$[taskcat_current_region]"
You need to replace the YOUR-KEYPAIR-HERE with your KeyPairName.
As you can notice, there are few parameter values which is in the format $[taskcat_*]. These are the identifiers which are used to auto-generate the values at runtime by TaskCat. For the complete list of pre-defined identifiers, see the TaskCat documentation.
As you can see above, there are few parameters like KeyPairName and EmailAddress which you may not want to hardcode a value and check it into the github repository. For such situations, TaskCat provide capability to override the project configuration and parameter values via override files. Following are the two override files supported by TaskCat:
To understand the precedence of values in different configuration files, read the Taskcat documentation
For this workshop, create a project override file and add the parameter values for the Email address and Key pair name. Your file should look like below:
EmailAddress: firstname.lastname@example.org KeyPairName: YOUR-KEYPAIR-HERE
Now that we have taskcat configuration file created, let’s run TaskCat to execute our tests.
Run the following command in your terminal window:
taskcat test run &> screen-logs.txt &
This will run TaskCat in the background and send logs/errors to screen-logs.txt file. TaskCat performs series of actions as part of executing a test, such as template validation, parameter validation, staging content into S3 bucket, and launching CloudFormation stack. It launches the stack creation in all the defined regions, for each test, simultaneously. And regularly polls the CloudFormation stack status to check if the stack creation is finished. How much time TaskCat takes to finish the testing, depends on how many tests you have defined in your TaskCat configuration file and how long each stack creation and deletion takes.
You can tail the logs by running following command.
tail -f screen-logs.txt
After the TaskCat run is complete, you will see a report genereated in HTML format in the current directory from where you are running TaskCat command. You can see the report by right click on taskcat_outputs/index.html, and click preview. Your report should look like below:
If you do not see all green, you can click on the View Logs link for the failed test and see the CloudFormation event logs for further troubleshooting.