This article explains how Qualibrate scenarios can be triggered from external tool such as AzureDevops CI/CD pipelines, in order to automate the process of test execution in the context of DevOps.
The below example is using AzureDevOps CI/CD capabilities, but this is not limited to only AzureDevOps. Similar approaches can be used on other CI/CD solutions such as GitLab, Jenkins, Atlassian Bamboo, CircleCI, etc.
There are 3 primary components on this integration:
Azure DevOps: Responsible to job orchestration (Pipelines).
Qualibrate CI Utility: Qualibrate’s official command line interface (CLI) utility responsible to abstracting test execution instructions into simple commands that can be triggered in a CI/CD pipeline
Qualibrate Cloud Service: Responsible for receiving execution jobs and reporting back results to the Qualibrate CLI. All target test scenarios are stored in Qualibrate, and are organized into projects.
The follow schematic visualizes how the 3 components are interacting with each other:
Pipeline configuration file
Azure CI/CD pipelines can be configured using a YAML file within each project. For more detailed information about how to configure Azure DevOps CI/CD pipelines, visit official Pipeline Configuration Reference documentation.
The following is a
azure-pipelines.yml example which already contains some Qualibrate-specific commands:
# Starter pipeline # Start with a minimal pipeline that you can customize to build and deploy your code. # Add steps that build, run tests, deploy, and more: # https://aka.ms/yaml trigger: - master pool: vmImage: 'ubuntu-latest' steps: - script: sudo apt-get update && sudo apt-get install -y wget && sudo rm -rf /var/lib/apt/lists/* displayName: 'Install `wget` to properly download binary from url' - script: sudo wget -q https://assets.qualibrate.com/downloads/qualibrate-cli displayName: 'Download `qualibrate-cli` binary from url' - script: sudo chmod +x qualibrate-cli displayName: 'Make `qualibrate-cli` executable (required for unix environments)' - script: ./qualibrate-cli exec -t=$(type) -i=$(id) -p=$(project) -k=$(api_key) -a=$(adapter) -e=$(execution_mode) $(verbose) displayName: 'Execute command'
If you would like to learn all the different options to use variables in a pipeline context, please refer to the official Azure Pipelines Documentation.
Qualibrate CLI Command Reference
PS C:\CLI> .\qualibrate-cli.exe --help Usage: qualibrate-cli [command] Available Commands: exec command line option to perform execution of Flow OR Scenario help Help about any command Flags: -h, --help help for qualibrate-cli -s, --stage Target stage environment Use "qualibrate-cli [command] --help" for more information about a command.
Execution command reference
To run scenarios, the
qualibrate-cli.exe exec command is used. The following is the help reference for that command:
PS C:\CLI> .\qualibrate-cli.exe exec --help Usage: qualibrate-cli exec [flags] Flags: -a, --adapter string adapter id -h, --help help for exec -i, --id string id of the target flow or scenario -k, --key string api key to access platform -p, --project string project id -t, --target string value can be 'flow' or 'scenario' Global Flags: -s, --stage Target stage environment
How to get the Qualibrate IDs?
The CLI requires 2 important variables to trigger executions from 3rd party tools;
project id and
To get these values, navigate to the target scenarios that will be executed. Both id values can be found in the browser URL as follows: