Background

This article explains how Qualibrate scenarios can be triggered from external tool such as AzureDevops CI/CD pipelines, in order to automate the process of test execution in the context of DevOps.

The below example is using AzureDevOps CI/CD capabilities, but this is not limited to only AzureDevOps. Similar approaches can be used on other CI/CD solutions such as GitLab, Jenkins, Atlassian Bamboo, CircleCI, etc.

Architecture

There are 3 primary components on this integration:

• Azure DevOps: Responsible to job orchestration (Pipelines).

• Qualibrate CI Utility: Qualibrate’s official command line interface (CLI) utility responsible to abstracting test execution instructions into simple commands that can be triggered in a CI/CD pipeline

• Qualibrate Cloud Service: Responsible for receiving execution jobs and reporting back results to the Qualibrate CLI. All target test scenarios are stored in Qualibrate, and are organized into projects.

The follow schematic visualizes how the 3 components are interacting with each other:

Pipeline configuration file

Azure CI/CD pipelines can be configured using a YAML file within each project. For more detailed information about how to configure Azure DevOps CI/CD pipelines, visit official Pipeline Configuration Reference documentation.

Example

The following is a azure-pipelines.yml example which already contains some Qualibrate-specific commands:

# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml

trigger:
- master

pool:
  vmImage: 'ubuntu-latest'

steps:

- script: sudo apt-get update && sudo apt-get install -y wget && sudo rm -rf /var/lib/apt/lists/*
  displayName: 'Install `wget` to properly download binary from url'

- script: sudo wget -q https://assets.qualibrate.com/downloads/qualibrate-cli
  displayName: 'Download `qualibrate-cli` binary from url'
  
- script: sudo chmod +x qualibrate-cli
  displayName: 'Make `qualibrate-cli` executable (required for unix environments)'
  
- script: ./qualibrate-cli exec -t=$(type) -i=$(id) -p=$(project) -k=$(api_key) -a=$(adapter) -e=$(execution_mode) $(verbose)
  displayName: 'Execute command'

If you would like to learn all the different options to use variables in a pipeline context, please refer to the official Azure Pipelines Documentation.

Qualibrate CLI Command Reference

Global Reference

PS C:\CLI> .\qualibrate-cli.exe --help

Usage:
  qualibrate-cli [command]

Available Commands:
  exec        command line option to perform execution of Flow OR Scenario
  help        Help about any command

Flags:
  -h, --help    help for qualibrate-cli
  -s, --stage   Target stage environment

Use "qualibrate-cli [command] --help" for more information about a command.

Execution command reference

To run scenarios, the qualibrate-cli.exe exec command is used. The following is the help reference for that command:

PS C:\CLI> .\qualibrate-cli.exe exec --help

Usage:
  qualibrate-cli exec [flags]

Flags:
  -a, --adapter string   adapter id
  -h, --help             help for exec
  -i, --id string        id of the target flow or scenario
  -k, --key string       api key to access platform
  -p, --project string   project id
  -t, --target string    value can be 'flow' or 'scenario'

Global Flags:
  -s, --stage   Target stage environment

How to get the Qualibrate IDs?

The CLI requires 2 important variables to trigger executions from 3rd party tools; project id and scenario id.

To get these values, navigate to the target scenarios that will be executed. Both id values can be found in the browser URL as follows: