Background

This article explains how Qualibrate scenarios can be triggered from external tool such as CI/CD servers, in order to automate the process of test execution in the context of DevOps.

The below example is using Jenkins CI/CD capabilities, but this is not limited to only Jenkins. Similar approaches can be used on other CI/CD solutions such as Azure DevOps, GitLab, Atlassian Bamboo, CircleCI, etc.

Architecture

There are 3 primary components on this integration:

• Jenkins: Responsible to job orchestration (Pipelines).

• Qualibrate CI Utility: Qualibrate’s official command line interface (CLI) utility responsible to abstracting test execution instructions into simple commands that can be triggered in a CI/CD pipeline

• Qualibrate Cloud Service: Responsible for receiving execution jobs and reporting back results to the Qualibrate CLI. All target test scenarios are stored in Qualibrate, and are organized into projects.

The follow schematic visualizes how the 3 components are interacting with each other:

Pipeline configuration file

Jenkins Pipeline is a suite of plugins that supports implementing and integrating continuous delivery pipelines into Jenkins.

Here is an example of how to setup a pipeline in Jenkins. (not using Docker, although we do recommend it for scaling purposes): https://www.jenkins.io/doc/book/pipeline/getting-started/.

If you decide to use Docker, please read this first: https://www.jenkins.io/doc/book/pipeline/docker/ 

Jenkins file example

What the following script does is spin up an ubuntu container, set environment variables, starts a testing stage by using the qualibrate-cli commands.

pipeline {
    agent {
        docker {
            image 'ubuntu:latest'
        }
    }
    
environment {
        GIT_STRATEGY = 'none'
        ADAPTER_ID = 'WIN-R4VQP0JPDBF-Administrator'
        COMMAND_TYPE = 'scenario'
        SCENARIO_ID = '5ede3cd446e0fb0001bf1b47'
        PROJECT_ID = '5e5786c8c9e77c0001dc4aa8'
    }

    stages {
        stage('Test') {
            steps {
                sh 'apt-get update && apt-get install -y wget && rm -rf /var/lib/apt/lists/*'
                sh 'wget -q https://assets.qualibrate.com/downloads/qualibrate-cli'
                sh 'chmod +x qualibrate-cli'
                sh './qualibrate-cli exec -i ${SCENARIO_ID} -a ${ADAPTER_ID} -t ${COMMAND_TYPE} -k ${API_KEY_PRD} -p ${PROJECT_ID}'
            }
        }
    }
}

Qualibrate CLI Command Reference

Global Reference

PS C:\CLI> .\qualibrate-cli.exe --help

Usage:
  qualibrate-cli [command]

Available Commands:
  exec        command line option to perform execution of Flow OR Scenario
  help        Help about any command

Flags:
  -h, --help    help for qualibrate-cli
  -s, --stage   Target stage environment

Use "qualibrate-cli [command] --help" for more information about a command.

Execution command reference

To run scenarios, the qualibrate-cli.exe exec command is used. The following is the help reference for that command:

PS C:\CLI> .\qualibrate-cli.exe exec --help

Usage:
  qualibrate-cli exec [flags]

Flags:
  -a, --adapter string   adapter id
  -h, --help             help for exec
  -i, --id string        id of the target flow or scenario
  -k, --key string       api key to access platform
  -p, --project string   project id
  -t, --target string    value can be 'flow' or 'scenario'

Global Flags:
  -s, --stage   Target stage environment

How to get the Qualibrate IDs?

The CLI requires 2 important variables to trigger executions from 3rd party tools; project id and scenario id.

To get these values, navigate to the target scenarios that will be executed. Both id values can be found in the browser URL as follows: