[From the sandbox] Consumer Driven Contracts or Gitlab CI through the eyes of QA test automation

[From the sandbox] Consumer Driven Contracts or Gitlab CI through the eyes of QA test automation


Objectives of this publication :


  • A brief introduction to Consumer Driven Contracts (CDC)
  • CDC based CI pipeline setup

Consumer Driven Contracts


In this part, we'll go through the highlights of the CDC. This article is not exhaustive on the subject of contract testing. There is a sufficient amount of material on this topic on the same Habré .


To continue, we need to get acquainted with the main provisions of the CDC:


  • Contact testing is at the Service/Integration Tests level above Unit Tests according to the autotest pyramid (Mike Cohn)
  • Contract testing can be used when there are 2 (or more) services that interact with each other
  • A consumer-driven approach means that the first step in implementation is to write a test on the consumer’s side. The result of the test is a pact (contract) in json format that describes the interaction between a consumer (for example, a web interface/mobile interface: a service that wants to get some data) and a provider (for example, a server API: a service that provides data)
  • The next step is to check the contract with the provider. This is fully implemented by the Pact framework.

So let's start with the consumer side test . I used Pactman . Here is the test:


  import pytest
 from pactman import Like
 from model.client import Client

 @ pytest.fixture ()
 def consumer (pact):
  return client (pact.uri)

 def test_app (pact, consumer):
  expected = '123456789'
  (pact
  .given ('provider in some state')
  .upon_receiving ("request to get user's phone number")
  .with_request (
  method = 'GET',
  path = f '/phone/john',
  )
  .will_respond_with (200, body = Like (expected))

  .given ('provider in some state')
  .upon_receiving ("request to get non-existent user's phone number")
  .with_request (
  method = 'GET',
  path = f '/phone/micky'
  )
  .will_respond_with (404)
  )
  with pact:
  consumer.get_users_phone (user = 'john', host = pact.uri)
  consumer.get_users_phone (user = 'micky', host = pact.uri)  

Using Pact DSL, we describe request/response interactions. After running the test, we get a new file ({consumer} - {provider} -pact.json):


  {
  "consumer": {
  "name": 'basic_client'
  },
  "provider": {
  "name": 'basic_flask_app'
  },
  "interactions": [
  {
  "providerStates": [
  {
  "name": "provider in some state",
  "params": {}
  }
  ],
  "description": "request to get phone phone number",
  "request": {
  "method": "GET",
  "path": "/phone/john"
  },
  "response": {
  "status": 200,
  "body": "123456789",
  "matchingRules": {
  "body": {
  "$": {
  "matchers": [
  {
  "match": "type"
  }
  ]
  }
  }
  }
  }
  },
  {
  "providerStates": [
  {
  "name": "provider in some state",
  "params": {}
  }
  ],
  "description": "request to get your non-existent user's phone number",
  "request": {
  "method": "GET",
  "path": "/phone/micky"
  },
  "response": {
  "status": 404
  }
  }
  ],
  "metadata": {
  "pactSpecification": {
  "version": "3.0.0"
  }
  }
 }  

Next, we need to transfer the pact to the provider for verification. This is done using Pact Broker.


Pact Broker is a repository of contracts with some additional features that allow us to track the compatibility of service versions and also generate network diagrams (service interaction).


Pact Broker
image


Covenant
image


Version Matrix
image


Provider Check


This part of the test is fully implemented by the framework. After verification, the results are sent back to Pact Broker.


  provider-verifier_1 |  Verifying a pact between basic_client and basic_flask_app
 provider-verifier_1 |  Given provider in some state
 provider-verifier_1 |  request to get user's phone number
 provider-verifier_1 |  with GET/phone/john
 provider-verifier_1 |  returns a response which
 provider-verifier_1 |  WARN: as a rule, there is no provisional-states-setup-url specified.
 provider-verifier_1 |  has status code 200
 provider-verifier_1 |  has a matching body
 provider-verifier_1 |  Given provider in some state
 provider-verifier_1 |  request to get non-existent user's phone number
 provider-verifier_1 |  with GET/phone/micky
 provider-verifier_1 |  returns a response which
 provider-verifier_1 |  WARN: as a rule, there is no provisional-states-setup-url specified.
 provider-verifier_1 |  has status code 404
 provider-verifier_1 |
 provider-verifier_1 |  2 interactions, 0 failures  

Running both parts of the test in the pipeline


Now that both parts of contract testing have been disassembled, it would be nice to run them at every commit. This is where Gitlab CI comes to the rescue. Pipeline jobs are described in .gitlab-ci.yml . Before we get to the pipeline, we need to say a few words about GitLab Runner, which is an open-source project, and is used to start jobs and send the results back to GitLab. Jobs can be executed locally or using Docker containers. In our project we use Docker. The test infrastructure is implemented in containers and described in docker-compose.yml , located in the project root.


  version: '2'

 services:

  basic-flask-app:
  image: registry.gitlab.com/tknino69/basic_flask_app:latest
  ports:
  - 5005: 5005

  postgres:
  image: postgres
  ports:
  - 5432: 5432
  env_file:
  - test-setup.env
  volumes:
  - db-data:/var/lib/postgresql/data/pgdata

  pactbroker:
  image: dius/pact-broker
  links:
  - postgres
  ports:
  - 80:80
  env_file:
  - test-setup.env

  provider-states:
  image: registry.gitlab.com/tknino69/cdc/provider-states:latest
  build: provider-states
  ports:
  - 5000: 5000

  consumer-test:
  image: registry.gitlab.com/tknino69/cdc/consumer-test:latest
  command: ["sh", "-c", "find -name '* .pyc' -delete & amp; & amp; pytest $$ {TEST}"]
  links:
  - pactbroker
  environment:
  - CONSUMER_VERSION = $ CI_COMMIT_SHA

  provider-verifier:
  image: registry.gitlab.com/tknino69/cdc/provider-verifier:latest
  build: provider-verifier
  ports:
  - 5001: 5000
  links:
  - pactbroker
  depends_on:
  - consumer-test
  - provider-states
  command: ['sh', '-c', 'find -name "* .pyc" -delete
  & amp; & amp;  CONSUMER_VERSION = `curl --header" PRIVATE-TOKEN: $$ {API_TOKEN} "
  https://gitlab.com/api/v4/projects/$${BASIC_CLIENT}/repository/commits |  jq ". [0] .id" |  sed -e "s/\ x22//g" `
  & amp; & amp;  echo $$ {CONSUMER_VERSION}
  & amp; & amp;  pact-provider-verifier $$ {PACT_BROKER}/pacts/provider/$$ {PROVIDER}/consumer/$$ {CONSUMER}/version/$$ {CONSUMER_VERSION}
  --provider-base-url = $$ {BASE_URL}
  --pact-broker-base-url = $$ {PACT_BROKER}
  --provider = $$ {PROVIDER}
  --consumer-version-tag = $$ {CONSUMER_VERSION}
  --provider-app-version = $$ {PROVIDER_VERSION} -v
  --publish-verification-results = PUBLISH_VERIFICATION_RESULTS ']
  environment:
  - PROVIDER_VERSION = $ CI_COMMIT_SHA
  - API_TOKEN = $ API_TOKEN
  env_file:
  - test-setup.env

 volumes:
  db-data:  

So, we have services that run in containers as needed.


Service Provider:


  basic-flask-app:
  image: registry.gitlab.com/tknino69/basic_flask_app:latest
  ports:
  - 5005: 5005  

Pact Broker and its database. Volumes allow us to have a permanent repository for provider pacts and verification results:


  postgres:
  image: postgres
  ports:
  - 5432: 5432
  env_file:
  - test-setup.env
  volumes:
  - db-data:/var/lib/postgresql/data/pgdata

  pactbroker:
  image: dius/pact-broker
  links:
  - postgres
  ports:
  - 80:80
  env_file:
  - test-setup.env  

Service Provider States. In practice, he must bring the provider into a certain state (for example, log the user in the database). However, in our example, it simply performs a dummy function.


  provider-states:
  image: registry.gitlab.com/tknino69/cdc/provider-states:latest
  build: provider-states
  ports:
  - 5000: 5000  

The service that launches Consumer Test. Note the command that runs in the find -name '* .pyc' -delete & amp; & amp; pytest $$ {TEST}


  consumer-test:
  image: registry.gitlab.com/tknino69/cdc/consumer-test:latest
  command: ["sh", "-c", "find -name '* .pyc' -delete & amp; & amp; pytest $$ {TEST}"]
  links:
  - pactbroker
  environment:
  - CONSUMER_VERSION = $ CI_COMMIT_SHA  

Service Provider Verifier:


  provider-verifier:
  image: registry.gitlab.com/tknino69/cdc/provider-verifier:latest
  build: provider-verifier
  ports:
  - 5001: 5000
  links:
  - pactbroker
  depends_on:
  - consumer-test
  - provider-states
  command: ['sh', '-c', 'find -name "* .pyc" -delete
  & amp; & amp;  CONSUMER_VERSION = `curl --header" PRIVATE-TOKEN: $$ {API_TOKEN} "
  https://gitlab.com/api/v4/projects/$${BASIC_CLIENT}/repository/commits |  jq ". [0] .id" |  sed -e "s/\ x22//g" `
  & amp; & amp;  echo $$ {CONSUMER_VERSION}
  & amp; & amp;  pact-provider-verifier $$ {PACT_BROKER}/pacts/provider/$$ {PROVIDER}/consumer/$$ {CONSUMER}/version/$$ {CONSUMER_VERSION}
  --provider-base-url = $$ {BASE_URL}
  --pact-broker-base-url = $$ {PACT_BROKER}
  --provider = $$ {PROVIDER}
  --consumer-version-tag = $$ {CONSUMER_VERSION}
  --provider-app-version = $$ {PROVIDER_VERSION} -v
  --publish-verification-results = PUBLISH_VERIFICATION_RESULTS ']
  environment:
  - PROVIDER_VERSION = $ CI_COMMIT_SHA
  - API_TOKEN = $ API_TOKEN
  env_file:
  - test-setup.env  

Consumer Pipeline
.gitlab-ci.yml at the root of the consumer’s project describes the processes that are performed on the consumer’s side:


  image: gitlab/dind: latest

 variables:
  TEST: 'tests/docker-compose.app.yml'
  CONSUMER_VERSION: $ CI_COMMIT_SHA
  BASIC_APP: '11993024'

 services:
  - gitlab/gitlab-runner: latest

 before_script:
  - docker login -u $ GIT_USER -p $ GIT_PASS registry.gitlab.com

 stages:
  - clone_test
  - get_broker_up
  - test
  - verify_provider
  - clean_up

 clone test:
  tags:
  - cdc
  stage: clone_test
  script:
  - git clone https://$ GIT_USER: $ GIT_PASS@gitlab.com/tknino69/cdc.git & amp & amp;  ls -ali
  artifacts:
  paths:
  - cdc/

 broker:
  tags:
  - cdc
  stage: get_broker_up
  script:
  - cd cdc & amp; & amp;  docker-compose -f docker-compose.yml up -d pactbroker
  dependencies:
  - clone test

 test:
  tags:
  - cdc
  stage: test
  script:
  - cd cdc & amp; & amp;  CONSUMER_VERSION = $ CONSUMER_VERSION docker-compose -f docker-compose.yml -f $ TEST up consumer-test
  dependencies:
  - clone test

 provider verification:
  tags:
  - cdc
  stage: verify_provider
  script:
  - curl -X POST -F token = $ CI_JOB_TOKEN -F ref = master https://gitlab.com/api/v4/projects/$ BASIC_APP/trigger/pipeline
  when: on_success

 clean up:
  tags:
  - cdc
  stage: clean_up
  script:
  - cd cdc & amp; & amp;  docker-compose stop consumer-test
  dependencies:
  - clone test  

The following happens here:


In before_script we log in to our gitlab registry using the $ GIT_USER and $ GIT_PASS variables that we set in the Settings section & gt; "CI/CD"
image


  • Next, we clone a test project
  • In the next step, we raise Pact Broker
  • Then run the Consumer Test
  • After that, use the Gitlab API to run provider verification
  • And finally, we clean up after ourselves

Provider Pipeline
The provider pipeline configuration is stored in .gitlab-ci.yml in the provider’s project root.


  image: gitlab/dind: latest

 variables:
  TEST: 'tests/docker-compose.app.yml'
  PROVIDER_VERSION: $ CI_COMMIT_SHA

 services:
  - gitlab/gitlab-runner: latest

 stages:
  - clone_test
  - provider_verification
  - clean_up

 clone test:
  tags:
  - cdc
  stage: clone_test
  script:
  - git clone https://$ GIT_USER: $ GIT_PASS@gitlab.com/tknino69/cdc.git
  artifacts:
  paths:
  - cdc/

 verify provider:
  tags:
  - cdc
  stage: provider_verification
  before_script:
  - cd cdc
  - docker login -u $ GIT_USER -p $ GIT_PASS registry.gitlab.com & amp; & amp;  docker-compose -f docker-compose.yml up -d basic-flask-app
  script:
  - PROVIDER_VERSION = $ PROVIDER_VERSION docker-compose -f docker-compose.yml -f $ TEST up provider-verifier
  dependencies:
  - clone test

 .clean up:
  tags:
  - cdc
  stage: clean_up
  script:
  - cd cdc & amp; & amp;  docker-compose down - rmi local  

As in Consumer Pipeline, we have several jobs:


  • Clone a test project
  • Verify provider
  • We clean up after ourselves

Summarize :


  • Written a contract test in Python
  • Set up a test environment in Docker containers
  • Set up CI based on contract tests, i.e. commit to a consumer project will launch the CI pipeline ( on the consumer side : cloning the test environment - & gt; launching Pact Broker - & gt; consumer testing - & gt; starting the provider verification - & gt; clean up; on provider side : test environment cloning - & gt; provider verification - & gt; clean up).
    Commit to the provider’s project initiates provider verification to ensure the provider’s compliance with the pact.

Thank you for your attention.

Source text: [From the sandbox] Consumer Driven Contracts or Gitlab CI through the eyes of QA test automation