Configuring GitHub Actions for Automated Python Testing in the CI / CD Pipeline
On the eve of the start of the course “Python QA Engineer” we traditionally publish translations of useful material.
We also invite you to join the open webinar on the topic “API Testing Automation”.
This article describes the operations for testing the client-side of an application using TestProject and pytestas well as ways to run tests through GitHub Actions. If you have a public GitHub repository, it’s all completely free. This feature is well suited for exploring TestProject and performing integration testing on your projects. If you want to perform these operations from a closed repository, then GitHub offers a very large number of free minutes, see. https://github.com/features/actions#pricing-details…
GitHub repository cards
First, I decided to explore the features available in the GitHub repositories, since they seemed to me quite simple to understand, and besides, I had not used it for a long time. Selenium…
I have pinned my favorite repos at the top of my site waylonwalker.com… The information to populate these cards is dynamically pulled to the client side via the GitHub API. This means that as the pages load, JavaScript executes scripts to fetch information from the GitHub API and then converts that data to the DOM and renders it on the page. This is what the GitHub repository cards look like:
Obtaining keys
First of all, you will need TPDEVTOKEN and TPAPIKEY… These keys will allow TestProject to access your account to automatically post results to your reporting panels…
In your GitHub repository, go to settings> Secrets (or add settings / secrets to your repository url) and add your secret tokens. GitHub will have secure access to tokens, while they will not be available to the general public (including project contributors), will not appear in log files, etc.
Setting up the development environment
To speed things up, I set up a development environment at Digital Ocean. This is optional: everything can work from your local computer or entirely from GitHub Actions. I just decided that by setting up Droplet with Ubuntu at Digital Ocean, I would get conditions close to the production environment, which means I could develop my tests faster. It also allowed me to make all my tests run a bit faster than running them through GitHub. The process was almost the same as when using GitHub. This way, I was able to learn in detail the principles of setting up TestProject without having to do a full installation every time I run GitHub Actions.
I will not go into detail about setting up a development machine here. You can read my notes on setting it up here: https://waylonwalker.com/notes/new-machine-tpio…
Pytest
All tests done with Pytest are given in github…
I decided to use Pytest. I loved the idea of using fixtures, automatically executing my test functions, and using some of Pytest’s features to generate reports as we go (and the TestProject platform would work without a testing framework like Pytest).
NOTE… Following the standard Pytest guidelines, I named the tests directory tests. In general, everything works, but the TestProject.io platform uses this directory as the default project name. If I could go back, I would either rename the directory to the name I want to see on TestProject.io, or I would give the project name in the configuration.
conftest.py
You can see the file conftest.py on GitHub.
In file conftest.py usually hosts fixtures that are used by multiple modules. Pytest will automatically import all modules conftest.py from the directory you are working in. This is a great place to place fixtures with TestProject drivers. Note, when you use a fixture as an argument in another function, the fixture will do the setup, pass everything from the statement yield into a test function, execute the test function, and then release resources.
conftest.py stores fixtures for all modules in the directory.
# tests/conftest.py
import time
import pytest
from src.TestProject.sdk.drivers import web driver
@pytest.fixture
def driver():
"creates a webdriver and loads the homepage"
driver = webdriver.Chrome()
driver.get("https://waylonwalker.com/")
yield driver
driver.quit()The above example is a bit simplified… In the real version, I ran into some inconsistencies and found that the passing percentage of some tests was higher when adding the time.sleep operator. In the full project, I settled on the driver and slow fixturesdriver. Thus, I still have a driver that waits a little longer for JavaScript to execute.
testrepos.py
Full version testrepos.py available on GitHub.
Initially, I set up three different tests for the repository cards. I formed a list of repositories that should have been displayed in cards. These tests are fairly easy to do with the TestProject.io framework as it uses Selenium and a headless browser to execute JavaScript. The REPOS area is created here as a global list. It can be easily converted into a configuration file if the need arises.
For those who do not know, I will say that headless browser works like a regular browser, only without a graphical user interface. JavaScript is fully loaded and parsed, and all interaction with the DOM is done programmatically.
Read the docstrings for each function. They describe what happens at each step.
"""
Test that GitHub repo data dynamically loads the client-side.
"""
REPOS = [
"find-kedro",
"kedro-static-viz",
"kedro-action",
"steel-toes",
]
def test_repos_loaded(slow_driver):
"""
Test that each repo-name exists as a title in one of the repo cards.
On waylonwalker.com repo cards have a title with a class of "repo-name"
"""
repos = slow_driver.find_elements_by_class_name("repo-name")
# get innertext from elements
header_text = [
header.text for header in repos
]
for repo in REPOS:
assert repo in header_text
def test_repo_description_loaded(slow_driver):
"""
Test that each repo has a description longer than 10 characters
On waylonwalker.com repo cards have a descriptiion with a class of "repo-description"
"""
repo_elements = slow_driver.find_elements_by_class_name("repo")
for el in repo_elements:
desc = el.find_element_by_class_name("repo-description")
assert len(desc.text) > 10
def test_repo_stars_loaded(slow_driver):
"""
Ensure that stars are correctly parsed from the API and loaded client-side
On waylonwalker.com repo cards have a stars element with a class of "repo-stars" and
is displayed as "n stars"
"""
repo_elements = slow_driver.find_elements_by_class_name("repo")
for el in repo_elements:
stars = el.find_element_by_class_name("repo-stars")
num_stars, label = stars.text.split()
assert int(num_stars) > 0
assert label == 'stars'Forum
I am a little confused with setting up TestProject.io in Actions. On TestProject forum I quickly found an answer with a link to exactly the example I needed. The example was written in Java, but it had the steps I needed to set up docker-compose.
GitHub Actions
So, the GitHub repository is set up, and mine tests work successfully in Pytest. Now let’s make them automatically run in GitHub Actions.
The Actions service is a GitHub solution for CI / CD. It allows you to run your code on a GitHub-managed virtual machine that can get additional information from your repository, such as the secret tokens we set up at the beginning. What, how and when is executed – all this is configured in the file yaml…
# .github/workflows/test-waylonwalker-com.yml
name: Test WaylonWalker.com
# Controls when the action will run. Triggers the workflow on push or pull request
# events but only for the master branch
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
schedule:
- cron: '*/10 * * * *'As you can see in the above piece of code, I have set the action to be executed when changes are pushed to the main branch (push command) or when I create a pull request affecting this branch. I also set a fairly aggressive test execution schedule – every ten minutes… This was necessary to make sure the tests were effective and to get more data for analysis in the reports. Later, I will most likely increase this interval.
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@main
- uses: actions/setup-python@v2
with:
python-version: '3.8'
- run: pip install -r requirements.txt
- name: Run TestProject Agent
env:
TP_API_KEY: ${{ secrets.TP_API_KEY }} # < Let Secrets handle your keys
run: |
envsubst < .github/ci/docker-compose.yml > docker-compose.yml
cat docker-compose.yml
docker-compose -f docker-compose.yml up -d
- name: Wait for Agent to Register
run: bash .github/ci/wait_for_agent.sh
- run: pytest
env:
TP_DEV_TOKEN: ${{ secrets.TP_DEV_TOKEN }} # < Let Secrets handle your tokens
TP_AGENT_URL: http://localhost:8585In the code for the test job, you can see that I decided to run it on ubuntu-latest. The first three steps are fairly boilerplate steps: switching to the repository branch, installing Python 3.8, and installing dependencies from the requirements.txt file via pip. Then the key TPAPIKEY substituted in docker-compose.yml using envsubst, then docker-compose starts and waits for the agent to be ready. I also provided pytest with our token TPDEVTOKEN and ran pytest.
docker-compose.yml
Next file docker-compose.yml was provided Vitaly Bukhovsky (one of the co-founders and security director of TestProject) in the repository testproject-io / java-sdk… This is where a template with a key is configured TPAPIKEY as a variable for envsubst, the headless Chrome and Firefox browsers, and the TestProject.io agent.
version: "3.1"
services:
testproject-agent:
image: testproject/agent:latest
container_name: testproject-agent
depends_on:
- chrome
- firefox
environment:
TP_API_KEY: "${TP_API_KEY}"
TP_AGENT_TEMP: "true"
TP_SDK_PORT: "8686"
CHROME: "chrome:4444"
CHROME_EXT: "localhost:5555"
FIREFOX: "firefox:4444"
FIREFOX_EXT: "localhost:6666"
ports:
- "8585:8585"
- "8686:8686"
chrome:
image: selenium/standalone-chrome
volumes:
- /dev/shm:/dev/shm
ports:
- "5555:4444"
firefox:
image: selenium/standalone-firefox
volumes:
- /dev/shm:/dev/shm
ports:
- "6666:4444"Waiting for agent registration
In my opinion, the most interesting part of the above workflow is how we expect the agent to register. The shell script is pretty laconic. It detects that the number of startup attempts exceeded (max_attempts) or the presence of a running agent by checking the status via the REST API (address: / api / status). This allows us to avoid wasting time caused by setting too long a wait time or starting a process too early, in which pytest runs when the agent is not yet running.
trap 'kill $(jobs -p)' EXIT
attempt_counter=0
max_attempts=100
mkdir -p build/reports/agent
docker-compose -f docker-compose.yml logs -f | tee build/reports/agent/log.txt&
until curl -s http://localhost:8585/api/status | jq '.registered' | grep true; do
if [ ${attempt_counter} -eq ${max_attempts} ]; then
echo "Agent failed to register. Terminating..."
exit 1
fi
attempt_counter=$(($attempt_counter+1))
echo
sleep 1
doneTestProject reporting panel
After running the tests, they are displayed in the TestProject panel. In the early stages of test development, there were a few glitches, but after troubleshooting, the tests run stably.
Panel: passing one test
The reporting panel allows you to find the individual tests that have been performed, select them, and view reports for each test. It automatically converts the steps taken by the driver into human-readable process diagram, while each step can be opened and viewed the values that were received by the driver from the site.
You can read more about TestProject dashboards at the following links:
This tutorial was created by Waylon Walker based on his article Integration Testing with Python, TestProject.io and GitHub Actions…
Happy testing!
– Learn more about the course “Python QA Engineer”.
– Sign up for an open webinar “API Testing Automation”.
