Learn/test GitHub Actions for running CI/CD.
To start, add a directory in your repo at .github/workflows
. GitHub will
automatically detect this when it's pushed.
This directory is populated with YAML (*.yml
) file that define different
automated workflows.
A basic workflow definition file example:
name: Build and Test
on:
pull_request:
branch:
- master
push:
branch:
- master
jobs:
ci:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: '3.x'
- name: Install dependencies
run:
python -m pip install --upgrade pip
pip install black hypothesis pytest
- name: Run tests
run:
python -m pytest
- name: Check syntax
run:
black --check .
The on
block defines what actions will trigger this workflow. In this
example the workflow will be triggered by either a push or a pull request to
the master
branch.
The jobs
block defines the workflow. Here we have one job called ci
, which
is defined to run on the latest version of Ubuntu, and which consists of four
steps (which are run sequentially). Each step is given a (human-readable) name
,
and a sequence of commands to run in the run
block. The first two steps set
up the Python environment in which to run the tests, and then the latter two
steps use PyTest to run the provided unit tests, and
Black to check the Python
formatting, respectively.
The uses
lines bring in preset GitHub Actions; first to check out the
repository, which is defined at a level so that it is applied across all steps;
and, specifically in the 'Set up Python' step, a pre-defined Action is used to
make Python available.
Setting up a matrix allows you to run a job in parallel in multiple different environments, e.g. operating systems, dependency versions, etc.
For example, rather than specifying a single OS via:
ci:
runs-on: ubuntu-latest
as above, the 'matrix' Strategy can be defined like so:
ci:
strategy:
matrix:
os: [ubuntu-latest, macos-latest]
runs-on: ${{ matrix.os }}
so that the ci
job is run on all of the specified operating systems.
The "matrix" of environments can have multiple dimensions; e.g. here we test over a list of Python versions as well as over the list of operating systems:
ci:
strategy:
matrix:
os: [ubuntu-latest, macos-latest]
python-version: [3.6, 3.7, 3.8]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v2
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
GitHub allows you to create 'Secrets' (as long as your are an admin of the repository), to supply workflows with confidential info like usernames and passwords.
GitHub's docs say:
Secrets are environment variables that are encrypted and only exposed to selected actions. Anyone with collaborator access to this repository can use these secrets in a workflow.
Secrets are not passed to workflows that are triggered by a pull request from a fork.
Consider the following workflow, to publish a Python package to PyPI (taken directly from the GitHub examples):
# This workflows will upload a Python Package using Twine when a release is
# created
name: Upload Python Package
on:
release:
types: [created]
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: '3.x'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install setuptools wheel twine
- name: Build and publish
env:
TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }}
TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
run: |
python setup.py sdist bdist_wheel
twine upload dist/*
Here it is clear in the 'Build and publish' step that the PyPI username and
password required for Twine to
upload the package to PyPI are contained in the secrets
data structure that
GitHub makes available to the workflow.
On the menu bar of the repository, click 'Settings' and then click 'Secrets' in the left pane.
A pre-made GitHub Action is just a Git repo. An example of the pre-defined
GitHub Actions is the setup-python
action used in the above and found here:
https://github.com/actions/setup-python.
You can create your own generic GitHub actions and publish them for others to
use; all you need to do is define an action.yml
file in the root of the
repository.
name: "Generic Action to install <dependency>"
description: "A generic action to be shared on GitHub for other workflows to use"
branding:
icon: ...
color: "gray-dark"
inputs:
<dependency>-version:
description: "The version of <dependency> to install"
required: true
default: "1.0"
runs:
using: ...
steps:
- run: |
pip install <dependency>==${{ inputs.<dependency>-version }}
shell: bash
To publish the Action, create a Release in the repository.
Other people can use this by adding:
- uses: <GITHUB_USER>/<REPO_NAME>@master
GitHub Actions: https://github.com/features/actions
GitHub Actions docs (with yml definitions): https://docs.github.com/en/actions
GitHub Actions Marketplace: https://github.com/marketplace?type=actions to browse existing solutions