For my current engagement, we have review apps (“recette jetable”): throwaway containers generated dynamically with a singe click out of a specific git branch on our PaaS, which comes with their own sample database and subdomain.
This code explains this feature in detail, with all the code we use. It’s open-source anyway, under license GNU Affero.
I think is a very interesting but underused feature for projects. For us, it’s a very convenient, low-cost way to show a new feature to other devs, designers or stakeholders before merging it. We use this a lot for demos and QA, in order for quick adjustements before the launch of complicated features.
We use Github Actions for automation, with Clever-Cloud as a PaaS for hosting −it’s great!−.
A key element is Clever-Cloud’s utility clever tools. It allows to create/update/delete a machine on the fly through the command line. With some modifications to the code in this article, you can adapt it to other PaaSes that provide similar features.
Credits goes to my colleague Celine who did all this, I’m simply sharing this idea.
The feature described here is not similar to ngrok or localtunnel. They expose your localhost on the internet, which is useful too (say, for remote pair programming). However, with those solutions, if your code change locally (because you work on something else) the local tunnel will reflect those changes.
Deploying an branch on an external container allows you to do something else with your local machine. Some of our review apps live for multiple days, so work is not interrupted.
The way it works is as follow. There are 3 parts that we’ll discuss in this article:
The code for this (as part of itou) has a lot of boilerplate (and may appear complex because yaml is very verbose), but it is surprisingly concise given how many things happen.
A dev opens a PR for his branch (my-awesome-feature), and attachs a label recette-jetable. A github action catchs the event “label recette-jetable has been added”. Inside the github action, it then:
clever tools and logs to our account. The credentials are stored in a Github Secret.clever tools to create a new Python instance, and attach all it needs to run the project: a configuration addon, a database…# See https://developer.github.com/v3/ and https://help.github.com/en/actions
name: 🕵 Review app creation
on:
pull_request:
types: [ labeled ] # run this pipeline when a label is added to a PR
env:
CLEVER_TOOLS_DOWNLOAD_URL: https://clever-tools.clever-cloud.com/releases/latest/clever-tools-latest_linux.tar.gz
CLEVER_TAR_FILE: clever-tools-latest_linux.tar.gz
CLEVER_CLI: clever-tools-latest_linux/clever
CLEVER_TOKEN: $
CLEVER_SECRET: $
REVIEW_APPS_ORGANIZATION_NAME: …
BRANCH: $
PYTHON_VERSION: 3.9
jobs:
create:
runs-on: ubuntu-latest
if: github.event.action == 'labeled' && github.event.label.name == 'recette-jetable'
steps:
- name: 📥 Checkout to the PR branch
uses: actions/checkout@v2
with:
ref: $
- name: 📥 Fetch git branches
run: git fetch --prune --unshallow
- name: 🏷 Set some necessary environnment variables
run:
echo "REVIEW_APP_NAME=`echo \"c1-review-$BRANCH\" | sed -r 's/[-;\\/._]+/-/g'`" >> $GITHUB_ENV
echo "REVIEW_APP_DB_NAME=`echo $REVIEW_APP_NAME | sed -r 's/-/_/g'`" >> $GITHUB_ENV
echo "DEPLOY_URL=`echo \"$REVIEW_APP_NAME.cleverapps.io\"`" >> $GITHUB_ENV
- name: 🧫 Create a review app on Clever Cloud
run: |
curl $CLEVER_TOOLS_DOWNLOAD_URL > $CLEVER_TAR_FILE
tar -xvf $CLEVER_TAR_FILE
$CLEVER_CLI login --token $CLEVER_TOKEN --secret $CLEVER_SECRET
# Create a new application on Clever Cloud.
# -t: application type (Python).
# --org: organization name.
# --region: server location ("par" means Paris).
# --alias: custom application name, used to find it with the CLI.
$CLEVER_CLI create $REVIEW_APP_NAME -t python --org $REVIEW_APPS_ORGANIZATION_NAME --region par --alias $REVIEW_APP_NAME
$CLEVER_CLI env set CC_PYTHON_VERSION $ --alias $REVIEW_APP_NAME
$CLEVER_CLI domain add $DEPLOY_URL --alias $REVIEW_APP_NAME
$CLEVER_CLI link $REVIEW_APP_NAME --org $REVIEW_APPS_ORGANIZATION_NAME
- name: 🗃 Create database addon
run: |
$CLEVER_CLI addon create postgresql-addon $REVIEW_APP_DB_NAME --org $REVIEW_APPS_ORGANIZATION_NAME --plan xxs_sml --yes
$CLEVER_CLI service link-addon $REVIEW_APP_DB_NAME
- name: 🗺 Add environment variables to the review app
run: |
$CLEVER_CLI link $REVIEW_APP_NAME --org $REVIEW_APPS_ORGANIZATION_NAME
$CLEVER_CLI service link-addon $CONFIGURATION_ADDON
$CLEVER_CLI env import-vars REVIEW_APP_DB_NAME
$CLEVER_CLI env import-vars DEPLOY_URL
- name: 🚀 Deploy to Clever Cloud
run: $CLEVER_CLI deploy --branch $BRANCH --force
- name: 🍻 Add link to pull request
uses: thollander/actions-comment-pull-request@main
with:
message: "🥁 La recette jetable est prête ! [👉 Je veux tester cette PR !](https://$)"
GITHUB_TOKEN: $
The code is running through a github action, and asks Clever Cloud to setup a machine for us through clever tools. The twist is on what triggers this action, then it’s mostly configuration and running commands. Some things to note:
Those containers are not free, but since there is very little trafic we create the smallest possible instances. This feature costs us in total less than 100€/month for hosting, which is quite a bargain given how useful it is.
We need to populate the database with initial data, and with some sample user accounts. Fixtures are stored with the project. A small piece of code triggers their import. It is referenced in the shared review app configuration (on clever-cloud’s end) through a post-deploy hook:
CC_RUN_SUCCEEDED_HOOK="$APP_HOME/clevercloud/review-app-after-success.sh"
Initial database data can be either a SQL dump or django fixtures.
#!/bin/sh
###################################################################
###################### Review apps entrypoint #####################
###################################################################
# Skip this step when redeploying a review app.
if [ "$SKIP_FIXTURES" = true ] ; then
echo "Skipping fixtures."
exit
fi
echo "Loading cities"
PGPASSWORD=$POSTGRESQL_ADDON_PASSWORD pg_restore -d $POSTGRESQL_ADDON_DB -h $POSTGRESQL_ADDON_HOST -p $POSTGRESQL_ADDON_PORT -U $POSTGRESQL_ADDON_USER --if-exists --clean --no-owner --no-privileges $APP_HOME/itou/fixtures/postgres/cities.sql
# `ls $APP_HOME` does not work as the current user does not have execution rights on the $APP_HOME directory.
echo "Loading fixtures"
ls -d $APP_HOME/itou/fixtures/django/* | xargs django-admin loaddata
When a developper pushes a fix to such a branch already attached to a review app:
# See https://developer.github.com/v3/
# and https://help.github.com/en/actions
name: 🕵 Review app update
# Run this pipeline when a label is present and when a push is made on this PR.
# `types: [ synchronize ]` targets a push event made on a PR.
on:
pull_request:
types: [ synchronize ]
env:
CLEVER_TOOLS_DOWNLOAD_URL: https://clever-tools.clever-cloud.com/releases/latest/clever-tools-latest_linux.tar.gz
CLEVER_TAR_FILE: clever-tools-latest_linux.tar.gz
CLEVER_CLI: clever-tools-latest_linux/clever
CLEVER_TOKEN: $
CLEVER_SECRET: $
REVIEW_APPS_ORGANIZATION_NAME: …
BRANCH: $
PYTHON_VERSION: 3.9
jobs:
redeploy:
runs-on: ubuntu-latest
# A push event when the 'recette-jetable' label is present triggers a new deployment.
if: github.event.action == 'synchronize' && contains( github.event.pull_request.labels.*.name, 'recette-jetable')
steps:
- name: 📥 Checkout to the PR branch
uses: actions/checkout@v2
with:
ref: $
- name: 📥 Fetch git branches
run: git fetch --prune --unshallow
- name: 🏷 Set review app name
run: echo "REVIEW_APP_NAME=`echo \"c1-review-$BRANCH\" | sed -r 's/[-;\\/._]+/-/g'`" >> $GITHUB_ENV
- name: 🤝 Find the application on Clever Cloud
run: |
curl $CLEVER_TOOLS_DOWNLOAD_URL > $CLEVER_TAR_FILE
tar -xvf $CLEVER_TAR_FILE
$CLEVER_CLI login --token $CLEVER_TOKEN --secret $CLEVER_SECRET
$CLEVER_CLI link $REVIEW_APP_NAME --org $REVIEW_APPS_ORGANIZATION_NAME
- name: ⏭ Skip fixtures
run: $CLEVER_CLI env set SKIP_FIXTURES true
- name: 🚀 Deploy to Clever Cloud
run: $CLEVER_CLI deploy --branch $BRANCH --force
When a user closes a PR or removes the “recette-jetable” label, a github action catches the event and destroy all the containers used here.
name: 🔪 Review app removal
# We’ll run this pipeline:
# - when a pull request having the label "review-app" is closed
# - when the "recette-jetable" label is removed
on:
pull_request:
types: [ unlabeled, closed ]
env:
CLEVER_TOOLS_DOWNLOAD_URL: https://clever-tools.clever-cloud.com/releases/latest/clever-tools-latest_linux.tar.gz
CLEVER_TAR_FILE: clever-tools-latest_linux.tar.gz
CLEVER_CLI: clever-tools-latest_linux/clever
CLEVER_TOKEN: $
CLEVER_SECRET: $
REVIEW_APPS_ORGANIZATION_NAME: $
BRANCH: $
jobs:
delete:
runs-on: ubuntu-latest
if: github.event.label.name == 'recette-jetable' || contains( github.event.pull_request.labels.*.name, 'recette-jetable')
steps:
- name: 📥 Checkout to the PR branch
uses: actions/checkout@v2
- name: 🏷 Set environnment variables
run:
echo "REVIEW_APP_NAME=`echo \"c1-review-$BRANCH\" | sed -r 's/[-;\\/._]+/-/g'`" >> $GITHUB_ENV
echo "REVIEW_APP_DB_NAME=`echo $REVIEW_APP_NAME | sed -r 's/-/_/g'`" >> $GITHUB_ENV
- name: 🤝 Find the application on Clever Cloud
run: |
curl $CLEVER_TOOLS_DOWNLOAD_URL > $CLEVER_TAR_FILE
tar -xvf $CLEVER_TAR_FILE
$CLEVER_CLI login --token $CLEVER_TOKEN --secret $CLEVER_SECRET
$CLEVER_CLI link $REVIEW_APP_NAME --org $REVIEW_APPS_ORGANIZATION_NAME
- name: 🗑 Delete the review app and its database
run: |
$CLEVER_CLI delete --yes
$CLEVER_CLI addon delete $REVIEW_APP_DB_NAME --org $REVIEW_APPS_ORGANIZATION_NAME --yes
That’s it for today! Hopefully this is clear enough.