Alternative Continuous Integration (CI) for BLT Projects

    See more
    • Updated

    Continuous Integration (CI) / Continuous Deployment (CD) is a critical part of any project. As such, Acquia's Build and Launch Tool (BLT) provides recipes out of the box for Travis CI and Acquia Pipelines as many of our customers rely on these products for their CI/CD solution(s). However, there are a significant number of solutions "out there" and we recognize that A: not everyone is going to use Travis CI or Acquia Pipelines and B: that we cannot possibly maintain recipes for "all the solutions." 

    This article will give you pointers for accomplishing the necessary tasks to use another CI/CD solution. At a high level, there are a few things that must be accomplished:

    1. Setup an environment in the build container that is similar to the Acquia cloud (e.g. Apache  not Nginx, correct PHP version, etc.)
    2. Execute build steps (e.g. install Drupal, execute automated tests)
    3. Build a deployment artifact 
    4. Authenticate and push the artifact into the Acquia cloud

    For reference, here is the Travis CI file that BLT generates for projects. 

    Environment Setup


    In this scenario, Travis allows for a Xenial container (which should closely align to Acquia's) -- https://github.com/acquia/blt/blob/11.x/scripts/travis/.travis.yml#L2. Each project should customize php version, but BLT defaults to 7.2 (at the time of this writing). Obviously, this should match the php version in local and cloud environments for the given project -- https://github.com/acquia/blt/blob/11.x/scripts/travis/.travis.yml#L7. MySQL is added (because obviously you need a database service -- https://github.com/acquia/blt/blob/11.x/scripts/travis/.travis.yml#L23) and the google-chrome browser is added for behat testing -- https://github.com/acquia/blt/blob/11.x/scripts/travis/.travis.yml#L28. 

    Executing Build Steps

    From here, we really get into the meat of the build. Note that many of the necessary build steps for Travis are executed using shell commands. For instance, the run tests command executes several key BLT commands in the shell script: 

    • blt validate: tests the quality of the code using linting and phpcs
    • blt setup: does a clean installation of Drupal, imports necessary configuration, compiles front end dependencies, etc.
    • blt tests: runs all automated tests from the project (including Behat, PHPUnit, etc.)

    Build and Deploy the Artifact

    In the Travis build, again, a shell command is called when merges are made into defined branches (or in the stock file, new tags). See https://github.com/acquia/blt/blob/11.x/scripts/travis/.travis.yml#L52. In this case, the shell command calls blt artifact:deploy. 

    Do it Yourself

    So! That brings us to setting up your own build process for <whatever other CI/CD tool you have>. First and foremost, familiarize yourself with the commands that BLT provides. Realistically, "most" of the scripting / build processes that you need to execute can be handled via BLT (and you shouldn't have to do a lot of custom drush or other commands on your own). In fact, you absolutely should rely on BLT for these scripts, as part of the benefit of BLT is that it automates all the things everywhere in the same order. You don't want your CI/CD process running things in a different order than your local and cloud processes! 

    High Level Overview of Commands

    For CI, I recommend the following:

    1. Setup environment (will vary)
    2. Install composer dependencies via composer install
    3. Validate code quality via blt validate
    4. Compile the front end via blt source:build:frontend
    5. Install Drupal via blt setup
    6. Execute tests via blt tests

    For CD, I recommend all of the above, plus blt deploy.

    Generate / Add SSH Keys

    First  things first, you need to generate a SSH key for the build executor and ensure that you can authenticate between your CI/CD solution and the Acquia Cloud. Acquia does have some specific requirements for ssh keys, and we always recommend separate keys (from your personal key) for build systems. 

    Each product handles authentication a little differently. Some allow you to authenticate at the product level, some require that encrypted keys be added to your repository and then decrypted during the build process. This step is a significant blocker to finishing the CI/CD process and we usually recommend starting here, as it can be quite challenging. 

    Create Build Script(s)

    You'll want to mirror the scripts that BLT provides for Acquia Pipelines and Travis CI. These shell scripts should be executable once committed to your repository and stored in an accessible location for your build process (although, the specific location is optional, we suggest /scripts in your repository root as it is outside of your docroot and not web accessible). 

    Example validate.sh file

    #!/usr/bin/env bash

    set​ -ev

    vendor/bin/blt validate:all --no-interaction

    set​ +v

    Create Build File

    Most CI/CD tools will give you a sample file to begin from. This file will serve as your environment definition file, as well as the order of operations that your scripts will be called in. Here is an example for Bitbucket Pipelines:

    image:​ ​php:7.3.8

    pipelines:
 default:

       - step:
       script:

    -​ ​apt-get​ ​update​ ​&&​ ​apt-get​ ​install​ ​-y​ ​unzip
    -​ ​apt-get​ ​install​ ​git​ ​-y
    -​ ​apt-get​ ​install​ ​-y​ ​libfreetype6-dev​ ​libjpeg62-turbo-dev​ ​libpng-dev
    -​ ​docker-php-ext-configure​ ​gd​ ​--with-freetype-dir=/usr/include/​ ​--with-jpeg-dir=/usr/include/ -​ ​docker-php-ext-install​ ​-j$(nproc)​ ​gd
    -​ ​curl​ ​-sS​ ​https://getcomposer.org/installer​ ​| php -- --install-dir=/usr/local/bin

    --filename=composer
         - bash scripts/bitbucket/build.sh

             - bash scripts/bitbucket/build_artifact.sh
         - bash scripts/bitbucket/deploy_tag.sh

    Define Build Behavior / Triggers

    Typically, you'll want to support both a CI and CD model with your builds. In the Github and Travis CI world this means that:

    • when a pull request is opened, Travis automatically executes a build (all build steps without the deploy step)
    • when a pull request is merged / a commit is made into a tracked branch OR a tag is created / pushed, Travis automatically executes a build including the deploy step

     

    Custom Variables

    The final piece of the puzzle is defining environment variables (e.g. https://github.com/acquia/blt/blob/11.x/scripts/travis/.travis.yml#L9). Each build system is going to be a little different on this topic. Some examples may be:

    • Composer Bin
    • BLT Directory
    • Docroot
    • Cache Directories for Drush and Composer
    • etc.

    Unfortunately, these may take some trial and error based on the build system. Use the Travis and Pipelines examples provided by BLT as a guide!

     

    Acquia Products

    Topics

    Related articles