CI/CD Platform Comparison: CircleCI, Codefresh, GitLab, GitHub Actions

Anais Urlichs -

This blog post showcases the same step in four different CI/CD pipelines. At the end of the blog post, I will provide my thoughts on the different pipelines.

You can find the example project that I am using for this blog post here: GitHub repository.

Note that the selection of CI/CD platforms is based on the most common and similar ones. I did not include something like Jenkins because I feel like Jenkins is a platform that only certain types of businesses use (not me).

Thank you for reading, have a lovely day!

Prerequisites

The step that I will demonstrate on each CI/CD platform will be for using Datree. Datree helps us to prevent Kubernetes misconfigurations from reaching production. You want to make sure whenever you change your Kubernetes Manifests that you check those are configured correctly – the best way to ensure that is by including a Datree step in your pipeline.

In order to run Datree as a step in each of the pipelines, you need to have the Account Token from your Datree account and set it as an environment variable in your CI/CD platform.

To create a Datree account, either sign-up through the website or install their CLI and run the first test command:

curl https://get.datree.io | /bin/bash

datree test ~/.datree/k8s-demo.yaml

This will provide you with a link to your account.

Note that we will use the exact same commands in our CI/CD pipelines.

GitLab CI/CD

Here is the link to the site.

Prerequisites

You add the variable in the following section: Settings > CI/CD > Variables

Your project then needs a gitlab-ci.yml file at its root. Once you have added it, the job may run automatically:

image: node:14

stages:
    - test

test:
    stage: test
    script:
        - curl <https://get.datree.io> | /bin/bash
        - datree test ~/.datree/k8s-demo.yaml

This is the output of the pipeline:

As you can see, 13 rules have been evaluated in total of which 2 rules failed. Note that this is using the Datree example. You would have to modify the path in the datree test command to a Kubernetes Manifest in your repository.

Note that this test command ran against my staging policy set up in Datree. Datree allows you to set up different policies for different use cases or environments. Please have a look at one of my previous videos for more details.

GitHub Actions

Prerequisites

To add the Datree account token to our environment variables, we are going to open the repository. Then we go to the following path: Settings > Secrets or Settings > Environments > Secrets

Next, you would add a .github folder with another folder called workflows in which you post your workflow:

name: CI

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]
    
env:
  DATREE_TOKEN: ${{ secrets.DATREE_TOKEN }} 

jobs:
  k8s-policy-check:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: Install Datree
        run: curl <https://get.datree.io> | /bin/bash
        
      - name: Run Datree's policy check
        run: datree test ~/.datree/k8s-demo.yaml

Running the pipeline looks something like this:

Note that GitHub Actions ran with the Datree default policy – that's why 21 policy rules got evaluated in total.

Circle CI

First, like with so many other things in life, you need to set up an account. This is pretty straightforward. You can choose to give CircleCI a project or skip right away to the platform. Let’s see what happens next.

Prerequisites

version: 2.1
jobs:
  build:
    docker:
      - image: circleci/node
    steps:
      - checkout
      - run: npm run build
  test:
    docker:
      - image: circleci/node
    steps:
      - checkout
      - run: curl <https://get.datree.io> | /bin/bash
      - run: datree test ~/.datree/k8s-demo.yaml

workflows:
  main:
    jobs:
      - build
      - test

Since we have not provided the required Account token in the variables, the workflow will not run.

So add Datree to your environment variable:

And then the job should run:

The output of our Datree test command is the same.

Codefresh

To use Codefresh you first need, like with any other CI/CD platform, an account. Once you have an account, you can link your project to it. Once you linked your project to it, you can then define the pipeline directly in the platform. It is very straightforward.

Prerequisites

Start a new project on your Codefresh account and create a new pipeline. Within the pipeline page, you can add any environment variables:

# More examples of Codefresh YAML can be found at
# <https://codefresh.io/docs/docs/yaml-examples/examples/>

version: "1.0"
# Stages can help you organize your steps in stages
stages:
  - "clone"
  - "test"

steps:
  clone:
    title: "Cloning repository"
    type: "git-clone"
    repo: "AnaisUrlichs/ci-cd-example"
    # CF_BRANCH value is auto set when pipeline is triggered
    # Learn more at codefresh.io/docs/docs/codefresh-yaml/variables/
    revision: "${{CF_BRANCH}}"
    git: "github"
    stage: "clone"

  test:
    title: "Running test"
    type: "freestyle" # Run any command
    image: "node:17" # The image in which command will be executed
    working_directory: "${{clone}}" # Running command where code cloned
    commands:
      - "curl <https://get.datree.io> | /bin/bash"
      - "datree test ~/.datree/k8s-demo.yaml"
    stage: "test"

Note that we do not need the clone step in this case. Usually, you would use the clone step to get access to your GH repository.

Once you run the pipeline, it will look something like this:

Comparison

To be honest, after trying out all four and having more experience with GitLab and Codefresh, here is what I would be doing:

  1. Using GitLab CI if my project is hosted on GitLab
  2. Using GitHub Actions if my project is hosted on GitHub
  3. Using Codefresh if I need anything more extensive or if I want to have lots of cloud native features

A lot of the things that you can do with Codefresh, you can also do with GitLab. However, I find it often too complex to figure it out on GitLab.

I might be too honest here but I don’t get for what I would use CircleCI. The platform does not seem to give me anything that GitHub or GitLab cannot. On the other hand, while I don’t see what value CircleCI is adding, I think Codefresh is providing too many different features. This can easily become irritating when people try to use multiple features in combination. Yes, lots of features are great but they all have to be maintained & people have to be educated on how to best use them.

Some platforms praise themselves with easy yaml configuration for the pipelines — I think this is missing the point. People will have to figure out how to configure the YAML either way — the best thing that you can do is to provide lots of examples that people can use. However, “easy YAML” is a highly subjective statement. I would say while GitHub has the most examples provided by the community, Codefresh has the best, most comprehensive documentation.

Sharing is Caring

If you like my content, please give me a shout-out on social media. I would highly appreciate it.

Also, make sure to sign-up for my weekly DevOps newsletter in which I share free online learning resources from across the DevOps space right to your inbox.