Continuous Integration with circleCI

Goals

By the end of this lesson, you will:

  • Understand what continuous integration is and why it’s important
  • Be familiar with a typical deployment flow for agile teams
  • Be able to set up CircleCI for continuous integration and automatic deployments for front-end or back-end applications.

Important Vocabulary

  • local environment: A version of an application that a developer can edit and run on their own machine (i.e. an application run on localhost).
  • production environment: A deployed version of an application that the end user can interact with (i.e. a final project deployed on Heroku or Fly.io)
  • lower environments: Any environments that are internal to a team/company and are used for development purposes. Lower environments usually include a testing/QA environment that’s used for testing and/or a staging environment that’s used for showcasing features before a release. Lower environments are not viewable by the end user.
  • deployment: The process of making software (with new features and changes) available in a target environment
  • deployment pipeline: A system of automated processes (often managed by a CI tool) responsible for facilitating the building, testing and deployment of code from a version control system to production.

For further reading on environments and the history of multi-environment deployment processes, refer to this lesson

What is Continuous Integration

Continuous integration (CI) is the practice of merging code to a main branch (“integrating” code) regularly and verifying the quality of merged code with the help of automated build and testing tools. This practice developed out of the pain and struggles developers felt after dealing with problems integrating different branches and features that had been developed in isolation over the course of weeks or even months. In order to properly practice continuous integration on software teams, developers should diligently monitor these automated tools so that when an error is detected, they are able to promptly fix it. Development teams often find that this practice allows them to more easily and quickly deploy integrated code with fewer bugs.

Continuous integration is not the same as continuous delivery, but it is closely connected. Continous delivery means keeping your application in a “deployable state” so that it can be released at any time. CD is often associated with releasing smaller changesets more often, rather than huge releases at a less frequent cadence. You cannot practice continuous delivery without also practicing continous integration. Many teams these days will rely on some sort of automated service to handle testing these changesets and facilitating frequent releases.

Some of the most common CI services you’ll hear about are TravisCI, CircleCI,Github Actions, and Jenkins. They all behave similarly, though each tool’s setup and configuration process is slightly different. Focus on integrating CircleCI to perform a full build of an application, test it using our test suite, and automatically deploy our changes to Heroku.

Why Use a CI Tool?

It is important to run and test a complete build of your application before you release it to production. Think of how easy it was to deploy new changes to your production apps on Heroku. You would make a commit, which would immediately be reflected on production after running git push Heroku main. What if that commit introduced breaking changes? There is no safeguard or filter against pushing up bad code. With a CI tool like CircleCI, we can still take advantage of automatic deployments, but we also get an extra layer of assurance that our app is in good condition before any changes are pushed live.

CircleCI will also help with catching errors that you might not find locally. By mimicking a production environment more closely, we can recognize any differences that might be causing inconsistent behavior between environments. For example, if I am developing an app locally and I did an npm install(JavaScript apps) or bundle install(Ruby apps) months ago, and I did not explicitly lock down the version numbers for each of my dependencies in my package.json or Gemfile, I might not notice that my production environment received more updated packages during the npm install/bundle install phase. Some of these packages may have included breaking changes that would only be exposed in production and impossible to reproduce locally until another npm install/bundle install was run.

Getting Started with CircleCI

  1. Sign up for a CircleCI account here and log in with your GitHub account.
  2. Make a repository on your Github
  3. Authorize CircleCI to access information about your GitHub repos (it’s ok)
  4. Skip adding any project yet to your CircleCI for now. We will add a config file to our project before adding it to circleCI.
  5. Clone down your Github project into your machine.
  6. In your root directory, make .circleci folder.
  7. Inside that folder, make config.yml file.

Getting Setup w/ Circle CI for FE Apps with React

  • Here is the sample yml file for React/Node projects

version: 2.1

orbs:
  node: circleci/node@4.7

jobs:
  build_and_test:
    executor: node/default
    steps:
      - checkout
      - node/install-packages: #If you are using Yarn change this to Yarn
          pkg-manager: npm
      - run:
          name: Run tests
          command: npm test # If you are using Yarn, change this to Yarn - Hence if you are testing your project with Cypress you need to add a new run for that depends on how you've defined that script in your project. 
      - run : 
          name : Run build
          command: npm run build #If you are using Yarn change this to Yarn
      - persist_to_workspace:
          root: ~/project
          paths:
            - .
  1. Always run your test before pushing your code up.
  2. Commit your code to Github.
  3. On your CircleCI dashboard, select your app.
  4. choose the main branch and Set Up the project.

What is a Build?

Think of all the steps you have to take if you want to collaborate on a classmate’s project. You have to fork their repo, clone it down locally, make sure you have an up-to-date version of Node (or some other platform) on your machine, install any dependencies, start up a server and maybe run a file watcher.

Sometimes more complex projects require additional steps. This setup process is called a build. It’s all the things you need to do to get your app up and running. CI tools will run through all of these steps (and then some!) to ensure the application is in a stable, working state before it goes to production.

Phases of the Build Process

The .yml file can define settings for various build lifecycle phases.

  • install: install any dependencies required (configure the virtual machine (VM) to be able to run your code)
  • script: run the build script (run your code and tests)
  • deploy: deploy your code to your web servers (such as deploying to Heroku)

Configure Automatic Deployments to Heroku

If a build passes on CircleCI, then that means the code is good enough to go to production. We can make CircleCI automatically deploy to Heroku for us after a build passes.

  • If you are working in React, Svelte, Vue, etc. you will want to utilize Circle CI to check that your build is stable AND deploy to Heroku (or other hosting site)

Review Documentation

This documentation page goes over the configuration needed to add automatic Heroku deployments to your CircleCI build. Take some time to review before continuing on! Here is an overview of how to configure Node.js applications on CircleCi

  1. log in to your Heroku dashboard and create a new app.
  2. Copy the name of your Heroku application
  3. On the CircleCI dashboard under project settings, Environment Variables add HEROKU_APP_NAME=your-app-name
  4. On your Heroku dashboard, copy your Heroku API key and add that to CircleCi Environment Variables HEROKU_API_KEY=Your-API-KEY
  5. In your yml file, add a new Heroku orb. (This should be the same for Ruby, Node, and React) - If you want to learn more about Orbs, refer to CircleCI docs.
  6. Orbs should look like this inside the yml file
orbs:
  node: circleci/node@4.7
  heroku: circleci/heroku@1.2.6 # Use the Heroku orb in your config
  1. Now we have this orb, we want to add the deploy job to our project. Under jobs, add this code. (This is the same for Ruby and Node and React)
deploy: # this can be any name you choose
    executor: heroku/default
    steps:
      - attach_workspace:
          at: ~/project
      - heroku/deploy-via-git:
          force: true # force push when pushing to the heroku remote, see: https://devcenter.heroku.com/articles/git

  1. Under workflow test_my_app jobs add the deployment requirement.
- deploy:
          requires:
            - build_and_test # only deploy if the build_and_test job has completed
          filters:
            branches:
              only: main # only deploy when on main
  • Your yml file should be like this (Remember this is a React app setup However the deploy job should be similar in Ruby - Refer to Ruby .yml file sample)
version: 2.1

orbs:
  node: circleci/node@4.7
  heroku: circleci/heroku@1.2.6 # Use the Heroku orb in your config

jobs:
  build_and_test:
    executor: node/default
    steps:
      - checkout
      - node/install-packages:
          pkg-manager: npm
      - run:
          name: Run tests
          command: npm test
      - run : 
          name : Run build
          command: npm run build
      - persist_to_workspace:
          root: ~/project
          paths:
            - .
  deploy: # this can be any name you choose
    executor: heroku/default
    steps:
      - attach_workspace:
          at: ~/project
      - heroku/deploy-via-git:
          force: true # force push when pushing to the heroku remote, see: https://devcenter.heroku.com/articles/git

workflows:
  test_my_app:
    jobs:
      - build_and_test
      - deploy:
          requires:
            - build_and_test # only deploy if the build_and_test job has completed
          filters:
            branches:
              only: main # only deploy when on main

To test with Cypress, do some research! Hint: add a Circle CI orb!

Sample config file with Cypress orb

  version: 2.1

orbs:
  node: circleci/node@5.0.3
  cypress: cypress-io/cypress@2.2.0

jobs:
  build:
    executor: node/default
    steps:
      - checkout
      - node/install-packages: 
          pkg-manager: npm
      - run : 
          name : Run build
          command: npm run build 
      - persist_to_workspace:
          root: ~/project
          paths:
            - .
workflows:
  test_my_app:
    jobs:
      - cypress/run:
          start: npm start
      - build

Getting Setup w/ Circle CI for BE Apps with Ruby

Below are sample config files. You will need to change the value associated with POSTGRES_DB. The first sample uses Fly.io to for deployment, and the second sample uses Heroku. When using Fly.io to deploy applications, you must set up authentication so that the CI server has permission to push to Fly. To do this, add an environment variable in your CircleCI project settings with the key FLY_ACCESS_TOKEN. Be sure to use this exact key name as flyctl (the command line tool for Fly.io) will be looking for this variable. The value of this variable will be the token that is returned when you run flyctl auth token in your command line after logging in to Fly running flyctl auth login.

version: 2.1 # use CircleCI 2.1
orbs:
  node: circleci/node@5.0.0 # Needed for javascript runtime
jobs: # a collection of steps
  build-and-test: # runs not using Workflows must have a `build` job as entry point
    docker: # run the steps with Docker
      - image: circleci/ruby:2.7.4 # if you're using ruby version >= 3.0, use cimg/ruby:3.x.y
        environment: # environment variables for primary container
          BUNDLE_JOBS: 3
          BUNDLE_RETRY: 3
          BUNDLE_PATH: vendor/bundle
          PGHOST: 127.0.0.1
          PGUSER: postgres
          RAILS_ENV: test
      - image: circleci/postgres:11 # database image
        environment: # environment variables for database, it is kind of obvious but it is important that they match with your test config in database.yml
          POSTGRES_USER: postgres
          POSTGRES_DB: mod_4_randomizer_test
          POSTGRES_HOST_AUTH_METHOD: trust
    steps: # a collection of executable commands
      - checkout # special step to check out source code to working directory
      # Which version of bundler?
      - node/install:
          install-yarn: true
          node-version: '16.13'
      - run:
          name: Force Bundler Version
          command: |
            echo 'export BUNDLER_VERSION=$(cat Gemfile.lock | tail -1 | tr -d " ")' >> $BASH_ENV
            source $BASH_ENV
            gem install bundler
      - run:
          name: Bundle Install
          command: bundle install
      - run:
          name: Wait for DB
          command: dockerize -wait tcp://localhost:5432 -timeout 1m
      - run:
          name: Database setup
          command: bin/rails db:{create,migrate} --trace
      - run:
          name: Run rspec in parallel
          command: bundle exec rspec
      # Save test results for timing analysis
      - store_test_results: # Upload test results for display in Test Summary: https://circleci.com/docs/2.0/collect-test-data/
          path: test_results
      # See https://circleci.com/docs/2.0/deployment-integrations/ for example deploy configs
  deploy:
    docker:
      - image: alpine:3.12
    steps:
      - checkout
      - run:
          name: Deploy app to Fly
          # The following steps install Flyctl and deploy the app
          # Your CircleCI project must have a FLY_ACCESS_TOKEN environment variable
          # in order for CircleCI to authenticate and deploy this app.
          command: |
            apk add --no-cache curl
            curl -L https://fly.io/install.sh | sh
            export FLYCTL_INSTALL="/root/.fly"
            export PATH="$FLYCTL_INSTALL/bin:$PATH"
            echo "Successfully Installed Flyctl"
            sh -c "flyctl status"
            sh -c "flyctl deploy"
            sh -c "flyctl info"
            exit 0
workflows:
  build-test-deploy:
    jobs:
      - build-and-test
      - deploy:
          requires:
            - build-and-test
          filters:
            branches:
              only: main

Below is the sample config file for an app that uses Heroku.

version: 2.1 # use CircleCI 2.1
orbs:
   node: circleci/node@5.0.0 # Needed for javascript runtime
   heroku: circleci/heroku@1.0.1 # Use the Heroku orb in your config
jobs: # a collection of steps
  build-and-test: # runs not using Workflows must have a `build` job as entry point
    docker: # run the steps with Docker
      - image: circleci/ruby:2.7.2 # if you're using ruby version >= 3.0, use cimg/ruby:3.x.y 
        environment: # environment variables for primary container
          BUNDLE_JOBS: 3
          BUNDLE_RETRY: 3
          BUNDLE_PATH: vendor/bundle
          PGHOST: 127.0.0.1
          PGUSER: postgres
          RAILS_ENV: test
      - image: circleci/postgres:11 # database image
        environment: # environment variables for database, it is kind of obvious but it is important that they match with your test config in database.yml
          POSTGRES_USER: postgres
          POSTGRES_DB: <Insert the name of your testing database, for example, cool_new_app_test>
    steps: # a collection of executable commands
      - checkout # special step to check out source code to working directory
      # Which version of bundler?
      - node/install:
          install-yarn: true
          node-version: '16.13'
      - run:
          name: Force Bundler Version
          command: |
            echo 'export BUNDLER_VERSION=$(cat Gemfile.lock | tail -1 | tr -d " ")' >> $BASH_ENV
            source $BASH_ENV
            gem install bundler
      - run:
          name: Bundle Install
          command: bundle install
      - run:
          name: Wait for DB
          command: dockerize -wait tcp://localhost:5432 -timeout 1m
      - run:
          name: Database setup
          command: bin/rails db:{create,migrate} --trace
      - run:
          name: Run rspec in parallel
          command: bundle exec rspec
      # Save test results for timing analysis
      - store_test_results: # Upload test results for display in Test Summary: https://circleci.com/docs/2.0/collect-test-data/
          path: test_results
      # See https://circleci.com/docs/2.0/deployment-integrations/ for example deploy configs
workflows:
  build-test-deploy:
    jobs:
      - build-and-test
      - heroku/deploy-via-git: # Use the pre-configured job, deploy-via-git
          requires:
            - build-and-test
          post-steps:
            - run:
                name: migrate
                command: heroku run -a $HEROKU_APP_NAME rails db:migrate
          filters:
            branches:
              only:
                - main

Lesson Search Results

Showing top 10 results