Continuous Integration and Automated Testing for Engelsystem

Every software development group tests its products, yet delivered software always has defects. Test engineers strive to catch them before the product is released but they always creep in and they often reappear, even with the best manual testing processes. Using automated testing is the best way to increase the effectiveness, efficiency and coverage of your software testing.

Manual software testing is performed by a human sitting in front of a computer carefully going through application screens, trying various usage and input combinations, comparing the results to the expected behavior and recording their observations. Manual tests are repeated often during development cycles for source code changes and other situations like multiple operating environments and hardware configurations.

Continuous integration (CI) has emerged as one of the most efficient ways to develop code. But testing has not always been a major part of the CI conversation.

In some respects, that’s not surprising. Traditionally, CI has been all about speeding up the coding, building, and release process. Instead of having each programmer write code separately, integrate it manually, and then wait until the next daily or weekly build to see if the changes broke anything, CI lets developers code and compile on a virtually continuous basis. It also means developers and admins can work together seamlessly since the programming and build processes are always in sync.

Continuous Integration (CI) is a development practice that requires developers to integrate code into a shared repository several times a day. Each check-in is then verified by an automated build, allowing teams to detect problems early.

By integrating regularly, you can detect errors quickly, and locate them more easily.

Solve problems quickly

Because you’re integrating so frequently, there is significantly less back-tracking to discover where things went wrong, so you can spend more time building features.

Continuous Integration is cheap. Not continuously integrating is costly. If you don’t follow a continuous approach, you’ll have longer periods between integrations. This makes it exponentially more difficult to find and fix problems. Such integration problems can easily knock a project off-schedule, or cause it to fail altogether.

Continuous Integration brings multiple benefits to your organization:

  • Say goodbye to long and tense integrations
  • Increase visibility which enables greater communication
  • Catch issues fast and nip them in the bud
  • Spend less time debugging and more time adding features
  • Proceed with the confidence you’re building on a solid foundation
  • Stop waiting to find out if your code’s going to work
  • Reduce integration problems allowing you to deliver software more rapidly

“Continuous Integration doesn’t get rid of bugs, but it does make them dramatically easier to find and remove.”

– Martin Fowler, Chief Scientist, ThoughtWorks

Continuous Integration is backed by several important principles and practices.

Practices in Continuous Integration:

  • Maintain a single source repository
  • Automate the build
  • Make your build self-testing
  • Every commit should build on an integration machine
  • Keep the build fast
  • Test in a clone of the production environment
  • Make it easy for anyone to get the latest executable
  • Everyone can see what’s happening
  • Automate deployment

How to do Continuous Integration:

  • Developers check out code into their private workspaces.
  • When done, commit the changes to the repository.
  • The CI server monitors the repository and checks out changes when they occur.
  • The CI server builds the system and runs unit and integration tests.
  • The CI server releases deployable artifacts for testing.
  • The CI server assigns a build label to the version of the code it just built.
  • The CI server informs the team of the successful build.
  • If the build or tests fail, the CI server alerts the team.
  • The team fixes the issue at the earliest opportunity.
  • Continue to continually integrate and test throughout the project.

The CI implemented in Engelsystem are as follows:

  • Travis-CITravis CI is a hosted, distributed continuous integration service used to build and test software projects hosted on GitHub. It is integrated using the .travis.yml file in the root folder.
    language: php
    php:
    - '5.4'
    - '5.5'
    - '5.6'
    - '7.0'
    script: cd test && phpunit
  • Nitpick-CI:Automatic comments on PSR-2 violations in one click, so your team can focus on better code review. It requires one click for integratting with the repository.
  • Circle-CI: CircleCI was founded in 2011 with the mission of giving every developer state-of-the-art automated testing and continuous integration tools. It is integrated using a circle.yml file in the root folder of the repository.
    machine:
    php:
    version: 5.4.5
    deployment:
    master:
    branch: master
    owner: fossasia
    commands:
    - ./deploy_master.sh
    dependencies:
    pre:
    - curl -s http://getcomposer.org/installer | php
    - php composer.phar install -n
    - sed -i 's/^;//' ~/.phpenv/versions/$(phpenv global)/etc/conf.d/xdebug.ini
    
    test:
    post:
    - php test/
    - bash <(curl -s https://codecov.io/bash)
  • Codacy: Check code style, security, duplication, complexity and coverage on every change while tracking code quality throughout your sprints.

Development: https://github.com/fossasia/engelsystem Issues/Bugs:https://github.com/fossasia/engelsystem/issues