travis-ci

Auto Deploying accounts.susi.ai on gh-pages

While migrating web apps from susi server repository to accounts.susi.ai repository. Our team autodeployed accounts.susi.ai on gh-pages branch i.e., each time changes are made to code, it gets auto deployed on gh-pages branch. And changes are automatically live and visible on site. To do this we have to setup travis in our repository. Travis can be easily set in github repository just by adding .travis.yml file into root directory your repo. Depending on type of repository you are dealing with, the configuration of travis changes. For accounts.susi.ai, It is a written on top of ReactJs framework. So we used the following code. sudo: required dist: trusty language: node_js node_js: - 6 script: - npm test deploy: provider: script script: "./deploy.sh" env: global: - ENCRYPTION_LABEL: "<.... encryption label from previous step ....>" - COMMIT_AUTHOR_EMAIL: "you@example.com" cache: directories: - node_modules branches: only: - master   The above travis configuration file, After every commit checks whether the build is passing or not by running npm test command. After checking the commit, next script deploy.sh is run on the commit. The last line tells us to only execute this script for commits in master branch. deploy.sh can be placed anywhere, we jus need to modify its path in .tavis.yml file. Here is code of deploy.sh : #!/bin/bash set -e SOURCE_BRANCH="master" TARGET_BRANCH="gh-pages" if [ "$TRAVIS_PULL_REQUEST" != "false" -o "$TRAVIS_BRANCH" != "$SOURCE_BRANCH" ]; then echo "Skipping deploy; just doing a build." doCompile exit 0 fi # Save some useful information REPO=`git config remote.origin.url` SSH_REPO=${REPO} SHA=`git rev-parse --verify HEAD` git clone $REPO out cd out git checkout $TARGET_BRANCH || git checkout --orphan $TARGET_BRANCH cd .. git config user.name "Travis CI" git config user.email "travis-ci@github.com" if git diff --quiet; then echo "No changes to the output on this push; exiting." exit 0 fi # Commit the "changes", i.e. the new version. # The delta will show diffs between new and old versions. git add -A . git commit -m "Deploy to GitHub Pages: ${SHA}" # Get the deploy key by using Travis's stored variables to decrypt deploy_key.enc ENCRYPTED_KEY_VAR="encrypted_${ENCRYPTION_LABEL}_key" ENCRYPTED_IV_VAR="encrypted_${ENCRYPTION_LABEL}_iv" ENCRYPTED_KEY=${!ENCRYPTED_KEY_VAR} ENCRYPTED_IV=${!ENCRYPTED_IV_VAR} openssl aes-256-cbc -K $ENCRYPTED_KEY -iv $ENCRYPTED_IV -in ../deploy_key.enc -out ../deploy_key -d chmod 600 ../deploy_key eval `ssh-agent -s` ssh-add deploy_key # Now that we're all set up, we can push. git push $SSH_REPO $TARGET_BRANCH   deploy.sh is automatically deploying master on gh pages branch. To perform this task, it needs admin authorisation of github repo. We do this authentication by encrypted keys. To create encrypted keys we need to generate a new SSH key, SSH keys can be generated by running following command in terminal. ssh-keygen -t rsa -b 4096 -C "your_email@example.com"   Then add this key to your project’s git repository at the github repo of project. https://github.com/<your name>/<your repo>/settings/keys Now, We can generate encrypted keys by running following command. travis encrypt-file deploy_key   This command generates deploy_key.enc file, which shall be placed in the repo. Its location needs to be updated in the placeholders of .travis.yml and deploy.sh. Hence, we achieved the task of auto deployment to…

Continue ReadingAuto Deploying accounts.susi.ai on gh-pages
Read more about the article Arch Linux for Travis CI

Arch Linux for Travis CI

Travis CI does not provide Arch Linux as a testing environment but if you need to test some script or some app specifically on Arch Linux, you would have to do that manually for every change made into code and that would waste time. I had a similar problem when I was trying to test the Event Linux build scripts based on Arch Linux which required the Arch Linux environment to be able to run build scripts which take around 25 to 30 minutes to be tested for every change How to use Arch travis script for Travis CI? To solve the problem we need to use Arch as chroot in Travis CI. To do so we are going to use this script (link) which will deploy Arch Linux chroot on travis for us to start testing For using it we need to configure the travis.yml like this :- sudo: required arch: repos:  - archlinuxfr=http://repo.archlinux.fr/$arch packages:  # pacman packages  - yaourt  - archiso script:  - ./build-repo  - sudo ./build.sh -v script: - "travis_wait 30 sleep 1800 &" - curl -s https://raw.githubusercontent.com/fossasia/arch-travis/master/arch-travis.sh | bash arch.packages defines a list of packages (from official repository or AUR) to be installed before running the build. For eg: packages:  # pacman packages  - python  - perl  # aur packages  - go-git  # packages from custom repo  - yaourt Here in arch.repos we define custom repository required for the package which is required during testing. For eg. in above code repo : http://repo.archlinux.fr/$arch is used to install yaourt we could have used AUR too but that will increase our build testing time as the AUR will first download the pkgbuild and then build the package which can result in increase of time arch.script defines a list of scripts to run as part of the build. Like in above example we have used ./build-repo and sudo ./build -v Anything defined in the arch.script list will run from the base of the repository as a normal user called travis CI . sudo is available as well as any packages installed in the setup. The path of the build dir (or repository base), is stored in the TRAVIS_BUILD_DIR environment variable inside the chroot. At the end of example we have used script: Which defines the scripts to be run by travis, this is where arch-travis was initialized and then we pipe it Usage of travis-wait function script: - travis_wait 30 sleep 1800 & - curl -s https://raw.githubusercontent.com/fossasia/arch-travis/master/arch-travis.sh | bash We cannot use travis wait function inside the chroot neither we can export it so we apply the travis wait function at the point where we have initialised the Arch-travis Example usage for pkgbuild scripts sudo: required # this to get sudo permissions arch: # anything to be tested under Arch is defined below this script:  - makepkg app.pkgbuild # this will test the pkgbuild script script: #for setting up arch chroot using the arch travis script - curl -s https://raw.githubusercontent.com/fossasia/arch-travis/master/arch-travis.sh | bash Limitations Increases build time by about 1-3…

Continue ReadingArch Linux for Travis CI

Integrating Travis CI and Codacy in PSLab Repositories

Continuous Integration Testing and Automated Code Review tools are really useful for developing better software, improving code and overall quality of the project. Continuous integration can help catch bugs by running tests automatically and to merge your code with confidence. While working on my GsoC-16 project, my mentors guided and helped me to integrate Travis CI and Codacy in PSLab github repositories. This blog post is all about integrating these tools in my github repos, problems faced, errors occurred and the test results. Travis CI is a hosted continuous integration and deployment system. It is used to build and test software projects hosted on github. There are two versions of it, travis-ci.com for private repositories, and travis-ci.org for public repositories. Read : Getting started with Travis CI Travis is configured with the “.travis.yml” file in your repository to tell Travis CI what to build. Following is the code from '.travis.yml' file in our PSLab repository. This repo contains python communication library for PSLab. language: python python:   - "2.6"   - "2.7"   - "3.2"   - "3.3"   - "3.4" # - "3.5" # command to install dependencies # install: "pip install -r requirements.txt" # command to run tests script: nosetests With this code everything worked out of the box (except few initial builds which errored because of missing 'requirements.txt' file) and build passed successfuly :) :) Later Mario Behling added integration to FOSSASIA Slack Channel. Slack notifications Travis CI supports notifying  Slack channels about build results. On Slack, set up a new Travis CI integration. Select a channel, and you’ll find the details to paste into your '.travis.yml'. Just copy and paste the settings, which already include the proper token and you’re done. The simplest configuration requires your account name and the token. notifications: slack: '<account>:<token>' notifications:   slack: fossasia:***tokenishidden**** Import errors in Travis builds of PSLab-apps Repository PSLab-apps repository contains PyQt bases apps for various experiments. The '.travis.yml' file mentioned above gave several module import errors. $ python --version Python 3.2.5 $ pip --version pip 6.0.7 from /home/travis/virtualenv/python3.2.5/lib/python3.2/site-packages (python 3.2) Could not locate requirements.txt. Override the install: key in your .travis.yml to install dependencies. 0.33s$ nosetests E ====================================================================== ERROR: Failure: ImportError (No module named sip) The repo is installable and PSLab was working fine on popular linux distributions without any errors. I was not able to find the reason for build errors. Even after adding proper 'requirements.txt' file,  travis builds errored. On exploring the documentation I could figure out the problem. Travis CI Environment uses separate virtualenv instances for each Python version. System Python is not used and should not be relied on. If you need to install Python packages, do it via pip and not apt. If you decide to use apt anyway, note that Python system packages only include Python 2.7 libraries (default python version). This means that the packages installed from the repositories are not available in other virtualenvs even if you use the –system-site-packages option. Therefore I was getting Import module errors. This…

Continue ReadingIntegrating Travis CI and Codacy in PSLab Repositories
Read more about the article Unit Testing and Travis

Unit Testing and Travis

Tests are an important part of any software development process. We need to write test codes for any feature that we develop to check if that feature is working properly. In this post, I am gonna talk about writing Unit tests and running those test codes. If you are a developer, I assume you have heard about unit tests. Most of you probably even wrote one in your life. Unit testing is becoming more and more popular in software development. Let's first talk about what Unit testing is: What is unit testing? Unit testing is the process through which units of source code are tested to verify if they work properly. Performing unit tests is a way to ensure that all functionalities of an application are working as they should. Unit tests inform the developer when a change in one unit interferes with the functionality of another. Modern unit testing frameworks are typically implemented using the same code used by the system under test. This enables a developer who is writing application code in a particular language to write their unit tests in that language as well. What is a unit testing framework? Unit testing frameworks are developed for the purpose of simplifying the process of unit-testing. Those frameworks enable the creation of Test Fixtures, which are classes that have specific attributes enabling them to be picked up by a Test Runner. Although it is possible to perform unit tests without such a framework, the process can be difficult, complicated and very manual. There are a lot of unit testing frameworks available. Each of the frameworks has its own merits and selecting one depends on what features are needed and the level of expertise of the development team. For my project, Engelsystem I choose PHPUnit as the testing framework. PHPUnit With PHPUnit, the most basic thing you’ll write is a test case. A test case is just a term for a class with several different tests all related to the same functionality. There are a few rules you’ll need to worry about when writing your cases so that they’ll work with PHPUnit: The test class would extend the PHPUnit_Framework_TestCase class. The test parameters will never receive any parameters. Below is an example of a test code from my project, Engelsystem <?php class ShiftTypes_Model_test extends PHPUnit_Framework_TestCase { private $shift_id = null; public function create_ShiftType(){ $this->shift_id = ShiftType_create('test', '1', 'test_description'); } public function test_ShiftType_create() { $count = count(ShiftTypes()); $this->assertNotFalse(create_ShiftType($shift_id)); // There should be one more ShiftTypes now $this->assertEquals(count(ShiftTypes()), $count + 1); } public function test_ShiftType(){ $this->create_ShiftType(); $shift_type = ShiftType($this->shift_id); $this->assertNotFalse($shift_type); $this->assertTrue(count(ShiftTypes()) > 0); $this->assertNotNull($shift_type); $this->assertEquals($shift_type['name'], 'test'); $this->assertEquals(count(ShiftTypes()), 0); $this->assertNull(ShiftTypes(-1)); } public function teardown() { if ($this->shift_id != null) ShiftType_delete($this->shift_id); } } ?> We can use different Assertions to test the functionality. We are running these tests on Travis-CI What is Travis-CI? Travis CI is a hosted, distributed continuous integration service used to build and test software projects hosted on GitHub. Open source projects may be tested at no charge via travis-ci.org. Private projects may be tested at…

Continue ReadingUnit Testing and Travis

Testing Docker Deployment using Travis

Hello. This post is about how to setup automated tests to check if your application’s docker deployment is working or not. I used it extensively while working on the Docker deployment of the Open Event Server. In this tutorial, we will use Travis CI as the testing service. To start testing your github project for Docker deployment, first add the repo to Travis. Then create a.travis.yml in the project’s root directory. In that file, add docker to services. services: - docker The above will enable docker in the testing environment. It will also include docker-compose by default. Next step is to build your app and run it. Since this is a pre-testing step, we will add it in the install directive. install: - docker build -t myapp . - docker run -d -p 127.0.0.1:80:4000 --name myapp myapp The 4000 in the above text is assuming your app runs on port 4000 inside the container. Also it is assumed that Dockerfile is in the root of the repo. So now that the docker app is running, it’s time to test it. script: - docker ps | grep -i myapp The above will test if our app is in one of the running docker processes. It is a basic test to see if the app is running or not. We can go ahead and test the app’s functionality with some sample requests. Create a file test.py with the following contents. import requests r = requests.get('http://127.0.0.1/') assert 'HomePage' in r.content, 'No homepage loaded' Then run it as a test. script: - docker ps | grep -i myapp - python test.py You can make use of the unittest module in Python to bundle and create more organized tests. The limit is the sky here. In the end, the .travis.yml will look something like the following language: python python: - "2.7" install: - docker build -t myapp . - docker run -d -p 127.0.0.1:80:4000 --name myapp myapp script: - docker ps | grep -i myapp - python test.py So this is it. A basic tutorial on testing Docker deployments using the awesome Travis CI service. Feel free to share it and comment your views.   {{ Repost from my personal blog http://aviaryan.in/blog/gsoc/docker-test.html }}

Continue ReadingTesting Docker Deployment using Travis

Unit Testing

There are many stories about unit testing. Developers sometimes say that they don’t write tests because they write a good quality code. Does it make sense, if no one is infallible?. At studies only a  few teachers talk about unit testing, but they only show basic examples of unit testing. They require to write a few tests to finish final project, but nobody really  teaches us the importance of unit testing. I have also always wondered what benefits can it bring. As time is a really important factor in our work it often happens that we simply resign of this part of process development to get “more time” rather than spend time on writing stupid tests. But now I know that it is a vicious circle. Customers requierments does not help us. They put a high pressure to see visible results not a few statistics about coverage status. None of them cares about some strange numbers. So, as I mentioned above, we usually focuses on building new features and get riid of tests. It may seem to save time, but it doesn’t. In reality tests save us a lot of time because we can identify and fix bugs very quickly. If a bug ocurrs because someone’s change we don’t have to spend long hours trying to figure out wgat is going out. That’s why we need tests.   It is especially visible in huge open source projects. FOSSASIA organization has about 200 contributors. In OpenEvent project we have about 20 active developers, who generate many lines of code every single day. Many of them change over and over again as well as interfere  with each other. Let me provide you with a simple example. In our team we have about 7 pull requests per day. As I mentioned above we want to make our code high quality and free of bugs, but without testing identifying if pull request causes a bug is very difficult task. But fortunately this boring job makes Travis CI for us. It is a great tool which uses our tests and runs them on every PR  to check if bugs occur. It helps us to quickly notice bugs and maintain our project very well. What is unit testing? Unit testing is a software development method in which the smallest testable parts of an application are tested Why do we need writing unit tests? Let me point all arguments why unit testing is really important while developing a project. To prove that our code works properly If developer adds another condition, test checks if method returns correct results. You simply don’t need to wonder if something is wrong with you code. To reduce amount of bugs It let you to know what inputs params’ function should get and what results should be returned. You simply don’t  write unused code To save development time Developers don’t waste time on checking every code’s change if his code works correctly Unit tests help to understand software design To provide quick feedback…

Continue ReadingUnit Testing

Push your apk to your GitHub repository from Travis

In this post I’ll guide you on how to directly upload the compiled .apk from travis to your GitHub repository. Why do we need this? Well, assume that you need to provide an app to your testers after each commit on the repository, so instead of manually copying and emailing them the app, we can setup travis to upload the file to our repository where the testers can fetch it from. So, lets get to it! Step 1 : Link Travis to your GitHub Account. Open up https://travis-ci.org. Click on the green button in the top right corner that says “Sign in with GitHub” < Step 2 : Add your existing repository to Travis Click the “+” button next to your Travis Dashboard located on the left. < Choose the project that you want to setup Travis from the next page Toggle the switch for the project that you want to integrate Click the cog here and add an Environment Variable named GITHUB_API_KEY. Proceed by adding your Personal Authentication Token there. Read up here on how to get the Token.  < Great, we are pretty much done here. Let us move to the project repository that we just integrated and create a new file in the root of repository by clicking on the “Create new file” on the repo’s page. Name it .travis.yml and add the following commands over there language: android jdk: - oraclejdk8 android: components: - tools - build-tools-24.0.0 - android-24 - extra-android-support - extra-google-google_play_services - extra-android-m2repository - extra-google-m2repository - addon-google_apis-google-24 before_install: - chmod +x gradlew - export JAVA8_HOME=/usr/lib/jvm/java-8-oracle - export JAVA_HOME=$JAVA8_HOME after_success: - chmod +x ./upload-gh-pages.sh - ./upload-apk.sh script: - ./gradlew build Next, create a bash file in the root of your repository using the same method and name it upload-apk.sh #create a new directory that will contain out generated apk mkdir $HOME/buildApk/ #copy generated apk from build folder to the folder just created cp -R app/build/outputs/apk/app-debug.apk $HOME/android/ #go to home and setup git cd $HOME git config --global user.email "useremail@domain.com" git config --global user.name "Your Name" #clone the repository in the buildApk folder git clone --quiet --branch=master https://user-name:$GITHUB_API_KEY@github.com/user-name/repo-name master > /dev/null #go into directory and copy data we're interested cd master cp -Rf $HOME/android/* . #add, commit and push files git add -f . git remote rm origin git remote add origin https://user-name:$GITHUB_API_KEY@github.com/user-name/repo-name.git git add -f . git commit -m "Travis build $TRAVIS_BUILD_NUMBER pushed" git push -fq origin master > /dev/null echo -e "Donen" Once you have done this, commit and push these files, a Travis build will be initiated in few seconds. You can see it ongoing in your Dashboard at https://travis-ci.org/. After the build has completed, you will can see an app-debug.apk in your Repository. IMPORTANT NOTE : You might be wondering as to why did I write [skip ci] in the commit message. Well the reason for that is, Travis starts a new build as soon as it detects a commit made on the master branch of your repository. So once the apk is uploaded, that will trigger…

Continue ReadingPush your apk to your GitHub repository from Travis