Deployment terms in Open Event Frontend

In Open Event Frontend, once a pull request is opened, we see some tests running on for the specific pull request like ‘Codacy’, ‘Codecov’, ‘Travis’, etc. New contributors eventually get confused what the tests are about. So this blog would be a walkthrough to these terms that we use and what they mean about the PR.

Travis: Everytime you make a pull request, you will see this test running and in some time giving the output whether the test passed or failed. Travis is the continuous integration test that we are using to test that the changes that the pull request you proposed does not break any other things. Sometimes you will see the following message which indicates that your changes is breaking something else which is not intended.

Thus, by looking at the Travis logs, you can see where the changes proposed in the pull request are breaking things. Thus, you can go ahead and correct the code and push again to run the Travis build until it passes.

Codacy: Codacy is being used to check the code style, duplication, complexity and coverage, etc. When you create a pull request or update the pull request, this test runs which checks whether the code followed certain style guide or if there is duplication in code, etc. For instance let’s say if your code has a html page in which a tag has an attribute which is left undefined. Then codacy will be throwing error failing the tests. Thus you need to see the logs and go correct the bug in code. The following message shows that the codacy test has passed.

Codecov:

Codecov is a code coverage test which indicates how much of the code change that is proposed in the pull request is actually executed. Consider out of the 100 lines of code that you wrote, only 80 lines is being actually executed and rest is not, then the code coverage decreases. The following indicates the codecov report.

Thus, it can be seen that which files are affected by what percent.

Surge:

The surge link is nothing but the deployment link of the changes in your pull request.

Thus, checking the link manually, we can test the behavior of the app in terms of UI/UX or the other features that the pull request adds.

References:

 

 

Continue ReadingDeployment terms in Open Event Frontend

Implementation of Responsive SUSI Web Chat Search Bar

When we were building the first phase of the SUSI Web Chat Application we didn’t consider about  the responsiveness as a main goal. The main thing we needed was a working application. This changed at a later stage. In this post I’m going to emphasize how we implemented the responsive design and problems we met while we were developing the design.

When we were moving to Material-UI from static HTML CSS we were able to make most of the parts responsive. As an example App-bar of the application. We added App-bar like as follows: We made a separate component for App-bar and it includes the “searchfield” element. Material-UI app bar handles the responsiveness for some extent. We have to handle responsiveness of other sub-parts of the app bar manually.

In “TopBar.react.js” I returned marterial-ui <Toolbar> element like this.

<Toolbar>
                <ToolbarGroup >
                </ToolbarGroup>
                <ToolbarGroup lastChild={true}> //inside of this we have to include other components of the top bar inside this element

                </ToolbarGroup>
             </Toolbar>

We have to add the search button inside the element.
In this we added search component.

This field has the ability to expand and collapse like this.

It looks good. But it appears on mobile screen in a different way. This is how it appears on mobile devices.

So we wanted to hide the SUSI logo on small sized screens. For that we wrote medial queries like this.

@media only screen and (max-width: 860px){
 .app-bar-search{
   background-image: none;
 }
 .search{
   width: 100px !important;
 }
}

Even in smaller screens it appears like this.

To avoid that we minimized the width of the search bar in different screen sizes using media queries .

@media only screen and (max-width: 480px){
 .search{
   width: 100px !important;
 }
}
@media only screen and (max-width: 360px){
 .search{
   width: 65px !important;
 }
}

But in even smaller screens it still appears in the same way. We can’t show the search bar on small screens because the screen size is not enough to show the search bar.
So we wrote another media query to hide all the elements of search component in small screens except close button. Because when we collapse the screen on search mode it hides all the search components and messagecomposer. To take it back to the chat mode we have to enable the close button on smaller screens.

@media only screen and (max-width: 300px){
 .displayNone{
   display: none !important;
 }
 .displayCloseNone{
     position: relative;
     top:6px  !important;
 }
}

We have to define these two classes in “SearchField.react.js” file.

<IconButton className='displayNone'
                   <SearchIcon />
               </IconButton>
               <TextField  name='search'
                   className='search displayNone'
                   placeholder="Search..."/>
               <IconButton className='displayNone'>
                   <UpIcon />
               </IconButton>
               <IconButton className='displayNone'>
                   <DownIcon />
               </IconButton>
               <IconButton className='displayCloseNone'>
                   <ExitIcon />
               </IconButton>

Since we have “Codacy” integrated into our Github Repository we have to change Codacy rules because we used “!important” in media queries to override inline style which comes from Material-UI.
To change codacy rules we can simply login to the codacy and select the “code pattern ” button from the column left side.

It shows the list of rules that Codacy checks. And you can see the “!important” rule under CSSlint category like this.

Just uncheck it. Then codacy will not check your source code for “!important” attributes.

Resources
Configuring Codacy: Use Your Own Conventions: https://blog.codacy.com/configuring-codacy-use-your-own-conventions-9272bee5dcdb
Media queries: https://www.w3schools.com/css/css_rwd_mediaqueries.asp

Continue ReadingImplementation of Responsive SUSI Web Chat Search Bar

Integrating Travis CI and Codacy in PSLab Repositories

Continuous Integration Testing and Automated Code Review tools are really useful for developing better software, improving code and overall quality of the project. Continuous integration can help catch bugs by running tests automatically and to merge your code with confidence.

While working on my GsoC-16 project, my mentors guided and helped me to integrate Travis CI and Codacy in PSLab github repositories. This blog post is all about integrating these tools in my github repos, problems faced, errors occurred and the test results.

travisTravis CI is a hosted continuous integration and deployment system. It is used to build and test software projects hosted on github. There are two versions of it, travis-ci.com for private repositories, and travis-ci.org for public repositories.

Read : Getting started with Travis CI

Travis is configured with the “.travis.yml” file in your repository to tell Travis CI what to build. Following is the code from ‘.travis.yml‘ file in our PSLab repository. This repo contains python communication library for PSLab.

language: python
python:
  - "2.6"
  - "2.7"
  - "3.2"
  - "3.3"
  - "3.4"
# - "3.5"
# command to install dependencies
# install: "pip install -r requirements.txt"
# command to run tests
script: nosetests

With this code everything worked out of the box (except few initial builds which errored because of missing ‘requirements.txt‘ file) and build passed successfuly 🙂 🙂

Later Mario Behling added integration to FOSSASIA Slack Channel.

Slack notifications

Travis CI supports notifying  Slack channels about build results. On Slack, set up a new Travis CI integration. Select a channel, and you’ll find the details to paste into your ‘.travis.yml’. Just copy and paste the settings, which already include the proper token and you’re done.

The simplest configuration requires your account name and the token.

notifications:
  slack: '<account>:<token>'     
notifications:
  slack: fossasia:***tokenishidden****

Import errors in Travis builds of PSLab-apps Repository

PSLab-apps repository contains PyQt bases apps for various experiments. The ‘.travis.yml‘ file mentioned above gave several module import errors.

$ python --version
Python 3.2.5
$ pip --version
pip 6.0.7 from /home/travis/virtualenv/python3.2.5/lib/python3.2/site-packages (python 3.2)
Could not locate requirements.txt. Override the install: key in your .travis.yml to install dependencies.
0.33s$ nosetests
E
======================================================================
ERROR: Failure: ImportError (No module named sip)

The repo is installable and PSLab was working fine on popular linux distributions without any errors. I was not able to find the reason for build errors. Even after adding proper ‘requirements.txt‘ file,  travis builds errored.

On exploring the documentation I could figure out the problem.

Travis CI Environment uses separate virtualenv instances for each Python version. System Python is not used and should not be relied on. If you need to install Python packages, do it via pip and not apt. If you decide to use apt anyway, note that Python system packages only include Python 2.7 libraries (default python version). This means that the packages installed from the repositories are not available in other virtualenvs even if you use the –system-site-packages option. Therefore I was getting Import module errors.

This problem was solved by making following changes in the ‘.travis.yml‘ file

language: python

python:
  #- "2.6"
  - "2.7"
  #- "2.7_with_system_site_packages"
  - "3.2"
  #- "3.2_with_system_site_packages"
  - "3.3"
  - "3.4"
before_install:
    - sudo mkdir -p /downloads
    - sudo chmod a+rw /downloads
    - curl -L http://sourceforge.net/projects/pyqt/files/sip/sip-4.16.5/sip-4.16.5.tar.gz -o /downloads/sip.tar.gz 
    - curl -L http://sourceforge.net/projects/pyqt/files/PyQt4/PyQt-4.11.3/PyQt-x11-gpl-4.11.3.tar.gz -o /downloads/pyqt4.tar.gz
    # Builds
    - sudo mkdir -p /builds
    - sudo chmod a+rw /builds

install:
    - export DISPLAY=:99.0
    - sh -e /etc/init.d/xvfb start
    - sudo apt-get install -y libqt4-dev
    - sudo apt-get install -y mesa-common-dev libgl1-mesa-dev libglu1-mesa-dev
#    - sudo apt-get install -y python3-sip python3-sip-dev python3-pyqt4 cmake
    # Qt4
    - pushd /builds
    # SIP
    - tar xzf /downloads/sip.tar.gz --keep-newer-files
    - pushd sip-4.16.5
    - python configure.py
    - make
    - sudo make install
    - popd
    # PyQt4
    - tar xzf /downloads/pyqt4.tar.gz --keep-newer-files
    - pushd PyQt-x11-gpl-4.11.3
    - python configure.py -c --confirm-license --no-designer-plugin -e QtCore -e QtGui -e QtTest
    - make
    - sudo make install
    - popd
 # - "3.5"
# command to install dependencies
#install: "pip install -r requirements.txt"
# command to run tests
script: nosetests

notifications:
  slack: fossasia:*****tokenishidden*******


codacy

Codacy is an automated code analysis and review tool that helps developers ship better software, faster. With Codacy integration one can get static analysis, code complexity, code duplication and code coverage changes in every commit and pull request.

Read : Integrating Codacy in github is here.

Codacy integration has really helped me to understand and enforce code quality standard. Codacy gives you impact of every pull request in terms of quality and errors directly into GitHub.

codacy check

Codacy also grades your project in different categories like Code Complexity, Compatibility, security, code style, error prone etc. to help you better understand the overall project quality and what are the areas you should improve.

Here is a screen-shot of Codacy review for PSLab-apps repository.

codacyreport

I am extremely happy to share that my learning adventure has got  Project Certification at ‘A’ grade. Project quality analysis shows that more than 90% of the work has A grade 🙂 🙂

Travis CI and Codacy Badges for my GSoC Repositories:

PSLab : Python Library for Communication with PSLab

Travis CI Badge         Codacy Badge

PSLab-apps : Qt based GUI applications for PSLab

Travis CI Badge         Codacy Badge

Pocket Science Lab : ExpEYES Programs, Sensor Plugins

Travis CI Badge         Codacy Badge

That’s all for now. Have a happy coding, testing and learning 🙂 🙂

Continue ReadingIntegrating Travis CI and Codacy in PSLab Repositories

Can solving lint bugs be interesting?

Today I am going to present you how we’ve changed monotonous solving bugs into motivating process.

PEP

Most developers need to improve their code quality. To do  that they can use style guide for e.g for Python code (PEP). PEP contains an index of all Python Enhancement Proposals.

Below you can find which logs PEP returned in a command line.

Do you think that this logs’ presentation is  good enough to interest a developer? Will he solve these  thousands of bugs?

Undoubtedly, there are much information about errors and warnings so PEP returns long logs. But developer can not even know how to start solving bugs. And even if she/he finally starts, after each commit he/she needs to run that script again to check if quantity of bugs are increased or decreased. It seems to be endless, exhausting and very monotonous.  Nobody is encouraged to do it.

logi.png

Quality monitoring

Open Event team wants to increase our productivity and code quality. Therefore we use a tool which allow us to check code style, security, duplication complexity and test coverage on every commit. That tool is Codacy and it fulfils our requirements in 100%. It is very helpful because it adds comments to pull requests and enables developer quickly find where a bug is located. It’s very comfortable, because you don’t need to check issues in above awful logs results. Take a look how it looks in Codacy.

-DO NOT MERGE  Ticketing Flow by niranjan94 · Pull Request  1927 · fossasia open event orga server.png

Isn’t it clear? Of course that it’s. Codacy shows in which line issue ocurres and which type of issue it’s.

Awesome statistics dashboard

I’d like to give an answer how you can engage your team to solve issues and make this process more interesting. On the main page codacy tool welcomes you with great statistics about your project.

open event orga server   Codacy   Dashboard

You can see number of issues, category like code complexity, code style, compatibility, documentation, error prone, performance, security and unused code. That params show in which stage of code quality your project is. I think that every developer’s aim is to have the highest code quality and increasing these statistics. But if project has many issues, developer sees only a few changes in project charts.

Define Goals

Recently I’ve discovered how you can motivate yourself more. You can define a goal which you’d like achive. It can be goal of category or goal of file. For example Open Event team has defined goal for a specific file to achieve. If you define small separate goals, you can quicker see the results of your work.

open event orga server_2   Codacy   Goals

On the left sidebar you can find a item which is named “Goals”. In this area you can easily add your projects goals. Everything is user friendly so you shouldn’t have a problem  to create own goals.

Continue ReadingCan solving lint bugs be interesting?

Configuring Codacy: Use Your Own Conventions

Screenshot from 2016-07-23 01:08:20

All the developers agree on at least one thing – writing clean code is necessary. Because as someone anonymous said, always write a code as if the developer who comes after you is a homicidal maniac who knows your address. So, yeah, writing clean code is very important. Codacy helps in code reviewing and code quality monitoring. You can set codacy in any of your github project. It automatically identifies new static analysis issues, code coverage, code duplication and code complexity evolution in every commit and pull request.

Continue ReadingConfiguring Codacy: Use Your Own Conventions

Read and Understand Codacy Reports

To begin understanding reports, let’s start with what Codacy is.

So. What is Codacy?

Codacy is an automated code review tool that helps developers to save time in code reviews and to tackle technical debt. It centralises customizable code patterns and enforces them within engineering teams. Codacy tracks new issues by severity level for every commit and pull request.

It can be integrated with the GitHub repository to review every commit and pull request in terms of quality and errors. It checks code style, security, duplication, complexity and coverage on every change while tracking code quality throughout your sprints.

You can integrate Codacy in your private/public repository by going here. Sign up with your Github account and follow the steps mentioned. More information regarding GitHub integration can be found here.

Features of Codacy:

  • SAVE TIME IN CODE REVIEWS
  • INTEGRATED IN YOUR WORKFLOW
  • TRACK YOUR PROJECT QUALITY EVOLUTION

Now we get to understanding Codacy reports.

Below shows and image of how a Codacy dashboard looks like, to evaluate a project. To evaluate a project we should know what are the Software Metrics”.

download

A software metric is a standard of measure of a degree to which a software system or process possesses some property. Even if a metric is not a measurement (metrics are functions, while measurements are the numbers obtained by the application of metrics), often the two terms are used as synonymous. Since quantitative measurements are essential in all sciences, there is a continuous effort by computer science practitioners and theoreticians to bring similar approaches to software development. The goal is obtaining objective, reproducible and quantifiable measurements, which may have numerous valuable applications in the schedule and budget planning, cost estimation, quality assurance testing, software debugging, software performance optimization, and optimal personnel task assignments. You can learn about software metrics by visiting here.

A Codacy Dashboard provides answer to the following 3 main things:

  • What is the state of your projects code quality?
  • How is it evolving throughout time?
  • What are the hotspots in your code?

Component of the Dashboard:

  • Introduction: The Dashboard is the central screen of any project on Codacy.
  • Project Certification: After running a complete on the project or the GitHub repository, Codacy provides an overall grade to the project from A-F. The grade depends on the following parameters.
    • Error Prone
    • Code Complexity
    • Code Style
    • Unused Code
    • Security
    • Compatibility
    • Documentation
    • Performancedashboard-certification
  • Issues Breakdown: Issues breakdown represents the different issues from different areas in a pictorial representation. It provides a quick overview of the total number of issues in the repository and the breakdown per category.
    Issues Brakedown
    Users can click on the specific category for more details.
  • Code Coverage: If you setup the code coverage on your repository, you will be able to see the overall covered percentage on the dashboard. It will also show the files with the worst code coverage allowing you to directly jump to the file to see the details.
    Coverage
  • Goals: Users can define individual goals to remove errors and get better grades for their projects.
  • Historic data: Codacy dashboard also provides an analysis of the Historic data, so as to keep a track of the progress on improving the code, milestones covered in reaching the goal.
    dashboard-historic-issues

Codacy provides a nice dashboard showing the metrics. Codacy saves hours in code review and code quality monitoring, from small teams to big companies. And as the Codacy team itself says “LOVED BY DEVELOPERS”, being a developer I wouldn’t deny this statement. It helped a lot in improving the code quality of my project Engelsystem.

Development: https://github.com/fossasia/engelsystem Issues/Bugs:https://github.com/fossasia/engelsystem/issues

Continue ReadingRead and Understand Codacy Reports

Continuous Integration and Automated Testing for Engelsystem

Every software development group tests its products, yet delivered software always has defects. Test engineers strive to catch them before the product is released but they always creep in and they often reappear, even with the best manual testing processes. Using automated testing is the best way to increase the effectiveness, efficiency and coverage of your software testing.

Manual software testing is performed by a human sitting in front of a computer carefully going through application screens, trying various usage and input combinations, comparing the results to the expected behavior and recording their observations. Manual tests are repeated often during development cycles for source code changes and other situations like multiple operating environments and hardware configurations.

Continuous integration (CI) has emerged as one of the most efficient ways to develop code. But testing has not always been a major part of the CI conversation.

In some respects, that’s not surprising. Traditionally, CI has been all about speeding up the coding, building, and release process. Instead of having each programmer write code separately, integrate it manually, and then wait until the next daily or weekly build to see if the changes broke anything, CI lets developers code and compile on a virtually continuous basis. It also means developers and admins can work together seamlessly since the programming and build processes are always in sync.

Continuous Integration (CI) is a development practice that requires developers to integrate code into a shared repository several times a day. Each check-in is then verified by an automated build, allowing teams to detect problems early.

By integrating regularly, you can detect errors quickly, and locate them more easily.

Solve problems quickly

Because you’re integrating so frequently, there is significantly less back-tracking to discover where things went wrong, so you can spend more time building features.

Continuous Integration is cheap. Not continuously integrating is costly. If you don’t follow a continuous approach, you’ll have longer periods between integrations. This makes it exponentially more difficult to find and fix problems. Such integration problems can easily knock a project off-schedule, or cause it to fail altogether.

Continuous Integration brings multiple benefits to your organization:

  • Say goodbye to long and tense integrations
  • Increase visibility which enables greater communication
  • Catch issues fast and nip them in the bud
  • Spend less time debugging and more time adding features
  • Proceed with the confidence you’re building on a solid foundation
  • Stop waiting to find out if your code’s going to work
  • Reduce integration problems allowing you to deliver software more rapidly

“Continuous Integration doesn’t get rid of bugs, but it does make them dramatically easier to find and remove.”

– Martin Fowler, Chief Scientist, ThoughtWorks

Continuous Integration is backed by several important principles and practices.

Practices in Continuous Integration:

  • Maintain a single source repository
  • Automate the build
  • Make your build self-testing
  • Every commit should build on an integration machine
  • Keep the build fast
  • Test in a clone of the production environment
  • Make it easy for anyone to get the latest executable
  • Everyone can see what’s happening
  • Automate deployment

How to do Continuous Integration:

  • Developers check out code into their private workspaces.
  • When done, commit the changes to the repository.
  • The CI server monitors the repository and checks out changes when they occur.
  • The CI server builds the system and runs unit and integration tests.
  • The CI server releases deployable artifacts for testing.
  • The CI server assigns a build label to the version of the code it just built.
  • The CI server informs the team of the successful build.
  • If the build or tests fail, the CI server alerts the team.
  • The team fixes the issue at the earliest opportunity.
  • Continue to continually integrate and test throughout the project.

The CI implemented in Engelsystem are as follows:

  • Travis-CITravis CI is a hosted, distributed continuous integration service used to build and test software projects hosted on GitHub. It is integrated using the .travis.yml file in the root folder.
    language: php
    php:
    - '5.4'
    - '5.5'
    - '5.6'
    - '7.0'
    script: cd test && phpunit
  • Nitpick-CI:Automatic comments on PSR-2 violations in one click, so your team can focus on better code review. It requires one click for integratting with the repository.
  • Circle-CI: CircleCI was founded in 2011 with the mission of giving every developer state-of-the-art automated testing and continuous integration tools. It is integrated using a circle.yml file in the root folder of the repository.
    machine:
    php:
    version: 5.4.5
    deployment:
    master:
    branch: master
    owner: fossasia
    commands:
    - ./deploy_master.sh
    dependencies:
    pre:
    - curl -s http://getcomposer.org/installer | php
    - php composer.phar install -n
    - sed -i 's/^;//' ~/.phpenv/versions/$(phpenv global)/etc/conf.d/xdebug.ini
    
    test:
    post:
    - php test/
    - bash <(curl -s https://codecov.io/bash)
  • Codacy: Check code style, security, duplication, complexity and coverage on every change while tracking code quality throughout your sprints.

Development: https://github.com/fossasia/engelsystem Issues/Bugs:https://github.com/fossasia/engelsystem/issues

Continue ReadingContinuous Integration and Automated Testing for Engelsystem