Engelsystem GSoC 2016 Summary

This blog post I would like to share my work done during the GSoC period (May - August). Doing GSoC has been one of the most wonderful experience in my life. Introduction ENGELSYSTEM - Engelsystem is a volunteer management application for events written in PHP. It is used for coordination among different angels who want to volunteer. END USERS FOR ENGELSYSTEM SYSTEM ADMINS It will make it easy for them to coordinate with the volunteers by the product we developed. VOLUNTEERS /USERS Those who want to volunteer finds it difficult to coordinate, contact admins. This platform helps all those who want to help. Volunteers can register and can allot their shifts. A better User Interface, Integration with different Asian languages will allow the system to be used globally and in their local languages and ease for the volunteers to use. As a part of the project I developed many new features for engelsystem. Implemented a workflow similar to wordpress, Implemented MVC model, Importing user data to database directly through script, Implemented a copy function for shifts, sending messages to entire group/ AngelType, Implemented a search for all Angels View, Date Picker for shifts, New Settings Page for admin where we can write event related information, Deployed the system on different platforms, Added Documentation, Re-factor the code, Improved the code standards, Blog Post tutorials on various technologies. I am really happy with the outcome of the project. I learnt lot of new technologies and was improving day by day. My initial proposal, time line of the project My GSOC 2016 Proposal. I have proposed a many UI changes for the system, Improving Documentation, Refactoring. All of these are done during the period. And I have implemented extra features and surpassed the proposal. I have finished most of the proposed features before the Mid Evaluation. There was a shift in the timeline but I am very happy that the outcome was excellent. I was able to finish all the issues assigned to me on time. What I have done to make Engelsystem better? Overview of my contributions Commits: 172 Additions: 87,528 Deletions: 80,555 Issue Completed: 60 Pull Requests Merged: 36 Scrum Reports: 99 Blog Posts: 15 Daily Timeline during GSOC. Everyday we need to write a scrum report about our progress and submit it before 10 AM local time. Scrum consists of the following information. 1.What did you do yesterday? (which areas, issues you worked on, links) 2.What do you plan to do today? (which areas, issues you want to work on, links) 3.What is currently preventing you from achieving your goals? What blockers do you encounter? I have written total of 99 scrum reports from May 10 - August 23. I have never missed a scrum report. My Activity on Engelsystem Google Group : https://groups.google.com/forum/#!forum/engelsystem Working Environment All the issues for engelsystem are available on github. Whenever we need to implement a new feature or resolve a bug we need to make a issue and start working on it. There are…

Continue ReadingEngelsystem GSoC 2016 Summary

Software Testing

In this post I would like to explain what is software testing and different methods, levels of software testing. What is software testing? Software Testing is the process of evaluating a system or its components with the intent to find whether it satisfies the specified requirements or not. Testing is executing a system in order to identify any gaps, errors, or missing requirements in contrary to the actual requirements. What are the Levels of software testing? Software products are tested at four levels: Unit testing Integration testing System testing Acceptance testing Unit testing During unit testing, modules are tested in isolation. If all modules were to be tested together, it may not be easy to determine which module has the error. Unit testing reduces debugging effort several folds. Programmers carry out unit testing immediately after they complete the coding of a module. Role of Unit Testing Assure minimum quality of units before integration into system Focus attention on relatively small units Testing forces us to read our own code – spend more time reading than writing Automated tests support maintainability and extendibility Integration testing After different modules of a system have been coded and unit tested: modules are integrated in steps according to an integration plan partially integrated system is tested at each integration step. Role of Integration Testing Gain confidence in the integrity of overall system design Ensure proper interaction of components Integration Testing Strategies Big-bang Top-down Bottom-up Critical-first Function-at-a-time As-delivered Sandwich System Testing Gain confidence in the integrity of the system as a whole Ensure compliance with functional requirements Ensure compliance with performance requirements Acceptance Testing Testing performed by the customer or end-user himself to determine whether the system should be accepted or reject Testing is very important step of software development. We have used unit testing for our project Engelsystem. We are developing new features. Interested Developers can work with us. Development: https://github.com/fossasia/engelsystem Issues/Bugs:https://github.com/fossasia/engelsystem/issues

Continue ReadingSoftware Testing

Working with Apache Web Server

In this article, I will examine what apache is how to setup apache for ubuntu or debian and how to view error logs for apache and how to understand what the logs contain which will be useful for web development. What is Apache? Apache is the most popular web server on the internet. It is used to serve more than half of all active websites. It is fast, reliable, and secure. It can be highly customized to meet the needs of many different environments by using extensions and modules. Installing Apache on Ubuntu and Debian If you do not already have Apache installed, you can do so now by issuing the following commands: sudo apt-get update sudo apt-get install apache2 If you visit your VPS's IP address in a web browser, you will get the default Apache index page. We get the following output. your_domain_name_or_ip_address It works! This is the default web page for this server. The web server software is running but no content has been added, yet. Looking into apache error.log files In order to effectively manage a web server, it is necessary to get feedback about the activity and performance of the server as well as any problems that may be occuring. The Apache HTTP Server provides very comprehensive and flexible logging capabilities. Error.log file The server error log, whose name and location is set by the ErrorLog directive, is the most important log file. This is the place where Apache httpd will send diagnostic information and record any errors that it encounters in processing requests. It is the first place to look when a problem occurs with starting the server or with the operation of the server, since it will often contain details of what went wrong and how to fix it. Explanation of log message There is certain information that is contained in most error log entries. For example, here is a typical message. [Sat Aug 20 08:09:38.958885 2016] [:error] [pid 1489] [client 127.0.0.1:40810] PHP Notice: Constant MIN_PASSWORD_LENGTH already defined in /var/www/html/test/engelsystem/config/config.php on line 30, referer: http://localhost/ [Sat Aug 20 08:09:38.958885 2016] The first item in the log entry is the date and time of the message. [:error] The second entry lists the severity of the error being reported. [pid 1489] Indicates process Id [client 127.0.0.1:40810] The fourth entry gives the IP address of the client that generated the error. Fifth one is error message explaining the error. Constant MIN_PASSWORD_LENGTH already defined. File and line number /var/www/html/test/engelsystem/config/config.php on line 30 Error logs are easy to understand. Using error logs we can easily figure out the problem in your server or your code and fix it. For more information about the log files and configuration please visit Apache website Apache provides excellent documentation. Apache log files were useful for me during web development project Engelsystem. We are developing new features. Interested developers can work with us. Development: https://github.com/fossasia/engelsystem Issues/Bugs:https://github.com/fossasia/engelsystem/issues

Continue ReadingWorking with Apache Web Server

Importing database with PHP script

In this post I would like to discuss how to import tables to database directly through PHP script. For our Project Engelsystem we need to import the tables manually by the command line or PHPMYADMIN. Now the user need not worry about the importing table. They are directly imported through the script. Initially we used to import the tables in a sql file by source command or mysql command. $ mysql -u root -p CREATE DATABASE engelsystem; use engelsystem; source db/install.sql; source db/update.sql; Script to import tables directly through PHP. function import_tables() { // get the database variables. global $DB_HOST, $DB_PASSWORD, $DB_NAME, $DB_USER; // file names to import $import_install = '../db/install.sql'; $import_update = '../db/update.sql'; // command to import both the files. $command_install = 'mysql -h' .$DB_HOST .' -u' .$DB_USER .' -p' .$DB_PASSWORD .' ' .$DB_NAME .' < ' .$import_install; $command_update = 'mysql -h' .$DB_HOST .' -u' .$DB_USER .' -p' .$DB_PASSWORD .' ' .$DB_NAME .' < ' .$import_update; $output = array(); // execute the command exec($command_install, $output, $worked_install); exec($command_update, $output, $worked_update); // test whether they are imported successfully or not switch ($worked_install && $worked_update) { case 0: return true; case 1: return false; } } Once we execute the above script the tables will be imported automatically. User need not import the tables manually. In this way we can import tables using script. For more information visit about project please visit here. Development: https://github.com/fossasia/engelsystem Issues/Bugs:https://github.com/fossasia/engelsystem/issues

Continue ReadingImporting database with PHP script

Configuring Document Root Apache2 Ubuntu

In this post I will explain how to setup document root for apache server. Initially the document root is set to /var/www/html by default. We need to change it when we have web applications in /var/www/html/folder. I will show an example how to configure document root for our project engelsystem. Engelsystem is downloaded to /var/www/html/ . We must make sure to point our apache2 document root to the Engelsystem directory to prevent any user from accessing anything other than the public/ directory for security reasons. Changing apache2 document root The default document root is set in the 000-default.conf file that is under /etc/apache2/sites-available folder. $ cd /etc/apache2/sites-available $ sudo nano 000-default.conf While the file is opened change DocumentRoot /var/www/ with your new folder e.g DocumentRoot /var/www/html/engelsystem/public where your index.php file resides Set the right Apache configuration The configuration of the /var/www folder is under /etc/apache2/apache2.conf. Edit this file to add the configuration of your new document root. $ sudo nano/etc/apache2/apache2.conf Copy the following: <Directory /var/www/> Options Indexes FollowSymLinks AllowOverride None Require all granted </Directory> and change the directory path: <Directory /var/www/html/engelsystem/public> Options Indexes FollowSymLinks AllowOverride None Require all granted </Directory> Restart Apache Now we can view the engelsystem at localhost/ or http://[i.p.address]/ $ sudoservice apache2 restart

Continue ReadingConfiguring Document Root Apache2 Ubuntu

Installing Engelsystem in Five Minutes

Here are the instructions to install engelsystem on your local server in just five minutes. Prerequisites 1.1 PHP 5.4.x (cgi-fcgi) 1.2 MySQL-Server 5.5+ or MariaDB 1.3 Webserver ( Apache/Nginx/lighttpd) Step 1: Download or Clone the repository git clone --recursive https://github.com/fossasia/engelsystem.git Download and unzip the engelsystem from here Step 2: Create a Database and a User Using the MySQL Client You can create MySQL/MariaDB user and database quickly and easily by running mysql from the shell. The syntax is shown below and the dollar sign is the command prompt: $ mysql -u adminusername -p Enter password: mysql> CREATE DATABASE databasename; Query OK, 1 row affected (0.00 sec) Step 3: Set up config.php Renname config/config-sample.php to config/config.php, and add your database information. // MySQL-Connection Settings $DB_HOST = "localhost"; $DB_USER = "username_here"; $DB_PASSWORD = "password_here"; $DB_NAME = "database_name_here"; Step 4: Upload the files and configure Document Root Now you will need to decide where on your domain you'd like your engelsystem to appear: Upload the engelsystem files to the desired location on your web server. For Apache server we need to upload to /var/www/html/ Now we need to configure document root. Change the document root in /etc/apache2/sites-available/000-default.conf from /var/www to /var/www/html/engelsystem/public. Now we can view engelsystem at localhost or http://[i.p.address]. Following command will change the document root from /var/www/ to /var/www/html/engelsystem/public. $ sudo sed -i -e 's/DocumentRoot \/var\/www\/html/DocumentRoot \/var\/www\/html\/engelsystem\/public/g' /etc/apache2/sites-available/000-default.conf Step 5: Run the Install Script Point a web browser to start the installation script. We will be redirected to install page where we need to fill in the information for installation. Once we fill the information and press install engelsystem. All the tables will be imported and we are redirected to login page. In this way we can install engelsystem using installation script in just five minutes. For more information visit here Development: https://github.com/fossasia/engelsystem Issues/Bugs:https://github.com/fossasia/engelsystem/issues        

Continue ReadingInstalling Engelsystem in Five Minutes

Writing Installation Script for Github Projects

I would like to discuss how to write a bash script to setup any github project and in particular PHP and mysql project. I would like take the example of engelsystem for this post. First, we need to make a list of all environments and dependencies for our github project. For PHP and mysql project we need to install LAMP server. The script to install all the dependencies for engelsystem are echo "Update your package manager" apt-get update echo "Installing LAMP" echo "Install Apache" apt-get install apache2 echo "Install MySQL" apt-get install mysql-server php5-mysql echo "Install PHP" apt-get install php5 libapache2-mod-php5 php5-mcrypt echo "Install php cgi" apt-cache search php5-cgi We can even add echo statements with instructions. Once we have installed LAMP. We need to clone the project from github. We need to install git and clone the repository. Script for installing git and cloning github repository are echo "Install git" apt-get install git cd /var/www/html echo "Cloning the github repository" git clone --recursive https://github.com/fossasia/engelsystem.git cd engelsystem Now we are in the project directory. Now we need to set up the database and migrate the tables. We are creating a database engelsystem and migrating the tables in install.sql and update.sql. echo "enter mysql root password" # creating new database engelsystem echo "create database engelsystem" | mysql -u root -p echo "migrate the table to engelsystem database" mysql -u root -p engelsystem < db/install.sql mysql -u root -p engelsystem < db/update.sql Once we are done with it. We need to copy config-sample.default.php to config.php and add the database and password for mysql. echo "enter the database name username and password" cp config/config-sample.default.php config/config.php Now edit the config.php file. Once we have done this we need to restart apache then we can view the login page at localhost/engelsystem/public echo "Restarting Apache" service apache2 restart echo "Engelsystem is successfully installed and can be viewed on local server localhost/engelsystem/public" We need to add all these instructions in install.sh file. Now steps to execute a bash file. Change the permissions of install.sh file $ chmod +x install.sh Now run the file from your terminal by executing the following command $ ./install.sh Developers who are interested in contributing can work with us. Development: https://github.com/fossasia/engelsystem Issues/Bugs:https://github.com/fossasia/engelsystem/issues

Continue ReadingWriting Installation Script for Github Projects

Dockerizing PHP and Mysql application

Docker Introduction Docker is based on the concept of building images which contain the necessary software and configuration for applications. We can also build distributable images that contain pre-configured software like an Apache server, Caching server, MySQL server, etc. We can share our final image on the Docker HUB to make it accessible to everyone. First we need to install docker on our local machine. Steps to install docker for ubuntu Prerequisites Docker requires a 64-bit installation regardless of your Ubuntu version. Your kernel must be 3.10 at minimum. The latest 3.10 minor version or a newer maintained version are also acceptable. To check your current kernel version, open a terminal and use uname -r to display your kernel version: $ uname -r 3.11.0-15-generic Update your apt sources Docker’s APT repository contains Docker 1.7.1 and higher. To set APT to use packages from the new repository: Log into your machine as a user with sudo or root privileges. Open a terminal window. Update package information, ensure that APT works with the https method, and that CA certificates are installed. $ sudo apt-get update $ sudo apt-get install apt-transport-https ca-certificates Add the new GPG key. $ sudo apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D Open the /etc/apt/sources.list.d/docker.list file in your favorite editor.If the file doesn’t exist, create it. Remove any existing entries. Add an entry for your Ubuntu operating system On Ubuntu Trusty 14.04 (LTS) deb https://apt.dockerproject.org/repo ubuntu-trusty main Save and close the /etc/apt/sources.list.d/docker.list file. Update the APT package index. $ sudo apt-get update For Ubuntu Trusty, Wily, and Xenial, it’s recommended to install the linux-image-extra kernel package. The linux-image-extra package allows you use the aufs storage driver. Install both the required and optional packages. $ sudo apt-get install linux-image-generic-lts-trusty INSTALL Log into your Ubuntu installation as a user with sudo privileges. Update your APT package index. $ sudo apt-get update Install Docker. $ sudo apt-get install docker-engine Start the docker daemon. $ sudo service docker start Verify docker is installed correctly. $ sudo docker run hello-world This command downloads a test image and runs it in a container. When the container runs, it prints an informational message. If it runs successfully then docker is installed. Docker Images Docker images are the basis of containers. An image can be considered a class definition. We define its properties and behavior. To browse the available images, we can visit the Docker HUB and run docker pull <image> to download them to the host machine. Listing images on the host $ docker images REPOSITORY TAG IMAGE ID CREATED SIZE ubuntu 14.04 1d073211c498 3 days ago 187.9 MB busybox latest 2c5ac3f849df 5 days ago 1.113 MB training/webapp latest 54bb4e8718e8 5 months ago 348.7 MB Working with Dockerfile Create a Dockerfile in your PHP project. This is the docker file for engelsystem. Our Dockerfile is now complete and ready to be built: Building the Image The docker build . command will build the Dockerfile inside the current directory: Our image is now labeled and tagged. The final…

Continue ReadingDockerizing PHP and Mysql application

Implementing Database Migrations

Database Migrations Using Phinx Database migrations can transform your database in many ways such as creating new tables, inserting rows, adding indexes and modifying columns. It avoids the use of writing MYSQL by hand and instead offers a powerful API for creating migrations using PHP code. Advantages of using Phinx Phinx keeps track of which migrations have been run so you can worry less about the state of your database and instead focus on building better software Each migration is represented by a PHP class in a unique file. We can write our migrations using the Phinx PHP API, or raw SQL. Phinx has an easy installation process and easy to use command line instructions and easy to Integrate with various other PHP tools (Phing, PHPUnit) and web frameworks. Installating Phinx Phinx should be installed using Composer. Composer is a tool for dependency management in PHP. We need to require the dependency in composer.json. php composer.phar require robmorgan/phinx Then run Composer: php composer.phar install --no-dev Now Create a folder in your database directory called migrations with adequate permissions. It is where we write our migrations. In engelsystem it is created in db directory Phinx can now be executed from within your project: php vendor/bin/phinx init Writing Migrations For SQL files Creating a New Migration Let’s start by creating a new Phinx migration. Run Phinx using the create command. This will create a new migration in the format YYYYMMDDHHMMSS_my_new_migration.php where the first 14 characters are replaced with the current timestamp down to the second. This will create a skeleton file with a single method. $ php vendor/bin/phinx create MyNewMigration The File looks something like this Explaining the File The AbstractMigration Class Abstraction class provides the necessary support to create your database migrations. All Phinx migrations extend from the AbstractMigration class. Phinx provides different methods in the abstraction class like change, up and down method. The Change Method This is the default migration method. I will explain how to write the change method for an example MYSQL query. For example following MYSQL query can also be executed using Phinx change method. MYSQL Query ALTER TABLE `AngelTypes` ADD `requires_driver_license` BOOLEAN NOT NULL; Equivalent change method public function change() { $table = $this->table('AngelTypes'); $table->addColumn('requires_driver_license', 'boolean', array('null' => 'false')) ->update(); } The Up Method We should use the up method to transform the database with your intended changes. For example following MYSQL query to create a new settings table can be executed using equivalent up method. MYSQL Query DROP TABLE IF EXISTS `Settings`; CREATE TABLE IF NOT EXISTS `Settings` ( `event_name` varchar(255) DEFAULT NULL, `buildup_start_date` int(11) DEFAULT NULL, `event_start_date` int(11) DEFAULT NULL, `event_end_date` int(11) DEFAULT NULL, `teardown_end_date` int(11) DEFAULT NULL, `event_welcome_msg` varchar(255) DEFAULT NULL ) ENGINE=InnoDB DEFAULT CHARSET=utf8 ; Equivalent up method public function up() { $table = $this->table('Settings'); $table->addColumn('event_name', 'string', array('limit' => 255)) ->addColumn('buildup_start_date', 'integer', array('limit' => 11)) ->addColumn('event_start_date', 'integer', array('limit' => 11)) ->addColumn('event_end_date', 'integer', array('limit' => 11)) ->addColumn('teardown_end_date', 'integer', array( 'limit' => 11)) ->addColumn('event_welcome_msg', 'string', array('limit' => 255)) ->save(); }We have now created…

Continue ReadingImplementing Database Migrations

Import Excel File Data in MYSQL Database using PHP

In this post I will explain how to Import Excel Sheet Data in MySQL Database using PHP. If you follow the below steps we will successfully achieve the target. For this tutorial we are going to work with a sample CSV file, which has the following fields. I will show an example of User Table of Engelsystem which contains the following fields Nick Name, First Name, Last Name, Email, Current City, Password, Mobile Number, Age. Steps to Import Excel File Data in MYSQL Database using PHP Step 1 First you have to create mysql database. mysql> CREATE DATABASE engelsystem; Step 2 Create table in your choosen database. mysql> use DATABASE engelsystem; The Table schema looks something like this. -- Table structure for table `User` mysql> CREATE TABLE IF NOT EXISTS `User` ( `UID` int(11) NOT NULL AUTO_INCREMENT, `Nick` varchar(23) NOT NULL DEFAULT '', `First Name` varchar(23) NOT NULL DEFAULT '', `Last Name` varchar(23) NOT NULL DEFAULT '', `email` varchar(123) DEFAULT NULL, `Age` int(4) DEFAULT NULL, `current_city` varchar(255) DEFAULT NULL, `Password` varchar(128) DEFAULT NULL, `Mobile` varchar(40) DEFAULT NULL, PRIMARY KEY (`UID`), UNIQUE KEY `Nick` (`Nick`), ) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=5 ; Step 3 create your php excelsheet data uploading file. This is a sample code which I used for my project. Ex: import_data.php <?php function admin_import() { if (isset($_REQUEST['upload'])) { $ok = true; $file = $_FILES['csv_file']['tmp_name']; $handle = fopen($file, "r"); if ($file == NULL) { error(_('Please select a file to import')); redirect(page_link_to('admin_export')); } else { while(($filesop = fgetcsv($handle, 1000, ",")) !== false) { $nick_name = $filesop[0]; $first_name = $filesop[1]; $last_name = $filesop[2]; $email = $filesop[3]; $age = $filesop[4]; $current_city = $filesop[5]; $password = $filesop[6]; $mobile = $filesop[7]; // example error handling. We can add more as required for the database. if ( strlen($email) && preg_match("/^[a-z0-9._+-]{1,64}@(?:[a-z0-9-]{1,63}\.){1,125}[a-z]{2,63}$/", $mail) > 0) { if (! check_email($email)) { $ok = false; $msg .= error(_("E-mail address is not correct."), true); } } // error handling for password if (strlen($password) >= MIN_PASSWORD_LENGTH) { $ok = true; } else { $ok = false; $msg .= error(sprintf(_("Your password is too short (please use at least %s characters)."), MIN_PASSWORD_LENGTH), true); } // If the tests pass we can insert it into the database. if ($ok) { $sql = sql_query(" INSERT INTO `User` SET `Nick Name`='" . sql_escape($nick_name) . "', `First Name`='" . sql_escape($first_name) . "', `Last Name`='" . sql_escape($last_name) . "', `email`='" . sql_escape($email) . "', `Age`='" . sql_escape($age) . "', `current_city`='" . sql_escape($current_city) . "', `Password`='" . sql_escape($password) . "', `mobile`='" . sql_escape($mobile) . "',"); } } if ($sql) { success(_("You database has imported successfully!")); redirect(page_link_to('admin_export')); } else { error(_('Sorry! There is some problem in the import file.')); redirect(page_link_to('admin_export')); } } } //form_submit($name, $label) Renders the submit button of a form //form_file($name, $label) Renders a form file box return page_with_title("Import Data", array( msg(), div('row', array( div('col-md-12', array( form(array( form_file('csv_file', _("Import user data from a csv file")), form_submit('upload', _("Import")) )) )) )) )); } ?> Step 4 The view of import_data.php looks something like this. Now that import_data.php is up and…

Continue ReadingImport Excel File Data in MYSQL Database using PHP