Allowing web-user on apache server to run scripts as root

Allowing web-user on apache server to run scripts as root

If you are new to this, you might be wondering, what the hell is a web user anyways?

So let’s say that you need a server which hosts a simple web page and does a particular task based on data entered into that web-page.

The normal way of doing this is to navigate to /var/www/html and place the web page you want to host here.

You also need to put your php script in this folder so that it is accessible from the website.
This php script will take in the data from your web-page and run the necessary commands that you need to be executed on the server.( I am assuming you are not using “The Real Dev Language” for now. :p )

I will be using a simple web page and script that I have made for this post.

<html>
<head>
  <title>Apk Generator</title>
  <meta charset="utf-8">
  <meta http-equiv="X-UA-Compatible" content="IE=edge">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <link href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.6/css/bootstrap.min.css" rel="stylesheet" integrity="sha384-1q8mTJOASx8j1Au+a5WDVnPi2lkFfwwEAa8hDDdjZlpLegxhjVME1fgjWPGmkzs7" crossorigin="anonymous">
  <link href='https://fonts.googleapis.com/css?family=Roboto:400,100' rel='stylesheet' type='text/css'>
  <link href="css/main.css" rel="stylesheet">
</head>
<body>
<div class="container"><br><br>
<form name="htmlform" id="form" enctype="multipart/form-data" class="col-md-offset-4 col-xs-offset-2 col-xs-8 col-md-4 form-group generator_form" >
  <label for="name">Email</label>
      <input type="email" class="form-control" id="Email" name="Email">
      <br>
      <input type="hidden" id="theme" name="theme" value="light">

      <label for="name">App's Name</label>
      <input type="text" class="form-control" id="App_Name" name="App_Name">
      <br>
      <label> Choose your data source </label>
      <ul style="list-style-type:none">
        <li><input type="radio" name="datasource" value="jsonupload"> Upload your own JSON files </input></li>
        <li><input type="radio" name="datasource" value="eventapi"> API endpoint of event on OpenEvent </input></li>
      </ul>
      <br>
      <section id="eventapi-input" style="display:none;">
        <label for="apiendpoint">Link to Open Event API endpoint</label>
        <input type="url" class="form-control"
        id="Api_Link" name="Api_Link">
      </section>
      <br>
      <section id="jsonupload-input" style="display:none;">
        <input type="file" name="uploadZip" id="uploadZip" class="form-control"/>
        <br>
      </section>
      <br>
      <input type="hidden" name="assetmode" value="download">
      <center>
        <br>
        <div id="status"></div>
        <br>
<tr>
 <td colspan="5" style="text-align:center">
  <button type="submit">Generate and Download app</button>
</td>
</tr>
</table>
</form>
<script src="https://www.gstatic.com/firebasejs/live/3.0/firebase.js"></script>
<script src="https://code.jquery.com/jquery-3.1.0.min.js"></script>
<script src="https://code.jquery.com/ui/1.12.0/jquery-ui.js"></script>
<script>
  $('input:radio[name="datasource"]').change(
  function() {
    if ($(this).is(':checked')) {

      if ($(this).val() === 'mockjson') {
        $('#jsonupload-input').hide(100);
        $('#eventapi-input').hide(100);
      }

      if ($(this).val() === 'jsonupload') {
        $('#jsonupload-input').show(100);
        $('#eventapi-input').hide(100);
      }

      if ($(this).val() === 'eventapi') {
        $('#eventapi-input').show(100);
        $('#jsonupload-input').hide(100);
      }
    }
  });
  var $ = jQuery;
  var timestamp = Number(new Date());
  var form = document.querySelector("form");
      form.addEventListener("submit", function(event) {
        event.preventDefault();
        var ary = $(form).serializeArray();
        var obj = {};
        for (var a = 0; a < ary.length; a++) obj[ary[a].name] = ary[a].value;
          console.log("JSON",obj);
        if(obj.Email == "" || obj.App_Name ==""){
          alert("It seems like you forgot to fill up your email address or the app's name");
          setTimeout("location.reload(true);", 1);
        }
        else{
	alert("Please wait while we generate the app, meanwhile you can stick around to directly download it.The app will also be emailed to you."); 
            $.ajax({
              type: "POST",
              url: "/test.php",
              data: { timestamp : timestamp },
              success: function(response){
                console.log("Success",response);
                window.location = response;
              }
            });
        }
      });
    </script>
    </div>
</body>
</html>

This is basically a web page with some inputText widgets which accept response and send it to a php file named test.php on the server via an AJAX post.

<?php
if(isset($_POST['timestamp']))
{
    $uid = escapeshellcmd($_POST['timestamp']);
    exec("sudo sh /var/www/email.sh $uid");
}
?>

This php script will call a bash script which in turns an email to me with the user’s timestamp as the subject.

Well, here is where the problem arises, as I am trying to run the bash file as root.

You might wonder as to why is this such a big issue?
Why can’t we do that?

Well, we can surely do that on the server but the point to be noted here is that we are not running this script directly from the server.

We are running it from a web page which is hosted on a server.

So our user here is a web user aka www-data rather than being a root user.

The web user is not provided root access by default, but there are ways to get this done.

Solution 1 :

Allow the web user to run only specific scripts as root.

Please note that this is not a ideal workaround.
Ideally your web user should not have root access in any case.
Since that’s cleared up, lets proceed.

This can be done by editing your sudoers list and adding www-data to it.
Open up your terminal and enter the following command.

sudo visudo

Next up, navigate to the end of the file and add the following command there

www-data = (root) NOPASSWD: /path/to/script.sh

In case you have to execute one script as root which in turn executes some more scripts as root, you don’t need to set the path to all of them over here.
Doing it only for the parent script will do the job.

Solution 2 :

Using SuExec

DigitalOcean blog has a very good article on how to execute python scripts as root via the web user through cgi.

You can go through the article here :https://www.digitalocean.com/community/tutorials/how-to-use-suexec-in-apache-to-run-cgi-scripts-on-an-ubuntu-vps

Well, that was all about my findings on properly handling sudo requirements for your web user on your apache server.

I’ll be adding more solutions as I find them along the way.
Meanwhile feel free to comment below your thoughts, suggestions and queries.

Cheers.

Sending mails using Sendgrid on Nodejs

The open-event webapp generator project needs to send an email to the user notifying him whenever generating the webapp is finished, containing the links to the preview and zip download.

For sending emails, the easiest service we found we could use was SendGrid  which provides upto 15000 free emails a month for students who have a Github Education Pack. (It anyway provides 10000 free emails to all users).

To use sendgrid, it’s best to use the sendgrid npm module that SendGrid officially builds. To get that installed just use the following command –

npm install --save sendgrid

Also, once you have made an account on Sendgrid, create an API key, and save it as an environment variable (so that your API key is not exposed in your code). For example in our project, we save it in the environment variable SENDGRID_API_KEY
To make it permanent you can add it to your ~/.profile file

export SEDGRID_API_KEY=xxxxxxxxxxxxxxxxxxx

The actual sending takes place in the mailer.js script in our project.

Basically we are using the mail helper class provided in the sendgrid module, and the bare minimum code required to send a mail is as follows

  var helper = require('sendgrid').mail
  from_email = new helper.Email([email protected]')
  to_email = new helper.Email([email protected]')
  subject = 'Hello World from the SendGrid Node.js Library!'
  content = new helper.Content('text/plain', 'Hello, Email!')
  mail = new helper.Mail(from_email, subject, to_email, content)
 
  var sg = require('sendgrid')(process.env.SENDGRID_API_KEY);
  var request = sg.emptyRequest({
    method: 'POST',
    path: '/v3/mail/send',
    body: mail.toJSON()
  });
 
  sg.API(request, function(error, response) {
    console.log(response.statusCode)
    console.log(response.body)
    console.log(response.headers)
  })

You need to replace the to and from emails to your requirements.

Also as you can see in our project’s code, if you want to send HTML formatted data, you can change the content type from text/plain to text/html and then add any html content (as a string) into the content.

Creating an API in PHP

One of the key components of my GSoC Project was to have a POST API for the Android App generator.

This was required so that the app generator could be plugged into the server and can be called directly instead of someone manually visiting the webpage and entering his/her details.

It takes in a JSON input and compiles and emails the app to the organizer based on his email address in the input JSON.

The input to the API will look something like this :

{
“email”: [email protected],
“app_name”: “MyApp”,
“endpoint”: “https://open-event-dev.herokuapp.com/api/v2
}

Once the data is sent, on the server I have a php file which intercepts the requests and performs an action based on the request.

<?php
function sendResponse($data) {
    header('Content-type: application/json');
    echo json_encode($data);
    die();
}
/* If the request isn't a POST request then send an error message*/
if ($_SERVER['REQUEST_METHOD'] != 'POST') {
    sendResponse([
        "status"=>"error",
        "code"=>405,
        "message"=>"Method Not Allowed",
    ]);
}
/* Store the input received in a variable named body */
$body = json_decode(file_get_contents('php://input'), true);
/* If the user is nissing any important input parameters, don't process the request */
if (!array_key_exists('email', $body) || !array_key_exists('app_name', $body) || !array_key_exists('endpoint', $body)) {
    sendResponse([
        "status"=>"error",
        "code"=>422,
        "message"=>"Unprocessable entity",
    ]);
}
$uid = mt_rand(1000,9999). "_" .time();  //A random User ID
/* Extracting variables from the body */
$email = escapeshellcmd($body['email']);
$appName = escapeshellcmd($body["app_name"]); 
$endpoint = escapeshellcmd($body["endpoint"]);

/* Run a script based on the input parameters */
exec("sudo python /var/www/html/api/appgenserver.py $email $appName $endpoint");

The code above is pretty much self explanatory.

So basically, first we check for a valid request (GET/POST) and throw an error if it is invalid.

Next up, for a valid request we store the body into a variable and then execute a followup script as per our needs using data from this response.

This PHP file should be located in the public-html (/var/www/data) of the server so as to be accessible from outside of the server.

You can test out this API by calling it directly by prepending the server’s ip address to the name of php file containing this code.

Something like :

domain-name.com/api/api.php

You can also use Postman for Chrome or RESTClient for Firefox for making API calls easily.

Well, that’s it then!

You can easily modify the PHP code provided and modify it to suite your needs for making you own API.

Let me know your thoughts and your queries in the “response” 😉 below.

Until next time.

Handling images inside ZIP or with URL’s

The Open-event webapp generator now provides the user a facility to upload the speakers and sponsors images inside the ZIP as images/speakers and images/sponsors.  After that the user will just have to specify the path of the images inside JSON such as /images/speakers/photoname.jpg  instead of URL’s.

How It Works ?

Whenever a user uploads a ZIP containing different files, it is extracted in dist/ folder according to the filename inside ZIP.

//fold.js... function to extract files according to folder name

var admZip = require('adm-zip');
const distPath = __dirname + '/../../../dist';
const uploadsPath = __dirname + '/../../../uploads';
const mockPath = __dirname + '/../../../mockjson'

copyUploads: function(appFolder, uploadedFile) {
 const appPath = distPath + '/' + appFolder;
 fs.mkdirpSync(appPath + '/json');
 var zip = new admZip(uploadedFile);
 var zipEntries = zip.getEntries(); 

 zipEntries.forEach(function(zipEntry) {
 
 switch(zipEntry.entryName){
 case 'images/speakers/':
 zip.extractEntryTo("images/speakers/", appPath ); 
 break;
 case 'images/sponsors/':
 zip.extractEntryTo("images/sponsors/", appPath ); 
 break;
 case 'audio/':
 zip.extractEntryTo("audio/", appPath);
 break;
 case 'sessions':
 zip.extractEntryTo("sessions", appPath +'/json/');
 break;
 case 'speakers':
 zip.extractEntryTo("speakers", appPath +'/json/');
 break;
 case 'microlocations' :
 zip.extractEntryTo("microlocations", appPath+'/json/');
 break;
 case 'event' :
 zip.extractEntryTo("event", appPath +'/json/');
 break;
 case 'sponsors' :
 zip.extractEntryTo("sponsors", appPath +'/json/');
 break;
 case 'tracks':
 zip.extractEntryTo("tracks", appPath +'/json/');
 break;
 default:
 }

This will send all the speaker images to images/speakers in dist and all sponsors images to images/sponsors in dist ( dist is the folder served to the user).

 

const appFolder = reqOpts.email + '/' + slugify(reqOpts.name);
 speakers.forEach((speaker) => {
  if ((speaker.photo !== null) && (speaker.photo.substring(0, 4) ===     'http')) {
 speaker.photo = urlencode(distHelper.downloadSpeakerPhoto(appFolder,    speaker.photo));
 }
 else {
 var reg = speaker.photo.split('');
 if(reg[0] =='/'){
 speaker.photo =    urlencode(speaker.photo.substring(1,speaker.photo.length));
 }
 
 }

 });
//dist.js

downloadSpeakerPhoto: function(appFolder, photoUrl) {
 const appPath = distPath + '/' +appFolder;
 const photoFileName = photoUrl.split('/').pop();
 const photoFilePath = 'images/speakers/' + photoFileName;

 console.log('Downloading photo : ' + photoFileName);
 downloadFile(photoUrl, appPath + '/' + photoFilePath);
 return photoFilePath;
 },

The code above shows how the image URL’s are split on the basis of ‘/’ to get the last element of the URL that is photoFileName. After that the image can be written inside the folder.

images/speakers/' + photoFileName

This code also handles the case when the images are downloaded by taking URL paths from JSON. If the photo in JSON has a string matched with ‘HTTP’ .Then it is downloaded in images/speakers/ . Finally, the image is downloaded with the same path as specified in JSON which can then be easily related in HTML image tag.

//dist.js Function to download files

const downloadFile = function(url, filePath) {
 const fileStream = fs.createWriteStream(filePath);

 fileStream.on('error', function(err) {
 console.log(err);
 });
 try {
 request(url).pipe(fileStream);
 } catch (err) {
 console.log(err);
 }
};

That’s how the images inside the ZIP and the image URL’s inside JSON files are handled in Open-event webapp generator.

Using Heroku pipelines to set up a dev and master configuration

The open-event-webapp project, which is a generator for event websites, is hosted on heroku. While it was easy and smooth sailing to host it on heroku for a single branch setup, we moved to a 2-branch policy later on. We make all changes to the development branch, and every week once or twice, when the codebase is stable, we merge it to master branch.

So we had to create a setup where  –

master branch –> hosted on –> heroku master

development branch –> hosted on –> heroku dev

Fortunately, for such a setup, Heroku provides a functionality called pipelines and a well documented article on how to implement git-flow

 

First and foremost, we created two separate heroku apps, called opev-webgen and opev-webgen-dev

To break it down, let’s take a look at our configuration. First step is to set up separate apps in the travis deploy config, so that when development branch is build, it pushed to open-webgen-dev and when master is built, it pushes to opev-webgen app. The required lines as you can see are –

https://github.com/fossasia/open-event-webapp/blob/master/.travis.yml#L25

https://github.com/fossasia/open-event-webapp/blob/development/.travis.yml#L25

Now, we made a new pipeline on heroku dashboard, and set opev-webgen-dev and opev-webgen in the staging and production stages respectively.

Screenshot from 2016-07-31 04-33-30 Screenshot from 2016-07-31 04-34-41

Then, using the “Manage Github Connection” option, connect this app to your github repo.

Screenshot from 2016-07-31 04-36-17

Once you’ve done that, in the review stage of your heroku pipeline, you can see all the existing PRs of your repo. Now you can set up temporary test apps for each PR as well using the Create Review App option.

Screenshot from 2016-07-31 04-37-38

So now we can test each PR out on a separate heroku app, and then merge them. And we can always test the latest state of development and master branches.

Working with Styles and Themes in Android

All those who have worked with styles and themes know that they’re hard to get right. We tend to get frustrated when we work with them. The hierarchy easily devolves into spaghetti code. How often did you want to change a style but feared you might break the continuity of the design of the app somewhere or the other.

I ran into a similar situation recently. I had to change the whole app’s style’s and theme by just changing the colors etc. in one location. This was for the Open Event android project where we wanted that while generating an apk by the apk generator we could change the color scheme of the app and could make it customisable for the needs of the organisations.

So, I’ll be talking about styling different views in this post. This shall be a long post!

When should we use styles

First of all, most of us get confused on when should we use styles instead of an inline attribute. Now I am going to show the rules that I follow:

When you have multiple views that should look identical ( Perhaps that do similar things)

Few Examples :

  • Payment screens. You want to get the user through a bunch of ordering and payment screens. You need similar kind of buttons there to make it look like a continuous process. Hence we make the Buttons follow one particular style
<style name="Payment_Buttons">
    <item name="android:minWidth">@dimen/button_min_width</item>
    <item name="android:minHeight">@dimen/button_min_height</item>
    <item name="android:background">@color/my_choice_color</item>
</style>

Try to use themes to tweak default styles

Themes provide a way of defining the default style of many widgets. For example :

If you want to define the default button for all of your payment screens in the example above, you can do something like :

<style name="ButtonTheme">
    <item name="android:buttonStyle">@style/MyButton</item>
</style>

But note that if you’re tweaking the default style, the only tricky part is to figure out the parent of your style but that’s really dificult due to a lot of variation within the different versions of android. If you’re using something that’s part of the AppCompat, then it’s okay. you don’t need to worry about the variations but when you want to style something not in AppCompat, then the main problem arises. So For example I want a button to be Holo until kitkat and then Material starting Lollipop, I’ll do something like this :

In values/styles.xml –

<style name="ButtonParent" Parent = "android:Widget.Holo.Button" />
<style name="ButtonParent.Holo">
    <item name="android:background">@drawable/my_bg</item>
</style>

Then in values-v21/styles.xml:

<style name="ButtonParent" parent ="android:Widget.Material.Button/>

This makes the button consistent with guidelines and the app looks perfect.

Now, Themes vs Styles

This is a topic which most of the developers don’t know about. They get confused on what is the difference between them. I was also not totally clear about this until recently. A theme is infact a style, the only difference is the usage.

  • We set a theme in the Manifest of the app or an activity
  • We set a style in a layout file or a widget
  • There are more styles than themes (Checkout styles.xml and themes.xml)
  • Definition of a theme is in the essence jsut a collection of references to stlyes that the theme will use.
  • To elaborate, let’s see the example of Theme.Holo :

It has a combination of

  1. Widget.Holo.Button
  2. Widget.Holo.Button.Small
  3. TextAppearence.Holo.Small
  4. TextAppearence.Holo.Small.Inverse

So, There can be different styles like this which can be referenced in a theme. Themes can be divided into 2 parts : General themes and sub themes

You can have a general them like Theme.ABC and if you want a variation of this general theme, for example no actionbar, you can add another theme like Theme.ABC.NoActionBar . This theme will not have the ActionBar

Inheritance

One of the interesting things that most people don’t know about is inhertance of styles/themes. What do I mean by this is that you can use existing styles and create some variations to suit different needs. There are 2 ways to use this inheritance. I’m going to try to explain and elorate on them :

  1. With a parent attribute

This is the most common way to use it and the way that most of the developers learn it while working on styles for the first time.

So how actually do we use it?

<style name = "Child" parent = "@style/Parent">
</style>

Here the child inherits all the properties of the style with the name “Parent” and define new properties in this style name “Child” where they can define new properties they want on top of the parent style.

2. With implicit style names

The other way to inherit styles/themes using the implicit way. Instead of setting a parent attribute, just prefix your new style/theme with the name of its parent and a dot. Something like this :

<style name = "Parent.Child">
</style>

This works the same as the previous method. Using this reduces some time to write additional parameter and is used by almost all experienced developers.

Plus you get some checks as well while writing code in Android studio. But be careful while using this as you need to take care of somethings :

For example,

  • The Parent style/theme needs to exist, Otherwise an error
  • You cannot inherit default themes and styles. For example you can’t create
<style name = "Theme.Holo.myTheme">

but you do this

<style name = "myTheme" parent = "Theme.Holo">

I know this can be overwhelming for a person who’s just starting with styles and themes. Trust me I was also not able to understand the concepts on the first go. I had to spend some time to grasp all that can be done using styles and themes. So I think this should be it for this blog. It’s already gotten pretty big.

Be sure to check out the Open event android project here and the usage of styles and themes there. Ciao till next time!

Creating a Widget for your Android App

Having a widget for your app, not only helps it to stand out among its alternatives but also provides user information on the go without having to open the app.

Keeping this thought in mind, I decided to make a widget for my GSoC project.

Do you want to know how to do it?
Hang around and I’ll guide you through the steps involved.

Step 1 :

Creating a new widget from Android Studio.

Open up your project for which you need a widget and navigate to the project’s java source.
Create a new sub-package there named widget.
Right click on the newly created sub-package and select the New->Widget option from there.

new_widget

Follow the instructions on the next screen.

screenshot-area-2016-07-30-002554
Most of the fields here are pretty much self explanatory

After doing this and running the app in your device, you will be able to see a widget for your app in the widget picker.

Screenshot_20160730-003515_01

Well, that’s it then? :p

Just kidding, this was the easy part, off to more harder things now!

Step 2 :

Populating the widget with data.

Now, there can be 2 broad type of widgets Information Widgets and Collection Widgets.

Information widgets are simple widgets that are used to display an information that changes with time, for example Weather Widget or a Clock Widget.

Whereas, collection widgets are widgets which display a collection of data, for example the GMail widget is a collection widget.
These are relatively complex and harder than the Information Widgets.

In this post, we will focus on making a Collection Widget.

For Collection widgets, we need two layout files, one for the widget and one for each item in the widget collection.

Go ahead and create the two layout files.
The wizard automatically generates the widget_layout.xml for you, you just need to edit it up.

stock_layout.xml
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:orientation="vertical">

    <LinearLayout
        android:layout_width="match_parent"
        android:id="@+id/widget_toolbar"
        android:layout_height="?android:attr/actionBarSize"
        android:background="@color/colorPrimary">

        <ImageView
            android:layout_width="wrap_content"
            android:layout_height="match_parent"
            android:layout_gravity="center"
            android:src="@drawable/stock_up"
            android:contentDescription="@string/stock_widget" />

        <ImageView
            android:layout_width="wrap_content"
            android:layout_height="match_parent"
            android:layout_gravity="center"
            android:src="@drawable/stock_down"
            android:contentDescription="@string/stock_widget" />

        <TextView
            android:layout_width="0dp"
            android:layout_height="match_parent"
            android:layout_weight="1"
            android:layout_marginStart="32dp"
            android:gravity="center_vertical"
            android:text="@string/your_stocks"
            android:textAppearance="@android:style/TextAppearance.DeviceDefault.Widget.ActionBar.Title"
            android:layout_marginLeft="32dp" />
    </LinearLayout>

    <ListView
        android:id="@+id/widget_listView"
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:background="@color/backGround"></ListView>

</LinearLayout>
list_item.xml
<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="72dp"
    android:gravity="center_vertical"
    android:orientation="horizontal"
    android:paddingLeft="16dp"
    android:paddingRight="16dp"
    >
  <TextView
      android:id="@+id/stock_symbol"
      style="@style/StockSymbolTextStyle"
      android:layout_width="wrap_content"
      android:layout_height="wrap_content"
      android:gravity="start|center_vertical"
      tools:text="List Item"
      />
</LinearLayout>

Next up, having a look at the modified files, we can see that the Widget creation wizard added some stuff into out AndroidManifest.xml and created a new java file.

Upon taking a closer look at the manifest, we can see that the widget’s java class has been registered as a <receiver/>

Next, opening up the NewAppWidget.java, we will see that it extends AppWidgetProvider and some methods are already overridden for you.

Time to edit up this file to reference to the layouts we have just created.

import android.annotation.TargetApi;
import android.app.PendingIntent;
import android.appwidget.AppWidgetManager;
import android.appwidget.AppWidgetProvider;
import android.content.Context;
import android.content.Intent;
import android.os.Build;
import android.support.annotation.NonNull;
import android.widget.RemoteViews;

/**
 * Implementation of App Widget functionality.
 */
public class StockWidgetProvider extends AppWidgetProvider {

    private static void updateAppWidget(Context context, AppWidgetManager appWidgetManager,
                                        int appWidgetId) {
        // Construct the RemoteViews object which defines the view of out widget
        RemoteViews views = new RemoteViews(context.getPackageName(), R.layout.widget_layout);
        // Instruct the widget manager to update the widget
        if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.ICE_CREAM_SANDWICH) {
            setRemoteAdapter(context, views);
        } else {
            setRemoteAdapterV11(context, views);
        }
        /** PendingIntent to launch the MainActivity when the widget was clicked **/
        Intent launchMain = new Intent(context, MainActivity.class);
        PendingIntent pendingMainIntent = PendingIntent.getActivity(context, 0, launchMain, 0);
        views.setOnClickPendingIntent(R.id.widget, pendingMainIntent);
        appWidgetManager.notifyAppWidgetViewDataChanged(appWidgetId,R.id.widget_listView);
        appWidgetManager.updateAppWidget(appWidgetId, views);
    }

    @Override
    public void onUpdate(Context context, AppWidgetManager appWidgetManager, int[] appWidgetIds) {
        // There may be multiple widgets active, so update all of them
        for (int appWidgetId : appWidgetIds) {
            updateAppWidget(context, appWidgetManager, appWidgetId);
        }

        super.onUpdate(context, appWidgetManager, appWidgetIds);
    }

    @Override
    public void onEnabled(Context context) {
        // Enter relevant functionality for when the first widget is created
    }

    @Override
    public void onDisabled(Context context) {
        // Enter relevant functionality for when the last widget is disabled
    }

  /** Set the Adapter for out widget **/

    @TargetApi(Build.VERSION_CODES.ICE_CREAM_SANDWICH)
    private static void setRemoteAdapter(Context context, @NonNull final RemoteViews views) {
        views.setRemoteAdapter(R.id.widget_listView,
                new Intent(context, StockWidgetService.class));
    }

    
    /** Deprecated method, don't create this if you are not planning to support devices below 4.0 **/
    @SuppressWarnings("deprecation")
    private static void setRemoteAdapterV11(Context context, @NonNull final RemoteViews views) {
        views.setRemoteAdapter(0, R.id.widget_listView,
                new Intent(context, StockWidgetService.class));
    }

}

Now, create a WidgetDataProvider which will provide us with data to be displayed inside the widget.

You can use a static data for now (like a prefilled ArrayList, but make sure that this data should be dynamic for making the widget meaningful)

import android.content.Context;
import android.content.Intent;
import android.database.Cursor;
import android.os.Binder;
import android.widget.RemoteViews;
import android.widget.RemoteViewsService;

/**
 * Created by the-dagger on 24/7/16.
 */

public class WidgetDataProvider implements RemoteViewsService.RemoteViewsFactory {

    private Context context;
    private Cursor cursor;
    private Intent intent;

    //For obtaining the activity's context and intent
    public WidgetDataProvider(Context context, Intent intent) {
        this.context = context;
        this.intent = intent;
    }

    private void initCursor(){
        if (cursor != null) {
            cursor.close();
        }
        final long identityToken = Binder.clearCallingIdentity();    
        /**This is done because the widget runs as a separate thread 
        when compared to the current app and hence the app's data won't be accessible to it
        because I'm using a content provided **/
        cursor = context.getContentResolver().query(QuoteProvider.Quotes.CONTENT_URI,
                new String[]{QuoteColumns._ID, QuoteColumns.SYMBOL, QuoteColumns.BIDPRICE,
                        QuoteColumns.PERCENT_CHANGE, QuoteColumns.CHANGE, QuoteColumns.ISUP},
                QuoteColumns.ISCURRENT + " = ?",
                new String[]{"1"},null);
        Binder.restoreCallingIdentity(identityToken);
    }

    @Override
    public void onCreate() {
        initCursor();
        if (cursor != null) {
            cursor.moveToFirst();
        }
    }

    @Override
    public void onDataSetChanged() {
        /** Listen for data changes and initialize the cursor again **/
        initCursor();
    }

    @Override
    public void onDestroy() {
    cursor.close();
    }

    @Override
    public int getCount() {
        return cursor.getCount();
    }

    @Override
    public RemoteViews getViewAt(int i) {
        /** Populate your widget's single list item **/
        RemoteViews remoteViews = new RemoteViews(context.getPackageName(), R.layout.list_item_quote);
        cursor.moveToPosition(i);
        remoteViews.setTextViewText(R.id.stock_symbol,cursor.getString(cursor.getColumnIndex(QuoteColumns.SYMBOL)));
        remoteViews.setTextViewText(R.id.bid_price,cursor.getString(cursor.getColumnIndex(QuoteColumns.BIDPRICE)));
        remoteViews.setTextViewText(R.id.change,cursor.getString(cursor.getColumnIndex(QuoteColumns.CHANGE)));
        if (cursor.getString(cursor.getColumnIndex(QuoteColumns.ISUP)).equals("1")) {
            remoteViews.setInt(R.id.change, "setBackgroundResource", R.drawable.percent_change_pill_green);
        } else {
            remoteViews.setInt(R.id.change, "setBackgroundResource", R.drawable.percent_change_pill_red);
        }
        return remoteViews;
    }

    @Override
    public RemoteViews getLoadingView() {
        return null;
    }

    @Override
    public int getViewTypeCount() {
        return 1;
    }

    @Override
    public long getItemId(int i) {
        return i;
    }

    @Override
    public boolean hasStableIds() {
        return true;
    }
}

Let’s also create a service that invokes the WidgetDataProvider after a fixed interval

import android.content.Intent;
import android.widget.RemoteViewsService;

/**
 * Created by the-dagger on 24/7/16.
 */

public class StockWidgetService extends RemoteViewsService {
    @Override
    public RemoteViewsFactory onGetViewFactory(Intent intent) {
        return new WidgetDataProvider(this,intent);
    }
}

Phew.. almost done with this now.

Finally edit up the widget_info.xml located inside /res/values/xml/ of your project.

Edit it to reference the time after which your widget will be updated, the preview image which should show up in the widget picker and minimum width and height of the widget.

<?xml version="1.0" encoding="utf-8"?>
<appwidget-provider xmlns:android="http://schemas.android.com/apk/res/android"
    android:initialKeyguardLayout="@layout/app_widget"
    android:initialLayout="@layout/app_widget"
    android:minHeight="110dp"
    android:minWidth="170dp"
    android:previewImage="@drawable/example_appwidget_preview"
    android:resizeMode="horizontal|vertical"
    android:updatePeriodMillis="86400000"
    android:widgetCategory="home_screen"></appwidget-provider>

Well, once this is done, go ahead and fire up your app.

You will be able to see the newly created and updated widget in your homescreen.

 widget

Pretty awesome right!
Congratulations on making your first widget.

For now the app only opens a specific activity on clicking it, but you can read up a bit on how to execute a separate task on clicking each item on the list by using a pendingIntent.

Well, that’s it for this week.
Thank you for going through this post and leave your suggestions and queries in the comments.

Cheers.

Unit testing JSON files in assets folder of Android App

So here is the scenario, your android app has a lot of json files in the assets folder that are used to load some data when in first runs.
You are writing some unit tests, and want to make sure the integrity of the data in the assets/*.json are preserved.

You’d assume, that reading JSON files should not involve using the Android Runtime in any way, and we should be able to read JSON files in local JVM as well. But you’re wrong. The JSONObject and JSONArray classes of Android are part of android.jar, and hence

 
JSONObject myJson = new JSONObject(someString);

The above code will not work when running unit tests on local JVM.

Fortunately, our codebase already using Google’s GSoN library to parse JSON, and that works on local JVM too (because GSoN is a core Java library, not specifically an Android library).

Now the second problem that comes is that when running unit tests on local JVM we do not have the getResources() or getAssets() functions.
So how do we retrieve a file from the assets folder ?

So what I found out (after a bit of trial and error and poking around with various dir paths), is that the tests are run from the app folder (app being the Android application module – it is named app by default by Android Studio, though you might have had named it differently)

So in the tests file you can define at the beginning

    public static final String  ASSET_BASE_PATH = "../app/src/main/assets/";

And also create the following helper function

    public String readJsonFile (String filename) throws IOException {
        BufferedReader br = new BufferedReader(new InputStreamReader(new FileInputStream(ASSET_BASE_PATH + filename)));
        StringBuilder sb = new StringBuilder();
        String line = br.readLine();
        while (line != null) {
            sb.append(line);
            line = br.readLine();
        }

        return sb.toString();
    }

Now wherever you need this JSON data you can just do the following

        Gson gson = new GsonBuilder().create();
        events = gson.fromJson(readJsonFile("events.json"),
                Event.EventList.class);
        eventDatesList = gson.fromJson(readJsonFile("eventDates.json"), EventDates.EventDatesList.class);

Push your apk to your GitHub repository from Travis

In this post I’ll guide you on how to directly upload the compiled .apk from travis to your GitHub repository.

Why do we need this?

Well, assume that you need to provide an app to your testers after each commit on the repository, so instead of manually copying and emailing them the app, we can setup travis to upload the file to our repository where the testers can fetch it from.

So, lets get to it!

Step 1 :

Link Travis to your GitHub Account.

Open up https://travis-ci.org.

Click on the green button in the top right corner that says “Sign in with GitHub”

screenshot-area-2016-07-15-205733.png<

Step 2 :

Add your existing repository to Travis

Click the “+” button next to your Travis Dashboard located on the left.

screenshot-area-2016-07-15-210630.png<

Choose the project that you want to setup Travis from the next page

screenshot-area-2016-07-15-210916.png
Toggle the switch for the project that you want to integrate

Click the cog here and add an Environment Variable named GITHUB_API_KEY.
Proceed by adding your Personal Authentication Token there.
Read up here on how to get the Token.

 screenshot-area-2016-07-15-213931.png<

Great, we are pretty much done here.

Let us move to the project repository that we just integrated and create a new file in the root of repository by clicking on the “Create new file” on the repo’s page.

Name it .travis.yml and add the following commands over there

language: android 
jdk:
  - oraclejdk8
android:
  components:
    - tools
    - build-tools-24.0.0
    - android-24
    - extra-android-support
    - extra-google-google_play_services
    - extra-android-m2repository
    - extra-google-m2repository
    - addon-google_apis-google-24
 before_install:
 - chmod +x gradlew
 - export JAVA8_HOME=/usr/lib/jvm/java-8-oracle
 - export JAVA_HOME=$JAVA8_HOME
 after_success:
 - chmod +x ./upload-gh-pages.sh
 - ./upload-apk.sh
 script:
 - ./gradlew build

Next, create a bash file in the root of your repository using the same method and name it upload-apk.sh

  #create a new directory that will contain out generated apk
  mkdir $HOME/buildApk/ 
  #copy generated apk from build folder to the folder just created
  cp -R app/build/outputs/apk/app-debug.apk $HOME/android/
  #go to home and setup git
  cd $HOME
  git config --global user.email "[email protected]"
  git config --global user.name "Your Name" 
  #clone the repository in the buildApk folder
  git clone --quiet --branch=master  https://user-name:[email protected]/user-name/repo-name  master > /dev/null
  #go into directory and copy data we're interested
  cd master  cp -Rf $HOME/android/* .
  #add, commit and push files
  git add -f .
  git remote rm origin
  git remote add origin https://user-name:[email protected]/user-name/repo-name.git
  git add -f .
  git commit -m "Travis build $TRAVIS_BUILD_NUMBER pushed"
  git push -fq origin master > /dev/null
  echo -e "Donen"

Once you have done this, commit and push these files, a Travis build will be initiated in few seconds.
You can see it ongoing in your Dashboard at https://travis-ci.org/.

After the build has completed, you will can see an app-debug.apk in your Repository.

IMPORTANT NOTE :

You might be wondering as to why did I write [skip ci] in the commit message.

Well the reason for that is, Travis starts a new build as soon as it detects a commit made on the master branch of your repository.

So once the apk is uploaded, that will trigger another build in Travis and hence forming an infinite loop.

We can prevent this in 2 ways :

First, simply write [skip ci] somewhere in the commit message and it will cause Travis to ignore the commit.

Or, push the apk to any other branch which is not configured for Travis build.

So well, that’s almost it.

I hope that you found this tutorial helpful, and if you have any doubts regarding this feel free to comment down below, I would love to help you out.

Cheers.

Using compression middleware in Node/Express

If you’re using ExpressJS to host your server, there’s a small performance tweak that you can do (works on universally on any kind of server – HTML templates to JSON REST APIs to image servers) – which is to enable GZIP compression.

The changed required to do it should be very simple. First install the compression npm package

 npm install --save compression

In you app.js (or wherever you start your express server), modify the code to include the compression middleware like this

var express = require('express);
var app = express();

var compression = require('compression');

app.use(compression());

// Rest of your app code
// app.get (endpoint, callback) 
// app.listen(port, callback)

This should easily reduce page loads time to the order of 15-20%.

For example, page load before compression middleware added 72f7abf2-4899-11e6-9cd5-68a7addaf3a6

 

And Page load time after compression is added69e6fba8-4899-11e6-9626-a25c26ea7d2b