Database Listener for User Centric Events

Badgeyay is an open-source utility developed by FOSSASIA to generate badges for conferences and events. The project is separated into two components to ease maintainability. First is the frontend part which is in ember and second part is backend which is in Flask. The choice of database to support backend is PostgreSQL.

Now comes the problem, whenever a user is registered in the database, he should receive  a verification mail, that he is successfully registered on the platform. For this case we have to listen to the database events on User model. This issue has greater extendibility than only sending greeting or verification mail to the user. We can extend this to trigger services that are dependent on user registration, like subscribing the user to some set of services based on the plan he opted while registration and many more.

These type of issues cannot be handled by normal relationship with tables and other entities, there has to be logic in place to support such functionalities. So the challenges for tackling the problem are as follows:

  • Listen to the insert_action in User model
  • Extracting the details necessary for the logic
  • Execute particular logic

Procedure

  1. Attaching insert_action listener to the User model. This function will get triggered whenever an entity is saved in the User model.

<!– HTML generated using hilite.me –>

@db.event.listens_for(User, "after_insert")
def logic(mapper, connection, target): {
......
}
  1. When the function gets triggered, extract the details of the saved user that is necessary for the logic. As currently we are sending greeting mail to the user,we only need the email of the user. Target is the actual saved user passed as argument to the listening function from the library.

<!– HTML generated using hilite.me –>

msg = {}
msg['subject'] = "Welcome to Badgeyay"
msg['receipent'] = target.email
msg['body'] = "It's good to have you onboard with Badgeyay. Welcome to " \
"FOSSASIA Family."
sendMail(msg)
  1. Now the details are passed to sendMail() function for sending mail which uses flask-mail library to send mail to the recipient.
    def sendMail(message):
    if message and message.receipent:
    try:
    msg = Message(
    subject=message.subject,
    sender=app.config['MAIL_USERNAME'], Response(200).generateMessage(
    recipients=[message.receipent],
    body=message.body)
    Mail(app).send(msg)
    except Exception as e:
    return jsonify(
    Response(500).exceptWithMessage(
    str(e),
    'Unable to send the mail'))
    return jsonify(
    Response(200).generateMessage(
    'Mail Sent'))
    else:
    return jsonify(
    Response(403).generateMessage(
    'No data received')) 'No data received'))
    
  2. This will send mail to the user who has been registered to the application.

Similarly we can use separate logics according to the need of the application.

 

The Pull Request for the above functionality is at this Link

Topics Involved

Working on the issue involve following topics:

  • Configuring mail service to allow insecure apps access.
  • Sending mail from the flask-mail to end user
  • Attaching listener to listen for database change
  • Extraction of data from saved object in database sqlalchemy.

Resources

  • Sending Mails Programmatically –  Link
  • Flask Mail Documentation – Link
  • Listening to database events – Link
  • Enabling access to GMAIL to send mails to recipient – Link

Adding multiple email support for users on Open Event Server

The Open Event Server enables organizers to manage events from concerts to conferences and meet-ups. It offers features for events with several tracks and venues. Event managers can create invitation forms for speakers and build schedules in a drag and drop interface. The event information is stored in a database. The system provides API endpoints to fetch the data, and to modify and update it.

The Open Event Server is based on JSON 1.0 Specification and hence build on top of Flask Rest Json API (for building Rest APIs) and Marshmallow (for Schema).

In this blog, we will talk about how to add support of multiple emails for a user in Open Event Server. The focus is on model and schema creation for this support.

Model Creation

For the UserEmail, we’ll make our model as follows

from app.models import db

class UserEmail(db.Model):
“””user email model class”””
__tablename__ = ‘user_emails’
id = db.Column(db.Integer, primary_key=True)
email = db.Column(db.String(120), unique=True, nullable=False)
verified = db.Column(db.Boolean, default=False)
user_id = db.Column(db.Integer, db.ForeignKey(‘users.id’, ondelete=’CASCADE’))
user = db.relationship(“User”, backref=”emails”, foreign_keys=[user_id])

def __init__(self, email=None, user_id=None):
self.email = email
self.user_id = user_id

def __str__(self):
return ‘User:’ + unicode(self.user_id).encode(‘utf-8’) + ‘ email: ‘ + unicode(self.email).encode(‘utf-8’)

def __unicode__(self):
return unicode(self.id)

Now, let’s try to understand the attributes of this model.

  1. id is most important Column required in every model to set it as primary key and to uniquely identify an UserEmail object.
  2. email is that attribute which is required hence should be unique and non-nullable.
  3. Verified attribute is used to check whether a email is verified or not (thus should be boolean)
  4. User_id is the attribute which specifies id of the user whose email is contained in the UserEmail object.
  5. Finally, a relationship with the user of id user_id and these emails (associated with the User.id == user_id) will be stored in the attribute emails in User Model.

Schema Creation

For the model UserEmail, we’ll make our schema UserEmailSchema as follows

from marshmallow_jsonapi import fields
from marshmallow_jsonapi.flask import Schema, Relationshipfrom app.api.helpers.utilities import dasherizeclass UserEmailSchema(Schema):
“””   API Schema for user email Model   “””class Meta:
“””  Meta class for user email API schema  “””
type_ = ‘user-emails’
self_view = ‘v1.user_emails_detail’
self_view_kwargs = {‘id’: ‘<id>’}
inflect = dasherize

id = fields.Str(dump_only=True)
email = fields.Email(allow_none=False)
user_id = fields.Integer(allow_none=False)
user = Relationship(attribute=’user’,
self_view=’v1.user_email’,
self_view_kwargs={‘id’: ‘<id>’},
related_view=’v1.user_detail’,
related_view_kwargs={‘user_id’: ‘<id>’},
schema=’UserSchema’,
type_=’user’
)

  • Marshmallow-jsonapi provides a simple way to produce JSON API-compliant data in any Python web framework.

Now, let’s try to understand the schema UserEmailSchema

  1. id : Same as in model id is used as uniquely identify an UserEmail object.
  2. email : Same as in model email is required thus allow_none is set to False.
  3. User_id : user_id is the id of user whose email is contained in a UserEmailSchema object.
  4. User : It tells whole attributes of the user to which this email belongs to.

So, we saw how to add multiple email support for users on Open Event Server. We just required to create a model and its schema to add this feature. Similarly, to add support for any database model in the project, we need to create Model and Schema with all the attributes as specified in the model too. This Schema creation is done with guidelines of JSONAPI 1.0 Specification using Marshmallow.

Resources

Variable Font Size Badgeyay

Badgeyay is a simple badge generator that aims for promoting an open-source tool for generation of badges in PDF format. The project has options to choose from predefined set of images or upload a background image. User can choose from set of fonts and color of the same. But now Badgeyay also has option to choose custom font-size in generation of badges.

To implement font size feature,  first, the component that is determining the font of the label has to be identified. The label that determines the text on the badge is the <text> label and within it, the label that determines the properties of the text is <tspan>. So mainly we need to alter the properties in the tspan.

The property that determines the font size for the badge is font-size and its default value is set to 31.25 px. If the property in the labels changed, then we can see the corresponding changes in the PDF generated from the svg.

Now the challenges were:

  • To Determine the font value from the frontend.
  • Using the same for the font-config.
  • Changing the built svg accordingly.

Procedure

  1. Firstly frontend component has to be changed to incorporate a slider to give input for the variable font size. So a range input is inserted with range from 15 px to 45 px and default as 30 px. The size_print label gets changed dynamically to show the value selected from the range slider.
<li>
<input type="radio" name="fontsize" id="font-size-picker"> Choose font size
</li>
<section id="font-size-input" style="display:none;">
<label for="inputFile" id="size_print"></label>
<div>
<input type="range" id="font-size" max=45 min=15 step=5 value=30  class="form-control" name="font_size">
</div>
</section>
  1. After adding the component, form script is changed to add toggle behaviour to the button. For adding the toggling behaviour in the component, checkbox is used and the value of the label is updated dynamically as the slider value is changed.
$("#size_print").text($("#font-size").val() + " px");

      $("#font-size-picker").click(function () {

          if ($(this).is(":checked")) {

              $("#font-size-input").css("display", "block");

          } else {

              $("#font-size-input").css("display", "none");

          }

      });

      $("#font-size").on('input', function () {

          $("#size_print").text($(this).val() + " px");

      });
  1. After completing the work on the frontend, it is necessary to modify the backend too. The method for choosing custom font has to be refactored. It now checks whether the custom font is set or font size variable is set, and creates a config file for fonts which after use gets deleted.
font_config = {}
   # custom font is specified
   if custom_font != '':
       font_config['font'] = custom_font
   if font_size != '':
       font_config['font_size'] = font_size
   if custom_font != '' or font_size != '':
       json_str = json.dumps(font_config)
       print(json_str)
       f = open(os.path.join(app.config['UPLOAD_FOLDER'], 'fonts.json'), "w+")
       f.write(json_str)
       f.close()
  1. The generator class is modified as well to accommodate the changes, by adding a new class attribute called font_size. We find the keys in the dict object loaded from the file and assign the same to class attribute.
if 'font_size' in self.DATA.keys():
               self.font_size = self.DATA['font_size']
  1. Make the necessary change in the svg, so that font size change can be represented in the generated PDF. Replace the old font size with the new font size specified.
if self.font_size:
           content = content.replace("font-size:31.25px",
                                     "font-size:" + str(self.font_size) + "px")
  1. After all the changes, badge generated will have a different font size.

The Pull request for the above change is at this Link

Topics Involved

Working on this Issue (Link) involve following topics:

  • SVG Label manipulation
  • Sending data from Ember frontend to Backend.
  • Javascript for the toggle radio button.

References

  • Extracting map information from the SVG (Link)
  • Python Documentation for class (Link)
  • About Github Pages- (Link)
  • Ajax Serialize method to serialize the form contents – (Link)

The Road to Success in Google Summer of Code 2017

It’s the best time when GCI students can get the overview experience of GSoC and all the aspiring participant can get themselves into different projects of FOSSASIA.

I’m a Junior year undergraduate student pursuing B.Tech in Electrical Engineering from Indian Institute of Technology Patna. This summer, I spent coding in Google Summer of Code with FOSSASIA organization. It feels great to be an open-source enthusiast, and Google as a sponsor make it as icing on the cake. People can learn new things here and meet new people.

I came to know about GSoC through my senior colleagues who got selected in GSoC in the year 2016. It was around September 2016 and I was in 2nd year of my college. At that time, last year, result of GSoC was declared.

What is GSoC?

Consider GSoC as a big bowl which has lots of small balls and those small balls are open-source organizations. Google basically acts as a sponsor for the open-source organizations. A timeline is proposed according to the applied organization and then student select their favorite organization and start to contribute to it. Believe me, it’s not only computer science branch specific, anyone can take part in it and there is no minimum CPI requirement. I consider myself to be one of the examples who have an electrical branch with not so good academic performance yet successfully being part of GSoC 2017.

How to select an organization?

This is the most important step and it takes time. I wandered around 100 organizations to find where my interest actually lies. But now, I’ll describe how to sort this and find your organization a little quicker. Take a pen and paper (kindly don’t use notepad of pc) and write down your field of interest in computer science. Number every point in decreasing order of your interest. Then for each respective field write down its basic pre-requisites. Visit GSoC website, go to organization tab and there is a slide for searching working field of the organization. Select only one organization, dig out its website, see the previous project and its application. If nothing fits you, repeat the same with another organization. And if that organization interests you, then look for a project of that organization. First of all, look at that application of the project, and give that application a try and must give a feedback to the organization. Then try to find that what languages, modules, etc that project used to work and how the project works. Don’t worry if nothing goes into your mind. Find out the developers mailing list, their chat channel, their code base area. And ask developers out there for help.

First Love It:

Open-Source, it’s a different world which exists on Earth. All organizations are open-source and all their codes are open and free to view. Find things that interests you the most and start to love the work. If you don’t understand a code, learn things by doing and asking. Most of the times we don’t get favorable responses, in such times we need to carry on and have patience for the best to happen.

My Favourite part:

GSoC has been my dream since the day I came to know about it. It’s only through this that one gets a chance to explore open-source softwares, and organizations get a chance to hire on board developers. This is the great initiative taken by Google which brings hope for the developers to increase the use of open-source. This is one of the ways through which one can look into the codes of the developers and help them out and even also get helped.

GSoC is the platform through which one can implement lots of new things, meet new people, develop new softwares and see the world around in a different way. That’s what happened with me, it’s just at the end of the first phase, my love towards open-source increased exponentially. Now I see every problem in my life as a way to solve it through the open-source. Rather it’s part of arranging an event or designing an invitation, I am encouraged to use open-source tools to help me out. It becomes very easy to distribute data and convey information through open-source, so the people can reach to you much easier.

You always see a thing according to your perspective and it’s always the best but the open-source gives it a view through the perspective of the world and gets the best from them through a compilation of all the sources. One can give ideas, their views, find something that other can’t even see and increase its karma through contribution. And all these things have been made possible through GOOGLE only. I became such that I can donate the rest of my life working for open-source. GSoC is responsible for including the open-source contribution in my daily life. It made me feel really bad if my Github profile page has 0 contributions at the end of the day. Open Source opens door to another world.

Challenging part:

To conclude, I would say that GSoC made me love the challenge. I became such that the things that come easily to me don’t taste good to me at all. Specifically, GSoC’s most challenging part is to get into it that is to get selected. I still can’t believe that I was selected. Now onwards it’s just fun and learning. Each and every day, I encountered several issues, bugs, etc but just before going to bed at night, there were things which collectively made me feel that whether the bug has been solved or not, but I was able to break the upper most covering of that conch shell. And such things increases the motivation and light up the enthusiasm to tackle the problem. Open-Source not only taught me to control different snapshots of software but also of time. I learn to manage different works of day efficiently and it includes the contribution in open-source as part of my daily life.

Advice to students:

The only problem new developers have is to get started. I’ll advise them to close their eyes and dive into it without thinking whether they would be able to complete this task or not. Believe me, you will gradually find that whether the task is completed or not but you are much above the condition than you were at the time of beginning the task.

Just learn by doing the things.

Make mistakes and enlist them as “things that will not work” so one may read it and avoid it.

GSoC Project link: https://summerofcode.withgoogle.com/projects/#5560333780385792
Final Code Submission: https://gist.github.com/meets2tarun/270f151d539298831ce542be5f733c82

Speeding up the Travis Build to Decrease the Building Time

Meilix is the repository which uses build script to generate community version of lubuntu as LXQT Desktop. It usually takes around 25-26 to build and deploy the ISO as a Github Release on master branch.
Observing the build log we can see that there are most of the packages like debootstrap, squashfs-tool, etc which are being fetch and setup at the time of building which results in increasing build time.

The issue is to decrease the build time supplying the packages from Travis so that when we run build.sh we won’t be required to download them again and thus few minutes will get reduced.
We included list of packages to be pre-downloaded in .travis.yml

include:
  - os: linux
    addons:
      apt:
        sources:
          - ubuntu-toolchain-r-test
        packages:
          - debootstrap
          - genisoimage
          - p7zip-full
          - squashfs-tools
          - ubuntu-dev-tools
          - dpkg-dev
          - debhelper
          - fakeroot
          - devscripts

These are some important packages included in the build.sh  as devtools=”debootstrap genisoimage p7zip-full squashfs-tools ubuntu-dev-tools” which are always fetched, so we included it into .travis.yml and downloaded it before entering into the chroot environment. By specifying those packages in the .travis.yml, Travis provides those packages beforehand into the docker container so it will run our script. Since the scripts also include package then when the script runs apt-get it won’t download those packages again. They are specified outside the chroot environment because they are expected to be at the system the build.sh script is run to get the iso. By this way, we get a sharp decrease in build time as the internet in the Travis CI container is not so fast so the package download time can be avoided. Now the build is taking around 15-16 minutes to build and deploy.

One thing to note that we didn’t remove those packages off build.sh so that build.sh works outside Travis CI as well.

References:
Pull #176 by @abishekvashok
Speeding up Travis Build by Travis CI
Faster Build by atchai.com

Implementing “Change Password” API in Open Event Frontend

In Open Event Frontend, users can change the password for their account in the ‘Settings’ section. Changing one’s password will require the previous password of the same account which ensures the security. To implement change in password API, we created a REST endpoint here since the password cannot be included in the user model and thereby exposed to the client.

There is also a check on the server side of the old password. Thus, if the old password entered matches the one on the server, the post is successful and the server saves the new password. We achieve this as follows:

We have a change password form located at ‘settings/change-password’, which contains three input fields for old password, new password and confirm new password.

On submitting the form, we pass the action from the component to the controller.

Components consist of two parts: a template written in Handlebars, and a source file written in JavaScript that defines the component’s behavior.

Controllers behave like a specialized type of Component that is rendered by the router when entering a Route.

We could have handled this action in component itself. But, Ember JS’ main principle is DDAU i.e data down actions up. That is the main reason why we handle the action in out controller.

submit() {
      this.onValid(() => {
      this.sendAction('changePassword', this.getProperties('passwordCurrent', 'passwordNew'));
      });
}

Thus, we handle the action in our controller as follows:

 changePassword(passwordData) {
      this.set('isLoading', true);
      let payload = {
        'data': {
          'old-password' : passwordData.passwordCurrent,
          'new-password' : passwordData.passwordNew
        }
      };
      this.get('loader')
        .post('/auth/change-password', payload)
        .then(() => {
          this.get('notify').success(this.l10n.t('Password updated successfully'));
        })
        .catch(error => {
          if (error.error) {
            this.get('notify').error(this.l10n.t(error.error));
          } else {
            this.get('notify').error(this.l10n.t('Unexpected error. Password did not change.'));
          }
        })
        .finally(() => {
          this.set('isLoading', false);
          this.setProperties({ 'passwordCurrent': '', 'passwordNew': '', 'passwordRepeat': '' });
        });
    }

Here, we are getting the old password and the new password passed from the form and making a POST to the endpoint:

v1/auth/change-password

If the old password check goes successful on the server side, the server returns a successful response:

{ 
 "email": "[email protected]",
 "id": "1",
 "name": "example",
 "password-changed": true
}

Thus, the user can change password in Open Event Frontend.
Resources: Docs on loader service in Ember JS

Showing “Get started” button in SUSI Viber bot

When we start a chat with SUSI.AI on Viber i.e. SUSI Viberbot, there should be an option on how to get started with the bot. The response to it are some options like “Visit repository”, “How to contribute” which direct the user to check how SUSI.AI bot is made and prompts him/her to contribute to it. Along with that an option of “start chatting” can be shown to add up some sample queries for the user to try.

To accomplish the task at hand, we will accomplish these sub tasks:

  1. To show the “Get started” button.
  2. To show the reply to “Get started” query.
  3. To respond to the queries, nested in the response of “Get started”

Showing “Get started”:

The Viber developers platform notifies us when a user starts a conversation with our bot. To be exact, a conversation_started event is sent to our webhook and can be handled accordingly. The Viberbot shows a welcome message to the user along with a Get started button to help him/her start.

To send just the welcome message:

if (req.body.event === 'conversation_started') {
       // Welcome Message
       var options = {
           method: 'POST',
           url: 'https://chatapi.viber.com/pa/send_message',
           headers: headerBody,
           body: {
               // some required body properties here
               text: 'Welcome to SUSI.AI!, ' + req.body.user.name + '.',
               // code for showing the get started button here.
        }
           json: true
       };
 
       request(options, function(error, res, body) {
           // handle error
       });
   }

The next step is to show the “Get started” button. To show that we use a keyboard tool, provided by Viber developers platform. So after the “text” key we have the “keyboard” key and a value for it:

keyboard: {
             "Type": "keyboard",
             "DefaultHeight": true,
             "Buttons": [{
                 "ActionType": "reply",
                 "ActionBody": "Get started",
             }]
         }

The action type as shown in the code, can be “reply” or “open-url”. The “reply” action type, triggers an automatic query sent back with “Get started” (i.e. the value of “ActionBody” key), when that button gets clicked.

Hence, this code helps us tackle our first sub task:

Reply to “Get started”:

We target to make each SUSI.AI bot generic. The SUSI FBbot and SUSI Tweetbot shows some options like “Visit repository”, “Start chatting” and “How to contribute?” for the “Get started” query. We render the same answer structure in Viberbot.

The “rich_media” type helps us send buttons in our reply message. As we ought to use three buttons in our message, the button rows are three in the body object:

if(message === "Get started"){
                   var options = {
                       method: 'POST',
                       url: 'https://chatapi.viber.com/pa/send_message',
                       headers: headerBody,
                       body: {
                           // some body object properties here
                           type: 'rich_media',
                           rich_media: {
                               Type: "rich_media",
                               ButtonsGroupColumns: 6,
                               ButtonsGroupRows: 3,
                               BgColor: "#FFFFFF",
                               Buttons: buttons
                           }
                       },
                       json: true
                   };
 
                   request(options, function(error, res, body) {
                       if (error) throw new Error(error);
                       console.log(body);
                   });

As said before, 2 type of Action types are available – “open-url” and “reply”. “Visit repository” button has an “open-url” action type and “How to contribute?” or “start chatting” has a “reply” action type.

Example of “Visit repository” button:

var buttons = [{
                Columns: 6,
                Rows: 1,
                Text: "Visit repository",
                "ActionType": "open-url",
                "ActionBody": "https://www.github.com/fossasia/susi_server",
                // some text styling properties here
             }];

To respond to the “reply” action type queries:

When the “reply” action type button gets clicked, it triggers an automatic query sent back to the bot with the value same as that of the “ActionBody” key. So we just need to apply a check if the message string recieved is “Start chatting” or “How to contribute?”

For the response to “Start chatting”, we plan to show sample queries for the user to try. This can be shown by using buttons with the action type as “reply”.

Code snippet to show a button with the text as “What is FOSSASIA?”:

var buttons = [{
                        Columns: 6,
                        Rows: 1,
                        Text: "What is FOSSASIA? ",
                        "ActionType": "reply",
                        "ActionBody": "What is FOSSASIA?",
                        // text styling here
                    }];

For the response to “How to contribute”, we show some messages to help the user contribute to SUSI.AI. These messages also just need buttons with it, to be able to apply the necessary action.

We respond with 2 messages to the user, both the messages have a button.

For example, a button to visit the SUSI.AI Gitter channel:

var buttons = [{
                    Columns: 6,
                    Rows: 1,
                       Text: "<font color=#323232><b>Chat on Gitter</b></font>",
                      ActionType: "open-url",
                      ActionBody: "https://www.gitter.im/fossasia/susi_server",
                      // text styling here
            }];

This way we have successfully added the “Get started” option to our Viberbot and handled all the subsequent steps.

Resources:

  1. Viber video managing chat extensions by Ingrid Lunden from Tech crunch.
  2. Develop a chat bot with node js by Slobodan Stojanović from smashing magazine.

Autolinker Component in Loklak Search

In Loklak Search the post items contain links, which are either internal or external. These links include the hashtags, mentions, and URLs. From the backend server we just received the message in the plain text format, and thus there is need to parse the plain text and render it as clickable links. These clickable links can be either internal or external. Thus we need an auto-linker component, which takes the text and render it as links.

The API of the Component

The component takes as the input the plain text, then four arrays of strings. Each containing the text to be linked. These are hashtags, mentions, links and the unshorten attribute which is used to unshorten the shortened URLs in the post. These attributes are used by the component to render the text in the appropriate format.

export class FeedLinkerComponent implements OnInit {
@Input() text: string;
@Input() hashtags: string[] = new Array<string>();
@Input() mentions: string[] = new Array<string>();
@Input() links: string[] = new Array<string>();
@Input() unshorten: Object = {};
}

The Logic of the Component

The basic logic of the component works as the following, we divide the text into chunks known as shards, we have three basic data structures for the component to work

  • The ShardType which is the type of the chunk it specifies whether it is plain, hashtags, mentions, and links.
  • The Shard which is the simple object containing the text to show, its type and the link it refers to

The StringIndexdChunks, they are utilized to index the chunks in the order in which they appear in the text.

const enum ShardType {
plain, // 0
link, // 1
hashtag, // 2
mention // 3
}

class Shard {
constructor (
public type: ShardType = ShardType.plain,
public text: String = '',
public linkTo: any = null,
public queryParams: any = null
) { }
}

interface StringIndexedChunks {
index: number;
str: string;
type: ShardType;
}

First we have a private method of the component which searches for all the elements (strings) in the text. Here we have an array which maintains the index of those chunks in the text.

private generateShards() {
const indexedChunks: StringIndexedChunks[] = [];

this.hashtags.forEach(hashtag => {
const indices = getIndicesOf(this.text, `#${hashtag}`, false);
indices.forEach(idx => {
indexedChunks.push({index: idx, str: `#${hashtag}`, type: ShardType.hashtag});
});
});

this.mentions.forEach(mention => {
const indices = getIndicesOf(this.text, `@${mention}`, false);
indices.forEach(idx => {
indexedChunks.push({index: idx, str: `@${mention}`, type: ShardType.mention});
});
});
}

Then we sort the chunks according to their indexes in the text. This gives us sorted array which consists of all the chunks sorted according to the indexes as they appear in the text.

indexedChunks.sort((a, b) => { return (a.index > b.index) ? 1 : (a.index < b.index) ? -1 : 0; });

The next part of the logic is to generate the shard array, an array which contains each chunk, once. To do this we iterate over the Sorted Indexed array created in the previous step and use it split the text into chunks. We iterate over the text and take substrings using the indexes of each element.

let startIndex = 0;
const endIndex = this.text.length;

indexedChunks.forEach(element => {
if (startIndex !== element.index) {
const shard = new Shard(ShardType.plain, this.text.substring(startIndex, element.index));
this.shardArray.push(shard);
startIndex = element.index;
}
if (startIndex === element.index) {
const str = this.text.substring(startIndex, element.index + element.str.length);
const shard = new Shard(element.type, str);
switch (element.type) {
case ShardType.link: {
if (this.unshorten[element.str]) {
shard.linkTo = str;
shard.text = this.unshorten[element.str];
}
else {
shard.linkTo = str;
}
break;
}

case ShardType.hashtag: {
shard.linkTo = ['/search'];
shard.queryParams = { query : str };
break;
}

case ShardType.mention: {
shard.linkTo = ['/search'];
shard.queryParams = { query : `from:${str.substring(1)}` };
break;
}
}
this.shardArray.push(shard);
startIndex += element.str.length;
}
});

if (startIndex !== endIndex) {
const shard = new Shard(ShardType.plain, this.text.substring(startIndex));
this.shardArray.push(shard);
}

After this we have generated the chunks of the text, now the only task is to write the view of the component which uses this Shard Array to render the linked elements.

<div class="textWrapper">
<span *ngFor="let shard of shardArray">
<span *ngIf="shard.type === 0"> <!-- Plain -->
{{shard.text}}
</span>
<span *ngIf="shard.type === 1"> <!-- URL Links -->
<a>{{shard.text}}</a>
</span>
<span *ngIf="shard.type === 2"> <!-- Hashtag -->
<a [routerLink]="shard.linkTo" [queryParams]="shard.queryParams">{{shard.text}}</a>
</span>
<span *ngIf="shard.type === 3"> <!-- Mention -->
<a [routerLink]="shard.linkTo" [queryParams]="shard.queryParams">{{shard.text}}</a>
</span>
</span>
</div>
  • This renders the chunks and handles the links of both internal and external type.
  • It also also makes sure that the links get unshortened properly using the unshorten API property.
  • Uses routerLink, angular property to link in application URLs, for asynchronous reloading while clicking links.

Resources and Links

This component is inspired from the two main open source libraries.

Earlier these libraries were used in the project, but as the need of unshortening and asynchronous linking appeared in the application, a custom implementation was needed to be implemented.

Restoring State after Orientation Change in Loklak Wok Android

During orientation change i.e. from portrait to landscape mode in Android, the current activity restarts again. As the activity restarts again, all the defined variables loose their previous value, for example the scroll position of a RecyclerView, or the data in the rows of RecyclerView etc. Just imagine a user searched some tweets in Loklak Wok Android, and as the user’s phone is in “Auto rotation” mode, the orientation changes from portrait to landscape. As a result of this, the user loses the search result and has to do the search again. This leads to a bad UX.

Saving state in onSavedInstanceState

The state of the app can be saved by inserting values in a Bundle object in onSavedInstanceState callback. Inserting values is same as adding elements to a Map in Java. Methods like putDouble, putFloat, putChar etc. are used where the first parameter is a key and the second parameter is the value we want to insert.

@Override
public void onSaveInstanceState(Bundle outState) {
   if (mLatitude != null && mLongitude != null) {
       outState.putDouble(PARCELABLE_LATITUDE, mLatitude);
       outState.putDouble(PARCELABLE_LONGITUDE, mLongitude);
   }
...
}

 

The values can be retrieved back when onCreate or onCreateView of the Activity or Fragment is called. Bundle object in the callback parameter is checked, whether it is null or not, if not the values are retrieved back using the keys provided at the time of inserting. The latitude and longitude of a location in TweetPostingFragment are retrieved in the same fashion

public void onViewCreated(View view, @Nullable Bundle savedInstanceState) {
   ...
   if (savedInstanceState != null) { // checking if bundle is null
       // extracting from bundle
       mLatitude = savedInstanceState.getDouble(PARCELABLE_LATITUDE);
       mLongitude = savedInstanceState.getDouble(PARCELABLE_LONGITUDE);
       // use extracted value
   }
}

Restoring Custom Objects, using Parcelable

But what if we want to restore custom object(s). A simple option can be serializing the objects using the native Java Serialization or libraries like Gson. The problem in these cases is performance, they are quite slow. Parcelable can be used, which leads the pack in performance and moreover it is provided by Android SDK, on top of that, it is simple to use.

The objects of class which needs to be restored implements Parcelable interface and the class must provide a static final object called CREATOR which implements Parcelable.Creator interface.

writeToParcel and describeContents method need to be override to implement Parcelable interface. In writeToParcel method the member variables are put inside the parcel, in our case describeContents method is not used, so, simply 0 is returned. Status class which stores the data of a searched tweet implements parcelable.

@Override
public int describeContents() {
   return 0;
}

@Override
public void writeToParcel(Parcel dest, int flags) {
   dest.writeString(mText);
   dest.writeInt(mRetweetCount);
   dest.writeInt(mFavouritesCount);
   dest.writeStringList(mImages);
   dest.writeParcelable(mUser, flags);
}

 

NOTE: The order in which variables are pushed into Parcel needs to be maintained while variables are extracted from the parcel to recreate the object. This is the reason why no “key” is required to push data into a parcel as we do in bundle.

The CREATOR object implements the creation of object from a Parcel. The CREATOR object overrides two methods createFromParcel and newArray. createFromParcel is the method in which we implement the way an object is created from a parcel.

public static final Parcelable.Creator<Status> CREATOR = new Creator<Status>() {
   @Override
   public Status createFromParcel(Parcel source) {
       return new Status(source); // a private constructor to create object from parcel
   }

   @Override
   public Status[] newArray(int size) {
       return new Status[size];
   }
};

 

The private constructor, note that the order in which variables were pushed is maintained while retrieving the values.

private Status(Parcel source) {
   mText = source.readString();
   mRetweetCount = source.readInt();
   mFavouritesCount = source.readInt();
   mImages = source.createStringArrayList();
   mUser = source.readParcelable(User.class.getClassLoader());
}

 

The status objects are restored the same way, latitude and longitude were restored. putParcelableArrayList in onSaveInstaceState and getParcelableArrayList in onCreateView methods are used to push into Bundle object and retrieve from Bundle object respectively.

@Override
public void onSaveInstanceState(Bundle outState) {
   super.onSaveInstanceState(outState);
   ArrayList<Status> searchedTweets = mSearchCategoryAdapter.getStatuses();
   outState.putParcelableArrayList(PARCELABLE_SEARCHED_TWEETS, searchedTweets);
   ...
}


// retrieval of the pushed values in bundle
@Override
public View onCreateView(LayoutInflater inflater, ViewGroup container,
                            Bundle savedInstanceState) {
   ...
   if (savedInstanceState != null) {
       ...
       List<Status> searchedTweets =
               savedInstanceState.getParcelableArrayList(PARCELABLE_SEARCHED_TWEETS);
       mSearchCategoryAdapter.setStatuses(searchedTweets);
   }
   ...
   return view;
}

Resources:

Fixing Infinite Scroll Feature for Susper using Angular

In Susper, we faced a unique problem. Every time the image tab was opened, and the user scrolled through the images, all the other tabs in the search engine, such as All, Videos etc, would stop working. They would continue to display image results as shown:

Since this problem occurred only when the infinite scroll action was called in the image tab, I diagnosed that the problem probably was in the url parameters being set.

The url parameters were set in the onScroll() function as shown:

onScroll () {
let urldata = Object.assign({}, this.searchdata);
this.getPresentPage(1);
this.resultDisplay = ‘images’;
urldata.start = (this.startindex) + urldata.rows;
urldata.fq = ‘url_file_ext_s:(png+OR+jpeg+OR+jpg+OR+gif)’;
urldata.resultDisplay = this.resultDisplay;
urldata.append = true;
urldata.nopagechange = true;
this.store.dispatch(new queryactions.QueryServerAction(urldata));
};

The parameters append and nopagechange were to ensure that the images are displayed in the same page, one after the other.
To solve this bug I first displayed the query call each time a tab is clicked on the web console.
Here I noticed that for the tab videos, nopagechange and append attributes still persisted, and had not been reset. The start offset had not been set to 0 either.
So adding these few lines before making a query call from any tab, would solve the problem.

urldata.start = 0;
urldata.nopagechange = false;
urldata.append = false;

Now the object is displayed as follows:

Now videos are displayed in the videos tab, text in the text tab and so on.
Please refer to results.component.ts for the entire code.

References:

  1. On how to dispatch queries to the store: https://gist.github.com/btroncone/a6e4347326749f938510
  2. Tutorial on the ngrx suite:http://bodiddlie.github.io/ng-2-toh-with-ngrx-suite/