Author: Bhavesh Anand

This author has written 18 articles
Read more about the article Adding System Image for Event Categories

Adding System Image for Event Categories

The Open Event Server is using the JSON 1.0 Specification and build on top of Flask Rest Json API (for building Rest APIs) and Marshmallow (for Schema). In this blog, we will talk about how to add feature of System Image for Event Categories on Open Event Server. The focus is on Model updation, Schema updation and migrating the Database.

Model Updation

For adding System Image, we’ll update our Model EventTopic.

In this feature, we are providing rights to the Admin to add a system image for each Event Category so that if no image is given by a organizer of event on event creation then it will use the system image of that Event Category as event image by default.

Here we are adding a Column named system_image_url which is of type String. This value cannot be nullable and having a default value.

Migrating the Database

For the migrating the Database we will use simple commands.

This command runs migrations. If it cause problems naming Multiple Migration Head, then you need to run

This problem is caused when two developers push a migration file without merging two heads to achieve one head.

The above command will give us ids of two migration heads.

This command is merging two migration heads.

This command is upgrading the migrations.

Finally, we migrate the Database using above command.

Schema Updation

For the system image, we’ll update the Schema EventTopicSchema as follows

In this feature, to provide system image for each Event Category we’ll add a field named system_image_url in the Schema.

Here we are adding a field named system_image_url which is of marshmallow field type URL. This value cannot be none.

Validating the Event Image and using System Image by default

In this step, we’ll check if a event image is provided by organizer. If that is not provided then we’ll use system image of Event Category as Event Image.

Here, we will first take the event topic of event as added by the organizer. Then we will fetch the the database row in Event Topic model which has id == event_topic_id . Then we will return the system image url of that event topic to the event image.

So we saw how we could provide a default image for any event.

Resources

Continue ReadingAdding System Image for Event Categories
Read more about the article Adding Event Roles concerning a User on Open Event Server

Adding Event Roles concerning a User on Open Event Server

The Open Event Server enables organizers to manage events from concerts to conferences and meetups. It offers features for events with several tracks and venues. Event managers can create invitation forms for speakers and build schedules in a drag and drop interface. The event information is stored in a database. The system provides API endpoints to fetch the data, and to modify and update it. The Open Event Server is based on JSON 1.0 Specification and hence build on top of Flask Rest Json API (for building Rest APIs) and Marshmallow (for Schema).

In this blog, we will talk about how to add different events role concerning a user on Open Event Server. The focus is on its model and Schema updation.

Model Updation

For the User Table, we’ll update our User Model as follows:

Now, let’s try to understand these hybrid properties.

In this feature, we are providing Admin the rights to see whether a user is acting as a organizer, co-organizer, track_organizer, moderator, attendee and registrar of any of the event or not. Here, _is_role method is used to check whether an user plays a event role like organizer, co-organizer, track_organizer, moderator, attendee and registrar or not. This is done by querying the record from UserEventsRole model. If the record is present then the returned value is True otherwise False.

Schema Updation

For the User Model, we’ll update our Schema as follows

Now, let’s try to understand this Schema.

Since all the properties will return either True or false so these all properties are set to Boolean in Schema. Here dump_only means, we will return this property in the Schema.

So, we saw how User Model and Schema is updated to show events role concerning a user on Open Event Server.

Resources

Continue ReadingAdding Event Roles concerning a User on Open Event Server
Read more about the article Adding Statistics of Event-Type on Open Event Server

Adding Statistics of Event-Type on Open Event Server

The Open Event Server enables organizers to manage events from concerts to conferences and meet-ups. It offers features for events with several tracks and venues. In this blog, we will talk about how to add statistics of event-type on Open Event Server. The focus is on to get number of events of a specific event type.

Number of events of a specific event type

Now, let’s try to understand this API. Here, we are using flask Blueprints to add this API to the API index.

  1. First of all, we are using the decorator of event_statistics which will append this API route with that of mentioned in the Blueprint event_statistics.
  2. We will just allow logged in user to access this API using JWT (JSON Web Token)
  3. To return the response having all the event types alongwith number of events of it, requires a lot of queries if tried to fulfilled by SQLALchemy ORM. So instead of using ORM we will query using SQL command so that we query the number of all the events of different event types in just one query, which will eventually reduces the time of server to return the response.
  4. In function event_types_count we are using db.engine.execute to run the SQL command of getting the statistics of events respective to event types.
  5. The response will include id of event_type, name of event_type and count of events of corresponding event_type.
  6. Finally, we jsonify the list having objects of statistics of events respective to event types.

Similarly, event topics statistics can be implemented to return the number of events of all the event topics.

Resources

Continue ReadingAdding Statistics of Event-Type on Open Event Server
Read more about the article Adding Modules API on Open Event Server

Adding Modules API on Open Event Server

The Open Event Server enables organizers to manage events from concerts to conferences and meet-ups. It offers features for events with several tracks and venues. Event managers can create invitation forms for speakers and build schedules in a drag and drop interface. The event information is stored in a database. The system provides API endpoints to fetch the data, and to modify and update it.

The Open Event Server is based on JSON 1.0 Specification and hence build on top of Flask Rest Json API (for building Rest APIs) and Marshmallow (for Schema).

In this blog, we will talk about how to add API for accessing the Modules on Open Event Server. The focus is on Schema creation and it’s API creation.

Schema Creation

For the ModuleSchema, we’ll make our Schema as follows

Now, let’s try to understand this Schema.

In this feature, we are providing Admin the rights to set whether Admin wants to include tickets, payment and donation in the open event application.

  1. First of all, we will provide three fields in this Schema, which are ticket_include, payment_include and donation_include.
  2. The very first attribute ticket_include should be Boolean as we want Admin to update it whether he wants to include ticketing system in the application from default one which is False.
  3. Next attribute payment_include should be Boolean as we want Admin to update it whether he wants to include payment system in the application from default one which is False.
  4. Next attribute donation_include should be Boolean as we want Admin to update it whether he wants to include donation system in the application from default one which is False.

API Creation

For the ModuleDetail, we’ll make our API as follows

Now, let’s try to understand this API.

In this API, we are providing Admin the rights to set whether Admin wants to include tickets, payment and donation in the open event application.

  1. First of all, there is the need to know that this API has two method GET and PATCH.
  2. Decorators shows us that only Admin has permissions to access PATCH method for this API i.e. only Admins can modify the modules .
  3. before_get method shows us that this API will give first record of Modules model irrespective of the id requested by user.
  4. Schema used here is default one of Modules
  5. Hence, GET Request is accessible to all the users.

So, we saw how Module Schema and API is created to allow users to get it’s values and Admin users to modify it’s values.

Resources

Continue ReadingAdding Modules API on Open Event Server
Read more about the article Adding multiple email support for users on Open Event Server

Adding multiple email support for users on Open Event Server

The Open Event Server enables organizers to manage events from concerts to conferences and meet-ups. It offers features for events with several tracks and venues. Event managers can create invitation forms for speakers and build schedules in a drag and drop interface. The event information is stored in a database. The system provides API endpoints to fetch the data, and to modify and update it.

The Open Event Server is based on JSON 1.0 Specification and hence build on top of Flask Rest Json API (for building Rest APIs) and Marshmallow (for Schema).

In this blog, we will talk about how to add support of multiple emails for a user in Open Event Server. The focus is on model and schema creation for this support.

Model Creation

For the UserEmail, we’ll make our model as follows

from app.models import db

class UserEmail(db.Model):
“””user email model class”””
__tablename__ = ‘user_emails’
id = db.Column(db.Integer, primary_key=True)
email = db.Column(db.String(120), unique=True, nullable=False)
verified = db.Column(db.Boolean, default=False)
user_id = db.Column(db.Integer, db.ForeignKey(‘users.id’, ondelete=’CASCADE’))
user = db.relationship(“User”, backref=”emails”, foreign_keys=[user_id])

def __init__(self, email=None, user_id=None):
self.email = email
self.user_id = user_id

def __str__(self):
return ‘User:’ + unicode(self.user_id).encode(‘utf-8’) + ‘ email: ‘ + unicode(self.email).encode(‘utf-8’)

def __unicode__(self):
return unicode(self.id)

Now, let’s try to understand the attributes of this model.

  1. id is most important Column required in every model to set it as primary key and to uniquely identify an UserEmail object.
  2. email is that attribute which is required hence should be unique and non-nullable.
  3. Verified attribute is used to check whether a email is verified or not (thus should be boolean)
  4. User_id is the attribute which specifies id of the user whose email is contained in the UserEmail object.
  5. Finally, a relationship with the user of id user_id and these emails (associated with the User.id == user_id) will be stored in the attribute emails in User Model.

Schema Creation

For the model UserEmail, we’ll make our schema UserEmailSchema as follows

from marshmallow_jsonapi import fields
from marshmallow_jsonapi.flask import Schema, Relationshipfrom app.api.helpers.utilities import dasherizeclass UserEmailSchema(Schema):
“””   API Schema for user email Model   “””class Meta:
“””  Meta class for user email API schema  “””
type_ = ‘user-emails’
self_view = ‘v1.user_emails_detail’
self_view_kwargs = {‘id’: ‘<id>’}
inflect = dasherize

id = fields.Str(dump_only=True)
email = fields.Email(allow_none=False)
user_id = fields.Integer(allow_none=False)
user = Relationship(attribute=’user’,
self_view=’v1.user_email’,
self_view_kwargs={‘id’: ‘<id>’},
related_view=’v1.user_detail’,
related_view_kwargs={‘user_id’: ‘<id>’},
schema=’UserSchema’,
type_=’user’
)
  • Marshmallow-jsonapi provides a simple way to produce JSON API-compliant data in any Python web framework.

Now, let’s try to understand the schema UserEmailSchema

  1. id : Same as in model id is used as uniquely identify an UserEmail object.
  2. email : Same as in model email is required thus allow_none is set to False.
  3. User_id : user_id is the id of user whose email is contained in a UserEmailSchema object.
  4. User : It tells whole attributes of the user to which this email belongs to.

So, we saw how to add multiple email support for users on Open Event Server. We just required to create a model and its schema to add this feature. Similarly, to add support for any database model in the project, we need to create Model and Schema with all the attributes as specified in the model too. This Schema creation is done with guidelines of JSONAPI 1.0 Specification using Marshmallow.

Resources

Continue ReadingAdding multiple email support for users on Open Event Server
Read more about the article Installing Susper Search Engine and Deploying it to Heroku

Installing Susper Search Engine and Deploying it to Heroku

Susper is a decentralized Search Engine that uses the peer to peer system yacy and Apache Solr to crawl and index search results.

Search results are displayed using the Solr server which is embedded into YaCy. All search results must be provided by a YaCy search server which includes a Solr server with a specialized JSON result writer. When a search request is made in one of the search templates, a HTTP request is made to YaCy. The response is JSON because that can much better be parsed than XML in JavaScript.

In this blog, we will talk about how to install Susper search engine locally and deploying it to Heroku (A cloud application platform).

How to clone the repository

Sign up / Login to GitHub and head over to the Susper repository. Then follow these steps.

  1. Go ahead and fork the repository
https://github.com/fossasia/susper.com

2.   Get the clone of the forked version on your local machine using

git clone https://github.com/<username>/susper.com.git

3. Add upstream to synchronize repository using

git remote add upstream https://github.com/fossasia/susper.com.git

Getting Started

The Susper search application basically consists of the following :

  1. First, we will need to install angular-cli by using the following command:
npm install -g @angular/cli@latest

2. After installing angular-cli we need to install our required node modules, so we will do that by using the following command:

npm install

3. Deploy locally by running this

ng serve

Go to localhost:4200 where the application will be running locally.

How to Deploy Susper Search Engine to Heroku :

  1. We need to install Heroku on our machine. Type the following in your Linux terminal:
wget -O- https://toolbelt.heroku.com/install-ubuntu.sh | sh

This installs the Heroku Toolbelt on your machine to access Heroku from the command line.

  1. Create a Procfile inside root directory and write
web: ng serve
  1. Next, we need to login to our Heroku server (assuming that you have already created an account).

Type the following in the terminal:

heroku login

Enter your credentials and login.

  1. Once logged in we need to create a space on the Heroku server for our application. This is done with the following command
heroku create
  1. Add nodejs buildpack to the app
heroku buildpacks:add –index 1 heroku/nodejs
  1. Then we deploy the code to Heroku.
git push heroku master
git push heroku yourbranch:master # If you are in a different branch other than master

Resources

Continue ReadingInstalling Susper Search Engine and Deploying it to Heroku
Read more about the article Installing the Loklak Search and Deploying it to Surge

Installing the Loklak Search and Deploying it to Surge

The Loklak search creates a website using the Loklak server as a data source. The goal is to get a search site, that offers timeline search as well as custom media search, account and geolocation search.

In order to run the service, you can use the API of http://api.loklak.org or install your own Loklak server data storage engine. Loklak_server is a server application which collects messages from various social media tweet sources, including Twitter. The server contains a search index and a peer-to-peer index sharing interface. All messages are stored in an elasticsearch index.

The site of this repo is deployed on the GitHub gh-pages branch and automatically deployed here: http://loklak.org

In this blog, we will talk about how to install Loklak_Search locally and deploying it to Surge (Static web publishing for Front-End Developers).

How to clone the repository

Sign up / Login to GitHub and head over to the Loklak_Search repository. Then follow these steps.

  1. Go ahead and fork the repository
https://github.com/fossasia/loklak_search
  1.   Get the clone of the forked version on your local machine using
git clone https://github.com/<username>/loklak_search.git
  1.   Add upstream to synchronize repository using
git remote add upstream https://github.com/fossasia/loklak_search.git

Getting Started

The Loklak search application basically consists of the following :

  1. First, we will need to install angular-cli by using the following command:
npm install -g @angular/cli@latest

2. After installing angular-cli we need to install our required node modules, so we will do that by using the following command:

npm install

3. Deploy locally by running this

ng serve

Go to localhost:4200 where the application will be running locally.

How to Deploy Loklak Search on Surge :

Surge is the technology which publishes or generates the static web-page demo link, which makes it easier for the developer to deploy their web-app. There are a lot of benefits of using surge over generating demo link using GitHub pages.

  1. We need to install surge on our machine. Type the following in your Linux terminal:
npm install –global surge

This installs the Surge on your machine to access Surge from the command line.

  1. In your project directory just run
surge
  1. After this, it will ask you three parameters, namely
Email
Password
Domain

After specifying all these three parameters, the deployment link with the respective domain is generated.

Auto deployment of Pull Requests using Surge :

To implement the feature of auto-deployment of pull request using surge, one can follow up these steps:

  • Create a pr_deploy.sh file
  • The pr_deploy.sh file will be executed only after success of Travis CI i.e. when Travis CI passes by using command bash pr_deploy.sh
#!/usr/bin/env bash
if [ “$TRAVIS_PULL_REQUEST” == “false” ]; then
echo “Not a PR. Skipping surge deployment.”
exit 0
fi
npm i -g surge
export SURGE_LOGIN=test@example.co.in
# Token of a dummy account
export SURGE_TOKEN=d1c28a7a75967cc2b4c852cca0d12206
export DEPLOY_DOMAIN=https://pr-${TRAVIS_PULL_REQUEST}-fossasia-LoklakSearch.surge.sh
surge –project ./dist –domain $DEPLOY_DOMAIN;

Here, Travis CI is first installing surge locally by npm i -g surge  and then we are exporting the environment variables SURGE_LOGIN , SURGE_TOKEN and DEPLOY_DOMAIN.

Now, execute pr_deploy.sh file from .travis.yml by using command bash pr_deploy.sh

Resources

Continue ReadingInstalling the Loklak Search and Deploying it to Surge
Read more about the article Installing Query Server Search and Adding Search Engines

Installing Query Server Search and Adding Search Engines

The query server can be used to search a keyword/phrase on a search engine (Google, Yahoo, Bing, Ask, DuckDuckGo and Yandex) and get the results as json or xml. The tool also stores the searched query string in a MongoDB database for analytical purposes. (The search engine scraper is based on the scraper at fossasia/searss.)

In this blog, we will talk about how to install Query-Server and implement the search engine of your own choice as an enhancement.

How to clone the repository

Sign up / Login to GitHub and head over to the Query-Server repository. Then follow these steps.

1. Go ahead and fork the repository

https://github.com/fossasia/query-server

2. Star the repository

3. Get the clone of the forked version on your local machine using

git clone https://github.com/<username>/query-server.git

4. Add upstream to synchronize repository using

git remote add upstream https://github.com/fossasia/query-server.git

Getting Started

The Query-Server application basically consists of the following :

1. Installing Node.js dependencies

npm install -g bower

bower install

2. Installing Python dependencies (Python 2.7 and 3.4+)

pip install -r requirements.txt

3. Setting up MongoDB server

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 7F0CEB10

echo "deb http://repo.mongodb.org/apt/ubuntu "$(lsb_release   -sc)"/mongodb-org/3.0   multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.0.list

sudo apt-get update

sudo apt-get install -y mongodb-org

sudo service mongod start

4. Now, run the query server:

python app/server.py

Go to http://localhost:7001/

How to contribute :

Add a search engine of your own choice

You can add a search engine of your choice apart from the existing ones in application.

  • Just add or edit 4 files and you are ready to go.

For adding a search engine ( say Exalead ) :

1. Add exalead.py in app/scrapers directory :

from __future__ import print_function

from generalized import Scraper


class Exalead(Scraper): # Exalead class inheriting Scraper

    """Scrapper class for Exalead"""


    def __init__(self):

       self.url = 'https://www.exalead.com/search/web/results/'

       self.defaultStart = 0

       self.startKey = ‘start_index’


    def parseResponse(self, soup):

       """ Parses the reponse and return set of urls

       Returns: urls (list)

               [[Tile1,url1], [Title2, url2],..]

       """

       urls = []

       for a in soup.findAll('a', {'class': 'title'}): # Scrap data with the corresponding tag

           url_entry = {'title': a.getText(), 'link': a.get('href')}

           urls.append(url_entry)


       return urls

Here, scraping data depends on the tag / class from where we could find the respective link and the title of the webpage.

2. Edit generalized.py in app/scrapers directory

from __future__ import print_function

import json

import sys

from google import Google

from duckduckgo import Duckduckgo

from bing import Bing

from yahoo import Yahoo

from ask import Ask

from yandex import Yandex

from exalead import Exalead   # import exalead.py



scrapers = {

    'g': Google(),

    'b': Bing(),

    'y': Yahoo(),

    'd': Duckduckgo(),

    'a': Ask(),

    'yd': Yandex(),

    't': Exalead() # Add exalead to scrapers with index ‘t’

}

From the scrapers dictionary, we could find which search engines had supported the project.

3. Edit server.py in app directory

@app.route('/api/v1/search/<search_engine>', methods=['GET'])

def search(search_engine):

    try:

       num = request.args.get('num') or 10

       count = int(num)

       qformat = request.args.get('format') or 'json'

       if qformat not in ('json', 'xml'):

           abort(400, 'Not Found - undefined format')


       engine = search_engine

       if engine not in ('google', 'bing', 'duckduckgo', 'yahoo', 'ask', ‘yandex' ‘exalead’): # Add exalead to the tuple

           err = [404, 'Incorrect search engine', qformat]

           return bad_request(err)


       query = request.args.get('query')

       if not query:

           err = [400, 'Not Found - missing query', qformat]

           return bad_request(err)

Checking, if the passed search engine is supporting the project, or not.

4.  Edit index.html in app/templates directory

     <button type="submit" value="ask" class="btn btn-lg  search btn-outline"><img src="{{ url_for('static', filename='images/ask_icon.ico') }}" width="30px" alt="Ask Icon"> Ask</button>

     <button type="submit" value="yandex" class="btn btn-lg  search btn-outline"><img src="{{ url_for('static', filename='images/yandex_icon.png') }}" width="30px" alt="Yandex Icon"> Yandex</button>

     <button type="submit" value="exalead" class="btn btn-lg  search btn-outline"><img src="{{ url_for('static', filename='images/exalead_icon.png') }}" width="30px" alt="Exalead Icon"> Exalead</button> # Add button for exalead
  • In a nutshell,

Scrape the data using the anchor tag having specific class name.

For example, searching fossasia using exalead

https://www.exalead.com/search/web/results/?q=fossasia&start_index=1

Here, after inspecting element for the links, you will find that anchor having class name as title is having the link and title of the webpage. So, scrap data using title classed anchor tag.

Similarly, you can add other search engines as well.

Resources

Continue ReadingInstalling Query Server Search and Adding Search Engines