Creating Feedback Logs for Analysis

The thumbs up and thumbs down feedback on the clients is meant for the improvement of the skills in SUSI.AI. So we need to scope the feedback system to a particular interaction rather than skill as a whole. The feedback logs can be used for various kinds of analysis and machine learning.

Server side implementation

Components of Feedback Log:

  • User ID – For identification of a feedback given by a particular user. For consistency in data, the user should not be able to change the feedback over the same interaction.
  • Interaction:
    • User query
    • SUSI Reply
  • Client location – The response of a skill may not be interesting for the users of a particular country. That means the skill should give localised results.
  • Skill path – The path on the server where the skill is stored.

Create a feedbackLogs.json file to store the logs of feedback given by the user and make a JSONTray object for that in src/ai/susi/DAO.java file. The JSON file contains the above mentioned components.

public static JsonTray feedbackLogs; 

Path feedbackLogs_per = susi_skill_rating_dir.resolve("feedbackLogs.json");
Path feedbackLogs_vol = susi_skill_rating_dir.resolve("feedbackLogs_session.json");
feedbackLogs = new JsonTray(feedbackLogs_per.toFile(), feedbackLogs_vol.toFile(), 1000000);
OS.protectPath(feedbackLogs_per);
OS.protectPath(feedbackLogs_vol);

Create FeedbackLogService.java file that acts as an API to create the feedback logs. The API accepts the feedback data from the client and stores it into the json file using DAO object. The user should be logged in to give feedback on an interaction. So keep the minimum user role as USER to access the API.

JSONObject feedbackLogObject = new JSONObject();
feedbackLogObject.put("timestamp", timestamp);
feedbackLogObject.put("uuid", idvalue);
feedbackLogObject.put("feedback", skill_rate);
feedbackLogObject.put("user_query", user_query);
feedbackLogObject.put("susi_reply", susi_reply);
feedbackLogObject.put("country_name", country_name);
feedbackLogObject.put("country_code", country_code);
feedbackLogObject.put("skill_path", skill_path);

The API is accessible at /cms/feedbackLog.json endpoint.

Send feedback log from Web Client

The feedback API should be called only if the user is logged in. When the user presses the feedback buttons fetch the required data for log (access token, user query, susi response, country and user feedback) and POST them on the feedbackLog.json API.

let rateEndPoint =   BASE_URL + '/cms/feedbackLog.json?model=' + skill.model + '&group=' + skill.group + '&language=' + skill.language + '&skill=' + skill.skill + '&rating=' + rating + '&access_token=' + accessToken + '&user_query=' + interaction.userQuery + '&susi_reply=' + interaction.susiReply + '&country_name=' + country.countryName + '&country_code=' + country.countryCode ;

$.ajax({
  url: rateEndPoint,
  success: function(response) {
      console.log('Skill rated successfully');
  }
})

References

Continue ReadingCreating Feedback Logs for Analysis

Showing only those languages for which skills are available

SUSI.AI is available for almost all the internationally recognised languages of the world. An author is allowed to create a skill in any of these languages. But there are some languages for which skills have not been created yet. So only those languages should be shown in the SUSI Skill CMS for which the skills are available. The approach is that all the languages must be listed while creating the skills but only non-empty languages must be listed while filtering skills on the CMS category page.

Updating the get languages API

  1. Add an API parameter in GetAllLanguages.java to fetch the group name. It is used to fetch the list of languages for which skills are available in that particular group. If no group is passed it means that the all the languages are to be listed. For that, we can use any group and show all the languages in that group. Say “Knowledge”.

String group_name = call.get("group", null);
if (group_name == null) {
    File group = new File(model, "Knowledge");
}

 

  1. Now check if the file inside the group folder is a directory. If yes then add it to the list of languages to be returned.

String[] languages = group.list((current, name) -> new File(current, name).isDirectory());

 

  1. If the languages corresponding to a particular category are to be fetched then first checked if the group is “All” or any specific group. Since the “All” category is not stored as such so we need to iterate over all the groups present in the parent directory ie, the model directory.

String[] group_names = model.list((current, name) -> new File(current, name).isDirectory());

 

  1. Now iterate over all the groups present in the group_names array and list the files present in it. Apply a filter to the list that accepts a file only if it is a directory and not empty ie, contains at least 1 language. Add that file to the list of languages.

group.list(new FilenameFilter() {
	@Override
	public boolean accept(File file, String s) {
		Boolean accepted = new File(file, s).list().length > 1;
		if (accepted) {
			if (!languages.contains(s)) {
		    	languages.add(s);
			}
		}
		return accepted;
	}
});

 

  1. The processing of getting languages for a particular group is same, just the iteration over the model directory is not required.

Resources

Continue ReadingShowing only those languages for which skills are available

STOP action implementation in SUSI iOS

You may have experienced, you can stop Google home or Amazon Alexa during the ongoing task. The same feature is available for SUSI too. Now, SUSI can respond to ‘stop’ action and stop ongoing tasks (e.g. SUSI is narrating a story and if the user says STOP, it stops narrating the story). ‘stop’ action is introduced to enable the user to make SUSI stop anything it’s doing.

Video demonstration of how stop action work on SUSI iOS App can be found here.

Stop action is implemented on SUSI iOS, Web chat, and Android. Here we will see how it is implemented in SUSI iOS.

When you ask SUSI to stop, you get following actions object from server side:

"actions": [{"type": "stop"}]

Full JSON response can be found here.

When SUSI respond with ‘stop’ action, we create a new action type ‘stop’ and assign `Message` object `actionType` to ‘stop’.

Adding ‘stop’ to action type:

enum ActionType: String {
... // other action types
case stop
}

Assigning to the message object:

if type == ActionType.stop.rawValue {
message.actionType = ActionType.stop.rawValue
message.message = ControllerConstants.stopMessage
message.answerData = AnswerAction(action: action)
}

A new collectionView cell is created to respond user with “stoped” text.

Registering the stopCell:

collectionView?.register(StopCell.self, forCellWithReuseIdentifier: ControllerConstants.stopCell)

Add cell to the chat screen:

if message.actionType == ActionType.stop.rawValue {
if let cell = collectionView.dequeueReusableCell(withReuseIdentifier: ControllerConstants.stopCell, for: indexPath) as? StopCell {
cell.message = message
let message = ControllerConstants.stopMessage
let estimatedFrame = self.estimatedFrame(message: message)
cell.setupCell(estimatedFrame, view.frame)
return cell
}
}

AVFoundation’s AVSpeechSynthesizer API is used to stop the action:

func stopSpeakAction() {
speechSynthesizer.stopSpeaking(at: AVSpeechBoundary.immediate)
}

This method immediately stops the speak action.

Final Output:

Resources – 

  1. About SUSI: https://chat.susi.ai/overview
  2. JSON response for ‘stop’ action: https://api.susi.ai/susi/chat.json?timezoneOffset=-330&q=susi+stop
  3. AVSpeechSynthesisVoice: https://developer.apple.com/documentation/avfoundation/avspeechsynthesisvoice
  4. AVFoundation: https://developer.apple.com/av-foundation/
  5. SUSI iOS Link: https://github.com/fossasia/susi_iOS
  6. SUSI Android Link: https://github.com/fossasia/susi_android
  7. SUSI Web Chat Link: https://chat.susi.ai/
Continue ReadingSTOP action implementation in SUSI iOS

Saving Sensor Data in CSV format

PSLab Android app by FOSSASIA provides a variety of features to its users. One of them is accessing various types of sensors both built into mobile phone and external sensors connected with PSLab device. In earlier versions users were only able to view the captured data. Moving forward, adding improvements to the app, now there is a feature to save those data displayed in graphs in csv format.

This feature is important in many ways. One is educational. In a classroom, teachers can ask students to perform an experiment and prepare a report using the data collected. By just visualizing they cannot do this. Actual data points must be made available. Another use is sharing data sets related to say environmental data over different demographics.

CSV, or comma-separated values file is a text file where stored data are separated by commas. The file stores these tabular data (numbers and text) in plain text format. Each line of the file represents a data record. Each data record consists of one or more fields, separated by commas. CSV files are commonly used to store sensor data because of its easy use. This post is about how PSLab device uses CSV file to write sensor data in it.

In PSLab android source code, there is a dedicated class to handle read sensor data from different instruments called “CSVLogger”. Developers can easily instantiate this class wherever they want a data logging as follows;

CSVLogger logger = new CSVLogger(<SUBFOLDER>); 
logger .writeCSVFile("Heading1,Heading2,Heading3\n");

 
This will create a blank folder in “PSLab” folder in device storage.  The CSV file is generated with the following convention according to the date and time where data is saved in the file.

yyyymmdd-hhmmss.csv

A sample file would have a name like 20180710-07:30:28.csv inside the SUBFOLDER which is specific to each instrument. Folder name will be the one used when initiating the CSVLogger.

With this method, logging data is pretty easy. Simply create a string which is a comma seperated and ended with a new line character. Then simply call the writeCSVFile(data) method with the string as a parameter added to it. It will keep appending string data until a new file is created. File creation can be handled by developers at their own interests and preferences.

String data = String.valueOf(System.currentTimeMillis()) + "," + item.getX() + "," + item.getY() + "\n";
logger.writeCSVFile(data);

 

To bring out an example let’s view how it’s implemented in Lux Meter instrument. This is a good source one can refer to when adding this feature in fragments

inside a main activity. In Lux Meter, there is the parent activity named Lux Meter and inside that there are two fragments, one is fragmentdata and the other one is fragmentsettings. Data capturing and saving occurs inside fragmentdata.

Menu icon controlling happens in the parent activity and we have bound a variable across the main activity and child fragment as follows;

LuxMeterActivity parent = (LuxMeterActivity) getActivity();
if (parent.saveData) {/* Save Data */}

 
This makes it easier listening menu icon clicks and start/stop recording accordingly. How to handle menu icons is beyond the scope of this blog and you can find tutorials on how to do that in the Resources section at the bottom of this blog post.

Once these CSV files are available, users can easily integrate them with advanced software like Matlab or Octave to do further analysis and processing to captured data sets.

Resources:

  1. CSV Logger: https://github.com/fossasia/pslab-android/blob/development/app/src/main/java/org/fossasia/pslab/others/CSVLogger.java
  2. Android Menu options: https://stackoverflow.com/questions/27984041/android-correct-use-of-invalidateoptionsmenu
Continue ReadingSaving Sensor Data in CSV format

Open Event Server – Export Event as a Pentabarf XML File

FOSSASIA‘s Open Event Server is the REST API backend for the event management platform, Open Event. Here, the event organizers can create their events, add tickets for it and manage all aspects from the schedule to the speakers. Also, once he makes his event public, others can view it and buy tickets if interested.

To make event promotion easier, we also provide the event organizer to export his event as a Pentabarf XML file. Pentabarf XML is used to store events/conferences in a format which most of the scheduling applications can read and add that particular event/conference to the user’s schedule.

Server side – generating the Pentabarf XML file

Here we will be using the pentabarf package for Python for parsing and creating the file.

from pentabarf.Conference import Conference
from pentabarf.Day import Day
from pentabarf.Event import Event
from pentabarf.Person import Person
from pentabarf.Room import Room
  • We define a class PentabarfExporter which has a static method export(event_id).
  • Query the event using the event_id passed and start forming the event in the required format:
event = EventModel.query.get(event_id)
diff = (event.ends_at - event.starts_at)

conference = Conference(title=event.name, start=event.starts_at, end=event.ends_at,
                       days=diff.days if diff.days > 0 else 1,
                       day_change="00:00", timeslot_duration="00:15",
                       venue=event.location_name)
dates = (db.session.query(cast(Session.starts_at, DATE))
        .filter_by(event_id=event_id)
        .filter_by(state='accepted')
        .filter(Session.deleted_at.is_(None))
        .order_by(asc(Session.starts_at)).distinct().all())
  • We have queried for the dates of the event and saved it in dates.
  • We will now iterate over each date and query the microlocations who have a session on that particular date.
for date in dates:
   date = date[0]
   day = Day(date=date)
   microlocation_ids = list(db.session.query(Session.microlocation_id)
                            .filter(func.date(Session.starts_at) == date)
                            .filter_by(state='accepted')
                            .filter(Session.deleted_at.is_(None))
                            .order_by(asc(Session.microlocation_id)).distinct())
  • For each microlocation thus obtained, we will query for accepted sessions to be held at those microlocations.
  • We will also initialize a Room for each microlocation.
for microlocation_id in microlocation_ids:
   microlocation_id = microlocation_id[0]
   microlocation = Microlocation.query.get(microlocation_id)
   sessions = Session.query.filter_by(microlocation_id=microlocation_id) \
       .filter(func.date(Session.starts_at) == date) \
       .filter_by(state='accepted') \
       .filter(Session.deleted_at.is_(None)) \
       .order_by(asc(Session.starts_at)).all()

   room = Room(name=microlocation.name)
  • We will now iterate over the aabove-obtained sessions and instantiate an Event for each session.
  • Then we will iterate over all the speakers of that session and instantiate a Person for each speaker.
  • Finally, we will add that Event to the Room we created earlier.
for session in sessions:

   session_event = Event(id=session.id,
                         date=session.starts_at,
                         start=session.starts_at,
                         duration=str(session.ends_at - session.starts_at) + "00:00",
                         track=session.track.name,
                         abstract=session.short_abstract,
                         title=session.title,
                         type='Talk',
                         description=session.long_abstract,
                         conf_url=url_for('event_detail.display_event_detail_home',
                                          identifier=event.identifier),
                         full_conf_url=url_for('event_detail.display_event_detail_home',
                                               identifier=event.identifier, _external=True),
                         released="True" if event.schedule_published_on else "False")

   for speaker in session.speakers:
       person = Person(id=speaker.id, name=speaker.name)
       session_event.add_person(person)

   room.add_event(session_event)
  • Then we will add the room to the day and then add each day to the conference.
day.add_room(room)
conference.add_day(day)
  • Finally, we will call the generate method of the conference to generate the XML file. This can be directly written to the file.
return conference.generate("Generated by " + get_settings()['app_name'])

Obtaining the Pentabarf XML file:

Firstly, we have an API endpoint which starts the task on the server.

GET - /v1/events/{event_identifier}/export/pentabarf

Here, event_identifier is the unique ID of the event. This endpoint starts a celery task on the server to export the event as a Pentabarf XML file. It returns the task of the URL to get the status of the export task. A sample response is as follows:

{
  "task_url": "/v1/tasks/b7ca7088-876e-4c29-a0ee-b8029a64849a"
}

The user can go to the above-returned URL and check the status of his Celery task. If the task completed successfully he will get the download URL. The endpoint to check the status of the task is:

and the corresponding response from the server –

{
  "result": {
    "download_url": "/v1/events/1/exports/http://localhost/static/media/exports/1/zip/OGpMM0w2RH/event1.zip"
  },
  "state": "SUCCESS"
}

The file can be downloaded from the above-mentioned URL.

Hence, now the event can be added to any scheduling app which recognizes the Pentabarf XML format.

References

Continue ReadingOpen Event Server – Export Event as a Pentabarf XML File

Add RSS feed and JSON output based on type specified with query param

The idea behind writing this blog post is to discuss the method on how RSS feed and JSON output sources have been included in loklak to provide respective data sources based on the type specified with query as a parameter.

Accessing Current Query in Info-box

Accessing link to RSS feed and JSON output of loklak requires the Query to be passed as a value with parameter ‘q’ (e.g. api/search.json?q=FOSSASIA or api/search.rss?q=FOSSASIA). In order to represent links as buttons in Info-box at sidebar of loklak.org, current Query needs to be accessed/stored inside Info-box from ngrx store.

public stringQuery;
...
this.store.select(fromRoot.getQuery).subscribe(
    query => this.stringQuery = query.displayString);

 

Firstly stringQuery variable is created to store the current Query from store. As the Query can be changed in store (User might search for several Queries), storing of current Query needs to be done inside ngOnChanges().

Checking type associated with Query

There are various types associated with Query to get different type of results like ‘from:FOSSASIA will give results specifically from ‘FOSSASIA’ which will have different query parameter from other types like ‘@FOSSASIA’ or ‘#FOSSASIA’. To assign appropriate Query parameter to each of these types, we need to check the Query pattern to apply Query param based on the type.

import { hashtagRegExp, fromRegExp, mentionRegExp }
      from ‘../../utils/reg-exp’;
...

if ( hashtagRegExp.exec(this.stringQuery) !== null ) {
	// Check for hashtag this.stringQuery
	this.queryString = ‘%23 + hashtagRegExp.exec(
	this.stringQuery)[1] + '' + hashtagRegExp.exec(
	this.stringQuery)[0];
} else if ( fromRegExp.exec(this.stringQuery) !== null ) {
	// Check for from user this.stringQuery
	this.queryString = ‘from%3A’ + fromRegExp.exec(
	this.stringQuery)[1];
} else if ( mentionRegExp.exec(this.stringQuery) !== null ) {
	// Check for mention this.stringQuery
	this.queryString = ‘%40 + mentionRegExp.exec(
	this.stringQuery)[1];
} else {
	// for other queries
	this.queryString = this.stringQuery;
}

 

Note: hashtagRegExp, fromRegExp, mentionRegExp are the utility functions created to match the pattern of given string (Query) in order to classify the preceding type associated with Query. These are provided here as a reference, which can be used to add more of the types.

Passing current queryString in RSS and JSON link

General link for both RSS and JSON data remains same for each Query passed, only the type associated would be changed in the Query value. So representing the link in an anchor tag in UI would be as –

<a class=“data rss” href=“
		http://api.loklak.org/api/search.
		rss?timezoneOffset=-330&q=
		{{stringQuery}}” target=“_blank”>
</a>
<a class=”data json” href=”http://api.
		loklak.org/api/search
		.json?timezoneOffset=-330&q=
		{{stringQuery}}” target=”_blank“>
</a>

 

{{stringQuery}} is the actual query parameter to be passed to get the required results.

Testing RSS feed and JSON data

Search for a query on loklak, and click on the the RSS or JSON button below sidebar on results page and compare with the results.

RSS and JSON button should be similar to –

Resources

Continue ReadingAdd RSS feed and JSON output based on type specified with query param

Stripe Authorization in Open Event Server

Stripe is a popular software platform for online payments. Since Open Event  allows the event organizers to sell tickets, an option to accept payments through Stripe is extremely beneficial to the organizer. Stripe allows accepting payments on other’s behalf using Connect. Connect is the Stripe’s full stack solution for platforms that need to process payments and process to multiple parties. This blog post goes over how Event organizers are able to link their Stripe accounts in order to accept payments later.

Registering the platform

The Admin of the Open Event Server will create an account on Stripe and register the platform. Upon creating the  account he/she will get a secret_key and publishable_key.  Moreover on registering the platform a client_id will be provided. These keys are saved in the application settings and only the Admin is authorized to view or change them.

Connecting the Organiser’s account

The Open Event Frontend has a wizard for creating an Event. It provides the organiser an option to connect his/her Stripe account in order to accept payments.

Upon clicking the following button, the organiser is directed to Stripe where he/she can fill the required details.  

The button directs the organizer to the following URL:

https://connect.stripe.com/oauth/authorize?response_type=code&client_id=client_id&scope=read_write&redirect_uri=redirect_uri 

The above URL has the following parameters:

  • client_id – The client ID acquired when registering your platform.required.
  • response_type – Response type. The value is always code. required.
  • redirect_uri – The URL to redirect the customer to after authorization.
  • scope – We need it to be read_write in order to be able to charge on behalf of the customer later.

After successfully entering the required details, the organizer is redirected to the redirect_url as specified in the above URL with a query parameter named as authorization_code. The Frontend sends this code to the Server using the Stripe Authorization endpoint which will be discussed in detail below.

Fetching Tokens from Stripe

The Server accepts the authorization_code by exposing the Stripe Authorization endpoint. It then uses it to fetch organizer’s details and token from Stripe and stores it for future use.

The schema for Stripe Authorization is extremely simple. We require the client to send an authorization_code which will be used to fetch the details. Stripe_publishable_key of the event organizer is exposed via the endpoint and will be used by the Frontend later.

class StripeAuthorizationSchema(Schema):
    """
        Stripe Authorization Schema
    """

    class Meta:
        """
        Meta class for StripeAuthorization Api Schema
        """
        type_ = 'stripe-authorization'
        self_view = 'v1.stripe_authorization_detail'
        self_view_kwargs = {'id': '<id>'}
        inflect = dasherize

    id = fields.Str(dump_only=True)
    stripe_publishable_key = fields.Str(dump_only=True)
    stripe_auth_code = fields.Str(load_only=True, required=True)

    event = Relationship(attribute='event',
                self_view='v1.stripe_authorization_event',
                self_view_kwargs={'id': '<id>'},
                related_view='v1.event_detail',
                related_view_kwargs={'stripe_authorization_id':                      '<id>'},
                schema="EventSchema",
                type_='event')

We use the Requests library in order to fetch the results. First we fetch the client_id that we had stored in the application settings using a helper method called get_credentials. We then use it along with the authorization_code in order to make a POST request to Stripe Connect API. The full method is given below for reference.

@staticmethod
def get_event_organizer_credentials_from_stripe(stripe_auth_code):
        """
        Uses the stripe_auth_code to get the other credentials for the event organizer's stripe account
        :param stripe_auth_code: stripe authorization code
        :return: response from stripe
        """
        credentials = StripePaymentsManager.get_credentials()

        if not credentials:
            raise Exception('Stripe is incorrectly configured')

        data = {
            'client_secret': credentials['SECRET_KEY'],
            'code': stripe_auth_code,
            'grant_type': 'authorization_code'
        }

        response = requests.post('https://connect.stripe.com/oauth/token', data=data)
        return json.loads(response.text)

We call the above method before creating the object using the before_create_object method of Marshmallow which allows us to do data preprocessing and validations.

If the request was a success, the response from Stripe connect API includes all the details necessary to accept payments on their behalf. We add these fields to the data and save it in the database.

{
  "token_type": "bearer",
  "stripe_publishable_key": PUBLISHABLE_KEY,
  "scope": "read_write",
  "livemode": false,
  "stripe_user_id": USER_ID,
  "refresh_token": REFRESH_TOKEN,
  "access_token": ACCESS_TOKEN
}

In case there was an error, an error_description would be returned. This error_description is sent back to the frontend and shown to the event organizer.

{
  "error": "invalid_grant",
  "error_description": "Authorization code already used:                                               
                        AUTHORIZATION_CODE"
}

After successfully fetching the results, we save it inside the database and return the stripe_publishable_key which will be used by the Frontend when charging the ticket buyers later.

Lastly we can go over the Stripe Authorization model as well. The stripe_secret_key will be used to charge the customers later.

id = db.Column(db.Integer, primary_key=True)
stripe_secret_key = db.Column(db.String)
stripe_refresh_token = db.Column(db.String)
stripe_publishable_key = db.Column(db.String)
stripe_user_id = db.Column(db.String)
stripe_auth_code = db.Column(db.String)

References

 

Continue ReadingStripe Authorization in Open Event Server

Open Event Server – Export Event as xCalendar File

FOSSASIA‘s Open Event Server is the REST API backend for the event management platform, Open Event. Here, the event organizers can create their events, add tickets for it and manage all aspects from the schedule to the speakers. Also, once he makes his event public, others can view it and buy tickets if interested.

To make event promotion easier, we also provide the event organizer to export his event as an xCalendar file. xCal is an XML representation of the iCalendar standard. xCal is not an alternative nor next generation of iCalendar. xCal represents iCalendar components, properties, and parameters as defined in iCalendar. This format was selected to ease its translation back to the iCalendar format using an XSLT transform.

Server side – generating the xCal file

Here we will be using the xml.etree.ElementTree package for Python for parsing and creating XML data.

from xml.etree.ElementTree import Element, SubElement, tostring
  • We define a class XCalExporter which has a static method export(event_id).
  • Query the event using the event_id passed and start forming the calendar:
event = Event.query.get(event_id)

tz = event.timezone or 'UTC'
tz = pytz.timezone(tz)

i_calendar_node = Element('iCalendar')
i_calendar_node.set('xmlns:xCal', 'urn:ietf:params:xml:ns:xcal')
v_calendar_node = SubElement(i_calendar_node, 'vcalendar')
version_node = SubElement(v_calendar_node, 'version')
version_node.text = '2.0'
prod_id_node = SubElement(v_calendar_node, 'prodid')
prod_id_node.text = '-//fossasia//open-event//EN'
cal_desc_node = SubElement(v_calendar_node, 'x-wr-caldesc')
cal_desc_node.text = "Schedule for sessions at " + event.name
cal_name_node = SubElement(v_calendar_node, 'x-wr-calname')
cal_name_node.text = event.name
  • We query for the accepted sessions of the event and store it in sessions
sessions = Session.query \
   .filter_by(event_id=event_id) \
   .filter_by(state='accepted') \
   .filter(Session.deleted_at.is_(None)) \
   .order_by(asc(Session.starts_at)).all()
  • We then iterate through all the sessions in sessions.
  • If it is a valid session, we instantiate a SubElement and store required details
v_event_node = SubElement(v_calendar_node, 'vevent')

method_node = SubElement(v_event_node, 'method')
method_node.text = 'PUBLISH'

uid_node = SubElement(v_event_node, 'uid')
uid_node.text = str(session.id) + "-" + event.identifier

dtstart_node = SubElement(v_event_node, 'dtstart')
dtstart_node.text = tz.localize(session.starts_at).isoformat()

…. So on
  • We then loop through all the speakers in that particular session and add it to the xCal calendar node object as well.
for speaker in session.speakers:
   attendee_node = SubElement(v_event_node, 'attendee')
   attendee_node.text = speaker.name
  • And finally, the string of the calendar node is returned. This is the xCalendar file contents. This can be directly written to a file.
return tostring(i_calendar_node)

Obtaining the xCal file:

Firstly, we have an API endpoint which starts the task on the server.

GET - /v1/events/{event_identifier}/export/xcal

Here, event_identifier is the unique ID of the event. This endpoint starts a celery task on the server to export the event as an xCal file. It returns the URL of the task to get the status of the export task. A sample response is as follows:

{
  "task_url": "/v1/tasks/b7ca7088-876e-4c29-a0ee-b8029a64849a"
}

The user can go to the above-returned URL and check the status of his Celery task. If the task completed successfully he will get the download URL. The endpoint to check the status of the task is:

and the corresponding response from the server –

{
  "result": {
    "download_url": "/v1/events/1/exports/http://localhost/static/media/exports/1/zip/OGpMM0w2RH/event1.zip"
  },
  "state": "SUCCESS"
}

The file can be downloaded from the above mentioned URL.

Hence, now the event can be added to any scheduling app which recognizes the xcs format.

References

Continue ReadingOpen Event Server – Export Event as xCalendar File

Displaying Top Hashtags by sorting Hashtags based on the frequency

It is a good idea to display top hashtags on sidebar of loklak. To represent them, it is really important to sort out all unique hashtags on basis of frequency from results obtained from api.loklak. The implementation of the process involved would be discussed in this blog.

Raw Hashtag result

The Hashtags obtained as a parameter of type array containing array of strings into the sortHashtags() method inside the component typescript file of Info-box Component is in Raw Hashtag form.

Making Array of all Hashtags

Firstly, all the Hashtags would be added to a new Array – stored.

sortHashtags( statistics ) {
    let stored = [];
    if (statistics !== undefined && 
        statistics.length !== 0) {
        for (const s in statistics) {
            if (s) {
                for (let i = 0;i <
                    statistics[s].length; i++) {
                    stored.push(statistics[s][i]);
                }
            }
        }
    }
}

 

stored.push( element ) will add each element ( Hashtag ) into the stored array.

Finding frequency of each unique Hashtag

array.reduce() method would be used to store all the Hashtags inside the stored array with frequency of each unique Hashtag (e.g. [ ‘Hashtag1’: 3, ‘Hashtag2’: 2, ‘Hashtag3’: 5, … ]), where Hashtag1 has appeared 3 times, Hashtag2 appeared 2 times and so on.

stored = stored.reduce(function (acc, curr) {
    if (typeof acc[curr] === undefined’) {
        acc[curr] = 1;
    } else {
        acc[curr] += 1;
    }
    return acc;
}, []);

 

stored.reduce() would store the result inside stored array in the format mentioned above.

Using Object to get the required result

Object would be used with different combination of associated methods such as map, filter, sort and slice to get the required Top 10 Hashtags sorted on the basis of frequency of each unique Hashtag.

this.topHashtags = Object.keys(stored)
    .map(key => key.trim())
    .filter(key => key !== ”)
    .map(key => ([key, stored[key]]))
    .sort((a, b) => b[1]  a[1])
    .slice(0, 10);

 

At last, the result is stored inside topHashtags array. First map method is used to trim out spaces from all the keys ( Hashtags ), first filter is applied to remove all those Hashtags which are empty and then mapping each Hashtag as an array with a unique index inside the containing array. At last, sorting each Hashtag on basis of the frequency using sort method and slicing the results to get Top 10 Hashtags to be displayed on sidebar of loklak.

Testing Top Hashtags

Search something on loklak.org to obtain sidebar with results. Now look through the Top 10 Hashtags being displayed on the Sidebar info-box.

Resources

Continue ReadingDisplaying Top Hashtags by sorting Hashtags based on the frequency