Linking Codecov to your Angular2 Project

As a developer, the importance of testing code is well known. Testing source code helps to prevent bugs and syntax errors by cross-checking it with an expected output.
As stated on their official website: “Code coverage provides a visual measurement of what source code is being executed by a test suite”. This information indicates to the software developer where they should write new tests in the effort to achieve higher coverage, and consequently a lower chance of being buggy. Hence nearly every public repository on git uses codecov, a tool to measure the coverage of their source code.

In this blog, we shall see how to link codecov, with a public repository on Github when the code has been written in Angular2(Typescript). We shall assume that the repository uses Travis CI for integration.

STEP 1:
Go to https://codecov.io/gh/ and login to your Github account.

It will now give you the option to chose a repository to add for coverage. Select your repository.

STEP 2:
Navigate to Settings Tab, you should see something like this:

Follow the above-mentioned instructions.

STEP 3:
We now come to one of the most important parts of Codecov integration. Writing the files in our repo to enable this.
We will need three main files:
Travis.yml- which will ensure continuous integration services on your git hosted project
Codecov.yml- to personalise your settings and override the default settings in codecov.”The Codecov Yaml file is the single point of configuration, providing the developers with a transparent and version controlled file to adjust all Codecov settings.” as mentioned in the official website
Package.json- to inform npm of the new dependencies related to codecov, in addition to providing all the metadata to the user.

In .travis.yml, Add the following line:
after_success:

 - bash <(curl -s https://codecov.io/bash)

In codecov.yml, Add the following

Source: https://github.com/codecov/support/wiki/Codecov-Yaml#
 codecov:
 url: "string" # [enterprise] your self-hosted Codecov endpoint
 # ex. https://codecov.company.com
 slug: "owner/repo" # [enterprise] the project's name when using the global upload tokens
 branch: master # the branch to show by default, inherited from your git repository settings
 # ex. master, stable or release
 # default: the default branch in git/mercurial
 bot: username # the username that will consume any oauth requests
 # must have previously logged into Codecov
 ci: # [advanced] a list of custom CI domains
 - "ci.custom.com"
 notify: # [advanced] usage only
 after_n_builds: 5 # how many build to wait for before submitting notifications
 # therefore skipping status checks
 countdown: 50 # number of seconds to wait before checking CI status
 delay: 100 # number of seconds between each CI status check

coverage:
 precision: 2 # how many decimal places to display in the UI: 0 <= value <= 4 round: down # how coverage is rounded: down/up/nearest range: 50...100 # custom range of coverage colors from red -> yellow -> green

notify:
 irc:
 default: # -> see "sections" below
 server: "chat.freenode.net" #*S the domain of the irc server
 branches: null # -> see "branch patterns" below
 threshold: null # -> see "threshold" below
 message: "template string" # [advanced] -> see "customized message" below

gitter:
 default: # -> see "sections" below
 url: "https://webhooks.gitter.im/..." #*S unique Gitter notifications url
 branches: null # -> see "branch patterns" below
 threshold: null # -> see "threshold" below
 message: "template string" # [advanced] -> see "customized message" below

status:
 project: # measuring the overall project coverage
 default: # context, you can create multiple ones with custom titles
 enabled: yes # must be yes|true to enable this status
 target: auto # specify the target coverage for each commit status
 # option: "auto" (must increase from parent commit or pull request base)
 # option: "X%" a static target percentage to hit
 branches: # -> see "branch patterns" below
 threshold: null # allowed to drop X% and still result in a "success" commit status
 if_no_uploads: error # will post commit status of "error" if no coverage reports we uploaded
 # options: success, error, failure
 if_not_found: success # if parent is not found report status as success, error, or failure
 if_ci_failed: error # if ci fails report status as success, error, or failure


patch: # pull requests only: this commit status will measure the
 # entire pull requests Coverage Diff. Checking if the lines
 # adjusted are covered at least X%.
 default:
 enabled: yes # must be yes|true to enable this status
 target: 80% # specify the target "X%" coverage to hit
 branches: null # -> see "branch patterns" below
 threshold: null # allowed to drop X% and still result in a "success" commit status
 if_no_uploads: error # will post commit status of "error" if no coverage reports we uploaded
 # options: success, error, failure
 if_not_found: success
 if_ci_failed: error

changes: # if there are any unexpected changes in coverage
 default:
 enabled: yes # must be yes|true to enable this status
 branches: null # -> see "branch patterns" below
 if_no_uploads: error
 if_not_found: success
 if_ci_failed: error

ignore: # files and folders that will be removed during processing
 - "tests/*"
 - "demo/*.rb"

fixes: # [advanced] in rare cases the report tree is invalid, specify adjustments here
 - "old_path::new_path"

# comment: false # to disable comments
 comment:
 layout: "header, diff, changes, sunburst, suggestions, tree"
 branches: null # -> see "branch patterns" below
 behavior: default # option: "default" posts once then update, posts new if delete
 # option: "once" post once then updates, if deleted do not post new
 # option: "new" delete old, post new
 # option: "spammy" post new

Your package.json should look like this:

{
 "name": "example-typescript",
 "version": "1.0.0",
 "description": "Codecov Example Typescript",
 "main": "index.js",
 "devDependencies": {
 "chai": "^3.5.0",
 "codecov": "^1.0.1",
 "mocha": "^2.5.3",
 "nyc": "^6.4.4",
 "tsd": "^0.6.5",
 "typescript": "^1.8.10"
 },
 "scripts": {
 "postinstall": "tsd install",
 "pretest": "tsc test/*.ts --module commonjs --sourcemap",
 "test": "nyc mocha",
 "posttest": "nyc report --reporter=json && codecov -f coverage/*.json"
 },
 "repository": {
 "type": "git",
 "url": "git+https://github.com/Myname/example-typescript.git"
 },
 /*Optional*/
 "author": "Myname",
 "license": "Lic.name",
 "bugs": {
 "url": "https://github.com/example-typescript-path"
 },
 "homepage": "https://github.com/Myname/example-typescript#readme"
 }

Most of the code in package.json is metadata.
Two major parts of the code above are the devDependencies and the scripts.
In devDependencies, make sure to include the latest versions of all the packages your repository is using.
In scripts:

  • Postinstall – indicates the actions to be performed, once installation is complete.
  • Pretest – is for just before running ng test.
  • Test – indicates what is used while testing.
  • Posttest – is what is run just after testing is complete.

Check this repository for the sample files to generate the reports to be uploaded for Codecov: https://github.com/codecov/example-typescript

Check https://docs.codecov.io/v4.3.6/docs/codecov-yaml for detailed step by step instructions on writing codecov.yaml and https://docs.codecov.io/v4.3.6/docs for any general information

Adding a page to Susper

As a project grows, it’s complexity increases. It suddenly seems to have a higher number of components and linked files. Performing a basic task, such as adding a new page to the website, might take more intricate knowledge. This blog deals with adding a new page to any Angular 2 project, in this case to Susper in specific.

STEP 1:

Use ng g component <component-name> command to generate a new component with your desired name. Make sure the component-name is relevant and describes the page you wish to add. Once the above command is run in the Angular CLI project, it will automatically generate the following files:

  • component-name.html
  • component-name.css
  • component-name.ts
  • component-name.spec.ts file

It also adds the new component name to src/app/app.module.ts in the declarations section, after importing it.

You can also do all of this manually without the ng g component command too.

STEP 2:

Write your HTML and CSS files. Ensure that your page looks how you intend it to. Using bootstrap for your CSS classes might help you. Ensure that you link your bootstrap modules in index.html and not in the individual component files.

STEP 3:

If your page uses any typescript functions, please link your functions to your HTML page, after defining them in typescript.

This is how your typescript file might look (This is how it looks in Susper):

You may want to import modules you will need first.

Notice this snippet of code:

export class DemoComponent implements OnInit {

constructor() { }

ngOnInit() {

 }

}

You can define all your variables and functions in the component class. ngOnInit() has already been listed as a demo.

Also, note that including anything in the constructor function will run as soon as the page is loaded or initialized.

STEP 4:

If you have a unit tests in place, then component-name.spec.ts is where they are listed. Make sure to update it.

This is the procedure in Susper. Initially, your component-name.spec.ts will look like this:

import { async, ComponentFixture, TestBed } from [email protected]/core/testing'

import { DemoComponent } from './demo.component'



describe('DemoComponent', () => {

let component: DemoComponent;

let fixture: ComponentFixture<DemoComponent>;



beforeEach(async(() => {

   TestBed.configureTestingModule({

     declarations: [ DemoComponent ]

   })

   .compileComponents();

 }));

beforeEach(() => {

   fixture = TestBed.createComponent(DemoComponent);

   component = fixture.componentInstance;

   fixture.detectChanges();

 });

it('should create', () => {

   expect(component).toBeTruthy();

 });

});

 

Here is what you need to add:

  1. Add imports under Testbed.configureTestingModule.

TIP:  Make sure to import all the modules from their file locations first.

imports: [

 RouterTestingModule,

 BrowserModule,

 CommonModule,

 FormsModule,

 HttpModule,

 JsonpModule,

 StoreModule.provideStore(reducer),

 StoreDevtoolsModule.instrumentOnlyWithExtension()

],
  1. Add all the other components under the declarations heading.

   TIP: Make sure to import all the components first.

It should look something like this:

declarations: [

   AppComponent,

   NavbarComponent,

   IndexComponent,

   ResultsComponent,

   NotFoundComponent,

   AdvancedsearchComponent,

   SearchBarComponent,

   FooterNavbarComponent,

   AboutComponent,

   ContactComponent,

   ModalComponent,

   InfoboxComponent,

 DemoComponent,

 ],

})

 

  1. Now add service providers under the providers heading if any.
  2. Finally, add any additional test cases using the standard syntax proved by Jasmine with it and expect statements.

STEP 5:

Update all .spec.ts files with your new component name under the declarations heading as seen in point 2 of step 4. This will notify all other spec.ts files about your new component, allowing ng test to run smoothly.

Make sure to import it each time you use it, to avoid compilation errors.

STEP 6:

To be able to reach your page, you can either

  • Embed it in another page using the selector mentioned in its .ts file.
@Component({

selector: 'app-demo',

Simply include the following tag in whichever page you wish to use the demo component, in the .html file:

<app-demo></app-demo>

  • Give it a route through which the user can reach it. Do this, by adding it in the Routes in app.module.ts
const appRoutes: Routes = [

 {path: 'search', component: ResultsComponent},

{path: '', component: IndexComponent},

You are done! You have successfully added a page to Susper!

Advanced customization of the Yaydoc Build Process

Introduction

Although, Yaydoc exposes many environment variables which can be used to configure various aspects of the build process, there may be cases where a user needs much more finer control over the build process. Yaydoc uses sphinx under the hood which uses a file named conf.py to allow users to customize the build. As part of the build process, Yaydoc generates a file named conf.py from a custom made jinja2 template. With this week’s update, now a user can extend the generated conf.py by providing their own conf.py whose content would be appended to the generated conf.py.

More Details

Why append you may ask. Why not just overwrite? This is because the generated conf.py has a lot of boilerplate code which when overwritten will need to be rewritten by the user. That is why the contents are appended so that the user will only need to specify any extra configuration options they may wish to add or override. This approach has the following advantages:

  • Ability to override or add any configuration option during build.
  • Since the conf.py file is execfile`d by sphinx during build, the user has the ability to execute arbitrary code to customize any part of the build process.

The following block of code implements this feature.

if [ -f $DOCPATH/conf.py ]; then
  echo >> BUILD_DIR/conf.py
  cat $DOCPATH/conf.py >> BUILD_DIR/conf.py
  rsync -a $DOCPATH. BUILD_DIR/ --exclude=conf.py
else
  cp -a $DOCPATH. BUILD_DIR/
fi

Here we check if user has provided a conf.py, we append it to the generated conf.py. To append we used the >> shell redirection feature. It redirects stdout to a file similar to > but instead of overwriting the file, it appends to it.

This brings us on parity with sphinx as  far as customization goes. We may expose some more configuration variables for easier setup in the future, but now you can always modify any aspects of the build process even if it is not exposed via a variable. This should be enough for most use cases. More changes are on the way. Stay tuned for more updates.

Adding support for Markdown in Yaydoc

Yaydoc being based on sphinx natively supports reStructuredText. From the official docs:

reStructuredText is an easy-to-read, what-you-see-is-what-you-get plaintext markup syntax and parser system. It is useful for quickly creating simple web pages, and for standalone documents. reStructuredText is designed for extensibility for specific application domains.

Although it being superior to markdown in terms of features, Markdown is still the most heavily used markup language out there. This week we added support for markdown into Yaydoc. Now you can use Markdown to document your project and Yaydoc would create a site with no changes required from your end. To achieve this, we used recommonmark, which enables sphinx to parse CommonMark, a strongly defined, highly compatible specification of Markdown. It solved most of the problem with 3 lines of code in our customized conf.py .

from recommonmark.parser import CommonMarkParser

source_parsers = {
'.md': CommonMarkParser,
}

source_suffix = ['.rst', '.md']

With this addition, sphinx can now use recommonmark to convert markdown to html. But not everything has been solved. Here is an excerpt from a previous blogpost which explains a problem yet to be solved.

Now sphinx requires an index.rst file within docs directory  which it uses to generate the first page of the site. A very obvious way to fill it which helps us avoid unnecessary duplication is to use the include directive of reStructuredText to include the README file from the root of the repository. But the Include directive can only properly include a reStructuredText, not a markdown document. Given a markdown document, it tries to parse the markdown as  reStructuredText which leads to errors.

To solve this problem, a custom directive mdinclude was created. Directives are the primary extension mechanism of reStructuredText. Most of it’s implementation is a copy of the built in Include directive from the docutils package. Before including in the doctree, mdinclude converts the docs from markdown to reStructuredText using pypandoc. The implementation is similar to the one also discussed in a previous blogpost.

class MdInclude(rst.Directive):

required_arguments = 1
optional_arguments = 0

def run(self):
    if not self.state.document.settings.file_insertion_enabled:
        raise self.warning('"%s" directive disabled.' % self.name)
    source = self.state_machine.input_lines.source(
        self.lineno - self.state_machine.input_offset - 1)
    source_dir = os.path.dirname(os.path.abspath(source))
    path = rst.directives.path(self.arguments[0])
    path = os.path.normpath(os.path.join(source_dir, path))
    path = utils.relative_path(None, path)
    path = nodes.reprunicode(path)

    encoding = self.options.get(
        'encoding', self.state.document.settings.input_encoding)
    e_handler = self.state.document.settings.input_encoding_error_handler
    tab_width = self.options.get(
        'tab-width', self.state.document.settings.tab_width)

    try:
        self.state.document.settings.record_dependencies.add(path)
        include_file = io.FileInput(source_path=path,
                                    encoding=encoding,
                                    error_handler=e_handler)
    except UnicodeEncodeError as error:
        raise self.severe('Problems with "%s" directive path:\n'
                          'Cannot encode input file path "%s" '
                          '(wrong locale?).' %
                          (self.name, SafeString(path)))
    except IOError as error:
        raise self.severe('Problems with "%s" directive path:\n%s.' %
                          (self.name, ErrorString(error)))

    try:
        rawtext = include_file.read()
    except UnicodeError as error:
        raise self.severe('Problem with "%s" directive:\n%s' %
                          (self.name, ErrorString(error)))

    output = md2rst(rawtext)
    include_lines = statemachine.string2lines(output,
                                              tab_width, 
                                              convert_whitespace=True)
    self.state_machine.insert_input(include_lines, path)
    return []

With this, Yaydoc can now be used on projects that exclusively use markdown. There are some more hurdles which we need to cross in the following weeks. Stay tuned for more updates.

Automatic Imports of Events to Open Event from online event sites with Query Server and Event Collect

One goal for the next version of the Open Event project is to allow an automatic import of events from various event listing sites. We will implement this using Open Event Import APIs and two additional modules: Query Server and Event Collect. The idea is to run the modules as micro-services or as stand-alone solutions.

Query Server
The query server is, as the name suggests, a query processor. As we are moving towards an API-centric approach for the server, query-server also has API endpoints (v1). Using this API you can get the data from the server in the mentioned format. The API itself is quite intuitive.

API to get data from query-server

GET /api/v1/search/<search-engine>/query=query&format=format

Sample Response Header

 Cache-Control: no-cache
 Connection: keep-alive
 Content-Length: 1395
 Content-Type: application/xml; charset=utf-8
 Date: Wed, 24 May 2017 08:33:42 GMT
 Server: Werkzeug/0.12.1 Python/2.7.13
 Via: 1.1 vegur

The server is built in Flask. The GitHub repository of the server contains a simple Bootstrap front-end, which is used as a testing ground for results. The query string calls the search engine result scraper scraper.py that is based on the scraper at searss. This scraper takes search engine, presently Google, Bing, DuckDuckGo and Yahoo as additional input and searches on that search engine. The output from the scraper, which can be in XML or in JSON depending on the API parameters is returned, while the search query is stored into MongoDB database with the query string indexing. This is done keeping in mind the capabilities to be added in order to use Kibana analyzing tools.

The frontend prettifies results with the help of PrismJS. The query-server will be used for initial listing of events from different search engines. This will be accessed through the following API.

The query server app can be accessed on heroku.

➢ api/list​: To provide with an initial list of events (titles and links) to be displayed on Open Event search results.

When an event is searched on Open Event, the query is passed on to query-server where a search is made by calling scraper.py with appending some details for better event hunting. Recent developments with Google include their event search feature. In the Google search app, event searches take over when Google detects that a user is looking for an event.

The feed from the scraper is parsed for events inside query server to generate a list containing Event Titles and Links. Each event in this list is then searched for in the database to check if it exists already. We will be using elastic search to achieve fuzzy searching for events in Open Event database as elastic search is planned for the API to be used.

One example of what we wish to achieve by implementing this type of search in the database follows. The user may search for

-Google Cloud Event Delhi
-Google Event, Delhi
-Google Cloud, Delhi
-google cloud delhi
-Google Cloud Onboard Delhi
-Google Delhi Cloud event

All these searches should match with “Google Cloud Onboard Event, Delhi” with good accuracy. After removing duplicates and events which already exist in the database from this list have been deleted, each event is rendered on search frontend of Open Event as a separate event. The user can click on any of these event, which will make a call to event collect.

Event Collect

The event collect project is developed as a separate module which has two parts

● Site specific scrapers
In its present state, event collect has scrapers for eventbrite and ticket-leap which, given a query, scrape eventbrite (and ticket-leap respectively) search results and downloads JSON files of each event using Loklak‘s API.
The scrapers can be developed in any form or any number of scrapers/scraping tools can be added as long as they are in alignment with the Open Event Import API’s data format. Writing tests for these against the concurrent API formats will take care of this. This part will be covered by using a json-validator​ to check against a pre-generated schema.

● REST APIs
The scrapers are exposed through a set of APIs, which will include, but not limited to,
➢ api/fetch-event : ​to scrape any event given the link and compose the data in a predefined JSON format which will be generated based on Open Event Import API. When this function is called on an event link, scrapers are invoked which collect event data such as event, meta, forms etc. This data will be validated against the generated JSON schema. The scraped JSON and directory structure for media files:
➢ api/export : to export all the JSON data containing event information into Open Event Server. As and when the scraping is complete, the data will be added into Open Event’s database as a new event.

How the Import works

The following graphic shows how the import works.




Let’s dive into the workflow. So as the diagram illustrates, the ‘search​’ functionality makes a call to api/list API endpoint provided by query-server which returns with events’ ‘Title’ and ‘Event Link’ from the parsed XML/JSON feed. This list is displayed as Open Event’s search results. Now the results having been displayed, the user can click on any of the events. When the user clicks on any event, the event is searched for in Open Event’s database. Two things happen now:

  • The event page loads if the event is found.
  • If the event does not already exist in the database, clicking on any event will

➢ Insert this event’s title and link in the database and get the event_id

➢ Make a call to api/fetch-event in event-collect which then invokes a site-specific scraper to fetch data about the event the user has chosen

➢ When the data is scraped, it is imported into Open Event database using the previously generated event_id. The page will be loaded using jquery ajax ​as and when the scraping is done.​When the imports are done, the search page refreshes with the new results. The Open Event Orga Server exposes a well documented REST API that can be used by external services to access the data.

Creating nested routes in Open event front end and navigating them with tabs via semantic UI – ember integration

Semantic UI is a modern development framework which helps build responsive and aesthetically beautiful layouts. While it is a really powerful framework in itself, it additionally offers seamless integrations with some of the other open source frameworks including ember js.

Open event front end is a project of fossasia organisation, which was created with the aim of decoupling the front end and the back end for the open event orga server. It is primarily based on ember JS and uses semantic UI for it’s UI.

Here we will be making a nested route /events/ with /events/live/, events/draft, events/past , events/import as it’s subroutes.

To get started with it, we simply use the ember CLI to generate the routes

$ ember generate route events

Then we go on to generate the successive sub routes as follows

$ ember generate route events/live
$ ember generate route events/past
$ ember generate route events/draft
$ ember generate route events/import

 

The router.js file should be looking like this now.

this.route('events', function() {
    this.route('live');
    this.route('draft');
    this.route('past');
    this.route('import');
  });

This means that our routes and sub routes are in place. Since we used the ember CLI to generate these routes, the template files for them would have generated automatically. Now these routes exist and we need to write the data in the templates of these routes which will get displayed to the end user.

Since the routes are nested, the content of the parent route can be made available to all the children routes via the outlet in ember js.

 

Next, we go to the template file of events/ route which is at templates/events.hbs And write the following code to create a menu and use ember integration of semantic UI link-to to link the tabs of the menu with the corresponding correct route. It will take care of selecting the appropriate data for the corresponding route and display it in the correct tab via the outlet

<.div class="row">
  <.div class="sixteen wide column">
    <.div class="ui fluid pointing secondary menu">
      {{#link-to 'events.live' class='item'}}
        {{t 'Live'}}
      {{/link-to}}
      {{#link-to 'events.draft' class='item'}}
        {{t 'Draft'}}
      {{/link-to}}
      {{#link-to 'events.past' class='item'}}
        {{t 'Past'}}
      {{/link-to}}
      {{#link-to 'events.import' class='item'}}
        {{t 'Import'}}
      {{/link-to}}
    <./div>
  <./div>
<./div>
<.div class="ui segment">
  {{outlet}}
<./div>

So finally, we start filling in the data for each of these routes. Let’s fill some dummy data at templates/events/live.hbs

<.div class="row">
  <.div class="sixteen wide column">
    <.table class="ui tablet stackable very basic table">
      <.thead>
        <.tr>
          <.th>{{t 'Name'}}<./th>
          <.th>{{t 'Date'}}<./th>
          <.th>{{t 'Roles'}}<./th>
          <.th>{{t 'Sessions'}}<./th>
          <.th>{{t 'Speakers'}}<./th>
          <.th>{{t 'Tickets'}}<./th>
          <.th>{{t 'Public URL'}}<./th>
          <.th><./th>
        <./tr>
      <./thead>
      <.tbody>
        <.tr>
          <.td>
            <.div class="ui header weight-400">
              <.img src="http://placehold.it/200x200" alt="Event logo" class="ui image">
              Sample Event
            <./div>
          <./td>
          <.td>
            March 18, 2016 - 09:30 AM
            <.br>(to)<.br>
            March 20, 2016 - 05:30 PM
          <./td>
          <.td>
            <.div class="ui ordered list">
              <.div class="item">[email protected] ({{t 'Organizer'}})<./div>
              <.div class="item">[email protected] ({{t 'Manager'}})<./div>
            <./div>
          <./td>
          <.td>
            <.div class="ui list">
              <.div class="item">{{t 'Drafts'}}: 0<./div>
              <.div class="item">{{t 'Submitted'}}: 0<./div>
              <.div class="item">{{t 'Accepted'}}: 0<./div>
              <.div class="item">{{t 'Confirmed'}}: 0<./div>
              <.div class="item">{{t 'Pending'}}: 0<./div>
              <.div class="item">{{t 'Rejected'}}: 0<./div>
            <./div>
          <./td>
          <.td>
            2
          <./td>
          <.td>
            <.div class="ui bulleted list">
              <.div class="item">{{t 'Premium'}} (12/100)<./div>
              <.div class="item">{{t 'VIP'}} (10/15)<./div>
              <.div class="item">{{t 'Normal'}} (100/200)<./div>
              <.div class="item">{{t 'Free'}} (100/500)<./div>
            <./div>
          <./td>
          <.td>
            <.div class="ui link list">
              <.a class="item" target="_blank" rel="noopener" href="http://nextgen.eventyay.com/e/ecc2001a">
                http://nextgen.eventyay.com/e/ecc2001a
              <./a>
            <./div>
          <./td>
          <.td class="center aligned">
            <.div class="ui vertical compact basic buttons">
              {{#ui-popup content=(t 'Edit event details') class='ui icon button'}}
                <.i class="edit icon"><./i>
              {{/ui-popup}}
              {{#ui-popup content=(t 'View event details') class='ui icon button'}}
                <.i class="unhide icon"><./i>
              {{/ui-popup}}
              {{#ui-popup content=(t 'Delete event') class='ui icon button'}}
                <.i class="trash outline icon"><./i>
              {{/ui-popup}}
            <./div>
          <./td>
        <./tr>
      <./tbody>
    <./table>
  <./div>
<./div>

           

Similarly we can fill the required data for each of the routes.And this is it, our nested route is ready. Here is a screenshot what you might expect.

Screenshot highlighting the tabs

Sorting language-translation in Open Event Server project using Jinja 2 dictsort.

Working on the Open Event Server project an issue about arranging language-translation listing in alphabetical order came up. To solve this issue of language listing arrangement i.e. #2817, I found the ‘d0_dictsort’ function in jinja2 to sort dictionaries. It is a defined in jinja2.filters. Python dicts are unsorted and in our web application we at times may want to order them by either their key or value. So this function comes handy.

This is what the function looks like:

do_dictsort(value, case_sensitive=False, by='key')

We can write them in three ways as:

{% for record in my_dictionary|dictsort %}
    case insensitive and sort the dict by key

{% for record in my_dictionary|dicsort(true) %}
    case sensitive and sort the dict by key

{% for record in my_dictionary|dictsort(false, 'value') %}
    sort the dict by value, normally sorted and case insensitive
  1.       The first way is easily understood that dict has been sorted by key not taking case into consideration. It is just in the same way written as dictsort(false).
  2.       Second way is basically the first being case sensitive. dictsort(true) here tells us that case is sensitive.
  3.      Third way is dictsort(false,’value’). The first parameter defines that case insensitive while second parameter defines that it is sorted by ‘value’.

The issues was to sort translation selector for the page in alphabetical order. The languages were stored in a dictionary which to change in order, I found this function very easy and useful.

Basically what we had was:

This is how the function was used in the code for the sort. Like this:

<ul class="dropdown-menu lang-list">
   {% for code in all-languages|dictsort(false,'value') %}
       <li><a  href="#" style="#969191" class="translate" id="{{ code[0] }}">{{  all_languages[code[0]] }}<>a><li>
    {% endfor %}
<ul>


Here:
{{ all_languages }} is the list which contained the languages like French, English, etc., which could be accessed with its global language code. code here(index for all_languages) is a tuple of {‘global_language_code’,’language’} (An example would be (‘fr’,’French’), so code[0] gave me the language_code.

Finally, the result:

This is one of the simple ways to sort your dictionaries.

Open Event Server: No (no-wrap) Ellipsis using jquery!

Yes, the title says it all i.e., Enabling multiple line ellipsis. This was used to solve an issue to keep Session abstract view within 200 characters (#3059) on FOSSASIA‘s Open Event Server project.

There is this one way to ellipsis a paragraph in html-css and that is by using the text-overflow property:

.div_class{
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
}’’

But the downside of this is the one line ellipis. Eg: My name is Medozonuo. I am…..

And here you might pretty much want to ellipsis after a few characters in multiple lines, given that your div space is small and you do want to wrap your paragraph. Or maybe not.

So jquery to the rescue.

There are two ways you can easily do this multiple line ellipsis:

1) Height-Ellipsis (Using the do-while loop):

//script:
if ($('.div_class').height() > 100) {
    var words = $('.div_class').html().split(/\s+/);
    words.push('...');

    do {
        words.splice(-2, 1);
        $('.div_class').html( words.join(' ') );
    } while($('.div_class').height() > 100);
}

Here, you check for the div content’s height and split the paragraph after that certain height and add a “…”, do- while making sure that the paragraphs are in multiple lines and not in one single line. But checkout for that infinite loop.

2) Length-Ellipsis (Using substring function):  

//script:
$.each($('.div_class'), function() {
        if ($(this).html().length > 100) {
               var cropped_words = $(this).html();
               cropped_words = cropped_words.substring(0, 200) + "...";
               $(this).html(cropped_words);
        }
 });

Here, you check for the length/characters rather than the height, take in the substring of the content starting from 0-th character to the 200-th character and then add in extra “…”.

This is exactly how I used it in the code.

$.each($('.short_abstract',function() {
   if ($(this).html().length > 200) {
       var  words = $(this).html();
       words = words.substring(0,200 + "...";
       $(this).html(words);
    }
});


So ellipsing paragraphs over heights and lengths can be done using jQuery likewise.

ember.js – the right choice for the Open Event Front-end

With the development of the API server for the Open Event project we needed to decide which framework to choose for the new Open Event front-end. With the plethora of javascript frameworks available, it got really difficult to decide, which one is actually the right choice. Every month a new framework arrives, and the existing ones keep actively updating themselves often. We decided to go with Ember.js. This article covers the emberJS framework and highlights its advantages over others and  demonstrates its usefulness.

EmberJS is an open-source JavaScript application front end framework for creating web applications, and uses Model-View-Controller (MVC) approach. The framework provides universal data binding. It’s focus lies on scalability.

Why is Ember JS great?

Convention over configuration – It does all the heavy lifting.

Ember JS mandates best practices, enforces naming conventions and generates the boilerplate code for the various components and routes itself. This has advantages other than uniformity. It is easier for other developers to join the project and start working right away, instead of spending hours on existing codebase to understand it, as the core structure of all ember apps is similar. To get an ember app started with the basic route, user doesn’t has to do much, ember does all the heavy lifting.

ember new my-app
ember server

After installing this is all it takes to create your app.

Ember CLI

Similar to Ruby on Rails, ember has a powerful CLI. It can be used to generate boiler plate codes for components, routes, tests and much more. Testing is possible via the CLI as well.

ember generate component my-component
ember generate route my-route
ember test

These are some of the examples which show how easy it is to manage the code via the ember CLI.

Tests.Tests.Tests.

Ember JS makes it incredibly easy to use test-first approach. Integration tests, acceptance tests, and unit tests are in built into the framework. And can be generated from the CLI itself, the documentation on them is well written and it’s really easy to customise them.

ember generate acceptance-test my-test

This is all it takes to set up the entire boiler plate for the test, which you can customise

Excellent documentation and guides

Ember JS has one of the best possible documentations available for a framework. The guides are a breeze to follow. It is highly recommended that, if starting out on ember, make the demo app from the official ember Guides. That should be enough to get familiar with ember.

Ember Guides is all you need to get started.

Ember Data

It sports one of the best implemented API data fetching capabilities. Fetching and using data in your app is a breeze. Ember comes with an inbuilt data management library Ember Data.

To generate a data model via ember CLI , all you have to do is

ember generate model my-model

Where is it being used?

Ember has a huge community and is being used all around. This article focuses on it’s salient features via the example of Open Event Orga Server project of FOSSASIA. The organizer server is primarily based on FLASK with jinja2 being used for rendering templates. At the small scale, it was efficient to have both the front end and backend of the server together, but as it grew larger in size with more refined features it became tough to keep track of all the minor edits and customizations of the front end and the code started to become complex in nature. And that gave birth to the new project Open Event Front End which is based on ember JS which will be covered in the next week.

With the orga server being converted into a fully functional API, the back end and the front end will be decoupled thereby making the code much cleaner and easy to understand for the other developers that may wish to contribute in the future. Also, since the new front end is being designed with ember JS, it’s UI will have a lot of enhanced features and enforcing uniformity across the design would be much easier with the help of components in ember. For instance, instead of making multiple copies of the same code, components are used to avoid repetition and ensure uniformity (change in one place will reflect everywhere)

<.div class="{{if isWide 'event wide ui grid row'}}">
  {{#if isWide}}
    {{#unless device.isMobile}}
      <.div class="ui card three wide computer six wide tablet column">
        <.a class="image" href="{{href-to 'public' event.identifier}}">
          {{widgets/safe-image src=(if event.large event.large event.placeholderUrl)}}
        <./a>
      <./div>
    {{/unless}}
  {{/if}}
  <.div class="ui card {{unless isWide 'event fluid' 'thirteen wide computer ten wide tablet sixteen wide mobile column'}}">
    {{#unless isWide}}
      <.a class="image" href="{{href-to 'public' event.identifier}}">
        {{widgets/safe-image src=(if event.large event.large event.placeholderUrl)}}
      <./a>
    {{/unless}}
    <.div class="main content">
      <.a class="header" href="{{href-to 'public' event.identifier}}">
        <.span>{{event.name}}<./span>
      <./a>
      <.div class="meta">
        <.span class="date">
          {{moment-format event.startTime 'ddd, MMM DD HH:mm A'}}
        <./span>
      <./div>
      <.div class="description">
        {{event.shortLocationName}}
      <./div>
    <./div>
    <.div class="extra content small text">
      <.span class="right floated">
        <.i role="button" class="share alternate link icon" {{action shareEvent event}}><./i>
      <./span>
      <.span>
        {{#if isYield}}
          {{yield}}
        {{else}}
          {{#each tags as |tag|}}
            <.a>{{tag}}<./a>
          {{/each}}
        {{/if}}
      <./span>
    <./div>
  <./div>
<./div>

This is a perfect example of the power of components in ember, this is a component for event information display in a card format which in addition to being rendered differently for various screen sizes can act differently based on passed parameters, thereby reducing the redundancy of writing separate components for the same.

Ember is a step forward towards the future of the web. With the help of Babel.js it is possible to write ES6/2015 syntax and not worry about it’s compatibility with the browsers. It will take care of it.

This is perfectly valid and will be compatible with majority of the supported browsers.

actions: {
  submit() {
    this.onValid(()=> {
    });
  }
}

 

Some references used for the blog article:

  1. https://www.codeschool.com/blog/2015/10/26/7-reasons-to-use-ember-js/
  2. https://www.quora.com/What-are-the-advantages-of-using-Ember-js
  3. Official Ember Guides: https://guides.emberjs.com

 
This page/product/etc is unaffiliated with the Ember project. Ember is a trademark of Tilde Inc

DetachedInstanceError: dealing with Celery, Flask’s app context and SQLAlchemy

In the open event server project, we had chosen to go with celery for async background tasks. From the official website,

What is celery?

Celery is an asynchronous task queue/job queue based on distributed message passing.

What are tasks?

The execution units, called tasks, are executed concurrently on a single or more worker servers using multiprocessing.

After the tasks had been set up, an error constantly came up whenever a task was called

The error was:

DetachedInstanceError: Instance <User at 0x7f358a4e9550> is not bound to a Session; attribute refresh operation cannot proceed

The above error usually occurs when you try to access the session object after it has been closed. It may have been closed by an explicit session.close() call or after committing the session with session.commit().

The celery tasks in question were performing some database operations. So the first thought was that maybe these operations might be causing the error. To test this theory, the celery task was changed to :

@celery.task(name='lorem.ipsum')
def lorem_ipsum():
    pass

But sadly, the error still remained. This proves that the celery task was just fine and the session was being closed whenever the celery task was called. The method in which the celery task was being called was of the following form:

def restore_session(session_id):
    session = DataGetter.get_session(session_id)
    session.deleted_at = None
    lorem_ipsum.delay()
    save_to_db(session, "Session restored from Trash")
    update_version(session.event_id, False, 'sessions_ver')


In our app, the app_context was not being passed whenever a celery task was initiated. Thus, the celery task, whenever called, closed the previous app_context eventually closing the session along with it. The solution to this error would be to follow the pattern as suggested on http://flask.pocoo.org/docs/0.12/patterns/celery/.

def make_celery(app):
    celery = Celery(app.import_name, broker=app.config['CELERY_BROKER_URL'])
    celery.conf.update(app.config)
    task_base = celery.Task

    class ContextTask(task_base):
        abstract = True

        def __call__(self, *args, **kwargs):
            if current_app.config['TESTING']:
                with app.test_request_context():
                    return task_base.__call__(self, *args, **kwargs)
            with app.app_context():
                return task_base.__call__(self, *args, **kwargs)

    celery.Task = ContextTask
    return celery

celery = make_celery(current_app)


The __call__ method ensures that celery task is provided with proper app context to work with.