One of the important features while creating server side code for some website/ web application is sending emails. But how do we send emails. There are different ways and packages using which you can setup SMTP ports and send emails. So why specifically sendgrid? Because along with SMTP relay, sendgrid also allows you to send emails using its Web API which makes work much easier. Here, we will discuss about using Web API.
For the Open Event android app we were using retofit 1.9 with an okhttp stack plus a gson parser but recently retrofit 2.0 was released and it was a major update in the sense that it a lot of things have been changed. For starters, you don’t have to declare synchronous and asynchronous requests upfront and you can just decide that while executing. The code for that will look something like this. This is how we define our request methods in our api service import retrofit.Call; public interface APIService { @POST(“/list”) Call<Repo> loadRepo(); } Now if we want to make a synchronous request, we can make it like Call<Repo> call = service.loadRepo(); Repo repo = call.execute(); and for an asynchronous request, we can call enqueue() Call<Repo> call = service.loadRepo(); call.enqueue(new Callback<Repo>() { @Override public void onResponse(Response<Repo> response) { // Get result Repo from response.body() } @Override public void onFailure(Throwable t) { } }); And another thing that changed in the async call throws a throwable on failure, so essentially the RetrofitError class is gone and since we were using that in our app, we had to modify the whole error handling in the app, basically from the grounds up. So, when we decided to move to retrofit 2 after the stable version was released, we had to change a lot of code and the main part that was affected was the error handling. So, replacing the retrofitError class, I used the throwable directly to retrieve the error type something like this if (error.getThrowable() instanceof IOException) { errorType = “Timeout”; errorDesc = String.valueOf(error.getThrowable().getCause()); } else if (error.getThrowable() instanceof IllegalStateException) { errorType = “ConversionError”; errorDesc = String.valueOf(error.getThrowable().getCause()); } else { errorType = “Other Error”; errorDesc = String.valueOf(error.getThrowable().getLocalizedMessage()); } This was ofcourse for all failure events. And to handle all response events I compared the HTTP status codes and displayed the errors : Integer statusCode = response.getStatusCode(); if (statusCode.equals(404)) { // Show Errors in a dialog showErrorDialog(“HTTP Error”, statusCode + “Api Not Found”); } This is how we can compare other HTTP errors in retrofit and assign the correct status accordingly. I personally think that this is a better implementation than Retrofit 1.9 and the RetrofitError was a bit tedious to work with. It wasn’t very thought of before implementation because it was not easy to tell what kind of error exactly occured. With Response codes, one can see what are the exact error one faces and can gracefully handle these errors.
{ Repost from my personal blog @ https://blog.codezero.xyz/adding-revisioning-to-sqlalchemy-models } In an application like Open Event, where a single piece of information can be edited by multiple users, it's always good to know who changed what. One should also be able to revert to a previous version if needed. That is where revisioning comes into picture. We use SQLAlchemy as the database toolkit and ORM. So, we wanted a versioning tool that would work well with our existing setup. That's when I came across SQLAlchemy-Continuum - a versioning extension for SQLAlchemy. Installation The installation of the module is just like any other python library. (don't forget to add it to your requirements.txt file, if you have one) pip install SQLAlchemy-Continuum Setup Now, it's time to enable SQLAlchemy-Continuum for the required models. Let's consider an Event model. import sqlalchemy as sa class Event(Base): __tablename__ = 'events' id = sa.Column(sa.Integer, primary_key=True, autoincrement=True) name = sa.Column(sa.String) start_time = sa.Column(db.DateTime, nullable=False) end_time = sa.Column(db.DateTime, nullable=False) description = db.Column(db.Text) schedule_published_on = db.Column(db.DateTime) We need to do three things to enable SQLAlchemy-Continuum. Call make_versioned() before the model(s) is/are defined. Add __versioned__ = {} to all the models that we want to be versioned Call configure_mappers from SQLAlchemy after declaring all the models. import sqlalchemy as sa from sqlalchemy_continuum import make_versioned # Must be called before defining all the models make_versioned() class Event(Base): __tablename__ = 'events' __versioned__ = {} # Must be added to all models that are to be versioned id = sa.Column(sa.Integer, primary_key=True, autoincrement=True) name = sa.Column(sa.String) start_time = sa.Column(db.DateTime, nullable=False) end_time = sa.Column(db.DateTime, nullable=False) description = db.Column(db.Text) schedule_published_on = db.Column(db.DateTime) # Must be called after defining all the models sa.orm.configure_mappers() SQLAlchemy-Continuum creates two tables: events_version which stores the version history for the Event model linked to the transaction table via a foreign key transaction which stores information about each transaction (like the user who performed the transaction, transaction datetime, etc) SQLAlchemy-Continuum also adds listeners to Event to record all create, update, delete actions. Usage All the CRUD (Create, read, update, delete) operations can be done as usual. SQLAlchemy-Continuum takes care of creating a version record for each CUD operation. The versions can be easily accessed using the versions property that is now available in the Event model. event = Event(name="FOSSASIA 2017", description="Open source conference in asia") session.add(event) # Event added to transaction session.commit() # Transaction comitted and event recored created # This would have created the first version record which can be accessed event.versions[0].name == "FOSSASIA 2017" # Lets make some changes to the recored. event.name = "FOSSASIA 2016" session.add(event) session.commit() # This would have created the second version record which can be accessed event.versions[1].name == "FOSSASIA 2016" # The first version record still remains and can be accessed event.versions[0].name == "FOSSASIA 2017" So, that's how basic versioning can be implemented in SQLAlchemy using SQLAlchemy-Continuum.
In this post, I will talk about how we can use the Amazon S3 (Simple Storage Service) for cloud storage. As you may know, S3 is a no-fuss, super easy cloud storage service based on the IaaS model. There is no limit on the size of file or the amount of files you can keep on S3, you are only charged for the amount of bandwidth you use. This makes S3 very popular among enterprises of all sizes and individuals. Now let’s see how to use S3 in Python. Luckily we have a very nice library called Boto for it. Boto is a library developed by the AWS team to provide a Python SDK for the amazon web services. Using it is very simple and straight-forward. Here is a basic example of uploading a file on S3 using Boto - import boto from boto.s3.key import Key # connect to the bucket conn = boto.connect_s3(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY) bucket = conn.get_bucket(BUCKET_NAME) # set the key key = 'key/for/file' file = '/full/path/to/file' # create a key to keep track of our file in the storage k = Key(bucket) k.key = key k.set_contents_from_filename(file) The above example uploads a file to s3 bucket BUCKET_NAME. Buckets are containers which store data. The key here is the unique key for an item in the bucket. Every item in the bucket is identified by a unique key assigned to it. The file can be downloaded from the urlBUCKET_NAME.s3.amazonaws.com/{key}. It is therefore essential to choose the key name smartly so that you don’t end up overwriting an existing item on the server. In the Open Event project, I thought of a scheme that will allows us to avoid conflicts. It relies on using IDs of items for distinguishing them and goes as follows - When uploading user avatar, key should be ‘users/{userId}/avatar’ When uploading event logo, key should be ‘events/{eventId}/logo’ When uploading audio of session, key should be ‘events/{eventId}/sessions/{sessionId}/audio’ Note that to store user ‘avatar’, I am setting the key as /avatar and not /avatar.extension. This is because if user uploads pictures in different formats, we will end up storing different copies of avatars for the same user. This is nice but it’s limitation is that downloading file from the url will give the file without an extension. So to solve this issue, we can use the Content-Disposition header. k.set_contents_from_filename( file, headers={ 'Content-Disposition': 'attachment; filename=filename.extension' } ) So now when someone tries to download the file from that link, they will get the file with an extension instead of a no-extension “Choose what you want to do” file. This covers up the basics of using S3 for your Python project. You may explore Boto’s S3 documentation to find other interesting functions like deleting a folder, copy one folder to another and so. Also don’t forget to have a look at the awesome documentation we wrote for the Open Event project. It provides a more pictorial and detailed guide on how to setup S3 for your project. {{ Repost from my…
So while making a sessions schedule for the open event app, I wanted to separate the sessions on the basis of the days they are scheduled for to improve the visual clarity. So to do this I had various approaches like add a filter to separate by date or add checkboxes to show only checked dates but I though they’d look ugly. Instead the best option was to add tabs in a fragment with a viewpager to scroll within them : It looks appealing, has simple and clean UI, easier to implement with the new design library. So, naturally I opted for using the Tablayout from the design Library. Earlier, when the Support design library was not introduce, it was really a tedious job to add it to our app since we had to extend Listeners to check for tab changes and we had to manually open fragments when a tab was selected or unselected or even when it was reselected. Essentially this meant a lot of errors and memory leaks. In the design library we essentially need to add tablayout and a viewpager to our layout like this : <android.support.design.widget.AppBarLayout xmlns:android="http://schemas.android.com/apk/res/android" xmlns:app="http://schemas.android.com/apk/res-auto" android:layout_width="match_parent" android:layout_height="match_parent"> <android.support.design.widget.TabLayout android:id="@+id/tabLayout" android:layout_width="match_parent" android:layout_height="wrap_content" app:tabGravity="fill" app:tabMode="fixed" app:theme="@style/ThemeOverlay.AppCompat.Dark.ActionBar" /> <android.support.v4.view.ViewPager android:id="@+id/viewpager" android:layout_width="match_parent" android:layout_height="wrap_content" android:background="@android:color/white" /> </android.support.design.widget.AppBarLayout> Next in our activity/ fragment, we can just inflate this view and create an adapter for the viewpager extending a FragmentPagerAdapter: public class OurAdapter extends FragmentPagerAdapter { private final List<Fragment> mFragmentList = new ArrayList<>(); private final List<String> mFragmentTitleList = new ArrayList<>(); public ScheduleViewPagerAdapter(FragmentManager manager) { super(manager); } @Override public Fragment getItem(int position) { return mFragmentList.get(position); } @Override public int getCount() { return mFragmentList.size(); } public void addFragment(Fragment fragment, String title, int day) { mFragmentList.add(fragment); mFragmentTitleList.add(title); } @Override public CharSequence getPageTitle(int position) { return mFragmentTitleList.get(position); } } Now I had to make dynamic number of tabs, since I wanted the app to customisable on the number of days listed in the json downloaded from the server. So, I made some changes in the traditional code. This is what we do in our activity/fragment’s onCreate/OnCreatView : viewPager = (ViewPager) view.findViewById(R.id.viewpager); for (int i = 0; i < daysofEvent; i++) { adapter.addFragment(new DayScheduleFragment(),title, dayNo); } viewPager.setAdapter(adapter); scheduleTabLayout = (TabLayout) view.findViewById(R.id.tabLayout); scheduleTabLayout.setupWithViewPager(viewPager); This is it. Now we have a basic working tablayout in a viewpager. This also has the capability to change according to the number of days specified in the json we have written. Earlier without the design library, we would have to even add switch cases in the FragmentPagerAdapter like this : public class OurAdapter extends FragmentPagerAdapter { public TabsPagerAdapter(FragmentManager fm) { super(fm); } @Override public Fragment getItem(int index) { switch (index) { case 0: return new FirstFragment(); case 1: return new SecondFragment(); case 2: return new ThirdFragment(); } return null; } Then we would have to override methods to listen to activities in tabs : @Override public void onTabReselected(Tab tab, FragmentTransaction ft) { } @Override public void onTabSelected(Tab tab, FragmentTransaction ft) { // on tab selected // show respected fragment view viewPager.setCurrentItem(tab.getPosition()); } @Override public void…
All the large scale applications require a permissions system. Thus we also implemented a permissions system in our open-event organization server. It consists of certain pre-decided roles: Super-Admin Admin Organizer Co organizer Track organizer Anonymous user Now we had to decide the permissions which each role would have. Hence we created a documentation regarding what URLs can be accessed by each role. We developed a list of services which the roles could use their permissions to access: Tracks Microlocations Speakers Sessions Sponsors Thus the final step was to implement the permissions system to the appropriate views or URLs. Here comes the power of Flask decorators . I created a individual decorators @is_organizer, @is_admin, @is_super_admin etc... to check the respective roles. I created one main decorator @can_access to see whether the role can access the particular URL or view function So in the above decorator I have simply take in the url and check whether it has 'create', 'edit' or 'delete' words in it. Depending on that the control goes in the particular IF statement. Now once it is decided what operation is being performed it checks what service is being accessed by the user. For example: if the operation is edit then it will check whether the service being edited is an event, session, sponsor etc... Similar checks are performed by each operation. A check is performed of the request.url to see whether the string for that service is present in it. After it knows what service is being accessed its just a matter of using the CRUD functions of user table to check if the role accessing the resource has the requested permission using the functions: user.can_create() user.can_read() user.can_update() user.can_delete() After this its just a matter of adding the decorator to each of the view functions and the system is implemented. :)
This week I have started testing my own code with Jasmine-Node. Testing seems beneficial only when we do it with a right approach. I have gone through various articles and find out Jasmine as one of the best frameworks for testing Javascript applications. For specifically NodeJS, a better option will be to use Jasmine-Node. How to start with Jasmine-Node ? For a quick-start, you need to have Jasmine-Node installed with npm. npm install jasmine-node -g After the installation, make a folder say spec/ in the root directory of the project. This will contain all the tests for the node app. We can include the command to run the tests in package.json file of the project. ./node_modules/.bin/jasmine-node spec Like here it is included with the key tester. Now, we are ready to write our first test file. So, in the Open-event-webapp spec/ folder I have created a file serverSpec.js. The file will test a basic function written in fold.js which is the file that includes all the functions of Open-event-webapp. //fold.js 'use strict'; var exports = module.exports = {}; const fs = require('fs-extra'); const moment = require('moment'); const handlebars = require('handlebars'); const async = require('async'); const archiver = require('archiver'); //function to test function speakerNameWithOrg(speaker) { return speaker.organisation ? `${speaker.name} (${speaker.organisation})` : speaker.name; } To test the above function, I have written the test like this //serverSpec.js 'use strict' const request = require("request"); const fold = require(__dirname +'/../src/generator/backend/fold.js'); const port = process.env.PORT || 5000; const base_url = "http://localhost:"+ port +"/" describe("Open Event Server", function() { describe("GET /", function() { it("returns status code 200", function(done) { request.get(base_url, function(error, response, body) { expect(response.statusCode).toBe(200); done(); }); }); }); }); describe("Testing Fold.js",function() { describe("Testing SpeakerWithOrg function",function(){ it("it will test speakername", function(){ let speaker= { name: "speakername", } expect(fold.speakerNameWithOrg(speaker)).toEqual("speakername"); }); it("it will test speakername and organisation", function(){ let speakerWithOrg= { name:"speakername", organisation:"Organisation", } expect(fold.speakerNameWithOrg(speakerWithOrg)).toEqual("speakername (Organisation)"); }); }); }); There are two describe blocks here one to test if the server is running properly and the second block includes two 'it' statements which are used to test the function behavior. The describe blocks are nested which is a nice feature provided by Node-Jasmine. To run the above test in Open-event-webapp , use the command: 1. npm run start 2. npm run tester On success, The output will be something like this Other Sources Jasmine Presentation HTML goodies
It's not typical team. We don't meet face-to-face each other on daily basis, one trouble is that we have to work from different time zones, and we have different operating system, the way of working, or even culture. But we are FOSSASIA Open Event team of six amazing developers so it does not discourage us to achieve our goals. But even if we experience all these things we have to learn how to work together successfully. We are trying, and I think it is becoming better and better every day, the Agile methodology. So first of all, before coding started, we had prepared user stories in issues. Every issue had been very well documented, and we had divided each issue to smaller ones. It brings us a lot of benefits, because it doesn't matter where you work. If sub issue is empty you can take it. Due to the fact that issue is very well documented you don't need to even ask someone what to do. This case saves a lot of time. I know that writing clear issues is very boring and It could seemed not to be beneficial, but it's. We are still trying to improve our performance and we are looking for new opportunities where and how we can improve our work, without lost quality of our development process. Then, we have one week sprint (milestones). It helps us to control our work and we know where are we, what have we done, and what is still to do. We know if our speed is enough or we have to move faster. And it also enables us to see our progress. Moreover, we have daily scrums. We answer three following questions - what did you do yesterday, what do you plan to do today and what is preventing you from achieving your goals. Furthermore, we follow continues integration system. We push code to our common Open Event repository on daily basis. It helps us to control our code and fix bugs quickly. I am sure we will continue along this path, and successfully finish to code an amazing Open Event web app.
This post discusses about the Organizer Server Permissions System and how it has been implemented using database models. The Organizer Server Permissions System includes Roles, and Services that these roles can access. Roles can broadly be classified into Event-specific Roles and System-wide Roles. System-Wide Roles System-wide roles can be considered a part of the staff maintaining the platform. We define two such roles: Admin and Super Admin. The Super Admin is the highest level user with permissions to access system logs, manage event-specific roles, etc. System-wide roles are the easiest to implement. Since they're directly related to a user we can add them as class variables in the User models. class User(db.Model): # other stuff is_admin = db.Column(db.Boolean, default=False) is_super_admin = db.Column(db.Boolean, default=False) @property def is_staff(self): return self.is_admin or self.is_super_admin Staff groups Admin and Super Admin roles. So to check if a user is a part of the staff (an admin or a super admin), the is_staff property can directly be used. user.is_staff Event-Specific Roles An Event itself can contain many entities like Tracks, Sponsors, Sessions, etc. Our goal was to define permissions for Roles to access these entities. We grouped these entities and put them under "Services". Services are nothing but database models associated with an event, that need to have restricted access for the Roles. We define the following Services for an Event: Track Session Speaker Sponsor Microlocation Each of these services can either be created, read, updated or deleted. And depending on the Role a user has been assigned for a particular event, he or she can perform such operations. There are four Event-Specific Roles: Organizer Co-organizer Track Organizer Moderator As soon as the user creates an event, he is assigned the role of an Organizer, giving him access to all the services for that event. An Organizer can perform any operation on any of the services. A Co-organizer also has access to all the services but can only update them. A Track Organizer can just read and update already created Tracks. The Moderator can only read Tracks. Although the initial distribution of permissions is kept as above, the Super Admin can (has permissions to) edit them later. To implement permissions for Event specific roles, three new database models were required: Role, Service and Permission. Role and Service would contain the above mentioned Roles and Services respectively. Permission would contain a Role column, a Service column and four other columns specifying what operation (create/read/update/delete) that Role is allowed to perform on the Service. The final objective was to define these methods for the User class: user.can_create(service, event_id) user.can_read(service, event_id) user.can_update(service, event_id) user.can_delete(service, event_id) Before this we needed a table specifying what Event Roles have been created for an event, and which users have been assigned these roles. The `UsersEventsRoles` model maintained this relationship. We also needed to check if a user has been assigned a particular role for an event. For this I created a method for each of the roles. # Event-specific Roles ORGANIZER = 'organizer'…
An important feature of websites these days with the advancement of smartphones is being responsive with device size. We nowadays not only worry about the various width of laptop or desktop, which don’t vary by a huge amount but also need to worry about tablets and phones which have a much lesser width. The website’s UI should not break and should be as easy to use on phones as it is on desktops/laptops. Using frameworks like bootstraps, Semantic-UI solves this problem to a large extent. But what if we need to modify certain parts by our own in case of mobile devices? How do we do that?
You must be logged in to post a comment.