Getting SUSI Skill at a Commit ID

Susi Skill CMS is a web app to edit and create new skills. We use Git for storing different versions of Susi Skills. So what if we want to roll back to a previous version of the skill? To implement this feature in Susi Skill CMS, we needed an API endpoint which accepts the name of the skill and the commit ID and returns the file at that commit ID. In this blog post I will tell about making an API endpoint which works similar to git show. First we will accept all the request parameters from the GET request. String model_name = call.get("model", "general"); String group_name = call.get("group", "Knowledge"); String language_name = call.get("language", "en"); String skill_name = call.get("skill", "wikipedia"); String commitID = call.get("commitID", null); In this we get the model name, category, language name, skill name and the commit ID. The above 4 parameters are used to make a file path that is used to find the location of the skill in the Susi Skill Data repository. This servlet need CommitID to work and if commit ID is not given in the request parameters then we send an error message saying that the commit id is null and stop the servlet execution. Repository repository = DAO.getRepository(); ObjectId CommitIdObject = repository.resolve(commitID); Then we get the git repository of the skill from the DAO and initialize the repository object. From the commitID that we got in the request parameters we create a CommitIdObject. (RevWalk revWalk = new RevWalk(repository)) { RevCommit commit = revWalk.parseCommit(CommitIdObject); RevTree tree = commit.getTree(); Now using commit's tree, we will find the find the path and get the tree of the commit. From the TreeWalk in the repository we will set a filter to find a file. This searches recursively for the files inside all the folders. revWalk = new RevWalk(repository)) { try (TreeWalk treeWalk = new TreeWalk(repository)) { treeWalk.addTree(tree); treeWalk.setRecursive(true); treeWalk.setFilter(PathFilter.create(path)); if (!treeWalk.next()) { throw new IllegalStateException("Did not find expected file"); } If the TreeWalk reaches to an end and does not find the specified skill path then it returns anIllegal State Exception with an message saying did not found the file on that commit ID. ObjectId objectId = treeWalk.getObjectId(0); ObjectLoader loader = repository.open(objectId); OutputStream output = new OutputStream(); loader.copyTo(output); And then one can the loader to read the file. From the treeWalk we get the object and create an output stream to copy the file content in it. After that we create the JSON and put the OutputStream object as as String in it. json.put("file",output); This Servlet can be seen working api.susi.ai: http://api.susi.ai/cms/getFileAtCommitID.json?model=general&group=knowledge&language=en&skill=bitcoin&commitID=214791f55c19f24d7744364495541b685539a4ee Resources JGit Documentation: https://eclipse.org/jgit/documentation/ JGit User Guide: http://wiki.eclipse.org/JGit/User_Guide JGit Repository access: http://www.codeaffine.com/2014/09/22/access-git-repository-with-jgit/ JGit Github: https://github.com/eclipse/jgit

Continue ReadingGetting SUSI Skill at a Commit ID

Implementing revisioning feature in Open Event

{ Repost from my personal blog @ https://blog.codezero.xyz/implementing-revisioning-feature-in-open-event } As I said in my previous blog post about Adding revisioning to SQLAlchemy Models, In an application like Open Event, where a single piece of information can be edited by multiple users, it's always good to know who changed what. One should also be able to revert to a previous version if needed. Let's have a quick run through on how we can enable SQLAlchemy-Continuum on our project. Install the library SQLAlchemy-Continuum with pip Add __versioned__ = {} to all the models that need to be versioned. Call make_versioned() before the models are defined Call configure_mappers from SQLAlchemy after declaring all the models. Example: import sqlalchemy as sa from sqlalchemy_continuum import make_versioned # Must be called before defining all the models make_versioned() class Event(Base): __tablename__ = 'events' __versioned__ = {} # Must be added to all models that are to be versioned id = sa.Column(sa.Integer, primary_key=True, autoincrement=True) name = sa.Column(sa.String) start_time = sa.Column(db.DateTime, nullable=False) end_time = sa.Column(db.DateTime, nullable=False) description = db.Column(db.Text) schedule_published_on = db.Column(db.DateTime) # Must be called after defining all the models sa.orm.configure_mappers() We have SQLAlchemy-Continuum enabled now. You can do all the read/write operations as usual. (No change there). Now, for the part where we give the users an option to view/restore revisions. The inspiration for this, comes from wordpress's wonderful revisioning functionality. The layout is well designed. The differences are shown in an easy-to-read form. The slider on top makes it intuitive to move b/w revisions. We have a Restore this Revision button on the top-right to switch to that revision. A similar layout is what we would like to achieve in Open Event. A slider to switch b/w sessions A pop-over infobox on the slider to show who made that change A button to switch to that selected revision. The colored-differences shown in side-by-side manner. To make all this a bit easier, SQLAlchemy-Continuum provides us with some nifty methods. count_versions is a method that allows us to know the number of revisions a record has. event = session.query(Event).get(1) count = count_versions(event) # number of versions of that event Next one is pretty cool. All the version objects have a property called as changeset which holds a dict of changed fields in that version. event = Event(name=u'FOSSASIA 2016', description=u'FOSS Conference in Asia') session.add(article) session.commit(article) version = event.versions[0] # first version version.changeset # { # 'id': [None, 1], # 'name': [None, u'FOSSASIA 2016'], # 'description': [None, u'FOSS Conference in Asia'] # } event.name = u'FOSSASIA 2017' session.commit() version = article.versions[1] # second version version.changeset # { # 'name': [u'FOSSASIA 2016', u'FOSSASIA 2017'], # } As you can see, dict holds the fields that changed and the content the changed (before and after). And this is what we'll be using for generating those pretty diffs that the guys and girls over at wordpress.com have done. And for this we'll be using two things. A library named diff-match-patch. It is a library from Google which offers robust algorithms to perform the…

Continue ReadingImplementing revisioning feature in Open Event