Implementing an Interface for Reading Configuration from a YAML File for Yaydoc

Yaydoc reads configuration specified in a YAML file to set various options during the build process. This allows users to customize various properties of the build process. The current implementation for this was very basic. Basically it uses a pyYAML, a yaml parser for python to read the file and convert it to a python dictionary. From the dictionary we extracted values for various properties and converting them to strings using various heuristics such as converting True to ”true”, False to ”false”, a list to comma separated string and None to an empty string. Finally, we exported variables with those values. Recently the entire code for this was rewritten using object-oriented paradigm. The motivation for this came from the fact that the implementation lacked certain features and also required some refactoring for long term readability. In the following paragraph, I have discussed the new implementation. Firstly a Configuration class was created which basically wraps around a dictionary and provide certain utility methods. The primary difference is that the Configuration class allows dotted key access. This means that you can use the following syntax to access nested keys. theme = conf[‘build.theme.name’] The class provides another method connect which is used to connect environment variables with configuration values. This method also takes a dotted key but provides an extension on top of that to handle the case when a certain option can take multiple values. For example, option: my_option Or, option: - my_option1 - my_option2 To indicate that a certain config is of this type, you can specify a “@” character at the end of the key. Anything after the “@” character is assumed to be an attribute of each element within the list. Let’s see an example of this whole process. build: subproject: - url: <url1> source: “doc” - url: <url2> Now to extract all urls from the above file, we’d need to do the following config.connect(‘SUBPROJECT_URLS’, ‘build.subproject@url’) To extract sources, we’ll also use the default parameter as the source option is optional. config.connect(‘SUBPROJECT_SOURCES’, build.subproject@source’, default=’docs’) Finally, The Configuration object also provides a getenv method which reads all connection and serializes values to string according to the previously described heuristics. It then returns a dictionary of all environment variables which must be set. Resources TutorialsPoint - Python Dictionary Official Yaml website

Continue ReadingImplementing an Interface for Reading Configuration from a YAML File for Yaydoc

Specifying Configurations for Yaydoc with .yaydoc.yml

Like many continuous integration services, Yaydoc also reads configurations from a YAML file. To get started with Yaydoc the first step is to create a file named .yaydoc.yml and specify the required options. Recently the yaydoc team finalized the layout of the file. You can still expect some changes as the projects continues to grow on but they shall not be major ones. Let me give you an outline of the entire layout before describing each in detail. Overall the file is divided into four sections. metadata build publish extras For the first three sections, their intention is pretty clear from their names. The last section extras is meant to hold settings related to integration to external services. Following is a description of config options under the metadata section and it’s example usage. Key Description Default author The author of the repository. It is used to construct the copyright text. user/organization projectname The name of the project. This would be displayed on the generated documentation. Name of the repository version The version of the project. This would be displayed alongside the project name Current UTC date debug If true, the logs generated would be a little more verbose. Can be one of true|false. true inline_math Whether inline math should be enabled. This only affects markdown documents. false autoindex This section contains various settings to control the behavior of the auto generated index. Use this to customize the starting page while having the benefit of not having to specify a manual index. - metadata: author: FOSSASIA projectname: Yaydoc version: development debug: true inline_math: false Following is a description of config options under the build section and it’s example usage. Key Description Default theme The theme which should be used to build the documentation. The attribute name can be one of the built-in themes or any custom sphinx theme from PyPI. Note that for PyPI themes, you need to specify the distribution name and not the package name. It also provides an attribute options to control specific theme options sphinx_fossasia_theme source This is the path which would be scanned for markdown and reST files. Also any static content such as images referenced from embedded html in markdown should be placed under a _static directory inside source. Note that the README would always be included in the starting page irrespective of source from the auto-generated index docs/ logo The path to an image to be used as logo for the project. The path specified should be relative to the source directory. - markdown_flavour The markdown flavour which should be used to parse the markdown documents. Possible values for this are markdown, markdown_strict, markdown_phpextra, markdown_github, markdown_mmd and commonmark. Note that currently this option is only used for parsing any included documents using the mdinclude directive and it’s unlikely to change soon. markdown_github mock Any python modules or packages which should be mocked. This only makes sense if the project is in python and uses autodoc has C dependencies. - autoapi If enabled, Yaydoc will…

Continue ReadingSpecifying Configurations for Yaydoc with .yaydoc.yml

Integrating Swagger with SUSI Server

The goal of Swagger is to define a standard interface for REST APIs which allows humans to understand the capabilities of the APIs without access to source code or documentation. SUSI Server is now completely API centric. As more and more people make their own bots using SUSI Server they will be needing documentation for the APIs. We can use swagger so that without looking at the javadocs or documentation people can consume the REST APIs. In this I post will walk you through the steps to integrate Swagger with SUSI Server which is running on Jetty. Add the following dependencies to build.gradle file. These add Swagger Annotations, Swagger Models, Swagger UI and Glassfish containers. compile group: 'io.swagger', name: 'swagger-annotations',version: swaggerVersion compile group: 'io.swagger', name: 'swagger-core', version: swaggerVersion compile group: 'io.swagger', name: 'swagger-jersey2-jaxrs', version: swaggerVersion compile group: 'io.swagger', name: 'swagger-models', version: swaggerVersion compile group: 'org.glassfish.jersey.core', name: 'jersey-server', version: versionJersey compile group: 'org.glassfish.jersey.containers', name: 'jersey-container-servlet-core', version: versionJersey compile group: 'org.webjars', name: 'swagger-ui', version: '2.1.4' Now SusiServer.class is the main file which initializes all the servlets and server handlers. Here, we need to tell the SusiServer to look for the swagger annotations and use them to build the documentation. In the main function, before starting the server we set the serverHandlers from setServerHandler function. public static void main(String[] args) throws Exception { // init the http server try { setupHttpServer(httpPort, httpsPort); } catch (Exception e) { Log.getLog().warn(e.getMessage()); System.exit(-1); } setServerHandler(dataFile); SusiServer.server.start(); SusiServer.caretaker = new Caretaker(); SusiServer.caretaker.start(); Now, we will modify the setServetHandler function for registering the Swagger Handler. There are already 2 handlers so I used a handerCollection object, so that we can give multiple handlers to handerCollection and set the serverHandler as handerCollection. private static void setServerHandler(File dataFile){ ServletContextHandler servletHandler = new ServletContextHandler(); handlerCollection.addHandler(ipaccess); ContextHandlerCollection contexts = new ContextHandlerCollection(); ServletContainer sc = new ServletContainer(resourceConfig); ServletHolder holder = new ServletHolder(sc); servletHandler.addServlet(holder, "/docs/*"); handlerCollection.addHandler(contexts); SusiServer.server.setHandler(handlerCollection); } servletHandler.addServlet(holder, "/docs/*"), this line in the setServerHandler method sets the default swagger.json path to api.susi.ai/docs/swagger.json This is all the basic setup to initialize swagger and now we need to modify our API endpoints and set annotations for the base URL and parameters. Now I will discuss how to add swagger annotations to our Servlets. For the demo, I will use GetAllUsers.class file, which returns the list of all users to Admins. @Path("/aaa") @Api(value = "/AAA") @Produces({"application/json"}) public class GetAllUsers extends AbstractAPIHandler implements APIHandler { Just before the class starts we will add the Path of the API endpoint and the result it produces. In this case, GetAllUsers returns JSON and is a part of aaa group. @GET @Path("/getAllUsers.json") @ApiOperation(value = "Get All Users Registered on SUSI.AI", notes = "This API Endpoint returns the list of all users registered on SUSI Server", responseContainer = "Object") @ApiResponses(value = { @ApiResponse(code = 200, message = "Success"), @ApiResponse(code = 401, message = "Base user role not sufficient") }) Inside the class, we will declare the path of the API endpoint. A description of this endpoint and the sample response code. In this file,…

Continue ReadingIntegrating Swagger with SUSI Server