HTTP Tunnelling

While developing the nextgen Open Event API Server, one has to let co-developers review the changes made for each new feature added or any bug fix made. One way to do this is to fetch code from pull request head and setup a local deployment from the PR branch. This sometimes causes issues in database due to faulty migrations, if one is not careful. One other way, which is discussed in the blog post here, is using ngrok.

What is ngrok?
Ngrok lets you create a secure tunnel to your localhost deployment over a public URL. This particular feature comes in handy when you have changes, say along with a populated database, to demo to someone else over a public url, which can be shared between as many people. It saves you from deploying the change branch over again and saves you from hassles of database migration issues.

How to setup and run ngrok?

To run ngrok, you need to download it from the ngrok website. The download page can be found here.

Once you have the zip installed, you’ll need to unzip it. On Linux or MacOS, run this in the terminal:

$ unzip /path/to/ngrok.zip

To expose the web server running on your local machine, run the following from inside the directory where you have unzipped ngrok:

./ngrok http 80

This syntax breakdowns to :
ngrok :: terminal command

http :: protocol of the server that is to be tunneled

( ngrok also lets you open and run TCP and TLS tunnels)

80 :: port on which the tunnel is to be run

( If you are not sure of the port on which your server is running, it might probably be 80 – the default for HTTP)

The Open Event API server runs on port 5000 and it provided HTTP API, so the command we’ll use here is

./ngrok http 5000

Once you run this command, ngrok opens its UI in the terminal itself. This will contain the public url of your tunnel along with other stats related to the requests being made and traffic on localhost.

Starting ngrok:

Public URL updated:

ngrok also offers a web interface where you can see the requests and other data which is shown in the terminal. For this go to http://localhost:4040/inspect/http. This web interface inspects and records each request made so that you can replay the requests again for debugging or cross-checking metrics. This feature can be turned off by passing an argument, so that the requests are not recorded anymore. While running a production server, it can help to both maintain security for the requests and also reduce request handling times when scaling. To read more about advanced options, please read the ngrok documentation.

Running Open Event API server on the public URL:
Since now we have localhost:5000 tunnelled over a public url, we’ll use that to make requests to the API server.

A GET request for /v1/events :

The request made to the public URL, which in this case here is:

http://9a5ac170.ngrok.io is equivalent to this url: http://localhost:5000  running on my local setup of the Open Event API Server. When the request is made, the EventList class is used and ResourceList class’ method which is build for the url endpoint ‘event_list’ is called. This returns a list of events from the current database which is being used on my server, thus my local database.

A DELETE request for /v1/events/1

In a similar fashion, when this request is made, event_id is parsed from view_kwargs and the following equivalent request is made: DELETE http://localhost:5000/v1/events/1 which deletes the  event with id = 1 and returns a success object as shown in the screenshot above.

ngrok tunnel is often initiated on the client-side first, however it can hash out a secure channel with the server, which is a very slick way to obtain a work around standard firewall configurations. One thing to keep in mind is that as soon as you quit the terminal UI, the tunnel will be closed and re-running ngrok will lead to the creation of a new public url, so you might have to share that again. To prevent this from happening, you can specify a custom url with a Cname configuration. You can also run ngrok with a custom url on your own domain. This can be used to run development servers for sub-projects under a particular domain. Adding auth improves security and can restrict usage to people from the same organization, let’s say.

You can also access the documentation page directly with the public url.

Adding auth to protect Open Event localhost:
Anyone with the public url can access your localhost and can make changes. To prevent this we can use the auth switch with ngrok command. This will enforce Basic Auth on each request.

ngrok http -auth="username:password" 5000

Apart from these, ngrok tunnels can be used for file system sharing, mobile device testing and also to build webhooks integrations.

Additional Resources

• ngrok – secure introspectable tunnels to localhost – ngrok Official Docs
• Using ngrok to test local site – Ray Bango,Tech Evangelist, Microsoft
• What is HTTP tunneling? – Pranay Rana, SF
• Protocols: HTTP, TCP, TLS – Thomas Pornin, Security Stackexchange