Loading front-end code at will with Scoutfiles

We've developed a new way how to load front-end code.

A look at front-end code

It’s hardly news that modern apps are written as single page webapps consuming a remote API. It became good practice to write web apps as static clients consuming the same public API as any other clients, perhaps with the exception of being hosted on the service’s main domain. Because we create highly dynamic webapps that create interdependent experiences, we assume that the user will download the entire web application into their browser. As such we attempt to minimize the size of the client code so that the user has the fastest experience possible.

This leads us into a world where webapps more resemble traditional software: we can assume the client to have the entire source present; we have a build process and build artefacts and we tend to end up with some concept of versions (even if we practice some form of continuous delivery).

We’ve developed an interesting methodology of dealing with this. Here’s a quick video demo of this system, the rest of the article describe the details of the system.

In the video, I make some changes to the styles and demonstrate how I can get a version of the app running on production with a shareable link. If you have a Cloud Analytics account, you can try Pony Analytics right here:

https://analytics.rightscale.com/dashboard?scout=6239b1e2f8a1963eff137c5ca7f7520f5fda8bc0

I also show how we gain separation between front-end and back-end code, because each can be run with the production version of the other.

Overview

Our code is split into a back-end and front-end system. The back-end system is deployed using our Cloud Management product (we eat our own dog food). The front-end system is ‘deployed’ on every git push to GitHub. This is not visible to users, since which of these ‘deployments’ is visible is governed by a database record on the back-end. This field propagates to the entry point of the front-end: the index.html. From here we load what we call a Scoutfile, this gives us considerable flexibility to load any other ‘deployment’ if specified, or default to the ‘production’ deployment.

We wanted the ability for front-end engineers and designers to be able to run the front-end against our staging or production back-ends. This allows them to work on the front-end without having to go trough the trouble of having to setup the back-end (which is admittedly painful) and of acquiring realistic datasets in their development environment.

Once we had that ability, we realised that it would be nice to have the ability to deploy and test every single commit and be able to share these deployments in our team.

Hosting

We push code to GitHub. Travis CI picks up the push and builds, tests, packages and finally uploads our build artefacts to S3. We serve our build artefacts from S3, but for security reasons we have a reverse proxy set up that we load the assets through, so users always see assets loading from our IP addresses.

Scoutfile

The production app then only loads a tiny file that checks a scout url parameter. If the parameter is present, it will load the entire front-end assets from the S3 bucket, from the folder corresponding to the scout parameter. It can also be used to load assets served by a local server.

This enables a much more powerful method of communication in the team, where any change can be instantly and automatically previewed by any other team member.

Chrome Extension

To make this form of communication even easier, I wrote a Chrome extension, that adds preview buttons to GitHub Pull Requests:

UI Previewer Chrome Extension

When you click on the preview button, you are redirected to the production application with the front-end corresponding to the latest commit in the pull request is loaded.

UI Previewer Chrome Extension
To avoid developer confusion about which version of the interface they are using, we added a little version indicator widget, that will show up if the user isn't using the default version. Clicking it will switch back to the default version and reload the page.

The extension is generic, so if you wish to use a similar approach, you can get it here and change the settings so that it work for your setup.

Technical Overview

Here I would like to show some of the key pieces of this infrastructure. This is a highly technical bit, and essentially this is almost literal programming, so consider yourself warned.

index.html - The Stringent Gatekeeper

We render our single page app through our Rails app. This is not strictly necessary - our original concept simply had a global file in the root of the S3 bucket. But having Rails handle the entry point into the application means that Rails can redirect the user to the login page and can easily bootstrap the app with some initial data. But more importantly, we store the Git SHA of the production version of the app in the database. This means that only people who have write access to the production database can release a new version of the application to the customer (which given my pony example, is probably a good thing), whereas every commit can (and indeed is) written to the production S3 bucket.

This renders the following view:

Rails.application.config.assets_endpoint is a configuration option that in production points to a proxy server that ensures that all requests are through RightScale IP addresses. In development it points to a different server on localhost.

Travis - The Ingenious Builder

Travis is a system that requires surprisingly little code to get very powerful things done:

scoutfile.js - The Courages Source Finder

Inspired by the influential post by Alex Sexton, The scoutfile is there to figure out what version of the app to load and then to do it.

ui_version_indicator_directive.js - The Handy Informant

Conclusion

This system allows for a big degree of flexibility, communication and easy customer prototypes. BTW, we’re looking for a front-end engineer to join our team if this is something that interests you.