Iteration 46: Real-time updates and performance improvements

One of the benefits of working with a physical card wall is that you know that your wall is an authorative version of your plan, and an accurate reflection of the team's progress.

This isn't necessarily the case if your card wall is on a web page. When somebody moves a card on their screen, anybody who has the same page open will be looking at an outdated plan. You often need to reload the web page to see the latest version.

Some of the teams using Planner have more than ten people working on a single board. With that many people on a board, significant changes can occur every quarter of an hour. Making decisions based on out of date web pages was starting to cause them problems.

The focus of iteration 46 was to fix it.

Notifications explain what's changed

The ideal solution would be to update everybody's boards in real time, replicating drag/drop events across all active browsers. Everybody's card wall would be kept up to date. It would have taken longer to write the code to do that than was available in the iteration, and we realised we could get most of the benefit for half the effort (the credit for spotting this goes to @tekin).

The big problem wasn't that people had to reload a web page; it was that they didn't know when they needed to reload a web page.

Next time you leave your browser open on the Current Iteration page you'll see a message like this one when somebody moves a card:

This is just an example, but it isn't the only message you might see – Planner will also tell you if somebody edits a card while you're looking at it.

In future I expect we'll be moving cards around before your eyes, as your teammates update your board.

Performance improvements

When considering how to implement real-time updates, I had a choice between two competing technologies:

1. Server Side Events (SSE), an HTML5 technology that allows the server to send data to the browser over the existing HTTP connection, and
2. Websockets, which support two-way communication between the server and the browser on a separate network port.

After a bit of research I decided to use SSE, as it looked like the simpler of the two options.

In order to be able to use SSE I needed to upgrade Planner to a newer version of Ruby and Rails (the programming language and framework that Planner is written in). I'd been wanting to upgrade for a while, but had prioritised new features instead. This was a perfect opportunity to do it.

Much to my surprise, the Ruby upgrade lead to a three-fold decrease in the time taken to serve its web pages. Requests that had been routinely taking over 300ms were now getting served in less than 100ms.

Retrospective

Shortly after I'd upgraded Ruby and Rails (so I could implement the notifications using SSE) I discovered that Rails isn't well suited to serving SSE at scale.

The Rails developers amongst you may now be saying "WAT?! He's said Rails doesn't scale!" I'll briefly explain what I'm saying. It's specifically about SSE, in Rails, running in a typical server environment. As currently implemented (Rails 4.x), SSE requires a threaded web server, and one active server thread for each user who keeps your app open in their browser. The thread is responsible for pushing data over an HTTP connection, and needs to maintain an open connection to your database.

If you're hosting your app in the cloud, a large number of concurrent database connections can become quite costly. I suspect that to make SSE a sensible approach we'd need threadsafe access to the database (which I expect will arrive in Rails before too long, for this very reason).

If you have a similar need for SSE and are hosting in the cloud, I'd recommend that you just use Websockets, and let Pusher (a hosted Websocket platform) deal with the complicated bits. Pusher turned out to be simpler to integrate into Planner than SSE anyway, as I didn't need to switch to a multi-threaded web server or setup an event bus to broadcast messages between threads.

So as it turned out I upgraded Ruby and Rails so I could use SSE, then switched to Websockets, and needn't have upgraded anything at all. Happily, the unexpected performance improvements alone have made the upgrade worth it.

Though it all turned out well, I dropped the ball here. I should have investigated SSE more thoroughly before committing so much time to the upgrade (which took several days).

I'll have to keep a closer eye on myself. I'm a loose cannon…