Saturday, 7 November 2015

Microscaling with Load-balancing in Amazon ECS (part 1)

Currently I am involved in the development of a digital imaging platform for libraries and other institutions (the Digital Library Cloud Service, see This platform uses a wide palette of Amazon Web Services for the orchestration of moving image assets between storage systems, retrieval of metadata and the presentation of deep-zoom image tiles. It’s sorta-kinda an “elastic image server” which supports the IIIF ( standard.
The system is split into two layers - Orchestration and Image-Serving. Both of these are intended to auto-scale independently, and this has been achieved with the Orchestration layer by deploying it as an Elastic Beanstalk application. Since this application is (currently) a .Net application that lives in IIS, then this environment suits it well.
The Image-Serving layer is Linux-based, comprising of a lightweight HTTP daemon (lighttpd) and Ruven Patel’s IIPSrv ( with optimised Jpeg-2000 support. This is quite easily deployed as a Docker container and that is what our current prototype uses.
Since we are using Docker anyway, I looked into how Amazon’s Elastic Container Service could be used to auto-scale the Image-Serving layer on demand.
What I found was that the concept of a Service in ECS did not cover how I thought it could be used.

Services under ECS

A Service in ECS represents parts of an application that are distributed across various Docker containers. A cluster of EC2 instances form a group of hosts that a Service may be deployed to. Each host may contain a maximum of one instance of a Service. When scaling occurs, the EC2 cluster can expand or contract with Auto-Scaling Group rules and the ECS Scheduler decides where new instances are placed.
The limitation of a maximum of a single Service instance per host is one that I was not expecting, but can understand given the way Elastic Load Balancers are configured. In ECS, the port numbers of containers in the Service are static because you’ll only get one instance of the Service per host, and therefore load balancing can occur across the cluster in a static way (from the Elastic Load Balancer’s point of view).
When I saw the setup dialog for a Task in ECS mention the internal and external ports for a Docker container, I had hoped that this implied that the external port on the container could be balanced across if it was left blank, but that turned out not to be the case.


What I wanted was the ability to work with multiple similar Docker containers populating EC2 cluster members dynamically in response to some metric - e.g. a CloudWatch alarm from our orchestration layer’s load balancer. As that Elastic Beanstalk application scales in response to demand, so would the number of deployed containers up to the available space of the hosts in the cluster as defined in ECS.
It turns out that this technique has a term - microscaling - and I’ve been researching it recently, including the offering from - ( - see for more details.
Once the number of containers is under dynamic control, the cluster itself could then expand or contract by responding to a separate Auto-Scaling Group rule.


The other issue is load balancing across the cluster. In the scenario I want, the ports on the deployed containers would need to be routed to directly by the load balancer, so that means multiple different ports per host. This is something which would bend the ELB architecture out of shape, I think.

UPDATE: Now that Amazon have released their Application Load Balancer feature, then the load balancing problem is more or less solved on AWS. I must have been one of many people who were after this kind of feature as a senior ECS developer talked to me about it on Twitter over a year ago.

Use case and solution

Our use case is that I want to have an auto-scaling group of EC2 instances in an ECS cluster which host multiple similar containers that will listen on the same internal port, but are routed to via their external port that is assigned by Docker (e.g. while still appearing to be port 80 inside the container). They will be available to be load-balanced across all the containers that are running in the cluster. These large EC2 instances could host multiple different ‘services’ comprised of similar containers.
Essentially, we’d like to have a number of large container hosts and fill them on-demand with web servers or other cookie-cutter services, with load balancing going direct to the containers, across a cluster, across AZs.
I have created a system for performing both microscaling and load-balancing within ECS, and I will show the solution in the next blog post.

Wednesday, 4 November 2015

Twitter - We should talk.

Hi Twitter.

Please, come in and sit down. I think it's time we had a little chat.

I know things haven't been great lately, and I think you're coming undone at the seams a bit.

I'd like to lay a few things out for discussion.

Your position.

You are in a unique position of being a vital, real-time connection between content producers and interested parties, who may also be content producers. You enjoy a position which is (blessedly) outside Facebook and is not a walled garden. You have made amazing technical advances to be able to support this amazing real-time platform. You are, in general, viewed as a force for good. You have friends.

I have witnessed more news, events, conversations and opinions on Twitter than almost any other medium I've used in the past twenty years that I've been on the Internet. It's addictive to be part of this huge, global conversation and I love that.

Recent activity.

But you keep doing these things.


Maybe these were justified because you had over-hired, but it looked reactionary which, if I was one of your investors, would make me quite nervous and/or irritated. Stop jerking your knees. Be more solid.


Experimental. I think it's good to separate curated items like this, which leaves timelines alone. More on that later. Definitely interesting as a thing for surrounding current affairs and getting people interested in them. The "You're caught up" is very Circa-like (RIP Circa). The Trending items have been kind of filling that purpose already, though, but the presentation in Moments is very clean, so I hope that this can build on that kind of attention into something cool. But I'm really not sure that this is the new user grabber that you think it is.

Please don't retire trending or hashtags though. That would be uncool.


The polling functionality is a really powerful thing. You can do a lot of good in this area. Keep up with the good work on this. Be a part of online democracy and live reaction. We like.


Still niche, but an excellent reporting tool. It would be nice to see better integration of that in things like the TweetDeck Chrome extension. Real problems here are bandwidth, video quality, people's data plans, vertical aspect video.


I know Instagram do 15 seconds, but 6 seconds is still cool, right? The integration of this service makes sense because it lends itself to the concise nature of a Tweet. However, the only people I have really seen (in the UK) using Vine are comedians. Think about that for one of those 6 seconds.

Apparent lack of action regarding the tackling of online abuse.

You really, REALLY need to sort this shit out. Provide tools, streamline reporting processes, support victims, unleash rogue AI... Literally anything apart from sitting on your hands about bullying and threatening behaviour.

Favourites vs likes.

Stars vs hearts. Connotations and implications. Organisation vs emotion. I could go on. In fact ...

The whole point of Web 2.0 was the ability to customise your web experience. What happened? How can it not be clear that this change makes it means something totally different? It smacks of flag-waving. You must stop doing these kinds of changes. People (read: your user base) get very used to these semantics and symbols, and you'd be amazed what could drive a person away if they felt aggrieved enough about it. As it is, it's now fairly evident that the people who make these kinds of decisions don't appear to use Twitter or "get it" - the blog post from Akarshan Kumar made it clear that you are going for lowest common denominator, and that's extremely patronising.

The best possible thing you can do here is to bring back the Star icon, and call the collection of things "Starred", because that is completely agnostic to emotional connections. Then this feature can go back to its meaning being implied from context (which includes timing) and also have zero connotations when used as an organisational method. Please do this because what you've done makes no sense to me and many others.


Most of Twitter is noisy. If I step outside of the accounts I follow then I find myself in a horrific sea of chatter. I'm actually cool with that. It supports the notion of 'tuning-in' to certain voices; apart from the Etsy bots - I'm f-ing sick of those. You should be able to spot them and many other types of spammer by now - so why aren't they automatically banned?

Concentration on in-house clients.

At the cost of pissing off every other development team who ever tried to make a 3rd party add on or client for Twitter. It's like you really don't want an ecosystem at all. You are a PLATFORM, a SERVICE, let people MAKE THINGS THAT USE YOUR PLATFORM. If this is about the whole advertising thing - i.e. that you can't control placement of adverts in 3rd party clients, then you're doing things backwards. FIND ANOTHER MODEL (see below). It's not like you're showing adverts in TweetDeck's chrome extension yet anyway (and please don't start now - I'll cry).

Perverse notion of messing with the chronology of timelines.

Twitter's utility is that it shows a chronological view of content. I have absolutely no idea why you think that modifying this is in any way a good thing to do. Even the the out-of-order replies reaction (the "blue line" fiasco) was a UX-disaster with people (like me) complaining about the cognitive dissonance it provoked and being the final nail in the coffin for staying the hell away from the web client. The considered plans for "Tweets-you-may-have-missed" is just barmy. If there are things that you think I've missed - PUT THEM INTO A SEPARATE LIST. Seriously, if you cock up the chronology of Tweets in a timeline then you might as well be a randomly ordered collection of things people have said at some point in the past X hours like Facebook. Just stop that line of thinking.

Consideration of Tweets longer than 140 characters.

No, no, no, no, NO. Twitter is concise. Keep it that way. How am I supposed to catch up on 500+ Tweets that happened overnight if they are a bunch of essays? That's what I have RSS for! If someone wants to post more than 140 characters then they should host content elsewhere which has meta-data on it that allows the previews in clients - just like they do now. CREATE AN ECOSYSTEM, NOT A WALLED GARDEN.

Also - please support RSS. RSS is your friend.

The web client.

I mean, seriously. I'm really glad that we got the Bootstrap project out of Twitter (I use it every day at work), but my God you guys really need to completely re-think your UX. Jesus F-ing Christ, what a mess. If I was approaching that project I would absolutely throw the entire template away and hope that no-one ever mentioned it at parties I went to.

Stock price.

By doubling down on bad ideas to please investors, all it goes to show is that you have collected the wrong investors, with the wrong promises whispered into the wrong ears.

The knock-on effects of promising swathes of new users is removing focus from what makes your platform great.

New users are only valuable if you can present advertising to them. And you've seen the current ad-block fuss.

What you need are customers.

Recommended actions.

Subscription model.

I would absolutely pay $5-10 dollars a month for a subscription to Twitter. That subscription would be ad-free, with a reasonable expectation of timely support for problems with my account, payment, online abuse, and the knowledge that my 3rd party Twitter client could be used without limitation.

I can envisage you running a subscription programme in parallel with your current operations, the only difference being the ad-free nature of the channel, a better SLA for support and the unlocked API access.

The more I think about it, (and I've been thinking about this for a few years), the more strongly I believe that this would be hugely successful for you.

But how much would subscription actually cost you? What change would that have on revenue when a (hopefully significant) percentage of your users are throwing you money each month? What would be the effect upon new user take-up, if they joined knowing that they could upgrade their experience? What would be the effect on API use if it was suddenly opened up to paying users?

I can't help but feel that these changes would be positive.

Content producer payment.

Given that many people join up to be able to follow people with something to say, celebrities, authors, musicians, news outlets, etc, then perhaps that should be rewarded somehow.

There are a few different services which allow users to make payment via Twitter. By taking that in-house this could be like your own version of Patreon, (minus the data leakage), offering content producers payback for being popular.

Again, this is about creating an ecosystem. Reward people for taking part.

Signing off.

Twitter, I don't know if you'll read this. I hope you do and I hope that some of it gets through to you. You are a vital, current thing, and it is in everyone's interest (apart from Facebook's) that you stop these self-harming actions and become something better.

Good day to you, whomever you are.