Sending data to our website via Arduino and end-to-end BDD

Quick links


Anyone who’s following our Tweets will be aware that at 5pm each day we halt client work and concentrate on some personal development activities. Rather than work individually on various things, this month we’ve decided to team up and run the personal development hour as a project involving all team members. We thought it would be a good idea to do so as we can practice some of our project methodology and have a bit more fun whilst learning new things. We also thought it would be great to have something at the end of it to show off.

Week 1 – defining project goals and user stories

Seeing as we wanted to run this project as with any other client project we set about defining our project goals in a workshop. It’s important to understand the goals of a project before venturing into user story workshop because user stories hang from the goals of the project. We can then refer back to the goals when we are defining user stories to make sure they bring value to the project. It’s the project sense check before committing to user stories that have gone off on a tangent.

We went around the project stakeholders (consisting of designer, front-end developer, back-end developer, project manager and business owner) who each in turn described what they wanted to get out of their personal development and therefore what the project we were going to do would have to facilitate.

Here’s a quick rundown:

“As a front-end developer I would like to be have a better understanding of how I can align my JavaScript web application development with the behaviour driven development (BDD) practices we are applying on the back-end”

“As a developer I would like to try out the Silex micro php framework (built with Symfony2 components) to understand when it might be appropriate to use in our web application development and I’d also like to start deploying using continuous integration tools”

“As a project manager I would like to practice running a project which uses BDD throughout so I can understand the best way of defining user stories for this process”

“As a business owner I would like to demonstrate to our clients and potential customers that we are working with some exciting and innovative technologies, underpinned by best practice methodologies and I would also like us to become better and more efficient (through practice) of delivering high value software”

What are we going to do?

Now we’ve defined the goals we can figure out what sort of mini project we can take on that delivers all of the above. One area on our site we had big ideas for which ultimately took a lower priority was our footer area which contains some interesting stats. We originally intended to make more of our coffee stats in particular and purchased an Arduino board in order to build a physical interface (button) which when pressed would push the data up to our webserver. But as this was by no means MVP the Arduino kit has been gathering dust for almost a year. Which is a shame because people often ask about these stats in particular and the answer is always a bit less inspiring than it should be (“well we were going to do something exciting but for now the data comes from a hard coded config file”).

With renewed vigour we set about defining a mini project that would meet the project goals and finally deliver what we originally intended by creating a new page related to the data visualisation of our coffee consumption whilst pulling the data from actual consumption (when a physical button in the office is pressed) as opposed to made up averages.

User story workshop

As always, once the project goals are set we are able to involve all stakeholders in a user story workshop. We usually start by asking everyone to spend a few minutes writing down all the stories they think would meet the project goals. It’s essential to involve everyone because you’ll miss a certain viewpoint if not and that will lead to missing user stories. Of course there are some duplicates but that doesn’t matter.

Once everyone has exhausted his or her ideas we stick the post-its on the wall and pick each story, one at a time, discussing each with the team. We are then able to loosely group these user stories (the grouping types appear organically as opposed to being prescriptive) and throw out any duplicates.

Finally we can do a MoSoCoW activity on each story so as to understand which ones offer the highest value:

  • Stories in the ‘Must have’ column are ones where we couldn’t go live unless they were done
  • Stories in the ‘Should have’ column are the ones which add a lot of value and often make the difference but at a push we would go live without them
  • Stories in the ‘Could have’ column are the ones that are valid but are not essential. Otherwise known as ‘nice to haves’
  • Stories in the ‘Won’t have column’ are not yet valid

Our aim is to deliver all the ‘Must haves’ and as many of the ‘Could haves’ as possible in the time frame for the first release.

What’s interesting is what tends to happen as a project is being developed is that user stories tend to flow right (toward ‘could have’ and ‘won’t have’) not because we are not able to deliver them in the timeframe but because once you’re into the design and development phase (sprint mode) of a project you can validate stories more accurately (you know more and consequently the uncertainty is less, you may even have put scamps, ideas or demo’s in front of the target audience). This is what ‘Agile’ is meant to do and why Waterfall sucks – because with Agile you are have a framework for continuously changing the plan so as to only deliver valid, high value features and to take into account the fact that things change as you move along whereas in waterfall you must deliver a set of agreed (up front) features half of which inevitably turn out to be a waste of time (and money).

We now have a valid set of user stories with an initial estimate as to their importance. Great. So we can just get going on them right? Not yet!

The thing about delivering a project (and this is a problem we commonly face with bespoke web application development) is that new technology or new design challenges provide the greatest amount of risk to any estimate. But when estimating the general size of a user story (as we do in order to understand project scale/cost) it is important to factor this in. Therefore we create what’s known as ‘Sprint 0’ and within this identify the main areas of risk to our estimations and set out a plan to find just enough information to be able to estimate with a greater certainty. This might involve research (or R&D in some cases), looking into frameworks, API’s or how others have tackled the same problems. In this project we identified a few areas to look at before we can tackle the all important user stories:

  • Cucumber.js: we’ve used Mocha.js for our JavaScript BDD but have felt that Cucumber might be more aligned with our back-end BDD which uses Behat
  • Silex: we use Symfony2 which is obviously the bigger cousin of Silex but we still need to do more reading on it to understand any pitfalls
  • Continuous integration tools: we like the look of Jenkins CI but need to look more thoroughly into other tools and to understand them a bit more
  • Dynamic data visualisation: our Digital Designer wanted to research how other designers and front-end developers have tackled the technical hurdles so that when it comes to designing the page she won’t be designing something that is impossible to implement

Week 2 – research time (sprint 0)

The aim of this week was to get everything out of the way so we could hit the user story execution with full velocity. In order to achieve this we’ve done a number of things which we’ve wrapped up into a ‘mini’ sprint we call Sprint 0 which doesn’t contain any user stories:

Ryan: has been getting up to speed with Cucumber.js and discussing with Rob a common language for describing our tests from back-end to front-end.

Rob: has been reading up on continuous integration tools and has settled on Jenkins CI. He then got Jenkins successfully building off a push to bitbucket account hosted on AWS. Finally he’s drawn some nice diagrams explaining how it’s going to fit within our deployment methodology which you’ll be able to read about shortly.

Natalie: has been working with Ryan and myself to come up with some data visualisation concepts. We’ve done this work up-front (which normally we would do as part of user story execution) so that we could get the design out of the way and Natalie could concentrate on her front-end as opposed to her design skills which are obviously already fantastic. Here’s where she got to:


We’re hoping that with all this stuff out of the way we can start on some user story action in the next week with Sprint 1 on the horizon. More to follow…

Week 3 – pivot time!

In all true Agile projects we have to be prepared to pivot on our idea: maybe the further you get the less valid a certain user story becomes or potentially new user stories need to be written. In fact it’s perfectly acceptable for the whole project to change direction.

It’s pretty trendy these days for a technology focussed agency such as ourselves to create some quirky Arduino based application and so I might disappoint a few people out there to say that in the cold light of day, having spent some research time I became increasingly skeptical about aligning the Arduino part of this project with the project goals. That’s why I ended up taking on the Arduino part of the work (something along the lines of allowing a user to click a physical button and the coffee stat is incremented on our webserver database): the Arduino part didn’t really interest anyone else. But I’m not really interested in upping my Arduino skills and what’s more, this left the front-end pretty static and not as useful for Ryan to test his BDD skills against. Therefore we took the decision to ditch the Arduino in order to concentrate on the goals of the project: the personal development of the team. So now we’re going to allow users to login via a mobile interface and record how many coffees they have just made. This increases workload on the front-end (good for Ryan to practice his BDD approach) and enables us to capture richer data which we can use in our statistics (individual users will now have a coffee made count).

Week 4 – user story execution

It’s time! We’ve got all our ducks in a row, know the approach we are taking and have even done a presentation on it at our Green Room tech event. So in this week’s workshop we decided to tackle our first user story:

As as barista I want the application to know it is me that has made the coffee

This has been planned out as a login screen where a user (member of staff) enters a unique 4 digit passcode. It’s a nice starter because it’s the very first part of the user journey and it requires design, front-end and back-end effort so everyone can work from the same Trello card.



Week 5 – user story execution: how lean can we go?

One of the purposes of this project has been to practice what we preach to our clients and that is to go for the lean approach. The lean start-up is a commonly banded around phrase but unless you practice it over and over again it’s actually really difficult to achieve for various reasons I won’t go into in this post. For us, the lean approach can be put into the following simple phrase:

To go lean is to deliver a viable, testable and deployable piece of software with as few user stories as possible

To put it into context, we have a product backlog of 14 user stories for this project which may or may not all get done. With that in mind we need to tackle the highest value stories that deliver something of worth and of use to us. There are activities that can be done around searching for value and finding out which user stories to tackle first which I’ll describe in a different post but suffice to say we’ve identified that we can release a first version of our mini project by completing 3 out of 14 user stories.

This may not sound very exciting until you consider the typical approach (Waterfall being one of them) which would generally require all 14 stories to be complete before delivering anything of worth (if it’s not live it’s not worth anything). Meaning that in this instance, we’re able to launch a viable product in only a fraction of the time (nearly 5 x faster by my reckoning). This provides a massive competitor advantage and means you’ll be gaining customers and improving your product way before anyone else has got to market.

More to follow once we’ve got to launch!