Email iconarrow-down-circleGroup 8arrow-rightGroup 4arrow-rightGroup 4Combined ShapeUntitled 2Untitled 2ozFill 166crosscupcake-icondribbble iconPage 1GitHamburgerPage 1Page 1LinkedInOval 1Page 1Email iconphone iconPodcastpushpinblog icon copy 2 + Bitmap Copy 2Fill 1medal copy 3Group 7twitter icontwitter iconPage 1

Introduction

One of the most successful digital products we’ve created is a deal arrangement platform for Limpid Markets. The product has evolved from allowing traders to arrange deals on a single precious metal derivative to a platform that facilitates deals in a number of diverse areas such as physical coal and physical precious metals (launching autumn 2017.)

The product has been through several significant changes since it launched, although the main stack on the backend has remained largely unchanged. However, as the platform has scaled, we’ve begun to notice a number of bottlenecks which are probably not fit for future scaling.


Eddy

Eddy

Developer

“I had been learning Elixir on and off for the last couple of years and each aha moment I had I realised a couple of things: firstly, I loved this, it was fun, it was easy to grok and test and it felt lighter and more modern than other languages that I’d been using. Secondly, it seemed a perfect fit for Limpid Markets. The immutable, concurrent-by-design nature of the language just lent itself so well to the problem space of a real-time orderbook. After a team-wide discussion we were satisfied of the potential benefits: speed, flexibility and not forgetting developer happiness. All we really needed was some practice…”

Practice practice practice

We used to run quarterly hack days but these became harder to organise when Limpid Markets took off. The hack days have always provided us with a time to experiment, to innovate and try out new ideas in a safe environment. We’ve learnt so much from these days: they’re one of the reasons we were able to create the Limpid Markets product in the first place.

Given the potential new direction for Limpid Markets, it seemed like a perfect opportunity to revisit our hack day past but this time to run an extended version into the week. An opportunity to discover the potential of a number of new technologies; to practice and experiment with them.

A safe place to experiment

Starting life as a hack day, we created MEMZ – a browser based photo sharing app allowing you to share your day’s experience with friends and family.

It’s been some time since we worked on MEMZ and the tech stack as well as the design and user experience has been in need of a revamp for a while. So it seemed like an ideal candidate for our hack week. We could revisit the entire application, rebuilding it with Elixir and Phoenix on the backend whilst throwing in a bit of experimentation on the frontend with the use of Elm. Not only that, there was an opportunity for Rachael to redesign the product whilst learning some new techniques.

Not just for developers


Rachael

Rachael

Design

“Not one to let the developers have all the fun, I was also keen to learn something new. After reviewing the latest designs for MEMZ, the whole team decided that they could do with a refresh. First I decided to change the typography to something a little more modern and then looked to incorporate the typeface into the logo. Cleaning up the logo has resulted in something more subtle to let the photographs be the standout elements of the application.”

logos

“Having the team working as the client was great as we got to include a fun onboarding process and a cute animation for when the photo feed is empty. I’ve always wanted to learn After Effects, so this was a great little project, and we took it a step further by using Body Moving and Air BnB’s tool Lottie to use the animation in the app.”

feed

Day one


Eddy

Eddy

Developer

“What an exciting first day to UVD’s first ever hack week! I started off the week with Alex, VJ and Kenny on the Elixir and Docker team. We each had a smallish amount of Elixir experience and are pretty au fait with Docker, so we soon had a new Phoenix 1.3 application bootstrapped and running in a nice containerised environment.

With time of the essence we utilised phoenix generators as much as possible and pretty soon had our first endpoint fully functioning. We managed to add in some pretty powerful tests with ExUnit, a joy in a functional environment, and rounded off the day by adding release generator Distillery, ready to deploy to the cloud tomorrow!”

James

James

Developer

“Day one of the MEMZ hackathon took the front-end head first into the Elm world: learning the basics of Elm on the web, getting to grips with the language syntax and taking our first stab at implementing HTTP requests. Towards the end of the day we also managed to implement our first route, creating the landing page and the create-event pages.”

Day two

Kenny

Kenny

Developer

“On the second day of our hack week, I learned how to use The Guardian authentication library, and how to implement JSON Web Tokens for authentication in our application. We also found an Elixir plug, CorsPlug, to add CORS and let our front-end team call our endpoint and test if our authentication was functioning properly. When they gave us the okay, we ended the day by adding slugs to deal with weird inputs.”

Ryan

Ryan

Developer

Building on our progress from the first day, today we had the slightly less glamorous job of handling some error cases that the backend could return from our create event request & handling an authorisation token.

We wrote our response error decoder first to convert the JSON response back into Elm values (tuples in our case)

type alias ServerError =
( String, List String )

....

errorResponseDecoder : Decode.Decoder (List ServerError)
errorResponseDecoder =
Decode.at [ "errors" ] (Decode.keyValuePairs (Decode.list Decode.string))

Then updated our model to store any messages back from the server and implemented the Messages.CreateEventResponse (Result.Err err) -> case of our update function which wrote those values from the response back to our model.

Our second task was twofold, we wanted to store a token we received back from the backend that we’ll use to authorise future requests. This was being sent on a header so we’d have to extract from our request, meaning we had to switch from an Http.post to a custom Http request to extract the full response. Secondly we wanted to persist this token beyond the current session meaning just writing it to our model wasn’t sufficient, we wanted to save our value to the browsers local storage also, which was our first foray into Elm’s Ports. This article proved a very useful introduction for us.

Lastly we did a bit of reorganisation as everything has ended up in our Main.elm file so far, we’ve just got some basic organisation at the moment but our longer term aim is to move to a similar architecture to the one outlined here.

Day three

Dave

Dave

Developer

“Today, we added the “event” page, which will probably be the main page of our application. The page is pretty basic at the moment, but the logic for loading the event (and handling errors) is now in place. When the page loads, we load the user’s authentication token from local storage. To achieve this, I had to learn about ports in elm. Ports are how elm communicates with javascript, and vice versa. We used them to fetch values out of localstorage. One port allows you to send a message to the “javascript world”, telling it to fetch a key from local storage. Once this value is retrieved, javascript then sends it to elm via another port, which causes the `update` function to be run.”

Alex

Alex

Developer

“Now we’ve had some experience with Guardian from yesterday, we’ve implemented it into our EventController and written automated tests. I have learnt how to test Phoenix applications rather than just small Elixir components. We took a deeper look into HTTP authentication and how to setup this within a single test with minimal setup. It’s been really useful to simplify and isolate tests within Phoenix.

We have a function called “show” in this controller which returns data about an event. In our first test, we checked that when the frontend makes a request to the show endpoint without authenticating, we receive a 401.


conn = get conn, event_path(conn, :show, 1)
assert conn.status == 401
assert conn.resp_body == "Unauthorized access"

In order for this to be any use the the client, we need allow access. When a user creates an event, an authorization token is set in the header. If this is set, they’ll have access to the event data. To test this, we had to manually put the token in the request header before making assertions on the show function, which then returns the 200 and data we expect.”


resource = %{:id => "Alex"}
{:ok, token, _} = Guardian.encode_and_sign(resource)

conn =
conn
|> put_req_header("authorization", "Bearer " <> token)

with {:ok, %Event{} = event} <- Events.create_event(event_params) do conn = get conn, event_path(conn, :show, event.id) assert json_response(conn, 200)["data"] == %{OUR EXPECTED STRUCT}

Day four

James

James

Developer

We've been utilising the elm-phoenix-socket library to make a web socket connection to the backend and allow us to connect to individual channels for each event.

What did we learn today? We learnt about Elm Flags, these allow us to use environment variables which is key for the deployment of the project. And as part of the work to enable photo uploads, we also learned about dealing with file inputs as well as how to base64 encode image files before POSTing them to the API.

VJ

VJ

Developer

Today was spent setting up deployment to our own shared hosting service via Docker on Amazon Web Services. We use Amazon's ECS (EC2 Container Service) to manage containers running across a cluster of nodes behind Traefik, a load balancer that also provides TLS encryption via Let's Encrypt.

We're using Hashicorp's Terraform to maintain our infrastructure via code. In order to maintain secret variables, we pass them as environment variables to the Docker container through our container definitions in ECS and have parameterised them with Terraform variables. Terraform does **not** secure variables in their state files thus extra care is taken to avoid exposing those statefiles.
We also started learning about Prometheus for monitoring our ECS clusters. Using an ECS exporter we came up with a simple dashboard in Grafana to visualise which services are running and more importantly which ones are down. We aim to go further by introducing alerts based on the powerful query engine Prometheus provides.

dashboard

Day five

James

James

Developer

All in all, the MEMZ hack week allowed us frontenders to really experiment with Elm. For the majority, this was the first project that we've started with a functional language, taking a fully functional approach. Over the week we had the time to not only learn the basics, but also explore the features of the language. With this new understanding, I'm sure it won't be long until we look to utilise the benefits of Elm within project work at UVD.

In terms of progress made, we managed to get the basics done. Users could create an event and upload photos to their event stream:

feed

Overall, I think I speak for all when I say that we had a very enjoyable, informative and productive week.

Eddy

Eddy

Developer

“We had a great time writing our Elixir Phoenix backend. As we had some experience with the language and framework already we had enough breathing space to explore some aspects we were not so familiar with, such as the awesome ExUnit. We managed to get our Guardian authentication system fully functional and authenticating Phoenix channels, and best of all we managed to get all of this covered by tests. Because our sign-up process was quite bespoke, we ended up doing a deep dive into Ecto changesets and learned a lot about how the new Phoenix 1.3 fallback controllers work.
The week was really exciting. We already had Elixir earmarked as a language we'd like to use for future project work and our learnings only enforced our opinion that this was an incredibly powerful tool. We can't wait to use it more!”

Share: