ClearScore Engineering Principles
On August 12th, something incredible happened at ClearScore. The preceding weeks had seen our International Deployment Team working day and night to get our India Beta Site ready. The result? That day, we witnessed a full, successful end-to-end registration. Our first Indian Credit Report rendered with vibrant, localised content.
Just a few days later, we officially went live in India.
Any product launch is a magical moment. But the platform for the India Launch was designed from the ground-up to support International markets. It’s made up of more than 50 microservices, running on Kubernetes, pipelines using Spinnaker deploying into multiple AWS regions, and an analytics data warehouse powered by Amazon Redshift. Oh, and a new front end built using Atomically Designed React components with multi-lingual capability.
It’s been three years since we launched in the UK. We can now proudly call ourselves a truly global business, providing free financial well-being to millions of people.
To achieve something like this so quickly you need a great team. But you also need an awesome culture. We pride ourselves on the fact that our culture is, and will always be, our own. Defined by what we call our ClearScore Operating System (csOS), it’s based around 8 fundamental behaviours.
Put simply, we believe that what we do matters, and that our company matters. We focus on achieving peak performance and creating an environment where we can do our best work and be our best selves.
From this, we’ve created 12 Engineering Principles:
1 - Hire smart and passionate people
Steve Jobs is famously quoted saying, “it doesn't make sense to hire smart people and then tell them what to do; we hire smart people so they can tell us what to do”. We’ve adopted this wisdom here at ClearScore.
Our interviews are behaviour-based - we assess whether someone has opinions and isn't afraid to argue around a whiteboard. But this doesn’t mean that we only hire extroverts. In fact, some of our most feisty engineers are introverts who bring gravitas to discussion by arriving armed with meticulously collected data.
Nor does it mean our interview process is male-centric. Currently, 34% of ClearScore are female, but we’re ambitiously pushing for gender equality by the end of 2019 (watch this space).
2 - Aligned autonomy
Jeff Bezos has talked about how Amazon is "stubborn on vision" but also "flexible on details". Common direction and strong alignment are important, but you also need flexibility and autonomy to move fast. In our opinion, combining these concepts accelerates success.
We operate in cross-functional, autonomous Agile squads that are aligned with the company vision. ClearScore has had numerous iterations of delivery models and we pride ourselves on our ability to restructure at pace. From a single squad in early 2015, to squads and business partners within delivery teams in 2017 and finally our current 2018 model.
Team roadmaps are driven by company goals. All of these goals, as well as individual objectives and key results (OKRs), are visible within our 7Geese Performance Management System. Our culture of transparency means you can see how anyone’s performing at any time, from Junior Developers to our CEO.
3 - T-shaped capability
The Scrum Guide describes cross-functionality as everyone having “all [the] competencies needed to accomplish the work without depending on others not part of the team”. Our challenge is to resource a team knowing that it can successfully deliver even when key members are away. We want to avoid a ‘Noah's Ark’ situation, where we have two of every type of Engineer to survive the storm.
This is why we promote T-shaped capabilities, to strike a balance between shared knowledge and specialist streams. In practice, this means that our Developers sometimes write stories, our Test Engineers often cut code and some of our Mobile Team can handle both iOS and Android. It’s not unheard of for our employees to change careers entirely, from User Delight to Product, or from Front End Developer to Delivery Manager, for example.
4 - Decoupled releases
When we launched in July 2015, our entire platform consisted of an Angular app on a single Java API. It moved data to and from a PostgreSQL database in an in-house data centre. Every part of the system was monolithic; a deliberate strategy to rapidly get to market. We always knew that we would eventually migrate to an ecosystem of micro services, and along the way, we created micro-frontends.
"Any organisation that designs a system... will produce a design whose structure is a copy of the organisation's communication structure", according to Conway’s Law. This rings true at ClearScore. We’ve created teams organised around the key parts of the user journey. Our architecture has subsequently become decentralised and broken down into product boundaries owned by squads. This allows each team to take end-to-end product ownership and autonomously release changes without blocking other teams or incurring the overhead of excessive regression testing.
5 - Small and frequent releases
In my experience, release frequency is one of the best measures of the health of a tech company. By using automation to work smart and work hard, the investment pays for itself by minimising the time between code being submitted to being in hands of our users.
Amazon recently disclosed that their AWS team performs 50 million code releases per year (that's more than one per second). Using the AWS Platform, Netflix has transformed their processes by creating Spinnaker as their continuous deployment tool. So, small and frequent releases mean you can enjoy watching Stranger Things without being interrupted by software upgrades.
In our first published Software Development Life Cycle (SDLC), we advocated daily releases, although Friday releases were generally frowned upon. Big bang releases do occasionally happen, but they’re becoming less frequent. A combination of Spinnaker and ‘pipelines as code’ meant our release frequency last month went above one per hour for the first time. It’s now not uncommon for a couple of releases to go live even after the beer fridge opens at 4pm on a Friday.
6 - Productivity over risk
The ability to fail and fix fast is championed by entrepreneur and author Eric Ries in his 2011 book, ‘The Lean Startup’. At ClearScore, we know that failures are inevitable. Instead of ruminating over them, we aim to reduce the time lag in detecting the failure and relaying it back to Engineers.
We fully embrace the ‘fail and fix fast’ philosophy, continually investigating metrics to make sure our product meets user needs before investing more time and money. It doesn't mean we’re reckless with quality standards or user data. But if there’s a choice between week-long delays or releasing a Minimum Lovable Product, we trust our guts and go with the latter.
7 - Continuously remove legacy
Technical debt takes many forms and, like a weed, will infest and strangle your garden if it is not continuously dealt with. Thoughtworks developed a great approach to managing this where their Technology Advisory Board create a Technology Radar several times a year, and we’ve borrowed from this.
Every three months, our Technology Leadership Team gets together to decide the best practices to follow for Development & Testing Techniques, Development & Testing Tools, Protocols & Platforms and Languages & Frameworks. It's one of our most heated and vocal technical meetings. Once completed, we put items into 4 categories: hold, assess, trial and adopt. This is then published, and serves as a guide for all the delivery teams.
8 - Security by design
From the moment someone registers with ClearScore, our priority is safeguarding their data.
Our Information Security policies and practices have evolved as our userbase has grown. This year, there’s been a major overhaul in our Information Security Policies and SDLC. And as a result, we’ve put in place data classification and controls, OWASP best practices, and automated security scanning with every deployment. We are also pursuing several certifications including ISO 27001 for India.
9 - You build it, you run it
By organising our company from the very beginning into product teams with a DevOps mindset, we embraced Werner Vogel’s (Amazon) mantra “you build it, you run it”. Decentralised release management and monitoring ensures greater ownership. In the UK, this has resulted in 7 months of straight 100% uptime this year.
We’re taking this one step further by moving to a ‘NoOps culture’ to really empower squads. Global Delivery Teams will be given the tools to create and run pipelines independently. Access to Instana Application Performance Management (APM) toolset (which uses AI to detect dysfunctional services and notifies the appropriate teams) means squads are able to act before adverse behaviour manifests in production.
10 - Data-driven quality
Without data, you're just another person with an opinion. This is the strapline for our Daily Dashboard, which is emailed to every single person at ClearScore each morning. It’s packed with charts of metrics for the Key Performance Indicators (KPIs) for every team. We can also see charts on bugs, uptime, releases and sobering projections as to whether targets will be hit.
We’re all responsible for quality. We use metrics to track this and surface data using a combination of systems, including Instana (Application Performance Monitoring), Amplitude (Real Time Monitoring) and Domo (Daily Dashboards). Lots of eyeballs on key metrics means that they never deviate far off course before the relevant team is all over it.
11 - Maximise learning
Research from the Centre for Creative Leadership suggests that everyone needs three types of experience, in a 70-20-10 ratio. Challenging assignments accounts for 70%, developmental relationships 20%, and coursework and training 10%.
In-house learning is one of our core differentiators, and one of the reasons why ‘we learn’ is one of the principles of our Operating System. Technology careers within ClearScore can take both a management or contribution path. But regardless of the path you pick, we believe that sharing (knowledge) is caring. To maximise learning, we use a combination of ClearScore University (CSU) modules, communities, mentoring, buddy systems, brown bags and demos.
12 - Continuously improve
The final principle is arguably the most important. Abraham Lincoln once said, "Give me six hours to chop down a tree and I will spend the first four sharpening the axe". One of my favourite quotes, it highlights how productivity can be maximised through efficiency.
Kaizen methodologies are regularly applied to our engineering to sharpen our approach. Everything from phasing out legacy, to using ‘five whys’ in running wash-ups, running regular retrospectives and never just accepting the status quo.
Heraclitus, a Greek philosopher, once said “the only constant is change”. Our response to that is often “so just don't get too comfortable" (we’re half joking).