Lead Data Engineer (Remote)

Job description

Location: Remote, UK based, as travel to our London hub will be required once a month.

Team: Engineering and Operations

As Ably enters its next stage of growth, we’re looking for talented and curious engineers to help us build and deliver the platform to power synchronized digital experiences in realtime for one billion devices every month by 2023.

Our everyday digital experiences are in the midst of a realtime revolution. Whether attending an event in a virtual venue, receiving realtime financial information, or monitoring live car performance data – consumers simply expect realtime digital experiences as standard. As such, synchronized data in realtime is now business critical for many organisations. Yet, at the same time, organizations don’t have the in-house capabilities nor the time to do this efficiently at scale. That’s where Ably comes in.

We provide a suite of APIs to build, extend, and deliver powerful digital experiences in realtime for more than 250 million devices across 80 countries each month. Organizations like Toyota, Bloomberg, HubSpot, and Hopin depend on Ably’s platform to offload the growing complexity of business-critical realtime data synchronization at global scale.

Working at Ably means helping to build the infrastructure and technology that will power and shape the future of the internet. The opportunity in front of us is immense. But we can’t rest yet. And we’d like your help.

To see what life’s like at Ably, check out our careers page at ably.com/careers and read the story of how Ably's core values came to be.

What we do

Ably’s engineering team is deeply technical, collectively bringing a wealth of experience and broad technology skills across infrastructure, security, distributed systems, and much more. The calibre of the code we produce is what excites and motivates us each day. If you enjoy solving hard distributed system problems that are mostly platform agnostic and theoretical in nature, then you'll love working at Ably.

In this role

Ably is an engineering company at its heart. Our audience ranges from software engineers to CIOs and it’s our job to make sure the world knows about Ably.

You will be responsible for laying down the foundations of Ably’s internal data processing pipelines, working closely with our website, realtime, product, operations, marketing and support teams to unlock our data and empower your colleagues to make informed decisions every day. This is a new capability we’re developing at Ably, so you’ll have a large influence on the technology and principles we employ to achieve our goals of unlocking all our data.

Day to day, you will:

  • Work closely with all engineering teams to gather and extract data from various internal systems and databases

  • Work with the sales, marketing, support and customer success teams to keep data synchronised between internal systems and SaaS products

  • Ensure our data reaches data warehouses/lakes and is surfaced in our BI, decision making and reporting tools

  • Work with the finance and operations team to to extract valuable data from our existing databases and feed it into decision making/reporting tools

  • Monitor the health of the various data processing pipelines

  • Work alongside all teams in company the to enhance the quality of the data we collect

  • Have a standup with your team to help everyone stay synchronised

  • Perform code reviews on pull requests to help the team ship quality software


It would be great if you have:
  • Architectural design and design patterns. You should have experience making architectural decisions for large data applications, and be well versed and a strong advocate of design patterns, where applicable.

  • Data warehouses/data lakes. Solid understanding of data warehouse and data lake concepts and designs, ideally with some experience of dealing with high volume realtime data.

  • Data synchronisation experience. Not all data pipelines will be used for reporting. Experience with reverse ETL tools or techniques will be helpful for future projects.

  • Hubspot API experience. Hubspot is a key service that Ably relies on for day-to-day operations, and any experience with Hubspot is hugely beneficial.

  • BI/reporting tools. You should have experience with business intelligence & reporting tools, and be able to help us make an informed decision on which tool will empower your colleagues to get the information they need. You should also be able to offer them basic support to get them productive quickly.

  • ETL pipelines. You should have experience using modern ETL tools, or having written/maintained your own, to construct an ETL process that is flexible and responsive to change - on the input and the output phases. As we’re developing this capacity within Ably there will be a lot of change to contend with.

It would be nice if you have:
  • Experience delivering and scaling large data pipelines on AWS, or other cloud providers.

  • Experience gathering data from PostgreSQL, and working with the teams that own the databases to minimise the impact of breaking changes on the data pipelines.

  • Experience gathering data from HTTP APIs, handling errors, networking issues and unpredictable responses without impacting the data pipelines.

  • Experience with Metabase, Looker, etc; helping to unlock data right at the source.

  • Experience in build processes that can easily be inspected and monitored for any health issues.

What’s it like to work at Ably?

Ably is entering a new phase of growth led by an experienced senior team. You’ll be part of a successful, highly motivated, and extremely collaborative team.

To provide the infrastructure required for the next generation of the internet, to tell the world about it, and to support everyone inside Ably to do so we hire people who are ready and capable of taking on challenging problems in a high-growth environment.

We want to work in a company of inclusive people who treat each other well. To really thrive at Ably, you’ll need to believe in and be prepared to do your work in line with our values, which are:

  • Open for all. Open is a mindset. When open, we’re better collaborators, communicators and creators. Whether open source, standards, or protocols, open hearts, minds, and eyes. It’s better when it’s open.

  • Bias for action. We avoid analysis paralysis at all costs. We lean into doing in order to move us forward. When we have enough of what we need to act, we act. 

  • Even the best do better. We want to be the best - but that doesn’t mean we rest when we are. Improvement and growth  - true betterment - is an ongoing journey. One fuelled by curiosity and taking risks. 

  • Technology needs humanity. Our technology means little without people - whether that’s how we treat our colleagues, or how we design developer experiences. Being human matters.

  • Dependable people make dependable things. Dependability is integral to our mission and technology, but being dependable starts with our people. When we care enough to take real ownership of our products and problems - all else will follow. 

  • Bold creativity leads to the greatest rewards.  When viewed through a different lens, what stands in the way can become the way. The bigger the challenge the easier the choice: bold creativity will always move us forward, create better outcomes, and set us apart.

What’s in it for you?
  • A great company culture

  • Share options

  • The chance to learn and grow with a fast-moving company, supported by a learning budget that gives you access to professional training, events, conferences, workshops etc.

  • Remote first: Unless you want to pop into our communal office space in Shoreditch, London, you’ll work from home most of the time, apart from when we have planning sessions and Summits. Travel to the office is at the employee's expense, apart from when we have Ably Summits

  • Your choice of high quality tech kit to work on

  • Home workstation set up budget of £500, plus an annual top-up of £150

  • Regular company away days and social events

  • Friday need-to-know drinks and snacks

  • Private health insurance, including Doctor@Hand

  • Access to private and confidential coaching with Sanctus

  • 24/7 Employee Assistance Programme with AXA

  • A monthly wellbeing budget with Juno

  • 25 days of paid annual leave (33 days including bank holidays)

  • 10 days of paid sick leave (including mental health days) per year

  • Dog friendly office for when you’re in the office!

If you enjoy working as part of a technology company, value open source, and love solving hard problems - then we’d love to hear from you, regardless of whether you fit the job description exactly or not. If in doubt, drop us an email; we’d be more than happy to give you some advice on your application.

We believe our differences as individuals is what makes us great. Our people are at the heart of what we do, and we encourage everyone to be their most authentic self at Ably: we see inclusivity and openness as fundamental to creating long-term success as an organization. We encourage applications from all backgrounds regardless of age, disability, gender, sexual orientation, parental status, race, religion, educational background or neurodiversity. Ably has an amazing opportunity ahead of us, and we want anyone to be able to contribute to that opportunity.