Anybody who has read about the Data space in the last year will have come across the meteoric rise of the Analytics Engineer—a role that sits in the liminal boundary between Data Engineering and Data Analytics. Why are companies willing to give such increases in salaries for candidates with such short exposure to a handful of off-the-shelf tools?
So it begs the question, what is an Analytics Engineer?
“You can think of an analytical engineer as the role that can answer technically hard questions using data engineering as a tool, but understanding data analytics as to the goal.” - Keboola
Of course, those who adopt the Analytics Engineering approach still require some Data Engineering knowledge to build, for example, technologies such as Airflow or Luigi as an Orchestration tool, Analysts to visualise, but what if this can be done much quicker and more efficiently so that the Data can be utilised in real-time with higher accuracy. This intersection is the genesis of Analytics Engineers and the adaptation of the ELT as opposed to your traditional ETL approach (undermentioned).
The premiss of this blog is to explain, for both Data professionals and companies looking to hire, the possibilities of Analytics Engineering.
For Data Professionals:
Recently, I was scrolling through LinkedIn and saw a post that mocked different jargon of the data field, it read “Analytics Engineer = anyone who can use dbt”. Although this was a joke it is hard to not think there could be some basis of truth for anyone who scours Analytics Engineering job adverts, as the likely tools of choice are dbt, Fivetran or Stitch. But this wildly undermines a truly sui generis position within a data function. Keboola outlined the three main responsibilities:
1 - Data pipelining: building clean, end-to-end data sets which can be used by both technical and non-technical users. This becomes increasingly important, in the efficiency goal of most tech lead businesses, of self-service analytics. However, most businesses adopting the Analytics Engineering approach have been keeping some Data Engineers for building pipelines.
2 - DataOps: somewhere between Data Engineering and DevOps, building the infrastructure of the platform. Increasingly, technologically advanced companies are looking for Engineers who can both drive optimisation and efficiency as well as monitor quality, building orchestration and validation. Progressively, many highly advanced Data and Analytics Engineers have varied DataOps-come-Analytics tech stacks: cloud warehousing (Big Query, Redshift or Snowflake), coding in SQL and Python, Kubernetes/Airflow for orchestration, Terraform for infrastructure-as-code, Fivetran or Stitch for the Extraction and Loading, dbt to transform data in warehouses by writing select statements and a modern visualisation tool (Looker).
3 - Data model building: designing models to optimise data warehousing and data consumption. This can include architectural design to analytics.
There is an obvious effect of a small group of engineers using very specific and niche tools: highly saturated demand. Our recent salary report saw the junior end of the Analytics Engineering salary bracket rise by over £10,000, a 22% increase on last year. Whilst senior salaries rose by £15,000 (25%) year-on-year and lead remuneration by £20,000 (again 25%) from 2021 to 2022.
Fundamental to the transition into Analytics Engineering is the ELT approach—whereby the data is in one’s warehouse before it is transformed— which is an easily transferrable skill from the traditional ETL approach and then requires technical acumen to model the raw data into defined datasets.
So what if you want to become an Analytics Engineer? The real requirement is very strong SQL at entry-level roles and beyond that an aptitude to learn. The two routes into the market that we are seeing at Burns Sheehan are entry-level roles where these skills are taught early in one’s career or when candidates upskill from Data Engineering or Analytics over the course of their tenure with a company.
If you’re interested in exploring Analytics Engineering roles, please reach out to me here.
For Data Leaders:
From a business case perspective there is one obvious question, why pay for more tools and higher salaries?
Dbt answered this question themselves “analytics engineers deliver well-defined, transformed, tested, documented, and code-reviewed data sets. Because of the high quality of this data and the associated documentation, business users can use BI tools to do their own analysis while getting reliable, consistent answers”. Put simply if your business would like to approach data in the most forward-thinking, efficient and ultimately more cost-efficient way possible, it’s time to hire an Analytics Engineer.
Unlike the traditional model within a data function, with an initial hire followed by successive Engineers and Analysts, the unique role of an Analytics Engineer means that you can scale to a very advanced data function without many hires due to the role sitting on the cusp between Engineering and Analysts. Of course, it is impossible to give an exact figure on cost savings but consider your data function and the benefit of having a single person who could: take ownership of multiple tools (Fivetran, Stitch, dbt), explain data to stakeholders, and, enable you to achieve self-service analytics—how much do you think you could save?
Lastly, if a company can achieve self-serve analytics—whereby non-technical employees are enabled to create their own dashboards with the data set on-demand in a very user-friendly way in the most efficient and advanced way, which will enable your business to turn their data from cost to a profit centre with live dashboards at your fingertips; then Analytics Engineering is for you.
As the Burns Sheehan Data team delve further into the world of Analytics Engineering, the constant debate we seem to have is why, and to what level, continue to hire Data Engineers, when an Analytics Engineering approach means leaner teams, fewer errors, more efficiency, more automation & simply, a cheaper workforce? We’d love to hear from you!
If you’re interested in hiring Analytics Engineers, please reach out to us here - data@burnssheehan.co.uk
