Inspiry #1 - Data Analyst vs Data Engineer vs Data Scientist - What are the differences?

That is right. It is time for us to delve into similarity and difference between the well-known data roles by having a closer look at different business expectations, responsibilities, skillset required and so on. This can potentially help to find our next job or identify which new expert your team needs. The following diagram explores the key differences between some of the most promising career pathways in the industry.

When overviewing the key differences above, several insights immediately come to mind:

1️⃣ In the context of a business, every individual role assumes a crucial function in facilitating the utilisation of data to drive business operations. These roles are instrumental in extracting the inherent value of business data at various stages of processing. As a business progresses and advances in its data-driven practices, the significance of each role becomes increasingly pronounced, particularly when the business endeavours to go beyond conventional "linear" analytics.

Business leaders today increasingly recognise that a shift is underway — from linear to non-linear processes, meaning that every stage of data is accessible throughout its processing.

2️⃣ Data engineers ingest enterprise data and transform them in a consumable format so that others can utilise for business operations. They live with engineering principles and practices such as continuous improvement / development (CI / CD) at heart.

3️⃣ Data analysts answer business questions raised during the BAU processes by organising data that are ingested by engineers and presenting them. They are data storytellers specialised in curating business data at heart.

4️⃣ Data scientists find patterns (or correlations) and making prediction (or forecasting) with feature data. They are researchers at heart as many of them have the high-level academic background and take the scientific approach of observation, hypothesis and testing to build a model.

5️⃣ Data analysts / scientists require the certain level of domain knowledge (or business expertise) against business functions they work with, to translate business questions into data questions and effectively communicate with stakeholders. On another note, data engineers are in need of understanding on application and service to keep their process well-managed.

6️⃣ A growing number of large organisations starts introducing data platforms such as Databrick and Snowflake, where enterprise data can be centralised and accessed through data tools and all three roles can carry out work in one environment.

7️⃣ The roles have a particular set of skillset in common, which are SQL and Python. Something to note for everyone 😊

8️⃣ It is common that each role wears more than one “hat“ to cover more than one duty in small medium businesses (SMB). For instance, a data engineer might play as a data analyst while supporting the ad-hoc data query to enrich the marketing performance data with customer infomation to understand demographical profile.