Career guide

Data Engineer career guide

Build the pipelines and platforms that get reliable data to everyone who needs it.

What does a data engineer do?

A data engineer designs and runs the systems that move data from where it is created into warehouses and lakes where it can be analysed. Analysts and data scientists depend on this work.

The role is closer to software engineering than analysis: writing production code, designing schemas, and making pipelines reliable, fast and cost-effective.

Typical responsibilities

Build and maintain batch and streaming data pipelines
Design data models for warehouses and lakes
Ensure data quality, testing and monitoring
Optimise queries and pipeline cost and performance
Manage access, security and governance of data
Work with analysts and data scientists on what data they need

Skills you need

Technical skills

Advanced SQL and query optimisation
Python for data processing
Data modelling (star schema, dimensional modelling)
ETL/ELT pipeline design
Distributed processing with Apache Spark
Cloud data services on AWS, Azure or Google Cloud
Orchestration (for example Apache Airflow)

Soft skills

Designing for reliability
Clear technical documentation
Working with data consumers
Debugging methodically

Tools and technologies

SQL
Python
Apache Spark
Apache Airflow
dbt
Apache Kafka
Snowflake
Databricks
AWS, Azure or Google Cloud data services

Certifications

AWS Certified Data Engineer – AssociateIntermediate
AWS
Microsoft Certified: Fabric Data Engineer Associate (DP-700)Intermediate
Microsoft · Replaced the retired DP-203
Google Cloud Professional Data EngineerAdvanced
Google Cloud
Databricks Certified Data Engineer AssociateIntermediate
Databricks

Certifications are optional for most roles; skills and projects matter more. Providers retire and rename exams regularly, so check the provider's website before you register. Last reviewed October 2026.

Data Engineer salary

We only publish salary ranges based on verified data, and we don't have a reliable source for this role yet. Meanwhile, if you have an offer or a target CTC, work out what you would actually take home.

In-hand salary calculator Salary hike calculator

Career path

Step 1
Associate Data Engineer
Step 2
Data Engineer
Step 3
Senior Data Engineer
Step 4
Lead Data Engineer
Step 5
Data Engineering Manager or Data Architect

Titles and the time between steps vary by company and individual.

Your first 90 days of learning

First 30 days

SQL, Python and data modelling

Master joins, window functions and query plans in SQL
Write Python scripts that read, transform and load files
Design a star schema for a sample sales dataset

Days 31-60

Pipelines and processing

Build a scheduled ETL pipeline with Airflow
Process a large dataset with Spark
Learn one cloud provider's storage and warehouse services

Days 61-90

Production habits and portfolio

Add data-quality tests and monitoring to your pipeline
Document the architecture of one end-to-end project
Practise explaining trade-offs: batch vs streaming, ETL vs ELT

Interview preparation

Common topics to prepare for data engineer interviews. These are preparation areas, not questions from a specific company's interview.

Advanced SQL and query optimisation
Data modelling: facts, dimensions and slowly changing dimensions
Designing an ETL pipeline end to end
Batch vs streaming processing
Spark fundamentals: partitions and shuffles
Ensuring data quality and handling late data

Company interview guides Take the aptitude assessment Practice questions

Resume guidance

Describe the scale and reliability of systems you built, accurately
Name the stack for each project: sources, processing, storage, orchestration
Show ownership: what you designed vs what you maintained
Link a repository with a documented pipeline project

Is your resume ready for data engineer roles?

Upload your resume to check ATS readiness, matched keywords and what to improve.