We are seeking a Senior Data Engineer to build cloud-based data products that power insurance analytics and reporting. You will design reliable pipelines on Microsoft Azure using Azure Databricks and Python, improve data quality and governance and partner with data and business teams to turn complex datasets into trusted insights.

Responsibilities

Design, build and maintain scalable ETL and ELT pipelines in Azure Databricks
Develop batch and streaming data solutions using Azure Data Factory, Azure Data Lake Storage Gen2, Azure Event Hubs and Kafka
Partner with data architects, analysts and business stakeholders to define requirements and deliver fit-for-purpose data assets
Implement data quality checks, validation, cleansing and governance including Informatica lineage, metadata and cataloging
Apply security and compliance controls including role-based access control (RBAC), managed identities and encryption aligned to Hong Kong Personal Data (Privacy) Ordinance (PDPO) and General Data Protection Regulation (GDPR)
Improve deployment reliability through GitHub Actions automation and solid operational practices
Troubleshoot pipeline issues, perform root cause analysis and drive performance tuning
Maintain clear documentation for transformations, testing and runbooks

Requirements

Hands-on experience building data pipelines with Azure Databricks and Apache Spark (PySpark or Scala)
Strong background with Microsoft Azure data services including Azure Data Factory and Azure Data Lake Storage Gen2
Proficiency in Python for data transformation and automation
Experience working with streaming platforms such as Apache Kafka or Azure Event Hubs
Knowledge of data governance and data quality practices including metadata and lineage tools such as Informatica
Understanding of security and compliance controls including RBAC, encryption and secrets management such as Azure Key Vault
Practical experience with continuous integration and continuous delivery (CI/CD) automation such as GitHub Actions
Clear communication in Cantonese, Chinese and English

We offer

By choosing EPAM, you're getting a job at one of the most loved workplaces according to Newsweek 2021 - 2023
Employee ideas are the main driver of our business. We have a very supportive environment where your voice matters
You will be challenged while working side-by-side with the best talent globally. We work with top-notch technologies, constantly seeking new industry trends and best practices
We offer a transparent career path and an individual roadmap to engineer your future & accelerate your journey
At EPAM, you can find vast opportunities for self-development: online courses and libraries, mentoring programs, partial grants of certification, and experience exchange with colleagues around the world. You will learn, contribute, and grow with us

How We Hire

Here, we summarize the typical journey to finding a job within EPAM.
Apply and tell us about yourself!
Go through some standard interviews:
- General interview with a recruiter
- Technical interview with our technology experts
- Manager interview or Offer interview with a hiring manager
Get ready to join the team!
Not sure if you meet all the requirements? No problem. Let's talk anyway and find out more!
It takes 1 min of application to start the journey with us. Apply now!

Life at EPAM

EPAM Systems Inc. (EPAM) is a leading digital transformation services and product engineering company.  
As the regional headquarters of EPAM in APAC, EPAM Hong Kong was established in 2013. 
EPAM Hong Kong has been growing and emerging rapidly as an international IT consulting company that continuously attracts the top technology and engineering professionals worldwide.
The integration of top-notch consulting services and intelligent engineering back-end systems shape our superior service to the world’s leading clients in various industries from banking and finance to insurance, technology and life sciences.
By choosing EPAM Hong Kong, you're getting a job at one of the Caring Companies by the Hong Kong Council of Social Service for 3 years consecutively since 2021 and  a dedicated, diverse community that will help you discover your fullest potential.

EPAM Systems jobs

Senior Data Engineer (Databricks)

Never Miss a New Opportunity