Senior Data Engineer - Databricks Expert

Senior Data Engineer - Databricks Expert

About the Role

Join our dynamic team as a Senior Data Engineer specializing in Databricks! We are on the lookout for a Databricks Champion who will take the lead in designing, building, and optimizing sophisticated data pipelines. You will work collaboratively with clients and our internal teams to deliver scalable and efficient data solutions that cater to diverse business needs.

Key Responsibilities

• Develop robust ETL/ELT pipelines utilizing Databricks and Delta Lake.
• Integrate and process data from a variety of sources for comprehensive analytics.
• Work closely with data scientists, architects, and analysts to implement strategic data solutions.
• Optimize performance and effectively manage Databricks clusters for maximum efficiency.
• Create innovative cloud-native solutions (with a preference for Azure, but AWS/GCP experience is also welcome).
• Establish and uphold data governance and quality best practices.
• Automate workflows and support continuous integration and deployment (CI/CD) processes.
• Thoroughly document architecture, processes, and data flows for team reference.

What We're Looking For

Required:

• 5+ years of experience in data engineering, with a strong focus on Databricks.
• Recognized Databricks Champion Status (Solution Architect / Partner).
• Expertise in Databricks, Delta Lake, Spark, Python, and SQL.
• Cloud experience, particularly with Azure; AWS or GCP experience is an added advantage.
• Outstanding problem-solving capabilities and excellent communication skills.
• Proven track record as a Databricks Champion.