Databricks Data Engineer- Manchester or London

FDM is a global business and technology consultancy seeking a Databricks Data Engineer to work for our client within the global consultancy sector. This is initially a 6-12 month contract with the potential to extend and will be a hybrid role based in either Manchester or London.

Our client is looking for Azure Databricks Data Engineer to design, develop and deliver cloud-based data integration and analytics solutions within their insurance portfolio. You’ll work across data engineering, modelling, and transformation initiatives- building scalable pipelines, optimising performance, and ensuring best practices in data quality and governance. This is a hands-on technical role requiring strong Azure and Databricks experience, coupled with solid insurance domain knowledge.

As part of this role, you’ll have the opportunity to work on high-impact Insurance data transformation programmes, collaborating with experienced data professionals in a forward-thinking, innovative environment.

Responsibilities:

• Design, build, and deploy scalable data pipelines using Azure Databricks, Azure Data Factory, and Azure SQL Database.
• Optimise Databricks environments for performance, cost, and security.
• Develop and maintain data integration, ETL/ELT processes, and data quality frameworks.
• Translate business requirements into data models, ensuring consistency and accuracy across data layers.
• Work with stakeholders to deliver effective data solutions, analytics..
• Create technical documentation including data flow diagrams, architecture design, and test plans.
• Stay current with emerging data engineering and Azure ecosystem technologies to drive continuous improvement.

• Insurance industry experience is essential, with a strong understanding of policy, claims, and regulatory data.
• Minimum of 5 years’ management experience required – leading technical delivery or overseeing project teams within data engineering or integration environments.
• 8–10 years’ experience in data engineering and integration, ideally in enterprise-scale environments.
• Deep expertise in Azure Databricks, Azure Data Factory, Azure Synapse, and Azure Data Lake.
• Hands-on experience with Spark (PySpark/SparkSQL), ETL/ELT design, and data pipeline orchestration.
• Strong data modelling skills (Dimensional, ODS, Data Vault) and experience with data warehousing concepts.
• Experience working with large, complex datasets and distributed computing environments.
• Proficiency in SQL, version control (e.g. Git), and CI/CD pipelines.
• Familiarity with Agile delivery and DevOps practices (Azure DevOps, JIRA, etc.).

Preferred Skills & Experience:

We are a business and technology consultancy and one of the UK's leading employers, recruiting the brightest talent to become the innovators of tomorrow. We have centres across Europe, North America and Asia-Pacific, and a global workforce of over 3,500 Consultants. FDM has shown exponential growth throughout the years, firmly establishing itself as an award-winning employer and is listed on the FTSE4Good Index. 

Diversity and Inclusion  

FDM Group is an equal opportunity employer, and all qualified applicants will receive consideration for employment without regard to race, colour, religion, sex, sexual orientation, national origin, age, disability, veteran status or any other status protected by federal, provincial or local laws.