Scala - Spark Developer - New York

This position requires the successful candidate to work on a W2 directly with FDM. We cannot accept C2C, 1099 or employment sponsorship (e.g. H1-B) for this position.

FDM is a global business and technology consultancy delivering client and industry driven solutions through our 5 core specialist Practices; Software Engineering, Data & Analytics, IT Operations, Change & Transformation, and Risk, Regulation & Compliance.

FDM is seeking an Expereinced Scala Spark Developer located in New York City to support a project in the Financial sector. Involvement in this project is anticipated to last initially 1 year but may be extended.

This role will be hybrid with requirements to be in office 3 days per week.

Overview

We are looking for a Scala - Spark Developer to join our regulatory reporting data engineering team. The application processes very large datasets, with business logic implemented using Scala/Spark workloads on Databricks and Snowflake as the core data platform. The selected candidate will work closely with senior engineers to develop data pipelines, implement controls, and support testing and operational activities.

What you'll do in the role:

• Assist in developing and enhancing Spark-based data pipelines written in Scala on Databricks

• Implement and maintain business logic for regulatory reporting under guidance of senior team members

• Develop automation utilities and operational controls using Python

• Work with datasets stored in Snowflake including data transformations and validations

• Support batch workflow setup and maintenance using Autosys scheduler

• Write and maintain BDD test cases and Python unit tests

• Participate in code reviews, design discussions, and Agile ceremonies

• Provide basic support during production issues and assist in root-cause analysis

• Ensure good documentation, coding standards, and quality practices

What you'll bring to the role:

• Bachelor’s degree in Computer Science, Engineering, MIS, or a related field

• 4+ years of experience in software development or data engineering

• Strong foundational knowledge of Scala and familiarity with Apache Spark

• Understanding of cloud data platforms (Snowflake preferred)

• Database knowledge and core database concepts(SQL, indexing, joins, transaction, query optimization)

• Ability to understand and debug spark application(DAGS, Job, Stages, Failure)

• Understanding of spark job optimization techniques

• Working knowledge of Python for scripting, automation, and testing

• Exposure to BDD frameworks and writing unit tests

• Strong understanding of SDLC, coding practices, and Agile methodology

• Good communication skills and willingness to learn complex financial datasets

Desired Skills:

• Basic understanding of Trade Lifecycle concepts

• Awareness of Equities and Options asset classes

• Exposure to Front Office (FO) trade/order data

• Experience with Unix/Linux commands and shell scripting
  Familiarity with batch job schedulers (Autosys is a plus)

• Basic exposure to Generative AI use cases

• Experience with large-scale data environments

Why Join Us?

• Hands-on learning with modern tech stack: Scala, Spark, Databricks, Snowflake

• Opportunity to work on large-scale, high-impact regulatory reporting systems

• Strong mentoring and skill-building environment with senior engineers

• Collaborative & respectful culture focused on growth, learning, and continuous improvement

FDM powers the people behind tech and innovation. We spot trends, find top talent, and help businesses stay ahead.

With 35+ years of experience, we coach, mentor, and launch fresh thinkers from diverse backgrounds into world-class careers. Partnering with top global companies, we deliver the right talent at the right time—while guiding our people toward exponential growth.

🌍 Global impact – 19 centers across North America, APAC, the UK, and Europe

🚀 25,000+ careers launched – and counting

🤝 300+ trusted client partners

Committed to Diversity, Equity, and Inclusion

Tech careers should be for everyone. With 80+ nationalities represented, FDM thrives on diversity, fuels innovation through unique perspectives, and celebrates success together. As an Equal Opportunity Employer and FTSE4Good-listed company, we ensure every qualified applicant gets a fair shot—no barriers, just opportunities.

Additional Considerations

FDM Group, Inc. is registered to operate and hire employees in select states within the US. We will consider employment applications exclusively from candidates who are either residing in one of the following states or willing to relocate to them: Arizona, California, Colorado, Delaware, Florida, Georgia, Illinois, Indiana, Massachusetts, Maryland, Minnesota, North Carolina, New Jersey, New York, Pennsylvania, Tennessee, Texas, Utah, and Virginia.