We are looking for an experienced PySpark Engineer to join a project focused on migrating and developing ETL/ELT processes within Palantir Foundry, which will serve as the organisation’s primary data lake platform.
Key information
- Start date: 1 October 2026
- Initial duration: Q4 2026, with a possible extension into 2027
- Engagement: Full-time
- Contract: B2B
- Location: Fully remote
- Methodology: Scrum with three-week sprints
Responsibilities
- Designing, developing and optimising ETL/ELT pipelines using PySpark and Python
- Building end-to-end data flows from source systems to data lake structures
- Migrating existing data processes to Palantir Foundry
- Developing scalable and reliable data engineering solutions
- Monitoring and improving pipeline performance
- Ensuring data quality and processing reliability
- Creating and maintaining technical documentation
- Estimating tasks and delivering assigned tickets on time
- Collaborating with data engineers, architects and business stakeholders in an international Scrum team
Requirements
- Commercial experience with PySpark and Python
- Strong background in data engineering
- Hands-on experience designing, developing and optimising ETL/ELT pipelines
- Experience building data flows from source systems to data lake environments
- Good understanding of distributed data processing
- Experience working with data lake platforms
- Ability to create clear technical and code documentation
- Experience working in Scrum teams
- Familiarity with Azure DevOps and/or Jira
- Experience using Confluence
- English proficiency allowing effective collaboration in an international environment
- Ability to work independently and take ownership of assigned tasks
Nice to have
- Hands-on experience with Palantir Foundry
- Experience migrating data pipelines between platforms
- Previous involvement in large-scale data lake transformation projects
What we offer
- A fully remote B2B engagement
- Full-time participation in an international data transformation project
- The opportunity to work with modern Big Data technologies and Palantir Foundry
- Potential contract extension into 2027
- Collaboration with an experienced, international project team