For our client in the financial sector, we are looking for an experienced Spark / Data Lakehouse Developer to join the Data Platform team.

The person in this role will be responsible for designing and developing data reconciliation and synchronization solutions within the ODS (Operational Data Store) layer, based on MongoDB, as well as building and maintaining data layers within the Data Lakehouse architecture — from raw data through Curated, Silver, and Gold layers.

Key Responsibilities

• Design and implementation of Spark SQL/PySpark processes for data validation and comparison (MongoDB/ODS).
• Implementation of data quality rules, duplicate, anomaly, and missing data detection, as well as reporting and alerting.
• Integration and optimization of Spark–MongoDB workflows (MongoDB Spark Connector, partitioning, push-down, schema-aware reads).
• Design of Raw → Bronze → Silver → Gold data layers using Delta Lake/Iceberg/Hudi, including CI/CD for schemas and migrations.
• Monitoring and observability of Spark/batch/streaming processes (Prometheus, Grafana, OpenTelemetry, Dynatrace) and definition of SLAs/SLOs.
• Automation of Spark application deployments using Docker/Kubernetes (Spark Operator).
• Troubleshooting data quality, performance, and processing latency issues.
• Creation and maintenance of technical documentation, runbooks, and data flow diagrams.
• Collaboration in Scrum/Kanban teams with Data Engineers, Business Analysts, QA, and DevOps; conducting workshops and code reviews.