Lead / Senior Data Engineer (Apache Spark / Flink / High-volume Data)

Highlights:

• Permanent job opportunity
• Great remuneration & benefits
• Great career progression
• Location: Central
• Experience in IceBerg, Apache Flink / Spark, dealing with high-volume data are welcome.

Responsibilities:

• Design and implement scalable batch and streaming data pipelines using technologies such as Apache Flink, Apache Spark, Debezium, and Seatunnel, an open-source data integration framework.
• Ingest and process large volumes of data from diverse sources, including APIs, operational databases, and semi-structured formats (e.g., JSON, CSV, XML, logs), to support a wide range of analytical and operational use cases.
• Develop reusable transformation pipelines to consolidate and standardize cross-domain data (e.g., user behavior, transactions), enabling efficient and reliable analytics-ready data marts.
• Architect and optimize data storage and modeling layers using modern platforms like MinIO/S3, Apache Iceberg, and ClickHouse, ensuring high performance, data integrity, and cost efficiency for OLAP and object-based workloads.
• Manage and maintain a multi-layered data warehouse architecture, including staging, core, and mart layers, aligned with evolving business requirements and data governance standards.
• Implement robust CI/CD processes, data lineage, observability, and compliance using modern data governance tools such as OpenMetadata and DataHub.
• Mentor and support junior data engineers, conduct thorough code reviews, and advocate for engineering best practices across the data organization.
• Collaborate cross-functionally with product managers, analysts, and stakeholders to understand business needs and translate them into scalable, high-impact data solutions.
• Stay up-to-date with the latest data engineering trends, tools, and frameworks, continuously driving innovation and improvement in the data platform and architecture.

Requirements:

• Bachelor’s degree in Computer Science, Information Systems, or equivalent qualification.
• With 5 - 6 years of experience in data engineering, possess leadership experience is a plus point.
• Proficient in Python, Java, and SQL, with strong expertise in schema design, performance tuning, and warehouse modeling.
• Hands-on experience with lakehouse architectures (e.g., Iceberg, Delta Lake), data warehouses (e.g., Hive, ClickHouse), and object storage (e.g., AWS S3, MinIO).
• Strong knowledge of orchestration tools (Airflow, Dolphin Scheduler), ETL/ELT design, and streaming frameworks (Kafka, Flink, Spark).
• Proven experience independently setting up and managing end-to-end data architecture in on-premise environments.
• Demonstrated success mentoring engineers in high-performance, cross-functional teams.
• Familiarity with Git-based workflows, CI/CD pipelines, and observability tools for production-grade data systems.
• Self-motivated with a strong ownership mindset, adaptability, and willingness to travel when needed.

• Last drawn salary
• Expected salary
• Notice period
• Reason for leaving

We regret only shortlisted candidates will be notified.

Please read our privacy statement on our corporate website www.trustrecruit.com.sg.

EA Personnel Reg No: R22106436