Toa Payoh - Manager - Data Engineering - APAC

Manager - Data Engineering - APAC

Are you excited about transforming data into actionable insights? Join DP World as a Data Engineer and become a key player in designing, building, and maintaining cutting-edge data pipelines that drive our business forward.

In this role, you will work closely with stakeholders to understand their data needs, ensuring that data is accurate, secure, and privacy compliant. Enjoy a stimulating work environment, abundant opportunities for professional growth, and the chance to contribute to industry-leading data initiative

Key Accountabilities

Data Integration and Transformation:

• Design and implement robust data ingestion techniques for both real-time and batch processing of structured and unstructured data.
• Build scalable and reusable data pipelines using Microsoft Azure Stack, ensuring seamless integration with various data sources.
• Utilize Azure Data Factory (ADF) to extract, transform, and load (ETL) data across platforms such as Azure SQL, Blob Storage, and Data Warehouse.

Cloud Solutions Design:

• Architect and develop modern data solutions leveraging Azure IaaS and PaaS services to meet business needs.
• Apply expertise in cloud migration and modern software design principles.
• Collaborate with stakeholders to identify business data requirements and translate them into secure, scalable solutions.

Big Data and Distributed Systems:

• Develop and deploy Spark applications using Databricks, focusing on Spark SQL for data extraction, transformation, and aggregation from various file formats.
• Ensure efficiency in data analysis and uncover insights into customer usage patterns using advanced Spark Architecture components, such as Core, SQL, DataFrames, and Streaming.
• Deploy distributed messaging systems like Kafka and integrate them with other Azure services.

Monitoring and Security:

• Manage, monitor, and ensure the privacy and security of data within the Azure ecosystem to comply with business and regulatory requirements.
• Utilize Debezium for CDC (Change Data Capture) from various database sources and integrate with Event Hubs.

Lifecycle and Project Expertise:

• Work on projects across all phases of the lifecycle, including design, analysis, implementation, and testing.
• Analyse the current production state of application to determine the impact of new implementations on existing business processes.

Visualization and Insights:

• Build solutions using Azure services to support data visualization and analytics.
• Support stakeholders by providing insights through data transformations and dashboards.

• Bachelor Degree/Masters in Computer Science, Information Technology, or a related field
• Certification in Databricks will be a plus (Not mandatory
• At least 5 years of experience in Data Engineering with a strong track record in designing and implementing data solutions
• Hands-on experience in PySpark, Databricks (will be good), SQL and Spark Architecture
• Experience with multiple cloud platforms, including Azure, AWS, and Google Cloud, to ensure flexibility and adaptability
• Proficiency in Azure ecosystem: Data Lake, Data Factory, Databricks, Event Hubs, and Purview.
• Experience with Azure databases: SQL Database, Synapse, and SQL Data Warehouse.
• Knowledge in ETL will be a plus
• Strong expertise in Azure Analytics Services
• Programming skills in Scala, Python, R, and Spark SQL
• Experience with message queuing systems such as Kafka or Azure MQ is a plus
• Knowledge of data visualization tools like Power BI is a plus
• Familiarity with data migration processes is a plus

Please note: The job title is being used for the purpose of Talent Attraction and job advertisement.

Equal Employment Opportunity (EEO)

DP World is committed to the principles of Equal Employment Opportunity (EEO). We strongly believe that employing a diverse workforce is central to our success and we make recruiting decisions based on your experience and skills. We welcome applications from all members of society irrespective of age, gender, disability, race, religion or belief.

By submitting your resume and application information, you authorize DP World to transmit and store your information in the world-wide recruitment database, and to circulate that information as necessary for the purpose of evaluating your qualification for this or other job vacancies.

About DP World

Trade is the lifeblood of the global economy, creating opportunities and improving the quality of life for people around the world. DP World exists to make the world’s trade flow better, changing what’s possible for the customers and communities we serve globally.

In Asia Pacific, DP World employs more than 7,000 people, and has ports and terminals in 17 locations.

We’re rapidly transforming and integrating our businesses -- Ports and Terminals, Marine Services, Logistics and Technology – and uniting our global infrastructure with local expertise to create stronger, more efficient end-to-end supply chain solutions that can change the way the world trades.

What's more, we're reshaping the future by investing in innovation. From intelligent delivery systems to automated warehouse stacking, we’re at the cutting edge of disruptive technology, pushing the sector towards better ways to trade, minimising disruptions from the factory floor to the customer’s door.

WE MAKE TRADE FLOW

TO CHANGE WHAT'S POSSIBLE FOR EVERYONE.

Disclaimer:

There may be fraudulent job offers and recruitment schemes on social media and job boards or communicated by unsolicited emails. DP World will never request any payment or sensitive financial information from job applicants. To explore genuine career opportunities with DP World, please apply only through our official careers page or trusted platforms like LinkedIn, where we are a verified employer.