Data Engineer
Company description:
SMRT Corporation Ltd is a public transport service provider. Our primary business is to manage and operate train services on the North-South Line, East-West Line, the Circle Line, the Thomson-East Coast Line and Bukit Panjang Light Rail Transit.This is complemented by our bus, taxi and private hire vehicle services.
Our core values are Respect, Integrity, Safety and Service, and Excellence. We are committed to provide safe, reliable and comfortable service for all our commuters.
Job description:
Job Purpose
The Data Engineer will be part of the team to develop operation & maintenance decision-support tools to enhance train reliability and maintenance efficiency. This position involves designing, developing, and maintaining data pipelines, APIs, and cloud infrastructure for various rail-oriented applications.The ideal candidate will have expertise in data analysis, transformation, ingestion, database design, API development, and preferably, cloud infrastructure setup. Collaborating closely with software engineers, data scientists, and frontend developers, the Data Engineer will contribute to building efficient, scalable, and reliable systems.
Responsibilities
The duties and responsibilities for Data Engineer, are as listed below. The list is not comprehensive and related duties and responsibilities may be assigned from time to time.
Data Engineering & Processing:
- Develop and maintain data pipelines for efficient data ingestion and transformation.
- Work with structured and unstructured data to ensure optimal storage and retrieval.
- Perform data analysis and report on results.
Database Design & Management:
- Design and implement relational and NoSQL database schemas for scalability.
- Optimize database performance through indexing, partitioning, and query tuning.
- Implement data security and compliance best practices.
API Development & Backend Engineering:
- Design and develop APIs for data access and application integration.
- Implement authentication, authorization, and API security best practices.
Cloud Infrastructure & Deployment (Supporting Role):
- Assist in design Azure cloud architectures
- Work with IT infrastructure team to set up cloud infrastructure for application hosting, data storage and processing.
Collaboration & Best Practices:
- Collaborate with internal stakeholders to understand their business needs.
- Work with software engineers, data scientist, frontend developer to understand the data requirement and design architecture of the data platform.
- Implement CI/CD pipelines for automated testing, deployment and monitoring.
- Write testable and maintainable code and documentation to deploy to production.
- Engage continuously with end-user for feedback and improvements.
- Degree in Science, Technology, Engineering or Mathematics (STEM)
- Previous experience as a data engineer or in a similar role
- Data engineering certification is a plus
- Knowledge of security best practices in cloud and database management is a plus
Skills
Technical skills include:
- Programming and Data processing: MATLAB, Python, SQL, or similar languages.
- Databases: My SQL, SQL Server, MongoDB or similar.
- Cloud Platforms: Azure
- DevOps & CI/CD: Git Lab CI/CD, Docker
Generic skills include:
- Strong inclination and eager for continual learning and development
- Strong team player
- Critical thinking and problem-solving skills
- Ability to understand and explain complex data and effective interactions with the stakeholders
- Ability to think independently and actively propose solutions to the team.