Senior AI Data Engineer (Data Lake & GenAI platform)
Direct visibility to CEO / Executive Directors
- Brokerage, Investment, or FSI domains are preferred
- The autonomy to shape our internal tech stack from the ground up
about the company
The company is at a pivotal juncture. They successfully consolidated data from various disparate systems into a central AWS Redshift Data Lake. Now, we are bringing everything in-house, scaling down our reliance on external vendors, and building the future of our financial analytics and AI research platform.
about the job- Take full ownership of our AWS-based data lake (Redshift, AWS Glue), focusing on data mobilization, cleansing, and robust pipeline optimization before feeding downstream financial systems.
- Lead the infrastructure setup for our next-gen AI capabilities. Spearhead the integration of AWS SageMaker (and related tools) and install/plug in local LLMs to ensure absolute data confidentiality and prevent data leakage.
- Create the foundational data grounding mechanisms to eliminate AI hallucination. Work will directly power real-time AI Advisors and Research Bots for our remisier.
- Develop and maintain live streaming pipelines for marketing data and proprietary stock prices.
- Learn our current setup fast, drive existing vendors to deliver, and systematically phase out vendors that aren't adding value.
skills and experience required
Minimum 5+ years of hands-on data engineering experience. Deep expertise in the AWS ecosystem, specifically AWS Glue, Redshift, S3, and Lambda.
Solid understanding of data lake concepts, data cleansing, and building highly reliable ETL/ELT pipelines.
Proven experience building and maintaining Flask APIs to connect enterprise data with applications.
Hands-on experience with AWS SageMaker and orchestrating Retrieval-Augmented Generation (RAG) architectures.
What’s an offer
You will have massive autonomy to take over, modernize, and enhance our AWS infrastructure. Senior AI Data Engineer (Data Lake & GenAI platform) won't just be maintaining code; you will be driving architectural decisions from deploying secure local LLMs to building real-time marketing and proprietary stock price streaming feeds.
To apply online please use the 'apply' function, alternatively you can reach me athttps://www.linkedin.com/in/Oliviatoh-032330132/. (EA: 94C3609 / R22109942).