Overview Join White Bay Recruitment as a Lead Data Platform Engineer in Montreal, where you will play a pivotal role in transforming urban data into actionable insights. Our client is dedicated to making cities more understandable through innovative data solutions, directly impacting how cities are developed and evolved. With substantial funding and a foundation rooted in academic research, this company is now operating on a large scale, harnessing vast datasets across North America to create practical applications that enhance urban living.
Responsibilities In this senior position, you will be at the forefront of enhancing a high-volume data platform that manages tens of terabytes of geospatial and real-world information. Your primary focus will be on refining and optimizing existing systems that drive APIs, analytics tools, and machine learning applications for both internal teams and external clients. You will not only construct data pipelines but also influence how data flows, ensuring reliability and scalability amidst increasing complexity.
Key responsibilities include: - Designing, building, and managing large-scale batch data pipelines and lakehouse datasets (approximately 30TB+). - Ensuring the reliability and cost-effectiveness of data systems, with regular updates across daily, monthly, and quarterly cycles. - Translating diverse requirements from data science, product, and engineering teams into a scalable data framework. - Establishing and maintaining standards for data quality, validation, lineage, and governance. - Enhancing observability, monitoring, and alerting mechanisms across the data platform. - Promoting best practices in software engineering, including testing, performance optimization, and security measures. - Mentoring fellow engineers through code reviews, design contributions, and architectural insights. - Facilitating data delivery into production systems, APIs, and downstream applications.
Requirements To be successful in this role, you should have: - Demonstrated experience in building and managing production-grade batch data pipelines at scale. - Proficiency in Python, emphasizing the creation of clean, testable, and production-ready code. - Familiarity with contemporary data stack tools, such as lakehouses, orchestration frameworks, and distributed processing technologies. - A solid grasp of data modeling, orchestration, observability, and cost optimization strategies. -
Experience in integrating pipelines with production databases while ensuring data integrity. - Knowledge of cloud-native environments (AWS preferred), containerization, and distributed systems. - A passion for leveraging data to create better urban environments.
Compensation The position offers a competitive salary ranging from $140,000 to $150,000 CAD, alongside equity and benefits.
Monday to Friday, 9 AM – 5 PM (full-time)
Submit your application directly to White Bay Recruitment.
🔗 Apply on Employer Site →