Big Data Platform Engineer

Company Overview

Tell us your story. Don't go unnoticed. Explain why you're a winning candidate. Think "TD" if you crave meaningful work and embrace change like we do. We are a trusted North American leader that cares about people and inspires them to grow and move forward.

Stay current and competitive. Carve out a career for yourself. Grow with us. Here's our story: jobs.td.com

Department Overview

Building a World-Class, Diverse and Inclusive Technology Team at TD

We can't afford to be boring. Neither can you. The scale and scope of what TD does may surprise you. The rapid pace of change makes it a business imperative for us to be smart and open-minded in the way we think about technology. TD's technology and business teams become more intertwined as new opportunities present themselves. This new era in banking does not equal boring. Not at TD, anyway.

The TDS Veritas Platform is the cross-asset pricing and risk management platform for TD Securities. TDS Veritas is also the strategic storage and compute infrastructure for numerous business-aligned products used by front office users as well as risk management functions in the Investment Bank.

These products include many user-facing applications used for real-time risk, scenario risk, electronic trading, and scalable data analytics and are utilized by a variety of users including traders, desk managers, risk managers, and sales teams.
Our team is made up of a number of technology and software engineering enthusiasts coming from a variety of diverse backgrounds and industries. Last year we welcomed the third Computer Science PhD to the team which further solidified our commitment to thought leadership and excellence in Computer Science and Software Engineering.

We participate in a number of industry events and engage with a variety of companies to find relevant and modern solutions to technical and business challenges that we face. In the past year various team members took part in:

• OSCON - Open Source Convention
• Strata + Hadoop World
• QCon
...and dozens of local and regional meetups that we attended as a team.

We also collaborated with Google and IBM to build innovative prototypes that used Natural Language Processing, Machine Learning, and Cloud Computing to demonstrate how we can utilize these technologies to build and deliver solutions tailored to the business challenges that we face.

We were really excited to win the bank-wide hackathon as well as the TDS Shark Tank challenge and show that we can be relied on as a source of good ideas as well as executing on those ideas when needed.

We also know how to have fun by organizing and participating in various social events. In the last year alone we organized team trips to the driving range, bowling, baseball games, ski trips, and of course: frequent trips to the bar.

There's room to grow in all of it.

Job Description

About This Role

We are looking for someone to manage a team of specialized engineering professionals in developing and implementing solutions that align with business needs, technology strategies and emerging industry standards. Planning, evaluating, recommending, designing, operationalizing, and supporting solutions - the list goes on! There's meaningful work everywhere you turn.

As a Senior Platform Engineer, you will be to lead and collaborate with the team to architect, design and implement solutions to help solve complex Big Data problems. We are a team that builds generalized tools to empower our users to have easy access to large sets of data.

Meaningful work is fueled by meaningful performance and career development conversations with your manager. Here's some of what you may be asked to perform:

• Draw on your deep technology expertise to ensure that all TD areas that provision, manage and support various technologies have what they need (tools, processes and documentation) to operate effectively.
• Execute on Engineering strategy by introducing tools and automating build, test, release and configure activities across applications, platform and infrastructure.
• Partner with the Operations team to automatically integrate with the right tools and processes as part of automated/self-serve application, platform or infrastructure releases.
• Act as the expert or lead innovator and 'change agent' for programs or services under management.
• Anticipate internal/external business and regulatory issues; recommend process/product/organizational improvements.
• Manage resource requirements and assignments, make sure project milestones are met, schedules are on track and within budget.
• Oversee business expenses and FTE plans and ensure controls are in place to monitor, effectively track and manage to established plans.
• Maintain knowledge and understanding of external development, engineering and emerging solutions, market conditions and their impact.
• Provide overall team leadership, training, coaching, development, succession, recruitment and resource management and provide on-going feedback and input into performance assessment and development plans for each team member.
• Clearly define and communicate strategies, objectives and accountabilities to team members and continually look for ways to boost value delivered to business and internal partners.
• As a lead contributor, you may participate in business strategy development.

Requirements

What can you bring to TD? Share your credentials, but your relevant experience and knowledge can be just as likely to get our attention. We are looking for a senior platform engineer to help build out our BI and analytic capabilities on top of our existing operational data stores.

Technology

The technology stack that is in place & evolving today:

• University or Post-Graduate Degree.
• Proficient operating Cloudera/Hortonworks Hadoop, HDFS/HBase/Yarn/Ranger/Spark/ZK, Ambari, Dremio, Druid, Airflow.
• A strong grasp of fundamental concepts of Linux (Redhat), data engineering, network, warehouses, operational databases, messaging and data management.
• Good knowledge with Azure Databricks, Azure Data Factory, ADLS.
• Working knowledge of data migration from HDFS to ADLS using ADF.
• Experience with configuration management tools such as Salt, Ansible.
• Strong scripting skills with experience of platform automation using python/bash with version control system Git/Bitbucket.
• Good knowledge with building automation and tooling for services monitoring using Datadog.
• A mindset to ensure stability of production environment, applying software engineering solutions to run/manage applications and work alongside application teams to troubleshoot and guide best practices.
• Strong experience supporting and working with cross-functional teams in a dynamic business environment.
• Required to be highly creative and collaborative.
• The ideal candidate will be expected to collaborate with both the IT teams to define the business problem, refine the requirements and design and develop data deliverables accordingly.
• The successful candidate will also be required to have regular discussions with data consumers on optimally refining the data pipelines developed in nonproduction environments and deploying them in production.
• Ability to gain the respect of stakeholders at all levels and roles within the company.
• Confident, energetic self-starter, with strong interpersonal skills.
• Good judgment, a sense of urgency and has demonstrated commitment to high standards of ethics, regulatory compliance, customer service and business integrity.
• A bachelor's or master's degree in computer science, statistics, applied mathematics, data management, information systems, information science or a related quantitative field or equivalent work experience is required.
• Candidate must be located within commuting distance of Toronto, Ontario or be willing to relocate to the area.
Desired Qualifications

• Experience in designing, implementing, and supporting highly scalable data systems in Scala and/or Java.
• Experience with Hadoop-ecosystem technologies in particular MapReduce, Spark / Spark-SQL / Spark Streaming, Hive, YARN.
• Experience building and running large-scale data pipelines, including distributed messaging such as Kafka, data ingest to/from multiple sources to feed batch and/or near-real-time/steaming compute components.
• Experience in data-modeling and data-architecture optimized for big data patterns. (i.e., warehousing concepts; efficient storage and query on HDFS; data privacy and security techniques).
Desired Interpersonal Skills

• Takes great personal pride in building robust software.
• Strong sense of ownership.
• Passionate about programming and computer science.
• Enjoys working in a fast-paced environment.
• Has excellent written and verbal communication skills.
• Has strong customer focus

Additional Information

Join in on what others in TD Technology Solutions are doing:

• Inspire a positive work environment and help champion quality, innovation, teamwork and service to the business.
• Learn voraciously, stretch your thinking, share your knowledge and educate others.
• Communicate and collaborate with both technical and non-technical professionals.
• Cultivate winning relationships by building trust with business and technology partners.
• Share our commitment to productivity, effectiveness and operational efficiency.

Hours
37.5

Inclusiveness

At TD, we are committed to fostering an inclusive, accessible environment, where all employees and customers feel valued, respected and supported. We are dedicated to building a workforce that reflects the diversity of our customers and communities in which we live and serve. If you require an accommodation for the recruitment/interview process (including alternate formats of materials, or accessible meeting rooms or other accommodation), please let us know and we will work with you to meet your needs.

Category

Information Technology