You're using an older version of Internet Explorer that is no longer supported. Please update your browser.
TD

Data Engineer

TD
Location
Markham, ON
Details
Full Time
6 days ago
Company Overview

Tell us your story. Don't go unnoticed. Explain why you're a winning candidate. Think "TD" if you crave meaningful work and embrace change like we do. We are a trusted North American leader that cares about people and inspires them to grow and move forward.

Stay current and competitive. Carve out a career for yourself. Grow with us. Here's our story: jobs.td.com

Department Overview

Building a World-Class Technology Team at TD

We can't afford to be boring. Neither can you. The scale and scope of what TD does may surprise you. The rapid pace of change makes it a business imperative for us to be smart and open-minded in the way we think about technology. TD's technology and business teams become more intertwined as new opportunities present themselves. This new era in banking does not equal boring. Not at TD, anyway.

TD Data and Information Management covers building, developing, designing, and implementing involved with tech data management solutions. Where do we begin? These experts analyze, design, develop and support TD's info management system; lead and support bank data curation; develop data road map and info management strategies; focus on complex data integration, and drive use of our data reference model. This area doesn't work in isolation. We love data and we feel the same about integration and our assignments prove it.

There's room to grow in all of it.

Job Description

We are seeking highly motivated and capable technologists to join our Data as a Services team to work on the forefront of enabling TD's big data strategy with a special focus on ETL/data curation on on-premise Akora and/or on-cloud Azure Platform.

Across multiple large-scale transformation programs and in a data management context, reporting to the curation practice lead, the developer will lead team members to develop ETL processes, design and build data stores and transformations on a Hadoop platform, coordinate team member's work into an integrated high quality product.

As a Data Engineer, you will:
  • Design and develop ETL processes based on functional and non-functional requirements in hive / python / pyspark within Hadoop platform
  • Design and implement component execution orchestration in Oozie
  • Understand the full end to end development activities from design to go live for ETL development and Cloudera Hadoop ecosystem
  • Recommend and execute improvements
  • Document component design for developers and for broader communication.
  • Understand and adopt an Agile (SCRUM like) software development mindset
  • Follow established processes/standards, business technology architecture for development, release management and deployment process
  • Execute and provide support during testing cycles and post-production deployment, engage in peer code reviews.
  • Elicit, analyze, interpret business and data requirements to develop complete business solutions, includes data models (entity relationship diagrams, dimensional data models), ETL and business rules, data life cycle management, governance, lineage, metadata and reporting elements.
  • Apply automation and innovation on new and on-going data platforms for those development projects aligned to business or organizational strategies.
  • Design, develop and implement reporting platforms (e.g. modeling, ETL, BI framework) and complex ETL frameworks that meet business requirements.
  • Deliver business or enterprise data deliverables (that adhere to enterprise frameworks) for various platforms/servers/applications/systems.
  • Provide data analysis and requirements within enterprise platform (e.g. Hadoop).
  • Implement processes aligned to data management standards, ensure data quality and requirements are embedded in/adhered to within system development deliverables.
  • Develop, maintain knowledge of data available from upstream sources and data within various platforms.
  • Perform data profiling using TD tooling and ad hoc system query languages to validate data analysis.
  • Engage with data managers, stewards and owners to implement, maintain, enhance data governance, quality and related policies, aligning with TD's standards and frameworks.
  • Ensure capture of requirements related to data privacy and security, governance and access controls.
  • Respect TD's technology delivery practices, standards, project management disciplines.


Requirements
  • Undergraduate Degree or Technical Certificate.
  • 5-7 years relevant design and development experience.
  • Senior work experience (>5 years):
    • Big Data technologies
    • Data Flow Processes
    • SQL Development
    • ETL
  • Preferred experience in:
    • Hive, Python, Pyspark, Spark, Sqoop
    • Experience with scheduling or orchestration tools such Oozie, Autosys
  • Working experience with data modeling, relational modeling and dimensional modeling.
  • Good understanding of Azure platform and tools like Azure Data Factory, Azure Databricks
  • Working knowledge of source code control tool such as GIT
  • Implementation experience in managing and working in multiple environments, release and change management and knowledge of firewall, network work protocols, file transfer - TIBCO
  • Familiar with Agile development methodologies
  • Sound to advanced knowledge of business, standards, infrastructure, architecture and technology from a design/support/ solutions perspective
  • Readiness and motivation (as an experienced developer and subject matter expert) to address and resolve complex issues, guide/advise/support clients, partners and project teams, often working on multiple medium-to-large sized projects.
  • Commitment to and belief in the quality of your deliverables.
  • Capacity and eagerness to work independently as a senior/lead role on multiple tasks and also coach/educate/guide/direct others.
  • Ability to assume assignments that are moderate- to highly- complex and multi-faceted, to be performed under management guidance.


Hours
37.5

Inclusiveness

At TD, we are committed to fostering an inclusive, accessible environment, where all employees and customers feel valued, respected and supported. We are dedicated to building a workforce that reflects the diversity of our customers and communities in which we live and serve. If you require an accommodation for the recruitment/interview process (including alternate formats of materials, or accessible meeting rooms or other accommodation), please let us know and we will work with you to meet your needs.
Category
Information Technology