Azure Data Engineer - Jobs in Toronto, Ontario, Canada, M4C

Job Description

Resource Profile

• Work with technology and business teams to provide technical leadership in planning, analysis, building, configure, testing, and post implementation support of Azure Cloud data technologies such as Azure Data Factory, Databricks, Azure Synapse and/or SQL Data Warehouse. Experience with enterprise ETL and data integration tooling such as IBM Infosphere Information Server DataStage solution is a plus.
• Provide support for Enterprise Data Warehouse operations (e.g. daily data refreshes of mirror and warehouse components) to ensure the delivery of timely, accurate and reliable data to CLIENT business users. This includes monitoring the update processes, updating/maintaining the appropriate system documentation, identifying potential problems and implementing solutions to ensure the efficiency and reliability of the IM operations.
• Maintain the ETL environments by monitoring their performance to ensure functionality, Integrity and security of the environments and compliance with BTS information security and change management processes. Identify and troubleshoot problems as they arise which may involve interacting with other areas within BTS for resolution
• Monitor and administer automated and manual data integration and ETL jobs to verify execution and measure performance
• Work with management to provide technical assistance, advice and guidance to client IT staff and business clients in response to enquiries on the functionality of BI/ETL tools, and data in warehouse and related data marts to ensure efficient and uninterrupted IM services to the
• business users.
• Candidate should be conversant with Cloud ETL methodologies and job design with a sound
• knowledge and experience of integrating input/jobs from disparate systems and be able to maintain the CLIENT #39;S standard of ETL best practices
• Worked on Data warehousing projects and involved in supporting data modelling activities as part of building the data warehouse in Azure. Strong understanding of data modeling (i.e. conceptual, logical and physical model design, experience with Operation Data Stores, Enterprise Data warehouses and Data Marts).

Resource Responsibilities

• Provide technical design, developing and implementing IT solutions
• Translate client business systems requirements and associated functional designs into technical design documentation, ensuring that the solution(s) satisfy all business requirements.
• Work with Analysts, Administrators, and Testing teams to support configurations, code customizations, code migrations, and testing from Unit, System, Integration, User Acceptance and performance.
• Participate in technical sessions to prototype new system functionality.
• Participate in workshops with project teams to explain and gain consensus on the solution(s) to be developed.
• Work with systems analysts to ensure the solution meets the approved solution design.
• Perform code reviews to identify potential issues.
• Perform unit testing and troubleshoot coding issues.
• Provide support during systems integration testing.
• Provide knowledge transfer and create/update technical documentation.
• Provide coordination in release and change management processes

RequirementsKnowledgeamp; SkillsMUST HAVE:

• 3+ Years of experience in designing, developing, testing and implementing Extract/Transform/load (ETL) data pipelines with Azure data technologies such as Azure Data Factory, Databricks, Azure Synapse, and/or SQL Data Warehouse
• 4+ years’ experience in data modeling and working with common data warehouse Models
• 4+ years’ experience doing analysis and developing with SQL (e.g. Oracle, MS SQL, DB2) and PySpark.
• 5+ years experience working DevOps tools and practices
• 5+ year experience working with accepted methods, standards, techniques and best practices for development in agile/scrum practices
• Oral communication skills to work effectively with and discuss development approaches to clients/senior management and external stakeholders; written communications skills to document and communicate development specifications
• 2+ years’ development experience with PeopleSoft V 9.1, PeopleTools V8 with data integration and ETL experience with relevant application design, development and testing experience

NICE TO HAVE:

• Good knowledge of popular data standards and formats (e.g, JSON, XML, Proto, Parquet, Avro, ORC)
• Experience working with high volume data exchange and transaction processing systems Experience writing well documented unit tests and executing them
• Experience of reverse-engineering data mappings based on application-layer code highly desirable Ability to express complex technical concepts effectively, both verbally and in writing
• Strong understanding of SDLC and Agile Methodologies
• Degree in computer science, math, engineering or equivalent work experience Excellent Communication Skills