Data Engineer (Cloud and DevOps), ETL ,AWS , SQL, Python
Toronto
Roles & Responsibilities
Build, design and implement scalable data lake architecture using AWS S3 and Lakeformation
Build and optimize ETL Pipelines using AWS Glue, EMR and Spark
Implement Event-Driven workflows using EventBridge, SNS and SQS
Design and query datasets using Athena
Manage metadata and data lineage using Amazon Neptune DB
Expose APIs for data subscription and sharing using API Gateway and AWS Lambda
Automate infrastructure provisioning using Terraform and CloudFormation
Ensure data security and compliance by implementing robust IAM policies and access controls.
Develop and maintain a self-serve data portal using Angular and integrate it with backend services
Must Have Technical/Functional Skills
Strong experience in automating infrastructure provisioning using Terraform or CloudFormation (Preferably with Terraform)
Strong Experience with AWS Services: S3, Lake Formation, Glue, EMR, Athena, Lambda, EventBridge, SNS, SQS
Proficiency in Apache Spark for distributed data processing
Familiarity with Amazon Neptune DB for graph-based metadata management
Strong understanding of Data lake architecture, data governance and security best practices
Proficiency in SQL, Python and Spark for Data Engineering tasks
...Position Summary As an EMT, your mission is to maximize effective unit hours to best serve our community with responsiveness and professionalism... ...or 24 hours in duration. Work weeks are from 48 60 hours. Travel Must be able to travel to assigned posts throughout Kings and...
Business Strategy Analyst Senior Location Remote in San Antonio, TX : Why USAA? Let's do something that really matters. At USAA, we have an important mission: facilitating the financial security of millions of U.S. military members and their families. Not all of our employees...
...through automating and streamlining the work of assurance and audit practitioners specifically... ...culture and product, you will find a home at Fieldguide. About the Role: We... ...the audit lifecycle and loves teaching, communicating, and creating clarity in complex...
...of a company that is helping transform the world around us. Make Your Mark at Prysmian! Reporting to the NA Digital Solutions Supply Chain Manager, the Supply Chain Manager is responsible for managing all aspects of the supply chain for the Jackson, Tennessee facility...
...Local MVT position. 6 months exp REQUIRED! Home daily 14 an hour Pay: $670 per week! JUST DELIVERIES TO TEXAS ! Days and night... ...Medical card Minimum 6 monthstractor trailer experience in the last 12 months Valid Class A driver's license NO SAP drivers...