Data Engineer (Cloud and DevOps), ETL ,AWS , SQL, Python
Toronto
Roles & Responsibilities
Build, design and implement scalable data lake architecture using AWS S3 and Lakeformation
Build and optimize ETL Pipelines using AWS Glue, EMR and Spark
Implement Event-Driven workflows using EventBridge, SNS and SQS
Design and query datasets using Athena
Manage metadata and data lineage using Amazon Neptune DB
Expose APIs for data subscription and sharing using API Gateway and AWS Lambda
Automate infrastructure provisioning using Terraform and CloudFormation
Ensure data security and compliance by implementing robust IAM policies and access controls.
Develop and maintain a self-serve data portal using Angular and integrate it with backend services
Must Have Technical/Functional Skills
Strong experience in automating infrastructure provisioning using Terraform or CloudFormation (Preferably with Terraform)
Strong Experience with AWS Services: S3, Lake Formation, Glue, EMR, Athena, Lambda, EventBridge, SNS, SQS
Proficiency in Apache Spark for distributed data processing
Familiarity with Amazon Neptune DB for graph-based metadata management
Strong understanding of Data lake architecture, data governance and security best practices
Proficiency in SQL, Python and Spark for Data Engineering tasks
...A FreshEdge LLC company , is hiring CDL A & B Local Reefer Drivers in Chicago, IL. Founded in 1991, Testa Produce is a wholesale... ...Maintaining proper trip records related to distance/miles and product delivery. Confirm all deliveries are correct with reference to the...
...Job Description: We are seeking a highly motivated Entry Level Data Analyst to join our team. As an Entry Level Data Analyst, you will be responsible for collecting, analyzing, and interpreting large datasets to help our clients make informed decisions. You will work...
...for by taking a minute to finish our online application. Qualifications No experience, Willing to train Ability to work within recognized turnaround times Must have exceptional social skills and the ability to organize simultaneous tasks Ability...
...for over 20 years.Conduent Project Management Office (PMO) has been... ...As the **Project Management Coordinator** , you will be assisting the... ...processes, including:+ Working with the project team.+ Leveraging... ...convenience of working from home and maximize your time by...
...# Can comprehend and utilize blueprints, sketches, drawings, technical. Manuals, gouge, and work process instruction.# Welds steel plates and pipes in the repair and/or construction of marine vessels, industrial components, and associated equipment.# Have the...