Lead Data Engineer Job at WorkHQ, Los Angeles, CA

RWd1SWl2UFhxZjNvbXUzT2JPdW1CUkFYa3c9PQ==
  • WorkHQ
  • Los Angeles, CA

Job Description

Company Context

Series A, well-funded US startup in HRTech developing WorkHQ.com and an AI Recruiter product.

This is a US-only, Remote role (Mainland).

Role Overview

Lead data infrastructure architect managing billions of data points across 250M+ professional profiles.

Hire data engineers to aid you in that journey.

Core Responsibilities

  • Design scalable data pipelines processing massive record volumes

  • Architect ETL processes using PySpark on Amazon EMR (Open to shifting to other solutions like Data Bricks / Snowflake)

  • Distribute enriched data through medallion architecture across Postgres, Athena, OpenSearch

  • Integrate new data sources into the main pipeline

  • Implement advanced data matching using Splink

Technical Requirements

  • 5-8 years professional data engineering experience

  • Good proficiency in:

    • PySpark and distributed computing

    • AWS data services (EMR, Glue, Athena)

    • Docker

    • Pandas and DataFrame manipulation

    • Complex data format handling (JSONL, Parquet)

  • Strong background in:

    • Big data processing architectures

    • Data warehouse design

    • Performance optimization

  • Advanced Python, SQL skills

Nice to Have

  • Probabilistic record linking expertise

  • OpenSearch/elasticsearch technologies

  • Machine learning data pipeline design

  • Recruitment tech ecosystem knowledge

Technical Stack

  • Big Data: PySpark, EMR

  • Databases: Postgres, OpenSearch

  • Cloud: AWS

  • Containerization: Docker

  • Data Formats: JSONL, Parquet

  • Analytics: Metabase, Athena, Glue

  • Data Processing: Pandas, Splink

Other Considerations

While this role has specific requirements - if you lack a few technical skills, but motivated to learn and lead the platform, please apply for consideration.

If you are coming from Director/Head of/VP levels that is relevant to this job, you can apply as well.

You will need to apply directly on our platform.

Thank you for your time.

Job Tags

Permanent employment, Shift work,

Similar Jobs

The Flowery

Dispensary Drivetender - Part Time - Clearwater Job at The Flowery

 ...Job Title: Dispensary Drivetender PART TIME | Reports to: Delivery Manager | Location:1201 N Highland Ave, Clearwater, FL 33755 Key Responsibilities: In this role, you are the face of the company. The Flowery delivers statewide and expects their team to practice... 

Zortech Solutions

Software Developer Job at Zortech Solutions

 ...5 platform, including: Play a key role in the design, development, and implementation of solutions that leverage the Microsoft Dynamics...  ...MS Dynamics 365 Configure and customize MS Dynamics/Power Apps Portal Troubleshoot problems and fix bugs on plug-ins,... 

Asset LifeCycle

Warehouse Associate Job at Asset LifeCycle

 ...Warehouse Associate - No Weekends! Entry Level Opportunity! Topeka, KS About Us: Asset LifeCycle is a dynamic company specializing...  ...team and help us sort and process used electronics. No prior experience is necessary we'll teach you everything you need to know!... 

SSI Fire & Safety Holdings, LLC.

Fire Alarm Technician Job at SSI Fire & Safety Holdings, LLC.

 ...State Systems, Inc.204 Commercial Ave Lowell, AR 72745 We are actively seeking to hire a full-time Fire Alarm Technician to install and service commercial fire alarms and other low voltage systems, and address customer needs. This position offers a competitive... 

上海恺英网络科技有限公司

Russian, Portuguese, Vietnamese, English Translator Job at 上海恺英网络科技有限公司

Responsibilities include1.You will search in the app store for apps suitable for our target users and publish them in our product. Make plans for contents update on a regular basis. 2.You will be responsible for the whole process of product update, from function design...