Senior Azure Data Engineer Job at eTeam, Redmond, WA

  • eTeam
  • Redmond, WA

Job Description

Job Title: Senior Azure Data Engineer

Location: Redmond, WA 98052

Work Setup: Hybrid

Duration: 12 Months

Experience: 8-10 Years


Job Description


We are looking for an experienced Senior Azure Data Engineer with 8-10 years of experience in data engineering and strong hands-on expertise in Python, SQL, Apache Spark, PySpark, and Azure Databricks .


The candidate will be responsible for developing and supporting scalable data pipelines, transforming large datasets, and optimizing data processing solutions using Azure cloud technologies.


Must Have Skills

  • Strong experience with Python and SQL
  • Hands-on experience with Apache Spark, PySpark, and Spark SQL
  • Strong experience with Azure Databricks , including:
    • Notebooks
    • Jobs/Workflows
    • Clusters
  • Experience with Azure Synapse
  • Strong knowledge of Delta Lake
  • Experience working with Parquet
  • Hands-on experience with ADLS Gen2
  • Strong understanding of data engineering and ETL/ELT concepts
Nice to Have Skills
  • Experience with Scala
  • ETL/ELT development
  • Experience with MERGE/Upsert operations
  • Knowledge of data partitioning and partition pruning
  • Strong Spark performance tuning experience
  • Experience handling data skew, shuffles, caching, broadcast joins, and partition optimization
  • Experience with Git
  • Experience with Azure DevOps and CI/CD
Key Responsibilities
  • Develop and maintain scalable ETL/ELT data pipelines
  • Extract, transform, clean, and load data from various source systems
  • Build data solutions using Azure Databricks, Spark, Delta Lake, Synapse, and ADLS Gen2
  • Implement MERGE/Upsert logic using Delta Lake
  • Design effective partitioning strategies for large datasets
  • Optimize Spark jobs for performance and cost
  • Manage data skew, shuffles, caching, partitions, and cluster resources
  • Collaborate with development and data teams to deliver reliable data solutions
  • Follow best practices for Git, Azure DevOps, and CI/CD where applicable

Job Tags

Similar Jobs

BTS Group

Senior Director-Level Industrial-Organizational Psychologist Job at BTS Group

 ...strategy into tangible results. Our innovative approach blends deep business insights with cutting-edge developments in industrial-organizational psychology, enabling targeted interventions that advance organizations and their talent simultaneously.About the Role: As a... 

KPMG

Specialist Director, Federal Cyber Security Solutions Architect Job at KPMG

 ...federal mandates Collaborate with senior architects to integrate security controls into systems design, evaluating new tools /...  ...Requirements Guide (SRG) Develop and tune use cases for Security Information and Event Management (SIEM) platforms and automate incident... 

McClure Harrison, Inc.

Legal Recruiter Job at McClure Harrison, Inc.

 ...on connecting with people? Are you hungry and motivated by uncapped earning potential? Consider joining McClure Harrison as a Legal Recruiter - no experience necessary!We're a boutique legal recruiting agency headquartered in New York City, specializing in placing attorneys... 

Anchor-Ventana

Residential Glass Installer Job at Anchor-Ventana

 ...Job Description Job Description Anchor-Ventana Glass is a fast paced, well established, commercial and residential glass and...  ...DRIVE FOR AT LEAST FIVE YEARS. Experience with fabrication and/or installation preferred, but not required. Previous construction experience... 

CSL

Principal Medical Writer Job at CSL

 ...The Principal Medical Writer is a key leadership position responsible for determining and driving themedical writing strategy for one or more Programs within Clinical Development.The incumbent is accountable for the efficient preparation of high quality, strategically...