Senior Data Engineer – Databricks / Azure Lakehouse
Vaco Recruiter Services, Tempe, AZ, United States
Our client is a technology-forward financial services organization operating in a highly regulated environment where data quality, governance, and reliability are non-negotiable. This team is responsible for building and modernizing a cloud-native data platform that supports analytics, reporting, and downstream application use across the business.
This is a hands-on Databricks-focused data engineering role centered on building production-grade pipelines and governed data models. The work here directly supports decision-making, compliance, and operational visibility across the enterprise.
What You’ll Do
Design, build, and optimize large-scale data pipelines using the Databricks Lakehouse platform
Develop ETL and ELT workflows using Python, PySpark, and Spark SQL
Implement and manage Delta Lake data models for performance, reliability, and scalability
Apply strong governance practices using Unity Catalog, including RBAC, lineage, and secure data sharing
Modernize and extend an Azure-based data ecosystem supporting enterprise analytics
Build reusable frameworks and workflows using Databricks Workflows, Repos, and Delta Live Tables
Enable downstream consumption for BI, analytics, and data science use cases
Collaborate with stakeholders across engineering, analytics, and the business
Support CI/CD pipelines, testing, and production deployments
Required Skills & Experience
5 plus years of professional data engineering experience in production environments
Strong expertise in Databricks and Lakehouse architectures
Advanced proficiency with Python, PySpark, and Spark SQL
Experience designing and optimizing Delta Lake data models
Hands-on experience with Azure cloud services supporting data platforms
Familiarity with Unity Catalog for governance, security, and metadata management
Experience building end-to-end data pipelines with ownership from ingestion through consumption
Strong understanding of data quality, compliance, and enterprise data standards
Preferred / Nice-to-Have
Experience working with financial, healthcare, or other regulated datasets
Familiarity with Azure Data Factory, Data Lake, Key Vault, and related services
Experience with orchestration tools such as Apache Airflow
Exposure to CI/CD using Azure DevOps or similar tooling
Experience supporting BI, analytics, or reporting teams
Background working in matrixed or cross-functional enterprise environments
Determining compensation for this role (and others) at Vaco/Highspring depends upon a wide array of factors including but not limited to the individual’s skill sets, experience and training, licensure and certifications, office location and other geographic considerations, as well as other business and organizational needs. With that said, as required by local law in geographies that require salary range disclosure, Vaco/Highspring notes the salary range for the role is noted in this job posting. The individual may also be eligible for discretionary bonuses, and can participate in medical, dental, and vision benefits as well as the company’s 401(k) retirement plan. Additional disclaimer: Unless otherwise noted in the job description, the position Vaco/Highspring is filing for is occupied. Please note, however, that Vaco/Highspring is regularly asked to provide talent to other organizations. By submitting to this position, you are agreeing to be included in our talent pool for future hiring for similarly qualified positions. Submissions to this position are subject to the use of AI to perform preliminary candidate screenings, focused on ensuring minimum job requirements noted in the position are satisfied. Further assessment of candidates beyond this initial phase within Vaco/Highspring will be otherwise assessed by recruiters and hiring managers. Vaco/Highspring does not have knowledge of the tools used by its clients in making final hiring decisions and cannot opine on their use of AI products.
#J-18808-Ljbffr
This is a hands-on Databricks-focused data engineering role centered on building production-grade pipelines and governed data models. The work here directly supports decision-making, compliance, and operational visibility across the enterprise.
What You’ll Do
Design, build, and optimize large-scale data pipelines using the Databricks Lakehouse platform
Develop ETL and ELT workflows using Python, PySpark, and Spark SQL
Implement and manage Delta Lake data models for performance, reliability, and scalability
Apply strong governance practices using Unity Catalog, including RBAC, lineage, and secure data sharing
Modernize and extend an Azure-based data ecosystem supporting enterprise analytics
Build reusable frameworks and workflows using Databricks Workflows, Repos, and Delta Live Tables
Enable downstream consumption for BI, analytics, and data science use cases
Collaborate with stakeholders across engineering, analytics, and the business
Support CI/CD pipelines, testing, and production deployments
Required Skills & Experience
5 plus years of professional data engineering experience in production environments
Strong expertise in Databricks and Lakehouse architectures
Advanced proficiency with Python, PySpark, and Spark SQL
Experience designing and optimizing Delta Lake data models
Hands-on experience with Azure cloud services supporting data platforms
Familiarity with Unity Catalog for governance, security, and metadata management
Experience building end-to-end data pipelines with ownership from ingestion through consumption
Strong understanding of data quality, compliance, and enterprise data standards
Preferred / Nice-to-Have
Experience working with financial, healthcare, or other regulated datasets
Familiarity with Azure Data Factory, Data Lake, Key Vault, and related services
Experience with orchestration tools such as Apache Airflow
Exposure to CI/CD using Azure DevOps or similar tooling
Experience supporting BI, analytics, or reporting teams
Background working in matrixed or cross-functional enterprise environments
Determining compensation for this role (and others) at Vaco/Highspring depends upon a wide array of factors including but not limited to the individual’s skill sets, experience and training, licensure and certifications, office location and other geographic considerations, as well as other business and organizational needs. With that said, as required by local law in geographies that require salary range disclosure, Vaco/Highspring notes the salary range for the role is noted in this job posting. The individual may also be eligible for discretionary bonuses, and can participate in medical, dental, and vision benefits as well as the company’s 401(k) retirement plan. Additional disclaimer: Unless otherwise noted in the job description, the position Vaco/Highspring is filing for is occupied. Please note, however, that Vaco/Highspring is regularly asked to provide talent to other organizations. By submitting to this position, you are agreeing to be included in our talent pool for future hiring for similarly qualified positions. Submissions to this position are subject to the use of AI to perform preliminary candidate screenings, focused on ensuring minimum job requirements noted in the position are satisfied. Further assessment of candidates beyond this initial phase within Vaco/Highspring will be otherwise assessed by recruiters and hiring managers. Vaco/Highspring does not have knowledge of the tools used by its clients in making final hiring decisions and cannot opine on their use of AI products.
#J-18808-Ljbffr