Lead Data Engineer
Circle K, Clemmons, NC, United States
Overview
Technical Lead Data Engineer. Lead the design, development, and implementation of data solutions to derive actionable insights from complex datasets. Guide a team of data engineers, collaborate with cross-functional teams, and fortify data infrastructure, CI/CD pipelines, and analytics capabilities.
Responsibilities
Apply advanced knowledge of data engineering principles to design and implement data loading and aggregation frameworks across the organization.
Gather and process raw, structured, semi-structured and unstructured data using batch and real-time processing frameworks.
Implement and optimize data solutions in enterprise data warehouses and big data repositories, with a focus on cloud transition.
Drive new and enhanced capabilities for Enterprise Data Platform partners to meet the needs of product, engineering, and business teams.
Leverage Python, Spark, and SQL to construct robust pipelines for data processing and analysis.
Implement CI/CD pipelines to automate build, test, and deployment of data solutions.
Design and optimize data schemas using data modeling techniques to ensure data integrity and performance.
Drive continuous improvement to enhance performance, reliability, and scalability of data infrastructure.
Collaborate with data scientists, analysts, and stakeholders to translate business requirements into technical solutions.
Adopt best practices for data governance, security, and compliance to protect data assets.
Qualifications
Bachelor’s or master’s degree in computer science, engineering, or a related field.
8+ years of experience in data engineering with expertise in designing and building data pipelines, ETL processes, and data warehouses.
Strong proficiency in SQL, Python, and Spark.
Hands-on experience with cloud platforms (AWS, Azure, or GCP).
Experience with big data technologies such as Hadoop, Spark, Kafka, and distributed computing frameworks.
Knowledge of data lake and data warehouse solutions (e.g., Databricks, Snowflake, Redshift, BigQuery, Azure Data Factory, Airflow).
Experience implementing CI/CD pipelines for data solutions.
Solid understanding of data modeling, data warehousing architectures, and data management best practices.
Excellent communication and leadership skills to collaborate with cross-functional teams and guide technical decisions.
Relevant certifications (e.g., Azure, Databricks, Snowflake) preferred.
Circle K is an Equal Opportunity Employer. The company complies with the Americans with Disabilities Act (the ADA) and applicable state and local disability laws. Applicants with disabilities may be entitled to reasonable accommodations under the ADA and state/local laws. Please inform the Human Resources representative if you need assistance completing any forms or participating in the application process.
Click below to review information about the company's use of the federal E-Verify program to check work eligibility:
In English
In Spanish
#J-18808-Ljbffr
Technical Lead Data Engineer. Lead the design, development, and implementation of data solutions to derive actionable insights from complex datasets. Guide a team of data engineers, collaborate with cross-functional teams, and fortify data infrastructure, CI/CD pipelines, and analytics capabilities.
Responsibilities
Apply advanced knowledge of data engineering principles to design and implement data loading and aggregation frameworks across the organization.
Gather and process raw, structured, semi-structured and unstructured data using batch and real-time processing frameworks.
Implement and optimize data solutions in enterprise data warehouses and big data repositories, with a focus on cloud transition.
Drive new and enhanced capabilities for Enterprise Data Platform partners to meet the needs of product, engineering, and business teams.
Leverage Python, Spark, and SQL to construct robust pipelines for data processing and analysis.
Implement CI/CD pipelines to automate build, test, and deployment of data solutions.
Design and optimize data schemas using data modeling techniques to ensure data integrity and performance.
Drive continuous improvement to enhance performance, reliability, and scalability of data infrastructure.
Collaborate with data scientists, analysts, and stakeholders to translate business requirements into technical solutions.
Adopt best practices for data governance, security, and compliance to protect data assets.
Qualifications
Bachelor’s or master’s degree in computer science, engineering, or a related field.
8+ years of experience in data engineering with expertise in designing and building data pipelines, ETL processes, and data warehouses.
Strong proficiency in SQL, Python, and Spark.
Hands-on experience with cloud platforms (AWS, Azure, or GCP).
Experience with big data technologies such as Hadoop, Spark, Kafka, and distributed computing frameworks.
Knowledge of data lake and data warehouse solutions (e.g., Databricks, Snowflake, Redshift, BigQuery, Azure Data Factory, Airflow).
Experience implementing CI/CD pipelines for data solutions.
Solid understanding of data modeling, data warehousing architectures, and data management best practices.
Excellent communication and leadership skills to collaborate with cross-functional teams and guide technical decisions.
Relevant certifications (e.g., Azure, Databricks, Snowflake) preferred.
Circle K is an Equal Opportunity Employer. The company complies with the Americans with Disabilities Act (the ADA) and applicable state and local disability laws. Applicants with disabilities may be entitled to reasonable accommodations under the ADA and state/local laws. Please inform the Human Resources representative if you need assistance completing any forms or participating in the application process.
Click below to review information about the company's use of the federal E-Verify program to check work eligibility:
In English
In Spanish
#J-18808-Ljbffr