Director, Data Engineering
Affinity Solutions, New York, NY, United States
Affinity Solutions (Affinity) is the leading consumer purchase insights company. We provide a complete view of U.S. and U.K. consumer spending, across and between brands, via exclusive access to fully permissioned data from over 100 million consumers. Our proprietary AI technology, Comet™, transforms these purchase signals into actionable insights for business and marketing leaders to drive optimal outcomes and build lasting customer relationships. Visit www.affinitysolutions.com to discover how we're shaping the future of consumer purchase insights.
We are seeking a Director, Data Engineering with
Master's degree in in Computer Science, Information Technology with 2 years of experience; Lead architecture and design decisions for Marketing Solutions' Core Data Asset Maintenance and Data Product development using advanced technologies including Snowflake Data Cleanrooms, Databricks, and Apache Kafka, with experience in Python, SQL, PySpark, Snowpark, Jinja SQL, and AWS EMR. Provide technical input and decide for all components of the data infrastructure, using AWS technologies (S3 data lake, Lambda, Glue, Airflow), Apache Hive, Amazon Redshift, Workflow Orchestration using Airflow for system design. Design and architect pipelines and data ingestion and transformation, requiring advanced skills in big data technologies including Apache Spark, distributed systems, and performance optimization using Bash scripting, and CI/CD pipelines with Jenkins. Define SLAs for data availability and architect automated monitoring solutions, utilizing experience in DevOps tools, CloudWatch, and incident response frameworks.
Skills: Snowflake Data Cleanrooms, Databricks, Apache Kafka, Python, SQL, PySpark, Snowpark, Jinja SQL, AWS EMR, S3 data lake, Lambda, Glue, Airflow, Apache Hive, Amazon Redshift, Workflow Orchestration using Airflow.
Send resume to Affinity Solutions, Inc, 112 WEST 34TH STREET, 18TH Floor, New York, NY 10120.
Salary: $131,997.00 Work Hours: 40 hours/week, Monday to Friday, 9:00am - 5:00pm
We are seeking a Director, Data Engineering with
Master's degree in in Computer Science, Information Technology with 2 years of experience; Lead architecture and design decisions for Marketing Solutions' Core Data Asset Maintenance and Data Product development using advanced technologies including Snowflake Data Cleanrooms, Databricks, and Apache Kafka, with experience in Python, SQL, PySpark, Snowpark, Jinja SQL, and AWS EMR. Provide technical input and decide for all components of the data infrastructure, using AWS technologies (S3 data lake, Lambda, Glue, Airflow), Apache Hive, Amazon Redshift, Workflow Orchestration using Airflow for system design. Design and architect pipelines and data ingestion and transformation, requiring advanced skills in big data technologies including Apache Spark, distributed systems, and performance optimization using Bash scripting, and CI/CD pipelines with Jenkins. Define SLAs for data availability and architect automated monitoring solutions, utilizing experience in DevOps tools, CloudWatch, and incident response frameworks.
Skills: Snowflake Data Cleanrooms, Databricks, Apache Kafka, Python, SQL, PySpark, Snowpark, Jinja SQL, AWS EMR, S3 data lake, Lambda, Glue, Airflow, Apache Hive, Amazon Redshift, Workflow Orchestration using Airflow.
Send resume to Affinity Solutions, Inc, 112 WEST 34TH STREET, 18TH Floor, New York, NY 10120.
Salary: $131,997.00 Work Hours: 40 hours/week, Monday to Friday, 9:00am - 5:00pm