Data Engineer II
Visual Lease, Arlington, VA, United States
Data Engineer II
This role will be responsible for helping build out sitewide tracking architecture on Homes.com and then feeding that data into user-friendly KPI dashboards which track site performance and product usage and provide day-to-day insights into consumer behavior. The Data Engineer will build and deliver the data engineering architecture for Homes.com and work across all departments to provide analysis and insights for future growth.
Responsibilities
Design, develop, and maintain scalable data pipelines and ETL processes using Databricks, Data Factory, and other AWS services.
Implement and optimize Spark jobs, data transformations, and data processing workflows in Databricks.
Build and deploy AI/ML models: integrate Machine Learning into data pipelines, leveraging Databricks ML and AWS ML to develop predictive models and drive business insights.
Understand various data domains and translate them through data transformations to suit business needs.
Oversee automated data infrastructure: collaborate with software engineering and product teams to streamline data tracking in products.
Integrate data from different sources via cross-domain data stitching into the existing data model.
Develop and maintain secure data warehouses and data lakehouses, implementing data models, data quality checks, and governance practices.
Hands‑on data optimization and query performance.
Liaise with key company teams (DBA, DevOps, SecOps, etc.) to ensure data accessibility and analysis.
Mentor and manage engineers and analysts in the normal execution of their responsibilities.
Successfully implement software development projects to specification and within scheduled timelines and budget parameters.
Leverage AWS DevOps and CI/CD best practices to automate the deployment, through Terraform, of data pipelines and infrastructure.
Basic Qualifications
Bachelor's degree from an accredited, not-for-profit, in-person college/university.
A track record of commitment to prior employers.
4+ years of hands‑on experience in software development, delivering high-quality solutions.
Strong proficiency in Python, PySpark, and SQL.
Solid foundation in Object‑Oriented Programming principles and best practices.
Demonstrated success in building and launching data‑driven products operating at terabyte scale.
Ability to translate technical requirements into robust architecture, data models, and ETL strategies.
Practical experience with cloud‑based databases, both relational and non‑relational systems.
Knowledge of business intelligence software (e.g., Power BI or similar tools).
Ability to retrieve, synthesize, and present critical data for ad‑hoc questions.
Preferred Qualifications
Hands‑on experience or familiarity with big data platforms such as Databricks or Snowflake.
Proficiency with cloud platforms, including AWS or Azure.
Experience with NoSQL databases (e.g., DynamoDB) and object storage systems (e.g., Amazon S3).
Practical knowledge of event streaming technologies, especially Kafka.
Strong analytical and problem‑solving skills for complex technical challenges.
Experience leveraging AWS compute and container services for scalable solutions.
Benefits
Comprehensive healthcare coverage: Medical / Vision / Dental / Prescription Drug.
Life, legal, and supplementary insurance.
Virtual and in‑person mental health counseling services.
Commuter and parking benefits.
401(k) retirement plan with matching contributions.
Employee stock purchase plan.
Paid time off.
Tuition reimbursement.
On‑site fitness center and/or reimbursed fitness center membership costs (location dependent), with yoga studio, Pelotons, personal training, group exercise classes.
Access to CoStar Group’s Diversity, Equity, & Inclusion Employee Resource Groups.
Complimentary gourmet coffee, tea, hot chocolate, fresh fruit, and other healthy snacks.
Additional Notes
We welcome all qualified candidates who are currently eligible to work full‑time in the United States to apply. However, CoStar Group is not able to provide visa sponsorship for this position.
CoStar Group is an Equal Employment Opportunity Employer; we maintain a drug‑free workplace and perform pre‑employment substance abuse testing.
#J-18808-Ljbffr
This role will be responsible for helping build out sitewide tracking architecture on Homes.com and then feeding that data into user-friendly KPI dashboards which track site performance and product usage and provide day-to-day insights into consumer behavior. The Data Engineer will build and deliver the data engineering architecture for Homes.com and work across all departments to provide analysis and insights for future growth.
Responsibilities
Design, develop, and maintain scalable data pipelines and ETL processes using Databricks, Data Factory, and other AWS services.
Implement and optimize Spark jobs, data transformations, and data processing workflows in Databricks.
Build and deploy AI/ML models: integrate Machine Learning into data pipelines, leveraging Databricks ML and AWS ML to develop predictive models and drive business insights.
Understand various data domains and translate them through data transformations to suit business needs.
Oversee automated data infrastructure: collaborate with software engineering and product teams to streamline data tracking in products.
Integrate data from different sources via cross-domain data stitching into the existing data model.
Develop and maintain secure data warehouses and data lakehouses, implementing data models, data quality checks, and governance practices.
Hands‑on data optimization and query performance.
Liaise with key company teams (DBA, DevOps, SecOps, etc.) to ensure data accessibility and analysis.
Mentor and manage engineers and analysts in the normal execution of their responsibilities.
Successfully implement software development projects to specification and within scheduled timelines and budget parameters.
Leverage AWS DevOps and CI/CD best practices to automate the deployment, through Terraform, of data pipelines and infrastructure.
Basic Qualifications
Bachelor's degree from an accredited, not-for-profit, in-person college/university.
A track record of commitment to prior employers.
4+ years of hands‑on experience in software development, delivering high-quality solutions.
Strong proficiency in Python, PySpark, and SQL.
Solid foundation in Object‑Oriented Programming principles and best practices.
Demonstrated success in building and launching data‑driven products operating at terabyte scale.
Ability to translate technical requirements into robust architecture, data models, and ETL strategies.
Practical experience with cloud‑based databases, both relational and non‑relational systems.
Knowledge of business intelligence software (e.g., Power BI or similar tools).
Ability to retrieve, synthesize, and present critical data for ad‑hoc questions.
Preferred Qualifications
Hands‑on experience or familiarity with big data platforms such as Databricks or Snowflake.
Proficiency with cloud platforms, including AWS or Azure.
Experience with NoSQL databases (e.g., DynamoDB) and object storage systems (e.g., Amazon S3).
Practical knowledge of event streaming technologies, especially Kafka.
Strong analytical and problem‑solving skills for complex technical challenges.
Experience leveraging AWS compute and container services for scalable solutions.
Benefits
Comprehensive healthcare coverage: Medical / Vision / Dental / Prescription Drug.
Life, legal, and supplementary insurance.
Virtual and in‑person mental health counseling services.
Commuter and parking benefits.
401(k) retirement plan with matching contributions.
Employee stock purchase plan.
Paid time off.
Tuition reimbursement.
On‑site fitness center and/or reimbursed fitness center membership costs (location dependent), with yoga studio, Pelotons, personal training, group exercise classes.
Access to CoStar Group’s Diversity, Equity, & Inclusion Employee Resource Groups.
Complimentary gourmet coffee, tea, hot chocolate, fresh fruit, and other healthy snacks.
Additional Notes
We welcome all qualified candidates who are currently eligible to work full‑time in the United States to apply. However, CoStar Group is not able to provide visa sponsorship for this position.
CoStar Group is an Equal Employment Opportunity Employer; we maintain a drug‑free workplace and perform pre‑employment substance abuse testing.
#J-18808-Ljbffr