Python Developer Job at Texas State Library and Archives Commision in Charlotte

Job Description:

Our Client is seeking a highly skilled and experienced Senior PySpark Developer to join our dynamic data engineering team. This role is ideal for a motivated professional with a strong background in big data technologies, ETL processes, and database systems. The successful candidate will play a key role in designing, developing, and optimizing data pipelines and analytics solutions that support critical business operations.

Key Responsibilities:

• Design, develop, and maintain scalable data pipelines using PySpark and Spark SQL.
• Collaborate with data architects, analysts, and business stakeholders to understand data requirements and deliver robust solutions.
• Optimize and troubleshoot complex ETL workflows and data transformation processes.
• Write efficient SQL queries for data extraction, transformation, and loading.
• Ensure data quality, integrity, and compliance with enterprise standards.
• Participate in code reviews, testing, and deployment activities.
• Monitor and maintain production workflows, including job scheduling and performance tuning.

Required Skills & Qualifications:

• 5+ years of experience in data engineering or software development.
• Strong proficiency in PySpark, Python, and Spark SQL.
• Solid experience with SQL and relational database systems.
• Proven expertise in ETL development and data pipeline orchestration.
• Familiarity with job scheduling tools; Autosys experience is a plus.
• Excellent problem-solving skills and attention to detail.
• Strong communication and collaboration abilities.

Preferred Qualifications:

• Experience working in the financial services industry.
• Familiarity with cloud platforms and big data ecosystems