Associate Product Architect

Competent Big Data/Databrick engineer, who is independent, results driven and is capable of taking business requirements and building out the technologies to take it to production. • Big Data Engineer with expert level experience in Hadoop ecosystem and real time analytics tools including PySpark/Scala Spark/Hive/Hadoop CLI/MapReduce/ Storm/Kafka/Lambda Architecture/Mongo • Familiar with job scheduling challenges in Hadoop • Experienced in creating and submitting Spark jobs • Experience in high performance tuning and scalability • Experience in working on real time stream processing technologies like Spark structured streaming, Kafka • Expertise in Python/Spark and their related libraries and frameworks • Experience in building pipeline and efforts involved in deployment • Unix/Linux expertise; comfortable with Linux operating system and Shell Scripting • Experience in Azure cache • Design, Development, Unit and Integration testing of complex data pipelines and to handle data volumes to derive insights • Ability to optimize code to be able to run efficiently with stipulated SLA • Excellent problem solving skills, with attention to detail, focus on quality and timely delivery of assigned tasks