Overview:
We are looking for a highly experienced and motivated Azure Data Engineer to join our team. The successful candidate will have a minimum of 8 years of hands-on experience in Azure and Databricks architecture, migration, and development. This role requires excellent communication skills and the ability to own and self-manage work. The candidate must be able to overlap with US CST work hours for up to 6 hours daily.
Key Responsibilities:
Design, develop, and implement advanced analytics solutions using Azure and Databricks.
Perform data migration and transformation tasks on Databricks.
Develop and maintain scalable data pipelines and build out new API integrations to support continuing increases in data volume and complexity.
Work closely with data engineers, data scientists, and business stakeholders to understand their requirements and provide expert guidance.
Optimize and tune the performance of Databricks and Azure-based solutions.
Manage and monitor the Databricks environment to ensure reliability and performance.
Lead the architecture and development of complex data integration projects.
Conduct code reviews, provide feedback, and ensure best practices are followed.
Own and self-manage assigned tasks, ensuring timely delivery with high quality.
Collaborate with cross-functional teams and stakeholders in a remote work environment.
Qualifications:
Bachelor’s or Master’s degree in Computer Science, Information Technology, or related field.
8+ years of experience with Azure and Databricks architecture, migration, and development.
Strong experience in building and optimizing data pipelines, architectures, and data sets.
Strong proficiency in Python, particularly for data engineering tasks like ETL (Extract, Transform, Load), automation, and working with libraries like Pandas, NumPy, and PySpark.
Experience with big data tools and technologies (Spark, Hadoop, etc.).
Solid understanding of SQL and relational databases.
Experience with cloud infrastructure management and monitoring tools.
Excellent problem-solving and analytical skills and the ability to troubleshoot complex issues.
Job Type: Full-time
Pay: Rs150,000.00 - Rs450,000.00 per month
Work Location: In person
