We are seeking an experienced Data Engineer with over 5 years of hands-on expertise in building and optimizing data pipelines and workflows. The ideal candidate will have strong experience with Azure Data Factory, Azure DevOps, Microsoft Fabric, and PySpark.
Responsibilities:
Design, develop, and maintain scalable data pipelines using Azure Data Factory.
Implement CI/CD pipelines using Azure DevOps for data solutions.
Work with Microsoft Fabric to enhance data warehousing and analytics capabilities.
Develop and optimize ETL processes using PySpark.
Ensure data integrity, quality, and availability across systems.
Collaborate with cross-functional teams including data scientists, analysts, and product managers.
Monitor and troubleshoot data workflows to ensure optimal performance and reliability.
Maintain documentation and follow best practices in data engineering.
Requirements:
Bachelor’s degree in Computer Science, Engineering, or a related field.
5+ years of experience in data engineering or similar roles.
Proven expertise in Azure Data Factory and Azure DevOps.
Hands-on experience with Microsoft Fabric.
Proficiency in PySpark and distributed data processing.
Strong understanding of data warehousing concepts and cloud architecture.
Excellent problem-solving and communication skills.
Ability to work independently and within a team environment.