Role description
Job Title: Data Engineer
Experience: 5 to 7 Years
Location: Hyderabad, India
Employment Type: Full-Time
Mandatory Skills:
Data Engineering: Strong foundation in building and maintaining scalable data pipelines.
AWS (Amazon Web Services): Proven experience with core AWS services like S3, Lambda, EC2, RDS, Glue, and Athena.
PySpark: Hands-on expertise in developing distributed data processing solutions using PySpark on big data platforms.
SNS (Simple Notification Service): Experience in implementing event-driven architectures using AWS SNS.
Key Responsibilities:
Design, build, and maintain efficient, scalable, and robust data pipelines and systems.
Work with large datasets in cloud-based environments (preferably AWS).
Develop ETL/ELT workflows using PySpark and other tools.
Implement data quality checks, monitoring, and logging in the pipeline processes.
Collaborate with cross-functional teams including Data Scientists, Analysts, and DevOps.
Use AWS services like Glue, EMR, S3, Lambda, and SNS to orchestrate data workflows.
Ensure data security and compliance with company and industry standards.
Preferred Qualifications:
Bachelor's or Master’s degree in Computer Science, Engineering, or related field.
Experience with other AWS services such as CloudWatch, SQS, Redshift, or Kinesis is a plus.
Familiarity with DevOps practices and tools such as Git, CI/CD pipelines.
Strong analytical and problem-solving skills.
Skills
Data Engineering,AWS,Pyspark,Sns
