Data Engineer / Developer

Overview

Accepts corp to corp applications
Contract - Term Contract

Skills

AWS
Python
PYSPARK

Job Details

Our client is looking for a Data Engineer / Developer for Long Term Contract project in 100 % Remote below is the detailed requirements.

Title: Data Engineer / Developer

Location: 100 % Remote

Duration: Long Term Contract

Note:

  • Overall 12+ years of experience required
  • Healthcare / Pharma domain experience is mandatory

Job Description:

  • Hands-on and should have all skills of Below Sr Data Engineer SQL skills to understand the transformation and perform analysis PySpark and MWAA DAGS skills to understand the data pipelines AWS Glue Data Catalog / LF skills to understand the access control Lakehouse concepts and databases to understand consumption patterns Architectural ownership
  • Cross functional collaboration Strategic vision.
  • The ideal candidate will have a strong background in data engineering and development, with expertise in various data skills and tools.
  • The role involves working on complex data projects, collaborating with cross-functional teams, and ensuring the efficient processing and management of data.

Key Responsibilities:

  • Develop and maintain data pipelines using Python and Pyspark.
  • Design and implement efficient CQL queries for data extraction and manipulation.
  • Work with Aurora Postgres and Redshift for data storage and management.
  • Utilize Informatica or other ETL tools for data integration and transformation.
  • Implement and manage AWS cloud services, including Athena, S3, ECS/Docker, EMR, EC2, Lambda, Cloudwatch, and EventBridge. Use Airflow or other orchestration tools for workflow management.
  • Ensure data quality and integrity through rigorous testing and validation.
  • Collaborate with other teams to understand data requirements and deliver solutions.

Required Skills:

  • Proficiency in Python and Pyspark.
  • Solid experience in AWS Glue
  • Strong SQL skills.
  • Experience with Aurora Postgres and Redshift.
  • Knowledge of Informatica or other ETL tools.
  • Experience with AWS cloud services, including Athena, S3, Lambda, CloudWatch, and EventBridge.

Preferred Skills:

  • ECS/ Gocker (or any docker experience/concepts)
  • EMR, EC2 (or knowledge of any other distributed computing)
  • Familiarity with Airflow or other orchestration tools.

Looking forward for your reply.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.