Overview
Remote
Depends on Experience
Accepts corp to corp applications
Contract - Independent
Contract - W2
Contract - 12 Month(s)
Skills
Senior Data Engineer
AWS
Kafka
Python
Airflow
ETL
ETL processes
NoSQL
data engineering
data pipelines
data warehouse
data warehousing
event - driven architecture
data security
data analysis
data engineer
AWS Glue
Redshift
EMR
ETL developer
SQL
Hive
SparkSQL
Vertica
Job Details
We are seeking a Senior Data Engineer with a strong background in software development, AWS cloud services, PySpark, S3 based datalake, EMR, Hive, SparkSQL. The successful candidate will collaborate with data analysts to design, develop, and maintain data pipelines and systems that support clinical and operational data use-cases.
Key Responsibilities:
- Design, develop, and maintain scalable data pipelines and systems to support clinical and operational data use-cases.
- Utilize Airflow/Kafka and event-driven architecture to build efficient real-time data pipelines.
- Collaborate with AI scientists and data analysts to align data solutions with business needs.
- Develop and maintain data warehouse and ETL processes, ensuring data quality and integrity.
- Implement data security, privacy, and compliance measures.
- Perform data analysis, troubleshoot data issues, and provide technical support to end-users.
- Integrate predictive models into the analytics platform.
- Monitor and optimize data pipelines for performance and scalability.
- Stay updated with the latest technologies in data engineering, AWS cloud services, and healthcare/pharmaceutical industry.
Skills Required:
- Bachelor’s degree in Computer Science or related field.
- 5+ years of experience in software or data engineering.
- Experience in designing and developing data pipelines in a cloud environment (preferably AWS).
- Proficiency in Python programming language.
- Expertise in Airflow/Kafka and event-driven architecture.
- Familiarity with SQL, NoSQL databases, and data warehousing concepts.
- Experience with data security, privacy, and compliance measures.
- Strong analytical skills and problem-solving abilities.
- Excellent communication skills.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.