AWS Data Engineer

Remote Fulltime - (EST - Canada)

Job Description

We are seeking a highly skilled and motivated Data Engineer with at least 4 years of experience, strong expertise in SQL, Python, and AWS services. The ideal candidate will have hands-on experience in building, managing, and optimizing data pipelines and architectures, as well as a deep understanding of data lake, data warehouse, and lakehouse concepts. You will play a crucial role in designing and implementing scalable ETL solutions for our data infrastructure, supporting a range of data driven initiatives.

Role and Responsibilities:

• Design and develop scalable and efficient ETL pipelines to process structured and unstructured data.
• Work extensively with AWS services including S3, Lambda, Step Functions, Glue, EC2, EMR, SNS, SQS, Redshift, and Athena to manage and optimize data pipelines and workflows.
• Develop, monitor, and troubleshoot workflows using Airflow and other scheduling/orchestration tools.
• Build and maintain data lakes, data warehouses, and lakehouse architectures, ensuring seamless integration and optimal data flow.
• Implement and optimize SQL queries and Python scripts for data extraction, transformation, and loading (ETL).
• Collaborate with data scientists, analysts, and stakeholders to understand business requirements and translate them into robust data solutions.
• Optimize data pipelines for performance, reliability, and scalability to handle growing datasets and evolving business needs.
• Leverage best practices for data security, compliance, and governance in cloud-based environments.
• Perform data quality checks and ensure the accuracy and reliability of the data.
• Stay up to date with the latest technologies and trends in data engineering, cloud platforms, and ETL development.

Qualifications:

• 4+ years of experience as a Data Engineer, ETL Developer, or in a similar role.
• Proficiency in SQL and Python for data manipulation, automation, and processing.
• Extensive experience with AWS services, including:
• S3, Lambda, Step Functions, Glue, EC2, EMR, SNS, SQS, Redshift, and Athena.
• Strong understanding of data lake, data warehouse, and lakehouse concepts and the ability to design solutions for these architectures.
• Experience with ETL frameworks and tools to build data pipelines.
• Hands-on experience with workflow orchestration tools like Airflow.
• Solid understanding of data governance, security, and compliance best practices.
• Experience in building data models, optimizing databases, and query performance tuning.
• Familiarity with data lake architectures and modern data integration patterns.