Google Cloud Platform Data Engineer

Overview

Remote
$160,000 - $180,000
Full Time
Accepts corp to corp applications
Able to Provide Sponsorship

Skills

Data Engineer
Google Cloud Platform
GCS
Dataflow
Pub/Sub
BigQuery
Scala
Python
ETL
Apache Airflow

Job Details

Lead engineer who can assess the current landscape, do data profiling, data mapping, build solutions etc.
Tech stack: Google Cloud Platform. Scala, Spark, Kafka, Database (many) with data profiling /data mining skills, end-to-end ownership
Responsibilities
Design, develop, and maintain robust and scalable ETL workflows and data pipelines using tools like Hive, Spark, and Airflow.
Implement and manage data storage and processing solutions using Apache Hudi and BigQuery.
Develop and optimize data pipelines for structured and unstructured data in Google Cloud Platform environments, leveraging GCS for data storage.
Write clean, maintainable, and efficient code in Scala and Python to process and transform data.
Ensure data quality, integrity, and consistency by implementing appropriate data validation and monitoring techniques.
Work with cross-functional teams to understand business requirements and deliver data solutions that drive insights and decision-making.
Troubleshoot and resolve performance and scalability issues in data processing and pipelines.
Stay updated with the latest developments in big data technologies and tools and incorporate them into the workflow as appropriate.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.