Overview
Hybrid
Depends on Experience
Contract - W2
Contract - Independent
Skills
Big Data
Data structures
HDFS
Spark
Python
Kubernetes
Data Pipelines
Job Details
Qualifications:
- Over 10+ years of total IT experience 5+ years of experience with Hadoop (Cloudera)/big data technologies
- Advanced knowledge of the Hadoop ecosystem and Big Data technologies Hands-on experience with the Hadoop eco-system (HDFS, MapReduce, Hive, Pig, Impala, Spark, Kafka, Kudu, Solr)
- Experience on designing and developing Data Pipelines for Data Ingestion or Transformation using Scala or Python.
- Experience with Spark programming (Pyspark )
- Expert level building pipelines using Apache Spark Familiarity with core provider services from AWS.
- Hands-on experience with Python/Pyspark and basic libraries for machine learning is required
- Exposure to containerization and related technologies (e.g. Docker, Kubernetes)
- Exposure to aspects of DevOps (source control, continuous integration, deployments, etc.)
- Proficient in programming in Python with prior Apache Beam/Spark experience a plus.
- System level understanding - Data structures, algorithms, distributed storage & compute
- Can-do attitude on solving complex business problems, good interpersonal and teamwork skills
- Possess team management experience and have led a team of data engineers and analysts.
- Experience with Snowflake is highly preferred.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.