Overview
Hybrid
Depends on Experience
Contract - W2
Contract - Independent
Skills
Analytical skill
Apache Spark
Big data
Cascading
Collaboration
Communication
Debugging
Distributed computing
IntelliJ IDEA
JetBrains
Kubernetes
Leadership
MapReduce
Migration
Problem solving
Scala
IDE
Job Details
Job Title: Spark Developer
Location: Columbus, OH / Remote
Job Description:
We are seeking a Sr. Spark Developer with extensive experience in Big Data technologies to lead the migration of workloads from Cascading/MapReduce to Spark 3 within a Kubernetes environment. The ideal candidate is skilled in Scala programming and has a strong background in designing and implementing Spark solutions.
Key Responsibilities:
- Migrate existing Big Data workloads from Cascading/MapReduce to Spark 3.
- Design and implement a Kubernetes environment for Spark 3 workloads.
- Perform daily code reviews and make required changes in Scala using IntelliJ IDE.
- Develop, test, and maintain efficient and scalable Spark solutions.
- Collaborate with cross-functional teams to ensure smooth migration and deployment.
Required Skills:
- Strong experience in converting Cascading/MapReduce jobs to Spark.
- Proficiency in Scala programming and Spark 3.
- Hands-on experience with Kubernetes and containerized environments.
- Familiarity with IntelliJ IDE for development and debugging.
- Strong analytical and problem-solving skills.
Preferred Qualifications:
- Prior experience with Big Data migration projects.
- Knowledge of distributed computing principles.
- Excellent communication and leadership skills.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.