Senior Data Engineer (Spark, Scala, Java, Flink, Spark Streaming, Kinesis)

The ideal candidate will have a strong background in software development with Java (Spring & Springboot), Scala for spark streaming & spark applications, or other JVM-based languages. You should be knowledgeable in using tools and frameworks such as Docker, Spark, Scala, Jupyter Notebook, Kubernetes, Feature Management Platforms, and SageMaker. Strong data engineering experience in AWS environment is required

Responsibilities:
- Develop software using Java (Spring & Springboot), Scala for spark streaming & spark applications, or other JVM-based languages.
- Work with XML, JSON, YML, and SQL; demonstrate strong Python and Linux skills.
- Utilize tools and frameworks including Docker, Spark, Scala, Jupyter Notebook, Kubernetes, Feature Management Platforms, and SageMaker.
- Apply advanced experience with scripting languages such as Python or Shell.
- Exhibit strong knowledge of software development methodologies and practices.
- Work in Agile development teams and demonstrate working knowledge of Agile (Scrum) development methodologies.
- Utilize experience with Amazon Web Services (EC2, S3, EMR) or equivalent cloud computing approaches.
- Showcase strong expertise in Data Warehousing and analytic architecture.
- Handle large data volumes efficiently.
- Develop stream-processing applications using Flink, Spark Streaming, Kinesis, etc.
- Conduct design and code reviews, defect remediation, and create technical design specifications.
- Develop automated unit tests and provide estimates and sequence of individual activities for project plans.
- Analyze and synthesize various inputs to create software and services.
- Identify dependencies and integrate 3rd party solutions.
- Collaborate effectively with peer engineers and architects to solve complex problems and deliver end-to-end quality.
- Communicate effectively with non-technical audiences including senior product and business management.
- Design and develop ETL jobs across multiple big data platforms and tools including S3, EMR, Scala, Python, and SQL.
- Evolve mature code bases into new technologies.
- Create and consume SOAP-based or JSON/REST web services and communicate with systems.

Skills: Java, Spring, Springboot, Scala, Spark Streaming, JVM, XML, JSON, YML, SQL, Python, Linux, Docker, Spark, Scala, Jupyter Notebook, Kubernetes, Feature Management Platforms, SageMaker, Agile, Scrum, AWS, EC2, S3, EMR, Data Warehousing, Flink, Kinesis, Unit Testing, 3rd Party Solutions, ETL, SOAP, REST, Web Services