Ingnieur Senior Big Data/Senior Big Data Engineer

Overview

On Site
Compensation information provided in the description
Full Time

Skills

Data Analysis
Game Development
Marketing
Monetization
Reporting
Analytics
Sourcing
Real-time
Root Cause Analysis
Issue Resolution
Extract
Transform
Load
Groovy
Tier 2
Mentorship
Computer Science
Big Data
Scala
Python
Java
Apache Spark
Apache Kafka
Streaming
API
Data Processing
File Formats
Apache Avro
JSON
Apache Parquet
Electronic Health Record (EHR)
Amazon EC2
Amazon DynamoDB
Amazon Redshift
Amazon Web Services
Data Lake
Management
Regulatory Compliance
Jenkins
Continuous Integration
Continuous Delivery
DevOps
Terraform
Facilitation
Presentations
Negotiations
Elasticsearch

Job Details

Job Description

Our customer provides data analytics services that support game teams and analysts across the organization. As players perform actions in-game, our platform collects telemetry in real-time. The data collected provides insights for guiding game development, marketing, and monetization.
The Senior Big Data Engineer is a key contributor to the design and development of the analytics platform. This role is responsible for building and maintaining optimized and highly available data pipelines that facilitate deeper analysis and reporting by the analytics teams. This role builds data processing frameworks that handle the analytics platform.

Responsibilities

  • Data pipeline development using Scala/Python/Java-based streaming frameworks, sourcing data from enterprise Kafka, working with test-driven methodologies.
  • Assistance in architecture and design of mostly real-time data pipeline, focusing on solutions that allow performant data transformation and aggregation within the pipeline.
  • PII data compliance - support of existing processes.
  • Troubleshooting production issues, leading root cause analysis and issue resolution efforts.
  • Data pipeline monitoring, defining metrics and alerting on issues with DataDog/PagerDuty.
  • Automation - build and test solutions using Groovy framework and Jenkins.
  • Production On-Call (Tier 2) support rotation.
  • Mentoring junior team members.

Requirements

  • Bachelor's in computer science or closely related discipline.
  • 6+ years of focused experience in Big Data space.
  • Experience with any combination of Scala, Python, or Java - any combination of 2/3.
  • Apache Spark Streaming, Spark, Kafka, or any other streaming API experience.
  • Experience building data processing pipelines at scale on AWS.
  • Experience working with different file formats like Avro, JSON, Parquet.
  • Experience in AWS components like EMR, EC2, Glue, DynamoDB, Redshift.
  • Deep understanding of the concepts involved with managing resources in AWS.
  • Strong understanding of data lake concepts and hands-on experience managing data lakes.

Additional Skills & Qualifications
  • PII data compliance tech.
  • Experience in Airflow and Jenkins or other CI/CD.
  • Experience with different monitoring, alerting, and dashboarding tools like Datadog and PagerDuty.
  • DevOps experience with Terraform or CloudFormation.
  • Ability to lead new and complex projects with multiple dependencies from the design phase to production. This may involve facilitating across multiple teams.
  • Other skills needed to perform the job (e.g., presentation skills, negotiation skills, etc.).

Pay and Benefits
The pay range for this position is $70.00 - $110.00/hr.
Workplace Type
This is a fully remote position.

propos de TEKsystems:

Nous sommes partenaires dans la transformation. Nous aidons les clients activer des ides et des solutions afin de profiter d'un nouveau monde d'opportunits. Nous sommes une quipe solide de 80 000 personnes, nous travaillons avec plus de 6 000 clients, dont 80 % faisant partie du Fortune 500, partout en Amrique du Nord, en Europe et en Asie. En tant que chefs de file de l'industrie des services technologiques complets, des services aux talents et des applications du monde rel, nous travaillons avec des leaders progressistes pour favoriser le changement. C'est le pouvoir d'un vrai partenariat. TEKsystems est une socit du groupe Allegis.

Qubec Numro du permis d'agence de placement de personnel AP-2000547

About TEKsystems:

We're partners in transformation. We help clients activate ideas and solutions to take advantage of a new world of opportunity. We are a team of 80,000 strong, working with over 6,000 clients, including 80% of the Fortune 500, across North America, Europe and Asia. As an industry leader in Full-Stack Technology Services, Talent Services, and real-world application, we work with progressive leaders to drive change. That's the power of true partnership. TEKsystems is an Allegis Group company.

The company is an equal opportunity employer and will consider all applications without regards to race, sex, age, color, religion, national origin, veteran status, disability, sexual orientation, gender identity, genetic information or any characteristic protected by law.

Quebec Personnel Placement Agency Permit Number AP-2000547
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.