Data Engineer

Primary Roles:

• Develop features and provide support for the Ground Speed Automation product
• Create and update APIs and backend processes running on AWS Lambda
• Write Python code to query from PostgreSQL/PostGIS databases
• Analyze data using SQL and Pyspark
• Create and maintain updates to AWS resources
• Integrate backend processes with existing APIs

Supporting Roles:

• Provide support to field testing and customer support operations related to data generated by the pipeline
• Provide support to data scientists building models and doing analysis based on the existing data pipeline and product testing results
• Work closely with the product management team and end users to understand customer experience and system requirements

Initial work:

• 1 week of onboarding - verify security, install software, technical overviews
• Work with senior engineer to improve system scalability, integrate image analysis ML models into pipeline
• Migration of CI/CD pipelines to Github Actions
• Improve Datadog monitoring and alerting for multiple systems

Required Skills:

• Experience requirements - at least 3 - 5 years of experience with SQL, Python and cloud computing (AWS preferred)
• Languages and Frameworks: Python, SQL
• Core understanding of AWS services including IAM, Lambda, S3, RDS
• Other: Docker, containerization concepts, REST API concepts (calling REST APIs using Python)

Additional Skills:
PostGIS or general GIS experience, MLOps (Sagemaker or MLflow), Terraform, Github Actions, Datadog, Databricks, REST API development in Python