Overview
On Site
Full Time
Skills
IT management
Decision-making
Collaboration
PySpark
Python
Apache Spark
Streaming
Debugging
Data warehouse
Data management
Data modeling
Data engineering
SQL Azure
Data
Microsoft
DevOps
Microsoft Azure
Cosmos
Software development
Productivity
Regulatory Compliance
Feasibility study
Status reports
Job Details
Job Description
Job Description
Job Summary:
We are seeking a skilled and motivated Data Engineer to join our team. As a Data Engineer, you will be responsible for designing, developing, and maintaining scalable data pipelines and infrastructure to support data-driven decision-making. You will collaborate with data scientists, analysts, and other stakeholders to ensure data is readily available, reliable, and of high quality.
Key Responsibilities:
Experience is a MUST with last 3-4 years Should have experience in using Pyspark Working experience on KUSTO for telemetry data Should be able to write python code in Azure Synapse Notebook. Good understanding working experience of Azure Synapse end-to-end. Should have experience in working with Spark structured streaming data possibly from Event hubs Should have experience in handing large volume of data possibly in 100s of GB Should be able to debug and fine tune the jobs note book for optimum memory consumption and processing. Should have knowledge of Data warehousing, data management for efficient data handling. Should have good knowledge of data modeling data engineering work Experience working with large data sets using SQLAzure Data LakePySparkADFSynapse, etc., to derive actionable insights is highly desirable. They should be productive soonest possible post joining. Nice To Have Local Redmond resourcePST working hour resource Microsoft ecosystem knowledge. Azure Devops Azure WebAppsfunctions Azure Cosmos Dbeventhub
(1.) To be responsible for providing technical guidance or solutions ;define, advocate, and implement best practices and coding standards for the team.
(2.) To develop and guide the team members in enhancing their technical capabilities and increasing productivity
(3.) To ensure process compliance in the assigned module, and participate in technical discussionsorreview as a technical consultant for feasibility study (technical alternatives, best packages, supporting architecture best practices, technical risks, breakdown into components, estimations).
(4.) To prepare and submit status reports for minimizing exposure and risks on the project or closure of escalations.
We are seeking a skilled and motivated Data Engineer to join our team. As a Data Engineer, you will be responsible for designing, developing, and maintaining scalable data pipelines and infrastructure to support data-driven decision-making. You will collaborate with data scientists, analysts, and other stakeholders to ensure data is readily available, reliable, and of high quality.
Key Responsibilities:
Experience is a MUST with last 3-4 years Should have experience in using Pyspark Working experience on KUSTO for telemetry data Should be able to write python code in Azure Synapse Notebook. Good understanding working experience of Azure Synapse end-to-end. Should have experience in working with Spark structured streaming data possibly from Event hubs Should have experience in handing large volume of data possibly in 100s of GB Should be able to debug and fine tune the jobs note book for optimum memory consumption and processing. Should have knowledge of Data warehousing, data management for efficient data handling. Should have good knowledge of data modeling data engineering work Experience working with large data sets using SQLAzure Data LakePySparkADFSynapse, etc., to derive actionable insights is highly desirable. They should be productive soonest possible post joining. Nice To Have Local Redmond resourcePST working hour resource Microsoft ecosystem knowledge. Azure Devops Azure WebAppsfunctions Azure Cosmos Dbeventhub
(1.) To be responsible for providing technical guidance or solutions ;define, advocate, and implement best practices and coding standards for the team.
(2.) To develop and guide the team members in enhancing their technical capabilities and increasing productivity
(3.) To ensure process compliance in the assigned module, and participate in technical discussionsorreview as a technical consultant for feasibility study (technical alternatives, best packages, supporting architecture best practices, technical risks, breakdown into components, estimations).
(4.) To prepare and submit status reports for minimizing exposure and risks on the project or closure of escalations.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.