Senior Data Engineer/Azure/Kubernetes
Manhattan, New York
Hybrid
Full Time
$160k - $200k
Job Description:
Our client is a leading company in the retail and consumer goods industry, dedicated to providing actionable insights into consumer behavior through advanced data analytics. They focus on leveraging technology and data to help clients make informed decisions that drive business growth.
They are looking for a highly skilled Senior Data Engineer with expertise in Azure technologies. The ideal candidate will be responsible for designing, building, and maintaining scalable data infrastructure and pipelines that support our data science and analytics initiatives. You will play a crucial role in ensuring our data systems are robust, efficient, and secure.
Responsibilities:
- Design, develop, and maintain scalable data pipelines and ETL processes using Azure Data Factory, Azure Databricks, and other Azure services.
- Implement containerization strategies using Kubernetes to deploy and manage data applications efficiently.
- Collaborate with data scientists and analysts to understand data requirements and ensure the availability and accessibility of high-quality data.
- Optimize data storage solutions and implement best practices for data management and security.
- Monitor and troubleshoot data infrastructure, ensuring high availability and performance.
- Participate in architecture reviews and contribute to the design of the overall data strategy.
- Mentor junior data engineers and foster a culture of continuous learning and improvement within the team.
Preferred Skills/Qualifications:
- Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field.
- 5+ years of experience in data engineering with a strong focus on Azure technologies.
- Proficiency in Azure services such as Azure Data Lake Storage, Azure SQL Database, Azure Data Factory, Azure Synapse, and Azure Databricks.
- Experience with containerization technologies, particularly Kubernetes and Docker.
- Strong programming skills in Python and SQL.
- Familiarity with big data processing frameworks such as Apache Spark. Benefits:
- Competitive salary and performance-based bonuses.
- Comprehensive health, dental, and vision insurance.
- 401(k) retirement plan with company match.
- Professional development opportunities and ongoing training.
- Generous PTO and holiday leave
**Applicants must be currently authorized to work in the US on a full-time basis now and in the future.**
#LI-LT2
Our client is a leading company in the retail and consumer goods industry, dedicated to providing actionable insights into consumer behavior through advanced data analytics. They focus on leveraging technology and data to help clients make informed decisions that drive business growth.
They are looking for a highly skilled Senior Data Engineer with expertise in Azure technologies. The ideal candidate will be responsible for designing, building, and maintaining scalable data infrastructure and pipelines that support our data science and analytics initiatives. You will play a crucial role in ensuring our data systems are robust, efficient, and secure.
Responsibilities:
- Design, develop, and maintain scalable data pipelines and ETL processes using Azure Data Factory, Azure Databricks, and other Azure services.
- Implement containerization strategies using Kubernetes to deploy and manage data applications efficiently.
- Collaborate with data scientists and analysts to understand data requirements and ensure the availability and accessibility of high-quality data.
- Optimize data storage solutions and implement best practices for data management and security.
- Monitor and troubleshoot data infrastructure, ensuring high availability and performance.
- Participate in architecture reviews and contribute to the design of the overall data strategy.
- Mentor junior data engineers and foster a culture of continuous learning and improvement within the team.
Preferred Skills/Qualifications:
- Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field.
- 5+ years of experience in data engineering with a strong focus on Azure technologies.
- Proficiency in Azure services such as Azure Data Lake Storage, Azure SQL Database, Azure Data Factory, Azure Synapse, and Azure Databricks.
- Experience with containerization technologies, particularly Kubernetes and Docker.
- Strong programming skills in Python and SQL.
- Familiarity with big data processing frameworks such as Apache Spark. Benefits:
- Competitive salary and performance-based bonuses.
- Comprehensive health, dental, and vision insurance.
- 401(k) retirement plan with company match.
- Professional development opportunities and ongoing training.
- Generous PTO and holiday leave
**Applicants must be currently authorized to work in the US on a full-time basis now and in the future.**
#LI-LT2