Detalhe da Oferta:
We are looking for a DataOps Engineer to join a project with one of our French clients in the fashion industry, supporting the data infrastructure behind a large-scale e-commerce platform.
The work can be done remotely, and communication with stakeholders will be mostly in English.
Responsibilities:
• Support and maintain the cloud data infrastructure, ensuring reliability and performance.
• Manage and optimize data pipelines and workflows using Airflow (Composer) and BigQuery.
• Collaborate with development and analytics teams to ensure smooth data delivery across systems.
• Implement Infrastructure-as-Code using Terraform, and manage CI/CD processes through GitLab.
• Monitor and troubleshoot data services across GCP components (Dataflow, Cloud Functions, Storage, etc.).
• Ensure secure and efficient access controls (IAM) and compliance with internal governance standards.
Requirements:
• Hands-on experience with Google Cloud Platform, including Composer (Airflow), BigQuery, Dataflow, Storage, IAM, Cloud Functions, and Compute Engine.
• 2+ years of experience with Apache Airflow (Composer).
• 4+ years of experience with BigQuery, including modeling, query optimization, and architecture.
• 2+ years of experience with Terraform and GitLab, particularly in managing IaC and CI/CD pipelines.
• Solid knowledge of DevOps/data workflows and cloud-native practices.
• Fluency in English.
Nice to Have:
• Experience with Azure (AKS, ADLS Gen2, Databricks with Unity Catalog).
• Familiarity with Pub/Sub, Kafka, and optimized usage of Google Cloud Storage.
• Experience with Apache Spark or PySpark.
If it sounds like you, let’s talk some more!
The work can be done remotely, and communication with stakeholders will be mostly in English.
Responsibilities:
• Support and maintain the cloud data infrastructure, ensuring reliability and performance.
• Manage and optimize data pipelines and workflows using Airflow (Composer) and BigQuery.
• Collaborate with development and analytics teams to ensure smooth data delivery across systems.
• Implement Infrastructure-as-Code using Terraform, and manage CI/CD processes through GitLab.
• Monitor and troubleshoot data services across GCP components (Dataflow, Cloud Functions, Storage, etc.).
• Ensure secure and efficient access controls (IAM) and compliance with internal governance standards.
Requirements:
• Hands-on experience with Google Cloud Platform, including Composer (Airflow), BigQuery, Dataflow, Storage, IAM, Cloud Functions, and Compute Engine.
• 2+ years of experience with Apache Airflow (Composer).
• 4+ years of experience with BigQuery, including modeling, query optimization, and architecture.
• 2+ years of experience with Terraform and GitLab, particularly in managing IaC and CI/CD pipelines.
• Solid knowledge of DevOps/data workflows and cloud-native practices.
• Fluency in English.
Nice to Have:
• Experience with Azure (AKS, ADLS Gen2, Databricks with Unity Catalog).
• Familiarity with Pub/Sub, Kafka, and optimized usage of Google Cloud Storage.
• Experience with Apache Spark or PySpark.
If it sounds like you, let’s talk some more!