Microsoft Azure Databricks – CL8

A full-time Microsoft Azure Databricks application support role in Pune, working in the office. The role covers production support for data-processing, ETL and data-pipeline applications, with 7.5 to 15 years of experience required. Albireo Recruiters is hiring for a technology company.

The role

This is a production support role for Microsoft Azure Databricks applications in Pune. You will support critical data-processing, analytics, ETL and data-pipeline applications. The work covers batch, streaming and data-ingestion workloads running on Azure Databricks. You will act as the subject matter expert for the platform and keep the business running smoothly.

A typical day involves monitoring Databricks jobs, notebooks, workflows, clusters and pipelines. You will look into incidents, read job, Spark, audit and cluster logs, and find the root cause. You will work with Data Engineering, Cloud, DevOps, Database, Security and Infrastructure teams to close issues end to end. You will also raise and track root-cause analysis, preventive actions, and documentation such as runbooks and SOPs.

We are looking for someone with hands-on experience in Azure Databricks support or data engineering support, and a sound understanding of Apache Spark. You should know Python, PySpark, Spark SQL and SQL well. Experience with Azure, Kafka, Databricks, ETL, data mining, API integration and data engineering is needed. Familiarity with Delta Lake, Unity Catalog, ADLS Gen2, Azure Data Factory and Azure Monitor is useful. You should be able to mentor junior team members.

The role is based in Pune and is an in-office position. The experience required is 7.5 to 15 years. This is a full-time role in the technology industry.

To apply, sign in on hyperwrk.com and apply in one step. A recruiter at Albireo Recruiters reads every application.

Must have

• Strong hands-on experience in Microsoft Azure Databricks production support, application support, data engineering support, or platform operations
• Strong understanding of Apache Spark architecture, distributed processing, executors, workers, partitions, stages, tasks, shuffle operations
• Hands-on knowledge of PySpark, Spark SQL, Python, and SQL
• Python
• SQL
• Strong log-analysis capability across Databricks logs, Spark logs, job logs, audit logs, cluster event logs, Azure Monitor, and Log Analytics
• Experience with Azure Databricks Workflows, Jobs, scheduling, dependencies, retries, alerts, and job-run monitoring
• Experience supporting Delta Lake, Delta tables, Unity Catalog, data quality checks, schema evolution
• Knowledge of ADLS Gen2, Azure Blob Storage, Azure Data Factory, Azure Key Vault
• Strong incident-management, problem-management, root-cause-analysis, ticket-management

Nice to have

• Experience with Azure DevOps, Git, CI/CD pipelines, Databricks Repos
• Exposure to Terraform, ARM templates, Bicep, or other Infrastructure as Code tools
• Knowledge of streaming technologies such as Structured Streaming, Kafka, Azure Event Hubs
• Experience with monitoring tools such as Azure Monitor, Log Analytics, Application Insights, Splunk, Dynatrace, Grafana, Prometheus, ELK Stack
• Azure certifications such as Azure Data Engineer Associate, Databricks Certified Data Engineer

What happens after you apply

The recruiter working this role reads your application. If it fits, they call you before anything is shared with the company. You see every stage in your account, in plain words.