Lead Confluent Kafka Engineer

We are seeking a seasoned Lead Confluent Kafka Engineer to oversee the deployment, configuration, administration, and production operations of Apache Kafka and the Confluent Platform. The ideal candidate will have extensive hands-on experience with Kafka, Java, Spring Boot, Linux, security, troubleshooting, and data streaming. This role is critical for maintaining reliable, secure, and highly available Kafka environments, ensuring seamless data streaming operations across various environments.

Key Responsibilities

- Deploy, configure, and maintain Apache Kafka and Confluent Platform across development, UAT, and production environments.
- Manage Kafka cluster provisioning, broker configuration, topic creation, partitioning, replication, retention, and capacity planning.
- Perform Kafka upgrades, patches, rolling restarts, configuration changes, and cluster expansions with minimal impact on production.
- Support deployment and maintenance of Kafka components such as Schema Registry, Kafka Connect, Control Center, and related services.
- Troubleshoot production issues related to brokers, partitions, ISR, replication, consumer lag, throughput, latency, storage, and connectivity.
- Configure and manage Kafka security, including TLS/mTLS, SASL, ACLs, certificates, keystores, and truststores.
- Support Java/Spring Boot Kafka application deployments, including producer/consumer configuration and integration troubleshooting.
- Collaborate with application, DevOps, infrastructure, and security teams to onboard new Kafka use cases and implement changes across environments.
- Support CI/CD pipelines, deployment automation, configuration management, and release activities for Kafka and Kafka-based applications.
- Conduct health checks, capacity planning, performance tuning, incident management, and root-cause analysis.
- Maintain operational documentation, deployment procedures, configuration standards, and recovery/runbook documentation.
- Ensure high availability, disaster recovery, backup/restore, and multi-cluster replication requirements are met where applicable.

Required Qualifications

- 5 to 10 years of hands-on experience with Apache Kafka and Confluent Platform in production environments.
- Deep understanding of Kafka architecture, including brokers, partitions, replication, ISR, consumer groups, offsets, retention policies, and delivery semantics.
- Proven expertise in Kafka deployment, configuration, upgrades, patching, and ongoing operational maintenance.
- Strong proficiency in Java and Spring Boot for Kafka producer and consumer integrations.
- Solid Linux, networking, JVM, troubleshooting, and production support skills.
- Experience with Kafka security configurations, particularly TLS/mTLS, SASL, ACLs, and certificate management.
- Familiarity with Confluent Kafka and data streaming technologies.
- Minimum Bachelor’s degree in a relevant field.

Preferred Qualifications and Additional Skills

- Experience with container orchestration platforms such as Kubernetes, OpenShift, Anthos, and Helm for containerized Kafka deployments.
- Knowledge of Kafka Connect, Schema Registry, and Confluent Control Center.
- Familiarity with CI/CD tools like Jenkins, GitLab CI, Azure DevOps, or equivalents.
- Monitoring experience using tools such as Splunk, AppDynamics, Prometheus, Grafana, or Confluent monitoring solutions.
- Exposure to Kafka disaster recovery, multi-cluster architectures, replication, and migration activities.
- Experience supporting Kafka in mission-critical environments such as banking or telecommunications.

This is a full-time onsite position based in Islamabad, offering the opportunity to lead Kafka initiatives within a dynamic and growing team. Candidates should possess a strong technical foundation and a proactive approach to maintaining high standards of Kafka platform reliability and security.