Senior Data Platform System Engineer

Ensure

high availability and reliability

of the enterprise data platform (Cloudera and Talend Big Data)

Deliver

scalable and performant data infrastructure

supporting batch and near real-time workloads

Enable

secure and compliant data operations

aligned with enterprise and regulatory requirements

Improve

platform automation and operational efficiency

through DevOps and Infrastructure-as-Code

Provide

platform support and partnership

to data engineering, analytics, and business teams

Key Responsibilities

Data Platform Engineering & Operations

Design, deploy, and manage large-scale data platform environments (primarily Cloudera and Talend Big Data) Operate and optimize Hadoop ecosystem components (HDFS, Hive, Spark, HBase, Impala, Iceberg, Ozone) Manage cluster lifecycle including provisioning, scaling, patching, and upgradesEnsure platform uptime, stability, and service reliability through proactive monitoring and maintenance

Performance, Scalability & Capacity Management

Monitor workload performance and optimize data processing efficiencyPerform capacity planning to support increasing data volumes and usageTroubleshoot performance bottlenecks across compute, storage, and query layers

Platform Delivery & Execution Planning

Plan and coordinate execution of platform upgrades, patching, and scaling activitiesManage deployment schedules and ensure minimal service disruptionCoordinate cross-team dependencies (infrastructure, network, security)Track and report execution progress against platform roadmap

DevOps, Automation & Platform Engineering

Implement Infrastructure-as-Code using tools such as Terraform and AnsibleDesign and maintain CI/CD pipelines for platform configuration Automate platform provisioning, deployment, and operational workflowsImprove observability through monitoring, alerting, and logging practices

Security, Governance & Compliance

Implement and manage platform security controls including authentication (Kerberos), authorization (e.g. Ranger), and encryptionEnsure compliance with enterprise data governance, security policies, and regulatory requirementsSupport auditability, access control, and data protection mechanisms

Collaboration & Platform Enablement

Work closely with data engineers, data scientists, and business teams to enable data solution delivery Partner with infrastructure, network, and security teams to ensure platform integration and stabilityProvide guidance, troubleshooting, and best practices to users of the platformDocument platform architecture, processes, and operational procedures

Skills &Experience

5- 10 years of experience in data platform engineering, big data operations, or infrastructure engineering

Hands-on experience managing big data platforms (Cloudera CDP, Hadoop ecosystem)

Deep understanding of Hadoop components: HDFS, Hive, Spark, HBase, and related services

Experience with cluster operations, performance tuning, and troubleshooting

Proficiency in automation and scripting (Python, Bash, Terraform, Ansible)

Experience with CI/CD pipelines and DevOps practices

Solid understanding of data platform security and access control concepts

Analytical and problem-solving skills with a proactive mindset

Ability to work in a fast-paced, cross-functional environment

Good communication skills with ability to translate technical topics to stakeholders

Ownership mentality with focus on platform reliability and continuous improvement

Good to have

Experience with Databricks, Talend, or hybrid data platforms

Knowledge of containerization and orchestration

Experience with streaming platforms (e.g. Kafka) or real-time processing

Familiarity with modern data lake-house concepts (e.g. Iceberg, Delta)

Exposure to cloud or hybrid environments (Azure)

Navneet

Goel | EA License No. : 02C3423 | Reg No. R1982194