M01 - Cloud Logging & Data Platform Engineer

Overview

We are looking for a Logging & Data Platform Engineer to design, build, and operate the logging and operational data-platform capabilities supporting the future SSOE platform.

You will build the platform that collects, transports, stores, indexes, searches, and serves operational data across MOE's technology environment, spanning on-premise infrastructure, networks, applications, GCC, AWS, Azure, and hybrid environments.

What You Will Be Working On

As a Logging & Data Platform Engineer, you will build the shared platform capabilities that enable engineering and operations teams to reliably collect and use operational data at scale.

You will work across logging, telemetry ingestion, data movement, storage, search, retention, and platform integration.

The role requires an engineer who understands both traditional enterprise infrastructure and modern cloud-native architectures and can design solutions that work securely and reliably across environment boundaries.

You will work closely with the Observability Engineer on telemetry requirements and with Data Engineering & Analytics on shared data-platform capabilities and integration patterns.

Key Responsibilities

Logging Platform Engineering

Design, build, and operate logging capabilities across on-premise infrastructure, networks, applications, cloud platforms, and hybrid environments

Collect logs from servers, network devices, applications, containers, databases, security appliances, cloud services, and other infrastructure sources

Build scalable log ingestion, routing, enrichment, storage, indexing, search, and retrieval capabilities

Define structured logging standards, schemas, metadata, tagging, and correlation conventions across services

Design appropriate retention, archival, lifecycle, and deletion policies for different classes of operational data

Support correlation between logs, metrics, events, and traces using common identifiers and telemetry standards

Work with the Observability Engineer to implement platform capabilities supporting end-to-end service observability

Data Collection & Integration

Design secure and resilient data movement between on-premise environments, GCC, and approved external services

Implement collection and forwarding patterns appropriate to different infrastructure, application, network, and security environments

Design for intermittent connectivity, network constraints, buffering, retry, back-pressure, and recovery between environments

Build event-driven and streaming patterns for moving operational data between producers and consumers

Integrate legacy and enterprise systems with modern cloud-native platform capabilities

Define clear interfaces and integration patterns between logging, observability, data engineering, and application platforms

Data Platform Engineering

Build shared platform capabilities for ingesting, storing, processing, querying, and serving operational data

Design scalable storage and query architectures appropriate to data volume, access patterns, retention requirements, and cost

Build ingestion, filtering, enrichment, and transformation pipelines for operational data

Provide APIs, query interfaces, or other serving mechanisms for authorised downstream consumers

Support operational datasets consumed by the User Portal, dashboards, reporting, automation, and Data Engineering & Analytics

Define schemas and data contracts for shared platform interfaces

Ensure platform changes remain backwards compatible or are coordinated with downstream consumers

Cloud & Platform Engineering

Design solutions using cloud-native logging, streaming, storage, search, and data capabilities

Build infrastructure and platform configuration using Infrastructure as Code

Automate build, test, deployment, configuration, and platform changes through CI/CD

Design for scalability, resilience, high availability, recoverability, and operational simplicity

Monitor platform capacity, performance, reliability, and cost

Security & Governance

Enforce MOE and Government data-classification requirements

Design secure routing and storage of operational data across security zones and environment boundaries

Apply appropriate encryption, access controls, authentication, and authorisation

Ensure logging pipelines do not unnecessarily expose credentials, secrets, or sensitive information

Implement audit-trail preservation and appropriate retention controls

Ensure data-residency requirements are considered when routing operational data between on-premise, GCC, cloud, and SaaS environments

Participate in security, architecture, and operational-readiness reviews

Reliability & Operations

Define SLOs and operational health indicators for logging and data-platform services

Build monitoring, alerting, failure detection, retry, and recovery into platform components

Monitor ingestion health, processing latency, data loss, storage utilisation, search performance, and platform availability

Participate in incident investigation, root-cause analysis, and post-incident reviews

Participate in operational support and on-call responsibilities for owned services

Maintain architecture documentation, operational procedures, and runbooks

What We Are Looking For

Experience

Minimum 3-5 years of experience in cloud engineering, platform engineering, DevOps, SRE, logging engineering, data platform engineering, or a related discipline

At least 2 years of hands-on experience building or operating production logging, telemetry, or data-platform capabilities

Demonstrated experience working with AWS and/or Azure cloud-native services

Experience integrating on-premise and cloud environments, or operating systems in a hybrid environment

Experience with production data ingestion, streaming, routing, storage, indexing, or search platforms

Experience implementing Infrastructure as Code and CI/CD for production environments

Experience designing systems for scalability, resilience, security, and operational support