TPU Performance Engineer

About the Role We are looking for a TPU Performance Engineer to optimize large-scale LLM inference performance on Google TPU . You will work across TPU, kernels, compilers, and runtime systems , improving inference latency, throughput, and overall efficiency.

Key Responsibilities

Optimize LLM inference workloads on Google TPU.

Develop and optimize TPU backends, kernels, compiler integrations, and runtime components .

Optimize performance-critical workloads such as Attention, GEMM, KV Cache, Sampling, and fused kernels .

Work with JAX, XLA, Pallas and related compiler/runtime technologies.

Build benchmarking and profiling infrastructure and identify bottlenecks across compute, memory, compilation, and runtime .

Collaborate with model, inference, compiler, and hardware teams to improve production performance.

Requirements

Bachelor's degree or equivalent experience in Computer Science, Engineering, Machine Learning, Systems , or a related field.

Hands-on experience with TPU performance optimization , preferably with JAX, XLA, Pallas , or related technologies.

Strong understanding of TPU architecture, memory behavior, compilation, and ML workload performance .

Experience with ML kernel optimization, LLM inference, backend/runtime development, or performance engineering .

Strong C++ and/or Python programming skills.

Solid experience with performance profiling and benchmarking .

Preferred Qualifications

Experience with vLLM, SGLang, TensorRT-LLM or other LLM inference frameworks.

Familiarity with LLM serving, batching, KV Cache, decoding, and inference optimization .

Experience with MLIR, LLVM, Pallas, XLA or other compiler technologies.

Knowledge of FP8, INT8, mixed precision, or quantization .

Contributions to vLLM, JAX/XLA, Pallas, PyTorch/XLA or other open-source AI infrastructure projects.