Nebius, a Nasdaq-listed AI cloud infrastructure company, is hiring a Senior ML Engineer (Token Factory) for its IT team in Berlin. This senior-level role requires a Bachelor's degree and 5+ years of experience, offering a chance to work on high-performance inference and fine-tuning platforms designed to push foundation models to their hardware limits across tens of thousands of GPUs.
Core Responsibilities and Technical Focus
- Inference Optimization: Identify LLM bottlenecks and maximize performance for architectures like GPT-OSS, Kimi K2.5, DeepSeek V3.1/V3.2, and GLM-5.
- Inference Engines: Implement novel speculative decoding architectures, optimize LLM design components, and contribute to open-source inference engines.
- Low Precision Training & Inference: Design and productionize FP8, NVFP4, and MXFP4 pipelines to improve throughput and cost-efficiency.
Requirements
- 5+ years of experience and a Bachelor's degree.
- Deep understanding of machine learning theory, transformer architecture, GPU memory hierarchy, and compute/memory tradeoffs.
- Experience profiling GPU workloads using tools like Nsight and PyTorch profiler.
- Familiarity with LLM concepts such as MHA, RoPE, KV-cache, Flash Attention, and quantization.
- Understanding of large neural network training performance aspects, including sharding strategies and custom kernels.
- Strong software engineering skills, primarily in Python, with deep experience in modern deep learning frameworks, CI/CD, version control, and unit testing.
- Strong communication and leadership abilities, alongside an excellent command of English.
Additional nice-to-have qualifications include open-source inference engine experience (vLLM, SGLang, TensorRT-LLM), kernel language familiarity (Triton, Cute, CUTLASS, CUDA), distributed systems experience, and relevant open-source projects.
Nebius offers a market competitive compensation package alongside career growth opportunities, flexible working, and an international, collaborative environment.