Job Description

Principal ML Engineer

About the Company

The Role

What You'll Do

• Build and own the training pipelines: data preparation, reproducible fine-tuning runs, experiment tracking, and release automation
• Build the evaluation infrastructure: automated eval runs, regression gates, dashboards, and dataset versioning. Research defines what good means. You build the machinery that measures it
• Own model serving in production: low-latency inference, batching, optimization, autoscaling, and cost
• Ship model updates safely with versioning, canarying, rollback, and drift monitoring
• Build repeatable workflows for adapting models to new domains and customer needs
• Turn expert labels and reviewer feedback into clean training and evaluation data
• Set the bar for ML infrastructure as the team grows

What We're Looking For

• 8+ years of software engineering experience, including 4+ years building infrastructure for ML or LLM systems in production
• Hands-on depth with the modern LLM stack: PyTorch, distributed training, fine-tuning at scale (LoRA, SFT), and inference engines such as vLLM or TensorRT-LLM
• Experience building eval harnesses, regression gates, or dataset pipelines, with a solid understanding of precision, recall, and calibration
• Production mindset — you have owned model serving with real latency, reliability, and cost constraints, not just notebooks
• Strong fundamentals: Python, containers, CI/CD, cloud infrastructure, observability
• High ownership on a small team: scope your own work, ship weekly, make pragmatic build-vs-buy calls
• You enjoy being the engineering counterpart to a research partner — tight collaboration, clear interfaces, no turf wars

Nice to Have

• Experience productionizing small or specialized language models
• Experience with structured-output serving or constrained decoding in production
• Prior work in a regulated or high-stakes domain such as fintech, healthcare, legal, or trust and safety
• Experience deploying models into customer-controlled environments

Compensation and Benefits

• $200,000–$250,000 base salary, depending on experience
• Performance bonus and meaningful early-stage equity
• Health, dental, and vision coverage
• Hybrid work from our New York office