Senior Architect, Model Performance and Post-Training

Job Description:

• Develop meticulous frameworks to measure and analyze coding model performance across code generation, debugging, code completion, repository-level reasoning, and multi-step software engineering tasks
• Develop and optimize post-training pipelines for coding models, including supervised fine-tuning (SFT), preference optimization, reinforcement learning (RL), and outcome-based or process-based reward modeling
• Analyze benchmark results, training dynamics, failure modes, and model behaviors to find opportunities for improvement
• Improve model performance on sophisticated coding tasks through targeted training interventions, reward design, curriculum development, and iterative error analysis
• Build scalable, reproducible infrastructure for training experiments, evaluation pipelines, and model performance analysis
• Collaborate with research scientists, infrastructure engineers, and product teams to turn model insights into production-ready capabilities

Requirements:

• Bachelor's degree (or equivalent experience) in Computer Science or AI
• 12+ years of meaningful work experience
• Excellent programming skills in Python
• Strong understanding of large language models, transformer architectures, and training and evaluation of generative models
• Foundational understanding of reinforcement learning algorithms and implementation, including MDPs, policies, and reward design
• Deep understanding of software engineering fundamentals, code generation, automated testing, and program correctness
• Proven track record to lead model development life cycles, including dataset versioning, experiment tracking, and evaluation pipelines
• Experience training or post-training state-of-the-art coding models or models designed for software engineering agents
• Experience with execution-based rewards, verifiable rewards, reward hacking mitigation, and automated code evaluation
• Familiarity with coding benchmarks and evaluation environments such as SWE-bench

Benefits:

• Highly competitive salaries
• Comprehensive benefits package
• Equity
• Benefits for you and your family