About Me

I am a Member of Technical Staff at Anthropic. Previously, I was a Research Scientist at Salesforce AI Research.

I earned my PhD at Georgia Tech, where I was fortunate to be advised by Mark Davenport and collaborate closely with Ashwin Pananjady. Prior to that, I earned my BSE at the University of Michigan.

In my free time, I enjoy cooking (and eating), reading, running, and watching basketball (NBA and college).

Publications and Preprints

• Foundational Automatic Evaluators: Scaling Multi-Task Generative Evaluator Training for Reasoning-Centric Domains
  Austin Xu, Xuan-Phi Nguyen, Yilun Zhou, Chien-Sheng Wu, Caiming Xiong, Shafiq Joty
  ICLR 2026
  
  
• Variation in verification: Understanding verification dynamics in large language models
  Yefan Zhou, Austin Xu, Yilun Zhou, Janvijay Singh, Jiang Gui, Shafiq Joty
  ICLR 2026
  
  
• On the Shelf Life of Fine-Tuned LLM Judges: Future Proofing, Backward Compatibility, and Question Generalization
  Janvijay Singh, Austin Xu, Yilun Zhou, Yefan Zhou, Dilek Hakkani-Tur, Shafiq Joty
  ICLR 2026
  
  
• LiveResearchBench: A Live Benchmark for User-Centric Deep Research in the Wild
  Jiayu Wang, Yifei Ming, Riya Dulepet, Qinglin Chen, Austin Xu, Zixuan Ke, Frederic Sala, Aws Albarghouthi, Caiming Xiong, Shafiq Joty
  ICLR 2026
  
  
• Hard2Verify: A Step-Level Verification Benchmark for Open-Ended Frontier Math
  Shrey Pandit*, Austin Xu*, Xuan-Phi Nguyen, Yifei Ming, Caiming Xiong, Shafiq Joty
  arXiv 2025
  
  
• Synthesizing Agentic Data for Web Agents with Progressive Difficulty Enhancement Mechanisms
  Shrey Pandit, Xuan-Phi Nguyen, Yifei Ming, Austin Xu, Jiayu Wang, Caiming Xiong, Shafiq Joty
  arXiv 2025
  
  
• SFR-DeepResearch: Towards Effective Reinforcement Learning for Autonomously Reasoning Single Agents
  Xuan-Phi Nguyen, Shrey Pandit, Revanth Gangi Reddy, Austin Xu, Silvio Savarese, Caiming Xiong, Shafiq Joty
  Technical Report, arXiv 2025
  
  
• J4R: Learning to Judge with Equivalent Initial State Group Relative Policy Optimization
  Austin Xu, Yilun Zhou, Xuan-Phi Nguyen, Caiming Xiong, Shafiq Joty
  arXiv 2025
  
  
• MAS-ZERO: Designing Multi-Agent Systems with Zero Supervision
  Zixuan Ke, Austin Xu, Yifei Ming, Xuan-Phi Nguyen, Caiming Xiong, Shafiq Joty
  arXiv 2025
  
  
• Direct Judgement Preference Optimization
  Peifeng Wang*, Austin Xu*, Yilun Zhou, Caiming Xiong, Shafiq Joty
  EMNLP 2025, Main Conference
  
  
• Topic-Guided Reinforcement Learning with LLMs for Enhancing Multi-Document Summarization
  Chuyuan Li, Austin Xu, Shafiq Joty, Giuseppe Carenini
  EMNLP 2025, Findings
  
  
• Evaluating Judges as Evaluators: The JETTS Benchmark of LLM-as-Judges as Test-Time Scaling Evaluators
  Yilun Zhou*, Austin Xu*, Peifeng Wang, Caiming Xiong, Shafiq Joty
  ICML 2025
  
  
• A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systems
  Zixuan Ke, Fangkai Jiao, Yifei Ming, Xuan-Phi Nguyen, Austin Xu, Do Xuan Long, Minzhi Li, Chengwei Qin, Peifeng Wang, Silvio Savarese, Caiming Xiong, Shafiq Joty
  TMLR 2025, Survey Certificate
  [arXiv]
  
  
• Does Context Matter? ContextualJudgeBench for Evaluating LLM-based Judges in Contextual Settings
  Austin Xu*, Srijan Bansal*, Yifei Ming, Semih Yavuz, Shafiq Joty
  ACL 2025, Main Conference
  [arXiv]
  
  
• SFR-RAG: Towards Contextually Faithful LLMs
  Xuan-Phi Nguyen, Shrey Pandit, Senthil Purushwalkam, Austin Xu, Hailin Chen, Yifei Ming, Zixuan Ke, Silvio Savarese, Caiming Xiong, Shafiq Joty
  Technical Report, arXiv 2024
  
  
• Large Language Model Augmented Exercise Retrieval for Personalized Language Learning
  Austin Xu, Will Monroe, Klinton Bicknell
  Learning Analytics and Knowledge (LAK) 2024
  Short version in the NeurIPS 2023 Workshop on Generative AI for Education (GAIED).
  [arXiv]
  
  
• Perceptual adjustment queries and an inverted measurement paradigm for low-rank metric learning
  Austin Xu, Andrew D. McRae, Jingyan Wang, Mark A. Davenport, and Ashwin Pananjady
  NeurIPS 2023
  Short version in the ICML 2023 Many Facets of Preference Learning Workshop
  [arXiv: extended version][code]
  
  
• HandsOff: Labeled dataset generation with no additional human annotations
  Austin Xu, Mariya I. Vasileva, Achal Dave, and Arjun Seshadri
  CVPR 2023
  Highlight Award (top 2.5% of submissions, 26% conference acceptance rate)
  Short version in the NeurIPS 2022 SyntheticData4ML Workshop
  [arXiv][website] [code]
  
  
• Active Metric Learning and Classification using Similarity Queries
  Namrata Nadagouda, Austin Xu, Mark A. Davenport
  UAI 2023
  Short version in the NeurIPS 2022 Workshop on Human in the Loop Learning.
  [arXiv]
  
  
• Simultaneous Preference and Metric Learning from Paired Comparisons
  Austin Xu and Mark A. Davenport
  NeurIPS 2020
  Spotlight Presentation (top 4% of submissions, 20% conference acceptance rate)
  [arXiv] [website] [talk]
  
  

Experience

Work Experience

• AI Research Intern at Duolingo (Summer 2023)
• Applied Scientist Intern at Amazon (Summer, Fall 2022)
• R&D Summer Intern at Sandia National Laboratories (Summer 2018)
• Student Intern at General Motors (Summer 2017)

Teaching Experience

• Spring 2022: Head TA, Statistical Machine Learning (ECE 6254) [website]
• Fall 2019/Spring 2020/Summer 2020: TA, Professional and Technical Communications (ECE 3005)
• Fall 2018/Spring 2019: IA, Discrete Mathematics (University of Michigan -- EECS 203)