Kaushik Subramanian

I am a Senior Research Scientist at SonyAI working on the flagship gaming project. I lead a team working on Gran Turismo Sophy.

My interests are in reinforcement learning, deep learning, and machine learning from human feedback. In particular, I like to find ways to take cutting-edge AI from research to practice.

I completed my Ph.D. in computer science at the Georgia Institute of Technology, where I was advised by Prof. Charles Isbell and Prof. Andrea Thomaz. Before that, I completed my M.S. in electrical and computer engineering at Rutgers University, advised by Prof. Michael Littman.

Email | Google Scholar | GitHub | LinkedIn

Highlights

Posts

• Traxion: How Sony AI created a new super-human racing agent that can ‘see’ using Gran Turismo 7, September 2024
• New Scientist: AI beats top racers at Gran Turismo – without cheating, July 2024
• SonyAI: From Hypothesis to Reality: The GT Sophy Team Explains the Evolution of the Breakthrough AI Agent, January 2024
• GTPlanet: How Sophy 2.0 is Different, and What Comes Next For Gran Turismo 7’s Super-Human AI, December 2023
• Ars Technica: Almost-unbeatable AI is now a permanent feature of Gran Turismo 7, November 2023
• S+T+ARTS Day interview for Frankfurt Book Fair, October 2022
• SonyAI: Training the World’s Fastest Gran Turismo Racer, June 2022

Videos

Invited Talks

• A panel on "Challenges Surrounding RL Deployment" at the Deployable RL: From Research to Practice workshop at the Reinforcement Learning Conference (RLC), August 2024
• A talk on Gran Turismo Sophy at the Connected Vehicles & Autonomous Driving workshop at the International Conference on Communication Systems & Networks, January 2023
• An invited talk and interview at the S+T+ARTS Day in the Frankfurt Book Fair, October 2022
• Keynote speaker at the Conference on Reinforcement Learning in Dresden, September 2022
• An invited speaker at the Fifth International Workshop on Intrinsically Motivated Open-ended Learning (IMOL), April 2022
• AI - Perspectives and Challenges of Real Data, Data Ninja Spring School, March 2022

Awards

• The paper on "A Super-human Vision-based Reinforcement Learning Agent for Autonomous Racing in Gran Turismo" won the Outstanding Paper Award on Applications of RL at RLC 2024.
• I am honored to have the won the 2023 Sony Outstanding Engineer Award for work on the permanent commercial deployment of a deep reinforcement learning agent in Gran Turismo® 7.
• The Interactive Machine Learning workshop I co-organized at IJCAI 2016 was awarded funding from the Artificial Intelligence Journal (AIJ).
• An “Introduction to Reinforcement Learning” video made with Prof. Michael Littman won the IJCAI 2009 AI Video Award for Best Narration.

Selected Publications

Journal

• Outracing champion Gran Turismo drivers with deep reinforcement learning
• Peter R. Wurman, Samuel Barrett, Kenta Kawamoto, James MacGlashan, Kaushik Subramanian, Thomas J. Walsh, Roberto Capobianco, Alisa Devlic, Franziska Eckert, Florian Fuchs, Leilani Gilpin, Piyush Khandelwal, Varun Kompella, HaoChih Lin, Patrick MacAlpine, Declan Oller, Takuma Seno, Craig Sherstan, Michael D. Thomure, Houmehr Aghabozorgi, Leon Barrett, Rory Douglas, Dion Whitehead, Peter Dürr, Peter Stone, Michael Spranger, Hiroaki Kitano
• Nature, 2022
• Abstraction from Demonstration for Efficient Reinforcement Learning in High-Dimensional Domains
• Luis C. Cobo, Kaushik Subramanian, Charles L. Isbell, Aaron D. Lanterman, Andrea L. Thomaz
• Artificial Intelligence Journal (AIJ), 2014

Conference

• SimBa: Simplicity Bias for Scaling Up Parameters in Deep Reinforcement Learning
• Hojoon Lee, Dongyoon Hwang, Donghu Kim, Hyunseung Kim, Jun Jet Tai, Kaushik Subramanian, Peter R. Wurman, Jaegul Choo, Peter Stone, Takuma Seno
• arXiv preprint, 2024
• Discovering Creative Behaviors through DUPLEX: Diverse Universal Features for Policy Exploration
• Borja G. Leon, Francesco Riccio, Kaushik Subramanian, Pete R. Wurman, Peter Stone
• NeurIPS, 2024
• A Super-human Vision-based Reinforcement Learning Agent for Autonomous Racing in Gran Turismo
• Miguel Vasco, Takuma Seno, Kenta Kawamoto, Kaushik Subramanian, Peter R. Wurman, Peter Stone
• *Outstanding Paper Award on Applications of RL (blog)
• Reinforcement Learning Conference (RLC), 2024
• Navigating Occluded Intersections with Autonomous Vehicles using Deep Reinforcement Learning
• David Isele, Reza Rahimi, Akansel Cosgun, Kaushik Subramanian, Kikuo Fujimura
• ICRA, 2018
• Exploration from Demonstration for Interactive Reinforcement Learning
• Kaushik Subramanian, Charles L. Isbell, Andrea L. Thomaz
• AAMAS, 2016
• Policy Shaping: Integrating Human Feedback with Reinforcement Learning
• Shane Griffith, Kaushik Subramanian, Jonathan Scholz, Charles L. Isbell, Andrea L. Thomaz
• NIPS, 2013
• Apprenticeship Learning about Multiple Intentions
• Monica Babes, Vukosi Marivate, Kaushik Subramanian, Michael L. Littman
• ICML, 2011

Academic Service

• Organizing Committee member for
  • Goal Specifications for Reinforcement Learning workshop at FAIM 2018
  • Adaptive and Learning Agents workshop at AAMAS 2017, 2018
  • Future of Interactive Learning Machines workshop at NIPS 2016
  • Interactive Machine Learning workshop at IJCAI 2016
• Program Committee member for
  • AAAI 2023
  • Adaptive and Learning Agents workshop at AAMAS (2014, 2015, 2016)
  • Interactive Machine Learning workshop at IUI 2013
• Invited Reviewer for
  • IEEE Robotics and Automation Letters, RA-L 2019
  • Adaptive and Learning Agents 2014, Special Issue of Connection Science (Taylor & Francis Journal)
  • Joint Conference on Development, Learning and Epigenetic Robotics, ICDL-EPIROB 2014
  • International Conference on Intelligent Robots and Systems, IROS 2014
• Teaching experience:
  • Teaching Assistant for Reinforcement Learning and Decision Making (CS 8803) by Prof. Charles Isbell and Prof. Michael Littman at Georgia Tech in Spring 2017 (for Online MS in CS degree)
  • Instructor for undergraduate machine learning (CS 4641) at Georgia Tech in Summer 2015

Adapted from Jon Barron's and Chelsea Finn's web page.