Akifumi Wachi

I am a Chief Research Scientist at LY Corporation. My research interests lie primarily in reinforcement learning (RL), and span the entire theory-to-application spectrum from fundamental advances to deployment in real-world systems. Especially, I am interested in how a policy should and can be trained and deployed in safety-critical problems.

I am from Japan. My Japanese name is 和地 瞭良. 日本語ページは こちら

Research Interests

My core research interest is in reinforcement learning (RL) for reliable sequential decision making and AI Safety. My current research interests can be broadly organized into three overlapping groups:

  • Safe RL: study how to develop RL approaches with certifiable safety guarantees.
  • RL × NLP: study how to apply RL approaches in NLP tasks (e.g., safety alignment of language models).
  • Adversarial Testing via RL: study how to utlize RL approaches for testing AI safety and finding failure cases.

Experiences

  • 2023 - present: Chief Research Scientist, LY Corporation
  • 2022 - 2023: Senior Research Scientist, LINE Corporation
  • 2018 - 2022: Research Scientist, IBM Research AI

  • 2021: Ph.D. in Computer Science, University of Tsukuba
  • 2018: M.S. in Aeronautics and Astronautics, University of Tokyo
  • 2016: B.S. in Aeronautics and Astronautics, University of Tokyo

Selected Publications


Stepwise Alignment for Constrained Language Model Policy Optimizations

Safe Exploration in Reinforcement Learning: A Generalized Formulation and Algorithms

Safe Policy Optimization with Local Generalized Linear Function Approximations

Safe Reinforcement Learning in Constrained Markov Decision Processes

Failure-Scenario Maker for Rule-Based Agent using Multi-agent Adversarial Reinforcement Learning and its Application to Autonomous Driving

Safe Exploration and Optimization of Constrained MDPs using Gaussian Processes

  • Akifumi Wachi, Yanan Sui, Yisong Yue, Masahiro Ono
  • AAAI, 2018
  • [PDF] [Slide]

Publications

  • Stepwise Alignment for Constrained Language Model Policy Optimizations
    Akifumi Wachi, Thien Q. Tran, Rei Sato, Takumi Tanabe, Youhei Akimoto
    Neural Information Processing Systems (NeurIPS), 2024.
    [arXiv] [Poster] [GitHub] [Huggigface (SACPO)] [Huggigface (P-SACPO)]

  • Flipping-based Policy for Chance-Constrained Markov Decision Processes
    Xun Shen, Shuo Jiang, Akifumi Wachi, Kazumune Hashimoto, Sebastien Gros
    Neural Information Processing Systems (NeurIPS), 2024.
    [arXiv]

  • A Survey of Constraint Formulations in Safe Reinforcement Learning
    Akifumi Wachi, Xun Shen, Yanan Sui
    International Joint Conference on Artificial Intelligence (IJCAI), 2024.
    [arXiv] [Poster] [Slide]

  • Safe Reinforcement Learning Using Model Predictive Control with Probabilistic Control Barrier Function
    Xun Shen, Akifumi Wachi, Wataru Hashimoto, Kazumune Hashimoto, Shigemasa Takai
    American Control Conference (ACC), 2024.
    [IEEE]

  • Long-term Safe Reinforcement Learning with Binary Feedback
    Akifumi Wachi, Wataru Hashimoto, Kazumune Hashimoto
    AAAI Conference on Artificial Intelligence (AAAI), 2024.
    [PDF] [Poster] [Slide]

  • Safe Exploration in Reinforcement Learning: A Generalized Formulation and Algorithms
    Akifumi Wachi, Wataru Hashimoto, Xun Shen, Kazumune Hashimoto
    Neural Information Processing Systems (NeurIPS), 2023.
    [PDF] [arXiv] [Poster] [Video]

  • Safe Policy Optimization with Local Generalized Linear Function Approximations
    Akifumi Wachi, Yunyue Wei, Yanan Sui
    Neural Information Processing Systems (NeurIPS), 2021.
    [PDF] [OpenReview] [arXiv] [Poster]

  • Polar Embedding
    Ran Iwamoto, Ryosuke Kohita, Akifumi Wachi
    The SIGNLL Conference on Computational Natural Language Learning (CoNLL), 2021.
    [PDF]

  • Neuro-Symbolic Reinforcement Learning with First-Order Logic
    Daiki Kimura, Masaki Ono, Subhajit Chaudhury, Ryosuke Kohita, Akifumi Wachi, Don Joven Agravante, Michiaki Tatsubori, Asim Munawar, Alexander Gray
    Empirical Methods in Natural Language Processing (EMNLP), Short paper, 2021.
    [PDF]

  • Language-based General Action Template for Reinforcement Learning Agents
    Ryosuke Kohita, Akifumi Wachi, Daiki Kimura, Subhajit Chaudhury, Michiaki Tatsubori, Asim Munawar
    Association for Computational Linguistics (ACL), Findings, 2021.
    [PDF]

  • Q-learning with Language Model for Edit-based Unsupervised Summarization
    Ryosuke Kohita, Akifumi Wachi, Yang Zhao, Ryuki Tachibana
    Empirical Methods in Natural Language Processing (EMNLP), 2020.
    [PDF] [arXiv]

  • Safe Reinforcement Learning in Constrained Markov Decision Processes
    Akifumi Wachi, Yanan Sui
    International Conference on Machine Learning (ICML), 2020.
    [PDF] [arXiv] [Slide] [Video]

  • Failure-Scenario Maker for Rule-Based Agent using Multi-agent Adversarial Reinforcement Learning and its Application to Autonomous Driving
    Akifumi Wachi
    International Joint Conference on Artificial Intelligence (IJCAI), 2019.
    [PDF] [arXiv] [Slide] [Poster] [Simulation]

  • Safe Exploration and Optimization of Constrained MDPs using Gaussian Processes
    Akifumi Wachi, Yanan Sui, Yisong Yue, Masahiro Ono
    AAAI Conference on Artificial Intelligence (AAAI), 2018.
    [PDF] [Slide]

  • Integral Design Method for Simple and Small Mars Lander System Using Membrane Aeroshell
    Ryo Sakagami, Ryohei Takahashi, Akifumi Wachi, Yuki Koshiro, Hiroyuki Maezawa, Yasko Kasai, Shinichi Nakasuka
    Acta Astronautica, 2018.
    [PDF]

  • The Conceptual Design of a Novel Simple and Small-sized Mars lander
    Ryohei Takahashi, Ryo Sakagami, Akifumi Wachi, Yasko Kasai, Shinichi Nakasuka
    IEEE Aerospace Conference, 2018.
    [PDF]

  • Mars Entry, Descent, and Landing by Small THz Spacecraft via Membrane Aeroshell
    Akifumi Wachi, Ryohei Takahashi, Ryo Sakagami, Yuki Koshiro, Yasko Kasai, Shinichi Nakasuka
    AIAA SPACE and Astronautics Forum and Exposition (AIAA SPACE), 2017.
    [PDF] [Slide]

  • Hazard Avoidance Control Using Stochastic Optimization for Mars Safe Landing
    Ryohei Takahashi, Akifumi Wachi, Ryu Funase, Shinichi Nakasuka
    International Symposium on Space Technology and Sciences (ISTS), 2017.

  • Low-Thrust Trajectory Design to Improve Overall Mission Success Probability Incorporating Target Changes in Case of Engine Failures
    Akifumi Wachi
    International Symposium on Space Technology and Science (ISTS), 2017.
    Best Student Paper Award (First Prize)
    [PDF]

  • Fault-Tolerant Low-Thrust Trajectory Design with Backups for Multiple Targets
    Akifumi Wachi, Naoya Ozaki, Shinichi Nakasuka
    IFAC Symposium on Automatic Control in Aerospace (ACA), 2016.
    Best Student Paper Award (First Prize)
    [PDF]

Demo

  • LOA: Logical Optimal Actions for Text-based Interaction Games
    Daiki Kimura, Subhajit Chaudhury, Masaki Ono, Michiaki Tatsubori, Don Joven Agravante, Asim Munawar, Akifumi Wachi, Ryosuke Kohita, and Alexander Gray
    Association for Computational Linguistics (ACL), Demo Paper, 2021.
    [PDF]

Patents

Professional Activities

Conference Program Committee Member

  • ICML (2021 - 2024), NeurIPS (2021 - 2024), ICLR (2022 - 2025), AISTATS (2024 - 2025)
  • AAAI (2020 - 2025), IJCAI (2020 - 2024)
  • COLM (2024), ACL (2020 - 2023), EMNLP (2020)

Journal Reviewer

  • ACM Transactions on Evolutionary Learning and Optimization (Associate Editor)
  • IEEE Transactions on Pattern Analysis and Machine Intelligence

Others

Awards

  • Top Reviewer of NeurIPS 2024 [link]
  • Dean’s Award (Second Prize), from Faculty of Engineering, The University of Tokyo, 2018
  • Best Student Paper Award (First Prize), Joint Conference: 31st ISTS, 26th ISSFD, and 8th NSAT, 2017
  • Best Student Paper Award (First Prize), IFAC Symposium on Automatic Control in Aerospace (ACA), 2016, [link]

Fellowships

  • Financial support to participate in Thirty-second AAAI conference on Artificial Intelligence by Society for Promotion of Space Science (SPSS), February, 2018
  • Financial support to participate in International Astronautical Congress (IAC) by the Japan Society for Aeronautical and Space Sciences (JSASS), September, 2017
  • Go Global Scholarships 2016 Short Study Abroad Scholarships: the University of Tokyo, 2016
  • Fellowship of the Leading Graduates Schools Program: Global Leader Program for Social Design and Management by the Ministry of Education, Culture, Sports, Science and Technology in Japan, 2016

Invited Talk

-->