Akifumi Wachi
I am a Chief Research Scientist at LY Corporation. My research interests lie primarily in reinforcement learning (RL), and span the entire theory-to-application spectrum from fundamental advances to deployment in real-world systems. Especially, I am interested in how a policy should and can be trained and deployed in safety-critical problems.
I am from Japan. My Japanese name is 和地 瞭良. 日本語ページは こちら
Research Interests
My core research interest is in reinforcement learning (RL) for reliable sequential decision making and AI Safety. My current research interests can be broadly organized into three overlapping groups:
- Safe RL: study how to develop RL approaches with certifiable safety guarantees.
- RL × NLP: study how to apply RL approaches in NLP tasks (e.g., safety alignment of language models).
- Adversarial Testing via RL: study how to utlize RL approaches for testing AI safety and finding failure cases.
Experiences
- 2023 - present: Chief Research Scientist, LY Corporation
- 2022 - 2023: Senior Research Scientist, LINE Corporation
- 2018 - 2022: Research Scientist, IBM Research AI
- 2021: Ph.D. in Computer Science, University of Tsukuba
- 2018: M.S. in Aeronautics and Astronautics, University of Tokyo
- 2016: B.S. in Aeronautics and Astronautics, University of Tokyo
Selected Publications
Safe Policy Optimization with Local Generalized Linear Function Approximations
- Akifumi Wachi, Yunyue Wei, Yanan Sui
- NeurIPS, 2021
- [PDF] [OpenReview] [arXiv] [Poster]
Failure-Scenario Maker for Rule-Based Agent using Multi-agent Adversarial Reinforcement Learning and its Application to Autonomous Driving
- Akifumi Wachi
- IJCAI, 2019
- [PDF] [arXiv] [Slide] [Poster] [Simulation]
Publications
- Stepwise Alignment for Constrained Language Model Policy Optimizations
Akifumi Wachi, Thien Q. Tran, Rei Sato, Takumi Tanabe, Youhei Akimoto
Neural Information Processing Systems (NeurIPS), 2024.
[arXiv] [GitHub] [Huggigface (SACPO)] [Huggigface (P-SACPO)] - Flipping-based Policy for Chance-Constrained Markov Decision Processes
Xun Shen, Shuo Jiang, Akifumi Wachi, Kazumune Hashimoto, Sebastien Gros
Neural Information Processing Systems (NeurIPS), 2024.
[PDF Forthcomming] - A Survey of Constraint Formulations in Safe Reinforcement Learning
Akifumi Wachi, Xun Shen, Yanan Sui
International Joint Conference on Artificial Intelligence (IJCAI), 2024.
[arXiv] [Poster] [Slide] - Safe Reinforcement Learning Using Model Predictive Control with Probabilistic Control Barrier Function
Xun Shen, Akifumi Wachi, Wataru Hashimoto, Kazumune Hashimoto, Shigemasa Takai
American Control Conference (ACC), 2024.
[PDF forthcoming] - Long-term Safe Reinforcement Learning with Binary Feedback
Akifumi Wachi, Wataru Hashimoto, Kazumune Hashimoto
AAAI Conference on Artificial Intelligence (AAAI), 2024.
[PDF] [Poster] [Slide] - Safe Exploration in Reinforcement Learning: A Generalized Formulation and Algorithms
Akifumi Wachi, Wataru Hashimoto, Xun Shen, Kazumune Hashimoto
Neural Information Processing Systems (NeurIPS), 2023.
[PDF] [arXiv] [Poster] [Video] - Safe Policy Optimization with Local Generalized Linear Function Approximations
Akifumi Wachi, Yunyue Wei, Yanan Sui
Neural Information Processing Systems (NeurIPS), 2021.
[PDF] [OpenReview] [arXiv] [Poster] - Polar Embedding
Ran Iwamoto, Ryosuke Kohita, Akifumi Wachi
The SIGNLL Conference on Computational Natural Language Learning (CoNLL), 2021.
[PDF] - Neuro-Symbolic Reinforcement Learning with First-Order Logic
Daiki Kimura, Masaki Ono, Subhajit Chaudhury, Ryosuke Kohita, Akifumi Wachi, Don Joven Agravante, Michiaki Tatsubori, Asim Munawar, Alexander Gray
Empirical Methods in Natural Language Processing (EMNLP), Short paper, 2021.
[PDF] - Language-based General Action Template for Reinforcement Learning Agents
Ryosuke Kohita, Akifumi Wachi, Daiki Kimura, Subhajit Chaudhury, Michiaki Tatsubori, Asim Munawar
Association for Computational Linguistics (ACL), Findings, 2021.
[PDF] - Q-learning with Language Model for Edit-based Unsupervised Summarization
Ryosuke Kohita, Akifumi Wachi, Yang Zhao, Ryuki Tachibana
Empirical Methods in Natural Language Processing (EMNLP), 2020.
[PDF] [arXiv] - Safe Reinforcement Learning in Constrained Markov Decision Processes
Akifumi Wachi, Yanan Sui
International Conference on Machine Learning (ICML), 2020.
[PDF] [arXiv] [Slide] [Video] - Failure-Scenario Maker for Rule-Based Agent using Multi-agent Adversarial Reinforcement Learning and its Application to Autonomous Driving
Akifumi Wachi
International Joint Conference on Artificial Intelligence (IJCAI), 2019.
[PDF] [arXiv] [Slide] [Poster] [Simulation] - Safe Exploration and Optimization of Constrained MDPs using Gaussian Processes
Akifumi Wachi, Yanan Sui, Yisong Yue, Masahiro Ono
AAAI Conference on Artificial Intelligence (AAAI), 2018.
[PDF] [Slide] - Integral Design Method for Simple and Small Mars Lander System Using Membrane Aeroshell
Ryo Sakagami, Ryohei Takahashi, Akifumi Wachi, Yuki Koshiro, Hiroyuki Maezawa, Yasko Kasai, Shinichi Nakasuka
Acta Astronautica, 2018.
[PDF] - The Conceptual Design of a Novel Simple and Small-sized Mars lander
Ryohei Takahashi, Ryo Sakagami, Akifumi Wachi, Yasko Kasai, Shinichi Nakasuka
IEEE Aerospace Conference, 2018.
[PDF] - Mars Entry, Descent, and Landing by Small THz Spacecraft via Membrane Aeroshell
Akifumi Wachi, Ryohei Takahashi, Ryo Sakagami, Yuki Koshiro, Yasko Kasai, Shinichi Nakasuka
AIAA SPACE and Astronautics Forum and Exposition (AIAA SPACE), 2017.
[PDF] [Slide] - Hazard Avoidance Control Using Stochastic Optimization for Mars Safe Landing
Ryohei Takahashi, Akifumi Wachi, Ryu Funase, Shinichi Nakasuka
International Symposium on Space Technology and Sciences (ISTS), 2017. - Low-Thrust Trajectory Design to Improve Overall Mission Success Probability Incorporating Target Changes in Case of Engine Failures
Akifumi Wachi
International Symposium on Space Technology and Science (ISTS), 2017.
Best Student Paper Award (First Prize)
[PDF] - Fault-Tolerant Low-Thrust Trajectory Design with Backups for Multiple Targets
Akifumi Wachi, Naoya Ozaki, Shinichi Nakasuka
IFAC Symposium on Automatic Control in Aerospace (ACA), 2016.
Best Student Paper Award (First Prize)
[PDF]
Demo
- LOA: Logical Optimal Actions for Text-based Interaction Games
Daiki Kimura, Subhajit Chaudhury, Masaki Ono, Michiaki Tatsubori, Don Joven Agravante, Asim Munawar, Akifumi Wachi, Ryosuke Kohita, and Alexander Gray
Association for Computational Linguistics (ACL), Demo Paper, 2021.
[PDF]
Patents
- I have over 10 filed patents. See Google Patents
Professional Activities
Conference Program Committee Member
- ICML (2021 - 2024), NeurIPS (2021 - 2024), ICLR (2022 - 2025), AISTATS (2024 - 2025)
- AAAI (2020 - 2025), IJCAI (2020 - 2024)
- COLM (2024), ACL (2020 - 2023), EMNLP (2020)
Journal Reviewer
- ACM Transactions on Evolutionary Learning and Optimization (Associate Editor)
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- ACM Transactions on Evolutionary Learning and Optimization (Associate Editor)
- IEEE Transactions on Pattern Analysis and Machine Intelligence
Others
Awards
- Dean’s Award (Second Prize), from Faculty of Engineering, The University of Tokyo, 2018
- Best Student Paper Award (First Prize), Joint Conference: 31st ISTS, 26th ISSFD, and 8th NSAT, 2017
- Best Student Paper Award (First Prize), IFAC Symposium on Automatic Control in Aerospace (ACA), 2016, [link]
Fellowships
- Financial support to participate in Thirty-second AAAI conference on Artificial Intelligence by Society for Promotion of Space Science (SPSS), February, 2018
- Financial support to participate in International Astronautical Congress (IAC) by the Japan Society for Aeronautical and Space Sciences (JSASS), September, 2017
- Go Global Scholarships 2016 Short Study Abroad Scholarships: the University of Tokyo, 2016
- Fellowship of the Leading Graduates Schools Program: Global Leader Program for Social Design and Management by the Ministry of Education, Culture, Sports, Science and Technology in Japan, 2016
Invited Talk
- Safe Reinforcement Learning,
Guest Speaker Session at Cohere for AI,
Online, February, 2024
[Cohere Website] [Presentation Slide]