和地 瞭良 / Akifumi Wachi

Enlgish version is here.

強化学習（Reinforcement Learning, RL）の理論と応用全般に関して興味をもって研究しています。特に、以下の三つの研究テーマに最近は興味をもっています。

  • Safe RL: 安全性に関する理論的保証のある強化学習アルゴリズムの設計
  • 基盤モデルのための RL:言語モデルのアライメントなど、強化学習と自然言語処理（NLP）の融合
  • RLによる敵対的検証: 強化学習を用いてAIシステムの安全性を検証する方法の開発

論文一覧・特許などの研究業績につきましては、英語ページ を御覧ください。

招待講演や共同研究、技術顧問等の依頼はメールにてご連絡ください。

経歴

  • 2025 May - present: Senior Chief Research Scientist, LINEヤフー株式会社
  • 2025 Jan - present: 技術顧問, 株式会社 Spakona
  • 2023 Oct - 2025 Apr: Chief Research Scientist, LINEヤフー株式会社
  • 2022 Sep - 2023 Sep: Senior Research Scientist, LINE株式会社
  • 2018 Apr - 2022 Aug: Research Scientist, IBM東京基礎研究所

  • 2021: 博士（情報工学）筑波大学（指導教員：佐久間淳 教授）
  • 2018: 修士（航空宇宙工学）東京大学（指導教員：中須賀真一 教授）
  • 2016: 学士（航空宇宙工学）東京大学（指導教員：中須賀真一 教授）

主要業績

  • How Neural Reward Models Learn Features for Policy Optimization: A Single-Index Analysis Rei Higuchi, Ryotaro Kawata, Akifumi Wachi, Shokichi Takakura, Kohei Miyaguchi, Taiji Suzuki Neural Information Processing Systems (NeurIPS), 2026. [PDF forthcoming]
  • Reward-Estimated Hypergradient for Bilevel Reinforcement Learning with Black-Box Follower Shigeki Kusaka, Mikoto Kudo, Takumi Tanabe, Akifumi Wachi, Youhei Akimoto Neural Information Processing Systems (NeurIPS), 2026. [PDF forthcoming]
  • Inference-Aware Meta-Alignment of LLMs via Non-Linear GRPO Shokichi Takakura, Akifumi Wachi, Rei Higuchi, Kohei Miyaguchi, Taiji Suzuki International Conference on Machine Learning (ICML), 2026. [ICML] [arXiv]
  • Cost-Minimized Label-Flipping Poisoning Attack to LLM Alignment Shigeki Kusaka, Keita Saito, Mikoto Kudo, Takumi Tanabe, Akifumi Wachi, Youhei Akimoto AAAI Conference on Artificial Intelligence (AAAI), 2026. [AAAI] [arXiv]
  • A Provable Approach for End-to-End Safe Reinforcement Learning Akifumi Wachi, Kohei Miyaguchi, Takumi Tanabe, Rei Sato, Youhei Akimoto Neural Information Processing Systems (NeurIPS), 2025. [NeurIPS] [arXiv]
  • Offline Guarded Safe Reinforcement Learning for Medical Treatment Optimization Strategies Runze Yan, Xun Shen, Akifumi Wachi, Sebastien Gros, Anni Zhao, Xiao Hu Neural Information Processing Systems (NeurIPS), 2025. (Spotlight) [NeurIPS] [arXiv]
  • Stepwise Alignment for Constrained Language Model Policy Optimizations Akifumi Wachi, Thien Q. Tran, Rei Sato, Takumi Tanabe, Youhei Akimoto Neural Information Processing Systems (NeurIPS), 2024. [arXiv] [Poster] [GitHub] [Hugging Face (SACPO)] [Hugging Face (P-SACPO)]
  • Flipping-based Policy for Chance-Constrained Markov Decision Processes Xun Shen, Shuo Jiang, Akifumi Wachi, Kazumune Hashimoto, Sebastien Gros Neural Information Processing Systems (NeurIPS), 2024. [arXiv]
  • A Survey of Constraint Formulations in Safe Reinforcement Learning Akifumi Wachi, Xun Shen, Yanan Sui International Joint Conference on Artificial Intelligence (IJCAI), 2024. [arXiv] [Poster] [Slide]
  • Long-term Safe Reinforcement Learning with Binary Feedback Akifumi Wachi, Wataru Hashimoto, Kazumune Hashimoto AAAI Conference on Artificial Intelligence (AAAI), 2024. [PDF] [Poster] [Slide]
  • Safe Exploration in Reinforcement Learning: A Generalized Formulation and Algorithms Akifumi Wachi, Wataru Hashimoto, Xun Shen, Kazumune Hashimoto Neural Information Processing Systems (NeurIPS), 2023. [PDF] [arXiv] [Poster] [Video]
  • Safe Policy Optimization with Local Generalized Linear Function Approximations Akifumi Wachi, Yunyue Wei, Yanan Sui Neural Information Processing Systems (NeurIPS), 2021. [PDF] [OpenReview] [arXiv] [Poster]
  • Neuro-Symbolic Reinforcement Learning with First-Order Logic Daiki Kimura, Masaki Ono, Subhajit Chaudhury, Ryosuke Kohita, Akifumi Wachi, Don Joven Agravante, Michiaki Tatsubori, Asim Munawar, Alexander Gray Empirical Methods in Natural Language Processing (EMNLP), Short paper, 2021. [PDF]
  • Language-based General Action Template for Reinforcement Learning Agents Ryosuke Kohita, Akifumi Wachi, Daiki Kimura, Subhajit Chaudhury, Michiaki Tatsubori, Asim Munawar Association for Computational Linguistics (ACL), Findings, 2021. [PDF]
  • Q-learning with Language Model for Edit-based Unsupervised Summarization Ryosuke Kohita, Akifumi Wachi, Yang Zhao, Ryuki Tachibana Empirical Methods in Natural Language Processing (EMNLP), 2020. [PDF] [arXiv]
  • Safe Reinforcement Learning in Constrained Markov Decision Processes Akifumi Wachi, Yanan Sui International Conference on Machine Learning (ICML), 2020. [PDF] [arXiv] [Slide] [Video]
  • Failure-Scenario Maker for Rule-Based Agent using Multi-agent Adversarial Reinforcement Learning and its Application to Autonomous Driving Akifumi Wachi International Joint Conference on Artificial Intelligence (IJCAI), 2019. [PDF] [arXiv] [Slide] [Poster] [Simulation]
  • Safe Exploration and Optimization of Constrained MDPs using Gaussian Processes Akifumi Wachi, Yanan Sui, Yisong Yue, Masahiro Ono AAAI Conference on Artificial Intelligence (AAAI), 2018. [PDF] [Slide]

論文一覧を見る（英語）

書籍

コンペティション

コラム

その他

受賞

  • Top Reviewer, NeurIPS, 2024 [link]
  • 工学系研究科長賞（副代表）, 東京大学, 2018
  • 最優秀学生論文賞, Joint Conference: 31st ISTS, 26th ISSFD, and 8th NSAT, 2017
  • 最優秀学生論文賞, IFAC Symposium on Automatic Control in Aerospace (ACA), 2016 [link]

講演

  • 第28回情報論的学習理論ワークショップ（IBIS2025）チュートリアル「深層基盤モデルのための強化学習：驚きから理論に基づく納得へ」
    [webpage] [slide]
  • 人工知能学会主催 第96回人工知能セミナー (2025.2.7) 「AIトレンド・トップカンファレンス報告会（NeurIPS2024）：世界最先端のAI研究開発動向が1日でわかる！」
    [webpage] [slide]
  • NLPコロキウム "Stepwise Alignment for Constrained Language Model Policy Optimization"
    [webpage] [YouTube] [slide]
  • 大阪大学 数理・データ科学教育研究センター AI・データ利活用研究会 「安全性を考慮した強化学習：基礎から応用まで」
    [webpage]
  • 鉄鋼協会 計測・制御・システム工学部会シンポジウム 「解釈性・信頼性の高いAI開発に向けたシステム技術」
    [webpage]
  • 人工知能学会主催 第90回人工知能セミナー (2024.3.26) 「AIトレンド・トップカンファレンス報告会（NeurIPS2023）：世界最先端のAI研究開発動向が1日でわかる！」
    [webpage] [YouTube（学会概要と参加報告）] [YouTube（受賞論文「DPO」徹底解説）] [slide]
  • Safe Reinforcement Learning, Cohere Guest Speaker Session, February, 2024
    [Cohere website] [slide]
-->