Haozhe Ji 计昊哲

I am Haozhe Ji, a penultimate-year Ph.D. student from CoAI Group in the Dept. of Computer Science and Technology, Tsinghua University, advised by Prof. Minlie Huang. Prior to that, I received my B.Eng. degree from the Dept. of Electronic Engineering, Tsinghua University. Please find my CV here [English].

My current research interests are focused on the theoretical foundations and scalable algorithms for generative language models, aiming to develop verifiable, consistent and robust AI systems capable of generating natural language indistinguishable from that of humans.

Specifically, my research is theoretically motivated to advance language models beyond the inherent limitations of Auto-Regressive (AR) modeling and Maximum Likelihood Estimation (MLE) objective by providing practical and scalable solutions.

• To overcome the capacity limitation of AR models, my research delves into a broader spectrum of expressive model families, including semi-parametric models 8 Language generation with multi-hop reasoning on commonsense knowledge graph
  Haozhe Ji, Pei Ke, Shaohan Huang, Furu Wei, Xiaoyan Zhu, Minlie Huang.
  EMNLP 2020.
  , memory-augmented models 5 LaMemo: Language modeling with look-ahead memory
  Haozhe Ji, Rongsheng Zhang, Zhenyu Yang, Zhipeng Hu, Minlie Huang.
  NAACL 2022.
  , latent variable models 6 DiscoDVT: Generating Long Text with Discourse-Aware Discrete Variational Transformer
  Haozhe Ji, Minlie Huang.
  EMNLP 2021.
  and energy-based models 2 Language Model Decoding as Direct Metrics Optimization
  Haozhe Ji, Pei Ke, Hongning Wang, Minlie Huang.
  ICLR 2024.
  .

• To tackle the biases inherent in the conventional MLE objective, my research introduces theoretically grounded and practically accessible training objectives 1 Towards Efficient and Exact Optimization of Language Model Alignment
  Haozhe Ji, Cheng Lu, Yilin Niu, Pei Ke, Hongning Wang, Jun Zhu, Jie Tang, Minlie Huang.
  arXiv:2402.00856. 2024.
  3 Tailoring Language Generation Models under Total Variation Distance
  Haozhe Ji, Pei Ke, Zhipeng Hu, Rongsheng Zhang, Minlie Huang.
  ICLR 2023.
  and decoding frameworks 2 Language Model Decoding as Direct Metrics Optimization
  Haozhe Ji, Pei Ke, Hongning Wang, Minlie Huang.
  ICLR 2024.
  , aiming to achieve better alignment with human language.

Publications

* indicates equal contribution.

1. Haozhe Ji, Cheng Lu, Yilin Niu, Pei Ke, Hongning Wang, Jun Zhu, Jie Tang, Minlie Huang.
   Towards Efficient and Exact Optimization of Language Model Alignment.
   arXiv:2402.00856.
   [paper] [repo]

2. Haozhe Ji, Pei Ke, Hongning Wang, Minlie Huang.
   Language Model Decoding as Direct Metrics Optimization.
   The Twelfth International Conference on Learning Representations, ICLR 2024.
   [paper]

3. Haozhe Ji, Pei Ke, Zhipeng Hu, Rongsheng Zhang, Minlie Huang.
   Tailoring Language Generation Models under Total Variation Distance.
   The Eleventh International Conference on Learning Representations, ICLR 2023.
   (Notable top 5%)
   [paper] [repo]

4. Pei Ke, Haozhe Ji, Zhenyu Yang, Yi Huang, Junlan Feng, Xiaoyan Zhu, Minlie Huang.
   Curriculum-Based Self-Training Makes Better Few-Shot Learners for Data-to-Text Generation
   Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI 2022.
   [paper]

5. Haozhe Ji, Rongsheng Zhang, Zhenyu Yang, Zhipeng Hu, Minlie Huang.
   LaMemo: Language modeling with look-ahead memory
   Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics, NAACL 2022. (Oral)
   [paper] [repo]

6. Haozhe Ji, Minlie Huang.
   DiscoDVT: Generating Long Text with Discourse-Aware Discrete Variational Transformer
   Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021. (Oral)
   [paper] [repo]

7. Pei Ke, Haozhe Ji, Yu Ran, Xin Cui, Liwei Wang, Linfeng Song, Xiaoyan Zhu, Minlie Huang.
   Jointgt: Graph-text joint representation learning for text generation from knowledge graphs
   Findings of the Association for Computational Linguistics, Findings of ACL 2021.
   [paper] [repo]

8. Zhengyan Zhang, Xu Han, Hao Zhou, Pei Ke, Yuxian Gu, Deming Ye, Yujia Qin, Yusheng Su, Haozhe Ji, Jian Guan, Fanchao Qi, Xiaozhi Wang, Yanan Zheng, Guoyang Zeng, Huanqi Cao, Shengqi Chen, Daixuan Li, Zhenbo Sun, Zhiyuan Liu, Minlie Huang, Wentao Han, Jie Tang, Juanzi Li, Xiaoyan Zhu, Maosong Sun.
   CPM: A large-scale generative Chinese pre-trained language model
   AI Open.
   [paper]

9. Haozhe Ji, Pei Ke, Shaohan Huang, Furu Wei, Xiaoyan Zhu, Minlie Huang.
   Language generation with multi-hop reasoning on commonsense knowledge graph
   Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020. (Oral)
   [paper] [repo]

10. Pei Ke*, Haozhe Ji*, Siyang Liu, Xiaoyan Zhu, Minlie Huang.
    Sentilare: Linguistic knowledge enhanced language representation for sentiment analysis
    Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020.
    [paper] [repo]

11. Haozhe Ji, Pei Ke, Shaohan Huang, Furu Wei, Minlie Huang.
    Generating commonsense explanation by extracting bridge concepts from reasoning paths
    Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics, AACL 2020.
    [paper]

12. Yankai Lin, Haozhe Ji, Zhiyuan Liu, Maosong Sun.
    Denoising Distantly Supervised Open-Domain Question Answering
    Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018.
    [paper] [repo]

Honors & Awards

• Tang Junyuan (唐君远) Scholarship, Tsinghua University, 2023
• Sohu Scholarship, Tsinghua University, 2022
• Yang Huiyan (杨惠妍) Scholarship, Tsinghua University, 2021
• Comprehensive Merit Scholarship, Tsinghua University, 2017/2019
• Gold Medal, 32nd Chinese Physics Olympiads (CPhO), 2015
• Distinguished Honor Roll (Top 1%), American Mathematics Contest 12A (AMC 12A), 2015

Education

• September, 2020 - present. Ph.D. student in CoAI Group, Dept. of Computer Science and Technology, Tsinghua University. Advisor: Prof. Minlie Huang
• September, 2016 - July, 2020. B.Eng. in Electronic Engineering, Tsinghua University.

Services

Reviewer/Program Committee: ACL, EMNLP, NAACL, ARR

Teaching

I was the Head TA of the undergraduate course Artificial Neural Network, instructed by Minlie Huang (2021 Fall, 2022 Fall, 2023 Fall).

Personal

I am a cellist in the Tsinghua University Symphony Orchestra (TUSO). Explore our 30th-anniversary concert, featuring a performance of Symphony No. 8 by Antonín Dvořák.