Publications

2024

1. EMNLP 2024

   NegotiationToM: A Benchmark for Stress-testing Machine Theory of Mind on Negotiation Surrounding

   Chunkit Chan, Cheng Jiayang, Yauwai Yim, Zheye Deng, Wei Fan, Haoran Li, Xin Liu, Hongming Zhang, Weiqi Wang, and Yangqiu Song

   arXiv preprint arXiv:2404.13627, 2024
2. Arxiv

   Evaluating and Enhancing LLMs Agent based on Theory of Mind in Guandan: A Multi-Player Cooperative Game under Imperfect Information

   Yauwai Yim, Chunkit Chan, Tianyu Shi, Zheye Deng, Wei Fan, Tianshi Zheng, and Yangqiu Song

   CoRR, 2024
3. Arxiv

   Constrained Reasoning Chains for Enhancing Theory-of-Mind in Large Language Models

   Zizheng Lin, Chunkit Chan, Yangqiu Song, and Xin Liu

   arXiv preprint arXiv:2409.13490, 2024
4. EMNLP 2024

   Text-Tuple-Table: Towards Information Integration in Text-to-Table Generation via Global Tuple Extraction

   Zheye Deng, Chunkit Chan, Weiqi Wang, Yuxi Sun, Wei Fan, Tianshi Zheng, Yauwai Yim, and Yangqiu Song

   arXiv preprint arXiv:2404.14215, 2024
5. ACL 2024

   CANDLE: Iterative Conceptualization and Instantiation Distillation from Large Language Models for Commonsense Reasoning

   Weiqi Wang, Tianqing Fang, Chunyang Li, Haochen Shi, Wenxuan Ding, Baixuan Xu, Zhaowei Wang, Jiaxin Bai, Xin Liu, Cheng Jiayang, Chunkit Chan, and Yangqiu Song

   In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2024, Bangkok, Thailand, August 11-16, 2024, 2024

   Bib

   @inproceedings{DBLP:conf/acl/0001FLS0XWBLJCS24,
     author = {Wang, Weiqi and Fang, Tianqing and Li, Chunyang and Shi, Haochen and Ding, Wenxuan and Xu, Baixuan and Wang, Zhaowei and Bai, Jiaxin and Liu, Xin and Jiayang, Cheng and Chan, Chunkit and Song, Yangqiu},
     editor = {Ku, Lun{-}Wei and Martins, Andre and Srikumar, Vivek},
     title = {{CANDLE:} Iterative Conceptualization and Instantiation Distillation
                       from Large Language Models for Commonsense Reasoning},
     booktitle = {Proceedings of the 62nd Annual Meeting of the Association for Computational
                       Linguistics (Volume 1: Long Papers), {ACL} 2024, Bangkok, Thailand,
                       August 11-16, 2024},
     pages = {2351--2374},
     publisher = {Association for Computational Linguistics},
     year = {2024},
     url = {https://doi.org/10.18653/v1/2024.acl-long.128},
     doi = {10.18653/V1/2024.ACL-LONG.128},
     timestamp = {Tue, 24 Sep 2024 10:55:43 +0200},
     biburl = {https://dblp.org/rec/conf/acl/0001FLS0XWBLJCS24.bib},
     bibsource = {dblp computer science bibliography, https://dblp.org}
   }

6. COLING 2024

   EventGround: Narrative Reasoning by Grounding to Eventuality-centric Knowledge Graphs

   Cheng Jiayang, Lin Qiu, Chunkit Chan, Xin Liu, Yangqiu Song, and Zheng Zhang

   In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC/COLING 2024, 20-25 May, 2024, Torino, Italy, 2024

   Bib

   @inproceedings{DBLP:conf/coling/JiayangQC0SZ24,
     author = {Jiayang, Cheng and Qiu, Lin and Chan, Chunkit and Liu, Xin and Song, Yangqiu and Zhang, Zheng},
     editor = {Calzolari, Nicoletta and Kan, Min{-}Yen and Hoste, V{\'{e}}ronique and Lenci, Alessandro and Sakti, Sakriani and Xue, Nianwen},
     title = {EventGround: Narrative Reasoning by Grounding to Eventuality-centric
                       Knowledge Graphs},
     booktitle = {Proceedings of the 2024 Joint International Conference on Computational
                       Linguistics, Language Resources and Evaluation, {LREC/COLING} 2024,
                       20-25 May, 2024, Torino, Italy},
     pages = {6622--6642},
     publisher = {{ELRA} and {ICCL}},
     year = {2024},
     url = {https://aclanthology.org/2024.lrec-main.587},
     timestamp = {Thu, 23 May 2024 16:47:05 +0200},
     biburl = {https://dblp.org/rec/conf/coling/JiayangQC0SZ24.bib},
     bibsource = {dblp computer science bibliography, https://dblp.org}
   }

2023

1. EACL 2024

   ChatGPT Evaluation on Sentence Level Relations: A Focus on Temporal, Causal, and Discourse Relations

   Chunkit Chan, Jiayang Cheng, Weiqi Wang, Yuxin Jiang, Tianqing Fang, Xin Liu, and Yangqiu Song

   Findings-EACL2024, 2023
2. ACL 2023

   DiscoPrompt: Path Prediction Prompt Tuning for Implicit Discourse Relation Recognition

   Chunkit Chan, Xin Liu, Jiayang Cheng, Zihan Li, Yangqiu Song, Ginny Y. Wong, and Simon See

   In Findings of the Association for Computational Linguistics: ACL 2023, Toronto, Canada, July 9-14, 2023, 2023

   Bib

   @inproceedings{DBLP:conf/acl/ChanLCLSWS23,
     author = {Chan, Chunkit and Liu, Xin and Cheng, Jiayang and Li, Zihan and Song, Yangqiu and Wong, Ginny Y. and See, Simon},
     editor = {Rogers, Anna and Boyd{-}Graber, Jordan L. and Okazaki, Naoaki},
     title = {DiscoPrompt: Path Prediction Prompt Tuning for Implicit Discourse
                       Relation Recognition},
     booktitle = {Findings of the Association for Computational Linguistics: {ACL} 2023,
                       Toronto, Canada, July 9-14, 2023},
     pages = {35--57},
     publisher = {Association for Computational Linguistics},
     year = {2023},
     url = {https://doi.org/10.18653/v1/2023.findings-acl.4},
     doi = {10.18653/v1/2023.findings-acl.4},
     timestamp = {Thu, 10 Aug 2023 12:35:51 +0200},
     biburl = {https://dblp.org/rec/conf/acl/ChanLCLSWS23.bib},
     bibsource = {dblp computer science bibliography, https://dblp.org}
   }

3. AACL 2023

   Self-Consistent Narrative Prompts on Abductive Natural Language Inference

   Chunkit Chan, Xin Liu, Tsz Ho Chan, Jiayang Cheng, Yangqiu Song, Ginny Wong, and Simon See

   In Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), Nov 2023

   Bib

   @inproceedings{chan-etal-2023-self,
     title = {Self-Consistent Narrative Prompts on Abductive Natural Language Inference},
     author = {Chan, Chunkit and Liu, Xin and Chan, Tsz Ho and Cheng, Jiayang and Song, Yangqiu and Wong, Ginny and See, Simon},
     editor = {Park, Jong C. and Arase, Yuki and Hu, Baotian and Lu, Wei and Wijaya, Derry and Purwarianti, Ayu and Krisnadhi, Adila Alfa},
     booktitle = {Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)},
     month = nov,
     year = {2023},
     address = {Nusa Dua, Bali},
     publisher = {Association for Computational Linguistics},
     url = {https://aclanthology.org/2023.ijcnlp-main.67},
     pages = {1040--1057}
   }

4. EMNLP 2023

   Lion: Adversarial Distillation of Proprietary Large Language Models

   Yuxin Jiang, Chunkit Chan, Mingyang Chen, and Wei Wang

   In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Dec 2023

   Abs Bib

   The practice of transferring knowledge from a sophisticated, proprietary large language model (LLM) to a compact, open-source LLM has garnered considerable attention. Previous works have focused on a unidirectional knowledge distillation way by aligning the responses of the student model with those of the teacher model to a set of instructions. Nevertheless, they overlooked the possibility of incorporating any “feedback”–identifying challenging instructions where the student model’s performance falls short–to boost the student model’s proficiency iteratively. To this end, we propose a novel adversarial distillation framework for a more efficient knowledge transfer. Leveraging the versatile role adaptability of LLMs, we prompt the teacher model to identify “hard” instructions and generate new “hard” instructions for the student model, creating a three-stage adversarial loop of imitation, discrimination, and generation. By applying this adversarial framework, we successfully transfer knowledge from ChatGPT to a student model (named Lion), using a mere 70k training data. Our results show that Lion-13B not only achieves comparable open-ended generation capabilities to ChatGPT but surpasses conventional state-of-the-art (SOTA) instruction-tuned models like Vicuna-13B by 55.4% in challenging zero-shot reasoning benchmarks such as BIG-Bench Hard (BBH) and 16.7% on AGIEval.

   @inproceedings{jiang-etal-2023-lion,
     title = {Lion: Adversarial Distillation of Proprietary Large Language Models},
     author = {Jiang, Yuxin and Chan, Chunkit and Chen, Mingyang and Wang, Wei},
     editor = {Bouamor, Houda and Pino, Juan and Bali, Kalika},
     booktitle = {Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing},
     month = dec,
     year = {2023},
     address = {Singapore},
     publisher = {Association for Computational Linguistics},
     url = {https://aclanthology.org/2023.emnlp-main.189},
     doi = {10.18653/v1/2023.emnlp-main.189},
     pages = {3134--3154}
   }

5. EMNLP 2023

   StoryAnalogy: Deriving Story-level Analogies from Large Language Models to Unlock Analogical Understanding

   Cheng Jiayang, Lin Qiu, Tsz Chan, Tianqing Fang, Weiqi Wang, Chunkit Chan, Dongyu Ru, Qipeng Guo, Hongming Zhang, Yangqiu Song, Yue Zhang, and Zheng Zhang

   In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Dec 2023

   Abs Bib

   Analogy-making between narratives is crucial for human reasoning. In this paper, we evaluate the ability to identify and generate analogies by constructing a first-of-its-kind large-scale story-level analogy corpus, StoryAnalogy, which contains 24K story pairs from diverse domains with human annotations on two similarities from the extended Structure-Mapping Theory. We design a set of tests on StoryAnalogy, presenting the first evaluation of story-level analogy identification and generation. Interestingly, we find that the analogy identification tasks are incredibly difficult not only for sentence embedding models but also for the recent large language models (LLMs) such as ChatGPT and LLaMa. ChatGPT, for example, only achieved around 30% accuracy in multiple-choice questions (compared to over 85% accuracy for humans). Furthermore, we observe that the data in StoryAnalogy can improve the quality of analogy generation in LLMs, where a fine-tuned FlanT5-xxl model achieves comparable performance to zero-shot ChatGPT.

   @inproceedings{jiayang-etal-2023-storyanalogy,
     title = {{S}tory{A}nalogy: Deriving Story-level Analogies from Large Language Models to Unlock Analogical Understanding},
     author = {Jiayang, Cheng and Qiu, Lin and Chan, Tsz and Fang, Tianqing and Wang, Weiqi and Chan, Chunkit and Ru, Dongyu and Guo, Qipeng and Zhang, Hongming and Song, Yangqiu and Zhang, Yue and Zhang, Zheng},
     editor = {Bouamor, Houda and Pino, Juan and Bali, Kalika},
     booktitle = {Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing},
     month = dec,
     year = {2023},
     address = {Singapore},
     publisher = {Association for Computational Linguistics},
     url = {https://aclanthology.org/2023.emnlp-main.706},
     doi = {10.18653/v1/2023.emnlp-main.706},
     pages = {11518--11537}
   }