NL4XAI Workshop on NLG Evaluation: Challenges and Tools

Join us for an insightful workshop focused on the evaluation of Natural Language Generation (NLG) and the challenges and tools used in this domain. This event is part of the NL4XAI project and promises to provide valuable perspectives and discussions on effective and reliable methods for assessing the quality of generated natural language outputs.

The seminar will commence with an introductory talk delivered by Ehud Reiter, Professor of Computing Science at the University of Aberdeen (UNIABDN) and an expert in NLG technology. Ehud will set the stage by highlighting the challenges associated with NLG evaluation and emphasizing the need for robust assessment methods.

Following the introductory talk, the workshop will feature a series of short presentations, each focusing on a specific evaluation tool used in NLG research. These presentations aim to shed light on the strengths and advantages of each tool, providing participants with valuable insights into their potential applications. Each presentation will be limited to 10 minutes, followed by a 5-minute question-and-answer session.

The first presentation will be conducted by Adarsa Sivaprasad, PhD candidate at the University of Aberdeen (UNIABDN), who will discuss the Qualtrics platform for survey design.

Ilia Stepin, Mariña Canabal, Nikolay Babakov, and Javier González, all PhD candidates at CiTIUS-USC, will present on different platforms for survey design and human assessment. Ilia Stepin will introduce the SurveyGenerator Python Platform. Mariña Canabal and Adarsa Sivaprasad will jointly present on the Prolific crowdsourcing platform. Lastly, Nikolay Babakov and Javier González will discuss the Amazon Mechanical Turk (MTURK) crowdsourcing platform.

The workshop will conclude with an open discussion session, allowing all participants to share their insights, experiences, and opinions on the presented tools. This interactive session will provide an opportunity for attendees to engage in meaningful discussions, exchange ideas, and explore potential improvements or alternative approaches to NLG evaluation.

Following the NL4XAI workshop, participants will have the privilege of attending a lecture by Ingrid Zukerman, a renowned Professor in the Department of Data Science and Artificial Intelligence at Monash University. Her talk, titled “Influence of context on users’ views about explanations for decision-tree predictions,” is part of the CiTIUS-USC ‘Prestigious Women Researchers Lecture Series.’

Don’t miss this opportunity to enhance your understanding of NLG evaluation and engage with leading experts in the field. We look forward to seeing you there!