ESR5: Multimodal Semantic Grounding and Model Transparency

Open Early Stage Researcher/PhD Position at Università ta’ Malta (UOM), Malta, for the NL4XAI project

Reference number: NL4XAI- ESR5

PhD research topic: Multimodal Semantic Grounding and Model Transparency

Objectives:

  • To develop methods for evaluating the sensitivity of neural generation models to their input, extending current techniques for sensitivity analysis;
  • to develop new multimodal architectures based on an in-depth exploration of the optimal  visual features for NLG tasks such as the generation of descriptive or inferred text from images or videos. While several architectures have been proposed at the Vision-Language interface, it is often observed that generation from visual input suffers from a lack of sensitivity to the input, which may be due to problems in the multimodal representations learned by the architecture, and/or issues of redundancy and preedictability in the trainning data;
  • to design training setups that carefully control for bias and predictability in training data and in the  output.

Expected Results:

Our main challenge will be to study deep multimodal architectures for generating language from visual data, looking ‘under the hood’ for evidence that the choices made by a generator are indeed grounded in the input, that is, motivated by (and explainable on the basis of) the sensory information. Thus, the expected outcomes are:

  1. In-depth theoretical and practical understanding of neural network methods and the underlying theoretical basis of multimodal neural architectures.
  2. To develop sensitivity analysis tools for NLG in general, and especially for NLG from visual input, which gobeyond the state of the art.
  3. Implementation of diverse multimodal neural architectures.
  4. New dataset for vision-text tasks (e.g., image captioning, grounded textual entailment generation) with an emphasis on high diversity and sensitivity of textual samples to visual inputs, and with an attention to design issues to avoid redundancy between modalities.
  5. Design and execution of evaluation experiments, both automatic and involving human participants, to evaluate sensitivity and the groundedness of multimodal models.

The above outcomes, especially (5), tie in with NL4XAI- ESR1 to a significant degree and candidates working on NL4XAI- ESR1 and NL4XAI- ESR5 would be expected to collaborate extensively, especially during the period of secondments of NL4XAI- ESR1 to UOM. There will be close collaboration with NL4XAI- ESR4 and NL4XAI- ESR6 in the secondments by NL4XAI- ESR5.

Host institution: Università ta’ Malta- UOM (Malta)

PhD Enrolment: Università ta’ Malta- UOM (Malta)

The PhD candidate will form part of the Institute of Linguistics and Language Technology (iLLT) at the University of Malta. The iLLT is a small, multidisciplinary group of researchers working on all area of linguistics and computational linguistics, including machine-learning approaches to NLP. iLLT researchers work within an extensive national and international network, with strong links to other departments and faculties in the University, including the Department of Artificial Intelligence and the Faculty of Engineering. iLLT members are highly research active and have a significant record of success in obtaining national and international research funding. Albert Gatt, the principal supervisor of this ESR, is regularly involved in high-level research events in the NLP community. He regularly serves on the programme committees of top conferences, including the ACL and EMNLP meetings, and has organised or co-organised several events in these venues, including the first series of shared tasks in NLG. He is a former elected member of the ACL Special Interest Group on Natural Language Generation (SIGGEN). He is currently involved in several projects spanning different areas of NLP, including the Vision-Language interface and Speech Recognition.

The University of Malta traces its origins to the founding of the Collegium Melitense by the Jesuits in 1592. The College was raised to University status by Grand Master Manoel Pinto de Fonseca in 1769. Situated at Msida, it is the highest teaching institution of the State by which it is mainly financed and is open to all those who have the requisite qualifications. The University has over 11,000 students, including a substantial international student body with individuals from over 90 different countries.

Further information:

Secondments: The ESR will enjoy three secondments of 3 months each at the premises of two project’s members as detailed in the following table.

Supervisors:

  • Main Supervisor: Dr. Albert Gatt, Institute of Linguistics and Language Technology – Università ta’ Malta (UOM), albert.gatt@um.edu.mt 
  • PhD Co-Supervisor: Prof. Kees van Deemter, Department of Information and Computing Sciences- Universiteit Utrecht (UU)

Inter-sectoral Secondment Supervisor:

  • Dr. Lina María Rojas Barahona, Learning and Natural Dialogue Teams – Orange  
  • Dr. Johannes Heinecke, Learning and Natural Dialogue Teams – Orange

General requirements:

  • Mobility: At the time of recruitment, the researcher must not have resided or carried out his/her main activity (work, studies, etc.) in Malta for more than 12 months in the 3 years prior to recruitment date. Time spent as part of a procedure for obtaining refugee status under the Geneva Convention is not taken into account.
  • Career: When starting their contract (expected in April 2020), selected researchers should be within the first four years of his/her research careers and not have been awarded a doctoral degree prior to the application.
  • The candidate must be working exclusively for the action.

Specific requirements:

The ideal candidate will have a strong background in Natural Language Processing, Artificial Intelligence and/or Machine Learning.

  • Degree: Degree in Computer Science or Artificial Intelligence. Master degree or equivalent in Computational Linguistics, Artificial Intelligence, Computer Science, or a related discipline, providing access to PhD program.
  • Experience in Neural modelling, Natural Language Generation and/or natural language processing (NLP) using deep learning techniques
  • Programming skills: Python, Familiarity with numerical computing libraries and neural modelling libraries such as PyTorch, Tensorflow etc
  • Language: Excellent command of English, together with good academic writing and presentation skills. Please refer to the pages of the UoM Doctoral School for details of the English language requirements for enrolment at the University of Malta.

Desirable skills:

  • Background in behavioural methods, such as the design and execution of experiments
  • Ability to work independently and as part of a team.
  • Strong motivation to pursue a PhD degree.
  • Strong interest in interdisciplinary scientific work

Estimated starting date: 1st April 2020

Contract: Full-time contract

Duration: 36 months, including 3 secondments of 3 months each, at other consortium members’ premises (see the Secondments section above)

Salary: EUR 2,230 / month (on a 4-weekly basis, i.e. with 13 payments per annum). This amount will be increased with the corresponding mobility allowance, and the family allowance depending on the family status of the  recruited researcher .

Application Documents:

  • Detailed CV in Europass format (template available in the following link) in English, highlighting the merits that are established as evaluation criteria;
  • Scans of BSc and/or MSc transcripts, with certified translation in English (if the degree qualification is not in English or in the language of the hosting country); 
  • A motivation letter in English, highlighting the consistency between the candidate profile and the chosen ESR position/s for which she/he is applying and describing why you wishes to be an NL4XAI ESR to carry out a PhD;
  • Contact details or recommendation letters of two referees in English or in certified translation;
  • Scanned copy of valid identification document;
  • Proof of excellent command of English as per the UoM requirements (IELTS, TOEFL or Cambridge). Please refer to the UoM Doctoral School pages for further details.

In addition, you can add any other documents which you find relevant for the applications such as Master thesis, publications or project reports.

Evaluation criteria:

  • Academic background (up to 40 points)
  • Knowledge and specific achievements (up to 35 points)
  • Personal interview, only for candidates achieving a minimum of 55 points (up to 25 points)

Deadline: February 14, 2020, at 23h59 CET (UCT + 01:00)

Enquiries about research content must be sent to the main PhD supervisor via email (see contact details in Supervisors section).

Closed

Share