Frank Keller receives European Research Council Starting Grant

Frank Keller, Member of the Institute for Communicating and Collaborative Systems, will be awarded a Starting Grant by the European Research Council (ERC).

The ERC is a new pan-European funding body set up to support investigator-driven frontier research. Its main aim is to stimulate scientific excellence by supporting and encouraging the best scientists to be adventurous and take risks in their research. The ERC's prestigious Starting Grant Scheme targets young researchers who have the potential of becoming independent research leaders. It supports them in establishing or consolidating an independent research team through financial support of up to 2 million euros over five years. According to a press release by ERC, the selection process for the first round of Starting Grants was very competitive, with only 300 out of 9000 applications being successful.

Frank Keller's Starting Grant will investigate the synchronous processing of linguistic and visual information. It starts from the observation that humans rarely process language in isolation. Linguistic input often occurs synchronously with visual input, e.g., in everyday activities such as attending a lecture or following directions on a map. The visual context constrains the interpretation of the linguistic input, and vice versa, making processing more efficient and less ambiguous. The grant will investigate key features of synchronous processing by tracking participants' eye movements when they view a naturalistic scene and listen to a speech stimulus at the same time. The aim is to understand synchronous processing better by studying the interaction of saliency and ambiguity, and the role of incrementality, object context, and task factors. The experimental results will feed into a series of computational models that predict the eye-movement patterns that humans exhibit when they view a scene and listen to speech at the same time. These models will incrementally construct aligned linguistic and visual representations, and will be evaluated against eye-tracking data.