Biblio

While language learning in infants and toddlers progresses somewhat seamlessly, in artificial systems the grounding of language in knowledge structures that are learned from sensorimotor experiences remains a hard challenge. Here we introduce LEARNA, which learns event-characterizing abstractions to resolve natural language ambiguity. LEARNA develops knowledge structures from simulated sensorimotor experiences. Given a possibly ambiguous descriptive utterance, the learned knowledge structures enable LEARNA to infer environmental scenes, and events unfolding within, which essentially constitute plausible imaginations of the utterance’s content. Similar event-predictive structures may help in developing artificial systems that can generate and comprehend descriptions of scenes and events.