Biblio

While language learning in infants and toddlers progresses somewhat seamlessly, in artificial systems the grounding of language in knowledge structures that are learned from sensorimotor experiences remains a hard challenge. Here we introduce LEARNA, which learns event-characterizing abstractions to resolve natural language ambiguity. LEARNA develops knowledge structures from simulated sensorimotor experiences. Given a possibly ambiguous descriptive utterance, the learned knowledge structures enable LEARNA to infer environmental scenes, and events unfolding within, which essentially constitute plausible imaginations of the utterance’s content. Similar event-predictive structures may help in developing artificial systems that can generate and comprehend descriptions of scenes and events.

Humans are often able to generalize knowledge learned from a single exemplar. In this paper, we present a novel integration of mental simulation and analogical generalization algorithms into a cognitive robotic architecture that enables a similarly rudimentary generalization capability in robots. Specifically, we show how a robot can generate variations of a given scenario and then use the results of those new scenarios run in a physics simulator to generate generalized action scripts using analogical mappings. The generalized action scripts then allow the robot to perform the originally learned activity in a wider range of scenarios with different types of objects without the need for additional exploration or practice. In a proof-of-concept demonstration we show how the robot can generalize from a previously learned pick-and-place action performed with a single arm on an object with a handle to a pick-and-place action of a cylindrical object with no handle with two arms.