Perception, Action & Cognition through Learning of Object-Action Complexes

Strategic Objective(s) Addressed
The objective of PACO-PLUS is to develop a new principle and methods to endow an
artificial robotic system with the ability to give meaning to objects through perception,
manipulation, and interaction with people. The paradigm of Object-Action Complexes, which
is at the center of this proposal, will be investigated to address hard problems such as
learning, decision making, and memorisation for situated agents. An anthropomorphic robotic
platform will be designed to validate the proposed approaches. The ultimate goal is to provide
an artificial system with higher-level cognitive abilities than state of the art systems proposed
in AI and robotics. As such PACO-PLUS addresses the basic issues of 2.4.8 Cognitive
Systems call, which are to develop artificial systems that can interpret data arising from realworld
events and processes, acquire situated knowledge of their environment, act, and make
or suggest decisions and communicate with people on human terms.
Proposal Abstract
The successful design of a cognitive system must rely on a theoretical and measurable basis
which on the one hand applies to humans and on the other hand to an artificial system,
ultimately allowing for its construction. PACO-PLUS aims at the design of a cognitive robot
that is able to develop perceptual, behavioural and cognitive categories in a self-emergent and
measurable way and communicate and share these with humans and other artificial agents.
PACO-PLUS brings together a consortium of robotics researchers, engineers, computer vision
scientists, linguists, theoretical neuroscientists and psychologists, which is reflected in the
management organization. Central to the approach are three almost axiomatic assumptions
which are linked to each other and which are the building blocks of a new approach required
to create cognitive artificial agents: 1) Objects and Actions are inseparably intertwined; the
resulting, so-called Object-Action Complexes are the entities on which cognition develops. 2)
Cognition is based on self-emergent recurrent processes involving nested feedback loops
operating on and re-interpreting object-action complexes. This is done while actively closing
the perception-action cycle which involves the loop through the environment. 3) Unified
measure of success and progress exist through minimization of contingencies which an
artificial cognitive system experiences while interacting with the environment.
To demonstrate the feasibility of our approach we will build robot systems with advanced
cognitive capabilities that operate in real-world scenarios and that are able to learn to interact
and perform basic communication with humans: 1) A robot system to augment human action,
and 2) a robot system that is able to explore and manipulate a limited set of objects in an
unconstrained environment.