Talk: "Changing the Environment based on Empowerment as Intrinsic Motivation"

December 18th, 2015, 12:00-13:00
Christoph Salge (University of Hertfordshire)
at the Sociolinguistic Lab (Meister-Eckehart-Str. 11, 1st floor)

 

Abstract: One of the remarkable feats of intelligent life is that it restructures the world it lives in for its own benefit. Beavers build dams for shelter and to produce better hunting grounds, bees build hives for shelter and storage, humans have transformed the world in a multitude of ways. Intelligence is not only the ability to produce the right reaction to a randomly changing environment, but is also about actively influencing the change in the environment, leaving artefacts and structures that provide benefits in the future. In this talk, I want to explore if the framework of intrinsic motivation can help us understand and possibly reproduce this phenomenon. In particular, I will show results on how empowerment, as one example of intrinsic motivation, can produce structures in the environment of an agent. The basic idea behind intrinsic motivation is that an agent's behaviour, or decision making, is not guided by some form of externally specified reward, but rather by the optimization of some agent-internal measurement, which then gives rise to complex and rich interactions with the world. A quantitative formulation of an intrinsic motivation should ideally be computable from an agent's perspective, be applicable to different sensory-motoric configurations, and should reflect different agent embodiments. One classic example of intrinsic motivation is Schmidhuber's artificial curiosity, where an agent acts in a way so that it learns the most about the environment. Other examples include Homeokinesis, or the predictive information framework. Here we want to focus on empowerment, formally defined as the channel capacity between an agent's actuators, and its own sensors at a later time. This measures the potential causal flow in an agent's action-perception loop, and can be thought of as an abstract measure of how much control an agent has over the world it can perceive. The more meaningful options an agent has to influence the world, the higher is its empowerment. I will show some results on how an empowerment driven agent will manipulated a minecraft-like gridworld and how these manipulations reflect the agent's morphological configuration.