Empowerment (artificial intelligence) explained

Empowerment in the field of artificial intelligence formalises and quantifies (via information theory) the potential an agent perceives that it has to influence its environment. An agent which follows an empowerment maximising policy, acts to maximise future options (typically up to some limited horizon). Empowerment can be used as a (pseudo) utility function that depends only on information gathered from the local environment to guide action, rather than seeking an externally imposed goal, thus is a form of intrinsic motivation.

The empowerment formalism depends on a probabilistic model commonly used in artificial intelligence. An autonomous agent operates in the world by taking in sensory information and acting to change its state, or that of the environment, in a cycle of perceiving and acting known as the perception-action loop. Agent state and actions are modelled by random variables (

S:s\inl{S},A:a\inl{A}

) and time (

). The choice of action depends on the current state, and the future state depends on the choice of action, thus the perception-action loop unrolled in time forms a causal bayesian network.

Definition

Empowerment (

ak{E}

) is defined as the channel capacity (

) of the actuation channel of the agent, and is formalised as the maximal possible information flow between the actions of the agent and the effect of those actions some time later. Empowerment can be thought of as the future potential of the agent to affect its environment, as measured by its sensors.

$\mathfrak(A^n_t \longrightarrow S_) = \max_ I(A_t,...,A_;S_)$

The unit of empowerment depends on the logarithm base. Base 2 is commonly used in which case the unit is bits.

Contextual Empowerment

In general the choice of action (action distribution) that maximises empowerment varies from state to state. Knowing the empowerment of an agent in a specific state is useful, for example to construct an empowerment maximising policy. State-specific empowerment can be found using the more general formalism for 'contextual empowerment'.

is a random variable describing the context (e.g. state).

$reinforcement learning agents playing video games, and in the control of underwater vehicles.$