Robot Simulator of the Robotics Group for Self-Organization of Control: ClassicReinforce Class Reference

# ClassicReinforce Class Reference

class for robot controller using several feedforward networks (satelite) and one selforg controller More...

`#include <classicreinforce.h>`

Inherits AbstractController.

Collaboration diagram for ClassicReinforce:
[legend]

List of all members.

## Public Member Functions
| ClassicReinforce (const ClassicReinforceConf &conf=getDefaultConf())
virtual void | init (int sensornumber, int motornumber, RandGen *randGen=0)
| initialisation of the controller with the given sensor/ motornumber Must be called before use.
virtual | ~ClassicReinforce ()
virtual int | getSensorNumber () const
| returns the number of sensors the controller was initialised with or 0 if not initialised
virtual int | getMotorNumber () const
| returns the mumber of motors the controller was initialised with or 0 if not initialised
virtual void | step (const sensor *, int number_sensors, motor *, int number_motors)
| performs one step (includes learning).
virtual void | stepNoLearning (const sensor *, int number_sensors, motor *, int number_motors)
| performs one step without learning. Calulates motor commands from sensor inputs.
void | setManualControl (bool mControl, int action_=0)
| enables/disables manual control, action_ is the sat network number to be used if mControl is false, action is ignored
virtual void | notifyOnChange (const paramkey &key)
| Is called when a parameter was changes via setParam().
virtual bool | store (FILE *f) const
| stores the controller values to a given file.
virtual bool | restore (FILE *f)
| loads the controller values from a given file.
virtual std::list< iparamkey > | getInternalParamNames () const
| The list of the names of all internal parameters given by getInternalParams().
virtual std::list< iparamval > | getInternalParams () const
virtual std::list< ILayer > | getStructuralLayers () const
| Specifies which parameter vector forms a structural layer (in terms of a neural network) The ordering is important.
virtual std::list< IConnection > | getStructuralConnections () const
| Specifies which parameter matrix forms a connection between layers (in terms of a neural network) The orderning is not important.
## Static Public Member Functions
static ClassicReinforceConf | getDefaultConf ()
## Protected Member Functions
virtual int | getStateNumber ()=0
| returns number of state, to be overwritten
virtual int | calcState ()=0
| returns state, to be overwritten
virtual int | getActionNumber ()=0
| returns number of actions, to be overwritten
virtual matrix::Matrix | calcMotor (int action)=0
| returns action Matrix from discrete actions, to be overwritten
virtual double | calcReinforcement ()=0
| returns the reinforcement (reward), to be overwritten
void | putInBuffer (matrix::Matrix *buffer, const matrix::Matrix &vec, int delay=0)
virtual void | fillSensorBuffer (const sensor *x_, int number_sensors)
| puts the sensors in the ringbuffer
virtual void | fillMotorBuffer (const motor *y_, int number_motors)
| puts the motors in the ringbuffer
virtual void | management ()
| handles inhibition damping etc.
## Protected Attributes
unsigned short | number_sensors
unsigned short | number_motors
unsigned short | buffersize
matrix::Matrix * | x_buffer
matrix::Matrix * | y_buffer
matrix::Matrix * | x_context_buffer
bool | manualControl
| True if actions (sats) are selected manually.
int | action
| action
int | oldaction
| old action
int | state
| current state
double | reward
| current reward
double | oldreward
| old reward (nicer for plotting)
ClassicReinforceConf | conf
bool | initialised
int | t
int | managementInterval
| interval between subsequent management calls

## Detailed Description

class for robot controller using several feedforward networks (satelite) and one selforg controller

## Constructor & Destructor Documentation

ClassicReinforce | ( | const ClassicReinforceConf & | conf = `getDefaultConf()` | ) |

~ClassicReinforce | ( |  | ) | `[virtual]`

## Member Function Documentation

virtual matrix::Matrix calcMotor | ( | int | action | ) | `[protected, pure virtual]`

returns action Matrix from discrete actions, to be overwritten

virtual double calcReinforcement | ( |  | ) | `[protected, pure virtual]`

returns the reinforcement (reward), to be overwritten

virtual int calcState | ( |  | ) | `[protected, pure virtual]`

returns state, to be overwritten

void fillMotorBuffer | ( | const motor * | y_,
|  | int | number_motors |
| ) |  |  | `[protected, virtual]`

puts the motors in the ringbuffer

void fillSensorBuffer | ( | const sensor * | x_,
|  | int | number_sensors |
| ) |  |  | `[protected, virtual]`

puts the sensors in the ringbuffer

virtual int getActionNumber | ( |  | ) | `[protected, pure virtual]`

returns number of actions, to be overwritten

static ClassicReinforceConf getDefaultConf | ( |  | ) | `[inline, static]`

list< Inspectable::iparamkey > getInternalParamNames | ( |  | ) | const`[virtual]`

The list of the names of all internal parameters given by getInternalParams().

The naming convention is "v[i]" for vectors and "A[i][j]" for matrices, where i, j start at 0.

Returns:
  : list of keys

Reimplemented from Inspectable.

list< Inspectable::iparamval > getInternalParams | ( |  | ) | const`[virtual]`

Returns:
  : list of values

Reimplemented from Inspectable.

virtual int getMotorNumber | ( |  | ) | const`[inline, virtual]`

returns the mumber of motors the controller was initialised with or 0 if not initialised

Implements AbstractController.

virtual int getSensorNumber | ( |  | ) | const`[inline, virtual]`

returns the number of sensors the controller was initialised with or 0 if not initialised

Implements AbstractController.

virtual int getStateNumber | ( |  | ) | `[protected, pure virtual]`

returns number of state, to be overwritten

list< Inspectable::IConnection > getStructuralConnections | ( |  | ) | const`[virtual]`

Specifies which parameter matrix forms a connection between layers (in terms of a neural network) The orderning is not important.

Returns:
  : list of layer names with dimension

Reimplemented from Inspectable.

list< Inspectable::ILayer > getStructuralLayers | ( |  | ) | const`[virtual]`

Specifies which parameter vector forms a structural layer (in terms of a neural network) The ordering is important.

The first entry is the input layer and so on.

Returns:
  : list of layer names with dimension

Reimplemented from Inspectable.

void init | ( | int | sensornumber,
|  | int | motornumber,
|  | RandGen * | randGen = `0` |
| ) |  |  | `[virtual]`

initialisation of the controller with the given sensor/ motornumber Must be called before use.

The random generator is optional.

Implements AbstractController.

void management | ( |  | ) | `[protected, virtual]`

handles inhibition damping etc.

void notifyOnChange | ( | const paramkey & | key | ) | `[virtual]`

Is called when a parameter was changes via setParam().

Note that it is not called of parameters of childs are changed, then there notifyOnChange() method is called. The key and of the changed parameter (use getParam() to retrieve its actual value). Overload this function when special actions have to be taken on parameter changes.

Reimplemented from Configurable.

void putInBuffer | ( | matrix::Matrix * | buffer,
|  | const matrix::Matrix & | vec,
|  | int | delay = `0` |
| ) |  |  | `[protected]`

bool restore | ( | FILE * | f | ) | `[virtual]`

loads the controller values from a given file.

Implements Storeable.

void setManualControl | ( | bool | mControl,
|  | int | action_ = `0` |
| ) |  |  |

enables/disables manual control, action_ is the sat network number to be used if mControl is false, action is ignored

void step | ( | const sensor * | x_,
|  | int | number_sensors,
|  | motor * | y_,
|  | int | number_motors |
| ) |  |  | `[virtual]`

performs one step (includes learning).

performs one step (includes learning). Calculates motor commands from sensor inputs.

Calulates motor commands from sensor inputs.

Implements AbstractController.

void stepNoLearning | ( | const sensor * | x,
|  | int | number_sensors,
|  | motor * | y,
|  | int | number_motors |
| ) |  |  | `[virtual]`

performs one step without learning. Calulates motor commands from sensor inputs.

Implements AbstractController.

bool store | ( | FILE * | f | ) | const`[virtual]`

stores the controller values to a given file.

Implements Storeable.

## Member Data Documentation

int action`[protected]`

action

unsigned short buffersize`[protected]`

ClassicReinforceConf conf`[protected]`

bool initialised`[protected]`

int managementInterval`[protected]`

interval between subsequent management calls

bool manualControl`[protected]`

True if actions (sats) are selected manually.

unsigned short number_motors`[protected]`

unsigned short number_sensors`[protected]`

int oldaction`[protected]`

old action

double oldreward`[protected]`

old reward (nicer for plotting)

double reward`[protected]`

current reward

int state`[protected]`

current state

int t`[protected]`

matrix::Matrix* x_buffer`[protected]`

matrix::Matrix* x_context_buffer`[protected]`

matrix::Matrix* y_buffer`[protected]`

The documentation for this class was generated from the following files:

- classicreinforce.h
- classicreinforce.cpp