BRAIN TO BRAIN VR MODULATOR

B2B VR Modulator>
Extended space>
Immersion>
Source>
Feedback>
Machine Learning>
Authors>

BRAIN TO BRAIN VR MODULATOR

The project examines systems of exchange between numerous brains in VR. We are using a brain interface (BCI), which reads the activity of cerebral cortex that is subsequently synthesized into complex visual manifolds. Reading of waves and synthesis take place in real time. The final project uses two human brains. Signals from one brain are sent to the other and back on the basis of oryginal interface that combines BCIs and VR helmets. Initial tests have shown the ability of establishing a sophisticated communicational connection between two brains and complex interferential patterns. The feedback between brains may deliver solutions, which can't be achieved by the brains working independently - new central nervous system consisting of many brains in VR environment.

EXTENDED SPACE

The project uses non-euclidean geometries and multi-dimensional manifolds as best for visual representation of mental states. We manipulate slices and projections of manifolds e.g.: 2d submanifolds of 4d space and 3d slices of 5-dimensional knotted objects. We research classes of embeddings, isotopies and energy of knots. These models are used in contemporary physics and have to be use in immersive art and other domains.

BB5 hyperbolic space by RBL

IMMERSION

The project explores immersive environments and sound spatialization. Brains impulses are transformed into multi-dimensional geometric objects and and low frequency sounds. Time is defined by the sequence of events and transformations. Each interval is created according to brain activity and its form is synchronized with spaces.

FEEDBACK

We place 2 performers in virtual reality. Signals from one brain are sent to the other and back based on BCI and VR. We represent states of brains of the performers as multidimensional objects in VR. In this approach we research 'feedback' between the performer and environment created by second performer. This model implicitly assumes a recursive process, namely a feedback between the output from environment and performer's mental actions. The trained model can modulate the overall system 'loops' and predict its possible long-term outputs. The project will find wide-ranging applications in architecture and art, and in many contemporary contexts that can be characterized by 'feedback' between many agents and space in dynamically evolving scenarios.

MACHINE LEARNING

In next step we use machine learning techniques. The reinforcement learning extends the idea of feedback. Meta-learning describes research that aims to create artificial agents/performers capable of multi-tasking intelligent actions. Simply, reinforcement learning is a modulation of agent's actions based on reward from environment. The agent takes actions so as to maximize some notion of cumulative reward. The selection of actions by agent is modeled as a map called 'policy'. The agent searches for a policy with maximum expected return. The method is powerful by the use of samples to optimize performance and the use of function approximation to deal with large complex environments.

Authors

concept and coding: Robert B. Lisek
support: SUBNET and Center for Human-Computer Interaction Dept of Computer Science Salzburg University
[special thanks for Martin Murer and Ruediger Wassibauer]
contact: lisek at fundamental dot art dot pl

produced by & Fundamental Research Lab & Subnet and HCI Salzburg