| |
|
|
| |
Neural DynamicsThe project proposes Neural Ordinary Differential Equations for
sound and video synthesis. My method provides a simple and
intuitive way to construct new objects and new types of
synthesis by manipulating matrices and maps. Combining of
objects was also trained, which ensures consistency of the
project. Differential equations are the fundamental
language of all physical law. Dynamic laws are differential
equations relating to operators and functions common to a wide
variety of patterns of motion, e.g. oscillators, elastic fluids
(Navier-Stokes equation) or gases.
SynthesisI have been working on new methods of video synthesis using machine learning techniques. I mainly focused on using differential equations and representing them with small neural networks. The main idea of ??artificial neural networks (ANN) is to build up representations for complicated functions using compositions of relatively simple functions called layers. I considered differential equations models. These systems describe the time evolution of the state of a system (x) in time, using an expression which involves the derivative of the current state. A differential equation fits into my neural network framework, as it takes in some parameters and produces the solution as output and it is differentiable. I can use a differential equation as a layer in a neural network. This is really neat for a few reasons.
RealityDifferential equations are the fundamental language of all physical laws and can have many application in art. Outside of physics and chemistry differential equations are an important tool in describing the behavior of complex systems. Dynamic laws are differential equations relating to operators and functions common to a wide variety of patterns of motion, e.g. oscillators, elastic fluids (Navier-Stokes equation) or gases. Using differential equations models in my neural networks allows these models to be combined with neural networks approaches. Building effective neural networks involves choosing a good underlying structure for the network. It is often easier to think of describing how a function changes in time, then it is to write down a function for the phenomena. Relatively simple rate laws can turn into very complex behavior (eg. attractors). They provide new visual techniques that can be used I performances and installations.
UNPREDICTED_1440012 from Robert B. Lisek on Vimeo. UniversalityMeta-learning. I developed method of combing many attractors to
build universal complex structures. This is step forward to
creating meta-learning system that can learn and generate many
different visual and sound objects. It provides also way for new
types of video synthesis. When string theory became popular in
the 1980s the idea was again to have strings propagating in a
background spacetime?though it turned out that for consistency
this spacetime had to satisfy the Einstein equations.
Consistency also typically required the basic spacetime to be
10-dimensional?with the reduction to observed 4D spacetime
normally assumed to occur through restriction of the other
dimensions to some kind of so-called Calabi?Yau manifold of
small extent, associating excitations with various particles
through an analog of the Kaluza?Klein mechanism. It has always
been hoped that this kind of seemingly arbitrary setup would
somehow automatically emerge from the underlying theory. And in
the late 1990s there seemed to be some signs of this when
dualities were discovered in various generalized string
theories?notably for example between quantum particle
excitations and gravitational black hole configurations. So
while it remains impossible to work out all the consequences of
string theories, it is conceivable that among the
representations of such theories there might be ones in which
matter can be viewed as just being associated with features of
space.
ImmersionThe proposed approach provides a platform for a catalogue of music research and composition that can explore topic of sound spatialization and immersion. I have developed machine learning methods that extend granular and pulsar synthesis in composing and new methods of building spatial sound. Restrictions introduced during the pandemic motivated me to research potential of AI and virtual reality for the evolution of society. I thought about implications of teleoperation, immersion and machine learning for organisation of society. I studied how various new methods of operation in virtual and online space can influence future social structures. I prepared plan for new Laboratory that deals with study of new social forms based on virtual, network and AI technologies. The goal of laboratory is to test and develop tools that will be useful to create decentralised network society with horizontal relationships between people, in which decisions can be developed during dialogue, based on directness (personal participation) and in which cases of "representation" as authorities and bureaucracy can be minimised by using intelligent systems. As an introduction to the the Laboratory, I organised and conducted meetings of the Fundamental Research Group, that deals with critical analysis of contemporary networks in the time of pandemic. The goal of laboratory is re-think critically ideas such as freedom, openness, pluralism, transparency and democracy, in the perspective of dynamically developing technologies such as AI and machine learning. The lab is interested also in the potential of machine learning and virtual reality within the field of art and its possible contribution to current political and social discourses.
Social contextSocial systems can be described by dynamical models. The power of the collected and analyzed data goes far beyond the functions of monitoring and supervision, enabling predictive control. Other aspects are related to the future social context. AI-enabled systems are now capable of generating disinformation at large scales. By producing greater volumes and variety of fake messages, these systems can obfuscate their true nature and optimize for success, improving their desired impact over time. Unnecessary tragedy may strike if we allow fiction to define our reality. But what choice is there when we can?t tell the difference between what is real and what is false in the digital world? The problem and questions posed above are important in the present social situation. The power of the collected and analyzed data goes far beyond the functions of monitoring and supervision, enabling predictive control. I was focused on a roles of Presence, Flow, Immersion, and Interactivity. I was particularly interested in the problem of presence and flow in VE. Presence is defined as the subjective experience of being in one place or environment, even when one is physically situated in another. Presence is a normal awareness phenomenon that requires directed attention and is based in the interaction between sensory stimulation, environmental factors that encourage involvement and enable immersion. Flow is a state of experience where someone is completely absorbed and immersed in an activity. I researched relations between presence, flow, immersion and interactivity, e.g. how interactivity and sound spatialization improves experience of presence.
Sound spatializationI have developed machine learning methods that extend granular and pulsar synthesis in composing and new methods of building and transforming virtual environments. In this approach composing and formations of complex sonic or visual structures is based on transformation of distribution and ordering of very large sets of small particles. I presented results of research and new compositions at the IRCAM Center Pompidou in Paris.
Authordevelopment: Robert B. Lisek |
|
| |
produced by Robert B. Lisek & Fundamental Research Lab
|
|
