AILWindow

“You may look at the world any way you please, through any window you choose. (It doesn’t mean) it always have to be the same window. Naturally how the world appears, what you see and what you miss, and the angle on what you see, depends on which window you are using, but how can a window be right or wrong? A window is a window. (This is) not to say that you should look through them, more to point out that there are other windows that sane, sensible, responsible, amiable, and otherwise normal people can and do look through, and it is perfectly OK to try another window if what you see through yours seems meaningless or inadequate…The number of different windows is endless. The unbridled conceit of the western nexus is to say that only certain windows are ‘sound,’ and that others are wrong, misleading, hallucinatory, etc.”

george spencer brown

Summary

AILWindow is an interactive installation. It’s core elements are two screens, centrally installed inside the building of Andreastrasse; a diverse set of sensors that collect various kind of data about the behavior of the people inside the building; an artificial life simulation that plays on the screens.

The intention of this paper is to prepare the discourse on the project AIL-window. It starts with an introduction to the framework of AILwindow and the specified requirements, defined in such a way to show it’s usability beyond the interactive installation.
Based on the framework it will propose a possible solution for an interactive installation, based on the concept of „collaboration“.

Background

The AILab with is multifaceted discourse upon robotics, behavior and intelligence is not visible in this sterile building it is located. From outside there are no hints that science is being done and answers are being found that will have an impact upon our future lives, the way we see the world around us and subsequently the way we see ourselves.

The intention of the framework described in this paper is to create an artefact that changes this situation.

Introduction

AILWindow’s form is defined by two screens attached to both sides of the bridge in the 2nd floor, connecting the two wings of the building at the andreasstrasse 15. This allows to make it visible from almost every spot inside the building and creates a virtual tunnel for those that choose to pass over this bridge.
Different sensors collect data about the behavior of people inside the building. This data will be used to make the interactive installation possible.

The screen displays

• an artificial life simulation, based on the simulation software GoApe, developed here at the AILab. This simulation is the core of the content, especially because it is designed to fit with the collected sensor information and create thus a meaningful interaction with a functioning feedback loop.
• Video showing current projects of the lab
• Sponsorship information
• Content provided by other institutes, schools or companies that participated in this project

Requirements

The AIL-window is designed as a framework which allows collaborators from different fields to participate. Because of it’s affiliation to the AILab, the location, the design of the screen and the planned interactive content, the following requirements are being identified:

• As an AILab installation:
  • A platform on which ideas and concepts from the AILab can be playfully explored or presented.
  • Test bed for Software created by the lab.
• As a house installation:
  • Because of it’s location the content has to be calm (means slow movements or changes), otherwise it would be a distraction to worker in it’s immediate neighborhood.
  • It must appear as a house installation and not as an AILab public relation tool.
  • The content and the form of interaction must be of interest for all the people in this house. They all should be able to understand it (as far as this is possible: it certainly excludes purely scientific simulations that are incomprehensible for the uninitiated). However, in order to work as art there needs to be ambiguity and/or space for multiple possibilities of interpretation.
  • The aesthetics fit to the space it is displayed in.
• As a display:
  • Other content then the proposed interactive simulation can be shown.
  • Content from other organizations in this house (or sponsors) can get „airtime“ too, depending on their support for this installation.
  • It should be avoided that the installation resp. it’s display becomes a pure advertising tool as we know it from the railway station.
  • The display has to be bright enough to be visible in daylight condition.
  • As an interactive installation:
  • A feedback loop that is recognizable by the spectators. This means the interaction schema should be self explanatory through careful observation and a playful approach.
  • The interaction schema fits together with the displayed content. This means whatever the spectator behavior is used as a sensory input it has to make sense and not appear arbitrary.
• As an art installation (my personal goals):
  • To bring all this requirements together and shape it into a unity that works in all relations between
    • Concept
    • Interactivity
    • Simulation
    • Aesthetics
    • Location
    • Other purposes for the framework.
  • It makes people forget themselves.
  • It makes people aware of themselves.

Interactive Installation

My personal motivation in this project is the creation of the interactive installation. However, as the requirements show, the framework proposed is shaped in such a way that it is open for a collaboration between scientists and artists and the final result can be used for different purposes than the interactive installation alone.

In the case of the interactive installation I defined the requirements for a solution that brings the key elements Concept, Interactivity, Simulation, Aesthetics and Location together into a unity that couldn’t be separated anymore. I will shortly show how this could look like.

Solution

Under solution I understand the description of the key elements mentioned above and the relationships between them. The reason for this definition is to draw a distinction to the term concept, which describes the basic ideas, reflections and content of the solution. Thus I want to point out that in the framework AILWindow many different solutions are possible, each with a different concept. Because I will propose a solution myself, I thus want to allow space for proposals by other collaborators, giving them an idea in which practical problem space I operate when trying to create an interactive installation. I hope to engage this way a discourse, starting here with my own solution.

Solution No 1: „Collaboration“

Concept

I found in the term „collaboration“ a common thread connecting the organizations and corporations inside the house, the people in this lab with it’s international affiliations and obviously the team that has been participating in this project. The hint for this simple principle came from a book by Douglas R. Homesteader, describing a computer tournament organized by Robert Axelrod, in which different strategies were played against each other in a setting similar to the prisoners dilemma. The interesting overall conclusion of this experiment was the emergence of collaboration (resp. TITforTAT) as the most successful strategy.

Even though, according to the philosophy of this lab, collaboration is an emergent behavior and cannot be a basic principle for life, I decided to make it the basic principle of this solution.

Before I show how the practical solution looks like I would like to lay down some theoretical groundwork.
Under interactivity I understand the form of interaction that occurs when a spectator (aware or unaware) influences the simulation and in turn the simulation influences the behavior of the spectator. In order to have a successful and interesting interaction, the behavior of the spectator and the simulation are closely coupled by the kind of sensor used, the feedback given and the metaphors used. At the moment I don’t have a formalized theory that describes this problem.
I dare to propose though, that the result of a successful interaction will be an „emergent sense“, something that only through the interaction itself can be grasped. Observing the behavior of the spectator or the behavior of the simulation separately would not point to this result. This argumentation follows the general philosophy “embodiment” of the lab, that intelligence /awareness needs a body that has to engage with it’s environment.

The “emergent sense” of this solution will deal with aspects of collaboration. The interesting aspect of this argumentation is the nature of collaboration itself as an emergent behavior between different agents. And to make it even more sexy, the spectators have to turn into collaborators themselves in order to interact with the simulation successfully.

How this will look like is not defined yet, but the following sketch gives an idea how this might work:

Interactivity

The location of the screen allows to use the court on both sides of the bridges as an interactive playground. Since it is the intention to keep beside the screen the impact on the house to a minimum, the use of sophisticated color and motion tracking ill be used to locate spectators in front of the screen. Through careful computer analysis it will be possible to see if a person tries to interact with the simulation and let it react accordingly.
A standard interaction might be a spectator/interactor that stands in front of the screen. The tracking software will signal the simulation the position of a nonmoving person (maybe after 5 seconds of immobility) and an agent species will start to react on this incoming signal. The agent species show collaborative behavior. The result of their behavior might be a structure or pattern. To give a positive feedback it’s location on the screen must resemble the location of the spectator. With a positive feedback I understand a reaction of the simulation that the spectator can trace back to his own behavior resp. influence.
Because there will be no user manual available, only careful observation of the previous interaction can lead to collaborative behavior among the spectators. Since this installation is designed to stay in its position for some time and the interaction space is occupied not only by spectators but of passersby as well, it would be ok if it takes some time until the influence of collaborative behavior is discovered by car full observers.

Hints will be necessary though: A different agent behavior will emerge if a group of people is involved, where the evolving pattern or structure will be different in size or form. Because the interaction space is used as a meeting place as well, clusters of people happen often throughout the day. It is therefore likely that this effect will happen unplanned and seemingly random, but hopefully observed and, depending on the effect and the incentive to recall it, consciously replayed. This will require collaboration, and it is hoped that once this step is done, the play fullness of the spectators will lead to different shapes. The interaction will be programmed thus, the more complex this collaboration will be, the more complex behavior the agents will show. An incentive to play could be the duration of the resulting effects. More complex behavior will leave structure/patterns/behavior that can be seen for hours or maybe days.
A very sophisticated scenario could be a web interface through which new agent behavior will be programmed and a collaboration pattern, by which a group must interact, is chosen. If the group plays accordingly, they will be able to support their agent and leave it with more fitness, thus defining through ingenuity and collaboration the overall appearance of the simulation. It might be possible to define different kinds of agent species that collaborate themselves with each other, in order to create even more complex environments. Thus a competition between different groups might emerge, challenging each other with new agents and collaborations to leave their marks, without getting a winner, since the result would only be a more complex and colorful environment.

This sketches show that there is reason to believe to tap into a huge set of possible combination and complexities. How this might be technically possible and stay comprehensible for the spectator is still an open question though.

Life simulation

Under the previous chapter much is already being described. The simulation will be an agent simulation with possibly many different species resp. behaviors, that create patterns or structures through collaboration, triggered by the collaboration of spectators. The species themselves will interact with each other, again maybe controlled/influenced by spectators collaboration.

Aesthetics

Nothing is defined yet, except the set requirements.

Summary

Collaboration is going through all aspects of this solution. As a unifying concept that all of human endeavors share, it starts with the people of the lab and brings them together with everybody else that works in this building and beyond. It is the basic principle of the agent simulation and the interaction scheme.