This easter I got the opportunity to work in a large theaterspace (BIT Teatergarasjen) to continue my work on relief projection (or masked projection).
I made 9 plywood boxes to use as my projection objects, and worked with two projectors, having a total of 16 projection surfaces.
One of the aims for this session was to also work with sound (each object would double as a speaker), and to create a depth in the placement of the objects.
I implemented my nodio system into the projection patch, which made it possible for me to create sequences of movement.
There are plenty of images from the session here.
I collected a library of field recordings I have made over the years. I set up an internet radio station for the project (using Nicecast), and played with different configurations mixing music and the field recordings, but ended up just using my own recordings.
I thought it could be interesting to stream internet radio, a global media, to very specific local areas. I found three locations in Oslo which would serve as the local radio stations. They were somehow connected to a clear visible cue in that location: A huge oak tree, a small sculpture, and a small pound in the roundabout. The range of the local stations would more or less correspond to these visual cues: If you saw them you would be able to pick up the signal from the radio stations. In numbers this would mean a range of 30-100 meters.
My original plan was to use the fm-senders for mp3 players which are mostly meant to be used to listen to the mp3 player through the car radio. This was partly because I was experimenting with solar energy as energy source, and because they were cheap. The range and quality of the signal wasnt good enough though, so I ended up getting the more powerful aareff fm transmitters .
The senders were placed with friendly hosts, letting me use their internet connection to pick up the internet radio stream.
The result was three very local radio stations sending out a continous soundtrack from other locations, so somehow these recorded locations came in dialogue with the physical locations of the radio stations.
The listening involved active participation from the public as you would need to tune in on your own radio to pick up the broadcast.
Soundpockets is a series of projects of intimate sound interventions in public spaces. By using fm radiowaves, soundbeams and miniature speakers to create local pockets of sound, the different projects have different scope and focus: creating private listening rooms, changing soundtracks of a location, displacement of time/or space and a bit of general disruption of everyday life.
I made Soundpocket 1 as part of the exhibition urban interface oslo fall 2007.
From urban interface osloblog:
“Hauntings? Dimension Doors? Time tunnels?
A boy heard what appeared to be the sound of a sheep coming from the wall of Strykejernet Art School.
A bartender at Blå was concerned when he heard running water like that from a leaking water pipe. The sound disappeared before he was able to locate it.
A seagull can be heard, but is nowhere to be seen.
Soundpocket artist HC Gilje is causing slight disturbances in the urban interfaces.”
Using a directional soundbeam to project a localized sound into a public space, this sound being only heard by the people within the sound beam which can be as narrow as 50 cm in diameter. It is similar to a lightbeam, only being sound instead. When it hits a surface it is reflected.
Soundpocket 1 was installed in a narrow passageway in Oslo, connecting two parts of the city. The soundbeam was mounted on a pan/tilt head making it possible to place the sounds very precisely in the passageway.
By bouncing the sound off surfaces, it seems as if the sound is coming from a window, door, elevator, a poster on the wall or just a more general presence. This made the piece into something which added another layer of sound to the existing soundscape, blending (sometimes disappearing) into the location.
Most of the sounds would appear to belong to the site, although dislocated (like the sound of the chandelier in the wind), the sounds of birds, telephones, babies crying, dogs barking, water running etc.
It was interesting to see how the piece was received. It was obvious for me that it wouldnt work very well as a typical art piece, it has a much more interventionist nature. I wanted it to be slight distortions to the regular soundscape of the passageway, and was pleased to see that the people who used this passageway regularly were noticing these disturbances. This could be described using the first of Barthes´ three listening modes: hearing involves “evaluation of the spatio-temporal situation“ and thus, it is linked to a “notion of territory“. It places the listener on alert when new sounds which dont´t “fit in” are heard.
By adding an extra layer of sound if also made people focus on the sounds which were already there.
The inspiration from this comes from when I studied in Trondheim in the 90ies, and I heard some stories about how a directional speaker had been used to cause a certain distress on a bridge over the local river: A person walking alone across the bridge suddenly hear whispering voices. An out of tune clarinet is projected into a marching band playing on the 17th of may (Norway´s national holiday).
If these stories are true or not, doesnt really matter, it is the idea of having a private experience in a public space which intrigued me.
Soundpockets is a series of projects of intimate sound interventions in public spaces. By using fm radiowaves, soundbeams and miniature speakers to create local pockets of sound, the different projects have different scope and focus: creating private listening rooms, changing soundtracks of a location, displacement of time/or space and a bit of general disruption of everyday life.
I found the title of this post in one of Michael Naimarks essays, I guess it also could have been called augmented reality, projection of a virtual object onto a physical object, projecting a virtual layer ontop of a physical geometry, masking of projections, etc.
I have been researching different ways of projecting on other things than flat surfaces: projections that project on objects, follow the shape of the room, and projections of virtual 3D shapes onto physical 3D shapes.
In my own work I have used projections as advanced light sources, masking as a way to fit flat projections on objects and surfaces, but also to create the illusion of multiple screens from a single source. Some examples here.
My goal has been to create tools which make it easy to start working with a physical space immediately, being able to make changes in realtime. I have mainly done this by using multiple opengl videoplane layers in max/msp jitter, with one of the layers having a drawing mode so you are able to draw the shape of a particular object after you have placed a opengl layer over it. I made a crude 3 layer tool for the workshop I did at KHIO this summer to enable the participants to immediately start relating to the physical space.
The master of multiple opengl videoplanes in Norway is Piotr Pajchel in his work with Verdensteatret.
I have done some experiments with projecting a 3D shape onto physical objects, but still have a long way to go in terms of having a simple setup for this.
Obviously I have been looking at what other people have been doing, but none of systems I have found seems to be available to the public, and few of them seem to have been used beyond the developing-period of the system, which might be a sign of them not being as flexible as wanted, and maybe also quite timeconsuming to prepare.
Most systems uses a method to track the shape/space they want to project onto in combination with custommade software, to be able to map the projected image correctly onto the physical object, which is related to the lens specifications of the projector, the placement of the projector in relation to the objects to be projected on, etc.
The LightTwist system developed at the University of Montreal (not much seems to have happened after 2004) use “a panoramic (catadioptric) camera to get correspondances between each projector pixel with the camera pixel. This camera represents the viewpoint of our futur observers. Then, from what the observer should see, we can build the projector images from their respective mapping.”
The videobjects from Whitevoid design in Germany is a software for realtime distortion of video to fit physical objects, but using predistorted video, and you calibrate it either with a helpgrid or by importing a model of the realworld setup. So you would need to first create the 3D shapes to project onto, and then decide how the video will map onto the 3D objects, and finally doing the calibration to match up the virtual objects with the physical ones.
I think the most spectacular callibration solution so far is the “automatic projector calibration with embedded light sensors” (pdf), a collaboration between people from Carnegie-Mellon, Mitsubishi Electric Research Lab and Stanford. They use fiberoptics and light sensors built into the objects/surfaces to be projected on, and by projecting a series of grey coded binary patterns, a custom-made software is able to adjust the image in less than a second to perfectly fit the projectionsurface, with a much higher resolution than a camerabased solution. Take a look at the impressive video:
The pdf and video seems to be from 2004, but I found some more information here. They are hoping to make the system fast enough for live tracking of moving objects, and also to make the calibration pattern invisible using infrared light.
If you have a big budget you could always invite Circus of Now to do the video for you (”We build skyscrapers of light”).
At Ars Electronica this year I had the pleasure to see Palbo Valbuena´s Augmented Sculpture (image at top of this post) which consists of a physical structure in the corner of the room, with the exact same virtual shape projected onto it using one projector. By then animating the color and lighting of this virtual shape, some very interesting light/shadowplays happen. Valbuena collaborates with some game developers in Spain who constructed the virtual model and animation in a standard 3D software.
This work shows the potential in augmented reality using videoprojection, and I hope to see more of his work soon (He has a big outdoor installation in Madrid at the moment, hopefully there will be some documentation soon.)
update feb 5th 2008: Valbuena has updated his website with documentation of several projects: different versions of the augmented sculpture and the public square installation in Madrid.
I was introduced to Martin Andersen last week, the artist behind the mirror project in Rjukan. Rjukan is in a valley where the sun disappears behind the mountains 5 months a year. Martin wants to construct a heliostat mirror (it follows the position of the sun) to get sunlight to the town square of Rjukan. This is actually an old idea from 1913 supported by Sam Eyde, the director of Norsk Hydro (which basically founded Rjukan for industrial purposes).
Unlike a similar project in Viganella in Italy which uses brushed steel as the reflection surface, the mirror project will use mirrors which focus the sun only to the town square (about 100m2).
This weekend I got the chance to see the two installations “Fortellerorkesteret” and “Louder” by Verdensteatret at Kunstnernes Hus in Oslo. Both come out of two theater performances by Verdensteatret. I had the opportunity to be a little bit involved in the production of Fortellerorkesteret so it was nice to see it in this huge beautiful space.
The installations are a mix of sculpture,sound,video,kinetic objects,light and shadows, and are inspiring examples of compositions in space. Fortellerorkesteret has a more theater structure, resemblant of old mechanical puppet theater, while Louder is more of a spatial experience dominated by the huge mechanical spider and the numerous speakers.
I was back in Berlin for a week in beginning of july, and got the chance to see the excellent exhibition BackJumps -the Live issue #3, which cover new and recent street art works in Germany and the rest of the world. It was partly a exhibition using the huge space at Kunstraum Bethanien, and partly documentation of work from urban spaces. I saw many interesting projects, but will mention two which relates directly to this project blog:
The first one is a animation shown on a monitor in a small room (see image). What´s interesting is that the animation was made on the walls in the same space and you see the trails of the animation on the walls. It is made by Bologna based Blu.
The other project was part of CubaBrasil, Los Berlin BeamBoys, which did huge video projections on buildings in Cuba, partly as a way of bypassing the censorship of certain political slogans and images.
I am doing a series of projects called soundpockets for urban interface oslo, which in different ways tries to create pockets of sound in public space. Some of the versions involve the fm sender- mp3 setup mentioned in the previous post, another one involves a directional speaker mounted on a pan/tilt unit, and this has been my second headache this summer, finding a controllable pan/tilt unit which is reasonably fast, which can handle a load of a few kilos and which is not ridiculously expensive.
Fortunately I have had good help from Soundscape studios. The first ideas was to use a existing movinghead light and refit it with the speaker, but it turned out that the motors wouldnt be able to handle the load. Pan tilt units are usually made for a specific purpose, either light or video, and the few ones which are available for general purpose use and which are controllable are very expensive. The one we have ended up with is quite expensive, but is controlled using serial protocol, is made for outdoor use, and is powerful enough to handle video projectors (for later projects). It is also very fast, up to 300 degrees per second pan, and 60 degrees per second tilt. Hopefully it arrives next week.
The most ambitious plan is to control it using an arduino microcontroller, which will also control a serial-controllable mp3 player, the daisy, so I should be able to place sounds quite accurately in a space, and also create movements with sounds.
If time runs short I will use a macmini with max controlling the sound and pan-tilt unit over the serial port.
I have been testing two different models of directional speakers which uses ultrasound as the carrier signal, I will probably have to go for the smaller one, although I am a bit worried it will disappear in the ambient sound. It is also challenging to find the sounds which works best, and also how to deal with the sounds both coming directly from the narrow beam of the speaker but also the reflections on surfaces in the space.
The inspiration for this projects comes from when I studied in Trondheim in the 90ies, and I heard some stories about how a directional speaker had been used to cause a certain distress on a bridge over the local river: A person walking alone across the bridge suddenly hearing whispering voices. An out of tune clarinet projected into a marching band playing on the 17th of may (Norway´s national holiday).
If these stories are true or not, doesn´t really matter, it is the idea of having a private experience in a public space which intrigued me.
I taught a one week workshop for the director-,choreography-, and scenography students at the National Academy of the Arts in Oslo last week.
My approach was to develop tools in the form of small software applications which made it very easy for the students to immediately start exploring ways of using video in a space.
We also tested out many different projection surfaces, privalite, dmx-controlled shutters, physical masks.
You can see a slideshow of some of the results here.
And even better, you can check out two of the applications I made (built using maxmsp and jitter), for both osX and windows. One is a simple capture program for recording either live stream, animation or recorded footage already on the camera, to your harddrive.
The other is a tool for positioning and scaling 3 planes of video, creating presets of the scenes you create and then make transitions between the scenes.
Google Earth has implemented a new technology called street view, developed by Immersive Media
This could roughly be seen as a mix of very advanced quicktime VR and a film effect from The Matrix movies. Interesting to me as another example of the interrelation between space,time and motion.
From the Google Earth Blog:
“One of the many secrets behind their technology is a patented 11 lens camera system that simultaneously takes photos in 11 directions based on a dodecahedron geometry. They can capture 30 frames a second of high resolution photography. That’s right - we’re talking high resolution video in digital 360. You can stop, start, back up, single-frame, etc”
