This easter I got the opportunity to work in a large theaterspace (BIT Teatergarasjen) to continue my work on relief projection (or masked projection).
I made 9 plywood boxes to use as my projection objects, and worked with two projectors, having a total of 16 projection surfaces.
One of the aims for this session was to also work with sound (each object would double as a speaker), and to create a depth in the placement of the objects.
I implemented my nodio system into the projection patch, which made it possible for me to create sequences of movement.
There are plenty of images from the session here.
I found the title of this post in one of Michael Naimarks essays, I guess it also could have been called augmented reality, projection of a virtual object onto a physical object, projecting a virtual layer ontop of a physical geometry, masking of projections, etc.
I have been researching different ways of projecting on other things than flat surfaces: projections that project on objects, follow the shape of the room, and projections of virtual 3D shapes onto physical 3D shapes.
In my own work I have used projections as advanced light sources, masking as a way to fit flat projections on objects and surfaces, but also to create the illusion of multiple screens from a single source. Some examples here.
My goal has been to create tools which make it easy to start working with a physical space immediately, being able to make changes in realtime. I have mainly done this by using multiple opengl videoplane layers in max/msp jitter, with one of the layers having a drawing mode so you are able to draw the shape of a particular object after you have placed a opengl layer over it. I made a crude 3 layer tool for the workshop I did at KHIO this summer to enable the participants to immediately start relating to the physical space.
The master of multiple opengl videoplanes in Norway is Piotr Pajchel in his work with Verdensteatret.
I have done some experiments with projecting a 3D shape onto physical objects, but still have a long way to go in terms of having a simple setup for this.
Obviously I have been looking at what other people have been doing, but none of systems I have found seems to be available to the public, and few of them seem to have been used beyond the developing-period of the system, which might be a sign of them not being as flexible as wanted, and maybe also quite timeconsuming to prepare.
Most systems uses a method to track the shape/space they want to project onto in combination with custommade software, to be able to map the projected image correctly onto the physical object, which is related to the lens specifications of the projector, the placement of the projector in relation to the objects to be projected on, etc.
The LightTwist system developed at the University of Montreal (not much seems to have happened after 2004) use “a panoramic (catadioptric) camera to get correspondances between each projector pixel with the camera pixel. This camera represents the viewpoint of our futur observers. Then, from what the observer should see, we can build the projector images from their respective mapping.”
The videobjects from Whitevoid design in Germany is a software for realtime distortion of video to fit physical objects, but using predistorted video, and you calibrate it either with a helpgrid or by importing a model of the realworld setup. So you would need to first create the 3D shapes to project onto, and then decide how the video will map onto the 3D objects, and finally doing the calibration to match up the virtual objects with the physical ones.
I think the most spectacular callibration solution so far is the “automatic projector calibration with embedded light sensors” (pdf), a collaboration between people from Carnegie-Mellon, Mitsubishi Electric Research Lab and Stanford. They use fiberoptics and light sensors built into the objects/surfaces to be projected on, and by projecting a series of grey coded binary patterns, a custom-made software is able to adjust the image in less than a second to perfectly fit the projectionsurface, with a much higher resolution than a camerabased solution. Take a look at the impressive video:
The pdf and video seems to be from 2004, but I found some more information here. They are hoping to make the system fast enough for live tracking of moving objects, and also to make the calibration pattern invisible using infrared light.
If you have a big budget you could always invite Circus of Now to do the video for you (”We build skyscrapers of light”).
At Ars Electronica this year I had the pleasure to see Palbo Valbuena´s Augmented Sculpture (image at top of this post) which consists of a physical structure in the corner of the room, with the exact same virtual shape projected onto it using one projector. By then animating the color and lighting of this virtual shape, some very interesting light/shadowplays happen. Valbuena collaborates with some game developers in Spain who constructed the virtual model and animation in a standard 3D software.
This work shows the potential in augmented reality using videoprojection, and I hope to see more of his work soon (He has a big outdoor installation in Madrid at the moment, hopefully there will be some documentation soon.)
update feb 5th 2008: Valbuena has updated his website with documentation of several projects: different versions of the augmented sculpture and the public square installation in Madrid.
I was introduced to Martin Andersen last week, the artist behind the mirror project in Rjukan. Rjukan is in a valley where the sun disappears behind the mountains 5 months a year. Martin wants to construct a heliostat mirror (it follows the position of the sun) to get sunlight to the town square of Rjukan. This is actually an old idea from 1913 supported by Sam Eyde, the director of Norsk Hydro (which basically founded Rjukan for industrial purposes).
Unlike a similar project in Viganella in Italy which uses brushed steel as the reflection surface, the mirror project will use mirrors which focus the sun only to the town square (about 100m2).
This weekend I got the chance to see the two installations “Fortellerorkesteret” and “Louder” by Verdensteatret at Kunstnernes Hus in Oslo. Both come out of two theater performances by Verdensteatret. I had the opportunity to be a little bit involved in the production of Fortellerorkesteret so it was nice to see it in this huge beautiful space.
The installations are a mix of sculpture,sound,video,kinetic objects,light and shadows, and are inspiring examples of compositions in space. Fortellerorkesteret has a more theater structure, resemblant of old mechanical puppet theater, while Louder is more of a spatial experience dominated by the huge mechanical spider and the numerous speakers.
Summer has seen few posts, but not because of lack of activity. I have been busy researching for the upcoming urban interface oslo exhibition. One of my challenges has been to find a setup with a short-range fm sender, a mp3 player and a solarcell/battery solution that would make it possible to place this unit anywhere and leave it running. This has proven a lot harder than I would have imagined.
First, there are now so many fm senders available, and it is really hard to distinguish good from bad (90 % bad), and the reviews out there are not trustworthy. There also seems to be so many factors affecting signal strength so the same sender might appear to have a strong signal one day and a really weak one the next day. Frustrating and time consuming, and I am still checking out different modules.
I guess I should ask Raghav Mahato in India, who claims to have built a radio sender for just over 1 usd.
I knew very little about solar power solutions before this summer, and the options range from build it yourself solutions to “how to survive after a nuclear attack” systems. I looked first at relatively new products advertised as solar chargers for mobile devices like ipods, cellphones and pdas.
Most of these mobile solar chargers combines small solar panels with an internal rechargeble battery with different types of breakout plugs.
Not all of them are able to provide power while charging, which was necessary for my setup. I gave the freeloader a try as it sounded like the perfect companion for a fm sender and mp3 player, it has a 2100 mAh battery, but the results were very disappointing. I guess the current from the solar panels is just too weak to provide enough power to keep the battery charged while in use.
There are not so many other ready-made solutions, either they have a smaller capacity than the freeloader or they are just chargers (so the batteries can´t be used while they are being charged).
I am now looking at more heavy-duty solutions, 6-15W solar panels in combination with a charge controller and a lead battery, but this becomes quite expensive and heavy. And all I need is about 100-150mAh. I am still hoping for the perfect solution to pop up.
This research will also be useful when looking for selfpowered xbee solutions. They seem to have had success with a solar powered xbee system at ITP.
I managed to do a little bit of work with the Xbees a few weeks ago, finally getting the directmode (connect a input of one xbee to the output of another xbee, without using an external microcontroller) to work, and also exploring the sleep options, which means being able to keep the power used to a minimum.
I have discovered lots of interesting alternative energy websites, here are a few of them:
This project has been mentioned enough places already I guess, but it fits too well to my research theme for me to leave it out: Morphovision by Toshio Iwai, shown at Siggraph and Ars Electronica last year. It is almost disturbing how frequency and patterns of light combined with motion make a solid model house morph into very liquid shapes.
“Morphovision is a unique display system that interactively transforms and animates a 3D solid object before our eyes. In this system, a model house is rotated at high speed, and is illuminated with special lighting from a digital projector. This enables the model to be distorted into various shapes.” More info here.
Video from Ars Electronica:
I was reminded of Morphovision when reading about a low-tech project posted on the serendipity blog, the time fountain, molding time and form with strobelight.
(Slideshow from the rehearsals and performances here)
My latest collaboration with Yannis Kyriakides is a piece based on old telegraph code books. I use scanned pages combined with microscope texture from these books and project onto the orchestra from two sides, using the musicians as screens in combination with a wide screen behind, trying to create a dynamic space using text fragments and letters as projected light.
It is a coproduction between ZKM in Karlsruhe and Musikfabrik in Cologne, and is performed may 17th in Cologne and on the 18th at ZKM.
(Slideshow of more images from the projection sketches here)
The second focus in this lab session was to work with video projections, and masking them to create several projection surfaces from one projector, and to be able to relate to physical shapes in the room, like my projection on a sphere in Iball, or the masking of video to fit the gallery spaces in one of the sleepers installation:
The simplest way to do it is to work with a 2 dimensional mask. I also wanted to try to work with 3D masks: projecting a virtual 3D version of an object back onto itself. In this way I managed to cover 4 sides of a cube with 2 projectors. Another thing I did was to link the individual projection surfaces (from one projector) so focus could move from one area to another, quite like how the nodes in nodio operate.
Finally, I did some simple tests using the projector as a light beam.
(Slideshow of more images of the light space modulator here)
One of my inspirations for the research fellow program has been the work of Lazlo Moholy-Nagy, both his ideas related to labs, and his art work dealing with creating spaces with movement and light. I went to see the replica of the Light Space Modulator at the Van Abbe Museum in Eindhoven, Holland, both to see how the physical space was altered, but also to investigate the translation from a 3D space created by the kinetic sculpture to the beautiful black-and-white film Schwartz-Weiss-Grau. I believe having read somewhere that Moholy-Nagy´s intention with the Light Space Modulator was as source material for the film, not as a freestanding sculpture. This would make sense to me, as it was a bit disappointing to see the installation in the museum, but interesting to look at it through my camera lens. Below you see a short recording I did, and an excerpt from Moholy-Nagy´s film.
“The model consists in a cubic box [...] with a circular opening (stage) at the front. Surrounding the opening, on the back side of the board, I have mounted a number of yellow, green, blue, red, and white electrical bulbs [...]. Inside the box, parallel to the front, there is a second board, also with a circular opening, around which a further set of light bulbs is mounted.
Single bulbs light up at different places according to a pre-set plan. They illuminate a continuously moving mechanism, made partly of transparent, partly of cut-out materials, in order to create linear shadows on the back wall of the closed box. (When the presentation takes place in a dark room, the back wall of the box can be removed and the color and shadow projection behind the box projected on a suitable screen of open dimensions.)“
This means it installed wrong in the Van Abbe museum, you are supposed to look at it from one position, through two holes in a box which contained the kinetic object. The Van Abbe Museum have put it in the middle of a open room, with static light, thus ruining a lot of the interplay between changing lights,movement and shadows which was Moholy-Nagy´s intention (which probably explains why the film is more interesting than seeing the replica).
