relief projection 2008

This easter I got the opportunity to work in a large theaterspace (BIT Teatergarasjen) to continue my work on relief projection (or masked projection).
I made 9 plywood boxes to use as my projection objects, and worked with two projectors, having a total of 16 projection surfaces.
One of the aims for this session was to also work with sound (each object would double as a speaker), and to create a depth in the placement of the objects.
I implemented my nodio system into the projection patch, which made it possible for me to create sequences of movement.
There are plenty of images from the session here.

relief projection

augmentedsculpture by pablo valbuena, originally uploaded by hc gilje.

I found the title of this post in one of Michael Naimarks essays, I guess it also could have been called augmented reality, projection of a virtual object onto a physical object, projecting a virtual layer ontop of a physical geometry, masking of projections, etc.

I have been researching different ways of projecting on other things than flat surfaces: projections that project on objects, follow the shape of the room, and projections of virtual 3D shapes onto physical 3D shapes.

In my own work I have used projections as advanced light sources, masking as a way to fit flat projections on objects and surfaces, but also to create the illusion of multiple screens from a single source. Some examples here. (Update may 2008: some more recent examples)

My goal has been to create tools which make it easy to start working with a physical space immediately, being able to make changes in realtime. I have mainly done this by using multiple opengl videoplane layers in max/msp jitter, with one of the layers having a drawing mode so you are able to draw the shape of a particular object after you have placed a opengl layer over it. I made a crude 3 layer tool for the workshop I did at KHIO this summer to enable the participants to immediately start relating to the physical space.
A prime example of multiple opengl videoplanes is Piotr Pajchels work with Verdensteatret.

I have done some experiments with projecting a 3D shape onto physical objects, but still have a long way to go in terms of having a simple setup for this.
Obviously I have been looking at what other people have been doing, but none of systems I have found seems to be available to the public, and few of them seem to have been used beyond the developing-period of the system, which might be a sign of them not being as flexible as wanted, and maybe also quite timeconsuming to prepare.

Most systems uses a method to track the shape/space they want to project onto in combination with custommade software, to be able to map the projected image correctly onto the physical object, which is related to the lens specifications of the projector, the placement of the projector in relation to the objects to be projected on, etc.

The LightTwist system developed at the University of Montreal (not much seems to have happened after 2004) use “a panoramic (catadioptric) camera to get correspondances between each projector pixel with the camera pixel. This camera represents the viewpoint of our futur observers. Then, from what the observer should see, we can build the projector images from their respective mapping.”

The videobjects from Whitevoid design in Germany is a software for realtime distortion of video to fit physical objects, but using predistorted video, and you calibrate it either with a helpgrid or by importing a model of the realworld setup. So you would need to first create the 3D shapes to project onto, and then decide how the video will map onto the 3D objects, and finally doing the calibration to match up the virtual objects with the physical ones.

I think the most spectacular callibration solution so far is the “automatic projector calibration with embedded light sensors” (pdf), a collaboration between people from Carnegie-Mellon, Mitsubishi Electric Research Lab and Stanford. They use fiberoptics and light sensors built into the objects/surfaces to be projected on, and by projecting a series of grey coded binary patterns, a custom-made software is able to adjust the image in less than a second to perfectly fit the projectionsurface, with a much higher resolution than a camerabased solution. Take a look at the impressive video:

The pdf and video seems to be from 2004, but I found some more information at Johnny Chung Lee´s website. They are hoping to make the system fast enough for live tracking of moving objects, and also to make the calibration pattern invisible using infrared light.
update: there is now more information on Lee´s website.

If you have a big budget you could always invite Circus of Now to do the video for you (”We build skyscrapers of light”).

At Ars Electronica this year I had the pleasure to see Palbo Valbuena´s Augmented Sculpture (image at top of this post) which consists of a physical structure in the corner of the room, with the exact same virtual shape projected onto it using one projector. By then animating the color and lighting of this virtual shape, some very interesting light/shadowplays happen. Valbuena collaborates with some game developers in Spain who constructed the virtual model and animation in a standard 3D software.
This work shows the potential in augmented reality using videoprojection, and I hope to see more of his work soon (He has a big outdoor installation in Madrid at the moment, hopefully there will be some documentation soon.)

update feb 5th 2008: Valbuena has updated his website with documentation of several projects: different versions of the augmented sculpture and the public square installation in Madrid.

The mirror project

I was introduced to Martin Andersen last week, the artist behind the mirror project in Rjukan. Rjukan is in a valley where the sun disappears behind the mountains 5 months a year. Martin wants to construct a heliostat mirror (it follows the position of the sun) to get sunlight to the town square of Rjukan. This is actually an old idea from 1913 supported by Sam Eyde, the director of Norsk Hydro (which basically founded Rjukan for industrial purposes).

Unlike a similar project in Viganella in Italy which uses brushed steel as the reflection surface, the mirror project will use mirrors which focus the sun only to the town square (about 100m2).


spletizizers, originally uploaded by hc gilje.

I am in Köln at the moment preparing for the “The queen is the supreme being of the realm” performance. As it happens, I am staying in the same hotel as last time I did a project in Köln:
In 2002 I was invited together with my co-pilot Kurt Ralske from 242.pilots to participate in a project initiated by Tillmann Roth. Roth built a flexible art container/platform (see image), basically a mobile art production and viewing space. He invited different media artists to do a project which would later be shown in the container. The common link in the project was that we should either make use of or be inspired by the sounds of Alan Splet (scroll down). Splet was the sound designer for David Lynch on all his films from the Grandmother to Blue Velvet. His use of ambient sounds very much shaped a space.

Kurt and I did a 45 minute video improvisation with his sounds as the soundtrack. This might become available through Audioframes at some point, or I might put some of it online.

Fictional architecture

This is the first day of my life 1, originally uploaded by hc gilje.

I had the chance to see the Elmgreen & Dragset exhibition “This is the first day of my life” when I was in Malmö last week. A very effective way of transforming the space:
“Upon entering Malmö Konsthall, the spectator is confronted with a long blank white wall and a series of anonymous-looking doors. Some of the doors are dysfunctional and can’t be opened; others will lead the viewer through a complex environment of secret rooms configured in a labyrinthine system.” From the catalogtext.

This reminds me of descriptions of some of Mike Nelson´s work, which I havent had the opportunity to see unfortunately.
I recommend reading an old interview from the Guardian, where his work is described as creating “emotionally charged false buildings within buildings”.

Liquid Space workshop at Club Transmediale07 in Berlin

I was participating in the LiquidSpace workshop organised by the artist-architecture group lab[au] from Brussels. They are developing a software for creating 3d environments to experience image and sound in immersive installation or performance setups. In Berlin this meant a cubic setup of 4 screens and 4 speakers with the audience either on the inside or the outside. I was interested in it in relation to my own research in creating and transforming spaces with image and sound, and the workshop clarified a few differences between lab[au]´s approach and mine. Their focus is creating a virtual 3d audiovisual space for the audience to immerse into. I am more interested in using physical sound positions instead of surround sound, and projecting on the structures of the physical space itself instead of having them images floating in simulated 3d space.

The software for creating the environment is still under development, but I was especially impressed by the sound possibilities of the system.

Here is a short excerpt from some of the results from the workshop.

video installation at 11 in amsterdam

nearuki at 11 in amsterdam 6, originally uploaded by hc gilje.

(Slideshow of more images from nearuki here)

In december 2006 I was invited by my dvd label lowave to do a 12 channel projection at the 11 space in Amsterdam. 11 serves as the café for the stedelijk museum in the daytime, and as bar/club/restaurant in the evening, with an amazing view over the city. I showed Nearuki (sleepwalkers), a series of slowmotion videoes of people crossing a street (one of the sequences is also in Night for Day). It was interesting to see how the slow movements interacted with the busy indoor space, and how it reflected onto the city outside.

Office Building in Almere by UN studio

UN_office3, originally uploaded by hc gilje.

(Slideshow of more images of the building here)

In Responsive Environments by Lucy Bullivant I read about the office building designed by UN studio in Almere, Holland. It looks relative normal on the outside with a silver/grey facade, but it has two openings leading into a spectacular colorful courtyard: the facade of the inner courtyard is covered with a magnetic film which makes the walls change color according to the light and from which angle you look. I had the chance to experience the building in december 2006.

More info


Get every new post delivered to your Inbox.

Join 174 other followers