video projection tool v2.5 + cornerpin keystone app

This post is really an announcement for an updated version of my videoprojectiontool + a new handy cornerpin keystone application.
But first some lines about another program, vvvv:

The last few months I have spent some time working with vvvv, which is a graphical programming language similar to max and pd, but with a few distinct differences.

First of all, it is windows only (It is based on directx instead of opengl), which is the main reason why I haven´t used it before.
The vvvv developer´s attitude is that it is better to make the program as good as possible on one platform rather than trying to make everybody happy.

Second, it has a clearer focus on 3D than max/jitter.
The main reason I have looked at vvvv is because the developers seem to be quite interested in making it a practical tool for real-life projects, to make it easier for the users to focus on content rather than trying to solve programming problems all the time.
Especially for people like me, who work with projectors, multi-screen setups, mapping etc, it was a relief to see dedicated modules for multiscreen blending, cornerpin keystone, and a projector module to make it easier to combine virtual models with physical environments. Also, there are easy-to-find modules and examples for how to interface vvvv with different hardware.

I had an interesting exchange with one of the jitter developers in the cycling74 forum a little while ago, where I was complaining about how such a useful thing as cornerpin keystoning is not available in jitter. I have attempted several times to make this in max/jitter without luck. It took the developer fifteen minutes to make it, but I would have never found this solution myself.
I guess my point is that I wish there was a set of useful abstractions and instructions, especially when working with 3D, instead of always being referred to the “red book” (the opengl reference book). Something like auv-i was for earlier versions of max/jitter for working with video.

And now, to the announcements: cornerpin keystone + v2.5 of videoprojectiontools
I have made a simple application, based on the above-mentioned exchange in the cycling74 forum, for a very simple but effective cornerpin keystoning. So, instead of tweaking the projector settings to get the projection to fit your projection surface, you can now just click in the corners on your desired projection surface, and the image will fit. The source code for the cornerpin keystoning is available on the cycling74 forums.

Videoprojectiontool v2.5 is now available, for mac osx and windows!
The new features are corner pin distortion of the individual layers, + enhanced functionality for drawing directly on the projected output for advanced masking. Please watch the video tutorials to learn more.

If you are working with mapping image content onto physical objects or surfaces, there are two main approaches: Creating a virtual 3D model which matches the physical environment, and then project that 3D model back onto the physical environment. This is not an easy task which involves a lot of calibration, as everything has to match exactly. For this I think vvvv is by far the best tool, check out their tutorial.

The second approach consists of positioning and masking individual layers in the physical environment. I think my videoprojectiontool is a good starting point with this approach. It involves no projector calibration, and you can get some great results in a very short time. By combining 3d placement of layers with advanced masking, and a sequencer for storing presets and creating transitions it has become a quite powerful tool.

Go to the resourcepage for the videoprojectiontool for the links to the new applications and several new video tutorials related to the update.

shift v2: relief projection installation

shift v.2, originally uploaded by hc gilje.

I decided to give my current series of relief projections a name, shift: moving from one place to another, changing the emphasis, direction or focus of something. It also has a loose relation to the idea of shapeshifting.
As mentioned in my previous posts about my relief projection projects, shift combines multichannel sequencing, audio generated from video (soundtransducer inside every box, the sound you hear is directly related to the video projected on that particular box), with masking/mapping a projection to fit physical objects. This creates a dynamic audiovisual landscape, a spatial light painting.
The software to create the installation has developed over almost two years and some workshops, and I have shown documentation of the development, but never exhibited it as a final work.
It is only this autumn that I have found the right opportunity to show it in an exhibition. I was invited to participate in the Total Aktion exhibition at Museet for Samtidskunst in Roskilde, Denmark. I had the opportunity to exhibit there in 2005 as part of Get Real, a exhibition with real-time art as the focus (which was also shown at Kiasma in Helsinki, Finland). It also resulted in the book where I wrote the essay “Within the space of a moment”.


Shift became a sort of drone installation, with slow light/colour changes of volume, sometimes cut off by sharp white planes. The video documentation is a cut version showing some of the different scenes. Here is a slide show of still images.

(youtube link to the same video, if someone prefers that)

The software used is an expansion of the videoprojectiontool available here.

relief projection 2008

This easter I got the opportunity to work in a large theaterspace (BIT Teatergarasjen) to continue my work on relief projection (or masked projection).
I made 9 plywood boxes to use as my projection objects, and worked with two projectors, having a total of 16 projection surfaces.
One of the aims for this session was to also work with sound (each object would double as a speaker), and to create a depth in the placement of the objects.
I implemented my nodio system into the projection patch, which made it possible for me to create sequences of movement.
There are plenty of images from the session here.

relief projection

augmentedsculpture by pablo valbuena, originally uploaded by hc gilje.

I found the title of this post in one of Michael Naimarks essays, I guess it also could have been called augmented reality, projection of a virtual object onto a physical object, projecting a virtual layer ontop of a physical geometry, masking of projections, etc.

I have been researching different ways of projecting on other things than flat surfaces: projections that project on objects, follow the shape of the room, and projections of virtual 3D shapes onto physical 3D shapes.

In my own work I have used projections as advanced light sources, masking as a way to fit flat projections on objects and surfaces, but also to create the illusion of multiple screens from a single source. Some examples here. (Update may 2008: some more recent examples)

My goal has been to create tools which make it easy to start working with a physical space immediately, being able to make changes in realtime. I have mainly done this by using multiple opengl videoplane layers in max/msp jitter, with one of the layers having a drawing mode so you are able to draw the shape of a particular object after you have placed a opengl layer over it. I made a crude 3 layer tool for the workshop I did at KHIO this summer to enable the participants to immediately start relating to the physical space.
A prime example of multiple opengl videoplanes is Piotr Pajchels work with Verdensteatret.

I have done some experiments with projecting a 3D shape onto physical objects, but still have a long way to go in terms of having a simple setup for this.
Obviously I have been looking at what other people have been doing, but none of systems I have found seems to be available to the public, and few of them seem to have been used beyond the developing-period of the system, which might be a sign of them not being as flexible as wanted, and maybe also quite timeconsuming to prepare.

Most systems uses a method to track the shape/space they want to project onto in combination with custommade software, to be able to map the projected image correctly onto the physical object, which is related to the lens specifications of the projector, the placement of the projector in relation to the objects to be projected on, etc.

The LightTwist system developed at the University of Montreal (not much seems to have happened after 2004) use “a panoramic (catadioptric) camera to get correspondances between each projector pixel with the camera pixel. This camera represents the viewpoint of our futur observers. Then, from what the observer should see, we can build the projector images from their respective mapping.”

The videobjects from Whitevoid design in Germany is a software for realtime distortion of video to fit physical objects, but using predistorted video, and you calibrate it either with a helpgrid or by importing a model of the realworld setup. So you would need to first create the 3D shapes to project onto, and then decide how the video will map onto the 3D objects, and finally doing the calibration to match up the virtual objects with the physical ones.

I think the most spectacular callibration solution so far is the “automatic projector calibration with embedded light sensors” (pdf), a collaboration between people from Carnegie-Mellon, Mitsubishi Electric Research Lab and Stanford. They use fiberoptics and light sensors built into the objects/surfaces to be projected on, and by projecting a series of grey coded binary patterns, a custom-made software is able to adjust the image in less than a second to perfectly fit the projectionsurface, with a much higher resolution than a camerabased solution. Take a look at the impressive video:

The pdf and video seems to be from 2004, but I found some more information at Johnny Chung Lee´s website. They are hoping to make the system fast enough for live tracking of moving objects, and also to make the calibration pattern invisible using infrared light.
update: there is now more information on Lee´s website.

If you have a big budget you could always invite Circus of Now to do the video for you (”We build skyscrapers of light”).

At Ars Electronica this year I had the pleasure to see Palbo Valbuena´s Augmented Sculpture (image at top of this post) which consists of a physical structure in the corner of the room, with the exact same virtual shape projected onto it using one projector. By then animating the color and lighting of this virtual shape, some very interesting light/shadowplays happen. Valbuena collaborates with some game developers in Spain who constructed the virtual model and animation in a standard 3D software.
This work shows the potential in augmented reality using videoprojection, and I hope to see more of his work soon (He has a big outdoor installation in Madrid at the moment, hopefully there will be some documentation soon.)

update feb 5th 2008: Valbuena has updated his website with documentation of several projects: different versions of the augmented sculpture and the public square installation in Madrid.

lab jan 2007: masking projections

lab jan 2007, originally uploaded by hc gilje.

(Slideshow of more images from the projection sketches here)

The second focus in this lab session was to work with video projections, and masking them to create several projection surfaces from one projector, and to be able to relate to physical shapes in the room, like my projection on a sphere in Iball, or the masking of video to fit the gallery spaces in one of the sleepers installation:

The simplest way to do it is to work with a 2 dimensional mask. I also wanted to try to work with 3D masks: projecting a virtual 3D version of an object back onto itself. In this way I managed to cover 4 sides of a cube with 2 projectors. Another thing I did was to link the individual projection surfaces (from one projector) so focus could move from one area to another, quite like how the nodes in nodio operate.

Finally, I did some simple tests using the projector as a light beam.

 

Iball Berlin

Iball by hc gilje, originally uploaded by hc gilje.

(Slideshow of more images from Iball here)

an outdoor video installation I did in Berlin in october 2006.

The first weeks of this blog will probably be more of a backlog, covering related artists,projects, research which I have done so far.

I also want to share practical and technical information which will hopefully be of use to others as well.

Follow

Get every new post delivered to your Inbox.

Join 175 other followers