Digital Puppetry Project Using Quartz Composer

posted by ian grant on February 12, 2007 at 11:11 pm | in installation, moving image, music, performance, quartz composer |

I have recently completed the first prototype of a major digital puppetry project that relies heavily on Quartz Composer in an arena of live performance familiar to vj-ers, visual artists and visualists. I have a set of predefined visuals (’scenes’) and effects and a complex mechanism that lets me composite my real-time singing mouth onto an a character that I (or someone else) can manipulate with a nintendo wii remote. Moving eyes are pre-recorded and in future versions, I indend for the eye movements to be controllable by the wii. The source code for the character control can be found elsewhere on this blog.

I do not intend to post the full patch as it is very dependent on other media, the wii controller and the Behringer midi controller, but you can view the root of the composition in the image below. I have split some of the more useful elements up and will be sharing them in other posts. I attempted (and will develop further) the idea of having a separate ‘buses’ for scenes, effects and transitions - a little like the way (I think) quartonian (and other vj-ing tools) work.

Total Qc To Wii Patch

Image (1.8mb large): Screenshot of Root of Performance Composition

The project used some of the following ideas:

  • instant chroma-keying and garbage matting of a performers blue face and the compositing of a hybrid character with live mouth and controllable pre-recorded eyes into multiple scenes
  • midi control
  • nintendo wii remote control of a virtual character (this meant I had to write a custom cocoa application to host the open source darwiinremote.framework ‘)
  • Quasi 3D (or 2-and-a-half-D) effects derived from 2D frames - I’ll post some movies of this.
  • Depth of focus and pulling focus effects
  • Dynamic real-time titles

Some Images and Commentary

A fuller walkthrough of the final images with a commentary can be found here:

Screenshots and Scene-by-Scene Descriptions Link: http://www.daisyrust.com/quartzcomposer/moocher/

Garbage Matte Demo 002

Image (above): Prototype of the Garbage Matte and Chroma-Keying Patch

Download Link: garbage_matte_bluescreen_demo_002.qtz.zip

This demo patch makes uses of Sam Kass’s excellent core image kernels available here:

Link: http://www.samkass.com/blog/

Dscf0001 Dscf0002

Images (above): Behringer BCF2000 MIDI controller controls scene sequencing and properties of various screen objects and parameters in real time

Quartz Composerscreensnapz003-1

Image: Real-time Mouth / Recorded (controllable) Eyes Composited into an Image in Real-Time

Full credit and copyright acknowledgment to the Fleischer Brothers Estate for frame grabs and stylistic inspiration.

2 comments

  1. Is the Quartz Composer patch available?
    Just been playing with the Wii and some of the Mac demos from various people, I’m keen to integrate it with some QC experiments.

    comment by Paul Bourke — March 26, 2007 #

  2. Thanks for your comment. I haven’t made the full ‘digital puppetry’ patch available yet.

    It has a few dependencies: a behringer bcf2000 (a midi controller), several large externally linked video files (amounting to over a gig of video) - so you would have to take it apart to understand it, mend it, before you could adapt it for use.

    However, the Wii to Quartz composer component of the monster ‘digital puppetry’ patch is available from the blog post below, including the full source code. Maybe that is what you are needing:

    http://www.daisyrust.com/2007/02/12/quartz-composer-some-projects/

    Direct download link (source):

    http://www.daisyrust.com/quartzcomposer/qcwii_v0.1_src.zip

    Direct download link (Application Only):

    http://www.daisyrust.com/quartzcomposer/qcwii_v0.1.zip

    Additionally:

    To find the QC composition:

    ctrl-click on the QC-Wii application ‘view package contents’ and dig down to ‘resources’ that is where the QC patch - wii_to_qc.qtz - lives. You can (carefully) edit that composition to do something different other than trigger the text display and move the teapot… Just don’t change the name of any of the root level published port ‘keys’:

    If you edit that composition, save it. Then when you re-launch the application, it will use the edited composition as it’s source.
    You don’t need to use xcode or re-build.

    The qc patch needs to run inside an application wrapper. The application handles all the blue-tooth connection wizardry provided by the wiiremote-framework, the calibration preferences etc and toggling full screen. So you can’t simply edit the composition and preview it using Quartz Composer itself and expect the WiiRemote to connect.

    Also, I’m sure the QC patch is included in the source code zip - in the xcode project directory.

    Kind Regards,

    Ian

    comment by ian grant — March 28, 2007 #

sorry, the comment form is now closed.

(cc) ian grant some rights reserved