… constructing the “Brain Games” music video …

CLICK HERE to download a frame grab of the master composition (36 times the size of 1080p HD). Look at it. Just LOOK AT IT.

THE INSPIRATION
“Brain Games” was inspired by the Atari 2600 game of the same name. The original game produced an incomprehensible jumble of symbols and bleeps in an effort to test the player’s memory and concentration skills. In order to shape the concept of the game into the song, I drew upon my thoughts about the scientific method and its influence over peoples thought processes over the course of modern history. LYRICS ARE HERE.

CONCEPT
Shortly after purchasing a Nintendo DSi and downloading Flipnote Studio, I decided to use the handheld console to make a music video for my album “BITS.” I thought it would be fun to use video game hardware to make a music video for an album inspired by video games. Flipnote is extremely easy to use — fast, fun, accessible … and very limited! The limited scope of the application was the first hurdle I had to work around. How could I get this thing to produce an HD music video (more on that later)?

From the start, I knew that I didn’t want to concern myself with a plot or characters. I wanted to make something abstract and “stream of consciousness.” My previous video work has been very meticulously planned out, storyboarded, pre-visualized ect. I wanted the “Brain Games” music video to be random and chaotic; to clash with the subject matter of the song (categorization, empiricism and the slow grind of accumulating knowledge using the scientific method).

I did take a relatively scientific approach to laying out the movie. I wanted each scene to be 1920 x 1080 pixels and about 6 seconds long. Given the length of the song, this gave me roughly 36 scenes.
I made a master composition in After Effects that was 6 full HD frames wide by 6 full HD frames high and 6 seconds long. Essentially, the movie only had to be 6 seconds long (played back 36 times focusing on different parts of the master composition). The viewer would see a section close up (at 1920×1080) for about 6 seconds … and then another and another. The last shot would reveal the mechanics of the whole thing.

THE SETUP
The master composition ended up being 11520 x 6480 pixels. This was very taxing on my Lenovo S10 netbook and even quite difficult for my quad-core desktop to render. At first, I envisioned that the video would take place inside a huge head. This early version was on a 5 x 7 grid.

This just started to look gross to me, so I ended up coming up with different settings for each of the 36 scenes (desert, hills, cliff, ocean, cave …). I brought back the idea of the face by placing seven TVs across the landscape to represent two eyes, a nose and a mouth.

I knew that the creatures that inhabited the video would be letters and numbers. I started drawing them … slapping eyes, teeth, little bits of hair where every I could find on their “bodies”. I had no idea what they would be doing, or how they would interact in the movie. I would drop them into the master composition and imagine what would look good with a particular character … a hot dog, dart, beach ball????

In order to keep some consistency within the composition, I scaled objects in increments of 100%. The letter/number creatures are all scaled (low quality nearest neighbor) at 200% relative to a 1920 x 1080 frame. Smaller things like the dart are at 100%. Using any high quality scaling caused all the beautiful jaggies to disappear.

In the end, my master composition in After Effects had 616 layers.

DOGMA?
I decided early on that all objects and characters would be drawn in Flipnote. Some objects run at 12fps, others at 24fps. All of them are looping animated gifs exported directly from Flipnote via the SD card on my DSi.

Here is a gallery of almost every image I created for use in this video:

Once I had established what would happen in a certain scene, I re-animated each character hopping, running or sliding around using the puppet tool in After Effects. This was often very labor intensive, frame by frame, slight nudging of pixels until the little creatures started to look even more alive. There is also a slight amount of drop shadow added to most of the characters and objects to make them pop off of the white page.

The video itself seemed to be missing an element so I decided to shoot some traditional footage of myself (using a Canon T2i) and down sample it to match the Flipnote material. I interjected this footage into the TVs to connect the music and video on a very literal level. The clips were too short on the little TVs (about 2 seconds per clip) so I added some full screen shots give the viewers a longer look.

It was never my intention to create the video entirely inside the DSi and only export it as a 256×192 Flipnote. If that were the case, I would have to import the audio by playing the song on my stereo, holding the DSi up to the speaker, and importing very low quality mono sound. The video would have been released in pieces … as an animated .gif! Sorry … no way.

My guiding principle was to use Flipnote, and have fun. I wanted to see what I could do with the DSi as the platform, but not limit myself to short, animated gif output it created. I worked while on the bus, on my lunch breaks, at midnight … whenever and wherever. The portability of the DSi/Flipnote was awesome.

OUTPUT
The original composition took about 10 hours for my computer to render down to 1920 x 1080 @ 24fps. OUCH!

I would love to see the main 6 second composition project at full resolution someday. That would be insane. I have uploaded “Brain Games” to youtube and vimeo. Strangely enough, youtube’s compression deals with the peculiarities of the video much better than vimeo’s. Unfortunately, neither of them are capable of showing the true pixelated goodness of the live action shots of me playing along with the songs. :(
It gets a little blurry and fuzzy even at 1080p.

This whole thing took me around nine months to put together. I’m happy to be finished. I think I’ll make a live action video next ;).