Tweet
--- article below taken from www.wired.com. full article here ---
I saw a video the other day that really stood out from the rest of the links making the rounds.
It depicts a man demonstrating software that appears to parse what he's saying fast enough to reassemble the same words by pulling and reordering bits from a recorded Michael Jackson interview. The result: Jackson appears to speak the same sentence right back to him.
The man goes on to explain how the software behind this process works, and his video closes with a live performance of the software in which a performer appears to employ the beat-box method to control the playback of audio and video on a large video screen behind him, in front of what I can only imagine must be a dazzled crowd.
If you haven't seen it yet, you can watch the video here.
It might look like a fancy parlor trick, but if the Scrambled Hackz (or "sCrAmBlEd?HaCkZ!") software does what it appears to do in the video, its ramifications extend far beyond. Sven K?nig does a fine job of explaining how the software works in the above video, but here's the general idea:
1) Scrambled Hackz analyzes the audio portion of a video file to determine the tempo of the incoming audio, and then slices it up into discrete chunks of a quarter note, eighth note, sixteenth note and so on (a process also used by audio editing programs such as Ableton Live and Sony's Acid software).
2) Using a large number of vectors, those slices are classified into a database according to their sonic characteristics.
3) When you send new audio information to the program (using, say, your voice and a microphone), it follows approximately the same process, becoming classified in the database. The software then outputs the pre-analyzed sample that is most similar to that newly cached sample.
4) The result, as you can see in the video, is that K?nig is able to reconfigure a Michael Jackson interview or any number of '80s music videos on the fly, so that they produce a sound similar to whatever he inputs. On screen, the software plays the frames of video that accompany the selected audio.
I must admit to having been enthralled by this video, because Scrambled Hackz appears to have countless possibilities. One could use the system as a virtual guitar pedal onstage, or as an audio interface to a massive library of longer samples. Producers looking for something that "sounds something like this" would be able to find exactly that in seconds. Or, live performers could use it as it's already been used (see video), as an audiovisual instrument in its own right.
Comment