home | current work | mechatronics | archive | dope sites
videos of wizardfingers in action.
There is of course a very long history of mechanical musical instruments including music boxes, player pianos, pipe organs, band organs, calliopes, etc. I find that one of the most important influences to this project are the carousel band organs from the last century. These electromechanical monsters were most often used to create a soothing childish music most often in the style of marches. They were mostly controlled by paper rolls much like a player piano. The standard configuration was typically a range of tuned pipes (like a pipe organ, but in wood or brass) combined with a bass drum, snare drum, and a small assortment of cymbals.
Here are some band organs:
There is still a thriving interest in these antique behemoths. Most notably is the Stinson Band Organ Company who speciallizes in building these devices even today. They have modernized the band organ by adding MIDI control and pushed the limits of size and gaudiness:
"Mighty Stinson 3000M" 75 Note Scale Orchestral Voice
This monster measures 23' wide by 12.5' high by 5' deep and weighs approximately 3,400 pounds. It even boasts 11 animated figures. No mention of how much these cost today, but you can email the sales office for more a quote.
While the Band Organ is a fascinating instrument for its scale and innovation, it is the bizarre and haunting sound of these instruments that compels me most. By some accounts they are dubbed the "happiest music on Earth", I find them to have an unnerving clunkyness and dischordant quality that makes them actually menacing and slightly sinister. They callously grind out music that played by a human musician may have some semblance of humanity (call it soul or whatnot). I find this inhuman quality to be the most fascinating aspect of these devices.
There are some current artists working with electro-acoustic instruments. The two most influential to this project are:
Trimpin: is an artist and inventor of electro acoustic instruments in Seattle.
Maywa Denki: are a pair of brothers that build amazing electro acoustic instruments in Japan.
Autonomously Generated Music:
Which brings us to the next topic: autonomously generated music. The history of this actually predates the use of computers with the algorithmic generation of music that has been employed in various ways for centuries. There are many methods from the stochastic to the precise and predictable. This is actually a rather large subject so I would like to refer you to the wikipedia article on the topic. Instead I would like to focus on exactly what I am interested in without straying too far. I am interested in creating a system that learns by example. This too is not new by any stretch. There are a variety of current projects that have been in development for years:
Neural Network music composition by prediction.
These are various implementations of creative machines for autonomous music creation.
The hardware components of the system are understood conceptually and merely require fabrication and the purchasing of appropriate parts. The software component is a bit more difficult. I have contacted Rajesh Rao in the CS department at The University of Washington to discuss the best AI algorithm for the system. He suggested a Naive Bayes Algorithm. Upon investigating the implementation of this algorithm I found an open-source software package called Orange:
"Orange is a component-based data mining software. It includes a range of preprocessing, modelling and data exploration techniques."
Orange implements a Naive Bayes Learning algorithm for processing an arbitrary size and format data set (as long as it is in a tab delimited format on input). It allows for Python scripting for any process or component in the system. So integrating it with MAX/MSP should be possible. I also spoke with Juan Pampin about what I am attempting to create and he explained more about Markov Chains, specifically that in order to get a reliable output from them they need to be provided with a very large and varied amount of learning material. So, I intend to experiment further with those as well.
I expect that when the system is completed it will behave autonomously and have the potential to self-generate musical compositions in a wide arry of styles. The system will also be well suited for collaborative improvisational performance with other musicians and dancers. I also believe that the system will likely be capable of defining it's own aesthetic and make it's own decisions, perhaps leading to a new aesthetic as defined by the system itself.
The drum set and bar chimes are struck by linear pneumatic actuators which are pressurized by pneumatic valves that are controlled by a Doepfer MTC64 MIDI-to-gate interface. The bar chimes are struck directly, but the drums use the actuators to move drum sticks attached to levers. The levers are made from door hinges and other parts found at Home Depot. The electro-mechanical drum set was designed and built by Zeb Ringer and myself for use in our band The Rogue Party. The bar chimes are a recent addition. The air organ will use solenoids to actuate each of its 87 keys. The solenoids will be controlled bt the same MTC64 that runs everything else. I use a common Midiman Midisport 2x2 MIDI interface to send control information from the laptop to the MTC64.
There is currently a need to redesign the hardware that supports the bar chime set. The current setup causes too much dampening of the chimes. Using nylon rope to suspend the chimes is a better solution that will be implemented soon. The air organ requires 87 solenoids. Solenoids cost about $20 each. I currently do not have the funding to buy the parts needed. It will have to wait. Two more drums (rack tom and floor tom) will be added to the drum set and perhaps a cymbal and tambourine. The levered strikers for these will need to be built.
The control electronics for the system consist of a Doepfer MTC-64 MIDI-to-Gate converter, a Doepfer MTC-64 power expansion board, and a Midiman 2x2 MIDI to USB interface. The power board allows for individual control of 16 separate 110V AC power connections. Each connection can control one pneumatic valve. The valves require 110-120V to switch. On the MTC64 there are four banks of 16 discrete gate outputs per bank, giving a total of 64 seperate gate signals for the whole system. This too is another limitation to building the air organ right now. I would need another controller. Of course by now I could build a controller with some Maxim LED driver chips and relays for a lot less.
The code involved in this project is written in MAX/MSP. It utilizes the Markov chain external and the jit.robosom Self-Organizing Map external written by Robin Meier as based on Kohonen's algorithm for SOMs. Training material is fed into both the Markov and the SOM. The Markov chain used is a 4th order chain and the SOM neurons consist of all notes performed in an entire bar of music, or 16 beats. In playback the Markov is used as a creative suggestion for the more structured SOM. Various other functionality is built in to provide input and control for dancers, including RFID reception via OSC messages and the wiiremote object for accelerometer input. I will upload the code here when it is finished.
The code would be better written in C++ and run as an application. I would like to give the system a running empirical memory that was maintained between instances of running and quitting the program. However, MAX/MSP does not provide this unless I implement some kind of dump function for the Self-Organizing Map object. I would rather spend time reimplementing this function as an OSX app that can send and receive a variety of impulses with the ability to adapt to changes in input and expected output. A better implementation of a neural network would greatly improve stability and output. Ideally I will rebuild all of this program using a variety of open-source resources during the summer.
The first and most important revision is to the 8 metal tubes that comprise a diatonic octave. Currently they are too small, too quiet and too musically uninteresting. I have decided to built a set of tuned pipes that are harmonically related to each other using Pythagoras' harmonic ratio series. The first pipe will be 48" in length producing the primary note. The next pipe being 3/4th the length at 36" will produce a perfect fourth of the primary note, with subsequent pipes being 2/3rd the length (32") producing a perfect fifth and the next being 1/2 the length (24") bringing us to an octave higher than the primary note. And so on for a total of 8 interrelated harmonic notes. This will produce more cohesive melodies regardless of note combinations.
The next revision will be in a better, more focused training of the neural nets. I intend to train a series of neural nets in particular styles, allowing the output to be more controlled and predictable. As well I intend to compose a "control track" for the dynamic triggering of the neural network on playback. Instead of using white noise as the trigger impulse the control track will serve to create further structure in the playback results without sacrificing the impulsive nature of the SOMs.
With dancers: The current intention for presenting the piece is as a collaborative device working with dancers in a nearly autonomous state. Ideally I would build the instrument into a vertical monolith of high volume ruckus. However, I currently have neither the money nor the means to transport the required materials to the studio. So for now it will reside on the grey plastic cart that it inhabits.
With other musicians: The other scenario I envision is improvising with live musicians and other machines. The system already has started to exhibit the ability to compose music to fit another performance. It just needs the development of a better interface, or a performer using a MIDI instrument of any kind. I believe that Wizardfingers would introduce an unpredictable improvisational component to a live situation.
LANGUAGE OF PRAISE:
Wizardfingers as a musical AI system was designed to create recognizable patterns of considerable complexity. Thereby creating music that seems to be reminiscent of a human style, but with limitless variation that extends the variety and complexity beyond the realm of what is capable for human performers. As a collaboration with dancers the intention is to create an analog connection between the dancers and an automaton. To create an experience that does not rely on an understanding of the underlying mechanisms of Wizardfingers, but to instead create an engaging spectacle. Ultimately the motivation to create this work was sheer impulsiveness, my own fascinations, and an exploration of musical structure and recomposition. The ultimate goal of this line of inquiry is to develop a system that may be trained to generate ideal pop songs to be sold to the music industry to fund other, more expensive research.
Acceptable praise is usually: dope!
week of [4/6]:
-speculate solenoids for use in air organ.
-gather necessary materials for the construction and perfection of the remaining hardware.
week of [4/13]:
-begin construction of remaining hardware.
-begin research into potential AI algorithms for machine learning and decision making.
-begin studying Python and Ruby with special attention to uses in AI.
week of [4/20]:
-continue construction, making corrections and adjustments as needed.
-continue AI algorithm research.
-continue studying most viable language.
week of [4/27]:
-should be finishing construction.
-should have a viable algorithm chosen and a plan for implementation.
-start noting useful operations and methods in chosen language.
week of [5/4]:
-draw up a system flowchart for the algorithm.
-annotating flow chart with appropriate ops and methods from language.
week of [5/11]:
-implementation of AI algorithm.
week of [5/18]:
-continued implementation, revision, and redirection.
week of [5/25]:
-testing revision and more testing.
week of [6/8]:
-final system testing and tuning.
Trimpin is an inventor of electro acoustic instruments in Seattle.
Maywa Denki are a pair of brothers that build amazing electro acoustic instruments in Japan.
Google has been a valuable source of information and advice for everything, mostly the top ten results on any topic...