Before we started working together, Justin had wanted to do a project relating to physical interface, and Alex had wanted to do a project related to music. So we combined our ideas and made a ‘mashup machine’. The goal was to let users play around with making music without needing any musical talent.
After getting an idea of what we were actually going to make, we decided to build the enclosure before starting the wiring. We picked up some wood, laser cut the pieces and holes that would house buttons and knobs, and then began wiring from there once we picked up the components we needed. Radio shack had a large selection of buttons and so we picked up four that had built in resistors so that we could reduce the amount of in box wiring we needed to do. We also purchased four knobs and potentiometers as well to go with the four tracks of each song that we were going to include. The buttons were to control what track was being changed; guitar, drums, bass, vocals. The knobs were to control the volume of each of these tracks.
Once we got all of those things out of the way, we could begin wiring. The Arduino was placed in first to get an idea of where it needed to be and how much room was needed in the rest of the enclosure. Then copper strips were placed on the back of the box, one connected to power and the other connected to ground. Each knob and button had two wires soldered to either of these, and then one extra connecting to the corresponding pin/analog output it was programmed for.
After we managed to get the wires connected in a way that allowed us to attach as many components with fewer wires than necessary, it all became about making sure everything would fit inside the enclosure once the wooden sides were connected to each other and the knobs and buttons were in the rightful place on the box.
We used the Firmata protocol to interface between the Arduino and the laptop, which handled the actual music mixing. Firmata sends the state of the Arduino inputs via serial communication to the computer. We then used a program called Pure Data, an open source visual programming language (http://puredata.info/), to do all of the work with the audio files. The Pure Data component of the project consisted of two ‘patches’ (the Pure Data term essentially for programs); one written by Hans-Christoph Steiner called pduino which intercepted the serial data from Arduino/Firmata and translated it into data that could be used directly in Pure Data, and another patch written by us to load and control the music tracks. Pure Data allows patches to communicate with each other as long as they are open simultaneously. So by adding send objects to the pduino patch for each of the knobs and buttons on the device, we could receive those signals into the patch that we created.
The mixing patch consists of four more or less identical segments, one for each instrument track. At the top (the mixing patch can be read basically from top to bottom…following the connecting lines) of each segment is a counter that increments every time the button of the related track is pushed, returning to zero when the max number of tracks is reached. Actually, it turned out that the counter incremented when the button was pressed and when it was released, making it skip a track, so an additional counter simply going from 0 to 1 and only letting the 1s through was added before the main counter. The counter then feeds into a ‘makefilename’ object that creates a string containing [instrument][X].wav, where [instrument] is drums, bass, guitar, or vocals depending on the segment, and [X] is the value of the counter. This information is then passed to the ‘open’ and ‘readsf’ objects, which actually open the correct file and start reading it. Whenever the file comes to the end, a ‘bang’ signal is sent back to an earlier part of the patch (after the counter) to open the same file again. This way the tracks all loop indefinitely. From this point the data goes to volume control, which is reading its value from the knobs, via the Arduino and the pduino patch. Volume display bars are attached just before and after the volume control object for debugging purposes. Finally it gets sent to the ‘dac’ object (digital to analog converter) which actually plays the files.
One of the key steps in the project was finding and formatting the audio tracks. In order to allow all the files to be mixed together without sounding completely awful, we needed them to be in the same key and tempo. Restricting ourselves to freely available samples that are in the same key and tempo would have drastically limited the number of tracks we could have included. Both tempo and key can be altered, but changing both for the number of files we wanted to use would have taken ages. Instead, we compromised by only using samples that were already in the key of E, but altering the tempos as necessary. This was still time consuming, but changing the tempo is faster and more reliable than changing the key, and we were able to accumulate a decent number of tracks. Then each of the tracks had to be named in the [instrument][X].wav format.
At the point of hooking the box up to the computer and testing the signals, we were getting a lot of weird results. The knobs worked correctly (after a bit of quick resoldering), but the buttons were much too sensitive. They would fire off and on rapidly just from moving your hand near them, without even touching them. After some research it was determined that we needed pull-up resistors on all of the buttons. Normally the Arduino has its own built in pull-up resistors to prevent this problem, but Firmata disables it. There may be software solutions, but the easiest fix for us was to add some physical pull-up resistors to each of the buttons.
Finished Machine in action (Pardon the background noise)
The next step to improve this project would be to modify the pure data patch to make sure that when the tracks are changed they stay in time with the rest of the music. Currently, whenever the track is changed, it begins playing the new track immediately from its beginning, regardless of where the rest of the other tracks are. This means that, although everything is playing at the same tempo, they can be a bit out of sync unless the user manually times their button presses to the beat of the music. I think the easiest way to fix this would be to use Pure Data’s metronome object to keep internal track of the song’s beat, and delay new tracks until the start of a measure. The downside to this approach would be that there would be a delay between the user pressing a button and any change taking place, which might negatively impact the user experience. This might be combated by adding some kind of display that indicates that new track is getting ready to play (lights flashing to the beat or something). Or a different approach might avoid this problem.
The other way that we would improve this project moving forward is to add some labeling or icons to the box itself, indicating what the buttons and knobs do. Currently they have to be explained verbally (or figured out through trial and error).