- Sound Engine
A bit of History...
Interest in keyboard instruments with dynamic and timbre control has a long history. The 15th century clavichord, although very quiet, provided dynamic control over individual notes as well as a sort of "aftertouch." By varying the amount of pressure on a key after initially striking it, the performer could produce a vibrato. The 18th and 19th century piano was loud enough to fill a recital hall, and it too provided dynamic control over the individual notes in a chord. To this day the piano remains the most popular acoustic keyboard instrument. However, the limitations of the piano action make crescendo and vibrato during sustained notes impossible.
In 1928 Maurice Martenot developed the Ondes Martenot, a monophonic instrument similar in sound to the Theremin. More information about this expressive instrument can be found here.
More recently, analog synthesizers often had ribbon controllers. These provided one-dimensional continuous control but could not track more than one finger. Some modern electronic keyboards offer key velocity and polyphonic aftertouch. These capabilities are extended by certain experimental keyboards, such as Moog's Clavier (Moog 1982), which also measures additional parameters including the exact horizontal and vertical location of the finger on each key. Several other innovative keyboard designs have been developed over the last two decades (Snell 1983, Johnstone 1985, Keislar 1987, Fortuin 1995).
Our work on the Continuum Fingerboard began in the early 1980s, with initial attempts incorporating photoelastics, conductive rubber, and capacitive techniques (Haken, Abdullah, and Smart 1992). These initial attempts were of limited success compared to our more recent approach that resulted in the current Continuum Fingerboard production models. This new approach is outlined below.
The Continuum Fingerboard
The Continuum Fingerboard, though roughly the size of a standard piano keyboard, is a continuous playing surface rather than a keyboard. The playing surface is constructed using 256 rods (128 for the half-sized), each 5.75 inches (14.6 cm) long, mounted on piano-wire springs. The rods are covered by neoprene and fabric so that the performer has the impression of a continuous surface rather than discrete keys. A magnet is mounted at both ends of each rod, and the rods are placed between two rows of Hall-effect sensors.
Hall Effect Sensors
Figure 1 shows a portion of the mechanical design of the Continuum Fingerboard's playing surface. The Hall-effect sensors are used to measure the positions of the magnets. When the performer applies finger pressure, the rods under the finger are depressed, and the magnets on those rods move closer to the sensors.
Figure 1: The Continuum Fingerboard uses Hall-effect sensors to detect the positions of magnets mounted on each rod.
Scanning software running on the Continuum Fingerboard's internal computer detects fingers by looking for any bar that has normalized sensor values greater than both of its neighboring bars. We call this the center bar, and the neighboring bars the left bar and right bar. The x, y, z position of the finger is calculated from the six sensor values on these three bars as follows.
The software tracks the front-to-back position (y position) using the normalized sensor values of the three back sensors divided by the normalized sensor values of all six sensors on the left, center, and right bars.
The exact pitch (x position) and finger pressure (z position) is computed using parabolic interpolation. A parabola is drawn through the normalized sensor values of the left, center, and right bars. The location of the minimum point of this parabola provides the x and z position of the note.
Figure 2 illustrates the use of these parabolas in detecting x variation during vibrato. In this example the center bar is always the same bar--it is always more depressed than its right and left neighbor. Still, the x position is accurately tracked as the performer rocks the finger back and forth, because the movement of the neighboring bars affects the minimum point of the parabola.
Figure 2: Exact pitch tracking using parabolic interpolation: A finger rocking left, centered, and rocking right.
Figure 3 shows fingers on the Continuum Fingerboard's playing surface, and the corresponding display on a computer screen. This computer output is used during the construction phase of the Continuum. The computer screen has a vertical line for each sensor. The length of the line corresponds to the normalized sensor value. Lines for the front sensors go down from the center of the display; lines for the back sensors go up. The display is updated as fingers move. This display information is the basis for the Midi interpretation of the performer's finger placements.
Figure 3: Fingers on the Continuum Fingerboard's playing surface, and the corresponding screen display showing normalized sensor values. The performer is using fingers 1 and 3 on the right hand and fingers 1, 2, and 4 on the left hand.