Electronic music controller using inertial navigation-2

Abstract

A percussion controller comprises an instrumented striker including devices for obtaining inertial measurements and a wireless transmitter, a sensor-enabled striking surface that receives an impact from the instrumented striker, and a data processing system that receives the inertial measurements and predicts at least one of the force or location of impact of the instrumented striker on the sensor-enabled striking surface before impact actually occurs.

Claims

1. A percussion controller comprising: an instrumented striker; and a data processing system, wherein the data processing system: a) generates a plurality of virtual impact zones, wherein each zone corresponds to a different musical event; b) receives first signals that convey information pertaining to movement of the instrumented striker; c) generates a location prediction and a force prediction based on information conveyed by the first signals, wherein: (i) the location prediction predicts a location of intersection of the instrumented striker and one of the virtual impact zones, (ii) the force prediction predicts a force with which the instrumented striker would strike the location of intersection if the virtual impact zone were physically manifested; d) relates the location of intersection to a musical event; and e) generates a musical event message based on the musical event.

2. The percussion controller of claim 1 and further wherein the location prediction is based, at least in part, on inertial navigation computations.

3. The percussion controller of claim 1 and further comprising a striking surface for striking with the instrumented striker, wherein the striking surface does not include any sensors.

4. The percussion controller of claim 3 and further wherein the data processing system maps at least some of the plurality of virtual impact zones to locations on the striking surface, thereby defining physical impact zones on the striking surface, wherein each physical impact zone corresponds to the musical event associated with virtual impact zone that defined the physical impact zone.

5. The percussion controller of claim 4 wherein the striking surface comprises a resilient surface.

6. The percussion controller of claim 5 and further wherein the striking surface comprises a plurality of lights, wherein the data processing system is operable to selectively illuminate some of the lights to demarcate the physical impact zones.

7. The percussion controller of claim 4 and further comprising an auxiliary instrumented mat that generates second signals, wherein the data processing system uses the second signals to perform at least one of the following tasks: (i) initialize inertial navigation computations, and (ii) provide on-going corrections to inertial navigation computations.

8. The percussion controller of claim 1 and further comprising a sensor-enabled striking surface including a resilient surface for striking with the instrumented striker and a plurality of sensors disposed beneath the sensor-enabled striking surface, wherein the data processing system: (f) receives second signals that convey information pertaining to the movement of the instrumented striker toward the sensor-enabled striking surface; (g) predicts, based on the information conveyed by the second signals, at least one of: (i) a force of impact of the instrumented striker on the sensor-enabled striking surface, and (ii) a location at which the instrumented striker will impact the sensor-enabled striking surface; (h) relates the location of impact to a musical event; and (i) generates a musical event message based on the musical event.

9. The percussion controller of claim 8 and further comprising an instrumented mat that controls one or more attributes of the sensor-enabled striking surface.

10. The percussion controller of claim 9 wherein striking the instrumented mat at a first location changes the musical event that corresponds to a first location on the sensor-enabled striking surface.

11. The percussion controller of claim 9 wherein striking the instrumented mat at a first location changes an instrument that the sensor-enabled striking surface simulates in conjunction with the data processing system.

12. The percussion controller of claim 10 wherein striking the instrumented mat a second location changes an instrument that the sensor-enabled striking surface simulates in conjunction with the data processing system.

13. The percussion controller of claim 9 wherein the sensor-enabled striking surface simulates a first instrument and the instrument mat simulates a second instrument.

14. The percussion controller of claim 8 and further comprising a foot switch, wherein the foot switch controls one or more attributes of the sensor-enabled striking surface.

15. The percussion controller of claim 1 and further wherein the data processing system alters a number of virtual impact zones in the plurality thereof.

16. The percussion controller of claim 15 and further wherein the data processing system increases the number of virtual impact zones, wherein additional virtual impact zones correspond to additional musical events.

17. The percussion controller of claim 1 and further wherein the data processing system changes the musical events that correspond to particular virtual impact zones.

18. The percussion controller of claim 1 wherein at least one of the virtual impact zones correspond to a cymbal.

19. The percussion controller of claim 1 and further wherein the data processing system: (f) compares the movement of the instrumented striker, as conveyed by the information in the first signals, to predetermined striker motion patterns that correspond to musical events; (g) characterizes the movement of the instrumented striker as a non-throwing motion when the striker's movement matches one of the predefined striker motion patterns; and (h) generates a second signal that conveys second information about the musical event corresponding to the matched predefined striker motion pattern.

20. The percussion controller of claim 1 and further wherein the data processing system stores information related to acceleration and position of the instrumented striker, wherein the information is indicative of a user's striker-throwing technique.

21. The percussion controller of claim 20 wherein the data processing system: generates a visual representation of the user's striker-throwing technique from the information indicative thereof; and displays the visual representation for viewing.

22. The percussion controller of claim 20 and further wherein the data processing system assesses the user's striker-throwing technique.

23. The percussion controller of claim 22 wherein the data processing system assesses the user's striker-throwing technique by comparing the information indicative of the user's striker-throwing technique to reference information pertaining to throwing technique.

24. The percussion controller of claim 23 wherein the reference information comprises a prerecorded reference performance.

25. A method comprising: predicting a location of intersection of an instrumented striker with a virtual impact zone based on signals received from the instrumented striker; predicting, based on the signals received from the instrumented striker, a force with which the instrumented striker would strike the virtual impact zone if the virtual impact zone were physically manifested; relating the location of intersection with a musical event; generating a first signal that conveys first information about the musical event; and transmitting the first signal to a device that generates a second signal that can be converted to sound that is related to the musical event.

26. The method of claim 25 and further comprising mapping the virtual impact zone onto a striking surface.

27. The method of claim 25 and further comprising: mapping predefined motion patterns to musical events; comparing motion of the instrumented striker to the predefined motion patterns; when the motion matches one of the predefined motion patterns, generating a third signal that conveys second information about the corresponding musical event; and transmitting the third signal to the device for generating signals that can be converted to a sound that is related to the corresponding musical event.

28. The method of claim 25 and further comprising storing information related to acceleration and position of the instrumented striker, wherein the information is indicative of a user's striker-throwing technique.

29. The method of claim 28 and further comprising assessing the user's striker-throwing technique.

30. The method of claim 29 wherein assessing the user's striker-throwing technique further comprises comparing the information indicative of the user's striker-throwing technique to reference information pertaining to throwing technique.

31. The method of claim 30 wherein the reference information comprises a prerecorded reference performance.

32. The method of claim 29 wherein assessing the user's throwing technique further comprises: generating a visual representation of the user's technique from the information indicative thereof; and displaying the visual representation for viewing.

33. The method of claim 25 and further comprising: generating, at the third device, the signals that can be converted to the sound that is related to the corresponding musical event; and generating the sound.

34. A method comprising: monitoring motion of a striker; predicting, using information obtained from the monitoring, at least one of a location or a force as follows: (a) a location at which the striker will impact a striking surface, (b) a location at which the striker will intersect a virtual impact zone, (c) a force with which the striker will impact the striking surface at the location, or (d) a force with which the striker would impact the virtual impact zone at the location of intersection, if the virtual impact zone were physically manifested; generating a visual representation of the monitored motion; displaying the visual representation for viewing; and generating a musical event message from the at least one predicted location or force.

35. The method of claim 34 and further comprising assessing a throwing technique of a user that is using the striker.

36. The method of claim 35 and wherein the throwing technique being assessed is selected from the group consisting of a wrist pivot, whether grip is slipping, whether the striker is being rolled, whether a pre-impact release of throwing force occurs, single stroke throw about wrist axis and bounce, and double stroke throw and bounce.

37. The method of claim 35 and further wherein assessing the throwing technique comprises comparing the throwing technique to reference information pertaining to striker throwing technique.

38. The method of claim 35 wherein predicting at least one of a location or a force is based, at least in part, on inertial navigation computations and further wherein assessing the throwing technique is based on at least some of the intertial navigation computations.

39. A percussion controller comprising: an instrumented striker; a resilient striking surface for striking with the instrumented striker, wherein the striking surface does not include any sensors; and a data processing system, wherein the data processing system: (a) receives first signals that convey information pertaining to kinetics of the instrumented striker; and (b) processes the first signals using inertial navigation techniques to predict at least one of: (i) a future location of the instrumented striker; (ii) a force with which the instrumented striker will impact a surface at the future location; (iii) a force with which the instrumented striker would impact a virtual impact zone at the future location, if the virtual impact zone were physically manifested.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) FIG. 1 depicts a first percussion controller in the prior art.

(2) FIG. 2 depicts a second percussion controller in the prior art.

(3) FIG. 3 depicts a third percussion controller in the prior art.

(4) FIG. 4a depicts percussion controller 400 in accordance with the illustrative embodiment of the present invention.

(5) FIG. 4b depicts a charging cradle for charging a rechargeable energy source within the instrumented strikers of percussion controller 400.

(6) FIG. 5 depicts an instrumented striker of percussion controller 400.

(7) FIG. 6a depicts a top view of a first embodiment of a sensor-enabled striking surface of percussion controller 400.

(8) FIG. 6b depicts a side view of the sensor-enabled striking surface of FIG. 6a.

(9) FIG. 6c depicts a top view of a second embodiment of a sensor-enabled striking surface of percussion controller 400.

(10) FIG. 7a depicts a top view of the sensor-enabled striking surface of FIG. 6c wherein lights for identifying impact zones are shown.

(11) FIGS. 7b-7d depict a top view of the sensor-enabled striking surface of FIG. 7a wherein different groups of lights are illuminated to identify different arrangements and sizes of impact zones.

(12) FIG. 8 depicts a block diagram of the salient components of an illustrative hardware platform for the data processing system of percussion controller 400.

(13) FIG. 9 depicts specialized software applications that are maintained in the data processing system's processor-accessible storage and used by the data processing system to perform the method depicted in FIG. 11.

(14) FIG. 10 depicts reference information that is maintained in data processing system's processor-accessible storage and used by the specialized software applications to perform required processing.

(15) FIG. 11 depicts a block diagram of a method in accordance with the illustrative embodiment of the present invention.

(16) FIG. 12a depicts a high level system sequence in accordance with the illustrative embodiment of the present invention.

(17) FIG. 12b depicts a high level processing sequence for use in conjunction with the illustrative embodiment of the present invention.

(18) FIG. 12c depicts a high level sequence of the instrumented striker.

(19) FIG. 13 depicts a block flow diagram of a method for scanning the sensor-enabled striking surface.

(20) FIG. 14 depicts a throw as a sequence of instrumented striker positions and predicted locations in relationship to the sensor-enabled striking surface and its Surface Frame, resulting in a predicted impact time and location.

(21) FIG. 15 depicts forces experienced by instrumented striker 402 during a throw.

(22) FIG. 16 depicts a sequence of instrumented striker positions and the shift of rotation during a throw.

(23) FIG. 17 depicts a sequence of instrumented striker positions and the shift of rotation during a rudimental bounce.

(24) FIG. 18 depicts the space volume boundaries of the instrumented striker during performance.

(25) FIG. 19 depicts the relationship of the sensed magnetic flux to the sensed gravity field, and resolving pitch, roll and yaw of the instrumented striker.

(26) FIG. 20 depicts the optional addition of a permanent magnet to the sensor-enabled striking surface.

DETAILED DESCRIPTION

(27) Although presented in the specific context of a percussion controller, the teachings of the present invention can be adapted to other applications, for example, and without limitation, to other human/computer interfaces such as touch panels, plasma panels, switch panels, computer keyboards, control panels, sound-mixing controls, or stage-lighting controls.

Definitions

(28) The terms appearing below are defined for use in this disclosure and the appended claims as follows: “Impact” means any physical contact, regardless of the severity thereof, between, for example, the instrumented striker and the sensor-enabled striking surface. Thus, a forceful “whack” as well as the gentle pressure of brushing movement are both “impacts.” “Instrumented mat” means a mat that is capable of controlling the sensor-enabled striking surface. For example, striking the instrumented mat at a first location can change the layout of a particular instrument simulated by the sensor-enabled striking surface and striking the instrumented mat at a second location can change the instrument that is simulated by the sensor-enabled striking surface. “Instrumented Striker” means a striker that includes devices/sensors that enable its kinetics to be determined for use, for example, with IN processing. In alternative embodiments in which striker force and position are determined based on measurements obtained through EM interrogation, the striker might not contain any sensors, etc. In such embodiments, “tags” that provide a reflective surface at the wavelength of the interrogating radiation can be present on the external surface of the striker. Such a “tagged” striker is considered to be an “instrumented striker,” as that term is used herein. The term “instrumented striker” collectively references a stick, mallet, beater, glove, etc. “Inter-network” means the wireless or wired communication network between the devices external to percussion controller and the percussion controller's processor, such as synthesizer(s), computer(s), other music controllers, and other percussion controllers. “Intra-network” means the wireless and wired communication network of the percussion controller's “edge” devices: foot switches, trigger sensors, sensor-enabled striking surface, instrumented mat(s), processor, strikers, cradle(s), and indicator panel(s). “MIDI” means “Musical Instrument Digital Interface,” which is an electronic musical instrument industry specification that enables a wide variety of digital musical instruments, computers and other related devices to connect and communicate with one another. MIDI equipment captures note events and adjustments to controls such as knobs and buttons, encodes them as digital messages (“musical event messages”), and transmits these messages to other devices where they control sound generation and other features. “Musical event” means something related to a musical performance, such as, a sound reproduced by a particular instrument, a musical note, tempo, pitch, volume (i.e., amplitude), and the like. “Sensor-enabled striking surface” means a layer of material having an upper surface that is intended to be struck by a striker. The layer of material, or at least a portion of it, is configured to provide a rebound or bounce when struck by the striker. That is, the material is elastic or resilient, or otherwise configured to provide such resilience. Sensors that are capable of sensing the impact or touch pressure of the striker on the upper surface are disposed beneath the upper surface. The sensors can be either within the layer of material or directly beneath it. “Striker” means an object that a performer strikes/touches to the sensor-enabled striking surface. The term “striker” collectively references a drum stick, a mallet, a beater, a gloved hand, etc.

(29) FIG. 4a depicts percussion controller 400 in accordance with the illustrative embodiment of the present invention. Percussion controller 400 includes instrumented strikers 402, sensor-enabled striking surface 404, data processing system 406, and striker cradle 408. Also depicted in FIG. 4a as part of percussion controller 400 are optional instrumented mat(s) 412, indicator panel 414, and foot pedal(s)/switch(es) 418. Percussion controller 400 is depicted in use with several devices that are not part of the percussion controller; that is, synthesizer 420, amplifier 422, and speaker(s) 424.

(30) In the illustrative embodiment, information about the kinetics of the instrumented striker 402 is obtained via inertial sensing from on-striker devices. That information is wirelessly transmitted, via wireless communications link 401, to data processing system 406. Applying Inertial Navigation techniques, the data processing system uses the inertial measurements to predict the force with which instrumented striker 402 will impact sensor-enabled striking surface 404. In some embodiments, such information is also used to predict the location that instrumented striker 402 will impact sensor-enabled striking surface 404. Instrumented striker 402 is described in more detail in conjunction with FIG. 5, sensor-enabled striking surface 404 is described in more detail in conjunction with FIGS. 6a-c and 7a-d, and data processing system 406 is described in more detail in conjunction with FIGS. 8-10.

(31) After mapping the predictions to virtual impact zones of sensor-enabled striking surface 404, data processing system 406 generates musical event messages, which are conveyed by signals 413 to music synthesizer 420. The musical event messages control synthesizer 420 in known fashion, causing it generate music signals 415 that are transmitted to amplifier 422 for amplification. The amplified music signals 417 are then transmitted to speakers 424, to actually generate the desired sounds; that is, the musical performance.

(32) Instrumented strikers 402 that are not in use (“cold”) reside in charging cradle 408. The cradle is operable to recharge a rechargeable energy source within each cold instrumented striker 402. In the illustrative embodiments, charging is performed inductively. In some embodiments, charging cradle 408 includes plural indicators 410, as shown in FIG. 4B, that provide an indication of the state of charge of instrumented strikers 402. Indicators 410 can be lights, wherein the state of the light (i.e., on or off) indicate charge. Alternatively, three lights each of different color, such as “red” (for depleted), “orange” (for partially charged), and “green” (for fully charged), can be used to indicate the charge state for each instrumented striker.

(33) To facilitate recharge, charging cradle 408 senses, via appropriate circuitry/sensors, the presence of an instrumented striker 402 before charging. The cradle transmits signals to data processing system 406 over communications link 405. The signals convey information pertaining to the presence and state of charge of any instrumented strikers within charging cradle 408. In the illustrative embodiment, communications link 405 is wired; in some other embodiments, this link is wireless. As discussed later in conjunction with FIG. 5, instrumented strikers include a coil (e.g., coil 536) in the tip thereof for inductive charging.

(34) Indicator panel 414 includes indicators 416 (e.g., lights, etc.) that provide an indication of the state of charge of the instrumented strikers that are currently in use (“hot”) by the performer. The state of charge of hot instrumented strikers is tracked by data processing system 406. The state of charge can be estimated by time-in-use or hot instrumented strikers can transmit the state of charge to data processing system 406. The data processing system transmits, via communications link 409, a signal to indicator panel 414 that conveys the status of the hot instrumented strikers. Indicator panel 414 can also provide an indication of the status of other elements of percussion controller 400.

(35) Optional instrumented pad 412 is used, in some embodiments, to supplement the capability of sensor-enabled striking surface 404. Instrumented pad 412 is a simply a smaller version of the sensor-enabled striking surface. Instrumented pad 412 communicates with data processing system 406 over wired communications link 407.

(36) In the illustrative embodiment, percussion controller 400 includes one more foot switch(es) 418b that control some aspects of the operation of sensor-enabled striking surface 404 and/or instrumented pad 412. For example, foot switch 418b can be used to change the layout of a particular instrument being simulated by sensor-enabled striking surface 404 (e.g., change the location of drums, etc. within a “virtual” trap set, etc.) by simply choosing from among several pre-programmed arrangements. For example, a first “click” on the switch provides a first layout and the second “click” on the switch provides a second layout. Or foot switch 418b can be use to change the instrument being simulated by the sensor-enabled striking surface. Again, it is simply a matter of “clicking” between pre-programmed selections. Foot switch 418b communicates with data processing system 406 over wired communications link 411b.

(37) Additional capability can be provided to the system via external pedal(s) 418a. Such pedals, which are conventional for electronic percussion systems, can, for example, actuate a virtual bass drum, etc. Pedal(s) 418a communicates with data processing system 406 over wireless communications link 411a. After reading the present disclosure, those skilled in the art will know how to integrate and use external pedal(s) 418a and foot switch(es) 418b with percussion controller 400.

(38) Instrumented Striker 402.

(39) Referring now to FIG. 5, instrumented striker 402 in accordance with the illustrative embodiment of the present invention comprises inductive coil 536, two 3-axis accelerometers 538 and 548, antenna 540, 3-axis digital compass 542, rechargeable energy source 544, and low power transmitter and logic circuits 546.

(40) In the illustrative embodiment, instrumented striker 402 is about the same size as a conventional striker. For example, a 5B standard drum stick is 16 inches in length and 7/16 inches in diameter. The location of the center-of-gravity should be about the same for both instrumented striker 402 and a conventional striker.

(41) In the illustrative embodiment, instrumented striker 402 comprises three sections: tip/taper section 530, shank 532, and butt 534. The diameter of each section near the interface to the adjacent section is appropriate for sliding one into the other and then bonding the adjacent sections together. As depicted in FIG. 5, coil 536 is disposed in the tip of tip/taper section 530. Accelerometer 538, antenna 540, and digital compass 542 are disposed in the taper of tip/taper section 530. Rechargeable energy source 544 is disposed in shank 532, and transmitter and logic circuits 546 and accelerometer 548 are disposed in butt 534.

(42) It will be appreciated that sections 530, 532, and 534 must be hollow or include hollowed-out regions to receive the various components. If any of the sections are hollow, after the components are positioned therein, fill is provided to prevent components from moving and to achieve the proper weight and weight distribution for striker.

(43) For inertial measurements, instrumented striker 402 includes at least one 3-axis accelerometer and at least one angular acceleration sensor (“AAS”). Accelerometer 538 measures acceleration of the striker's reference frame along each of three orthogonal axes: up/down, left/right, forward/back.

(44) Accelerometers do not resolve all the forces present on the three axes (i.e., throwing force, gravity, and angular acceleration [centripetal] forces). Another measurement device, such as an AAS, is required so that angular acceleration forces acting on the striker can be resolved, leaving gravity and the throwing forces combined. Using the fixed rotation, measured at initialization, between the Earth's magnetic field and the gravity field, local gravity can be accurately resolved, such that the throwing forces on instrumented striker 402 can be isolated. In the illustrative embodiment, the AAS is 3-axis digital compass 542.

(45) 3-axis digital compass 542 measures the attitude of the instrumented striker frame with respect to the Earth's magnetic field. This information is used, in the illustrative embodiment, to provide angular accelerations for roll, yaw, and pitch about the instrumented striker's frame axes and provides a reference to accurately calculate the direction of Earth's gravity field. As an alternative to digital compass 542, a 3-axis gyroscope can be used. Due to the concerns as to the affect of repeated forceful impacts of instrumented striker 402 on sensor-enabled striking surface 404, digital compasses are currently preferred over gyroscopes.

(46) A second 3-axis accelerometer 548 is used to decrease measurement errors, thereby improving the accuracy of calculations based on the measurements obtained from these devices. Alternatively, a second AAS device (e.g., 3-axis digital compass) could be used.

(47) In some alternative, but less preferred embodiments, the kinetics of the striker is determined by interrogating the striker with electromagnetic energy (“EM”). For example, in some embodiments, a high speed camera is used to track the movements of the strikers during a performance. The images from the camera are then processed and, using IN, the force and/or location of a strike is predicted. In additional embodiments, very high frequency (e.g., K.sub.u band, etc.) radio can be used to interrogate the strikers. The energy is projected at the striker's tip and butt locations and, for example, the Doppler shift is measured at multiple sensors (a minimum of three) and processed in known fashion (e.g., triangulation, etc.) to obtain striker velocities and derive the striker positions, etc., either augmenting or replacing the IN processing. The location of the EM emitters is important so that the percussionist does not obstruct the emissions. In conjunction with the present disclosure, those skilled in the art will be able to make and use such alternative embodiments of the invention.

(48) Information pertaining to the kinetics of instrumented strikers 402 must be transmitted to the data processing system without interfering with percussion performance techniques. To that end, in the illustrative embodiment, instrumented striker 402 includes wireless transmitter/logic circuits 546 and compact antenna 540 for transmitting the measurements obtained by accelerometers 538 and 548 and digital compass 542 to data processing system 406. The logic circuits implement link-layer logic and the conventional wireless physical link.

(49) Power is required to operate transmitter and logic circuits 546. To that end, instrumented striker 402 includes rechargeable energy source 544. In the illustrative embodiment, the rechargeable energy source is a capacitor (e.g., super capacitor, etc).

(50) Rechargeable energy source 544 must be routinely recharged. In the illustrative embodiment, metal coil 536 is disposed within the tip of instrumented striker 402 to facilitate inductive charging of rechargeable energy source 544 in charging cradle 408. Coil 536 is electrically coupled (not depicted) to rechargeable energy source 544.

(51) In some other embodiments, instrumented striker 402 includes an energy-harvester, such as a piezoelectric crystal, etc., which charges the rechargeable energy source. The energy harvester captures energy, such as the energy released as the instrumented striker impacts sensor-enabled striking surface 404 and uses that energy to power the on-striker electronics. In such embodiments, the resiliency/elasticity of the resilient surface of sensor-enabled striking surface 404 is appropriately tailored so that a desired amount of the energy available from the strike is absorbed by deflection of the mat leaving a suitable amount of energy available for harvesting.

(52) Although not depicted, some embodiments of percussion controller 400 include an instrumented glove (e.g., to be worn on the hands for hand percussion, etc.). The instrumented glove includes: (i) two or six accelerometers (one for each finger and one redundant); (ii) one or five 3-axis digital compasses (one for each finger); (iii) a replaceable energy source (e.g., a battery); (iv) a low-power transmitter and matched compact antenna; and (v) circuits to implement a link-layer logic and the conventional wireless physical link.

(53) The Sensor-Enabled Striking Surface 404.

(54) FIGS. 6a and 6b depict, via top and side views, a first embodiment of sensor-enabled striking surface 404. In this embodiment, the sensor-enabled striking surface has a round shape, like a drum head. In some other embodiments, such as one shown in FIG. 6c, sensor-enabled striking surface 404 has a rectangular shape. The sensor-enabled striking surface can have any of a variety of forms as convenient.

(55) Referring again to FIGS. 6a and 6b, sensor-enabled striking surface 404 comprises resilient striking surface 650, sensor mesh 652, and light mesh 654, arranged as depicted.

(56) Resilient striking surface 650 provides a “rebound” upon striker impact, thereby mimicking the rebound response of an actual acoustic percussive instrument (e.g., drum heads, etc.).

(57) Mesh of individually-addressable contact (force/pressure) sensors 652 underlies resilient striking surface 650. The contact sensors can be strain gauges, load cells, or the like, such as commercially available from Tekscan, Inc. of Boston, Mass. Sensor mesh spacing is typically less than about 2 centimeters, and more preferably less than about 1 centimeter. The smaller the spacing between sensors, the greater number of zones can be established on the striking surface.

(58) Mesh of individually-addressable lights 654 underlies sensor mesh 652. The lights are positioned in the space between adjacent sensors. The use of the lights is discussed later in conjunction with FIGS. 7A through 7D.

(59) Although not directly used for force and/or location determination of a strike, sensor-enabled striking surface 404 provides certain important functionality. In particular, sensor mesh 652 is used for at least the following purposes: Initialization for IN calculations; IN error correction; and Verification of striker impact (i.e., force and/or predicted impact location).

(60) As will be appreciated by those skilled in the art, IN needs to be initialized before it is used and requires ongoing error corrections. In accordance with the illustrative embodiment of the present invention, initialization and navigation error correction are accomplished by striking sensor-enabled striking surface 404. Data processing system 406 keeps track of each striker's state of initialization and the estimated error, and every strike or touch on the sensor-enabled striking surface can be used to fix the navigation solution.

(61) As discussed further below, to relate the (predicted) location of a strike of instrumented striker 402 to a musical event, sensor-enabled striking surface 404 is “virtually” segregated into a plurality of impact zones via data processing system 406. More particularly, the data processing system “virtually” segregates sensor mesh 652 into impact zones. Each such impact zone typically represents a different musical event. Prior to a first performance, a user programs, in conjunction with data processing system 406, a variety of impact zone arrangements. The arrangements are stored in data processing system 406. A desired arrangement is recalled by the performer before a performance.

(62) In the illustrative embodiment, data processing system 406 selectively activates lights within the mesh thereof to display the boundaries of the impact zones for the performer. FIG. 7a depicts a top view of sensor-enabled striking surface 404 showing (un-lit) lights 654. FIGS. 7b through 7d depict arrangements of impact zones of increasing complexity. The layout of each arrangement is revealed by activated lights 754.

(63) FIG. 7b depicts an arrangement having four impact zones, 755a through 755d. FIG. 7c depicts an arrangement having six impact zones, 757a through 757f. And FIG. 7d depicts an arrangement having twenty-four impact zones. The various impact zones can map to different instruments, or different regions on an instrument, or both.

(64) Sensor-enabled striking surface 404 will typically have dimensions of 14 inches×32.5 inches, 25 inches×32 inches, or 25 inches×39 inches, although other sizes are acceptable. A master percussionist can reliably strike within a square region that is about 1% on a side. With a sensor-enabled striking surface 404 having dimensions of 25 inches×32 inches, 252 impact zones can be created.

(65) The location and force predictions of the “strike” will be issued a few milliseconds before actual impact on sensor-enabled striking surface 404. As a consequence, prediction accuracy will be very high, but there remains the possibility of extremely infrequent prediction errors. In such cases, at the time of impact, data processing system 406 might determine that there was a prediction error wherein:

(66) (1) Synthesizer 420 begins to generate the wrong note; or

(67) (2) Synthesizer 420 begins to generate the right note but with the incorrect force.

(68) The solution to scenario “2” is to do nothing. “MIDI” velocity is used to convey “force” (at 127 different energy levels) and most force errors will be very small and barely noticeable in the generated sound. Scenario “1” represents the more significant error. The “note” error must be corrected; an uncorrected note will detract from the musical performance. The processor will issue a “note-off” command to the synthesizer for the wrong note. This is followed by a “note-on” command for the correct note. The result of this will be a barely perceptible, several-millisecond “click” sound (due to the incorrect note) followed by the sounding of the correct note.

(69) It is notable that IN error reduction is well established; many conventional techniques are known and applicable to achieve one-in-a-million occurrences of error. Two textbooks that are particularly useful to an understanding of the IN algorithms, causes of IN error and rates of occurrence, and IN error correction techniques are: Britting, Kenneth R. “Inertial Navigation Systems Analysis” (ISBN-13 978-1-60807-078-7) and Bekir, Esmat “Introduction to Modern Navigation Systems” (ISBN-13 978-981-270-765-9).

(70) The dependence of the predictive aspects of the present invention on making very accurate IN predictions is the reason why it is preferable to use two accelerometers, rather than one, in a stick/mallet/beater and up to six accelerometers, rather than five (one for each finger) in a glove. The extra accelerometer provides information critical to reducing errors.

(71) In some alternative embodiments, the striking surface is not sensor-enabled; it is simply a resilient striking pad. In such embodiments, an auxiliary instrumented pad is used to provide the initialization and updating functions. Since the percussionist would have to occasionally strike the auxiliary instrumented pad during a performance, such embodiments are less desirable than the illustrative embodiment in which the striking surface is instrumented. Furthermore, in such embodiments, the percussion controller will not be able to correct prediction errors.

(72) Data Processing System 406.

(73) FIG. 8 depicts a block diagram of the salient components of an illustrative hardware platform for implementing data processing system 406. In the embodiment depicted in FIG. 8, data processing system 406 comprises transceiver 856A and 8556B, processor 858, and processor-accessible storage 860, interrelated as shown.

(74) Transceiver 856A is a wireless transceiver (including antenna, not depicted) and transceiver 856B is a wireline transceiver. These transceivers enable data processing system 406 to (i) transmit information-conveying signals to other elements of percussion controller 400 and (ii) to receive information-conveying signals from such other elements. For example, in the illustrative embodiment depicted in FIG. 4a, transceiver 856A is used for communications with instrumented strikers 402 and indicator panel 414. Transceiver 856B is used for communications with sensor-enabled striking surface 404, charging cradle 408, and instrumented pad 412. In some other embodiments, percussion controller 400 includes additional wireless and/or wireless transceivers. For example, in some of such embodiments, one wireless transceiver is used for communications between data processing system 406 and instrumented striker 402, another wireless transceiver is used for communications between data processing system 406 and indicator panel 414. It will be clear to those skilled in the art, after reading this specification, how to make and use transceivers 856A and 856B.

(75) In the illustrative embodiment, processor 858 is a general-purpose processor that is capable of, among other tasks, running Operating System 862, executing Specialized Applications 864, and populating, updating, using, and managing Reference Data and Intermediate Results 866 in processor-accessible storage 860. In some alternative embodiments of the present invention, processor 858 is a special-purpose processor. It will be clear to those skilled in the art how to make and use processor 858.

(76) Processor-accessible storage 860 is a non-volatile, non-transitory memory technology (e.g., hard drive(s), flash drive(s), etc.) that stores Operating System 862, Specialized Applications 864, and Reference Database and Intermediate Results 866. It will be clear to those skilled in the art how to make and use alternative embodiments that comprise more than one memory, or comprise subdivided segments of memory, or comprise a plurality of memory technologies that collectively store Operating System 862, Specialized Applications 864, and Reference Database and Intermediate Results 866.

(77) It is to be understood that FIG. 8 depicts one embodiment of data processing system 406; a variety of other hardware platforms or arrangements can suitably be used. For example, system 406 can be implemented in a virtual computing environment. In some embodiments, multiple processors can be used, wherein different processors execute different Specialized Applications. The use of multiple processors may be advantageous or necessary as a function of the rate at which information is being processed.

(78) Furthermore, in some embodiments, the various elements of data processing system 406 are co-located with one another. In some other embodiments, one or more of the elements is not co-located with the remaining elements. For example, in some embodiments, processor-accessible storage 860 is not co-located with processor 858.

(79) FIG. 9 depicts the contents of Specialized Applications 864. The routines stored in this “component” of processor-accessible storage 860 enable percussion controller 400 to perform the various tasks for required for operation, including, without limitation, the prediction of the force and location of the impact of instrumented striker 402, mapping of impact zones to musical events, as well as keeping track of all the strikers that are actively being used, setting the computational priority of IN on active strikers, background tracking on dropped strikers and on strikers that are recharging in the cradle, as well as to perform various optional tasks.

(80) The software routines stored in Specialized Applications 864 include the following: Striker Initialization 970. This routine determines the initial conditions required for IN calculations. This routine requires data obtained by touching instrumented striker 402 to sensor-enabled striking surface 404. Also, rolling the instrumented striker on the sensor-enabled striking surface will reveal any misalignments in the 3-axis sensors (i.e., accelerometers 538 and 548 and digital compass 542). As required, corrections can be applied during processing to account for any such misalignments. Surface initialization 971. This routine determines where (geographically) sensor-enabled striking surface 404 is residing and its altitude. This establishes the orientation of sensor-enabled striking surface 404 with respect the Earth's gravity and magnetic fields. This routine utilizes latitude and longitude data, GPS readings, input from Performance Locations Profile 1092 and Geocentric Dataset 1093 as available in Reference Database 866 within processor-accessible storage 860, etc., to the extent available. Impact-Surface Zone Boundaries Illumination 972. This routine illuminates the appropriate lights in light mesh 654 to demarcate the boundaries of the impact zones established on sensor-enabled striking surface 404. The pre-defined Zone Boundaries 1085 are recalled from Reference Database 866 within processor-accessible storage 860. Inertial Navigation: Acceleration, Velocity, and Location-of-Striker 973. With every sensor sample from instrumented striker 402, inertial navigation calculations are performed to predict striker location. Next Location-of-Striker Prediction 974. This routine use the results of routine 973, which performs the IN computations for acceleration, velocity, and location-of-striker to then predict the future location-of-striker at exactly the next sequential time when the striker's sensor will again be sampled (or forward to two sample cycles in the future). If the predicted future location-of-striker is not entirely above the sensor-enabled striker surface 404, then the time of impact is computed and then the predicted future location-of-striker is computer for the condition of bouncing off sensor enabled striking surface 404. If the time of impact is computed, then Striker Impact Location Prediction 975 must be run using this time of impact parameter. Striker Impact Location Prediction 975. This routine predicts the striker impact location based on the time of impact solution obtained from Next Location-of Striker Prediction 974 (usually the striker's velocity, etc.). The predicted location is mapped into an appropriate predefined impact zone, as obtained from Zone Boundaries 1085 in Reference Database 866 within processor-accessible storage 860. Force-of-Impact Prediction 976. This routine predicts the force of impact of instrumented striker 402 on the sensor-enabled striking surface using the location prediction obtained via routine 975. That is, based on the predicted location, the velocity of the striker at impact, etc., the force of impact is predicted. Correction of Inertial Navigation from Measure Striker Impact Errors 977. This routine compares the actual location (and optionally force) of the instrumented striker's impact with the predicted values. To the extent any discrepancy that is deemed significant is observed, corrective parameters are computed and then provided to IN routine 973, which performs the correction on the next (sampling) cycle. Event Message Generation 978. Having mapped the predicted strike location to a impact zone via routine 975, this routine accesses Musical Event Mappings 1087 from Reference Database 866 to correlate the impact location to a musical event. Position-Matching and Force Matching 979. These routines track a performer's technique and enable comparison to Reference Throwing Techniques 1088 in Reference Database 866. These routines are also used to build User Profile 1090 in Reference Database 866. Tracking of Human Factors Grip Points and Pivot Points 980. This routine persists a history of results from IN routine 973 and then performs a calculation of the grip pivot point of the striker. A history of up to about 5000 results of the grip pivot points is used with IN routine 973 computations to compute the wrist pivot, elbow pivot and shoulder pivot point locations. Establish Impact Zones 981. This routine is used prior to performance to create pre-defined impact zones. The predefined impact zones are stored in Zone Boundaries 1085 in Reference Database 866. Musical Event-to-Impact Zone Mapping 982. This routine maps musical events to impact zones. This routine is used prior to performance in conjunction with the pre-defined Zone Boundaries 1085 in Reference Database 866 to create Defined Musical Event Mappings 1087. User Profile Determination 983. This routine performs statistical averages of the information from Tracking routine 980 to supply generalized parameters for grip pivot, wrist pivot, elbow pivot, and should pivot for User Profile 1090 in Reference Database 866. Non-throwing Motion Correlation 984. This routine persists a history of results from IN routine 973, and then performs a correlation matching algorithm on that history against a record of acceleration, velocity, and location-of-striker pre-recorded patterns. When the correlation result exceeds a threshold value, the musical event associated with that pattern is issued to Event Message Generation 978.

(81) FIG. 10 depicts the contents of Reference Database and Intermediate Results 866 in processor-accessible storage 860. The information stored in Reference Database 866 are accessed by many of the routines comprising Specialized Application 864. The information stored in Reference Database 866 include: Delay Configurations 1084. Parameters (both preprogrammed factory presets and user defined data) to set up the anticipated delays from the user's MIDI equipment and sound generation equipment, including musical event message transmission and routing devices, computers with musical event latencies, and sound generation hardware with signal processing latencies. These transmission, sound-generation processing, buffering delays are corrected by issuing musical event messages with a total pre-delay in advance of the actual impact that will eliminate unwanted delay. Zone Boundaries 1085. Parameters (both preprogrammed factory presets and user defined data) to establish the boundaries of the impact zones on sensor-enable striking surface 404. Virtual Impact Zones 1086. Parameters (both preprogrammed factory presets and user defined data) to establish the boundaries of the virtual zones not located on any Surface. These zones are then mapped to musical events. Musical Event Mappings 1087. Parameters (both preprogrammed factory presets and user defined data) to establish the mapping of both the physical impact zones (on a Surface) and the virtual (not on a Surface) zones to the musical event that shall be issued for that zone. Reference Throwing Techniques 1088. Preprogrammed data for instructional applications; data that provide expert throws of the striker for comparison and reference by the user. Non-Throwing Motions 1089. Profile Parameters (both preprogrammed factory presets and user defined data) to establish the definitions for non-throwing striker motions, such as muting a cymbal, muting a ringing drum, conducting like a baton to a tempo, or conducting like a baton for a volume swell. User Profile 1090. User defined data for the historical human factors associated with throwing and bouncing strikers, such as striker grip points, wrist and elbow pivot radii, shoulder pivot radius, etc. User's Striker Profile 1091. Parameters (both preprogrammed factory presets and user defined data) that keep historical data about each of the strikers used or associated to the system, including Striker unique identification codes, historical 3-Axes sensor alignments, Striker warp, and Striker sensor sensitivity. Performance Locations Profile 1092. Parameters (user defined data) that keep data about frequently used locations of the system where performance or rehearsal would occur, and any corrections of the default geocentric data at that location. For example, frequent locations might be at home, band rehearsal, 5th St. Grill, etc. . . . Geocentric Dataset 1093. Preprogrammed factory data about the latitude, longitude, elevation, and magnetic flux direction at the Earth's surface.

(82) FIG. 11 depicts method 1100 in accordance with the illustrative embodiment of the present invention. Task 1102 recites predicting a force of impact of a striker on a striking surface before impact occurs. As previously discussed, this task involves obtaining kinetics information about instrumented striker 402 and applying inertial navigation techniques thereto.

(83) Task 1104 recites determining a location of impact of the striker on the striking surface. As previously discussed, in some embodiments, this task involves obtaining kinetics information about instrumented striker 402 and applying inertial navigation techniques thereto. In some other embodiments, the location of impact is measured on sensor-enabled striking surface 404; that is, only the force of impact is predicted.

(84) Task 1106 recites relating the location of impact with a musical event. As previously disclosed, this task involves determining the impact zone on the sensor-enabled striking surface in which impact is predicted to occur, and determining the musical event that corresponds to an impact at that zone.

(85) Task 1108 recites generating a signal that conveys information pertaining to the musical event. As previously discussed, this can be done in conventional fashion via MIDI protocol.

(86) Task 1110 recites transmitting the signal to a device that generates a signal that can be converted to sound that is related to the musical event.

(87) Additional considerations and details about some of the methods and routines disclosed herein are presented in conjunction with FIGS. 12a-c and 13 through 19.

(88) FIGS. 12a through 12c depict the sequence of system states and automatic processing. The system is in OFF State when it is de-energized. Packing, shipping, hauling, unpacking, and mechanical and electrical installation all occur in this state. During installation, assembly of sensor-enabled striking surface 404, charging cradle 408, and any other assemblies are mounted on a stand. (See, e.g., FIGS. 4a and 4b.) Power cables and electrical system cables are the connected. Instrumented strikers 402 are typically be placed in the charging cradle. When power is applied, the OFF state terminates, and Surface Initialization begins. When power is de-energized, the OFF state immediately resumes.

(89) In the Striking Surface Initialize state, just after power is applied, instrumented strikers 402 in charging cradle 408 will begin receiving power, processor 858 (see, e.g., FIG. 8) begins booting operating system 862 and initializing various Specialized Applications 864. Indicator panel 414 and charging cradle 408 are initialized. Initialization requires input of external information for the latitude and longitude and elevation of the system, which could optionally be provided via wireless or wired USB communications to a GPS application on a handheld device, or through a user interaction using indicator panel 414. (See, e.g., FIG. 10, Performance Locations Profile 1092 and Geocentric Dataset 1093.)

(90) Sensors of the sensor-enabled striking surface 404 take initial readings and set system parameters used during performance. The direction and strength of the gravity field to the Striking Surface frame is read via an included 3-axis accelerometer (not depicted in sensor-enabled striking surface). Alternatively, readings from the 3-axis accelerometer 538 (see, e.g., FIG. 5) in instrumented striker 402, which must be held motionless on the sensor-enabled striking surface, can be used instead. The magnetic attitude of the Striking Surface frame is read by an included digital compass (not depicted in sensor-enabled striking surface). Alternatively, readings from digital compass 542 in the instrumented striker, which must be held motionless on the sensor-enabled striking surface, can be used instead. The gravity attitude of the Striking Surface frame is computed from the gravity field calculation and the gravity field to the Striking Surface frame. The transceiver is initialized and, upon completion, processor 858 begins issuing a discovery request message to instrumented strikers 402. Other systems of percussion controller 400 in the vicinity may also respond to the discovery request. The system then proceeds to Striker Initialization state.

(91) In the Striker Initialization state, as instrumented strikers 402 individually energize, they respond to the discovery requests, and processor 858 registers them in a Striker Protocol Table. Gradually, processor 858 reduces the rate of issuing discovery request messages and increases the rate of polling instrumented strikers 402 for data from their sensors. When instrumented strikers 402 report that they are fully energized, indicator panel 414 requests that the operator performs a Striker Initialization. For this process, each instrumented striker 402 is first placed motionless on sensor-enabled striking surface 404, and then rolled across the striking surface. After each instrumented striker is initialized, the system proceeds to the Performance state.

(92) The Performance mode is a real-time loop of process execution control. Instrumented strikers 402 and sensor-enabled striking surface 404 must be sampled and processed at consistent rates of approximately 1000 Hz; that is, once per millisecond, in order to the achieve psychoacoustic performance criteria required by professional musicians.

(93) The Performance mode processing loop (FIG. 12b) begins with scanning of sensor data from active instrumented strikers, then executing the inertial navigation computations for each such striker, computing the striker kinematics and predicting the striker impacts on sensor-enabled striking surface 404. In each polling cycle, one additional inactive instrumented striker 402 is polled for its status. In each polling cycle, a different inactive striker is polled for status.

(94) With continued reference to FIGS. 12a through 12c, and now referencing FIG. 13, the process of scanning the sensor-enabled striking surface is executed. From the striker scan it was determined if instrumented striker 402 would impact sensor-enabled striking surface in the next one or two update cycles along with the prediction for where on that surface the instrumented striker would impact. If there is no immediate surface impact predicted, then the processing continues for a normal surface scan proceeding sequentially through every row and column; measuring each sensor of sensor-enabled striking surface 404. This is performed between impacts to detect any finger touches that a performer uses, for example, to control the musical performance (e.g., muting a sound, etc.).

(95) If an immediate surface impact predicted, then the prediction for where the striker would impact on sensor-enabled striking surface 404 is used to create an impact scan list of the sensors surrounding the predicted point of impact. Process control is then passed to the normal surface scan process, after triggering an immediate interrupt to scan the predicted impact area. The interrupt causes a process to scan the predicted impact area using the impact scan list, recording the time of the scan and the impact location if an impact is discovered.

(96) If no impact is detected, a delay is triggered of approximately 100 microseconds to repeat interrupt to scan the predicted impact area. If an impact is detected, processing begins for that instrumented striker's impact to: calculate the error corrections (as necessary), recording the striker's Navigation error offsets to be used in future striker inertial navigation updates, and returning to the normal processing from the interrupt. To avoid an infinite interrupt loop, a time-out control is used to conditionally trigger the delayed interrupt.

(97) Continuing with FIG. 12b, charging cradle 408 is scanned for the presence of instrumented strikers 402, and then passed to the application controller to run various Specialized Applications 864 in the remaining execution time left in the performance mode real-time cycle.

(98) The instrumented striker sequence is depicted in FIG. 12c. Strikers are initially de-energized and may return to that state during the performance. The depleted state can occur during charging from a de-energized state or just from normal use in an active state during performance. In this state, there is insufficient stored energy in the striker to assure correct operation. A depleted striker can lose energy if it is not charged and will shut off. Through continued charging of the striker, the charged state is obtained. There are three sub-states: barely charged, adequately charged, and fully charged. These sub-states are useful indications to the performer for which instrumented striker 402 to select during emergencies (e.g., a dropped stick, etc.), so that a barely charged striker in hand may be swapped for a fully charged striker in charging cradle 408. An instrumented striker 402 that is not present in the charging cradle and that is sensed to be in motion is defined to be in the active state. An instrumented striker that is not present in the charging cradle and that is sensed to be without motion is defined to be in the inactive state. Active and Inactive strikers may become depleted over time. The depleted state should be indicated to the user via indicator panel 414.

(99) FIG. 14 depicts the prediction of the impact of instrumented striker 402 on a tilted sensor-enabled striking surface 404. The Striking Surface Frame (“SF”) axes are shown overlaying the sensor-enabled striking surface 404 with the elevation axis perpendicular thereto. The perspective of FIG. 14, which is viewing into the left side of the sensor-enabled striking surface shows the mathematical relevance of the SF for making impact calculations.

(100) In the SF, the calculated predicted locations of the instrumented striker trace points can be easily checked for a negative elevation (i.e., below the axes in the plane of the sensor-enabled striking surface). Both the elevation of the last striker trace point prior to impact (i.e., position “5” in FIG. 14) and the magnitude of the predicted negative elevation are used for precisely interpolating to the time and location of the striker's impact. This striker position is identified as “X,” the dashed line indicating the projected location and time of impact. This information is used to compute predicted velocity of the instrumented striker at the time of impact (using the previously computed velocity at position 5). The velocity is used to compute predicted energy of impact using the known mass of the striker (i.e., E=½ mV.sup.2). Then the magnitude of the predicted negative elevation can again be used for predicting the elevation of the point of the actual instrumented striker after bouncing back (not depicted) from sensor-enabled striking surface 402. The call-out “X” indicates a next predicted position from the measured and computed velocity, where points along the striker trace have negative elevation in the Surface Frame. It is to be understood that at actual sample rates a professional percussionist's throw will have twenty or more samples taken and computed; the six positions shown in FIG. 14 are simply for pedagogical purposes.

(101) With continuing reference to FIG. 14, the wrist pivot of the throw is illustrated in the Surface Frame point of view, which is a significant point of view for purposes of instructing throwing techniques. Specialized Applications for aiding instruction (e.g., Position-matching & Force matching 979, etc.) are optionally executed by the system to access the stream of Inertial Navigation computations and/or striker traces that can be, for example, recorded to an external bulk storage device, streamed over a network, or streamed to an external video display.

(102) FIG. 15 illustrates forces experienced by instrumented striker 402 during a throw, the important wrist pivot is in both the Striker Frame and the Surface Frame. The Grip Force between the Thumb and Pointer fingers counter balances the centripetal force of the mass at the center of gravity of the striker (not depicted). The throwing force on the instrumented striker is also applied between the Thumb and Pointer fingers. The accelerometers experience the same Gravity force and rotational torque about the wrist pivot, yet experience very different local centripetal forces.

(103) The Inertial Navigation computations, as taught for example by Britting, address the centripetal and gravity force implications, but instructional value can also be derived from applications that assess these forces. For example, a rapid decrease in centripetal force can indicate the instrumented striker is slipping the grip, which could be detected by instructional applications. As another example, rolling the striker during a throw is inefficient and this could be detected by instructional applications. Also, immediately prior to impact there should be a release of the throwing force on the instrumented striker, which could be detected by instructional applications. Finally, the pivot of throw should remain stable in both the Striker Frame and the Striking Surface Frame which could be detected by instructional applications. Instructional applications would also be concerned with the accuracy of impact placement and timing that could make use of information from the surface impact scans. Parameters inside the Inertial Navigation computations or the surface scan procedures are made available to the instructional applications. The software architecture of the system provides, at minimum, Application Program Interfaces (API) for subscribing to the striker Inertial Navigation parameters or surface scan parameters.

(104) To automate a throwing technique assessment for an instructional application, the primary rotational axis for each accelerometer is computed at every striker sample from a multitude of past samples. Then, calculating the short term weighted average of approximately 3 to 12 samples across both accelerometers, positional tracking algorithms are used to detect the nearness of the pivot to the Wrist Axis. This should be near the stick Butt, and of much shorter radius than an Elbow Axis. Additional calculations then utilize inertial navigation parameter streams to detect the pitching force about the wrist pivot and detect throwing-axis stability. These are recorded and can be displayed externally in real-time to the instructor and student.

(105) FIG. 16 depicts a single stroke throw about wrist axis, wherein impact requires shifting the axis to the grip point. Instrumented striker 402 is allowed to pivot on impact about the grip point as the hand simultaneously reverses to lifting about the wrist pivot. The stick is then recovered, lifted about the wrist axis for the next throw. Positions 1, 2, and 3 depicts a sequence of throwing about the wrist pivot, position 4 in the sequence indicates impact bounce about the grip pivot, and positions 5, 6, and 7 in the sequence indicate lift about the wrist pivot. To automate a single-bounce-technique assessment for an instructional application, the primary rotational axis for each accelerometer is computed at every striker sample from a multitude of past samples. The primary rotational axis for each accelerometer (e.g., accelerometers 538 and 548) is computed at every sample from a multitude of samples, with the weighted averaging as discussed previously. During the bounce, the grip axis should be through the shank of the instrumented striker, approximately ⅓ of the distance from the striker butt. An improper grip is detected when the grip axis is underneath the instrumented striker (not through the striker) or at the wrong location along the length of the instrumented striker. The bouncing axis stability is recorded and can be displayed externally in real-time to the instructor and student. Additional instructional applications provide prerecorded master percussionist throws and bounces, which are correlated against the student's striker positions and velocities. Real-time and replay displays (external) of striker throws and bounces—master vs. student—are provided.

(106) FIG. 17 illustrates a double stroke throw and bounce. After a throw about wrist axis (positions 1, 2, and 3), the first impact requires shifting the axis to the grip point (position 4). After the first impact against sensor-enabled striking surface 404, instrumented striker 402 is freely pivoting about the grip point (positions 5 and 6) when a double stroke pull is executed by the performer (i.e., a finger pulled bounce during positions 5, 6, and 7) reversing the rotation about the axis of the grip point. The stick is allowed to pivot following the second impact about the grip point (positions 8 and 9). Then the stick is lifted about the wrist axis for the next throw (positions 10, 11, and 12). The automation of a rudimental double bounce technique assessment follows similarly to the previously discussed single stroke throw assessment application, now with the additional capability to assess the timing of the finger pull forces to bounce the striker.

(107) FIG. 18 depicts the highly constrained volume of space where an instrumented striker will travel and for which accurate inertial navigation solutions are required. FIG. 18 depicts both a front and side view of the area around sensor-enabled striking surface 404. The striker volume A-A is shown as a dashed line to indicate the boundary for the right hand instrumented striker 402 (solid line). The striker volume for the left hand instrumented striker 402 (dashed line) is not shown. There is a natural overlap of the striker volumes. For a drum-set performance using a single sensor-enabled striking surface, each instrumented striker will require approximately 1.5 cubic meters of space, whereas the combined space for both instrumented strikers 402 is approximately 2 cubic meters. Active instrumented strikers should not be outside of this combined space during performance. Calculated elevations outside of the combined volume are a possible indication of the vertical divergence problem recognized by Britting. This would be indicated to the percussionist (e.g., via indicator panel 414 of FIG. 4a) and require re-initialization of that instrumented striker. A dropped instrumented striker exits the combined volume in a state of free-fall, so there will be no external forces being measured on the striker's accelerometers (only centripetal forces would be experienced and measured). Thus a dropped-striker condition can be detected. An instrumented striker that is removed from charging cradle 408 and then enters the combined volume requires initialization. In this case, there will be an indication to the percussionist on the indicator panel to initialize that particular striker.

(108) The magnetic and gravitational fields should be constant in the combined striker volume. For the AAS approach to sensing motion of instrumented striker 402, this means that magnets and ferrous materials must not influence the uniformity of the magnetic field in the combined striker volume. Structural supports and stands should be made of non-ferrous material such as aluminum or carbon fiber composites. Loudspeakers will need to be kept approximately a few meters away from the combined striker volume. The performance location should not occur near structural steel beams or near metal walls because these might focus the Earth's magnetic field and distort AAS readings. One compensation that is possible for magnetic field distortion is to make measurements of the magnetic field across the combined volume during surface initialization, such as by using a conventional magnetometer device (not depicted). A mapping of the magnetic field in the combined volume is then created that is used during performance to correct the AAS readings based on the IN computed positions.

(109) Dynamically varying magnetic fields nearby or inside the combined striker volume are not compatible with the AAS sensing approach; these fields from devices such as lapel microphones, headsets, earphones, or vocal microphones will distort the AAS measurements in a way that is very difficult to compensate. Thus, when instrumented strikers include an AAS device, a close microphone on the percussionist's voice should be avoided. Rather, a distant, highly directional microphone is preferred.

(110) Referring now to FIG. 19, this Figure depicts the transformation of the measured direction of the magnetic attitude to obtain the gravity attitude. The Magnetic Frame and Gravity Frame are each measured during initialization activities, either in instrumented striker 402 with its 3-axis AAS and accelerometers or with the striker when it is placed motionless along sensor-enabled striking surface 404. From the Magnetic Frame and the Gravity Frame, a constant coordinate frame direction cosine matrix “DCM” is computed for performing a coordinate transformation, as taught by Britting in section 2.1.3 on page 13.

(111) In FIG. 19, the magnetic attitude is illustrated by a pair of arrows, one on the symmetric axis of the striker, and the other parallel to the magnetic flux lines. As depicted, the magnetic attitude is influenced by the pitch, roll and yaw of the instrumented striker, which is significant to accurately solving the gravity attitude of the striker. The magnetic attitude is used with the Magnetic to Gravity DCM to compute the Gravity Attitude of the instrumented striker, a 3-axis unit vector that points in the direction of gravity relative to the Striker Frame. The previously measured gravity magnitude is then multiplied upon the Gravity Attitude (a unit vector) to accurately compute the 3-axis gravity acceleration force relative to the Striker Frame. Finally, as taught by Britting, the gravity acceleration force is subtracted from the 3-axis accelerometer measurements.

(112) Britting teaches sensor axis alignment and platform alignment error corrections in Chapter 8; alignments are applied to magnetic attitude and the accelerometer measurements. A DCM is computed for aligning the AAS sensor, and another DCM is computed for each of the 3-axis accelerometers during the striker initialization, when the performer first places the instrumented striker on the sensor-enabled striking surface motionless, and then rolls it on the surface. Following Brittings teachings, measurements taking by the sensors in the instrumented striker at known times and positions (sensed by the sensor-enabled surface on the Surface Frame) are then converted into the AAS alignment DCM and the alignment DCM for each accelerometer.

(113) FIG. 20 depicts the installation of a permanent magnet beneath sensor-enabled striking surface 404. FIG. 20 depicts the magnet centered beneath the sensor-enabled striking surface producing magnetic field lines through the striker volume above sensor-enabled striking surface 404. The striker volume is shown as a dashed line to indicate the boundary for the right hand instrumented striker 402. The installation of a loudspeaker type of magnet (approximately 1 to 2 Tesla) provides approximately five orders of magnitude improved field strength over Earth's Magnetic Field. The magnetic field direction and strength is measured at the manufacturing facility (of percussion controller 400) and stored in processor-accessible storage 860. This data is used to correct the AAS measurements. In this way, the dynamically varying magnetic concerns from devices such as lapel microphones, headsets, earphones, or vocal microphones are eliminated by the strength of the fixed magnet under sensor-enabled striking surface 404.

(114) It is to be understood that the disclosure teaches just one example of the illustrative embodiment and that many variations of the invention can easily be devised by those skilled in the art after reading this disclosure and that the scope of the present invention is to be determined by the following claims.