Smart Acoustic Drum and Sing Music System

Abstract

A system that wirelessly integrates actual drum equipment and singing equipment with a computer and the internet to allow musicians and or singers remotely located from one another to play music and or a competitive real or simulated music and or singing competition. An individual musician may opt to play music solo or practice to improve basic music, drumming, and or singing techniques. The system includes smart music and drum systems, an audio detection system, and drum motion sensing devices, all containing circuits and contact sensors or motion sensors, coupled with signal processing and radio frequency transmitter circuitry, thereby wirelessly communicating game performance information to a remote receiver-computer. The computer displays musician and or singing information, and visually simulates and controls a music and or a singing competition between two or more musicians, via the internet, having similar equipment and remotely located from each other. Standard drum sets may be retrofitted with the sensors and associated circuitry to convert such drum sets into smart drum sets for use with the system. Standard microphones may be retrofitted with the sensors and associated circuitry to convert such microphones into smart microphones for use with the system. The system employs specially developed computer software to process musician performance data, control the music competition, communicate music and information between musicians and or singers, generate and control visual simulations, and display singer and or musician performance information.

Claims

1. A computerized interactive music and singing system comprising a local, and a remote player site; comprising a music implement, comprising: a drum; a drumstick; a musical instrument; and a smart microphone, comprising: a microphone with a sensor, attachable and or detachable to said musical instrument, said drum, and or said drumsticks; a first array of sensors, mounted on the face side of said drumhead of said drum; a second array of sensors, comprising a contact sensor mounted to said microphone; a first computer, programmed to process data derived from data acquired by said first and said second sensor arrays; a first communication link, for transmitting data derived from said data acquired by said first sensor array to said first computer; a second communication link, for transmitting data derived from said data acquired by said second sensor array to said first computer; a second computer, programmed to communicate with a remote player site; a display monitor, connected to said second computer; a third communication link, for transmitting data derived from said data acquired by said first computer to said second computer; and a fourth communication link, for transmitting data derived from said data acquired by said second computer to said first computer; wherein said second computer is further programmed to analyze the performance data of a person drumming said drum and or playing said musical instrument, and or acoustic data from said microphone.

2. The computerized interactive music and singing system as recited in claim 1, wherein said first computer is further programmed to send the results of said performance analysis to said second computer upon completion of said performance analysis via said third communications link.

3. The computerized interactive music and singing system as recited in claim 1, wherein said second sensor array, comprises a motion sensing device and or a motion detector, mounted internally and or externally to said drum set and or said musical instrument, and or internally or externally to said smart microphone or a sound implement.

4. The computerized interactive music and singing system as recited in claim 1, wherein said first sensor array comprises piezo-active transducers, and or drum-triggers, and or pressure sensors, and or force sensors.

5. The computerized interactive music and singing system as recited in claim 1, further comprising electronic circuitry for outputting to said first communications link, a signal representing the energy, and momentum, of a drumstick contact event with said drum, derived from data acquired by said first sensor array.

6. The computerized interactive music and singing system as recited in claim 5, wherein said electronic circuitry comprises a computing device mounted to said drum, and or a drum set, and or said musical instrument, and or said smart microphone, programmed to convert data acquired by said first sensor array, and said second sensor array, into a time-multiplexed serial digital data stream, containing a respective binary-coded word for each channel.

7. The computerized interactive music and singing system as recited in claim 1, wherein said second computer further comprises a competition server, connected to said second computer via a network, comprising a fourth communication link, wherein said second computer is further programmed to process data from said music implement, and or the competition server, representing the user performance of a competition at a remote site during the turn of said competitor; wherein said competition server selects remote players from a queue of awaiting players in response to a first musician, and or singer, indicating a readiness to compete.

8. The competition server system as recited in claim 7, wherein said music implement, and or said competition server, is programmed to establish connections amongst subscribing players at a plurality of remote locations via said network.

9. The competition server system as recited in claim 7, wherein said second computer is further programmed to create, and manage, a plurality of music and singing games, and or music and singing competitors, wherein said music and singing games comprise one or more first computers of opponents connected to said first computers, wherein each first computer comprises a network port for connecting to said network, a serial data port for receiving a serial data stream from a respective set of sensors designed to detect the motion of a respective music implement and or singing equipment items being manipulated by a respective musician and or a singer, and a port for connecting to its respective display monitor, wherein each first computer is programmed to perform the following: processing the digital data stream from said respective set of sensors into music and or singing game data; having a format representing music and or singing game results for said first computer player; controlling the respective display monitor via said second computer to provide visual feedback concerning the progress of the game; transmitting said game data from said first computer to said second computer that comprises data to be transmitted to said competition server; and said second computer polling said competition server for receipt of game data, and or messages, from opponents at remote sites.

10. The computerized interactive music and singing system as recited in claim 1, further comprising an electronic drum set and or a standard drum set, retrofitted with electronic devices, and an electronic smart microphone and or a standard microphone, retrofitted with electronic devices, operatively connected to said first computer.

11. A computerized interactive drum and music system, wherein said first computer programming comprises: a serial port listener software program, that receives acquired data from said first sensor array and said second sensor array; a socket event listener, that receives data from said competition server; and a main thread for alternately processing data received by either said socket event listener or said serial port listener in accordance with a sports competition format; wherein acquired data from said first sensor array, and said second sensor array, are processed by said first computer and the results are sent to said competition server only if data has been previously received from said competition server indicating that it is that musician and or singer's turn to play a music implement, and or sing, and or play a sound implement, otherwise, acquired data from said drum set, said drum set motion sensing device, said microphone, and or said computer, and or an audio system, is ignored if data has been previously received from said competition server indicating that it is a remote player's turn to play a musical implement and or sing; wherein controlling the reception of data received from said first sensor array, and said second sensor array is allowed only during that musician's and or singer's turn to play, thus allowing and controlling sequential play, the use of one or more said music implements, and or said drum sets, and the use of one or more said microphones.

12. The computerized interactive music and singing system as recited in claim 5, wherein said computer programming further comprises musician training software, providing training as a function of the data received from said music implement, said drum set, said drum motion sensing device, and said audio system.

13. The computerized interactive music and singing system as recited in claim 1, wherein said second computer further comprises audible drumming music image simulation and display software for displaying images that simulate the results of said performance analysis.

14. A computerized interactive drum system comprising: a drum; a drumstick; and a smart microphone, comprising a microphone with a sensor, attachable and or detachable to said drum; a drumstick; a row of sensors, mounted on said face of said drum set, for acquiring data from respective channels, and said acquired data representing a force, a time of contact, an energy, a frequency, and a position of impact of a drumstick relative to each impacted sensor on drumhead; an accelerometer, attached to said drum set, a music and or singing tool, a music implement, and or an audio implement, to detect spatial translational motion and or rotational orientation of a drumstick, adapted to sense acceleration along three orthogonal axes; a first computer having a data input port for receiving data and an output port for communicating with said display monitor of said first computer; a first communications link, for communicating data derived from said impact data acquired by said sensor array in said respective channels to said data input port.

15. The computerized interactive music and singing system as recited in claim 1, wherein said first communications link comprises a wireless transmitter housed inside and or attached to said drum set, and or smart microphone, and a wireless receiver coupled to an input port of said first computer.

16. The computerized interactive music and singing system as recited in claim 7, wherein said second computer further comprises audible image simulation and or display software for the display of real and or simulated musical implement images, and of participants, and the results of said performance analysis.

17. The computerized interactive music and singing system as recited in claim 7, wherein said second computer programming further comprises single local and remote musician training software providing training as a function of the data received from said drum, said drum motion sensing device, and said audio system.

18. The computerized interactive music and singing system as recited in claim 5, wherein said second communications link further comprises a wireless transmitter mounted inside said audio system, and a wireless receiver, coupled to an input port of said first computer.

19. The computerized interactive drum system of claim 14, wherein the first computer is adapted for at least one of identifying a stroke, recognizing an event, inferring the commencement of the event, inferring the conclusion of the event, identifying the stroke cycle, a measure of force, and a time duration, based on at least some of the information sensed by said first sensor array.

20. The computerized interactive music and smart singing system as recited in claim 17, wherein said second computer is further programmed to create and manage a plurality of games for musicians, wherein said games comprise one or more said computers of musical opponents connected to said first computers; wherein, each first computer comprises a network port for connecting to said network, a serial data port for receiving a serial data stream from a respective set of sensors designed to detect the motion of respective musical implements and or equipment items being manipulated by a respective musician, wherein each first computer is programmed to perform the following: processing the digital data stream from said respective set of sensors into game data, having a format representing game results of said musician; providing visual feedback concerning the progress of the game to said second computer operatively connected to the display monitor; transmitting said game data from said first computer to said second computer, comprising data from said music implements and or said drum sets.

Description

BRIEF DESCRIPTION OF DRAWINGS

[0015] FIG. 1 is a diagrammatic illustration of components of a computer implemented drum system according to this invention.

[0016] FIG. 2 is a top plan view of a drum with sensors and circuitry and used in the computer implemented system of FIG. 1.

[0017] FIG. 3 Smart Microphone with sensor and hand-grasping Smart Microphone with sensor.

[0018] FIG. 4 is a diagrammatic front plan view of a putter with a drumhead and circuitry forming a further alternative embodiment of a drum, for use with the computer implemented system of FIG. 1.

[0019] FIG. 5 is a schematic block diagram of a drumhead electronics installation for use with the drumheads of Figures L2, and 4.

[0020] FIG. 6 Tri-pad Sensor with 3 different activation areas or top plan view of a drumstick sensing element with 3 activation areas for use in the drumming surface strokes of FIGS. 1, 2, and 4.

[0021] FIG. 6 was previously FIG. 7. FIG. 7 is a schematic block diagram of a singing and drum electronics installation for communicating with the computer in a computer implemented system according to FIG. 1 and FIG. 3.

[0022] FIG. 7 was previously FIG. 8. FIG. 8 is a block diagram of a computer installation for use as the computer and information receiving interconnect of the system of FIG. 1.

[0023] FIG. 8 replaces FIG. 10. FIG. 9 is a functional block diagram of the software operation of the computer of FIG. 8.

[0024] FIG. 10 is a flowchart illustrative of a portion of the operation of the computer of FIG. 8 operating as indicated in the block diagram of FIG. 9.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENT

[0025] As shown in FIG. 1, a preferred embodiment of the invention includes a wireless smart drum system 20, a wireless audio detection system 22, a wireless drum motion sensing plate 24, a wireless receiver 26 connected to a computer 28, and a display or monitor 30, with speakers 31 operated under the control of drum system software 32, and connected via the internet to an internet drum game server 34 (called herein the GGC server)

1. Smart Drum System

[0026] The smart drum system 20 has a head 40 and a stick 42. As shown in FIGS. 2 and 3, the head 40 has a drumstick opening 42, a plurality of embedded contact sensors 46 (three are illustrated in the preferred embodiment), and the internal electronics circuitry 48, including a wireless radio frequency transmitter (58 in FIG. 5). As shown, at least one of the sensors 46 is located at or proximate to the optimal location on a drum face 47 for contact with the drumstick. The contact sensors may be, but are not limited to, sensors employing piezo-active type transducers or drum-triggers, specifically, either piezoelectric or piezoresistive transducers (similar, but is not limited to the Cooper Instruments LPM 562).

[0027] In an alternative embodiment, FIG. 2, three sensors 46 are applied to the face of an adapted drum by a mylar tape or other means 49. Again, the electronic circuitry is internal to the drumhead 40 and connects to the sensors 46 by leads 27.

[0028] In a second alternative embodiment, to retrofit a standard drum, contact sensors 46 are part of an adapter 40 attached to an ordinary drumhead as seen in FIG. 4 and wire connected to an electronic circuitry 48, attached to the drumstick 42, or elsewhere on the drum.

[0029] A drumstick contacting any sensor 46, produces a detectable variance indicating the magnitude, and duration of a sensor-ball impact. The variance may be a change in resistance of a piezoresistive transducer or a voltage change in the case of a piezoelectric transducer. As shown in FIG. 5, the variance is detected and amplified by an associated amplifier 52 and is the input to an associated integration circuit 54, the output of which represents the energy of a ball-drum contact event. Connected to the integration circuit 54, a microcontroller 56 is a multi-input signal processing circuit (similar, but not limited to a Motorola #68HC05) having analog-to-digital signal converting circuits (ADCs), one for each input channel, and a sequential digital signal encoding circuit connected so as to convert the ADC outputs into a time multiplexed serial digital data stream containing a binary-coded word for each channel, indicating the energy of the associated sensor-ball impact event.

[0030] A radio frequency transmitting circuit 58, receives the serial digital data from the microcontroller 56 and wirelessly transmits the information via an internal antenna 60 to a receiver 26 (FIG. 1) for subsequent processing by the computer 28. Using programming as contained in the accompanying microfiche appendix, one skilled in the art can readily accomplish the game programming described. Alternative programming will be apparent as well from the foregoing functional description and the illustrations contained in the appended drawings.

2. Smart Microphone

[0031] The smart microphone uses 22 in FIG. 1, to allow voice inputs as shown in FIG. 3. The smart microphone has a contact sensor pad 64, shown in FIG. 6, containing at least one contact sensor (three different activation areas 65, 66, and 67 are illustrated in the preferred embodiment). The internal circuitry includes a wireless radio frequency transmitter (as shown in FIG. 5). Additional sensor activation area 65 and 67 are adjacent, one on either side of the center area 66. In the preferred embodiment of FIGS. 1, 2, and 3, and like the sensor used at the face of the drum and smart microphone, the sensors may be, but are not limited to, sensors employing piezo-active type transducers, specifically, either piezoelectric and or piezo-transducers.

[0032] A hand touching the smart microphone 60 and containing the sensor pad 65, 66, or 67 produces a detectable variance indicating the hand-sensor smart microphone event. The variance may be a change in resistance in the case of a piezoresistive transducer (similar, but not limited to Cooper Instruments LPM 562) or a voltage change in the case of a piezoelectric transducer. As illustrated in FIG. 7, the variance is detected and amplified by an associated amplifier 71. The amplified signal is then input into a microcontroller 72, having an analog-to-digital signal converting circuit (ADC) and a digital signal encoding circuit connected so as to convert the ADC output representing the sensors signals into a serial digital data stream containing a binary coded word indicating the sensor-hand contact event. The microcontroller 72 may be the same and or similar to the microcontroller 56 of the drum system electronics.

[0033] A radio frequency transmitter circuit 74, receives the serial digital data from the microcontroller 72 and wirelessly transmits the information via an internal antenna 76 to the receiver 26 (FIG. 1) for subsequent processing by the computer 28. The smart microphone configuration is susceptible to much variation. The smart microphone illustrated and described above is well suited to indoor use, on carpet for example.

3. Wireless Signal Receiver and Computer

[0034] At each local and remote player site, a wireless radio frequency signal receiver 26 is connected to the computer 28 by either the serial (USB) or parallel computer ports, as shown in the functional block diagram, FIG. 8. The wireless signal receiver 26 detects digitally coded radio frequency transmissions from the communication circuit associated with any of a smart drum system 20, a smart microphone 22, as shown in FIG. 1. The received transmissions are demodulated by the RF receiver circuitry 122 (FIG. 10), connected to a microcontroller 124, which converts the demodulated data signal into serial binary-coded data, suitable for communications to a computer 28. The computer 28, under the control of the internally installed music and sing system software program, monitors and directs the flow of communications between remotely located participants via the internet, and displays the game simulations and performance information. In appropriate installations the wireless electromagnetic signals that communicate with the receiver that further comprise the communication links may be infrared communications.

5. Computer Drumming Software

[0035] At each remote player site, the computer 28 (FIG. 1) under the control of the drumming software program (shown in the drumming software system functional block diagram, FIG. 9) monitors and controls initialization and the sequential play of the drumming and singing game, or alternatively, the individual musician and or singer player practice session. Upon start up by a musician and or singer at a particular site, the system input parameters are then set and the system internet and music and or singer player port interfaces are initialized 130, as indicated by the arrows 130A and 130B. For internet communications, the serial port listener of the computer 28 is enabled in the preferred embodiment. A remote player event listener is initialized. It then communicates events from one or more of the smart drum systems, the smart microphone, and the motion sensor plate. The main operational software (program) thread is run 130, and the system awaits data input from the appropriate computer communications ports at 132 (port), 133 (remote player Socket Event Listener).

[0036] If competitive play mode has been selected, the program generates a player participation request and sends 134 the request to the GGC game internet server (GGC server) 34 (FIG. 1). Upon identification of a player opponent at 150 (FIG. 12) by the GGC server, the program initiates the player identification sequence 152, and sequential play begins 154 (this software sequence and control routine occurs at each remote player site where play has been initiated. During the game play sequences 154, the program generates the appropriate animation, display, and audio data and commands 136 and 138 (FIG. 9) and communicates with the associated display and speaker devices 30 and 31 (FIG. 1). Upon the occurrence of a local player event, detected at 133, the main operating program at 130, displays the event at 136, and communicates the event at 132 by causing a device transmission at 137 to be sent at 134 via the internet GGC server 135, which displays the event for the opposing player, and alerts the opposing player that it is his/her turn to play. The local player event may be, but is not limited to, the smart drum system impacting a drumstick, the swing of a golf club across the sensing plate, or the balls entry into the receptacle. The program contains time delay limits for the player action, and delays of play beyond these limits generate play quit and disconnect signals. The event at 133 also has the effect of indicating at 139 that it is no longer the local players turn and enables (as indicated by line 139) the serial port listener at 132 to detect an event from the remote player, again via the internet.

[0037] If single music and or singer player practice mode is selected, the internet communications sequences are disabled, other software sequential operating routines continue as described above, and the player's drum system stroke, hand-smart microphone contact, and audio feedback information are communicated only to the computer located at that local players site. and the performance information analyzed and displayed only at the local players site.

[0038] When a game is won, lost, or terminated, the music and sing software system generates the appropriate output signals 156 (FIG. 10, displays the player performance information, and resets to initial pre-game conditions. If one player opponent quits the game or is timed out (due to excessive delay in play) and the remaining player wishes to continue play, the software resumes an internet search for another opponent 152 and 153.

6. Motion Sensor Plate

[0039] The motion sensor plate 80 having a top motion plate 82 and a bottom motion plate 84 is diagrammatically shown in FIGS. 9A-D, wherein the top motion plate 82 contains a plurality of capacitor-forming electrically isolated platelets 83 (twelve platelets are illustrated in this exemplary preferred embodiment). They are evenly distributed at or just below the top plate's exterior upper surface 82. The bottom plate 84 has a homogenous electrically conductive interior surface 85 underlying the platelets 83. Each capacitive platelet 83 contained in the top motion plate 82 forms a capacitive component when the top and bottom motion plates are vertically closely spaced to form the motion sensor plate. A suitable dielectric insulator may be sandwiched between the two plates. The structure is adhesively, or otherwise mechanically joined and it may be covered or coated as desired. The result is a golf club motion sensor plate 80 containing a capacitor matrix (a 34 capacitor matrix is illustrated in the preferred embodiment. The capacitive components 83 are connected to form a capacitive network 88 as is indicated in FIG. 9E.

[0040] Applying an energizing high frequency alternating electrical signal having a frequency in the range from 100 MHz to 200 MHz from an oscillator 87 to the motion plate capacitive network 88 produces an electromagnetic field above the surface of each platelet 83 of the capacitive components of the motion sensor plate 80. Any object, including a golf club, passing near the surface of the energized motion plate will cause a perturbation of the electromagnetic field as illustrated by the sample possible pathways 90 across the plate in FIG. 9C. A network 92 of electrical comparator amplifiers (FIG. 9B) is connected to the capacitor network. The comparators of the network 92 are connected one-to-one with the capacitive elements of the capacitive network 88. The comparators of the network 88 detect voltage variations occasioned by the electromagnetic field disturbance due to a golf cub moving over certain of the capacitive elements of the motion plate. Each different golf club motion over the energized motion plate will produce a uniquely identifiable signal from the comparator amplifier network. There are a variety of known proximity sensors that could be gathered together in an array like that of the platelets 83 to serve as the transducer portion of the golf club and or sports implement motion detector.

[0041] The electrical signal from the comparative amplifier network 92 is applied to an analog-to-digital signal converter 94 (ADC) and the ADC digitized output signal is converted into a serial digital data stream by a multiplexer 96. This data identifies each platelet having had its field disturbed. The serial digital data can be input directly by wire from a multiplexer 96 to the computer 28 located at the site of the player and motion sensor plate 80, or as in the preferred embodiment, illustrated in FIG. 1, the serial data can be transmitted 100 and an antenna 102, included in the motion detector electronic transmitter communication circuitry from FIG. 1.

[0042] The computer 28, under the control of the game system software, will analyze the serial digital club motion signal, recognize from the transmitted signals the platelets 83 over which the club head passed and display the golf club swing motion.

[0043] The motion sensors further comprise spatial orientation devices such as a gyro meter and an accelerometer to derive spatial orientation and or translational acceleration data housed inside or mounted to the golf club, sports implement, or gaming item. A gyroscope or equivalently a gyro meter is hereon and heretofore understood to be, and or comprise, spatial orientation devices, and each of the latter is understood to be included in the former.

[0044] When a game is won, lost, or terminated, the gaming software system generates the appropriate output signals 156 (FIG. 12), displays the player performance information, and resets to initial pre-game conditions. If one player opponent quits the game or is timed out (due to an excessive delay in play) and the remaining player wishes to continue play, the software resumes an internet search for another opponent 152 and 153.

[0045] Using programming as contained in the accompanying microfiche appendix, one skilled in the art can readily accomplish the game programming described. Alternative programming will be apparent as well from the foregoing functional description and the illustrations contained in the appended drawings.

[0046] While a preferred embodiment has been described, it will be appreciated that many variations and modifications in the system, its operation, and its various components may be made without departure from the spirit and scope of invention as set forth in the appended claims.