SYSTEM AND A METHOD FOR DETERMINING POSITIONS OF SENSOR UNITS

Abstract

A system (1) is provided. The system comprises two primary sensor units (10) and two secondary sensor units (20). The secondary sensor units are configured to receive ultrasonic pulses during time windows, wherein a time window of the time windows comprises a corresponding transmit time of predetermined transmit times. The system is configured to determine a time-of-flight of an ultrasonic pulse of the ultrasonic pulses transmitted at a transmit time of the transmit times based on when the ultrasonic pulse was received during the corresponding time window. The system is further configured to determine a distance between two of the sensor units based on the determined time-of-flight between said sensor units. The system is configured to determine the positions of the sensor units in real-time based on measured movements and the determined distances.

Claims

1. A system for determining positions sensor units, comprising two primary sensor units and two secondary sensor units, wherein each sensor unit comprises an inertial measurement unit, IMU, configured to measure movement of the sensor unit; wherein the primary sensor units and a first sensor unit of the secondary sensor units are configured to transmit ultrasonic pulses at predetermined transmit times, wherein the secondary sensor units are configured to receive the ultrasonic pulses during time windows, wherein a time window of the time windows comprises a corresponding transmit time of the predetermined transmit times, and wherein the system is configured to determine a time-of-flight of an ultrasonic pulse of the ultrasonic pulses transmitted at a transmit time of the predetermined transmit times based on when the ultrasonic pulse was received during the corresponding time window; wherein the system is further configured to determine a distance between two of the sensor units, based on the determined time-of-flight between said sensor units; and wherein the system is configured to determine the positions of the sensor units in real-time based on the measured movements and the determined distances.

2. The system according to claim 1, wherein each primary sensor unit comprises a transducer unit configured for ultrasonic transmission; and wherein each secondary sensor unit comprises a transducer unit configured for ultrasonic reception, wherein the transducer unit of the first sensor unit is further configured for ultrasonic transmission.

3. The system according to claim 1, wherein a time window of the time windows begins at a corresponding transmit time of the transmit times.

4. The system according claim 1, wherein each sensor unit comprises a micro-controller unit, MCU, configured for radio communication; and wherein the primary sensor units and the secondary sensor units are configured for time synchronisation via radio communication, wherein the time synchronisation comprises transmitting a synchronisation packet from the first sensor unit to the primary sensor units and a second sensor unit of the secondary sensor units, wherein the synchronisation packet comprises a master time reference; in response to receiving the synchronisation packet, setting an internal clock of the receiving primary sensor unit or the second sensor unit to the master time reference, and transmitting a synchronisation response to the first sensor unit; and in response to receiving a synchronisation response, determining that the primary sensor unit or the second sensor unit which transmitted said synchronisation response is time synchronised.

5. The system according to claim 1, wherein the system is for a virtual drum kit and is further configured to determine that a virtual percussion instrument of the virtual drum kit has been hit based on the determined positions of one sensor unit.

6. The system according to claim 5, wherein the system is further configured to convert the determined virtual percussion instrument hits to Musical Instrument Digital Interface, MIDI, data.

7. The system according to claim 5, wherein the system is configured to transmit at least one of the determined hits and the MIDI data in real-time to an auxiliary device.

8. The system according to claim 7, wherein the first sensor unit further comprises a second MCU configured for wireless communication with the auxiliary device, and wherein the additional MCU is configured to transmit at least one of the determined hits and the MIDI data in real-time to the auxiliary device.

9. The system according to claim 1, wherein the system is further configured for performing an environment characterisation process comprising the steps of: transmitting a characterisation ultrasonic pulse from a primary sensor unit; recording during an echo time window by the secondary sensor units; determining a time-of-flight of a recorded echo of the characterisation ultrasonic pulse recorded during the echo time window; and determining an interval time between the predetermined transmit times such that the interval is different from any determined time-of-flight of a recorded echo.

10. The system according to claim 1, wherein the system is further configured for performing a calibration comprising: transmitting a first calibration ultrasonic pulse from the first sensor unit; receiving the first calibration ultrasonic pulse by a second sensor unit of the secondary sensor units; determining a time-of-flight of the first calibration ultrasonic pulse; transmitting a second calibration ultrasonic pulse from each of the primary sensor units; receiving the second calibration ultrasonic pulses by the secondary sensor units; determining the time-of-flight of each of the calibration ultrasonic pulses to each of the secondary sensor units; and determining distances between the sensor units based on the determined time-of-flights.

11. The system according to claim 10, wherein the calibration further comprises determining, by each IMU of the sensor units, a reference position.

12. The system according to claim 10, wherein the second sensor unit comprises a transducer unit comprising at least two transducer elements, and wherein the first calibration ultrasonic pulse is received by each of the transducer elements, and wherein the calibration further comprises the steps of: determining a time-of-flight of the first calibration ultrasonic pulse for each of the transducer elements, and determining the relative orientation and/or location of the secondary sensor units based on the determined time-of-flights of the first calibration pulse received by the transducer elements.

13. The system according to claim 1, wherein the system is further configured to determine the positions of the sensor units in real-time by using a digital filter, wherein the digital filter is fed with the measured movements and the determined distances.

14. The system according to claim 1, wherein at least one primary sensor unit of the first sensor units further comprises a haptic feedback device.

15. A method for determining positions of sensor units of a system, wherein the system comprises two primary sensor units and two secondary sensor units, wherein each of the sensor units comprises an inertial measurement unit, IMU, configured to measure movement of the sensor unit, wherein the method comprises the steps of: transmitting ultrasonic pulses at predetermined transmit times from a primary sensor unit of the primary sensor units and a first sensor unit of the secondary sensor units; receiving the ultrasonic pulses by the secondary sensor units during time windows, wherein a time window of the time windows comprises a corresponding transmit time of the predetermined transmit times; determining a time-of-flight of an ultrasonic pulse transmitted at a transmit time based on when the ultrasonic pulse was received during the corresponding time window; and determining the positions of the sensor units in real-time based on the measured movements and the determined distances.

Description

BRIEF DESCRIPTION OF DRAWINGS

[0076] Exemplifying embodiments will now be described in more detail, with reference to the following appended drawings:

[0077] FIG. 1 is a schematic illustration of a primary sensor unit and a secondary sensor unit, in accordance with some embodiments;

[0078] FIG. 2 is schematic illustration of a system, in accordance with some embodiments.

[0079] As illustrated in the figures, the sizes of the elements and regions may be exaggerated for illustrative purposes and, thus, are provided to illustrate the general structures of the embodiments. Like reference numerals refer to like elements throughout.

DETAILED DESCRIPTION

[0080] Exemplifying embodiments will now be described more fully hereinafter with reference to the accompanying drawings in which currently preferred embodiments are shown. The present disclosure may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided for thoroughness and completeness, and fully convey the scope of the present disclosure to the skilled person.

[0081] With reference to FIG. 1, a primary sensor unit 10 and a secondary sensor unit 20 of a system (not shown; see FIG. 2), in accordance with some embodiments, will be described. It is to be understood that the system may comprise more than one primary sensor unit 10 and/or more than one secondary sensor unit 20. For example, the system may comprise two primary sensor units 10 and two secondary sensor units 20.

[0082] The primary sensor unit 10 and the secondary sensor unit each comprise a micro-controller unit, MCU, 15, 25. The primary sensor unit 10 and the secondary sensor unit 20 may alternatively comprise a processor or a micro-processor, instead of an MCU 15, 25. The MCUs 15, 25 may be configured for radio communication.

[0083] The primary sensor unit 10 and the secondary sensor unit 20 each comprise an inertial measurement unit, IMU, 31. Each inertial measurement unit 31 may be configured to measure movement of the sensor unit 10, 20. Each IMU 31 is communicatively connected to the MCU 15, 25 of the sensor unit 10, 20.

[0084] The primary sensor unit 10 comprises a transducer unit 13. The transducer unit 13 is configured for ultrasonic transmission. The transducer unit 13 comprises two transducer elements 13′ and a driver circuit. The transducer elements 13′ are arranged in opposite directions. The transducer unit 13 is not limited to comprising two transducer elements 13′, and may comprise, for example, one, two, three, four, or more, transducer elements 13′. The transducer elements 13′ are connected to the MCU 15 via the driver circuit of the transducer unit 13.

[0085] The primary sensor unit 10 further comprises a haptic feedback device 17. The haptic feedback device 17 comprises an actuator 17′ configured to produce haptic feedback, such as vibration, and a driver circuit. The actuator 17′ is connected to the MCU via the driver circuit of the haptic feedback device 17. The present disclosure is not limited to a primary sensor unit 10 comprising a haptic feedback device 17. For example, a system, in accordance with some embodiments, may comprise two primary sensor units 10, wherein one, both, or none of the primary sensor units 10 comprises a haptic feedback device 17.

[0086] The secondary sensor unit 20 comprises two transducer units 23, 24. A first transducer unit 23 of the two transducer units 23, 24 is configured for ultrasonic transmission. The first transducer unit 23 comprises a transducer element 23′ and detector circuit. The transducer element 23′ is connected to the MCU 25 via the converter circuit of the first transducer unit 23. A second transducer unit 24 of the two transducer units 23, 24 is configured for ultrasonic transmission and for ultrasonic reception. The second transducer 24 further comprises a transducer element 24′, a switch, a detector circuit connected and a driver circuit. The detector unit and the driver circuit are configured to be connected to the transducer element 24′ via the switch. The second transducer unit 24 may be configured to be operated in an ultrasonic transmission configuration or in an ultrasonic reception configuration. In the ultrasonic transmission configuration, the switch may be arranged to connect the driver unit of the second transducer unit 24 to the transducer element 24′. Correspondingly, in the ultrasonic reception configuration, the switch may be arranged to connect the detector unit of the second transducer unit 24 to the transducer element 24′. The present disclosure is not limited to a secondary sensor unit 20 comprising two transducer units 23, 24, as illustrated in FIG. 1. For example, a second sensor unit 20 may comprise one or more first transducer units 23, and/or one or more second transducer units 24. Accordingly, a secondary sensor unit 20 may only comprise one or more first transducer units 23, or only one or more second transducer units 24.

[0087] The secondary sensor unit 20 further comprises a second MCU 26. The second MCU 26 is configured for wireless communication. The second MCU 26 is communicatively connected to the MCU 25. The system may comprise one secondary sensor unit 20 which comprises a second MCU 26. Thus, the present disclosure is not limited to a sensor unit 20 comprising a second MCU 26.

[0088] The primary sensor unit 10 and the secondary sensor unit 20 each comprise a rechargeable battery 32, a charger 33, and a charging connector 34. The battery 32 is connected to the charging connector 34 via the charger 33. The charging connector 34 is configured for connecting the corresponding sensor unit 10, 20 to an electrical energy source. The battery 32 is configured to provide the corresponding sensor unit 10, 20 with electrical energy. The battery 32 of the primary sensor unit 10 is electrically connected to the transducer unit 13, the haptic feedback unit 17 and the MCU 15. The battery 32 of the secondary sensor unit 20 is electrically connected to the first transducer unit 23, the second transducer unit, and the MCU 25. The present disclosure is not limited to sensor units 10, 20 comprising a rechargeable battery 32, a charger 33, and a charging connector 34, as shown in FIG. 1. For example, the sensor units 10, 20 may comprise replaceable batteries.

[0089] FIG. 2 is schematic illustration of a system 1 for a virtual drum kit 90, in accordance with some embodiments. FIG. 2 illustrates the system 1 when in use by a user. The perspective in FIG. 2 is from the eyes of the user.

[0090] It should be noted that present disclosure is not limited to a system 1 for a virtual drum kit 90. The present disclosure is provided to facilitate an increased understanding of the inventive concept. The system 1 may alternatively be for, for example, gaming, industrial applications, sports, and/or skills training.

[0091] The system 1 comprises two primary sensor units 10 and two secondary sensor units 20, 21, 22. Each primary sensor unit 10 is arranged inside a drum stick 61. The two drum sticks 61 are held in the user's hands 51.

[0092] A first sensor unit 21 of the two secondary sensor units 20 is arranged on the user's right foot 52. A second sensor unit 22 of the two secondary sensor units 20 is arranged on the user's left foot 52. The secondary sensor units 20, 21, 22 are arranged to the feet of the user by straps 62. The secondary sensor units 20, 21, 22 are arranged on the upper side of the user's feet 52.

[0093] FIG. 2 illustrates a virtual drum kit 90 comprising four virtual percussion instruments 95. The virtual drum kit 90 is shown with dashed lines to indicate that they are not physically present. Rather, the shown virtual drum kit 90 is (virtually) arranged at the shown location. The virtual drum kit 90 is not limited to the shown configuration of virtual percussion instruments 95. For example, the virtual drum kit 90 may comprise substantially any number of virtual percussion instruments 95, such as one, two, three, four, five, six, seven, eight, or more. Further, the virtual percussion instruments 95 are not limited to being virtual drums 95, as shown in FIG. 2. A virtual percussion instrument 95 may be configured as, for example, pedals, snares, hi-hats, bells, or any kind of percussion instrument.

[0094] Each sensor units 10, 20 comprises an IMU (not shown; see FIG. 1) configured to measure movement of the sensor unit 10, 20. The IMU may comprise an accelerometer and/or a gyroscope. The primary sensor units 10 and the second sensor unit 22 may be configured to send the measured movements to the first sensor unit 21 via radio communication. The sensor units 10, 20 may comprise an MCU (not shown; see FIG. 1) configured for radio communication. The measured movements may be sent, i.e. transmitted, by the MCUs. One or more MCU may be configured to apply a digital filter on the measured movements to produce filtered measured movements. For example, the MCUs of the primary sensor units 10 and the second sensor unit 22 may be configured to produce filtered measured movements and to send the filtered measured movements, via radio communication, to the first sensor unit 21. Alternatively, the primary sensor units 10 and the second sensor unit 22 may be configured to send the measured movements, via radio communication, to the first sensor unit 21. The MCU of the first sensor unit 21 may be configured to apply a digital filter to the measured movements received from the primary sensor units 10 and the second sensor unit 22.

[0095] Further, the primary sensor units 10 may be configured to buffer the measured movements before sending the measured movements to the first sensor unit 21. Thus, the required bandwidth may be reduced. The buffering of the measured movements may comprise sending first and last samples of the buffered measured movements. The first and last samples may be used to linearise the buffered measured movements, i.e. determining a line between the first and last samples of the buffered measured movements.

[0096] The IMUs of the primary sensor units 10 may be arranged within the drum sticks 61 such that they are arranged under the hand of a user holding the drum stick 61 in an ordinary fashion. Thus, the primary sensor units 10 may be arranged close to a pivot point of the drum stick 61 when the drum stick 61 is used. Thus, the acceleration of the IMU may be reduced or minimized, which may prevent the IMUs of the primary sensor units 10 from being maxed out when the primary sensor units 10 are used by a user.

[0097] Data recorded by the IMUs of the primary sensor units 10 may be processed, by the MCU of the corresponding primary sensor unit 10, through a gesture recognition filter. The gesture recognition filter may be configured to determine that a user has moved a primary sensor unit 10 in accordance with a predetermined gesture. Upon a determination that a predetermined gesture has been made, the system 1 may perform a number of actions, which may include, for example, switching the virtual drum kit 90, adding audio effects, start and stop recordings, or send preconfigured (MIDI) commands.

[0098] The IMUs of the primary sensor units 10 may be configured to determine the orientation of the drum stick 61 in which it is arranged. The orientation of a drum stick 61 may comprise a roll, a pitch, and/or yaw.

[0099] A primary sensor unit 10 of the system 1 may be configured to determine a hit. A primary sensor unit 10 may determine a hit if a determined pitch and determined yaw of the drum stick 61 are above a predetermined hit threshold. The predetermined hit threshold may be compared to the root of the squared sum of angular velocity of the pitch and yaw. Thus, the roll, measured by the IMU, of the drum stick 61 may be cancelled or disregarded when determining a hit.

[0100] Determining hits by the secondary sensor units 20 requires less precision than determining hits by the primary sensor units 10, as the secondary sensor units 20 are limited by the floor on which the feet 52 is resting on. An accelerometer of the IMU of a secondary sensor unit 20 may be used to determine the distance travelled by the secondary sensor unit 20. Further, the orientation of the secondary sensor unit 20 may be used to determine if toes or a heel of the foot 52 were used for the hit.

[0101] The primary sensor units 10 and the first sensor unit 21 are configured to transmit ultrasonic pulses at transmit times. The primary sensor units 10 and the first sensor unit 21 may each comprise a transducer unit (not shown; see FIG. 1) configured for ultrasonic transmission, wherein the ultrasonic pulses are transmitted by the transducer units. The secondary sensor 20 units are configured to receive the ultrasonic pulses during time windows. The secondary sensor units 20 may each comprise a transducer unit configured for ultrasonic reception, wherein the ultrasonic pulses are received by the transducer units. The first sensor unit 21 may comprise a transducer unit which may be configured for ultrasonic transmission and reception.

[0102] A time window of the time windows comprises a corresponding transmit time of the transmit times. The system 1 is configured to determine a time-of-flight of an ultrasonic pulse of the ultrasonic pulses transmitted at a transmit time of the transmit times based on when the ultrasonic pulse was received during the corresponding time window. Determining a time-of-flight of an ultrasonic pulse may be determined by comparing the transmit time of the ultrasonic pulse with the time when the ultrasonic pulse was received. The second sensor unit 22 may be configured to determine the time-of-flight of an ultrasonic pulse and to transmit the determined time-of-flight to the first sensor unit 21. Alternatively, the second sensor unit 22 may be configured to transmit the time when the ultrasonic pulse was received to the first sensor unit 21 which may then determine the time-of-flight of the ultrasonic pulse.

[0103] The system 1 is further configured to determine distances between the secondary sensor units 20, and between the secondary sensor units 20 and the primary sensor units 10, based on the determined time-of-flights between said sensor units 10, 20. The second sensor unit 22 may be configured to determine the distance from itself to the primary sensor units 22 based on the determined time-of-flights. Alternatively, the first sensor unit 21 may be configured to determine the distances between the sensor units 10, 20 based on the determined time-of-flights, wherein the first sensor unit 21 may receive time-of-flights determined by the second sensor unit 22. In other words, some of the determination of distances may be performed by the second sensor unit 22 which are then sent to the first sensor unit 21, or, alternatively, all of the determination of distances may be performed by the first sensor unit 21.

[0104] The system 1 is configured to determine the positions of the sensor units 10, 20 in real-time based on the measured movements and the determined distances. The first sensor unit 1 may be configured to determine the positions of the sensor units 10, 20 in real-time based on the measured movements and the determined distances.

[0105] The sensor units 10, 20 may be configured for time synchronisation via radio communication, which may ensure that the transmit times and the time windows are time synchronised, thereby increasing the precision of determining the time-of-flights, which may result in more precisely determined distances between the sensor units 10, 20.

[0106] The first sensor unit 21 may be configured as a time synchronisation master, and the other sensor units 10, 22 may be configured as time synchronisation slaves. The sensor units 10, 20 may each comprise an internal clock, wherein the internal clock of the first sensor unit 21 may be understood as a master clock, wherein the master clock may be considered to be correct.

[0107] The time synchronisation may be operating in super cycles. A super cycle period, T.sub.super, may be, for example, 500 ms. At the beginning of a super cycle, the master clock may be reset to zero. The first sensor unit 21, i.e. the master 21, may transmit a synchronisation, SYN, packet, via radio communication, at the beginning of each super cycle. The SYN packet may contain an indication of the master time, T.sub.SYN, and/or which other sensor units 10, 22, i.e. the slaves 10, 22, that may be unsynchronised, and/or if the first sensor unit 21 requests that any of the slaves 10, 22 should respond to the SYN packet.

[0108] Unsynchronised slaves 10, 22 may be constantly listening for the SYN packet. When the SYN packet is received by a slave 10, 22, the slave 10, 22 knows that the correct time, i.e. the time of the master clock, is equal to the master time, T.sub.SYN, minus the time-of-flight of the SYN packet. The slave 10, 22 may respond to the SYN packet with an alive, ALV, packet if the slave was unsynchronised or if the master 21 requested a response. The slaves 10, 22 may be configured to transmit the ALV packet with a predetermined delay after receiving a SYN packet, in order to avoid that ALV packets from the slaves 10, 22 are transmitted at the same time. The master 21 may consider a slave 10, 22 to be synchronised if the slave 10, 22 responds to a SYN packet when requested.

[0109] When a slave 10, 22 receives a SYN packet, the slave 10, 22 may reset its internal clock, and may expect another SYN packet in T.sub.super. A slave 10, 22 may expect another SYN packet in T.sub.super*K.sub.drift, wherein K.sub.drift is the drift of the internal clock of the slave 10, 22. K.sub.drift may be initially be equal to 1 for each slave. A slave 10, 22 may be configured to update K.sub.drift by comparing its internal clock against the received master clock. Further, a the updating of K.sub.drift may be fed through a (digital) low pass filter in order to avoid transient updates.

[0110] The sensor units 10, 20 may be configured to divide the super cycle, T.sub.super, into a number of frames. The master 21 may be configured to transmit the number of frames to the slaves 10, 22. One of the primary sensor units 10 may be configured to transmit ultrasonic pulses at the beginning of every other frame, and the other primary sensor unit 10 may be configured to transmit the ultrasonic pulses at the beginning of the remaining frames. Thus, the sensor units 10, 20 may know when the sensor units 10, 20 are configured to transmit and receive ultrasonic pulses, which may reduce the required bandwidth. Thus, when the sensor units 10, 20 are time-synchronised, the secondary sensor units 20 may therefore only need to keep track of how many frames that have passed when an ultrasonic pulse is received in order to determine a time-of-flight of the ultrasonic pulse.

[0111] Thus, the system 1 may be configured to determine that a virtual percussion instrument 95 of the virtual drum kit 90 has been hit based on the determined positions of one sensor unit 10, 20, wherein the determined movements may be used to determine that a hit has been determined, or detected, and the determined distances may be used to determine which virtual percussion instrument 95 that has been hit.

[0112] The system 1 may be configured to convert determined virtual percussion instrument hits to Musical Instrument Digital Interface, MIDI, data. The MCU of the first sensor unit 1 may be configured to convert the determined virtual percussion instrument hits to MIDI data. The system 1 may be configured to transmit the determined hits and/or the MIDI data in real-time to an auxiliary device 80, wherein the determined hits and/or the MIDI data may be transmitted via a wireless interface, such as BLE. The auxiliary device 80 may be, for example, a smartphone, a tablet, a computer, a speaker, a recording device, or another smart device. The first sensor unit 21 may comprise a second MCU (not shown; see FIG. 1) configured for wireless communication with the auxiliary device 80. The additional MCU may be configured to transmit at least one of the determined hits and the MIDI data in real-time to the auxiliary device 80. However, the system 1 is not limited to transmitting the determined hits and/or the MIDI data to an auxiliary device 80 via a wireless interface. For example, the system 1 may be configured to transmit the determined hits and/or the MIDI data to the auxiliary device 80 via wire or cable.

[0113] The person skilled in the art realizes that the present invention by no means is limited to the preferred embodiments described above. On the contrary, many modifications and variations are possible within the scope of the appended claims.

[0114] Although features and elements are described above in particular combinations, each feature or element can be used alone without the other features and elements or in various combinations with or without other features and elements.

[0115] Additionally, variations to the disclosed embodiments can be understood and effected by the skilled person in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word “comprising” does not exclude other elements, and the indefinite article “a” or “an” does not exclude a plurality. The mere fact that certain features are recited in mutually different dependent claims does not indicate that a combination of these features cannot be used to advantage.