Device for the determination and analysis of the motor skill and the oculomotor skill of a person

Abstract

The invention relates to a device for the determination and analysis of the motor skill and the oculomotor skill of a person (100), with a headset comprising at least the following components: a display unit (9) for displaying an image to the eyes of a person (100), when the headset is mounted on the head of the person (100); an optical sensor system (3, 4, 6) for estimating the position and shape of an object in three-dimensional space and for estimating the position of the head set in three dimensional space, wherein the optical sensor system (3, 4, 6) is arranged and designed for the detection and registration of the hands and fingers of the person (100); an eye-tracking module (8) that is configured to determine a point of gaze of the person (100) wearing the device. The invention furthermore relates to various methods for using the device.

Claims

1. Method for analyzing the motor skill and the oculomotor skill of a person (100) with a device, comprising the steps of: Estimating a three dimensional surrounding of the person with the device, Displaying an object arranged in the three-dimensional space surrounding the person with the display unit of the device, Determining the position and shape of the hand and the fingers of the person pointing towards the displayed object with the optical sensor system, Determining a pointing direction or a point of contact of the object and a finger of the person from the position and shape of the hand and the fingers, Determining a deviation of the position of the hand or the pointing direction of the fingers and the position of the displayed object, wherein the devise comprises: a headset, a display unit (9) for displaying an image to the eyes of a person (100), when the headset is mounted on the head of the person (100), an optical sensor system (3, 4, 6) for estimating the position and shape of an object in three-dimensional space and for estimating the position of the head set in three dimensional space, wherein the optical sensor system (3, 4, 6) is arranged and designed for the detection and registration of the hands and fingers of the person (100), an eye-tracking module (8) that is configured to determine a point of gaze of the person (100) wearing the device.

2. Method according to claim 1, wherein the method further comprises the steps of: Determining a point of gaze with the eye-tracking module, Determining a deviation between the point of gaze and the objects position.

3. Method according to claim 1, wherein the displayed object is moving along a predefined trajectory, the method further comprises the steps of: Determining a temporospatial deviation between the position of the hand or the pointing direction of the fingers and the displayed object trajectory, Determining a temporospatial deviation between the point of gaze and the displayed object trajectory.

4. Method according to claim 1, wherein the method further comprises the steps of: Providing a predefined minimum step length and a minimum step frequency, Displaying an object to the person with the display unit, when the determined step length and/or step frequency is lower than the predefined minimum step length and/or step frequency, and/or Issuing an audio signal particularly with an audio unit such as a speaker comprised by the headset, when the determined step length and/or step frequency is lower than the predefined minimum step length and/or step frequency.

5. Method for determining eye movements and hand tremor with a device comprising the steps of: Determining the eye movements with the eye-tracking module, Determining the position and shape of the hands of the person wearing the device with the optical sensor system, Comparing the determined eye movement and the determined position and shape of the hands, wherein the devise comprises: a headset, a display unit (9) for displaying an image to the eyes of a person (100), when the headset is mounted on the head of the person (100), an optical sensor system (3, 4, 6) for estimating the position and shape of an object in three-dimensional space and for estimating the position of the head set in three dimensional space, wherein the optical sensor system (3, 4, 6) is arranged and designed for the detection and registration of the hands and fingers of the person (100), an eye-tracking module (8) that is configured to determine a point of gaze of the person (100) wearing the device.

6. Method for determining irregular eye movements and irregular hand movements with a device, comprising the steps of: Determining an eye movement of the person with the eye-tracking module, Determining the position and shape of the hands of the person wearing the headset, Comparing the determined eye movement and the determined movement and/or shape of the hands against one or more predefined temporospatial parameters and/or patterns, wherein the devise comprises: a headset, a display unit (9) for displaying an image to the eyes of a person (100), when the headset is mounted on the head of the person (100), an optical sensor system (3, 4, 6) for estimating the position and shape of an object in three-dimensional space and for estimating the position of the head set in three dimensional space, wherein the optical sensor system (3, 4, 6) is arranged and designed for the detection and registration of the hands and fingers of the person (100), an eye-tracking module (8) that is configured to determine a point of gaze of the person (100) wearing the device.

Description

(1) In the following the invention is illustrated by means of a detailed figure description and exemplary embodiment.

(2) It is shown in

(3) FIG. 1 a schematic drawing of the device according to the invention;

(4) FIG. 2 Illustration of the downward pointing sensor field of view;

(5) FIG. 3 flowchart depicting the parameters estimated by the various methods according to the invention, and how these parameters are determined form the measurement data acquired by the components of the device.

(6) In FIG. 1 the device according to the invention is shown schematically. The device comprises a headset that is worn on the head of the person.

(7) The headset comprises a headband 1 that is configured to stably arrange the headset above the eyes of the person. The headband 1 can be made of rubber or another elastic material. However, also other length-adjustable textile materials are suitable. The headband 1 might comprise a means for adjusting the length.

(8) Speakers 2 are comprised by the headset, wherein the speakers 2 in this example are formed as headphones 2, which plug in the ear. With the headphones 2 audio signals, operator instructions or other sounds can be transmitted to the person.

(9) Furthermore, the device comprises forward oriented optical sensors for estimating the position in three dimensional space; the sensors each can be an IMU.

(10) Additionally the headset comprises forward oriented optical sensors 4 for hand and finger tracking.

(11) Furthermore, an IMU 5 is arranged at the headband 1.

(12) The headband comprises a downward oriented optical sensor 6, also referred to a body sensor, for estimating the position in three-dimensional space and for body, feet and hand tracking.

(13) A microphone or a microphone array 7 for recording the voice of the person is arranged at the headband 1.

(14) An eye tracking module 8 for determining the eyes position is integrated in the headset as well. Also, a display unit 9 is arranged in front of the eyes, so that images or instructions can be displayed to the person wearing the headset.

(15) At the backside of the headset, a computing unit 10 is arranged, that also comprises an IMU 11 and a battery 12.

(16) All electronic and electric components, such as the optical sensors 3, 5 and 6, the eye tracking module 8, the headphones 2 and the display unit 9 are connected to the computing unit 10 and can be controlled by the computing unit 10.

(17) In FIG. 2 a visualization (a side view and a top view) of measurement area (schematically indicated by broken lines) of the 3D body sensor (see FIG. 1 ref. 6) is shown. The 3D body sensor points downwards and is configured to record the surroundings, particularly the floor, the body limbs feet and the hands, all of which are visible to the sensor during normal walk and the floor. Motion signals of the specific body parts can then be derived from these recordings. Hands and feet are visible to the sensor during normal walk. By usage of the surrounding information, speed and body movement data, the spatial foot positions are measured and computed over time. From these foot position signals, gait parameters like step and stride length, as well as the step width can be calculated. Further derived parameters include for example the cadence (steps per minute), the knee amplitude (mean amplitude of knee movements in anterio-posterior direction), the knee amplitude asymmetry (log of the ratio between the knee amplitudes from the smaller body side and the larger body side, see Nantel et al., Gait & Pos 34(2011) 329-333), the maximum stride time (maximum time needed to perform one stride as the mean of both body sides), the coefficient of variation of the stride time, the stride time asymmetry (log of the ratio between stride time from the faster body side and the slower body side, see Nantel et al., Gait & Pos 34(2011) 329-333), the hand amplitude (mean amplitude of hand excursions in anterio-posterior direction), the hand amplitude asymmetry (log of the ratio between the hand amplitudes from the smaller body side and the larger body side, see Nantel et al., Gait & Pos 34(2011) 329-333) and the coefficient of variation of the hand swing time.

(18) Since the 3D body sensor directly provides spatial coordinates of specific body parts, corresponding specific amplitude parameters and relative distances between body parts like width can be computed, in contrast to the IMU accelerator based data, from which only time and frequency parameters can be generated.

(19) FIG. 3 shows a flowchart for the methods according to the invention. The determined parameters and variables can be used in subsequent evaluation methods.

(20) Boxes that exhibit a dotted background refer to components of the device according to the invention, wherein the arrows indicate the transfer and/or provision of measurement data acquired by the respective component.

(21) Arrows between plain boxes (no dotted background) indicate method steps for determining the respective parameter listed in the box or to achieve the result listed in the box the arrow points to.

(22) In case a decision has to be made the box is diamond shaped and lists the decision that has to be made. The various possible outcomes of the decision is indicated by arrows leading away from the diamond shaped box.