Intelligent Attention Rehabilitation System

Abstract

The present invention provides an intelligent attention rehabilitation system, which includes a gaze tracking module, an attention evaluation module, an intelligent computation and program push module, a data storage module, and an assessment and feedback module. In the invention, based on the theoretical CMA, data on attention level is acquired; and the advantages of simple configuration requirements and low cost of the gaze tracking technology, and intelligent computation of AI algorithms are used, to digitally evaluate attention of a subject. Besides, the DQN algorithm is used to realize intelligent push of related attention training guidance and programs, to solve the problem of family attention rehabilitation training under the lack of scientific guidance in the market currently, thereby providing a more scientific and accurate family attention rehabilitation evaluation and training system for the need.

Claims

1. An intelligent attention rehabilitation system, comprising a gaze tracking module, an attention evaluation module, an intelligent computation and program push module, a data storage module, and an assessment and feedback module, wherein the gaze tracking module comprises a server and a camera, the camera is used to acquire information of a facial image, and the server is used to position iris centers of human eyes according to the facial image; the attention evaluation module is used to evaluate focused attention, sustained attention, selective attention, alternating attention, divided attention, and Conners Parent Symptom Questionnaire (PSQ); the intelligent computation and program push module is used to compare and analyze evaluation scores of a subject to norm data in a database by receiving real-time data of the attention evaluation module and the gaze tracking module, and intelligently push an optimal training program through a Deep Q-Network (DQN) algorithm; the data storage module is used to receive data transmitted by the gaze tracking module and the attention evaluation module and data of intelligent training, and upload the data to the database; and the assessment and feedback module is used for an operator of the system to check historical data of all users stored in the data storage module, and/or, to receive specific user information sent by the system.

2. The system according to claim 1, wherein the server of the gaze tracking module is particularly used to: S1, acquire information of a facial image; S2, detect a position of a human face frame by using an Adaboost cascade algorithm; S3, calculate facial feature points by a face alignment algorithm, to acquire an eye area image; S4, perform iris center detection to the eye area image, calculate a gray-scale differential on a circle of an iris image by a calculus operator, and take a maximum value from all differential results, so as to accurately position iris centers of human eyes; S5, perform coordinate positioning of the iris centers, wherein in an equation of $\underset{\left(x_0,y_0,r\right)}{\max }.Math.|\frac{\partial }{\partial r}.Math.\underset{x,y,r}{\oint }.Math.\frac{I\left(x,y\right)}{2.Math..Math.r}.Math.\mathrm{ds}.Math.,$ I (X, Y) is an image array, (x, Y) is the center of a circle, and r is a radius; and S6, acquire information of eye movement data, comprising gaze points, gaze duration, gaze frequency, and the time for gazing a stimulus point for the first time, generate an eye movement score, and transmit the eye movement score to the attention evaluation module and the intelligent computation and program push module in real time.

3. The system according to claim 1, wherein a calculation method of the eye movement score comprises: the time the subject stays at a stimulus point is counted as a score a1, the frequency the subject gazes the stimulus point before completing a task is counted as a score a2, the time the subject gazes the stimulus point for the first time is counted as a score a3, and the scores a1, a2 and a3 are added up to obtain the eye movement score A.

4. The system according to any one of claim 1, wherein the camera is a light-source-free single camera.

5. The system according to any one of claim 2, wherein the camera is a light-source-free single camera.

6. The system according to any one of claim 3, wherein the camera is a light-source-free single camera.

7. The system according to claim 1, wherein the attention evaluation module is used to display one or more preset visual stimulations through a display screen of the sever, and provide a voice prompt required to be completed for the visual stimulations; and the subject completes a corresponding task according to the prompt, so that corresponding attention scores are generated according to task completion degree and time.

8. The system according to claim 7, wherein the attention scores comprise: a focused attention score B: select a specific number or character or symbol from randomly arranged stimuli of the same type within a limited time; a sustained attention score C: delete as many specific targets as possible from randomly arranged stimuli of the same type within a limited time; a selective attention score D: select a specific target from randomly arranged stimuli of various different types within a limited time; an alternating attention score E: alternatively select a specific target, according to a voice prompt, from randomly arranged stimuli of two types within a limited time; a divided attention score F: select a specific target from randomly arranged stimuli of the same type within a limited time, and tick a box if a specific syllable is heard during a task; and the Conners PSQ comprises 48 items, and is completed by the father or the mother of a subject child; a questionnaire score G of 6 factors, comprising conduct problems, learning disorders, psychosomatic problems, impulsivity-hyperactivity, anxiety, and hyperactivity indexes, is obtained according to scores of the Conners PSQ.

9. The system according to claim 1, wherein the intelligent computation and program push module intelligently pushes the optimal training program by using the DQN algorithm: the DQN algorithm continuously extracts data features from the database for learning, and through a large amount of data extraction and learning, the module learns experience and knowledge to realize selection and matching of training programs; the intelligent computation and program push module automatically matches and adjusts difficulty and level of the next task according to a task completion status of the subject in a task, provides a corresponding voice prompt, and conducts special training for project push programs with lower scores.

10. The system according to claim 1, wherein the data storage module is particularly used to expand capacity of the database, which comprises basic data on the attention level of normal children and children with different degrees of ADHD; historical data of each evaluation and training of a user will be stored in a file of the subject; and the same user may directly call the historical data when using the system.

11. The system according to claim 1, wherein the operator of the system is a doctor or a therapist.

12. The system according to claim 11, wherein the specific user information comprises user information selected according to preset sending standards and user scores; the assessment and feedback module is further used to remind the doctor or the therapist to give advice and guidance within a set time.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0053] FIG. 1 is a flowchart of an implementation process of a system according to an embodiment of the disclosure;

[0054] FIG. 2 is a module schematic diagram of a system according to an embodiment of the disclosure; and

[0055] FIG. 3 is a module schematic diagram of an attention evaluation module according to an embodiment of the disclosure.

DETAILED DESCRIPTION OF THE EMBODIMENTS

[0056] To make the implementation purposes, technical solutions and advantages of the disclosure clearer, the disclosure will be further described below in detail by referring to the accompanying drawings.

[0057] Embodiments of the disclosure provide an intelligent attention rehabilitation evaluation and training system, which uses gaze tracking and a Deep Q-Network (DQN) algorithm, and includes a gaze tracking module, an attention evaluation module, an intelligent computation and program push module, a data storage module, and an assessment and feedback module, as shown in FIG. 2.

[0058] The gaze tracking module includes a computer or a tablet computer, a common light-source-free single camera, without invasive devices.

[0059] As shown in FIG. 1, the steps for using the gaze tracking module are as below.

[0060] At S1, information of a facial image is acquired through the common light-source-free single camera.

[0061] At S2, a position of a human face frame is detected by an Adaboost cascade algorithm. (Patent search results: Adaboost was used as the search term to acquire 4536 pieces of data, which directly used Adaboost, and the classifier algorithm was marked.)

[0062] At S3, facial feature points are calculated by a face alignment algorithm (a Supervised Descent Method, SDM), to acquire an eye area image. (Requirements: keep the head basically still or turn the head slightly at the angle less than 30 degrees)

[0063] At S4, iris center detection is performed to the eye area image, a gray-scale differential on a circle of an iris image is calculated by a calculus operator (a Daugman algorithm), and a maximum value is taken from all differential results, so that iris centers of human eyes are accurately positioned. (Requirements: good lighting conditions, and avoid using a template matching method for rough positioning, which has similar effects)

[0064] At S5, coordinate positioning of the iris centers is performed.

[0065] In an equation of

[00002]$\underset{\left(x_0,y_0,r\right)}{\max }.Math.|\frac{\partial }{\partial r}.Math.\underset{x,y,r}{\oint }.Math.\frac{I\left(x,y\right)}{2.Math..Math.r}.Math.\mathrm{ds}.Math.,$

I (X, Y) is an image array, (x, Y) is the center of circle, and r is a radius.

[0066] The source of functions and methods of the above-mentioned gaze tracking module is: “A Gaze Tracking Method Using Geometric Features of the Human Eyes” in Chinese Journal of Image and Graphics, published on June 2019.

[0067] The gaze tracking module acquires, by the above steps, information of eye movement data, including gaze points, gaze duration, gaze frequency, and the time for gazing a stimulus point for the first time, and generates an eye movement score. The eye movement score includes the time the subject stays at the stimulus point counted as a score a1, the frequency the subject gazes the stimulus point before completing a task counted as a score a2, and the time the subject gazes the stimulus point for the first time counted as a score a3; the scores a1, a2 and a3 are added up to obtain the eye movement score A (a1+a2+a3=A); and the eye movement score A is transmitted to the attention evaluation module and the intelligent computation and program push module in real time.

[0068] The attention evaluation module, as shown in FIG. 3, is based on the theory of a Clinical Model of Attention (CMA) that is classic and widely used clinically at present. Evaluation content includes six subtasks of focused attention, sustained attention, selective attention, alternating attention, divided attention, and Conners Parent Symptom Questionnaire (PSQ).

[0069] The evaluation module displays one or more preset visual stimulations through a display screen of the computer or the tablet computer, and provides a voice prompt required to be completed against the visual stimulations, and the subject is required to complete a corresponding task according to the prompt. According to the task completion degree and time, the system automatically generates corresponding attention scores. The attention scores comprise: [0070] (1) A focused attention score B: select a specific number or character or symbols from randomly arranged stimuli of the same type within a limited time. [0071] (2) A sustained attention score C: delete as many specific targets as possible from randomly arranged stimuli of the same type within a limited time. [0072] (3) A selective attention score D: select a specific target from randomly arranged stimuli of various different types within a limited time. [0073] (4) An alternating attention score E: alternatively select a specific target, according to a voice prompt, from randomly arranged stimuli of two types within a limited time. [0074] (5) A divided attention score F: select a specific target from randomly arranged stimuli of the same type within a limited time, and tick a box if a specific syllable is heard during a task. [0075] (6) The Conners PSQ includes 48 items, and is completed by the father or the mother of a subject child. A questionnaire score G of 6 factors, including conduct problems, learning disorders, psychosomatic problems, impulsivity-hyperactivity, anxiety, and hyperactivity indexes, is obtained according to scores of the Conners PSQ.

[0076] After the six subtasks are completed, all the evaluation data is transmitted to the intelligent computation and program push module.

[0077] The intelligent computation and program push module, by receiving real-time data of the attention evaluation module and the gaze tracking module, compares evaluation scores (A to G) of the subject to norm data of the database and analyzes the evaluation scores, and intelligently pushes an optimal training program through the DQN algorithm. The DQN algorithm continuously extracts data features from the database for learning, and through a large amount of data extraction and learning, the module learns experience and knowledge to realize selection and matching of training programs. The intelligent computation and program push module automatically matches and adjusts difficulty and level of the next task according to a task completion status of the subject, provides a corresponding voice prompt, and conducts special training for project push programs with lower scores. For example, when the distance score a3 between a gaze point and a stimulus point of the subject is lower than the norm data, the system of the disclosure will lower the task completion standard; under the voice prompt, a task is considered completed if the gaze point of the subject is within a range of a circle that takes the stimulus as the center of a circle and has the radius of N cm, and the function is improved by gradually reducing the voice prompt and the radius of the circle.

[0078] The data storage module is used to receive data of gaze tracking, attention evaluation and intelligent training, and upload the data to the database to expand capacity of the database, which may include basic data on the attention level of normal children and children with different degrees of ADHD. Historical data of each evaluation and training of a user will be stored in a file of the subject. The same user may directly call the historical data when using the system.

[0079] In use of the assessment and feedback module, an assessment and feedback module entry may be displayed when the system (i.e., a doctor port) is logged in through a specific account (the module is hidden and the entry is not displayed when other common users log in to the system), a doctor or a therapist may directly check historical data of all users in the data storage module through the doctor port. The system will selectively send user information (e.g., user information with low scores, A+B+C+D+E+F+G<XX), according to the preset sending standards and user scores, to the background of the doctor port, to remind the doctor or the therapist to give advice or guidance within 2 working days.

[0080] To sum up, the implementation of the system of the disclosure includes the following steps. [0081] (1) After logging in to the system, the user enters basic information such as name, age, symptom duration and so on. [0082] (2) The gaze tracking module tracks coordinate information of the iris centers of the subject in real time throughout the whole process, to acquire information of eye movement data, including gaze time, gaze frequency, a distance between a gaze point and a stimulus, and the like, and to generate the eye movement score. [0083] (3) The evaluation module gives the visual stimulations, and obtains corresponding attention scores based on the task completion status of the subject and the result of the Conners PSQ. [0084] (4) The intelligent computation and program push module analyzes and compares the evaluation data, automatically generates the optimal training program, and intelligently adjusts the next training program according to the eye movement information and the task completion status during the training process. [0085] (5) The data storage module stores data and uploads the data to the database to expand the capacity of the norm database. When the same user uses the system again, there is no need to evaluate again, and the historical data can be directly called to continue the next training program. [0086] (6) The assessment and feedback module of the system selectively sends user data information to the background of the doctor port according to user scores, and reminds the doctor or the therapist to give advice and guidance within 2 working days.

[0087] According to the disclosure, the intelligent attention rehabilitation evaluation and training system is provided. Evaluation and training indexes may be quantified based on the CMA in combination with the existing gaze tracking technology and the DQN algorithm. The attention index is assigned through the gaze tracking technology and scores of games that the user actively participates in, so that objective quantification of the attention evaluation index may be achieved. Children eye movement information may be dynamically monitored according to characteristics of children. Thus, accuracy and scientificity of attention evaluation and training for children with ADHD will be improved, and the scientific guidance for family training for children with ADHD may also be provided, thereby alleviating the shortage of children rehabilitation medical resources in China to a certain extent.