INTERACTIVE BRAIN TRAINER
20170337834 · 2017-11-23
Inventors
Cpc classification
A61B5/002
HUMAN NECESSITIES
G09B5/02
PHYSICS
International classification
G09B5/02
PHYSICS
A61B5/00
HUMAN NECESSITIES
Abstract
A brain training system and method are provided that incorporates a first phase and a second phase of brain training. The first phase is designed to increase attention by requiring the completion of an attention-based activity or task. This may require the identification of match or mismatched images. The second phase is designed to increase working memory by requiring the completion of a working memory-based task that requires the repetition of an illumination sequence. Each phase may be implemented in a self-contained handheld device, similar to the shape of a tablet, or may be implemented in a computer application, such as software. The answers of each phase are recorded in a central database and the system may provide recommendations based on the answers.
Claims
1. A method configured to improve brain functioning comprising the steps of: equipping a test subject with a neurofeedback monitoring device; providing a first task to be completed, wherein the first task is an attention training technique incorporating an image for identification; effectuating the completion of the first task; displaying, in real time, neurofeedback during the completion of the first task to the test subject; measuring a completion time of the first task and recording the completion time in a register; providing a second task to be completed, wherein the second task is a working memory training technique; effectuating the completion of the section task; displaying, in real time, neurofeedback during the completion of the second task to the test subject; measuring a completion time of the second task and recording the completion time in the register; comparing the completion times of the first and second task with predetermined standard completion times; and providing a recommendation of attention and working memory of a test subject based on the comparison of the completion times of the first and second task with predetermined standard completion times, wherein if one of the completion times is less than the predetermined standard completion times then indicating one of the following (i) that the test subject is not improving attention span and (ii) that the test subject is not improving working memory, wherein if both the completion times are greater than or equal to the predetermined standard completion times then indicated that the test subject is increasing attention span and working memory.
2. The method of claim 1, wherein the step of effectuating the completion of the first task is accomplished over a first time frame in a range from one to three weeks.
3. The method of claim 2, wherein the step of effectuating the completion of the second task is accomplished over a second time frame in a range from one to three weeks.
4. The method of claim 1, wherein the step of effectuating the completion of the first task is accomplished over a first time frame in a range from 10 sessions to 30 sessions, wherein in each session is about 20 minutes in length.
5. The method of claim 4, wherein the step of effectuating the completion of the second task is accomplished over a second time frame in a range from 10 sessions to 20 sessions, wherein each session is about 20 minutes in length.
6. The method of claim 5, further comprising: capturing and recording data from only a portion of the session to normalize results with other test subjects.
7. The method of claim 6, wherein only the results from the first ten minutes of a session is captured.
8. The method of claim 1, wherein the first task to be completed is identifying a match/mismatch between first and second images.
9. The method of claim 8, wherein the first and second images are displayed in a self-contained handheld device.
10. The method of claim 8, wherein the first and second images are displayed on a computer screen and a test subject identifies the match/mismatch via a mouse.
11. The method of claim 1, wherein the working memory training technique comprises the steps of: illuminating lights in a first sequence; receiving, via user input, repeated sequential illuminations; determining whether the repeated sequential illuminations matches the first sequence.
12. The method of claim 11, wherein the first sequence and the repeated sequential illuminations are accomplished entirely in a self-contained handheld device.
13. The method of claim 11, wherein the first sequence and the repeated sequential illuminations are accomplished in a computer application controlled via a mouse or a touch screen.
14. The method of claim 1, wherein the method is implemented in a school curriculum and the test subject is a school-age student.
15. The method of claim 1, wherein effectuating the completion of the first task comprises: generating, in a first round of testing, a first visual cue and a second visual cue simultaneously, wherein the first completion time begins being measured from the time the first and second visual cues are provided, wherein the first and second visual cues are one of the following: (i) a matched pair, and (ii) a mismatched pair; receiving a response as to whether first and second cues are a matched pair or mismatched; determining whether the received response is correct, wherein if the response is correct then increasing difficulty in a second round of testing.
16. A system comprising: at least one non-transitory computer readable storage medium having instructions encoded thereon, that when executed by one or more processors, perform operations to train attention and working memory in a test subject, the operations comprising: (i) conduct an attention-based task by presenting first and second images including geometric shapes, wherein the attention-based task requires identification, via user input, as to whether the first and second images match; (ii) conduct a working memory task by presenting sequenced illuminations, wherein the working memory task requires repetition, via user input, of sequenced illuminations; a display for displaying at least one of the attention based game/test and the working memory game/test; and at least one user input device in operative communication with the display.
17. The system of claim 16, wherein the at least one user input device includes a first handheld controller, wherein the display is carried by the first handheld controller; and wherein the first display presents the first and second images during the attention-based task.
18. The system of claim 17, wherein the working memory task is accomplished by the first handheld controller.
19. The system of claim 17, further comprising: a second handheld controller including at least one light for implementing the illumination sequence of the working memory task.
20. The system of claim 16, further comprising: at least one attention indicator including a plurality of lights adapted to be illuminated in response to neurofeedback information generated by the test subject; wherein the plurality of lights are illuminated in real-time so as to allow the test subject to visualize their attention during at least one of (i) the attention-based task and (ii) the working memory-based task; and wherein the lights are positioned below an object shaped in the form of a human brain.
Description
BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS
[0055] A sample embodiment of the disclosure is set forth in the following description, is shown in the drawings and is particularly and distinctly pointed out and set forth in the appended claims. The accompanying drawings, which are fully incorporated herein and constitute a part of the specification, illustrate various examples, methods, and other example embodiments of various aspects of the disclosure. It will be appreciated that the illustrated element boundaries (e.g., boxes, groups of boxes, or other shapes) in the figures represent one example of the boundaries. One of ordinary skill in the art will appreciate that in some examples one element may be designed as multiple elements or that multiple elements may be designed as one element. In some examples, an element shown as an internal component of another element may be implemented as an external component and vice versa. Furthermore, elements may not be drawn to scale.
[0056]
[0057]
[0058]
[0059]
[0060]
[0061]
[0062]
[0063]
[0064]
[0065]
[0066]
[0067]
[0068] Similar numbers refer to similar parts throughout the drawings.
DETAILED DESCRIPTION
[0069] The present disclosure relates generally to a method and system of a brain-computer interface (BCI) utilized to improve attention and working memory by implementing a two-phase approach.
[0070]
[0071] The claimed methods, programmed memory and apparatus are operational with numerous other general purpose or spatial purpose computing system environments or configurations. Examples of well-known computing systems, environments, and/or configurations that may be suitable for use with the invention include, but are not limited to, personal computers, server computers, hand-held or laptop devices, smart phones (such as an Apple iPhone or a Samsung Galaxy or the like), multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.
[0072] The claimed methods, apparatus and programmed memory may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
[0073] With reference to
[0074] Computer 12 typically includes a variety of computer readable media. Computer readable media can be any available media that can be accessed by computer 12 and includes both volatile and nonvolatile media, removable and non-removable media. By way of example, and not limitation, computer readable media may comprise computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. This may also include non-transitory computer readable storage media. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by computer 110. Communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of the any of the above should also be included within the scope of computer readable media.
[0075] The system memory 16 includes computer storage media in the form of volatile and/or nonvolatile memory such as read only memory (ROM) 20 and random access memory (RAM) 22. A basic input/output system 24 (BIOS), including the basic routines that help to transfer information between elements within computer 12, such as during start-up, is typically stored in ROM 20. RAM 22 typically includes data and/or program modules that are immediately accessible to and/or presently being operated on by processing unit 14. By way of example, and not limitation,
[0076] The computer 12 may also include other removable/non-removable, volatile/nonvolatile computer storage media. By way of example only,
[0077] The drives and their associated computer storage media discussed above and illustrated in
[0078] A user may enter commands and information into the computer 12 through input devices such as a keyboard 56 and pointing device 58, commonly referred to as a mouse, trackball or touch pad. Other input devices (not shown) may include a touchscreen, buttons, individual keys, microphone, joystick, game pad, satellite dish, scanner, or the like. These and other input devices are often connected to the processing unit 14 through a user input interface 60 that is coupled to the system bus 18, but may be connected by other interface and bus structures, such as a parallel port, game port or a universal serial bus (USB). A monitor 62 or other type of display device is also connected to the system bus 18 via an interface, such as a video interface 64. In addition to the monitor 62, computers may also include other peripheral output devices such as speakers 66 and printer 68, which may be connected through an output peripheral interface 70.
[0079] The computer 12 may operate in a networked environment using logical connections to one or more remote computers, such as a remote computer 72. The remote computer 72 may be a personal computer, a server, a router, a network PC, a peer device, a smart phone or other common network node, and typically includes many or all of the elements described above relative to the computer 12, although only a memory storage device 74 has been illustrated in
[0080] When used in a LAN networking environment, the computer 12 is connected to the LAN 76 through a network interface or adapter 80. When used in a WAN networking environment, the computer 12 typically includes a modem 82 or other means for establishing communications over the WAN 78, such as the Internet. The modem 82, which may be internal or external, may be connected to the system bus 18 via the user input interface 60, or other appropriate mechanism. In a networked environment, program modules depicted relative to the computer 12, or portions thereof, may be stored in the remote memory storage device. By way of example, and not limitation,
[0081] Now that computing environment 10 has been described, reference is made to the interactive system, method, and article of manufacture for training a brain of the present disclosure, to which the computing environment 10 helps implement.
[0082] The present disclosure relates to a two-phase system for increasing attention (phase one) and working memory (phase two) in test subjects by selectively training the brain. The method of training the brain of present disclosure may be implemented in multiple ways. The two phases of the system of the present disclosure may be implemented in either a hardware device that the test subject (i.e., student) can pick up and hold, or may be implemented in an application run on a computer and displayed in a monitor, such as monitor 62, that the test subject interacts with.
[0083]
[0084] Interaction is the dialog between the test subject and the computer 12, while the interface is the vehicle for that dialog. In the field of human computer interaction, the capacity to provide user (i.e., the test subject) control over the interface is a vital component and provides users with comfort in using the interface. Some exemplary properties of tangible user interfaces are physical interaction, rich feedback, and high levels of practicality. There has been an increase in tangible user interfaces for complex real-world application domains such as learning and health. When designing a tangible physical manipulation interface, the user should feel as if he or she is interacting with the domain rather than with a computer interface, so the user focuses on the tasks instead of the technology.
[0085] In mathematics, tangible teaching instruments (e.g., blocks, rods, board games) can reinforce mathematical understanding of numbers, lines, and other mathematical entities. These instruments can give learners ways to construct physical models of abstract mathematical ideas. When children learn by using an instrument, information is encoded in a different neural location than when they learn by methodology. To design an interface that is effective in supporting children's learning, it is important to understand the limitations and advantages of the physical directive manipulation. The tangible designs may support learning of mathematics by building on the advantages of physical manipulation, while avoiding memorization of mathematical theories, which supports ongoing research into children's use of physical materials to solve numerical problems, and comparative performance using virtual materials. Physical manipulation interfaces are seen as likely candidates to influence advanced user interfaces. User control and responsibility are highly desirable. Traditional computer interfaces frame interaction in terms of “input” and “output”. The computer output is delivered in the form of “digital illustrations”, while the input is obtained from controls such as keyboard and mouse.
[0086]
[0087] New
[0088]
[0089] In one example, a matching task is executed from instructions encoded on a non-transitory computer readable storage medium, that when executed by at least one processor, performs an operation of training subjects on how to pay attention and focus on the task at hand (Phase one 124). The spatial-span step or task (phase two 126) is executed from instructions encoded on a non-transitory computer readable storage medium, that when executed by at least one processor, performs an operation of training subjects to view how long they are paying attention. The goal was to enhance working memory capacity primarily by increasing the amount of visuospatial information to be retained and by observing attention feedback on the screen (non-physical memory brain training matching application) or the LEDs (physical memory-brain training matching application).
[0090] During the matching task (phase one 124), a new task appears on the screen showing two matched or mismatched images. The subject makes a choice to press the match or mismatch button. Time to complete the task is measured from the beginning to the successful completion of the task. When the correct answer is displayed, a green LED lights up and the level for the next task increases by one. The score for the task is also increased by one and the next task begins.
[0091]
[0092]
[0093] The first complex image 136 includes at least one geometric shape, such as a square. Further, the second complex image 138 includes at least one geometric shape, such as a square. Further, in each complex image, the geometric shapes are associated with a specific location. For example, the first and second complex images may generally utilize a 4×4 array wherein each box of the array (16 boxes total) can be illuminated or highlighted to form a geometric square. Thus, when a box of the array is selected, it forms a shape which the test subject identifies relative to other portions of the array. Thus, in addition to each complex image having geometric shapes, the shapes are in specific locations of the image. Thus, when the test subject must identify whether the first and second complex images are “matched,” the test subject must identify the correct number of geometric shapes as well as the correct location/positioning of the shapes in each respective image. In one example, it may be considered critical that the complex images are formed from geometric shapes, and not other deigns or figures, such as faces or geographic landscapes. The present disclosure has found, through empirical evidence that geometric shapes test more effectively to increase attention over other designs or figures, such as faces.
[0094] The attention bar 130 is linked with the neurofeedback device 114 worn on the test subject's 116 head. As the computer 12 receives neurofeedback input from the device 130 worn on the head, it is display in real-time (without perceivable delay to the test subject) on the display in the form of the attention bar 130.
[0095] If the test subject 116 identifies a correct match, the level indicator 144 increases and the complexity of the first and second images increases. For example, in the first round, the first and second images may have one, two or three geometric shapes to identify a match/mismatch. However, in the second round, the first and second images may have three, four, or five geometric shaped to identify a match/mismatch. The task of identifying a match or mismatch of first and second complex images is an attention-based task (phase one) that task requires the user to concentrate or focus attention on complex images. Thus, at the frequency of completing phase one continues, the test subject should be able to improve their attention ability. Furthermore, the period of phase one may be considered critical to the functionality of the system in some implementation, for example, in one embodiment, the test subject should perform the attention-based task for frequency at least once a day for at least 10 minutes at a time, for a period of at least two weeks.
[0096] With respect to
[0097] For each round, a new task with a new set of possible solutions to the task appears. In one embodiment, the images for each round are randomly generated each time, however the code/instructions may indicate that about 50% of the time the images should match, but how they match/what patterns the form to create the match may be random. Since humans are selective in focus of attention, in this step, children were asked to identify whether the images were matched or mismatched.
[0098] Subject(s) 116 make a selection of either match/mismatch using any of the interfaces (either 1. hardware based handheld controller or 2. the application based computer game/program). A similar algorithm or set of instructions is embedded with the application to control the LEDs on or off in the physical handheld controller interface and the attention bar and the level bar in the application-based interface. The algorithm measures and records attention and other EEG waves into the database.
[0099] In one non-limiting example, the test subject 116 is a student in an elementary school class room, the subject 116 is called from the classroom to attend the training in an isolated area in the library. The subject dons and continues to wear the NeuroSky MindSet (i.e. the neurofeedback device 114) that forwards brain wave signals to the software application and logs into the application using a unique username and password executed via processing unit on a computer, such as computer 12 or the like. Upon signing in, the application recognizes the sign in role as a subject or a teacher. A welcome message then appears on the screen and the subject clicks “Start” to begin the session. Two images appear on the computer screen if the subject is using the ATA (Attention Training Application) 128 or on the LCD if using the ATD (handheld Attention Training Device, described in greater detail below) and training begins. The timer 140 is used to document the total amount of time the subject is engaged in each task, with brain computer interface readings captured every second, for example. The software application calculates the average attention every second and records the average attention information into a database (Recommended average reading to really capture an action pattern). The classification system may continuously analyze the incoming brain waves, which are mapped at the conclusion of the subject's session. Student actions control some feature(s) of the running session. The timer captures the amount of time the students spends in each learning session. The amount of time per session is not limited as students can participate as long as they chose. However in one example, the data analysis only captures the first 10 minutes of each session to normalize the session with other test subjects who may have a period of time more than 10 minutes to play the tasks/games. The subject can view their amount of time in each session, level, and score at all times during the training session. Students make their selections by clicking on the buttons to match or mismatch if ATA or by clicking on the yellow or the green push button and in the ATD to indicate match or mismatch. A new task appears as soon as the student completes the current task and the task completion time is recorded to measure the difference between the ending task time and beginning task time. The application automatically calculates the level of each task. If the subject gets two tasks in a row correctly, it automatically moves to the next level. Subjects receive 1 point for each correct answer.
[0100]
[0101] The brainwave feedback indicator 152 is generally indicated in
[0102]
[0103]
[0104] With respect to the attention-based activity/task, each of the ATA and the ATD calculate the level per session, if the test subject answers correctly for two tasks in the row, then they receive 2 points and if they answer only one they get 1 points. Similarly, if the student answers incorrectly, the instructions are encoded to deduct 1 point.
[0105] The system may include a software application and/or hardware direct manipulative interface device, such as a physical memory manipulative device (ATD, WMTD) and and/or a non-physical memory manipulative (ATA, WMTA).
[0106] Three levels of attention threshold are defined in the application as shown in Table 1 and Table 2. While the three levels of attention presented in Table 1 and Table 2 are exemplary, the present disclosure is not limited to the three levels of attention. Other levels could be implemented, for example five levels or ten levels. In the example wherein the attention bar on either the attend-based device or application and the attention bar on either the working memory based-deice or application utilizes LEDs or other lights, one LED or light corresponds to one level. So for example, if the system divides the attention threshold into three levels, then the attention bar consists of three LEDs (one for each level). If the system divides the attention threshold into five levels, then the attention bar consists of five LEDs (one for each level). If the system divides the attention threshold into ten levels, then the attention bar consists of ten LEDs (one for each level), and so on.
[0107] In each instance, the subject 116 wears a noninvasive brain computer interface 114 around their head. For each task level, the subject is challenged to reach the highest attention level (i.e. the third level, or fifth level, or tenth level depending on the total number of LEDs). The attention indicator, which may be a LED strip bar, increases based on the level of attention.
TABLE-US-00001 TABLE 1 The Attention Level and Visual feedback Average Visual Feedback Attention Type Action 0 to 30 LED LEVEL 1 Turn On >=31 and <=60 LED LEVEL 2 Turn On >=61 and <=100 LED LEVEL 3 Turn On
[0108] With respect to attention of the test subject, the present disclosure attempts to maximize Beta waves (>13 Hz) and minimize Theta waves. As the subject completes the attention based task, either with the ATA or ATD, the neurofeedback device worn around the head registers the brain wave frequencies (identified below in Table 2). As the number of Brainwaves in the Beta range are maximized, the threshold levels identified in Table 1 are registered in the systems (either in the device or the computer) and control logic illuminates the appropriate LED or light to identify the attention level.
TABLE-US-00002 TABLE 2 Brain wave frequencies and associated mental states Type Hz = Cycles Per Second Delta <4 Hz Sleep, unconscious; increased in some ADHD, normal or decreased in others Theta 4-7 Hz Drowsiness, unfocused; increased in frontal and central area of brain in ADHD, continues into adulthood Alpha 8-12 Hz Eyes closed, relaxed, but alert; mixed findings, perhaps depending on age and gender Beta >13 Hz Mental activity, concentration; decreased in some but not all ADHD children, may normalize in adults
[0109] As the test subject continues to complete the attention-based task (identify the match/mismatch), a performance indicator (which may be separate and distinct from the attention indicator) can identify the number of correct answers in a row. Three levels of performance threshold are defined in the application indicating the number of times the subject reaches each level, time, and errors (See Table 3 below). The subject uses a touchscreen, trackpad, or mouse to interact with the system when implemented in an application based environment. Alternatively, the subject interacts with a physical button when the brain training system of the present disclosure is implemented in a hardware based environment (i.e., a handheld tablet-like device).
TABLE-US-00003 TABLE 3 Performance Levels and Visual Feedback Correct Answers Visual feedback type Action 0 to 30 LED Level 1 Turn On >=31 and <=60 LED Level 2 Turn On >=61 LED Level 3 Turn On
[0110] The Neurosky application-programming interface is developed to integrate the brain computer interface with software (which may be coded in for example the C# paradigm programming language). The brain computer interface calculates the subject's attention and generates a reading from 0 to 100 during every second of real time with the average response of the headset recorded in the database. An attention threshold is calculated (e.g., Table 1 and Table 3) to control the blue LEDs of the attention indicator in the ATD interface and the attention indicator in the ATA.
[0111] In some example embodiments, there is provided, as noted, a child-friendly physical memory brain training system. The system is configured to have a positive effect on a child's engagement and performance, leading to an increase in attention. The system may include a software application and/or hardware direct manipulative interface device, such as a physical memory manipulative device (ATD {for attention-based tasks} and WMTD {for working memory-based tasks}) and/or a non-physical memory manipulative (ATA {for attention-based tasks} and WMTA {for working memory-based tasks}).
[0112] Phase two 126 of the present disclosure relates to a working memory-based task which occurs subsequent to the attention based task identified above. It has been empirically discovered that the frequency and period of performing phase on (attention-based task) prior to phase two (working memory-based tasks may be critical in improving overall brain function).
[0113] In some example embodiments, a software application (i.e., the WMTA 148) and a hardware physical memory application (i.e., the WMTD 174) are provided to effectuate a working memory-task/activity based on the spatial-span concept. The spatial-span concept is a tool used in clinical neuropsychology to exercise visuospatial working memory. Neuroscience studies have shown that visuospatial memory can be improved through “chunking” strategies. This is another way to take advantage of meaningful knowledge that one already possesses to facilitate retrieval and enhance working memory. In this task, the subject must remember the pattern of the lighted squares and repeat it by using the interface to interact with the application.
[0114] With respect to phase two 126, the subject 116 may have to remember the pattern of the lighted squares and repeat it by using the interface to interact with the application. In one example, there are 16 levels of illumination patters.
[0115] As depicted in
[0116]
[0117] The brain computer interface may connect to a computer via a wired and/or wireless link, such as a Bluetooth link. The MindSet brain computer interface uses Bluetooth to transmit data. The manufacturer designed the headset to be adjustable for various sizes of heads. The forehead sensor arm is designed to fit the forehead comfortably; however, in the current study it was difficult to adjust the arm to fit each child's head. An ordered headset size reduction did not work, so a simple modification to the sensor's position was made by adding an ear clip for better contact with the sensors.
[0118]
[0119] An exemplary feedback of the present disclosure provides a the WMTD device (or the ATD device) which actively incorporated brain neurofeedback of working memory (or attention, as the case may be) in a three dimensional (3D) object, such as the neurofeedback indicator 184. More particularly, the reference to the WMTD and the ATD being “three dimensional” refers to the ability of the test subject to hold the device in their hand and control the object (as opposed to simply interacting with a computer screen). In one implementation, the three dimensional devices. Furthermore, the three dimensional aspect of the present disclosure further includes the keypad on the WMTD to which the test subject interacts with. More particularly, the keypad is formed from a tactilely-pleasing substance so as to stimulate the test subjects sense of touch, as well as their sense of pressure. The keypad is housed and sized to receive lights 180 there beneath. The keypad may be at least partially transparent or translucent to allow the illuminations of the lights 180 beneath the keypad to light up during the course of the repetitive working memory-based activity.
[0120] The neurofeedback indicator 184 may be a three dimensional object shaped in the form of a human brain. When in the shape of a human brain, the indicator 184 rises above the housing 190 in a prominent manner. The indicator 184 may include a plurality of lights positioned beneath the outer surface of the brain shaped object. These lights are in operative communication 12 with the processor that receives brainwave feedback from the device 114 worn by the test subject 116. In this way, the test subject 116 can see their brain activity in real-time in the lights that are positioned in the brain of the indicator 184. The indicator must be at least partially transparent or translucent to enable light from the lights positioned below the brain shaped object to illuminate in real time so as to provide accurate neurofeedback for the test subject as to their real-time (without perceivable delay to the user) brain activity.
[0121] The brain training system of the present disclosure may further comprises a central computer or central database, which may operate within computing environment 10 so as to be considered a second computer 12. In this instance, the central computer stores and registers in a database all data sets of test participants.
[0122] A database program executed by at least one processor of the central processor processes the information on a per test subject basis and can compare each individual test subject to a set of averages (either mean, medians, or modes) of other test subjects. The central server may then provide a recommendation of attention and working memory of a test subject based on the comparison of the completion times of the first and second activity/task with either predetermined standard completion times or average completion times, wherein if one of the completion times is less than the predetermined standard completion time or average of the remaining test subjects then indicating one of the following (i) that the test subject is not improving attention span and (ii) that the test subject is not improving working memory, wherein if both the completion times are greater than or equal to the predetermined standard completion times then indicated that the test subject is increasing attention span and working memory. This may be generally referred to as a “system recommendation.”
[0123] With continued reference to the system recommendations, some exemplary system recommendations may recommend to the teacher or parent after the student has completed either one training session or the entire course. In one example, the system will generate a recommendation report to the parents or teachers. The report may include quantitative data and graphs. The data and graphs may comprise the average attention span of the test subject 116 while working on a task during a pre-assessment phase (i.e., before the first training exercise or phase one 124). The data and graphs may further comprise the attention span of the test subject 116 while working on a task after completing the first training exercise or phase one 124. The data and graphs may further comprise average attention span of the test subject 116 while working on a task after completing a second training or phase two 126. The data and graphs may further comprising average attention span of the test subject while working on a task after completing both trainings (i.e., completing phase one 124 and phase two 126. The data and graphs may further comprise the a “retain score” which is the working memory capacity, after completing each session of phase two 126. The data and graphs may further comprise the average retain score (working memory capacity) of the test subject after completing all sessions (for example, all twenty sessions) of phase two 126. The data and graphs may further comprise the performance of the test subject 116 after completing each session of either phase one 124 or phase two 126. The data and graphs may further comprising the average performance of the test subject after completing all sessions of both phase one and phase two.
[0124] Based on historical data of each test subject, the system develops a predictive model to calculate an overall “BrainStem” Score. The attributes needed to calculate the overall BrainStem score (Brain Waves, Attention score, Working memory capacity, Performance, Time to complete the task, User Actions (Number of clicks, pressing on the buttons), Emotion Score (which may be identified in a level or score from 1-5). In accordance with one aspect, an exemplary advantage of the present disclosure provides a novel approach to build a Working memory, attention capacity assessment system that help assess and evaluate the child's challenges and reduced the effort excreted by teachers and parents.
[0125] The study that led to the development of the brain training system of the present disclosure was guided by two hypotheses. Hypothesis 1: The physical memory brain training matching application disclosed herein will have a better effect on subject engagement and be more efficient and accurate, leading to a better result in training working memory capacity and attention than the non-physical brain training matching application. Hypothesis 2: The intervention will improve learning behavior in the classroom by building awareness among children with poor academic achievement and their understanding of working memory and attention.
[0126] The research protocol and consent, and assent forms were approved by the New Mexico State University Institutional Review Board. Before entering the study, written consent was obtained from the parent or legal guardian of each subject, and written assent was obtained from the subject. The study was conducted in a local elementary school. The selection criteria below was utilized to select subjects with the following criteria: (a) poor attention span and high levels of destructibility, (b) poor academic progress in mathematics, (c) demonstrated difficulty in following instructions, (d) demonstrated problems in combining processing with storage, (e) place-keeping difficulty, and (f) short attention span and highly destructible actions.
[0127] A total of 18 subjects completed the study: Nine used the AT and WM training devices and nine used the AT and WM training application. Subjects were either eight or nine years old, with no significant age difference between groups. A between-group design was adopted. A blind selection was used for group assignment. Each subject was exposed to only one interface for both training steps.
[0128] Subjects were brought to an isolated location in the school library. The setup included a foldable board to isolate the subject to minimize distraction. The subject was seated in front of a laptop that displayed the application and the interface. The subject wore noninvasive BCI, such as device 114.
[0129] This research describes the design of the PM-Brain Training Matching Application. This application was highly rated by 85% of the subjects who participated. Subjects were from three local elementary schools and presentations to teachers at these schools about the brain training application also scored highly. Feedback was collected via a questionnaire. A doctor of neurology reviewed the evaluation and feedback that was received from subjects and teachers.
[0130] The R Project for Statistical Computing, referred to herein as “R” (https://www.r-project.org) is a statistical computing software and was used to analyze collected data. R is an extremely flexible statistical programming language and environment for statistical computing; it is open source and freely available for all mainstream operating systems.
[0131] Attention (AT) data were collected by using the Neurosky brain computer interface; readings ranged from 0 to 100 per second. AT was recorded in the database every second. It was predicted that subjects could increase attention and working memory capacity brainwaves (beta) and suppress drowsiness brainwaves (theta) through training.
[0132] Accuracy (n correct) was recorded in the database to represent the number of correct answers per session. Human interaction interface studies have shown that multimodality speeded task completion by 10%, while users made 36% fewer errors. The level of engagement in the domain of the application and control was studied to enhance task performance (time on task and task errors), as well as user rating. They provide improved support for users' preferred interaction style, since 95% to 100% of users prefer multimodal interaction over unimodal interaction.
[0133] Efficiency, measured as task completion time (TCT), was recorded and calculated per task. This measure was calculated as the differences between task starting time and task ending time, measured in seconds.
[0134] Number of items retained by the subject was recorded for each task completed. This measure was calculated by counting the number of retained items per task and averaging the total per session. For example, if the task was 2, 3, 5 and the subject pressed boxes 2, 3, 5 in the correct order, then the total number of correct items was 3. In the application, visuospatial information can be retained through brainwaves. Subjects' attention feedback was observed based on the screen (WMTA 148) or the LEDs (WMTD 174). Training Session Time (TT) was recorded and calculated as the difference between beginning and ending times of the session. Usability is a key concept in HCl. It is concerned with making systems easy to learn and use. Data on usability (U) were collected via a questionnaire. Many everyday systems and products seem to be designed with little regard for usability, which can lead to frustration on the part of the user. Data on frustration (F) were collected via a questionnaire.
[0135] For each of the eleven response variables, two-sample tests for equal mean (or median) were conducted. To address concerns about statistical assumptions, data were first tested for normality using the Shapiro-Wilk test. Here, a significance level of α=0.10 was used (as opposed to α=0.05) to be conservative in terms of more quickly rejecting the normality assumption so as to lean towards more conservative two-sample tests that do not have the normality assumption. If normality was not rejected, Levene's test for equal variances was conducted at a significance level of α=0.10. Depending on the outcome of Levene's test, either a two-sample t-test assuming equal variances was conducted or the Welch two-sample t-test (for unequal variances) was conducted. If normality was rejected, the nonparametric Fligner-Killeen test for equal variances was conducted, as was a bootstrapped Kolmogorov-Smirnov test for testing that the shapes of the underlying distributions are the same. If neither was rejected, then a Wilcoxon Rank Sum test for equal medians was conducted for the two-sample test. If neither a t-test nor a Wilcoxon test had been appropriate, a randomization test would have been conducted, but this was not necessary with any of the eleven response variables. The results of the analyses are summarized in Tables 4, 5, 6.
TABLE-US-00004 TABLE 4 Data Analysis for the ATA and ATD using match-mismatch task. Mean ATD Mean ATA Attention (AT) 46.67 33.78 Accuracy (N_correct) 0.6856 0.3422 Efficiency, measured as a task 2.34 4.136 completion time (TCT)
TABLE-US-00005 TABLE 5 Data Analysis for the WMTA and WMTD using the spatial span task Mean WMTD Mean WMTA Attention (AT) 49.61 36.83 Accuracy (N_correct) 0.06077 0.048 Efficiency, measured as a task 7.81889 12.62778 completion time (TCT) Mem. Retained 4.05556 2.43889
TABLE-US-00006 TABLE 6 Data Analysis summary Efficiency measured task mem. Completion Attention Accuracy Retained time P-value p-value p-value p-value WMTD and 0.0427 0.1808 0.004457 0.000716 ATD WMTA and 0.002526 0.001769 0.001234 ATA
[0136] The study was concerned with making systems easy to learn and use. A usable system is easy to learn, easy to remember how to use, effective to use, safe to use, and enjoyable to use. Many everyday systems and products seem to be designed with little regard to usability, which can lead frustration, wasted time, and errors. The evaluation questionnaire was designed based on The Fun Sorter and the Again-Again questionnaire (which asks whether the respondent would do the activity again). It asked the subjects to rank items against one or more constructs. This was intended to record their opinions of the technology or activity and to gain a measure of their engagement. 100% of test subjects surface said they would like to participate again when using the physical devices (ATD and WMTD) and the non-physical application based tasks (ATA and WMTA). However, 100% of test subjects indicated that using the physical devices (ATD and WMTD) was helpful in provide attention feedback, whereas only 89% of test subjects indicated that using the non-physical devices (ATA and WMTA) was helpful.
[0137] The goal of this project was to design a system interface that would not cause users to become bored, leading to lack of attention, nor so overtasked and stressed as to miss clues or make decision errors. Since the amount of information processing and decision making required in task performance affects the workload experienced by the user, the workload factor was calculated and analyzed carefully.
[0138] Subjects participated in pre/post intervention activities. Data analysis showed a significant difference in results between the two tests, indicating a clear increase in performance from pretest to posttest. However, due to the small sample size, these results should be viewed with caution.
[0139]
[0140] With continued reference to
[0141] The timeline may further include a pre-assessment phase which is shown generally at 194. The pre-assessment may include an assessment given in the form of a test, or quiz, or questionnaire of the test subjects to identify their level of knowledge pertaining to working memory and attention.
[0142]
[0143] With continued reference to
[0144] In one example, the system of the present disclosure is account-based; not per-student-based. (i.e., each student has an account and can login to any workstation). For example, about ten students can use the same interface. Students may login as a different user and a control unit or central computer would be a station for the teacher. The teacher would see all of the units she has and then she would turn them on with a control computer so they can use it. And all of these are networked wirelessly to one location.
[0145] Feedback 196 monitors the brain and performance activities during phase one 124 and phase two 126. In one example, two weeks of phase one 124 for example of manipulative integrated system attention training, then two weeks of phase two 126 for example of working memory manipulative integrated system. Alternatively, the attention training could be 20 sessions and the working memory training could be 20 session.
[0146] A post assessment is completed subsequent to phase one and phase two. The post assessment is shown generally at 198. Then, the system may report and monitoring the cognitive activities and performance could then be evaluated subsequent to training, shown generally at 200. In one example, the system may be used as part of a student's daily activity (for example, at least 20 minutes a day). This timeline is believed to support the position that enhancing working memory may affect the student's math achievement and performance; this will overcome a major STEM (Science, Technology, Engineering and Mathematics) statistics in American society.
[0147] In accordance with one aspect, one exemplary non-limiting of the present disclosure is that the interactive system, method, and article of manufacture for training a brain provides a brain training system in schools or outside of school that is available, affordable and interactive (3D), fun, non-invasive, easy to use that provides awareness, assessment, treatment and post assessment to overcome the issue that our nation faces regarding Attention Disorder, low Math achievement, and low working memory performance. The system is overall: unique, interactive, fun to use, and easy to use.
[0148]
[0149] From the perspective of the test subject 116, some exemplary method steps include: go to the training area; complete pre-test questionnaire for assessment; place the headset in place and make necessary adjustments to ensure proper fit; sign in with your user name and password; read the welcome message and press “Start”; view the objects on the screen, if the two objects match, press the “Match” button, and if they do not match, press the “MisMatch” button, wherein if a green light displays, the answer is correct and if a red light displays, the answer is incorrect; continue to answer for each exercise until you have completed all exercises; log out of the system; remove the headset; complete post-test questionnaire for assessment; review performance and analysis with teacher; and return to your classroom.
[0150] From the teacher's perspective, some exemplary method steps include prepare the system, set it up, turn it on, get headset ready to use; retrieve the student (or test subject) from classroom; assist student with pre-test questionnaire for assessment; assist student with installing headset and make necessary adjustments to ensure proper fit; assist student with logging into system; assist student as needed in completing exercises; assist student in logging out of system; assist student with removing headset; assist student with post-test questionnaire for assessment; download or receive student performance and analysis along with the system recommendations and review the same; share student performance and analysis with student and answer any questions; provide positive feedback and coaching; take student back to classroom; file student performance analysis appropriately; and clean system and put it away.
[0151] All definitions, as defined and used herein, should be understood to control over dictionary definitions, definitions in documents incorporated by reference, and/or ordinary meanings of the defined terms.
[0152] “Logic”, as used herein, includes but is not limited to hardware, firmware, software and/or combinations of each to perform a function(s) or an action(s), and/or to cause a function or action from another logic, method, and/or system. For example, based on a desired application or needs, logic may include a software controlled microprocessor, discrete logic like a processor (e.g., microprocessor), an application specific integrated circuit (ASIC), a programmed logic device, a memory device containing instructions, an electric device having a memory, or the like. Logic may include one or more gates, combinations of gates, or other circuit components. Logic may also be fully embodied as software. Where multiple logics are described, it may be possible to incorporate the multiple logics into one physical logic. Similarly, where a single logic is described, it may be possible to distribute that single logic between multiple physical logics.
[0153] Furthermore, the logic(s) presented herein for accomplishing various methods of this system may be directed towards improvements in existing computer-centric or internet-centric technology that may not have previous analog versions. The logic(s) may provide specific functionality directly related to structure that addresses and resolves some problems identified herein. The logic(s) may also provide significantly more advantages to solve these problems by providing an exemplary inventive concept as specific logic structure and concordant functionality of the method and system. Furthermore, the logic(s) may also provide specific computer implemented rules that improve on existing technological processes. The logic(s) provided herein extends beyond merely gathering data, analyzing the information, and displaying the results.
[0154] While various inventive embodiments have been described and illustrated herein, those of ordinary skill in the art will readily envision a variety of other means and/or structures for performing the function and/or obtaining the results and/or one or more of the advantages described herein, and each of such variations and/or modifications is deemed to be within the scope of the inventive embodiments described herein. More generally, those skilled in the art will readily appreciate that all parameters, dimensions, materials, and configurations described herein are meant to be exemplary and that the actual parameters, dimensions, materials, and/or configurations will depend upon the specific application or applications for which the inventive teachings is/are used. Those skilled in the art will recognize, or be able to ascertain using no more than routine experimentation, many equivalents to the specific inventive embodiments described herein. It is, therefore, to be understood that the foregoing embodiments are presented by way of example only and that, within the scope of the appended claims and equivalents thereto; inventive embodiments may be practiced otherwise than as specifically described and claimed. Inventive embodiments of the present disclosure are directed to each individual feature, system, article, material, kit, and/or method described herein. In addition, any combination of two or more such features, systems, articles, materials, kits, and/or methods, if such features, systems, articles, materials, kits, and/or methods are not mutually inconsistent, is included within the inventive scope of the present disclosure.
[0155] The above-described embodiments can be implemented in any of numerous ways. For example, embodiments of technology disclosed herein may be implemented using hardware, software, or a combination thereof. When implemented in software, the software code can be executed on any suitable processor or collection of processors, whether provided in a single computer or distributed among multiple computers.
[0156] Further, it should be appreciated that a computer may be embodied in any of a number of forms, such as a rack-mounted computer, a desktop computer, a laptop computer, or a tablet computer. Additionally, a computer may be embedded in a device not generally regarded as a computer but with suitable processing capabilities, including a Personal Digital Assistant (PDA), a smart phone or any other suitable portable or fixed electronic device.
[0157] Also, a computer, such as computer 12, may have one or more input and output devices. These devices can be used, among other things, to present a user interface. Examples of output devices that can be used to provide a user interface include printers or display screens for visual presentation of output and speakers or other sound generating devices for audible presentation of output. Examples of input devices that can be used for a user interface include keyboards, and pointing devices, such as mice, touch pads, and digitizing tablets. As another example, a computer may receive input information through speech recognition or in other audible format.
[0158] Such computers may be interconnected by one or more networks in any suitable form, including a local area network or a wide area network, such as an enterprise network, and intelligent network (IN) or the Internet. Such networks may be based on any suitable technology and may operate according to any suitable protocol and may include wireless networks, wired networks or fiber optic networks.
[0159] The various methods or processes outlined herein may be coded as software that is executable on one or more processors that employ any one of a variety of operating systems or platforms. Additionally, such software may be written using any of a number of suitable programming languages and/or programming or scripting tools, and also may be compiled as executable machine language code or intermediate code that is executed on a framework or virtual machine.
[0160] In this respect, various inventive concepts may be embodied as a computer readable storage medium (or multiple computer readable storage media) (e.g., a computer memory, one or more floppy discs, compact discs, optical discs, magnetic tapes, flash memories, circuit configurations in Field Programmable Gate Arrays or other semiconductor devices, or other non-transitory medium or tangible computer storage medium) encoded with one or more programs that, when executed on one or more computers or other processors, perform methods that implement the various embodiments of the invention discussed above. The computer readable medium or media can be transportable, such that the program or programs stored thereon can be loaded onto one or more different computers or other processors to implement various aspects of the present invention as discussed above.
[0161] The terms “program” or “software” or “algorithm” are used herein in a generic sense to refer to any type of computer code or set of computer-executable instructions that can be employed to program a computer or other processor to implement various aspects of embodiments as discussed above. Additionally, it should be appreciated that according to one aspect, one or more computer programs that when executed perform methods of the present invention need not reside on a single computer or processor, but may be distributed in a modular fashion amongst a number of different computers or processors to implement various aspects of the present invention.
[0162] Computer-executable instructions may be in many forms, such as program modules, executed by one or more computers or other devices. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Typically the functionality of the program modules may be combined or distributed as desired in various embodiments.
[0163] Also, data structures may be stored in computer-readable media in any suitable form. For simplicity of illustration, data structures may be shown to have fields that are related through location in the data structure. Such relationships may likewise be achieved by assigning storage for the fields with locations in a computer-readable medium that convey relationship between the fields. However, any suitable mechanism may be used to establish a relationship between information in fields of a data structure, including through the use of pointers, tags or other mechanisms that establish relationship between data elements.
[0164] Also, various inventive concepts may be embodied as one or more methods, of which an example has been provided. The acts performed as part of the method may be ordered in any suitable way. Accordingly, embodiments may be constructed in which acts are performed in an order different than illustrated, which may include performing some acts simultaneously, even though shown as sequential acts in illustrative embodiments.
[0165] The indefinite articles “a” and “an,” as used herein in the specification and in the claims, unless clearly indicated to the contrary, should be understood to mean “at least one.” The phrase “and/or,” as used herein in the specification and in the claims (if at all), should be understood to mean “either or both” of the elements so conjoined, i.e., elements that are conjunctively present in some cases and disjunctively present in other cases. Multiple elements listed with “and/or” should be construed in the same fashion, i.e., “one or more” of the elements so conjoined. Other elements may optionally be present other than the elements specifically identified by the “and/or” clause, whether related or unrelated to those elements specifically identified. Thus, as a non-limiting example, a reference to “A and/or B”, when used in conjunction with open-ended language such as “comprising” can refer, in one embodiment, to A only (optionally including elements other than B); in another embodiment, to B only (optionally including elements other than A); in yet another embodiment, to both A and B (optionally including other elements); etc. As used herein in the specification and in the claims, “or” should be understood to have the same meaning as “and/or” as defined above. For example, when separating items in a list, “or” or “and/or” shall be interpreted as being inclusive, i.e., the inclusion of at least one, but also including more than one, of a number or list of elements, and, optionally, additional unlisted items. Only terms clearly indicated to the contrary, such as “only one of” or “exactly one of,” or, when used in the claims, “consisting of,” will refer to the inclusion of exactly one element of a number or list of elements. In general, the term “or” as used herein shall only be interpreted as indicating exclusive alternatives (i.e. “one or the other but not both”) when preceded by terms of exclusivity, such as “either,” “one of,” “only one of,” or “exactly one of.” “Consisting essentially of,” when used in the claims, shall have its ordinary meaning as used in the field of patent law.
[0166] As used herein in the specification and in the claims, the phrase “at least one,” in reference to a list of one or more elements, should be understood to mean at least one element selected from any one or more of the elements in the list of elements, but not necessarily including at least one of each and every element specifically listed within the list of elements and not excluding any combinations of elements in the list of elements. This definition also allows that elements may optionally be present other than the elements specifically identified within the list of elements to which the phrase “at least one” refers, whether related or unrelated to those elements specifically identified. Thus, as a non-limiting example, “at least one of A and B” (or, equivalently, “at least one of A or B,” or, equivalently “at least one of A and/or B”) can refer, in one embodiment, to at least one, optionally including more than one, A, with no B present (and optionally including elements other than B); in another embodiment, to at least one, optionally including more than one, B, with no A present (and optionally including elements other than A); in yet another embodiment, to at least one, optionally including more than one, A, and at least one, optionally including more than one, B (and optionally including other elements); etc.
[0167] In the claims, as well as in the specification above, all transitional phrases such as “comprising,” “including,” “carrying,” “having,” “containing,” “involving,” “holding,” “composed of,” and the like are to be understood to be open-ended, i.e., to mean including but not limited to. Only the transitional phrases “consisting of” and “consisting essentially of” shall be closed or semi-closed transitional phrases, respectively, as set forth in the United States Patent Office Manual of Patent Examining Procedures.
[0168] An embodiment is an implementation or example of the present disclosure. Reference in the specification to “an embodiment,” “one embodiment,” “some embodiments,” “one particular embodiment,” or “other embodiments,” or the like, means that a particular feature, structure, or characteristic described in connection with the embodiments is included in at least some embodiments, but not necessarily all embodiments, of the invention. The various appearances “an embodiment,” “one embodiment,” “some embodiments,” “one particular embodiment,” or “other embodiments,” or the like, are not necessarily all referring to the same embodiments.
[0169] If this specification states a component, feature, structure, or characteristic “may”, “might”, or “could” be included, that particular component, feature, structure, or characteristic is not required to be included. If the specification or claim refers to “a” or “an” element, that does not mean there is only one of the element. If the specification or claims refer to “an additional” element, that does not preclude there being more than one of the additional element.
[0170] In the foregoing description, certain terms have been used for brevity, clearness, and understanding. No unnecessary limitations are to be implied therefrom beyond the requirement of the prior art because such terms are used for descriptive purposes and are intended to be broadly construed.
[0171] Moreover, the description and illustration of the preferred embodiment of the disclosure are an example and the disclosure is not limited to the exact details shown or described.