ANALYZING BRAIN FUNCTIONING USING BEHAVIORAL EVENT MARKERS FROM PORTABLE ELECTRONIC DEVICE

Abstract

A method of monitoring a user's neuronal activity (10) includes recording a user's behavioural output with a portable electronic device, such as a mobile cellphone. The user's behavioural output (14) is compared (18) with predefined behavioural outputs associated with known event-related neuronal activations (20). The user's event-related neuronal activation (22) is determined based upon the comparison so as to provide an indication of the user's neuronal activity.

Claims

1. A method of monitoring a user's neuronal activity, the method comprising recording a user's behavioural output with a portable electronic device, such as a mobile cellphone, comparing the user's behavioural output with predefined behavioural outputs associated with known event-related neuronal activations; and determining the user's event-related neuronal activation based upon the comparison so as to provide an indication of the user's neuronal activity.

2. The method of claim 1, comprising recording a plurality of behavioural outputs of the user; and using the plurality of behavioural outputs to determine the neuronal activity of the user.

3. The method of claim 1, comprising determining the user's neuronal activity over a period of time, sequentially, to identify a development in the user's neuronal activity.

4. The method of claim 1, comprising associating the user's neuronal activity with one or more of: physical wellbeing; mental wellbeing; physical development/s; mental development/s; treatment; disease; diagnosis.

5. The method of claim 1, comprising comparing the user's behavioural output with an event-related neuronal activation matching a pattern associated with a behavioural output recorder with known patterns to identify an event-related neuronal activation; and the known patterns are previously established using a neuronal recorder.

6. The method of claim 1, wherein the method comprises compiling a database of a plurality of neuronal activities and corresponding behavioural outputs.

7. The method of claim 6, wherein the method comprises compiling the database in advance of performing the monitoring of the user's neuronal activity.

8. The method of claim 6, wherein the method comprises pattern matching of behavioural output with neuronal activity to enable identification of one of behavioural output or neuronal activity based on only one of the other of neuronal activity or behavioural output.

9. The method of claim 6, wherein the database enables the identification of neuronal activity based solely on recording or observing behavioural output, without requiring direct neuronal recording.

10. The method of claim 1, wherein the method does not comprise synchronising the behavioural output recorder and the neuronal activity recorder.

11. The method of claim 1, wherein the method comprises the asynchronous recording of behavioural output and neuronal activity.

12. The method of claim 1, wherein the event-related neuronal activation is associated with a provision of an input towards the portable electronic device.

13. The method of claim 12, wherein the input comprises one or more of: a gesture; a touch; a sound input, such as voice command; a sequence; a series.

14. The method of claim 1, wherein the method comprises a method of diagnosis, the user comprising a patient.

15. The method of claim 1, wherein the monitoring comprises assessing the user's cognitive function.

16. A non-transitory computer readable carrier medium carrying computer readable code to carry out the method of claim 1.

17. A computer program product executable on a processor so as to implement the method of claim 1.

18. A non-transitory computer readable medium loaded with the computer program product of claim 17.

19. A processor arranged to implement the method of claim 1.

20. A system comprising the portable electronic device of claim 1 any and a computer program product executable on a processor.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0035] FIG. 1 shows an example of a method of compiling a database, with event-related neuronal activations.

[0036] FIG. 2 shows an example of a method of monitoring a user's neuronal activity, using a behavioural output recorder without requiring a neuronal recorder.

[0037] FIG. 3 shows the SmRP of touchscreen events with the rapid engagement of distinct cortical processes surrounding the events. (a) Shows SmRP isolated by aligning the touchscreen events while users used their own smartphones with the recorded EEG signals. (b) Shows the time-course of the population of grand average of signals detected over the scalp from select electrodes and the corresponding standard error of the mean. (c) Shows the topology of grand average signals detected over the scalp. (d) Shows the corresponding results of one-sample t-tests. (e) Shows the latency to the statistically significant signal onsets. (f) Shows the offsets to the statistically significant signal onsets.

[0038] FIG. 4 shows how the signals over the sensorimotor cortex are depressed when on social apps vs. non-social apps. (a) Shows population grand average of the kinematic profiles of the thumb movements used towards social and non-social Apps. The shaded area depicts the standard error of the mean. (b) Show the time-course of the population average of the regression-adjusted—for trial-to-trial kinematic variation—EEG signals. Insert shows the un-adjusted signals; the shaded are depicted the standard error. (c) Shows the topology of population means of adjusted signals and the outcomes of the paired t-test comparing SmRP gathered from the social vs. non-social Apps.

[0039] FIG. 5 shows how the signals over the sensorimotor cortex are over-turned for ‘air touches’ vs. real touchscreen touches. (a) Shows population grand average of the kinematic profiles of the thumb movements during ‘air touches’, i.e., when the thumb flexion occurs without a touchscreen event, and real touchscreen events. (b) Shows the time course of the population average EEG signals recorded over the sensorimotor cortex. (c) Shows the topology of the grand average signals detected over the scalp for the air touches vs. touchscreen touches, and the outcomes of the paired t-test comparing the two event types.

DETAILED DESCRIPTION

[0040] FIG. 1 illustrates an example of a method 10 according to the present disclosure. The method 10 shown here comprises compiling a database 20 of a plurality of neuronal activities and corresponding behavioural outputs. The method 10 comprises compiling a database of event-related neuronal activations 22, here being a database of previously-recorded or observed event-related neuronal activations. The method comprises a pre-process collation of unmatched data 16 from a neuronal recorder 12 and a behavioural output recorder 14. The method 10 comprises compiling the database 20 by matching 18 data from the behavioural output recorder 14 and the neuronal recorder 12. The data matching comprises pattern matching 18 to identify event-related neuronal activations 22, the events being associated with recorded behavioural outputs. In at least some examples, the pattern matching 18 comprises synchronised time-based pattern matching. For example, the database 20 compilation includes synchronous, synchronised neuronal and behavioural output recordings. Accordingly, an event 22 is identified from a behavioural output recording 14 and a corresponding neuronal activation identified based at least partially upon identification of recorded neuronal activity at a corresponding recorded time or within a corresponding time window or interval. For example, the neuronal activity associated with the behavioural output is often instigated or identified as being initiated in advance of the behavioural output, such as by a time interval associated with a lag or delay between recorded neuronal activity 12 to instigate motor control and the recorded behavioural output 14 caused by the motor control. It will be appreciated that the method 10 of FIG. 1 can be supplemented or improved, such as with subsequent iterations or steps to expand and/or refine the database 20. For example, the patterns identified herein can be further improved with increasing data. More details in the patterns identified herein may emerge as the data collection pipeline improves. For instance, a small peak that can occur at roughly 50 ms after a touch (capturing the touch-related brain activity) may not show up with the presently-illustrated method (e.g. present resolution), but can be included in the method 10 and database 20 subsequently (e.g. with increased resolution or refinement).

[0041] FIG. 2 illustrates an example of a method 110 according to the present disclosure. The method is provided for monitoring a user's neuronal activity. The method comprises recording a user's behavioural output 114 with a behavioural output recorder, typically a portable electronic device, such as a mobile cellphone. The method 110 comprises comparing the user's behavioural output with predefined behavioural outputs associated with known event-related neuronal activations. As shown here, the method comprises utilising pattern recognition 118 of the behavioural output and pattern matching with the pattern database 120 to determine 122 the user's event-related neuronal activation based upon the comparison so as to provide an indication of the user's neuronal activity. Accordingly, the neuronal activation 122 is effectively modelled in the method of FIG. 2, being inferred or derived without direct measurement of neuronal activity with a neuronal recorder as such.

[0042] In at least some examples, the method comprises recording a plurality of behavioural outputs of the user; and using the plurality of behavioural outputs to determine the neuronal activity of the user. The method comprises determining the user's neuronal activity over a period of time, sequentially, to identify a development in the user's neuronal activity. The method comprises associating the user's neuronal activity with one or more of: physical wellbeing; mental wellbeing; physical development/s; mental development/s; treatment; disease; diagnosis. The method comprises comparing the user's behavioural output with an event-related neuronal activation matching a pattern associated with a behavioural output recorder with known patterns to identify an event-related neuronal activation; and the known patterns are previously established using a neuronal recorder. The method comprises compiling a database of a plurality of neuronal activities and corresponding behavioural outputs. It will be appreciated that the method comprises compiling the database in advance of performing the monitoring of the user's neuronal activity, such as with the method as shown in FIG. 1. T will be appreciated that the database 120 shown in FIG. 2 may be the same as the database 20 developed or established in the method of FIG. 1. The method comprises pattern matching of behavioural output with neuronal activity to enable identification of one of behavioural output or neuronal activity based on only one of the other of neuronal activity or behavioural output. The database 120 enables the identification of neuronal activity based solely on recording or observing behavioural output, without requiring direct neuronal recording. Here, the method of FIG. 2 does not comprise synchronising the behavioural output recorder and the neuronal activity recorder. The method comprises the asynchronous recording of behavioural output and neuronal activity.

[0043] As will be described in more detail below, the event-related neuronal activation is associated with a provision of an input towards the portable electronic device. The input comprises one or more of: a gesture; a touch; a sound input, such as voice command; a sequence; a series. In at least some examples, the method comprises a method of diagnosis, the user comprising a patient. Similarly, in at least some examples (potentially overlapping examples), the monitoring comprises assessing the user's cognitive function. For example, cognitive tasks may involve the processing of salient information regardless of the modality used for the inputs. The MP of an individual may be depressed by an increased cognitive load with no overt motor impact. Emotionally laden stimuli may depress MP related signals. Accordingly, assessment of the behavioural output (via the input to the device) may provide indications of the user's cognitive function.

[0044] FIG. 3 illustrates a method of analyzing a functioning of neuronal circuits of a brain of an individual, which circuits are engaged in the individual's voluntary, self-paced motor control of a button press with a finger, the method comprising: measuring the smartphone related potential “SmRP” of the brain of the individual when the individual uses, particularly with the individual's thumb, a touch screen of a smartphone; and then comparing the measured SmRP of the brain of the individual with standard measured values of SmRP of brains of other individuals when the other individuals use, particularly with the other individual's thumbs, touch screens of smartphones. In “a” of FIG. 3, a series of sequential ‘phone taps’ is shown (as dots) over a time interval, along with the corresponding EEG readings (in u).

[0045] As used herein, the term “smartphone related potential” or SmRP preferably means one or more, preferably all, of the following: the readiness potential (“RP”), the motor potential (“MP”), the reafferent potential (“RAP”) of the brain of an individual, the consecutive post movement sensory processing involving the tactile, visual, frontal & parietal electrodes.

[0046] The method involves comparing the smartphone related potential (“SmRP”) of an individual's touchscreen events with the rapid engagement of distinct cortical processes of the individual surrounding the events.

[0047] Initially, SmRP is measured prior to and following any touchscreen event by the individual. For this purpose, EEG signals of the individual are measured while the individual is engaged in spontaneous right-handed (thumb) touchscreen touches on his/her own smartphone to reveal the neuronal activity surrounding the touchscreen event. The population median of inter-touch intervals of the analyzed events can be 2 s (a 700 ms inter-touch interval cutoff can be used to eliminate the fast touchscreen events). It is estimated that the EEG signal population average to capture the statistically significant deviations from a 1 s long baseline starting at 4 s prior to the touch. A flat recording can persist for up to 704 ms prior to the touch and the earliest signal can be detected at the right parietal and occipital electrodes (FIG. 3). According to the population average, this posterior positive signal can be briefly followed by the simultaneous activation of the frontal (negative) and the parietal & occipital (positive) electrodes. The gap seen at signal onsets between the posterior and anterior electrodes can also be apparent at the corresponding signal peaks. By 400 ms prior to the touch, the negative signals over the contralateral (left) sensorimotor cortex can dominate the topology. At the time of the touchscreen event (0 ms from the event), the negative signals can additionally occupy the parietal and occipital electrodes bilaterally.

[0048] Then, SmRP is measured after a touchscreen event by the individual. With the touchscreen event, the signals over the sensorimotor cortex can begin to reverse from the negativity. The bilateral negative components over the parietal and occipital electrodes, which can develop prior to the touch, can peak in the first 100 ms after the touch (FIG. 3). In the grand average signal, the negative peak latency can be the shortest over the sensorimotor cortex followed by the frontal electrodes and then the parietal electrodes. In the subsequent 200 ms, these negative components can be entirely replaced by a distinct positive component occupying the central and the frontal electrodes. By 400 ms after the touchscreen event, the positive component can occupy the central and parietal electrodes. This propagation towards the posterior electrodes can continue with activation over the parietal and occipital electrodes at 600 ms. This pattern of sequential activation from the frontal-to-occipital electrodes can also be apparent in the latency to the signal peaks. Although this wave of activation can subside by 700 ms, the signals over the left sensorimotor cortex can remain higher than the baseline until 1995 ms after the touchscreen event.

[0049] The variations in the amplitude of the negative sensorimotor signal detected before the touch between different individuals shows that the pre-touch negativity can be correlated with the post-touch activity. The pre-touch activity can be correlated almost exclusively over the parietal & occipital electrodes. The higher the amplitude of pre-touch activity the larger is the positive component over the parietal and occipital electrodes between 200-600 ms.

[0050] The effect on pre-touch neuronal activity of an individual between social and non-social Apps is also measured. Apart from measuring the brain signals of the individual, the thumb flexion and the extension of the thumb of the individual are also preferably measured, preferably by using bend sensor recordings (see e.g. “a” of FIGS. 4 and 5). Kinematically, the amplitudes of the thumb movements are generally similar but with a tendency for the movements being of higher amplitudes when using non-social Apps compared to social Apps (the differences are not generally statistically significant after multiple comparison correction (FIG. 4a). For either category, the touchscreen events generally start with a brief thumb extension and a descent towards the screen (flexion) at .sup.˜600 ms prior to the touch (based on the population mean, 619 ms for social Apps and 571 ms for non-social Apps). After the touch, the thumb is generally more rapidly withdrawn from the screen than the descent towards the screen reaching the maximum flexion already at .sup.˜400 ms (based on the population mean, 341 ms for social Apps and 426 ms for non-social Apps). The SmRPs of individuals are found to differ according to the behavioral context. In this regard, the pre-touch SmRPs over the sensorimotor cortex are depressed when engaged in social vs. non-social Apps in terms of signal amplitude (FIG. 4c). The reduced signal amplitude is apparent at 500 ms before the touchscreen event. The differences mainly occur in the electrodes over the sensorimotor cortex, but the negative components engaging the left parietal and occipital electrodes are also depressed, and this depression can last for up to 100 ms after the touch. The depressed sensorimotor negativity is also present in the analysis of the kinematically unadjusted potentials.

[0051] The effects of ‘air touches’ on SmRP is also measured. In this regard, while individuals are using their smartphone, their thumbs generally are at times flexed towards the screen without resulting in any touchscreen event (FIG. 5a). These ‘air touches’ account for a mean of 31.66% (±3.0% SE) of all the thumb flexions towards the screen. Indeed, ‘air touches’ have been found to be inversely proportional to the number of real touchscreen events (β=−0.0004, R2=0.667, p=2.01×10−07, t=−7, linear regression analysis). As real touchscreen events occur at maximum thumb flexions, EEG analysis can be correlated to the maximum flexions. Both the “air touches” and the real touchscreen events are then seen to share similar movement profiles, starting with a thumb extension and then a flexion towards the screen (at 437 ms before the air touch, based on the population mean) followed by withdrawal (extension) away from the screen. However, the final extension for the “air touches” is not as extensive as for the real touchscreen touches.

[0052] The SmRP prior to an “air touch” is also compared to the SmRP prior to an actual touch, starting from 680 ms prior to the touch (FIG. 5b & c). Significantly, an actual touch yields a strong pre-touch negative component over the sensorimotor cortex while an air touch yields a positive component over the sensorimotor cortex. The positive component peaks at 487 ms (based on the population mean) over the sensorimotor electrodes before the air touch. The method of measuring the different SmRP potentials, i.e., the RP, the MP and the RAP associated with a touchscreen event involving a smartphone shows that a series of neuronal activations are generally involved. In this regard, a touch is preceded by posterior-to-anterior EEG signal flow and strong activation of the sensorimotor cortex. It is followed by an opposite anterior- to-posterior signal flow, unraveling the distinct directions of cortical information flow associated with touching the screen and processing the consequences of the touch respectively. The activation of the sensorimotor cortex is strongly modulated by the behavioral context in terms of the App in use and the near-term consequences of the thumb movements (as in if a movement was followed by touch or not).

[0053] It has been found, by this method, that touchscreen movements by an individual are rapidly prepared and that the crucial decision by the individual to touch or not to touch the screen of a smartphone can occur with movement initiation. The first consistently visible signals before the touchscreen event are detected over the frontal and parietal (and occipital) electrodes about 700 ms before the touchscreen event, while the thumb was already extended to descend towards the screen at .sup.˜600 ms before the touch. Such frontal-parietal signals seen prior to dominant negativity over the sensorimotor cortex are associated with visuomotor attention and response selection. This suggests that neuronal activity precedes the movements by only 100 ms-20× faster than the 2 s preparatory time observed in slow laboratory finger tapping tasks. However, an extended thumb does not always lead to a touchscreen event and such an event is accompanied with a distinct positive component over the sensorimotor cortex starting at .sup.˜700 ms prior to the ‘air touch’. Therefore, the decision process underlying a touchscreen event and the motor control processes can be highly compressed in time on the smartphone. Although screen touches separated by 2 s are common, more rapid are frequent, separated by less than 500 ms (median).

[0054] It has also been found that the pre-touch negativity over the sensorimotor cortex is depressed when engaged in social Apps compared to the non-social Apps. This difference is apparent from .sup.˜500 ms prior to the touch up to .sup.˜100 ms after the touch, even when any contribution of motor amplitude fluctuations are regressed out at the level of individual trials. This suggests that the sensorimotor computations remain connected to the behavioural context through much of the ongoing action.

[0055] The SmRP potentials, measured by this method, suggest the following. Firstly, a touch is followed by continued negativity over the contralateral sensorimotor cortex and enlargement of the negativity bilaterally to occupy the parietal and occipital electrodes. These signals most likely reflect action monitoring and tactile-visual confirmation of the touchscreen event. Secondly, there is a positive component that sequentially recruited the anterior to the posterior electrodes. This pattern of activity and the signal latency is consistent with the P3 (P300) component which reflects the attention and memory-related brain. This wave is commonly observed in cognitive tasks involving the processing of salient information regardless of the modality used for the inputs. The neuronal generators underlying this signal likely inhibit extraneous information flow and thus enhance the flow of information from an attention-grabbing input from the frontal to the parietal cortical structures to ‘sharpen memory. The post-touch cognitive processing may, in fact, be shaped by the pre-touch sensorimotor activity, as the amplitude of the sensorimotor signals recorded prior to the touch was correlated to the post-touch activity over the parietal and occipital electrodes.

[0056] Accordingly, it can be seen that patterns matching behavioural output with neuronal activity can be identified. Such patterns can be stored in a database. For example, particular touches associated with particular smartphone functions can be matched to particular neuronal activities. These patterns can then be used, such as in the method of FIG. 2, to identify event-based neuronal activations based purely on the behavioural output. Accordingly, the smartphone can be used as a behavioural output recorder, such as with software (e.g. a background app) recording behavioural output of the user. The behavioural output can be matched, either realtime or subsequently, to patterns in the database to identify the user's neuronal activity. Particularly over a longer period of time, developments in a user's neuronal activity can be identified. For example, temporal changes in a user's event-related neuronal activations may be identified. Such changes may be associated with improvement and/or deterioration of a user. For example, where a user is a patient, such as a neurological patient, changes in neuronal activity over a period of time may be identified based solely, or at least predominantly, on smartphone use—particularly, normal daily unprompted smartphone use by the user in loco. Such changes may be associated or associatable with physical or mental changes, such as of health. Accordingly, the changes may represent an improvement, such as recovery or healing; or the changes may represent a deterioration, such as a medical setback. Accordingly, the behavioural output as monitored by the smartphone app may act as a trigger to take action, such as to request or instigate a consultation or laboratory analysis or follow-up with the user.

[0057] The measured SmRP of the brain of an individual can be compared with standard measured values of SmRPs of brains of other individuals when the other individuals use, particularly with the other individual's thumbs, touch screens of smartphones. From this comparison, one can readily analyse one or more of the individual's wellbeing, health, preferences, predilections, fears, biases, loyalties and the like.

[0058] It will be appreciated that the method can be we can include estimates of ‘accuracy’ of alignment, and optionally distinguish outcomes in healthy and diseased individuals. It will also be appreciated that the exact detailed shape of the SmRP may be refined. For example, further peaks and valleys may be revealed or identified as the database 20 grows. In at least some examples, the coarse features as shown here may remain stable but the finer features may be altered.

[0059] With over 20% of the global population on smartphones, the touchscreen movements are one of the most common actions and yet one of the simplest in appearance. Understanding how these actions are generated can not only reveal fundamental insights into how the brain engages in complex behavior but also offers a new avenue to measure brain functions relevant to the real world.

[0060] Also in accordance with this invention, a system is provided for analyzing a functioning of neuronal circuits of a brain of an individual, which circuits are engaged in the individual's voluntary, self-paced motor control of a button press with a finger, the system comprising: a smartphone with a touch screen; an apparatus for scanning the brain of the individual to measure the SmRP of the brain of the individual when the individual uses, particularly with the individual's thumb, the touch screen of the smartphone; and means for comparing the measured SmRP of the brain of the individual with standard measured values of SmRP of brains of other individuals when the other individuals use, particularly with the other individuals's thumbs, touch screens of smartphones.

[0061] Also in accordance with this invention, a use of a smartphone for analyzing a functioning of neuronal circuits of a brain of an individual, which circuits are engaged in the individual's voluntary, self-paced motor control of a button press with a finger, the use comprising; determining the SmRP of the brain of the individual when the individual uses, particularly with the individual's thumb, a touch screen of a smartphone; and comparing the determined SmRP of the brain of the individual with standard determined values of SmRP of brains of other individuals when the other individuals use, particularly with the other individuals's thumbs, touch screens of smartphones.

Example

[0062] A total of 45 people were recruited. The sample age ranged from 18 to 45 (median age, 23).

[0063] Smartphone Activity Recordings

[0064] The timestamps of the touchscreen interactions and the corresponding Apps in use were recorded by using a background App attached to a cloud-based data collection platform (TapCounter, QuantActions Ltd, Lausanne, Switzerland). The background App was installed for a period of 3-5 weeks prior to the laboratory-based EEG recordings. This data were downloaded from the cloud in a compressed form and further processed using MATLAB (Mathworks, Natick, USA) by using the data unpacking processes made available by QuantActions.

[0065] Categorization of Social and Non-Social Apps

[0066] The classification of social and non-social Apps was primarily based on the definitions used in a previous report. To elaborate, an app label captured on the phone was categorized as social if the main purpose was to allow users to communicate with others (friends or strangers). The app had to also contain the tools to enable these interactions such as direct messaging, public posts or voice chat, personalized profiles and being able to rate or follow the profile. The apps that did not fit the main purpose and have the tools were labeled as non-social. A slightly altered definition was applied when categorizing gaming apps—they were categorized as social only if they engaged other users during the game rather than sharing the results after playing the game solo.

[0067] Movement Sensor Recordings

[0068] During the laboratory measurements of smartphone behavior, the thumb flexions were tracked using a bend sensor (Flex Sensor, 4.5″, Spectra Symbol, Salt Lake City, USA). The sensor was attached to the thumb (dorsum) using a custom-built jacket that allowed the sensor to bend within the jacket without the sensor being pulled. The thumb was further covered with a conductive surface (aluminum foil) ensuring that all the touches were translated to touchscreen events and that the same part of the thumb was used to target the screen. The analog signals from the sensor were digitized at 1 kHz using Labview via the USB 6008 DAQ (National Instruments, Austin, USA). The same DAQ was also used to power the sensor. The thumb was all able to freely move on the touchscreen under this configuration. The movements were recorded along with synchronizing triggers emitting from the EEG recording set-up.

[0069] EEG Recordings

[0070] The EEG recordings were conducted in a faraday shielded room (Holland Shielding Systems BV, Dordrecht, The Netherlands) with optic fiber transmitted internet connectivity. The users used their own smartphone while comfortably reclined on a chair. White noise was presented through the experiment using earphones. Sixty-four channel EEG caps with equidistant electrodes were used (Easycap GmbH, Worthsee, Germany) in conjunction with ABRALYT HiCl electrode gel. Prior to recording the EEG signals, the contact impedances were reduced to less than 10 kΩ by rubbing the gel on the skin. The capsizes were matched using head circumference measurements. The EEG recordings were conducted using BrainAmp DC amplifiers (Brainproducts, Gilching, Germany). The data sample rate was set at 1 kHz and no online filters were applied during the recordings. As the EEG, the smartphone and the movement sensors operated on different clocks they were synchronized using common TTL pulse bursts generated by using an IBM T 42 motherboard running MATLAB.

[0071] To measure the EEG signals surrounding the smartphone behavior, the users were provided with a list of their own top 4 apps-top 2 social and non-social ranked based on the number of touches generated over the previous weeks. Video Apps such as youtube were eliminated prior to the ranking. The users were given 12 minutes to use each App, and the usage was separated by 2 minute short breaks where the subsequent App was launched and ready to be used. The order of the Apps was randomized. Social interactions were anticipated to be dominated by higher levels of text messaging compared to the non-social interactions. To enable comparison between the social and non-social interactions we instructed the participants to ‘not engage in extensive text messaging’—and this was further monitored by the experimenter by using the thumb flexion bend sensor measures online. However, users were permitted .sup.˜50 characters of typing in situations where they judged that typing was essential for continued engagement (for instance when writing a short reaction to a posted picture). According to debriefing interviews, users mainly browsed older posts, read and liked the posts, and responded with brief comments using emojis and a few characters.

[0072] EEG Analysis and Statistics

[0073] The temporal alignment between the smartphone touchscreen events and the bend sensor recordings was confirmed using cross-correlation analysis (Signal processing toolbox, MATLAB). In 8 of the 45 participants the alignment could not be confirmed (R2<0.8) due to recording gaps or trigger alignment failures and they were disregarded from further analysis in social vs. non-social comparisons (kinematically adjusted), and in air touches vs. real touchscreen event categories. For further analysis, a threshold of 700 acceptable trials had to be met for the establishment of the SmRP and 350 trials (per category) in the cross-category comparisons. A key focus of our analysis was to compare social vs. non-social interactions. In addition to the instructions to diminish typing interactions we also determined that in the real world the typing interactions were dominated by intervals of 200 ms (median) and towards our analysis interactions separated by less than 3.5× threshold (700 ms) gap were excluded. The population median of median separations was 2 s (for both social and non-social interactions) during the laboratory testing. After these layers of exclusions, we were left with 34 participants for the establishment of SmRPs, 20 participants for the comparison between social vs. non-social interactions (without kinematic adjustments), and 24 participants for the comparison between air touches vs. real touchscreen events.

[0074] The EEG recordings were band-pass filtered between 0.1 to 70 Hz, and independent component analysis was run for subtracting blink artifacts from the EEG signals (Icablinkmetrics implemented in MATLAB). This was followed by another band-pass filter between 0.1 to 30 Hz (and a parallel set was created with 0.1 to 3 Hz focused on the slower signals). Trials above 1 mV were eliminated. The epoch durations were −4 s to 4 s from the touchscreen event, and signals were baseline corrected between −4 to −3 s from the touchscreen event. The signals were then processed using the hierarchical linear modelling toolbox LIMO EEG using ordinary least squares regression. The SmRP was tested using one sample t-test at the population level. For comparison of social and non-social touches, the trial-to-trial motor amplitudes were used towards an ANCOVA model at the single subject level and the resultant β values for the social and no-social categories were used towards the paired t-test at the population level. For comparison of air touches vs. real touchscreen events paired t-test was used at the population level. The statistics were for corrected for multiple comparison correction (MCC) using bootstrapped signals and 2-D spatiotemporal clustering implemented in LIMO EEG (α=0.05). The statistical masks of the main and parallel streams were merged using the logical ‘or’ operator. For the simple behavioural regression analysis robust (bi-square) linear regression was used.

[0075] Subsequent Monitoring

[0076] Accordingly, it can be seen that patterns matching behavioural output with neuronal activity can be identified, along the lines above and illustrated as an example in FIG. 1. Such patterns can be stored in a database. For example, particular touches associated with particular smartphone functions can be matched to particular neuronal activities. These patterns can then be used, such as in the method of FIG. 2, to identify event-based neuronal activations based purely on the behavioural output. Accordingly, the smartphone can be used as a behavioural output recorder, such as with software (e.g. a background app) recording behavioural output of the user. Based upon such identified patterns as outlined hereabove, subsequent monitoring of users can be based solely on in loco smartphone use, recording behavioural outputs with a background app running on the smartphone. For example, a detection or recording of a smartphone touch by the background app allows a corresponding neuronal activity to be identified from the database of event-related neuronal activations. A change in identified event-related neuronal activations over a period of time may be associated with a particular function or area of the brain such that the change may be associated with a corresponding change in the function and/or area of the brain. The change may be associated with an impairment or disease and/or a treatment thereof—thereby enabling a monitoring of change in the user's brain.

[0077] The applicant hereby discloses in isolation each individual feature described herein and any combination of two or more such features, to the extent that such features or combinations are capable of being carried out based on the present specification as a whole in the light of the common general knowledge of a person skilled in the art, irrespective of whether such features or combinations of features solve any problems disclosed herein, and without limitation to the scope of the claims.

[0078] The applicant indicates that aspects of the present invention may consist of any such individual feature or combination of features. It should be understood that the embodiments described herein are merely exemplary and that various modifications may be made thereto without departing from the scope or spirit of the invention. For example, it will be appreciated that although shown here relating to a smartphone touch, in other examples other events, behaviours and associated neuronal activities are included.