Systems and methods for diagnosing a stroke condition
11699529 · 2023-07-11
Assignee
Inventors
- Nadav Eichler (Haifa, IL)
- Shmuel Raz (Kfar Vradim, IL)
- Rotem Sivan-Hoffmann (Kfar Bialik, IL)
- Alex Frid (Tirat-Karmel, IL)
- Oren Dror (Herzeliya, IL)
Cpc classification
Y02A90/10
GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
A61B5/7282
HUMAN NECESSITIES
A61B5/0077
HUMAN NECESSITIES
G16H50/20
PHYSICS
G16H50/30
PHYSICS
A61B5/4076
HUMAN NECESSITIES
G16H50/70
PHYSICS
A61B5/4803
HUMAN NECESSITIES
A61B5/743
HUMAN NECESSITIES
International classification
Abstract
A method for estimating a likelihood of a stroke condition of a subject, the method comprising: acquiring clinical measurement data pertaining to said subject, said clinical measurement data including at least one of image data, sound data, movement data, and tactile data; extracting from said clinical measurement data, potential stroke features according to at least one predetermined stroke assessment criterion; comparing said potential stroke features with classified sampled data acquired from a plurality of subjects, each positively diagnosed with at least one stroke condition, defining a positive stroke dataset; and determining, according to said comparing, a probability of a type of said stroke condition, and a probability of a corresponding stroke location of said stroke condition with respect to a brain location of said subject.
Claims
1. A method for quantitatively estimating a likelihood of a stroke condition of a subject, the method comprising: acquiring non-invasive clinical measurement data pertaining to said subject, said clinical measurement data including at least one of image data, sound data, movement data, and tactile data; constructing, via machine learning in an initial training phase, a positive stroke model from at least part of a positive stroke dataset acquired from a plurality of subjects positively diagnosed with at least one stroke condition and, in a steady-state operation phase continuously updating by training through machine learning said positive stroke model via its defining parameters through parameter estimation and optimization; extracting from said clinical measurement data, potential stroke features according to at least one predetermined stroke assessment criterion; comparing said potential stroke features with classified sampled data of said positive stroke dataset; and determining, according to said comparing and said positive stroke model, without neuroimaging of said subject, a probability of a type of said stroke condition, and a probability of a corresponding stroke location of said stroke condition with respect to a particular brain location of said subject.
2. The method according to claim 1, further comprising constructing, via machine learning in said initial training phase, a negative stroke model from at least part of a negative stroke dataset acquired from a plurality of subjects negatively diagnosed with a stroke condition, and in said steady-state operation phase continuously updating by training through machine learning said negative stroke model via its defining parameters through parameter estimation and optimization, wherein said comparing is further performed on classified sampled data of said negative stroke dataset.
3. The method according to claim 2, wherein said acquiring, said extracting, said comparing, and said determining are performed for constructing a baseline profile of said subject, wherein said baseline profile defines a time-dependent estimated neurological state of said subject.
4. The method according to claim 3, further comprising comparing between at least two said baseline profiles acquired at different times to determine changes in said clinical measurement data at said different times.
5. The method according to claim 4, further comprising generating a report from comparison between said at least two said baseline profiles.
6. The method according to claim 2, wherein said comparing involves pre-configuration to enable classification of said potential stroke features to said positive stroke dataset, and to said negative stroke dataset.
7. The method according to claim 2, wherein said comparing involves pre-training via at least one machine learning classifier (MLC) to enable classification of said potential stroke features to said positive stroke dataset, and to said negative stroke dataset.
8. The method according to claim 1, further comprising preprocessing of at least part of said clinical measurement data, prior to said extracting.
9. The method according to claim 1, wherein said extraction is of at least one of a region of interest (ROI), and a point of interest (POI) in at least one of a spatial domain, and a temporal domain.
10. The method according to claim 9, wherein said comparing further involves assessing a statistical correlation between said image data, said sound data, said movement data, and said tactile data.
11. The method according to claim 1, wherein said at least one predetermined stroke assessment criterion is selected from a list consisting of: a standardized test; a National Institutes of Health Stroke Scale (NIHSS) test; a face-arm-speech-time (FAST) test; a ABCD.sup.2 score; a CHADS.sub.2 score; a CHA.sub.2DS.sub.2VASc score; a Los Angeles Pre-hospital Stroke Screen (LAPSS) test a non-standardized test; a modified test based on a standardized test; a modified NIHSS (mNIHSS) test; a customized test based on a standardized test; and at least one characterizing mark.
12. The method according to claim 2, wherein said positive stroke dataset includes entries, each entry includes at least two fields: a stroke type and corresponding brain location.
13. The method according to claim 2, wherein said determining uses results outputted from said comparing that respectively represent quantitative measures indicating how extracted said stroke features match with corresponding said entries in said positive stroke dataset and entries in said negative stroke dataset.
14. The method according to claim 1, wherein said at least one positive stroke model is constructed for each of said potential stroke features.
15. The method according to claim 1, further comprising communicating information pertaining to said probability for said type of said stroke condition, and said probability of said corresponding stroke location to at least one device that is associated with at least one of said subject, a physician, and a medical facility.
16. The method according to claim 15, further comprising presenting least one of a region of interest (ROI), and a point of interest (POI) in extracted said clinical measurement data that corresponds with a highest estimated said likelihood of said stroke condition, according to determined said probability of said type of said stroke condition, and said probability of said corresponding stroke location.
17. A system for quantitatively estimating a likelihood of a stroke condition of a subject, the system comprising: a database, containing classified sampled datasets acquired from a plurality of subjects, each positively diagnosed with at least one stroke condition, defining a positive stroke dataset; and a processor, configured to receive non-invasive clinical measurement data pertaining to said subject, and acquired from at least one sensor that is configured to acquire at least one of image data, sound data, movement data, and tactile data pertaining to said subject, said processor configured to construct, via machine learning in an initial training phase, a positive stroke model from at least part of said positive stroke dataset and, in a steady-state operation phase, continuously update by training through machine learning said positive stroke model via its defining parameters through parameter estimation and optimization, to extract from said clinical measurement data, potential stroke features according to at least one predetermined stroke assessment criterion; to compare said potential stroke features with said classified sampled datasets; and to determine according to said positive stroke model, without neuroimaging of said subject, a probability of a type of said stroke condition, and a probability of a corresponding stroke location of said stroke condition with respect to a particular brain location of said subject.
18. The system according 17, wherein said processor is further configured to construct, via machine learning in said initial training phase, a negative stroke model from at least part of a negative stroke dataset acquired from a plurality of subjects negatively diagnosed with a stroke condition, and in said steady-state operation phase continuously updating by training through machine learning said negative stroke model via its defining parameters through parameter estimation and optimization, and to compare said potential stroke features with classified sampled data of said negative stroke dataset.
19. The system according to claim 18, wherein said processor said acquires, said extracts, said compares, and said determines is for constructing a baseline profile of said subject, wherein said baseline profile defines a time-dependent estimated neurological state of said subject.
20. The system according to claim 19, wherein said processor is further configured to compare between at least two said baseline profiles acquired at different times to determine changes in said clinical measurement data at said different times.
21. The system according to claim 20, further wherein said processor is further configured to generate a report from comparison between said at least two said baseline profiles.
22. The system according to claim 18, wherein said comparing involves pre-configuration to enable classification of said potential stroke features to said positive stroke dataset, and to said negative stroke dataset.
23. The system according to claim 18, wherein said comparing involves pre-training via at least one machine learning classifier (MLC) to enable classification of said potential stroke features to said positive stroke dataset, and to said negative stroke dataset.
24. The system according to claim 17, further wherein said processor is configured to preprocess of at least part of said clinical measurement data, prior to said extraction of said potential stroke features.
25. The system according to claim 17, wherein said extraction is of at least one of a region of interest (ROI), and a point of interest (POI) in at least one of a spatial domain, and a temporal domain.
26. The system according to claim 25, wherein said comparing further involves assessing a statistical correlation between said image data, said sound data, said movement data, and said tactile data.
27. The system according to claim 17, wherein said at least one predetermined stroke assessment criterion is selected from a list consisting of: a standardized test; a National Institutes of Health Stroke Scale (NIHSS) test; a face-arm-speech-time (FAST) test; a ABCD.sup.2 score; a CHADS.sub.2 score; a CHA.sub.2DS.sub.2VASc score; a Los Angeles Pre-hospital Stroke Screen (LAPSS) test a non-standardized test; a modified test based on a standardized test; a modified NIHSS (mNIHSS) test; a customized test based on a standardized test; and at least one characterizing mark.
28. The system according to claim 18, wherein said positive stroke dataset includes entries, each entry includes at least two fields: a stroke type and corresponding brain location.
29. The system according to claim 18, wherein said determining uses results outputted from said comparing that respectively represent quantitative measures indicating how extracted said stroke features match with corresponding said entries in said positive stroke dataset and entries in said negative stroke dataset.
30. The system according to claim 17, wherein said at least one positive stroke model is constructed for each of said potential stroke features.
31. The system according to claim 17, further comprising a communication module enabled for communication with said processor, said communication module is configured to communicate information pertaining to said probability for said type of said stroke condition, and said probability of said corresponding stroke location to at least one device that is associated with at least one of said subject, a physician, and a medical facility.
32. The system according to claim 17, further including a user interface configured to interface between said system and at least one of said subject, an operator of said system, a manager of said system, and a physician using said system.
33. The system according to claim 32, wherein said user interface is configured to provide an indication to at least one of said probability of said stroke type, and said probability of said corresponding stroke location.
34. The system according to claim 33, wherein said user interface is configured to present at least one of a region of interest (ROI), and a point of interest (POI) in extracted said clinical measurement data that corresponds with a highest estimated said likelihood of said stroke condition, according to determined said probability for said type of said stroke condition, and said probability of said corresponding stroke location.
35. A system for quantitatively estimating a likelihood of a stroke condition of a subject, the system comprising: a client device including: at least one sensor, configured to acquire at least one of image data, sound data, movement data, and tactile data, all of which constitute non-invasive clinical measurement data pertaining to said subject; a user interface, configured to provide an indication of a probability for a type of said stroke condition, and a probability of a corresponding stroke location of said stroke condition with respect to a particular brain location of said subject, without neuroimaging of said subject; and a communication module, enabled for communication with a remote computer, said communication module configured to send said clinical measurement data to said remote computer, and to receive from said remote computer said indication; wherein said indication is based on a comparison between potential stroke features extracted from said clinical measurement data according to at least one predetermined stroke assessment criterion, with classified sampled data in a database acquired from a plurality of subjects, each positively diagnosed with at least one stroke condition, defining a positive stroke dataset, and based on a positive stroke model constructed via machine learning in an initial phase from at least part of said positive stroke dataset that is in a steady-state operation phase continuously updated by training machine learning via its defining parameters through parameter estimation and optimization.
36. The system according 37, wherein said remote computer includes a processor that is configured to compare said potential stroke features with classified sampled data acquired from a plurality of subjects negatively diagnosed with a stroke condition, defining a negative stroke dataset, based on a negative stroke model constructed via machine learning in said initial training phase from at least part of said negative stroke dataset that is in said steady-state operation phase continuously updated by training through machine learning via its defining parameters through parameter estimation and optimization.
37. The system according to claim 36, wherein said remote computer is configured to construct a baseline profile of said subject, wherein said baseline profile defines a time-dependent estimated neurological state of said subject.
38. The system according to claim 37, wherein said processor is further configured to compare between at least two said baseline profiles acquired at different times to determine changes in said clinical measurement data at said different times.
39. The system according to claim 38, further wherein said processor is further configured to generate a report from comparison between said at least two said baseline profiles.
40. The system according to claim 36, wherein said comparing involves pre-configuration to enable classification of said potential stroke features to said positive stroke dataset, and to said negative stroke dataset.
41. The system according to claim 36, wherein said comparing involves pre-training via at least one machine learning classifier (MLC) to enable classification of said potential stroke features to said positive stroke dataset, and to said negative stroke dataset.
42. The system according to claim 36, said processor is configured to preprocess of at least part of said clinical measurement data, prior to said extraction of said potential stroke features.
43. The system according to claim 36, wherein said extraction is of at least one of a region of interest (ROI), and a point of interest (POI) in at least one of a spatial domain, and a temporal domain.
44. The system according to claim 43, wherein said comparing further involves assessing a statistical correlation between said image data, said sound data, said movement data, and said tactile data.
45. The system according to claim 35, wherein said at least one predetermined stroke assessment criterion is selected from a list consisting of: a standardized test; a National Institutes of Health Stroke Scale (NIHSS) test; a face-arm-speech-time (FAST) test; a ABCD.sup.2 score; a CHADS.sub.2 score; a CHA.sub.2DS.sub.2VASc score; a Los Angeles Pre-hospital Stroke Screen (LAPSS) test a non-standardized test; a modified test based on a standardized test; a modified NIHSS (mNIHSS) test; a customized test based on a standardized test; and at least one characterizing mark.
46. The system according to claim 36, wherein said positive stroke dataset includes entries, each entry includes at least two fields: a stroke type and corresponding brain location.
47. The system according to claim 36, wherein said determining uses results outputted from said comparing that respectively represent quantitative measures indicating how extracted said stroke features match with corresponding said entries in said positive stroke dataset and entries in said negative stroke dataset.
48. The system according to claim 35, wherein said at least one positive stroke model is constructed for each of said potential stroke features.
49. The system according to claim 36, wherein said communication module is configured to communicate information pertaining to said probability for said type of said stroke condition, and said probability of said corresponding stroke location to at least one device that is associated with at least one of said subject, a physician, and a medical facility.
50. The system according to claim 35, wherein said user interface is configured to interface between said system and at least one of said subject, an operator of said system, a manager of said system, and a physician using said system.
51. The system according to claim 36, wherein said user interface is configured to provide an indication to at least one of said probability of said stroke type, and said probability of said corresponding stroke location.
52. The system according to claim 51, wherein said user interface is configured to present at least one of a region of interest (ROI), and a point of interest (POI) in extracted said clinical measurement data that corresponds with a highest estimated said likelihood of said stroke condition, according to determined said probability of said type of said stroke condition, and said probability of said corresponding stroke location.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1) The disclosed technique will be understood and appreciated more fully from the following detailed description taken in conjunction with the drawings in which:
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
(13)
(14)
(15)
(16)
(17)
(18)
(19)
(20)
(21)
(22)
(23)
(24)
DETAILED DESCRIPTION OF THE EMBODIMENTS
(25) The disclosed technique overcomes the disadvantages of the prior art by providing systems and a method for electronically estimating a likelihood of a cerebral stroke condition (cerebrovascular accident (CVA), a “stroke” for short) of a subject (e.g., an individual, a patient). The disclosed technique allows for remote (as well as on-site) neurological and neurophysiological assessment of the subject (e.g., telemedicine via a physician) so as to allow shortening of “time to treatment” in case it was determined that the subject is suffering from a stroke condition with a high-probability (e.g., above a threshold value). The systems of the disclosed technique are configured and operative to provide an indication of a stroke as soon (i.e., immediate, in real-time) as it is detected (i.e., estimated at a high likelihood, i.e., over a threshold probability). According to one implementation, the system includes a patient database (“database” for brevity), and a processor. The patient database contains classified sampled datasets acquired from a plurality of subjects positively diagnosed with a stroke condition. The patient database may further contain classified sample datasets acquired from a plurality of subjects negatively diagnosed with a stroke condition (i.e., do not have a stroke condition). The processor is configured to receive clinical measurement data pertaining to the subject. The clinical measurement data is acquired from at least one sensor that is configured to sense at least one of image data, sound data, movement data, and tactile data pertaining to the subject. The processor is configured to extract from the clinical measurement data, potential stroke features according to at least one predetermined stroke assessment criterion (e.g., a test, a standard, a characterizing mark). The processor is configured to compare the potential stroke features with the classified sample data in the patient database, and to determine a probability for a type of stroke condition, and a probability of a corresponding stroke location of the stroke condition with respect to a brain location of the subject. The stroke location corresponds to the type of stroke for that stroke location. The brain location of the subject is an estimate that is fine-tuned by a brain image of the subject acquired, for example, by neuroimaging techniques. The brain location may be specified by the particular anatomical brain feature (e.g., blood vessel, area, etc.), as well as via three-dimensional coordinates of a brain volume with respect to reference point(s).
(26) According to another aspect of the disclosed technique, there is thus provided a method for estimating a likelihood of a stroke condition of a subject. The method includes acquiring clinical measurement data pertaining to the subject, extracting potential stroke features from the clinical measurement data, comparing the potential stroke features with classified sampled data in a patient database potential stroke features, and determining, according to the comparing, a probability for a type of the stroke condition, and a probability of a corresponding stroke location of the stroke condition with respect to a brain location of the subject. The clinical measurement data includes at least one of image data, sound data, movement data, and tactile data. The extraction of potential stroke features from the clinical measurement data is according to at least one predetermined stroke assessment criterion. The patient database is acquired from a plurality of subjects, each positively diagnosed with at least one stroke condition, and optionally a plurality of subjects negatively diagnosed with a stroke condition.
(27) According to a further aspect of the disclosed technique, there is thus provided a system for estimating a likelihood of a stroke condition of a subject, in which the system includes a client device enabled for communication with a remote computer. The client device includes at least one sensor, a user-interface, and a communication module. The at least one sensor is configured to acquire at least one of image data, sound data, movement data, and tactile data, all of which constitute clinical measurement data pertaining to the subject. The user-interface is configured to provide an indication of a probability for a type of the stroke condition, and a probability of a corresponding stroke location of the stroke condition with respect to a brain location of the subject. The communication module is enabled for communication with the remote computer. The communication module is configured to send the clinical measurement data to the remote computer, and to receive from the remote computer the indication. The indication is based on a comparison between potential stroke features extracted from the clinical measurement data according to at least one predetermined stroke assessment criterion, with classified sampled data in a patient database acquired from a plurality of subjects, each positively diagnosed with at least one stroke condition. The terms “stroke”, “stroke event”, and “stroke condition” are used interchangeably herein.
(28) Reference is now made to
(29) According to another implementation, system 100.sub.1 is a split (i.e., not standalone), in which typically both database 102, and processor 104, are separate and remote from acquisition unit(s) 106. In this typical implementation, the optional components of communication module 108 and user interface 110 are typically located with processor 104 and database 102. For example, database 102 and processor 104 are located in a cloud server (e.g., a data center, a server farm, etc.), and acquisition unit(s) 106 are dispersed at different and remote locations (e.g., different clinics). In this implementation, acquisition unit(s) 106 is/are enabled for communication with processor 104.
(30) An overview of the block elements of system 100.sub.1 now follows. Generally, each acquisition unit 106 includes at least one sensor (not shown in
(31) According to another implementation of the disclosed technique, there is provided a system that is configured and operative in accordance with server-client architecture. To further explicate the particulars of this implementation, reference is now made to
(32) On the client side, there are generally N clients, where each i-th client device (1≤i≤N; i∈) includes at least one acquisition unit 106C.sub.i and a communication module 108C.sub.i. Each i-th client device may further include optionally, a client processor 104C.sub.i and a user interface 110C.sub.i. Additionally, client devices 101C.sub.1, 101C.sub.2, . . . , 101C.sub.N may typically further include a memory device (not shown) for storing data acquired by acquisition unit(s).
(33) Reference is now made to
(34) Prior to the process of estimating a likelihood of a stroke condition, subject 10 (or via an intermediary thereof) is usually required to set-up a user account on server 101S via client device 101C.sub.1 that is enabled for this purpose. Typically, subject 10 (or via an intermediary thereof) may be required to input her/his identifying information into client device 101C.sub.1 that is configured and operative to run software (e.g., an application, a program that may be downloaded to the client device, be pre-installed on the client device, etc.) and enabled for communication and the exchange of data with server 101S (
(35) Following the initial set-up stage, the system and method of the disclosed technique are configured and operative to acquire and construct at least one baseline profile of subject 10. The baseline profile defines a time-dependent state of that subject's detected neurological state (i.e., a personalized profile) that includes an estimation to a likelihood of a stroke condition at a particular time. The disclosed technique employs a plurality of baseline profiles that are time-stamped, recorded and stored in database 102. The baseline profiles may be acquired and recorded on a timely basis (e.g., in a scheduled manner), on an initiation/prompt basis (e.g., patient initiated, medical professional initiated, third-party initiated (e.g., by a family member, relative, etc.), on the basis of measurements indicators triggers, a non-scheduled manner, and the like. Should the baseline profile of a particular individual be indicative of a high likelihood of a stroke condition (i.e., with respect to a particular threshold), systems 101.sub.1, and 101.sub.2 are configured and operative to alert the user, the user's relatives, and medical professionals, as will be detailed hereinbelow. Attaining a current estimation of a likelihood of a stroke condition (which can serve as a time-stamped baseline profile) is facilitated by acquiring clinical measurement data via the acquisition units. According to one implementation, the acquirement of the clinical measurement data involves prompting subject 10 to follow instructions, directions or guidance, provided by user interface 110C.sub.1 (e.g., via a program installed in client device 101C.sub.1, via a phone call, an Internet website, etc.). According to another implementation, clinical measurement data is acquired automatically, with or without user intervention. The baseline profile enables systems 100.sub.1 and 100.sub.2 to monitor, detect, and alert to changing trends in the clinical measurement data (e.g., speech irregularities get progressively worse, etc.), so as to facilitate early estimation and detection of a stroke condition before it occurs (upcoming stroke event). Furthermore, the baseline profile enables systems 100.sub.1 and 100.sub.2 to compare different baseline profiles (amongst themselves) of a particular subject acquired at different times (e.g., current baseline profile as well as past baseline profiles) and generate respective comparison reports (i.e., between at least two different baseline profiles).
(36) Prior to use, systems 101.sub.1 and 101.sub.2 are configured (e.g., via a program, software, hardware configuration, firmware configuration, algorithm, self-modifiable program, or combinations thereof) (also denoted herein as “pre-configured”) or trained (i.e., via machine learning (ML) techniques, such as machine learning classification/classifier (MLC)) (also denoted herein as “pre-trained”) so as to be enable to classify input data (e.g., distinguish, identify) among two main classes of potential stroke features stored in two different and main datasets, namely, a positive stroke dataset, and a negative stroke dataset. The positive stroke dataset includes a plurality of entries (labeled data) that are sampled from individuals positively diagnosed with at least one stroke condition. The negative stroke dataset includes a plurality of entries that are sampled from individuals negatively diagnosed for a stroke condition (i.e., are verified not to have a stroke condition). Given a tested potential stroke feature input, systems 101.sub.1 and 101.sub.2 are configured and/or trained to classify, i.e., associate the input potential stroke feature with either one of the positive stroke dataset (with a particular probability of match), the negative stroke dataset (with a particular probability of match), or (untypically) be indeterminate (i.e., neither). The configuration or training is achieved at different hierarchies (i.e., types and levels of data), from the data type to a particular attribute in the data, such as per clinical measurement type (e.g., image data, sound data), per sub-type (e.g., image feature, sound feature), and so forth according to the resolution required. Following the initial configuration or training phase, systems 101.sub.1 and 101.sub.2 are enabled for “steady-state” operation. The MLC is trained on dataset entries that may include data pertaining or based on computer tomography (CT) scans marked and evaluated by a trained physician, as well as digital reports of subjects and their respective image data, sound data, movement data, and tactile data, and optionally, blood pressure data.
(37) Image sensor 120C.sub.1 in client device 101C.sub.1 is typically part of a camera system assembly configured and operative to acquire image data 130 usually in the form of at least one image, and typically a plurality of images 130.sub.1, 130.sub.2, 130.sub.3, . . . of at least a part of subject 10 (e.g., face, torso and face, entire body, etc.). Images 130.sub.1, 130.sub.2, 130.sub.3 may be outputted as individual still images, as well as in the form of video. The camera system assembly may employ a plurality of individual camera modules each having its own image sensor, lens, and image software. The camera system may further be augmented by employing range imaging techniques (not shown) that capture depth information (i.e., distance between points in an external scene with respect to at least one reference point (e.g., the sensor's image plane)) that may be presented as a two-dimensional (2-D) range image. Such techniques include for example, time-of-flight (ToF) techniques, structured light techniques, stereophotogrammetry techniques, interferometry techniques, and the like. Images 130.sub.1, 130.sub.2, 130.sub.3, . . . are inputted into a preprocessor 132 that is configured and operative to preprocess the images by various techniques that include for example, image cropping, scaling, correction of distortions, isolation of image background from image foreground, color adjustment, exposure adjustment, sharpening, removal of noise, edge detection, etc. Image preprocessing may typically be performed but is optional.
(38) Sound sensor 122C.sub.1 (e.g., a microphone) in client device 101C.sub.1 is configured and operative to acquire sound produced by subject 10 (i.e., typically voice, speech, and the like) and to produce corresponding sound data 134 that is graphically represented in
(39) Systems 100.sub.1 and 100.sub.2 enable sensor fusion of the acquired clinical measurement data from the acquisition units (also denoted herein as “multi-modal” data defined as clinical measurement data that is acquired from different types of sources (e.g., sensors)) in the temporal domain as well as in the spatial domain so as allow for more accurate results than clinical measurement data acquirement from a single modality (i.e., one source type, e.g., image data) (e.g., by using Kalman filtering, and the like). Sensor fusion may be complete (i.e., data fused or combined from all data source types or modalities), or alternatively, may be partial (i.e., “partial sensor fusion”) where data is not fused or combined from all data source types.
(40) After acquiring the clinical measurement data from the acquisition units (i.e., the multi-modal), systems 100.sub.1 and 100.sub.2 are configured and operative to extract potential stroke features (e.g., attributes and their corresponding value) from the clinical measurement data, according to at least one predetermined stroke assessment criterion. A predetermined stroke assessment criterion is any characterizing mark, trait, standard, or rule for evaluating, assessing, deciding, or testing a likelihood to a presence of a stroke condition.
(41) Reference is now further made to
(42)
(43) The extraction of potential stroke features from different types of clinical measurement data (i.e., acquired from different sources (e.g., sensors) of data, i.e., “multi-modal data”) may time-wise correspond to each other (i.e., be synchronized in time), may overlap in time (at least partially or fully), or may be mutually exclusive in time. The example in
(44) The POIs and ROIs (in the time and spatial domains) are extracted according to least one predetermined stroke assessment criterion (typically a plurality of individual criteria) that may be: (1) a standardized test (e.g., the National Institutes of Health Stroke Scale (NIHSS), the face-arm-speech-time (FAST) test, the ABCD.sup.2 score, the CHADS.sub.2 score and its refinement the CHA.sub.2DS.sub.2VASc score (calculates stroke risk for subjects with non-rheumatic atrial fibrillation (“AF” or “A-fib”) (early stage diagnosis), Los Angeles Pre-hospital Stroke Screen (LAPSS) test, etc.); (2) a non-standardized test; (3) a modified test based on a standardized test (e.g., a modified NIHSS (mNIHSS); (4) a customized test based on a standardized test (e.g., NIHSS), where the customized version doesn't necessarily include all sub-tests of the standardized test, and may include variations of sub-tests, as well as additional sub-tests, etc.); and (5) at least one characterizing mark or trait that can serve as a direct and/or indirect possible indication in the assessment of the likelihood of a stroke condition (e.g., a determined statistical correlation between clinical measurement data and likelihood to a stroke condition). Systems 100.sub.1 and 100.sub.2 are configured and operative to run a computerized version of each selected stroke assessment test (whether standardized or non-standardized). Tables 1-12 hereinbelow show examples of predetermined stroke assessment criteria based on NIHSS, a computerized version of which according to the disclosed technique is denoted interchangeably herein as “modified NIHSS” (mNIHSS), and “adopted NIHSS”. As aforementioned, the extraction of clinical measurement data by the acquisition units may be with user intervention (e.g., prompting the subject to perform instructions, such as raising hands, speaking, etc.), be without user intervention (e.g., automatic), or be hybridized between (partial) user intervention and (partial) user non-intervention.
(45) In the alternative implementation, the acquirement of clinical measurement data is achieved without user (subject) intervention (i.e., non-interactive approach), for example automatically, by monitoring the subject's normal activities (e.g., during walking, sitting, standing, talking, during computer use, smartphone use, etc.). Systems 101.sub.1 and 101.sub.2 acquire the clinical measurement data and extract potential stroke features from the acquired clinical measurement data without prompting the user to perform tasks required for standardized tests (e.g., NIHSS) or other types of user interactive tests. This implementation may typically employ machine learning techniques for modeling the user's various routine activities via training data that is inputted into and/or acquired by systems 101.sub.1 and 101.sub.2.
(46) Following the extraction of the potential stroke features from the clinical measurement data, the extracted potential stroke features are then compared with classified sampled data in a patient database (interchangeably denoted herein as “database”) acquired from a plurality of subjects, each positively diagnosed with at least one stroke condition. To further detail this step of the disclosed technique, reference is now made to
(47) Baseline(s) dataset 186 includes at least one entry 186.sub.i that is a time-dependent baseline profile of subject 10 (where i denotes a general index of the i-th entry in baseline dataset 186 at a particular point in time). There may typically be a plurality of baseline entries for subject 10 that are time-wise ordered, as shown in
(48) Negative stroke dataset 184 includes a plurality of entries 184i where each entry 184i includes data sampled from an individual negatively diagnosed for a stroke condition (i.e., are verified not to have a stroke condition (“ground truth”)). Likewise, there may be only one entry sampled from a particular individual in a population, or a plurality of entries sampled from the same individual.
(49) Processor 104S includes a main comparator block 190, which in turn may include a plurality of individual comparators 190.sub.1, 190.sub.2, 190.sub.3, 190.sub.4, 190.sub.5, 190.sub.6, 190.sub.7, 190.sub.8, 190.sub.9 (collectively denoted herein as “comparators 190.sub.1-190.sub.9”). Main comparator block 190 may be implemented in at least one of hardware, software, firmware, and a combination thereof. Main comparator block 190 is configured and operative to compare subject-specific extracted potential stroke features 160.sub.1-7, 162.sub.1, 164.sub.1, and, 166.sub.1 with classified sampled data in positive stroke dataset 182. Specifically, comparator 190.sub.1 compares extracted potential stroke feature 160.sub.2 with positive stroke dataset 182 so as to produce a result that represents a quantitative measure that indicates how extracted potential stroke feature 160.sub.2 matches with corresponding entries 182i of the same type (i.e., image data). Similarly, comparators 190.sub.2-190.sub.9 respectively compare extracted potential stroke features 160.sub.3-7, 162.sub.1, 164.sub.1, and 166.sub.1 with positive stroke dataset 182, so as to produce respective outputs that represent quantitative measures that indicate how these extracted potential stroke features match with corresponding entries 182i of their same type. An output of the comparison is a quantitative measure to how a particular extracted potential stroke feature matches either one of positive stroke dataset 182, negative stroke dataset 184, or both (i.e., an indeterminate result, e.g., in case there's a 50% match to positive stroke dataset 182 and 50% match to negative stroke dataset 184). In addition (and optionally), comparators 190.sub.1-190.sub.9 are configured and operative to compare extracted potential stroke features 160.sub.1-160.sub.7, 162.sub.1, 164.sub.1, and 166.sub.1 with negative stroke dataset 184, so as to produce respective outputs that represent quantitative measures indicating how these extracted potential stroke features match with corresponding entries 184.sub.i of their same type. Generally, the use of both positive stroke dataset 182 and negative stroke dataset 184 in the comparison enhances the estimation of the likelihood in determining the presence of a stroke condition of the subject.
(50) Alternatively, there is one comparator associated for each modality type (e.g., image data, sound data, etc.) (not shown). According to this alternative configuration, one comparator is used to compare extracted potential stroke features 160.sub.1, 160.sub.2, 160.sub.3, 160.sub.4, 160.sub.5, 160.sub.6, and 160.sub.7 (image data) with classified data in positive stroke dataset 182, and optionally with negative stroke dataset 184. Similarly, there are separate and distinct comparators, respectively employed to compare extracted potential stroke feature 162.sub.1 (sound data), extracted potential stroke feature 164.sub.1 (movement data), as well as extracted potential stroke feature 166.sub.1 (tactile data) with classified data in positive stroke dataset 182, and optionally with negative stroke dataset 184. Further alternatively, there is one comparator that is configured and operative to perform all the required comparisons.
(51) According to a particular configuration, main comparator block 190 is implemented as a machine learning classifier (denoted herein “MLC”) that is configured and operative to employ both positive stroke dataset 182 as well as negative stroke dataset 184, both of which constitute as training data in which the MLC bases and produces an output that corresponds to an input of an extracted potential stroke feature. Generally, the input to the MLC is an extracted (and preprocessed) potential stroke feature, and the corresponding output of the MLC is a quantitative measure to how the inputted extracted potential stroke feature fits to the trained data, the latter of which can be represented by a mathematical model, as will be further detailed hereinbelow. In one implementation, there is a plurality of different MLCs (i.e., equal to the number of comparators 190.sub.1-190.sub.9) for each subject-specific extracted potential stroke feature. According to another implementation, there is one MLC for each modality type (e.g., image data, sound data, etc.) (not shown). According to a further implementation, there is one MLC (e.g., main comparator 190 is implemented by one MLC). Typical examples of MLCs include artificial neural networks (ANNs), decision trees, support vector machines (SVMs), Bayesian networks, k-nearest neighbor (KNN) classifiers, regression analysis (e.g., linear, logistic), etc.
(52) To further explicate the particulars of the disclosed technique, reference is now further made to
(53) Each comparator (also herein MLC) 190.sub.1-190.sub.9 is configured and operative to receive as input the extracted and preprocessed potential stroke features (as detailed in conjunction with
(54) In determining a probability for a type of a stroke condition, and a probability of a corresponding stroke location, processor 104S is configured and operative to use the results of the comparisons between the potential stroke features and the classified sampled data in the positive stroke dataset (as well as optionally with the negative stroke dataset). To further detail the particulars of this aspect of the disclosed technique, reference is further made to
(55) In addition, communication module 108S is configured and operative to communicate outputs 202.sub.1 and 202.sub.2 through signals encoding data pertaining to P.sub.T and P.sub.L to external communication devices 220 (also denoted herein interchangeably as mobile or immobile “patient management console units”, “management console units”, and “management console”) of various entities such as: (1) a medical emergency response service (e.g., operating an ambulance service); (2) medical professional(s) (e.g., a doctor specialized in treating strokes, a personal doctor of subject 10, paramedics, etc.); (3) a hospital emergency room (ER) including a neuroimaging department (e.g., employing computerized tomography (CT), magnetic resonance imaging (MRI) in general and functional-MRI (fMRI) in particular, positron-emission tomography (PET), and the like); (4) subject's 10 relatives (e.g., family member(s)); (5) an operator of systems 101.sub.1 and 101.sub.2 of the disclosed technique; and the like. Probabilities P.sub.T and P.sub.L transmitted to external communication devices 220 also include information about subject 10 that can include name, identification number, age, current location, etc. The system and method of the disclosed technique are configured and operative to present (e.g., provide, display) at least one ROI, and POI in the extracted clinical measurement data that corresponds with a highest estimated likelihood of the stroke condition, according to the determined probabilities P.sub.T and P.sub.L so as to reduce time for treatment by medical staff, physician, etc.
(56) Reference is now made to
(57) In procedure 254, from the clinical measurement data, potential stroke features are extracted according to at least one predetermined stroke assessment criterion. With reference to
(58) In procedure 256, the potential stroke features are compared with classified sampled data acquired from a plurality of subjects, each positively diagnosed with at least one stroke condition, defining a positive stroke dataset. With reference to
(59) In procedure 258, a probability of a type of stroke condition, and a probability of a corresponding stroke location of the stroke condition with respect to a brain location of the subject are determined according to the comparing procedure. With reference with
(60) A real-world example implementation of the disclosed technique now follows. Reference is now made to
(61) Reference is now further made to
(62) Reference is now further made to
(63) Reference is now further made to
(64) Reference is now further made to
(65) Reference is now further made to
(66) Reference is now further made to
(67) Reference is now further made to
(68) Reference is now further made to
(69) Reference is now made to
(70) Reference is now made to
(71) Another aspect of the disclosed technique involves using the infrastructure of systems 100.sub.1 and 100.sub.2 to estimate diseases, conditions, and neurological disorders other than stroke, such as Parkinson's disease, dementia, psychiatric and mental diseases, facial visual disorders, etc. Estimation to a likelihood of a variety of medical conditions can be covered by a modified version of a stroke scale described herein and/or can be covered by other medical scales, e.g., Unified Parkinson's Disease Rating Scale (UPDRS) for Parkinson's disease). For example, some symptoms of Parkinson's disease can be detected during diagnostic tests for stroke (such as the NIHSS test).
(72) TABLE-US-00001 TABLE 1 Adopted/ Modified NIHSS Computer- NIHSS NIHSS ized Extraction of potential Category Task Test stroke features Level of The examiner Acquire and POI and ROI extraction: Conscious- first assesses record image and Processors 104 and ness if the subject is sound data from 104S are configured and (LOC) fully alert to subject who is operative to extract responsive- his/her instructed to potential stroke features ness surroundings. provide verbal in the image data and in If the subject is feedback when the sound data by not completely touched on both detecting and extracting alert, examiner sides of body and POI and ROI (e.g., attempts a then asked image and voice verbal stimulus several basic segments of the subject to arouse the questions (e.g., voice segments of a subject. Failure age, the current narrator, while the of verbal month). subject is being touched stimuli leads to during questioning). an attempt to In case there's no arouse the feedback from the subjec via subject, the algorithm repeated outputs the maximum physical score for this category. stimuli. If none Measurements/Features of these stimuli extraction: are successful Processors 104 and in eliciting a 104S (e.g., employing an response, the algorithm) are configured subject can be and operative measures considered the response time totally intervals between the unresponsive. narrator's commands LOC Subject is (human or synthetic) and questions verbally asked the patient's feedback. his/her age Medical report outputs: and for the 1) The test name of the recordings. current month. 2) TRUE/FASE indication for responsiveness in each test. 3) Response time interval values between the narrator's instructions and the patient's responses for each test. 4) NIHSS score—guided by the MLC. Note: MLC outputs a score according to the current NIHSS category (the classifier is pre- trained with previous analyzed subjects and their NIHSS scores as ground truth).
(73) TABLE-US-00002 TABLE 2 Adopted/ Modified NIHSS Computer- NIHSS NIHSS ized Extraction of potential Category Task Test stroke features LOC The Acquire and POI and ROI extraction: commands subject is record Processors 104 and 104S are instructed images of the configured and operative to to first subject's face extract potential stroke features open and with sound in the image data and in the close his/ while the sound data by detecting and her eyes subject is extracting POI and ROI (e.g., and then instructed to image features, and voice grip and close and segments of the subject patient release open eyes, and a narrator, while the his/her then record subject is visually responding hand subject's to the instructions: open-close body while he eyes, grip-release hand. is instructed If patient feedback does not to grip and exist, the algorithm outputs the release his maximum score for this hand. category. Measurements/Features extraction: Processors 104 and 104S (e.g., via an algorithm) detect and tracks the eyes in the video and also detects blinking or closing of the eyes. The algorithm measures the response time intervals between the narrator's commands (human or synthetic) and the patient's visual feedback. Another algorithm detects and tracks the hands of the patient in the video, and also detects the grip and release gestures in the video. Medical report outputs: 1) The test recordings. 2) TRUE/FALSE value for each test regarding the responsiveness. 3) Response time intervals between the narrator's instructions and the patient's responses for each test. 4) NIHSS score—guided by the MLC. 5) According to the detected region of interest: a. Cropped videos for each test. b. Cropped videos for each test with eye tracker algorithm animation/hand tracking animation. c. Snapshots from the video, for example, eyes close, eyes open, hand grip and release. Note: The MLC outputs a score according to the current NIHSS category (the classifier is pre- trained with previous analyzed subjects and their NIHSS scores as ground truth).
(74) TABLE-US-00003 TABLE 3 Adopted/ Modified NIHSS Computer- NIHSS NIHSS ized Extraction of potential Category Task Test stroke features Horizontal Assesses Record the POI and ROI extraction: eye ability of patient's face Processors 104 and 104S are movement the patient with camera configured and operative (e.g., to track a and via an algorithm) to analyze pen or microphone the video and voice signals finger from while he is and detect moments when the side to side instructed to narrator gives the current only using look straight instructions, and the moments his or her into the when the patient responds to eyes. This camera. the instructions, looking is designed straight at the camera. to assess If patient feedback does not the motor exist, the algorithm outputs the ability to maximum score for this gaze category. towards Measurements/Features the hemi- extraction: sphere Processors 104 and 104S are opposite configured and operative (e.g., to injury. an algorithm) to detect and track the facial landmarks of the patient, specifically the eyes and the symmetry axis of the face. The algorithm analysis of the patient's gaze is quantified by calculating the head pose of the patient relative to the camera during this test. The algorithm measures every frame of the video, if one side of the patient's face is more gaze- deviated (relative to the camera plane) compared to the other side of his face. Medical report outputs: 1) The test recordings. 2) TRUE/FALSE value for each test regarding the responsiveness. 3) Response time intervals between the narrator's instructions and the patient's responses for each test. 4) NIHSS score—guided by the MLC. 5) According to the detected region of interest: a. Cropped videos for each test. b. Cropped videos for each test with eye tracker/facial symmetry axis algorithm animation. c. Snapshots from the video, for example, maximum gaze asymmetry frame, minimum gaze asymmetry frame. d. Eye coordination and the facial symmetry axis position for all video frames (including calculation of more measurements from these data such as variance, average speed, distance, etc.). Note: The MLC outputs a score according to the current NIHSS category (the classifier is pre- trained with previous analyzed subjects and their NIHSS scores as ground truth).
(75) TABLE-US-00004 TABLE 4 Adopted/ Modified NIHSS Computer- NIHSS NIHSS ized Extraction of potential Category Task Test stroke features Visual Assess the Record the POI and ROI extraction: field test patient's patient's face Processors 104 and 104S are vision in with camera configured and operative (e.g., each visual and via an algorithm) to analyze field. Each microphone the video and voice signals eye is while he and detects the moments tested instructed to when the narrator gives the individually, cover one of current instructions, and the by his eyes and moments when the patient covering then say the responds to the instructions, one eye number that covers the eye, says the and then he sees, from presented number, for both the other. a screen or sides separately. Each upper by the If patient feedback does not and lower fingers exist, the algorithm outputs the quadrant is of the maximum score for this tested by instructor, category. asking the This test is Measurements/Features patient to conducted extraction: indicate for Processors 104 and 104S are how many both sides configured and operative (e.g., fingers the separately. via an algorithm) to detect and investigator track the eyes of the patient. is The analysis of the patient's presenting visual field test is quantified by in each detecting the moments that the quadrant. patient covers one of his eyes until he recognizes and says the presented number and the voice of the spoken number is analyzed. The algorithm measures the response time intervals between the narrator's commands (human or synthetic) and the patient's verbal feedback. The algorithm also analyzes the speech of the patient by trying to recognize a valid number within the voice recording. Medical report outputs: 1) The test recordings. 2) TRUE/FALSE value for each test regarding the responsiveness. 3) Response time intervals between the narrator's instructions and the patient's responses for each test. 4) NIHSS score—guided by the MLC. 5) According to the detected region of interest: a. Cropped videos for each test. b. Cropped videos for each test with eye tracker algorithm animation. c. Snapshots from the video, for example, right eye covered, left eye covered, neutral face and face while speaking. d. Eye coordination for all video frames (including calculation of more measurements from these data such as variance, average speed, distance, etc.). Note: The MLC outputs a score according to the current NIHSS category (the classifier is pre- trained with previous analyzed subjects and their NIHSS scores as ground truth).
(76) TABLE-US-00005 TABLE 5 Adopted/ Modified NIHSS NIHSS Computer- Cat- NIHSS ized Extraction of potential stroke egory Task Test features Facial Facial Record the POI and ROI extraction: palsy palsy is patient's Processors 104 and 104S are partial or face with configured and operative (e.g., complete camera and via an algorithm) to analyze paralysis microphone the video and voice signals of while he and to detect the moments portions instructed when the narrator gives the of to smile current instructions, and the the face. and show moments when the patient Typically, his teeth responds to the instructions, this smiles and shows the teeth. paralysis If patient feedback does not is most exist, the algorithm outputs the pro- maximum score for this nounced category. in the Measurements/Features lower half extraction: of one Processors 104 and 104S are facial side. configured and operative (e.g., via an algorithm) to detect and track the facial landmarks of the patient, including the patient's eyes, lips and facial symmetry axis. The analysis of the patient's facial palsy is quantified by measuring the asymmetry between correlated face part coordinates and their relative distance from the facial symmetry axis during the entirev ideo. The algorithm also measures the response time intervals between the narrator's commands (human or synthetic) and the patient's visual feedback. Medical report outputs: 1) The test recordings. 2) TRUE/FALSE value for each test regarding the responsiveness. 3) Response time intervals between narrator's instructions and patient's responses for each test. 4) NIHSS score—guided by the MLC. 5) According to the detected region of interest: a. Cropped videos for each test. b. Cropped videos for each test with face tracker algorithm animation. c. Snapshots from the video, for example, neutral face, most asymmetrical face, smile climax. d. Face part coordinates and symmetry axis position for all video frames (including calculation of more measurements from these data such as variance, average speed, distance, etc.). Note: The MLC outputs a score according to the current NIHSS category (the classifier is pre- trained with previous analyzed subjects and their NIHSS scores as ground truth).
(77) TABLE-US-00006 TABLE 6 Adopted/ Modified NIHSS Extraction of NIHSS NIHSS Computerized potential stroke Category Task Test features Motor With Record the POI and ROI extraction: arm palm patient's Processors 104 and 104S are facing upper body configured and operative (e.g., down- with camera via an algorithm) to analyze wards, and the video and voice signals have the microphone and to detect the moments patient while he is when the narrator gives the extend instructed to current instructions, and the one arm lift his arms moments when the patient 90 simultane- responds to the instructions, degrees ously lifting his hands. out in to 90 If patient feedback does not front degrees. exist, the algorithm outputs the if the maximum score for this patient is category. sitting, Measurements/Features and 45 extraction: degrees An algorithm detects and out in tracks the hands of the patient; front the analysis of the patient's if the motor arm is quantified by patient is measuring the distance, height, lying and angle of each hand down. separately from the body. It calculated from the start of the motion to the end of the motion, and then asymmetry measurements between the hands are calculated relating to the whole video. The algorithm also measures the response time intervals between the narrator's commands (human or synthetic) and the patient's visual feedback. Medical report outputs: 1) The test recordings. 2) TRUE/FALSE value for each test regarding the responsiveness. 3) Response time intervals between the narrator's instructions and the patient's responses for each test. 4) NIHSS score-guided by the MLC. 5) According to the detected region of interest: a. Cropped videos for each test. b. Cropped videos for each test with hand tracker algorithm animation. c. Snapshots from the video, for example, neutral hands, max hand lift, most asymmetric frame between hand height. d. Hand and body distances, heights, and angles for all video frames. e. Summarizing asymmetry measurements between hands during the whole video. Note: The MLC outputs a score according to the current NIHSS category (the classifier is pre-trained with previous analyzed subjects and their NIHSS scores as ground truth).
(78) TABLE-US-00007 TABLE 7 Adopted/ Modified NIHSS Computer- Extraction of NIHSS NIHSS ized potential stroke Category Task Test features Motor With the Record the POI and ROI extraction: leg patient in patient's Processors 104 and 104S are the supine lower body configured and operative (e.g., position, with camera via an algorithm) to analyze one leg is and the video and voice signals placed 30 microphone and to detect the moments degrees while he is when the narrator gives the above instructed to current instructions and the horizontal, lift each one moments that the patient As soon of his legs responds to theinstructions as the separately to to lift his legs. If patient patient's 30 degrees. feedback does not exist, the leg is in algorithm outputs the position, maximum score for this the category. investigator Measurements/Features should extraction: begin An algorithm detects and verbally tracks the legs of the patient. counting The analysis of the patient's down from motor leg is quantified by 5 while measuring the distance, height, simultane- and angle of each leg ously separately from the body. It counting calculates from the start of the down on motion to the end of the his or her motion, and then asymmetry fingers in measurements between the full view of legs are calculated relating to the patient. the whole video. Observe The algorithm also measures any the response time intervals downward between the narrator's leg drift commands (human or prior to the synthetic) and the patient's end of the visual feedback. 5 seconds. Medical report outputs: 1) The test recordings. 2) TRUE/FALSE value for each test regarding the responsiveness. 3) Response time intervals between the narrator's instructions and the patient's responses for each test. 4) NIHSS score—guided by the MLC. 5) According to the detected region of interest: a. Cropped videos for each test. b. Cropped videos for each test with hand tracker algorithm animation. c. Snapshots from the video, for example, neutral legs, max leg lift for each leg, most asymmetric frames between legs. d. Leg and body distances, heights, and angles for all video frames. e. Summarizing asymmetry measurements between legs during the whole video. Note: The MLC outputs a score according to the current NIHSS category (the classifier is pre- trained with previous analyzed subjects and their NIHSS scores as ground truth).
(79) TABLE-US-00008 TABLE 8 Adopted/ Modified NIHSS Computer- Extraction of NIHSS NIHSS ized potential stroke Category Task Test features Limb This tests Record the POI and ROI extraction: ataxia for the patient's Processors 104 and 104S are presence face and configured and operative (e.g., of upper via an algorithm) to analyze a unilateral body with the video and voice signals cerebellar camera and and to detect the moments lesion, and micro- when the narrator gives the distinguishes phone current instructions, and the between while he is moments that the patient general instructed visually responds to the weakness to touch instructions, the first touch and inco- the screen of his finger to instructor's ordination. or the finger or screen, and the The instructor's second touch of the same patient finger, and finger with his nose. If should be then touch patient feedback does not instructed his nose exist, the algorithm outputs to first with the the maximum score for touch his same this category. or her finger. Measurements/Features finger to extraction: the An algorithm detects and examiner's tracks the hands and the finger finger, then used by the patient, and also move that detects and tracks the patient's finger back facial landmarks, including his to his or nose. The analysis of the her nose patient's limb ataxia is quantified by measuring the distance and speed of motion between the finger-to-finger touching and between finger- to-nose touching during this test. The video is analyzed from the start of the motion to the end of the motion, then a total score is calculated for the patient's motion performance (success/failure/partial success) relating the whole video. The algorithm also measures the response time intervals between the narrator's commands (human or synthetic) and the patient's visual feedback. Medical report outputs: 1) The test recordings. 2) TRUE/FALSE value for each test regarding the responsiveness. 3) Response time intervals between the narrator's instructions and the patient's responses for each test. 4) NIHSS score—guided by the MLC. 5) According to the detected region of interest: a. Cropped videos for each test. b. Cropped videos for each test with hand tracker algorithm animation and face tracker algorithm animation. c. Snapshots from the video, for example, touch between fingers, touch between finger to nose, closest point between nose and finger. d. Finger-to-nose and finger-to- finger distances and speeds for all video frames. e. Summarizing score measure for success failure/partial success of the touching during the whole video. Note: The MLC outputs a score according to the current NIHSS category (the classifier is pre- trained with previous analyzed subjects and their NIHSS scores as ground truth).
(80) TABLE-US-00009 TABLE 9 Adopted/ Modified NIHSS Computer- Extraction of NIHSS NIHSS ized potential stroke Category Task Test features Language This item Record the POI and ROI extraction: measures patient's Processors 104 and 104S are the responses configured and operative (e.g., patient's with camera via an algorithm) to analyze language and the video and voice signals skills. After microphone and to detect the moments completing while the when the narrator gives the items instructor is current instructions, and the (Tables) 1- guiding the moments that the patient 8, it is patient to visually and verbally responds likely the read to the instructions, reading a investigator sentences sentence or naming an object has gained and describe from a picture. an approx- a picture of If patient feedback does not imation several exist, the algorithm outputs the of the objects, maximum score for this patient's which is category. language presented to Measurements/Features skills; the patient extraction: however, it on the An algorithm detects and is important mobile tracks the patient's facial to confirm device landmarks, including his this screen. mouth. The algorithm also measure- detects the voice segments ment that the narrator or the patient at this speaks based on the mouth time The movement and audio signals. stroke The analysis of the patient's scale language and speech is includes a quantified by measuring the picture of a similarity between the recorded picture of a voice segments of the patient scenario, a and the words and objects that list of are presented to him during the simple test. sentences, The video is analyzed from the a figure of start of the test to the end of assorted the test, then a total score is random calculated for the patient's objects, verbal feedback and a list of (success/failure/partial words. The success) relating to the whole patient video. The algorithm also should be measures the motion of the asked to patient's mouth. explain the The algorithm also measures scenario the response time intervals depicted in between the narrator's the first commands (human or figure. synthetic) and the patient's Next, he or visual feedback. she should Medical report outputs: read the 1) The test recordings. list of 2) TRUE/FALSE value for sentences each test regarding the and name responsiveness. each of the objects depicted in the next figure.
(81) TABLE-US-00010 TABLE 10 Adopted/ Modified NIHSS Computer- Extraction of NIHSS NIHSS ized potential stroke Category Task Test features Speech Dysarthria is Record the POI and ROI extraction: the lack of patient's Processors 104 and 104S are motor skills responses configured and operative (e.g., required to with via an algorithm) to analyze produce camera and the video and voice signals understand- microphone and to detect the moments able speech. while the when the narrator gives the Dysarthria is instructor is current instructions, and the strictly a guiding the moments that the patient motor patient to visually and verbally responds problem and read to the instructions, reading a is not sentences sentence or naming an object related to and from a picture. the patient's describe a If patient feedback does not ability to picture of exist, the algorithm outputs the comprehend several maximum score for this speech. objects, category. Strokes that which is Measurements/Features cause presented to extraction: dysarthria the patient An algorithm detects and typically on the tracks the patient's facial affect areas mobile landmarks, including his such as the device mouth. The algorithm also anterior screen. detects the voice segments opercular, that the narrator or the patient medial speaks based on the mouth prefrontal movement and audio signals. and The analysis of the patient's premotor, language and speech is and anterior quantified by measuring the cingulate similarity between the recorded regions. voice segments of the patient These brain and the words and objects that regions are are presented to him during the vital in test. coordinating The video is analyzed from the motor start of the test to the end of control of the test, then a total score is the tongue, calculated for the patient's throat, lips, verbal feedback and (success/failure/partial lungs. To success) relating to the whole perform this video. The algorithm also test, the measures the motion of the patient is patient's mouth. asked to The algorithm also measures read from the response time intervals the list of between the narrator's words commands (human or provided synthetic) and the patient's with the visual feedback. stroke scale Medical report outputs: while the 1) The test recordings. examiner a. TRUE/FALSE observes value for each test the patient's regarding the articulation responsiveness. and clarity of speech.
(82) TABLE-US-00011 TABLE 11 Adopted/ Modified NIHSS Computer- Extraction of NIHSS NIHSS ized potential stroke Category Task Test features Sensory Sensory Record the POI and ROI extraction: testing is patient's Processors 104 and 104S are performed responses configured and operative (e.g., via pinpricks with via an algorithm) to analyze in the camera and the video and voice signals proximal micro- and to detect the moments portion of phone when the narrator speaks the all four while the current instructions, and limbs. While instructor while he touches the patient. applying is applying The algorithm also analyzes pinpricks, pinpricks the moments that the patient the on the visually and verbally responds investigator patient's to the instructions and should ask body on touching. whether or both sides If patient feedback does not not the separately. exist, the algorithm outputs the patient feels maximum score for this the pricks, category. and if he or Measurements/Features she feels the extraction: pricks An algorithm detects the voice differently segments when the narrator on one side asks the patient if he feels his when touching. The algorithm also compared to detects the voice feedback of the other the patient to the touching. The side. analysis of the patient's responses is quantified by analyzing the voice feedback to the touching, specifically if the feedback is positive or negative. The video is analyzed from the start of the test to the end of the test, then a total score is calculated to summarize the verbal feedback of the touching (negative/positive/partial) relating to the whole video. The algorithm also measures the response time intervals between the narrator's commands (human or synthetic) and the patient's visual feedback. Medical report outputs: 1) The test recordings. 2) TRUE/FALSE value for each test regarding the responsiveness. 3) Response time intervals between the narrator's instructions and the patient's responses for each test. 4) NIHSS score—guided by the MLC. 5) According to the detected region of interest: a. Cropped videos for each test. b. Cropped videos for each test with mouth tracker algorithm animation and the moments of answering of the patient. c. Snapshots from the video, for example, while patient is being touched. d. Summarizing score measure for negative/positive response of patient touching feedback during the whole video. Note: The MLC outputs a score according to the current NIHSS category (the classifier is pre- trained with previous analyzed subjects and their NIHSS scores as ground truth).
(83) TABLE-US-00012 TABLE 12 Adopted/ Ex- Modified trac- NIHSS tion of Computer- potential NIHSS NIHSS ized stroke Category Task Test features Extinction Sufficient information regarding Note: The extinction and this item may have been obtained and inattention inattention by the examiner to properly score category is covered the patient in items 1-10. However, by the other NIHSS if any ambiguity exists, the categories described examiner should test this item via in this table (For a technique referred to as “double example, see “LOC simultaneous stimulation”. This is Commands” performed by having the patient category, close his or her eyes and asking specifically the eyes him or her to identify the side on closing test). which they are being touched by the examiner. During this time, the examiner alternates between touching the patient on the right and left a sides. Next, the examiner touches the patient on both sides at the same time. This should be repeated on the patient's face, arms, and legs. To test extinction in vision, the examiner should hold up one finger in front of each of the patient's eyes and ask the patient to determine which finger is wiggling or if both are wiggling. The examiner should then alternate between wiggling each finger and wiggling both fingers at the same time.
(84) After quantifying each category of the NIHSS, the total score can define the stroke severity according as follows: a score of 0 indicates no stroke symptoms; a score between 1 and 4 indicates a minor stroke; a score between 5 and 15 indicates a moderate stroke; a score between 16 and 20 indicates a moderate to severe stroke; and a score of 21-42 indicates a severe stroke. The disclosed technique is configured and operative to calculate the total severity score in a “decision-making” mode. The quantified scores can also be treated as recommendations for the physician, when the system configured to “decision support mode”.
(85) It will be appreciated by persons skilled in the art that the disclosed technique is not limited to what has been particularly shown and described hereinabove. Rather the scope of the disclosed technique is defined only by the claims, which follow.