COMPUTER-BASED SYSTEMS AND DEVICES CONFIGURED FOR DEEP LEARNING FROM SENSOR DATA NON-INVASIVE SEIZURE FORECASTING AND METHODS THEREOF
20230141496 · 2023-05-11
Assignee
Inventors
Cpc classification
G16H20/30
PHYSICS
G16H50/70
PHYSICS
G16H50/20
PHYSICS
A61B5/7275
HUMAN NECESSITIES
G16H20/10
PHYSICS
A61B5/4094
HUMAN NECESSITIES
A61B5/746
HUMAN NECESSITIES
International classification
Abstract
To enable real-time seizure warnings, systems and methods of the present disclosure include a wearable sensor in communication with processors that are configured to receive from the wearable sensor data streams associated with a user that include biomarker data parameters. The processors utilize a seizure forecasting machine learning model to predict a pre-ictal period probability associated with a forecasted time segment based on values of the data streams. The processors determine a segment value for an integration window of a history pre-ictal period probabilities for the forecasted time segment and previously forecasted time segments and determine a pre-ictal period based on the segment value exceeding a pre-ictal probability threshold. The processors determine a pre-ictal risk indication include a seizure treatment administration and cause a computing device to produce the pre-ictal risk indication to indicate a predicted risk of a seizure.
Claims
1. A method comprising: receiving, by at least one processor, at least one data stream comprising wearable sensor data associated with a user; wherein the at least one data stream comprises biomarker data parameters; utilizing, by the at least one processor, seizure forecasting machine learning model to predict a pre-ictal period probability associated with a forecasted time segment based at least in part on values of the at least one data stream; determining, by the at least one processor, a segment for an integration window of a history pre-ictal period probabilities for the forecasted time segment and at least one previously forecasted time segment; determining, by the at least one processor, a pre-ictal period based at least in part on the segment exceeding a pre-ictal probability threshold; determining, by the at least one processor, a pre-ictal risk indication including a seizure treatment administration responsive to the pre-ictal risk indication; and causing to produce, by the at least one processor, the pre-ictal risk indication at a computing device associated with the user to alert the user of a predicted risk of a seizure.
2. The method as recited in claim 1, further comprising communicating, by the at least one processor, with a wearable device to receive the at least one data stream in real-time.
3. The method as recited in claim 2, wherein the wearable device includes a biomarker sensor worn by the user.
4. The method as recited in claim 1, wherein the at least one data stream comprises: i) electrodermal activity, ii) heart rate, iii) blood volume pulse, iv) temperature, v) accelerometer-based movement data vi) electroencephalogram measurements, vii) time, viii) date, ix) global positioning system data, x) medication, xi) self-reported seizures, xii) clinical patient data, or xiii) combinations thereof.
5. The method as recited in claim 1, wherein the time segment used to calculate forecasts comprises thirty seconds.
6. The method as recited in claim 1, wherein the integration window comprises a rolling three hundred second period of the history of pre-ictal period probabilities.
7. The method as recited in claim 1, further comprising determining, by the at least one processor, an inter-ictal period upon the pre-ictal period probability falling below the pre-ictal probability threshold.
8-9. (canceled)
10. The method as recited in claim 1, further comprising modifying, by the at least one processor, a time-span of the integration window, a time span of the forecasted time segment, the pre-ictal probability threshold, seizure occurrence period, or combinations thereof, based on an accuracy of the pre-ictal risk alert for the user.
11. A system comprising: at least one sensor; and at least one processor in communication with the at least one sensor and configured to perform steps of instructions stored in a non-transitory memory, the steps comprising: receive from the at least one sensor at least one data stream associated with a user; wherein the at least one data stream comprises biomarker data parameters; utilize seizure forecasting machine learning model to predict a pre-ictal period probability associated with a forecasted time segment based at least in part on values of the at least one data stream; determine a segment value for an integration window of a history pre-ictal period probabilities for the forecasted time segment and at least one previously forecasted time segment; determine a pre-ictal period based at least in part on the segment value exceeding a pre-ictal probability threshold; determine a pre-ictal risk indication including a seizure treatment administration responsive to the pre-ictal risk indication; and cause to produce a pre-ictal risk indication at a computing device associated with the user to indicate a predicted risk of a seizure.
12. The system as recited in claim 11, wherein the at least one processor is further configured to generate a pre-ictal risk alert to alert the user of the predicted seizure.
13. The system as recited in claim 11, wherein the at least one processor is further configured to generate a risk profile based on a history of pre-ictal risk indicators associated with the user.
14. The system as recited in claim 13, wherein the at least one processor is further configured to generate treatment plan optimizations for mitigating seizures.
15. The system as recited in claim 11, wherein the at least one processor is further configured to generate a seizure mitigation suggestion based on the pre-ictal risk indicator and the at least one data stream.
16. The system as recited in claim 15, wherein the seizure mitigation suggest comprises one or more of: i) a medication administration, ii) a release of stimulation, or iii) a combination thereof.
17. The system as recited in claim 11, wherein the at least one processor is further configured to communicate with a wearable device to receive the at least one data stream in real-time.
18. The system as recited in claim 17, wherein the wearable device includes a wrist worn sensor.
19. The system as recited in claim 11, wherein the at least one data stream comprises: i) electrodermal activity, ii) heart rate, iii) blood volume pulse, iv) temperature, v) accelerometer-based movement data, or vi) electroencephalogram measurements, vii) time, viii) date, ix) global positioning system data, x) medication, xi) self-reported seizures, xii) clinical patient data, or xii) combinations thereof.
20-21. (canceled)
22. The system as recited in claim 11, wherein the at least one processor is further configured to determine an inter-ictal period upon the pre-ictal period probability falling below the pre-ictal probability threshold.
23-24. (canceled)
25. A method comprising: receiving, by at least one processor, a training dataset from a plurality of ground-truth time-series electrophysiological datasets; wherein each ground-truth time-series electrophysiological data of the plurality of ground-truth time-series electrophysiological datasets comprises a series of labelled epochs; determining, by the at least one processor, an epoch average of electrophysiological data values in each labelled epoch of each series of labelled epochs of each ground-truth time-series electrophysiological data; training, by the at least one processor, a seizure forecasting machine learning model using leave-one-out cross validation with of the training datasets based on labels associated with each labelled epoch and the epoch average associated with each labelled epoch; wherein machine learning model is trained on data from single or multiple patients to predict a pre-ictal period probability associated with a forecasted time segment based at least in part on values of the at least one data stream; wherein optimal values for integration window, a time span of the forecasted time segment, a pre-ictal probability threshold, a seizure occurrence period, or combinations thereof are determined by a leave-one-out cross-validation approach or are set according to individual preference; storing, by the at least one processor, the regression machine learning model in a memory upon being trained to predict the pre-ictal period probability.
26. The method as recited in claim 25, wherein the at least one data stream comprises: i) electrodermal activity, ii) heart rate, iii) blood volume pulse, iv) temperature, v) accelerometer-based movement data, or vi) electroencephalogram measurements, vii) time, viii) date, ix) global positioning system data, x) medication, xi) self-reported seizures, xii) clinical patient data, or xiii) combinations thereof.
27-30. (canceled)
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0049] Various embodiments of the present disclosure can be further explained with reference to the attached drawings, wherein like structures are referred to by like numerals throughout the several views. The drawings shown are not necessarily to scale, with emphasis instead generally being placed upon illustrating the principles of the present disclosure. Therefore, specific structural and functional details disclosed herein are not to be interpreted as limiting, but merely as a representative basis for teaching one skilled in the art to variously employ one or more illustrative embodiments. The FIGS. including:
[0050]
[0051]
[0052]
[0053]
[0054]
[0055]
[0056]
[0057]
[0058]
[0059]
[0060]
[0061]
[0062]
[0063]
[0064]
[0065]
[0066]
DETAILED DESCRIPTION
[0067] Various detailed embodiments of the present disclosure, taken in conjunction with the accompanying figures, are disclosed herein; however, it is to be understood that the disclosed embodiments are merely illustrative. In addition, each of the examples given in connection with the various embodiments of the present disclosure is intended to be illustrative, and not restrictive.
[0068] In order to make seizure risk assessments available for broader clinical use methods that build on non-invasive, easily recordable data streams and that can be readily used without the need of an adjustment phase or expert parameter setting are desirable. Peripheral signals recorded using wearable devices, such as wearable sensors, are particularly interesting in this respect since these signals permit continuous, non-invasive recording of several physiological parameters, such as electrodermal activity, body temperature, blood volume pulse and actigraphy. At the same time, the compact design may limit the risk of stigmatization, affords more easy application, and may altogether increase patient adherence relevant for long-term ambulatory use. Furthermore, the data streams may also include signals including additional physiological parameters and patient data such as, e.g., heart rate, accelerometer-based movement data, electroencephalogram measurements, time, date, global positioning system data, medication, self-reported seizures, clinical patient data, electronic medical record data, and combinations thereof. Monitoring of such physiological parameters has already been demonstrated to assist in the detection of generalized tonic-clonic seizures. Similar autonomous system measures may also provide information on detection of preictal patterns or periods.
[0069] Deep learning has been shown to exhibit strong classification performance from complex feature sets. It therefore constitutes a promising technique to differentiate pre- from interictal periods based on complex, multi-modal wearable sensor data. While more traditional machine learning approaches rely on hand-designed feature sets, deep learning uses multiple layers of connections to perform classification tasks without the need of feature designing, which may be an advantage in relatively under-explored, multi-modal datasets, such as data from wrist-worn devices.
[0070]
[0071]
[0072] In some embodiments, a seizure monitoring system 110 using wearable sensor data 102 to forecast seizure risks in a user without the need for expensive and cumbersome EEG, EKG and ECOG tests that would ordinarily be used for assessing seizures. The wearable sensor data 102 can be provided to the seizure monitoring system 110 from a wearable sensor 101 as a continuous stream of data. Thus, the seizure monitoring system 110 may monitor the user's sensor data 102 in real-time, thus enabling timely intervention or mitigation of impending seizures using discrete, wearable sensing devices. As a result, the seizure monitoring system 110 uses peripheral signals from a device having a compact, wearable design that may limit the risk of stigmatization, affords more easy application, and may altogether increase patient adherence relevant for long-term ambulatory use, while also providing effective, real-time monitoring beyond the typical occasional and expensive EEG, ECG and ECOG testing.
[0073] As such, in some embodiments, the wearable sensor 101 can include any suitable sensing device for sensing physiological parameters. In some embodiments, the wearable sensor 101 can include a device for sensing parameters, such as, e.g., electrodermal activity, body temperature, blood volume pulse, heart rate, heart rate variability, blood oxygen content, blood glucose data, electrocardiographic data and actigraphy (accelerometer-based and location-based activity data), electroencephalogram measurements, time, date, medications, self-reported seizures, and combinations thereof. For example, the wearable sensor 101 can include, e.g., a smartwatch, a wristband sensor, a chest strap, a smart ring, or other health tracking sensor device, and combinations thereof. The wearable sensor data 102 may be continuously collected, e.g., at about 60 hertz (Hz), 30 Hz, 20 Hz, 15 Hz, 10 Hz, 5 Hz, 1 Hz or other sampling rate. Thus, the seizure monitoring system 110 is provided with continuous streams of each physiological parameter in the wearable sensor data 102.
[0074] In some embodiments, the seizure monitoring system 110 may receive the wearable sensor data 102 as a continuous data stream of each of the physiological parameters. In some embodiments, the seizure monitoring system 110 may use a combination of software and hardware components to record and process the data to forecast a risk of the user experiencing a seizure and generate an alert to the user indicating the risk.
[0075] Examples of software components may include programs, applications, operating system software, middleware, firmware, software modules, routines, subroutines, functions, methods, procedures, software interfaces, application program interfaces (API), instruction sets, computer code, computer code segments, words, values, symbols, or any combination thereof. Determining whether an embodiment is implemented using hardware elements and/or software elements may vary in accordance with any number of factors, such as desired computational rate, power levels, heat tolerances, processing cycle budget, input data rates, output data rates, memory resources, data bus speeds and other design or performance constraints.
[0076] Examples of hardware components may include processors 111, microprocessors, circuits, circuit elements (e.g., transistors, resistors, capacitors, inductors, and so forth), integrated circuits, application specific integrated circuits (ASIC), programmable logic devices (PLD), digital signal processors (DSP), field programmable gate array (FPGA), logic gates, registers, semiconductor device, chips, microchips, chip sets, and so forth. In some embodiments, the one or more processors may be implemented as a Complex Instruction Set Computer (CISC) or Reduced Instruction Set Computer (RISC) processors; x86 instruction set compatible processors, multi-core, or any other microprocessor or central processing unit (CPU). In various implementations, the one or more processors may be dual-core processor(s), dual-core mobile processor(s), and so forth.
[0077] In some embodiments, the hardware components may also include a data storage 112. In some embodiments, the data storage 112 may include, e.g., a suitable memory or storage solutions for providing electronic data to the seizure monitoring system 110. For example, the data storage 112 may include, e.g., a centralized or distributed database, cloud storage platform, decentralized system, server or server system, among other storage systems, or the data storage 112 may include, e.g., a hard drive, solid-state drive, flash drive, or other suitable storage device, or the data storage 112 may include, e.g., a random access memory, cache, buffer, or other suitable memory device, or any other data storage solution and combinations thereof.
[0078] In some embodiments, the data storage 112 may receive and record the continuous stream of wearable sensor data 102 among other patient data, including electronic medical record data, clinical data, radiological and other imagery and test results, medication and medication dosages, and any other health-related data, such as any data from an electronic medical health record or other health record. The wearable sensor data 102 may be accessible by, e.g., a seizure forecasting model 120 and an alert engine 130, e.g., via the processor 111. However, in some embodiments, the wearable sensor data 102 may be provided directly to the forecasting model 120 and the alert engine 130 before or instead of being stored in the data storage 112.
[0079] In some embodiments, the seizure forecasting model 120 includes a combination of hardware and/or software for predicting seizure risk at a given time based on the wearable sensor data 102, or a subset of the wearable sensor data 102 pertaining to a selected segment of time preceding the given time. In some embodiments, the seizure forecasting model 120 may predict a seizure risk level once every prediction period. In some embodiments, the prediction period may be, e.g., every second, every ten seconds, every fifteen seconds, every twenty seconds, every thirty seconds, every minute, or other suitable period. Thus, for each prediction period, the seizure forecasting model 120 may develop a seizure risk prediction for that prediction period based on the wearable sensor data 102 and other health-related data associated with the prediction period. In some embodiments, the prediction periods are continuous, non-overlapping segments of time including the wearable sensor data 102 during that time.
[0080] However, in some embodiments, the prediction periods may overlap, such that the time segment preceding the given time at which a seizure risk prediction is made overlaps with a previous time segment for predicting seizure risk at a previous time. For example, a first 30 second prediction period may include the time segment from t=0 seconds to t=30 seconds for a prediction at t=30 of seizure risk, with a second 30 second prediction period including the time segment from t=15 seconds to t=45 seconds for a prediction at t=45 of seizure risk. Thus, the prediction period may include a moving time window approach that may form predictions based on windows having a size according to the prediction period and move according to, e.g., an update period. In some embodiments, the update period may be any suitable increment of time less than the prediction period.
[0081] In some embodiments, the seizure forecasting model 120 may include, e.g., a suitable machine learning algorithm for classifying a seizure risk based on the physiological parameters of the wearable sensor data 102 and health-related data, such as electronic medical record data. The classification of seizure risk can include a period type relative to a seizure, or ictal period. For example, the seizure forecasting model 120 may forecast a seizure risk based on a classification of any given time period as, e.g., preictal, postictal or interictal.
[0082] Thus, in some embodiments, the prediction based on the wearable sensor data 102 for a particular prediction period results in a seizure risk forecast including the classification of that the prediction period as preictal, postictal or interictal. However, in some embodiments, since postictal periods follow a seizure event, the predictive power of such periods are low. Thus, the seizure forecasting model 120 may be configured to predict whether a prediction period is preictal or not preictal (e.g., interictal and/or postictal). For example, the seizure forecasting model 120 may be trained to recognize physiological parameters during a time period that would indicate that preictal period correspond to a seizure being imminent within, e.g., about 61 minutes of a current time.
[0083] Thus, in some embodiments, the seizure forecasting model 120 may include a machine learning model for differentiating between the periictal periods, and in particular, in distinguishing a preictal period indicating a high risk of an impending seizure, e.g., a high risk of a seizure occurring within a seizure occurrence period, e.g., within about 61 minutes of the high risk indication. In some embodiments, a data segment may be preictal if it occurs between 61 minutes and 1 minute prior to a seizure, thus leaving a one-minute buffer prior to seizure onset. This preictal window definition is commensurate with other seizure forecasting research using EEG and ECoG and may account for potential small ambiguities in determining the exact seizure onset between EEG and wristband. A data segments may classified as interictal or not preictal to indicate that the associated prediction period is two hours or more away from any seizure.
[0084] To do so, in some embodiments, the seizure forecasting model 120 may include artificial intelligence (AI) or machine learning techniques for forecasting a preictal period based on wearable sensor data 102 of physiological parameters and physiological data from, e.g., electronic medical health records, the techniques chosen from, but not limited to, decision trees, boosting, support-vector machines, neural networks, nearest neighbor algorithms, Naive Bayes, bagging, random forests, and the like. In some embodiments and, optionally, in combination of any embodiment described above or below, an exemplary neutral network technique may be one of, without limitation, feedforward neural network, radial basis function network, recurrent neural network, convolutional network (e.g., U-net), long short-term memory network or other suitable network. In some embodiments and, optionally, in combination of any embodiment described above or below, an exemplary implementation of Neural Network may be executed as follows: [0085] i) define Neural Network architecture/model, [0086] ii) transfer the input data to the exemplary neural network model, [0087] iii) train the exemplary model incrementally, [0088] iv) determine the accuracy for a specific number of timesteps, [0089] v) apply the exemplary trained model to process the newly-received input data, [0090] vi) optionally and in parallel, continue to train the exemplary trained model with a predetermined periodicity.
[0091] In some embodiments and, optionally, in combination of any embodiment described above or below, the exemplary trained neural network model may specify a neural network by at least a neural network topology, a series of activation functions, and connection weights. For example, the topology of a neural network may include a configuration of nodes of the neural network and connections between such nodes. In some embodiments and, optionally, in combination of any embodiment described above or below, the exemplary trained neural network model may also be specified to include other parameters, including but not limited to, bias values, functions and aggregation functions. For example, an activation function of a node may be a step function, sine function, continuous or piecewise linear function, sigmoid function, hyperbolic tangent function, or other type of mathematical function that represents a threshold at which the node is activated. In some embodiments and, optionally, in combination of any embodiment described above or below, the exemplary aggregation function may be a mathematical function that combines (e.g., sum, product, etc.) input signals to the node. In some embodiments and, optionally, in combination of any embodiment described above or below, an output of the exemplary aggregation function may be used as input to the exemplary activation function. In some embodiments and, optionally, in combination of any embodiment described above or below, the bias may be a constant value or function that may be used by the aggregation function and/or the activation function to make the node more or less likely to be activated.
[0092] In some embodiments, the classification from the seizure forecasting model 120 can be provided to an alert engine 130 for generating an alert that a seizure event may be imminent based on the prediction period being a preictal period. In some embodiments, the seizure forecasting model 120 may first provide each seizure classification for each prediction period to the data storage 112 to record the prediction period with an indication of the associated classification. In some embodiments, the classification may be provided to the data storage 112, which may then be accessed by the alert engine 130, or the seizure forecasting model 120 may provide the classification directly to the alert engine 130, either before, after or concurrently with recording the classification in the data storage 112.
[0093] In some embodiments, the classification from the seizure forecasting model 120 can include a binary classification (e.g., preictal or not preictal). The binary classification can include a probability that a given prediction period is a preictal period based on the wearable sensor data 102. For example, the classification may include a numerical value on a scale from 0 to 1, where 0 indicates a zero percent probability of the prediction period being a preictal period, and where 1 indicates a one hundred percent probability of the prediction period being a preictal period. In practice, any given prediction period is unlikely to be a 0 or a 1, but likely may be classified somewhere in between.
[0094] In some embodiments, the alert engine 130 may determine that the probability of the classification indicates a preictal period using, e.g., a risk threshold. For example, where the probability rises above a risk threshold of, e.g., 0.5, 0.52, 0.54, 0.56, 0.58, 0.6, the alert engine 130 may determine that the prediction period is a preictal period, thus indicating that a seizure is imminent within about an hour. Thus, the alert engine 130 may generate an alert to the user. In some embodiments, the probability for each prediction period may be compared to the risk threshold.
[0095] However, irregularities may occur at any particular prediction period that may give rise to a high probability of the preictal period classification for one prediction period. Thus, a seizure may only be actually imminent where the physiological parameters consistently indicate a preictal period according to the associated classification probabilities. Therefore, in some embodiments, an integration window may be employed where the preictal classification probabilities for each prediction period within a windowed time span may be aggregated and then compared to the risk threshold. For example, the integration window may encompass a time span including, e.g., 90, 120, 150, 300, 600, or 1200 seconds.
[0096] In some embodiments, the alert generator 130 may generate an alert including, e.g., a visual indication via a graphical user interface, an audible indication, a vibration or tactile indication, or other alert notifying the user of the risk of a seizure based on the preictal classification. In some embodiments, the alert may be provided to a user computing device 103 or to the wearable sensor 101, or both. In some embodiments, the user computing device 103 may include, e.g., a personal computer, mobile device, wearable device, tablet computer, or other computing device associated with the user. For example, the user computing device 103 and/or the wearable sensor 101 may display the visual indication and/or emit the audible, vibration and/or tactile indication such that the user may perceive the alert of the risk of an imminent seizure. As a result, the user may receive a real-time warning for imminent seizures, enabling the user to take preventative or mitigating steps to avoid harm that may result from a seizure. Similarly, the user may receive a real-time indication that seizure risk is low, a real-time indication of the current seizure risk at any time, or other real-time seizure risk indication techniques. The alert generator 130 may also be configured to determine a mitigation or treatment strategy along with the alert, such as, e.g., a notification to stop a car, lie down, ingest a prescribed medication, etc. The alert generator 130 may also be embedded in a closed-loop setup linked to a device to administer treatment and thereby lower the risk of a seizure or prevent it completely. This treatment device may include a system to apply a fast-acting antiseizure medication or a neuromodulatory device which, for example, administers electrical stimulation to the brain in order to decrease seizure risk.
[0097]
[0098] In some embodiments, seizure forecasting builds on the notion that a preictal period, during which a seizure is more likely to occur soon, can be reliably distinguished from interictal periods. However, previous studies have focused on data recorded either from ECoG and EEG or from ECG. ECG has thus been a long-standing example that peri- and preictal changes can not only be detected within the central nervous system but are also reflected in a variety of cardiac effects. Cardiac activity is controlled by parasympathetic and sympathetic branches of the autonomic nervous system, with the former producing an inhibitory response and the latter producing an excitatory response on heart rate. Preictal changes in brain activity that occur in or propagate to autonomic control centers may affect this autonomic balance and, consequently, affect cardiac activity during the leadup to a seizure.
[0099] In some embodiments, it is assumed that single-channel ECG contains a comparable amount of information to multi-channel EEG. Therefore, peripheral sensors, such as the wearable sensor 101 may be relevant for seizure forecasting. In some embodiments, autonomous nervous system changes are correlated to the wristband sensor data 102 in several ways: electrodermal activity is known to be sensitive to sympathetic innervation; blood volume pulse curves contain information about heartrate which is controlled by the parasympathetic and sympathetic interplay; and body temperature is similarly known to be maintained by the autonomic nervous system. In some embodiments, the peripheral biomarker parameters are built on by monitoring both these autonomous nervous system functions along with actigraphy, which indirectly also monitors resting periods and sleep. Thus, in some embodiments, the seizure forecasting model 120 and alert engine 130 are configured to leverage such multi-modal wearable sensor data 102, going beyond more traditional ECoG/EEG and ECG approaches to enable real-time, ever-present seizure monitoring to warn users of impending seizures.
[0100] In particular, in some embodiments, the seizure forecasting model 120 receives the multi-modal wearable sensor data 102 and analyzes the data with a sampler 222, segment extractor 224 and seizure forecasting neural network 226 to predict a seizure risk according to the inferences described above correlating multimodal peripheral biomarker data to preictal period signatures.
[0101] In some embodiments, the seizure forecasting model 120 employs the sampler 222 to standardize and down-sample the wearable sensor data 102 streams of each parameter to standardize data vector lengths. For example, as described above, the parameters may include, e.g., blood volume pulse, electrodermal activity, body temperature, and actigraphy. In some embodiments, the actigraphy includes accelerometer based body movement data, including x-axis acceleration measurement, y-axis acceleration measurement and z-axis acceleration measurements. Thus, the wearable sensor data 102 may include six data streams including a blood volume pulse data stream, an electrodermal activity data stream, a body temperature data stream, an x-axis acceleration data stream, a y-axis acceleration data stream and a z-axis acceleration data stream. However, not all of these data streams may collect data at the same sample rate. Thus, the sampler 222 may down-sample all data streams to a sample rate associated with the lowest sample rate of the data streams. However, the data streams may be down-sampled further to, e.g., reduce unnecessary data, thus improving efficiency. Moreover, a lower data stream may reduce overfitting of the seizure forecasting neural network 226 upon optimization. In some embodiments, the sampler 222 is configured to downsample the data streams of the wearable sensor data 102 to, e.g., between about 2 Hz and about 10 Hz, and may be about 4 Hz.
[0102] Moreover, the sampler 222 may also sample other medical and clinical data pertaining to the wearer of the wearable sensor device 101. For example, the sampler 222 may sample, e.g., test results, medication and medication dosages, EEG data, ECoG data, diagnoses, genetic information such as gene presence and gene expression, among other information. While some of medical and clinical data may be static between visits with a doctor or clinician, the sampler 222 may convert parameters of the medical and clinical data as a continuous data stream, e.g., as a 4 Hz continuous signal similar to the sampled wearable sensor data 102 by representing a data point as a constant value signal between each change. For example, where a patient is given a particular dosage of a particular medication, a signal representing the dosage of the medication may be employed as a constant value four times per second (e.g., for a 4 Hz sample rate) until a clinician or doctor prescribes a new dosage or new medication, or both. Thus, each medical parameter of the medical and clinical data may be represented as a time-series for ease of used alongside the wearable sensor data 102.
[0103] In some embodiments, the sampler 222 may filter and/or down-sample the wearable sensor data 102 and medical and clinical data streams in real-time as it is received by the seizure forecasting model 120. The filtered and/or down-sampled data may then be extracted in time segments based on the prediction period described above. As such, the segment extractor 224 may receive the filtered and/or down-sampled data streams and extract overlapping or non-overlapping continuous segments of data from each data stream, where each segment includes the data within the prediction period, e.g., about 30 seconds. As a result, the segment extractor 224 may generate a series of, e.g., 30 second segments of 4 Hz data streams (about 120 data points) from each of the six data streams associated with the six peripheral biomarker parameters.
[0104] In some embodiments, the segment extractor 224 may extract the time segments of data from the data streams in the wearable sensor data 102 before the data streams are sampled by the sampler 222. Thus, the original data streams may have segments extracted therefrom, and then the sampler 222 may downsample the data segments. Where overlapping segments are employed, such an approach may result in increased computation by down-sampling the overlapping portions of segments more than once. However, accuracy and effectiveness may nevertheless be comparable.
[0105] In some embodiments, the segments of down-sampled data streams are received by the seizure forecasting neural network 226 to classify each segment according to a probability of being within a preictal period. In some embodiments, there exists a preictal signature in autonomous nervous system and actigraphy data, which, despite possibly not being detectable by visual inspection, may be picked up by deep learning. This signature may be learned across patients and is therefore not patient-specific. This is an advance from most traditional seizure prediction work, which mostly succeeded when using algorithms designed specifically for each individual patient. An algorithm that can be trained across patients has the advantage that it can be readily employed to a new patient, without any training to learn patient-specific factors or expert knowledge to set parameters.
[0106] In some embodiments, the deep learning used to form the non-patient-specific algorithm may be a suitable classification algorithm with robust classification performance based on multi-dimensional timeseries data, while being resistant to overfitting. For example, a network of long short-term memory (LSTM) units or neurons may be employed. In some embodiments, to limit LSTMs from overfitting, network architecture was kept simple and shallow. For example, the LSTM network may employ, e.g., about 20 or fewer nodes, or about 10 or fewer nodes with a dropout rate between about, e.g., 0.4 and 0.6 and a recurrent dropout rate between about, e.g., 0.4 and 0.6. Thus, an LSTM based seizure forecasting neural network 226 may be configured according to Table 1 below:
TABLE-US-00001 TABLE 1 Layer Layer number type Nodes/units Layer parameters 1 LSTM 10 dropout rate = 0.5, recurrent dropout rate = 0.5 2 Dense 10 3 Dropout N/A dropout rate = 0.7 4 Dense 1 activation = sigmoid
[0107] However, in some embodiments, 1-dimensional convolutional neural networks (1DConv) may also or alternatively be employed due to their good performance on timeseries data while often being easier and faster to train than LSTM networks. Thus, in some embodiments, the seizure forecasting neural network 226 may employ a 1DConv, for example, with a network of about, e.g., 100 or fewer nodes, or about 81 or fewer nodes, or about 64 or fewer nodes, and about, e.g., 2, 3, 4, or 5 units. In some embodiments, the 1DConv may employ any suitable activation function, such as, e.g., a sigmoid function, a tanh function, rectified linear units (ReLu), leaky ReLu, a maxout function, exponential linear units (ELU), or other activation function. For example, the seizure forecasting neural network 226 may employ a 1DConv as set forth in Table 2 below:
TABLE-US-00002 TABLE 2 Layer Layer number type Nodes/units Layer parameters 1 Conv1D 64/2 activation = ReLu 2 MaxPooling1D 2 N/A 3 Dropout N/A dropout rate = 0.7 4 Dense 50 activation = ReLu 5 Dropout N/A dropout rate = 0.7 6 Dense 1
[0108] In our approach, the same seizure forecasting neural network 226 may be employed for all users. In some embodiments, the seizure forecasting neural network 226 may be trained separately for each user, however, in some embodiments, the seizure forecasting neural network 226 is trained on a pool of training patients and then used for individual users. In some embodiment, the seizure forecasting neural network 226 may be trained initially on all patients and the further specified to the individual patient by additional training on the individual patient's data. While it is possible that model hyperparameters may be individualized for each patient, which may bring about better performance, the same model architecture may nevertheless be employed across patients to facilitate an “out-of-the-box” solution for future prospective settings and users.
[0109] In some embodiments, the seizure forecasting neural network 226 ingests each time segment of down-sampled wearable sensor data and generates a preictal probability 203 of each time segment being part of a preictal period, e.g., on a scale from 0 to 1, as described above. However, to increase the usability of the preictal probabilities 203, the alert engine 130 utilizes the preictal probabilities 203 to alert a user when the user is likely in a preictal period, thus warning the user of an impending seizure.
[0110] In some embodiments, the alert engine 130 employs a sliding window approach in which the individual 30-second segment preictal predictions are statistically aggregated over an integration window. If this statistically aggregated value crosses a risk threshold, an alarm may be initiated which may last for the duration of a seizure occurrence period. Thus, in some embodiments, a new alarm may only be initiated once the seizure occurrence period has passed. This post-processing thus employs three additional variables: integration window, threshold and seizure occurrence period. In long-term recordings, parameters like this can in principle be optimized at the individual patient level, for example by optimizing these parameters during an initial adjustment phase. However, in some embodiments, these parameters may be determined, e.g., using a grid search of training data for the parameters yield a greatest improvement-over-chance (IoC) (for example, see,
[0111] Accordingly, in some embodiments, the preictal predictions 203 may be received by the time window generator 232 to implement the integration window. As such, the time window generator 232 may generate a super-segment of all time segment data and the associated preictal predictions 203 within the period of the integration window. For example, where the integration window value is 600 seconds, the time window generator 232 may identify all of the time segments of wearable sensor data 102 within a 600 second time span and generate a time window of preictal predictions 203 associated with the time segments to compile all preictal predictions 203 within the time window.
[0112] In some embodiments, the time window generator 232 compiles preictal predictions 203 in a rolling time window fashion, continuously updating the set of preictal predictions 203 to update the time windowing of preictal predictions 203 with each new preictal prediction. However, other time windowing techniques may be employed.
[0113] In some embodiments, the alert generator 234 may statistically aggregate the preictal predictions 203 of each time window of preictal predictions 203 as described above to generate a time window preictal prediction value. In some embodiments, the statistical aggregation may include an averaging of all preictal predictions 203 within the time window. However, in some embodiments, the preictal predictions 203 in a time window may be summed or the segment median may be determined, a weighted average or other statistical operation may be performed, or the individual preictal predictions 203 in the time window may be integrated in a weighted manner, for example, using leaky-integrate-and-fire neural networks, among other aggregation techniques to form a value representing the segment of preictal predictions 203 of the time-window.
[0114] In some embodiments, the alert generator 234 may determine whether the time window is a likely to be a preictal period by comparing the time window preictal prediction value with the threshold 236. As described above, the threshold 236 may be, e.g., about 0.50, 0.52, 0.54, 0.56, 0.58, 0.60 or other suitable threshold. In some embodiments, the threshold 236 may be about 0.54. Thus, where the time window preictal prediction value exceeds the threshold 236 of 0.54, the alert generator 234 may determine that the time window corresponds to a preictal period and generate an impending seizure alert 204. In some embodiments, the impending seizure alert 204 may be provided as, e.g., an audible, visual, and/or tactile indication at a user's device (e.g., the wearable sensor 101 or a user computing device 103, as described above), such that the user may be quickly and effectively warned of impending seizures in real-time.
[0115] However, continually receiving alerts 204 with every new preictal prediction 203 as the alert engine 130 updates may be annoying and even harmful when a seizure may be imminent. Thus, the occurrence period calculator 238 may utilize the above described occurrence period to prevent new alerts 204 until after the occurrence period has passed. This is because once the user is in a preictal period, each successive prediction period resulting in an additional preictal prediction is likely to indicate a continuation of the preictal period until after the seizure has occurred. Thus, based on the optimal occurrence period identified above, further alerts 204 can be prevented until the occurrence period calculator 238 determines that the seizure occurrence has passed.
[0116] Accordingly, in some embodiments, the time at which the alert 204 is generated may be logged by the occurrence period calculator 238 and a timer may be started. The timer may run until the end of the occurrence period. During the occurrence period prior to the end of the timer, the occurrence period calculator 238 may prevent the alert generator 234 from generating new alerts. In some embodiments, the occurrence period calculator 238 may even prevent the alert engine 130 in general from updating with new preictal predictions 203. However, in some embodiments, the time window generator 232 may continue to receive new preictal predictions 204 for rolling the integration window. The alert generator 234 may also continue to statistically aggregate the preictal predictions of the updated time window and, e.g., rescind the alert 204 when the statistical aggregation falls below the threshold 236.
[0117] In some embodiments, one may tune the integration window, threshold, and occurrence period parameters for an individual user's needs, for example during an initial adjustment phase, for optimal future performance and patient preferences.
[0118]
EXAMPLE 1
Training an LSTM-Based Seizure Forecasting Model
[0119] Data Recording and Preprocessing
[0120] In some embodiments, patients 301 and 302 with epilepsy were selected for training the seizure forecasting neural network 224 and admitted to long-term video-EEG monitoring (LTM) unit and fit with a biosensor wristband on either left or right wrist or ankle for long-term recording. In this example, all patients with wristband recordings were considered from February 2015 until October 2018. Data from one wristband per patient only was considered; when a patient recording involved multiple wristbands (e.g. from wrist and ankle), the data from the biosensor wristband with the longest total recording time was selected for further analysis. From all the patients monitored by wristband recording (about 317 patient), only the patients with at least one seizure during the wristband recording period were included, limiting the analysis to 69 patients (see, for example Table 3 below).
TABLE-US-00003 TABLE 3 Age of First Age Seizure Wristband Patient Gender [Years] [Years] Seizure Types MRI Findings Etiology Location 1 male 15 13 focal onset not noted structural left wrist 2 male 13 0 focal onset normal unknown left ankle 3 female 17 unknown focal onset normal unknown left wrist 4 female 2 unknown focal onset volume loss, structural left ankle unspecified 5 female 5 4 unclassified malformation structural left wrist 6 female 15 3 generalized onset, gliosis, structural right wrist unclassified unspecified 7 female 22 10 focal onset infarction structural right ankle 8 female 16 15 focal onset normal unknown left ankle 9 male 17 7 focal onset normal unknown right wrist 10 male 3 2 focal onset dysplasia structural right ankle 11 female 14 unknown generalized onset cyst unknown left wrist 12 male 11 1 focal onset normal unknown left wrist 13 male 10 1 focal onset malformation structural right wrist 14 female 13 11 focal onset, volume loss, structural left wrist unclassified unspecified 15 male 16 10 focal onset normal unknown left wrist 16 male 8 7 generalized onset normal unknown left ankle 17 female 2 unknown focal onset, tuberous genetic left ankle unclassified sclerosis/hamartoma 18 male 9 1 focal onset, not noted unknown right ankle generalized onset 19 male 10 8 focal onset volume loss, unknown right wrist unspecified 20 male 9 5 focal onset, normal unknown left ankle generalized onset 21 male 5 0 generalized onset normal unknown left wrist 22 female 14 7 focal onset infarction structural right wrist 23 female 15 13 focal onset, volume loss, genetic left ankle generalized onset unspecified 24 female 3 0 generalized onset, resection structural right ankle unclassified 25 male 13 0 focal onset tuberous genetic right ankle sclerosis/hamartoma 26 male 0 0 focal onset malformation structural right ankle 27 female 27 14 generalized onset normal unknown right wrist 28 male 17 15 focal onset tumor structural right ankle 29 male 13 0 focal onset resection unknown left wrist 30 female 2 1 generalized onset not noted unknown right ankle 31 male 8 1 focal onset volume loss, structural right wrist unspecified 32 female 15 0 focal onset, hippocampal structural right ankle unclassified sclerosis 33 male 7 4 focal onset infarction structural left wrist 34 female 17 1 generalized onset not noted unknown right wrist 35 female 3 unknown focal onset volume loss, structural right ankle unspecified 36 female 6 3 generalized onset, gliosis, structural left wrist unclassified unspecified 37 male 1 0 generalized onset not noted unknown left ankle 38 female 11 7 generalized onset normal unknown right wrist 39 male 12 6 generalized onset volume loss, structural right wrist unspecified 40 male 7 unknown generalized onset volume loss. metabolic right ankle unspecified 41 male 3 1 focal onset resection structural right wrist 42 female 10 1 focal onset, dysplasia structural right wrist unclassified 43 female 13 0 focal onset infarction structural left wrist 44 male 2 0 generalized onset, tuberous genetic left ankle unclassified sclerosis/hamartoma 45 male 4 2 focal onset volume loss, structural left ankle unspecified 46 female 5 0 focal onset infarction structural right ankle 47 male 8 5 focal onset tumor structural right ankle 48 female 13 9 focal onset, normal unknown left ankle unclassified 49 male 12 7 focal onset normal unknown right wrist 50 male 0 0 generalized onset, not noted unknown right ankle unclassified 51 male 9 unknown generalized onset normal unknown right wrist 52 female 13 3 focal onset volume loss, structural right wrist unspecified 53 female 11 1 focal onset hippocampal structural right ankle sclerosis 54 male 1 unknown focal onset dysplasia structural left ankle 55 male 14 0 focal onset, resection structural left wrist unclassified 56 female 11 10 generalized onset tumor structural right wrist 57 female 19 0 generalized onset volume loss, unknown right ankle unspecified 58 male 7 1 focal onset normal unknown left ankle 59 male 14 7 generalized onset dysplasia structural left ankle 60 male 2 0 generalized onset dysplasia unknown left ankle 61 male 12 1 focal onset not noted structural right wrist 62 male 9 8 focal onset malformation unknown left ankle 63 male 0 0 unclassified infarction structural right ankle 64 male 9 1 focal onset normal unknown right ankle 65 male 10 0 focal onset cyst unknown left ankle 66 male 5 unknown focal onset malformation structural right ankle 67 female 3 1 generalized onset, normal genetic left ankle unclassified 68 male 11 unknown generalized onset normal unknown left wrist 69 male 21 3 focal onset, volume loss, genetic right ankle generalized onset unspecified
[0121] In some embodiments, the patient monitoring data may be used to evaluate whether forecasting solely based on wristband data can deliver better-than-chance and clinically meaningful performance. A prerequisite for seizure risk assessment is the reliable distinction between pre- and interictal periods. For this purpose, continuous, non-overlapping 30-second segments of wristband recordings composed of six sensor data streams (electrodermal activity (EDA), accelerometer data in three dimensions, blood volume pulse (BVP), and temperature (TEMP); see,
[0122] Preparation of Training, Validation and Test Data
[0123] For the main results, a leave-one-out cross-validation approach was applied where data from 68 patients were used for training (training set 301), and testing was done on the full dataset of the one remaining out-of-sample patient (testing set 302). This allows for maximization of the number of patients included in training and testing. Additional analyses were performed to train on lower number of patients and use other patients for validation before testing on one out-of-sample test patient (see,
[0124] For the preparation of the training datasets 301, all 30-second preictal segments (preictal 331, preictal 341 through preictal 351) were identified and matched with an equal number of randomly chosen interictal segments in each patient (interictal 332, interictal 342 through interictal 352, see
[0125] Analysis may also be performed on training data with less than 68 patients where the remaining patients (apart from the test patient) are used for validation. Based on monitoring these validation data, a prevention of overfitting may be validated (see,
[0126] Neural Networks and Training
[0127] In some embodiments, the seizure forecasting neural network 224 employs long short-term memory networks (LSTMs), as they are specifically designed for learning underlying representations in timeseries data and have been shown to provide robust classification performance based on multi-dimensional timeseries data. To use the wristband sensor data in a LSTM networks, data was down-sampled (e.g., using the sampler 222 described above) to 4 Hz for all sensors in order to provide the same vector length for each 30-second segment (i.e. 120 sampling points). To limit LSTMs from overfitting, network architecture may kept simple and shallow (Table 1,
[0128] In some embodiments, the training may be performed by using the LSTMs of the seizure forecasting neural network 224 to predict a binary classification of preictal periods, and comparing the classification to the labeled data of the training dataset 301 with an optimizer 326 to determine a loss. In some embodiments, the optimizer 326 may employ an optimization function such as, e.g., gradient descent with backpropagation through time, connectionist temporal classification (CTC), neural evolution, or other optimization technique. Thus, the optimizer 326 may determine an error between the predictions of the seizure forecasting neural network 224 and adjust LSTM parameters to improve accuracy. In some embodiments, the seizure forecasting neural network 224 was trained for 200 epochs (see,
[0129] Performance Metrics and Statistical Tests
[0130] Seizure forecasting performance may be assessed on the full timeseries data (after removal of potential dropouts, e.g. when the wristband was replaced by a new one for charging purposes) from out-of-sample test patients.
[0131] As clinically useful metrics to evaluate forecasting algorithm performance, the loss validation engine 328 may employ metrics including: sensitivity (i.e. the true positive seizure prediction rate; seizures outside the recording period were not considered), time in warning (i.e. the fraction of time spent in warning) and improvement over chance (IoC, defined as the difference between sensitivity and time in warning). Mean prediction scores may be determined for these variables over ten independent runs for each patient where each run corresponds to an independently trained seizure forecasting neural network 224.
[0132] A two-sided Wilcoxon signed-rank test may used by the loss validation engine 328 to assess significance of IoC-values in each patient. Multiple comparisons may be controlled for using a Benjamini and Hochberg false discovery rate using a threshold of 0.05.29 A two-sided Mann-Whitney U test may be applied for comparison between groups. In some embodiments, a p of 0.05 or less may be significant.
[0133] Results
[0134] In some embodiments, the dataset 301 and 302 includes multi-day recordings from 69 epilepsy patients (mean age 9.8±5.9 years (mean±s.d.), 28 female, total duration 2311.4 hours, 452 seizures; Table 3). The performance of the proposed seizure forecasting system is evaluated in terms of sensitivity, time in warning and improvement over chance (IoC). A leave-one-out cross-validation approach is applied where matched pre-/interictal data from 68 patients are used for training (
[0135] For practical application as a warning system for patients the expected time between alarm onset and seizure onset, the prediction horizon (
[0136] Prediction performance being dependent on seizure type may also be assessed, in particular whether performance depends on seizures of focal or generalized onset. Thus, prediction performance between patients was compared with only focal onset seizures (n=35 patients) and patients with only generalized onset of seizures (n=16 patients; Table 3). Group comparison reveals no significant difference in IoC values (p=0.44). Similarly, no significant dependence on where the device was worn (wrist: n=31 patients, ankle: n=38 patients) in terms of IoC performance values was revealed by group comparison (p=0.24).
[0137] Machine leaning, and particularly deep learning, benefits from large datasets that afford learning of the underlying data representations while also containing enough variability to permit generalization to unseen data. In such a use-case, performance may take a certain amount of data and, more generally, benefits from training on larger datasets. To determine the relationship between seizure forecasting performance and size of the training dataset, and to obtain a better understanding of how a deep learning approach might benefit from more data in the future, performance may be evaluated under different amounts of training data. For this purpose, instead of training on all 68 patients in a leave-one-out approach, as described above, the amount of training data may be systematically reduced by considering only a smaller number of patients (n=4, 8, 16, 32 or 55 patients) for training. Specifically, performance for each test patient was calculated for ten independently trained networks where training data was composed of only n number of randomly chosen patients in each run.
EXAMPLE 2
Training a 1DConv-Based Seizure Forecasting Model
[0138] In some embodiments, the patients 301 and 302 were also used to train and test a seizure forecasting neural network 224 using the leave one out cross validation approach described above for the LSTM-based seizure forecasting model. For the 1DConv-based seizure forecasting neural network 224, preprocessing and the preparation of the training dataset 301, validation and testing dataset 302 are the same. However, for the 1DConv-based seizure forecasting neural network 224, the seizure forecasting neural network 224 employs a 1-dimensional convolutional neural network. 1DConv networks may be easier and faster to train than LSTM networks while also exhibiting good performance on timeseries data. Table 2 shows a summary of the 1DConv network parameters used. Similar to the LSTM network, the 1DConv network was trained for 200 epochs with analyses performed with in- house written code using, e.g., Python and Keras with Tensorflow backend.
[0139] Similar to the LSTM networks described above, data was down-sampled (e.g., using the sampler 222 described above) to 4 Hz for all sensors in order to provide the same vector length for each 30-second segment (i.e. 120 sampling points) for input to the 1DConv networks. To limit 1DConv networks from overfitting, network architecture may kept simple and shallow (Table 2), and training may be performed on matched data, i.e. where both classes appeared equally often.
[0140] In some embodiments, the training may be performed by using the 1DConv-based seizure forecasting neural network 224 to predict a binary classification of preictal periods, and comparing the classification to the labeled data of the training dataset 301 with an optimizer 326 to determine a loss. In some embodiments, the optimizer 326 may employ an optimization function such as, e.g., gradient descent with backpropagation through time, connectionist temporal classification (CTC), neural evolution, or other optimization technique. Thus, the optimizer 326 may determine an error between the predictions of the seizure forecasting neural network 224 and adjust 1DConv parameters to improve accuracy. In some embodiments, the seizure forecasting neural network 224 was trained for 200 epochs.
[0141] Performance Metrics and Statistical Tests
[0142] Seizure forecasting performance may be assessed on the full timeseries data (after removal of potential dropouts, e.g. when the wristband was replaced by a new one for charging purposes) from out-of-sample test patients.
[0143] Using leave-one-out cross-validation approach, the training data is used to find the optimal parameters using a grid search implemented by the loss validation engine 328 (
[0144] As clinically useful metrics to evaluate forecasting algorithm performance, the loss validation engine 328 may employ metrics including: sensitivity (i.e. the true positive seizure prediction rate; seizures outside the recording period were not considered), time in warning (i.e. the fraction of time spent in warning) and improvement over chance (IoC, defined as the difference between sensitivity and time in warning). Mean prediction scores may be determined for these variables over ten independent runs for each patient where each run corresponds to an independently trained seizure forecasting neural network 224.
[0145] A two-sided Wilcoxon signed-rank test may used by the loss validation engine 328 to assess significance of IoC-values in each patient. Multiple comparisons may be controlled for using a Benjamini and Hochberg false discovery rate using a threshold of 0.05.29 A two-sided Mann-Whitney U test may be applied for comparison between groups. In some embodiments, a p of 0.05 or less may be significant.
[0146] Results
[0147] In some embodiments, the dataset 301 and 302 includes multi-day recordings from 69 epilepsy patients (mean age 9.8±5.9 years (mean±s.d.), 28 female, total duration 2311.4 hours, 452 seizures; Table 3). The performance of the proposed seizure forecasting system is evaluated in terms of sensitivity, time in warning and improvement over chance (IoC). A leave-one-out cross-validation approach is applied where matched pre-/interictal data from 68 patients are used for training (
[0148] Prediction performance between patients was compared with only focal onset seizures (n=35 patients) and patients with only generalized onset of seizures (n=16 patients; Table 3). Group comparison reveals no significant difference in IoC values (p=0.44). Similarly, no significant dependence on where the device was worn (wrist: n=31 patients, ankle: n=38 patients) in terms of IoC performance values was revealed by group comparison (p=0.24).
[0149]
[0150]
[0151] In some embodiments, referring to
[0152] In some embodiments, the exemplary network 1405 may provide network access, data transport and/or other services to any computing device coupled to it. In some embodiments, the exemplary network 1405 may include and implement at least one specialized network architecture that may be based at least in part on one or more standards set by, for example, without limitation, Global System for Mobile communication (GSM) Association, the Internet Engineering Task Force (IETF), and the Worldwide Interoperability for Microwave Access (WiMAX) forum. In some embodiments, the exemplary network 1405 may implement one or more of a GSM architecture, a General Packet Radio Service (GPRS) architecture, a Universal Mobile Telecommunications System (UMTS) architecture, and an evolution of UMTS referred to as Long Term Evolution (LTE). In some embodiments, the exemplary network 1405 may include and implement, as an alternative or in conjunction with one or more of the above, a WiMAX architecture defined by the WiMAX forum. In some embodiments and, optionally, in combination of any embodiment described above or below, the exemplary network 1405 may also include, for instance, at least one of a local area network (LAN), a wide area network (WAN), the Internet, a virtual LAN (VLAN), an enterprise LAN, a layer 3 virtual private network (VPN), an enterprise IP network, or any combination thereof. In some embodiments and, optionally, in combination of any embodiment described above or below, at least one computer network communication over the exemplary network 1405 may be transmitted based at least in part on one of more communication modes such as but not limited to: NFC, RFID, Narrow Band Internet of Things (NBIOT), ZigBee, 3G, 4G, 5G, GSM, GPRS, WiFi, WiMax, CDMA, satellite and any combination thereof. In some embodiments, the exemplary network 1405 may also include mass storage, such as network attached storage (NAS), a storage area network (SAN), a content delivery network (CDN) or other forms of computer or machine readable media.
[0153] In some embodiments, the exemplary server 1406 or the exemplary server 1407 may be a web server (or a series of servers) running a network operating system, examples of which may include but are not limited to Microsoft Windows Server, Novell NetWare, or Linux. In some embodiments, the exemplary server 1406 or the exemplary server 1407 may be used for and/or provide cloud and/or network computing. Although not shown in
[0154] In some embodiments, one or more of the exemplary servers 1406 and 1407 may be specifically programmed to perform, in non-limiting example, as authentication servers, search servers, email servers, social networking services servers, SMS servers, IM servers, MMS servers, exchange servers, photo-sharing services servers, advertisement providing servers, financial/banking-related services servers, travel services servers, or any similarly suitable service-base servers for users of the member computing devices 1401-1404.
[0155] In some embodiments and, optionally, in combination of any embodiment described above or below, for example, one or more exemplary computing member devices 1402-1404, the exemplary server 1406, and/or the exemplary server 1407 may include a specifically programmed software module that may be configured to send, process, and receive information using a scripting language, a remote procedure call, an email, a tweet, Short Message Service (SMS), Multimedia Message Service (MMS), instant messaging (IM), internet relay chat (IRC), mIRC, Jabber, an application programming interface, Simple Object Access Protocol (SOAP) methods, Common Object Request Broker Architecture (CORBA), HTTP (Hypertext Transfer Protocol), REST (Representational State Transfer), or any combination thereof.
[0156]
[0157] In some embodiments, member computing devices 1502a through 1502n may also comprise a number of external or internal devices such as a mouse, a CD-ROM, DVD, a physical or virtual keyboard, a display, or other input or output devices. In some embodiments, examples of member computing devices 1502a through 1502n (e.g., clients) may be any type of processor-based platforms that are connected to a network 1506 such as, without limitation, personal computers, digital assistants, personal digital assistants, smart phones, pagers, digital tablets, laptop computers, Internet appliances, and other processor-based devices. In some embodiments, member computing devices 1502a through 1502n may be specifically programmed with one or more application programs in accordance with one or more principles/methodologies detailed herein. In some embodiments, member computing devices 1502a through 1502n may operate on any operating system capable of supporting a browser or browser-enabled application, such as Microsoft™, Windows™, and/or Linux. In some embodiments, member computing devices 1502a through 1502n shown may include, for example, personal computers executing a browser application program such as Microsoft Corporation's Internet Explorer™, Apple Computer, Inc.'s Safari™, Mozilla Firefox, and/or Opera. In some embodiments, through the member computing client devices 1502a through 1502n, users, 1512a through 1502n, may communicate over the exemplary network 1506 with each other and/or with other systems and/or devices coupled to the network 1506. As shown in
[0158] In some embodiments, at least one database of exemplary databases 1507 and 1515 may be any type of database, including a database managed by a database management system (DBMS). In some embodiments, an exemplary DBMS-managed database may be specifically programmed as an engine that controls organization, storage, management, and/or retrieval of data in the respective database. In some embodiments, the exemplary DBMS-managed database may be specifically programmed to provide the ability to query, backup and replicate, enforce rules, provide security, compute, perform change and access logging, and/or automate optimization. In some embodiments, the exemplary DBMS-managed database may be chosen from Oracle database, IBM DB2, Adaptive Server Enterprise, FileMaker, Microsoft Access, Microsoft SQL Server, MySQL, PostgreSQL, and a NoSQL implementation. In some embodiments, the exemplary DBMS-managed database may be specifically programmed to define each respective schema of each database in the exemplary DBMS, according to a particular database model of the present disclosure which may include a hierarchical model, network model, relational model, object model, or some other suitable organization that may result in one or more applicable data structures that may include fields, records, files, and/or objects. In some embodiments, the exemplary DBMS-managed database may be specifically programmed to include metadata about the data that is stored.
[0159] In some embodiments, the illustrative computer-based systems or platforms of the present disclosure may be specifically configured to operate in a cloud computing/architecture such as, but not limiting to: infrastructure a service (IaaS), platform as a service (PaaS), and/or software as a service (SaaS).
[0160] In some embodiments, exemplary inventive, specially programmed computing systems and platforms with associated devices are configured to operate in the distributed network environment, communicating with one another over one or more suitable data communication networks (e.g., the Internet, satellite, etc.) and utilizing one or more suitable data communication protocols/modes such as, without limitation, IPX/SPX, X.25, AX.25, AppleTalk™, TCP/IP (e.g., HTTP), near-field wireless communication (NFC), RFID, Narrow Band Internet of Things (NBIOT), 3G, 4G, 5G, GSM, GPRS, WiFi, WiMax, CDMA, satellite, ZigBee, and other suitable communication modes. In some embodiments, the NFC can represent a short-range wireless communications technology in which NFC-enabled devices are “swiped,” “bumped,” “tap” or otherwise moved in close proximity to communicate. In some embodiments, the NFC could include a set of short-range wireless technologies, typically requiring a distance of 10 cm or less.
[0161] The material disclosed herein may be implemented in software or firmware or a combination of them or as instructions stored on a machine-readable medium, which may be read and executed by one or more processors. A machine-readable medium may include any medium and/or mechanism for storing or transmitting information in a form readable by a machine (e.g., a computing device). For example, a machine-readable medium may include read only memory (ROM); random access memory (RAM); magnetic disk storage media; optical storage media; flash memory devices; electrical, optical, acoustical or other forms of propagated signals (e.g., carrier waves, infrared signals, digital signals, etc.), and others.
[0162] One or more aspects of at least one embodiment may be implemented by representative instructions stored on a machine-readable medium which represents various logic within the processor, which when read by a machine causes the machine to fabricate logic to perform the techniques described herein. Such representations, known as “IP cores” may be stored on a tangible, machine readable medium and supplied to various customers or manufacturing facilities to load into the fabrication machines that make the logic or processor. Of note, various embodiments described herein may, of course, be implemented using any appropriate hardware and/or computing software languages (e.g., C++, Objective-C, Swift, Java, JavaScript, Python, Perl, QT, etc.).
[0163] In some embodiments, one or more of illustrative computer-based systems or platforms of the present disclosure may include or be incorporated, partially or entirely into at least one personal computer (PC), laptop computer, ultra-laptop computer, tablet, touch pad, portable computer, handheld computer, palmtop computer, personal digital assistant (PDA), cellular telephone, combination cellular telephone/PDA, television, smart device (e.g., smart phone, smart tablet or smart television), mobile internet device (MID), messaging device, data communication device, and so forth.
[0164] In some embodiments, illustrative computer-based systems or platforms of the present disclosure may be configured to handle numerous concurrent users that may be, but is not limited to, at least 100 (e.g., but not limited to, 100-999), at least 1,000 (e.g., but not limited to, 1,000-9,999), at least 10,000 (e.g., but not limited to, 10,000-99,999), at least 100,000 (e.g., but not limited to, 100,000-999,999), at least 1,000,000 (e.g., but not limited to, 1,000,000-9,999,999), at least 10,000,000 (e.g., but not limited to, 10,000,000-99,999,999), at least 100,000,000 (e.g., but not limited to, 100,000,000-999,999,999), at least 1,000,000,000 (e.g., but not limited to, 1,000,000,000-999,999,999,999), and so on.
[0165] In some embodiments, illustrative computer-based systems or platforms of the present disclosure may be configured to output to distinct, specifically programmed graphical user interface implementations of the present disclosure (e.g., a desktop, a web app., etc.). In various implementations of the present disclosure, a final output may be displayed on a displaying screen which may be, without limitation, a screen of a computer, a screen of a mobile device, or the like. In various implementations, the display may be a holographic display. In various implementations, the display may be a transparent surface that may receive a visual projection. Such projections may convey various forms of information, images, or objects. For example, such projections may be a visual overlay for a mobile augmented reality (MAR) application.
[0166] In some embodiments, illustrative computer-based systems or platforms of the present disclosure may be configured to be utilized in various applications which may include, but not limited to, gaming, mobile-device games, video chats, video conferences, live video streaming, video streaming and/or augmented reality applications, mobile-device messenger applications, and others similarly suitable computer-device applications.
[0167] In some embodiments, the illustrative computer-based systems or platforms of the present disclosure may be configured to securely store and/or transmit data by utilizing one or more of encryption techniques (e.g., private/public key pair, Triple Data Encryption Standard (3DES), block cipher algorithms (e.g., IDEA, RC2, RCS, CAST and Skipjack), cryptographic hash algorithms (e.g., MD5, RIPEMD-160, RTRO, SHA-1, SHA-2, Tiger (TTH), WHIRLPOOL, RNGs).
[0168] The aforementioned examples are, of course, illustrative and not restrictive.
[0169] Publications cited throughout this document are hereby incorporated by reference in their entirety. While one or more embodiments of the present disclosure have been described, it is understood that these embodiments are illustrative only, and not restrictive, and that many modifications may become apparent to those of ordinary skill in the art, including that various embodiments of the inventive methodologies, the illustrative systems and platforms, and the illustrative devices described herein can be utilized in any combination with each other. Further still, the various steps may be carried out in any desired order (and any desired steps may be added and/or any desired steps may be eliminated).