METHOD FOR ADAPTIVE CONTROL OF A MEDICAL DEVICE USING BAYESIAN OPTIMIZATION
20200391037 ยท 2020-12-17
Inventors
- Logan Grado (Minneapolis, MN, US)
- Theoden Netoff (Minneapolis, MN)
- Andy Lamperski (Minneapolis, MN, US)
- Bryan Moore (Minneapolis, MN, US)
Cpc classification
A61M5/1723
HUMAN NECESSITIES
A61N1/365
HUMAN NECESSITIES
A61M2005/14208
HUMAN NECESSITIES
G16H20/10
PHYSICS
International classification
Abstract
Systems and methods for adaptively controlling an electrical stimulation device, such as a closed-loop stimulation device, based in part on a Bayesian optimization of the operational parameters of the device are described. An adaptive dual control of the stimulation device can be provided. In a first control loop parameters are extracted from signals recorded from the subject by the stimulation device, and in a second control loop a Bayesian optimization is implemented with a hardware processor and memory to compute updated operational parameters for the stimulation device. As noted, the stimulation device is an electrical stimulation device, and may be a closed-loop stimulation device. Such devices can be used for deep brain stimulation (DBS), cardiac resynchronization therapy (CRT), and other electrophysiological stimulation applications.
Claims
1. An adaptive controller for controlling a controllable medical device, comprising: an input that receives feedback data representative of a treatment response or effect in a subject; a processor in communication with the input and programmed to: receive the feedback data from the input and generate a posterior distribution therefrom; estimate an acquisition function from the posterior distribution; and generate updated control parameter settings based on the acquisition function; a memory in communication with the input and the processor, wherein the memory stores instructions for generating updated control parameter settings, the feedback data received from the input, and the updated control parameter settings generated by the processor; and an output that communicates the updated control parameter settings to a controllable medical device.
2. The adaptive controller as recited in claim 1, wherein the feedback data received from the input comprise at least one of behavior metrics or user preferences, and wherein the processor is programmed to generate a response surface based on the feedback data.
3. The adaptive controller as recited in claim 2, wherein the processor is programmed to generate the response surface using a probit function.
4. The adaptive controller as recited in claim 3, wherein the feedback data comprise user preferences between two different control parameter settings.
5. The adaptive controller as recited in claim 1, wherein the feedback data received from the input comprise at least one of physiological data or behavioral metrics.
6. The adaptive controller as recited in claim 5, wherein the physiological data comprises at least one of , cardiac signals, or chemical signals.
7. The adaptive controller as recited in claim 6, wherein the physiological data comprises chemical signals including at least one of insulin levels or glucose levels.
8. The adaptive controller as recited in claim 1, wherein the feedback data received from the input comprise user preferences.
9. The adaptive controller as recited in claim 8, wherein the user preferences comprise user preference responses to a questionnaire.
10. The adaptive controller as recited in claim 1, wherein the controllable medical device is at least one of an electrical stimulation device, a drug pump, a hearing aid, or a cochlear implant.
11. A method for controlling a closed-loop stimulation system having one or more electrodes configured to apply stimulating electrical signals and to receive electrophysiological signals, the method comprising: (a) measuring, with the one or more electrodes of the closed-loop stimulation system, electrophysiological signals from a subject; (b) estimating a biomarker from the electrophysiological signals; (c) determining electrical stimulation settings based on a Bayesian optimization to which the biomarker is input; and (d) controlling the closed-loop stimulation system based on the determined electrical stimulation settings.
12. The method as recited in claim 11, further comprising receiving user preference data corresponding to a preference of the subject between two different electrical stimulation settings and updating the determined electrical stimulation settings based on the user preference data.
13. The method as recited in claim 12, wherein updating the determined electrical stimulation settings includes generating a probit function from the user preference data and updating the electrical stimulation settings using the probit function.
14. The method as recited in claim 10, wherein the biomarker comprises at least one of amplitude data or phase data.
15. The method as recited in claim 10, wherein the biomarker is at least one of a phase-amplitude coupling or an evoked compound action potential.
16. A closed-loop stimulation system, comprising: one or more electrodes configured to apply stimulating electrical signals and to receive electrophysiological signals; and a controller configured to: measure electrophysiological signals received at the one or more electrodes; estimate at least one of amplitude data or phase data from the measured electrophysiological signals; determine electrical stimulation settings based on a Bayesian optimization to which the at least one of amplitude data or phase data are input; and apply electrical stimulation based on the determined electrical stimulation settings using the one or more electrodes.
17. The closed-loop stimulation system as recited in claim 16, further comprising an input in communication with the controller, and wherein the controller is programmed to: receive from the input user preference data corresponding to a preference a user between two different electrical stimulation settings; and update the determined electrical stimulation settings based on the user preference data.
18. The closed-loop stimulation system as recited in claim 17, wherein the controller is programmed to update the determined electrical stimulation settings by generating a probit function from the user preference data and updating the electrical stimulation settings using the probit function.
19. The closed-loop stimulation system as recited in claim 16, wherein the controller is programmed to measure the electrophysiological signals and estimate the at least one of amplitude data or phase data in a parameterized feedback control loop, and to determine the electrical stimulation settings in a parameter adjustment loop that implements the Bayesian optimization.
20. The closed-loop stimulation system as recited in claim 19, wherein the controller is programmed to input a model based on a local field potential to the parameterized feedback control loop and to estimate the at least one of amplitude data or phase data using the model based on the local field potential.
21. (canceled)
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0014]
[0015]
[0016]
[0017]
[0018]
[0019]
[0020]
[0021]
[0022]
[0023]
[0024]
[0025]
DETAILED DESCRIPTION
[0026] Described here are systems and methods for adaptively controlling a medical device based in part on a Bayesian optimization of the control parameters of the device. The Bayesian optimization provides automatic tuning of the control parameters of the medical device based on feedback data, such as user response data, to achieve a user-specific therapy or effect. Examples of medical devices that can be adaptively controlled in the manner include electrical stimulation devices, such as peripheral nerve stimulators, central nervous system stimulators, cardiac pacemakers, and cardiac resynchronization therapy (CRT) devices; drug pumps; hearing aids; cochlear implants; and other such devices with controllable parameters. Examples of feedback data that can be input to the Bayesian optimization include physiological data, such as neural signals, cardiac signals, and chemical signals (e.g., insulin levels, glucose levels). The feedback data may also include behavioral metrics and user preferences (e.g., preferences based on a questionnaire or between two presented options). To implement user preferences, the user preferences can be made into a response surface using a probit function or other suitable statistical or mathematical model.
[0027] As one non-limiting example, an adaptive control of the medical device can be provided. Feedback data can be measured or otherwise provided to a controller, and a Bayesian optimization is implemented with a hardware processor and memory to compute updated control parameters for the medical device based in part on the feedback data.
[0028] An adaptive controller for use in medical devices is described in the present disclosure. The adaptive controller is implemented using a hardware processor and memory and generally includes a feedback driven control parameter optimization, as shown in
[0029] The feedback driven control parameter optimization of the adaptive controller employs Bayesian optimization to intelligently sample the parameter space and select the optimal set of parameters. In some implementations, the optimization generally operates on a timescale that is on the order of a few seconds, but in some other implementations may operate on longer timescales (e.g., days to weeks). After selecting a new control parameter set, the optimization can wait a delay time in order to allow the subject to settle into a steady state. As an example, the delay time can be on the order of 10 seconds. In some embodiments, the optimization can then estimate one or more measurement parameters from the subject. As one non-limiting example, in DBS applications the optimization can estimate the amplitude of the beta oscillations, or other neuronal signals, over a measurement time. For instance, the amplitude can be estimated by keeping a running average of the oscillations amplitude. The measurement time can be on the order of 10 seconds as well.
[0030] Bayesian optimization is well-suited for selecting the optimal parameters of a controllable medical device, such as an electrical stimulation device, drug pump, or hearing aid. For instance, Bayesian optimization offers advantages because direct access to the objective function is often not available for such devices and, thus, noisy observations are made instead; the objective function is expensive to evaluate; there is no access to derivatives; and the optimization problem is not necessarily convex.
[0031] In the case of electrical stimulation, the objective function is the user's response to a set of feedback stimulator parameters. It may take seconds, minutes, hours, or even days to obtain a good (likely noisy) measure of the effect of a parameter set. Additionally, in DBS and other electrical stimulation applications there is typically no access to derivatives, and so gradient descent methods cannot be used, and the problem at hand cannot be assumed to be convex, so a global exploration is preferred.
[0032] Bayesian optimization address each of these challenges. The Bayesian optimization algorithm generally includes treating the unknown objective function as a random function over which a prior is placed. The prior generally represents the believed behavior of the unknown objective function. This prior is updated based in part on feedback data that are representative of evaluations of the unknown objective function to form a posterior distribution over the unknown objective function. The posterior distribution is then used to generate an acquisition function, which models the utility of sampling across the space. The acquisition function is used to determine the next sample points. As one non-limiting example, the posterior distribution can be a Gaussian process regressor (GPR) that estimates the objective function with a response surface, and the acquisition function predicts the utility of sampling by incorporating the mean and variance of the GPR.
[0033] A Gaussian process is an extension of the multivariate Gaussian distribution to an infinite-dimension stochastic process. A Gaussian process can be thought of as a distribution over functions, specified by a mean function, m, and covariance function (also known as the kernel), k:
f(x)GP(m(x),k(x,x)) (1).
[0034] The prior mean is often assumed to be the zero function m(x)=0, or the mean of the training data, m(x)=
[0035] where .sup.2 is the variance and is a length scale parameter. In the Bayesian optimization task, the GP is fit to previous observations, D.sub.1:n={x.sub.i:n,f.sub.i:n}, in order to obtain the posterior for any point x.sub.n+1. The predictive distribution can be derived as,
P(f.sub.n+1|D.sub.1:n,x.sub.n+1)=N(.sub.n(x.sub.n+1), .sup.2(x.sub.n+1)) (3);
[0036] where
.sub.n(x.sub.n+1)=k.sup.TK.sup.1f.sub.1:n (4);
.sub.n.sup.2(x.sub.n+1)=k(x.sub.t+1, x.sub.t+1)k.sup.TK.sup.1k (5).
[0037] Thus, given a set of previous observations, D, the mean and variance of any point x.sub.n+1 can be predicted, which can be used to determine which point should be sampled next using the acquisition function.
[0038] The acquisition function, u(. . . ), serves to guide the search to the optimum by modeling the expected utility of sampling at any point, x.sub.n+1. An acquisition function such as the Gaussian process lower confidence bound (GP-LCB) function achieves low values in regions where either the prediction is low, the uncertainty is high, or both.
GPLCB(x)=(x)(x) (6);
[0039] where 0. Other acquisition functions can also be used. The Bayesian optimization algorithm thus selects the next evaluation point, x.sub.n+1, by minimizing the acquisition function, such as by sampling at argmax.sub.x u(x|D) .
[0040] The acquisition function also governs the trade-off between exploration and exploitation. In GP-LCB, the parameter determines the exploration-exploitation trade-off, where high values of encourage exploration and low values of encourage exploitation.
[0041] With,
[0042] it can be shown that this method is no regret with high probability.
[0043] In general, feedback data provided to the adaptive controller can be used to generate the response surface estimated by the GPR. As noted above, the feedback data can include physiological feedback data measured from the user, which may include electrophysiological signals, chemical signals, and so on. Such physiological feedback data can be measured by the controllable medical device, or can be provided to the controllable medical device via an input. For instance, the physiological feedback data could be measured by a different medical device and transmitted over a wired or wireless connection to the adaptive controller used for the controllable medical device.
[0044] As mentioned above, in some embodiments, the feedback data can include behavioral or user preference data that can be quantified and provided as an additional input to the Bayesian optimization. For instance, the user can qualitatively describe their interpretation of whether a current batch of settings for the controllable medical device are effective or not. As one example, the user preference data can be recorded as responses to a questionnaire. This qualitative feedback can be used to build out a Gaussian process to be used in the Bayesian optimization. As one example, a probit function may be used to build the GPR. A probit function may be, for instance, a regression-based model where the dependent variable can take only two values. Within the context of the systems and methods described in the present disclosure, the probit function may be built using dependent variables that are associated with user preferences on control parameter settings. For instance, the user preferences could be a user's preferred choice between two different control parameter settings. The preference could be a forced choice (e.g., better/worse), or better/same/worse, or much better/better/worse/much worse, or so on.
[0045] The user's specific trade-off between exploration and exploitation can also be incorporated into the Bayesian optimization. For instance, some users may be more willing to try more therapy and thus that user's tolerance for exploitation versus exploration may be different from other users.
[0046] Implementing user preference to design the control parameter settings for the controllable medical device effectively amounts to a changing cost function over time. The methods described in the present disclosure are capable of optimizing this changing cost function to select the settings that are contained within the current set of user preferences.
[0047] In an example study, the adaptive controller described in the present disclosure was tested for controlling a neurostimulator used in a DBS application. In this example, an adaptive dual controller (ADC) was implemented using a hardware processor and memory. The ADC generally included an inner loop and an outer loop, as shown in the example configurations in
[0048] In this example, the inner loop of the ADC included a closed-loop feedback stimulator. The inner loop received information from a model that was based on a local field potential (LFP) of the GPi, as shown in
[0049] DBS was added to the model by representing a DBS pulse as a direct current injection into the target structure. Integrating Ohm's law for capacitors results in the following:
[0050] DBS was therefore modeled by directly adding the total charge of a monophasic DBS pulse (divided by the membrane capacitance) to the first derivative of the voltage.
[0051] Referring again to
[0052] As shown in the example in
[0053] To implement phase and amplitude feedback stimulation, a real-time method of accurately estimating both phase and amplitude of an oscillation can be used. As one example, phasic stimulation has been accomplished by band-pass filtering the signal and then using the time since the preceding zero crossing. Amplitude-based stimulation has been achieved by rectifying and smoothing the band-passed signal for 400 ms. A visualization of amplitude feedback stimulation, phase feedback stimulation, and combined phased/amplitude feedback stimulation is shown in
[0054] As another example, a sliding Fourier transform technique referred to as the Sliding Windowed Infinite Fourier Transform (SWIFT) and described in co-pending U.S. Provisional Patent Appin. No. 62/520,265, which is herein incorporated by reference in its entirety, can be used. Unlike other methods of phase/amplitude estimation, the SWIFT technique directly and efficiently calculates the windowed Fourier transform of the signal, centered on =2f/f.sub.s and windowed with an infinite length, causal exponential window. In a variation referred to as SWIFT, the function window, which is the difference between two exponentials with different time constants, is used to achieve improved frequency resolution.
[0055] The SWIFT algorithm has two parameters that control its behavior: the center frequency, ; and the time constant, . For the SWIFT algorithm two time constants are used: .sub.slow and .sub.fast. The center frequency, , can be set to match frequency information of the model. For instance, the center frequency can be set to match the center frequency of a beta peak in the model. The time constant (.sub.slow in SWIFT) controls the time-frequency tradeoff: a shorter time constant leads to higher temporal resolution, but lower frequency resolution (i.e., wider frequency response). As one example of tuning this tradeoff, the width of the SWIFT algorithm's frequency response can be matched to the width of the model's beta peak at 6 dB (or 50% power reduction).
[0056]
[0057] Referring now to
[0058] In some embodiments, the input 1016 is capable of sensing feedback data from the user. As one example, the feedback data can be electrophysiological activity, and the input 1016 can be one or more electrodes. As another example, the feedback data can be chemical signal data, such as measured levels of chemicals. In such instances, the input 1016 can include a suitable sensor for measuring the chemical signal data. As noted above, such a sensor could be a part of the controllable medical device, or could be a separate sensor that is in communication with the controller 1010 via the input 1016, whether through a wired or wireless connection. Such chemical data could also be measured through other means, such as via a blood sample taken from the user, and transmitted to the controller 1010 via the input 1016. The input 1016 can thus more generally include a wired or wireless connector for receiving feedback data, which as noted above may also include behavioral or user preference data. In these latter examples, the feedback data can include a response surface generated from the behavioral or user preference data, such as a probit function, that is transmitted to the controller 1010 via the input 1016.
[0059] The processor 1012 includes at least one hardware processor to execute instructions embedded in or otherwise stored on the memory 1014 to implement the methods described in the present disclosure. The memory can also store measured feedback data for processing, as well as settings to be provided to the processor 1012 for generating control signals to be provided to a controllable medical device via the output 1018. As described above, these settings can be stored and also updated by the adaptive control implemented by the controller 1010.
[0060] The output 1018 communicates control signal to a controllable medical device. As one example, where the controllable medical device is an electrical stimulation device, the control signals provided to the output 1018 can control one or more electrodes to operate under control of the controller 1010 to sense electrophysiological activity in a subject and to deliver electrical stimulations to the subject in response thereto. Sensing circuitry in the controller 1010 can detect and processes electrophysiological activity sensed by the one or more one electrodes via the input 1016 to determine the optimized stimulation settings (e.g., phasic burst stimulation settings) based on the methods and algorithms described above. The optimized settings are provided as instructions to a pulse generator in the electrical stimulation device via the output 1018, which in response to the instructions provides an electrical signal to the one or more electrodes to deliver the electrical stimulations to the subject.
[0061] The controller 1010 can also include a transceiver 1020 and associated circuitry for communicating with a programmer or other external or internal device. As one example, the transceiver 1020 can include a telemetry coil. In some embodiments, the transceiver 1020 can be a part of the input 1016.
[0062] In operation, the controller 1010 receives feedback data from the subject via the input 1016. These feedback data are provided to the processor 1012 where they are processed. For example, the processor 1012 analyzes the feedback data and generates an appropriate response surface, or otherwise generates a GPR to be used for the Bayesian optimization to update the control parameter settings for the controllable medical device.
[0063] In one non-limiting example, the processor 1012 can process electrophysiological signal data to estimate biomarkers such as amplitude data, phase data, or both, from the measured data. In these instances, the processor 1012 can analyze the electrophysiological signals using a SWIFT or SWIFT algorithm to extract the relevant amplitude and phase data. In other example, other biomarkers can be extracted or estimated from the electrophysiological signals, such as phase-amplitude coupling, evoked compound action potentials, or other parameters or characteristics that can be extracted or estimated from the electrophysiological signals. The extracted biomakrers are then input to a Bayesian optimization algorithm implemented by a hardware processor and memory 1014 of the controller 1012 to determine optimized settings for the delivery of electrical stimulation to the subject, as described above in detail. The optimized settings are provided to the electrical stimulation device via the output 1018 to control one or more electrodes to generate electrical stimulation that will achieve the desired effect in the subject, such as preventing an anticipated pathological electrophysiological event.
[0064] The present disclosure has described one or more preferred embodiments, and it should be appreciated that many equivalents, alternatives, variations, and modifications, aside from those expressly stated, are possible and within the scope of the invention.