Feedback system with stimulation of sensory nerves for improved balance control

Abstract

A feedback device for measuring pressure related information, and for providing that information in a way that is useful to the wearer of the device. At least one sensor detects pressure information and transmits at least one pressure information signal to a signal processing subsystem. The signal processing subsystem converts the received pressure information signal into at least one stimulation control signal. The signal processing subsystem then transmits the stimulation control signal to at least one stimulator, which provides stimulation to a wearer of the device reflecting the stimulation control signal received from the signal processing subsystem.

Claims

1. (canceled)

2. A system for improving balance control, the system comprising: a. at least one sensor associated with a prosthetic lower leg of the user, wherein the at least one sensor is configured to generate user balance information signals and transmit the user balance information signals, wherein the at least one sensor is configured to sense an angle between the prosthetic lower leg and an upper leg of the user; b. a signal processing subsystem, the subsystem configured to receive the user balance information signals and generate balance control signals comprising temporal and spatial information reflecting the angle between the prosthetic lower leg and an upper leg of the user; and c. at least one stimulator configured to be responsive to the balance control signals to provide stimulation based on the balance control signals, thereby providing feedback to the user.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The invention will be more fully understood by reference to the following detailed description of the invention in conjunction with the drawings, of which:

[0010] FIG. 1 shows the structure and operation of an illustrative embodiment of the disclosed system;

[0011] FIG. 2 shows a circuit diagram of a portion of an illustrative embodiment of the disclosed system;

[0012] FIG. 3 is a flow chart showing steps performed during operation of an illustrative embodiment of the disclosed system;

[0013] FIG. 4 shows an illustrative embodiment of the disclosed system using a single level stimulation array;

[0014] FIG. 5 shows an illustrative embodiment of the disclosed system using a multi-level stimulation array;

[0015] FIG. 6 shows an illustrative embodiment of the disclosed system using a multi-level stimulation array in combination with a bi-axial goniometer;

[0016] FIG. 7 illustrates an example of force magnitude encoding in a multilevel stimulator embodiment; and

[0017] FIGS. 8-12 show the mapping of balance information to stimulation points in an illustrative embodiment.

DETAILED DESCRIPTION

[0018] All disclosures of provisional patent application Ser. No. 60/372,148 entitled SENSORY PROSTHETIC FOR IMPROVED BALANCE CONTROL, and filed Apr. 12, 2002, are hereby incorporated by reference herein.

[0019] FIG. 1 shows the structure and operation of an illustrative embodiment of the disclosed system. As shown in FIG. 1, sensors 10 detect balance information that may represent foot force/pressure, such as normal and/or shear pressure, ankle angle, such as sagittal and/or coronal ankle angle, or knee angle, such as sagittal knee angle. The balance information detected by the sensors 10 is passed to a signal processing subsystem 12, which converts the balance information received from the sensors 10 into a stimulation control signal that is passed to the stimulators 14. The stimulators 14 operate in response to stimulation control signal, producing a stimuli to a user of the disclosed system representing the balance information detected by the sensors 10. The stimulators 14 may be of various types, including external and/or internal stimulators, and provide various types of stimuli, including vibratory, visual, auditory, and/or electrical stimuli. The stimulators 14 may be located on various parts of the body, including one or both legs, head, arms or trunk of the user.

[0020] FIG. 2 shows a circuit diagram of a portion of an illustrative embodiment of the disclosed system. As shown in FIG. 2, a number of foot pressure sensors 16 transmit at least one balance information signal to one or more amplifiers 20 within a signal processing subsystem 18. The amplifiers 20 may, for example, include one or more LM324 analog amplifiers supplied by National Semiconductor Corporation. The amplified signals output from the amplifiers 20 are passed to a signal processor 22. In the example of FIG. 2, the signal processor includes analog to digital processing functionality, as well as program code storage for code executable on the signal processor 22. The signal processor 22 may, for example, consist of a PIC 16F877 microcontroller, supplied by Microchip Technology, Inc. The signal processor 22 passes a digital stimulation control signal to one or more digital to analog converters 24, which convert the digital stimulation control signal to an analog stimulation control signal that is passed to one or more vibratory feedback devices 26. The vibratory feedback devices 26 provide a vibratory stimulus to a user wearing the vibratory feedback devices 26, such that the vibratory stimulus encodes and/or represents the balance information provided from the foot pressure sensors 16 to the signal processing subsystem 18. The digital to analog converters 24 may, for example, include one or more TLC7226IN, 8 bit, 4 channel digital to analog converters. The vibratory feedback devices may, for example, include one or more analog amplifiers, such as AD8534 analog amplifiers, supplied by Analog Devices, Inc., as well as a number of vibrators.

[0021] FIG. 3 is a flow chart showing steps performed during operation of an illustrative embodiment of the disclosed system. The steps shown in FIG. 3 may, for example be performed at least in part by the signal processing subsystem 18 shown in FIG. 2, for example under control of an assembly language program stored within and executed on the signal processor 22 shown in FIG. 2. As shown in FIG. 3, at step 28, the disclosed system reads balance information signals output from one or more sensors, and stores, the values for subsequent processing. The disclosed system then operates to convert the stored sensor values into stimulation control values at step 30. The stimulation control values are then output at step 32.

[0022] An example of the processing performed by the signal processing subsystem to convert sensor values into stimulation control values is now provided. A formula describing the steps performed by the signal processing subsystem to convert sensor values into stimulation control values is as follows:

[00001]$\begin{array}{cc}{}_l\left(t\right)=\underset{n=1}{\overset{N}{.Math.}}.Math.f_{n.Math.l}\left(t\right).Math.{}_{n.Math.l},R_l\left(t\right)=\underset{n=1}{\overset{N}{.Math.}}.Math.f_{n.Math.l}\left(t\right).Math.r_{n.Math.l},.Math.F_l\left(t\right)=\frac{1}{W}.Math.\underset{n=1}{\overset{N}{.Math.}}.Math.f_{n.Math.l}\left(t\right),& \left(\mathrm{Eq}..Math.1\right)\end{array}$

[0023] where

[0024] f.sub.nl=normal force, approximately perpendicular to the plane of the foot sole, measured by sensor array element n under leg 1,

[0025] .sub.nl=angular position of sensor array element n under leg 1,

[0026] r.sub.nl=radial position of sensor array element n under leg 1,

[0027] N=total number of sensor array elements under leg 1,

[0028] W=total body weight of the wearer,

[0029] .sub.l=angular position of center-of-pressure under leg 1,

[0030] R.sub.l=radial position of center-of-pressure under leg 1,

[0031] F.sub.l=portion of body weight supported by leg 1, and,

[0032] t=a variable representing discrete time

[00002]$\begin{array}{cc}{Q}_{.Math..Math.l}\left(t\right)=P_{}.Math.{}_l\left(t\right)+D_{}.Math.\frac{d.Math..Math.{}_l\left(t\right)}{\mathrm{dt}}+I_{}.Math.{.Math.}_{x=t-u_{}}^t.Math.{}_l\left(z\right)+P_{}.Math.{}_l\left(t\right)+D_{}.Math.\frac{d.Math.{}_l\left(t\right)}{\mathrm{dt}}+I_{}.Math.{.Math.}_{x=t-u_{}}^t.Math.{}_l.Math.\left(z\right),& \left(\mathrm{Eq}..Math.2\right)\\ Q_{R.Math.l}\left(t\right)=P_R.Math.R_l\left(t\right)+D_R.Math.\frac{d.Math.R_l\left(t\right)}{\mathrm{dt}}+I_R.Math.{.Math.}_{x=t-u_R}^t.Math.R_l\left(z\right)+P_{}.Math.{}_l\left(t\right)+D_{}.Math.\frac{d.Math.{}_l\left(t\right)}{\mathrm{dt}}+I_{}.Math.{.Math.}_{x=t-u_{}}^t.Math.{}_l\left(z\right)+P_{}.Math.{}_l\left(t\right)+D_{}.Math.\frac{d.Math.{}_l\left(t\right)}{\mathrm{dt}}+I_{}.Math.{.Math.}_{x=t-u_{}}^t.Math.{}_l\left(z\right),& \left(\mathrm{Eq}..Math.3\right)\\ .Math.Q_{F.Math.l}\left(t\right)=P_F.Math.F_l\left(t\right)+D_F.Math.\frac{d.Math.F_l\left(t\right)}{\mathrm{dt}}+I_F.Math.{.Math.}_{z=t-u_F}^t.Math.F_l\left(z\right),& \left(\mathrm{Eq}..Math.4\right)\end{array}$

[0033] where

[0034] .sub.l=the magnitude of the angle existing between the approximate longitudinal axis of foot 1 and the projection of the approximate longitudinal axis of the ipsilateral shank onto a plane substantially parallel to the sole of foot 1,

[0035] .sub.l=the magnitude of the angle existing between the approximate longitudinal axis of shank 1 and a plane substantially parallel to the sole of foot 1,

[0036] .sub.l=the magnitude of the angle existing between a line substantially parallel to the longitudinal axis of shank 1 and a line substantially parallel to the longitudinal axis of the ipsilateral femur,

[0037] z=a variable representing discrete time,

[0038] d/dt=the operator indicating a discrete-time estimation of the first derivative,

[0039] =the summation operator,

[0040] P.sub., D.sub., u.sub., P.sub., D.sub., I.sub., u.sub., P.sub.R, D.sub.R, I.sub.R, u.sub.R, P.sub., D.sub., I.sub., u.sub., P.sub., D.sub., I.sub., u.sub., P.sub.F, D.sub.F, I.sub.F, u.sub.F are time-invariant coefficients.

[0041] An example of an algorithm for the processing performed by software or firmware stored in and executed by a microcontroller in the signal processing subsystem is as follows:

TABLE-US-00001 ALGORITHM: for v = 1 to V for p = 1 to P if {(A.sub.V Q.sup.J1(t) < A.sub.V + .sub.1) (B.sub.p Q.sub.1(t) B.sub.p+)} = true, then S.sub.[v, p]1 (t) = h(Q.sub.K1 (t),Q.sub.1 (t)) else S.sub.[v, p]1 (t) = 0 end end
where

[0042] subscripts J and K correspond to subscripts F and R in Equations 3 & 4: (J=F and K=R) or (J=R and K=F),

[0043] [v,p] denotes the spatial coordinates of an individual stimulator within the stimulation array, v being an integer denoting the vertical index and p being an integer denoting the horizontal index of the stimulator at location [v,p],

[0044] denotes the logical AND operation,

[0045] V=total number of horizontal rows in the stimulation array,

[0046] P=total number of vertical columns in the stimulation array,

[0047] A.sub.j=the threshold of Q.sub.J1(t) for activation of a stimulator in the jth row of the stimulator array,

[0048] B.sub.j=the minimum value of Q.sub.1(t) for activation of a stimulator in the jth column of the stimulator array,

[0049] B.sub.J+=the maximum value of Q.sub.1(t) for activation of a stimulator in the jth column of the stimulator array,

[0050] h(Q.sub.K1(t), Q.sub.1(t)) is a piecewise-continuous function of Q.sub.K1(t) and Q.sub.1(t), and

[0051] S.sub.[v,p]1 is the magnitude of activation of stimulator [v,p] within the stimulator array attached to leg 1. S.sub.[v,p]1 may denote the amplitude or frequency of stimulation produced by stimulator [v,p].

[0052] In a preferred embodiment, with regard to the stimulators, 3 vibrator rows are used to encode load, 4 vibrator columns are used to encode polar center-of-pressure (COP) orientation, and vibrator activation voltage (proportional to frequency) is used to encode polar radius of COP. Vibrator frequency is normalized over its active range to an elliptical ring with ranges of 1 to 8 mm in the mediolateral direction and 2 to 20 mm in the anteroposterior direction. These ranges were selected based upon the typical range of human movement during quiet stance and would be increased in order to optimize the device for dynamic activities such as walking. The position of force transducers under the foot soles corresponds to a foot corresponding to a U.S. Men's size 9 shoe. Illustrative parameter values for an example of the preferred embodiment are therefore provided for purposes of explanation as follows:

[0053] N=7,

[0054] r.sub.1=75, r.sub.2=80, r.sub.3=90, r.sub.4=120, r.sub.5=70, r.sub.6=80, r.sub.7=115, (units in mm)

[0055] .sub.1=0.70, .sub.2=1.05, .sub.3=1.39, .sub.4=1.74, .sub.5=2.09, .sub.6=4.36, .sub.7=4.89, (units in radians)

[0056] V=3, P=4,

[0057] JF, KR,

[0058] P.sub.=P.sub.R=P.sub.F=1; P.sub.=P.sub.=P.sub.=0; D.sub.=D.sub.R=D.sub.F=D.sub.=D.sub.=D.sub.=0, I.sub.=I.sub.R=I.sub.F=I.sub.=I.sub.=I.sub.=0,

[0059] A.sub.1=0.25 W, A.sub.2=0.5 W, A.sub.3=0.75 W,

[0060] B.sub.1=0.25 rad, B.sub.1+=0.25 rad, B.sub.2=0.25 rad, B.sub.2+=0.75 rad,

[0061] B.sub.3=0.75 rad, B.sub.3+=1.25 rad, B.sub.4=1.25 rad, B.sub.4+=1.75 rad

[00003]$h\left(Q_{R.Math.l}\left(t\right)-R_o\right).Math.\frac{V_{\max }-V_{\min }}{1.Math.7}.Math..Math.\mathrm{for}.Math..Math.Q_{R.Math.l}\left(t\right)>R_o$$h0.Math..Math.\mathrm{for}.Math..Math.Q_{R.Math.l}\left(t\right)R_o$$\mathrm{where}$$R_o=\sqrt{\frac{1}{{\cos }^2.Math.Q_{.Math..Math.l}\left(t\right)+\frac{{\sin }^2.Math.Q_{.Math..Math.l}\left(t\right)}{9}}}$$V_{\max }=5.Math..Math.\mathrm{volts}.Math..Math.\mathrm{and}.Math..Math.V_{\min }=2.5.Math..Math.\mathrm{Volts}$

[0062] In an alternative embodiment, Equation 1 above also includes calculation of the total shear force impinging on the sole of foot 1, and Equation 2 and/or 3 includes terms pertaining to the total shear force, the derivative of the total shear force, and the integral summation of the total shear force impinging on foot 1. Thus the representation of balance information by the stimulator array reflects information regarding the shear forces impinging on the soles of the user's feet.

[0063] Neuropathic patients often encounter sustained elevated pressures under parts of their feet that result in skin damage and the development of an ulcer. Consistent with the algorithm above, the disclosed system may operate such that signal processing subsystem transmits signals to the stimulators reflecting the time histories of pressures or forces impinging on individual transducers within a sensor array. The resulting stimulation to a wearer of such an embodiment indicates when and where the forces or pressures impinging on an individual transducer within the sensor array have exceeded a predetermined instantaneous or time integral threshold.

[0064] While in the above algorithm the terms foot, shank and femur are used to describe body parts of a wearer of an embodiment of the disclosed system, those skilled in the art will recognize that other terms may be used in the alternative to describe the same parts. For example, alternative, corresponding terms to those used in the algorithm above include lower leg for shank, and upper leg for femur.

[0065] FIG. 4 shows an illustrative embodiment of the disclosed system using a single level stimulation array 40. As shown in FIG. 4, the illustrative embodiment includes a vibrotactile feedback array 40, a microprocessor controller subsystem 42, and a foot sole pressure sensor array 44. In the illustrative embodiment of FIG. 4, vibrotactile or electrotactile cutaneous feedback provided through the stimulation array 40 encodes position of foot Center-Of-Pressure and/or weight distribution by modulating one or more of the following: stimulus frequency, stimulus amplitude, location of stimulus or number of active stimulators. This stimulation array 40 is located adjacent to the skin of the leg or thigh. The location of active stimulator(s) on the skin in the transverse plane directly reflects the location of the foot Center-of-Pressure in the transverse plane.

[0066] The microprocessor controller subsystem 42 operates to convert electrical or mechanical signal(s) from the sensor array 44 into signal(s) which control the activity of elements within the feedback array 40. The microprocessor controller subsystem 42 may be implemented as a discrete system component or be imbedded within the other components. The microprocessor controller subsystem 42 estimates the position of the Center-of-Pressure (COP) under the foot and/or the fraction of the body weight supported by the foot. These estimates are then used to produce an appropriate output signal to the feedback array 40. A dead-zone may be implemented such that Center-of-Pressure position within a certain range and/or foot load below a certain threshold may produce no output to the feedback elements.

[0067] FIG. 5 shows an illustrative embodiment of the disclosed system using a multi-level stimulation array. The illustrative embodiment of FIG. 5 includes an array of force sensing resistors (FSRs) 50 placeable under the soles of one or more feet of a user. Balance information from the array of force sensing resistors 50 is passed to a microprocessor data acquisition and processing subsystem 54. The subsystem 54 operates to convert the balance information it receives into stimulation control signals sent to a vibro-tactile array 52 located on one or more legs of the user. The vibro-tactile array 52 includes three vibrator levels 56, 58, and 59. A top view 60 of the vibrotactile array on the leg or legs of the user illustrates that each of the levels 56, 58 and 59 include four vibrators located in the front, back, and both sides of the leg or legs.

[0068] FIG. 6 shows an illustrative embodiment of the disclosed system using a multi-level stimulation array in combination with a bi-axial goniometer. As shown in FIG. 6, a force pressure sensor array 62 and a bi-axial goniometer 64 operate to provide balance information to a microprocessor data acquisition and processing subsystem 66. The bi-axial goniometer 64 provides information regarding detected ankle angle with regard to the angle of the user's foot to the corresponding lower leg. The stimulation array 68 on the user leg is shown as a multilevel vibrator array, and the top view 70 of the stimulation array 68 illustrates that each level of vibrators in the stimulation array 68 includes four vibrators, mounted in the front, back, and both sides of the leg.

[0069] FIG. 7 illustrates an example of force magnitude encoding in a multilevel stimulator embodiment. FIG. 7 illustrates that a first, relatively low level force 72, may be encoded as a vibrational stimulus 74 on a lowest vibrator layer (or level) of the stimulation array. Similarly, a relatively high level force 80, may be encoded as a vibrational stimulus 82 on a highest vibrator layer (or level) of the stimulation array. Along these same lines, a force 76 between the forces 72 and 80 may be encoded as a vibrational stimulus 78 within a middle vibrator layer of the stimulation array. Thus FIG. 7 shows how a magnitude represented in the balance information signal, such as a ground reaction magnitude, can be communicated to the user through levels within a stimulator array attached or proximate to the user.

[0070] FIGS. 8-12 show the mapping of balance information to stimulation points in an illustrative embodiment, and thus illustrate mapping of COP information onto a body part, in this case, a leg of the user. FIG. 8 shows a dead zone 86, representing a predetermined physical area under a user's foot. In an illustrative embodiment of the disclosed system, if a center of pressure is estimated to be located within the dead zone, no stimulus is provided to the user. In such a case, the user is significantly balanced on the foot in question. As shown in FIG. 8, in the event that a center of pressure 88 is determined to be within the predetermined dead zone, no vibrational stimulus is provided in any of the vibrators 90, 92, 94 or 96, shown within a single level of stimulators mounted on a user's leg in a plane parallel to an array of sensors mounted under the user's foot or feet.

[0071] FIG. 9 illustrates operation of an embodiment of the disclosed system in the event that a center of pressure 98 is estimated to be located towards the back of the user's foot. Under such circumstances, a vibrator 96 located towards the back of the user's leg is shown providing a vibrational stimulus to the user.

[0072] FIG. 10 illustrates operation of an embodiment of the disclosed system in the event that a center of pressure 100 is estimated to be located towards the front of a user's foot. Under such circumstances, a vibrator 92 located towards the front of the user's leis shown providing a vibrational stimulus to the user.

[0073] FIG. 11 illustrates operation of an embodiment of the disclosed system in the event that a center of pressure 102 is estimated to be located towards the front and right side of the user's foot. Under such circumstances, a vibrator 92 located towards the front of the user's leg, and a vibrator 94 located towards the right of the user's leg are shown providing a vibrational stimulus to the user.

[0074] FIG. 12 illustrates frequency encoded magnitude stimulus in an illustrative embodiment. In the embodiment illustrated by FIG. 12, a relative increase in vibration frequency in one or more vibrators within the stimulation array is triggered by an increase in the polar radial distance of the center of pressure from a coordinate frame located near the center of the foot, or from the edge of the dead zone.

[0075] As shown in FIG. 12, the radial distance of the center of pressure from the center of the user's foot may be encoded using the frequency of vibration caused to occur in one or more vibrators in the stimulation array. As shown in FIG. 12, an estimated center of pressure 104, located towards the front of the user's foot, and being relatively farther from the center of the user's foot, may be represented to the user by causing the vibrator 92 in the stimulation array to vibrate at a relatively higher frequency. Such a higher frequency vibration thus represents the further distance of the estimated center of pressure from the center of the foot to the user. For example, the distance of the estimated center of pressure 104 from the center of the foot in FIG. 12 is greater than the distance of the estimated center of pressure 102 from the center of the foot in FIG. 11. Accordingly, the frequency of vibration of the vibrator 92 in FIG. 12 is greater than the frequency of vibration of the vibrators 92 and 94 in FIG. 11.

[0076] The disclosed system provides many and various advantages over previous systems. Specifically, the simplification of the balance information feedback provided by the disclosed system can more easily be integrated into the user's unconscious postural control system. The reduction of individual pressure signals by the disclosed system into an estimate of COP position and magnitude under each foot is easier to integrate into the postural control system than information regarding a number of separate pressure transducers.

[0077] A further advantage of the disclosed system relates to the coding of balance information using frequency modulation in addition to or as an alternative to amplitude modulation. Cutaneous stimulation has been shown to excite cutaneous mechanoreceptors on a 1 to 1 basis for a wide range of input frequencies. As a result, some cutaneous mechanoreceptors will respond to an artificial stimulation (vibrotactile or electrotactile) in the same manner as they would respond to a pressure stimulus. Simulating a natural pressure stimulus with an artificial one in this manner should facilitate the integration of this information into the unconscious balance control system.

[0078] Moreover, the location of feedback stimulators on the legs and oriented in a plane parallel to the plane of the foot sole should facilitate the integration of feedback information into the unconscious balance control system.

[0079] A number of specific variations and modifications are foreseen within the scope of the present invention. The following are some examples of variations and modifications:

[0080] 1. The sensor array and/or stimulation array may be incorporated into a stocking, shoe, or boot.

[0081] 2. The sensors may be embodied to acquire, encode, and provides feedback regarding shear forces under the user's foot or feet.

[0082] 3. The sensors may be embodied to acquire, encode, and provide balance information regarding angle and or angular velocity of the lower leg with respect to the foot.

[0083] 4. The disclosed system may be embodied to stimulate the cutaneous foot sole for the purpose of reducing postural deficits associated with long-term exposure to reduced foot loads, such as those incurred by bedridden patients on earth or astronauts in microgravity.

[0084] 5. The disclosed system may be embodied to stimulate the cutaneous foot sole for the purpose of producing an artificial feeling of pressure or shear force, such as might be used in virtual environments.

[0085] 6. The disclosed system may be embodied to stimulate the skin of a part of the body other than the foot sole for the purpose of producing an artificial feeling of pressure or shear force, such as might be used in virtual environments.

[0086] 7. The disclosed system may be embodied to stimulate the cutaneous foot sole in response to pressure under the foot for the purpose of amplifying the sensation of pressure.

[0087] 8. The disclosed system may be embodied to implement a signal processing method such that a range of COP positions and/or magnitudes produce no output from the feedback array (i.e. sensory dead zone).

[0088] 9. The mode of feedback may be embodied as tactile, vibrotactile, electrotactile, visual, thermal, and/or auditory.

[0089] 10. The sensor array is implanted into or under the skin or within the body.

[0090] 11. The feedback array may be implanted into or under the skin or within the body.

[0091] 12. The stimulation array may be implanted such that the feedback elements are adjacent to or in contact with one or more sensory neurons or sensory nerves.

[0092] 13. The sensor array may be affixed to or embedded within a prosthetic limb.

[0093] 14. The communication between any or all of the device components may be wireless.

[0094] 15. The sensor signals and/or feedback signals may be monitored remotely or recorded for the purpose of evaluating the effect or function of the device.

[0095] Those skilled in the art should appreciate that while the illustrative embodiments may implement the functions of the signal processing subsystem in computer software, these functions may alternatively be embodied in part or in whole using hardware components such as Application Specific Integrated Circuits, Field Programmable Gate Arrays, or other hardware, or in some combination of hardware components and software components.

[0096] While the invention is described through the above exemplary embodiments, it will be understood by those of ordinary skill in the art that modification to and variation of the illustrated embodiments may be made without departing from the inventive concepts herein disclosed. Accordingly, the invention should not be viewed as limited except by the scope and spirit of the appended claims.