METHOD FOR CONTROLLING AT LEAST ONE ACTUATOR OF AN ORTHOPEDIC DEVICE, AND ORTHOPEDIC DEVICE

Abstract

The invention relates to a method for controlling at least one actuator (4) of an orthopedic device (2) with an electronic control device (E), which is coupled to the actuator (4) and at least one sensor (8) and which has an electronic processor (μC) for processing sensor data (s), wherein at least one state machine (SM) in which states (z) of the orthopedic device (2) and state transitions of the actuator (4) are determined is stored in the control device (E), wherein a classifier (K) in which sensor data (s) and/or states (z) are automatically classified within the scope of a classification method is stored in the control device (E), wherein the state machine (SM) and the classification method are used in combination and, on the basis of the classification and the states (z), a decision is made about the manner of activating or deactivating the actuator (4) as a control signal.

Claims

1. A method for controlling at least one actuator of an orthopedic device, comprising: providing an electronic control device which is coupled to the at least one actuator and to at least one sensor, wherein the electronic control device comprises and has an electronic processor for processing sensor data from the at least one sensor, at least one state machine stored in the control device in which states of the orthopedic device and state transitions of the actuator are determined, automatically classifying in a classification process using a classifier stored in the control device sensor data and/or states of the orthopedic device, wherein the at least one state machine and the classification process are used in combination; and signaling the at least one actuator with a control signal for activation or deactivation on the basis of the classification and the states of the orthopedic device.

2. The method as claimed in claim 1, signals from the at least one sensor as input parameters to the classifier and as classified sensor data to the at least one state machine, or as input parameters to the at least one state machine and as states of the orthopedic device to the classifier.

3. The method as claimed in claim 1 further comprising supplying both the at least one state machine and the classifier with the sensor data of the at least one sensor as input parameters.

4. The method as claimed in claim 1 further comprising storing several classification processes in the classifier, and wherein the at least one state machine feeds a determined state as an input variable to the classifier.

5. The method as claimed in claim 1 further comprising feeding an output of the classifier back to an input of the classifier via the at least one state machine.

6. The method as claimed in claim 5, wherein a feedback signal is generated in the at least one state machine from the sensor data, and from results of the classifier and wherein the feedback signal changes at least one parameter of the classifier, wherein the classifier selects a different classification process, or wherein the feedback signal is fed to the classifier as a current state of the current states of the orthopedic device.

7. The method as claimed in claim 1 wherein the control signal is generated in the at least one state machine and/or in the classifier.

8. The method as claimed in claim 1 wherein the automatically classifying step performs several classification processes simultaneously, and wherein each classification process of the several classification processes can have different goals.

9. The method as claimed in claim 1 further comprising selecting the classification process from amongst several classification processes, or parameterizing the classifier, wherein the selecting or parameterizing takes place as a function of a current state of the states of the orthopedic device determined by the one or more state machines.

10. The method as claimed in claim 1 wherein the one or more state machines make discrete decisions or make decisions on the basis of fuzzy logic or fuzzy sets.

11. The method as claimed in claim 1 further comprising parameterizing the classifier by removal of one or more classes.

12. The method as claimed in claim 1 further comprising feeding a result of the classifier as an input variable to the one or more state machines.

13. An orthopedic device, comprising: with an electronic control device is coupled to an actuator and to at least one sensor, wherein the electronic control device comprises has an electronic processor for processing sensor data, wherein the electronic control device is configured to perform at least one classification based on the sensor data or on data calculated therefrom, wherein a classification process of the at least one classification depends on a state of the orthopedic device.

Description

[0023] Exemplary embodiments of the invention are explained in more detail below with reference to the accompanying figures, in which:

[0024] FIG. 1 shows a first series connection of classifier and state machine;

[0025] FIG. 2 shows a variant of FIG. 1, with parallel application of sensor data to the state machine;

[0026] FIG. 3 shows a parallel connection of classifier and state machine, with classifier state input;

[0027] FIG. 4 shows an activation of the state machine and classifier;

[0028] FIG. 5 shows the data flow with a feed-back classifier;

[0029] FIG. 6 shows different classifiers;

[0030] FIG. 7 shows an example of the data flow of an expanded state machine;

[0031] FIG. 8 shows a diagram of a classification problem with no state restriction;

[0032] FIG. 9 shows a diagram of a classification problem with state restriction;

[0033] FIG. 10 shows a diagram of an expanded state machine;

[0034] FIG. 11 shows a schematic representation of a prosthesis of an upper extremity; and

[0035] FIG. 12 shows a schematic representation of a prosthetic leg.

[0036] FIGS. 1 to 4 show different arrangements and sequences of classifiers K and state machines SM, which are supplied with sensor data s or with classified sensor data k or with states z, in order to send a signal to a processor or microcontroller μC, via which the further control of the orthopedic device, which is not shown in any more detail, is to be carried out.

[0037] In FIG. 1, the classifier K is arranged in series with the state machine SM. The original sensor data s are first fed to the classifier K and processed therein. The classifier K calculates the probabilities of the associated classes from the sensor data s or features and feeds the sensor data k thus classified to the state machine SM. In this exemplary embodiment, the state machine SM then no longer processes the original sensor data s, but already processed sensor data, such that, based on said processed sensor data or classified sensor data k, a distinction is made within the state machine SM with regard to the current state of the orthopedic device. On the basis of the established or fixed state, a control signal or a corresponding signal is then transmitted to the microcontroller μC, via which corresponding actuators are then activated or deactivated.

[0038] FIG. 4 shows a reversal of the order in which the original sensor data s are processed in comparison with FIG. 1. The original sensor data s are first fed to the state machine SM. In the state machine SM, it is possible to evaluate these sensor data and to predict the possible next state changes or considerably reduce the number of the possible next state changes. This is achieved through the network structure of the state machine SM and the associated and implemented knowledge of the current state of the orthopedic device. These states z or this state are/is transmitted to the downstream classifier K, which transmits a signal therefrom to the microcontroller μC or processor in order then to carry out the further control measures. With such a configuration and combination of a classifier K with a state machine SM, the classifier K does not always have to classify all data or features, but calculates the probabilities only from the options that appear to be practicable and that have been selected by the state machine SM. For example, if a prosthetic hand is in the “lateral grip” state, then there are only two options for a further command, namely “close lateral grip” or “open lateral grip”.

[0039] In FIG. 2, analogously to the procedure in FIG. 1, the sensor data s are first fed to the classifier K, from which the classified sensor data k are then fed to the state machine SM. However, the classified sensor data k are not the sole input variables of the state machine SM; rather, the original sensor data s are also transmitted to the state machine SM, such that the state machine SM also used, in addition to the classified sensor data k, raw sensor data or otherwise processed sensor data in order to determine the respective state and the associated measure. The state machine SM can thus make better decisions on the basis of additional information that is made available by the classifier K.

[0040] A similar structure of the combination of a state machine SM and a classifier K is shown in FIG. 3 in which, in addition to the sensor raw data s, state data z from the state machine SM are also transmitted to the classifier K. It is possible that the state machine SM does not just use state data or states z as further influencing variables to influence the classifier K, but also changes in the classes within the classifier K.

[0041] FIG. 5 shows the data flow of a combination of a classifier K with a feedback state machine SM. Features F are derived for the specific classifier K from the sensor raw data s. In addition, the sensor raw data s are sent directly to the state machine SM. Probabilities P are calculated in the classifier K from the features F. These probabilities P are fed to the state machine SM as an input variable. The state machine SM can use the probabilities P or classified sensor data z to generate an output signal y2, on the basis of which the orthopedic device, for example the prosthesis, can be controlled. The state machine SM runs through different states Z on the basis of the history of the input signals and the results of the classifier K. In the simplest case, the state Z corresponds exactly to the class with the highest probability. The current state Z can now be fed directly as an input to the classifier K. Alternatively, the state machine SM influences the classifier K in that its parameters are modified or different classifiers K are selected. The respective state z, which was generated on the basis of the sensor raw data s and possibly the classified sensor data from the feedback with the classifier K, can be fed to the classifier K from the state machine SM. The classified sensor data can then be passed through a microcontroller μC to control the orthopedic device.

[0042] The parameterization and the selection of different classifiers K make it possible to use the optimal and leaner classifier K in each case for different decisions. In addition to influencing state transitions in the state machine SM, the classifiers K can also be used to influence the control within a state. If, for example, the closing of a prosthetic hand is specified as the state, the way in which the hand is closed can be influenced via a classifier K. On the basis of the sensor raw data, the classifier generates further control signals, which are used either via a microcontroller μC or directly to an actuator to influence the orthopedic device. In principle, it is also provided that several classifiers K are executed at the same time, but each classifier K has different goals. When the classifiers K have been completely developed, they work very simply and quickly and can be executed in parallel without significantly increasing the computational effort.

[0043] Specific classifiers K and their selection are shown schematically in FIG. 6. The respective classifier K is selected, for example, as a function of the current state, which is fixed via a state machine SM. The parameterization of the respective classifier K can likewise be selected or changed as a function of the current state of the control or of the orthopedic device. The structure of the classifier can likewise be varied as a function of the respective state, for example the number of classes, number of neurons, arrangement of the layers and the like.

[0044] The data flow of an expanded state machine SM is shown in FIG. 7. The central component of the expansion of the state machine SM is a library of generally parameterizable classifiers that are coupled to the state machine SM. The respective classifiers can be optimized for their specific task using feature engineering, feature extraction and machine learning. In principle, it is also possible for a plurality of state machines SM to be coupled to the library of classifiers. The state machine SM or the respective state machine SM and also the respective classifier K or also several classifiers K communicate with one another during the period of use and exert an interaction on one another during the running time, such that the two components interact dynamically with each other over the running time. The respective state machine SM selects those classifiers K as a function of the state or the respective parameter set which is best suited to optimally solve the currently pending decision. On the left, next to the combination of classifier K and state machine SM, the courses of the different sensor signals s are shown, for example forces, moments, myoelectric signals, angle courses or the like. The features F derived therefrom are sent to the classifiers or the classifier K. The calculated probabilities within the classifier K are sent to the state machine SM, the states Z are sent to the classifier K in order to change the parameters within the classifier K. In the state machine SM, for example, discrete decisions are made, that is to say certain states Z exist; alternatively, decisions are made on the basis of probabilities or approximations, so-called fuzzy control. In principle, it is also possible for the processed features to be used by the classifier K directly for control.

[0045] An example of a control is the movement control of a prosthetic hand, in which, for example, two types of grip A and B can be implemented. In principle, both types of grip A or B can be performed from the open position of the prosthetic hand, whereas, with a closed or partially closed prosthetic hand, only the opening of the prosthetic hand is possible. As soon as the prosthetic hand begins, for example, to perform the type of grip A and to close the prosthetic hand, on the basis of a classification result, it is only necessary for further control to distinguish between type of grip A and opening. The other grip type B can be neglected in the classification. The class boundaries are accordingly simplified. The second type of grip B is included in the decision-making process again only after the prosthetic hand has been fully opened. In the case of a conventional control with only one classifier, the second type of grip B could also be recognized during the closing and opening of the first type of grip A, as a result of which the closing or opening of the prosthetic hand would need to have been interrupted.

[0046] In order to rule this out, the control method according to the invention provides, for example, that the current state, namely the type of grip currently present, is returned as a feature F to the classifier K, by which means the feature space is expanded by a unique feature. A distinction is made between the adopted state of the prosthesis, namely the respective type of grip and the open state, and the last recognized class. Alternatively, it is possible to change the parameters of the classifier K over the running time, if a certain state has been recognized, for example by removing a class from the classifier K.

[0047] This example is explained in more detail with reference to FIG. 8 and FIG. 9. In FIG. 8, there is no restriction of a state in the state machine SM. Therefore, a distinction has to be made at each time between all the existing classes, namely prosthetic hand open, grip A and grip B. Due to the lack of a sharp separation between the two grip types A and B in the feature space x1-x2, incorrect classifications may occur. The classification Open->Grip B->Grip A->Grip B->Open may be output over the course of time of the feature vector. In the case of a combination of a classifier K with a state restriction via the state machine SM, as is shown in FIG. 9, the distinction between grip type A and grip type B only has to be made initially. When performing grip type B, a distinction only needs to be made between gripping and opening. The class for the grip type can be ignored. The reduction of the classes results in a much simpler classification problem. The plotted time course of the feature vector can only lead to the correct classification Open->Grip B->Open.

[0048] FIG. 10 shows an expanded state machine SM which can be used to control a prosthetic knee joint or orthotic knee joint. First of all, two classifiers are used to make local decisions of a state machine SM. After the end of a swing phase sw, with the heel strike HS, a hidden Markov model (HMM) is used to determine whether a transition to walking on the level L or to walking downward D is controlled. For walking downward D, a neural network NN is again used, the output of which forms an argument in a control characteristic area. Both classifiers HMM and NN are highly specialized and are used locally, depending on the current state—walking on the level or walking downward—for a typical discrete decision.

[0049] When walking on the level, walking down a slope or walking down stairs with an artificial knee joint, different flexion resistances have to be approached. This is achieved by combining a state machine SM with an assignment of the gait phases in combination with classifiers K. First of all, the trajectory of the foot is recorded in the swing phase and, in the event of a heel strike HS, is fed to a hidden Markov model, possibly in a simplified mathematical representation. The hidden Markov model can use the trajectory to differentiate whether the step concerned is a flat step or a downward step. The output probabilities of the hidden Markov model are the basis for branching into two different branches of the state machine. The first branch is for walking on the level, the second for downward steps. Once a decision has been made to walk downward, a distinction must be made between walking on a ramp and descending stairs. The neural network NN is used for this purpose. The courses of the segment angles of the prosthesis or orthosis and the loads after the heel strike HS, for example, serve as input variable s. The outputs in the form of probabilities for descending stairs or walking downhill on a ramp are not used in this case for a further branching of the state machine, but as input parameters of a control characteristic area which, from the probabilities and the knee angle, calculates the flexion resistance that is necessary for the respective movement scenario.

[0050] The combination or entanglement of classifiers K with state machines SM thus has advantages over the respectively isolated application of these components. Particularly in the case of pattern recognition of electromyographic signals or other neural signals of the upper extremity, it is advantageous to add a state machine to the control of a prosthesis. Likewise, the tried and tested state machine control of the lower extremity can easily be supplemented with a classifier-based control component.

[0051] For all of the exemplary embodiments and for all of the controls and variants described, it is provided that the classifiers are designed to be self-learning or can be designed to be self-learning. Self-learning classifiers can, for example, use a cluster analysis to independently carry out or evaluate a class formation for an unclassified set of feature vectors and in doing so take into account their own classification results.

[0052] FIG. 11 shows a schematic representation of an embodiment of the orthopedic device 2 in the form of an actuatable prosthesis device of a lower extremity. The prosthesis device 2 is partially shown along its longitudinal axis in the manner of an exploded view. The prosthesis device 2 is designed as a prosthetic lower arm and has several actuators 4 in the form of electromotive drives, a sensor arrangement 6 with a plurality of electrode pairs 8, and a control device E in the form of an electronic data processing device.

[0053] The drive 4 shown to the right in FIG. 11 is used to rotate a prosthetic hand 10 of the prosthesis device 2 relative to a forearm socket 12 of the prosthesis device 2 about the longitudinal axis of the forearm socket 12. In addition, the prosthetic hand 10 has a plurality of further drives or actuators 4 within the prosthetic hand 10, which are used to actuate the prosthetic fingers 14 of the prosthetic hand 10.

[0054] The sensor arrangement 6 has four electrode pairs 8. These are designed, for example, as individual electrode pairs 8, each of which can be attached to the skin of a prosthesis wearer. According to a further embodiment, the electrode pairs 8 are applied to a prosthesis liner (not shown), for example glued on, or connected to or formed integrally with a liner. According to a further embodiment, the electrode pairs 8 are arranged on an inner side of the forearm socket 12.

[0055] The electrode pairs 8 are each connected to the control device E via sensor lines 16. The signals detected by the electrodes are passed to the control device E via the sensor lines 16.

[0056] The signals are evaluated in the control device E. In addition to the illustrated electrode pairs 8 as sensors for detecting myoelectric signals, other sensors 8 are arranged on the forearm socket and can be designed as inertial sensors, IMU, spatial position sensors, acceleration sensors, force sensors, angle sensors, temperature sensors or other sensors. A plurality of sensors 8 can also be arranged on the orthopedic device, which sensors detect different measured variables or conditions. Likewise, sensors 8 can be arranged in the prosthetic hand, for example position sensors that detect the position of the prosthetic fingers.

[0057] In the embodiment shown in FIG. 11, only one of the schematically depicted drive lines 18 is used for the actuator 4 shown. The other drive lines 18 are passed through this or are attached to other lines and supply the other actuators in the form of the motor drives 4 in the prosthetic hand 10 for actuating the phalanges 14.

[0058] In the present case, the control device E is in wireless communication, for example via radio, with an input device 20. Signals received from the sensor arrangement 6, for example, can be visualized via this input device 20. In addition, it is preferably possible to change the programs, classifiers K or state machines SM stored in the control device E.

[0059] FIG. 12 shows a further embodiment of the orthopedic device 2, here in the form of a prosthetic leg for a thigh amputee patient. The prosthesis device 2 has a sensor arrangement 6 with several, for example six, electrode pairs 8, only four of which are visible in FIG. 12.

[0060] The prosthesis device 2 has a prosthetic foot 22 and a lower-leg part 24. A prosthetic knee joint 26, which has an upper part 28, is arranged on the lower-leg part 24. A damper serving as actuator 4 is assigned to the prosthetic knee joint 26. Drives or actuators for adjusting valves or for setting other manipulated variables can be arranged in the damper itself. Actuators are not limited to electric motors, but are generally used to influence and/or change properties or positions of components of the orthopedic device. A prosthesis socket is arranged on the joint upper part 28; however, for the sake of clarity, it is not shown in FIG. 12. Instead, the underlying limb stump 32 of a prosthesis wearer is indicated.

[0061] The prosthetic knee joint 26 can also be assigned an active drive (not shown) as an actuator, which actuates a flexion and/or extension movement of the prosthesis device 2. Control signals from the control device E are sent to this drive via drive lines 18, which are likewise not shown.

[0062] The sensor arrangement 6 has a flexible, in particular elastic belt 34 on which the electrode pairs 8 are arranged. This belt is arranged around the limb stump 32, such that the electrode pairs 8 rest on it and can accordingly detect signals. The other sensors 8 mentioned in connection with FIG. 11 can likewise be arranged on the prosthetic leg and/or the stump and connected to the control device. In addition to prostheses, orthopedic devices within the meaning of the invention also include in particular orthoses and exoskeletons.