SYSTEM AND METHOD FOR PREDICTING POSTOPERATIVE BED TYPE

Abstract

A system and method are provided for generating a predictive model for predicting a postoperative bed type to be used by a patient after surgery. The predictive model is trained on features extracted from medical data and using a postoperative bed type as prediction target in the training. The predictive model is configured to output a probability on a scale 400 which corresponds to, at its lower end, a prediction of a first postoperative bed type and, at its upper end, a prediction of a second postoperative bed. A hybrid model is generated which applies a lower 410 and an upper threshold 420 to the probability scale. If the output probability of the predictive model is in between both thresholds, an expert selection of the bed type is recommended, while otherwise, the prediction of the predictive model is output. The values of the thresholds are optimized using a performance metric.

Claims

1. A system for generating a predictive model for predicting a postoperative bed type for use by a patient after surgery, comprising: an input interface for accessing medical data comprising records of surgeries, wherein a record of a surgery is indicative of a postoperative bed type used by a patient after the surgery, wherein the postoperative bed type is one of at least two possible bed types, wherein the medical data comprises data characterizing the surgery and the patient; a processor subsystem configured to generate a predictive model for predicting the postoperative bed type to be used by a patient after surgery by: training a predictive model on the medical data, wherein the training uses the postoperative bed type as prediction target, wherein the predictive model is configured to output a probability on a scale which corresponds to, at its lower end, a prediction of a first one of the at least two possible bed types and, at its upper end, a prediction of a second one of the at least two possible bed types; generating a hybrid predictive model by establishing an upper threshold and a lower threshold within or at an endpoint of the scale, wherein the hybrid predictive model is configured to, during use: if the probability is below the lower threshold, output as the prediction the first one of the at least two possible bed types; if the probability is above the upper threshold, output as the prediction the second one of the at least two possible bed types; if the probability is in between the lower threshold and the upper threshold, recommend or refer to an expert selection of the bed type, wherein generating the hybrid model comprises selecting the upper threshold and the lower threshold to optimize a performance metric using the postoperative bed type indicated by the medical data as prediction target.

2. The system according to claim 1, wherein the at least two postoperative bed types differ in level of care provided to the patient.

3. The system according to claim 1, wherein the at least two bed types comprise: an intensive care unit (ICU) bed type and a post-anaesthesia care unit (PACU) bed type; an intensive care unit (ICU) bed type and a general ward bed type; or a post-anaesthesia care unit (PACU) bed type and a general ward bed type.

4. The system according to claim 2, wherein the performance metric penalizes a first type of erroneous prediction by which a lower level of care bed type is predicted more than a second type of erroneous prediction by which a higher level of care bed type is predicted.

5. The system according to claim 4, wherein the performance metric rewards minimization of occurrences of the first type of erroneous prediction while maintaining occurrences of the second type of erroneous prediction below an acceptability threshold.

6. The system according to claim 1, wherein selecting the upper threshold and the lower threshold comprises one of: evaluating different combinations of values for the upper threshold and the lower threshold; selecting the lower threshold to be equal to the upper threshold and evaluating different values for both the lower threshold and the upper threshold; selecting the lower threshold at the lower endpoint of the scale and evaluating different values for the upper threshold; and selecting the upper threshold at the upper endpoint of the scale and evaluating different values for the lower threshold.

7. The system according to claim 1, wherein the processor subsystem is further configured to: use a feature extraction technique to identify a set of features in the medical data, which set of features is predictive of the postoperative bed type; and train the predictive model using the set of features as input.

8. The system according to claim 1, wherein the processor subsystem is further configured to: receive an identification of a subset of records in the medical data, wherein in surgeries represented by the subset of records, the postoperative bed type is determined by external factors to be disregarded by the predictive model; determine if the medical data excluding the subset of records is sufficient for training the predictive model; if the medical data excluding the subset of records is determined not to be sufficient for the training of the predictive model: train the predictive model on the medical data including the subset of records; when selecting the upper threshold and the lower threshold to optimize the performance metric, exclude the subset of records from the medical data.

9. The system according to claim 8, wherein the processor subsystem is configured to receive the identification of the subset of records in form of a time range in which, or a time after or before which, a respective surgery is performed.

10. The system according to claim 1, wherein the performance metric is a user-definable metric.

11. A system for using a predictive model for predicting a postoperative bed type for use by a patient after surgery, comprising: an input interface for accessing: a predictive model for predicting the postoperative bed type to be used by the patient after surgery, wherein the predictive model is configured for outputting a probability on a scale which corresponds to, at its lower end, a prediction of a first one of the at least two possible bed types and, at its upper end, a prediction of a second one of the at least two possible bed types; values for an upper threshold and a lower threshold within or at an endpoint of the scale; input data characterizing a planned surgery of the patient and/or characterizing the patient; a processor subsystem which is configured to use the predictive model with the values for the upper threshold and the lower threshold to predict a postoperative bed type for use by a patient after surgery by: using the input data as input to the predictive model to obtain a probability; if the probability is below the lower threshold, output as the prediction the first one of the at least two possible bed types; if the probability is above the upper threshold, output as the prediction the second one of the at least two possible bed types; and if the probability is in between the lower threshold and the upper threshold, recommend or refer to an expert selection of the bed type.

12. The system according to claim 11, wherein the expert selection is a selection by a clinician.

13. A computer-implemented method for generating a predictive model for predicting a postoperative bed type for use by a patient after surgery, comprising: accessing medical data comprising records of surgeries, wherein a record of a surgery is indicative of a postoperative bed type used by a patient after the surgery, wherein the postoperative bed type is one of at least two possible bed types, wherein the medical data comprises data characterizing the surgery and the patient; generating a predictive model for predicting the postoperative bed type to be used by the patient after surgery by: training a predictive model on the medical data, wherein the training uses the postoperative bed type as prediction target, wherein the predictive model is configured to output a probability on a scale which corresponds to, at its lower end, a prediction of a first one of the at least two possible bed types and, at its upper end, a prediction of a second one of the at least two possible bed types; generating a hybrid predictive model by establishing an upper threshold and a lower threshold within or at an endpoint of the scale, wherein the hybrid predictive model is configured to, during use: if the probability is below the lower threshold, output as the prediction the first one of the at least two possible bed types; if the probability is above the upper threshold, output as the prediction the second one of the at least two possible bed types; if the probability is in between the lower threshold and the upper threshold, recommend or refer to an expert selection of the bed type, wherein generating the hybrid model comprises selecting the upper threshold and the lower threshold to optimize a performance metric using the postoperative bed type indicated by the medical data as prediction target.

14. A computer-implemented method for predicting a postoperative bed type for use by a patient after surgery, comprising: accessing: a predictive model for predicting the postoperative bed type to be used by the patient after surgery, wherein the predictive model is configured to output a probability on a scale which corresponds to, at its lower end, a prediction of a first one of the at least two possible bed types and, at its upper end, a prediction of a second one of the at least two possible bed types; values for an upper threshold and a lower threshold within or at an endpoint of the scale; input data characterizing a planned surgery of the patient and/or characterizing the patient; using the predictive model with the values for the upper threshold and the lower threshold to predict a postoperative bed type for use by the patient after surgery by: using the input data as input to the predictive model to obtain a probability; if the probability is below the lower threshold, output as the prediction the first one of the at least two possible bed types; if the probability is above the upper threshold, output as the prediction the second one of the at least two possible bed types; and if the probability is in between the lower threshold and the upper threshold, recommend or refer to an expert selection of the bed type.

15. A transitory or non-transitory computer-readable medium comprising data representing a computer program, the computer program comprising instructions for causing a processor system to perform the method according to claim 13.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0076] These and other aspects of the invention will be apparent from and elucidated further with reference to the embodiments described by way of example in the following description and with reference to the accompanying drawings, in which

[0077] FIG. 1 shows an exemplary system for generating and/or using a predictive model for predicting a postoperative bed type, which system comprises an input interface for accessing medical data, a processor subsystem for generating and/or using the predictive model and a user interface subsystem for displaying a graphical user interface to a user;

[0078] FIG. 2 shows an error/confusion matrix for the recommendation of a postoperative bed type by a clinician, with the postoperative bed type being either an ICU or a PACU bed;

[0079] FIG. 3 illustrates surgeries for which a type 1 error occurred in the clinician's recommendation, i.e., a PACU bed was recommended but the patient ended up in an ICU bed, together with the cumulative percentage of such surgeries depending on the time slot during the day when the surgery occurred;

[0080] FIG. 4 shows an error/confusion matrix for the prediction of the postoperative bed type by a predictive model which was trained to predict the postoperative bed type;

[0081] FIG. 5 shows a probability scale in which two thresholds are used to select between the prediction of the predictive model and the clinician's recommendation;

[0082] FIG. 6 illustrates a selection of thresholds in which false negatives are kept below 10% while optimizing the performance of the hybrid model for a performance metric;

[0083] FIG. 7 shows an error/confusion matrix for the prediction of the postoperative bed type by a hybrid model, wherein the hybrid model selects between the prediction of the predictive model and the clinician's recommendation using thresholds;

[0084] FIGS. 8A and 8B each show an example of a graphical user interface of a system which uses the hybrid model to recommend a postoperative bed type, wherein in the example of FIG. 8A the system recommends a PACU bed while in the example of FIG. 8B the system refers to an expert selection of the postoperative bed type; and

[0085] FIG. 9 shows a non-transitory computer-readable medium comprising data.

[0086] It should be noted that the figures are purely diagrammatic and not drawn to scale. In the figures, elements which correspond to elements already described may have the same reference numerals.

LIST OF REFERENCE NUMBERS

[0087] The following list of reference numbers is provided for facilitating the interpretation of the drawings and shall not be construed as limiting the claims. [0088] 20 data storage [0089] 22 medical data [0090] 24 data representing trained predictive model [0091] 26 data representing thresholds [0092] 60 display [0093] 62 display data [0094] 80 user input device(s) [0095] 82 user input data [0096] 100 system for generating and/or using predictive model [0097] 120 data storage interface [0098] 140 processor subsystem [0099] 142-146 data communication [0100] 160 memory [0101] 180 user interface subsystem [0102] 182 display output interface [0103] 184 user input interface [0104] 200 error/confusion matrix of clinician's recommendation [0105] 202 error/confusion matrix of trained predictive model [0106] 204 error/confusion matrix of hybrid model [0107] 210 predicted bed type [0108] 220 actual bed type [0109] 300 time [0110] 310 time interval [0111] 320 morning surgeries [0112] 330 percentages of surgeries in time interval or in time intervals lower on the vertical axis [0113] 400 probability scale [0114] 410 first threshold [0115] 420 second threshold [0116] 500 visualization of prediction performance as a function of threshold values and false negative rate [0117] 510 prediction performance metric output [0118] 600 user interface window [0119] 610 element showing patient data [0120] 620 element showing surgery data [0121] 630 element showing output of predictive model [0122] 640 element showing recommendation [0123] 642 recommendation by predictive model [0124] 644 action button to enter clinician's recommendation [0125] 700 non-transitory computer-readable medium [0126] 710 data representing computer program

DETAILED DESCRIPTION OF EMBODIMENTS

[0127] FIG. 1 shows a system 100 which may in some embodiments represent a system for generating a predictive model for predicting a postoperative bed type for use by a patient after surgery, in other embodiments a system for using the predictive model to predict the postoperative bed type for a patient for which a surgery is scheduled, and in yet other embodiments a system for both generating and using the predictive model.

[0128] The system 100 is shown to comprise a data storage interface 120 to a data storage 20. In some embodiments, the data storage 20 may store medical data 22 comprising records of surgeries. These records, or other parts of the medical data 22, may characterize a respective surgery and the patient undergoing the surgery. In some embodiments, the data storage 20 may store model data 24 representing a machine learnable predictive model. In some embodiments, the data storage 20 may store threshold data 26 of thresholds described elsewhere in this specification. In general, the data storage 20 may serve as short term storage and/or as long-term data storage. In the example of FIG. 1, the data storage interface 120 is shown to be connected to an external data storage 20, such as a network-accessible data storage 20. Alternatively, the data storage 20 may be an internal data storage of the system 100 (not shown in FIG. 1). In general, the data storage interface 120 may take various forms, such as a hard disk or solid-state disk interface to one or more hard disks and/or solid-state disks, or a network interface to a Local Area Network (LAN) or to a Wide Area Network (WAN). In general, the data storage interface 120 may represent an example of an input interface as described elsewhere in this specification.

[0129] The system 100 is further shown to comprise a processor subsystem 140 configured to internally communicate with the data storage interface 120 via data communication 142, with a memory 160 via data communication 144 and with a user interface subsystem 180 via data communication 146. The memory 160 may for example be a volatile memory in which a computer program may be loaded which may cause the processor subsystem 140 to carry out functions which are described in this specification, for example in relation to generating a predictive model and/or to using the predictive model.

[0130] In some embodiments, the system 100 may comprise a user interface subsystem 180, which user interface subsystem may be configured to, during operation of the system 100, enable a user to interact with the system 100, for example using a graphical user interface. In particular, as also described elsewhere, the graphical user interface may enable the user to obtain a postoperative bed type recommendation. For that and other purposes, the user interface subsystem 180 is shown to comprise a user input interface 184 configured to receive user input data 82 from one or more user input devices 80 operable by the user. The user input devices 80 may take various forms, including but not limited to a keyboard, mouse, touch screen, microphone, etc. FIG. 1 shows the user input devices to be a keyboard and mouse 80. In general, the user input interface 184 may be of a type which corresponds to the type of user input device(s) 80, i.e., it may be a thereto corresponding type of user device interface. The user interface subsystem 180 is further shown to comprise a display output interface 182 configured to provide display data 62 to a display 60 to visualize output of the system 100. In the example of FIG. 1, the display is an external display 60. Alternatively, the display may be an internal display of the system 100.

[0131] In some embodiments, the processor subsystem 140 may be configured to, during operation of the system 100, generate a predictive model for predicting a postoperative bed type for use by a patient after surgery. For that purpose, the processor subsystem 140 may be configured to train a predictive model on the medical data 22 using the postoperative bed type as prediction target. The predictive model may be configured, e.g., by design, to output a probability on a scale which corresponds to, at its lower end, a prediction of a first one of the at least two possible bed types and, at its upper end, a prediction of a second one of the at least two possible bed types. The processor subsystem 140 may be further configured to generate a hybrid predictive model by establishing an upper threshold and a lower threshold within or at an endpoint of the scale. The hybrid predictive model may be configured to, during use, if the probability is below the lower threshold, output as the prediction the first one of the at least two possible bed types, if the probability is above the upper threshold, output as the prediction the second one of the at least two possible bed types, and if the probability is in between the lower threshold and the upper threshold, recommend or refer to an expert selection of the bed type. The processor subsystem 140 may be further configured to, when generating the hybrid model, select the upper threshold and the lower threshold to optimize a performance metric using the postoperative bed type indicated by the medical data as prediction target.

[0132] In other embodiments, the processor subsystem 140 may alternatively or additionally be configured to, during operation of the system 100, use the generated predictive model for prediction purposes. For that purpose, the processor subsystem 140 may be configured to access the predictive model as described elsewhere in this specification, values for an upper threshold and a lower threshold as described elsewhere in this specification, and input data characterizing a surgery of a patient and/or characterizing the patient. The processor subsystem 140 may be further configured to use the predictive model with the values for the upper threshold and the lower threshold to predict a postoperative bed type for use by the patient after surgery by using the input data as input to the predictive model to obtain a probability, and if the probability is below the lower threshold, output as the prediction the first one of the at least two possible bed types, if the probability is above the upper threshold, output as the prediction the second one of the at least two possible bed types, and if the probability is in between the lower threshold and the upper threshold, recommend or refer to an expert selection of the bed type.

[0133] These and other operations of the system 100, and various optional aspects thereof, will be explained in more detail with reference to FIG. 2 et seq.

[0134] In general, the system 100 may be embodied as, or in, a single device or apparatus. The device or apparatus may be a general-purpose device or apparatus, such as a workstation or a computer, but may also be application-specific, such as a patient monitor. The device or apparatus may comprise one or more microprocessors which may represent the processor subsystem, and which may execute appropriate software. The software may have been downloaded and/or stored in a corresponding memory, e.g., a volatile memory such as RAM or a non-volatile memory such as Flash. Alternatively, the functional units of the system, e.g., the input interface, the user interface subsystem, and the processor subsystem, may be implemented in the device or apparatus in the form of programmable logic, e.g., as a Field-Programmable Gate Array (FPGA). In general, each functional unit of the system 100 may be implemented in the form of a circuit. It is noted that the system 100 may also be implemented in a distributed manner, e.g., involving different devices or apparatuses. For example, the distribution may be in accordance with a client-server model, e.g., using a server and workstation. For example, the user input interface and the display output interface may be part of the workstation, while the processor subsystem may be a subsystem of the server. It is noted that various other distributions are equally conceivable.

[0135] FIG. 2 shows an error/confusion matrix 200 for the recommendation of a postoperative bed type by a clinician, with the postoperative bed type being either an ICU or a PACU bed, which shows along the horizontal axis 210 the predicted bed type and on the vertical axis the actual bed type. This error matrix was generated based on actual medical data of cardiothoracic surgeries, in which both the recommendation of the clinician before the surgery and the actual bed type in which the patient ended up in after surgery were identified and depicted as columns and rows of the confusion matrix, respectively. The values in the error matrix 200, as well as those of other error matrixes described in this specification, are normalized to 1.0 (=100%). It can be seen that for 34% of the patients a PACU bed is predicted by the clinician before surgery, and a PACU bed is indeed sufficient after surgery (sometimes also referred to as fast-track), and that for 43% of the patients an ICU bed is predicted by the clinician before surgery, and an ICU bed is indeed needed after surgery. For the error/confusion matrix 200 of FIG. 2 and that of FIGS. 4 and 7, the correct prediction of a PACU bed may be considered as a true positive (TP), while the correct prediction of an ICU bed may be considered as a true negative (TN). However, there are also erroneous predictions in the form of false positives (FP) and false negatives (FN). Namely 19% of the patients ended up in an ICU bed although the clinician recommended a PACU bed (false positive, or type 1 error, a lower level of care was predicted than needed) Similarly, 4% of the patients ended up in a PACU bed although the clinician recommended an ICU bed (false negative, or type 2 error, a higher level of care was predicted).

[0136] The postoperative bed type may be determined based on medical factors, e.g., the type of surgery and the age and BMI of a patient, as well as external factors such as start and end time of the surgery. Detecting surgeries where external factors were prevailing may lead to more precise analysis of errors and to training of a more precise predictive model. FIG. 3 shows all the cardiothoracic surgeries for which a type 1 error (a lower level of care was predicted than needed) occurred in the clinician's recommendation with respect to the time slot when the surgery was performed. The vertical axis 310 shows different time intervals in which the surgeries were performed, the horizontal axis 300 being a time axis (in hours of the day), and the bars being again a visualization of the time intervals set out on the vertical axis 310. The percentages 330 next to the bars indicate the percentage of all depicted surgeries performed in the respective time slot, or in any other time slot shown lower on the vertical axis 310 and are therefore calculated in a cumulative manner. The PACU unit may be closed at night, and patients who were operated late in the morning or in the afternoon and required at least the level of care provided by in a PACU unit may thus be assigned an ICU bed. Hence, for all the surgeries that were performed late in the morning or in the afternoon, the external factors may prevail in assigning a bed over the medical factors. On the other hand, for morning surgeries, which are indicated in FIG. 3 by the bounding box 320, the choice of ICU bed or PACU bed may be mainly determined by medical factors. The conclusion may be that only 39% of all type 1 error surgeries should be considered as an erroneous recommendation, while the rest only appear to be erroneous predictions, while in reality the recommendation was not followed due to external factors.

[0137] A predictive model was trained to predict the postoperative bed type for use by a patient after surgery based on the medical data of the aforementioned cardiothoracic surgeries. In this specific example, records of 2363 cardiothoracic surgeries were accessed, which were filtered down to 1481 cardiothoracic surgeries, e.g., by selecting only elective surgeries, selecting surgeries where the postoperative bed type is specified, etc. Also, by way of the filtering, some surgeries were omitted from which it was certain that the choice of bed type was determined by external factors rather than by medical factors. From these 1481 surgeries, 1049 were used for training the predictive model and 251 for testing purposes. The model itself was a binary classification model which was trained using CatBoost to predict either an ICU bed (prediction target=0) or a PACU bed (prediction target=1.0). The features used as input during the training included patient features, such as the ASA score, the number of medications taken by a patient at home, the creatine level, the gender, the BMI, the age group to which the patient belonged, the number of procedures that the patient will undergo, etc., as well as surgery-related features, such as the type of surgery, which for example included aortic valve repairs and coronary artery bypass grafts.

[0138] FIG. 4 shows an error matrix 202 for the prediction of the postoperative bed type by the aforementioned predictive model. It can be seen that for the predictive model, for 31% of the patients a PACU bed is predicted by the predictive model, and a PACU bed is sufficient after surgery, and that for 40% of the patients an ICU bed is predicted by the predictive model, and an ICU bed is needed after surgery. However, for 22% of the patients a PACU bed is predicted by the predictive model, but an ICU bed (i.e., a higher level of care than originally predicted) is needed after surgery, and for 7% of the patients the predictive model predicts an ICU bed, but a PACU bed (i.e., a lower level of care than predicted) is sufficient after surgery. When comparing the error matrix 202 of the predictive model with the error matrix 200 of the clinician's recommendation as discussed with reference to FIG. 2, it appears at first glance that the predictive model does not improve upon the clinician's recommendation. This is seemingly confirmed in Table 1, which shows well known performance metrics for classification models in term of Accuracy ((TP+TN)/(TP+TN+FP+FN)), Recall (Sensitivity=TP/(TP+FN)), Precision (PPV=TP/(TP+FP)).

TABLE-US-00001 TABLE 1 Comparison of clinician's and predictive models' performance Metric Clinician's recommendation Predictive model Accuracy 0.77 0.71 Recall (Sensitivity) 0.89 0.84 Precision (PPV) 0.64 0.58

[0139] However, when the predictive model is only evaluated on morning surgeries (being a subset of the test data in which the selection of bed type was considered not or at least less influenced by external factors), an improvement could be seen over the clinician's recommendation in terms of number of patients classified correctly for the PACU bed type. To quantify this improvement, a custom performance metric may be used, which is defined by (TP+FP.sub.corrected)/FP.sub.corrected with TP referring to the number of true positives and FP.sub.corrected to the number of false positives which were scheduled in the morning, since if a false positive surgery was not scheduled in the morning one may not be certain whether it is a false positive due to medical reasons or due external factors, this is why FP.sub.corrected may be used rather than FP. This performance metric may be considered as a one-in metric which expresses one in how many patients were recommended for PACU but ended up in ICU, which is a prediction error which poses more health risks than the other way around and therefore is more relevant in the assessment of the performance of the prediction.

[0140] When applying this one-in performance metric to both the clinician's recommendations as discussed with reference to FIG. 2 and the predictive model as discussed with reference to FIG. 4, an improvement is visible for the surgeries scheduled in the morning, in that for the clinician, approximately 1 in 5 patients was recommended for PACU yet ended up in an ICU bed, while for the predictive model, only 1 in 7 patients was predicted for PACU yet ended up in an ICU bed.

[0141] FIG. 5 shows a probability scale 400 associated with the predictive model. Here, the lower endpoint of the scale, which corresponds to a prediction score of 0.0, represents a prediction of an ICU bed while the higher endpoint of the scale, which corresponds to a prediction score of 1.0, represents a prediction of a PACU bed. To further improve on the predictive model's performance, a hybrid predictive model may be generated by establishing an upper threshold 420 and a lower threshold 410 each within or at an endpoint of the scale 400, with the thresholds being used to select whether the prediction of the predictive model is to be followed or whether an expert selection is recommended. In particular, below the lower threshold 410, the predictive model's prediction of an ICU bed may be followed, while above the upper threshold 420, the predictive model's prediction of a PACU bed may be followed, while in-between the lower threshold 410 and the upper threshold 420, a user may be recommended or referred to the expert selection.

[0142] To determine optimal values for the upper threshold 420 and the lower threshold 410, a performance metric may be used. This performance metric may be, or may comprise, the aforementioned one-in performance metric to quantify how many patients which were recommended for PACU ended up in an ICU bed. In optimizing the threshold values, the performance metric may be optimized so that the number of patients recommended for PACU yet ending up in an ICU bed is reduced or even minimized.

[0143] FIG. 6 illustrates the selection of thresholds, showing a graph 500 in which along one axis a value of the lower threshold (th_IC) is set out, along another axis a value of the upper threshold (th_PA), and along yet another axis a percentage of false negative predictions (fn) (i.e., a type 2 error). In the graph 500, a surface is shown which, for different combinations of thresholds, indicates the percentage of false negative predictions, with the surface being color-coded using the value of the one-in performance metric (one_in) for the combination of th_IC and th_PA. In this example, best results are achieved with th_IC=0.49 and th_PA=0.76, which keeps the percentage of false negatives below 10% (corresponding to <0.1 in the graph) and obtains the best one-in performance, meaning the lowest misclassification of an PACU bed when it should be an ICU bed.

[0144] FIG. 7 shows an error matrix 204 for the prediction of the postoperative bed type by the hybrid model as discussed above. It can be seen that for 29% of the patients a PACU bed is predicted by the hybrid model, and a PACU bed is sufficient after surgery, and that for 50% of the patients an ICU bed is predicted by the hybrid model, and an ICU bed is indeed needed after surgery. However, there are also erroneous predictions. Namely, for 11% of the patients a PACU bed is predicted by the hybrid model, but an ICU bed is actually needed, and for 10% of the patients the hybrid model predicts an ICU bed, but a PACU bed is sufficient after surgery. According to the aforementioned one-in metric, for approximately 1 in 10 patients, a PACU bed was predicted even though the patient ended up in an ICU bed, which is a significant improvement over the 1-in-5 score of the clinician recommendation and 1-in-7 score of the prediction by the predictive model alone. Such improved performance is also demonstrated in Table 2.

TABLE-US-00002 TABLE 2 Comparison of clinician's and hybrid models performance Metric Clinician's recommendation Hybrid model Accuracy 0.77 0.79 Recall (Sensitivity) 0.89 0.75 Precision (PPV) 0.64 0.72

[0145] In alternative embodiments of the hybrid model, both thresholds may be set to the same value and thereby represent one and the same threshold. This way, the optimization may be reduced in complexity as one dimension is removed from the solution space. However, then the clinician's recommendation is excluded as well. In other embodiments, one of both thresholds may be fixed to a respective endpoint of the probability scale. For example, the lower threshold may be set to the lower endpoint of the probability scale while different values for the upper threshold may be evaluated. In this case, the output of the hybrid model will be either PACU if probability>th_PA or the clinician's recommendation otherwise. Another example is that the upper threshold may be set to the upper endpoint of the probability scale while different values for the lower threshold may be evaluated. In this case, the output of the hybrid model will be either ICU if probability<th_IC or the clinician's recommendation otherwise. Also here, the optimization may be reduced in complexity as one dimension is removed from the solution space. In addition, such embodiments may take into account that the clinician's recommendation may be better than that of the predictive model at one end of the scale.

[0146] With continued reference to the features used during the training and subsequent use, such features may be extracted from the medical data, for example using univariate and/or multivariate inferential analysis. Various types of features may be extracted. For example, some types of features may characterize the surgery while other types of features may characterize the patient. Examples of the former type of features are given in Table 3 below, while examples of the latter type of features are given in Table 4 below. It is noted that these features may be considered as potentially relevant features for the prediction of the postoperative bed type in the field of cardiothoracic surgeries, but that in other fields, e.g., in other surgical disciplines, other features may be of relevance. It is further noted that typically, all features are available before the surgery is performed, e.g., at the point of surgery planning, and typically even before the patient is hospitalized.

TABLE-US-00003 TABLE 3 Features based on surgery characteristics Features Examples of values Surgeon ID unclustered ID_6791, ID_86 clustered Hierarchical clustering - ID_C1, ID_C2 Surgeon type categorical Attendings vs. Residents Surgery Procedures unclustered CABG, AVR, etc. (Px) clustered According to: medical subspecialties Hierarchical clustering - Px_C1, Px_C2 Ave Px time per surgeon 183 min, 316 min Number of Px categorical Single, double, multiple (>=3) discrete 1, 2, 3, 4, 5, . . . Surgery urgency categorical Acute, Elective Post OR type of bed categorical ICU, PACU, general ward (GW)

TABLE-US-00004 TABLE 4 Features based on patient characteristics Features Examples of values Age discrete 67 years, 86 years categorical [18-29], [30-44], [45-69], . . . Gender categorical Male vs. Female BMI categorical Underweight, Normal, Overweight, Obese ASA score discrete 1, 2, 3, 4, 5 Medication discrete 1, 2, . . . , 22 numbers categorical [1, 5], [6-10], [11-15], [16++] Creatinine levels categorical renal failure, severe decrease, moderate decrease, mild decrease, normal

[0147] FIGS. 8A and 8B each show an example of a graphical user interface 600 of a system which uses the hybrid model to recommend a postoperative bed type. In both figures, the graphical user interface 600 comprises various graphical elements, such as an element 610 which shows information of a patient for which surgery is planned, such as the name, age, gender, number of medications, etc., and an element 620 which shows information about the upcoming surgery, such as the procedure code, procedure description, number of procedures to be performed, surgeon, etc. Some of the information shown in these elements may also be used as feature input to the hybrid model. In element 630, an output of the predictive model is shown, showing the probability scale, the thresholds of the hybrid model (0.4 and 0.75) and the output of the predictive model for this patient and surgery. In the example of FIG. 8A, this probability score is 0.9 which exceeds the hybrid model's upper threshold of 0.75. As such, the hybrid model may follow the predictive model's recommendation, as depicted in element 640, namely a PACU bed is recommended to the user. In some embodiments, the graphical user interface may further indicate the most important predictors for this recommendation, being in this example the age of the patient, the e-GFR (CDK-EPI) value (estimated Glomerular Filtration Rate) and the number of procedures to be carried out during the surgery. In the example of FIG. 8B, the bed prediction probability score is 0.6, which is below the upper threshold of 0.75 but above the lower threshold of 0.4. As such, the hybrid model refers to a recommendation by a clinician. This recommendation may for example be obtained by the system by prompting the user to click on a button 644, which may open an input field in which the clinician's recommendation can be entered or in which the bed type can be chosen, e.g., from PACU or ICU as options. In other examples, the expert's recommendation may already be available to the system or hybrid model and may be output to the user instead of the predictive model's prediction.

[0148] It will be appreciated that any method described in this specification may be implemented on a computer as a computer implemented method, as dedicated hardware, or as a combination of both. As also illustrated in FIG. 9, instructions for a computer, e.g., in the form of executable code, may be stored on a computer readable medium 700, e.g., in the form of a series 710 of machine-readable physical marks and/or as a series of elements having different electrical, e.g., magnetic, or optical properties or values. The executable code may be stored in a transitory or non-transitory manner. Examples of computer readable mediums include memory devices, optical storage devices, integrated circuits, servers, online software, etc. FIG. 9 shows a memory device 700.

[0149] In accordance with an abstract of the present application, a system and method are provided for generating a predictive model for predicting a postoperative bed type to be used by a patient after surgery. The predictive model is trained on features extracted from medical data and using a postoperative bed type as prediction target in the training The predictive model is configured to output a probability on a scale which corresponds to, at its lower end, a prediction of a first postoperative bed type and, at its upper end, a prediction of a second postoperative bed. A hybrid model is generated which applies a lower threshold and an upper threshold to the probability scale. If the output probability of the predictive model is in between both thresholds, an expert selection of the bed type is recommended, while otherwise, the prediction of the predictive model is output. The values of the thresholds are optimized using a performance metric.

[0150] Examples, embodiments or optional features, whether indicated as non-limiting or not, are not to be understood as limiting the invention as claimed.

[0151] It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim Use of the verb comprise and its conjugations does not exclude the presence of elements or stages other than those stated in a claim. The article a or an preceding an element does not exclude the presence of a plurality of such elements. Expressions such as at least one of when preceding a list or group of elements represent a selection of all or of any subset of elements from the list or group. For example, the expression, at least one of A, B, and C should be understood as including only A, only B, only C, both A and B, both A and C, both B and C, or all of A, B, and C. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the device claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.