TECHNIQUE FOR DETERMINING AN INDICATION OF A MEDICAL CONDITION
20230154614 · 2023-05-18
Inventors
Cpc classification
G01R33/5608
PHYSICS
G06F18/254
PHYSICS
G16H50/20
PHYSICS
A61B6/501
HUMAN NECESSITIES
G06V10/87
PHYSICS
G06F18/285
PHYSICS
A61B6/5217
HUMAN NECESSITIES
G16H50/30
PHYSICS
G16H50/70
PHYSICS
G06V10/809
PHYSICS
International classification
G16H50/20
PHYSICS
Abstract
A medical data processing technique for determining an indication of a medical condition is disclosed. A method implementation of the technique comprises selecting (202), based on at least one property associated with medical data of a test instance, at least one model out of a plurality of models, wherein each of the plurality of models is generated by a learning algorithm and configured to provide a model-specific indication of the medical condition, determining (204), using each of the at least one selected model, a respective model-specific indication, and determining (206), based on the model-specific indications, the indication of the medical condition.
Claims
1-15. (canceled)
16. A medical data processing method for determining an indication of a medical condition, the method comprising: selecting, based on at least one property associated with medical data of a test instance, at least two models out of a plurality of models, wherein each of the plurality of models is generated by a learning algorithm and configured to provide a model-specific indication of the medical condition based on the medical data; determining, using each of the selected models, a respective model-specific indication of the medical condition based on the medical data; and determining, based on the model-specific indications, the indication of the medical condition.
17. The method of claim 16, wherein the at least one property associated with the medical data comprises a feature of a medical image comprised in the medical data.
18. The method of claim 16, wherein the at least one property associated with the medical data comprises a characteristic of a patient to which the medical data relates.
19. The method of claim 16, wherein the step of selecting comprises comparing, individually for each of the plurality of models, the at least one property associated with the medical data of the test instance and at least one property associated with training data used for generating the individual model.
20. The method of claim 19, wherein the indication is determined further based on at least one attribute chosen from a result of the comparing, empirical performances of each of the plurality of models and a degree of explainability of each of the plurality of models.
21. The method of claim 16, wherein at least one of the models comprised in the plurality of models is generated by an unsupervised learning algorithm using unlabeled training data of healthy patients and, optionally, configured to provide an anomaly detection as the model-specific indication of the medical condition.
22. The method of claim 16, wherein the model-specific indication of the medical condition and/or the indication of the medical condition comprises at least one result chosen from probabilities of an anomaly for different parts of a medical image comprised in the medical data and a numerical value describing a probability of an anomaly of the overall medical data, wherein the numerical value is optionally derived from the probabilities of the anomaly for the different parts of the medical image.
23. The method of claim 16, further comprising determining that a reliable determination of the indication is impossible, if the at least one property associated with the medical data of the test instance does not indicate suitability of the at least two models.
24. The method of claim 16, wherein at least one of the models comprised in the plurality of models correlates parts of a medical image comprised in the medical data with parts of a reference image, and compares an image value of at least one part of the medical image with information associated with a correlated part of the reference image to obtain the model-specific indication of the medical condition, wherein the information has been generated by: matching a plurality of training images to a base image to correlate parts of each of the training images with parts of the base image; determining image values of at least one part of each of the plurality of training images correlated with a part of the base image, wherein the part of the base image is assigned to the correlated part of the reference image using a predetermined transformation; and determining the information based on the determined image values of the at least one part of each of the plurality of training images.
25. The method of claim 24, wherein the information comprises or is a statistical distribution function of image values of the at least one part of the plurality of training images.
26. The method of claim 24, wherein the information comprises an average image value of the at least one part of all of the plurality of training images and, optionally, a mean deviation of the image values of the at least one part of all of the plurality of training images from the average image value.
27. A medical data processing method for determining an indication of a medical condition, the method comprising: correlating parts of a medical image comprised in medical data of a test instance with parts of a reference image; and comparing an image value of at least one part of the medical image with information associated with a correlated part of the reference image to obtain the indication of the medical condition, wherein the information has been generated by: matching a plurality of training images to a base image to correlate parts of each of the training images with parts of the base image; determining image values of at least one part of each of the plurality of training images correlated with a part of the base image, wherein the part of the base image is assigned to the correlated part of the reference image using a predetermined transformation; and determining the information based on the determined image values of the at least one part of each of the plurality of training images, wherein the information is a statistical distribution function of image values of the at least one part of the plurality of training images.
28. An apparatus comprising at least one processor and at least one memory, the at least one memory containing instructions executable by the at least one processor such that the apparatus unit is operable to perform the method of claim 16.
29. A computer program product comprising program code portions for performing the method of claim 16 when the computer program product is executed on one or more processors.
30. The computer program product of claim 29, stored on one or more computer readable recording media.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0046] Further details and advantages of the technique presented herein will be described with reference to exemplary implementations illustrated in the figures, in which:
[0047]
[0048]
[0049]
[0050]
[0051]
[0052]
[0053]
[0054]
[0055]
DETAILED DESCRIPTION
[0056] In the following description, for purposes of explanation and not limitation, specific details are set forth in order to provide a thorough understanding of the present disclosure. It will be apparent to one skilled in the art that the present disclosure may be practiced in other implementations that depart from these specific details.
[0057] Those skilled in the art will further appreciate that the steps, services and functions explained herein below may be implemented using individual hardware circuitry, using software functioning in conjunction with a programmed micro-processor or general-purpose computer, using one or more Application Specific Integrated Circuits (ASICs) and/or using one or more Digital Signal Processors (DSPs). It will also be appreciated that when the present disclosure is described in terms of a method, it may also be embodied in one or more processors and one or more memories coupled to the one or more processors, wherein the one or more memories are encoded with one or more programs that perform the steps, services and functions disclosed herein when executed by the one or more processors.
[0058]
[0059]
[0060] The method further comprises a step 204 of determining, using (e.g., only) each of the at least one selected model, a respective model-specific indication of the medical condition based on the medical data.
[0061] The method also comprises a step 206 of determining (e.g., by or using the aggregator described herein), based on the model-specific indications, the (e.g., final) indication of the medical condition.
[0062]
[0063] Training data, such as medical images of a plurality of patients, for example, may be used by a learning algorithm, also referred to herein below as base learner, to generate one of the base learner models. A learner configuration (“config”) file may define the type and/or structure of the base learner model. The learner configuration file may comprise at least one hyperparameter defined in or used by the (e.g., unlearned) base learner model. The learner configuration file may define the learning algorithm used to generate the base learner model and, optionally, hyperparameters defined in or used by the learning algorithm. Some preferred examples of base learners and base learner models will be described in detail with reference to
[0064]
[0065]
[0066]
[0067] A test instance, in particular medical data of a test instance, may be used as input for a selector which may select the at least one model from the plurality of base learner models BLS 1 to BLS k. The selector may correspond to the selector described above with reference to the method of the first aspect. The selector may be defined by a selector configuration (“config”) file. The selected models may then be used to determine the respective model-specific indications. In the shown example, the base learner model BLS 2 and the base learner model BLS k may be selected from the plurality of base learner models BLS 1 to BLS k. In this example, only the selected based learner models BLS 2 and BLS k may be used to determine the respective model-specific indications, as indicated in
[0068] Based on the model-specific indications, the (e.g., final) indication of the medical condition may be determined, for example by or using an aggregator. The aggregator may correspond to the aggregator described above with reference to the method of the first aspect. The aggregator may determine or provide the (e.g., final) indication based on the determined model-specific indications, the model characteristics, test instance metadata and features extracted from the test instance. The aggregator may be defined by an aggregator configuration (“config”) file. Note that the aggregator may not necessarily be a hardware component but may be embodied in software. The aggregator configuration file may define how the inputs to the aggregator are combined into the output “O”, namely, into the indication of the medical condition. The aggregator may comprise or consist of the aggregation model described herein above. The aggregation model may be defined by the aggregator configuration file. Not all of the inputs shown in
[0069] As mentioned with reference to
[0070] The step of selecting may comprise comparing, individually for each of the plurality of models, the at least one property associated with the medical data of the test instance and at least one property associated with the training data used for generating the individual model. The comparing may result in a degree of suitability for each of the plurality of models. For example, models that have been trained with training images having a low slice thickness may have a low degree of suitability when compared with a higher slice thickness of the medical image of the test instance. Models that have been trained with training images having a certain contrast may have a low degree of suitability when compared with a contrast of the medical image of the test instance which deviates from the certain contrast more than a predefined threshold. Also, suitability of a base learner model that has been trained with training data of patients above the age of 50 may be low for medical data of the test instance which relates to a 5-year-old patient. As a further example, suitability of a base learner model that has been trained only with training data of Asian patients may be low for medical data of the test instance which relates to a patient of another ethnicity such as an African or a European patient. Also, suitability of a base learner model that has been trained only with training data of female patients may be low for medical data of the test instance which relates to a male patient.
[0071] The method may further comprise determining that a reliable determination of the indication is impossible, if the at least one property associated with the medical data of the test instance does not indicate suitability of the at least one model. In this case, a notification such as a warning tone or a visual warning may be triggered to be output on the output device, e.g., by sending a corresponding trigger signal via the interface 106 to the output device.
[0072] The indication may be determined (e.g., by or using the aggregator) based on a majority vote of the model-specific indications or based on a mean aggregation for continuous numerical model-specific indications. Other aggregation functions are possible. For example, in case the indication shall be very sensitive, the indication may be that the patient to which the medical data of the test instance relates is unhealthy, if one or more of the model-specific indications indicate such unhealthiness.
[0073] The indication may be determined further based on at least one of attribute chosen from a result of the comparing, empirical performances (e.g., comprised in the model characteristics) of each of the plurality of models and a degree of explainability (e.g., comprised in the model characteristics) of each of the plurality of models. The degree of explainability may be determined based on the learner configuration file or predetermined and indicates how well a user is able to understand the functioning of the base learner (model). The degree of explainability may be obtained from a database.
[0074] For example, only model-specific indications may be taken into account (e.g., used or considered) for determining the (e.g., final) indication, attributes of which fulfil a predetermined criterion. In case no attribute fulfils the predetermined criterion, it may be determined that a reliable determination of the indication is impossible and a corresponding notification may be output as noted above.
[0075] Alternatively, a weighted average of the model-specific indications may be determined as the (e.g., final) indication of the medical condition. The weights of the respective model-specific indications may be determined based on the attribute, in particular based on the empirical performances. Weights w_i (e.g., that sum up to 1) may be defined for each of the learner models BLS1 to BLS k and proportional to the base learner's empirical performance. For example, w_i may be proportional to exp(p_i), wherein p_i may be the empirical performance of base learner model i.
[0076] Still alternatively, the (e.g., final) indication may be determined using an aggregation model, e.g., the aggregation model described herein above. In particular, the aggregator may comprise or consist of the aggregation model described herein above and/or be generated by a supervised learning algorithm, for example. The supervised learning algorithm may be trained with a set of determined model-specific indications, and optionally, a set of model characteristics, test instance metadata and/or extracted features and the generated aggregation model may be capable of providing the indication of the medical condition based on the inputs of the “aggregator”. The aggregation model learning algorithm may be defined by the aggregator configuration file, for example.
[0077] The indication of the medical condition may be used to trigger output of a notification on the output device, the notification informing the user of the indication. For example, the interface 106 may output a trigger signal to a display which then displays a visualization of areas of the medical image comprised in the medical data of the test instance, which areas exhibit the medical condition. Alternatively, a score or a binary visualization such as a color red or green may be displayed informing the user of the indication of the medical condition. In one example, the indication itself may be visualized on the display. Alternatively, or additionally, a medical diagnosis may be determined based on the indication of the medical condition. For example, if the indication of the medical condition represents an anomalous volume in a medical image, further properties of the anomalous volume such as image values, color values, volume size or else may be taken into account (e.g., used or considered) for determining the diagnosis, e.g., the diagnosis that the patient to which the medical data relates has a brain tumor.
[0078] As will be apparent for the skilled person, other ways of determining the (e.g., final) indication based on the model-specific indications may be possible. In the following, examples of base learners and base learner models will be described.
[0079] Generally, supervised learning algorithms may need labeled training data. A label may indicate whether a certain medical condition or a set of medical conditions applies to the training data. A supervised learning algorithm may then be trained using the labeled training data in order to be able to predict, when applied to new data which has not been used for training, a label as the model-specific indication. At least one of the base learner models BLS 1 to BLS k may in one variant be generated by a supervised learning algorithm using labeled training data (e.g., of at least one healthy patient and of at least one unhealthy patient with a disease).
[0080] Especially in medical diagnostic tasks, the structure of the labels to be predicted may be hierarchical. As shown in
[0081] If a model is generated by a supervised learning algorithm using labeled training data, the model may only be able to provide (e.g., determine) a model-specific indication of whether a test instance is associated with one of the labels for which the model has been trained. In this case, a determination of a model-specific indication of the medical condition associated with the medical data of the test instance, the medical condition being “healthy” or “disease”, may be possible only if the training data represents balanced amounts of all possible diseases D1 to DZ that could potentially be found. In other words, labeled data of each possible subclass D1 to DZ may need to be available for training the supervised learning algorithm to enable a correct prediction of a lower-level label of the subclass (highlighted subclass “D2” in
[0082] A reliable machine-based indication of the medical condition “healthy” or “anomalous” (or “disease”) may allow a doctor to prioritize cases and allocate time appropriately even before having a look at the examination. As noted above, the method described herein may further comprise determining a medical diagnosis based on the indication of the medical condition, thereby improving a medical workflow for the doctor.
[0083] In view of the above, a method of determining the model-specific indication of the medical condition may be provided, as schematically illustrated in
[0084] The method comprises a step 902 of correlating parts of a (e.g., the) medical image comprised in (e.g., the) medical data of a (e.g., the) test instance with parts of a reference image, for example, using the first registration described above.
[0085] The method also comprises a step 904 of comparing an image value of at least one part of the medical image with information associated with a correlated part of the reference image to obtain the indication of the medical condition.
[0086] The information has been generated, e.g., by the different apparatus, by: matching a plurality of training images to a base image (e.g., using the second registration described above) to correlate parts of each of the training images with parts of the base image, determining image values of at least one part of each of the plurality of training images correlated with a part of the base image, wherein the part of the base image is assigned to the correlated part of the reference image using a predetermined transformation (e.g., the third registration described above), and determining the information based on the determined image values of the at least one part of each of the plurality of training images.
[0087] The base image may be an atlas image generated based on a plurality of medical images. The base image may be determined by applying a rigid registration of a plurality of medical images to a common frame of reference and averaging image values of all these medical images. By applying the rigid registration, each part of each of the patient images may be correlated with the common frame of reference. That is, the rigid registration may comprise a transformation matrix describing a transformation of a coordinate system of a patient image into a common coordinate system. Different rigid registrations may be used for different ones of the plurality of medical images. The base image may be an MR atlas image. CT images of the training data may be matched to the MR atlas image using mattes-mutual-information. In one variant, each of the learning images may be first rigidly and affinely transformed to the base image before the SyN non-linear transformation may be applied. Each of the aforementioned registration steps may be iteratively repeated until a convergence or a maximum number of iteration is reached. Also, each of these steps may be performed on different levels of resolution, starting with a low resolution and then going to higher resolutions, to obtain a coarse registration which is then improved to a fine-grained registration.
[0088] A dataset of a plurality of 3D CT images (dimensions m×n×p) of healthy patients may be mapped as the training images to the base image which may be a CT volume. This mapping may be performed by applying a registration (e.g., the second registration described above) between each of the plurality of 3D CT images of healthy patients to the base image. By applying the registration, each part of each of the patient images may be correlated with parts of the base image. That is, the registration may comprise a transformation matrix describing a transformation of a coordinate system of a patient image into a coordinate system of the base image and/or a plurality of transformations of different parts of the patient image into the coordinate system of the base image. Of course, different registrations may be used for different patient images. The registration may be determined using a symmetric diffeomorphic image registration with cross-correlation and a SyN algorithm comprised in the Advanced Normalisation Tools library available on https://github.com/ANTsX/ANTs, for example.
[0089] In a first implementation, the information may comprise or be a statistical distribution function of image values of the at least one part of the plurality of training images. The information may have been determined as follows: For each voxel position x.sub.a,b,c with 1<=a<=m, 1<=b<=n, 1<=c<=p, a statistical distribution function may be fitted over all voxel values of all training images. The distribution may be a Gaussian distribution or a kernel density distribution (KDE), for example. This may result in m×n×p distribution functions p.sub.a,b,c that have been independently estimated based on the learning images.
[0090] The base learner model of the first implementation may use the medical data to determine or provide the model-specific indication of the medical condition as follows: When the model-specific indication of the medical condition is to be determined based on the medical image, the medical image may first be registered to the reference image which is registered to the base image with a predetermined transformation or which is the base image. If a voxel value of the medical image is higher than the upper q/2 quantile or lower than the lower q/2 quantile as defined by the corresponding distribution function, this voxel may be tagged or identified as anomalous. A percentile filter may be used to smooth the results, and thresholding may be used to obtain a binary segmentation mask. The binary segmentation mask may indicate parts of the medical image of the test instance which are anomalous. The binary segmentation mask may be determined as the model-specific indication. An overall anomaly score for the medical image of the test instance may be determined, e.g., as the model-specific indication, by counting the number of anomalous voxels in the medical image or by dividing this number by the total number of voxels of the medical image.
[0091] In a second implementation, the information may comprise an average image value of the at least one part of all of the plurality of training images and, optionally, a mean deviation of the image values of the at least one part of all of the plurality of training images from the average image value. The information may have been determined as follows: For each voxel position x.sub.a,b,c with 1<=a<=m, 1<=b<=n, 1<=c<=p, a voxel value may be obtained of all training images. For each voxel position, an average voxel value of all training images may be determined. In other words, the training images may be averaged voxel-wise. Beforehand, the training images may be normalized so that all voxel values lie in a predetermined range (e.g., [0, 1]). In addition, a mean error map may be computed by calculating an average over the voxel-wise difference of all training images to the average voxel values.
[0092] The base learner model of the second implementation may use the medical data to determine or provide the model-specific indication of the medical condition as follows: When the model-specific indication of the medical condition is to be determined based on the medical image, the medical image is first registered to the reference image which is registered to the base image with a predetermined transformation or which is the base image. The absolute difference of the medical image to the average voxel values may then be determined per voxel. The mean error map may be subtracted from the absolute differences. A percentile filter may be used to smooth the results, and thresholding may be used to obtain a binary segmentation mask. The binary segmentation mask may indicate parts of the medical image of the test instance which are anomalous. The binary segmentation mask may be determined as the model-specific indication. An overall anomaly score for the medical image of the test instance may be determined, e.g., as the model-specific indication, by counting the number of anomalous voxels in the medical image or by dividing this number by the total number of voxels of the medical image.
[0093] The method of
[0094] As explained above, the model-specific indication of the medical condition and/or the (e.g., final) indication of the medical condition may comprise at least one result chosen from probabilities of an anomaly for different parts of a medical image comprised in the medical data and a numerical value describing a probability of an anomaly of the overall medical data, wherein the numerical value is optionally derived from the probabilities of the anomaly for the different parts of the medical image.
[0095] As has become apparent from the above, the present disclosure provides a technique for determining an indication of a medical condition. At least one of the base learner models may be generated using unlabeled training data of healthy patients. Such a base learner model may provide (e.g., determine) a model-specific indication of a medical condition, which may be a score indicating whether a new, prior unseen test instance is in- or out-of-distribution compared to the distribution of the training data. Accordingly, (e.g., only) training data of healthy patients, which is easily obtainable, may be used to generate the at least one model.
[0096] A plurality of the base learner models may represent an ensemble. A (most suitable) subset of a plurality of base learner models may be selected based on the test instance at hand. In other words, models to be included in the ensemble may be dynamically selected based on the medical data. This may enable a more reliable determination of the model-specific indications. Model-specific indications of the selected base learner models may be aggregated to determine the indication of the medical condition. The aggregation may further improve reliability of the prediction (i.e., of the determined indication).
[0097] Especially if the base learners' errors are uncorrelated, combining them in an ensemble may enhance robustness. Further, model-specific indications may be combined in such a way that explainable base learners are preferred over non-explainable deep learning models. This may result in higher interpretability of the model-specific indications. False negative predictions may be avoided by the selection of the at least one base learner model and the determination of the indication based on the model-specific indications.
[0098] By using learning data of a higher-level “normal” class (that has no or only a few subclasses, e.g., learning data of only healthy patients), a robust, reliable indication of whether the medical data of the test instance belongs to an in-distribution-instance (“normal” or “healthy” instance) or not may be possible. Through the coupling mechanism (rule-based or learned) by the proposed ensemble, in particular, by determining the indication based on the model-specific indications, an advanced predictive performance, as well as increased robustness of the predictions may be ensured. The techniques described herein may be applicable to any type of diagnostic procedure, and may provide a prediction with respect to the higher-level classes without having to use training data with a detailed sub-class labeling, and even without having to use training data of out-of-distribution samples.
[0099] It is believed that the advantages of the technique presented herein will be fully understood from the foregoing description, and it will be apparent that various changes may be made in the form, constructions and arrangement of the exemplary aspects thereof without departing from the scope of the disclosure or without sacrificing all of its advantageous effects. Because the technique presented herein can be varied in many ways, it will be recognized that the disclosure should be limited only by the scope of the claims that follow.