System and Method for Generating Training Images

Abstract

The present invention relates to training data sets and a system and method for generating training images especially those which are medical images. Especially disclosed is a method of training a machine learning model to recognize movement of a body part in an acquired medical image The machine learning model is trained by varying/modifying a blur convolution kernel constructed with pixels oriented in a direction of the movement; the method including determining at least one motion weighting factor corresponding to a motion time period when the body part is moving during acquisition of the medical image, and using the motion weighting factor to vary/modify the blur convolution kernel.

Claims

1. A method of training a machine learning model to recognise movement of a body part in an acquired medical image by modifying a blur convolution kernel constructed with pixels oriented in a direction of the movement; the method includes determining at least one motion weighting factor corresponding to a motion time period when the body part is moving during acquisition of the medical image, and using the motion weighting factor to vary/modify the blur convolution kernel.

2-4. (canceled)

5. The method of training a machine learning model according to claim 1 including scaling the motion weighting factor directly proportional to an imaging resolution measure in length per pixel in a line of the pixels.

6. The method of training the machine learning model according to claim 1 including determining a first static weighting factor corresponding to a first static time period when the body part is motionless during acquisition of the medical image and using the static weighting factor to vary the blur convolution kernel.

7. The method of training the machine learning model according to claim 6 including scaling the static weighting factor proportional to the static time period and inversely proportional to an imaging time period during which the medical image is acquired.

8. The method of training the machine learning model according to claim 1 including constructing the blur convolution kernel as an array in which each one of the pixels is associated with a respective kernel value.

9. (canceled)

10. The method of training the machine learning model according to claim 1 including convolving a sharp image with the blur convolutional kernel oriented in the direction of movement to generate a simulated motion blur training image.

11-15. (canceled)

16. The method of training the machine learning model according to claim 6 to train the model to recognize the movement as motion characterised by the static time period followed by the motion time period including using the static weighting factor to modify a kernel value of at least one pixel on a first portion of the kernel.

17. The method of training the machine learning model according to claim 16 including using the static weighting factor to modify the kernel value of only the first pixel in the kernel.

18. (canceled)

19. The method of training the machine learning model according to claim 17 including using the motion weighting factor to modify a kernel value of at least one pixel on a second portion of the kernel following the first portion.

20. The method of training the machine learning method according to claim 19 including adding a first proportion of the motion weighting factor to the pixels on the first portion of the kernel.

21. The method of training the machine learning method according to claim 20 wherein the first proportion is equal to the fraction of ti m e during acquisition of the image that the body part is motionless; and a second proportion of the motion weighting factor added to pixels on a second portion of the line is larger than the first proportion.

22. (canceled)

23. The method of training the machine learning model according to claim 6 to train the model to recognize the movement to be continuing when the image acquisition time ends, including scaling the at least one of the static weighting factors to be equal to the final motion weighting factor.

24. The method of training the machine learning model according to claim 6 including determining a second static weighting factor corresponding to a second static time period when the body part is motionless during acquisition of the medical image, in which the first static time period leads to the motion time period which leads to the second static time period, and using the second static weighting factor to vary the blur convolution kernel.

25. The method of training the machine learning model according to claim 24 including using the motion weighting factor to modify respective kernel values of the first pixel and last pixel.

26. The method of training the machine learning model according to claim 24 including scaling the first static weighting factor to be greater than or equal to half the motion weighting factor and scaling the second static weighting factor to be greater than or equal to half the motion weighting factor.

27. (canceled)

28. The method of training the machine learning model according to claim 1 wherein at least one motion weighting factor corresponds to a first motion time period when the body part is accelerating from a first speed to a second speed, and scaling a kernel value of a pixel associated with the time period immediately before the first motion period to be greater than a kernel value of a pixel associated with the first motion time period.

29. The method of training the machine learning model according to claim 28 including scaling the kernel value of a pixel associated with the first motion time period to be greater than a kernel value of a pixel associated with the time period immediately after the first motion period.

30. A system designed to train a machine learning model to recognise movement of a body part in an acquired medical image comprising: a convolution engine to train the machine learning model; a convolution kernel constructor to construct and modify a blur convolution kernel constructed with pixels oriented in a direction of the movement; and a weighting factor calculator to determine at least one motion weighting factor corresponding to a motion time period when the body part is moving during acquisition of the medical image, wherein the convolution kernel constructor is configured to use the motion weighting factor to modify the blur convolution kernel.

31. The system designed to train the machine learning model according to claim 30 comprising a digital image memory device with an interface with the convolution engine to store and transfer digital data representing a sharp image to be convolved with the pixels by the convolution engine.

32-33. (canceled)

34. The system designed to train the machine learning model according to claim 30, wherein the convolution engine comprises a digital data processor hardwired and programmed to convolve digital data representing the sharp image with the pixels by the convolution engine.

Description

BRIEF DESCRIPTION OF THE FIGURES

[0044] FIG. 1 shows constant movement throughout the image acquisition process;

[0045] FIG. 2 shows stepped movement during part of the image acquisition process;

[0046] FIG. 3 shows an example of a temporally weighted convolution kernel;

[0047] FIG. 4 shows examples of straight-line constant weight, temporally weighted, and temporally weighted with acceleration convolution kernels; and

[0048] FIG. 5 shows a temporally weighted motion kernel with acceleration.

DETAILED DESCRIPTION OF THE INVENTION

[0049] In an illustrative embodiment movement of body parts during imaging results in motion blur in 2D medical images.

[0050] For example, a constant movement model assumes body part movement is constant throughout the image acquisition process, as illustrated by FIG. 1.

[0051] A stepped movement where the body part is moving during part of the image acquisition time and is stationary for remainder of the image acquisition time is illustrated by FIG. 2.

[0052] This is modelled by varying blur convolution kernel with two or more different weighting factors including, static weighting factors corresponding to each respective stationary or ‘static’ period, and motion weighting factors corresponding to each respective motion period, as depicted in FIG. 3 and FIG. 4 (middle).

[0053] With the additional aspect of modelling the acceleration and deceleration of the body part, the convolution kernel becomes smoothly varying between weighted regions, as depicted in FIG. 4 (bottom) and FIG. 5.

[0054] In FIG. 3 and FIG. 5. kernel values are on the ordinate axis. The abscissa is labelled the kernel pixel. The blur convolution kernel comprises array of each kernel value paired with each respective kernel pixel. The figures show a weighted motion blur vector field comprising different convolution kernel values for each corresponding part of the image.

[0055] A first example explains a method of modelling movement of one or more body parts. In the example an imaging time is between 1.0 and 2.0 seconds. During the imaging time, a body part starts as stationary then the body moves and is stationary again For the purposes of the illustrative example, imaging time is 1.5 seconds, the body part is stationary for 0.3 seconds, the body part moves 0.7 mm over 0.6 seconds, and the body part is then stationary for 0.6 seconds. Motion blur can be modelled by determining a blur convolution kernel as follows.

[00001]$\mathrm{Imaging}\mathrm{resolution}:0.07\mathrm{mm}\mathrm{per}\mathrm{pixel}$$\mathrm{Static}\mathrm{weighting}\mathrm{factor}1=\mathrm{stationary}\mathrm{time}/\mathrm{imaging}\mathrm{time}=0.3s/15s=0.2$$\mathrm{Motion}\mathrm{weighting}\mathrm{factor}=\mathrm{motion}\mathrm{time}/\mathrm{imaging}\mathrm{time}/\mathrm{motion}\mathrm{distance}*\mathrm{imaging}\mathrm{resolution}$$=0.6s/1.5s/0.7\mathrm{mm}*0.07\mathrm{mm}=0.04$$\mathrm{Static}\mathrm{weighting}\mathrm{factor}2=\mathrm{stationary}\mathrm{time}/\mathrm{imaging}\mathrm{time}=0.5s/1.5s=0.4$

[0056] Kernel Construction:

First pixel kernel value=(Static weighting factor 1)+0.5−(Motion weighting factor)=0.22

Motion pixels kernel values=Motion weighting factor=0.04

Last pixel kernel value={Static weighting factor 2}+0.5−(Motion weighting factor)=0.42

[0057] A temporally weighted blur convolution kernel in which each kernel pixel is assigned a pixel weight according to the kernel construction of the first example above is illustrated in FIG. 3. This is an example of a weighted motion blur convolution kernel comprising different kernel values for each corresponding part of the kernel. The Kernel value is 0.22 for the First pixel which is labeled as Kernel pixel 0. The Kernel value is 0.04 for the Motion pixels labeled as Kernel pixels ‘1’ to ‘9’. The Kernel value is 0.42 for the Last pixel which is labeled as Kernel pixel ‘10’. The motion blur convolution kernel values sum to 1.

[0058] As FIG. 3 shows the method results in a weighted motion blur vector field comprising different convolution kernel values for each corresponding part of the image. There are static pails with respective first and last kernel pixels having respective first and last kernel values. There is a motion part with respective motion kernel pixels having respective motion kernel values.

[0059] For random initialization of kernel values, integration of the Motion weighting factor into first and last pixels can be assumed as long as the first static weighting factor is greater than or equal to half the motion weighting factor and the second or last static weighting factor is also greater than or equal to half the motion weighting factor. Expressed mathematically: (Static weighting factor 1)>=0.5 Motion weighting factor, and (Static weighting factor 2) >=0.5−Motion weighting factor

[0060] With reference to FIG. 3, the weighted motion blur kernel has two static weighting factors corresponding to stationary periods, separated by a series of movement weighting factors corresponding to the time the body part was in motion.

[0061] With reference to FIG. 4, examples are shown of straight-line constant weight and temporally weighted blur convolution kernels. Whereas FIG. 3 shows kernel values as bar heights for each pixel number of the blur convolution kernel, FIG. 4 shows the kernel valuess for each pixel according to a shaded scale.

[0062] In FIG. 4 Top Shaded Scale there is an illustration of a constantly weighted convolution kernel that approximates constant motion throughout the imaging time. The motion time equals the imaging time. All the pixels are motion pixels. A second example showing determination of kernel values for the motion pixels is as follows:

[0063] For the purposes of the illustrative example, imaging time is 1.0s, modelling constant movement of a body part moving 0.7 mm over the 1.0s.

[00002]$\mathrm{Imaging}\mathrm{resolution}:0.07\mathrm{mm}\mathrm{per}\mathrm{pixel}$$\mathrm{Motion}\mathrm{weight}\mathrm{factor}:=\mathrm{motion}\mathrm{time}/\mathrm{imaging}\mathrm{time}/\mathrm{motion}\mathrm{distance}*\mathrm{imaging}\mathrm{resolution}$$=1.s/1.s/0.7\mathrm{mm}*0.07\mathrm{mm}=0.1$$\mathrm{Motion}\mathrm{pixels}\mathrm{kernel}\mathrm{values}=\mathrm{Motion}\mathrm{weighting}\mathrm{factor}=0.1$

[0064] In FIG. 4 Middle Shaded Scale there is an illustration of the temporally weighted blur convolution kernel that has three weighted regions previously described by the first example above and illustrated wt FIG. 3. Single pixels at the left and right of the kernel correspond with time spent stationary and have a higher weight than the pixels in between, which correspond to the time the body part is in motion.

[0065] In FIG. 4 Bottom Shaded Scale there is an illustration of a temporally weighted blur convolution kernel that models a combination of stationary and moving phases, with the addition of acceleration and deceleration at the start and end of motion, resulting in gradual weight changes between stationary and motion phases. The temporally weighted convolution kernel which is illustrated in the FIG. 4 Bottom Shaded Scale is determined as explained by a third example below.

[0066] In the third illustrative example, a simple time step acceleration model (using the Euler method) is used to model a constant acceleration of 2 mm/s.sup.2 for a first interval of 0.4s, followed by constant motion for a second interval 0.6s and a constant deceleration of 2 mm/s.sup.7 for a third interval 0.4s along with 0.1s stationary at both the start and end of the movement. The simple acceleration model is used to generate a smoothly varying onvolution kernel as follows.

Imaging resolution 0.08 mm per pixel

Timing resolution=0.1s per time point

Acquisition time=1.7s

Acceleration (mm pers 5.sup.2)=[0, 0, 2, 2, 2, 2, 0, 0, 0, 0, 0, 0, −2, −2, −2, −2 0] mm/s.sup.2

Acceleration (pixels per 0.1s.sup.2)=[0, 0, 0.25, 0.25, 0.25, 0.25, 0, 0, 0, 0, 0.25, 0.25, 0.25, 0.25, 0]

Speed (pixels per 0.1s.sup.2)=[0.0, 0.0, 0.25, 0.5, 0.75, 1.0. 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.75. 0.5, 0.25, 0.0]

Location (pixel)=[0 0, 0.0, 0.25, 0.75, 1.5, 2.5, 3.5, 4.5, 5.5, 6.5, 7.5, 8.5, 9.25, 9.75, 10.0, 10.0]

Time Weighted Kernel (time at pixel per 0.1s)=[5, 1.5, 1, 1 1, 1, 1, 1, 1, 1.5, 5]

Scaled Kernel Value (normalized to sum to 1)=[0.25, 0, 0.75, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.075, 0.25]

[0067] FIG. 5 snows kernel values as bar heights for each pixel number of a convolution kernel whereas FIG. 4 Bottom Shaded shows the kernel values according to a shaded scale. So FIGS. 4 and 5 show that the kernel has two static weighted kernel values corresponding to respective stationary periods, separated by a series of movement weighted kernel values corresponding to the time the body part was in motion, including periods of acceleration and deceleration. The blur convolution kernel values sum to 1.

[0068] The invention has been described by way of examples only. Therefore, the foregoing is considered as illustrative only of the principles of the invention. Further, since numerous modifications and changes will readily occur to those skilled in the art, it is not desired to limit the invention to the exact construction and operation shown and described, and accordingly, all suitable modifications and equivalents may be resorted to, falling within the scope of the claims.