METHOD FOR AIDING AN OPERATOR TO DETERMINE A POSITION OF A REGION OF INTEREST IN A BREAST AND MAMMOGRAPHY APPARATUS IMPLEMENTING IT

Abstract

The method is for aiding an operator to determine a position of a region of interest in a breast so as to subsequently guide a biopsy procedure; the method is adapted to be implemented in a mammography apparatus; the method provides performing a 2D imaging procedure of the dual energy mammography type and generating a composite 2D image, and performing a 3D imaging procedure of the tomosynthesis type, firstly generating a 3D image and then a virtual 2D image; based, for example, on the composite 2D image and the virtual 2D image, a two-dimensional mapping and transformation are determined which allow the same marking elements to be accurately displayed on different images, in particular on a 2D section of the 3D image and on a recorded 2D image derived from the composite 2D image.

Claims

1. Method for aiding an operator to determine a position of a region of interest in a breast so as to subsequently guide a biopsy procedure in said breast, comprising the steps of: A) performing a 2D imaging procedure of said breast comprising a first delivery of X-rays from a predetermined emission point at a first energy to obtain a first image on a predetermined acquisition plane and a second delivery of X-rays from said predetermined emission point at a second energy to obtain a second image on said predetermined acquisition plane, said predetermined emission point and said predetermined acquisition plane being predetermined geometric parameters of said 2D imaging procedure; B) generating a composite 2D image consisting of a 2D matrix of pixels corresponding to a combination of said first image and said second image; C) performing a tomosynthesis-type 3D imaging procedure of said breast by means of delivering X-rays from various emission points at a third energy to obtain a 3D image consisting of a 3D voxel matrix; D) generating a virtual 2D image by means of a predetermined mapping of said 3D image on a virtual plane corresponding to said predetermined acquisition plane according to geometric mapping parameters corresponding to said predetermined geometric parameters; E) determining a two-dimensional transformation that links a portion of said first image or said second image or said composite 2D image and a corresponding portion of said virtual 2D image in such a way as to record said composite 2D image with respect to said virtual 2D image and to generate a recorded 2D image on the basis of said two-dimensional transformation; and F) using said geometric parameters of said mapping to display in a section of said 3D image at least one marking point previously identified by the operator or by a computer program in said recorded 2D image or to display in said recorded 2D image at least one marking point previously identified by the operator or by a computer program in a section of said 3D image; wherein said 2D imaging procedure and said 3D imaging procedure are performed on a breast in a compressed state resulting from a single compression operation, and wherein a contrast medium is present.

2. Method according to claim 1, wherein said 2D imaging procedure is performed using an X-ray grid.

3. Method according to claim 1, wherein said first energy is lower than said second energy, and wherein said second energy and said third energy are equal.

4. Method according to claim 1, wherein said first energy is lower than said second energy, and wherein said second energy and said third energy are different and in particular selected according to an energy of a photon absorption peak by said contrast medium.

5. Method according to claim 4, wherein said third energy is lower than said second energy by 10-30%.

6. Method according to claim 1, comprising one or more repetitions of a 2D imaging procedure performance according to step A and a generation of a 2D composite image according to step B prior to performing said steps A, B, C, D, E, F.

7. Method according to claim 6, wherein a repeat is made if an estimation result on the reachability of the biopsy procedure is negative.

8. Method according to claim 6, wherein said step E is performed on the basis of the last first image obtained or last second image obtained or last composite 2D image generated.

9. Method according to claim 1, wherein said image portion used in step E is defined by said operator.

10. Method according to claim 1, wherein said image portion used in step E corresponds to an area of maximum reachability of the biopsy procedure.

11. Method according to claim 1, wherein in said step F the operator or the computer program identifies a marking area surrounded by a closed marking line.

12. Method according to claim 11, wherein in said step F said geometric parameters of said mapping are used to display said closed marking line.

13. Method according to claim 1, wherein said geometric parameters of said mapping are used to display points, lines or areas belonging to a volume calculated by a computer program in one or more sections of said 3D image or in said recorded 2D image.

14. Method according to claim 1, wherein said recorded 2D image or one or more sections of said 3D image as well as a marking point or a marking line or a marking area are displayed on a screen.

15. Method according to claim 1, wherein, in step D, said mapping comprises: defining a 2D matrix of pixels in said predetermined acquisition plane, creating a plurality of projection axes in said 3D image, wherein said projection axes pass respectively through the pixels of said 2D matrix and through said predetermined emission point, associating a corresponding projection volume of said 3D image which surrounds the projection axis to each of said projection axes, identifying for each projection volume the voxels of said 3D image inside the projection volume, for the pixels of said 2D matrix, choosing at least one function which defines an intensity value with respect to the intensity of the voxels of the corresponding projection volume.

16. Method according to claim 15, wherein said function is a combination of maximum or minimum or mean or median.

17. Method according to claim 1, wherein, in step E, said two-dimensional transformation is of the geometric type and comprises: translation or rotation or uniform scaling or non-uniform scaling or shear displacement or perspective displacement.

18. Method according to claim 1, wherein, in step E, said two-dimensional transformation is determined by preliminarily extracting features from said portion of said first image or said second image or said composite 2D image and from said portion of said virtual 2D image.

19. (canceled)

20. (canceled)

21. Method according to claim 1, wherein the step of using said geometric parameters of said mapping to display in the section of said 3D image comprises using at least one marking point previously identified by the operator and by the computer program in said recorded 2D image or to display in said recorded 2D image at least one marking point previously identified by the operator and by the computer program in a section of said 3D image.

22. Method according to claim 6, wherein each repetition corresponds to a preliminary compression operation distinct from said single compression operation.

Description

LIST OF FIGURES

[0020] The present invention shall become more readily apparent from the detailed description that follows to be considered together with the accompanying drawings in which:

[0021] FIG. 1 shows a block diagram of an embodiment of an apparatus according to the present invention,

[0022] FIG. 2 schematically shows the performance of a 3D tomosynthesis-type imaging procedure with the apparatus of FIG. 1,

[0023] FIG. 3 schematically shows how a virtual 2D image is generated by mapping a 3D tomosynthesis-type image with the apparatus of FIG. 1,

[0024] FIG. 4 shows a flow chart of an embodiment of the method according to the present invention,

[0025] FIG. 5 shows an example of a “low energy” image acquired during a 2D breast CESM imaging procedure,

[0026] FIG. 6 shows an example of a “high energy” image acquired during a 2D breast CESM imaging procedure,

[0027] FIG. 7 shows a “composite” image corresponding to a combination of the images in FIG. 5 and FIG. 6,

[0028] FIG. 8 shows an example of a “virtual” breast image obtained by mapping a 3D tomosynthesis-type image as shown in FIGS. 3, and

[0029] FIG. 9 shows the display, according to the present invention, of corresponding marking elements in various images.

[0030] As it can be easily understood, there are various ways of practically implementing the present invention which is defined in its main advantageous aspects in the appended claims and is not limited either to the following detailed description or to the appended claims.

DETAILED DESCRIPTION

[0031] The apparatus 1 of FIG. 1 comprises an X-ray emitter 2 and an X-ray detector 6 adapted to detect the X-rays emitted by emitter 2.

[0032] As it is clear, FIG. 1, FIG. 2 and FIG. 3 correspond to each other and (partially) show the same components as apparatus 1. FIG. 2 and FIG. 3 focus on certain aspects of the apparatus 1 and therefore some components (in particular the filter assembly 3 and the grid assembly 5) are not shown, but only for the sake of clarity.

[0033] In particular and as graphically highlighted, the detector 6 is of electronic type and integrates a two-dimensional array, or matrix, of elementary sensors, for example one per pixel, which defines an acquisition plane 61; in FIG. 1, FIG. 2 and FIG. 3, the plane 61 is shown as if it was external to the detector 6, on the contrary it is internal and may correspond to the top surface of the detector or to a parallel plane at a small distance (e.g. 0.3-3.0 mm). It should be noted that alternative electronic sensors integrating a one-dimensional array of elementary sensors, e.g., one per pixel, adapted to perform a translational image scanning motion, are known in the literature.

[0034] A first function of the apparatus 1 is therefore to perform breast imaging procedures; in FIG. 1, FIG. 2 and FIG. 3, a breast is schematised with an oval M in a position such to intercept the X-rays emitted by the emitter 2 before they hit the detector 6; obviously, part of the X-rays emitted by the emitter 2 are not intercepted by the breast M.

[0035] FIG. 1 shows that the emitter 2 may be moved, in particular rotated about an axis that lies in the acquisition plane 61 and is perpendicular to the plane of FIG. 1; this rotation may be better understood by looking at FIG. 2 and is used to perform 3D tomosynthesis-type imaging procedures. However, in general, other possible movements of the emitter 2, in particular translation and/or rotation, should not be excluded.

[0036] A second function of the apparatus 1 is therefore to perform breast biopsy procedures. In FIG. 1 a biopsy assembly 8 equipped with a suitable biopsy needle 81 and its withdrawal mechanism for inserting and withdrawing the needle to take the sample is schematically shown. In general, the assembly 8 can move in different ways, in particular translating (e.g., in one, two or three directions) and rotating (e.g., around one, two or three axes), in order to best fulfil its function.

[0037] The apparatus 1 further comprises a breast positioning assembly 4 conceptually consisting of an upper contact element 41 and a lower contact element 42 between which a breast M is compressed. In particular, the element 42 is fixed with respect to the detector 6 and the element 41 is movable with respect to the element 42 in that it can firstly move away to insert the breast M and then move closer by compressing the breast M. The element 41, called “compression plate” and hereinafter “compressor” for ease of brevity, may be adapted to receive electrical position control signals and/or transmit electrical measurement signals, such as force and/or position. In FIG. 1, the compressor 41 is equipped with several windows to allow the passage of a biopsy needle, in particular the needle 81; the figure shows for example three windows 43a, 43b and 43c, but the number could be different; moreover, it cannot be excluded that, thanks to the range of possible movements of the assembly 8, the needle 81 could penetrate the breast M from a side rather than from above.

[0038] In FIG. 1, two other possible components are shown, a filter assembly 3 and a grid assembly 5. The former, in combination with the emitter 2, is used to shape the spectrum of X-rays entering the breast; in particular, to perform DEM or CESM procedures, the “low energy”/“high energy” effect can be obtained by choosing the high (anodic) voltage to be supplied as input to the emitter 2 and the assembly 3 filter to be used for filtering. The second one is for improving the quality of the image detected by detector 6 as it reduces the contribution of scattering, which results in a general blurring of details.

[0039] No further details of the components mentioned above are provided herein because they are in themselves known in the field.

[0040] Two important components of the apparatus 1 shown in FIG. 1 need to be highlighted for the purposes of the present invention: a processing and control unit 7 and a human-machine interface unit 9.

[0041] Unit 7 is electrically connected to all the other components and is the one that basically determines the operation of apparatus 1. It is, in general, a computerised unit equipped with a system software and possibly an application.

[0042] Unit 9 is connected to unit 7 and allows the operator to provide “inputs”, e.g., data and/or commands, to the apparatus 1 in particular unit 7, and to receive “outputs” from the apparatus 1 in particular from unit 7. Typically, the unit 9 is a computerised unit equipped with an application software, in particular a PC, and comprises at least one screen, a keyboard and a mouse and/or joystick, as well as other devices for controlling, for example, the movements of the various components of the apparatus and the delivery of X-rays.

[0043] Typically, thanks to the appropriate software of the processing and control unit and/or thanks to the human-machine interface unit with relative appropriate software, it is possible to carry out the methods according to the present invention. The components of unit 7 and unit 9 are not shown in FIG. 1 for ease of simplicity. It is worth bearing in mind that FIG. 1 is highly schematic and therefore no deductions should be made about the position and composition of the processing and control unit.

[0044] FIG. 2 schematises the performance of a 3D tomosynthesis-type imaging procedure by means of the apparatus of FIG. 1 on the breast M (taking into account that in FIG. 2 the assembly 3 is not shown but is present as in FIG. 1); this figure assumes, by way of example, that the resulting 3D image is derived from five consecutive detections by the detector 6.

[0045] While the breast M is compressed between elements 41 and 42: [0046] the emitter 2 is positioned at position P−2 and delivers X-rays, for example, pointing to a central point of the detector 6, detector 6 detects a 2D image on the acquisition plane 61 and sends it to the unit 7, then [0047] the emitter 2 is positioned at position P−1 and delivers X-rays, for example, pointing to the central point of the detector 6, the detector 6 detects a 2D image on the acquisition plane 61 and sends it to the unit 7, then [0048] the emitter 2 is positioned at position P0 and delivers X-rays, for example, pointing to the central point of the detector 6, the detector 6 detects a 2D image on the acquisition plane 61 and sends it to the unit 7, then [0049] the emitter 2 is positioned at position P+1 and delivers X-rays, for example, pointing to the central point of the detector 6, the detector 6 detects a 2D image on the acquisition plane 61 and sends it to the unit 7, then [0050] the emitter 2 is positioned at position P+2 and delivers X-rays, for example pointing to the central point of the detector 6, the detector 6 detects a 2D image on the acquisition plane 61 and sends it to the unit 7, finally [0051] the unit 7 processes all the 2D images received and generates a 3D tomosynthesis-type image consisting of a three-dimensional matrix of voxels; [0052] the positions P−2, P−1, P0, P+1 and P+2 correspond to the positions of the “focal point” (it is more precisely a “focal spot”) of the emitter 2 and are ideally located on a circumference centred for example on the central point of the detector 6 in the acquisition plane 61, which we could call “acquisition centre”; the axis joining the “focal spot” and the “acquisition centre” is rotated, for the various positions, for example by −20°, −10°, 0°, +10° and +20° respectively.

[0053] Delivering X-rays on various occasions for acquiring the above 3D image is performed using a certain voltage (anodic), which is always the same, and with a certain filtering, which is always the same.

[0054] Rather than the point of the detector 6 in the acquisition plane 61, another point can be used that can be located for example on a central axis perpendicular to the detector 6 preferably between the acquisition plane 61 and the plane associated with the element 41.

[0055] Performing a 2D DEM or CESM imaging procedure with the apparatus of FIG. 1 on the breast M may still be described with the aid of FIG. 2, bearing in mind that in this figure the assemblies 3 and 5 are not shown but are present (as in FIG. 1). While the breast M is compressed between elements 41 and 42: [0056] the emitter 2 is positioned in position P0, the assembly 3 is set for a first type of filtering, the emitter 2 performs a first delivery of X-rays using a first voltage (anodic) and pointing for example (perpendicularly) to the central point of the detector 6, the detector 6 detects a first 2D image on the acquisition plane 61 which is for example filtered through the assembly 5 and sends it to unit 7—it can be for example a “low energy” image (for example 20 KeV), and then [0057] the emitter 2 remains in position P0, the assembly 3 is set for a second type of filtering, the emitter 2 makes a second X-ray delivery using a second (anodic) voltage and pointing for example (perpendicularly) to the central point of the detector 6, the detector 6 detects a second 2D image on the acquisition plane 61 which is for example filtered through the assembly 5 and sends it to the unit 7—this may be for example a “high energy” image (e.g. 36 KeV).

[0058] Typically, the unit 7 processes the two received 2D images and generates a composite 2D image comprising a 2D matrix of pixels corresponding to a combination of the first image and the second image.

[0059] FIG. 3 will be considered hereinafter in detail, at the moment it is sufficient to note that in this figure the emitter 2 is in the same position P0 as used for the 2D DEM or CESM imaging procedure, that the breast M is compressed between elements 41 and 42 as for the 2D DEM or CESM imaging procedure, that the detector 6 (and its acquisition plane 61) is in the same position P0 as used for the 2D DEM or CESM imaging procedure. The breast M is subdivided into voxels as from a 3D image resulting from a previous tomosynthesis 3D imaging procedure: in particular, the various “sections” of the 3D image are highlighted—all horizontally aligned voxels placed at the same distance from the acquisition plane 61 belong to a “section”; they are therefore, slices with a small thickness (e.g. 0.5-1.0 mm) corresponding to one side of the voxel. In FIG. 3, we see the cone or pyramid of X-rays created by the emitter 2 (typically the emitter is associated with a “collimator” adapted to determine the shape and size of the cone or pyramid) which is intercepted by the breast M, conceptually subdivided into voxels, and which reaches the pixels of the detector 6.

[0060] The method according to the present invention will be illustrated hereinafter with the aid of FIGS. 1 to 3 as well as the flowchart 400 in FIG. 4; the references in brackets are for exemplary purposes only. Diagram 400 shows a method start block 410 and a method end block 490 with possible starting and final activities.

[0061] The method according to the present invention is for aiding an operator to determine a spatial position of a region of interest in a breast (M) in order to subsequently guide a biopsy procedure in the breast itself, for example, to firstly move appropriately the biopsy assembly (8) and then to withdraw tissue by inserting and extracting the biopsy needle (81).

[0062] In general, the method comprises the steps of: [0063] A) performing a 2D breast imaging procedure (M); this step is subdivided into two sub-steps; a first sub-step (block 420) involves a first delivery of X-rays from a predetermined emission point (P0) at a first energy to obtain a first image (see for example image 1001 in FIG. 5), in the form of a 2D matrix of pixels, on a predetermined acquisition plane (61); a second sub-step (block 430) involves a second delivery of X-rays from the predetermined emission point (P0) at a second energy to obtain a second image (see for example image 1002 in FIG. 6), in the form of a 2D matrix of pixels, on the predetermined acquisition plane (61); the emission point (P0) and the acquisition plane (61) are to be considered as predetermined geometric parameters of the 2D imaging procedure; [0064] B) generating a composite 2D image (see for example image 1003 in FIG. 7) (block 440) consisting of a 2D matrix of pixels corresponding to a combination of the first image (1001) and the second image (1002); [0065] C) performing a 3D tomosynthesis-type imaging procedure (block 450) of the breast M by delivering X-rays from various emission points (P−2, P−1, P0, P+1, P+2 [0066] see for example FIG. 2) at a third energy to obtain a 3D image consisting of a 3D matrix of voxels (see for example FIG. 3); [0067] D) generating a virtual 2D image (see image 1004 in FIG. 8) (block 460) by means of a predetermined mapping of the 3D image onto a virtual plane corresponding to the predetermined acquisition plane (61) according to geometric mapping parameters corresponding to the predetermined geometric parameters of the 2D imaging procedure referred to in step A—this step is schematically shown in FIG. 3 and will be better explained hereinafter, but, to get an idea of step D, one can think of the virtual 2D image as corresponding, for example, to a projection of the 3D image onto the acquisition plane; [0068] E) determining a two-dimensional transformation (block 470) linking a portion of the composite 2D image (1003) and a corresponding portion of the virtual 2D image (1004) in such a way as to record the composite 2D image (1003) with respect to the virtual 2D image (1004) and generating a recorded 2D image (block 480) based on this two-dimensional transformation—this step will be better explained hereinafter, but by way of example, it can be disclosed here that its function is to take into account that external and/or internal micro-movements of the breast under examination certainly occurred between the moment when the two images of the dual energy mammography in step A were obtained and the moment when the images of the tomosynthesis in step C were obtained,—it should be noted that, alternatively, rather than using the composite 2D image of step B to determine the transformation, either the first image or the second image of the 2D imaging procedure of step A can be used; e [0069] F) using the geometric parameters of the mapping to display (block 490) at least one marking point; this point has been previously located manually by an operator or automatically by a computer program or by an operator with the aid of a program; this will typically also be highlighted; there are in fact two possibilities: either the point is first located in a section of the 3D image and then displayed in the 2D (dual energy) recorded image or the point is first located in the 2D (dual energy) recorded image and then displayed in a section of the 3D image.

[0070] It should be noted that the 2D imaging procedure and the 3D imaging procedure are performed on a breast in a compressed state resulting from a single compression operation, and wherein a contrast medium is present. Such breast preparation may be considered, for example, as an “initial activity” and may fall within block 410 of FIG. 4.

[0071] It can be understood from the above that a way has been found to determine a precise relationship (combination of the geometric parametrics of mapping and geometric transformation) between the 2D mammographic dual-energy image and the 3D tomosynthetic image; such a relation passes through the virtual 2D image. The virtual 2D image can be thought of as a “virtual” mammography that is not performed on a physical breast, but on a 3D reconstruction of the breast obtained by tomosynthesis.

[0072] Once this precise relationship has been determined, it is possible to switch precisely from dual energy mammography to tomosynthesis and vice versa; in particular, to display corresponding points and to know their spatial position, i.e., their coordinates.

[0073] This is useful when preparing for a biopsy to ensure that the biopsy needle withdraws a sample of tissue exactly at the point of the breast deemed as most relevant by the operator. Thus, the marking point referred to in step F of the method is conceptually related to the position of interest.

[0074] Suppose, for example, that a hypothetical lesion has been identified in the recorded dual-energy mammography (this could derive from an operator observing it) and there is interest in identifying the lesion in the tomosynthesis and its spatial position; the operator may create, for example, a marking point on the 2D mammography image by selecting the point with a mouse (which will be highlighted); then, the operator can view one or more of the sections of the 3D tomosynthetic image and, thanks to the method according to the present invention, the marking point is automatically displayed in such sections; finally, the operator can evaluate if such marking point is suitable and signal it to the apparatus as a point to be reached with the biopsy.

[0075] From this example, it can be understood that the operator may be interested in operating (i.e., identifying marking points such as selecting points and/or moving points that have already been selected with the mouse) in the mammographic image and/or tomosynthetic section images. According to preferred embodiments of the present invention, both things are possible.

[0076] The aid method according to the present invention may provide displaying images on one or more screens; for example, it may be provided that a mammographic image (with possible marking elements) and an image of a tomosynthetic section (with possible marking elements) are displayed simultaneously on one or two screens or alternately on the same screen.

[0077] Typically, according to the present invention, steps A-F are performed in the order indicated above; indeed, it is very advantageous to perform mammography before tomosynthesis. However, changes of order are not excluded; for example, step B could be performed after step C.

[0078] It should be considered that the core of the present invention corresponds to steps D-F. In other words, according to certain embodiments, the aid method according to the present invention may start from a 2D mammographic dual energy image already acquired and from a 3D tomosynthetic image already acquired, under the assumption that they refer to the same breast in the same position, that is, without macro-movements of the breast as it always remains compressed.

[0079] According to the present invention, as is common practice, the 2D imaging procedure and the 3D imaging procedure are performed on a breast in a compressed state. Advantageously, according to the present invention, the 2D imaging procedure and the 3D imaging procedure (at least those used for steps E and F of the method) are performed on a breast in a compressed state resulting from a single compression operation.

[0080] Advantageously, according to the present invention, the 2D imaging procedure and the 3D imaging procedure (at least those used for steps E and F of the method) are performed on a breast in which a contrast medium is present. In particular, a dose of a contrast medium may be injected only once before the 2D imaging procedure because the timing of steps A, B and C is such that the contrast medium remains in circulation and both can benefit (even if steps A and B are repeated two or three times).

[0081] Preferably, according to the present invention, the 2D imaging procedure is carried out with the use of an X-ray grid, which is then removed before carrying out the 3D imaging procedure.

[0082] The method according to the present invention advantageously provides one or more repetitions of a performance of a 2D imaging procedure according to step A and a generation of a 2D composite image according to step B (i.e., repetitions of one or more dual energy mammographies) prior to performing steps A, B, C, D, E, F. Each of these repetitions typically derives from a specific request of an operator and is typically preceded by a preliminary operation of repositioning the breast with a subsequent preliminary operation of compressing the breast (which is distinct from the only compression operation mentioned above and on the basis of which steps D and E and F of the method are carried out).

[0083] It is useful to carry out a repetition if it is estimated that the biopsy procedure will not be positively reached. For example, an operator might look at a mammographic image and identify a hypothetical lesion that should be biopsied; then the operator, possibly assisted by a computer program, might estimate whether it is reachable, taking into account for example the estimated spatial position of the breast, the biopsy assembly and the position(s) of the access/withdrawal window(s) of the compressor; if this estimate is negative, i.e. the lesion is not reachable, the breast must be repositioned and the dual energy mammography must be repeated. According to the present document, as is common practice, the second energy mentioned in step A is greater than the first energy mentioned in step A; conceptually, it is irrelevant in a dual-energy mammography to first perform a “low energy” acquisition and then a “high energy” acquisition or vice versa; in practice, it may be preferable to start with the “low energy” acquisition (which also implies a lower total energy in the detector) to avoid “ghost” effects particularly evident in some types of X-ray detectors.

[0084] Preferably, according to the present invention, the third energy mentioned in step C is equal to or about equal to the second energy mentioned in step A, i.e., “high energy”; for example, the third energy may be lower than the second energy by, for example, 10-30%. In the case of different energies, the tomosynthesis imaging is performed at an energy such that there is a peak absorption of photons by the contrast medium, neglecting the optimisation of the visibility of the surrounding tissues (in contrast to conventional tomosynthesis). In this case, it may be appropriate to determine the transformation in step E of the method using, for example, the “high energy” image instead of the composite image.

[0085] In explaining step E of the method, reference was made to portions of the images to determine the transformation; this allows for instance to exclude irrelevant parts of the images (such as the “background”) and/or to take in consideration only parts that are highly relevant for the biopsy. The portion of the image used in step E can be defined by the operator by, for example, drawing the outline of this portion on the screen. Alternatively, the portion of the image used in step E may be defined by a computer program and may correspond, for example, to an area of maximum reachability of the biopsy procedure, in particular enclosed by the profile of an access/withdrawal window of the compressor displayed on the screen.

[0086] In case of dual energy mammographies being repeated, it is preferable that the transformation referred to in step E of the method is determined on the basis of a 2D image deriving from the last performance of these two steps prior to step E, in particular the last composite 2D image generated or the last “high energy” 2D image acquired; in this way, there will be a greater relation between the 2D image and the 3D image.

[0087] Previously, when generally describing the method according to the present invention, reference was made to “at least one marking point”. However, it is typical and advantageous of the present invention that in step F the operator and/or the computer program identifies (and highlights) a marking area surrounded by a closed marking line. In this case, the display can be focused on, for example, and be limited to the closed marking line.

[0088] According to the present invention, it is possible and advantageous that geometric mapping parametrics are used to display in one or more sections of the recorded 3D image and/or in the 2D image points, lines or areas belonging to a volume calculated by a computer program; this may be useful, for example, to display the (simulated) passage cylinder of the biopsy needle.

[0089] Thus, in general according to the present invention, it is possible to display on a screen the recorded 2D image and/or one or more sections of the 3D image as well as a marking point or a marking line or a marking area.

[0090] We will now consider more in detail image processing.

[0091] In step D, mapping may include: [0092] defining a 2D matrix of pixels in the predetermined acquisition plane, [0093] creating a plurality of projection axes in the 3D image, wherein these projection axes pass respectively through the pixels of the 2D matrix and through the predetermined emission point, [0094] associating a corresponding projection volume of the 3D image which surrounds the projection axis to each of the projection axes, wherein said projection volume is preferably cone- or pyramid-shaped, [0095] identifying for each projection volume the voxels of the 3D image that are internal (or at least partly internal) to the projection volume, [0096] for the pixels of the 2D matrix, choosing at least one function which defines an intensity value with respect to the intensity of the voxels of the corresponding projection volume.

[0097] Such a function may be “binary” (i.e., defined over two values) or “numeric” (i.e., defined over a discrete range of values) or a mixture of “numeric” and “binary” elements—since processing according to the present invention will be carried out by a computer, strictly speaking, one cannot speak of a “continuous range of values”.

[0098] This function may be a combination (e.g., a linear combination) of maximum and/or minimum and/or mean and/or median.

[0099] In step E, the transformation may be of the geometric type; in particular, it may comprise: translation and/or rotation and/or uniform scaling and/or non-uniform scaling and/or shear deformation and/or perspective deformation (also called homographic transformation which is much preferred).

[0100] In step E, the transformation may be determined by a preliminary feature extraction from a portion of the reference 2D image (in particular the composite 2D image or the “high energy” 2D image) and from the corresponding portion of the virtual 2D image; consider for example the “high energy” image 1002 in FIG. 6 and the composite image 1003 in FIG. 7 and virtual image 1004 in FIG. 8.

[0101] The algorithm that allows the transformation to be achieved is complex and typically includes: a step of extracting the relevant characteristics or “features” from the two images, a step of creating the couplings for the features extracted from the two images and a step of identifying the geometric transformation that adapts one image to the other according to the couplings created.

[0102] Two aspects of the algorithm need to be considered with particular care during practical implementation: the choice of a number of “features” to be extracted from the two images and the constraints to be imposed on the “feature” couplings in order to filter out false matches. As the number of “features” increases, if the couplings are sufficiently reliable according to the constraints imposed, the accuracy with which the transformation is identified increases, but so does the time required to calculate the couplings.

[0103] According to the present invention, a “feature” extraction method of the ORB (=Oriented Fast and Rotated Brief) type is preferably employed in which a maximum number of “features” equal to about 1% of the total pixels composing the image under examination is preferably set, which is preferably segmented with respect to the background of the image (typically, therefore the background pixels are not considered); in order to reduce the calculation time, this limit could also be reduced for example to 0.1%. As already mentioned, extracting the features from an image under examination may only concern a portion of the image.

[0104] The ORB method basically consists of a combination of two algorithms called FAST (=Features from Accelerated Segment Test), which deals with “feature” extraction and BRIEF (=Binary Robust Independent Elementary Feature), which deals with the description of each individual feature. The FAST algorithm is based on the detection of contrast differences for each pixel in the image to which a “feature” will eventually be associated, and also allows features of different shapes and sizes to be identified thanks to a multi-scale analysis of the image (FAST can be iterated several times, each time sub-sampling the original image according to a geometric pyramid model). The BRIEF algorithm assigns a string of bits to each identified feature based on, for example, the shape and size of the feature, which constitutes the feature descriptor; the advantage of this bit-string representation is that feature descriptors can be processed more efficiently during the coupling process.

[0105] In order to determine the matches between the identified features, a matching method may advantageously be used that compares the descriptors (in pairs and in such a way that there is always a descriptor for each of the two images) for all the features identified in the two images and creates a coupling based, as a quantitative evaluation criterion, on the distance measure between the descriptors; preferably, the Hamming distance between two bit strings (which encode the descriptors) is calculated.

[0106] These methods are generally applicable to the analysis of any images and, therefore, the resulting matches do not take into account specific aspects related to, for example, the typical size of the lesion of interest and/or the typical size of the anatomical structures involved and/or their positioning. Therefore, according to the present invention, which is in the specific field of breast imaging, it is advantageous to provide for an additional filtering step which eliminates all obtained couplings not satisfying a number of spatial constraints. Among them, preferably an upper limit is set to the (Euclidean) distance for pairs of corresponding features (i.e. coupled according to the previous calculations) within the area of the breast under consideration in order to restrict the analysis only to consistent displacements with respect to the real movement of breast tissues during compression; this constraint can also preferably be obtained by estimating a weighted value (e.g. average displacement) from the analysis of the distribution of dislocations for pairs of corresponding features identified in the area of the breast under consideration. Feature pairs that do not fulfil the imposed constraint criteria are then discarded from the analysis, which leads to obtaining the transformation, ensuring a greater reliability.

[0107] Once the corresponding feature pairs of interest have been defined, in order to obtain the transformation used for recording, two vectors, usually called “source” and “target”, are constructed, which contain the points with the two-dimensional coordinates of the features belonging respectively to the source image (e.g., the composite 2D image or a portion thereof—see image 1003 of FIG. 7) and the target image (e.g. the virtual 2D image or a portion thereof—see image 1004 of FIG. 8), ordered in such a way that the coupling matches are respected according to the vector index. Based on these two vectors, it is possible to derive a matrix that describes a transformation between the source image and the target image. This matrix can be used to transform the composite image and obtain a recorded composite image. It should be noted that the transformation could also be applied to only a portion of the composite image, e.g., only to the portion used to calculate the transformation; in this case, a resulting image would be obtained that partly corresponds to the original composite image and partly corresponds to the recorded composite image (a smoothing filter could be applied to the border).

[0108] FIGS. 5 to 8 relate to the same breast and are examples of images useful to understand the present invention. FIG. 5 and FIG. 6 show respectively a “low energy” and a “high energy” image captured during a 2D CESM imaging procedure; FIG. 7 shows a 2D “composite” image corresponding to a combination of the images in FIG. 5 and FIG. 6—this image highlights a hypothetical lesion. FIG. 8 shows a “virtual” 2D image obtained by mapping a 3D tomosynthesis-type image onto the acquisition plane. In these images, the outline of the breast can be seen, as well as a darker frame due to a lower attenuation of the X-rays where they do not intercept the compressor at its access/withdrawal window for the biopsy needle (in case of access from a side, the compressor may not have any opening, but a simulated outline defining a zone of maximum reach for the needle may still be displayed). The accuracy purpose of the present invention is equal to or less than the accuracy imposed by the International Standard IEC 60601-2-45 regarding biopsy pointing, i.e., ±1 mm. This accuracy leads to recording the image in FIG. 7 with respect to the image in FIG. 8, although visually (given the scale of these two images) this would not seem necessary; it should be noted that displacements greater than 1 mm are unlikely for a breast under compression.

[0109] FIG. 9 shows the display of marking elements according to the present invention; the top image (FIG. 9A) is a recorded “composite” 2D image and the images (FIG. 9B to FIG. 9F) below are related to (e.g. five) 2D sections of a 3D tomosynthetic image at different distances from the acquisition plane (as it can be understood from FIG. 3); the term “sections” is used even if they are slices of a small thickness (e.g. 0.5-1.0 mm).

[0110] In the image above (FIG. 9A), a radiologist has drawn a marking element; for example, a pentagon has been drawn around a hypothetical lesion. In all the images below (FIG. 9B to FIG. 9F), corresponding marking elements were displayed (automatically) by the method according to the present invention; it is noted that in some sections (FIG. 9B, FIG. 9C and FIG. 9D) within the marking element there is a figure, more or less clear, which corresponds to the hypothetical lesion, and that in other sections (FIG. 9E and FIG. 9F) there is nothing within the marking element corresponding to the hypothetical lesion, or rather suggesting a lesion. The radiologist (or a computer program) chooses the most suitable section for “biopsy pointing” based for example on the portion of the image within the marking element; for example, he or she might choose the section with the clearest figure (FIG. 9C). At this point, the radiologist can perform the “biopsy pointing” by typically identifying a point on the chosen section (FIG. 9C) as a “biopsy target”; consequently, a cross (centred on the point) is for example displayed in the section and subsequently (typically after an imperceptible time), a cross is for example displayed in the “composite” image by the method according to the present invention (note the cross both in FIG. 9A in FIG. 9C); alternatively or additionally, the outline of the section of a needle that will be used for sampling could be displayed. Different hatches and/or colours can help the operator distinguish between the markings he/she made directly from those produced automatically by the computer program. Once the radiologist is satisfied with the pointing he/she has carried out by the method according to the present invention, he/she can confirm the selection of the coordinates to the computer, which will then convert them into coordinates for the pointing system of the biopsy assembly so as to proceed with the biopsy examination.

[0111] The method according to the present invention is specifically adapted to be implemented in a mammography apparatus.

[0112] For example, the apparatus 1 in FIG. 1 includes means to implement this method. In particular, the processing and control unit and the human-machine interface unit are the essential means for implementing the method.

[0113] The core of the method for aiding an operator according to the present invention described above consists of steps D, E and F.

[0114] It can thus be defined as a computer-aided method which is carried out entirely by a computer electronic unit (such as a PC) and which comprises only steps D, E and F (and possibly also step B) and which operates on images stored in advance in a memory, e.g., an internal electronic or magnetic or optical memory or an external electronic or magnetic or optical memory. In this case, steps A and C (and possibly also step B) wherein there is (electromagnetic) interaction with a breast are carried out in advance (e.g., a few seconds before, or even less).