2D-3D IMAGE REGISTRATION METHOD AND MEDICAL OPERATING ROBOT SYSTEM FOR PERFORMING THE SAME

Abstract

An image registration method and apparatus, a medical operating robot system using the same, and a computer program medium are described. The image registration method includes: extracting digitally reconstructed radiograph (DRR) images in an anterior-posterior (AP) direction and a lateral-lateral (LL) direction from the 3D image; acquiring 2D images for an AP image and an LL image of the patient's surgical; determining a first rotation angle between a reference position and a predetermined first reference position of the patient's surgical site corresponding to the first reference position of the AP image or LL image; determining a second rotation angle between the reference position and the second reference position of the AP image or LL image corresponding to the reference position; and determining a transformation relationship between the 2D image and the DRR image based on the first and second rotation angles.

Claims

1. An image registration method, steps of which are performed by an image registration apparatus comprising a processor, the method comprising: acquiring a 3D image of a patient's surgical site from a 3D imaging apparatus before an operation; extracting digitally reconstructed radiograph (DRR) images in an anterior-posterior (AP) direction and a lateral-lateral (LL) direction from the 3D image; acquiring 2D images for an AP image and an LL image of the patient's surgical site from a 2D imaging apparatus during an operation; determining a first rotation angle between a reference position and a predetermined first reference position of the patient's surgical site corresponding to the first reference position of the AP image or LL image, based on a first rotation axis passing through a predetermined first origin and parallel to a cross product vector of first normal vectors for planes of the AP image and the LL image, from a geospatial relationship between a source and a detector with respect to the DRR image; determining a second rotation angle between the reference position and the second reference position of the AP image or LL image corresponding to the reference position, based on a second rotation axis passing through a predetermined second origin and parallel to a cross product vector of second normal vectors for planes of the AP image and the LL image, from a geospatial relationship between a source and a detector with respect to the 2D image; and determining a transformation relationship between the 2D image and the DRR image based on the first and second rotation angles, from the geospatial relationships between the sources and the detectors of the DRR and 2D images.

2. The method of claim 1, wherein the first reference position and the second reference position comprises a center of the AP image or LL image for each of the 2D image and the DRR image, or a line or plane comprising the center.

3. The method of claim 1, further comprising performing operation planning based on the 3D image by the image registration apparatus, wherein the first origin for the DRR image is determined based on a relative relationship of a trajectory of a surgical instrument for mounting an implant or a mounting position of the implant applied to the operation planning.

4. The method of claim 1, wherein the reference position for the DRR image or 2D image is determined based on a user's input.

5. The method of claim 1, wherein the geospatial relationship between the source and the detector for the DRR image comprises an orthogonal projection relationship, and the geospatial relationship between the source and the detector for the 2D image comprises a perspective projection relationship.

6. The method of claim 1, further comprising: determining, by the image registration apparatus, a first volume of interest where planes intersect as the plane of the AP image and the plane of the LL image are moved in directions of the first normal vectors, from the geospatial relationship between the source and the detector for the DDR image; and determining, by the image registration apparatus, a second volume of interest where planes intersect as the AP image and the LL image are moved in directions of the second normal vectors within a perspective projection range, wherein the geospatial relationship between the source and the detector for the 2D image comprises a perspective projection relationship.

7. The method of claim 6, wherein the first origin comprises a center of the first volume of interest, and the second origin comprises a center of the second volume of interest.

8. The method of claim 1, further comprising: determining, by the image registration apparatus, a first region of interest for each of the AP image and LL image of the DRR image; and determining, by the image registration apparatus, a second region of interest corresponding to the first region of interest for each of the AP image and LL image of the 2D image, wherein the first reference position is positioned within the first region of interest, and the second reference position is positioned within the second region of interest.

9. The method of claim 8, further comprising: determining, by the image registration apparatus, a first volume of interest where planes intersect as a region of interest on the AP image and a region of interest on the LL image are moved in directions of the first normal vectors, from the geospatial relationship between the source and the detector for the DDR image; and determining, by the image registration apparatus, a second volume of interest where planes intersect as a region of interest on the AP image and a region of interest on the LL image are moved in directions of the second normal vectors within a perspective projection relationship, wherein the geospatial relationship between the source and the detector for the 2D image comprises a perspective projection relationship.

10. The method of claim 9, wherein the first origin comprises a center of the first volume of interest, and the second origin comprises a center of the second volume of interest.

11. The method of claim 9, wherein the first origin comprises a center between target positions of a patient's spine pedicle screws.

12. The method of claim 11, wherein the first rotation angle comprises an angle formed between a line segment that connects the first origin and a midpoint between the pedicle screw entry points, and the first normal vector that passes through the center of the first volume of interest, with respect to the first origin.

13. The method of claim 8, wherein each first region of interest for the AP image and LL image of the DRR image comprises a rectangle, and regarding the DRR image, the method comprising: a first step of calculating, by the image registration apparatus, first intersection points between an epipolar line on the LL image for the vertices of the region of interest on the AP image and a midline connecting midpoints of an outer circumference or lateral sides of a region of interest on the LL image; a second step of acquiring, by the image registration apparatus, four reconstructed points by orthogonal projection of the first intersection points to the normal vectors from the vertices of the region of interest on the AP image; a third step of calculating, by the image registration apparatus, second intersection points between an epipolar line on the AP image for the vertices of the region of interest on the LL image and a midline connecting midpoints of an outer circumference or lateral sides of a region of interest on the AP image; a fourth step of acquiring, by the image registration apparatus, four reconstructed points by orthogonal projection of the second intersection points to the normal vectors from the vertices of the region of interest on the LL image; and a fifth step of calculating, by the image registration apparatus, a first volume of interest in a hexahedron formed based on eight reconstructed points obtained through the first to fourth steps.

14. The method of claim 8, wherein the determining the second volume of interest comprises, regarding the 2D image, a first step of calculating, by the image registration apparatus, first intersection points between an epipolar line on the LL image for the vertices of the region of interest on the AP image and a midline connecting midpoints of an outer circumference or lateral sides of a region of interest on the LL image; a second step of acquiring, by the image registration apparatus, four reconstructed points by perspective projection of the first intersection points to perspective projection vector from the vertices of the region of interest on the AP image toward the source; a third step of calculating, by the image registration apparatus, second intersection points between an epipolar line on the AP image for the vertices of the region of interest on the LL image and a midline connecting midpoints of an outer circumference or lateral sides of a region of interest on the AP image; a fourth step of acquiring, by the image registration apparatus, four reconstructed points by perspective projection of the second intersection points to the perspective projection vectors from the vertices of the region of interest on the LL image toward the source; and a fifth step of calculating, by the image registration apparatus, a second volume of interest in a hexahedron based on eight reconstructed points obtained through the first to fourth steps.

15. An image registration method, steps of which are performed by an image registration apparatus comprising a processor, the method comprising: acquiring a 3D image of a patient's surgical site from a 3D imaging apparatus before an operation; extracting digitally reconstructed radiograph (DRR) images in an anterior-posterior (AP) direction and a lateral-lateral (LL) direction from the 3D image; acquiring 2D images for an AP image and an LL image of the patient's surgical site from a 2D imaging apparatus during an operation; determining a first region of interest for each of the AP image and the LL image of the DRR image; determining a second region of interest corresponding to the first region of interest with respect to each of the AP image and the LL image of the 2D image; determining a first volume of interest formed by intersection of planes upon parallel translation of a region of interest on the AP image and a region of interest on the LL image in a direction of a first normal vector to the planes of the AP image and the LL image, from a geospatial relationship between a source and a detector with respect to the DRR image; determining a second volume of interest formed by intersection of planes upon translation of a region of interest on the AP image and a region of interest on the LL image in a direction of a second normal vector to the AP image and the LL image of the 2D image within a perspective projection range, wherein the geospatial relationship between the source and the detector for the 2D image comprises a perspective projection relationship; determining first displacement between a first reference position within the first volume of interest corresponding to a predetermined first reference position in the first region of interest and a predetermined reference position corresponding to the first reference position; determining second displacement between the reference position and a second reference position within the second volume of interest for a predetermined second reference position within the second region of interest corresponding to the reference position; and determining a transformation relationship to minimize a Euclidean distance between vertices of the first region of interest and vertices of the second region of interest based on a transformation relationship, as the transformation relationship between the 2D image and the DRR image is determined from geospatial relationships for the source and the detector of each of the DRR image and the 2D image, based on the first displacement and the second displacement.

16. The method of claim 15, wherein the determining the first displacement comprises determining a first rotation angle based on an angle between the reference position and the first reference position, with respect to a first rotation axis passing through a predetermined first origin and parallel to a cross product vector of the first normal vectors for planes of the AP image and the LL image; and the determining the second displacement comprises determining a second rotation angle based on an angle between the reference position and the second reference position, with respect to a second rotation axis passing through a predetermined second origin and parallel to a cross product vector of the second normal vectors for planes of the AP image and the LL image.

17. The method of claim 15, wherein the determining the first and second volumes of interest comprise forming a polyhedron by projecting an epipolar line of vertices of the first and second regions of interest to the first and second normal vectors.

18. An image registration apparatus comprising a processor to perform the image registration method based on claim 1.

19. A medical operating robot system comprising: a 2D imaging apparatus configured to acquire a 2D image of a patient's surgical site during an operation; a robot arm comprising an end effector to which a surgical instrument is detachably coupled; a position sensor configured to detect a real-time position of the surgical instrument or the end effector; a controller configured to control the robot arm based on predetermined operation planning; a display; and a navigation system configured to display the planning information about the surgical instrument or implant on a 2D image acquired during an operation or display the real-time position of the surgical instrument or implant on the 2D image or a 3D image acquired before the operation, through the display, by performing the image registration method based on claim 1.

20. A computer program medium storing software to perform the image registration method based on claim 1.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The above and/or other aspects will become apparent and more readily appreciated from the following description of embodiments, taken in conjunction with the accompanying drawings, in which:

[0027] FIG. 1 is a block diagram of an image registration apparatus according to an embodiment of the disclosure;

[0028] FIG. 2 is a flowchart of an image registration method according to an embodiment of the disclosure;

[0029] FIGS. 3A and 3B show examples for describing the process of making an operation plan on a CT image and extracting a volume of interest from the CT image;

[0030] FIG. 4 shows an example for describing that a DRR image is generated from a CT image based on orthogonal projection;

[0031] FIG. 5 shows an example for describing the process of automatically generating a region of interest after labeling through machine learning in an AP image;

[0032] FIG. 6A and 6B shows an example of the result of generating a first region of interest as the DRR image in an AP image and an LL image;

[0033] FIGS. 7A and 7B show an example of the result of generating a second region of interest as a C-arm image in the AP image and the LL image;

[0034] FIG. 8 shows an example of a user interface for resizing, positioning, rotating, setting SP, and the like for the region of interest;

[0035] FIGS. 9A to 9D are views for describing a process of reconstructing vertices of a first region of interest in a CT volume;

[0036] FIG. 10 shows an example to describe the result of reconstructing eight vertices of a first region of interest in a CT volume;

[0037] FIG. 11 is a view for describing a process of reconstructing vertices of a second region of interest in a C-arm volume;

[0038] FIG. 12 is a view for describing the result of reconstructing eight vertices of a second region of interest in a C-arm volume;

[0039] FIGS. 13A and 13B are views for describing a case where an axial direction of a first volume of interest and a tip line of a spinous process are aligned and a caser where they are misaligned and rotated;

[0040] FIG. 14 shows an example for describing a method of determining a first rotation angle based on a planning object position of which an operation plan is made before an operation;

[0041] FIG. 15 shows an example for describing a user interface through which a user sets a tip line of a spinous process in a C-arm AP image;

[0042] FIG. 16 shows an example for describing a process of calculating a second rotation angle based on a C-arm AP image;

[0043] FIG. 17 shows a relationship between eight points reconstructed in a CT volume and points rotated by a first rotation angle.

[0044] FIG. 18 is a view for describing a local coordinate system V, the origin of which is the center of a first volume of interest; and

[0045] FIG. 19 is a schematic diagram of a medical operating robot system according to an embodiment of the disclosure.

DETAILED DESCRIPTION OF EMBODIMENTS

[0046] Below, embodiments of the disclosure will be described with reference to the accompanying drawings.

[0047] FIG. 1 is a block diagram schematically showing the configuration of an image registration apparatus 10 for performing image registration according to an embodiment of the disclosure. AS an electronic apparatus capable of performing the image registration, the image registration apparatus 10 refers to a computing apparatus such as a computer, a notebook computer, a laptop computer, a tablet personal computer (PC), a smartphone, a mobile phone, a personal media player (PMP), and a personal digital assistant (PDA). An image registration method according to an embodiment of the disclosure may be applied to medical image processing software such as medical navigation software. The image registration apparatus 10 according to an embodiment of the disclosure may execute image processing software such as an operation plan or planning as well as the medical navigation software.

[0048] Referring to FIG. 1, the image registration apparatus 10 according to an embodiment of the disclosure includes a memory 11 for storing data and a program code, and a processor 13 for executing the data and the program code.

[0049] The memory 11 refers to a computer-readable recording medium, and stores at least one computer program code to be performed by the processor 13. Such a computer program code may be loaded from a floppy drive, a disk, a tape, a digital versatile disc (DVD)/compact disc read only memory (CD-ROM) drive, a memory card, etc., which are separated from the memory 11, into the memory 11. The memory 11 may store software for the image registration, a patient's medical image or data, etc.

[0050] The processor 13 is to execute and process a computer program instruction through basic logic, calculations, operations, etc., and the computer program code stored in the memory 11 is loaded into and executed by the processor 13. The processor 13 may execute an algorithm stored in the memory llto perform a series of 2D-3D image registration.

[0051] As shown in FIG. 1, the image registration apparatus 10 according to an embodiment of the disclosure may further include a display 15 for displaying data and images, a user interface 17 for receiving a user's input, and a communication interface 19 for interworking with the outside. The user interface 17 may for example include input devices such as a microphone, a keyboard, a mouse, or a foot pad.

[0052] Below, the image registration method performed by the image registration apparatus 10 (or the processor 13) according to an embodiment of the disclosure will be described.

[0053] FIG. 2 is a flowchart of the image registration method according to an embodiment of the disclosure, and FIGS. 3 to 14 are schematic diagrams illustrating the steps in the flowchart of FIG. 2.

[0054] Referring to FIG. 2, the image registration method according to an embodiment of the disclosure includes pre-operative planning steps.

[0055] First, a 3D image is acquired by taking a pre-operative a computed tomography (CT) image for a patient's surgical site through an imaging apparatus (S1). Here, another imaging apparatus for acquiring the 3D image, e.g., a magnetic resonance imaging (MRI), etc., may be used as an alternative to the CT, and the disclosure is not limited to a specific imaging apparatus.

[0056] A doctor uses operation planning software to make an operation plan for a patient's 3D image (S2). For example, in the case of an operation of inserting and fixing a screw into a pedicle during a spinal operation, the selection of a screw product based on the diameter, length, material, etc. of the screw, a pedicle entry point for the screw, a target where the end of the screw is settled, etc. may be set and displayed on the 3D image. Such planning software is provided by many companies in various operative fields.

[0057] Next, the image registration apparatus 10 extracts a volume of interest with respect to the surgical site for which the operation plan is established (S3). Such extraction may be automatically performed by an automated algorithm with respect to the position of a planning object, e.g., the screw displayed on the 3D image, or may be performed as large as a given volume boundary is adjusted by a doctor in person.

[0058] Referring to FIG. 3A, a doctor may set or adjust the volume of interest by controlling a white box provided on a left 3D image. Unlike FIG. 3A, referring to FIG. 3B, left and right margins (a, b) having default sizes with respect to, for example, the planning object, i.e., the screw are automatically extracted.

[0059] From such extracted volume of interest, an anterior-posterior (AP) image and a lateral-lateral (LL) image are generated as digitally reconstructed radiograph (DRR) images (S4). In other words, as shown in FIG. 4, the AP image and the LL image are virtual C-arm images based on a CT coordinate system given by CT equipment, in which orthogonal projection is used to generate the AP image by an AP source and a detector and to generate the LL image by an LL source and a detector.

[0060] Next, the AP image and the LL image of the surgical site are acquired through the C-arm equipment during the operation (S5), and the C-arm equipment is registered to a spatial coordinate system based on a marker placed on a part of a patient's body (hereinafter, referred to as a PM marker) or a marker to be referenced in other operative space (S6). Korean Patent No. KR2203544 filed by the present applicant's discloses technology of registering the C-arm equipment to a 3D space or registering a 2D image to the 3D space, and Patent 544' is incorporated by reference into the present disclosure in its entirety. The technology for registering the C-arm equipment to the space have been publicly known in many other documents in addition to Patent 544', and the disclosure is not limited to specific spatial registration technology.

[0061] The image registration apparatus 10 sets a first region of interest (ROI) and a second region of interest (ROI) for the DRR image and the C-arm image, respectively (S7 and S8). Each region of interest may be set in units of vertebral bones in the case of an operation of fixing a pedicle screw. As shown in FIG. 5, the region of interest may be extracted and labeled in units of vertebral bones through machine learning, and may be defined as a rectangle including each vertebral bone.

[0062] FIGS. 6A and 6B show the results of extracting the first region of interest in the AP image and the LL image of the labeled DRR image, and FIGS. 7A and 7B show the results of extracting the second region of interest in the AP image and the LL image of the labeled C-arm.

[0063] Here, it is noted that the first region of interest for the DRR image and the second region of interest for the C-arm image are extracted equivalently to each other. At least the vertices of the rectangle of the region of interest are selected as points having equivalences corresponding to each other. The equivalence does not mean a perfect match, but for example means that the first region of interest and the second region of interest are extracted or selected so that a relationship between the image feature of the vertebral bone and four vertices the selected region of interest is kept constant.

[0064] In this embodiment, for example, with respect to the vertebral bone, the first region of interest and the second region of interest are selected so that the tip of the spinous process can be disposed at the center of the region of interest for the AP image, and the outer margin of the vertebral bone can be uniformly applied to the first and second regions of interest in each of the AP/LL images.

[0065] Referring to FIG. 8, a doctor can resize, reposition, rotate, etc. the extracted region of interest through a given user interface, and can create a line pointing at a specific body part, e.g., the tip of the spinous process or adjust the region of interest to be aligned with a center line.

[0066] Next, the image registration apparatus 10 reconstructs the first region of interest displayed on the AP image and the LL image for the DRR image to a space, i.e., a volume of interest (S9). Intuitively, when the first region of interest in the AP image is moved parallel to a normal vector of the AP image on the assumption that the AP image is positioned at the virtual detector and at the same time the first region of interest in the LL image is moved parallel to the normal vector of the LL image on the assumption that the LL image is positioned at the virtual detector, a space where two planes intersect, i.e., a space where the first regions of interest on the two planes intersect each other will be called a first volume of interest.

[0067] The process of calculating the first volume of interest based on a coordinate system will be described with reference to FIGS. 9 to 13.

[0068] Referring to FIG. 9, a first vertex A1 of the first region of interest on the AP image and a virtual detector plane on which the AP image is positioned in the CT coordinate system are defined for the first region of interest with respect to the vertebral bone labeled with L3. Further, the virtual detector plane on which the LL image is positioned is defined, and the first region of interest on the LL image is also marked. Intuitively, when a volume formed by the intersection between the

[0069] AP image and the LL image plane in a space is called the CT volume, the CT volume is expressed as a hexahedron that has six boundary faces and a three-axial CT coordinate system.

[0070] Because the DRR image is based on the orthogonal projection, such a hexahedral CT volume is formed, and the normal vector passing through A.sub.1 reaches a certain height of A.sub.1 while intersecting the top and bottom boundary faces of the CT volume.

[0071] Therefore, A.sub.1 and the points I.sub.1 and I.sub.2 intersecting the top and bottom boundary faces are obtained as follows.

A.sub.1=?.sub.1N.sub.AP+A.sub.1 [Equation 1]

I.sub.1=f.sub.inter(?.sub.Top,A.sub.1,A.sub.1) [Equation 2]

I.sub.2=f.sub.inter(?.sub.Bot,A.sub.1,A.sub.1) [Equation 3]

[0072] Where, ?.sub.1 is an arbitrary number, N.sub.AP is a normal vector to the AP plane, and f.sub.inter is a function that takes two points and one plane as input variables and obtains intersection points between a line connecting the two points and the plane.

[0073] Therefore, I.sub.1 is obtained as follows.

[00001]$\begin{array}{cc}{I}_1=A_1+\frac{N_{\mathrm{AP}}^T\left(P_{\mathrm{Top}}-A_1\right)}{N_{\mathrm{AP}}^T\left(A_1^{}-A_1\right)}\left(A_1^{}-A_1\right)& \left[\mathrm{Equation}4\right]\end{array}$

[0074] Where, P.sub.Top is ?.sub.Top, and I.sub.2 is obtained in the same way as I.sub.1. Referring to FIG. 9B, let I.sub.1 and I.sub.2 be arbitrary points on arbitrary normal starting from I.sub.1 and I.sub.2 and perpendicularly penetrating the LL image plane, respectively, I.sub.1 and I.sub.2 are expressed as follows.

I.sub.1=?.sub.2N.sub.LL+I.sub.1 [Equation 5]

I.sub.2=?.sub.2N.sub.LL+I.sub.2 [Equation 6]

[0075] Where, ?.sub.2 is an arbitrary number, and N.sub.LL is a normal vector of the LL plane.

[0076] Next, referring to FIG. 9C, the epipolar line of the vertex A.sub.1 is obtained by connecting intersection points between the LL image plane and line segments from I.sub.1 and I.sub.2 to I.sub.1 and I.sub.2. Further, intersection points between the epipolar line and any position in the region of interest on the LL image are obtained. In FIG. 9C, the intersection point C.sub.3 between the epipolar line and a line segment connecting the centers C.sub.1 and C.sub.2 on the left and right sides in the first region of interest is obtained.

[0077] By projecting the intersection point C.sub.3 between the region of interest and the epipolar line obtained as above onto the normal A.sub.1-A.sub.1 from the vertex A.sub.1, P.sub.1 is obtained as shown in FIG. 9D. Here, if the position of the region of interest intersecting the epipolar line is taken as a top side, P.sub.1 is positioned parallel to the top side of the first region of interest of the LL image. Therefore, it is noted that the point P.sub.1 in the CT volume corresponding to A.sub.1 is selectable on the normal from A.sub.1 at any position within positions parallel to the top and bottom sides of the region of interest on the LL image.

[0078] By applying the same process as that of A.sub.1 to the other three vertices, it will be understood that the plane of the first region of interest on the AP image is reconstructed as transferred to the CT volume. Likewise, it will be understood that the vertices on the LL image are reconstructed as transferred to the CT volume.

[0079] FIG. 10 shows an example that eight vertices A.sub.1 to A.sub.8 defining the first region of interest are reconstructed as eight points .sup.CTP.sub.1, .sup.CTP.sub.2, . . . .sup.CTP.sub.8 within the CT volume.

[0080] FIG. 11 shows a relationship in which one vertex A.sub.1 defining the second region of interest on the C-arm image is reconstructed to the C-arm volume.

[0081] Compared to FIGS. 9A to 9D, the C-arm image is obtained based on perspective projection, and thus a vector between the source and the detector is defined as a direction vector based on a PM coordinate system using the PM marker instead of using the unit vectors on the orthogonal CT coordinate system.

[0082] Therefore, I.sub.1, I.sub.2, I.sub.1, I.sub.2, C.sub.3, and P.sub.1 in FIG. 11 are obtained as follows.

[00002]$\begin{array}{cc}{I}_1=f_{\mathrm{inter}}\left({?}_{\mathrm{Top}},S_{\mathrm{AP}},A_1\right)& \left[\mathrm{Equation}7\right]\end{array}$$\begin{array}{cc}{I}_2=f_{\mathrm{inter}}\left({?}_{\mathrm{Bot}},S_{\mathrm{AP}},A_1\right)& \left[\mathrm{Equation}8\right]\end{array}$$\begin{array}{cc}{I}_1^{}={?}_2\frac{I_1-S_{\mathrm{LL}}}{.Math.I_1-S_{\mathrm{LL}}.Math.}+S_{\mathrm{LL}}& \left[\mathrm{Equation}9\right]\end{array}$$\begin{array}{cc}{I}_2^{}={?}_2\frac{I_2-S_{\mathrm{LL}}}{.Math.I_2-S_{\mathrm{LL}}.Math.}+S_{\mathrm{LL}}& \left[\mathrm{Equation}10\right]\end{array}$$\begin{array}{cc}{C}_3=f_{\mathrm{inter}}\left({?}_e,C_1,C_2\right)& \left[\mathrm{Equation}11\right]\end{array}$$\begin{array}{cc}{P}_1=\left\{{\left(I_1-C_3\right)}^{T}N\right\}N+C_3& \left[\mathrm{Equation}12\right]\end{array}$

[0083] Where, S.sub.LL is a source position on the LL image, ?.sub.2 is an arbitrary number,

[00003]$N=\frac{C_3-S_{\mathrm{LL}}}{.Math.C_3-S_{\mathrm{LL}}.Math.},$

and C.sub.1 and C.sub.2 are the centers of the left and right sides in the region of interest on the LL image.

[0084] By repeating the same process with respect to eight vertices A.sub.1 to A.sub.8, eight points .sup.PMP.sub.1, .sup.PMP.sub.2, . . . .sup.PMP.sub.8 are reconstructed in the C-arm volume as shown in FIG. 12.

[0085] Referring to FIGS. 10 and 12, eight points reconstructed in the CT volume and eight points reconstructed in the C-arm volume look as if the region of interest on the AP image and the region of interest on the LL image intersect at their centers. It will be understood that these eight points serve as the midlines of a hexahedron while forming a volume of interest.

[0086] Referring back to the flowchart of FIG. 2, the image registration apparatus 10 reconstructs the eight points or the first and second volumes of interest in the CT volume and the C-arm volume, respectively, and calculates a rotation angle between the reference position of a patient's surgical site and the corresponding reference position with respect to a predetermined axis (S10).

[0087] In this embodiment, the reference position is the tip of the spinous process, and the corresponding first reference position corresponds to the normal vector of an AXIAL image defined in the same direction as the vector of the cross product between the normal vector of the AP image plane and the normal vector of the LL image plane. However, a first origin determining the position of the rotation axis ideally reflects the rotation center of the spine, but it is difficult to define this. Therefore, one of two options (to be described later) is selected.

[0088] First, referring to FIG. 13A, it will be understood that there is no rotation in this case because the end of the spinous process SP and a line segment between .sup.CTP.sub.5 and .sup.CTP.sub.6 shown in FIG. 10 are different only in height and have no lateral deviation.

[0089] On the other hand, referring to FIG. 13B, it will be understood that the axis passing through the center of the first volume of interest is tilted by an angle of ? because there is lateral deviation between the end of the spinous process SP and the line segment between .sup.CTP.sub.5 and .sup.CTP.sub.6.

[0090] To obtain the tilted rotation angle ? (hereinafter referred to as a first rotation angle), a doctor may use a pre-operative planning object as shown in FIG. 14. Referring to FIG. 14, an angle between a line E.sub.c-T.sub.c that connects the center E.sub.c between the entry points E.sub.r and E.sub.1 at which a pair of screws enter the left and right pedicles with respect to the axis and the center T.sub.c between the positions T.sub.1 and T.sub.r (hereinafter referred to as targets) to which the distal ends of the screws are finally mounted, and a longitudinal line segment that passes through the center of the axial plane of the first volume of interest, i.e., the line segment between .sup.PTP.sub.6 and .sup.PTP.sub.8 is obtained as a first rotation angle ?.sub.DRR.

[0091] In this case, it will be understood that the first origin through which the first rotation axis passes is selected as the center T c between the left and right targets T.sub.1 and T.sub.r, and different in height by d from the center of the first volume of interest. This makes it easy to calculate a rotation value based on a doctor's operation plan, and helps to perform quick registration processing. As discussed for the rotation of the C-arm image, the center of the first volume of interest may be used instead of the target center as the first rotation origin.

[0092] Meanwhile, as shown in FIG. 15, it is possible to acquire the true AP and LL images of the C-arm image. Therefore, a user interface may be provided so that a doctor can move the center line of the second region of interest on the AP image to match the tip of the spinous process in order to set the tip of the spinous process in person in this stage even though the center of the second region of interest on the AP image often matches the tip of the spinous process. This may also be applied to the DRR image.

[0093] As shown in FIG. 16, when the tip of the spinous process set by a user passes through P.sub.12 and is parallel to P.sub.5-P.sub.6, a second rotation angle of the C-arm image may be obtained as an angle between a line P.sub.9-P.sub.10 of an input end spinous process tip projected at a height of P.sub.6 and a line segment P.sub.5-P.sub.6 with respect to an intersection line between the plane formed by P.sub.1 to P.sub.4 and the plane formed by P.sub.5 to P.sub.8.

[00004]$\begin{array}{cc}{P}_{11}=f_{\mathrm{inter}}\left({?}_H,P_6,P_8\right)& \left[\mathrm{Equation}13\right]\end{array}$$\begin{array}{cc}N=\frac{P_2-P_4}{.Math.P_2-P_4.Math.}& \left[\mathrm{Equation}14\right]\end{array}$$\begin{array}{cc}{P}_{12}=\left\{{\left(P_{10}-P_{11}\right)}^{T}N\right\}N+P_6& \left[\mathrm{Equation}15\right]\end{array}$$\begin{array}{cc}{?}_{C-\mathrm{arm}}=a\cos \left({\left(\frac{P_{12}-P_{11}}{.Math.P_{12}-P_{11}.Math.}\right)}^T\left(\frac{P_6-P_{11}}{.Math.P_6-P_{11}.Math.}\right)\right)& \left[\mathrm{Equation}16\right]\end{array}$

[0094] Where, P.sub.1 to P.sub.8 are points to which eight vertices of the second region of interest are reconstructed, and P.sub.9 and P.sub.10 are a spinous process tip line designated by a user.

[0095] It is noted that FIG. 16 illustrates a case where a horizontal plane formed by P.sub.1 to P.sub.4 and a vertical plane formed by P.sub.5 to P.sub.8 do not intersect except an intersection line.

[0096] FIG. 17 shows a relationship between eight points .sup.CTP.sub.1 to .sup.CTP.sub.8 reconstructed in the CT volume and points .sup.CTP.sub.1 to .sup.CTP.sub.8 rotated from the points .sup.CTP.sub.1 to .sup.CTP.sub.8 by the first rotation angle ?.sub.DRR calculated with reference to FIG. 14. This shows the positions of the points reflecting the rotation of the spine with respect to the tip of the spinous process after reconstructing the first region of interest in the CT coordinate system to the CT volume. Similarly, for the second region of interest on the C-arm image, the point positions .sup.PMP.sub.1 to .sup.PMP.sub.8 calculated reflecting the rotation of a patient's spine during the operation may be obtained in the same way.

[0097] When a transformation relationship between the PM coordinate system and the CT coordinate system is applied to the rotated points, i.e., .sup.PMP.sub.1 to .sup.PMP.sub.8 and .sup.CTP.sub.1 to .sup.CTP.sub.8, the Euclidean distance therebetween should be 0 ideally. The optimal registration may be performed under the condition that the sum or average of the Euclidean distances between the eight point pairs is the smallest, and the purpose of initial registration is to obtain a transform matrix satisfying this condition (S11).

[00005]$\begin{array}{cc}{T_{\mathrm{PM}}^{\mathrm{CT}}?\underset{T_{\mathrm{PM}}^{\mathrm{CT}}}{\min }\underset{i=1}{\overset{N}{.Math.}}{.Math.}^{\mathrm{CT}}{P}_i^{}-T_{\mathrm{PM}}^{\mathrm{CT}}{}^{\mathrm{PM}}{P}_i^{}.Math.}^2& \left[\mathrm{Equation}17\right]\end{array}$

[0098] Where, T.sub.PM.sup.CT is a transformation matrix from the PM coordinate system to the CT coordinate system.

[0099] Referring to FIG. 18, a transformation relationship from the CT coordinate system to a V-local coordinate system will be described. Here, the origin of the V-local coordinate system is defined as follows based on the midpoint of the intersection line between the horizontal plane and the vertical plane.

[00006]$\begin{array}{cc}{}^{V}O=\frac{P_9+P_{10}}{2}& \left[\mathrm{Equation}18\right]\end{array}$

[0100] Where, P.sub.9=f.sub.inter(?.sub.V,P.sub.1,P.sub.3) P.sub.10=f.sub.inter(?.sub.V,P.sub.2,P.sub.4).

[0101] Thus, three axes .sup.VX, .sup.VY, and .sup.VZ of the V-local coordinate system are defined as follows.

[00007]$\begin{array}{cc}{V}^{}{X}^{}=\frac{P_2-P_4}{.Math.P_2-P_4.Math.}& \left[\mathrm{Equation}19\right]\end{array}$$\begin{array}{cc}{}^{V}Z=\frac{P_9-P_{10}}{.Math.P_9-P_{10}.Math.}& \left[\mathrm{Equation}20\right]\end{array}$$\begin{array}{cc}{}^{V}Y={}^{V}Z?{}^V{X}^{}& \left[\mathrm{Equation}21\right]\end{array}$$\begin{array}{cc}{}^{V}X={}^{V}Y?{}^{V}Z& \left[\mathrm{Equation}22\right]\end{array}$

[0102] In the V-local coordinate system, the position .sup.VP.sub.i of P.sub.i is defined as follows.

.sup.VP.sub.i=R.sub.CT.sup.V.sup.CTP.sub.i+t.sub.CT.sup.V [Equation 23]

[0103] Where, R.sub.CT.sup.V is a rotational transformation matrix from the CT coordinate system to the V-local coordinate system, and t.sub.CT.sup.V is a translation vector between the CT coordinate system and the V-local coordinate system.

[0104] In addition, the points in the rotated V-local coordinate system are obtained as follows.

.sup.VP.sub.t=Rodrigues(.sup.VZ,?.sub.DRR).sup.VP.sub.i [Equation 24]

[0105] Where, the Rodrigues function is defined as a function that rotates an object by an input rotation angle with respect to the input rotation axis.

[0106] Thus, the rotated point .sup.CTP.sub.I in the CT coordinate system is defined as follows.

.sup.CTP.sub.i=(R.sub.CT.sup.V).sup.?1.sup.VP.sub.i?(R.sub.CT.sup.V).sup.?1t.sub.CT.sup.V [Equation 25]

[0107] Thus, the foregoing processes are applied to the eight points, and .sup.PMP.sub.i rotated from .sup.PMP.sub.i in the PM coordinate system is calculated and input to the following equation.

[00008]$\begin{array}{cc}{T}_{\mathrm{PM}}^{\mathrm{CT}}?\underset{T_{\mathrm{PM}}^{\mathrm{CT}}}{\min }\underset{i=1}{\overset{N}{.Math.}}{.Math.{}^{\mathrm{CT}}{P}_i^{}-{T_{\mathrm{PM}}^{\mathrm{CT}}}^{\mathrm{PM}}{P}_i^{}.Math.}^2& \left[\mathrm{Equation}26\right]\end{array}$

[0108] When the initial registration is completed by finding the optimal transformation matrix, the image registration apparatus 10 derives the optimal transformation matrix while adjusting a search range of the DRR image and performs a registration optimization process, thereby completing the image registration (S12). The optimization process is publicly known based on global search, and thus detailed descriptions thereof will be omitted.

[0109] As described above, the image registration method has the advantage of increasing the accuracy of the image registration according to the rotation of a human body, and quickly performing the image registration processing.

[0110] The disclosure may be implemented as a computer program recording medium in which a computer program is recorded to perform the image registration method on a computer.

[0111] Further, the disclosure may also be implemented by the medical operating robot system based on the foregoing image registration method.

[0112] Referring to FIG. 19, the medical operating robot system 1 according to an embodiment of the disclosure includes a C-arm imaging apparatus 100, a medical operating robot 200, a position sensor 300, and a navigation system 400, and the medical operating robot 200 includes a main body 201, a robot arm 203 with an end effector 203a, and a robot controller 205.

[0113] The C-arm imaging apparatus 100 is used to acquire the AP image and the LL image of a patient's surgical site during the operation.

[0114] The robot arm 203 is secured to the robot main body 201, and includes the end effector 203a, to which a surgical instrument is detachably coupled, at a distal end thereof. The position sensor 300 is implemented as an OTS that tracks the real-time position of the surgical instrument or the end effector 203a by recognizing the marker. The controller 205 is provided in the robot main body 201, and controls the robot arm 203 according to predetermined operation planning and control software.

[0115] The navigation system 400 performs the foregoing image registration method to display planning information about a surgical instrument or implant on a C-arm image acquired during an operation or display a real-time position of the surgical instrument or implant on the C-arm image or a 3D image acquired before the operation through a display, thereby assisting a doctor in performing the operation. To this end, the navigation system 400 may further include the display connected thereto so that a doctor can view the real-time position of the surgical instrument or the like as the operation plan and the operating status by his/her naked eyes during the operation. A person having ordinary knowledge in the art may easily understand that other elements than the navigation system 400 of FIG. 15 are the same as those commonly used in a medical operating robot system.

[0116] According to the disclosure, there is accurate and quick image registration processing which can compensate for movement due to rotation of a patient's body part

[0117] Although embodiments of the disclosure have been described so far, it will be appreciated by those skilled in the art that modifications or substitutions may be made in all or part of the embodiments of the disclosure without departing from the technical spirit of the disclosure.

[0118] Accordingly, the foregoing embodiments are merely examples of the disclosure, and the scope of the disclosure falls into the appended claims and equivalents thereof.