Augmented Reality Guidance System For Guiding Surgical Operations On An Articulating Portion Of A Bone

Abstract

An augmented reality guidance system for guiding surgical operations on an articulating portion of a bone includes a tracking element to be attached to the articulating portion and-having a surface bearing against the portion that is custom-designed from a virtual model , and a three-dimensional reference mark ; a probe with a three-dimensional reference mark for probing at least three regions of a portion apart from the cartilage area; a camera for displaying and tracking the reference marks ; and a processing system for processing virtual bone model data and real-time image data from the camera to determine the position and relative direction of the tracking element and probe, to display on a display device , a three-dimensional virtual representation of a geometric datum relating to the bone model having position and direction determined by the position and direction of the tracking device and corrected after probing.

Claims

1. An augmented reality system for guiding a surgical operation on an articulating portion of a bone, the system comprising: a tracking element configured to be attached to the articulating portion of the bone, the tracking element comprising, on the one hand, at least one face for bearing on the articulating portion made to measure from a virtual model of the articulating portion of the bone and, on the other hand, a three-dimensional visual marker a camera for visualizing and tracking the marker a processing system configured to process data from the virtual bone model and, in real time, image data obtained from the camera in order to determine the position and orientation of the tracking element (2), in such a manner as to display, on a display unit secured to the camera and superimposed on the field of view of a user, a three-dimensional virtual representation of at least one geometric datum relating to the bone model the position and orientation of which are determined by the position and orientation of the tracking element wherein theguidance system comprises a sensor which is also provided with a three-dimensional visual marker and to palpate at least three zones of a portion of the bone outside the cartilage zone, the marker of the sensor also being be tracked by the camera, and the processing system being further configured in order to determine the position and the orientation of the sensor with respect to the position and the orientation of the tracking element in a manner such as to correct, on the display unit and after palpation, the position and the orientation of the three-dimensional virtual representation of the geometric datum relating to the bone model with respect to the tracking element .

2. The guidance system as claimed in claim 1, comprising a cutting or drilling instrument provided with a three-dimensional visual marker, the markerbeing configure to be tracked by the camera and the processing systembeing further configured in order to determine the position and the orientation of the instrument with respect to the position and the orientation of the tracking element in a manner such as to display, on the display unit the position and the orientation of a cutting plane or drilling axis linked to the instrument with respect to the tracking element.

3. The guidance system as claimed in claim 1, the display unit is a screen displaying the images obtained from the camera.

4. The guidance system as claimed in claim 1, whereinthe display unit is at least one spectacle lens configured to be worn by the user.

5. The guidance system as claimed in claim 4, whereinthe at least one spectacle lens carries the camera.

6. The guidance system as claimed in claim 1, wherein the geometric datum relating to the bone model corresponds to the external surface of said virtual bone model, and the processing system is further configured, after correction, to process the data for the virtual bone model in order to increase its dimensions at the level of the bearing surface of the tracking element so that the display unit represents the bearing face of the tracking element in contact with the external surface of the virtual bone model.

7. The guidance system as claimed in claim 1, whereinthe visual marker is attached to the tracking element in a removable or separable manner.

8. The guidance system as claimed in claim 1, wherein the tracking element comprises means for securing a guide block.

9. The guidance system as claimed in claim 1, wherein the tracking element comprises at least one orifice for guiding a pin for securing to the bone, and the system comprises a guide block having at least one orifice for the passage of the pin the guide block being to be secured to the bone in a position and at an orientation which is determined and guided by the pin.

10. The guidance system as claimed in claimwherein guide block comprises at least one slot and/or at least one guide orifice the position and/or orientation of which can be adjusted with respect to the guide block.

11. The guidance system as claimed in claim 9, wherein the guide block comprises at least one slot and/or at least one guide orifice the position and/or orientation of which can be adjusted with respect to the guide block.

12. The guidance system as claimed in claim 1, comprising a pin configured to secure a guide block to the bone.

13. The guidance system as claimed in claim 12, wherein the guide block comprises at least one slot and/or at least one guide orifice the position and/or orientation of which can be adjusted with respect to the guide block.

Description

DESCRIPTION OF THE FIGURES

[0044] Other features and advantages of the invention will become apparent from the following description, which is given by way of non-limiting indication and made with reference to the accompanying figures, in which:

[0045] FIG. 1 is a diagrammatic perspective view illustrating a tracking element of the system in accordance with the invention, positioned on an articulating portion of a femur.

[0046] FIG. 2FIG. 2 is a diagrammatic view illustrating the display unit, processing system, and the camera of the augmented reality guidance system of the invention.

[0047] FIG. 3FIG. 3 is a diagrammatic perspective view illustrating a tracking element of the system in accordance with the invention, positioned on an articulating portion of a femur, on the hip side.

[0048] FIG. 4FIG. 4 is a diagrammatic perspective view illustrating a tracking element of the system in accordance with the invention positioned on an articulating portion of a humerus, on the shoulder side.

[0049] FIG. 5FIG. 5 is a diagrammatic perspective view illustrating a tracking element of the system in accordance with the invention, positioned on an articulating portion of a scapula.

[0050] FIG. 6FIG. 6 is a diagrammatic perspective view illustrating a tracking element of the system in accordance with the invention, positioned on an articulating portion of a tibia, on the knee side.

[0051] FIG. 7FIG. 7 is a diagrammatic perspective view illustrating a guide block attached to a tibia and comprising a guide slot the position and/or orientation of which are adjustable relative to the guide block.

[0052] FIG. 8FIG. 8 is a diagrammatic perspective view of a sensor provided with a second three-dimensional visual marker.

[0053] FIG. 9FIG. 9 is a diagrammatic perspective view illustrating an instrument, such as a drill, provided with a three-dimensional visual marker.

DETAILED DESCRIPTION OF THE INVENTION

[0054] With reference to FIGS. 1 to 9, the invention concerns an augmented reality system for guiding a surgical operation on an articulating portion (1) of a bone.

[0055] The invention is not limited to a particular joint, and makes it possible to guide the positioning, i.e. the production of a bone cut or drilling, of any type of orthopaedic joint implant, such as a knee, shoulder or hip prosthesis, etc.

[0056] In known manner, the guidance system comprises a tracking element (2) intended to be secured to an articulating portion (1) of a bone. The tracking element (2) comprises at least one face (21) bearing on the articulating portion, made to measure from a virtual bone model.

[0057] This makes it possible to create a unique relationship between the virtual bone model and the tracking element (2) in a manner such that if the position and orientation in space of the tracking element (2) are known, the position and orientation in space of the virtual bone model are known extremely accurately.

[0058] In order to be able to know the position and orientation in space of the tracking element (2), in known manner, this comprises a three-dimensional visual marker (22) integrating information relating to the position and orientation of the tracking element (2) which is capable of being processed by a processing system (3), in particular a computer system.

[0059] In known manner again, the augmented reality system comprises a camera (4) intended to display and to track the three-dimensional visual marker (22) of the tracking element (2), and a processing system (3) configured to process data from the virtual bone model and, in real time, image data obtained from the camera (4) in order to determine the position and the orientation of the tracking element (2).

[0060] In a manner known to the person skilled in the art, the camera (4) is firmly attached, in particular in terms of a frame of reference, to a display unit (6) making it possible to display information transmitted by the processing system (3), superimposed on the field of view of a user.

[0061] The display unit (6) is, for example, in the form of a screen enabling the images obtained from the camera (4) to be visualized directly and the information transmitted by the processing system (3) to be superimposed thereon, or alternatively it is in the form of augmented reality spectacles, i.e. spectacles comprising at least one lens forming the display unit (6) and through which the user can observe a scene, and onto which the display unit (6) can superimpose the information.

[0062] In the latter embodiment, and in a known manner, the augmented reality spectacles may also carry the camera (4).

[0063] In this manner, when the processing system (3) detects that the camera (4) is visualizing the three-dimensional visual marker (22) of the tracking element (2), the processing system (3) displays on the display unit (6) a three-dimensional virtual representation of at least one geometric datum relating to the bone model positioned relatively with respect to the tracking element (2). In other words, the virtual representation of the geometric datum is attached to the tracking element (2), in accordance with a single position and orientation which in particular depends on those of said tracking element (2).

[0064] The geometric datum relating to the bone model represented virtually and displayed on the display unit (6) may be of any type, such as a geometrical surface or contours of the bone model, a mechanical axis, planned cutting planes or drilling axes, etc.

[0065] The augmented reality system is well known in the art and will not be described in further detail. The invention in particular consists in correcting the inaccuracy of this system, which is inherent to the presence of cartilage or fat on the articulating portion to which the tracking element (2) is to be secured.

[0066] To accomplish this, the system in accordance with the invention comprises a sensor (7) also provided with a three-dimensional visual marker (71) carrying information relating to the position and the orientation of the sensor (7) in space.

[0067] When the tracking element (2) is secured to the actual bone, the display unit (6) will virtually represent the geometric datum relating to the bone model superimposed on the actual scene. However, given the presence of cartilage, there will be a slight discrepancy between the real image of the bone and the virtual representation. In fact, and in practice, the tracking element (2) is not directly in contact with the bone, but in contact with an extra thickness of cartilage.

[0068] To correct this discrepancy, the surgeon will palpate at least three zones with the marked sensor (7), for example three zones of approximately 2 mm.sup.2 which are also non-cartilaginous.

[0069] Since the processing system (3) accurately determines the position and orientation of the elements secured to the markers (22, 71), the position and spatial orientation of the sensor (7) with respect to the position and orientation of the tracking element (2) is known.

[0070] In theory, the three zones which the sensor (7) will palpate must normally be zones which correspond to equivalent zones of the virtual bone model. The processing system (3) is configured to rectify this discrepancy and very precisely reposition the virtual bone model, and in particular the virtual representation of the geometric datum, with respect to the three-dimensional visual marker (22) of the tracking element (2), and therefore with respect to the tracking element (2) itself.

[0071] From the foregoing, this makes it possible to know the position of the articulating portion of the bone very precisely, and to visualize it in three dimensions and virtually on the display unit (6) superimposed on the real bone. It is also possible to visualize the mechanical axis of the bone, or in fact the planned cutting planes or drilling axes.

[0072] In order to then facilitate and guide the surgical operation, the system in accordance with the invention also comprises an instrument (8), such as a cutting or drilling instrument, which also comprises a three-dimensional visual marker (81) carrying information relating to the position and the orientation of said instrument (8) in the space which the processing system (3) is capable of processing when it is visualized by the camera (4).

[0073] In other words, it is possible to know the position and the orientation of the instrument (8) precisely with respect to the visual marker (22) of the tracking element (2), and therefore with respect to the bone itself. The information integrated into the three-dimensional visual marker (81) of the instrument (8) enables the processing system (3) to display a drilling axis or a cutting plane linked to the instrument (8), for example, on the display unit (6) and superimposed on the field of vision of the surgeon, which is perfectly positioned with respect to the real bone, without inaccuracy relating to the extra thickness of cartilage.

[0074] The tracking element (2) may also comprise means for securing a guide block (9), in particular in a known and determined position and orientation relative to the tracking element (2).

[0075] In this configuration, the three-dimensional visual marker (22) of the tracking element (2) may be attached to the tracking element (2) in a removable or separable manner, allowing the surgeon to remove it so as not to interfere with the subsequent drilling cutting operation.

[0076] In accordance with another embodiment, the tracking element (2) comprises at least one, and preferably two orifices (10) adapted to receive two pins (11) intended to be secured in the bone. The tracking element (2) may then be removed by sliding it along the pins (11), and a guide block (9) also comprising two identical orifices (91) may be secured to the bone by positioning it by means of the two pins (11), see FIG. 7. The guide block (9) comprises a slot (92) and/or guide orifice the position and orientation of which are known with respect to the bone.

[0077] In order to be able to adapt to the surgeon’s requirements, the position and/or orientation of the slots (92) or guide orifice are adjustable with respect to the guide block (9).

[0078] As an example, with reference to FIG. 7, the guide block (9) comprises a principal portion (93) intended to be secured to the bone by the securing pins (11), and a guide slot (92) formed in a secondary portion (94) of the block, connected to the principal portion via a ball joint (95). Means for clamping (96), for example by screwing, are secured to the ball joint (95) to prevent it from pivoting. The secondary portion (94) of the guide block (9) comprises a three-dimensional visual marker (97) carrying information relating to the position and the orientation of the slot (92) in space which the processing system (3) is capable of processing when it is visualized by the camera (4).

[0079] In other words, it is possible to know the position and the orientation of the slot (92) precisely with respect to the visual marker (22) of the tracking element (2), and therefore with respect to the bone itself. The information integrated into the three-dimensional visual marker (97) of the secondary portion (94) of the guide block (9) enables the processing system (3) to display a cutting plane or drilling axis linked to the secondary portion (94) of the guide block (9) on the display unit (6), for example, which is superimposed on the field of vision of the surgeon and perfectly positioned with respect to the real bone, without inaccuracy relating to the extra thickness of cartilage.