LIGHT-POWERED LIGHT-EMITTING TISSUE MARKER

Abstract

Method and related system for marker-based navigation. A first image is capturing (S530) with a medical imaging apparatus (ENDO), whilst a light source (LS1) of an implanted marker device (MD) is in a low intensity state and a second light source (LS2) of the medical imaging apparatus (IA) is in a high intensity state, higher than the low intensity state of the marker. A second image is captured (S560) with the medical imaging apparatus (ENDO), whilst the light source (LS1) of the implanted marker device is in a high intensity state and the second light source (LS2) of the medical imaging apparatus (IA) is in a low intensity state, lower than the high intensity state of the marker. The two images may then be combined to obtain a combined image that represents the location of the marker at high signal-to-noise-ratio.

Claims

1. System for marker-based navigation, comprising: an implantable marker device having a first light source, switchable into at least two intensity states, a low intensity state and a high intensity state; a synchronizer logic; and an imaging apparatus with a second light source to provide incident light, the said second light source switchable into at least two intensity states, a low intensity state and a high intensity state; wherein the first light source of the marker device and the second light source are operable in synchrony by the synchronizer logic controlling the switching of the two light sources, the system thereby operable in two modes, in a beacon mode and in an exploration mode, so that, in exploration mode, the first light source is switched into low intensity state and the second light source is switched into high intensity state, and so that in beacon mode, the first light source is switched into high intensity state and the second light source is switched into low intensity state, wherein the imaging apparatus is configured to be controlled by the synchronizer logic, or by a user, so as to capture at least one respective image in each of the two modes.

2. System of claim 1, including an image processor configured to combine image information from the at least two images to form a combined image.

3. System of claim 1, the image processor comprising a contrast enhancer to enhance contrast in the one or more images acquired in beacon mode.

4. System of claim 1, the imaging apparatus including a light filter component having its frequency response adjusted to a bandwidth of the light emittable by the light source of the marker device.

5. System of claim 1, the image processor configured to spatially register the image captured in beacon mode and the image captured in exploration mode.

6. System of claim 1, wherein the synchronizer logic comprises a light sensitive sensor, the light sensitive sensor operable to effect the switching into the two intensities of the first or the second light source, in dependence on incident light received from second or the first light source, respectively.

7. System of claim 6, wherein the light sensitive sensor is configured to facilitate i) the first light source to switch into high intensity state in response to the sensor sensing a light intensity below an intensity threshold and/or ii) the first light source to switch into low intensity state in response to the sensor sensing a light intensity above the threshold.

8. System of claim 7, wherein the light intensity sensed at the sensor is caused by the incident light from the second light source.

9. System of claim 1, wherein the imaging apparatus comprises an endoscope.

10. System of claim 6, wherein the synchronizer logic is configured to automatically switch the marker device into beacon mode when an intensity of the incident light received from the second light source drops under an intensity threshold.

11. A medical imaging device, comprising an image sensor to capture an image and a light source to provide ambient light for the light sensor, and a synchronizer logic operable to cause the light source being switched into at least two intensity states, a low intensity state and a high intensity state, whilst capturing images, the said switching on and off being set at a preset frequency or wherein the switching between the two intensities is in response to an external switching signal received by the synchronizer logic through an interface of the device.

12. Method for marker-based navigation, comprising the steps of: capturing a first image with a medical imaging apparatus, whilst a light source of an implanted marker device is in a low intensity state and a second light source of the medical imaging apparatus is in a high intensity state, higher than the low intensity state of the marker; and capturing a second image with the medical imaging apparatus, whilst the light source of the implanted marker device is in a high intensity state and the second light source of the medical imaging apparatus is in a low intensity state, lower than the high intensity state of the marker.

13. Method of claim 12, comprising: combining the two images and, optionally, displaying the combined image on a display device, and or displaying the two images.

14. A computer program element, which, when being executed by at least one processing unit, is adapted to cause the processing unit to perform one or more steps of the method as per claim 12.

15. A computer readable medium having stored thereon the program element of claim 14.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Exemplary embodiments of the invention will now be described with reference to the following drawings, which are not to scale, wherein:

[0074] FIG. 1 shows a block diagram of a system for marker based navigation;

[0075] FIG. 2 shows switching cycles of two synchronized light sources;

[0076] FIG. 3 shows a block diagram of an implantable marker device;

[0077] FIG. 4 shows circuitry of a marker device; and

[0078] FIG. 5 shows a method for marker based navigation.

DETAILED DESCRIPTION OF EMBODIMENTS

[0079] With reference to FIG. 1, this shows a schematic block diagram of a system SYS for marker assisted navigation as may be used in medical interventions such as minimally invasive surgery (MIS) interventions, in-situ inspections or similar. In particular, the system SYS preferably supports imaging during such interventions.

[0080] Medical applications, in particular MIS applications, include lung interventions such as VATS. VATS interventions include the removal of lesions from lung tissue, for example wedge resections and similar. Other MIS medical applications may include inspection of intestines, urethra, esophagus, liver, and other organs or groups thereof.

[0081] Although medical applications are mainly envisaged herein, non-medical applications are also envisaged, such as navigation in plumbing systems, in cave systems or in other difficult to access environments where a certain target needs to be located.

[0082] The system SYS includes broadly a marker device MD which is implantable in human or animal patient PAT. The marker device includes a light source LS1, herein also referred to as “first light source”. The marker further includes a switch SW1 to switch the light source LS1 on and off, the purpose of which will be explained more fully below. The system SYS further includes a medical imaging device ENDO, such as an endoscope. Although main reference will be made in the following to endoscopic imaging, other imaging modalities are also envisaged herein, such as a digital camera, a fiberscope, a rhinoscope, an arthroscope, a laparoscope, a surgical microscope, a telescope, and other. Capsule endoscopes are also envisaged, that are administered into the patient as whole.

[0083] A part of the endoscope ENDO is introducible into the patient to acquire images to provide visual support for navigation and intervention. A light source LS2 (herein also referred to as “second light source”) of the endoscope ensures sufficient exposure when acquiring images from inside the patient PAT. The marker device MD was implanted into the patient PAT close to a lesion, in a procedure prior to the endoscope supported intervention to be described herein.

[0084] Very broadly, in the system SYS as envisaged herein, the second light source LS2 of the endoscope ENDO and the first light source LS1 of the marker device MD act together to assist the user in navigating to the marker location and hence to the lesion. The two light sources are switched in synchrony to acquire imagery that reveals the location of the marker MD and hence of the lesion. The imagery may be processed by an image processor IP to better visualize the marker location. The image processor IP may facilitate displaying of the imagery on a display device DD to yet better support the user during the intervention.

[0085] Before explaining operation of the system SYS in more detail, we refer first to the imaging device ENDO to describe some of its components.

[0086] The endoscopic imaging device ENDO allows the user to “see” inside the patient. The imaging apparatus ENDO operates preferably using non-ionizing radiation, such as visible light. The imaging device includes a supply unit SU, optionally a hand-piece HP and an insertion tube IT.

[0087] The supply unit SU may include the light source LS2. The light source LS2 may be powered by an onboard power source PS2 in the supply unit SU, or it may be connectible to an external power source. The light source LS2 is switchable on and off by a switching element SW2. Specifically, the switch SW2 may operate to connect or reconnect the light Source LS2 to the power source PS2.

[0088] The insertion tube IT may be connected to the hand piece HP. The hand piece allows operation by the user of the endoscope ENDO. The insertion tube is for at least partial insertion into the patient PAT. The insertion tube IT has sufficient stiffness so that it can be urged into the patient and may be advanced through the patient. The insertion tube IT may include a metallic mesh coated by a polymer. The insertion tube terminates in a distal tip portion TP.

[0089] In embodiments, there the imaging apparatus may include a steering mechanism. In particular, a distal portion of the tube TP including the tip TP forms a bendable section. The bendable section is articulated by a series of inter-joined segments. A plurality of wires, such 2, 3, 4 or more are attached to the tip at different anchor points. The wires run parallel the bending section, inside the tube IT and the full length thereof, from tip TP to hand piece HP. In the hand piece, the wires couple into an actuator such as a mechanical chain-sprocket arrangement or other. The user can tension the wires by way of the actuator, to force the tip to curl and bend into different spatial directions as desired, depending on which wire is tensioned most. The actuator responds to knobs or levers at the hand piece operable by the user. The bendable section thus allows the user to navigate and negotiate obstacles.

[0090] To the lower right in FIG. 1, there is shown a cross section of tube IT at the tip TP through a plane p. Inside the insertion tube there are a number of channels, such C1-C3. One channel, the working channel C1, allows introduction of medical tools to perform the intervention or the removal of re-sected tissue by application of a vacuum for instance. The other two channels C2 and C3 include respective fiber cables FCD, FCU which run the full length of the insertion tube IT from the distal tip TP portion back into the supply unit SU. The fiber cables may terminate at tip portion TP in respective lenses (not shown).

[0091] The light source LS2 is to provide illumination, in particular so that imagery at sufficient exposure can be acquired. To this end, and in some, but not all embodiments, the light source LS2 is optically coupled to one of the optical fibers FCD to send light down the FCD cable, to the tip portion TP and into the surroundings to provide illumination thereof. Light reflects off structures encountered by the outgoing light and is captured by the second fiber cable FCU. The captured light travels up that second fiber cable FCU which also runs the full length of the tube IT, back to the supply unit. The second fiber cable FCU, at its proximal end, is optically coupled to a camera system that includes the image sensor LD to capture imagery. Acquisition Circuity (not shown) allows capturing imagery at a suitable frame rate, such as 20-30 fps or other.

[0092] The imagery so captured may then be processed by the image processor IP, on which more further below. The imagery is processed into a video feed which can be displayed on the display device DD in grey scale or in color or in any other suitable rendering. Still imagery may also be acquired and presented for display, if required. There may also be an optical eye piece to observe the inside of the patient instead of or in addition to the camera system described.

[0093] It will be understood that the described endoscopic system ENDO is merely according to one non-limiting embodiment. Numerous other endoscopic imaging system in a multitude of variations are also envisaged herein and may be used in the context of the described system SYS. For instance, the camera system may be situated outside the supply unit SU. Not all endoscopic imagers may have a dedicated hand piece HP as described. The tip portion may be structured differently.

[0094] Not all endoscopes may necessarily have an articulated bending section as described and the number of channels may be different from what was described above.

[0095] Also, the optical fibers UFC, DFC are not necessarily required in all embodiments. For example, in alterative embodiments, the camera system including the image sensor TD (such as a CCD) and one or more lenses may be arranged at the tip TP. In addition or instead, he the light source(s) LS2, such as one or more LEDs, might also be arranged at the tip TP.

[0096] Furthermore, surgical endoscopes such as laparoscopces, thoracoscopes, arthroscopes, etc., may not necessarily have a working channel. In surgical procedures, instruments are usually introduced into the patient via through additional access points.

[0097] Endoscopic imaging is not necessarily required herein in all embodiments. Specifically, in embodiment as surgical instrument where the insertion tube is a needle or trocar. In this or similar cases, the insertion tube is stiff, does not include a bendable section, and there is no steering mechanism. The camera system TD and light source LS2 are again both arranged at the tip of this stiff insertion tube.

[0098] As indicated, the system SYS allows the user to localize, within a patient PAT, a site of interest, such as lesion, in proximity of which the marker MD has been implanted in an earlier procedure. Specially, the marker MD may have been introduced into the patient through a medical needle, a catheter, a working channel of an endoscope, or other delivery device, in a preparatory phase before the endoscope or other imaging device supported intervention. The marker may be delivered without any endoscope, e.g. transthoracically with a needle under CT guidance or similar. At delivery, the marker MD may be plunged into tissue TS, close to where the lesion is located. More than one marker may be implanted so as to circumscribe the location of the lesion. The implanting of the marker may have been carried during X-ray imaging guidance, such as CT. Later, the site may need to be revisited in a different procedure to carry out the intervention, and it is these and later procedures that are the main focus herein. Instead of the described endoscopic imaging device ENDO, other imaging devices may be used in such procedures to navigate to the lesion site.

[0099] Because some imaging devices, such as some endoscopic imaging devices ENDO, use non-ionizing radiation for imaging, a line of sight through air is required to image structures of interest. It may be the case that the lesion, and hence the marker MD planted close to it, is located inside the tissue or organ, behind a tissue/organ outer surface SR. So even if the endoscope ENDO is introduced into a cavity CAV bounded by the surface SR, there may still be no line of sight between the endoscope and the marker device MD because of occlusion by intervening tissue TS.

[0100] To address this, the light source LS1 of the marker device MD is configured to emit light of sufficient intensity so the light is capable to penetrate through the intervening tissue for a reasonable distance, and possibly into the cavity CAV (if any), to be detectable, albeit attenuated, by the image sensor LD of the endoscopic imager ENDO.

[0101] It is envisaged herein to switch on and off, in synchrony, the two light sources LS1, LS2 of the marker device MD and of the endoscope ENDO, respectively. This allows the user to better localize the marker device possibly buried in surrounding tissue with no direct line of sight. More specifically, it is envisaged herein that the light source of the marker MD is switched off when the light source LS2 of the imaging device ENDO is switched on, and, vice versa, that the light source LS1 of the marker MD is switched on when light source LS2 of the endoscope is switched off. In addition, imaging by the imager ENDO is also synchronized so that, in either of the two synchronized states, one or more images are acquired by the imager ENDO.

[0102] As the distal tip portion TP of the endoscope ENDO resides during the intervention inside the patient there is very low ambient light, or even next to complete darkness. Imaging whilst the illuminating source LS2 of the endoscope is off will therefore lead to dark images with flat signal distribution, expect for an isolated light signal that is recorded as a local bright “splodge” because the marker MD's light source is on whilst the endoscope's light LS2 is off. The one (or more) dark image with the isolated bright signal may be referred to herein as a “beacon image”. The beacon image may thus provide the user with a clear clue as to the location, or at least a direction, where the marker device MD is located. Once one or more of such beacon images, with the localization information encoded, are acquired, the light source of the endoscope ENDO is switched on again but now the light source of the marker MD is switched off. The endoscope ENDO continues to acquire one or more images, but this time the images, which may be referred to herein as “exploratory images”, are now fully illuminated to reveal of anatomic structures of surroundings that were missing in the beacon images. However, in the exploratory images it is now the location of the marker that is missing.

[0103] The synchronization of switching the two light sources LS1 and LS2 in this manner during imaging hence leads to two intertwined streams of images, the fully illuminated ones and the dark images. The imagery from the two streams may be displayed in alternate fashion. It is proposed herein to process by the image processor IP the images from the two streams in combination to obtain the complete information, namely the location of the marker and the anatomical information.

[0104] By selectively switching the two light sources on/off in synchrony, the system SYS overall operates in two modes, in beacon mode and in an exploration mode. When in beacon mode, only the marker device light source LS1 is on whilst the endoscope light source LS2 is off and the imager ENDO captures the beacon or dark images. When in exploration mode, the roles are reversed, in that the marker MD light is now off whilst the light source LS2 of the endoscope is on to capture the exploratory, fully illuminated images.

[0105] Oscillation between those two modes, that is the synchronization of the switching operations at the marker MD and endoscope ENDO, may be administered and controlled by a synchronizer logic SL. At least a part of the synchronizer logic SL may be located external to the endoscope and the marker device, implemented on one or more data processing units, coupled through wired or wireless communication with remaining parts of the synchronizer logic SL. Alternatively, the synchronizer logic may be partly or fully integrated into the marker device or partly or fully integrated into the endoscope. One part of the synchronizer logic SL may be integrated into the endoscope and another part in the marker MD, as required.

[0106] The synchronizer logic SL ensures out-off phase switching cycles or pulsing frequencies of the two light sources LS1 and LS2. A pattern of switching cycles of the light sources as envisaged herein is shown in diagrams a) and b) in FIG. 2. Diagram a) pertains to the imager light source LS2. Diagram b) pertains to the marker MD light source LS1. The two diagrams show respective light intensity I-versus-time-t curves for light sources LS1 and LS2. Zero intensity period (referred to herein as “switch off period”) corresponds to a period where the respective light source LS1, LS2 is off. The non-zero intensity periods (referred to herein as “switch on period”), that is the light pulse length, corresponds to a period during which the respective light source LS1, LS2 is on. The flanks mark the switching events, either from “on” to “off” or from “off” to “on”. Specifically, curve a) represents the switching cycle of the endoscope light LS2 and curve b) represents the switching cycle of the marker device source LS1. In particular, a switch-on period of the marker device in b) falls within a switch-off period of the endoscope cycle and, vice versa, the switch on period of the endoscope falls within the switch off period of the marker device.

[0107] Preferably the switch on period of the marker light source is shorter than the switch on period of the endoscope, such as in circumstances, where fewer beacon images are acquired than exploratory images. In in extreme example, the switch on period is only for a single beacon image to be acquired so the switch on period of LS1 is quasi-instantaneous, with a spiked pulse and with a relatively long switch off period. In contrast, the switch on period of the endoscope light SL2 is longer, allowing the acquisition of a plurality of exploratory images.

[0108] In embodiments, a switch off period of one of the light sources coincides with the switch on period of the other light source.

[0109] The frequency of the switching cycle of the marker device light source may be the same or may differ from the frequency of the switching cycle of the endoscope light source.

[0110] The cycles may not to be static, but one or both may be variable, adjustable by the user. In one embodiment, one setting may correspond to the images of the two streams strictly alternating, every one beacon image followed by one single exploratory image which is then followed by one single beacon image and so forth. This cycle setting may be useful if the user has difficulties locating the marker. However once found, the marker light frequency can be decreased. Generally, however the cycles are such that one or more beacon images are followed by a run of exploratory images. Furthermore, cycles may not necessarily be regular. In fact, in some embodiments, the cycle of the imager light source LS2 may be irregular, the light LS2 being switched on and off at the user's leisure.

[0111] Preferably, some or each switch off event at the imager light source LS2 causes a switch on event at the maker device light SL1. Preferably, a switch on event at the imager light source LS2 causes the light source LS1 of the marker to switch off, although this is not necessarily required in all embodiments, as the light source LS1 in the marker may switch off, for instance on its own, before the imager light source LS2 switched on as shown in FIG. 2.

[0112] When in beacon mode, the light source LS1 of the marker may give of light continuously for a preset period of time or the light is given off in pulsation, as a series of pulses at a frequency for a preset period of time, or until the imager light source LS2 is switched on again.

[0113] It is not necessarily the case in all embodiments that the imager's illumination source LS2 switches on at each and every instant when the marker light source is off, such as during pulsation of the marker light source LS1 as triggered by an earlier switch off event at the imager's illumination source LS2.

[0114] In general, the synchronizer logic SL may be implemented as causally linked or where there is no causal link between the two switching of switches SW1, SW2. These embodiments may be referred to herein as a causally linking synchronizer logic SL or a non-causally linking synchronizer logic SL.

[0115] Referring now first to the causally linked synchronizer logic SL, this may be implemented partly at the marker device as will now be explained with reference to FIG. 3. The synchronizer logic SL may implement a cycle pattern as shown in FIG. 2.

[0116] Specifically, FIG. 3 shows a schematic block diagram of a marker device MD arrangement according to one embodiment. FIG. 3A shows the marker device illuminated by ambient light, for instance incoming IL light that is coming from the light source LS2 of the imaging apparatus ENDO. FIG. 3B shows a state where there is no incoming illumination IL, but instead it is now the marker device's own light source LS1 that is activated and outgoing light OL issues forth from the marker device.

[0117] The circuitry of the marker device includes the above mentioned light source LS1 and an onboard power source PS1 that powers the light source LS1. The power source may be a re-chargeable battery, a non-rechargeable battery or a capacitator or other. The switch SW1 is operable to connect and reconnect the light source LS1 to the power source PS1. For ease of representation, in FIG. 3 the marker MD's switch SW1 is not shown.

[0118] The light source LS1 may be arranged as a light emitting diode (LED), or similar. In embodiments, red or NIR radiation is preferred in case of LEDs as this ensures sufficient tissue penetration depth. Alternative, a high power (white) light bulb or a stroboscope light is used.

[0119] In a preferred embodiment, the circuitry of the marker device MD further includes a transducer PC. The transducer PC may be arranged as photo cell or photo diode. The transducer PC is capable of converting incoming light IL, such as from the endoscope light source LS2, into electrical energy so as to power the light source LS1 of the marker MD.

[0120] The transducer PC may either supply directly the light source with electrical energy but preferably there is a buffer storage element, such as the capacitator or re-chargeable battery PS mentioned above, that stores the energy converted by the transducer PC and the light source LS1 draws the electrical energy from the power storage element PS.

[0121] Some or all of the mentioned components, that is, the light source LS1, the power source PS and the converter PC, if any, are preferably arranged on a single semi-conductor chip or a PCB although this may not be necessarily so in all embodiments where some or all components are distributed across two or more semi-conductor chips.

[0122] In a preferred embodiment, but not in all embodiments, the marker device MD further includes a light sensitive sensor LXS. The light sensitive sensor forms part of the causally linking synchronizer logic SL. The light sensor LXS facilitates switching on the light source LS1, if no ambient light is sensed by the sensor LXS, for instance, in response to switching off the endoscope light source LS2. In addition, the light sensitive sensor LXS facilitates switching off light source LS1 in response to sensing incoming light, such as light form the endoscope source LS2. The light sensitive sensor LXS and the transducer PC, if any, may be arranged in a single unit such as a photo diode or photo cell, or may be arranged discretely and separately as different components. In simpler embodiments, there is no light sensor LXS and the power source is a rechargeable battery or, less preferred, a non-rechargeable battery.

[0123] The light sensitive sensor establishes a causal link between the switching events at the endoscope and the marker MD. The switching at the marker responds to events at the endoscope. The switching events and imaging at the endoscope may be purely user controlled and triggered by the user operating a button or other actuator. Alternatively, there is an automatic switching circuitry in the imager ENDO, such as a multiplexer to toggle/switch between different predefined modes or a multi-vibrator or other which the user can set up by specifying through a user interface the switching cycle. The source LS2 is then repeatedly switched on and off at the specified frequency and period lengths and whilst the imaging continues. The marker MD, thanks to sensor LXS will then respond accordingly as per pattern as in FIG. 2 b). In this and similar embodiments, the imager may include an interface IFE to receive the switching command from the switching circuitry.

[0124] If the imaging device ENDO is of the endoscope-type, the insertion tube IT may include a further channel (not shown in FIG. 1) dedicated for delivering light from a different, second light source of the imager. For example, a non-visible near infrared (NIR) light channel may be used to power up the marker so as not to interfere with light through another channel to provide the visible illumination light from source LS2 for imaging purposes. There may also be an additional working channel (not shown), in addition to the working channel C1, suitably dimensioned with a lumen for delivery or removal of the marker.

[0125] Although in some embodiments the light source LS2 is integrated in the medical imaging device ENDO, this is not necessarily so in all embodiments, where the illumination source is separate from the imager imaging device ENDO. The illumination source may be introduced into the patient through a different delivery tool.

[0126] FIG. 4 shows further details of the marker MD circuitry configured to implement the marker device MD as per FIG. 3 according to one embodiment. The marker device MD includes the light sensitive sensor LXS and, optionally, the transducer PC. The synchronizer logic SL comprises the light sensitive sensor LXS and logic circuitry such as a flip-flop FF as shown in FIG. 4. Incident light IL, such as coming from the endoscope light source LS2, is shown in dashed arrows. When light is received, the transducer converts this into energy and stores the same in the power source PS such as a battery through charging circuit CC. The circuitry is operable in charging mode and in a mode that give rise to the earlier mentioned beacon mode.

[0127] More specifically, when incident light IL is sensed at the sensor LXS and if a transducer PC is present, the power source PS is charged by transducer PC and the light source LS1 is switched off, with no power supplied to the light source LS1. However, when no light is received at the sensor LXS, the transducer is no longer charging and the device MD switches into beacon mode. Now, power source PS1 does provide power to the light source LS1 to release light (shown in solid arrows).

[0128] In more detail, and with continued reference to FIG. 4, the marker's illumination source LS1 can be operated in a pulsed on/off mode. During the switch on period, the sensor LXS captures photons and charges the energy storage element PS1, such as capacitor. When the illumination IL from the second light source LS2 is discontinued, the marker device is automatically switched into beacon mode by means of the synchronizer logic SL. In this mode, the storage element PS1 now powers the light source LS1 and the light source emits a light signal, either pulsed or in continuous wave mode (cw). When the sensor LXS receives photons, it is in charging mode. In the charging mode, the output voltage at the sensor LXS is higher than the voltage at the energy storing element PS1. When the illuminating beacon light source PS1 is switched off, the output voltage at the sensor LXS drops and will be lower than the voltage supplied by energy storing element. These oscillating voltages U1, U2, can be processed by a flip-flop trigger circuitry FF as part of the synchronizer logic SL to trigger switch SW1. Once so triggered, the stored energy from the storing element PS1 can flow as an electrical current through the light source LS1 which causes emitting of light. The voltage drawn from the power source PS1 in beacon mode power may be provided by an LED driver circuit DCR which enables the light source LS1 to illuminate.

[0129] In embodiments when the power storage PS1 of the marker MD is recharged by light captured at the light sensor LXS or other charging arrangement, the switch off period preferably is at least as long as the recharge period as shown in FIG. 2. However, it is in particular in non-causally linked embodiments of the logic SL where there is no sensor LXS required, and one or both light sources are powered by batteries or by wired or wireless connection to an external power supply.

[0130] The pulse length of the emitted light pulse, that is the length of the switch on period, may depend on the amount of energy stored in the capacitor. If the sensor LXS is in close vicinity to the illuminating source LS2 of the imager, more energy is stored and the maker light source LS1, the “beacon”, may emit light for a longer period of time with a longer switch on period. The switch on period of the marker may hence be used as a “surrogate” measure to determine the distance from the illuminating source LS2 and the beacon LS1. In other words, in embodiments, a processing unit PU, either integrated in the imager ENDO or external thereto, can convert the switch on period as measured through a series of beacon images into a distance. The distance so computed may be displayed on the display device, overlaid on the current exploratory image.

[0131] In some embodiments of FIGS. 3,4, it is the user who switches on/off, with a manual switch, the light source LS2 at the imager ENDO, as required, preferably when requesting imaging in exploratory or beacon mode. When switching off the light source LS2 at the endoscope, and as a direct response thereto, the light from illumination source LS2 is now received at the light sensor of LXS of the marker device, and the synchronizer logic SL then operates the onboard switch SW1 in the marker device to switch on the light source of the marker device as described above. It will be understood that in this embodiment, a light intensity at which the sensor LXS causes the switching off of the marker source LS1 may need to be adjusted, for instance in a calibration setup, to ensure that his threshold is not set too high. For instance, the maker light source LS1 should preferably be not merely switch on when the illumination source LS2 is pointed away from a direction of the marker as the user examines the surroundings.

[0132] A converse arrangement to the one of FIGS. 2,3 is also envisaged, where the light sensor LXS is instead integrated into the endoscope. In embodiments, the sensor LXS is arranged on the insertion tube at its bending section or at the insertion portion of a delivery device such as a needle, etc. Specifically, the sensor LXS may be integrated in the tip TP portion. In this embodiment it is now the marker device that controls the switching off of the endoscope light source LS2 as opposed to the above mentioned embodiments FIGS. 3,4, where it is the endoscope light that controls the switching off of the light source at the marker device. In this converse embodiment, the switching signal interface IFE may include the light sensitive sensor in order to receive the switching on/off commands, that is, the light signals from the marker light source LS1, to switch the switch SW2 accordingly. This converse arrangement may be useful to ascertain the optical transparency of the surrounding tissue.

[0133] The embodiments in FIGS. 3,4 described above, where the marker device includes the light sensor LXS to facilitate switching on/off the onboard light source LS1, implement an example of causal synchronization logic SL. Switching events, in particular a switch off event at the marker device responds flexibly and dynamically to the light received from the endoscope light LS2. But FIGS. 3,4 are merely one embodiment for a causally linked synchronization logic SL. Other, switching event triggered logics that detect hard switching events are also envisaged herein in alternative embodiments. In these embodiments, interface and event handlers at the marker MD and/or endoscope ENDO capture “hard” switching events and then transmit a trigger signal to effect switching on or off the other switch of imager ENDO or marker MD to achieve a similar switching cycle pattern as in FIG. 2. In these embodiments that are driven by hard switching events, the endoscope ENDO and the marker device MD may each include preferably a wireless transmitter/receiver arrangement that are coupled to the respective switches SW1, SW2. The switch event one in the endoscope or the marker device will then be transmitted, for instance from the marker device to the endoscope and to effect switching the endoscope on if the light source in the marker has been switched off or vice versa.

[0134] In the above described embodiments with causally linked synchronizer logic SL, it may not necessarily be required for the synchronizer logic to issue a specific switch off signal to cause the marker device light source LS1 to switch off so that the imager ENDO can operate in exploration mode, although this may still be implemented in some embodiments. Rather, as shown in the switch cycles of FIG. 2, the marker light source LS1 may switch off automatically, either by expiry of a timing circuit or simply because of the power source being drained after sustaining a beacon pulse. The illumination source LS2 at the endoscope may then at one point be switched on again, e.g. at user request, and the marker power source PS1 is then recharged.

[0135] Turning now to the earlier mentioned non-causally linking synchronizer logic LS, in such embodiments the lights sources LS1, LS2 may be switched autonomously, and yet the switching cycles as per FIG. 2 or variants thereof can still be achieved. Such non-causally linking synchronizer logics SL may be implemented as two respective oscillating switching circuitries, one in the marker MD and one in the imager ENDO. The switching cycles are set up with pulse lengths for LS1, LS2 suitably timed, and out of phase. The user adjusts through a suitable interface at the imager and marker, frequency, phase and the respective lengths of switch off and switch on times so that the two cycles run autonomously out of phase similar to the diagrams of FIG. 2. The marker may include a wireless set up functionality for the user to change its switching cycle post-implantation. There is no more a causal connection between the two switching logics and the system SYS can operate as described above. Two oscillator or multi-vibrators may be used, respectively integrated in the marker MD and the imager ENDO. Preferably, the switching off of the illumination light LS2 of the imager ENDO should be timed so as to allow the marker's MD power source PS1 to recharge if a light sensor LXS is used.

[0136] Referring now back to FIG. 3 to describe the marker device MD more fully, the circuitry of the marker device is encapsulated or enclosed wholly or partly by a bio-compatible material CP. The material is preferably wholly translucent or may be only translucent at window portions W but is opaque elsewhere. In embodiment it is the whole of the encapsulation CP that is translucent. Translucent encapsulation ensures that more light can be sensed and released.

[0137] The translucent window portion(s) W are arranged where the light source and optionally the transducer PC are located. A single window portion W may be as shown in FIG. 3 with suitable width to serve both, the light source and the transducer and/or light sensor. Alternatively, the transducer PC, if any, and the light sensor LXS and the light source may each have a separate discrete window portion (not shown).

[0138] Glass or other biocompatible materials are envisaged herein such as suitable polymers, or organic or inorganic translucent materials. Biocompatibility ensures in particular that toxic substances are not released into the surrounding tissue. The marker is not intended to stay long term in the patient but may be removed by a tool passed through the working channel of the endoscope ENDO at the conclusion of the intervention or in a follow-up intervention or may be removed together with the surrounding tissue during a surgical excision. Biocompatibility should be ensured at least for periods of days, preferably weeks or months because, due to pressure on health systems, it may well happen that considerable time may pass between implanting the marker and the concluding intervention.

[0139] The overall shape of the marker device can be any one of ball shape, ellipsoid shape but may also be arranged as a strip. Any other shapes are also envisaged. The overall dimensions may be in the millimeter or sub-millimeter range. Preferably the marker device is miniaturized so as to be able to be administered through a medical surgical needle. Examples include, biopsy needles of gauge 12-18, about 2-1.25 mm in diameter, but thicker needles in the range of 3 mm-5 mm, such as for ablation purposes, may also be envisaged.

[0140] The overall shape of the encapsulation CP is generally convex so that it can be easily administered through the needle as mentioned. Alternatively, however, the encapsulation CP may not necessarily be convex, in particular, it may include protruding structures to form hooks or other anchorages so as to assist in affixing the marker to the intended tissue type or organ. The encapsulation CP may be monolithic or may be assembled from components or parts.

[0141] With continued reference to FIG. 3, the marker device MD may optionally include further circuitry or components that allow the marker device to collect measurement data of physical, chemical or physiological quantities. One or more sensors or probes S may be so arranged. The measurement values may be stored in an onboard memory and can then be evaluated once the marker device is removed. Alternatively, there is a dedicated transmitter that transmits to the outside measurement data which can be evaluated by a receiver, such as a computing device. In one embodiment there is no dedicated transmitter, but a modulator MOD that modulates the measurement values onto the light emitted by light source LS1. The modulation may occur during beacon mode, or may occur instead in a separate operational phase when light source LS1 is solely activated to transmit the measured values. Modulation may be through frequency modulation or intensity modulation or a combination.

[0142] The transducer TD, although preferred, is optional, and the marker MD may be powered by an on-board battery or other energy source, without the transducer, especially when used with the non-causally linking synchronizer logic SL. Preferably, however a transducer is used in the marker MD to recharge the marker light source LS1 during exploration mode, suing the light from the imager ENDO light source LS2. The transducer TD may be used in the causal or non-causal embodiments of the synchronizer logic SL.

[0143] Turning now in more detail to the operation of the image processor IP, this may extract image information from the dark or beacon images. In particular, the location of the isolated signal, the bright spot, may be determined by thresholding or segmentation. The location so determined may then be suitably indicated in the follow up one or more exploratory images. A suitable graphical symbol such as hair-cross or other may be overlaid in the exploratory images at the corresponding in-image location in the one or more subsequent exploratory images to provide a clue to the user of the location of the marker device. Alternatively, the whole beacon image is overlaid onto the exploratory image. “Dark” areas of the beacon image are rendered completely transparent, whilst light areas that represent the marker light signal, are rendered opaque or only partially transparent. The marker location, and the associated graphical symbol, is dynamically updated when a new beacon image is acquired, and so on. In other embodiments, the bright spot is segmented, and it is the segmentation that is then overlaid.

[0144] If the movement of the endoscope in between two frames, in particular between acquisition of the beacon image and an exploratory frame, is zero or negligible, the two images are naturally registered spatially with one another. In other words, the same in-image co-ordinates in the two images correspond to the same spatial position. In this case, no dedicated registration operation is required, and the marker position that can be readily indicated in the exploratory image based on the signal in the earlier or later beacon image. Otherwise, if there is non-negligible motion between the frames and hence no direct native registration may be assumed, a registration functionality of the image processor may be employed to attempt spatially registering one of the earlier or later beacon frames and a follow-up or earlier exploratory frame. In situations in which only exploratory images can registered with each other, a beacon image is then registered on temporally neighbored exploratory image(s), which are then in turn registered to later acquired exploratory images, so as to ensure that also those later exploratory images may be faithfully registered with the beacon image.

[0145] Morphing algorithms, optical flow methods or others may be used to effect the registration. The image registration processing may be beneficial in particular in situations where the time to recharge the marker power source PS1 is relatively long, and where the period between subsequent LS1-pulses might lead to spatial a mismatch between the beacon image and the current exploratory image. The proposed image registration allows to address this.

[0146] In embodiments, before attempting to locate the marker position in the beacon image, a contrast enhancer CE may be used to contrast enhance the image and to then attempt the identification. A color change may also be attempted instead or in addition to contrast enhancement.

[0147] In order to capture better beacon images of enhanced quality, the image apparatus ENDO may include a filter component FC. The filter component has a frequency response function that corresponds to the bandwidth of the light source LS1 of the marker. The light from the marker device MD captured by the endoscope ENDO during beacon mode is then first filtered by the component FC, and it is the so filtered image that is then processed as described above.

[0148] In sum, as per the above embodiments, the image processor is configured to combine at least a part of the beacon image with the exploratory images to form a combined image that may include additional graphical information to indicate the location of the marker device to better assist the user in navigating towards the device.

[0149] However other visualization options of the marker location are also envisaged. For instance, in embodiments it is the beacon image itself that is displayed, for instance by interleaving the beacon images into the stream of exploratory images. The so interleaved beacon images may be displayed for a certain amount of time. The time may be set by the user. The time may be short, such as a fraction of a second, or in the order of one or more seconds.

[0150] Alternatively, the beacon images are displayed as a separate stream alongside the stream of exploratory images, either in different, respective, screen portions of a single display device, or on two display devices, respectively.

[0151] Reference is now made to FIG. 5 which slows a flow chart of a method for marker based navigation. The method can be used to implement the system as described above in FIGS. 1-4 but the following steps of the method may be understood as a teaching in its own right.

[0152] At step S510, the light source of the implanted device MD is switched off.

[0153] In step S520, the light source LS2, the illumination source, at the medical imaging apparatus such as an endoscope is switched on.

[0154] The switching on of the light source may occur in a response to a signal received in connection with the switching off of the light source of the implanted device. The order of the two steps S510 and S520 may be reversed. Now that the second light source of medical imaging device that illuminates the field of view of the medical device is on, whilst the light source at the implanted device is switched off, one (or more) image is captured with the imaging apparatus at step S530. This image may be referred to as an exploratory image and constitutes an image suitably exposed thanks to the second light source.

[0155] Once one or more of such exploratory images have been captured, the light source at the medical device is switched off at step S540. The light source at the marker device is switched on at step S550. Again, the steps S540 and S550 may be reversed.

[0156] In embodiments, the switching on in step S540 of the marker light source responds to the switching off in step S550 of the illumination source at the imager. Now that the second light source at the imager is off and the light source at the implanted device is switched on, a second (one or more) image is captured by the medical imager at step S560. This second image may be referred to as the beacon image. This image will capture an isolated light signal, a light spot, from the marker device to so indicate the location of the implanted marker device (and hence of the lesion) although the marker it may not necessarily be in a line of sight relative to the medical imaging apparatus. Because the beacon image is now underexposed and may not reveal the surroundings as the illumination source is off, the location of the marker is precisely indicated thanks to the isolated light signal from the marker device as recorded in the beacon image.

[0157] At an optional step S570, at least a part of the image information in the dark image and at least a part of an exploratory image may be combined.

[0158] In particular, the light spot representing the light from the marker device may be identified in the beacon image through segmentation, signal thresholding or other image processing. The in-image position of the so identified light spot may be indicated in the exploratory image, to so reveal to the user the location of the, possibly occluded, marker and hence of the lesion. The location may be indicated by overlaying a visual marker at the corresponding location in the exploratory image.

[0159] At step S580 the combined image may be displayed on a display device as part of a video feed or a still image. The new imagery can be used to support an intervention and to reliably guide the user to the marker and hence the lesion marked by the marker.

[0160] The proposed method as such results in two streams, one for the beacon images and one for the exploratory images. The image streams may be interleaved, so as to generate a stream of exploratory images runs, occasionally interrupted by one or more beacon images, when a beacon image becomes available. In embodiments there may not necessarily be a strict, regular alteration between beacon image and exploratory image.

[0161] Steps S540, S550 may be performed before the step S520, S530.

[0162] In embodiments there may be causal link between steps S540 and S550 or between steps S520 and S530.

[0163] The above steps may be repeated, to obtain new pairs of exploratory and beacon images.

[0164] In particular, the switching on of the marker light may be caused by the switching off of the illumination source at the imager. This will constitute a causal link synchronization but this is not necessarily required in all embodiments, as non-causal synchronization is also envisaged. In non-causal synchronization the two light sources may be pre-set to pulse in frequencies suitably adjusted out of phase with pulse lengths suitably chosen as explained above at FIG. 2.

[0165] For best results, it may be advisable to darken the operating theatre to reduce ambient light that may trigger undesirable switching operations. In embodiments, a calibration procedure may be advisable to calibrate the marker sensor to the light source of the imaging apparatus. The light emitted by the marker and/or the illumination source may include visible light, but other frequencies such as IR or NIR are also envisaged.

[0166] Preferably, the marker as used in the proposed method is powered by light, in particular by light of the imager's illumination source.

[0167] The capturing of the beacon and exploratory image at steps S530, S560 may include using a lensed eye piece or view finder such as may be used in an endoscope. The location of the marker as identified may be overlaid as a visual indication in the field of view of the eye piece or view finder.

[0168] The components of the image processing system IP, the modulation module MD and or part or all of the synchronizer logic SL may be implemented as software modules or routines run on a computing unit PU such as a workstation associated with the imager ENDO. The image processing system IP, the modulation module MD and/or a part or all of the synchronizer logic SL may be arranged in a distributed architecture and connected in a suitable communication network. The image processing system IP, the modulation module MD and/or a part or all of the synchronizer logic SL may be arranged in the marker device or in the imager ENDO, as a suitably programmed microprocessor or microcontroller, such as an FPGA (field-programmable-gate-array) or as hardwired IC chip such as an ASIC, system-on-a-chip (SOC), and combinations thereof.

[0169] The switching on and off of one or both of the light sources LS1, LS2 may be done by (re-)connection or disconnection, respectively, from the respective power source. Alternatively, the light may be blocked/released by a shutter mechanism or filter element.

[0170] In all of the above embodiments, instead of switching off one or both of the lights sources LS1, LS2, the light intensities may be merely reduced to a non-zero intensity level by a dimmer functionality or by insertion of a filter element or other. In particular, the light source LS2 at the imager may be so dimmed when the light source LS1 of the marker is on. The marker is preferably not merely dimmed, but switched off when the light source of the imager is switched to the higher intensity. In other embodiments, it is the light source at the marker that is dimmed whilst the light source of the imager is illuminating at a higher intensity level. In a further embodiment, each light source is merely dimmed whilst the other light source illuminates at a higher intensity, and there is no complete switch off at either light source.

[0171] In another exemplary embodiment of the present invention, a computer program or a computer program element is provided that is characterized by being adapted to execute the method steps of the method according to one of the preceding embodiments, on an appropriate system.

[0172] The computer program element might therefore be stored on a computer unit, which might also be part of an embodiment of the present invention. This computing unit may be adapted to perform or induce a performing of the steps of the method described above. Moreover, it may be adapted to operate the components of the above-described apparatus. The computing unit can be adapted to operate automatically and/or to execute the orders of a user. A computer program may be loaded into a working memory of a data processor. The data processor may thus be equipped to carry out the method of the invention.

[0173] This exemplary embodiment of the invention covers both, a computer program that right from the beginning uses the invention and a computer program that by means of an up-date turns an existing program into a program that uses the invention.

[0174] Further on, the computer program element might be able to provide all necessary steps to fulfill the procedure of an exemplary embodiment of the method as described above.

[0175] According to a further exemplary embodiment of the present invention, a computer readable medium, such as a CD-ROM, is presented wherein the computer readable medium has a computer program element stored on it which computer program element is described by the preceding section.

[0176] A computer program may be stored and/or distributed on a suitable medium (in particular, but not necessarily, a non-transitory medium), such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the internet or other wired or wireless telecommunication systems.

[0177] However, the computer program may also be presented over a network like the World Wide Web and can be downloaded into the working memory of a data processor from such a network. According to a further exemplary embodiment of the present invention, a medium for making a computer program element available for downloading is provided, which computer program element is arranged to perform a method according to one of the previously described embodiments of the invention.

[0178] It has to be noted that embodiments of the invention are described with reference to different subject matters. In particular, some embodiments are described with reference to method type claims whereas other embodiments are described with reference to the device type claims. However, a person skilled in the art will gather from the above and the following description that, unless otherwise notified, in addition to any combination of features belonging to one type of subject matter also any combination between features relating to different subject matters is considered to be disclosed with this application. However, all features can be combined providing synergetic effects that are more than the simple summation of the features.

[0179] While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive. The invention is not limited to the disclosed embodiments. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing a claimed invention, from a study of the drawings, the disclosure, and the dependent claims.

[0180] In the claims, the word “comprising” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude a plurality. A single processor or other unit may fulfill the functions of several items re-cited in the claims. The mere fact that certain measures are re-cited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. Any reference signs in the claims should not be construed as limiting the scope.