System and method for localization of fluorescent targets in deep tissue for guiding surgery
11636647 · 2023-04-25
Assignee
Inventors
- Brian Zahler Bentz (Albuquerque, NM, US)
- Kevin J. Webb (West Lafayette, IN, US)
- Dergan Lin (West Lafayette, IN, US)
Cpc classification
G06T2207/10101
PHYSICS
G06T17/20
PHYSICS
International classification
Abstract
A method for identifying a source of florescence is disclosed which includes shining light on a subject at a first wavelength, causing emission of light at a second wavelength from the source of fluorescence, filtering out light at the first wavelength, capturing at least one 2 dimensional (2D) image of a subject having a plurality of pixels at the second wavelength, and establishing information about approximate location of a source of florescence within a tissue of the subject, selectively generating a 3D geometric model where the model is adapted to provide a model representation of the at least one 2D captured image, comparing the modeled at least one 2D captured image to the captured at least one 2D image and iteratively adjusting the model to minimize the difference, and outputting location and geometric configuration of the source of fluorescence within the tissue within the region of interest.
Claims
1. A system for identifying a source of fluorescence in tissue, comprising: a light source configured to be shone on a subject, the light source configured to illuminate tissue of a subject at a first wavelength, and in response cause emission of light at a second wavelength from a source of fluorescence; an optical filter configured to filter out light having the first wavelength and allow passage of light having the second wavelength; an image capture device configured to capture images of the tissue at the second wavelength; and a processor having software encoded on a non-transitory computer readable medium configured to: capture at least one 2 dimensional (2D) image of a subject having a plurality of pixels; establish information about approximate location in the captured 2D image of the source of fluorescence within the tissue of the subject; identify a region of the interest about the approximate location of the source of fluorescence; establish a 3 dimensional (3D) topography data of the subject at least about the region of interest; map each pixel of the region of interest of the at least one captured 2D image to the 3D topography data; selectively generate a 3D geometric model based on a plurality of parameters including optical parameters of the tissue as well as the mapped pixels of the region of interest on the 3D topography data, wherein the 3D geometric model outputs a modeled 2D image representation of the at least one 2D captured image; compare the modeled 2D image representation to the at least one captured 2D image and generate an error signal representing a difference therebetween; iteratively adjust the plurality of parameters of the 3D geometric model to minimize the error signal; and output a location and a geometric configuration with respect to the 3D geometric model of the source of fluorescence within the tissue within the region of interest.
2. The system of claim 1, wherein establishment of the 3D topography data is based on laser scanning.
3. The system of claim 1, wherein establishment of the 3D topography data is based on stereoscopic image acquisition.
4. The system of claim 3, the stereoscopic image acquisition using at least two image capture devices.
5. The system of claim 1, wherein the image capture device is a charge coupled device, or a plurality of photodiodes.
6. The system of claim 1, wherein the establishment of information about approximate location in the captured 2D image of a tumor is by causing the tumor to fluoresce.
7. The system of claim 1, the establishment of information about the approximate location of the source of fluorescence is by analyzing scattered light initially shone from the light source.
8. The system of claim 1, the establishment of information about the approximate location of the source of fluorescence is by analyzing absorption of light initially shone from the light source.
9. The system of claim 1, wherein the light source is a laser light.
10. The system of claim 9, wherein the laser light is spread using a light spreader.
11. A method for identifying a source of fluorescence in tissue, comprising: shining a light on a subject by a light source at a first wavelength, causing emission of light at a second wavelength from a source of fluorescence; optically filtering out light at the first wavelength and allowing passage of light at the second wavelength; capturing at least one 2 dimensional (2D) image of a subject having a plurality of pixels at the second wavelength; establishing information about approximate location in the captured 2D image of the source of fluorescence within the tissue of the subject; identifying a region of the interest about the approximate location of the source of fluorescence; establishing a 3 dimensional (3D) topography data of the subject at least about the region of interest; mapping each pixel of the region of interest of the at least one captured 2D image to the 3D topography data; selectively generating a 3D geometric model based on a plurality of parameters including optical parameters of the tissue as well as the mapped pixels of the region of interest on the 3D topography data, wherein the 3D geometric model outputs a modeled 2D image representation of the at least one 2D captured image; comparing the modeled 2D image representation to the at least one captured 2D image and generate an error signal representing a difference therebetween; iteratively adjusting the plurality of parameters of the 3D geometric model to minimize the error signal; and outputting a location and a geometric configuration with respect to the 3D geometric model of the source of fluorescence within the tissue within the region of interest.
12. The method of claim 11, wherein establishment of the 3D topography data is based on laser scanning.
13. The method of claim 11, wherein establishment of the 3D topography data is based on stereoscopic image acquisition.
14. The method of claim 13, the stereoscopic image acquisition uses at least two image capture devices.
15. The method of claim 11, wherein the image capture device is a charge coupled device, or a plurality of photodiodes.
16. The system of claim 1, wherein the establishment of information about approximate location in the captured 2D image of a tumor is by causing the tumor to fluoresce.
17. The system of claim 1, the establishment of information about the approximate location of the source of fluorescence is by analyzing scattered light initially shone from the light source.
18. The system of claim 1, the establishment of information about the approximate location of the source of fluorescence is by analyzing absorption of light initially shone from the light source.
19. The system of claim 1, wherein the light source is a laser light.
20. The method of claim 19, wherein the laser light is spread using a light spreader.
Description
BRIEF DESCRIPTION OF DRAWINGS
(1) The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.
(2)
(3)
(4)
(5)
(6)
(7)
DETAILED DESCRIPTION
(8) For the purposes of promoting an understanding of the principles of the present disclosure, reference will now be made to the embodiments illustrated in the drawings, and specific language will be used to describe the same. It will nevertheless be understood that no limitation of the scope of this disclosure is thereby intended.
(9) In the present disclosure, the term “about” can allow for a degree of variability in a value or range, for example, within 10%, within 5%, or within 1% of a stated value or of a stated limit of a range.
(10) In the present disclosure, the term “substantially” can allow for a degree of variability in a value or range, for example, within 90%, within 95%, or within 99% of a stated value or of a stated limit of a range.
(11) A novel imaging approach is provided in the present disclosure that can be used to assist a surgeon to resect tissue. In particular, imaging beyond 1 mm is desirable. Imaging at tissue depths beyond 1 mm is achievable with diffuse optical imaging (DOI), where the light propagation is approximated as a diffusion process. The diffusion process includes both light scattering as well as absorption. In fluorescence diffuse optical tomography (FDOT), a DOI method, computational imaging allows formation of three dimensional (3D) images of optical properties. FDOT provides utility for in vivo studies in mice and rats, especially when combined with another imaging modality such as computed tomography (CT) or magnetic resonance imaging (MRI) to improve spatial resolution.
(12) FDOT and folate-targeted fluorescent contrast agents can be used to image the kidneys and liver in dead mice as well as tumors in live mice. FDOT has potential to be a useful tool for fluorescence guided surgery, where tumor nodules are identified for a surgeon to remove. However, the full volumetric reconstruction performed by FDOT requires extensive computational time, making it ill-suited for an intraoperative environment where real-time imaging is required over a period of hours. As a result, an alternative approach using fast localization methods is provided in the present disclosure where only the position of a source of florescence is determined. A mouse model is used to show that this method can find tumors in deep tissue, and can provide depth information to assist in guided surgery.
(13) Referring to
(14) The detector 104 is coupled to a processor (not shown) which has analyzing software on a non-transitory computer readable medium configured to determine the characteristics of the tumor 118 (or other tissue that has been treated with a florescent material). The analysis is based on a process 200 depicted as a flowchart in
(15) Prior to describing the steps in the process 200 shown in
(16)
where r denotes the position,
ϕ(r, ω) (W/mm.sup.2) is the photon flux density,
ω is the angular modulation frequency,
D=1/[3(μ.sub.s′+μ.sub.a)] (mm) is the diffusion coefficient,
μ′.sub.s (mm) is the reduced scattering coefficient,
μ.sub.a (mm) is the absorption coefficient,
c is the speed of light in the medium,
the subscripts x and m denote parameters at the fluorophore excitation and emission wavelengths, λ.sub.x and λ.sub.m, respectively,
S.sub.x (W/mm.sup.3) is the excitation source term, and
S.sub.f=η(1+jωτ).sup.−1 (mm.sup.−1) is the fluorescence source term.
(17) Equations (1) and (2) are coupled through the ϕ.sub.x(r, ω) term on the right hand side of (2). These equations represent a set of partial differential equations (PDE), that can be solved numerically using the Green's function, as known to a person having ordinary skill in the art. In an infinite homogeneous space, the diffusion equation's Green's function is
(18)
where r′ is the position of a point source, and
k.sup.2=−μ.sub.aD−jω/(Dc), where
μ.sub.a and D can be calculated at λ.sub.x or λ.sub.m in (1) or (2), respectively. Equation (3) represents the analytical solution of propagation of photons.
(19) Based on equations (1), (2), and (3), a model can thus be generated based on the assumption that tissue surrounding the tumor is homogenous and thus diffuses light uniformly. Equations (1) and (2) can be solved on an unstructured finite element method (FEM) mesh on the assumption that the tissue is heterogeneous. However, the FEM solution requires extensive computational time, limiting its application in an operating environment. For this reason, a closed-form analytic solution can be adopted that allows fast computation.
(20) Referring to
(21)
where w is a multiplicative constant that incorporates η.sub.f, μ.sub.a.sub.
g.sub.x(r.sub.s, r.sub.f) represent the diffusion equation Green's function for (1) at λ.sub.x, and
g.sub.m(r.sub.f, r.sub.i) represent the diffusion equation Green's function for (2) at λ.sub.m. The Green's functions are derived by assuming a single boundary exists such that r.sub.s and r.sub.i are on the boundary and r.sub.f is below the boundary, as shown in
(22) If a source of florescence is present, its position can be estimated by finding the value of r.sub.f that minimizes the cost function
(23)
where y is a vector of N measurements,
f (r.sub.f) is a vector of N forward calculations f.sub.i(r.sub.f) from (5),
γ=αdiag[|y.sub.1|, . . . , |y.sub.N|] is the noise covariance matrix, for which we assume a shot noise model characterized by α. For an arbitrary vector v, ∥v∥.sub.γ.sub.
where H denotes the Hermitian transpose. Only the case where a single excitation source is present at position r.sub.s is considered. In this case, g.sub.x(r.sub.s, r.sub.f) at r.sub.f is constant, and can therefore be pulled out of f (r.sub.f), giving
(24)
where h.sub.i(r.sub.f)=g.sub.m(r.sub.f, r.sub.i) is the i.sup.th component of h(r.sub.f). Because g.sub.x(r.sub.s, r.sub.f) is a constant at r.sub.f, it can be incorporated into w as
(25)
(26) For localization, one goal is to find the r.sub.f that minimizes (8), and we note that the inverse problem is linear in w.sub.s and nonlinear in r.sub.f. Equation (8) can therefore be minimized using a two-step procedure. First, we set the derivative of ∥y−w.sub.sh(r.sub.f)∥.sub.γ.sub.
(27)
(28) Second, we calculate (10) at a set of positions r.sub.f within a region of interest that encompasses the true location. The maximum likelihood estimates are then
(29)
(30) An important step in our derivation that differentiates it from previous derivations is the incorporation of g.sub.x(r.sub.s, r.sub.f) into w.sub.s. This step implies that the inverse problem can be solved without consideration or modeling of the excitation source, and only g.sub.m(r.sub.f, r.sub.i) needs to be computed for the forward model. This is of great utility because complicated illumination patterns (such as an expanded beam) do not need to be modeled.
(31) The localization of a source of florescence can thus be demonstrated numerically, according to the present disclosure, in , according to (11).
(32) To demonstrate the efficacy of the process 200 (see
(33) In order to localize a tumor, tumor cells need to be first targeted with a fluorescent compound. Over forty percent of human cancer cells over-express folate receptors, enabling the cells to be identified using folate-targeted fluorescence imaging. In a typical study, a fluorophore is attached to the targeting agent (folate) and injected into the blood stream of an animal. The fluorescent agent is then distributed to the extracellular extravascular space, where it remains in circulation or is eliminated. Roughly 30 minutes after injection, the fluorescent agent is mostly cleared from the blood, and is concentrated in the kidneys, the liver, and any tumors that are present. This process introduces a contrast in fluorescence throughout the tissue, enabling fluorescence-guided surgery. In a previous study, it was shown that a surgeon can detect 5 times more malignant masses with the aid of fluorescence than with the naked eye. However, once a tumor has been identified, additional information about its location, such as its depth, could be used to minimize damage to the surrounding healthy tissue. Here, we use a mouse model to demonstrate that the location of a tumor can be estimated using our localization method. Expanded beam illumination is commonly used in fluorescence-guided surgery, further motivating the use of our approach.
(34) Female nu/nu mice purchased from NCI CHARLES RIVER LABORATORIES™ were maintained on folate deficient rodent chow for 3 weeks prior to experimental study and kept on a standard 12 hour light-dark cycle. Tumor cells (106 of L1210A) were injected intravenously into the tail vein of a six-week-old female nu/nu mouse. The cancer cells were allowed to metastasize for 30 days, at which point 10 nmol of a folate-targeted fluorescent agent (OTL0038) dissolved in saline was injected intravenously via the tail vain. The OTL0038 attached itself to the folate-receptors present in the tumors, allowing for fluorescent imaging. Two hours after injection of OTL0038, the mouse was euthanized through CO2 asphyxiation. The mouse was then placed on its side in the experimental setup shown in
(35) The peak excitation of the OTL0038 is 770 nm and the peak emission is 790 nm. An OD4 emission bandpass filter (see filter 122 in
(36) The 3D topography of the mouse was captured using the laser line scanner. Laser light was focused through a cylindrical lens to form a line, which was scanned along the length of the mouse to 92 positions. At each position, a CCD camera image was captured.
(37) Localization of the tumor requires that the μ.sub.s′ and μ.sub.a of the tissue are known so that g.sub.m(r.sub.f, r.sub.i) can be calculated using (3) subject to the boundary condition. The μ.sub.s′ and μ.sub.a can be determined from the literature, or they can be estimated by incorporating them into the optimization problem. μ.sub.a was estimated according to the present disclosure in order to improve the accuracy of the localization. This was accomplished by fixing μ.sub.s′=1.6 mm.sup.−1 and, for each r.sub.f within the region of interest, calculating the cost in (10) for values of μ.sub.a between 0 and 0.05 mm.sup.−1 separated by increments of 0.005 mm.sup.−1. The position of the source of florescence was then estimated as the position r.sub.f that minimized the cost. Because the tissue is heterogeneous and the model assumes that it is homogeneous, estimated values of μ.sub.s′ and μ.sub.a will not be quantitative. Therefore, μ.sub.s′ and μ.sub.a can be treated as fitting parameters, since for localization only position of the source of florescence is of interest.
(38) The results of this localization procedure using data from h(
). Since the surface is slowly varying, and only one source of florescence dominates the contribution to the data at the detectors, the model fits the data well. The discrepancy could be due to autofluorescence, fluorescence from the kidney, errors in the 3D topography, or assumptions made in the forward model derivation.
(39) The localization method was implemented in MATLAB® and run on a 12 core computer with 3.47 GHz INTEL® X5690 processors and 96 GB RAM. In order to improve the computational time, an effort was made to parallelize the computation of the cost function across multiple processors using the MATLAB® parallel computing toolbox. Without parallel processing, the computational time for the results in
(40) Referring to
(41) Those having ordinary skill in the art will recognize that numerous modifications can be made to the specific implementations described above. The implementations should not be limited to the particular limitations described. Other implementations may be possible.