INTRAORAL OCT WITH COLOR TEXTURE

Abstract

An apparatus for acquiring intraoral images of a subject has an OCT imaging apparatus having an OCT light source, a scanner that conveys OCT light toward the subject and returned from the subject, and an interferometer having a reference arm and a sample arm. A reflectance imaging apparatus has a visible light source directed toward the subject and an image sensor that forms a reflectance image from returned light. Processing and control logic are configured to process and combine the returned reflectance image to the OCT measured data. A display shows the combined reflectance image and OCT measured data.

Claims

1. An apparatus for acquiring intraoral images of a subject comprising: an OCT imaging apparatus comprising an OCT light source, an interferometer having a reference arm and a sample arm, and a scanner that conveys OCT light along an OCT light path toward the subject and returned from the subject in the sample arm; a reflectance imaging apparatus comprising a visible light source to direct visible light along a visible light path toward the subject and an image sensor that forms a reflectance image from returned visible light from the subject; at least one processor configured to process and combine the reflectance image to OCT measured data from the interferometer at least in part by: calibrating the visible light path to the OCT light path; associating the reflectance image and the OCT measured data based at least in part on timing synchronization; generating a projection image based at least in part on the OCT measured data from the interferometer; and a display that shows the combined reflectance image and the OCT measured data reflecting final color texture.

2. The apparatus of claim 1, wherein the at least one processor is further configured for: registering color texture from the reflectance image to the OCT measured data by projecting a 3D volume generated using OCT measured data to generate a 2D grayscale OCT image.

3. The apparatus of claim 2, wherein the at least one processor is further configured for: applying a scanner distortion model to correct for OCT distortion to restore correct geometry to surface data of the subject.

4. The apparatus of claim 2, wherein registering color texture from the reflectance image to the OCT measured data further comprises interpolating, for a plurality of pixels of the OCT measured data, a final color texture based at least in part on one or more color values of pixels of the reflectance image registered with neighboring pixels.

5. The apparatus of claim 1, wherein calibrating the visible light path to the OCT light path comprises stereo vision calibration applied to the scanner and the image sensor.

6. The apparatus of claim 5, wherein calibrating the visible light path to the OCT light path further comprises calculating a matrix to translate and rotate a two-dimensional color camera image of the image sensor to a two-dimensional grayscale OCT image.

7. The apparatus of claim 1, wherein associating the reflectance image and the OCT measured data further comprises registering the reflectance image to the OCT measured data based on operations of the scanner.

8. The apparatus of claim 1, wherein the OCT imaging apparatus is a swept-source OCT imaging apparatus or the OCT light source is a wide-bandwidth light source.

9. The apparatus of claim 1, wherein the scanner determines an optical path to the subject.

10. A method for acquiring intraoral images comprising: generating a 3D OCT volume having pixel dimensions L?M?N using an OCT imaging apparatus comprising an OCT light source, a scanner, and an interferometer having a reference arm and a sample arm; generating a 2D color image having pixel dimensions L?M using a reflectance imaging apparatus comprising a visible light source and a color image sensor; combining, using combining optics, the OCT light source and visible light directed toward a 2D arrangement of a plurality of points of an intraoral feature onto a same optical path for projection onto the intraoral feature and that separate the visible light primary components from OCT light for light returning from the intraoral feature to separately obtain OCT measured data and corresponding color reflectance image data in sequence and correlated for the 2D arrangement of the plurality of points of the intraoral feature; processing and combining returned color reflectance image data and the OCT measured data to overlay the L?M?N OCT volume with the L?M 2D color image to create an overlaid image; and providing for display of the combined color reflectance image and OCT measured data.

11. The method of claim 10, wherein the combining optics provide laser diode visible light.

12. The method of claim 10, wherein the same optical path includes the scanner.

13. The method of claim 10, wherein the combining optics comprise a fiber combiner, a beam splitter, or beam splitter prism.

14. The method of claim 10, further comprising processing and combining the returned color reflectance image data and an OCT surface detected in the OCT measured data.

15. The method of claim 10, wherein the combining optics comprise a wave division multiplexer or a grating.

16. A method for acquiring intraoral images of a subject comprising: conveying OCT light along an OCT light path toward the subject and returned from the subject in a sample arm using an OCT imaging apparatus comprising an OCT light source, an interferometer having a reference arm and the sample arm, and a scanner; directing visible light along a visible light path toward the subject and forming a reflectance image from returned visible light from the subject using a reflectance imaging apparatus comprising a visible light source and an image sensor; processing and combining the reflectance image to OCT measured data from the interferometer at least in part by: calibrating the visible light path to the OCT light path; associating the reflectance image and the OCT measured data based at least in part on timing synchronization; generating a projection image based at least in part on the OCT measured data from the interferometer; and displaying a combination of the reflectance image and the OCT measured data reflecting final color texture.

17. The method of claim 16, further comprising: registering color texture from the reflectance image to the OCT measured data by projecting a 3D volume generated using OCT measured data to generate a 2D grayscale OCT image.

18. The method of claim 17, further comprising: applying a scanner distortion model to correct for OCT distortion to restore correct geometry to surface data of the subject.

19. The method of claim 17, further comprising: interpolating, for a plurality of pixels of the OCT measured data, a final color texture based at least in part on one or more color values of pixels of the reflectance image registered with neighboring pixels.

20. The method of claim 16, wherein calibrating the visible light path to the OCT light path comprises stereo vision calibration applied to the scanner and the image sensor.

21. The method of claim 20, wherein calibrating the visible light path to the OCT light path further comprises calculating a matrix to translate and rotate a two-dimensional color camera image of the image sensor to a two-dimensional grayscale OCT image.

22. The method of claim 16, wherein associating the reflectance image and the OCT measured data further comprises registering the reflectance image to the OCT measured data based on operations of the scanner.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The foregoing and other objects, features, and advantages of the invention will be apparent from the following more particular description of the embodiments of the invention, as illustrated in the accompanying drawings.

[0023] The elements of the drawings are not necessarily to scale relative to each other. Some exaggeration may be necessary in order to emphasize basic structural relationships or principles of operation. Some conventional components that would be needed for implementation of the described embodiments, such as support components used for providing power, for packaging, and for mounting and protecting system optics, for example, are not shown in the drawings in order to simplify description.

[0024] FIG. 1 is a schematic diagram showing an imaging apparatus for combined OCT scanning and color image acquisition.

[0025] FIG. 2 is a schematic diagram showing an alternate embodiment for an imaging apparatus for combined OCT scanning and color image acquisition.

[0026] FIG. 3 is a schematic diagram showing another alternate embodiment for an imaging apparatus for combined OCT scanning and color image acquisition.

[0027] FIGS. 4A and 4B show functional and geometric aspects of OCT imaging.

[0028] FIG. 5 shows a combined OCT and color scanning scheme in schematic form.

[0029] FIGS. 6A through 6G show various embodiments of a color light emitter/detector.

[0030] FIG. 6H shows an embodiment in which RGB light detection is performed at the OCT spectrometer.

[0031] FIG. 7 is a logic flow diagram that shows a sequence for combined color calibration.

[0032] FIG. 8 is a graph that shows spectral ranges for OCT and reflectance imaging.

[0033] FIGS. 9A-9E show various embodiments of a combined OCT and color image acquisition apparatus using a preview camera.

[0034] FIGS. 10A and 10B are schematic diagrams that show aspects of registration processing for a tooth as the imaged object using a pinhole camera paradigm.

DESCRIPTION OF EXEMPLARY EMBODIMENTS

[0035] The following is a description of exemplary method and/or apparatus embodiments of the application, reference being made to the drawings in which the same reference numerals identify the same elements of structure in each of the several figures.

[0036] Where they are used in the context of the present disclosure, the terms first, second, and so on, do not necessarily denote any ordinal, sequential, or priority relation, but are simply used to more clearly distinguish one step, element, or set of elements from another, unless specified otherwise.

[0037] As used herein, the term energizable relates to a device or set of components that perform an indicated function upon receiving power and, optionally, upon receiving an enabling signal.

[0038] In the context of the present disclosure, the term optics is used generally to refer to lenses and other refractive, diffractive, and reflective components or apertures used for shaping and orienting a light beam. An individual component of this type is termed an optic.

[0039] In the context of the present disclosure, the terms viewer, operator, and user are considered to be equivalent and refer to the viewing practitioner, technician, or other person who may operate a camera or scanner and may also view and manipulate an image, such as a dental image, on a display monitor. An operator instruction or viewer instruction is obtained from explicit commands entered by the viewer, such as by clicking a button on the camera or scanner or by using a computer mouse or by touch screen or keyboard entry.

[0040] In the context of the present disclosure, the phrase in signal communication indicates that two or more devices and/or components are capable of communicating with each other via signals that travel over some type of signal path. Signal communication may be wired or wireless. The signals may be communication, power, data, or energy signals. The signal paths may include physical, electrical, magnetic, electromagnetic, optical, wired, and/or wireless connections between the first device and/or component and second device and/or component. The signal paths may also include additional devices and/or components between the first device and/or component and second device and/or component.

[0041] In the context of the present disclosure, the term camera relates to a device that is enabled to acquire a reflectance, 2-D digital image from reflected visible or NIR light, such as structured light that is reflected from the surface of teeth and supporting structures.

[0042] The general term scanner relates to an optical system that projects a scanned light beam of broadband near-IR (BNIR) illumination that is directed to the tooth surface through a sample arm and acquired, as scattered light returned in the sample arm, for detecting interference with light from a reference rum used in OCT imaging of a surface. The term raster scanner relates to the combination of hardware components that scan light toward a sample, as described in more detail subsequently.

[0043] The term subject refers to the tooth or other portion of a patient that is being imaged and, in optical terms, can be considered equivalent to the object of the corresponding imaging system. This corresponds to the term sample that is typically used with reference to OCT imaging.

[0044] In the context of the present disclosure, the phrase broadband light emitter refers to an illumination or light source that emits a continuous spectrum output over a range of wavelengths at any given point of time. Short-coherence or low-coherence, broadband illumination sources can include, for example, super luminescent diodes, short-pulse lasers, many types of white-light sources, and supercontinuum light sources. Most low coherence length sources of these types have a coherence length on the order of tens of microns or less.

[0045] In the context of the present disclosure, the terms color light, polychromatic light, and RGB light describe visible light illumination that is provided for reflectance imaging. The color image can be considered a reflectance image or color texture image. As is well known in the color imaging ruts, a color combiner, such as a dichroic surface that transmits one spectral band and reflects another spectral band, can be used to combine colors for light traveling in one direction and to separate colors for light traveling in the opposite direction. Thus, the general term combiner is typically used for a combiner/separator device that both combines and separates light according to wavelength and direction along an optical path.

[0046] As is well-known to those skilled in the OCT imaging ruts, the axial resolution is related to the coherence length of the light source. Thus, the shorter the coherence length, the higher the axial resolution.

[0047] Certain exemplary method and/or apparatus embodiments herein can utilize any of the various types of OCT scanning methods, including time-domain or spectral or frequency-domain OCT. Because the speed advantage is of particular interest, the description that follows is primarily directed to embodiments that employ swept-source OCT, a type of frequency-domain OCT that is generally advantageous for faster speed and overall scanning throughput. However, it should be noted that the compressive sampling methods or other available OCT methods can be used to improve the response of time-domain OCT and other types of OCT as well as with SS-OCT. Methods of the present disclosure can also be used where a spectrometer is used for sensing in the OCT system.

[0048] According to an embodiment of the present disclosure, there is provided a hybrid imaging apparatus that obtains OCT scanned data with accompanying color texture content for intraoral features.

[0049] Referring to the schematic diagram of FIG. 1, there is shown an imaging apparatus 100 for combined OCT scanning and color image acquisition, wherein the color data is inherently registered to the OCT scan data. Imaging apparatus 100 has two light paths for imaging that can combine the OCT and color image content and share the same light guiding components.

[0050] In an OCT light path 40, an OCT light source 10 provides illumination for OCT image scanning. Light source 10 can employ a superluminescent diode (SLD) or other source that emits continuous wavelength broadband light. Alternately, light source 10 can be some other type of suitable light source, such as a swept source that emits light with continuously varying spectral content. This light is directed through a first fiber coupler FC1 or wavelength division multiplexer WDM to a second fiber coupler FC2. Fiber coupler FC2 splits the light path into a reference rum 42 and a sample arm 44. Light in reference arm 42 reflects back from a reference mirror 48; this light is coupled back through fiber coupler FC2 and goes to OCT signal detector 46. Light directed to sample rum 44 is directed to the subject or sample S by a scanner 24. Reflected and scattered light from sample S is coupled back through sample rum 44 to fiber coupler FC2 and is conveyed to OCT signal detector 46. The light from reference rum 42 interferes with light from reference arm 44 to provide the OCT scan data for processing and reconstruction.

[0051] In a color reflectance imaging path 50, polychromatic or color light is emitted from a color light emitter/detector (CLED) 52 and directed through fiber coupler FC1 or WDM to the second fiber coupler FC2. Coupler FC2 acts as a combiner/separator. The polychromatic visible light is combined with the OCT sample light and is simultaneously directed to sample S through scanner 24, part of an intraoral probe 30. Returned reflected color light from the surface of the tooth or other intraoral feature is conveyed through fiber coupler FC2 back to CLED 52. CLED 52 senses the color content from the reflected light. A control logic processor 60 is in signal communication with OCT signal detector 46, CLED 52 and light source 10 to record and process the OCT output data from interference and combine this data with the color data from the intraoral surface. The resulting combined image content can then be presented on a display 72 and can alternately be transmitted and stored.

[0052] The schematic diagrams of FIGS. 2 and 3 show similar imaging apparatus 120 and 140 respectively, having slightly different light path arrangements for combining the OCT and reflectance imaging functions. In the FIG. 2 arrangement of imaging apparatus 120, the OCT path is the same as that described previously with respect to imaging apparatus 100 in FIG. 1. The light from color light emitter/detector (CLED) 52 is directed through a fiber coupler FC3 and shares the sample arm with OCT light; this combined light is directed to the subject or sample S through scanner 24. Backscattered color light from the intraoral surface is conveyed through fiber coupler FC3 to color light emitter/detector (CLED) 52 to measure the color content, recorded and processed by processor 60. The resulting combined image content can then be presented on a display 72 and can alternately be transmitted and stored.

[0053] In the FIG. 3 configuration of imaging apparatus 140, the OCT path is the same as that described previously with respect to imaging apparatus 100 in FIG. 1. The light from color light emitter/detector (CLED) 52 is directed into the sample path through a dichroic combiner 54, that has a reflective surface and a dichroic surface in the configuration shown. Backscattered color light from the intraoral surface is conveyed back to CLED 52 through combiner 54 for detection and measurement, recorded and processed by processor 60. The resulting combined image content can similarly be presented on a display 72 and can alternately be transmitted and stored.

[0054] In each of the FIG. 1-3 configurations, CLED signal detection and OCT signal detection are synchronized and share the same optical path in the sample arm, to and from the sample probe 30 and its scanner 24. This arrangement provides inherent registration of the polychromatic image content to the OCT data.

Scanning Sequence for OCT Imaging

[0055] The schematic diagrams of FIGS. 4A and 4B show a scan sequence that can be used for forming tomographic images using the OCT apparatus of the present disclosure in a Fourier domain acquisition. The sequence shown in FIG. 4A shows how a single B-scan image is generated. Raster scanner 24 (FIG. 1) scans the selected light sequence over the subject, sample S, point by point. A periodic drive signal 92 as shown in FIG. 4A is used to drive the raster scanner 24 galvo mirrors to control a lateral scan or B-scan that extends across each row of the sample, shown as discrete points 82 extending in the horizontal direction in FIGS. 4A and 4B. At each of a plurality of points 82 along a line or row of the B-scan, an A-scan or depth scan, acquiring data in the z-axis direction, is generated using successive portions of the selected wavelength band. FIG. 4A shows drive signal 92 for generating a straightforward ascending sequence using raster scanner 24, with corresponding micro-mirror actuations, or other spatial light modulator pixel-by-pixel actuation, through the wavelength band. The retro-scan signal 93, part of drive signal 92, simply restores the scan mirror back to its starting position for the next line; no OCT data is obtained during retro-scan signal 93.

[0056] It should be noted that the B-scan drive signal 92 drives the galvo minor for raster scanner 24 as shown in FIGS. 1-3. At each incremental position, point 82 along the row of the B-scan, an A-scan is obtained. To acquire the A-scan data, the tuned laser or other OCT light source sweeps through a spectral sequence controlled by a programmable filter in the OCT source 10. Thus, in an embodiment in which the light source sweeps through a 30 nm range of wavelengths, this sequence is carried out at each point 82 along the B-scan path. As FIG. 4A shows, the set of A-scan acquisitions executes at each point 82, that is, at each position of the scanner 24. By way of example, there can be 2048 measurements for generating the A-scan at each position 82.

[0057] FIG. 4A schematically shows the information acquired during each A-scan. An interference signal 88, shown with DC signal content removed, is acquired over the time interval for each point. 82, wherein the signal is a function of the time interval required for the spectral sweep, with the signal that is acquired indicative of the spectral interference fringes generated by combining the light from reference and feedback arms of the OCT interferometer components. The Fourier transform generates a transform T for each A-scan. One transform signal corresponding to an A-scan is shown by way of example in FIG. 4A.

[0058] From the above description, it can be appreciated that a significant amount of data is acquired over a single B-scan sequence. In order to process this data efficiently, a Fast-Fourier Transform (FFT) is used, transforming the time-based signal data to corresponding frequency-based data from which image content can more readily be generated.

[0059] In Fourier domain OCT, the A scan corresponds to one line of spectrum acquisition which generates a line of depth (z-axis) resolved OCT signal. The B scan data generates a 2-D OCT image along the corresponding scanned line.

[0060] Raster scanning is used to obtain multiple B-scan data by incrementing the raster scanner 24 acquisition in the C-scan direction. This is represented schematically in FIG. 4B, which shows how 3-D volume information is generated using the A-, B-, and C-scan data.

[0061] As noted previously, the wavelength or frequency sweep sequence that is used at each A-scan point 82 can be modified from the ascending or descending wavelength sequence that is typically used. Arbitrary wavelength sequencing can alternately be used. In the case of arbitrary wavelength selection, which may be useful for some particular implementations of OCT, only a portion of the available wavelengths are provided as a result of each sweep. In arbitrary wavelength sequencing, each wavelength can be randomly selected, in arbitrary sequential order, to be used in the OCT system during a single sweep.

[0062] FIG. 5 shows the combined OCT and color scanning scheme in schematic form. Scanner 24 steers both the color light beam and OCT light beam to sample Sin a two-dimensional (x, y) raster scan at each point 82, with x[0, L?1] indexed along the x scanning axis. The orthogonal component, y[0, M?1] is indexed along they scanning axis. The color signal with three reflectance values (R(x, y), G(x, y), B(x, y)) and the OCT signal I.sub.OCT(x,y) with dimension N (for N data points) in the depth direction are acquired corresponding to each scanned position (x, y).

[0063] When the 2D scanner 24 scans continuously, a 2D color image is populated with a number L?M of color pixels; correspondingly a 3D OCT volume is reconstructed with values L?M?N. The (R(x, y), G(x, y), B(x, y)) values are inherently registered with I.sub.OCT(x,y) along lateral directions. FIG. 5 part (b) shows the inherent mapping provided for the color content at each point 82 corresponding to the OCT scan, with values (R(x, y), G(x, y), B(x, y)) mapped to corresponding I.sub.OCT(x,y) measurements. FIG. 5 part (c) shows I.sub.OCT(x, y, z.sub.i) with the surface point intensity at (x, y, z.sub.i), wherein Zi is depth of surface from a zero delay line along the A-line OCT signal. The color texture (R(x, y), G(x, y), B(x, y)) is thus mapped directly to the OCT signal at (x, y, z.sub.i).

CLED Structure and Functional Components

[0064] FIGS. 6A, 6B, and 6C show different embodiments of color light emitter/detector CLED 52. Laser diodes LD1, LD2, and LD3 are red, green and blue laser diodes respectively. Lenses L1, L2 and L3 are the corresponding collimation lenses used with each of LD1, LD2, and LD3 to generate collimated light beams.

[0065] In the FIG. 6A arrangement, collimated beams are combined onto the same light path by dichroic mirrors DM1 and DM2. DM1 and DM2 have appropriate cutoff wavelengths for corresponding routing of light to and from the light path, such as center wavelengths of red and green laser diodes LD1 and LD2. Lens L4 couples the light from the shared path into a single mode optical fiber 74 to provide full color or polychromatic light. Full color light that has been back-scattered from the sample S is coupled back to CLED 52. Each color light is coupled back to its original channel through beam splitters BS1, BS2, and BS3 and a portion of the light power is directed to corresponding photodiodes PD1, PD2, and PD3 for measurement.

[0066] In the FIG. 6B configuration, a trichroic beam splitter TBS with dichroic filters F1 and F2, similar to a Philips prism, combines the light from Red, Green, and Blue laser diodes LD1, LD2, and LD3 onto optical fiber 74, coupled through lens L4. Full color light that has been back-scattered from the subject, sample S, is coupled back to CLED 52. Each color light is coupled back to its original channel through trichroic beam splitter TBS.

[0067] In the FIG. 6C configuration, a wave division multiplexer WDM is used for combining and separating the Red, Green, and Blue light from their respective color channels.

[0068] In the FIG. 6D configuration, a fiber combiner 76 is used for combining and separating the Red, Green, and Blue light from their respective color channels.

[0069] In the FIG. 6E configuration, two fiber combiners 76 are used, one for combining the outgoing Red, Green, and Blue light onto a single channel, the other for separating the returned Red, Green, and Blue light to their respective color channels.

[0070] In the FIG. 6F configuration, a wide-bandwidth visible light source, such as a supercontinuum laser SCL, serves as the light source for color imaging. The SCL has a continuous visible spectrum output. Wavelength division multiplexer WDM in the path of returned light separates the backscattered light and redirects the light to each respective photodiode PD1, PD2, PD3. A fiber coupler FC is used to couple the light to and from fiber 74.

[0071] In the FIG. 60 configuration, a pair of wave division multiplexers WDM and variable attenuators VA are used to modulate the SCL light in the emission path. A WDM in the path of returned light separates the backscattered light and redirects the light to each respective photodiode PD1, PD2, PD3. A fiber coupler FC is used to couple the light to and from fiber 74.

[0072] The FIG. 6H configuration shows an embodiment in which RGB light detection is performed at the OCT spectrometer. Input polychromatic light is coupled to the OCT scanning system, as shown in previous FIGS. 6A-6G. Detection of the polychromatic light utilizes a spectral separator 124, such as a grating or prism to provide spectral separation of the light, directing red, green, and blue light. to corresponding detectors 126r, 126g, 126b at appropriate angles, as determined by grating, prism, or other separator characteristics.

Color Image Processing and Calibration

[0073] For system calibration and imaging, the reflectance imaging apparatus should be calibrated to a reference standard. R, G, B laser emission is adjusted to provide equal light intensities. Background signals are captured with sample S removed from the sample arm. The R, G, B photodiodes, PD1, PD2, and PD3 respectively, detect background signals that are reflected from the components in the light path. Background signals are subtracted from the R, G, and B signals, respectively. The color image calibration method is similar to that used in color photography which is also adapted in the calculation flow chart of FIG. 7.

[0074] FIG. 7 shows a color calibration sequence that can be used for the color scanner that performs both OCT and RGB imaging. RGB signals are obtained from a calibration target, such as a white light reference patch, in a reference imaging step 700. Values from a standard color model, such as sRGB of the reference white patch or other calibration target are acquired in a standard color model step 710. A least squares calculation or other suitable method for obtaining the calibration transform between the RGB signals and sRGB is performed at a calibration step 720, generating a calibration matrix at a transform step 730. The calibration matrix is applied to the RGB signals obtained from a reflectance imaging step 732 to generate a calibrated RGB signal 734. This is combined with OCT surface detection obtained in an OCT surface imaging step 740. An attachment step 750 then combines the OCT surface detection data in register with calibrated RGB data to provide a combined output.

[0075] The difference in spectral ranges for the two imaging modes makes the combination of OCT light and RGB color light possible, either using spectral division or amplitude division.

[0076] FIG. 8 shows the spectrum distribution of visible R, G, B light and infrared OCT light waves. As can be readily seen from this mapping, the spectral ranges are non-overlapping. Visible light ranges from about 390 nm to about 700 nm. The infrared light ranges from above 700 nm to about 1600 nm. FIG. 8 also shows related dichroic mirror cut off and bandpass wavelengths for WDM operation. Additional color content can be added to provide more accurate shade matching, such as a violet V wavelength (less than about 450 nm), as shown in the spectral diagram of FIG. 8.

[0077] It should be noted that configurations described with reference to FIGS. 1-6H provide inherent registration of the color texture data with OCT data. Because the same optical path in the sample arm is shared, no additional processing is needed in order to register the two different types of data to each other.

Alternate Approach for Combined OCT and Color Texture Imaging

[0078] An alternate approach to meeting the need for combined OCT and color texture image data uses an OCT scanner that is coupled with a color preview camera for obtaining the needed image content. When using this alternate approach, processing is needed in order to register the color texture data with the OCT scan content. FIGS. 9A through 9E show different embodiments of an imaging apparatus 200 providing this feature.

[0079] In the schematic diagram of FIG. 9A, imaging apparatus 200 has a light source 202 that directs collimated light to an OCT scanner 204 for OCT characterization of the subject tooth or other sample S. Lens L5 focuses the light onto the sample S, such as a tooth or other intraoral feature. A mirror 212 folds the optical path to direct light to and from sample S. A camera 210, positionally coupled to the field of view (FOV) of the OCT scanner apparatus, acquires a reflectance (RGB color) image from subject or sample S. White light or RGB light that serves as illumination for reflectance imaging is directed to sample S along the camera imaging path (not shown). Timing synchronization, for example, can be used to associate the reflectance image with the corresponding OCT imaging content for the same portion of the sample S surface. The RGB imaging path must be calibrated to the OCT light path, as described in more detail subsequently.

[0080] FIG. 9B shows an alternate embodiment of imaging apparatus 200 with a collimated light source 202 and OCT scanner 204 arrangement similar to that of FIG. 9A. Here, lens L5 also forms part of the optical path for camera 210 imaging. Lens L5 focuses the source light for reflectance imaging, as shown.

[0081] FIG. 9C shows an alternate embodiment of imaging apparatus 200 having a different. arrangement of scan mirrors for OCT scanners 204, 206. Here, each OCT scanner is a single-axis scanner. Again, lens L5 also forms part of the optical path for camera 210 imaging.

[0082] FIG. 9D shows an alternate embodiment of imaging apparatus 200 without lens L5 optics for focus. Here, light source 202 provides a focused OCT beam. RGB illumination goes directly to mirror 212 and is reflected toward the subject, sample S. Returned light from the subject is directed back to camera 210 by mirror 212. Camera 210 contains imaging optics (not shown) for imaging the light reflected back from sample S.

[0083] FIG. 9E shows an alternate embodiment of imaging apparatus 200 in which a dichroic surface 208 acts as a hot mirror, reflecting infrared light from OCT light source 202 and transmitting visible light to and from the reflectance imaging apparatus provided by camera 210.

Registration of Color Texture to OCT Imaging Data

[0084] For imaging apparatus that uses OCT scanner coupling with a color preview camera, registration of color texture from the reflectance image to OCT depth characterization data can be performed using various interrelated data acquired in the imaging process.

[0085] Additional data for registration can be obtained by projecting the 3D volume generated using OCT in order to generate a 2D grayscale OCT image. Applying a suitable scanner distortion model to correct for OCT distortion can then restore the correct geometry to surface data.

[0086] Image projection along the A-line direction mimics 2D pinhole camera imaging in terms of FOV and focus length. This treatment can allow stereo vision calibration methods to be applied to the scanner and preview color camera. Matrix calculation can then be used to translate and rotate the 2D color camera image to a 2D grayscale OCT image. This completes lateral registration. Following this, the 3D surface from OCT imaging can be used for digital impression, with color texture associated with surface locations.

[0087] FIGS. 10A and 10B show aspects of registration processing for a tooth 220 as the imaged object using the pinhole camera paradigm. In FIG. 10A, color camera 210 can be modeled as a pinhole camera, with its image sensor at the 2D projection sensor 218 plane. The focus length between the pinhole and sensor 218 plane is labeled f_camera. Labeling O_camera indicates the pinhole origin position for the camera.

[0088] Similarly, the three-dimensional OCT volume can back-project onto a virtual projection plane Q along the direction of each A-line, with an OCT scanner pivot 222 regarded as a pinhole source. The focus length f_oct of a virtual OCT camera can be defined such that it generates a projection plane Q of approximately the same size as the color sensor of camera 210. Through OCT scanner calibration, each A-line beam direction can be obtained in a predefined OCT scanner coordinate. For example, a scanner pivot is defined as origin O_oct. The direction of an OCT scan beam can be calibrated with coordinate (k.sub.x, k.sub.y, k.sub.z). Its projection onto a virtual plane is along coordinate (?k.sub.x, ?k.sub.y, ?k.sub.z). The focus length f_OCT and direction determine the projection position. Due to scanner scanning properties, a fan beam projection shape results, as shown in the OCT projection matrix of FIG. 10B. The pattern shown correctly reflects the actual geometric shape of the scanned object.

[0089] After obtaining the OCT projection image and color image, a 3?3 rotation matrix R and a 3?1 translation matrix T can be obtained using well-known stereovision calibration methods. By translation and rotation, two images can thus be registered. However, due to the projection shape difference, a precise matching color for each A-line is not available. Instead, the final color texture for each A-line can be calculated by interpolating the RGB color with surrounding values from the nearest neighbor pixels.

[0090] Color texture attachment to the corresponding surface point of the A-line is done similarly to the color texture mapping or correlation method previously described with reference to FIG. 5. The color signal with reflectance values (R(x, y), G(x, y), B(x, y)) and the OCT signal I.sub.OCT(x,y) with dimension N (for N data points) in the depth direction are acquired corresponding to each scanned position (x, y). These (R(x, y), G(x, y), B(x, y)) values can be registered with corresponding I.sub.OCT(x,y) data along lateral directions.

[0091] Consistent with an embodiment of the present invention, a computer program utilizes stored instructions that perform on image data that is accessed from an electronic memory. As can be appreciated by those skilled in the image processing arts, a computer program for operating the imaging system in an embodiment of the present disclosure can be utilized by a suitable, general-purpose computer system operating as control logic processor 60 as described herein, such as a personal computer or workstation. However, many other types of computer systems can be used to execute the computer program of the present invention, including an arrangement of networked processors, for example. The computer program for performing the method of the present invention may be stored in a computer readable storage medium. This medium may comprise, for example; magnetic storage media such as a magnetic disk such as a hard drive or removable device or magnetic tape; optical storage media such as an optical disc, optical tape, or machine readable optical encoding; solid state electronic storage devices such as random access memory (RAM), or read only memory (ROM); or any other physical device or medium employed to store a computer program. The computer program for performing the method of the present disclosure may also be stored on computer readable storage medium that is connected to the image processor by way of the internet or other network or communication medium. Those skilled in the art will further readily recognize that the equivalent of such a computer program product may also be constructed in hardware.

[0092] It should be noted that the term memory, equivalent to computer-accessible memory in the context of the present disclosure, can refer to any type of temporary or more enduring data storage workspace used for storing and operating upon image data and accessible to a computer system, including a database, for example. The memory could be non-volatile, using, for example, a long-term storage medium such as magnetic or optical storage. Alternately, the memory could be of a more volatile nature, using an electronic circuit, such as random-access memory (RAM) that is used as a temporary buffer or workspace by a microprocessor or other control logic processor device. Display data, for example, is typically stored in a temporary storage buffer that is directly associated with a display device and is periodically refreshed as needed in order to provide displayed data. This temporary storage buffer is also considered to be a type of memory, as the term is used in the present disclosure. Memory is also used as the data workspace for executing and storing intermediate and final results of calculations and other processing. Computer-accessible memory can be volatile, non-volatile, or a hybrid combination of volatile and non-volatile types.

[0093] It will be understood that the computer program product of the present disclosure may make use of various image manipulation algorithms and processes that are well known. It will be further understood that the computer program product embodiment of the present disclosure may embody algorithms and processes not specifically shown or described herein that are useful for implementation. Such algorithms and processes may include conventional utilities that are within the ordinary skill of the image processing arts. Additional aspects of such algorithms and systems, and hardware and/or software for producing and otherwise processing the images or co-operating with the computer program product of the present disclosure, are not specifically shown or described herein and may be selected from such algorithms, systems, hardware, components and elements known in the art. Exemplary embodiments according to the application can include various features described herein (individually or in combination).

[0094] While the invention has been illustrated with respect to one or more implementations, alterations and/or modifications can be made to the illustrated examples without departing from the spirit and scope of the appended claims. In addition, while a particular feature of the invention can have been disclosed with respect to only one of several implementations/embodiments, such feature can be combined with one or more other features of the other implementations/embodiments as can be desired and advantageous for any given or particular function. The term at least one of is used to mean one or more of the listed items can be selected. The term about indicates that the value listed can be somewhat altered, as long as the alteration does not result in nonconformance of the process or structure to the illustrated embodiment. Finally, exemplary indicates the description is used as an example, rather than implying that it is an ideal. Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by at least the following claims.