METHOD AND APPARATUS FOR SPECTRAL REFLECTANCE IMAGING USING DIGITAL CAMERAS

Abstract

A method and spectral light-based apparatus with an embedded (built-in) spectral calibration module for acquiring multi-spectral reflectance images from a digital camera are disclosed. The apparatus may be an attachment device, which may be integrated with a consumer digital camera (such as smartphone camera), and may measure and/or estimate spectral reflectance and true color values for an object recorded by the camera. An example apparatus comprises an array of monochromatic light sources, preferably pulsed LEDs, irradiating in a time-multiplexed manner to generate light spectra in the range of 400 nm-1000 nm, an optical lens to limit the field of view of the attached camera, an electro-mechanical shutter or plate with its inner (reflection) surface coated with a diffuse reflectance standard to ensure flat spectral response, and an interface module for synchronizing the time-multiplexed light spectra with the coated shutter opening and closing and with the digital frames acquired by the camera, such that the true spectral reflectance and true color value of an object can be measured.

Claims

1. A multi-spectral imaging system comprising: a digital camera comprising an imaging lens having an opening; an electro-mechanical shutter covering the opening; a control module configured to synchronize electrical triggering of a light source, with opening or closing of the shutter such that the light source is triggered sometimes when the shutter is closed and sometimes when the shutter is open.

2. A multi-spectral imaging system comprising: a digital camera comprising an imaging lens having an opening; a light source disposed within the digital camera; an electro-mechanical shutter covering the opening; and a control module operable to synchronize electrical triggering of the light source, with opening or closing of the shutter.

3. A spectral light apparatus that when integrated to a digital camera (such as smartphone camera) produces spectral reflectance and color value measurements of an object/scene recorded by the digital camera, the apparatus comprising a housing supporting: a. spectral light source operable to selectively produce a spectral flash light sequence comprising a sequence of narrow-band light to illuminate a target; b. an optical lens system comprising at lease one lens arranged to align the digital camera field of view to the enclosure opening at the distal end which defines the spectral image field of view; c. an electro-mechanical shutter or plate covering the spectral image field of view when closes and having an inner surface having a known reflectance spectrum; d. a control module operable to synchronize electrical triggering of the spectral flash light sequence with a sequence of open and shut states of the shutter with a sequence of image acquisitions by the camera.

4. Apparatus according to claim 3 comprising processing software configured to execute on a processor of the apparatus to acquire a spectral reflectance image of an object by processing the images captured by the digital camera.

5. Apparatus according to claim 4 wherein the software is configured to calculate the object spectral reflectance using the acquired spectral frames from an object and from the electro-mechanical shutter.

6. Apparatus according to claim 3 wherein the shutter or plate comprises a white or grey reflectance surface.

7. Apparatus according to claim 3 wherein the shutter or plate is movable between a closed configuration position in which it shields the optical field of view of the camera and an open configuration in which but clears the field of view of the camera.

8. Apparatus according to claim 3 wherein the control module interfaces wirelessly and/or through wire connection with the smartphone circuitry.

9. Apparatus according to claim 1 wherein the spectral light source comprises at least 3 LEDs or Laser Diodes which emit light of different wavelengths.

10. Apparatus according to claim 9 wherein the wavelengths are in the range of 400 nm-800 nm.

11. Apparatus according to claim 8 wherein the LEDs or laser diodes are mounted in a circular or rectangular array central around an optical axis of the digital camera.

12. Apparatus according to claim 3 wherein a control module triggers a plurality of the LEDs to emit light of a corresponding plurality of different wavelengths simultaneously.

13. Apparatus according to claim 12 wherein the control module triggers three of the LEDs to emit light of three corresponding different wavelengths simultaneously.

14. Apparatus according to claim 1 wherein the lens is arranged to control optical coupling of the digital camera to capture only reflected light originating from the interaction of light from the spectral light source with an object being imaged.

15. Apparatus according to claim 1 wherein the lens comprises one or more color-corrected aspheric lenses for optimum focusing of the different wavelength light.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings illustrate non-limiting example embodiments of the invention.

[0029] FIG. 1A is a schematic block diagram of an example apparatus for estimating reflectance image spectra using a multi-spectral imaging system with an embedded (built-in) white reflectance-coated shutter calibration module.

[0030] FIG. 1B shows an example reflectance spectrum for the FIG. 1A white reflectance-coated shutter calibration module.

[0031] FIGS. 2A and 2B are schematic cross section views of example spectral imaging attachment devices. The device of FIG. 2A is adapted for macro-imaging. The device of FIG. 2B is adapted for distance imaging.

[0032] FIGS. 3A and 3B are schematic views of a multi-spectral imaging camera with a built-in white reflectance-coated shutter calibration module.

[0033] FIGS. 4A and 4B respectively illustrate example spectral imaging attachments for a smartphone and a webcam or digital camera.

[0034] FIG. 5 is a block diagram illustrating a system for electronic synchronization between an electro-optical attachment and a smartphone/digital camera processor.

[0035] FIGS. 6A, 6B, and 6C illustrate example shutter types suitable for white reflectance-coating.

[0036] FIG. 7 is a process flow diagram illustrating the operation of a mobile app or algorithm for quantifying the depth of skin lesions.

[0037] FIG. 8A is a schematic block diagram of an example RGB digital camera having an embedded standard reflectance-coated shutter calibration module.

[0038] FIG. 8B is an example of the FIG. 8A standard reflectance-coated shutter calibration module, with some example possible reflectance spectra.

[0039] FIG. 9 is a process flow diagram illustrating a dynamic spectral calibration algorithm.

[0040] FIGS. 10A and 10B are schematic illustrations of an example digital camera having an embedded standard reflectance-coated shutter calibration module.

DETAILED DESCRIPTION

[0041] Throughout the following description, specific details are set forth in order to provide a more thorough understanding of the invention. However, the invention may be practiced without these particulars. In other instances, well known elements have not been shown or described in detail to avoid unnecessarily obscuring the invention. Accordingly, the specification and drawings are to be regarded in an illustrative, rather than a restrictive sense.

[0042] One embodiment of the present invention relates to a digital camera-based device having a built-in spectral intensity calibration module. This device may be attached to any digital camera including, for example, a smartphone camera. The device allows the digital camera to perform multi-spectral imaging and/or color shade measurement. The device may dynamically re-calibrate the digital camera when viewing or illuminations conditions change.

[0043] The general principles of spectral imaging using a calibration surface when producing images will now be described, followed by a description of some specific example embodiments of the invention.

[0044] A digital camera system equipped to perform multi-spectral imaging may apply the following principle. The measured signal reflected from some target (TARGT) and recorded by the digital camera system can be described by the following model:

I.sub.TARGT(t,x,y)=F()E()S()R()d,

where is the wavelength of illumination, I.sub.TARGT is the measured spectral reflectance of the target at a certain location and point in time, F() is the response of the optical system (including the response of the optical lens and/or filters), E() is the spectrum of the light source, S() is the camera sensitivity, and R() is the target's real reflectance spectrum.

[0045] A sequentially flashed light (such as the light emitted by a pulsed LED ring), illuminating both a target and a calibration surface at the same distance as the target, may be described by the following model:

I.sub.TARGT(,t.sub.1,x,y)=F.sub.1(,t.sub.1,x,y)E.sub.1(,t.sub.1,x,y)S.sub.1(,t.sub.1,x,y)R(,t.sub.1,x,y);

I.sub.cal(,t.sub.2,x,y)=F.sub.2(,t.sub.2,x,y)E.sub.2(,t.sub.2,x,y)S.sub.2(,t.sub.2,x,y)R(,t.sub.2,x,y);

[0046] where I.sub.cal is the measured reflectance spectrum of the calibration surface (which may comprise a white-coated electro-mechanical shutter for example), R and L are respectively the real reflectance spectra of the target and calibration surface, the subscript 1 refers to properties of the target and the calibration surface at a first time, and the subscript 2 refers to properties of the target and the calibration surface at a second, later time. The difference between t.sub.1 and t.sub.2 (i.e. the length of time between sequential flashes of illuminating light) may be one the order of seconds or milliseconds, for example.

[0047] For simultaneous (or sequential) imaging of the target surface and the calibration surface at the same location as the target, the model becomes:

F.sub.1(,t.sub.1,x,y)=F.sub.2(,t.sub.2,x,y)=F.sub.i();

E.sub.i(,t.sub.t,x,y)=E.sub.2(,t.sub.2,x,y)=E.sub.i();

S.sub.1(,t.sub.1,x,y)=S.sub.2(,t.sub.2,x,y)=S.sub.i();

R()=[I.sub.TARGT/I.sub.cal]L().

[0048] To measure the true target reflectance (or true color image of the target), the reflectance L() of the calibration surface should have a known spectral response within the measured wavelength range. Additionally, measurement of the target and the calibration surface should be done under similar illumination and viewing conditions. It is convenient but not mandatory for the known spectral response of the calibration surface to be flat over the measured wavelength range.

[0049] Some embodiments of the invention integrate a calibration module within a digital camera system. Such a built-in calibration module may estimate the effective spectral sensitivities of the detectors in the camera at the place and time that the camera is used to image a target.

[0050] In some embodiments, a calibration module uses images of a calibration surface coated with a standard flat response diffuse reflectance coating such as SpectraReflect diffuse reflectance coating (which is a water-based barium sulfate coating).

[0051] In some embodiments a calibration surface having a known reflectance at different wavelengths is provided on an inside face of a shutter covering the lens of a digital camera. The spectral response of the shutter may be established by coating the inner surface of the shutter with a standard reflectance coating so that the real spectral response L() is a known, preferably flat, spectral response (that is, L() is preferably relatively constant over the desired wavelength range).

[0052] To correct for variations with wavelength in a spectral response of a calibration surface (e.g. a coated shutter) the calibration module can use a mapping table which indicates a difference between L() and the reflectance of a diffuse reflectance standard at different wavelengths. The difference corresponds to an input value in a lookup table (LUT). The LUT based algorithm then corrects the measured spectral response of the calibration surface, by transforming the input value into an output value.

[0053] Some embodiments automatically set a camera to acquire images of an object illuminated with light of one or more wavelengths and images of a calibration surface illuminated in the same manner. For example, the camera may automatically acquire images of an object and images of a calibration surface (e.g. the inside of a shutter) in alternation. The shutter may be automatically opened and closed to facilitate such a sequence of images. Other possibilities include: [0054] acquiring images of an object between two images of a calibration surface (each of these images may be acquired under the same illumination conditions); or [0055] acquiring images of an calibration surface between two images of an object (each of these images may be acquired under the same illumination conditions); or [0056] acquiring a sequence of images of a calibration surface and an object with the images of the calibration surface acquired within a short time interval of the images of the object (collectively the sequence may include one or more images of the object acquired under each of a plurality of illumination conditions and one or more images of the calibration surface acquired under each of the same plurality of illumination conditions).

[0057] Synchronizing shutter reflectance images with the acquired spectral images of the target may provide a feedback control to ensure that the digital camera system is spectrally calibrated during measurement.

[0058] For an RGB color digital camera, an illuminator may be controlled to perform simultaneous double or triple band illumination to speed up spectral image acquisition and measurements. Each triple band illumination may include light of three wavelength bands with each of the wavelength bands detectable by one of the RGB sensors of the digital camera.

[0059] With these principles in mind, some non-limiting example embodiments of the invention will now be described.

[0060] FIG. 1A illustrates a multi-spectral imaging system 100 with an embedded (built-in) reflectance-coated shutter calibration module. System 100 includes an electro-mechanical shutter 104 coated with a material that provides a known, preferably reasonably high reflectivity. For example, the inner surface of the shutter may be white or gray. The shutter is embedded (built-in) within the housing of multi-spectral imaging system 100 together with a spectral sampling module comprising imaging lens 106; and a control module 108 to synchronize electrical triggering of the light source with opening or closing of shutter 104. Electro-mechanical shutter 104 has a known spectral response, such as the spectrum 110 shown in FIG. 1B.

[0061] In operation, system 100 sends a trigger signal from the control module 108 to close electro-mechanical shutter 104. System 100 sends another trigger signal from the control module 108 to spectral sampling module 106 to illuminate the closed shutter 104 with spectral light. System 100 then produces a series of multi-spectral images Is(,x,y,t) of the closed shutter 104. System 100 then sends a trigger signal from the control module 108 to open the white or grey coated electro-mechanical shutter 104. When the white or grey coated electro-mechanical shutter 104 is open, system 100 produces multi-spectral images It(,x,y,t) of a target object. System 100 may then process the acquired images to estimate the spectral reflectance of the surface of the target object using the following equation: Rt(,x,y,t)=It(,x,y,t)/Is(,x,y,t). This estimation may be performed on a pixel-by-pixel basis.

[0062] System 100 may be configured to acquire a series of multi-spectral images It(,x,y,t) of the surface of a target object and a series of multi-spectral images Is(,x,y,t) of the inner surface of electro-mechanical shutter 104 within a short time frame. In each image, the surface of a target object and the shutter 104 may be illuminated by light of a different wavelength (or different combination of discrete wavelengths). These images may then be processed on a per-pixel basis, to determine a spectral calibration which relates a detected light intensity to a reflectance of a target object at a given wavelength, using the following equation: Rt(,x,y,t)=It(,x,y,t)/Is(,x,y,t).

[0063] System 100 may be provided as an attachment to any standard digital camera 102, such as a smartphone camera.

[0064] Synchronization between operation of the camera to acquire images, operation of the light source of spectral sampling module 106 and operation of shutter 104 may be facilitated by software running on the camera or on a computer that controls the camera. For example, where the camera is the camera of a smartphone the software may comprise an app running on the smartphone or may be built into an operating system of the smart phone. where the camera is a stand-alone digital camera the software may be included in firmware of the digital camera, where the camera is an accessory connected to a computer the software may comprise application software executing on the computer. The software may communicate to control module 108 by a wired interface (e.g. by sending signals to control module 108 by way of a USB interface or other digital data interface provided as part of the camera or a computer controlling the camera) or by way of a wireless interface (e,g, by way of a Bluetooth or WiFi or other wireless signal transmitted from a wireless interface of the camera or a computer controlling the camera and received by a wireless interface connected to control module 108.

[0065] In a non-limiting example embodiment, the software executing on the camera or associated computer controls the camera to take a series of images (either as still images or a series of frames of a video image). For each image the software sends a synchronization signal to control module 108. Control module 108 controls light source of spectral sampling module 106 to emit light of one or more colors desired for the current image and controls shutter 104 to be open or closed depending on whether the current image is intended to be a calibration image or an image of the object.

[0066] Control of which images are to be calibration images and what colors of light illuminate each image may be in control module 108 or in the software that controls the camera. Where control over these parameters are in the software that controls the camera then the software may include information specifying whether shutter 104 should be open or closed and what spectral bands should be enabled for illumination either as part of the synchronization signals or in a separate communication.

[0067] The software may automatically associate together the acquired images. The software may optionally perform analysis of the acquired images.

[0068] FIGS. 2A and 2B illustrate multi-spectral imaging attachments for a smartphone camera. FIG. 2A shows an attachment useful for macro imaging, such as dermoscopy imaging in the field of dermatology. FIG. 2B shows an attachment useful for distance imaging, such as clinical imaging in the field of dermatology.

[0069] As shown in FIG. 2A, apparatus 200 includes a device housing 202 that contains: a light source 204 comprising one or more monochromatic light sources for supplying spectral light illumination; an optical lens system 208 for controlling the field of view of digital camera 102 and for capturing reflected spectral light; and an electrically-controlled mechanical shutter or plate 212 coated with standard white reflectance for calibrating the spectral response of digital camera 102. Apparatus 200 further includes an electronic interface module 206 that selects the monochromatic light source(s) of light source 204 to be triggered and synchronizes the triggering sequence with the frames acquired by the attached digital camera 102. The synchronization may be achieved through an electrical communication protocol. For example, the acquisition time of image frames by the camera may be synchronized with the times that spectral light source 204 is on.

[0070] A communication protocol may initiate the on signal for spectral light source 204 when the attached digital camera 102 receives a trigger signal to acquire images. Another communication protocol may synchronize the frequency of the on time for spectral light with the frame rate of the attached digital camera 102. Apparatus 200 further includes a hood 210 to collect and diffuse light. In addition, hood 210 is useful as a fixed spacer between a target 216 (e.g. skin) and optical lens system 208 to facilitate focusing light to the target 216 for macro-imaging.

[0071] Light source 204 provides light in the visible or in the visible and near-infrared region, and may be a single unit or a combination of light sources. Light source 204 comprises an array of LEDs which emit light at different wavelengths. In order to construct a full spectral reflectance image of target 216, the number of different wavelengths may be more than ten. In some embodiments, light source 204 comprises ten or more groups of LEDs or other substantially monochrome light sources. Each group of LEDs emits light of a particular wavelength. Each group of LEDs comprises one or more LEDs. The LEDs may be arranged in two or more concentric circles. An angular optical diffuser may be used to direct light from light source 204 to evenly illuminate target 216.

[0072] Other possible light sources include: one or more lasers (e.g. one or more laser diodes (LDs)) or a broadband light source used in conjunction with narrow-band filters. The broadband light source may comprise, for example, a xenon lamp, a mercury lamp, a tungsten halogen lamp, or a metal halide lamp. The narrow-band filters may be attached to filter wheels. Light emitted from light source 204 can be modulated into a plurality of illuminating segments.

[0073] Electronic interface module 206 triggers illumination by light source 204 such that target 216 is illuminated by light of one or more selected wavelengths in a manner that is synchronized with acquisition of image frames captured by attached digital camera 102. The digital camera 102 may be operating in a video acquisition mode or a still-image acquisition mode. The captured image frames coming from digital camera 102 and synchronized with the illumination sequence of light source 204 are collected and analyzed by a processing unit (not shown) to estimate the reflectance spectrum of target 216. Processing may occur automatically upon acquisition of the images or may be performed subsequently.

[0074] A processing unit receives multi-spectral images of a target 216, captured at different wavelengths. By correlating the On/Off timing of spectral light with the frame acquisition time, the processing unit (not shown) marks each image with its corresponding wavelength(s) of illumination. For example, the processing unit (not shown) receives multi-spectral images of the target 216 It1(x,y, 1), It2(x,y, 2), It3(x,y, 3), . . . Itn(x,y, n) and multi-spectral images of the coated shutter 212 Is1(x,y,1), Is2(x,y, 2), Is3(x,y, 3), . . . Isn(x,y, n) in real time. To estimate the spectral reflectance images R(x, y, ) of the target 216, the processing unit normalizes the multi-spectral images of the target 216 with the multi-spectral images of the coated shutter 212.

[0075] The processing unit may use the estimated spectral reflectance images to estimate perceptual color components, e.g. the RGB values. This relies on the CIE XYZ color space model: a

X=dx()I()

Y=dy()I()

Z=dz()I()

Where x (), y (), and z () are the CIE color matching functions, and I () is the target reflectance spectra.

[0076] The processing unit may be embedded in camera 102 or housing 202, or may be external. In some embodiments, processing is performed in all or in part by a smartphone app (or a web app) that runs on the device hosting camera 102 instead of by a separate processing unit.

[0077] Optical lens system 208 may comprise macro lenses with variable optical fields of view. The macro lenses may be corrected by suitable coatings for use in the spectral range of 400 nm-800 nm. Diffuse white reflectance material is coated or mounted on the side of shutter 212 which is facing lens system 208 Apparatus 200 uses the spectral measurements obtained from shutter 212 to calibrate the spectral response of the color sensor in digital camera 102.

[0078] FIG. 2B shows an example embodiment of the multi-spectral imaging device with an attachment for use with distance imaging (together apparatus 250). Apparatus 250 includes a diffuser 258 to diffuse light to illuminate larger areas. Apparatus 250 may be used at a distance and not in direct contact with the target 216. Diffuse reflection is calibrated using near light measurements 258 and then diffused 260 for wide-field illumination. The distance imaging device may then function in the same manner as apparatus 200 as described above.

[0079] FIG. 3A shows a multi-spectral imaging digital camera system 300 with built-in spectral calibration, according to another example embodiment of the invention. System 300 comprises a camera sensor 302 housed in a camera enclosure 312, a lens 304 (typically a macro lens), an LED ring 308 comprising emitters of different monochromatic wavelengths (e.g. wavelengths in the range of 400 nm-1000 nm) and coupled to a polarizer 306, and an electrically-controlled shutter 310 (such as rotary drive shutter) coated with standard white reflectance. Digital camera system 300 may operate to acquire multi-spectral images as described above.

[0080] FIG. 3B shows an example absorption filter-based spectral imaging digital camera system with a built-in spectral calibration module, together apparatus 350. For example, the digital camera system may comprise tunable filters or acousto-optic filters 356. Possible embodiments of light source for apparatus 350 include a xenon lamp, a mercury lamp, a tungsten halogen lamp, a metal halide lamp, a laser, one or more laser diodes (LDs), and/or natural light. Apparatus 350 includes an electrically-controlled shutter 358. The shutter 358 has an opening to allow light to illuminate the shutter 358. Light reflected from the shutter 358 is filtered through tunable filters or acousto-optic filters 356 to produce light of discrete wavelengths. The filtered light is then used to estimate the spectral reflectance images R(x, y, ) of a target.

[0081] FIG. 4A shows a smartphone attachment 400, according to another example embodiment of the invention. Multi-spectral imaging attachment 400 may be connected to a smartphone 402. A smartphone app may be used to synchronize the illumination sequence of a spectral light source with the acquired image frames of a target from the smartphone camera 404. In operation, to obtain spectral reflectance measurements using the integrated device and smartphone camera 404, an initial measurement is obtained with a white or grey reflectance shutter 410 covering the field of view of camera 404. A smartphone app sends a trigger signal to close the shutter 410. The smartphone app then sends a trigger signal to turn on spectral light of a specified wavelength. The smartphone app then sends a trigger signal to the smartphone camera to acquire an image of the shutter 410 in real time. Once an image is acquired, the smartphone app sends a trigger signal to turn off spectral light of a the specified wavelength and to turn on spectral light of a different wavelength. This process repeats until spectral light of all different wavelengths have been used to produce multi-spectral images of the coated shutter 410: Is1(x,y,1), Is2(x,y, 2), Is3(x,y, 3), . . . Isn(x,y, n) in real time. The smart app then sends a trigger signal to open the coated shutter 410. With the shutter 410 removed from the field of view of camera 404, multi-spectral images of a target may be acquired: It1(x,y, 1), It2(x,y, 2), It3(x,y, 3), . . . Itn(x,y, n). The smartphone app may synchronize the frequency of the on time for spectral light with the frame rate of the attached digital camera. The acquired multi-spectral images may be stored in a data store of the smartphone 402 and/or transmitted to another destination (e.g. the acquired images may be uploaded to a FTP site or to a cloud-based data store or transmitted to an email account or the like.

[0082] An algorithm, which may be implemented by an app stored on smartphone 402 or by a processor on a separate device may be used to determine the reflectance spectra of the imaged target on a pre-pixel basis, based on the calibration surface (e.g. shutter 410) and target surface measurements. To estimate the spectral reflectance images R(x, y, ) of the target, the algorithm normalizes the multi-spectral images of the target with the multi-spectral images of the coated shutter 410.

[0083] In another example embodiment, an attachment 452 can be used with a web cam 454, as shown in FIG. 4B. A computer program may be used to synchronize the illumination sequence of a spectral light source with the acquired image frames of a target from webcam 454.

[0084] FIG. 5 is a block diagram 500 showing selected electrical components in a multi-spectral imaging attachment for use with a smartphone camera, which may be used to generate synchronized video frames for different illumination wavelengths. This embodiment includes an interface portion 502 which receives frame timing signals from the camera. The timing signals may be electrical signals, such as TTL signals and pulsed electrical signals. Interface portion 502 may also interact with a digital camera controller 506 and a shutter control circuit 508 in order to facilitate illumination of the selected LEDs. Interface portion 502 triggers an LED driving circuit 504 to control illumination of selected LEDs in a synchronized manner. The synchronization can be wireless, preferably using Bluetooth, or can be implemented through a wired connection. For example, interface portion 502 sends concurrent electrical signals to the camera 506 to start frame acquisition; to the shutter 508 to close the shutter; and to LED driving circuit 504 to start illumination by LEDs of a specified wavelength. When interface portion 502 receives a trigger signal from the camera 506 indicating that multi-spectral images have been acquired from light of a specified wavelength, interface portion 502 sends a trigger signal to stop illumination by light of a specified wavelength but start illumination by light of a different wavelength. This process repeats until spectral light of all different wavelengths have been used to produce multi-spectral images.

[0085] For a typical RGB color camera, illumination can be triggered at three different wavelengths simultaneously, with one wavelength in the blue band of the spectrum, a second wavelength in the green band of the spectrum, and a third wavelength in the red band of the spectrum.

[0086] FIG. 6 shows some examples of electro-mechanical shutter mechanisms 602, 604, and 606, which are suitable for use with a built-in spectral calibration module. Shutter 602 is a leaf shutter comprising overlapping metal blades. When the shutter control circuit receives a trigger signal, shutter 602 opens by moving the blades to uncover the lens. Shutter 604 is a focal plane shutter. The two curtains of shutter 604 move in opposite directions to uncover the lens. Shutter 606 is a flag shutter. The blade of shutter 606 pivots downward to uncover the lens. Other possible electro-mechanical shutter mechanisms include rotor drive shutters solenoid shutters, and stepper motor shutters.

[0087] Apparatus as described herein may be used to acquire digital images for skin diagnosis and computer-aided diagnosis (CAD), for follow-ups using image archiving, and for tele-dermatology, among other applications.

[0088] FIG. 7 is a block diagram of an example method 700 for generating depth-resolved skin images from acquired spectral images. Method 700 accesses optical properties of the tissue being imaged. The optical properties of the tissue relate to the penetration depth of light of different wavelengths. These properties may include optical absorption coefficient and optical scattering coefficient. For example, skin and the coated shutter may be imaged while being illuminated with light of multiple different wavelengths to produce a set of multi-spectral images. The images may then be used to measure the reflectance spectra of skin as a function of wavelength. An inverse model algorithm using a radiative transfer model is used to calculate an expected depth of penetration at each of the plurality of wavelengths, while taking into account the optical properties of the tissue being imaged.

[0089] In an example embodiment apparatus as described above is used to image a skin lesion. The apparatus yields images of the skin lesion illuminated by light of a plurality of corresponding wavelengths. A computer implemented algorithm is used to calculate the contrast of the skin lesion to the surrounding tissue. The algorithm transforms the contrast images at different wavelengths into contrast images at different depths. As such, the algorithm creates 2D cross sectional images of skin lesion at different depths, which allow the user to visualize the skin lesion.

[0090] Method 700 will now be described in detail. Method 700 may be implemented, for example, using apparatus 200 as shown in FIG. 2A. In block 702, light source 204 (e.g. a plurality of LEDs) sequentially illuminates target 216 with different monochromatic wavelengths (for example, in total ten or more wavelengths in the range of 400 nm-1000 nm). This produces a plurality of spectral images 704 of target 216. Spectral measurements 706 of shutter 212 may be combined with spectral images 704 to create a reflectance estimate 708 of target 216. This results in a plurality of reflectance images 710 of target 216 indexed by the wavelength of the illumination.

[0091] Reflectance images 710 are then provided to inverse model algorithm 712. Algorithm 712 may use a radiative transfer model, such as a diffusion approximation model, to calculate the expected depth of penetration at each wavelength of illumination, taking into account the optical properties of target 216 as measured in block 702. Algorithm 712 outputs a plurality of reflectance images 714 of target 216 indexed by the expected depth of penetration of target 216.

[0092] Reflectance images 714 are then input into algorithm 716, which calculates the contrast of target 216 to its surroundings. For example, target 216 may be a skin lesion, and algorithm 716 may calculate the contrast in the reflectance of the skin lesion with the reflectance of tissue surrounding the lesion. Optionally, at block 718, the maximum depth of target 216 (for example, a skin lesion) may be calculated, and a corresponding depth image may be produced.

[0093] FIG. 8 is a schematic block diagram of an example RGB digital camera 802 having an embedded shutter calibration module. The shutter calibration module may comprise, for example, an electro-mechanical shutter 804, such as a focal plane shutter, coated with a number of different color reflectance surfaces 850 having known reflectance spectra (e.g. spectra 852 and 854), as shown in FIG. 8B.

[0094] Digital camera 802 may be used to enhance the effect of natural light and to provide an independent light source. For example, digital camera 802 may capture a plurality of RGB digital images of a target. These images may then be re-calibrated under the same viewing and illumination conditions (i.e. the same time and the same place) using shutter 804. Digital camera 802 may measure the surrounding natural light reflected from the coated shutter 804 either before or during the image acquisition of a target. Digital camera 802 may be configured to use measurements of the coated shutter 804 to calibrate the spectral reflectance matrix of the target. This process produces a more accurate spectral reflectance matrix (H.sub.t). H.sub.t may then be used to calibrate the acquired spectral images to estimate the spectral reflectance of the target.

[00001]$\stackrel{H_t}{}.Math.\left[\begin{array}{c}1\\ 2\\ 3\\ 4\\ .Math..Math.k\end{array}\right]=\left[\begin{array}{ccc}h.Math..Math.1.Math.R& h.Math..Math.1.Math.G& h.Math..Math.1.Math.B\\ h.Math..Math.2.Math.R& h.Math..Math.2.Math.G& h.Math..Math.2.Math.B\\ h.Math..Math.3.Math.R& h.Math..Math.3.Math.G& h.Math..Math.3.Math.B\\ & h.Math..Math.4& \\ & h.Math..Math.5& \\ & \mathrm{hk}& \end{array}\right].Math.\left[\begin{array}{c}R\\ G\\ B\end{array}\right]$

[0095] A spectral reflectance matrix may be obtained from the calibration procedure described above. To estimate the spectral reflectance matrix , a regression analysis, such as the Wiener estimation, may be use to analyze the RGB measurements obtained from a number of different color reflectance surfaces 850 having known reflectance spectra. The spectral reflectance matrix may be used to estimate spectral reflectance of other targets, assuming the images of these targets have been acquired under the same illumination and viewing conditions.

[0096] The measurements of coated shutter 804 may be used as reference measurements.

[0097] Digital camera 802 may capture a plurality of RGB digital images of target 216. These images may then be re-calibrated under the same viewing and illumination conditions (i.e. the same time and the same place) using shutter 804 and a dynamic calibration algorithm.

[0098] A dynamic calibration algorithm for updating the reflectance spectrum matrix, using the coated shutter reflectance measurement, is proposed to produce updated estimation matrix.

[0099] FIG. 9 is a block diagram of an example dynamic calibration algorithm 900. In block 902, the initial calibration of RGB digital camera 802 is performed using a color standard placed at target 216 to obtain the target RGB reflectance estimator matrix H.sub.t. In block 904, shutter calibration may then be performed to obtain the coated shutter RGB reflectance estimator matrix H.sub.s, as well as the correlation between H.sub.t and H.sub.s. The system calibration matrix H=H.sub.t/H.sub.s should be fixed for a particular RGB digital camera 802. For new measurements under different viewing and illumination conditions, the reflectance of shutter 804 may be evaluated dynamically. If the reflectance of shutter 804 is changed in response to new viewing or illumination conditions, then H.sub.s is updated (as in block 906) and used to calculate an updated target RGB reflectance estimator matrix H.sub.t=H.sub.s.Math.H.sup.1 (block 908).

[0100] In this way, RGB images obtained under different viewing and/or illumination conditions may be re-calibrated under the same viewing and illumination conditions as the initial calibration conditions. For example, when the built-in calibration device detects a change in viewing and/or illumination conditions, it may dynamically re-calibrate the camera system by updating H based on current viewing and illumination conditions.

[0101] FIGS. 10A and 10B are schematic illustrations of an example digital camera 1000 having an embedded standard reflectance-coated shutter calibration module and equipped to implement dynamic calibration (e.g. using method 900), according to an example embodiment of the invention. FIG. 10A shows camera 1000 with built-in coated shutter calibration module 1008. This system may be used to estimate the reflectance spectra of a target 1014, as shown in FIG. 10B. Camera 1000 supports a camera lens 1002 (preferably an autofocus lens, variable lens, or other lens with a large depth of field), which is connected to module 1008.

[0102] Module 1008 may be made of transparent plastic or glass so that incident light from the surroundings is able to illuminate shutter surface 1010. This allows the surroundings to be sampled and measured during shutter image acquisition. The camera system is able to trigger the shutter to open and close, and is able to synchronize shutter calibration with the acquisition of digital camera frames. Dynamic re-calibration of module 1008 may remove the confounding effect of illuminant color, such that camera 1000 only acquires surface color images (i.e. RGB bands measured for a surface viewed under canonical lighting conditions).

[0103] While preferred embodiments of present invention are shown and described above, it is envisioned that those skilled in the art may devise various modifications of the present invention without departing from the spirit and scope presented herein.

Interpretation of Terms

[0104] Words that indicate directions such as vertical, transverse, horizontal, upward, downward, forward, backward, inward, outward, vertical, transverse, left, right, front, back, top, bottom, below, above, under, and the like, used in this description and any accompanying claims (where present), depend on the specific orientation of the apparatus described and illustrated. The subject matter described herein may assume various alternative orientations. Accordingly, these directional terms are not strictly defined and should not be interpreted narrowly.

[0105] While elements are at times shown as being performed sequentially, they may instead be performed simultaneously or in different sequences. It is therefore intended that the following claims are interpreted to include all such variations as are within their intended scope.

[0106] Where a component (e.g. a camera, sensor, shutter, control, software module, processor, assembly, device, circuit, etc.) is referred to above, unless otherwise indicated, reference to that component (including a reference to a means) should be interpreted as including as equivalents of that component any component which performs the function of the described component (i.e., that is functionally equivalent), including components which are not structurally equivalent to the disclosed structure which performs the function in the illustrated exemplary embodiments of the invention.

[0107] Specific examples of systems, methods and apparatus have been described herein for purposes of illustration. These are only examples. The technology provided herein can be applied to systems other than the example systems described above. Many alterations, modifications, additions, omissions, and permutations are possible within the practice of this invention. This invention includes variations on described embodiments that would be apparent to the skilled addressee, including variations obtained by: replacing features, elements and/or acts with equivalent features, elements and/or acts; mixing and matching of features, elements and/or acts from different embodiments; combining features, elements and/or acts from embodiments as described herein with features, elements and/or acts of other technology; and/or omitting combining features, elements and/or acts from described embodiments.