Time-resolved hyper-spectral single-pixel imaging

Abstract

A time-resolved hyper-spectral imaging system for imaging a sample, includes a radiation source suitable for illuminating the sample repeatably, a first optical system configured to form an image I of the sample on a spatial light modulator forming a transmission or reflection mask P, a processor connected to the spatial light modulator and configured to make the transmission or reflection mask P vary for each repetition of the illumination, a second optical system suitable for focusing the radiation transmitted or reflected by the spatial light modulator so as to form, in its image focal plane, a partial image S=P.Math.I; the imaging system being wherein it comprises: a dispersive device comprising a slit placed in the image focal plane of the second optical system, the dispersive device being suitable for spatially splitting the various wavelengths of the radiation transmitted or reflected by the spatial light modulator; a streak camera arranged so as to be illuminated by the radiation issuing from the dispersive device and configured to acquire a plurality of time-resolved partial images of the sample, the images being associated with respective and different transmission or reflection masks P, the streak camera being connected to the processor and the processor also being configured to combine the partial images of the sample so as to construct a 4D image cube I.sub.tot forming an image resolved in time and in wavelength of the sample; and corresponding time-resolved hyper-spectral imaging method for imaging a sample.

Claims

1. A time-resolved hyper-spectral imaging system for imaging a sample, comprising: a radiation source suitable for illuminating the sample repeatably, a spatial light modulator forming a transmission or reflection mask P, a first optical system configured to form an image I of the sample on a spatial light modulator, a processor connected to the spatial light modulator and configured to make the transmission or reflection mask P vary for each repetition of the illumination, a second optical system suitable for focusing the radiation transmitted or reflected by the spatial light modulator so as to form, in its image focal plane, a partial image S=P.Math.I, said imaging system being wherein it comprises: a dispersive device comprising a slit placed in the image focal plane of the second optical system, said dispersive device being suitable for spatially splitting the various wavelengths of the radiation transmitted or reflected by the spatial light modulator; a streak camera arranged so as to be illuminated by the radiation issuing from the dispersive device and configured to acquire a plurality of partial images resolved in time and in wavelength of the sample, said images being associated with respective and different transmission or reflection masks P, said streak camera being connected to the processor and said processor also being configured to combine said partial images of the sample so as to construct a 4D image cube I.sub.tot forming an image resolved in time and in wavelength of the sample.

2. The imaging system according to claim 1, comprising a third optical system suitable for focusing the radiation coming from the radiation source onto the sample and for collecting the radiation issuing from the sample, said imaging system moreover comprising a beamsplitter arranged so as to split the radiation coming from the sample and the radiation coming from the radiation source.

3. The imaging system according to claim 2, comprising a linear stage suitable for moving the sample along the optical axis of the third optical system.

4. The imaging system according to claim 1, wherein the dispersive device comprises a diffraction grating.

5. The imaging system according to claim 1, wherein the streak camera and the dispersive system are suitable for simultaneously acquiring partial images with more than 20 different wavelength channels.

6. The imaging system according to claim 1, wherein the radiation source is connected to the processor so that the repetition of the illumination emitted by the radiation source is synchronized with the change of the transmission or reflection mask of the spatial light modulator.

7. The imaging system according to claim 1, wherein the spatial light modulator is a digital micromirror device, forming a binary reflective intensity mask.

8. The imaging system according to claim 7, wherein the processor is configured so that the binary reflection masks of the digital micromirror device that are associated with the partial images of the sample that are combined to form the image resolved in time and in wavelength of the sample form a Hadamard, Fourier or wavelet basis.

9. The imaging system according to claim 8, wherein the processor is configured to combine said partial images of the sample so as to construct a 4D cube forming an image resolved in time and in wavelength of the sample by implementing a matching-pursuit or basis-pursuit algorithm.

10. The imaging system according to claim 1, wherein the radiation source is a pulsed source.

11. The imaging system according to claim 1, wherein the radiation source is a laser source, a laser diode for example.

12. A time-resolved hyper-spectral imaging method for imaging a sample, comprising the following steps: a. illuminating the sample with a radiation source; b. forming the image of the sample with a first optical system on a spatial light modulator forming a transmission or reflection mask P; c. focusing the radiation transmitted or reflected by the spatial light modulator using a second optical system onto a dispersive device, and spatially splitting the various wavelengths of the radiation transmitted or reflected by the spatial light modulator using said dispersive device; d. acquiring, using a streak camera arranged so as to be illuminated by the radiation issuing from the dispersive device, a time-resolved partial image S=P.Math.I of the sample, which image is associated with one transmission or reflection mask P; e. repeating steps a) to c) while varying the binary transmission or reflection mask P of the spatial light modulator using a processor connected to the spatial light modulator, in order to acquire, with the streak camera, a plurality of time-resolved partial images of the sample, said images being associated with respective transmission or reflection masks P; f. combining, using a processor connected to the streak camera, said respective time-resolved partial images of the sample so as to construct a 4D cube I.sub.tot forming an image resolved in time and in wavelength of the sample.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) Other features, details and advantages of the invention will become apparent on reading the description, which is given with reference to the appended drawings, which are given by way of example and show, respectively:

(2) FIG. 1, a schematic view of a time-resolved hyper-spectral imaging system for imaging a sample according to a first embodiment of the invention.

(3) FIG. 2, a schematic view of the dispersive device end of the streak camera of the imaging system of FIG. 1.

(4) FIG. 3, a schematic view of a time-resolved hyper-spectral imaging system for imaging a sample according to a second embodiment of the invention.

(5) FIG. 4A an image integrated with respect to wavelength and time of a signal of photoluminescence brightness of a point source, and FIG. 4B an image of the time-resolved and wavelength-resolved variation in the photoluminescence signal of the brightest pixel of FIG. 4A.

(6) Below, when the same references are used in more than one figure, they designate the same elements.

DETAILED DESCRIPTION

(7) FIG. 1 illustrates a schematic view of a time-resolved hyper-spectral imaging system 10 for imaging a repeatable event according to a first embodiment of the invention. The system comprises a radiation source 1. In the embodiment of FIG. 1, this source is a pulsed source of monochromatic radiation such as a solid-state laser. The laser source is connected to a processor that is configured to control the synchronization of the laser pulse with the rest of the system. In another embodiment, the radiation source is continuous-wave and intensity-modulated. In another embodiment, the radiation source may be a vapour lamp.

(8) The radiation 2 generated by the source 1 illuminates a sample 3. This sample may absorb the incident radiation 2 and emit a photoluminescence signal 4 or quite simply reflect or transmit some of the incident radiation 2. In the embodiment of FIG. 1, the sample emits a polychromatic photoluminescence signal or radiation 4 after having absorb some of the incident radiation 2.

(9) Below, the event that it is desired to resolve temporally and spectrally is considered to be a repeatable event, the effect being triggered again with a new laser pulse for example.

(10) Below, the expressions radiation emitted by the sample, radiation transmitted by the sample and radiation reflected by the sample are considered to be equivalent.

(11) The radiation emitted by the sample 4 is focused by a first optical system 4 onto a spatial light modulator (SLM) 6, which is located in the image focal plane of the optical system. This optical system 4 is an optical objective. Preferably, it possesses a high numerical aperture. It may be a single lens, for example a convergent biconvex lens, or a convergent complex set of lenses. The optical system therefore forms an image I of the sample 3 in its image focal plane.

(12) The SLM is configured to form a binary reflection or transmission intensity mask P. In the embodiment of FIG. 1, this SLM 6 is a digital micromirror device (DMD).

(13) By way of example, the DMD of the imaging system 10 consists of 1024×768 side-by-side micro-mirrors, the side length of each mirror being about 13 μm. The mirrors are controllable electrostatically and may be inclined (state “1”) or not (state “0”) by a fixed angle with respect to the normal to the DMD, in general by an angle of 12°, the radiation reflected by the mirrors in state 1 making an angle of 24° to the incident radiation 4.

(14) Since the mirrors in state 1 are coupled to the rest of the optical setup, they have a reflection coefficient equal to 1, the mirrors in state 0 corresponding to a reflection coefficient of zero. Thus, the DMD forms a controllable binary intensity mask.

(15) The inclination of the micro-mirrors and therefore the pattern of the binary mask P of the DMD is controllable by a processor 12 connected to the DMD.

(16) In another embodiment the SLM is a device based on liquid crystals able to operate in transmission or in reflection and forming a binary intensity mask P.

(17) The imaging system 10 comprises a second optical system 7 of focal length f, suitable for focusing the radiation transmitted or reflected by the spatial light modulator so as to form, in its image focal plane, a partial image of the sample S=P.Math.I.

(18) In the image focal plane of the second optical system, the imaging system 10 comprises a dispersive device 8 allowing the various wavelengths of the radiation to be spatially split and thus a spectral resolution to be obtained. This dispersive device is coupled to a streak camera 11 allowing a temporal resolution to be obtained.

(19) In the embodiment of FIG. 1, this dispersive system is composed of a slit placed in the image focal plane of the second optical system, of a plurality of mirrors and of a diffraction grating, which are arranged so as to spatially split the various wavelengths of the radiation in a defined xz-plane. The z-direction is the optical axis of the first optical system. Thus, on exiting the dispersive system, each wavelength of the radiation 9 propagates simultaneously in a different direction located in the xz-plane. A schematic view of the dispersive device 8 and of the streak camera 11 is illustrated in FIG. 2.

(20) The slit 21, the direction of the slit being the y-direction, is placed in the focal plane of the second optical system 7, acts as a spatial filter and allows the beam to be cleaned by removing scatter of the radiation emitted by the source 1. A first collimating mirror 22 allows the radiation to be directed toward a reflective diffraction grating 23. The diffraction grating allows the various wavelengths of the radiation in the reflective radiation 9 to be spatially split, each wavelength possessing a different propagation direction located in the same horizontal xz-plane. A second mirror 24 allows the radiation to be directed toward the streak camera 11.

(21) As illustrated in FIG. 2, the streak camera comprises a photocathode 25. The radiation 9 striking the photocathode produces, via the photoelectric effect, electrons that are accelerated by an electric field generated between two deviation plates 26 that are connected to a voltage generator 27. The electrons are diverted vertically in a direction y parallel to the normal to the deviation plates under a variable electric field. By modulating the voltage difference between the plates 26, the electric field rapidly varies, thus providing a deviation of the electrons that varies over time. The later the time at which the electrons arrive between the deviation plates, the higher the electric field, which thus produces a greater vertical movement. The deviation of the electrons is imaged using a microchannel plate (MCP), which allows the number of deviated electrons to be multiplied, and a phosphorus screen 29. The impacts of the electrons on the phosphorus screen are observed using a two-dimensional sensor 20, a CCD sensor for example.

(22) The voltage generator 27 is connected to the processor in order to synchronize the variation in the electric field between the deviation plates of the streak camera with the laser pulse emitted by the source 1 of laser radiation.

(23) The vertical axis of the CCD camera therefore allows the dynamic behaviour of the partial image S of the sample, which image is associated with the binary reflection mask P, to be temporally resolved and the horizontal axis allows the partial image to be split wavelengthwise and therefore the sample or the luminous event to be viewed hyper-spectrally.

(24) The temporal resolution is set by the CCD sensor and the scan frequency of the voltage of the streak camera; it is typically about 1 picosecond with a set scan frequency and about 15 picoseconds with a variable scan frequency. It will be noted that when the duration of the laser pulse is greater than the temporal resolution of the streak camera, then it is this duration that sets the temporal resolution of the imaging system.

(25) The spectral resolution is set by the diffraction grating, provided that the pixel size of the CCD camera is smaller than the minimum wavelength resolution of the diffraction grating.

(26) The number of temporal channels T detectable by the imaging system 10 is set by the vertical dimension of the CCD sensor. The number A of wavelength channels simultaneously detectable by the imaging system 10 is set by the horizontal dimension of the CCD sensor. By suitably choosing the sensor, it is therefore possible to carry out hyper-spectral imaging, i.e. to simultaneously image 20 or more channels of wavelengths of the photoluminescent radiation issuing from the sample.

(27) As briefly explained above, the complete reconstruction of the image of the repeatable event or of the sample requires a plurality M of measurements to be performed in order to acquire, with the streak camera 11, a plurality M of partial images S.sub.1, . . . , S.sub.M, which are associated with distinct respective binary transmission or reflection masks P.sub.1, . . . , P.sub.M of the same event or of the sample. The partial images S.sub.1, . . . , S.sub.M are obtained by illuminating the sample with a new light pulse emitted by the radiation source 1, by changing the binary mask P of the DMD between each pulse using the processor and by carrying out a new acquisition with the streak camera. As mentioned above, the radiation source, the DMD and the streak camera all being connected to the processor, the latter is configured to suitably synchronize the light pulse, the change of the binary mask of the DMD and the acquisition by the streak camera.

(28) When a sufficient number of partial images associated with distinct binary patterns has been acquired, it is possible to reconstruct a complete image I.sub.tot of the sample by post-processing. The reconstructed image is a 4D image cube I(x, y, λ, t), with (x, y) the spatial coordinates of the image, λ the wavelength and t time. The image cube I.sub.tot ∈.sup.N×N×ΛT of N×N pixels is defined, T being the number of temporal channels detected with the streak camera and A being the number of wavelength channels detected with the streak camera, where I.sub.tot=(I.sub.1,1, . . . , I.sub.λ,t, . . . , I.sub.Λ,T), I.sub.λ,T ∈.sup.N×N is the reconstructed image corresponding to the λ-th spectral channel and to the t-th temporal channel measured. The partial image or signal measured by the m-th binary pattern is denoted S.sub.m ∈.sup.1×ΛT. The binary patterns P.sub.1, . . . , P.sub.M associated with the M measurements providing partial images S.sub.1, . . . , S.sub.M of the sample form a matrix P.sub.tot=(P.sub.1; . . . ; P.sub.M) with P.sub.tot ∈.sup.M×N×N. Writing S.sub.tot=(S.sub.1; . . . ; S.sub.M), S.sub.tot ∈ .sup.M×ΛT then:
S.sub.tot=P.sub.tot.Math.I.sub.tot.  [Math. 1]

(29) In the embodiment, the processor 12 is configured to carry out this post-processing step and to combine the partial images of the sample so as to construct the 4D image cube I.sub.tot forming a time- and wavelength-resolved image of the sample, i.e. to invert the equation [Math. 1].

(30) A plurality of image-reconstruction methods suitable for SP imaging are known and may be applied to the invention. See for example: Duarte, Marco F., et al. “Single-pixel imaging via compressive sampling.” IEEE signal processing magazine 25.2 (2008): 83-91, or indeed Edgar, Matthew P., Graham M. Gibson, and Miles J. Padgett. “Principles and prospects for single-pixel imaging.” Nature Photonics (2018): 1.

(31) In the embodiment of FIG. 1, the binary patterns P.sub.1, . . . , P.sub.M associated with the M measurements providing partial images S.sub.1, . . . , S.sub.M of the sample, which images are combined to form the complete time- and wavelength-resolved image I.sub.tot=(I.sub.1,1, . . . , I.sub.λ,t, . . . , I.sub.Λ,T), are advantageously chosen so as to form a Hadamard, Fourier or wavelet basis. The processor is then configured so that the sample is scanned by the binary patterns using a predefined basis or an adaptive basis. The advantage of performing a scan using an adaptive basis is that it decreases the acquisition time required by selecting only the binary patterns that contain the most spatial information. In the case of a scan using an adaptive basis, the processor is configured to extract spatial information from the sample during the acquisition of partial images in order to determine which binary patterns are most suitable for representing the sample. The complete image is then reconstructed using the processor via a suitable inverse transformation. These methods possess the advantage of the image being rapidly reconstructed algorithmically using a suitable inverse transform, but in general require a high number of patterns and therefore a long acquisition time.

(32) In another embodiment, the processor may be configured so that the binary patterns associated with the partial images are pseudo-random, the processor being able to reconstruct the complete time-resolved image by implementing a compressed-acquisition algorithm (l.sub.1-minimization) such as matching pursuit or basis pursuit. These methods have the advantage of allowing reconstruction with a lower number of patterns than with scanning methods based on a predefined basis or adaptive basis, and therefore require a shorter acquisition time. However, they require greater computing power and substantial computing time.

(33) It is also possible to track a given point of interest of the sample with adaptive pseudo-random binary patterns. This may be useful in the characterization of photovoltaic cells for example, in which the source of photoluminescent (PL) illumination is point-like and may be considered to be radially symmetric. The use of a radial basis of binary patterns thus allows the number of acquisitions performed to be decreased. The centre of a PL source in the sample is determined by observing the variation in the intensity acquired by the streak camera, by integrating an image with respect to wavelength and time and by activating in turn different regions of the DMD by using a dichotomy system.

(34) Thus, when the sample has particular properties (centro-symmetric, a low total variation norm, etc.) it is necessary to perform about M=N×log (N) measurements with distinct binary patterns P.sub.1, . . . , P.sub.M with P.sub.i ∈.sup.N×N to reconstruct an image of N×N pixels, the pixel size being set by the DMD. Otherwise, it is in general necessary to perform N×N measurements.

(35) FIG. 3 illustrates an imaging system 30 according to a second embodiment of the invention similar to that of FIG. 1, with the exception that it comprises a beamsplitter 14 arranged so as to split the radiation coming from the sample and the radiation coming from the radiation source. The beamsplitter may be a plate beamsplitter.

(36) The imaging system 30 in addition comprises a third optical system 13 suitable for focusing the radiation coming from the radiation source onto the sample, the sample being placed in the focal plane of the third optical system. In order to easily control the position of the sample in the focal plane of the third optical system, the sample is fastened to a linear stage suitable for moving the sample along the optical axis of the third optical system (which is represented by the direction z in FIG. 3). The third optical system is an objective optical system similar to those used in confocal microscopy for example. In the embodiment of FIG. 3, it possesses a high numerical aperture. It may be a single lens, for example a convergent biconvex lens, or a convergent complex set of a plurality of lenses.

(37) Comparatively to the imaging system of the embodiment of FIG. 1, the imaging system of the embodiment of FIG. 3 allows a larger amount of the light flux emitted/transmitted/reflected by the sample to be collected.

(38) FIG. 4A illustrates an image of a PL signal originating from a point source acquired with the imaging system of the embodiment of FIG. 3 and integrated with respect to wavelength and time. By determining the centre of the point source and by scanning around this point, a 4D image of the point source may be reconstructed. To make viewing easier, FIG. 4A shows the reconstructed image of the point source integrated with respect to wavelength and time. FIG. 4B shows the variation in the PL signal as a function of time and as a function of wavelength for the brightest pixel in FIG. 4A. This pixel corresponds to the centre of the point source. By cutting horizontal sections through FIG. 4B for various time values, it is possible to observe the variation in the spectrum over time of the signal of the central pixel.

(39) In another embodiment the SLM is configured to form a non-binary reflection or transmission intensity mask. A non-binary mask allows the step of reconstructing the image of the sample to be accelerated. Alternatively, the mask may be a phase mask.