DEVICE FOR OBSERVING A SAMPLE AND METHOD FOR OBSERVING A SAMPLE
20200033580 ยท 2020-01-30
Assignee
Inventors
- Cedric Allier (Grenoble, FR)
- Thomas Bordy (Grenoble, FR)
- Olivier Cioni (Grenoble, FR)
- Lionel Herve (Corenc, FR)
- Sophie Morel (Douai, FR)
Cpc classification
G02B21/362
PHYSICS
G03H2001/005
PHYSICS
G03H1/0443
PHYSICS
G03H1/0866
PHYSICS
G02B21/367
PHYSICS
G02B21/361
PHYSICS
International classification
Abstract
There is provided a device allowing a sample to be observed in a first mode, by lensless imaging using a first sensor. The first mode allows a first image to be obtained, on the basis of which a region of interest of the sample may be identified. The device then allows, via a relative movement, the region of interest to be analyzed using a more precise second mode and in particular using an optical system coupled to a second sensor.
Claims
1.-18. (canceled)
19. A device for observing a sample, comprising: a holder, configured to hold the sample; a first light source, configured to emit an incident light wave that propagates to the sample; a first image sensor, configured to acquire a first image of the sample illuminated by the incident light wave, the holder being further configured to hold the sample between the first light source and the first image sensor such that no magnifying optics are placed between the sample and the first image sensor, the first image sensor being exposed to an exposure light wave, the first image defining a first field of view of the sample; a second image sensor, optically coupled to an optical system having a magnification higher than 1, the second image sensor being configured to acquire a second image of the sample, held on the holder, in a second field of view that is smaller than the first field of view; a mechanism configured to move the sample relative to the first image sensor and to the optical system, so as to alternate between: a first mode, in which the sample is placed in the field of view of the first image sensor, so as to acquire the first image, and a second mode, in which the sample is placed in the field of view of the second image sensor, so as to acquire the second image.
20. The device of claim 19, further comprising a second light source configured to illuminate the sample during the acquisition of the second image of the sample.
21. The device of claim 19, further comprising: a selector, configured to select a region of interest in the first image; and a processor, configured to determine a relative position of the sample with respect to the optical system, in which position the selected region of interest lies in the second field of view, wherein the mechanism is further configured to automatically position the sample relative to the optical system in the relative position determined by the processor.
22. The device of claim 19, further comprising a processor configured to apply a numerical propagation operator to the first image, so as to: calculate a complex expression for the exposure light wave on a reconstruction surface lying facing the first image sensor, and form a reconstructed image from a modulus and/or a phase of the complex expression, wherein, in the second mode, a position of the sample with respect to the optical system is defined depending on a region of interest selected in the reconstructed image.
23. The device of claim 21, wherein the first image sensor lies in a detection plane, and the device further comprises a processor configured to apply a digital focus to the first image so as to estimate a distance between the sample and the detection plane in the region of interest, such that the relative position of the sample with respect to the optical system is determined depending on the distance thus estimated.
24. The device of claim 19, wherein the first image sensor and the second image sensor are fixed, and wherein the moving mechanism is configured to move the sample: to face the first image sensor in the first mode, and to face the optical system in the second mode.
25. The device of claim 19, wherein the sample is fixed and the mechanism is further configured to move the first image sensor to face the sample, in the first mode, and/or to move the optical system to face the sample, in the second mode.
26. A method for observing a sample, comprising: a) illuminating the sample using a first light source; b) acquiring a first image of the sample using a first image sensor, the first image sensor being exposed to an exposure light wave, the sample being held between the first light source and the first image sensor, no magnifying optics being placed between the first image sensor and the sample; c) selecting a region of interest of the sample in the first image; d) moving the sample relative to the first image sensor and to an optical system having a magnification higher than 1, the optical system being optically coupled to a second image sensor, the movement being carried out automatically by a moving mechanism, such that the region of interest of the sample selected in step c) is located in a field of view of the second image sensor; and e) illuminating the sample using a second light source and acquiring a second image of the region of interest of the sample, using the second image sensor.
27. The method of claim 26, wherein the relative movement of the sample results in automatically passing between: a first mode, in which the sample is placed in a field of view of the first image sensor, so as to acquire the first image, and a second mode, in which the sample is placed in the field of view of the second image sensor, so as to acquire the second image.
28. The method of claim 26, wherein, in step c), the region of interest is selected, in the first image, using a manual selector or via an analysis of the first image, the analysis being based on a predefined selection criterion and being implemented by a processor.
29. The method of claim 26, wherein the first image sensor lies in a detection plane, and step c) further comprises: ci) applying a propagation operator to the first image, so as to calculate a complex expression for the exposure light wave on a reconstruction surface lying facing the detection plane, cii) forming a reconstructed image, based on a modulus and/or a phase of the complex expression calculated in ci), and ciii) selecting the region of interest in the reconstructed image.
30. The method of claim 29, wherein, in step ci), the propagation operator is applied to an intermediate image, obtained from the first image and comprising a number of pixels lower than a number of pixels of the first image, a field of view of the intermediate image being similar to a field of view of the first image, and wherein step c) further comprises: applying a propagation operator to the first image, in the region of interest selected in step ciii), so as to calculate a complex expression for the exposure light wave on a reconstruction surface lying facing the detection plane, thereby defining a complex image of interest, from the calculated complex image of interest, forming a reconstructed image of interest, based on the modulus and/or the phase of the complex expression, and displaying the reconstructed image of interest.
31. The method of claim 29, wherein, in step ciii), the region of interest is selected, in the reconstructed image, using a manual selector or via an analysis of the reconstructed image, the analysis being based on a predefined selection criterion and being implemented by a processor.
32. The method of claim 29, wherein, in step ci), the reconstruction surface is a sample plane in which the sample lies.
33. The method of claim 29, further comprising, prior to step c), calibrating a position of the sample with respect to the detection plane, the calibrating comprising: selecting a plurality of calibration points in the first image acquired; defining an elementary calibration region about each selected calibration point; implementing, with a processor, a digital focusing algorithm, so as to estimate a calibration distance, between the sample and the detection plane, for each elementary calibration region; and partitioning the first image acquired into various elementary images, and associating, with each elementary image, a distance between the sample and the detection plane, depending on the calibration distance estimated for each elementary calibration region, the partitioning comprising: applying a propagation operator to each elementary image, depending on the distance associated with the elementary image, so as to calculate, for each elementary image, a complex expression for the exposure light wave in an elementary reconstruction plane, and forming an elementary reconstructed image from the modulus or the phase of the complex expression calculated in step ci), in each elementary reconstruction plane, the reconstructed image being obtained by concatenation of each elementary reconstructed image.
34. The method of claim 33, wherein the digital focusing algorithm comprises: applying a numerical propagation operator to each elementary calibration region so as to obtain, for said each elementary calibration region, a calibration image, which is a complex image of the exposure light wave, the numerical propagation operation being applied in various reconstruction planes that are respectively spaced apart by various distances from the detection plane, for each elementary calibration region, calculating, for each reconstruction plane, an indicator of clearness of an obtained reconstructed image from the phase and/or the modulus of the complex calibration expression calculated in the reconstruction plane, and determining a calibration distance between the sample and the detection plane for said each elementary calibration region, depending on the calculated clearness indicators.
35. The method of claim 26, wherein step d) further comprises: implementing, with a processor, a digital focusing algorithm, so as to estimate a distance between the sample and a detection plane in which the image sensor lies, in the region of interest selected in step c); and moving the sample relative to the optical system, while taking into account the distance thus estimated, such that the sample is placed in a focal plane of the optical system.
36. The method of claim 35, wherein the digital focusing algorithm comprises: applying a numerical propagation operator to the first image, so as to calculate a complex expression for the exposure light wave in a plurality of reconstruction planes respectively located at various reconstruction distances from the detection plane, obtaining a reconstruction image at each reconstruction distance, from a phase or an amplitude of the complex expression determined in each reconstruction plane, determining an indicator of clearness of each reconstruction image, and determining the distance between the sample and the detection plane, in the region of interest, depending on the clearness indicator determined for each reconstruction image.
Description
FIGURES
[0091]
[0092]
[0093]
[0094]
[0095]
[0096]
[0097]
[0098]
[0099]
DESCRIPTION OF PARTICULAR EMBODIMENTS
[0100]
[0101] The sample 10 is placed on a sample holder 10s. The sample may be a medium, for example a liquid medium, in which particles are submerged, or on the surface of which particles are found. It may for example be a bodily or biological liquid. By particle, what is for example meant is objects the diameter of which is smaller than 1 mm, or even than 100 m, or objects inscribable in a circle of such a diameter. The particles may be cells, microorganisms (for example bacteria or spores) or microbeads. The medium may also be agar, which is propitious for the development of bacterial colonies, or a solid. The sample 10 may also be a tissue slide intended for histological analysis, or a pathology slide comprising a small thickness of tissue deposited on a transparent slide. By small thickness, what is meant is a thickness that is preferably smaller than 100 m, and preferably smaller than 10 m, and typically a view microns. Such a tissue slide may be obtained using known preparation methods, a tissue sample being obtained via a biopsy or smear then prepared so as to take the form of a small thickness deposited on a transparent slide, the latter serving as holder. Such methods are well known in the histology field. They for example comprise slicing frozen tissue, or embedding a tissue sample into a paraffin matrix. The tissue may be stained, for example using a staining agent such as HES (hematoxylin eosin saffron).
[0102] Generally, the thickness of the sample 10, along the propagation axis Z, is preferably comprised between 20 m and 500 m. The sample lies in at least one plane P.sub.10, called the sample plane, preferably perpendicular to the propagation axis Z. It is held on the holder 10s at a distance d from a first image sensor 16.
[0103] Preferably, the optical path traced by the first light wave 12 before reaching the sample 10 is longer than 5 cm. Advantageously, the light source, seen by the sample, may be considered to be point-like. This means that its diameter (or its diagonal) is preferably smaller than one tenth, better still one hundredth of the optical path between the sample and the light source. The light source 11 may, for example, be a light-emitting diode or a laser source, for example a laser source. It may be associated with a diaphragm 18, or a spatial filter. The aperture of the diaphragm 18 is typically comprised between 5 m and 1 mm, and preferably between 50 m and 500 m. In this example, the diaphragm is that sold by Thorlabs under the reference P150S and its diameter is 150 m. The diaphragm may be replaced by an optical fiber, a first end of which is placed facing the first light source 11 and a second end of which is placed facing the sample 10.
[0104] The device may comprise a diffuser 17, placed between the light source 11 and the diaphragm 18. Use of such a diffuser allows constraints on the centeredness of the light source 11 with respect to the aperture of the diaphragm 18 to be relaxed. The function of such a diffuser is to distribute the light beam, produced by the light source, into a cone of angle . Preferably, the scattering angle varies between 10 and 80. The presence of such a diffuser makes it possible to make the device more tolerant to an off-centeredness of the light source with respect to the diaphragm. The diaphragm is not essential, in particular when the light source is sufficiently point-like, and notably when it is a laser source.
[0105] Preferably, the emission spectral band of the incident light wave 12 has a width smaller than 100 nm. By spectral bandwidth, what is meant is a fullwidth at half maximum of said spectral band.
[0106] The device, such as shown in
[0107] The first image sensor 16 is able to form a first image I.sub.1 in a detection plane P.sub.0. In the shown example, it is a question of an image sensor comprising a matrix array of pixels, of CCD or CMOS type, and the area of which is generally larger than 10 mm.sup.2. The so-called detection area of the matrix array of pixels depends on the number of pixels and on their size. It is generally comprised between 10 mm.sup.2 and 50 mm.sup.2. The detection plane P.sub.0 preferably lies perpendicular to the propagation axis Z of the incident light wave 12. The distance d between the sample 10 and the matrix array of pixels of the image sensor 16 is preferably comprised between 50 m and 2 cm, and preferably comprised between 100 m and 2 mm.
[0108] The absence of magnifying optics between the first image sensor 16 and the sample 10 will be noted. This does not prevent focusing micro-lenses optionally being present level with each pixel of the first image sensor 16, the function of these micro-lenses not being to magnifying the image acquired by the first image sensor.
[0109] Because of the proximity between the first image sensor 16 and the sample 10, the first image I.sub.1 is acquired in a first field of observation .sub.1 that is slightly smaller than the area of the image sensor, i.e. typically comprised between 10 mm.sup.2 and 50 mm.sup.2. This field of observation is large in comparison to the field of observation of a high-magnification microscope objectives, for example an objective of magnification higher than 10. Thus, the first image I.sub.1 allows exploitable information on the sample to be obtained in a first large field of observation .sub.1. An important element of the invention is to take advantage of this large field of observation to select a region of interest ROI of the sample in the first image I.sub.1, then to analyse the selected region of interest with a conventional microscope objective 25 having a magnification higher than 1, or even higher than 10.
[0110] Under the effect of the first incident light wave 12, the sample may generate a diffracted wave 13 that is liable to interfere, in the detection plane P.sub.0, with that portion of the incident first light wave 12 which is transmitted by the sample. Thus, the light wave 14 (called the exposure light wave) transmitted by the sample 10 and to which the first image sensor 16 is exposed, may comprise: [0111] a component 13 resulting from diffraction of the first incident light wave 12 by the sample; [0112] a component 12, transmitted by the sample, and resulting from absorption of the first incident light wave 12 by the sample.
[0113] These components interfere in the detection plane. Thus, the first image I.sub.1 acquired by the image sensor comprises interference patterns (or diffraction patterns), each interference pattern being generated by the sample. For example, when the sample contains particles, it is possible to associate, with each particle, one interference pattern. The first image I.sub.1 then allows the particles to be located and counted, or even a particle to be identified on the basis of the morphology of the diffraction pattern that is associated therewith, as described for example in WO2008090330. It is then possible to select a region of interest in the first image I.sub.1, then to carry out a more thorough analysis of the region of interest using the second mode described below, with reference to
[0114] A processor 40, for example a microprocessor, is configured to process each image I.sub.1 acquired by the image sensor 16, and to for example allow the region of interest ROI to be selected and any of the holographic reconstruction or image-processing operations described in this patent application to be carried out. In particular, the processor is a microprocessor connected to a programmable memory 42 in which a sequence of instructions for carrying out the image-processing operations and calculations described in this description is stored. The processor may be coupled to a display 44 allowing images acquired by the image sensor 16 or calculated by the processor 40 to be displayed.
[0115] As described above, the first image I.sub.1 may suffice to locate a region of interest ROI, which it seems judicious to analyse more thoroughly. This is for example the case when the sample 10 contains particles, the latter possibly being subjected to a more detailed analysis implementing the second mode described below.
[0116] The device shown in
[0117] The sample may be moved relative to the first image sensor 16 into the optical system 25, so as to be placed: [0118] either in a first large-field observation mode using the first image sensor 16, as described above, this first mode being shown in
[0120] Preferably, the sample 10 is kept immobile, whereas the image sensor 16 and the optical system 25 are moved with respect to the sample to switch between the two observation modes.
[0121] The optical system 25 is in particular a microscope objective, the magnification of which is preferably higher than or equal to 5 or even than 10. The device may comprise a plurality of optical systems 25, 25 of different magnifications. In the second mode, the sample is illuminated by the second light source 21, for example a white light source. The second light source 21 is not necessarily different from the first light source 11. The second image I.sub.2, which is acquired by the second image sensor, through the objective 25, allows a detailed representation of the region of interest ROI identified in the first image I.sub.1 to be obtained.
[0122] Preferably, the relative movement of the first image sensor 16 and of the optical system 25 is automatically calculated by the processor 40, depending on the region of interest ROI of the sample selected by an operator in the first image I.sub.1. For example, the first image I.sub.1 may be displayed on the display 44. The operator then selects the region of interest ROI using a selector 41, the latter notably being a peripheral accessory of the processor 40, such as a computer mouse or a keyboard. The selection of the region of interest results in the stage 30 being actuated to place the sample in the second analysis mode, i.e. facing the objective 25. The selector allows a manual selection of the region of interest ROI of the sample, but an automatic selection may be carried out by a processor, for example the processor 40, as described below with reference to
[0123] It will be understood that the combination of these two modes allows time to be saved during the analysis of the sample, by allowing fine analysis, carried out with the objective 25, to be concentrated on a limited number of regions of interest, the latter being determined using the large-field image I.sub.1 acquired in the first mode. A needless and time-consuming scan of the entire area of a sample using a microscope objective is thus avoided.
[0124]
[0125] The first image sensor 16 may comprise a Bayer filter, such that each pixel is sensitive to one spectral band chosen from the blue, the red or the green. Thus, when the sample 10 is exposed to such a first light source 11, the first image sensor 16 acquires a first image I.sub.1 that is able to be decomposed into: [0126] a first image I.sub.1(.sub.1) in the first emission spectral band .sub.1 of the first light-emitting diode 11.sub.1, this image being formed from pixels exposed to a wavelength transmitted by the blue filter of the Bayer filter; [0127] a first image I.sub.1(.sub.2) in the second emission spectral band .sub.2 of the second light-emitting diode 11.sub.2, this image being formed from pixels exposed to a wavelength transmitted by the green filter of the Bayer filter; [0128] a first image I.sub.1(.sub.3) in the third emission spectral band .sub.3 of the third light-emitting diode 11.sub.3, this image being formed from pixels exposed to a wavelength transmitted by the red filter of the Bayer filter.
[0129] Generally, in this embodiment, the first image sensor 16 allows first images I.sub.1(.sub.i) of the sample 10 to be acquired in various spectral bands .sub.i. Each first image I.sub.1(.sub.i) is representative of a light wave 141 to which the first image sensor 16 is exposed in each spectral band .sub.i. Preferably, there is no overlap between the various spectral bands; a negligible overlap, for example concerning less than 25% and better still less than 10% of the emitted light intensity, is however envisionable.
[0130] Other configurations are possible, for example it is possible to use a monochrome image sensor that acquires first images I.sub.1(.sub.i) of the sample when the latter is successively illuminated, by an incident wave 121, in various spectral bands .sub.i. Each incident wave 121 may be emitted by a light source 11i that emits one of said spectral bands, or by a white light source that is filtered by an optical filter the passband of which corresponds to said spectral band .sub.i.
[0131]
[0132]
[0133]
[0134] Step 100: illuminating the sample 10 using the first light source 11, the sample being placed facing the first image sensor 16.
[0135] Step 110: acquiring a first image I.sub.1 of the sample 10 using the first image sensor 16.
[0136] Step 130: selecting, manually or automatically, a region of interest ROI in the first image I.sub.1.
[0137] Step 140: moving the sample 10 relative to the objective 25, so as to place the region of interest ROI of the sample 10 facing this objective.
[0138] Step 150: Illuminating the sample 10 using the second light source 21.
[0139] Step 160: acquiring a second image I.sub.2 representing the region of interest ROI using the second images sensor 26, through the objective 25.
[0140] Step 170: exiting from the algorithm or moving the sample relative to the first image sensor 16 so as to place the sample facing the first image sensor 16.
[0141] When the number of diffracting elements of the sample increases, the first image I.sub.1 acquired by the first image sensor 16 may not allow the region of interest to be reliably selected. This may be the case when the sample contains particles and the concentration of the particles is high. This is also the case when the sample is a thin tissue slide such as described above. In this case, the region of interest is not selected in the first image I.sub.1, but in an image I.sub.z that is said to be reconstructed from the first image. Such an embodiment is shown in
[0142] Step 120: determining an image I.sub.z, called the reconstructed image, that is representative of the sample. This image is obtained by applying a holographic propagation operator h, such as described below, to the first image I.sub.1, so as to calculate a complex image A.sub.z representing the complex amplitude of the exposure light wave 14 on a surface lying substantially parallel to the detection plane P.sub.0, at a distance z, called the reconstruction distance, from the latter. The reconstructed image I.sub.z is obtained from the modulus and/or phase of the complex amplitude A.sub.z thus calculated. By substantially parallel, what is meant is parallel, an angular tolerance of plus or minus 10 or 20 being acceptable.
[0143] Step 130: selecting, manually or automatically, a region of interest ROI in the reconstructed image I.sub.z.
[0144] In this description, the term reconstructed image designates an image I.sub.z formed from the modulus or phase of the exposure light wave 14 on a reconstruction surface parallel to the detection plane. The reconstructed image is determined from the modulus or phase of the complex image A.sub.z. This surface may be a plane P.sub.z, located at a reconstruction distance z from the detection plane P.sub.0. It may also be a question of a plurality of planes that lie parallel to the detection plane, and that are located at various distances z.sub.w from the detection plane, so as to take into account an inclination of the plane P.sub.10 in which the sample 10 lies with respect to the detection plane P.sub.0.
[0145] The reconstructed image I.sub.z is obtained by applying a holographic propagation operator h to the first image I.sub.1 acquired by the first image sensor 16. Such a method, designated by the term holographic reconstruction, in particular allows an image of the modulus or phase of the exposure light wave 14 in a reconstruction plane P.sub.z parallel to the detection plane P.sub.0, and in particular in the plane P.sub.10 in which the sample lies, to be reconstructed. To do this, the first image I.sub.1 is convoluted with a propagation operator h. It is then possible to reconstruct a complex expression A for the light wave 14 at every point of spatial coordinates (x, y, z), and in particular in a reconstruction plane Pz located at a reconstruction distance Izi from the image sensor 16, called the reconstruction distance, this reconstruction plane preferably being the sample plane P.sub.10, with: A(x,y,z)=I.sub.1(x,y,z)*h, * designating the convolution operator. In the rest of this description, the coordinates (x,y) designate a radial position in a plane perpendicular to the propagation axis Z. The coordinate z designates a coordinate along the propagation axis Z. The complex expression A is a complex quantity the argument and the modulus of which are respectively representative of the phase and intensity of the exposure light wave 14. The convolution of the first image I.sub.1 with the propagation operator h allows a complex image A.sub.z to be obtained that represents a spatial distribution of the complex expression A in a reconstruction plane P.sub.z lying at a coordinate z from the detection plane P.sub.0. In this example, the equation of the detection plane P.sub.0 is z=0. The complex image A.sub.z corresponds to a complex image of the exposure wave 14 in the reconstruction plane P.sub.z. It also represents a two-dimensional spatial distribution of the optical properties of the exposure wave 14.
[0146] The function of the propagation operator h is to describe the propagation of light between the image sensor 16 and a point of coordinates (x,y,z), located at a distance |z| from the first image sensor. It is then possible to determine the modulus M(x,y,z) and/or phase (x,y,z) of the light wave 14, at this distance |z|, which is called the reconstruction distance, with:
M(x,y,z)=abs[A(x,y,z)];
(p(x,y,z)=arg[A(x,y,z)];
[0147] The operators abs and arg designate the modulus and argument, respectively.
[0148] The propagation operator is for example the Fresnel-Helmholtz function, such that:
[0149] In other words, the complex expression for the light wave 14, at every point of spatial coordinates (x, y, z), is such that A(x, y, z)=M(x, y, z)e.sup.j(x,y,z). It is possible to form images, called reconstructed images, M.sub.z and .sub.z respectively representing the modulus and phase of the complex expression A in a plane P.sub.z located at a distance |z| from the detection plane P.sub.0, with M.sub.z=mod (A.sub.z) and .sub.z=arg(A.sub.z).
[0150] However, simply applying the propagation operator h to the first image generally leads to a complex image A.sub.z that is affected by substantial reconstruction noise. This is due to the fact that the first image I.sub.1, acquired by the image sensor 16, contains no information on the phase of the exposure light wave 14. It is possible to implement interactive algorithms, so as to gradually estimate the phase of the exposure light wave 14 in the detection plane P.sub.0, this then allowing a more exact complex image A.sub.z of the light wave 14 in a reconstruction plane P.sub.z to be obtained.
[0151] The inventors have developed an iterative algorithm, which is described in the publication S. N. A. Morel, A. Delon, P. Blandin, T. Bordy, O. Cioni, L. Herv, C. Fromentin, J. Dinten, and C. Allier, Wide-Field Lensfree Imaging of Tissue Slides, in Advanced Microscopy Techniques IV; and Neurophotonics II, E. Beaurepaire, P. So, F. Pavone, and E. Hillman, eds., Vol. 9536 of SPIE Proceedings (Optical Society of America, 2015) and in patent application FR1554811 filed 28 May 2015, and more precisely in steps 100 to 500 described in this patent application. In this algorithm, the sample is successively or simultaneously illuminated in various spectral bands .sub.i, using a light source 11 such as illustrated in
[0152] Step 121: initialization, from the first image I.sub.1(.sub.i) acquired by the image sensor 16 in each spectral band .sub.i. This corresponds to step 100 described in the aforementioned patent application FR1554811. This initialization allows an initial complex image A.sub.0(.sub.i) representative of the exposure light wave 14 in the detection plane P.sub.0, in each spectral band .sub.i, to be obtained.
[0153] Step 122: propagation of each first image I.sub.1i(.sub.i) to a propagation plane P.sub.z located at a reconstruction distance z from the detection plane P.sub.0. This corresponds to step 200 described in patent application FR1554811. A complex image A.sub.z(.sub.i) representing the exposure wave 14, in the reconstruction plane P.sub.z, and in each spectral band .sub.i, is then obtained. The reconstruction plane is preferably the plane P.sub.10 of the sample.
[0154] Step 123: combination of each complex image A.sub.z(.sub.i) so as to obtain a weighting function in the reconstruction plane P. This corresponds to step 300 described in patent application FR1554811. The weighting function may be a weighted sum of each complex image A.sub.z(.sub.i).
[0155] Step 124: propagation of the weighting function to the detection plane P.sub.0, by application of the propagation operator h to the weighting function. This corresponds to step 400 described in patent application FR1554811.
[0156] Step 125: update of the complex image A.sub.0(.sub.i) representative of the exposure light wave 14 in the detection plane P.sub.0, in each spectral band .sub.i. This update is carried out on the basis of the weighting function propagated to the detection plane P.sub.0 in the preceding step. The phase of the complex image A.sub.0(.sub.i), in each spectral band .sub.i, is updated by being replaced by the phase of the weighting function propagated to the detection plane P.sub.0. This corresponds to step 500 described in patent application FR1554811.
[0157] Each complex image thus updated in the detection plane is then propagated to the reconstruction plane, in step 122. Steps 122 to 125 are implemented iteratively until the phase of the complex image in the detection plane P.sub.0 or in the sample plane P.sub.10 is considered to be correctly estimated.
[0158] From the complex image A.sub.z(.sub.i) obtained, in the reconstruction plane P.sub.z, in each spectral band .sub.i, it is possible to obtain a reconstructed image I.sub.z, for example: [0159] by averaging a modulus or phase of the complex image A.sub.z(.sub.i) in each spectral band .sub.i: the reconstructed image I.sub.z is then a grayscale image; [0160] by combining the modulus and/or phase of the complex image A.sub.z(.sub.i) in each spectral band .sub.i, this allowing a colour reconstructed image I.sub.z representing the modulus or phase to be obtained.
[0161] Other algorithms are usable to obtain a complex image representing the exposure wave 14, on a reconstruction surface P facing the first image sensor 16. Such algorithms are for example described in patent application FR1652500, which was filed 23 Mar. 2016. Such algorithms may be implemented on the basis of a first light source 11 that emits a first light wave 12 in a single spectral band, without departing from the scope of the invention.
[0162]
[0163] The experimental conditions of this trial were the following: [0164] light source 11: CREE Xlamp MCE; [0165] first image sensor: monochrome IDS MT9J003, 3840*2748 pixels, each pixel having a side length of 1.67 mm, i.e. a detection area of 6.4*4.6 mm (29.4 mm.sup.2); [0166] distance d between the sample 10 and the first image sensor 16: 1.5 mm; [0167] spatial filter: 150 m aperture; [0168] objective 25: Olympusmagnification 20; [0169] second image sensor 26: Mightex SE-C050-Ucolor sensor comprising 2560*1920 pixels.
[0170] According to one embodiment, the region of interest ROI selected in the first image I.sub.1 is the subject of a reconstruction, so as to obtain a reconstructed image that is representative of the light wave 14, only from the region of interest.
[0171]
[0172] According to one variant, a reconstructed image I.sub.z is obtained from a first image referred to as an intermediate image I.sub.1, which is obtained by applying binning to the first image I.sub.1. Carrying out a reconstruction on an intermediate image I.sub.1 allows calculation time to be decreased, and a reconstructed image I.sub.z to be obtained more rapidly. In this reconstructed image, a region of interest ROI is selected. It is then possible to obtain a reconstructed image of interest I.sub.z,ROI, limited to the region of interest selected beforehand. This reconstructed image of interest is obtained by applying holographic reconstruction algorithms, such as described or mentioned above, not to the entirety of the first image I.sub.1, but only to the portion of the first image I.sub.1,ROI corresponding to the selected region of interest.
[0173]
[0174] Step 120 corresponds to the reconstruction of the complex expression for the exposure light wave 14, in a reconstruction plane P.sub.z, not from the first image I.sub.1, but from the intermediate image I.sub.1. Thus a complex image A.sub.z is obtained from which is extracted the reconstructed image I.sub.z, in the reconstruction plane P.sub.z from the modulus and/or phase of the complex image A.sub.z.
[0175] Step 130 is the selection of a region of interest ROI, in the reconstructed image I.sub.z.
[0176] Step 132 corresponds to selection of a portion I.sub.1,ROI of the first image corresponding to the region of interest ROI selected in the preceding step.
[0177] Step 134 is a reconstruction of the complex expression for the exposure light wave 14, in a reconstruction plane P.sub.z, from the portion I.sub.1,ROI of the first image I.sub.1 corresponding to the region of interest ROI selected in step 130. A reconstructed image of interest I.sub.z,ROI is thus obtained in the plane of reconstruction P.sub.z, from the modulus and/or phase of the complex image, which image is called the complex image of interest A.sub.z,ROI reconstructed in the region of interest.
[0178] Steps 140 to 170, such as described above, may then be implemented, so as to obtain an image of the region of interest selected in step 130 through the objective 25, using the second image sensor 26. However, in certain cases, the reconstructed image of interest may be sufficient to obtain a correct representation of the sample 10.
[0179] According to one embodiment, the region of interest ROI of the sample is not selected manually, using a selector 41 such as a mouse or keyboard, but automatically, via implementation of an image-processing algorithm, using a processor, for example the processor 40. The region of interest is selected depending on a predefined selection criterion. This criterion is for example a morphological criterion, in which case a region of interest is automatically detected in case of correspondence with the morphological criterion.
[0180]
[0181]
[0182] Moreover, as described with reference to the prior art, the first image I.sub.1, or the image I.sub.z reconstructed from the first image, may be used to track moving particles. Each particle may be associated with a region of interest, the latter being able to be periodically analyzed using the second mode.
[0183] According to one embodiment, illustrated in
[0184] Step 136: The first image I.sub.1, or a portion of the first image corresponding to the region of interest ROI, is propagated using a numerical propagation operator h, such as described above, to a plurality of reconstruction planes P.sub.z, each reconstruction plane lying at a different distance z from the detection plane P.sub.10. A complex expression is then obtained for the light wave 14 in these various reconstruction planes P.sub.z, thus forming the same number of complex images A.sub.z. A stack of complex images is thus obtained. From each complex image A.sub.z, a reconstruction image A.sub.z representing the modulus and/or phase of the complex expression in each reconstruction plane P.sub.z in question is established.
[0185] Step 137: A clearness indicator q.sub.z is assigned to each reconstruction image A.sub.z. The clearness indicator q.sub.z may be an indicator that quantifies a dispersion of each image A.sub.z, for example a standard deviation or a variance. It may also be defined by convoluting each reconstruction image with a Sobel operator. For example, it is possible to define a Sobel operator S.sub.x along the X-axis and a Sobel operator S.sub.y along the Y-axis. If (x, y) designate the pixels of the reconstruction image A.sub.z, the clearness indicator q.sub.z associated with each reconstruction image A.sub.z may then be such that
[0186] It is possible for:
[0187] Step 138: The variation in the clearness indicator q.sub.z as a function of depth allows the distance actually corresponding to the distance z.sub.ROI between the detection plane P.sub.0 and the sample, level with the region of interest ROI, to be identified. Depending on the clearness indicator employed, the distance z.sub.ROI generally corresponds to a minimum or a maximum, along the Z-axis, in the value of the clearness indicator. The distance z.sub.ROI is then used, in step 140, to automatically place the region of interest ROI of the sample in the focal plane of the objective 25. Manual adjustment of the focus of the objective 25 before the acquisition of the second image I.sub.2 is then avoided or limited.
[0188] The complex images A.sub.z forming the stack of complex images may be obtained from a complex image, called the initial image, A.sub.z=d formed with a pre-established distance d between the first image sensor 16 and the sample. This initial complex image is obtained by implementing an iterative reconstruction algorithm, as described above. It is then propagated to the various reconstruction planes P.sub.z via a simple convolution with a propagation operator h. For example, a complex image A.sub.z, in the reconstruction plane P.sub.z, is obtained simply via the operation: A.sub.z=A.sub.z=d*h.sub.d,z, where h.sub.d,z, is a propagation operator representing the propagation of light between the plane P.sub.z=d and the plane P.sub.z=z. The distance between two adjacent propagation planes is set depending on the precision with which it is desired to determine the distance z.sub.ROI between the detection plane P.sub.0 and the region of interest ROI of the sample.
[0189] One of the advantages of the first modes resides in the observed first field, the latter being extensive with respect to the field observed by a microscope objective. However, in such a field, the sample may not be rigorously parallel to the image sensor 16, as shown in
[0190] In order to solve this problem, it is proposed, when implementing step 120, to consider not a single propagation distance z, corresponding to the distance between the sample plane P.sub.10 and the detection plane P.sub.0, but to take into account a potential variation in the distance between the sample and the detection plane. To do this, prior to step 120, the image I.sub.1 acquired by the first image sensor 16 may be partitioned into various elementary portions I.sub.1,w, each elementary portion corresponding to a reconstruction distance z.sub.w representative of the distance between the sample and the first image sensor 16 level with the elementary portion. Step 120 is then implemented separately on each elementary portion I.sub.1,w, so as to reconstruct the exposure light wave 14 reaching the image sensor 10 level with said elementary portion, using the reconstruction distance z.sub.w associated with the elementary portion I.sub.1,w. In other words, a reconstruction distance z.sub.w corresponding to an estimation, level with said elementary portion, of the distance between the sample plane P.sub.10 and the detection plane P.sub.0, is attributed to each elementary portion I.sub.1,w.
[0191] However, the position of the sample with respect to the first image sensor 16 is generally not known beforehand. A calibration phase, comprising the steps shown in
[0192] Step 111: selection of a plurality of calibration points U.sub.n in the acquired first image I.sub.1. It is a question of determining at least two points, and preferably at least three points, in the first image I.sub.1 acquired, in step 100, by the first image sensor. These points are preferably as spaced apart from each other as possible. It may for example be a question of four points located at the corners of the first image I.sub.1, to which may optionally be added one or two points at the centre of the image. The number of calibration points must be sufficiently low for the calibration to be able to be carried out rapidly. It may be comprised between 2 and 10. In
[0193] Step 112: definition of a calibration region V, about each calibration point. Typically, a calibration region stretches at least 10 pixels about a calibration point U.sub.n. A calibration region may thus comprise 1010 pixels, or even 2020 pixels, or more, for example 5050 pixels.
[0194] Steps 113 to 115 aim to implement a digital focusing algorithm, such as described above, on each calibration region V.sub.n, so as to estimate a distance z.sub.n between the sample 10 and the detection plane P.sub.0 level with each calibration region V.sub.n.
[0195] Step 113: application of a propagation operator h to each elementary calibration region V.sub.n in order to obtain, for each thereof, a complex image A.sub.n,z, called the calibration image, of the exposure light wave 14.sub.n reaching the detection plane P.sub.0 level with said elementary calibration region V.sub.n, in various reconstruction planes P.sub.z respectively spaced by different distances z from the detection plane. The reconstruction planes P.sub.z are for example spaced apart by a few microns, for example 10 m, in a distance range liable to contain the sample 10. Thus a complex image A.sub.n,z of the light wave 14.sub.n to which the first image sensor 16 is exposed, level with the elementary calibration region V.sub.n, is obtained, in the various reconstruction planes P.sub.z.
[0196] Step 114: for each elementary calibration region V.sub.n, association, with each reconstruction plane P.sub.z, of an indicator q.sub.n,z of the dispersion of a so-called reconstruction image A.sub.n,z obtained from the phase and/or modulus of the complex expression of the exposure light wave 14, obtained from the complex calibration image A.sub.n,z reconstructed in step 113. The dispersion indicator q.sub.n,z may be a standard deviation of the modulus or phase of the complex expression reconstructed in each reconstruction plane.
[0197] Step 115: for each elementary calibration region V.sub.n, determination of a calibration distance z.sub.n, depending on the various clearness indicators q.sub.n,z. The calibration distance z.sub.n is the distance between the sample 10 and the detection plane P.sub.0, level with each elementary calibration region V.sub.n. Generally, this step amounts to applying a digital autofocus so as to determine the reconstruction plane P in which an image of the phase or of the modulus of the complex image A.sub.n,z is the clearest, this reconstruction plane then being considered to be located level with the sample 10. The clearness indicator q.sub.n,z may be one of those described with reference to
[0198] Step 116: partition of the acquired first image I.sub.1 into various elementary portions I.sub.1,w, and association, with each elementary portion, of a distance z.sub.w separating the sample 10 from the detection plane P.sub.0. Each of these distances is determined depending on the calibration distances z.sub.n established for each elementary calibration region V.sub.n, for example via a two-dimensional interpolation. The interpolation may be a linear interpolation. The number W of elementary portions I.sub.1,w may be determined beforehand, or depend on the difference between various calibration distances. The greater the inclination of the sample, the higher the number of elementary portions may be.
[0199] Step 120 then comprises reconstructing the complex amplitude of the exposure light wave 14 by independently processing each elementary portion I.sub.1,w of the first image I.sub.1. Thus, each elementary portion of the first image is propagated by the distance z.sub.w that was attributed thereto in step 116. As many reconstructions are carried out as there are elementary portions I.sub.1,w, these reconstructions being carried out simultaneously or successively, but independently of one another. Complex images A.sub.z,w of the exposure light wave 14 in various elementary reconstruction planes P.sub.z,w are then obtained, each elementary reconstruction plane lying at the detection plane-sample distance z.sub.w that was determined level with the elementary portion I.sub.1,w.
[0200] It is then possible to form a reconstructed image I.sub.z,w, called the elementary reconstructed image, from the modulus or the phase of the complex image A.sub.z,w, in the various elementary reconstruction planes P.sub.z,w.
[0201] This embodiment may be used for a simple observation of the sample, by virtue of the reconstructed image I.sub.z. Moreover, when a region of interest is selected then observed by a second image sensor, through an objective 25, the distances z.sub.w may be used to automatically position the selected region of interest in the focal plane of the objective 25. The fact that the position of the sample with respect to the detection plane is well known is then capitalized upon.
[0202]
[0203]
[0204] The invention is possibly applicable to the field of health, or to other fields in which it is necessary to obtain an overview of a sample, while allowing easy analysis of selected regions of interest of the latter. Thus, the invention is applicable to the field of environmental inspection, to food processing, to biotechnologies or to the monitoring of industrial processes.