OPTICALLY SUPER-RESOLVED INFRARED IMPULSE SPECTROSCOPY (OSIRIS)

Abstract

The present invention provides a system and method for spectroscopic imaging. In one embodiment, a system includes: a sample stage for holding a sample to be analyzed; a first light source for generating a pulse of infrared light; a second light source for generating a probing beam of infrared light; an optical system to direct the pulses of infrared light and the probing beam of infrared light at the sample, such that a temperature change is induced in an area of the sample. The duration of the pulse of infrared light is shorter than or equal to a cooling time constant of resolution scale inclusions within the sample such that the temperature change is independent of inclusion size. Light detectors are configured to detect light from the sample and digitization electronics convert data from the light detectors into signal data indicative of a chemical composition of the sample.

Claims

1. A method for spectroscopic imaging comprising: probing a sample to obtain reflective light data by: directing a pulse of infrared light from a first light source at an area of the sample to be measured such that a temperature change is induced in the area to be measured, wherein a duration of the pulse of infrared light is shorter than or equal to a cooling time constant of resolution scale inclusions within the sample such that the temperature change is independent of inclusion size; directing at least one probing beam of infrared light from a second light source at the sample incident with the area to be measured; and detecting a signal based on light reflected, transmitted, re-emitted, or combinations thereof, from the area to be measured while illuminated by the at least one probing beam of infrared light; and repeating the probing for different areas of the sample, thereby generating multidimensional signal data for the sample.

2. The method of claim 1, wherein the repeating the probing for different areas of the sample is performed at different wavelengths of the pulse of infrared light.

3. The method of claim 1, further comprising moving one of the second light source and the sample with respect to one another prior to each probing instance.

4. The method of claim 1, wherein a period between pulses of infrared light is longer than a thermal cooling time constant of the respective areas of the sample, thereby enabling inclusions within the sample to return to equilibrium with respect to the inclusions' surrounds after each pulse of infrared light.

5. The method of claim 1, wherein the at least one probing beam of infrared light has a wavelength shorter than a wavelength of the pulse of infrared light.

6. The method of claim 1, further comprising generating a multidimensional infrared spectroscopic image of the sample based on the multidimensional signal data.

7. The method of claim 1, further comprising determining at least one chemical characteristic of the sample based on the multidimensional signal data.

8. The method of claim 1, wherein the wavelength of the at least one probing beam of infrared light is between 400 nanometers (nm) and 700 nm.

9. The method of claim 1, wherein the duration of the pulse of infrared light is less than or equal to 20 nanoseconds (ns).

10. The method of claim 1, wherein the directing at least one probing beam of infrared light at the sample occurs within 300 nanoseconds of the pulse of light.

11. The method of claim 1, further comprising performing a probe signal trace during the pulse of infrared light; and calculating a relationship between the probe signal trace and a temperature of the area to be measured.

12. A system for spectroscopic imaging comprising: a sample stage configured to hold a sample to be analyzed; a first light source configured to generate a pulse of infrared light having a predetermined duration; a second light source configured to generate at least one probing beam of infrared light; an optical system configured to direct pulses of infrared light from the first light source, and direct the at least one probing beam of infrared light from the second light source, at the sample held by the sample stage, such that a temperature change is induced in an area of the sample to be measured, wherein the duration of the pulse of infrared light is shorter than or equal to a cooling time constant of resolution scale inclusions within the sample such that the temperature change is independent of inclusion size; one or more light detectors configured to detect light reflected, transmitted, or re-emitted from the sample while the sample is illuminated by the at least one probing beam of infrared light; and digitization electronics configured to convert data from the one of more light detectors into signal data indicative of a chemical composition of the sample.

13. The system of claim 12, wherein the duration of the pulse of infrared light is shorter than or equal to 20 ns.

14. The system of claim 12, a means for moving one of the at least one probing beam of infrared light and the sample stage with respect to one another.

15. The system of claim 14, wherein the means for moving the at least one probing beam of infrared light and the sample with respect to one another comprises one of: the optical system, which is configured to selectively reposition the at least one probing beam of infrared light with respect to the sample held by the sample stage; and the sample stage, wherein the sample stage is a sample motion stage.

16. The system of claim 12, wherein at least one of the light detectors is a photomultiplier light detector.

17. The system of claim 12, further comprising a signal processor.

18. The system of claim 12, wherein the digitization electronics have a temporal resolution to observe the resolution scale inclusions during the duration of the pulse of infrared light, which is greater than 20 ns.

19. The system of claim 12, wherein the optical system includes a beam splitter configured to split the at least one probing beam of infrared light into a plurality of probing beams.

20. The system of claim 12, wherein the one or more light detectors comprises a photomultiplier tube.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0030] FIG. 1 is a graph showing simulated temperature at the center of a moderately sized inclusion after successive IR pump pulses.

[0031] FIG. 2 is a graph showing relative heating simulated for infrared pulse lengths incident on spheres of various diameters in water when pulse energy is held constant.

[0032] FIG. 3 is a graph showing estimated relative pulse energy required to reach damage threshold and resulting relative SNR in the presence of shot-(random) noise. Detection that is not shot noise limited is worse for short pulse periods.

[0033] FIG. 4A is a cartoon representation depicting thermal waves emanating from a highly absorbing inclusionthe hotspot. The differing delay between probe locations 1 and 2 permits the triangulation of the hotspot.

[0034] FIG. 4B shows the results (right) using the Photothermal GPS method to reconstruct the ground truth (left) from the down-sampled image (middle).

[0035] FIG. 5 shows the Bayesian NMF model. The assumptions are incorporated into the model using maximum entropy distributions. For those unfamiliar with model notation, the plain English interpretation is included explicitly. Note that the tilde should be read as distributed as.

[0036] FIG. 6 shows chosen ground truth values for the mixing matrix (top left) and spectral matrix (top right). The ground truth spectrogram (bottom left) is compared to the synthetic (noised) spectrogram (bottom right).

[0037] FIG. 7 shows inferred results as a function of model order. Note that models of incorrect order are quite weak, at least for this synthetic data. The order=2 model (top) has a very high (bad) Bayesian information criterion (BIC), indicative of a poor fit (likelihood) of the data. The order=4 model (bottom) the extra component is primarily noise. The correct order=3 model (middle) has the best (lowest) BIC.

[0038] FIG. 8 shows higher dimensional inference demonstration. Analytes are localized within the sample in a Gaussian profile (top). Uncertainty in the inferred spectra is also shown (bottom).

[0039] FIG. 9 is a schematic of transmission mode OSIRIS.

[0040] FIG. 10 is a schematic of reflection mode OSIRIS.

DETAILED DESCRIPTION OF THE INVENTION

[0041] The present invention provides a method for spectroscopic imaging. According to this method, a sample is illuminated with a short infrared pulse that is shorter than or equal to the thermal cooling time constant of resolution scale inclusions within the sample to induce a temperature change independent of inclusion size or surface area (fractal dimension). Then one or more probe beams are directed to the sample such that it is incident within the area heated by the infrared pulse, and wherein the one or more probe beams has a shorter wavelength than the infrared pulse. Light that is reflected, transmitted, and/or re-emitted inelastically is measured for each probe beam to deduce the infrared induced heating. These steps (illuminating sample with an infrared pulse, directing probe beam to the sample, and measuring the light) are repeated multiple times with varying wavelengths of the infrared pulse and with the sample and the probe beams being moved relative to each other. This allows for the collection of a multidimensional infrared spectroscopic image of the sample, which characterizes the chemical makeup of the sample.

[0042] In a preferred embodiment, a pulse period of the infrared pulse is long compared to the thermal cooling time constant of the area heated by the infrared pulse, wherein the sample cools fully before the next infrared pulse. In another preferred embodiment the probe beam wavelength is in the range of 400 to 700 nm. In yet another preferred embodiment, the probe beam is modulated to only illuminate the sample at times within a few hundred nanoseconds of the infrared pulse to limit damage to the sample caused by the probe beam. In another preferred embodiment, a probe signal trace during the infrared pulse is used to exploit the constant rate of heating during the infrared pulse to calibrate the complicated relationship between the probe signal level and the temperature of the samplewhat we call probe thermometry.

[0043] The present invention also provides methods for analyzing the data. Spectrum data can be analyzed using spectral de-mixing, such as non-negative matrix factorization (NMF), to extract locations and spectra of distinct analytes within the sample, and a Bayesian approach can be employed to quantify the uncertainty of the data analysis. The measured light can be analyzed in the time domain, such as by photothermal GPS (deconvolution) or the inverse operation (convolution) on some proposed model of the sample, to infer information about the one or more probe beams and the surrounding area either to reduce the uncertainty of the collected image or upsample an undersampled image, and a Bayesian approach can be employed to quantify the uncertainty. A time domain analysis can be combined with spectral de-mixing to extract concentration maps. This could be applied in fully sampled images to reduce uncertainty, or in undersampled images to increase the data acquisition rate at the cost of increased uncertainty. Again, a Bayesian approach can be employed to quantify the uncertainty. Infrared pump optical probe images can be analyzed using non-negative matrix factorization (NMF) to extract locations and spectra of distinct analytes within the sample, and a Bayesian approach is used to quantify the uncertainty implicit in the data analysis.

[0044] This invention also provides a system for spectroscopic imaging. The system includes a pulsed infrared light source, an optical system for generating one or more probe beams, means for directing the infrared light source and one or more probe beams to a sample, a device for moving the one or more probe beams and the sample relative to each other, light detectors and filtering optics for the one or more probe beams, digitization electronics for the probe beams, and a computer with control software to collect, process, analyze and display relevant information. In a preferred embodiment, the means of directing the infrared light source and probe beams may be mirrors, lenses, or both. In another preferred embodiment, the pulsed infrared light is produced by nonlinear optics as in an optical parametric oscillator (OPO) or optical parameteric amplifier (OPA). In yet another preferred embodiment, multiple probe beams are generated by a diffractive optic, such as a spatial light modulator. In another preferred embodiment, the sample is moved relative to the one or more probe beams via a motion stage and/or the probe beams are moved relative to the sample via a scan mirror. In yet another preferred embodiment, the light detector is a photomultiplier tube or an array of photomultiplier tubes, an avalanche photodiode or an array of avalanche photodiodes, or a photodiode or an array of photodiodes. In a preferred embodiment, the digitization electronics have temporal resolution greater than or equal to the cooling time constant of resolution scale objects. In another preferred embodiment, the light detection is performed interferometrically by splitting the probe beams into two paths, one of which, the sample arm, going to the sample before being combined with the other arm, the reference arm, at the light detector, wherein either arm length is controlled by a piezo motion stage or by other means to cause destructive interference at the detector to reduce the shot noise associated with detecting the one or more probe signals.

Notional Design

[0045] A notional design is presented, supported by argument and simulation, of the ideal way to overcome the infrared diffraction limit with an optical probe. All simulations of thermal properties assume water as a medium, since aqueous samples are the primary application for OSIRIS. The thermal diffusivity a of water is similar to that of other organics. Inorganic materials generally have much higher thermal diffusivities that make the low-duty cycle of OSIRIS even more attractive. Finally, a schematic implementation of OSIRIS is presented in both reflection and transmission geometries.

[0046] The OSIRIS signal is related to both the magnitude of temperature change induced by the pump T and the intensity of the probe beam. That in most cases the temperature change results in some small perturbation in the optics of the probe implies the following relation:

[00003]$I_{\mathrm{probe}}T$

This relation is true with the caveat that the proportionality constant is itself a complicated function of the optical configuration (i.e. reflection or transmission mode, confocality, etc.), the thermo-mechanical and thermo-optic properties of the sample, and the degree of focus of the imaging system.

[0047] It is instructive to first consider what happens when the sample is illuminated by repeated infrared pulses. The general case of an infrared absorbing inclusion within a medium is presented in FIG. 1. While the inclusion initially heats rapidly in response to the pump, it is not linear due to heat transfer out of the inclusion. Once the pump is turned off, the inclusion cools until it reaches equilibrium with its surroundingsprimarily the volume heated by the infrared pump. The pumped volume will eventually equilibrate back to the ambient temperature, but in principle the equilibrium temperature of the pumped volume increases with successive pulses. Since this increased heating is of absorbent (on resonance) inclusions within the infrared spot is the fundamental physical process being attempted to measure with the probe to beat the diffraction limit, the best experimental method takes this physics and the associated timescales into account.

Assertion 1

[0048] The infrared pulses should be shorter than the cooling time constant of resolution scale objects.

[0049] Generally speaking, the size and shape (fractal dimension) of inclusions will vary considerably within and between samples. The cooling time of a given inclusion, , depends greatly on its size and shape. The cooling time for resolution scale spherical objects, .sub.res, is on the order of 20 ns in water. As the size of the inclusion increases, the cooling time scales quadratically, which follows from the heat transport equation:

[00004]$\frac{T}{t}={}^{2}T.fwdarw.\mathrm{size}{}^2.$

The cooling of a feature is related to the fraction of available space into which to cool (relative to its size). For example, a small spherical inclusion can cool into all 3 directions. A thin line can cool into 2 dimensions (i.e. in the plane perpendicular to its axis) and so on. The more dimensions the cooling can proceed into, the faster the particle will lower its temperature and so it becomes critical to excite the heating with very short pulses. Moreover, inclusions with complex surfaces will cool faster than inclusions with low surface area for their volume, such as a sphere. This complexity can be characterized by the fractal dimension of the inclusion.

[0050] FIG. 2 shows how distributing the same pulse energy over finite time effects the induced heating for various particle sizes. Note that small particles heat much less than large particles unless the pulse length is very short. This effect is formalized by the following relationship, which results from convolving a square pulse with the exponential cooling of the inclusion:

[00005]$I_{\mathrm{probe}}Tk\left(\right)P_{\mathrm{pulse}}\left(1-e^{\frac{-t_{\mathrm{pulse}}}{}}\right)\underset{t}{.Math.}k\left(\right)E_{\mathrm{pulse}}^{\mathrm{IR}}.fwdarw.\left(\right)C{E}_{\mathrm{pulse}}^{\mathrm{IR}}.$

Thus, the experimental advantage of using pulses that are short relative to .sub.res is clear: the OSIRIS signal will not depend on the size or shape of the inclusion through . Instead, the OSIRIS signal will depend on the absorption k(), which is proportional to the concentration C of the analyte and its characteristic infrared cross section (). This simplifies the data analysis.

Assertion 2

[0051] Each pulse should heat the sample near the damage threshold, after which the sample must be allowed to cool in preparation for the next pulse.

[0052] Recall from FIG. 1 that successive pulses can increase the temperature of the pumped volume appreciably if they occur rapidly relative to the cooling time of the pumped volume .sub.IR. The pumped volume is a large volume that can be heated considerably after many pulses. While the degree of heating is of course highly dependent on sample composition and pump power, it is certainly possible to damage many samples with a modern infrared laser source under the high Numerical Aperture of a microscope objective. By approximating the heated volume as a sphere with the same diameter as the IR spot, it is found that .sub.IR is at least

[00006]${\left(\frac{2{}_{\mathrm{IR}}}{{}_{\mathrm{probe}}}\right)}^2$

times larger than .sub.res for a diffraction limited probe

[00007]$\left(\mathrm{resolution}\frac{{}_{\mathrm{probe}}}{2}\right).$

For the current experimental setup, the factor

[00008]${\left(\frac{2{}_{\mathrm{IR}}}{{}_{\mathrm{probe}}}\right)}^2$

is roughly 2500. Thus, .sub.IR is on the order of 50 s in water. Although at first glance this seems like an unacceptably long time to wait after each pulse, note that letting the sample relax permits heating to the damage threshold by each pulse. That this is desirable can be seen from the fact that the signal to noise will be roughly linear in E.sub.pulse, while it is only proportional {square root over (N.sub.pulses)} in the best case of shot-noise limited detection. Thus, one should always demand more than 4 pulses in exchange for a pulse that is half as intense. As such derating of the pump is necessary to avoid the damage threshold, it is plausible that fewer, more intense pulses could provide some benefit. Of course, the pulse period T.sub.pulse, must be finite to permit co-adding to improve signal to noise, or to collect a nearby point. In the case of a finite period, the pulse energy must be decreased by a factor of

[00009]$\frac{1}{1-e^{-T_{\mathrm{pulse}}/{}_{\mathrm{IR}}}}$

to remain below the damage threshold. Given the assumption of shot-noise limited detection, the effect of different periods on SNReach with appropriate pulse energy to reach damage threshold is shown in FIG. 3. Clearly, the optimal pulse period is somewhere around 1.3.sub.IR.

[0053] These long periods provide an additional optical advantage. The successive heating after each pulse causes the sample to expand towards the objective, which means that the probe interrogates a slightly different position in the sample after each pulse. Consider a 100 micron thick sample with a coefficient of thermal expansion around 100E-6/K, which is reasonable proxy for water and various organics. The surface of the sample moves 10 nm per degree kelvin that it is heated. Certainly, in this situation, it is quite possible that the sample moves several pixels between the first pulse and whenever some steady-state is established. Such movement would not only compromise the resolution in the z direction, but also change the focus significantly, particularly in reflectance mode. For example, if the microscope focus is very slightly above the sample surface for a measurement, the first pulse will improve the focus, which will result in a peak in the probe detector signal. After several pulses, the focus could be precisely at the sample surface, or even slightly below. Then, each pump pulse would instead result in troughsnot peaks like beforein the probe signal. Such sign changes would cause real problems for the simple demodulation used in the MIP method.

[0054] The pump duty cycle for this type of microscopy most consistent with the thermal dynamics is related to the wavelengths of the pump and probe, where the 5.1 in the denominator comes from the diffraction limit and the optimum pump period being 1.3 .sub.IR.

[00010]$D\frac{{}_{\mathrm{probe}}^2}{5.2{}_{\mathrm{IR}}^2}.$

For the current pump and probe wavelength, the duty cycle should be less than 300 ppm. The consequences of this low duty cycle will echo through the rest of the instrument design.

Assertion 3

[0055] The probe should be pulsed.

[0056] Note that the most interesting data points are before the pulse to establish a baseline, and at the peak temperature to quantify the induced change. Several units of .sub.res later, the inclusion will equilibrate with the medium and any hope of meaningful microscopy is lost. Thus, except for a brief instant slightly before and after the pump pulse, perhaps 100 nanoseconds all-told, probe light is a detriment. In the best case, this probe light is an additional heat load on the sample. A worse case is possible at the short probe wavelengths that permit the maximum resolution, because the photon energy can be sufficiently high to alter the chemistry of the sample. In biological systems, this is referred to as photo-toxicity for obvious reasons. As this sort of damage is non-reversible, it is very different from the thermal damage threshold that we were concerned about with the pump. This means that there will be some limit of probe photons that can illuminate each pixel of the sample over the entire experiment. If we pulse the probe at a duty cycle that is 6 times larger than the pump duty cycle, we can increase the number of usable photons by roughly 500 times. Given shot-noise limited detection of the probe, this will increase the signal to noise by a factor of {square root over (N.sub.probephtons)}=22. Thus, pulsing the probe has immense consequences, potentially resulting in roughly 22 times more sensitivity.

Assertion 4

[0057] Multi-channel operation is critical for imaging dynamics.

[0058] This is necessary to address the reality that the image acquisition rate of OSIRIS is limited to one pixel per pump pulse. In water, this limits the pixel acquisition rate to 10 kHz at full sensitivity, which is not sufficient for video like imaging. The optimistic scenario would still place the pixel acquisition rate around 270 kHz. This would be sufficient to image a 300 pixel square image at 3 spectral points in a second. Note that this fundamental limit for the pixel acquisition rate of OSIRIS is quite slow; for studying chemical dynamics, one would want to be able to sample more spectral pointsat least 10at a higher framerateperhaps 20 frames per second. This can be addressed by splitting the probe beam into multiple channels with a diffractive optic or spatial light modulator and using multiple data acquisition channels. To fill the diffraction limited pump spot with sufficient spacing requires approximately 61 probes. For each probe, there are effectively 7 distinct spots on the confocal plane for coherent detection (0.33 Airy diameter detection fibers). Thus, the potential improvement from multiplexing is huge. In principle, so long as the pump energy is sufficient to expand the pump spot, there is no real limit on the number of detection channels. However, while the pixel acquisition rate is linear in the number of detection channels, there are significant diminishing returns on how additional probes increase the object size that can be studied at full resolution. The high bandwidth requirement of OSIRIS results in a significant marginal cost per channel. Thus, the practical limit to channel count depends on the implementation and the application.

Assertion 5

[0059] Signal processing should take place in the time domain.

[0060] The essential idea of demodulation is to project the raw time-domain signal onto a sinusoidal basis set. The resulting outputthe frequency domain signalcan be a more intuitive way to analyze highly periodic signals. Historically, demodulation had advantages in terms of isolating a weak signal from a noisy background because the noise could be averaged out via an analog low-pass filter. However, modern lock-in amplifiers, which are perhaps the most sensitive demodulation tool, are digital instruments which first collect the raw time-domain signal before performing the appropriate complex Fourier transform. Since the raw time-domain signal will be collected anyway, it is natural to ask whether demodulation, or some other algorithm, is the most appropriate way to analyze the OSIRIS data.

[0061] For MIP experiments, where duty cycles range from 5%-50% duty cycle, demodulation at the first harmonic has been used successfully. However, as the duty cycle decreases, the Fourier components are distributed over higher harmonics. Consider the case of a 5% duty cycle square pulse. For this case, which is a far higher duty-cycle than is appropriate for OSIRIS, the Fourier coefficient at the first harmonic is roughly 8% of the amplitude of the pulse. It follows that simple demodulation for low duty cycle waveforms results in lower signal and sacrifices information that is contained in the higher harmonics. In the high-duty cycle MIP regime, the disadvantages of first harmonic demodulation have thus far been accepted because of its usefulness in compressing the complicated waveform into a single number. However, in the low duty cycle regime of OSIRIS, the time-domain signal provides increased information in a more intuitive form that we can put to good use. Two approaches could prove useful: [0062] 1) Thermometry CalibrationThe probe is a poor thermometer. Whether the signal on the photodetector increases or decreases depends on whether the pump induced heating improves the focus. The photodetector signal may even be non-monotonic with temperature. The behavior of the probe cannot be calibrated by simple means because the initial focus will change as the probe scans. This can be affected by a number of factors including thermal drift, field curvature, and the structure of the sample. However, the heating portion of the signal trace provides information about how the signal changes with temperature. Using the knowledge that the pump pulse is sufficiently short that the temperature will increase at a constant rate, this part of the signal trace can be used to crudely calibrate the probe thermometry. The degree of calibration that can be accomplished depends on how many data points can be measured during the short pulse. A two-point calibrationone before and one after the pump pulsereveals whether the heating improved or degraded the focus. Adding a third point provides information about the degree of linearity of the probe thermometer. By correcting for the nonideality of the probe thermometry, this method is a necessary step towards a technique that could measure the quantitative chemical composition of the sample. [0063] 2) Photo-thermal GPSAt timescales greater than .sub.res, appreciable heat can flow between pixels. Neighboring regions of high absorption will manifest in the signal trace as a delayed peak. This implies that it may be possible to undersample the image, and then analyze the signal trace at a sparse array of points to interpolate the intermediate points, Upsampling. Although the potential of this method is limited because the signal weakens as it radiates out from the source, if this method allows one to interpolate every other point in practice, that amounts to an 8-fold increase in image acquisition rate for 3D imaging. FIGS. 4A and 4B shows the results of some simulations of using this method to upsample an image, which we call photo-thermal GPS because it uses a similar approach to localize the origin of a signal by triangulating based on time-lag (FIG. 4A). A test of photothermal GPS on synthetic data is presented in FIG. 4B. Although this is a very early attempt, the results are promising. A regularization step and other improvements can be added after doing some tests on experimental data. Even when the image is fully sampled, this method can be used reduce the uncertainty present in the image since measuring neighboring points provides additional information that one wouldn't have had if one had measured a solitary point. The size of the convolution kernel in photo-thermal GPS is set by the thermal conductivity of the sample. This parameter can either be provided a priori by the user, if the nature of the sample is known. For example, in biological cells, the water content is very high so the conductivity can be approximated as that of water. In samples where multiple phases or analytes exist and where the conductivity differs by a lot between these phases, it is possible (though time-consuming) to make an initial estimate of the average conductivity using the measured points and including conductivity as one of the parameters to be optimized. Furthermore, in all cases, one or more, iteration step(s) can be performed in which the thermal conductivity is determined for each analyte (most likely using a small subset of the image) and then the photo-thermal GPS procedure is applied using the modified conductivity values. In this case, instead of using the Green's function kernels, the convolution is applied using the Laplace operator and average thermal conductivity in each sampled point (derived based on analyte concentration from the previous iteration step).

Assertion 6

[0064] Concentration is proportional to the OSIRIS signal allowing spectral de-mixing, preferably NMF, to be used to extract concentration maps.

[0065] In the full embodiment, a static OSIRIS image is measured in 5 dimensions3 spatial dimensions, the time delay relative to the pump pulse, and the infrared pump frequency. For video imaging, there is, of course, an additional time (frame) dimension. With 5 or 6 dimensions, the raw dataset is a challenge to even visualize in a meaningful way. In general, the raw data is much too complicated to provide the intuitive understanding that makes imaging techniques so desirable. The raw data of MIP shares this impenetrable quality, especially when the sample is a priori unknown. Instead of focusing on the raw dataset, the goal of either technique, or indeed any chemical microscopy technique, should be to infer the spectrum and locations (concentration map) of each analyte.

[0066] That the raw data is so unintuitive, and the inference algorithm so indispensable, means that the inference algorithm is intimately tied to the technique. That the concentration cannot be straightforwardly inferred from the MIP data is the fatal flaw of that technique. However, the inference is tractable in OSIRIS since the OSIRIS signal is proportional to the concentration and absorption. This linear behavior opens the door to leverage existing matrix-based techniques. Since this reconstruction is quite nontrivial, it is preferable to adopt a Bayesian approach to quantify the uncertainty implicit in the data analysis. This is important, especially in research applications, to reduce the possibility of analysis artefacts or other reconstruction errors being used to draw unsubstantiated conclusions.

[0067] Thus, the first step towards inference is to write down the measurement model. The connection between , the concentration of an analyte .sub.analyte, and the infrared spectrum (cross-section) (.sub.IR) is shown by: [0068] =.sub.probe .sub.IR F(Geometry, Thermo-optics, . . . ) .sub.analyte .sub.IR, where: .sub.probe .sub.IR is addressed by normalization, F(Geometry, Thermo-optics, . . . ) is addressable by calibration or future modeling, and .sub.analyte .sub.IR is a critical term for qualitative understanding.

[0069] The OSIRIS signal depends linearly on the probe power .sub.probe and infrared power .sub.IR. This effect can be taken care of by normalizing with respect to the power of both beams. In addition, the OSIRIS signal involves a complicated proportionality factor F which depends on geometric details like the degree of optical focus and various thermo-optic properties of the sample such as the coefficient of thermal expansion (CTE), heat capacity (C.sub.p), and the thermo-optic coefficient

[00011]$\frac{n}{T}.$

While F is not presently well understood, it can in principle be addressed by the calibration strategy discussed previously.

[0070] For this work, the focus is on the product of the analyte concentration and the infrared response-since it is the most critical term for qualitative understanding. The matrix form of this factor is shown by: [0071] .sub.x M.sub.xa S.sub.a, wherein Q.sub.xis Osiris data, M.sub.xa is the map of analytes' abundance, and S.sub.ais the spectra of each analyte.

[0072] In this illustrative method, the multi-dimensional OSIRIS data is unwrapped into a 2 dimensional matrix where the columns contain spectral information, while the rows index the spatial/temporal coordinates. The first matrix M.sub.xathe mixing matrixis proportional to the concentration of each analyte a at each space-time coordinate x. The second matrix S.sub.athe spectral matrixtabulates the infrared absorption spectrum of an analyte in each row. A matrix equation of this form is a common situation in signal processing and dimensionality reduction, known as the Cocktail Party Problem. Solving this problem is often referred to as Spectral De-mixing. The various common methods for the cocktail problem such as Principal Component Analysis (PCA), Independent Component Analysis (ICA), and Non-Negative Matrix Factorization (NMF) are essentially pre-packaged sets of assumptions about the values of M and S. Note that failure to use the right assumptions for the technique, either implicitly or explicitly, in the factorization procedure will yield absurd results. For example, PCA assumes that the spectra are orthogonal. This is counter to the physical reality of infrared spectra, since analytes often share vibrational modes that are associated with common structure elements, i.e., the C-H stretching mode. Although the principal components may be interesting to think about in some cases, they should NOT be interpreted as infrared spectra since the assumptions of PCA do not match the physical meaning of the infrared spectra. Of the common cocktail party methods, the most sensible assumptions for OSIRIS are those of NMF, since the infrared cross-section and the analyte concentration cannot be negative. The Bayesian version of NMF with Gaussian observation noise is shown in FIG. 5. Note that the model has remaining issues which will need to be dealt with: [0073] The proper number of analytesoften referred to as model orderis not known at the outset. This can be inferred from the data using some Bayesian model comparison. Bayesian model comparison is the theoretical foundation of Occam's razor, in the sense that it penalizes the fitting of more parameters than are necessary to explain the data. [0074] There is an undetermined scale factor between the concentration map (column of M) and the Spectra of each analyte. This is unimportant in this qualitative analysis. This ambiguity can be removed by matching the spectra to a library. [0075] There is a permutation symmetry between various components. Failure to account for this will result in overestimating the uncertainty of each element of M and S because, for example, any of the analytes could be associated with component 1 (first column of M and row of S). This can be handled by matching the components in samples from the output (posterior) distribution based on the maximum correlation.

[0076] The first step towards testing our inference procedure is to generate some synthetic data. We first consider an example with 2 dimensional OSIRIS data (1 spatial coordinate and the frequency coordinate). Significant noise is added to the ground truth as shown in FIG. 6. We note that the spectra and spatial distributions chosen as the ground-truth are challenging since the spectra share infrared modes and the spatial distributions have significant mixing.

[0077] The Gibbs sampling procedure following Schmidt et al., Bayesian Non-negative Matrix Factorization In: Adali et al., (eds) Independent Component Analysis and Signal Separation. ICA 2009. Lecture Notes in Computer Science, vol 5441. Springer, Berlin, Heidelberg (2009) https://doi.org/10.1007/978-3-642-00599-2_68 can be used. When the number of analytes is known (3 in this case), both methods generally agree on the uncertainty and perform extremely well (see FIG. 7). However, in practice, the number of analytes present in the sample will not be known a priori, and the model order will need to inferred from the data. This step is critical because models of incorrect order will necessarily involve unphysical components (spectra and spatial distributions) (see FIG. 7). The simplest method of inferring the model order is to fit a number of different models with different orders and perform some method of model comparison.

[0078] Here, a model comparison is performed using the Bayesian information Criterion (BIC) at:

[00012]$\mathrm{BIC}{N}_{\mathrm{parameters}}\ln \left(N_{\mathrm{data}\mathrm{points}}\right)-2\ln (P\left(\mathrm{Data}|\right)\left(\mathrm{Smaller}\mathrm{is}\mathrm{better}\right).$

For this synthetic dataset, the BIC is quite successful in determining the correct model order (See FIG. 7). Note that the BIC relies on approximations that leave it only as a heuristic measure of model quality. More computationally expensive methods of model comparison such as bridge sampling exist if the BIC turns out to not be sufficient with real data.

[0079] While simple one (spatial) dimensional examples are discussed for clarity of presentation, this method is of course generalizable to multi-dimensional OSIRIS data. A proof of concept for this is presented in FIG. 8 for a 2 dimensional region with 3 analytes localized in Gaussian profiles similar to the earlier examples.

[0080] Note that we've simplified the observation model described in this section to aid in the illustration of our approach to extracting the concentration maps. A more interesting observation model (Likelihood) would be the following (Note priors are omitted):

[00013]${}_{x,u,z,t,,t}{(K_{\mathrm{thermal}})}^t{M}_{x,y,z,t,a}{S}_{a,}+N\left(0,{}_{\mathrm{noise}}\right).$

Variables are described in the following table.

TABLE-US-00001 TABLE 1 Observation Model Variables Variable Description .sub.x,y,z,t,,t Measured OSIRIS Data (K.sub.thermal {circle around (*)}).sup.t Convolution from thermal diffusion: Inverse of photothermal GPS K.sub.thermal {circle around (*)} Convolution operator for the convolution kernel K corresponding to thermal diffusion in one timestep. Depends on the thermal properties of the medium. Photothermal-GPS is the inverse operation (de- convolution). t The number of timesteps since the infrared pump pulse. M.sub.x,y,z,t,a Concentration Map mixing matrix/tensor S.sub.a, Spectral Matrix N(0, .sub.noise) Observation Noise x, y, z, t Space-time coordinates of the image. The time index t denotes the frame for video imaging. Index corresponding to the infrared pump wavelength a Analyte index The OSIRIS image S The spectral matrix M The mixing matrix/tensor N Distribution of the observation noise .sub.noise Standard deviation of the observation noise

[0081] Statistical inference with this model will combine photothermal GPS and spectral de-mixing in one step. The OSIRIS image could be undersampled to improve the image acquisition rate at the cost of increased uncertainty at the unsampled coordinates (see photothermal GPS section). The Bayesian approach allows any uncertainty induced by such undersampling to be quantified in a consistent way by the inference algorithm. Quantifying this uncertainty is imperative to provide confidence increasing the degree of undersampling so long as the uncertainty is acceptable to the user.

[0082] The computational cost of the convolution step is not prohibitive because the data will be sampled on the .sub.res timescale, which means that only heat flow from near neighbor pixels likely needs to be considered. Consequently, K.sub.thermal will have relatively few elements. This is the regime in which it is more computationally efficient to perform the convolution in real space (as opposed to Fourier space, which is common when the kernel is large). This real-space analysis presents an additional opportunity when undersampling is used, since the convolution need only be performed at data points that were actually measured.

[0083] Both transmission and reflection embodiments have been considered, which are appropriate for different types of samples (FIGS. 9 and 10). FIG. 9 shows a Transmission Embodiment including: a pulse waveform driver 1; a pulsed laser source 2, such as a Quantum Cascade Laser (QCL) or an Optical Parametric Oscillator (OPO); a probe laser 3, such as a pulsed laser or continuous wave (CW) laser; an infrared (IR) focusing optic 4; an IR reference detector 5; a preamplifier 6; data acquisition (DAQ) hardware 7; and an objective lens 8.

[0084] A sample 10 on a sample motion stage 11 is depicted with analyte spheres 9. The system of FIG. 9 also includes a condenser 12; a dichroic mirror 13; a confocal lens 14; a fast visible detector 15, such as a Photomultiplier Tube (PMT) or a Silicon Photomultiplier (SiPM); a preamplifier 16; an IR detector 17; a preamplifier 18; DAQ hardware 19; DAQ hardware 20, a signal processor 21 for signal processing, such as a field programmable gate array (FPGA), DAQ hardware, or a personal computer (PC); a dichroic mirror 22; a pick-off mirror 23; a confocal pinhole 24; a confocal pinhole 25; and scan mirrors 28.

[0085] FIG. 9 further depicts a pulse of light from the pulsed laser source 2, having a pulse waveform 26, and light from a probe laser 3, being directed to and through the sample 10. This results in probe time series data indicated at 27.

[0086] FIG. 10 shows a Reflection Embodiment including: a pulse waveform driver 1; a pulsed laser source 2, such as a Quantum Cascade Laser (QCL) or an Optical Parametric Oscillator (OPO); a probe laser 3, such as a pulsed laser or continuous wave (CW) laser; an infrared (IR) focusing optic 4; an IR reference detector 5; a preamplifier 6; data acquisition (DAQ) hardware 7; and an objective lens 8.

[0087] A sample 10 on a sample motion stage 11 is depicted with analyte spheres 9. The system of FIG. 10 also includes a condenser 12; a dichroic mirror 13; a confocal lens 14; a fast visible detector 15, such as a Photomultiplier Tube (PMT) or a Silicon Photomultiplier (SiPM); a preamplifier 16; an IR detector 17; a preamplifier 18; DAQ hardware 19; DAQ hardware 20, a signal processor 21 for signal processing, such as a field programmable gate array (FPGA), DAQ hardware, or a personal computer (PC); a dichroic mirror 22; a pick-off mirror 23; a confocal pinhole 24; a confocal pinhole 25; scan mirrors 28, and a beamsplitter 29.

[0088] FIG. 10 further depicts a pulse of light from the pulsed laser source 2, having a pulse waveform 26, and light from a probe laser 3, being directed to and through the sample 10. This results in probe time series data indicated at 27.

[0089] With minimal compromise, modern laser technology is capable of meeting the performance requirements for OSIRIS. In one example, an Optical Parametric Oscillator (OPO) from Ekspla (EKSPLA NT277-XIR) was chosen for the pump laser. The pulse length of 7 ns is ideal-well below the 20 ns requirement. Moreover, it has a maximum pulse energy of 12 J, which is sufficient to heat an IR Point Spread Function (PSF) sized voxel of water by 100 K. As water has a particularly high specific heat, 12 J is plenty of power to reach the damage threshold with each pulse for all samples. Unfortunately, the repetition rate of 1 kHz is significantly below the optimal value of 1 kHz, which will reduce the pixel acquisition rate 15 fold. However, the estimated loss in sensitivity (SNR) from the lower repetition rate is only a factor of three. The compromise on repetition rate is justified because this laser has the best available combination of tuning speed and repetition rate. Together, they will result in the fastest image acquisition rate for hyperspectral images.

[0090] In implementations, for pulsed probe operation to work properly, the probe pulse needs to be somewhat broader than the pump pulse to allow multiple data points and arrive somewhat before the pump pulse to allow the collection of a baseline probe signal. To implement this mode-locked scheme, some stand-alone digital logic is required to trigger one laser based on another. In one scheme, an FPGA then generates the appropriate probe waveform from the pump trigger. Some software code enables one to vary the details of the probe pulse timing. The FPGA serves as a fixed piece of glue logic to generate the desired probe waveform at all times. Since the probe has the potential to damage the sample, the implementation provides an external Probe_Enable signal that must be asserted high for the probe to pulse.

[0091] In implementations, the requirements for the OSIRIS detection system are a time constant comfortably below .sub.res and a noise level that is below the statistical fluctuation of the photons in the probe beam, hereafter referred to as shot noise. The detector and preamp should be sufficiently cheap and compact to make it suitable for multi-channel operation.

[0092] In one as-built prototype at a sensible probe beam power and a confocal pinhole appropriate for super-resolution microscopy (0.3 Airy Diameters), 350 nW of power is incident on the detector. Although we anticipate much higher peak power may be utilized when pulsing the probe, designing the detection system around this value will ensure that it can cope with the worst case scenario.

[0093] The photodetectors that might be useful for this application are photodiodes (PD), avalanche photodiodes (APD), silicon photomultipliers (SiPM), and photomultiplier tubes (PMT). In these detectors some fraction, called the quantum efficiency of the incident photons, are absorbed and contribute to a current. In the APD, SiPM, and PMT, the output photocurrent is enhanced by a factor called the internal gain y. These detectors are suitably similar that they can be considered within the context of a generic preamplifier, shown in FIG. 9, for example. For a suitably chosen preamplifier, the Johnson noise is the dominant electronic noise source. Thus, the necessary condition to achieve shot noise limited detection is as follows:

[00014]$\sqrt{R}>\frac{1}{e}\sqrt{\frac{2k_B\mathrm{Thc}}{P_{\mathrm{probe}}{}_{\mathrm{probe}}}\frac{4.449\sqrt{W}}{\sqrt{P_{\mathrm{probe}}}}}$

[0094] The proper detection element should accommodate the pulsed probe case as well as the current CW probe case, where P.sub.probe=350 nW. In the CW probe case, the right hand side is quite large, roughly 7.6E3. The finite gain-bandwidth product GBP of real op-amps provides a further constraint on R. As the fastest commercially available op-amps, such as the LTC6268, have a GBP around 4 GHz, the load resistor should be significantly less than 250 (2tGBP) to avoid unnecessary distortion of the signal. This implies that an internal gain around 1000 will be necessary to ensure shot-noise limited operation, ruling out the simple photodiode and normal APDs, which typically have internal gains of around 100. While both PMTs and SiPMs have sufficient internal gain, modern SiPMs do not have sufficient dynamic range to handle powers beyond a few hundred nW, which will certainly be the case in the pulsed probe regime. Thus, the Photomultiplier Tube is the best choice to accommodate both the CW and pulsed probe regimes.

[0095] As a relic of the vacuum tube era, PMTs are a curious piece of technology. That they are still widely used today is because of their particular niche: an extremely high gain and detection area which is decoupled from a consequent reduction in bandwidth, which occurs in solid state detectors and amplifiers. This is quite helpful, if not necessary, to amplify the photocurrent, which before internal gain is on the order of 100 nA, to the single volt level necessary to fill the input range of the analog-digital converter. One exemplary photodiode module is from Hamamatsu (H9305-03). This module allows for adjustment of the gain by 4 orders of magnitude to test both CW and pulsed probe modalities. Regardless of the internal gain used, the maximum amplified current is limited by the last dynone, approximately 10 mA for the chosen module. This means that when the PMT internal gain is set appropriately for the probe, a single fixed gain trans-impedance preamp with a sensible gain value, such as 50V/A (R=50 Ohms, C=200 pF) will not only set the time constant (=10 ns) but amplify the signal to a sensible value (0.5V) for the ADC.

[0096] The required bit-depth for the ADC should be sufficiently high that any sampling noise is less than the shot noise. For the CW case (P.sub.probe=350 nw), around 8750 photons are incident on the detector in each 10 ns sample. For such a sample, the shot-noise limited signal to noise ({square root over (N.sub.photons)} ) will be approximately 100. Thus, the CW case can be sampled at full precision with around 7 bits of precision. As discussed previously, the pulsed probe case could increase the SNR by a further factor of 22, which would require 5 additional bits. Thus, 12 bits of precision are sufficient for OSIRIS data acquisition. In one implementation, an ATS9350 (12 bit, 500 MS/s) from Alazartech was chosen as the DAQ card for single channel operation. This will satisfy the aforementioned precision requirements and provides sufficient resolution for both thermal GPS and a 4 point probe thermometry calibration.

[0097] A FPGA is much more appropriate for highly parallel computations than the CPU of the host PC. Doing such computation on the FPGA saves CPU time and could dramatically improve the usability of the tool and the experience of the user. However, the FPGA is a particularly rigid, inflexible platform that is difficult to troubleshoot. Thus, the proper approach is to first use the commercial multi-channel DAQ to test and iterate on the signal processing algorithms. Then, the refined algorithm can be implemented in the FPGA if it will yield significant benefit.

[0098] The above descriptions are those of the preferred embodiments of the invention. Various modifications and variations are possible in light of the above teachings without departing from the spirit and broader aspects of the invention. It is therefore to be understood that the claimed invention may be practiced otherwise than as specifically described. Any references to claim elements in the singular, for example, using the articles a, an, the, or said, is not to be construed as limiting the element to the singular.