Methods and Systems for Processing of Microscopy Images
20230179885 · 2023-06-08
Assignee
Inventors
Cpc classification
H01J37/1471
ELECTRICITY
H01J37/244
ELECTRICITY
H04N25/75
ELECTRICITY
H01J2237/24495
ELECTRICITY
International classification
H04N25/75
ELECTRICITY
H01J37/147
ELECTRICITY
H01J37/244
ELECTRICITY
H01J37/26
ELECTRICITY
Abstract
Techniques for acquiring an electron energy loss spectrum in two dimensions are disclosed herein. The technique at least includes exposing an electron sensor to an electron spectrum projected in two dimensions, wherein one of the two dimensions corresponds to a dispersive axis, and the other of the two dimensions corresponds to a non-dispersive axis, receiving an electron sensor readout frame from the electron sensor, where the electron sensor readout frame comprises a plurality of values representative of the electron spectrum in each of the two dimensions, and reducing a resolution of the electron sensor readout frame in at least one of the two dimensions, where reducing the resolution includes reducing the number of values in the at least one of the two dimensions, where the electron sensor readout frame comprises a plurality of values in each of the two dimensions after the reduction in resolution.
Claims
1. A method of acquiring an electron energy loss spectrum in two dimensions, comprising: exposing an electron sensor to an electron spectrum projected in the two dimensions, wherein one of the two dimensions corresponds to a dispersive axis, and the other of the two dimensions corresponds to a non-dispersive axis; receiving an electron sensor readout frame from the electron sensor, wherein the electron sensor readout frame comprises a plurality of values representative of the electron spectrum in each of the two dimensions; and reducing a resolution of the electron sensor readout frame in at least one of the two dimensions, wherein reducing the resolution comprises reducing the number of values in the at least one of the two dimensions, and wherein the electron sensor readout frame comprises a plurality of values in each of the two dimensions after the reduction in resolution.
2. The method according to the preceding claim, wherein the method further comprises sending the electron sensor readout frame after reducing its resolution.
3. The method according to claim 1, wherein the reduction in resolution is carried out in the dimension corresponding to the non-dispersive axis.
4. The method according to claim 1, wherein the reduction in resolution comprises summing up, at least in part, the electron spectrum.
5. The method according to claim 1, wherein the reduction in resolution comprises convolution of, at least part of, the electron spectrum with a window function.
6. The method according to claim 1, wherein the method comprises receiving a plurality of electron sensor readout frames from the electron sensor.
7. The method according to claim 6, wherein the method further comprises summing together a plurality of electron sensor readout frames.
8. A microscopy system comprising: an electron sensor configured to detect electrons impinging on the sensor in two dimensions, wherein one of the two dimensions corresponds to a dispersive axis and the other of the two dimensions corresponds to a non-dispersive axis; and a sensor processing unit configured to receive an electron sensor readout frame from the electron sensor, wherein the electron sensor readout frame comprises a plurality of values representative of the electron spectrum in each of the two dimensions, and wherein the sensor processing unit is further configured to reduce a resolution of the electron sensor readout frame in at least one of the two dimensions, wherein reducing the resolution comprises reducing the number of values in at least one of the two dimensions, and wherein the electron sensor readout frame comprises a plurality of values in each of the two dimensions after the reduction in resolution.
9. The microscopy system according to claim 8, wherein the electron sensor comprises a direct detection sensor.
10. The microscopy system according to claim 8, wherein the electron sensor comprises an indirect detection sensor.
11. The microscopy system according to claim 8, wherein the sensor processing unit is further configured to receive a target resolution in any of the two dimensions, and wherein it is further configured to reduce the resolution of the electron sensor readout frame to the target resolution.
12. Use of the method according to claim 1 to align an electron beam.
13. The use according to claim 12, wherein aligning the electron beam comprises determining a drift of the electron spectrum based on the electron sensor readout frame.
14. The use according to claim 13, wherein aligning the electron beam further comprises using the drift as feedback.
15. The use according to claim 13, wherein aligning the electron beam further comprises using the drift as feedforward.
16. The use according to claim 12, wherein the use further comprises aligning an electron sensor.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0043]
[0044]
[0045]
DETAILED DESCRIPTION OF EMBODIMENTS
[0046]
[0047] Also depicted is a deflector 8, which (inter alia) can be used to effect scanning motion of the beam B. The sample 18 may be held on a sample holder i6 that can be positioned in multiple degrees of freedom by a positioning device/stage 14, which moves a cradle 14′ into which holder 16 is (removably) affixed; for example, the sample holder 16 may comprise a finger that can be moved (inter alia) in the XY plane (see the depicted Cartesian coordinate system; typically, motion parallel to Z and tilt about X/Y will also be possible). Such movement allows different parts of the sample 18 to be illuminated/imaged/inspected by the electron beam B traveling along axis B′ (in the Z direction) (and/or allows scanning motion to be performed, as an alternative to beam scanning). If desired, an optional cooling device (not depicted) can be brought into intimate thermal contact with the sample holder 16, so as to maintain it (and the sample 18 thereupon) at cryogenic temperatures, for example.
[0048] The electron beam B will interact with the sample 18 in such a manner as to cause various types of “stimulated” radiation to emanate from the sample 18, including (for example) secondary electrons, backscattered electrons, X-rays and optical radiation (cathodoluminescence). If desired, one or more of these radiation types can be detected with the aid of analysis device 22, which might be a combined scintillator/photomultiplier or EDX (Energy-Dispersive X-Ray Spectroscopy) module, for instance; in such a case, an image could be constructed using basically the same principle as in a Scanning Electron Microscope (SEM). However, alternatively or supplementally, one can study electrons that traverse (pass through) the sample 18, exit/emanate from it and continue to propagate (substantially, though generally with some deflection/scattering) along axis B′.
[0049] Such a transmitted electron flux enters an imaging system (energy filter) 24, which will generally comprise a variety of electrostatic/magnetic lenses, deflectors, correctors (such as stigmators), etc. In particular, when the microscopy system M is used for electron energy loss spectroscopy, the imaging system 24 may comprise the offset drift tube 26. The offset drift tube 26 may comprise a region where a magnetic field (not shown) may be applied to the electron beam B. The magnetic field may be applied in a direction substantially parallel to the Y-direction in the configuration depicted in
[0050] The electrons emitted from the offset drift tube 26 may then enter an imaging sub-system 28 that may also comprise a variety of electrostatic/magnetic lenses, deflectors, correctors (such as stigmators), etc. The imaging sub-system 28 may be configured, for example, to cause a spread of the electron beam B in the Y-direction (the non-dispersive dimension in the configuration of
[0051] The sensor 30 may send the 2-dimensional electron spectrum 100 to a sensor processor, that may also be called a sensor processing unit, 32. The 2-dimensional electron spectrum 100 (that may also be called an electron sensor readout frame or simply a sensor readout frame) may comprise a plurality of values in each of the two dimensions corresponding to a dispersive dimension (X) and a non-dispersive dimension (Y). Note that the Z-direction has been chosen to correspond to the optic axis, or the direction in which the electron beam propagates. Consequently, while the Z axis points in the vertically downward direction in the vacuum enclosure 2, it gets rotated by the offset drift tube 26 to point in the horizontally right direction in the imaging system 24 and later on.
[0052] The sensor processor 32 may be configured to reduce a resolution of the 2-dimensional sensor readout frame 100 without significant processing overhead. The sensor processor 32 may also be configured to send the sensor readout frame 110 with the reduced resolution to an external device 40, wherein the external device 40 may comprise, for example, a computer, a tablet, a laptop, a smartphone, or any other data processing device. Further, the sensor processor 32 may be configured to reduce the resolution of the 2-dimensional electron spectrum 100 by reducing the number of values representative of the electron spectrum in at least one of the two dimensions corresponding to the dispersive and non-dispersive dimensions. Preferably, the sensor processor 32 may reduce the resolution in the non-dispersive dimension. The number of values representative of the electron spectrum in each of the two dimensions corresponding to the dispersive and non-dispersive dimensions of the electron sensor readout frame no may be greater than 1 after the reduction in resolution by the sensor processor 32.
[0053] For example, the sensor 30 may send a sensor readout frame 100 comprising a total of 50 values in a 5×10 grid in the X-Y plane corresponding to the number of electrons detected at 50 pixels of the sensor 30. Thus, the readout frame 100 comprises values representative of the electron spectrum in 10 energy bands (along to the X-direction) and 5 bands along the non-dispersive direction (along the Y-direction). The sensor processor 32 may then be configured to reduce the resolution of this frame 100 to a 2×10 grid in the X-Y plane representing an electron spectrum 110 in 10 energy bands but only 2 bands in the non-dispersive direction. Since the sensor processor 32 may only reduce the resolution of an input electron sensor readout frame loo and output the electron sensor readout frame 110 with the reduced resolution, it may be configured to have a frame processing rate substantially identical to the rate at which electrons are detected by the sensor 30.
[0054] Note that a controller (that may be a computer processor) 20 is connected to various illustrated components via control lines (buses) 20′. This controller 20 can provide a variety of functions, such as synchronizing actions, providing setpoints, processing signals, performing calculations, and displaying messages/information on a display device (not depicted). Needless to say, the (schematically depicted) controller 20 may be (partially) inside or outside the enclosure 2, and may have a unitary or composite structure, as desired. The skilled artisan will understand that the interior of the enclosure 2 does not have to be kept at a strict vacuum; for example, in a so-called “Environmental TEM/STEM”, a background atmosphere of a given gas is deliberately introduced/maintained within the enclosure 2. The skilled artisan will also understand that, in practice, it may be advantageous to confine the volume of enclosure 2 so that, where possible, it essentially hugs the axis B′, taking the form of a small tube (e.g., of the order of 1 cm in diameter) through which the employed electron beam passes, but widening out to accommodate structures such as the source 4, sample holder 16, offset drift tube 26, sensor 30, sensor processor 32, etc.
[0055]
[0056] According to an embodiment of the present invention as depicted in
[0057] The reduction in the number of values (that may also be called reduction in resolution as described above) may be carried out by any of summing up, averaging, convolving with a window function, and any other (many-to-one or many-to-many) mathematical transformation of the values in the dimension in which the resolution is reduced. Further, any two sets of values that may be mathematically transformed, as described above, into values representative of the electron spectrum (with the reduced resolution) may be disjoint, or may have a non-zero intersection, i.e., the reduction in resolution may be performed by transforming regions of the electron sensor readout frame 100 that may or may not be disjoint.
[0058] For example, in the embodiment depicted in
[0059] Embodiments of the present invention may be of particular advantage in alignment of the electron beam B and/or the electron sensor 30 (as shown in
[0060] By making use of features characteristic of a misalignment in the 2-dimensional sensor readout frame 110 even with the reduced resolution, the alignment may be corrected. Such features may be identified by visual inspection by human or automatically by means of a computer-implemented method. The computer-implemented method may comprise, for example, an artificial intelligence-based method that may accept the sensor readout frame 110 as an input and that may produce a measure of misalignment such as a drift of the electron spectrum, or a measure of correction such as an angle by which the electron beam B or the electron sensor 30 may be tilted, as an output. The artificial intelligence-based method may comprise, at least in part, any of a neural network, such as a deep convolutional neural network, a recurrent neural network, or any other architecture of neural network, a random forest, a gradient boosting machine, a support vector machine, or any other artificial intelligence-based model. The artificial intelligence-based method may be trained appropriately by, for example, varying the alignment of the electron beam B and/or the electron sensor 30 and acquiring the electron sensor readout frame 110 for each position of alignment. The output of the computer-implemented method (or also the visual inspection described above) may be used either as feedback or feedforward into the microscopic system M, preferably the controller 20 of the microscopic system M, to correct for the misalignment.
[0061] Further, for example, the electron sensor readout frame 110 with the reduced resolution may be convolved with a step function along the non-dispersive axis such that a difference between the spectra along the non-dispersive axis is effectively calculated. The result of this convolution may be used as an input for a feedback control that controls the electron sensor 30 or the beam deflection in the non-dispersive direction. The controller may be an artificial intelligence-based PID controller or may employ any other suitable control method. Thus, a shift/drift of the electron spectrum may be detected not only in a direction orthogonal to the dispersive axis but also at an angle. Additionally, multiple electron spectra may be correlated to determine an angle that may be used for feedback or feedforward leading to improvement in the energy resolution or a modulation transfer function (characteristic of the resolution and performance of the microscopy system M) of the microscopy system M.
[0062] Overall, embodiments of the present technology thus provide a method to enhance the amount of information that may be extracted from a microscopic image, preferably an electron microscopic image, that is robust and efficient.
[0063] Whenever a relative term, such as “about”, “substantially” or “approximately” is used in this specification, such a term should also be construed to also include the exact term. That is, e.g., “substantially straight” should be construed to also include “(exactly) straight”.
[0064] Whenever steps were recited in the above or also in the appended claims, it should be noted that the order in which the steps are recited in this text may be accidental. That is, unless otherwise specified or unless clear to the skilled person, the order in which steps are recited may be accidental. That is, when the present document states, e.g., that a method comprises steps (A) and (B), this does not necessarily mean that step (A) precedes step (B), but it is also possible that step (A) is performed (at least partly) simultaneously with step (B) or that step (B) precedes step (A). Furthermore, when a step (X) is said to precede another step (Z), this does not imply that there is no step between steps (X) and (Z). That is, step (X) preceding step (Z) encompasses the situation that step (X) is performed directly before step (Z), but also the situation that (X) is performed before one or more steps (Y1), . . . , followed by step (Z). Corresponding considerations apply when terms like “after” or “before” are used.
[0065] While in the above, a preferred embodiment has been described with reference to the accompanying drawings, the skilled person will understand that this embodiment was provided for illustrative purpose only and should by no means be construed to limit the scope of the present invention, which is defined by the claims.
[0066] The present invention is also defined by the following numbered embodiments.
[0067] Below system embodiments will be discussed. These embodiments are abbreviated by the letter ‘S’ followed by a number. Whenever reference is herein made to system embodiments, these embodiments are meant.
[0068] S1. A microscopy system comprising: [0069] an electron sensor configured to detect electrons impinging on the sensor in two dimensions, wherein one of the two dimensions corresponds to a dispersive axis and the other of the two dimensions corresponds to a non-dispersive axis; and [0070] a sensor processing unit configured to receive an electron sensor readout frame from the electron sensor, wherein the electron sensor readout frame comprises a plurality of values representative of the electron spectrum in each of the two dimensions, and wherein the sensor processing unit is further configured to reduce a resolution of the electron sensor readout frame in at least one of the two dimensions, wherein reducing the resolution comprises reducing the number of values in at least one of the two dimensions.
[0071] S2. The microscopy system according to the preceding embodiment, wherein the sensor processing unit is further configured to send the electron sensor readout frame with the reduced resolution.
[0072] S3. The microscopy system according to any of the preceding system embodiments, wherein the sensor processing unit is configured to reduce the resolution of the electron sensor readout frame in the dimension corresponding to the non-dispersive axis.
[0073] S4. The microscopy system according to any of the preceding system embodiments, wherein the electron sensor readout frame comprises a plurality of values in each of the two dimensions after the reduction in resolution.
[0074] S5. The microscopy system according to any of the preceding system embodiments, wherein the electron sensor comprises a direct detection sensor.
[0075] S6. The microscopy system according to any of the preceding system embodiments but without the features of the preceding embodiment, wherein the electron sensor comprises an indirect detection sensor.
[0076] S7. The microscopy system according to any of the preceding system embodiments, wherein the sensor processing unit is further configured to receive a target resolution in any of the two dimensions, and wherein it is further configured to reduce the resolution of the electron sensor readout frame to the target resolution.
[0077] S8. The microscopy system according to any of the preceding system embodiments, wherein the sensor processing unit is configured to receive a plurality of electron sensor readout frames.
[0078] S9. The microscopy system according to the preceding embodiment, wherein the sensor processing unit is configured to reduce the resolution of a plurality of electron sensor readout frames.
[0079] S10. The microscopy system according to any of the 2 preceding embodiments, wherein the sensor processing unit is configured to sum together a plurality of electron sensor readout frames.
[0080] S11. The microscopy system according to any of the preceding system embodiments, wherein the microscopy system is further configured to determine a drift of the electron spectrum based on the electron sensor readout frame.
[0081] S12. The microscopy system according to the preceding embodiment, wherein the drift of the electron spectrum is determined after the resolution of the electron sensor readout frame has been reduced.
[0082] S13. The microscopy system according to any of the 2 preceding embodiments, wherein the microscopy system is further configured to align the electron sensor based on the drift of the electron spectrum.
[0083] S14. The microscopy system according to any of the 3 preceding embodiments, wherein the microscopy system is configured to direct a beam of electrons at a sample, and wherein the microscopy system is configured to align the electron beam based on the drift of the electron spectrum.
[0084] Below method embodiments will be discussed. These embodiments are abbreviated by the letter ‘M’ followed by a number. Whenever reference is herein made to method embodiments, these embodiments are meant.
[0085] M1. A method of acquiring an electron energy loss spectrum in two dimensions, comprising: [0086] exposing an electron sensor to an electron spectrum projected in the two dimensions, wherein one of the two dimensions corresponds to a dispersive axis, and the other of the two dimensions corresponds to a non-dispersive axis; [0087] receiving an electron sensor readout frame from the electron sensor, wherein the electron sensor readout frame comprises a plurality of values representative of the electron spectrum in each of the two dimensions; and [0088] reducing a resolution of the electron sensor readout frame in at least one of the two dimensions, wherein reducing the resolution comprises reducing the number of values in the at least one of the two dimensions.
[0089] M2. The method according to the preceding embodiment, wherein the method further comprises sending the electron sensor readout frame after reducing its resolution.
[0090] M3. The method according to any of the 2 preceding embodiments, wherein the electron sensor readout frame comprises a plurality of values in each of the two dimensions after the reduction in resolution.
[0091] M4. The method according to any of the preceding method embodiments, wherein the reduction in resolution is carried out in the dimension corresponding to the non-dispersive axis.
[0092] M5. The method according to any of the preceding method embodiments, wherein the reduction in resolution comprises summing up, at least in part, the electron spectrum.
[0093] M6. The method according to any of the preceding method embodiments, wherein the reduction in resolution comprises convolution of, at least part of, the electron spectrum with a window function.
[0094] M7. The method according to any of the preceding method embodiments, wherein the method comprises receiving a plurality of electron sensor readout frames from the electron sensor.
[0095] M8. The method according to the preceding embodiment, wherein the method further comprises reducing the resolution of a plurality of electron sensor readout frames.
[0096] M9. The method according to any of the 2 preceding embodiments, wherein the method further comprises summing together a plurality of electron sensor readout frames.
[0097] S15. The microscopy system according to any of the preceding system embodiments, wherein the microscopy system is configured to perform the method according to any of the preceding method embodiments.
[0098] Below use embodiments will be discussed. These embodiments are abbreviated by the letter ‘U’ followed by a number. Whenever reference is herein made to use embodiments, these embodiments are meant.
[0099] U1. Use of the method according to any of the preceding method embodiments to align an electron beam.
[0100] U2. The use according to the preceding embodiment, wherein aligning the electron beam comprises determining a drift of the electron spectrum based on the electron sensor readout frame.
[0101] U3. The use according to the preceding embodiment, wherein aligning the electron beam comprises determining a drift of the electron spectrum based on a plurality of electron sensor readout frames.
[0102] U4. The use according to any of the preceding use embodiments and with the features of embodiment U2, wherein aligning the electron beam further comprises using the drift as feedback.
[0103] U5. The use according to any of the preceding use embodiments and with the features of embodiment U2, wherein aligning the electron beam further comprises using the drift as feedforward.
[0104] U6. The use according to any of the preceding use embodiments and with the features of embodiment U2, wherein the drift is determined using a machine learning model.
[0105] U7. The use according to the preceding embodiment, wherein an input to the machine learning model comprises the electron sensor readout frame.
[0106] U8. The use according to any of the 2 preceding embodiments, wherein an output of the machine learning model comprises at least one value based on which the electron sensor is aligned.
[0107] U9. The use according to any of the preceding use embodiments, wherein the use comprises using the method according to any of the preceding method embodiments to align an electron sensor.
[0108] Below computer program product embodiments will be discussed. These embodiments are abbreviated by the letter ‘P’ followed by a number. Whenever reference is herein made to computer program product embodiments, these embodiments are meant.
[0109] P1. A computer program product comprising instructions, when run on a data processing unit of a system according to any of the preceding system embodiments, to at least carry out a reduction in resolution of at least one electron sensor readout frame.