METHOD OF AND APPARATUS FOR ANALYSIS OF A SAMPLE OF BIOLOGICAL TISSUE CELLS
20170160536 ยท 2017-06-08
Inventors
Cpc classification
G02B21/0056
PHYSICS
G06V20/69
PHYSICS
G02B21/367
PHYSICS
International classification
G02B21/36
PHYSICS
G01N21/25
PHYSICS
Abstract
At least first and second digital images of the sample are acquired having different focal heights relative to a platform on which the cells are disposed. A contrast matrix is produced having elements computed in dependence upon the difference between the values of the corresponding pixels in the first and second images. A phase matrix is produced by convolution of the contrast matrix with a predetermined distance matrix. The phase matrix is used to assess characteristics of the sample, such as the presence of cells in the sample or the heights of cells in the sample.
Claims
1. A method of analysis of a sample of biological tissue cells disposed on a platform, the sample being a thin phase object and the platform being substantially transparent, the method comprising: notionally dividing the sample up into an array of portions; determining a measure of the height of the platform on which each portion of the sample is disposed; and acquiring from light passing through the platform and the sample, using an optical system including a camera and an objective optical element disposed between the camera and the platform, a set of image patches of the portions of the sample with the focal height of each image patch being adjusted in dependence upon the respective measure of the height of the platform; stitching the image patches together to form an image of the sample; and using the image to assess characteristics of the sample; wherein the measure of the height of the platform is determined by: projecting a beam of light into the objective optical element substantially parallel to and offset from the optical axis so that: the beam is projected from the objective optical element to the platform at an inclined angle to the platform so that the beam is reflected, and the reflected beam is projected back into the objective optical element so that the reflected beam is projected from the objective optical element to the camera; using the camera to acquire an image of the reflected beam; and detecting the position of the reflected beam in the image.
2. The method of claim 1, wherein: the image patches for the image are of overlapping portions of the sample; and in the step of stitching together the image patches, the image patches are blended together in the areas of overlap.
3. The method of claim 2, wherein the reflected beam is reflected by an under surface of the platform; a second beam is reflected by an upper surface of the platform and is projected back into the objective optical element so that the second reflected beam is also projected from the objective optical element to the camera; and choosing whether to determine the measure of the height of the platform on which each portion of the sample is disposed from the reflected beam or the second reflected beam.
4. The method of claim 1, wherein acquiring the image comprises: illuminating the sample with light of a first color and while doing so acquiring a first monochrome image component of the sample; subsequently illuminating the sample with light of a second different color and while doing so acquiring a second monochrome image component of the sample; and combining the first and second image components to produce a color image.
5. The method of claim 4, and further comprising: illuminating the sample with light of a third different color and while doing so acquiring a third monochrome image component of the sample; and combining the third image components with the first and second image components to produce the color image.
6. The method of claim 1, and further comprising scanning the platform and the optical system relative to each other.
7. The method of claim 6, wherein acquiring the image patches is performed while the platform and the optical system are moving relative to each other.
8. The method of claim 1, wherein focal heights of the image patches are adjusted by moving the optical element.
9. An apparatus for analysis of a sample of biological tissue cells on a platform, the apparatus comprising: an optical system including a camera and an objective optical element disposed between the camera and the platform, the optical system configured to: notionally divide the sample up into an array of portions; determine a measure of the height of the platform on which each portion of the sample is disposed; and acquire from light passing through the platform and the sample a set of image patches of the portions of the sample with the focal height of each image patch being adjusted in dependence upon the respective measure of the height of the platform; and a computer system configured to: stitch the image patches together to form an image of the sample; and use the image to assess characteristics of the sample.
10. The apparatus of claim 9, wherein the optical system is configured to measure the height of the platform by: projecting a beam of light into the objective optical element substantially parallel to and offset from the optical axis so that: the beam is projected from the objective optical element to the platform at an inclined angle to the platform so that the beam is reflected; and the reflected beam is projected back into the objective optical element so that the reflected beam is projected from the objective optical element to the camera; using the camera to acquire an image of the reflected beam; and detecting the position of the reflected beam in the image.
11. The apparatus of claim 9, wherein the image patches for the image are of overlapping portions of the sample, and the computer system is configured to blend the image patches together in the areas of overlap.
12. The apparatus of claim 11, wherein the platform comprises an under surface to reflect the reflected beam, and an upper surface to reflect a second beam that is projected back into the objective optical element to also project the second beam from the objective optical element to the camera; and wherein the computer system is further configured to choose whether to determine the measure of the height of the platform on which each portion of the sample is disposed from the reflected beam or the second reflected beam.
13. The apparatus of claim 9, wherein the platform and the optical system are movable relative to each other.
14. The apparatus of claim 9, wherein the optical system further comprises: a light source, a collecting lens, a collimating lens, a first aperture, a mirror, a first focusing lens, a second aperture, and a condenser lens aligned in order on an optical axis on a first side of the sample to project light from the source onto the sample, and a movable mechanical slide with the objective optical element mounted thereon, a beam splitter, a second focusing lens, and the camera aligned in order on the optical axis on a second side of the sample, the optical system configured to acquire the image of the sample from light passing through the platform and the sample.
15. A non-transitory computer readable medium containing program instructions for a computer in the analysis of a sample of biological tissue cells disposed on a platform, the sample being a thin phase object and the platform being substantially transparent, the program instructions for causing the computer to: notionally divide the sample up into an array of portions; determine a measure of the height of the platform on which each portion of the sample is disposed; cause an optical system including a camera and an objective optical element disposed between the camera and the platform to acquire from light passing through the platform and the sample a set of image patches of the portions of the sample with the focal height of each image patch being adjusted in dependence upon the respective measure of the height of the platform; stitch the image patches together to form an image of the sample; and use the image to assess characteristics of the sample.
16. The non-transitory computer readable medium of claim 15, wherein the program instructions further cause the computer to: cause the optical system to obtain the image patches for the image as overlapping portions of the sample; and blend the image patches together in the areas of overlap during stitching.
17. The non-transitory computer readable medium of claim 15, wherein the program instructions further cause the computer to: cause the optical system to illuminate the sample with light of a first color and while doing so acquire a first monochrome image component of the sample; cause the optical system to subsequently illuminate the sample with light of a second different color and while doing so acquire a second monochrome image component of the sample; and combine the first and second image components to produce a color image.
18. The non-transitory computer readable medium of claim 17, wherein the program instructions further cause the computer to: cause the optical system to illuminate the sample with light of a third different color and while doing so acquire a third monochrome image component of the sample; and combine the third image components with the first and second image components to produce the color image.
19. The non-transitory computer readable medium of claim 15, wherein the program instructions further cause the computer to: cause the optical system to scan the platform and the optical system relative to each other.
20. The non-transitory computer readable medium of claim 19, wherein the program instructions further cause the computer to: cause the optical system to adjust focal heights of the image patches by moving the optical element.
Description
[0027] A specific embodiment of the present invention will now be described, purely by way of example, with reference to the accompanying drawings, in which:
[0028]
[0029]
[0030]
[0031]
[0032]
[0033]
[0034]
[0035]
[0036]
[0037]
[0038] As mentioned above, the invention results from a realization that two simplifications can be applied to the TIE when working with typical tissue cells, and that by working in the spatial domain, rather than the frequency domain, the phase information can be simply deduced. Consider a central focal plane at a particular position along a Z axis, and consider a pair of parallel focal planes equispaced a distance of f to either side of the central focal plane. Digital images h, I.sub.1, I.sub.2 are acquired of each of the focal planes. For a phase target, an image I.sub.0 of the central plane is simply the pixel-by-pixel mean of the acquired images, I.sub.0=(I.sub.1+I.sub.2)/2. A contrast matrix C can be calculated on a pixel-by-pixel basis such that C=(I.sub.1I.sub.2)/I.sub.0. The TIE equation for the wave field is
where: I is the intensity in the plane; the gradient operator in the place is denoted .sub.; k is the wave number of the radiation;
is the intensity derivative or the rate of change of intensity; is the phase; and z is the direction in which the light wave travels.
[0039] Two assumptions can be made about the object being images, namely: (1) it is a thin phase object meaning that light can pass through it and has its phase changed only slightly; and (2) its density is not too high. With these assumptions, the TIE equation is simplified from
in which can be expressed as
where: q is the pixel size for the optical system, D is the depth of field for the optical system; is the typical wavelength of the optical system; (CN) is the 2-D convolution of the contrast matrix C and a distance index matrix N.
[0040] Referring to
[0041] Referring in particular to
[0042] In addition to being able to illuminate the sample and acquire images thereof, the optical system is also arranged to enable a determination to be made of the height of the portion of the floor 16 of the container 12 which is currently in the view of the camera 62 so that the focus of the image acquired by the camera 62 can be appropriately set. Specifically, a laser source 64 is disposed behind the beam splitter 59 and projects a laser beam 66 through the beam splitter 59 and the objective lens 50 so that it impinges at an inclined angle on the container 12 adjacent the optical axis 34. A first reflected beam 68 is produced at the underside of the container 12 and is reflected back through the objective lens 50 and passes into the camera 62 where, as shown in
[0043] The computer 28 has a user interface 78 including, for example, a monitor, keyboard and mouse, and can produce an output 80, for example, to the monitor and/or to a printer and/or network connection.
[0044] The computer 28 is programmed to cause the apparatus 10 of
[0045] The first step 82 in the method of
[0046] The camera 62 captures an image of only a small portion of the container 12 and its contents with each operation of the camera 62. These small images portions will be referred to as patches. In order to produce an image of the whole container 12 and its contents, a rectangular array of rows and columns of patches is acquired, and the patches are subsequently stitched together. The patches in a particular row are acquired with the same Y position of the XY stage, and for simplicity the Y positions will be taken to be integer numbers between and including 1 and Y.sub.MAX. Similarly, the patches in a particular column are acquired with the same X position of the XY stage, and for simplicity the X positions will be taken to be integer numbers between and including 1 and X.sub.MAX.
[0047] The aim is to acquire a pair of patches P.sub.1(X,Y), P.sub.2(X,Y) for each (X,Y) position, with each first patch P.sub.1(X, Y) having a first focal height relative to the upper surface of the floor 16 at that (X,Y) position, and with each second patch P.sub.2(X, Y) having a second focal height relative to the upper surface of the floor 16 at that (X,Y) position. However, if the height of the upper surface of the floor 16 cannot be ascertained at a particular (X,Y) position (for example because of the presence of a dividing wall 18 between adjacent compartments 14), the focal heights are determined relative to the lower surface of the floor 16 at that (X, Y) position. As mentioned above, the upper and lower surfaces of the floor 16 of the container 12 are not necessarily planar.
[0048] In order to achieve the above aim, for each row of patches from Y=1 to Y=Y.sub.MAX, the apparatus 10 firstly scans the row in order to determine the focusing that should be employed for each patch in that row, and then scans the row again in order to acquire the patches. Therefore, referring again to
[0049] After the XY stage 20 has been moved to the appropriate row in step 110, then as shown in
[0050] As mentioned above, after a laser scan has been performed for each row Y in step 112 of
[0051] It will be appreciated from the above that, if the positions Z(X,Y) are calculated based on the positions of the second laser spot (in step 132), the patches P.sub.1(X,Y) are captured with a first focal height F.sub.1=Fo+f above the upper surface of the floor 16 of the container 12 at the position (X,Y) and the patches P.sub.2(X,Y) are captured with a first focal height F.sub.2=F.sub.0f above the upper surface of the floor 16 of the container 12 at the position (X,Y), despite variations in the height of the upper surface of the floor 16 of the container 12. If the positions Z(X,Y) are calculated based on the positions of the first laser spot (in step 138), the same holds true, provided that the thickness of the floor 16 of the container 12 is uniform.
[0052] Referring back to
where q is the pixel size, D is the depth of field for the optical system; is the typical wavelength of the optical system. For example, in a system having a pixel size q of 2 m, a depth of field D of 12.8 m, a focal height offset f of 12.8 and a typical wavelength of 550 nm, the phase image is given by 2(CN).
[0053] The distance index is a matrix having odd numbers of rows and columns The number of elements in the distance index matrix is of the same order as the number of pixels occupied by a typical cell in the images I.sub.1, I.sub.2. In an example of the distance index matrix, the value of each of its elements, other than its center element, is inversely proportional to the distance between that element and the central element, and the value of the central element is zero. For example, a 55 distance index matrix N may be used as follows:
[0054] The technique of 2-D convolution of matrices is well-known to those skilled in the art of image processing. For further information, reference is directed to John C. Russ, The Image Processing Handbook, ISBN 978-1-4398-4045-0.
[0055] The phase matrix illustrates the phase change in the light as it passes through the container 12 and therefore illustrates the thickness of the cells in the sample in the container 12.
[0056] In addition to producing the phase matrix , in step 178 the computer 28 also computes a high contrast image H, which is easy for the human eye to interpret. Each pixel H(X, Y) in the high contrast image H is given by H(X,Y)=I.sub.0+{A.[I.sub.1(X,Y)I.sub.2(X,Y)]/2}, where A is an arbitrary coefficient.
[0057] The phase image and the high contrast H may be output from the system in step 178 and/or they may be further processed by the computer 28 is any desired manner.
[0058] Referring to
[0059]
[0060] Many modifications and developments may be made to the apparatus and method described above.
[0061] For example,
[0062] Also,
[0063] The light source 35 described above may be a broad band white light source, or it may be a narrow band colored light source. However, if a monochrome camera 62 is employed, the patches and images will be grey scale. In a modification to the apparatus, the light source 35 may be replaced by two or more light sources having different colors, such as the primary colors red, green and blue. Each patch acquisition may then be replaced by a set of two or more patch acquisitions, with a different colored light source being used for each patch in a set. The patches of each color may then be stitched together to produce an image of that color, and the images of the different colors may then be combined to produce a color image. For example, if three 8-bit grey scale images are produced while the object is illuminated with red, green and blue light, respectively, having pixel values R(X,Y), G(X, Y) and B(X,Y), respectively, at pixel position (X,Y), then a 24-bit RGB color image may be produced so that its pixel value I(X,Y) at each pixel position is given by I(X,Y)=[2.sup.16R(X,Y)]+[2.sup.8G(X,Y)]+[B(X,Y)]. This technique may be employed for acquiring the images at different focal heights, e.g. F.sub.1, F.sub.2, relative to the floor of the container 12, which may then be used for example in the production of a high contrast image in color. Alternatively, the technique may be employed to acquire a single image at a single focal height relative to the floor of the container 12.
[0064] In the initialization routine shown in
[0065] In the case where the camera 62 and optical system has a large enough field of view and sufficient resolution, each of the images I.sub.1, I.sub.2 may be acquired as a single image, rather than as an array of patches which are subsequently stitched together.
[0066] In addition to producing the two images I.sub.1, I.sub.2 having focal heights; offset by a distance f to either side of the central focal plane F.sub.0, other pairs of images may be acquired with different offsets and processed in a similar way to that described above. The multiple phase matrices and high contrast matrices may then be averaged or otherwise filtered.
[0067] It should be noted that the embodiment of the invention has been described above purely by way of example and that many other modifications and developments may be made thereto within the scope of the present invention.