Method of and apparatus for analysis of a sample of biological tissue cells
09600710 ยท 2017-03-21
Assignee
Inventors
Cpc classification
G02B21/0056
PHYSICS
G06V20/69
PHYSICS
G02B21/367
PHYSICS
International classification
G02B21/36
PHYSICS
G01N21/25
PHYSICS
Abstract
At least first and second digital images of the sample are acquired having different focal heights relative to a platform on which the cells are disposed. A contrast matrix is produced having elements computed in dependence upon the difference between the values of the corresponding pixels in the first and second images. A phase matrix is produced by convolution of the contrast matrix with a predetermined distance matrix. The phase matrix is used to assess characteristics of the sample, such as the presence of cells in the sample or the heights of cells in the sample.
Claims
1. A method of analysis of a sample of biological tissue cells disposed on a platform, the sample being a thin phase object and the platform being substantially transparent, the method comprising the steps of: acquiring at least first and second images of the sample from light passing through the platform and the sample, the images having different focal heights relative to the platform, and each of the acquired images comprising a matrix of pixels; computing a contrast matrix having elements corresponding to the pixels in the acquired images by setting the value of each element of the contrast matrix in dependence upon the difference between the values of the corresponding pixels in the first and second images; computing a phase matrix having elements corresponding to the elements in the contrast matrix by performing a convolution of the contrast matrix and a predetermined distance index matrix; and using the phase matrix to assess characteristics of the sample, wherein said predetermined distance index matrix comprises elements where each element, other than a center element, has a value that is inversely proportional to a distance between that image and the center element.
2. A method as claimed in claim 1, wherein: each element of the contrast matrix is computed so that its value is also dependent on the sum of the values of the corresponding pixels in the first and second images.
3. A method as claimed in claim 2, wherein: each element of the contrast matrix is computed so that its value is substantially proportional to the difference between the values of the corresponding pixels in the first and second images divided by the sum of those values.
4. A method as claimed in claim 1, wherein: each element of the phase matrix is computed so that its value is substantially proportional to a weighted average of the values of a respective predetermined set of elements in the contrast matrix neighbouring that one of the elements in the contrast matrix corresponding to the respective element of the phase matrix.
5. A method as claimed in claim 4, wherein: in computing the weighted average, each weighting is substantially inversely proportional to the spacing between the respective element and that one of the elements in the contrast matrix corresponding to the respective element of the phase matrix.
6. A method as claimed in claim 1, wherein: each element of the phase matrix is computed so that its value is substantially inversely proportional to the difference in focal heights of the first and second acquired images.
7. A method as claimed in claim 1, wherein: an optical system is employed including: a camera for acquiring the images; and an objective optical element disposed on an optical axis between the camera and the platform.
8. A method as claimed in claim 1, wherein: the step of acquiring each of the first and second images comprises the steps of: acquiring a set of image patches of an array of respective portions of the sample; and stitching the image patches together to form the respective image.
9. A method as claimed in claim 8, wherein: the image patches for each of the first and second images are of overlapping portions of the sample; and in the step of stitching together the image patches, the patches are blended together in the areas of overlap.
10. A method as claimed in claim 8, wherein: the set of image patches for the second image is staggered, in the plane of the images, with respect to the set of image patches for the first image.
11. A method as claimed in claim 8, wherein: the image patches are acquired alternately for the first and second images.
12. A method as claimed in claim 8, further including the steps of: determining a measure of the height of the platform on which each portion of the sample is disposed; and adjusting the focal heights of the image patches in dependence upon the respective measures of the height of the platform.
13. A method as claimed in claim 12, wherein: the measure of the height of the platform is determined by: projecting a beam of light at the platform at an inclined angle to the platform so that the beam is reflected; and detecting the reflected beam.
14. A method as claimed in claim 1, further including the steps of: an optical system is employed including: a camera for acquiring the images; and an objective optical element disposed on an optical axis between the camera and the platform; the step of acquiring each of the first and second images comprises the steps of: acquiring a set of image patches of an array of respective portions of the sample; and stitching the image patches together to form the respective image; and further including the step of: scanning the platform and the optical system relative to each other.
15. A method as claimed in claim 1, wherein: at least some of the image patches are acquired while the platform and optical system are moving relative to each other; and the sample is illuminated by a flashing light source synchronised with movement of the platform and optical system relative to each other.
16. A method as claimed in claim 15, wherein: the camera is responsive to a trigger signal to commence, after a delay, acquisition of such an image patch provided that the camera is not already acquiring an image patch; the light source is responsive to a trigger signal to produce a flash of light; and common trigger signals are provided in pairs to both the camera and the light source so that: the first trigger signal of each pair triggers the camera and the light source; and the second trigger signal of each pair triggers the light source while the camera is acquiring an image patch.
17. A method as claimed in claim 1, including: an optical system is employed including: a camera for acquiring the images; and an objective optical element disposed on an optical axis between the camera and the platform, wherein the step of acquiring each of the first and second comprises the steps of: acquiring a set of image patches of an array of respective portions of the sample; and stitching the image patches together to form the respective image, further including the steps of: determining a measure of the height of the platform on which each portion of the sample is disposed; and adjusting the focal heights of the image patches in dependence upon the respective measures of the height of the platform, the focal heights of the image patches are adjusted by moving the objective optical element.
18. A method as claimed in claim 17, wherein: the first-mentioned reflected beam is reflected by an under surface of the platform; a second beam is reflected by an upper surface of the platform and is projected back into the objective optical element so that the second reflected beam is also projected from the objective optical element to the camera; and further including the step of choosing whether to determine the measure of the height of the platform on which each portion of the sample is disposed from the first reflected beam or the second reflected beam.
19. A method as claimed in claim 1, wherein: an optical system is employed including: a camera for acquiring the images; and an objective optical element disposed on an optical axis between the camera and the platform, wherein the step of acquiring each of the first and second comprises the steps of: acquiring a set of image patches of an array of respective portions of the sample; and stitching the image patches together to form the respective image, further including the steps of: determining a measure of the height of the platform on which each portion of the sample is disposed; and adjusting the focal heights of the image patches in dependence upon the respective measures of the height of the platform, wherein: the measure of the height of the platform is determined by: projecting a beam of light at the platform at an inclined angle to the platform so that the beam is reflected; detecting the reflected beam; and the beam of light is projected into the objective optical element substantially parallel to and offset from the optical axis so that the beam is projected from the objective optical element to the platform at said inclined angle; the reflected beam is projected back into the objective optical element so that the reflected beam is projected from the objective optical element to the camera; and the reflected beam is detected by the camera.
20. A method as claimed in claim 1, wherein: the step of acquiring at least one of the first and second images includes the steps of: illuminating the sample with light of a first colour and while doing so acquiring a first monochrome image component of the sample; subsequently illuminating the sample with light of a second different colour and while doing so acquiring a second monochrome image component of the sample; and combining the first and second image components to produce a colour image.
21. A method as claimed in claim 20, further including: illuminating the sample with light of a third different colour and while doing so acquiring a third monochrome image component of the sample; and combining the third image components with the first and second image components to produce the colour image.
22. An apparatus for analysis of a sample of biological tissue cells that is configured to perform the steps of the method recited in claim 20, the apparatus comprising: an optical system including a camera and an objective optical element; a platform on which the sample of biological tissue cells can be disposed; a motor system for moving the platform and the objective optical element relative to each other in three orthogonal directions; and a controller for causing the optical system and motor system to perform the method steps.
23. An apparatus as claimed in claim 22, wherein: the controller includes a programmable computer.
24. An apparatus for analysis of a sample of biological tissue cells on a platform, the apparatus comprising: an optical system comprising a light source, a collecting lens, a collimating lens, a first aperture, a mirror, a first focusing lens, a second aperture, and a condenser lens aligned in order on an optical axis on a first side of the sample to project light from the source onto the sample, and a movable mechanical slide with an objective lens mounted thereon, a beam splitter, a second focusing lens, and a camera aligned in order on the optical axis on a second side of the sample, the optical system configured to acquire at least first and second images of the sample from light passing through the platform and the sample, the images having different focal heights relative to the platform, and each of the acquired images comprising a matrix of pixels; and a computer system configured to: compute a contrast matrix having elements corresponding to the pixels in the acquired images by setting the value of each element of the contrast matrix in dependence upon the difference between the values of the corresponding pixels in the first and second images; compute a phase matrix having elements corresponding to the elements in the contrast matrix by performing a convolution of the contrast matrix and a predetermined distance index matrix; and use the phase matrix to assess characteristics of the sample, wherein said predetermined distance index matrix comprises elements where each element, other than a center element, has a value that is inversely proportional to a distance between that image and the center element.
25. A non-transitory computer readable medium containing program instructions for a computer in the analysis of a sample of biological tissue cells disposed on a platform, the sample being a thin phase object and the platform being substantially transparent, the program instructions for causing the computer to: take at least first and second image matrices of values of pixels representing images of the sample having different focal heights relative to the platform; compute a contrast matrix having elements corresponding to the pixels in the image matrices, by setting the value of each element of the contrast matrix in dependence upon the difference between the values of the corresponding pixels in the first and second image matrices; compute a phase matrix having elements corresponding to the elements in the contrast matrix, by performing a convolution of the contrast matrix and a predetermined distance index matrix; and compute an assessment of characteristics of the sample from the phase matrix, wherein the predetermined distance index matrix comprises elements where each element, other than a center element, has a value that is inversely proportional to a distance between that image and the center element.
Description
(1) A specific embodiment of the present invention will now be described, purely by way of example, with reference to the accompanying drawings, in which:
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12) As mentioned above, the invention results from a realisation that two simplifications can be applied to the TIE when working with typical tissue cells, and that by working in the spatial domain, rather than the frequency domain, the phase information can be simply deduced. Consider a central focal plane at a particular position along a Z axis, and consider a pair of parallel focal planes equispaced a distance of f to either side of the central focal plane. Digital images I.sub.1,I.sub.2 are acquired of each of the focal planes. For a phase target, an image I.sub.0of the central plane is simply the pixel-by-pixel mean of the acquired images, I.sub.0=(I.sub.1+I.sub.2)/2. A contrast matrix C can be calculated on a pixel-by-pixel basis such that C=(I.sub.1I.sub.2)/I.sub.0. The TIE equation for the wave field is
(13)
where: I is the intensity in the plane; the gradient operator in the place is denoted .sub.; k is the wave number of the radiation;
(14)
is the intensity derivative or the rate of change of intensity; is the phase; and z is the direction in which the light wave travels.
(15) Two assumptions can be made about the object being images, namely: (1) it is a thin phase object meaning that light can pass through it and has its phase changed only slightly; and (2) its density is not too high. With these assumptions, the TIE equation is simplified from
(16)
in which can be expressed as
(17)
where: q is the pixel size for the optical system, D is the depth of field for the optical system; is the typical wavelength of the optical system; (CN) is the 2-D convolution of the contrast matrix C and a distance index matrix N.
(18) Referring to
(19) Referring in particular to
(20) In addition to being able to illuminate the sample and acquire images thereof, the optical system is also arranged to enable a determination to be made of the height of the portion of the floor 16 of the container 12 which is currently in the view of the camera 62 so that the focus of the image acquired by the camera 62 can be appropriately set. Specifically, a laser source 64 is disposed behind the beam splitter 59 and projects a laser beam 66 through the beam splitter 59 and the objective lens 50 so that it impinges at an inclined angle on the container 12 adjacent the optical axis 34. A first reflected beam 68 is produced at the underside of the container 12 and is reflected back through the objective lens 50 and passes into the camera 62 where, as shown in
(21) The computer 28 has a user interface 78 including, for example, a monitor, keyboard and mouse, and can produce an output 80, for example, to the monitor and/or to a printer and/or network connection.
(22) The computer 28 is programmed to cause the apparatus 10 of
(23) The first step 82 in the method of
(24) The camera 62 captures an image of only a small portion of the container 12 and its contents with each operation of the camera 62. These small images portions will be referred to as patches. In order to produce an image of the whole container 12 and its contents, a rectangular array of rows and columns of patches is acquired, and the patches are subsequently stitched together. The patches in a particular row are acquired with the same Y position of the XY stage, and for simplicity the Y positions will be taken to be integer numbers between and including 1 and Y.sub.MAX. Similarly, the patches in a particular column are acquired with the same X position of the XY stage, and for simplicity the X positions will be taken to be integer numbers between and including 1 and X.sub.MAX.
(25) The aim is to acquire a pair of patches P.sub.1(X,Y), P.sub.2(X,Y) for each (X,Y) position, with each first patch P.sub.1(X,Y) having a first focal height relative to the upper surface of the floor 16 at that (X,Y) position, and with each second patch P.sub.2(X,Y) having a second focal height relative to the upper surface of the floor 16 at that (X,Y) position. However, if the height of the upper surface of the floor 16 cannot be ascertained at a particular (X,Y) position (for example because of the presence of a dividing wall 18 between adjacent compartments 14), the focal heights are determined relative to the lower surface of the floor 16 at that (X,Y) position. As mentioned above, the upper and lower surfaces of the floor 16 of the container 12 are not necessarily planar.
(26) In order to achieve the above aim, for each row of patches from Y=1 to Y=Y.sub.MAX, the apparatus 10 firstly scans the row in order to determine the focussing that should be employed for each patch in that row, and then scans the row again in order to acquire the patches. Therefore, referring again to
(27) After the XY stage 20 has been moved to the appropriate row in step 110, then as shown in
(28) As mentioned above, after a laser scan has been performed for each row Y in step 112 of
(29) It will be appreciated from the above that, if the positions Z(X,Y) are calculated based on the positions of the second laser spot (in step 132), the patches P.sub.1(X,Y) are captured with a first focal height F.sub.1=F.sub.0+f above the upper surface of the floor 16 of the container 12 at the position (X,Y) and the patches P.sub.2(X,Y) are captured with a first focal height F.sub.2=F.sub.0f above the upper surface of the floor 16 of the container 12 at the position (X,Y), despite variations in the height of the upper surface of the floor 16 of the container 12. If the positions Z(X,Y) are calculated based on the positions of the first laser spot (in step 138), the same holds true, provided that the thickness of the floor 16 of the container 12 is uniform.
(30) Referring back to
(31)
where q is the pixel size, D is the depth of field for the optical system; is the typical wavelength of the optical system. For example, in a system having a pixel size q of 2 m, a depth of field D of 12.8 m, a focal height offset f of 12.8 m and a typical wavelength of 550 nm, the phase image is given by 2(CN).
(32) The distance index is a matrix having odd numbers of rows and columns. The number of elements in the distance index matrix is of the same order as the number of pixels occupied by a typical cell in the images I.sub.1,I.sub.2. In an example of the distance index matrix, the value of each of its elements, other than its centre element, is inversely proportional to the distance between that element and the central element, and the value of the central element is zero. For example, a 55 distance index matrix N may be used as follows:
(33)
(34) The technique of 2-D convolution of matrices is well-known to those skilled in the art of image processing. For further information, reference is directed to John C. Russ, The Image Processing Handbook, ISBN 978-1-4398-4045-0.
(35) The phase matrix illustrates the phase change in the light as it passes through the container 12 and therefore illustrates the thickness of the cells in the sample in the container 12.
(36) In addition to producing the phase matrix , in step 178 the computer 28 also computes a high contrast image H, which is easy for the human eye to interpret. Each pixel H(X,Y) in the high contrast image H is given by H(X,Y)=I.sub.0+{A.[I.sub.1(X,Y)I.sub.2(X,Y)]/2}, where A is an arbitrary coefficient.
(37) The phase image and the high contrast H may be output from the system in step 178 and/or they may be further processed by the computer 28 is any desired manner.
(38) Referring to
(39)
(40) Many modifications and developments may be made to the apparatus and method described above.
(41) For example,
(42) Also,
(43) The light source 35 described above may be a broad band white light source, or it may be a narrow band coloured light source. However, if a monochrome camera 62 is employed, the patches and images will be grey scale. In a modification to the apparatus, the light source 35 may be replaced by two or more light sources having different colours, such as the primary colours red, green and blue. Each patch acquisition may then be replaced by a set of two or more patch acquisitions, with a different coloured light source being used for each patch in a set. The patches of each colour may then be stitched together to produce an image of that colour, and the images of the different colours may then be combined to produce a colour image. For example, if three 8-bit grey scale images are produced while the object is illuminated with red, green and blue light, respectively, having pixel values R(X,Y), G(X,Y) and B(X,Y), respectively, at pixel position (X,Y), then a 24-bit RGB colour image may be produced so that its pixel value I(X,Y) at each pixel position is given by I(X,Y)=[2.sup.16R(X,Y)]+[2.sup.8G(X,Y)]+[B(X,Y)]. This technique may be employed for acquiring the images at different focal heights, e.g. F.sub.1, F.sub.2, relative to the floor of the container 12, which may then be used for example in the production of a high contrast image in colour. Alternatively, the technique may be employed to acquire a single image at a single focal height relative to the floor of the container 12.
(44) In the initialisation routine shown in
(45) In the case where the camera 62 and optical system has a large enough field of view and sufficient resolution, each of the images I.sub.1,I.sub.2 may be acquired as a single image, rather than as an array of patches which are subsequently stitched together.
(46) In addition to producing the two images I.sub.1,I.sub.2 having focal heights offset by a distance f to either side of the central focal plane F.sub.0, other pairs of images may be acquired with different offsets and processed in a similar way to that described above. The multiple phase matrices and high contrast matrices may then be averaged or otherwise filtered.
(47) It should be noted that the embodiment of the invention has been described above purely by way of example and that many other modifications and developments may be made thereto within the scope of the present invention.