Imaging a well of a microplate

Abstract

An imaging system and method are provided in which a well of a microplate 050 is imaged by a camera 110 comprising magnification optics 112. The camera is controlled to acquire a series of images of the well with different exposures. The series of images comprise a base image with a base exposure and at least one further image with a larger exposure than the base exposure. The series of images are then merged into an output image which comprises in a center region of the well image content from the base image and at a peripheral region of the well image content from the at least one further image. Advantageously, the output image may allow for better assaying or analysis of the samples in the well than any of the individual images.

Claims

1. An imaging system for imaging individual wells of a microplate, the imaging system comprising: a camera comprising magnification optics to establish a field of view encompassing a well when the well is positioned within the field of view of the camera; a camera interface including wired or wireless data communication circuitry configured to provide control data to the camera to control the imaging by the camera, and to receive image data acquired by the camera; a light source for illuminating the well; a memory comprising instruction data representing a set of instructions; and a processor configured to communicate with the camera interface and the memory and to execute the set of instructions, wherein the set of instructions, when executed by the processor, cause the processor to: control the camera to acquire a series of images of the well with different exposures, the series of images comprising a base image with a base exposure and at least one further image with a larger exposure than the base exposure, and merge the series of images into an output image which comprises, in a center region of the well, image content from the center region in the base image, and which comprises, in a peripheral region of the well, image content from the peripheral region in the at least one further image, wherein the set of instructions, when executed by the processor, cause the processor to merge the series of images into the output image by selecting a different set of frequency components from each of the series of images for inclusion in the output image.

2. The imaging system according to claim 1, wherein the set of instructions, when executed by the processor, cause the processor to select the base exposure to minimize clipping in the center region of the well in the base image.

3. The imaging system according to claim 1, wherein the selecting the different set of frequency components comprises: selecting at least lower frequency components from the base image; and selecting higher frequency components, while omitting selecting the lower frequency components, from the further image.

4. The imaging system according to claim 3, wherein the set of instructions, when executed by the processor, cause the processor to select the different set of frequency components from each of the series of images by: generating a Laplacian of Gaussian pyramid representation of the base image; and generating a Laplacian of Gaussian pyramid representation of the further image which comprises fewer levels than the Laplacian of Gaussian pyramid representation of the base image and thereby omits the lower frequency components.

5. The imaging system according to claim 4, wherein the set of instructions, when executed by the processor, cause the processor to generate the output image by: calculating an intermediate base output image from the Laplacian of Gaussian pyramid representation of the base image; calculating an intermediate further output image from the Laplacian of Gaussian pyramid representation of the further image; and merging the intermediate base output image and the intermediate further output image.

6. The imaging system according to claim 5, wherein the merging the intermediate base output image and the intermediate further output image comprises summing or averaging the intermediate base output image and the intermediate further output image.

7. The imaging system according to claim 4, wherein the set of instructions, when executed by the processor, cause the processor to generate the output image by: combining the Laplacian of Gaussian pyramid representation of the base image and the Laplacian of Gaussian pyramid representation of the further image to obtain a combined Laplacian of Gaussian pyramid representation; and deriving the output image from the combined Laplacian of Gaussian pyramid representation.

8. The imaging system according to claim 1, wherein the set of instructions, when executed by the processor, cause the processor to: control the camera to acquire a first series of images of the well having a first focus position with respect to the well; control the camera to acquire a second series of images of the well having a second focus position with respect to the well; merge the first series of images into a first output image; merge the second series of images into a second output image; and generate a bright field output image as a sum or average of the first output image and the second output image.

9. The imaging system according to claim 8, wherein the set of instructions, when executed by the processor, cause the processor to generate a phase difference output image as a difference of the first output image and the second output image.

10. The imaging system according to claim 1, further comprising a light source control interface for controlling the light source.

11. The imaging system according to claim 1, wherein the imaging system is a microplate reader.

12. The imaging system according to claim 1, wherein the microplate is a 96-well microplate.

13. A method of imaging individual wells of a microplate with an imaging system, the method comprising: providing an imaging system, the imaging system comprising: a camera comprising magnification optics to establish a field of view encompassing a well when the well is positioned within the field of view of the camera; a camera interface including wired or wireless data communication circuitry configured to provide control data to the camera to control the imaging by the camera, and to receive image data acquired by the camera; and a light source for illuminating the well; controlling the camera to acquire a series of images of the well with different exposures, the series of images comprising a base image with a base exposure and at least one further image with a larger exposure than the base exposure; and merging the series of images into an output image which comprises, in a center region of the well, image content from the center region in the base image, and which comprises, in a peripheral region of the well, image content from the peripheral region in the at least one further image, wherein merging the series of images includes merging the series of images into the output image by selecting a different set of frequency components from each of the series of images for inclusion in the output image.

14. A non-transitory computer-readable medium comprising a computer program executable by a processor system of an imaging system comprising a camera comprising magnification optics to establish a field of view encompassing a well when the well is positioned within the field of view of the camera, a camera interface including wired or wireless data communication circuitry configured to provide control data to the camera to control the imaging by the camera and to receive image data acquired by the camera, and a light source for illuminating the well, the computer program comprising instructions for causing the processor system to perform functionality comprising: controlling the camera to acquire a series of images of the well with different exposures, the series of images comprising a base image with a base exposure and at least one further image with a larger exposure than the base exposure; and merging the series of images into an output image which comprises, in a center region of the well, image content from the center region in the base image, and which comprises, in a peripheral region of the well, image content from the peripheral region in the at least one further image, wherein merging the series of images includes merging the series of images into the output image by selecting a different set of frequency components from each of the series of images for inclusion in the output image.

15. The method of claim 13 further comprising: selecting the base exposure to minimize clipping in the center region of the well in the base image.

16. The method of claim 13, wherein the selecting the different set of frequency components comprises: selecting at least lower frequency components from the base image; and selecting higher frequency components, while omitting selecting the lower frequency components, from the further image.

17. The method of claim 16, wherein the selecting the different set of frequency components comprises: generating a Laplacian of Gaussian pyramid representation of the base image; and generating a Laplacian of Gaussian pyramid representation of the further image which comprises fewer levels than the Laplacian of Gaussian pyramid representation of the base image and thereby omits the lower frequency components.

18. The method of claim 17 further comprising generating the output image by: calculating an intermediate base output image from the Laplacian of Gaussian pyramid representation of the base image; calculating an intermediate further output image from the Laplacian of Gaussian pyramid representation of the further image; and merging the intermediate base output image and the intermediate further output image.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) These and other aspects of the invention are apparent from and will be elucidated with reference to the embodiments described hereinafter. In the drawings,

(2) FIG. 1 shows an imaging system for imaging individual wells of a microplate, which is shown to comprise a camera and a processor subsystem;

(3) FIG. 2A shows an image of a well acquired with a base exposure setting;

(4) FIG. 2B indicates a well-exposed central region in the image of FIG. 2A;

(5) FIG. 2C shows an image of the well acquired with a larger exposure setting;

(6) FIG. 2D indicates a well-exposed peripheral region in the image of FIG. 2C;

(7) FIG. 3A shows an intermediate base output image which may be obtained by generating a Laplacian of Gaussian pyramid representation of the base image and converting the representation into an output image;

(8) FIG. 3B shows an intermediate further output image which may be obtained by generating a Laplacian of Gaussian pyramid representation of the further image and converting the representation into an output image;

(9) FIG. 4A shows a bright field output image;

(10) FIG. 4B shows a phase difference output image;

(11) FIG. 5 shows a method of imaging individual wells of a microplate; and

(12) FIG. 6 shows a computer readable medium comprising non-transitory data, with said data representing a computer program for performing the method.

(13) It should be noted that items which have the same reference numbers in different figures, have the same structural features and the same functions, or are the same signals. Where the function and/or structure of such an item has been explained, there is no necessity for repeated explanation thereof in the detailed description.

LIST OF REFERENCE AND ABBREVIATIONS

(14) The following list of references and abbreviations is provided for facilitating the interpretation of the drawings and shall not be construed as limiting the claims.

(15) 050 microplate

(16) 060 well

(17) 100 imaging system

(18) 110 camera

(19) 112 magnification optics

(20) 114 field of view

(21) 120 light source

(22) 122 emitted light

(23) 130 processor subsystem

(24) 140 camera interface

(25) 142 camera control data

(26) 144 image data

(27) 146 internal data communication

(28) 150 light source control interface

(29) 152 light source control data

(30) 154 internal data communication

(31) 160 processor

(32) 170 memory

(33) 172 memory data communication

(34) 200 base image

(35) 202 central portion

(36) 210 further image

(37) 212 peripheral portion

(38) 300 intermediate base output image

(39) 310 intermediate further output image

(40) 400 bright field output image

(41) 410 phase contrast output image

(42) 500 method for imaging wells with imaging system

(43) 510 controlling camera to acquire image series

(44) 520 merging image series into output image

(45) 600 computer readable medium

(46) 610 non-transitory data

DETAILED DESCRIPTION OF EMBODIMENTS

(47) FIG. 1 shows an example of an imaging system 100 for imaging individual wells of a microplate 050. The imaging system 100 is shown to comprise a camera 110, which may be of any suitable type for imaging samples in a microplate. For example, the camera may be a still image or video camera equipped with a CMOS or CCD-type sensor. The camera is shown to comprise magnification optics 112, thereby establishing a certain field of view 114. The field of view 114 is chosen such that when a microplate 050 is brought into an imaging position, the imaging system can acquire an image of substantially an entire well of the microplate 050. For that purpose, the type of magnification optics, the distance of the microplate 050 to the camera and optical parameters may be chosen such that the well fits within the field of view of the camera. For example, the imaging system 100 may comprise a microplate holder which puts the microplate 050 at a predetermined or determinable distance from the camera 110. The selection of such design parameters is within reach of the skilled person. In a specific example, the camera may be a CMOS camera having a 5 Megapixel resolution and sensor size, the optics may comprise a tubus 0.5 lens and a 2 magnification objective, e.g., with an numerical aperture (NA) of 0.08, a working distance of 6.20 mm, a Field of View of 13.25 mm and an XY resolution of 4.19 m. Moreover, the microplate 050 may be an ANSI-size 96-well microplate.

(48) The imaging system 100 is further shown to comprise a light source 120 for illuminating the well. For example, the light source 120 may be a light emitting diode (LED) or an arrangement of LEDs, or in general may be embodied by any another suitable type of light source. In the example of FIG. 1, the light source 120 is shown to illuminate the well from above, whereas the camera 110 is arranged below the well. Alternatively, the placement of camera 110 and light source 120 may be reversed, in that the light source 120 may illuminate the well from below, and the camera 110 may be arranged above the well. It is noted that, in general, the light 122 captured by the camera may be light 122 which is transmitted through the sample in the well due to the camera 110 and the light source 120 being positioned at opposite sides of the microplate. To enable such transmission of light 122 through the sample, the well may have a transparent bottom portion, and/or the entire microplate 050 may be transparent. Alternatively, the camera 110 and the light source 120 may be positioned at a same side of the microplate, in which case the light 122 may be reflected light. Although not shown in FIG. 1, one or more lenses may be used to image the light source 120 at infinity. As a result, the light emitted by the light source 120 and incident on the microplate 050 may form parallel rays of light, e.g., a parallel illumination beam.

(49) The imaging system 100 is further shown to comprise a processor subsystem 130, which may comprise a camera interface 140, a memory 170 and a processor 160, with the processor 160 communicating with the camera interface 140 via data communication 146 and with the memory 170 via data communication 172.

(50) The camera interface may be configured to provide control data 142 to the camera 110 to enable the processor 160 to control the imaging by the camera 110, and to enable the processor 160 to receive image data 144 acquired by the camera 110.

(51) The memory 170 may comprise instruction data representing a set of instructions, which when executed by the processor 160 during operation of the imaging system 100, cause the processor 160 to control the camera 110 to acquire a series of images of the well with different exposures. The series of images may comprise a base image with a base exposure and at least one further image with a larger exposure than the base exposure. The processor 160 may be further configured by the instruction data to merge the series of images into an output image which comprises in a center region of the well image content from the base image and at a peripheral region of the well image content from the at least one further image.

(52) Optionally, the processor subsystem 130 may comprise a light source control interface 150 for controlling the light source 120. The processor 160 may communicate with the light source control interface 150 via data communication 154.

(53) Various examples of the operation of the imaging system 100, including optional aspects thereof, will be further explained with reference to FIGS. 2A-4B.

(54) In general, the camera interface 140 may take any suitable form, including but not limited to USB, FireWire, Wi-Fi, and similar types of wired and wireless data communication interfaces. It is noted that the camera interface may comprise or consist of multiple interfaces, e.g., arranged in parallel and/or series. For example, the camera interface 140 may comprise a HDMI interface for receiving the image data from the camera, and an 12C interface for controlling the camera. Although not shown explicitly in FIG. 1, the camera 110 may comprise a corresponding camera interface.

(55) In some embodiments, the processor subsystem 130 may be implemented as a device, apparatus or system separate from the camera 110. For example, the processor subsystem 130 may be embodied by a workstation, a server, a personal computer, etc., which may be connected to the camera 110 via the camera interface 140. In a specific embodiment, the camera 110 may be part of a microplate reader, which may further include the light source 120 and a microplate transport (not shown in FIG. 1) and which may be connected to the device, apparatus or system embodying the processor subsystem 130 via one or more data communication interfaces, including but not limited to the camera interface 140. Alternatively, the camera 110 and the processor subsystem 130 may be part of a same device, apparatus or system, e.g., a microplate reader having one or more embedded processors capable of and configured for performing the operations described with reference to the processor 160.

(56) In general, the processor subsystem 130 may be embodied asor ina device, apparatus or system. The device, apparatus system may comprise one or more processors, such as microprocessors, which execute appropriate software. Software implementing the described operations of processor 160 may have been downloaded and/or stored in a corresponding memory 170 or memories, e.g., in volatile memory such as RAM or in non-volatile memory such as Flash. Alternatively, the processor 160 may be implemented in the device, apparatus or system in the form of programmable logic, e.g., as a Field-Programmable Gate Array (FPGA). The camera interface and the optional light source control interface may each be implemented by respective interfaces of the device, apparatus or system. In general, each unit of the processor subsystem may be implemented in the form of a circuit and/or in a distributed manner, e.g., involving different devices or apparatuses. For example, the distribution may be in accordance with a client-server model, e.g., using a server and a client workstation.

(57) FIG. 2A shows an image 200 of a well of a microplate which may be acquired with an imaging system as described in this specification. For example, the image 200 may be acquired of a single well of a 96-plate microplate with 2 magnification optics. The image 200 is acquired with a base exposure setting. Accordingly, the image 200 may be referred to as a base image 200. The base exposure setting may be a setting which avoids over-exposure. In a specific example, and as illustrated in FIG. 2B, the base exposure setting may be selected to well-expose a central region 202 of the well 060. Such exposure setting may be pre-determined, or dynamically determined using any known technique for selecting an exposure setting, e.g., as known per se in the technical field of photography and related technical fields. It can be seen in FIGS. 2A and 2B that the base image 200 shows image details such as cells in the central region 202, but is underexposed in the periphery of the well, causing image details to be poorly visible or not visible at all. Such underexposure may at least in part be caused by a meniscus formed by liquid in the well 060.

(58) The base image 200 may be part of a series of images which may be acquired by the imaging system with increasingly larger exposure settings. For example, the exposure may be incrementally increased, or each time increased by a same factor, e.g., a factor of two or four. Accordingly, the series of images comprises at least two images, e.g., the base image and a further image, but possibly more images.

(59) It is noted that the series of images may also be acquired while incrementally decreasing the exposure setting, in which case the base image may be the last image to be acquired, or in any other suitable acquisition order.

(60) FIGS. 2C and 2D show such a further image acquired with larger exposure setting. To emphasize the difference with respect to the base image of FIGS. 2A and 2B, the further image 210 of FIGS. 2C and 2D has been acquired with a 4 as large exposure setting, e.g., a 4 as slow shutter speed. It can be seen that the larger exposure setting causes over-exposure in the central region of the well 060, thereby causing image details to clip and thereby to be poorly visible or not visible at all. However, as also shown in FIG. 2D, a peripheral region 212 (denoted by a white dashed outer line and a black dashed inner line, thus being ring-shaped) which was underexposed in the base image is better exposed in the further image 210, thereby increasing the visibility of image detail such as cells in the peripheral region 212.

(61) Having acquired the base image and the further image, or in general a series of images with different exposure settings, the images may be merged into an output image which comprises in a center region of the well image content from the base image and at a peripheral region of the well image content from the further image(s). For that purpose, various techniques may be used. Generally speaking, the output image may be generated by, for a given region in the output image, selecting the image content of image(s) which provide a well-exposed imaging of the region. It will be appreciated that whether a particular region is well-exposed or not may be determined dynamically, e.g., using known techniques including exposure detection techniques or clipping detection techniques, or may be predetermined. In a specific example, clipped pixels may be detected in each of the images, and the output image may be generated as an average, median or other statistical function of the non-clipped pixels across the different images. Another example is that a fixed central region of the output image may be selected for inclusion from the base image, whereas surrounding ring-shaped region(s) may be selected for inclusion from the further image(s).

(62) In an embodiment, the series of images may be merged into an output image by selecting a different set of frequency components from each of the series of images for inclusion in the output image. For example, a Laplacian of Gaussian pyramid representation of the base image may be generated, as well as a Laplacian of Gaussian pyramid representation for each of the further images. Such a Laplacian of Gaussian pyramid representation may henceforth also be simply referred to as a LoG pyramid. The LoG pyramid of the further image(s) may comprise fewer levels than the LoG pyramid of the base image, thereby omitting lower frequency components from the representation. In case of more than one further image, each representation may progressively comprise fewer levels, thereby obtaining representations which progressively contain fewer low-frequency components. In a specific example, if the LoG pyramid of the base image is generated having N levels, the LoG pyramid of the further image may be generated having N1 levels and the LoG pyramid of yet a further image may be generated having N2 levels. Thereby, increasingly more low-frequencies may be omitted from the representations of the respective images.

(63) The LoG pyramids may then be combined to obtain an output image. This may be done in several ways. For example, said representations may be first combined, e.g., by averaging, summing or in any other way combining LoG pyramids per level, thereby obtaining a combined LoG pyramid which may be converted into an output image by known means, e.g., by summing across all levels and adding an offset. For example, the offset may be 0.5 in case the resulting image should have normalized intensity values between 0 and 1. Another example is that each pyramid may be individually converted into an intermediate output image, with the intermediate output images then being merged into a single output image, e.g., by averaging, summing or in any other way. The latter example is illustrated in FIGS. 3A and 3B: FIG. 3A shows the intermediate output image as generated from the LoG pyramid of the base image of FIGS. 2A and B2B and FIG. 3B shows the intermediate output image as generated from the LoG pyramid of the further image of FIGS. 2C and 2D. Both images may then be combined, e.g., by averaging or summing, which may yield in the case of averaging an output image similar to that shown in FIG. 4A.

(64) Instead of a Laplacian of Gaussian pyramid representation, also a Difference of Gaussian pyramid representation may be used, or any other suitable pyramidal representation which represents a frequency decomposition of an image.

(65) However, FIG. 4A shows a slight variation of such an output image, in that the output image 400 of FIG. 4A is obtained by the processor controlling the camera to acquire a first series of images of the well having a focus position above the bottom of the well (Z-plus), controlling the camera to acquire a second series of images of the well having a focus position below the bottom of the well (Z-minus), merging the first series of images into a first output image, merging the second series of images into a second output image, and generating a bright field output image 400 as a sum or (weighted) average of the first output image and the second output image. Effectively, output images obtained from image series acquired at different focal distances may be combined by summing or averaging, resulting in an output image 400 which is termed bright field output image. It is noted that the bottom of the well may refer to the boundary between well-bottom and liquid. Auto-focus may focus on this boundary. The above and below focus positions may then be determined as offsets with respect to the auto-focus position. Alternatively, the bright field output image 400 may be generated only as the first output image, e.g., from the Z-plus focus position.

(66) Moreover, as shown in FIG. 4B, additionally or alternatively a phase difference output image 410 may be generated as a difference of the first output image and the second output image. The bright field output image 400, the difference output image 410 and/or any of the other (intermediate) output images may be then displayed to a user, but also archived on a storage medium or further processed by a processing entity, such as the imaging system itself or another workstation, server, etc.

(67) It will be appreciated that FIGS. 2A-4B have been contrast- and brightness-enhanced for facilitating reproduction in a patent publication and may not be directly representative of the output images as generated by the imaging system. Of these figures, FIGS. 3A-4B have been enhanced with the largest gain value as otherwise their contents would not be (easily) legible when reproduced in a patent publication.

(68) It will be appreciated that any of the (intermediate) output images as described in this specification may in some embodiments comprise signed values. If such images are to be displayed on a display or in archived in a standard image format, it may be needed to add an offset to the signed values, e.g., of 128 if the display range is 8 bit (from 0 to 255). Moreover, a scaling factor may be applied before adding the offset to better use the dynamic range of the display or of the standard image format. For example, if an intermediate output image comprises signed values in the range of 0.5 to 0.5, a scaling by a factor of 256 may be applied after which an offset of 128 may be added. Such data conversion is within reach of the skilled person.

(69) It will be appreciated that the terms merging and combining may, when used in reference to generating an output image based on multiple input images, refer to a particular region in an output image being generated as a function of, or based on, the image content of one or more of the these input images at the same or similar region. A non-limiting example is a pixel-wise processing. As such, the terms are not to be understood as including simply showing the input images side-by-side, e.g., by image stitching, or the generating of entirely different types of output images.

(70) FIG. 5 shows a computer-implemented method for imaging individual wells of a microplate with an imaging system. The imaging system may be of a type as described in this specification, but may also be of a different type provided that it comprises a camera, a light source and a camera interface and allows the camera to be configured to acquire a series of images of a well and to obtain the image data.

(71) The method 500 comprises, in an operation titled CONTROLLING CAMERA TO ACQUIRE IMAGE SERIES, controlling 510 the camera to acquire a series of images of the well with different exposures, the series of images comprising a base image with a base exposure and at least one further image with a larger exposure than the base exposure. The method 500 further comprises, in an operation titled MERGING IMAGE SERIES INTO OUTPUT IMAGE, merging 520 the series of images into an output image which comprises in a center region of the well image content from the base image and at a peripheral region of the well image content from the at least one further image.

(72) The method 500 may be implemented on a processor system, e.g., on a computer as a computer implemented method, as dedicated hardware, or as a combination of both. As also illustrated in FIG. 6, instructions for the computer, e.g., executable code, may be stored on a computer readable medium 600, e.g., in the form of a series 610 of machine readable physical marks and/or as a series of elements having different electrical, e.g., magnetic, or optical properties or values. The executable code may be stored in a transitory or non-transitory manner. Examples of computer readable mediums include memory devices, optical storage devices, integrated circuits, servers, online software, etc. FIG. 6 shows an optical disc 600.

(73) Examples, embodiments or optional features, whether indicated as non-limiting or not, are not to be understood as limiting the invention as claimed.

(74) It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims.

(75) In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. Use of the verb comprise and its conjugations does not exclude the presence of elements or steps other than those stated in a claim. The article a or an preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the device claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.