Three-dimensional confocal microscopy apparatus and focal plane scanning and aberration correction unit
09835843 · 2017-12-05
Assignee
Inventors
Cpc classification
G02B21/365
PHYSICS
B33Y80/00
PERFORMING OPERATIONS; TRANSPORTING
G02B21/006
PHYSICS
G02B21/367
PHYSICS
International classification
G02B21/36
PHYSICS
Abstract
Provided is a 3-dimensional confocal microscopy apparatus which is manufactured by combining a confocal microscope and an optical tweezers technique, wherein a pair of lenses for focal plane displacement where one lens is movable in the optical axis direction is arranged between a fixed objective lens and a fluorescent light imaging camera, and the 3-dimensional confocal microscopy apparatus also includes a mean which corrects the aberration of a fluorescent confocal image obtained by the fluorescent imaging camera. Accordingly, it is possible to provide a 3-dimensional confocal microscopy apparatus which can acquire a 3-dimensional image of a specimen during a manipulation of the specimen using optical tweezers without affecting an optical trap.
Claims
1. A 3-dimensional confocal microscopy apparatus, comprising: a first laser for confocal imaging excitation; a 2-dimensional scanning optical system for scanning the first laser on a specimen; an objective lens for focusing the first laser on a fluorescent specimen; a camera for detecting a fluorescent confocal microscopy image based on fluorescent light emitted from the fluorescent specimen, and a computer for outputting a 3-dimensional structural image of the specimen, wherein: a pair of lenses for focal plane displacement is arranged between the objective lens and the camera, wherein one lens of the pair of lenses for focal plane displacement is an intermediate imaging lens which forms an intermediate image of the fluorescent confocal microscopy image of the specimen and another lens of the pair of lenses for focal plane displacement is an intermediate objective lens arranged in proximity to the intermediate imaging lens such that a focal point of the intermediate objective lens overlaps with a focal point of the intermediate imaging lens on the same optical axis, and at least one of the intermediate imaging lens and the intermediate objective lens being movable along an optical axis by a lens drive means, and an optical component is arranged between the pair of lenses for focal plane displacement and the objective lens, and the optical component changes the optical axis of the first laser from the pair of lenses for focal plane displacement and a second laser for optical tweezers which passes through the objective lens and is focused on the specimen, wherein there are no lenses between the objective lens and the specimen, and the computer has a processor configured to: calculate aberrations of the fluorescent confocal microscopy images detected by the camera, based on the relationship between degree of aberration of images and displacement of a focal plane, wherein positions of the focal plane are changed when at least one of the intermediate imaging lens and the intermediate objective lens moves, correct the detected fluorescent confocal microscopy images using the calculated aberrations, generate the 3-dimensional structural image from the corrected fluorescent confocal microscopy images, and output the 3-dimensional structural image of the specimen.
2. The 3-dimensional confocal microscopy apparatus according to claim 1, wherein the 2-dimensional scanning optical system comprises a line-scanning type scanning system.
3. The 3-dimensional confocal microscopy apparatus according to claim 1, wherein the 2-dimensional scanning optical system comprises a nipkow disk.
4. The 3-dimensional confocal microscopy apparatus according to claim 1, wherein the lens drive means for the pair of lenses for focal plane displacement comprises a piezoelectric element.
5. The 3-dimensional confocal microscopy apparatus according to claim 1, wherein the lens drive means comprises a magnetostrictive actuator.
6. The 3-dimensional confocal microscopy apparatus according to claim 1, wherein the lens drive means for the pair of lenses comprises a voice coil.
7. The 3-dimensional confocal microscopy apparatus according to claim 1, wherein the timing for driving the lens drive means for the pair of lenses and the timing for acquiring an image by a fluorescent imaging camera are adjusted so as to be synchronous with each other.
8. The 3-dimensional confocal microscopy apparatus according to claim 1, wherein: the processor comprises a graphics processing unit.
9. The 3-dimensional confocal microscopy apparatus according to claim 1, further comprising an imaging lens and a bright field imaging camera for imaging a 2-dimensional image of a specimen separated by the optical component.
10. A focal plane scanning and aberration correction unit to be incorporated into a 3-dimensional confocal microscopy apparatus, comprising: a pair of lenses for focal plane displacement, wherein one lens of the pair of lenses is an intermediate imaging lens which forms an intermediate image of a fluorescent confocal microscopy image of a specimen and another lens of the pair of lenses for focal plane displacement is an intermediate objective lens arranged in proximity to the intermediate imaging lens such that a focal point of the intermediate objective lens overlaps with a focal point of the intermediate imaging lens on the same optical axis, at least one of the intermediate imaging lens and the intermediate objective lens being movable along an optical axis by a lens drive means; a computer for outputting a 3-dimensional structural image of the specimen; and a display which displays the 3-dimensional structural image; wherein: the 3-dimensional confocal microscopy apparatus comprises a first laser; an objective lens for focusing the laser on a fluorescent specimen; and a camera for detecting a fluorescent confocal microscopy image based on fluorescent light emitted from the fluorescent specimen; the focal plane scanning and aberration correction unit is arranged between the objective lens and the camera; and an optical component is arranged between the pair of lenses for focal plane displacement and the objective lens, and the optical component changes the optical axis of the first laser from the pair of lenses for focal plane displacement and a second laser for optical tweezers which passes through the objective lens and is focused on the fluorescent specimen, wherein there are no lenses between the objective lens and the specimen, and the computer has a processor configured to: calculate aberrations of the fluorescent confocal microscopy images detected by the camera, based on the relationship between degree of aberration of images and displacement of a focal plane, wherein positions of the focal plane are changed when at least one of the intermediate imaging lens and the intermediate objective lens moves, correct the detected fluorescent confocal microscopy images using the calculated aberrations, generate the 3-dimensional structural image from the corrected fluorescent confocal microscopy images, and output the 3-dimensional structural image of the specimen.
Description
BRIEF DESCRIPTION OF DRAWINGS
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
REFERENCE SIGNS LIST
(9) 1: confocal imaging laser 2: objective lens 3: focal plane 4: optical tweezers laser 5: dichroic mirror 6: specimen 7: lower position 8: focal plane 9: lower position 10: excitation laser light source 11: confocal unit 12: objective lens 13: specimen 14: intermediate imaging lens 15: intermediate objective lens 16: confocal imaging camera 17: piezoelectric actuator 18: infrared laser light source 19: optical tweezers driving optical system 20: dichroic mirror 21: half mirror 22: bright field imaging camera 23: image processing process 24: cross-sectional image acquisition process 25: optical tweezers control process 26: columnar cross section 27: side surface 28: columnar cross section 29: side surface 30: body 31: fixed arm 32: movable arm 33: columnar projection portion 34: cell 35: minute structural body 36: movable arm 37: columnar projection portion 38: ring structure 39: cell which is object to be manipulated
DESCRIPTION OF EMBODIMENTS
(10) Next, a mode for carrying out the present invention including the best mode for carrying out the present invention is explained.
(11) [3-Dimensional Confocal Microscopy Apparatus]
(12) A 3-dimensional confocal microscopy apparatus according to the present invention includes the following constitutions (1) to (4).
(13) (1) A 3-dimensional confocal microscopy apparatus including: a light source for a first laser for confocal imaging excitation; a 2-dimensional scanning optical system for scanning the first laser on a specimen; an objective lens for focusing the first laser supplied from the light source on a fluorescent specimen which is an object to be observed; and a fluorescent light imaging camera for detecting a fluorescent confocal microscopy image based on a fluorescent light emitted from the fluorescent specimen, wherein
(14) (2) a pair of lenses for focal plane displacement is arranged between the objective lens and the fluorescent light imaging camera, the pair of lenses for focal plane displacement being constituted of an intermediate imaging lens which forms an intermediate image of the fluorescent confocal microscopy image of the specimen and an intermediate objective lens arranged such that a focal point of the intermediate objective lens overlaps with a focal point of the intermediate imaging lens on the same optical axis, and at least one lens out of these lenses being movable in an optical axis direction by a lens drive means,
(15) (3) the 3-dimensional confocal microscopy apparatus further includes a computer having an arithmetic operation means for acquiring a 3-dimensional structural image of the specimen by correcting aberration of the fluorescent confocal microscopy image caused by the movement of the lens of the pair of lenses for focal plane displacement in real time using the fluorescent confocal microscopy image acquired by the fluorescent light imaging camera and aberration correction data which is acquired in advance by performing provisional measurement of a sample specimen, and a means which displays a corrected image obtained by the arithmetic operation means, and
(16) (4) a beam splitter or a dichroic mirror is provided between the pair of lenses for focal plane displacement and the objective lens, and a second laser supplied from a light source for a second laser for optical tweezers passes through the objective lens via the beam splitter or the dichroic mirror and is focused on the specimen.
(17) In the three-dimensional confocal microscopy apparatus having the above-mentioned constitution, the objective lens, the intermediate imaging lens and the intermediate objective lens which constitute a pair of lenses for focal plane displacement and, further, other various lenses described later in examples may be constituted from a single lens body, and may be constituted of a plurality of lens bodies which are arranged parallel to each other on the same optical axis and are accommodated in a cylindrical housing.
(18) [First Constitutional Part of 3-dimensional Confocal Microscopy Apparatus]
(19) In the above-mentioned constitution (1), “2-dimensional scanning optical system” is an optical system for acquiring a thin cross-sectional image of a specimen in the vicinity of a specific depth by 2-dimensionally scanning a focal point of excitation laser beams on the specimen, and known various two-dimensional scanning optical system can be used. However, particularly, a nipkow disc or a line-scanning-type scanning system can be preferably used.
(20) The nipkow disc is, as is well known, a rotary disc on which a large number of pin holes are arranged in a vortex shape, for example, and can acquire a confocal image of a specimen with high-speed scanning performance by multi-beam scanning. However, the use of only such a rotary disc has drawbacks such as an insufficient optical amount or a so-called poor S/N ratio and hence, there has been also known a nipkow disc of an improved type where a micro lens array disc having a large number of micro lenses at positions corresponding to pin holes formed in a nipkow disc is formed into the integral structure with the nipkow disc, and both discs are synchronously rotated. Further, a dichroic mirror may be incorporated between the above-mentioned nipkow disc and micro lens array disc. The “nipkow disc” according to this invention includes all of discs of the above-mentioned types.
(21) The line-scanning-type scanning system is disclosed in patent literature 5 or the like and is known. The line-scanning-type scanning system is a scanning optical system where an excitation laser beam is scanned in a focused manner in a line shape and, for example, an illumination light focused in a line shape is irradiated to a specimen, the illumination light in a line shape is scanned in the predetermined direction on the specimen, and light generated in an illuminated region on the specimen is detected by an optical detector thus acquiring 2-dimensional image data of the specimen.
(22) The objective lens which focuses the first laser on the specimen which is an object to be observed is a fixed lens. However, a lens which is configured to be movable along an optical axis can achieve such an object when the lens is used without moving the lens at the time of using the lens and hence, such a lens can be also used as the objective lens of the present invention in this context.
(23) [Second Constitutional Part of 3-dimensional Confocal Microscopy Apparatus]
(24) In the above-mentioned constitution (2), an intermediate imaging lens and an intermediate objective lens which constitute the pairs of lenses for focal plane displacement are arranged such that a focal point of the intermediate imaging lens and a focal point of the intermediate objective lens overlap with each other on the same optical axis, and at least one lens is movable in the optical axis direction by the lens drive means. That is, either one of lenses is fixed and the other lens is made movable or both lenses are made movable. However, when both lenses are made movable, an accurate control of the drive means becomes difficult. Accordingly, it is preferable that either one of lenses is fixed and the other lens is made movable. It is more preferable that the fixed intermediate imaging lens is arranged on an objective lens sides, and the movable intermediate objective lens is arranged on a fluorescent imaging camera side.
(25) Although a kind of the drive means which moves the lens is not limited, it is preferable to adopt a means capable of moving a lens at a high speed under an arcuate control. As such a drive means, for example, any one of a piezoelectric element, a magnetostrictive actuator and a voice coil which are well known is named.
(26) Due to the reason set forth with respect to the fourth invention in the above-mentioned column “advantageous effects of the invention”, it is preferable that timing of driving the lens drive means in the pair of lenses for focal plane displacement and timing of acquiring an image by the fluorescent imaging camera are adjusted so as to be synchronous with each other.
(27) [Third Constitutional Part of 3-dimensional Confocal Microscopy Apparatus]
(28) The meaning of “arithmetic operation for acquiring a 3-dimensional structural image of the specimen” in the above-mentioned constitution (3) is explained in conjunction with
(29) When the intermediate objective lens is at the position shown in
(30) On the other hand, when the intermediate objective lens is moved to the upper position expressed as “Image 1” and approaches the intermediate imaging lens, the focal points of both lenses overlap with each other on a focal plane expressed as “Image 1′” and hence, the focal point of the objective lens is shifted to an upper position beyond the specimen stage. Accordingly, the magnification of the lens is enlarged so that an enlarged imaged image which is the body having a checkerboard pattern is transmitted to the fluorescent light imaging part. To the contrary, when the intermediate objective lens is moved to a lower position expressed as “Image 3” so that the intermediate objective lens is moved away from the intermediate imaging lens, the focal points of both lenses overlap with each other on a focal plane expressed as “Image 3′” and hence, the focal point of the objective lens is shifted to a lower position which does not reach the specimen stage. Accordingly, a contracted imaged image of the body having a checkerboard pattern is transmitted to the fluorescent imaging part.
(31) In this manner, when the intermediate objective lens is made to approach the intermediate imaging lens or to move away from the intermediate imaging lens to obtain a 3-dimensional structural image of the specimen, the deviation from designed focal lengths of both lenses is generated so that a fluorescent confocal microscopy image including aberration such as enlargement or contraction of the image is generated in the fluorescent imaging camera. Accordingly, it is necessary to acquire an accurate 3-dimensional structural image of the specimen by correcting such an aberration.
(32) Accordingly, by performing the provisional measurement of a sample specimen such as the above-mentioned body having a checkerboard pattern, for example, the relationship of the degree of aberration such as enlargement or contraction of a fluorescent confocal microscopy image corresponding to a moving direction and a moving distance of the intermediate objective lens is acquired by a computer in advance as aberration correction data, and the aberration of the fluorescent confocal microscopy image based on the displacement of a focal plane in the pair of lenses for focal plane displacement is calculated and is corrected in real time thus accurately acquiring a 3-dimensional structural image of the specimen.
(33) The above-mentioned calculation for acquiring the 3-dimensional structural image of the specimen can be preferably performed using a known GPU. The GPU is an auxiliary arithmetic operation unit or a dedicated circuit which is dedicated to 3-dimensional image processing in the computer.
(34) [Fourth Constitutional Part of 3-dimensional Confocal Microscopy Apparatus]
(35) The above-mentioned constitution (4) is provided for combining the optical tweezers technique with the confocal microscope. A second laser for optical tweezers is introduced onto an optical path between the pair of lenses for focal plane displacement and the objective lens in the confocal microscope via the beam splitter or the dichroic mirror and is focused on the specimen after passing through the objective lens.
(36) The beam splitter is an optical part which reflects some of incident light and divides a light flux by allowing other portion of the incident light to pass therethrough. The optical part where the intensity of reflection light and the intensity of transmitting light are approximately 1:1 is referred to as a half mirror, and this half mirror is also included in the beam splitter. The dichroic mirror is an optical part which reflects light having a specified wave length and allows lights having other wave lengths to pass therethrough.
(37) [Other Constitutional Part of 3-dimensional Confocal Microscopy Apparatus]
(38) The 3-dimensional confocal microscopy apparatus can further include, besides the above-mentioned respective constitutional parts, a beam splitter or a dichroic mirror which is arranged between the objective lens and the pair of lenses for focal plane displacement, and an imaging lens and a bright field imaging camera for imaging a 2-dimensional image of the specimen separated by the beam splitter or a dichroic mirror.
(39) [Focal Plane Scanning and Aberration Correction Unit]
(40) The focal plane scanning and aberration correction unit according to the present invention is constituted of the above-mentioned second constitutional part and third constitutional part of the 3-dimensional confocal microscopy apparatus.
(41) A representative usage of the focal plane scanning and aberration correction unit is a case where the focal plane scanning and aberration correction unit is incorporated into the 3-dimensional confocal microscopy apparatus manufactured by combining a confocal microscope with an optical tweezers technique. To be more specific, in the 3-dimensional confocal microscopy apparatus which includes the light source for a laser for confocal imaging excitation, the fixed objective lens which focuses the laser supplied from the light source on the fluorescent specimen, and the fluorescent imaging camera which detects a fluorescent confocal microscopy image based on fluorescent light emitted from the specimen, the focal plane scanning and aberration correction unit is arranged between the fixed objective lens and the fluorescent imaging camera.
EXAMPLES
(42) Hereinafter, one example of the 3-dimensional confocal microscopy apparatus and the focal plane scanning and aberration correction unit according to the present invention is explained. The technical scope of the present invention is not limited by the following examples.
Example 1
Constitution, Manner of Operation and Advantageous Effects of 3-dimensional Confocal Microscopy Apparatus
(43) The constitution of the 3-dimensional confocal microscopy apparatus according to this example is shown in
(44) The confocal unit 11 is of a type where a nipkow disc on a right side and a microlens array disc on a left side both shown in the cross-sectional view are formed into the integral structure, and both discs are synchronously rotated. A dichroic mirror is incorporated between the nipkow disc and the microlens array disc. Further, an imaging lens is provided below the dichroic mirror in the drawing.
(45) A fluorescent light emitted from the specimen 13 by the excitation laser passes through a pair of lenses for focal plane displacement (UPlanSApo100× made by Olympus) which is arranged behind the objective lens 12 and is constituted of a fixed intermediate imaging lens 14 and a movable intermediate objective lens 15 which is arranged such that a focal point of the intermediate imaging lens 14 and a focal point of the intermediate objective lens 15 overlap with each other on the same optical axis, and is focused on a confocal imaging camera 16 (ImagEM: EMCCD camera made by Hamamatsu Photonics) which is a high sensitive fluorescent light imaging camera for detecting a fluorescent confocal microscopy image by an imaging lens of the confocal unit 11.
(46) A piezoelectric actuator 17 (P-721 made by PI-Japan Corporation) is mounted on the intermediate objective lens 15. By driving the intermediate objective lens 15 in the optical axis direction with the actuation of the piezoelectric actuator 17, it is possible to displace an imaging focal plane in the vertical direction on the drawing without moving the objective lens 12 of the confocal microscope. The driving of the piezoelectric actuator 17 is performed at a high speed that approximately 30 slices of cross-sectional images can be acquired within 1 second at intervals of 1 μm of the specimen.
(47) An infrared laser irradiated from an infrared laser light source 18 for optical tweezers passes through a pair of scanning mirrors of an optical tweezers driving optical system 19, and is focused on the specimen 13 by a dichroic mirror 20 arranged below the objective lens 12.
(48) Further, a half mirror 21 is arranged below the objective lens 12 and, using this half mirror 21, a non-fluorescent image of the specimen 13 is focused on a bright field imaging camera 22 via the imaging lens thus enabling the observation of a bright field image.
(49) Next, the flow of a process for processing data or the like in this embodiment is explained in conjunction with
(50) In the image processing process 23, firstly, an observation view point is instructed when an operator manipulates a mouse on a screen. A set of cross-sectional views of the specimen 13 which is sequentially transferred from the cross-sectional image acquisition process 24 and is updated is stored in a memory of a GPU (GTX295 made by NVIDIA) of a computer, pre-calibrated aberration correction calculation is applied to the respective cross-sectional images and, thereafter, a 3-dimensional image is formed from an observation viewpoint instructed by volume rendering, and is displayed on a screen which constitutes a means for displaying a corrected image. A 3-dimensional image formed of 512×512×512 pixels can be displayed at a frame rate of 30 frames per second.
(51) In the cross-sectional image acquisition process 24, firstly, the intermediate objective lens 15 is moved to a predetermined position by driving the piezoelectric actuator 17 and, thereafter, a confocal image of the focal plane is acquired by the confocal imaging camera 16, and is stored in the main memory. The confocal image is readily transferred to the set of cross-sectional images on the GPU memory in the image processing process 23. In this example, since the frame rate of the confocal imaging camera 16 is 30 frames per second, when the specimen 13 is observed in cross sections at 10 layers, for example, a 3-dimensional image of the specimen 13 is, as a whole, updated 3 times for every second. Further, the confocal image in the main memory is recorded in a hard disc together with focal depth information and acquisition time information, and can be used for an analysis after an experiment is finished.
(52) In the optical tweezers control process 25, firstly, a position of the optical tweezers is instructed by a manipulator using a 3-dimensional mouse on a screen, a scanning amount of the optical tweezers driving optical system 19 is calculated based on the instruction, and a focal point of the optical tweezers is moved.
(53) Through the above-mentioned three processes, by freely manipulating an optical driving nano machine in a 3-dimensional space, positions and the deformation of the machine and an object to be manipulated can be 3-dimensionally observed in real time from an arbitrary direction.
Example 2
Use of 3-dimensional Confocal Microscopy Apparatus
(54)
(55) Next, the driving of a minute structural body prepared by a two-photon absorption stereolithography and a manipulation of a cell are performed using the 3-dimensional confocal microscopy apparatus of the example 1.
(56) Further,
(57) A specimen can be observed only in plane in the case of the conventional system and hence, it is difficult to grasp the vertical positional relationship between a cell and a minute structural body driven by optical tweezers so that the manipulation requires skills. By using the 3-dimensional confocal microscopy apparatus of the present invention, the 3-dimensional positional relationship between a minute structural body and a cell can be grasped at a glance so that the manipulability of the 3-dimensional confocal microscopy apparatus is greatly enhanced. In this manner, with the use of the 3-dimensional confocal microscopy apparatus of the present invention, the 3-dimensional confocal observation can be realized at an arbitrary view point simultaneously with the manipulation of an object to be observed such as a cell by driving a minute structural body using optical tweezers.
INDUSTRIAL APPLICABILITY
(58) According to the present invention, a 3-dimensional image of a specimen can be acquired during the manipulation of the specimen using optical tweezers without affecting an optical trap. Further, the accuracy of the 3-dimensional image of the specimen acquired in this manner can be also ensured.