System and methods for automated vitrification of biological materials

Abstract

The present invention relaters to a system and methods for automated vitrification of mammalian oocytes or embryos. The system and methods enable automated processing of oocytes or embryos in vitrification solutions; robotically moving vitrification devices that carry processed cells for freezing in liquid nitrogen; automated sealing of the frozen devices; and transferring the sealed devices to an automated storage system for long-term cryopreservation.

Claims

1. An automated biological material cryopreservation and thawing system comprising: (a) a carrier plate having (i) an area for holding unprocessed biological material, (ii) a multi-well area for holding biological material processing solutions, and (iii) a straw area for holding one or more vitrification straws; (b) a first robot connected to a manipulator to manipulate the biological material and the processing solutions; (c) a multiple degrees of freedom second robot to manipulate the vitrification straws; (d) a microscope and an image capturing device operatively linked to the microscope to capture images of the biological material, the manipulator and the vitrification straws; (e) a computer readable medium having executable instructions; (f) an automated vitrification straw-sealing machine and an automated cryopreservation storage device; and (g) a processor for executing the executable instructions of the computer readable medium; said executable instructions including instructions for automatically: (i) processing the captured images, and (ii) operatively controlling the microscope, the image capturing device, the first robot, and the second robot, and wherein the instructions for operatively controlling the second robot, include instructions to seal the vitrification straw in the vitrification straw-sealing machine, and to transfer the sealed vitrification straw into the automated cryopreservation storage device.

2. The system of claim 1, wherein the executable instructions further include instructions for automatically operatively controlling the straw-sealing machine and the automated cryopreservation storage device.

3. The system of claim 1, wherein the executable instructions further include instructions for: (i) when the biological material is a cell, embryo or oocyte, detecting the cell, embryo or oocyte at different development stages from the captured images; (ii) tracking the biological material inside the processing solutions in three dimensional space; (iii) monitoring the biological material's volume; (iv) generating a concentration gradient in the treatment solutions; (v) placing the biological material on the vitrification straws; and (vi) removing excessive treatment solution from biological material placed on the vitrification straw.

4. The system of claim 1, wherein the executable instructions further include instructions to automatically change focus and magnification of the microscope.

5. The system of claim 1, wherein said microscope includes an X-Y stage for receiving the carrier plate, the X-Y stage including an X-axis linear motion system and a Y-axis linear motion system, and wherein the executable instructions further include instructions to operatively controlling the X-Y stage.

6. The system of claim 5, wherein the microscope is operationally linked to a controller having a driver for each of said X and Y axis linear motion system.

7. The system of claim 6, wherein said X-axis linear motion system and the Y-axis linear motion system are independently controllable.

8. The system of claim 1, wherein said first and second robots have at least three degrees of freedom.

9. The system of claim 1, wherein said area for holding the biological material is integrated with a heating plate.

10. The system of claim 2, wherein said a straw sealing machine includes a preset straw loading position, a preset straw cap loading position, and a mechanical guiding mechanism to move the cap to the vitrification straw.

11. The system of claim 2, wherein said automated cryopreservation storage system includes a cryopreservation solution tank for holding the cryopreservation solution, a movable sub-container and a 3-axis transport system automated to locate and transport the sub-container.

12. The system of claim 11, wherein said movable sub-container has at least one position to hold at least one vitrification straw.

13. The system of claim 11, wherein said movable sub-container has a freezing region with an opening space for directly freezing vitrification straws.

14. The system of claim 11, wherein said movable sub-container has at least one handle for connection to the built-in transport system.

15. The system of claim 14, wherein the built-in transport system has at least one linear motion part, wherein said the linear motion part is controlled to move reciprocally to take the sub-container out of the tank, or put it into the tank.

16. The system of claim 3, wherein said computer executable instruction for detecting cell, embryo or oocyte at different development stages includes: image binarization to produce a binarized image, detecting potential cell targets from foreground objects in the binarized image, and fitting circles on potential cell targets.

17. The system of claim 16, wherein said detection involves automatically changing the magnification of microscope and coordinately moving X-Y stage.

18. The system of claim 17, wherein said detection starts with the detection of the cell, embryo or oocyte under lower magnification, and then X-Y stage is controlled automatically to move the detected cell, embryo or oocyte to the center of field of view.

19. The system of claim 3, wherein said computer executable instruction for tracking biological material in three-dimensional space further detects the biological material's Z position.

20. The system of claim 19, wherein said the detection of biological material's Z position involves the automated adjustment of the microscope's focal plane along the Z axis.

21. The system of claim 1, wherein said computer executable instruction for automatically placing the biological material on the vitrification straw further includes detection of the manipulator tip's contact on the vitrification straw tip.

22. The system of claim 21, wherein said detection of contact on the vitrification straw tip is based on detecting the vitrification straw tip's deflection arising from the contact of the manipulator's tip with the vitrification straw tip.

23. The system of claim 22, wherein said detection of straw tip's deflection is via motion detection.

24. The system of claim 1, wherein the manipulator is a micropipette, and the system further comprises a motorized syringe to generate pressure for micropipette aspiration or dispensing.

25. The system of claim 1, wherein the biological material is a cell, an oocyte and/or an embryo.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) The following figures illustrate various aspects and preferred and alternative embodiments of the invention.

(2) FIG. 1 illustrates an automated robotic vitrification system in accordance to one embodiment of the present invention.

(3) FIG. 2 illustrates a top view of a carrier plate that holds an embryo culture dish, a multi-well plate, and multiple vitrification straws in accordance to one embodiment of the present invention.

(4) FIG. 3 is a graph illustrating an example protocol for vitrification and thawing process. ES: equilibrium solution, VS: vitrification solution, TS; thawing solution, DS: diluent solution, WS: Washing solution.

(5) FIG. 4 is a graph illustrating an example schedule for parallel vitrification by the robotic system of the present invention. In this example, processing time follows the protocol from Kitazato Company (Kitazato Inc., Cryotop Safety Kit Protocol, 2014).

(6) FIG. 5 are images showing an example algorithm for micropipette detection of using area and shape information of abbots contained in the images. (a) The rectangle is the detection result. (b) Binarized image shows the defected foreground objects. (c) Contour image shows the micropipette shape; a Hough line transform is applied to reject false detections of micropipette.

(7) FIG. 6 is a graph illustrating contact detection algorithms for determining micropipette's Z position in accordance to one embodiment of the present invention. (a) Front view illustrating that the micropipette tip produces horizontal sliding movement when contacting the embryo culture dish substrate. The Z position of the micropipette relative to the culture dish bottom is determined by detecting this horizontal sliding motion. (b) Front view showing that the micropipette tip deforms the vitrification straw tip when contacting the straw tip surface. The Z position of the micropipette tip relative to the vitrification straw tip is determined by detecting the straw tip's deformation.

(8) FIG. 7 is a graph illustrating an example method for embryo detection in accordance to one embodiment of the present invention. (a) The circle shows the detection result. (b) Binarized image shows the detected foreground objects. (c) Contour image shows the shape information: a Hough circle transform is applied to fit a circle to the potential embryo target, in order to avoid false detections.

(9) FIG. 8 Control diagram for positioning embryos inside micropipette in accordance to one embodiment of the present invention.

(10) FIG. 9 Graph illustrating an example method for generating a concentration gradient when the system transfers the embryo from vitrification solution 1 (VS1) to VS2.

(11) FIG. 10 are images illustrating automated removal of excessive medium after placing an embryo on the vitrification straw tip in accordance to one embodiment of the present invention. (a) A relatively large medium obtaining the embryo is deposited onto the straw surface. (b) A thin film of medium is formed on the straw surface by the robot's dragging the micropipette tip to the left. (c) Excessive medium is aspirated away automatically by applying a negative pressure. Embryo stays in initial position because of friction.

(12) FIG. 11 Schematic showing an example of a straw cap seeing machine that places a cap on the vitrification straw in accordance to one embodiment of the present invention.

(13) FIG. 12 is a graph illustrating an example design of an automated liquid nitrogen storage system in accordance to one embodiment of the present invention.

(14) FIG. 13 illustrates an example design of a liquid nitrogen sub-container with slots for vitrified sample storage in accordance to one embodiment of the present invention: left region has open space to freeze vitrification straws, while the right region has a number of slots for storage of frozen sample. A lid can be placed on the sub-container in order to keep the vitrification straws in place.

DESCRIPTION OF THE INVENTION

(15) Definitions

(16) Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Also, unless indicated otherwise, except within the claims, the use of “or” includes “and” and vice versa. Non-limiting terms are not to be construed as limiting unless expressly stated or the context clearly indicates otherwise (for example “including”, “having” and “comprising” typically indicate “including without limitation”). Singular forms including in the claims such as “a”, “an” and “the” include the plural reference unless expressly stated otherwise. All documents cited are incorporated herein by reference in their entirety. In order to aid in the understanding and preparation of the within invention, the following illustrative, non-limiting, examples are provided.

(17) “Automatic/automated” as used in this document means free of human operator control (for example manual and/or joy-stick based control) or where human intervention is limited to entering input data. Software (image processing and motion control) and hardware are integrated in a computer machine to reduce or eliminate the operator's intervention.

(18) “Motorized” as used in this document means that devices such as positioning devices, syringes and so forth, are equipped with a motor or motors which are controlled via control algorithms and strategies.

(19) Overview

(20) The present invention relates to hardware components and computer vision algorithms to automate visitation process, including full automation. The present invention may be used to implement vitrification kits/protocols. For instance, the Kitazato's kit/protocol (Kitazato Inc., Cryotop Safety Kit Protocol, 2014) and the Origio Vitrification protocols (Origio MediCult Media, Medicult Vitrification Cooling Protocol, 2014) can be followed in the robotic system of the present invention by simply changing the oocyte/embryo washing time in vitrification solutions.

(21) The robotic system of the present invention is designed to be fully automated, including automated biological material loading and retrieval. The system monitors the response of individual cells in vitrification solution (VS) to determine the optimal washing time in VS. After washing cells in VS, the verified embryos are automatically placed onto vitrification straws. The straws are then sealed with a cap on a cap-sealing machine. The sealed straws are picked up and frozen in a cryopreservation sub-container, such as a liquid nitrogen sub-container. The sub-container containing the frozen vitrification straws is finally placed in an automated liquid nitrogen system for long-term cryopreservation. The robotic system performs tasks in an optimized parallel way to process multiple cells or cell lines. The system also allows users to readily specify the operation time of each step for realizing different verification protocols or for developing new protocols.

(22) A. Automated Vitrification System

(23) Embodiments of the invention relate to a system that includes a motorized X-Y stage microscope movable in two axes for moving sample in the X-Y plane, a camera for capturing images, a robot for positioning a manipulator, such as a micropipette, a motorized syringe connected with the micropipette for aspirating and dispensing embryos/oocytes/cells (biological materials), a carrier plate for holding the biological material culture dish, a multi-well plate for holding processing solutions, such as vitrification and thawing solutions, and multiple vitrification straws, a second robot for handling vitrification straws, and an automated liquid nitrogen storage system for long-term cryopreservation of the vitrified samples. This section describes an embodiment of the hardware system, the system work flow, and technical details.

(24) Vitrification straws may be described as physical carriers to hold embryos for freezing and cryopreserving in, for example, liquid nitrogen. Vitrification straws can be classified into three groups: opening strip based straws (e.g., Cryotop), micropipette based straws (e.g., Cryopipette), and CPA (cryoprotective agent) film based straws (e.g., Cryoloop). The opening strip based straws often consist of a polypropylene strip attached to a hard plastic handle. Vitrified embryos are placed directly on top of the strip. After placing embryos on strip, the straws are sealed with plastic caps to avoid contaminations in long-term cryopreservation. The micropipette based straws use micropipettes to directly aspirate vitrified embryos. After the vitrified embryos are aspirated into the micropipette, the opening of the micropipette is sealed by a heat sealer. The CPA film based straw, taking Cryoloop for example, involves a plastic loop that can be integrated into a lid of a cryovial via a stainless steel stick. During vitrification, the Cyroloop.sup.T is immersed into a cryoprotective solution to create a thin film of the solution inside the loop. The vitrified embryos are then transferred into the film and sealed in the cryovial. In this document, the term “vitrification straw” refers to all three classification groups. Among all the vitrification straws, Cryotop is a well-accepted method because it involves least residual solutions surrounding vitrified embryos for producing the highest cooling rate and highest post-thawing cell survival rate. Accordingly, herein Cryotop is used as an example to explain the present invention.

(25) In one embodiment with reference to FIG. 1, the vitrification system 1 of the present invention may include a microscope 100 which may be equipped with motorized magnification and motorized focusing functions. Mounted on or linked to the microscope may be an XY motorized stage 200. A custom designed carrier plate 300 with a heating plate 400 may be placed on the XY stage 200. As further described herein below and illustrated in FIG. 2, the carrier plate 300 may be configured to hold a biological material culture dish 301, a multi-well plate 302 containing biological material processing solutions, and multiple vitrification straws 303. A first robot 501 or robotic manipulator carrying a micropipette 510 may be used to manipulate biological materials such as oocytes/embryos/cells which may be held in culture dish 301. A motorized syringe 600 may be controlled to aspirate or dispense biological materials into and out of the micropipette 510. A camera 700 may be coupled or connected to the microscope 100 to provide visual feedback that may be used to realize vision-based robotic control. The camera 700 may be at a frame rata of 30 frames per second or higher to provide the system with continuous visual information. A second robot 502 may be used to handle vitrification straws for freezing in liquid nitrogen. As part of the system 1, a straw-sealing machine 800, which may also be custom aligned, may be used to seal vitrification straws with caps. Additionally, a liquid nitrogen (or other cryopreservation solution) sub-container 901 may be designed to carry frozen straws and is stored in an automated liquid nitrogen storage system 902. The system 1 may also include a central computer 50 having instructions that when executed control all the aforementioned hardware and functions via running custom developed control software.

(26) The system 1 shown in FIG. 1 may include an upright microscope 100. The microscope 100, may include a motorized X-Y stage 200. The stage 200 may have a travel range of centimetres and a resolution better than 1 μm along each axis. The microscope 100 may have a range of magnifications (e.g., 0.7×-11.5×), which may be automatically changed by the system's control program. In addition, the focus of the microscope may also be changed by the system automatically via auto-focusing algorithms.

(27) As shown in FIG. 2, the carrier plate 300 may include any number of working areas, for example three working areas; a biological material loading area 301, a biological material processing (i.e. washing, equilibration, vitrification, thawing and so forth) area 302, and a vitrification straw area 304. The biological material loading area 301 may be for holding a standard biological material culture dish (e.g., 35 mm Petri dish). The biological material processing area 302 is for holding a multi-well plate that may include processing solutions such as equilibrium solution (ES), vitrification solution (VS), washing solution, flawing solution, which may be used for washing/soaking biological material. The vitrification straw area 304 may include multiple slots for holding vitrification straws 303. A carrier plate having more or less than three working areas may also be possible.

(28) The biological material, such as embryos, oocytes or cells, in dish 301 may be manipulated by using a manipulator, such as micropipette 510. The micropipette may have a tip diameter suitable for aspirating or dispensing the biological material. In the case of embryos, the micropipette may have a tip diameter in the range of about 100-200 micrometer. The micropipette may be inserted into a micropipette holder 511 connected to the motorized syringe 600. The micropipette holder 511 is clamped to a robotic manipulator 501, which has a travel range of centimetres and a resolution better than 1 μm along each axis.

(29) After processing biological material on the carrier plate 300 (i.e., washing biological material with a series of VS, then placing biological material on the vitrification straw), a vitrification straw 303 with vitrified biological material on top may be picked up by a second robot 502 that plunges or dips the straw into liquid nitrogen in a sub-container 901 for fast freezing. The frozen vitrification straw may then be placed by the robot to a cap-sealing machine 800 for sealing vitrification straw with a plastic cap. The sealed straw may finally be inserted into the storage slots of the sub-container, and the sub-container is transferred by the robot 502 into an automated liquid nitrogen storage system 902.

(30) Robots 501 and 502 may be multiple degrees of freedom robots. In one embodiment, robots 501 and 502 may have at least three degrees of freedom. One robot may serve to position the micropipette to pick and place cells and solutions, and the other robot may be used to handle vitrification straws.

(31) With reference to FIG. 11, the straw-sealing machine 800 may include a fixed part 801 and a moving part 802. The fixed part 801 may include a shoulder 803 extending from one end of the fixed part 801. The moving part 802 may be configured to slide on an upper surface of the fixed part 801 towards the shoulder 803. In one embodiment of the present invention, the shoulder 803 may be configured for receiving and holding a straw cap 305. The moving part 802 may be configured to receive and hold a vitrification-type straw 303. In this case, sliding movement of the moving part 802 will result in the vitrification straw 303 sliding into cap 305 and sealing of the vitrification straw (see FIGS. 11(a) and 11(b). In another embodiment, the shoulder 803 may receive and hold the straw while the moving pert 802 may receive and hold the cap such that the cap moves towards straw thereby sealing it.

(32) B. Automated Embryo Vitrification Work Flow

(33) The vitrification work flow described below is explained by using the implementation of Kitazato vitrification protocol as an example. The system of the present invention may also be capable of automating the implementation of other protocols, such as that of Origio Vitrification protocols. FIG. 3 schematically shows major steps of a method of automated biological material vitrification in accordance to one embodiment of the present invention. The examples is with regards to embryos, however it should be understood that the system is also capable of automatically processing other types of biological materials, such as oocytes and any other biological cells. In the following example, the micropipette's tip is shown to point in the right direction. Accordingly the terms “left” and “right” are used relative to the micropipette's tip direction. It should be understood that the system may be reversed and have the micropipette's tip pointing to the left direction. The example processes described below and illustrated in FIG. 3, with reference to FIG. 1, may be implemented as machine-accessible instructions utilizing any of different programming codes stored in computer 50.

(34) Step 0. System Setup and Preparation

(35) The set up step is the only step which may include sub-steps that may involve an operator. All other steps in the method are fully automated. (a) Upon being turned on, the system 1 automatically moves the X-Y stage to a home position (i.e., the bottom right corner) and turns on the heating plate. (b) Embryo culture dish is taken out from incubator and placed to the embryo loading area on the carder plate 300. The culture medium may be covered with mineral oil to prevent evaporation for maintaining a constant pH. (c) ES (equilibrium solution) and VS (vitrification solution) are added to the multi-well plate 302 according to the protocol provided by the vitrification kit supplier. (d) Vitrification straws 303 are placed in the straw slots 304 of the carrier plate 300.

(36) Step 1. Automated Pick-Up of Embryos from Culture Dish (a) The system portions the embryo loading area into the field of view (FOV) under the microscope 100. (b) Autofocusing is performed to focus on embryos. (c) Detect embryos from the captured images. (d) Detect micropipette 510 tip and position it to the center of FOV under the microscope. (e) Detect the tip's contact on dish substrate by lowering the micropipette towards the dish bottom. (f) Move a target embryo close to the micropipette tip position by moving the X-Y stage 200 via closed-loop visual servo control. (g) Aspirate the embryo into the micropipette and position it to a desired location inside the micropipette by controlling the motorized syringe. (h) Move micropipette up (e.g., by 3000 μm) above the dish bottom surface, and move X-Y stage to ES area of multi-well plate 302.

(37) Step 2. Alternated Washing of Embryos in ES (a) System 1 performs autofocusing to focus on the bottom of ES region on the multi-well dish area 302. (b) The micropipette is lowered until contact with the bottom of ES region is detected. (c) The embryo is dispensed into ES by applying a positive pressure from the motorized syringe. (d) Embryo is washed for several times by repealing micropipette aspiration and dispensing. (e) The embryo is kept in ES for a suitable period of time (e.g., 12 min as in Kitazato protocol). The embryo shrinks to the minimum volume and then gradually returns to is ordinal volume, which indicates that equilibration is complete. The washing time in ES may be optimized for each of embryos according to their individual volume change in ES, enabled by constantly monitoring embryo volume by the system. (f) Aspirate and position the embryo in micropipette by controlling the motorized syringe. (g) Move micropipette up (e.g., by 3000 μm) above the dish bottom, and move X-Y stage to VS area of the multi-well dish 302.

(38) Step 3. Automated Washing of Embryos in VS (a) System performs autofocusing to focus on the bottom of VS region on the multi-well dish area 302. (b) The micropippette is lowered until contact with the bottom of VS region is detected. (c) The embryo is dispensed into VS. Due to the higher concentration of VS, the embryo equillibrated in ES floats upwards in VS, which cause embryo's position to change along the Z axis. The system 1 detects the embryo in 3D space by real-time processing 2D images and moving the focal plane upwards to keep the embryo in focus. Sub-steps (a), (b) and (c) may be repeated if multiple washing steps in VS is required by a protocol (e.g., Kitazato protocol requires VS1 and VS2). (d) Aspirate and position the embryo in micropipette when the embryo is detected to have its minimum volume or at fixed time indicated by the vacation protocol. (e) Move micropipette up (e.g., by 3000 μm) above the dish bottom, and move X-Y stage to the vitrification straw area 304.

(39) Step 4. Automated Placing of Embryo on Vitrification Straw (a) System 1 performs visual servo control of the X-Y stage 200 to position a vitrification straw 303 to the center of FOV. (b) The micropippette is lowered until its tip contacts the vitrification straw's surface. (c) Control the motorized syringe to infuse a relatively large volume (e.g., 5 μL) of vitrification solution (VS) in order to ensure that embryo is dispensed out of the micropipette and onto the surface of the vitrification straw. (d) Move the micropipette tip away from is initial position on the straw surface to the left side of the FOV. Medium follows the micropipette tip because of surface tension. (f) Aspirate away excessive medium until the volume of embryo droplet does not change further on the straw. (g) Move micropipette up (e.g., by 3000 μm) above the vitrification straw, and move the X-Y stage to the home position.

(40) Step 5. Automated Freeze, Seal and Cryopreserve the Vitrification Straw (a) The vitrification straw 303 with vitrified embryo on top of its surface is picked up by the second robot 602 and immediately frozen in the sub-container 901 containing liquid nitrogen. (b) Place the frozen straw on the cap sealing machine 800. (c) Move a plastic cap to seal the straw tip via a guiding mechanism on the sealing machine. (d) Insert the sealed frozen straw into a slot in the liquid nitrogen sub-container 901. (e) Transfer the sub-container into the automated liquid nitrogen storage system for long-term cryopreservation

(41) Typically, more than one oocyte/embryo/cell needs to be processed within a batch. In order to shorten system idle time, task scheduling may be made/optimized. FIG. 4 shows a scheduling example, assuming Kitazato vitrification protocol is implemented. Accordingly to this scheduling, robot 501 starts to pick up the second embryo from the culture dish while the first embryo is inside ES.

(42) C. Thawing Process

(43) The set up step is the only step which may include sub-steps that may involve an operator. All other steps are fully automated.

(44) Step 0. System Setup and Preparation (a) System 1 is turned on for thawing process and moves the X-Y stage to home position. (b) Pre-warmed thawing solution (TS), dilution solution (DS), and washing solution (WS) are added into the multi-well dish, according to the protocol. (c) The frozen straw with embryos is taken out from the alternated liquid nitrogen storage system. The sealing cap is removed from the straw by the second robot 502. (d) The straw is plunged into TS. The straw is gently shaken to let embryos fall off from the straw tip.

(45) Step 1. Automated Thaw Embryos in TS (a) The system moves the TS region to the center of FOV under the microscope. (b). Autofocusing is performed to focus on the embryos. (c). Embryos are detected and tracked from captured images. (d). Embryos are kept in TS for a suitable period of time (e.g., about 1 min), according the protocol being used. (e). Micropipette tip is detected and positioned to the center of FOV under the microscope. (f). The tip's contact on dish substrate is detected by lowering the micropipette towards the dish bottom. (g). A thawed embryo is moved close to the micropipette tip position. (h). The embryo is aspirated into the micropipette and positioned to a desired location inside the micropipette by controlling the motorized syringe. (i). Micropipette is moved up (e.g., by about 3000 μm) above the dish bottom, and move X-Y stage to DS area.

(46) Step 2. Automated Transfer Embryos from TS to DS (a). Autofocusing algorithm is performed to focus on the bottom of DS region on the multi-well dish. (b). The micropipette is lowered until contact is detected. (c). The embryo is gently dispensed to the bottom of DS by applying a positive pressure from the motorized syringe. (d). The embryo is kept in ES for a period of time (e.g., 3 min), according to the protocol. (e). The embryo is aspirated and positioned in the micropipette by controlling the motorized syringe. (f). Micropipette is moved up (e.g., by about 3000 μm) above the dish bottom, end move X-Y stage to WS area.

(47) Step 3. Automated Wash Embryos and Transfer to Culture Medium (a). The steps as in Step 2 may be repeated to transfer embryo from DS to WS, and keep embryo in WS for a suitable period of time (e.g., 5 min) according to the protocol being followed. (b). Embryos are washed in WS for a number of times, for example two, three, four or more times, by aspirating the embryos into micropipette and gently placing it on the top of WS. The embryo may be let to freely fall to the bottom of WS. System detects embryos and tracks them in 3D space during washing in WS. (c). Washed embryos are transferred to a culture dish containing the culture medium, similar to Step 2. (d) Put the embryo culture dish back to a 37° C. incubator for complete recovery.

(48) D. Techniques for Automated Vitrification

(49) Techniques for implementing the work flow discussed in the previous section are described below. These techniques include: micropipette/vitrification straw detection, contact detection, autofocusing, embryo detection/tracking, positioning embryos in micropipette, generation of a moderate concentration gradient in VS, removal of excessive VS from the vitrification straw, a vitrification straw sealing machine, and an automated liquid nitrogen storage system. The techniques described below and illustrated in the referred figures may be implemented as machine-accessible instructions utilizing any of different programming codes stored in computer 50.

(50) Detection of Micropipette and Vitrification Straw

(51) The first step may be to binarize an image of the micropipette or vetrification straw and extract target objects from the background. In IVF clinics, stereo up-right microscope is often used for examining the inner morphological detail of embryos. Stereo microscope produces a lighting gradient on the captured images. The system runs a histogram-based thresholding algorithm to discriminate foreground objects (e.g., micropipette, or vitrification straw) from background. At first, a histogram of gray scale value of all pixels may be calculated. The histogram may include two peaks, with one representing the background pixels and the other representing the pixels of the foreground objects. The middle point of the two pixels may be used by the system as threshold to bitwise the image for extracting foreground objects, as shown in FIG. 5(a)(b).

(52) After image binarization, morphological transformations (i.e., erosion and dilation) are performed to remove the noisy objects of small sizes. Since the micropipette is much larger than embryos, the largest object in the binarized image is taken by the system to be the micropipette. In addition to using size as a criterion for detecting micropipette, the shape information is also used by the system to ensure correct defection. The outside walls of the micropipette can be modeled as two lines, therefore, Hough line transform is applied on the contours of detected objects (see FIG. 5(e)). If the two lines are on the largest object, micropipette detection is confirmed to be correct. Otherwise, it is considered as a false detection.

(53) In the robotic system, the micropipette tip points to the right direction. According to this specific system setup, the point (on the detected micropipette object) with the largest distance to the left boundary of the image is considered by the system to be the micropipette tip position. Within the detected micropipette, a smaller region is extracted as a region of interest (ROI) for detecting embryos inside micropipette.

(54) The detection of vitrification straw is similar to micropipette detection. The difference is that the orientation of the vitrification straw (with the tip pointing to the left side) is opposite to the micropipette.

(55) Contact Detection

(56) The relative distance along Z axis between micropipette tip and dish bottom or vitrification straw's surface may be critical for successful automation of picking/placing embryos from/to dish or straw.

(57) The relative distance between micropipette tip and dish bottom may be detected based on detecting micropipette tip's sliding motion on the dish bottom. The system constantly visually tracks the tip's position in the x-y plane, while lowering the micropipette downwards to the dish bottom. As illustrated in FIG. 6(a), when the micropipette 510 tip contacts the dish bottom 601, further downward movement induces the tip's horizontal sliding motion on the dish bottom, which changes the tip's position in the x-y plane. The detected Z position of the robotic manipulator 501 is then recorded as the reference height of the dish bottom.

(58) The detection of the micropipette tip's contact on vitrification straw is based on a different principle. The vitrification straw 303 has low stiffness and is easily deflected when contacted by the micropipette tip. Therefore, the robotic system of the present invention detects the contact between micropipette tip and straw by detecting the straw's deformation motion (see FIG. 6(b)). Various motion detection algorithms can be used to detect straw's deformation, such as frame subtraction, background subtraction, optical flow, or the motion history image method.

(59) Another method for detecting the tip's contact with vitrification straw is based on detecting the spreading of medium on the straw. When micropipette tip contacts the straw, medium inside the micropipette passively flows out to the hydrophilic straw surface, due to surface tension. Therefore, the contact position can be determined by detecting medium spreading around the micropipette tip. When an object appears around the tip position in the binarized image, contact is considered detected. The robotic manipulator 501 then stops downward movements and records the reference Z position.

(60) Autofocusing

(61) One prerequisite stop for the aforementioned detection processes is to focus on a reference surface (i.e., on dish bottom or on straw surface). Accordingly, the system conducts auto-focusing. The autofocus algorithm uses a focus measure based on Tenenbaum gradient to determine the focus level of the captured image. This algorithm convolves an image with Sobel operators, and then sums the square of the gradient vector components. The system adjusts the microscope focal plane until the focus measure reaches maximum. The first order derivative of focus measure is used to determine the maximal value. When the focal plane moves to the target objects (e.g., embryos or micropipette tip), the focus measure is increasing and the first order derivative is positive. Once the first order derivative turns to negative, the local plane has reached the optimal position at the turning point and starts to move away from the target objects.

(62) Embryo Detection/Tracking

(63) In order to avoid the loss of embryos through the entire vitrification process, robustly detecting and tracking embryos is important. With reference to FIG. 7, the first step in the system to detect embryos is image binarization [see FIG. 7(a)(b)]. In the binarized image, the system detects embryos from foreground objects by using area and shape information. Objects with an area ranging from 500 to 1000 pixels are considered to be potential embryo targets. In addition, a Hough circle transform is applied to fit circles on the potential targets. The objects with circular shapes are considered as embryos. The number of fitted circular shapes indicates the embryo developmental stages (e.g., 2-cell, 4-cell, blastocysts stage). The area of an individual embryo is monitored in real time by the system to provide a criterion for optimizing the immersion time in VS. When washing embryos in VS, the embryos shrink in the beginning due to osmatic pressure. Then they re-expand to equilibrate with VS. Since the solute of VS is toxic, the equilibration with VS should ideally be avoided. Therefore, embryos ideally should be taken out of VS once they reach their minimum size/volume. However, in present manual operation, all embryos undergo the same timing.

(64) In order to rapidly locate embryos in the relatively large culture dishes, the embryo detection algorithm further involves the change of the microscope magnifications and the movement of the X-Y stage. The system first detects potential embryo objects under a lower magnification (i.e., larger FOV). Then the potential embryo objects are positioned to the center of FOV by moving the X-Y stage. The system then automatically switches the microscope to a higher magnification, and performs embryo detection again to verify the detection result from under lower magnifications and analyze the inner morphology of the embryo.

(65) When transferring embryo from lower-concentration ES to VS or during the washing steps, embryos can move up and down inside the solutions, due to the buoyancy force and/or fluidic flow caused by micropipette movement/dispensing. In order to tracking embryos in 3D space, the embryo detection algorithm is integrated with the autofocusing algorithm by dynamically moving the focal plane to keep the tracked embryo(s) in focus.

(66) Positioning Embryos in Micropipette

(67) During embryo aspiration and dispensing, the robotic system quickly and accurately positions the embryo in micropipette. The embryo's position is visually tracked to provide position feedback for the system to realize closed-loop visual servo control. The desired position inside the micropipette is at a pre-specified distance to the tip. This distance cannot be too long because it would introduce too large a volume of the solution when dispensing the embryo into the next vitrification solution. On the other hand, the distance cannot be too short because the embryo can possibly ‘escape’ out of the micropipette tip, resulting in oocyte/embryo loss. Based on the error between desired position and detected positions, the system runs a controller (e.g., proportional-integral-derivative (PID) or robust controller) algorithm to control the motorized syringe and aspirate or dispense until the oocyte/embryo reaches the desired position. FIG. 8 shows a control diagram for positioning embryos in micropipette.

(68) Generation of a Moderate Concentration Gradient

(69) In vitrification, ES often has the same solute as VS but at a lower concentration. After equilibrated in ES, the embryo encounters an osmotic shock when transferred into VS due to the concentration change. In order to lessen cell damage, this osmotic shock may be minimized. Therefore, the robotic system may generate a moderate concentration gradient in VS. To generate a moderate concentration change, the system moves the micropipette along the X axis to the left while dispensing ES into VS. ES medium is dispensed at a constant rate (e.g., at 5 μl/sec) while the speed of micropipette movement keeps increasing until micropipette tip is moved to the edge of FOV. Since the mixing rate between ES and VS is constant, a concentration gradient is generated along the micropipette's moving path.

(70) When an embryo is dispensed out from the micropipette tip, they are first positioned at the locations where the concentration is lower (i.e., similar to ES). Then the embryos are gently moved along the concentration gradient increasing direction (e.g., from right to left). After that, the embryos are washed several times by aspirating into and dispensing out of the micropipette.

(71) FIG. 9 illustrates an example method for generating a concentration gradient. VS2 often has the same solutes as VS1 but at a higher concentration. In order to generate a concentration gradient, VS1 is infused into VS2 at a constant rate while the robot moves the micropipette to the left direction at an increasing speed with a constant acceleration.

(72) Another method to minimize osmotic shock pressure is to use a series of vitrification solutions with moderate concentration changes. However, this method involves more types of VS and more steps of embryos pick-and-place.

(73) Removal of Excessive VS from the Vitrification Straw

(74) After robot 501 places a vitrified embryo onto the straw, the system removes excessive medium to achieve a high cooling rate. As shown in FIG. 10(a)(b), the system first dispenses the embryo in a relatively large volume of VS solution onto the vitrification straw. Robot 501 then moves the micropipette on the straw surface away from the initial dispensing location to form a thin VS film. The robotic system controls the motorized syringe to aspirate VS until the volume of the embryo droplets stops changing [FIG. 10(c)]. While VS medium is aspirated into the micropipette, friction force keeps the embryos in its original place.

(75) Throughout this step, the system utilizes image processing to monitor droplet volume change and micropipette tip positions. The embryo is also detected from the straw ROI by using the methods described in the previous section “Embryo Detection/Tracking”. Since the embryo is surrounded by the excessive medium, the volume of the detected object is proportional to the volume of the excessive medium. Accordingly, when the detected embryo volume stops changing, it is considered that all excessive medium has been removed from the vitrified embryo.

(76) Vitrification Straw Sealing

(77) The vitrification straw sealing machine is design to automatically place a plastic cap onto the straw tip in order to avoid contamination during the long-term cryopreservation. As shown is FIG. 11, the sealing machine comprises a mechanical guiding mechanism, which allows a sliding part to move towards the fixed part. The vitrification straws and plastic taps are mounted on predefined slots/positions on the sliding part and fixed part, respectively. After the straws are placed by robot 502 on the sealing machine, the system controls a motor to drive the sliding part to the fixed part until a limit switch is triggered, which completes the capping/sealing process.

(78) Automated Liquid Nitrogen Storage System

(79) As part of the robotic vitrification system, an automated liquid nitrogen storage system is designed. The automated storage system comprises a liquid nitrogen tank, a built-in 3-axis transport system, and liquid nitrogen sub-containers. The three-axis transport system 903 is automated to locate a sub-container 901 and transfer it into or out of the storage tank 904 containing liquid nitrogen. As shown in FIG. 12, the transport system 903 may include three translational motion axes or two translation axes plus one rotational axis to rotate sub-containers insider the storage tank. As the end-effector of the transport system, a mechanical mating loop 905 holds the handle of the sub-container.

(80) As shown in FIG. 13, the sub-container 901 is designed as the sub-unit of the automated storage system 902 for freezing and storing vitrification straws. The sub-container 901 has a handle 906 for the end-effector of the transport system to connect to, a vitrification freezing region 907, and a storage region 908 having multiple slots for frozen straws to insert in. To prevent the vitrification straws' free motion in the storage region, the sub-container further includes a lid 909 to cover the storage region.

(81) E. Advantages

(82) The system of the present invention automates the implementation of well accepted vitrification protocols. Without changing well-established protocols or modifying embryo handling approach, the system of the present invention can be easily accepted in clinics.

(83) Embryos are monitored in real time throughout the entire vitrification process, which can effectively avoid embryo loss.

(84) The systems and methods of the present invention are fully automated including the loading and retrieving embryos (vs. manual loading in microfluidic vitrification), and detecting the embryos position in solutions (as opposed to the manual input in patient by Ru et al. in Chinese Pat. Appl. Publ. No. 202918907U).

(85) The systems and methods of the present invention can generate a moderate concentration gradient which can minimize the osmotic pressure and further increase the cell survival rate.

(86) The systems and methods of the present invention can automatically place vitrified embryos onto vitrifications straws and remove excessive surrounding solutions to achieve ‘minimal volume’, in order to achieve high cooling rates for freezing.

(87) The systems and methods of the present invention is capable of processing multiple embryos with an optimized schedule for saving the total processing time.

(88) The systems and methods of the present invention includes an automated sealing machine that can be used to place caps to vitrification straws, and an automated cryopreservation storage system which can eliminate human errors in embryo handing.

(89) The oocytes, embryos and cells are exposed to the different processing solutions (i.e. washing, equilibrium, vitrification, thawing solutions) as uniformly as in the standard manual protocols.

(90) Through the embodiments that are illustrated and described, the currently contemplated best mode of making and using the invention is described. Without further elaboration, if is believed that one of ordinary skill in the art can, based on the description presented herein, utilize the present invention to the full extent.

(91) Although the description above contains many specificities, these should not be construed as limiting the scope of the invention, but as merely providing illustrations of some of the presently embodiments of this invention. Thus the scope of the invention should be determined by the appended claims and their legal equivalents, rather than by the examples given.