DROPLET PROCESSING METHODS AND SYSTEMS

Abstract

A method of processing droplets in a microfluidic system. The method may comprise capturing a time sequence of images of a droplet as it passes through a channel in a microfluidic system. The method may further comprise processing each image of the sequence of images using a convolutional neural network to count a number of cells or other entities visible in each image the droplet. The method may further comprise processing the count of the number of cells or other entities visible in each image of the droplet to determine an estimated number of cells or other entities in the droplet. The method/system may further comprise controlling a microfluidic process performed on the droplet responsive to the estimated number of cells or other entities in the droplet. Implementations of the method use the changing orientation and disposition of droplet contents in combination with machine learning to improve monoclonality assurance.

Claims

1-21 (canceled)

22. A method of processing droplets in a microfluidic system, the method comprising: capturing a sequence of images of a droplet as it passes through a channel in a microfluidic system; processing each image of the sequence of images using a convolutional neural network to count a number of cells or other entities visible in each image of the droplet; processing the count of the number of cells or other entities visible in each image of the droplet to determine an estimated number of cells or other entities in the droplet; and controlling a microfluidic process performed on the droplet responsive to the estimated number of cells or other entities in the droplet.

23. A method as claimed in claim 22 wherein processing the count of the number of cells or other entities visible in each image of the droplet to determine an estimated number of cells or other entities in the droplet comprises determining the estimated number of cells in the droplet to be a modal value of the counts for the images.

24. A method as claimed in claim 22 wherein processing the count of the number of cells or other entities visible in each image of the droplet to determine an estimated number of cells or other entities in the droplet comprises determining the estimated number of cells in the droplet to be a maximum value of the counts for the images.

25. A method as claimed in claim 22, wherein processing each image of the sequence of images using the convolutional neural network to count the number of cells or other entities visible in each image the droplet comprises classifying each image of the droplet into one of a plurality of categories, wherein the categories comprise at least three categories, a category for no cell or other entity in the droplet, a category for just one cell or other entity in the droplet; and at least one category for more than one cell or other entity in the droplet.

26. A method as claimed in claim 22 wherein controlling the microfluidic process comprises, in response to the estimated number of cells or other entities, selectively performing one or more of: sorting the droplet; dispensing the droplet; incubating the droplet; splitting the droplet; fusing the droplet; reducing a volume of the droplet; taking an aliquot of the droplet; increasing a volume of the droplet.

27. A method as claimed in claim 22 wherein the convolutional neural network is a convolutional recurrent neural network, and wherein the sequence of images of the droplet comprises a video sequence of images of the droplet.

28. A method as claimed in claim 22 wherein capturing the sequence of images comprises capturing a sequence of (x, y) images at different depths in the z-direction.

29. A method as claimed in claim 22 further comprising localizing a cell or other entity in a captured image and displaying an image of the localized cell or other entity to a user.

30. A method as claimed in claim 29 wherein localizing the cell or other entity comprises using the convolutional neural network to define a position or boundary for the cell or other entity; and wherein displaying the image of the localized cell or other entity comprises displaying the image with the position/boundary superimposed.

31. A method as claimed in claim 22 further comprising localizing one or more cells or other entities in a captured image; wherein controlling the microfluidic process is performed responsive to a determined location of the one or more cells or other entities.

32. A method as claimed in claim 22 wherein processing each image of the sequence of images using a convolutional neural network comprises processing each image at a first resolution; the method further comprising localizing a cell or other entity in a captured image, using the localizing to provide a second, higher resolution image of the cell/entity, processing the second, higher resolution image of the cell or other entity using a set of one or more characterization neural networks to characterize the cell or other entity and/or an event associated with the cell or other entity, and outputting characterization data for the cell or other entity.

33. A method as claimed in claim 22 further comprising capturing a fluorescence-time or luminescence-time signal for the droplet, processing the fluorescence-time or luminescence-time signal using a set of one or more further signal processing neural networks to provide droplet characterization data characterizing the contents of the droplet, and controlling the or another microfluidic process performed on the droplet responsive to the droplet characterization data.

34. A method as claimed in claim 32 further comprising illuminating the droplet with a substantially uniform sheet of light to provide the fluorescence-time or luminescence-time signal for the droplet and/or the second, higher resolution image of the cell or other entity.

35. A method as claimed in claim 22 used for identifying a transient cell/entity-associated signal, the method comprising activating a cell/entity-associated event with a controlled timing upstream of a location of capturing the sequence of images of the droplet such that a transient cell/entity-associated optical signal is produced at the location of capturing the sequence of images; the method further comprising using the or another convolutional neural network to identify presence of the transient cell/entity-associated signal in one or more of the captured sequence of images.

36. A method of processing droplets containing one or more entities in a microfluidic system, the method comprising: capturing a time sequence of optical signals from a droplet as it passes through a channel in a microfluidic system; processing the time sequence of optical signals using a set of one or more classifier neural networks to determine data characterizing one or more entities in the droplet; and controlling a microfluidic process performed on the droplet responsive to the data characterizing the one or more entities in the droplet.

37. An instrument for microfluidic droplet-based processing of cells or other entities, the instrument comprising: a droplet generation system to generate one or more emulsions of droplets comprising cells or other entities; a microfluidic droplet processing system to process the droplets; and a droplet dispensing system to dispense the processed droplets into one or more reservoirs; wherein the droplet processing system comprises: an image capture device to capture a sequence of images of a droplet as it passes through a channel in the microfluidic droplet processing system; a convolutional neural network to process each image of the sequence of images to count a number of cells or other entities visible in each image the droplet; and a processor configured to: determine an estimated number of cells or other entities in the droplet from the count of the number of cells or other entities visible in each image of the droplet, and to control the droplet dispensing system and/or a microfluidic process performed on the droplet prior to the droplet dispensing system, responsive to the estimated number of cells or other entities in the droplet.

38. An instrument for microfluidic droplet-based processing of cells or other entities, the instrument comprising: a droplet generation system to generate one or more emulsions of droplets comprising cells or other entities; a microfluidic droplet processing system to process the droplets; and a droplet dispensing system to dispense the processed droplets into one or more reservoirs; wherein the droplet processing system comprises: an optical signal capture device to capture a time sequence of optical signals from a droplet as it passes through a channel in the microfluidic droplet processing system; a set of one or more classifier neural networks to process the time sequence of optical signals to determine data characterizing one or more cells or entities in the droplet; and a processor configured to control the droplet dispensing system and/or a microfluidic process performed on the droplet prior to the droplet dispensing system, responsive to the data characterizing the one or more cells or entities in the droplet.

39. A carrier carrying processor control code or data to implement the method of claim 22.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0038] These and other aspects of the invention will now be further described, by way of example only, with reference to the accompanying figures in which:

[0039] FIG. 1 shows a schematic diagram of a droplet generation, detection, sorting and dispensing system;

[0040] FIG. 2 shows a schematic diagram of a further example of a droplet generation, incubation, detection, sorting and dispensing system;

[0041] FIG. 3 shows a schematic diagram of a further example of a droplet generation, fusion, incubation, detection, sorting and dispensing system;

[0042] FIG. 4 shows an example physical configuration of a droplet processing system;

[0043] FIG. 5 shows a flow diagram of a method of processing droplets in a microfluidic system in an implementation of the techniques described herein;

[0044] FIG. 6 shows a block diagram of processor for implementing the method of FIG. 5;

[0045] FIG. 7 shows example images from a microfluidic system for processing droplets implementing the techniques described herein; and

[0046] FIG. 8 schematically illustrates a two stage procedure for identifying and classifying a cell/entity.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

[0047] FIG. 1 shows a schematic illustration of a first example of a droplet generation, detection, sorting and dispensing system. In this example, multiple cells are provided in a fluid, in this example water, usually including a growth medium. In the first step, individual droplets are formed from the fluid. As outlined above, this may be achieved by using, for example, T-junctions, flow focussing devices, or other devices. The droplets which have been generated are, in this example, transported in a fluid of oil.

[0048] The individual droplets, which may or may not contain one or more cells, are then guided through the microfluidic device in an oil emulsion.

[0049] In this example, the droplet cell suspension, i.e. the droplets in the oil emulsion, are guided towards a detection and sorting device. Whether or not a single droplet contains one or more cells may be detected in the analyser, based on one or more of electrical, optical, thermal, acoustic, mechanical, temporal, spatial, and other physical characteristics of the droplets. Based on the analysis in the analyser, i.e. the determination as to whether a single droplet contains one or more target cells, the droplet may be sorted in the droplet sorting device. In this example, droplets which do not contain one or more cells are put to waste. Furthermore, droplets which contain, in this example, the single cell of interest are guided towards a decoupler of the microfluidic system.

[0050] Droplets which contain one or more cells of interest are then extracted from the first fluidic flow path and transferred into a second fluidic flow path. In this example, the target droplets are extracted from the first fluidic flow path in a growth media fluid. A droplet which contains a target cell, whereby the droplet is incorporated in the growth media fluid, is then dispensed into a microtitre plate via pressurised fluid ejection. A pressure source is, in this example, attached to the flow path at which the growth media fluid is injected. The droplets may thereby be diluted. A robotic xy translational stage is provided in this example in order to dispense droplets into different wells of the microtitre plate.

[0051] The droplet detecting, sorting and cell dispensing may, in this example, be performed in a temperature controlled environment.

[0052] In this example, droplets which contain a single cell and which are not to be disposed into the microtitre plate, may be guided in the first fluidic flow path to waste.

[0053] Using the techniques described herein the probability for finding a single cell in a single droplet which is disposed into the microtitre plate may be higher than 95% or 99%.

[0054] In the example of FIG. 1, an automated plate stacker interface may be provided, in order to stack various microtitre plates on top of each other.

[0055] In some preferred implementations of the system of FIG. 1, and similarly of the later described systems, an imaging device such as a camera is provided at or close to the point at which the droplets are dispended (into the microtitre plate). The imaging device may be used to image a droplet to determine whether the droplet contains just a single cell (or other biological entity), or alternatively two or more cells, or no cells. In other approaches optical detection of scatter from a droplet may be detected for this purpose. Such an optical system may then be used as a tool to monitor monoclonality, more particularly in conjunction with a control system to identify and log when a well does not contain just a single cell (or vice-versa), for example so that the system may then flag an error for the identified well.

[0056] FIG. 2 shows a schematic illustration of a further example of a droplet detection, sorting and dispensing system.

[0057] In this example, cells are provided in a fluid together with assay reagents. Individual droplets are then formed from the fluid as outlined in the example shown in FIG. 1.

[0058] Droplets which have been prepared from the fluid containing cells and assay reagents, are then guided into an incubator. The incubator may be used to grow and/or maintain the cells in the droplets. As outlined above, the incubator may comprise a stability test unit which allows for performing stability tests on the cells during the incubation. Performing a stability test on a cell in a single droplet allows for sorting only viable cells during the detection and sorting steps in the analyser and droplet sorting device, which have not degraded or died during the stability test.

[0059] Further steps of determining the content of a droplet, sorting the droplet based on the determination, and a potential extraction of a droplet of interest in the decoupler are performed as outlined with regard to the schematic illustration of FIG. 1.

[0060] FIG. 3 shows a schematic illustration of a further example of a droplet detection, sorting and dispensing system.

[0061] In this example a droplet fusion assay mode is illustrated. The first cell type A is provided in a first fluid. Individual droplets are then formed from this first fluid. A second cell type B is provided in a second fluid, from which individual droplets are formed. Droplets which have been prepared from the first fluid, as well as droplets which have been prepared from the second fluid, are guided towards a fusion device (electrodes in FIG. 3). In this example, two droplets from the first and second fluids, respectively, are prepared by electro-coalescence. The fused droplets may then be processed further as shown in the schematic illustrations of FIG. 1 and/or FIG. 2.

[0062] As outlined above, the droplet fusion device may be placed, for example, behind the analyser and droplet sorting device in a fluid flow direction of the microfluidic system. Such a configuration may allow for fusing droplets in the droplet fusion device only for droplets which have been determined to contain, in this example, cells which are of interest for growth and/or further analysis and processing.

[0063] FIG. 4 shows an example physical configuration of a droplet processing instrument 1200. The system/instrument 1200 may incorporate a (disposable) microfluidic cartridge 410 and may have a reservoir 402 holding an aqueous medium, generally including growth media, within which biological entities such as mammalian cells, bacteria or the like are held for processing. A set of reservoirs 404a-d (which may include pumps) hold oil for providing an emulsion at various stages of the system. Although reservoirs 402, 404 are shown as part of the instrument rather than part of the cartridge, in other configurations they may be part of the cartridge; depending upon the cartridge function, for example whether it is to be used for a fusion or functional assay, there may be more than one aqueous reservoir 402. The fluids from reservoirs 402, 404 are pumped by respective pumps. A set of fluid flow lines may connect to respective ports on the cartridge when the cartridge is inserted into the instrument (as shown in the figure). The cartridge 410 may comprises a droplet generation region, and/or a droplet incubation/storage region, and/or a droplet sorting region, and/or a flow buffer region, and/or a droplet dispenser. In implementations of the cartridge each of these regions may be modular and an emulsion flow processing configuration may be selected by selecting modules to attach to a cartridge base or holder so that the selected modules interface with one another to perform a desired function for a particular cartridge.

[0064] The droplet dispenser may apply a pressure pulse to the emulsion flowing in an output channel, for example via a compressed air line, to eject a slug of emulsion containing a selected, target droplet for collection in a well of a multi-well reservoir 484. The emulsion may be created off-cartridge and added to a reservoir.

[0065] FIG. 4 shows locations of one or more cameras 1202, 1204, 1206 for example for, respectively, droplet generation, sorting and dispensing. These may have respective (LED) illumination 1114, 1144, 1164, for example under control of an LED controller 1208. The figure also illustrates a heater plate 1210 of an incubator for the cartridge, and valve actuators 1212 for droplet generation and incubation, either or both of which may be present in some implementations. The multi-well plate 484 may be mounted on an X-Y stage 1214. The instrument may also includes an internal oil container 1216 to supply these reservoirs. Excess oil may flow to waste through into a reservoir 436 from which, potentially, the oil may be recycled. The instrument may include a control system; and may further include a user interface 1218.

[0066] FIG. 5 shows a flow diagram of a procedure for processing droplets in a microfluidic system in an implementation of the techniques described herein. Thus at step S400 the procedure captures a sequence of images, for example 5 images, of the same droplet as it passes through the field of view of a camera. Each image may then be processed by cropping around the droplet to create a smaller, cropped image (S402). Optionally the image resolution may also be reduced, for example to provide an 80×80 monochrome image.

[0067] Each cropped image is then processed by a convolutional neural network which classifies a cell count of the droplet image as either: 0, 1, or 2+ cells (S404). The convolutional neural network architecture may be varied according to the input image resolution, number of category outputs, quantity and quality of training data, degree of regularization employed and so forth. A manual or automated tool may be used to select/optimise a network architecture and number of nodes. Merely by way of example, in one implementation the convolutional network was trained with 180,000 labelled images and employed the architecture below; such a convolutional network by may implemented using software such as TensorFlow or the like:

TABLE-US-00001 Layer type Shape Parameters Conv2d_1 (78, 78; 20) 560 Activation_1 (78, 78; 20) 0 Conv2d_2 (76, 76; 53) 9593 Activation_2 (76, 76; 53) 0 Conv2d_3 (74, 74; 86) 41108 Activation_3 (74, 74; 86) 0 Conv2d_4 (72, 72; 119) 92225 Activation_4 (72, 72; 119) 0 Conv2d_5 (70, 70; 152) 162944 Activation_5 (70, 70; 152) 0 MaxPooling_2d_1 (35, 35; 152) 0 Flatten_1 (186200) 0 FullyConnected_1   (200) 37240200 Activation_6   (200) 0 FullyConnected_2    (3) 603 Total parameters 37,547,233

[0068] The set of cell counts is then processed to determine an estimated number of cells in the droplet, for example by determining the mode of the count for each image, or by determining a maximum number of cells counted (S406) e.g. to identify monoclonality of a droplet. In applications of the system which relate to assuring monoclonality (i.e. a single cell per droplet) it may be advantageous to err on the side of false positive non-monoclonal identifications. The microfluidic system may then be controlled (S408) according to the number of cells identified in a droplet, for example to discard droplets with 0 or 2+ cells per droplet, or to split droplets with 2 cells, and so forth.

[0069] FIG. 6 shows a block diagram of a processor 600 configured to implement the procedure of FIG. 5. Thus the processor comprises a microprocessor 610 and optionally a GPU/TPU or similar 612 and/or other hardware acceleration such as an FPGA 613, coupled to working memory 614 and to various interfaces 616 such as communications, control, and user interfaces. Non-volatile memory 618 stores code to implement the method and data, in particular trained neural network parameters to implement the one or more neural networks of the system such as the above described convolutional neural network. These different components may be interconnected via a bus 620.

[0070] FIGS. 7a and 7b illustrate some of the difficulties in accurately determining the number of cells in a droplet from an image of the droplet—for example a cell may be stuck to the droplet edge and/or partly hidden in shadow and/or two cells may be touching and/or a cell may have an irregular shape.

[0071] The convolutional neural network may be trained to identify the number of cells in a droplet and after training may accurately identify the number of cells in a droplet using the above described method. A large data set is advantageous for training. To increase the size of the dataset a labelled image of a droplet may be transformed e.g. by reflection/rotation as illustrated in FIG. 7b (which shows, by way of example, one image transformed into three images for training). In one implementation around 180,000 training images were available. It can be advantageous to approximately balance the number of examples in each category of image identified by the neural network (so that a likely correct answer cannot simply be obtained by selecting the most populous category). Optionally training images may be captured with different camera focus accuracy.

[0072] FIG. 8 schematically illustrates a two stage procedure for identifying and classifying a cell/entity. This technique may be used together with or independent from the above described techniques.

[0073] In broad terms a first neural network, e.g. a convolutional neural network (CNN), receives and processes image data from a captured image to identify whether a cell or other entity is present and (if present) to output data identifying a location(s) of the cell(s)/entity(entities).

[0074] Then the image data is processed using the output of the first neural network to generate a version of the image data for processing by a second neural network, e.g. convolutional neural network. The version of the image data may be a higher resolution version of the image data, and/or may be cropped around the location of the cell(s)/entity(entities); thus the version of the image data may define a smaller physical region of the image at higher resolution, including the target cell(s)/entity(entities). This may then be processed by the second neural network to classify the cell(s)/entity(entities) into one of a plurality of categories. The categories may define shapes/morphologies of a cell/entity and/or other properties as previously described. Optionally a plurality of second neural networks may be provided to classify a cell/entity according to a set of parameters each with different respective categories; the set of categories for the parameters into which a cell/entity is classified may characterise the cell/entity.

[0075] In more detail, a two-step neural network system may be used for efficient use of image data and for accurate classification of a cell/other entity. A two-step network with two different CNNs can facilitate efficient training and execution of a CNN system, potentially in real time, potentially without having to sacrifice accuracy (which could occur if the image resolution was decreased). Thus in an example of such a system: [0076] CNN Network 1—identifies if a cell/other entity is present and the location of cell/other entity [0077] CNN Network 2—classifies cell/other entity through recognition of features

[0078] CNN Network 1 may be trained to identify if a cell/entity is present with a classification head, and then within the droplet the actual location of the identified cell/entity with a regression head. It or another process then uses the coordinates of the identified cell/entity to crop and single out the cell/entity from the original high-resolution image creating a very small (e.g. 200×200) but high-resolution image of just the cell/entity, which is sent to Network 2.

[0079] CNN Network 2 may be trained to identify what the cell/entity is i.e. cell/entity type, optionally along with any other characteristics the CNN has been trained to identify.

[0080] Thus referring again to FIG. 8, at step 800 a high resolution image of a microfluidic region, e.g. a channel, is captured; this may be, e.g., a 50 megapixel image. The image resolution is reduced (802), and then optionally cropped around a droplet (804). This reduced resolution, cropped image is provided to a first neural network (NN1) which identifies whether a cell/entity is present and (if present) a location of the cell/entity in the image e.g. as (x,y) coordinates of the cell/entity (804).

[0081] The location, e.g. (x,y) coordinates of the cell/entity is then used to extract a higher resolution image of the cell/entity from the original high resolution image (806). This higher resolution image is cropped around the cell/entity and covers a region which is smaller than that of the droplet e.g. substantially just containing the cell/entity itself.

[0082] The smaller, high resolution extracted image of the cell/entity cut-out is then provided to a second neural network (NN2) which processes the image e.g. to identify and/or characterize (classify) the cell/entity.

[0083] No doubt many other effective alternatives will occur to the skilled person. For example although some example implementations have been described with reference to cells the techniques may be applied with droplets containing other entities including, but not limited to, beads, particles, and quantum dots.

[0084] Although methods and systems which capture a 2D image have been described, in principle the techniques may be employed with 1D images captured, for example, by a line-scan camera. Similarly, where there are references to capturing optical and other data from a droplet and/or its contents, this data may be collected by techniques including, but not limited to: PMT signal data, diffraction pattern data, single pixel detection data, non-optical detection methods such as electrical and acoustic signal data, and other types of data collection.

[0085] It will be understood that the invention is not limited to the described embodiments and encompasses modifications apparent to those skilled in the art lying within the spirit and scope of the claims appended hereto.