VERIFICATION PIPETTE AND VISION APPARATUS

Abstract

Manually operated pipettors, widely used in clinical, forensics, pharmaceutical research, hospital and biotech laboratories to transfer small volumes of liquid, may be subject to positional errors, operator use errors and hidden performance degradation. Manual pipette performance cannot be accepted without monitoring and reporting. This invention concerns a computer controlled vision tracking and lighting system working in conjunction with a sensor controlled fluid dispensing device and controller to confirm pipette tip positional locations during aspiration and dispensing operations with automatic monitoring of liquids entering and leaving a pipette apparatus to digitally track a manual pipetting operation with a digital output file of validated liquid transfer results. The invention may also monitor possible error conditions and prevent improper liquid transfers during the manual process.

Claims

1: A system for fluid sample processing validation, comprising in combination: a pipette including at least one tip and a fluid aspirator/dispenser coupled to said tip, said pipette having a hand grippable size and including at least one actuator thereon to cause fluid aspiration and/or dispensing relative to said tip; at least one target location; and at least one camera having said at least one fluid aspiration/dispensing target location within a field of view of said camera.

2: The system of claim 1 wherein a camera image collection actuator is coupled to said camera to cause said camera to acquire an image including said target location, said actuator also coupled to said fluid aspirator/dispenser of said pipette.

3: The system of claim 1 wherein a controller is coupled to said pipette and said camera.

4: The system of claim 3 wherein said camera actuator is coupled to said aspirator/dispenser of said pipette such that said camera acquires an image when said aspirator/dispenser is operated to transfer fluid relative to said tip of said pipette.

5: The system of claim 1 wherein said pipette includes a plurality of manual buttons thereon for actuation of said fluid aspirator/dispenser, said camera actuator at least indirectly coupled to at least one of said manual buttons for camera image acquisition when at least one of said manual buttons is actuated.

6: The system of claim 1 wherein a controller is coupled to said pipette and said camera, said controller including a protocol including a series of desired steps and with at least one of said desired steps to be performed at said target location, said controller inhibiting operation of said fluid aspirator/dispenser if a function not specified by said protocol is initiated.

7: The system of claim 6 wherein said fluid aspirator/dispenser of said pipette includes a signal transmitter and a signal receiver coupled to said controller, said signal transmitter sending a signal when actuation of the aspirator/dispenser is attempted, and said signal receiver receiving a signal allowing or blocking said fluid aspirator/dispenser at least partially dependent upon pipette tip location information gathered by said camera.

8: The system of claim 6 wherein said controller is configured to allow function of said fluid aspirator/dispenser which are consistent with said protocol and to block functions which are inconsistent with said protocol.

9: The system of claim 1 wherein said fluid aspirator/dispenser includes fluid volume control.

10: The system of claim 1 wherein said at least one target location is a known position relative to said at least one camera.

11: The system of claim 10 wherein a plurality of target locations are provided within said field of view of said camera, each of said plurality of target locations having a known position relative to said at least one camera, such that a fluid analysis protocol can occur involving multiple different target locations for fluid transfer by said tip of said pipette, and with said camera acquiring images, verifying tip location and aspiration/dispensing function, as well as a time associated with each image, for fluid analysis verification.

12: The system of claim 11 wherein said aspirator/dispenser is prevented from operating unless said camera detects said tip of said pipette at a proper location for aspiration/dispensing.

13: The system of claim 12 wherein said system includes an operator communication output which communicates a warning when a fluid aspiration/dispensing function is attempted that is outside of said protocol.

14: The system of claim 11 wherein a plurality of cameras are provided adjacent to said plurality of target locations, each of said plurality of cameras acquiring images which are utilized together at least partially to further verify pipette tip location relative to said plurality of target locations.

15: The system of claim 14 wherein said pipette includes a plurality of tips spaced apart from each other by a known fixed distance, and with said plurality of target locations including at least some target locations which are spaced apart by a distance similar to spacing between said tips, and wherein said fluid aspiration/dispensing can occur for multiple lines of said tips and said target locations substantially simultaneously.

16: The system of claim 12 wherein said at least one camera includes at least two cameras spaced from each other to provide a stereoscopic image.

17: A method for validation of processing of a liquid sample, including the steps of: moving a tip of a pipette to at least one target location; performing a fluid transfer between the tip and the target location; monitoring tip presence relative to the target location with at least one camera recording at least one image of the at least one target location; and using the at least one image to validate tip location at a time of said performing step.

18: The method of claim 17 wherein said performing step includes aspirating fluid into said pipette, and said target location includes a location of a specimen to be tested, and with said monitoring step including said camera collecting at least one image during said performing step to verify aspiration of the sample located at the target location.

19: The method of claim 18 wherein said monitoring step includes noting time when at least one image is collected showing location of the tip during said performing step.

20: The method of claim 19 including the further step of blocking fluid transfer relative to the tip when a location for aspiration/dispensing function of the pipette is out of compliance with an analysis protocol.

21: The method of claim 17 wherein said monitoring step includes collection of a video image and saving at least a portion of the video image for later use in verifying pipette tip position.

22: The method of claim 21 including the further step of coding the images with indications of when the pipette is aspirating, when the pipette is dispensing and a time stamp on the image.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0049] FIG. 1 is a perspective view of a system incorporating a workbench, a handheld pipette apparatus and a vision system according to this invention.

[0050] FIG. 2 is a perspective view of an example handheld pipette apparatus that may be used with the system shown with respect to FIG. 1.

[0051] FIG. 3 is a perspective view of an example of a multi-channel handheld pipette apparatus used with the single channel for high speed liquid transfers to wells in a microplate, or test tubes in a uniform rack.

[0052] FIG. 4 is a perspective view of an alternative system to that depicted in FIG. 1, but with a stereoscopic vision subsystem.

[0053] FIG. 5 is a flowchart showing operation of the system in a “teach” mode.

[0054] FIG. 6 is a flowchart showing operation of the system in a “recipe” mode.

DESCRIPTION OF THE PREFERRED EMBODIMENT

[0055] Referring to the drawings, wherein like reference numerals represent like parts throughout the various drawing figures, reference numeral 100 (FIG. 1) is directed to a system for processing a fluid sample in a verified manner. FIG. 1 illustrates an exemplary embodiment system 100 for collecting and monitoring the aspiration and dispensing of liquid using a handheld pipette apparatus 120, such as described above.

[0056] As shown in FIG. 1, the system 100 includes a pipetting controller 110 and the handheld pipette apparatus 120. The handheld pipette apparatus 120 may be connected to the pipetting controller 110 wirelessly, or using various communication, air, power and/or pressure and vacuum tubes. The pipetting controller 110 may be used to monitor and control the dispensing and aspiration of liquid such as described above, or just for passive documentation of pipette 120 tip 140 position (and optionally orientation) at meaningful times, such as when aspirating or dispensing.

[0057] The handheld pipette apparatus 120 may be stored in a stand 130. One or more sensors associated with or otherwise integrated with the stand 130 (and/or the handheld pipette apparatus 120) may sense when the handheld pipette apparatus 120 is secured or is otherwise placed on the stand 130 and when the handheld pipette apparatus 120 has been removed from the stand 130. In one embodiment, when the sensors detect that the handheld pipette apparatus 120 has been removed from the stand 130, the pipetting controller 110 may enable the handheld pipette apparatus 120 to aspirate and dispense liquid in the manner described above. In addition, the sensors may also indicate that the vision system and lighting system associated with the system 100 should also begin monitoring the movement and/or location of the handheld pipette apparatus 120, such as described above.

[0058] As shown in FIG. 1, the handheld pipette apparatus 120 includes a pipette tip attachment 140 and tip eject sleeve. In some embodiments, the pipette tip attachment 140 may be tapered such that pipette tips having various sizes may be coupled to the handheld pipette apparatus. The tapered configuration may also reduce the air volume between the distal end of a disposable pipette tip and the sensors that monitor the flow rate of the liquid. The pipette tip attachment 140 may also function to eject the tip from the tip adapter end. In addition, the stand 130 may be used to change pipette tips that are coupled to the handheld pipette apparatus 120. The system 100 can log and validate details such as when the tip adapter 140 had a new pipette tip attached to the pipette 120 and the location where the new tip was taken from.

[0059] The system 100 also includes a deck workbench 150. Preferably, the workbench 150 has a staging area for storing or otherwise placing the various test tubes, racks, plates, trays 160 and other labware which can hold volumes of fluid to be used as part of the recipe, testing protocol, assay, etc. Also, equipment such as disposable tips on a tip support stand can be located in this staging area. In some embodiments, the staging area workbench 150 may contain between 1-18 positional locators 162 that hold the labware such as, for example, disposable tip racks, liquid reservoirs, trays, tube racks and microplates. The workbench may be setup using locators 162 that may be screwed or pinned to the deck or implement magnets within both the deck and the locators to have a flat surface that is easily cleaned and sterilized. The locators 162 cause position of the “labware” to be at precise locations that can be correlated with the pipette 120 position information to determine what labware was present when the pipette 120 performed some step in the process (e.g. aspiration, dispensing, tip exchange, mixing, etc.). In some embodiments, shorter plates are placed nearer the camera 170 (or cameras) in order to enable the camera 170 to capture images of other labware further from the camera 170.

[0060] As described above, the labware may include bar codes that are readable by the cameras such as described above, such as to automate the process of correlating location information with labware details and fluid content details. The lights 180 may be used to increase visibility for the cameras 170 placed in various positions to illuminate the containers, racks, trays, plates and other labware to be used on the system. The lighting acts to differentiate the workbench 150 labware from the disposable pipette tip 140 on the handheld device 120. The light can be visible light or some other non-visible light frequency that is visible to the cameras 170.

[0061] The system 100 may also include a computing device 190. The computing device 190 may be used to output various notifications to the operator on a display thereof, including the volumes of liquid dispensed, the volumes of liquid aspirated, errors in recipes or methods and so on. The computing device 190 may also control the cameras 170, the lights 180 and provide instructions to the pipetting controller 110. The controller 110 could be merely software running on the computing device 190 or hardware coupled to the computing device 190.

[0062] In some implementations, the pipetting controller 110 includes a vacuum pump and reservoir, a pressure pump and a reservoir, a microprocessor controlling unit and a power supply. The pipetting controller 110 may be communicatively coupled to the handheld pipette apparatus 120 such as described above.

[0063] FIG. 2 illustrates a handheld pipette apparatus 200 according to some embodiments of the present disclosure. The handheld pipette apparatus 200 may be similar to the handheld pipette apparatus 120 described above. The handheld pipette apparatus 200 includes various buttons 210 for aspirating, dispensing, and/or mixing a liquid. The handheld pipette apparatus 200 also includes a pipette tip adapter 230 that removably supports a tip which is typically disposable and replaceable with other tips of the same or different sizes and configurations. The tip can be ejected off of the tip adapter 230 using an eject button 220. A ring 235 can be fixed to or removably attachable to the tip adapter 230 (or disposable tip), and of a unique color or other visually distinct character, to enhance tip 230 visibility to the cameras 170.

[0064] The handheld pipette apparatus 200 may also include a display 240. The display 240 may show the mode of operation of the handheld pipette apparatus 200, whether the handheld pipette apparatus 200 is in a correct position or location on the staging area of the deck 150, or provide other such notifications. A cord 250 having various vacuum and pressure tubes, as well as various power and communication cords, may be coupled to the handheld pipette apparatus 200. As an alternative, the pipette 200 could be wireless and generate vacuum/pressure through onboard equipment and communicate wirelessly to other parts of the system 100.

[0065] As described above, the handheld pipette apparatus 200 may be used to automatically fill (aspirate) a pipette tip with a desired amount of liquid. This liquid may then be dispensed into various wells of a tray such as described. The amount of liquid aspirated and dispensed can also be validated by the system 100 such as described above.

[0066] More specifically, the handheld pipette apparatus 200 may include or otherwise be associated with a micro-controller that detects the actuation of the buttons 210. For example, when an aspirate button is actuated, the vision system is activated and liquid is drawn into the pipette tip. Likewise, when a dispense button is actuated, the vision system is activated and liquid is dispensed from the pipette tip. The micro-controller may also be configured to communicate with the system computer system 190 and the pipetting controller 110 to coordinate pipetting activities. In some embodiments, when a pipette tip is attached to the pipette tip adapter 230 of the handheld pipette apparatus 200, a pressure or hall sensor may be triggered to activate the vision system for tip validation or otherwise ensure that the tip is uncontaminated and/or property installed.

[0067] The aspirate button causes liquid to be drawn into the pipette tip. It also signals the microprocessor to communicate with the pipetting controller 110 to monitor the volume of liquid being drawn into the tip as well as the volume that is accumulating in the tip. Actuation of the aspirate button also causes the vision system 170 and lighting system 180 to validate and/or record the well/tube (or other) location in the plate or tray at one of the multiple locations on the deck 150. The vision system will validate this process using the two stages described above.

[0068] In some embodiments, a dial may be present on the handheld pipette apparatus 200. The dial may be used to indicate the amount of liquid that will be aspirated or dispensed by the handheld pipette apparatus 200. In other embodiments, the desired volume may be input on an associated computing device 190 or controller 110 or a display of the pipette 120. Pressing the aspirate button also enables the micro-controller to reach the target volume set by the operator.

[0069] As the liquid is being aspirated or dispensed, the vision system 100 will validate source well/tube position, etc. if a stored recipe/method is being executed. The system 100 will also output an error if an incorrect source or destination position of the handheld pipette apparatus 200 is detected. The system 100 may deactivate the pipette buttons 210 on the handheld pipette apparatus 200 until a proper location is accessed. The computer apparatus 190 may also display the target and actual volume aspirated into the pipette tip on the computer screen interface and/or on the display screen 240 of the handheld pipette apparatus 200.

[0070] The handheld pipette apparatus 200 and the associated computing system may also record the fill volume as well as the source plate/rack ID bar code and the source volume well/tube number in a memory device.

[0071] When the dispense button is actuated, the micro-controller of the handheld pipette apparatus 200 communicates with the pipetting controller 110 to monitor liquid evacuation from the pipette tip and determines when the desired aliquot has been dispensed. Actuation of the dispense button also causes the vision and lighting system to validate and record the well/tube location in the plate or tray. The vision system may have two validation stages such as described above. The dispense button may also be used to signal the micro-controller to communicate with the pipetting controller 110 to record the target and actual volume of the desired aliquot in a memory.

[0072] When the tip eject button 220 is actuated, the handheld pipette apparatus 200 may activate software to record the tip ejection to ensure that contaminated tips are not reused in a subsequent liquid transfer step. This button may also signal the recording of the source plate/rack ID bar code and destination volume well/tube number in a memory device.

[0073] With particular reference to FIG. 3, a multi-channel pipette 300 is disclosed as an accessory to substitute for or use along side of a pipette 200 (FIG. 2) or the pipette 120 (FIG. 1). This multi-channel pipette 300 includes an eject button 320, operation buttons 310, a display 340, and a tip assembly 330, as well as associated lines 350 (in non-wireless embodiments) which are each similar to those of other embodiments described herein, except that the tip assembly 330 is configured for faster operations and higher throughput and improved duty cycle.

[0074] The multi-channel tip assembly 330 preferably includes an array of tips which are in a line and with spacing therebetween which matches spacing between wells 164 (FIG. 1) on a plate-type fluid support 160 or which align with multiple test tubes housed within a test tube rack type fluid support 160. Most preferably, such fluid supports 160 are provided in rectangular arrays with a certain number of uniformly spaced fluid locations in each row and in each column. Preferably the number of tips on the tip assembly 330 of the multi-channel pipette 300 match either a total number of fluid locations in a row or a total number of locations in a column of one such fluid support 160. In this way, the multi-channel tip assembly 330 of the multi-channel pipette 300 can aspirate or dispense fluid in multiple locations associated with the fluid supports 160 simultaneously, greatly increasing speed with which laboratory procedures performed by the system 100 can be achieved.

[0075] As an alternative, the multi-channel tip assembly 330 could have a number of tips which is some whole fraction of the number of fluid supports 160 in a row or in a column. For instance, if the fluid support 160 in the form of a microplate could have an 8×12 array of wells 164 provided as fluid locations within the tray type fluid support 160. The tip assembly 330 could then have eight tips for maximum effectiveness, but could also be efficient with four tips or two tips. Other less efficient numbers of tips could also be provided and still provide some benefit over single tipped pipettes 120, 200. As another option, the tip assembly 330 could have a two-dimensional array of tips such as a 2×8 array (or even an 8×12 array) so that fluid locations can be even more expeditiously accessed.

[0076] With the multi-channel pipette 300, the vision system 100 utilize the cameras 170 not only to pinpoint the location of the tip assembly 330, but also the orientation of the tip assembly 330, so that well or other fluid location specific information is accurately gathered by the vision system disclosed herein to facilitate and verify accuracy of tip location adjacent to fluid locations.

[0077] With particular reference to FIG. 4, an alternative embodiment system 400 is disclosed as an alternative to the system 100 (FIG. 1). In this embodiment similar structures are provided with similar reference numerals. The primary unique feature of the system 400 is the provision of a stereoscopic camera assembly 470 of two or more cameras, in place of (or in addition to) one or more separate cameras 170 (FIG. 1). Thus, this system 400 utilizes a controller 410 which interfaces with a handheld pipette 420 held upon a stand 430. A tip attach/eject sleeve 440 is provided on the pipette 420. A dock/workbench 450 is provided with multiple locators 265 where fluid supports 460, such as micro-plates 462, racks 466 to support test tubes 468 or other fluid supports can be precisely located, preferably in arrays of similar labware. In this manner, position information on the workbench 450 can be correlated with the containment of specific fluids or other equipment (e.g. removable tips/sleeves 440), so that particular locations detected by the camera 470 can be correlated with the aspiration or dispensing of particular liquids, or the selection of a particular tip 440, or other actions.

[0078] The camera 470 is preferably a stereoscopic camera that has a left camera sensor 472 and a right camera sensor 474 (or optionally a triscopic camera box to achieve a similar result). Optionally an auxiliary camera 476 can also be provided, such as in the form of a video camera or still camera. The camera sensors 472, 474 can be still, video or combined type cameras. The left and right camera sensors 472, 474 are spaced a known distance apart and have their signals integrated in a standardized fashion so that a three-dimensional image is constructed stereoscopically.

[0079] The overall stereoscopic camera system 470 also preferably includes a light bar 478 which extends horizontally, and optionally also a vertical light bar 479 which in this embodiment also acts as a tower to which the lines extending from the pipette 420 can attach, so that these lines stay out of the way of the workbench 450, but allow the pipette 420 to reach all of the locations on the workbench 450. If needed, a tensioning system can be provided to play out and draw up excess amounts of line to further keep these lines from being a nuisance or preventing the pipette 420 from reaching all of the required locations on the workbench 450. By providing one horizontal light bar 478 and one vertical light bar 479, shadow minimization is to a greater extent achieved than with lights of a single point, line or other limited orientation. Other lighting configurations could alternatively be provided. The system 400 can also work without lighting, relying on ambient light, infrared radiation, etc.

[0080] With particular reference to FIGS. 5 and 6, flow charts are provided which show protocols and other methods which can be implemented by the systems 100, 400 described herein. In FIG. 5 a flow chart defines the steps in a “teach mode” according to one form of this invention. In FIG. 6 a similar flowchart, but for a “recipe mode” is depicted. As a first step disclosed in these procedures, the systems 100, 400 can be configured so that a microprocessor serial number validation system is provided. The systems 100, 400 are prevented from working unless a valid number is first inputted into the system. Such a number acts as a “key” for the system and can enable payment/billing systems, so that the systems 100, 400 generates a payment obligation for each session in which they are put to use, and also allow for case numbers or other identifying information to be logged into the various databases at the beginning of a particular session. Most preferably, the systems 100, 400 can still operate in a demo mode when no appropriate number or other key is inputted in this first step. Such a demo mode can be used for training purposes or to communicate to others how the systems 100, 400 works when in full operation.

[0081] The processes are further implemented as illustrated in the respective flow charts shown in FIGS. 5 and 6. In a teach mode (FIG. 5), one is able to teach the system 100, 400 what the proper specific locations are, volumes are, and other details are for running a particular assay, testing protocol, or other “recipe.” Once the system 100, 400 has been taught the proper “recipe,” this enables the system 100, 400 to then later monitor the operation of the system 100, 400 and compare it to the proper operation. If anomalies are detected, error codes can be attached to validation information, or the aspirate/dispense functions (and other functions) can optionally be prevented (at least initially), with typically the opportunity for override, but typically not allowing any tampering with the validation system which carefully logs pipette 120, 200, 300, 420 position information, volume information, and other validation information, typically also with respect to a timestamp and any fluids, etc. identified as being at any of the locations visited by the pipette 120, 200, 300, 420. Once the system 100, 400 has been “taught” a recipe/protocol, the recipe/protocol can optionally be saved and/or transferred to other similar systems 100, 400 so that the other systems 100, 400 could be preprogrammed to provide validation and other assistance with the same recipes/protocols, thus potentially having multiple recipes/protocols the systems 100, 400 are ready to support.

[0082] As shown in the flow charts, interactive information can be provided onto the display 240, 340 of the pipette 200, 300, or onto the display associated with the computing device 190 (FIG. 1). This allows a user (or other person) to be provided with instructions, and to allow for inputting of information into the system 100, 400, such as a case number, other identifying information, operator information, and any other information relevant to the recipe or other protocol being performed.

[0083] Multiple different protocols can be loaded into the system 100, 400 in preferred embodiments, so that different “recipes” can be followed by the system 100, 400, such as with the different recipes selectable from a list of recipes for which the system 100, 400 has been “taught.” As an alternative, the system 100, 400 could be configured to just perform one type of recipe, such as in a particular instance where a common recipe is to be performed multiple times. In one embodiment a certified entity first “teaches” the system 100, 400 to perform the recipe and then other operators use the system 100, 400 to practice the method defined by the recipe, and with the system 100, 400 verifying proper operation.

[0084] The system 100, 400 is disclosed along with a manually held and manipulated pipette 120, 200, 300, 420. This pipette 120, 200, 300, 420 could be electric or non-electric. Also, the pipette 120, 200, 300, 420 could alternatively be robotized with appropriate carriers and actuators coupled to the pipette 120, 200, 300, 420 and causing the pipette 120, 200, 300, 420 to move to desired fluid locations and perform desired functions in an automated fashion. With such a robotic system, the cameras 170 would still be utilized to verify that the tip of the pipette 120, 200, 300, 420 truly is in the required locations but various procedures, such as an aspirate procedure, dispense procedure, mix procedure, tip change step, or other procedure was performed at a proper location. Validation that a robotic system as a variation to the system 100, 400, is properly operating could thus be provided with this invention as well.

[0085] The system described herein has various advantages over current systems. For example, the combination of the lights and cameras of the system described herein, along with a sensor controlled handheld pipette apparatus, enable the validation of information needed for quality and defensible data in a lab. This data includes information about the picture and source information of the liquid, the picture and destination information of the liquid as well as position data and actual aspiration and dispensing volumes, bar codes or plates and tips, lot numbers of tips, operator information, runtime information, method information, and time and date stamps.

[0086] In some embodiments, the handheld pipette apparatus 120 includes sensors that communication with the pipetting controller 110 that monitor the liquid as it enters and leaves the pipette tip. This monitoring may be done in real time and with a closed loop system. For example, the sensors may include a MEMS flow sensor with associated solenoid valves that provide access to vacuum or pressure that move liquid into or out of the disposable tip. Using the sensors, the system 100 described herein directly measures the flow rate of the liquid entering or leaving the pipette tip. The measured value may be accumulated over time in order to determine the actual liquid volume. As discussed above, the system 100 may incorporate temperature sensors so the system can account for changes in the viscosity of the liquids due to temperature changes.

[0087] This system is also configured to notify the operator of an error before the operator transfers liquid from one location to another location. For example, the system 100 may prevent the handheld pipette apparatus 120 from dispensing or aspirating liquid when it is determine the handheld pipette apparatus 120 is in an incorrect location compared to a recipe or other protocol. The system 100 may also flag a well or tube as being contaminated by an operator error and therefore need to be skipped or rerun.

[0088] This disclosure is provided to reveal a preferred embodiment of the invention and a best mode for practicing the invention. Having thus described the invention in this way, it should be apparent that various different modifications can be made to the preferred embodiment without departing from the scope and spirit of this invention disclosure. When structures are identified as a means to perform a function, the identification is intended to include all structures which can perform the function specified. When structures of this invention are identified as being coupled together, such language should be interpreted broadly to include the structures being coupled directly together or coupled together through intervening structures. Such coupling could be permanent or temporary and either in a rigid fashion or in a fashion which allows pivoting, sliding or other relative motion while still providing some form of attachment, unless specifically restricted.