COUPON READER
20170300779 · 2017-10-19
Assignee
Inventors
Cpc classification
H04N23/74
ELECTRICITY
International classification
Abstract
When reading a coupon that displays a pattern when a target substance is detected, recognition of the target pattern can be hindered by nonuniform illumination of the coupon. In one aspect, methods and devices are disclosed for uniform illumination of a coupon using a negative axicon lens and a light diffusing assembly. In another separate aspect, methods are disclosed for mathematically compensating for nonuniform illumination of a coupon.
Claims
1. A method of reading a coupon channel that displays a pattern after being exposed to a target substance, said method comprising: a) providing a device having a computer readable memory, a digital camera, a logic assembly and a user interface; b) providing an indicator pattern representation, representative of an exposed and at least partially developed coupon channel control line; c) placing said digitized target pattern into said memory; d) exposing said coupon channel to a test fluid mixture; e) automatically repeatedly using said digital camera to take a digital image of said coupon channel after said exposure; f) after each of said digital images is taken, automatically using said logic assembly to compare it to said indicator pattern representation to determine if it passes a similarity threshold; and g) in response to a digital image passing said similarity threshold, automatically providing a human perceptible indication that the target substance has been detected.
2. The method of claim 1, wherein said coupon channel includes a reference pattern, that develops after exposure to said test fluid mixture, without regard to the presence of said target substance in said test fluid mixture, and wherein said target pattern representation is provided by digital imagery from said digital images repeatedly taken of said coupon channel.
3. The method of claim 1, wherein said target pattern representation is provided by taking a digital photograph of a coupon channel that has been exposed to said target substance after a predetermined development time has elapsed since said exposure.
4. The method of claim 1, wherein said target pattern representation is one of a set of target pattern representations stored in said computer readable memory and is user selected using said user interface.
5. The method of claim 4, wherein said target pattern representation is an abstract description of said target pattern.
6. The method of claim 4, wherein said target pattern representation is an image taken of a fully developed indicator pattern.
7. The method of claim 1, wherein said device has a position guide for assisting a user in placing said coupon into a predetermined geometric relationship to said camera.
8. The method of claim 1, wherein said device includes a substantially light tight enclosure, which is lighted with an internal lighting assembly, and wherein said coupon is placed into said enclosure and illuminated with said internal lighting assembly prior to repeatedly using said digital camera to take a digital image of said coupon, after said exposure, thereby illuminating said coupon in a manner that is repeatable between coupon exposures.
9. The method of claim 1, wherein said internal lighting assembly includes at least one light source and a light diffusing assembly, causing said coupon to be evenly illuminated.
10. The method of claim 1, wherein said similarity threshold is user adjustable by said user interface.
11. The method of claim 1, wherein automatically using said logic assembly to compare said digital image to said target pattern representation to determine if it passes a similarity threshold, includes finding the coefficient of the target pattern pixels that minimizes the sum of the squared differences between a nonlinear function representing the indicator pattern and the corresponding pixels of the digital image, and thresholding this coefficient to determine if a test pattern of sufficient clarity has formed.
12. The method of claim 1, wherein a recommended time to develop is provided by the manufacturer for said coupon, said time to develop being a time duration recommended for users to wait after said coupon channel is exposed to a test fluid mixture, prior to viewing the coupon to form a conclusion concerning the presence of said target substance, and wherein said automatic repeated usage of said digital camera to take a digital image of said coupon after said exposure to said test fluid mixture begins at least one minute prior to the passing of said recommended time to develop, after exposure to said test fluid mixture.
13. The method of claim 1, wherein a first digital image is formed no more than three minutes after said coupon channel is exposed to said test fluid mixture.
14. The method of claim 1, wherein a first digital image is formed no more than 1 minute after said coupon channel is exposed to test fluid mixture.
15. The method of claim 1, wherein said threshold is user adjustable and further including the initial step of adjusting said threshold.
16. The method of claim 1, wherein said coupon channel is hosted on a coupon having a single channel only.
17. The method of claim 1, wherein said coupon channel is hosted on a coupon having multiple channels.
18. An apparatus for reading coupons that display a pattern after the passage of a predetermined development time after being exposed to a target substance, said system comprising: a) a computer readable memory; b) a digital camera; c) a logic assembly, controlling and responsive to said digital camera; d) a user interface; e) a user perceptible output; f) a substantially light tight chamber; g) a positional guide for placing one or more coupons into said substantially light tight chamber in predetermined and repeatable positions; h) a lighting assembly for substantially evenly illuminating said predetermined and repeatable positions; i) wherein said logic assembly controls said camera to repeatedly take an image of said coupon, after being user activated and after each of said digital images is taken, automatically comparing it to a target indicator pattern representation to determine if it passes a similarity threshold, and in response to a digital image passing said similarity threshold, automatically activating said user perceptible output.
19. The apparatus of claim 18, wherein said internal lighting assembly includes at least one light source and a light diffusing assembly, causing said coupon to be even illuminated.
20. The apparatus of claim 19, wherein substantially no light travels directly from said at least one light source to said coupon, but all of said light illuminating said coupon has reflected off of a reflective surface on its path to said coupon.
21. The apparatus of claim 19, wherein said at least one light source is substantially opposed to said positional guide, and wherein light from said light source is redirected by a negative axicon lens.
22. The apparatus of claim 18, wherein said light tight chamber has reflective walls, to reflect light from said light source.
23. The apparatus of claim 22, wherein said reflective walls extend outwardly as they extend from said light source.
24. The apparatus of claim 22, wherein said reflective walls are coated with a white reflective coating.
25. The apparatus of claim 22, wherein said reflective walls are coated with a colored coating.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0018]
[0019]
[0020]
[0021]
[0022]
DETAILED DESCRIPTION
[0023] Referring to
[0024] In one embodiment, means are provided to introduce a target pattern representation (an image of a developed coupon) into a computer readable memory 14 of reader 10. This data entry may be performed by placing a developed coupon or a control section into reader 10, and using user interface 12 to command reader 10 to use a digital camera 16 to take a digital photograph of the developed coupon and store it in the computer readable memory 14, properly labeled as a digitized image of a target pattern. In another preferred embodiment, reader 10 is provided with digitized target images already stored. Otherwise digitized target images may be introduced into reader 10 by way of the USB port {not shown}, or may be downloaded by way of a cellular network transceiver 18.
[0025] After an exposed coupon is introduced into reader 10, the microcontroller 24 controls the digital camera 16 to repeatedly form images of each coupon channel and effectively compare each image with the digitized target image stored in computer readable memory 14.
[0026] Coupon manufacturers typically identify a specific time period that should elapse between exposing the coupon to the substance to be tested and reading the results on the coupon. In some cases, results may be visible before this predetermined development time has elapsed and time may be wasted by continuing to wait for the entire predetermined time period. The ability of the coupon reader 10 to determine when an indicator pattern has been detected and to alert any nearby people that detection has occurred helps to eliminate this wasted time. It will be appreciated that this ability is particularly important when the coupon reader is being used to detect the presence of toxic or harmful compounds in a public safety context, where time may be of the essence.
[0027] Many other algorithms, including least squares detection and various linear algorithms are used in alternative embodiments.
[0028] Prior to channel analysis, all embodiments require that the coupon image is recognized in the digital image being analyzed. Immediately following coupon image capture, image recognition routines analyze the captured image and identify channel locations based on the detection of characteristic coupon features. In embodiments, the user has entered the coupon type, and this information facilitates the recognition of the channel locations. The processing described below relates to each channel analyzed.
[0029] Known optical features of the coupon are pre-loaded into the device before use. Image recognition tools identify the coordinates of these characteristic features for each different coupon type. The pixel coordinates of these features provide reference points and allow each coupon image to be overlaid by a coordinate map which can then be used to locate the assay channels.
[0030] Upon positive recognition and satisfaction of location criteria, coupon analysis is allowed to continue. Upon detection of an abnormal situation, such as the coupon being inserted 90 or 180 degrees from the correct orientation, the user is alerted to the type of error. Once the error is remedied and location criteria are satisfied, sample analysis procedures may proceed.
[0031] In one embodiment, each time a reader 10 processes a coupon, the results are time, position and device code stamped, with position being provided by a GPS reader 20 and time being provided by a clock 22 controlled by microcontroller 24, and stored in computer readable memory 14. In a preferred embodiment, this information is uploaded periodically via cell network link 28 using cell network transceiver 18, to a remote station 30. In one embodiment, remote station 30 includes a display and may be commanded to display a map of an area, showing the locations, times, results and device codes of each coupon reading.
[0032] Referring to
[0033] Referring now to
[0034] In one embodiment, the light diffusing assembly comprises side walls 36 which form a rectangular cross-section approximately 8 cm on a side. The side walls 36 meet the coupon surface 44 at angles that maximize uniformity of illumination over the coupon surface and compensate for the light source's off-center location. In a preferred embodiment, the included angle 46 between the side walls 36 and the coupon surface 44 is:
[0035] Face 1=70 degrees
[0036] Face 2=70 degrees
[0037] Face 3=70 degrees
[0038] Face 4=70 degrees
[0039] In one embodiment, the distance between the coupon surface 44 and the negative axicon lens 40 is 15 cm.
[0040] Diffuse reflection may be obtained from the reflective interior surfaces 38 by covering them with a flat or gloss white paint, or by constructing the structure using a white polymer. However, commercially available white polymers may absorb a significant amount of the optical power impinging on them, and this may need to be taken into consideration.
[0041] Optionally, the reflective interior surfaces 38 may be covered with a colored coating rather than a white coating. A colored coating may be desirable in circumstances where a specific light spectrum distribution is preferred for illuminating certain types of coupon. In a further option, the reflective interior surfaces may be covered by a hybrid coating that provides a mixture of specular and diffuse reflection.
[0042] Axicons are special lenses that have at least one active surface that is conic in shape. They will typically take a point source of light and transform it into either a line or circle of light. A negative axicon lens is used for coupon reader 10. Typical commercially available axicon lenses are designed to form a specific geometric shape, and for that reason are considered “positive axicons.” The negative axicon, in contrast, takes a point of light and transforms it into an annular fan of light. Instead of a centrally located solid cone of material as in a positive axicon, the negative axicon has a conic pocket into which source light expands.
[0043] Refraction at the conic top surface and at the optionally planar lower surface cause rays from a point source to be distributed over a range of angles determined by the conic profile, the distance of the source from the lens, the lens thickness, and the refractive index of the lens material. Angular emission of the lens can be tuned so that no light hits the coupon surface 44 without first reflecting off the reflective interior surfaces 38. That is, in a preferred embodiment, no light passing through the negative axicon lens 40 falls directly on the coupon surface 44, particularly at the spot immediately below the lens where a hotspot would normally occur. In this way, all light impinging on the coupon surface 38 is diffuse light re-emitted or reflected by the reflective interior surfaces and the uniformity of illumination over the coupon is greatly improved, even if the coupon is in close proximity to the light source and lens.
[0044] It is not necessary that the conic profile of the negative axicon lens 40 be a simple cone. By adjusting the surface profile of the lens, the distribution of emitted optical power can be modified in an angular sense. Since the objective of the negative axicon lens and light diffusing assembly is to provide diffuse illumination, minor errors in the surface profiles are not critical. In one embodiment, the lenses are manufactured on a CNC lathe, followed by vapor polishing to remove macroscopic tooling marks.
[0045] In one embodiment, a negative axicon lens with planar output surface for supplying optical power to the reflective interior surfaces is constructed of clear polycarbonate. In one embodiment, the negative axicon lens has the following dimensions: the overall height of the lens may be 3.5 mm, the diameter may be 10 mm, and the maximum depth of the cone may be 2.5 mm.
[0046] In one embodiment, the cone profile is described as: Radius=1.98543z−0.18142z.sup.2+0.04361z.sup.3(where z is the distance in mm from the cone's vertex).
[0047] In one embodiment, nonuniformity of illumination across the coupon surface is compensated for by color-based ratiometric analysis of pixel data. Most color cameras provide at least three color channels; typically red, green, and blue. It has been found that light intensity variations over a coupon due to geometric issues are very similar in the three color bands. Most lateral flow immunoassay coupons use colloidal gold as the line colorant and its peak absorption is typically in the wavelength range of 540 nm to 580 nm, which is principally apparent in the camera pixels sensitive to green and red. It has been discovered that by forming a pixel-by-pixel ratio of color channel intensities, a remarkable reduction in background nonuniformity is seen. For the common colloidal gold colorant, the red and green channels may be most useful in this regard, but creating a ratiometric intensity variable using other color channels may be preferred for other colorants.
[0048]
[0049] In one embodiment, a new variable is created when the green pixel intensities are divided by the corresponding red pixel intensities and the resultant curve is normalized to an arbitrary value of 1.0 (or 100%) at some arbitrary pixel location distant from the test and control lines. The resulting ratiometric variable is also shown in
[0050] The fluid dynamic and chemical processes that create the test and control lines are typically identical or very similar. For most coupons, even those with multiple identification channels, all test and control lines produce pixel signal responses that are similar in shape and substantively differ only in overall intensity. Therefore, the relative concentration of a targeted material may be estimated by simply comparing the maximum deviation of the ratiometric signal from background and comparing it to the maximum deviation of the fully-formed control line. However, this may not be the most suitable method for test lines that have a small peak intensity because there may be too high of a signal to noise ratio to determine the peak value accurately.
[0051] In addition, the test and target lines are of finite width and vary in intensity over the line's width. Therefore, the quantity of target material in the sample is more appropriately proportional to the integrated total response over the entire line width. Using the integrated response over the entire line width will also serve to smooth out pixel signal noise. For example, if a signal is averaged over 100 pixels, random background noise as a fraction of the mean signal value will typically be reduced by a factor of 10 times compared to the noise associated with a single pixel measurement process.
[0052] In one embodiment, the following mathematical strategy and corresponding algorithm automatically provide a measure of the test and control line integrated areas and the pixel locations of each. The control and test line locations may indicate whether the assay coupon was correctly assembled or not, or if it was subsequently mishandled by the user.
[0053] The coupon typically includes a control line, which develops even in the absence of the target substance to indicate that the coupon is functioning properly and that the correct procedures were used to expose and develop the coupon. In one embodiment, the control line is used as the source of reference information that permits the coupon reader to determine whether a test line is present or absent. In other embodiments, a different source of reference information may be used, such as a pre-printed line on the coupon, data from a different coupon, or an abstract representation of a developed control line, showing the expected result when the tested for substance is present.
[0054] Because the test and control lines extend across the width of each coupon channel, the pixels are divided into columns running parallel to the test and control lines and then summed across each column. Each column sum defined in this way is identified by pixel position, and the value of the sum at that position is the pixel signal value for that position. The pixel signal values associated with a coupon's test and control lines can be approximated as the sum of a slowly varying background signal and a characteristically-shaped nonlinear absorbance signature, Y*(i), where “i” is pixel position. The aforementioned ratiometric signal between approximate pixel positions 400 and 500 in
Y.sub.f(i)=a+b×i+c×Y*(i)
[0055] In this equation, “a” is the average baseline value, “b” is the rate of baseline change per pixel, and “c” is a measure of the goodness-of-fit of experimental data to the reference signature, Y*(i). To obtain an unbiased measure of this goodness-of-fit, we minimize the sum of squares of the deviations between the experimental data and the curve fit, Y.sub.f(i),
[0056] The procedure determining the values of the parameters “a”, “b,” and “c” to minimize η are well-known in statistical mathematics. When this procedure is performed, it is found that the best-fit value of “c” is given by the following equation:
C=K.sub.1×S.sub.iY+K.sub.2×S.sub.Y−K.sub.3×S.sub.YS
[0057] The values K.sub.1, K.sub.2, and K.sub.3 are constants that involve the reference signature Y*(i), and are easily calculated by someone knowledgeable in statistical mathematics. The other equation factors are calculated for each sectional pixel array of N+1 pixels length, [0, i, N], to be examined as follows:
[0058] The three summations are easily and quickly calculated and the constants K.sub.1, K.sub.2, and K.sub.3 are only calculated once: After that they may be stored as look-up values for the channel. If the pixel data presented for comparison is the reference pixel array Y*.sub.i itself, the value of “c” will be exactly 1.0 at the pixel location identical to the midpoint of the control line.
[0059] As previously discussed, the test line in most cases is essentially identical in shape to the control line, but less intense overall. Hence, when the value of “c” is computed by scanning the test line section of the coupon channel pixel data with an array of [0, N] size, it will have maximum values when the signal data most closely matches the control line shape Y*(i). This allows identifying the location of each test line center point while the value of “c” represents the relative integral size of the discovered line as compared to the control line. For the example ratiometric pixel signal data in
This method is favored for several reasons: [0060] Small variations in the baseline are common due to lighting variations, shadows, and residual color in the coupon substrate. This method automatically estimates and corrects for a non-constant baseline. The integral value “c” is an unbiased best estimate of the baseline-corrected integral value. [0061] Due to the use of all data points associated with a line, signal-to-noise is improved and a wide dynamic range is realized of as much as 100 to 1. That is, a test line with a value of “c” on the order of 0.01 may be detected. [0062] The method allows development of a calibration curve that quantitatively correlates the targeted material's concentration with the test line integral value. [0063] The method allows a quantitative determination that a test line's response is statistically above background noise. [0064] The method provides the lines' peak value location, which is useful in identification of manufacturing or user errors. [0065] While the method can be applied to raw data as well as ratiometric data, the integral value from ratiometric data provides a measure of signal strength that is independent of absolute light intensity or light exposure time as long as the color channels all change proportionately the same when these factors are varied. This is a reasonable assumption for changes in exposure time or temperature, since both the control line and test line will see the same environmental shifts.
[0066] In a preferred embodiment, for a multichannel coupon the three constants K.sub.1, K.sub.2, and K.sub.3 are empirically determined for each individual channel by running one or more assays and calculating these least-squares constants for each channel's control line. This may be of particular value if there are differences in channel construction or if assay reagents are significantly different among the channels.
[0067] In addition, in a multi-channel coupon the channel control lines may be compared one against another and these relative “c” values stored in the device's memory. During later use, the control lines can be compared to each other to see if any have deviated from the expected relative “c” values. The device may then issue a warning to the user that there may be a problem with one or more channels.
[0068] In a preferred embodiment, digital images of the coupon are repeatedly taken over a period of time, starting shortly after addition of the sample and well before the time period suggested by the manufacturer has passed. These images are processed sequentially to determine if a test line has developed. As soon as detection positive result is found, a human perceivable signal is given. Depending on the coupon type, it is possible to gather useful information as early as 1 minute after development has been initiated. Some explosives coupons have a peak color at 2 minutes after application of a test fluid, and noticeably fade after 3 minutes. Accordingly, the time after exposure at which a first digital image is formed may be as little as 10 seconds or even 1 second. The techniques described herein have been shown to reduce the time to detection for bio-coupons to as little as 5 minutes, a third of the time period that the manufacturer recommends waiting, before evaluating an exposed coupon.
[0069] Accordingly, in embodiments the taking of digital imagery begins well before the manufacturers development time has passed. In various embodiments, the taking of digital imagery begins at least 4, 3, 2 and 1 minute prior to the passing of the recommended time to develop, after exposure to said test fluid mixture. Described in different terms, in various embodiments a first digital image is formed no more than 1, 2, 3 and 4 minutes after the coupon channel is exposed to said test fluid mixture.
[0070] In a further inventive feature, digital imagery is also repeatedly taken of the control section of each channel, beginning shortly after exposure to a test fluid mixture. This permits early detection of a coupon that has been ineffectively exposed to the test fluid mixture. As soon as ineffective exposure is determined, the test can be terminated and a new test begun, thereby greatly decreasing the time before an accurate reading is determined.
[0071] In a further embodiment, the sequence of images are integrated together, to enhance the signal to noise ratio and, in some instances, provide earlier detection than would otherwise be possible. In one variant, time integration is performed on the raw pixels, but in another least squares fitting as described above is applied to the data from each digital image and the multiple results are then integrated over time.
[0072] The invention may be embodied in other specific forms besides and beyond those described herein. The foregoing embodiments are therefore to be considered in all respects illustrative rather than limiting, and the scope of the invention is defined and limited only by the appended claims and their equivalents, rather than by the foregoing description.