ANALYSIS METHOD

Abstract

A computer implemented method of analysing assays performed at respective assay sites of an array or microarray that comprises a plurality of assay sites, the method comprising receiving at least one image of the assay sites of the array or microarray; for each of the assay sites, processing the at least one image to determine at least one metric representative of the degree of reaction at that assay site; for each of the assay sites, determining one or more parameters for that assay site, wherein the one or more parameters for at least one of the assay sites of the array or microarray are different from the one or more parameters for at least one other of the assay sites of the array or microarray; and for each of the assay sites, determining an extent of the reaction at that assay site from the at least one metric for that assay site and the one or more parameters for that assay site. Also disclosed is a correspondingly configured computerized analysis system.

Claims

1. A computer implemented method of analysing assays performed at respective assay sites of an array or microarray that comprises a plurality of assay sites, the method comprising: receiving at least one image, the at least one image collectively or individually imaging the plurality of assay sites of the array or microarray, processing the at least one image to determine at least one metric representative of the degree of reaction at that assay site; for each of the assay sites, determining one or more parameters for that assay site, wherein the one or more parameters for at least one of the assay sites of the array or microarray are different from the one or more parameters for at least one other of the assay sites of the array or microarray; and for each of the assay sites, determining an extent of the reaction at that assay site from the at least one metric for that assay site and the one or more parameters for that assay site.

2. The method of claim 1, wherein: the parameters comprise one or more criteria, thresholds or ranges; each criterion, threshold or range is indicative of a different degree of reaction; and the determining of the extent of the reaction at an assay site comprises determining whether or not the at least one metric for that site is above or below the one of more thresholds or within or out with the one or more ranges or does or doesn't meet at least one of the criteria to determine the degree of reaction.

3. The method of claim 1, wherein the parameters for a site depend on one or more of: an assay type performed at the site, a location of the site on the array or microarray, and/or a targeted assay type

4. The method of claim 1, wherein the at least one metric for a site comprises or is representative of pixel intensities for an area of the image representing a spot at that site, the spot being formed by reaction of an analyte and having a property indicative of the degree of reaction.

5. The method of claim 4, wherein the at least one metric for a site comprises signal to standard deviation ratio or a delta that is the difference between mean pixel intensity value for the spot and a mean background pixel intensity value.

6. The method of claim 1 comprising at least one of: cropping the image around the array of sites; and/or gridding the image into segments wherein each segment encloses a spot and/or assay site.

7. The method of claim 6, wherein at least one of: the cropping, gridding and/or the determining of the one or more metrics for the assay site uses common parameters that are shared with one or more or each or every other spot or site with the same assay type.

8. The method of claim 1, wherein the parameters for metrics for determining an extent of the reaction at the assay site are unique for that site.

9. The method of claim 4, wherein the method comprises detecting or identifying the parts of the images corresponding to the spots in the reaction sites comprising one or both of: detecting or identifying shapes in the image that comprise circles of a diameter within a predefined interval; and/or edge detection to determine the edges of the spots in the at least one image.

10. The method of claim 9, comprising filtering out identified spots having a measure of pixel intensity less than a threshold; and/or outside a predefined geometric area corresponding to the location of the assay sites on the array or microarray and positioned based on at least one other identified spot, control spot, or reference point on the array or microarray.

11. The method of claim 1, wherein the parameters used to determine the extent of the reaction at the assay site comprise at least one of: parameters received from a user input device and/or parameters retrieved from a data store corresponding to one or more identifiers associated with the array or microarray or one or more or each of the assay sites or the assays being performed therein.

12. The method of claim 11, wherein the identifiers are obtained by an input device comprising at least one of a user input device for receiving user input, a barcode reader, a QR code reader or other machine readable code reader, an RFID tag reader, and/or an infra-red signal reader.

13. The method of claim 1, wherein the determining of the extent of the reaction at the assay site comprises performing one or more logic tests to the at least one metric for that assay site and the one or more parameters for that assay site, where the result of the one or more logic tests is the extent of the reaction at the assay site comprising an indication of at least one of: whether or not there has been a reaction at that assay site; a degree of reaction at that assay site; whether or not there is activity at that assay site; a degree of activity at that assay site; and/or whether or not an analyte is present at that assay site.

14. An assay analysis system for analyzing assays performed at respective assay sites of an array or microarray that comprises a plurality of assay sites, the assay analysis system comprising at least one processing device, data storage and a communications system for receiving images and outputting an indication of an extent of a reaction at an assay site; the processing device being configured to: receive at least one image, the at least one image collectively or individually imaging the plurality of assay sites of the array or microarray, processing the at least one image to determine at least one metric representative of the degree of reaction at that assay site; determine, for each of the assay sites, one or more parameters for that assay site, wherein the one or more parameters for at least one of the assay sites of the array or microarray are different from the one or more parameters for at least one other of the assay sites of the array or microarray; and determine, for each of the assay sites, an extent of the reaction at that assay site from the at least one metric for that assay site and the one or more parameters for that assay site.

15. The assay analysis system of claim 14 comprising: at least one imaging device configured or configurable to collect at least one image of the array or microarray and communicate the at least one image to the processing system; and an output device configured to receive the an indication of an extent of a reaction at an assay site from the processing system and to output the an indication of an extent of a reaction at the assay site.

16. A computer program product comprising a non-transient computer usable medium having computer readable instructions embodied therein, the computer readable instructions being such that when implemented on a processing system, cause the processing system to perform an assay analysis for analyzing assays performed at respective assay sites of an array or microarray that comprises a plurality of assay sites that comprises: receiving at least one image, the at least one image collectively or individually imaging the plurality of assay sites of the array or microarray, processing the at least one image to determine at least one metric representative of the degree of reaction at that assay site; determining, for each of the assay sites, one or more parameters for that assay site, wherein the one or more parameters for at least one of the assay sites of the array or microarray are different from the one or more parameters for at least one other of the assay sites of the array or microarray; and determining, for each of the assay sites, an extent of the reaction at that assay site from the at least one metric for that assay site and the one or more parameters for that assay site.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0053] These and other aspects of the present disclosure will now be described, by way of example only, with reference to the accompanying Figures, in which:

[0054] FIG. 1: a schematic of a computerized assay analysis system;

[0055] FIG. 2: an example of an annotated image of assays on an array collected using the system of FIG. 1;

[0056] FIGS. 3 to 7: charts showing the variation of assay attributes for a specific metric used in a spot grading process;

[0057] FIG. 8: an overview of an assay analysis performed by a computerized assay analysis system, such as that of FIG. 1;

[0058] FIG. 9A: an overview of an image analysis that forms part of the assay analysis shown in FIG. 8;

[0059] FIG. 9B: a detailed flowchart of the image analysis of FIG. 9A;

[0060] FIG. 10: another flowchart of the image analysis of FIGS. 9A and 9B;

[0061] FIG. 11: a schematic of a grid matching process performed as part of a cropping process;

[0062] FIG. 12: a schematic of a grid for a micro-array produced by the gridding process of FIG. 11;

[0063] FIG. 13: a flowchart of a process of determining metrics that forms part of the image analysis of FIGS. 9A, 9B and 10;

[0064] FIG. 14: a flowchart of a segmentation process performed as part of the metric determination shown in FIG. 17;

[0065] FIG. 15: a flowchart of operation of a grading rule engine for use in the assay analysis of FIG. 8;

[0066] FIG. 16: a flowchart illustrating logic for determining suitability for analysis for use in the grading rule engine illustrated with respect to FIG. 15;

[0067] FIG. 17: a flowchart illustrating spot determination logic for use in the grading rule engine illustrated with respect to FIG. 15;

[0068] FIG. 18: a flowchart illustrating object detection logic for use in the grading rule engine illustrated with respect to FIG. 15;

[0069] FIG. 19: a flowchart illustrating bright object determination logic for use in the grading rule engine illustrated with respect to FIG. 15;

[0070] FIG. 20: a flowchart illustrating spot classification logic for use in the grading rule engine illustrated with respect to FIG. 15; and

[0071] FIG. 21: a flowchart illustrating spot classification confirmation logic for use in the grading rule engine illustrated with respect to FIG. 15.

DETAILED DESCRIPTION OF THE DRAWINGS

[0072] FIG. 1 shows an assay analysis system 5 for analysing assays carried out in an array 10, such as a multiplexed micro-array or hybridized array, that comprises a plurality of assay sites or print areas 15, wherein an individual assay can be performed in each assay site 15 or print area. The system 5 comprises one or more sensors, in this example in the form of a digital camera 17, configured to collect images of the array 10. Although only one camera 17 is shown in FIG. 1, more than one camera 17 could be provided. The camera 17 in this example is configured to capture images of the whole of the array 10, e.g. the field of view 18 of the camera 17 encompasses the whole of the array 10, or at least all of the assay sites 15 of the array 10. However, the camera 17 could be configured to capture images of only some of the assay sites 15 of the array 10, or different cameras 17 could be configured to capture images of different subsets of assay sites 15 of the array 10 or to capture the array 10 from different angles. The system 5 comprises an analysis system 20 that receives the images collected by the camera 10 and is configured to analyse the images to determine states of reaction in specific assay sites 15.

[0073] The analysis system 20 comprises a processing system 25, data storage 30, a communications module 35, one or more output devices 45 and one or more user input devices 40. The processing system 25 comprises one or more processors, which could be single core or multi-core processors. The one or more processors include one or more central processing units, and optionally also one or more graphics processing units, maths co-processors, tensor processing units, and/or the like. The data storage 30 could comprise solid state memory, magnetic memory, optical memory, and/or the like. The communications unit 35 can be configured to communicate via wired and/or wireless communications. In this example, the communications unit 35 is configured to communicate with remote and/or local systems, e.g. via a network such as a LAN, a WAN, the internet, one or more cellular networks, an Ethernet network, an fibre optic network, and/or the like. The at least one output device 45 could comprise a display or other visual output device, an audio output device, and/or a haptic output device. The at least on input device 40 could comprise one or more of: a keyboard, a touchscreen, a trackball, a touchpad, a joystick, a speech recognition based input device, an RFID tag reader, a barcode or QR code reader, and/or the like.

[0074] The analysis system 20 may be configured to communicate with at least one remote processing system 50. The remote processing system 50 could comprise a server, a cloud computing resource, a workstation, a personal computer, and/or the like. The remote processing system 50 comprises remote data storage 55. In this way, any or all of the method steps described herein, particularly any relating to data processing and/or data storage, could be performed using the processing system 25, the at least one remote processing system 50 or the method steps described herein may be distributed between the processing system 25 and the at least one remote processing system 50.

[0075] An example of an image of the array 10 collected by the system of FIG. 1 is shown in FIG. 2. Some of the assay sites 15a comprise spots representative of a reaction, e.g. a reaction indicative of a presence of an analyte, some of the assay sites 15b are indicative of no reaction, which may be indicative of lack of an analyte, and some of the assay sites 15c are empty. The spots are generally associated with a change in opacity or colour or the like that is representative of the extent or degree of reaction. Importantly, the present inventors have identified situations in which different assay sites 15a, 15a′ of the array 10 are used to carry out a reactions having a different reaction strength. The different reaction strength could be associated with different assays, i.e. different types of assay are carried out in different assay sites 15 and at least one or each of the assays may be associated with a different reaction strength to at least one other of the assays. In another example, the reaction strength could be sample dependent or vary from assay site to assay site.

[0076] The analysis of the reaction in each assay site 15 determines one or more metric for the reaction in the assay site 15 and applies logic, e.g. a comparison to a threshold, that depends on one or more parameters to yield a measure of reaction state (e.g. reacted or not, i.e. a qualitative determination, or a value for extent or degree of reaction or activity, i.e. a quantitative determination, or an indication of active or not active or the like). This could be done by selecting a threshold for all of the reactions, e.g. assays and samples, for all of the assay sites. However, the present inventors have noted that the dependency of attributes of each analysis on a given metric can vary differently and significantly for different assays.

[0077] This can be seen from FIGS. 3 to 7, which each show the variation of different attributes (in this case accuracy, specificity, and sensitivity) with an exemplary metric (edge metric in this case) for different assays. As such, in these examples, if a common edge metric threshold is set for the assay shown in FIG. 3, then an edge metric threshold of 20 or above may be desirable. However, for the assay shown in FIG. 4, an edge metric threshold of 20 or above may result in an undesirably low sensitivity for that assay and an edge metric threshold of between 5 and 10 may be more preferable. It can be seen from each of the assays shown in FIGS. 3 to 7, that different threshold values of the metric may be better for different assays. Having identified these issues, the assay analysis system 5 is specifically configured to allow enhanced analysis of arrays that are used to perform different assays in different assay sites 15 of the same array 10.

[0078] An overview of a computerized method of analysing the array 10 to determine an extent or degree of reaction in the assay sites 15 of the array 10 is shown in FIG. 8. Although the method can be performed by the processing system 25 shown in FIG. 1, one or more or each of the steps could be performed by the remote processing system 50 or different steps could be performed by the processing system 25 and the remote processing system 50 such that the processing is effectively distributed between them.

[0079] Step 705 indicates the performance of assays in each of a plurality of assay sites 15 of the microarray 10. The performance of individual assays is known in the art, although in the present case different assays are performed in at least some of the assay sites 15 of the microarray 10, and captured in an image 805 collected by the assay analysis system 5 in step 710.

[0080] Algorithm 810 provides image analysis as part of the assay analysis, and is shown in further detail in FIGS. 9A, 9B and 10.

[0081] The processing system 25 and/or the remote processing system 50 receives the image 805 of the array 10 collected by the camera 17 (see FIG. 1). The processing system 25 and/or the remote processing system 50 implements the algorithm 810 shown in FIGS. 8 and 9A that processes the image 805 to determine metrics indicative of an extent or degree of reaction in each assay site 15. The algorithm 810 uses various parameters 815 to determine metrics indicative of the extent or degree of reaction for each assay site 15.

[0082] The parameters 815 used to determine the metrics are global and apply regardless of the assay being performed. However, the analysis of the metrics by the grading engine uses parameters that are specific and non-generic, i.e. different parameters can be used for different assay sites 15 or groups or subsets of assay sites 15 of the same array 10. For example, different assay types can be associated with different parameters. As the array 10 is used to perform different assays that are included in the image 805, the parameters for at least one of the assay sites 15 will be different to those for at least one other assay site 15 in the same array 10. The parameters for each assay type are generally predetermined and the appropriate predetermined parameters are selected for each different assay site depending on the type of assay being performed at that site. The assay type being performed in each different assay site and/or the parameters associated with that assay type is generally provided in a configuration file. For completeness, the provision of the parameters is not essentially limited to this and in other possible examples the assay type being performed in each different print area can be determined from user input received from one of the input devices such as a keyboard or a reader for an identifier provided on or with the array such as a QR code, RFID tag, holographic tag, barcode, or the like. For example, the assay type for each assay site 15 can be input by a user using the keyboard or the identifier for an array can be read by the reader, and pre-stored data for that assay type can be retrieved from a database, look up table or other form of data file that could be stored on the data storage 30 and/or in the remote data storage 55, and/or the like. Additionally or alternatively, the parameters can vary by sample type, sample size, by individual assay site position in the array 10, and/or by other factors that affect the reaction strength or extent or degree of response to the reaction. In this case, it will be appreciated that the parameters for each assay site 15 or group of assay sites 15 can be determined in a similar way to that described above.

[0083] The algorithm 810 is configured to analyse the image, identify the regions of the image associated with different assay sites 15, determine one or more metrics for the regions of the image associated with the different assay sites 15, and determine an extent or degree of reaction (such as reactive or unreactive or a value for extent or degree of reaction or activity) for the different assay sites 15 based on the metrics and the parameters 815.

[0084] As shown in FIGS. 9A, 9B and 10, the output from the algorithm 810 can include metrics 820 indicative of the extent or degree of reaction for one or more or each of the assay sites 15. The metrics for the assay sites 15 are processed by the grading engine to determine an extent or degree of reaction (e.g. spot grades) 720, which could be a simple indication of a reaction or no reaction, or could be more nuanced, e.g. a value giving an extent or degree of the reaction in the assay site. The determined spot grades can then be presented for interpretation by a user 725. The output could also include one or more errors and/or alarms 825, e.g. when the algorithm has failed to determine an extent or degree of reaction for a given assay site 15 or some other error in the process has occurred.

[0085] A more detailed overview of the process shown in FIGS. 8 and 9A is shown in FIG. 9B. As shown in FIG. 9B, the process has as an input the image 805 of the array 10 collected by the camera 17.

[0086] The input image 805 is cropped 905 so that the image 805 is tightly cropped to the area of the image 805 containing the assay sites 15 (and any other features used by the process such as control assay sites/spots) based on cropping parameters 910 that control the cropping process 905. Micro-arrays 10 can be provided in standardized sizes, dimensions and layouts. The cropping process is intended to reduce the area of the original image 805 to the area of the image associated with the array of assay sites 15 and any other features used by the process, such as control assay sites 15/spots. Cropping processes in general are known and any suitable cropping process could be used. An example of a suitable cropping process comprises identifying corner control spots and using these to determine an array area in the image where the array of assay sites 15 is located. The area is defined based on the position of the four control spots and a safe margin necessary to take the spot placement tolerance into account. The image can then be cropped around the array area.

[0087] The cropped image output from the cropping stage 905 is subjected to gridding 915 that divides the cropped image into segments 925 (see FIG. 2), each segment 925 of the image containing a single assay site 15. The gridding 915 is performed based on gridding parameters 920 that control the gridding process 915. The gridding parameters 915 can be common parameters specific to the assay under test. The grid can be overlaid over the image. This comprises determining the most probable position of the grid based on the position of the corner spots. Segments 925 of a grid model corresponding to the corner control spots are known for the arrays 10, allowing the gird model to be applied to the image 805 using the determined positions of the corner control spots, as shown in FIG. 11.

[0088] The gridding process 915 involves building the model of the array of assay sites 15 in the array 10 based on a geometric definition of a layout of the array of assay sites and on a position tolerance of the spot in the assay site 15. The grid is shown in FIG. 12 and organised in rows and columns of segments 925, each segment 925 encompassing a single assay site 15. A print domain corresponding to a single assay site 15 contains a single spot and a spot can't be in contact with a neighbouring spot. The result is a grid organised in rows and columns of print domains corresponding to the segments 925 of the grid. The model of the grid (the grid descriptor) is stored in the data storage 30 and/or the remote data storage 55.

[0089] Once the segments 925 of the image have been generated, spot metrics 930 for a spot in a given assay site 15 can be generated based on metric generation parameters 935.

[0090] The spot metrics 930 can be any metric of the image associated with a spot that can be used to characterize the spot indicative of a reaction in a given assay site 15. For example, at least one of the spot metrics 930 of a given spot can comprise or be representative of pixel intensities for an area of the image representing that spot. A particular example of this is a spot metric 930 of a given spot that comprises or is representative of a mean value of the pixel intensities for at least part of the area of the image representing that spot of that reaction site and/or a difference between a mean value of the pixel intensities for the area of the image representing that spot and a mean value of background pixel intensities. A specific example of a suitable metric for each spot is standard deviation ratio (SSR), which is the difference between the mean value of the pixel intensities for at least part of the image representing the given spot and the mean value of the pixel intensities for at least part of the background, all divided by a measure of spread (e.g. standard deviation) of the background pixel intensities for at least the part of the background.

[0091] The spot metric calculations 930 can be designed such that the metric calculation is repeatable and generally the same regardless of the spot being analysed, e.g. by using average pixel intensity values, by using masks to select pixels from specified areas of spots and background, and/or the like. As such, the metric generation parameters 935 can also generally be generic or common between spots, regardless of assay type.

[0092] Once the spot metrics 930 have been determined for any spots of interest, then the spot metrics can be assessed using a grading engine 940 to determine the extent or degree of reaction for the respective reaction being performed in the respective assay site 15 containing the respective spot. The grading engine 940 applies spot specific (that are also assay specific and/or sample type specific and/or the like) grading parameters 945 and/or grading rules 950 to the spot metrics determined for a given spot in order to determine the extent or degree of reaction (e.g. spot grade) for that given spot/assay site 15. As noted above, as different spots/assay sites 15 are for different assays and/or sample types, then the grading parameters 945 and/or rules 950 applied by the grading engine 940 for one or more spots/assay sites 15 are different to those applied for some other spots/assay sites 15 in order to determine the extent or degree of reaction (spot grade) for each spot/assay site 15.

[0093] In this way, common or generic parameters are applied where appropriate, e.g. for one or more of the cropping 905, gridding 915 and/or calculation of spot metrics 930, which promotes efficient processing for those processes for which individual parameterization has less of an effect. In contrast, bespoke rules 950 and/or parameters 945 are determined and used by the grading engine 940 for different spots/assay sites 15 and/or different groups of spots/assay sites 15 on the same array 10, where using parameters specific to that assay, sample type or other factor can have a significant difference on the performance and quality of the assays being performed. However, it will be appreciated that, in other examples, one or more or each of the cropping 905, gridding 915 and/or calculation of spot metrics 930 can be carried out using assay site 15 specific (e.g. assay, spot position and/or sample type specific) parameters.

[0094] An alternative flowchart showing the method of FIG. 9B is shown in FIG. 10, which shows the input image 805 being provided and loaded 1005. Parameters in the form of a build grid descriptor 1007 are generated and used to crop 905 and grid 915 the input image 805. Metrics 1012 for each segment 925 of the grid are determined (step 1008) and output in an image analysis output file to the spot grading engine 940, which uses the metrics to grade the spot in each segment 925/well 15. The results of the image analysis are also output (step 1014). The output results in step 1014 could contain, for example, the determined metrics 1012 and any additional image analysis data, such as error indications, that might be useful or requested for further investigation.

[0095] Segments 925 of the grid model corresponding to the corner control spots 1747 are known for the arrays 10, allowing the gird model to be applied to the image 805 using the determined positions of the corner control spots 1747 and other spots, as shown in FIGS. 11 and 12.

[0096] More detail of the generation of the metrics 930/1008 shown in FIGS. 9B and 10 is shown in FIG. 13. The metrics 930/1008 are determined from the image for each segment 925/print domain in the gridded image. The metrics 930/1008 for the segments (print domains) 925 are input into the grading rule engine 940, which uses the metrics to determine a grade or reactivity for each assay performed in the associated segment (print domain) 925.

[0097] As shown in FIG. 13, the metric generation process comprises calculating 1732 shape (e.g. circle) metrics 1805. The shape metrics are identified by using object recognition techniques to identify a shape (such as but not limited to a circle) indicative of a reaction spot in a segment and then determining metrics of the shape. The shape metrics can comprise, for example, one or more of: circle centre coordinates, circle radius, circle mean (i.e. mean pixel value of a circle area defined by the circle coordinates and radius), circle standard deviation (i.e. a standard deviation of pixel values in the circle area), circle background mean (i.e. a mean of the pixel values in an area of the background), circle background standard deviation (i.e. a standard deviation of the pixel values in an area of the background), circle SSR (a signal to standard deviation ratio for the circle area), any alternative metrics that are indicative of any of the above, and/or the like.

[0098] The metric generation process also comprises calculating domain metrics 1810 for each segment (print domain). The domain metrics can include metrics such as a domain mean (mean of the pixel values of an area defined by the entire segment coordinates and radius) and domain standard deviation (standard deviation of the pixel values of an area defined by the entire segment coordinates and radius), and/or the like. The domain metrics are calculated based on the position and size of individual segments of the grid.

[0099] In addition to calculating domain metrics 1810, the process also comprises calculating default domain metrics 1815 for each segment (print domain), the default domain being a circle of a default radius placed at the centre of the respective segment. Examples of default domain metrics include one or more or each of: a mean of the pixel intensities of the default domain, a standard deviation of the pixel intensities of the default domain, a mean of pixels of an area of background area of the default domain, a standard deviation of pixels of an area of background area of the default domain, and a signal to standard deviation ration (SSR) of the default domain, and/or the like.

[0100] The metric generation process further comprises calculating spot metrics. Spot metrics are extracted from each segment (print domain) by a segmentation method. The purpose of the segmentation step is to detect if an object (e.g. some form of activity or a “blob”, which may or may not be a spot caused by and/or indicative of the extent of the assay reaction) is present in the segment (print domain) under consideration and, if so, determine the position and size of the object. Corresponding metrics are calculated for both the object and for the background surrounding the object. If no object is found in a segment (print domain), no spot metrics are generated. If an object is detected, then the spot metrics are determined and subsequently used in the analysis performed by the grading rule engine to determine if the object is a spot and, if so, will determine the grade of the spot. Examples of possible spot metrics include: spot centre coordinates, spot radius, mean of the pixel values in an area defined by the spot mean and spot radius, standard deviation of the pixel values in an area defined by the spot mean and spot radius, mean and/or standard deviation of an area defined by the background, spot threshold, spot edge metric, spot isoperimetric coefficient, and/or signal to standard deviation ratio of the spot, and/or the like.

[0101] The determination of the spot metrics starts with a segmentation step 1820. The segmentation step 1820 is detailed in FIG. 14.

[0102] The segmentation process 2005 shown in FIG. 14 comprises creation of an eroded image 2405. This comprises applying a structuring element of a size and shape set by erosion parameters 2410, which include data setting a shape and size of the erosion element. The erosion element is a matrix that is placed on each image pixel, the value of the eroded pixel being the minimum input pixel found on the matrix. The erosion is used to determine an estimate of the local background. The structuring element can be chosen so that, after erosion, the centre of the segment reflects the local background.

[0103] For each segment 925, the segmentation process shown in FIG. 14 comprises determining a pixel intensity based metric of the segment 925. In this example, the metric for each segment 925 is an IQ3 metric 2415, which is calculated for the respective segments 925. The IQ3 metric for a segment 925 is the pixel intensity of the image at the centre of that segment 925.

[0104] In step 2430 of FIG. 14, thresholding is applied to segments 925. The thresholding converts colour or greyscale images into black and white images by applying a threshold on the pixel intensities, above which the pixel is determined to be white and below which is it determined to be black. The threshold can optionally be a dynamically determined threshold (i.e. to apply adaptive thresholding) or can be provided as a parameter.

[0105] In step 2435 of FIG. 14, the thresholded image output from step 2430 is analysed to determine any contours in the image. Various techniques can be used to identify contours and edges, and a suitable technique can be selected.

[0106] In step 2440 of FIG. 14, the contours found in step 2435 are checked on a segment 925 by segment 925 basis to see if they approach to within a border width threshold (which is provided as an input parameter 2005) of the border of the segment 925 and, if so, are removed. If more than one contour is left after this process, then the largest contour is selected for further processing, step 2450, and the remaining contours are discarded. If no contours are found for a given segment 925, then that segment 925 is labelled as “not segmented” or “blob not found” and no further segmentation processing is performed with respect to that segment 925.

[0107] For those segments 925 that have not already been labelled as “not segmented”, the convex hulls 2705 for the spots 2710 in those segments are determined in step 2455. The convex hull or convex envelope 2705 of a spot is the smallest possible convex shape that contains that spot 2710.

[0108] In step 2460, a minimum enclosing circle is determined for the respective segments. The minimum enclosing circle is the smallest possible circular shape that contains the convex hull. The minimum enclosing circle centre coordinates and radius are the output of the segmentation stage.

[0109] For each segment 925, if no blob is found in the segmentation process 1820 (i.e. it is unsegmented) then no spot metrics are determined for that segment 925. However, if a spot is determined to be present in a given segment 925, that segment 925 is partitioned into spot portions and background portions in step 1825.

[0110] The part of the image representing an assay site 15 associated with the segment 925 can be partitioned into portions representing the spot and portions representing a background. The portion representing the spot is used to calculate the average intensity of the spot, e.g. by taking a mean value of all of the pixel intensity values for pixels that are contained within the area of the portion representing the spot.

[0111] The portion representing the background is used to calculate the average intensity and standard deviation (or other measure of spread) of the background in the part of the segment that does not include the spot (i.e. the mean value and standard deviation of the pixel intensity values of pixels representative of the background).

[0112] For segments or spots that are determined to be segmented (i.e. a blob or some form of activity is present), then the process can proceed to determining the spot metrics needed for assessing the spot grade or extent or degree of reaction for each spot or assay site 15 that so far hasn't been deemed non-reactive or a technical error. Step 1830 of the process of FIG. 13 comprises pixel filtering the images to remove pixels that are atypical by being extremely high or low in intensity. Image artefacts typically have very high intensities and this process removes any remaining residues of these artefacts. Pixels with very low intensity may be indicative of noise. The pixel filtering applies high and low pixel intensity thresholds, which are parameters of the process, to both the spot pixel population and the background pixel population and removes those pixels below the low intensity threshold and above the high intensity threshold.

[0113] In step 1835, the spot metrics used in the spot grading/determination of the extent or degree of reaction are determined. Various metrics could be used. One metric is the signal to standard deviation ratio (SSR), which can be calculated for each spot/segment 925/assay site 15 using the following formula:

[00001]$SSR=\frac{\mathrm{Mean}\left(\mathrm{Signal}\right)-\mathrm{Mean}\left(\mathrm{Background}\right)}{\mathrm{StD}\left(\mathrm{Background}\right)}$

[0114] Where: [0115] Mean(Signal) is the mean value of the pixels intensities in the filtered spot area; [0116] Mean(Background) is the mean value of the pixels intensities in the filtered background area; and [0117] StD(Background) is the Standard Deviation of the pixels intensities in the filtered background area

[0118] Other examples of metrics that could be used in addition or as alternatives to the SSR include IQ1, IQ2, IQ3 and IQ4, which are:

[00002]$\mathrm{IQ}1=\mathrm{abs}\left(\frac{\mathrm{Mean}\left(\mathrm{signal}\right)-\mathrm{Mean}\left(\mathrm{background}\right)}{\begin{array}{c}\mathrm{Background}\mathrm{Factor}*\mathrm{StD}\left(\mathrm{background}\right)+\\ \mathrm{SignalFactor}*\mathrm{StD}\left(\mathrm{signal}\right)\end{array}}\right)-1$

[00003]$IQ2=\frac{A_{\mathrm{signal}}}{A_{\mathrm{spot}}},$

where A.sub.signal is the area of the spot obtained using the spot radius, and A.sub.spot is either the area of the spot obtained from segmentation prior to pixel filtering (which is a measure of the number of pixels outside the Spot Radius) or the area of the spot within the spot radius excluding any pixels removed through filtering (which is a measure of the number of missing pixels inside the Spot Radius). [0119] IQ3=the Pixel intensity of the resulting image at the centre of the segment 925 [0120] IQ4 is the ratio of the % of pixels at or above an intensity target to the % of pixels at camera saturation value (usually 255).

[0121] Other suitable metrics may be apparent to a skilled person from the teaching of the present disclosure. Any determined metrics are output to the grading rules engine 940. The grading rules engine takes the metrics, e.g. circle metrics, domain metrics and/or spot metrics for various of the assay sites, which are generally indicative of image properties, and uses these to determine spot grade or extent or degree of reactions, which are indicative of properties of the assay being carried out at the associated assay site.

[0122] The grading rules engine 940 operates according to a logic that is outlined in FIG. 15. FIG. 15 is a flowchart giving an overview of rules used in the spot grading processes, i.e. the generation of spot/assay site 15 metrics 930 and outlines operation of the grading engine 940. The application of individual rules by the grading engine 940 is described in more detail below with respect to FIGS. 24 to 29.

[0123] In this example, the grading parameters 945 and rules 950 (as shown in FIG. 9B) used by the grading engine 940 to determine the spot grade or extent or degree of reaction can be either global or unique. Global parameters are parameters specific to a type of assay, whereas unique parameters are defined as a function of both the assay and the spot/assay site 15 position, i.e. the parameters are dependent on both the assay site 15/spot and on the assay.

[0124] The spot grading determination performed by the grading rules engine 940 considers the image 805 of the array of assay sites 15 in the array 10 (suitably cropped 905 and gridded 915 into segments) along with the metrics for each segment 925/assay site 15 and delivers the grading result for the corresponding spot/assay site 15/segment 925. In the specific example shown in FIG. 23, the grading engine 940 applies a set of six rules for each segment 925/assay site 15 of interest, namely: [0125] Rule 00: A rule 1205 that identifies images not suitable for analysis; [0126] Rule 01: A rule 1210 that identifies if the segment 925 is empty or contains an object; [0127] Rule 02: A rule 1215 that identifies if the segment 925 contains a spot, e.g. by segmentation or circle search; [0128] Rule 03: A rule 1220 that will grade a spot as reactive, non-reactive or either a technical error or the degree of reaction in of the spot/assay site 15 is indeterminate; [0129] Rule 04: A rule 1225 that will detect bright objects; and [0130] Rule 05: A rule 1230 that will confirm a result.

[0131] Although a useful set of rules 1205-1230 is provided above for grading the spots/reactions in assay sites 15 by the grading rules engine 940, in other examples only some but not all of the rules 1205-1230, or additional or alternative rules could be applied. The operation of the system will be described further with respect to FIGS. 16 to 21.

[0132] An example of flowchart of a process or rule for determining whether an image is suitable for analysis is shown in FIG. 16. The process or rule shown in FIG. 16 is operable to determine if an image of the array of assay sites 15 in the array 10 is suitable for analysis if the background of the image of the array 10 is sufficiently dark and uniform. The process uses 1405 metrics of the overall image, wherein the metrics of the overall image include, in this example, the mean of the pixel intensities for the background parts of the overall image and the standard deviation (or some other suitable measure of spread) of the pixel intensities for the background parts of the overall image. The rule uses global parameters 1410, including an image background mean maximum acceptable value (or threshold) and image standard deviation or spread maximum acceptable value (or threshold). The process comprises implementing a rule 1415 in which it is determined if either the mean of the pixel intensities for the background parts of the overall image exceeds the image background mean maximum acceptable value (or threshold) or the standard deviation (or some other suitable measure of spread) of the pixel intensities for the background parts of the overall image exceeds the image standard deviation or spread maximum acceptable value (or threshold) respectively. If so, then a technical error is flagged 1420 and the process ends 1425 for that image. Otherwise, it is determined that the image is suitable for analysis and the overall process implemented by the grading engine 940 continues to rule 1215, which is illustrated in FIG. 17.

[0133] FIG. 17 is a schematic of a process 2205 for identifying if an object detected in a segment 925 is a spot indicative of a reaction. If the object can't be identified as a spot with a required certainty, then the process proceeds to the process 2305 of FIG. 18, which determines if the segment 925 is empty or not. The process 2205 of FIG. 17 determines an object in a given segment 925 is a spot indicative of a reaction if it satisfies certain conditions relating to the detectability, shape, size and position in the segment 925 of the spot.

[0134] The process includes receiving parameters 2215, such as maximum and minimum blob radius, maximum and minimum isoperimetric coefficient, maximum and minimum blob edge metric and the like. The parameters are unique for the individual assay site/assay being performed.

[0135] The process comprises applying rules 2220 to determine if metrics found in the segment support recognition of the object in a given segment are indicative of a reaction that is suitable for grading. If so, the graded reaction can be determined uniquely for the given segment.

[0136] The application of the rules 2220 utilise circle and domain metrics 2210, such as those generated during the metrics generation described above in relation to FIG. 13. The application of the rules 2220 can include, for example, determining if a circle has been found in a given segment 925. If a circle is found for the segment under analysis, then it is determined that a spot indicative of a reaction is present in that segment, and the radius and position of the circle are determined.

[0137] Additional or alternative rules can be used to determine the presence of a spot indicative of a reaction. For example, the application of the rules 2220 can also include, determining if an object in the form of a non-circular “blob” has been found based on SSR. If so, various metrics of the “blob” such as radius, edge metric, isoperimetric co-efficient, position and the like are determined. It is determined that a spot indicative of a reaction is present in the segment if all blob criteria are met, wherein the blob criteria comprise one or more of, for example, the blob radius is greater than or equal to the blob minimum radius and less than or equal to the blob maximum radius; the blob isoperimetric coefficient is greater than or equal to the blob minimum isoperimetric coefficient and less than or equal to the blob maximum isoperimetric coefficient; the blob edge metric is greater than or equal to the blob minimum edge metric and less than or equal to the blob maximum edge metric, and/or the like. If all of the blob criteria are met, then it is also determined that a spot has been detected.

[0138] If a circle is identified in the segment 925 or if all of the blob criteria are met, then the process 2420 shown in FIG. 19 is applied, which checks if the spot identified in the process 2205 can be identified as a bright object artefact rather than a spot indicative of a reaction.

[0139] If no circle was found and any of the blob criteria are not met, then the process 2305 of FIG. 18 is applied, which determines if a segment is empty or not.

[0140] If a segment 925 is determined to neither contain a circle nor meet the blob criteria, the process 2305 determines if the segment 925 contains an object or if it is empty. It does this by determining 2310 if the segment 925 is blank (i.e. the mean pixel value is below a certain level) or if it is uniformly filled with a level of grey that does not allow an object to be distinguished (i.e. no transitions or edges detected) and is consistent with the overall array image grey level. The determination 2310 comprises comparing certain metrics for the segment 925 (such as SSR value) to one or more thresholds that are set as parameters 2312 (such as a segment SSR limit parameter). This check can be specific for the printed area or assay in that segment 925.

[0141] For example, if an SSR value for the segment 925 is less than a segment SSR limit parameter, a mean pixel value for the segment 925 is less than a maximum mean pixel value parameter, a standard deviation (or other spread metric) for the segment 925 is less than a maximum standard deviation parameter, and an indication of a detection of an object to background noise less a measure of background noise is less than a maximum threshold parameter, then the segment 925 is determined to be empty 2315 and the process is ended 2320 for that segment 925. Otherwise, a technical error is determined 2325, that is, the domain contains an object but it cannot be said with sufficient certainty that the object qualifies as a spot indicative of a reaction.

[0142] As noted above in relation to the process shown in FIG. 17, if the process of FIG. 17 identifies that the object inside a given segment 925 can potentially be identified as a spot indicative of a reaction, the process comprises, as step 2420, removing bright object artefacts from each segment 925, which is shown in more detail in FIG. 19.

[0143] This bright object artefact removal step 2420, as shown in FIG. 19, comprises applying rules 2505 for determining if an object in a segment is a bright object anomaly by comparing a pixel intensity metric to one or more thresholds that are set as parameters 2425 and/or derived from other metrics. As examples, the rules 2505 may specify that an object identified in a segment 925 will be considered as a bright object anomaly if a signal mean value is above a detection threshold that is calculated from the IQ3 value or if the Signal Standard Deviation is above a value defined by a unique parameter, or the like.

[0144] The threshold is defined using a detection gradient and detection intercept that are provided as unique parameters 2425 for a given segment 925, and the detection threshold is defined as detection gradient for that segment 925 multiplied by the IQ3 value for that segment 925 plus the detection intercept for that segment 925. For example, the rules 2505 may specify that the object in a given segment 925 can be determined to be a bright object anomaly if any of the following criteria apply: [0145] (1) The mean spot signal (e.g. pixel values) for the spot is greater than or equal to the detection threshold; [0146] (2) The standard deviation or other measure of spread of the spot signal is greater than an associated threshold that is provided as a unique parameter 2425; or [0147] (3) A mean of a circle signal is greater than or equal to the detection threshold; [0148] (4) The standard deviation or other measure of spread of the circle signal is greater than an associated threshold that is provided as a unique parameter 2425.

[0149] If none of these criteria are met, then the process, as step 2510, performs the process 2025 shown in FIG. 20, which applies rules to classify the spot. If any of the above criteria apply, then a bright object anomaly is detected and a technical error 2515 is recorded for that segment 925 (i.e. the detected object is likely an anomaly); and the process 2420 repeats for another segment 925 until all of the segments 925 in the image 10 have been analysed.

[0150] It will be appreciated that the bright object anomaly detection may use only one or some of the above criteria, and/or may use different criteria. In an alternative or additional step, the process may remove any pixels that have a pixel intensity above the detection threshold rather than rejecting the entire segment 925 on the grounds of a technical error.

[0151] As noted above, with respect to FIG. 19, if the process of FIG. 19 determines that the object is not a bight object artefact, then the process 2025 shown in FIG. 20 is applied, which applies rules to classify the spot.

[0152] In process 2025 shown in FIG. 20, the spot grade/degree of reaction for each assay site 15/spot/segment 925 is determined from the metrics determined using the process shown in FIG. 13 and parameters that are assay and/or assay site 15 specific. The spot grading/determination of the degree of reaction is only carried out for those spots/segments 925 that have not been rejected by any of the preceding rules applied by the grading engine 940.

[0153] Determining a spot grade/degree of reaction based on SSR is provided as a beneficial example. It will be appreciated that the determination of spot grade/degree of reaction could additionally or alternatively be based on other metrics such as, but not limited to, IQ1, IQ2, IQ3, IQ4, and/or the like.

[0154] In the example, four thresholds are set, which are set as assay dependent and optionally also spot specific and/or sample type dependent parameters. The thresholds are, in increasing value, Tel, Tl, Th and Teh, The spot grade/degree of reaction for a given spot/assay site 15/segment 925 is determined based on the SSR according to the following table 1:

TABLE-US-00001 TABLE 1 SSR Value Range Spot Grade >Teh Technical Error [Teh-Th] Reactive ]Th to Tl[ Indeterminate [Tl to Tel] Non-Reactive <Tel Technical Error

[0155] The process 2025 comprises applying rules 3105 for determining spot grade/degree of reaction for each spot/assay site 15/segment 925, as shown in FIG. 20. According to these rules 3105, the spot will be graded as reactive, non-reactive or indeterminate based on values of metrics that reflect its intensity, such as the SSR or delta (the delta is the difference between the mean spot or circle pixel intensity value and the mean background pixel intensity value).

[0156] In the specific example shown in Table 1, if the SSR value of a segment is below a lowest threshold Tel or above a highest threshold then a technical error is determined. If the SSR is above the lowest threshold Tel but below an upper non-reactive threshold Tl, then a non-reactive grading is determined. If the SSR is above the upper non-reactive threshold Tl but below a lower reactive threshold Th, then an indeterminate state is determined, which is indicative of it not being possible to determine the reactive and non-reactive state gradings with sufficient certainty. If the SSR is between lower and upper reactive thresholds Th and Teh, then a reactive grading is determined. Although the specific example given in Table 1 uses SSR as the metric, it will be appreciated that the same concept could also be applied to other metrics.

[0157] Beneficially, the rules 3105 may comprise logic to dynamically switch between the metrics used for determining spot value/degree of reaction, to select the most suitable metric, for example switching being based on the standard deviation or other spread metric of pixel intensities of the background. In the example, the logic uses the SSR as a default and switches to another metrics, e.g. the delta, if it is found that the SSR is unsuitable. The switch between SSR and delta is made based on a measure of spread in background values, for example by using background CV values which is 100× the standard deviation of the background divided by the mean of the background.

[0158] For example, in some cases, the background surrounding a spot is non-uniform, causing the background standard deviation to have values well in excess of the average. As a result, the SSR is strongly decreased, which can cause false non-reactive determination even when the spot is clearly indicative of a reaction. In this case (invalid SSR), an alternate spot reactivity evaluation using the delta is used and the background uniformity is evaluated using the CV instead of the standard deviation to take into account the average level of the background relative to the standard deviation.

[0159] Threshold parameters 3110 for the determination of spot grade/degree of reaction are all assay dependent, and optionally also dependent on the spot position and/or sample type, i.e. they vary for different assay sites 15 in the same array 10. The threshold parameters 3110 required include a maximum acceptable background CV value, reactive SSR value range, non-reactive SSR value range, reactive spot delta value range, non-reactive delta value range, and/or the like.

[0160] From the process described above in relation to FIG. 17 (rule 2), the spot may have been determined by segmentation or by circle detection. The threshold parameters 3110 that are applied may also depend on whether the spot is determined as a circle from the circle search or as a spot or “blob” via the segmentation process, with different threshold parameters being applied for each.

[0161] The application of the rules 3105 starts by determining if the background CV value is less than the maximum acceptable background CV value. If so, then the process proceed by using the SSR metric. If not, then the process proceeds using the delta metric, thereby providing the dynamic switching described above.

[0162] If proceeding using the SSR metric, it is determined if the SSR (either for the blob or the circle, depending on how the spot was determined) is within the reactive SSR value range (which is a threshold parameter) and if so determines that the spot/assay site 15/segment 925 is indicative of the associated assay being reactive 3115. If the SSR is not within the reactive SSR value range, then it is determined if the SSR is within the non-reactive SSR value range (which is also a threshold parameter) and if so then it is determined that the spot/assay site 15/segment 925 is indicative of being unreactive or not-reacted 3120. If the SSR for the spot is determined to be out with both the reactive SSR value range and non-reactive SSR value range, then it is determined that the spot/assay site 15/segment 925 is indeterminate or a technical error has arisen 3125.

[0163] If it is determined that the delta metric should be used, then a determination process similar to the above but using delta and threshold parameters 3110 (such as reactive delta value range and unreactive delta value range) associated with the delta instead of SSR and the threshold parameters 3110 associated with SSR are used.

[0164] Regardless of the determination made in the process 2025 shown in FIG. 20, a check step 3205, shown in FIG. 21, is made. The check step 3205 comprises applying rules 3215 to the determination of spot grade/degree of reaction determined from the spot qualification step 2025 shown in FIG. 17. The check step 3205 shown in FIG. 21 is an optional step and can be switched on or off by setting an appropriate check parameter value 3210. If the spot qualification step 2025 of FIG. 20 determines that the spot/assay site 15/segment 925 should be graded as “non-reactive” 3120 and the same spot/assay site 15/segment 925 is indicated as “empty” 2315 by the process in FIG. 18, then the rules 3215 confirm the spot/assay site 15/segment 925 as non-reactive 3220. However, if the spot qualification step 2025 determines that the spot/assay site 15/segment 925 should be graded as “non-reactive” 3120 and the same spot/assay site 15/segment 925 is not indicated as “empty” 2315 by the process in FIG. 26, then a “technical error” 3225 is output for the spot/assay site 15/segment 925 instead of “non-reactive”. If the spot qualification step 2025 determines that the spot/assay site 15/segment 925 should be graded as “reactive” 3115, then this is maintained by the rules 3215, then the “reactive” determination is output 3230.

[0165] Beneficially, at least the threshold parameters used in the spot qualification process 2025 described above in relation to FIG. 20 are separately variable depending on factors such as one or more of: the assay being performed, the specific location of the assay site 15 on the assay 10, the type of sample being analysed, and/or the like. In this way, the effectiveness of the assay determination can be improved by selecting better thresholds for any given assay, assay site 15 and/or sample type, regardless of what other (different) assays are being performed on the same array 10. That is, less compromise in the setting of threshold parameters is needed to account for performing different assays in different assay sites 15 on the same array 10.

[0166] Various steps prior to the spot qualification are carried out, which need not be essential, but individually or in combination, may improve the accuracy or efficiency of the assay result determination.

[0167] Although a specific example is described above, it will be appreciated that this is provided to give the skilled person one possible way to put the invention into practice, but variations to the methods and apparatus described above are possible within the scope of the claims.

[0168] For example, various metrics such as SSR, delta, IQ3 and the like are used, but it will be appreciated that alternative metrics could be used. Furthermore, although various techniques for detecting spots in the assay sites 15 are described above, such as edge detection, thresholding, and shape, e.g. circle identification, and the like, it will be appreciated that alternative techniques could be used.

[0169] Digital cameras 17 are advantageously used to collect the images of the assays, but it will be appreciated that other sensor devices that are usable for determining metrics representative of a degree of reaction such as thermal cameras, ultra-violet or infra-red sensors and the like could be used.

[0170] Method steps of the invention can be performed by one or more programmable processors executing a computer program to perform functions of the invention by operating on input data and generating output. Method steps can also be performed by special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit) or other customised circuitry. Processors suitable for the execution of a computer program include CPUs and microprocessors, and any one or more processors. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. Information carriers suitable for embodying computer program instructions and data include all forms of non-volatile memory, including by way of example semiconductor memory devices, e.g. EPROM, EEPROM, solid state memory such as SSD and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in special purpose logic circuitry.

[0171] The method steps can be performed by a local processing device, on a remote processing device or certain steps can be performed on the local processing device and certain method steps can be performed on the remote device in a distributed processing arrangement.

[0172] To provide for interaction with a user, the invention can be implemented on a device having a screen, e.g., a CRT (cathode ray tube), plasma, LED (light emitting diode) or LCD (liquid crystal display) monitor, for displaying information to the user and an input device, e.g., a keyboard, touch screen, a mouse, a trackball, and the like by which the user can provide input to the computer. Other kinds of devices can be used, for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0173] A suitable processing device for performing the method described above may comprise a mobile, fixed or network enabled device, comprising or configured to implement a controller or processing system. The device may be or comprise or be comprised in a mobile phone, smartphone, PDA, tablet computer, laptop computer, and/or the like. The controller or processing system may be implemented by a suitable program or application (app) running on the device. The device may comprise at least one processor, such as a central processing unit (CPU), maths co-processor (MCP), graphics processing unit (GPU), and/or the like. The at least one processor may be a single core or multicore processor. The device may comprise memory and/or other data storage, which may be implemented on DRAM (dynamic random access memory), SSD (solid state drive), HDD (hard disk drive) or other suitable magnetic, optical and/or electronic memory device. The at least one processor and/or the memory and/or data storage may be arranged locally, e.g. provided in a single device or in multiple devices in in communication at a single location or may be distributed over several local and/or remote devices. The device may comprise a communications module, e.g. a wireless and/or wired communications module. The communications module may be configured to communicate over a cellular communications network, Wi-Fi, Bluetooth, ZigBee, near field communications (NFC), IR, satellite communications, other internet enabling networks and/or the like. The communications module may be configured to communicate via Ethernet or other wired network or connections, via a telecommunications network such as a POTS, PSTN, DSL, ADSL, optical carrier line, and/or ISDN link or network and/or the like, via the cloud and/or via the internet, or other suitable data carrying network. The communications module may be configured to communicate via optical communications such as optical wireless communications (OWC), optical free space communications or Li-Fi or via optical fibres and/or the like. The device and/or the controller or the at least one processor or processing unit may be configured to communicate with the remote server or data store via the communications module. The controller or processing unit may comprise or be implemented using the at least one processor, the memory and/or other data storage and/or the communications module of the device.