METHOD AND SYSTEM FOR DETECTING DEFECTIVE PIXELS AND SCREEN IMPERFECTIONS OF A MOBILE DEVICE
20170278232 · 2017-09-28
Assignee
Inventors
- Gerald J. Parsons (Chester Springs, PA, US)
- Michael P. Parsons (West Chester, PA, US)
- Leo D. Parsons (Chester Springs, PA, US)
- Nidhin Davis (Avondale, PA, US)
- Andrew Sieben (Wilmington, DE, US)
- Sean Parsons (West Chester, PA, US)
- Mohammad R. Paknejad (Glenmoore, PA, US)
- Marc Aurel Kamsu Tennou (West Chester, PA, US)
- John J. McGrath (Kennett Square, PA, US)
Cpc classification
International classification
Abstract
A method for detecting defective pixels and screen imperfections is provided and include the steps of: positioning a mobile device having a display screen in a testing chamber, capturing an image of the display screen using a camera directed to a the display screen, and performing a perform a pixel detection analysis of the image.
Claims
1. A method for detecting defective pixels and screen imperfections, comprising the steps of: positioning a mobile device having a display screen in a testing chamber; capturing an image of the display screen using a camera directed to the display screen, the image has a plurality of image pixels and is a photographic representation of rows and columns of a plurality of display pixels of the display screen, each image pixel having Red, Green, and Blue values (RGB values) varying from 0-255; performing a pixel detection analysis of the image; and determining an image pixel of the plurality of image pixels is a questionable pixel if at least two of the R, G, and B values of the image pixel are greater than 30.
2. The method according to claim 1, further comprising the step: loading a defective pixel mobile application to the mobile device.
3. The method according to claim 2, wherein the defective pixel mobile application is loaded through a connector or communication protocol.
4. The method according to claim 2, wherein the defective pixel mobile application sets the display screen black with a backlight turned ON to a maximum setting.
5. The method according to claim 2, wherein the defective pixel mobile application is an application that functions on a mobile device operating system.
6. (canceled)
7. The method according to claim 1, wherein a pure white pixel color has values of R=255, G=255 and B=255 and a pure black pixel color has values of R=0, G=0 and B=0.
8. (canceled)
9. The method according to claim 1, further comprising the steps of: comparing the questionable pixel against 8 surrounding pixels.
10. The method according to claim 9, further comprising the steps of: categorizing the questionable pixel as a drastic change pixel when more than 4 of the 8 surrounding pixels have a significant variance from the questionable pixel.
11. The method according to claim 10, further comprising the steps of: identifying a defective display pixel when there are more than 5 drastic change pixels in a 15×15 image pixel range.
12. A method for detecting defective pixels and screen imperfections, comprising the steps of: positioning a mobile device having a screen in a testing chamber; capturing an image of a display screen using a camera directed to a the display screen; and performing a perform a scratch detection analysis of the image to determine a scratch on the screen.
13. The method according to claim 12, further comprising the step: loading a scratch detection mobile application to the mobile device.
14. The method according to claim 13, wherein the scratch detection mobile application is loaded through a connector or communication protocol.
15. The method according to claim 13, wherein the scratch detection mobile application sets the display screen white with a backlight turned ON to a maximum setting.
16. The method according to claim 12, wherein the scratch detection mobile application is an application that functions on a mobile device operating system.
17. The method according to claim 12, further comprising the steps of: grouping pixels of the image into pixel groupings.
18. The method according to claim 17, wherein the pixel grouping includes 8×8 pixel groupings.
19. The method according to claim 17, further comprising the steps of: determining if the pixel grouping includes a defective pixel if standard deviation of a RGB value for the pixel grouping is above 8.
20. The method according to claim 19, wherein the defective pixel is represented by a single pixel with white color.
21. The method according to claim 19, further comprising the steps of: converting the image to a gray scale converted image.
22. The method according to claim 21, further comprising the steps of: applying a grayscale spec map to the grayscale converted image.
23. The method according to claim 22, further comprising the steps of: dilate the grayscale converted image.
24. The method according to claim 23, further comprising the steps of: perform further dilatation of the grayscale converted image.
25. The method according to claim 24, further comprising the steps of: invert the grayscale converted image.
26. The method according to claim 25, further comprising the steps of: filter the grayscale converted image to eliminate small artifacts.
27. The method according to claim 26, further comprising the steps of: inverting a filtered image of the grayscale converted image into a final image.
28. The method according to claim 27, further comprising the steps of: analyze the final image for RGB values corresponding to any white pixels.
30. The method according to claim 28, further comprising the steps of: determining the scratch on the screen when a white pixel is identified in the final image.
31. The method according to claim 10, wherein the categorizing step includes calculating a difference for each of the R, G, and B values between the questionable pixel and the surrounding pixel.
32. The method according to claim 31, wherein the surrounding pixel has the significant variance from the questionable pixel if the difference of one of the R, G, and B values is at least 25 and the difference of at least one of the other two R, G, and B values is greater than 3.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0007] The invention will now be described by way of example with reference to the accompanying Figures of which:
[0008]
[0009]
[0010]
[0011]
[0012]
[0013]
[0014]
[0015]
[0016]
[0017]
[0018]
[0019]
[0020]
[0021]
[0022]
[0023]
[0024]
[0025]
DETAILED DESCRIPTION OF THE EMBODIMENT(S)
[0026] Now with reference to the drawings, a method and system for detecting defective pixels and screen imperfections according to the invention will be described.
[0027] First with reference to
[0028] The mobile device 1 is loaded with a defective pixel mobile application to analyze the pixels and the display 2. The defective pixel mobile application can be loaded through known cables, connectors and communications protocols. The defective pixel mobile application includes an algorithm that uses high quality images taken by one or more testing cameras 20 to identify defective pixels. It is important that each of the pixels of the mobile device 1 needs to be clearly visible for analysis and the changes in the pixel RGB values need to be clearly identified. In particular, in the exemplary embodiment, the defective pixel mobile application sets the whole display 2 black with a backlight turned ON to a maximum setting. The black display and backlight set to ON combine to illuminate the defective pixel as a white pixel, making any defective pixel readily identifiable in contrast to the surrounding black pixels. The defective pixel mobile application, in an exemplary embodiment, is a custom Android application, but may be an application that functions on other mobile device operating systems known to those with ordinary skill in the art.
[0029] The mobile device 1 is manually positioned on a retention device 5 with the display 2 facing upward and way from the retention device 5. The retention device 5 is then positioned in the testing chamber 10, for instance, using a sliding shelf. The testing chamber 10 is then closed and sealed from exterior light. The pair of cameras 20a, 20b are positioned facing the display 2 from a top of the testing chamber 10, as shown in
[0030] In an exemplary embodiment of the invention, the method and system for detecting defective pixels and screen imperfections of a mobile device 1 uses a two camera system for large screens, such as tablet screens, so that the pixels of the mobile device 1 are clearly visible for analysis. However, for mobile devices having smaller screens, such as mobile phones, the method and system for detecting defective pixels and screen imperfections of a mobile device 1 may use a one camera system.
[0031] As shown in the two camera system, the method and system can detect changes in the pixel RGB values. The first testing camera 20a is positioned to capture and image of a top portion of the display 2, while the second camera 20b is positioned to capture an image of a bottom portion of the display 2. The first testing camera 20a is positioned side by side with the second camera 20b. The use of a two camera system makes defective pixels clearly visible when captured images are analyzed using the system and method described below.
[0032] An exemplary embodiment of the camera arrangement in the testing chamber is clearly shown in
[0033] In the shown embodiment, the cameras 20a, 20b used for analysis are the Basler GigE #acA2500-14gc (see
[0034] Table 1 below shows the comparison of the resolution of the camera versus the resolution of the mobile device 1 (5:1) in an exemplary embodiment of the invention. Table 1 shows that the first testing camera 20a captures about 5 million pixels in which the display 2 crop area is 2 million pixels and the pixels that fall into the camera 20a view is 0.33 million pixels. The second testing camera 20b captures another 5 million pixels in which the display 2 crop area is 1.67 million pixels and the pixels that fall into the camera view is 0.31 million pixels. This correlates to a total of approximately 3.5 million pixels of the captured image corresponds to 0.64 million pixels on the actual display 2 of the mobile device 1. This means 1 pixel on the display 2 of the mobile device 1 corresponds to approximately 5 pixels on the actual captured image taken by the cameras 20a, 20b.
TABLE-US-00001 TABLE 1 Exemplary Camera Pixel ratio comparison to Tablet HD Camera Tablet 1.sup.st Testing Camera 20a 2590 × 942 Resolution (5 million Pixels) 2.sup.nd Testing Camera 20b 2590 × 1942 Resolution (5 million Pixels) Display Area resolution for 1200 × 1720 560 × 600 Top Section of Display 2 (2 million Pixels) (.33 million Pixels) (Testing Camera 20a) Display Area resolution for 1200 × 1395 520 × 600 Bottom Section of Display 2 (1.67 million Pixels) (.31 million Pixels) (Testing Camera 20b) Total Pixel ratio 3.5 million Pixels (5:1) .64 million Px
[0035] After the mobile device 1 is positioned in the testing chamber 10, the defective pixel mobile application commands the mobile device 1 to make the display 2 black with backlight turned ON to a maximum setting. The first testing camera 20a captures a first image of the top section of the display 2 and the second testing camera 20b captures a second image of the bottom section of the display 2. In general, the first image and the second image are then analyzed for any sudden drastic changes in the intensity of light across the black display 2. Any changes are recorded and are compared against a set of parameters that will determine if the RGB value falls under damaged pixel criterion or a good pixel criterion, as will be described in more detail below.
[0036] In general, each captured image is analyzed to find any defective pixels and then analyzed to determine if the defective pixel is large enough to be considered as a failure. If it falls into fail criteria, a circle is applied around the defective pixel to visually show the location of the defective pixel in the captured image. The same analysis is conducted for captured images of both the top and bottom sections of the display 2 using the first testing camera 20a and the second testing camera 20b. An overlap area is also considered.
[0037] Now with respect to
[0038] Firstly,
[0039] Firstly, the captured images of the top and bottom sections of the display 2 are analyzed and cropped to the display 2 size by a computer which then runs several steps to perform a defective pixel detection analysis. An example of a picture after cropping to the display 2 size is shown in
[0040] Each captured image from the first testing camera 20a and the second testing camera 20b is a photographic representation of rows and columns of pixels of the display 2, for instance an LED display, and each pixel is made of Red, Green and Blue values (RGB values) in the shown embodiment. However, one skilled in the art should appreciate that the method could be easily applied to other color models, including CMYK.
[0041] In the shown embodiment, and according to the invention, the RGB values vary from 0-255. For example, a pure white pixel color has values of R=255, G=255 and B=255, and a pure black pixel color has values of R=0, G=0 and B=0. As shown in
[0042]
[0043] The criteria check step is performed by comparing the questionable pixels against 8 surrounding pixels. In the event more than 4 of the surrounding pixels show a significant variance from the questionable pixel, then the questionable pixel is categorized as a drastic change pixel. All the questionable pixels are analyzed in this method and grouped within a 15×15 pixel range. If there are more than 5 drastic change pixels in one group, then it shows that there is a defective pixel in the mobile device 1 which is then considered a failure.
[0044] A pixel is considered questionable if it shows a drastic change when compared to adjacent pixels. A drastic change is defined as a drastic change in RGB values for the questionable pixel when compared to 8 surrounding pixels. If any of the RGB value changes should be above 25, and if the other 2 values are greater than 3, then it is considered a questionable pixel.
[0045] In an exemplary embodiment, shown in Table 2 and Table 3, the difference of R, G, and B values are calculated by comparing all the 8 surrounding pixels against a center pixel. After finding the differences, each set of RGB values is then checked to see if the difference of one of them changed by a value of at least 25, and if one of the 2 values should be greater than 3. This indicates that there is a drastic change. In the below example, Table 2, 6 out of 8 surrounding pixels marked in red falls into that criterion. Since it is higher than the set limit of 4, the center pixel is considered a questionable pixel. This analysis is conducted for all the pixels for which at least two of the R, G and B have values greater than 30.
[0046] A pixel is considered good if it does not show a drastic change. A drastic change is defined as a significant variance in RGB values compared to 8 surrounding pixels. In order to be a good pixel, the variance in any RGB value must be below 25 as compared to the 8 surrounding pixels under analysis. In the event one value is greater than 25, then each of the remaining 2 values must have a variance below 3. Tables 4 and 5 show the analysis of a good pixel.
[0047] Next, questionable pixels are grouped within a 15×15 pixel range. This is grouped to find a defect in a damaged area. Up to four questionable pixels within a 15×15 pixel range in the captured picture is acceptable, since this equates to less than one defective pixel on the display 2. Five or more questionable pixels within a 15×15 pixel range in the camera picture will be categorized as a defective pixel. A few examples of a pixel damage group are shown in Table 6.
TABLE-US-00002 TABLE 6 Example of 3 Defective Pixel Groups - More than 4 questionable pixels in a 10 × 10 range Group #1: 8 Questionable Pixels X: 385 Y: 1036 R: 45 G: 21 B: 34 X: 384 Y: 1037 R: 74 G: 52 B: 38 X: 385 Y: 1037 R: 44 G: 33 B: 67 X: 386 Y: 1037 R: 33 G: 22 B: 56 X: 384 Y: 1038 R: 65 G: 64 B: 44 X: 385 Y: 1038 R: 23 G: 50 B: 67 X: 384 Y: 1039 R: 64 G: 40 B: 36 X: 385 Y: 1039 R: 24 G: 41 B: 49 Group #2: 16 Questionable Pixels X: 265 Y: 359 R: 56 G: 35 B: 42 X: 266 Y: 359 R: 59 G: 38 B: 45 X: 267 Y: 359 R: 26 G: 46 B: 53 X: 265 Y: 360 R: 99 G: 46 B: 42 X: 266 Y: 360 R: 109 G: 56 B: 52 X: 267 Y: 360 R: 116 G: 70 B: 55 X: 268 Y: 360 R: 98 G: 52 B: 37 X: 265 Y: 361 R: 100 G: 69 B: 48 X: 266 Y: 361 R: 113 G: 82 B: 61 X: 267 Y: 361 R: 114 G: 113 B: 85 X: 268 Y: 361 R: 94 G: 93 B: 65 X: 265 Y: 362 R: 37 G: 57 B: 46 X: 266 Y: 362 R: 71 G: 91 B: 80 X: 267 Y: 362 R: 113 G: 135 B: 86 X: 268 Y: 362 R: 102 G: 124 B: 75 X: 269 Y: 362 R: 17 G: 59 B: 45 Group #3: 7 Questionable Pixels X: 372 Y: 1488 R: 41 G: 53 B: 51 X: 373 Y: 1488 R: 55 G: 56 B: 40 X: 374 Y: 1488 R: 40 G: 41 B: 25 X: 373 Y: 1489 R: 55 G: 107 B: 58 X: 374 Y: 1489 R: 40 G: 92 B: 43 X: 373 Y: 1490 R: 28 G: 75 B: 57 X: 374 Y: 1490 R: 25 G: 72 B: 54
[0048] Table 6 shows that group#1 has 8 questionable pixels which correspond to approximately 2 defective pixels in one area on the tablet. Group#2 has 16 questionable pixels that correspond to 3 defective pixels in another area. Group#3 has 7 questionable pixels which correspond to approximately 2 defective pixels in another area of the tablet.
[0049] With respect to
[0050] Now with respect to
[0051] The mobile device 1 is loaded with a scratch detection mobile application to analyze scratches to the display 2 of the mobile device 1. The scratch detection mobile application can be loaded through known cables, connectors and communications protocols. The scratch detection mobile application includes an algorithm that uses a high quality images taken by one or more testing cameras 20 to identify defective pixels. In particular, in the exemplary embodiment, the scratch detection mobile application sets the whole display 2 white with backlight turned ON to a maximum setting. The scratch detection mobile application can be loaded with the pixel detection mobile application, as a single application, or separately. The scratch detection mobile application, in an exemplary embodiment, is a custom Android application, but may be an application that functions on other mobile device operating systems known to those with ordinary skill in the art.
[0052] If the scratch detection mobile application is loaded separately, the mobile device 1 is again manually positioned on a retention device 5 with the display 2 facing upward and way from the retention device 5. The retention device 5 is then positioned in the testing chamber 10. The testing chamber 10 is then closed and sealed from exterior light. The pair of cameras 20a, 20b are positioned from a top of the testing chamber 10 and face the display 2, as shown in
[0053] As shown in
[0054] In order to confirm the defective screen, a computer performs various image processing steps on captured images by the first testing camera 20a and the second testing camera 20b. These steps are described below.
[0055] Firstly, the pixels of the captured images are grouped. In an exemplary embodiment, each captured image is divided into small group of 8×8 pixel groupings (64 pixels). The standard deviation of RGB value for the whole group of 64 pixels is calculated and used in the next step.
[0056] Next, if the standard deviation of the RGB value for the whole group of 64 pixels is above a deviation threshold, it is determined that the group of 64 pixels includes a defective pixel and is represented by a single pixel with white color. If the standard deviation of the whole group of 64 pixels is below the deviation threshold, then the whole group of 64 pixels is considered a good pixel and is represented by a single pixel of black color. In an exemplary embodiment, the threshold is calculated by:
Equation 1:
[0057]
[RGB value−RGB value for the whole group (i.e. 64 pixels)]<Standard Deviation s=8
[0058] The analysis is done for all the 8×8 pixel groupings (64 pixels) of the captured image, and then the captured image with these 8×8 pixel groupings (64 pixels) is represented by an grouped image ( 1/64th in size to the captured image) with each pixel of the grouped image representing a 8×8 pixel grouping (64 pixels) of the captured image.
[0059] The grouped image is then converted to gray scale for further analysis.
[0060] Next, the grayscale converted image is applied with a grayscale spec map to make the grayscale converted image more clear for analysis.
[0061] The grayscale converted image, with an applied grayscale spec map applied, is then dilated to join the nearby dots together as shown in
[0062] Next, the grayscale converted image, which has been dilated three times and a grayscale spec map applied, is inverted. As a result, the white pixels are converted to black and the black pixels are converted to white so that a filter analysis can be performed. An inverted image of the grayscale converted image from
[0063] Next, the inverted image is then filtered to eliminate small artifacts that have zero consequence as part of the analysis. The filtered image of
[0064] As shown in
[0065] Finally, the final image is analyzed by the computer and checked for RGB values to find if there are any white pixels in the final image. If a white pixel is found, the scratch detection analysis identifies a scratch on the screen of the mobile device 1. In the example shown in
[0066] The foregoing illustrates some of the possibilities for practicing the invention. Many other embodiments are possible within the scope and spirit of the invention. Therefore, more or less of the aforementioned components can be used to conform to that particular purpose. It is, therefore, intended that the foregoing description be regarded as illustrative rather than limiting, and that the scope of the invention is given by the appended claims together with their full range of equivalents.