NON-DESTRUCTIVE METHOD TO PREDICT SHELF LIFE AND MATURITY OF PERISHABLE COMMODITIES

Abstract

A non-destructive method to predict the shelf life and maturity of perishable commodities using an intelligent vision system is presented here. The system includes a conventional camera and a vision processor (including shelf-life matrix, defect matrix and maturity matrix specific to each perishable commodity) which automatically determines ready for harvest condition, and the remaining shelf life of the perishable commodity.

Claims

1. A method for predicting the remaining shelf life of fruits and vegetables, the steps comprising: generating an image of said fruit/vegetable; resizing and cropping the image to the area of interest separating the Red, Green, Blue Channels from the original image; dissecting said image into a plurality of CIELAB colorspace; comparing the data with the combination of CIELAB colorspace based shelf life, defect and maturity matrices finding the match between the CIELAB color space and the colorspace from the shelf life, defect and maturity matrices predicting the remaining shelf life based upon the matching process

2. A method for predicting the ready for harvest condition for fruits and vegetables, the steps comprising: generating an image of said fruit/vegetable; resizing and cropping the image to the area of interest separating the Red, Green, Blue Channels from the original image; dissecting said image into a plurality of CIELAB colorspace; comparing the data with the combination of CIELAB colorspace based disease, defect and maturity matrices finding the match between the CIELAB color space and the colorspace from the disease, defect and maturity matrices predicting the harvest status based upon the matching process

3. A prediction method of claim 1, further configured to integrate into permanently affixed refrigerated/non-refrigerated drawers and/or cabinets.

4. A method of claim 2, further configured to integrate into robotic arm, equipped with handgrip mechanism to allow for automated harvesting.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Reference is made to the accompanying drawings in which is shown an illustrative embodiment of the invention, from which its novel features and advantages will be apparent.

[0026] FIG. 1 is a simplified illustration of the Non-Destructive Method to predict the maturity and the Remaining Shelf-Life Prediction methodology.

[0027] FIG. 2 is a simplified illustration of the Non-Destructive Method to predict the ready for harvest and documenting disease information of the fruit while on the tree. maturity and the Remaining Shelf-Life Prediction methodology.

[0028] FIG. 3 is a simple illustration of the CIELAB parameters.

[0029] FIG. 4a is an illustration of the process to analyze the state of the produce (in this case banana)

[0030] FIG. 4b is the analysis result of the scanned banana predicting that the banana is ready for harvest.

[0031] FIG. 5a is an illustration of the process to analyze the Remaining Shelf Life of the produce (in this case mango).

[0032] FIG. 5b is the analysis result of the scanned mango predicting the maturity stage and the shelf life remaining.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0033] Referring to FIG. 1, it will be seen that an illustrative includes there is produce at a distribution/retail center 1, an image of the produce is captured via an imaging device 2, three matrices, 3 Shelf Life Matrix, 4 Defect Matrix, and 5 is the Maturity Matrix either resides on the cloud server or on the imaging device itself, 6 is the analysis based upon the comparison of the image with the three matrices, 3, 4 & 5. 7 is the results panel which documents the maturity stage and remaining shelf life of the produce.

[0034] Referring to FIG. 2, it will be seen that an illustrative includes there is produce on a tree 1, an image of the produce is captured via an imaging device 2, 3 is the Maturity Matrix, 4 is disease matrix which either reside on the cloud server or on the imaging device itself, 5 is the analysis based upon the comparison of the image with the three matrices, 6 is the results panel which documents the maturity stage and determines if the produce is ready for harvest or not. Based upon the disease matrix, if the produce is found to be diseased, the results panels will state the produce is diseased.

[0035] Referring to FIG. 3, the CIELAB color chart is explained, with the depiction of Hue 1, and Chroma 2.

[0036] Referring to FIG. 4a, the analysis for ready for harvest for fruits and vegetables is demonstrated.

[0037] Referring to FIG. 4b, the results for the produce item are demonstrated.

[0038] Referring to FIG. 5a, the analysis for remaining shelf life for fruits and vegetables is demonstrated.

[0039] Referring to FIG. 5b, the results for the remaining shelf life and maturity of the produce item are demonstrated.

[0040] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Although any methods and materials similar or equivalent to those described herein can be used in the practice or testing of the present invention, example methods and materials are now described.

[0041] As used in the specification and the appended claims, the singular forms “a,” “an” and “the” include plural referents unless the context clearly dictates otherwise.

[0042] In one aspect, the present disclosure provides a method for predicting the ready for harvest condition of fruits and vegetables, the method comprising generating an image of the said fruits/vegetables, resizing and cropping the image, separating the red, green and blue channels from the image, and converting the image to CIELAB color space and identifying parameters such as, l*, a* and be, hue, and chroma, and comparing with the produce specific Maturity Matrix to predict harvest ready or not, FIGS. 4a and 4b. In another aspect, the present disclosure provides a method for predicting the remaining shelf life of the fruits and vegetables, the method comprising generating an image of the said fruits/vegetables, resizing and cropping the image, separating the red, green and blue channels from the image, and converting the image to CIELAB color space and identifying parameters such as, 1*, a″ and b*, hue, and chroma, and comparing with the produce specific Maturity Matrix, Shelf Life Matrix and the Defect Matrix to predict remaining shelf life of the fruit/vegetable, FIGS. 5a and 5b.

EXAMPLES

[0043] The invention will now be illustrated, but not limited, by reference to the specific embodiments described in the following examples.

Example 1

[0044] A mango photo is taken as in FIG. 5a.

[0045] Resize Image to fixed size of 1816×2688. Next crop image to obtain minimum area image. To crop the image, we first check if the image is 3D, if yes, then convert to 2D gray image. Convert 2D gray image to binary black and White Image. Calculate areas and bounding boxes of the objects inside the black and white image. Find the maximum area object from calculated areas and corresponding bounding box for that object. Crop the image using the found bounding box.

[0046] One draws a free selection region of interest on the image or automatically select the fruit/vegetable. All values of pixels in original image except masked pixels are set to zero. Drawn region of interest is deleted. The overall image analysis is conducted thereafter. Based upon the selected area of interest, separate Red, Green, Blue channels from the original image are captured. The masked image is converted to CIELAB color space. The 1, a and b values are separated. Remove all lab and color values which are zero. Calculate huemin, huemax and hue. Histogram of Hue is plotted. There are 7 scenarios, and the image is analyzed based upon that: [0047] Based on L max [0048] Based on L min [0049] Based on A max [0050] Based on A min [0051] Based on B max [0052] Based on B min [0053] Based on Average values [0054] The based value is found and then corresponding to that value rest of the value is found. Data is shown in Table 1 below.

TABLE-US-00001 TABLE 1 CIELAB and RGB Parameters for the Mango Photo taken R G B l a b Hue Huemin Huemax Defect % 255 255 254 89.4302 −2.4083 60.2537 92.2888 92.2888 92.2888 0 240 180 74 52.7246 23.6763 60.1477 68.5136 68.5136 68.5136 0 255 255 255 65.9943 34.4015 71.5891 64.3338 64.3338 64.3338 0 255 255 255 87.4243 +3.6636 63.9843 93.277 93.277 93.277 0 255 255 255 82.8101 7.3091 78.1241 84.6551 84.6551 84.6551 0 255 255 255 88.8711 −1.5904 58.2346 91.5643 91.5643 91.5643 0 239 198 119 69.7938 14.9872 71.1195 78.1 78.1 78.1 0

[0055] Calculate Maturity by Reading the maturity file data, which is shown in Table 2. First the Huemax from Table 1 is matched with Huemax from Table 2. For matching we calculate difference of calculated HUEMAX with each stage's HUEMAX from Table 2 and then taking the stage with least difference. After that we match HUEAVG using same above technique. Then we check if day calculated from HUEMAX and HUEAVG are same. If same then that's the maturity stage if not same then we will compare AMIN and AMAX further to decide maturity stage.

[0056] Suppose the HUEMAX and HUEAVG matched with Stage 3-to Stage 5 respectively, so now we will match AMIN and AMAX from stage 3 to stage 5

[0057] Match calculated AMAX with Table 2's AMAX for stage 3. Match calculated AMIN with Table 2's AMIN for stage 3. Find and save value which has least difference from above two values Repeat above three steps for all the stages from stage 3 to stage 5. This gives total 3 least values corresponding to stages 3, 4 and 5.

[0058] Find the minimum difference value among all the calculated values among above three values.

[0059] This is our maturity stage that is closest to our calculated value. We find the maturity stage of the Mango is Stage 5.

TABLE-US-00002 TABLE 2 Maturity Matrix - Maturity Data for Mango (ready to harvest) Stage AMAX AMIN HUEMAX HUEMIN HUEAVG STAGE 1- −2.8898 −23.4647 116.3376 95.3556 114.673 UNRIPE STAGE 2- 3.1741 −17.9865 111.0686 86.355 100.4461 EARLY RIPE STAGE 3 - 14.4234 −8.1765 96.4347 78.7858 90.5588 PARTIALLY RIPE STAGE 4 - 19.6962 −2.4537 91.8921 74.6495 85.2115 RIPE STAGE 5 - 20.8387 −1.7419 99.0804 27.0151 81.096 OVERRIPE/ DECAY

[0060] Read the shelf life data, which is shown in Table 3. First the Huemax from Table 1 is matched with Huemax from Table 3. For matching we calculate difference of calculated HUEMAX with each stage's HUEMAX from Table 3 and then taking the stage with least difference. After that we match HUEAVG using same above technique. Then we check if day calculated from HUEMAX and HUEAVG are same. If same then that's the maturity stage if not same then we will compare AMIN and AMAX further to decide maturity stage.

[0061] Suppose the HUEMAX and HUEAVG matched with Days 7 to 9 so now we will match AMIN and AMAX for these days.

[0062] Match calculated AMAX with Table 3's AMAX for Day 7. Match calculated AMIN with, Table 2's AMIN for day 7. Find and save value which has least difference from above two values

[0063] Repeat above three steps for all the days from Day 7 to Day 9. This gives total 3 least values corresponding to days 7, 8 and 9. Find the minimum difference value among all the calculated values among above three values. This is our shelf life that is closest to our calculated value. We find the remaining shelf life of the Mango by subtracting from the total number of shelf life to the matches shelf life. The remaining shelf life is found to be 12 days.

TABLE-US-00003 TABLE 3 Shelf-Life Matrix - Shelf Life Data for Mango (Day 1 to Day 19) DAYS AMAX AMIN HUEMAX HUEMIN HUEAVG 1 −9.34464 −22.9436 120.3509 106.1455 114.7813 2 −9.54247 −26.1497 109.3918 98.49451 104.5212 3 7.30502 −11.28 99.13905 82.8271 93.42326 4 3.773863 −11.6731 100.551 87.0441 93.45909 5 3.343492 −19.2558 109.0611 86.53275 101.2904 6 3.343492 −19.2558 109.0611 86.53275 90.63397 7 14.42341 −8.17647 96.43475 78.78582 90.63397 8 14.42341 −8.17647 96.43475 78.78582 90.63397 9 14.42341 −8.17647 96.43475 78.78582 90.63397 10 14.42341 −8.17647 96.43475 78.78582 90.63397 11 14.42341 −8.17647 96.43475 78.78582 90.63397 12 19.96494 −3.41899 100.3274 71.90615 84.92258 13 19.96494 −3.41899 100.3274 71.90615 84.92258 14 19.96494 −3.41899 100.3274 71.90615 84.92258 15 19.96494 −3.41899 100.3274 71.90615 84.92258 16 19.96494 −3.41899 100.3274 71.90615 84.92258 17 19.96494 −3.41899 100.3274 71.90615 84.92258 18 21.48506 −1.74188 99.08042 27.01506 80.93653 19 21.48506 −1.74188 99.08042 27.01506 80.93653

TABLE-US-00004 TABLE 4 Defect Matrix - Data for Browning Index for Mango Categories LowerRange UpperRange Shelf Life Loss (%) No Browning 0 5 0 Slight Browning 5.1 15 50 Moderate Browning 15.1 25 75 Serious Browning 25.1 100 100

[0064] Defect is calculated based upon L less than 45. Define the min (0) and maximum range (45). Find the L values in the min and max range. Find percentage by dividing by the number of found L values to total L values. Compare data with the defect matrix in Table 4. Match the defect with the ranges of the browning index. If matched then find the shelf life loss from the given percentage loss. Subtract this calculated percentage lost shelf life from the Remaining Shelf Life. In this example, the browning index was less than 2%, hence the Remaining Shelf Life is 12 days.

Example 2

[0065] A banana photo is taken as in FIG. 4a on a tree.

[0066] The image is resized and cropped. Conduct analysis using Table 6 based upon the following: [0067] a. Based on A max [0068] b. Based on A min [0069] c. Based on Average Values

[0070] Black-White Defect

[0071] Maturity stage calculation—if stage is 1 then product ready for harvesting.

TABLE-US-00005 TABLE 5 CIELAB and RGB Parameters for the Banana Photo taken R G B l a b Hue Huemin Huemax Defect % 255 255 255 85.2254 −22.7847 44.2865 117.2251 117.2251 117.2251 0 254 252 252 68.667 −30.0754 59.1601 116.9476 116.9476 116.9476 0 255 255 255 81.8865 −19.3354 40.0035 115.7732 115.7732 115.7732 0 255 255 255 77.9736 −33.306 64.8766 117.1748 117.1748 117.1748 0 255 255 255 74.4756 −30.5795 66.7264 114.6212 114.6212 114.6212 0 255 255 255 82.905 −22.4563 38.7684 118.9623 118.9623 118.9623 0 243.8153 246.6088 235.3764 76.8541 −27.8779 56.6651 116.1961 116.1961 116.1961 0

[0072] For finding if there is any disease on the banana read the disease file data, which is shown in Table 6. First the Amax from Table 1 is matched with Amax from Table -. For matching we calculate difference of calculated AMAX with each disease's7 AMAX from Table 6 and then taking the stage with least difference. After that we match AMIN using same above technique. Then we check if day calculated from AMAX and AMIN are same. If same then that's the maturity stage if not same then we will compare HUEAVG and HUEMAX further to decide disease.

[0073] Suppose the AMAX and AMIN matched with Stage 3, Stage 4 and to Stage 5 respectively, so now we will match AMIN and AMAX for these stages from stage 3 to stage 5 Match calculated HUEMAX with Table 6-'s HUEMAX for stage 3. Match calculated HUEAVG with Table -'s HUEAVG for stage 3. Find and save value which has least difference from above two values

[0074] Repeat above three steps for all the stages from stage 3 to stage 5. This gives total 3 least values corresponding to stages 3, 4 and 5.

[0075] Find the minimum difference value among all the calculated values among above three values. This is our disease that is closest to our calculated value. We found no disease on the banana.

TABLE-US-00006 TABLE 6 Disease Matrix - for Banana Disease AMAX AMIN HUEMAX HUEMIN HUEAVG Anthracnose, 29 −17 103 68 87.7 Medium Anthracnose, 37 −18 310 90 59.6 Severe Crown 26.9 −17.4 294 22.9 71.2 Rotting Gray Mold −29 12.3 298 68 116.7 Fusarium 38.6 −31 353 61.5 88.8 Roseum Healthy −3.3873 −34.5897 140.27 102.2007 121.9718

TABLE-US-00007 TABLE 7 Maturity Matrix - for Banana Stage AMAX AMIN HUEMAX HUEMIN HUEAVG STAGE 1 −3.3873 −34.5897 140.27 102.2007 121.9718 STAGE 2 −3.2526 −31.906 124.5533 96.8086 111.9377 STAGE 3 22.1084 −26.7638 115.8208 21.4318 100.6394 STAGE 4 9.4485 −19.3766 108.0123 76.3147 93.3819 STAGE 5 17.3249 −10.0394 99.3006 66.5324 89.186 STAGE 6 12.3982 −4.1874 93.5746 77.3122 88.726 STAGE 7 22.2797 −7.5335 108.8402 11.2733 83.7259