FOOD CONTAINER SYSTEM WITH IMAGE PROCESSING AND METHOD

Abstract

A method and container for providing an enclosure for a food item includes a plurality of grading marks and a docking station to dock an electronic device. Yet further, the system includes a processor configured to take one or more pictures of the food item using the electronic device, transmit the one or more pictures to a cloud, receive recommended recipes for the food item and display the recommended recipes.

Claims

1. A method comprising: obtaining image data relating to a food item; delivering the data to a processor, and using the processor to carry out the steps of: obtaining information relating to the weight of the food item by; calculating the volume occupied by the food item based on one or more pictures of the food item; accessing the food database to obtain the density of the food item based on the type of food item; and obtaining the weight of the food item from the density of the food item, and the volume of the food item; comparing the image data with comparative data in a database; determining, based on the comparative data and the additional information, the type and quantity of the food item; selecting, based on the determined type and quantity, a recipe relating to the food item, the recipe comprising food preparation information relating to a plurality of food preparation steps, including configuration information relating to at least one food preparation step to be carried out by a kitchen appliance; providing the configuration information to the kitchen appliance; and configuring the kitchen appliance to carry out the at least one food preparation step based on the configuration information.

2. Image processing apparatus comprising: an electronic device arranged to obtain image data relating to a food item, and to deliver the data to a processor, the processor being configured to: obtain information relating to the weight of the food item by; calculating the volume occupied by the food item based on one or more pictures of the food item; accessing the food database to obtain the density of the food item based on the type of food item; and obtaining the weight of the food item from the density of the food item, and the volume of the food item; compare the image data with comparative data in a database; determine, based on the comparative data and the additional information, a type and quantity of the food item; select, based on the determined type and quantity, a recipe relating to the food item, the recipe comprising food preparation information relating to a plurality of food preparation steps, including configuration information for operating a kitchen appliance to carry out at least one of the plurality of food preparation steps; provide the configuration information to the kitchen appliance; and configure the kitchen appliance to carry out the at least one food preparation step based on the configuration information.

3. Apparatus as claimed in claim 2, in which the processor is configured to provide the recipe to the electronic device.

4. Apparatus as claimed in claim 2, in which the recipe comprises instructions for programming a kitchen appliance.

5. Apparatus as claimed in claim 2, in which the image data comprises a bar code providing information about the type and/or quantity of the food item.

6. Apparatus as claimed in claim 2 in which: the database comprises pictures and recipes relating to a plurality of food items; and the processor is configured to: select one or more recipes based on the type of the food item: and transmit information about the one or more recommended recipes to the electronic device.

7. Apparatus as claimed in claim 6 in which the image data comprises one or more pictures and wherein the processor is configured to match the one or more pictures of the food item with the pictures of the plurality of food items available in the database.

8. Apparatus as claimed in claim 2, wherein the processor is configured to obtain information relating to the weight of the food item by: calculating the volume occupied by the food item based on grading marks visible in the one or more pictures of the food item, wherein the grading marks indicate the size of the food item in each or any of the three dimensions.

9. Apparatus as claimed in claim 2, wherein the processor is configured to calculate the volume occupied by the food item by creating a 3D digital model of the food item based on the one or more pictures of the food item and then enumerating the number of voxels in the 3D digital model to determine volume of the food item.

10. Apparatus as claimed in claim 2, wherein the food database further includes nutritional information about a plurality of food items, wherein the nutritional information includes amount of calories, carbohydrates and protein in each food item in the plurality of food items.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The present invention is now described, purely by way of example, with reference to the accompanying diagrammatic drawings, in which:

[0020] FIG. 1A and FIG. 1B show, in perspective view, one example of a food-grading device according to the present disclosure;

[0021] FIG. 1C shows a side view of the food-grading device and FIG. 1D shows a top view of the food-grading device according to the present disclosure;

[0022] FIG. 2 is a block diagram of a food-grading system according to the present disclosure;

[0023] FIG. 3 is a flowchart of a method for grading food and recommending recipes according to the present disclosure; and

[0024] FIG. 4 illustrates a user interface of a mobile phone application that receives and displays recipes according to the present disclosure.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0025] Referring now to FIGS. 1A and 1B, in which corresponding features carry the same reference numbers, a food-grading device 100 comprises a container 102 which acts as an enclosure for a food item 104; for example an apple. The container 102 may include one or more grading marks 106 on the surface of the container 102. Each mark in the grading marks 106 may be an etch, a depression, a raised portion, a print, or any other structure that is clearly observable from inside and outside of the container 102. The grading marks 106 indicate a unit of measurement, such as centimeters or inches.

[0026] The container 102 is cube shaped. However, the container 102 may be of any other suitable shape such as spherical, cuboid or conical. The container 102 may have one or more openings that allow a user to place the food item 104 inside the container 102. For example, the one or more side faces of the container 102 may open as a gate or the top face of the container 102 may be a lid that can be lifted of the container 102 to allow the user to place the food item 104 in the container 102. Further, the container 102 may be transparent such that the food item 104 is visible to the user at all times. Yet further, the container 102 is may be hermitically sealable which ensures that no extraneous elements enter the container 102 when the food item 104 is placed in the container 102. The container 102 may further be dishwasher-safe which makes it easy for the user to clean the food-grading device 100. The food-grading device 100 may also include a weighing scale to determine weight of the food item 104. Further, the food-grading device 100 may include one or more sensors to perform chemical analysis of the food item 104. For example, the food-grading device 100 may include an odor sensor that is used to capture the odor given out by the food item 104 and detect chemical nature of odor molecules. The food-grading device 100 may also be connected to a network that includes, but is not limited to, the Internet, LAN, MAN, WAN, or the like.

[0027] Further, the food-grading device 100 may include a docking station 108, which is used to dock an electronic device 110 including a smart phone, a tablet, a camera, a 3D scanner and a barcode scanner. In the example shown in FIG. 1B, a smartphone is docked on the docking station 108. The docking station 108 may electronically connect the food-grading device 100 and the electronic device 110 using one or more ports available on the docking station 108. When activated, the electronic device 110 is configured to take pictures of the food item 104. For example, the user may use controls (not shown) available on the outer surface of the food-grading device 100 to instruct the food-grading device 100 to initiate taking pictures using the electronic device 110. Alternatively, the user may directly activate the electronic device 110 to take pictures.

[0028] Further, the container 102 may have a turntable installed on the bottom face of the food-grading device 100. The turntable rotates the food item 104 at a predefined rate, which allows the electronic device 110 to take multiple pictures of the food item 104 from various angles. This helps in grading the food item 104 more accurately. Further, a 3D digital model of the food item 104 may be created using the multiple pictures of the food item 104 taken from various angles. FIG. 1C shows a front view of the food-grading device 100 and FIG. 1D shows a top view of the food-grading device 100.

[0029] FIG. 2 shows a block diagram of a food-grading system 200 according to the present disclosure. The food-grading system 200 includes the food-grading device 100 connected to a cloud 202. Examples of clouds are Windows Azure™, Amazon EC2™, Bungee Connect™, Google App Engine™, and others. The cloud 202 includes a food database 206 includes details of a plurality of food items including names, pictures and contents of the food items.

[0030] Further, the cloud 202 includes a food-grading service unit 204, which receives one or more pictures for the food item 104 from the food-grading device 100. Further, the food-grading service unit 204 may also receive additional information including weight and chemical analysis report of the food item 104. Thereafter, the food-grading service unit 204 performs image processing on the one or more pictures obtained to determine the type and quantity of the food item 104. Image processing may include matching the one or more pictures of the food item 104 with the pictures available in the food database 206. For example, the food-grading service unit 204 may determine that the food item 104 is an apple based on image processing. Then, the food-grading service unit 204 may determine weight of the food item 104. Accordingly, the food-grading service unit 204 first calculates the volume occupied by the food item 104. To accomplish this, the food-grading service unit 204 obtains the position of the grading marks 106 with respect to the food item 104 from image processing. The grading marks 106 in the one or more pictures help the estimation of the site of the food item 104 in each of the three dimensions, which is used to estimate the volume of the food item 104. Alternatively, a 3D digital model of the food item 104 may be created using the one or more pictures of the food item 104; thereafter, voxels (volume pixel) may be used to determine the volume of the food item 104. Once volume is known, the food-grading service unit 204 may access the food database 206 to obtain density of the food item 104 based on the type of the food item 104 detected. Thereafter, equation (1) below may be used to determine weight of the food item 104.

[00001]$\begin{array}{cc}\mathrm{Weight}=\mathrm{Density}\mathrm{of}\mathrm{the}\mathrm{food}\mathrm{item}\times \mathrm{Volume}\mathrm{of}\mathrm{the}\mathrm{food}\mathrm{item}\times \mathrm{Acceleration}\mathrm{due}\mathrm{to}\mathrm{gravity}& \left(1\right)\end{array}$

Alternatively, the food-grading device 100 includes a weighing scale and sends the weight information to the food-grading service unit 204.

[0031] Moreover, the food database 206 may include nutritional information about each food item in the food database 206. For example, the nutritional information and other information including pictures in the food database 206 may advantageously be obtained on an ad-hoc basis depending on requirements of the food-grading service unit 204 from databases such as the U.S. Department of Agriculture (USDA) nutrient database. The nutritional information may indicate the amount of calories, carbohydrates, protein, etc. of various food items. Yet further, the food database 206 includes recipes for various food items. Therefore, the food-grading service unit 204 may provide the nutritional information about the detected food item 104 to the user and recommend recipes for the detected food item 104. This information may be sent to one or more of the electronic device 110 and a kitchen appliance 208 such that user may use these devices to access the information. This is explained in further detail in conjunction with FIG. 4 below. For example, the user may use a touch display 210 on the kitchen appliance 208 (the food processor 208) to access the information. The user may optionally use the touch display 210 to provide feedback to the food-grading service unit 204. Further, the food-grading service unit 204 may send instructions to be followed by the kitchen appliance 208 for various recipes. Yet further, the user may have a user account with the food-grading service unit 204. Accordingly, the food-grading service unit 204 may recommend recipes to balance meals of the user and to help the user achieve predetermined health goals, or on feedback received from the user or from other users. The food-grading service unit 204 may also determine quality of the food item 104 and share this information with the user.

[0032] In another embodiment, image processing may detect a barcode, which may directly provide information about type and quantity of the food item 104.

[0033] The food-grading service unit 204 also comprises a processor, coupled to the food database 206 for generating the recipe recommendations based on the detected food item 104.

[0034] In an exemplary embodiment, a user may place a cucumber in the food-grading device 100. When the cloud 202 detects the food item as cucumber, it recommends multiple recipes to the user. The recommended recipes may be displayed on the touch display 210 of the food processor 208. The user may select a recipe titled “Cold Cucumber Soup Recipe” from the recommended recipes. Thereafter, the touch display 210 displays full details (shown below) of the selected recipe including name of the recipe, number of serves, preparation time, electronic appliances required, ingredients required and the method steps.

Recipe: “Cold Cucumber Soup Recipe”

Serves: 4-6

[0035] Preparation time: 10 minutes
Can be made with these appliances: Food processor

Ingredients:

[0036] 1. Cucumbers 2 (about 750 g)

2. Salt 01 tsp

[0037] 3. Fresh garlic 01 to 1 clove
4. Spring onions, white 6

5. Yoghurt 115 g

[0038] 6. Dill 1 tbsp
7. Buttermilk or cream 200 g

Method:

[0039] 1. Peel and cut the cucumber length ways then scoop out the seeds.
2. Attach the knife blade to the food processor,
3. Place the cucumber, salt, garlic, spring onion, yoghurt and dill into the food processor bowl.
4. Process to a smooth puree on high speed.
5. To finish the soup attach the whisk.
6. Switch the machine to low speed and whisk in the buttermilk.
7. Traditionally we serve such soup in a bowl set over a bowl of ice. To finish the soup top with some chopped dill and garlic croutons,

[0040] Further, the cloud 202 may send instructions to the food processor 208 to program the food processor 208 based on the method steps. For example, the food processor 208 is programmed to process at high speed at fourth method step and then switch to low speed at sixth method step.

[0041] As the user performs the tasks corresponding to each method step, the user may select the corresponding method steps one-by-one on the touch display 210. Therefore, when user selects the method step four on the touch display 210, the food processor 208 automatically processes the food at high speed. Similarly, when the user selects the method step six on the touch display 210, the food processor 208 automatically processes the food at low speed.

[0042] It will be appreciated that, in such circumstances, the cloud 202 may send instructions to two or more kitchen appliances including food processors, blenders mixers, fryers and ovens, to program each appliance based on the method steps of a selected recipe.

[0043] In another embodiment, the touch display 210 also allows user to order one or more ingredients for a selected recipe from an online store.

[0044] Referring now to FIG. 3, is a flowchart 300 illustrating method for grading food and recommending recipes according to the present disclosure. At step 302, a user places the food item 104 in the container 102. Then at step 304, the food-grading device 100 takes pictures of the food item 104 using the electronic device 110. The user may instruct the electronic device 110 to initiate taking pictures if the food item 104. Alternatively, the user may use controls (not shown) available on the food-grading device 100 to instruct the food-grading device 100 to initiate taking pictures using the electronic device 110. Further, the food-grading device 100 may determine weight using weighing scale available in the food-grading device 100. Yet further, the food-grading device 100 may perform chemical analysis of the food item 104 using available sensors that employ food-safe chemical analysis techniques.

[0045] Next at step 306, the food-grading device 100 transmits the one or more pictures and optionally the additional information obtained to the cloud 202. The additional information includes weight and chemical analysis report of the food item 104. The food-grading device 100 may directly transmit the one or more pictures and additional information to the cloud 202 through a connection to a computer network such as the Internet. Alternatively, the food-grading device 100 may instruct the electronic device 110 to transmit the one or more pictures and additional information to the cloud 202.

[0046] Optionally, to avoid unnecessary processing, the food-grading device 100 may consult a local memory (not shown) of the food grading device 100 before transmitting (or instructing the electronic device 110 to transmit) the one or more pictures or the additional information to the cloud 202. If the one or more pictures or the additional information corresponds to data already stored in the local memory of the grading device 100, the food-grading device will not transmit it to the cloud 202, but instead display nutritional information of the food item 104 and one or more stored recipes stored on the local memory of the grading device 100. Alternatively, instead of displaying the information, the food-grading device 100 could send the information to one or both of the electronic device 110 and the kitchen appliance 208. The data stored on the local memory of the grading device 100 could be previously sent to or received from the cloud 202 in a previous query and stored by the local memory of the grading device 100, or it could be preloaded data for common foodstuffs.

[0047] Once the cloud 202 receives the one or more pictures and additional information sent by the food-grading device 100, the food-grading service unit 204 available on the cloud 202 processes the one or more pictures and additional information at step 308. This includes performing image processing to determine the type and quantity of the food item 104. Further, the food-grading service unit 204 may use the food database 206 available with the cloud 202 to obtain nutritional information of the food item 104 and one or more recipes to recommend to the user. Further, the user may have an account with the food-grading service unit 204. Therefore, the food-grading service unit 204 may select recipes based on user preferences, health goals, availability of ingredients etc. Thereafter, the food-grading service unit 204 sends the information to one or more of the food-grading device 100, the electronic device 110 and the kitchen appliance 208. Finally, at step 310, the information is received at the user's end. The information may also include instructions to program the kitchen appliance 208 for various recipes.

[0048] Referring now to FIG. 4, is a user interface 400 of a mobile phone application installed on the electronic device 110 according to an embodiment of the present disclosure. The mobile phone application receives and displays recipes recommended by the food-grading service unit 204. In an example embodiment, the mobile phone application displays a recipe 402 for “chocolate steamed pudding”, The recipe 402 includes a picture 404 of the final dish. Further, the recipe 402 includes information and instructions 406 for preparing the recipe. The user may view further details using an “Ingredients” tab 408 and a “Method” tab 410. The mobile phone application may also allow the user to send the instructions 406 to one or more kitchen appliances to program the kitchen appliances for the selected recipe. The instructions 606 from the electronic device 110 may be sent to the one or more kitchen appliances via the cloud 202 or directly using a short-range wireless communication protocol such as Bluetooth, Zigbee, Infrared, Wi-Fi etc. The user interface 400 may optionally allow the user to send feedback to the food-grading service unit 204.

[0049] It will be understood that the present invention has been described above purely by way of example, and modifications of detail can be made within the scope of the invention.

[0050] Each feature disclosed in the description, and (where appropriate) the claims and drawings may be provided independently or in any appropriate combination.

[0051] Reference numerals appearing in the claims are by way of illustration only and shall have no limiting effect on the scope of the claims.