Method for estimating the volume of a remaining fluidum in a non-translucent recipient using an image of the recipient

Abstract

A method of estimating a volume of a remaining fluidum in a non-translucent recipient having an opening which area is wider than or equal to the area of a bottom of the recipient. The method includes taking a picture of the opening at the top side of the recipient by with camera, deriving from the picture characteristics of a first and second number of pixels representing a visible part of the recipient and a visible part of the fluidum in the recipient, respectively, requesting from a user an identification of the recipient, matching the identification with a stored identification in order to retrieve reference data about an empty recipient, and estimating a volume of the remaining fluidum in the recipient by means of the retrieved reference data and a ratio of the characteristics of the first number of pixels and the characteristics of the second number of pixels.

Claims

1. A method of estimating a volume of a remaining fluidum in a non-translucent recipient having an opening at a top side of the recipient which area of the opening is wider than or equal to the area of a bottom of the recipient, the method being implemented as an application program for an electronic device having a camera, said method comprising the steps of: taking a picture of the opening at the top side of the recipient by means of the camera; the electronic device: deriving from the picture characteristics of a first number of pixels representing a visible part of the recipient; deriving from the picture characteristics of a second number of pixels representing a visible part of the fluidum in the recipient; requesting from a user an identification of the recipient; receiving from the user the identification of the recipient; matching the received identification with a stored identification in order to retrieve reference data about an empty recipient; and estimating a volume of the remaining fluidum in the recipient by means of the retrieved reference data and a ratio of the characteristics of the first number of pixels and the characteristics of the second number of pixels, wherein the picture is taken from a first arbitrary angle to the recipient and from a first arbitrary distance to the recipient as long as the remaining fluidum and the recipient are visible for the camera, wherein a picture for creating the reference data is taken from a second arbitrary angle to the recipient and from a second arbitrary distance to the recipient as long as the remaining fluidum and the recipient are visible for the camera, and wherein the second arbitrary angle and the second arbitrary distance are independent from the first arbitrary angle and from the first arbitrary distance respectively.

2. The method according to claim 1, wherein the reference data comprises at least the volume of the fluidum when the recipient is fully filled.

3. The method according to claim 2, wherein the electronic device has a user interface screen and said method further comprises the step of displaying the estimated volume of the remaining fluidum on the user interface screen.

4. The method according to claim 3, wherein the estimated volume is expressed in volume units of the fluidum.

5. The method according to claim 1, wherein the electronic device comprises a storage device configured to store the reference data and the application program.

6. The method according to claim 1, wherein the electronic device is connectable to a server via a network, the reference data and the application program being stored on the server, said method further comprising the steps of transmitting the picture to the server and transmitting the estimated volume from the server to the electronic device.

7. An electronic device comprising: a user interface screen; a camera; a storage device configured to store reference data about an empty recipient; and the application program for implementing the method according to claim 1, wherein said electronic device is configured to invoke the application program via the user interface screen and to display the estimated volume of remaining fluidum in the recipient at the user interface screen.

8. The electronic device according to claim 7, wherein the electronic device is a mobile apparatus.

9. The electronic device according to claim 8, wherein the mobile apparatus is a mobile phone.

10. An electronic device comprising: a user interface screen; a camera; wherein the user interface screen and the camera are connectable to a server on which reference data about an empty recipient are stored, wherein the application program for implementing the method according to claim 1 is stored on the server, and wherein said electronic device is configured to invoke the application program via the user interface screen and to display the estimated volume of remaining fluidum in the recipient at the user interface screen.

11. The electronic device according to claim 10, wherein the electronic device is a mobile apparatus.

12. The electronic device according to claim 11, wherein the mobile apparatus is a mobile phone.

13. A computer program product embodied on a non-transitory computer readable medium that, when executed on a processor, performs the method according to claim 1.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) The present invention will become more fully understood from the detailed description given herein below and the accompanying drawings which are given by way of illustration only, and thus are not limitative of the present invention, and wherein:

(2) FIG. 1 is a schematic view of an image reproduction apparatus comprising a roll of media;

(3) FIG. 2 is a schematic view of an electronic device according to the present invention;

(4) FIG. 3 is a schematic view of a picture taken of an end grain of the roll;

(5) FIG. 4 is a picture taken from an end grain of the roll and the width of the roll;

(6) FIGS. 5-8 are schematic views of a picture taken of the end grain of the roll and a representation of the width of the roll; and

(7) FIG. 9 is a schematic view of a picture taken of a top side of a recipient filled with a remaining amount of fluidum.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

(8) The present invention will now be described with reference to the accompanying drawings, wherein the same or similar elements are identified with the same reference numerals throughout the several views.

(9) FIG. 1 shows a reproduction system comprising workstations 8A-8C connected to a network N. A reproduction apparatus 5 is available for a user working with one of the workstations 8A-8C and intending to send a print job comprising a set of documents from one of the workstations 8A-8C to the reproduction apparatus 5. The reproduction apparatus 5 is connected to the network N and suited for receiving print jobs from the workstations 8A-8C. The network N may be wireless. The reproduction apparatus 5 comprises a local user interface unit 2 suitable for displaying print jobs, selecting a print job and editing settings of a print job. A control unit 3 is suited to store a queue of print jobs, to check whether or not the queue is empty and to submit a document from a queued print job to the print engine of the reproduction apparatus 5 in order to be printed. Documents of a print job are stored in a storing device 7 contained in the control unit 3 at least for the time period that the document is printed by the print engine and finished by a finisher.

(10) The reproduction apparatus 5 works up a roll 1 of media having a width indicated by an arrow 1c and a length. The length of the media on the roll is defined as a dimension of the media perpendicular to the width of the media in the plane of the media when completely unrolled. Media material may be paper, textile, metal plates, printed circuit plates, solar cell plates, overhead sheet material, plastic or any other kind of suitable receiving material or substrate. The roll 1 has been put in a holder. An end grain of the roll 1 comprises a part 1a of the core of the roll 1 and a side part 1b of the media of the roll 1. In FIG. 1, the core of the roll 1 and the side part of the media on the roll 1 are visible, but image reproduction apparatuses exist that have a housing that shields the roll such that the core of the roll and the media on the roll are not visible from the outside of the reproduction apparatus 5.

(11) FIG. 2 is a schematic view of an electronic device 11 according to the present invention. The electronic device 11 comprises a user interface screen 15 for displaying user interactive dialogs and for requesting data to be input by a user, a storage unit 13 for storing pictures, applications, etc., and a camera 14 for taking pictures of objects in front of the eye of the camera 14. The storage unit 13 may comprise the reference data according to the present invention. When an application program according to the present invention is started, a picture is asked to be taken by means of the camera 14 from an end grain of a roll to be investigated. The picture is stored in the storage unit 13 for further pixel counting and calculations. The user interface screen 15 is used to display the result of the method according to the present invention, i.e. the at least one characteristic of the roll comprising the remaining media.

(12) In another embodiment, the electronic device 11 is connected via a network to a server. The reference data and/or the application program resides on the server. According to this embodiment, the picture is transmitted to the server for further pixel counting and calculations and the results of the calculations are transmitted to the electronic device 11.

(13) FIG. 3 is a schematic view of a picture 20 taken of an end grain of the roll 1 at a background represented by a grey color. (Roll 1 has been removed from the image reproduction apparatus 5 or has been taken from a stock of rolls). The picture is taken in front of the end grain of the roll, so that no part of the roll that is behind the end grain is visible in the picture 20. A core of the roll is represented by a black filled-up ellipse 21. Media on the roll are represented by a white area 22 between the black filled-up ellipse 21 and another larger ellipse 23.

(14) A picture like FIG. 3 may be taken by the camera 14 of the electronic device 11 for an unused roll and stored in the storage unit 13 of the electronic device 11 as a second picture. The picture in FIG. 3 is taken by the camera 14 of the electronic device 11 for a used roll and stored in the storage unit 13 of the electronic device 11.

(15) An application stored in the storage unit 13 is able to count the pixels of the picture 20 which represent the core of the roll, i.e. the pixels within and on the black filled-up ellipse 21 resulting in a first number N.sub.1 of pixels. The application is also able to count the pixels of the picture 20 which represent the remaining media on the roll, i.e. the pixels of the picture 20 which have a white color within the larger ellipse 23, resulting in a second number N.sub.2 of pixels. The pixels of the grey background are not counted.

(16) Such counting of a first number and a second number is also established for the second picture of an unused roll—in the storage unit 13 or at an external server—resulting in a first reference number N.sub.1ref and a second reference number N.sub.2ref. The second picture may be taken from another angle and/or from another distance. The storage unit 13 also has stored a number L.sub.ref of the length of an unused roll of the same kind as roll 1. The goal of the present invention is to estimate a length L of the remaining media on the roll 1.

(17) The following formula (1) reveals the calculation of the wanted length L:
L=L.sub.ref*(N.sub.2/N.sub.1)/(N.sub.2ref/N.sub.1ref)  (1)

(18) For convenience reasons, the picture 20 has been taken just in front of the end grain of the roll 1, but may be taken from any angle as long as the end grain is still visible for the camera's eye. The concentric ellipses 21, 23 will become elongated or broadened. At each angle shot at each distance, the eccentricity of the smaller ellipse 21 will be the same as the eccentricity of the larger ellipse 23. Therefore the elongation or broadening of the ellipses 21, 23 does not effect the truth of the formula (1) here-above. It is noted that a change in distance from the camera to the end grain also does not effect the truth of formula (1)

(19) The term N.sub.2ref/N.sub.1ref may be stored as a quotient Q.sub.ref in the reference data. The quotient Q.sub.ref and the term N.sub.2/N.sub.1 are invariant qua shot angle with the end grain and qua shot distance to the end grain.

(20) FIG. 4 shows a picture 30 taken of the end grain of the roll and the width of the roll, wherein the picture of the end grain of the roll is taken from a position outside a virtual roll being an extension of the real roll. A picture of the entire roll is taken. First, the remaining media length on the roll is calculated. Then, the width of the roll is calculated.

(21) It is noted that the calculation of the remaining media length described here-below differs from the calculation of the remaining media length corresponding to FIG. 3, in that the picture according to FIG. 3 is taken from directly above the roll standing on end. The calculation described here-below is done from a picture taken at an angle (around 45°) to the horizontal plane in order to make the calculation of the roll width with the same picture.

(22) It is assumed that the picture is taken in a way so the roll axis is more or less in the widest direction of the picture frame (in order to maximize the roll image on the picture) called “y-direction” or “ordinate axis”. The axis orthogonal to “y-direction” is called the “x-direction” or “abscissa axis”. It is also assumed that a reference calibration of the Q ratio of a full roll (Q.sub.ref as introduced before) has been done as described with regard to FIG. 3. In the shot conditions of the picture, the perspective effect (due to the camera focal length) in the roll core plan is neglected. The picture is binarized to transform all the paper pixels 31 as white pixels, and the roll core 32 and background pixels as black pixels.

(23) A computation of the remaining length on the core is now elucidated on behalf of FIG. 5. Assuming that in the picture the apparent roll width (along the y-direction) is larger than the apparent diameter of the roll (in the x-direction), a roll axis 33 can be found by calculating a linear regression on the position of the white pixels from the binarized bitmap. The Cartesian equation of the axis 33 is: b=y−ax.

(24) The ‘a’ and ‘b’ coefficients are estimated in the following way. The white pixels coordinates are named (x.sub.w,y.sub.w). We define the usual averages, centered variances and covariance on these coordinates:

(25) $\stackrel{\_}{x_w}=\frac{.Math.x_w}{n};\stackrel{\_}{y_w}=\frac{.Math.y_w}{n};$$\stackrel{\_}{x_w^2}=\frac{.Math.{\left(x_w\stackrel{\_}{x_w}\right)}^2}{n};\stackrel{\_}{y_w^2}=\frac{.Math.{\left(y_w\stackrel{\_}{y_w}\right)}^2}{n};$$\stackrel{\_}{{\mathrm{xy}}_w}=\frac{.Math.\left(x_w-\stackrel{\_}{x_w}\right).Math.\left(y_w-\stackrel{\_}{y_w}\right)}{n};$${\sigma }_{x_w}^2=\stackrel{\_}{x_w^2}-\stackrel{\_}{x_w^2};{\sigma }_{y_w}^2=\stackrel{\_}{y_w^2}-\stackrel{\_}{y_w^2};{\sigma }_{{\mathrm{xy}}_w}=\stackrel{\_}{{\mathrm{xy}}_w}-\stackrel{\_}{x_w}.Math.\stackrel{\_}{y_w};$

(26) The chosen estimator for the ‘a’ coefficient of this axis is:

(27) ${\tilde{a}}_w=\frac{{\sigma }_{y_w}^2-{\sigma }_{x_w}^2+\sqrt{{\left({\sigma }_{y_w}^2-{\sigma }_{x_w}^2\right)}^2+4.Math.{\sigma }_{{\mathrm{xy}}_w}^2}}{2.Math.{\sigma }_{{\mathrm{xy}}_w}}$

(28) The estimator of ‘b’ is then:
{tilde over (b)}.sub.w=y.sub.w−ā.sub.w.Math.x.sub.w

(29) After the binarization, the roll core appears as a black ellipse 32 included totally inside the white pixels 31 on the processed picture as shown in FIG. 6. The ellipsis parameters are determined: center coordinates (x.sub.k,y.sub.k), a semi-major axis q.sub.k and a semi-minor axis p.sub.k.

(30) The black pixels coordinates are named (x.sub.k,y.sub.k) and, similar to the white pixels, the averages, centered variances and covariance are calculated:
x.sub.k; y.sub.k; σ.sub.x.sub.b.sup.2; σ.sub.y.sub.b.sup.2; σ.sub.xy.sub.b;

(31) The estimators for x.sub.k,y.sub.k,p.sub.k and q.sub.k are:

(32) ${\tilde{x}}_k=\stackrel{\_}{x_k};{\tilde{y}}_k=\stackrel{\_}{y_k};$${\tilde{q}}_k=\sqrt{2.Math.\left({\sigma }_{x_k}^2.Math.{\sigma }_{y_k}^2.Math.\sqrt{4.Math.{\sigma }_{{\mathrm{xy}}_k}^2.Math.{\left({\sigma }_{x_k}^2{\sigma }_{y_k}^2\right)}^2}\right)};$${\tilde{p}}_k=\sqrt{2.Math.\left({\sigma }_{x_k}^2+{\sigma }_{y_k}^2-\sqrt{4.Math.{\sigma }_{{\mathrm{xy}}_k}^2+{\left({\sigma }_{x_k}^2-{\sigma }_{y_k}^2\right)}^2}\right)};$

(33) The picture is taken in a way that q.sub.k is following the x axis. In any case, we can determine if it is really the case by testing {tilde over (q)}.sub.k>{tilde over (p)}.sub.k.

(34) Like the white pixels, the semi-major axis equation of the core ellipse is evaluated from:

(35) ${\tilde{a}}_k=\frac{{\sigma }_{y_k}^2-{\sigma }_{x_k}^2+\sqrt{{\left({\sigma }_{y_k}^2-{\sigma }_{x_k}^2\right)}^2+4.Math.{\sigma }_{{\mathrm{xy}}_k}^2}}{2.Math.{\sigma }_{{\mathrm{xy}}_k}}$${\tilde{h}}_k=\stackrel{\_}{y_k}-{\tilde{a}}_k.Math.\stackrel{\_}{x_k}$

(36) As shown in FIG. 7, by construction, the ellipse 32 of the roll core (in black) and the ellipse of the roll paper seen from aside (in white) have a common center and common semi-axis lines 34.

(37) We compute the projected length 2q.sub.w, of the remaining paper semi-axis 34 (in dark color on FIG. 7) by searching the minimal and maximal abscissas x.sub.w.sub.min and x.sub.w.sub.max of the white pixels on the semi-axis equation:
{tilde over (b)}.sub.k=y−ã.sub.k.Math.x
2{tilde over (q)}.sub.w=(x.sub.w.sub.maxx.sub.w.sub.min).Math.√{square root over (1|ã.sub.k.sup.2)}

(38) The area ratio Q.sub.m of the paper ellipse area by the core ellipse area is then:

(39) $Q_m=\frac{{\tilde{q}}_w^z}{{\tilde{q}}_k^2}$

(40) An estimation of the remaining length can be deduced as described with regard to FIG. 3.

(41) Secondly, the roll width is computed. Since it is assumed that the core is physically a circle, the estimated angle θ between the camera optical axis and the roll core plan is evaluated with:

(42) $\tilde{\theta }={\sin }^{-1}\left(\frac{{\tilde{p}}_k}{{\tilde{q}}_k}\right)$

(43) It is also assumed that the user provides by the application program the physical diameter of the core Φ.sub.c. Therefore, the {tilde over (p)}.sub.k estimator can be used to calibrate the scale factor f.sub.c and to measure the lengths on the semi-minor axis of the core:

(44) $f_c=\frac{{\Phi }_c}{2{\tilde{p}}_k}$

(45) Minimal and maximal ordinates y.sub.w.sub.min and y.sub.w.sub.max are determined of the white pixels on the roll axis equation:
{tilde over (b)}.sub.w=y.sub.w−ã.sub.w.Math.x.sub.w.Math.

(46) The semi-minor axis 35 of the white ellipse according to FIG. 8 is deduced from its semi-major axis and the black ellipse eccentricity:

(47) ${\tilde{p}}_{\mathrm{kw}}={\tilde{q}}_w.Math.\frac{{\tilde{p}}_k}{{\tilde{q}}_k}$

(48) The projected width W.sub.pr of the roll 36 is then given by:

(49) $W_{\mathrm{pr}}=\frac{\left(y_{w_{\max }}-y_{w_{\min }}\right).Math.\sqrt{1+{\tilde{a}}_w^2}}{{\tilde{a}}_w}-2.\frac{{\tilde{p}}_k.Math.{\tilde{q}}_w}{{\tilde{q}}_k}$

(50) In this evaluation, the perspective effect is neglected and an evaluator {tilde over (W)}.sub.r of the physical roll width is:

(51) 0${\tilde{W}}_r=f_c\times \frac{W_{\mathrm{pr}}}{\cos \tilde{\theta }}.$

(52) A similar method may be used to estimate a remaining fluidum in a non-translucent recipient by means of an electronic device having a camera and storage means comprising reference data about an empty recipient. The reference data may also be stored on a server that has a connection to the electronic device via a network. At the server, the reference data and/or the application program resides.

(53) The recipient has an opening at a top side of the recipient. The area of the opening is wider than or equal to the area of a bottom of the recipient. The reference data comprises the dimensions of the recipient and the method comprises the steps of taking a picture of the maximal opening at the top side of the recipient by means of the camera, deriving from the picture a first number of pixels representing a visible part of the recipient, deriving from the picture a second number of pixels representing the visible part of the fluidum in the recipient, and estimating the volume of the remaining fluidum in the recipient based on the reference data, the first number and the second number, wherein the reference data comprises at least the volume of the fluidum when the recipient is fully filled. The volume of the fluidum when the recipient is fully filled is equal to the volume of the recipient.

(54) By doing so, the volume is estimated by taking a photo with the camera of the electronic device and performing calculations with an application installed on the electronic device or on an external server connected to the electronic device. When a picture of the opening at the top side of the recipient is taken, the picture will show the recipient, the fluidum in the recipient and a background. The fluidum forms an inner part of the picture, while the recipient forms an outer part of the picture circumventing the inner part of the picture representing the fluidum. Usually, the fluidum and the recipient are easily distinguishable from each other. If this is not the case, a recipient may be selected having a deviating color than the color of the fluidum. Usually, the recipient and the background are also easily distinguishable. If this is not the case, a background may be selected having a deviating color than the color of the recipient. A vertical axis of the recipient is to be deduced from the picture. From this vertical axis, a first part corresponds to a distance from the top of the recipient to the fluidum and a second part corresponds to a distance from the bottom of the recipient to the top of the recipient. Knowing this, a simple formula for the remaining volume of the fluidum may be derived from these distances, and a volume of the recipient that is contained in the reference data.

(55) If in a calibration phase more than one recipient is measured, the reference data will comprise identification data for each kind of recipient having a wide opening at the top side of the recipient. The identification data are requested to be entered and added to the reference data during the calibration phase. The identification may be performed by scanning an identification code on the recipient. The recipient from which the first picture is taken may also be identified by entering identification data before the calculation of the remaining fluidum is performed. The entered identification data will be compared with the identification data already present in the reference data. According to a further embodiment, the amount of the fluidum in the recipient is expressed as a number of volume units of the fluidum in the recipient.

(56) Such a method comprises the steps of taking a picture from above the recipient with the camera, deriving from the picture a first number of pixels representing a fluidum in the recipient, deriving from the picture a second number of pixels representing the shape of the recipient, and estimating the remaining amount of fluidum in the recipient based on the reference data, the first number and the second number. The reference data may comprise the dimensions of the empty recipient and/or data derived from a picture of the empty recipient.

(57) For example, the volume of coffee in a cup or the amount of paint remaining in a bucket may be estimated by taking a picture from above as shown in FIG. 9, comparing with reference data present at the smart phone or at a server connected to the smart phone and deriving the remaining amount from the reference data and the recognition of the shape 40 of the recipient and an amount of the fluidum in the recipient 40 from the picture.

(58) The reference data contains the dimensions of the recipient object 40, i.e. a total volumn V.sub.total of the recipient is known. An image processing algorithm suitable for recognizing the shape of the recipient 40 and an amount of fluidum in the recipient is used to detect a level 43 of remaining fluidum in the recipient and the recipient shape. The recipient dimensions can be different due to the angle of the camera when taking the picture, but the deformations are isomorphic. Therefore, it is possible to estimate the remaining fluidum in the recipient from the liquid level.

(59) FIG. 9 shows a 3-dimensional container as a recipient and 2-dimensional shape on a top 41 of the recipient and a bottom 42 of the recipient. For convenience reasons, the top 41 and the bottom 42 are parallel, e.g. a bucket of paint. The total volume V.sub.total of the recipient 40 is known. The fluidum to be measured can be differentiated from the inside of the recipient 40. The diameter of the recipient is small enough to make negligible the difference of the apparent heights of the front side and the rear side. The recipient 40 is not translucent. However, for convenience reasons FIG. 9 shows all line curves necessary to explain the estimation of the remaining fluidum.

(60) The linear regression for detecting the vertical axis 33 in FIG. 5 explained in the previous embodiment is applied to detect the vertical axis of the recipient 40. From the vertical axis we can count the number of pixels of the height 44 of the front of the recipient (H) and we can count the number of pixels from the rear top edge to the inside-level of the contents (h) on line segment 45.

(61) The remaining volume of the fluidum is computed to be equal to:
V.sub.remaining=h/H*V.sub.total.

(62) In FIG. 9 the vertical sides 46, 47 are assumed to be parallel, but a recipient having a top opening that is wider than the bottom of the recipient may be envisaged. The formula of V.sub.remaining mentioned here-above is easily to be adapted to be suitable for such a recipient. For such a recipient the area size of the top opening and the area size of the bottom (determining an angle between two sides similar to the vertical sides 46, 47) may be part of the reference data and may be used in the calculations. If the diameter of the top opening equals X and the diameter of the bottom equals x, then the volume of the complete filled recipient is 1/12*π*(X.sup.3−x.sup.3). If the same heights H and h are deducted from the picture, we get the following approximating formula for the remaining fluidum:
V.sub.remaining=1/12*π*((X−(X−x)*(h/H)).sup.3−x.sup.3).

(63) It may be clear to the skilled person that the various embodiments of the method, the electronic devices and the image reproduction apparatuses may be combined in one way or another in order to establish an appropriate and applicable embodiment of the method according to the invention.

(64) The invention being thus described, it will be obvious that the same may be varied in many ways. Such variations are not to be regarded as a departure from the spirit and scope of the invention, and all such modifications as would be obvious to one skilled in the art are intended to be included within the scope of the following claims.