METHOD OF CONTROL, CONTROL SYSTEM AND GLASS FURNACE, IN PARTICULAR FOR TEMPERATURE/THERMAL CONTROL

Abstract

The invention relates to a method of monitor and/or control of operation of an in-dustrial furnace for processing a heated material, in particular for processing a melt in a melting end of a kiln or the like industrial furnace, wherein the industrial furnace has an inner furnace space comprising a furnace crown, a furnace superstructure and a furnace material basin, wherein in the method: an image process of at least a part of the furnace space is provided. namely provided with a series of images in the course of time, wherein an image of the series is provided by means of a camera sensor of a camera. the camera being installed at the furnace with a camera view to the furnace space, and the image of the furnace space is related to a technical map of at least one process parameter of the furnace space during operation of the furnace by means of an image point read out, andwherein a process parameter (P) is used in the monitor and/or control of operation, andan image point (i, j) of the image, in particular pixel image A(i, j), corresponds to an object-image position assigned to an object location (x, y, z) of an object in the furnace space B(x, y, z), wherein the image point is related with a sensor point of the camera sensor.

Claims

1. A method of monitor and/or control of operation of an industrial furnace for processing a heated material, in particular for processing a melt in a melting end of a kiln or the like industrial furnace, wherein the industrial furnace has an inner furnace space comprising a furnace crown, a furnace superstructure and a furnace material basin, wherein in the method: an image process of at least a part of the furnace space is provided, namely provided with a series of images in the course of time, wherein an image of the series is provided by means of a camera sensor of a camera, the camera being installed at the furnace with a camera view to the furnace space, and the image of the furnace space is related to a technical map of at least one process parameter of the furnace space during operation of the furnace by means of an image point read out, and wherein a process parameter is used in the monitor and/or control of operation, and an image point of the image, in particular pixel image, corresponds to an object-image position assigned to an object location of an object in the furnace space, wherein the image point is related with a sensor point of the camera sensor, wherein: a reference image of the furnace space is provided at an initial time during the course of time, an actual image is provided at a further time during the course of time, a characteristic object-image position is identified in the actual image and a corresponding characteristic object-image position is identified in the reference image, a deviation is identified for the characteristic object-image position in the actual image as compared to the characteristic object-image position in the reference image, a deviation-compensation is provided to the object-image position in the actual image, in particular to apply for a varying camera view, wherein the deviation-compensation is based on the deviation identified, and a process parameter is determined by means of the deviation-compensated object image position in the actual image.

2. The method as claimed in claim 1, wherein a deviation-compensation is provided to the actual image wherein the deviation-compensation is based on the deviation identified, and the image of the furnace space which is used for relating into the technical map of process parameters, wherein the process parameters are determined by means of a deviation-compensated actual image.

3. The method as claimed in claim 1, wherein the camera view is subject to a camera view variation in the course of time, in particular due to ambient conditions of the furnace, such that the camera view variation causes a deviation of the image point from the object-image position.

4. The method as claimed in claim 1, wherein the camera having a camera position and/or being directed to the furnace space with a camera orientation, wherein a camera pose is assigned to the position and/or orientation of the camera, and/or with varying camera pose in the course of time the image point deviates from the corresponding object-image position.

5. The method as claimed in claim 1, characterized in that the image is provided with a number of image points, in particular as a pixel image, wherein an image point is assigned to a sensor point, in particular sensor pixel, of the camera sensor.

6. The method as claimed in claim 1, wherein the reference image of the furnace space is provided at the initial time, and a characteristic object in the furnace space is selected at the initial time, wherein the characteristic object-image position of the object corresponds to an original image point in the reference image, and the actual image is provided at the further time, wherein the original image point, being related with the sensor point of the camera sensor in the actual image, corresponds to another object-image position of another object, and/or the characteristic object-image position of the object is identified to correspond to another image point being related with the sensor point of the camera sensor in the actual image.

7. The method as claimed in claim 6, wherein: the deviation-compensation is provided to the object-image position in the actual image and applies for the varying camera pose, in that the deviation-compensation is determined by means of identifying the deviation between the original image point in the actual image and the another image point in the actual image.

8. The method as claimed in claim 1, wherein a characteristic object-image position corresponds to an original image point in the reference image in that an image point of the image, in particular pixel image, corresponds to the characteristic object-image position and/or an object-image position is assigned to a characteristic object, in particular fixed object, wherein the characteristic object corresponds to one or more selected points of a group or cluster of points of interest, in particular in the reference image, and/or at least one point of interest in the reference image is selected, such that the point of interest is assigned to the characteristic object and the object-image position of the point of interest is determined as an image point of the image, in particular pixel image.

9. The method as claimed in claim 8, wherein: a characteristic object-image position is identified to correspond to another image point in the actual image as compared to the reference image, wherein at least some of the points of interest are identified in the actual image.

10. The method as claimed in claim 6, wherein: the deviation between the original image point and the another image point in the actual image corresponds to a deviation-relation between at least some of selected points of interest, and the deviation is used to determine the deviation-compensation to the object-image position in the actual image.

11. The method as claimed in claim 1, wherein: the camera view variation in the course of time is related to a drift of camera pose, and/or the camera view variation results in that the image point assigned to a sensor pixel of the camera sensor deviates from a corresponding object-image position and results into a difference between an image point read out at the initial time and an image point read out at the further time, and/or the deviation-compensation to the object-image position in the actual image provides deviation-compensated object-image position in the actual image, in particular wherein the sensor pixel of the camera sensor is again related to the corresponding object-image position such that the difference is compensated.

12. The method as claimed in claim 10, wherein: the object-image position is assigned to an object location by an assignment which is evaluated by means of a transformation function, and/or a deviation-compensating transformation function is applied to the object-image position in the actual image.

13. The method as claimed in claim 12, wherein: the deviation-relation between said at least some of the selected points of interest, namely as identified in the reference image and in the actual image, is used to determine the deviation-compensating transformation function to the object-image position in the actual image.

14. The method as claimed in claim 11, wherein: a deviation-compensation to object-image position in the actual image is applied, such that the image point of the image of the deviation-compensated transformed object-image position in the actual image is assigned to the original location of the object, in particular wherein the assignment is evaluated by means of an inverse transformation function.

15. The method as claimed in claim 1, wherein: points of interest are identified in the reference image by means of an image analysis, in particular a graphical image analysis, wherein: processed material, in particular melt, and/or an interface between melt and furnace structure is identified enabling discriminating between the furnace crown and a superstructure and a material basin, and/or processed material is excluded from selecting the points of interest.

16. The method as claimed in claim 1, wherein: means of image analysis for discriminating between furnace crown and furnace material basin and selection of points of interest are selected from the group of means comprising: image analysis means for identifying high values or extrema of pixel amplitude and/or pixel-to-pixel gradient image analysis means for identifying confined or line or sharp image structures, in particular edge or point like structures image analysis means for identifying fixed objects as stand-still or essentially static, neural network image analysis means for learning of discriminating between furnace crown and superstructure, furnace material basin and other objects.

17. A control system adapted to execute the method as claimed in claim 1, wherein a camera is adapted in that an image process of at least a part of the furnace space is provided, namely provided with a series of images in the course of time, wherein an image of the series is provided by means of a camera sensor of a camera, the camera being installed at the furnace with a camera view to the furnace space, and a read out module is adapted in that the image of the furnace space is related to a technical map of at least one process parameter of the furnace space during operation of the furnace by means of an image point read out, and wherein a process parameter is used in the monitor and/or control of operation, and an image point of the image, in particular pixel image, corresponds to an object-image position assigned to an object location of an object in the furnace space, wherein the image point is related with a sensor point of the camera sensor characterized in that an image taking module is adapted in that a reference image of the furnace space is provided at an initial time during the course of time, an actual image is provided at a further time during the course of time, and a deviation-compensation module is adapted in that a characteristic object-image position is identified in the actual image and a corresponding characteristic object-image position is identified in the reference image, a deviation is identified for the characteristic object-image position in the actual image as compared to the characteristic object-image position in the reference image, and a deviation-compensation module is adapted in that a deviation-compensation is provided to the object-image position in the actual image, in particular to apply for a varying camera view, wherein the deviation-compensation is based on the deviation identified, and a control unit is adapted in that a process parameter is determined by means of the deviation-compensated object image position in the actual image.

18. The control system of claim 17, wherein enhanced process parameters are based on corrected camera imaging results and the enhanced process parameters are used to monitor the industrial furnace operation and/or to control the industrial furnace operation in a feed-forward loop or a feed-back control loop of control.

19. The control system as claimed in claim 17, wherein images obstructed by the deposits and/or blurred images and/or barrel and pincushion distortion or the like optical system distortions are compensated.

Description

[0138] Further advantages, features and details of the invention result from the following description of the preferred embodiments as well as from the drawings, which show in:

[0139] FIG. 1A a furnace image captured by the furnace camera;

[0140] FIG. 1B a thermal image showing group of selected points around the burner port;

[0141] FIG. 2 a process control using conventional sensors and PID control strategy;

[0142] FIG. 3A a schematic geometrical representation of the furnace and camera position;

[0143] FIG. 3B a variation of a given observed point position with camera inclination angle;

[0144] FIG. 3C a relation between the lengths of the displayed furnace floor by one pixel of the camera sensor as a function of the distance of this point from the camera; a vertical pixel resolution is 1080 pixels per viewed area;

[0145] FIG. 4A a schema of the 3D-furnace space (on the left) and the corresponding 2D-image (on the right) taken by a camera at initial pose;

[0146] FIG. 4B a schema of the 3D-furnace space (on the left) and the corresponding 2D-image (on the right) taken by the camera after a change of its pose;

[0147] FIG. 4C a schema of the 3D-furnace space (on the left) and the corresponding 2D-image (on the right) showing correction performed by the deviation-compensation using the inverse transformation function T.sup.?1;

[0148] FIG. 5A a reference image with example of identified points of interest and a glass melt surface;

[0149] FIG. 5B a comparison of reference image (on the right) with the actual image (on the left) as a basis for transformation function T determination;

[0150] FIG. 6 the edges of visible objects inside the furnace identified by image analysis; therein two consequent images (reference image on the right and the actual image on the left) serving as a basis for transformation function T determination;

[0151] FIG. 7 fixed objects (coloured in redfurnace superstructure, coloured in bluemelt level, coloured in greyobservation hole edges) inside the furnace identified by means of neural network as a basis for determination of the transformation function T;

[0152] FIG. 8 an original image (left) obstructed by deposits (see dark corner areas at the bottom of the image) and a neural network analysis of the image (right side); it reveals to show identified deposits (grey color), and clearly distinguishes these from the batch (ochre), flame (purple), bubbler (pink), glass melt (violet), and a superstructure (green) of the furnace;

[0153] FIG. 9 a reference not blurred image (left) and the actual blurred image (right);

[0154] FIG. 10A a new control approach using enhanced control parameters based on corrected camera imaging results;

[0155] FIG. 10B a control scheme using model based predictive control and enhanced control parameters based on corrected camera imaging results;

[0156] FIG. 11 a scheme of furnace and an embodiment of control for the furnace to implement the concept of the invention;

[0157] FIG. 12 a flow chart of a method according to a preferred embodiment.

[0158] Latest solutions and trends in the industrial practice suggest to use visual and thermal imaging cameras or the like imaging means for measurement of the industrial kilns and glass furnaces.

[0159] FIG. 1A shows a visible furnace image 1A of a furnace 1 captured by the furnace camera. The visible furnace image 1A of FIG. 1A in the visible range immediately reveals the structural properties of the furnace 1. The structural properties of the furnace 1 are adapted for processing a melt M in a melting end of a kiln or the like industrial furnace. Among others, the industrial furnace 1 is established with the inner furnace space 10 comprising a furnace crown 11, a furnace superstructure 12 and a furnace material basin 13, with the melt M and batch material B and flame F for heating as visible in furnace image 1A.

[0160] Corresponding features are also shown in the thermal furnace image 1B of FIG. 1B. FIG. 1B depicts an exemplary outcome of an exemplifying present camera system with a camera CAM, which provide 2D-images of the furnace interior, i.e. inner furnace space 10, wherein the 2D-image is a thermal image. One or more camera systems, this is in particular a number of cameras CAM, can be used for a method of monitor and/or control as will be described below in detail.

[0161] The furnace superstructure 12 among others comprises a batch opening 14 or other kind, forms and devices for material supply and a heating device or the like for heating the material. In particular, in this and other embodiments the heating device is formed as burner arrangement 16 with a group of burners and burner ports, oxidiser inlet and combustion products outlet 18. Also, other kinds and forms of heating devices can be provided. Further the furnace structure 12 provides a bubbling system (below the melt level, not shown) which is known in the art to be adapted for creating a controlled disturbance in the glass melt; air or other gases are blown through special bubbler nozzles into the furnace.

[0162] In the thermal furnace image 1B of FIG. 1B one can select a picture point A.sub.0 as an example. As will be described with FIG. 4A to FIG. 4C in detail a picture point A.sub.0 with regard to A.sub.0=A(i.sub.0, j.sub.0) is an example of a number or multitude of points. One can automatically select any group of points A_G (as will be described with FIG. 5A, FIG. 5B in detail).

[0163] A read out of this picture point A.sub.0 (or multitude of points A_G as shown in FIG. 5A, FIG. 5B) or any other picture point of the furnace image 1A is subject of the fact that related with the picture point A.sub.0, A_G is a camera pixel. In other words an image point (i, j) of the image (more precisely the coordinates (i,j) of the point), in particular the pixel image A(i, j), corresponds to an object-image position assigned to an object location (x, y, z) of an object in the furnace space B(x, y, z), wherein the image point is related with a sensor point of the camera sensor CAM_S.

[0164] Thus, the sensor point of the camera sensor CAM_S can be used to determine a physical value (like temperature) for the inner furnace space 10. The physical value (like temperature) can be stored with its value and coordinates (i,j) of the point in the database with time information t.sub.1, when the image was taken in a series SER in a course of time of operation of the furnace 1.

[0165] Obviously the picture of a visible furnace image 1A of FIG. 1A, i.e. in the visible range, is only of limited use in this regardthe thermal furnace image 1B of FIG. 1B, i.e. in a thermal range, i.e. infrared or far-infrared range (IR or FIR range), as shown in FIG. 1B is a better instrument in this regard.

[0166] An example of the selection of the group of points taken by thermal imaging camera is shown only as an example in FIG. 1B, wherein the thermal image as a furnace thermal image 1B show a group of selected points around the burner ports of the burner arrangement 16.

[0167] An image of a series of a furnace thermal image 1B is provided by means of a camera sensor CAM_S of a camera CAM, the camera CAM being installed at the furnace 1 with a camera view to the furnace space 10. Above mentioned example shows seven selected points of temperature measurements Ti (T1, T2 . . . T7), where temperature T.sub.i can be identified from the furnace thermal image 1B in the course of time t.sub.i (t.sub.1, t.sub.2, . . . t.sub.n) and data can be organized in the matrix form as e.g. shown in:

TABLE-US-00001 TABLE I Data structure for group of points defined in FIG. 1B. Time Point t.sub.1 t.sub.2 . . . t.sub.n T1 T.sub.1(t.sub.1) ? C. T.sub.1(t.sub.2) ? C. . . . T.sub.1(t.sub.n) ? C. T2 T.sub.2(t.sub.1) ? C. T.sub.2(t.sub.2) ? C. . . . T.sub.2(t.sub.n) ? C. T3 T.sub.3(t.sub.1) ? C. T.sub.3(t.sub.2) ? C. . . . T.sub.3(t.sub.n) ? C. T4 T.sub.4(t.sub.1) ? C. T.sub.4(t.sub.2) ? C. . . . T.sub.4(t.sub.n) ? C. T5 T.sub.5(t.sub.1) ? C. T.sub.5(t.sub.2) ? C. . . . T.sub.5(t.sub.n) ? C. T6 T.sub.6(t.sub.1) ? C. T.sub.6(t.sub.2) ? C. . . . T.sub.6(t.sub.n) ? C. T7 T.sub.7(t.sub.1) ? C. T.sub.7(t.sub.2) ? C. . . . T.sub.7(t.sub.n) ? C.

[0168] FIG. 2 depicts a general control strategy 20C for operation of an industrial furnace 1 for processing a heated material M using conventional sensors S. For the control unit CU, yr defines the desired process response, yp the real process response and u the process input which also is considered as a control signal. This general control strategy using conventional sensors S however exhibits significant shortcomings. This is because these sensors S measure the control parameters D values as depicted therein in only a limited number of locations (e.g., several thermocouples for temperatures T.sub.i as indicated above). The critically important process data D such as internal thermal situation, batch shape and melting rate, flame shape and chemistry, or process parameters P, which need to be controlled are represented only partially by the measurement. The control unit CU in this case comprises a standard PID controller. Consequently, the conventional techniques provide only a limited control solution. It is currently possible to record a Temperature/Time function (T over t) for temperature values T measured in the course of time t as shown in FIG. 1B as a data set and attempt to use it for the control purposes. However, there are critical shortcomings of this method preventing its use for the consistent and reliable control as follows.

[0169] A first important problem is resided in a mismatch of measurement points due to camera movement; this aspect is the part of the major object addressed by the inventive concept. The camera CAM is installed at the furnace with a camera view to the furnace space 10. More particular the camera CAM having a camera position and/or being directed to the furnace space with a camera orientation, wherein a camera pose CAM_P is assigned to the position and/or orientation of the camera CAM.

[0170] The objective of the control (such as temperature control) is to maintain certain real part of the furnace at predefined set point value. This could be for example group of seven points of a temperature measurement by thermal imaging as shown in FIG. 1B, which represents among others a situation around the furnace port of the burner arrangement 16. The corresponding data structure is shown as an example in Table I.

[0171] These points, selected in the image, correspond to actual locations within the real 3D space of the furnace interior, i.e. the inner furnace space 10.

[0172] The camera projects these 3D locations (x, y, z) as points in 2D-image with no information on its perspective or (in other words) camera view CAM_V; this is due to camera position and/or camera orientation also referred to as camera pose CAM_P.

[0173] It can be assumed that for a fixed camera position and no changes in optical settings, these points of temperature measurements Ti (1=1 . . . 7, . . . N) in the 2D-image represent always the same locations of objects in the furnace, i.e. the inner furnace space 10. In this case, such selected points of temperature measurements Ti can be used for the control purposes, because the data D recorded are consistent with respect to their location of objects whose temperature is to be determined.

[0174] Whereas the camera CAM having a camera position and/or being directed to the furnace space with a camera orientation, wherein a camera pose CAM_P is assigned to the position and/or orientation of the camera, it has been, however, discovered with the approach of the inventive concept that in case of a real furnace and its operations the camera pose CAM_P is not stable due to the various ambient conditions like e.g. operational disturbances such as camera holder mechanical instability, camera maintenance and cleaning, reproducibility of the camera positioning, camera and lens replacement, etc. This is the process faces inevitably a varying camera pose in the course of time t.

[0175] In other words, the camera view CAM_V is subject to a camera view variation ?CAM_V in the course of time t, in particular due to ambient conditions of the furnace, such that the camera view variation ?CAM_V causes a deviation of an image point from an object-image position. In particular therein with varying camera pose CAM_P in the course of time t the image point deviates from an initially corresponding object-image positionthe result will be described in detail with FIG. 4A, FIG. 4B and FIG. 4C.

[0176] The true problem is, that these disturbances cause slight changes in the camera position and/or orientation CAM_P, so that the alignment between the 3D locations in the furnace, respectively inner furnace space 10, and corresponding points in the 2D-image is mismatched. Therefore, the points in the 2D-image, which are fixed by the initial selection are corresponding to other locations in the real furnace interior each time when the camera position and/or orientation -and thus the camera view CAM_Vis changed. In essence, every such camera position and orientation change means a change in the location inside the furnace interior, which is measured for control purposes. This is with varying camera pose CAM_P in the course of time t an image point deviates from the corresponding object-image position.

[0177] This results in a mismatch in measured data consistency and therefore prevents an effective furnace controlthe inventive concept proposes to improve the control strategy 20C to an improved control strategy 20A, 20B as shown in examples of embodiments in FIG. 10A and FIG. 10B.

[0178] It can be shown, that even a slight change in the camera position and/or orientation can cause due to the camera view variation ?CAM_V a significant mismatch between the original alignment between the 3D locations in the furnace and corresponding points in the 2D-image and the actual alignment.

[0179] As has been recognized with the inventive concept the root of this problem is found to be severely impacted with the furnace size and geometry together with observation from one single point of camera view CAM_V. Thus, the concept of the invention starts from the condition that an image point (i, j) of a pixel image A(i, j) corresponds to an object-image position assigned to an object location (x, y, z) of an object B(x, y, z) in the furnace space (x,y,z).

[0180] The problem solved is explained in more detail in FIG. 3A to FIG. 3C. The typical furnace chamber dimensions, i.e. dimensions of a furnace crown 11 and dimensions of a furnace material basin 13 in the inner furnace space 10 are: Length 5 m-50 m, width 2 m-12 m and height 1 m-6 m.

[0181] The camera(s) is (are) typically located in the opening(s) through one (or more) of the furnace walls. The schematic geometrical representation is shown in FIG. 3A.

[0182] Therein a schematic geometrical representation of the furnace and camera position is given. FIG. 3A shows the situation when the camera CAM with a camera sensor CAM_S is located at the furnace 1, this is with a camera view CAM_V to the inner furnace space, wherein mostly the camera CAM is inside the furnace with length L, wherein the camera is located at the wall at the height h above the furnace floor (or melt surface) aiming down at the angle alpha with viewing angle 2?beta; i.e. the camera CAM having a certain camera pose CAM_P to gain the camera view CAM_V. In this case, at given angles alpha and beta 2D-image will display area of the furnace floor (or melt surface) between points x1 and x2 around central point c which is in the middle of the camera field of view.

[0183] An example of FIG. 3B illustrates how significantly the position of given observed point x1this can be an object location (x, y, z) of an object in the furnace space 10changes with a camera view variation ?CAM_V; in this case exemplified by a small change of camera inclination angle alpha. Therein a variation of given observed point x1 position with camera inclination angle is shown. In this example, the dimensions are: L=35 m, h=1.9 m, beta=30 deg and alpha varying from 34.16 deg to 33.14 deg. It can be seen, that a very slight change in the camera inclination angle leads to a very significant shift of the observed point, i.e., one degree in the angle alpha results in shift of 8.5 m in the real furnace. Similar adverse effects as with inclination angle are obtained if the position of camera is shifted alongside the x, y, z axis or by any other angular orientation change within the spherical coordinates.

[0184] It is obvious that such sensitivity to camera position and orientation CAM_P without additional treatment prevents effective furnace control in scheme of control 20C.

[0185] A second problem is resided in the errors of the camera optical system; this aspect is part of the object addressed by a development of the inventive concept. Optical distortions such as vignetting, barrel distortion etc. are related to camera optical system. Vignetting is influencing the radiation intensity impacting on the camera sensor and thus creating artifacts of the thermal imaging. Barrel and pincushion distortions contribute further to the mismatch of measuring points, as explained before.

[0186] A third problem is resided in the image obstruction by the furnace atmosphere deposits; this aspect is part of the object addressed by a development of the inventive concept. Further, a complication for the furnace imaging is caused by the deposits blocking camera view in the form of stalagmites in front of the camera or directly depositing on camera lens creating fogging. These deposits are created by condensation of the volatile species from the furnace atmosphere and/or by settling of solid particles. Specifically, undetected fogging makes thermal imaging inadequate while deposits blocking camera view make measurement impossible. These effects can happen only at part of the image and the current state of art technology cannot identify them and compensate their influence.

[0187] A fourth problem is resided in the various area size of the furnace displayed on one pixel; this aspect is part of the object addressed by a development of the inventive concept. The further complexity arises from the fact that each pixel within an image A(i, j) is corresponding not to a discrete point inside the furnace, but is representing an area which size depends on the distance from the camera and on the angle of view.

[0188] FIG. 3C provides a relation between the length of the displayed furnace floor by one pixel of the camera sensor as a function of the distance of this point from the camera. Vertical pixel resolution is 1080 pixels per viewed area. FIG. 3C illustrates the case for dimensions of L=35 m, h=1.9 m, alpha=33.14 deg, beta=30 deg. It can be seen that, e.g., the length of floor (or surface of melt M) element displayed on one pixel dx significantly vary with the distance x from the wall where the camera is installed.

[0189] Generally, a method of monitor and/or control of operation of an industrial furnace for processing a heated material is applied to an industrial furnace. As described herewith the industrial furnace is adapted for processing a melt in a melting end of a kiln or the like industrial furnace 1. The industrial furnace has an inner furnace space 10 comprising a furnace crown 11, a furnace superstructure 12 and a furnace material basin 13. The method provides the steps of: [0190] an image process of at least a part of the furnace space is provided, namely provided with a series of images 1B in the course of time, [0191] wherein an image of the series is provided by means of a camera sensor CAM_S of a camera CAM, the camera CAM being installed at the furnace with a camera view CAM_V to the furnace space 10. In particular it can be assumed that with the camera having a camera position and/or being directed to the furnace space with a camera orientation, therein a camera pose CAM_P is assigned to the position and/or orientation of the camera.

[0192] Therein the image 1B is provided as a pixel image A(i, j) with a number of image points (i, j), wherein an image point is assigned to a sensor pixel of the camera sensor CAM_S. The image of the furnace space 10 is related to a technical map of at least one process parameter P of the furnace space during operation of the furnace by means of an image point read out, i.e. at a sensor pixel of the camera sensor CAM_S. Therein a process parameter P is used in the monitor and/or control of operation of the furnace.

[0193] In the following at first an exemplifying description is given as how to provide an improved method and control schemeas explained in detail with FIG. 10A, FIG. 10B, FIG. 11 and FIG. 12, namely in that a varying camera pose CAM_P in the course of time t.sub.i (i=1 . . . n , wherein t.sub.i indicate a series of points of time), can be applied for by image analysis as such. Thus, the concept of invention is unlike sophisticated modelling of the changes to the setting of camera and furnace.

[0194] The approach of inventive concept is to provide a deviation-compensation to the object-image position in the actual image 1B to apply for a camera view variation ?CAM_V. More particularly due to the varying camera pose CAM_P this is considered to be particular useful. In order to overcome the shortcomings of the present state of art, this invention concept proposes a new method how to use camera imaging in furnace control as explained with FIG. 10A, FIG. 10B, FIG. 11 and FIG. 12.

[0195] As explained above, the concept of the invention starts from the condition that an image point (i, j) of a pixel image A(i, j) corresponds to an object-image position assigned to an object location (x, y, z) of an object B(x, y, z) in the furnace space (x,y,z).

[0196] In a preferred embodiment the first important problem is resided in a mismatch of measurement points due to a camera movement. As addressed as follows to overcome the errors resulting from the camera movement is part of the major object and thus is addressed by the inventive concept. A good exemplifying non-restrictive solution within the concept of the invention is to apply for the mismatch of measurement points due to camera movement given as follows.

Introduction of the Transformation function

[0197] With view to FIG. 4A it is to be understood, that in case that any point of interest within the furnace interior (furnace space coordinates (x, y, z) can be chosen arbitrarily as intended for control purposes using the camera imaging, its projection in the 2D-imagenamely the mage coordinates (i, j)are selected. The present invention introduces a transformation function T such that to each point (i, j) within the 2D-imagecaptured by the cameraassigns a point inside the furnace interior (x, y, z) within the camera field of view:

A(i, j).fwdarw.T.B(x, y, z); (1)

Where:

[0198] A(i, j) is a 2D-matrix of points within the 2D-image;
B(x, y, z) is a 3D-matrix of points inside 3D-furnace space displayed within the camera field of view;
T is the transformation function.

[0199] Thus, it can be said, that an image point (i, j) of the pixel image A(i, j) corresponds to an object-image position assigned to an object location (x, y, z) of an object B in the furnace space B(x, y, z).

[0200] The transformation function can be described by the following expressions:

[00001]$\begin{array}{cc}T\left(x,y,z\right)=T\left(f\left(i,j\right),g\left(i,j\right),h\left(i,j\right)\right);& \left(2\right)\end{array}$$\begin{array}{cc}x=f\left(i,j\right);y=g\left(i,j\right);z=h\left(i,j\right);& \left(2-1,2-2,2-3\right)\end{array}$

Where:

[0201] f(i, j), g(i, j), h(i, j) are the functions expressing the alignment between the coordinates [i, j] of points within the 2D-image A(i, j) and furnace space coordinates [x, y, z].

[0202] In case the camera position is not changed the alignment is matching, i.e., the transformation function T remains unchanged, and any point within the image can be used for control; the original position A.sub.0=A(i.sub.0, j.sub.0) of the selected control point within the image will always correspond to location B.sub.0 in this hypothetical case.

[0203] With view to FIG. 4B it is to be understood, that with varying camera pose in the course of time however, the image point (i, j) deviates from the corresponding object-image position. Thus, on the other hand, in case that camera position or orientation changes, the transformation function is changed to Tn and the original position A.sub.0=A(i.sub.0, j.sub.0) of the selected control point within the image will correspond to other location B1 inside the furnace as shown by development from the image shown in FIG. 4A to FIG. 4B.

[0204] Therefore, after the camera position or orientation change ?CAM_V, the following relation is valid:

A(i, j).fwdarw.Tn.B(x, y, z); (3)

Where:

[0205] A(i, j) is a 2D-matrix of points within the 2D-image;
B(x, y, z) is a 3D-matrix of points inside 3D-furnace space displayed within the camera field of view;
Tn is the new transformation function after change of the camera position or orientation.

[0206] Severity of the problem is obvious from the example when a camera inclination angle alpha (see FIG. 3A) changes by 1 degree, the representation of the position of point x1 is shifted by 8.5 m further from its original location in the furnace (see Fehler! Verweisquelle konnte nicht gefunden werden.). This effectively means that the measurement is conducted 8.5 m far away from the originally selected point within the furnace.

[0207] The invention solves the above-mentioned problem using transformation function as described in the following below.

Selection of the Control Point in the Furnace and its Representation the 2D-Image

[0208] Originally selected control point B.sub.0 in the furnace B(x.sub.0, y.sub.0, z.sub.0) has coordinates (x.sub.0, y.sub.0, z.sub.0)see FIG. 4A which depicts a schema of the 3D-furnace space FS3 (on the left) and the corresponding projection furnace space FS2 thereof into the 2D-image of the 3D-furnace space, i.e. projection thereof (on the right) taken by the camera at initial pose. In the initial state, a point in the furnace B(x.sub.0, y.sub.0, z.sub.0) (hereinafter B.sub.0) corresponds to the image point A(i.sub.0, j.sub.0) (hereinafter A.sub.0) and the following relation applies:

A(i.sub.0, j.sub.0).fwdarw.T0.B(x.sub.0, y.sub.0, z.sub.0); (4)

Where:

[0209] A(i.sub.0, j.sub.0)=A.sub.0 is the control point with coordinates [i.sub.0, j.sub.0] within the 2D-image corresponding to the point B.sub.0=B(x.sub.0, y.sub.0, z.sub.0);
B(x.sub.0, y.sub.0, z.sub.0)=B.sub.0 is the initially selected control point with coordinates [x.sub.0, y.sub.0, z.sub.0] inside 3D-furnace space within the camera field of view;
T0 is the initial transformation function.

[0210] In this regard, so to say, a characteristic object-image position A.sub.0 for an object B.sub.0 is identified in the reference image at time t.sub.0 for an object B.sub.0 at an object location (x.sub.0, y.sub.0, z.sub.0) of an object in the furnace space (B(x, y, z)) as shown in right part of FIG. 4A. Such situation can be provided in a reference image IMG_ref according to the concept to the invention and as described below in an example.

[0211] So to say in the image provided in FIG. 4A on the right hand side, a number of image points (i, j), in particular in a pixel image therein, an image point (i, j) is assigned to a sensor point, in particular sensor pixel (i, j), of the camera sensor CAM_S.

Drift of the Measured Point Representation Caused by the Camera Movement

[0212] After the change of camera position or orientation the initially selected control point A.sub.0=A(i.sub.0, j.sub.0) corresponds to another point inside 3D-furnace space B(x.sub.1, y.sub.1, z.sub.1)=B.sub.1; this is A.sub.0=A(i.sub.0, j.sub.0) now corresponds to B.sub.1 as the another point inside 3D-furnace spacethe same camera pixel formerly aiming to B.sub.0 now with changing camera view aims to B.sub.1see FIG. 4B which depicts a schema of the 3D-furnace space (on the left) and the corresponding 2D-image (on the right) taken by the camera after a change of its orientationand the following relation applies:

A(i.sub.0, j.sub.0).fwdarw.Tn.B(x.sub.1, y.sub.1, z.sub.1); (5)

Where:

[0213] A.sub.0=A(i.sub.0, j.sub.0) is the originally selected control point with coordinates [i.sub.0, j.sub.0] within the 2D-image;
B.sub.1=B(x.sub.1, y.sub.1, z.sub.1) is the new point in the furnace with coordinates [x.sub.1, y.sub.1, z.sub.1] corresponding to the image point A.sub.0=A(i.sub.0, j.sub.0);
Tn is the new transformation function (where index n=1, 2, . . . , k identifies the new transformation function after each change of the camera position and/or orientation).

[0214] Such situation e.g. can be provided in an actual image IMG_act according to the concept to the invention and as described below. In this regard, so to say (i.sub.0, j.sub.0) is related to the originally selected control point.

Compensation of the Drift

[0215] A characteristic object-image position A.sub.0 referring to object location B.sub.1 is identified in the actual image IMG_act and a corresponding characteristic object-image position A.sub.0 referring to object location B.sub.0 is identified in the reference image IMG_ref. In order to continue to measure the initially selected control point B(x.sub.0, y.sub.0, z.sub.0)=B.sub.0, the new coordinates within the 2D-image [i.sub.1, j.sub.1] must be determinedsee FIG. 4C which depicts a schema of the 3D-furnace space (on the left) and the corresponding 2D-image (on the right) showing correction performed by using the inverse transformation function T1-inversefor which the following relation applies:

B(x.sub.0, y.sub.0, z.sub.0).fwdarw.T1-inverse.A(i.sub.1, j.sub.1); (6)

Where:

[0216] B.sub.0=B(x.sub.0, y.sub.0, z.sub.0) is the initially selected control point with coordinates [x.sub.0, y.sub.0, z.sub.0] inside 3D-furnace space within the camera field of view;
A.sub.1=A(i.sub.1, j.sub.1) (now related with B.sub.0) is the corresponding new control point with coordinates (i.sub.1, j.sub.1) within the 2D-image as shown in FIG. 4B and FIG. 4C;
T1-inverse is an inverse function to the new transformation function T1.

[0217] Generally, the inverse transformation function T-inverse can be described by the following expressions:

[00002]$\begin{array}{cc}T-\mathrm{inverse}\left(i,j\right)=T\left(u\left(x,y,z\right);v\left(x,y,z\right)\right);& \left(7\right)\end{array}$$\begin{array}{cc}i=u\left(x,y,z\right);j=v\left(x,y,z\right);& \left(7-1,7-2\right)\end{array}$

Where:

[0218] u(x, y, z) and v(x, y, z) are the functions expressing the alignment between the furnace space coordinates [x, y, z] and the coordinates [i, j] of points within the 2D-image A(i, j);
T is the transformation function.

[0219] Such situation can be considered as a deviation ?POS identified for a characteristic object-image position A.sub.1 in the actual image IMG_act as compared to the characteristic object-image position A.sub.0. More precisely the characteristic object-image position A.sub.1 in the actual image IMG_act is now the object-image position A.sub.0 referring to object location B.sub.1 as the aiming of the respective camera pixel changed with changing camera view CAM_V at time t.sub.1. in the actual image IMG_act as compared to the characteristic object-image position A.sub.0. More precisely the object-image position A.sub.0 refers to object location B.sub.0 as the original aiming of the respective camera pixel with original camera view CAM_V at time t.sub.0 in the reference image IMG_ref; this is according to the concept to the invention and as described below.

Periodic Alignment Correction Procedure

[0220] The above mentioned procedure according to sections Drift of the measured point representation caused by the camera movement and Compensation of the drift (B.sub.1=B(x.sub.1, y.sub.1, z.sub.1) related to A.sub.0=A(i.sub.0, j.sub.0) at time t.sub.1actual image IMG_act) can be applied whenever a deviation from the original alignment between B.sub.0=B(x.sub.0, y.sub.0, z.sub.0) and A.sub.0=A(i.sub.0, j.sub.0) at time to (reference image IMG_ref) as defined by Eq. 4 is detected.

[0221] Thus, a deviation-compensation can provided to the object-image position in the actual image (A(i.sub.n, j.sub.n)) (FIG. 4C), to apply for the varying camera pose, wherein the deviation-compensation is based on a deviation identified for a characteristic object-image position in the actual image as compared to the reference image.

Determination of the Transformation Function

[0222] The alignment between the 2D-image and the 3D-furnace space is described by means of the transformation function as described in Eq. 1. The present invention uses image analysis for finding the initial alignment between the points within the furnace space B(x, y, z) and the points within the 2D-image A(i, j). The image analysis employs for example but not limited to methods of edge detection, image feature extraction and matching, segmentation by use of Neural Networks, genetic algorithms.

[0223] The following procedure discloses use of automated image analysis employing techniques of segmentation and Neural Network-based object identification within the image.

[0224] FIG. 5A shows a reference image IMG_ref with a glass level of melt M and examples of identified arbitrary points A of interest indicating mostly the structure of the furnace; i.e. those points with original alignment between B.sub.0=B(x.sub.0, y.sub.0, z.sub.0) and A.sub.0=A(i.sub.0, j.sub.0) at time t.sub.0reference image IMG_ref. More particular a group of points A_G is shown therein.

[0225] A point is denoted as a point A of interest as it has some feature identifiable and distinguishable from its nearest surrounding. This distinguishable feature is identifiable e.g. by one or more of the features selected from the group consisting of: high or extreme gradient of contrast, high or extreme gradient of luminance, extremely low or extremely high intensity, change of colour. The gradient or change or extreme can be present as a pixel-to-pixel gradient or change or extreme.

[0226] This kind of distinguishable feature is understood regularly to denote some interesting structure or line in the image like a border between melt and/or furnace crown and/or furnace structure or an item of the superstructure. Such kind of points of interest are understood mostly in that they can be considered as fixed as compared to moving parts (melt, batch, flame) of the furnace interior when the furnace is in operation. Thus, a point A of interest is best suited to have a characteristic object-image position assigned to an object location of the object.

[0227] The points A of interest can be selected arbitrarily in that they can be selected in various areas of the image and at best somewhat distributed over the image; if not distributed uniformly (which is unlike) but distributed accurately to balance out the various deviation ?POS identification for the characteristic object-image position in the actual image as compared to the characteristic object-image position in the reference image for each point of interest. In that sense the points A of interest can be understood to be selected arbitrarily.

[0228] Said deviation ?POS is understood to be different for each pair of a point A of interest in the actual image and reference image and this depending on where the point A of interest is located in the image. Thus, an average of all deviations ?POS identified will be better, the more accurate the distribution is; thus to balance out the various deviations ?POS and come to a good value for ?POS.

[0229] In other words, when identifying some points A of interest in the reference image, only part of them will be found in the actual image. This is also visible in comparing the right and left part of FIG. 5B.

[0230] FIG. 5B depicts a comparison of the reference image (reference image IMG_ref on the right) with the actual image IMG_act (on the left) as a basis for transformation function Tn determination. Some of the identified points of interest A.sub.0 originally corresponding to characteristic objects B.sub.0 image-locations in the reference image IMG_ref at the time t.sub.0denoted as A.sub.0.fwdarw.[B.sub.0, t.sub.0]are later found at image-locations A.sub.1 in the actual image IMG_act at the time t.sub.1denoted as A.sub.1.fwdarw.[B.sub.0, t.sub.1]. In other words, A.sub.0 and A.sub.1 indicate one and the same object B.sub.0=B(x.sub.0, y.sub.0, z.sub.0) at time t.sub.1 and t.sub.0 respectively. Therefore, transformation function Tn can be derived from the relation between A.sub.0 and A.sub.1.

[0231] To illustrate an alternative or additional approach based on the same approach, FIG. 6 shows the edges A_E of visible objects inside the furnace identified by image analysis. Two consequent images (reference image on the right and the actual image on the left) serve as a basis for transformation function Tn determination.

[0232] To illustrate an alternative or additional approach, FIG. 7 shows fixed objects (redfurnace superstructure, bluemelt level, greyobservation hole edges) inside the furnace identified by means of neural network; neural identified objects A_N-serve as a basis for determination of the transformation function Tn.

Reference Image Creation

[0233] In the first step of the core process according to the concept of the invention a reference image IMG_ref (A.sub.reference(i, j)) of the furnace space B(x, y, z) is provided at an initial (point of) time in the course of time. Thus a reference image IMG_ref is taken (see FIG. 5A, FIG. 6, FIG. 7) by the camera CAM at an initial position and orientation CAM_P and thus camera view CAM_V resulting therefrom as explained above. The matrix of image points A.sub.reference(i, j) correspond to points within the 3D-furnace space B(x, y, z).

A.sub.reference(i, j).fwdarw.T0.B(x, y, z); (8)

Where:

[0234] A.sub.reference(i, j) is the matrix of points of the reference image;
B(x, y, z) is a 3D-matrix of points inside 3D-furnace space displayed within the camera field of view;
T0 is the initial transformation function.

Reference Image Analysis

[0235] In the second step of the core process according to the concept of the invention, an image point (i, j) of the pixel image A(i, j) corresponds to an object-image position assigned to an object location (x, y, z) of an object in the furnace space B(x, y, z). More particularly the reference image (A.sub.reference(i, j)) of the furnace space B(x, y, z) is provided at the further (point of) time t.sub.0 in the course of time, and a characteristic object B.sub.0 in the furnace space B(x.sub.0, y.sub.0, z.sub.0) is selected at the initial time t.sub.0, wherein the characteristic object-image position A.sub.0 [B.sub.0, t.sub.0] of this characteristic object B.sub.0 at time t.sub.0 corresponds to an original image point (i.sub.0, j.sub.0) in the reference image A(i.sub.0, j.sub.0) (FIG. 4A). It can be said, that a characteristic object-image position A.sub.0 [B.sub.0, t.sub.0] is identified at original image point (i.sub.0, j.sub.0) in the reference image A(i.sub.0, j.sub.0).

[0236] Thus, points of interest A, A_G, A_E, A_N as indicated for example in FIG. 5A, FIG. 5B and FIG. 6 and FIG. 7 respectively can be related with object-image positions assigned to respective objects and these points of interest A, A_G, A_E, A_N comprising the object-image positions within the 2D reference image are identified by an automated image analysis procedure. Output of this procedure constitutes a set of reference image characteristics and the determined transformation function T0.

[0237] In case that the arbitrary points or objects are fixed within the furnace 3D space, e.g., objects which are parts of the furnace superstructure, complete glass level etc., the relation expressed in Eq. 8 can be approximated by a simplified form using relationship between two images taken at different time:

A.sub.reference(i, j)=Tn.A.sub.n(i, j); (9)

Where:

[0238] A.sub.reference(i, j) is the reference image and;
A.sub.n(i, j) is the actual image;
Tn is the actual transformation function.

[0239] A position of the fixed object in the images is then identified as:

A.sub.reference(iF.sub.reference, jF.sub.reference)=Tn.A.sub.n(iF.sub.n, jF.sub.n); (10)

Where:

[0240] A.sub.reference(iF.sub.reference, jF.sub.reference) is the position of the fixed object image identified by image analysis in the reference image;
A.sub.n(iF.sub.n, jF.sub.n) is the position of the fixed object identified by image analysis in any other image;
Tn is the actual transformation function.

[0241] From the relation between A.sub.reference and A.sub.n, prescription for the inverse transformation function Tn-inverse is derived in the form:

A.sub.n(i.sub.n, j.sub.n)=Tn-inverse. A.sub.reference(iF.sub.reference, jF.sub.reference); (11)

Where:

[0242] A.sub.reference(iF.sub.reference, jF.sub.reference) is the position of the fixed object image identified by image analysis in the reference image;
A.sub.n(i.sub.n, j.sub.n) is the position of the fixed object image in the actual image;
Tn-inverse is the actual inverse transformation function.

[0243] The inverse transformation function Tn-inverse can be described by the following expressions:

[00003]$\begin{array}{cc}\mathrm{Tn}-\mathrm{inverse}\left(i,j\right)?T\left(f\left({\mathrm{iF}}_{\mathrm{reference}},{\mathrm{jF}}_{\mathrm{reference}}\right);g\left({\mathrm{iF}}_{\mathrm{reference},}{\mathrm{jF}}_{\mathrm{reference}}\right)\right)& \left(11\right)\end{array}$$\begin{array}{cc}{i}_n?f\left({\mathrm{iF}}_{\mathrm{reference}},{\mathrm{jF}}_{\mathrm{reference}}\right);j_n?g\left({\mathrm{iF}}_{\mathrm{reference},}{\mathrm{jF}}_{\mathrm{reference}}\right);& \left(11-1;11-2\right)\end{array}$

Where:

[0244] i.sub.n and j.sub.n are the coordinates of the fixed object image in the actual image corresponding to the original position of the same object image in the reference image before camera pose change;
iF.sub.reference and jF.sub.reference are the coordinates of the fixed object image in the reference image;
f and g are inverse transformation functions of (i, j) coordinates between the actual and the reference image. These functions are applied by means of interpolation/extrapolation.

[0245] In other words an actual image (A(i.sub.n, j.sub.n)) is provided at a further time (FIG. 4B) during the course of time. As explained above a characteristic object-image position is identified in the actual image and a corresponding characteristic object-image position is identified in the reference image.

[0246] The invention recognized that due to the course of time the camera view is subject to a camera view variation in the course of time, in particular due to ambient conditions of the furnace, such that the camera view variation ?CAM_V causes a deviation of the image point from the object-image position, in particular wherein with varying camera pose in the course of time the image point (i, j) deviates from the corresponding object-image position.

Determination of the Object-Image Position Shift Represented by Transformation Function Tn

[0247] In the third step of the core process according to the concept of the invention, an actual image IMG_act (A(i.sub.n, j.sub.n)) is provided at a further time (FIG. 4B), and a deviation-compensation is provided to the object-image position in the actual image (A(i.sub.n, j.sub.n)) (FIG. 4C), to apply for the varying camera pose, wherein the deviation-compensation is based on a deviation identified for a characteristic object-image position in the actual image as compared to the reference image. More particularly the actual image A(i.sub.n, j.sub.n) is provided at the further time t.sub.1 (FIG. 4B), wherein the original image point (i.sub.0, j.sub.0) in the actual image A(i.sub.0, j.sub.0) (FIG. 4B) corresponds to another object-image position A.sub.1 [B.sub.1, t.sub.1]; this is at the further time t.sub.1 the camera's original image point (i.sub.0, j.sub.0) aims to another object B.sub.1 at the further time t.sub.1. Also, the characteristic object-image position A.sub.1 [B.sub.0, t.sub.1] is identified to correspond to another image point (i.sub.1, j.sub.1) in the actual image (A(i.sub.1, j.sub.1)) (FIG. 4C).

[0248] Here the points of interest or objects can be identified as A, A_G, A_E, A_N (set of reference image characteristics) in the reference image are automatically searched and identified in all other consequent imagessee FIG. 5B, FIG. 6, FIG. 7.

[0249] Those fixed points or objects A, A_G, A_E, A_Nwhich can be identified in both images (i.e., in the reference one and in the actual one) with predefined degree of certaintyare used for determining the actual transformation function Tn according to Eq. 10.

[0250] Based thereon thereby a deviation-compensation COMP??POS is provided to the object-image positions A.sub.1 [B.sub.0, t.sub.1] in the actual image (A(i.sub.n, j.sub.n)) (FIG. 4C), to apply for the varying camera pose. Therein the deviation-compensation is based on a deviation identified for a characteristic object-image position A.sub.1 [B.sub.0, t.sub.1] in the actual image IMG_act as compared to the characteristic object-image position A.sub.0 [B.sub.0, t.sub.0] in the reference image IMG_ref.

[0251] More particularly the deviation-compensation is provided to the object-image position in actual image (A(i.sub.n, j.sub.n)) (FIG. 4C) and applies for the varying camera pose, in that the deviation-compensation is determined by means of identifying a deviation (A(i.sub.0, j.sub.0)?A(i.sub.1, j.sub.1)) between the original image point (i.sub.0, j.sub.0) and the another object image point (i.sub.1, j.sub.1) in the actual image; the relation with the deviation ?POS is shown in FIG. 4C.

[0252] In other words, it can be said that within the concept of the invention this example shows a preferred way of a method in that [0253] a characteristic object-image position corresponds to an original image point (i.sub.0, j.sub.0) in the reference image A(i.sub.0, j.sub.0) (FIG. 4A) in that an image point (i, j) of the pixel image (A(i, j)) corresponds to the characteristic object-image position and/or [0254] an object-image position is assigned to a characteristic, in particular fixed, object corresponding to one or more selected points of a group or cluster of points of interest, in particular in the reference image (A.sub.reference(i, j), FIG. 5A), in particular wherein at least one point of interest in the reference image (A.sub.reference(i, j) is selected, (FIG. 5A) such that the point of interest is assigned to the characteristic object and the object-image position of the point of interest is determined as an image point (i, j) of the pixel image (A.sub.reference(i, j)).

[0255] Further [0256] a characteristic object-image position is identified to correspond to another image point (i.sub.1, j.sub.1) in the actual image (A(i.sub.1, j.sub.1)) (FIG. 4C) as compared to the reference image, wherein at least some of the points of interest are identified in the actual image (A(i.sub.n, j.sub.n) (FIG. 5B), and [0257] the deviation (A(i.sub.0, j.sub.0)?A(i.sub.1, j.sub.1)) between the original image point (i.sub.0, j.sub.0) and the another image point (i.sub.1, j.sub.1) in the actual image (FIG. 4C) results from a deviation-relation between at least some of the selected points of interest and is used to determine the deviation-compensation to the object-image position in the actual image (A(i.sub.n, j.sub.n)) (FIG. 5B, left part).

[0258] The alternative way of selection the objects in the reference image makes use of the edge detection of all distinctly visible objects inside the furnace, such as ports, joints, arches, etc.see in FIG. 6. These objects are identified together with their numerical descriptors.

[0259] Any following image is identified in the same way and its descriptors are compared with those of the reference image. In case a significant difference is found, a new transformation function Tn is found.

[0260] Another way of selection the objects in the reference image is based on Neural Network identification of the principal parts of the furnace A_N such as superstructure, glass melt surface, observation holes and similar objects (see FIG. 70). Difference in position of these identified objects in the reference image in comparison to any other consequent image is used for determination of the actual transformation function Tn.

Determination of the Inverse Transformation Function Tn-Inverse for Drift Compensation

[0261] In the fourth step, the deviation of the actual transformation function Tn from the initial transformation function T0 is evaluated. In case the deviation is greater than a predefined threshold, inverse transformation function Tn-inverse is calculated and used for determination of the position of the initially selected control point in the actual image A(i.sub.1, j.sub.1)see Eq. 6.

Other Camera Related Corrections for Changes in Camera View

[0262] Also the following errors of the camera optical system, image obstruction by the furnace atmosphere deposits, camera blocking by the deposits and camera lens fogging are considered as changes processes which result in changes in camera view as explained in detail below. The changes in camera view can apply to one or more cameras in use for the method of the inventive concept.

Solution for Errors of the Camera Optical System

[0263] It is further beneficial to calibrate and compensate the optical system of the camera for optical errors such as vignetting, barrel or pincushion distortion. Techniques of the calibration and compensation are based on measurement of standard calibration imagehere standard calibration image can be compared to effect of barrel and pincushion distortion. The measured deviations caused by the optical errors are quantified and used for image compensation in order to receive an undistorted image. This improved image is preferable for data processing according to procedures described in sections Solution for mismatch of measurement points due to camera movement and Determination of the transformation function

Solution for Image Obstruction by the Furnace Atmosphere Deposits

[0264] The present invention uses an automated method for identification of either deposits blocking camera view or camera lens fogging.

Solution for Camera Blocking by the Deposits

[0265] The method is based on Neural Network object identification where calibration image with no deposit is recorded. Artificial Neural Network identifies visible objects within the image. Any other consequent image is identified using Neural Network and presence of the deposit including its location is identified. Neural Network is specifically trained such a way that it distinguishes other objects such as bubbling, batch, and flame from the specific of the depositsee FIG. 8 Original image (left) obstructed by the depositssee dark corner areas at the bottom of the image. Neural Network analysis of the image (right) shows identified deposits DEP (grey color), and clearly distinguishes these from the batch of material B (ochre), flame F (purple), bubbler BBL (pink), glass melt M (violet), and superstructure 12 (green).

Solution for Camera Lens Fogging

[0266] Fogging identification is based on quantification of the blur index BI where integral of the radiation intensity in the reference image is compared to integral of the radiation intensity in any consequent image. Shown in FIG. 9 is a result of the fogging evaluation together with the reference image IMG_ref and actual image IMG_act: Reference IMG_ref not blurred image (left) and the actual blurred image IMG_act (right). Charts of the blur index BI are shown below the images.

[0267] The evaluated blurring index BI can be used for correction of the thermal imaging using a general formula:

I-corrected=I-actual.(1/BI); (12)

Where:

[0268] I-corrected is the corrected radiation intensity value used for thermal imaging;
I-actual is the measured radiation intensity;
BI is the blurring index, which is a number between zero and one indicating the relative attenuation of the radiation intensity by the semi-transparent deposit layer.
The temperature can then be obtained using a general formula:

T-measured=k(I-corrected); (13)

Where:

[0269] T-measured is the measured temperature;
k(I-corrected) is function converting the radiation intensity received by the camera into the temperature taking into account camera thermal calibration and material emissivity;
I-corrected is the corrected radiation intensity value used for thermal imaging.

Solution for Various Area Size of the Furnace Displayed on One Pixel

[0270] It is beneficial to calculate based on known furnace geometry and position and orientation of the camera within the furnace (which is identified from the image analysis by comparing image, known furnace geometry, using, e.g., genetic programming) the physical area of the furnace corresponding to each pixel of the image. This information is used for determination of the precision of the object spatial identification. This is specifically beneficial for identification of the batch, bubbler, flame, temperature of the object.

New Control Method Using Corrected Camera Imaging

[0271] It has been found, based on the concept of the invention, that the furnace control can be greatly improved by the use of the camera CAM and image analysis as described above. The concept recognizes that the camera view is subject to a camera view variation in the course of time, in particular due to ambient conditions of the furnace, such that the camera view variation causes a deviation of the image point from the object-image position,

in particular wherein with varying camera pose in the course of time the image point (i, j) deviates from the corresponding object-image position.

[0272] Thus, information from such imaging is processed through several enhancing modules performing at least the following functions: [0273] Correction of the mismatch of measurement points due to camera view variation ?CAM_V; this is basically due to camera movement, which results in a variation of the camera pose CAM_P.

[0274] This correction is basically achieved in that [0275] a reference image (A.sub.reference(i, j)) of the furnace space B(x, y, z) is provided at an initial time during the course of time, [0276] an actual image (A(i.sub.n, j.sub.n)) is provided at a further time (FIG. 4B) during the course of time, [0277] a characteristic object-image position is identified in the actual image and a corresponding characteristic object-image position is identified in the reference image, [0278] a deviation is identified for the characteristic object-image position in the actual image as compared to the characteristic object-image position in the reference image. [0279] Determination of the transformation function for correction of the mismatch of measurement points due to camera movement.

[0280] This determination of the transformation function basically allows providing a deviation-compensation to the object-image position in the actual image (A(i.sub.n, j.sub.n)) (FIG. 4C), to apply for the varying camera view, wherein the deviation-compensation is based on a the deviation identified.

[0281] Further enhancing modules performing the following functions have been found to be of particular use with advantages as described above: [0282] Correction of errors of the camera optical system; [0283] Detection and correction for image obstruction by the furnace atmosphere deposits; [0284] Correction for various area size of the furnace displayed on 1 pixel.

[0285] The above procedure converts information from the imaging into the reliable and reproducible data, which can be used for dependable furnace control. One or several major features of this data can be achieved at least thereby as listed below: [0286] Measured points are continuously correctly identified in the image always corresponding to the same location in the furnace; [0287] The precision of the points identification is known based on area size of the furnace displayed on 1 pixel.

[0288] Further, the optional feature of this data can be achieved in that: [0289] Any image obstruction is identified and automated correction is taken.

[0290] Further, the major feature of this data can be achieved in that: [0291] Correct temperature reading can be made using compensation for the image obstruction and correction for the camera optical system errors.

[0292] More particular this is due to that the image of the furnace space, which is used for relating into the imaging technical map of process parameters, is determined by means of the deviation-compensated actual image.

[0293] Further one or more optional features of this data can be achieved in that: [0294] Batch, foam, bubbling, furnace walls, glass melt level, and flame are identified using automated image analysis (Neural Networks, genetic programming etc.) and their precise location in the furnace is measured as a function of time; [0295] Temperature reading can be further enhanced using known emissivity and analysis of angular radiation components.

[0296] This new precise data allow determining at least one or more of new enhanced control parameters as listed below: [0297] Temperature of the furnace superstructure; [0298] Temperature of the glass surface, batch and foam or scum layer; [0299] Temperature of the exhaust ports, exhaust ducts, and regenerators; [0300] Throat area and riser area temperature; [0301] Batch melting rate; [0302] Batch position; [0303] Batch migration patterns and velocity; [0304] Flame chemistryCO and NO.sub.x generated by combustion; [0305] Identification of the flame hot zones; [0306] Refractory corrosion, cracking or dis-integration; [0307] Refractory overheating.

[0308] These new enhanced control parameters are very beneficial for new level of the furnace control with either conventional control approachsee FIG. 10Aor with usage of model based predictive controlsee FIG. 10B.

[0309] Thus, the conventional control as described in FIG. 2 is modified and new process parameters as described above are used. In this regard, FIG. 10A, and FIG. 10B depict a general control strategy using conventional sensors S. However, therein yr* defines the desired process response of improved kind, yp* the improved process response and u* the improved process input which also is considered as a control signal. Thus, the general control strategy using conventional sensors S measuring the control parameters values as depicted therein in only a limited number of locations (e.g., several thermocouples) is improved as follows. The critically important process data D* such as internal thermal situation, batch shape and melting rate, flame shape and chemistry, or parameters are controlled in an improved way according to the concept of the invention; this is they are partially represented by the (conventional) measurement and partly by the (inventive) camera based monitor and/or control of operation of the industrial furnace. As a result the improvement provides that the conventional techniques go beyond a limited control solution. Corrected camera imaging results I* are received and used for enhanced process parameters P*. The enhanced process parameters P* are used in the control scheme as shown in FIG. 10A, and FIG. 10Bthis is, they are fed back to control the furnace 1 as the improved process response yp* in addition to the conventional process response yp; thus this results in a new improved process output yem*.

[0310] The improvement is executed by the inventive control system 100 as is described below with FIG. 11 and FIG. 12. The inventive control system 100 exhibits a camera K, an image taking module 130, a read out module 110 and a deviation-compensation module DELTA which gives data to a control unit 120; an image of the furnace space which is used in a read out module 110 for relating into the imaging technical map of process parameters is determined by means of the deviation-compensated object-image position in the actual image.

[0311] The inventive control system 100 allows for gaining the corrected camera imaging results I* to be received and used for enhanced process parameters P* as shown in FIG. 10A, and FIG. 10B. Implementation thereof in the control loop as shown in FIG. 10A, and FIG. 10B with the enhanced process response and new process output yem* provides an improved, i.e. desired enhanced process response yr*.

[0312] FIG. 11 depicts a scheme of an industrial furnace 1 for processing a heated material M, in particular for processing a melt in a melting end of a kiln or the like industrial furnace 1, wherein the industrial furnace has an inner furnace space 10 comprising a furnace crown 11, a furnace superstructure 12 and a furnace material basin 13 and further comprising the control system 100 adapted to execute the method of the inventive concept. Further an inventive control system 100 is connected to at least the structural actuator elements as shown in FIG. 11, which serve to supply air or the like oxidizer and fuel to the furnace space 10.

[0313] The inventive control system 100 preferably further provides for a camera CAM adapted in that [0314] an image process of at least a part of the furnace space is provided, namely provided with a series SER of images in the course of time t.sub.i, [0315] wherein an image of the series SER is provided by means of a camera sensor CAM_S of the camera CAM, the camera being installed at the furnace 1 with a camera view CAM_V, i.e. position and/or directed to the furnace space with a camera orientation wherein a camera pose CAM_P is assigned to the position and/or orientation of the camera CAM. The setting has been further exemplified with FIG. 1A and FIG. 1B and FIG. 10A and FIG. 10B and the same reference signs are used in this regard. The series SER of images is taken by an image taking module 130 and a read out module 110 is provided as illustrated below.

[0316] Therein, an image 1B is provided as a pixel image (A(i, j)) with a number of image points (i, j), wherein an image point is assigned to a sensor pixel of the camera sensor CAM_S. An image point (i, j) of the pixel image A(i, j) corresponds to an object-image position assigned to an object location (x, y, z) of an object in the furnace space B(x, y, z);

[0317] The inventive control system 100 preferably further provides for the read out module 110 adapted in that

the image of the furnace space is related to a technical map of at least one process parameter of the furnace space during operation of the furnace by means of an image point read out, and
a control unit 120 is adapted in that [0318] a process parameter P* is used in the monitor and/or control of operation as shown in FIG. 10A and FIG. 10B.

[0319] It can be assumed that with varying camera view CAM_V due to varying camera pose CAM_P in the course of time the image point (i, j) deviates from the corresponding object-image position.

[0320] The inventive control system 100 preferably further provides for the image taking module 130 adapted in that [0321] a reference image (A.sub.reference(i, j)) of the furnace space B(x, y, z) is provided at an initial time [0322] an actual image (A(i.sub.n, j.sub.n)) is provided at a further time (FIG. 4B).

[0323] The inventive control system 100 preferably further provides for a deviation-compensation module DELTA adapted in that [0324] a deviation-compensation is provided to the object-image position in the actual image (A(i.sub.n, j.sub.n)) (FIG. 4C), to apply for the varying camera view ?CAM_V and pose ?CAM_P, wherein the deviation-compensation is based on a deviation identified for a characteristic object-image position in the actual image as compared to the reference image.

[0325] The control unit 120, CU is further adapted in that [0326] the image 1B of the furnace space 10 which is used for relating into the imaging technical map of process parameters P* is determined by means of the deviation-compensated object-image positions in the actual image.

[0327] Thus an industrial furnace 1 for processing a heated material M, in particular for processing a melt in a melting end of a kiln or the like industrial furnace, is shown herewith, wherein the industrial furnace has an inner furnace space 10 comprising a furnace crown, a furnace superstructure and a furnace material basin and comprising a control system 100 as described hereinbefore and/or adapted to execute the method as described hereinbefore.

[0328] The industrial furnace can be formed as a

a glass furnace with a furnace space comprising a furnace crown and a superstructure over a material basin with a glass melt, and/or with the control applying to control of a high temperature process.

[0329] FIG. 12 shows a view graph to illustrate a corresponding method 1000 of monitor and/or control of operation of an industrial furnace for processing a heated material, in particular for processing a melt in a melting end of a kiln or the like industrial furnace, wherein the industrial furnace has an inner furnace space comprising a furnace crown, a furnace superstructure and a furnace material basin as described above with FIG. 1A, FIG. 1B and FIG. 10A, FIG. 10B. In the method 1000: [0330] in step 1100 an image process of at least a part of the furnace space 10 is provided, namely provided with a series SER of images in the course of time t.sub.i, [0331] wherein an image 1B of the series SER is provided by means of a camera sensor CAM_S of a camera CAM, the camera CAM being installed at the furnace 1 with a camera view CAM_V to the furnace space 10, [0332] in particular with the camera having a camera position and/or being directed to the furnace space with a camera orientation, wherein a camera pose CAM_P is assigned to the position and/or orientation of the camera.

[0333] Therein in step 1200 the image IMG is provided as a pixel image (A(i, j)) with a number of image points (i, j), wherein an image point is assigned to a sensor pixel of the camera sensor CAM_S, [0334] and the image of the furnace space is related to a technical map MAP of at least one process parameter of the furnace space during operation of the furnace by means of an image point read out. Therein a process parameter P is used in the monitor and/or control of operation.

[0335] An image point (i, j) of the pixel image A(i, j) corresponds to an object-image position assigned to an object location (x, y, z) of an object in the furnace space B(x, y, z),

[0336] In step 1300 it is illustrated that the camera view is subject to a camera view variation ?CAM_V in the course of time, in particular due to ambient conditions of the furnace, such that the camera view variation causes a deviation ?POS of the image point from the object-image position. As respective deviation ?POS of the image point from the object-image position is also illustrated with FIG. 4C and FIG. 5B. In particular therein with varying camera pose CAM_P in the course of time the image point (i, j) deviates ?POS from the corresponding object-image position. More particular a camera sensor CAM_S pixel PIX is assigned to differing object-image positions A.sub.0, A.sub.1 in the course of time t.sub.i such that the camera view variation ?CAM_V causes a deviation ?POS of the image point from the object-image position.

[0337] For addressing this problem, the method provides an image treatment process 1400 with steps 1410, 1420, 1430, 1440 as shown below.

[0338] In step 1410 a reference image IMG_ref , A.sub.reference(i, j) of the furnace space B(x, y, z) is provided at an initial time to during the course of time t.sub.i. More particular the reference image (A.sub.reference(i, j)) of the furnace space B(x, y, z) is provided at the initial time, and a characteristic object in the furnace space B.sub.0=B(x.sub.0, y.sub.0, z.sub.0) is selected at the initial time, wherein the characteristic object-image position corresponds to an original image point (i.sub.0, j.sub.0) in the reference image A.sub.reference(i, j) (FIG. 4A). This is, the original alignment between B.sub.0=B(x.sub.0, y.sub.0, z.sub.0) and A.sub.0=A(i.sub.0, j.sub.0) at time to is given for the reference image IMG_ref; this alignment is denoted by: A.sub.0.fwdarw.A.sub.0[B.sub.0, t.sub.0]

[0339] In step 1420 an actual image IMG_act, A(i.sub.n, j.sub.n) is provided at a further time t.sub.1 (FIG. 4B) during the course of time t.sub.i. More particular the actual image (A(i.sub.n, j.sub.n)) is provided at the further time (FIG. 4B).

[0340] Characteristic object-image position A.sub.0 originally corresponding to characteristic object B.sub.0 in the reference image (IMG_ref), denoted as A.sub.0.fwdarw.[B.sub.0, t.sub.0], is corresponding to another object B.sub.1 in the actual image (IMG_act), denoted as A.sub.0.fwdarw.A.sub.1[B.sub.1, t.sub.1] New object-image position A.sub.1 corresponding to characteristic object B.sub.0 is determined in the actual image (IMG_act), denoted as A.sub.1.fwdarw.A.sub.1[B.sub.0, t.sub.1].

[0341] In step 1430 [0342] a deviation ?POS is identified for the characteristic object-image position A.sub.1 in the actual image as compared to the characteristic object-image position A.sub.0 in the reference image. Thereby it is recognized that a characteristic object-image position corresponds to an original image point (i.sub.0, j.sub.0) in the reference image A(i.sub.0, j.sub.0) (FIG. 4A) in that an image point (i, j) of the pixel image A(i, j) corresponds to the characteristic object-image position.

[0343] In operation more particularly in an automated image analysis [0344] an object-image position is assigned to a characteristic, in particular fixed, object corresponding to one or more selected points of a group or cluster of points of interest, in particular in the reference image A.sub.reference(i, j), (FIG. 5A), in particular wherein at least one point of interest in the reference image A.sub.reference(i, j) is selected, (FIG. 5A) such that the point of interest is assigned to the characteristic object and the object-image position of the point of interest is determined as an image point (i, j) of the pixel image A.sub.reference(i, j). This implies determining the transformation function Tn as explained in the chapter Determination of the transformation function.

[0345] In step 1440 [0346] a deviation-compensation COMP??POS is provided to the object-image position in the actual image A(i.sub.n, j.sub.n), (FIG. 4C) to apply for the varying camera view, wherein the deviation-compensation is based on the deviation identified. More particular, the deviation-compensation is provided to the object-image position in the actual image A(i.sub.n, j.sub.n), (FIG. 4C) and applies for the varying camera pose, in that [0347] the deviation-compensation is determined by means of identifying the deviation (A(i.sub.0, j.sub.0)?A(i.sub.1, j.sub.1)) between the original image point (i.sub.0, j.sub.0) in the actual image and the another image point (i.sub.1, j.sub.1) in the actual image (FIG. 4C).

[0348] Thus, the camera view variation in the course of time is related to a drift of camera pose. And the camera view variation results in that the image point (i, j) assigned to a sensor pixel of the camera sensor deviates from a corresponding object-image position and results into a difference between an image point read out at the initial time and an image point read out at the further time, and/or [0349] the deviation-compensation to the object-image position in the actual image provides deviation-compensated actual image wherein the sensor pixel of the camera sensor is again related to the corresponding object-image position such that the difference is compensated. This implies using the transformation function Tn as explained above.

[0350] Further, in the method 1000

[0351] In step 1500 a process parameter is determined by means of the deviation-compensated object-image position in the actual image. More particular as an option the image of the furnace space which is used for relating into the imaging technical map MAP of improved process parameters P* is determined by means of the deviation-compensated object-image position in the actual image.

[0352] In step 1600 it is depicted for the method 1000 to use the improved process parameters P* in a control unit CU* as shown in FIG. 10A, FIG. 10B.