ROCK PROCESSING MACHINE INCLUDING IMAGE ACQUISITION AND IMAGE PROCESSING BY A NEURAL NETWORK

Abstract

A rock processing machine is disclosed for crushing and/or grain size-dependent sorting of pourable rock material. The rock processing machine comprises a rock processing device having a crushing crusher device and a sorting screen, at least one camera system in whose field of view in the operation of the machine a surface of the pourable rock material is located, and a data processing device configured to process image data of the camera system via an artificial neural network. The data processing device is configured to ascertain at least one image area portion in image data recorded and transmitted by the camera system as an image object and to classify the ascertained image object by using the artificial neural network with respect to at least one object property from among a group of object properties comprising: object shapes; object sizes; object types; and/or object materials.

Claims

1-16. (canceled)

17. A rock processing machine for crushing and/or grain size-dependent sorting and/or conveying of pourable rock material, the rock processing machine comprising: at least one rock processing device comprising at least one crushing crusher device and at least one sorting screen; at least one conveyor device configured to convey the pourable rock material; at least one camera system in the field of view of which, in the operation of the machine, a surface of the pourable rock material is located; and a data processing device configured to process image data of the camera system by using an artificial neural network, ascertain at least one image area portion in image data recorded and transmitted by the camera system as an image object, and classify the ascertained image object by using the artificial neural network with respect to at least one object property selected from among an object shape, an object size, an object type, and/or an object material.

18. The rock processing machine of claim 17, wherein the at least one camera system is situated upstream and/or downstream from the rock processing device in the flow of the pourable rock material.

19. The rock processing machine of claim 17, wherein the data processing device is configured to change at least one operating parameter of the rock processing machine based on the at least one classified object property and/or to inform a machine operator about a recommended change of operating parameters based on the at least one classified object property.

20. The rock crushing machine of claim 19, wherein the at least one operating parameter of the at least one rock processing device changeable by the data processing device comprises a: crushing gap width of a crusher device; drive speed of a crusher device; filling ratio of a crusher device; conveying speed of a conveyor device; movement frequency of at least one screen; movement amplitude of at least one screen; identification of at least one discharge conveyor device to be controlled; inclination and orientation of at least one conveyor device; distance of a magnetic separator from a device or surface; magnetic performance of the magnetic separator; drive speed of a wind sifter; and/or volumetric flow of the wind sifter.

21. The rock processing machine of claim 19, wherein the data processing device is configured to ascertain, based on the at least one classified object property, at least one quantitative value comprising a: weighted average value of an object size distribution of one and the same object type; number per unit of time of classified objects; number of different object types classified per unit of time; number of different object shapes classified per unit of time; number of different object materials classified per unit of time; weighted average value of a parameter representing different object types and/or object shapes and/or object materials and/or object sizes; and/or statistical evaluation of at least one of the aforementioned parameters.

22. The rock processing machine of claim 19, wherein the rock processing device is configured, upon classifying an object as a foreign object that at least one rock processing device of the rock processing machine is unable to process, to output a message to a machine operator indicating the foreign object, and/or start a separation process for separating the foreign object from a flow of material of the rock processing machine via a separation device.

23. The rock processing machine of claim 19, wherein the rock processing device is configured, upon determining a reaching or undershooting of a predetermined threshold number of objects per unit of time classified as processable by the at least one rock processing device, to initiate at least one action comprising: transmitting corresponding information to a charging device cooperating with the rock processing machine; transferring the rock processing machine into a mode consuming less energy per unit of time; and/or stopping the rock processing machine.

24. The rock processing machine of claim 17, comprising: at least two camera systems having fields of view each detecting along the flow of material at different locations in the rock processing machine, wherein the data processing device is configured to classify ascertained image objects in image data of at least two of the at least two camera systems based on the same ground truth regarding at least one object property.

25. The rock processing machine of claim 17, wherein the data processing device comprises a training mode, enabling training of the artificial neural network used by the data processing device based on image data of at least one camera system of the rock processing machine.

26. The rock crushing machine of claim 25, wherein the training mode enables at least one action comprising: assigning object properties to image data by an operator; and/or entering object properties of a known rock material loaded into the rock processing machine and automatically assigning the entered object properties to ascertained image objects in acquired image data.

27. The rock processing machine of claim 17, comprising: a data transmission device configured to transmit image data of at least one camera system to a remote data processing device situated at a distance from the rock processing machine.

28. The rock processing machine of claim 27, wherein the data transmission device is coupled at least temporarily to the remote data processing device in a data-transmitting manner, and wherein the remote data processing device is configured to allow an assignment of object properties to image data transmitted from the rock processing machine and thereby to generate an expanded ground truth of the artificial neural network of the rock processing machine, the expanded ground truth being transmittable to the data processing device of the rock processing machine for use by the artificial neural network of the rock processing machine.

29. The rock processing machine of claim 17, wherein the artificial neural network is a convolutional neural network.

30. The rock processing machine of claim 17, wherein the data processing device is configured to ascribe a quality value representing a quality of the image data to the image data of the at least one camera system during the image processing or in a separate quality assurance process.

31. The rock processing machine of claim 30, wherein the data processing device is further configured, upon determining that a quality value ascribed to the image data does not reach a predetermined minimum quality value, to output a warning signal and/or to terminate an automated process management of the rock processing in the machine.

32. A rock processing plant comprising: at least two rock processing machines sequentially arranged in a common rock material flow of the plant; each of the at least two rock processing machines configured for crushing and/or grain size-dependent sorting and/or conveying of pourable rock material, and comprising: at least one rock processing device comprising at least one crushing crusher device and at least one sorting screen; at least one conveyor device configured to convey the pourable rock material; at least one camera system in the field of view of which, in the operation of the machine, a surface of the pourable rock material is located; and a data processing device configured to process image data of the camera system by using an artificial neural network, ascertain at least one image area portion in image data recorded and transmitted by the camera system as an image object, and classify the ascertained image object by using the artificial neural network with respect to at least one object property selected from among an object shape, an object size, an object type, and/or an object material.

33. A method for updating and developing an artificial neural network used, in a rock processing machine or an associated rock classifying plant, for classifying ascertained image objects, wherein the rock processing machine comprises at least one rock processing device comprising at least one crushing crusher device and at least one sorting screen, at least one conveyor device configured to convey the pourable rock material, at least one camera system in the field of view of which, in the operation of the machine, a surface of the pourable rock material is located, and a data processing device configured to process image data of the camera system by using an artificial neural network, ascertain at least one image area portion in image data recorded and transmitted by the camera system as an image object, and classify the ascertained image object by using the artificial neural network with respect to at least one object property selected from among an object shape, an object size, an object type, and/or an object material; wherein the method comprises steps of: a) acquiring image data of a pourable rock material in the rock processing machine; b) ascertaining image objects in the acquired image data via an image processing device; in a manual updating method: c1a) assigning object properties to the ascertained image objects by an operator; and c1b) weighting connections between neurons of the artificial neural network on the basis of the generated assignment of image objects and object properties; or in an automated updating method: c2a) entering object properties of a known rock material into the data processing device; c2b) prior to steps a) and b): loading the known rock material into the rock processing machine; c2c) after steps a) and b): automated assignment of object properties to the ascertained image objects by the data processing system; and c2d) weighting connections between neurons of the artificial neural network on the basis of the generated assignment of image objects and object properties; or in a remote updating method: c3a) transmitting the image data with or without the ascertained image objects to a remote data processing device; c3b) assigning object properties to the transmitted image data; c3c) weighting connections between neurons of the artificial neural network based on the generated assignment of image objects and object properties; and c3d) transmitting the ascertained connection weights to at least one rock processing machine or rock processing plant.

34. The method of claim 33, wherein in step c3d) the ascertained connection weights are transmitted to at least two of the rock processing machines.

35. The method of claim 34, wherein the rock processing plant comprises the at least two rock processing machines sequentially arranged in a common rock material flow of the rock processing plant, and in step c3d) the ascertained connection weights are further transmitted to the rock processing plant.

36. The method of claim 33, wherein in step c3d) the ascertained connection weights are transmitted to the rock processing plant, wherein the rock processing plant comprises the at least two rock processing machines sequentially arranged in a common rock material flow of the rock processing plant.

Description

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

[0092] FIG. 1 is a rough schematic elevation view of a specific embodiment of a rock processing machine as disclosed herein.

DETAILED DESCRIPTION

[0093] An exemplary rock processing machine is generally denoted by 10 in FIG. 1. The machine 10 comprises a machine frame 12, which stands on a contact subsurface U via a crawler undercarriage 14 that is known per se. The machine 10 is therefore a mobile rock processing machine 10, which with its crawler undercarriage 14 is able to drive independently at least from a transport device, such as a flat-bed vehicle for example, to its site of operation.

[0094] The machine 10 comprises a combustion engine 16, for example a diesel engine, which forms a central power plant of machine 10. The combustion engine 16 may drive for example a hydraulic motor 18 and an electrical generator 20, so that when the combustion engine 16 is in operation, a predetermined hydraulic pressure level and an electrical energy supply beyond electrical energy merely stored in batteries is available.

[0095] The machine 10 has a first rock processing device, namely, a jaw crusher 22. The right crushing jaw 24 in FIG. 1 is driven by an eccentric 26 to perform a reciprocal movement toward and away from the left crushing jaw 28 in FIG. 1, which is fixed to the machine frame, in a pulsating change of the crushing gap 29 existing between the crushing jaws 24 and 28. The movement of the eccentric 26 is provided by the combustion engine 16.

[0096] Via a charging unit 30, the jaw crusher 22 is charged with material 32 to be crushed in the jaw crusher 22. As a conveyor device, the machine 10 has a charging chute 34, which conveys the material 32 charged therein as a vibrating conveyor to a double-deck primary screen 36. The double-deck primary screen 36 in operation is driven to circular vibration and forms a second rock processing device. Here, a fine fraction 35 and a fraction 37 having medium-sized grain are separated from material 32 and are conveyed separately from the rest of the material 32. The fine fraction 35 may be discharged from the machine 10, for example. The fraction 37 having a medium grain size may be conveyed directly onto the crusher discharge conveyor belt 38 as a further conveyor device, which also conveys the crushed material 40 emerging from the jaw crusher 22 after passing through it away from jaw crusher 22 to a discharge location, from where the material 40, which was crushed as intended, is heaped up.

[0097] Along the conveyor path from the crushing gap 29 to the discharge location 42, the material 37 and 40 is conveyed past a magnetic separator 44. The electrically driven magnetic separator 44, which magnetically separates ferromagnetic portions, such as steel reinforcements, from the crushed material 37 and 40 and conveys the separated ferromagnetic material away from machine frame 12 in a direction projecting from the drawing plane of FIG. 1, is a separation device in the sense of the introduction of the description.

[0098] The machine 10 may be operated and controlled via an operating console 46 situated laterally on the machine frame 12 by way of example. The operating console 46 is connected for data transmission to a data processing device 48 and a data storage device 50. Data storage device 50 is likewise connected to the data processing device 48.

[0099] The data processing device 48 is very generally designed for data processing on the rock processing machine 10, also for image processing. It is also used for controlling operational sequences and is therefore also a control device of the rock processing machine 10. The control device of machine 10 may be developed separately from the data processing device 48 as an independent device, so that the machine 10 then comprises a control device normally comprising microprocessors and a data processing device 48 normally also comprising microprocessors. The two separate devices then differ with respect to the data processing operations running within them.

[0100] Data storage device 50 stores parameters for a neural network, which is developed as hardware in data processing device 48 by an appropriate interconnection of microprocessors. Additionally or alternatively, the neural network may also be implemented as software by a corresponding program.

[0101] The ground truth on which the neural network was trained is stored for example in data storage device 50. The neural network is a convolutional neural network especially suited for the present data processing purpose of an automated classification of ascertained image objects.

[0102] The data processing device 48 and with it the data storage device 50 are connected to a transmitting and receiving data transmission device 52, for example a radio antenna. The data transmission device 52 may operate for example in accordance with the UMTS or 5G standard. Other data transmission devices 52 are likewise conceivable, for example WLAN access points, which may be in radio communication with further WLAN access points outside of machine 10. The further WLAN access points may then be connected by a cable to a data transmission network.

[0103] Above the charging unit 30, more precisely the charging chute 34, a frame 54 is situated, which supports a first camera system 56. The field of view 56a of the first camera system 56 is directed onto the carrying side of the charging chute 34 and thus onto the rock material 32, which in the operation of machine 10 is fed to the rock processing devices 36 (double-deck primary screen) and 22 (jaw crusher).

[0104] The first camera system 56 supplies image data to the data processing device 48, which ascertains image objects in the image data, for example as contiguous, sufficiently homogeneous image areas. On the basis of the ground truth available to it, the neural network implemented in the data processing device 48 then classifies the image objects according to object shape, object size, object type and object material. On the basis of a brightness or relative brightness of a recognized object material, the neural network is possibly also able to infer an object moisture.

[0105] Starting from the object properties of the ascertained image objects classified in this manner, the data processing device 48 is able to ascertain the processing conditions to be expected by application of statistical methods known per se and to derive therefrom the required operating parameters, with which the rock processing devices 22 and 36 as well as the conveyor devices 34 and 38 are to be operated in order to achieve the desired processing result, for example regarding the achieved grain size.

[0106] Above the crusher discharge conveyor belt 38, a further frame 58 is situated including a second camera system, whose field of view 60a is directed onto the carrying side of the crusher discharge conveyor belt 38 and thus, during the operation of machine 10, onto the pourable rock material 37 and onto the pourable crushed rock material 40 on the crusher discharge conveyor belt 38.

[0107] The second camera system 60 also provides image data of the rock materials 37 and 40 to the data processing device 48, in which image objects are ascertained in the same manner as from the image data of the first camera system 58 and are classified using the same neural network of the data processing device 48 on the basis of the same ground truth. The classified image objects in turn may be processed further by the data processing device 48 by application of statistical methods for assessing the rock materials 37 and 40.

[0108] On the basis of the data processing results of the image data of the second camera system 60, the data processing device 48 is able to check whether the desired work result was obtained, for example whether the rock material was crushed sufficiently, and is able, if indicated, to change the operating parameters of the rock processing devices 22 and 36 on the basis of the image data of the second camera system 60 and their processing by the neural network and, if indicated, by using statistical methods, in order to adapt the actually obtained processing result to the desired target processing result. At the same time, contaminations (binding material, foreign bodies) may be detected and characteristics regarding the quality of the end product or of an intermediate product may be ascertained at the rock processing machine 10.

[0109] At the same time, the neural network is able to use this correlation of input rock and processed or crushed rock from the image data of the camera systems 56 and 60, in order to improve the control behavior of data processing device 48 on the basis of the rock material 32 input into machine 10. Thus, in subsequent rock processing sequences, immediately after classifying the input pourable rock material, possibly following further statistical processing of the classification data, it is possible to set more appropriate operating parameters for operating the rock processing devices 22 and 36 at an earlier point in time in the processing sequence than in an earlier rock processing sequence.

[0110] If one of the camera systems 56 or 60 records image data, which the neural network existing in machine 10 is only insufficiently capable of classifying, these image data may be transmitted via data transmission device 54 to a remote data processing device 62, which likewise has a data transmission device 64. There, the image data may be processed and image objects ascertained therein may be classified by manual semantic segmentation. The ground truth developed further in this manner may be transmitted back to machine 10 via data transmission devices 54 and 64. In this manner, it is possible continuously to improve and develop the neural network of machine 10.

[0111] In the event that a processing result of machine 10, for example the rock materials 37 and 40 on the crusher discharge conveyor belt 38, forms the starting material for a further rock processing machine 66, the second camera system 58 of the rock processing machine 10 is able to serve as the first camera system 58 of the further rock processing machine 66. The data transmission device 54 may then transmit the image data or the already classified and possibly statistically processed classified image object information to the data transmission device of the further rock processing machine 66, which either controls its rock processing devices using its control and/or data processing device on the basis of this image object information or which compares or adjusts the transmitted image object information with image object information obtained by its own camera system or its own neural network and then takes control measures such as selecting operating parameters and adjusting rock processing devices and conveyor devices in accordance with the selected operating parameters.

[0112] The rock processing machines 10 and 66 form a rock processing plant 70.