Mixed Reality Method And System For Precision Mining
20210047804 ยท 2021-02-18
Inventors
Cpc classification
E02F9/262
FIXED CONSTRUCTIONS
E21C47/00
FIXED CONSTRUCTIONS
E02F9/265
FIXED CONSTRUCTIONS
G06T17/20
PHYSICS
G05D1/0038
PHYSICS
E02F9/205
FIXED CONSTRUCTIONS
International classification
G05D1/00
PHYSICS
G06T17/20
PHYSICS
Abstract
A precision mining system having mining equipment manipulable to recover material with mineral resources, the system comprising: an image sensor for capturing real-time imagery of a geographical location having material with mineral resources and at least one element of the mining equipment; a storage device comprising instructions and the real-time imagery; and a processor configured to execute the instructions to receive, from the image sensor, real-time imagery; at least one sensing device associated with at least one element of the mining equipment configured to acquire data corresponding to at least one of position and motion of the at least one element of the mining equipment in the real-time imagery; and the processor configured to execute the instructions to generate at least one production polygon corresponding to a volume of interest with the mineral resources and combine the real-time imagery with the at least one production polygon to generate real-time composite imagery.
Claims
1. A precision mining system having mining equipment manipulable to recover material with mineral resources, the system comprising: an image sensor for capturing real-time imagery of a geographical location having material with mineral resources and at least one element of the mining equipment; a display device; a storage device comprising instructions and the real-time imagery; and a processor configured to execute the instructions to receive, from the image sensor, real-time imagery; at least one sensing device associated with at least one element of the mining equipment configured to acquire data corresponding to at least one of position and motion of the at least one element of the mining equipment in the real-time imagery; the processor configured to execute the instructions to generate at least one production polygon corresponding to a volume of interest with the mineral resources; an image processing system adapted to receive the real-time imagery and the at least one production polygon, the image processing system comprising a second set of instructions stored in the storage device and the instructions executable by the processor to cause the processor to combine the real-time imagery with the at least one production polygon to generate real-time composite imagery; and a display unit for presenting the real-time composite imagery as a guide for manipulating the at least one element of the mining equipment to accurately recover the material with mineral resources.
2. The precision mining system of claim 1, wherein the at least one production polygon is generated using at least one of geological data, survey data, and site location data corresponding to a geographical location having the material with mineral resources.
3. The precision mining system of claim 2, wherein the processor is configured to receive, from a modelling engine, the at least one production polygon corresponding to a volume of interest with the mineral resources.
4. The precision mining system of claim 3, wherein the at least one sensing devices provides at least one of positional data and velocity data of the at least one element of the mining equipment a 3D space.
5. The precision mining system of claim 4, wherein at least one of positional data and velocity data is received and analyzed by the processor to track the movement of the at least one element of the mining equipment in real time, wherein the at least one element of the mining equipment is one of a boom, a dipper arm and an excavation bucket.
6. The precision mining system of claim 5, wherein the processor provides navigational data instructive for placement of the mining equipment for excavation of the material in an excavation event, and subsequent loading of the materials into a haulage truck positioned in a predetermined location based on the least one of positional data and velocity data.
7. The precision mining system of claim 6, wherein the processor receives and processes the at least one of positional data and velocity data to permit orientating and positioning of the excavation bucket during at least one of an excavation event, loading event and classification event.
8. The precision mining system of claim 7, wherein the at least one production polygon comprises a plurality of blocks, and each block comprises at least one block attribute which reflects athe geological body's properties of the at least one production polygon.
9. The precision mining system of claim 8, wherein the at least one block attribute comprises at least one of density, rock type, and reserves grade.
10. The precision mining system of claim 9, wherein the at least one block attribute is employed to classify a rock material as at least one of ore material, ore/waste material and waste material.
11. The precision mining system of claim 10, wherein the at least one block attribute is associated with at least one of a color, shading and transparent gradient.
12. The precision mining system of claim 11, wherein the at least one production polygon is updated following each excavation event to show unexcavated material.
13. The precision mining system of claim 12, wherein the display unit comprises a graphical portion comprising statistics pertaining to the excavation event.
14. The precision mining system of claim 13, wherein the statistics comprise at least one of a percentage of ore recovered and a percentage of waste recovered.
15. The precision mining system of claim 14, wherein a control system receives at least one block attribute, positional data, excavation data, and generates instructions to position and orient the bucket to excavate a desired material of the volume of interest, and place an excavated material in at least one of an ore stream, an ore/waste stream and a waste stream, thereby minimizing misclassification of the ore material, ore waste material and the waste material.
16. The precision mining system of claim 15, wherein the control system autonomously or semi-autonomously positions and orients the excavation bucket during at least one of an excavation event, a loading event, and a classification event of the material in the at least one of an ore stream, an ore/waste stream and a waste stream.
17. The precision mining system of claim 7, wherein positional data associated with the mining equipment and the haulage truck is used to streamline dispatch operations.
18. A computer-implemented method for improving the accuracy of at least one element of a mining equipment manipulable to recover material with mineral resources, the method comprising the steps of: acquiring at least one a of geological data, a survey data, and a site location data corresponding to a geographical location having a material with mineral resources; with a processor, executing a first set of instructions stored in a memory to cause the processor to generate at least one synthetic production polygon corresponding to a volume of interest with the mineral resources; acquiring data corresponding to at least one of position and motion of the at least one element of the mining equipment in a 3D space; determining whether the at least one element of the mining equipment is properly positioned and/or oriented to recover the material with mineral resources; and when the at least one element of the mining equipment is not positioned and/or oriented properly, then causing the at least one element of the mining equipment to change location and orientation to suit a desired location and orientation for recovery of the material with mineral resources; else when the at least one element of a mining equipment is properly positioned and/or oriented then capturing a real-time imagery associated with the geographical location and the at least one element of the mining equipment; with the processor, executing a second set of instructions stored in the memory to cause the processor to combine the real-time imagery with the at least one synthetic production polygon and generate a real-time composite imagery; and presenting the real-time composite imagery on a display as a guide for manipulating the at least one element of the mining equipment to accurately recover the material with mineral resources.
19. The method of claim 18, wherein the at least one synthetic production polygon comprises a plurality of blocks, and each block comprises an at least one block attribute reflecting the geological body's properties of the at least one synthetic production polygon.
20. The method of claim 19, wherein the at least one block attribute comprises at least one of density, rock type, and reserves grade.
21. The method of claim 20, wherein the at least one block attribute is associated with at least one of a color, shading and transparent gradient.
22. The method of claim 21, wherein the at least one synthetic production polygon comprises a wire frame outline.
23. The method of claim 22, wherein with the processor, executing a third set of instructions stored in the memory to cause the processor to present the at least one synthetic production polygon comprising the wire frame outline on a display means.
24. The method of claim 23, wherein the display means is at least one of a windshield of the mining equipment, a display screen, a heads-up display (HUD), and a head-mounted display (HMD).
25. The method of claim 20, wherein a control system receives the at least one block attribute, the geological data, the survey data, the site location data, and a positional data, and generates instructions to position and orient the at least one element of the mining equipment to excavate or recover the material with mineral resources, and place an excavated material in at least one of an ore stream, an ore/waste stream and a waste stream.
26. The method of claim 18, wherein a control system autonomously or semi-autonomously positions and orients the at least one element of the mining equipment during an excavation event, a loading event, and a classification event of the material in at least one of an ore stream, an ore/waste stream and a waste stream.
27. A computer program product comprising at least one non-transitory computer-readable storage medium having instructions stored therein, the instructions executable by a processor to at least: acquire at least one of a geological data, a survey data, and a site location data corresponding to a geographical location having a material with mineral resources; acquire data corresponding to at least one of a position and a motion of an at least one virtual articulating structure associated with an earth moving vehicle in a 3D space, said earth moving vehicle being located at the geographical location; generate a virtual earth moving vehicle having the at least one virtual articulating structure in the 3D space; generate an at least one virtual production polygon corresponding to a volume of interest having mineral resources at the geographical location in the 3D space; acquire a captured image data of the geographical location; combine the at least one virtual production polygon corresponding to a volume of interest having mineral resources at the geographical location and the captured image data of the geographical location to generate a first mixed-reality view for presentation on a first portion of a user interface; combine the at least one virtual production polygon corresponding to a volume of interest having mineral resources at the geographical location, and the virtual earth moving vehicle having the at least one virtual articulating structure and the captured image data of the geographical location to generate a second mixed-reality view for presentation on a second portion of the user interface; and based on the at least one of the first mixed-reality view displayed on the first portion of a user interface and the second mixed-reality view displayed on the second portion of the user interface, determine whether the at least one virtual articulating structure is properly positioned and/or oriented to excavate the material with mineral resources based on the position data and motion location data of the at least one virtual articulating structure, the site location data and the at least one virtual production polygon.
28. The computer program product of claim 27, comprising instructions executable to receive commands to cause an excavation session information to be displayed on at least one of the first portion of the user interface, the second portion of the user interface, and a third portion of the user interface.
29. The computer program product of claim 28, wherein the excavation session information comprises statistics pertaining to an excavation session.
30. The computer program product of claim 29, wherein the least one virtual articulating structure is one of a boom, a dipper arm and an excavation bucket.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0037] Several exemplary embodiments of the present invention will now be described, by way of example only, with reference to the appended drawings in which:
[0038]
[0039]
[0040]
[0041]
[0042]
DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS
[0043] Various embodiments of the disclosure are discussed in detail below. While specific implementations are discussed, it should be understood that this is done for illustration purposes only. A person skilled in the relevant art will recognize that other components and configurations may be used without parting from the spirit and scope of the disclosure.
[0044]
[0045] Precision mining platform 10 comprises exemplary mining equipment 12, such as a mining shovel and a rope shovel, and mining platform controller 13. Other mining equipment 12 may comprise excavators, backhoes, front-end loaders, electric mining shovels, and so forth. As an example, rope shovel 12 may have dipper payloads of 109 tonnes or more, and dipper capacities of 60 m3 or more, with boom lengths exceeding 20 m, such as the Cat7495 Electric Rope Shovel, manufactured by Caterpillar Inc. of Peoria, Ill., U.S.A.
[0046] As can be seen in
[0047] Mining shovel 12 further comprises sensing devices associated with boom 18, dipper arm 28, bucket 32, and mining shovel body 22. For example, sensing devices may comprise location sensing devices 50 associated with any remote positioning system, whether terrestrial or non-terrestrial. For example, satellite based systems currently available include the global positioning system (GPS), and land-based systems can include Wi-Fi positioning systems, among others. Location sensing devices 50 track the position of mining shovel 12, including orientation thereof. Other sensing devices include motion sensing devices, such as inertial-movement unit sensors 52, which may be associated with articulated arm 16 and mining shovel 12, and responsive to the movement of articulated arm 16 and/or shovel 12, to permit orientation and positioning of mining shovel 12. Sensing devices 50, 52 may be communicatively coupled to relay transceiver 54, such as a base station or wireless access point to transmit the sensed signals via communication network 55 to mining platthrm controller 13, or receive control signals from mining platthrm controller 13. Haulage trucks 42, 44, 46 may include location sensing devices 50, as described above, and may be communicatively linked to mining shovel 12, mining platform controller 13 and dispatch system 56 via communication network 55.
[0048] As shown in
[0049] Processor 57 receives sensed data from the sensing devices, such as GPS sensors 50 and inertial-movement unit sensors 52, via transceiver 70, and image processing system 59 processes the real imagery of open-pit surface 14 captured by image capture device 72. Generally, image processing system 59 comprises a set of instructions stored in memory 58 associated with mining platform controller 13, with a plurality of lines of code executable by processor 57 to combine the real imagery with graphical block 62 locating the volume of interest, in conjunction with the sensed data, and for storage in memory 58 or database 69, and for access and retrieval in real-time. Accordingly, production polygon 62 is overlaid on imagery of open-pit surface 14 to create composite real-time image 79, and composite real-time image 79 is displayed on visual display screen 74, which may be a heads-up display (HUD), head-mounted display (HMD), hand held display unit, cabin windshield with display images from a projection unit, or any other graphical display. Image processing system 59 may thus comprise an augmented reality module, which mixes or juxtaposes synthetic block model elements 62 with real world elements, open-pit surface 14, in such a way that synthetic block model elements 62 appear to be part of open-pit surface 14. In
[0050] In more detail, looking again at
[0051]
[0052]
[0053] In step 116, imaging processing system 59 combines real-time video imagery from step 114 with synthetic block models 62 generated in step 104, and real-time composite image 79 is displayed on display 74 (step 118), such as cabin windshield or a video display monitor. As noted above, synthetic block models 62 represent the volume of interest, and depending on the geological nature of the volume of interest, synthetic block model elements 62 may include blocks 62a, 62b indicative of the presence of mineral ore deposit, or varying concentrations thereof within a single block model element 62. Accordingly, an equipment operator can be visually guided to precisely position bucket 32 using the displayed composite image 79 (step 120). Having the knowledge of the attributes of block model elements 62a, 62b enables an equipment operator to systematically orient and position bucket 32 via control system 75, and excavate the desired material 15 of the volume of interest, and place the excavated material 15 in the correct stream, that is, ore stream, ore/waste stream and waste stream (step 122). Control system 75 may comprise a set of instructions stored in memory 58, with a plurality of lines of code executable by processor 57 to position bucket 32 automatically. After each bucket 32 load, block 62 is updated to remove portion 62a or 62b of block 62 representative of the material 15 that was loaded into bucket 32 for placement in one of the material streams (step 124), and portion 62a or 62b is replaced by portion 62a or 62b, respectively, to represent an empty block or removed material 15. Accordingly, during the session block 62 is deformed with each bucket 32 load and the percentage of recovered mineral ore and waste is tracked in real-time (step 126), which may be recorded and displayed on display 74.
[0054] In addition, haulage trucks 42, 44, 46 comprise location sensing devices 50, such as GPS sensors or Wi-Fi position sensors for tracking movement thereof, and for allowing operators of haulage trucks 42, 44, 46 to receive directions and allow proper parking of haulage trucks 42, 44, 46 to receive excavated material 15 from mining shovel 12.
[0055] Mining platform controller 13 may further comprise other components, such as a storage device, video processor, and I/O controller for coupling a number of input/output (I/O) devices thereto, such as a keyboard, touch screen, pointing device, and a communications interface device. The components of mining platform controller 13 may be coupled by an interconnection mechanism, which may include one or more buses (e.g., between components that are integrated within a same machine) and/or a network (e.g., between components that reside on separate discrete machines).
[0056] In another implementation, communication between mining equipment assets 12, 42, 44, 46, equipment operators, dispatch center or dispatch system 56, within precision mining platform 10 allows for the streamlining of dispatch operations and management of all assets 12, 42, 44, 46, and personnel.
[0057] In another implementation, precision mining platform 10 may control mining shovel 12 to accurately load excavated materials into waiting haulage trucks 42, 44, 46, based on the classification of the material. Therefore, haulage trucks 42, 44, 46 may be represented as a synthetic element in the composite video, while bucket 32, materials 15 and open-pit surface 14 represent real-time video imagery.
[0058] In another implementation, sensing devices are communicatively linked to mining platform controller 13 via a wired or wireless connection controller; or communicatively linked to platform transmitter 54 via a wired or wireless connection.
[0059] In another implementation, sensing devices may include load sensors, proximity sensors, pressure sensors, temperature, flow, mass, heat sensors, humidity sensors, position sensors, velocity sensors, accelerometers, vibration sensors, gyroscope sensors, among others.
[0060] In another implementation, display 74 is wirelessly linked to image processor system 59.
[0061] In another implementation, precision mining platform 10 may guide an operator in performing an excavation remotely.
[0062] In another implementation, mining shovel 12 autonomously or semi-autonomously positions itself on open-pit surface 14 based on location data, and orients and positions bucket 32 during an excavation event, a loading event, and a classification event of material 15 into one or more streams.
[0063] In another implementation, precision mining platform 10 automates surveying of the geographical area, post-blast inspection, shovelling of material 15 and hauling of material 15, sorting of material 15 into a stream (ore stream/ore-waste stream and waste stream) and real-time assay of material 15.
[0064] In another implementation, haulage trucks 42, 44, 46 may be a rail car, flexible conveyor train, in-pit crushing hopper, and/or truck with an open bed trailer, etc.
[0065] In another implementation, database 69 may be any type of data repository or combination of data repositories, which store records or other representations of data comprising any data associated with mining platform 10, such as open-pit profile data, geological data, survey data, sensed data, positioning data, operational data, production data, equipment operator profile data, mining operator data, and statistical data.
[0066] In another implementation, as shown in
[0067] In another implementation, one or more of the components and/or one or more additional components of the example environment of
[0068] In another implementation, mining platform controller 13 may comprise a server which executes a web server application, examples of which may include but are not limited to: Microsoft IIS or Apache Webserver, that allows for HTTP (i.e., HyperText Transfer Protocol) access to server computer 70 via communication network 55.
[0069] In another implementation, operator computing device 85 include any device, such as, a personal computer, laptop, tablet, computer server, or smartphone.
[0070] The logic of
[0071] In other implementations, various aspects of the invention may be distributed among one or more computer systems (e.g., servers) configured to provide a service to one or more client computers, or to perform an overall task as part of a distributed system. For example, various aspects of the invention may be performed on a client-server, hybrid client-server, or multi-tier system that includes components distributed among one or more server systems that perform various functions according to various embodiments of the invention.
[0072] In other implementations, processor 57 may be configured to execute hard-coded functionality.
[0073] In one embodiment, processor 57 may be embodied as a multi-core processor, a single core processor, or a combination of one or more multi-core processors and one or more single core processors. For example, processor 57 may be embodied as one or more of various processing devices, such as a coprocessor, a microprocessor, a controller, a digital signal processor (DSP), a processing circuitry with or without an accompanying DSP, or various other processing devices including integrated circuits such as, for example, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a microcontroller unit (MCU), a hardware accelerator, a special-purpose computer chip, Application-Specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), Graphics Processing Units (GPUs), and the like. For example, some or all of the device functionality or method sequences may be performed by one or more hardware logic components.
[0074] Communications network 55 can include a series of network nodes (e.g., the clients and servers) that can be interconnected by network devices and wired and/or wireless communication lines (such as, public carrier lines, private lines, satellite lines, etc.) that enable the network nodes to communicate. The transfer of data between network nodes can be facilitated by network devices, such as routers, switches, multiplexers, bridges, gateways, etc., that can manipulate and/or route data from an originating node to a server node regardless of dissimilarities in the network topology (such as, bus, star, token ring, mesh, or hybrids thereof), spatial distance (such as, LAN, MAN, WAN, Internet), transmission technology (such as, TCP/IP, Systems Network Architecture), data type (such as, data, voice, video, multimedia), nature of connection (such as, switched, non-switched, dial-up, dedicated, or virtual), and/or physical link (such as, optical fiber, coaxial cable, twisted pair, wireless, etc.) between the correspondents within the network. Communication network 55 may be connected to one or more secondary networks, examples of which may include but are not limited to: a local area network; a wide area network; or an intranet, for example.
[0075] Database 69 may be, include or interface to, for example, the Oracle relational database sold commercially by Oracle Corp. Other databases, such as Informix, DB2 (Database 2), Sybase or other data storage or query formats, platforms or resources such as OLAP (On Line Analytical Processing), SQL (Standard Query Language), a storage area network (SAN), Microsoft Access or others may also be used, incorporated or accessed in the invention. Alternatively, database 69 is communicatively coupled to mining platform controller 13.
[0076] Benefits, other advantages, and solutions to problems have been described above with regard to specific embodiments. However, the benefits, advantages, solutions to problems, and any element(s) that may cause any benefit, advantage, or solution to occur or become more pronounced are not to be construed as critical, required, or essential features or elements of any or all the claims. As used herein, the terms comprises, comprising, or any other variations thereof, are intended to cover a nonexclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Further, no element described herein is required for the practice of the invention unless expressly described as essential or critical.
[0077] The preceding detailed description of exemplary embodiments of the invention makes reference to the accompanying drawings, which show the exemplary embodiment by way of illustration. While these exemplary embodiments are described in sufficient detail to enable those skilled in the art to practice the invention, it should be understood that other embodiments may be realized and that logical and mechanical changes may be made without departing from the spirit and scope of the invention. For example, the steps recited in any of the method or process claims may be executed in any order and are not limited to the order presented. Further, the present invention may be practiced using one or more servers, as necessary. Thus, the preceding detailed description is presented for purposes of illustration only and not of limitation, and the scope of the invention is defined by the preceding description, and with respect to the attached claims.