REMOTE OPERATION TERMINAL AND MOBILE CRANE PROVIDED WITH REMOTE OPERATION TERMINAL

Abstract

A remote operation terminal (32) of a crane (1) provided with a GNSS receiver (30) for calculating the current position of the tip of a boom (9); acquires attitude information of the crane (1) and the current position of the tip of the boom (9) from a control device (31) of the crane (1); generates a three-dimensional image (G) of the crane (1) at the current time and planimetric features on the basis of the information acquired from the control device (31) and three-dimensional information acquired by a three-dimensional information acquisition unit (42); displays, on a display device (40), the generated three-dimensional image (G) at the position of and in the direction of a viewpoint input to a viewpoint changing operation tool (34); and generates a control signal for the crane (1) so that the tip of the boom (9) of the three-dimensional image (G) moves in a moving direction input to a suspended load. moving operation tool (35) in the three-dimensional image (G) of the crane (1) displayed on the display device (40).

Claims

1. A remote operation terminal of a mobile crane including a position detection unit that detects a current position of a tip of a boom, the remote operation terminal comprising: a three-dimensional information acquisition unit that acquires three-dimensional information of each of the mobile crane and a planimetric feature within a movable range of the mobile crane; a display unit that displays, in a virtual space, a three-dimensional image of each of the mobile crane and the planimetric feature within the movable range of the mobile crane; a crane operation unit that receives a crane operation done by an operator and remotely operates the mobile crane so as to correspond to the crane operation; an image operation unit that receives an image display changing operation done by the operator and controls a viewpoint position and a viewpoint direction in the three-dimensional image displayed on the display unit so as to correspond to the image display changing operation; and a control unit that is configured to be communicable with a control device of the mobile crane and controls each unit of the remote operation terminal, wherein the control unit acquires attitude information of the mobile crane, movable range information of the mobile crane, and current position information of the tip of the boom from the control device of the mobile crane, generates the three-dimensional image of the current mobile crane and the three-dimensional image of the planimetric feature within the movable range of the current mobile crane, on the basis of the information acquired from the control device of the mobile crane and the three-dimensional information acquired by the three-dimensional information acquisition unit, displays the generated three-dimensional image in the virtual space of the display unit at the viewpoint position and the viewpoint direction input to the image operation unit, and when the crane operation is input to the crane operation unit, associates a coordinate system of a horizontal plane of the three-dimensional image displayed on the display unit and an operation direction or the crane operation on the basis of the viewpoint position and the viewpoint direction, and generates a control signal for remotely operating the mobile crane so that the tip of the boom displayed on the display unit moves horizontally toward the operation direction of the crane operation.

2. The remote operation terminal according to claim 1, wherein the crane operation unit is configured to be capable of operating the three-dimensional image of the mobile crane displayed on the display unit, and the control unit generates a control signal for remotely operating the mobile crane on the basis of operation information of the three-dimensional image by the crane operation unit.

3. The remote operation terminal according to claim 1, wherein the control unit acquires an image captured by a suspended load camera included in the mobile crane, and causes the display unit to display the acquired image together with the three-dimensional image.

4. A remote operation terminal of a mobile crane including a position detection unit that detects a current position of a tip of a boom, the remote operation terminal comprising: an aerial camera that images the mobile crane and a planimetric feature within a movable range of the mobile crane; an imaging information acquisition unit that acquires an image captured by the aerial camera, and position coordinates and a capturing direction of the aerial camera; a display unit that displays the image; a crane operation unit that receives a crane operation done by an operator and remotely operates the mobile crane so as to correspond to the crane operation; and a control unit that is configured to be communicable with a control device of the mobile crane and controls each unit of the remote operation terminal, wherein the control unit acquires attitude information of the mobile crane and current position information. of the tip of the boom of the mobile crane from the control device of the mobile crane, displays the image acquired by the imaging information acquisition unit on. the display unit, and when the crane operation is input to the crane operation unit, associates a coordinate system of a horizontal plane of the image displayed on the display unit and an operation direction of the crane operation on the basis of the position coordinates and the imaging direction of the aerial camera, and generates a control signal for remotely operating the mobile crane so that the tip of the boom displayed on the display unit moves horizontally toward the operation direction of the crane operation.

5. The remote operation terminal according to claim 4, further comprising: an unmanned aerial vehicle having the aerial camera; and an aerial vehicle operation. unit that operates the unmanned aerial vehicle, wherein the imaging information acquisition unit acquires position coordinates of the of the unmanned aerial vehicle and a direction of the aerial camera as position coordinates and an imaging direction of the aerial camera, and the display unit displays the image captured by the aerial camera of the unmanned aerial vehicle operated by the aerial vehicle operation unit.

6. A mobile crane comprising the remote operation terminal according to claim 1, wherein a boom is operated on the basis of the control signal received from the remote operation terminal.

Description

BRIEF DESCRIPTION OF DRAWINGS

[0053] FIG. 1 is a side view illustrating an overall configuration of a crane.

[0054] FIG. 2 is a block diagram illustrating a control configuration of a crane.

[0055] FIG. 3 is a plan view illustrating a schematic configuration of a first embodiment of a remote operation terminal.

[0056] FIG. 4 is a block diagram illustrating a control configuration. of the remote operation terminal.

[0057] FIGS. 5(A) and 5(B) illustrate the first embodiment of the remote operation terminal. FIG. 5(A) illustrates a state in which a three-dimensional image viewed from the side of the crane is displayed on the remote operation terminal, and FIG. 5(B) illustrates a state in which a three-dimensional image viewed from the upper front side of the crane is displayed on the remote operation terminal.

[0058] FIGS. 6(A) and 6(B) illustrate the first embodiment of the remote operation terminal. FIG. 6(A) is an enlarged view of a three-dimensional image of the crane displayed on the remote operation terminal, and FIG. 6(B) illustrates a state in which a suspended load moving operation tool of the remote operation terminal is operated.

[0059] FIG. 7 is a partially enlarged view illustrating the display device when the suspended load moving operation tool is operated in a state where the three-dimensional images of the crane and planimetric features are displayed on the remote operation terminal.

[0060] FIG. 8 is a plan view illustrating the crane controlled by the remote operation terminal operated in FIG. 7.

[0061] FIG. 9 is a plan view illustrating a schematic configuration of a second embodiment of the remote operation terminal.

[0062] FIGS. 10(A) and 10(B) illustrate the second embodiment of the remote operation terminal. FIG. 10(A) illustrates an unmanned aerial vehicle controlled by the remote operation terminal, and FIG. 10(B) illustrates a control image captured by the unmanned aerial vehicle.

DESCRIPTION OF EMBODIMENTS

[0063] Hereinafter, a crane 1 which is a mobile crane according to a first embodiment of the present invention will be described with reference to FIGS. 1 and 2. Note that while a rough terrain crane will be described in the present embodiment, the crane may be an all-terrain crane, a truck crane, a truck loader crane, or the like.

[0064] As illustrated in FIG. 1, the crane 1 is a mobile crane that can be moved to an unspecified place. The crane 1 includes a vehicle 2, a crane device 6 as a work device, a control device 31, and a remote operation terminal 32 (see FIG. 2) capable of remotely operating the crane device 6.

[0065] The vehicle 2 is a mobile body that transports the crane device 6. The vehicle 2 has a plurality of wheels 3 and travels using an engine 4 as a power source. The vehicle 2 is provided with an outrigger 5. The outrigger 5 includes an overhanging beam that can be extended by hydraulic pressure on both sides in the width direction of the vehicle 2, and a hydraulic jack cylinder that can be extended in a direction perpendicular to the ground. The vehicle 2 can expand the workable range of the crane 1 by extending the outrigger 5 in the width direction of the vehicle 2 and grounding the jack cylinder.

[0066] The crane device 6 is a device that lifts a load W with a wire rope. The crane device 6 includes a slewing base 7, a boom 9, a jib 9a, a main hook block 10, a sub hook block 11, a hydraulic derricking cylinder 12, a main winch 13, a main wire rope 14, a sub winch 15, a sub wire rope 16, a cabin 17, and the like.

[0067] The slewing base 7 is a device that slews the crane device 6. The slewing base 7 is rotatable about the center of an annular bearing. The slewing base 7 is provided with a hydraulic slewing motor 8 that is an actuator. The slewing base 7 is slewable in one and the other direction by the hydraulic stewing motor 8.

[0068] The hydraulic slewing motor 8, which is an actuator, is rotationally operated by a slewing valve 23 (see FIG. 2), which is an electromagnetic proportional switching valve. The slewing valve 23 can control the flow rate of hydraulic oil supplied to the hydraulic stewing motor 8 to an arbitrary flow rate. The slewing base 7 is provided with a slewing sensor 27 (see FIG. 2) that detects a slewing position (angle) and a slewing speed of the stewing base 7.

[0069] The boom 9 is a beam member that supports the wire rope in a state where the wire rope can lift the load W. The base end of a base boom member of the boom 9 is swing ably provided substantially at the center of the stewing base 7. The boom 9 is capable of expanding and contracting in the axial direction by moving each boom member with a hydraulic expansion/contraction cylinder (not illustrated) which is an actuator. Additionally, the boom 9 is provided with the jib 9a.

[0070] The hydraulic expansion/contraction cylinder (not illustrated) which is an actuator is expanded and contracted by an expansion/contraction valve 24 (see FIG. 2) which is an electromagnetic proportional switching valve. The boom 9 is provided with an expansion/contraction sensor 28 that detects the length of the boom 9, a weight sensor that detects the weight of the load W, and the like.

[0071] A suspended load camera 9b (see FIG. 2) is an imaging device that images the load W and planimetric features around the load W. The suspended load camera 9b is provided in a tip part of the boom 9. The suspended load camera 9b is capable of imaging the load W and planimetric features and topography around the crane 1 from vertically above the load W.

[0072] The main hook block 10 and the sub hook block 11 are members for suspending the load W. The main hook block 10 is provided with a plurality of hook sheaves around which the main wire rope 14 is wound, and a main hook 10a for suspending the load W. The sub hook block 11 is provided with a sub hook 11a for suspending the load W.

[0073] The hydraulic derricking cylinder 12 is an actuator that raises and lowers the boom 9 and holds the attitude of the boom 9. The hydraulic derricking cylinder 12 is operated to expand and contract by a derricking valve 25 (see FIG. 2) which is an electromagnetic proportional switching valve. The boom 9 is provided with a derricking sensor 29 (see FIG. 2) that detects a derricking angle of the boom 9.

[0074] The main winch 13 and the sub winch 15 spool (wind up) and feed (wind out) the main wire rope 14 and the sub wire rope 16. The main winch 13 is configured such that a main drum around which the main wire rope 14 is wound is rotated by a main hydraulic motor (not illustrated) as an actuator, and the sub winch 15 is configured such that a sub drum around which the sub wire rope 16 is wound is rotated by a sub hydraulic motor (not illustrated) as an actuator.

[0075] The main hydraulic motor is rotationally operated by a main valve 26m (see FIG. 2) which is an electromagnetic proportional switching valve. By controlling the main hydraulic motor with the main valve 26m, the main winch 13 can be operated at an arbitrary spooling and feeding speed. Similarly, by controlling the sub hydraulic motor with a sub valve 26s (see FIG. 2), which is an electromagnetic proportional switching valve, the sub winch 15 can be operated at an arbitrary spooling and feeding speed.

[0076] The cabin 17 is mounted on the slewing base 7. An unillustrated cockpit is provided. The cockpit is provided with an operation tool for operating traveling of the vehicle 2, a slewing operation tool 18 for operating the crane device 6, a derricking operation tool 19, an expansion/contraction operation tool 20, a main drum operation tool 21m, a sub drum operation tool 21s, and the like (see FIG. 2).

[0077] A communication device 22 (see FIG. 2) is a device that receives a control signal from the remote operation terminal 32 through a wide-area information. communication network or the like, and transmits control information and. the like from the crane device 6 through the wide-area information communication network or the like. The communication device 22 is provided in the cabin 17. The communication device 22 is configured to transfer a control signal or the like to the control device 31 of the crane 1, upon receipt of the control signal or the like from the remote operation terminal 32. Additionally, the communication. device 22 is configured to transfer control information from the control device 31 and a suspended load image i1 from the suspended load camera 9h to the remote operation terminal 32.

[0078] The GNSS receiver 30 (see FIG. 2), which is a position detection unit, is a receiver included in a global navigation satellite system, and is a device that receives a ranging electric wave from a satellite and detects latitude, longitude, and altitude, which are position coordinates of the receiver. The GNSS receiver 30 is provided at the tip of the boom 9 and in the cabin 17 (hereinafter, GNSS receivers provided at tip of boom 9 and in cabin 17 are collectively referred to as “GNSS receiver 30”). That is, the crane 1 can acquire position coordinates of the tip of the boom 9 and position coordinates of the cabin 17 with the GNSS receiver 30. Additionally, the crane 1 can detect the arrangement of the boom 9 with respect to the tip (position of load W) of the boom 9 with the GNSS receiver 30. Note that while the crane 1 detects the tip position of the boom 9 using the GNSS receiver 30 in the following embodiment, the crane 1 may detect the tip position of the boom 9 using known position detection means such as a fired point camera or near field wireless communication.

[0079] As illustrated in FIG. 2, the control device 31 is a device that controls an actuator of the crane I through each operation. valve. The control device 31 is provided in the cabin 17. Substantially, the control device 31 may be configured such that a CPU, a ROM, a RAM, an HDD, and the like are connected by a bus, or may be configured of a one-chip LSI and the like. The control device 31 stores various programs and data in order to control the operation of each actuator, switching valve, sensor, and the like.

[0080] The control device 31 is connected to the suspended load camera 9b, the slowing operation tool 18, the derricking operation tool 19, the expansion/contraction operation tool 20, the main drum operation tool 21m, and the sub drum operation tool 21s, can acquire the suspended load image i1 of the suspended load camera 9h, and can acquire the operation amount of each of the stewing operation tool 18, the derricking operation tool 19, the main drum operation tool 21m, and the sub drum operation tool 21s.

[0081] The control device 31 is connected to the communication device 22, can acquire a control signal from the remote operation terminal 32, and can transmit control information from the crane device 6, the suspended load image i1 from the suspended load camera 9b, and the like.

[0082] The control device 31 is connected to the slewing valve 23, the expansion/contraction valve 24, the derricking valve 25, the main valve 26m, and the sub valve 26s, and can transmit a control signal to the slewing valve 23, the derricking valve 25, the main valve 26m, and the sub valve 26s.

[0083] The control device 31 is connected to the slewing sensor 27, the expansion/contraction sensor 28, and the derricking sensor 29, and can acquire attitude information such as the slewing position of the slewing base 7, the boom length, and the derricking angle, and the weight of the load W.

[0084] The control device 31 is connected to the GNSS receiver 30, and can acquire position coordinates of the tip of the boom 9 and position coordinates of the cabin 17. Additionally, the control device 31 can calculate the arrangement of the boom 9 with respect to the tip (load W) of the boom 9 from the acquired position coordinates of the tip of the boom 9 and position coordinates of the cabin 17.

[0085] The control device 31 generates a control signal corresponding to each operation tool on the basis of the operation amount of the slewing operation tool 18, the derricking operation tool 19, the main drum operation tool 21m, and the sub drum operation tool 21s.

[0086] The crane 1 configured as described above can move the crane device 6 to an arbitrary position by causing the vehicle 2 to travel. Additionally, the crane 1 can expand the lifting range and the work radius of the crane device 6 by causing the boom 9 to stand at an arbitrary derricking angle by operating the hydraulic derricking cylinder 12 with the derricking operation tool 19 and extending the boom 9 to an arbitrary boom length by operating the expansion/contraction operation tool 20. Additionally, the crane 1 can lift the load W by the sub drum operation tool 21s or the like, and can convey the load W by slewing the slewing base 7 by operating the slewing operation tool 18.

[0087] Next, the remote operation terminal 32 that remotely operates the crane 1 will be described with reference to FIGS. 3 to 5(B).

[0088] As illustrated in FIG. 3, the remote operation terminal 32 is a device used for remotely operating the crane 1. The remote operation terminal 32 includes a housing 33, a viewpoint changing operation tool 34 which is an image operation unit provided on the operation surface of the housing 33, a suspended load moving operation tool 35 which is a crane operation unit provided on the operation surface of the housing 33, a terminal-side stewing operation tool 36, a terminal-side expansion/contraction operation tool 37, a terminal-side main drum operation tool 38m, a terminal-side sub drum operation tool 38s, a terminal-side derricking operation tool 39, a display device 40 which is a display unit, a terminal-side communication device 41, a terminal-side control device 43 (see FIG. 4), and the like. The remote operation terminal 32 transmits a control signal of an operation valve of each actuator for moving the load W by operating the suspended load moving operation tool 35 or various operation tools to the crane device 6 through a wide area information communication network (the Internet or the like).

[0089] The housing 33 is a main component of the remote operation terminal 32. The housing 33 is formed in a size that can be held by the operator's hand. The housing 33 is provided with the viewpoint changing operation tool 34, the suspended load moving operation tool 35, the terminal-side slewing operation tool 36, the terminal-side expansion/contraction operation tool 37, the terminal-side main drum operation tool 38m, the terminal-side sub drum operation tool 38s, the terminal-side derricking operation tool 39, the display device 40, and the terminal-side communication device 41 (see FIGS. 2 and 4) on the operation surface.

[0090] The viewpoint changing operation tool 34 is an operation tool that receives input of an instruction to change the position of the viewpoint and the direction of the viewpoint of a three-dimensional image G displayed on the display device 40. The viewpoint changing operation tool 34 includes a rotatable operation stick protruding from the operation surface of the housing 33, and a sensor (not illustrated) that detects a rotation direction and rotation amount, and a tilt direction and tilt amount of the operation stick. With art operation of the operation stick, the viewpoint changing operation tool 34 transmits, to the terminal-side control device 43, a signal regarding the position of the viewpoint and the direction of the viewpoint of the three-dimensional image G in the virtual space displayed on the display device 40. That is, with an operation of the operation stick, the viewpoint changing operation tool 34 transmits, to the terminal-side control device 43, an operation signal regarding the displacement amount of the three-dimensional image G in the virtual space displayed on the display device 40.

[0091] The suspended load moving operation tool 35, which is a first operation unit, is an operation tool that receives input of an instruction to move the load W at an arbitrary speed in an arbitrary direction on an arbitrary horizontal plane. The suspended load moving operation tool 35 includes an operation stick standing substantially vertically from the operation surface of the housing 33 and a sensor (not illustrated) that detects a tilt direction and a tilt amount of the operation stick. The suspended load moving operation tool 35 is configured such that the operation stick can be tilted in an arbitrary direction. The suspended load moving operation tool 35 is configured to transmit, to the terminal-side control device 43, a signal regarding the angle and the tilt amount between the upward direction of the remote operation terminal 32 and the tilt direction of the operation stick detected by the sensor. The remote operation terminal 32 displays, as indicators of the tilting operation of the suspended load moving operation tool 35, an arrow Aa indicating an upward direction at the time of facing the operation surface of the housing 33, an arrow Ab indicating a rightward direction at the time of facing the operation surface, an arrow Ac indicating a downward direction at the time of facing the operation surface, and an arrow Ad indicating a leftward direction at the time of facing the operation surface.

[0092] The terminal-side slewing operation tool 36 is an operation tool that receives input of an instruction to slew the crane device 6 at an arbitrary moving speed in an arbitrary moving direction. The terminal-side slewing operation tool 36 includes an operation stick standing substantially vertically from the operation surface of the housing 33 and a sensor (not illustrated) that detects a tilt direction and a tilt amount of the operation stick.

[0093] The terminal-side expansion/contraction operation tool 37 is an operation tool that receives input of an instruction to expand and contract the boom 9 at an arbitrary speed. The terminal-side expansion/contraction operation tool 37 includes an operation stick standing from the operation surface of the housing 33 and a sensor (not illustrated) that detects a tilt direction and a tilt amount of the operation stick.

[0094] The terminal-side main drum operation tool 38m is an operation tool that receives input of an instruction to rotate the main winch 13 at an arbitrary speed in an arbitrary direction. The terminal-side main drum operation tool 38m includes an operation stick standing from the operation surface of the housing 33 and a sensor (not illustrated) that detects a tilt direction and a tilt amount of the operation stick. The terminal-side sub drum operation tool 38s is similarly configured.

[0095] The terminal-side derricking operation tool 39 is an operation tool that receives input of an instruction to raise or lower the boom 9 at an arbitrary speed. The terminal-side derricking operation tool 39 includes an operation stick standing from the operation surface of the housing 33 and a sensor (not illustrated) that detects a tilt direction and a tilt amount of the operation stick.

[0096] The display device 40 is a display device 40 that displays various information such as attitude information of the crane 1 and information of the load W. The display device 40 includes an image display device such as a liquid crystal screen. The display device 40 is provided on the operation surface of the housing 33. The display device 40 displays the three-dimensional image G of the crane 1 arranged in the virtual space and the three-dimensional image G of a ground surface, planimetric features, and the like within the movable range of the crane 1 (hereinafter, both images are collectively referred to as “three-dimensional image G”). Note that while the display device 40 is provided in the remote operation terminal 32 in the present embodiment, the display device 40 may he a monitor separated from the remote operation terminal 32.

[0097] Moreover, the display device 40 displays, as indicators of the tilting direction of the suspended load moving operation tool 35, an arrow Aa in an upward direction at the time of facing the operation surface of the housing 33, an arrow Ab indicating a rightward direction at the time of facing the operation surface, an arrow Ac indicating a downward direction at the time of facing the operation surface, and an arrow Ad indicating a leftward direction at the time of facing the operation surface.

[0098] As illustrated in FIG. 4, the terminal-side communication device 41 is a device that receives control information and the like of the crane device 6 through a wide-area information communication network or the like, and transmits control information and the like from the remote operation terminal 32 to the crane device 6. The terminal-side communication device 41 is provided. inside the housing 33. The terminal-side communication device 41 is configured to transmit the suspended load image i1 (see FIG. 7), the control signal, and the like to the terminal-side control device 43, upon receipt of the suspended load image i1, a control signal, and the like from the crane device 6. Additionally, the terminal-side communication device 41 is configured to transmit control information from the terminal-side control device 43, from the wide-area information communication network or the like to the crane device 6 through the communion device 22 of the crane 1.

[0099] The terminal-side control device 43 as a control unit is a device that controls the remote operation terminal 32. The terminal-side control device 43 is provided inside the housing 33 of the remote operation terminal 32. Substantially, the terminal-side control device 43 may be configured such that a CPU, a ROM, a PJM, an HDD, and the like are connected by a bus, or may be configured of a one-chip LSI and the like. The terminal-side control device 43 includes a three-dimensional information acquisition unit 42. The terminal-side control device 43 stores various programs and data in order to control operations of the suspended load moving operation tool 35, the viewpoint changing operation tool 34, the terminal-side slewing operation tool 36, the terminal-side expansion/contraction operation tool 37, the terminal-side main drum operation tool 38m, the terminal-side sub drum operation tool 38s, the terminal-side derricking operation tool 39, the display device 40, the terminal-side communication device 41, and the like.

[0100] The terminal-side control device 43 is connected to the suspended load moving operation tool 35, the terminal-side stewing operation tool 36, the terminal-side expansion/contraction operation tool 37, the terminal-side main drum operation tool 38m, the terminal-side sub drum operation tool 38s, and the terminal-side derricking operation tool 39, and can acquire an operation signal including a tilt direction and a tilt amount of the operation stick of each operation tool. Additionally, the terminal-side control device 43 is connected to the viewpoint changing operation tool 34, and can acquire an operation signal including the rotation direction and rotation amount, and the tilt direction and tilt amount input to the viewpoint changing operation tool 34.

[0101] The terminal-side control device 43 is connected to the terminal-side communication device 41, and can transmit and receive various types of information to and from the control device 31 through the communication device 22 of the crane device 6 connected through the wide-area information communication network. Specifically, the terminal-side control device 43 can acquire, from the control device 31, attitude information of the crane 1, the suspended load image i1 captured by the suspended load camera 9b, the movable range of the crane 1, and the current position of the tip of the boom 9.

[0102] The three-dimensional information acquisition unit 42 of the terminal-side control device 43 can acquire three-dimensional information of the crane 1 (e.g., information on three-dimensional shape of crane 1), three-dimensional information of a planimetric feature within a movable range of the crane 1 (e.g., information on three-dimensional shape of planimetric feature), and the like from the control device 31 of the crane 1 or a building information modeling (BIM) such as a management server (not illustrated) through the wide-area information communication network.

[0103] The terminal-side control device 43 can generate a control signal of a corresponding one of the slewing valve 23, the expansion/contraction valve 24, the derricking valve 25, the main valve 26m, and the sub valve 26s from the operation signal of each operation stick acquired from each sensor of the terminal-side slewing operation tool 36, the terminal-side expansion/contraction operation tool 37, the terminal-side main drum operation tool 38m, the terminal-side sub drum operation tool 38s, and the terminal-side derricking operation tool 39.

[0104] The terminal-side control device 43 can generate the three-dimensional image G of the crane 1 in the attitude of the current crane 1 and planimetric features within the movable range of the current crane 1, on the basis of the attitude information of the crane 1, the movable range of the crane 1 and current position of the tip of the boom 9, the three-dimensional information of the crane 1 and three-dimensional information of planimetric features within the movable range of the crane 1, and the like acquired from the control device 31 of the crane 1.

[0105] The terminal side control device 43 is connected to the display device 40, and can cause, the display device 40 to display the three-dimensional image G of the crane 1 in the attitude of the current crane 1 and the planimetric features within the movable range of the current crane 1, the suspended load image i1 from the suspended load camera 9b, and various types of information acquired from the crane 1. Additionally, the terminal-side control device 43 can display the three-dimensional image G displayed on the display device 40 as a three-dimensional image G from an arbitrary viewpoint position and viewpoint direction in synchronization with the rotation direction and rotation amount and the tilt direction and tilt amount input to the viewpoint changing operation tool 34.

[0106] The terminal-side control device 43 generates a control signal of the crane 1 when the suspended load moving operation tool 35 is tilted, with reference to a display state which is a position and an orientation on a screen of the three-dimensional image S of the crane 1 in the virtual space displayed on the display device 40. For example, when the suspended load moving operation tool 35 is tilted in the direction of the arrow Aa or the direction of the arrow Ac (hereinafter simply referred to as “vertical direction of remote operation terminal 32”), which is the vertical direction of the remote operation terminal 32, the terminal-side control device 43 generates a control signal of the crane 1 in which the tip of the boom 9 is moved horizontally as well as in the vertical direction at the time of facing the operation surface of the housing 33 in the virtual space. Additionally, when the suspended load moving operation tool 35 is tilted in the direction of the arrow Ab or the direction of the arrow Ad (hereinafter simply referred to as “left-right direction of remote operation terminal 32”), which is the left-right direction at the time of facing the operation surface of the housing 33, the terminal-side control device 43 generates a control signal of the crane 1 in which the tip of the boom 9 is moved horizontally and in the left-right direction at the time of facing the operation surface of the housing 33 in the virtual space. As described above, when the suspended load moving operation tool 35 is tilted in an arbitrary direction, the terminal-side control device 43 generates a control signal of the crane 1 in which the tip of the boom 9 is moved horizontally and in the tilting direction of the suspended load moving operation tool 35 in virtual space, in the display state of the three-dimensional image P of the crane 1 displayed in virtual space.

[0107] Moreover, the terminal-side control device 43 generates a control signal of the crane 1, and. also moves the tip of the boom 9 in the three-dimensional image G of the crane 1 displayed on the display device 40 in the tilting direction of the suspended load moving operation tool 35. That is, the remote operation terminal 32 can confirm the operating state of the crane 1, in advance, by the movement of the three-dimensional image G of the crane 1 in the virtual space displayed on the display device 40. When a transmission operation. is given or a predetermined condition (e.g., predetermined time elapses from generation of control signal) is satisfied for a generated control signal, the terminal-side control device 43 transmits the control signal to the control device 31 of the crane 1.

[0108] As illustrated in FIGS. 5 (A) and 5(B), the terminal-side control device 43 (see FIG. 4) displays, on the display screen, the three-dimensional image G rotated and moved on the basis of an operation signal related to the rotation direction and rotation amount and the tilt direction and tilt amount acquired from the rotated and tilted viewpoint changing operation tool 34. In the present embodiment, a forward direction of the crane 1 is defined as an X-axis direction, a direction perpendicular to the forward direction of the crane 1 is defined as a Y direction, and the vertical direction of the crane 1, which is a direction perpendicular to the ground surface in the virtual space, is defined as a Z direction.

[0109] As illustrated in FIG. 5(A), the terminal-side control device 43 causes the display device 40 to display the three-dimensional image G in the virtual space on the basis of attitude information of the crane 1, the movable range of the crane 1, the current position of the tip of the boom 9 acquired from the control device 31 of the crane 1, and the three-dimensional information of the crane 1 and the three-dimensional information of planimetric features within the movable range of the crane 1, and the like acquired from the BIM. In the present embodiment, the display device 40 displays the three-dimensional image G of the crane 1 in which the direction. of the line of sight is the direction along the Y-axis at the position of the viewpoint from the left side in the forward direction of the crane 1.

[0110] As illustrated in FIG. 5(B), when the viewpoint changing operation tool 34 is rotated and tilted, the terminal-side control device 43 rotates and moves the three-dimensional image G of the crane 1 displayed on the display device 40 on the basis of the rotation direction and rotation amount and the tilt direction and tilt amount of the viewpoint changing operation tool 34. When the viewpoint changing operation tool 34 is rotated counterclockwise and tilted downward with respect to the remote operation terminal 32, the terminal-side control device 43 rotates the three-dimensional image G of the crane 1 displayed on the display device 40 in the state of FIG. 5(A) counterclockwise about the Z-axis, and displays the state of FIG. 5(B) rotated counterclockwise about the X-axis on the display device 40.

[0111] As illustrated in FIGS. 6(A) and 6(B), the terminal-side control device 43 (see FIG. 4) generates a control signal of the crane 1 on the basis of the operation signal related to the tilting direction and tilting amount acquired from the tilted suspended load moving operation tool 35, and simultaneously moves the tip of the boom 9 in the three-dimensional image G of the crane 1 in the tilting direction of the suspended load moving operation tool 35. When the suspended load moving operation tool 35 is tilted in the direction of the arrow Aa (see FIG. 6(A)), which is the upward direction of the remote operation terminal 32, the terminal-side control device 43 moves the tip of the boom 9 in the three-dimensional image G of the crane 1 displayed on the display device 40 horizontally as well as in a direction perpendicular to the operation surface of the remote operation terminal 32 in the virtual space from the display state (see FIG. 6(B)). That is, the terminal-side control device 43 moves the tip of the boom 9 in the display state of the three-dimensional image G of the crane 1 in the upward direction of the remote operation terminal 32 which is the same direction as the tilting direction of the suspended load moving operation tool 35 on the screen of the display device 40 (see white arrow). In the crane 1 in the virtual space, the tip of the, boom 9 moves toward the −X-axis direction which is the backward direction as well as the Y-axis which is the left direction in the forward direction.

[0112] Hereinafter, control of the crane 1 by the remote operation terminal 32 will be described with reference to FIGS. 7 and 8. In the present embodiment, assume that the crane 1 is disposed at a position away to such an extent that the operator who operates the remote operation terminal 32 cannot visually recognize the crane 1. Assume that the crane 1 and the remote operation terminal 32 are communicable through the wide-area information communication network. As a result, the crane 1 can be operated by the remote operation. terminal 32 regardless of whether it is inside or outside the country as long as the crane 1 can be connected to the wide-area information communication network. Additionally, also assume that the remote operation terminal 32 acquires attitude information of the crane 1, the movable range of the crane 1, and the current position of the tip of the boom 9 of the crane 1 from the control device 31 of the crane 1. Moreover, assume that the remote operation terminal 32 acquires three-dimensional information of the crane 1 and three-dimensional information of planimetric features and the like of the work site from a BIM included in a server or the like (not illustrated).

[0113] As illustrated in FIG. 7, the display device 40 of the remote operation terminal 32 displays the three-dimensional image G of the crane 1 and planimetric features within a movable range of the crane 1. The three-dimensional image P displayed on the display device 40 is rotated and moved in an arbitrary viewpoint position and viewpoint direction by the viewpoint changing operation tool 34.

[0114] When the suspended load moving operation tool 35 is tilted in the direction of the arrow Aa (see arrow in FIG. 6(A)), which is the upward direction of the remote operation terminal 32, the terminal-side control device 43 (see FIG. 4) generates a control signal for moving the tip of the boom 9 horizontally as well as in a vertical direction at the time of facing the operation surface of the housing 33 in the virtual space in the display state of the three-dimensional image G of the crane 1 displayed on the display device 40. At the same time, the terminal-side control device 43 moves the tip of the boom 9 in the three-dimensional image G of the crane 1 horizontally as well as in a vertical direction at the time of facing the operation surface of the housing 33 in the virtual space (see white arrow). When the transmission operation of the generated control signal is performed, the terminal-side control device 43 transmits the control signal to the control device 31 (see FIG. 2) of the crane 1.

[0115] When the terminal-side main drum operation tool 38m or the terminal-side sub drum operation tool 38s is operated in the vertical direction of the remote operation terminal 32, the terminal-side control device 43 winds up or winds out the main wire rope 14 or the sub wire rope 16. That is, regardless of the display state or the three-dimensional image G of the crane 1 displayed on the display device 10, when the terminal-side main drum operation tool 38m or the terminal-side sub drum operation tool 38s is operated in the upward direction of the remote operation terminal 32, the terminal-side control device 43 generates a control signal so that the load W moves in a direction away from the ground. Additionally, when the terminal-side main drum operation tool 38m or the terminal-side sub drum operation tool 38s is operated in the downward direction of the remote operation terminal 32, the terminal-side control device 43 generates a control signal so that the load W moves in a direction approaching the ground.

[0116] As illustrated in FIG. 8, the control device 31 (see FIG. 2) of the crane 1 acquires a control signal from the terminal-side control device 43 (see FIG. 4) of the remote operation terminal 32 using a wide-area information communication network or the like. The control device 31 controls each valve of each actuator on the basis of the acquired control signal. The crane 1 moves the tip of the boom 9 so as to correspond to the movement of the tip of the boom 9 in the display state of the three-dimensional image G of the crane 1 of the display device 40 illustrated in FIG. 7 (see white arrow).

[0117] With this configuration, the remote operation terminal 32 (see FIG. 4) generates the control signal so that the tip of the boom 9 in the display state of the three-dimensional image G (see FIG. 7) of the crane 1 in the virtual space displayed from an arbitrary viewpoint and line-of-sight direction moves in the tilting direction. of the suspended load moving operation tool 35 (i.e., operating direction of crane operation). Hence, the remote operation of the crane 1 is intuitively performed without being conscious of the positional relationship of the viewpoint with respect to the crane 1. Additionally, since the control signal of the crane 1 is generated on the basis of the movement of the crane 1 using the three-dimensional image G in the virtual space, the crane 1 is operated consideration of the surrounding situation and operation conditions. As a result, an erroneous operation at the time of remote operation of the crane 1 can be prevented, and the remote operation of the crane 1 can be performed in an easy and simple manner.

[0118] Note that as illustrated in FIG. 7, the display device 40 of the remote operation terminal 32 may display the suspended load image i1 captured by the suspended load camera 9b (see FIG. 2) of the crane 1. The terminal-side control device 43 causes the display device 40 to display the suspended load image i1 acquired from the control device 31 of the crane 1 such that the suspended load image i1 is rotated by a rotation angle around the Z-axis of the three-dimensional image G of the crane 1.

[0119] With this configuration, information of a planimetric feature or the like that is not reproduced in the three-dimensional image G in the virtual space is displayed by the suspended load image i1. As a result, an erroneous operation at the time of remote operation of the mobile crane 1 can be prevented, and the remote operation of the mobile crane 1 can be performed in an easy and simple manner.

[0120] Next, a crane 1 according to the present invention and a remote operation terminal 32 as a second embodiment of the remote operation terminal of the crane 1 will be described with reference to FIGS. 9, 10(A), and 10(B). Note that the crane 1 and the remote operation terminal 32 according to the following embodiment are applied in place of the crane 1 and the remote operation terminal 32 to the crane 1 and the remote operation terminal 32 illustrated in FIGS. 1 to 8, and the names, figure numbers, and reference numerals used in the description are used to indicate the same components. In the following embodiment, points similar to those of the previously described embodiment will not be specifically described, and differences will be mainly described.

[0121] As illustrated in FIG. 9, the remote operation terminal 32 is a device used for remotely operating the crane 1. The remote operation terminal 32 includes a housing 33, a camera position change operation tool 44 which is an aerial vehicle operation unit, a suspended load moving operation tool 35 of the housing 33, a terminal-side slewing operation tool 36, a terminal-side expansion/contraction operation tool 37, a terminal-side main drum operation tool 38m, a terminal-side sub drum operation tool 38s, a terminal-side derricking operation tool 39, a display device 40, a terminal-side communication device 41, an unmanned aerial vehicle 45, an aerial camera 46, a terminal-side control device 47, and the like.

[0122] The camera position change operation tool 44 is an operation tool that receives input of an operation of the unmanned aerial vehicle 45 capturing an image displayed on the display device 40. That is, the camera position change operation tool 44 is an operation tool that changes the position coordinates and the imaging direction of the aerial camera 46 by operating the unmanned aerial vehicle 45. The camera position change operation tool 44 includes a rotatable operation stick protruding from an operation surface of the housing 33, and a sensor (not illustrated) that detects a rotation direction and rotation amount, and a tilt direction and tilt amount of the operation stick.

[0123] With an operation of the operation stick, the camera position change operation tool 44 transmits, to the terminal-side control device 47, a signal regarding the position coordinates of the unmanned aerial vehicle 45 holding the aerial camera 46 and the direction of the camera. That is, with an operation of the operation stick, the camera position change operation tool 44 transmits, to the terminal-side control device 47, a signal regarding the camera coordinates and the displacement amount in the photographing direction of a control image i2, which is an image from the aerial camera 46 displayed on the display device 40.

[0124] The display, device 40 is a display device 40 that displays various information such as attitude information of the crane 1 and information of the load W. The display device 40 includes an image display device such as a liquid crystal screen. The display device 40 is provided on the operation surface of the housing 33. The display device 40 displays the control image i2 of the crane 1 and the ground surface, the ground object, and the like within. the movable range of the crane 1 captured by the aerial camera 46.

[0125] The terminal-side communication device 41 is a device that receives control information of a crane device 6, the unmanned aerial vehicle 45, and the aerial camera 46, the control image i2 from the aerial camera 46, and the like through a wide-area information communication network or the like, and transmits the control information and the like from the remote operation terminal 32 to the crane device 6 and the unmanned aerial vehicle 45. Upon receipt of a suspended load image i1 from the crane device 6, the control image i2 from the aerial camera 46, control signals of the crane device 6 and the unmanned aerial vehicle 45, and the like, the terminal-side communication device 41 transmits the suspended load image i1, the control image i2, the control signals, and the like to the terminal-side control device 47. Additionally, the terminal-side communication device 41 transmits control information from the terminal-side control device 47 to the crane device 6 or to the unmanned aerial vehicle 45 through the communication device 22 of the crane 1 from the wide-area information communication network or the like.

[0126] The unmanned aerial vehicle 45 is a moving device that transports the aerial camera 46. The unmanned aerial vehicle 45 has a plurality of rotary blades 45a (see FIG. 10(A)). The unmanned aerial vehicle 45 can fly while maintaining an arbitrary attitude or stop flight (hover) by controlling the plurality of rotary blades 45a. The unmanned aerial vehicle 45 can move to an arbitrary position and stop by a control signal from the remote operation terminal 32. The unmanned aerial vehicle 45 is provided with an aerial vehicle-side GNSS receiver 45b (see FIG. 9) and the aerial camera 46. The unmanned aerial vehicle 45 can acquire position coordinates of the unmanned aerial vehicle 45 with the aerial vehicle-side GNSS receiver 45b. The unmanned aerial vehicle 45 can image an object from camera coordinates which are position coordinates of the unmanned aerial vehicle 45 and an imaging direction by stop flight.

[0127] The terminal-side control device 47 as a control unit is a device that controls the remote operation terminal 32. The terminal-side control device 47 includes an imaging information acquisition unit 48. The terminal-side control device 47 stores various programs and data in order to control operations of the suspended load moving operation tool 35, the terminal side slewing operation tool 36, the terminal-side expansion/contraction operation tool 37, the terminal-side main drum operation tool 38m, the terminal-side sub drum operation tool 38s, the terminal-side derricking operation tool 39, the display device 40, the terminal-side communication device 41, the camera position change operation tool 44, and the like.

[0128] The terminal-side control device 47 is connected to the terminal-side communication device 41, and can transmit and receive various types of information to and from the unmanned aerial vehicle 45 connected through a wide-area information communication network or the like. Specifically, the terminal-side control device 47 can acquire the camera coordinates, the imaging direction, and the control image i2 captured by the aerial camera 46 from the unmanned aerial vehicle 45.

[0129] The terminal-side control device 47 is connected to the display device 40, and can cause the display device 40 to display the control image 12 in which the aerial camera 46 captures the current crane 1 and surrounding planimetric features or the like from an arbitrary capturing direction at arbitrary camera coordinates.

[0130] The terminal-side control device 47 generates a control signal of the crane 1 when the suspended load moving operation tool 35 is tilted, with reference to a display state which is a position and an orientation on a screen of the control image i2 of the crane 1 displayed on the display device 40. Specifically, when the suspended load moving operation tool 35 is tilted in the vertical direction of the remote operation terminal 32, the terminal-side control device 47 generates a control signal of the crane 1 in which the tip of the boom 9 moves horizontally as well as in a direction corresponding to the vertical direction at the time of facing the operation surface of the housing 33 from the display state of the control image i2. Additionally, when the suspended load moving operation tool 35 is tilted the left-right direction of the remote operation terminal 32, the terminal-side control device 47 generates a control signal of the crane i in which the tip of the boom 9 moves horizontally as well as in a direction corresponding to the left-right direction at the time of facing the operation surface of the housing 33 from the display state of the control image i2. That is, the terminal-side control device 47 generates a control signal in which the tip of the boom 9 in the display state of the control image i2 of the crane 1 moves in a direction corresponding to the same direction as the tilting direction of the suspended load moving operation tool 35 on the screen of the display device 40.

[0131] As illustrated in FIG. 10(A), the unmanned aerial vehicle 45 performs stop flight at an arbitrary positron by a control signal from. the remote operation terminal 32. The unmanned aerial vehicle 45 transmits the position coordinates and the attitude during the stop flight to the remote operation terminal 32 as camera coordinates and an imaging direction. At the same time, the unmanned aerial vehicle 45 transmits the control image i2 captured by the aerial camera 46 to the remote operation terminal 32.

[0132] As illustrated in FIG. 10(B), the control image i2 captured by the aerial camera 46 is displayed on. the display device 40 of the remote operation terminal 32. The control image i2 displayed on the display device 40 is captured from an arbitrary capturing direction at arbitrary camera coordinates by the camera position change operation tool 44. The terminal-side control device 47 of the remote operation terminal 32 generates a control signal for moving the tip of the boom 9 of the crane 1 in a direction corresponding to the tilting direction of the suspended load moving operation tool 35 on the screen of the display device 40.

[0133] With this configuration, the terminal-side control device 47 of the remote operation terminal 32 generates the control signal so that the tip of the boom 9 of the crane 1 displayed on the display device 40 moves in the direction corresponding to the tilting direction of the suspended load moving operation tool 35 in the display state. Hence, the remote operation of the crane 1 is intuitively performed without being conscious of the positional relationship between the camera coordinates of the aerial camera 46, which is the viewpoint for the crane 1, and the imaging direction. As a result, an erroneous operation at the time of remote operation of the crane 1 can be prevented, and the remote operation of the crane 1 can be performed is an easy and simple manner.

[0134] In the present embodiment, the remote operation terminal 32 acquires the control image i2 from the aerial camera 46 of one unmanned aerial vehicle 45. However, the remote operation terminal 32 may acquire the control image i2 from the aerial camera 46 of each of a plurality of unmanned aerial vehicles 45. The remote operation terminal 32 can arbitrarily switch among a plurality of control images i2 captured from different capturing directions at different camera coordinates and use the control images i2 for controlling the crane 1. With this configuration, it is not necessary to move the unmanned aerial vehicle 45 each time, so that the remote operation of the crane 1 can be performed in an easy and simple manner. Additionally, instead of the aerial camera 46 included in the unmanned aerial vehicle 45, an image from a ground surface camera installed at an arbitrary position may be used.

[0135] The above-described embodiments are merely representative forms, and various modifications can be made without departing from the gist of one embodiment. It is a matter of course that the present invention can be implemented in various other forms. The scope of the present invention is indicated by the description of the claims, and further includes equivalents of the description of the claims and all modifications within the scope.

[0136] The entire disclosure of the specification, drawings, and abstract included in Japanese Patent Application No. 2019 096382 filed on May 22, 2019 is incorporated herein by reference.

REFERENCE SIGNS LIST

[0137] 1 crane [0138] 6 crane device [0139] 9 boom [0140] 30 GNSS receiver [0141] 31 control device [0142] 32 remote operation terminal [0143] 34 viewpoint changing operation tool [0144] 35 suspended load moving operation tool [0145] 40 display device [0146] 41 terminal-side communication device [0147] 43 terminal-side control device [0148] 44 camera position change operation tool [0149] 45 unmanned aerial vehicle [0150] 46 aerial camera [0151] 47 terminal-side control device