Method and system for marine loading, computer readable medium and computer program for a marine loading system

Abstract

A method for marine loading includes: providing a marine loading system including: a marine loading arm, a fluid transfer line provided with a coupling device, an actuator being adapted to move the marine loading arm and to position the coupling device and a imaging system for capturing a plurality of images; receiving image data associated with the plurality of images of a scene; constructing, based on the image data, a representation of an object in the scene; determining whether the object represents an open target duct being present in the scene; determining an estimated position of the open target duct with respect to the coupling device or the imaging system; constructing a 3D representation of the open target duct; determining a characteristic property of the open target duct; determining whether the coupling device is compatible with the open target duct; and moving the coupling device towards the open target duct.

Claims

1. A method for marine loading, the method comprising: providing a marine loading system comprising: a marine loading arm comprising: at least one fluid transfer line provided with a coupling device; at least one actuator being adapted to move the marine loading arm and to position the coupling device; an imaging system configured to capture a plurality of images, wherein the position and orientation of the imaging system with respect to the coupling device is determinable; and a) receiving, from the imaging system, image data associated with the plurality of images of a scene; b) constructing, based on the image data, a representation of an object in the scene; c) determining, based on matching the representation of the object in the scene with a plurality of predetermined representations comprising representations of open target ducts, whether the object represents an open target duct being present in the scene; d) determining, if it is determined that the open target duct is present in the scene, an estimated position of the open target duct with respect to the coupling device; e) constructing, based on the image data, a 3D representation of the open target duct; f) determining a characteristic property of the open target duct, based on matching the 3D representation of the open target duct with a plurality of predetermined 3D representations comprising 3D representations of open target ducts with predetermined characteristic properties; g) determining, based on the determined characteristic property of the open target duct, whether the coupling device is compatible with the open target duct, wherein a coupling device is compatible with the open target duct if the coupling device can be connected to the open target duct; and h) moving, if the coupling device is compatible with the open target duct, the coupling device towards the open target duct, based on the estimated position of the open target duct with respect to the coupling device.

2. The method according to claim 1, wherein the characteristic property of the open target duct comprises information on a diameter of the open target duct.

3. The method according to claim 1, further comprising: determining, after step c and before step d, based on matching the representation of the object in the scene with a plurality of predetermined representations comprising representations of closed target ducts, whether the object represents a closed target duct being present in the scene.

4. The method according claim 1, further comprising: indicating whether the property of the open target duct is determined; and/or indicating whether the coupling device is compatible with the open target duct.

5. The method claim 1, further comprising: repeating steps b to e for every object in the scene; indicating a number of open target ducts that are present in the scene; and selecting, if the number of open target ducts is greater than one, the open target duct to which the coupling device should connect.

6. The method according to claim 1, further comprising: receiving, after step h, detailed image data associated to least one detailed image of the open target duct; and determining whether the open target duct has defects, based on the detailed image data and on matching the detailed image data with image defect data associated to a plurality of predetermined images of open target ducts with defects and a plurality of predetermined images of open target ducts without defects.

7. The method according to claim 1, further comprising: moving the coupling device at a first height towards the open target duct and then transitioning the coupling device to a second height, wherein the first height is higher than the second height, and wherein the second height is substantially the same height as the height of the open target duct.

8. The method according to claim 1, further comprising: moving, if it is determined that the open target duct is present in the scene, the imaging system along a trajectory such that the open target duct remains present in the scene.

9. The method according to claim 1, further comprising: moving the imaging system together with the coupling device, wherein the relative position of the imaging system with respect to the coupling device remains substantially the same.

10. The method according to claim 1, wherein the open target duct comprises a flange.

11. The method according to claim 10, wherein the characteristic property of the open target duct comprises information on a plurality of holes in the flange.

12. The method according to claim 1, further comprising: moving, if the estimated position of the open target duct has been determined, the imaging system along a trajectory such that the orientation of the open target duct with respect to the imaging system changes; capturing additional images of the scene, while the imaging system moves; and improving, based on the additional images of the scene captured, a representation accuracy of the representation or the 3D representation of the open target duct.

13. The method according to claim 12, further comprising: improving, based on the 3D representation of the open target duct with the improved representation accuracy, a position accuracy of the estimated position of the open target duct with respect to the coupling device or the imaging system.

14. A marine loading system comprising: a marine loading arm comprising: at least one fluid transfer line provided with a coupling device; at least one actuator being adapted to move the marine loading arm and to position the coupling device; an imaging system configured to capture a plurality of images, wherein the position and orientation of the imaging system with respect to the coupling device is determinable; and a control system operatively connected to the at least one actuator and to the imaging system, wherein the control system is configured to: a) receive, from the imaging system, image data associated to the plurality of images of a scene; b) construct, based on the image data, a representation of an object in the scene; c) determine, based on matching the representation of the object in the scene with a plurality of predetermined representations comprising representations of open target ducts, whether the object represents an open target duct being present in the scene; d) determine, if the open target duct is present in the scene, an estimated position of the open target duct with respect to the coupling device; e) construct, based on the image data, a 3D representation of the open target duct; f) determine a characteristic property of the open target duct, based on matching the 3D representation of the open target duct with a plurality of predetermined 3D representations comprising 3D representations of open target ducts with predetermined characteristic properties; g) determine, based on the determined characteristic property of the open target duct, whether the coupling device is compatible with the open target duct, wherein a coupling device is compatible with the open target duct if the coupling device can be connected to the open target duct; and h) control, if the coupling device is compatible with the open target duct, the at least one actuator to move the coupling device towards the open target duct, based on the estimated position of the open target duct with respect to the coupling device.

15. The marine loading system according to claim 14, wherein the characteristic property of the open target duct comprises information on a diameter of the open target duct.

16. The marine loading system according to claim 14, wherein the open target duct comprises a flange.

17. The marine loading system according to claim 16, wherein the characteristic property of the open target duct comprises information on a plurality of holes in the flange.

18. The marine loading system according to claim 14, wherein the control system is further configured to: determine, after step c and before step d, based on matching the objects in the scene with a plurality of predetermined representations comprising representations of closed target ducts, whether the object represents a closed target duct being present in the scene.

19. The marine loading system according to claim 14, wherein the control system further is configured to: generate a first signal, indicating whether the property of the open target duct is determined; and/or generate a second signal, if the property of the open target duct is determined, the second signal indicating whether the coupling device is compatible with the open target duct.

20. The marine loading system according to claim 14, wherein the control system further is configured to: repeat steps b to e for every object in the scene; generate a third signal, indicating a number of open target ducts that are present in the scene; and select, if the number of open target ducts is greater than one, the open target duct to which the coupling device should connect.

21. The marine loading system according to claim 14, wherein the control system further is configured to: receive, after step h, a detailed image data associated to at least one detailed image of the open target duct; and determine whether the open target duct has defects, based on the detailed image data and on matching the detailed image data with image defect data associated to a plurality of predetermined images of open target ducts with defects and a plurality of predetermined images of open target ducts without defects.

22. The marine loading system according to claim 14, wherein the control system further is configured to: control the at least one actuator to move the coupling device at a first height towards the open target duct and then transitioning the coupling device to a second height, wherein the first height is higher than the second height, and wherein the second height is substantially the same height as the height of the open target duct.

23. A non-transient computer readable medium containing program instructions for a control system of the marine loading system according to claim 14, to cause the control system to perform steps a to h.

24. A computer program comprising computer instructions which, when loaded in a control system of the marine loading system according to claim 14, cause the control system to perform steps a to h.

25. The marine loading system according to claim 14, further comprising means to move the imaging system.

26. The marine loading system according to claim 25, wherein the imaging system is mounted on the marine loading arm, on a drone and/or on a movable arm.

27. The marine loading system according to claim 25: wherein the control system further is configured to let, if it is determined that the open target duct is present in the scene, the imaging system move along a trajectory such that the orientation of the open target duct with respect to the imaging system changes; wherein the imaging system further is configured to capture additional images of the scene, while the imaging system moves; and wherein the control system further is configured to improve, based on the additional images of the scene captured, a representational accuracy of the representation or the 3D representation of the open target duct.

28. The marine loading system according to claim 27, wherein the control system further is configured to: improve, based on the 3D representation of the open target duct with the improved representation accuracy, a position accuracy of the estimated position of the open target duct with respect to the coupling device or the imaging system.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) FIG. 1 depicts a perspective view, partially schematic, of an embodiment of a machine loading system.

(2) FIG. 2 depicts a perspective view of an embodiment of a coupling device and a imaging system of the machine loading system in greater detail.

(3) FIG. 3 depicts a side view of a ship comprising target ducts.

(4) FIG. 4 depicts a 2D front view of an open target duct.

(5) FIG. 5 depicts a 2D front view of a closed target duct.

(6) FIG. 6 depicts a 3D perspective view of an open target duct.

(7) FIG. 7 depicts a 3D perspective view of a closed target duct.

(8) FIG. 8 depicts a first detailed 3D perspective view of an open target duct.

(9) FIG. 9 depicts a second detailed 3D perspective view of an open target duct.

(10) FIG. 10 depicts a detailed 2D front view of an open target duct without defect.

(11) FIG. 11 depicts a detailed 2D front view of an open target duct with defect.

(12) FIG. 12 illustrates a preferred trajectory of the coupling device.

(13) FIG. 13 depicts a first flow diagram of steps associated with the invention.

(14) FIG. 14 depicts a second flow diagram of steps associated with the invention.

DETAILED DESCRIPTION OF EMBODIMENTS

(15) FIG. 1 depicts an embodiment of a machine loading system 10 according to the invention. The machine loading system 10 comprises an apex 12, a balancer link 13 and a counterweight 14. The marine loading system 10 further comprises a fluid transfer line 16 which is connected to a coupling device 18 and along which a fluid such as a liquid or a gas enters or leaves the marine loading system. Actuators 17 are present to move and position the marine loading arm and its coupling device 18. An imaging system 20 is configured to capture a plurality of images of a scene and to send image data via a wired or wireless communication link transmitting a imaging signal 21, schematically indicated by a dashed line, to a control system 30. The imaging system 20 is mounted on the marine loading arm, close to the coupling device 18. Consequently, the imaging system 20 moves together with the coupling device 18, wherein the relative position of the imaging system 20 with respect to the coupling device 18 remains substantially the same. Alternatively or additionally, the imaging system 20 is (partly) mounted on a drone 80 and/or a (moveable) arm 85. The control system 30 is configured to perform a plurality of steps based on the received image data. In particular, the control system 30 is configured to process the image data and to generate an actuation signal 31, transmitted by a communication link as schematically indicated by a dashed line, to control the actuator 17 such that the marine loading arm and the coupling device 18 are moved and positioned properly.

(16) FIG. 2 depicts in greater detail the coupling device 18 of the marine loading system. In particular, FIG. 2 shows that the coupling device 18 comprises a flange 19 comprising a plurality of holes 19b. The imaging system 20 comprises a stereo camera comprising two lenses 20a, 20b. The imaging system 20 is mounted on the marine loading arm, close to the coupling device 18. The stereo camera is fastened to the fluid transfer line 16 via a camera pole 23 that is connected to a pole clamp 22. A plurality of clamps 19c are arranged to clamp an open target duct 40 comprising a flange 41, when the coupling device 18 is connecting to it. To permit proper clamping of the open target duct 40 comprising a flange 41, the clamps 19c have to be arranged at a proper distance. It is therefore important to determine whether the open target duct 40 is compatible with the coupling device 18 comprising the clamps 19c. The coupling device 18 is articulated at its end with three degrees of rotational freedom.

(17) FIG. 13 depicts a first flow diagram of steps associated with the invention.

(18) In particular, FIG. 13 depicts steps of a marine loading method for a marine loading system 10, wherein the arrows indicate an efficient order to perform the plurality of steps.

(19) In step 101, the control system 30 receives image data from imaging system 20 via an imaging signal 21 associated to the plurality of images of a scene. As shown in FIG. 3, the scene may comprise a ship 60 on which at least one open target duct 40 with flange is mounted.

(20) In step 102, the control system 30 constructs, based on the image data, a representation of an object in the scene. The representation may be a 2D or a 3D representation. An example of a non-detailed 2D representation of an open target duct is provided in FIG. 4. In particular, FIG. 4 depicts an open target duct 40, whereas FIG. 5 depicts a closed target duct, showing a cover 51. The inner circle and outer circle in FIG. 4 are related to an inner and an outer diameter of a flange 41, respectively. In case more images from various orientations are captured and/or the target duct is closer to the imaging system, a related 3D representation may be constructed. FIG. 6 depicts a 3D representation of an open target duct 40 connected to a fluid transfer line 42, whereas FIG. 7 depicts a 3D representation of a closed target duct.

(21) In step 103, the control system 30 determines, based on matching the representation of the object in the scene with a plurality of predetermined representations comprising representations of open target ducts, whether or not an open target duct 40 is present in the scene. In case it is determined that the open target duct 40 is present in the scene, it is efficient to move the imaging system 20 along a trajectory such that the open target duct 40 remains present in the scene.

(22) If, in step 103, the control system 30 has determined that an open target duct 40 is present in the scene, the flow continues with step 104.

(23) If, in step 103, the control system 30 has determined that there is no open target duct 40 present in the scene, the flow returns to step 101.

(24) In step 104, the control system 30 determines, if the open target duct 40 is present in the scene, an estimated position of the open target duct 40 with respect to the coupling device 18 or the imaging system 20. Determining the position may be based on the (3D) representation of the open target duct 40.

(25) In step 106, the control system 30 constructs, based on the image data, a 3D representation of the open target duct 40.

(26) In step 107, the control system 30 receives, via the imaging signal 21, additional images of the scene captured by the imaging system 20. During this step, the control system 30 may control the at least one actuator 17 to move the coupling device 18 towards the open target duct 40 to enable the imaging system 20 to capture more and better images of the open target duct 40. In particular it may be beneficial when the imaging system 20 moves along a trajectory such that the orientation of the open target duct 40 with respect to the imaging system 20 changes, such that the imaging system 20 may view the open target duct 40 from different angles and/or orientations.

(27) In step 108, the control system 30 improves, based on the additional images of the scene captured, an representation accuracy of the 3D representation of the open target duct 40. Although it may be clear according to FIGS. 4 and 6 whether the target duct is open or closed, the images may not be detailed enough to determine whether the coupling device 18 is compatible with the (open) target duct 40. Improving the representation accuracy of the 3D representation may continue until the 3D representation of the open target duct 40 can be matched with predetermined 3D representations of open target ducts. FIGS. 8 and 9 depict, from different viewpoints, a detailed 3D representation of the open target duct that may be used to determine characterizing properties such as its shape, inner diameter, outer diameter, diameter of holes 43 and the plurality of the holes 43.

(28) In step 109, the control system 30 tries to determine a characteristic property of the open target duct 40, based on matching the 3D representation with the improved representation accuracy of the open target duct 40 with a plurality of predetermined 3D representations comprising 3D representations of open target ducts with predetermined characteristic properties.

(29) If, in step 109, the control system 30 is unable to determine a characteristic property of the open target duct 40, the flow returns to step 107.

(30) If, in step 109, the control system 30 is able to determine a characteristic property of the open target duct 40, the flow continues with step 110.

(31) In step 110, the control system 30 determines whether a characteristic property of the open target duct 40 can be determined, based on matching the 3D representation of the open target duct 40 with a plurality of predetermined 3D representations comprising 3D representations of open target ducts with predetermined characteristic properties.

(32) In step 111, the control system 30 determines, based on the determined characteristic property of the open target duct 40, whether or not the coupling device 18 is compatible with the open target duct 40, wherein a coupling device 18 is compatible with the open target duct 40 if the coupling device 18 can be reliably connected to the open target duct 40.

(33) If, in step 111, the control system 30 has determined that the coupling device 18 is compatible with the open target duct 40, the flow continues with step 112.

(34) If, in step 111, the control system 30 has determined that the coupling device 18 is not compatible with the open target duct 40, the flow continues with step 113.

(35) In step 112, the control system 30 controls, if the coupling device 18 is compatible with the open target duct, the at least one actuator 17 via the actuation signal 31 to move the coupling device 18 towards the open target duct 40. A preferred trajectory 75 of the coupling device 18 towards the open target duct 40 is depicted in FIG. 12, wherein the coupling device avoids an obstacle 70 that may be mounted on the ship 60, in particular mounted on a deck 61 of the ship. In particular, the coupling device 18 moves at a first height h1 towards the open target duct 40 and then transitions the coupling device 18 to a second height h2, wherein the first height h1 is higher than the second height h2, and wherein the second height h2 is substantially the same height as the height of the centre of the open target duct 40. The first height h1 and the second height h2 are heights with respect to a (virtual) horizontal plane 90, that may has an identical height as the height of a water surface, a pier or a shore.

(36) In step 113, the control system 30 aborts the coupling process if in step 111 it is determined that open target duct 40 is not compatible with the coupling device 18.

(37) FIG. 14 depicts a second flow diagram of steps associated to the invention.

(38) In particular, FIG. 14 depicts steps of a marine loading method for a marine loading system 10, wherein the arrows indicate an efficient order to perform the plurality of steps. Some steps of the second flow diagram are similar to steps of the first flow diagram depicted in FIG. 13, but the flow diagram of FIG. 14 comprises a few modifications and additional steps as described in the following.

(39) In step 201, after the control system 30, in step 102, has constructed a representation of an object in the scene, the control system 30 checks whether or not for all objects in the scene a representation is constructed. This check is performed in case there may be multiple target ducts present in the scene. It may be preferable to first determine whether a ship 60 is present in the scene and thereafter whether target ducts are present on the ship 60.

(40) If, in step 201, the control system 30 has determined that not for all objects in the scene a representation is constructed, the flow returns to step 102 to construct a representation of another object.

(41) If, in step 201, the control system 30 has determined that for all objects in the scene a representation is constructed, the flow continues with step 103.

(42) In step 203, after the control system 30, in step 103, has determined that at least one open target duct 40 is present in the scene, the open target duct 40 is selected to which the coupling device 18 should connect. In case only one open target duct 40 is present in the scene, this one open target duct 40 is selected. In case multiple open target ducts 40 are present in the scene, it may be preferable to communicate the number and approximate locations of the open target ducts 40 to an operator who then decides to which open target duct 40 the coupling device 18 should connect. Alternatively, selecting the open target duct 40 may be performed by a computer, without human intervention.

(43) In step 204, after the control system 30, in step 103, has determined that no open target duct 40 is present in the scene, the control system 30 determines, based on matching the objects in the scene with a plurality of predetermined 3D representations comprising 3D representations of closed target ducts, whether or not a closed target duct 50 is present in the scene. In case a closed target duct 50 is present in the scene and/or in the case a closed target duct 50 is not present in the scene, it is preferable to continue receiving image data to continue looking for open target ducts 40.

(44) If, in step 204, the control system 30 has determined that no closed target duct 50 is present in the scene, the flow returns to step 101.

(45) If, in step 204, the control system 30 has determined that a closed target duct 50 is present in the scene, the flow continues with step 205. Additionally, the flow may return to step 101 as well.

(46) In step 205, the control system 30 informs an operator or computer that a closed target duct 50 is present in the scene, that may use this information to initiate the opening of the closed target duct 50, which may be done by removing a cover 51.

(47) In step 206, after the control system 30, in step 108, improved the representational accuracy, the control system 30 improves, based on the 3D representation of the open target duct 40 with the improved representational accuracy, a position accuracy of the estimated position of the open target duct 40 with respect to the coupling device 18. This increased accuracy of the position is beneficial to move the coupling device 18 more precise towards the open target duct 40.

(48) In step 207, after the coupling device 18, in step 112, has moved towards the open target duct 40, the control system 30 receives, via the imaging signal 21, detailed image data associated to least one detailed image of the open target duct 40. Since this image is taken when the coupling device 18 has moved relatively close to the open target duct 40, an image of the open target duct 40 can be obtained that comprises relatively fine details, such that even small defects 43 may be visible on the detailed image (see FIG. 11).

(49) In step 208, the control system 30 determines whether or not the open target duct 40 has a defect 45, based on the detailed image data and on matching the detailed image data with image defect data associated to a plurality of predetermined images of open target ducts with defects and a plurality of predetermined images of open target ducts without defects. In particular defects on the seal face 44 are important to detect.

(50) If, in step 208, the control system 30 has determined that the open target duct 40 has no defect, the flow continues with step 209.

(51) If, in step 208, the control system 30 has determined that the open target duct 40 has a defect 45, the flow continues with step 210.

(52) In step 209, the coupling device connects to the open target duct 40 after it is determined that the open target duct 40 is compatible with the coupling device 18 and that the open target duct 40 has no defects.

(53) In step 210, the control system 30 aborts the coupling process because the open target duct 40 is not compatible with the coupling device 18 or because the open target duct 40 has a defect 45.

(54) To illustrate that a characteristic property may not be determinable, holes that may be present in the flange 41 are not depicted in FIG. 4 and FIG. 6. Also, bolts that may be present in the cover 51 are not depicted in FIG. 5 and FIG. 7.

(55) As explained above, a method for marine loading is disclosed, the method comprising: providing a marine loading system comprising: a marine loading arm, at least one fluid transfer line provided with a coupling device, at least one actuator being adapted to move the marine loading arm and to position the coupling device and a imaging system configured to capture a plurality of images. The method further comprises: receiving image data associated with the plurality of images of a scene; constructing, based on the image data, a representation of an object in the scene; determining whether the object represents an open target duct being present in the scene; determining an estimated position of the open target duct with respect to the coupling device or the imaging system; constructing a 3D representation of the open target duct; determining a characteristic property of the open target duct, based on matching the 3D representation of the open target duct with a plurality of predetermined 3D representations comprising 3D representations of open target ducts with predetermined characteristic properties; determining whether the coupling device is compatible with the open target duct; and moving the coupling device towards the open target duct.

(56) As required, detailed embodiments of the present invention are disclosed herein; however, it is to be understood that the disclosed embodiments are merely exemplary of the invention, which can be embodied in various forms. Therefore, specific structural and functional details disclosed herein are not to be interpreted as limiting, but merely as a basis for the claims and as a representative basis for teaching one skilled in the art to variously employ the present invention in virtually any appropriately detailed structure. Further, the terms and phrases used herein are not intended to be limiting, but rather, to provide an understandable description of the invention.

(57) The terms a/an, as used herein, are defined as one or more than one. The term plurality, as used herein, is defined as two or more than two. The term another, as used herein, is defined as at least a second or more. The terms including and/or having, as used herein, are defined as comprising (i.e., open language, not excluding other elements or steps). Any reference signs in the claims should not be construed as limiting the scope of the claims or the invention.

(58) The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

(59) The term coupled, as used herein, is defined as connected, although not necessarily directly, and not necessarily mechanically (e.g. magnetically).

(60) A single processor or other unit, or multiple processors or other units may fulfil control the functions of several test routine steps recited in the claims.

LIST OF REFERENCE SIGNS

(61) 10 marine loading system 12 apex 13 balancer link 14 counterweight 15 base riser 16 fluid transfer line 17 actuator 18 coupling device 19 connection flange 19b hole 19c clamp 20 imaging system 20a lens 20b lens 21 imaging signal 22 pole clamp 23 camera pole 30 control system 31 actuation signal 40 open target duct 41 flange 42 fluid transfer line 43 hole 44 seal face 45 defect 50 closed target duct 51 cover 52 bolt 60 ship 61 deck 70 obstacle 75 preferred trajectory 80 drone 85 arm h1 first height h2 second height 90 horizontal plane