INFORMATION PROCESSING METHOD, INFORMATION PROCESSING DEVICE, AND MOBILE BODY CONTROL SYSTEM

Abstract

The present disclosure relates to an information processing method, an information processing device, and a mobile body control system that enable real-time notification to a user of a movement plan modified during the automatic movement of a mobile body. The mobile body control system outputs, in reference to modification information input during automatic movement of a mobile body based on a movement plan, a modified plan obtained by modifying the movement plan, and displays the modified plan during the automatic movement of the mobile body based on the modified plan. The technology according to the present disclosure can be applied, for example, to a drone system configured to allow intervention by manual operation for drones in automatic flight.

Claims

1. An information processing method comprising: outputting, in reference to modification information input during automatic movement of a mobile body based on a movement plan, a modified plan obtained by modifying the movement plan; and displaying the modified plan during the automatic movement of the mobile body based on the modified plan.

2. The information processing method according to claim 1, wherein a movement route of the mobile body included in the movement plan is modified in reference to the modification information, and the movement route modified is displayed as the modified plan.

3. The information processing method according to claim 2, wherein position information representing the movement route is modified as a whole in reference to the modification information, and the movement route represented by the position information modified as a whole is displayed as the modified plan.

4. The information processing method according to claim 2, wherein position information representing the movement route is sequentially modified in reference to the modification information, and the movement route represented by the position information sequentially modified is sequentially displayed as the modified plan.

5. The information processing method according to claim 2, wherein the movement route before modification is displayed together with the movement route modified.

6. The information processing method according to claim 5, wherein the movement route before modification includes at least one of a first movement route set before movement of the mobile body and a second movement route set during automatic movement standby of the mobile body, and at least one of the first movement route and the second movement route is displayed together with the movement route modified.

7. The information processing method according to claim 6, wherein the second movement route is the first movement route modified in reference to the modification information input during the automatic movement standby of the mobile body.

8. The information processing device according to claim 1, wherein a movement route of the mobile body included in the movement plan is modified in reference to the modification information, and a numerical value representing a modification amount of the movement route modified is displayed as the modified plan.

9. The information processing method according to claim 1, wherein the modification information includes operation information representing a modification operation for the movement plan by a user, and the modified plan modified in reference to the operation information is output.

10. The information processing method according to claim 9, wherein, in a case where the operation information has been input, the modified plan modified in reference to the operation information is output, and, in a case where the operation information has not been input, the movement plan before modification is output as it is.

11. The information processing method according to claim 9, wherein the operation information is input from a ground-based device capable of communicating with the mobile body.

12. The information processing method according to claim 1, wherein the modification information includes surrounding information representing a situation around the mobile body.

13. The information processing method according to claim 12, wherein the surrounding information includes distance information representing a certain distance to be maintained between the mobile body and an approach avoidance target, and the modified plan modified in reference to the distance information is output.

14. The information processing method according to claim 13, wherein the distance information is input in a case where the mobile body and the approach avoidance target approach each other within a threshold distance determined in advance.

15. The information processing method according to claim 14, wherein whether the mobile body and the approach avoidance target have approached each other within the threshold distance or not is determined in reference to sensor information acquired by a distance measurement sensor mounted on the mobile body.

16. The information processing method according to claim 13, wherein the modification information further includes operation information representing a modification operation for the movement plan by a user, and the modified plan modified in reference to the operation information is further modified in reference to the distance information.

17. The information processing method according to claim 1, wherein the mobile body includes a drone capable of automatic flight based on a flight plan.

18. An information processing device comprising: a plan modification unit configured to output, in reference to modification information input during automatic movement of a mobile body based on a movement plan, a modified plan obtained by modifying the movement plan; and a display control unit configured to display the modified plan during the automatic movement of the mobile body based on the modified plan.

19. A mobile body control system comprising: a mobile body; a plan modification unit configured to output, in reference to modification information input during automatic movement of the mobile body based on a movement plan, a modified plan obtained by modifying the movement plan; and a display control unit configured to display the modified plan during the automatic movement of the mobile body based on the modified plan.

Description

BRIEF DESCRIPTION OF DRAWINGS

[0011] FIG. 1 is a diagram illustrating a flow of an automatic flight of a drone.

[0012] FIG. 2 is a diagram illustrating intervention by manual operation during the automatic flight.

[0013] FIG. 3 is a diagram illustrating a display example of a flight plan on a ground-based device.

[0014] FIG. 4 is a block diagram illustrating a hardware configuration example of a mobile body control system.

[0015] FIG. 5 is a block diagram illustrating a schematic functional configuration example of the mobile body control system.

[0016] FIG. 6 is a flowchart illustrating the flow of processing of the mobile body control system.

[0017] FIG. 7 is a diagram illustrating a display example of a modified plan.

[0018] FIG. 8 is a diagram illustrating a display example of a modified plan.

[0019] FIG. 9 is a diagram illustrating a display example of a modified plan.

[0020] FIG. 10 is a diagram illustrating a display example of a modified plan.

[0021] FIG. 11 is a diagram illustrating a display example of a modified plan.

[0022] FIG. 12 is a block diagram illustrating the configuration of a drone system of a first embodiment.

[0023] FIG. 13 is a flowchart illustrating the flow of the processing of the drone system.

[0024] FIG. 14 is a block diagram illustrating a modified example of the drone system.

[0025] FIG. 15 is a block diagram illustrating the configuration of a drone system of a second embodiment.

[0026] FIG. 16 is a block diagram illustrating a first specific configuration example of the drone system.

[0027] FIG. 17 is a flowchart illustrating the flow of flight plan modification processing.

[0028] FIG. 18 is a block diagram illustrating a second specific configuration example of the drone system.

[0029] FIG. 19 is a flowchart illustrating the flow of flight plan modification processing.

[0030] FIG. 20 is a diagram illustrating an example of flight plan modification based on an approach prevention plan.

[0031] FIG. 21 is a diagram illustrating a configuration example of a computer.

DESCRIPTION OF EMBODIMENTS

[0032] Now, modes for carrying out the present disclosure (hereinafter referred to as embodiments) are described. Note that the descriptions are given in the following order. [0033] 1. Automatic Flight of Drone and Manual Operation Intervention [0034] 2. Overview of Mobile Body Control System According to Present Disclosure and Display Examples of Modified Plans [0035] 3. First Embodiment (Configuration for Achieving Visualization of Modified Plan) [0036] 4. Second Embodiment (Configuration for Achieving Flight Desired by User) [0037] 5. Configuration Example of Computer

<1. Automatic Flight of Drone and Manual Operation Intervention>

[0038] Hitherto, there has been known a technology in which a flying device such as a drone automatically flies according to a flight plan set in advance.

[0039] The flight plan includes a flight route and flight speed at which the drone is to fly. Specifically, the flight plan is set by a user specifying in advance position information (latitude and longitude), time, aircraft attitude (direction), the posture of a gimbal camera mounted on the drone, or the like on map data.

[0040] The flight plan set in such a manner is transferred to the drone, and when the user instructs the execution of automatic flight, the drone starts automatic flight. Note that automatic flight herein refers to a flight mode that enables flight without manual operation, which is called autopilot, semi-autonomous flight, fully autonomous flight, or the like.

[0041] FIG. 1 is a diagram illustrating the flow of the automatic flight of a drone.

[0042] As illustrated in FIG. 1, in Step S1, a drone DR performs preparatory flight. Specifically, the drone DR rises from a takeoff point to a predetermined altitude and automatically flies linearly toward a start point(S).

[0043] In Step S2, the drone DR performs pre-route flight standby. Specifically, when arriving at the start point (S) by preparatory flight, the drone DR waits for a start instruction from the user at that start point(S). At this time, the drone DR may wait for the start instruction from the user while hovering.

[0044] Then, in Step S3, the drone DR performs route flight. That is, the drone DR automatically flies to follow a flight route included in a flight plan.

[0045] For this series of automatic flight of the drone DR executed as described above, the user can intervene by manual operation by operating an operation stick of a transmitter.

[0046] FIG. 2 is a diagram illustrating intervention by manual operation during automatic flight.

[0047] Each step illustrated in FIG. 2 corresponds to the respective steps described with reference to FIG. 1.

[0048] That is, when the drone DR is performing preparatory flight in Step S1, if the user operates the operation stick of the transmitter, the drone DR flies in response to the user operations. However, when the user releases his/her hand from the operation stick and stops operating, the drone DR automatically flies linearly toward the start point(S) again.

[0049] Further, when the drone DR is performing pre-route flight standby in Step S2, if the user operates the operation stick of the transmitter, the entire flight route including the start point(S) is moved (shifted) in response to the user operations. Here, even if the user releases his/her hand from the operation stick and stops operating, the start point(S) and the flight route are not returned to their original positions.

[0050] Moreover, when the drone DR is performing route flight in Step S3, if the user operates the operation stick of the transmitter, for example, the entire flight route is moved (shifted) in response to the user operations, and the drone DR automatically flies to follow the moved flight route. Here as well, even if the user releases his/her hand from the operation stick and stops operating, the flight route is not returned to its original position.

[0051] Note that, in FIG. 2, the flight route of the drone DR is changed forward, backward, left, or right (in the horizontal direction with respect to the ground) by user operations, but the flight route may be changed up or down (in the vertical direction with respect to the ground).

[0052] The flight plan (flight route) of the drone DR as described with reference to FIG. 1 and FIG. 2 can be set using, for example, an application (app) for managing the operation and settings of the drone DR. Such an app is installed on a ground-based device operated by the user on the ground. The ground-based device can include a transmitter, a portable terminal, such as a tablet terminal or a smartphone, connected (attached) to that transmitter, a computer such as a PC (Personal Computer) forming the control system of the drone DR, or the like.

[0053] FIG. 3 is a diagram illustrating a display example of a flight plan on the ground-based device.

[0054] In the example of FIG. 3, as the flight plan of the drone DR, a flight route R1 that connects a takeoff point (H) to the start point(S) and a flight route R2 that connects the start point(S) to a goal point (E) are illustrated. Note that, in reality, the takeoff point (H), the start point(S), the goal point (E), and the flight routes R1 and R2 are displayed on map data, but in FIG. 3, the display of the map data is omitted.

[0055] The flight route R1 is a route for the drone DR to perform preparatory flight and connects the takeoff point (H) to the start point(S) in a straight line regardless of the intention of the user. Meanwhile, the flight route R2 is a route for the drone DR to perform route flight (automatic flight) and can be set by the user as desired. Note that, since the takeoff point (H) is determined by the position of the drone DR at the start of flight, in a strict sense, the flight route R1 from the takeoff point (H) to the start point(S) is not included in the flight plan. However, the present disclosure is not limited to this, and the start point(S) in the flight plan may be treated as identical to the takeoff point (H).

[0056] In such a manner, the flight plan set on the ground-based device is transferred to the drone DR, and when the user instructs the execution of automatic flight, the drone DR starts automatic flight following the flight routes R1 and R2.

[0057] However, as described above, even if the flight route is changed by the user operating the operation stick of the transmitter when the drone DR is performing route flight, the change could not be reflected in the flight plan displayed on the ground-based device. In such a manner, in a case where the flight route of the drone DR during automatic flight is modified, the user could not know the modified flight plan in real time.

[0058] Therefore, in the technology according to the present disclosure, in reference to modification information input during the automatic movement of a mobile body based on a movement plan, a modified plan obtained by modifying the movement plan is output, and the modified plan is displayed during the automatic movement of the mobile body based on the modified plan, so that real-time notification to the user of the movement plan modified during the automatic movement of the mobile body is achieved.

<2. Overview of Mobile Body Control System According to Present Disclosure and Display Examples of Modified Plans>

(Hardware Configuration of Mobile Body Control System)

[0059] FIG. 4 is a block diagram illustrating a hardware configuration example of a mobile body control system to which the technology according to the present disclosure is applied.

[0060] A mobile body control system 1 illustrated in FIG. 4 includes a mobile body 10, a transmitter 20, and a portable terminal 30.

[0061] In the mobile body control system 1, the mobile body 10 and the transmitter 20 are connected to be mutually communicable via a wireless communication path.

[0062] The mobile body 10 can include an autonomous mobile robot, such as a drone, an autonomous vehicle, a maritime autonomous surface ship, and an autonomous vacuum cleaner, or the like.

[0063] The mobile body 10 includes an internal sensor 11, an external sensor 12, a communication unit 13, a controller 14, a drive unit 15, a storage unit 16, and an application processor 17. This configuration is an example, and the mobile body 10 is not necessarily limited to this.

[0064] The internal sensor 11 is a sensor configured to detect the internal state of the mobile body 10. The internal sensor 11 includes, for example, an IMU (inertial measurement unit) capable of detecting the angle and angular velocity of the mobile body 10. Sensor information obtained by the internal sensor 11 is supplied to the controller 14 and the application processor 17.

[0065] The external sensor 12 is a sensor configured to detect the state of the external environment of the mobile body 10. The external sensor 12 includes, for example, a camera capable of capturing images of the surroundings of the mobile body 10, or a barometer capable of measuring atmospheric pressure. Sensor information obtained by the external sensor 12 is also supplied to the controller 14 and the application processor 17.

[0066] The communication unit 13 wirelessly communicates with the transmitter 20. Further, the communication unit 13 can also communicate with the portable terminal 30 connected to the transmitter 20 via the transmitter 20.

[0067] For example, the communication unit 13 receives operation information representing a user operation on the transmitter 20 and supplies the operation information to the controller 14 and the application processor 17, or receives a movement plan output from the portable terminal 30 via the transmitter 20 and supplies the movement plan to the storage unit 16.

[0068] The controller 14 includes, for example, an arithmetic device (processor) configured to control the movement of the mobile body 10. For example, the controller 14 controls the drive of the drive unit 15 in reference to the sensor information from the internal sensor 11 and the external sensor 12 and the operation information from the communication unit 13, and thereby achieves the manual movement of the mobile body 10. Moreover, the controller 14 controls the drive of the drive unit 15 in reference to control information from the application processor 17, and thereby achieves the automatic movement of the mobile body 10.

[0069] Further, the controller 14 supplies movement situation information representing the current movement situation of the mobile body 10 to the communication unit 13 at time intervals set in advance. The communication unit 13 transmits the movement situation information from the controller 14 to the transmitter 20.

[0070] The drive unit 15 is a mechanism for causing the mobile body 10 to move and includes a flight mechanism, a travel mechanism, a propulsion mechanism, or the like. In a case where the mobile body 10 is configured as a drone, the drive unit 15 includes motors, propellers, and the like as a flight mechanism. Further, in a case where the mobile body 10 is configured as an autonomous vehicle, the drive unit 15 includes motors, wheels, and the like as a travel mechanism. In a case where the mobile body 10 is configured as a maritime autonomous surface ship, the drive unit 15 includes screw propellers and the like as a propulsion mechanism.

[0071] The storage unit 16 stores the movement plan output from the portable terminal 30 and received by the communication unit 13. The movement plan includes the movement route of the mobile body 10 desired by the user. The movement route may be represented by a series of pieces of position information indicated by latitude and longitude, or by position information and altitude, speed, and time for each piece of position information. Moreover, the movement plan may include, for each piece of position information representing the movement route, the attitude (imaging direction) of a gimbal camera, which is not illustrated, mounted on the mobile body 10, the model direction and acceleration of the mobile body 10, and other information associated with the movement of the mobile body 10.

[0072] The application processor 17 reads out the movement plan stored in the storage unit 16, to generate control information for the mobile body 10 to automatically move, in reference to the read movement plan and the sensor information from the internal sensor 11 and the external sensor 12. With the control information being supplied to the controller 14, the automatic movement of the mobile body 10 that avoids collisions with obstacles and the like while following the movement plan is achieved.

[0073] Further, the application processor 17 modifies the movement plan read out from the storage unit 16 in reference to the operation information from the communication unit 13 and other modification information. Control information generated according to the modified plan obtained through modification is also output to the controller 14.

[0074] The transmitter 20 can include a remote controller operated by the user who operates and controls the mobile body 10.

[0075] The transmitter 20 includes an operation unit 21 and a communication unit 22. This configuration is an example, and the transmitter 20 is not necessarily limited to this.

[0076] The operation unit 21 includes operation sticks, switches, buttons, or the like, and supplies operation information representing user operations to the communication unit 22. For example, the operation unit 21 can include two operation sticks that achieve each of forward/backward and left/right turning operations and up/down and left/right movement operations of the mobile body 10 by up/down/left/right operations. Further, the operation unit 21 may include an input interface such as a touch panel capable of receiving up/down/left/right operations.

[0077] The communication unit 22 can transmit operation information representing user operations on the operation unit 21 to the mobile body 10 by wirelessly communicating with the mobile body 10.

[0078] Further, the communication unit 22 can transmit to the mobile body 10 a movement plan generated in the portable terminal 30 and supply to the portable terminal 30 movement situation information transmitted from the mobile body 10, by communicating with the portable terminal 30 connected to the transmitter 20.

[0079] The portable terminal 30 includes a tablet terminal, a smartphone, or the like connected (attached) to the transmitter 20.

[0080] The portable terminal 30 includes a control unit 31, an input unit 32, a communication unit 33, and a display unit 34. This configuration is an example, and the portable terminal 30 is not necessarily limited to this. For example, instead of being configured separately from the transmitter 20 described above, the portable terminal 30 may be configured integrally with the transmitter 20 such that a single terminal has the respective configurations of the transmitter 20 and the portable terminal 30.

[0081] The control unit 31 includes a processor such as a CPU (Central Processing Unit) and executes a predetermined program in response to an input signal or the like from the input unit 32, to control each unit of the portable terminal 30. For example, the control unit 31 launches an app for managing the operation and settings of the mobile body 10, according to the input signal from the input unit 32, and generates a movement plan for the mobile body 10.

[0082] The input unit 32 includes a touch panel, buttons, or the like, and supplies an input signal corresponding to a user operation to the control unit 31.

[0083] The communication unit 33 communicates with the transmitter 20 to which the portable terminal 30 is connected (attached), and indirectly sends and receives information to and from the mobile body 10. Further, the communication unit 33 may directly communicate wirelessly with the mobile body 10.

[0084] The display unit 34 includes a liquid crystal display, an organic EL display, or the like, and displays various types of information under the control of the control unit 31. For example, the display unit 34 displays the movement plan generated by the control unit 31 and movement situation information transmitted from the mobile body 10, according to the control of the control unit 31.

(Schematic Functional Configuration of Mobile Body Control System)

[0085] FIG. 5 is a block diagram illustrating a schematic functional configuration example of the mobile body control system 1.

[0086] The mobile body control system 1 illustrated in FIG. 5 includes a movement control unit 51, a plan modification unit 52, a display control unit 53, and a display unit 54. The movement control unit 51 corresponds to the controller 14 of FIG. 4, and the plan modification unit 52 corresponds to the application processor 17 of FIG. 4. Further, the display control unit 53 corresponds to the control unit 31 of FIG. 4, and the display unit 54 corresponds to the display unit 34 of FIG. 4.

[0087] The movement control unit 51 controls the automatic movement of the mobile body 10 according to a movement plan MP output via the plan modification unit 52 and sensor information from the various sensors included in the mobile body 10.

[0088] The plan modification unit 52 outputs, in reference to modification information input during the automatic movement of the mobile body 10 based on the movement plan MP, a modified plan obtained by modifying the movement plan. The movement control unit 51 controls the automatic movement of the mobile body 10 according to the modified plan output by the plan modification unit 52.

[0089] Specifically, the plan modification unit 52 modifies the movement route of the mobile body 10 included in the movement plan, in reference to the modification information.

[0090] The modification information input during the automatic movement of the mobile body 10 includes, for example, surrounding information representing the situation around the mobile body 10 and operation information representing movement plan modification operations by the user. Specifically, the surrounding information is information based on the presence or absence of objects around the mobile body 10, due to which the modification of the movement route of the mobile body 10 in question would be required. The surrounding information includes, for example, distance information representing a certain distance to be maintained between the mobile body 10 and an approach avoidance target that the mobile body 10 in question is to avoid approaching. Further, the surrounding information can include, for example, area information representing the existence of entry prohibited areas (flight prohibited areas), weather information representing weather conditions such as temperature, humidity, wind, rain, and snow, or mobile body information representing the positions of other mobile bodies (other drones, other aircrafts, and the like) acquired through communication or from sensor information. Further, the movement plan modification operations by the user may be performed on the movement route of the mobile body 10 included in the movement plan displayed on the display unit 54.

[0091] For example, the plan modification unit 52 modifies position information representing the movement route of the mobile body 10 as a whole, in reference to the modification information.

[0092] In a case where an operation input in the forward-backward or left-right direction is made to the operation stick (operation unit 21) of the transmitter 20 during the automatic movement of the mobile body 10, the change distance [m] in the horizontal direction of the mobile body 10 is input as modification information. Further, in a case where an operation input in the up-down direction is made to the operation stick (operation unit 21) of the transmitter 20 during the automatic movement of the mobile body 10, the change distance [m] in the altitude direction of the mobile body 10 is input as modification information.

[0093] The plan modification unit 52 moves (shifts) the position information regarding the current and subsequent positions included in the movement plan by a change distance represented by the modification information, and thereby modifies the movement route of the mobile body 10 as a whole. At this time, the goal point in the movement plan may not be modified, and the movement route may be modified to eventually reach the goal point. Further, the goal point in the movement plan may be moved (shifted) together with the position information representing the movement route of the mobile body 10 such that the movement route including the goal point is modified as a whole.

[0094] Here, the movement control unit 51 controls the movement speed of the mobile body 10 according to at least the relation between the current movement position of the mobile body 10 and the movement position newly determined through modification.

[0095] Further, for example, the plan modification unit 52 sequentially modifies the position information representing the movement route of the mobile body 10, in reference to the modification information.

[0096] In a case where an operation input in the forward-backward or left-right direction is made to the operation stick (operation unit 21) of the transmitter 20 during the automatic movement of the mobile body 10, the change distance displacement amount [m] in the horizontal direction of the mobile body 10 is sequentially input as modification information. Further, in a case where an operation input in the up-down direction is made to the operation stick (operation unit 21) of the transmitter 20 during the automatic movement of the mobile body 10, the change distance displacement amount [m] in the altitude direction of the mobile body 10 is sequentially input as modification information.

[0097] The plan modification unit 52 repeatedly moves, each time the modification information is input, the current position information included in the movement plan by a change distance displacement amount represented by the modification information, thereby sequentially modifying the movement route of the mobile body 10.

[0098] Here, the movement control unit 51 controls the movement speed of the mobile body 10 according to at least the relation between the current movement position of the mobile body 10 and the movement position newly determined through modification.

[0099] The display control unit 53 controls the display unit 54 to display, during the automatic movement of the mobile body 10 based on the modified plan output by the plan modification unit 52, the modified plan in question. Specifically, the display control unit 53 (display unit 54) displays, as a modified plan, the movement route of the mobile body 10 which is included in the movement plan and which has been modified by the plan modification unit 52 in reference to the modification information.

[0100] For example, in a case where the position information representing the movement route of the mobile body 10 has been modified as a whole by the plan modification unit 52, the display control unit 53 (display unit 54) displays a movement route represented by the position information modified as a whole. Further, in a case where the position information representing the movement route of the mobile body 10 has been sequentially modified by the plan modification unit 52, the display control unit 53 (display unit 54) sequentially displays a movement route represented by the sequentially modified position information.

[0101] Each functional block forming the mobile body control system 1 of FIG. 5 described above may be achieved by either the mobile body 10 or the ground-based device including the transmitter 20 and the portable terminal 30, which form the mobile body control system 1 of FIG. 4.

[0102] For example, as indicated by the dotted line in FIG. 5, the plan modification unit 52, the display control unit 53, and the display unit 54 may be achieved by a ground-based device GS1. In this case, only the movement control unit 51 is achieved by the mobile body 10.

[0103] Further, as indicated by the dotted line in FIG. 5, the display control unit 53 and the display unit 54 may be achieved by a ground-based device GS2. In this case, the movement control unit 51 and the plan modification unit 52 are achieved by the mobile body 10.

(Flow of Processing of Mobile Body Control System)

[0104] With reference to the flowchart of FIG. 6, the flow of the processing of the mobile body control system 1 of FIG. 5 is described. The processing of FIG. 6 is executed during the automatic movement of the mobile body 10 based on a movement plan.

[0105] In Step S1, the plan modification unit 52 determines whether modification information has been input or not.

[0106] In a case where it is determined in Step S1 that modification information has been input, the processing proceeds to Step S2, and the plan modification unit 52 modifies the movement plan in reference to the input modification information. Specifically, the plan modification unit 52 modifies the movement route of the mobile body 10 included in the movement plan, in reference to the modification information. The modified movement plan (modified plan) is output to the movement control unit 51 and the display control unit 53.

[0107] In Step S3, the movement control unit 51 controls the automatic movement of the mobile body 10, in reference to the modified plan output by the plan modification unit 52.

[0108] In Step S4, the display control unit 53 controls the display unit 54 to display the modified plan output by the plan modification unit 52. Specifically, the display control unit 53 displays the modified movement route as a modified plan.

[0109] Here, the control of the automatic movement of the mobile body 10 in Step S3 and the display of the modified plan in Step S4 may be performed in parallel.

[0110] Note that, in a case where it is determined in Step S1 that modification information has not been input, Step S2 to Step S4 are skipped.

[0111] In Step S5, the movement control unit 51 determines whether to end the automatic movement based on the movement plan (modified plan) or not.

[0112] In a case where it is determined in Step S5 not to end the automatic movement, the processing returns to Step S1, and the subsequent processing is repeated.

[0113] On the other hand, in a case where it is determined in Step S5 to end the automatic movement, the movement control unit 51 ends the automatic movement of the mobile body 10, and the processing ends.

[0114] With the processing described above, it becomes possible to achieve real-time notification to the user of a movement plan modified during the automatic movement of the mobile body, and to allow the user to easily know the modified movement plan.

(Display Examples of Modified Plans)

[0115] Here, display examples of modified plans displayed on the display unit 54 are described. In the following, an example of a case where the flight plan described with reference to FIG. 3 for the mobile body 10, which serves as a drone, has been modified is given.

[0116] FIG. 7 is a diagram illustrating a first display example of a modified plan.

[0117] In FIG. 7, a flight route R11, which is the flight route R2 of FIG. 3 modified to be moved (shifted) to the left in FIG. 7, is illustrated. An icon M10 indicating the current flight position of the mobile body 10 is illustrated on the flight route R11, and the icon M10 moves along the flight route R11 as the mobile body 10 automatically flies.

[0118] With this, the user can know the modified flight route.

[0119] FIG. 8 is a diagram illustrating a second display example of a modified plan.

[0120] In FIG. 8, similar to FIG. 7, the flight route R11, which is the flight route R2 of FIG. 3 modified to be moved (shifted) to the left in FIG. 8 as a whole, is illustrated together with the flight route R2 (dotted line) used before modification.

[0121] With this, the user can know how the flight route before modification has been modified.

[0122] Note that, other than the modified flight route R11 being drawn, as illustrated in FIG. 8, a numerical value SA representing the modification amount (shift amount) of the modified flight route may also be displayed. In the example of FIG. 8, as the numerical value SA, it is illustrated that the flight route R2 before modification has been shifted 3 m to the west.

[0123] FIG. 9 is a diagram illustrating a third display example of a modified plan.

[0124] In FIG. 9, a flight route R12, which is the flight route R2 of FIG. 3 sequentially modified in reference to input modification information, is indicated by the arrow. The icon M10 indicating the current flight position of the mobile body 10 is illustrated at the distal end of the flight route R12, and the icon M10 moves in accordance with the route following the input modification information. In FIG. 9 as well, the flight route R2 before modification is indicated by the dotted line.

[0125] Further, as illustrated in FIG. 9, together with the flight route of the mobile body 10, an FPV (First Person View) camera image CAM captured by a front camera or a gimbal camera, which are not illustrated, may be displayed. In this case, an arrow r12 representing the modified flight route R12 may be displayed to be superimposed on the camera image CAM.

[0126] In the example of FIG. 9, for example, while the modification information is continuously input, such as when the user continues to operate the operation stick (operation unit 21) of the transmitter 20, the mobile body 10 moves along the route following that modification information. On the other hand, when the modification information is no longer input, such as when the user stops operating the operation stick (operation unit 21) of the transmitter 20, as illustrated in FIG. 10, the mobile body 10 automatically flies to follow the flight route R2 used before modification (returns to the original flight route R2), and that state is displayed on the ground-based device.

[0127] FIG. 11 is a diagram illustrating a fourth display example of a modified plan.

[0128] In FIG. 11, a flight route R21, which is the flight route R2 of FIG. 3 modified in reference to modification information input during the route flight standby of the mobile body 10, and a flight route R22, which is that flight route R21 modified to be moved (shifted) to the left in FIG. 11, are illustrated. The icon M10 indicating the current flight position of the mobile body 10 is illustrated on the flight route R22, and the icon M10 moves along the flight route R22 as the mobile body 10 automatically flies.

[0129] With this, the user can know how the flight route set before flight and the flight route set (modified) during route flight standby have each been modified.

[0130] Note that the display examples of modified plans are not limited to those described above. For example, the display examples of FIG. 8 to FIG. 11 may be combined and displayed as appropriate, or only a part of any of them may be displayed.

[0131] In the following, the configurations of embodiments of the above-mentioned mobile body control system 1 are described.

3. First Embodiment

[0132] It is an object of a drone system of the present embodiment to achieve the visualization of modified plans.

(Configuration of Drone System)

[0133] FIG. 12 is a block diagram illustrating the configuration of the drone system of the first embodiment of the technology according to the present disclosure.

[0134] A drone system 100 illustrated in FIG. 12 includes a drone 110 and a ground-based device 120.

[0135] The drone 110 corresponds to the mobile body 10 forming the mobile body control system 1 of FIG. 4 and can automatically fly following a flight route included in a flight plan set in advance.

[0136] The drone 110 includes a plan modification amount storage control unit 111, a plan modification amount retention unit 112, a plan modification unit 113, and an automatic flight control unit 114.

[0137] The plan modification amount storage control unit 111 converts a flight plan modification operation (plan modification operation) by the user input as modification information (operation information) from the ground-based device 120 during the automatic flight of the drone 110 into a flight plan modification amount (plan modification amount), and causes the plan modification amount retention unit 112 to store the plan modification amount.

[0138] The plan modification amount stored (retained) in the plan modification amount retention unit 112 is read out to the plan modification unit 113 as appropriate as a manual modification plan.

[0139] The plan modification unit 113 outputs a modified plan obtained by modifying a flight plan FP set in advance and retained by the drone 110, in reference to the manual modification plan read out from the plan modification amount retention unit 112, to the automatic flight control unit 114.

[0140] The automatic flight control unit 114 controls the automatic flight of the drone 110 following a modified flight route, in reference to the modified plan output from the plan modification unit 113.

[0141] Further, the modified plan including the modification amount from the original flight plan FP is transmitted to the ground-based device 120.

[0142] The ground-based device 120 corresponds to the transmitter 20 or the portable terminal 30, which forms the mobile body control system 1 of FIG. 4, and can manage the operation and settings of the drone 110 according to apps installed in advance.

[0143] The ground-based device 120 includes an operation unit 121 and a modified plan visualization unit 122.

[0144] The operation unit 121 includes operation sticks, switches, buttons, or the like, and receives flight plan modification operations by the user. Plan modification operations, which serve as operation information representing the modification operations in question, are transmitted to the drone 110.

[0145] The modified plan visualization unit 122 presents a modified flight route to the user in reference to the modified plan including the modification amount from the original flight plan FP from the drone 110.

(Flow of Processing of Drone System)

[0146] With reference to the flowchart of FIG. 13, the flow of the processing of the drone system 100 of FIG. 12 is described. The processing of FIG. 13 is executed during the automatic flight of the drone 110 based on a flight plan.

[0147] When receiving a flight plan modification operation by the user in Step S111, in Step S112, the ground-based device 120 transmits the plan modification operation to the drone 110.

[0148] When receiving the plan modification operation from the ground-based device 120, in Step S113, the drone 110 stores the plan modification operation from the ground-based device 120 as a plan modification amount in the plan modification amount retention unit 112.

[0149] In Step S114, the drone 110 executes flight plan modification processing to modify the flight plan FP set in advance, in reference to the plan modification amount stored in the plan modification amount retention unit 112.

[0150] In Step S115, the drone 110 transmits the modified plan obtained by the flight plan modification processing to the ground-based device 120.

[0151] Then, in Step S116, the drone 110 controls its own automatic flight in reference to the modified plan. Meanwhile, when receiving the modified plan from the drone 110, in Step S117, the ground-based device 120 presents a modified flight route to the user in reference to the received modified plan.

[0152] With the processing described above, it becomes possible to achieve real-time notification to the user of a flight plan modified during the automatic flight of the drone, and to allow the user to easily know the modified flight plan.

(Modified Example of Drone System)

[0153] FIG. 14 is a block diagram illustrating a modified example of the drone system 100.

[0154] Note that, in the drone system 100 of FIG. 14, the components that are identical to the components of the drone system 100 of FIG. 12 are denoted by the same reference signs, and the descriptions thereof are omitted as appropriate.

[0155] That is, the drone system 100 of FIG. 14 differs from the drone system 100 of FIG. 12 in that the ground-based device 120 newly includes a modified plan reconstruction unit 131 and retains a flight plan FP that is the same as the flight plan FP retained by the drone 110.

[0156] Note that, in the drone system 100 of FIG. 14, instead of a modified plan including the modification amount from the original flight plan FP, only the modification amount is transmitted from the drone 110 to the ground-based device 120.

[0157] The modified plan reconstruction unit 131 creates (reconstructs) the modified plan obtained by flight plan modification processing by the drone 110, in reference to the flight plan FP retained by the ground-based device 120 and the modification amount from the drone 110.

[0158] The modified plan visualization unit 122 presents a modified flight route to the user in reference to the reconstructed modified plan.

[0159] With the configuration described above, it is possible to present a modified flight route to the user by only sending and receiving the modification amount from the original flight plan FP, rather than the entire modified plan, between the drone 110 and the ground-based device 120. That is, it becomes possible to notify the user of the modified flight plan with less communication volume.

4. Second Embodiment

[0160] For the drone in automatic flight based on a flight plan, the user can intervene by manual operation by operating the operation stick of the transmitter.

[0161] However, it is difficult to precisely modify the flight route by manual operation in a similar manner to, for example, drawing a flight route on the app of the ground-based device, for the purpose of a flight desired by the user. Further, it is also difficult to instantly modify the flight route by manual operation, such as when avoiding obstacles.

[0162] Therefore, it is an object of a drone system of the present embodiment to achieve a flight desired by the user even in a case where intervention by manual operation has been performed on the drone during automatic flight.

(Configuration of Drone System)

[0163] FIG. 15 is a block diagram illustrating the configuration of the drone system of the second embodiment of the technology according to the present disclosure.

[0164] Note that, in a drone system 200 of FIG. 15, the components that are identical to the components of the drone system 100 of FIG. 12 are denoted by the same reference signs, and the descriptions thereof are omitted as appropriate.

[0165] That is, the drone system 200 of FIG. 15 differs from the drone system 100 of FIG. 12 in that the drone 110 newly includes an automatic plan modification unit 211.

[0166] The automatic plan modification unit 211 acquires a plan modification amount different from a plan modification amount (manual modification plan) stored (retained) in the plan modification amount retention unit 112, and outputs the different plan modification amount as an automatic modification plan to the plan modification unit 113.

[0167] The plan modification unit 113 appropriately modifies the flight plan FP retained by the drone 110, in reference to the manual modification plan read out from the plan modification amount retention unit 112 and the automatic modification plan output from the automatic plan modification unit 211. The modified flight plan is supplied to the automatic flight control unit 115 as an appropriate modified plan that enables a flight desired by the user, which cannot be achieved by the user's intervention by manual operation alone.

[0168] In the following, specific configuration examples of the drone system 200 described above are described.

(First Specific Configuration Example of Drone System)

[0169] FIG. 16 is a block diagram illustrating a first specific configuration example of the drone system 200.

[0170] In the drone system 200 of FIG. 16, a drone 110A includes a pre-plan retention unit 221 as the automatic plan modification unit 211 of FIG. 15.

[0171] The pre-plan retention unit 221 retains a pre-plan, which is a flight plan set before the flight of the drone 110A. The pre-plan may be the flight plan FP itself, which has been set in advance and retained by the drone 110A, or may be the flight plan FP modified in reference to operation information input during the pre-route flight standby of the drone 110A.

[0172] The pre-plan retained in the pre-plan retention unit 221 is read out to the plan modification unit 113 as appropriate.

[0173] Further, in the drone system 200 of FIG. 16, the operation unit 121 of the ground-based device 120 outputs, to the drone 110A, a no-operation instruction indicating that there is no plan modification operation by the user. The no-operation instruction is output when it is detected that, for example, the stick values in the directions of all four axes of the operation stick, which serves as the operation unit 121, are smaller than a predetermined threshold.

[0174] The plan modification unit 113 determines whether to modify the flight plan FP in reference to either the manual modification plan of the plan modification amount retention unit 112 or the pre-plan of the pre-plan retention unit 221, depending on whether there is an input of a no-operation instruction from the ground-based device 120 or not.

[0175] Here, with reference to the flowchart of FIG. 17, the flow of flight plan modification processing by the drone 110A is described.

[0176] In Step S221, the plan modification unit 113 of the drone 110A determines whether a no-operation instruction has been input from the ground-based device 120 or not.

[0177] In a case where it is determined in Step S221 that a no-operation instruction has not been input, that is, in a case where a plan modification operation, which serves as operation information, has been input, the processing proceeds to Step S222.

[0178] In Step S222, the plan modification unit 113 modifies the flight plan FP in reference to a manual modification plan retained in the plan modification amount retention unit 112. That is, in a case where operation information has been input, the plan modification unit 113 outputs a modified plan obtained through modification in reference to the operation information.

[0179] On the other hand, in a case where it is determined in Step S221 that a no-operation instruction has been input, that is, in a case where a plan modification operation, which serves as operation information, has not been input, the processing proceeds to Step S223.

[0180] In Step S223, the plan modification unit 113 modifies the flight plan FP in reference to a pre-plan retained in the pre-plan retention unit 221. That is, in a case where operation information has not been input, for example, the plan modification unit 113 outputs the flight plan FP used before modification as it is or outputs the flight plan FP modified during pre-route flight standby.

[0181] With this, as described with reference to FIG. 10, when the user stops operating the operation stick of the transmitter, the drone 110A automatically flies to follow the flight route used before modification (returns to the original flight route).

[0182] With the configuration and the processing described above, when there is no modification operation by the user, the drone automatically flies according to a precise flight route drawn in advance by the user. That is, it becomes possible to achieve a delicate flight desired by the user, which cannot be achieved by the user's intervention by manual operation alone.

[0183] Further, not only a flight route changed by modification operations by the user, but also a flight route changed without any modification operation by the user is presented to the user in real time. With this, the user can know not only how he/she has modified the flight route himself/herself, but also how the system has modified the flight route, so that he/she can continue the flight of the drone with peace of mind.

(Second Specific Configuration Example of Drone System)

[0184] FIG. 18 is a block diagram illustrating a second specific configuration example of the drone system 200.

[0185] In the drone system 200 of FIG. 18, a drone 110B newly includes a distance measurement unit 231 and an approach prevention plan change unit 232, which serves as the automatic plan modification unit 211 of FIG. 15.

[0186] The distance measurement unit 231 is a distance measurement sensor that can measure the distance to an approach avoidance target, which is an object that the drone 110B is to avoid approaching. The distance measurement unit 231 includes, for example, a stereo camera, LiDAR (Light Detection And Ranging), a millimeter-wave radar, or a ToF (Time of Flight) sensor mounted on the drone 110B. The distance measurement unit 231 supplies sensor information obtained by distance measurement to the approach prevention plan change unit 232.

[0187] The approach avoidance target may be, for example, an obstacle existing in the flight environment of the drone 110B, or may be a geofence, which is a flight restricted area set for the drone 110B.

[0188] The approach prevention plan change unit 232 inputs distance information which represents a certain distance to be maintained between the drone 110B and the approach avoidance target, as modification information (surrounding information), to the plan modification unit 113, as an approach prevention plan, in reference to the sensor information acquired by the distance measurement unit 231.

[0189] Specifically, the approach prevention plan change unit 232 determines whether the drone 110B and the approach avoidance target have approached each other to reach a threshold distance determined in advance or not, in reference to the sensor information from the distance measurement unit 231. Then, in a case where the drone 110B and the approach avoidance target have approached each other within the threshold distance, the approach prevention plan change unit 232 inputs the approach prevention plan (distance information) to the plan modification unit 113.

[0190] Note that the approach prevention plan change unit 232 may determine whether the drone 110B and the approach avoidance target have approached each other within the threshold distance or not, in reference to, in addition to the sensor information from the distance measurement unit 231, the self-position of the drone 110B estimated using the sensor information.

[0191] The plan modification unit 113 determines whether to modify the flight plan FP or not, in reference to only a manual modification plan from the plan modification amount retention unit 112, depending on whether there is an input of an approach prevention plan from the approach prevention plan change unit 232 or not.

[0192] Here, with reference to the flowchart of FIG. 19, the flow of flight plan modification processing by the drone 110B is described.

[0193] In Step S231, the plan modification unit 113 of the drone 110B determines whether an approach prevention plan (distance information) has been input from the approach prevention plan change unit 232 or not.

[0194] In a case where it is determined in Step S231 that an approach prevention plan has not been input, that is, in a case where the drone 110B and the approach avoidance target have not approached each other within the threshold distance, the processing proceeds to Step S232.

[0195] In Step S232, the plan modification unit 113 modifies the flight plan FP in reference to a manual modification plan corresponding to a plan modification operation. Note that, in a case where a plan modification operation (operation information) has not been input, the plan modification unit 113 outputs the flight plan FP set in advance as it is.

[0196] On the other hand, in a case where it is determined in Step S231 that an approach prevention plan has been input, that is, in a case where the drone 110B and the approach avoidance target have approached each other within the threshold distance, the processing proceeds to Step S233.

[0197] In Step S233, the plan modification unit 113 modifies the flight plan FP in reference to the approach prevention plan and the manual modification plan. Specifically, the plan modification unit 113 further modifies the modified plan obtained by modifying the flight plan FP based on the manual modification plan, in reference to the approach prevention plan. Note that, in a case where a plan modification operation (operation information) has not been input, the plan modification unit 113 outputs the modified plan obtained by modifying the flight plan FP, in reference to only the approach prevention plan.

[0198] It is assumed that, for example, as illustrated in the left part of FIG. 20, when the drone 110B is automatically flying to follow a flight route R31, the distance to an obstacle AT approaches the threshold distance.

[0199] In this case, as illustrated in the right part of FIG. 20, a modified plan including a flight route R32, which is obtained by moving (shifting) position information representing the flight route R31 in a direction away from the obstacle AT by a certain distance represented by an approach prevention plan (distance information), is sequentially created. The drone 110B can continue to automatically fly to follow the flight route R32, which avoids a collision with the obstacle AT, by flying a route R31 for a transition from the flight route R31 to the flight route R32.

[0200] With the configuration and the processing described above, when approaching an approach avoidance target within the threshold distance, the drone automatically flies according to a flight route that instantly avoids approach with the approach avoidance target. That is, it is possible to achieve a safe flight desired by the user, which cannot be achieved by the user's intervention by manual operation alone.

[0201] Further, not only a flight route changed by a modification operation by the user, but also a flight route changed without any modification operation by the user is presented to the user in real time. With this, the user can know not only how he/she has modified the flight route himself/herself, but also how the system has modified the flight route, so that he/she can continue the flight of the drone with peace of mind.

<5. Configuration Example of Computer>

[0202] The series of processing processes described above can be executed by hardware or software. In a case where the series of processing processes is executed by software, a program configuring that software is installed on a computer. Here, examples of the computer include computers incorporated in dedicated hardware and, for example, general-purpose personal computers capable of executing various functions with various programs installed thereon.

[0203] FIG. 21 is a block diagram illustrating a configuration example of the hardware of a computer configured to execute the series of processing processes described above by the program.

[0204] In a computer 300, a CPU 301, a ROM (Read Only Memory) 302, and a RAM (Random Access Memory) 303 are connected to each other through a bus 304.

[0205] An input-output interface 305 is further connected to the bus 304. The input-output interface 305 is connected to an input unit 306, an output unit 307, a storage unit 308, a communication unit 309, and a drive 310.

[0206] The input unit 306 includes a keyboard, a mouse, a microphone, or the like. The output unit 307 includes a display, a speaker, or the like. The storage unit 308 includes a hard disk, a non-volatile memory, or the like. The communication unit 309 includes a network interface or the like. The drive 310 drives a removable medium 311 such as a magnetic disk, an optical disc, a magneto-optical disk, or a semiconductor memory.

[0207] In the computer 300 configured as described above, for example, the CPU 301 loads the program stored in the storage unit 308 into the RAM 303 through the input-output interface 305 and the bus 304 and executes the program to perform the series of processing processes described above.

[0208] The program that the computer 300 (CPU 301) executes can be recorded on the removable medium 311, which serves as a package medium or the like, to be provided, for example. Further, the program can be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

[0209] In the computer 300, the program can be installed on the storage unit 308 through the input-output interface 305 with the removable medium 311 mounted on the drive 310. Further, the program can be received by the communication unit 309 via a wired or wireless transmission medium to be installed on the storage unit 308. Besides, the program can be installed on the ROM 302 or the storage unit 308 in advance.

[0210] Note that the program that the computer executes may be a program whose processing processes are performed chronologically in the order described herein or in parallel. Alternatively, the program may be a program whose processing processes are performed at necessary timing such as when the program is called.

[0211] Embodiments of the present technology are not limited to the embodiments described above, and various modifications can be made without departing from the gist of the present technology.

[0212] For example, the present technology can employ the configuration of cloud computing in which a single function is shared and collaboratively processed by multiple apparatuses via a network.

[0213] Further, each step of the flowcharts described above can be executed by a single apparatus or can be shared and executed by multiple apparatuses.

[0214] Moreover, in a case where multiple processing processes are included in a single step, the multiple processing processes included in the single step can be executed by a single apparatus or can be shared and executed by apparatuses.

[0215] Further, the effects described herein are merely exemplary and not limiting, and there may be other effects.

[0216] Furthermore, the technology according to the present disclosure can take the following configurations.

(1)

[0217] An information processing method including: [0218] outputting, in reference to modification information input during automatic movement of a mobile body based on a movement plan, a modified plan obtained by modifying the movement plan; and [0219] displaying the modified plan during the automatic movement of the mobile body based on the modified plan.
(2)

[0220] The information processing method according to (1), in which [0221] a movement route of the mobile body included in the movement plan is modified in reference to the modification information, and [0222] the movement route modified is displayed as the modified plan.
(3)

[0223] The information processing method according to (2), in which [0224] position information representing the movement route is modified as a whole in reference to the modification information, and [0225] the movement route represented by the position information modified as a whole is displayed as the modified plan.
(4)

[0226] The information processing method according to (2), in which [0227] position information representing the movement route is sequentially modified in reference to the modification information, and [0228] the movement route represented by the position information sequentially modified is sequentially displayed as the modified plan.
(5)

[0229] The information processing method according to any one of (2) through (4), in which the movement route before modification is displayed together with the movement route modified.

(6)

[0230] The information processing method according to (5), in which [0231] the movement route before modification includes at least one of a first movement route set before movement of the mobile body and a second movement route set during automatic movement standby of the mobile body, and [0232] at least one of the first movement route and the second movement route is displayed together with the movement route modified.
(7)

[0233] The information processing method according to (6), in which the second movement route is the first movement route modified in reference to the modification information input during the automatic movement standby of the mobile body.

(8)

[0234] The information processing device according to (1), in which [0235] a movement route of the mobile body included in the movement plan is modified in reference to the modification information, and [0236] a numerical value representing a modification amount of the movement route modified is displayed as the modified plan.
(9)

[0237] The information processing method according to any one of (1) through (8), in which [0238] the modification information includes operation information representing a modification operation for the movement plan by a user, and [0239] the modified plan modified in reference to the operation information is output.
(10)

[0240] The information processing method according to (9), in which, [0241] in a case where the operation information has been input, the modified plan modified in reference to the operation information is output, and, [0242] in a case where the operation information has not been input, the movement plan before modification is output as it is.
(11)

[0243] The information processing method according to (9), in which the operation information is input from a ground-based device capable of communicating with the mobile body.

(12)

[0244] The information processing method according to any one of (1) through (8), in which the modification information includes surrounding information representing a situation around the mobile body.

(13)

[0245] The information processing method according to (12), in which [0246] the surrounding information includes distance information representing a certain distance to be maintained between the mobile body and an approach avoidance target, and [0247] the modified plan modified in reference to the distance information is output.
(14)

[0248] The information processing method according to (13), in which the distance information is input in a case where the mobile body and the approach avoidance target approach each other within a threshold distance determined in advance.

(15)

[0249] The information processing method according to (14), in which whether the mobile body and the approach avoidance target have approached each other within the threshold distance or not is determined in reference to sensor information acquired by a distance measurement sensor mounted on the mobile body.

(16)

[0250] The information processing method according to any one of (13) through (15), in which [0251] the modification information further includes operation information representing a modification operation for the movement plan by a user, and [0252] the modified plan modified in reference to the operation information is further modified in reference to the distance information.
(17)

[0253] The information processing method according to any one of (1) through (16), in which the mobile body includes a drone capable of automatic flight based on a flight plan.

(18)

[0254] An information processing device including: [0255] a plan modification unit configured to output, in reference to modification information input during automatic movement of a mobile body based on a movement plan, a modified plan obtained by modifying the movement plan; and [0256] a display control unit configured to display the modified plan during the automatic movement of the mobile body based on the modified plan.
(19)

[0257] A mobile body control system including: [0258] a mobile body; [0259] a plan modification unit configured to output, in reference to modification information input during automatic movement of the mobile body based on a movement plan, a modified plan obtained by modifying the movement plan; and [0260] a display control unit configured to display the modified plan during the automatic movement of the mobile body based on the modified plan.

REFERENCE SIGNS LIST

[0261] 1: Mobile body control system [0262] 10: Mobile body [0263] 11: Internal sensor [0264] 12: External sensor [0265] 13: Communication unit [0266] 14: Controller [0267] 15: Drive unit [0268] 16: Storage unit [0269] 17: Application processor [0270] 20: Transmitter [0271] 21: Operation unit [0272] 22: Communication unit [0273] 30: Portable terminal [0274] 31: Control unit [0275] 32: Input unit [0276] 33: Communication unit [0277] 34: Display unit [0278] 51: Movement control unit [0279] 52: Plan modification unit [0280] 53: Display control unit [0281] 54: Display unit [0282] 30: Drone system [0283] 110, 110A, 110B: Drone [0284] 120: Ground-based device [0285] 113: Plan modification unit [0286] 114: Automatic flight control unit [0287] 122: Modified plan visualization unit [0288] 131: Modified plan reconstruction unit [0289] 211: Automatic plan modification unit [0290] 221: Pre-plan retention unit [0291] 231: Distance measurement unit [0292] 232: Approach prevention plan change unit