REMOTE PERCEPTION STATION AS MAINTENANCE TRIGGER FOR AUTONOMOUS VEHICLES DEPLOYED IN AUTONOMOUS TRANSPORT SOLUTIONS

Abstract

A system for detecting a maintenance need of an autonomous vehicle operating as part of a cargo transport system is described. The system includes a control unit and a remote sensor arranged separate from the autonomous vehicle. The control unit is arranged to obtain, via the remote sensor, a current driving behaviour of the autonomous vehicle in a test area; obtain a baseline driving behaviour of the autonomous vehicle in the test area; determine a deviation of the current driving behaviour from the baseline driving behaviour; and detect a maintenance need of the autonomous vehicle based on the determined deviation.

Claims

1. A system for detecting a maintenance need of an autonomous vehicle operating as part of a cargo transport system, the system comprising: a control unit and a remote sensor arranged separate from the autonomous vehicle, where the control unit is arranged to obtain, via the remote sensor, a current driving behaviour of the autonomous vehicle in a test area, obtain a baseline driving behaviour of the autonomous vehicle in the test area, determine a deviation of the current driving behaviour from the baseline driving behaviour, and detect a maintenance need of the autonomous vehicle based on the determined deviation, wherein the baseline driving behaviour is obtained from one or more previous observations of the autonomous vehicle in the test area.

2. The system according to claim 1, wherein the control unit is further arranged to determine a fault based on the determined deviation.

3. The system according to claim 2, wherein the fault comprises any of tire wear, mechanical defects, suspension system failure, and brake failure.

4. The system according to claim 2, wherein the remote sensor comprises any of a camera, an IR camera, a lidar, a sonar, a microphone, and a radar.

5. The system according to claim 2, wherein the current driving behaviour and baseline driving behaviour comprise one or more locations or travelled path of the autonomous vehicle in the test area.

6. The system according to claim 2, wherein the current driving behaviour and baseline driving behaviour comprise any of velocity, acceleration, yaw, and yaw rate of the autonomous vehicle.

7. The system according to claim 2, wherein maintenance is recommended if the deviation is a above a predetermined threshold deviation.

8. The system according to claim 2, wherein maintenance is recommended based on a computer-implemented classification model arranged to determine a maintenance need based on the determined deviation.

9. The system according to claim 8, wherein the computer-implemented classification model is based on any of a look up table and an analytical function.

10. The system according to claim 8, wherein the computer-implemented classification model is based on any of a neural network, a random forest structure, a support vector machine model, a logistic regression algorithm, a Bayes algorithm, a decision tree algorithm, and a K-nearest neighbours' algorithm.

11. The system according to claim 2, wherein the baseline behaviour is obtained from one or more previous observations of one or more autonomous vehicles in the test area.

12. The system according to claim 2, wherein the baseline driving behaviour is obtained from a planned driving behaviour of the autonomous vehicle.

13. The system according to claim 2, where the control unit is arranged to prompt the autonomous vehicle to perform a test case manoeuvre in the test area.

14. A method for detecting a maintenance need of an autonomous vehicle, the method comprising: obtaining, via a remote sensor arranged separate from the autonomous vehicle, a current driving behaviour of the autonomous vehicle in a test area, obtaining a baseline driving behaviour of the autonomous vehicle in the test area, wherein the baseline driving behaviour is obtained from one or more previous observations the autonomous vehicle in the test area, determining a deviation of the current driving behaviour from the baseline driving behaviour, and detecting a maintenance need of the autonomous vehicle based on the determined deviation.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0027] With reference to the appended drawings, below follows a more detailed description of embodiments of the invention cited as examples. In the drawings:

[0028] FIG. 1 illustrates a system for detecting a maintenance need of an autonomous vehicle;

[0029] FIG. 2 is a plot showing deviation versus time;

[0030] FIG. 3 is a flow chart illustrating an example method of the present disclosure;

[0031] FIG. 4 schematically illustrates a control unit; and

[0032] FIG. 5 shows an example computer program product.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS OF THE INVENTION

[0033] The invention will now be described more fully hereinafter with reference to the accompanying drawings, in which certain aspects of the invention are shown. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments and aspects set forth herein; rather, these embodiments are provided by way of example so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Like numbers refer to like elements throughout the description.

[0034] It is to be understood that the present invention is not limited to the embodiments described herein and illustrated in the drawings; rather, the skilled person will recognize that many changes and modifications may be made within the scope of the appended claims.

[0035] FIG. 1 shows a heavy-duty vehicle 110. This particular example comprises a tractor unit towing a trailer unit, which in turn tows an additional trailer unit using a dolly unit. The vehicle combination 110 may of course also comprise additional vehicle units. The techniques disclosed herein are applicable to rigid trucks, and also to passenger cars, although the main benefit of the proposed technique is obtained when used with heavy-duty vehicle for autonomous transport system.

[0036] As mentioned, there is a need for improved systems that can detect maintenance needs of autonomous vehicles. In particular, internal sensors of autonomous vehicles typically detects faults and maintenance needs later than desired. Detecting a drifting accuracy of such internal sensor is also challenging. As of today, any diagnostics in autonomous vehicles are based on the detection of a failure, not on the prevention of it. Moreover, an autonomous vehicle typically believes its path/trajectory is followed as it should, even when an external system would not agree.

[0037] Therefore, there is herein disclosed a system 100 for detecting a maintenance need of an autonomous vehicle 110 operating as part of an autonomous transport system. The system comprises a control unit 130 and a remote sensor 120 arranged separate from the autonomous vehicle 110. The control unit is arranged to obtain, via the remote sensor 120, a current driving behaviour of the autonomous vehicle 110 in a test area 121; obtain a baseline driving behaviour of the autonomous vehicle 110 in the test area 121; determine a deviation of the current driving behaviour from the baseline driving behaviour; and detect a maintenance need of the autonomous vehicle 110 based on the determined deviation.

[0038] The disclosed system is suitable for detecting a maintenance needs of an autonomous vehicle. This includes fully autonomous vehicles and partially autonomous vehicles. In general, however, the system 100 and corresponding method disclosed herein can detect a maintenance need of vehicles where it is possible to compare a current driving behaviour to a baseline driving behaviour. To detect a maintenance need herein means to detect an anomaly in the driving behaviour resulting from a faulty component, system, subsystem etc. before critical failure.

[0039] The remote sensor 120 can be located at specific places where the driving behaviour is representative to parts of or the whole operation route. If anything in the vehicle does not look good, i.e., if the current driving behaviour deviates too much from the baseline driving behaviour according to some pre-determined metric, the vehicle 110 can be removed from operation and be brought to the maintenance workshop autonomously.

[0040] Arranging the sensor 120 separate from the autonomous vehicle 110 enables the capture of data that may be difficult to obtain from internal sensors. Arranged separate can, e.g., mean that the sensor is arranged stationary with respect to a road, such as a camera aimed at a section of the road. The system 100 may use a plurality of remote sensors 120, but a single sensor will suffice, which is cost-effective. Furthermore, using a remote sensor rather than internal sensors means that no modifications of the vehicles are necessary, neither by adding additional sensors nor by adapting existing sensors.

[0041] The system 100 can operate independently from other control systems related to the vehicle. The obtaining of the current driving behaviour, the obtaining of the baseline driving behaviour, the determination of the deviation, and the detection of maintenance can, according to aspects, all be performed within the system 100. This facilitates data management since, e.g., there is no need for wireless connections loaded with large amounts of data from, e.g., movies captured by a camera constituting the sensor 120.

[0042] The control unit 130 can be connected to vehicle 110 and other relating systems for autonomous driving. For example, the control unit can be connected to a cloud system with traffic planner functionality, which coordinates which vehicles to send to the workshop and optionally also which selects what kind of service is needed.

[0043] In general, the system 100 can take all kinds of variations possible in the driving behaviour over time. Based on that, it is possible to predict when the vehicle will should to go through maintenance and/or calibration. This way, it is possible to do maintenance before a complete breakdown.

[0044] FIG. 2 is a plot showing a deviation of driving behaviour versus time. A first line 210 shows a deviation increasing over time. A breakdown occurs at time instance 211. This breakdown could have been avoided if maintenance had been done well ahead of the time instance 211. A second line 220 also shows a deviation increasing over time. Here, on the other hand, the vehicle is taken out of the cargo transport system for maintenance after the deviation has reached a threshold value 230, and the breakdown therefore is avoided.

[0045] As mentioned, it may be difficult to detect a drifting accuracy of internal sensors using the internal sensors of a vehicle. The disclosed system 100, however, can detect such drift. Usually, when internal sensors are drifting, a small degradation of the driving behaviour is noticed over time. This degradation can be observed by the system 100 as a larger and larger deviation from a baseline driving behaviour.

[0046] In the example of FIG. 1, the heavy-duty vehicle 110 is driving along a lane on road 101 and has just passed through a 90-degree corner. A single sensor 120 is arranged at the corner and is arranged to capture the current driving behaviour of the vehicle in the test area 120, which, in this case, covers the corner. In this example, the travelled paths 111, 112, and 113 of three points on the back of the vehicle are observed. More generally, the current driving behaviour and baseline driving behaviour may comprise one or more locations or travelled path of the autonomous vehicle 110 in the test area 121. The driving behaviour may also or alternatively comprise other data. In particular, the current driving behaviour and baseline driving behaviour may comprise any of velocity, acceleration, yaw, and yaw rate of the autonomous vehicle 110. In general, however, the driving behaviour may comprise any data representative of the operation of the vehicle.

[0047] The baseline driving behaviour may be obtained from one or more previous observations the autonomous vehicle 110 in the test area 121. In the example embodiment of FIG. 1, the three travelled paths across the test area may at a first instance be captured and saved as the baseline driving behaviour. Any succeeding passes across the test area may thereafter be compared to the stored baseline behaviour. The baseline behaviour may also be obtained from a plurality of observations of the vehicles in the test area, such as the average of ten controlled test runs.

[0048] The baseline driving behaviour may be unique for each single vehicle. Alternatively, or in combination of, the base line driving behaviour may be representative for a fleet of the same model of vehicles, or even same type of vehicles. Therefore, the baseline behaviour may be obtained from one or more previous observations of one or more autonomous vehicles 110 in the test area 121. Thus, the base line behaviour can be established from a different vehicle than the one the current driving behaviour is obtained from.

[0049] According to aspects, the system 100 has access to an intended behaviour of the vehicle 110, e.g., the path the autonomous driving system is prompting the vehicle to follow. Therefore, the baseline driving behaviour may be obtained from a planned driving behaviour of the autonomous vehicle 110. In this case, the system 100 may return a diagnostic to the vehicle or other system part of the autonomous driving. The system may be integrated as part of the vehicle perception system network via, e.g., V2I (V2X) communication. Additionally, the system may be used to create a distributed perception network for the autonomous cargo transport system.

[0050] The baseline driving behaviour may be obtained from a plurality of different ways, such as the ways mentioned above or more generally any way establishing an expected behaviour of the vehicle.

[0051] The remote sensor 120 may comprise any of a camera, an IR camera, a lidar, a sonar, a microphone, and a radar. The cameras, lidar, sonar, and radar may be used to track one or more points of the vehicle 110. A microphone may be used to detect sounds from the engine, gear boxes, and actuators etc. More generally, the remote sensor can be any type of sensor that can capture data representative of the driving behaviour of the vehicle. As mentioned, the system 100 may comprise a plurality of sensors 120, which may be any combination of different types of sensors. The one or more sensors may be comprised in a single unit or be distributed. Similarly, the control unit 130 may be integrated with one or more sensors 120 or be in a separate unit. The control unit may also be distributed; it can also be cloud based.

[0052] The remote sensor 120 obtains driving behaviour of the autonomous vehicle 110 in a test area 121. This can, e.g., mean a field of view for a camera, radar, lidar etc., and/or an area where the sensor can capture data with a predetermined fidelity, such as resolution, bandwidth, dynamic range etc. As an example, the test area can include a five-by-five meter squared cross section of a road.

[0053] According to aspects, maintenance may be recommended by the control unit 130 if the deviation is a above a predetermined threshold deviation. For example, if any of the tracked paths in FIG. 1 of the current observation deviates more than 20 cm from a previous pass in the test area constituting the baseline, maintenance is recommended. Different behavioural parameters, such as the travelled path of a point or the sound of the engine, may be associated with respective threshold deviations. It is also possible that a threshold deviation is obtained from a weighted averaging of deviations of different behavioural parameters.

[0054] To identify if maintenance is needed, a computer-implemented classification model can be utilized with advantage. In other words, maintenance may be recommended by the control unit 130 based on a computer-implemented classification model arranged to determine a maintenance need based on the determined deviation. In particular, such model may be a machine learning model. Machine learning generally relates to techniques where a model with a pre-determined structure is modified to provide a desired function by means of some form of training. Any machine learning model may be trained using one or more test runs in the test area and/or using simulations. For example, the computer-implemented classification model may be based on any of a neural network, a random forest structure, a support vector machine model, a logistic regression algorithm, a Bayes algorithm, a decision tree algorithm, and a K-nearest neighbours' algorithm. These models are known in general and will therefore not be discussed in more detail herein.

[0055] The classification model may also be based on other algorithms/models. In particular, the computer-implemented classification model may be based on any of a look up table and an analytical function.

[0056] According to aspects, the control unit 130 may further be arranged to determine a fault based on the determined deviation. Certain types of faults may result in a predictable deviation in the current driving behaviour compared to the baseline. A fault can be detected based on, e.g., the magnitude of the deviation of a behaviour parameter of the driving behaviour, such as how much a tracked point of the vehicle deviates from a baseline path. Faults can also be identified using a computer-implemented classification model. The fault may comprise any of tire wear, mechanical defects, suspension system failure, and brake failure. The fault can also be other type defects, wear, or such.

[0057] The system 100 can further be used to send test cases to the vehicles and monitor the feedback. Such tests can be used as a confirmation of system integrity. In other words, the control unit 130 may be arranged to prompt the autonomous vehicle 110 to perform a test case manoeuvre in the test area 121. The baseline driving behaviour may be captured during a previous test. Alternatively, or in combination of, the baseline can be established by simulations. A simulation platform may run in the control unit 130 in the system 100 and can simulate various test cases.

[0058] There is also disclosed herein a method for detecting a maintenance need of an autonomous vehicle 110, as is shown in FIG. 3. The method comprises obtaining S1, via a remote sensor 120 arranged separate from the autonomous vehicle 110, a current driving behaviour of the autonomous vehicle 110 in a test area 121,

[0059] obtaining S2 a baseline driving behaviour of the autonomous vehicle 110 in the test area 121,

[0060] determining S3 a deviation of the current driving behaviour from the baseline driving behaviour, and

[0061] detecting S4 a maintenance need of the autonomous vehicle 110 based on the determined deviation.

[0062] FIG. 4 schematically illustrates, in terms of a number of functional units, the components of the control unit 130 according to an embodiment of the discussions herein. Processing circuitry 410 is provided using any combination of one or more of a suitable central processing unit CPU, multiprocessor, microcontroller, digital signal processor DSP, etc., capable of executing software instructions stored in a computer program product, e.g., in the form of a storage medium 430. The processing circuitry 410 may further be provided as at least one application specific integrated circuit ASIC, or field programmable gate array FPGA.

[0063] Particularly, the processing circuitry 410 is configured to cause the control unit 130 to perform a set of operations, or steps, such as the methods discussed in connection to FIG. 3. For example, the storage medium 430 may store the set of operations, and the processing circuitry 410 may be configured to retrieve the set of operations from the storage medium 430 to cause the control unit 130 to perform the set of operations. The set of operations may be provided as a set of executable instructions. Thus, the processing circuitry 410 is thereby arranged to execute methods as herein disclosed.

[0064] The storage medium 430 may also comprise persistent storage, which, for example, can be any single one or combination of magnetic memory, optical memory, solid state memory or even remotely mounted memory.

[0065] The control unit 130 may further comprise an interface 420 for communications with at least one external device. As such the interface 420 may comprise one or more transmitters and receivers, comprising analogue and digital components and a suitable number of ports for wireline or wireless communication.

[0066] The processing circuitry 410 controls the general operation of the control unit 130 e.g., by sending data and control signals to the interface 420 and the storage medium 430, by receiving data and reports from the interface 420, and by retrieving data and instructions from the storage medium 430. Other components, as well as the related functionality, of the control node are omitted in order not to obscure the concepts presented herein.

[0067] FIG. 5 schematically illustrates a computer program product 500, comprising a set of operations 520 executable by the control unit 130. The set of operations 520 may be loaded into the storage medium 430 in the control unit 130. The set of operations may correspond to the methods discussed above in connection to FIG. 3.

[0068] In the example of FIG. 5, the computer program product 500 is illustrated as an optical disc 510, such as a CD (compact disc) or a DVD (digital versatile disc) or a Blu-Ray disc. The computer program product could also be embodied as a memory, such as a random-access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), or an electrically erasable programmable read-only memory (EEPROM) and more particularly as a non-volatile storage medium of a device in an external memory such as a USB (Universal Serial Bus) memory or a Flash memory, such as a compact Flash memory. Thus, while the computer program is here schematically shown as a track on the depicted optical disk, the computer program can be stored in any way which is suitable for the computer program product.