APPARATUS FOR RECORDING DATA TO PRODUCE A LOCALIZED PANORAMIC IMAGE OF A STREET AND METHOD RELATED THERETO

Abstract

The invention relates to an apparatus for recording data to produce a localized panoramic image of a street with a camera and a device for satellite-based position and time determination as well as a storage unit. According to the invention a preparation device is provided which encodes time data of the device for satellite-based position and time determination into a format recordable for the camera and forwards this to the camera, wherein the camera is designed to simultaneously record these data and record a continuous film. The storage unit stores the film with the time data as well as position and time data of the device for satellite-based position and time determination. Furthermore, the invention relates to a method for this.

Claims

1. An apparatus for recording data to produce a localized panoramic image of a street with a camera, with a device for satellite-based position and time determination, with a storage unit, wherein a preparation device is provided which is designed to prepare time data of the device for satellite-based position and time determination into a format recordable for the camera as coded time data and to forward this to the camera for recording, in that the camera is designed to simultaneously record a continuous film and the coded time data as a film with time data, in that the storage unit is designed and configured to store the film with coded time data, in that the storage unit is designed and configured to store position and time data of the device for satellite-based position and time determination, and the preparation device is configured to produce the coded time data in a SMPTE ltc-compliant manner whilst inserting data on the millisecond and microsecond as user bit load.

2. The apparatus according to claim 1, wherein the preparation device is configured to produce the coded time data as acoustic signal and forward this to the camera.

3. The apparatus according to claim 1, wherein the preparation device is configured to produce the coded time data as a continuous time data stream with a frame for each time-stamp.

4. The apparatus according to claim 1, wherein the preparation device is configured to produce the coded time data in a SMPTE ltc-compliant manner.

5. The apparatus according to claim 1, wherein the preparation device is configured to adjust the coded time data by means of a time signal, in particular a seconds signal, of the device for satellite-based position and time determination.

6. The apparatus according to claim 1, wherein for adjustment of the coded time data the preparation device is configured to stop and/or repeat one or several frames in order to adjust the coded time data by means of the time signal of the device for satellite-based position and time determination.

7. The apparatus according to claim 1, wherein a LIDAR sensor unit with a LIDAR sensor is provided which is designed to receive position and/or time data from the device for satellite-based position and time determination and which is configured to transmit distance data of the LIDAR sensor together with position and time data of the device for satellite-based position and time determination.

8. A method for recording data to produce a localized panoramic image of a street with a camera, with a device for satellite-based position and time determination and with a preparation device, wherein the preparation device prepares time data of the device for satellite-based position and time determination into a format recordable for the camera as coded time data and forwards this to the camera, wherein the camera produces and issues a continuous film simultaneously with the coded time data as a film with time data.

9. The method according to claim 9, wherein the coded time data are produced as acoustic signal and recorded on the audio track of the camera.

10. The method according to claim 9, wherein the coded time data are produced as a continuous time data stream with a frame for one time-stamp in each case.

11. The method according to claim 9, wherein the coded time data are SMPTE ltc-compliant and in that as user bit load data on the millisecond and microsecond are inserted.

12. The method according to claim 9, wherein the coded time data are adjusted by means of a time signal, in particular a seconds signal, of the device for satellite-based position and time determination.

13. The method according to claim 9, wherein for adjustment of the coded time data one or several frames are stopped and/or repeated.

14. The method according to claim 9, wherein a localized panoramic image of a street is produced from the continuous film with the coded time data whilst using the position and time data of the device for satellite-based position and time determination.

Description

[0039] The invention is explained in greater detail hereinafter by way of a schematic, exemplary embodiment with reference to the Figures, wherein show:

[0040] FIG. 1 a schematic view of an apparatus according to the invention for recording data to produce a localized panoramic image of a street;

[0041] FIG. 2 the basic structure of a time data stream according to the invention;

[0042] FIG. 3 the principle of dropping a time data stream frame;

[0043] FIG. 4 the principle of renewed transmission of a time data frame; and

[0044] FIG. 5 a highly simplified panoramic image of a street.

[0045] FIG. 1 shows a highly schematic, simplified illustration of an apparatus 10 according to the invention for recording data to produce a localized panoramic image of a street 60.

[0046] This apparatus 10 has a camera 12 and a device 14 for satellite-based position and time determination. The device 14 for satellite-based position and time determination is a GPS-receiver for example. In the case of a GPS signal different satellites transmit a high-precision time signal, from which the corresponding position can then be ascertained in the GPS-apparatus or GPS-receiver.

[0047] Furthermore, in the apparatus 10 according to the invention a preparation device 18 is provided, which will be dealt with in greater detail in the following. In addition, the apparatus 10 according to the invention has a LIDAR sensor unit 22 with a LIDAR sensor 23. Both the device 14 for satellite-based position and time determination and the camera 12 as well as the LIDAR sensor unit 22 are connected to a storage unit 16.

[0048] In the following the functioning of the apparatus 10 according to the invention will be dealt with in greater detail. The objective is to record data with the apparatus 10 according to the invention to produce a localized panoramic image of a street 60. For this purpose, the apparatus 10 according to the invention is placed in a vehicle that moves on a street. While the vehicle is moving a film is recorded by means of the camera 12. In addition to the actual image data, position and time data are ascertained via the device 14 for satellite-based position and time determination, which are to be harmonized with the film so that this information or these data that are stored on the storage unit 16 can subsequently be prepared for a localized panoramic image of a street 60.

[0049] For this purpose, at least a time signal of the device 14 for satellite-based position and time determination is transmitted to the preparation device 18. In the latter, the time signal is converted into a time data stream which is in turn present as audio stream. The precise structure of this audio stream will be dealt with later on with reference to FIGS. 2, 3 and 4.

[0050] This audio stream is transmitted to the camera 12 to an audio input present there. In the audio stream high-precision information on the current time is contained. The camera then records its film, with the audio data being present on the audio track as acoustic signal. This film with the acoustic signal is then stored on the storage unit 16. For reasons of data protection for example this can be an encrypted memory or a server that directly stores the data in encrypted form so that they do not exist in raw form.

[0051] In this way, it is possible to establish and store the precise point in time of the production of the film, i.e. of an image of the film.

[0052] Parallel to this the precise position, just as the precise time, is also transmitted from the device 14 for satellite-based position and time determination to the storage unit 16 and stored there.

[0053] Thus, in an evaluation implemented downstream it is possible by way of the precise time code on the film and the parallel time code of the position data to determine the position, at which each corresponding image of the film was produced.

[0054] In addition, in the illustrated embodiment the apparatus 10 according to the invention has the LIDAR sensor unit 22 with a LIDAR sensor 23. However, this is not absolutely necessary.

[0055] In the embodiment illustrated here at least time data are also transmitted from the device 14 for satellite-based position and time determination to the LIDAR sensor unit 22. The LIDAR sensor unit 22 prepares these data together with the distance, ascertained by the LIDAR sensor 23 from the LIDAR sensor to the next object, and also stores these by means of the storage unit 16.

[0056] Basically, it is also possible to transmit not only the time data of the device 14 for satellite-based position and time determination but also the position data. In this case either only the position data with the corresponding distance data or the time data, the position data and the distance data would be prepared by the LIDAR sensor unit 22 and stored in the storage unit 16. In analogy to the film data, these data can then be used to obtain further information for the panoramic image of a street. These too are of highly accurate localized nature so that the precise position can be determined.

[0057] In the following the simplified structure of the time data stream 30 is explained in greater detail with reference to FIG. 2. Basically, the time data stream 30 constitutes an audio signal that codes a bit-sequence 37. The audio stream or time data stream is preferably designed to be SMPTE-ltc-compliant. According to this standard a specific point in time, i.e. a time-stamp, is in each case coded per frame 31, 32, 33, 34, 35, 36. The simplified time data stream 30 illustrated in FIG. 2 only has five frames per second. In line with this standard, in each frame the current hour, minute and second as well as the current frame is coded in relation to the recorded data. In simplified terms, in the case of a recording with 24 images per second this means the information to which image in the respective second the corresponding frame 31, 32, 33, 34, 35, 36 belongs.

[0058] According to the invention this format was extended in that in the user bit load provided according to the standard, data on the millisecond and microsecond are additionally inserted.

[0059] The respective time-stamp that is coded in each frame 31, 32, 33, 34, 35, 36 always corresponds to the start of the respective frame t1, t2, t3, t4, t5, t6. For the sake of simplification, it is assumed that only five images are produced per second, i.e. five frames are present in one second.

[0060] The preparation device 18 has a quartz crystal to produce the high-precision time signal that serves as a basis for the time data stream 30. Due to the fact that quartz crystals, just as other time measurement devices, are often temperature-dependent it is necessary to adjust the time in each case. Especially in the application presented here this is necessary because otherwise the high-accuracy localization of the images cannot be achieved. For adjustment the time signal of the device 14 for satellite-based position and time determination is used for example. In this case, a seconds signal is used for this. This means that the device 14 for satellite-based position and time determination transmits a signal to the preparation device 18 every second. By means of this signal the time data stream 30 is then newly adjusted in each case.

[0061] In the following the two cases are described in FIGS. 3 and 4, in which the time elapses too slowly or too fast on the preparation device 18.

[0062] According to FIG. 3 the time calculated by the preparation device 18 is too slow. As a consequence, the second has already elapsed before the fifth frame 35 is terminated. According to the invention provision is then made for the transmission of this frame 35 to be stopped immediately and for transmission of the next, correct frame, i.e. frame 36 to be started.

[0063] Conversely, in FIG. 4 the case is illustrated if the preparation device 18 runs too slowly, i.e. if its time is too slow. In this case the frame 36 already starts before the next second has elapsed. This too is detected by means of the seconds signal emitted by the device 14 for satellite-based position and time determination. As a consequence, the transmission of the frame 36 is in turn stopped but is then started anew so that the time data stream 30 again runs synchronously to the time of the device 14 for satellite-based position and time determination.

[0064] Finally, in FIG. 5 a highly schematic example of a produced panoramic image of a street 60 is illustrated. When preparing the data the continuous film data produced by means of the camera 12 are prepared and assembled to a panoramic image of a street 60 that has no or hardly any optical distortions where possible. On the panoramic image of a street 60 individual objects 64, such as cars or trucks located on parking spaces but also signs 65 can then be recognized. On these signs 65 parking information can be present for example. Likewise, complete streetlamps 66 or sections thereof are illustrated for example. What is important here is that the curbside is present in order to ascertain a sufficient amount of data for static information for the parking management system. The objects shown hatched in FIG. 5 are for example houses 67 located in the background.

[0065] By way of the apparatus according to the invention employed pursuant to the method according to the invention it is thus possible to efficiently generate data that can be used to produce a localized panoramic image of a street.