Abstract
A method for securely displaying ASIL-relevant data on a display device of a motor vehicle, where the device includes a transmitter unit, a receiver unit, and a display unit to show the images generated by the receiver unit. Images having higher safety ratings are shown on the display unit with less modification by an image enhancement process in comparison with images having lower safety ratings. Information about the safety rating of an image to be displayed is introduced by the transmitter unit into a data stream by a classifying binary code in color bit information of at least one pixel of the image to be displayed via a pixel matrix, in order to control a change of the image during the image enhancement process in an image enhancer associated with an image-data-outputting receiver unit for images of all safety ratings.
Claims
1-9. (canceled)
10. A method for securely displaying ASIL-relevant data on a display device of a motor vehicle, the display device comprising a transmitter unit (110), a receiver unit (120), and a display unit (130) configured to display images output by the receiver unit (120), wherein images having higher safety ratings are shown on the display unit with less modification by an image enhancement process in comparison with images having lower safety ratings, and information about the safety rating of an image to be displayed is introduced by the transmitter unit (110) into a data stream by a classifying binary code in color bit information of at least one pixel of the image to be displayed via a pixel matrix to control a change to the image during the image enhancement process in an image enhancer associated with the image-data-outputting receiver unit for images of all safety ratings.
11. The method of claim 10, wherein prior to transmission of the image to be displayed comprising the pixel to the display unit (130), the classifying binary code of the at least one pixel is read out from the data stream received from the receiver unit (120) that outputs the image data, wherein, depending on the binary code of the at least one pixel, a decision is made about performing an image enhancement.
12. The method of claim 10, wherein a read-out binary code of the at least one pixel is replaced by the color value bit originally stored in the pixel prior to the binary code, wherein the image to be displayed is modified in accordance with an associated classifying binary code in the image enhancement process.
13. The method of claim 10, wherein color value bits of the image to be displayed that are output after the image enhancement process are checked to establish whether the image to be output is output in accordance with safety ratings specified by the associated binary code.
14. The method of claim 10, wherein one bit is reserved in a specified color of RGB information for the binary code.
15. The method of claim 10, wherein one bit of each color of RGB information is reserved for the binary code.
16. The method of claim 10, wherein one bit of a specified color is reserved alternately in data frames for the binary code.
17. The method of claim 10, wherein for the binary code, additional least significant bit data is provided in a frame with a specified color depth or bits are reserved in a supplementary channel of an RGBA or RGBW color encoding.
18. A device for securely displaying ASIL-relevant data on a display device of a motor vehicle, the device comprising: an image-data-generating transmitter unit; an image-data-outputting receiver unit, which is configured to communicate with the image-data-generating transmitter unit via a data stream; and a display unit configured to display images output by the receiver unit, wherein the image-data-generating transmitter unit is configured to introduce additional information about a safety rating of image data to be displayed into a pixel of the image data as classifying binary code to control a modification of the image during an image enhancement process in an image enhancer associated with the image-data-outputting receiver unit for images of all safety ratings.
Description
BRIEF DESCRIPTION OF THE DRAWING FIGURES
[0017] In the drawing:
[0018] FIG. 1 shows a schematic diagram of the device according to the invention,
[0019] FIG. 2 shows an exemplary embodiment of the method according to the invention with the device as per FIG. 1 with a two-stage functional safety process in the receiver unit,
[0020] FIG. 3 shows an exemplary embodiment of a flowchart in the receiver unit,
[0021] FIG. 4 shows an exemplary embodiment for processing ASIL-classifying pixels in the receiver unit,
[0022] FIG. 5 shows an exemplary embodiment for processing non-ASIL-classifying pixels in the receiver unit, and
[0023] FIG. 6 shows an exemplary embodiment of introducing a classifying bit and subsequently replacing it with a bit of the original image data.
DETAILED DESCRIPTION
[0024] FIG. 1 shows a schematic diagram of the device according to the invention. The device 100 consists of a transmitter unit 110 and a receiver unit 120, both of which process image data that is displayed in a display unit 130, which is part of the receiver unit 120. The transmitter unit 110, which is a rendering unit for generating an image from raw data, comprises a first layer 140, in which non-critically classified pixel layers are introduced into an image. In a second layer 150, which is a secure unit, safety-critically classified picture elements are introduced into the image. The criticality of the pixel layers used to generate the images depends, for example, on whether the images render displays for an ASIL-relevant system such as a system for automated driving or for non-secure systems such as an entertainment system. The image content, which is rendered in full by the transmitter unit 110 and classified at pixel level, is formed in a serializer 160 into a compressed data stream, which is transmitted via a video transmission link 170 to the receiver unit 120, where it is decompressed again by means of a deserializer 180 and converted into image information. The receiver unit 120 receives the transmitted image information by means of a controller 190. Depending on whether the pixels of the image information are safety-critical (ASIL-relevant) or non-critical, which is determined by a pixel content mapping that enables a classification into safety ratings, the image to be displayed is modulated in an image enhancer (125), i.e., in an image enhancement process, or not, and supplied to the display unit 130 of the receiver unit 120.
[0025] FIG. 2 shows an exemplary embodiment of the method according to the invention with the device 100 as per FIG. 1 with a two-stage functional safety process in the receiver unit 120. In function 1, internal and external information relevant to visualization, preferably vehicle information, is collected in the transmitter unit 110. The visualization-relevant parameters are linked to the image information, in this case a pixel color content, for the image synthesis. Subsequently, in function 2, the image information is merged, i.e., rendered, with metadata, which includes a safety rating of the binary code classifying image information, for pixel-precise type casting to form image information. Following the transmission of the rendered image information from the transmitter unit 110 to the receiver unit 120 (function 3), a pixel-by-pixel separation of the data stream of the image to be displayed into display information (color and position) and metadata (classification and position) takes place in function 4. The metadata provides a pixel-precise statement about ASIL-relevant classifications in safety ratings, while the display information determines the pixel color. In a first functional safety level 5, the rating classification of each pixel is checked. In response to detected ASIL-relevant metadata, the system receives corresponding feedback with rules for the image enhancement process and the display process according to predefined parameters, derived e.g., in a lookup table. These rules are supplied to the image enhancement process and the display unit 130. Depending on these rules and the results of the functional safety level 5, in function 6 the image enhancement process processes each pixel as per the applicable rules as a function of the pixel classification. In accordance with function 7 it is checked whether the applicable rules of the pixel-based modification or optimization for the intended display have been complied with. This involves a pixel-by-pixel comparison of the values received by the receiver unit with the values to be output to the display unit, for example with a hash value algorithm. If the values match, in function 8 the image information is transferred to the drivers for visualization in the display unit.
[0026] As described, for each frame transmitted between the transmitter unit 110 and the receiver unit 120 on the video transmission link 170, the transmitter unit 110 adds information to the pixels of the image as to what type of image it is. This is done by an RGB pixel classification mapping. The classification at RGB pixel level depends on the number of bits provided and is carried out in the respective LSB data (Least Significant Bit data) of the RGB matrix. The classification possibilities shall be illustrated using an 8-bit RGB pixel:
[0027] Hereinafter, some possible characteristics of the RGB-based pixel formatting scheme will be explained by way of example. [0028] 1. Reservation of one LSB pixel in the color blue B, resulting in two classification states [0029] R=8, G=8, B=8.fwdarw.R=8, G=8, B=7+x RRRRRRRR GGGGGGGG BBBBBBBB.fwdarw.RRRRRRRR GGGGGGGG BBBBBBBx [0030] 2. Reservation of 1 bit per color, resulting in 8 classification states [0031] R=8, G=8, B=8.fwdarw.R=7+x, G=7+x, B=7+x RRRRRRRR GGGGGGGG BBBBBBBB.fwdarw.RRRRRRRx GGGGGGGx BBBBBBBx [0032] 3. Reservation of 1 bit of one color per data frame, resulting in 8 classification states over three data frames. This is a frame-based iterative pixel classification without significant image losses. With each frame, the classification bit alternates between the bit code of the colors red, green and blue. [0033] Data frame 1 [0034] R=8, G=8, B=8.fwdarw.R=7+x, G=8, B=8 RRRRRRRR GGGGGGGG BBBBBBBB.fwdarw.RRRRRRRx GGGGGGGG BBBBBBBB [0035] Data frame 2: [0036] R=8, G=8, B=8.fwdarw.R=8, G=7+x, B=8 RRRRRRRR GGGGGGGG BBBBBBBB.fwdarw.RRRRRRRR GGGGGGGx BBBBBBBB [0037] Data frame 3: [0038] R=8, G=8, B=8.fwdarw.R=8, G=8, B=7+x RRRRRRRR GGGGGGGG BBBBBBBB.fwdarw.RRRRRRRR GGGGGGGG BBBBBBBx [0039] 4. Additional provision of LSB information in the frame with 10-bit color depth, resulting in classification states 1 to N [0040] R=10, G=10, B=10.fwdarw.R=10n+n*x, G=10n+n*x, B=10n+n*x; for n=3 [0041] RRRRRRRRRR GGGGGGGGGG BBBBBBBBBB.fwdarw.RRRRRRRRxxx GGGGGGGGxxx BBBBBBBBxxx [0042] 5. Additional provision of meta-information by encoding in an additional channel (alpha and white) [0043] R=8, G=8, B=8, A=8.fwdarw.R=8, G=8, B=8, A=(A+x)=8 RRRRRRRR GGGGGGGG BBBBBBBB aaaaaaaa.fwdarw.RRRRRRRR GGGGGGGG BBBBBBBB aaaaxxxx
In all classification cases presented, x stands for the reserved bit for the LSB formatting.
[0044] FIG. 3 shows an exemplary embodiment of a flowchart in the controller 190 of the receiver unit 120. In block 200, the entire image is read in the form of matrix pixel data. The individual pixels are extracted in block 210 and read in in a checking process in block 220. Subsequently, in block 230, the individual pixels are checked to determine which pixel type they are. If safety-critical pixel content is detected using a classifying binary code, the next step is block 240, in which restrictive criteria for a pixel manipulation are defined. This can be, for example, a protection against changes. The process then continues to block 250, to which the data of pixels whose non-critical content was detected in block 230 is also forwarded. Image enhancement of the image data takes place in this block 250 if the checked pixels comprise non-critical pixel content. If a pixel with safety-critical content has been detected in block 230, the image data is only processed to a limited extent within the scope of the permitted restrictive criteria. The process proceeds from block 250 to block 260, where a check is made for compliance with the applicable check criteria, which are also defined in block 240 for the pixels with safety-critical content. In block 270, the pixel classification is removed from the image data, which data is replaced by the original single pixel data in block 280.
[0045] FIG. 4 shows an exemplary embodiment for processing ASIL-classified pixels in the receiver unit 120. In function 3, as already explained, the image data is transmitted from the transmitter unit 110 to the receiver unit 120 and converted into an entire image pixel matrix in the receiver unit 120. In function 4, the entire image is separated pixel by pixel. In the following, for example the 8th bit of the blue value, (B8), is considered, which comprises as display information (color 00000000 00000000 00000001) and metadata (classification 1). In the first functional level 5, the rating classification of the bit B8 is checked. The result of the evaluation gives classification level 1=ASIL pixel. In the function step 6, the classification bit is replaced by a buffered bit associated with the original image information, i.e., in the present case the bit B8 is replaced, i.e., the bit is reset from 1 to the image value 0. Further, in function 6, the image enhancement process is performed to process the pixels of the image information RGB according to the applicable criteria depending on the parameters given by the ASIL pixel classification. Thereafter, the binary code 1 means that no pixel image enhancement of the image associated with bit B8 may be performed (first functional safety level 5). A check is performed in function 7 as to whether the criteria for pixel-based modification or optimization specified in the first functional safety level 5 have been complied with. If the result is positive, the image data is transferred in function 8 to the drivers for visualization in the display unit 130. If the result is negative, an emergency strategy is adopted, for example the function of the image enhancer is switched off, a replacement value is formed or a message about a malfunction is issued to a vehicle user.
[0046] FIG. 5 shows an exemplary embodiment for processing non-ASIL-classified pixels in the receiver unit 120. The functions 3 and 4 correspond to those explained in connection with FIG. 4. In this case, it is not necessary to exchange the classification bit for a color value bit of the original image, as both have the value 0. However, in this case, the classifying bit B8 includes a 0 as classification bit information, which indicates a non-safety-critical content, which is why changes to the image associated with the bit B8 are permitted during the image enhancement process, which also takes place in function 6. Here, too, function 7 is used to check the criteria specified in the first functional safety level 5. Since the image that comprises pixel B8 has non-critical content, there are no additional checking criteria, so the pixel values are output to the display unit 130 in the modified-in-function-6, i.e., image-enhanced, form.
[0047] As shown in more detail in FIG. 6, the meta-information is replaced by its original color information of the last or previous image, which is done, in particular, by means of buffering a number of possibly changed color value bits per pixel in the controller 190. In the present example, the buffering is explained using the pixel formatting scheme described in FIG. 3 above, in which the classification bit is alternately written into the bit information of the RGB color values. In the LSB bit of the color blue, the ASIL classification bit 1 is encoded as a classifying binary code as per the description with regard to FIG. 2 in function 2 or in the associated method step. The LSB color value bit of the original pixel of blue that was valid before encoding is buffered with the color value bits of red and green, as indicated by an arrow, see FIG. 6a. The classification bit is read out to define the ASIL relevance of the pixel or image associated with the bit in accordance with function 5 in FIG. 4. In function 6, the classification bit is overwritten again by the buffered bit for the color value, i.e., the classification bit is replaced by the value 0 as shown in FIG. 6b. The original pixel or the original image associated with the pixel that is thus present prior to the encoding is treated in the image enhancer in accordance with the read-out classification bit, i.e., output in the present case without enhancement. Accordingly, a classification bit encoded in the color red or blue is replaced by the buffered LSB bits of the original image prior to transmission to the image enhancer.
[0048] Although the invention has been illustrated and described in detail by way of preferred embodiments, the invention is not limited by the examples disclosed, and other variations can be derived from these by the person skilled in the art without leaving the scope of the invention. It is therefore clear that there is a plurality of possible variations. It is also clear that embodiments stated by way of example are only really examples that are not to be seen as limiting the scope, application possibilities or configuration of the invention in any way. In fact, the preceding description and the description of the figures enable the person skilled in the art to implement the exemplary embodiments in concrete manner, wherein, with the knowledge of the disclosed inventive concept, the person skilled in the art is able to undertake various changes, for example, with regard to the functioning or arrangement of individual elements stated in an exemplary embodiment without leaving the scope of the invention, which is defined by the claims and their legal equivalents, such as further explanations in the description.
