Method and apparatus for embedding key information in an image

Abstract

The present disclosure relates to a method for embedding key information in an image, the method comprising reserving a range of DMZ values, in a predetermined range of 2.sup.N values used for storing useful data in the image, the reserved range being used for storing a key information associated with at least one coordinates in the image, with N>0 and DMZ<<2.sup.N.

Claims

1. A method for encoding an image corresponding to a map obtained after projection onto a parametric surface of a point cloud representing at least one part of an object of a multimedia content, the method comprising: reserving a first range of values within a predetermined second range of values used for storing values in pixels of the image, wherein the first range is used to store a key value indicating that the pixel is a non-useful pixel, and wherein the predetermined second range of values, deprived of the first range of values, is used to store useful values of the pixels of the image; and encoding, in a data stream, the image and information representative of the first range.

2. The method of claim 1, wherein the first range is the beginning of the second range.

3. The method of claim 1, wherein the first range is the end of the second range.

4. The method according to claim 1, comprising, for at least one pixel in the image: when a non-useful value is associated with the pixel, storing a key value in the pixel, the key value belonging to the first range; when a useful value is associated with the pixel: applying a modified quantization to the useful value for providing a modified useful value, the modified quantization being determined according to the number of values of the first range, and storing the modified value in the pixel.

5. The method of claim 1, wherein the first range is determined by: (a) encoding the image according to the first range; (b) decoding the image; and (c) when the decoded image comprises a number of artifacts greater than a given number, increasing the first range and iterating (a), (b), and (c).

6. A method for decoding an image corresponding to a map obtained after projection onto a parametric surface of a point cloud representing at least one part of an object of a multimedia content, the method comprising: decoding, from a data stream, the image and information representative of a reserved range provided for storing key information indicating that a pixel is a non-useful pixel; and for a value W read for at least one pixel in the image: when the value W belongs to the reserved range, discarding the value W; when the value W does not belong to the reserved range, recovering the useful value corresponding to the value W by applying an inverse modified quantization to the value W, the inverse modified quantization determined according to the reserved range.

7. A non-transitory computer-readable storage medium having instructions that, when executed by a processor, cause the processor to: generate data representative of an image embedding depth values, wherein the image stores key values in a first range reserved within a second range used to store useful values in the image, wherein the key values are associated with at least one pixel of the image, and wherein the second range of values, deprived of the first range of values, is used to store values of the pixels of the image; and generate information representative of the first range used to store key values indicating that a pixel is a non-useful pixel.

8. An apparatus configured to encode an image corresponding to a map obtained after projection onto a parametric surface of a point cloud representing at least one part of an object of a multimedia content, the apparatus comprising at least one processor and at least one memory having stored instructions operative, when executed by the at least one processor, to cause the apparatus to: reserve a first range of values within a second range of values used for storing useful values in pixels of the image, wherein the first range is used to store a key value indicating that the pixel is a non-useful pixel, and wherein the second range of values, deprived of the first range of values, is used to store useful values of the pixels of the image; and encode, in a data stream, the image and information representative of the first range.

9. The apparatus according to claim 8, wherein the instructions further cause the apparatus to, for at least one pixel in the image: when a non-useful value is associated with the pixel, store a key value in the pixel, the key value belonging to the first range; when a useful value is associated with the pixel: apply a modified quantization to the useful value for providing a modified useful value, the modified quantization being determined according to the number of values of the first range, and store the modified value in the pixel.

10. The apparatus of claim 8, wherein the first range is the beginning of the second range.

11. The apparatus of claim 8, wherein the first range is the end of the second range.

12. The apparatus according to claim 8, wherein the instructions further cause the apparatus to determine the first range by: (a) encoding the image according to the first range; (b) decoding the image; and (c) when the decoded image comprises a number of artifacts greater than a given number, increasing the first range and iterating (a), (b), and (c).

13. An apparatus configured to decode an image corresponding to a map obtained after projection onto a parametric surface of a point cloud representing at least one part of an object of a multimedia content, the apparatus comprising at least one processor and at least one memory having stored instructions operative, when executed by the at least one processor, to cause the apparatus to: decode, from a data stream, the image and information representative of a reserved range provided for storing a key information indicating that a pixel is a non-useful pixel; and for a value W read for at least one pixel in the image: when the value W belongs to the reserved range, discard the value W; when the value W does not belong to the reserved range, recover the useful value corresponding to the value W by applying an inverse modified quantization to the value W, the inverse modified quantization determined according to the reserved range.

14. A non-transitory computer-readable storage medium having instructions that, when executed by a processor, cause the processor to encode an image corresponding to a map obtained after projection onto a parametric surface of a point cloud representing at least one part of an object of a multimedia content, wherein the encoding comprises: reserving a first range of values within a predetermined second range of values used for storing values in pixels of the image, wherein the first range is used to store a key value indicating that the pixel is a non-useful pixel, and wherein the predetermined second range of values, deprived of the first range of values, is used to store useful values of the pixels of the image; and encoding, in a data stream, the image and information representative of the first range.

Description

4. BRIEF DESCRIPTION OF DRAWINGS

(1) Other features and advantages of embodiments shall appear from the following description, given by way of indicative and non-exhaustive examples and from the appended drawings, of which:

(2) FIGS. 1a and 1b illustrate respectively an example of images with undefined values and a key map, according to prior art;

(3) FIGS. 2a and 2b are respectively a flowchart of an embodiment of the disclosed method and an example of a reserved range of values according to an embodiment of the disclosed method; FIG. 2c illustrates curves corresponding respectively to a known quantization of useful data and to a modified quantization applied to useful data according to an embodiment of the disclosed method;

(4) FIG. 3 is a schematic illustration of the structural blocks of an exemplary apparatus that can be used for implementing the disclosed method according to the embodiment disclosed in relation with FIGS. 2a to 2c.

5. DESCRIPTION OF EMBODIMENTS

(5) In all of the figures of the present document, the same numerical reference signs designate similar elements and steps.

(6) The subject matter is now described with reference to the drawings, wherein like reference numerals are used to refer to like elements throughout. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the subject matter. It can be evident, however, that subject matter embodiments can be practiced without these specific details.

(7) The present description illustrates the principles of the present disclosure. It will thus be appreciated that those skilled in the art will be able to devise various arrangements that, although not explicitly described or shown herein, embody the principles of the disclosure.

(8) The present principles will be described in reference to a particular embodiment of a method of embedding key information in an image, so as to optimize the decoding of this image, as illustrated in FIGS. 2a and 2b.

(9) The general principle of the disclose method is based on reserving, in a predetermined range of 2.sup.N values used for storing useful data in an image, a smaller range of values (of size DMZ<<2.sup.N) for storing a key information. For example, the key information is associated with at least one coordinates in the image and need to be recognized as a value to be discarded on the decoding side, and not as a useful data.

(10) As no specific values are predetermined for embedding key information but a range of values is reserved and used, the disclosed method is thus robust to encoding artifacts. Indeed, even if the encoding modifies the key information, it should not vary outside the reserved range of values, so that, on decoding side, it can be recognized as a value to be discarded and not decoded.

(11) For that, the size DMZ of the reserved range may be adaptively refined depending on the compression level used for encoding the image (to be robust enough to encoding artifacts). Indeed, a too high value for DMZ (i.e. a too large reserved range of values for key information) may waste a part of the dynamic allowed to the useful data of the image whereas a too low value (i.e. a too small reserved range of values for key information) may induce too much encoding artifacts (bad transmission of the key information).

(12) The size DMZ of the reserved range may also be adaptively refined depending on the image itself, i.e. on the real dynamic needed for storing useful data in the image.

(13) Thus, a multi-pass strategy could be considered in order to set the optimal value of DMZ, for a given image and a given level of compression. For each pass, a candidate value would be tested, on the encoding side as well as on the decoding side, starting from a minimal initialization and increased progressively if too much encoding artifacts are observed. In any case the DMZ value should be known at the decoding time to properly decode the image information.

(14) For example, such multi-pass strategy may be implemented so as to adapt the size DMZ for each GOP (Group Of Picture).

(15) According to a particular embodiment of the disclosed method, the image embedding the key information, as well as useful data, corresponds to a distance map, for example obtained from previous steps of encoding a source image. Thus, in that case, the predetermined range of 2.sup.N values corresponds to the dynamic range of values used for storing distance values, for each coordinates in the image corresponding to a useful distance value.

(16) However, as this distance map/image may present some areas that contain undefined values (for example where no projection occurred) that should not be decoded, the reserved range of size DMZ thus corresponds to a part of this distance map dynamic, that will be used for storing a key information for each coordinates in the image with no associated distance value.

(17) A distance map/image may be obtained after a projection on a parametric surface of a 3D point cloud representing at least a part of an object in a multimedia content. Such a distance map/image contains the information of distance z between the projected point and the original 3D point. This information of distance z, when existing, is required on the decoding side to properly recover the point cloud and is classically represented as a clipped reverse function Z(z) of the distance so as to give more dynamic and favor the close distances. For example, this clipped reverse function Z(z) can be formulated as:

(18) $Z\left(z\right)=\frac{z_{\min }\left(\frac{z_{\max }}{z}-1\right)}{z_{\max }-z_{\min }}$

(19) where z.sub.min and z.sub.max define the clipping range with Z(z.sub.min)=1 and Z(z.sub.max)=0.

(20) This quantity Z(z) is then quantified and stored as an unsigned integer U(z), on N bits (typically 10 or 12), such as:
U(z)=floor(2.sup.NZ(z)).

(21) For example, U(z) is illustrated in FIG. 2c, in continuous line, for N=12, z.sub.min=1 and z.sub.max=5

(22) Thus, the disclosed method proposes to use a part of this dynamic given to U(z) to store the key information, when no useful data is associated with a coordinates in an image.

(23) For example, in a first variant illustrated in FIG. 2b, a reserved range of DMZ values is defined at the beginning of the admissible dynamic, with DMZ<<2.sup.N.

(24) Indeed, as explained above, it is still less prejudicial to reserve the beginning of the range values, as the first values are used for distant objects (according to the clipped reverse function Z(z)).

(25) In another variant, for example in case where the image do not correspond to a distance map, it can be better to use the end of the range values if the last values are used for storing less important information.

(26) As already said, the size DMZ may be adapted so as to anticipate the fact that the encoding step will change the values stored in the distance map, for example in a maximum range of +/−(DMZ/2). This maximum range of change is adaptively adjusted depending on the compression level. This is illustrated on FIG. 2b, in relation with the first variant, wherein the dotted vertical line represents the value DMZ that may vary to split up the predetermined range of 2.sup.N values into two ranges: the reserved range and the useful data range. Thus, the useful data stored in the distance image may take all the values from (DMZ+1) to (2.sup.N−1).

(27) Then, once the size DMZ has been defined, dedicated encoding and decoding process are implemented.

(28) For example, on the encoding side, let's note V the value to store in the distance map/image, for a given position/coordinates in the image. As already said, the value V may be a useful data corresponding to the distance information associated to a coordinates or a key information for the undefined areas.

(29) Thus, if no useful data is known for a current coordinates (e.g. no projection occurs at this specific part of the image) then a key information V.sub.k equal to 0 is stored in the distance image, 0 being a value belonging to the reserve range of size DMZ, at the beginning of the dynamic range of the distance values, in the first variant.

(30) Else, if a distance value is known for a current coordinates (i.e. a projection occurs at this specific part of the image), a useful data should be stored, in the useful data range, i.e. in the predetermined range of 2.sup.N values, except the beginning reserved for key information. Thus, in order to obtain the value to be stored, from the distance information, a modified quantization is applied to Z(z) and the value V.sub.u is stored, such as:
V.sub.u=U.sub.DMZ(z)=floor((2.sup.N−DMZ)Z(z)+DMZ).

(31) For example, U.sub.DMZ(z) is also illustrated in FIG. 2c, in dotted line, for N=12, z.sub.min=1 and z.sub.max=5.

(32) Thus, according to the disclosed method, for each coordinates in the distance image, a value V.sub.u or V.sub.k (equal to 0 in the first variant) is stored, depending on whether a projection has occurred or not for the corresponding point. Then, the encoded image is compressed using an image or video compression method.

(33) On the decoding side, for a received and decompressed image, let's note W the value read, possibly affected by a compression artifact. For a received image, the value DMZ is known by the decoder. The value DMZ may have been transmitted in the stream in association with the image. In a variant, a predefined value is set for DMZ. In another variant, the decoder implements a method to determine the value DMZ according to the received image.

(34) According to the disclosed method, if the value W belongs to the reserved range of size DMZ (i.e. if W<DMZ, in the first variant), then the current value W may correspond to an undefined/unused point or may result from an encoding error of a «true» point (i.e. a useful data close to the reserved range before encoding but modified by the compression and thus belonging to the reserved range when decompressed by the decoder). As a consequence, such a point is discarded and considered as unused by the decoder. Indeed, even if a “true” point is discarded, due to compression artifact, it is less prejudicial to “loose” few useful data, moreover if they correspond to distant objects, than interpreting an undefined data as a useful data.

(35) On the contrary, if the value W do not belong to the reserved range (i.e. if W>DMZ, in the first variant) then the current point is considered as a «valid» point and the associated distance value is recovered by computing U.sub.DMZ.sup.−1(W) (i.e. the inversed quantization as the quantization applied on the encoding size).

(36) Thus, according to the disclosed method, for each coordinates in a received distance image, a value W can be interpreted as a useful data or can be discarded if a key information is recognized.

(37) Thus, the disclosed method allows a compact storage of a key information in an image, while providing robustness to encoding artifacts in a content/image and level of compression adaptive way.

(38) For that, the disclosed method may take advantage of a dynamic range of values that could be considered as oversized regarding the values to be stored and/or the image itself and allows embedding a key information in an image without transmitting an additional image, thus optimizing the final bitrate for the image transmission.

(39) Referring now to FIG. 3, the structural blocks of an exemplary apparatus that can be used for implementing the method for embedding a key information in an image, is illustrated, according to any of the embodiments disclosed above in relation with FIGS. 2a to 2c.

(40) In an embodiment, an apparatus 300 for implementing the disclosed method comprises a non-volatile memory 303 (e.g. a read-only memory (ROM) or a hard disk), a volatile memory 301 (e.g. a random access memory or RAM) and a processor 302. The non-volatile memory 303 is a non-transitory computer-readable carrier medium. It stores executable program code instructions, which are executed by the processor 302 in order to enable implementation of the method described above in its various embodiments.

(41) Upon initialization, the aforementioned program code instructions are transferred from the non-volatile memory 303 to the volatile memory 301 so as to be executed by the processor 302. The volatile memory 301 likewise includes registers for storing the variables and parameters required for this execution.

(42) Naturally, the present disclosure is not limited to the embodiments previously described.

(43) In particular, the present disclosure is not limited to a method and device for embedding a key information in an image but also extends to a method for encoding/decoding a packet comprising data representative of an object of a multimedia content and to any device implementing this method and notably any devices comprising at least one CPU and/or at least one GPU.

(44) For example, the disclosed method may apply to volumetric or immersive videos.

(45) The implementations described herein may be implemented in, for example, a method or a process, an apparatus, a computer program product, a data stream, or a signal. Even if only discussed in the context of a single form of implementation (for example, discussed only as a method or a device), the implementation of features discussed may also be implemented in other forms (for example a program). An apparatus may be implemented in, for example, appropriate hardware, software, and firmware. The methods may be implemented in, for example, an apparatus such as, for example, a processor, which refers to processing devices in general, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include communication devices, such as, for example, Smartphones, tablets, computers, mobile phones, portable/personal digital assistants (“PDAs”), and other devices that facilitate communication of information between end-users.

(46) Implementations of the various processes and features described herein may be embodied in a variety of different equipment or applications, particularly, for example, equipment or applications associated with data encoding, data decoding, view generation, texture processing, and other processing of images and related texture information and/or depth information. Examples of such equipment include an encoder, a decoder, a post-processor processing output from a decoder, a pre-processor providing input to an encoder, a video coder, a video decoder, a video codec, a web server, a set-top box, a laptop, a personal computer, a cell phone, a PDA, and other communication devices. As should be clear, the equipment may be mobile and even installed in a mobile vehicle.

(47) Additionally, the methods may be implemented by instructions being performed by a processor, and such instructions (and/or data values produced by an implementation) may be stored on a processor-readable medium such as, for example, an integrated circuit, a software carrier or other storage device such as, for example, a hard disk, a compact diskette (“CD”), an optical disc (such as, for example, a DVD, often referred to as a digital versatile disc or a digital video disc), a random access memory (“RAM”), or a read-only memory (“ROM”). The instructions may form an application program tangibly embodied on a processor-readable medium. Instructions may be, for example, in hardware, firmware, software, or a combination. Instructions may be found in, for example, an operating system, a separate application, or a combination of the two. A processor may be characterized, therefore, as, for example, both a device configured to carry out a process and a device that includes a processor-readable medium (such as a storage device) having instructions for carrying out a process. Further, a processor-readable medium may store, in addition to or in lieu of instructions, data values produced by an implementation.

(48) As will be evident to one of skill in the art, implementations may produce a variety of signals formatted to carry information that may be, for example, stored or transmitted. The information may include, for example, instructions for performing a method, or data produced by one of the described implementations. For example, a signal may be formatted to carry as data the rules for writing or reading the syntax of a described embodiment, or to carry as data the actual syntax-values written by a described embodiment. Such a signal may be formatted, for example, as an electromagnetic wave (for example, using a radio frequency portion of spectrum) or as a baseband signal. The formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information that the signal carries may be, for example, analog or digital information. The signal may be transmitted over a variety of different wired or wireless links, as is known. The signal may be stored on a processor-readable medium.

(49) A number of implementations have been described. Nevertheless, it will be understood that various modifications may be made. For example, elements of different implementations may be combined, supplemented, modified, or removed to produce other implementations. Additionally, one of ordinary skill will understand that other structures and processes may be substituted for those disclosed and the resulting implementations will perform at least substantially the same function(s), in at least substantially the same way(s), to achieve at least substantially the same result(s) as the implementations disclosed. Accordingly, these and other implementations are contemplated by this application.