METHODS AND IMAGE PROCESSING DEVICES FOR ENCODING AND DECODING PRIVATE DATA

Abstract

Methods and image processing devices for encoding and decoding private data are proposed. The method for encoding private data includes to receive an original video frame, mask at least one private area in the original video frame to generate a protected video frame, generate a first encoded frame by encoding the protected video frame, and generate at least one output bitstream for streaming or storage according to the first encoded frame. The method for decoding private data includes to receive at least one input video bitstream to obtain a first encoded bitstream and a second encoded bitstream, decode the first encoded bitstream to generate a protected video frame including image data associated with at least one private area, and output the protected video frame to a display queue such that the at least one private area is displayed.

Claims

1. A method for encoding private data comprising: receiving an original video frame; masking at least one private area in the original video frame to generate a protected video frame; generating a first encoded frame by encoding the protected video frame; and generating at least one output bitstream for streaming or storage according to the first encoded frame.

2. The method of claim 1 further comprising: generating a second encoded frame by encoding the original video frame, and wherein the step of generating the at least one output bitstream for streaming or storage according to the first encoded frame comprises: generating the at least one output bitstream for streaming or storage according to the first encoded frame and the second encoded frame.

3. The method of claim 2 further comprising: encrypting the second encoded frame to generate an encrypted second encoded frame, wherein the step of generating the at least one output bitstream for streaming or storage according to the first encoded frame comprises: generating the at least one output bitstream for streaming or storage according to the first encoded frame and the encrypted second encoded frame.

4. The method of claim 3, wherein the at least one output bitstream comprises a first output bitstream and a second output bitstream, and wherein the step of generating the at least one output bitstream for streaming or storage according to the first encoded frame and the second encoded frame comprises: packing the first encoded frame into the first output bitstream; and packing the encrypted second encoded frame into the second output bitstream.

5. The method of claim 3, wherein the at least one output bitstream comprises a combined output bitstream, and wherein the step of generating the at least one output bitstream for streaming or storage according to the first encoded frame and the second encoded frame comprises: packing the first encoded frame and the encrypted second encoded frame into the combined output bitstream.

6. The method of claim 2, wherein a reference frame is generated when the protected video frame is encoded, and the original video frame is encoded by using the reference frame.

7. The method of claim 1 further comprising: duplicating the original video frame to generate a duplicated original video frame; and performing masking on the duplicated original video frame.

8. An image processing device comprising: a receiving circuit, configured to receive an original video frame; a masking circuit, configured to mask at least one private area in the original video frame to generate a protected video frame; an encoding circuit, configured to generating a first encoded frame by encoding the protected video frame; and an output circuit, configured to generate at least one output bitstream for streaming or storage according to the first encoded frame.

9. The image processing device of claim 8, wherein the encoding circuit is further configured to generate a second encoded frame by encoding the original video frame, and wherein the output circuit is configured to generate the at least one output bitstream for streaming or storage according to the first encoded frame and the second encoded frame.

10. The image processing device of claim 9 further comprising: an encrypting circuit, configured to encrypt the second encoded frame to generate an encrypted second encoded frame, wherein the output circuit is further configured to generate the at least one output bitstream for streaming or storage according to the first encoded frame and the encrypted second encoded frame.

11. The image processing device of claim 10, wherein the at least one output bitstream comprises a first output bitstream and a second output bitstream, and wherein the output circuit is configured to generate the at least one output bitstream for streaming or storage according to the first encoded frame and the second encoded frame by packing the first encoded frame into the first output bitstream and packing the encrypted second encoded frame into the second output bitstream.

12. The image processing device of claim 10, wherein the at least one output bitstream comprises a combined output bitstream, and wherein he output circuit is configured to generate the at least one output bitstream for streaming or storage according to the first encoded frame and the second encoded frame by packing the first encoded frame and the encrypted second encoded frame into the combined output bitstream.

13. The image processing circuit of claim 9, wherein the encoding circuit is configured to generate a reference frame when encoding the protected video frame, and wherein the encoding circuit is configured to encode the original video frame by using the reference frame.

14. The image processing circuit of claim 8, wherein the receiving circuit is further configured to duplicate the original video frame to generate a duplicated original video frame, and wherein the masking circuit is configured to perform masking on the duplicated original video frame.

15. An image processing device comprising: a processor, configured to: receive an original video frame; mask at least one private area in the original video frame to generate a protected video frame; generate a first encoded frame by encoding the protected video frame; and generate at least one output bitstream for streaming or storage according to the first encoded frame.

16. A method for decoding private data comprising: receiving at least one input video bitstream to obtain a first encoded bitstream and a second encoded bitstream; decoding the first encoded bitstream to generate a protected video frame comprising image data associated with at least one private area; and outputting the protected video frame to a display queue such that the at least one private area is displayed.

17. The method of claim 16, further comprising: decoding the first encoded bitstream to generate a reference frame not having the image data of at least one private area; and outputting the reference frame to the display queue.

18. The method of claim 17, wherein the second encoded bitstream is decoded by using the reference frame.

19. The method of claim 17 further comprising: decrypting the second encoded bitstream when a key is authorized before the second encoded bitstream is decoded.

20. The method of claim 19, wherein both the first encoded bitstream and the second bitstream are decoded to generate both the reference frame and the protected video frame which are output to the display queue when the key is authorized such that a whole image is displayed without the at least one private area being blurred.

21. The method of claim 19 further comprising: not decrypting the second encoded bitstream when the key is not authorized.

22. The method of claim 21, wherein only the first encoded bitstream is decoded and only the reference frame is output to the display queue and the second encoded bitstream is not decoded and the protected video frame is not output to the display queue when the key is not authorized, such that the at least one private area is blurred when only the reference frame is being displayed.

23. An image processing device comprising: a receiving circuit, configured to receive at least one input video bitstream to obtain a first encoded bitstream and a second encoded bitstream; a decoding circuit, configured to decode the first encoded bitstream to generate a protected video frame comprising image data of at least one private area; and an output circuit configured to output the protected video frame to a display queue such that the at least one private area is displayed.

24. The image processing device of claim 23, wherein the decoding circuit is further configured to decode the first encoded bitstream to generate a reference frame not having the image data of at least one private area, and wherein the output circuit is further configured to output the reference frame to the display queue.

25. The image processing device of claim 24, wherein the decoding circuit is further configured to the second encoded bitstream by using the reference frame.

26. The image processing device of claim 24 further comprising: a decrypting circuit, configured to decrypt the second encoded bitstream when a key is authorized before the second encoded bitstream is decoded.

27. The image processing device of claim 25, wherein the decoding circuit is configured to decode both the first encoded bitstream and the second bitstream to generate both the reference frame and the protected video frame, and wherein the output circuit is configured to output both the reference frame and the protected video frame to the display queue when the key is authorized such that a whole image is displayed without the at least one private area being blurred.

28. The image processing device of claim 26, wherein the decrypting circuit is not configured to decrypt the second encoded bitstream when the key is not authorized.

29. The image processing device of claim 28, wherein the decoding circuit is configured to decode only the first encoded bitstream, and wherein the output circuit is configured to output only the reference frame to the display queue when the key is not authorized, such that the at least one private area is blurred when only the reference frame is being displayed.

30. An image processing device comprising: a processor, configured to: receive at least one input video bitstream to obtain a first encoded bitstream and a second encoded bitstream; decode the first encoded bitstream to generate a protected video frame comprising image data of at least one private area; and output the protected video frame to a display queue such that the at least one private area is displayed.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings are included to provide a further understanding of the disclosure, and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0017] FIG. 1 illustrates a schematic diagram of a street view image with face blur in accordance with an existing technology.

[0018] FIG. 2A illustrates a schematic diagram of a first conventional approach for face blur in NVR.

[0019] FIG. 2B illustrates a schematic diagram of a second conventional approach for face blur in NVR.

[0020] FIG. 3A and FIG. 3B illustrate schematic diagrams of proposed image processing devices in accordance with one of the exemplary embodiments of the disclosure.

[0021] FIG. 4 illustrates a flowchart of a proposed method for encoding private data in accordance with one of the exemplary embodiments of the disclosure.

[0022] FIG. 5 illustrates a functional diagram of a proposed method for encoding private data in accordance with one of the exemplary embodiments of the disclosure.

[0023] FIG. 6A and FIG. 6B illustrate schematic diagrams of proposed image processing devices in accordance with one of the exemplary embodiments of the disclosure.

[0024] FIG. 7 illustrates a flowchart of a proposed method for decoding private data in accordance with one of the exemplary embodiments of the disclosure.

[0025] FIG. 8A illustrates a functional diagram of a proposed method for decoding private data in accordance with one of the exemplary embodiments of the disclosure.

[0026] FIG. 8B illustrates a schematic diagram of private data and protected data for playback in accordance with one of exemplary embodiments of the disclosure.

[0027] To make the above features and advantages of the application more comprehensible, several embodiments accompanied with drawings are described in detail as follows.

DESCRIPTION OF THE EMBODIMENTS

[0028] Some embodiments of the disclosure will now be described more fully hereinafter with reference to the accompanying drawings, in which some, but not all embodiments of the application are shown. Indeed, various embodiments of the disclosure may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will satisfy applicable legal requirements. Like reference numerals refer to like elements throughout.

[0029] FIG. 3A and FIG. 3B illustrate schematic diagrams of proposed image processing devices in accordance with one of the exemplary embodiments of the disclosure. Note that both image processing devices proposed in FIG. 3A and FIG. 3B would be used for encoding private data. All components of the image processing devices and their configurations are first introduced in FIG. 3A and FIG. 3B. The functionalities of the components are disclosed in more detail in conjunction with FIG. 4.

[0030] Referring to FIG. 3A, an image processing device 300A would be implemented as an integrated circuit (IC) including a receiving circuit 310, a masking circuit 320, an encoding circuit 330, and an output circuit 340, where the masking circuit 320 is coupled to the receiving circuit 310, the encoding circuit 330 is coupled to the masking circuit 320, and the output circuit 340 is coupled to the encoding circuit 330. Note that in one of exemplary embodiments, an encrypting circuit 335 coupled between the encoding circuit 330 and the output circuit 340 may be optionally included in the image processing device 300A.

[0031] Referring to FIG. 3B, an image processing device 300B would be an electronic system or a computer system that includes a processor 350. The processor 350 may be may be one or more of a North Bridge, a South Bridge, a field programmable array (FPGA), a programmable logic device (PLD), an application specific integrated circuit (ASIC), other similar devices, or a combination thereof. The processor may also be a central processing unit (CPU), a programmable general purpose or special purpose microprocessor, a digital signal processor (DSP), a graphics processing unit (GPU), other similar devices, or a combination thereof.

[0032] FIG. 4 illustrates a flowchart of a proposed method for encoding private data in accordance with one of the exemplary embodiments of the disclosure. The steps of FIG. 4 could be implemented by the proposed image processing device 300A and 300B as respectively illustrated in FIG. 3A and FIG. 3B.

[0033] Referring to FIG. 4 in conjunction to FIG. 3A, the receiving circuit 310 would receive an original video frame (Step S402), and the masking circuit 320 would mask at least one private area in the original video frame to generate a protected video frame (Step S404). Herein, such private area may be a facial area of a person and may be masked through any face blurring techniques as known per se. In other words, the identity of the person appearing in the protected video frame may be protected and anonymized. Next, the encoding circuit 330 would generate a first encoded frame by encoding the protected video frame (Step S406), and the output circuit 408 would generate at least one output bitstream for streaming or storage according to the first encoded frame (Step S408). More details would be provided later. Note that the steps S402-S406 could also be performed by the processor 350 of the image processing device 300B in FIG. 3B. The descriptions would be omitted for brevity's sake.

[0034] For better comprehension, FIG. 5 illustrates a functional diagram of a proposed method for encoding private data in accordance with one of the exemplary embodiments of the disclosure. The steps of FIG. 5 could be implemented by the proposed image processing device 300A and 300B as respectively illustrated in FIG. 3A and FIG. 3B. In the following description, only one frame would be presented for illustration purposes.

[0035] Referring to FIG. 5 in conjunction to FIG. 3A, the receiving circuit 310 would receive an original video frame Fn and duplicate (S510) the original frame Fn to generate a duplicated original video frame Cn. Next, the masking circuit 320 would mask (S520) the duplicated original video frame Cn to generate a protected video frame Cn′.

[0036] The encoding circuit 330 would encode (S530) the protected video frame Cn′ to generate a reference frame Rn and a first encoded frame Xn. Note that the protected video frame Cn′ would be encoded with a reference frame R(n−1) if coded to p-frame. The encoding circuit 330 would also encode (S530) the original video frame Cn by using the reference frame Rn to generate a second encoded frame Yn. Next, the encrypting circuit 335 would encrypt the second encoded frame Yn to generate an encrypted second encoded frame Zn using, for example, the AES encryption algorithm.

[0037] The output circuit 340 would generate at least one output bitstream for streaming or storage according to the first encoded frame Xn (considered as a normal bitstream) and the encrypted second encoded frame Zn (considered as an encrypted bitstream). Herein, the output circuit 340 would pack (S540) the first encoded frame Xn and the encrypted second encoded frame Zn into one bitstream or separated bitstreams for storage or streaming. In one exemplary embodiment, the output circuit 340 may pack the first encoded frame Xn into a first output bitstream and pack the encrypted second encoded frame into a second output bitstream. In another exemplary embodiment, the output circuit 340 may pack the first encoded frame Xn and the encrypted second encoded frame Zn into a combined output bitstream.

[0038] Note that the flow in FIG. 5 could also be performed by the processor 350 of the image processing device 300B in FIG. 3B. The descriptions would be omitted for brevity's sake.

[0039] In terms of video encoding, note that the second encoded frame Yn keeps the difference between the original frame Fn and the reference frame Rn only. The bitstream Zn would only contain private data (private area) and would be considered as “a private layer”. The private layer would be packed in the supplemental enhancement information (SEI) in the network abstraction layer (NAL) unit. In terms of video decoding and playback, the SEI would be packed with a normal bitstream and would be ignored by NVR while playback and liveview, and NVR would play the private layer only with an authorized key (i.e. a decryption key).

[0040] In detail, FIG. 6A and FIG. 6B illustrate schematic diagrams of proposed image processing devices in accordance with one of the exemplary embodiments of the disclosure. Note that both image processing devices proposed in FIG. 6A and FIG. 6B would be used for decoding private data. All components of the image processing devices and their configurations are first introduced in FIG. 6A and FIG. 6B. The functionalities of the components are disclosed in more detail in conjunction with FIG. 7.

[0041] Referring to FIG. 6A, an image processing device 600A would be implemented as an integrated circuit (IC) including a receiving circuit 610, a decoding circuit 620, and an output circuit 630, where the decoding circuit 620 is coupled to the receiving circuit 610, and the output circuit 630 is coupled to the decoding circuit 620. Note that in one of exemplary embodiments, a decrypting circuit 625 coupled between the decoding circuit 620 and the output circuit 630 may be optionally included in the image processing device 600A.

[0042] Referring to FIG. 6B, an image processing device 600B would be an electronic system or a computer system that includes a processor 650. The processor 650 may be may be one or more of a North Bridge, a South Bridge, a FPGA, a PLD, an ASIC, other similar devices, or a combination thereof. The processor may also be a CPU, a programmable general purpose or special purpose microprocessor, a DSP, a GPU, other similar devices, or a combination thereof.

[0043] FIG. 7 illustrates a flowchart of a proposed method for decoding private data in accordance with one of the exemplary embodiments of the disclosure. The steps of FIG. 7 could be implemented by the proposed image processing device 600A and 600B as respectively illustrated in FIG. 6A and FIG. 6B.

[0044] Referring to FIG. 7 in conjunction to FIG. 6A, the receiving circuit 610 would receive at least one input video bitstream to obtain a first encoded bitstream and a second encoded bitstream (Step S702). Next, the decoding circuit 620 would decode the first encoded bitstream to generate a protected video frame including image data of at least one private area (Step S704), and the output circuit 630 would output the protected video frame to a display queue such that the at least one private area is displayed (Step S706). More details would be provided later. Note that the steps S702-S706 could also be performed by the processor 650 of the image processing device 600B in FIG. 6B. The descriptions would be omitted for brevity's sake.

[0045] For better comprehension, FIG. 8A illustrates a functional diagram of a proposed method for decoding private data in accordance with one of the exemplary embodiments of the disclosure. The steps of FIG. 8A could be implemented by the proposed image processing device 600A and 600B as respectively illustrated in FIG. 6A and FIG. 6B. In the following description, only one frame would be presented for illustration purposes.

[0046] Referring to FIG. 8A in conjunction to FIG. 6A, the receiving circuit 610 would receive an input bitstream (Xn, Zn) (S810) as packed in (S540) in FIG. 5, where Xn and Zn would be referred to as “a first encoded bitstream” and “a second encoded bitstream” hereafter. Next, decoding circuit 620 would decode the first encoded bitstream Xn to reconstruct a reference frame

[0047] Rn not having image data of at least one private area (S820). Note that the first bitstream Xn would be decoded with a reference frame R(n−1) if coded to p-frame. Next, the decrypting circuit 625 would determine whether the second encoded bitstream Zn is authorized to be decoded (i.e. whether an authorized key is available) (S830).

[0048] If the second encoded bitstream Zn is not authorized to be decoded, the output circuit 630 would send the reference frame Rn to a display queue (S840). Herein, only the first encoded bitstream Xn is decoded, and only the reference frame Rn is output to the display queue. The second encoded bitstream Zn is not decoded, and the protected video frame Cn′ is not output to the display queue. Since only the reference frame Rn is displayed with the private area being blurred.

[0049] On the other hand, if the second encoded bitstream Zn is authorized to be decoded, the decrypting circuit 625 would decrypt the second encoded bitstream Zn to a second encoded frame Yn (S850) and decode the second encoded frame Yn with the reference frame Rn to reconstruct a protected video frame Cn′ (S860), and the output circuit 630 would send the protected video frame to the display queue (S870). Herein, both the first encoded bitstream Xn and the second encoded bitstream Zn are decoded to generate the reference frame Rn and the protected video frame Cn′ that are output to the display queue. Therefore, a whole image (i.e. an original frame) would be displayed without the private area being blurred. Once the input bitstream (Xn, Zn) has been processed, the flow moves to S810 to process a subsequent bitstream (X(n+1), Z(n+1)).

[0050] Note that the flow in FIG. 8A could also be performed by the processor 650 of the image processing device 600B in FIG. 6B. The descriptions would be omitted for brevity's sake.

[0051] FIG. 8B illustrates a schematic diagram of private data and protected data for playback in accordance with one of exemplary embodiments of the disclosure. During video playback, if the authorized key does not exist, only the protected data 81, 83, 85, 87 would be displayed. If the authorized key exists, the protected data 81, 83, 85, 87 would be alternately combined with the protected data 82, 84, 86, 88, and the combination of the protected data and the private data 81-88 would be displayed.

[0052] Note that in the flows in FIG. 6 and FIG. 8A, the private data is able to be secured at a data level by using a single video stream with a protected private layer. Non-private video would be decoded in NVR without certification, and the private layer requires an authorized key for decoding with instant authorization and expiration. For example, the authorized key may be different for each camera, each zone, each time interval, and so forth. Moreover, reduced spaces (approximately only up to 1.2 times of memory spaces) are required for storage and streaming for transmission.

[0053] In view of the aforementioned descriptions, the proposed methods and the image processing devices would be able to provide video transmission in a secure and efficient fashion.

[0054] No element, act, or instruction used in the detailed description of disclosed embodiments of the present application should be construed as absolutely critical or essential to the present disclosure unless explicitly described as such. Also, as used herein, each of the indefinite articles “a” and “an” could include more than one item. If only one item is intended, the terms “a single” or similar languages would be used. Furthermore, the terms “any of” followed by a listing of a plurality of items and/or a plurality of categories of items, as used herein, are intended to include “any of”, “any combination of”, “any multiple of”, and/or “any combination of multiples of the items and/or the categories of items, individually or in conjunction with other items and/or other categories of items. Further, as used herein, the term “set” is intended to include any number of items, including zero. Further, as used herein, the term “number” is intended to include any number, including zero.

[0055] It will be apparent to those skilled in the art that various modifications and variations can be made to the structure of the disclosed embodiments without departing from the scope or spirit of the disclosure. In view of the foregoing, it is intended that the disclosure cover modifications and variations of this disclosure provided they fall within the scope of the following claims and their equivalents.