Video masking processing method and apparatus

Abstract

A video masking processing method and a video masking processing apparatus using the same are provided. The video masking processing method photographs frames constituting a video, detects at least one masking area to be masked in the photographed frames, masks an image in the detected at least one masking area, inserts masking information including information on the masking into the photographed frame, and stores the frame which has undergone the masking and has the masking information inserted thereinto. Therefore, since images related to personal information in a photographed video are masked, a user does not infringe others' privacy in a video normally photographed by a CCTV or a dashboard camera.

Claims

1. A video masking processing method, comprising: detecting a masking area to be masked in a frame constituting a video; masking an image in the detected masking area; inserting masking information on the masked image into a predetermined pixel area of the frame; storing the frame comprising the inserted masking information; displaying a vehicle plate number or a human face included in the video using the inserted masking information; and when one of the vehicle plate number or human face is selected, reproducing a frame of the video that includes the selected vehicle plate number or human face, wherein the masking information comprises information on a number of masking areas in the frame, and masking area information corresponding to the number of the masking areas, wherein the masking area information comprises an identifier of a corresponding masking area, information on a masking technique code applied to the corresponding masking area, coordinates information of a left upper point of the corresponding masking area, and coordinates information of a right lower point of the corresponding masking area, and wherein the image in the detected masking area is masked using a different key according to an object type of the image.

2. The video masking processing method of claim 1, wherein the detected masking area comprises an image of a human face or a vehicle number plate.

3. The video masking processing method of claim 1, further comprising: transmitting a request to reproduce the video constituted by the frame comprising the inserted masking information; receiving a key for unmasking the video corresponding to the request; and unmasking the video using the key and reproducing the unmasked video.

4. A video masking processing apparatus, comprising: a processing unit configured to detect a masking area to be masked in a frame constituting a video, mask an image in the detected masking area, and insert masking information on the masked image into a predetermined pixel area of the frame; and a storage unit configured to store the frame comprising the inserted masking information, wherein the masking information comprises information on a number of masking areas in the frame, and masking area information corresponding to the number of the masking areas, wherein the masking area information comprises an identifier of a corresponding masking area, information on a masking technique code applied to the corresponding masking area, coordinates information of a left upper point of the corresponding masking area, and coordinates information of a right lower point of the corresponding masking area, and wherein the image in the detected masking area is masked using a different key according to an object type of the image.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) For a more complete understanding of the present disclosure and its advantages, reference is now made to the following description taken in conjunction with the accompanying drawings, in which like reference numerals represent like parts:

(2) FIG. 1 is a view schematically showing a configuration of a video masking processing apparatus according to an exemplary embodiment of the present disclosure;

(3) FIG. 2 is a flowchart showing a video masking processing method according to an exemplary embodiment of the present disclosure;

(4) FIG. 3 is a view showing a structure of masking information according to an exemplary embodiment of the present disclosure;

(5) FIGS. 4A and 4B are views showing original images according to an exemplary embodiment of the present disclosure;

(6) FIGS. 5A and 5B are views showing images in which vehicle plate numbers and persons' faces are masked according to an exemplary embodiment of the present disclosure;

(7) FIG. 6 is a view showing a process of reproducing a masking-processed video according to an exemplary embodiment of the present disclosure;

(8) FIG. 7 is a view showing a key in the form of an image according to an exemplary embodiment of the present disclosure;

(9) FIG. 8 is a view showing a screen which displays a list of vehicle plate numbers and a list of person's faces included in a masking-processed video according to an exemplary embodiment of the present disclosure;

(10) FIG. 9 is a view showing a screen which displays a video of a part including a vehicle plate number when the vehicle plate number is selected according to an exemplary embodiment of the present disclosure;

(11) FIG. 10 is a view showing a screen which displays a video of a part including a person's face when the person's face is selected according to an exemplary embodiment of the present disclosure;

(12) FIGS. 11A and 11B are views showing a process of searching a vehicle plate number included in a masking-processed video according to an exemplary embodiment of the present disclosure; and

(13) FIGS. 12A and 12B are views showing a process of searching a person's face included in a masking-processed video according to an exemplary embodiment of the present disclosure.

DETAILED DESCRIPTION OF THE INVENTION

(14) Reference will now be made in detail to the embodiment of the present general inventive concept, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the like elements throughout. The embodiment is described below in order to explain the present general inventive concept by referring to the drawings.

(15) FIG. 1 is a view schematically showing a configuration of a video masking processing apparatus 100 according to an exemplary embodiment of the present disclosure. The video masking processing apparatus 100 removes others' vehicle plate number or others' faces from a photographed video to protect others' personal information which is not necessary for the photographed video. In an image monitoring technique for monitoring vehicles and persons in an image photographed by a camera such as a dashboard camera, there is a problem that the photographed image is recorded randomly and thus personal information is leaked, and thus the problem of infringement of privacy has become a social issue. Therefore, to prevent leakage of others' personal information, the video masking processing apparatus 100 detects pedestrians, pedestrians' faces, vehicles ahead, and vehicle plate numbers using an image pattern recognition technique, and then masks the recognized object area and voice and stores the same as a video using a masking technique.

(16) The video masking processing apparatus 100 may be provided in a camera device, a computer or a smart phone, or a CCTV or a dashboard camera. As shown in FIG. 1, the video masking processing apparatus 100 includes a photographing unit 110, a processing unit 120, and a storage unit 130.

(17) The photographing unit 110 photographs frames constituting a video. That is, the photographing unit 110 photographs a video on a frame basis, and may be any device that can photograph a video such as a video camera, a camcorder, etc.

(18) The processing unit 120 masks the frame of the photographed video. Specifically, the processing unit 120 detects at least one masking area to be masked from the photographed frame, masks an image in the detected at least one masking area, and inserts masking information including information on the masking into the photographed frame. The processing unit 120 may be applied to a processing device such as a graphic processing device or a CPU.

(19) The storage unit 120 stores the frame which has undergone the masking process and has the masking information inserted thereinto. The storage unit 120 stores all of the frames of the video which has been processed as described above, and as a result, stores the masking-processed video. In addition, the storage unit 120 may store a variety of other data and may be a flash memory, a hard disk, a solid state disk (SDD), etc.

(20) The video masking processing apparatus 100 having the above-described configuration performs the masking process with respect to an area related to personal information in the photographed image and stores the image, thereby protecting personal privacy which was inadvertently photographed.

(21) FIG. 2 is a flowchart to illustrate a video masking processing method of the video masking processing apparatus 100 according to an exemplary embodiment of the present disclosure.

(22) First, the video masking processing apparatus 100 photographs frames constituting a video (S210). The video is formed of a plurality of frames, and the video masking processing apparatus 100 performs a masking process with respect to each of the frames of the photographed video.

(23) Next, the video masking processing apparatus 100 detects at least one masking area to be masked in the photographed frame (S220). Herein, the masking area refers to an area which includes others' personal information unnecessarily existing in the frame of the photographed video. For example, the masking area may indicate an area which includes a person's face or an area which includes a vehicle number plate. Since a single frame includes a plurality of vehicle number plates or a plurality of persons' faces, a single masking area or a plurality of masking areas may be included in the single frame. In this case, the video masking processing apparatus 100 may recognize a person's face or a vehicle number plate using an image pattern recognition technique, and, when the vehicle number plate is recognized, may recognize what number the corresponding vehicle is.

(24) Thereafter, the video masking processing apparatus 100 masks an image in the detected at least one masking area (S230). Herein, the masking refers to a process of making an image of a corresponding area unrecognizable, and is performed in various ways. For example, in the case of an image masking technique, the video masking processing apparatus 100 masks the image by encoding through Discrete Cosine Transform (DCT), quantization, zig-zag scan, run-length technique, and Huffman coding. In this case, the video masking processing apparatus 100 may use a specific key value to mask the image, and the specific key value may be used for unmasking afterward.

(25) Next, the video masking processing apparatus 100 inserts masking information including information on the masking into the photographed frame (S240). Herein, the masking information indicates information on how the masking area has been masked. The detailed structure of the masking information is illustrated in FIG. 3.

(26) FIG. 3 is a view showing a structure of masking information 300 according to an exemplary embodiment of the present disclosure. As shown in FIG. 3, the masking information 300 includes start information (M-Start) 310, information on the number of masking areas in a frame 320, masking area information 330-1, 330-2 corresponding to each of the masking areas, and end information (M-end) 340.

(27) The start information (M-start) 310 refers to a start pixel from which the masking information is recorded. The end information (M-end) 340 refers to a last pixel where the masking information is recorded.

(28) The information on the number of masking areas in the frame 320 is an area indicating how many masking areas exist in a corresponding frame.

(29) The masking area information 330-1, 330-2 includes location information on one masking area and information on the masking technique, and as many pieces of masking area information as the number of masking areas exist. Since two masking area exist, the masking information includes two pieces of masking area information, first masking area information 330-1 and second masking area information 330-2 in FIG. 3.

(30) Specifically, the first masking area information 330-1 includes an identifier 331-1 of the corresponding masking area, masking technique code information 332-1 applied to the masking area, coordinates information 333-1, 334-1 of a left upper point of the corresponding masking area, and coordinate information 335-1, 336-1 of a right lower point of the corresponding masking area. The second masking area information 330-2 has the same structure as the first masking area information 330-1, and thus is omitted from FIG. 3.

(31) The masking information 300 having the above-described structure indicates how many masking areas are included in the corresponding frame, where each of the masking areas is located, and how each of the masking areas is masked.

(32) In addition, the video masking processing apparatus 100 inserts the masking information 300 generated as described above into a predetermined pixel area of the corresponding frame. Since the masking information 300 has a relatively small amount of data, the masking information 300 may be recorded using only one or a few pixels in the frame.

(33) Referring back to FIG. 2, the video masking processing apparatus 100 stores the frame which has undergone the masking process and have the masking information inserted thereinto (S250). That is, the video masking processing apparatus 100 can store the masking information 300 with the video by simply storing only the corresponding frame without having to store the masking information 300 in a separate storage area.

(34) Thereafter, the video masking processing apparatus 100 determines whether the photographing ends or not (S260). When the photographing does not end (S260-N), the video masking processing apparatus 100 resumes step S210. That is, the video masking processing apparatus 100 repeats steps 210 to 250 described above for all of the frames of the photographed video.

(35) Through the above-described process, the video masking processing apparatus 100 can perform the masking process with respect to the photographed video and store the video, and store the information related to the masking in the corresponding video.

(36) FIGS. 4A and 4B are views showing originals images according to an exemplary embodiment of the present disclosure, and FIGS. 5A and 5B are views showing images in which vehicle plate numbers and persons' faces are masked according to an exemplary embodiments of the present disclosure.

(37) As shown in FIGS. 4A and 4B, in the originals images, the vehicle number plates or the persons' faces are displayed as they are. However, in FIGS. 5A and 5B, the masking-processed vehicle plate numbers and persons' faces are displayed.

(38) As described above, the video masking processing apparatus 100 masks vehicle plate numbers and persons' faces in order to protect personal information of passing walkers or vehicles, and stores the image as shown in FIGS. 5A and 5B.

(39) However, the vehicle plate number or person's face may need to be identified to be used as evidence of an accident or a crime afterward. Therefore, there is a need for a method for unmasking a masking-processed video. This will be explained in detail below.

(40) FIG. 6 is a view showing a process of reproducing a masking-processed video according to an exemplary embodiment of the present disclosure. In FIG. 6, a user device 600 may indicate the video masking processing apparatus 100 or an apparatus which receives a video which has been masking-processed and recorded by the video masking processing apparatus 100 to unmask and then reproduce the video. The user device 600 capable of unmaking may be used in a public institution or a police station.

(41) In addition, a manager device 601 indicates a device of a manager who manages reproduction authority of a masked video. The manager device 601 may be a device of a company which provides masking processing and video services.

(42) The user device 600 and the manager device 601 may be computers or servers.

(43) First, the user device 600 transmits, to the manager device 601, a request to reproduce a video which is formed of frames which have undergone a masking process and have masking information inserted thereinto (S610). In this case, the user device 600 may also transmit its own MAC address, user information, and using time information. In addition, the user device 600 may use a dedicated player to reproduce the masked video.

(44) Then, the manager device 601 transmits a key for unmasking the video to the user device 600 in response to the reproduction request (S620). In this case, the key may be generated using the MAC address, and may be a key in the form of a text or an image. FIG. 7 is a view showing a key in the form of an image according to an exemplary embodiment of the present disclosure. As shown in FIG. 7, when the key in the form of the image is used, the security of the key can be improved.

(45) After receiving the key, the user device 600 unmasks the video using the key and reproduces the video. In this case, the user device 600 may reproduce the video using a dedicated player.

(46) Through the above-described process, the user device 600 unmasks the making-processed video and reproduces the video.

(47) Hereinafter, additional functions using masking information will be explained with reference to FIGS. 8 to 12B.

(48) FIG. 8 is a view showing a screen which displays a list of vehicle plate numbers and a list of persons' faces included in a masking-processed video according to an exemplary embodiment of the present disclosure.

(49) The masking information includes information on vehicle plate numbers and persons' faces in a video. Therefore, the video masking processing apparatus 100 may recognize all of the vehicle plate numbers and the persons' faces included in the entire video, and list them or separately store them in a database.

(50) As shown in FIG. 8, the video masking processing apparatus 100 may display at least one vehicle plate number or person's face included in the video in the form of a list using the masking information. In FIG. 8, three vehicle plate numbers 811, 812, and 813 and three persons' faces 821, 822, and 823 are included in the video.

(51) Thereafter, when the user selects one of the vehicle plate numbers or the persons' faces, the video masking processing apparatus 100 may reproduce a part of the video that includes the selected vehicle plate number or person's face. This will be explained below with reference to FIGS. 9 and 10.

(52) FIG. 9 is a view showing a screen which, when a vehicle plate number is selected, displays a video of a part including the selected vehicle plate number. When the second vehicle plate number 812 in FIG. 8 is selected, the video masking processing apparatus 100 directly reproduces the part including the second vehicle plate number 812 as shown in FIG. 9.

(53) FIG. 10 is a view showing a screen which, when a person's face is selected, displays a video of a part including the selected person's face. When the third person's face 823 in FIG. 8 is selected, the video masking processing apparatus 100 directly reproduces the part including the third person's face 823 as shown in FIG. 10.

(54) Through the above-described function, the user can easily search and identify the vehicle plate number or person's face in the masked video.

(55) In addition, the video masking processing apparatus 100 may provide a function of searching a vehicle plate number or a person's face included in a video. This function will be explained in detail below with reference to FIGS. 11A, 11B, 12A and 12B.

(56) FIGS. 11A and 11B are views showing a process of searching a vehicle plate number included in a masking-processed video according to an exemplary embodiment of the present invention.

(57) As shown in FIGS. 11A and 11B, when a vehicle plate number is input by the user, the video masking processing apparatus 100 recognizes at least one vehicle plate number included in the video using masking information included in the video, and reproduces a part of the video including the inputted vehicle plate number. In FIGS. 11A and 11B, the user inputs 13A 0001 as a number that the user wishes to search, and the video masking processing apparatus 100 may directly reproduces a part including 13A 0001.

(58) FIGS. 12A and 12B are views showing a process of searching a person's face included in a masking-processed video according to an exemplary embodiment of the present invention.

(59) When a photo of a person's face is inputted by the user, the video masking processing apparatus 100 recognizes at least one person's face included in the video using masking information included in the video, compares the persons' faces included in the video with the inputted person's face photo through a face recognition technique, and directly reproduces a part of the video including the person's face matching the inputted person's face. In FIGS. 12A and 12B, the user inputs the photo of the person's face 823 that the user wishes to find, and the video masking processing apparatus 100 directly reproduces the part including the person's face 823.

(60) Through the above-described function, the video masking processing apparatus 100 can easily search a vehicle plate number or a person's face included in the masking-processed video.

(61) In addition, masking/unmasking may be performed for a person's face and a vehicle plate number using different keys. Specifically, in a video, masking areas where persons' faces appear may be masked/unmasked using key-1, and masking areas where vehicle plate numbers appear may be masked/unmasked using key-2.

(62) Accordingly, in the video, the vehicle plate numbers may remain masked and only the persons' faces may be unmasked, or the persons' faces remain masked and only the vehicle plate numbers may be unmasked. The former case may be useful when only the persons' faces need to be identified since the vehicle plate numbers are not exposed, and the latter case may be useful when only the vehicle plate numbers in the video need to be identified since the persons' faces are not exposed.

(63) However, to achieve this, the masking information should additionally include information on a type of masking area, that is, information on whether the masking area includes persons' faces or vehicle plate numbers. To unmask the video, the type of masking area included in the masking information may be determined, and key-1 may be used for the masking area including the persons' faces and key-2 may be used for the masking area including the vehicle plate numbers.

(64) The technical idea of the present disclosure can be applied to a computer-readable recording medium which records a computer program for implementing the functions and methods of the apparatus according to the exemplary embodiments. In addition, the technical idea according to various exemplary embodiments of the present disclosure may be implemented in the form of a computer-readable programming language code which is recorded on the computer-readable recording medium. The computer-readable recording medium may be any data storage device that can be read by a computer and can store data. For example, the computer-readable recording medium may be a Read Only Memory (ROM), a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical disk, a hard disk drive, a flash memory, a Solid State Disk (SSD), etc. In addition, a computer-readable code or program stored in the computer-readable recording medium may be transmitted through a network connected between computers.

(65) The foregoing exemplary embodiments and advantages are merely exemplary and are not to be construed as limiting the present inventive concept. The exemplary embodiments can be readily applied to other types of apparatuses. Also, the description of the exemplary embodiments is intended to be illustrative, and not to limit the scope of the claims, and many alternatives, modifications, and variations will be apparent to those skilled in the art.