SECURE QR CODE USING NONLINEARITY OF SPATIAL FREQUENCY IN LIGHT

Abstract

This invention relates to a secure QrCode communication method based on nonlinear spatial frequency characteristics, comprising: camera modeling: modeling according to the spatial frequency of the color filter matrix of the scanning device's camera; QrCode encryption: using the CFA spatial frequency of scanning device and modeling results, as well as the spatial frequency of the display device, generate an encrypted picture of the target QrCode on the display device; QrCode decryption: the camera of the scanning device takes the picture of the display at a specified position and a specified angle, and parses the image to recover the target QrCode. Compared with the prior art, the present invention utilizes the nonlinear characteristics of the optical spatial frequency, and uses the spatial frequency of the camera's own color filter array and the spatial frequency of the display to modulate the target two-dimensional code through phase to achieve the effect of encryption.

Claims

1. A secure QrCode communication method based on nonlinear spatial frequency is characterized by the following three parts: Camera modeling: modeling based on the spatial frequency of the color filter array of the scanning device's camera; QrCode encryption: using the CFA spatial frequency of the scanning device's camera and its modeling result, and the spatial frequency of the display device, to generate an encrypted picture of the QrCode on the display device; QrCode decryption: The camera of the scanning device captures the encrypted QrCode shown on the display at a specified position and a specified angle, analyzes the target QrCode and recovers it.

2. According to claim 1, the secure QrCode communication method based on nonlinear spatial frequency is characterized that the modeling process specifically models the color filter array of the green filter layer.

3. According to claim 2, the secure QrCode communication method based on nonlinear spatial frequency is characterized wherein the modeling result is:
m.sub.cfa(x,y)=p.sub.cfa(.sub.cfa(x,y))
p.sub.cfa(u)=0.5+0.5 cos(2u)
.sub.cfa(x,y)=((x,y)mod 2)/2 Where, m.sub.cfa(x,y) is the green receiving information of the green filter layer on the camera image sensor at coordinates (x,y), p.sub.cfa() is the periodic function of the green filter layer in the CFA, and .sub.cfa (x,y) is the phase function of the green filter layer.

4. According to claim 1, the secure QrCode communication method based on the nonlinear spatial frequency is characterized wherein the QrCode encryption process specifically comprises: Step A1: performing phase and frequency modulation on the original QrCode image picture by using the CFA spatial frequency of the scanning device's camera and the modeling result, and the spatial frequency of the display device; Step A2: Add Gaussian white noise to the modulated picture to blur the contour in the encrypted QrCode.

5. According to claim 3, the secure QrCode communication method based on the nonlinear spatial frequency is characterized wherein the QrCode decryption process specifically comprises: Step B1: acquiring a plurality of consecutive pictures captured by the camera in the specified position and the specified angle to encrypt the QrCode in the display; Step B2: selecting an unprocessed image and converting the RGB image to the HSV coordinates to maximize the saturation dimension; Step B3: according to the positions of the three positioning marks, determining the size of the QrCode and dividing the QrCode for the converted picture to obtain a plurality of blocks; Step B4: extracting a green square, for each pixel point, when the green intensity is greater than a predetermined threshold, it is judged to be white, and vice versa, it is judged to be black; Step B5: for each square, determine the color of the square according to the distribution of white and black pixel points in the square; Step B6: traversing all the black squares and mark all adjacent black squares to the same index value; Step B7: determining whether all the blocks are marked or unmarked squares are all surrounded by the marked blocks, if yes, proceed to step B8, otherwise, return to step B2; Step B8: coloring according to the index value of each block, if two adjacent squares have the same index value, they are colored into the same color, and vice versa.

6. According to claim 5, the secure QrCode communication method based on the nonlinear spatial frequency is characterized, wherein the step B6 specifically comprises: Step B61: traversing all the black squares, and determining an index value of each block under the frame based on the principle that all adjacent black squares mark the same index value; Step B62: determining whether the index value exists in each block, and if not, the index value of the block in the current frame is used as the index value of the block, otherwise, step B63 is performed. Step B63: determining whether the index value of the block in the current frame is consistent with the previous index value. If not, all the blocks marked with the index value of the block in the previous frame are marked with the index value of the block under the frame.

7. According to claim 5, the secure QrCode communication method based on the nonlinear spatial frequency is characterized, wherein the step B8 specifically includes: Step B81: coloring the squares in the three positioning mark areas; Step B82, based on the coloring result of the squares in the three positioning areas, coloring according to the index values of the blocks, if the two adjacent squares have the same index value, the colors are colored the same color, and vice versa.

8. According to claim 1, the secure QrCode communication method based on nonlinear spatial frequency is characterized, wherein the designated position is between 20 and 45 centimeters from the display.

9. According to claim 8, a secure QrCode communication method based on nonlinear spatial frequency is characterized, wherein said designated position is 30 cm from the display.

10. According to claim 1, the secure two-dimensional code communication method based on nonlinear spatial frequency is characterized, wherein the specified angle is perpendicular to an angle of the display plane.

Description

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

[0038] FIG. 1 is a schematic flow chart of main steps of the method of the present invention;

[0039] FIG. 2 is a nonlinear phenomenon generated by superposition of two spatial frequencies in optical and a schematic diagram of a moir generated by a camera-capture display;

[0040] FIG. 3 is a schematic diagram of a Bayer color filter matrix in a camera;

[0041] FIG. 4 (a) is a schematic diagram of the original two-dimensional code;

[0042] FIG. 4 (b) is a schematic diagram of an encryption effect based on a green filter layer;

[0043] FIG. 4(c) is a schematic diagram of an encryption effect based on a red filter layer;

[0044] FIG. 4(d) is a schematic diagram of an encryption effect based on a blue filter layer;

[0045] FIG. 5 is an encryption effect diagram based on display spatial frequency modulation;

[0046] FIG. 6 is a decoding flowchart of a camera taking a photo;

[0047] FIG. 7(a) is a schematic diagram of shooting distance and decoding rate in a safety two-dimensional code communication experiment result based on nonlinear spatial frequency characteristics;

[0048] FIG. 7(b) is a schematic diagram of the shooting angle and decoding rate in the experimental results of secure QrCode communication based on nonlinear spatial frequency characteristics.

DETAILED DESCRIPTION OF THE INVENTION

[0049] The invention will be described in detail below with reference to the drawings and specific embodiments. This embodiment is implemented on the premise of the technical solution of the present invention, and detailed implementation manners and specific operation procedures are given, but the scope of protection of the present invention is not limited to the following embodiments. To solve the security problem, we propose mQRCode, which encrypts and decrypts the QR code using nonlinear features of optically specific spatial frequencies. When the target receiver holds the camera at the specified position (for example, 30 cm and 0), the encrypted QR code will be revealed as the original QrCode due to the mole phenomenon. A malicious attacker will only see the encrypted camouflage QrCode anywhere else. Our experiments show that the decryption rate of mQRCode is 95% and the average decoding rate is above 0.82 seconds. When the camera is 10 cm or 15 away from the specified position, the decoding rate drops to zero. So it directly isolates the attacker's attack and is a secure means of communication. A secure QrCode communication method based on nonlinear spatial frequency characteristics, as shown in FIG. 1 to FIG. 6, includes two parts: encryption and decryption: 1) a color filter array spatial frequency and a display through a known camera spatial frequency, using phase modulation and frequency modulation to generate an encrypted QrCode pattern on the display; 2) taking a picture of the encrypted pattern with a camera at a specified position and angle from the display, and finally parsing the original QrCode pattern by using a decryption algorithm and recovering to get relevant information. By carefully performing the phase and frequency modulation of the encrypted two-dimensional code, the attacker cannot use the camera to obtain the target QrCode pattern at any position other than the specified position, thus ensuring that the QrCode communication is visually eliminating the information leaked. Optically, when two patterns with regular spatial frequencies are superimposed, an additional spatial frequency pattern called moir is created, as shown in FIG. 2. Assume that the spatial frequencies of the two superimposed patterns are f.sub.1 and f.sub.2, respectively. Through the multiplication model, the superimposed pattern will appear with two additional spatial frequencies: (f.sub.1+f.sub.2) and (f.sub.1f.sub.2), but The eye is more sensitive to the low frequency pattern, so the picture on the left side of FIG. 2 is the pattern of (f.sub.1f.sub.2) frequency. The camera is also a nonlinear optical system, because the color filter matrix in the camera sensor has a moir pattern when photographing some specific regular spatial frequency patterns, as shown in the right figure of FIG. 2.

[0050] In the encryption process, more specific, because the camera's color filter array (FIG. 3) contains spatial frequency features of three colors, different color filter array types will have different distribution of colors. The color filter matrix of Bayer distribution in FIG. 3 is the most common color filter matrix in current mobile smartphone cameras, consisting of three different spatial patterns of red, green and blue. When the camera is used to capture the display, the pixel on the display is projected onto the camera sensor to form a spatial pattern of the first layer of spatial frequency f.sub.1, and the CFA of the camera sensor itself forms a spatial pattern of the second layer with a spatial frequency of f.sub.2. When the camera is placed in the correct position and angle, the spatial frequency can be seen by (f.sub.1f.sub.2) due to the mole phenomenon, which results in a ripple pattern due to nonlinear optical interaction. Specifically, as shown in FIG. 1, it includes:

[0051] Camera Modeling: Modeling based on the spatial frequency of the camera's color filter matrix of the scanning device, where the camera color filter matrix parameters of the scanning device are known. The present application utilizes nonlinear optical interaction between the color filter matrix of the camera and the camouflage pattern to encrypt and decrypt the two-dimensional code. We assume that the spatial model of the color filter matrix is m.sub.cfa(x,y) and the original two-dimensional code is m.sub.dec(x,y). The encryption process is to calculate the encrypted two-dimensional code model m.sub.enc(x,y) to satisfy: m.sub.sec (x,y)=m.sub.cfa(x,y)*m.sub.enc(x,y). First, we need to model the color filter matrix, m.sub.cfa (x,y)=p.sub.cfa (.sub.cfa(x,y)). FIG. 3 shows the color sensor distribution of the Bayer color filter array. We choose the color of the green filter layer. The filter array is modeled. Because 1) the QrCode contains only two colors, and it has been experimentally proved that the use of the green filter layer can fully cope with the requirements of encryption and decryption; 2) The green filter layer is completely symmetrical in the 2D array, and the number is the most, the human eye is difficult to make a distinction when the phase changes, which is conducive to the encryption of the pattern. We model the spatial model of the green filter layer as follows:

m.sub.cfa(x,y)=p.sub.cfa(.sub.cfa(x,y))

p.sub.cfa(u)=0.5+0.5 cos(2u)

.sub.cfa(x,y)=((x,y)mod 2)/2

[0052] Where: m.sub.cfa(x,y) is the receiving information of the green filter layer on the camera image sensor at coordinates (x,y), p.sub.cfa() is the periodic function of the green filter layer in the CFA, and .sub.cfa (x,y) is in the CFA The phase function of the green filter layer. To calculate m.sub.enc(x,y)=p.sub.enc(.sub.enc(x,y)), we define p.sub.enc(u)=0.5+0.5 cos(2u). We perform phase modulation by mapping black and white in the two-dimensional code to different phases. Since m.sub.dec(x,y) and m.sub.cfa (x,y) are known, combined with the Moore theory and the Fourier transform theory:

m.sub.dec(x,y)=p.sub.dec(.sub.dec(x,y))=p.sub.dec(.sub.cfa(x,y).sub.enc(x,y))

.sub.enc(x,y)=.sub.cfa(x,y)p.sub.dec.sup.1(m.sub.dec(x,y))+2k, kZ

[0053] FIG. 4 shows the encryption effect after modeling with three filter layers of green, red and blue and phase modulation. It is found that the effect of using the green filter layer is the best.

[0054] QrCode encryption: using the CFA spatial frequency and modeling result of the scanning device, and the spatial frequency of the display device display, generating an encrypted image of the target two-dimensional code on the display device, specifically including:

[0055] Step A1: performing phase and frequency modulation on the original QrCode image picture by using the CFA spatial frequency of the scanning device and the modeling result, and the spatial frequency of the display device;

[0056] Step A2: Add Gaussian white noise to the modulated picture to blur the contour in the encrypted QrCode.

[0057] The use scene of the daily QrCode requires the QrCode to have a plurality of communication distances. In order to support a plurality of communication distances, the present invention expands the green filter layer space model function by modulating the frequency to generate a new space model. FIG. 5 illustrates the effect of mQRCode re-encoding the image with frequency modulation to accommodate different communication distances in a variety of application scenarios. But when we adjust the spatial frequency down, the contour of the target QR code becomes more obvious and the encryption effect is worse as shown in FIG. 5(d). Therefore, the present invention also adds 10% of Gaussian white noise after phase and frequency modulation to blur the target QrCode profile.

[0058] QrCode decryption: The camera of the scanning device captures the encrypted QrCode in the display at a specified position and a specified angle, and restores the target QrCode and parses it, including:

[0059] Step B 1: acquiring a plurality of consecutive pictures captured by the camera in the specified position and the specified angle to encrypt the QrCode in the display;

[0060] Step B2: selecting an unprocessed image and convert the RGB image to the HSV coordinates to maximize the saturation dimension. FIG. 6(a) is a picture taken by the camera. The converted picture is shown in FIG. 6(b).

[0061] Step B3: determining, according to the positions of the three positioning marks, the size of the QrCode and dividing the QrCode for the converted picture to obtain a plurality of blocks;

[0062] Among them, there are two versions of common QR codes (version1 is 2121 squares and version2 is 177177 squares). The QR code has three positioning marks and is fixed in position, so we can distinguish the saturated image and determine the width and height, then calculate the size of each square and divide it, as shown in FIG. 6(c).

[0063] Step B4: extracting a green square, for each pixel point, when the green intensity is greater than a predetermined threshold, it is judged to be white, and vice versa, it is judged to be black;

[0064] In order to reliably identify the squares with the same phase, we first separate the green squares from blue/red. For each pixel, when the green intensity is greater than the specified threshold, the pixel will be judged as white, otherwise it will be judged as black. The converted picture is shown in FIG. 6(d).

[0065] Step B5: For each square, determine the color of the square according to the distribution of white and black pixel points in the square; wherein the noise added during the blurring of the QrCode contour and the noise generated during the shooting process which made in each square includes both black and white pixels. To classify each block into black or white, we use a weighted block filter to filter each block. The weight block filter is the same size as each small block in the QR code. The filter has more weight at the center of the block and a smaller weight at the edge of the block. After each block and the weight block filter are multiplied, the resulting weights are sorted into black and white according to a predetermined threshold.

[0066] Step B6: traversing all the black squares. Since the adjacent squares are all black, the adjacent two squares have the same modulation phase value, so all the adjacent black squares are marked to the same index value. The detailed methods are included:

[0067] Step B61: traversing all the black squares, and determining an index value of each block under the frame based on the principle that all adjacent black squares mark the same index value;

[0068] Step B62: determining whether the index value exists in each block, and if not, the index value of the block in the current frame is used as the index value of the block, otherwise, step B63 is performed:

[0069] Step B63: determining whether the index value of the block in the current frame is consistent with the previous index value. If not, all the blocks marked with the index value of the block in the previous frame are marked with the index value of the block under the frame.

[0070] Step B7: determining whether all the blocks are marked or unmarked squares are all surrounded by the marked blocks, if yes, proceed to step B8, otherwise, return to step B2;

[0071] Step B8: coloring according to the index value of each block. If two adjacent blocks have the same index value, they are colored into the same color, and vice versa, and then colored into different colors, the detail methods are shown in the following:

[0072] Step B81: coloring the squares in the three positioning mark areas;

[0073] Step B82, coloring according to the index values of the blocks based on the coloring result of the squares in the three positioning areas, if the two adjacent squares have the same index value, the colors are colored the same color, and vice versa. color the inverted.

[0074] The designated position is between 20 and 45 cm from the display. Preferably, the designated position is 30 cm from the display. In addition, the specified angle is perpendicular to the angle of the display plane.

[0075] We will use the Samsung S7 mobile phone to display the encrypted QR code, and then use the iPhone6 and HuaweiPLE to shoot and decrypt at the 30 cm and 0 position from the Samsung S7 mobile phone. As shown in FIG. 7, we found that when the distance is 30 cm, the decoding rate reaches 100%. When the distance deviation reaches 5 cm, the decoding rate drops to 0% instantaneously. Similarly, when the angle is 0, the decoding rate is as high as 100%. When the angular deviation reaches 5, the decoding rate instantaneously drops to 0%.

[0076] The present application provides a QrCode communication encryption method based on nonlinear spatial frequency characteristics, which uses a moir generated by an optical nonlinear spatial frequency to encrypt a target QrCode and is specified by a camera in a smartphone. The position and angle of the encrypted QR code are captured and decrypted to achieve secure communication of the two-dimensional code. Since the electronic screen displaying the QrCode itself has spatial frequency characteristics, the present application eliminates the need for special optical hardware and provides frequency modulation for a plurality of shooting distances. The present application systematically analyzes the spatial frequency characteristics of the screen and the camera, and realizes the encrypted communication of the QrCode on the common electronic display and the smart phone on the market.