MACHINE VISION SYSTEM USING QUANTUM MECHANICAL HARDWARE BASED ON TRAPPED ION SPIN-PHONON CHAINS AND ARITHMETIC OPERATION METHOD THEREOF

Abstract

Disclosed are a quantum system-based pattern recognition computation apparatus and method for artificial intelligence or machine learning. The computation apparatus recognizes patterns between input data by using a quantum system. The computation apparatus includes a modeling unit and an interpretation unit. The modeling unit sets up an objective function based on the similarity between a first pattern derived from the relationships between points of interests of a first data and a second pattern derived from the relationships between points of interests of a second data. The interpretation unit finds an optimum first pattern and an optimum second pattern, in which the similarity between the first pattern and the second pattern is optimized, by interpreting a final quantum state obtained through an adiabatic evolution process of the quantum system in which the objective function is optimized.

Claims

1. A quantum system-based pattern recognition computation apparatus for artificial intelligence or machine learning, the computation apparatus recognizing and optimizing artificial intelligence or machine learning-based patterns, the computation apparatus being connected to a quantum system, the computation apparatus comprising: a memory configured to store program instructions; and a processor configured to execute the program instructions, the program instructions when executed configured to: set up an objective function based on similarity between a first pattern derived from relationships between elements of input data and a second pattern derived from relationships between elements of reference data; find an optimum first pattern and an optimum second pattern, in which the similarity between the first pattern and the second pattern is optimized, by interpreting a final quantum state obtained through an adiabatic evolution process of the quantum system in which the objective function is optimized; and recognize that the input data matches the reference data if the optimized similarity is greater than a predetermined threshold value, wherein the quantum system comprises a physical model that depends on interaction between dipoles, and a Hamiltonian is set up as the objective function, wherein the Hamiltonian is adapted to solve the physical model, and the Hamiltonian includes a qubit operator term associating a superposition of qubit states.

2. The quantum system-based pattern recognition computation apparatus of claim 1, wherein the program instructions are further configured to: vectorize the relationships between the elements of the input data, and generate the first pattern by modeling a set of the vectorized relationships between the elements of input data as the first pattern; and vectorize the relationships between the elements of the reference data, and generate the second pattern by modeling a set of the vectorized relationships between the elements of the reference data as the second pattern.

3. The quantum system-based pattern recognition computation apparatus of claim 1, wherein the data include text data or voice sound data.

4. The quantum system-based pattern recognition computation apparatus of claim 1, wherein the quantum system comprises an Ising model that depends on dipole interaction of a magnetic body.

5. The quantum system-based pattern recognition computation apparatus of claim 4, wherein the Ising model is a physical model that depends on trapped ion-based spin-phonon coupling.

6. The quantum system-based pattern recognition computation apparatus of claim 1, wherein the quantum system comprises a physical model that has energy corresponding to the objective function.

7. A quantum system-based pattern recognition computation method for artificial intelligence or machine learning using a computation apparatus recognizing and optimizing artificial intelligence or machine learning-based patterns, the computation method comprising: setting up, by the computation apparatus, an objective function based on similarity between a first pattern derived from relationships between elements of input data and a second pattern derived from relationships between elements of reference data; connecting the computation apparatus to a quantum system that is configured to model the first pattern and the second pattern as physical state variables; finding an optimum first pattern and an optimum second pattern, in which the similarity between the first pattern and the second pattern is optimized, by interpreting a final quantum state obtained through an adiabatic evolution process of the quantum system in which the objective function is optimized; and recognizing that the input data matches the reference data if the optimized similarity is greater than a predetermined threshold value, wherein the quantum system comprises a physical model that depends on interaction between dipoles, and a Hamiltonian is set up as the objective function, wherein the Hamiltonian is adapted to solve the physical model, and the Hamiltonian includes a qubit operator term associating a superposition of qubit states.

8. The quantum system-based pattern recognition computation method of claim 7, wherein the setting up comprises: vectorizing the relationships between the elements of the input data, and modeling a set of the vectorized relationships between the elements of the input data as the first pattern; and vectorizing the relationships between the elements of the reference data, and modeling a set of the vectorized relationships between the elements of the reference data as the second pattern.

9. The quantum system-based pattern recognition computation method of claim 7, wherein the data include text data or voice sound data.

10. The quantum system-based pattern recognition computation method of claim 7, wherein the quantum system comprises an Ising model that depends on dipole interaction of a magnetic body.

11. The quantum system-based pattern recognition computation method of claim 10, wherein the Ising model is a physical model that depends on trapped ion-based spin-phonon coupling.

12. The quantum system-based pattern recognition computation method of claim 7, wherein the quantum system comprises a physical model that has energy corresponding to the objective function.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The above and other objects, features and advantages of the present invention will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings, in which:

[0060] FIG. 1 is a view showing the modeling of the interaction between relation vectors between points of interest at principal points of interest according to an embodiment of the present invention;

[0061] FIG. 2 is a diagram showing conventional pattern recognition machine vision modeling;

[0062] FIG. 3 shows a process of modeling feature points within two images of Part (a) and Part (b) of FIG. 2 as vectors and a labeled graph;

[0063] FIG. 4 is a diagram showing the relationships between points of interest at principal points of interest according to an embodiment of the present invention;

[0064] FIG. 5 is a detailed diagram showing an image pattern recognition computation apparatus for machine vision according to an embodiment of the present invention;

[0065] FIG. 6 is a detailed diagram showing a quantum system-based machine vision apparatus according to an embodiment of the present invention;

[0066] FIG. 7 is a flowchart showing an image pattern recognition computation method for machine vision according to an embodiment of the present invention; and

[0067] FIG. 8 is a diagram showing an example of a process of distinguishing between and recognizing the face of a human and the face of a cat by using deep learning.

DETAILED DESCRIPTION OF THE DISCLOSURE

[0068] Other objects and features of the present invention will be apparent through descriptions of embodiments given in conjunction with the accompanying diagrams.

[0069] Embodiments of the present invention will be described in detail below with reference to the accompanying drawings. In the following description, detailed descriptions of related known components or functions that may unnecessarily make the gist of the present invention obscure will be omitted.

[0070] In the embodiments of the present invention, the lengths, scales and proportions of elements may be exaggerated for ease of description. The prevent invention is not limited or restricted to the embodiments. Throughout the accompanying drawings, the same reference symbols are assigned to the same components.

[0071] FIG. 1 is a view showing the modeling of a process of pattern recognition between different images in a machine vision apparatus according to an embodiment of the present invention.

[0072] In a machine vision apparatus, a computer or a robot recognizes an image by comparing previously stored reference patterns with a photographed image. This process is advanced through training. Generally, an algorithm called a heuristic algorithm is applied to a particular type of image. Various heuristic algorithms may be applied based on the types of images.

[0073] In general, to perform pattern matching between different images, the vision of a human and the brain capable of recognizing visual information extract feature points that can describe a pattern represented by each of the images, recognize the locations and arrangement of the feature points and the distances and orientations between the feature points, and comprehensively recognize the feature points and the relative location information between the feature points as a pattern. Thereafter, the brain of the human determines whether the two images match each other by comparing pieces of pattern information extracted from the two images. This process cannot be performed simply based on the translation between a single point and a single point, but can be understood as obtaining a match between patterns each including surround points.

[0074] However, it is generally extremely difficult for machine vision having limited intelligence to completely imitate the operation of the brain of a human or the interpretation of sensory data of a human. The imitation of the operation of the brain of a human is known as a so-called NP-hard problem.

[0075] FIG. 1 shows the modeling of a process of pattern recognition between different images, and indicates that the process corresponds to a general NP-hard problem. Although the images of Part (a) and Part (b) of FIG. 1 are different images, the images may correspond to the different versions of an image having the same physical structure.

[0076] To perform machine vision-related pattern recognition computation, a type of combination of relation vectors indicative of relative location information between individual points of interest (feature points) in each image is used, and such a combination of relation vectors may be understood to have a meaning similar to that of a pattern that is a reference for the recognition of each image.

[0077] Meanwhile, a process of extracting a reference pattern for each image may include a process of extracting feature points, and a process of determining specific relative location information between feature points that can be recognized as relation vectors. This does not correspond to a deterministic problem, but corresponds to a non-deterministic problem that can be solved by finding optimized values while comparing results.

[0078] Assuming that a reference pattern composed of a set of relation vectors between points of interest (feature points) i, j and k within the image of Part (a) of FIG. 1 is X and a reference pattern composed of a set of relation vectors between points of interest (feature points) , and within the image of Part (b) of FIG. 1 is Y, a problem arises in that it is difficult to compute the reference pattern X of the image of Part (a) of FIG. 1 and the reference pattern Y of the image of Part (b) of FIG. 1.

[0079] To describe mapping between the image of Part (a) of FIG. 1 and the image of Part (b) of FIG. 1, the reference pattern X capable of describing Part (a) of FIG. 1 best and the reference pattern Y capable of describing Part (b) of FIG. 1 best must be obtained. The displacement from the reference pattern X to the reference pattern Y must be derived through computation. That is, finding relations descriptive of mapping between the image of Part (a) of FIG. 1 and the image of Part (b) of FIG. 1 may be viewed as finding the most optimized match of the reference pattern X and the reference pattern Y. In the present invention, this process is set up as a single objective function, and will be considered as an optimization problem that is intended to minimize the objective function. In this case, the optimization problem is known as an NP-hard problem as described above.

[0080] In the present invention, it is intended that the optimization problem is solved using quantum computing. In each image, arrows indicative of connections between points of interest (feature points) are modeled as dipoles. In this case, each of the arrows refers to the orientation and length between points of interest (feature points), and may be represented by a vector.

[0081] For this purpose, there may be defined (i) a term representative of a mismatch between a feature point of the image of Part (a) of FIG. 1 and a location corresponding to the feature point in the image of Part (b) of FIG. 1, and (ii) a term representative of the spatial consistency between matches of adjacent points obtained by measuring the divergence between the matches of the adjacent points.

[0082] Using a physical model of quantum computing, the most optimized pattern X in Part (a) of FIG. 1 and the most optimized pattern Y in Part (b) of FIG. 1 may be simultaneously found. The optimized reference pattern X and the optimized reference pattern Y may be simultaneously obtained by simultaneously modeling a combination of relation vectors between the points of interest (feature points) i, j and k in the image of Part (a) of FIG. 1 and a combination of relation vectors between the points of interest (feature points) in the image of Part (b) of FIG. 1 as a physical model for quantum computing and then observing the physical model. An optimized state may be obtained by taking the state of the physical model when the energy of the physical model is in a ground state. For example, when the physical model is implemented within a black box in which quantum computing is possible and the physical properties of the physical model within the black box are observed when a target state (a ground state) has been reached through an adiabatic evolution process by using the physical model, the optimized reference pattern X and the optimized reference pattern Y may be obtained as a result.

[0083] In this case, a physical model including dipoles as physical properties so that a combination of vectors between points of interest (feature points) in each image is described by a dipole model may be selected as a physical model within the black box.

[0084] A relation vector in which a point of interest (a feature point) i is a start point and a point of interest (a feature point) j is an end point in the image of Part (a) of FIG. 1 may be represented by {right arrow over (g)}.sub.i,j. In this case, the relationship from the point of interest (a feature point) i to the point of interest (a feature point) j includes not only the translation between the feature points but also a local scale and an orientation. The relation vector {right arrow over (g)}.sub.i,j may be normalized for global translation, rotation, and scaling.

[0085] When a graph of the feature points i, j and k in Part (a) of FIG. 1 is defined as G.sub.A and a graph of the points of interest (feature points) , and in Part (b) of FIG. 1 is defined as G.sub.B, Equation 1 represents the definition of the distance between the feature points (iG.sub.A, and G.sub.B) derived from each image. When the number of feature points in Part (a) of FIG. 1 is M, G.sub.A is represented by a labeled graph having M nodes. When the number of feature points in Part (b) of FIG. 1 is N, G.sub.B is represented by a labeled graph having N nodes. In this case, {right arrow over (f)}.sub.i refers to a normalized feature vector for the i.sup.th vertex of G.sub.A, and {right arrow over (f)}.sub. refers to a normalized feature vector for the .sup.th vertex of G.sub.B. Such a normalized feature vector is referred to as a local descriptor in some documents. For example, the normalized feature vector may be a vector based on Gabor wavelets of varying scale and orientation around points of interest. The edges of the graphs G.sub.A and G.sub.B represent the geometric relationship between the feature vectors. The similarity between the image of Part (a) of FIG. and the image of Part (b) of FIG. 1 may be found by obtaining the similarity between the two labeled graphs G.sub.A and G.sub.B.

d(i,)=d.sub.feature({right arrow over (f)}.sub.i, {right arrow over (f)}.sub.)(1)

[0086] d(i,) is the scalar product of the feature vectors {right arrow over (f)}.sub.i and {right arrow over (f)}.sub., may be interpreted as a measure of similarity between associated feature vectors, and is a normalized value. In this case, a match of a point i obtained from Part (a) of FIG. 1 and a point obtained from Part (b) of FIG. 1 may be defined as (i,). When a measure indicative of whether the match (i,) is a point-to-point potential match appropriate to describe a pattern is defined as a point-wise threshold T.sub.feature the match (i,) may be interpreted as a potential point-to-point match appropriate to describe a pattern match if the match (i,) satisfies d(i,)>T.sub.feature.

[0087] A conflict graph G.sub.C may be generated from the graphs G.sub.A and G.sub.B as a measure used to measure the similarity between the graphic representations of Part (a) and Part (b) of FIG. 1. The conflict graph G.sub.C may be generated by sequentially adding potential point-to-point matches, appropriate to describe the pattern, from the match (i,) having the largest d(i,) value as the vertices V.sub.i of the conflict graph G.sub.C. The process of generating the conflict graph G.sub.C may be repeated until all appropriate point-to-point matches are included.

[0088] The edges (i,;j,) within the conflict graph G.sub.C may be interpreted as measures representative of the geometric consistency between vectors {right arrow over (f)}.sub.i and {right arrow over (f)}.sub.j within the labeled graph G.sub.A corresponding to the image of Part (a) of FIG. 1 and the geometric consistency between vectors {right arrow over (f)}.sub. and {right arrow over (f)}.sub. within the labeled graph G.sub.B corresponding to the image of Part (b) of FIG. 1 through encoding.

[0089] The geometric consistency d(i,, j, )=d.sub.geometic({right arrow over (g)}.sub.ij, {right arrow over (g)}.sub.) of two pairs between the points of interests may be computed for all vertex pairs (V.sub.i,V.sub.j) (in the case where ij and ) within the conflict graph G.sub.C.

[0090] In this case, the geometric consistency d(i,, j, )=d.sub.geometic({right arrow over (g)}.sub.ij, {right arrow over (g)}.sub.) may be a normalized value.

[0091] The geometric consistency may become a means for measuring the geometric compatibility of matched point-to-point pairs (i,) and (j,). In this case, since the geometric consistency may depend on the residual differences of influences attributable to the local displacement, scaling and rotation of associated points of interests after changes attributable to the global translation, rotation and scaling between the matched point-to-point pairs (i,) and (j, ) have been normalized, the geometric consistency may become a means for measuring geometric compatibility.

[0092] When the residual difference after the normalization is excessively large, the pairs (i,) and (j, ) are in geometric conflict, and this is interpreted as not being compatible, with the result that matching is not allowed.

[0093] When the geometric consistency d(i,, j, )=d.sub.geometic({right arrow over (g)}.sub.ij, {right arrow over (g)}.sub.) satisfies the condition d(i, , j, )<T.sub.geometic for the geometric conflict threshold T.sub.feature the point-to-point pairs (i,) and (j, ) are considered to be in geometric conflict.

[0094] The conflict graph G.sub.C has an edge for vertex pair (V.sub.i,V.sub.j).sub.if in geometric conflict. In this case, ij and , as described above. In this manner, the conflict graph G.sub.C may have a maximum of L vertices. The maximum independent set of the conflict graph is equivalent to the maximum common subgraph of the unlabeled graphs G.sub.A and G.sub.B.

[0095] A process of finding the maximum independent set of the conflict graph G.sub.C may be interpreted as a quadratic unconstrained binary optimization problem, as shown in Equation 2 below:

[00001]$\begin{array}{cc}{\stackrel{.fwdarw.}{x}}_{\mathrm{opt}}=\mathrm{argmin}.Math.\left\{\stackrel{N}{\underset{i.Math..Math.<j.Math..Math.}{.Math.}}.Math..Math.Q_{i.Math..Math.,j.Math..Math.}.Math.x_{i.Math..Math.}.Math.x_{j.Math..Math.}\right\},x_{i.Math..Math.}\left\{0,1\right\}& \left(2\right)\end{array}$

[0096] In this case, Q.sub.i, i=1 for all the vertices, and Q.sub.i, j=L for the point-to-point pairs (i,) and (j, ) having edges on the conflict graph.

[0097] In the minimum energy configuration, x.sub.i=1 only when V.sub.i belongs to the maximum independent set, and x.sub.i=0 in the other cases. Equation 2 corresponds to a well-known NP-hard problem. It is well known that as L increases, computation time increases at a very fast speed.

[0098] In the following, Equation 2 is modified to be applied to adiabatic quantum computation for a quantum Ising model.

[0099] When the column vector of N Boolean variables is

[00002]$X=\left(\begin{array}{c}{x}_1\\ x_2\\ x_3\\ \mathrm{.Math.}\\ x_N\end{array}\right)$

and an NN matrix is Q, Equation 2 may be represented by Equation 3 below:

X.sub.opt=arg min X.sup.QX where x.sub.i{0,1}(3)

[0100] Meanwhile, a quantum Ising problem may be represented by Equation 4 by applying the relation S=2X 1 to Equation 3 and changing a variable from X to S:

S.sub.opt=arg min{S.sup.JS+h.sup.554 S} where S.sub.i{1,1}(4)

[0101] The variable S is referred to as a quantum-mechanical spin, and a solution to the quantum Ising model can be found by adiabatic quantum computing (AQC), i.e., a specific model of quantum computing.

[0102] In quantum mechanics, S=1 of spin states can be represented by qubits that refer to orthogonal vectors in a Hilbert space. The two states of qubits may be described by the vectors of Equation 5 below:

[00003]$\begin{array}{cc}.Math.0=\left(\begin{array}{c}1\\ 0\end{array}\right).Math..Math.\mathrm{and}.Math..Math..Math.1=\left(\begin{array}{c}0\\ 1\end{array}\right)& \left(5\right)\end{array}$

[0103] Qubits may be extended to vectors by linear combination called superposition, and this process is represented by Equation 6 below:

|=|0+|1 with ||.sup.2+||.sup.2=1(6)

[0104] A large quantum system is configured via the tensor product of individual qubit vector spaces. For example, this may be represented by Equation 7 below:

[00004]$\begin{array}{cc}.Math.01=.Math.0.Math..Math.1=\left(\begin{array}{c}0\\ 1\\ 0\\ 0\end{array}\right)& \left(7\right)\end{array}$

[0105] The superposition of N qubit states may have associated amplitudes representative of probabilities that individual N spin states can be observed.

[0106] Furthermore, a single qubit operator may be defined as

[0107] Equation 8 below:

[00005]$\begin{array}{cc}I=\left(\begin{array}{cc}1& 0\\ 0& 1\end{array}\right).Math..Math.\mathrm{and}.Math..Math.{}^Z=\left(\begin{array}{cc}1& 0\\ 0& -1\end{array}\right)& \left(8\right)\end{array}$

[0108] When the operator of Equation 8 is applied to the qubit vector of Equation 5, Equation 9 can be obtained:

.sup.Z|0=|0, .sup.Z|1=|1(9)

[0109] In a two-qubit state, the operator .sup.Z .Math.I may extract the classical spin of a first qubit, the operator I.Math..sup.Z may extract the classical spin of a second qubit, and the operator .sup.Z.Math..sup.Z may extract the product of the two classical spins.

[0110] The N spin quantum Ising model is represented by 2.sup.N2.sup.N Hamiltonians, as shown in Equation 10 below:

[00006]$\begin{array}{cc}{H}_I=\underset{i,j}{.Math.}.Math..Math.J_{\mathrm{ij}}.Math.{}_i^Z.Math.{}_j^Z+\underset{i}{.Math.}.Math..Math.h_i.Math.{}_i^Z& \left(10\right)\end{array}$

where .sup.Z.sub.i refers to the operator .sup.Z that acts upon an i.sup.th qubit.

[0111] In order to initiate the quantum system, another type of spin operator .sup.Z may be defined, as shown in Equation 11 below:

[00007]$\begin{array}{cc}{}^x=\left(\begin{array}{cc}1& 0\\ 0& 1\end{array}\right)& \left(11\right)\end{array}$

[0112] The spin operator of Equation 11 may flip the state of a qubit.

[0113] In this case, a ground state Hamiltonian may be represented by a spin operator acting upon an i.sup.th qubit, as shown in Equation 12 below:

[00008]$\begin{array}{cc}{H}_0=.Math.\underset{i}{.Math.}.Math..Math.{}_i^x& \left(12\right)\end{array}$

[0114] Furthermore, the eigenstate of the ground state Hamiltonian of Equation 12 may be represented by Equation 13 below:

[00009]$\begin{array}{cc}{.Math.}_0=.Math.\left(\frac{{.Math.0}_i-{.Math.1}_i}{\sqrt{2}}\right)& \left(13\right)\end{array}$

[0115] The time dependency of the quantum state of Equation 13 may be represented by the Schrodinger equation of Equation 14 below:

[00010]$\begin{array}{cc}i.Math..Math..Math.\frac{}{t}.Math..Math.\left(t\right)=H\left(t\right).Math..Math.\left(t\right)& \left(14\right)\end{array}$

[0116] Using an initial condition, given as |(0)|=|, at t=0, to adiabatically solve the quantum Ising model, the convex form of the adiabatic Hamiltonian may be represented by

[0117] Equation 15 below:

[00011]$\begin{array}{cc}H\left(t\right)=\left(1-\frac{t}{T}\right).Math.H_0+\frac{t}{T}.Math.H_I& \left(15\right)\end{array}$

[0118] At t=0, the quantum system has the lowest energy state. In this case, the lowest energy state may provide an equivalent probability for all classical configurations. In contrast, at t=T, design is made to correspond to the quantum Ising model problem adapted to solve the machine vision problem.

[0119] In this manner, the NP-hard problem that cannot be processed in a classical manner may be solved through the quantum-mechanical adiabatic evolution of the given quantum system.

[0120] The globally lowest classical configuration obtained through adiabatic quantum computing (AQC) may be a solution to the quadratic unconstrained binary optimization problem, which is a complex computational problem related to machine vision. It has been mathematically proved that quantum computing can provide exponential speed-up to the solution of the NP-hard problem compared to a classical method.

[0121] A process of training the adiabatic quantum computing system to solve the quadratic unconstrained binary optimization problem defined by Equation 2 starts with the training of hardware using a classification algorithm. The classification algorithm is represented by Equation 16 below:

[00012]$\begin{array}{cc}y=\mathrm{sign}\left(\underset{i=1}{\overset{N}{.Math.}}.Math..Math.{}_i.Math.h_i\left(x\right)\right)& \left(16\right)\end{array}$

[0122] In this case, xR.sup.M is an input pattern to be classified, and y{1,1} is an output classified by a classifier. h.sub.i:R.sup.M{1,1} is a function of x adapted to detect a feature, and .sub.i{0,1} is a weight for optimization during training.

[0123] The training may be performed by solving a discrete optimization problem represented by Equation 17 below:

[00013]$\begin{array}{cc}{\underset{\_}{}}_{\mathrm{opt}}=\underset{}{\mathrm{argmin}}\left(\underset{\_}{{\underset{\_}{}}^{}\left(\frac{1}{N^2}.Math.\underset{s=1}{\overset{S}{.Math.}}.Math..Math.\underset{\_}{h}\left({\underset{\_}{x}}_s\right).Math.{\underset{\_}{h}\left({\underset{\_}{x}}_s\right)}^{}\right).Math.\underset{\_}{}+\underset{\_}{}\left(.Math..Math.I-2.Math.\underset{s=1}{\overset{S}{.Math.}}.Math..Math.\frac{\underset{\_}{h}\left({\underset{\_}{x}}_s\right)}{N}.Math.y_s\right)}\right)& \left(17\right)\end{array}$

[0124] In this case, Equation 17 is described for S training samples {(x.sub.s, y.sub.s)|s=1,2, . . . , S}.

[0125] In the following, equations capable of implementing hardware for solving the machine vision problem and describing the hardware will be described. The hardware may be implemented based on the quantum Ising model of trapped ion spin-phonon chains. The quantum Ising model of trapped ion spin-phonon chains is a dipole-based physical model. An optimized solution to a specific vector for the solution of the machine vision problem may be obtained by matching a dipole to the specific vector for the solution of the machine vision problem and optimizing the state variable of a dipole through quantum-mechanical adiabatic evolution. The dipole is a physical property having an orientation and a scale attributable to a magnetic field or the like. The dipole is each element of the quantum Ising model, and may be matched to a specific vector for the solution of the machine vision problem.

[0126] In theory, the dipole may model the system to describe the feature vectors {right arrow over (f)}.sub.i and {right arrow over (f)}.sub., and may model the system to describe relation vectors {right arrow over (g)}.sub.i,j in which a point of interest (a feature point) i is a start point and a point of interest (a feature point) j is an end point in the image of Part (a) of FIG. 1. When the dipole is modeled to describe the feature vectors {right arrow over (f)}.sub.i and {right arrow over (f)}.sub., the similarity d(i,)=d.sub.feature({right arrow over (f)}.sub.i, {right arrow over (f)}.sub.) between associated feature vectors may be obtained by obtaining the scalar product of dipoles. When the dipole is modeled to describe the relation vectors {right arrow over (g)}.sub.i,j, the geometric consistency d(i,,j,)=d.sub.geometic({right arrow over (g)}.sub.ij, {right arrow over (g)}.sub.of two pairs may be obtained by computing the dipoles.

[0127] A Hamiltonian adapted to solve the quantum Ising model of trapped ion spin-phonon chains may be represented by Equation 18:

[00014]$\begin{array}{cc}H=\underset{i,j}{.Math.}.Math..Math.J_{j,l}^{.Math..Math.k}.Math.{}_j^Z.Math.{}_l^Z+H_{\mathrm{ph}}& \left(18\right)\end{array}$

[0128] In this case, J.sup.k.sub.j,i is a value that can be tuned by an external bias, and represents Ising interaction for spin-phonon chains. Furthermore, H.sub.ph represents a chain that is described by a hopping phonon. Meanwhile, since the Hamiltonian refers to an energy state, setting up the Hamiltonian as an objective function and obtaining an optimized Hamiltonian (a state in which energy is minimum) may be understood as obtaining an optimized objective function.

[0129] Individual charge detection may be performed by a spin-dependent single-electron tunneling event through the single-shot detection of a sensitive charge detector capacitively coupled to a quantum dot.

[0130] The Hamiltonian of the N quantum dot system may be represented by Equation 19 below:

[00015]$\begin{array}{cc}{H}_I=\underset{i<j}{.Math.}.Math..Math.J_{\mathrm{ij}}.Math.{}_i^Z.Math.{}_j^Z+\underset{i}{.Math.}.Math..Math.h_i.Math.{}_i^Z& \left(19\right)\end{array}$

[0131] The parameters J.sub.ij and h.sub.i are positive variables, and the variables may be changed by the application of external voltages. These bias voltages of gates may be controlled to obtain a Hamiltonian appropriate for a required quantum operator.

[0132] An initial Hamiltonian at t=0 may be obtained by setting a bias so that J.sub.ij=0 for all I and j. The variable T of Equation 15 may be defined as T=/E. In this case, E represents energy separation between initial ground state energy and global minimum energy.

[0133] FIG. 2 is a diagram showing conventional pattern recognition machine vision modeling.

[0134] The images of FIGS. 2 and 3 were quoted from Image recognition with an adiabatic quantum computer, I. Mapping to quadratic unconstrained binary optimization, Hartmut Neven et al., Apr. 28, 2008.

[0135] Part (a) of FIG. 2 shows a reference image, and Part (b) of FIG. 2 shows an input image.

[0136] General pattern recognition machine vision modeling is performed by pairing feature points in two images having the same structure in the real world. In this case, similarity and geometric consistency may become criteria that are used to determine the extent to which two images are similar to each other. According to this, the vertices of a pyramid, the eye, nose and mouth portion of the face of the Sphinx, the boundary points of a collapsed portion of a pyramid, etc. may become the reference points of pairing. However, strictly speaking, each of these items is not a single point, but is defined as a single feature region having a plurality of feature points. It may be represented by different scales and orientations in Part (a) and Part (b) of FIG. 2 depending on the angle at which it is seen.

[0137] FIG. 3 shows a process of modeling feature points within two images of Part (a) and Part (b) of FIG. 2 as vectors and a labeled graph. As shown in Part (a) of FIG. 3, the relation vectors between feature points i, j and k may be defined for the image of Part (a) of FIG. 2. In Part (b) of FIG. 3, the relation vectors between feature points , and may be defined for the image of Part (b) of FIG. 2. A process of describing each image by using a labeled graph, i.e., a set of relation vectors between feature points, as described above, is shown via FIG. 3.

[0138] From the embodiment of FIG. 3, it can be intuitively seen that the feature points i, j and k of Part (a) of FIG. 3 correspond to the feature points , and of Part (b) of FIG. 3. Since pattern recognition has been performed already by the eyes of a human, the correspondences of the feature points between the images can be clearly recognized. However, in machine vision, all the computation starting from computation related to the determination of which of the feature points , and corresponds to the feature point i or the selection of a point as a feature point must be performed, and thus this computation corresponds to an NP-hard problem that requires more computational load in proportion to the number of feature points.

[0139] FIG. 4 is a diagram showing the relationships between points of interest at principal points of interest according to an embodiment of the present invention.

[0140] In FIG. 4, i, j and k refer to the feature points of Part (a) of FIG. 3, and , and refer to the feature points of Part (b) of FIG. 3. Assuming that Part (b) of FIG. 3 is formed by moving or twisting Part (a) of FIG. 3, Part (b) of FIG. 3 and Part (a) of FIG. 3 may be compared by observing changes in feature locations in the present invention. The displacements of the feature points of Part (a) and Part (b) of FIG. 3 may not be determined using only the viewpoint of a camera. However, it may be expected that adjacent feature points have been displaced by a similar method.

[0141] That is, it can be expected that adjacent feature points will be moved together. As a result, image matching may be considered to be a type of optimization problem. This optimization problem may be modeled by an attempt to minimize an objective function between two terms.

[0142] An example shown in FIG. 4 is an embodiment showing mapping between the feature points of Part (a) of FIG. 3 and the feature points of Part (b) of FIG. 3. That is, although the feature point i may be mapped to one of the feature points , and , a case where the feature point is mapped to the feature points is shown in FIG. 4. The matched point-to-point pair (i, ) is shown as a vertex of the graph of FIG. 4.

[0143] If an objective function is designed to depend on the differences between the relationships (a set of arrows) between feature points in Part (a) of FIG. 3 and the relationships between feature points in Part (b) of FIG. 3, the pattern of Part (a) of FIG. 3 and the pattern of Part (b) of FIG. 3, which are described to be most similar to each other, may be found by minimizing the objective function.

[0144] The objective function is the energy of a quantum system in quantum computing. When energy is minimized, the quantum system converges into the most stabilized state, and thus the pattern of Part (a) of FIG. 3 and the pattern of Part (b) of FIG. 3 in the most stabilized state and the displacement relationships between Part (a) of FIG. 3 and Part (b) of FIG. 3 may be calculated. In this case, the displacement relationships refer to translation, rotation, and twisting between the images of Part (a) and Part (b) of FIG. 3.

[0145] In this case, the relationships between feature points are not necessarily formed to be a single pattern, and may be formed to be various patterns. Accordingly, an optimized pattern may be extracted as a first pattern. The minimum energy may be given a different value depending on the setting to being global or local. Depending on a set range, the result of the computation of an optimized pattern may vary slightly.

[0146] FIG. 5 is a detailed diagram showing an image pattern recognition computation apparatus for machine vision according to an embodiment of the present invention.

[0147] In machine vision using a quantum system, a computation apparatus 500 for recognizing patterns between images includes a modeling unit 510 and an interpretation unit 520.

[0148] The modeling unit 510 sets up an objective function based on the similarity between a first pattern derived from the relationships between points of interests of a first image and a second pattern derived from the relationships between points of interests of a second image.

[0149] In this case, the first image refers to an input image, e.g., the image of Part (a) of FIG. 1 for ease of description. The second image refers to a reference image, e.g., the image of Part (b) of FIG. 1. Furthermore, the points of interest have the same meaning as feature points of each of the images. However, in another embodiment, an input image and a reference image do not necessarily follow this example, but may correspond to the second and first images in a reversed manner.

[0150] The interpretation unit 520 finds an optimum first pattern and an optimum second pattern, in which the similarity between the first pattern and the second pattern modeled by the modeling unit 510 is optimized, by interpreting a final quantum state obtained through the adiabatic evolution process of the quantum system in which the objective function is optimized.

[0151] In this case, the quantum system is characterized by including a physical model that depends on the interaction between dipoles, and may model the pattern of an image, i.e., a combination of vectors using the interaction between physical dipoles of the image.

[0152] Furthermore, the quantum system may include an Ising model that depends on the dipole interaction of a magnetic body.

[0153] In this case, the Ising model uses a physical model that depends on trapped ion-based spin-phonon coupling.

[0154] Furthermore, the quantum system may include a physical model that has energy corresponding to the objective function.

[0155] Accordingly, machine vision-related complex computational problem may be modeled as the interaction between relation vectors between points of interest at principal points of interest, and the machine vision-related complex computational problem may be solved through the modeling.

[0156] FIG. 6 is a detailed diagram showing a quantum system-based machine vision apparatus according to an embodiment of the present invention.

[0157] A machine vision apparatus 600 for recognizing patterns between images using a quantum system includes an optical module 610, memory 620, and a processor 630. The processor 630 includes a modeling unit 631, and an interpretation unit 632.

[0158] The optical module 610 acquires a first image. The optical module 610 is the generic terms for devices which form an image of an object in a space by using reflection, refraction, absorption, interference and diffraction, i.e., the characteristics of light radiated by the sun or an electric light, or which are used to investigate the characteristics of a specific object by analyzing radioactive rays emitted from the object.

[0159] The processor 630 derives a first pattern from the relationships between points of interests of the first image acquired by the optical module 610, and derives a second pattern from the relationships between points of interests of a second image. In this case, the second image is the reference image of the first image, and refers to an image into which the first image has been changed.

[0160] The memory 620 functions to store the first pattern and the second pattern that are derived by the processor 630.

[0161] The modeling unit 631 included in the processor 630 sets up an objective function based on the similarity between the first pattern and the second pattern stored in the memory 620. The interpretation unit 632 finds an optimum first pattern and an optimum second pattern, in which the similarity between the first pattern and the second pattern are optimized, by interpreting a final quantum state obtained through the adiabatic evolution process of the quantum system in which the objective function is optimized.

[0162] In this case, the processor 630 vectorizes the relationships between the points of interests of the first image, models a set of the vectorized relationships between the points of interests of the first image as the first pattern, vectorizes the relationships between the points of interests of the second image, and models a set of the vectorized relationships between the points of interests of the second image as the second pattern.

[0163] Furthermore, the quantum system may include a physical model that depends on the interaction between dipoles.

[0164] FIG. 7 is a flowchart showing an image pattern recognition computation method for machine vision according to an embodiment of the present invention.

[0165] A computation method for recognizing patterns between images in machine vision by using a quantum system includes step S710 of setting up an objective function based on the similarity between a first pattern derived from the relationships between points of interests of a first image and a second pattern derived from the relationships between points of interests of a second image.

[0166] In this case, the relationships between the points of interests of the first image may be vectorized, a set of the vectorized relationships between the points of interests of the first image may be modeled as the first pattern, the relationships between the points of interests of the second image may be vectorized, and a set of the vectorized relationships between points of interests of the second image may be modeled as the second pattern.

[0167] In this case, the first image refers to an input image, and the second image refers to an image into which the first image has been changed, i.e., the reference image of the first image.

[0168] At step S720, an optimum first pattern and an optimum second pattern in which the similarity between the first pattern and the second pattern are optimized are found by interpreting a final quantum state obtained through the adiabatic evolution process of the quantum system in which the objective function is optimized.

[0169] In this case, the quantum system is characterized by including a physical model depending on the interaction between dipoles, and is characterized by including an Ising model depending on the dipole interaction of a magnetic body.

[0170] In this case, the Ising model is characterized in that it is a physical model depending on trapped ion-based spin-phonon coupling, and the quantum system may include a physical model having energy corresponding to the objective function.

[0171] Accordingly, machine vision-related complex computational problem may be modeled as the interaction between relation vectors between points of interest at principal points of interest, and the machine vision-related complex computational problem may be solved through the modeling.

[0172] In the above embodiments, technologies related to the modeling of machine vision-related complex computational problem using the adiabatic evolution of a quantum system and quantum computing have been disclosed. The idea of the present invention contemplates that the objective function of Equation 2 may be set up, a quantum system may be selected and implemented to include a physical model depending on the interaction between dipoles, and the pattern of an image, i.e., a combination of vectors of the image, may be modeled as the interaction between physical dipoles. The modeled Hamiltonian of the quantum system is given as Equation 10.

[0173] Although the idea of the present invention has been disclosed with a focus on the embodiments of the image pattern recognition of machine vision, the fields of application of the present invention are not limited only to machine vision. The problem of the pattern recognition disclosed above may be equivalently converted and applied to a problem of artificial intelligence or machine learning capable of representing pattern recognition as a combination of vectors.

[0174] Since the optimization problem in the field of artificial intelligence or machine learning may model an optimization process as a process of finding the optimum state of a Hamiltonian by using the adiabatic evolution of a quantum system, the ideal of the present invention may be applied to various fields.

[0175] An example of an application in the field of machine learning to which the idea of the present invention is applicable is a deep learning technique that has attracted attention recently. The deep learning technique is a technique by which a computer machine performs learning so that it can operate in accordance with the human way of thinking. For example, the deep learning technique is designed to represent patterns between elements of data by combinations of vectors based on interrelationships and combine the patterns, thereby recognizing the combination of the patterns as a higher grouped pattern. In the case of image information, an example in which the relationships between pixels are represented by column vectors is used. The deep learning technique refers to a process in which a computer performs learning to distinguish objects by imitating humans' information processing method in which the brain finds patterns among a massive amount of data and distinguishes objects. In the case of the deep learning technique, a process of finding a pattern is performed through the recognition and reasoning of a computer itself.

[0176] Conventional general deep learning techniques employ specific techniques, such as a deep neural network, a convolutional deep neural network, a deep belief network, etc., in order to represent a complex pattern recognition process so that computers can recognize the process. The conventional general deep learning techniques are applied to the fields of computer vision, voice recognition, natural language processing, voice/signal processing, etc.

[0177] Deep learning techniques based on pattern recognition entails a process of selecting only significant pattern candidates from among a number of potential pattern candidates. This process is shown in FIG. 8. FIG. 8 introduces a process of distinguishing between and recognizing the face of a human and the face of a cat by using deep learning. Although a number of pattern candidates are set up, only significant pattern candidates are finally selected, and the comprehensive evaluation of the images is performed based on the selected patterns.

[0178] The idea of the present invention is intended to replace the computation of all or part of a technique, such as a conventional complex deep neural network, through adiabatic evolution using a quantum system. It is well known that there are many cases where the neural network does not reach a global optimum due to an initial value and converges into a local optimum. To overcome this constraint, a hierarchical deep neural network or the like is used. However, fundamentally, it cannot be certain due to the characteristics of the neural network that a presented solution is a global optimum. There are cases where an exhaustive search for an initial state must be performed to obtain a global optimum. Furthermore, techniques, such as a deep neural network or the like, require considerable computational times, and thus cannot provide sufficiently desirable quality to real-time pattern recognition applications. The idea of the present invention is intended to reduce the time required for optimization computation by using the adiabatic evolution of a quantum system and to provide sufficiently desirable quality to real-time pattern recognition applications.

[0179] The idea of the present invention is applicable to cases where the relationships between the entries of data can be represented by respective vectors and the similarity between grouped patterns represented by combinations of vectors is analyzed. In connection with the combinations of vectors, for example, in the case of news, the relationships between pieces of text, i.e., contexts, are extracted, and may be provided as vectorized representations. In the case of voice data, changes in the frequency components over time may be extracted, and may be provided as vectorized representations. In this case, the vectors may be modeled as the physical state variables of the quantum system. For example, the Hamiltonian of Equation 10 is applicable to a case where the states of the quantum system may be described via dipoles, as described above.

[0180] The idea of the present invention may be also applicable to the reading and diagnosis of medical images, applications designed to predict the effectiveness of a new medicine or detect a side effect in advance in the pharmaceutical industry, pattern recognition for the planning of marketing or automation of customer management, and applications designed to extract significant information from a massive amount of data obtained via the Internet of Things and analyze the extracted information.

[0181] The quantum system-based image pattern recognition computation method for machine vision according to an embodiment of the present invention may be implemented in the form of program instructions that can be executed by a variety of computer means, and may be stored in a computer-readable storage medium. The computer-readable storage medium may include program instructions, a data file, and a data structure solely or in combination. The program instructions that are stored in the medium may be designed and constructed particularly for the present invention, or may be known and available to those skilled in the field of computer software. Examples of the computer-readable storage medium include magnetic media such as a hard disk, a floppy disk and a magnetic tape, optical media such as CD-ROM and a DVD, magneto-optical media such as a floptical disk, and hardware devices particularly configured to store and execute program instructions such as ROM, RAM, and flash memory. Examples of the program instructions include not only machine language code that is constructed by a compiler but also high-level language code that can be executed by a computer using an interpreter or the like. The above-described hardware components may be configured to act as one or more software modules that perform the operation of the present invention, and vice versa.

[0182] According to the present invention, machine vision-related complex computational problem can be accurately solved by modeling the machine vision-related complex computational problem as the interaction between relation vectors between points of interest at principal points of interest in each image.

[0183] The present invention provides the advantage of easily solving machine vision-related complex computational problem by applying a physical model regarding the interaction between dipoles to interaction between vectors and using Ising model-based quantum computing hardware.

[0184] Furthermore, the present invention provides the advantage of configuring quantum computing hardware by using an Ising model, particularly trapped ion-based spin-phonon coupling, and solving a machine vision-related complex computational problem based on a physical model capable of solving the configured quantum computing hardware.

[0185] While the present invention has been described in conjunction with specific details, such as specific elements, and limited embodiments and diagrams, above, these are provided merely to help an overall understanding of the present invention. The present invention is not limited to these embodiments, and various modifications and variations can be made based on the foregoing description by those having ordinary knowledge in the art to which the present invention pertains.

[0186] Therefore, the technical spirit of the present invention should not be determined based only on the described embodiments, and not only the following claims but also all equivalents to the claims and equivalent modifications should be construed as falling within the scope of the spirit of the present invention.