IMAGING APPARATUS AND IMAGING METHOD

Abstract

An imaging apparatus according to an aspect of the present disclosure includes: a waveguide that transmits light emitted from a light source; a detector that detects power of a speckle pattern generated by the light passing through the waveguide and applied to an object; at least one memory storing a set of instructions; and at least one processor configured to execute the set of instructions to reconstruct an image of the object based on the power obtained by making the light enter the waveguide under different conditions.

Claims

1. An imaging apparatus comprising: a waveguide that transmits light emitted from a light source; a detector that detects power of a speckle pattern generated by the light passing through the waveguide and applied to an object; at least one memory storing a set of instructions; and at least one processor configured to execute the set of instructions to reconstruct an image of the object based on the power obtained by making the light enter the waveguide under different conditions.

2. The imaging apparatus according to claim 1, further comprising a modulator that changes the conditions under which the light enters the waveguide.

3. The imaging apparatus according to claim 2, wherein the modulator changes the conditions by changing an incident angle of the light on the waveguide.

4. The imaging apparatus according to claim 2, wherein the modulator changes the conditions by changing a wavefront of the light.

5. The imaging apparatus according to claim 1, wherein the waveguide is a multimode waveguide.

6. The imaging apparatus according to claim 1, wherein the at least one processor is configured to execute the set of instructions to reconstruct the image based on an alternating direction method of multipliers (ADMM) algorithm.

7. The imaging apparatus according to claim 6, wherein the at least one processor is configured to execute the set of instructions to reconstruct the image based on the ADMM algorithm by using a transformation matrix obtained for the conditions and a speckle pattern generated under the conditions.

8. The imaging apparatus according to claim 1, wherein the light source generates the light of a plurality of wavelengths, and the at least one processor is configured to execute the set of instructions to: separate the light generated by the light source regarding the plurality of wavelengths; and reconstruct the image concerning the light for each of the plurality of wavelengths based on the power of the light, the power being detected for each of the plurality of wavelengths.

9. The imaging apparatus according to claim 1, wherein the light source is capable of generating light of a selected wavelength among a plurality of wavelengths, and the at least one processor is configured to execute the set of instructions to: control a wavelength of the light generated from the light source; and reconstruct the image concerning the light for each of the plurality of wavelengths based on the power of the light, the power being detected for each of the plurality of wavelengths.

10. The imaging apparatus according to claim 2, wherein the modulator changes the conditions by changing a wavelength of the light.

11. An imaging method comprising: detecting power of a speckle pattern generated by light passing through a waveguide and applied to an object; and reconstructing an image of the object based on the power obtained by making the light enter the waveguide under different conditions.

Description

BRIEF DESCRIPTION OF DRAWINGS

[0011] FIG. 1 is a diagram illustrating an imaging apparatus according to an example embodiment of the present invention.

[0012] FIG. 2 is a diagram illustrating a detailed exemplary configuration of the imaging apparatus according to the example embodiment of the present invention.

[0013] FIG. 3 is a diagram illustrating an exemplary configuration for obtaining a transformation matrix to be used in the imaging apparatus according to the example embodiment of the present invention.

[0014] FIG. 4 is a view illustrating an exemplary speckle pattern to be used in the imaging apparatus according to the example embodiment of the present invention.

[0015] FIG. 5 is a diagram illustrating a relationship between the transformation matrix and the speckle pattern to be used in the imaging apparatus according to the example embodiment of the present invention.

[0016] FIG. 6 is a flowchart illustrating operation of the imaging apparatus according to the example embodiment of the present invention.

[0017] FIG. 7 is a diagram illustrating an exemplary imaging apparatus according to a variation of the example embodiment of the present invention.

[0018] FIG. 8 is a diagram illustrating another exemplary imaging apparatus according to the variation of the example embodiment of the present invention.

[0019] FIG. 9 is a diagram illustrating another exemplary imaging apparatus according to the variation of the example embodiment of the present invention.

[0020] FIG. 10 is a diagram illustrating another exemplary imaging apparatus according to the variation of the example embodiment of the present invention.

[0021] FIG. 11 is a view illustrating an exemplary image to be reconstructed in a simulation example of the present invention.

EXAMPLE EMBODIMENT

[0022] An example embodiment of the present invention will be described with reference to the accompanying drawings. FIG. 1 is a diagram illustrating an imaging apparatus according to the example embodiment of the present invention.

[0023] As illustrated in FIG. 1, the imaging apparatus according to the example embodiment of the present invention includes a waveguide 110, a detector 120, and a reconstruction unit 130. The waveguide 110 transmits light. The detector 120 detects power of a speckle pattern generated by the light passing through the waveguide and applied to an object. The reconstruction unit 130 reconstructs an image of the object on the basis of a plurality of pieces of power obtained by emitting light into the waveguide under different conditions.

[0024] An imaging apparatus 100 has a configuration illustrated in FIG. 2 as a more specific example. In the exemplary configuration illustrated in FIG. 2, the imaging apparatus 100 includes, in addition to each element mentioned above, a light source 11, a modulator 12, a modulation control unit 13, an optical system 14, a beam splitter 15, and a lens probe 16. The imaging apparatus 100 reconstructs an image of a target 17.

[0025] In FIG. 2, a solid line arrow indicates a direction in which light generated from the light source 11 travels, and a dotted line arrow indicates a direction in which light reflected from the target 17 travels.

[0026] The light source 11 generates light to be applied to the object. In the present example embodiment, the light source 11 primarily generates light of a single wavelength. The modulator 12 modulates the light generated by the light source 11. Modulation of light in the present example embodiment will be described in detail later. The modulation control unit 13 controls a state of modulation by the modulator 12. The light modulated by the modulator 12 passes thorough the appropriately provided optical system 14, and enters the waveguide 110. The beam splitter 15 separates the light to be made incident on the waveguide 110 and applied to the object from the light reflected from the object.

[0027] In FIG. 2 and in the example of FIG. 3 to be described later, the light generated by the light source 11 enters the waveguide 110 from the left end of the waveguide 110 in each drawing, and is emitted from the right end in each drawing. In the following descriptions, among the end portions of the waveguide 110, an end portion (left end in FIG. 2 or 3) at which the light generated from the light source 11 enters may be referred to as an incident end, and an end portion (right end in FIG. 2 or 3) at which the light generated from the light source 11 enters may be referred to as an exit end.

[0028] The imaging apparatus 100 reconstructs an image using the compressed sensing technique described above. More specifically, the imaging apparatus 100 reconstruct the image on the basis of the power of light emitted from the waveguide 110, applied to the object, and detected by the detector 120.

[0029] In general, when the light emitted from the exit end of the waveguide 110 is applied to an object of some kind, a speckle pattern is generated due to interference or the like. The generated speckle pattern varies depending on the light incident on the waveguide 110. The imaging apparatus 100 uses such a diversity of near-field speckle patterns of light emitted from the waveguide 110 to reconstruct an image on the basis of the compressed sensing technique. By using such a method, it becomes possible to reconstruct an image with less observation data compared with the number of pixels of the image to be reconstructed. In the following descriptions, the speckle pattern generated by light emitted from the exit end of the waveguide 110 may be simply referred to as a speckle pattern.

[0030] The speckle pattern will be further described. As described above, the speckle pattern is generated by near-field light emitted from the exit end of the waveguide 110. The direction in which the waveguide 110 extends is defined as a z direction (direction indicated by the solid line arrow in the waveguide 110 in FIGS. 2 and 3), the cross-sectional direction of the waveguide 110 is defined as an x direction (upward direction orthogonal to the solid line arrow in the waveguide 110 in FIGS. 2 and 3), and a direction from the bottom to the top of the drawing is defined as a y direction. It is assumed that a light beam having an angular velocity is made incident on the waveguide 110. With respect to the speckle pattern in this case, if the distribution of the optical electric field intensity at the incident end of the waveguide 110 serving as a multimode waveguide is represented by E.sub.in and the distribution of the optical electric field intensity at the exit end is represented by E.sub.out, a relationship between the E.sub.in and E.sub.out can be expressed by the following expression (1).

[Math. 1]

E.sub.out=E.sub.in(x,y)e.sup.(i(k.sup.z.sup.zt) (1)

[0031] In the expression (1), and in the expression (1), a wavenumber k.sub.z in the z direction is k.sub.z=2/*cos . In the equation expressing k.sub.z, represents an incident angle of incident light, and represents a wavelength of the incident light.

[0032] That is, assuming that the waveguide 110 serves as a multimode waveguide, the incident light having been made incident on the waveguide 110 excites light of different waveguide modes depending on the incident angle and the like. Furthermore, the incident light is reflected on the inner surface of the waveguide 110 multiple times. For those reasons, the intensity distribution of E.sub.out, which is emitted light, becomes a pseudo-random distribution, for example. A pseudo-random distribution possesses regularity unlike an inherently random distribution. In addition, the regularity is not immediately apparent in the pseudo-random distribution unlike traditional distribution such as Gaussian distribution and Bernoulli distribution. The pseudo-random distribution of speckles on the fiber exit face can be estimated by tracing incident rays and linearly adding the multiple incident rays.

[0033] The intensity distribution of E.sub.out changes according to the wavelength and the incident angle of the incident light as expressed by the expression (1). That is, the speckle pattern generated by the light emitted from the waveguide 110 changes according to the wavelength and the incident angle of the incident light. In addition, the speckle pattern may also change depending on the wavefront of the incident light (i.e., plane constituted by a collection of points of the same phase).

[0034] Accordingly, the imaging apparatus 100 uses a plurality of speckle patterns, which changes when conditions of the incident light including the incident angle, wavelength, wavefront, and the like of the incident light are changed, to reconstruct an image on the basis of an ADMM algorithm. In the following descriptions, to change the wavefront means to change the phase of light at the same point.

[0035] In the following descriptions, changing conditions of the incident light, such as the incident angle, wavelength, and wavefront of the incident light to be made incident on the waveguide 110, will be called modulation.

[0036] Image reconstruction is carried out on the basis of the ADMM algorithm, which is one of techniques of the compressed sensing, using a transformation matrix D obtained according to the speckle pattern. Hereinafter, a procedure for obtaining the transformation matrix D and a procedure for reconstructing an image based on the ADMM algorithm using the transformation matrix D will be described.

[0037] The procedure for obtaining the transformation matrix D will be described. For example, the transformation matrix D is obtained using the configuration illustrated in FIG. 3. The number of rows and the number of columns of the transformation matrix D are determined depending on the number of pixels of the image to be reconstructed. Hereinafter, it is assumed that the image to be reconstructed is x, and the image x is an image of m pixels in width and n pixels in length. Hereinafter, the procedure for obtaining the transformation matrix D may be called a calibration process or simply called calibration.

[0038] In the configuration illustrated in FIG. 3, the waveguide 110, the light source 11, the modulator 12, the modulation control unit 13, and the optical system 14 are elements similar to those illustrated in FIG. 2. In addition, a camera 18 is provided to face the exit end of the waveguide 110 in FIG. 3. The camera 18 includes, for example, a sensor in which the number of pixels is equal to or more than the number of pixels of the image to be reconstructed by the imaging apparatus 100. The speckle pattern of the near-field light emitted from the exit end of the waveguide 110 is imaged on the sensor surface of the camera 18.

[0039] A camera according to the wavelength of light to be generated by the light source 11 is used as the camera 18. In a case where the light to be generated by the light source 11 is visible light, a camera of a general charge coupled device (CCD), a complementary metal-oxide-semiconductor (CMOS) image sensor, or the like is used as the camera 18. Furthermore, as the camera 18, an ultraviolet camera is used when the light is ultraviolet light, an indium gallium arsenide (InGaAs) camera is used in the case of near-infrared light, and a thermal image camera or the like is used in the case of mid/far-infrared light, for example. The arrangement of pixels including the number of pixels and the aspect ratio of the camera 18 is determined depending on the number of pixels and the arrangement of pixels of the image to be reconstructed. The sensitivity characteristic of the camera 18 is preferably close to that of the detector 120.

[0040] It is necessary to use the same speckle pattern in both the case of obtaining the transformation matrix D and the case of reconstructing the image. Therefore, the elements same as those of the imaging apparatus 100 using the obtained transformation matrix D are generally used as the elements of the waveguide 110, the light source 11, the modulator 12, and the optical system 14 illustrated in FIG. 3. In addition, the positional relationship at the time of disposing those elements needs to be the same condition as that of the imaging apparatus 100 using the obtained transformation matrix D.

[0041] As expressed by the expression (1) mentioned above, the distribution of the speckle pattern changes according to the modulation of the incident light. That is, the speckle pattern generated by the light emitted from the waveguide 110 changes according to the wavelength and the incident angle of the incident light. In addition, the speckle pattern may also vary depending on the wavefront of the incident light. In the following example, the transformation matrix D is obtained on the basis of a plurality of speckle patterns obtained by modulating the conditions of the incident light including the incident angle, wavelength, wavefront, and the like of the incident light.

[0042] At the time of obtaining the transformation matrix D, first, the light source 11 generates light, and the modulator 12 modulates the light generated by the light source 11. In the example illustrated in FIG. 3, an exemplary case where the incident angle of the incident light to the waveguide 110 is changed to modulate the incident light is assumed. In this example, the modulator 12 is a galvanometer mirror capable of changing an angle of a reflective surface.

[0043] The light modulated by the modulator 12 passes thorough the optical system 14, and enters the waveguide 110. The near-field light emitted from the exit end of the waveguide 110 is imaged on the sensor surface of the camera 18. FIGS. 4A to 4C are exemplary speckle patterns obtained in a case where the incident angle of the incident light to the waveguide 110 is changed.

[0044] FIG. 5 illustrates a relationship between the transformation matrix D and a speckle pattern 180 detected by the camera 18.

[0045] FIG. 5 schematically illustrates an exemplary case where the power of the speckle pattern 180 is detected in two gradations. That is, in the example illustrated in FIG. 5, a region where the power of the speckle pattern 180 is large is represented by a black square, and a region where the power of the speckle pattern 180 is small is represented by a white square for each pixel of the camera 18.

[0046] The power of the speckle pattern 180 for each pixel detected by the camera 18 is expressed as an mn dimensional vector Di (i=1, 2, . . . , k) relevant to the number of pixels (mn) of the image to be reconstructed. Therefore, in the exemplary transformation matrix D illustrated in FIG. 5, the power of the speckle pattern 180 detected by the camera 18 is expressed as an mn dimensional vector. The power of the speckle pattern 180 may be detected in multiple gradations depending on, for example, sensitivity resolution of the detector 120.

[0047] Such detection of the speckle pattern 180 by the camera 18 is repeated for different speckle patterns 180 as illustrated in FIGS. 4A to 4C. As described above, the speckle pattern 180 changes by the incident light to the waveguide 110 being modulated. For example, in a case where the modulator 12 is a galvanometer mirror, which is one of mechanisms for changing a reflection angle of reflected light, the speckle pattern 180 changes by light being made incident on the waveguide 110 at different incident angles.

[0048] Therefore, observation of such different speckle patterns 180 is carried out using the camera 18 while changing the state of modulation by the modulator 12.

[0049] FIG. 5 schematically illustrates an exemplary case where the speckle pattern 180 is changed. In the repeated detection of the speckle pattern 180, a plurality of vectors is obtained from the power of the signal for each pixel detected by the camera 18. Vectors D.sub.1 to D.sub.k of the number of k obtained by observations of the different speckle patterns 180 of k times constitute the transformation matrix D. That is, each row of the transformation matrix D is an mn dimensional vector D.sub.i (1ik) obtained for one speckle pattern 180.

[0050] The number of observations (k) of the speckle pattern 180 may be determined appropriately depending on conditions such as image quality required for the image to be reconstructed and a type of the modulator 12 or the detector 120.

[0051] Next, a procedure for reconstructing an image using the transformation matrix D will be described. Image reconstruction is carried out on the basis of the ADMM algorithm. As described above, the image x to be reconstructed is assumed to be an image of m pixels in width and n pixels in length.

[0052] In the imaging apparatus 100, an image is reconstructed using a plurality of total sums of power of the speckle patterns generated when light from the light source 11 is made incident on the waveguide 110 and imaged on the target.

[0053] As described above, it is necessary to use the same speckle pattern in both the case of obtaining the transformation matrix D and the case of reconstructing the image. Accordingly, the modulator 12 is controlled by the modulation control unit 13 in such a manner that the same speckle pattern is generated in both the case of obtaining the transformation matrix D and the case of reconstructing the image. In both the case of obtaining the transformation matrix D and the case of reconstructing the image, observation is carried out the same number of times, which is k times.

[0054] In the present example embodiment, it is assumed that the detector 120 detects only the power of the speckle pattern applied to the target 17 without detecting positional information such as power distribution of the speckle pattern. Therefore, the total sum of power detected by the detector 120 is the sum of signal intensity of all the pixels of the image to be reconstructed.

[0055] In this case, power j.sub.i of the signal to be detected by the detector 120 with respect to the speckle pattern relevant to the vector D.sub.i constituting any row of the transformation matrix D for the image x to be reconstructed by the imaging apparatus 100 is expressed by the following expression (2).

[00001]$\left[\mathrm{Math}..Math.2\right]$$\begin{array}{cc}{j}_i=\underset{k=1}{\overset{m*n}{.Math.}}.Math..Math.d_k^i*x_k& \left(2\right)\end{array}$

[0056] In the expression (2), d.sup.i.sub.k represents one of the mn elements included in the vector D.sub.i described above (see the transformation matrix D in FIG. 5), and x.sub.k represents a value obtained by sequentially arranging a value representing the signal intensity of each of the mn pixels included in the image x. Values of d.sup.i.sub.k and x.sub.k represent values at relevant positions in the image.

[0057] It is assumed that power of a signal is detected by the detector 120 for speckle patterns of the number of k relevant to D.sub.1 to D.sub.k. In this case, the relationship of the following expression (3) is obtained with respect to vector j.sub.k1=(j.sub.1, j.sub.2, . . . , j.sub.k) of the power j.sub.i (i=1 to k) of the k signals detected for D.sub.1 to D.sub.k.

[Math. 3]

D.sub.kmnx.sub.mn1=j.sub.k1 (3)

[0058] In the expression (3), the superscript in each of D.sub.kmn, x.sub.mn1, and j.sub.k1 indicates the number of elements of the matrix representing each value. D.sub.kmn represents a transformation matrix of k rows and (mn) columns. That is, the transformation matrix D.sub.kmn is composed of k vectors D.sub.i (i=1 to k) each containing (mn) elements. A vector of (mn) rows is represented by x.sub.mn1. A vector of k rows is represented by j.sub.k1. In the following descriptions, D, x, and j without subscripts represent the contents same as D.sub.kmn, x.sub.mn1, and j.sub.k1, respectively.

[0059] In the compressed sensing, a solution to the minimization problem expressed by the following expression (4) is obtained to reconstruct the image. That is, by obtaining the L1 norm solution of the expression (4), the image x is reconstructed from the detection results of the power of the k signals described above.

[Math. 4]

minx.sub.xs.t. j=Dx (4)

[0060] In the expression (4), x.sub.1 represents the L1 norm of x.

[0061] The solution to the expression (4) mentioned above is obtained according to the procedure to be described below using the ADMM algorithm. First, a cost function represented by the following expression (5) is considered.

[Math. 5]

L(x)=x.sub.1+v.sup.T(jDx) (5)

[0062] The expression (5) is a cost function associated with the method of Lagrange multiplier. In the expression (5), v represents a Lagrange undetermined multiplier.

[0063] In the expression (4), a new variable z is introduced to distinguish x for the L1 norm from another x. The problem of minimizing the L(x) expressed in the expression (5) is replaced with the conditional minimization problem expressed in the following expression (6). In the expression (6), represents a cost coefficient associated with the method of Lagrange multiplier.

[00002]$\left[\mathrm{Math}..Math.6\right]$$\begin{array}{cc}\underset{x,z}{\min }.Math.\left\{{.Math.z.Math.}_1+\frac{1}{2.Math.}.Math.{.Math.j-\mathrm{Dx}.Math.}_2^2\right\}.Math..Math.s.t..Math.z-j=0& \left(6\right)\end{array}$

[0064] According to the procedure of the augmented Lagrangian method, a new cost function expressed in the following expression (7) is minimized.

[00003]$\left[\mathrm{Math}..Math.7\right]$$\begin{array}{cc}{L}_{\mathrm{aug}}.\left(x,z;u\left[t\right]\right)={.Math.z.Math.}_1+v^T\left(j-\mathrm{Dx}\right)+\frac{}{2}.Math.{.Math.x-z+u\left[t\right].Math.}_2^2& \left(7\right)\end{array}$

[0065] In the expression (7), u[t] represents an auxiliary term for conversion to the optimum solution when the constrained optimization problem is solved by a gradient method for performing iterative calculation from an appropriate initial point. The following expression (8) is obtained by differentiating the expression (7) with respect to x.

[Math. 8]

.sub.xL.sub.aug.Math.(x, z; u[t])=D.sup.Tv+(xz+u[t]) (8)

[0066] Assuming that x is a value with which the expression (8) is zero, the following expression (9) is obtained.

[00004]$\left[\mathrm{Math}..Math.9\right]$$\begin{array}{cc}x=\frac{1}{}.Math.D^T.Math.v+\left(z-u\left[t\right]\right)& \left(9\right)\end{array}$

[0067] By substituting x obtained in the expression (9) into the cost function of the original expression (7), the expression (7) becomes the following expression (10).

[00005]$\left[\mathrm{Math}..Math.10\right]$$\begin{array}{cc}{L}_{\mathrm{aug}}.\left(x,z;u\left[t\right]\right)={.Math.z.Math.}_1+v^T\left(j-D\left(z-u\left[t\right]\right)\right)-\frac{1}{2.Math.}.Math.{.Math.D^T.Math.v.Math.}_2^2& \left(10\right)\end{array}$

[0068] The expression (10) can be considered to be a quadratic function with respect to v. Accordingly, the optimum solution by which the expression (10) is maximized is expressed by the following expression (11).

[Math. 11]

v=(DD.sup.T).sup.1[jD(zu[t])](11)

[0069] By substituting the obtained expression (11) into the expression (9), the following expression (12) is obtained.

[Math. 12]

x=D.sup.T(DD.sup.T).sup.1j+(ID.sup.T(D.sup.TD).sup.1D)(zu[t]) (12)

[0070] A expression (13) is obtained by the gradient method using iterative calculation described above. When the gradient method is applied to the expression (12), the following three expressione (13) are obtained as expressione representing a t+1 value of the iterative calculation for x, z, and u.

[Math. 13]

x[t+1]=D.sup.T(D.sup.TD).sup.1j+(ID.sup.T(D.sup.TD).sup.1D)(z[t]u[t])

z[t+1]=W.sub.1/(x[t+1]+u[t])

u[t+1]=u[t]+(x[t+1]z[t+1]) (13)

where

[00006]$W_{1.Math.\text{/}.Math.}\left(x\right)=\left\{\begin{array}{c}x-\frac{1}{}.Math..Math.\left(x>\frac{1}{}\right)\\ 0.Math..Math.\left(-\frac{1}{}x\frac{1}{}\right)\\ x+\frac{1}{}.Math..Math.\left(x<-\frac{1}{}\right)\end{array}\right\}$

[0071] In the present example embodiment, it is assumed that the iterative calculation is performed only once. In this case, z[t+1] and u[t+1] do not need to be considered. While z[t] and u[t] need to be initialized, z[0] and u[0] may be zero. Accordingly, the following expression (14) is obtained as the image x to be reconstructed.

[Math. 14]

x[1]=D.sup.T(DD.sup.T).sup.1j (14)

[0072] Meanwhile, the ADMM algorithm exerts a great effect in the case where a base expected to be sparse is found through a certain transformation. A sparse property of a signal indicates a property in which a number of components of the signal are zero. Accordingly, it is generally necessary to convert x into a space having sparsity when reconstructing the image x. That is, the image is reconstructed in a space having sparsity.

[0073] Sparsification is achieved by, for example, performing a discrete Fourier transform or a wavelet transform on the image x. Accordingly, as expressed by the following expression (15), the image x is made sparse using a sparse transformation matrix . The sparse transformation matrix is, for example, either a discrete Fourier transformation matrix or a wavelet transformation matrix. Then, Q is obtained on the basis of the ADMM algorithm described above.

[Math. 15]

.Math.x.sub.mn1=Q.sub.mn1 (15)

[0074] The expression (15) is converted into the form expressed by the following expression (16) using an inverse matrix .sup.1 of the transformation matrix .

[Math. 16]

.sup.1.Math.Q.sub.mn1=x.sub.mn1 (16)

[0075] The expression (16) is further converted into the form expressed by the following expression (17) using the relationship expressed by the expression (3).

[Math. 17]

D.sub.kmn.Math..sup.1.Math.Q.sub.mn1=j.sub.k1 (17)

[0076] In a case where the sparse transformation matrix is the discrete Fourier transform or the wavelet transformation matrix, a conjugate transposed matrix of is expressed by the subscript +, and when expressed as .sup.+, and .sup.+ are inverse matrixes of each other.

[0077] Accordingly, the following expression (18) is obtained from the expression (17).

[Math. 18]

D.sub.mn1.Math..sup.+.Math.Q.sub.mn1=j.sub.k1 (18)

[0078] In the expression (18), an unknown is Q. Accordingly, an approximate solution Q of Q is obtained by obtaining the minimum solution of the L1 norm as expressed by the following expression (19).

[Math. 19]

Q.sub.mn1=argminQ.sub.mn1.sub.1 s.t.D.sub.kmn+.sup.+Q.sub.mn1=j.sub.k1 (19)

[0079] In a similar manner to the example of obtaining the expression (17) with respect to the expression (4) described above, the following expression (20) is obtained with respect to the expression (19).

[Math. 20]

Q.sub.mn1[1]=(D.sub.kmn.Math..sup.+).sup.+.Math.inv[(D.sub.kmn.Math..sup.+).Math.(D.sub.kmn.Math..sup.+).sup.+].Math.j.sub.k1=P.Math.j (20)

[0080] In the expression (20), P represents (D.sub.kmn.sup.+).sup.+inv[(D.sub.kmn.sup.+)(D.sub.kmn.sup.+).sup.+]. In addition, j on the right side of the expression (20) represents the content same as j.sub.k1 on the left side.

[0081] That is, Q, which is the approximate solution of Q, is obtained, and the obtained Q and .sup.+ described above are used to obtain the image x to be reconstructed.

[0082] The number of observations k performed by the camera 18 or the detector 120 may be generally smaller than mn, which is the number of pixels of the image x to be reconstructed. For example, in a case where the sparsification described above is properly performed, the number of observations k by the camera 18 or the detector 120 may be at about several percent of the number of pixels. That is, the imaging apparatus 100 uses the ADMM algorithm, which is a method of compressed sensing, to enable image reconstruction from a small amount of data obtained by changing the speckle pattern.

[0083] The number of observations k by the camera 18 or the detector 120 is not limited to the example described above, and may be appropriately determined depending on the degree of sparsification or the image quality required for the image to be reconstructed.

[0084] Next, each element of the imaging apparatus 100 according to the present example embodiment will be described in detail.

[0085] The light source 11 generates light to be applied to the object. In the present example embodiment, the light source 11 is a light source that primarily generates light of a desired single wavelength. It is sufficient if conditions such as intensity and a wavelength of light to be generated by the light source 11 are appropriately determined depending on a target to be a subject of an image and other factors. The light source 11 may be a white light source that generates light of various wavelengths, or may be capable of changing a wavelength of light to be generated, such as a wavelength-tunable laser. A specific type and the like of the light source 11 is not particularly limited, and it is sufficient if desired light can be generated.

[0086] The modulator 12 modulates the light generated by the light source 11. As described above, the light generated from the light source 11 is modulated by, for example, changing the wavefront, the wavelength, or the incident angle to the waveguide 110. Accordingly, the modulator 12 changes those. That is, the modulator 12 changes either the incident angle of the light generated from the light source to the waveguide, the wavefront of the light generated from the light source, or the wavelength of the light generated from the light source, for example.

[0087] In a case where the modulator 12 changes the incident angle of the light generated from the light source 11 to the waveguide 110, a mechanism for changing the traveling direction of the light, such as a galvanometer mirror, a piezoelectric element mirror, and a movable stage, is used as the modulator 12. In a case where the modulator 12 changes the wavefront of the light generated from the light source 11, a mechanism for changing the wavefront of the light, such as an optical space modulator, a digital mirror device, and a shape-variable mirror, is used as the modulator 12.

[0088] Furthermore, in a case where the modulator 12 changes the wavelength of the light generated from the light source 11, a white light source is used as the light source 11 when the modulator 12 is a mechanism for changing the wavelength of the light generated from the light source 11 for extracting light of a specific wavelength such as a diffraction grating and a prism. In this case, instead of the modulator 12, the light source 11 may have a mechanism capable of changing the wavelength of light to be generated, such as the wavelength-tunable laser mentioned above.

[0089] The modulation control unit 13 controls a state of modulation by the modulator 12. For example, in a case where the modulator 12 is a galvanometer mirror, the modulation control unit 13 changes the orientation of the mirror surface to perform control to change the incident angle of the incident light generated from the light source 11 and made incident on the waveguide 110. Even in a case where another mechanism is used as the modulator 12, it is sufficient if the modulation control unit 13 appropriately controls the modulator 12 in such a manner that the light generated from the light source 11 is modulated.

[0090] As described above, it is necessary to use the same speckle pattern in both the case of obtaining the transformation matrix D and the case of reconstructing the image. Accordingly, the same light source 11 and the modulator 12 are generally used in both the case of obtaining the transformation matrix D and the case of reconstructing the image. The modulation control unit 13 controls the modulator 12 in such a manner that the modulation by the modulator 12 is performed under the same condition in both the case of obtaining the transformation matrix D and the case of reconstructing the image.

[0091] The beam splitter 15 separates the light to be made incident on the waveguide 110 and travel toward the target from the light obtained from the object. In the example illustrated in FIG. 2, the beam splitter 15 is provided in such a manner that the light to be made incident on the waveguide 110 is allowed to pass and the light reflected by the target and traveling toward the detector is separated.

[0092] In the example illustrated in FIG. 2, the detector 120 is configured to detect the reflected light of the speckle pattern from the target. However, as described later, the detector 120 may detect the transmitted light of the speckle pattern transmitted through the target depending on an application purpose of the imaging apparatus 100. In this case, the detector 120 is disposed at a position opposite to the position at which the waveguide 110 is disposed with respect to the target 17. In other words, in this case, the target 17 is disposed between the exit end of the waveguide 110 and the detector 120. Accordingly, in this case, the beam splitter 15 is not required.

[0093] That is, it is sufficient if the beam splitter 15 is provided when necessary depending on conditions for reconstructing the image, such as a type of the target.

[0094] The waveguide 110 transmits light generated from the light source. In the present example embodiment, a multimode waveguide is used as the waveguide 110. With the multimode waveguide being used, it becomes possible to obtain different speckle patterns according to modulation of light incident on the waveguide 110.

[0095] For example, a multimode optical fiber, a rectangular or circular waveguide, or a photonic crystal waveguide is used as the waveguide 110. Other types of optical waveguides may be used as the waveguide 110 as long as different speckle patterns can be obtained depending on a degree of modulation of light incident on the waveguide 110.

[0096] Furthermore, when the waveguide 110 having a large number of modes is used, an information volume increases, and resolution can be improved. Accordingly, it is preferable to use the waveguide 110 having a large diameter within the realm of availability.

[0097] As described above, the detector 120 detects the power of the speckle pattern of the near-field light emitted from the waveguide 110 and applied to the target.

[0098] In the present example embodiment, the detector 120 detects the power of the speckle pattern. That is, the detector 120 is not necessarily detect positional information such as power distribution of the speckle pattern. The detector 120 may be any one-pixel sensor, and is not necessarily a detector that detects power positional information, such as sensors arranged in an array. The imaging apparatus 100 makes it possible to obtain a two-dimensional image without using, for example, sensors arranged in an array, which may be expensive depending on a wavelength band or the like to be detected.

[0099] A general detector capable of detecting the power of the speckle pattern is appropriately used as the detector 120 depending on the wavelength of light to be generated by the light source 11 and other conditions. A camera including a CCD camera or a CMOS image sensor, an ultraviolet camera, an InGaAs camera, a thermal image camera, or the like is appropriately used as the detector 120 depending on conditions such as a wavelength of light to be generated by the light source 11.

[0100] Gradation of the power magnitude of the signal that can be detected by the detector 120 is not particularly limited. In a case where sensitivity resolution of the detector 120 is high, that is, the power of the signal is detected by the detector 120 in a larger number of gradations, it becomes possible to reconstruct an image with less noise.

[0101] In the example illustrated in FIG. 2, as described above, the detector 120 detects the power of the speckle pattern applied to and reflected by the target 17. That is, a speckle emitted from the waveguide 110 is imaged on the target 17 spaced apart from the lens probe 16 by the lens probe 16 provided at the exit end of the waveguide 110. The detector 120 detects the power of the light reflected from the target 17 and transmitted through the lens probe 16, waveguide 110, and the beam splitter 15.

[0102] However, the detection of the power by the detector 120 may be carried out under conditions different from those in the example illustrated in FIG. 2. For example, the waveguide 110 or the lens probe 16 may be in intimate contact with the target 17. Furthermore, the detector 120 may detect the power of the speckle pattern transmitted through the target 17. That is, conditions for the detection by the detector 120, including a positional relationship with the waveguide 110 and the like, are not particularly limited as long as the power of the speckle pattern reflected from the target or the speckle pattern transmitted through the target can be detected.

[0103] Furthermore, as illustrated in FIGS. 2 and 3, the positional relationship between the detector 120 and other elements may be different from or the same as the positional relationship between the camera 18 and other elements in the case of obtaining the transformation matrix D.

[0104] The reconstruction unit 130 reconstructs an image using the transformation matrix D obtained in advance as described above on the basis of the power of a plurality of speckle patterns generated by the light modulated by the modulator 12 and detected by the detector 120.

[0105] More specifically, the reconstruction unit 130 obtains Q using the relationship of the expression (19) mentioned above on the basis of the power of the signal obtained by the observation of k times performed by the detector 120. As described above, Q is a value obtained by performing the discrete Fourier transform or the wavelet transform on the image x. After Q is obtained, the reconstruction unit 130 reconstructs the image using the inverse matrix .sup.1 of the transformation matrix . In the present example embodiment, it is assumed that the image to be reconstructed in this case is a monochromatic image.

[0106] The reconstruction unit 130 is implemented by, for example, appropriately combining hardware including a central processing unit (CPU) and a memory and software for reconstructing an image. A specific configuration of the reconstruction unit 130 is not particularly limited, and may be implemented by, for example, a field programmable gate array (FPGA) or dedicated hardware. The reconstruction unit 130 may have a function of obtaining the transformation matrix D through the procedure of the calibration process described above.

[0107] Next, exemplary operation of the imaging apparatus 100 will be described with reference to the flowchart illustrated in FIG. 6. In the following descriptions of the operation, it is assumed that the transformation matrix D is obtained in advance with respect to the speckle pattern of the waveguide 110 through the procedure of the calibration process described above.

[0108] First, the light generated by the light source 11 passes through the waveguide 110 and irradiates the target 17 (step S101).

[0109] The detector 120 detects the power of the speckle pattern generated by the light from the light source 11 irradiating the target 17 (step S102).

[0110] Next, the reconstruction unit 130 determines whether the power is observed in step S102 k times, which is the predetermined number of observations (step S103). If the number of observations does not reach the predetermined number (No in step S103), for example, the modulation control unit 13 changes the state of modulation performed by the modulator 12 in such a manner that the speckle pattern changes (step S104). In this case, the state of modulation performed by the modulator 12 is controlled in such a manner that the speckle pattern similar to the case of obtaining the transformation matrix D is generated. Returning to step S102, the detector 120 detects the power of the speckle pattern.

[0111] If the number of observations has reached the predetermined number (Yes in step S103), the reconstruction unit 130 reconstructs an image (step S105). That is, in step S102, the reconstruction unit 130 reconstructs the image using the transformation matrix D obtained in advance on the basis of the power of the speckle pattern observed by the detector 120 k times.

[0112] As described above, the imaging apparatus 100 according to the present example embodiment reconstructs the image using the ADMM algorithm that is one of techniques of the compressed sensing. Various speckle patterns generated by the waveguide 110 are used in the image reconstruction.

[0113] A type of the waveguide 110 is not particularly limited as long as it is a multimode waveguide. That is, it is possible to reconstruct an image using a common waveguide 110 having a small diameter such as a micrometer-order single waveguide 110. Therefore, the imaging apparatus 100 serves as an imaging apparatus usable in a narrow space.

[0114] Furthermore, according to the imaging apparatus 100, the resolution in reconstruction can be increased by obtaining the transformation matrix D relevant to a larger number of speckle patterns and carrying out a larger number of observations relevant to the speckle patterns. That is, by using the imaging apparatus 100, it becomes possible to obtain an image with a resolution relevant to an application purpose without depending on the diameter of the waveguide.

[0115] (Variations)

[0116] Variations of the imaging apparatus 100 described above are conceivable.

[0117] In the imaging apparatus 100, the light source 11 for generating light of a single wavelength is used. That is, according to the imaging apparatus 100, a monochromatic image is reconstructed for a specific wavelength. However, the imaging apparatus 100 may be what is called a multicolor apparatus, that is, an apparatus that reconstructs images for a plurality of wavelengths.

[0118] Each of FIGS. 7 to 10 illustrates an exemplary configuration in the case where the imaging apparatus 100 reconstructs images for a plurality of wavelengths.

[0119] In the example illustrated in FIG. 7, an imaging apparatus 101 includes a wavelength-tunable laser 21 and a wavelength control unit 22 instead of the light source 11 of the imaging apparatus 100. The wavelength-tunable laser 21 generates laser light having a wavelength under the control of the wavelength control unit 22. A wavelength range of the laser light that can be generated by the wavelength-tunable laser 21 is not particularly limited. It is sufficient if an appropriate wavelength-tunable laser 21 is used depending on, for example, conditions required for an image to be reconstructed.

[0120] In the imaging apparatus 101, the wavelength control unit 22 performs control in such a manner that the wavelength-tunable laser 21 generates laser light having a specific wavelength. An image for the specific wavelength is then reconstructed. Subsequently, the wavelength control unit 22 performs control in such a manner that the wavelength of the laser light generated by the wavelength-tunable laser 21 changes, and an image for a different wavelength is reconstructed.

[0121] A transformation matrix D to be used in reconstructing an image is different for each wavelength. Accordingly, the transformation matrix D is obtained in advance for each wavelength. When reconstructing an image, the transformation matrix D relevant to the wavelength is used. Images for multiple wavelengths are reconstructed by repeatedly reconstructing images for different wavelengths.

[0122] In the example illustrated in FIG. 8, an imaging apparatus 102 includes a white light source 31 instead of the light source 11 of the imaging apparatus 100. The imaging apparatus 102 further includes a wavelength separation unit 32.

[0123] The white light source 31 is a light source that generates light of various wavelengths. In the present variation, a spectral distribution of light emitted from the white light source 31 is not particularly limited, and the range of the spectral distribution and the intensity of the light of each wavelength may not be uniform. In the present variation, it is sufficient if a light source that emits light of a wavelength required to reconstruct an image is appropriately used as the white light source 31 depending on, for example, conditions required for the image to be reconstructed. That is, the white light source 31 may be any light source that generates light of a plurality of wavelengths required to reconstruct an image.

[0124] The wavelength separation unit 32 separates the light generated by the white light source 31 for each wavelength. That is, the wavelength separation unit 32 serves as a mechanism for extracting separation of a desired wavelength from the light generated by the white light source 31. While a diffraction grating, a prism, a filter, an electro-optic crystal, an acousto-optic crystal, or a magneto-optic crystal is used as the wavelength separation unit 32, for example, another mechanism for separating white light for each wavelength may be used. In addition, the wavelength separation unit 32 is also provided with a mechanism for extracting monochromatic light of a specific wavelength.

[0125] According to the imaging apparatus 102, the light of a specific wavelength separated by the wavelength separation unit 32 is made incident on the waveguide 110 to reconstruct an image. Then, the wavelength of the light to be made incident on the waveguide 110 is changed to reconstruct an image, thereby reconstructing images for multiple wavelengths in a similar manner to the imaging apparatus 101.

[0126] In the example illustrated in FIG. 9, an imaging apparatus 103 includes a white light source 41 instead of the light source 11 of the imaging apparatus 100. The imaging apparatus 103 further includes a filter 42.

[0127] The white light source 41 is a light source similar to the white light source 31 described above. The filter 42 is a filter that transmits light of a specific wavelength. That is, in a similar manner to the wavelength separation unit 32 described above, the filter 42 is a mechanism for extracting separation of a desired wavelength from the light generated by the white light source 41, and can be said that another implementation example of the wavelength separation unit 32.

[0128] For example, multiple filters having different wavelengths of light to be transmitted depending on the image to be reconstructed are used as the filter 42. The filter 42 may be a filter that varies the wavelength of light to be transmitted.

[0129] In the imaging apparatus 103, the filter 42 transmits light of a specific wavelength, thereby reconstructing an image for the specific wavelength. Then, the filter 42 that transmits light of a different wavelength is used or the filter 42 changes the wavelength of light to be transmitted, thereby reconstructing images for multiple wavelengths in a similar manner to the imaging apparatus 101 and the like.

[0130] In the example illustrated in FIG. 10, an imaging apparatus 104 includes a white light source 51 and a Fourier spectrometer 52 instead of the light source 11 of the imaging apparatus 100. The white light source 51 is a light source similar to the white light source 31 and the like described above. For example, a Michelson interferometer is used as the Fourier spectrometer 52.

[0131] In the imaging apparatus 104, light of different wavelengths is made incident on the waveguide 110 every prescribed time by the Fourier spectrometer 52. A detection result by the detector 120 is subject to a Fourier transform every prescribed time, thereby obtaining a detection result of light of a specific wavelength. It becomes possible to reconstruct images for multiple wavelengths by using a transformation matrix D or other parameters according to the wavelength.

[0132] In each of the imaging apparatuses 101 to 104, the detector 120 is a detector capable of detecting light having a wavelength for an image to be reconstructed. In addition, in each of the imaging apparatuses 101 to 104, the detector 120 is assumed to be a modulator that changes an incident angle of light generated from the light source 11 to the waveguide 110 or a modulator that changes a wavefront of light generated from the light source 11.

[0133] Each of the imaging apparatuses 101 to 104 enables reconstruction of images for multiple wavelengths. In other words, each of the imaging apparatuses 101 to 104 enables hyperspectral imaging.

[0134] (Simulation Example) [0135] It was confirmed by simulation that an image is reconstructed by the imaging apparatus 100 described above.

[0136] The image to be reconstructed was set to an image of 128 pixels in both width and length. A multimode fiber having a diameter of equal to or more than 125 m (micrometer) was used as the waveguide 110. The light source 11 was set to a light source that emits light of 632.8 nm (nanometer), and the modulator 12 that changes the incident angle to the waveguide 110 was used as the modulator 12.

[0137] Under those conditions, FIG. 11 illustrates an original image and an image reconstructed when 10% (percent) of the number of pixels of the original image is sampled. As illustrated in FIG. 11, comparing an original image 11A with a reconstructed image 11B, it was confirmed that the outline of the person, variations of light and shade of the original image 11A, and a part of the accessories of the hat were restored in the reconstructed image 11B.

[0138] As described above, in this example, a micrometer-order waveguide 110 is used. That is, it was confirmed that the imaging apparatus 100 is capable of performing imaging in a very narrow space.

[0139] While the invention has been particularly shown and described with reference to exemplary embodiments thereof, the invention is not limited to these embodiments. It will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present invention as defined by the claims. The configurations of the example embodiments may be combined with each other without departing from the scope of the present invention.

[0140] Although a part or all of the present invention is expressed as the following supplementary notes, it is not limited thereto.

[0141] (Supplementary Note 1)

[0142] An imaging apparatus including:

[0143] a waveguide that transmits light emitted from a light source;

[0144] a detector that detects power of a speckle pattern generated by the light passing through the waveguide and applied to an object; and

[0145] reconstruction means for reconstructing an image of the object on the basis of the power obtained by making the light enter the waveguide under different conditions.

[0146] (Supplementary Note 2)

[0147] The imaging apparatus according to Supplementary Note 1, further including:

[0148] a modulator that changes the conditions under which the light enters the waveguide.

[0149] (Supplementary Note 3)

[0150] The imaging apparatus according to Supplementary Note 2, wherein

[0151] the modulator changes the conditions by changing an incident angle of the light on the waveguide.

[0152] (Supplementary Note 4)

[0153] The imaging apparatus according to Supplementary Note 2, wherein

[0154] the modulator changes the conditions by changing a wavefront of the light.

[0155] (Supplementary Note 5)

[0156] The imaging apparatus according to Supplementary Note 2, wherein

[0157] the waveguide is a multimode waveguide.

[0158] (Supplementary Note 6)

[0159] The imaging apparatus according to any one of Supplementary Notes 1 to 5, wherein

[0160] the waveguide is a multimode waveguide.

[0161] (Supplementary Note 7)

[0162] The imaging apparatus according to any one of Supplementary Notes 1 to 6, wherein

[0163] the reconstruction means reconstructs the image on the basis of an alternating direction method of multipliers (ADMM) algorithm.

[0164] (Supplementary Note 8)

[0165] The imaging apparatus according to Supplementary Note 7, wherein

[0166] the reconstruction means reconstructs the image on the basis of the ADMM by using a transformation matrix obtained for the conditions and a speckle pattern generated under the conditions.

[0167] (Supplementary Note 9)

[0168] The imaging apparatus according to any one of Supplementary Notes 1 to 8, further including:

[0169] the light source that generates the light of a plurality of wavelengths; and

[0170] wavelength separation means for separating the light generated by the light source regarding the plurality of wavelengths, wherein

[0171] the reconstruction means reconstructs the image concerning the light for each of the plurality of wavelengths on the basis of the power of the light, the power being detected for each of the plurality of wavelengths.

[0172] (Supplementary Note 10)

[0173] The imaging apparatus according to any one of Supplementary Notes 1 to 8, further including:

[0174] the light source capable of generating light of a selected wavelength among a plurality of wavelengths; and

[0175] wavelength control means for controlling a wavelength of the light generated from the light source, wherein

[0176] the reconstruction means reconstructs the image concerning the light for each of the plurality of wavelengths on the basis of the power of the light, the power being detected for each of the plurality of wavelengths.

[0177] (Supplementary Note 11)

[0178] An imaging method including:

[0179] detecting power of a speckle pattern generated by light passing through a waveguide and applied to an object; and

[0180] reconstructing an image of the object on the basis of the power obtained by making the light enter the waveguide under different conditions.

[0181] This application is based upon and claims the benefit of priority from Japanese Patent Application No. 2018-052223, filed on Mar. 20, 2018, the disclosure of which is incorporated herein in its entirety by reference.

REFERENCE SIGNS LIST

[0182] 100, 101, 102, 103, 104 imaging apparatus [0183] 110 waveguide [0184] 120 detector [0185] 130 reconstruction unit [0186] 11 light source [0187] 12 modulator [0188] 13 modulation control unit [0189] 14 optical system [0190] 15 beam splitter [0191] 16 lens probe [0192] 17 target [0193] 18 camera