Optical tensor core, Method, and Applications

Abstract

An optical tensor core includes a device platform, a broadband, phase-locked, optical frequency comb generator, an array of electro-optic (EO) modulators, wherein a vector, {X.sub.j} and a matrix, {A.sub.ij} are encoded onto optical comb mode fields, and a non-linear waveguide, wherein a sum frequency generation (SFG) operation occurs directly on the optical comb mode fields so as to realize a multiply-accumulate (MAC) operation between the vector {X.sub.j} and the matrix {A.sub.ij}. An integrated system and related methodologies and applications are disclosed.

Claims

1. An optical tensor core, comprising: a device platform; a broadband, phase-locked, optical frequency comb generator; a plurality of electro-optic (EO) modulators adapted to EO modulate a respective plurality of comb mode fields of a broadband, phase-locked, optical comb produced by the comb generator, wherein a vector, {X.sub.j}, and a matrix, {A.sub.ij} are encoded onto the optical comb mode fields; and a non-linear waveguide operationally integrated with an output port of the E-O modulators, wherein a sum frequency generation (SFG) operation occurs directly on the optical comb mode fields so as to realize a multiply-accumulate (MAC) operation between the vector, {X.sub.j}, and the matrix, {A.sub.ij}, further wherein at least one of the frequency comb generator, the plurality of electro-optic (EO) modulators, and the non-linear waveguide are disposed on the non-linear photonics platform.

2. The optical tensor core of claim 1, wherein the broadband, phase-locked, optical frequency comb generator is a high-Q microresonator.

3. The optical tensor core of claim 1, further comprising a wavelength division demultiplexer (WDM DEMUX) and a wavelength division multiplexer (WDM MUX) operationally disposed intermediate the high-Q microresonator and the non-linear waveguide.

4. The optical tensor core of claim 3, wherein the frequency comb generator, the EO modulator array, the nonlinear optical waveguide, and the WDM MUX and WDM DEMUX are disposed on separate, operationally integrated platforms.

5. The optical tensor core of claim 1, wherein the device platform is one of a lithium niobate (LiNBO.sub.3), lithium tantalate (LiTaO.sub.3), potassium niobate (KNbO.sub.3), III-V semiconductors (AlN, GaN, GaP, GaAs, AlGaAs, InP), barium titanate (BaTiO.sub.3), silicon nitride, silica, tantalum pentoxide (Ta.sub.2O.sub.5), or a composite medium formed by integrating one of these materials with a dielectric material such as silicon nitride or silicon dioxide.

6. An integrated optical tensor core system, comprising: a photonic computing engine that includes: an optical tensor core, a pump laser suitably adapted to produce a broadband, phase-locked, optical frequency comb, and an optical receiver array suitably adapted to detect a computed optical output from the optical tensor core; and an electrical I/O circuit suitably adapted to provide programming voltages to the optical tensor core and to process the received signals from the optical receiver array, and to characterize the computing performance.

7. The integrated optical tensor core system of claim 6, wherein the optical tensor core further comprises: a device platform; a broadband, phase-locked, optical frequency comb generator; a plurality of electro-optic (EO) modulators adapted to EO modulate a respective plurality of comb mode fields of a broadband, phase-locked, optical comb produced by the comb generator, wherein a vector, {X.sub.j}, and a matrix, {A.sub.ij} are encoded onto the optical comb mode fields; and a non-linear waveguide operationally integrated with an output port of the E-O modulators, wherein a sum frequency generation (SFG) operation occurs directly on the optical comb mode fields so as to realize a multiply-accumulate (MAC) operation between the vector, {X.sub.j}, and the matrix, {A.sub.ij}.

8. A method for performing a multiply-accumulate (MAC) operation between a vector element, {X.sub.j}, and a matrix element, {A.sub.ij} using an optical tensor core, comprising: generating a broadband, phase-locked optical comb having a spectral bandwidth; encoding a computing vector, {X.sub.j}, and a matrix, {A.sub.ij} on different spectral bands of the optical comb mode field; and performing a sum frequency generation (SFG) operation directly on the vector- and matrix-encoded optical comb mode fields so as to realize a multiply-accumulate (MAC) operation between the vector, {X.sub.j}, and the matrix, {A.sub.ij}.

9. The MAC method of claim 8, wherein the step of generating the broadband, phase-locked optical comb mode field further comprises one of four-wave mixing, parametric down-conversion, and electro-optic modulation.

10. The MAC method of claim 8, wherein the step of encoding a computing vector with a length of N, {X.sub.j} (j=1, . . . , N), on the spectral bands of the optical comb mode field further comprises selecting a band of N comb modes at an equally spaced frequency set of {.sub.j.sup.s} (j=1, . . . , N) and electro-optically modulating the amplitude and phase of the selected band of comb modes to generate the computing vector, {X.sub.ij}.

11. The MAC method of claim 8, wherein the step of encoding a matrix {A.sub.ij} with a dimension of MN on the spectral bands of the optical comb mode field further comprises one of: i) (a) encoding a row of the matrix {A.sub.ij} (j=1, . . . , N) with a length of N onto a band of N comb modes at an equally spaced optical frequency set of {.sub.j.sup.l} (j=1, . . . , N), (b) encoding different rows of the matrix {A.sub.ij} onto different spectral bands of the comb; and ii) (a) selecting a band of N comb modes at an equally spaced frequency set of {.sub.j.sup.l} (j=1, . . . , N), (b) splitting the whole comb band equally into M parts; and (c) using each part to encode a row of {A.sub.ij}.

12. The MAC method of claim 8, wherein the step of encoding vector and matrix further comprises biasing the EO modulators with appropriate voltages so as to compensate for a spectral non-uniformity of the input comb.

13. The MAC method of claim 8, wherein the step of encoding vector/matrix and the step of MAC operation further comprise encoding the vector {X.sub.j} with appropriate order on a comb band at an equally spaced frequency set {.sub.j.sup.s}, encoding a row of the matrix {A.sub.ij} with appropriate order on another comb band at an equally spaced frequency set {.sub.j.sup.l}, and SFG between the two comb bands to produce an optical field at a single frequency .sup.c=.sub.j.sup.s+.sub.j.sup.l (j=1, . . . , N), with an amplitude of Y.sub.i=.sub.jA.sub.ijX.sub.j (where is a slope efficiency of the SFG process), wherein the MAC operation realizes a dot product between the two vectors.

14. The MAC method of claim 8, wherein, in conjunction with a matrix encoding method comprising (a) encoding a row of the matrix {A.sub.ij} (j=1, . . . , N) with a length of N onto a band of N comb modes at an equally spaced optical frequency set of {.sub.j.sup.l} (j=1, . . . , N), and (b) encoding different rows of the matrix {A.sub.ij} onto different spectral bands of the comb; and a dot product operation comprising encoding the vector {X.sub.j} with appropriate order on a comb band at an equally spaced frequency set {.sub.j.sup.s}, encoding a row of the matrix {A.sub.ij} with appropriate order on another comb band at an equally spaced frequency set {.sub.j.sup.l}, and SFG between the two comb bands to produce an optical field at a single frequency .sup.c=.sub.j.sup.s+.sub.j.sup.l (j=1, . . . , N), with an amplitude of Y.sub.i=.sub.jA.sub.ijX.sub.j (where is a slope efficiency of the SFG process), the step of MAC operation further comprises SFG between the comb band carrying the vector {X.sub.j} and different comb bands carrying different rows of the matrix {A.sub.ij} to produce a set of converted optical fields at equally spaced frequencies .sub.i.sup.c (i=1, . . . , M), each with an amplitude of Y.sub.i=.sub.jA.sub.ijX.sub.j (i=1, . . . , M), wherein the MAC operation realizes a matrix-vector multiplication.

15. The MAC method of claim 8, wherein, in conjunction with a matrix encoding method comprising (a) selecting a band of N comb modes at an equally spaced frequency set of {.sub.j.sup.l} (j=1, . . . , N), (b) splitting the whole comb band equally into M parts; and (c) using each part to encode a row of {A.sub.ij} and a dot product operation comprising encoding the vector {X.sub.j} with appropriate order on a comb band at an equally spaced frequency set {.sub.j.sup.s}, encoding a row of the matrix {A.sub.ij} with appropriate order on another comb band at an equally spaced frequency set {.sub.j.sup.l}, and SFG between the two comb bands to produce an optical field at a single frequency .sup.c=.sub.j.sup.s+.sub.j.sup.l (j=1, . . . , N), with an amplitude of Y.sub.i=.sub.jA.sub.ijX.sub.j (where is a slope efficiency of the SFG process), the step of MAC operation further comprises SFG in M separate nonlinear waveguides to realize dot product operations between the vector {X.sub.j} and individual rows of the matrix {A.sub.ij}, each in one nonlinear waveguide, to produce up-converted optical fields at a same frequency .sup.c=.sub.j.sup.s+.sub.j.sup.l but with different amplitudes of Y.sub.i=.sub.jA.sub.ijX.sub.j (i=1, . . . , M), output from the set of nonlinear waveguides, wherein the MAC operation realizes a matrix-vector multiplication.

16. The MAC method of claim 8, wherein the step of vector/matrix encoding and the step of MAC operation further comprises encoding the vector {X.sub.j} with appropriate order on a comb band at an equally spaced frequency set {.sub.j.sup.s} and encoding another vector {W.sub.j} with appropriate order on another comb band at an equally spaced frequency set {.sub.j.sup.l}, and performing SFG between the two comb bands to produce optical fields at equally spaced frequencies of .sup.c+m (m=N, . . . , +N) with amplitudes of amplitudes of Y.sub.m=.sub.jW.sub.jX.sub.(Nj+m), respectively, where is the mode spacing of the combs, wherein the MAC operation realizes a convolution between {X.sub.j} and {W.sub.j}.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0026] FIG. 1 is a schematic drawing conceptually illustrating related device and method elements, according to a non-limiting, exemplary embodiment.

[0027] FIG. 2 is a schematic drawing conceptually illustrating a layout of device elements and related functionalities, according to a non-limiting, exemplary embodiment.

[0028] FIG. 3 is a schematic drawing showing the layout of a hyperspectral optical tensor core device based upon a lithium niobate photonic integrated circuit platform that supports all required functionalities, according to a non-limiting, exemplary embodiment.

[0029] FIG. 4 is a schematic drawing showing a comb generation mechanism inside a high-Q microresonator via a) four-wave mixing, b) parametric down conversion, and c) electro-optic modulation, according to a non-limiting, exemplary embodiment.

[0030] FIG. 5 is a schematic drawing illustrating encoding a computing vector {X.sub.j} onto comb modes via electro-optic modulation (EOM), according to a non-limiting, exemplary embodiment.

[0031] FIG. 6(a) and (b) schematically illustrate two approaches for encoding a computing matrix {A.sub.ij} onto comb modes, according to a non-limiting, exemplary embodiment.

[0032] FIG. 7 is a schematic drawing illustrating sum frequency generation (SFG) performing an inner product operation, according to a non-limiting, exemplary embodiment.

[0033] FIG. 8(a) and (b), respectively, illustrate two approaches performing matrix-vector multiplication, corresponding to the matrix encoding schemes in FIGS. 6(a) and (b), according to a non-limiting, exemplary embodiment.

[0034] FIG. 9 schematically illustrates a SFG process performing a convolution operation, wherein the frequency ordering of {W.sub.j} is opposite to that of {A.sub.ij} in FIG. 7 for dot product, according to a non-limiting, exemplary embodiment.

[0035] FIG. 10(a) and (b), respectively, illustrate two approaches for realizing the optical tensor core, according to a non-limiting, exemplary embodiment.

[0036] FIG. 11 is a schematic drawing illustrating an integrated optical tensor core system, according to a non-limiting, exemplary embodiment.

DETAILED DESCRIPTION OF NON-LIMITING, EXEMPLARY EMBODIMENTS

[0037] Disclosed embodiments are described with reference to the attached figures, wherein like reference numerals are used throughout the figures to designate similar or equivalent elements. The figures are not drawn to scale and they are provided merely to illustrate aspects disclosed herein. Several disclosed aspects are described below with reference to example applications for illustration. It should be understood that numerous specific details, relationships and methods are set forth to provide a more complete understanding of the embodiments disclosed herein.

[0038] Described herein below is a hyperspectral optical tensor core for low-power, ultrafast, parallel computation of matrix operations, based upon a novel multiply-accumulate (MAC) operation principle, 100, the basic conceptual elements of which are illustrated in FIG. 1. Most generally, the optical tensor core 100 includes a broadband, phase-locked, optical frequency comb generator 102 to generate optical comb 103, an electro-optic (EO) modulator array 104 adapted to EO modulate a respective plurality of comb mode fields 105 of the optical comb, wherein a vector {X.sub.j} 107 and a computing matrix {A.sub.ij} 109 (FIG. 2) are encoded onto the optical comb mode fields, and a non-linear waveguide 106 operationally integrated with an output port 112 of the EO modulator array, wherein a sum frequency generation (SFG) operation 111 occurs directly on the optical comb mode fields so as to realize a multiply-accumulate (MAC) operation 113 between the vector, {X.sub.j}, and the matrix, {A.sub.ij}.

[0039] FIG. 2 is a schematic drawing further illustrating a layout 200 of device elements and related functionalities, following that of FIG. 1. A broadband phase-locked optical microcomb 103 is generated by a high-Q microresonator 202. The computing vector 107 and matrix 109 are both encoded on different spectral bands of the comb teeth via electro-optic modulators (EOMs) 104. MAC operation between the vector and the matrix is realized via sum-frequency generation (SFG) 113 inside a nonlinear waveguide 106. A wavelength-division demultiplexer (WDM DEMUX) 214 is employed to separate the microcomb into individual comb modes 105 for encoding the vector/matrix elements 107, 109. Similarly, a WDM MUX 216 is employed to combine the comb modes into the nonlinear waveguide 106 for SFG.

[0040] FIG. 3 schematically illustrates an optical tensor core device 300 fabricated on a thin-film lithium niobate (LN; LiNbO.sub.3) photonic integrated circuit (PIC) device platform 301, which will support all the underlying functionalities of the device. As will be appreciated, other nonlinear photonics platforms such as lithium tantalate (LiTaO.sub.3), potassium niobate (KNbO.sub.3), III-V semiconductors (AlN, GaN, GaP, GaAs, AlGaAs, InP), barium titanate (BaTiO.sub.3), silicon nitride, silica, tantalum pentoxide (Ta.sub.2O.sub.5), or a composite medium formed by integrating one of these materials with a dielectric material such as silicon nitride or silicon dioxide can be used as well, as long as they support the underlying functionalities. As will be further appreciated by a PHOSITA, all or some of the device components can be physically integrated on the same device platform, or on separate device platforms, or on separate device systems.

[0041] Advantageously, SFG offers nature-based, scalable, and ultrafast MAC operation, with unlimited size of vector/matrix and extremely low latency, and MAC operation directly on the optical field (rather than optical intensity), thus supporting MAC operation of complex variables. Moreover, LN EOM offers extremely high-speed encoding of both the vector and the matrix, with low power consumption while LN EOM and SFG offer high-precision programming of the vector/matrix and MAC operation. The foregoing combination of features enables a novel matrix-vector multiplication (MVM) modality that promises to transform the state of the art of in-memory computing with unprecedented speed, scalability, parallelism, accuracy, and power consumption.

[0042] In the description hereinbelow, a LN PIC platform will serve as an example platform. FIG. 4 is a schematic drawing showing well understood optical comb generation mechanisms inside a high-Q microresonator via a) four-wave mixing (FWM), b) parametric down conversion (PDC), and c) electro-optic modulation (EOM). With the strong nonlinear effect, a pump laser launched into the resonator produces a massive number of coherent comb modes at equally spaced frequencies whose phases are locked with each other. The phase locking among the comb lines leads to a common phase across the entire comb spectrum and a well-defined spectral profile that corresponds to chirp-free optical solitons in time domain. Consequently, the phase-locked microcombs exhibit exceptionally low phase noise and low timing jitter. Such an ultralow-noise phase-locked broadband microcomb is particularly advantageous for the embodied application in the instant hyperspectral tensor core. Moreover, in addition to the three mechanisms listed above, the phase-locked optical comb can also be produced by a mode-locked laser.

[0043] FIG. 5 is a schematic drawing illustrating encoding a computing vector {X.sub.j} onto comb modes via electro-optic modulation (EOM). To encode the computing vector {X.sub.j} (j=1, . . . , N) onto the comb, we select a band of N consecutive comb modes with optical frequency .sub.j.sup.s (j=1, . . . , N). The comb mode field at .sub.j.sup.s is electro-optically modulated in both amplitude and phase to produce the vector element X.sub.j. The same set of amplitude modulators can be used to compensate the deterministic spectral non-uniformity of the input microcomb whenever necessary, by biasing the EOMs with appropriate known voltages.

[0044] We describe two ways to encode the computing matrix {A.sub.ij} onto the comb as illustrated in FIG. 6. In the first approach (FIG. 6(a)), the whole matrix is encoded onto a band of MN consecutive comb modes at optical frequency of .sub.ij.sup.l (i=1, . . . , M; j=1, . . . , N). In the second approach (FIG. 6(b)), we select a band of N consecutive comb modes with optical frequency of (j=1, . . . , N) and split the whole comb band equally into M parts. Each part is used to encode a row of {A.sub.ij}. An LN EOM requires only capacitive driving and the energy needed to program an element A.sub.ij is CV.sub.ij.sup.2/2 where C is the EOM input capacitance and V.sub.ij is the driving voltage. The programming voltages {V.sub.ij} will be generated by a peripheral I/O electrical circuit that is used to support the optical tensor core chip.

[0045] As shown in FIG. 7, the SFG process combines two optical fields of X.sub.j and A.sub.ij at frequencies .sub.j.sup.s and .sub.j.sup.l, respectively, and produces an up-converted optical field with an amplitude of A.sub.ijX.sub.j at frequency .sup.c=.sub.j.sup.s+.sub.j.sup.l (where is the slope efficiency). Therefore, SFG performs the MAC operation by its very nature. In particular, SFG between two comb bands {X.sub.j} and {A.sub.ij} (j=1, . . . , N) at two equally-spaced frequency sets of {.sub.j.sup.s} and {.sub.j.sup.l} will produce an up-converted field with an amplitude of Y.sub.i=.sub.jA.sub.ijX.sub.j at a single frequency .sup.c=.sub.j.sup.s+.sub.j.sup.l (j=1, . . . , N), which naturally perform a dot product between two vectors. Note advantageously that SFG performs MAC operation on the optical field directly (rather than optical power), thus capable of MAC operation of complex variables.

[0046] The unique SFG-based MAC operation principle thus enables direct hyperspectral operation of the matrix-vector product as shown in FIG. 8. The matrix-vector product can be realized in two ways, depending on how the matrix is encoded on the comb. If the matrix is encoded on a whole set of comb modes (FIG. 6(a)), SFG between the vector comb and the matrix comb will produce a set of up-converted fields at equally spaced frequencies .sub.i.sup.c (i=1, . . . , M), each with an amplitude of Y.sub.i=.sub.jA.sub.ijX.sub.j. This approach is shown in FIG. 8(a). However, if the matrix rows are encoded on a same frequency set of comb modes according to FIG. 6(b), the set of up-converted fields will be produced at the same frequency .sup.c=.sub.j.sup.s+.sub.j.sup.l, while each with an amplitude of Y.sub.i=.sub.jA.sub.ijX.sub.j. This second approach is shown in FIG. 8(b). In this case, SFG for different matrix rows will be realized in different nonlinear waveguides. The approach in FIG. 6(b) has an advantage that it utilizes only a limited number of comb modes for matrix programming, while the approach in FIG. 6(a) has an advantage that SFG can be realized in a single nonlinear waveguide.

[0047] In addition to these two hyperspectral matrix-vector product operations, the SFG-based MAC operation also uniquely supports a third hyperspectral operationconvolution between two vectors. FIG. 9 illustrates the operation principle. For two vectors {X.sub.j} and {W.sub.j} (j=1, . . . , N) encoded on two equally-spaced frequency sets of comb modes, SFG between them can not only produce an up-converted field at frequency .sup.c with an amplitude of Y.sub.0=.sub.jW.sub.jX.sub.(Nj) (similar to FIG. 7), but also produce up-converted fields at equally spaced frequencies of .sup.c+m (m=N, . . . , +N) with amplitudes of Y.sub.m=.sub.jW.sub.jX.sub.(Nj+m), respectively, where is the mode spacing of the combs (see FIG. 9). Clearly, {Y.sub.m} is the convolution between {X.sub.j} and {W.sub.j}. Such a nature-based convolution operation will have vastly broad applications. Note that the whole convolution process is realized with a single SFG nonlinear waveguide as shown in FIG. 9. In practice, the up-converted comb modes at .sup.c+m carrying the vector {Y.sub.m}, output from the nonlinear waveguide, can be separated into individual comb modes with a wavelength-division demultiplexer (WDM DEMUX) for optical detection of individual vector elements. Note also that the hyperspectral convolution operation shown in FIG. 9 is distinctive to the hyperspectral matrix-vector product shown in FIG. 8(a) since the optical fields after the convolution operation appear at a frequency set of .sup.c+m that are near .sup.c (FIG. 9), while the optical fields after matrix-vector product operation appear at frequencies far separated from .sup.c with frequency difference equal to those between the matrix-encoded comb bands (FIG. 8(a)).

[0048] In general, SFG is a very powerful MAC operation approach. Together with difference frequency generation (DFG), more matrix operation can be realized, such as cross product and outer product. Furthermore, in addition to the SFG based upon the quadratic optical nonlinearity (.sup.(2) nonlinearity) that is utilized in FIGS. 7-9, SFG and FWM based upon the third-order optical nonlinearity (.sup.(3) nonlinearity) can also be used for MAC operations and more complex MAC operations.

[0049] The hyperspectral optical tensor core device can also be realized with parallel multimode interferometer (MMI) networks 1001, 1002 as shown in FIGS. 10(a) and 10(b). The parallel approach splits the MN matrix operation into the Minner products of 1N and N1 vectors, i.e., the matrix row vector and input column vector. As shown at 1001 in FIG. 10(a), the input CW laser is equally split into N parts with a 1N splitter MMI, each of which is electro-optically modulated in amplitude and phase to encode the input vector element X.sub.j(t). Each channel is then again split into M parts with a 1N splitter MMI, each of which is electro-optically modulated to encode the matrix element T.sub.ij to obtain T.sub.ijX.sub.j(t). After that, all of T.sub.ijX.sub.j(t) (j=1, 2, . . . , N) is grouped together and launched into a N1 combiner MMI for the summing function to obtain the output vector element Y.sub.i(t)=.sub.jT.sub.ijX.sub.j(t). The MAC operation can be realized with a similar configuration 1002 shown in FIG. 10(b), where the input CW laser is split into MN parts each of which is electro-optically modulated sequentially to encode X.sub.j(t) and T.sub.ij. The advantage of FIG. 10(a) is that it uses just the right number of EOMs but it requires appropriate design of the waveguide crossing to eliminate potential crosstalk. This crossing challenge is eliminated in FIG. 10(b), while it will require extra EOMs for encoding the input vector element X.sub.j(t). For hyperspectral operation, the input CW laser is replaced with a microcomb.

[0050] FIG. 11 schematically illustrates an integrated optical tensor core system 1100. The system includes two major parts: 1) the photonic computing engine 1101 that includes the optical tensor core, a pump laser to produce the microcomb, and an optical receiver array to detect the computed optical waves output from the optical tensor core; 2) the electrical I/O circuits 1102 to provide programming voltages to the optical tensor core and to process the received signals from the optical receiver array, and to characterize the computing performance.

[0051] While various disclosed embodiments have been described above, it should be understood that they have been presented by way of example only and not as a limitation. Numerous changes to the disclosed embodiments can be made in accordance with the specification herein without departing from the spirit or scope of this specification. Thus the breadth and scope of this specification should not be limited by any of the above-described embodiments; rather, the scope of this specification should be defined in accordance with the appended claims and their equivalents.