Provided are integrated circuits and methods for transposing a tensor using processing element array operations. In some cases, it may be necessary to transpose elements of a tensor to perform a matrix operation. The tensor may be decomposed into blocks of data elements having dimensions consistent with the dimensions of a systolic array. An identity multiplication may be performed on each block of data elements loaded into a systolic array and the multiplication products summed in column partitions of a results buffer. The data elements in the column partitions of results buffer can then be mapped to row partitions of a buffer memory for further processing.

Systems and methods that include: providing input information in an electronic format; converting at least a part of the electronic input information into an optical input vector; optically transforming the optical input vector into an optical output vector based on an optical matrix multiplication; converting the optical output vector into an electronic format; and electronically applying a non-linear transformation to the electronically converted optical output vector to provide output information in an electronic format. In some examples, a set of multiple input values are encoded on respective optical signals carried by optical waveguides. For each of at least two subsets of one or more optical signals, a corresponding set of one or more copying modules splits the subset of one or more optical signals into two or more copies of the optical signals. For each of at least two copies of a first subset of one or more optical signals, a corresponding multiplication module multiplies the one or more optical signals of the first subset by one or more matrix element values using optical amplitude modulation. For results of two or more of the multiplication modules, a summation module produces an electrical signal that represents a sum of the results of the two or more of the multiplication modules.

An optoelectronic computing system includes a first semiconductor die having a photonic integrated circuit (PIC) and a second semiconductor die having an electronic integrated circuit (EIC). The PIC includes optical waveguides, in which input values are encoded on respective optical signals carried by the optical waveguides. The PIC includes an optical copying distribution network having optical splitters. The PIC includes an array of optoelectronic circuitry sections, each receiving an optical wave from one of the output ports of the optical copying distribution network, and each optoelectronic circuitry section includes: at least one photodetector detecting at least one optical wave from the optoelectronic operation. The EIC includes electrical input ports receiving respective electrical values. The first semiconductor die and the second semiconductor die are electrically coupled in a controlled collapse chip connection, with the electrical output port of the PIC connected to one of the electrical input ports of the EIC.

Systems and methods that include: providing input information in an electronic format; converting at least a part of the electronic input information into an optical input vector; optically transforming the optical input vector into an optical output vector based on an optical matrix multiplication; converting the optical output vector into an electronic format; and electronically applying a non-linear transformation to the electronically converted optical output vector to provide output information in an electronic format. In some examples, a set of multiple input values are encoded on respective optical signals carried by optical waveguides. For each of at least two subsets of one or more optical signals, a corresponding set of one or more copying modules splits the subset of one or more optical signals into two or more copies of the optical signals. For each of at least two copies of a first subset of one or more optical signals, a corresponding multiplication module multiplies the one or more optical signals of the first subset by one or more matrix element values using optical amplitude modulation. For results of two or more of the multiplication modules, a summation module produces an electrical signal that represents a sum of the results of the two or more of the multiplication modules.

Systems and methods that include: providing input information in an electronic format; converting at least a part of the electronic input information into an optical input vector; optically transforming the optical input vector into an optical output vector based on an optical matrix multiplication; converting the optical output vector into an electronic format; and electronically applying a non-linear transformation to the electronically converted optical output vector to provide output information in an electronic format.

In some examples, a set of multiple input values are encoded on respective optical signals carried by optical waveguides. For each of at least two subsets of one or more optical signals, a corresponding set of one or more copying modules splits the subset of one or more optical signals into two or more copies of the optical signals. For each of at least two copies of a first subset of one or more optical signals, a corresponding multiplication module multiplies the one or more optical signals of the first subset by one or more matrix element values using optical amplitude modulation. For results of two or more of the multiplication modules, a summation module produces an electrical signal that represents a sum of the results of the two or more of the multiplication modules.

Systems and methods that include: providing input information in an electronic format; converting at least a part of the electronic input information into an optical input vector; optically transforming the optical input vector into an optical output vector based on an optical matrix multiplication; converting the optical output vector into an electronic format; and electronically applying a non-linear transformation to the electronically converted optical output vector to provide output information in an electronic format. In some examples, a set of multiple input values are encoded on respective optical signals carried by optical waveguides. For each of at least two subsets of one or more optical signals, a corresponding set of one or more copying modules splits the subset of one or more optical signals into two or more copies of the optical signals. For each of at least two copies of a first subset of one or more optical signals, a corresponding multiplication module multiplies the one or more optical signals of the first subset by one or more matrix element values using optical amplitude modulation. For results of two or more of the multiplication modules, a summation module produces an electrical signal that represents a sum of the results of the two or more of the multiplication modules.

Photonic processors are described. The photonic processors described herein are configured to perform matrix multiplications (e.g., matrix vector multiplications). Matrix multiplications are broken down in scalar multiplications and scalar additions. Some embodiments relate to devices for performing scalar additions in the optical domain. One optical adder, for example, includes an interferometer having a plurality of phase shifters and a coherent detector. Leveraging the high-speed characteristics of these optical adders, some processors are sufficiently fast to support clocks in the tens of gigahertz of frequency, which represent a significant improvement over conventional electronic processors.

Photonic processors are described. The photonic processors described herein are configured to perform matrix multiplications (e.g., matrix vector multiplications). Matrix multiplications are broken down in scalar multiplications and scalar additions. Some embodiments relate to devices for performing scalar additions in the optical domain. One optical adder, for example, includes an interferometer having a plurality of phase shifters and a coherent detector. Leveraging the high-speed characteristics of these optical adders, some processors are sufficiently fast to support clocks in the tens of gigahertz of frequency, which represent a significant improvement over conventional electronic processors.

Using a processor and a memory at a worker machine, a gradient vector is computed corresponding to a set of weights associated with a set of nodes of a neural network instance being trained in the worker machine. In an ISA vector corresponding to the gradient vector, an ISA instruction is constructed corresponding to a gradient in a set of gradients in the gradient vector, wherein a data transmission of the ISA instruction is smaller as compared to a data transmission of the gradient. The ISA vector is transmitted from the worker machine to a parameter server, the ISA vector being responsive to one iteration of a training of the neural network instance, the ISA vector being transmitted instead of the gradient vector to reduce an amount of data transmitted from the worker machine to the parameter server for the one iteration of the training.

An optoelectronic computing system includes a first semiconductor die having a photonic integrated circuit (PIC) and a second semiconductor die having an electronic integrated circuit (EIC). The PIC includes optical waveguides, in which input values are encoded on respective optical signals carried by the optical waveguides. The PIC includes an optical copying distribution network having optical splitters. The PIC includes an array of optoelectronic circuitry sections, each receiving an optical wave from one of the output ports of the optical copying distribution network, and each optoelectronic circuitry section includes: at least one photodetector detecting at least one optical wave from the optoelectronic operation. The EIC includes electrical input ports receiving respective electrical values. The first semiconductor die and the second semiconductor die are electrically coupled in a controlled collapse chip connection, with the electrical output port of the PIC connected to one of the electrical input ports of the EIC.