SYSTEMS AND METHODS FOR SWITCHING
20260129321 ยท 2026-05-07
Assignee
Inventors
- David Lazovsky (Los Gatos, CA, US)
- Philip Winterbottom (San Jose, CA, US)
- Francisco Jose MAIA DA SILVA (Porto, PT)
- Martinus Bos (San Jose, CA, US)
Cpc classification
International classification
Abstract
Systems and methods for switching are disclosed. In an example, a switching system includes a photonic integrated circuit (PIC) having optical ports, an electric integrated circuit (EIC) stacked on the PIC and having switching circuitry, and photonic transceivers optically coupled to the optical ports of the PIC and electrically coupled to the switching circuitry of the EIC, and the switching circuitry is configured to transfer digital signals, which are generated from optical signals received at a first photonic transceiver of the photonic transceivers, to a second photonic transceiver of the photonic transceivers based on information in the digital signals, wherein the first photonic transceiver is optically coupled to a first optical port of the PIC and the second photonic transceiver is optically coupled to a second optical port of the PIC.
Claims
1. An integrated circuit device comprising: switching circuitry; and a plurality of first portions of photonic transceivers, each first portion of a photonic transceiver including a driver and a driver interface that is exposed at a major surface of the integrated circuit device and an amplifier and an amplifier interface that is exposed at the major surface of the integrated circuit device; wherein the switching circuitry is configured to transfer digital signals, which are generated from electrical signals received from the first portion of a first photonic transceiver, to the first portion of a second photonic transceiver based on information in the digital signals.
2. The integrated circuit device of claim 1, wherein the major surface of the integrated circuit device is bottom major surface of the integrated circuit device.
3. The integrated circuit device of claim 1, wherein: the major surface of the integrated circuit device is a bottom major surface of the integrated circuit device; the driver interface of each photonic transceiver is located on the bottom major surface of the integrated circuit device to align with a modulator interface on a photonic integrated circuit (PIC); and the amplifier interface of each photonic transceiver is located on the bottom major surface of the integrated circuit device to align with a photodetector interface on the PIC.
4. The integrated circuit device of claim 1, wherein: the driver interface of each photonic transceiver is located on the integrated circuit device to align with a modulator interface on a photonic integrated circuit (PIC); and the amplifier interface of each photonic transceiver is located on the integrated circuit device to align with a photodetector interface on the PIC.
5. The integrated circuit device of claim 1, wherein the switching circuitry includes a switch fabric.
6. The integrated circuit device of claim 1, wherein the switch fabric is a crosspoint matrix.
7. The integrated circuit device of claim 1, wherein the switching circuitry is configured to identify a photonic transceiver of the integrated circuit device to transfer digital signals to based on information in a media access control (MAC) table that is stored in the integrated circuit device.
8. The integrated circuit device of claim 1, wherein a first set of the plurality of first portions of photonic transceivers surrounds a second set of the plurality of first portions of photonic transceivers.
9. The switching system of claim 1, wherein the integrated circuit device has a footprint, and wherein the first portions of the photonic transceivers are located in an interior region of the footprint of the integrated circuit device.
10. The integrated circuit device of claim 1, wherein a first set of the plurality of first portions of photonic transceivers corresponds to spine photonic transceivers and a second set of the plurality of first portions of photonic transceivers corresponds to leaf photonic transceivers.
11. The integrated circuit device of claim 1, wherein a first set of the plurality of first portions of photonic transceivers corresponds to spine photonic transceivers and a second set of the plurality of first portions of photonic transceivers corresponds to leaf photonic transceivers, wherein the switching circuitry surrounds the spine photonic transceivers and the leaf photonic transceivers surround the switching circuitry.
12. A method comprising: amplifying analog electrical signals that are received at an amplifier interface of a first portion of a first photonic transceiver of an integrated circuit device, wherein the amplifier interface is exposed at a major surface of the integrated circuit device and electrically coupled to an amplifier of the first photonic transceiver; converting the amplified analog electrical signals to digital signals within the integrated circuit device; transferring the digital signals to a second photonic transceiver of the integrated circuit device via switching circuitry of the integrated circuit device based on information in the digital signals; and driving a driver of the second photonic transceiver in response to the digital signals to generate analog electrical signals at a driver interface of a first portion of the second photonic transceiver, wherein the driver interface is exposed at the major surface of the integrated circuit device.
13. The method of claim 12, wherein transferring the digital signals to the second photonic transceiver of the integrated circuit device via the switching circuitry based on information in the digital signals involves identifying an output optical port based on a media access control (MAC) table in the switching circuitry.
14. The method of claim 12, wherein the EIC includes spine photonic transceivers and leaf photonic transceivers, and wherein the first photonic transceiver is a leaf photonic transceiver and the second photonic transceiver is a spine photonic transceiver.
15. The method of claim 12, wherein the EIC includes spine photonic transceivers and leaf photonic transceivers, and wherein the first photonic transceiver is a spine photonic transceiver and the second photonic transceiver is a leaf photonic transceiver.
16. The method of claim 12, wherein the EIC includes interior photonic transceivers surrounded by exterior photonic transceivers, and wherein the first photonic transceiver is an exterior photonic transceiver and the second photonic transceiver is an interior photonic transceiver.
17. The method of claim 12, wherein the EIC includes interior photonic transceivers surrounded by exterior photonic transceivers, and wherein the first photonic transceiver is an interior photonic transceiver and the second photonic transceiver is an exterior photonic transceiver.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0003]
[0004]
[0005]
[0006]
[0007]
[0008]
[0009]
[0010]
[0011]
[0012]
[0013]
[0014]
[0015]
[0016]
[0017]
[0018]
[0019]
[0020]
[0021]
[0022]
[0023]
[0024]
[0025] Throughout the description, similar reference numbers may be used to identify similar elements.
DETAILED DESCRIPTION
[0026] A multiport network switch is used to direct network traffic from an input port of the multiport network switch to an output port of the multiport network switch based on some information in the network traffic, e.g., some header information in data packets of the network traffic. With higher levels of integration, much of the functionality of a multiport network switch can be integrated into a single chip, e.g., a single IC device. While various examples of single-chip multiport network switches exist, the data interfaces to and from the IC devices, often referred to as media independent interfaces (MIIs), are physically located at the edge, or beachfront, of the IC devices and physical layer PHY devices (e.g., optical fiber PHY devices and/or copper wire PHY devices) are typically located off-chip near a perimeter edge of the IC device and coplanar with the IC device. For example, a single-chip switch is typically mounted on a printed circuit board (PCB) and the PHY devices are connected to the PCB at a perimeter edge, or edges, of the single-chip switch. In particular, for a switch that has multiple optical ports, the transformation of signals from the optical domain to the electrical domain and the transformation of signals from the electrical domain to the optical domain happens exclusively at the beachfront, or perimeter edges, of the IC device. As the desired number of ports on a multiport network switch increases and/or the size of an IC device decreases, the amount of physical real estate at the edges of an IC device can become a limiting factor with regard to the number of ports that can be supported by a single-chip multiport network switch. However, it has been realized that photonic transceivers formed by an electronic integrated circuit (EIC) stacked on a photonic integrated circuit (PIC) can be coupled with switching circuitry in the EIC to produce a multiport network switch that can switch data between optical ports of the multiport network switch without being limited to the beachfront of the EIC. In an example, at least some of the photonic transceivers are physically located in an interior region of the EIC as opposed to conventional single-chip switches in which the data interfaces (e.g., MIIs) to the IC device are exclusively located at perimeter edges of the IC device. Such a multiport network switch enables high speed network switching with a port density and power efficiency that heretofore has not been achieved.
[0027] In an example, the multiport network switch can be deployed in a data center that is implementing, for example, machine learning (ML) models and/or artificial neural networks (ANNs). In one specific example, the multiport network switch can be deployed as a Top-of-Rack (ToR) switch in a data center that utilizes a leaf-spine network architecture to connect nodes, e.g., compute nodes and/or memory nodes, which are running ANN workloads, across multiple racks of such nodes.
[0028]
[0029] Additionally, the PIC may be attached to a planar substrate that includes electrical connections to the PIC and to the EIC. In an example, the EIC and PIC are physically and electrically connected to each other by electrical interconnects, e.g., solder bumps, and the distance between the bottom major surface of the EIC and the top major surface of the PIC is less than 2 mm and in many cases less than 50 microns.
[0030]
[0031] In an example, an FAU is a device used in optical communication systems that combines or separates optical signals from multiple fibers into a single optical signal or multiple optical signals, respectively. The FAU can be used for a variety of applications, such as wavelength division multiplexing (WDM), parallel optical interconnects, and optical sensing. There are two main types of fiber array units that can be used: linear and circular. Linear FAUs combine or separate optical signals along a straight line, while circular fiber array units combine or separate optical signals in a circular configuration. Both types of FAUs are typically made from a precision-molded optical plastic or ceramic material and can have anywhere from a few to hundreds of fibers arranged in a specific pattern. The choice of FAU depends on the specific requirements of an application, such as the number of fibers, the arrangement of the fibers, the wavelength of light being used, and the coupling efficiency desired.
[0032] In an example, the EIC 102 includes high-speed integrated circuits configured to support the photonic transceivers and the data switching. The EIC may be an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or a System on Chip (SoC) that is designed and fabricated using state of the art CMOS nodes. The EIC may include circuits for serialization/deserialization, clock and data recovery, modulator drivers, and amplifiers that implement the photonic transceivers and circuits that implement the switching circuitry.
[0033] A waveguide may be a structure that guides and/or confines light waves to facilitate the propagation of the light along a desired path and to a desired location. For example, a waveguide may be an optical fiber, a planar waveguide, a glass-etched waveguide, a photonic crystal waveguide, a free-space waveguide, any other suitable structure for directing optical signals, and combinations thereof. In some embodiments, one or more internal waveguides are formed in the PIC 104. In some embodiments, one or more external waveguides are implemented external to the PIC, such as the optical fibers 114 or a ribbon comprising multiple optical fibers.
[0034] The PIC 104 may include one or more internal waveguides that are optically coupled to the optical ports 116 of the multiport network switch. For example, as will be discussed below in more detail, one or more of the optical ports may be optically coupled to another optical port on another computing device. In some examples, an internal waveguide of the PIC is implemented (e.g., formed) in the PIC to connect photonic elements internally within the PIC. In another example, one or more optical ports of the PIC may be optically coupled to an optical port of another computing device located in a separate circuit package or separate chip to form inter-chip connections. In some embodiments, an external waveguide is implemented in connection with the PIC in order to connect photonic ports across multiple chips. For example, the optical ports of the PIC may be connected via optical fibers across multiple chips. In some embodiments, an external waveguide (e.g., optical fiber) connects directly to photonic ports of respective computing devices across multiple chips. In some embodiments, an external waveguide is implemented in connection with one or more internal waveguides formed in the PIC of one or more of the chips. For example, one or more internal waveguides may internally connect one or more of the photonic ports to one or more additional optical components located at another portion of the circuit package (e.g., another portion of the PIC) to facilitate coupling with the external waveguides. For example, the internal waveguides within the PIC may connect to one or more optical coupling structures including FAUs located over grating couplers (GCs), or edge couplers. In some embodiments, one or more FAUs are implemented to facilitate coupling the external waveguides to the internal waveguides to facilitate chip-to-chip interconnection to another circuit package to both transmit and receive optical signals. In some embodiments, one or more FAUs are implemented to supply optical power from an external laser light source to the PIC to drive the photonics (e.g., provide one or more optical carrier signals) in the PIC.
[0035] In an example, the EIC 102 and the PIC 104 may be manufactured using standard wafer fabrication processes, including, e.g., photolithographic patterning, etching, ion implantation, etc. Further, in some embodiments, heterogeneous material platforms and integration processes are used. For example, various active photonic components, such as laser light sources and optical modulators and/or photodetectors used in the photonic transceivers, may be implemented using group III-V semiconductor components.
[0036] As will be appreciated by those of ordinary skill in the art, the depicted structure of the circuit package 100 is merely one of several possible ways to assemble and package the various components. In some examples, some or all of the EIC 102 is disposed on the substrate 110. In some examples, it is also possible to create the EIC and the PIC 104 in different layers of a single semiconductor chip. In some examples, the photonic circuit layer includes or is made of multiple PICs. Multiple layers of PICs, or a multi-layer PIC may help to reduce waveguide crossings. Moreover, the structure depicted in
[0037] In an example, a light source, or light sources, is/are optically coupled to the circuit package 100, e.g., a multiport network switch. The light source or light sources may include laser light sources that are implemented either in the circuit package or externally. When implemented externally, a connection to the circuit package may be made optically using a grating coupler in the PIC 104 underneath an FAU 112 and/or using an edge coupler. In some embodiments, lasers are implemented in the circuit package by using an interposer containing several lasers that can be co-packaged and edge-coupled with the PIC. In some embodiments, the lasers are integrated directly into the PIC using heterogenous or homogenous integration. Homogenous integration allows lasers to be directly implemented in the silicon substrate in which the waveguides of the PIC are formed, and allows for lasers of different materials, such as indium phosphide (InP), and architectures such as, quantum dot lasers. Heterogenous assembly of lasers on the PIC allows for group III-V semiconductors or other materials to be precision-attached onto the PIC and optically coupled to a waveguide implemented on the PIC.
[0038] In an example, data is communicated between the EIC 102 and the PIC 104 using photonic transceivers in which each photonic transceiver includes a first portion in the EIC and a second portion in the PIC.
[0039]
[0040] In an example, the driver 230 and the amplifier 232 of the first portion 226 of the photonic transceiver 220 include electronic circuits that are fabricated in the EIC 202. In an example, the first portion of the photonic transceiver is an analog/mixed signal (AMS) block that includes circuits for processing analog signals or circuits for processing analog signals and circuits for processing digital signals. The driver of the first portion of the photonic transceiver may include digital control and analog amplifier circuits. In an example, the driver includes a driver interface (not shown) that is exposed at the bottom major surface 240 of the EIC. The amplifier of the first portion of the photonic transceiver may include a transimpedance amplifier (TIA). In an example, the amplifier includes an amplifier interface (not shown) that is also exposed at the bottom major surface of the EIC. In an example, the driver interface and the amplifier interface include one or more conductive contacts or pads that are electrically coupled to electronic circuits of the respective components and that are exposed at the bottom major surface of the EIC.
[0041] The modulator 234 of the second portion 228 of the photonic transceiver 220 may include an Electro-Absorption Modulator (EAM) that is fabricated into the PIC 204, for example, the EAM may be a Germanium-Silicon (GeSi) EAM. Other examples of optical modulators include, but are not limited to, micro-ring resonators (MRRs), or any suitable optical component with sufficient thermal stability over the operating ranges of the photonic transceivers. In an example, the modulator includes a modulator interface (not shown) that is exposed at the top major surface 242 of the PIC. For example, the modulator interface may include one or more conductive contacts or pads that are electrically coupled to the modulator and that are exposed at the top major surface of the PIC.
[0042] The photodetector 236 of the second portion 228 of the photonic transceiver 220 includes electronic circuits that are fabricated into the PIC 204, for example, the photodetector may be a GeSi photodetector. In an example, the photodetector includes a photodiode and a photodetector interface (not shown) that are fabricated into the PIC. For example, the photodetector interface may include one or more conductive contacts that are exposed at the top major surface of the PIC.
[0043]
[0044] As stated above, it has been realized that photonic transceivers formed by an EIC stacked on a PIC as described with reference to
[0045] In an example, a first portion of each photonic transceiver is in the EIC and a second portion of each photonic transceiver is in the PIC and interfaces of the first portion of each photonic transceiver in the EIC are vertically aligned with interfaces of the second portion of the corresponding photonic transceiver in the PIC as described with reference to
[0046]
[0047] As illustrated in the top view of
[0048] As illustrated in the top view of
[0049]
[0050]
[0051] With reference to the left side of
[0052] The PIC 304 may include grating couplers (GCs) 360, multiplexers, demultiplexers, and waveguides that connect the optical ports to the modulators 334 and to the photodetectors 336. In the example of
[0053]
[0054] The photonic transceivers may facilitate converting a message or a signal between the electronic domain and the photonic domain. For example, the photonic transceivers may each include an EO interface for converting electronic signals to optical (e.g., photonic) signals, and may include an OE interface for converting optical signals to electronic signals. While
[0055] Examples of switching data from node A to node D and from node D to node A are now described with reference back to
[0056] First, an example of switching data from node A to node D via the multiport network switch 300 is described. In the example, some digital data at node A is intended to be transmitted to node D but to get to node D, the digital data travels through the multiport network switch. In an operation, the digital data is transmitted via optical signals from node A to optical port, P4, of the multiport network switch in the optical path 351, e.g., via an optical fiber. The optical signals enter into the multiport network switch via the FAU 312 and GC 360 corresponding to optical port, P4, and travel on optical path 351 in an optical waveguide within the PIC to the photodetector 336 of the photonic transceiver corresponding to optical port, P4. The photodetector converts the optical signals to electrical signals and the electrical signals are conducted via the electrical interconnect 338 to the amplifier 332 of the photonic transceiver. The amplifier of the photonic transceiver amplifies the electrical signals and provides the amplified electrical signals via electrical path 353 to the interface 358 that corresponds to the photonic transceiver. The interface converts the electrical signals to digital signals, e.g., to binary digital data, and the switching circuitry 350 determines, based on information in the digital signals, where to transfer the digital signals. For example, the switching circuitry determines from a 12-bit value in a header portion of the digital signals where to transfer the digital signals. In an example, the digital signals are transmitted as digital data in protocol data units (PDUs), also referred to as packets, with each PDU including a header. The header includes information that is used to determine where to transfer the digital data within the multiport network switch.
[0057] In the example of
[0058] A similar operation is implemented for digital data that is transmitted from node D to node A via the multiport network switch 300. In the example, some digital data at node D is intended to be transmitted to node A but to get to node A, the data travels through the multiport network switch. In an operation, the digital data is transmitted via optical signals from node D to optical port, P2, of the multiport network switch in the optical path 357, e.g., via an optical fiber. The optical signals enter into the multiport network switch via the FAU 312 and GC 360 corresponding to optical port, P2, and travel on optical path 357 in an optical waveguide within the PIC 304 to the photodetector 336 of the photonic transceiver corresponding to port, P4. The photodetector converts the optical signals to electrical signals and the electrical signals are conducted via the electrical interconnect 338 to the amplifier 332 of the photonic transceiver. The amplifier of the photonic transceiver amplifies the electrical signals and provides the amplified electrical signals via electrical path 359 to the interface 358 that corresponds to the photonic transceiver. The interface converts the electrical signals to digital signals, e.g., to binary digital data, and the switching circuitry 350 determines, based on information in the digital signals, where to transfer the digital signals. For example, the switching circuitry determines from a 12-bit value in a header portion of PDUs where to transfer the digital signals. In the example of
[0059] In the example described with reference to
[0060] A common architecture in data centers involves a leaf-spine configuration of end nodes and switches. Leaf-spine network architectures are efficient at providing resilient connectivity between clusters of nodes, which may include, for example, compute resources and/or memory resources. In an example, a multiport network switch is configured for use in a leaf-spine network architecture that connects multiple racks of nodes, e.g., compute nodes and/or memory nodes, to each other. In one particular example, the multiport network switch is configured with twenty optical ports for communicating data between nodes in the leaf-spine network architecture, sixteen of which are configured for use as leaf ports and four of which are configured for use as spine ports. An example of a multiport network switch that is configured for use in a leaf-spine network architecture is described with reference to
[0061]
[0062] In the example of
[0063] As illustrated in the top view of
[0064]
[0065]
[0066] With reference to
[0067] The PIC 504 may also include grating couplers (GCs), splitters, multiplexers, demultiplexers, and waveguides that are optically coupled between the optical ports and the modulators and the photodetectors. In the example of
[0068] In an example, the multiport network switch 500 described with reference to
[0069] In the example of
[0070] In an example and with reference back to
[0091] In a leaf-spine network architecture 676 as shown in
[0092] In the example described with reference to
[0093] Thus, both the leaf photonic transceivers and the spine photonic transceivers are located in an interior region of the EIC and are close to the switching circuitry. Because the multiport network switch employs photonic transceivers that are formed by an EIC stacked on a PIC, the transformation of signals from the optical domain to the electrical domain and the transformation of signals from the electrical domain to the optical domain does not have to happen at the beachfront, or edges, of the EIC, but can instead happen wherever the drivers and amplifiers can be fabricated on the EIC. Thus, the number of ports that can be included on such a multiport network switch is not limited to the beachfront, or edge dimensions, of the EIC and more photonic transceivers may be included in the EIC than if the location of the photonic transceivers was limited exclusively to the beachfront (e.g., the edges) of the EIC. Additionally, because the photonic transceivers are not limited exclusively to the beachfront of the EIC, the photonic transceivers can be strategically located near the switching circuitry in a manner that reduces the physical distance that electrical signals must travel through the switching circuitry to be transferred from one photonic transceiver to another photonic transceiver to implement multiport network switching. For example, having the spine photonic transceivers close to the leaf photonic transceivers on the EIC can reduce the amount of physical distance that electrical signals have to propagate across the switching circuitry of the EIC to get from a photonic transceiver corresponding to an input optical port of the multiport network switch to a photonic transceiver corresponding to an output optical port of the multiport network switch, which can reduce power consumption and reduce latency across the multiport network switch.
[0094] Although an example of a multiport network switch with twenty optical ports (16 leaf ports and 4 spine ports) is described, other numbers of optical ports are possible. In an example, a multiport network switch may have an EIC in which some photonic transceivers are located at an interior region of the EIC and some other photonic transceivers are located at the edge (e.g., the beachfront) of the EIC. Additionally, the photonic transceivers can be located in positions on the EIC that are different from those shown in
[0095] In an example, the ability to locate photonic transceiver in the interior region of the EIC and not at a perimeter edge of the EIC may enable the photonic transceivers to be spaced apart from each other so as to improve the temperature stability around the photonic transceivers.
[0096] In the examples described herein, the optical ports are shown in certain locations on the PICs. Although some examples are provided, the locations of the optical ports on a PIC are not limited to the locations as shown herein. For example, the optical ports could all be located on the same side of the PIC, or the optical ports could be located at an interior region of the PIC.
[0097] In one example, the photonic transceivers support a bidirectional photonic channel that has a data transmission rate of 56 Gbps in each direction, although other data transmission rates are possible. In an example, multiple photonic channels combine to provide data transmission speeds of 1.8 terabits per second (Tbps) for each optical port. In another example, a multiport network switch has 64 optical ports with a total bandwidth capacity of 115.2 Tbps (e.g., 64 ports1.8 Tbps)
[0098] As used herein, a photonic path may be implemented in an optical transmission medium. The optical transmission medium may include an optical waveguide in a PIC, an optical fiber or other optical transmission medium (such as free space optics or glass-etched waveguide), or some combination of the foregoing.
[0099] As used herein, a major surface of the EIC or the PIC is a large planar surface of the EIC and the PIC that is distinct from an edge surface of the EIC or the PIC. Typically, each EIC and PIC has a rectangular shape with two major surfaces and a perimeter edge that includes four edge surfaces. The major surfaces of the EIC and PIC may be referred to as the top and bottom major surfaces, upper and lower major surfaces, and/or first and second major surfaces. In an example, the top major surface and the bottom major surface are terms that refer to the top and bottom planar surfaces of the EIC and PIC relative to the ground or floor as the multiport network switch is intended to be installed in a rack unit of a data center.
[0100] The switching circuitry is configured to direct PDUs (e.g., data packets) from one photonic transceiver corresponding to an optical port to another photonic transceiver corresponding to another optical port within the multiport network switch.
[0101] In an example, some of the switching functionality may be supported by the interfaces (e.g.,
[0102]
[0103] In the examples described above with reference to
[0104] Although some configurations of photonic transceivers and switching circuitry have been described above, other configurations of photonic transceivers and switching circuitry are possible. In another example, photonic transceiver cells are configured in a matrix, with each photonic transceiver cell including at least one photonic transceiver, an interface, and switching circuitry.
[0105]
[0106] As illustrated in the top view of
[0107]
[0108] The switching circuitry corresponding to each photonic transceiver is not represented in
[0109]
[0110] In one example, network traffic may be received on optical port 4 and sent out of the multiport network switch on optical port 6. In this example, the network traffic is transformed from the optical domain to the electrical domain by the photonic transceiver, P4, transferred in the electrical domain of the EIC from the switching circuitry corresponding to photonic transceiver P4 to the switching circuitry corresponding to photonic transceiver P6 via the electrical paths in the EIC that run between the switching circuitry of the adjacent photonic transceiver cells. Then, the network traffic is transformed from the electrical domain to the optical domain by the photonic transceiver P6 and transmitted out optical port P6. In another example, network traffic may be received on optical port 5 and sent out of the multiport network switch on optical port 10. In this example, the network traffic is transformed from the optical domain to the electrical domain by the photonic transceiver, P5, transferred in the electrical domain of the EIC from the switching circuitry corresponding to photonic transceiver P5 to the switching circuitry corresponding to photonic transceiver P10 via the electrical paths in the EIC that run between the switching circuitry of the adjacent photonic transceiver cells. For example, the network traffic may be communicated on a path that hops from the switching circuity corresponding to photonic transceiver cell P5, to the switching circuity corresponding to photonic transceiver cell P6, to the switching circuity corresponding to photonic transceiver cell P13, to the switching circuity corresponding to photonic transceiver cell P14, to the switching circuity corresponding to photonic transceiver cell P12, and finally to the switching circuity corresponding to photonic transceiver cell P10. Then, the network traffic is transformed from the electrical domain to the optical domain by the photonic transceiver P10 and transmitted out optical port P10. Because the multiport network switch employs a matrix of photonic transceiver cells 1180 that are formed by the EIC 1102 stacked on the PIC 1104, the transformation of signals from the optical domain to the electrical domain and the transformation of signals from the electrical domain to the optical domain does not have to happen at the beachfront, or edges, of the EIC, but can instead happen wherever the drivers and amplifiers can be fabricated on the EIC. Thus, the number of ports that can be included on such a multiport network switch is not limited to the beachfront, or edge dimensions, of the EIC and more photonic transceivers may be included in the EIC than if the location of the photonic transceivers was limited exclusively to the beachfront (e.g., the edges) of the EIC. Additionally, because the photonic transceivers are not limited exclusively to the beachfront of the EIC, the photonic transceivers can be strategically located near each other in a manner that reduces the physical distance that electrical signals must travel through the EIC to be transferred from one photonic transceiver to another photonic transceiver to implement multiport network switching. For example, having a closely spaced matrix of photonic transceiver cells can reduce the amount of physical distance that electrical signals have to propagate across the EIC to get from a photonic transceiver corresponding to an input optical port of the multiport network switch to a photonic transceiver corresponding to an output optical port of the multiport network switch, which can reduce power consumption and reduce latency across the multiport network switch.
[0111]
[0112]
[0113] As described above, the EIC and the PIC may be fabricated as separate devices and then physically and electrically coupled to each other via electrical interconnects. In one example, the EIC alone includes novel features of switching circuitry and a plurality of first portions of photonic transceivers, with each first portion of a photonic transceiver including a driver interface exposed at a major surface of the integrated circuit device and an amplifier interface exposed at the major surface of the integrated circuit device, wherein the switching circuitry is configured to transfer digital signals, which are generated from electrical signals received from the first portion of a first photonic transceiver, to the first portion of a second photonic transceiver based on information in the digital signals. In an example, a first set of the plurality of first portions of photonic transceivers corresponds to spine photonic transceivers, and a second set of the plurality of first portions of photonic transceivers corresponds to leaf photonic transceivers. In another example, a first set of the plurality of first portions of photonic transceivers corresponds to spine photonic transceivers, and a second set of the plurality of first portions of photonic transceivers corresponds to leaf photonic transceivers, wherein the switching circuitry surrounds the spine photonic transceivers and the leaf photonic transceivers surround the switching circuitry. In another example, a first set of the plurality of first portions of photonic transceivers corresponds to interior photonic transceivers, and a second set of the plurality of first portions of photonic transceivers corresponds to exterior photonic transceivers, wherein the exterior photonic transceivers surround the interior photonic transceivers. In another example, the EIC has a footprint, wherein the first portions of the photonic transceivers include a set of first portions located in an interior region of the footprint of the integrated circuit device, and a set of first portions located in an exterior region of the footprint of the integrated circuit device.
[0114] The connections as discussed herein may be any type of connection suitable to transfer signals or power from or to the respective nodes, units, or devices, including via intermediate devices. The connections may be illustrated or described in reference to being a single connection, a plurality of connections, unidirectional connections, or bidirectional connections. However, different embodiments may vary the implementation of the connections. For example, single connections carrying multiple signals may be separated out into various different connections carrying subsets of these signals. The term coupled or similar language may include a direct physical connection or a connection through other intermediate components even when those intermediate components change the form of coupling from a source to a destination.
[0115] The present disclosure provides computing systems, implemented by one or more circuit packages (e.g., SIPs), that achieve reduced power consumption and/or increased processing speed. In accordance with various embodiments, power consumed for, in particular, data movement is reduced by maximizing data locality in a circuit package and reducing energy losses when data movement is needed. Power-efficient data movement, in turn, can be accomplished by moving data over small distances in the electronic domain, while leveraging photonic channels for data movement in scenarios where the resistance in the electronic domain and/or the speed at which the data can move in the electronic domain leads to bandwidth limitations that cannot be overcome using existing electronic technology. Thus, in some embodiments, each circuit package includes an electronic integrated circuit (EIC) that includes switching circuitry and photonic transceivers that are connected by bidirectional photonic channels (e.g., implemented in a PIC in a separate layer or chip of the package) into a hybrid, electronic-photonic (or electro-photonic) multiport network switch. The multiport network switch may be connected, by bidirectional photonic channels, to other multiport network switches and/or network nodes (e.g., compute nodes and/or memory nodes). While the described computing system and its various novel aspects are generally applicable to a wide range of networking applications, the computing system is particularly well suited to data centers that are implementing ML models, in particular ANNs. In one example, the computing system is particularly well suited to data centers that utilize leaf-spine network architectures.
[0116] As described herein, the present disclosure includes a number of practical applications having features described herein that provide benefits and/or solve problems associated with providing a multiport network switch with sufficient port density, data processing speed, and energy efficiency for effective operation in a data center, e.g., a data center that for processing AI and/or ML models. Some example benefits are discussed herein in connection with various features and functionalities provided by the computing system as described. It will be appreciated that benefits explicitly discussed in connection with one or more embodiments described herein are provided by way of example and are not intended to be an exhaustive list of all possible benefits of the computing system.
[0117] The various network topologies enabled by the multiport network switch may provide significant speed and energy savings. For example, photonic transport of data is typically more efficient than an equivalent high-bandwidth electrical interconnect in an EIC of the circuit package itself. By implementing one or more multiport network switches in a data center, the electrical cost of transmitting data may be significantly reduced. A further added benefit is that of being able to control the power density of the system by spacing the location of photonic transceivers on the EIC to optimize cooling efficiency, as the distances and arrangements are not dictated by electrical interfaces.
Particular Implementations
[0118] Described implementations of the subject matter can include one or more features, alone or in combination, as described in the following clauses.
[0119] Clause 1. A switching system, the switching system comprising: [0120] a photonic integrated circuit (PIC) having optical ports; [0121] an electric integrated circuit (EIC) stacked on the PIC and having switching circuitry; and [0122] photonic transceivers optically coupled to the optical ports of the PIC and electrically coupled to the switching circuitry of the EIC; and [0123] wherein the switching circuitry is configured to transfer digital signals, which are generated from optical signals received at a first photonic transceiver of the photonic transceivers, to a second photonic transceiver of the photonic transceivers based on information in the digital signals, wherein the first photonic transceiver is optically coupled to a first optical port of the PIC and the second photonic transceiver is optically coupled to a second optical port of the PIC.
[0124] Clause 2. The switching system of clause 1, wherein each photonic transceiver includes a first portion in the EIC, a second portion in the PIC, and electrical interconnects that electrically couple the first portion in the EIC and the second portion in the PIC.
[0125] Clause 3. The switching system of clause 1 or clause 2, wherein: [0126] the first portion of each photonic transceiver in the EIC includes a driver and an amplifier; and [0127] the second portion of each photonic transceiver in the PIC includes a modulator and a photodetector.
[0128] Clause 4. The switching system of any of the clauses 1 to 3, wherein: [0129] the driver of each photonic transceiver includes a driver interface at a bottom major surface of the EIC; [0130] the amplifier of each photonic transceiver includes an amplifier interface at the bottom major surface of the EIC; [0131] the modulator of each photonic transceiver includes a modulator interface at a top major surface of the PIC; [0132] the photodetector of each photonic transceiver includes a photodetector interface at the top major surface of the PIC; [0133] wherein the driver interface is coupled to the modulator interface by a first electrical interconnect and the amplifier interface is coupled to the photodetector by a second electrical interconnect.
[0134] Clause 5. The switching system of any of the clauses 1 to 4, wherein the first portion of each photonic transceiver is vertically aligned with a second portion of a corresponding photonic transceiver.
[0135] Clause 6. The switching system of any of the clauses 1 to 5, wherein the second portion of each photonic transceiver in the PIC includes an electro-absorption modulator.
[0136] Clause 7. The switching system of any of the clauses 1 to 6, wherein: [0137] each photonic transceiver includes a first portion in the EIC, a second portion in the PIC, and electrical interconnects that electrically couple the first portion in the EIC and the second portion in the PIC; [0138] the first portion of each photonic transceiver in the EIC includes a driver and an amplifier; and [0139] the second portion of each photonic transceiver in the PIC includes a modulator and a photodetector; and [0140] wherein the first portion of each photonic transceiver is vertically aligned with a second portion of a corresponding photonic transceiver.
[0141] Clause 8. The switching system of any of the clauses 1 to 7, wherein the photonic transceivers are located in an interior region of the EIC.
[0142] Clause 9. The switching system of any of the clauses 1 to 8, wherein the EIC has a footprint, and wherein the photonic transceivers include photonic transceivers located in an interior region of the footprint of the EIC.
[0143] Clause 10. The switching system of any of the clauses 1 to 9, wherein the photonic transceivers include a first set of photonic transceivers surrounded by the switching circuitry and a second set of photonic transceivers surrounding the switching circuitry.
[0144] Clause 11. The switching system of any of the clauses 1 to 10, wherein the photonic transceivers include spine photonic transceivers and leaf photonic transceivers.
[0145] Clause 12. The switching system of any of the clauses 1 to 11, wherein the photonic transceivers include spine photonic transceivers surrounded by the switching circuitry, and leaf photonic transceivers surrounding the switching circuitry.
[0146] Clause 13. The switching system of any of the clauses 1 to 12, wherein the electrical interconnects are less than 50 microns.
[0147] Clause 14. The switching system of any of the clauses 1 to 13, wherein the switching circuitry includes a switch fabric.
[0148] Clause 15. The switching system of claim 14 wherein the switch fabric is a crosspoint matrix.
[0149] Clause 16. The switching system of clause 15, wherein the PIC includes an optical port configured for connection to a light engine.
[0150] Clause 17. A method comprising: [0151] receiving optical signals at a first optical port of a photonic integrated circuit (PIC) that has multiple optical ports; [0152] processing the optical signals at a first photonic transceiver, which is optically coupled to the first optical port of the PIC and electrically coupled to an electric integrated circuit (EIC) that is stacked on the PIC, to generate electrical signals; [0153] converting the electrical signals to digital signals within the EIC; [0154] transferring the digital signals to a second photonic transceiver via switching circuitry of the EIC based on information in the digital signals, wherein the second photonic transceiver is optically coupled to a second optical port of the PIC; and [0155] outputting optical signals from the second optical port of the PIC based on the digital signals that are transferred to the second photonic transceiver.
[0156] Clause 18. The method of clause 17, wherein: [0157] processing optical signals at the first photonic transceiver includes converting the optical signals to electrical signals at the PIC and amplifying the electrical signals at the EIC; and [0158] outputting optical signals at the second photonic transceiver includes generating driver signals at the EIC and modulating an optical carrier at the PIC in response to the driver signals.
[0159] Clause 19. The method of clause 17 or clause 18, wherein each optical port of the PIC includes a corresponding photonic transceiver, wherein each photonic transceiver includes a first portion in the EIC, a second portion in the PIC, and electrical interconnects that electrically couple the first portion in the EIC and the second portion in the PIC.
[0160] Clause 20. The method of any of the clauses 17 to 19, wherein: [0161] the first portion of each photonic transceiver in the EIC includes a driver and an amplifier; and [0162] the second portion of each photonic transceiver in the PIC includes a modulator and a photodetector.
[0163] Clause 21. The method of any of the clauses 17 to 20, wherein the first portion of each photonic transceiver is vertically aligned with a second portion of a corresponding photonic transceiver.
[0164] Clause 22. The method of any of the clauses 17 to 21, wherein the second portion in the PIC of each photonic transceiver includes an electro-absorption modulator.
[0165] Clause 23. The method of any of the clauses 17 to 22, wherein: [0166] the driver of each photonic transceiver includes a driver interface at a bottom major surface of the EIC; [0167] the amplifier of each photonic transceiver includes an amplifier interface at the bottom major surface of the EIC; [0168] the modulator of each photonic transceiver includes a modulator interface at a top major surface of the PIC; [0169] the photodetector of each photonic transceiver includes a photodetector interface at the top major surface of the PIC; [0170] wherein the driver interface is coupled to the modulator interface by a first electrical interconnect and the amplifier interface is coupled to the photodetector by a second electrical interconnect.
[0171] Clause 24. The method of any of the clauses 17 to 23, wherein the EIC includes spine photonic transceivers and leaf photonic transceivers, and wherein the first photonic transceiver is a leaf photonic transceiver and the second photonic transceiver is a spine photonic transceiver.
[0172] Clause 25. The method of any of the clauses 17 to 24, wherein the EIC includes spine photonic transceivers and leaf photonic transceivers, and wherein the first photonic transceiver is a spine photonic transceiver and the second photonic transceiver is a leaf photonic transceiver.
[0173] Clause 26. The method of any of the clauses 17 to 25, wherein the EIC includes a first set of photonic transceivers surrounded by a second set of photonic transceivers, and wherein the first photonic transceiver is in the second set of photonic transceivers and the second photonic transceiver is in the first set of photonic transceivers.
[0174] Clause 27. The method of any of the clauses 17 to 26, wherein the EIC includes a first set of photonic transceivers surrounded by a second set of photonic transceivers, and wherein the first photonic transceiver is in the first set of photonic transceivers and the second photonic transceiver is in the second set of photonic transceivers.
[0175] Clause 28. The method of any of the clauses 17 to 27, wherein transferring the digital signals to the second photonic transceiver via the switching circuitry based on information in the digital signals involves identifying an output optical port based on a media access control (MAC) table in the switching circuitry.
[0176] Clause 29. The method of any of the clauses 17 to 28, further comprising receiving an optical carrier at the PIC.
[0177] Clause 30. An integrated circuit device comprising: [0178] switching circuitry; and [0179] a plurality of first portions of photonic transceivers, each first portion of a photonic transceiver including a driver and a driver interface that is exposed at a major surface of the integrated circuit device and an amplifier and an amplifier interface that is exposed at the major surface of the integrated circuit device; [0180] wherein the switching circuitry is configured to transfer digital signals, which are generated from electrical signals received from the first portion of a first photonic transceiver, to the first portion of a second photonic transceiver based on information in the digital signals.
[0181] Clause 31. The integrated circuit device of clause 30, wherein the major surface of the integrated circuit device is bottom major surface of the integrated circuit device.
[0182] Clause 32. The integrated circuit device of clause 30 or clause 31, wherein: [0183] the major surface of the integrated circuit device is a bottom major surface of the integrated circuit device; [0184] the driver interface of each photonic transceiver is located on the bottom major surface of the integrated circuit device to align with a modulator interface on a photonic integrated circuit (PIC); and [0185] the amplifier interface of each photonic transceiver is located on the bottom major surface of the integrated circuit device to align with a photodetector interface on the PIC.
[0186] Clause 33. The integrated circuit device of any of the clauses 30 to 32, wherein: [0187] the driver interface of each photonic transceiver is located on the integrated circuit device to align with a modulator interface on a photonic integrated circuit (PIC); and [0188] the amplifier interface of each photonic transceiver is located on the integrated circuit device to align with a photodetector interface on the PIC.
[0189] Clause 34. The integrated circuit device of any of the clauses 30 to 33, wherein the switching circuitry includes a switch fabric.
[0190] Clause 35. The integrated circuit device of any of the clauses 30 to 34, wherein the switch fabric is a crosspoint matrix.
[0191] Clause 36. The integrated circuit device of any of the clauses 30 to 35, wherein the switching circuitry is configured to identify a photonic transceiver of the integrated circuit device to transfer digital signals to based on information in a media access control (MAC) table that is stored in the integrated circuit device.
[0192] Clause 37. The integrated circuit device of any of the clauses 30 to 36, wherein a first set of the plurality of first portions of photonic transceivers surrounds a second set of the plurality of first portions of photonic transceivers.
[0193] Clause 38. The switching system of any of the clauses 30 to 37, wherein the integrated circuit device has a footprint, and wherein the first portions of the photonic transceivers are located in an interior region of the footprint of the integrated circuit device.
[0194] Clause 39. The integrated circuit device of any of the clauses 30 to 38, wherein a first set of the plurality of first portions of photonic transceivers corresponds to spine photonic transceivers and a second set of the plurality of first portions of photonic transceivers corresponds to leaf photonic transceivers.
[0195] Clause 40. The integrated circuit device of any of the clauses 30 to 39, wherein a first set of the plurality of first portions of photonic transceivers corresponds to spine photonic transceivers and a second set of the plurality of first portions of photonic transceivers corresponds to leaf photonic transceivers, wherein the switching circuitry surrounds the spine photonic transceivers and the leaf photonic transceivers surround the switching circuitry.
[0196] Clause 41. A method comprising: [0197] amplifying analog electrical signals that are received at an amplifier interface of a first portion of a first photonic transceiver of an integrated circuit device, wherein the amplifier interface is exposed at a major surface of the integrated circuit device and electrically coupled to an amplifier of the first photonic transceiver; [0198] converting the amplified analog electrical signals to digital signals within the integrated circuit device; [0199] transferring the digital signals to a second photonic transceiver of the integrated circuit device via switching circuitry of the integrated circuit device based on information in the digital signals; and [0200] driving a driver of the second photonic transceiver in response to the digital signals to generate analog electrical signals at a driver interface of a first portion of the second photonic transceiver, wherein the driver interface is exposed at the major surface of the integrated circuit device.
[0201] Clause 42. The method of clause 41, wherein transferring the digital signals to the second photonic transceiver of the integrated circuit device via the switching circuitry based on information in the digital signals involves identifying an output optical port based on a media access control (MAC) table in the switching circuitry.
[0202] Clause 43. The method of clause 41 or clause 42, wherein the EIC includes spine photonic transceivers and leaf photonic transceivers, and wherein the first photonic transceiver is a leaf photonic transceiver and the second photonic transceiver is a spine photonic transceiver.
[0203] Clause 44. The method of any of the clauses 41 to 43, wherein the EIC includes spine photonic transceivers and leaf photonic transceivers, and wherein the first photonic transceiver is a spine photonic transceiver and the second photonic transceiver is a leaf photonic transceiver.
[0204] Clause 45. The method of any of the clauses 41 to 44, wherein the EIC includes interior photonic transceivers surrounded by exterior photonic transceivers, and wherein the first photonic transceiver is an exterior photonic transceiver and the second photonic transceiver is an interior photonic transceiver.
[0205] Clause 46. The method of any of the clauses 41 to 45, wherein the EIC includes interior photonic transceivers surrounded by exterior photonic transceivers, and wherein the first photonic transceiver is an interior photonic transceiver and the second photonic transceiver is an exterior photonic transceiver.
[0206] Although the operations of the method(s) herein are shown and described in a particular order, the order of the operations of each method may be altered so that certain operations may be performed in an inverse order or so that certain operations may be performed, at least in part, concurrently with other operations. In another embodiment, instructions or sub-operations of distinct operations may be implemented in an intermittent and/or alternating manner.
[0207] It should also be noted that at least some of the operations for the methods described herein may be implemented using software instructions stored on a computer useable storage medium for execution by a computer. As an example, an embodiment of a computer program product includes a computer useable storage medium to store a computer readable program.
[0208] It will be readily understood that the components of the embodiments as generally described herein and illustrated in the figures could be arranged and designed in a wide variety of different configurations. Thus, the following more detailed description of various examples, as represented in the figures, is not intended to limit the scope of the present disclosure, but is merely representative of various embodiments. While the various aspects of the examples are presented in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.
[0209] The disclosed technology may be embodied in other specific forms without departing from its spirit or essential characteristics. The described examples are to be considered in all respects only as illustrative and not restrictive. The scope of the disclosed technology is, therefore, indicated by the appended claims rather than by this detailed description. All changes which come within the meaning and range of equivalency of the claims are to be embraced within their scope.
[0210] Reference throughout this specification to features, advantages, or similar language does not imply that all of the features and advantages that may be realized with the disclosed technology should be or are in any single example. Rather, language referring to the features and advantages is understood to mean that a specific feature, advantage, or characteristic described in connection with an example is included in at least one example of the disclosed technology. Thus, discussions of the features and advantages, and similar language, throughout this specification may, but do not necessarily, refer to the same example.
[0211] Furthermore, the described features, advantages, and characteristics of the disclosed technology may be combined in any suitable manner in one or more examples. One skilled in the relevant art will recognize, in light of the description herein, that the disclosed technology can be practiced without one or more of the specific features or advantages of a particular example. In other instances, additional features and advantages may be recognized in certain examples that may not be present in all examples of the disclosed technology.
[0212] Reference throughout this specification to one example, an example, or similar language means that a particular feature, structure, or characteristic described in connection with the indicated example is included in at least one example of the disclosed technology. Thus, the phrases in one example, in an example, and similar language throughout this specification may, but do not necessarily, all refer to the same example.
[0213] Although specific examples of the disclosed technology have been described and illustrated, the disclosed technology is not to be limited to the specific forms or arrangements of parts so described and illustrated. The scope of the disclosed technology is to be defined by the claims appended hereto and their equivalents.