Permutated ring network interconnected computing architecture

Abstract

A computer architecture that connects a plurality of compute engines and memory banks using one or more permutated ring networks to provide a scalable, high-bandwidth, low-latency point-to-point multi-chip communications solution.

Claims

1. A computer architecture comprising: a plurality of computing slices, each including a plurality of compute engines, a plurality of memory banks, a communication node and a first-level interconnect structure coupling each of the plurality of compute engines, the plurality of memory banks and the communication node; and a second-level interconnect structure that includes a permutated ring network having a plurality of bi-directional source-synchronous ring networks, each including a plurality of data transport stations, wherein each communication node of the plurality of computing slices is coupled to one of the data transport stations in each of the plurality of bi-directional source-synchronous ring networks.

2. The computer architecture of claim 1, wherein each first-level interconnect structure includes a slice-level permutated ring network having a plurality of bi-directional source-synchronous ring networks, each including a plurality of data transport stations, wherein each of the plurality of compute engines, the plurality of memory banks, and the communication node of the corresponding computing slice is coupled to one of the data transport stations in each of the plurality of bi-directional source-synchronous ring networks of the slice-level permutated ring network.

3. The computer architecture of claim 1, wherein each first-level interconnect structure includes a crossbar switch coupled to the plurality of compute engines, the plurality of memory banks, and the communication node of the corresponding computing slice.

4. The computer architecture of claim 1, wherein each first-level interconnect structure includes a ring network coupled to the plurality of compute engines, the plurality of memory banks, and the communication node of the corresponding computing slice.

5. The computer architecture of claim 1, further comprising: a memory interface communication node coupled to the second-level interconnect structure, wherein the memory interface communication node is coupled to one of the data transport stations in each of the plurality of bi-directional source-synchronous ring networks: and a memory device coupled to the memory interface communication node.

6. The computer architecture of claim 5, wherein the memory device is a dynamic random access memory (DRAM) device.

7. The computer architecture of claim 1, further comprising a first network communication node coupled to the second-level interconnect structure, wherein the first network communication node is coupled to one of the data transport stations in each of the plurality of bi-directional source-synchronous ring networks of the second-level interconnect structure.

8. The computer architecture of claim 7, further comprising a system-level interconnect structure coupled to the first network communication node.

9. The computer architecture of claim 8, wherein the system-level interconnect structure comprises a plurality of network communication nodes coupled to a third-level interconnect structure, wherein a first one of the plurality of network communication nodes is coupled to the first network communication node.

10. The computer architecture of claim 9, further comprising a host system processor coupled to a second one of the plurality of network communication nodes.

11. The computer architecture of claim 10, further comprising a system memory coupled to a third one of the plurality of network communication nodes.

12. The computer architecture of claim 8, further comprising a a second plurality of computing slices, each including a second plurality of compute engines, a second plurality of memory banks, a second communication node and a second first-level interconnect structure coupling each of the second plurality of compute engines, the second plurality of memory banks and the second communication node; a second second-level interconnect structure that includes a second permutated ring network having a second plurality of bi-directional source-synchronous ring networks, each including a second plurality of data transport stations, wherein each second communication node of the second plurality of computing slices is coupled to one of the second data transport stations in each of the second plurality of bi-directional source-synchronous ring networks; and a second network communication node coupled to the second second-level interconnect structure, wherein the second network communication node is coupled to one of the data transport stations in each of the second plurality of bi-directional source-synchronous ring networks of the second second-level interconnect structure, wherein the second network communication node is coupled to the system-level interconnect structure.

13. The computer architecture of claim 9, wherein the third-level interconnect structure includes a system-level permutated ring network having a plurality of bi-directional source-synchronous ring networks, each including a plurality of data transport stations, wherein each of the plurality network communication nodes is coupled to one of the data transport stations in each of the plurality of bi-directional source-synchronous ring networks of the system-level permutated ring network.

14. The computer architecture of claim 9, wherein the third-level interconnect structure includes a crossbar switch coupled to the plurality of network communication nodes.

15. The computer architecture of claim 9, wherein the third-level interconnect structure includes a ring network coupled to the plurality of network communication nodes.

16. The computer architecture of claim 1, wherein each communication node includes a connection to each of the other communication nodes, wherein the connection includes a direct connection between adjacent data transport stations.

17. The computer architecture of claim 1, wherein a unique pair of adjacent data transport stations provides a connection between each pair of the communication nodes.

18. The computer architecture of claim 1, wherein the plurality of bi-directional source-synchronous ring networks operate in a first clock domain, and the plurality of compute engines and the plurality of memory banks operate in a second clock domain, different than the first clock domain.

19. The computer architecture of claim 1, wherein the plurality of computing slices and the second level interconnect structure are located on the same semiconductor chip.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) FIG. 1 is a block diagram of a conventional computer architecture that is optimized for a shared memory programming model.

(2) FIG. 2 is a block diagram of a computer architecture that uses permutated ring networks to connect a plurality of compute engines and memory devices, in accordance with one embodiment of the present invention.

(3) FIG. 3 is a modified view of the computer architecture of FIG. 2 in accordance with one embodiment of the present invention.

(4) FIG. 4 is a block diagram of first level permutated ring network used in the computer architecture of FIGS. 2-3 in accordance with one embodiment of the present invention.

(5) FIG. 5 is an interconnect matrix of the four communication channels of the first level permutated ring network of FIG. 4, in accordance with one embodiment of the present invention.

(6) FIG. 6 is a routing table that defines the flow of traffic one the permutated ring network of FIG. 4 in accordance with one embodiment of the present embodiment.

(7) FIG. 7 is a block diagram of a computer architecture in accordance with an alternate embodiment of the present invention.

(8) FIG. 8 is a block diagram of a computer architecture in accordance with another alternate embodiment of the present invention.

DETAILED DESCRIPTION

(9) The present invention uses a permutated ring network (PRN) architecture to provide a better solution for the interconnect systems of a machine learning computing system. The PRN architecture includes a flat memory hierarchy, which allows compute engines on different chips (and on the same chip) to communicate directly among one another using a common communication protocol. The interconnect system is non-cache coherent. In one embodiment, the interconnect system uses a single interconnect structure (i.e., a plurality of permutated ring networks).

(10) In an alternate embodiment, a PRN structures are used only at specific locations within the interconnect structure (e.g., to connect a plurality of computing slices on the same chip).

(11) FIG. 2 is a block diagram of a computer system 200 in accordance with one embodiment of the present invention. Computer system 200 includes a plurality of processor chips 201-204, host processor system 205, system memory 206, system level interconnect chip 210 and DRAM devices 211-214. Although only four processor chips 201-204 are illustrated in FIG. 2, it is understood that computer system 200 can be easily modified to include other numbers of processor chips in other embodiments. Moreover, although only processor chip 201 is illustrated in detail in FIG. 2, it is understood that processor chips 202-204 include the same internal elements as the processor chip 201 in the described embodiments. In alternate embodiments, the processor chips 201-204 can include different numbers of computing slices, compute engines and/or memory banks, in accordance with the descriptions provided below.

(12) In the illustrated embodiment, processor chip 201 includes four computing slices 1, 2, 3, and 4, and a permutated ring network (PRN) based interconnect structure 21. Although four slices are illustrated in FIG. 2, it is understood that other numbers of slices can be included on processor chip 201 in other embodiments. Each slice includes a plurality of compute engines, a plurality of memory banks, a communication node and a first level PRN-based interconnect structure. More specifically, slices 1, 2, 3 and 4 include compute engine sets CE.sub.1, CE.sub.2, CE.sub.3 and CE.sub.4, respectively, memory bank sets M.sub.1, M.sub.2, M.sub.3 and M.sub.4, respectively, first level PRN-based interconnect structures 11, 12, 13 and 14, respectively, and communication nodes CN.sub.1, CN.sub.2, CN.sub.3 and CN.sub.4, respectively. Each of the compute engine sets CE.sub.1, CE.sub.2, CE.sub.3 and CE.sub.4 includes a plurality of compute engines (e.g., local processors). Each of the compute engine sets CE.sub.1, CE.sub.2, CE.sub.3 and CE.sub.4 includes four compute engines in the illustrated example. However, it is understood that other numbers of compute engines can be included in each compute engine set in other embodiments. Similarly, each of the memory bank sets M.sub.1, M.sub.2, M.sub.3 and M.sub.4 includes a plurality of memory banks. Each of the memory bank sets includes four memory banks in the illustrated example. However, it is understood that other numbers of memory banks can be included in each memory bank set in other embodiments. In one embodiment, each of the memory banks in memory bank sets M.sub.1, M.sub.2, M.sub.3 and M.sub.4 is a static random access memory (SRAM), which enables relatively fast memory accesses to be implemented.

(13) Within each of the computing slices 1, 2, 3 and 4, the corresponding first level PRN-based interconnect structures 11, 12, 13 and 14 couple the corresponding compute engine sets CE.sub.1, CE.sub.2, CE.sub.3 and CE.sub.4 and the corresponding memory bank sets M.sub.1, M.sub.2, M.sub.3 and M.sub.4. This allows each of the compute engines to access each of the memory banks within the same slice using the corresponding first level PRN-based interconnect structure. For example, each of the four compute engines in the compute engine set CE.sub.1 of computing slice 1 is able to access each of the four memory banks of the memory bank set M.sub.1 of computing slice 1 through the corresponding first level PRN-based interconnect structure 11 of slice 1.

(14) The first level PRN-based interconnect structures 11, 12, 13 and 14, are also coupled to corresponding communication nodes CN.sub.1, CN.sub.2, CN.sub.3 and CN.sub.4, within the corresponding slices 1, 2, 3 and 4. The communication nodes CN.sub.1, CN.sub.2, CN.sub.3 and CN.sub.4 are coupled to the second level PRN-based interconnect structure 21. As described in more detail below, the communication nodes CN.sub.1, CN.sub.2, CN.sub.3 and CN.sub.4 pass messages and data between the corresponding first level PRN-based interconnect structures 11, 12, 13 and 14 and the second level PRN-based interconnect structure 21.

(15) This configuration allows each of the compute engines on processor chip 201 to access each of the memory banks on the processor chip 201 using the first level PRN-based interconnect structures 11-14 and the second level PRN-based interconnect structure 21 (if necessary). For example, each of the compute engines in the compute engine set CE.sub.1 of computing slice 1 is able to access each of the memory banks of the memory bank set M.sub.4 of slice 4 through a path that includes: the corresponding first level PRN-based interconnect structure 11 of computing slice 1, the communication node CN.sub.1, the second level PRN-based interconnect structure 21, the communication node CN.sub.4, and the first level PRN-based interconnect structure 14 of computing slice 4.

(16) This configuration also allows each of the compute engines on processor chip 201 to communicate with each of the other compute engines on the processor chip 201 using the first level PRN-based interconnect structures 11-14 and the second level PRN-based interconnect structure 21 (if necessary). For example, each of the compute engines in the compute engine set CE.sub.2 of computing slice 2 is able to communicate with each of the compute engines in the compute engine set CE.sub.3 of computing slice 3 through a path that includes: the corresponding first level PRN-based interconnect structure 12 of slice 2, the communication node CN.sub.2, the second level PRN-based interconnect structure 21, the communication node CN.sub.3, and the first level PRN-based interconnect structure 13 of slice 3.

(17) The second level PRN-based interconnect structure 21 is also coupled to external DRAM 211 through memory interface communication node CN.sub.5. This configuration allows each of the compute engines of processor chip 201 to access the DRAM 211 through the first level PRN-based interconnect structures 11-14 and the second level PRN-based interconnect structure 21. For example, each of the compute engines in the compute engine set CE.sub.1 of computing slice 1 is able to access DRAM 211 through a path that includes: the corresponding first level PRN-based interconnect structure 11 of computing slice 1, the communication node CN.sub.1, the second level PRN-based interconnect structure 21 and the communication node CN.sub.5.

(18) The computer system 200 of FIG. 2 also includes a third level PRN-based interconnect structure 31, which is fabricated on system level interconnect chip 210. The third level PRN-based interconnect structure 31 is coupled to a plurality of communication nodes CN.sub.11-CN.sub.16 on chip 210. As described in more detail below, the third level PRN-based interconnect structure 31 enables the transmission of messages and data between the communication nodes CN.sub.11-CN.sub.16. Communication nodes CN.sub.11, CN.sub.12, CN.sub.13 and CN.sub.14 are coupled to processor chips 201, 202, 203 and 204, respectively. Communication nodes CN.sub.15 and CN.sub.16 are coupled to host system processor 205 and system memory 206, respectively.

(19) The system level interconnect chip 210 allows for the transmission of data and messages between host system processor 205, system memory 206 and each of the processor chips 201-204. More specifically, host processor 205 can communicate with any of the compute engines on processor chips 201-204 or any of the memory banks on processor chips 201-204. For example, host processor 205 is able to access the compute engines in the compute engine set CE.sub.1 of computing slice 1 (or the memory banks of the memory bank set M.sub.1 of computing slice 1) through a path that includes: communication node CN.sub.15, the third level PRN-based interconnect structure 31, network communication nodes CN.sub.11 and CN.sub.6, second level PRN-based interconnect structure 21, communication node CN.sub.1 and first level PRN-based interconnect structure 11.

(20) Host processor 205 can also communicate with any of the DRAMs 211-214. For example, host processor 205 is able to access the DRAM 211 through a path that includes: communication node CN.sub.15, the third level PRN-based interconnect structure 31, network communication nodes CN.sub.11 and CN.sub.6, second level PRN-based interconnect structure 21 and communication node CN.sub.5. Host processor 205 can access DRAMs 212-214 through similar paths in processor chips 202-204, respectively.

(21) Host processor 205 can also communicate with the system memory 206 through a path that includes: communication node CN.sub.15, the third level PRN-based interconnect structure 31 and communication node CN.sub.16.

(22) In addition, each of the compute engines on any of the processor chips 201-204 can communicate with any of the compute engines or memory banks on any of the other processor chips 201-204, as well as the DRAMs 211-214 coupled to these other processor chips.

(23) In accordance with one embodiment, the various memory banks, compute engines and communication nodes located on processor chips 201-204, DRAMs 211-214, host system 205, system memory 206 and the communication nodes CN.sub.11-CN.sub.16 on system level interconnect chip 210 are assigned unique system addresses, thereby enabling each of these system elements to be easily addressed by (and therefore communicate with) any of the other system elements.

(24) FIG. 3 is a block diagram of PRN-based computer system 200, which shows processor chip 202 in detail. Similar elements in processor chips 201 and 202 are labeled with similar reference numbers. Thus, processor chip 202 includes computing slices 1, 2, 3 and 4, which include memory bank sets M.sub.1, M.sub.2, M.sub.3, M.sub.4, respectively, compute engine sets CE.sub.1, CE.sub.2, CE.sub.3, and CE.sub.4, respectively, first level PRN-based interconnect structures 11, 12, 13 and 14, respectively, and communication nodes CN.sub.1, CN.sub.2, CN.sub.3 and CN.sub.4, respectively. Processor chip 202 also includes second level PRN-based interconnect structure 21, memory interface communication node CN.sub.5 and network communication node CN.sub.6, which are coupled to DRAM 212 and network communication node CN.sub.12, respectively.

(25) This configuration allows each of the compute engines in the compute engine set CE.sub.1 (of processor chip 201) to access each of the compute engines in the compute engine set CE.sub.3 (of processor chip 202) through a path that includes: first level PRN-based interconnect structure 11, communication node CN.sub.1, second level PRN-based interconnect structure 21, network communication nodes CN.sub.6 and CN.sub.11, the third level PRN-based interconnect structure 31, network communication nodes CN.sub.12 and CN.sub.6, second level PRN-based interconnect structure 21, communication node CN.sub.3 and first level PRN-based interconnect structure 13. Similarly, each of the compute engines in the compute engine set CE.sub.1 (of processor chip 201) is able to access each of the memory banks in the memory bank set M.sub.3 (of processor chip 202) using the same path.

(26) This configuration also allows each of the compute engines of each processor chip to access the DRAMs coupled to the other processor chips. For example, each of the compute engines in the compute engine set CE.sub.1 of slice 1 (of processor chip 201) is able to access the DRAM 212 (coupled to processor chip 202) through a path that includes: the corresponding first level PRN-based interconnect structure 11 of slice 1, the communication node CN.sub.1, the second level PRN-based interconnect structure 21, communication nodes CN.sub.6 and CN.sub.11, the third level PRN-based interconnect structure 31, communication nodes CN.sub.12 and CN.sub.6, second level PRN-based interconnect structure 21, and communication node CN.sub.5.

(27) As described above, the PRNA interconnected computer system 200 has three levels of hierarchies, including slice level, chip level and system level, wherein each level is defined by its physical construction boundary.

(28) The slice level, represented by computing slices 1-4 (and computing slices 1-4), is the basic building block of the computer system 200. Each computing slice, by itself, can be implemented as a small scale machine learning processor via a bridge between the host system processor 205 and the first level PRN-based interconnect structure.

(29) The chip level, represented by processor chips 201-204, is defined by the subsystems included on a die, including a plurality of computing slices and the corresponding second level PRN-based interconnect structure. Each processor chip can be implemented as a medium scale machine learning system via a bridge between the host system processor 205 and the second level PRN based interconnect structures.

(30) The system level, which includes the host system processor 205, is built on a plurality of processor chips and the system level interconnect chip 210. The processor chips 201-204 communicate through the system level interconnect chip 210. The third level PRN-based interconnect structure 31 implemented by the system level interconnect chip 210 advantageously operates with a high bandwidth, a low latency and a high power efficiency. By implementing the first, second and third level interconnect structures using permutated ring networks, the same communication protocol can be maintained across the entire system. This greatly simplifies the shared memory and message passing protocols across the system. As described above, computer system 200 enables any compute engine to access all of the memory bank sets (e.g., memory bank sets M1-M4 and M1-M4) and all of the DRAMs (e.g., DRAMs 211-214) in the system 200 via the PRN-based interconnect structures. Hence, computer system 200 is a highly flexible shared memory computing system.

(31) Moreover, all of the compute engines of computer system 200 can communicate directly among each other via the PRN-based interconnect structures. Advantageously, software support is not required to translate messages exchanged between compute engines of different computing slices or chips, thereby resulting in a highly efficient message passing computing system.

(32) The PRN-based interconnect structure used to implement the level 1, level 2 and level 3 PRN interconnect structures of FIGS. 2 and 3 is described in more detail in commonly owned, co-pending U.S. Published Patent Application No. 2018/0145850, which is incorporated by reference in its entirety. The use of PRN interconnect structures in computer system 200 accordance with various embodiments is described in more detail below.

(33) FIG. 4 is a block diagram of first level permutated ring network 11 in accordance with one embodiment of the present invention. The other first level permutated ring networks of computer system 200 (e.g., permutated ring networks 12-14 and 11-14) may be identical to first level permutated ring network 11. In the illustrated embodiment, first level permutated ring network 11 includes four bi-directional source synchronous ring networks 401, 402, 403 and 404. Each of the ring networks 401-404 functions as a communication channel. Although the illustrated permutated ring network 11 includes nine communication nodes (i.e., communication node CN.sub.1, compute engines CE.sub.1A, CE.sub.1B, CE.sub.1C and CE.sub.ID of compute engine set CE.sub.1 and memory banks M.sub.1A, M.sub.1B, M.sub.1C and M.sub.1D of memory bank set M.sub.1) and four communication channels 401-404, it is understood that other numbers of communication nodes and communication channels can be used in other embodiments. In general, the number of communication nodes in the first level permutated ring network 11 is identified by the value, N, and the number of bi-directional ring networks in the first level permutated ring network 11 is identified by the value M. The number of communication channels (M) is selected to provide an appropriate tradeoff between the bandwidth requirements of the communication network and the area-power constraints of the communication network.

(34) Each of the communication channels 401-404 includes a plurality of data transport stations connected by bi-directional links (interconnects). More specifically, communication channel 401 includes nine data transport stations A0-A8, communication channel 402 includes nine data transport stations B0-B8, communication channel 403 includes nine data transport stations C0-C8 and communication channel 404 includes nine data transport stations D0-D8. The bi-directional links of communication channel 401 are shown as solid lines that connect the data transport stations A0-A8 in a ring. The bi-directional links of communication channel 402 are shown as long dashed lines that connect the data transport stations B0-B8 in a ring. The bi-directional links of communication channel 403 are shown as dashed-dotted lines that connect the data transport stations C0-C8 in a ring. The bi-directional links of communication channel 404 are shown as short dashed lines that connect the data transport stations D0-D8 in a ring. The bi-directional links allow for the simultaneous transmission of data and clock signals in both the clockwise and counterclockwise directions.

(35) In general, each of the data transport stations A0-A8, B0-B8, C0-C8 and D0-D8 provides an interface that enables the transfer of data between the nine communication nodes and the communication channels 401-404.

(36) In general, each of the communication channels 401-404 is coupled to receive a master clock signal. Thus, in the example of FIG. 4, communication channels 401, 402, 403 and 404 are coupled to receive master clock signals CKA, CKB, CKC and CKD, respectively. In the embodiment illustrated, data transport stations A0, B0, C0 and D0 are coupled to receive the master clock signals CKA, CKB, CKC and CKD, respectively. However, in other embodiments, other data transport stations in communication channels 401, 402, 403 and 404 can be coupled to receive the master clock signals CKA, CKB, CKC and CKD, respectively. Although four separate master clock signals CKA, CKB, CKC and CKD are illustrated, it is understood that each of the master clock signals CKA, CKB, CKC and CKD can be derived from a single master clock signal. In the described embodiments, each of the master clock signals CKA, CKB, CKC and CKD have the same frequency.

(37) Conventional clock generation circuitry (e.g., a phase locked loop circuit) can be used to generate the master clock signals CKA, CKB, CKC and CKD. In the described embodiments, the master clock signals can have a frequency of about 5 GHz or more. However, it is understood that the master clock signals can have other frequencies in other embodiments. The frequency and voltage of the master clock signals can be scaled based on the bandwidth demands and power optimization of the ring network architecture. In the illustrated embodiments, data transport stations A0, B0, C0 and D0 receive the master clock signals CKA, CKB, CKC and CKD, respectively. Each of the other data transport stations receives its clock signal from its adjacent neighbor. That is, the master clock signals CKA, CKB, CKC and CKD are effectively transmitted to each of the data transport stations of communication channels 401, 402, 402 and 404, respectively, in series.

(38) Each of the communication channels 401, 402, 403 and 404 operates in a source synchronous manner with respect to its corresponding master clock signal CKA, CKB, CKC and CKD, respectively.

(39) In general, each data transport station can transmit output messages on two paths. In the first path, a message received by from an upstream data transport station is forwarded to a downstream data transport station (e.g., data transport station A0 may forward a message received from downstream data transport station A8 to upstream data transport station A1 on the clockwise path, or data transport station A0 may forward a message received from downstream data transport station A1 to upstream data transport station A8 on the counterclockwise path). In the second path, a message provided by a communication node coupled to the data transport station is routed to a downstream data transport station (e.g., data transport station A0 may forward a message received from compute engine CE.sub.1A to downstream data transport station A1 on the clockwise path, or to downstream data transport station A8 on the counterclockwise path). Also in the second path, a message received by a data transport station is routed to an addressed communication node (e.g., data transport station A0 may forward a message received from downstream data transport station A8 on the clockwise path to compute engine CE.sub.1A, or data transport station A0 may forward a message received from downstream data transport station A0 on the counterclockwise path to compute engine CE.sub.1A). Note that the wires and buffers used to transmit the clock signals and the messages between the data transport stations are highly equalized and balanced in order to minimize setup and hold time loss.

(40) The clock signal path and the message bus operate as a wave pipeline system, wherein messages transmitted between data transport stations are latched into the receiving data transport station in a source-synchronous manner using the clock signal transmitted on the clock signal path. In this manner, messages are transmitted between data transport stations at the frequency of the master clock signals CKA, CKB, CKC and CKD, allowing for fast data transfer between data transport stations.

(41) Because point-to-point source-synchronous communication is implemented, the wire and buffer delays of the clock signal line structure and the message bus structure will not degrade the operating frequency of the communication channels 401-404.

(42) Because the data transport stations have a relatively simple design, the transmission of messages on the permutated ring network 11 can be performed at a relatively high frequency. Communication node CN.sub.1, compute engines CE.sub.1A, CE.sub.1B, CE.sub.1C and CE.sub.1D, and memory banks M.sub.1A, M.sub.1B, M.sub.1C and M.sub.1D typically include a more complicated design, and may operate at a slower frequency than the frequency of the master clock signals CKA, CKB, CKC and CKD.

(43) Note that the circular configuration of the communication channels 401-404 necessitates that messages received by the originating data transport stations A0, B0, C0 and D0 (e.g., the data transport stations that receive the master clock signals CKA, CKB, CKC and CKD) must be resynchronized to the master clock signals CKA, CKB, CKC and CKD, respectively. In one embodiment, resynchronization circuitry (not shown) performs this synchronizing operation by latching the incoming message into a first flip-flop in response to the incoming clock signal received from a downstream data transport station. The message provided at the output of this first flip-flop is then latched into a second flip flop in response to the master clock signal (e.g., CKA). The second flip-flop provides the synchronized message to the originating data transport station (e.g., data transport station A0). This synchronized message is stored in the originating data transport station (A0) in response to the master clock signal (CKA).

(44) Returning now to the topography of the first level permutated ring network 11, each of the communication node CN.sub.1, compute engines CE.sub.1A, CE.sub.1B, CE.sub.1C and CE.sub.1D and memory banks M.sub.1A, M.sub.1B, M.sub.1C and M.sub.1D is coupled to a unique one of the data transport stations A0-A8, B0-B8, C0-C8 and D0-D8 in each of the four communication channels 401-404. For example, compute engine CE.sub.1A is connected to data transport station A0 in communication channel 401, data transport station B8 in communication channel 402, data transport station C7 in communication channel 403 and data transport station D6 in communication channel 404. Table 1 below defines the connections between communication node CN.sub.1, compute engines CE.sub.1A, CE.sub.1B, CE.sub.1C and CE.sub.1D and memory banks M.sub.1A, M.sub.1B, M.sub.1C and M.sub.1D, and the data transport stations A0-A8, B0-B8, C0-C8 and D0-D8 in accordance with one embodiment. Note that the physical connections between communication node CN.sub.1, compute engines CE.sub.1A, CE.sub.1B, CE.sub.1C and CE.sub.1D and memory banks M.sub.1A, M.sub.1B, M.sub.1C and M.sub.1D, and the data transport stations A0-A8, B0-B8 and C0-C8 are not explicitly shown in FIG. 4 for clarity.

(45) TABLE-US-00001 TABLE 1 DATA DATA DATA DATA TRANS- TRANS- TRANS- TRANS- PORT PORT PORT PORT STATION STATION STATION STATION IN COMM IN COMM IN COMM IN COMM CHANNEL CHANNEL CHANNEL CHANNEL NODE 401 402 403 404 CE.sub.1A A0 B8 C7 D6 CE.sub.1B A8 B1 C2 D3 CE.sub.1C A2 B7 C0 D4 CE.sub.1D A4 B6 C6 D2 M.sub.1A A1 B4 C3 D1 M.sub.1B A3 B0 C4 D0 M.sub.1C A B2 C8 D8 M.sub.1D A6 B5 C1 D7 CN1 A7 B3 C5 D5

(46) FIG. 5 re-orders the data of Table 1 to provide an interconnect matrix 500 of the four communication channels 401-404, wherein the interconnect matrix 500 is ordered by the data transport stations in each of the communication channels 401-404. This interconnect matrix 500 makes it easy to determine the number of hops between communication node CN.sub.1, compute engines CE.sub.1A, CE.sub.1B, CE.sub.1C and CE.sub.1D and memory banks M.sub.1A, M.sub.1B, M.sub.1C and M.sub.1D, on each of the communication channels 401-404. Note that communication node CN.sub.1, compute engines CE.sub.1A, CE.sub.1B, CE.sub.1C and CE.sub.1D and memory banks M.sub.1A, M.sub.1B, M.sub.1C and M.sub.1D, are coupled to data transport stations having different relative positions in the four communication channels 401-404. As described in more detail below, this configuration allows for the versatile and efficient routing of messages between the communication nodes.

(47) FIG. 6 is a routing table 600, which defines the flow of traffic among communication node CN.sub.1, compute engines CE.sub.1A, CE.sub.1B, CE.sub.1C and CE.sub.1D and memory banks M.sub.1A, M.sub.1B, M.sub.1C and M.sub.1D, through the permutated ring network 11 in accordance with the present embodiment. For example, communication node CN.sub.1 and compute engine CE.sub.1A communicate using the path between data transport stations D5 and D6 on communication channel 404. The number of hops along this path is defined by the number of segments traversed on the communication channel 404. Because data transport stations D5 and D6 are adjacent to one another on communication channel 404 (i.e., one segment exists between data transport stations D5 and D6), the communication path between communication node CN.sub.1 and compute engine CE.sub.1A consists of one hop (1H).

(48) As illustrated by routing table 600, all of the relevant communication paths between communication node CN.sub.1, compute engines CE.sub.1A, CE.sub.1B, CE.sub.1C and CE.sub.1D and memory banks M.sub.1A, M.sub.1B, M.sub.1C and M.sub.1D include unique one hop communication paths. In other embodiments, one or more of the communication paths may include more than one hop. In yet other embodiments, multiple communication paths may be provided between one or more pairs of communication node CN.sub.1, compute engines CE.sub.1A, CE.sub.1B, CE.sub.1C and CE.sub.1D and memory banks M.sub.1A, M.sub.1B, M.sub.1C and M.sub.1D. In other embodiments, different pairs of communication nodes can share the same communication paths.

(49) Communication among the data transport stations A0-A8, B0-B8, C0-C8 and D0-D8 will operate at the highest frequency allowed by the source synchronous network. This frequency is not reduced as the number of communication nodes and the number of communication channels scale up. It is understood that each of the communication channels 401-404 includes provisions for initialization, arbitration, flow control and error handling. In one embodiment, these provisions are provided using well established techniques.

(50) Each of compute engines CE.sub.1A, CE.sub.1B, CE.sub.1C and CE.sub.1D and memory banks M.sub.1A, M.sub.1B, M.sub.1C and M.sub.1D transmits messages (which may include data) on permutated ring network 11 in accordance with the routing table 600. For example, compute engine CE.sub.1A may transmit a data request message to memory bank M.sub.1C using communication channel 404. More specifically, compute engine CE.sub.1A may transmit a data request message to the clockwise transmit path of data transport station C7. This data request message addresses data transport station C8 and memory bank M.sub.1C. Upon receiving the data request message, data transport station C8 determines that the data request message addresses memory bank M.sub.1C, and forwards the data request message to memory bank M.sub.1C. After processing the data request message, memory bank M.sub.1C may transmit a data response message to the counterclockwise transmit path of data transport station C8. This data response message addresses data transport station C7 and compute engine CE.sub.1A. Upon receiving the data response message, data transport station C7 determines that the data response message addresses compute engine CE.sub.1A, and forwards the data response message to compute engine CE.sub.1A.

(51) Messages can be transmitted into and out of permutated ring network 11 through communication node CN.sub.1. For example, compute engine CE.sub.1A of slice 1 may transmit a data request message to memory bank M.sub.2A of computing slice 2 using communication channel 404. More specifically, compute engine CE.sub.1A may transmit a data request message to the counterclockwise transmit path of data transport station D6. This data request message addresses data transport station D5 and communication node CN.sub.1 (as well as communication node CN.sub.2 of computing slice 2 and memory bank M.sub.2A within computing slice 2). Upon receiving the data request message, data transport station D5 determines that the data request message addresses communication node CN.sub.1, and forwards the data request message to communication node CN.sub.1. In response, communication node CN.sub.1 determines that the data request message addresses communication node CN.sub.2 within computing slice 2, and forwards the data request message on second level PRN interconnect 21 (using a routing table implemented by second level PRN interconnect 21). Note that second level PRN interconnect 21 uses a PRN structure similar to first level PRN interconnect 11 to route messages among communication nodes CN.sub.1-CN.sub.6. Note that the implementation of second level PRN interconnect 21 may be different than the implementation of first level PRN interconnect 11 (e.g., different number of communication channels, different routing table), due to the different number of communication nodes serviced by the second level PRN interconnect 21. In accordance with one embodiment, the second level PRN-based interconnect structure 21 includes three communication channels (i.e., three bi-directional ring networks), wherein each communication channel includes six data transport stations. In this embodiment, each of the communication nodes CN.sub.1-CN.sub.6 is coupled to a corresponding one of the data transport stations in each of the three communication channels.

(52) The data transport station associated with communication node CN.sub.2 receives the data request message transmitted on second level PRN interconnect 21, and determines that the data request message addresses communication node CN.sub.2, and forwards the data request message to communication node CN.sub.2. In response, communication node CN.sub.2 determines that the data request message addresses memory bank M.sub.2A within computing slice 2, and forwards the data request message on the first level PRN interconnect 12 (using the routing table implemented by the first level PRN interconnect 12). Note that the first level PRN interconnect 12 uses a PRN structure similar to first level PRN interconnect 11 to route messages among communication nodes CN.sub.2, compute engines CE.sub.2A, CE.sub.2B, CE.sub.2C, CE.sub.2D (of compute engine set CE.sub.2), and memory banks M.sub.2A, M.sub.2B, M.sub.2C and M.sub.2D (of memory bank set M.sub.2).

(53) The data transport station associated with memory bank M.sub.2A receives the data request message transmitted on first level PRN interconnect 12, and determines that the data request message addresses memory bank M.sub.2A, and forwards the data request message to memory bank M.sub.2A. Memory bank M.sub.2A may then respond to the data request message. For example, memory bank M.sub.2A may retrieve a stored data value and return this data value to compute engine C.sub.1A using a data response message. This data response message is transmitted to compute engine C.sub.1A using the reverse path of the original data request message.

(54) In accordance with one embodiment, the third level PRN-based interconnect structure 31 includes three communication channels (i.e., three bi-directional ring networks), wherein each communication channel includes six data transport stations. In this embodiment, each of the communication nodes CN.sub.11-CN1.sub.6 is coupled to a corresponding one of the data transport stations in each of the three communication channels.

(55) Using the above-described flat computer architecture and messaging system, messages can be transmitted between any of the various elements of computer system 200 via the first, second and third level PRN interconnect structures, without requiring a change in the messaging protocol. In accordance with one embodiment, each of the elements of computer system 200 is assigned a unique (system) address. Address mapping the various elements of the system 200 in this manner allows these elements to be consistently accessed across the first, second and third level PRN interconnect structures. Note that the computer system 200 is a non-coherent system, because this computer system 200 does not explicitly ensure the coherency of data stored by the memory banks within the computing slices, DRAM 211-214 or system memory 206. Instead, the user is required to control the data stored by these memories in the desired manner. Computer system 200 is therefore well suited to implement a Producer-Consumer execution model, such as that implemented by the forward propagation of a neural network. That is, computer system 200 is able to efficiently process data in neural network/machine learning applications. The improved network topology of computer system 200 is advantageously able to span multiple chips, without requiring cache coherency protocol between the multiple chips. Computer system 200 is therefore easily scalable, and capable of providing communication between many different chips.

(56) In the embodiments described above, the first, second and third level interconnect structures 11, 21 and 31 are all implemented using bi-directional source synchronous permutated ring networks. However, in an alternate embodiment of the present invention, the first level interconnect structures can be implemented using a non-PRN based structure.

(57) FIG. 7 is a block diagram of a computer system 700 in accordance with an alternate embodiment of the present invention. Because computer system 700 is similar to computer system 200, similar elements in FIGS. 7 and 2 are labeled with similar reference numbers. Thus, computer system 700 includes a plurality of processor chips 701-704, host processor system 205, system memory 206, system level interconnect chip 210 and DRAM devices 211-214. Although only four processor chips 701-704 are illustrated in FIG. 7, it is understood that computer system 700 can be easily modified to include other numbers of processor chips in other embodiments. Moreover, although only processor chip 701 is illustrated in detail in FIG. 7, it is understood that processor chips 702-704 include the same (or similar) internal elements as the processor chip 701 in the described embodiments. As described in more detail below, processor chip 701 replaces the first level PRN-based interconnect structures 11-14 of processor chip 201 with simple network interconnect structures 711-714. Simple network interconnect structures 711-714 can be, for example, crossbar switch-based interconnect structures, or simple ring networks.

(58) In the illustrated embodiment, processor chip 701 includes four computing slices 71, 72, 73, and 74, which are coupled to second level permutated ring network interconnect structure 21. Although four computing slices are illustrated in FIG. 7, it is understood that other numbers of computing slices can be included on processor chip 701 in other embodiments. Each computing slice includes a plurality of compute engines, a plurality of memory banks, a communication node and a simple network interconnect structure. More specifically, slices 71, 72, 73 and 74 include compute engine sets CE.sub.1, CE.sub.2, CE.sub.3 and CE.sub.4, respectively, memory bank sets M.sub.1, M.sub.2, M.sub.3 and M.sub.4, respectively, simple network interconnect structures 711, 712, 713 and 714, respectively, and communication nodes CN.sub.1, CN.sub.2, CN.sub.3 and CN.sub.4, respectively. Compute engine sets CE.sub.1, CE.sub.2, CE.sub.3 and CE.sub.4 and memory bank sets M.sub.1, M.sub.2, M.sub.3 and M.sub.4 are described in more detail above in connection with FIGS. 2 and 3.

(59) Within each of the slices 71, 72, 73 and 74, the corresponding simple network interconnect structures 711, 712, 713 and 714 couple the corresponding compute engine sets CE.sub.1, CE.sub.2, CE.sub.3 and CE.sub.4 and the corresponding memory bank sets M.sub.2, M.sub.2, M.sub.3 and M.sub.4. This allows each of the compute engines to access each of the memory banks within the same slice using the corresponding simple network.

(60) The simple network interconnect structures 711, 712, 713 and 714, are also coupled to corresponding communication nodes CN.sub.1, CN.sub.2, CN.sub.3 and CN.sub.4 within the corresponding computing slices 71, 72, 73 and 74. The communication nodes CN.sub.1, CN.sub.2, CN.sub.3 and CN.sub.4 are coupled to the second level PRN-based interconnect structure 21 in the manner described above. The communication nodes CN.sub.1, CN.sub.2, CN.sub.3 and CN.sub.4 pass messages and data between the corresponding simple network interconnect structures 711, 712, 713 and 714 and the second level PRN-based interconnect structure 21. Note that messages transmitted between the simple network interconnect structures 711, 712, 713 and 714 and the corresponding communication nodes CN.sub.1, CN.sub.2, CN.sub.3 and CN.sub.4 must be converted to a protocol consistent with the receiving system. Such conversions may be implemented by an interface within simple network interconnect structures 711-714, or an interface within communication nodes CN.sub.1, CN.sub.2, CN.sub.3 and CN.sub.4. While this protocol conversion complicates the operation of computer system 700, it allows the use of simple network interconnect structures within each computing slice, which may reduce the required layout area of the computing slices 71-74.

(61) In another embodiment of the present invention, the third level PRN based interconnect structure 31 is replaced with a simple network interconnect structure, such as a crossbar switch based interconnect structure or a simple ring network (in the same manner that the first level PRN-based structures 11-14 are replaced by simple network structures 711-714 in FIG. 7 above). FIG. 8 is a block diagram of a computer system 800 in accordance with this alternate embodiment, which replaces the third level PRN based interconnect structure 31 with a simple network interconnect structure 81 on a system level interconnect chip 810 in the manner suggested above. The simple network interconnect structure 81, which can include, for example, a crossbar switch based interconnect structure or a simple ring network, provides connections between communication nodes CN.sub.11-CN.sub.16. Note that messages transmitted between processor chips 701-704, host processor system 205 and system memory 206 and the corresponding communication nodes CN.sub.11, CN.sub.12, CN.sub.13, CN.sub.14, CN.sub.15 and CN.sub.16 must be converted to protocols consistent with the receiving systems. Such conversions may be implemented by and interface within simple network interconnect structure 81, or interfaces within communication nodes CN.sub.11-CN.sub.16. While this protocol conversion complicates the operation of computer system 800, it allows the use of simple network interconnect structures within system level interconnect chip 810.

(62) Although the simple network interconnect structure 81 of system level interconnect chip 810 is shown in combination with computing slices 71-74 having simple network interconnect structures 711-713, it is understood that the simple network interconnect structure 81 of system level interconnect chip 810 can also be used in combination with computing slices 1-4 having first level PRN-based interconnect structures 11-14, as illustrated by FIG. 2.

(63) Several factors can be used to determine whether the first and third level interconnect structures should be implemented with a bi-directional source synchronous permutated ring networks (FIGS. 2A-2B) or simple network interconnect structures, such a crossbar switches or single ring networks (FIGS. 7-8). Permutated ring networks will provide better performance (but require a larger layout area) than a simple single ring network. Permutated ring networks will also typically provide better performance (and may require a larger layout area) than a crossbar switch. In general, as more communication nodes are connected by the interconnect structure, it becomes more efficient (in terms of layout area and performance) to use permutated ring networks instead of single ring networks or crossbar switches. In accordance with one embodiment, permutated ring networks are used when the number of communication nodes to be connected is four or greater.

(64) Although the invention has been described in connection with several embodiments, it is understood that this invention is not limited to the embodiments disclosed, but is capable of various modifications, which would be apparent to a person skilled in the art. Accordingly, the present invention is limited only by the following claims.