INTEGRATED RACK ARCHITECTURE FOR DISTRIBUTING TWO PHASE COOLANT

Abstract

An electronic rack includes condensing, coolant distribution, and server regions. The condensing region includes a condensing container housing condensing coils and a coolant container to contain two phase coolant. The coolant distribution region includes a set of rack manifolds having at least a rack liquid supply line to receive coolant from the coolant distribution region, and a vapor line to return vapor to the coolant distribution region, a liquid return line. The server region is coupled to the condensing region and the coolant distribution region, the server region includes a number of server slots to receive a number of servers, where each of the servers is at least partially submerged within two phase liquid coolant, where, when the servers operate, the servers generate heat that is extracted by the two phase liquid coolant thereby causing at least some of the two phase liquid coolant to turn into a vapor.

Claims

1. An electronic rack of a data center, comprising: a condensing region comprising: a condensing container housing condensing coils, and a coolant container situated at a height lower than the condensing container to contain two phase coolant; a coolant distribution region, comprising: a set of rack manifolds having at least a rack liquid supply line to receive a two phase coolant from the coolant distribution region, and a vapor line to return a vapor to the coolant distribution region; and a server region coupled to the condensing region and the coolant distribution region, the server region comprising a plurality of server slots to receive a plurality of servers, wherein each of the servers is at least partially submerged within two phase liquid coolant, wherein, when the servers operate, the servers generate heat that is transferred to the two phase liquid coolant thereby causing at least some of the two phase liquid coolant to turn into a vapor, wherein the server, coolant distribution, and condensing regions are integrated within the electronic rack.

2. The electronic rack of claim 1, wherein the set of rack manifolds further comprise a rack liquid return line to return two phase coolant back to the condensing region.

3. The electronic rack of claim 1, wherein the coolant container is directly connected to the condensing container of the condensing region.

4. The electronic rack of claim 1, wherein a rack liquid supply line supplies a two phase liquid coolant from the coolant container to the plurality of servers, via a charging pump.

5. The electronic rack of claim 1, wherein a rack liquid return line returns a two phase liquid coolant from the plurality of servers to the coolant container via a return pump.

6. The electronic rack of claim 1, wherein each of the plurality of servers is housed within a corresponding immersion housing unit.

7. The electronic rack of claim 1, wherein the set of rack manifolds are fixed on the electronic rack, wherein the server region includes an adjustable server populating structure, wherein the adjustable server populating structure is adjustable for a server to connect to the set of rack manifolds.

8. The electronic rack of claim 1, wherein the condensing coils are coupled to ceiling mounted facility liquid supply/return lines situated above the electronic rack.

9. The electronic rack of claim 1, wherein the coolant container is coupled to a facility coolant supply line and a facility coolant return line situated beneath the electronic rack.

10. The electronic rack of claim 9, wherein the facility coolant supply line is connected to an inlet of a return pump and the facility coolant return line is connected to an outlet of a charging pump.

11. A data center system, comprising: an array of electronic racks, wherein each electronic racks comprises: a condensing region comprising: a condensing container housing condensing coils, and a coolant container situated at a height lower than the condensing container to contain two phase coolant; a coolant distribution region, comprising: a set of rack manifolds having at least a rack liquid supply line to receive a two phase coolant from the coolant distribution region, and a vapor line to return a vapor to the coolant distribution region; and a server region coupled to the condensing region and the coolant distribution region, the server region comprising a plurality of server slots to receive a plurality of servers, wherein each of the servers is at least partially submerged within two phase liquid coolant, wherein, when the servers operate, the servers generate heat that is transferred to the two phase liquid coolant thereby causing at least some of the two phase liquid coolant to turn into a vapor, wherein the server, coolant distribution, and condensing regions are integrated within the electronic rack.

12. The data center system of claim 11, wherein the set of rack manifolds further comprise a rack liquid return line to return two phase coolant back to the condensing region.

13. The data center system of claim 11, wherein the coolant container is directly connected to the condensing container of the condensing region.

14. The data center system of claim 11, wherein a rack liquid supply line supplies a two phase liquid coolant from the coolant container to the plurality of servers, via a charging pump.

15. The data center system of claim 11, wherein a rack liquid return line returns a two phase liquid coolant from the plurality of servers to the coolant container via a return pump.

16. The data center system of claim 11, wherein each of the plurality of servers is housed within a corresponding immersion housing unit.

17. The data center system of claim 11, wherein the set of rack manifolds are fixed on the electronic rack, wherein the server region includes an adjustable server populating structure, wherein the adjustable server populating structure is adjustable for a server to connect to the set of rack manifolds.

18. The data center system of claim 11, wherein the condensing coils are coupled to ceiling mounted facility liquid supply/return lines situated above the electronic rack.

19. The data center system of claim 11, wherein the coolant container is coupled to a facility coolant supply line and a facility coolant return line situated beneath the electronic rack.

20. The data center system of claim 19, wherein the facility coolant supply line is connected to an inlet of a return pump and the facility coolant return line is connected to an outlet of a charging pump.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Embodiments of the invention are illustrated by way of example and not limitation in the figures of the accompanying drawings in which like references indicate similar elements.

[0007] FIG. 1 is a block diagram illustrating a front view of a cooling system with one fluid loop according to one embodiment.

[0008] FIG. 2 is a block diagram illustrating a front view of a cooling system with two fluid loops according to one embodiment.

[0009] FIG. 3 is a block diagram illustrating a server populating structure of a cooling system according to one embodiment.

[0010] FIG. 4 is a block diagram illustrating a data center with a plurality of cooling systems according to one embodiment.

[0011] FIG. 5 is a block diagram illustrating a data center with a plurality of cooling systems according to one embodiment.

DETAILED DESCRIPTION

[0012] Various embodiments and aspects of the inventions will be described with reference to details discussed below, and the accompanying drawings will illustrate the various embodiments. The following description and drawings are illustrative of the invention and are not to be construed as limiting the invention. Numerous specific details are described to provide a thorough understanding of various embodiments of the present invention. However, in certain instances, well-known or conventional details are not described in order to provide a concise discussion of embodiments of the present inventions.

[0013] Reference in the specification to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in conjunction with the embodiment can be included in at least one embodiment of the invention. The appearances of the phrase “in one embodiment” in various places in the specification do not necessarily all refer to the same embodiment.

[0014] The current solution includes a cooling system of integrated electronic racks to distribute two phase coolant to servers populated at the racks, the solution having facility level and rack level co-designs. As further described in FIGS. 1-5, the solution can improve the design and deployment efficiency of high power density compute clusters and data centers.

[0015] According to a first aspect, an electronic rack of a data center includes a condensing region that includes a condensing container housing condensing coils and a coolant container situated at a height lower than the condensing container to contain two phase coolant. The condensing region in this specification can be understood as the region where vapor condensation is located and liquid coolant are collected. The electronic rack includes a coolant distribution region. The coolant distribution region includes a set of rack manifolds having at least a rack liquid supply line to receive a two phase coolant from the coolant distribution region and a vapor line to return a vapor to the coolant distribution region.

[0016] The electronic rack includes a server region coupled to the condensing region and the coolant distribution region, the server region includes a number of server slots to receive a number of servers, where each of the servers is at least partially submerged within two phase liquid coolant, where, when the servers operate, the servers generate heat that is transferred to the two phase liquid coolant thereby causing at least some of the two phase liquid coolant to turn into a vapor. The server, coolant distribution, and condensing regions are integrated within the electronic rack.

[0017] The rack manifolds further include a rack liquid return line to return two phase coolant back to the condensing region. The coolant container is directly connected to the condensing container of the condensing region.

[0018] The rack liquid supply line supplies a two phase liquid coolant from the coolant container to the plurality of servers, via a charging pump. The rack liquid return line returns a two phase liquid coolant from the plurality of servers to the coolant container via a return pump

[0019] Each of the plurality of servers is housed within a corresponding immersion housing unit. The rack manifolds are fixed on the electronic rack, wherein the server region includes an adjustable server populating structure, wherein the adjustable server populating structure is adjustable for a server to connect to the rack manifolds.

[0020] The condensing coils are coupled to ceiling mounted facility liquid supply/return lines situated above the electronic rack. The coolant container is coupled to facility coolant supply/return lines situated beneath the electronic rack. The coolant supply/return lines are situated beneath a facility floor. In one embodiment, the coolant supply and return lines are designed on the surface of the facility floor.

[0021] FIG. 1 is a block diagram illustrating a front view of a cooling system 100 with one coolant loop according to one embodiment. Cooling system 100 can include an electronic rack 102 populated with servers 101A-101E and a set of coolant manifolds (or lines) 111A-111C. The servers 101A-101E can be situated in a server region, and the set of coolant manifolds 111A-111C can be situated in a coolant distribution region of electronic rack 102.

[0022] Servers 101A-101E can be configured to provide IT services. Specifically, servers 101A-101E may include a host server (referred to as a host node) and/or one or more compute servers (also referred to as computing nodes, such as CPU server and GPU server). The host server (having one or more CPUs) typically interfaces with clients (not shown in FIG. 1) over a network (e.g., Internet) to receive a request for a particular service such as storage services (e.g., cloud-based storage services such as backup and/or restoration), executing an application to perform certain operations (e.g., image processing, deep data learning algorithms or modeling, etc., as a part of a software-as-a-service or SaaS platform). In response to the request, the host server distributes the tasks to one or more of the performance computing nodes or compute servers (having one or more GPUs) managed by the host server. In one embodiment, servers 101A-101E can perform any type of computing task and/or can be any type of computing device (e.g., a server, a storage device, etc.). In one embodiment, servers 101A-101E can be edge computing devices. Thus, while servers 101A-101E provide the IT services, each of servers 101A-101E generates heat that is transferred to two phase coolant.

[0023] In one embodiment, coolant/rack manifolds (or lines) 111A-111C can include a liquid supply line 111C, a vapor return line 111B, and a liquid return line 111A. In one embodiment, each of servers 101A-101E include a set of liquid inlet, liquid outlet, and vapor outlet connectors (not shown). In one embodiment, each of manifolds 111A-111C include a set of connectors 121-125 assembled thereon. Liquid return line 111A can return a two phase liquid coolant from servers 101A-101E to secondary container 109 (e.g., via connectors 121A-121E). Liquid supply line 111C can supply a two phase liquid coolant to servers 101A-101E (e.g., via connectors 125A-125E). In one embodiment, the vapor return line 111B is situated in between liquid supply line 111C and liquid return line 111A. Vapor return line 111B can return a two phase coolant in vapor phase to fluid container 105 (e.g., via connectors 123A-123E). In one embodiment, the connectors are dripless blind-mating ones.

[0024] In one embodiment, each of servers 101A-101E can include an immersion module housing the server therein. The servers can include electronics which can generate heat during operation. The electronics can be submerged in the immersion cooling fluid within the server. The immersion module can include a set of ports, such as a two phase supply port to interface with liquid supply line 111C, a two phase coolant release port to interface with liquid return line 111A, and a vapor release port to interface with vapor return line 111B. Although three ports are described, in some embodiments, servers 101A-101E include only a two phase supply port and a vapor release port.

[0025] In one embodiment, electronic rack 102 can include a condensing region. The condensing region can include fluid container 105 and condensing coils 103 situated above fluid container 105. Fluid container 105 can receive two phase coolant in vapor phase from vapor manifold 111B via line 106. Condensing coils 103 can receive a supply of liquid cooling fluid (such as water) from an external source, via lines 115, to condense two phase coolant from vapor phase to liquid phase. A secondary container 109 can be situated beneath fluid container 105. Fluid container 105 and secondary container 109 can be connected directly via line 107.

[0026] In one embodiment, secondary container 109 is connected to liquid supply line 111C, via line 113. In one embodiment, line 113 includes an active pump to facilitate liquid distribution from secondary container 109 to servers 101A-101E. In one embodiment, liquid return line 111A is connected to secondary container 109 via line 117. In one embodiment, line 117 includes a release pump to actively pump two phase liquid coolant from servers 101A-101E to secondary pump 109. In one embodiment, server, condensing, and coolant distribution regions are integrated within electronic rack 102 and/or is fabricated and co-design as part of electronic rack 102.

[0027] FIG. 2 is a block diagram illustrating a front view of a cooling system 200 with two fluid loops according to one embodiment. Racks 202 can be similar to racks 102 of FIG. 1, except for the additional line 207. For example, fluid container 105 can be directly connecting to fluid supply line 111C, via line 207. Since fluid container 105 and fluid supply line 112C can be co-designed, therefore, line 207 can be prefabricated or assembled on rack 202 before the servers are being deployed.

[0028] FIG. 3 is a block diagram illustrating a server populating structure of a cooling system 200 according to one embodiment. In one embodiment, cooling system 200 includes one or more server populating structures 301-303. Although shown as rack 202, cooling system 200 can include rack 102 of FIG. 1. In one embodiment, server populating structures 301-303 are adjustable modular units, which can be adjusted, removed, and/or inserted as a placeholder for server slots. Thus, server populating structures 301-303 can form one or more server slots to support servers that are contained therein. The adjustable features of the server populating structures 301-303 can adjust a height of the server slots, such that, servers, when populated, can be connected to manifolds 111A-111C.

[0029] In one embodiment, on each of manifolds 111A-111C, multiple connectors 121-125 may be assembled thereto.

[0030] The server populating structures 301-303 also enables the connectors of servers being populated to the rack 202 to match a height of the individual corresponding manifold connectors more accurately and, thus, connectors on the servers and connectors of the manifolds can be mated.

[0031] FIG. 4 is a block diagram illustrating a data center system 400 with a plurality of cooling systems according to one embodiment. In one embodiment, data center system 400 can be a heterogeneous cluster including an array of electronic racks, each having a number of server nodes. A heterogeneous cluster environment can contain processors and devices with different bandwidth and computational capabilities. Electronic racks 102A-102C can be racks 102 of FIG. 1 or racks 202 of FIG. 2. It can be seen that the system includes a cooling fluid recirculation loop 401 and a two phase coolant fluid distribution loop 411. In one embodiment, cooling fluid recirculation loop 401 includes a facility liquid supply line 403 and a facility liquid return line 405. The supply and returns lines 403-405 can be connected to condensing coils of each of the racks, via lines 115A-115C. In one embodiment, two phase coolant fluid distribution loop 411 can be connected to secondary containers 109A-109C of respective racks to interface individual racks to a facility level two phase coolant distribution. Two phase coolant fluid distribution loop 411 can include facility coolant supply line 413 and facility coolant return line 415. Facility coolant supply line 413 can be connected to supply lines 113A-113C and facility coolant return line 415 can be connected to liquid return lines 117A-117C of the racks.

[0032] In one embodiment, cooling fluid recirculation loop 401 can be mounted to a ceiling of data center facility. In one embodiment, two phase coolant fluid distribution loop 411 can be implemented beneath a facility flooring. In an embodiment, two phase coolant fluid distribution loop 411 can be directly implemented on a surface of the facility flooring.

[0033] FIG. 5 is a block diagram illustrating a data center 500 with a plurality of cooling systems according to one embodiment. Data center 500 is similar to data center 400 of FIG. 4, except that two phase coolant fluid distribution loop 411 is connected to secondary containers 109A-109C directly, via one or more bi-directional lines or supply/return lines 501. In one embodiment, bi-directional lines or supply/return lines 501 can include valves (not shown) to control a flow to each of the racks. In this implementation, two phase coolant fluid can be stored at a facility pool (not shown), and be supplied to the racks or withdrawn from the racks, thus, increasing the data center system resilience.

[0034] In the foregoing specification, embodiments of the invention have been described with reference to specific exemplary embodiments thereof. It will be evident that various modifications may be made thereto without departing from the broader spirit and scope of the invention as set forth in the following claims. The specification and drawings are, accordingly, to be regarded in an illustrative sense rather than a restrictive sense.