Abstract
A novel method and apparatus are described that can be used to equalize the latency in fiber optic distribution links within data centers containing multiple pods (clusters of servers) and thereby improve the overall operation and utility of the data center for multiple customers. Specifically, the apparatus serves to add precisely measured latency (signal delays) to data transmission in certain fiber optic cable links so that there are negligible differences in signal transmission times from the central switch (core router) to each of the distributed pods within a data center. While that purposeful addition of latency may, at first, seem counterintuitive to optimizing the performance of a data center, the effect achieved is quite the opposite. That is because all pods will have equal access to received and transmitted data thereby reducing signal congestion and the unbalanced time favoritism of one pod operator over another to the access incoming data.
Claims
1. A multilink equalizer apparatus suitable for equalizing link latency (time delay) in a group of fiber optic data links within a data center wherein the said multilink equalizer apparatus is comprised of a multiplicity of equalizer modules that are inserted into a single equipment rack mounted chassis and each of said modules contains a multiplicity of spools wound with optical fibers of varying lengths that have been precisely cut to provide various data transmission delay times (latencies) necessary to equalize the data transmission delays in said group of optical fiber data links.
2. A multilink equalizer apparatus as in claim 1 suitable for mounting in a standard 19 inch wide equipment rack space in a data center that is 3 RU (5.25 inches) high.
3. A multilink equalizer apparatus as in claim 1 containing a multiplicity of spools that have been wound with optical fibers that have been cut to various predetermined lengths of up to 800 meters for each spool.
4. A multilink equalizer apparatus as in claim 3 wherein the said predetermined fiber lengths are precise to within 10 centimeters of the nominal specified length.
5. A multilink equalizer apparatus as in claim 1 containing spools that hold wound optical fibers that are manufactured by 3D printing, such as Fused Deposition Modeling (FDM) or injection molded processes.
6. A multilink equalizer apparatus as in claim 5 wherein the polymer material, such as polyethylene terephthalate (PETG), is used to form the spools.
7. A method for equalizing latency in a group optical fiber data links within a data center employing the following steps: (1) measuring the data transmission latency (time delay) in each optical fiber within a group that has been selected for equalization using an optical time domain reflectometer (OTDR) or some other suitable equipment, (2) determining the optical fiber link in the selected group that has the maximum latency, (3) subtract the latency for each of the other optical fiber links in the group from the one with the maximum latency to determine the latency difference that must be added to each link for the purpose of equalization, (4) cut a separate length of optical fiber cable having a length that corresponds to the latency difference for each optical fiber link in the group, (5) wind each of these fibers onto a spool and mark the spool with the corresponding latency value, (6) insert all of these spools into a single equalizer module or multiple equalizer modules that are subsequently inserted into the equipment rack of a single multilink equalizer apparatus, (7) connect the cable input ports in the single multilink equalizer apparatus to the core switch in the data center using a group of equal length fiber optic jumper cables, (8) connect the cable output ports of the multilink equalizer apparatus to the corresponding fiber optical data links that are to be equalized so that the time delays (latency) in all links become equal.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0021] The above SUMMARY OF THE INVENTION as well as other features and advantages of the present invention will be more fully appreciated by reference to the following detailed descriptions of illustrative embodiments in accordance with the present invention when taken in conjunction with the accompanying drawings, wherein:
[0022] FIG. 1 is a schematic of a typical data center with pods having varying cable lengths from the core switch.
[0023] FIG. 2 is similar to FIG. 1 with one exception; all of the optical fiber link lengths between the central switch and the pods have been cut to have equal length.
[0024] FIG. 3 is similar to FIG. 1 with one exception; all of the optical fiber links between the central switch and the pods have been made to all have equal latency by the addition of varying amounts of latency at various locations within each link.
[0025] FIG. 4 is similar to FIG. 1 with one exception; all of the optical fiber links between the central switch and the pods have been made to have equal latency by the addition of a specialized piece of equipment known as a multilink equalizer apparatus adjacent to the core switch that adds selected amounts of latency to each of the outgoing links.
[0026] FIG. 5 is an isometric view of the multilink equalizer apparatus shown in FIG. 4.
[0027] FIG. 6 is an isometric view of the interior of a single equalizer module that can be inserted into the multilink equalizer apparatus.
[0028] FIG. 7 is an isometric view of one of the twelve spools that can be contained in each module shown in FIG. 6.
DETAILED DESCRIPTION OF THE DRAWINGS
[0029] With reference to the attached drawings, embodiments of the present invention will be described in the following:
[0030] FIG. 1 is a schematic of a typical data center 1 contained within a large dashed box 1. A multiplicity of external data cables, 2(a), 2(b), etc. (implying more similar data cables may exist in a typical data center) connect to carry data to and from a multiplicity of edge routers 3(a), 3, (b), etc. (implying more similar edge routers may exist in a typical data center) which, in turn, direct the data to and from an aggregated group of routers called the core switch 4 through cables 2(c), and 2(d), etc. (implying more similar data cables may exist in a typical data center). From there, the data is directed to and from dispersed data processing pods (sometimes referred to as modules, containers, or clusters) 5(a), 5(b), 5(c), 5(d), 5(e), 5(f), etc. (implying more similar data processing pods may exist in a typical data center) through fiber optic cable links 6(a), 6(b), 6(c), 6(d), 6(e), 6(f), etc. (implying more similar fiber optic cable links may exist in a typical data center). Some larger data centers may contain a multiplicity of core switches similar to core switch 4 that are linked to other groups of pods similar to the group 5(a), 5(b), 5(c), 5(d), 5(e), 5(f), etc. shown in FIG. 1
[0031] FIG. 2 is a schematic, similar to FIG. 1 with one exception; all of the optical fiber links 7(a), 7(b), 7(c), 7(d), 7(e),7(f), etc. (implying more similar optical fiber links may exist in a typical data center) between the core switch 4 and the pods 5(a), 5(b), 5(c), 5(d), 5(e), 5(f), etc. have been made to have equal lengths. Specifically, the optical fibers within these links have all been cut to the equal lengths prior to installation. This ensures that the latency associated with each of the said optical fiber links is the same. When employing this cabling design strategy in a data center, some of the optical fiber links 7(a), 7(b), 7(c), 7(d), 7(e), 7(f), etc. may be physically longer that the links 6(a), 6(b), 6(c), 6(d), 6(e), 6(f), etc. shown in FIG. 1. In this case, any excess cable lengths may be coiled as shown in locations 7C(a), 7C(c), 7C(d), 7C(e), 7C(f), etc. or simply bunched together and stored in cable troughs located within the data center. Longer excess cable lengths may be wrapped around the pod to which the cable link is directed or stored or placed in any other convenient location.
[0032] Cabling is an important aspect of data center design. Poor cable deployment can be more than just messy to look at—it can restrict airflow, preventing hot air from being expelled properly and blocking cool air from coming in. Over time, cable-related air damming can cause equipment to overheat and fail, resulting in costly downtime. As a consequence, there is an advantage in not having coils or bunches of cables scattered throughout a data center.
[0033] FIG. 3 is similar to FIG. 1 with one exception; all of the optical fiber links 8(a), 8(b), 8(c), 8(d), 8(e),8(f), etc. (implying more similar optical fiber links may exist in a typical data center) between the core switch and the pods 5(a), 5(b), 5(c), 5(d), 5(e), 5(f), etc. have been made to all have equal latency by splicing in additional sections of optical fiber cables 9(a), 9(b), 9(c), 9(d), 9(e),9(f), etc. to the links, as needed, at various locations 10(a), 10(b), 10(c), 10(d), 10(e),10(f), etc. that are accessible. The strategy for determining the lengths of the additional optical fiber cables 9(a), 9(b), 9(c), 9(d), 9(e),9(f), etc. would be to measure the time delay in each of the fiber optic cable links 6(a), 6(b), 6(c), 6(d), 6(e), 6(f), etc. in FIG. 1. Then use this information to calculate, measure and cut additional optical fiber cables 9(a), 9(b), 9(c), 9(d), 9(e),9(f), etc. (implying more similar optical fiber cables may exist in a typical data center) so that the time delay in each of the resulting spliced cables 8(a), 8(b), 8(c), 8(d), 8(e),8(f), etc. are equal. Measurement of the time delay can be accomplished using a conventional optical time delay reflectometer (OTDR) or some other suitable equipment. While this cabling design strategy equalizes link latency, it suffers from cluttering the data center with optical fiber cables 9(a), 9(b), 9(c), 9(d), 9(e),9(f), etc.
[0034] FIG. 4 is similar to FIG. 1 with one exception; all of the optical fiber links between the core switch 4 and the pods 5(a), 5(b), 5(c), 5(d), 5(e), 5(f), etc. have been made to have equal latency by the addition of a specialized piece of equipment known as a multilink equalizer apparatus 20 that is located adjacent to the core switch 4 and connected to the core switch 4 by a series of equal length fiber optic jumper cables 13(a), 13(b), 13(c), 13(d), 13(e),13(f), etc. (implying more similar fiber optic jumper cables may exist in a typical data center) that connect to the input ports of the multilink equalized apparatus 20. The output ports of the multilink equalized are connected to the outgoing fiber optic data links 12(a), 12(b), 12(c), 12(d), 12(e),12(f), etc. (implying more similar outgoing fiber optic links may exist in a typical data center). This strategy is employed to make the total latency from the core switch 4 to the pods 5(a), 5(b), 5(c), 5(d), 5(e), 5(f), etc. equal with minimum additional space requirements for the storage of cables. The design and operation of the multilink equalizer apparatus is discussed in FIG. 5, FIG. 6 and FIG. 7.
[0035] FIG. 5 is an isometric view of the multilink equalizer apparatus 20 that is shown schematically in FIG. 4. This is a novel product for equalizing link latency and system synchronization in data centers. It offers a space-efficient and scalable approach for data center engineering teams deploying time delays for latency-driven applications that require equalization.
[0036] While the size and configuration of the multilink equalizer apparatus 20 may vary depending on the application, a standard unit has a rack mounted chassis 21 is 3 RU (5¼ inches) high. This multilink equalizer apparatus accommodates up to 12 high-density modules, 22(a) through 22(l), one of which is shown in greater detail in FIG. 6. Each of these modules holds up to 12 fiber delay spools, shown in greater detail in FIG. 7, for a total of 12×12=144 time delays that can be individually specified by a data center engineering team. Each module has a group of 24 fiber optic connector ports 23(a) through 23(l). Twelve these ports are input ports connect to the core switch 4 through jumper cables 13(a), 13(b), 13(c), 13(d), 13(e),13(f), etc. shown in FIG. 4. The remaining 12 fiber optic connector ports are output ports that connect to data links 12(a), 12(b), 12(c), 12(d), 12(e),12(f), etc., also shown in FIG. 4. Having as many as 144 time delays in a single apparatus saves considerable rack space while enabling a data center engineering team to add, re-configure, and completely control their setup configuration as their data center business needs evolve. Furthermore, each time delay can be achieved with sub-nanosecond accuracy by carefully monitoring the length of optical fiber wound on each spool, delivering a superior performance level not seen before in the data center industry.
[0037] FIG. 6 is an isometric view of the interior of a single equalizer module 30 that can be inserted into the multilink equalizer apparatus 20. The module's cover has been removed and is not shown in this figure. While the size of a module can vary depending on the application, the standard module shown in this figure is 1.1 inches wide, 5.25 inches tall and 24.1 inches long. This standard module contains six pairs of spools 31(a), 31(b), 31(c), 31(d), 31 (e), and 31(f). Each pair of spools is mounted on keyed shafts 32(a), 32(b), 32(c), 32(d), 32 (e), and 32(f) that accommodate four different rotational positions for the spools. The lengths of the individual optical fibers are precisely measured as they are wound on the individual spools. And the delay time for each spool is equal to the length of the optical fiber on the spool times the velocity of light in this fiber. For example, a spool containing 100 meters of wound optical fiber would have a delay time of 0.4897 microseconds (100 meters×4.897 nanoseconds per meter).
[0038] FIG. 7 is an isometric view of one of the twelve spools that is contained in each standard module shown in FIG. 6. The standard spool 40, shown in FIG. 7, has end flanges, 41(a) and 41(b) of 3.5 inches in diameter and a central core 42 that is 2.5 inches in diameter. The standard spools are made in two sizes; one with a core width of 0.17 inches that can accommodate up to 250 meters of optical fiber and the other has a larger core width of 0.66 inches that can accommodated up to 500 meters of optical fiber with an outside diameter, including coating, of 250 microns. The two ends, 43(a) and 43(b), of optical fiber, 44, that is wound on the stool 40 can be terminated with fiber optic connectors 44(a) and 44(b). Since it is often easier to fusion splice optical fibers than to terminate them with connectors, a short fiber optic jumper cable that has been pre-terminated with connectors on both ends may be cut in half to produce two short lengths of optical fibers 45 (a) and 45(b) that can be fusion spliced at locations 46(a) and 46(b) to the two ends of the wound fiber 43(a) and 43(b). A cost-effective way to produce these spools is by using 3D Fused Deposition Modeling (FDM) printing employing a polyethylene terephthalate (PETG) polymer material. The keyway 47 on the rotational axis of the spool allows the spool to be set on and secured in four different rotational position on any one of the six keyed shafts 32(a), 32(b), 32(c), 32(d), 32 (e), and 32(f) inside of the module 30, as shown in FIG. 6. With four different rotational positions allowed for each spool, the person attaching the spool onto one of the keyed shafts in the module 30 can select the most favorable rotational position to connect the fiber optic connectors 44(a) and 44(b) on the ends of the wound fiber to mating connectors in the group of panel mounted fiber optic connectors 23.
[0039] In case one of the wound fibers in a module 30 breaks or it is desired to change a wound fiber with another one having a different time delay, a technician can pull the module from its equipment rack, remove the module's cover, disconnect the two ends of the wound fiber selected for replacement, and then slide the corresponding spool off of its keyed shaft. These steps can be reversed for the replacement of a new spool of optical fiber into the module.
[0040] While the above drawings provide representative examples of specific embodiments of the multilink equalization apparatus, numerous variations in the shape and design details of this apparatus are possible.
