Networking
Fat Trees, Dragonflies and Tori: A Field Guide to Network Topologies
Spine-leaf is not the only way to wire a cluster. This guide introduces the main network shapes used in data centers and supercomputers and the tradeoffs behind each.
The pattern in which switches and servers are connected, the topology, shapes a network’s speed, cost and resilience. Data centers commonly use spine-leaf, but supercomputers and AI clusters draw on a wider menu. Here is a field guide.
Fat tree
A fat tree is a tree-shaped network where the links get fatter, meaning higher in capacity, as you move toward the top, so the upper levels are not a bottleneck. In practice, data centers build fat trees out of many identical switches arranged in layers, which is closely related to the Clos network design described by Charles Clos for telephone switching in the 1950s.
Spine-leaf is essentially a two-tier version of this idea. Larger clusters add a third tier. Fat trees provide many equal paths between any two endpoints and can deliver full bandwidth between all servers when built without oversubscription. The cost is a lot of switches and cables, especially at very large scale.
Dragonfly
A dragonfly topology groups switches into tightly connected clusters, then links every group directly to every other group with long global links. Most traffic can reach any destination in a small number of hops: within the local group, across one global link and into the destination group.
Dragonflies reduce the number of expensive long-distance optical links compared with a large fat tree, and they are used in some major supercomputers. The challenge is routing, since traffic between two busy groups can congest their direct link, so adaptive routing that sends some traffic on detours is important.
Torus
In a torus, each node connects directly to its neighbors in a grid, and the edges wrap around so the last node in a row connects back to the first. Tori can have two, three or more dimensions.
Torus networks need no large central switches, scale in a regular way and suit workloads where nodes mostly talk to neighbors, such as many physics simulations. Some AI accelerator systems have used torus-style connections between chips. The downside is that traffic between distant nodes must take many hops.
Other shapes worth knowing
- Rail-optimized designs, common in AI clusters, connect the same-numbered accelerator in every server to the same set of switches, so the most frequent communication patterns stay on dedicated paths.
- Optical circuit switching, where mirrors or other optical components physically redirect light between ports, lets operators reconfigure the topology itself, an approach some large operators have used in their data center networks.
- Hybrid designs mix a tightly connected scale-up domain within each rack and a scale-out topology across racks.
How builders choose
No topology is best for every job. Designers weigh how many endpoints must be connected, how traffic flows, how much cabling and optics the budget allows, how failures are handled and how easily the network can grow. AI has made these choices more consequential, because the network can determine how efficiently expensive accelerators are used. When you see a new cluster described, the topology is often a quiet clue to the builder’s priorities.
Comparing the Shapes Side by Side
A quick summary helps keep the options straight. These are general characteristics; real designs vary.
Fat tree at a glance
- Hops between distant endpoints: few and consistent.
- Strengths: many equal paths, predictable performance, full bandwidth when not oversubscribed.
- Costs: many switches, cables and optics as scale grows.
- Best fit: general data centers and AI clusters needing broad, even connectivity.
Dragonfly at a glance
- Hops between distant endpoints: few, typically through one global link.
- Strengths: fewer long-distance links than a comparable fat tree.
- Costs: needs adaptive routing to avoid congestion on busy global links.
- Best fit: very large systems where long links are expensive.
Torus at a glance
- Hops between distant endpoints: many, growing with system size.
- Strengths: no large central switches, regular structure, efficient for neighbor communication.
- Costs: distant traffic crosses many nodes, and failures can affect routing through neighbors.
- Best fit: workloads that mostly communicate locally.
A Worked Example: Hops Across a Torus
A simple calculation shows why distance matters in a torus. Imagine a two-dimensional torus arranged as a 16 by 16 grid, 256 nodes in all, where each node links to its four neighbors and the edges wrap around.
- Thanks to wraparound, the farthest you can be from any node in one direction is 8 steps, half of 16.
- Across two dimensions, the worst case is 8 steps in each, or 16 hops.
- Neighbors, by contrast, are just one hop away.
If the workload mostly talks to neighbors, the torus is very efficient. If it often needs to reach distant nodes, those 16 hops add delay and load on the links in between. Adding a third dimension shortens the worst-case path for the same node count, which is one reason three-dimensional and higher tori have been used in large systems.
Common Misunderstandings
“There is one correct topology for AI”
Different builders make different choices depending on scale, workloads and budget. The variety in real systems reflects real tradeoffs.
“Fewer hops always means faster”
Hop count matters, but so does congestion. A path with fewer hops that is overloaded can be slower than a longer, quieter one.
“Topology is fixed once built”
Physical cabling is hard to change, but routing policies, job placement and, in some designs, optical circuit switches can change how the network behaves.
Terms Worth Knowing
- Bisection bandwidth: the total capacity across a cut that splits the network into two equal halves, a common measure of how well it handles all-to-all traffic.
- Diameter: the largest number of hops between any two endpoints.
- Adaptive routing: choosing paths dynamically based on current congestion.
- Radix: the number of ports on a switch, which shapes how flat a network can be.
Questions Readers Ask
Why are long links expensive?
Longer links generally need optics rather than copper, and optics add cost and power. Topologies that reduce long links can save on both.
What should I look for in a cluster description?
Note the topology, the number of tiers, whether it is oversubscribed and whether it uses rail-optimized or hybrid designs. Together they hint at what the builder prioritized, whether that is even performance for any job, lower cabling cost or efficiency for one particular kind of workload.
Can different topologies be combined?
Yes. Many systems use one shape inside a rack or group and another to connect those groups, matching each layer of the network to the traffic it carries most often.
This field guide is general education and not investment advice. For financial decisions related to networking companies, consult a qualified financial professional.
Latest stories
View allNetworking
The Control Plane Can Be a Shared Dependency Across Separate Systems
Two independent-looking systems can still rely on one service to decide what they should do. Configuration management, identity, name resolution and orchestration can become common dependencies even when the data paths are physically separate.
Photo: Scott Rodgerson / Unsplash
Two independent-looking systems can still rely on one service to decide what they should do. Configuration management, identity, name resolution and orchestration can become common dependencies even when the data paths are physically separate.
The operating distinction
The data plane moves the work, while control functions establish or manage its behavior. Some equipment continues operating through a temporary control interruption; other actions may fail until control returns. The distinction matters because successful ongoing traffic does not necessarily prove that new sessions or changes will work.
Management dependencies can extend beyond the network team. Identity services and access systems may be owned by another group, while the people restoring equipment assume those services are available. That creates an organizational boundary as well as a technical one.
A practical example
Picture a hypothetical network that keeps existing connections running while its management service is unavailable. Monitoring may report that traffic is flowing, yet a new deployment cannot receive the configuration it needs. Both observations can be true at the same time.
Questions worth asking
Duplicated data links do not explain the availability of the services that configure them. Identifying control dependencies helps distinguish uninterrupted traffic from the broader ability to manage, replace and restart equipment after a fault.
- Which functions keep working when the shared control service is unavailable?
- Can new sessions and replacement devices be brought into service?
- How is access to the management system restored during a broader incident?
What the evidence supports
A shared control service is not automatically an unacceptable design. Centralization can simplify management. The task is to identify what depends on it, how interruption affects different operations and how restoration is tested. A diagram should show management dependencies as well as the visible cables.
A useful tabletop discussion asks what can be observed, changed and recovered if the common management service is absent. The answer should distinguish a service that can continue automatically from one that needs human intervention or new configuration. It can then identify the approved route for that intervention. This provides a better picture of recovery capability than a diagram that shows duplicated data links but omits who or what controls them.
Separate the ability to keep current work moving from the ability to manage or restart it.
Quick check
How much did you catch? Tap a question to see the answer.
1. How can separate systems share a dependency?
Answer: Through control functions like identity, configuration management, name resolution or orchestration.
2. Does flowing traffic prove new changes will work?
Answer: No. Existing connections can run while new sessions or deployments fail.
3. Why is this an organizational issue too?
Answer: Identity and access services may belong to another team.
A Map of Common Control Dependencies
Control dependencies are often invisible on a physical diagram because they do not occupy their own cables. Listing them explicitly is a useful first step. In many environments, the shared services include:
- Identity and authentication. The systems that confirm who an engineer or automated process is and what they are allowed to change.
- Name resolution. The services that translate names into addresses for both applications and management tools.
- Configuration management. The repositories and tools that hold intended device settings and push them out.
- Orchestration and scheduling. The systems that decide where workloads run and start or stop them.
- Certificates and secrets. The services that issue and renew the credentials systems use to trust each other.
- Time synchronization. Accurate time underpins logging, certificate validity and coordination.
- Out-of-band management. The separate access path engineers use when the main network is unavailable.
Running a Control-Plane Tabletop Exercise
A tabletop exercise lets a team walk through a scenario without touching live systems. A simple structure works well.
- Pick one shared service. Assume it is unavailable for a defined period.
- List what keeps running. Identify functions expected to continue automatically, such as existing traffic flows.
- List what stops. Identify actions that would fail, such as new deployments, logins, configuration pushes or device replacements.
- Trace the recovery route. Determine how engineers would reach the affected systems, and whether that route itself depends on the missing service.
- Check emergency access. Confirm that break-glass credentials or local access methods exist, are controlled and have been tested.
- Record corrections. Capture missing documentation, unclear ownership or circular dependencies discovered during the walk-through.
Common mistakes
A frequent discovery is a circular dependency, where the tool needed to restore a service relies on that same service. Another is an out-of-band network that quietly shares authentication or name resolution with the main environment. A third is assuming that the identity team will be available and informed during an infrastructure incident, when no joint procedure exists. Finally, emergency credentials are sometimes created and never tested, so nobody knows whether they still work.
What This Means for Businesses and Readers
For organizations using cloud and hosted services, control-plane dependencies help explain why an outage can affect services that appear unrelated. A disruption in a shared management or identity function can prevent changes and new connections across many systems while existing work continues. For readers trying to understand an incident report, distinguishing between data-plane and control-plane effects often clarifies what customers actually experienced.
Separate cables can carry the work. Something still has to tell them what to do.
This is general education about system design. It does not assess any particular provider’s architecture.
Questions Readers Ask
Is centralizing control a mistake?
Not necessarily. Centralized management can improve consistency and security. The important part is understanding and testing what depends on it.
What is the difference between the data plane and the control plane?
The data plane carries the actual traffic or work. The control plane decides how that traffic is routed, configured and managed.
Why might existing connections survive while new ones fail?
Established sessions may already have the information they need, while new sessions require lookups, authentication or configuration from the unavailable service.
What is break-glass access?
A tightly controlled emergency method of reaching systems when normal access paths are unavailable, used under defined conditions and recorded.
How often should control dependencies be reviewed?
Whenever a shared service is introduced, replaced or reorganized, and periodically in between, because new dependencies tend to accumulate quietly as tools and teams change over time.
AI infrastructure explainers and news on iPhone, iPad and Apple Watch.
Networking
Two Fiber Contracts Can Still Share One Excavation Risk
Buying connections from different suppliers does not establish that the physical routes are independent. Somewhere beyond the facility fence, those services may pass through the same duct, bridge or building entrance, creating a common exposure.
Photo: Tom Shamberger / Unsplash
Buying connections from different suppliers does not establish that the physical routes are independent. Somewhere beyond the facility fence, those services may pass through the same duct, bridge or building entrance, creating a common exposure.
The operating distinction
Network service is a chain of physical assets and operational arrangements. Separate billing relationships can sit on shared infrastructure. Route diversity therefore asks where the cables actually travel and which crossings, access points and equipment rooms they share. Logical paths alone cannot answer every physical question.
The same reasoning applies to restoration plans. An alternate service that survives one local cable fault may still be affected by a wider event. Naming the scenario avoids an all-or-nothing conversation about whether a network is diverse.
A practical example
Imagine two connections that enter a campus from opposite sides but meet at the same nearby road crossing. A fault at that crossing could affect both services. The separate entrances remain useful, but they do not remove that shared section of the route.
Questions worth asking
Different contracts can provide useful commercial options without proving physical independence. Reading the route evidence separately helps identify the actual benefit of the arrangement and the remaining shared exposure that needs another response.
- Which physical segments are shared outside the property boundary?
- When was the route information last confirmed with the providers?
- What restoration options remain if that common segment is unavailable?
What the evidence supports
Route documentation can be incomplete or subject to confidentiality limits. It can also become stale after changes. The practical goal is to establish the important shared risks and the evidence available, rather than assume that a second contract eliminates every possible network interruption.
A route sketch should distinguish confirmed information from an assumption or a supplier statement awaiting clarification. It can then show the important uncertainty without revealing sensitive route details publicly. The point is to make procurement and operating discussions more precise: which kind of damage is covered, which common dependencies remain and what evidence would justify a stronger claim. That is more useful than assuming that different service names imply different physical journeys.
Diversity is a property of the complete path, not simply the number of names on the invoices.
Quick check
How much did you catch? Tap a question to see the answer.
1. Do two suppliers guarantee independent routes?
Answer: No. They may share a duct, bridge or building entrance.
2. What does route diversity ask?
Answer: Where the cables physically travel and which crossings they share.
3. What could two entrances on opposite sides still share?
Answer: The same nearby road crossing.
How Shared Segments Usually Hide
Common exposure rarely announces itself on a contract. It tends to sit in places that are easy to overlook because they are outside the property and outside the day-to-day attention of the facility team.
- Shared conduit or duct banks. Different providers may lease space in the same underground structure along a road.
- Bridge and rail crossings. Physical geography often funnels routes toward a small number of practical crossing points over rivers, highways or rail lines.
- Common meet points. Carriers can hand traffic off in the same nearby building or cabinet before it reaches the facility.
- Resold or leased capacity. One supplier may carry traffic over another supplier’s fiber, so two names on two invoices can describe one physical cable.
- The same building entrance. Even when routes are separate for miles, they may converge on a single entrance room or cable vault.
A Practical Route Review, Step by Step
Facility teams and buyers can approach route diversity as a structured review rather than a yes-or-no question.
- Request route information from each provider. Ask for the level of detail they are able to share, recognizing that some route data is confidential.
- Mark known convergence points. Identify crossings, meet points, entrances and any segments where the routes appear to run near each other.
- Label the quality of each claim. Separate confirmed information from provider statements awaiting verification and from assumptions.
- Name the scenarios. Consider a single cable cut near the site, construction damage along a shared road, a bridge closure and a problem in a shared equipment room.
- Map restoration options to each scenario. Decide which services would remain available and which would require a separate response.
- Set a refresh date. Routes change when providers reroute, rebuild or sell assets. Agree on how often the information will be confirmed again.
Common mistakes
One mistake is accepting a provider’s diversity label without asking what it covers. Another is checking diversity once at contract signing and never again. A third is focusing entirely on the long-haul portion of a route while ignoring the last stretch near the facility, where excavation work is often most likely to occur. A fourth is assuming that a wireless or alternate technology backup has no shared dependency, when it may rely on the same power, building or tower infrastructure.
What This Means for Businesses Relying on Connectivity
For organizations that depend on a data center or cloud region, network diversity is part of the resilience they are buying, whether or not they negotiate it directly. A provider that can explain its known shared exposures clearly is usually giving a more honest picture than one that simply states its routes are diverse. For readers following infrastructure companies, route quality is one of the less visible factors behind service reliability, and it rarely appears in a headline.
The most useful diversity statement describes which failures the arrangement is expected to survive and which it is not.
Questions Readers Ask
Why do providers keep route details confidential?
Detailed route maps can create security concerns and may be commercially sensitive. Buyers can often obtain enough information to assess shared risk under appropriate agreements.
Is physical diversity always worth the cost?
It depends on how much an interruption would cost the business and which scenarios matter most. The decision benefits from naming the risk rather than buying diversity as a general label.
Can two routes ever be fully independent?
Complete independence is difficult to prove. The practical aim is to reduce the most likely and most damaging shared exposures and to know which ones remain.
How does excavation risk relate to fiber?
Buried cable can be damaged during digging for roads, utilities or construction. Shared corridors mean one digging accident can affect more than one service.
AI infrastructure explainers and news on iPhone, iPad and Apple Watch.
Networking
InfiniBand vs. Ethernet for AI Clusters
InfiniBand offers a tightly integrated path for synchronized AI traffic, while Ethernet offers broader compatibility and choice; the right fabric depends on workload, scale, software, and operating priorities.
The networking choice inside an AI cluster is not a simple bandwidth comparison. Training jobs repeatedly move parameters, gradients, and activations between accelerators. If the network stalls, expensive GPUs wait. The result can be lower utilization even when every link’s headline speed looks impressive.
InfiniBand and Ethernet can both build large clusters. They differ in how much of the transport, congestion control, topology, and software stack is standardized or integrated for the target workload.
Why AI traffic is unusual
AI training produces synchronized bursts. Many accelerators finish a step and communicate at nearly the same time. That creates a traffic pattern that is sensitive to tail latency, congestion, packet loss, and collective-operation performance.
A general enterprise network can tolerate variable paths and occasional retransmission. A large training cluster may lose efficiency if a small number of links become hot and delay an all-reduce operation. The network is part of the compute system, not just plumbing.
InfiniBand’s case
InfiniBand was designed for high-performance computing. It supports low-latency communication, remote direct memory access, and features intended to keep data moving between nodes with limited CPU intervention.
NVIDIA’s April 2025 description of a DGX SuperPOD deployment cited Quantum-2 InfiniBand networking and a digital-twin validation process. The company said a SoftBank deployment of 510 DGX B200 systems was completed in three weeks, compared with a typical six-plus months, and that NVIDIA estimated the avoided downtime at roughly $1 million per day for a 1,000-server, 8,000-GPU data center. That estimate is NVIDIA’s analysis, not an independent industry average, but it illustrates why network deployment speed can have financial value.
InfiniBand’s strength is a validated stack. Hardware, drivers, collective libraries, and topology can be designed together. That can reduce the integration burden for a customer that wants a high-performing training appliance.
Its weaknesses are ecosystem and flexibility. It can require specialized adapters, switches, expertise, and operational tools. A company with a large Ethernet organization may prefer a fabric it already knows.
Ethernet’s case
Ethernet is widely deployed across data centers and connects servers, storage, offices, and the public network. It offers a large supplier base, familiar management tools, and multiple implementation choices.
AI Ethernet systems commonly use lossless or low-loss designs, remote direct memory access over Converged Ethernet, explicit congestion notification, and careful buffer and topology planning. The performance depends heavily on configuration. “Ethernet” by itself does not say whether a cluster will perform well.
Intel’s Gaudi 3 materials emphasize standard Ethernet and a 32-node all-Ethernet reference design. The pitch is not merely lower equipment cost. It is the ability to use existing operational skills and to avoid a completely closed networking path.
Speed changes the economics
Arista announced new 800G platforms in October 2025 and said 800GbE port shipments more than tripled sequentially in the second quarter of 2025, according to cited market research. The company described 3.2-terabit-per-second HyperPorts and AI uses including lossless training and inference.
800G matters because an AI rack can have many high-bandwidth links. If each accelerator needs 800 gigabits per second of east-west bandwidth, a 72-accelerator rack creates enormous switching demand. Moving to 1.6T transceivers can reduce port counts or support larger bisection bandwidth, but it also raises optical, thermal, and signal-integrity requirements.
The comparison should be made at job completion time
A network that delivers higher link speed but poor congestion behavior may lose to a slower network with better collective performance. The useful test is end-to-end: time to train, tokens per second, scaling efficiency from one node to 32 or 128 nodes, tail latency, and behavior during failures.
The test should include the actual model, precision, batch size, topology, and software release. A vendor’s ideal benchmark can be informative, but it is not a deployment guarantee.
Operations may decide the outcome
A cluster can fail financially through poor utilization. If the team cannot troubleshoot the fabric, the expensive accelerators sit idle. A familiar Ethernet operation may be worth more than a theoretical performance advantage. Conversely, a dedicated AI service with a specialized team may benefit from the tighter InfiniBand stack.
A hybrid architecture is common. InfiniBand or specialized Ethernet can carry accelerator-to-accelerator traffic, while ordinary Ethernet handles management, storage, and user access. The boundary must be clear so a failure in one network does not quietly degrade the other.
The decision rule
Choose InfiniBand when predictable scale-out training, validated deployment, and low-latency collectives dominate. Choose Ethernet when supplier choice, operational familiarity, interoperability, and mixed workloads matter more. Measure both on the workload that pays the bills.
The procurement decision should include the people who will operate the fabric. Ask how many engineers are trained on the selected switches, how firmware is qualified, how telemetry is exported, and how a bad optic is isolated. Also ask whether the vendor can support a mixed-generation cluster during a refresh. A network that is technically fast but difficult to diagnose can reduce accelerator utilization for weeks. In that sense, operational simplicity is a performance feature, not an administrative preference.
At very large scale, fabric choice also affects the power budget. More ports, higher-speed optics, and additional switch tiers consume electricity and occupy rack space. That overhead should be included in the same total-cost model as accelerator power. A lower-cost accelerator that needs a larger network may not be cheaper at the cluster level. The fair comparison is the amount of useful work delivered by the complete system after networking, cooling, software, and operations are included.
Sources
- NVIDIA, “NVIDIA Helps Build AI Factories Faster Than Ever with DGX SuperPOD,” April 11, 2025: developer.nvidia.com
- NVIDIA, “Blackwell: Born for Extreme-Scale AI Inference,” 2025: blogs.nvidia.com
- Intel, “Intel Gaudi AI Accelerator Products,” accessed October 2026: intel.com
- Arista, “Arista Networks Unveils Next Generation Data and AI Centers,” October 29, 2025: arista.com
- NVIDIA, “Reference Architectures: Deep Dive,” accessed October 2026: docs.nvidia.com
BOSSTOX Research articles are drafted with AI assistance and reviewed by our editors before publication. See our editorial standards.
-
Power & Grid1 month agoTransformer Lead Times Are Becoming a Data-Center Schedule Risk
-
Power & Grid1 month agoGas Turbines on Data-Center Campuses
-
Cooling2 months agoWater Use and Dry Coolers: The Tradeoff AI Sites Cannot Avoid
-
Power & Grid1 month agoBattery Storage at AI Sites
-
Compute & Silicon1 month agoGPU Supply Chains: How AI Hardware Gets Made, Moved, and Bottlenecked
-
Power & Grid4 weeks agoFERC’s Large-Load Orders: The November 16 Grid Deadline Behind the AI Build-Out
-
Capital & Backlog4 weeks agoAI Infrastructure’s Rough Week: Chips Fall 6.7% as Meta Lifts Capex Floor
-
Cooling4 weeks agoLiquid-Cooling Leak Detection and Reliability
