Why the next improvement in AI infrastructure may come from changing the rack itself
Open the back of a modern AI rack and you see a strange result of progress: the faster the chips get, the more engineering it takes to connect them.
NVIDIA’s GB200 NVL72 joins 72 GPUs into one high-bandwidth domain. Behind its compute trays sits a carefully engineered spine of more than 5,000 copper connections. The newer Vera Rubin NVL72 still uses roughly 5,000 copper cables, packaged into four preassembled cartridges. Together, those cables run more than two miles. [1][2]
Copper is doing important work here. At these short distances, it remains an effective way to move enormous amounts of data with low latency. But the cables also tell us something about where AI infrastructure is going. A rack has become a computer in its own right, and its internal connections now shape what that computer costs, how it is built, and how it is serviced.
Figure 1. Anatomy of a 72-GPU rack-scale system. The copper network behind the trays is now a major part of the product. Industry example drawn from public NVIDIA sources [1-4]; schematic, not to scale.
The rack is becoming the unit of compute
For years, a server rack was mostly a place to put separate machines. In an AI system, those machines increasingly need to act together. Data moves among GPUs during training and inference, and the interconnect helps determine how much useful work they can do.
That is why NVIDIA designed GB200 NVL72 as a 72-GPU NVLink domain with 130 TB/s of rack-level GPU communication bandwidth. NVIDIA states that Vera Rubin NVL72 raises that figure to 260 TB/s. These are substantial advances. They also require the compute trays, switches, connectors, cable spine, power delivery, and cooling system to be designed as one machine. [3][4]
The cost of a connection is therefore larger than its purchase price. It includes the space reserved for routing it, the connectors at both ends, the work of assembling and testing it, and the constraints it places on the next rack design.
Figure 2. The full cost of a high-speed connection goes well beyond the cable or transceiver price. Illustrative categories only.
SemiAnalysis has documented the GB200 cable architecture and its component economics in detail. Its coverage of Vera Rubin also describes the engineering work involved in improving the copper backplane. The interesting conclusion is not that copper has failed. It is that making copper work at this scale has become a major part of the product. [5][6]
Removing cables from a tray does not remove the network behind it
NVIDIA describes Vera Rubin’s compute trays as cable-free. That is a meaningful improvement for assembly and service. NVIDIA says the revised design cuts compute tray assembly time from hours to one minute. Yet its rack-level NVLink spine still contains roughly 5,000 copper cables. [2][7]
Both statements can be true. The tray becomes easier to handle while the rack continues to depend on a substantial wired backplane.
This distinction matters because “cable-free” can describe several different designs. A tray might have no loose internal cables. It might plug into a preassembled cable cartridge. Or its data connection to another module might be contactless. Each removes a different physical constraint.
The first two simplify manufacturing and service. The third could also change how modules are placed and connected inside the rack. It has to earn that flexibility without giving up the bandwidth, latency, reliability, or efficiency that a wired rack already delivers.
Figure 3. “Cable-free” can mean three different designs. A and B are industry approaches in use today; C is RIVVOR’s Bounded Electromagnetic Corridor concept, still under validation.
Where contactless connections have a useful job
RIVVOR is exploring two ways to make selected data connections contactless inside a rack.
Bounded Electromagnetic Corridors build guided sub-THz paths into structures such as rack sidewalls or chassis partitions. Modules couple to those paths across a controlled gap. Directional sub-THz links use the rack’s known geometry to form connections through its enclosed space.
Figure 4. In a BEC-enabled rack, selected short-reach data paths move from cables and connectors into the rack structure itself. Concept adapted from the RIVVOR Technical White Paper.
The two approaches serve different needs. A corridor can provide a designed path for connections used frequently or persistently. A directional link could provide a direct or alternate path where routing a dedicated connection through the structure makes less sense. Neither means that any module can connect to any other module at any time. The available paths depend on the rack’s geometry, endpoint placement, isolation, and installed hardware.
The appeal is straightforward. If a module can exchange data without a conventional high-speed mating connector, it may become easier to replace or rearrange. Some connections may no longer need individual cable routes. Those are possibilities to test, not savings we can assume.
A rack is a useful place to test them. The distances are short. The geometry is known. The propagation environment can be engineered as part of the machine.
The comparison that matters
A contactless link should be judged against the complete wired path it would replace, doing the same job.
That means measuring delivered bandwidth, end-to-end latency, error performance, and power at both endpoints. It also means testing many links operating together and measuring what happens when a module moves, a path is obstructed, or a component needs replacement. Assembly and repair time belong in the comparison too.
The wired baseline is formidable. Copper works well over short reaches, and vendors continue to improve how it is packaged. Optics remains valuable where its reach and bandwidth justify its power and cost. There is no universal “optical port power” figure from which to calculate an automatic wireless saving. For example, Arista lists under 16 W typical power per port for one 800G switch configuration; other modules and configurations differ. [8]
Figure 5. Copper and optics keep improving, but inside the rack every option still needs a routed medium and a mated interface. Conceptual, adapted from the RIVVOR Technical White Paper.
The opportunity is more specific: find the intra-rack connections whose routing, assembly, connector, placement, or service costs justify a different approach. Then prove that the complete contactless system performs the job better.
What comes next
The industry is already spending heavily to make rack-scale interconnects easier to build. Preassembled cable cartridges and cable-free trays are evidence of that effort. RIVVOR’s question is whether some of the connections between modules can be made contactless as well.
We cannot yet put a credible percentage on the rack-level cost, energy, or GPU utilization that would save. Those figures require a defined rack, a comparable wired design, and measurements under real workloads.
What we can say is that the physical network has become too important to treat as an afterthought. When thousands of connections must fit inside one rack, the way they are made affects the architecture of the machine. That is a good reason to test whether some of them still need a wire.
Want to see contactless links running live? Join us at RIVVOR Demo Night on November 12 in Santa Clara, or read more about RIVVOR’s technology.
References
NVIDIA, NVIDIA Blackwell: Born for Extreme-Scale AI Inference. Describes the GB200 NVL72’s copper cable spine.
NVIDIA, NVIDIA Vera Rubin POD: Seven Chips, Five Rack-Scale Systems, One AI Supercomputer. Describes Vera Rubin’s four cable cartridges, approximately 5,000 copper cables, and more than two miles of cable.
NVIDIA, GB200 NVL72. GB200 rack configuration and NVLink bandwidth.
NVIDIA, NVIDIA NVLink: The Scale-Up Network for AI Factories. Vera Rubin NVL72 bandwidth and scale-up architecture.
SemiAnalysis, GB200 Hardware Architecture: Component Supply Chain & BOM. Analysis of the GB200 interconnect design and components.
SemiAnalysis, Vera Rubin: Extreme Co-Design, An Evolution from Grace Blackwell Oberon. Analysis of the rack architecture and copper backplane.
NVIDIA, NVIDIA Vera Rubin Driving Performance Per Watt, Lower Token Costs for Partners Worldwide. Describes the cable-free compute tray and NVIDIA’s assembly-time claim.
Arista Networks, 7060X6 Series Quick Look. Example of a published 800G port power specification.
