Key takeaways:
- TE executives say data center infrastructure is becoming a tightly coupled system across power, cooling, sensing and resilience.
- TE’s Mike Tryson said interconnect is probably the first bottleneck for AI data centers, with power a close second.
- TE’s executives named water as the bigger long-term constraint on data centers, ahead of power.
Power delivery and interconnect, both copper and optical, are two of the main bottlenecks facing data centers today. Not surprisingly, improving efficiency in these sectors has become a critical focus for data center vendors and chip suppliers.
TE Connectivity sees these two sectors as part of a changing focus for data centers.
“Data center infrastructure is changing from a collection of components into what I call a tightly coupled architecture across power, cooling, sensing, and overall resilience and reliability,” said Mike Tryson, vice president (VP) and chief technology officer (CTO) for digital data networks at TE Connectivity. “We're managing those interfaces from the utility connection, through the facility, to the rack, all the way to compute.”
GlobalSpec talked with a trio of executives from TE Connectivity for an exclusive discussion about some of the major inflection points facing data centers as well as how sensing is enabling the emergence of software-defined vehicles. The executives include Tryson; Karthik Krishnamurthy, VP and CTO for industrial solutions at TE; and Corneliu Tobescu, VP and CTO for sensors at TE.
GlobalSpec: Where does TE Connectivity fit in the whole data center conversation? This is a hot topic for a variety of reasons and crosses numerous electronic sectors. What role does TE play today, and where might it play in the future?
Mike Tryson is the VP and CTO for digital data networks at TE Connectivity. Source: TE ConnectivityMike Tryson: Relative to the data center, we're the infrastructure that allows the data to move. TE provides both scale-up and scale-out interconnect solutions, as well as power delivery. Power in AI clusters is going to grow by a factor of five over the next few years. We've brought some novel technology as data rates go from 100 gigabits a second to 400 and eventually 800. Somewhere between 400 and 800, there will start to be a transition to optics, and we have a play there that we're starting to build out.
As you think about scale-out, you're going from thousands of GPUs to millions, so there are far more interconnects involved. Interconnect is probably the first bottleneck to AI in data centers, with power a close second. We provide integrated solutions on both fronts.
GlobalSpec: You touched on going from thousands of GPUs to millions and from 800 gigabits on up. Do we keep expanding? Is there a limit that can be hit? Will it come to a point where the industry will need to catch up to the expansion?
Tryson: No slowdown, is the short answer.
There are some technology inflections along the way. Going to millions of GPUs will be more optically enabled, to interconnect the GPUs. You'll need longer transmission distances at higher speeds, so newer technologies come in. How do you get more data into the same fiber? Whether it's multicore fiber or [wavelength division multiplexing] WDM, there are multiple solution paths for higher density.
A GPU today has about one terabyte of aggregate bandwidth at its interface. That's going to go up by probably a factor of 20 over the next few years. So that's the interconnect bandwidth bottleneck. Copper has a certain reach, and people will use copper as long as they can. It's the most mature, and the ecosystem is very mature. The optics ecosystem is not nearly as mature, both for pluggable systems and as they move to [near-packaged optics (NPO) and co-packaged optics (CPO)] solutions. So, the industry will keep asking how to extend the life of copper, which is what we do best.
GlobalSpec: There is a lot of talk about on-device AI, and we are starting to see vendors come out with first devices. There was a security system announced recently that supposedly has on-device AI. As AI gets moved to the edge and on device, how does that evolve the role of the data center, and will it play as big of a role as they do today?
Karthik Krishnamurthy is the VP and CTO for industrial solutions at TE Connectivity. Source: TE ConnectivityKarthik Krishnamurthy: When it concerns mission-critical or important topics, the decisions might be made at the edge, but there's going to be a backup. There's going to be resiliency and redundancy. We've seen 2N resiliency being built out, so from a market and growth perspective, that tells us growth is still pointed up.
To connect it to your previous question, we're dealing with bandwidth growth along with power density growth driven by that resiliency, plus the thermal elements we're hitting. So, we're becoming a multi-physics problem across multiple domains. If anything, [edge and on-device AI drive it more], in some cases because the device must connect to the mothership.
Tryson: I agree with Karthik. As AI gets deployed more at the component level, it just means more overall network bandwidth. Today the big infrastructure is around training. The next wave is more inferencing on local models, and that's going to happen at the edge and down at the device level. That wave is just starting. It hasn't really become a wave yet. It's a bit of a trickle, but it's going to come. It's an eventuality.
GlobalSpec: What about future bottlenecks? You have mentioned a few already, but say power delivery gets solved and interconnect gets solved. Are there others you see cropping up in the next couple of years?
Tryson: Nobody wants these mega data centers in their backyard, whether that's because of power, the grid, taxes or water. Power and water are the two big ones getting talked about. Water is probably the more concerning one long term.
I think the world will figure out how to make more power. Whether people like how it's made, or whether it's clean enough, is a question. But water is a scarce commodity, and at some point people may have to choose between the comfort of modern living and using AI for certain tasks. I'll say that's the extreme case, but those are the two looming bottlenecks on the horizon that need solution sets.
Krishnamurthy: I'll add to what Mike said. Power can be generated in one place and moved across the grid. A watershed typically isn't that easily moved, and that's what generates the concern.
But if you look at TE, on power we're also engaged in battery electric systems, grid resiliency and grid modernization, anywhere from an efficiency play to helping customers deliver power more intelligently. Likewise, on the cooling side, water isn't the only solution. There are more efficient ways to cool: immersion cooling, dielectric fluids, direct-to-chip cooling. We share the same concerns Mike raised, so we're supporting both efficiency and responsible use of resources.
Corneliu Tobescu is the VP and CTO for sensors at TE Connectivity. Source: TE ConnectivityCorneliu Tobescu: Speaking from the sensors perspective, we're engaged in the challenge that comes from heat. We believe the real challenge going forward is no longer just processing data but managing the heat that comes with it.
As Karthik said, every watt consumed by a GPU ultimately becomes heat, and as [power levels] continue to increase, cooling is becoming one of the technology challenges of our time. The industry is rapidly moving from traditional air cooling to liquid cooling as Karthik said, direct-to-chip architecture that can efficiently support high-density AI infrastructure.
We see the future of data centers being defined not only by faster processors but by smart infrastructure too. There we want to play a role by providing sensing technology that lets operators monitor coolant flow, optimize pump performance, detect anomalies early, and protect mission-critical AI infrastructure. There is no way a data center can fail; reliability is a must. Our portfolio supports everything from precision pressure [sensing] to temperature monitoring and ultrasonic level sensing for advanced liquid cooling systems.
GlobalSpec: Turning to electric vehicles (EVs), beyond the politics surrounding EVs, countries outside the U.S. are expanding EVs and continuing to develop their fleets. Not that American automakers aren't, but it's slower, I guess is the best way to put it. Looking ahead, where do you see the next technology breakthrough in electric vehicles, and how will it change where we are in the market today?
Tobescu: The automotive industry is experiencing a convergence of three powerful forces: electrification, software-defined vehicles and increasing automation. They're fundamentally changing the role of sensing. In combustion vehicles, sensors primarily helped optimize engines and emissions. In electric vehicles, sensing becomes a direct enabler of efficiency, safety, and intelligence.
Electric motors can operate at speeds far beyond traditional engines. We're talking about 30,000 RPM versus 7,000 to 8,000 on combustion engines, which requires highly accurate position sensing and real-time control. At the same time, battery performance, charging speed, range, and reliability increasingly depend on sophisticated thermal management and cooling, through monitoring capabilities.
We also see a shift from mechanical and hydraulic systems toward by-wire systems: brake-by-wire, steer-by-wire, and all-electronic control architectures. Sensors become the so-called trusted source of truth. Reliability is important, redundancy is important, diagnostics are important, and so is cybersecurity. You don't want sensor data altered by a cybersecurity event. These matter as much as measurement accuracy.
TE Connectivity executives said sensors are becoming the “nervous system” of software-defined vehicles as electrification and automation grow. Source: TE Connectivity
GlobalSpec: So, the number of sensors in vehicles is going to increase even further than what we've already seen. There are a lot in there already, but it sounds like exponential growth.
Tobescu: Exactly. At the same time, systems will be using information from multiple sensors with [low] latency, and for autonomous driving.
Information from wheel speed, motor position, braking, thermal management, even occupant sensing is increasingly shared across multiple vehicle systems. That transforms sensors from components into the nervous system of the software-defined vehicle.
So, the next generation of automotive innovation won't be defined by batteries or software alone. It will be defined by how effectively vehicles sense; how they understand; and how they respond. Those are the trends I see.
