Engineers discuss how rapid growth in AI-driven workloads is reshaping data center design and driving higher power densities.
Learning objectives
- Understand how AI workloads are transforming infrastructure requirements and pushing the limits of traditional cooling and electrical systems.
- Evaluate emerging cooling and power strategies including the shift from air- to liquid-cooled systems.
- Assess the role of modularization and off-site construction using prefabricated systems and skid-mounted equipment to improve construction speed.
Data center insights
- Artificial intelligence is forcing a fundamental redesign of data center infrastructure due to explosive increases in rack power density and synchronized graphics processing unit workloads.
- Speed, scalability and power constraints are accelerating modular and hybrid solutions such as prefabricated systems, on-site generation and phased buildouts to overcome grid limitations and aggressive timelines.
Respondents:
- Brook Gummere, PE, FPE, ATD, Colorado BES Market Sector Leader, HDR, Denver
- Anvay Joshi, PE, Mechanical Engineer II, Affiliated Engineers Inc., Madison, Wisconsin
- Bill Kosik, PE, CEM, LEED AP, Mission Critical Sector Leader, HED, Chicago
- Kenneth Kutsmeda, PE, LEED AP, Global Technology Leader – Data Centers, Jacobs, Philadelphia
- Daniel Noto, PE, LEED AP, Southeast Market Leader, Fitzemeyer & Tocci Associates Inc., Alpharetta, Georgia
- Brian Schlosser, PE, Principal Fire Protection Engineer, Jensen Hughes, Columbia, Maryland
- Ken Urbanek, PE, LEED AP, ASHRAE HBDP, ATD, Client Executive and Senior Principal, IMEG, Denver
What are some current trends in data centers?
Brook Gummere: There is growing conversation, though not yet a broad shift, about some organizations reconsidering on-premises data centers or expanding to hybrid environments. Increasing concerns around intellectual property protection when data is stored off-site are driving part of this dialogue.
Advances in server hardware and chip technologies are also enabling companies to create their own language models without sending sensitive datasets off-site. This gives some businesses more confidence in keeping proprietary data fully under their control. While a widespread move to in-house data centers has not yet materialized, data sovereignty requirements continue to expand. Regulations such as the U.S. CLOUD Act and various country-specific data residency laws are making cross-border data transfers more complex.
Anvay Joshi: A major trend in data centers is the rapid increase in rack densities and cooling splits, especially when artificial intelligence (AI) workloads are involved. A couple of years ago, I worked on a hyperscale project that was predominantly air-cooled. Projects today are moving toward up to 80% liquid and 20% air-cooled splits. Additionally, the load profiles of AI data centers usually show sudden ramp-ups and ramp-downs, which call for the use of buffers in cooling systems. Because the cooling equipment cannot ramp up or down at the same rate, thermal energy storage (TES) or large buffer tanks are becoming more common.
On the electrical side, there is increasing state-level conversation around on-site generation, battery storage and renewable energy procurement for data centers. For sites with space restrictions and those located in denser neighborhoods, cooling equipment is being placed on dunnage platforms with canopies to avoid hot air reentrainment. Additionally, a greater focus is placed on sound mitigation options from equipment manufacturers to ensure compliance with local laws.
Bill Kosik: Over the past 12 months, there has been a sea change in the data center design and construction industry. After a few years of straightforward power and cooling system design, AI clusters have completely rewritten the rules for data centers. The relatively high power densities of the AI power stack will approach a 10-time increase in power and cooling. These challenges are technically feasible to be solved, but there are not many examples to draw on as a starting point.
Kenneth Kutsmeda: Grid constraints are pushing new data center campuses into remote geographies where land is available for on-site generation and where limited grid capacity still exists. These locations often lack the skilled labor force needed for traditional, labor-intensive construction methods.
To address this, developers are adopting skid-mounted, modular or premanufactured systems that can be assembled, integrated and tested off-site. This approach offers several advantages including reduced on-site labor requirements, shorter construction schedules, higher quality control and parallelization of work, which allows mechanical and electrical systems to be built before the data hall structure is complete.
A clear illustration of this trend is the move toward modular hot aisle containment (HAC) assemblies. The overhead infrastructure in a data hall — liquid cooling piping, power distribution, telecom trays, hot aisle containment and structural supports — traditionally requires multiple trades working sequentially on site. This process is slow, coordination-intensive and highly dependent on local labor availability. By fabricating HAC modules in a warehouse environment, developers can integrate all overhead systems into a single prebuilt assembly, perform alignment, leak testing and commissioning off site and ship the completed module to the data hall for rapid installation. This approach significantly reduces man hours, minimizes schedule risk and improves installation consistency across large deployments.
Daniel Noto: In the data center industry, the key is planning, whether that means leaving room for additional racks and expansion or planning for what information can go to the cloud or even using colocation strategies. Regardless, it’s planning for increased needs for data storage that is driving data center conversations today.
Ken Urbanek: The elephant in the room is AI and this trend is driving heavy energy use and new techniques for mechanical, electrical and plumbing (MEP) system design.
Within the next three years, what trends should engineers or designers expect for such projects?
Bill Kosik: Current AI cabinet power density is still ramping up. Manufacturers of graphics processing units (GPUs) and tensor processing units (TPUs) have published cabinet densities in the 300-kilowatt (kW) range. But with current AI cabinet densities in the 150-kW range, a more practical three-year forecast for AI systems is 120 to 300 kW, with maximum AI cabinet densities of 300 to 400 kW.
Anvay Joshi: The recent growth of AI workload needs has resulted in a push to build data centers at a faster rate than previously seen. With speed to market at the forefront, packaged skids are already becoming products that engineers are turning to. Pushing the assembly of equipment off-site improves installation and setup time. Coolant distribution units (CDUs), which are essentially pumps, heat exchangers and other auxiliary equipment skidded into a single packaged unit are a good example of this trend. CDUs are already ubiquitous in the data center industry.
As site procurement becomes expensive and air-cooled data centers age, there will be massive interest in retrofit projects for existing data centers going through equipment replacements. Lastly, with power availability becoming a major constraint, phased data center projects will be more common as owners work to procure the power required for their sites.
Brook Gummere: Over the next one to three years, engineers can expect data center projects to focus on sharply rising rack-level power demands driven by next-generation AI hardware such as specialized GPUs that deliver significantly higher performance per watt and require greater electrical and cooling capacity. Likewise, the growing adoption of TPUs also will lead to larger campuses on gigawatt scales. To support large-scale AI training, tightly clustered GPUs will become essential to reduce latency between computing elements.
Kenneth Kutsmeda: Power densities in AI data centers are accelerating far beyond the historical 20% generation-over-generation growth rate, creating requirements that exceed the practical limits of traditional alternating current (AC) distribution. To support these unprecedented densities, high-voltage direct current (DC) architectures are being evaluated, drawing on established practices from the electric vehicle, photovoltaic and traction power industries, where high-voltage DC is already mature and widely deployed.
Adopting 750 or 800 volts (V) DC provides several technical advantages over conventional 480 VAC systems. This includes: elimination of multiple conversion stages between AC to DC and DC to AC, improving overall system efficiency; lower distribution losses due to reduced current for the same power level; significant reduction in copper cabling and associated thermal management requirements; and direct compatibility with energy storage systems, which inherently operate on DC and can be integrated without additional conversion hardware. These benefits collectively enable higher rack densities, improved electrical efficiency and more scalable power delivery for large GPU clusters. New classes of equipment are emerging around these technologies, including solid-state transformers capable of high-efficiency AC/DC conversion and voltage regulation, solid-state dc breakers that provide fast, precise fault interruption and high-speed generators designed for direct DC output and rapid response to load transients. Together, these advancements are positioning 800 VDC distribution as a central trend for future AI data center power architectures.
Ken Urbanek: Proximity and system density are critical to optimizing AI computations. As a result, rack densities will continue pushing the limits of power and cooling requirements at the rack level.
How is the rapid growth of AI workloads and high-density computing changing cooling, power distribution and mechanical/electrical design strategies in data centers?
Daniel Noto: AI training/inference clusters are pushing rack power densities from “typical enterprise” levels of approximately 5 to 15 kW per rack into 30 to 60 kW per rack and, for leading GPU racks, toward roughly 100 kW per rack and higher. Once you’re in that range, air cannot move heat fast enough (or efficiently) without extreme airflow, noise and fan power — so data center design is shifting from room-level air cooling and standard electrical rooms to chip-to-facility thermal and electrical engineering.
Ken Urbanek: Rack densities have gone from 10, 20 and 30 kW per rack being considered high density a few years ago to regular AI deployments going well beyond 100 kW per rack and in many cases two to five times higher than this.
Brook Gummere: The speed at which power densities and cooling technologies have been evolving has resulted in a focus on system flexibility. With significant uncertainty around the requirements of next-generation servers over the next one to five years, mechanical and electrical designs will need to be adaptable and agile. Mechanical systems need to support large swings between air and liquid cooling approaches, while electrical systems must be capable of handling large, fast-changing power demands.
Anvay Joshi: Cooling systems in data centers are shifting from traditional room-level air cooling to more localized solutions, such as direct-to-chip liquid cooling. For example, the recently announced NVIDIA Vera Rubin can handle inlet water temperatures up to 113°F. The acceptable server inlet water temperatures have increased over the years, allowing for more efficient cooling strategies.
On the power distribution side, greater density calls for larger busways or power distribution units. Coordination between mechanical and electrical disciplines is critical at an early stage to define redundancy requirements for equipment.
Bill Kosik: If we look back 10 years on the forecasted versus actual server cabinet power demand, there was concern on how to provide power and cooling to “high-density” cabinets. At that time, the forecast for server cabinet power ranged from 10 to 20 kW. Based on these forecasts, the data center engineering community, along with power and cooling equipment manufacturers, developed new approaches to tackle this problem.
From these efforts, two lessons emerged: First, few colocation and enterprise data centers ever got to these densities, but if they did, it was for specialized computing needs and not widespread across the data halls. This led to stranded power and cooling infrastructure due to overestimating power densities. (It must be noted that cabinet power density for high-performance and supercomputing applications at this time ranged from 50 to over 100 kW.) Second, the AI cabinet power densities are growing at a far higher rate than what we saw in the past. Unlike the time when high-density server cabinet loads were growing at a reasonable incremental rate, the GPU and TPU AI stack has been increasing at a rate that challenges data center power and cooling design, especially when developing plans for infrastructure growth over the near term.
Kenneth Kutsmeda: Traditional data centers rely on central processing units (CPUs), which execute instructions in a largely linear fashion. CPU-based workloads are typically asynchronous and uncorrelated, causing individual servers to draw power at different times. This diversity smooths the aggregate load profile and produces relatively stable power demand across the facility.
AI-focused data centers, by contrast, are built around GPUs designed for massively parallel computation. GPU workloads operate synchronously, with large clusters executing identical operations on shared datasets. This synchronization creates highly correlated power patterns, resulting in rapid transitions between low- and peak-load states. In addition, GPU cabinets exhibit significantly higher power densities, commonly 60 to 200 kW per rack, far exceeding traditional CPU deployments.
The shift to GPU-centric architectures drives substantial changes in facility design. Mechanical systems are moving from air-cooled architectures to direct-to-chip or liquid-cooling solutions, required to manage the thermal output of high-density GPU racks. Electrical distribution systems must support higher continuous and peak loads, pushing transformers, switchgear and distribution equipment toward their upper standard ratings, e.g., 3,750 kilovolt-ampere (kVA) transformers and 5,000 amperes (A) switchgear. In addition, integration of battery energy storage systems (BESS) and advanced power-management controls helps stabilize the dynamic load behavior from synchronous GPU operation. These systems absorb short-duration spikes during GPU ramp-up, fill valleys during rapid load drop-off and shape the overall ramp rate to remain within grid-interconnection and generator-response limits.
What are the biggest challenges around utility coordination, power availability and grid interconnection for new data center developments?
Ken Urbanek: The biggest challenge is finding suitable data center locations that can either provide power on a reasonable timeframe or that have space to deploy on-site power generation such as fuel cells.
Bill Kosik: As data center developers and owners requested generation capacity and transmission for AI data centers, the legacy process that utilities used to enter into an agreement with the data center no longer worked for AI facilities, which put the spotlight on the utilities and grid operators. For the first time, the Federal Energy Regulatory Commission became involved in the oversight of current methods and mandated that new processes and protocols be implemented to ensure electricity requests were executed fairly, at a reasonable cost and safely.
These processes were put in place in late 2025. It appears that using this process is improving the method for estimating data center load and includes penalties for data center operators that do not achieve the agreed-upon minimum power estimates.
Kenneth Kutsmeda: The primary constraint is the limited available capacity on the utility grid. Data center load growth is outpacing the timelines for transmission and distribution system expansion, creating persistent deficits in deliverable power. As a result, developers can no longer rely on the utility as a single, firm source of supply. Many facilities are shifting to hybrid power architectures that integrate utility service with on-site generation, temporary feeders or mobile substation assets.
As these hybrid strategies proliferate, microgrid control becomes critical. Coordinating multiple sources requires precise management of interconnection points, relay protection schemes, anti-backfeed measures and load-demand response. Effective control is essential to maintain high power availability, ensure stable transitions between sources and protect both the data center infrastructure and the utility network.
Utilities are also imposing new requirements for large electronic loads. Data centers must now ride through defined voltage excursions and return to pre-disturbance operating levels within one second after voltage restoration. Meeting these requirements — while also supporting the performance characteristics of information technology (IT) equipment — necessitates detailed power quality and dynamic response analysis. This analysis informs the optimal integration of BESS and advanced power-management controls to ensure compliance, stability and resilience.
How is the growth of cloud-based storage and virtualization impacting colocation projects?
Daniel Noto: Demand is shifting from many small single-tenant deployments to fewer, larger, high-density and highly flexible environments designed for cloud providers and hybrid information technology (IT). Instead of traditional cabinets with predictable loads, colocation facilities now need scalable power blocks, higher rack densities and network-rich interconnection to support hyperscale and multi-cloud ecosystems.
Virtualization allows customers to run far more workloads on fewer physical servers, which reduces the number of racks per tenant but increases power density, bandwidth demand and resilience expectations, pushing operators to design facilities with modular expansion, software-defined infrastructure compatibility and stronger connectivity to public cloud on-ramps. As a result, modern colocation developments emphasize carrier neutrality, rapid deployment suites, flexible lease structures and robust power/cooling infrastructure to attract enterprises adopting hybrid and multi-cloud strategies.
Ken Urbanek: This continues to be an area of growth as exponentially more data is generated each day.
What kinds of challenges do you encounter for these types of projects that you might not face on other structures?
Anvay Joshi: Heating, ventilation and air conditioning in data center design differs significantly from comfort-cooling applications in hospitals or higher education buildings. Data centers are more akin to industrial projects where you could break the building into blocks or colocation suites that repeat throughout the facility. As rack densities increase, a project could encounter space constraints for mechanical equipment. Fan walls inside the building might require more width than the total span of all cabinets in the data hall. Chillers or dry coolers in the yard could be pushed together, requiring site validation with manufacturers and the design team.
An iterative process of internal and external computational fluid dynamics (CFD) simulations is also undertaken to analyze varying wind speeds, direction and temperature conditions. Another major consideration for AI data centers is concurrent maintainability. Design engineers need to be mindful of multiple paths of failure — such as valves, electrical feeds, electrical/mechanical equipment and pipes. Any failure should not hinder a data center’s uptime requirements.
Lastly, with schedules being compressed, engineers may be asked to select a wide array of equipment. Long-lead equipment could include larger items like chillers and dry coolers as well as valves and other specialty piping.
Ken Urbanek: These projects present several challenges: expensive MEP components, long lead times, significant density of rack power and cooling systems, large MEP support spaces and more. Project site location and programming are driven by MEP systems within the grey space more than ever before. In some cases of ultra-high-density racks, the grey space can greatly exceed white space requirements.
In what ways are you working with IT experts to meet the needs and goals of a data center?
Daniel Noto: We have worked with a data center consultant on several projects as they are the ones who are truly connected to the pulse of the industry. As MEP engineers, we need to be able to provide the proper support infrastructure, but these consultants help guide the actual design of the racks, servers and other data-related equipment.
Ken Urbanek: We are gaining greater insight into rack operations than ever before, and these computing machines are more than just black box components in a rack. We need to understand their true operation so that MEP systems can best support those significant changes in power and cooling demands.
Tell us about a recent project you’ve worked on that is innovative, large-scale or otherwise noteworthy.
Anvay Joshi: For a recent large-scale project, we used a hybrid dry cooler and air-cooled chiller (ACC) plant design with TES tanks. The CDUs would be served primarily by dry coolers, but on peak days, the TES tanks provide blended chilled water with the dry coolers. The ACC plant serves the air side load and charges the TES tanks during off-peak hours, improving plant flexibility. A challenge for the site was the limited space available for equipment placement. We had to conduct extensive CFD analysis to validate the spacing between ACCs and dry coolers to ensure reentrainment was kept at a minimum. Thorough coordination between different disciplines was necessary to iterate the design from mechanical, structural, electrical and civil perspectives.
Brook Gummere: Recently, we have been leading the design of several large-scale data centers in rural communities where existing infrastructure cannot support the increased utility demands. These projects have required collaborative partnerships with local governments and utilities to upgrade existing water, wastewater, power, fiber and roadway systems. A key challenge and opportunity has been looking beyond the data center to understand what the surrounding community needs to sustain both construction and long-term operations.
Our teams have helped clients develop training programs with trade schools and community colleges, building excitement about careers in mission-critical infrastructure with students. Additionally, we are increasingly exploring locations with direct access to renewable energy generation such as offshore wind paired with energy storage systems to reduce dependence on constrained transmission networks and also to accelerate project delivery.
Ken Urbanek: One recent example is an AI high-performance computing (HPC) center we are deploying on a northern latitude university campus. We can extract the low-grade heat from the computing operations and push it into the campus district energy system. This HPC is relatively small, only 15 megawatts, but it has the potential to be an energy source for heat pump heating of millions of square feet of campus classroom space. Deploying this requires communication between HPC operators, broader campus facilities and alignment with the campus decarbonization plan.
This is a great example of coupling these data centers with adjacent uses that can use waste heat. This can be especially beneficial if the adjacent uses such as health care or higher education campuses are looking for electrification opportunities to decarbonize their campuses. This synergistic relationship is much preferred over throwing waste heat into the atmosphere, which sadly is all too common in data centers across the U.S. We are seeing data centers outside the U.S. consider other unique uses such as heating greenhouses and even heating a trout farm.
How are engineers designing these kinds of projects to keep costs down while offering appealing features, complying with relevant codes and meeting client needs?
Brian Schlosser: The code development process takes time and often lags the need for developing, for example, fire protection schemes for new mechanical and electrical technologies associated with data centers. Historical examples include development of hot/cold aisle containment systems and the use of lithium-ion batteries. Building information modeling has proven to be extremely effective in coordinating and collaborating among various design disciplines to reduce clashes between building systems. This helps ensure that the construction of a data center proceeds as smoothly and efficiently as possible, potentially saving both time and money.
Ken Urbanek: This is a challenge. The equipment is expensive, installation is expensive and demand is driving these costs up everywhere. Early-stage conversations with end users could help ensure that the criticality of the facility is in alignment with their goals. If needed, extra criticality should be provided, but if dropping redundancy, concurrent maintainability and other factors can be tolerated, it can significantly reduce cost.
Additionally, as rack densities and temperatures increase, mechanical systems are shifting toward non-compressor cooling options for direct liquid cooling, which can in some cases reduce costs. This is highly dependent on the system’s operating temperatures and ambient conditions at the site. Lastly, phasing can be another great way to defer project costs.
How are you preparing for future phases of data center operation?
Brook Gummere: Concurrent maintainability has a renewed focus in data centers to ensure both maintenance activities and system upgrades can occur without disrupting ongoing operations. With technology requirements changing so rapidly, we often design new infrastructure within the data center to support the new servers before the existing data center is fully operational. It is critical to keep existing systems operational while integrating the next generation of infrastructure needed to support emerging needs. An approach we often deploy is our partnership with the client’s facility engineering team and installing contractors to understand the impact of change to the existing infrastructure to avoid downtime. We prepare a risk assessment and mitigation plan with input from all stakeholders who can provide direction during the modification as well as a response plan to mitigate against unforeseen events.
Bill Kosik: Determining phasing and reliability targets is a fundamental part of data center design. The phasing plan must include incremental growth for the first phases and then design and building the subsequent phases without impacting the parts of the data center in use. For example, placing roof-mounted chillers and working above live data halls needs to be planned early in the design to make sure logistics are worked out and the crane has the required clearances to put equipment in place. The same is true for electrical gear including generators. Ken Urbanek: Phasing is very common in data center deployments. Providing infrastructure to accommodate expansion of the future phases is critical so that equipment (both MEP components and computing) can be easily plugged in upon delivery. Also, isolating parallel systems to avoid complex cross-connectivity later in the project is highly recommended wherever possible. Thorough review of concurrent maintainability not only helps with operational reliability but also with phasing and future connections.
