Data centers used for bitcoin mining
Data centers used for bitcoin mining have significant differences from their commercial data center counterparts.
- Learn about the differences in cryptocurrency data centers as compared with their traditional data center counterparts.
- Understand the nuances in cooling and power systems for cryptocurrency data centers.
Back in 2009 Satoshi Nakamoto, regarded as the father of cryptocurrency, announced the first release of bitcoin. Nakamoto described it as a new electronic currency system, completely decentralized with no singular oversight, using a peer-to-peer network to prevent double-spending. Simply put, cryptocurrency is analogous to entries in a database that no one can change without fulfilling specific requirements. If you think about it, this is not so different than how a traditional bank account works.
Since bitcoin started in 2009, the use of cryptocurrency has continued to grow and gain in popularity. How data centers are used for mining cryptocurrency differ from other data centers in other industries, especially in regard to their power and cooling systems.
Defining bitcoin mining and mining data centers
At a high level, the secure hash algorithm (SHA) is a function that is used to validate bitcoin transactions and ensure the security for the bitcoin network’s public ledger, also known as the blockchain. The speed at which bitcoins are mined is measured in hashes per second. The servers used in mining (referred to as "miners" or "mining servers") bundle recent bitcoin transactions into "blocks," then work to solve cryptographic problems to help validate each block, making sure the ledger entries are accurate. These cryptographic problems are where the mining servers and data centers come into play. Solving these problems requires heavy-duty computational power operating for long periods of time.
The bitcoin network pays bitcoin miners for their time, effort, and financial investment by releasing bitcoins to those who contribute the needed computational power to validate the transactions. The greater computational power a miner has, the greater the portion of compensation—this is the overarching driver for why individuals and corporations are building megawatt bitcoin mining data centers, either to be used by themselves or for paying customers who then have access to mining servers without having to make major capital investments in information technology (IT) and facilities. In either scenario, minimizing first costs and ongoing energy costs is critical to maximizing return on investment (ROI).
Fundamentally, a mining data center shares the same basic design and operational principles as other types of data centers: Power is brought to the building and distributed to the equipment, air-distribution systems maintain the required environmental conditions, and the building provides protection from outside conditions and security from external threats. Although on a deeper level, there are significant differences to data centers that are used for mining than their commercial data center counterparts. This divergence is readily seen in the examination of the following categories:
- Impacts of mining server design
- Data center structure and envelope
- Cooling and air distribution
- Energy use and efficiency.
Analyzing the data on the growth of bitcoin from 2014 to 2018 indicates a tremendous growth in mining activities. This is an important indicator of energy use linked to mining operations. The growth of mining (and energy use) has been extremely strong from 2014 to 2018, and trend analyses show a continued positive, aggressive growth. Taking into consideration the very rapid growth that is occurring, and projections, it is imperative to focus on energy-reduction strategies (Figure 1).
Mining server design
Two major considerations when investing in mining servers is the first-cost per hash and the electrical efficiency stated in watts per hash. Higher-performing computers have higher hash rates, providing greater computational power to the mining operation. In contrast to enterprise servers, miners are designed to accomplish only one task-mining. Currently, a common type of architecture used for mining servers is based on the application-specific integrated circuit (ASIC) chips, often referred to as system on a chip (SoC).
When developing a cooling system strategy, an important consideration is that miners can operate with inlet conditions of 80 to 90°F and10% to 80% non-condensing relative humidity. A powerful mining server can have an electrical demand of 1,400 W or more, dissipating the equivalent quantity of heat to the data center. To minimize the possibility of server failure due to high interior temperatures, some of the server manufacturers include a controller that varies the speed of the fan in the server, the voltage, and clock speed of the machine based on the temperature. Additionally, the larger cross-sectional area of the miner allows for better airflow across the ASIC chips, enabling effective heat dissipation. Being able to take advantage of these higher operating temperatures, which can reduce or eliminate the requirements of the cooling system, is a central organizing principle for mining data centers. The extent of the reduction of cooling equipment is dependent on the size, location, and physical characteristic of the data center building.
Structure and envelope
Typically, mining data centers use buildings that are constructed of lightweight materials-including the exterior walls, roof, and windows—such as a storage facility or warehouse. This construction is akin to a Level 1 basic facility as defined in the Telecommunications Industry Association (TIA) 942 Standard. A Level 1 basic facility has the least resiliency of the four levels in terms of systems reliability, handling extreme weather events, security, and many other criteria. The standard further defines a Level 1 basic facility as being prone to operational errors or spontaneous failures of site infrastructure components that result in a data center disruption. A thorough risk assessment and analysis is necessary when contemplating constructing this type of data center, as power/cooling outages or damage to the structure could lead to a catastrophic outcome.
A Level 1 basic facility data center will have little or no redundancy in the cooling systems. Lower reliability systems do not use redundant equipment, such as pumps, chillers, and air-handling equipment. In this scenario, mechanical rooms are smaller because there is less equipment. This "found" space will be used for mining servers, which is advantageous for the mining operation but increases the required power and cooling capacity. The inside of the facility is typically a large, high-bay warehouse-style interior, which allows for the flexibility and working space necessary when installing thousands of mining servers.
Cooling and air distribution
In a traditional data center, the servers are mounted in a cabinet or rack that secures the server in place, allows for cable management, and aids in airflow. In a mining data center, the servers are mounted on industrial shelving units, allowing for quick replacement in case of server failure. This shelving arrangement offers cost advantages for procuring the products and labor to install the shelving.
One of the advantages of using the industrial-type shelving to hold the computers is the openness of the installation. The miners are placed on the shelving in a manner that permits the air to flow above, below, and on both sides. In this arrangement, there is no formal airflow management, such as a hot-aisle/cold-aisle configuration; the air temperature entering the servers is nonhomogeneous and will vary greatly. The temperatures are generated by a combination of convective forces and air mixing at the discharge of the server fans. When comparing this with an enterprise data center, there are major differences. A typical enterprise data center has servers stacked within cabinets, and the cabinets are placed side by side. The intent is to create airflow patterns that are generally very controlled, with the inlet at the face of the server and discharge from the back. This arrangement results in much more predictable and controlled air-temperature gradients.
In data centers, the cooling systems are among the most expensive and energy-hungry (behind the servers). Since this is the case, reducing or eliminating components like chillers, cooling towers, pumps, piping, and ductwork will reduce or eliminate most of the cooling system. Reducing the use or eliminating these systems will also address the first-cost and energy-cost issues. Since the primary job of the cooling system is to keep the IT equipment operating at a prescribed temperature and moisture level, easing the indoor environmental requirements of the servers (for example, allowing them to operate at a higher internal temperatures) will reduce energy consumption and, in some cases, reduce the size of the cooling systems.
As previously mentioned, the mining servers can operate with air temperatures ranging from 80 to 90°F and beyond. If the outdoor air is approximately equal to the maximum allowable server temperature, no mechanical cooling is required. Therefore, the data center’s geographic location and the server’s maximum operating temperature must be taken into consideration in tandem; cooler locations and hotter server operating temperatures will have the lowest energy use, while the hottest locations and lowest server temperatures will use the most energy (Figure 2).
Energy efficiency for mining operations
Energy use is a primary concern for mining operations. If operating costs are higher than what is needed for a favorable financial return on the mining operation, the business model will be a non-starter. To illustrate this point, in 2009 when bitcoin launched, each block created was worth 50 bitcoins. By design, this figure is scheduled to fall by half every 4 years: 25 bitcoins in 2012, 12.5 bitcoins in 2016, and 6.25 bitcoins in 2020. When the mining industry’s revenue falls by half, its energy consumption must fall proportionately. If it doesn’t fall, mining would become an unprofitable activity. It is necessary to control energy consumption and cost by upfront analysis on location, system type, server performance, etc.
To understand how different components impact the total cost of ownership (simple payback in this example), Figure 3 illustrates the sensitivity of simple payback to different variables.
- The mining server has an electrical demand of 1,620 W.
- The server has a hash rate of 18 T-H/s.
- The server’s first cost is $4,800.
- The electricity rate is $0.10/kWh.
- The server will mine the equivalent of $3,200/year in bitcoins.
- The data center’s cooling system power (watts per watt of server power) is 0.392.
- Data center construction costs (dollars per watt of server power) equal $3 per watt.
Discussion on Results
- When performing this type of analysis, one of the assumptions is that when a single variable is altered, the other variables remain static. This is generally true for this analysis, except for Case 3 where the cooling energy was reduced commensurate with the reduction in server power.
- As in all data center projects, reducing the electrical load for the IT equipment will have the greatest impact on reducing overall energy use.
- In any analysis of data center energy consumption, studying the IT load is the starting point as all energy use cascades down to the other systems.
- Increasing the server operating temperature will reduce the payback by 5%. The risk of an increased server failure rate must be taken into consideration, including annual replacement costs.
- Case 4 assumed the ability to obtain the base case servers, but for 20% less. The power and hash rates are the same.
- The electrical utility rate is a fixed-rate structure: Time-of-day rates or demand charges are not included. Depending on the area of the United States, these can have a significant impact on total cost of ownership.
- The type of building assumed in this analysis is a Tier 1 facility of simple construction. Building a more robust building, like a Tier 2 or Tier 3, will add a significant amount of costs.
This analysis is meant to provide a high-level snapshot of how a simple payback is impacted by manipulating different variables. It is not exhaustive and can’t account for different situations and parameters that could further influence the total cost of ownership. As an example, installations that are highly electrically dense will likely need supplemental cooling to avoid areas of extreme heat buildup, even if outdoor air is being used to cool the data center. A nuance like this is not covered in this type of analysis.
Unlike enterprise servers where it is difficult (if not impossible) to draw a one-to-one correlation between server energy use and financial return, this correlation is readily obtainable from mining operations. This is possible, in part, because while enterprise servers will handle a multitude of different applications, the mining servers are designed to do only one thing—mining. Understanding the influencing parameters when planning a new mining data center will provide valuable data and analysis techniques to maximize the owner’s ROI.
Bill Kosik is a data center energy efficiency strategist. In the mission critical industry, he is a subject matter expert in research, analysis, strategy, and planning in the reduction of data center energy consumption, water use, and indirect greenhouse gas emissions. Kosik is a member of the Consulting-Specifying Engineer editorial advisory board.