The Dell PowerEdge R640 10 Bay is one of the most popular server platforms that came out of the Intel Xeon Scalable generation. It features 10 SFF drive bays in the front. Unlike the 8 bay model, the 10 bay does have support for NVME drives. This makes the R640 10 bay an extremely versatile 1U server chassis. In the rear it can be configured with 3 half height expansion slots or 2 full height slots. Configuration depends entirely on chassis selection and riser setup. The most common configuration is 3 half height rear chassis.
Essential components
When assembling this server from an empty chassis, there are several essential components that need to be identified and verified before you begin. This guide is intended for anyone who has a server chassis in front of them and wants to turn it into a working system, but may not be familiar with the hardware or know exactly what is needed to get the server powered on and running a functional operating environment.
Motherboard
The chassis should already have the motherboard securely installed. The motherboard is the primary hub of the server, with virtually every other component connecting to it in some way. The power supplies connect to the motherboard from the rear of the chassis, while the processors and memory are installed directly on the board. The PCIe risers connect to their respective slots, the backplane is wired to allow drives to be installed from the front of the chassis, and the hot-swap fans connect directly to the motherboard.
Before continuing, verify that the motherboard is properly installed and secured inside the chassis.
Backplane and Required Cables
The second, and perhaps less obvious, piece of hardware you will need is the backplane, along with the required cables used to connect it to the motherboard and storage controller.
In most configurations, a backplane will require three separate connections. This also applies to rear flex-bay backplanes. The first is the signal cable, which connects the backplane to the system board. The second is the power cable, which supplies power to the backplane. The third is the SAS PERC cable, which connects the backplane to the RAID or HBA controller.
With this particular chassis, you will most likely be using one of Dell’s PERC cables to connect the backplane to a Mini Mono RAID/HBA controller. The 10 Bay model has a second power cable that supplies power to the expansion board.
Take a close look at the backplane and verify that all four required cables are present and properly connected. Without the necessary signal, power, and SAS connections, the server will not be able to properly communicate with or manage the installed drives.
Click to see picture of signal cable
Click to see picture of backplane power cable
Click to see picture of backplane expansion power cable
Click to see picture of PERC cable
Cooling
Proper cooling is essential to the operation of the R640, and several components work together to keep the processors and other internal hardware within their required operating temperatures.
The most obvious of these components are the cooling fans, highlighted in orange in the image above. On Dell servers, orange is generally used to identify components that are designed to be hot-swappable.
The fans slide into their designated positions in the chassis and make electrical contact with the motherboard through a set of pins. The R640 uses two different fan types: standard-performance and high-performance fans. The appropriate fan type depends on the server’s configuration. High-performance fans are recommended when using processors with a TDP above 165 watts, while standard-performance fans are suitable for lower-power configurations.
Regardless of which fan type you use, the chassis should have all 8 fan modules installed. Operating the server without the complete fan complement can result in hardware warnings and logged errors and may also lead to inadequate cooling under load.
The processors also require appropriate heatsinks. A dual-processor configuration will require two heatsinks, with the same standard and high-performance options available. Standard heatsinks are constructed entirely from aluminum, while high-performance heatsinks incorporate a copper contact surface to improve heat transfer away from the processor.
Processors with a TDP of 165 watts or greater should be paired with the high-performance heatsinks. Each heatsink also has a black plastic retention clip on its underside. This clip holds the processor against the heatsink and helps maintain the correct alignment during installation. The retention clip is a required part of the heatsink assembly and should not be removed.
Click to see standard-performance heatsinks
Click to see high-performance heatsinks
The final major cooling component is the air shroud. The shroud sits over the motherboard and directs airflow from the fans across the processors, memory, and other critical components. Without the shroud in place, airflow can be disrupted and the server may not cool its internal components as intended.
The fans, heatsinks, and air shroud should all be considered essential components of the R640 cooling system. Verify that all three are present and properly installed before attempting to operate the server.
Risers
The R640 10-bay configuration uses the same basic PCIe riser architecture as the 8-bay model. The most common setup consists of two risers providing three half-height, low-profile PCIe slots.
Riser 1 provides two low-profile x16 slots, both of which are connected to CPU1. Riser 2 is associated with CPU2 and provides an additional x16 slot. Because the PCIe lanes for Riser 2 are provided by the second processor, this riser will not function unless CPU2 is installed.
There is also a less common configuration that provides two full-height PCIe slots. These are also x16 slots, although the three-slot low-profile configuration is encountered more frequently.
If the server has the available risers, it is generally a good idea to install them even if you do not currently need every PCIe slot. Having the risers installed gives you additional expansion options if you decide to add a network adapter, HBA, storage controller, or other PCIe device later.
Click to see NVME configuration guide
NVME Configuration
The Dell PowerEdge R640 10 Bay supports NVME storage with the addition of some specialized cabling and an NVME controller. This is one of the major advantages of the 10 Bay chassis over the 8 Bay configuration. With the appropriate hardware, the system can be configured for 2, 4, 8, or all 10 front NVME drive bays.
There are three different Dell cable assemblies used to enable NVME connectivity in the 10 Bay chassis. We’ll start with the cable responsible for connecting the first two NVME bays.
This cable is essentially a Slim SAS cable that connects the NVME backplane to a dedicated NVME controller or expander card. Unlike the other NVME cables used in this configuration, this cable requires a separate controller card.
With the server facing you from the front, locate the connector on the lower-left side of the backplane. One end of the cable is labeled BP, which stands for backplane. Connect the BP end of the cable to the backplane.
There will likely be several other cables in this area that are in the way. You do not necessarily need to disconnect them, although temporarily moving them aside can make the installation easier.
The other end of the cable is labeled CTRL. Route this end alongside the backplane and then up the right side of the chassis.
Once the cable has been routed into position, you can install the NVME controller card.
Install the controller card into Riser 1 and connect the cable to the first port on the card.
At this point, the first two drive bays are enabled for NVME. If you only need two NVME drives, you can stop here. No additional cabling is required. Install the drives and the server is ready to use them.
If you want to enable additional bays, two more cable assemblies are required. Each of these cable sets provides NVME connectivity for four additional drive bays.
The cables labeled A0 and B0 provide connectivity for bays 6 through 9. The cables labeled A1 and B1 provide connectivity for bays 2 through 5. The cable we installed above provides connectivity for bays 0 and 1.
These cables can be somewhat difficult to install if you have never worked with this chassis before. There is very little room to work with and getting everything routed neatly takes some patience.
Here is the next cable assembly:
Move to the right side of the backplane where the remaining Slim SAS connectors are located.
The connectors on this cable are labeled A0 and B0. Connect each one to the matching A0 and B0 ports on the backplane.
These cables then need to be routed all the way toward the left side of the chassis. Follow the cable channel and route them toward the Slim SAS connectors located at the rear-left side of the motherboard.
Now we can install the second cable assembly, which is labeled A1 and B1.
Do your best to keep these cables tucked neatly into the chassis. Alongside the fans there are several hooks designed to hold the cables in place and prevent them from moving around.
When routing the cables through the left-side channel, I recommend temporarily pulling the existing cables out of the channel. You do not have to disconnect them. Simply moving them out of the way makes it much easier to route the NVME cables through the channel.
Once the cables are routed into position, you will see the corresponding connectors at the rear of the motherboard.
Locate the connectors labeled M1, M2, M3, and M4 and match the cables to their corresponding connectors.
Once all three cable assemblies have been installed, the R640 10 Bay will have NVME connectivity for all 10 front drive bays.
You do not have to install all three cable assemblies if you do not need all 10 bays. The NVME configuration can be built incrementally depending on how many drives you intend to use. If you only need a portion of the chassis configured for NVME, install only the cable assemblies required for those bays.
With all three cable assemblies installed, the R640 10 Bay is fully configured to support 10 NVME drives.
Building the Server
Once you have verified that the chassis contains the required components, you can begin assembling the server. Before installing every component, you can perform an initial power-on test using only the hardware required for the system to POST.
At a minimum, you will need:
- At least one CPU
- Memory
- At least one power supply
Required Tools
Fortunately, assembling an R640 does not require a large collection of tools. The basic tools and supplies you should have on hand are a Phillips #2 screwdriver, a T30 Torx bit, thermal paste, and either an electric air blower or a can of compressed air.
Keep in mind that a single-CPU configuration will disable some of the server’s hardware. Depending on the configuration, certain PCIe slots and even entire risers may be unavailable when CPU2 is not installed.
Installing the Processors
The two processor sockets are located near the center of the motherboard and are labeled CPU1 and CPU2. Depending on how the server was previously configured, the sockets may be covered by heatsinks or protective CPU socket covers.
Before installing a processor, remove the existing heatsink or socket cover. The blue retention clips must be released to remove these components. These clips can become brittle after years of exposure to heat, so do not be surprised if one breaks during removal. Replacement clips are available, although they are not strictly required for the server to operate.
With the socket exposed, take the opportunity to inspect the CPU pins carefully. Use compressed air to remove dust and debris, then examine the entire socket for bent, damaged, or crushed pins. The pins are extremely delicate, and even a small amount of damage can cause problems with memory channels, PCIe devices, or other portions of the system.
Once the socket has been inspected and the pins appear to be in good condition, the processor can be installed.
There are two common approaches to installing the CPU and heatsink. Some people attach the processor to the heatsink first and then install the assembly into the socket. Others, myself included, prefer to place the CPU directly into the socket and then install the heatsink on top.
For older R640 systems, I prefer the second method because the plastic retention clips on the heatsinks can become fragile with age. A processor that is not securely retained by the heatsink can fall out while the assembly is being positioned, potentially damaging the CPU socket pins.
Regardless of which method you choose, pay very close attention to the orientation of the processor. The gold alignment triangle on the CPU must correspond with the orientation marking on the motherboard socket.
Never force a processor into the socket. Installing the CPU in the wrong orientation can damage both the processor and the delicate socket pins.
After the CPU is correctly positioned, apply an appropriate amount of thermal paste to the processor. You want enough paste to provide full coverage between the CPU and heatsink without applying an excessive amount.
Position the heatsink over the processor, making sure the black plastic retention clip is installed correctly, and then secure the heatsink using a T30 bit.
If you are installing a second processor, repeat the same procedure for CPU2.
Installing the Memory
The R640 has six memory channels per processor, with two DIMM positions available per channel. This gives each CPU access to 12 DIMM slots, or 24 slots total in a dual-processor system.
The easiest way to populate the memory is to start with the primary DIMM in each channel. On the R640, these are the white DIMM slots. Once the primary slots have been populated, the secondary black slots can be used.
Ideally, memory should be distributed across as many channels as possible rather than filling both DIMMs in a single channel before using the remaining channels. When running two processors, the memory configuration should also be balanced between CPU1 and CPU2.
For example, if you have eight identical DIMMs to install in a dual-CPU system, a balanced configuration would be A1 through A4 on CPU1 and B1 through B4 on CPU2.
Click here to view the R640 memory channel mapping
| CPU (Socket) | Memory Channel | Primary DIMM (1 DPC) | Secondary DIMM (2 DPC) |
|---|---|---|---|
| Socket 01 (CPU1) | Channel 0 | A1 | A7 |
| Channel 1 | A2 | A8 | |
| Channel 2 | A3 | A9 | |
| Channel 3 | A4 | A10 | |
| Channel 4 | A5 | A11 | |
| Channel 5 | A6 | A12 | |
| Socket 02 (CPU2) | Channel 0 | B1 | B7 |
| Channel 1 | B2 | B8 | |
| Channel 2 | B3 | B9 | |
| Channel 3 | B4 | B10 | |
| Channel 4 | B5 | B11 | |
| Channel 5 | B6 | B12 |
If you are starting with a single DIMM, install it in A1.
For a dual-processor configuration, install the matching DIMM in B1.
Continue adding DIMMs according to the recommended population sequence, keeping the memory configuration balanced between the two processors whenever possible.
Installing a RAID Controller
The next step is to connect the drive backplane to a storage controller. The R640 can use the motherboard’s onboard SATA controller, but that configuration is limited to SATA storage. If you want to use SAS drives or take advantage of hardware RAID, you will need an appropriate PERC or HBA controller.
For this build, we will be installing an H730P Mini Mono controller.
Before installing the controller, make sure you have the correct PERC cable identified earlier in the guide. The backplane has two Mini-SAS HD connectors. The cable connects to these ports and is routed along the right side of the chassis before terminating at the Mini Mono controller location.
Position the controller in the slot, sliding the edge of the card underneath the black plastic retention clips.
Once the controller is seated, secure the PERC cable to the controller using a Phillips #2 screwdriver.
Installing the NDC
The Network Daughter Card, or NDC, provides the R640 with its primary onboard network interfaces. Installing it requires temporarily removing Riser 2 and the small black plastic cover that protects the NDC installation area.
Align the NDC so that its network ports line up with the opening in the rear of the chassis. Once the card is correctly positioned, press it down firmly until the connector on the bottom of the NDC seats into the corresponding connector on the motherboard.
Secure the NDC using a Phillips #2 screwdriver.
Once the NDC is installed, reinstall Riser 2 and the black plastic cover.
Installing the Power Supplies
The final major components to install are the power supplies. The PSUs are installed from the rear of the chassis and slide directly into their respective bays.
Push each power supply firmly into its bay until the locking mechanism engages and the PSU clicks into place.
At this point, the major components required to start the server have been installed. The R640 is now ready for its initial power-on test and, assuming the required hardware is present, should be ready to begin POST.
