Blog

  • Dell PowerEdge R640 8 Bay Configuration and Build Guide

    Dell PowerEdge R640 8 Bay Configuration and Build Guide

    The Dell PowerEdge R640 8 bay is a 1U chassis outfitted with 8 SFF SAS/Sata drive bays. The platform supports both generation 1 and generation 2 Intel Xeon Scalable processors as well as DDR4 ECC RDIMMs and LRDIMMs. This particular chassis has rather simple configuration options compared to its 10 bay older brother, which supports u.2 NVME drives. The 8 bay chassis has no support for NVME in the front drive bays due to no built in connections on the backplane.

    In the rear the most common configuration is 3 half height expansion slots supported by risers 1 and 2 but Dell did make a chassis with 2 full height slots. Both of these configurations require different risers.

    Essential components

    If you’re building this server from the ground up you should first verify a few critical components required. These guides are written for someone who has a chassis in front of them, needs to get the chassis operational, but has almost no clue what he’s looking at or what is required to boot a functional environment.

    Motherboard

    The chassis should obviously have the motherboard screwed in. This is the primary hub where everything else feeds in. The power supplies will connect to the board from the rear, Both processors and memory will be installed on the board, the risers connect into the appropriate slots, the backplane will be wired up so you can stick drives through the front, and of course the hot swap fans will go straight into the motherboard. Verify the board is in the husk of the chassis.

    Backplane and required cables

    The second perhaps not so obvious piece of hardware is the backplane and the 2 required cables needed to connect the backplane to the motherboard. In most cases, any backplane (this includes rear flex bay backplanes as well) will have 3 required cables. The first cable is a signal cable, the second cable is a power cable, and the third cable will be a SAS PERC cable used to connect the backplane to a RAID/HBA controller. With this particular chassis you will most likely be using one of Dells PERC cables to attach a mini mono RAID/HBA controller. Take a look at the backplane and ensure all 3 cables are present. Without all 3 you will not have the ability to configure any kind of storage arrays.

    Click to see picture of backplane
    Backplane for the Dell PowerEdge R640 094J5V
    Here is a picture of the backplane outside of the server. Inside the server it will snap into position and connect to the motherboard using the signal and power cables
    Click to see picture of signal cable
    BP signal cable for the Dell PowerEdge R640 8 Bay server
    Signal cable
    Click to see picture of power cable
    BP Power Cable for the Dell PowerEdge R640
    Power cable
    Click to see picture of PERC cable
    PERC storage cable for the Dell PowerEdge R640 8 Bay server
    PERC storage cable.

    Cooling

    Dell PowerEdge R640 cooling fans

    There are a number of critical components the server needs for proper cooling and thermal regulation.

    The first main component is of course the fans identified by the orange highlights. In general, the color orange indicates that a particular component is able to be hot swapped.

    These are partially slotted in to fixed positions on the server chassis and then connect to the motherboard through a set of pins. There are 2 variations of fans – standard performance and high performance. Certain configurations may require high performance fans. If you’re using a CPU with a TDP higher than 165 watts they come recommended. Otherwise, standard performance will work. Either way you want to make sure the server has all 8 fans installed otherwise the server will not only complain and generate logs but you may run into serious thermal issues.

    Another component critical for cooling is the heatsink. You will need 2 of these if you have 2 processors. These also come in standard and high performance variations. Standard is all aluminum. High performance have a copper face that draws more heat away from the CPU. Any processor >=165 watts should use high performance heatsinks. On the bottom of the heatsink is a black plastic clip. The purpose of these clips are to retain the CPU onto the heatsink and ensure proper alignment with the CPU pins. These are necessary.

    Click to see standard performance heatsinks

    R640 standard performance heatsinks

    Click to see high performance heatsinks

    Dell PowerEdge R640 high performance heatsinks

    The last component related to cooling is the airshroud. The airshroud directs the cool air coming in from the server to critical components.

    Both the fans, heatsinks, and air shroud are considered critical components to ensure proper operation of the server.

    Risers

    Dell PowerEdge R640 risers 1 and 2

    Both the R640 8 and 10 bay models use the same risers to provide PCIe connectivity to the system. The most common configuration is the 3 half height slot setup. This configuration has 2 risers. Riser 1 provides two low profile x16 half height slots. Both these slots are linked to CPU1. Riser 2 is linked to the second processor and provides a single x16 slot to the system. Riser 2 will not function without the presence of CPU2.

    As mentioned there is also a chassis that has 2 full height slots. These are also x16 slots although this configuration is less common than the 3 slot low profile chassis. It is a good idea to have all available risers installed. Even if you don’t need them now you may need them in the future.

    Building the server

    You can test the server by installing the minimum amount of components required to boot. To boot you will need at least one of the following:

    • CPU
    • Memory
    • Power supply

    Required tools

    tools needed to build R640

    Not many tools are needed for this job. You will at the bare minimum need a Phillips head #2. A torque bit #30, some thermal paste, and electric air blower or can of compressed air.

    Once again please note that in single CPU server configurations some components may be disabled – for example some slots or even entire risers could be rendered unusable.

    Installing the processors

    You will notice the slots for the CPU near the center of the chassis. They might be covered by the heatsinks or they could have a dummy CPU cover installed. The sockets will be labeled CPU1 and CPU2. Remove the heatsink or the cover prior to installing the CPU. Removal of the heatsink will reveal the CPU pins. You will have to depress the blue clips to remove the heatsinks and/or the CPU dummy cover. Don’t be alarmed if the clips break – this is common and unavoidable. You can replace them but its not entirely necessary.

    Dell PowerEdge R640 heatsink retention clips
    Make note of the blue heatsink retention clips and don’t worry if they snap off from years of heat exposure

    Now is the time to do a proper visual inspection of the CPU pins. Start by grabbing a can of compressed air and blow out any debris and dust. Look closely at the pins. They should not be bent or smashed in. Bent pins can cause a host of issues with memory and other aspects of the server like PCIe devices.

    Dell PowerEdge R640 CPU pins

    If you’ve determined the pins are in good condition you can now place the CPUs into the socket. Some people like to first snap the CPU into the heatsink prior to installing. I like to place the CPU into the socket by hand and then snap the heatsink on top of it. Do whatever is most comfortable for you. The reason I don’t always install the CPU into the heatsink is the fragility of the retention clips. At this point these servers are getting quite old and the plastic doesn’t always secure the CPU. I’ve had processors fall out of the heatsink as I flipped them down to install into the socket, and I’ve damaged pins as a result.

    The most important part of this step is aligning the gold arrow on the CPU with the orientation arrow on the motherboard. Installing the CPU backwards can damage both the pins and the CPU if enough force is applied.

    Aligning the CPU arrow with the motherboard arrow
    Make note of the alignment arrows and place the CPU into the socket with great care

    Apply thermal paste in an amount that will spread evenly across the entire face of the processor. Screw down the heatsink with a T30 bit.

    This is just about the right amount of thermal paste.
    After applying thermal paste snap the heatsink on top. Be sure the black plastic retention clip is installed prior to installation. Use the T30 bit to screw it down securely

    If you have a second CPU go ahead and populate the socket using the same method.

    Installing the memory

    Next, the memory can be installed. For the R640 platform there are 6 channels per CPU with each channel supporting 2 DIMMs each. This means for each processor there are a total of 12 DIMMs. I will include a channel mapping table below but you don’t really need it – simply populate the white DIMMs first and if you run out of those start populating the black DIMMs. Each white DIMM is a different channel and it’s recommended to spread memory across as many channels as you can. However you populate the DIMMs on one processor you should mirror the configuration on the other.

    Let’s say you have a total of 8 sticks you want to spread across both processors. The recommended configuration would be to populate DIMMs A1,A2,A3,A4 and then DIMMs B1,B2,B3,B4 on the second processor.

    Click here to see R640 memory channel mapping table
    CPU (Socket) Memory Channel Primary DIMM (1 DPC) Secondary DIMM (2 DPC)
    Socket 01 (CPU1) Channel 0 A1 A7
    Channel 1 A2 A8
    Channel 2 A3 A9
    Channel 3 A4 A10
    Channel 4 A5 A11
    Channel 5 A6 A12
    Socket 02 (CPU2) Channel 0 B1 B7
    Channel 1 B2 B8
    Channel 2 B3 B9
    Channel 3 B4 B10
    Channel 4 B5 B11
    Channel 5 B6 B12

    Now install the DIMM into the primary DIMM of the first channel (A1)

    Install a stick of ram into the DIMM
    The air shroud is a great way to see where you’re installing the DIMMs.

    Now mirror the configuration for the second CPU (B1):

    Installing B1 memory module on R640

    If you have more memory continue to populate the DIMMs in the order suggested above.

    Installing a RAID controller

    In order for drives to be exposed on the system you will need to wire the backplane to a RAID or HBA controller. You can wire the backplane directly to the motherboards onboard controller but this will only expose Sata drives. If you want to use both SAS and Sata drives you will need to install a controller. In this guide we will install an H730P mini mono controller.

    First make sure you have the cable mentioned at the beginning of this guide. There are 2 mini SAS HD ports on the backplane. The cable will plug into these ports and be routed along the right side of the chassis where it terminates at the mini mono slot below:

    Here is the installation point for the raid controller

    Place the controller in the slot by sliding edge of the controller under the black plastic retention clips:

    The controller installs like this

    finally, screw down PERC cable with a phillips head #2:

    screwing down the PERC raid controller

    Installing the NDC

    Now it’s time to install the NDC (network daughter card.) To achieve this you must first remove riser 2 as well as the black plastic cover shown in the photo below:

    R640 NDC installation point
    Riser 2 and the little black plastic cover must be removed prior to installation of the NDC
    line up the side of the NDC with networking ports with rear expansion slot. Once fitted in the slot press down so the NDC snaps into the connector on the motherboard.
    Now screw the NDC into place using a phillips head #2.

    At this point reinstall both riser 2 and the black plastic cover.

    Installing the power supplies

    The last component to install are the power supplies. They slide in from the rear of the chassis.

    installing power supplies on the R640
    Orient the power supplies like so in the rear of the chassis
    Slide them in firmly. You will hear them click into place.

    The server is now ready to be powered up.

    Compatible parts

    For a complete list of compatible parts for this server please read the following post. 

  • UEFI0401: Unable to use system memory because the DIMMS are populated in an unsupported configuration

    UEFI0401: Unable to use system memory because the DIMMS are populated in an unsupported configuration

    This is an error you see on 15 gen Dell servers like the PowerEdge R750 and the VxRail V670F. In almost all cases it’s because you have the DIMMs in the wrong slots. The memory population rules for this generation are similar to the generations before it – You populate A1, A2, A3, B1, B2, B3 etc. In this case I had DIMMS A1 and A2 populated correctly but I had the DIMMs for CPU2 populated in B1 and B8 instead of B1 and B2.

    The exact slot numbering varies depending on the server model and the number of DIMMs being installed, but the key principle remains the same: populate the lowest-numbered slot in each memory channel before moving to the next slot.

    You may also see the following log entry in iDRAC:

    A diagnostic warning event occurred in the memory device at location identified in the message. The device may be operating in a degraded state. Checking the device and system configuration is recommended for remediation.

    Because the second DIMM was installed in B8 instead of B2, the system generated memory diagnostic warnings even though both DIMMs were fully functional.

    Once the DIMMs were moved into the correct slots, the warning disappeared and the system reported the memory configuration as healthy.

    How to Troubleshoot

    If you encounter this error on a Dell 15G server, work through the following checks before assuming a hardware failure:

    1. Verify that each DIMM is fully seated.
    2. Check that every DIMM is installed in the correct slot according to the Dell Memory Population Guidelines for your server model.
    3. Ensure both CPUs have memory installed symmetrically if your configuration requires it.
    4. Confirm that all DIMMs are of compatible type, speed, and capacity.
    5. Clear any existing hardware logs, reboot the server, and verify whether the warning returns.

    If the warning persists after confirming the population order, then it becomes worthwhile to investigate individual DIMMs by swapping modules or running the built-in Dell diagnostics.

    Why Population Order Matters

    Modern Intel Xeon platforms use multiple memory channels per processor. Installing DIMMs in the wrong slots can leave channels partially populated or cause the memory controller to operate outside its recommended configuration.

    The result may include:

    • Memory operating in a degraded configuration
    • Reduced memory bandwidth
    • POST warnings
    • iDRAC diagnostic events
    • In some cases, memory not being detected at all

    The hardware itself is often perfectly healthy—the issue is simply that the DIMMs are not installed in the order expected by the memory controller.

    Final Thoughts

    When working with Dell 15th Generation servers such as the PowerEdge R750 or VxRail V670F, don’t immediately assume a DIMM has failed when you see memory diagnostic events.

    The first thing to check should always be the memory population layout. A quick comparison against Dell’s population diagram can save a significant amount of troubleshooting time. In my case, moving a single DIMM from B8 to B2 completely resolved the issue with no hardware replacement required.

  • Dell PowerEdge R640 Memory Channel Mapping Table

    Dell PowerEdge R640 Memory Channel Mapping Table

    Sometimes memory errors don’t give you the exact problematic DIMM slot in the log. Instead you’ll see something like the following:

    One or more memory errors have occurred during the Double Data Rate (DDR) memory channel initialization on the memory slot of Socket 02, Channel 3.

    This is not exactly intuitive.

    Use the following table to track the error to the actual DIMM slot. Note that since there are 2 DIMMs per channel you may have to troubleshoot 2 different sticks if both slots in the channel are populated. If only one stick in the channel is populated then the error always refers to that specific DIMM. Such errors will only ever refer to the problematic processor (Socket 01/02) and the associated channel, which can include up to 2 DIMMs if you happen to have them populated.

    CPU (Socket) Memory Channel Primary DIMM (1 DPC) Secondary DIMM (2 DPC)
    Socket 01 (CPU1) Channel 0 A1 A7
    Channel 1 A2 A8
    Channel 2 A3 A9
    Channel 3 A4 A10
    Channel 4 A5 A11
    Channel 5 A6 A12
    Socket 02 (CPU2) Channel 0 B1 B7
    Channel 1 B2 B8
    Channel 2 B3 B9
    Channel 3 B4 B10
    Channel 4 B5 B11
    Channel 5 B6 B12
  • Generating Supermicro OOB Per Node License

    Generating Supermicro OOB Per Node License

    I came across a 4 Bay LFF Supermicro server at work (X11SSH-F) and needed to give it an update.

    This particular server allowed me to update the BMC but required a license to install the BIOS as seen in the following screenshot:

    This license is required by Supermicro to update the bios

    This is always frustrating when you’re just trying to complete the most bonehead level maintenance. Googling the error led me to the following reddit post which led me to the following perl script on Github.

    I thought I would expand upon these resources with a step by step guide of downloading the script, running the script, and taking care of a few dependencies to make the script work.

    I accomplished all of the following by booting a live version of Ubuntu. If you already have Ubuntu or some other distribution installed on your server that is just as good.

    So the first order of business is booting into Ubuntu before following the next steps.

    Step 1: Check if Perl Is Installed

    Most Ubuntu installations already include Perl. To verify, open a terminal and run:

    perl -v

    If Perl is installed, you’ll see version information displayed in the terminal:

    Checking to see if Perl is installed

    If you get a “command not found” message, install Perl with:

    sudo apt update
    sudo apt install perl

    Step 2: Install the Required Perl Module

    The script uses the Digest::SHA module to calculate an HMAC-SHA1 hash. Ubuntu provides this module as a package.

    Install it by running:

    sudo apt install libdigest-sha-perl

    Alternatively, if you prefer using CPAN, you can install it with:

    cpan Digest::SHA
    
    Install lib digest

    Step 3: Save the Script

    Copy the Perl code from this github page into a new file. For example:

    supermicro-ipmi-key.pl

    Save the file somewhere convenient, such as your home directory.

    Step 4: Make the Script Executable (Optional)

    You can run the script directly by making it executable:

    chmod +x supermicro-ipmi-key.pl

    While this step isn’t strictly necessary, it makes running the script a little cleaner.

    Step 5: Run the Script

    To run the script you simply invoke it and pass the MAC address of your BMC as an argument.You can find the MAC here in the IPMI web admin page:

     

    You have two options.

    Run it directly:

    ./supermicro-ipmi-key.pl 00:25:90:12:34:56

    Or execute it with Perl:

    perl supermicro-ipmi-key.pl 00:25:90:12:34:56

    Replace the sample MAC address with the MAC address of your server’s BMC (IPMI) interface.

    If everything is configured correctly, you’ll receive a generated key similar to:

    3A1F 7B82 C4D0 E1F9 A5B3 8C2D

    Punch this into the license field located here:

    Troubleshooting Common Errors

    “Can’t locate Digest/SHA.pm in @INC”

    This means the required module isn’t installed.

    Simply run:

    sudo apt install libdigest-sha-perl

    and try again.

    “Usage: supermicro-ipmi-key <MAC>”

    This message appears when no MAC address is provided.

    Be sure to include the BMC MAC address when running the script.

    “Invalid mac address”

    The script expects the MAC address in the standard format:

    AA:BB:CC:DD:EE:FF

    Double-check that your address uses six hexadecimal pairs separated by colons.

  • SOLARFLARE SFC-9120 Will Spin Up The Fans On R740xd

    SOLARFLARE SFC-9120 Will Spin Up The Fans On R740xd

    This is just an observation and public service announcement. If you have the Solarflare SFC-9120 10Gb SFP+ card installed in an R740xd it will spin up the fans and be louder than it needs to be. Most likely due to iDRAC not being able to read the temperature of the card. This is common with Solarflare cards.

    Here’s a screenshot of the fan speed at idle load without the card:

    And here’s a shot with the card installed:

    At 48% PWM the server is quite loud even with high performance fans installed.