Category: Troubleshooting

This is an error you see on 15 gen Dell servers like the PowerEdge R750 and the VxRail V670F. In almost all cases it’s because you have the DIMMs in the wrong slots. The memory population rules for this generation are similar to the generations before it – You populate A1, A2, A3, B1, B2, B3 etc. In this case I had DIMMS A1 and A2 populated correctly but I had the DIMMs for CPU2 populated in B1 and B8 instead of B1 and B2.

The exact slot numbering varies depending on the server model and the number of DIMMs being installed, but the key principle remains the same: populate the lowest-numbered slot in each memory channel before moving to the next slot.

You may also see the following log entry in iDRAC:

A diagnostic warning event occurred in the memory device at location identified in the message. The device may be operating in a degraded state. Checking the device and system configuration is recommended for remediation.

Because the second DIMM was installed in B8 instead of B2, the system generated memory diagnostic warnings even though both DIMMs were fully functional.

Once the DIMMs were moved into the correct slots, the warning disappeared and the system reported the memory configuration as healthy.

How to Troubleshoot

If you encounter this error on a Dell 15G server, work through the following checks before assuming a hardware failure:

1. Verify that each DIMM is fully seated.
2. Check that every DIMM is installed in the correct slot according to the Dell Memory Population Guidelines for your server model.
3. Ensure both CPUs have memory installed symmetrically if your configuration requires it.
4. Confirm that all DIMMs are of compatible type, speed, and capacity.
5. Clear any existing hardware logs, reboot the server, and verify whether the warning returns.

If the warning persists after confirming the population order, then it becomes worthwhile to investigate individual DIMMs by swapping modules or running the built-in Dell diagnostics.

Why Population Order Matters

Modern Intel Xeon platforms use multiple memory channels per processor. Installing DIMMs in the wrong slots can leave channels partially populated or cause the memory controller to operate outside its recommended configuration.

The result may include:

• Memory operating in a degraded configuration
• Reduced memory bandwidth
• POST warnings
• iDRAC diagnostic events
• In some cases, memory not being detected at all

The hardware itself is often perfectly healthy—the issue is simply that the DIMMs are not installed in the order expected by the memory controller.

Final Thoughts

When working with Dell 15th Generation servers such as the PowerEdge R750 or VxRail V670F, don’t immediately assume a DIMM has failed when you see memory diagnostic events.

The first thing to check should always be the memory population layout. A quick comparison against Dell’s population diagram can save a significant amount of troubleshooting time. In my case, moving a single DIMM from B8 to B2 completely resolved the issue with no hardware replacement required.

Sometimes memory errors don’t give you the exact problematic DIMM slot in the log. Instead you’ll see something like the following:

One or more memory errors have occurred during the Double Data Rate (DDR) memory channel initialization on the memory slot of Socket 02, Channel 3.

This is not exactly intuitive.

Use the following table to track the error to the actual DIMM slot. Note that since there are 2 DIMMs per channel you may have to troubleshoot 2 different sticks if both slots in the channel are populated. If only one stick in the channel is populated then the error always refers to that specific DIMM. Such errors will only ever refer to the problematic processor (Socket 01/02) and the associated channel, which can include up to 2 DIMMs if you happen to have them populated.

CPU (Socket)	Memory Channel	Primary DIMM (1 DPC)	Secondary DIMM (2 DPC)
Socket 01 (CPU1)	Channel 0	A1	A7
	Channel 1	A2	A8
	Channel 2	A3	A9
	Channel 3	A4	A10
	Channel 4	A5	A11
	Channel 5	A6	A12
Socket 02 (CPU2)	Channel 0	B1	B7
	Channel 1	B2	B8
	Channel 2	B3	B9
	Channel 3	B4	B10
	Channel 4	B5	B11
	Channel 5	B6	B12

I came across a 4 Bay LFF Supermicro server at work (X11SSH-F) and needed to give it an update.

This particular server allowed me to update the BMC but required a license to install the BIOS as seen in the following screenshot:

This is always frustrating when you’re just trying to complete the most bonehead level maintenance. Googling the error led me to the following reddit post which led me to the following perl script on Github.

I thought I would expand upon these resources with a step by step guide of downloading the script, running the script, and taking care of a few dependencies to make the script work.

I accomplished all of the following by booting a live version of Ubuntu. If you already have Ubuntu or some other distribution installed on your server that is just as good.

So the first order of business is booting into Ubuntu before following the next steps.

Step 1: Check if Perl Is Installed

Most Ubuntu installations already include Perl. To verify, open a terminal and run:

perl -v

If Perl is installed, you’ll see version information displayed in the terminal:

If you get a “command not found” message, install Perl with:

sudo apt update
sudo apt install perl

Step 2: Install the Required Perl Module

The script uses the Digest::SHA module to calculate an HMAC-SHA1 hash. Ubuntu provides this module as a package.

Install it by running:

sudo apt install libdigest-sha-perl

Alternatively, if you prefer using CPAN, you can install it with:

cpan Digest::SHA

Step 3: Save the Script

Copy the Perl code from this github page into a new file. For example:

supermicro-ipmi-key.pl

Save the file somewhere convenient, such as your home directory.

Step 4: Make the Script Executable (Optional)

You can run the script directly by making it executable:

chmod +x supermicro-ipmi-key.pl

While this step isn’t strictly necessary, it makes running the script a little cleaner.

Step 5: Run the Script

To run the script you simply invoke it and pass the MAC address of your BMC as an argument.You can find the MAC here in the IPMI web admin page:

 

You have two options.

Run it directly:

./supermicro-ipmi-key.pl 00:25:90:12:34:56

Or execute it with Perl:

perl supermicro-ipmi-key.pl 00:25:90:12:34:56

Replace the sample MAC address with the MAC address of your server’s BMC (IPMI) interface.

If everything is configured correctly, you’ll receive a generated key similar to:

3A1F 7B82 C4D0 E1F9 A5B3 8C2D

Punch this into the license field located here:

Troubleshooting Common Errors

“Can’t locate Digest/SHA.pm in @INC”

This means the required module isn’t installed.

Simply run:

sudo apt install libdigest-sha-perl

and try again.

“Usage: supermicro-ipmi-key <MAC>”

This message appears when no MAC address is provided.

Be sure to include the BMC MAC address when running the script.

“Invalid mac address”

The script expects the MAC address in the standard format:

AA:BB:CC:DD:EE:FF

Double-check that your address uses six hexadecimal pairs separated by colons.