r/LocalLLaMA · · 3 min read

Need assistance in getting my 4 AMD GPU setup running on a Z10PA-U8 mainboard

Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.

I'm trying to run 4x AMD RX 6900 XT and an NVIDIA GT 710 (x8) display card on an ASUS Z10PA-U8 mainboard and Ubuntu Server 24.04. But so far this doesn't work so well.

I have an old GA-Z77-DS3H mainboard which has 1 PCIe x16 (Gen3), 2 PCIe x1 (Gen2) and 1 PCIe x16 (Gen2) slots. All 4 GPUs work on that board with the Delock x16->x16 and x16->x8 riser cables I have. So I know the GPUs are fine. But I'm only getting ~7 t/s output with a 40B model. I was hoping that switching to a mainboard that supports 4 x8 Gen3 slots would result in a good performance boost.

The ASUS Z10PA-U8 mainboard has the following PCI(e) slots setup:

Slot Description
1 PCI-E x8 (x4 Gen2 link)
2 PCI
3 PCI-E x8 (x8 Gen3 Link)
4 PCI-E x16 (x16 Gen3 Link) (Auto switch to x8 Link if slot 3 is occupied)
5 PCI-E x8 (x8 Gen3 Link)
6 PCI-E x16 (x16 Gen3 Link) (Auto switch to x8 Link if slot 5 is occupied)

The idea was to put the AMD GPUs in slot #6, #5, #4, #3 and the display card in slot #1. But with this setup only the display card in slot #1 and the AMD GPUs in slot #6 and slot #4 are recognized. Also slot #4 is producing a lot of AER error messages during Ubuntu booting. I was able to get rid of these error messages by forcing slot #4 to run on Gen2 using setpci.

But the AMD GPUs in slot #5 and slot #3 are not recognized at all. The system doesn't see them, they don't show up in lspci | grep AMD, only the two AMD GPUs in slot #6 and #4.

However, I can get 3 AMD GPUs running in slot #1 (PCIe x8 Gen2), slot #4 (PCIe x16 Gen3 forced to Gen2) and slot #6 (PCIe x16 Gen3) while having the display card in slot #5 or slot #3. The same 40B model generates ~30 t/s.

So I haven't found a solution to get all 4 AMD GPUs running on this board. Weird thing is that the x8 NVIDIA GT 710 works fine in slot #3 and slot #5. So the slots are working, they just don't want to communicate with any of my x16->x16 or x16->x8 riser cables. I've already tried different riser cables and also a riser card, with the same results:

  • slot #1 -> any riser card / any riser cable -> any RX / GT 710 = works
  • slot #3/#5 -> riser cable -> RX = not recognized
  • slot #3/#5 -> riser card -> RX = not recognized
  • slot #3/#5 -> riser cable -> GT 710 = works
  • slot #3/#5 -> riser card -> GT 710 = works
  • slot #4/#6 -> quality x16->x16 riser cable -> any RX = works

Unfortunately it seems the BIOS of the Z10PA-U8 mainboard doesn't offer the option to configure the PCIe slots. I've already got the latest version.

Does anyone have any idea what I can try or can give me a (budget) mainboard suggestion that supports 4 GPUs at least at x8 Gen3?

submitted by /u/nuclearping
[link] [comments]

Discussion (0)

Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.

Sign in →

No comments yet. Sign in and be the first to say something.

More from r/LocalLLaMA