106
[Solved] Any idea why those two drives are dramatically slower than the rest? (feddit.org)
submitted 4 days ago* (last edited 4 days ago) by to c/selfhosted@lemmy.world
8 comments fedilink hide all child comments
 

I have eight identical drives connected to an LSI 9207-8i ASRock X470 Taichi Ultimate mainboard.

They're currently all running badblocks -b 8192 -c 2048 -p 0 -s -t 0 -v -w /dev/disk/by-id/ata-WDC_WUH722424ALE6L4_[XXXXXXXX] each.

While sda and sdb achieve almost 300MB/s, sdg and sdh struggle to even get to 200MB/s. That's a 50% difference!

All of my drives are recognized as SATA3, confirmed by smartctl -a /dev/sd[a-h] | grep SATA showing SATA Version is: SATA 3.5, 6.0 Gb/s (current: 6.0 Gb/s)

There's no CPU bottleneck either AFAICT:

Any idea what could be the issue of that? Is such a dramatic speed difference to be expected of Ultrastar drives?
Are these drives just bad?

Edit: Mainboard configuration:

Edit2: Okay, apparently the ASM1061 which provides the two extra SATA3 ports is connected to the CPU via a single PCIe 2.0 x1 lane, which provides only 4gbps (=500MB/s) of raw throughput.
I think I have found the culprit and the problem should disappear once I connect the drives to my LSI 9207-8i HBA card.

all 8 comments

sorted by: hot top controversial new old
[–] 63 points 4 days ago* (3 children)

Without googling the specs, are those two on a different channel? ie the chipset has 6 and then another chip handles 2 or vice versa

  • permalink
  • fedilink
  • source
  • hideshow 6 child comments
    •
  • [–] 14 points 4 days ago

    Based on OP's edit, that looks correct. Unless OP checks the topology and can make the claim that drives from the main chipset are the slow ones, secondary chipset drives will tend to perform slower, and the ASMedia ASM1061 has several reports online of poor to trash performance compared to baseline.

    Best bet if the full performance is necessary out of every drive would be finding a decent quality PCI-E SATA III riser card that doesn't use a known flawed driver chip rather use the two 'extra' ports on the motherboard.

  • permalink
  • fedilink
  • source
  • parent
    • [–] 13 points 4 days ago (1 child)

    As a dude from the peanut gallery still learning, this was an interesting read. Never thought about the importance of what channel the sata cables are plugged into.

  • permalink
  • fedilink
  • source
  • hideshow 2 child comments
    •
  • [–] 3 points 4 days ago

    It's one of the reasons why getting a SAS controller is really nice, backplanes make it even better.

    I was preforming a burn with 10 X 8TB drives on a controller that already had a couple operating ZFS pools totaling 18 X 4TB drives and a couple SSD's and there was no slow downs. A SAS3008 controller can support over 6000 MB/s and has a PCI Express X8 3.0 bus.

    I've been building my servers and network out of old enterprise gear for a while now. It uses more power but the things you can do are truly amazing. Sadly the ai boom drove prices way up compared to a couple years ago, what I bought for 900 in 2024 is running around 1500 today.

  • permalink
  • fedilink
  • source
  • parent
    • [–] 2 points 3 days ago* (last edited 3 days ago) (1 child)

    What are "d/s" and "f/s"?

    E: I rtfm and.... It's discard requests and flush requests

  • permalink
  • fedilink
  • source
  • hideshow 2 child comments
    •
  • [–] [S] 1 point 3 days ago*

    I actually needed to look that up, since I didn't know either. So, thank you, I've learned something new because of your question.

              d/s    The number (after merges) of discard requests
                 completed per second for the device.

          f/s    The number (after merges) of flush requests
                 completed per second for the device.  This counts
                 flush requests executed by disks. Flush requests are
                 not tracked for partitions.  Before being merged,
                 flush operations are counted as writes.

    https://man7.org/linux/man-pages/man1/iostat.1.html

    (Discards being TRIM operations on SSDs, I imagine.)

  • permalink
  • fedilink
  • source
  • parent