Curious: just how fast can you get? Nvme Raid 0?

[JD]

it seemed to rapidly accelerate their failure rates and killed the drives.

I think RAID also removed TRIM, at least back then. Not sure if the same issue exists today or not.

For those curious, the drives I'm considering (because theyew so cheap) are these: 10x Timetec drives for $369. So, 3x to use in my 3x new thin clints, and 7x to play with? Sadly they're only rated for 2GB/s though, which is supposedly half of 1GB/s/lane of Pcie 3.0. However... I can't seem to find top-end pcie3x4 nvme drives anymore?

I would think a PCIe 4.0 drive would still work, just run slower? Also 512GB SSDs generally weren't the fastest either, I think it's usually the 1-2TB range where performance peaks.

You'd probably be better off looking at OEM type drives on eBay though. Lots of sellers with large lots. You can get 256GB ones for under $20 which I'd think would be fine for your HTPCs.

And if you do want to pursue the RAID idea, perhaps look at the more Enterprise level drives with PLP (power loss protection)? I think you can get say the Samsung 983DCT or Micron 7300 Pro for somewhere around $100/each. Might give you better stability.

 

[FreeKnight]

I think RAID also removed TRIM, at least back then. Not sure if the same issue exists today or not.

*SNIP*

Yep. IIRC it wasn't until like the 2nd gen of SSDs with sandforce controllers were out that the RAID controllers on the Mobos would still manage TRIM (or something like that, it's been ages). I think I did a RAID 0 with the intel 'Skull' SSDs (730? 520?) at one point, but after testing I didn't see an appreciable speed difference in boot or loading games, so went back to separating them just on the chance one failed after having had a couple vertex drives fail very quickly.

Obviously newer drives are very different and likely not going to suffer from the issues the first gen of mainstream consumer drives did, but I suspect there's a reason RAID 0 is basically never talked about anymore. Easier to just get a faster performing SSD.

 

[CMetaphor]

@JD the 512gb timetec ones seemed to be (according to their own spec sheet) the minimum size (aka has enough nand chips) to get full speed performance, matching or being very close to that of 1TB or larger drives. 256GB drives are definitely too slow to even fully utilize Pcie3.0, not enough chips AFAIK. And getting more expensive PCIE4.0 drives that are more expensive really sorta defeats the purpose of "bang for your buck"

TRIM? Shouldn't be an issue. We're talking software Raid 0 here (weird for me, I know) so the OS should still see all the individual drives, and therefore shouldn't hurt them or cause anything weird.

Again, I'm a frugal sorta guy (kinda Have to be, bleh), so I'm trying to get the most out of my purchases. Buying a 10-pack of 512gb nvme drives (well, a 10 pack and one spare, so 11 total IF I use them ALL) still gives me: two "cards" of 4x 512GB, running in a single pcie3.0 slot. So 2TB software Raid 0, running at a max theoretical speed of around 8GB/s, for around $220 ($160 of drives plus a $60 pcie -> 4x nvme drive thing, with a heatsink). A 2TB pcie4.0x4 nvme drive that does around 7GB/s costs around $280 or so, so I'm saving $60 bucks. If I put both "cards" into one system (again, neither B nor Odyssey have Pcie4.0, at all, sadly) and they continue to scale decently, I could approach a 4TB drive running at a theoretical 16GB/s. That's faster than most PCIe drive can manage, which at 4TB cost around 600-700 (or much more) and I'd only be spending $440.


IDK, the math sorta works out. The price is definitely low enough to give it a shot. And even if it doesn't scale well beyond 4x drives on one card, I'll have one card for B, one for Odyssey, and 3 more loose 512GB drives for the three new HTPCs (assuming I buy 11x of those timetec drives).

 

[CMetaphor]

Reconsidering the timetec drives, they're cheap sure, but at 2GB/s max speed they're barely used 2 out of the 4 pcie lanes per drive, they might even be 2x lane drives (their white papers claim they're using 4x lanes, but it doesn't really seem like it? Or maybe the nand chips are just slow, even if there's "enough " of them to get to its so-called max speeds).

Regardless, for not much more money (about 3 dollars more per drive) I can buy partiot P320 512gb Pcie3.0x4 drives. Rated for 3GB/s max read speeds (compared to the Timetec's 2GB/s). So that *could* mean that one of the pcie "cards" I'll throw together - using 4x of thse P320s - could hit a theoretical max read speed of 12GB/s? And two cards together, IF they scale well, could hit 24GB/s? Wild speeds! I do agree with the doubt that they'll scale well, especially two "cards" running together, but price per performance wise? Getting 12GB/s out of anything under $500-600 is unheard of. And I'll be paying just a bit over half that much.

Someone's gonna have to convince me this is a stupid idea realllllll soon, haha

 

[lowfat]

Do you transfer files back and forth all the time?

You could limit a WD 850X to x1 lanes and you wouldn't see a difference in real world. And I don't consider transferring large files all the time real world, as how often do people actually do this?

If you want to do this for fun, then by all means do so. But as been said by others, there are reasons why no one does this.

I've done a decent of SSD raid in my days. Hell, my first ever SSD setup was a 4 drive 33GB hardware raid setup in late 2008. I've done 2 drive NVMe raid with Samsung 950 Pros. I've always gone back to a single drive with the best small file read performance I could afford.

 

[JD]

Probably the closest comparison point is the Crucial T700 2TB that goes for about $300 and claims to do 12GB/s as well. I'm guessing your 4x 512GB + 4x NVMe card is around $250?

However without buying a Gen5 motherboard, I suppose this is your only path if you actually need to transfer large files at 12GB/s. But I'd question that without a 25G network, or at least 2 of these arrays for local transfers between them.

It would be interesting to see if a Crucial T700 offered "smoother" performance in actual usage, even capped at Gen3 speeds, compared to this RAID0 setup. But that's a costly experiment

 

[Marzipan]

twice as fast as a single...or if you have more than two striped, multiply it by adding 1 every time you put in another SSD.

theoretically, if you were to use a The Threadripper 9000 series (WRX90 platform), which offers a total of 148 PCIe lanes, with 144 usable lanes, including up to 128 PCIe 5.0 lanes, 32x PCIe x4 NVMe SSD could reach speeds of up to 448GB/s (32x 14,000MB/s).

that's crazy fast, but nothing compared to the new photonics switch Nvidia is making with 400Tb/s of bandwidth...50TB/s!!!

 

[CMetaphor]

Actually... in my PC room (when it's ready) I'll have a 10Gbe mini network for moving files between my streaming/YT PC, my File server, and Odyssey. Will be working often to re-encode captured in-game footage as well as, eventually, live footage. The biggest reason I got Odyssey's parts was because, everywhere I've read, CPU -bound encoding produces better quality files than say Nvidia Hardware-based encoding. Nvidia can be faster, in certain circumstances, but quality is what I'm after.

So yeah, technically I'll be moving many GBs around, fairly often, to get them to Odyssesy to be "processed", with footage backups going to the file server (not sure about this part yet, my FS is already ridiculously full) / or another storage server. So these nvme raid 0 cards might not be a complete waste?

IDK, I've always kinda wanted to try and big Raid 0 with nvme drives, and just never have had the opportunity. Plus, I still think they may be a slight performance advantage to reading/writing from 4x nvme drives, through 4x pcie lanes each (16x lanes total) as compared to one PCIe5.0 drive.

I'll admit I didn't know about the T700 card, but then again, I don't actually have Any pcie5.0 m.2 slots anywhere in my home. My gamer PC upgrade parts are the AM4 + 5900x, no PCIe5.0 anywhere AFAIK. And buying an $800 mobo, a $600-700 AM5 CPU, DDR5 ram... yeah, that's not in the budget for me, not for a while still. $400-500 -ish to upgrade 2 servers and 3 thin clients, all of which don't have any nvme drives at all really set aside for them? That's a bit easier to stomach... and even then I still might skip upgrading the thin clients for now... we'll see.

 

[JD]

A 10Gbps network is only going to get you (at most) 1250MB/s, realistically around 1GB/s.

I still think you'd probably want to look at a SFP28 (25G) NIC in both client + server and connect them directly with a DAC if they're near each other. That's still only going to be 3125MB/s though. I think 100G NICs are a bit cost prohibitive

 

[CMetaphor]

A 10Gbps network is only going to get you (at most) 1250MB/s, realistically around 1GB/s.

I still think you'd probably want to look at a SFP28 (25G) NIC in both client + server and connect them directly with a DAC if they're near each other. That's still only going to be 3125MB/s though. I think 100G NICs are a bit cost prohibitive

Welp, I'm still on a very tight budget for the foreseeable future, and I already have the 10gbe stuff here.

The point sorta was, if I dump all the files I need to reencode onto one of these nvme raid 0 cards, I could get the max speed out of the cpus - so these "cards" I'm thinking about might actually have a use, beyond curiosity and some fun.