Late update, but then I've been busy and Jonesing to get this beastie up and running correctly, and with 2 jobs and normal computer "fixes" for friends/family I've not had time until now.
Like I mentioned, I picked up the Natex deal for dual E5-2760s on a S2600CP2J board, with 128GB hynix ram. Originally in my den I had a main rig with a 2500k for gaming, and general use. (OC to 4.4Ghz.) A second rig with a Kodi setup and a VBox running win with my security camera BlueIris system and AI. I'm also running an UnRaid server for media-feeding the family. (22TB already. They are insatiable!)
What I ended up doing was making a plexi motherboard tray, and tapping new screw holes in my Phantex Enthoo Evolv. It's not clean inside (minor angle which I can live with and doesn't interfere with mounting) but I get to keep this case. It's quite the luxury box as far as I'm concerned.
So, installed with a Intel 480GB SSD and 1TB Black drive. A R9 290 GPU and 2x4port 3.0 usb pcie cards. (one's split to connect with the front ports)
I'm running Proxmox from a board mounted 32 GB USB (formatted as 26GB for extra fail room) with everything running virtual. I've replicated my main gaming rig with USB and GPU passthrough. Got a second VM for downloads with Sickbeard and SABnzbd, a WIN8 for those windows apps I gotta have (StudioTax among others) and a Win10 just because, and one little linux VM I call Quarantine because that's the places in the internet it travels too. Added to that I've got a few Raspberry Pi 3s running as thin clients as recommended in this thread for 3 htpcs and my wife's Win 7 setup.
All in all it's been a fantastic learning experience. I love that I can tinker and return to a Snapshot if I get too far off track, and the power of this thing is fantastic.
Still to do: Install my H240X and expand the loop to include the second cpu. I haven't done this yet as the connector needed for the reservoir is no longer available from Swiftech and my wife will skin me if I put more money into this before I sell my old gear.
Eventually I hope to set up a private VPN to remote into the system from outside, but networking is my least knowledgeable facet of computers, so I'm in no rush. I do love how easy backups are atm, I tend to fubar things up a lot!
There's still a fair bit of these chips/systems out in the wild. I can only advise to go for it if you're remotely interested. Home labs are a lot of fun! Hell, switching to the RPi3s for thin clients alone should make up for any expenses with hydro power savings. Have fun folks! :funky:
How is GPU passthrough working in Proxmox? Which GPU are you using?
Two minor updates for this build.
Ordered some LSI SAS3008 HBAs off eBay and a set of SFF-8643 to SATA cables from Newegg.
The drive cage in my P4000m is only SAS2 but I'm thinking of putting a 2.5" cage in the 5.25" drive bays for some SSDs. The 3008 HBAs will either go into this build, or bump out the SAS2008 from my Freenas build to see if it helps my drive throughput.
The stock Nidec fan had a really annoying quality to it, so I busted out some GT AP30s I've had sitting around forever. Also tried modding a Cougar fan but it's a fake pad for PWM... connected to nothing.
The problem is it sits in a fan bracket, and the GT is 25mm thick instead of the Nidec's 38mm. Maybe oing to glue some kind of shim to hold it in place. Not even sure if it's a good idea to reduce max RPM on the CPU area. We'll see.
I've hot glued fans in before, but for a server or something running all the time - I wouldn't. I would probably look at getting a 38mm fan replacement, but I used to have some in-line connectors from Noctua with integrated resistors to drop the RPM from 2200 to 1200 or 800rpm. If you have soldering skill you could always throw a resistor on the 12v lead to drop the RPM down if that's your problem.
The noise quality of the stock Nidec is a bit too "groany" compared to the GT. Even the other Nidec fan for the PCI-E area has a better quality motor noise. Wonder if I can pick up any cheap 38mm Sanyo Denki fans somewhere.
It's working well, actually. However it was temperamental to set up as I'm using an Asus R9 290 CUII. The Hawaii cards all work but have "glitches". In my case, everything works perfect EXCEPT that if I attempt to stop/reboot the VM with the passthrough, it forces a reboot of the host. Frustrating but not a deal breaker as I only really reboot when changing it's configuration, and I'm settled now. Gaming has been clean, and I don't even notice the difference vs bare metal.
USB passthrough and soundcard passthrough work without issue. It's a lot of fun and there's so many places to easily snapshot/backup that mistakes are quickly and easily rectified.
There's less info on my level of skill for Proxmox than I would like, but since it's basically Debian with go faster stripes there's usually something out there written.
Glad to see you're rig taking shape. Do you plan on buying new or getting some refurbished enterprise level SSDs for it? Just picked up one myself and from what I read in the community it's definately the way to go. Some awesome deals, I just wish more shipped to Canada. :blarg:
Hardware Canucks is focused in-depth reviews of the the latest graphics cards, CPUs, storage, smartphones, notebooks and gaming peripherals. Join us as we unbox, review and benchmark the best computer hardware on the market.