A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.
Resources:
Any issues on the community? Report it using the report flag.
Questions? DM the mods!
It's viable, but when you're buying a DAS for the drives, figure out what the USB chipset is and make sure it's not a flaky piece of crap.
Things have gotten better, but some random manufacturers are still using trash bridge chips and you'll be in for a bad time. (By which I mean your drives will vanish in the middle of a write, and corrupt themselves.)
I've had "trash controllers" in Orico units which rewite the drive details which makes them annoying to work with but I've never heard of
your drives will vanish in the middle of a write, and corrupt themselves
That sounds likes its underpowered and when the draw is up the supply can't handle it, which could happen for internal drives if your supply isn't up to powering enough drives.
Use a reputable brand like TerraMaster and you'll not have those sorts of problems.
Seconded. It's not a bad idea, but it's another point of failure for sure.
To add on - several of those USB controllers implement stuff in non-standard ways, so if your board fails you either need another one of the exact same model and firmware to read the data off those drives again. It's very likely if you just bought another DAS/USB controller that the drives/partitions would be unreadable, and you'd have to start over again.
OP, it's not a bad idea unless you don't have a backup plan. If you are planning on having no backups, then this is much more risky than just an external hard drive, this could very well lead to complete data loss. If you have a solid backup plan, then go for it! If you don't, then for the love of god you aren't ready for it.
Honestly, if you only need a few bays, just look at getting a Synology.
Always have a offside backup for any data you deeply care about it's never worth he risk.
So as a TLDR a DAS really should only be used as an offside backup plan or secondary storage then? I might really have to do a DIY NAS with a desktop PC then. Thank you for the warning!
I mean it's up to you, your decision. In my experience going the usb route though only leads to more cost later, to me it's better to just save your money and go with a solid solution in a couple of months. You don't have to go crazy. An old desktop with some extra sata ports is a fine start
I have the QNAP TL-D800S. It’s an 8 bay DAS but there is also a 4 bay variant. Works well for me. It uses SFF cables to connect to the PC and comes with the appropriate PCIe card which seems more robust to me than anything USB for this.
Yeah this option seems like a more robust option than using an USB one. Unfortunately my mini PC doesn't offer a sata connection, but thank you for the suggestion!
It would need a PCIe slot, not a SATA connection. But I assume it doesn’t have that either then.
I use the usb version of the qnap 4 bay and never had an issue. Into a nuc holding all media. Boot and docker config all on the nuc nvme.
I’ve personally gone with an N100 Mini PC running Proxmox and two of these daisy-chained (purchased on sale). https://www.amazon.com/MAIWO-Enclosure-Cooling-Storage-Expansion/dp/B0D28Q187R/
The MAIWO DAS uses garbage JMicron firmware by default, and there are significant issues with their sleep functions. Because of that, it took me forever to figure out why SnapRAID kept failing mid-sync. Fortunately, new firmware seems to have fixed their issues and they’ve been rock solid ever since. I specifically had to update the firmware for all 4 of the USB controllers on each DAS.
Direct link to firmware that worked for me. https://gbatemp.net/attachments/bin-16028_jms578_std_v00-04-01-04_self_power_odd_20190611-zip.230929/
JMS578_STD_v00.04.01.04_Self Power + ODD.bin
MD5: 7701fb7a968e3ad4ca926dd7854806ff
Firmware updater tool for Windows found here. I ran this from a Virtualbox Windows 10 VM inside my Arch install: https://gbatemp.net/attachments/jmicron-jms578-sata-crystal-enclosure-fwupdate-zip.216335/
FwUpdateTool_v1_19_16_24.exe
MD5: 735ec8d9f99c457ce793739480c55706
Mirrors for posterity:
https://files.catbox.moe/e4121s.zip
https://mega.nz/file/OJAX2KhQ#67kIDJun92nqi56mFur_9vALSi2yTJXXv7ew5pYSJVY
Blog post detailing firmware update procedure for an external drive: https://ralimtek.com/posts/2021/jms578/
Detailed post on JMS578: https://gbatemp.net/threads/how-to-update-firmware-of-jmicron-jms578-usb3-0-sata-enclosure-black-screen-lock-music-stop.569158/
Alternate FOSS software for flashing I found later, but never used. https://github.com/BertoldVdb/jms578flash
Wow thank you for these links, you're insane. And if you would start this homelab process again, then would you go with this mini PC + DAS layout again, or with desktop pc as a NAS?
I built my own and it honestly wasn't that expensive (at the time back in 2018). I just started with the basics but built it to be expandable. I used a Define 6 case which gave me room for 12 HDDs, a mobo with the highest number of SATA ports, processor, RAM, etc and then just added drives 1-2 at a time as they filled up. My only regret is that I didn't and still haven't learned Linux well enough to rely on it because it runs Windows, the PC is showing its age now, and I need to think about the replacement solution and how I'll be able to migrate 70+ TB of media and all my configurations to the new machine.
If you do run Windows, Drivepool and SnapRAID are useful for pooling everything into a single virtual disk and setting up a software RAID that will protect from drive failures without locking your data away.
You can also use snapraid (and mergefs) on Linux to do the same thing! I'm excited to recognize this because I recently turned an old PC into a crappy nas with Open Media Vault and used these two.
And what kinda mobo have you went with? A microATX perhaps?
It's a midtower case so I just went with an ATX board. I would like to figure out a compact solution for the future but it's hard to house and control a bunch of HDDs in a small footprint. I don't want to spend thousands on a NAS and I haven't found a trustworthy DAS solution that will hold all my drives.
You might play around with PCPartPicker since it allows for so many filtering options for things like SATA ports on a mobo or drive slots in a case and see what you can come up with.
If you use a reputable brand, such as TerraMaster you'll avoid all of the scare stories you hear.
Almost everything bad seems to be along the lines of
I bought "off brand" XzzYyG from Amazon/EBay/Wish and it failed
Something like this:
https://www.terra-master.com/uk/products/homesoho-das/d6-320.html
It'll be faster than your spinning rust anyway, as long as you have high speed ports on your mini PC.
Consider the machine being on 24/7 and cooling.
Furthermore, depending on the current power supply, you might need to upgrade it to keep everything running.
I use an M1 Mac Mini running Asahi Linux with a USB 3.0 4-bay enclosure. Works great so far.
I have used mini PCs as a servers for years with file serving being a major duty of them. Granted my storage needs aren't excessive, but most NUCs or Nuc-likes can hold two drives, some can have a third if you include 2.5" drives. My AsRock A300 can hold 4 drives (two of each), but its m.2 support sucks so that's not as much of a boon as it sounds. If you need significant storage, there is no replacement for something that can hold 3.5" drives though since those can now reach 20+ GB a drive.
I had a setup similar to this for a year or two that ended with an hdd destroying itself one night. Probably because of the drive and not the usb enclosure I was using. Until then it worked fine, it’s definitely a viable route.
If you can swing a desktop pc case it’ll probably end up easier and cheaper and have some headroom for upgrades, that’s the route I went down after trying an escalating series of mini-pcs and running into their limits one too many times.
I would either get a used HP Proliant microserver Gen 8 (I own one. Pretty sweet) or get a mini ITX nas case, 4 or more bays. Depending on USB doesn't seem like a good idea to me. You can always sell your mini PC. I have slowly learned to avoid the sunk cost fallacy.
Those microservers are dope!
Yes, and you might want to ask in the datahoarders community.
While I dont use a mini-pc, I have a server with 48TB in it on spinning disks, and I've built a hybrid DAS/NAS that I back up to.
I use this 4-bay DAS: https://www.amazon.com/gp/aw/d/B078YQHWYW I chose it because it supports USB 3.2 Gen 2 and I've been pretty happy with it.
It's usually plugged into my server directly, and I use ZFS to snapshot and send to it. However, I also can plug it into a Pi5 and use ZFS send over SSH to treat it like a NAS. The Pi can of course run Samba/CIFS and SSH for sshfs.
The biggest downside to this structure is probably the metadata speeds for ZFS over USB (looking up snapshot names), but you could always use a cache drive with ZFS.
I highly, highly recommend ZFS and figuring out your software requirements before picking hardware.
Happy to answer any specific questions, too.
Reviews on that page are kind of dodgy, but they are for all 3 products listed which makes it difficult to tell which review is for what.
Have you had any of the listed issues? Heat, unrecognized success, etc?
To follow up on this, I just used it in a 72F room, and the drives hit 60C. So heat is an issue, I've just never run into it before.
None.
I think the unrecognized issues are because people think it will behave like a device with a controller vs just USB pass-through. Every disk I've plugged in just shows up fine on the host. I also have only used it with linux.
As for heat, my drives go to sleep when not in use, but even for long stints of backups it wasn't an issue.
Reviews for just the 4-bay: https://www.amazon.com/product-reviews/B078YQHWYW?formatType=current_format&pageNumber=1
Thanks for the reply! I have a couple USB 3.0 2-drive docks that just sit out in the open - consolidating in to a single, enclosed unit with a fan would be nice (since mine or open, you really hear the HD spin up/click when accessing it).
What do you use to adjust your drives' spin down? hdparm? hd-idle? I have one drive that is constantly spinning/accessed so the thermal concerns with the unit do weigh on me.
Oh man it's been a while, but I think I used hdparm. They basically show up as individual USB drives and were SMART capable.
The DAS is disconnected now for a road test (this system will eventually live in a van), but once I reconnect it I could check if you would like.
Have you had data loss occurences in these bay enclosures? Some other commenters have said, that using it as a primary storage is really risky because some crappy controllers could ruin the drives's data for example.
I have not, but I also run it as a RAIDZ, so if I did have issues with one, the parity should cover it. That being said it is a backup for me.
Get a USB-C DAS (enclosure) for your disks, those use their own power supply. Since it is USB-C performance will be very good and stable and you'll be happy with it.
Get old HP thinclient T510, or Igel M340C. Got a few of those practically free online. Has Via Eden 1.2Ghz 2 core. Not powerful at all. But cold. Mine runs on hot summer days approximately between 40-50C. HP has I think 19V power source, Igel runs on 12V brick.
I used intel nuc j5005 for 4 years iirc with 2 bay hdd enclosure, Orico brand. Never had any problem that i hear about such as random diaconnecting. Granted i used it to host my media. So i dont pratically mind if the data is gone.
I used openmediavault with usb boot drive. Internal 2.5’ drive as docker appdata. All my media files is stored on the external enclosure. The only problem i noticed is the temperature but thats because of high ambient temp.
If youre planning to use ext enclosure for critical data. Might wanna think twice. But, if you have proper backup plan. Sure why not.
Did that for 3 years. Not pretty but it worked
I use a couple of mini pcs in exactly the setup you are talking about. Only downside is throughput. Anything built in is gonna be faster for read write ops, but usb3 is plenty fast for most things including media and data sync. I run Ubuntu with ZFS, and created raid arrays for data redundancy. It works really well. I virtualize using Incus with docker inside of those tiny VMs. It’s awesome.
I am currently getting a replacement for my old win10+drivepool setup using hand me down parts from my old main pc. I decided to go with UNRAID this time and other than a few headaches, its been pretty simple to get going. I am already a big fan of the container/docker/app setup. My setup is 3 12tb seagate iron wolf nas drives recertified/refurbished from ebay, the nvme ssd I used as my prior boot drive as a cache drive, and a r7 1700 and ab350m pro 4 mobo with a pny xlr8 1070gpu. It was mostly free since I was replacing it with a newer gaming build. I think I spent $280 on the hdds?
You may not be able to do RAID or other redundant/performant arrays with USB. You can definitely achieve a big JBOD array but it will be less resilient and slower than a RAID array. Enclosures often don’t cool as well so heat may degrade your disks faster as well. I did this for a while with some old disks and some $30 HDD toasters. I only put data on there I could afford to lose. I wish there was a standalone hardware RAID solution… like a NAS without the network. That would have a huge draw for hobbyists that don’t want to buy an expensive NAS. I’ve searched for this but haven’t found anything. Message me if you know of such a product! Maybe consider building your own NAS with an old PC. Way cheaper than a prebuilt and fun to build! I had an old Dell Optiplex 990 that is now a 32 TB NAS. Had to get a new case but it’s a decent backup to my Synology.
This is completely untrue.
You can get plently of performant arrays over USB. You do know how much USB 3.1 or 4 can transmit?
Enclosures often don’t cool as well so heat may degrade your disks faster as well.
DAS enclosures can do a great job of cooling by separating and not sharing the air inside a single case.
I used USB enclosures for my RAIDs for over 20 years. The turning point has been usb3 and then usb-c even better, but I found really no difference as in the bottleneck where the mechanical drives.
Moved to an all internal sata setup a few months back because I upgraded the space and moved to a desktop form factor.
Can still recommend the USB approach tough.
BUY A QUALITY EBCLOSURE.
I always used Linux software raid, but purchased a 4 slots USB raid/jbod enclosure to keep the number of used USB ports down.
I never ever had issues with the setup, but I purchased a known-brand enclosure, one with also e-SATA, which unfortunately was/is more a fad than even been really used.
There are small SATA backplanes that allow you to fit 3 HDDs into two 5.25" slots (or 4 HDDs in 3 slots). You can find used ones for cheap (mine was 30€), and with some cheap tower case you could get something NAS-like with hot-swap drive bays for way cheaper