Data Hoarder

168 readers
1 users here now

We are digital librarians. Among us are represented the various reasons to keep data -- legal requirements, competitive requirements, uncertainty of permanence of cloud services, distaste for transmitting your data externally (e.g. government or corporate espionage), cultural and familial archivists, internet collapse preppers, and people who do it themselves so they're sure it's done right. Everyone has their reasons for curating the data they have decided to keep (either forever or For A Damn Long Time (tm) ). Along the way we have sought out like-minded individuals to exchange strategies, war stories, and cautionary tales of failures.

founded 10 months ago
MODERATORS
1
 
 

I'm wondering if anyone who uses Drivepool with Snapraid can help with a question I have. I've searched all over and can't find an answer.

I have a Drivepool set up with some aging drives, mainly for Emby as well as photos etc. My essential files are duplicated as well as backed up on Backblaze but the media files are just freeballing.

I would like to set up a parity drive using Snapraid for the media files since it would be a PIA to figure out whats missing and then replace it in the event of a drive failure. Snapraid would just be for these media files which are static and don't get modified.

I understand automatic rebalancing should be turned off so files do not get moved around too much for Snapraid.

My question is I also have Stablebit Scanner and one of the options is automatic evacuation if it detects a drive failure.

Am I right to assume I should disable this since it would try to remove files off the dying drive to the remaining drives as it would screw up the parity?

Typing it out I think I know the answer but just want confirmation as I've never used a RAID config before.

2
 
 

Hello! If this is the wrong place to ask this question, please let me know and kindly push me into the right direction :)

So, I am looking for a USB Flash Drive. I do not care about form factor, storage capacity or speed, since the file on there will only be a few bytes.

BUT, I need it to be reliable. I've had many USBs not working / breaking out of nowhere, without any impact from outside. They just simply stopped working.

So, what are your recommendations for a really reliable USB flash drive?

3
 
 

I am looking for sata to usb enclosures that has USAP and TRIM support. And if more features like smart reporting work well then thats a plus. I will be using it with WD Blue SSD.

So far I looked at https://www.amazon.in/ORICO-External-Enclosure-Supported-25PW1-U3/dp/B0B936X2XH. It uses JMS578 and the firmware can be flashed to support trim.

I found some other enclosures but I don't know how to find the chipset used in them.

Which are some good enclosures that work without any issues in linux/bsd ?

4
 
 

I still use google photos and I maybe intend on getting a nas in the future to deal with my 400gb and growing of personal photos and videos but I want to start doing some backups for example one to an hdd and another to a cold storage cloud thingy, the one I heard most about is backblaze but I also heard about Idrive which seems way cheaper, is that one good? Also any ideas on how to handle the camera roll backups easily? it should be something like Android -> Main PC -> Cloud/HDD or for the backblaze / idrive thing can I go Cameral Roll/ Android -> Cloud? Ty!

5
 
 

Am looking at ServerPartDeals for 18TB HDDs and its amazing but they're USA based so I am worried about international shipping complications or damages

I tried finding alternative places in Australia but all the ones i found have like max 300gb HDDs

Am I being too paranoid about international shipping?

6
 
 

Couldnt think of an eye catching title to summarize the issue but here's what's been going on cause I'm losing my mind.

- Originally was transferring a video on my external HDD (Western Digital 4TB). All of a sudden the last 2 videos I was tranferring aren't transferring. They're moving at a very slow pace. KBs per minute almost. So I was confused but thought it was the video length.

- I cancel video transfer, but notice the music off my hdd is taking forever to buffer but it loads up slowly. I try to unmount my hdd (but am told the HDD is in use and cannot be unmounted? Never seen this before) so I restart mac. I try to use disk utility and its failing cause it can't unmount. I unplug the hdd (also was using HDD with a standard usb to usb c adapter cause apple). One thing to mention the HDD when it gets unplugged makes a click sound.

- I try plugging HDD back in but now the mac won't read it. However disk utility does, it mounts and unmounts it and says repair successful, but since then the hdd won't even show up in my disk utlity or mac. However the HDD is running, the blue light is blinking and there isn't any clicking. I try changing cables nothing.

- My sister has the standard usb port so I plug the hdd into it. At first same issue, HDD isn't being read by mac but disk utility does read it and then it shows back up on my mac. So I figure best way of approach let me drag and drop the files. It takes foreverrrrrr to prepare the files (because of these weird slow speeds) and then it says its going to take multiple days (okay fine lets see) and it does copy some for a bit eventually failing after several hours to due "error -36" on the mac and then it cancels the entire copy and paste process. And sometimes the HDD randomly disconnects and reconnects. I can try to copy and paste the folders one by one but this is going to take months and if the HDD is failing, its risky to wait. On back ups, had a backup drive but that also failed.

- Any time I try to unmount on the Mac, it gives the error the HDD is in use. Always that only disk utility can sometimes unmount it or pulling out the cable. I tried to see if I can change the enclosure maybe that's the is the issue but stupid western digital apparently has the port attached to their HDDs smfh and idk if there is even an alternative.

Been searching all over for answers on this but can't really get a proper answer, hoping someone knows what's going on here?

7
 
 

Hey I've been mostly WD Rust spinner since days of IDE, yeah IDE not EIDE LOL. But prices on spinning rust don't seem to be coming down much lately and I am in need of expansion. I have seen people say I have a misguided trust in and just lottery luck of my 30 years of WD. Also ran some Maxtors back then too with success.

That being said, review sites don't seem to be what they used to be with HDDs even SSDs for that matter. Youtubers seems to be sponsored by /via free hardware. So what is the popular opinion around here about current Seagate Quality? Because have almost exclusively been WD, and the only Seagate I owned guess 20 years also died on me sometime within the few years of owning it.

Seagate Exos 7E10 ST10000NM017B 10TB 7200 RPM 256MB Cache SATA is what I am looking at.

8
 
 

I'm a data hoarder for sure. Have many SCSI Hard Drives in my closet to prove it... I sure miss the olden days when it comes to downloading (and not necessarily pirating). Anyone remember Hotline and/or Carracho? People would set up little servers and take pride in building little communities around specific content. English Dubbed Anime, Web UI resources and files, porn of course, Freeware, foriegn movies, even FONT servers were always fun... It was not just straight forward (you could even chat or email an admin and get a response fairly fast) but it was fun and you could meet like minded people.

I guess I'm just venting because my mom got sick and I haven't logged into any of my trackers in a year and now I'm back to square one with invites. Uhhhg.

9
 
 

I have an old qnap 569 pro that I am thinking about replacing with a custom-built NAS. I was wondering if there was an easy way to keep the data when moving the hard drives without having to format them at all. Of course, I would then install a new os on the custom built, but i would prefer not to have to move all my data before doing this.

10
 
 

I might have messed up... I recently bought a 4TB SSD, Samsung 870 EVO, for the purposes of moving all my data from fragmented external Hard drives to one SSD that I can have SATA'd on my desktop.

Then I realized I've always wanted a RAID setup and the peace of mind that comes from the array. The first thing that comes to mind is a NAS setup, but for me to buy that and another two disks is about $400 and I've already spent $220 on the 4TB mentioned above, which I'd probably like to keep anyway, because like yourselves, I'm a Data hoarder.

Doing some searches online, others seemed to imply a NAS and RAID are nice but my original idea of moving everything to an SSD is also sound, even without RAID array protection.

Now I'm paralyzed with indecision and really want to move all my data from the external hard drives I have to a single source, and because I manage my Family's pictures and videos, it has a lot of sensitive data I never want to lose.

Am I over thinking it, should I just move everything to the Samsung 4Tb attached to my home desktop? I also start to worry about malware risk of keeping it on my workstation, which is why I bought smaller external hard drives over the years, to begin with.

You guys are the kings here, what would the community recommend? Is buying a NAS really the next natural step? Maybe just buy another really large external hard drive which is cheaper? $100 would get me a nice 4TB one, and I could move everything to that, again it wouldn't be in a RAID though.

The cost of the NAS is what is what is making this decision hard for me, and the fact that I just got this nice Samsung SSD.

11
 
 

Hey guys. This might be a long shot, but I thought I might as well ask.

Is there any way to find a deleted YouTube channel to watch the videos? I was listening to the Stormlight Archives Graphic Audio on YouTube and found that all the videos are gone as of a few days ago. In my YT history I have the URLs for the videos I’ve already watched, but I can’t remember the name of the channel. Is there a way to recover the videos on this channel so I can continue listening? I doubt it since it’s not my channel 😂 but any help is appreciated, cheers

12
 
 

The listing here is a CMR drive for 30 dollars. I want a drive for my jellyfin media server with the only requirement being the higher sequential access speed than the bitrate of my media which is around 6390 kbps. Will this drive work?

13
 
 

EDIT: 7 Seagate Backup Hub 8TB and 1 WD Elements external drive (the target drive).

Computer is a Dell Vostro 260 with i5-2400, 8GB RAM, 2TB Patriot P210 SSD boot drive, and 8 Seagate Backup Hub 8TB external hard drives connected to two PCI-E USB 3.0 cards with 5 USB 3.0 ports each. The computer is set to Never for when the screen and computer will shut down and the power profile is Performance.

Been trying since Monday to transfer a little over 6TB of data. Before the data transfer I checked for updates and the computer downloaded all the available updates and installed some and required a restart for another. After the restart I checked for updates again and nothing showed up so I began the transfer before going to bed. I think the estimated time was 15 or 16 hours. I turned off the monitor and went to bed. The open windows were task manager, two file explorer windows showing drives :K (source drive) and drive :L (target drive), and the little file transfer window showing about 130 MB/s.

The next morning I turned on the monitor and it was just the desktop and no windows open. I opened file explorer and only about 1TB transferred over. Usually if I restart the computer with task manager still open the computer will restart and have task manager open.

I went to settings to check for updates and none were found. I paused updates for 7 days and retried the transfer again before going to work. I turned off the monitor. Came home from work and turned on the monitor and again it was just the desktop. I opened file explorer and about 3TB total transferred, including the 1TB from the original attempt.

I went to Device Manager and have three instances of "USB Mass Storage." I went to properties and then Power Management tab and unchecked, "Allow the computer to turn off device to save power" for all three. I then retried the file transfer. The next morning I turn on the monitor to just the desktop. I turned off the computer and went to work. Got home and before bed I retried the transfer. Of course I wake up and turn on the monitor to nothing but the desktop.

Any other settings I need to check and change? It's annoying not being able to transfer all that data in one go.

14
 
 

Hi all been lurking around trying to get some info on here. Figured Id try my luck and ask my Q. Ive never run an NAS , just have external hard drives (even a few zip disks) laying around for the last 2 decades or so. Im very interested in the idea of a central place to secure everything. I probably need around 15tb of storage (without redundancy). was looking at the Synology DiskStation DS923+. is the higher price of SSD worth it for NAS? What would you guys recommend for me for a diskstation and hard drives. I dont have an unlimited budget would love to stay below 1500$ if possible. Would alos love if this could double as a mediaserver. thanks in advance

15
 
 

A recent post by a person headed to Antarctica has me wondering...

What's the most remote / disconnected data hoard you're aware of? I'm picturing a cabin in the mountains with terabytes of data on a solar powered server, that kind of thing. Starlink has given us connectivity pretty much planet-wide, but there's no substitute for a massive server with your own data on a local network.

16
 
 

I'm looking to build a new server. I want to go all ssd. I don't care to mount the ssds properly. I will be installing 14 2.5 ssds. What case would have enough space to cram 14 ssds.

17
 
 

I have been using an 8 tb external for my home lab for a year and a half now and am wanting to expand. I have been thinking of getting a usb c/3.1 hard drive rack and just buying a pallet of 20 4 tb used hard drives and stress testing all of them until i get a solid 10 or so that dont fair under stress testing. Does anyone have a solution? I was thinking e waste recycling plants or amazon liquidation warehouses but I don't know where to start. Thank you.

18
 
 

https://forums.aida64.com/topic/667-share-your-sensorpanel/

AIDA64 sensor panels are not archived anywhere as far as I know and trying to find them is horrible, it's a nearly 500 page thread.

Gotta be a solution out there already right? I foumd this https://creativeandcritical.net/fups but doesn't work with Invision forums.

19
 
 

hi, I have ~1G of personal documents that include all documents that I wrote/edited since high school. Most of them are docx/pptx/txt/markdown/pdf, and mostly text with a small fraction of pictures. I wonder if there is a rock solid backup against almost all possible corrupt in my data files? There are not large files (very few photos/videos) so I do not mind using 10x storage space (with huge redundancy to protect against any corruption) to back up data. Any ideas?

20
 
 

In the past, I have used a USB to transfer a few video files from one computer to another. This time I have over 1 terabyte of movies and shows I'd like to transfer. What is the best method?

I know it will take a very long time

21
 
 

I read many posts talking about importance of having multiple copies. but the problem is, even if you have multiple copies, how do you make sure that EVERY FILE in each copy is good. For instance, imagine you want to view a photo taken a few years ago, when you checkout copy 1 of your backup, you find it already corrupted. Then you turn to copy 2/3, find this photo is good. OK you happily discard copy 1 of backup and keep 2/3. Next day you want to view another photo 2, and find that photo 2 in backup copy 2 is dead but good in copy 3, so you keep copy 3, discard copy 3. Now some day you find something is wrong in copy 3, and you no longer have any copies with everything intact.

Someone may say, when we find that some files for copy 1 are dead, we make a new copy 4 from copy 2 (or 3), but problem is, there are already dead files in this copy 2, so this new copy would not solve the issue above.

Just wonder how do you guys deal with this issue? Any idea would be appreciated.

22
 
 

Hours worth of archived television programming across multiple channels on the night before 9/11, as a documentation of the final day of a certain time and era. The original broadcasted commercials are of course included. Revisit that night on your own time and take a trip back to get sucked back into the last day of the normalcy of 2001.

Nearly 4-Hours of Primetime Television in a "channel-changing" format: https://archive.org/details/september-10-2001-primetime-television-on-a-vintage-panasonic-tv_202312

ABC's Monday Night Football: https://archive.org/details/september-10-2001-monday-night-football-on-a-vintage-panasonic-tv

This media is highly important to preserve. It represents the calm before the storm and acts as a time-capsule. Keep these saved and preserved.

23
 
 

Hey everyone,

I travel a lot and I have 2TB SSD and a 2TB HDD and most important documents backed up into cloud as a 3rd spot.

I want to unload the 2TB HDD backup with something lighter if possible and looked into MicroSD cards.

I’ve often read about MicroSD being less reliable than other storages but I did some reading and I came up with a plan and want to pass it by people who actually know their stuff for a sanity check:

I’m planning to replace 2TB HDD with 2 1 TB MicroSDs. I know it’s not cost efficient and it may not be worth it but I really want to try it unless it’s super stupid even outside of the cost factor.

Two points of concern:

I heard MicroSDs biggest weakness is the limited writes before it breaks?

I heard MicroSDs cannot be without power for a long time.

Plan:

My plan is to write the backup once (one write), and never use them as working drives but still power them up every couple months.

When backing up, I currently delete all of my HDD and just copy everything over, but I heard there are programs that detect the changes and differences and just update those, I’m hoping those will not count as full rewrite and not do a big hit on MicroSD life.

If I do it like that, would MicroSDs be near similar reliability as other storage methods?

(And also, I feel little stupid for asking, but you can encrypt MicroSD in Disk Utility in Mac just like any other drives, right?)

Thanks for the help.

24
 
 

Howdy. So I'm in the process of building a new pool for my NAS. I have eight 18tb drives that I've burnin with Spearfoot's disk-burnin-and-testing script from GitHub. Everything has been going smooth so far at 65 hours in. I do have one possible issue however, one of the drives is testing at a slower rate then the others.

90.82% done, 65:24:36 elapsed. (0/0/0 errors) 95.90% done, 65:24:51 elapsed. (0/0/0 errors) 88.11% done, 65:24:43 elapsed. (0/0/0 errors) 53.61% done, 65:24:35 elapsed. (0/0/0 errors)
88.58% done, 65:24:38 elapsed. (0/0/0 errors) 89.32% done, 65:24:49 elapsed. (0/0/0 errors) 88.89% done, 65:24:46 elapsed. (0/0/0 errors) 94.82% done, 65:24:37 elapsed. (0/0/0 errors)

So these look good so far, the majority of the drives which are chugging along within + or - 5% of one another. While that top right drive that has fallen behind and is now -40% behind its peers and it continues to slowly fall away from the pack.

So my question is does this drive have an issue where its running at a slower speed to its peers or is this nothing to worry about? Once the burnin has finished I was thinking I would speed test each drive and see if anything shows up.

Any ideas or comments?

Thanks all.

25
 
 

Since the pandemic I’ve been collecting DVDs and Blu-rays, because I started getting into filmmaking and valued the importance of physical media. One of my reasons was the horror stories I’ve read about licenses on DRM-protected purchases being revoked.

After we moved to a much smaller house, my Billy bookshelf containing around 200+ titles has been taking a huge amount of space. And the cases just sit there looking pretty. We never use the discs. There’s no Blu-ray player in our house. We all watch digital content on portable devices. I’ve filled up several hard drives with so many obscure, international films that will never get distribution here. And so, I’ve stopped buying discs. It’s also much more convenient to be able to play MKVs on every device in my house.

I was one of those people who constantly purchased discs to remux and encode them myself for use on a future server, but that’s a waste of time, energy and money as there are dozens of release groups who’ve done the work already for me.

It doesn’t make sense to keep all the clutter around. I also have 500+ DVDs in a binder with the cover art stored in folders, but it seems like a gigantic waste of money to buy a storage system for outdated standard definition media, when most studios have remastered editions readily available.

I’m thinking of selling the Blu-rays that aren’t rare to buy a cheapo Optiplex. The discs are already pretty worthless. I’m just scared that I might regret this decision.

view more: next ›