this post was submitted on 23 Nov 2023
1 points (100.0% liked)

Data Hoarder

168 readers
1 users here now

We are digital librarians. Among us are represented the various reasons to keep data -- legal requirements, competitive requirements, uncertainty of permanence of cloud services, distaste for transmitting your data externally (e.g. government or corporate espionage), cultural and familial archivists, internet collapse preppers, and people who do it themselves so they're sure it's done right. Everyone has their reasons for curating the data they have decided to keep (either forever or For A Damn Long Time (tm) ). Along the way we have sought out like-minded individuals to exchange strategies, war stories, and cautionary tales of failures.

founded 1 year ago
MODERATORS
 

So I'll fully admit this is probably user error and not due to any bugs with the software. But I've been on a bit of a tear lately utilizing YT-DLP to download videos from YouTube. Now I'm not a stranger to this process, I'd used Youtube-DL successfully for years now. But I did recently take advantage of using it in conjunction Aria2c with the understanding that it would give me a bit more of a boost . Well...

I don't know if it's YT-DLP or Aria or my NAS or my Internet connection, but I went through to organize these downloads and found TONS and TONS of video files ending with ".part-frag###." There are thousands and thousands, and upon a cursory glance, seems to affect hundreds of video downloads. Each folder has dozens of these files and if it doesn't, it has split mp4/m4a. Rarely does a folder seem to have one clean video file like I would expect. I skimmed through and there seems to be finished files amongst the mess that seem to play fine, I can't be 100% sure though these files are as complete as they should be (surely the part-frags exist for a reason).

So firstly, what program or setting is causing this plague of fragments? Secondly, is there anything I can do to resolve theses files at this point? Yes, I can delete all the frags, but that doesn't give me any comfort in the integrity of whatever video file remains (and it won't solve the problem for future downloads). And as I archive to a txt file, I can't just re-download as YT-DLP considers them downloaded (I've tried redownloading to see it forces a re-check and cleans things up, but no dice). Is it just refusing to merge or delete for some reason?

you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 1 points 11 months ago

May be the NAS is the problem.

Aria2c will get these fragments. But at the end ffmpeg or equivalent will merge everything and delete all the temporary fragments.

Somehow it is not getting deleted

Ask in yt-dlp discussions (github).