HDDs or SSDs locally?
Local server has SSDs with zfs and well enough memory for arc.
Healthy storage pool? Any disk errors?
No errors and I have separate local backup server with spinning drives too which works just fine. Local backup server has ~5,7TB of data and verifications complete in about 5,5 hours. Actual (incremental) backups to S3 take about two hours with that ~2TB on VMs which seems decent enough.
Have a look at your AWS billing console, since data egress is charged and downloading to verify is considered egress.
AWS S3 supports data checksums where a checksum is calculated at AWS, which you can compare against a checksum that you calculate locally.
This is an article that goes into how it works, but I've not (yet) tested it, but I'll be following in your footsteps pretty soon.
As an aside, make sure that versioning is OFF on your backup bucket unless you specifically require and understand it, because even when you delete objects, they persist as a previous, all but invisible, and charged(!), version.
My former backup software "helpfully" enabled versioning and I was left with a $600 monthly bill for six months while there was no actual backup being done due to a local hardware failure, until I figured out what was happening. I used that software for years and shudder to think just how much extra it actually cost.
I will note that while I had a catastrophic hardware failure, I didn't lose any data.
Finally, if you're storing data in Glacier, retrieval is charged at different rates, depending on timelines of access, so it might be that your backup software is using the slow tier to "save" you money.
Edit: OP advises that they're not using AWS, instead they're using OVH. The object storage solutions appear to be mostly compatible, but I was unable to discover if the OVH implementation supports checksums.
I'm using OVH, not AWS. Their console gives estimation of ~20€/month for the ~2TB I have stored. Versioning is disabled and i'm currently runnign on their signup offer of 200€ credit, so I'm good to go for few weeks more. The storage I'm using includes the traffic, it's just practically unusable due to verification speeds.
I apologise, I saw S3, never even noticed the "OVH", nor had I ever heard of it.
I'll leave my original reply as is with an added disclaimer for anyone who follows down the same path.
Selfhosted
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
-
Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
-
No spam posting.
-
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
-
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
-
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
-
No trolling.
-
No low-effort posts. This is subjective and will largely be determined by the community member reports.
Resources:
- selfh.st Newsletter and index of selfhosted software and apps
- awesome-selfhosted software
- awesome-sysadmin resources
- Self-Hosted Podcast from Jupiter Broadcasting
Any issues on the community? Report it using the report flag.
Questions? DM the mods!