I run some services on machine A. I want to backup the data to machine B. It's around 40 gigabytes in thousands of small files. I would prefer to compress the data, but there's not enough space on machine A to keep the data twice (original + tar archive). I would like to avoid copying that many files via scp/rsync since it's taking forever and a bit fragile.

Any solutions on how to solve this? Is it possible to compress live on machine A and stream into the archive on machine B without the need to keep the big archive file on A?

Eventually, it's supposed to be automated and B has like the last 3 dailys, 1 weekly, 1 montly.

B can ssh into A.

you are viewing a single comment's thread
view the rest of the comments
[–] [S] 1 point 3 months ago (4 children)
  • [–] 2 points 3 months ago (3 children)

    Certainly it is a good way to go, but in my experience try the usual work flow, it is way more stable and the cpu burden is share between both nodes -> faster backups

  • source
  • parent
  • hideshow 3 child comments