[nmglug] Debian, how to send large zip file 95GB?

[Anthony J. Bentley]

Hi,

a writes:
> Send 95GB zip file, would email attachment work?

Any common mail provider will block such a message, if you could even
send it at all.

The best way, if possible, would be to transfer the file via physical
USB or hard disk, perhaps by mail. As the saying goes, you can't
beat the throughput of sneakernet, but man, that latency...

If that's not possible, I would host a HTTPS or SFTP server, giving a
link to the recipient via phone call or email. I'd also split the file
into sub-1GB chunks (with split: https://man.openbsd.org/split.1) and
provide checksums (with cksum: https://man.openbsd.org/cksum.1), because
odds are a 95 GB file will have short reads or other transient network
issues, and you don't want to download the whole thing again. The
file splitting procedure would be something like:

$ split -b 500m bigfile.zip splitfile.zip.
$ cksum -a sha256 splitfile.zip.* > checksums.txt

Copy those to a web server, and the receive procedure would be something
like:

$ wget https://example.com/{checksums.txt,splitfile.zip.{a..z}{a..z}}
$ cksum -C checksums.txt && cat splitfile.zip.* > bigfile.zip

If all that seems like too much trouble, or the recipient won't
have access to shell tools like cksum and cat, a third-party service
like Dropbox or Mega would be the way, I suppose. But I bet you'd
still have trouble getting a complete 95 GB file over a single
connection without corruption or a short read.

-- 
Anthony J. Bentley