Sharing your awesome private linux iso collection with your friends
tehdartherer
OG
So, imagine you have this private large super secret linux iso collection you call your own and not even that, you actually know your friends have similar super awesome collections of even more isos. And all of these collections are growing by themselves.
I'm looking for suggestions to share these collections between your friends privately, i.e. make a centralized shared archive.
The obvious way would be to setup a big storage server, put wireguard and some file server on it and call it a day. Maybe also include super fancy iso streaming.
But maybe your friends are living scattered across Europe and whatever choice of provider you make, some of your friends will always end up with bad peering, waiting for their newest classic isos for way too long.
Another possibility would be a shared cloud storage account, but if you have more than 3 friends accessing it from all over Europe the account will probably get suspended soon.
Alternative could be a business account with multiple users, but that can get expensive and you are not running a company with your friends.
Most of these are classic isos and won't be touched much. So there is not much traffic, just a regular inflow and in case something gets downloaded it has to be fast.
What would be your solution to this, without spending tons of money?
Maybe this could be useful for ppl with similar demands.
Comments
How many different distros are you talking about? Like more than 10TB? More than 100TB? Or what? Would it be enough to have (say) a primary in Germany and a mirror in Romania? If a user has crappy peering to those places, maybe there could be an impromptu CDN made of a few bottom-end VPS's? I understand the need for secrecy since the Debian users wouldn't want to be caught out by the Fedora users or vice versa, but you could always encrypt the distros for particular recipients if you're leaving them in public places.
Can't stop laughing, sorry.
Why would you download isos from peering network than official mirrors.
The only problem are the dead distros. But still I don't think it would amount to too much of space.
The Linux isos don't even take that much space if you think about it. Maybe Google drive is the best option. They have mirrors everywhere. Doesn't even cost that much if you go with Gsuite.
For netcup offers visit: Netcup Codes Root server and VPS vouchers || Deutsche Version - Netcup Gutschein
Wouldn't it be possible to set up some entry servers with good peering to your friends? And those entry servers acces the data somewhere else (eg. where it's stored right now)... Normally servers have better peering than home connections.
Though, this would have some drawbacks afaik.. At first you might need tons of bandwidth (which can be expensive).
Secondly, if mounted (with eg NFS, SMB or any ohter), the data per se might be somewhat scattered. Perhaps something like glusterfs (or even NFS, dunno) can bypass this by creating a big storage pool.
The more I think of it, it really sounds like a self built cdn
What are you talking about i have people hoarding every version of ubuntu debian whatever released every season in uncompressed form you really cannot trust google they might inject spyware on your isos
Want free vps ? https://microlxc.net
First of all thanks for your comments. I'm enjoying to read them
While I'm not particularly concerned about encryption - there will be a strong anti-Fedora policy enforced - it could be a viable option for cloud storage. I mean, who knows, maybe someday will Google scan all of our isos and decides to ban all Debian related archives? That would be a pity.
But then the average friend of mine will be displeased having to download and decrypt it himself, without having the possibility to just mount and grep through the contents of a particular iso on-the-fly.
This is what @willie calls a sort of impromptu CDN and doesn't seem to be a bad idea. It would certainly work if ppl are grouped at specific places, so that one has to look up only a few instances with good peering to the given locations. But if that's not the case, one has to come up with multiple instances for every user having bad peering to the central one. At the same time one has to make sure that the peering between the central server and these "CDN" nodes is and stays good too.
Having that said, this can also get expensive if you want to rely on a good network for all of them.
Would the ISOs need to be converted on the fly while downloading them? If not, a bunch of encrypted of drives (that rhyme with cream), all mounted on one decent VPS or machine that works adequately for the majority of users will suffice. For the users who don't get adequate service, they can get a cheap lifetime VPN subscription, which should solve most of the issues.
Yeah right. Google will break the law and add spyware to files of paid customers.
I am not talking about free 15 GB space here. I am taking about the GSuite that is trusted by a million companies to store their confidential documents.
I am not defending Google here but atleast their is credibility there.
Self managed servers are not risk free. That's what I am saying here.
For netcup offers visit: Netcup Codes Root server and VPS vouchers || Deutsche Version - Netcup Gutschein
How is VPN going to solve most problems? From experience, I can tell using VPN with any home broadband network only adds to the overhead. For example, if I am getting slow connection from Japan to Germany, even if I use a Japanese VPN to connect, will add my Japanese home connection overhead to the vpn which still needs to connect from Japan to Germany. In most of the cases It will actually create a bigger overhead on the network. I am sorry, but correct me if I am wrong.
From what I can tell, using a good cdn will solve the issues not a VPN.
For netcup offers visit: Netcup Codes Root server and VPS vouchers || Deutsche Version - Netcup Gutschein
Datacenter to datacenter connections tend to be better than residential to datacenter, and geographically closer connections tend to be better. So, the easiest way to get a better experience is to connect to a good, nearby PoP, so that last mile connectivity is less of an issue, and then use the good DC to DC connection.
That is purely subjective and totally depends on the routing. I don't think it will solve the issues for most users. Maybe some but not most.
But still, I think using a good CDN will solve issues for all users. And they won't have to rely on VPNs. The overhead is still considerable.
For netcup offers visit: Netcup Codes Root server and VPS vouchers || Deutsche Version - Netcup Gutschein
You’re thinking the wrong way.
Let’s say my ISP has a bad connection to a hosting provider in Germany, possibly due to bad routing.
But I have a VPN that has a great connection to both my isp and the hosting provider. Then my connection to that provider will improve greatly even though the server is still as far away.
Won’t solve connections to freaking Japan. But OP was talking about Europe. I wouldn’t recommend someone a CDN for sharing their illegal media collection with friends. Might as well subscribe to all streaming services.
It's not really illegal right. It's open source Linux distros.
For netcup offers visit: Netcup Codes Root server and VPS vouchers || Deutsche Version - Netcup Gutschein
@FridayNightGamer some people define it different https://www.urbandictionary.com/define.php?term=Linux ISO
https://webhorizon.net
From .img to .zip to .iso... Depending on what needed.. If the connection is rather slow then one might only get a minimal version, and not the full blown desktop type.
Question is, will you get i386 support as well then?
Gave me great laughs, thanks!
it's super secret linux iso collection.
exactly, only for private use.
I think this actually a good idea. Will play around with this a bit.
@willie I'm talking about ~> 10 TB and growing steadily, but slowly.
Yes, on-the-fly conversion would be optimal, sure. Having the full flexibility of any compression formats for our precious boot media.
Wait, you are actually talking about linux ISOs? I thought you were talking about "linux ISOs".
I hear you say: "Whats the difference?"
You use linux ISOs to install linux OS on your computer. You normally download it over http/ftp/https from the official distro websites.
You download "linux ISOs" via torrent from shady websites using private torrent trackers. You use VPN to hide what you are downloading and later enjoy the blueray quality "linux ISOs" on your existing OS by using your favorite media player.
Somik.org - Server admins cheat codes
What's the state of fuse overlay filesystems that do transparent file crypt/transform?
Anything like that floating around the data hoarder scene?
There's always ' rclone crypt' IIRC, though I've never used rclone personally. Sftp is fine if your ISP latency doesn't really let you pull more than 17mbit from Europe (per thread)
Ohh. But still. Most people here are giving recommendations on actual Linux isos. I would advise not to create storage of copyrighted content for public sharing. Closed access will solve everything
For netcup offers visit: Netcup Codes Root server and VPS vouchers || Deutsche Version - Netcup Gutschein
Yeah I was going to say, you can fuse-mount a remote fs with one file that's an encrypted iso, then have local dmcrypt to a local mount point that lets you access individual files in the iso without having to download the whole thing.
Also, even if the international peering is terrible, individual isos are rarely more than a few GB. You might have hundreds of them on a cloud server (like all the Debian point releases all the way back to version 1, which was on floppies) but if someone just wants a specific one, they could start a transfer from the remote server to a VPS in their geo region. That could take a while because of the slow international pipe, but it would be server to server so they wouldn't have to pay attention to it til it was finished. Then they'd use their fast regional connection to retrieve the iso from the vps. The vps would only need enough file space for the specific iso rather than the whole collection.
As for Japan, Vultr and Linode both have Tokyo locations. The amount of traffic involved shouldn't cost much.
Have you considered high definition Linux iso dead drops?
The only ones that should know your Linux ISO preference are your partner, Google and Pornhub. Anyone else is just creepy AF.
hi can I be your friend
I bench YABS 24/7/365 unless it's a leap year.