51
ZeroFS Turns S3 Buckets Into Linux Filesystems and Block Devices (linuxiac.com)
submitted 1 month ago by to c/linux@programming.dev
27 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 9 points 1 month ago* (1 child)

This sounds like a very expensive way to abstract away access to S3-type files, unless I'm missing something. Sounds pretty easy to make a very costly mistake here.

Might be sort of useful on R2 since they don't charge for ingress/egress.

  • permalink
  • fedilink
  • source
  • hideshow 2 child comments
    •
  • [–] 7 points 1 month ago (1 child)

    You can self host S3.

    https://garagehq.deuxfleurs.fr/

    I use it as the joplin (note taking app) sync target. I originally tried webdav but I couldn't get it to work behind the reverse proxy (nginx proxy manager) so I tried garage/s3 and got it working.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 4 points 1 month ago (2 children)

    I'm aware, but why would you self-host an S3-compatible storage implementation when you can host what this tool does via NFS or NBD? Makes no sense.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 10 points 1 month ago* (1 child)

    the most simple answer is that you selected s3 as your sole storage backend for whatever services/apps you have deployed. But now, you want to integrate something that does not support s3 into your stack. So you'll look at stuff like this.

    Now, why would you self host S3 in the first place:

    The big reason is security. NFS is a security nightmare that believes connections when they claim to be root. It has no encryption and basically no authentication by default [1]

    NBD is a little better, but both NBD and NFS have the problem in that services run in the kernel, exposing frightening attack surface to privileged parts of linux systems. Also, NBD is block storage and a lot less supported as a storage backend for various services.

    S3 on the other hand, is actually designed to be exposed over the internet. http based, you can use https for encryption, and http basic auth or other methods for authentication trivially. So if you ever end up running something distributed over the internet, NFS is basically not an option, and S3 is a much better solution.

    [1] Was posted to lemmy: https://programming.dev/post/34520407

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] -1 points 1 month ago (1 child)

    That's just...a bad idea then. You have every opportunity to not do that, and you're using a solution like this as a patch to solve for other issues you had. Not a good use-case.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 5 points 1 month ago (1 child)

    This is one of the most popular methods to handle apps that are using a storage backend directly over the internet for increased bandwidth, and separation of deployment and state.

    If I'm hosting something like Nextcloud at a massive scale, it simply isn't feasible to use internal networks because they don't have enough bandwidth, and overlay/vpn network solutions have too much overhead. A common solution is to just run the services and directly connect to them over the internet. So I point my 10,000 node Nextcloud instance at S3, either my own cluster or somebody else's and S3 handles encryption while remaining reasonably performant. And scalable.

    Garage uses their software internally for something similar: https://garagehq.deuxfleurs.fr/documentation/design/goals/ . On this page, they describe using it as their matrix image/file storage cache. Same thing there, they probably have a large distributed matrix cluster that needs storage that scales with it, while also being secure over different networks.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] -1 points 1 month ago (1 child)

    Okay, as I said before, you had a technical issue you couldn't fix for one specific use-case. Jumping down to a less efficient (which this is) abstract to solve for that problem isn't a good solution, ESPECIALLY if you're self-hosting as you describe.

    If I go to a store to buy hot dogs, and they're out of hot dogs, I wouldn't buy hamburgers to cut in half and try to pass it off as hot dogs just because they fit in a hot dog bun.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 1 month ago (1 child)

    It is useful because you don't need to migrate the fs if you need to expand later

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 0 points 1 month ago (3 children)

    This has zero to do with anything. Please explain why you wouldn't be able to expand storage otherwise, and how this solution helps with that. You're aware of LVM, btrfs, or ZFS?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •