you are viewing a single comment's thread
view the rest of the comments
[–] 115 points 1 day ago (3 children)

something very similar in mainframe land where they call files "datasets." if you call a dataset a file, get ready to get an earfull!

which is ironic considering IBM themselves often call them files and one of the most popular dataset utilities is called FileAid. but if we acknowledge that, we lose a valuable opportunity to belittle and exclude newcomers

  • source
  • hideshow 6 child comments
  • [–] 65 points 1 day ago (6 children)

    In AWS S3, they're objects, not files. But you upload files to objects. And can download an object to a file. But they're not files. Trust me bro.

  • source
  • parent
  • hideshow 12 child comments
  • [–] 6 points 1 day ago

    There is a reason for that since S3 is not block storage and doesn't have a Hierarchal filesystem. That's why when you create folder in the S3 console what you are really creating is a prefix meaning its just a placeholder to auto add the prefix string to the beginning of all objects. But all objects in the bucket are flat when it comes to the API. Plus the way the files are stored in the shards in S3. You can limit Throughput by treating prefixes like a hierarchical filesystem too much.

  • source
  • parent
  • [–] 26 points 1 day ago (2 children)

    You know, the same deal kind of applies about "losing newcomers". Because I avooded using AWS because everything seemed cryptic and weird with its terminology. So other companies got my business.

  • source
  • parent
  • hideshow 4 child comments
  • The more confusing thing I've had to explain to too many developers is that it's not a directory regardless of what it looks like on the AWS website view. It's a "prefix" meaning the entire thing, including the slashes, is just part of the filename, there is no real structure inside a bucket. So there's no such thing as relative paths to search for things. When you call the list objects function with a prefix it's the same as doing a "starts with" on a string. You can't give it a partial "path" relative to another object that appears to be in another "folder".

  • source
  • parent
  • [–] 7 points 1 day ago (1 child)

    That's mostly because it doesn't have real directories, it just simulates them with string prefixes and if you forget that on large buckets you are going to have a bad time.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 1 point 1 day ago (1 child)

    What makes that different from a "real directory"?

  • source
  • parent
  • hideshow 2 child comments
  • [–] 1 point 13 hours ago

    Imagine you have dir1/dir2 with "directories" inside, each one with millions of "files"/objects. When you try to "ls" the dir1/dir2 by querying Prefix='dir1/dir2/' and Delimiter='/' S3 will still scan all the millions of objects starting with the prefix, and then filter out the ones that have a / after the prefix.

  • source
  • parent