127
We have to solve the money problem! (blenderdumbass.org)
submitted 1 year ago by to c/fediverse@lemmy.world
174 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 1 year ago (2 children)

I don’t think the general architecture scales that well (think of all the duplicate storage ...

That's my hunch too, although haven't studied in detail - so I wonder how we can fix it ?
Is there an forum that discusses this scaling issue (in general, across fediverse) ?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 2 points 1 year ago (1 child)

    I suppose this community is as good as any. But it's difficult to talk in general about this as each fediverse app has different performance needs/characteristics, so I'm not sure if you can extrapolate anything in general. But perhaps?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 1 year ago (2 children)

    Well problem with any Lemmy community as such a forum, is that current usage (not necessarily intrinsic to the software) is so ephemeral. So it's good for discussing breaking news, but not to gradually accumulate discussion of solutions to complex problems, over years. I wish this were not the case, but doubt anybody will even notice this comment, as no longer 'hot', and folded away ... Rather, a few weeks later the same topic will be reopened under a different post, and we start over again.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 2 points 1 year ago (1 child)

    Well, that's the nature of link aggregators. Lemmy's and Reddit's style is a link aggregator, not really what you would consider an old-fashioned forums. It's a different sort of use case with different pros and cons. A con is that you don't get these super long lived threads cause they disappear in the stream of new threads. A pro is that... you don't get these super long lived threads cause they disappear in the stream of new threads. :P

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 1 year ago (1 child)

    "New comments" sort helps with that

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 1 year ago (1 child)

    Sort of, but doesn't it just sort by the latest comment? I.e. any thread would be bumped to the top by a single comment? I might be wrong. But that makes it kind of less than ideal if true.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 1 year ago (1 child)

    That's indeed how it works (and that's I was able to see those comments).

    It works for me, how would you like to have it differently? IIRC on old school forums a single message would also bump the thread

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 1 year ago (1 child)

    Yes, but that doesn't scale. If there are thousands of comments being submitted constantly, the All feed would just be a new page every time you refresh for the new comments sort. It would be chaotic.

    It should instead be based on a recent rate of comments for instance. Much like normal votes but comments instead and not based on the age of the post.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 1 year ago (1 child)

    Piefed partially solves that with multicommunities

    That way you can follow your hobbies feeds, your tech feed, your art feed separately.

    For All it would indeed be messy.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 1 year ago (1 child)

    Yea it's still only a partial solution. Even those feeds could get very active over time (we can hope 😅). The way Piefed implemented feeds is interesting but seems almost overengineered? Sharing feeds could have been done via a simple query parameter I feel like.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 1 year ago

    Feeds can be personal, so you can decide how granular to you want them. So far I'm able to follow every conversation on the different topics I like, so I would say it works quite well so far.

    They can be public too, but personally I prefer to customize my own.

  • permalink
  • fedilink
  • source
  • parent
  • [–] [S] 1 point 1 year ago (1 child)

    Storage Duplication is I think not necessarily an issue of ActivityPub, it's an issue of implementation of it. Because all posts can technically live on their respective servers. And rendered directly or almost directly. Like it can be copied over for the time it is relevant, and then discarded to be available only from the original server.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 0 points 1 year ago (1 child)

    That makes sense, to store only popular stuff, or temporarily - especially for 'heavier' images (although as we see with lemm.ee, that leads to issues when an instance dies). Yet I also wonder about the scalability of just the minimum meta-info, whose size does depend on the protocol design.
    For example with Lemmy every upvote click propagates across the network (if i understand correctly, mastodon doesn't propagate 'likes' so consistently, presumably for efficiency, but this can make it seem 'empty'). Maybe such meta-info could be batched, or gathered by a smaller set of 'node' instances, from which others pick up periodically - some tree to disperse information rather than directly each instance to each other instance ?
    As the fediverse grows, gathering past meta-info might also become a barrier to new entrant instances ?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 1 year ago (1 child)

    mastodon doesn’t propagate ‘likes’ so consistently, presumably for efficiency.

    It is not a matter of efficiency, but solely of how AP works. All it takes is someone one an server to to follow a community for that server to receive every vote/post/comment, while to get a whole conversation thread on Mastodon you'd need to be on the same server as the original poster or your server would need to have at least one person following every server involved in the conversation.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 1 year ago (1 child)

    Thanks, that makes sense if I think about it, but maybe users shouldn't have to - i.e. the Mdon part-conversation way still seems confusing to me (despite being a climate modeler and scala dev), although haven't used Mdon much since I found Lemmy. And I still feel that both ways seem intrinsically inefficient - for different reasons - if we intend to scale up the global numbers (relating OP).

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 1 year ago (1 child)
  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 1 year ago (1 child)

    I agree with most of what you say. I'm a long-time fan of calculating more complex things client side, as you can see from my climate model (currently all calcs within web browser, evolved from java applet to scalajs).
    Also, in regarding social media, keeping the data client side could make the network more resilient in autocratic countries (many), and thelp this become truly a global alternative.
    On the other hand, some 'trunk' server interactions could also doing more not less, bundling many 'activity' messages together for efficiency - especially to reduce the duplication of meta-info headers in clunky json, and work of authentification-checking (which I suppose has to happen to propagate every upvote in Lemmy?).

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 1 year ago

    bundling many ‘activity’ messages together for efficiency - especially to reduce the duplication of meta-info headers in clunky json

    Seems like an optimization that is not really needed. The data format is not really the bottleneck, there are ActivityPub relays that can send messages in bulk and ActivityPub is built on LinkedData, which means that there plenty of powerful libraries in most languages that can parse and produce JSON in a way that keeps application developers with a consistent semantics. The more people try to change the data format in the sake of "efficiency", the less portable and useful it would be.

    and work of authentification-checking (which I suppose has to happen to propagate every upvote in Lemmy?)

    Yes and no. Most of the current software do authentication by using HTTP Message Signatures, so after you fetch the actor's public key every request is authenticated by seeing an HTTP header, which makes it no different most common authentication schemes.

  • permalink
  • fedilink
  • source
  • parent