Lemmy.World is looking for 4 new Systems operators to help with our growing community.

Volunteers will assist our existing systems team with monitoring and maintenance.

We’re ideally looking for chill folks that want to give back to their community and work on our back-end infrastructure. Must have 4+ years of professional experience working in systems administration. We are not looking for junior admins at this time. Please keep in mind that, while this is a volunteer gig, we would ask you to be able to help at least 5-10 hours a week. We also understand this is a hobby and that family and work comes first.

Applicants must be okay with providing their CV and/or LinkedIn profile AND sitting for a video interview. This is due to the sensitivity of the infrastructure you will have access to.

We are an international team that works from both North America EST time (-4) and Europe CEST (+2) so we would ask that candidates be flexible with their availability.

If you are in AEST (+10) or JST (+9) please let us know, as we are looking for at least one Sysadmin to help out during our overnight.

You may be asked to participate in an on-call pool. Please keep in mind that this is a round-robin style pool, so it's alright if you're busy as it will just move along the chain.

If you're interested and want to apply, click here.

top 50 comments

sorted by: hot top controversial new old
[–] 310 points 3 years ago (2 children)

I have the experience, but not the energy nor passion as I am almost burned out already. I hope you find some awesome people.

  • source
  • hideshow 4 child comments
  • [–] 71 points 3 years ago (3 children)

    If it was a paying gig would you consider it? 5 to 10 hours a week, let's say 10. What kind of salary would you expect?

    Just curious.

  • source
  • parent
  • hideshow 5 child comments
  • [–] 104 points 3 years ago

    I also have the desired skill set and experience far surpassing what they’re asking for but not the time or energy to do this since my work already demands 60+ hours a week and on-call from me. Yes I’m American.

    To answer your pay question; around 4-500 would be the average pay for 10 hours this position in the working world. Since the fediverse instances have next to zero reliable income (donations can’t be counted as reliable) I understand this is a difficult if not impossible bill to pay. This is why they’re asking for volunteers whose work schedule is more sane and therefore have the energy and time to commit. I wish I was available to do so, maybe if my current job search is successful at finding something more chill.

  • source
  • parent
  • [–] 43 points 3 years ago* (last edited 3 years ago) (1 child)

    Knowing this is a volunteer project, I'd never request renumeration. If I were contracting with a large company, I guess I'd charge 300-500 per day. That's just based on quotes I get on LinkedIn, as I've never worked as a contractor. Also I couldn't have it interfere with my main job, where I'm also on call, so it would be lower priority.

  • source
  • parent
  • hideshow 2 child comments
  • load more comments (1 reply)
  • [–] 230 points 3 years ago* (last edited 3 years ago) (1 child)

    In my opinion Lemmy.world should start selling a bit of merchandising (t-shirt and so on), just to add a little on the donation side.

    BTW. the donation links are in the group info.

  • source
  • hideshow 1 child comment
  • load more comments (1 reply)
    [+] 141 points 3 years ago* (last edited 1 year ago) (3 children)
  • load more comments (2 replies)
    [–] 133 points 3 years ago (3 children)

    A CTO of 5 years with many more years of experience here. I would be really glad to help, but not in scenario where I have to prepare a CV for international readers and have no pay at all as this looks to me like a job application with no job.

    Considering you are running on Digitalocean infrastructure, I am completely unsure why you would ever need Ansible and Terraform as it just adds complexity without certain benefit, especially if you mention Kubernetes which DO already provides with two clicks.

    I'd personally suggest trying out ArgoCD for declarative clusters. With this thing, I've seen 2 companies maintained by a single DevOps engineer with no problems. Huge timesaver and makes everything transparent.

    In case this process changes and becomes less corporate-y and more transparent, I'll be ready to apply. Hope you're going to find the right people! Long live Lemmy World!

  • source
  • hideshow 6 child comments
  • [–] 17 points 3 years ago

    I'm a big fan of Kubernetes, and for larger projects the flexibility and power it brings is unrivalled. But for smaller projects, assuming equal levels of competence, delivery teams using managed Kubernetes are almost universally later and have more issues than teams that use simpler solutions. Container-as-a-service solutions like GCP CloudRun or AWS FarGate help somewhat, but are not cheap for a given amount of compute time.

    Terraform (or IaC in general) absolutely has a place, because even if you use Kubernetes, most projects have more infrastructure to manage than just the cluster - at the very least, lemmy.world has a CloudFlare proxy to manage - and clicking buttons in a management portal is not a repeatable way of deploying that, or deploying the Kubernetes clusters themselves.

    Ansible also has a place, particularly if you're deploying onto bare metal. I wouldn't use it for new deployments unless I had bare metal to configure and maintain, but lemmy.world is deployed onto a bare metal server as I understand it. Plus, the most effective tooling is generally the one your team understands.

  • source
  • parent
  • [–] 14 points 3 years ago

    IaC is the right move. It transfers complexity, it necessarilt doesn't add it. It makes your deployments reproduceable and automated.

    Which is a baseline to having highly available infrastructure. Not everyone will be familiar, or have the right mindset for that sort of DevOps.

  • source
  • parent
  • [–] 88 points 3 years ago

    Senior Network engineer with lots of experience in the field (servers + network), 15+ years if you need help let me know I'm happy to lend a hand.

  • source
  • [–] 78 points 3 years ago

    I'm qualified, but 5-10 hours can mean a lot of different things.

    Are you looking just for oncall/incident response, or are there more active reliability projects that you need help on?

  • source
  • [–] 74 points 3 years ago (2 children)

    4 applicants x 5-10 hours is .5 to 1 full time employees. Very generously speaking the ask here is for 100k/yr in free labor. The stringent interview process is going to be very limiting on potential candidates.

    The experience isn't going to be a learning experience since you're looking for people that already know it all and I wouldn't even put it on a resume, it just advertises to employers you're ok being lowballed.

    Perhaps this is a necessity for an instance of this size, but to me that seems to indicate that lemmy.world has reached the upper end of reasonable scalability, which given the workings of the fediverse would be fine.

  • source
  • hideshow 4 child comments
  • [+] 26 points 3 years ago (1 child)
  • [–] 76 points 3 years ago (1 child)

    I don't think that user was personally complaining, more pointing out that LW might be facing it's first crisis of resources. Good sys-ops people don't come cheap, and bad one often can do far more harm than help.

    It's tricky trying to handle something like this when LW is foundationally not an enterprise or anything close to a business.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 64 points 3 years ago (9 children)

    Out of curiosity, will you be able to weed out bad faith volunteers? I am sure there are a variety of interests that would be more than willing to pay a junior admin to be a Lemmy Sysop and it’s not like the candidate will volunteer that information.

  • source
  • hideshow 10 child comments
  • [–] 48 points 3 years ago (1 child)

    I think they'll be fine, the form asks for a CV + video call + lemmy user name and optional github profile.

    I'd be surprised if a bad faith candidate got through that. A probation period could work here, where their access is a bit restricted at first

  • source
  • parent
  • hideshow 2 child comments
  • [–] 29 points 3 years ago

    Hum, I hate to be that guy, but the good (experienced) bad actors have multiple profiles curated exactly for this reason. There's a reason why companies require proof of identity and a background security check.

  • source
  • parent
  • load more comments (8 replies)
    [–] 57 points 3 years ago

    I threw my hat in the ring. Hope you all get some good people!

  • source
  • [–] 50 points 3 years ago (1 child)

    How about another approach?

    There is no good reason for Lemmyworld to keep on growing to an extent that this kind of overhead is necessary. The idea of Lemmy is decentralization and not creating a new reddit instance. Close your registration, limit your amount of communities and let Lemmy grow in other directions.

  • source
  • hideshow 2 child comments
  • [–] 37 points 3 years ago (1 child)

    The thing is. Not to grow. LW wants to get stable.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 16 points 3 years ago (1 child)

    Understandable, but aren't growth and instability related in this case? There are many instances with capacity that are already run by capable people. Just spread the load (ahem) across the Lemmy verse and only handle as much as you can. But maybe I'm missing a point, I just think that this would be the best for Lemmy in the long run.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 14 points 3 years ago* (1 child)

    I mean is that even up to them? People gravitate to the places with the most content, and right now that's lemmy world. I think the only way they could combat that is to make lemmy world private, but it might lead to people not using lemmy at all instead of spreading out to other instances.

    The other option I see is to make the instances more specialized and basically do away with generalist instances like lemmy world. So you have an instance focused on news with its own subcommunities, one for gaming, one for politics etc. But that could hurt usability. It's not an easy problem to solve.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 17 points 3 years ago*

    I agree with your points and like the idea of more specialized instances and also country related instances. I think it's solvable if the different admins work together.

    Lemmy.world doesn't have to go private, they could just not accept more users and communities for a while. It wouldn't change much since everyone will still be able to post and comment on Lemmy.world from all instances. New users would just have to choose a different instance that's all.

    For me that's the whole point, I don't see any benefit of a big instance, the Lemmyverse doesn't need one.

  • source
  • parent
  • [–] 49 points 3 years ago (2 children)

    Definitely misread this as Lemmy World PsyOp

  • source
  • hideshow 3 child comments
  • load more comments (1 reply)
    [–] 49 points 3 years ago (1 child)

    You may be asked to participate in an on-call pool

    Unfortunately, this is where I noped out. But I ditch most paid positions where I can't avoid standby-time.

  • source
  • hideshow 2 child comments
  • [–] 31 points 3 years ago

    Threw my hat in the ring, I'm a senior devops engineer.

    Don't have any Lemmy experience though. I have no desire to self host it, but I wouldn't mind being part of the team to maintain a large instance.

  • source
  • [–] 29 points 3 years ago

    Get rid of TLS 1.0 and 1.1 - good luck πŸ‘

  • source
  • [–] 25 points 3 years ago (1 child)

    I spent 4-5 years running a high traffic server using Linux, nginx, apache, php and whatever we did with Python, and would be glad to help. This was in 2010 though, so….

  • source
  • hideshow 2 child comments
  • [–] 13 points 3 years ago

    I know, I meet all of the Soft skills, like four of the Systems knowledge, and maybe 0.5 of the Ideal Devops skills. But I have certifications, love 90s cartoons, and hate oatmeal raisin cookies - so I’m thinking I’m perfect.

  • source
  • parent
  • [–] 17 points 3 years ago*

    Very cool! Would be nice to have folks from different timezones to help with outages that currently occur mostly when everyone in the team is sleeping. Good luck!

  • source
  • [–] 17 points 3 years ago

    Lol. I read the header as "Lemmy World PsyOp" and was like "well, that's disappointing," lol.

  • source
  • [–] 15 points 3 years ago (2 children)

    Sysop... reminds of the good ol' BBS days. What a great time that was.

  • source
  • hideshow 2 child comments
  • load more comments (2 replies)
    [–] 14 points 3 years ago (1 child)

    This is really tempting actually. Do you by chance need someone with skills in various storage technologies?

  • source
  • hideshow 2 child comments
  • load more comments
    view more: next β€Ί