this post was submitted on 04 Jul 2025

115 points (96.0% liked)

Selfhosted

selfhosted@lemmy.world

61,168 readers

6,543 active

397 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Detailed Rules Post

Be civil.
No spam.
Posts are to be related to self-hosting.
Don't duplicate the full text of your blog or readme if you're providing a link.
Submission headline should match the article title.
No trolling.
Promotion posts require active participation, with an account that is at least 30 days old. F/LOSS without a paywall has exceptions, with requirements. See the rules link for details. Tags [CBH] or [AIP] are required, see the links in Rule 8 for details.
AI-related discussions and AI-involved promotional posts have additional requirements for tagging, as noted in Rule 7 and the AI & Promotional Post Expanded Rules post, and find example disclosures here.

Resources:

selfh.st Newsletter and index of selfhosted software and apps
awesome-selfhosted software
awesome-sysadmin resources
Self-Hosted Podcast from Jupiter Broadcasting

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 3 years ago

moderators

ruud@lemmy.world

Loki@lemmy.world

CannaVet@lemmy.world

devve@lemmy.world

ayyy@sh.itjust.works

curbstickle@anarchist.nexus

curbstickle_lw@lemmy.world

How to combat large amounts of Ai scrapers (sh.itjust.works)

submitted 1 year ago by DrunkAnRoot@sh.itjust.works to c/selfhosted@lemmy.world

54 comments fedilink hide all child comments

everytime i check nginx logs its more scrapers then i can count and i could not find any good open source solutions

you are viewing a single comment's thread
view the rest of the comments

[–] HelloRoot@lemy.lol 4 points 1 year ago (1 child)

crowdsec is arguably not completely open source, but I'm very satisfied with it.

hideshow 2 child comments

[–] lemonuri@lemmy.ml 3 points 1 year ago (1 child)

The scraper blocklist on crowdsec requires a paid subscription, though, or did you find another workaround?

parent

hideshow 2 child comments

[–] HelloRoot@lemy.lol 1 point 1 year ago

I don't remember how I set it up a long time ago. But when I look at my server logs I only see myself.

Afaik I just added the biggests lists. But I don't remember.

parent