▲ 57 ▼ Should we block OpenAI from scrapping the server? (platform.openai.com) submitted 3 years ago by mp3@lemmy.ca [M] to c/lemmy_ca_support@lemmy.ca 21 comments fedilink hide all child comments Right now, robots.txt on lemmy.ca is configured this way User-Agent: * Disallow: /login Disallow: /login_reset Disallow: /settings Disallow: /create_community Disallow: /create_post Disallow: /create_private_message Disallow: /inbox Disallow: /setup Disallow: /admin Disallow: /password_change Disallow: /search/ Disallow: /modlog Would it be a good idea privacy-wise to deny GPTBot from scrapping content from the server? User-agent: GPTBot Disallow: / Thanks!
[–] jman269@lemmy.world 6 points 3 years ago (3 children) Probably want == instead else we will all be forbidden permalink fedilink source parent hideshow 6 child comments replies: [–] Shadow@lemmy.ca 3 points 3 years ago* (last edited 3 years ago) I would have thought so too, but == failed the syntax check 2023/08/07 15:36:59 [emerg] 2315181#2315181: unexpected "==" in condition in /etc/nginx/sites-enabled/lemmy.ca.conf:50 You actually want ~ though because GPTBot is just in the user agent, it's not the full string. permalink fedilink source parent [–] nbailey@lemmy.ca 2 points 3 years ago Strangely, = works the same as == with nginx. It's a very strange config format... https://nginx.org/en/docs/http/ngx_http_rewrite_module.html#if permalink fedilink source parent [–] quesomodo@programming.dev 1 point 3 years ago Look at me! I'm the GPTBot now! permalink fedilink source parent
[–] Shadow@lemmy.ca 3 points 3 years ago* (last edited 3 years ago) I would have thought so too, but == failed the syntax check 2023/08/07 15:36:59 [emerg] 2315181#2315181: unexpected "==" in condition in /etc/nginx/sites-enabled/lemmy.ca.conf:50 You actually want ~ though because GPTBot is just in the user agent, it's not the full string. permalink fedilink source parent
[–] nbailey@lemmy.ca 2 points 3 years ago Strangely, = works the same as == with nginx. It's a very strange config format... https://nginx.org/en/docs/http/ngx_http_rewrite_module.html#if permalink fedilink source parent
[–] quesomodo@programming.dev 1 point 3 years ago Look at me! I'm the GPTBot now! permalink fedilink source parent