▲ 82 ▼ Internet Archive | Software Engineer, Archiving & Data Services (Remote) (app.trinethire.com) submitted 2 years ago by SmartmanApps@programming.dev to c/programming@programming.dev 7 comments fedilink hide all child comments
[–] ICastFist@programming.dev 2 points 2 years ago they should advocate for a more robust robots.txt on their instances server. Only a couple of bots respect that file anyway permalink fedilink source parent