Have you ever tried writing a scrapper? I have for offline reference material. You'll make a mistake like that a few times and know but there are sure to be other times you don't notice. I usually only want a relatively small site (say a Khan Academy lesson which doesn't save text offline, just videos) and put in a large delay between requests but I'll still come back after thinking I have it down and it's thrashed something
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments