22
Git scraping: track changes over time by scraping to a Git repository (simonwillison.net)
submitted 3 years ago by [M] to c/git@programming.dev
5 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 3 years ago* (1 child)

And that's just a single repository. Although for a single JSON scrape it shouldn't take a full minute

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 3 years ago

    According to the Actions log for that repository it generally takes about 11-16 seconds per scrape, nowhere near a full minute. Worst case, if it's 16 seconds every time, for a 31-day month that's 31 days * 72 actions/day * 16 sec/action = 595.2 minutes, well within the free tier limit.

    And if it does become a problem it can easily be scaled down to once per hour or something with a single git commit.

  • permalink
  • fedilink
  • source
  • parent