▲ 1018 ▼ Chad scraper (sh.itjust.works) submitted 2 years ago by dramaticcat@sh.itjust.works to c/lemmyshitpost@lemmy.world 93 comments fedilink hide all child comments
[–] Wakmrow@lemmy.world 2 points 2 years ago (1 child) How so? permalink fedilink source parent hideshow 2 child comments replies: [–] idiosynk@lemmy.world 10 points 2 years ago (1 child) When a customer needs a part replaced, they send in shipping data. This data has to be entered into 3-4 different web forms and an email. This allows me to automate it all from a single form that has built in error checking so human mistakes are limited. Company could probably automate this all in the backend but they won’t :shrug: permalink fedilink source parent hideshow 2 child comments replies: [–] Lemmesee@lemm.ee 4 points 2 years ago (1 child) Using Selenium for this is probably overkill. You might be better off sending direct HTTP requests with your form data. This way you don't actually have to spin up an entire browser to perform that simple operation for you. That said, if it works - it works! permalink fedilink source parent hideshow 2 child comments replies: [–] kattenluik@feddit.nl 2 points 2 years ago (1 child) I'm guessing forms like this have CSRF protection, so you'd probably have to obtain that token and hope they don't make a new one on every request. permalink fedilink source parent hideshow 2 child comments replies: [–] Lemmesee@lemm.ee 2 points 2 years ago Good point. This is also possible to overcome with one additional HTTP request and some HTML parsing. Still less overhead than running Selenium! In any event, I was replying in a general sense: Selenium is easy to understand and seems like an intuitive solution to a simple problem. In 99% of cases some additional effort will result in a more efficient solution. permalink fedilink source parent
[–] idiosynk@lemmy.world 10 points 2 years ago (1 child) When a customer needs a part replaced, they send in shipping data. This data has to be entered into 3-4 different web forms and an email. This allows me to automate it all from a single form that has built in error checking so human mistakes are limited. Company could probably automate this all in the backend but they won’t :shrug: permalink fedilink source parent hideshow 2 child comments replies: [–] Lemmesee@lemm.ee 4 points 2 years ago (1 child) Using Selenium for this is probably overkill. You might be better off sending direct HTTP requests with your form data. This way you don't actually have to spin up an entire browser to perform that simple operation for you. That said, if it works - it works! permalink fedilink source parent hideshow 2 child comments replies: [–] kattenluik@feddit.nl 2 points 2 years ago (1 child) I'm guessing forms like this have CSRF protection, so you'd probably have to obtain that token and hope they don't make a new one on every request. permalink fedilink source parent hideshow 2 child comments replies: [–] Lemmesee@lemm.ee 2 points 2 years ago Good point. This is also possible to overcome with one additional HTTP request and some HTML parsing. Still less overhead than running Selenium! In any event, I was replying in a general sense: Selenium is easy to understand and seems like an intuitive solution to a simple problem. In 99% of cases some additional effort will result in a more efficient solution. permalink fedilink source parent
[–] Lemmesee@lemm.ee 4 points 2 years ago (1 child) Using Selenium for this is probably overkill. You might be better off sending direct HTTP requests with your form data. This way you don't actually have to spin up an entire browser to perform that simple operation for you. That said, if it works - it works! permalink fedilink source parent hideshow 2 child comments replies: [–] kattenluik@feddit.nl 2 points 2 years ago (1 child) I'm guessing forms like this have CSRF protection, so you'd probably have to obtain that token and hope they don't make a new one on every request. permalink fedilink source parent hideshow 2 child comments replies: [–] Lemmesee@lemm.ee 2 points 2 years ago Good point. This is also possible to overcome with one additional HTTP request and some HTML parsing. Still less overhead than running Selenium! In any event, I was replying in a general sense: Selenium is easy to understand and seems like an intuitive solution to a simple problem. In 99% of cases some additional effort will result in a more efficient solution. permalink fedilink source parent
[–] kattenluik@feddit.nl 2 points 2 years ago (1 child) I'm guessing forms like this have CSRF protection, so you'd probably have to obtain that token and hope they don't make a new one on every request. permalink fedilink source parent hideshow 2 child comments replies: [–] Lemmesee@lemm.ee 2 points 2 years ago Good point. This is also possible to overcome with one additional HTTP request and some HTML parsing. Still less overhead than running Selenium! In any event, I was replying in a general sense: Selenium is easy to understand and seems like an intuitive solution to a simple problem. In 99% of cases some additional effort will result in a more efficient solution. permalink fedilink source parent
[–] Lemmesee@lemm.ee 2 points 2 years ago Good point. This is also possible to overcome with one additional HTTP request and some HTML parsing. Still less overhead than running Selenium! In any event, I was replying in a general sense: Selenium is easy to understand and seems like an intuitive solution to a simple problem. In 99% of cases some additional effort will result in a more efficient solution. permalink fedilink source parent
