339
A missing underscore sent innocent man to prison for 18 months (arstechnica.com)
submitted 2 days ago by to c/technology@lemmy.world
51 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 17 points 2 days ago (1 child)

This is a failure of the courts. But I agree that tools like chat control or using AI to find child abuse (as has been inappropriately used by Google and Meta to falsely accuse people of such crimes) should not be the norm. I blame NCMEC for that and a lot of the current situation. That organization just wants to have numbers of reports go up year after year to secure funding. They also want to get rid of all adult content. So now we end up with situations where unshared innocent family photos and legal adult content get reported. Which reporting itself is not an issue, but then what follows where innocent people are often never given their account data back is an issue. It sometimes takes multiple layers to resolve the issue, such as click workers, NCMEC, LEOs, the courts, etc. This case shows that all levels of checks can fail.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 3 points 2 days ago* (last edited 2 days ago) (1 child)

    I'm okay with using AI to find child abuse as long as we have human verification at the end. It's more of a privacy issue for me, like how google runs their detection models in private drives and then autobans, very easy to get false positives I imagine.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 8 points 2 days ago* (last edited 2 days ago)

    Same. After the NYT articles (and several others from other news agencies), I stopped using Google for storage of personal files as I no longer trust their AI. Private (technically they call them personal) files should not be analyzed, unless there is a very high level of necessity. Scanning around the clock should not be done, just like we don’t need cameras in every home because bad things happen in homes. I also get that people will say make backups and that the cloud is optional. But an email address itself can be vital and is unable to be backed up (as far as I’m aware you can only back up the emails themselves). I doubt they are doing any better than if they just reported accounts at random. I mean reporting millions of accounts a year will probably catch some bad actors. And that they lock you out of all other data and won’t give your account back even after you are proven innocent is beyond insane. I don’t necessarily even think that emails should be scanned either. I get anything that is (or is about to be) made public needs to be scrutinized. Just my two cents.

    Edit: Google claims humans are in the reviewing the accounts, but you also have to ask what conditions those people are under. Are they highly trained? Are they looking at the worst of the worst every day? Are they pressured to just follow the recommendations of the AI? Or are they just trusting the AI and rubber stamping the results? Do they just pass the buck? I don’t think the current system was ever ment to handle the volume of accusations spurred on by AI. Google and Meta are at the forefront, but it’s the NCMEC that is the one really pushing for more volume.

  • permalink
  • fedilink
  • source
  • parent