Apple: Our production version won’t make mistakes like this. It’s secret and magic.

Also Apple: We will have furtive human reviewers in the loop, so you’ll never know how often we screw up and our people have to check out your photos. That’s how “magic” works in the real world. https://t.co/x1TVU9QKr9

— Zak Doffman (@UKZak) August 18, 2021

Apple's hash function has been found. The thing to look for isn't collisions, though those are useful, it's if you can use it to search for say "all pictures of a gun" or all pictures with this text in it. Instead of just a specific picture. https://t.co/zplFBGYJkk

— Ian Miers (@secparam) August 18, 2021

Hash collisions with a tech company reporting you to law enforcement for some of the most heinous of crimes at stake. What could go wrong? https://t.co/8i68SsSVIW

— Richard Hoeg (@HoegLaw) August 18, 2021

If things like this can be built, so can the exact opposite: Fake CSAM designed to, if it is in the training set, cause a chosen non-CSAM document to flag. Which means plausible deniability for state actors while enabling arbitrary flagging. https://t.co/RZ4zpLkTQG

— Kim Reece (@feonixrift) August 18, 2021

It's been thing after thing since Apple has announced thishttps://t.co/P3wYh3t8a6

— Jason Koebler (@jason_koebler) August 18, 2021

OK. This is quite technical. But TL;DR - the neural hash system #Apple uses for their CSAM detection has been confronted with its first possible collision by some good hackers. This dog might be marked by the system as suspicious. Ouch. Issue 1 at https://t.co/VerUBTOFCU pic.twitter.com/MogRcrl3Y7

— ? Jan Wildeboer (@jwildeboer) August 18, 2021

New: researchers are abuzz with claims that NeuralHash, tech underpinning Apple's CSAM, may mistake two clearly different photos as being the same. But Apple says this is a generic version of the tech, not final, the overall system is built to expect this https://t.co/I2wsvu5uIs

— Joseph Cox (@josephfcox) August 18, 2021

Well that didn't take long.

“Can you verify that these two images collide?“

“Yes! I can confirm that both images generate the exact same [NeuralHash] hashes on my iPhone. And they are identical to what you generated here.”

“59a34eabe31910abfb06f308”https://t.co/cXS4wvBMG9 pic.twitter.com/FFIhNuGIBo

— Kenn White (@kennwhite) August 18, 2021

Surprise: Someone already built a first working collision/preimage-attack for Apple's NeuralHash model for CSAM detection.https://t.co/7e8gnqPVFp pic.twitter.com/PnfRnHbtBD

— stacksmashing (@ghidraninja) August 18, 2021

Apple is going to claim that they didn’t expect the hash function to remain secret, but this is nonsense. If they wanted this code public, they would have released its design as part of the documentation they published last week.

— Matthew Green (@matthew_d_green) August 18, 2021

It's almost as if the "screeching voices of the minority" saw this coming.https://t.co/5uXikGRdJ2

— Zack Whittaker (@zackwhittaker) August 18, 2021

Instead of just respecting privacy now Apple is going to need a build a team of people to sit around all day investigating CSAM hash collisions https://t.co/3QNzvFUbs2

— Cyber (@r0wdy_) August 18, 2021

So Apple's CSAM detection thing just turned into a weapons system for discrediting just about anyone. Outch. https://t.co/nP2s3vXFVs

— Friedrich Lindenberg (@pudo) August 18, 2021

tl;dr The hashing model Apple uses to scan iPhones for CSAM has reportedly been found by security researchers. Further, a method was found to create hash collisions between images. In theory, this could allow the creation of unrelated images that are detected as CSAM. Not good. https://t.co/pmUKxki68X

— Charles Perry (@DazeEnd) August 18, 2021

And guess what has now been proven--
"Apple’s child-abuse scanner has serious flaw, researchers say"
"Researchers have produced a collision in Apple’s NeuralHash system"https://t.co/Lv35BygN76 pic.twitter.com/tzjuMt3xvJ

— Chris Vickery (@VickerySec) August 18, 2021

Independent security researchers appear to have generated images that could fool Apple's controversial new CSAM detection/surveillance tech into flagging false matches https://t.co/4NodfR6OUZ

— Freedom of the Press (@FreedomofPress) August 18, 2021

researchers say they extracted the NeuralHash engine at the heart of Apple's CSAM-scanning system and used it to generate a collision — two images that produce the same hash, potentially a serious flaw in the system https://t.co/nXA8ns4Uyg

— James Vincent (@jjvincent) August 18, 2021

What this means is, according to Apple's "never wrong" photo detection algorithm, these two pictures are identical and there is no difference whatsoever in their appearance.
Apple wants prosecutions based on the software that thinks these two pictures are identical: https://t.co/uaUOEIzTb6 pic.twitter.com/pomWkjun4E

— Chris Vickery (@VickerySec) August 18, 2021

Stop using your iPhone for pictures.

Unless you want a real person to see all of your private pictures. https://t.co/61cRLdYXd5

— MeanHash ₿ ✪ (@MeanHash) August 18, 2021

Apple told Motherboard that the version of NeuralHash that users on GitHub tested is a generic version, and not the one final version that will be used for iCloud Photos CSAM detection. https://t.co/E5PkLQnRfW

— Motherboard (@motherboard) August 18, 2021

"the only thing you can do [...] is annoy Apple's response team with some garbage images until they implement a filter to eliminate those garbage false positives in their analysis pipeline"https://t.co/I2wsvu5uIs pic.twitter.com/VgaXhKnK7k

— Joseph Cox (@josephfcox) August 18, 2021

The next question is whether the latter image could be applied as a perturbation to an existing image? Or to answer the question of whether NerualHash collisions could be created to make an arbitrary benign image appear to be a known bad image. https://t.co/PwxgidIy6R

— Dino A. Dai Zovi (@dinodaizovi) August 18, 2021

This could allow an attacker to generate false matches to the CSAM database, triggering apparent matches from innocent-looking photos. The PSI protocol doesn't protect against this.

(n.b. I haven't checked this collision or the claim it matches NeuralHash.) @bipr @rossjanderson https://t.co/a7xBl0qtQy

— Vanessa Teague (@VTeagueAus) August 18, 2021

"There's no way we're going to ever have hash collisions" https://t.co/3VstwNY2ds

— Eric Richards (@EricRichards22) August 18, 2021

Apple said security researchers could verify its claims about the child safety feature. So they did. These two images generate the same hash.https://t.co/lO8BEbWr89

— Runa Sandvik (@runasand) August 18, 2021

Apple responds to reports of trivial NeuralHash collisions. TL;DR: A second, blackbox server-side image scanning algo (possibly PhotoDNA) would also have to be triggered as well before flagging for human review.https://t.co/kWuYDOLYJT

— Kenn White (@kennwhite) August 18, 2021

someone found Apple's Neurohash CSAM hash system already embedded in iOS 14.3 and later, and managed to export the MobileNetV3 model and rebuild it in Python

how many months until there's a GAN that creates innocuous images that're detected as CSAM?https://t.co/gkzjm4pmCX

— site specific carnivorous occurrence (@atomicthumbs) August 18, 2021

Researchers claim they were able to trick the system into saying two completely different images were the same. Apple says it's supposed to work that way. https://t.co/0yT6NMN3mg

— VICE (@VICE) August 18, 2021

Reports raise concerns about flaws in iPhone child abuse scanning tech https://t.co/pNX7CJuwg1

— iMore (@iMore) August 18, 2021