AI detector verdicts need careful reading

TLDR

• A reporter's election-night photo drew AI-detector flags; she said she used ChatGPT to "brighten and sharpen" it.¹²
• Indicator says SynthID's "made or edited with OpenAI" verdict does not necessarily mean the image was fully AI-generated.¹
• Pangram's new image detector claims 99.96% accuracy on its own test set; Indicator says aggressive cropping still fools it.¹³

Indicator reported on 6 November that detector results on an election-night photo shared by a Free Press reporter were widely read as proof of an AI-generated image, when the reporter says she used ChatGPT to edit a real photo.¹ The photo was posted on X on 4 November and showed designer items at a Democratic gathering in Michigan.² Observers noticed a glossy sheen, and a Vanity Fair writer pointed to details of a handbag and shoes that looked inconsistent with the brands' real products.¹²

Why it matters: both verdicts quoted below allow for editing as well as generation, so the detector output alone did not settle which of the two happened. The original file and the poster's account carried that distinction.¹

Driving the news:

• Detector results: per Indicator, the SynthID detector said "This media was made or edited with OpenAI." Pangram flagged the image as "AI-generated or edited with AI tools".¹
• The poster's account: she said she "used ChatGPT to brighten and sharpen" the image and posted what she said was the unaltered original.¹²
• Comparison: Indicator says ChatGPT "did more than just 'brighten and sharpen' the image" and "distorted it". Futurism says the bag texture differs between the supposed original and the edited version, and that the lighting changed so the subject looks brighter.¹²
• Reaction: Pangram's chief executive wrote that "there is no such thing as a faithful AI upscale", according to Futurism.²

The big picture: Indicator's point is that many people assume such verdicts mean an image was created entirely by AI, which "is not necessarily the case". It asks whether labelling the photo "AI-generated" is accurate given it was not a camera original, whether "edited with AI" undersells how much it changed, and notes that the latest phone cameras also use AI.¹ Indicator also quotes a New York Times journalist saying that even apparently harmless AI editing gives an image "an unmistakable AI sheen to skin and the background" and changes the actual objects in it.¹

Detector claims: Pangram announced Image 1.0 on 4 November. It says the model scored 99.96% on 1,200 synthetic images, made as mirrors of 100 real photos using 12 image generators. It also says the model outputs a heatmap of where AI appears in an image, accepts images down to 256 by 256 pixels with a reduced-confidence disclaimer below 512, and has a separate video model.³ Indicator adds that aggressive cropping still fools the detector.¹

Single-source note: the account of the detector results rests on Indicator, which showed them as screenshots that were not reviewed here, and the edit comparison rests on Indicator and Futurism. Pangram's accuracy figures are the vendor's own claims and were not tested independently.