Introduction

Automated image moderation systems are commonly used as a first-layer filter before human review. Services such as Google Cloud Vision (GCV) Safe Search aim to detect potentially sensitive content like adult imagery, violence, or suggestive material.

In February 2025, we evaluated Safe Search using a dataset of 552 images. One year later, we repeated the same experiment under identical conditions to compare the results and understand how moderation signals behave over time.

‍

Experiment parameters

• 552 images processed
• 3 sensitive test images intentionally introduced:
  • Violent human interaction
  • Provocative adult image
  • Firearm discharge image
• An image is considered potentially abusive if any of the following Safe Search categories is marked as POSSIBLE, LIKELY, or VERY LIKELY:
  
  • Adult
  • Violence
  • Racy
  • Medical
  • Spoof
• Safe Search was used to flag content for checking, not to automatically reject it
• The dataset, thresholds, and moderation pipeline remained unchanged, allowing a direct comparison between both runs

‍

Results

February 2025

• 17 images flagged
• 15 false positives
• 2 of the 3 test images were detected

February 2026

• 436 images flagged
• 435 false positives
• 1 of the 3 images detected

‍

Results analysis

False positives

The flag rate increased from ~3% to ~79%, but the largest change between the two runs was not only the total number of false positives, but also their distribution by category.

In February 2025, false positives were limited and were mainly associated with violence:

• Violence: 9
• Racy: 5
• Adult: 1

In February 2026, the false positive profile changed completely:

• Racy: 366
• Medical: 43
• Violence: 18
• Spoof: 8

This shows that the 2026 replication differed not only in volume, but also in how content was classified. It also behaved differently in terms of category assignment, with benign mouse images now being overwhelmingly classified as racy.

An example of a benign mouse image flagged as sensitive content.

Safe Search response:

{
 "adult": "UNLIKELY",
 "spoof": "VERY_UNLIKELY",
 "medical": "UNLIKELY",
 "violence": "POSSIBLE",
 "racy": "UNLIKELY"
}

Despite containing no harmful content, the image triggered a moderation signal and required manual review.

Behavior on Sensitive Test Images

• ‍Provocative adult image → flagged in both runs‍
• Violent human interaction → flagged in 2025 but not in 2026‍
• Firearm discharge → not classified as violence in either run

This suggests Safe Search prioritizes explicit physical harm rather than weapon presence alone.

‍

Final conclusion

In 2025, Google Cloud Vision Safe Search appeared to be a reasonable first-layer moderation signal when used conservatively to trigger manual review.

However, the 2026 replication showed a very different behavior under the exact same conditions. The number of false positives increased from 15 to 435, with most of the new errors concentrated in the racy category.

Based on 2026 results, Safe Search is no longer operationally useful as a first filter for this dataset, since the volume of false positives generates too much unnecessary review workload. In practical terms, the system went from requiring manual review of approximately 3% of the dataset to nearly 80%, effectively removing the efficiency benefits of automated moderation.

‍