skip to content
← back

DSO National Laboratories · Feb 2023 – Feb 2024

SINDEX

Multimodal Misinformation Verification

with Ashley Goh Rou Hui

Recognised by Singapore's Minister of Defence

input · claim + imagephoto shared with the claimevidence · retrieved livekeywordsnews search25 closest articlestop 3 articles · 2 sentences eachthree checkstruthfulnessclaim + image + evidencesatireis it exaggerated for effect?out of contextdoes the image belong here?misinformation · satire · out-of-contextverdict+ explanation citingthe evidence it foundschematic of the pipeline · not experimental data

overview

SINDEX is an end-to-end multimodal misinformation detector built for Singapore's social and linguistic context. Given a claim and the image shared with it, it retrieves fresh news evidence, decides whether the claim holds up, flags satire, checks whether the image is being used out of context, and explains its verdict in plain language.

The paper is titled "Chat, is this real? mm-DoubleConfirm!" and was written with Ashley Goh Rou Hui.

motivation

Misinformation now travels as text and images together. Photos lend a claim "truthiness", out-of-context images are highly persuasive, and satire adds irony and exaggeration on top. People are not reliably good at spotting it: in a survey the paper cites, four in five Singaporeans were confident they could identify fake news, yet about 90% misidentified at least one of five false headlines.

Existing automated fact-checkers fall short. The paper notes that none addresses Singapore-context misinformation, end-to-end systems are scarce, most only output true or false, they are trained on fixed datasets with limited topics, and evidence retrieval is not automated, which makes them impractical in the real world.

problem

Given a claim and an accompanying image, predict whether it is true or false, say whether it is satire, say whether the image is used out of context, and give an explanation a reader can follow, all grounded in up-to-date evidence.

technical approach

  1. Automated evidence retrieval

    A keyword-extraction transformer (H2-keywordextractor) pulls keywords from the claim; named-entity recognition keeps names, places and time phrases, and stop words are dropped. The keywords query NewsAPI, giving a live evidence bank over 80,000 news sources. SBERT embeds the keywords and each article, and the 25 most similar articles are re-scored by their similarity to the whole claim. From the top three, the two sentences closest to the claim form the evidence.

  2. Claim verification: CLIP classifier

    A Custom Truthfulness Classifier encodes the claim, image and text evidence with CLIP. Attention between evidence and the claim, and between evidence and the image, yields stance representations. Two fully connected layers with a ReLU and a sigmoid output give a binary verdict, trained with binary cross-entropy on the Singapore dataset.

  3. Claim verification: parallel LLM system

    A second, parallel system prompts ChatGPT-4-32k with the claim, context from Wikipedia articles on each keyword, and the same retrieved news evidence, and asks for a true or false label with reasoning.

  4. Satire detection

    A fine-tuned bert-base-uncased binary classifier (satire or not), trained on balanced satirical and non-satirical fake claims. Only BERT layers 10 and 11 are unfrozen; inputs are padded to 512 tokens, with dropout and a linear sigmoid head, Adam, binary cross-entropy and 100 epochs.

  5. Out-of-context image detection

    The pretrained COSMOS model (self-supervised out-of-context detection) is used. Its dataset-preparation code was not released, so SINDEX re-implements that stage with custom code: spaCy NER for entities and modified captions, SBERT for similarity, and detectron2 for bounding boxes, together with a reverse image search caption from Google Cloud Vision.

  6. Explanation generation

    ChatGPT-4-32k is prompted with the claim, its truthfulness and satire labels, and the retrieved evidence, and asked for a reasoning explanation that cites that evidence.

main contributions

  1. 01

    Singapore multimodal dataset

    A large original dataset of Singapore-context claim-image sets, collected, cleaned and manually annotated (117 hours of annotation), plus a Singapore-context satire dataset.

  2. 02

    Evidence retrieval and verification

    An automated framework that retrieves live news evidence and verifies claims against it, instead of relying on a fixed evidence bank.

  3. 03

    Custom satire detector

    A text classifier that separates satire from non-satirical fake claims, so exaggerated content is not mistaken for malicious misinformation.

  4. 04

    Multistage model

    A pipeline that combines a pretrained out-of-context image detector with large language models for zero-shot prediction and explanation.

experimental setup

  • Verification dataset: 295 Singapore-context claim-image sets, collected from Black Dot Research, Factually, Facebook and WhatsApp for false and misleading claims, and from The Straits Times and CNA for true examples. 80/20 train-test split.
  • Satire dataset: 184 Singapore-context satirical claims (written with ChatGPT's help) and 104 non-satirical fake claims, combined with 203 satirical and 283 non-satirical articles from an external dataset, balanced between satire and non-satire. 80/20 split.
  • Evidence retrieval benchmark: evidence collected by hand for 10 claims, compared with six module-retrieved sentences per claim (60 in total).
  • Metrics: precision, recall, accuracy and F1 for classification.

results

F1 · CLIP truthfulness classifier
95.2%
F1 · ChatGPT system, with evidence
95.4%
F1 · satire detection
93.2%
evidence sentences · matched the hand-picked ones
58 / 60

Figures are from the paper. Table 1 is reproduced from it.

  • Evidence retrieval: filtering by similarity alone left 65% of the 60 selected sentences irrelevant. Adding the second filter, relevance to the whole claim, brought it to 58 of 60 matching the hand-picked ones; the remaining differences were in phrasing rather than meaning.
  • Evidence is what makes the LLM system work. Without it the ChatGPT system reached only 62.9% F1, mostly guessing from how credible the claimed source sounded; with retrieved evidence it reached 95.4% F1 and made two errors, both with correct explanations but mislabelled.
  • Explanations were evaluated against hand-written gold explanations. In the paper's example, the generated explanation reaches the same conclusion as the gold one: that a claim about road closures is false because the news evidence says the road will be closed.

ChatGPT-4-32k misinformation detection, with and without retrieved evidence

MetricWithout evidenceWith evidence
Precision0.5950.969
Recall0.6670.939
Accuracy0.6210.954
F1-score0.6280.954

Same test set as the CLIP classifier. Adding evidence is what lifts the system.

figures

The full SINDEX pipeline, from claim and image to verdict and explanation.
Automated evidence retrieval: keywords → news search → similarity filter → relevance filter → six evidence sentences.
The CLIP-based Custom Truthfulness Classifier: claim, image and evidence attend to one another before classification.
The satire detector: BERT with layers 10 and 11 unfrozen, then dropout and a linear classifier.

poster & report

Chat, is this real? mm-DoubleConfirm! : A Multimodal Network for Singapore-context Misinformation Detection

recognition

  • IRC-SET Best Paper Award
  • Springer publication
  • National STEM Talent Search: Top 8 Overall; Top Computer Science Finalist
  • Additional national research awards
  • Recognised by Singapore's Minister of Defence