|
by Ana Romero-Vicente
EU
DisinfoLab Researcher
December 02, 2021
from
DisinfoLab Website
Introduction
...have all taken additional steps to
crack down on Covid-19 falsehoods, although it is proven that they
are
not always effective, and their
AI detection system fails to remove most of the
'problematic' content.
More worryingly,
malicious actors are a step ahead and have already been developing
strategies to escape platforms' content moderation systems.
One of
these strategies consists of using twisted keywords to escape
platforms' COVID-19 disinformation policies.
Therefore, misleading
and fearmongering coronavirus-related content can continue to spread
quickly on traditional social media platforms in subtle disguises.
This case study
briefly explores how misinformers camouflage virus-related Spanish
keywords to spread bogus content.
This represents a challenge for
the platforms that seem unable to detect it and, therefore, to label
or remove it in many cases.
Although limited in
scope, the present contribution should serve as a case study and a
call to action for platforms to improve their systems and keep
implementing additional methods to improve their content moderation
and mitigate disinformation related to the 'pandemic'.
Methodology
The research
focuses on three Spanish words related to Covid-19 that have been
subtly modified in order not to be detected by Facebook, Instagram,
YouTube, or Twitter.
These pieces of unverified
disinformation have
been collected through various social media tracking tools, with the
goal of collecting concrete and significant cases, rather than
providing a comprehensive evaluation of the phenomenon.
The twisted Spanish
words that have been analyzed are the following:
-
"vacunas"
(vaccines)
-
"pandemia" ('pandemic')
-
"Covid-19"
Even though our
study focuses on the 'pandemic', camouflaged words are also used to
spread other narratives.
To illustrate this, we recall a
pro-QAnon
Facebook
video claiming that,
"there is an operation against
p3d0fili14 (pedophilia) and 4dr3n0cr0m0 (adrenochrome)",
...which collected 48k views and over 2k likes.
In addition to our
main blog post, we offer in an attached document with additional
material that covers other modified keywords linked to the
'pandemic',
namely,
"virus", "grafeno" (graphene), "mascarillas" (masks), or "Ivermectina"
(Ivermectin).
These twisted keywords were also used to escape
content moderation and spread disinformation:
Annex
1...
The word camouflage
technique, or how to keep out of the sight of content moderation
systems
Misinformers tend
to subtly tweak keywords so that they are understandable for users
while remaining undetected to social networks' content moderation
systems.
Although they do not pursue the same goal, their techniques
are similar to those used by cybercriminals while performing the
practice called
cybersquatting, playing on the spelling of domain names.
To a
certain extent, it also reminded us of our investigation "Indian
Chronicles", where some Indian actors regularly inverted letters in
names, very likely in an attempt to confuse people and hide online
tracks linked to their foreign influence operation.
In particular, we
noticed a number of recurring strategies:
-
Vowels are
replaced by numbers due to their pictographic similarity:
-
Vowel "A"
replaced by the number 4: e.g. VACUNA (vaccine) → V4CUN4.
-
Vowel "E"
replaced by the number 3: e.g. GRAFENO (graphene) → GR4F3N0.
-
Vowel "O"
replaced by the number 0: e.g. COVID → C0V1D.
-
Vowel "I"
replaced by the number 1: e.g. PANDEMIA ('pandemic') → P4ND3M14.
-
Consonants
are replaced by other consonants that, in Spanish, sound the
same or very similar:
-
"V"
replaced by "B": e.g. COVID → COBID;
VANUNA → B4CUN4.
-
"D"
replaced by "T": e.g. COVID→ COVIT.
Or a combination of
both: e.g. COVID→ COBIT.
NACUVA
instead of VACUNA.
-
Punctuation
marks and symbols are inserted between the letters (.,
…,!, $,@, π, €, ∆, etc.).
Bicho 19 (bug 19) instead of Covid-19.
From the above,
there are many possible combinations:
-
V4.kuna,
b4cun4, V.A.C.U.N.A.S, va-cuna, va…cuna, v4-kuna, b4qVn4z,
V@(u¬a, K4KVNA, v@kN4s instead of "vacuna/s"
(vaccine/s).
-
V1RU5, V1ru$
instead of "virus".
-
@#plan#demia,
pl@πd€m1∆ instead of "pandemia"
('pandemic').
-
K0 B1T, k0b1d,
C0 V ¡D, c0*vid, C0%VID-19, C(o(v(i(d-19, C0 •VID•19, CV I9
instead of "Covid-19", etc.
In addition, we
observed other camouflage methods in videos and images:
-
Some of the
analyzed posts contain images (photos and screenshots),
including altered keywords. Social networks seem, in some cases,
unable to recognize Optical Character Recognition (OCR), i.e.
the text encoded in visuals.
-
In some videos,
the speaker silences a keyword when pronouncing it and, instead,
that keyword is superimposed on the video screen duly
camouflaged:
#1:
Screenshot taken
from Instagram in October 2021
It is important to
note that the Covid-19 words camouflaged by some social media users
do not always have misinformative intentions.
However, the majority
are malicious actors who deliberately use this method to spread
disinformative messages to escape being deplatformed or
labeled.
In this regard,
content moderation by Facebook, Instagram, Twitter, and YouTube
sometimes seems haphazard since the implementation of platforms'
policies has flaws, leaving out considerable amounts of
disinformation.
From our observation, many of the posts containing
camouflaged words within a misinformation message go entirely
unnoticed.
However, there are times when they are detected and
either moderated or addressed by applying standard information
panels - these panels are designed to give more context about a
topic regardless of the presence or absence of disinformation in a
social media post.
An example:
A June 2021
post wrote:
"Maddie de Garay,
12 years volunteer for the study of b4.cu.n4s (vaccines) k0vbld (Covid), she is now in a
wheelchair and eats through a tube".
#2:
Screenshot taken
from Facebook in November 2021
In this case,
Facebook detected that the post contained a message linked to
Covid-19 vaccines.
The platform added an information panel saying
that,
"Covid-19 vaccines go through many tests for safety and
effectiveness and are monitored closely".
It is unclear if the
information panel was added by Facebook because of the platform
detecting the twisted keywords or due to some other elements in the
post.
In this case, we
can notice that the generic information panel doesn't address the
misleading information here, which consists in using the very
specific case of a young American woman to suggest 'unproven'
conclusions about
the overall dangerousness of COVID-19 vaccines.
Moderation
gaps
The items below are
a brief selection of several posts containing hidden keywords that
have not been detected, labeled by independent fact-checkers, or
removed from traditional social media platforms, despite explicitly
breaking their Covid-19 policies (i.e.
Facebook,
Instagram,
Twitter, and
YouTube).
Concerning the
narratives, word camouflage techniques are mainly spreading
anti-vaccine stances, conspiracy messages, and denialist content.
Keyword 1: Vacuna (vaccine) vs. V4CUN4, B.4.C.U.N.4, NACUVAS,
V.A.C.U.N.A.S, V4 CUN 4
Facebook
#3:
Screenshots taken
from Facebook in October 2021
The screenshots in Figure 3 include a post with 84k views and 5k
shares, claiming that,
"the V4CUN4 (vaccine) has four potential
dangers and that they can lead to the death of fetuses, female
infertility and serious side effects".
The other posts state that
there are safer treatments,
"than the B.4.C.U.N.4 (vaccine)" or that "Covid-19 has a 99,96% recovery rate. We do not need quarantines,
nor protocols, we do not need V.A.C.U.N.A.S (vaccines)".
Twitter
#4:
Screenshots taken
from Twitter in October 2021
The third group of figures shows a tweeted video that reached 13k
views, claiming that,
"this is not a V4CUN4 (vaccine),it is a horror
experiment on humans. It's a war".
Another tweet maintains that,
"the
so-called NACUVAS (vaccines) are not FDA-approved".
Another item
declares that,
"Margarita's 38-year-old brother died after being V.A.C.U.N.A.R.S.E (vaccinated), a victim of experimental
V.A.C.U.N.A.S. C.O.V.I.D.-1.9 (Covid-19 vaccines)".
Instagram
The selected examples show how Instagram's OCR detection system
failed to tackle disinformation.
#5:
Screenshots taken
from Instagram in October 2021
The first post obtained over 2k likes by claiming that,
"being
anti-cov1t (Covid) V4CUN4s (vaccines) is not being anti-vax (…)
V4CUN4s (vaccines) are experiments and f4rm4ceuticas (pharmaceutical
companies) make us sign a contract that says they are not
responsible for any adverse effects".
The other photo, which received almost 3k likes, people hold a
banner that reads:
"The manufacturer is not responsible; the
government is not responsible; doctors are not responsible. But if
you do not get V.A.C.U.N.A.S (vaccinated), you are irresponsible.
They are cheating you. Wake up now".
YouTube
The caption of the video below reads:
"The most dangerous V4 CUN 4
(vaccine)", as the video proceeds to proclaim the vaccine a mass
experiment to turn every patient into "Bill Gates' mutant pet".
Interestingly, the title of the video also suggests that YouTube
might have already taken some actions against this video when it was
published on another YouTube channel.
#6:
Screenshots taken
from YouTube in December 2021
Keyword 2: Covid-19 vs. K0 B1T, K0b1d 1984, KOBiD, Bicho 1'9, ¢ 0V-!
D, C.O.V.I.D
Facebook
The first post states that,
"Italy follows France and announces the
green pass as a requirement to enter restaurants.
The Italian people
have taken to the streets to claim their right to free movement and
to protest against the tyranny of the K0b1d 1984 (Covid)
dictatorship".
The second post is from a disinformative outlet
according to which,
"38.700 deaths and 3.968.000 adverse reactions
have been registered due to the K0 B1T (Covid-19) injection".
#7:
Screenshots taken
from Facebook in October 2021
Twitter
The tweet below claims that,
"forcing more than 99%, which is the
healthy population, to get vaccinated, while less than 1% are sick
with the bicho (bug - Covid) is like asking healthy people to
undergo chemotherapy".
#8:
Screenshot taken
from Twitter in November 2021
Instagram
#9:
Screenshot taken
from Twitter in October 2021
The above post received almost 2k views and claims that,
"the circus
is over. On May 6, 2021, Texas physicians testified before the
Senate to oppose the mandatory v @ q-n4s (vaccines) for ¢ 0V-! D (Covid)…"
YouTube
#10:
Screenshots taken
from YouTube in October 2021
The first video, with over 11k views, has the following caption:
"Doctors from around the world explain how they Fals3an (falsify)
deaths by the bicho (bug)".
In the other video, extracted from a TV
interview, which has more than 60k views and 3k likes, former Miss
Spain talks about,
"Data of KOBiD_B4Kun4s (Covid-19 vaccines)",
...and
expresses several Covid-19 denialist messages on the allegedly low
Covid-19 mortality rate as opposed to the danger of vaccines.
Keyword 3: pandemia ('pandemic') vs. P4ND3M14, Pandemi4, P4ndemi4,
P4nd3m1A
Facebook
#11:
Screenshot taken
from Facebook in October 2021
The post (with more than 1k views) spread the conspiracy that
Tanzanian President John Magufuli was disappeared because,
"He was
the only world leader to fully expose the entire p4ndemi4
('pandemic')" and "to go against the v4cun4s (vaccines) plot" aimed at
poisoning citizens, but "now the mainstream fake news media say he
died of c0v1d" (Covid).
This example is particularly relevant to
highlight the widespread use of word camouflage beyond the
virus-related lingo, as terms like,
"T4NZ4NI4NO" (from Tanzania),
ASES1N4D0 (killed), or "desapar3ció" (disappeared) employ the same
technique.
Twitter
#12:
Screenshots taken
from Twitter in October 2021
The first item (left) is relevant as it shows that disinformation
travels across platforms.
In fact, the tweet asks:
"What do people
die of if the P4nd3m14 ('pandemic') is not real?" and attaches a video
from a smaller, fringe platform (Odysee) where the video received
more than 5k views.
Similarly, the other tweet (right) asks:
"Doctor
Luis de Benito, where is the P4nd3m1A ('pandemic')?" and contains a
link to a Covid-19 denialist video on YouTube, which has not been
removed by the platform, with 30k views.
Instagram
#13
Screenshot taken
from Instagram in December 2021
This post claims to be,
"an excerpt from an interview by
James
Corbett (The Corbett Report) in which the p4nd3m1a ('pandemic') is
mentioned as a smokescreen to what could happen", projecting a
'deep
state' conspiracy.
Conclusions and recommendations
-
The skill of
misinformers in developing new techniques to continue spreading
their messages cannot be ignored.
Although there are other
platforms where false and deceptive content spreads without any
restriction, mainstream social media platforms have the widest
audiences, drawing the interest of malicious actors.
Besides
escaping moderation, the use of word camouflage also reinforces
in-group identity, the idea of being part of a community sharing
the same secret code and opposing the censorship power of big
tech companies.
-
Tackling the
phenomenon of camouflaging keywords to spread false and harmful
messages is a complex challenge.
An excessively limited list of
disinformation-related keywords can allow misinformation content
to still find its way onto the platforms.
However, the opposite
can lead to the removal of content that does not violate the
rules, like
Twitter
labeling as COVID-19 disinformation all tweets that
mentioned 5G and oxygen together, irrespective of their
non-conspiratorial nature.
Blocking lists and techniques must be
carefully and continually re-evaluated and optimized to achieve
this delicate balance.
-
Overall, there
are actions that platforms should embrace, such as making better
use of Optical Character Recognition (OCR) to tackle content
that violates their terms and conditions, or improving other
detection techniques including their automated moderation
systems, to deal with current difficulties.
|
