Twitter's Information Operations - An OSINT Analysis

Key Takeaways 🔗

• Twitter is doing better than other platforms by releasing datasets, albeit partial, on Information Operations (IO).
  • There is so much more information yet to be disclosed. Recommendations are given.
• Attribution blindspots seem to be a common problem with social media companies.
• Aggregated Twitter data and Python scripts are available on Github - and will be kept up-to-date.
• Beautiful dynamic data visualization for Twitter’s IO datasets, generated in real time from our GitHub datasets.
• A similar study for other platforms such as YouTube would be interesting. Maybe Google’s Threat Analysis Group could start publishing comprehensive datasets? :)

In our last OSINT analysis of Facebook’s Coordinated Inauthentic Behavior we highlighted the pitfalls of Facebook’s data-sharing policies and the lack of transparency when it comes to processes and awareness of influence campaigns on the platform. Although, previous work has been done on some of the Twitter datasets - in this analysis, we extend our work to examine Twitter’s Information Operations (IO) and the measures they are taking (or neglecting) to combat the rampant growth of disinformation, misinformation, and influence campaigns. All the data used in this analysis was downloaded from Twitter’s archives of suspended accounts. The data from Twitter can be accessed on their transparency report, whereas our aggregated data for this analysis is available through GitHub, including the script used to generate the datasets – feel free to send us pull requests.

Platforms like Facebook and Twitter allow ordinary people, civic groups, and journalists to reach a vast and global audience. Controversially, they have also provided an extremely efficient and inexpensive platform for malign influence operations by foreign and domestic actors alike. It’s been well documented how those platforms are being used to construct people’s digital DNA, steer public debate, set the agenda of what journalists are covering, recruit terrorists, reshape warfare itself, and even “change reality”.

The intensification of election meddling, the widespread false information dissemination, and rise of populism and extremism coincide with the growth of online mobs that include both authentic users and automated spam accounts. They intend to build large audiences around similar interests.

Digital Tribes 🔗

Influence campaigns thrive on basic crowd psychology tactics that are being mobilized and manipulated by both domestic and foreign actors alike. Instead of building relationships and groups to push a meticulous and strategic message, Twitter is used strategically to join conversations and amplify the dominant narrative. This instigates a psychological bias based on tribal affiliations, creating an ecosystem enmeshed in distrust, paranoia, cognitive blind spots, and one dimensional critical thinking –operating in a very similar manner to cults.

Although there are many ways to tackle this from a psychological perspective, one broad way to look at Twitter’s Information Operations is through the lens of Cultural Cognition, which exploits the basic processes of identity formation in humans. Once we identify with a group (joining a cause, following a trendy narrative, or contributing online to the public debate and discourse) we shape our opinions to conform to the views of the groups with which we most strongly identify with. Leading to two outcomes: it creates solidarity in the group, which increases the chances that our group’s views will prevail online (or even in society at large), and it strengthens the group’s acceptance of us as members in good standing.

Once the threat of the “other” is created (whether bona fide or totally fabricated threats), the more we circle the wagons of our opinions to keep the tribe together and keep our identities intact. This creates an inflexible war of polarities that impede compromise and progress.

Social media platforms, like Twitter, offer fertile grounds to not only create echo chambers that circulate and amplify narratives, but to amass a receptive audience. In this environment, confirmation bias is algorithmically propagated, on a mass scale.

Identifying Malign Behavior 🔗

There is still ambiguity concerning how Twitter identifies Information Operations on its platform. A lot of details are yet to be unearthed.

So, how do they identify IO and how do they link accounts together to assume that they are operating together? According to Vijaya Gadde, Legal, Policy and Trust & Safety Lead at Twitter, metadata is used to link accounts’ phone numbers, or email addresses, and in some cases IP addresses. They also rely on online reporting, and tips from external firms.

According to Renee DiResta, from the Stanford Internet Observatory, there are three criteria commonly used to assess whether a given page, account cluster, or channel is manipulative.

• Account authenticity. Meaning, are the accounts authentic run and created by real people, or are they a collection of automated accounts?
• Dissemination pattern. Are the messages distributed in an organic manner or are they spreading in ways that look anomalous to how information spreads? Meaning, are the scale, timing of posts, and volume of posting appear coordinated?
• Content integrity. This is identified by examining whether the domains in question are known to be of suspicious quality. This criteria, more than the others, requires a judgement call.

Perils of Censorship 🔗

Tech giants are responsible for public discourse on a scale unprecedented in human history. Given that centralized global policies at scale are almost impossible to draft and apply, some exceptions include the case of communities on Reddit that have their own moderators that enforce policies.

Although the question of censorship, and free speech vs free reach, go far beyond this analysis it is stills important, however, to bring attention to the protection that those companies are relying on: Section 230 of the Communications Decency Act. Part of the Telecommunications Act of 1996, this piece of legislation, which has been established well before Twitter and other platfroms, gives social media companies broad immunity from being sued for user behavior. Necessary but urgently needing a timely upgrade, this legislation has come under increasing scrutiny, with many critics arguing that tech firms need more accountability.

The road to more transparency and accountability is long, albeit sluggish:

“We are not done. We are not finished.”
- Jack Dorsey, CEO of Twitter, on the Joe Rogan podcast 

Data 🔗

In this analysis, we focus on compiling and presenting the released datasets by Twitter pertaining to Operation Information takedowns. This analysis, similar to our previous work on Facebook’s Coordinated Inauthentic Behavior, is an ongoing and open-source project. Contributions, suggestions, and feedback are all encouraged! – Access the full dataset on Github. Moreover, the Python script that was used to compile the data is also available in the Twitter folder.

According to Twitter’s transparency report, Information Operations specifically pertain to alleged state-backed foreign influence campaigns. This leaves room for speculation and (mis)interpretation of the published datasets. Twitter’s definition of IO, in and of itself, excludes organized campaigns operating domestically, or sophisticated campaigns operating on behalf of foreign actors in a fragmented fashion. To tackle parts of this shortcoming, we wanted to make use of the Georeverse code tagging, but it was only possible in the Iranian dataset. All other geolocation data was removed by Twitter from their published dataset, and the columns longitude and latitude were just displaying “present” in those other datasets. This is unfortunate as we were hoping to be be able to draw additional conclusions from the geographical contexts. Tweet languages, account languages, hashtags, and urls (which we used to extract unique domains), have enabled us to draw a wider context to conduct this analysis. Unlike Facebook, which only discloses numbers without additional information around contexts.

Twitter’s published datasets seem to exhibit another pitfall. Although defined under another category (i.e., Platform Manipulation), spam behavior is included as part of Information Operations, this is most notable when examining published datasets pertaining to the Saudi Arabia takedowns, although, an analysis by the Stanford Internet Observatory revealed that some of the spam accounts appeared to attempt to conceal their commercial and political activity by mass-tweeting of religious, sports, and poetry content. According to the same report, approximately 7% of tweets came from client apps that automatically tweeted religious invocations, Dua’. This article on the Emojitracker captured an interesting trend of using the emoji “♻️” as part of the religious bots posting Dua’ tweets on behalf of authentic users.

Given that we are also tracking Facebook’s efforts to combat influence campaigns (termed Coordinated Inauthentic Behavior) on their platform, we couldn’t help but compare and contrast the difference in their strategies and processes in relation to Twitter. A deeper look at the differences between Facebook’s CIB takedowns and Twitter’s Information Operation datasets reveals the discrepancy in the shared information:

	Facebook	Twitter
Partial vs Full Disclosure	No Data	Good
Data Discrepancy	No Data	Medium
Verified Accounts Information	No Data	Not Present
Attribution Blindspots	High	High
Domestic Information Operations	Low	None
Users Notification	Low	Low

• Partial vs Full Disclosure. Twitter does not disclose all information about suspended accounts. For example, as part of account takedowns from China, only 2% of the accounts and information were made public. Similarly, Twitter disclosed only 7% of the total accounts suspended for violating their platform manipulation policies from Saudi Arabia. We observe this pattern of inconsistent and non-comprehensive data disclosure when Twitter encounters wide-scale spam behavior.

• Data Discrepancy. We have also observed that in other cases, the numbers of total removed accounts deviate from the numbers published in the datasets. We are unable to explain the reason behind those discrepancies. See accounts (number of unique accounts in the *_users_csv_hashed.csv files) vs accounts_reported (number of accounts mentioned in the Twitter blogposts) in our twitter-data.csv for the files:

  • ira_users_csv_hashed.csv
  • iran_201901_1_users_csv_hashed.csv
  • venezuela_201901_2_users_csv_hashed.csv
  • iran_201906_3_users_csv_hashed.csv
  • iran_201906_2_users_csv_hashed.csv
  • iran_201906_1_users_csv_hashed.csv

• Verified Accounts Information. Twitter gives no information whether the suspended accounts were verified. There is no is_verified column in the user accounts datasets.

• Attribution Blindspots. There also seems to be a blindspot for certain state-backed actors or content. So far, there has been no public mentions of India’s troll farms (maybe because it will upset a few), or other documented operations by states like Ukraine and Israel. The reason we pinpoint these two countries is because they previously operated influence campaigns on Facebook. This begs the question: do some influence campaigns only operate on one platform? Or is there a bias in Twitter’s reporting of Information Operations? Do they exhibit a blindspot for some state-backed influence campaigns and not the other?

  • According to a report by the Computational Propaganda Research Project at Oxford University, there has been a 150% increase in countries using organized social media influence campaigns between 2017-2019. The report also lists the countries that are most active with social media manipulation on Twitter. Of the 47 countries listed in the report, Twitter had released datasets for only 9 of them. This raises questions about the other countries. It is highly improbable that Twitter is unaware of the campaigns conducted by the other foreign states, but are they choosing to selectively share datasets that follow a certain political narrative?

• Domestic Information Operations. Twitter datasets only reveal information on foreign IO. When can we expect them to elaborate on domestic campaigns? Is there a bias in their reporting? Or perhaps a bias in the way they define Information Operation as “alleged foreign influence campaigns”?

• Users Notification. Similar to Facebook, Twitter rarely notifies accounts directly affected by influence campaigns, nor does it conduct public briefings.

Attribution Bias vs (mis)Attribution 🔗

Although it is important to acknowledge the role that foreign actors play in public discourse, it is more important to remain critical of attribution patterns. The Russian tactics are being closely studied and replicated by other groups, such as political parties during Senate elections in the US. Twitter also admitted to misidentifying and falsely attributing around 230 accounts originally thought to be linked to the Russian Internet Research Agency. The accounts were later found (by an independent researcher) to be associated with a Venezuelan operation.

We argue that the lack of transparency and inconsistency by Twitter restricts independent researchers from conducting thorough investigations of claims and attributions. This feeds into the popular narrative of collective fear/paranoia and constructed political foes.

Another important point to raise is: correlation does not equal causation. While foreign operations have definitely seeped into the public life, we ought to question whether they actually act as instigators or whether they simply jump on the trending hashtag bandwagon and magnify their presence.

Conclusion 🔗

Disinformation, misinformation, and influence campaigns have become a normal part of the digital public sphere. Employed tactics are continuously evolving at a pace no longer containable by social media platforms. Unless Facebook, Twitter, and others, increase their transparency about shared information, and encourage more open-source investigations, those companies will continue to play a non-stop game of whack-a-mole with influence campaigns.

As tactics and Information Operations are evolving, the modus operandi is also evolving: influence campaigns are also using other platforms such as Reddit, YouTube, or even Tiktok. Last year, Google’s Threat Analysis Group closed 210 YouTube accounts “to combat coordinated influence operations” but there is very little information on the other accounts that they have closed. It would also be nice if Google’s Threat Analysis Group released Twitter-like datasets about those accounts.

Unfortunately, very few companies provide comprehensive datasets and we can only ask ourselves what new and creative ways domestic and foreign actors are utilizing to upgrade their influence campaigns. Until then, those campaigns will keep seeping into the collective conciousness… For instance, Netflix docuseries are growing in popularity, could they also be subconsciously used against us?