State-backed fake profiles use GenAI to launder disinformation

At a glance

What this is: NHI Mgmt Group's analysis, drawing on Cyabra's blog post: State-backed fake profile networks use GenAI, persistence and coordinated amplification to launder disinformation into seeming legitimacy.

Why it matters: Identity and trust teams need to treat social platforms as influence infrastructure, because fake personas and coordinated networks can undermine public confidence and brand reputation at scale.

By the numbers:

• While fake profiles typically make up 5% to 12% of conversations on social media, election-related discussions can rise to 30%, 40% or even 50% fake profiles.

Context

Social media manipulation is an identity and trust problem before it is a content problem. Fake personas, coordinated bot activity and synthetic media distort who appears to be speaking, which changes how audiences assign legitimacy and how quickly narratives spread. Social engineering matters here because the objective is persuasion through deception, not just volume.

The governance gap is that most monitoring still treats posts as the primary unit of review. State-backed campaigns exploit persistence, account aging and narrative repetition, which means the real control challenge is proving whether an account, network or source is authentic over time.

Key questions

Q: How do fake social media profiles gain credibility over time?

A: They gain credibility by looking active, consistent and socially validated before they are used in a campaign. Age, cadence, cross-engagement and profile completeness can make a fabricated account appear routine, which is why defenders need to assess lifecycle signals, not just the content it publishes.

Q: Why do disinformation campaigns use bot networks instead of single accounts?

A: Disinformation campaigns use bot networks because many accounts can amplify the same message at speed, mask coordination, and survive takedowns better than one account. Shared infrastructure, synchronised posting, reused content, and abnormal follower or engagement patterns help operators simulate independent voices and evade platform controls that flag single-account abuse.

Q: What are the signs of a coordinated influence operation online?

A: Signs of a coordinated influence operation include many accounts posting the same claims, links, or hashtags in a tight time window, with reused language, profile patterns, or sudden bursts from newly created accounts. The phrase behavioural, temporal and graph analysis describes the main detection methods: studying posting behaviour, timing, and connection patterns to expose coordination.

Q: How should organisations detect GenAI-assisted disinformation?

A: Organisations should detect GenAI-assisted disinformation by combining content analysis, provenance checks, and campaign-level monitoring. Look for mass-produced variants of the same claim, unnatural posting cadence, synthetic media artefacts, repeated prompts or templates, and account clusters that share distribution paths. Human review and platform telemetry help separate automated persuasion from ordinary user discussion.

Technical breakdown

How fake profile networks build credibility over time

Fake profile operations do not usually depend on one dramatic post. They rely on account ageing, intermittent posting, cross-engagement and profile enrichment so that a fabricated identity looks dormant at first and then gradually appears normal. That lifecycle matters because detection systems that focus only on sudden bursts of activity miss the slow construction of trust. In practice, the network is the product, while any single account is only one node in the campaign. The operational goal is to make synthetic activity blend into genuine conversation long enough to influence perception.

Practical implication: Track account creation patterns, dormancy, interaction clusters and narrative reuse together instead of reviewing posts in isolation.

Why GenAI changes disinformation operations

GenAI lowers the cost of producing text, images and video that appear locally relevant and emotionally plausible. That matters because influence campaigns no longer need large manual teams to keep a fake persona active or to generate convincing supporting material. Synthetic media can also reinforce other fake accounts, creating a self-supporting illusion of consensus. The risk is not merely false content. It is scalable identity fabrication, where personas, evidence and endorsement all become machine-assisted.

Practical implication: Assume synthetic content can be produced at volume and design verification workflows that check source authenticity, not just content quality.

Why election periods amplify deception

Election periods compress attention, increase emotional response and create a ready-made reason to share unverified claims. That gives influence operators a stronger operating environment because audiences are already primed to interpret ambiguous content as politically meaningful. The article's figures show that fake profiles can become a much larger share of the conversation during these windows. The mechanism is not a technical exploit in the classic sense. It is a trust exploit that benefits from timing, repetition and social validation.

Practical implication: Increase scrutiny around election-adjacent narratives and treat volume spikes as a cue to investigate coordination, not as proof of public sentiment.

Threat narrative

Attacker objective: The attacker wants to shape public opinion, weaken trust in institutions and make disinformation endure as part of the narrative ecosystem.

1. Entry begins with the creation or purchase of fake profiles and bot infrastructure that can be aged before any public campaign starts.
2. Credentialed-looking personas then amplify one another with GenAI text, synthetic images and deepfakes to simulate authentic participation and consensus.
3. Escalation occurs when the network gains enough credibility to reshape conversations, seed doubt and push false narratives into wider circulation.
4. Impact is the erosion of trust in institutions, brands or candidates, with false narratives persisting long after the original campaign ends.

Breaches seen in the wild

• New York Times GitHub breach 2024: An exposed GitHub token gave an attacker The New York Times' repositories; the 270GB leak held 4,875 unique secrets.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Fake personas are now infrastructure, not just content. The article shows that state actors are not relying on isolated false posts. They are building account fleets, engagement loops and synthetic media pipelines that function like an operational influence stack. That changes the governance problem from content moderation to identity assurance at scale. Practitioners should treat network persistence and persona lifecycle as the real control surface.

Credentialed-looking trust is the new attack surface. Social platforms reward aged accounts, familiar names and repeated interaction, which means deceptive actors can accumulate credibility before they push a narrative. The security failure is not only that false content exists. It is that the system grants legitimacy to accounts that look socially validated. Practitioners need to ask how authenticity is established before reach is granted.

Disinformation campaigns exploit the same trust assumptions as social engineering. The article's examples show influence operations using persuasion, urgency and social proof in ways that mirror classic deception tactics. That makes social engineering controls relevant outside the help desk and email inbox. Where the audience itself is the target, social engineering governance becomes a broader trust discipline. Practitioners should align monitoring, identity checks and escalation rules across brand, policy and public-facing channels.

Election spikes expose a verification gap in audience trust. The article's numbers show that fake profiles can make up a much larger share of election-related discussion than in ordinary conversation. That is a governance warning, not a media curiosity. High-stakes public moments create fertile ground for synthetic consensus, so practitioners should prepare for coordinated narrative spikes before they appear. The control objective is early detection of coordinated identity behaviour, not just post removal.

What this signals

Disinformation governance now needs identity-style verification. When synthetic personas can be generated, aged and coordinated at scale, the verification problem starts before publication and continues after engagement. The practical shift is toward source credibility, behavioural correlation and network lineage, not just content takedown.

Audience trust is becoming a measurable security asset. Influence campaigns succeed when they can simulate consensus long enough to shape perception. Practitioners should therefore watch for repeated narratives, abrupt engagement spikes and clusters of accounts that behave like managed identities rather than real communities.

For practitioners

• Audit persona authenticity signals Track account age, posting cadence, follower graph changes and repetition patterns to distinguish organic communities from coordinated networks.
• Correlate content with network behaviour Review whether the same claim is appearing across linked accounts, reused phrasing, mirrored timelines and synchronized amplification.
• Set escalation thresholds for election periods Treat rapid spikes in engagement around sensitive topics as a trigger for deeper investigation into coordination and synthetic identity behaviour.
• Verify source authenticity before amplifying Require secondary checks on the origin of images, video and quoted claims when they are likely to influence reputation or public confidence.

Key takeaways

• Fake profile campaigns are effective because they blend persona construction, synthetic media and repeated amplification into a single trust operation.
• Election-related discussion can contain a far higher share of fake profiles than ordinary social conversation, which raises the odds of coordinated manipulation.
• Defenders need to monitor identity behaviour, network structure and narrative reuse together, because content-only review misses how influence campaigns actually scale.

Related NHI Mgmt Group resources: Social Engineering Guide · Top 10 Social Engineering Attacks · Deepfakes, Social Engineering and AI Impersonation Guide

Key terms

• Disinformation: Disinformation is false or misleading content created with the intent to deceive. Unlike accidental misinformation, it is designed to manipulate beliefs or behaviour, making it a common tool in social engineering, fraud, and influence operations.
• Bot Network: A bot network is a coordinated set of automated or semi-automated accounts used to amplify messages, imitate genuine users, or overwhelm moderation systems. When paired with crypto-funded infrastructure, bot networks can support disinformation campaigns, account creation, and rapid operational scaling across platforms.
• Fake Profiles: Fake profiles are fabricated online identities created to impersonate real people or to appear as legitimate users. They are used in social networks, messaging platforms and fraud schemes to gain trust, collect data, spread scams or bypass controls that rely on profile reputation or account history.
• Deepfake: Synthetic or altered media created with AI or machine learning so that a person appears to say or do something they never did. In security terms, deepfakes are trust attacks that can distort identity verification, approval workflows, and fraud detection.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, secrets management and workload identity. It helps identity and security practitioners connect lifecycle control to operational risk.