Deepfake Technology Issues

As conflict rages in the middle-east, deepfakes have made the 'fog of war' thicker than ever. I spoke to BBC News about why this time is different compared to the other wars i've monitored for deepfakes since 2018. The use of deepfakes to spread disinformation and doubt in war isn't new. Conflicts including the ongoing Russian invasion of Ukraine, Israel Gaza war, and brief 2025 India Pakistan crisis all saw deceptive AI generated images, videos, and audio being circulated. This was in addition to the 'evergreen' cheapfakes such as video game footage, out of context photos, and crude edits that often circulate in acute low information crisis scenarios. What's different with the current conflict, as I shared with Thomas C., is the volume and realism of AI generated content being created as powerful AI tools have become commodified. "The number of different tools that are now available to create a wide range of highly realistic AI manipulations is unprecedented," says Henry Ajder, a generative AI expert. "We have never seen these tools so available, so easy and so cheap to use," he says. The result is the volume has been overwhelming at the same time that discerning authentic from deepfake/synthetic footage has become more laborious and riddled with doubt. This isn't just for everyday people trying to navigate a choked X/Twitter feed, but also for governments and other institutions that need to make informed decisions about how to respond to fast evolving and high risk situations. Last year, I posted about how a fake AI generated images of a damaged train bridge caused UK train operators to immediately cancel services. With lives on the line, a decision was made to act with caution based on the consequences if it were real and the lack of confidence in making that determination. Unsurprisingly, military and intelligence have far better digital forensic capabilities than a train operator when assessing content authenticity, but as we saw with the tragic bombing of an Iranian girl's school, intelligence flaws exist and in the deepfake context detection tools are far from invulnerable. Floods of highly realistic deepfakes in fraught scenarios is the new norm. Now more than ever, we need better 'fog lamps' to cut through these increasingly murky times (see Coalition for Content Provenance and Authenticity (C2PA), Content Authenticity Initiative, GetReal Security, and Google SynthID) Read the full piece below, with excellent insights from WITNESS' Mahsa Alimardani, Timothy Graham, and Victoire Rio. https://lnkd.in/d6cSPpA8

The New Corporate Threat: Deepfakes That Even Experts Can't Detect Welcome to the new reality where AI doesn’t just generate content, it manufactures convincing lies. You’ve probably seen it: - A CEO announces a fake acquisition. - A politician "says" something they never did. - A voice note "from your boss" requests a fund transfer. It all looks real. But it’s not. It’s a deepfake AI-generated audio, video, or images designed to deceive. Why it matters: Deepfakes are no longer just internet tricks or entertainment. They’re now: - Financial fraud enablers (voice clones used to scam employees) - Corporate risk vectors (fake news impacting stock prices) - Political weapons (manipulated clips used to sway public opinion) - Personal threats (identity misuse, blackmail, defamation) How to spot a deepfake  Look for: - Unnatural blinking or awkward lip sync - Plastic skin or weird lighting - Robotic tone or emotionless speech - Out-of-character statements - No credible source backing the video If it feels off, it probably is. What you can do: - Pause before sharing - Use tools like Deep ware, Microsoft Video Authenticator, or Adobe Verify - Train your teams especially PR, legal, and finance - Push for content provenance in your organization In the GenAI era, trust is currency. Don’t spend it on content you didn’t verify. #artificialintelligence

AI PR Nightmares Part 2: When AI Clones Voices, Faces, and Authority. What Happened: Last week, a sophisticated AI-driven impersonation targeted White House Chief of Staff Susie Wiles. An unknown actor, using advanced AI-generated voice cloning, began contacting high-profile Republicans and business leaders, posing as Wiles. The impersonator requested sensitive information, including lists of potential presidential pardon candidates and even cash transfers. The messages were convincing enough that some recipients engaged before realizing the deception. Wiles’ personal cellphone contacts were reportedly compromised, giving the impersonator access to a network of influential individuals. This incident underscores a huge growing threat: AI-generated deepfakes are becoming increasingly realistic and accessible, enabling malicious actors to impersonate individuals with frightening accuracy. From cloned voices to authentic looking fabricated videos, the potential for misuse spans politics, finance, and way beyond. And it needs your attention now. 🔍 The Implications for PR and Issues Management: As AI-generated impersonations become more prevalent, organizations must proactively address the associated risks as part of their ongoing crisis planning. Here are key considerations: 1. Implement New Verification Protocols: Establish multi-factor authentication for communications, especially those involving sensitive requests. Encourage stakeholders to verify unusual requests through secondary channels. 2. Educate Constituents: Conduct training sessions to raise awareness about deepfake technologies and the signs of AI-generated impersonations. An informed network is a critical defense. 3. Develop a Deepfakes Crisis Plan: Prepare for potential deepfake incidents with a clear action plan, including communication strategies to address stakeholders and the public promptly. 4. Monitor Digital Channels: Utilize your monitoring tools to detect unauthorized use of your organization’s or executives’ likenesses online. Early detection and action can mitigate damage. 5. Collaborate with Authorities: In the event of an impersonation, work closely with law enforcement and cybersecurity experts to investigate and respond effectively. ———————————————————— The rise of AI-driven impersonations is not a distant threat, it’s a current reality and only going to get worse as the tech becomes more sophisticated. If you want to think and talk more about how to prepare for this and other AI related PR and issues management topics, follow along here with my series or DM if I can help your organization prepare or respond.

The rise of AI-powered fraud reached a critical inflection point in 2024, and the numbers are staggering. Studies from this year paint a sobering picture of our digital landscape. According to MiTek's 2024 Identity Intelligence Index, 76% of financial institutions report that fraud cases have become more sophisticated, with deepfakes emerging as a primary attack vector. Research from Sift this year reveals that 52% of businesses now face deepfake attacks daily or weekly, creating unprecedented risks to critical communications. A September 2024 Medius study found that 87% of finance professionals admit they would make a payment if "called" by their CEO/CFO — yet 53% have already experienced attempted deepfake scamming attacks. Most concerning: iProov's August 2024 research shows that while 70% of industry leaders believe AI-generated attacks will significantly impact their organizations, 62% worry their organizations aren't taking the threat seriously enough. At Reality Defender, our mission is clear: secure critical communication channels by detecting deepfake impersonations in real-time. We're working tirelessly with enterprises to build resilience against this rapidly evolving threat landscape. The trust gap in our AI-powered world is widening. Yet through proactive defense and cutting-edge detection capabilities, we can help organizations interact with confidence in an era of synthetic media.

Deepfakes aren’t a tech story. they’re a trust story A few days ago, a doctor in Hyderabad lost money to a #deepfake video that showed a cabinet minister “endorsing” an investment scheme on #Instagram. If that sounds distant, it isn’t. This is the new fraud funnel: authority, urgency, proof… all manufactured at scale. As #communicators and leaders, we can’t outsource this to compliance or IT. #Trust is now an operational KPI. What we as communicators need to do? •      Treat digital hygiene like fire safety. Run quarterly drills that teach people how fakes travel and how to report them •      Publish an authenticity sheet. List official handles, verified domains, escalation numbers and a simple “how to verify” flow for customers and employees •      Watermark outbound content and adopt content credentials where possible. Make the real easier to prove than the fake is to spread. •      Rewrite influencer and media contracts with an “authenticity clause” and takedown SLAs. If your face or footage is misused, minutes matter. •      Stand up a rapid debunk protocol. Pre-approved copy, visuals, spokespeople and a single public link that carries all corrections. •      Close the platform loop. Nominate a trust lead who keeps warm lines with platform policy teams so your takedown requests don’t start cold. Silence helps the scammer. Clarity helps the vulnerable. What would you add to this deepfake playbook? If you’ve seen a convincing fake lately, share it below and let’s decode why it worked. #digitalsafety #misinformation #brandprotection #reputationmanagement #contentauthenticity #aiethics #factchecking #onlinescams #communications

I tested Google's Nano Banana Pro this morning. We're cooked. Look at these images. Both AI generated. Same person. Same lighting. Same realism. You can't tell the difference anymore. Here's the part that should worry you: The barrier to creating deepfakes just disappeared. Anyone can do this now. No technical skills needed. No expensive software. Just a prompt and 30 seconds. Deepfakes aren't just a tech risk. They're a trust collapse multiplier. Think about what this enables: → Political campaigns can fabricate evidence → Corporate fraud becomes easier to execute → Reputation attacks can be launched in minutes → Romance scams will scale exponentially → Court evidence loses credibility Yes, Google added watermarks. Yes, detection tools exist. But there's a problem: Most people won't check. Social media moves faster than fact-checkers. A fake image goes viral in minutes. The damage is done before truth catches up. This is the perfect recipe for large-scale deception. We're not ready for this reality. The technology is advancing faster than: Our ability to regulate it Our digital literacy programs Our collective understanding We need honest conversations about guardrails. Right now. Not next year. Not when the first major scandal hits. Now. The window for preparation is closing faster than anyone expected. Smart people are already preparing for this: They verify with multiple sources They check credibility before sharing They assume images could be fabricated Most people are unprepared to face deepfakes. The gap between technology and digital literacy has never been wider. And it's growing every day. Every organization will face this, but only a few are preparing for it. What's your company doing to prepare for a world where seeing is no longer believing? If you care about staying ahead of AI & tech risks, this is the place to be. #AI #Deepfakes #DigitalTransformation #FutureOfWork #RiskManagement

AI deepfake nude images are now being created of schoolchildren in at least 28 countries. A new analysis by WIRED and Indicator found nearly 90 schools and over 600 pupils affected globally since 2023. Boys are downloading photos from Instagram and Snapchat and using freely available AI apps to create fake nude images of girls they know. This is not a niche technology problem. It is happening in schools right now, and the children affected are your employees' children. The problem in the UK : 1.50% of UK children aged 8-15 have seen at least one deepfake online in the last six months (Safer Internet) 2.43% of those aged 16+ have also encountered deepfakes online Only 20% of children aged 8-15 feel confident they can identify (SWGL) 3. Internet Watch Foundation reports of AI-generated child sexual abuse material more than doubled year-on-year, rising from 199 reports in 2024 to 426 reports in 2025 (IWF) What this means for working parents: Parents dealing with this at home are carrying it into the workplace. When a child is targeted, the fallout includes school refusals, police involvement, legal processes, and ongoing emotional distress. That pressure shows up as reduced focus, unplanned absence, and disengagement at work. For HR leads and people teams: 1. Does your employee wellbeing programme include any support for digital parenting challenges? 2.Parents facing this kind of crisis need somewhere to turn before it escalates 3.A structured digital parenting support resource reduces the time employees spend trying to navigate this alone For safeguarding leads and school staff: 1.Staff need to understand how nudify apps work and what pupils may be experiencing 2.Victims often do not disclose because they fear no action will be taken 3.Schools need clear protocols before an incident happens, not after For parents: 1.Check what photos of your child are publicly visible on social media 2.Talk to your children about what deepfakes are and why sharing images of others is harmful and illegal 3.Know that victims are not to blame and early disclosure leads to better outcomes The scale of this is almost certainly larger than reported. One Unicef estimate puts the number of children affected globally at 1.2 million last year alone. Organisations that support parents with digital challenges now are building a more resilient, better-supported workforce. Get in touch to find out how Kids N Clicks can support parents in your organisation. Image credit : Wired

Deepfake, Grok, and the global ethics crisis in AI When AI can fabricate bodies, mimic voices, ignore consents - the harm is engineered. 1️⃣ Consent ignored. Boundaries erased. Last week, Ashley St. Clair - public figure and mother of Elon Musk’s youngest son - says Grok, the AI built into X (formerly Twitter), generated sexually explicit deepfakes of her, including photos when she was a minor. She revoked consent. Grok acknowledged it. Then it kept going. After she spoke out, her ability to earn income on X was revoked. Musk’s response? A threat to seek full custody of their toddler. 2️⃣ A symbol of design failure Grok is one of many, since 2024 deepfakes are escalating globally: ▫️ Taylor Swift deepfakes flooded X ▫️ Teen girls targeted in AI-generated nudes across South Korea & Europe ▫️ President Biden's voice mimicked in robocalls before U.S. primaries ▫️ Crypto AI-generated audio scams of political leaders in Malta and India ➡️ When AI lacks ethics, the fallout is human. A forensic audit of 20k+ Grok-generated images revealed the scale of harm: 🔹 53% showed individuals in minimal clothing 🔹 81% of those were women 🔹 2% impact minors ➡️ A system without ethics now under legal scrutiny worldwide. 3️⃣ Governments are drawing red lines on AI deepfakes: 💠 EU: Non-consensual sexual deepfakes must be criminalised by Jun 27 💠 UK: Ofcom launches investigation into X under the Online Safety Act 💠 Spain: Draft law bans unauthorized use of AI-generated images & voices 💠 Malta: Criminal penalties for AI-enabled harassment and deepfake abuse 💠 Indonesia & Malaysia: Blocks/bans Grok, citing risks to women & children 💠 Canada: Declares deepfake abuse a form of “violence” & drafts legislation 💠 Australia: Uses removal powers under existing online safety laws 4️⃣ Ethical standards are emerging, slowly The Council of Europe  is drafting the world’s first binding AI treaty with safeguards against deception and abuse. OECD - OCDE, UNESCO, the G7 call for: 🔸 Accountability for harmful design 🔸 Consent and dignity online 🔸 Transparency in AI media ➡️ None are binding. No AI model must assess deepfake risks. 5️⃣ This is not just an AI crisis. It’s a moral one. Grok has demonstrated that an AI product can: ▪️ Undress a child in simulation ▪️ Confirm it lacks consent ▪️ Continue generating content anyway all within milliseconds, and without external intervention. ➡️ This a system working as designed and ethically abandoned. Final thoughts AI ethics means stopping systems that predictably harm and are built to evade responsibility. Grok exposed the truth: AI can generate abuse, even when told to stop. The global tide is shifting toward prohibition. The moral cost of delay is rising fast. #AI #GenerativeAI #AIGovernance #Deepfakes #AIEthics

In January 2024, a finance worker in Hong Kong joined a video call with his CFO and several colleagues. He recognised every face. He heard every voice. He transferred $25.6 million across fifteen transactions. A week later, it turned out that no one had been on that call. Every face, every voice — generated by an algorithm trained on publicly available footage. That's the headline version. Here's the one that should keep you up at night. The cost of producing that deepfake was negligible. A few seconds of source video. An open-source tool anyone can download. The technology that once required a Hollywood studio now runs on a laptop. And it's accelerating at a pace that defies linear thinking. We've spent decades building trust infrastructure - credentials, reputation, personal brands, professional authority. Deepfake technology doesn't just attack that infrastructure. It renders the very concept of visual evidence meaningless. A libelous article lies about you with words. A deepfake lies as you — with your face, your voice, your gestures. The victim watches herself say things she never said, endorse products she never endorsed, appear in contexts she never entered. Research confirms what intuition suggests: victims describe the experience in terms closer to physical violation than defamation. And a neurocognitive experiment at Humboldt University found something even more disturbing — the mere awareness that deepfakes exist changes how our brains process social signals. Smiles perceived as "potentially synthetic" trigger weaker emotional responses. Threat signals retain full power. The asymmetry is corrosive: trust erodes, fear doesn't. This is not a technology problem waiting for a technology solution. Detection tools are perpetually one generation behind generation tools. Watermarking schemes assume good faith from bad actors. Platform takedowns happen after millions of views. The real question is what the law can do — now, with the tools it already has. I wrote this essay to answer that question for a specific scenario: a deepfake used in a competitor's advertising. Your face, selling someone else's product. Your reputation, hijacked to build someone else's business. It sounds niche. It isn't. It's the commercial frontier of a problem that will define the next decade of intellectual property, identity rights, and the meaning of personhood in the digital age. One thing is certain: the question is no longer whether your face will be used without your consent. It's whether you'll know what to do when it happens. 📰 Read the full article via the link in comments 👇

We’ve crossed a threshold with deepfakes. Not because the technology is impressive (which it is), but because the volume is now overwhelming. Estimates suggest that the number of deepfake videos online has grown from hundreds of thousands just a few years ago to millions today, with growth continuing to accelerate. Voice clones can be created in seconds. Digital faces can be swapped convincingly on a laptop. Entire personas can be manufactured on demand. At that scale, this is more than a technical issue. It’s a governance problem. When anyone can convincingly sound like a CEO or look like a public official, the foundations of trust begin to fracture. Verification slows everything down. Skepticism rises. And in the worst cases, people stop believing anything at all. That’s the real risk. Yes, deepfakes are false content. But beyond that, they force societies, organizations, and individuals into a constant state of doubt, where every digital interaction carries friction and a bit more risk. Cognitive freedom depends on our ability to orient ourselves in reality. When authenticity collapses, people either disengage or hand judgment over to automated systems to decide what’s “real enough.” Neither outcome preserves human agency. I enthusiastically support detection tools (and I’m pleased to work with the company Gartner called “the company to beat” in this space). But technology is not quite the full solution. We also need policies and verification practices that recognize a new reality: identity itself is now an attack surface. The question today is whether we can build systems (both human and institutional) that preserve trust, judgment, and freedom in the new age of synthetic identity. That’s the governance challenge. #Deepfakes #DigitalTrust #AIandSecurity #MindSovereignty