It starts the way a lot of online harm starts these days: fast, public, and completely out of the victim's control. An Indonesian singer and public figure, Fransisca Saraswati (also known professionally as Sisca Saras), discovered an AI-generated image of her spreading across X. The account behind it was anonymous, but what made it even more unsettling was that the image wasn't just posted randomly. The tool used to create it, Grok, reportedly tagged her publicly, even though she had not agreed to anything.

Her reaction was blunt and painfully human: humiliated, angry, and deeply upset. And it wasn't only personal. She described it as professionally damaging too, because it distorts identity, attacks dignity, and creates mistrust between public figures and the audiences they depend on.

When CNA spoke to victims in Malaysia, the emotional impact sounded very similar.

Why This Problem Escalated So Quickly

A big turning point was Grok introducing easier image editing features. When tools make it simple to manipulate real photos with a few words, misuse doesn't stay theoretical. It becomes a button people press, a trend people copy, and a flood of content that spreads faster than anyone can report it.

Critics say these features have been abused to generate sexualised deepfakes, including content targeting women and, in some cases, minors. Victims describe it as degrading and violating, because it's not just "a fake image." It's a public attack on someone's dignity, reputation, and sense of safety.

And the real kicker is the speed. By the time a victim sees it, it may already be everywhere.

This Is Not Just "Bad Users" Anymore

One of the most important points raised by experts is that regulators are starting to frame this as more than individual user behaviour.

The argument is basically this: if a platform ships a tool that predictably enables harm, then responsibility doesn't stop at "people misused it." Responsibility also includes how the tool was designed, what safeguards were included, and whether warnings were ignored.

Nuurrianti Jalli of the ISEAS-Yusof Ishak Institute described this as a shift in how accountability is being defined. Instead of treating it purely as misuse, authorities are pointing to systemic failures in deployment and safety design.

In plain terms: if the guardrails were weak or missing, that becomes part of the story.

Southeast Asia Pushes Back With Blocks And Legal Pressure

As backlash grew, several countries in Southeast Asia moved to suspend Grok.

Indonesia was reported to be the first to temporarily block the chatbot. Malaysia followed shortly after, with the Malaysian Communications and Multimedia Commission (MCMC) also announcing plans to pursue legal action against X over what it viewed as insufficient safeguards.

Malaysia's Communications Minister, Fahmi Fadzil, indicated that restrictions would only be lifted once X demonstrates that proper protections are in place to prevent misuse.

Not long after, the Philippines also signaled it intended to block Grok as well, joining the regional response.

This matters because it shows something bigger than one isolated policy decision. It suggests a growing regional willingness to draw a line around what's considered acceptable when it comes to the safety and dignity of users, especially women and children.

X Responds, But The Bigger Question Remains

X has since announced measures aimed at preventing certain forms of image-based abuse involving real people. That response is important, but it also raises the obvious question: why did it take international outrage and government action for meaningful changes to arrive?

Elina Noor of the Carnegie Endowment for International Peace pointed to Malaysia's legal stance as highlighting failures in systemic safeguards, and the perception that the platform was not acting decisively despite prior warnings.

That theme keeps repeating across tech: "We'll fix it" often arrives after damage has already been done.

What Makes Grok Different From Other AI Tools

Another reason Grok is drawing special scrutiny is the perception that it has been more permissive than many mainstream AI platforms when it comes to manipulating images of real people.

That comparison matters because it undercuts the idea that "this is unavoidable." Other platforms have shown that stricter guardrails are technically possible, even if they come with trade-offs. That's why some experts argue AI safety is not just a technical limitation, but a product decision.

Which leads to a harsh but fair conclusion: if harm is predictable, prevention is a choice.

Is Banning The Tool Enough?

A block can slow harm down, send a message, and force urgent action. But it doesn't solve the root problem on its own.

Because even if one tool is banned, the underlying reality stays the same:

A ban is a starting point, not a finish line. The long-term question is what safeguards become mandatory, how quickly platforms must respond, and what consequences exist when safety is treated as optional.

Final Thoughts

What stands out most in stories like this isn't the technology. It's the human impact.

Victims describe feeling humiliated and violated, not because of a "feature," but because their identity was manipulated and broadcast without consent. When governments respond by blocking tools and threatening legal action, it signals that the conversation is shifting from "users did something bad" to "platforms shipped something unsafe."

The big takeaway is simple: generative AI tools don't exist in a vacuum. The choices made in design, defaults, and safeguards shape what the public experiences. And when those choices fall short, the damage doesn't land on the company first.