South Korea investigates Telegram over alleged sexual deepfakes

SEOUL, South Korea – Students of all ages, teachers, soldiers and now journalists, more and more ordinary South Korean women are learning they are victims of a fast-growing form of digital sexual abuse: deepfakes.

South Korean authorities are desperately trying to respond after local media and crowdsourcing initiatives recently uncovered a large number of chat rooms on the messaging app Telegram that are spreading fake sexual images and videos created using artificial intelligence.

The Korean National Police, which last week announced a crackdown on sexually explicit deepfakes, said on Monday that it had launched an investigation into Telegram over possible charges of aiding and abetting the spread of sexually explicit deepfakes on the platform.

The agency said this was the first time South Korean law enforcement authorities had investigated the company, whose founder Pavel Durov was arrested and charged in France last month for alleged illegal activities on the platform.

Telegram spokesperson Remi Vaughn told NPR that the company has “actively removed content reported from Korea that violates the terms of service and will continue to do so.”

The Korea Communications Standards Commission, the government's media regulator, said Telegram had complied with its request and removed 25 deepfakes identified by the commission.

According to journalists and activists who have monitored some of the chatrooms, the vast majority of deepfake victims in South Korea are women and young girls.

Local media reports that perpetrators steal images of victims from social media without their knowledge or consent. Or they secretly take photos of women in their environment at home or school. They then alter the images using artificial intelligence and share the results on Telegram with strangers or users who know the victim.

Some of the chat rooms that appear when searching for terms such as “mutual acquaintance room” or “humiliation room” have thousands of participants.

The number of such chatrooms on Telegram and the extent of alleged abuses are unclear. Many chatrooms are closed and accessible only with an invitation link or permission from the chatroom administrator. Some have reportedly been shut down since activists and media started tracking them.

In a Telegram post on Thursday, the platform's founder and CEO Durov said the company was “committed to working with regulators to find the right balance” between privacy and security, while acknowledging that it has now “become easier for criminals to abuse the platform.”

But data from South Korean law enforcement and government agencies show a sharp increase in digital sex crimes involving fake images in the country.

The state media regulator said it received nearly 6,500 requests to combat sexually abusive deepfake videos between January and July this year – four times as many as in the same period last year.

According to police, 297 cases of crimes related to sexually explicit deepfakes were reported in the first seven months of this year, compared to 180 in all of 2023.

Many of the victims and perpetrators are teenagers. Of the 178 suspects police arrested in the seven-month period, 74% were between the ages of 10 and 19, up from 65% in 2021. And more than half of the deepfakes tracked down and deleted this year by the state's Advocacy Center for Online Sexual Abuse Victims involved minors.

Perpetrators harass women

South Korea has long struggled with sex crimes, including illegal filming, non-consensual distribution of sexually explicit images, online grooming and sexual blackmail.

According to experts on online sex crimes, creators of the type of deepfakes that are prevalent on Telegram often target women they know personally, rather than random strangers.

For victims, the harm caused by such assaults by someone they know goes beyond the violation of their privacy, says Chang Dahye, a research fellow at the Korea Institute of Criminology and Justice in Seoul who has studied online sexual assault.

“They lose trust in their community,” says Chang. “They fear they will no longer be able to maintain their daily lives with the people around them. Basically, their trust in social relationships breaks down.”

What also distinguishes sexually abusive deepfakes from other crimes, according to Chang, is their purpose.

Some perpetrators are motivated by money or resentment.

But Chang says, “The goal of most men who consume this content is to degrade women in general.”

She explains that deepfakes emphasize recognizable faces and are often accompanied by verbal sexual harassment.

“It's a way of expressing misogyny and anger toward women. By mocking and degrading women, they get validation from each other,” Chang says.

In a joint statement last week, women's rights groups said the “root cause” of recurring digital sexual abuse is sexism, accusing President Yoon Suk Yeol's government of failing to recognize this and allowing the problem to grow.

Yoon declared that there was “no more structural sexism” in South Korea and promised to abolish the country’s Ministry of Gender Equality and Family.

The minister's position has been vacant since February, and the ministry's budget for preventing violence against women and supporting victims has been cut significantly this year. In a recently announced draft budget for next year, funding for the Advocacy Center for Online Sexual Abuse Victims, which removes online sexual abuse material, was cut from last year, even though the center has an increasing amount of work to do.

Despite these cuts, a multi-agency government emergency task force and the ruling People Power Party recently promised to toughen investigations and punishments for deepfake crimes and increase support for victims.

Laws on digital sexual abuse have evolved piecemeal as they have tried to keep up with the new types of crimes created by new technologies. This leaves a constant gap between what victims perceive as harm and what the law considers to be a crime, according to Chang of the Korea Institute of Criminology and Justice.

Even if an act is punishable, perpetrators often evade punishment because manipulated or faked material that is “likely to cause sexual desire or shame” and is created “for the purpose of distribution” is currently punishable.

According to police statistics, the arrest rate for fake sexual materials was 48% last year, far lower than the rate for other forms of digital sexual assault.

And even if the perpetrators are brought to justice, according to an analysis by the South Korean broadcaster MBC, about half of them receive suspended sentences.

According to Chang, the legal system still struggles to recognize digital sexual abuse as a serious crime with actual victims. “In many cases, judges believe that the harm is not as serious as in sexual violence with direct physical contact,” she says.

NPR correspondent Anthony Kuhn reported from Seoul, South Korea.

Copyright: NPR