South Korea faces a deepfake porn crisis

South Korea has been rocked by a wave of sexually explicit deepfake images created and shared online, indiscriminately targeting women and girls using their school photos, social media selfies and even military portraits. A Telegram channel with more than 220,000 participants was reportedly used to create and share these artificial intelligence-generated pornographic images, serving as a global reminder of the grim consequences of the widespread rush to this technology.

Users could upload photos and create explicit content using the faces of their friends, classmates or partners in seconds. Initial reporting by local news outlets prompted more women to come forward as victims and more Telegram channels were uncovered, revealing the true extent of the problem. Many of the victims affected are minors. Late last month, protesters gathered in Seoul wearing white masks over their eyes, demanding justice.

As outrage grew, South Korean President Yoon Suk Yeol called on his government to crack down on digital abuse, and authorities said they would form a task force to tackle the problem. As police investigated, another disturbing trend emerged. Preliminary data suggests that the vast majority of suspected perpetrators in the latest wave of cases were teenagers.

“Some may dismiss it as a mere prank, but it is clearly a criminal act that exploits technology under the cloak of anonymity,” Yoon said during a cabinet meeting late last month, acknowledging that many of the victims and perpetrators were minors.

Yoon is right. This is not just a joke, and the impact of this type of digital sexual violence can be devastating for victims. Much of the blame is rightly placed on Telegram, especially since the scandal comes at the same time as the platform's CEO, Pavel Durov, was arrested and charged in France for alleged complicity in crimes, including the distribution of child pornography, committed through his app. Korean authorities said Telegram is cooperating with the investigation and is asking for content to be removed.

Yet Yoon's words will ring hollow to some, as he came to power in 2022 by courting young male voters with proposals to abolish the Ministry of Gender Equality, which he accused of treating men like “potential sex criminals.” He also claimed that there is no systematic gender discrimination in South Korea and suggested that feminism is to blame for the country's low birth rate. At the same time, women earn about 30% less than their male counterparts, the highest gender pay gap in the developed world. Even in two-income households, women bear the brunt of housework and childcare.

And before the recent flood of AI tools made it much easier to create explicit deepfake images, advocacy groups had already drawn attention to a spate of digital sex crimes that typically involved intimate images without consent or hidden cameras.

There are a number of reasons why South Korean women, like women in other developed countries, are choosing not to have children, including rising labor force participation and the unequal burden of child-rearing. Rather than blaming feminism, perhaps it would be more useful to look at this mountain of deeply disturbing data. Ironically, one of the main Telegram groups sharing these images reportedly had around 227,000 members—roughly the number of babies born last year.

South Korean women are bravely bringing this issue to light with protests and activism. This is the first step in bringing about change in a crisis that is affecting people all over the world. Compared to other countries, the country is also leading the way in regulating deepfake porn. In fact, there are laws that provide penalties of up to five years in prison and fines for people convicted of creating images with the intent to distribute them. In the US, federal legislation has gained support from lawmakers of both parties but is still making its way through Congress at a snail's pace.

Regulation is important, but the cases in Korea also show how difficult enforcement can be on such a widespread problem, and how easy it is to create and share such content in the first place. Expanding laws that make it illegal to possess such material could help. But victims also continue to bear a huge responsibility to digitally trace who created the content that turned their lives upside down.

The recent arrest of the Telegram CEO shows that there is a growing push around the world to hold tech companies accountable for their role in the harms that occur on their platforms. This is a step in the right direction; as more regulators shift responsibility to these powerful companies, more industry-led solutions would be welcome. Universities and research centers are developing some promising tools to protect images from AI manipulation. But not enough resources are being devoted to ensuring responsible use of AI from the industries that develop it.

Technology companies must be held more accountable to find proactive solutions. Many tech giants have cut ethical and responsible AI teams in recent years as part of comprehensive cost-cutting measures. But the dignity of women and girls should not be sacrificed for the sake of profit.

South Korea is currently the epicenter of this crisis. But it's a global problem that can affect everyone, from celebrities like Taylor Swift to middle school girls from Seoul to New Jersey. The tech sector can't shirk its responsibility much longer.

(This story has not been edited by NDTV staff and is auto-generated from a syndicated feed.)