OpenAI has a voice cloning tool, there is fear it will affect elections

(Patria) - OpenAI on Friday unveiled a voice cloning tool that it plans to keep under strict control
until safeguards are established to prevent fake audio intended to deceive
listeners, AFP reported.

The model called "Voice Engine" can essentially duplicate someone's speech based on a fifteen-second audio sample, according to a blog post by OpenAI sharing results of a small test of the tool.

"We are aware that generating speech that resembles people's voices carries serious risks, which are especially at the forefront in an election year," said the San Francisco-based company.

"We are working with US and international partners from across governments, media, entertainment, education, civil society and beyond to ensure we incorporate their feedback as we build," they added.

Disinformation researchers fear the misuse of artificial intelligence-powered applications in a key election year thanks to growing voice cloning tools, which are cheap, easy to use and difficult to trace. Acknowledging these issues, OpenAI said it is "taking a cautious and informed approach to a broader release due to the potential for misuse of synthetic voice."

The cautious disclosure came several months after a political consultant working for a long-shot Democratic presidential rival to Joe Biden admitted to being behind a robocall that impersonated the US leader.

The AI-generated call, the brainchild of an operative for Minnesota Congressman Dean Phillips, featured what sounded like Biden's voice urging people not to vote in the January New Hampshire primary. The incident sparked alarm among experts who fear a flood of AI-powered fake disinformation in the race for the White House in 2024, as well as in other key elections around the world this year.

OpenAI said that partners testing Voice Engine agreed to rules including a requirement for explicit and informed consent from each person whose voice is duplicated using the tool. Also, audiences must be made clear that the voices they hear are generated by artificial intelligence, the company added.

"We have implemented a set of safety measures, including watermarking to track the origin of any audio generated by Voice Engine, as well as proactive monitoring of how it is being used," OpenAI said.