Illness took her voice away, but AI created a replica that she carries in her phone | Western

PROVIDENCE — The voice Alexis “Lexi” Bogan had before last summer was exuberant.

She loved belting out Taylor Swift and Zach Bryan ballads in the car. She laughed all the time. In high school she was a soprano in the choir.

Then that voice disappeared.

Doctors removed a life-threatening tumor at the back of her brain in August. When the breathing tube came out a month later, Bogan had trouble swallowing and struggled to say “hello” to her parents. Months of rehabilitation helped her recovery, but her speech is still impaired.

In April, the 21-year-old got her old voice back. Not the real one, but a voice clone generated by artificial intelligence that she can summon from a phone app. Trained on a 15-second time capsule of her teenage voice – taken from a cooking demonstration video she recorded for a high school project – her synthetic but remarkably realistic-sounding AI voice can now say almost anything she wants.

Experts have warned that rapidly improving AI voice cloning technology could amplify phone fraud, disrupt democratic elections and violate the dignity of people – living or dead – who have never consented to having their voices simulated to say things that they never spoke to.

It has been used to produce deepfake robocalls for New Hampshire voters impersonating President Joe Biden. In Maryland, authorities recently accused a high school athletic director of using AI to generate a fake audio clip of the school’s principal making racist comments.

But Bogan and a team of doctors from the Lifespan hospital group in Rhode Island believe they have found an application that justifies the risks. She is one of the first people and the first with her condition to work with ChatGPT maker OpenAI to replicate a lost voice.

“We hope that Lexi will be a pioneer as the technology continues to develop,” says Dr. Rohaid Ali, a neurosurgery resident at Brown University Medical School and Rhode Island Hospital. Millions of people with debilitating strokes, throat cancer or neurogenerative diseases could benefit, he said.

Bogan had to go back a few years to find a suitable recording of her voice to ‘train’ the AI ​​system on the way she spoke. It was a video in which she explained how to make a pasta salad.

Her doctors deliberately fed the AI ​​system only a 15-second clip. Cooking sounds make other parts of the video imperfect. It was also everything OpenAI needed: an improvement over previous technology that required much longer samples.

Getting something useful out of 15 seconds could be crucial for future patients who no longer have a trace of their voice on the Internet. A short voicemail left for a family member may be sufficient.

When they tested it for the first time, everyone was amazed by the quality of Bogan’s voice clone. “Every time I hear her voice I get so emotional,” her mother, Pamela Bogan, said through tears.

Bogan types a few words or sentences into her phone and her custom app immediately reads them out loud.

She now uses her AI voice about 40 times a day, sending feedback that she hopes will help future patients. One of her first experiments was to talk to the children at the kindergarten where she works as a teaching assistant.

She uses it in stores to ask where to find items. It helped her reconnect with her father, who has hearing loss and had difficulty understanding her. And it has made it easier for her to order fast food.

“Hello, can I please have a grande espresso with brown sugar and oat milk,” Bogan’s AI voice said as she held the phone out the window of her car at a Starbucks drive-thru.

“I love that I can have that sound again,” she said. It helps boost her self-confidence and restore a part of her identity that she thought she would lose forever.

Bogan’s doctors have begun cloning the voices of other willing Rhode Island patients and hope to bring the technology to hospitals around the world. OpenAI said it is proceeding cautiously as it expands use of the tool it calls Voice Engine, which is not yet publicly available.

Other companies with commercially available voice generation services say they prohibit impersonation or abuse, but vary in how they enforce their terms of use.

“We want to ensure that everyone whose voice is used in the service provides ongoing consent,” said Jeff Harris, OpenAI’s head of product. “We want to make sure it is not used in political contexts.”

Harris said OpenAI’s next step is to develop a secure “voice authentication” tool so users can replicate only their own voice, with a possible exception for trusted medical providers working with a patient.

While she’ll have to fiddle with her phone to get the voting machine to talk for now, Bogan envisions an AI voting machine that improves on older voice-repair solutions by merging with the human body or repeating words in real time. translate.

She is less sure of what will happen as she gets older and her AI voice continues to sound like it did as a teenager. Perhaps the technology can “aging out” her AI voice, she said.

For now, “even though I don’t have my voice completely back, I have something to help me find my voice again,” she said.