Connect with us

Technology & Innovation

Hour One’s building a “character economy”

Diane Davis

Published

on

A woman named Hepburn is on my screen speaking my words, warmly, professionally, smiling in a sharp blue blazer, like a TV reporter or one of those improbably clear-skinned creators who make their living on YouTube. Then there’s a small clue: she mispronounces COVID like Ovid. The truth is, Hepburn lives in the cloud, and had been summoned just a few minutes earlier by Oren Aharon, the CEO and cofounder of Hour One, a Tel Aviv-based startup that builds human-seeming avatars, each capable of speaking some 20 languages. Watching her, I can’t suppress a small laugh.

Deepfakes, digital puppets, metahumans; Hour One calls these fake people “reals,” and without irony, because they are based on actual humans. Dozens of these talking heads are now doing tutorials, customer service, client presentations, interoffice communications, and silly videos on Cameo. Imagine a company-wide meeting, says Aharon: Why should the CEO record a presentation on video if his avatar can just do it? “You get something amazing in two minutes, you can send it to everyone, and nobody needs to waste their time,” he adds.

[Screenshot: Hour One]

“And of course, it’s not replacing any personal connection that they can also do,” cofounder and CTO Lior Hakim is quick to add. “But [real-world interactions] are not scalable, just by the fact that they’re bound by time and the physical aspect of, well, us.”

In 2017, the two entrepreneurs first saw an AI-generated Obama, and witnessed the start of a new era in which video gets automated by code. That was also the year that the work of anonymous Reddit user named Deepfakes quickly came to represent all AI-powered threats to democracy. Nonconsensual porn was how it started, but synthetic media has since been used to falsely implicate enemies and steal millions of dollars. This year, voice clones for Anthony Boirdain and Val Kilmer sparked fresh ethical and economic concerns, particularly among voice actors worried about their jobs. Earlier this year, one actor sued TikTok for using her voice in its text-to-speech feature without compensation or consent.

Hour One is trying to put a happier face on all that, literally, by focusing on business uses and prioritzing the people behind the faces. And now that pandemic precautions have upended the workplace and the way we think about it—and companies devote billions to blockchains and “the metaverse”—the startup is riding a growing wave of interest in synthetic media. “The idea of, ‘You can’t do it from a studio anymore, so let’s try new technology’—that was really the perfect storm for us,” Aharon says.

The company of 15 raised $5 million in funding last year, and says it’s racked up dozens of customers. Berlitz is now using Hour One to “scale” some of its language teachers, and AliceReceptionist has tapped the company to welcome visitors to lobbies in English, Spanish, Arabic, and French Canadian. A German TV network hired its avatars to report soccer scores, and Cameo and DreamWorks recently worked with Hour One and the voice startup Lovo to debut its first “deepfake”: For 20 bucks, you can now get a jokey, semi-personalized greeting from the Boss Baby, the animated character voiced by Alec Baldwin. Cameo hints that more human celebrities may be coming.

Hepburn and most of Hour One’s characters are based on a diverse group of about 100 actual people, many from around Tel Aviv, who receive micropayments each time their likeness is used. Capturing a face using a high-resolution camera now takes only about half an hour. (Recording voices, as some opt to do, is a more time-consuming process).

To get Hepburn to read my story, Aharon opens a dashboard, pasted in an article I wrote, uploaded a few images, selected her from a gallery of talking heads, and added a voice. (The article was about, of all things, disinformation.) While the video is processing, he scrolls through other options: We can change backgrounds, select new camera movements, switch colors, text, or imagery, even get Hepburn to read the article in Mandarin. We could spin up hundreds of her at once.

[Screenshot: Hour One]

You can see where this is going, or hear it: Synthetic voices are already spreading in the wilds of TikTok and YouTube, thanks to text-to-speech software that lets creators “narrate” videos without using their own voice. Hour One extends that capability to the entire video, including the visuals.

“Not everyone is a YouTuber or a podcaster, and necessarily wants to spend all day recording themselves just to reach his audience,” says Hakim. “But [they do] want to create those personal connections.”

Seeing photo-realistic human faces “creates some kind of psychological effect where you basically feel the connection because you know this person is out there, this is a true person,” he says. And seeing a human “also creates a situation where people are saying, ‘Well, I could become a character, and there is a way to do it.’”

Aharon imagines this opening the door to what he calls “the character economy.” Eventually we could all become Reals.

“This is an asset that every one of us has, the likeness of us, the voice and the face, an asset that people will be able to use and scale, once digitized,” says Aharon. “There are what, 600 million people on LinkedIn? All of these people are potentially characters, potentially presenters.”

Advances in processors and GANs, or generative algorithmic networks, are cutting the time it takes to capture and render a talking head. They’ve also made it possible to render these faces instantaneously, and even make them interactive. In tests it’s done with the AI-writing system GPT-3, Hakim says that with only a few keyword prompts “the machine basically creates the entire scene.”

Generating substitute teachers

My Spanish instructor was realistic enough, at least enough to get to me to focus on my pronunciation, rather than on her computer-generated articulation. For Berlitz, Hour One made 13,000 videos of these deepfaked teachers, speaking English, Spanish, and German, in about 15 hours. The language-training company still offers online classes with human instructors, but digital teachers mean it can drastically lower production costs (and perhaps subscription costs), while still providing what its CEO Curt Uehlein called in a statement “a very human-centric experience.”

Still, it’s not clear if fake humans, even very realistic ones, can do the heavy lifting of an actual human teacher: Research has shown that even real humans communicating through a screen don’t activate the same parts of the brain associated with live social contexts. And it seems hard to spin a “human-centric experience” as good news for actual human teachers. Still, Hakim insists: “We are not looking to replace the jobs of workers. We are actually putting creative tools in the hands of people in the world of work, so they can actually focus on the creative process.”

The founders say they are mindful of other perils, too. While Hour One doesn’t let people dictate how and where their likenesses will be used, its ethics policy and agreements with customers and talent forbid any uses for what Aharon calls “extreme” things: adult entertainment, profanity, politics, “inappropriate ads,” self-harm, or “any opinion that may incite controversy.” For “known” personalities like XPrize founder Peter Diamandis, or the YouTube star Taryn Southern, both of whom have been scanned, any use must be personally approved.

The company is also labeling its videos to avoid deceiving audiences. My Spanish teacher had a small watermark in the corner of her video—the letters “AV”—but I saw no obvious explanation that this meant “altered visual.” A proposed bill in the U.S. Congress would mandate such watermarks, though experts say more protections are needed.

There’s a tension here: Even as it labels its characters as fakes, Hour One is also trying to make them realistic enough to trick us into feeling a human connection. Amir Konigsberg, a tech entrepeneur who sits on Hour One’s board, suggests this tension could be what makes its avatars so compelling: not because they’re human but because they’re obviously fake—but look so human. “The fact that it’s very realistic, very high quality, but also marked as clearly synthetic, that’s why it’s engaging,” he says.

This may be why I couldn’t help laughing a little at Hepburn and what she portends. We’ve crawled out of the uncanny valley into another strange place, where our brains are increasingly stuck between what we know and what we see. It’s nervous laughter.


Technology & Innovation

Google DeepMind’s new generative model makes Super Mario-like games from scratch

Diane Davis

Published

on

example of game generated from a crayon sketch

“It’s cool work,” says Matthew Gudzial, an AI researcher at the University of Alberta, who developed a similar game generator a few years ago. 

Genie was trained on 30,000 hours of video of hundreds of 2D platform games taken from the internet. Others have taken that approach before, says Gudzial. His own game generator learned from videos to create abstract platformers. Nivida used video data to train a model called GameGAN, which could produce clones of games like Pac-Man.

But all of these examples trained the model with input actions, button presses on a games controller, as well as video footage: a video frame showing Mario jumping was paired with the “jump” action, and so on. Tagging video footage with input actions takes a lot of work, however. This has limited the amount of training data available. 

In contrast, Genie was trained on video footage alone. It then learned which of eight possible actions would cause the game character in a video to change its position. This turned countless hours of existing online video into potential training data. 

Genie can generate simple games from hand-drawn sketches

GOOGLE DEEPMIND

Genie generates each new frame of the game on the fly depending on the action the player takes. Press jump and Genie updates the current image to show the game character jumping; press left and the image changes to show the character moved to the left. The game ticks along action by action, each new frame generated from scratch as the player plays. 

Future versions of Genie could run faster. “There is no fundamental limitation that prevents us from reaching 30 frames per second,” says Tim Rocktäschel, a research scientist at Google DeepMind who leads the team behind the work. “Genie uses many of the same technologies as contemporary large language models, where there has been significant progress in improving inference speed.” 

Genie learned some common visual quirks found in platformers. Many games of this type use parallax, where the foreground moves sideways faster than the background. Genie often adds this effect to the games it generates.  

While Genie is an in-house research project and won’t be released, Gudzial notes that the Google DeepMind team says it could one day be turned into a game-making tool—something he’s working on too. “I’m definitely interested to see what they build,” he says.

Continue Reading

Technology & Innovation

Tackling long-haul diseases | MIT Technology Review

Diane Davis

Published

on

""

Tal, who has been obsessed with infectious disease since losing an uncle to HIV/AIDS and a cousin to meningococcal meningitis, wondered what this striking diversity could reveal about our immune response to infection. According to one hypothesis, the wide array of these receptors is the result of an evolutionary arms race between disease-causing microbes and the immune system. Think of the receptor as a lock, and the “Nothing to see here” message as a key. Pathogens might evolve to produce their own chemical mimics of this key, effectively hiding from the immune system in plain sight. In response, the human population has developed a wide range of locks to frustrate any given impostor key. 

Wanting to test this hypothesis, Tal found herself walking the halls of Stanford, asking colleagues, “Who’s got a cool bug?” Someone gave her Borrelia burgdorferi, the bacterium that causes Lyme disease. Previous research from Tal’s collaborator Jenifer Coburn, a microbiologist now at the Medical College of Wisconsin, had established that Lyme bacteria sport a special protein crucial for establishing a lasting infection. Knock this protein out, and the immune system swiftly overwhelms the bugs. The big question, however, was what made this protein so essential. So Tal used what’s known as a high-affinity probe as bait—and caught the Borrelia’s mimic of our “Don’t eat me” signal binding to it. In other words, she confirmed that the bacteria’s sneakyprotein was, as predicted, a close match for a healthy cell’s signal.  

Sex differences in Lyme infection

Until then, Tal says, she had never given Lyme disease much thought. But the more she learned, the more disturbed she grew. Even after timely antibiotic treatment, roughly 10% of all Lyme patients go on to develop chronic symptoms that can include crushing pain, debilitating fatigue, and cognitive changes that make basic tasks a struggle.  

This confocal micrograph depicts Borrelia burgdorferi bacteria, which cause Lyme disease when transmitted to humans by ticks. These Borrelia were genetically engineered to produce a green fluorescent protein.

COURTESY OF THE TAL RESEARCH GROUP

Perhaps even more alarming than the disease has been the medical community’s response to it. “I realized that there’s this public health debacle around Lyme, and it’s, for lack of a better word, obscene,” Tal says. Chronic Lyme patients skew female, and for decades, clinicians have dismissed their symptoms as signs of mental illness. The medical establishment has “done nothing but call them crazy,” Tal says, “instead of admitting that they just don’t understand what’s going on.” 

Today, there is no objective way to diagnose chronic Lyme, and no medically accepted therapy. For some patients, lengthy treatments with high doses of antibiotics can ease symptoms, but these come with their own serious risks. (They can, for example, damage the microbiome, leading to significant negative effects on health.) And because the antibiotic used currently only prevents bacteria from replicating, Tal notes, it’s up to the immune system to actually kill off the invaders. If immune cells can’t tell friend from foe, the utility of antibiotics may be limited. 

Chronic Lyme patients skew female, and for decades, the medical establishment has “done nothing but call them crazy,” Tal says, “instead of admitting that they just don’t understand what’s going on.”

For Tal, these revelations were electrifying. She dove into the immunology of Lyme disease, focusing in particular on sex differences. In one mouse experiment, she discovered that Lyme bacteria “completely disfigured” the uterus. Yet after delving through decades of Lyme research, she could find only one other study that even documented uterine infection. 

This shortfall mirrors larger problems in medical research. “We’ve let men dictate the direction of research funding for so long,” Tal says. Traditionally, studies focused on male subjects, and a 1977 FDA policy barred women from participating in most clinical trials in the US in the wake of birth defects caused by thalidomide. It wasn’t until 1993 that federal law required studies to include women and minorities. This, coupled with other sex- and gender-based medical biases, means that many female-dominated diseases remain under-researched. “So much of this research is being done on males, male mice—male, male, male,” Tal says. “And I’m like, no.” 

Tal suspects that the sex disparities seen in chronic Lyme and other pathogen-­triggered chronic diseases might come down to the fact that men mount a more robust response to acute infection. This no-holds-barred approach is risky—“Your immune system has the power to kill you,” she notes—but it may mean that men, on average, can kill off more viruses or bacteria in the critical first week of infection. After that window closes, the immune system largely settles back down, Tal says. Pathogens that escaped the initial blitz could take up long-term residence in the body, potentially causing persistent symptoms. And women have a higher chance of chronic illness.

Continue Reading

Technology & Innovation

We are beavers all | MIT Technology Review

Diane Davis

Published

on

old black and white photo of William Miller in a canoe

As efficient creators and stewards of wetlands, beavers provide a hospitable ecosystem for dozens of other creatures, from insects, frogs, and turtles to owls, otters, great blue herons, and even moose and deer. What’s more, by harvesting undergrowth for their dams and creating ponds and bogs that raise the moisture content of the soil, beavers lessen the likelihood that forest fires will spread. As forest fires devastated Oregon in 2021, beaver wetlands remained green and lush, acting as natural firebreaks. On aerial images of the charred landscape, the beaver’s habitat stands out, a wide and verdant ribbon running through the blackened trees.

While not all property owners who live near beaver habitats appreciate the animals’ tree removal services, the pro-­beaver movement seems to be getting more organized. In November 2023, some 300 beaver restoration advocates from North America and Europe gathered in the Beaver State (Oregon) for the annual State of the Beaver conference. “Seventy-five percent of the artificial wetland restoration projects done in America over the past 30 years have failed,” conference cofounder Stanley Petrowski told the Daily Yonder. “But when beavers do it, they do it perfectly.” 

BeaverCon, held near Baltimore in June of 2022, and the Midwest Beaver Summit, held in Chicago and online in September 2023, attracted similar crowds of humans interested in promoting beaver welfare.

It is, in fact, possible to find ways to allow beavers to continue creating their watery habitats in ways that minimize damage to human infrastructure. For example, devices such as the Beaver Deceiver can be installed to prevent beavers from damming culverts, which often leads to flooding of roads. Skip Lisle, founder of Beaver Deceivers International of Grafton, Vermont, first developed the device in the 1990s to beaver-proof the Penobscot Nation’s 130 miles of roads in Maine. “In all likelihood, they are the first large landowner to completely beaver-proof their property nonlethally,” he says.

Living organic chemical factory

At the base of their tail, all beavers have two castor sacs that store castoreum, a complex, granular substance with a strong and long-lasting musky smell. It is made up of at least 24 different compounds, primarily derived from the barks of the various trees in the beaver’s diet. Beavers deposit castoreum atop foot-high mounds of mud, sticks, and grass to mark the edges of their territory. 

Humans have long valued castoreum. About 400 BCE, Hippocrates, a chronicler of natural cures, wrote of its wonderful medical properties. Around 77 CE, the Roman naturalist Pliny listed castoreum as a cure for headaches, constipation, and epilepsy. In the Middle Ages the list of maladies castoreum was said to cure expanded to include dysentery, worms, fleas, pleurisy, gout, rheumatism, insomnia, hysteria, memory loss, and liver problems. 

Author William Miller ’51, SM ’52, reports that his foot once crashed through a beaver dam while he was dragging his canoe over it to get to the next lake in Jasper National Park in Alberta, Canada. About 100 feet away, a watching beaver immediately began to slap its tail on the pond surface. Having just unleashed a string of curses directed at the beavers, Bill assumed that the beaver was cursing at him. But he now suspects it was sending a warning signal to the other beavers—or possibly urging them to come quickly to repair the damage caused by the trespassing human oaf.

COURTESY OF WILLIAM MILLER

As it turns out, quite a few of the tree barks that beavers prefer contain compounds with known medicinal benefits. Phenols, for example, are often anti-­inflammatory and antiseptic and can have antiviral properties. They include salicylic acid (a precursor to aspirin), which can be found in the bark of willow, poplar, and alder trees—all beaver favorites. The beaver’s system functions as a natural pharmacy, extracting these compounds (among others) and secreting them in the form of castoreum. 

Humans have also used castoreum for several nonmedical applications, such as in high-end “leather note” perfumes including Shalimar, Givenchy III, and Chanel’s Antaeus. It is an ingredient in some bourbons and vodkas and has been used in Sweden to flavor “Bäverhojt” (literally, beaver shout) schnapps.

Today, most castoreum is harvested in a sterile environment by anesthetizing beavers and expressing the castor sacs near their tails. As a food additive, castoreum extract is “generally recognized as safe,” according to the FDA. But at close to $100 per pound, it’s used sparingly. The total annual US consumption of dried castoreum is around 300 pounds.

Continue Reading

Trending