Amazon’s all-new Alexa voice assistant is coming soon, powered by the new Alexa LLM

Amazon’s Alexa is about to come out of its shell, and what emerges could be very interesting. At its fall hardware event on Wednesday, the company unveiled an all-new Alexa voice assistant powered by the new Alexa large language model. According to Dave Limp, Amazon’s current senior vice president of devices and services, the new Alexa can understand conversational phrases and respond appropriately, interpret context more effectively, and complete multiple requests from a single command.

Voice assistants need to change. A general lack of innovation and imperceptible improvements around understanding has turned them into essential tools rather than the exciting technological advances we hoped for when they came onto the scene over a decade ago.

Generative AI has looked like its best chance of survival for a while. But although these digital assistants have always had an AI element, they lack the complex processing capabilities and more human-like interactions that AI can generate. This is a big moment for the smart home, as it could take home automation to the next level, taking it from a remote control experience to a truly smart home.

In an interview with the edge Before the event, Limp explained that the new Alexa LLM “is a real large language model that is generalizable and highly optimized for the Alexa use case; that’s not what you find with Bard or ChatGPT or any of that stuff.”

However, the all-new Alexa won’t be unleashed everywhere, on everyone, all at once. The company is rolling it out slowly through a preview program “in the coming months” — and in the US only. It is clear that there are lessons to be learned from the mistakes made by Microsoft and Google, and that Amazon is acting cautiously.

“When you connect the MBA to the real world, you want to minimize hallucinations — and while we think we have the right systems in place… there’s no substitute for putting it out there in the real world,” Limp says. If you want to be notified when you can join the preview, tell your Echo device, “Alexa, let’s talk,” and your interest will be registered.

It’s no surprise that this Alexa superpower may not always be free. Although Alexa, as it is today, will remain free, Limp said, “the idea of ​​a super-assistant that can augment your smart home and, more than that, do complex tasks for you, could provide enough benefit that we end up charging something.” “. So on the road.

The first big change in the new Alexa will be a more conversational assistant, one that can understand more of what you say and requires less specific labels to do what you ask. This is one of the most common reasons for frustration with voice assistants — having to repeat yourself when you ask them to turn down the thermostat or when you ask them to respond, “There are some things that share the name ‘lights.’ Please choose unique names and run discovery again,” For the 900th time when all you want is to know where you left the remote.

With the new Alexa, you can say a phrase like, “Alexa, I’m cold,” and the assistant should raise the temperature of your connected thermostat. Or as Limp explained, “Say, ‘Alexa, make this room look like Seahawk colors,’ and it will know what room I’m in and what Seahawk colors are and do those translations between the APIs.”

APIs are key, Limb says. “We have converted a large number of smart home APIs, over 200, into our MBA.” This data, combined with Alexa’s knowledge of what devices in your home and which room you’re in based on which Echo speaker you’re talking to, will give Alexa the context needed to manage your smart home more proactively and seamlessly.

This contextual understanding will extend beyond knowing what other connected devices you might want to control for things like inferring when something changes in your home. “If you add a new device to your home, you can say, ‘Alexa, turn on the new light,’ and it will know what the new light is,” Limb explains. “It will demystify things, so if you put in a plug or a new smart light, it will Easier to control.”

Another new ability is to respond to multiple requests at once. These aren’t just the basic things it can already do (to some extent), like “Alexa turns off the lights and locks the door.” This is more advanced. “You can say, ‘Alexa, turn on the sprinklers, open my garage door, turn off the outside lights,’ and it will figure it all out,” Limb says.

This capability will extend to creating on-the-go routines entirely by voice – without any manual programming in the Alexa app. “I set one up this morning for my child just by saying, ‘Alexa, every morning at 8 a.m., turn on the light, play wake-up music for my child in their bedroom, and turn on the coffee maker,’” Limb says. “This can get complicated.” As obscure as you want, and instantly, it will appear in your app as a routine.”

At first, the multi-command feature will only work with a subset of device types, including lights, smart plugs and some others, Lemp says. But the team is working on adding everything.

Developers will also be able to take advantage of Alexa’s new cognitive functions and integrate their products and services into this more conversational format. Amazon is offering two tools that allow the new Alexa to control some unique features of third-party manufacturers’ products that aren’t necessarily in Amazon’s smart home ecosystem toolkit. These are called dynamic controller and action controller.

The Dynamic Controller will enable features such as preset scenes to control lighting to appear more natural. So, if you have GE Cync color light bulbs and you say, “Alexa, make it look scary here,” Alexa will know what to do without having to program a routine or import scenes into the Alexa app.

Likewise, the Actions console allows developers to add simple actions that Alexa can act on. For example, if you say, “Alexa, the floor is dirty,” the assistant will know that you want the robot vacuum to start.

Amazon says it’s already working with GE Cync, Philips, GE Appliances, iRobot, Roborock, and Xiaomi on these features, and developers can Register now to request participation. Amazon’s developer blog has more details about the new capabilities and tools.

Limp says this is just the beginning of Alexa’s new journey. “We’ve built a new LLM AI program that will – over time – power a range of Alexa domains, including a range of new smart home experiences,” he says. “The first set is to try to simplify these daily tasks.” Where she goes next will be an interesting journey to watch.

The new Alexa LLM-powered voice assistant will debut in preview in the US and will be available to anyone with an Echo device. Amazon has not announced a preview date, and the new Alexa LLM-powered smart home features will be part of an additional invitation-only preview. You can request an invitation once you’re part of the preview. Amazon says it will be available later.

Updated Thursday, September 21 at 9:20 a.m.: Added details on how developers can sign up to integrate their devices and services with Alexa LLM.