When Apple first launched Siri alongside the iPhone 4S in 2011, the company created a series of very convincing ads showing how you could use this newfangled voice assistant thing. In one, Zooey Deschanel asks her phone about tomato soup delivery; in another, John asks Malkovich for existential life advice. There’s also one where Martin Scorsese shuffles his schedule from the back of a taxi in New York City. They showed reminders, weather, alarms and more. The point of the ads was that Siri was a useful, constant companion, who could tackle whatever you needed. No apps or taps required. Just ask.
Siri was a big deal for Apple. At the launch event for the 4S, Apple’s Phil Schiller said Siri was the best feature of the new device. “Technologists have been teasing us for decades with the dream that you can talk to technology and it will do things for us,” he said. “But it will never come true!” All we really want to do, he said, is talk to our device any way we want and get information and help. In a moment of classic Apple bravado, Schiller exclaimed that Apple had solved it.
Apple hadn’t solved it. In the thirteen years since its initial launch, Siri has either become a way for most people to set timers, or a useless feature to be avoided at all costs. Siri has been bad for a long time, long enough that for years it seemed like Apple forgot about it or simply chose to pretend it didn’t exist.
But if the rumors and reports are true, we’re about to meet the real Siri for the first time at WWDC next week – or at least something much closer to it. According to Bloomberg, The New York Times, and others, Apple is set to unveil a massive overhaul for the Assistant, making Siri more reliable thanks to major language models, but without much new functionality. Even that would be a victory. But Apple also appears to be working on, and may be close to launching, a version of Siri that will actually be integrated into apps, meaning the assistant can take action on your device on your behalf. In theory, at least, Siri will soon be able to do everything for you.
This has clearly been the vision for Siri all along. You can even see it in those iPhone 4S commercials: these celebrities ask Siri for help, and Siri almost never finishes the job. It gives Deschanel a list of restaurants that mention delivery but don’t offer to order anything or show her the menu. It tells Scorsese there’s traffic, but doesn’t reroute him – and shouldn’t it already know he’ll be late for his meeting? Siri tells Malkovich to be nice to people and read a good book, but offers no practical help. So far, using Siri is like having a virtual assistant whose only job is to Google things for you. That is something! But it’s not much.
Siri’s inabilities are all the more frustrating because all it needs to be useful is right there on your phone. If I want pizza, why can’t Siri look at my email for the receipt from the last time I ordered, open DoorDash, enter the same order, pay with one of the cards in my Apple Wallet, and be done? When I’m having a Scorsese-level busy day, Siri seems to be right there next to all my contacts, my Slack, my email, and whatever else is needed to quickly move things on my behalf. If Siri could take over my phone as one of those remote access tools that let someone else move your computer’s cursor, it would be unstoppable.
There are actually two reasons why Siri never lived up to its potential in this way. The first is simple: the underlying technology was not good enough. If you’ve used Siri, you know how often Siri misinterprets names, misunderstands commands, and falls back on “Here are some things I found on the Internet” when all you wanted to do was play a podcast. This is where large language models are undeniably very exciting, because we’ve seen how much better speech-to-text tools like Whisper are and how much more broadly these models can understand language. They’re not perfect, but they’re a huge improvement over what we’ve had before. That’s why Amazon is also turning Alexa into LLMs and Google’s Assistant is being overrun by Gemini.
The second reason Siri never quite worked is simply that neither Apple nor third-party developers ever figured out how it works should work. How are you supposed to know what Siri can do or how to ask it? How should developers integrate Siri? Even now, if you want to add a task to your to-do list app, Siri can’t just figure out which app you’re using. You gotta say, Hey Siri, remind me to water the grass in Todoist, which is a weird sentence that makes no sense and, in my experience, fails half the time anyway. If you want to perform a multi-step action, your only option is to mess around in Shortcuts, which is a very powerful tool, but doesn’t require you to write any code. For most people it’s too much.
AI could also give Apple a chance to solve the whole problem. The researchers published a paper earlier this year about a system called Ferret-UI, which uses an AI model to understand small details of an image on the screen. The researchers even describe how a general app that uses Siri might work: OpenAI’s GPT-4 does a good job of understanding broadly what an image is, and then Ferret can understand small regions and details. In practice, this could mean that one system says: “This is the Ticketmaster app!” and the other says, “There’s the buy button.”
We should be skeptical of the claims Apple makes about Siri. More than a decade ago, Schiller stood on stage and proclaimed that Apple had built a better voice assistant, and that wasn’t the case. The same could be true now, as the hype for AI continues to move much faster than the actual technology. Humane, Rabbit, Google and others are all working on similar ideas – “agent” is the buzzword of the summer in the AI world – and none have shown that it’s ready yet.
But if Apple has cracked something here, this could be the first time we ever get to see the real Siri – the Siri we were promised all those years ago. Maybe in the next commercial, Deschanel’s tomato soup will just magically appear at her house, and the Headspace app will activate to bring Malkovich some inner peace. Maybe we’ll finally get the Siri that Apple always wanted to make.