Every new iOS 18 AI feature for the new Siri has just been leaked

We heard a lot of rumors during the 2024 Worldwide Developers Conference (WWDC) about Apple’s alleged plans to give its digital assistant Siri a superpower. Today through Apple Insiderwe have the most extensive leak yet. The leak, from “people familiar with Apple’s AI initiative,” details virtually everything Siri can do in more than a dozen initial iPhone apps.

The full leak is worth checking out, especially if you’re an iPhone user. However, we’ll give you the general gist of Apple’s goals for the “new” Siri and share some highlights that we think will most impact the daily lives of iPhone users.

What Apple wants from the ‘new’ Siri

The overall goal of Siri seems to be to make it more powerful and better at understanding voice commands given in natural language. According to the leak, Apple trained Siri to do this by having Apple engineers give commands that were purposefully obtuse. For example, instead of asking something like, “Hey Siri, show me pictures of my cat,” it tests vaguer commands, such as, “I want to create a blog” or, “I’m feeling nostalgic right now.” These are not specific commands that instruct Siri to do one particular thing, but instead commands will require Siri to first interpret and then decide how best to deliver what he thinks the user would want/need.

The benefit of this is obvious: train Siri to be better for users who don’t know (or don’t want to use) the proper syntax needed to execute a command. For example, a user who says, “Hey Siri, I want some coffee,” may or may not turn on the smart coffee machine, while saying, “Hey Siri, turn on the coffee machine,” probably will. The former is a natural explanation, while the latter is a direct command. Apple wants this embedded syntax to be reduced, making Siri much easier to use.

The Apple Insider However, leak does not mention how this works. For example, are these Siri features powered by “Ajax,” the codename for Apple’s internal large language model (LLM)? Or are these based on ChatGPT, as Apple is reportedly working with OpenAI for some of its AI-based systems? It could be a bit of both, but we don’t know for sure yet.

New Siri features: a list of highlights

As mentioned, the full leak is extensive and covers no fewer than 18 first-party apps for the iPhone and how Siri can work with each app. Here are a few that we think are really interesting:

• Camera: Siri will be able to control the camera via voice commands. You can turn video recording on or off, open the camera in a specific mode (photo, portrait, video, etc.) and then start a shutter timer and switch to the front or rear camera. Theoretically, this could allow you to set up your iPhone for a group photo, walk away, and then use voice commands to take the photo remotely.
• Mail: The Mail app is getting a complete overhaul. It will apparently be able to automatically classify email using machine learning, something Gmail users are probably already familiar with. Moreover, Siri can also perform detailed functions through voice commands alone. This includes things like composing an email, sending it, scheduling it, marking an email as junk, and setting a reminder to read an email at a later time. It will also be able to summarize emails and create ‘smart replies’, a feature that will no doubt be similar to Smart Reply on Android and Help Me Write in Gmail.
• Photos: Apple is likely to introduce many photo editing features based on generative AI. Pixel users will probably recognize a lot of it, because so far we’ve only heard about features you can already use on Pixels with Magic Editor and Google Photos, like moving/removing an object from the photo and filling in the blanks with generative AI, find specific photos with specific people/animals and apply generative AI filters.
• Safari: Apple’s web browser will use Siri for web page summaries, something Google has already brought to Android via Gemini. Safari can also create new tab groups or open a new private tab via voice commands.
• Voice memos: Voice memos allow you to work with Siri completely hands-free. For example, you can ask Siri to make a new voice recording and then start talking. You can then stop the recording, save it under a specific name, and then even move it to a specific folder – all without laying a finger on your iPhone.

When will these be launched?

According to serial Apple leaker Mark Gurman, at least some Siri features won’t actually land at WWDC. Apple will almost certainly announce some, but not all will be available in 2024. Gurman suggests it will be 2025 before the majority of these features will be available via a software update.

Of course, that doesn’t mean Apple won’t roll out at least a few at or around WWDC. However, it’s probably best not to expect iOS 18 to include all the features from this Siri leak, as they’re much more likely to leak in the coming months.