Among all the features Apple recently unveiled, the “Audio Intelligence” features on the Apple Watch Series 12 and Apple Watch Ultra 4 stand out as some of the most impressive, and perhaps simultaneously, the most suspicious and unsettling technologies. At first glance, the idea seems like a scenario from a dark science fiction film: a smartwatch on your wrist, with an ear tuned to every word spoken in your surroundings all day long to provide you with smart tools to track your conversations. This system currently includes two main tools: “Siri Recap” to summarize your day, and “Live Rewind” to catch what you missed while someone was talking to you. But does this mean Apple is actually spying on us all the time?
Live Rewind: A rewind button for speech in real life
However, for the watch to be able to retrieve the last 15 seconds at any moment, it necessarily means it is listening constantly. Apple is aware of the extreme sensitivity of this matter; therefore, it has published a detailed guide explaining the strict privacy measures it has taken to ensure that no party, including Apple itself, can access what you say. The system starts with the new S11 chip, which includes an isolated hardware section known as the “Secure Exclave.” This is completely isolated from the watchOS operating system and third-party apps and cannot be breached via software.
This chip relies on a constantly refreshing cache memory with very limited capacity, meaning old audio is automatically replaced by new audio, and no recordings accumulate at all. These 15 seconds will not be released except by a direct physical press on the Digital Crown, similar to how the side button works with Apple Pay, which prevents any malicious app from activating the feature via software.
Because the watch chip is small, the text transcription does not happen inside it. Instead, the temporary audio clip is securely transmitted via an encrypted wireless connection to the Secure Exclave in the paired iPhone. There, the voice recognition model converts it to text and immediately deletes the audio file. The feature will not work if the iPhone is not nearby. Furthermore, this feature cannot be used secretly; as soon as the Digital Crown is pressed, the watch emits an audible alert tone from its speaker, even if you are wearing headphones, and a microphone indicator and clear animation appear on the screen to alert those present.
Siri Recap: How does the watch understand your day without recording your conversations?
The second feature in the suite is “Siri Recap,” a tool aimed at summarizing the conversations you have throughout the day to create smart notes that remind you of the key points, so you can stay focused in the moment and recall what was discussed later. The fundamental difference here is that this feature does not transcribe text word-for-word, nor does it attribute statements to their speakers; it simply drafts general summaries, much like those you might jot down yourself after a meeting ends.
This feature is entirely optional and requires manual activation, whether by leaving it on all the time or enabling it for specific times and places via the Control Center. To avoid battery drain and random recording, the S11 chip employs a lightweight, local AI model on the watch whose only task is to sense the presence of human speech nearby without recording or transcribing it. If the model confirms the presence of a real conversation, the encrypted audio is passed to the paired iPhone, and the data is immediately erased from the watch once sent.
Once the encrypted audio reaches the Secure Exclave on the iPhone, it is decrypted and transcribed into raw text. It then enters a local language model that cleans it up, removing filler words, vocal tone, and useless repetition, while retaining only the core ideas. At that exact moment, the original audio is permanently deleted. A security model is then applied to block sensitive terms, and no financial data, government numbers, or authentication information is included in this condensed text.
The final step is sending the encrypted condensed text to Apple’s “Private Cloud Compute” servers, accompanied by some general context such as calendar appointments and general location classification (e.g., office, school, store) without the exact location, to generate the final summary. Because these servers are encrypted with a strict privacy system that even Apple engineers cannot access, the summary returns directly to your devices and is synced with full end-to-end encryption, and it is automatically deleted after seven days unless you choose to save it manually.
Operating requirements and launch date: Who is supported and who is not?
These features impose strict hardware and compatibility conditions, as they necessarily require an Apple Watch Series 12 or Apple Watch Ultra 4, paired with an iPhone 16 or newer models that support AI, excluding the iPhone 16e from the list. Interestingly, the iPhone 15 Pro and iPhone 15 Pro Max have been completely excluded from this feature despite their support for Apple Intelligence, which reflects audio processing and hardware isolation requirements that were only available in the latest generation of phones.
The features will not be available as soon as the watches are taken out of their boxes; Apple indicated that they will be available in a beta phase later with upcoming updates. They will initially be limited to the English language and will not reach European Union countries at this time due to regulatory requirements. Apple also prohibits their use by those under 13 years of age and requires those under 18 to obtain parental consent within family accounts.
In conclusion, Apple has managed to strike a delicate balance between features that seemed frightening at first glance and a formidable fortress of encryption and physical and software restrictions, to provide tools that enhance the user’s daily productivity without making them a commodity exposed to hacking or eavesdropping.
Source:
Leave a Reply