Apple does it again: Here are the awesome features of Apple Intelligence

The Apple conference was boring until Apple started talking about the Apple Intelligence system. Amazing features and an amazing step towards a different future. Apple Intelligence is a personal intelligence system for iPhone, iPad and Mac devices that combines the power of generative models and integrates them with personal context to provide intelligence characterized by an amazing level of... Benefit and convenience.

Apple Intelligence will be deeply integrated into iOS 18, iPadOS 18, and macOS Sequoia, and harnesses the power of Apple silicon to understand and create language and images, take actions across apps, and extract data from personal information, to speed up and simplify everyday tasks. All of this while setting a new standard for privacy in the world of artificial intelligence by introducing a new type of server called Private Cloud Compute, with the ability to provide the required computing power between on-device processing and larger models based on dedicated servers running Apple chips.

New abilities to understand and create language

Apple Intelligence launches new ways for users to write and communicate more effectively. Users are waiting for the all-new Writing Tools feature across the entire system, so they can rewrite, proofread, and summarize texts wherever they want, including in Mail, Notes, and Pages applications, as well as external applications.

Whatever writing tasks range from organizing class notes, proofreading a blog post, or making sure an email is crafted perfectly, users will feel more confident in their writing with Writing Tools. With the Rewrite tool, Apple Intelligence will allow users to choose from different versions of what they have written, and adjust the style to reach what suits the intended audience and the task at hand. The Rewrite tool also makes it possible to present the right article in the right place, help refine a cover letter, and add humor and creativity to a party invitation. The Proofread tool verifies that grammar rules are followed, appropriate words are chosen, and sentences are structured correctly, while at the same time providing suggestions for modifications, accompanied by an explanation of these modifications so that the user can quickly review or accept them. The Summarize tool enables users to select text and summarize it into an easy-to-read paragraph, bulleted form for key points, a table, or a list.

In the Mail app, you'll never miss emails. Priority Messages introduces a new section at the top of your inbox where your most urgent emails, like a same-day dinner invitation or a plane boarding pass, are displayed.

Through a user's inbox, instead of viewing the first lines of each email, users can view summaries without having to open the message. As for long threads, users can view relevant details with just a click. The Smart Reply feature also provides suggestions for quick replies, and identifies questions in the email to ensure all of them are answered.

The deep understanding of language extends to notifications. Priority Notifications appear at the top of the notification stack to highlight the most important ones, and Summaries helps users quickly sift through long or cluttered notifications, showing key details right on the lock screen, like what happens in active group chats.

As for the new focus feature (Reduce Interruptions), it helps users focus while completing tasks, by highlighting only notifications that require immediate attention, such as text messages related to receiving children from daycare.

Users can now record, transcribe, and summarize audio clips in Notes and Phone apps. When you start recording a callParticipants are automatically notified, and when it's over, Apple Intelligence creates a summary to help remember the most important points.

Finally, record calls on iPhone

Image Playground feature

Apple Intelligence supports exciting image creation capabilities that help users communicate and express themselves in new ways. With Image Playground, users can create fun images in seconds, choosing from three styles: Animation, Illustration, or Sketch.

The Image Playground feature is very easy to use as it is built into applications including messaging, and is also available in a dedicated application, and is ideal for testing different concepts and styles. All images are created on-device, giving users the freedom to test this feature with as many images as they like.

Image Playground allows users to choose from a range of concepts from various categories such as themes, outfits, accessories, and locations, write a description to define the image, choose a person from their own image library to include in the image, and choose their favorite style.

The Image Playground experience in Messages gives users the ability to quickly create fun photos for their friends, and even see suggested character concepts relevant to their conversations. For example, if a user sends a message to a group about going hiking, they will be shown suggested concepts related to friends, destination, and activity, making image creation faster and more relevant.

Notes users can access the Image Playground feature via the new Image Wand widget in the Apple Pencil tool palette, adding more visual appeal to notes. Priority diagrams can also be turned into colorful images, and users can select an empty space to create an image using context from the surrounding area. Image Playground is also available in apps like Keynote, Free Space, and Pages, as well as third-party apps that support the new Image Playground API.

Creating Genmoji codes

Emoji will get a whole new upgrade, as users can create original Genmoji to express themselves. By simply writing a description that shows the Genmoji along with additional options, users can also create Genmojis for their friends and family using their photos. Genmoji codes can be added to messages or shared as a sticker or interactive response code, just like emoji codes.

New features in pictures

Searching for photos and videos just got easier with Apple Intelligence, so natural language can be used to search for specific photos, like “Maya skating in a tie-dye shirt” or “Katy with stickers on her face.” As for searching in videos, it has also become more powerful with the ability to find specific moments in clips so that users can reach directly to the part in question. What's more, the new Clean Up tool can identify and remove distracting elements in the background of an image, without accidentally replacing the element in the image.

Memories allows users to create the story they want to see simply by writing a description. By understanding the language and image, Apple Intelligence will be able to select the best photos and videos based on the description, creating a story composed of chapters based on themes inspired by the images. He arranges it to present a film with its own narrative curve. As with all Apple Intelligence features, photos and videos remain private on the device and are not shared with Apple or anyone else.

Siri is entering a new era

With richer language understanding capabilities, Siri becomes more natural, offering more personalized and contextual suggestions, as well as the ability to simplify and speed up everyday tasks. It can pick up if a speaker stutters, and it can maintain context from one request to the next. In addition, users can write to Siri, and switch between text and voice when communicating with him according to what they prefer at that moment. In this context, Siri has a completely new design with an elegant glowing light that wraps around the edge of the screen when Siri is active.

Siri can now provide all-round device support to users, answering thousands of questions about getting things done. Users can learn everything from how to schedule emails in the Mail app, to how to switch from light mode to dark mode.

With awareness of what's happening on devices' screens, over time Siri will be able to understand and take action on users' content in more apps. For example, if a friend sends their new address in the Messages app, the recipient can say, “Add this address to your contact card.”

Thanks to Apple Intelligence, Siri will have the ability to take hundreds of new actions on and across Apple devices and third-party apps. For example, a user can say: “Pick up that article about cicadas from your reading list,” or “Send Malia photos of a barbecue.” on Saturday,” and Siri will perform this action.

Siri will have the ability to provide intelligence tailored to the user and the information on their device. For example, a user can say, “Play the podcast that Jimmy recommended,” and Siri will identify and play episodes without the user having to remember whether the recommendation was mentioned in a text or message. Email. Or to ask, “When will my mother’s plane arrive?” Siri will find the flight details and base on them to track the flight in real time and answer the arrival time.

A new standard for privacy in artificial intelligence

Apple Intelligence relies on a deep understanding of personal context while preserving privacy to deliver great benefit. On-device processing is the foundation of Apple Intelligence, and many models that support it run entirely on-device. To meet more complex operational demands that require more processing power, Private Cloud Compute extends the privacy and security of Apple devices into the cloud to deliver even more intelligence.

Private Cloud Compute allows Apple Intelligence to expose and expand its compute power and leverage larger server-based models for more complex requests. These models run on servers powered by Apple silicon, so data related to the user's task is only processed on Apple's servers, and is not stored or made accessible to Apple.

Integrating ChatGPT into Apple platforms

Apple is integrating ChatGPT into iOS 18, iPadOS 18, and macOS Sequoia experiences, giving users access to its expertise, as well as its image and document understanding capabilities, without having to navigate multiple tools.

Siri can also leverage ChatGPT's expertise when it's useful. Before sending any question to ChatGPT, users are alerted, along with any documents or photos, and Siri will then provide the answer directly.

Additionally, ChatGPT will be available in Apple's Writing Tools system-wide, helping users create content for whatever they're writing about. The creation tool also allows users to access ChatGPT's image tools to create images in a wide range of genres to complement what they write about.

There is built-in privacy protection for users accessing ChatGPT, as their IP addresses are masked, and OpenAI does not store requests. ChatGPT's data usage policies apply to users who choose to link their accounts.

ChatGPT will arrive for iOS 18, iPadOS 18, and macOS Sequoia later this year, with support from GPT-4o. Users can access the service for free without having to create an account, and ChatGPT subscribers can link their accounts and access paid features directly from these experiences.

Availability

Apple Intelligence is free to users and will be available in beta as part of iOS 18, iPadOS 18, and macOS Sequoia this fall in English (US). Additional features, software platforms, and languages ​​will be available over the next year. The Apple Intelligence system will be available on iPhone 15 Pro, iPhone 15 Pro Max, iPad, and Mac devices equipped with the M1 chip and later, with Siri and the device language required to be set to English (United States).