WWDC24 has arrived, bringing many exciting new features to Apple’s operating systems. In this article, we focus on Vision OS 2.0, the latest update to Apple’s augmented reality platform. This major update includes many features that were missing in the first version, such as camera access, object detection, and better management of windows and volumes. These improvements allow developers to create even more powerful and immersive applications.
Enterprise APIs
Enterprise APIs introduce new functionalities, granting enhanced access to device sensors and unlocking the full potential of Vision OS. With the ability to access the main camera, developers can now implement solutions that were previously impossible. This feature is a significant game-changer for Vision Pro applications, enabling image processing with CoreML models.
Additionally, Vision OS 2.0 allows the sharing of both physical and virtual content. Another notable feature is the ability to scan QR codes and barcodes, integrating their content into applications. To fully utilize these APIs, developers need specific entitlements, requiring an Apple Developer account.
TabletopKit
TabletopKit is a new framework focused on content attached to plane surfaces, primarily for building board games. It offers several features to facilitate this purpose. Developers can create a table with any shape and attach Equipment Entities, a new entity type designed to hold the table’s content. Interactions can be built around these entities to establish game rules and animations.
To integrate players into the game, seats can be defined around the table, with each seat corresponding to a player. This setup allows for specific equipment to be limited to individual players, ensuring personalized gameplay. Moreover, multiplayer functionality is easily achievable through SharePlay sessions.
Volumetric APIs
Volumes have received updates to enhance their flexibility. These new features address gaps from the first Vision OS version. One significant update is the ability to attach ornaments to volumes. This addition simplifies the process of integrating UI elements with 3D content within volumes. Unlike the previous version, where a window was needed for UI interactions, ornaments provide a more straightforward and integrated solution, moving with the user to ensure accessibility.
Other updates include volume resizability and the option to choose between fixed or dynamic scaling, allowing 3D objects to maintain constant size or change based on user proximity.
Object Tracking
The Object Tracking Provider, a highly anticipated feature, is now available. It allows for the recognition of specific objects in the user’s environment, enabling the placement of 3D content around these objects using RealityKit. To utilize this feature, developers must create reference objects by training ML models using accurate 3D models in .usdz format. These models can specify the tracking direction, enhancing accuracy. Once trained, the reference objects can be integrated into projects using Reality Composer Pro for scene creation.
ARKit updates
ARKit has also received several updates. Room Tracking enables the system to track specific rooms and place content accordingly. It ensures that anchors are only tracked within the current room, avoiding misplaced content. The system also recognizes room transitions, updating anchors as the user moves.
Hand Tracking has been enhanced with movement prediction, reducing delays in animations that depend on hand positioning. The Plane Detection Provider now recognizes slanted surfaces, in addition to vertical and horizontal ones.
Other features
Vision OS 2.0 includes several smaller yet valuable features:
- Custom Hover Effects: Developers can create custom hover effects, adding animations to make interactions more intuitive and engaging.
- Force Effects: New force effects enhance object interactions, adding another layer of interactivity.
- Spatial Photos: 2D photos can now be transformed into spatial photos.
- Window Management: Developers can choose where new windows appear before they open, resulting in a more organized and user-friendly interface.
- Multiple Video Views: Applications can now include multiple video views simultaneously, enhancing multimedia capabilities.
Conclusion
Vision OS 2.0 is a significant update that introduces powerful new tools to take immersive applications to the next level. The inclusion of previously missing features opens up many new possibilities for developers. The Vision OS 2.0 beta is available now, along with Xcode 16 beta, providing the perfect opportunity to explore and implement these exciting new features.
