Augmented reality (AR) startup Niantic has opened up its first set of AR tools, allowing developers to bring mixed reality experiences to the real world.
Called the Niantic Lightship Augmented Reality Developer Kit (ARDK), the platform is the foundation for Niantic’s products built on the experience of developing and running AR titles from “Ingress” to “Pokémon GO” and “Pikmin Bloom.”
ARDK is a cross-platform software kit that runs on Android and iOS devices featuring tools and technologies such as real-time mapping, understanding and sharing.
Using a smartphone camera sensor helped to build depth understanding for developers and meshing APIs allowed them to create real-time 3D mesh maps, understanding the topography and surfaces around the user. A mesh persists through the app session and can be expanded or improved depending on user movements.
Virtual objects appear in the right depth to real-world objects, meaning if a user kicks a virtual ball over a hedge, the hedge will remain in sight, but the ball disappears. Or if a user kicks a virtual ball against a tree, it bounces off and travels in a way a real ball would.
The depth technology adapts to whatever smartphone camera is being used from a standard RGB sensor to lidar-powered devices to create 3D mesh maps that allow developers to create more visually realistic layers.
Niantic’s Semantic Segmentation APIs allow virtual objects to correctly interact with different surfaces in the real world and appear, or not appear, where they realistically would. Based on computer vision, the ARDK platform instantly identifies different elements in a real-world environment such as the sky, ground, water, buildings and more.
Semantic Segmentation for developers gives tools to bring realistic creations to new games or interactive mixed reality sports.
In the platform, the ARDK’s multiplayer APIs enable developers to create AR sessions supporting up to five players concurrently. The system allows more interactive and collaborative applications that keeps players in sync in real time with all the virtual content shared among the players.
ARDK is backed by a peer-to-peer networking stack and managed server, so developers don’t need to write code or support multiplayer experiences. ARDK also provides lightweight multiplayer gaming functions such as a lobby system, a synchronized clock and session-persistent storage.
Current AR projects
Niantic showcased various AR projects that are under development based on the Lightship ARDK platform, which will be available in 2022, including:
- Historic Royal Palaces — AR that transformed the Tower of London into a new landscape filled with flowers.
- Shueisha — Mixed reality that makes the characters from the Japanese company come to life.
- PGA of America — An AR app that helps create shared-mixed reality experiences in golf to entice new generations of players to the sport.
- Coachella — The annual concert event’s Spectra tower, the seven-story rainbow walkway tower, will show mixed reality butterflies behaving in the real world through semantic segmentation.
- Tripp — This app will bring mediation to AR.
- Science Museum Group — The group will connect people to science all around museums through an AR app.