Audio and Video

Video: Niantic opens AR platform to global designers

10 November 2021

Augmented reality (AR) startup Niantic has opened up its first set of AR tools, allowing developers to bring mixed reality experiences to the real world.

Called the Niantic Lightship Augmented Reality Developer Kit (ARDK), the platform is the foundation for Niantic’s products built on the experience of developing and running AR titles from “Ingress” to “Pokémon GO” and “Pikmin Bloom.”

ARDK is a cross-platform software kit that runs on Android and iOS devices featuring tools and technologies such as real-time mapping, understanding and sharing.


Using a smartphone camera sensor helped to build depth understanding for developers and meshing APIs allowed them to create real-time 3D mesh maps, understanding the topography and surfaces around the user. A mesh persists through the app session and can be expanded or improved depending on user movements.

Virtual objects appear in the right depth to real-world objects, meaning if a user kicks a virtual ball over a hedge, the hedge will remain in sight, but the ball disappears. Or if a user kicks a virtual ball against a tree, it bounces off and travels in a way a real ball would.

The depth technology adapts to whatever smartphone camera is being used from a standard RGB sensor to lidar-powered devices to create 3D mesh maps that allow developers to create more visually realistic layers.

Object interaction

Niantic’s Semantic Segmentation APIs allow virtual objects to correctly interact with different surfaces in the real world and appear, or not appear, where they realistically would. Based on computer vision, the ARDK platform instantly identifies different elements in a real-world environment such as the sky, ground, water, buildings and more.

Semantic Segmentation for developers gives tools to bring realistic creations to new games or interactive mixed reality sports.


In the platform, the ARDK’s multiplayer APIs enable developers to create AR sessions supporting up to five players concurrently. The system allows more interactive and collaborative applications that keeps players in sync in real time with all the virtual content shared among the players.

ARDK is backed by a peer-to-peer networking stack and managed server, so developers don’t need to write code or support multiplayer experiences. ARDK also provides lightweight multiplayer gaming functions such as a lobby system, a synchronized clock and session-persistent storage.

Current AR projects

Niantic showcased various AR projects that are under development based on the Lightship ARDK platform, which will be available in 2022, including:

  • Historic Royal Palaces — AR that transformed the Tower of London into a new landscape filled with flowers.
  • Shueisha — Mixed reality that makes the characters from the Japanese company come to life.
  • PGA of America — An AR app that helps create shared-mixed reality experiences in golf to entice new generations of players to the sport.
  • Coachella — The annual concert event’s Spectra tower, the seven-story rainbow walkway tower, will show mixed reality butterflies behaving in the real world through semantic segmentation.
  • Tripp — This app will bring mediation to AR.
  • Science Museum Group — The group will connect people to science all around museums through an AR app.
To contact the author of this article, email

Powered by CR4, the Engineering Community

Discussion – 0 comments

By posting a comment you confirm that you have read and accept our Posting Rules and Terms of Use.
Engineering Newsletter Signup
Get the Engineering360
Stay up to date on:
Features the top stories, latest news, charts, insights and more on the end-to-end electronics value chain.
Weekly Newsletter
Get news, research, and analysis
on the Electronics industry in your
inbox every week - for FREE
Sign up for our FREE eNewsletter
Find Free Electronics Datasheets