Experience Data in Augmented Reality

How can we visualize and interact with data in augmented reality?
Three mobile experiments explore this new field.


New technologies such as the iPhone X makes it possible to interact with data heavy visualizations in augmented reality (AR). Let's explore this new ways of experience.

The three experiments are part of a self-initated side project which deals with the new technical and interactive possibilities of data with AR while focusing on two map visualizations and a scatterplot.


Visualizing the »Tour du Mont Blanc« in a garden – and explore the hiking route & mountain huts.

We mapped one of the most popular long-distance walks in Europe, the »Tour du Mont Blanc«, covering a distance of roughly 170 kilometres (110 mi) with 10 kilometres (6.2 mi) of ascent/descent and passing through parts of Switzerland, Italy and France (pink/lila circles visualize the GPS coordinates of the route in the pictures).

To plan the 11 days long trip around the Mont Blanc, the route has been mapped with a 3D terrain map on a table with markers for the mountain huts along the route (blue rectangles with a line for the precise location).
Looking at the route and the mountain massif in reality provided another feeling for the terrain and enhanced a playful exploration while moving closer and further away from the mountains with the smartphone, similar to zooming in and out of a map.

Another fun experience was to map the whole terrain in a larger scale into the garden and literally walk the route along the hike.



With the new iOS 11.3 and ARKit 1.5 it's possible to project objects on vertical planes.

Although, it didn't improve the map and 3D maps still work best in a horizontal view.



Explore the three most common street trees in Manhattan, New York City with AR – while flying over the city and walking into the streets.

There are thousands of street trees in NY city and it's hard to distinguesh the tree varities from a bird's-eye view only. In fact, only the top 3 most common street trees in Manhattan are summing up to over 24k trees
Instead of showing so many data points on a 2D map, the experience of exploring street trees and buildings in a room provides a new way and playful exploration of this dataset. It’s like flying over the city to see some patterns and physically moving down to a street level to see different trees side-by-side.

We were also quite impressed of the amount of data that can be fluidly visualized in AR with an iPhone: over 24k data points consisting of 11,629 Honeylocust (green bars), 7,297 Callery pear (red bars) and 5,859 Ginkgo (yellow bars) trees, in addition to many buildings (blue).

How to get started with data visualizations on maps in augmented reality?

The Mapbox SDK for Unity, Unity, the newest Xcode and an iPhone X or a newer iPad works quite well together. The article »How I built an augmented reality app with the Mapbox Unity SDK and ARKit« on the mapbox blog helps a lot to get started. I can recommend to get into Unity and Mapbox first before getting started with ARKit, e.g. with this article
Unity is sometimes quite a pain with an outdated, complicated interface and coding isn't easy either. Nevertheless, it's quite powerful and works well together with the new augmented reality features from Apple with Unity’s ARKit plugin.


Visualizing ~10k soccer players in a physical space – and analyse them with a 3D scatterplot.

We visualized around 10k soccer players with a 3D scatterplot in a physical space. On the x-axis there is the defence, y-axis offence, z-axis minutes played value. The color highlights the overall rating. The scatterplot has been visualized tested in desktop-based 3D environment but didn’t keep up with the exciting experience to the immersive visualization.

The immersive web-experience has been implemented with three.ar.js – a helper three.js library for building AR web experiences that run in WebARonARKit and WebARonARCore
Download the code for the 3D AR scatterplot here or write me an email.