How to Download a Mesh Using the API

With the Mesh Download API, you can download and create a mesh of any Site at runtime, allowing you to dynamically create mesh overlays in AR scenes. This feature makes it easier to test AR experiences by allowing you to check that your mesh lines up with the real world without having to leave the test environment. For example, you can download a stored mesh after localizing to see how far offset your localization is and figure out how it needs to change. Mesh downloading also allows developers to place content in scenes and explore environmental interactions without needing to stop testing and set up each mesh they want to try.

A Site mesh is delivered as a stream of chunks rather than as a single download. Identify the map by the VPS map asset ID of the map you localized to. The first request starts the download process, and mesh chunks are received via polling the API. Chunks arrive closest-first relative to a geographic position you supply, so the geometry nearest the user renders while the rest is still downloading. You can cap how much mesh data the request downloads to help with rendering large maps.

Anchor payload mesh download APIs are deprecated

The APIs for downloading meshes using a Site's anchor payload are deprecated. These APIs do not support maps captured by 360 cameras or location-based streaming. Site IDs and asset IDs are the preferred mechanisms for identifying maps in NSDK. Use the streaming, asset-based API described on this page instead.

Platform	Deprecated API	Migrate To
Unity	LocationMeshManager.GetLocationMeshForPayloadAsync	LocationMeshManager.GetLocationMeshChunksByVpsMapAssetIdAsync
Swift	NSDKMeshDownloader.requestLocationMesh(payload:)	NSDKMeshDownloader.requestLocationMeshByAsset(vpsMapAssetId:)
Kotlin	MeshDownloaderSession.download(payload =)	MeshDownloaderSession.downloadByAsset(vpsMapAssetId =)

Prerequisites

You will need:

• a Unity project with NSDK installed and configured
• an NSDK access token configured for the app; see Authorization
• the VPS map asset ID of the Site, obtained through the Sites API
• a geographic position to order the chunks by, ideally the device's current location

For project and scene setup, see Set up the Niantic SDK for Unity.

Get the VPS map asset ID

Fetch the Site's assets and use the ID of its production VPS asset. AssetInfo.Id is the VPS map asset ID to use with Mesh Downloader.

using NianticSpatial.NSDK.AR.Sites;

[SerializeField]
private SitesClientManager sitesClientManager;

var assetsResult = await sitesClientManager.GetAssetsForSiteAsync(siteId);

if (assetsResult.Status != SitesRequestStatus.Success)
{
    return;
}

string vpsMapAssetId = null;
foreach (var asset in assetsResult.Assets)
{
    if (asset.VpsData.HasValue && asset.Deployment == AssetDeploymentType.Production)
    {
        vpsMapAssetId = asset.Id;
        break;
    }
}

Stream a Site mesh

LocationMeshManager is a component included with NSDK. Add Location Mesh Manager to a GameObject, then reference that component from the script that downloads the mesh:

using NianticSpatial.NSDK.AR.Subsystems;

[SerializeField]
private LocationMeshManager locationMeshManager;

In the Inspector, drag the GameObject containing Location Mesh Manager into this field. Then iterate GetLocationMeshChunksByVpsMapAssetIdAsync with await foreach. Each iteration yields one chunk as a GameObject with its transform already applied.

View the Unity mesh streaming code

var meshRoot = new GameObject("DownloadedMesh");
meshRoot.transform.SetParent(siteAsset.transform, false);

int chunkCount = 0;

await foreach (var chunk in locationMeshManager.GetLocationMeshChunksByVpsMapAssetIdAsync(
                   vpsMapAssetId,
                   siteLatitude,
                   siteLongitude,
                   getTexture: true))
{
    // Chunks arrive inactive so they never flash at the scene origin. Set the hierarchy before activating it.
    chunk.transform.SetParent(meshRoot.transform, false);
    chunk.SetActive(true);
    chunkCount++;
}

// A finished stream that produced no chunks is the signal that no mesh was downloaded.
if (chunkCount == 0)
{
    Destroy(meshRoot);
}

latitude and longitude only decide the order the chunks arrive in. They do not affect the position of the mesh itself.

Position the mesh chunks

Downloading a mesh does not localize the device or position the mesh in the AR scene, and each chunk's transform is relative to the VPS map's own origin. To know the correct real-world position for the mesh, we first need to localize to its map. Track the same VPS map asset and parent the chunks to its trackable, as in the code above:

if (!arVps2Manager.TryTrackAsset(vpsMapAssetId, out ARVps2Asset siteAsset))
{
    return;
}

// Update visibility as tracking changes in your app.
meshRoot.SetActive(siteAsset.trackingState == TrackingState.Tracking);

TryTrackAsset accepts any valid asset ID, but the trackable only receives a pose once the asset's Site has been submitted with TryLocalize and localized. See Place virtual content with VPS2 for that flow.

Unity scene requirements and sample files

A project that downloads and renders a mesh needs authorization; a Location Mesh Manager; a compatible mesh material; and a source for the Site's VPS asset. Placing the mesh at its physical Site also requires an AR Session, an XR Origin with an AR camera and AR VPS2 Manager, and camera and location permissions.

The NSDK Unity sample project shows how these pieces are connected:

• Assets/Samples/VPS2/Scenes/VPS2Localization.unity contains the scene components.
• Assets/Samples/VPS2/Scripts/VPS2AssetLocalizeDemo.cs contains the asset tracking and mesh streaming flow.
• Assets/Samples/VPS2/Scripts/SitesTargetListManager.cs obtains the VPS asset ID and the Site's coordinates.

Limit how much mesh data you download

GetLocationMeshChunksByVpsMapAssetIdAsync also accepts options for download size, collision geometry, textures, and cancellation:

await foreach (var chunk in locationMeshManager.GetLocationMeshChunksByVpsMapAssetIdAsync(
                   vpsMapAssetId,
                   siteLatitude,
                   siteLongitude,
                   getTexture: true,
                   maxChunks: 8,
                   maxSizeKb: 10240,
                   addCollider: true,
                   cancelOnDisable: true))

• maxChunks downloads only the given number of nearest chunks. The default, 0, downloads all chunks.
• maxSizeKb stops the request once the downloaded mesh and texture data reaches this size. The default, 0, sets no limit.
• getTexture requests texture data. When it is false, chunks contain geometry with no texture image.
• addCollider adds collision geometry to each chunk.
• cancelOnDisable stops the stream if LocationMeshManager is disabled.

The mesh must use a material compatible with the project's render pipeline. Set Textured Mesh Material and Vertex Color Material on the Location Mesh Manager: chunks that carry a texture use the textured material, and chunks without one use the vertex color material. Refer to the Location Mesh Manager in the sample scene for the setup used by the installed NSDK version. Textured meshes contain more download data; untextured meshes are smaller and require a material that renders vertex colors.

Textures and 360 captures

Meshes generated from 360 camera captures are served as decimated, untextured meshes, so it's recommended to render them with a vertex color or wireframe material to avoid covering the AR camera view with opaque surfaces. Due to the decimation, their level of geometry detail will be lower than the original reconstruction in Scaniverse.

Known Issues

• Some meshes can be quite large. If download size or runtime performance is a concern, limit the number of chunks or set a maximum download size.
• A maximum download size is checked after each chunk is delivered, so the chunk that crosses the limit is still delivered, and the total downloaded size can exceed the limit by up to one chunk.

More Information

See LocationMeshManager in the Unity API reference.