Skip to main content

Release Notes 4.4.0

Highlights​

Features/Fixes​

New VPS2 stop-localize and pose-to-geolocation APIs​

Developers can now stop localizing against a requested site, or every requested site, using stopLocalize(siteId:) in Swift, stopLocalize(siteId) in Kotlin, or TryStopLocalize(...) in Unity. Stopping removes the site's assets, their anchors, and their map data from the localization set.

Developers can also convert any AR pose to a geolocation using the latest VPS2 localization with getGeolocation(pose:) in Swift, getGeolocation(pose) in Kotlin, or TryGetGeolocation(...) in Unity. This is the inverse of getPose; see Getting VPS2 geoposition.

VPS2 site localization and asset tracking in Unity​

Unity developers can now localize to a site or to nearby sites with TryLocalize(...) on ARVps2Manager, and follow a VPS asset's origin in world space through the new ARVps2AssetManager trackables. Anchors move to the new ARVps2AnchorManager; see Breaking Changes and Placing virtual content.

Starting VPS2 is optional​

VPS2 now starts itself when an app calls a site or coordinate localize function, so an explicit start() is only needed to collect sensor data before a localization target is known. Asset tracking data is served for nearby assets before the device localizes against them, at reduced accuracy, and keeps being served after the device localizes elsewhere. See Adding VPS2.

Site assets expose their areas​

A VPS map too large for one mapping run is published as several areas. Sites now report every area of an asset through AssetInfo.areas and the new AssetArea type in Swift, Kotlin, and Unity, each with its own job status, anchor, nodes, and geographic bounds. VPS2 targets each area by its bounds, and the localized asset reports the area the device localized in.

Georeferenced mesh downloads​

Downloaded mesh chunks now carry a georeference, the geodetic origin and transform that place the mesh geographically, through geoReference on MeshDownloaderChunk in Swift, MeshDownloaderData in Kotlin, and the MeshGeoReference component in Unity. Each chunk also reports the VPS map asset ID and area ID it belongs to, and downloads by asset ID now cover every area of the map. See Mesh download.

Changelog​

Added​

  • Added stopLocalize(siteId) to stop localizing against a requested site, or every site when null (ARDK_VPS2_StopLocalize, Unity TryStopLocalize).
  • Added getGeolocation(pose) to convert an AR pose to a geolocation using the latest VPS2 localization (ARDK_VPS2_GetGeolocation, Unity TryGetGeolocation).
  • Added geolocation to Vps2AssetTrackingData, reporting the asset origin's geolocation when the asset is georeferenced.
  • Swift, Kotlin: Added areaId to Vps2LocalizedAsset and Vps2AssetTrackingData, naming the area of a split asset the device localized in.
  • Added AssetInfo.areas and AssetArea, describing every area of a site asset with its job status, anchor, nodes, and bounds.
  • Added the Published and Draft asset deployment types and the Canceled pipeline job status.
  • Added geoReference to downloaded mesh chunks, carrying the geodetic origin and transform needed to place the mesh geographically.
  • Added the VPS map asset ID and area ID to each downloaded mesh chunk (vpsMapAssetId, areaId; Unity MeshChunkInfo).
  • Unity: Added ARVps2Manager.TryLocalize overloads for site and coordinate VPS2 localization.
  • Unity: Added ARVps2AssetManager, ARVps2Asset, and ARVps2Manager.TryGetAssetTrackingData to follow VPS asset origins in world space.
  • Kotlin: Added captureDepth to createLiveDataSource to feed ARCore Depth API images to NSDK as platform depth.
  • Native: Added LiDAR depth and confidence to playback of datasets that record them.
  • Added a Unity guide for VPS2 site localization and placing content on localized assets.

Changed​

  • VPS2 site and coordinate localize calls now start VPS2 themselves; calling start() first is optional.
  • Asset tracking data is now served for nearby assets before localization, at reduced accuracy, and after the device localizes elsewhere.
  • VPS2 now filters map nodes by an asset's geographic bounds, using anchorDistanceGateMeters only for assets without bounds.
  • Mesh download by VPS map asset ID now downloads every area of a map published in several areas.
  • AssetInfo.pipelineJobId and sourceScanIds are now always empty, and assetStatus is always unspecified.
  • Sessions that use only platform depth, such as meshing with LiDAR, no longer download or initialize the depth model.
  • Native: Playback now loads frame images on demand instead of holding the whole dataset in memory.
  • Kotlin: Maven artifacts are now served per release line, from the nsdk-MAJOR.MINOR-release repository URL.

Deprecated​

  • Deprecated identifying maps by anchor payload, including LocalizationTarget.defaultAnchorPayload; localize by site ID or coordinate instead.
  • Deprecated AssetInfo typed asset data, assetStatus, pipelineJobId, pipelineJobStatus, and sourceScanIds; read AssetInfo.areas instead.

Fixed​

  • Fixed VPS2 discarding cloud localizations whose responses arrived after the device's tracking node changed.
  • Fixed VPS2 dropping from precise to coarse tracking when a new tracking node had not yet reached the map graph.
  • Fixed VPS2 device geolocation reporting a fabricated position and heading before the first camera pose arrived.
  • Fixed a crash when reading depth or scene segmentation results while the feature was stopping.
  • Fixed depth and scene segmentation acting on a stale model initialization after being stopped and restarted.
  • Fixed a crash at shutdown when NSDK logged after its logger was torn down.
  • Kotlin: Fixed incorrect colors in playback frames converted from bitmaps to YUV.
  • Kotlin: Fixed playback content misaligning with the camera image when the viewport aspect ratio differed from the image.

Breaking Changes​

  • All platforms: The VPS2 config fields assetTargetingEnterMarginMeters and assetTargetingExitMarginMeters are renamed to areaTargetingEnterMarginMeters and areaTargetingExitMarginMeters. Native and Python callers must use area_targeting_enter_margin_meters and area_targeting_exit_margin_meters.
  • Unity: ARVps2Manager.TryGetLatestLocalization now returns true whenever the subsystem exists, including while the tracking state is Unavailable. Callers that used the return value to decide whether the geolocation fields are valid must check XRVps2Localization.TrackingState instead.
  • Unity: ARVps2Manager is no longer an ARTrackableManager; anchors are served by the new ARVps2AnchorManager and XRVps2AnchorSubsystem. Anchor calls through ARVps2Manager still compile, but casts to ARTrackableManager and subclass Update() overrides must use the AnchorManager property instead. Scenes must add ARVps2AnchorManager in the editor and re-wire the anchor prefab and trackablesChanged listeners on it, and callers of the anchor methods on XRVps2Subsystem must move to XRVps2AnchorSubsystem.

Known Issues​

  • Importing the NSDK UPM in a project started from Universal 3D template might cause a few build errors reported in the console. Restarting the editor should fix this.
  • On iPad Pro running iOS 17 in the Unity External Sample, rotating the device from landscape to portrait may cause a soft lock and screen flickering.