# Place virtual content with VPS2 Source: https://www.nianticspatial.com/docs/nsdk/how-to/vps2/placing_virtual_content/?platform=swift This guide explains how to place virtual content in an AR scene relative to a [localized asset](https://www.nianticspatial.com/docs/nsdk/core_concepts/#anchor-and-place-content) or at known geo coordinates. Virtual content is any digital object rendered in AR, such as a 3D model, label, or visual effect. To appear in the correct location and remain stable as the user moves, it needs a reliable spatial reference. Depending on the placement approach, the app either attaches the content to a localized asset's origin or repeatedly calculates its AR pose from geo coordinates. When VPS2 localizes to a Site, it matches the camera view against the Site's **VPS map asset**. The asset's origin defines a tracked position and orientation in the real world. Placing virtual content relative to that origin allows the AR system to keep the content aligned with the environment over time. VPS2 handles the conversions between AR space, [map-relative space](https://www.nianticspatial.com/docs/nsdk/core_concepts/#map-relative-space), and [global space](https://www.nianticspatial.com/docs/nsdk/core_concepts/#global-space), so content placed against an asset can be expressed in whichever frame your app needs. Because the asset is a mapped real-world location, content placed relative to it appears consistently across sessions and devices. Follow this workflow: 1. [Choose a placement approach](#choose-how-to-place-virtual-content) based on where the content's position comes from and how your app needs to restore it. 2. Follow the readiness guidance for that approach. For [asset-relative placement](#place-content-relative-to-an-anchor), localize to the Site and evaluate the localized asset's tracking data and confidence. For [geo-coordinate placement](#place-content-using-geo-coordinates), evaluate the device's VPS2 tracking state and reported geolocation accuracy. 3. Use the instructions for the selected approach to place the content and save the data needed to restore it. 4. [Test the placement](#test-your-placement) and verify that the content remains stable as the AR session updates. Choose the approach once based on your application's placement and persistence requirements. If its readiness conditions are not met, wait for localization to improve or show a localizing state rather than switching approaches. --- ## Prerequisites Before you start, make sure you already have: - a project with NSDK installed and a basic AR scene set up - a Site in Scaniverse for workflows that localize to a Site or use Site-derived content For setup details, see [Set up the Niantic SDK](https://www.nianticspatial.com/docs/nsdk/setup/), [Set up a basic AR scene](https://www.nianticspatial.com/docs/nsdk/setup/#set-up-a-basic-ar-scene), and [First Localization with NSDK](https://www.nianticspatial.com/docs/nsdk/first_localization/). --- ## Choose how to place virtual content There are multiple ways to place virtual content in an AR scene. Each approach defines how **content is positioned**, how it **behaves over time**, and **what data is stored** so it can be reused. Choose an approach based on where the content's position comes from and how it must be restored. The current device or asset tracking state does not determine which approach to use. #### Key considerations - **Using multiple approaches** - Different features in your app can use different placement approaches. You do not need to choose only one for the entire application. - **Persistence** - Each approach defines what data, if any, is stored for reuse, and whether that reuse is limited to the current session or can work across sessions and devices: - **VPS-based placement** stores data relative to a real-world location, which can allow content to appear in the same place across sessions and devices. - **Local placement** can keep content stable during the current session, including user-driven placement such as hit tests, but it does not necessarily restore to the same real-world position later unless your app persists additional data. Use the following table to compare the available approaches based on when to use them, what data they save, and how each one restores content later. | Approach | Best for | Saved data | Restoration | | --- | --- | --- | --- | | [Place content relative to a localized asset](#place-content-relative-to-an-anchor) | You localize to a Site (by Site ID, or by coordinates for nearby Sites), then want content to stay attached to that asset's origin across sessions. | The Site ID, the asset ID, and each content item's asset-local transform. | Re-localizes to the same Site and reapplies the saved asset-local transform. | | [Place content using geo coordinates](#place-content-using-geo-coordinates) | Content is authored at a known latitude, longitude, and altitude. | Geolocation data such as latitude, longitude, altitude, and optional heading. | Reconstructs placement from geospatial coordinates and reapplies the resulting pose. | ## Know which readiness signal your approach uses The two approaches read **different** readiness signals, and using the wrong one is a common source of misplaced content: | Approach | Check this before placing | Why | | --- | --- | --- | | Place content relative to a localized asset | The **asset's** tracking data | The content is attached to the asset's origin, so the asset's pose is what has to be reliable. | | Place content using geo coordinates | The **device's** `Vps2TrackingState` | The content's position comes from the device's localization, not from a tracked asset. | These two states change independently -- the VPS2 tracking state can be `precise` while an asset is not yet returning tracking data. See [Tracking states](https://www.nianticspatial.com/docs/nsdk/features/vps2/#tracking-states). --- ### Place content relative to a localized asset Use this approach when the user decides where content should appear during a localized session. Save the content's local transform relative to the asset's origin, then apply that same local transform again the next time the asset is tracked. When you check an asset's tracking data, follow these rules: | Rule | Why it matters | What to do | |---|---|---| | **Evaluate each asset separately.** | If you localize to more than one Site, each asset is tracked on its own. Do not combine them into a single session-wide flag, or a distant asset that is not tracked will misreport a nearby asset that is. | Store and evaluate readiness by asset ID. | | **Tracking data means a pose is available; its confidence says how good it is.** | When VPS2 returns asset tracking data, the origin pose is available to place content against. The tracking confidence (`confidence`, `0.0`-`1.0`) estimates how likely the pose is to be correct. It rises as localizations agree, and falls as they disagree or grow stale while the device moves. | Place content once tracking data is available. If your content must align closely with the real world, also require a minimum tracking confidence, tuned by testing at your Site. | | **Check again after AR tracking is disturbed.** | If the camera is covered, the device is moved abruptly, or the session is interrupted, the asset can stop returning tracking data while content stays on screen at an out-of-date position. | Hide the placed content while tracking data is unavailable, and show it again once it returns. | Follow this workflow to place, save, and later restore content relative to a localized asset: 1. Localize to a Site with one of the `localize` overloads. - To localize to a specific Site, [create a Site](https://www.nianticspatial.com/docs/nsdk/first_localization/#create-a-private-site) in [Scaniverse](https://scaniverse.nianticspatial.com/signin) and get its Site ID (from Scaniverse or the [Sites API](https://www.nianticspatial.com/docs/nsdk/how-to/sites/getting_started/#3-query-sites)), then call [`localize(siteId:)`](https://www.nianticspatial.com/docs/api/swift/NSDK.NSDKVps2Session.method-localize). - To localize to nearby Sites, call [`localize(latitude:longitude:radiusMeters:)`](https://www.nianticspatial.com/docs/api/swift/NSDK.NSDKVps2Session.method-localize) with the device's location. You do not need a Site ID. 2. Poll [`assetTrackingData()`](https://www.nianticspatial.com/docs/api/swift/NSDK.NSDKVps2Session.method-assetTrackingData) for the localized asset and keep a matching `AnchorEntity` updated. `assetTrackingData()` returns data only while the asset is tracked; check its `confidence` if your content must align closely with the real world. When it is available, apply its `assetToTrackingTransform` to the `AnchorEntity` so content stays aligned as VPS2 refines the asset's pose. 3. Parent content under the asset's `AnchorEntity` using an asset-local `Transform`. Save that local transform together with the Site ID and asset ID. 4. Restore in a later session by localizing to the same Site again, waiting for tracking data to become available, then reapplying the saved local `Transform` under the asset's `AnchorEntity`. The following runnable example shows one complete class with localization, asset-pose updates, placement, save, and restore: #### Runnable example: Place and restore content relative to a localized asset Create a new file named `SiteAssetContentController.swift` and add the following code: ```swift import Combine import NSDK import RealityKit import simd import UIKit struct SavedAssetRelativeContent { let siteId: String let assetId: String let localTransform: Transform } /// Core VPS2 asset-relative placement flow. /// Trigger `startLocalizing`, `placeCubeAtAsset`, and `restoreSavedContent` /// from your own UI actions or app lifecycle code. final class SiteAssetContentController { private let vps2Session: NSDKVps2Session private weak var scene: Scene? private var siteId: String? private var currentAssetId: String? private let assetAnchor = AnchorEntity(world: .zero) private var cancellables = Set() private var assetReady = false private(set) var savedContent: SavedAssetRelativeContent? init(vps2Session: NSDKVps2Session, scene: Scene) { self.vps2Session = vps2Session self.scene = scene scene.addAnchor(assetAnchor) } func startLocalizing(siteId: String) throws { self.siteId = siteId try vps2Session.localize(siteId: siteId) } /// `$latestLocalization` fires every frame, so it doubles as the polling clock for the /// asset's pose. func beginAssetUpdates() { vps2Session.$latestLocalization .sink { [weak self] _ in self?.updateAssetPose() } .store(in: &cancellables) } private func updateAssetPose() { // Poll the currently localized asset. Data is served only while it is tracked. guard let tracking = vps2Session.assetTrackingData() else { assetReady = false assetAnchor.isEnabled = false // hide placed content while the asset is not tracked return } currentAssetId = tracking.assetId assetAnchor.transform = Transform(matrix: tracking.assetToTrackingTransform) assetAnchor.isEnabled = true assetReady = true } /// Call this from your placement UI action once the asset is tracked. func placeCubeAtAsset() { guard assetReady, let siteId, let currentAssetId else { return } let content = makeCubeEntity(size: 0.15, color: .green) let localTransform = Transform( scale: .one, rotation: simd_quatf(), translation: SIMD3(0, 0, -0.5) ) content.transform = localTransform assetAnchor.addChild(content) savedContent = SavedAssetRelativeContent(siteId: siteId, assetId: currentAssetId, localTransform: localTransform) } /// Call this in a later session after localizing to the same Site again. func restoreSavedContent() { // Restore only once the saved asset is tracked again. guard let savedContent, vps2Session.assetTrackingData(assetId: savedContent.assetId) != nil else { return } let restored = makeCubeEntity(size: 0.15, color: .yellow) restored.transform = savedContent.localTransform assetAnchor.addChild(restored) } private func makeCubeEntity(size: Float, color: UIColor) -> ModelEntity { ModelEntity( mesh: .generateBox(size: size), materials: [SimpleMaterial(color: color, isMetallic: false)] ) } } ``` Expected result: after you save a local `Transform` and localize to the same Site again, restored content returns to the same real-world location. ## Place content using geo coordinates When you place content from geo coordinates, follow these rules: | Rule | Why it matters | What to do | |---|---|---| | **At a Site, wait for the precise state before placing.** | `getPose` returns a pose even in the `coarse` tracking state, but it is only accurate enough to anchor virtual content once the state is `precise`. The state can fall back to `coarse`, for example when AR tracking resets. | Gate content placement on the `precise` state, keep checking it on each update, and treat `coarse` poses as approximate. | | **Expect `coarse` away from mapped Sites.** | Only localizing to a VPS map reaches `precise`. Local sensor fusion and universal localization stop at `coarse`, so content gated on `precise` does not appear away from mapped Sites. The runnable examples below gate on `precise`. | For content outside mapped Sites, place it in the `coarse` state and size or fade it using the reported horizontal and heading accuracy. | Follow this workflow to place content from geo coordinates: 1. Define the target latitude, longitude, and altitude for the real-world location where content should appear. These values can come from authored data, backend content, or a known location. 2. Pass those coordinates to the placement feature that already owns your VPS2 session and RealityKit scene objects. 3. On each localization update, convert the target latitude, longitude, and altitude to an AR pose with [`getPose(location:)`](https://www.nianticspatial.com/docs/api/swift/NSDK.NSDKVps2Session.method-getPose) and apply its transform to your RealityKit entity. `getPose(location:)` uses the latest localization and returns `nil` until VPS2 has localized. The returned `Vps2Pose` carries its `trackingState`; when your content needs accurate placement, wait for the `.precise` state. 4. Repeat step 3 every frame to keep content aligned as localization improves. The following runnable example shows one complete class with placement updates: #### Runnable example: Place and restore content from geo coordinates Create a new file named `GeoCoordinateContentController.swift` and add the following code: ```swift import Combine import NSDK import RealityKit struct SavedGeoPlacement { let latitude: Double let longitude: Double let altitude: Double } /// Core VPS2 geo-coordinate placement flow. /// Trigger `setTargetLocation` and `restoreSavedPlacement` /// from your own UI actions or app lifecycle code. final class GeoCoordinateContentController { private let vps2Session: NSDKVps2Session private weak var scene: Scene? private let contentRootAnchor = AnchorEntity(world: .zero) private let contentEntity: ModelEntity private var targetLocation: GeolocationData? private var cancellables = Set() private(set) var savedPlacement: SavedGeoPlacement? init(vps2Session: NSDKVps2Session, scene: Scene) { self.vps2Session = vps2Session self.scene = scene self.contentEntity = ModelEntity( mesh: .generateBox(size: 0.75), materials: [SimpleMaterial(color: .yellow, isMetallic: false)] ) contentRootAnchor.addChild(contentEntity) scene.addAnchor(contentRootAnchor) } func setTargetLocation(latitude: Double, longitude: Double, altitude: Double) { targetLocation = GeolocationData( latitude: latitude, longitude: longitude, altitude: altitude ) savePlacement(latitude: latitude, longitude: longitude, altitude: altitude) updateContentPose() } /// Call this after VPS2 starts to keep geo content updated. func beginLocalizationUpdates() { vps2Session.$latestLocalization .sink { [weak self] _ in self?.updateContentPose() } .store(in: &cancellables) } func updateContentPose() { guard let targetLocation else { return } // getPose returns a pose even in the coarse tracking state, but it is only accurate // enough to place content once the pose's tracking state is precise. guard let pose = vps2Session.getPose(location: targetLocation), pose.trackingState == .precise else { return } contentEntity.transform = Transform(matrix: pose.transform) } private func savePlacement(latitude: Double, longitude: Double, altitude: Double) { savedPlacement = SavedGeoPlacement( latitude: latitude, longitude: longitude, altitude: altitude ) } /// Call this in a later run after your app has started VPS2 again. func restoreSavedPlacement() { guard let savedPlacement else { return } setTargetLocation( latitude: savedPlacement.latitude, longitude: savedPlacement.longitude, altitude: savedPlacement.altitude ) } } ``` For reading the device's geolocation, see [Use geoposition with VPS2](https://www.nianticspatial.com/docs/nsdk/how-to/vps2/getting_vps2_geoposition/). ## Test Your Placement In addition to testing your app on-location, you can also use Playback in order to iterate quickly without traveling to a VPS-enabled Site. Playback runs NSDK against a prerecorded AR session dataset instead of live camera input. It can exercise the full asset-relative and geo-coordinate workflows, as long as the dataset was recorded at the Site. Still, Playback is only a simulation of a live session, so it is best for iteration and debugging rather than final validation. On-location device testing is the most accurate way to confirm that content appears in the correct real-world place for the actual Site. See [How to set up Playback](https://www.nianticspatial.com/docs/nsdk/how-to/playback/setting_up_playback/).