# How to set up playback Source: https://www.nianticspatial.com/docs/nsdk/how-to/playback/setting_up_playback/?platform=kotlin ## Description This guide explains how to set up Playback in an Android application so you can run NSDK using a prerecorded dataset instead of a live AR session. Playback uses recorded AR session data to simulate device input, allowing you to test and iterate without needing to move a physical device. When a dataset is loaded, frames are fed into the NSDK session from the playback pipeline. If no dataset is available, your app can run using a live AR session instead. ## Prerequisites You will need an Android project with the NSDK Kotlin library integrated. For more information, see [Set up the NSDK in Kotlin](https://www.nianticspatial.com/docs/nsdk/setup/#set-up-the-nsdk-in-kotlin). ## Steps To set up Playback in your Android app, complete the following steps: 1. [Add your dataset to the Android assets directory](#add-your-dataset-to-the-android-assets-directory) -- Include the dataset in your app's assets so it can be loaded at runtime. 2. [Load the dataset with AssetPlaybackDatasetLoader](#load-the-dataset-with-assetplaybackdatasetloader) -- Parse the dataset metadata and prepare it for playback. 3. [Create a PlaybackSessionDataSource and assign it to NSDKSession](#create-a-playbacksessiondatasource-and-assign-it-to-nsdksession) -- Wire up the pull-based data source so the SDK can read frames, camera data, and sensors. 4. [Create a PlaybackSession and drive the frame loop](#create-a-playbacksession-and-drive-the-frame-loop) -- Feed playback frames into the data source and call `NSDKSession.update()` each frame. 5. [Start and stop playback](#start-and-stop-playback) -- Control the playback lifecycle and release resources when finished. ### Add your dataset to the Android assets directory Copy your playback dataset folder into `src/main/assets/` in your Android module. The folder must contain a `capture.json` file along with the recorded frame images (for example, `src/main/assets/playback/MyDataset/`). (image: Android Studio Project view shows assets under module root though it is located in src/main/assets.) > **Note:** > > In Android Studio's project view, `src/main/assets/` is displayed as `assets/` directly under your module root, for example `app/assets/playback/MyDataset/`). This is a display alias reflecting how the directory is presented in the UI. The actual on-disk path remains `src/main/assets/playback/MyDataset/`. ### Load the dataset with AssetPlaybackDatasetLoader Construct an `AssetPlaybackDatasetLoader` with the path under `assets/`, then call `loadDataset()` to parse the capture JSON and get a `PlaybackDataset`. Call `hasDepth()` on the result to determine whether the recording contains depth data, and pass that to `NSDKSession`: ```kotlin import com.nianticspatial.nsdk.playback.AssetPlaybackDatasetLoader val loader = AssetPlaybackDatasetLoader(context, "playback/MyDataset") val dataset = loader.loadDataset() // null if capture.json is missing or invalid val nsdkSession = NSDKSession( accessToken = accessToken, refreshToken = refreshToken, useLidar = dataset?.hasDepth() ?: false ) ``` `AssetPlaybackDatasetLoader` loads frame images on-demand, so only metadata is read upfront. ### Create a PlaybackSessionDataSource and assign it to NSDKSession `PlaybackSessionDataSource` implements the pull-based data source interface. It prepares the YUV camera image for the SDK on a background thread and exposes it via `NsdkSessionDataSource`. Assign it to `NSDKSession.dataSource` so `update()` can read from it each frame. The `frameProvider` lambda is how the latest available `PlaybackFrame` is injected into the data source -- it is called by the data source when it is time to process a new frame. The `orientationProvider` lambda should return the **current device display orientation**, not the orientation recorded in the dataset. This is required for ML-based features such as depth and scene segmentation to work correctly during playback. `PlaybackSessionDataSource` implements `DefaultLifecycleObserver`, so registering it with your `Lifecycle` handles resource cleanup automatically when the component is destroyed. The following code example configures a playback data source and connects it to the SDK session: ```kotlin import com.nianticspatial.nsdk.helpers.PlaybackSessionDataSource import com.nianticspatial.nsdk.Orientation import com.nianticspatial.nsdk.playback.PlaybackFrame var latestFrame: PlaybackFrame? = null val dataSource = PlaybackSessionDataSource( frameProvider = { latestFrame }, orientationProvider = { deviceOrientation } // current device display orientation ) lifecycle.addObserver(dataSource) // cleans up image resources in onDestroy nsdkSession.dataSource = dataSource ``` ### Create a PlaybackSession and drive the frame loop Create a `PlaybackSession` from the loaded dataset and register a frame listener. In the listener, update `latestFrame` and call `dataSource.prepareFrame()` followed by `nsdkSession.update()`. The following code example processes each playback frame and forwards it to the SDK session: ```kotlin import com.nianticspatial.nsdk.playback.PlaybackSession import kotlinx.coroutines.CoroutineScope import kotlinx.coroutines.Dispatchers import kotlinx.coroutines.SupervisorJob import kotlinx.coroutines.launch val scope = CoroutineScope(SupervisorJob() + Dispatchers.Main.immediate) val playbackSession = PlaybackSession(dataset) playbackSession.setOnFrameListener { frame -> scope.launch { latestFrame = frame dataSource.prepareFrame() // bitmap -> YUV on background thread nsdkSession.update() // pulls from dataSource and sends to native } } ``` ### Start and stop playback Call `play()` to begin the frame loop and `pause()` to stop it. During teardown, call `dispose()` to release all held references and cancel the listener, and `cancel()` on your coroutine scope: ```kotlin playbackSession.play() // start // when pausing: playbackSession.pause() // in teardown (e.g. onDestroy): playbackSession.dispose() scope.cancel() ``` All NSDK features (Depth, Scene Segmentation, Object Detection) work the same as in a live session.