# How to enable scan visualization Source: https://www.nianticspatial.com/docs/nsdk/how-to/ar/scan_visualization/?platform=kotlin # How to enable scan visualization in AR Scan visualization gives users real-time feedback while they scan. Areas that have been sufficiently captured appear in full color, while incomplete areas remain striped. This makes it easier to understand scan coverage while recording. This guide is for developers who are building custom scan capture flows in Unity, Swift, or Kotlin and want to add that feedback to their app. It shows how to enable raycast visualization, connect it to the scanning session, and render the striped overlay during capture so users can spot gaps before saving a scan. **Raycast visualization** is the scan visualization mode that draws this striped overlay from the scanning session's raycast data. It is useful during development because it helps you confirm that scan coverage is complete, spot gaps before saving a scan, and verify that your capture UI is showing the visualization at the right time. (image: Example of raycast visualization in an AR scene) **Figure:** Raycast visualization overlays stripes on incomplete areas of a scan. In Kotlin, scan visualization is enabled by configuring `ScanningSession` with **raycast visualization** and displaying the resulting `raycastBuffer` through overlay content. The current sample uses a **Compose-based** capture screen and a **Filament** material, where Filament is the rendering engine used to draw the striped visualization. To enable scan visualization in Kotlin: 1. [Prepare your Kotlin project](#kotlin-prepare-your-project) 2. [Enable raycast visualization in the scanning configuration](#kotlin-enable-raycast-visualization-in-the-scanning-configuration) 3. [Add overlay content to your AR screen](#kotlin-add-overlay-content-for-the-visualization) 4. [Start and stop the visualization with capture](#kotlin-start-and-stop-the-visualization-with-capture) 5. [Update the visualization while scan data arrives](#kotlin-render-the-latest-raycast-buffer) --- ### Prepare your Kotlin project This section shows how to start from the existing Kotlin Capture sample so you can add scan visualization to a working AR scene instead of building the setup from scratch. It also identifies the sample app and screen used throughout the rest of the guide. Use the following steps to prepare your project using that sample as the starting point: 1. Clone [nsdk-samples-kotlin](https://github.com/nianticspatial/nsdk-samples-kotlin): ```bash git clone https://github.com/nianticspatial/nsdk-samples-kotlin.git ``` 2. Open the `NsdkSamples` project in Android Studio. 3. Run the `NsdkSamples` app on an Android device. 4. In the sample app, open the **Capture** sample and verify that it opens successfully. This walkthrough assumes: - the sample app builds and runs on your device - the Capture screen already has an active `NSDKSession` - the app already provides overlay content to the AR view For general setup steps, see [Set up the NSDK in Kotlin](https://www.nianticspatial.com/docs/nsdk/setup/#set-up-the-nsdk-in-kotlin). --- ### Enable raycast visualization in the scanning configuration This section shows how to configure the scanning session so it produces the raycast visualization data needed for the striped overlay. It also explains which configuration flags turn raycast visualization on and how that setup fits into the sample app's `CaptureManager`, which also handles capture, save, export, and overlay state. Update your capture manager so the scanning session enables raycast visualization before capture begins. In `nsdk-samples-kotlin`, that code lives in `NsdkSamples/NsdkSamples/src/main/java/com/nianticspatial/nsdk/externalsamples/capture/CaptureManager.kt`. The following code example shows the portion of the sample app's `CaptureManager` that configures scan visualization: ```kotlin import com.nianticspatial.nsdk.ScannerConfig fun startCapture() { val config = ScannerConfig().apply { useNsdkDepthsIfPlatformUnavailable = true enableRaycastVisualization = true enableVoxelVisualization = false } scanningSession.configure(config) scanningSession.start() } ``` In the previous code example: - `ScannerConfig` is used to tell the scanning session which visualization outputs to generate. - `enableRaycastVisualization = true` turns on the striped raycast overlay data. - `enableVoxelVisualization = false` leaves voxel visualization disabled, since this sample only displays the raycast overlay. - `useNsdkDepthsIfPlatformUnavailable = true` allows the session to keep generating visualization data on devices without native depth support. - `scanningSession.configure(config)` applies those settings before `scanningSession.start()` begins capture. #### Expand to reveal a minimal Kotlin example for raycast visualization setup This example reduces the Capture sample to the code required for this step. It lets you focus on enabling raycast visualization and starting a scanning session that can produce overlay data. - ⅰ. Create `NsdkSamples/NsdkSamples/src/main/java/com/nianticspatial/nsdk/externalsamples/capture/ScanVisualizationCaptureManager.kt` so you have a minimal manager that only configures raycast visualization and starts or stops scanning, then paste in the following code: ```kotlin package com.nianticspatial.nsdk.externalsamples.capture import android.util.Log import androidx.compose.runtime.getValue import androidx.compose.runtime.mutableStateOf import androidx.compose.runtime.setValue import androidx.lifecycle.LifecycleOwner import com.nianticspatial.nsdk.ScannerConfig import com.nianticspatial.nsdk.scanning.ScanningSession import com.nianticspatial.nsdk.externalsamples.FeatureManager class ScanVisualizationCaptureManager( private val scanningSession: ScanningSession ) : FeatureManager() { var isRecording by mutableStateOf(false) private set fun startCapture() { if (isRecording) return val config = ScannerConfig().apply { useNsdkDepthsIfPlatformUnavailable = true enableRaycastVisualization = true enableVoxelVisualization = false } scanningSession.configure(config) scanningSession.start() isRecording = true Log.i("ScanVisualizationCaptureManager", "Capture started") } fun stop() { if (!isRecording) return isRecording = false scanningSession.stop() Log.i("ScanVisualizationCaptureManager", "Capture stopped") } override fun onDestroy(owner: LifecycleOwner) { stop() scanningSession.close() } } ``` - ⅱ. Create `NsdkSamples/NsdkSamples/src/main/java/com/nianticspatial/nsdk/externalsamples/capture/ScanVisualizationCaptureView.kt` so you have a minimal screen that can call that manager without the rest of the sample app's Capture flow, then paste in the following code: ```kotlin package com.nianticspatial.nsdk.externalsamples.capture import android.app.Activity import android.util.Log import androidx.compose.material3.Button import androidx.compose.material3.Text import androidx.compose.runtime.Composable import androidx.compose.runtime.DisposableEffect import androidx.compose.runtime.MutableState import androidx.compose.runtime.getValue import androidx.compose.runtime.remember import androidx.compose.runtime.setValue import androidx.compose.ui.Modifier import androidx.compose.ui.platform.LocalLifecycleOwner import com.nianticspatial.nsdk.externalsamples.HelpContent import com.nianticspatial.nsdk.externalsamples.NSDKSessionManager import com.nianticspatial.nsdk.externalsamples.common.OverlayContent @Composable fun ScanVisualizationCaptureView( context: Activity, nsdkManager: NSDKSessionManager, helpContentState: MutableState, overlayContentState: MutableState ) { val captureManager = remember(nsdkManager) { ScanVisualizationCaptureManager(nsdkManager.session.scanning.acquire()) } val lifecycleOwner = LocalLifecycleOwner.current DisposableEffect(captureManager) { lifecycleOwner.lifecycle.addObserver(captureManager) Log.i("ScanVisualizationCaptureView", "Capture view started") onDispose { captureManager.onDestroy(lifecycleOwner) lifecycleOwner.lifecycle.removeObserver(captureManager) } } Button( onClick = { Log.i("ScanVisualizationCaptureView", "Capture button tapped") if (captureManager.isRecording) { captureManager.stop() } else { captureManager.startCapture() } }, modifier = Modifier ) { Text(if (captureManager.isRecording) "Stop Capture" else "Start Capture") } } ``` - ⅲ. Open `NsdkSamples/NsdkSamples/src/main/java/com/nianticspatial/nsdk/externalsamples/NSDKDemoView.kt`, then add this import so the file can reference the new minimal screen: ```kotlin import com.nianticspatial.nsdk.externalsamples.capture.ScanVisualizationCaptureView ``` - ⅳ. In the same `NSDKDemoView.kt` file, replace this line so the app opens directly into the Capture route instead of the sample picker: ```kotlin NavHost(navController = navController, startDestination = SelectorRoute, modifier) { ``` with this line: ```kotlin NavHost(navController = navController, startDestination = CaptureRoute(), modifier) { ``` - ⅴ. In the same `NSDKDemoView.kt` file, replace this line so the Capture route shows the minimal screen from this section instead of the full sample's `CaptureView(...)`: ```kotlin CaptureView(activity, nsdkSessionManager, helpContentState, overlayContentState) ``` with this line: ```kotlin ScanVisualizationCaptureView(activity, nsdkSessionManager, helpContentState, overlayContentState) ``` In the previous code example: - `ScanVisualizationCaptureManager` keeps only the scanning configuration and simple start and stop behavior needed to validate raycast visualization. - `ScanVisualizationCaptureView` gives the sample app a minimal Compose screen that can create that manager and toggle capture. - Adding the import for `ScanVisualizationCaptureView` makes the new composable available inside `NSDKDemoView.kt`. - Replacing `startDestination = SelectorRoute` with `startDestination = CaptureRoute()` makes the sample app open directly to the Capture route instead of the sample picker. - Replacing `CaptureView(...)` with `ScanVisualizationCaptureView(...)` makes that route render the minimal screen from this section. - The `Log.i(...)` calls are optional temporary debug lines that you can remove after you finish testing. At this stage, the Kotlin sample is configuring raycast visualization and starting and stopping scanning, but it is not yet creating an overlay to display the output. When you test this step, the app should open directly into the minimal Capture screen instead of the sample picker. You should see a single button labeled **Start Capture**. Tap it once to start scanning and confirm that the button label changes to **Stop Capture**. At this stage you should not expect any visualization on screen yet, because the overlay has not been added. --- ### Add overlay content to your AR screen This section shows how to create the overlay content that renders scan visualization over the AR view. It also explains how the sample app uses a Filament material and overlay class to draw the striped visualization from the scanning output. Create an overlay content class that provides the material used to render the scan visualization. The following code example shows the portion of the sample app's `ScanVisualizationCaptureOverlayContent` that sets up the striped visualization material: ```kotlin class ScanVisualizationCaptureOverlayContent( private val captureManager: ScanVisualizationCaptureManager ) : OverlayContent() { override fun onCreateMaterial( engine: Engine, materialLoader: MaterialLoader ): MaterialInstance { val material = materialLoader.createMaterial("materials/capture_viz.filamat") val materialInstance = material.createInstance() materialInstance.setParameter("stripeStride", 2) materialInstance.setParameter("stripeRedPixels", 1) return materialInstance } } ``` This material renders the striped overlay over the camera feed. #### Expand to reveal a minimal Kotlin example for overlay content setup This example builds on the previous step and keeps only the code needed to define the overlay. It lets you focus on how the overlay content creates the Filament material used to render the striped scan visualization. - ⅰ. Keep `ScanVisualizationCaptureManager.kt` from the previous section so the minimal screen can continue to start and stop scanning. - ⅱ. Create `NsdkSamples/NsdkSamples/src/main/java/com/nianticspatial/nsdk/externalsamples/capture/ScanVisualizationCaptureOverlayContent.kt` so the app has a minimal overlay class that can render the striped visualization material, then paste in the following code: ```kotlin package com.nianticspatial.nsdk.externalsamples.capture import android.util.Log import com.google.android.filament.Engine import com.google.android.filament.MaterialInstance import com.nianticspatial.nsdk.externalsamples.common.OverlayContent import io.github.sceneview.loaders.MaterialLoader class ScanVisualizationCaptureOverlayContent( private val captureManager: ScanVisualizationCaptureManager ) : OverlayContent() { override fun onCreateMaterial( engine: Engine, materialLoader: MaterialLoader ): MaterialInstance { Log.i("ScanVisualizationCaptureOverlay", "Overlay material created") val material = materialLoader.createMaterial("materials/capture_viz.filamat") val materialInstance = material.createInstance() materialInstance.setParameter("stripeStride", 2) materialInstance.setParameter("stripeRedPixels", 1) return materialInstance } override fun onFrame(engine: Engine, materialInstance: MaterialInstance): Boolean { return false } override fun onDestroy(engine: Engine) { } } ``` - ⅲ. Open `NsdkSamples/NsdkSamples/src/main/java/com/nianticspatial/nsdk/externalsamples/capture/ScanVisualizationCaptureView.kt`, then add the following line immediately after `val lifecycleOwner = LocalLifecycleOwner.current` so the minimal screen instantiates the overlay object during startup: ```kotlin val captureOverlayContent = remember(captureManager) { ScanVisualizationCaptureOverlayContent(captureManager) } ``` - ⅳ. In the same `ScanVisualizationCaptureView.kt` file, add the following block immediately after that `remember(...)` call so the screen writes a confirmation log as soon as the overlay object is created: ```kotlin DisposableEffect(captureOverlayContent) { Log.i("ScanVisualizationCaptureView", "Overlay content created") onDispose { } } ``` In the previous code example: - `ScanVisualizationCaptureOverlayContent` is the minimal overlay class that owns the striped-material setup for this walkthrough. - `onFrame(...)` and `onDestroy(...)` are included as no-op implementations so the minimal overlay compiles before the later section adds the real frame update logic. - `Log.i("ScanVisualizationCaptureOverlay", "Overlay material created")` is kept for the next section, when the overlay is attached to the AR scene and Filament creates its material. - Adding `val captureOverlayContent = remember(captureManager) { ... }` to `ScanVisualizationCaptureView.kt` makes the minimal screen instantiate the overlay object in this section, even though it is not attached yet. - `DisposableEffect(captureOverlayContent)` gives you a log message from `ScanVisualizationCaptureView.kt`, which is more reliable for this step than waiting for the overlay material to be created. At this stage, the app has the manager and a minimal overlay implementation, but the overlay is not yet attached to the AR scene. When you test this step, you should not expect a visible change yet. This step only adds the overlay class; the next section attaches it to the AR scene. To confirm that this code is running: - ⅰ. Open Android Studio's **Logcat** tool window. - ⅱ. In the process or app selector at the top of Logcat, select your running sample app so Logcat stops showing unrelated system logs. - ⅲ. Click the search field in Logcat and enter `tag:ScanVisualizationCaptureView Overlay content created`. - ⅳ. Run the app. - ⅴ. Confirm that Logcat shows the message `Overlay content created`. That message confirms that `ScanVisualizationCaptureView.kt` created a `ScanVisualizationCaptureOverlayContent` instance during startup. `Overlay material created` will not appear until the next section, when the overlay is attached to the AR scene and Filament creates its material. --- ### Start and stop the visualization with capture This section shows how to connect overlay visibility to the existing capture flow so the visualization appears when scanning starts and disappears when scanning stops. It also explains how the sample app attaches overlay content to the AR scene and keeps it synchronized with capture state. Update the Capture screen so it attaches the overlay and shows or hides it based on whether capture is active. The following code example shows the portion of the sample app's `ScanVisualizationCaptureView` that controls overlay visibility: ```kotlin val captureOverlayContent = remember(captureManager) { ScanVisualizationCaptureOverlayContent(captureManager) } DisposableEffect(captureOverlayContent) { overlayContentState.value = captureOverlayContent onDispose { captureOverlayContent.hide() overlayContentState.value = null } } LaunchedEffect(captureManager.isRecording) { if (captureManager.isRecording) { captureOverlayContent.show() } else { captureOverlayContent.hide() } } ``` This matches the sample flow: the overlay appears while capture is running and hides when capture ends. In the previous code example: - `remember(captureManager)` keeps a stable overlay instance tied to the current capture manager. - `DisposableEffect(captureOverlayContent)` attaches the overlay to `overlayContentState` when the screen appears and removes it when the screen is disposed. - `LaunchedEffect(captureManager.isRecording)` shows the overlay only while the capture manager is actively recording. #### Expand to reveal a minimal Kotlin example for starting and stopping visualization This example builds on the previous steps and keeps only the code needed to attach and control the overlay. It lets you focus on how CaptureView shows and hides the visualization in sync with capture state. - ⅰ. Keep the existing `ScanVisualizationCaptureManager.kt` and `ScanVisualizationCaptureOverlayContent.kt` files from the previous sections unchanged. In this section, the only file you need to update is `NsdkSamples/NsdkSamples/src/main/java/com/nianticspatial/nsdk/externalsamples/capture/ScanVisualizationCaptureView.kt`. Open that existing file, delete its current contents, and replace them with the following code so the same minimal screen now attaches the overlay to the shared AR scene and shows or hides it with capture state: ```kotlin package com.nianticspatial.nsdk.externalsamples.capture import android.app.Activity import android.util.Log import androidx.compose.material3.Button import androidx.compose.material3.Text import androidx.compose.runtime.Composable import androidx.compose.runtime.DisposableEffect import androidx.compose.runtime.LaunchedEffect import androidx.compose.runtime.MutableState import androidx.compose.runtime.remember import androidx.compose.ui.Modifier import androidx.compose.ui.platform.LocalLifecycleOwner import com.nianticspatial.nsdk.externalsamples.HelpContent import com.nianticspatial.nsdk.externalsamples.NSDKSessionManager import com.nianticspatial.nsdk.externalsamples.common.OverlayContent @Composable fun ScanVisualizationCaptureView( context: Activity, nsdkManager: NSDKSessionManager, helpContentState: MutableState, overlayContentState: MutableState ) { val captureManager = remember(nsdkManager) { ScanVisualizationCaptureManager(nsdkManager.session.scanning.acquire()) } val lifecycleOwner = LocalLifecycleOwner.current val captureOverlayContent = remember(captureManager) { ScanVisualizationCaptureOverlayContent(captureManager) } DisposableEffect(captureManager) { lifecycleOwner.lifecycle.addObserver(captureManager) Log.i("ScanVisualizationCaptureView", "Capture view started") onDispose { captureManager.onDestroy(lifecycleOwner) lifecycleOwner.lifecycle.removeObserver(captureManager) } } DisposableEffect(captureOverlayContent) { overlayContentState.value = captureOverlayContent Log.i("ScanVisualizationCaptureView", "Overlay attached") onDispose { overlayContentState.value = null } } LaunchedEffect(captureManager.isRecording) { if (captureManager.isRecording) { Log.i("ScanVisualizationCaptureView", "Showing overlay") captureOverlayContent.show() } else { Log.i("ScanVisualizationCaptureView", "Hiding overlay") captureOverlayContent.hide() } } Button( onClick = { Log.i("ScanVisualizationCaptureView", "Capture button tapped") if (captureManager.isRecording) { captureManager.stop() } else { captureManager.startCapture() } }, modifier = Modifier ) { Text(if (captureManager.isRecording) "Stop Capture" else "Start Capture") } } ``` In the previous code example: - `DisposableEffect(captureOverlayContent)` attaches the overlay to the shared AR scene used by the sample app. - `LaunchedEffect(captureManager.isRecording)` makes the overlay show while capture is active and hide when capture stops. - The `Log.i(...)` lines are optional temporary debug lines that you can remove after you finish testing. - At this stage, the overlay can appear and disappear, but it still does not receive per-frame raycast texture updates. When you test this step: - ⅰ. Clear Android Studio's **Logcat** output. - ⅱ. Run the app again. - ⅲ. Search for `Overlay attached`. - ⅳ. Confirm that the message appears during startup. - ⅴ. Tap **Start Capture** once. - ⅵ. Search for `Showing overlay`. - ⅶ. Confirm that the message appears after capture starts. What these logs mean: - `Overlay attached` confirms that `overlayContentState.value = captureOverlayContent` ran and the overlay was passed into the shared AR scene. - `Showing overlay` confirms that the overlay is shown when capture starts. At this stage, the overlay may still look blank because the next section has not yet uploaded any raycast image data into it. --- ### Update the visualization while scan data arrives This section shows how to update the visualization continuously during capture so the overlay reflects the latest scan coverage in real time. It also explains how the sample app polls `raycastBuffer()`, converts the returned image data into a Filament texture, and uploads it to the overlay. Update the capture manager and overlay so the latest raycast data is polled and rendered during capture. The following code examples show the portions of the sample app's `ScanVisualizationCaptureManager` and `ScanVisualizationCaptureOverlayContent` that read the raycast buffer and display it in the overlay: ```kotlin coroutineScope.launch { while (isRecording) { try { raycastBuffer = scanningSession.raycastBuffer() } catch (e: Exception) { Log.e("CaptureManager", "Error getting raycast buffer", e) } delay(RAYCAST_BUFFER_POLL_DELAY_MS) } raycastBuffer = null } ``` Then convert the returned image data into a Filament texture in your overlay: ```kotlin override fun onFrame(engine: Engine, materialInstance: MaterialInstance): Boolean { val raycastBuffer = captureManager.raycastBuffer ?: return false val intImage = raycastBuffer.rgba if (intImage.width <= 0 || intImage.height <= 0) return false val buffer = ByteBuffer.allocateDirect(intImage.data.size * 4) .order(ByteOrder.nativeOrder()) for (pixel in intImage.data) { buffer.put((pixel shr 0 and 0xFF).toByte()) buffer.put((pixel shr 8 and 0xFF).toByte()) buffer.put((pixel shr 16 and 0xFF).toByte()) buffer.put((pixel shr 24 and 0xFF).toByte()) } buffer.flip() val pixelBuffer = Texture.PixelBufferDescriptor( buffer, Texture.Format.RGBA, Texture.Type.UBYTE ) texture?.setImage(engine, 0, pixelBuffer) return true } ``` In the previous code example: - `raycastBuffer = scanningSession.raycastBuffer()` is the point where the capture manager pulls the latest visualization data while recording is active. - `onFrame(...)` is the point where the overlay turns that buffer into a Filament texture that can actually render on screen. #### Expand to reveal a minimal Kotlin example for frame-driven visualization updates This example builds on the previous steps and keeps only the code needed to refresh the overlay during capture. - ⅰ. Keep `NsdkSamples/NsdkSamples/src/main/java/com/nianticspatial/nsdk/externalsamples/capture/ScanVisualizationCaptureView.kt` unchanged from the previous section. - ⅱ. Open `NsdkSamples/NsdkSamples/src/main/java/com/nianticspatial/nsdk/externalsamples/capture/ScanVisualizationCaptureManager.kt`, delete its current contents, and replace them with the following code so the capture manager stores the latest `raycastBuffer` while capture is active: ```kotlin package com.nianticspatial.nsdk.externalsamples.capture import android.util.Log import androidx.compose.runtime.getValue import androidx.compose.runtime.mutableStateOf import androidx.compose.runtime.setValue import androidx.lifecycle.LifecycleOwner import com.nianticspatial.nsdk.ScannerConfig import com.nianticspatial.nsdk.externalsamples.FeatureManager import com.nianticspatial.nsdk.scanning.RaycastBuffer import com.nianticspatial.nsdk.scanning.ScanningSession import kotlinx.coroutines.CoroutineScope import kotlinx.coroutines.Dispatchers import kotlinx.coroutines.SupervisorJob import kotlinx.coroutines.cancel import kotlinx.coroutines.delay import kotlinx.coroutines.launch class ScanVisualizationCaptureManager( private val scanningSession: ScanningSession, private val coroutineScope: CoroutineScope = CoroutineScope(Dispatchers.Default + SupervisorJob()) ) : FeatureManager() { companion object { const val RAYCAST_BUFFER_POLL_DELAY_MS = 16L } var isRecording by mutableStateOf(false) private set var raycastBuffer by mutableStateOf(null) private set fun startCapture() { if (isRecording) return val config = ScannerConfig().apply { useNsdkDepthsIfPlatformUnavailable = true enableRaycastVisualization = true enableVoxelVisualization = false } scanningSession.configure(config) scanningSession.start() isRecording = true Log.i("ScanVisualizationCaptureManager", "Capture started") coroutineScope.launch { while (isRecording) { try { raycastBuffer = scanningSession.raycastBuffer() Log.i("ScanVisualizationCaptureManager", "Updated raycast buffer") } catch (e: Exception) { Log.e("ScanVisualizationCaptureManager", "Error getting raycast buffer", e) } delay(RAYCAST_BUFFER_POLL_DELAY_MS) } raycastBuffer = null } } fun stop() { if (!isRecording) return isRecording = false raycastBuffer = null scanningSession.stop() Log.i("ScanVisualizationCaptureManager", "Capture stopped") } override fun onDestroy(owner: LifecycleOwner) { stop() scanningSession.close() coroutineScope.cancel() } } ``` - ⅲ. Open `NsdkSamples/NsdkSamples/src/main/java/com/nianticspatial/nsdk/externalsamples/capture/ScanVisualizationCaptureOverlayContent.kt`, delete its current contents, and replace them with the following code so it uploads the latest `raycastBuffer` into the overlay texture: ```kotlin package com.nianticspatial.nsdk.externalsamples.capture import android.util.Log import com.google.android.filament.Engine import com.google.android.filament.MaterialInstance import com.google.android.filament.Texture import com.google.android.filament.TextureSampler import com.nianticspatial.nsdk.externalsamples.common.OverlayContent import io.github.sceneview.loaders.MaterialLoader import io.github.sceneview.safeDestroyTexture import java.nio.ByteBuffer import java.nio.ByteOrder class ScanVisualizationCaptureOverlayContent( private val captureManager: ScanVisualizationCaptureManager ) : OverlayContent() { private var texture: Texture? = null override fun onCreateMaterial(engine: Engine, materialLoader: MaterialLoader): MaterialInstance { val material = materialLoader.createMaterial("materials/capture_viz.filamat") val materialInstance = material.createInstance() materialInstance.setParameter("stripeStride", 2) materialInstance.setParameter("stripeRedPixels", 1) return materialInstance } override fun onFrame(engine: Engine, materialInstance: MaterialInstance): Boolean { val raycastBuffer = captureManager.raycastBuffer ?: return false val intImage = raycastBuffer.rgba if (intImage.width <= 0 || intImage.height <= 0) return false if (texture == null || texture!!.getWidth(0) != intImage.width || texture!!.getHeight(0) != intImage.height) { texture?.let { engine.safeDestroyTexture(it) } texture = Texture.Builder() .width(intImage.width) .height(intImage.height) .levels(1) .format(Texture.InternalFormat.RGBA8) .sampler(Texture.Sampler.SAMPLER_2D) .build(engine) materialInstance.setParameter( "visualizationTexture", texture!!, TextureSampler( TextureSampler.MinFilter.LINEAR, TextureSampler.MagFilter.LINEAR, TextureSampler.WrapMode.CLAMP_TO_EDGE ) ) } val buffer = ByteBuffer.allocateDirect(intImage.data.size * 4) .order(ByteOrder.nativeOrder()) for (pixel in intImage.data) { buffer.put((pixel shr 0 and 0xFF).toByte()) buffer.put((pixel shr 8 and 0xFF).toByte()) buffer.put((pixel shr 16 and 0xFF).toByte()) buffer.put((pixel shr 24 and 0xFF).toByte()) } buffer.flip() val pixelBuffer = Texture.PixelBufferDescriptor( buffer, Texture.Format.RGBA, Texture.Type.UBYTE ) texture?.setImage(engine, 0, pixelBuffer) Log.i("ScanVisualizationCaptureOverlay", "Displayed composited texture") return true } override fun onDestroy(engine: Engine) { texture?.let { engine.safeDestroyTexture(it) } texture = null } } ``` In the previous code examples: - `raycastBuffer` stores the latest scan output while capture is running. - `onFrame(...)` converts that output into a Filament texture and uploads it to the overlay. - The `Log.i(...)` lines are optional temporary debug lines that you can remove after you finish testing. When you test this step: - ⅰ. Clear Android Studio's **Logcat** output. - ⅱ. Run the app again. - ⅲ. Tap **Start Capture**. - ⅳ. Move the device around the scene for a few seconds. - ⅴ. Confirm that the scan visualization overlay now appears on top of the AR view. - ⅵ. Confirm that incomplete areas remain striped while scanned areas update as you move. - ⅶ. Tap **Stop Capture**. - ⅷ. Confirm that the button returns to **Start Capture** and the visualization stops updating. Optional Logcat checks: - Search for `Updated raycast buffer` to confirm that the capture manager is polling fresh scan data during capture. - Search for `Displayed composited texture` to confirm that the overlay is uploading that data into the Filament texture. What these results mean: - A visible updating overlay confirms that the polling loop and the overlay texture upload path are both working together. - `Updated raycast buffer` confirms that `scanningSession.raycastBuffer()` is returning new visualization data. - `Displayed composited texture` confirms that `onFrame(...)` is converting that data and pushing it into the overlay material. After you finish validating the Kotlin example, you can remove the temporary `Log.i(...)` debug lines. --- ## Troubleshooting This section helps you diagnose common issues in the Kotlin sample where scan visualization does not appear, does not update, or reduces performance. It focuses on the Compose-based capture screen, the overlay wiring, and the Filament texture upload path used in this walkthrough. ### Visualization doesn't appear - Verify that the `composable` entry in `NSDKDemoView.kt` is still launching `ScanVisualizationCaptureView(...)` instead of the original `CaptureView(...)`. - Confirm that `Overlay attached` appears during startup and that `Showing overlay` appears after you tap **Start Capture**. - Check that `overlayContentState.value = captureOverlayContent` still runs inside `DisposableEffect(captureOverlayContent)` in `ScanVisualizationCaptureView.kt`. - Verify that `ScanVisualizationCaptureOverlayContent.kt` still loads `materials/capture_viz.filamat` in `onCreateMaterial(...)`. - Ensure the app has camera permission and that the AR scene is active before you start capture. ### Overlay doesn't update - Confirm that `Updated raycast buffer` appears in logcat while capture is active. - Confirm that `Displayed composited texture` appears after the overlay is shown. - If the overlay appears but stays blank, check that `captureManager.raycastBuffer` is not `null` inside `onFrame(...)`. - If `Updated raycast buffer` appears but `Displayed composited texture` does not, verify that `captureOverlayContent.show()` still runs from `LaunchedEffect(captureManager.isRecording)`. - If logcat shows warnings about the source image being upscaled, treat them as quality or performance warnings rather than as a sign that the visualization path is broken. ### Performance issues - Polling `raycastBuffer()` and uploading a new texture every frame can reduce performance on lower-end devices. - Reduce validation logging after testing, since repeated `Log.i(...)` calls in the polling loop and `onFrame(...)` can add overhead. - The minimal example allocates a new `ByteBuffer` for each frame, so expect this version to be a validation path rather than a fully optimized production implementation. - Avoid running other expensive AR features at the same time unless necessary.