# How to enable scan visualization Source: https://www.nianticspatial.com/docs/nsdk/how-to/ar/scan_visualization/?platform=swift # How to enable scan visualization in AR Scan visualization gives users real-time feedback while they scan. Areas that have been sufficiently captured appear in full color, while incomplete areas remain striped. This makes it easier to understand scan coverage while recording. This guide is for developers who are building custom scan capture flows in Unity, Swift, or Kotlin and want to add that feedback to their app. It shows how to enable raycast visualization, connect it to the scanning session, and render the striped overlay during capture so users can spot gaps before saving a scan. **Raycast visualization** is the scan visualization mode that draws this striped overlay from the scanning session's raycast data. It is useful during development because it helps you confirm that scan coverage is complete, spot gaps before saving a scan, and verify that your capture UI is showing the visualization at the right time. (image: Example of raycast visualization in an AR scene) **Figure:** Raycast visualization overlays stripes on incomplete areas of a scan. In Swift, scan visualization is enabled by configuring `NsdkScanningSession` with **raycast visualization** and then compositing the returned `raycastBuffer` textures into an overlay view. The current sample uses a **Metal-backed** `TextureView` to display the visualization while capture is active. To enable scan visualization in Swift: 1. [Prepare your Swift project](#swift-prepare-your-project) 2. [Enable raycast visualization in the scanning configuration](#swift-enable-raycast-visualization-in-the-scanning-configuration) 3. [Add a visualization view to your AR screen](#swift-add-a-visualization-view-to-your-ar-screen) 4. [Start and stop the visualization with capture](#swift-start-and-stop-the-visualization-with-capture) 5. [Update the visualization while AR frames arrive](#swift-update-the-visualization-while-ar-frames-arrive) --- ### Prepare your Swift project This section shows how to start from the existing Swift Capture sample so you can add scan visualization to a working AR scene instead of building the setup from scratch. It identifies the sample app and screen used throughout the rest of the guide. Use the following steps to prepare your project: 1. Clone [nsdk-samples-swift](https://github.com/nianticspatial/nsdk-samples-swift): ```bash git clone https://github.com/nianticspatial/nsdk-samples-swift.git ``` 2. Open `NsdkSamples.xcworkspace` in Xcode. 3. Run the `NsdkSamples` app on an iOS device. 4. In the sample app, open the **Capture** sample and verify that it opens successfully. This walkthrough assumes: - the sample app builds and runs on your device - the Capture screen already has an active `nsdkSession` - the AR view controller already receives `ARSession` frame updates For general setup steps, see [Set up the NSDK in Swift](https://www.nianticspatial.com/docs/nsdk/setup/#set-up-the-nsdk-in-swift). --- ### Enable raycast visualization in the scanning configuration This section shows how to configure the scanning session so it produces the raycast visualization data needed for the striped overlay. It also explains which configuration flags turn raycast visualization on and how that setup fits into the sample app's `CaptureManager`, which also includes capture, save, and export logic. Create or update your capture manager to enable raycast visualization in the scanning session. In `nsdk-samples-swift`, that code lives in `NsdkSamples/NsdkSamples/NSDK/CaptureManager.swift`. The following code example shows the portion of the sample app's `CaptureManager` that configures scan visualization: ```swift import SwiftyNsdk import Metal class CaptureManager : NsdkFeatureManager { private let enableRaycastVisualization: Bool private let enableVoxelVisualization: Bool init( nsdk: NsdkSession, enableRaycastVisualization: Bool, enableVoxelVisualization: Bool ) { self.enableRaycastVisualization = enableRaycastVisualization self.enableVoxelVisualization = enableVoxelVisualization super.init(session: nsdk.createScanningSession()) } override func configuration() -> NsdkScanningSession.Configuration { print("ScanVisualizationCaptureManager configuration called") var config = NsdkScanningSession.Configuration() config.enableRaycastVisualization = enableRaycastVisualization config.enableVoxelVisualization = enableVoxelVisualization config.generateDepthsIfLidarUnavailable = true return config } } ``` In the previous code example: - `CaptureManager` subclasses `NsdkFeatureManager`, so it owns and configures the scanning session used by the sample. - `enableRaycastVisualization` and `enableVoxelVisualization` are stored as properties so the manager can decide which visualization outputs to request. - The initializer receives those flags and creates the scanning session from `NsdkSession`. - `configuration()` creates an `NsdkScanningSession.Configuration` and turns on `enableRaycastVisualization` so the session produces the striped raycast overlay data. - `enableVoxelVisualization = false` keeps voxel visualization disabled, since this sample only displays the raycast overlay. - `generateDepthsIfLidarUnavailable = true` ensures the session can still generate visualization data on devices without LiDAR. This configuration enables the scanning session to produce raycast visualization data. Later sections show how the app reads that data and renders it on screen. #### Expand to reveal a minimal Swift example for raycast visualization setup This example reduces the Capture sample to the code required for this step. It lets you focus on how `CaptureManager` enables raycast visualization and produces the scan output needed for the overlay. - ⅰ. Use the following code in a file called `ScanVisualizationCaptureManager.swift` as a replacement for `CaptureManager.swift`: ```swift import Foundation import SwiftyNsdk import Metal class ScanVisualizationCaptureManager: NsdkFeatureManager { private let enableRaycastVisualization: Bool private let enableVoxelVisualization: Bool private let device: MTLDevice? private(set) var rgbaTexture: MTLTexture? private(set) var normalsTexture: MTLTexture? private(set) var positionAndConfidenceTexture: MTLTexture? private var cachedRgbaWidth = 0 private var cachedRgbaHeight = 0 private var cachedNormalsWidth = 0 private var cachedNormalsHeight = 0 private var cachedPositionWidth = 0 private var cachedPositionHeight = 0 init( nsdk: NsdkSession, enableRaycastVisualization: Bool, enableVoxelVisualization: Bool ) { self.enableRaycastVisualization = enableRaycastVisualization self.enableVoxelVisualization = enableVoxelVisualization self.device = MTLCreateSystemDefaultDevice() super.init(session: nsdk.createScanningSession()) } override func configuration() -> NsdkScanningSession.Configuration { var config = NsdkScanningSession.Configuration() config.enableRaycastVisualization = enableRaycastVisualization config.enableVoxelVisualization = enableVoxelVisualization config.generateDepthsIfLidarUnavailable = true return config } func updateRaycastTextures() -> Bool { guard let device = device else { return false } guard let raycastBuffer = session.raycastBuffer() else { return false } rgbaTexture = createOrUpdateTexture( from: raycastBuffer.rgba, existing: rgbaTexture, cachedWidth: &cachedRgbaWidth, cachedHeight: &cachedRgbaHeight, pixelFormat: .rgba8Unorm, device: device ) normalsTexture = createOrUpdateTexture( from: raycastBuffer.normals, existing: normalsTexture, cachedWidth: &cachedNormalsWidth, cachedHeight: &cachedNormalsHeight, pixelFormat: .rgba8Unorm, device: device ) positionAndConfidenceTexture = createOrUpdateTexture( from: raycastBuffer.positionAndConfidence, existing: positionAndConfidenceTexture, cachedWidth: &cachedPositionWidth, cachedHeight: &cachedPositionHeight, pixelFormat: .rgba16Float, device: device ) return rgbaTexture != nil && normalsTexture != nil && positionAndConfidenceTexture != nil } private func createOrUpdateTexture( from image: RawImage, existing: MTLTexture?, cachedWidth: inout Int, cachedHeight: inout Int, pixelFormat: MTLPixelFormat, device: MTLDevice ) -> MTLTexture? { let width = Int(image.width) let height = Int(image.height) var texture = existing if texture == nil || cachedWidth != width || cachedHeight != height { let descriptor = MTLTextureDescriptor.texture2DDescriptor( pixelFormat: pixelFormat, width: width, height: height, mipmapped: false ) descriptor.usage = [.shaderRead] descriptor.storageMode = .shared guard let newTexture = device.makeTexture(descriptor: descriptor) else { return nil } texture = newTexture cachedWidth = width cachedHeight = height } guard let validTexture = texture else { return nil } let bytesPerRow: Int switch pixelFormat { case .rgba8Unorm: bytesPerRow = width * 4 case .rgba16Float: bytesPerRow = width * 8 default: return nil } validTexture.replace( region: MTLRegionMake2D(0, 0, width, height), mipmapLevel: 0, withBytes: image.data, bytesPerRow: bytesPerRow ) return validTexture } } ``` - ⅱ. Use the following code in a file called `ScanVisualizationCaptureViewController.swift` as a replacement for `CaptureViewController.swift`: ```swift import UIKit import ARKit import SwiftyNsdk class ScanVisualizationCaptureViewController: BaseARViewController { private var nsdkCapture: ScanVisualizationCaptureManager? private var isCapturing = false private var captureButton: UIButton? override func viewDidLoad() { super.viewDidLoad() print("ScanVisualizationCaptureViewController started") } override func setupUI() { super.setupUI() captureButton = addButton( buttonTitle: "Start Capture", onClickAction: #selector(handleCaptureButtonTap) ) } @objc private func handleCaptureButtonTap() { print("ScanVisualizationCaptureViewController button tapped") guard let session = nsdkManager?.session else { return } if nsdkCapture == nil { nsdkCapture = ScanVisualizationCaptureManager( nsdk: session, enableRaycastVisualization: true, enableVoxelVisualization: false ) } if isCapturing { nsdkCapture?.stop() isCapturing = false captureButton?.setTitle("Start Capture", for: .normal) } else { nsdkCapture?.start() isCapturing = true captureButton?.setTitle("Stop Capture", for: .normal) } } } ``` In the previous code example: - `nsdkCapture` stores the scan-visualization manager so the controller can reuse the same session across button taps. - `setupUI()` adds a button that toggles capture, using the sample app's existing `BaseARViewController` helpers. - `handleCaptureButtonTap()` creates `ScanVisualizationCaptureManager` on first use, then starts and stops the scanning session without any save or export behavior. - ⅲ. To run this minimal example from the sample app menu, update the `Capture` entry in `MainViewController.swift` so it points to `ScanVisualizationCaptureViewController.self`: ```swift ViewControllerItem(title: "Capture", viewControllerType: ScanVisualizationCaptureViewController.self), ``` At this stage, the session is producing raycast visualization data, but the app is not displaying it yet. The next section adds the visualization view needed to render that data on screen. When you test this step, the `print(...)` statements help confirm that the sample app is launching `ScanVisualizationCaptureViewController` and configuring `ScanVisualizationCaptureManager`. After you verify the flow, you can remove those debug lines. --- ### Add a visualization view to your AR screen This section shows how to add an overlay view that can display scan visualization over the live AR content. It also explains how the sample app uses a `TextureView` to render the composited visualization and position it so it appears only while capture is active. Create a `TextureView` and add it as an overlay over the AR view. The following code example shows the portion of the sample app's `CaptureViewController` that creates and configures the visualization view: ```swift private var visualizationView: TextureView? override func setupUI() { super.setupUI() visualizationView = TextureView( frame: view.bounds, vertexShader: "scanningVertexShader", fragmentShader: "scanningFragmentShader" ) visualizationView!.isOpaque = true visualizationView!.translatesAutoresizingMaskIntoConstraints = false visualizationView!.clearColor = MTLClearColor(red: 0, green: 0, blue: 0, alpha: 0) visualizationView!.isHidden = true arView.addSubview(visualizationView!) NSLayoutConstraint.activate([ visualizationView!.leadingAnchor.constraint(equalTo: view.leadingAnchor), visualizationView!.topAnchor.constraint(equalTo: view.topAnchor), visualizationView!.trailingAnchor.constraint(equalTo: view.trailingAnchor), visualizationView!.bottomAnchor.constraint(equalTo: view.bottomAnchor) ]) } ``` In the previous code example: - `visualizationView` is stored as a property so the controller can show it when capture starts, hide it when capture stops, and push new textures into it while frames update. - `TextureView` is added as a subview of `arView`, so the visualization is drawn directly over the live AR content. - `vertexShader: "scanningVertexShader"` and `fragmentShader: "scanningFragmentShader"` use the sample app's existing Metal shader setup instead of introducing a second rendering path. - `clearColor` uses a transparent alpha value so the visualization can appear as an overlay instead of replacing the camera view. - `isHidden = true` keeps the view off-screen until the capture flow explicitly enables it. - The constraints pin the visualization view to the edges of the screen so it fills the same area as the AR view. #### Expand to reveal a minimal Swift example for visualization view setup This example builds on the previous step and keeps only the code needed to add the overlay view. It lets you focus on how `CaptureViewController` creates, places, and prepares `TextureView` to display scan visualization. - ⅰ. Keep `ScanVisualizationCaptureManager.swift` from the previous section. - ⅱ. Use the following code in `ScanVisualizationCaptureViewController.swift` as a replacement for the earlier minimal controller to create a minimal runnable code example through visualization view setup: ```swift import UIKit import ARKit import SwiftyNsdk import Metal import MetalKit class ScanVisualizationCaptureViewController: BaseARViewController { private var nsdkCapture: ScanVisualizationCaptureManager? private var isCapturing = false private var captureButton: UIButton? private var visualizationView: TextureView? override func viewDidLoad() { super.viewDidLoad() print("ScanVisualizationCaptureViewController started") } override func setupUI() { super.setupUI() captureButton = addButton( buttonTitle: "Start Capture", onClickAction: #selector(handleCaptureButtonTap) ) visualizationView = TextureView( frame: view.bounds, vertexShader: "scanningVertexShader", fragmentShader: "scanningFragmentShader" ) visualizationView!.isOpaque = true visualizationView!.translatesAutoresizingMaskIntoConstraints = false visualizationView!.clearColor = MTLClearColor(red: 0, green: 0, blue: 0, alpha: 0) visualizationView!.isHidden = true arView.addSubview(visualizationView!) print("Visualization view configured") NSLayoutConstraint.activate([ visualizationView!.leadingAnchor.constraint(equalTo: view.leadingAnchor), visualizationView!.topAnchor.constraint(equalTo: view.topAnchor), visualizationView!.trailingAnchor.constraint(equalTo: view.trailingAnchor), visualizationView!.bottomAnchor.constraint(equalTo: view.bottomAnchor) ]) } @objc private func handleCaptureButtonTap() { guard let session = nsdkManager?.session else { return } if nsdkCapture == nil { nsdkCapture = ScanVisualizationCaptureManager( nsdk: session, enableRaycastVisualization: true, enableVoxelVisualization: false ) } if isCapturing { nsdkCapture?.stop() isCapturing = false captureButton?.setTitle("Start Capture", for: .normal) visualizationView?.isHidden = true } else { nsdkCapture?.start() isCapturing = true captureButton?.setTitle("Stop Capture", for: .normal) visualizationView?.isHidden = false } } } ``` In the previous code example: - `viewDidLoad()` keeps the earlier debug print so you can confirm the sample app is still launching `ScanVisualizationCaptureViewController`. - `visualizationView` is now part of the minimal controller, so the app has a dedicated view ready to display the composited overlay in later steps. - `print("Visualization view configured")` gives you a lightweight validation point that the new view was created and added to the AR screen. - `print("ScanVisualizationCaptureViewController button tapped")` confirms that the capture button is still wired to this controller while you test the updated setup. - `handleCaptureButtonTap()` now keeps the earlier start and stop behavior, while also showing and hiding the visualization view as capture starts and stops. At this stage, the app has the controller, manager, and visualization view in place, and the capture button now shows and hides that view while the scanning session starts and stops. The next section adds the frame-driven update path that renders scan output into the view. --- ### Start and stop the visualization with capture This section shows how to connect scan visualization to the existing capture flow so the overlay appears when scanning starts and disappears when scanning stops. It also explains how the sample app keeps visualization state aligned with the same UI controls and lifecycle used for capture. Update your capture button logic so it starts and stops both scanning and visualization. The following code example shows the portion of the sample app's `CaptureViewController` that controls visualization visibility during capture: ```swift @objc private func handleCaptureButtonTap() { guard let session = nsdkManager?.session else { return } if nsdkCapture == nil { nsdkCapture = ScanVisualizationCaptureManager( nsdk: session, enableRaycastVisualization: true, enableVoxelVisualization: false ) } if isCapturing { isCapturing = false visualizationView?.isHidden = true visualizationView?.reset() nsdkCapture?.stop() } else { nsdkCapture?.start() isCapturing = true visualizationView?.isHidden = false } } ``` In the previous code example: - `handleCaptureButtonTap()` keeps the same manager-creation logic from earlier sections, so the controller still uses `ScanVisualizationCaptureManager`. - `visualizationView?.isHidden = false` shows the overlay view when capture starts, and `visualizationView?.isHidden = true` hides it again when capture stops. - `visualizationView?.reset()` clears any previously assigned texture so the view starts cleanly the next time capture begins. - This step removes the sample app's save and export behavior so the button only controls scanning and visualization visibility. #### Expand to reveal a minimal Swift example for starting and stopping visualization This example builds on the previous steps and keeps only the code needed to control visualization during capture. It lets you focus on how `CaptureViewController` shows and hides the overlay as scanning starts and stops. - ⅰ. Keep `ScanVisualizationCaptureManager.swift` from the previous sections. - ⅱ. Use the following code in `ScanVisualizationCaptureViewController.swift` as a replacement for the earlier minimal controller to create a minimal runnable code example through start and stop: ```swift import UIKit import ARKit import SwiftyNsdk import Metal import MetalKit class ScanVisualizationCaptureViewController: BaseARViewController { private var nsdkCapture: ScanVisualizationCaptureManager? private var isCapturing = false private var captureButton: UIButton? private var visualizationView: TextureView? override func viewDidLoad() { super.viewDidLoad() print("ScanVisualizationCaptureViewController started") } override func setupUI() { super.setupUI() captureButton = addButton( buttonTitle: "Start Capture", onClickAction: #selector(handleCaptureButtonTap) ) visualizationView = TextureView( frame: view.bounds, vertexShader: "scanningVertexShader", fragmentShader: "scanningFragmentShader" ) visualizationView!.isOpaque = true visualizationView!.translatesAutoresizingMaskIntoConstraints = false visualizationView!.clearColor = MTLClearColor(red: 0, green: 0, blue: 0, alpha: 0) visualizationView!.isHidden = true arView.addSubview(visualizationView!) print("Visualization view configured") NSLayoutConstraint.activate([ visualizationView!.leadingAnchor.constraint(equalTo: view.leadingAnchor), visualizationView!.topAnchor.constraint(equalTo: view.topAnchor), visualizationView!.trailingAnchor.constraint(equalTo: view.trailingAnchor), visualizationView!.bottomAnchor.constraint(equalTo: view.bottomAnchor) ]) } @objc private func handleCaptureButtonTap() { print("ScanVisualizationCaptureViewController button tapped") guard let session = nsdkManager?.session else { return } if nsdkCapture == nil { nsdkCapture = ScanVisualizationCaptureManager( nsdk: session, enableRaycastVisualization: true, enableVoxelVisualization: false ) } if isCapturing { print("Stopping scan visualization") nsdkCapture?.stop() isCapturing = false captureButton?.setTitle("Start Capture", for: .normal) visualizationView?.isHidden = true visualizationView?.reset() } else { print("Starting scan visualization") nsdkCapture?.start() isCapturing = true captureButton?.setTitle("Stop Capture", for: .normal) visualizationView?.isHidden = false } } } ``` In the previous code example: - `print("Starting scan visualization")` and `print("Stopping scan visualization")` confirm that the button now drives the start and stop path for capture. - The button title changes between `Start Capture` and `Stop Capture`, which gives you a visible UI signal that the controller state is updating. - The visualization view is now shown and hidden in sync with the scanning session, even though no textures are rendered into it yet. At this stage, the app can start and stop scanning and show and hide the visualization view, but the view is still blank. The next section adds the frame update path that renders the raycast visualization into the view. --- ### Update the visualization while AR frames arrive This section shows how to update the visualization continuously as new AR frames arrive so the overlay reflects the latest scan coverage in real time. It also explains how the sample app reads raycast output from the scanning session and sends the resulting texture to the visualization view. Use the AR session frame callback to refresh the visualization while capture is active. The following code example shows the portion of the sample app's `ScanVisualizationCaptureViewController` that updates the overlay using data from the scanning session: ```swift override func session(_ session: ARSession, didUpdate frame: ARFrame) { super.session(session, didUpdate: frame) if isCapturing { updateVisualization() } } private func updateVisualization() { guard let capture = nsdkCapture, let textureView = visualizationView else { return } if capture.updateRaycastTextures() { if let rgbaTexture = capture.rgbaTexture { compositeVisualization(from: rgbaTexture) if let composited = compositedTexture { textureView.setTexture(copyFrom: composited) } } } } ``` In the previous code example: - `session(_:didUpdate:)` is the point where ARKit frame updates drive the visualization refresh while capture is active. - `updateVisualization()` reads the latest scan output from `ScanVisualizationCaptureManager`, composites it, and sends the finished texture to `TextureView`. The following code example shows the portion of the sample app's `ScanVisualizationCaptureManager` that calls `session.raycastBuffer()` and converts the returned buffers into Metal textures: ```swift func updateRaycastTextures() -> Bool { print("ScanVisualizationCaptureManager updating raycast textures") guard let device = device else { return false } guard let raycastBuffer = session.raycastBuffer() else { return false } TextureUtils.createOrUpdateTexture(from: raycastBuffer.rgba, texture: &rgbaTexture, device: device) TextureUtils.createOrUpdateTexture(from: raycastBuffer.normals, texture: &normalsTexture, device: device) TextureUtils.createOrUpdateTexture(from: raycastBuffer.positionAndConfidence, texture: &positionAndConfidenceTexture, device: device) return rgbaTexture != nil && normalsTexture != nil && positionAndConfidenceTexture != nil } ``` In the previous code example: - `print("ScanVisualizationCaptureManager updating raycast textures")` gives you a direct log signal that the manager is receiving raycast data while the device moves through the scene. #### Expand to reveal a minimal Swift example for frame-driven visualization updates This example builds on the previous steps and keeps only the code needed to refresh the overlay during capture. It lets you focus on how AR frame updates drive texture updates and how the composited scan visualization is rendered on screen. - ⅰ. Keep `ScanVisualizationCaptureManager.swift` from the previous sections. - ⅱ. Use the following code in `ScanVisualizationCaptureViewController.swift` as a replacement for the earlier minimal controller to create a minimal runnable code example through frame updates: ```swift import UIKit import ARKit import SwiftyNsdk import Metal import MetalKit class ScanVisualizationCaptureViewController: BaseARViewController { private var nsdkCapture: ScanVisualizationCaptureManager? private var isCapturing = false private var captureButton: UIButton? private var visualizationView: TextureView? private var device: MTLDevice? private var commandQueue: MTLCommandQueue? private var compositeKernel: MTLComputePipelineState? private var compositedTexture: MTLTexture? private var cachedCompositeWidth = 0 private var cachedCompositeHeight = 0 private var stripeStride: UInt32 = 7 private var stripeRedPixels: UInt32 = 5 override func viewDidLoad() { super.viewDidLoad() print("ScanVisualizationCaptureViewController started") setupMetalResources() } override func setupUI() { super.setupUI() captureButton = addButton( buttonTitle: "Start Capture", onClickAction: #selector(handleCaptureButtonTap) ) visualizationView = TextureView( frame: view.bounds, vertexShader: "scanningVertexShader", fragmentShader: "scanningFragmentShader" ) visualizationView!.isOpaque = true visualizationView!.translatesAutoresizingMaskIntoConstraints = false visualizationView!.clearColor = MTLClearColor(red: 0, green: 0, blue: 0, alpha: 0) visualizationView!.isHidden = true arView.addSubview(visualizationView!) print("Visualization view configured") NSLayoutConstraint.activate([ visualizationView!.leadingAnchor.constraint(equalTo: view.leadingAnchor), visualizationView!.topAnchor.constraint(equalTo: view.topAnchor), visualizationView!.trailingAnchor.constraint(equalTo: view.trailingAnchor), visualizationView!.bottomAnchor.constraint(equalTo: view.bottomAnchor) ]) } private func setupMetalResources() { device = MTLCreateSystemDefaultDevice() guard let device = device else { print("Failed to create Metal device") return } commandQueue = device.makeCommandQueue() guard let library = device.makeDefaultLibrary(), let kernelFunction = library.makeFunction(name: "scanning_composite_kernel") else { print("Failed to load scanning composite kernel") return } do { compositeKernel = try device.makeComputePipelineState(function: kernelFunction) } catch { print("Failed to create compute pipeline state: \(error)") } } @objc private func handleCaptureButtonTap() { print("ScanVisualizationCaptureViewController button tapped") guard let session = nsdkManager?.session else { return } if nsdkCapture == nil { nsdkCapture = ScanVisualizationCaptureManager( nsdk: session, enableRaycastVisualization: true, enableVoxelVisualization: false ) } if isCapturing { print("Stopping scan visualization") nsdkCapture?.stop() isCapturing = false captureButton?.setTitle("Start Capture", for: .normal) visualizationView?.isHidden = true visualizationView?.reset() } else { print("Starting scan visualization") nsdkCapture?.start() isCapturing = true captureButton?.setTitle("Stop Capture", for: .normal) visualizationView?.isHidden = false } } override func session(_ session: ARSession, didUpdate frame: ARFrame) { super.session(session, didUpdate: frame) if isCapturing { updateVisualization() } } private func updateVisualization() { print("ScanVisualizationCaptureViewController updating visualization") guard let capture = nsdkCapture, let textureView = visualizationView else { return } if capture.updateRaycastTextures() { if let rgbaTexture = capture.rgbaTexture { compositeVisualization(from: rgbaTexture) if let composited = compositedTexture { print("ScanVisualizationCaptureViewController displayed composited texture") textureView.setTexture(copyFrom: composited) } } } } private func compositeVisualization(from source: MTLTexture) { guard let device = device, let kernel = compositeKernel, let queue = commandQueue, let commandBuffer = queue.makeCommandBuffer(), let encoder = commandBuffer.makeComputeCommandEncoder() else { return } let width = source.width let height = source.height if compositedTexture == nil || cachedCompositeWidth != width || cachedCompositeHeight != height { let descriptor = MTLTextureDescriptor.texture2DDescriptor( pixelFormat: .rgba8Unorm, width: width, height: height, mipmapped: false ) descriptor.usage = [.shaderWrite, .shaderRead] descriptor.storageMode = .shared compositedTexture = device.makeTexture(descriptor: descriptor) cachedCompositeWidth = width cachedCompositeHeight = height } guard let output = compositedTexture else { return } struct CompositeUniforms { var stripeStride: UInt32 var stripeRedPixels: UInt32 } var uniforms = CompositeUniforms( stripeStride: stripeStride, stripeRedPixels: stripeRedPixels ) encoder.setComputePipelineState(kernel) encoder.setTexture(output, index: 0) encoder.setTexture(source, index: 1) encoder.setBytes(&uniforms, length: MemoryLayout.stride, index: 0) let threadgroupWidth = min(kernel.threadExecutionWidth, width) let threadgroupHeight = min(kernel.maxTotalThreadsPerThreadgroup / threadgroupWidth, height) let threadgroupSize = MTLSize(width: threadgroupWidth, height: threadgroupHeight, depth: 1) let threadgroupCount = MTLSize( width: (width + threadgroupWidth - 1) / threadgroupWidth, height: (height + threadgroupHeight - 1) / threadgroupHeight, depth: 1 ) encoder.dispatchThreadgroups(threadgroupCount, threadsPerThreadgroup: threadgroupSize) encoder.endEncoding() commandBuffer.commit() commandBuffer.waitUntilCompleted() } } ``` In the previous code example: - `print("ScanVisualizationCaptureManager updating raycast textures")` confirms that the manager is reading scan output while capture is active. - `print("ScanVisualizationCaptureViewController updating visualization")` confirms that frame updates are reaching the controller during capture. - `print("ScanVisualizationCaptureViewController displayed composited texture")` confirms that the controller produced a composited texture and sent it to `TextureView`. - `setupMetalResources()` and `compositeVisualization(from:)` provide the minimal Metal path needed to turn the raw raycast texture into the striped overlay shown on screen. At this stage, the minimal Swift example should show the overlay and update it as you move through the scene. After you finish validating the Swift example, you can remove the temporary `print(...)` debug lines. --- ## Troubleshooting This section helps you diagnose common issues where scan visualization does not appear, does not update, or does not perform as expected. It focuses on verifying the sample app setup, rendering path, and runtime behavior while testing your implementation. ### Visualization doesn't appear - Verify that `MainViewController.swift` is launching `ScanVisualizationCaptureViewController.self` while you test the minimal example. - Confirm that `ScanVisualizationCaptureViewController started` and `ScanVisualizationCaptureManager configuration called` appear in the Xcode console after you tap **Start Capture**. - Check that `visualizationView?.isHidden = false` runs when capture starts and that `TextureView` is still attached to `arView`. - Make sure `setupMetalResources()` succeeds and does not print `Failed to create Metal device`, `Failed to load scanning composite kernel`, or `Failed to create compute pipeline state`. - Ensure the app has camera permission and that the AR session is producing frames for `session(_:didUpdate:)`. ### Overlay doesn't update - Confirm that `ScanVisualizationCaptureViewController updating visualization` appears repeatedly while capture is running. - Confirm that `ScanVisualizationCaptureViewController displayed composited texture` appears after capture starts. If it does not, the controller is not yet producing or assigning a composited texture. - If the visualization view appears but stays blank, verify that `capture.updateRaycastTextures()` is returning `true` and that `rgbaTexture` is non-`nil`. - If the view flashes or appears misaligned, check that `TextureView` is pinned to the full `arView` bounds and that `visualizationView?.reset()` is only called when capture stops. ### Performance issues - Scan visualization composites textures every frame, so lower-end devices may show reduced performance while capture is active. - Reduce debug logging after validation, since repeated `print(...)` calls in the frame update path can affect performance. - Avoid running other expensive AR features at the same time unless necessary.