Testing and development | Vision SDK | iOS

Testing visual detection and classification

You can point your device at a prerecorded video playing on a separate monitor to test some features that rely only on visual detection and classification.

Device requirements

You must use a physical iOS device for all development and testing with Vision-related products. You can use the camera built into the device or an external camera connected to a physical device. You cannot use iOS Simulator with Xcode.

Video requirements

You can use prerecorded videos from a dashboard-mounted camera to test some features like detections, segmentations, and sign classifications. You can use any dashboard camera video (for example, from YouTube) or record your own video to be sure to capture local traffic signs and road markings important to your application.

For the best results when recording your own videos, you should use the same in-car setup described in the Requirements section of this documentation. Some tips for recording a video:

The Vision SDK works best under good lighting conditions. Lighting conditions may impact the reliability of certain behaviors so you should collect video in any lighting conditions you’d like to test.
Plan your route ahead of time to include a diversity of driving situations that may be important to your application (roads with different lane configurations, various speed limits, encounters with pedestrians and cyclists, etc).

Development environment setup

After you have selected or recorded a video, you will need to set up your physical development environment. You will need:

A physical device with a built-in camera (or set up an external camera source following the code example)
A monitor to play the video on
A method for positioning the device to point at the video

Position the device so the camera is pointed at the video. The video should fill the entire screen of your device.

What to expect

This approach to development can help you test some features of the SDK, but will not work for testing all features since some features rely on GPS and other sensors.

Examples of features that can be tested using this setup:

Road sign classification to determine the signType for road signs that appear in the video.
Lane Detection.
Vehicle, pedestrian, and bicycle detection for the 2D screen location of objects only.

AR Navigation and Safety features cannot be tested with this setup. See the Testing with sensor data section below instead.

Testing with sensor data

Several features of the Vision SDK require GPS and other sensors including Vision AR and Vision Safety. You can test these features in the real world in a vehicle using a device. Or, you can use the session recording and replay mechanisms of the Vision SDK to debug and iterate more rapidly.

Record a session

To record a session including video, GPS, sensors, and other necessary data, you'll need to:

Prepare a Vision application and load it on your device.


let camera = CameraVideoSource()
let visionManager = VisionManager.create(videoSource: camera)

// setup AR and/or Safety if needed
let visionARManager = VisionARManager.create(visionManager: visionManager)

visionManager.start()

// will throw an exception if `visionManager` hasn't been started beforehand
try? visionManager.startRecording(to: "/path/to/sessions/1/")

// when you're ready to finish recording
visionManager.stopRecording()

// make another recording
try? visionManager.startRecording(to: "/path/to/sessions/2/")
visionManager.stopRecording()

visionManager.stop()

Set up a device in a vehicle according to the instructions in the requirements section of this documentation.
Drive and record.

Replay a session

After you've recorded a session, you can replay it using VisionReplayManager, which is a replacement of VisionManager for replay scenario.

Provide the path to the recorded session to VisionReplayManager.create(recordPath:) method:


let path = "/path/to/sessions/1/"

// will throw an exception if directory at `path` does not exist
guard let visionReplayManager = try? VisionReplayManager.create(recordPath: path) else {
    fatalError("\(path) doesn't exist")
}

// setup AR and/or Safety the same way you do with `VisionManager` instance
let visionARManager = VisionARManager.create(visionManager: visionReplayManager)

visionReplayManager.start()