Tutorials
intermediate
Swift

Draw safety alerts using the Mapbox Vision SDK for iOS

Prerequisite

Familiarity with Xcode and Swift, and completion of the Mapbox Vision SDK for iOS Install and configure guide.

The Mapbox Vision SDK for iOS is a library for interpreting road scenes in real time directly on iOS devices using the device’s built-in camera. The Vision SDK detects many types of objects including cars, people, road signs, and more.

In this tutorial, you'll learn how to use the Vision Safety module from the Mapbox Vision SDK and apply it to detect overspeeding and potential collisions with cars.

Note

In the example used in this tutorial, all custom alerts have predefined images. In your own application, you can use your own images.

Getting started

Here are the resources that you need before getting started:

  • An application including Mapbox Vision SDK for iOS. Before starting this tutorial, go through the Install and configure steps in the Vision SDK for iOS documentation. This will walk you through how to install the Vision SDK and configure your application.
  • Recorded session. This tutorial is based on a recorded driving session through a city and the replay capabilities of MapboxVision. You will use VisionReplayManager for a recorded session.

    • Check our Testing and development guide to familiarize yourself with record and replay functionality.
    • You can download the recorded session used in this tutorial below.
Download sample session

To configure your application with a prerecorded session:

  1. Unzip the contents to a folder on your local machine.
  2. Go your Xcode project Info.plist and set YES for UIFileSharingEnabled thus enabling file sharing through Finder.
  3. Install the app to the device ( + R).
  4. Connect your device, choose it in Finder under Locations section, and select Files tab.
  5. Drag and drop the folder with the recorded session onto your app. Now the session is available in the Documents folder inside the app container.
  6. In the code use VisionReplayManager.create(recordPath:) method to create an instance of VisionReplayManager by providing a path to a recorded session.

Configure Vision SDK lifecycle

In this tutorial, you'll use VisionReplayManager to run a prerecorded session (which includes video and telemetry data) and find speed signs and cars in the video. The VisionReplayManager class is the main object for registering for events from the Vision SDK and controlling its delivery. For production applications or testing in a live environment, use VisionManager instead of VisionReplayManager. See the Next steps section for details.

To set up the Vision SDK:

  1. Create a VisionReplayManager instance with a recorded session path.
show hidden lines
private var visionManager: VisionReplayManager!
show hidden lines
// Documents directory path with files uploaded via Finder
let documentsPath =
NSSearchPathForDirectoriesInDomains(.documentDirectory,
.userDomainMask,
true).first!
let path = documentsPath.appending("/safety-alerts-drawing")
// create VisionReplayManager with a path to recorded session
visionManager = try? VisionReplayManager.create(recordPath: path)
show hidden lines
  1. Register its delegate.
show hidden lines
// register its delegate
visionManager.delegate = self
show hidden lines
  1. Create an instance of VisionSafetyManager configured with VisionReplayManager instance.
show hidden lines
// create VisionSafetyManager and register as its delegate to receive safety related events
visionSafetyManager = VisionSafetyManager.create(visionManager: visionManager)
show hidden lines
  1. Register its delegate to receive safety related events.
show hidden lines
// register its delegate
visionSafetyManager.delegate = self
show hidden lines
  1. Create VisionPresentationViewController and configure it with VisionReplayManagerto display camera frames.
show hidden lines
private let visionViewController = VisionPresentationViewController()
show hidden lines
// configure Vision view to display sample buffers from video source
visionViewController.set(visionManager: visionManager)
// add Vision view as a child view
addVisionView()
show hidden lines
private func addVisionView() {
addChild(visionViewController)
view.addSubview(visionViewController.view)
visionViewController.didMove(toParent: self)
}
show hidden lines
  1. Start delivering events by calling start on VisionReplayManager.
show hidden lines
override func viewWillAppear(_ animated: Bool) {
super.viewWillAppear(animated)
visionManager.start()
}
show hidden lines
  1. Stop delivering events by calling stop on VisionReplayManager.
show hidden lines
override func viewDidDisappear(_ animated: Bool) {
super.viewDidDisappear(animated)
visionManager.stop()
}
show hidden lines

Set up views to draw alerts

Overspeeding alert

To show an overspeeding alert you'll use a UIView element named alertOverspeedingView:

show hidden lines
private var alertOverspeedingView: UIView!
show hidden lines
private func addOverspeedingAlertView() {
alertOverspeedingView = UIImageView(image: UIImage(named: "alert"))
alertOverspeedingView.isHidden = true
alertOverspeedingView.translatesAutoresizingMaskIntoConstraints = false
view.addSubview(alertOverspeedingView)
NSLayoutConstraint.activate([
alertOverspeedingView.topAnchor
.constraint(equalToSystemSpacingBelow: view.safeAreaLayoutGuide.topAnchor, multiplier: 1),
view.safeAreaLayoutGuide.trailingAnchor
.constraint(equalToSystemSpacingAfter: alertOverspeedingView.trailingAnchor, multiplier: 1)
])
}
show hidden lines

Collision detections

You will draw a collision state using a custom CollisionDetectionView with a custom border color to show a bounding box around potential collision objects:

show hidden lines
// Custom UIView to draw a red bounding box
class CollisionDetectionView: UIView {
override init(frame: CGRect) {
super.init(frame: frame)
// Transparent view with a red border
backgroundColor = .clear
layer.borderWidth = 3
layer.borderColor = UIColor.red.cgColor
}
required init?(coder: NSCoder) {
fatalError("init(coder:) has not been implemented")
}
}
show hidden lines

Having a custom class also helps to track detection views in UIView's subviews. If you need to remove all CollisionDetectionView objects from superview, you can use the following code:

show hidden lines
// remove `CollisionDetectionView` objects from the view
for subview in view.subviews {
if subview.isKind(of: CollisionDetectionView.self) {
subview.removeFromSuperview()
}
}
show hidden lines

Respond to SDK events

Implement VisionManagerDelegate

Note

All delegate methods are called on a background thread, so consider dispatching execution to the DispatchQueue.main when working with UI elements or the queue of your choice for data synchronization.

You need to implement several methods from VisionManagerDelegate to obtain the necessary data.

Use visionManager(_:didUpdateVehicleState:) to get the latest vehicle state. You need to save the latest state of the vehicle:

show hidden lines
private var vehicleState: VehicleState?
show hidden lines
func visionManager(_ visionManager: VisionManagerProtocol,
didUpdateVehicleState vehicleState: VehicleState) {
// dispatch to the main queue in order to sync access to `VehicleState` instance
DispatchQueue.main.async { [weak self] in
// save the latest state of the vehicle
self?.vehicleState = vehicleState
}
}
show hidden lines

Use visionManagerDidCompleteUpdate(_:) to know the whole update iteration is completed. That means all the data that came from delegate methods is in sync. When the update is completed, you can use all the data to update UI:

show hidden lines
func visionManagerDidCompleteUpdate(_ visionManager: VisionManagerProtocol) {
// dispatch to the main queue in order to work with UIKit elements
DispatchQueue.main.async { [weak self] in
// update UI elements
self?.updateOverspeedingDrawing()
self?.updateCollisionDrawing()
}
}
show hidden lines

Implement VisionSafetyManagerDelegate

Note

All delegate methods are called on a background thread, so consider dispatching execution to the DispatchQueue.main when working with UI elements or the queue of your choice for data synchronization.

You need to implement several methods from VisionManagerSafetyDelegate.

Use visionSafetyManager(_:didUpdateRoadRestrictions:) to know when the road restrictions were updated. When VisionSafetyManagerDelegate provides you with new information about road restrictions (speed alerts, more specifically), you store it in a local variable.

Then you'll use this information in the visionManagerDidCompleteUpdate(_ visionManager: VisionManagerProtocol) method to decide if a speed limit has been exceeded:

show hidden lines
private var speedLimits: SpeedLimits?
show hidden lines
func visionSafetyManager(_ visionSafetyManager: VisionSafetyManager,
didUpdateRoadRestrictions roadRestrictions: RoadRestrictions) {
// dispatch to the main queue in order to sync access to `SpeedLimits` instance
DispatchQueue.main.async { [weak self] in
// save currenly applied speed limits
self?.speedLimits = roadRestrictions.speedLimits
}
}
show hidden lines

Use visionSafetyManager(_:didUpdateCollisions:) to get collision objects. In this method you need to filter collision objects that have type Car and store them in a local array:

show hidden lines
private var carCollisions = [CollisionObject]()
show hidden lines
func visionSafetyManager(_ visionSafetyManager: VisionSafetyManager,
didUpdateCollisions collisions: [CollisionObject]) {
// we will draw collisions with cars only, so we need to filter `CollisionObject`s
let carCollisions = collisions.filter { $0.object.detectionClass == .car }
// dispatch to the main queue in order to sync access to `[CollisionObject]` array
DispatchQueue.main.async { [weak self] in
// update current collisions state
self?.carCollisions = carCollisions
}
}
show hidden lines

Handle overspeeding

As soon as all necessary data is in sync within the visionManagerDidCompleteUpdate(_ visionManager: VisionManagerProtocol) method, you can draw an overspeeding alert:

show hidden lines
private func updateOverspeedingDrawing() {
// when update is completed all the data has the most current state
guard let vehicle = vehicleState, let limits = speedLimits else { return }
// decide whether speed limit is exceeded by comparing it with the current speed
let isOverSpeeding = vehicle.speed > limits.speedLimitRange.max
alertOverspeedingView.isHidden = !isOverSpeeding
}
show hidden lines

Handle collisions

As soon as all necessary data is in sync within the visionManagerDidCompleteUpdate(_ visionManager: VisionManagerProtocol) method, you can draw custom collision detection views.

Remove CollisionDetectionView objects from the view:

show hidden lines
// remove `CollisionDetectionView` objects from the view
for subview in view.subviews {
if subview.isKind(of: CollisionDetectionView.self) {
subview.removeFromSuperview()
}
}
show hidden lines

Iterating through the CollisionObject array, perform several steps to show the collision detection view for each item.

First, calculate absolute coordinates of the bounding box, given the relative coordinates of the rectangle around the detected object on the camera frame:

show hidden lines
let relativeBBox = carCollision.lastDetection.boundingBox
let cameraFrameSize = carCollision.lastFrame.image.size.cgSize
// calculate absolute coordinates
let bboxInCameraFrameSpace = CGRect(x: relativeBBox.origin.x * cameraFrameSize.width,
y: relativeBBox.origin.y * cameraFrameSize.height,
width: relativeBBox.size.width * cameraFrameSize.width,
height: relativeBBox.size.height * cameraFrameSize.height)
// at this stage, bbox has the coordinates in the camera frame space
// you should convert it to the view space saving the aspect ratio
show hidden lines

Convert the coordinates of the bounding box from the camera frame space to the view space saving the aspect ratio:

show hidden lines
// first, construct left-top and right-bottom coordinates of a bounding box
var leftTop = CGPoint(x: bboxInCameraFrameSpace.origin.x,
y: bboxInCameraFrameSpace.origin.y)
var rightBottom = CGPoint(x: bboxInCameraFrameSpace.maxX,
y: bboxInCameraFrameSpace.maxY)
// then convert the points from the camera frame space into the view frame space
leftTop = leftTop.convertForAspectRatioFill(from: cameraFrameSize,
to: view.bounds.size)
rightBottom = rightBottom.convertForAspectRatioFill(from: cameraFrameSize,
to: view.bounds.size)
// finally, construct a bounding box in the view frame space
let bboxInViewSpace = CGRect(x: leftTop.x,
y: leftTop.y,
width: rightBottom.x - leftTop.x,
height: rightBottom.y - leftTop.y)
show hidden lines

Having the bounding box with coordinates in the view space, draw a collision detection alert:

show hidden lines
// draw a collision detection alert
let view = CollisionDetectionView(frame: bboxInViewSpace)
self.view.addSubview(view)
show hidden lines

Final result

Here is the complete code for the demo:

SafetyAlertsViewController
import MapboxVision
import MapboxVisionSafety
import UIKit
/**
* "Safety alerts" example demonstrates how to utilize events from MapboxVisionSafetyManager
* to alert a user about exceeding allowed speed limit and potential collisions with other cars.
*/
// Custom UIView to draw a red bounding box
class CollisionDetectionView: UIView {
override init(frame: CGRect) {
super.init(frame: frame)
// Transparent view with a red border
backgroundColor = .clear
layer.borderWidth = 3
layer.borderColor = UIColor.red.cgColor
}
required init?(coder: NSCoder) {
fatalError("init(coder:) has not been implemented")
}
}
class SafetyAlertsViewController: UIViewController {
private var visionManager: VisionReplayManager!
private var visionSafetyManager: VisionSafetyManager!
private let visionViewController = VisionPresentationViewController()
private var alertOverspeedingView: UIView!
private var vehicleState: VehicleState?
private var speedLimits: SpeedLimits?
private var carCollisions = [CollisionObject]()
override func viewDidLoad() {
super.viewDidLoad()
// Documents directory path with files uploaded via Finder
let documentsPath =
NSSearchPathForDirectoriesInDomains(.documentDirectory,
.userDomainMask,
true).first!
let path = documentsPath.appending("/safety-alerts-drawing")
// create VisionReplayManager with a path to recorded session
visionManager = try? VisionReplayManager.create(recordPath: path)
// register its delegate
visionManager.delegate = self
// create VisionSafetyManager and register as its delegate to receive safety related events
visionSafetyManager = VisionSafetyManager.create(visionManager: visionManager)
// register its delegate
visionSafetyManager.delegate = self
// configure Vision view to display sample buffers from video source
visionViewController.set(visionManager: visionManager)
// add Vision view as a child view
addVisionView()
// add view to draw overspeeding alert
addOverspeedingAlertView()
}
override func viewWillAppear(_ animated: Bool) {
super.viewWillAppear(animated)
visionManager.start()
}
override func viewDidDisappear(_ animated: Bool) {
super.viewDidDisappear(animated)
visionManager.stop()
}
deinit {
// free up VisionSafetyManager's resources
visionSafetyManager.destroy()
// free up VisionManager's resources
visionManager.destroy()
}
private func addVisionView() {
addChild(visionViewController)
view.addSubview(visionViewController.view)
visionViewController.didMove(toParent: self)
}
private func addOverspeedingAlertView() {
alertOverspeedingView = UIImageView(image: UIImage(named: "alert"))
alertOverspeedingView.isHidden = true
alertOverspeedingView.translatesAutoresizingMaskIntoConstraints = false
view.addSubview(alertOverspeedingView)
NSLayoutConstraint.activate([
alertOverspeedingView.topAnchor
.constraint(equalToSystemSpacingBelow: view.safeAreaLayoutGuide.topAnchor, multiplier: 1),
view.safeAreaLayoutGuide.trailingAnchor
.constraint(equalToSystemSpacingAfter: alertOverspeedingView.trailingAnchor, multiplier: 1)
])
}
// MARK: - Handle VisionSafety events
private func updateCollisionDrawing() {
// remove `CollisionDetectionView` objects from the view
for subview in view.subviews {
if subview.isKind(of: CollisionDetectionView.self) {
subview.removeFromSuperview()
}
}
// iterate the collection of `CollisionObject`s and draw each of them
for carCollision in carCollisions {
let relativeBBox = carCollision.lastDetection.boundingBox
let cameraFrameSize = carCollision.lastFrame.image.size.cgSize
// calculate absolute coordinates
let bboxInCameraFrameSpace = CGRect(x: relativeBBox.origin.x * cameraFrameSize.width,
y: relativeBBox.origin.y * cameraFrameSize.height,
width: relativeBBox.size.width * cameraFrameSize.width,
height: relativeBBox.size.height * cameraFrameSize.height)
// at this stage, bbox has the coordinates in the camera frame space
// you should convert it to the view space saving the aspect ratio
// first, construct left-top and right-bottom coordinates of a bounding box
var leftTop = CGPoint(x: bboxInCameraFrameSpace.origin.x,
y: bboxInCameraFrameSpace.origin.y)
var rightBottom = CGPoint(x: bboxInCameraFrameSpace.maxX,
y: bboxInCameraFrameSpace.maxY)
// then convert the points from the camera frame space into the view frame space
leftTop = leftTop.convertForAspectRatioFill(from: cameraFrameSize,
to: view.bounds.size)
rightBottom = rightBottom.convertForAspectRatioFill(from: cameraFrameSize,
to: view.bounds.size)
// finally, construct a bounding box in the view frame space
let bboxInViewSpace = CGRect(x: leftTop.x,
y: leftTop.y,
width: rightBottom.x - leftTop.x,
height: rightBottom.y - leftTop.y)
// draw a collision detection alert
let view = CollisionDetectionView(frame: bboxInViewSpace)
self.view.addSubview(view)
}
}
private func updateOverspeedingDrawing() {
// when update is completed all the data has the most current state
guard let vehicle = vehicleState, let limits = speedLimits else { return }
// decide whether speed limit is exceeded by comparing it with the current speed
let isOverSpeeding = vehicle.speed > limits.speedLimitRange.max
alertOverspeedingView.isHidden = !isOverSpeeding
}
}
extension SafetyAlertsViewController: VisionManagerDelegate {
func visionManager(_ visionManager: VisionManagerProtocol,
didUpdateVehicleState vehicleState: VehicleState) {
// dispatch to the main queue in order to sync access to `VehicleState` instance
DispatchQueue.main.async { [weak self] in
// save the latest state of the vehicle
self?.vehicleState = vehicleState
}
}
func visionManagerDidCompleteUpdate(_ visionManager: VisionManagerProtocol) {
// dispatch to the main queue in order to work with UIKit elements
DispatchQueue.main.async { [weak self] in
// update UI elements
self?.updateOverspeedingDrawing()
self?.updateCollisionDrawing()
}
}
}
extension SafetyAlertsViewController: VisionSafetyManagerDelegate {
func visionSafetyManager(_ visionSafetyManager: VisionSafetyManager,
didUpdateRoadRestrictions roadRestrictions: RoadRestrictions) {
// dispatch to the main queue in order to sync access to `SpeedLimits` instance
DispatchQueue.main.async { [weak self] in
// save currenly applied speed limits
self?.speedLimits = roadRestrictions.speedLimits
}
}
func visionSafetyManager(_ visionSafetyManager: VisionSafetyManager,
didUpdateCollisions collisions: [CollisionObject]) {
// we will draw collisions with cars only, so we need to filter `CollisionObject`s
let carCollisions = collisions.filter { $0.object.detectionClass == .car }
// dispatch to the main queue in order to sync access to `[CollisionObject]` array
DispatchQueue.main.async { [weak self] in
// update current collisions state
self?.carCollisions = carCollisions
}
}
}
// This comment is here to assure the correct rendering of code snippets in a public documentation

Next steps

Use real-time data

When you're done testing, follow these steps to start working with real-time data.

  1. Change the type of visionManager var to VisionManager
  2. Create and save a CameraVideoSource instance
// create a video source obtaining buffers from camera module
cameraVideoSource = CameraVideoSource()
  1. Create VisionManager with the cameraVideoSource you created above
// create VisionManager with video source
visionManager = VisionManager.create(videoSource: cameraVideoSource!)
  1. Start CameraVideoSource along with VisionManager in viewWillAppear(_:)
  2. Stop CameraVideoSource along with VisionManager in viewDidDisappear(_:)
  3. Update drawing asserts and logic in updateCollisionDrawing and updateOverspeedingDrawing methods if it's needed
Was this page helpful?