Vision SDK for Android
- Real-time classification of road signs
- Semantic segmentation of the roadway
- Object detection
- Following distance detection
The Vision SDK uses highly efficient neural networks to process imagery directly on user’s mobile or embedded devices, turning any connected camera into a second set of eyes for your car. In doing so, the Vision SDK enables the following user-facing features:
- Augmented reality navigation with turn-by-turn directions
- Classification and display of regulatory and warning signs
- Object detection for vehicles, pedestrians, road signs, and traffic lights
- Semantic segmentation of the roadway into 14 different classes (other, road, road_markup, flat_non_road, sky, building, car, cycle, person, road_markings_other, curb, double_yellow, traffic_sign, traffic_light)
- Distance detection that shows spacing to lead vehicle
Developers of native navigation applications can integrate the Vision SDK into their applications to enhance navigation experiences in real time and collect data that will improve the map data made available by Mapbox.
The Vision SDK, unlike similar products, provides functionality, safety alerts, or a heads-up display on customer’s existing devices without requiring video uploads or storage on device. Most importantly, the functionality of the Vision SDK is tied into the rest of Mapbox’s ecosystem: maps, navigation, customizable data layers, and more.
The Vision SDK for Android is a framework written in Java that allows developers to interact with Mapbox Vision Core using a convenient user-facing API that handles platform-dependent tasks. The Vision SDK for Android can be used independently or with Vision AR for Android to create AR navigation experiences.
There are three components to the Vision SDK: VisionCore, VisionSDK, and VisionAR.
VisionCore is the core logic of the system, including all machine learning models; it exists as compiled library for each platform with a user-facing API.
VisionSDK is a framework written in native language (Kotlin for Android, Swift for iOS) that encapsulates core utilization and platform-dependent tasks. It calls VisionCore.
VisionAR is a native framework with dependency on the Mapbox Navigation SDK. It takes information from the specified navigation route, transfers it to VisionCore via VisionSDK, receives instructions on displaying the route, and then finally renders it on top of camera frame using native instruments.
VisionSDK requires Android 6 (API 23) and higher, with QC Snapdragon 650 // 710 // 8xx with Open CL support
Some of devices that will work with VisionSDK:
- Samsung Galaxy S8, S8+ // S9, S9+ // Note 8
- Xiaomi Mi 6 // 8
- HTC U11, U11+ // U12, U12+
- OnePlus 5 // 6
You can also check more details at Vision SDK FAQ.
Besides software and hardware requirements, use of the Vision SDK requires that the device is pointed with a view of the road. Use a dashboard or windshield mount to keep your phone oriented correctly while you drive, such as Car Phone Mount Windshield and Car Mount Holder.
The Vision SDK is in limited public beta. Contact our team to request access and receive installation instructions.
Besides installing the framework, you will need to set up the device in the vehicle. Some things to consider when choosing and setting up a mount:
- Generally, shorter length mounts will vibrate less. Mounting to your windshield or to the dashboard itself are both options.
- Place the phone near or behind where your rearview mirror. Note that your local jurisdiction may have limits on where mounts may be placed.
- Make sure the phone’s camera view is unobstructed (you will be able to tell with any of the video screens open).