Skip to main content

Draw points of interest using the Mapbox Vision SDK for Android

Additional Developer Resources
Prerequisite

Familiarity with Android Studio and Kotlin or Java. Completion of the Mapbox Vision SDK for Android Install and configure guide.

The Mapbox Vision SDK for Android is a library for interpreting road scenes in real time directly on Android devices using the device’s built-in camera. The Vision SDK detects many types of objects including cars, people, road signs, and more.

In this tutorial, you'll learn how to use geo-to-world and world-to-screen coordinate transformations available in the Mapbox Vision SDK and apply it to rendering points of interest (POIs) on the screen.

Note

In the example used in this tutorial, all POIs have predefined images and hardcoded locations. In your own application, you can use your own POI data or data from other Mapbox products.

Getting started​

Here are the resources that you need before getting started:

  • An application including Mapbox Vision SDK for Android. Before starting this tutorial, go through the Install and configure steps in the Vision SDK for Android documentation. This will walk you through how to install the Vision SDK and configure your application.
  • Recorded session This tutorial is based on a recorded driving session through a city and replay capabilities of MapboxVision. You may use VisionManager and VisionReplayManager interchangeably for live and recorded session respectively.
    • Check our Testing and development guide to familiarize yourself with record and replay functionality.
    • You can download the recorded session used in this tutorial below.
arrow-downDownload sample session

Coordinate systems​

The Vision SDK uses three coordinate systems: frame, world, and geo. You can translate your coordinates from one system to another with help of the VisionManager and VisionReplayManager methods geoToWorld(), worldToPixel() and inverse ones worldToGeo(), pixelToWorld().

Note

The accuracy of the transformation functions is highly dependent on camera calibration so it's recommended to use them when camera.getCalibrationProgress() value is equal to 1.

Frame coordinates​

In the Mapbox Vision SDK for Android, frame coordinates are represented by a PixelCoordinate class.

This coordinate system represents the position of an object relative to the frame received from the camera. The origin is the left top corner of the frame. The position of an object is an x, y pair where x is the horizontal distance from the origin to the object in pixels, and y is the vertical distance from the origin to the object in pixels.

Note

This system can be also referred to as the screen coordinate system. Dimensions of the "screen" are defined by the dimensions of the frame received from the camera or other video source.

World coordinates​

In the Mapbox Vision SDK for Android, the world coordinate is represented by the WorldCoordinate class.

This coordinate system represents the position of an object relative to the device camera in the physical world. The origin of the system is a point projected from the camera to a road plane. The coordinate of the object is a triplet x, y, z where x is a distance to the object in front of the origin, y is a distance on the left of the origin, and z is a distance above the origin. Distance in the world coordinate system is expressed in meters.

Geographic coordinate​

In the Mapbox Vision SDK for Android, the geographic coordinate is represented by the GeoCoordinate class.

This coordinate system is used to locate an object's geographic position as it would appear on a map. Each point is specified using longitude, latitude pair.

Longitude ranges from -180 to 180 degrees, where 0 is the Greenwich meridian, the positive direction (+) is to the East, and the negative direction (-) is to the West.

Latitude ranges from -90 to +90 degrees, where 0 is the equator, the positive direction (+) is to the North, and the negative direction (-) is to the South.

Configure layout​

You'll use three different views in your layout file: VisionView, ImageView, and TextView.

VisionView: Use VisionView to render the image that was produced by Vision SDK.

ImageView: Use a regular Android ImageView at the top of the VisionView for drawing POIs in the FrameLayout. A custom Bitmap with the size of camera or video frame will be used to draw POIs and will be placed into ImageView.

Note

Using android:scaleType="centerCrop" is important because the frame's aspect ratio coming from Vision SDK likely does not match the aspect ratio of actual device.

TextView: Use TextView to show the Vision SDK's camera calibration progress.

Note

This isn't related to POIs, but is useful to see the actual video cutoff when calibration is finished and you can safely convert POI geo coordinates to world coordinates.

<FrameLayout
        xmlns:android="http://schemas.android.com/apk/res/android"
        xmlns:app="http://schemas.android.com/apk/res-auto"
        android:layout_width="match_parent"
        android:layout_height="match_parent">

    <com.mapbox.vision.view.VisionView
        android:id="@+id/vision_view"
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        app:visualization_mode="clear" />

<ImageView
        android:id="@+id/poi_view"
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        android:scaleType="centerCrop"
        android:background="@android:color/transparent"/>

<TextView
        android:id="@+id/camera_calibration_text"
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        android:layout_gravity="top|start"
        android:layout_margin="10dp"
        android:textColor="@android:color/black"
        android:textSize="18sp"
        android:background="@android:color/transparent" />

</FrameLayout>

Set up the recorded session​

To configure your application with a prerecorded session (see Getting started to download the sample session used in this tutorial):

  1. Unzip the contents to a folder on your local machine.
  2. Push all unpacked files to device to preferable folder.
  3. Set up the session's path from device to variable SESSION_PATH for VisionReplayManager.

Start the Vision SDK​

In this tutorial, you'll use VisionReplayManager to run a prerecorded session (which includes video and telemetry data) and find POIs in the video. For production applications or testing in a live environment, use VisionManager instead of VisionReplayManager.

Inside the Activity.onStart() callback:

  1. Check that all permissions are granted and that you have not already started the Vision SDK.
  2. Create a VisionReplayManager instance with the SESSION_PATH for your prerecorded session.
  3. Set up VisionEventsListener.
  4. Start delivering events from the VisionReplayManager.
@Override
public void onStart() {
    super.onStart();
    startVisionManager();
}

private void startVisionManager() {
if (allPermissionsGranted() && !visionReplayManagerWasInit && visionView != null) {
VisionReplayManager.create(SESSION_PATH);
VisionReplayManager.setVisionEventsListener(visionEventsListener);
visionView.setVisionManager(VisionReplayManager.INSTANCE);
VisionReplayManager.start();
visionReplayManagerWasInit = true;
}
}

You should also stop the Vision SDK on Activity.onStop().

@Override
public void onStop() {
    super.onStop();
    stopVisionManager();
}

private void stopVisionManager() {
if (visionReplayManagerWasInit) {
VisionReplayManager.stop();
VisionReplayManager.destroy();
visionReplayManagerWasInit = false;
}
}

Compile and run your code. Make sure video from SESSION_PATH is played on the screen after starting your application.

Calculate POI positions​

Next, you'll prepare POI data for drawing on the screen. To draw a POI label on the screen, you need to take the geographic coordinate (longitude, latitude) of the POI and make two coordinate transformations (from geographic coordinates to world coordinates and from world coordinates to screen coordinates).

Specify a list of POIs​

You need to provide a list of POIs including a geographic location for each. Start by defining a new POI data class. Each POI will require a longitude, latitude, and an image (Bitmap) to display as a label for the POI.

public static class POI {
    @NonNull
    private final Bitmap bitmap;
    private final double longitude;
    private final double latitude;

    POI(double longitude, double latitude, @NonNull Bitmap bitmap) {
        this.longitude = longitude;
        this.latitude = latitude;
        this.bitmap = bitmap;
    }

    double getLongitude() { return longitude; }

    double getLatitude() { return latitude; }

    @NonNull
    Bitmap getBitmap() { return bitmap; }

}

This example uses a list of two hardcoded POIs defined in the providePOIList() method. In your application, you will need to generate your own list of POIs including longitude, latitude, and an image (Bitmap) for each.

Here are the sample POIs included in this tutorial:

private List<POI> providePOIList() {
    POI poiGasStation = new POI(27.674764394760132, 53.9405971055192, getBitmapFromAssets("ic_gas_station.png"));
    POI poiCarWash = new POI(27.675944566726685, 53.94105180084251, getBitmapFromAssets("ic_car_wash.png"));
    return Arrays.asList(poiGasStation, poiCarWash);
}
Note

In your own application, you could use your own POI data, data from the Mapbox Geocoding API, or data from the Mapbox Streets tileset via the Mapbox Tilequery API.

Calculate POI WorldCoordinate and distance from vehicle​

Next, calculate the distance between the vehicle and each POI. This will allow you to only draw POIs within a certain distance of the vehicle and style the POI labels based on how far they are from the vehicle.

Use the geographic coordinates (longitude, latitude) of both the vehicle (currentVehicleGeoCoordinate) and each POI to calculate the distance between them. Then, convert the POI's geographic coordinate (longitude, latitude) to WorldCoordinate with the origin at the camera position. This will allow you to calculate some physical parameters in later steps.

private List<POIState> calculatePOIStateListRegardingVehicle(@NonNull GeoCoordinate currentVehicleGeoCoordinate) {
    final List<POIState> poiStateList = new ArrayList<>();
    final LatLng currentVehicleLatLng = new LatLng(currentVehicleGeoCoordinate.getLatitude(),
                                                    currentVehicleGeoCoordinate.getLongitude());
    for (POI poi: poiList) {
        final LatLng latLng = new LatLng(poi.getLatitude(), poi.getLongitude());
        final GeoCoordinate geoCoordinate = new GeoCoordinate(latLng.getLatitude(),
                                                               latLng.getLongitude());
        final WorldCoordinate worldCoordinate = VisionReplayManager.geoToWorld(geoCoordinate);
        if (worldCoordinate == null) {
            continue;
        }
        int distanceToVehicle = (int)latLng.distanceTo(currentVehicleLatLng);
        POIState poiState = new POIState(poi, distanceToVehicle, worldCoordinate);
        poiStateList.add(poiState);
    }
    return poiStateList;
}

Results will be saved to POIState list, which describes current state of each POI with regard to the vehicle.

private static class POIState {

    @NonNull
    private POI poi;
    @NonNull
    private WorldCoordinate worldCoordinate;
    private int distanceToVehicle;

    POIState(@NonNull POI poi, int distanceToVehicle, @NonNull WorldCoordinate worldCoordinate) {
        this.poi = poi;
        this.distanceToVehicle = distanceToVehicle;
        this.worldCoordinate = worldCoordinate;
    }

    @NonNull
    POI getPOI() { return poi; }

    @NonNull
    WorldCoordinate getWorldCoordinate() { return worldCoordinate; }

    int getDistanceToVehicle() { return distanceToVehicle; }

}

Filter POI by distance to vehicle​

Then, use the distances in the last step to filter the complete list of POIs to only POIs within a certain distance (less than MIN_DISTANCE_METERS_FOR_DRAW_LABEL) from the vehicle. You will only draw these POIs.

@NonNull
private List<POIState> filterPOIByDistance(List<POIState> poiStateList) {
    final List<POIState> result = new ArrayList<>();
    for (POIState poiState: poiStateList) {
        double x = poiState.worldCoordinate.getX();
        if (x > 0 && poiState.getDistanceToVehicle() < MIN_DISTANCE_METERS_FOR_DRAW_LABEL) {
            result.add(poiState);
        }
    }
    return result;
}

Update POI state and prepare for drawing​

Now that you've calculated all values you need, create a new function called updatePOIStateAndDraw() that will use:

  • calculatePOIStateListRegardingVehicle() to get the WorldCoordinates and the physical distance in meters from the POI to the vehicle.
  • filterPOIByDistance() to filter and select POIs that is less than a specified distance from the vehicle.
private fun updatePOIStateAndDraw(newVehicleGeoLocation: GeoLocation) {
    if (poiList.isEmpty()) {
        return
    }
    val poiStateList = calculatePOIStateListRegardingVehicle(newVehicleGeoLocation.geoCoordinate)
    val poiStateListToShow = filterPOIByDistance(poiStateList)
    if (poiStateListToShow.isEmpty()) {
        return
    }
    val poiDrawDataList = preparePOIDrawData(poiStateListToShow)
    updateBitmapByPOIList(bitmapCameraFrame, poiDrawDataList)
    runOnUiThread {
        poi_view.setImageBitmap(bitmapCameraFrame)
    }
}

Keep track of the vehicle position​

Then, call updatePOIStateAndDraw() every time the vehicle's position updates using VisionEventsListener.onVehicleStateUpdated. This will continuously update the geographic location of the vehicle and allow you to update the position of POIs on the screen while the vehicle is moving.

// Callback is invoked on background thread
public void onVehicleStateUpdated(@NonNull VehicleState vehicleState) {
    if (cameraCalibrated) {
        updatePOIStateAndDraw(vehicleState.getGeoLocation());
    }
}

Calculate POI frame coordinates for drawing​

Finally, convert world coordinates to frame coordinates so you can tell the application where on the screen to draw the image for each POI.

You already have the WorldCoordinate for each POI with actual X and Y values (Z always comes as 0 from callback). This value is point on the ground in the physical world.

Physical sizes like LABEL_SIZE_METERS and LABEL_ABOVE_GROUND_METERS are required to calculate POI left-top and right-bottom WorldCoordinate. Then you should convert these two points to points with frame coordinates with help of the VisionReplayManager.worldToPixel.

private Rect calculatePOIScreenRect(@NonNull WorldCoordinate poiWorldCoordinate) {

    // Calculate left top coordinate of POI in real world using POI world coordinate
    final WorldCoordinate worldLeftTop = poiWorldCoordinate.copy(
        poiWorldCoordinate.getX(),
        poiWorldCoordinate.getY() + (float) LABEL_SIZE_METERS / 2,
        poiWorldCoordinate.getZ() + LABEL_ABOVE_GROUND_METERS + LABEL_SIZE_METERS
    );

    // Calculate right bottom coordinate of POI in real world using POI world coordinate
    final WorldCoordinate worldRightBottom = poiWorldCoordinate.copy(
        poiWorldCoordinate.getX(),
        poiWorldCoordinate.getY() - (float) LABEL_SIZE_METERS / 2,
        poiWorldCoordinate.getZ() + LABEL_ABOVE_GROUND_METERS
    );

    Rect poiBitmapRect = new Rect(0, 0, 0, 0);

    // Calculate POI left top position on camera frame using real world coordinates
    PixelCoordinate pixelLeftTop = VisionReplayManager.worldToPixel(worldLeftTop);
    if (pixelLeftTop == null) {
        return poiBitmapRect;
    }
    poiBitmapRect.left = pixelLeftTop.getX();
    poiBitmapRect.top = pixelLeftTop.getY();

    // Calculate POI right bottom position on camera frame from real word coordinates
    PixelCoordinate pixelRightTop = VisionReplayManager.worldToPixel(worldRightBottom);
    if (pixelRightTop == null) {
        return poiBitmapRect;
    }
    poiBitmapRect.right = pixelRightTop.getX();
    poiBitmapRect.bottom = pixelRightTop.getY();
    return poiBitmapRect;

}

Draw POIs​

Now that you have all the required information about each POI, you can draw them on the screen.

Prepare Canvas​

You'll use Android's Canvas to draw a label on the screen when a POI comes into view. In VisionEventsListener.onCameraUpdated:

  1. Check if camera calibration is complete.
  2. Create a Bitmap using the frame size from Camera.
  3. Create a Canvas using the resulting Bitmap.
Note
Calibration progress is tracked and shown on screen.
// Callback is invoked on background thread
public void onCameraUpdated(@NonNull Camera camera) {
	if (camera.getCalibrationProgress() == 1.0f && !cameraCalibrated) {
	    cameraCalibrated = true;
       bitmapCameraFrame = Bitmap.createBitmap(camera.getFrameWidth(),
                                               camera.getFrameHeight(),
                                               Bitmap.Config.ARGB_8888);
       canvasCameraFrame = new Canvas(bitmapCameraFrame);
       runOnUiThread (() -> {
       	if (cameraCalibrationView != null) {
              cameraCalibrationView.setVisibility(GONE);
          }
       });
   }
   else {
       runOnUiThread ( () -> {
           if (cameraCalibrationView != null) {
               String text = getString(R.string.camera_calibration_progress, (int)(camera.getCalibrationProgress() * 100));
               cameraCalibrationView.setText(text);
           }
       });
   }
}

Set POI label opacity by distance to vehicle​

Increase the opacity of POI labels as POIs get closer to the vehicle by calculating the value of the ImageView alpha property (1 to 0) based on the distance between the vehicle and the POI.

private int calculatePOILabelAlpha(@NonNull POIState poiState) {
    int minDistance = Math.min(MIN_DISTANCE_METERS_FOR_DRAW_LABEL - poiState.distanceToVehicle, DISTANCE_FOR_ALPHA_APPEAR_METERS);
    return (int)((minDistance / (float)DISTANCE_FOR_ALPHA_APPEAR_METERS) * 255);
}

Aggregate POI parameters for drawing​

To prepare a to draw POI, combine previously calculated parameters including:

  1. Top-left and bottom-right POI frame coordinates in the Rect.
  2. Alpha for POI transparency.
  3. POI label.
@NonNull
private List<POIDrawData> preparePOIDrawData(List<POIState> poiStateList){

    // Prepare bounding rect for POI in mobile screen coordinates
    final List<POIDrawData> poiDrawDataList = new ArrayList<>();
    for (POIState poiState: poiStateList) {
        final Rect poiBitmapRect = calculatePOIScreenRect(poiState.getWorldCoordinate());
        final int poiLabelAlpha = calculatePOILabelAlpha(poiState);
        final POIDrawData poiDrawData = new POIDrawData(poiState.getPOI().getBitmap(), poiBitmapRect, poiLabelAlpha);
        poiDrawDataList.add(poiDrawData);
     }
     return poiDrawDataList;

}

Additional data class POIDrawData is used to aggregate this POI properties.

private static class POIDrawData {

    @NonNull
    private Bitmap poiBitmap;
    private Rect poiBitmapRect;
    private int poiBitmapAlpha;

    POIDrawData(@NonNull Bitmap poiBitmap, Rect poiBitmapRect, int poiBitmapAlpha) {
        this.poiBitmap = poiBitmap;
        this.poiBitmapRect = poiBitmapRect;
        this.poiBitmapAlpha = poiBitmapAlpha;
    }

    @NonNull
    Bitmap getPOIBitmap() {
        return poiBitmap;
    }

    Rect getPOIBitmapRect() {
        return poiBitmapRect;
    }

    int getPOIBitmapAlpha() {
        return poiBitmapAlpha;
    }

}

Draw POIs on the Bitmap​

Draw POIs on the Bitmap object created in VisionEventsListener.onCameraUpdated. Use Canvas to clean previous state of the global Bitmap and draw the current List of POIDrawData objects.

private void updateBitmapByPOIList(@NonNull List<POIDrawData> poiDrawDataList) {
	canvasCameraFrame.drawColor(Color.TRANSPARENT, PorterDuff.Mode.CLEAR);
   for (POIDrawData drawPoi : poiDrawDataList) {
       paint.setAlpha(drawPoi.getPOIBitmapAlpha());
       canvasCameraFrame.drawBitmap(drawPoi.getPOIBitmap(), null, drawPoi.getPOIBitmapRect(), paint);
   }
}

Set Bitmap with POIs to ImageView​

The Bitmap with all POIs is ready. But it was prepared on background thread in the VisionEventsListener.onVehicleStateUpdated callback. That means you should switch to the main thread and set up Bitmap with drawn POIs to ImageView.

runOnUiThread(() -> {
    if (poiView != null) {
        poiView.setImageBitmap(bitmapCameraFrame);
    }
});

Final product​

POIActivity.java
github View on GitHub
package com.mapbox.vision.examples;

import android.content.res.AssetManager;
import android.graphics.Bitmap;
import android.graphics.BitmapFactory;
import android.graphics.Canvas;
import android.graphics.Color;
import android.graphics.Paint;
import android.graphics.PorterDuff;
import android.graphics.Rect;
import android.os.Environment;
import android.widget.ImageView;
import android.widget.TextView;

import androidx.annotation.NonNull;
import androidx.annotation.Nullable;

import com.mapbox.mapboxsdk.geometry.LatLng;
import com.mapbox.vision.VisionReplayManager;
import com.mapbox.vision.mobile.core.interfaces.VisionEventsListener;
import com.mapbox.vision.mobile.core.models.AuthorizationStatus;
import com.mapbox.vision.mobile.core.models.Camera;
import com.mapbox.vision.mobile.core.models.Country;
import com.mapbox.vision.mobile.core.models.FrameSegmentation;
import com.mapbox.vision.mobile.core.models.classification.FrameSignClassifications;
import com.mapbox.vision.mobile.core.models.detection.FrameDetections;
import com.mapbox.vision.mobile.core.models.frame.PixelCoordinate;
import com.mapbox.vision.mobile.core.models.position.GeoCoordinate;
import com.mapbox.vision.mobile.core.models.position.GeoLocation;
import com.mapbox.vision.mobile.core.models.position.VehicleState;
import com.mapbox.vision.mobile.core.models.road.RoadDescription;
import com.mapbox.vision.mobile.core.models.world.WorldCoordinate;
import com.mapbox.vision.mobile.core.models.world.WorldDescription;
import com.mapbox.vision.view.VisionView;

import java.io.InputStream;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;

import static android.view.View.GONE;

public class POIActivity extends BaseActivity {

    // POI will start to appear at this distance, starting with transparent and appearing gradually
    private static final int MIN_DISTANCE_METERS_FOR_DRAW_LABEL = 400;

    // POI will start to appear from transparent to non-transparent during this first meters of showing distance
    private static final int DISTANCE_FOR_ALPHA_APPEAR_METERS = 150;
    private static final int LABEL_SIZE_METERS = 8;
    private static final int LABEL_ABOVE_GROUND_METERS = 4;
    // Download session from tutorial and push to device
    private static final String SESSION_PATH = Environment.getExternalStorageDirectory().getAbsolutePath() + "/session";

    private List<POI> poiList = new ArrayList<>();
    private boolean visionReplayManagerWasInit = false;

    @Nullable
    private ImageView poiView = null;
    @Nullable
    private VisionView visionView = null;
    @Nullable
    private TextView cameraCalibrationView = null;

    private VisionEventsListener visionEventsListener = new VisionEventsListener() {

        private boolean cameraCalibrated = false;
        private Paint paint = new Paint();
        private Canvas canvasCameraFrame = new Canvas();
        private Bitmap bitmapCameraFrame = Bitmap.createBitmap(1, 1, Bitmap.Config.ARGB_8888);

        public void onAuthorizationStatusUpdated(@NonNull AuthorizationStatus authorizationStatus) { }

        public void onFrameSegmentationUpdated(@NonNull FrameSegmentation frameSegmentation) { }

        public void onFrameDetectionsUpdated(@NonNull FrameDetections frameDetections) { }

        public void onFrameSignClassificationsUpdated(@NonNull FrameSignClassifications frameSignClassifications) { }

        public void onRoadDescriptionUpdated(@NonNull RoadDescription roadDescription) { }

        public void onWorldDescriptionUpdated(@NonNull WorldDescription worldDescription) { }

        public void onCameraUpdated(@NonNull Camera camera) {
            if (camera.getCalibrationProgress() == 1.0f && !cameraCalibrated) {
                cameraCalibrated = true;
                bitmapCameraFrame = Bitmap.createBitmap(camera.getFrameWidth(), camera.getFrameHeight(), Bitmap.Config.ARGB_8888);
                canvasCameraFrame = new Canvas(bitmapCameraFrame);
                runOnUiThread (() -> {
                    if (cameraCalibrationView != null) {
                        cameraCalibrationView.setVisibility(GONE);
                    }
                });
            } else {
                runOnUiThread ( () -> {
                    if (cameraCalibrationView != null) {
                        String text = getString(R.string.camera_calibration_progress, (int)(camera.getCalibrationProgress() * 100));
                        cameraCalibrationView.setText(text);
                    }
                });
            }
        }

        public void onCountryUpdated(@NonNull Country country) { }

        public void onUpdateCompleted() { }

        public void onVehicleStateUpdated(@NonNull VehicleState vehicleState) {
            if (cameraCalibrated) {
                updatePOIStateAndDraw(vehicleState.getGeoLocation());
            }
        }

        private void updatePOIStateAndDraw(GeoLocation newVehicleGeoLocation) {
            if (poiList.isEmpty()) {
                return;
            }

            List<POIState> poiStateList = calculatePOIStateListRegardingVehicle(newVehicleGeoLocation.getGeoCoordinate());
            poiStateList = filterPOIByDistance(poiStateList);
            if (poiStateList.isEmpty()) {
                return;
            }
            final List<POIDrawData> poiDrawDataList = preparePOIDrawData(poiStateList);
            updateBitmapByPOIList(poiDrawDataList);
            runOnUiThread(() -> {
                if (poiView != null) {
                    poiView.setImageBitmap(bitmapCameraFrame);
                }
            });
        }

        // Calculate POI distance to vehicle and WorldCoordinates regarding the vehicle
        private List<POIState> calculatePOIStateListRegardingVehicle(@NonNull GeoCoordinate currentVehicleGeoCoordinate) {
            final List<POIState> poiStateList = new ArrayList<>();
            final LatLng currentVehicleLatLng = new LatLng(currentVehicleGeoCoordinate.getLatitude(), currentVehicleGeoCoordinate.getLongitude());
            for (POI poi: poiList) {
                final LatLng latLng = new LatLng(poi.getLatitude(), poi.getLongitude());
                final GeoCoordinate geoCoordinate = new GeoCoordinate(latLng.getLatitude(), latLng.getLongitude());
                final WorldCoordinate worldCoordinate = VisionReplayManager.geoToWorld(geoCoordinate);
                if (worldCoordinate == null) {
                    continue;
                }
                int distanceToVehicle = (int)latLng.distanceTo(currentVehicleLatLng);
                POIState poiState = new POIState(poi, distanceToVehicle, worldCoordinate);
                poiStateList.add(poiState);
            }
            return poiStateList;
        }

        // Show only POI which is close enough and behind the car
        @NonNull
        private List<POIState> filterPOIByDistance(List<POIState> poiStateList) {
            final List<POIState> result = new ArrayList<>();
            for (POIState poiState: poiStateList) {
                double x = poiState.worldCoordinate.getX();
                if (x > 0 && poiState.getDistanceToVehicle() < MIN_DISTANCE_METERS_FOR_DRAW_LABEL) {
                    result.add(poiState);
                }
            }
            return result;
        }

        @NonNull
        private List<POIDrawData> preparePOIDrawData(List<POIState> poiStateList){

            // Prepare bounding rect for POI in mobile screen coordinates
            final List<POIDrawData> poiDrawDataList = new ArrayList<>();
            for (POIState poiState: poiStateList) {
                final Rect poiBitmapRect = calculatePOIScreenRect(poiState.getWorldCoordinate());
                final int poiLabelAlpha = calculatePOILabelAlpha(poiState);
                final POIDrawData poiDrawData = new POIDrawData(poiState.getPOI().getBitmap(), poiBitmapRect, poiLabelAlpha);
                poiDrawDataList.add(poiDrawData);
            }
            return poiDrawDataList;
        }

        private Rect calculatePOIScreenRect(@NonNull WorldCoordinate poiWorldCoordinate) {

            // Calculate left top coordinate of POI in real world using POI world coordinate
            final WorldCoordinate worldLeftTop = poiWorldCoordinate.copy(
                    poiWorldCoordinate.getX(),
                    poiWorldCoordinate.getY() + (float) LABEL_SIZE_METERS / 2,
                    poiWorldCoordinate.getZ() + LABEL_ABOVE_GROUND_METERS + LABEL_SIZE_METERS
            );

            // Calculate right bottom coordinate of POI in real world using POI world coordinate
            final WorldCoordinate worldRightBottom = poiWorldCoordinate.copy(
                    poiWorldCoordinate.getX(),
                    poiWorldCoordinate.getY() - (float) LABEL_SIZE_METERS / 2,
                    poiWorldCoordinate.getZ() + LABEL_ABOVE_GROUND_METERS
            );

            Rect poiBitmapRect = new Rect(0, 0, 0, 0);

            // Calculate POI left top position on camera frame from real word coordinates
            PixelCoordinate pixelLeftTop = VisionReplayManager.worldToPixel(worldLeftTop);
            if (pixelLeftTop == null) {
                return poiBitmapRect;
            }
            poiBitmapRect.left = pixelLeftTop.getX();
            poiBitmapRect.top = pixelLeftTop.getY();

            // Calculate POI right bottom position on camera frame from real word coordinates
            PixelCoordinate pixelRightTop = VisionReplayManager.worldToPixel(worldRightBottom);
            if (pixelRightTop == null) {
                return poiBitmapRect;
            }
            poiBitmapRect.right = pixelRightTop.getX();
            poiBitmapRect.bottom = pixelRightTop.getY();

            return poiBitmapRect;
        }

        private int calculatePOILabelAlpha(@NonNull POIState poiState) {
            int minDistance = Math.min(MIN_DISTANCE_METERS_FOR_DRAW_LABEL - poiState.distanceToVehicle, DISTANCE_FOR_ALPHA_APPEAR_METERS);
            return (int)((minDistance / (float)DISTANCE_FOR_ALPHA_APPEAR_METERS) * 255);
        }

        private void updateBitmapByPOIList(@NonNull List<POIDrawData> poiDrawDataList) {
            canvasCameraFrame.drawColor(Color.TRANSPARENT, PorterDuff.Mode.CLEAR);
            for (POIDrawData drawPoi : poiDrawDataList) {
                paint.setAlpha(drawPoi.getPOIBitmapAlpha());
                canvasCameraFrame.drawBitmap(drawPoi.getPOIBitmap(), null, drawPoi.getPOIBitmapRect(), paint);
            }
        }
    };

    @Override
    public void onPermissionsGranted() {
        startVisionManager();
    }

    @Override
    public void initViews() {
        setContentView(R.layout.activity_poi);
        poiView = findViewById(R.id.poi_view);
        visionView = findViewById(R.id.vision_view);
        cameraCalibrationView = findViewById(R.id.camera_calibration_text);
        poiList = providePOIList();
    }

    @Override
    public void onStart() {
        super.onStart();
        startVisionManager();
    }

    @Override
    public void onStop() {
        super.onStop();
        stopVisionManager();
    }

    @Override
    public void onResume() {
        super.onResume();
        if (visionView != null) {
            visionView.onResume();
        }
    }

    @Override
    public void onPause() {
        super.onPause();
        if (visionView != null) {
            visionView.onPause();
        }
    }

    private void startVisionManager() {
        if (allPermissionsGranted() && !visionReplayManagerWasInit && visionView != null) {
            VisionReplayManager.create(SESSION_PATH);
            VisionReplayManager.setVisionEventsListener(visionEventsListener);
            visionView.setVisionManager(VisionReplayManager.INSTANCE);
            VisionReplayManager.start();
            visionReplayManagerWasInit = true;
        }
    }

    private void stopVisionManager() {
        if (visionReplayManagerWasInit) {
            VisionReplayManager.stop();
            VisionReplayManager.destroy();
            visionReplayManagerWasInit = false;
        }
    }

    private List<POI> providePOIList() {
        POI poiGasStation = new POI(27.674764394760132, 53.9405971055192, getBitmapFromAssets("ic_gas_station.png"));
        POI poiCarWash = new POI(27.675944566726685, 53.94105180084251, getBitmapFromAssets("ic_car_wash.png"));
        return Arrays.asList(poiGasStation, poiCarWash);
    }

    private Bitmap getBitmapFromAssets(@NonNull String asset) {
        final AssetManager assetManager = this.getAssets();
        try {
            final InputStream stream = assetManager.open(asset);
            return BitmapFactory.decodeStream(stream);
        } catch (Exception ex) {
            throw new RuntimeException(ex);
        }
    }

    public static class POI {

        @NonNull
        private final Bitmap bitmap;
        private final double longitude;
        private final double latitude;

        POI(double longitude, double latitude, @NonNull Bitmap bitmap) {
            this.longitude = longitude;
            this.latitude = latitude;
            this.bitmap = bitmap;
        }

        double getLongitude() {
            return longitude;
        }

        double getLatitude() {
            return latitude;
        }

        @NonNull
        Bitmap getBitmap() {
            return bitmap;
        }
    }

    private static class POIDrawData {

        @NonNull
        private Bitmap poiBitmap;
        private Rect poiBitmapRect;
        private int poiBitmapAlpha;

        POIDrawData(@NonNull Bitmap poiBitmap, Rect poiBitmapRect, int poiBitmapAlpha) {
            this.poiBitmap = poiBitmap;
            this.poiBitmapRect = poiBitmapRect;
            this.poiBitmapAlpha = poiBitmapAlpha;
        }

        @NonNull
        Bitmap getPOIBitmap() {
            return poiBitmap;
        }

        Rect getPOIBitmapRect() {
            return poiBitmapRect;
        }

        int getPOIBitmapAlpha() {
            return poiBitmapAlpha;
        }
    }

    private static class POIState {

        @NonNull
        private POI poi;
        @NonNull
        private WorldCoordinate worldCoordinate;
        private int distanceToVehicle;

        POIState(@NonNull POI poi, int distanceToVehicle, @NonNull WorldCoordinate worldCoordinate) {
            this.poi = poi;
            this.distanceToVehicle = distanceToVehicle;
            this.worldCoordinate = worldCoordinate;
        }

        @NonNull
        POI getPOI() {
            return poi;
        }

        @NonNull
        WorldCoordinate getWorldCoordinate() {
            return worldCoordinate;
        }

        int getDistanceToVehicle() {
            return distanceToVehicle;
        }
    }
}

Next steps​

Use real-time data​

When you're done testing, follow these steps to start working with real-time data.

  1. Replace all VisionReplayManager references with VisionManager.
  2. Use default VisionManager.create() without parameters to get a video stream from the device's camera.
  3. Provide your own POI GeoCoordinates.
Additional Developer Resources
Was this page helpful?