Atlas for Docker Compose

Introduction

Atlas supports Docker Compose, a tool for defining and running multi-container Docker applications. Atlas for Docker Compose lets you run Atlas on a single host. If you need scaling, fault tolerance, high availability, or other advanced features, you may be interested in Atlas for Kubernetes.

Atlas installer

The Atlas for Docker Compose installer is a command line interface (CLI) utility used to download and set up Atlas on a host server. The installer has multiple commands that can support different deployment environments. The installer is available for both macOS and Linux.

Connectivity requirements

The Atlas installer has been specifically designed for situations in which the host server has no internet connectivity. You must run the installer's install or download command from a connected server to download required dependencies. Once the dependencies have been downloaded, they can be copied over to the host to complete the set up process.

Architecture

The diagram below shows how Atlas operates with external and internal services via ports, protocols, and data flows. Atlas listens on a single port and protocol for data requests. Internally, Atlas runs on a single host with routing, Mapbox APIs, data, and caching running in Docker containers.

Dependencies

To download and set up Atlas for Docker Compose, you need to download several dependencies first:

• Docker
• Docker Compose
• wget
• unzip
• coreutils (macOS only for v1.6.0 and later of the Atlas installer)
• pv (optional for v1.6.0 and later of the Atlas installer)
• OpenSSL (optional)

wget, pv, OpenSSL, and unzip are all dependencies of the Atlas installer. wget is used as the download manager for the installer's download command. OpenSSL is used by Atlas to create a self-signed certificate, and is optional. pv provides user feedback during file operations. Docker and Docker Compose are used as the primary distribution and containerization strategy for Atlas. unzip extracts the installer download package. Mac users will need to install coreutils, which contains the realpath utility used by v1.6.0 and later of the Atlas installer for obtaining the pathname of the installation directory.

The installation process for these dependencies will vary depending on which operating system you are using.

You may need to reboot after installing the dependencies.

Configuring Docker

The Atlas deployment will require about 25 GB of disk space within your Docker context. Depending on how your Docker storage is configured you may have increase the disk space available to Docker. Get details about your configuration with docker info and visit the Docker storage driver documentation. We recommend configuring overlay2 and explicitly setting your data-root to live on NVMe SSD storage:

{
  "data-root": "/nvme/docker",
  "storage-driver": "overlay2"
}

Supported operating systems

You can run Atlas for Docker Compose on the following operating systems:

• macOS^*
• Ubuntu 16.04
• Ubuntu 18.04
• Ubuntu 20.04
• CentOS 7
• Red Hat Enterprise Linux 7
• Red Hat Enterprise Linux 8

* Recommended for development only

macOS

1. Download and install Docker and Docker Compose for macOS from the Docker website.
2. Use Homebrew to install Wget, unzip, pv, coreutils, and, optionally, OpenSSL:

$ brew update
$ brew install wget unzip pv coreutils openssl

3. Docker needs at least 4 GB of RAM to run on macOS. Open the Docker control panel. Click Preferences > Advanced. Adjust the memory by sliding the tab to 4 GB.

Ubuntu

# Update the package database
$ sudo apt-get update

# Install Docker and Atlas dependencies
$ sudo apt-get install -y wget openssl unzip pv

# Add the Docker repository to APT sources
$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
$ sudo add-apt-repository \
   "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable"

# Update the package database with the Docker packages
$ sudo apt-get update

# Make sure you are about to install from the Docker repo
$ sudo apt-cache policy docker-ce

# Install Docker
$ sudo apt-get install -y docker-ce

# Add your user to the docker group
$ sudo usermod -aG docker ubuntu
# If necessary, log out and log back in to use the new group permissions

# Check the current release and, if necessary, update it in the command below
$ sudo curl -L \
    "https://github.com/docker/compose/releases/download/1.26.2/docker-compose-$(uname -s)-$(uname -m)" \
    -o /usr/local/bin/docker-compose

# Set execute permissions on docker-compose
$ sudo chmod +x /usr/local/bin/docker-compose

# Link an available version of python
$ sudo ln -s /usr/bin/python3 /usr/local/bin/python

CentOS 7

# Most of these commands need to be run by the root user
$ sudo su

$ yum check-update
$ yum install -y epel-release
$ yum install -y wget openssl python-pip unzip pv

# Install Docker
$ curl -fsSL https://get.docker.com/ | sh

# Add your user to the Docker group. i.e. centos
$ usermod -aG docker centos

# Enable the Docker daemon to start at system boot
$ systemctl enable docker.service

# Start the Docker daemon
$ systemctl start docker.service

# Install Docker Compose
$ pip install docker-compose

Red Hat Enterprise Linux 7

This guidance was tested with RHEL 7.6 3.10.0-957; per Docker documentation the OverlayFS driver requires a kernel version of at least 3.10.0-514. The OverlayFS storage driver must sit on top of an XFS file system. The following guidance assumes a storage disk is accessible at /dev/xvdf.

# Red Hat Labs Registration
# SKIP IF: You have already registered your instance with Red Hat
$ subscription-manager register --username <username> --password <password> --auto-attach

# Initialize a disk w/ the XFS file system for OverlayFS & Docker
$ parted -a optimal /dev/xvdf  mklabel gpt
$ parted -a optimal /dev/xvdf  mkpart primary xfs 0% 100%
$ mkfs.xfs -n ftype=1 /dev/xvdf1
$ mkdir /docker
$ mount /dev/xvdf1 /docker

# Setup Docker users & groups:
$ useradd -M -r docker
$ usermod -aG docker ec2-user

# Enable Red Hat Subscription based repositories
$ subscription-manager repos --enable=rhel-7-server-rpms \
  && subscription-manager repos --enable=rhel-7-server-extras-rpms \
  && subscription-manager repos --enable=rhel-7-server-optional-rpms

# Update System
$ yum update -y

# Enable EPEL
$ yum install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm

# Install Docker & Atlas dependencies
$ yum install -y docker device-mapper-libs device-mapper-event-libs \
  wget openssl unzip pv

# Configure Docker to use disk space
$ echo '{"graph":"/docker"}' > /etc/docker/daemon.json

# Set SELinux file context type rules (https://success.docker.com/article/how-to-set-selinux-file-contexts-when-using-a-custom-docker-data-root)
$ semanage fcontext -a -e /var/lib/docker "/docker"
$ restorecon -R /docker

# Enable the Docker daemon to start at system boot
$ systemctl enable docker.service

# Start the Docker daemon
$ systemctl start docker.service

# Install Docker Compose
# Check the current release and, if necessary, update it in the command below
$ curl -L \
  "https://github.com/docker/compose/releases/download/1.24.1/docker-compose-$(uname -s)-$(uname -m)" \
  -o /usr/local/bin/docker-compose

# Set execute permissions on docker-compose
$ chmod +x /usr/local/bin/docker-compose

# Link docker-compose into the PATH
$ ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose

Red Hat Enterprise Linux 8

The OverlayFS storage driver must sit on top of an XFS file system. The following guidance assumes a storage volume is accessible at /dev/nvme1n1.

# Red Hat Labs Registration
# SKIP IF: You have already registered your instance with Red Hat
$ subscription-manager register --username <username> --password <password> --auto-attach

# Initialize a disk w/ the XFS file system for OverlayFS & Docker
$ mkfs -t xfs -n ftype=1 /dev/nvme1n1
$ mkdir /data
$ mount /dev/nvme1n1 /data
$ chown ec2-user /data # Replace "ec2-user" with the Atlas service account name
# Add the new volume to /etc/fstab to mount on reboot

# Update System
$ yum update -y

# Enable EPEL and Docker repository
$ rpm --import https://download.docker.com/linux/centos/gpg
$ rpm --import http://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-8
$ yum install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm
$ yum install -y yum-utils
$ yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
$ yum makecache

# Install Docker & Atlas dependencies
$ yum remove -y podman
$ yum install -y docker-ce wget openssl unzip pv python36

# Configure Docker to use allocated volume
$ mkdir -p /etc/docker
$ mkdir /data/docker
$ echo '{"graph":"/data/docker"}' > /etc/docker/daemon.json

# Setup Docker group
$ usermod -aG docker ec2-user

# Set SELinux file context type rules (https://success.docker.com/article/how-to-set-selinux-file-contexts-when-using-a-custom-docker-data-root)
$ semanage fcontext -a -e /var/lib/docker "/data/docker"
$ restorecon -R /data/docker

# Enable the Docker daemon to start at system boot
$ systemctl enable docker.service

# Start the Docker daemon
$ systemctl start docker.service

# Install Docker Compose
# Check the current release and, if necessary, update it in the command below
$ curl -L \
  "https://github.com/docker/compose/releases/download/1.28.0/docker-compose-$(uname -s)-$(uname -m)" \
  -o /usr/local/bin/docker-compose

# Set execute permissions on docker-compose
$ chmod +x /usr/local/bin/docker-compose

# Link docker-compose into the PATH
$ ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose

# Link an available version of python
$ ln -s /usr/bin/python3 /usr/local/bin/python

Install Atlas

This guide will walk you through the following components of the Atlas installation:

• Downloading the installer.
• Using the installer to download and install Atlas.

Check prerequisites

To download Atlas, your system must have at least enough disk space for the Docker images and tilesets. The disk space requirements will be different depending on whether you download the development or production tilesets.

Tileset size

Below are approximate sizes for each tileset. Exact tileset file sizes are subject to change with new data from periodic updates.

Docker images

In total, Docker images are about 15 GB.

Total disk space recommendations

Because installation maintains two copies of your data, we recommend hosting Atlas on a server with double the disk space that the data will require. If you are downloading production tilesets and geocoding data, our recommendation is to host Atlas on a server with at least 1.5 TB of disk space.

Download the installer

1. Go to atlas.mapbox.com/install.
2. Click the Download button for your operating system, either Linux or macOS.
3. Unzip the downloaded zip file: unzip atlas-installer.zip.
4. Make sure the binary is executable: chmod +x ./atlas-installer.

Wizard installation method

The Atlas installer is a command line interface (CLI) utility used to download and setup Atlas. The installer has been specifically designed for air-gapped environments where the host computer does not have access to the internet.

We recommend using the install command. This command will prompt you through a series of questions to configure your setup, execute download to download your Atlas data, then run setup if you've specified that you're installing Atlas on the same machine. To start the installation process run:

$ ./atlas-installer install

Before starting the installation process, be prepared with the following information:

• Your Mapbox access token with atlas:read scope. See Access Token for instructions on how to create this token.
• Whether you plan to run Atlas on the same machine you are running the install command on.
• Which version of Atlas you want to download.
• What data you would like to download including development data.

For a full reference of the prompts you will see when running install, see Installer prompts reference

Install Atlas on the same machine

If you are running this command on the machine you will be installing Atlas on, you should also be prepared with the following information:

• If you plan to run Atlas on HTTPS.
• If you have an SSL certificate. If yes, where it is located.
• The path where you would like to install Atlas.
• Your Atlas server hostname, port, and a password.

After you have answered all the prompts, the installer will download all the Atlas files you specified and put them in a directory named atlas-server-files. If you are installing Atlas on the same machine (answered yes to the first prompt), the installer will run the setup command and configure your installation based on the information passed in to the wizard. Once complete, you can skip to the run Atlas section.

Install Atlas on a different machine

If you are planning to install Atlas on a different machine (for example, a machine without an internet connection), you should only run install to download Atlas. You will then need to run the install.sh script on the machine you plan to use Atlas on.

To do this, answer no to the first prompt if you plan to run Atlas on a different machine, or if you are running Atlas in an air-gapped environment. If you answered no to the first prompt, the installer wizard will only ask you the questions needed to download data. Once the installer has finished downloading data, you will need to copy the downloaded files and run the install.sh command.

If you are planning to install Atlas on a different machine (for example, a machine without an internet connection), you should only run install to download Atlas. You will then need to run setup on the machine you plan to use Atlas on.

To do this, answer no to the first prompt if you plan to run Atlas on a different machine, or if you are running Atlas in an air-gapped environment. If you answered no to the first prompt, the installer wizard will only ask you the questions needed to download data. Once the installer has finished downloading data, you will need to copy the downloaded files along with the installer and run the setup command.

Manual installation method

We recommend using the installer wizard for a better install experience.

Download Docker images, tilesets, and data

The download command will download the files required to a directory to be used by the install.sh command (or setup command if using Atlas installer version 1.5.0). These two steps are separated to enable an offline installation.

Usage

$ ./atlas-installer download <destination> --token <your-atlas-token> [options]

Positional arguments

• <destination> (required): The directory where the downloaded files will be saved.

Options

  --token, -t              Mapbox secret access token with "atlas:read" scope, e.g "sk.abc123"
  --version, -v            Version of Atlas to download.
  --docker                 Download docker images.
  --dev                    Download development data where possible.
  --tilesets               Specify which production tilesets to download
  --geocoding              Download geocoding data.
  --boundaries             Download boundaries tilesets.
  --navigation             Download navigation data.
  --tileset-checksums      Calculate checksums for tileset downloads.
  --help                   Show help

If no flags are passed in to the download command nothing will be downloaded. Depending on your selections, geographic region, and available bandwidth this step can be time-consuming. All files must be downloaded before proceeding to the setup command.

Downloading development data

If you are testing or evaluating Atlas, we recommend that you execute the download command with the --dev flag to test your Atlas configuration without committing to downloading the full set of production tilesets. These are low-zoom (z0-8) versions of the Mapbox Streets, Satellite, and Terrain tilesets that you can use for testing.

$ ./atlas-installer download atlas-server-files/ --token <your-atlas-token> --dev --docker --tilesets streets --tilesets satellite --tilesets terrain

Downloading a specific version of Atlas

By default, the download command will download the latest release of Atlas. You can use the list command to see the list of Atlas versions that are available to download. For more information on the list command, see the List available Atlas versions section. If you require an Atlas version older than 2.6.0, contact Mapbox support for help.

For example, to download v2.6.0 of Atlas, you would run:

$ ./atlas-installer download atlas-server-files/ --token <your-atlas-token> --version v2.6.0 --docker

Downloading specific tilesets

If you only need to download a single tileset, you can limit the tilesets downloaded with the --tilesets flag. In this example only the development version of streets is downloaded:

$ ./atlas-installer download atlas-server-files/ --token <your-atlas-token> --dev --tilesets streets

Downloading only Docker images

To only download Docker images (e.g., to upgrade Atlas without replacing data), you would run:

$ ./atlas-installer download atlas-server-files/ --token <your-atlas-token> --docker

Setup Atlas

Move or copy files downloaded from the download command to the Atlas host machine. Execute the install.sh command on the Atlas host machine. The install.sh command will copy files from the source directory into the target directory and will load Docker images into the Docker context. The install.sh command does not change the downloaded data, enabling the install.sh command to be executed repeatedly.

Usage

$ ./atlas-server-files/install.sh <target>

Positional arguments

• <target> (required): The directory where Atlas will be installed on the host.

Move or copy files downloaded from the download command to the Atlas host machine. Execute the setup command on the Atlas host machine. The setup command will copy files from the source directory into the target directory and will load Docker images into the Docker context. The setup command does not change the downloaded data, enabling the setup command to be executed repeatedly.

Usage

./atlas-installer setup <source> <target>

Positional arguments

<source> (required): The directory that contains all the files downloaded with the download command.<target> (required): The directory where Atlas will be installed on the host.

List available Atlas versions

The list command lists of all downloadable Atlas versions.

You can determine the version of Atlas you have installed by listing the files in <atlas-server>/docker directory. You will find the version of Atlas in the docker tarballs.

atlas-api-accounts_a053a96-d95e584-53eb9ce_v2.7.0.tar.gz --> Atlas version 2.7.0

You can also get your version from the atlas.sh script:

$ ./atlas.sh version

Usage

$ ./atlas-installer list --token <your-atlas-token>

Options

• --token <token> (required): A Mapbox access token with atlas:read scope.

Once you have found the version of Atlas you would like to install, run the download command and specify the --version.

$ ./atlas-installer download atlas-server-files/ --version <your-atlas-version> --token <your-atlas-token> --dev

Reference

Commands

Installer prompts reference

Do you plan to run Atlas on this machine?

Defaults to yes. Select no if you only want to download dependencies.

Enter your sk token with atlas:read scope

Copy and paste your secret token you created on mapbox.com with atlas:read scope.

Which Atlas version would you like to download?

You will be presented with a drop down menu of available versions to download. The first version listed is the latest stable version. Use your arrow keys to select a different version, and press <enter> to continue.

What data would you like to download?

By default, Docker images, mapbox-streets-v7, and Geocoding data are selected as options. Use your arrow keys to navigate the options. You can press <space> to select, <a> to toggle all, or <i> to invert selection.

Would you like to download development data where possible?

Defaults to no. Select yes if you want to download development datasets.

Would you like to verify tileset hash values?

Defaults to no. Select yes if you would like to verify file integrity for downloaded tilesets. For large tilesets, hash verification may add significant time to the download process.

Where do you plan to install Atlas?

Only asked if installing Atlas on the same machine.

Defaults to atlas-server. Enter the name of the directory you wish to install Atlas. The installer will create this directory for you.

Enable high-resolution tilesets

If your Atlas license includes access to high-resolution terrain data, use the set-tileset-zoom command to enable access to these tilesets. After installing Atlas and downloading terrain data, start Atlas and run the following commands from your Atlas server directory:

./atlas.sh set-tileset-zoom mapbox.terrain-rgb 14
./atlas.sh set-tileset-zoom mapbox.mapbox-terrain-v2 15

Configure Atlas

Atlas uses a single config file in the Atlas root directory to manage all configuration settings.

The default configuration

# Atlas username is atlas-user
# This is the password for atlas-user
atlasUserPassword=password
atlasLicense=

# This is the URL where users will be accessing Atlas
# This could be
# - the machine's local IP Address
# - the DNS name of your load balancer
# - etc
atlasURL=http://localhost:2999

# This is the port, protocol, and network interface that the Atlas nginx Docker
# container will listen on.
#
# If https is used, Atlas will expect SSL certificates to be placed
# in the $ATLAS_ROOT/ssl_cert folder with names:
# - cert.pem
# - key.pem
atlasPort=2999
atlasProtocol=http
atlasInterface=0.0.0.0

# This sets the Atlas log level to help control the verbosity of logs.
# Valid values:
# - error: limits log messages to important messages (default)
# - info: log more informational messages in addition to errors & warnings.
# Note that request logs are output for both log levels.
atlasLogLevel=error

# Sets the access keys for Minio s3 backend.
# These are the default access keys and they should be changed before starting Atlas the first time.
# Key rotation can be done following these instructions:
# https://github.com/minio/minio/blob/master/docs/config/README.md#rotating-encryption-with-new-credentials
# Uncomment and set the MINIO_ACCESS_KEY_OLD and MINIO_SECRET_KEY_OLD keys
# directly in the atlas-minio service in the docker-compose.yml
AWS_ACCESS_KEY_ID=EXAMPLE
AWS_SECRET_ACCESS_KEY=EXAMPLEKEY

# Atlas username is atlas-user
# This is the password for atlas-user
atlasUserPassword=password

# This is the URL where users will be accessing Atlas
# This could be
# - the machine's local IP Address
# - the DNS name of your load balancer
# - etc
atlasURL=http://localhost:2999

# This is the port and protocol that the Atlas nginx Docker
# container will listen on.
#
# If https is used, Atlas will expect SSL certificates to be placed
# in the $ATLAS_ROOT/ssl_cert folder with names:
# - cert.pem
# - key.pem
atlasPort=2999
atlasProtocol=http

# Sets the access keys for Minio s3 backend.
# These are the default access keys and they should be changed before starting Atlas the first time.
# Key rotation can be done following these instructions:
# https://github.com/minio/minio/blob/master/docs/config/README.md#rotating-encryption-with-new-credentials
# Uncomment and set the MINIO_ACCESS_KEY_OLD and MINIO_SECRET_KEY_OLD keys
# directly in the atlas-minio service in the docker-compose.yml
AWS_ACCESS_KEY_ID=EXAMPLE
AWS_SECRET_ACCESS_KEY=EXAMPLEKEY

Configuration reference

• atlasUserPassword should be changed and is generally only shared with Atlas administrators.
• atlasLicense is the Atlas license key, also available from the Atlas installation page.
• atlasURL is the URL that Atlas Studio and Atlas APIs use (e.g. local IP address, public DNS hostname, etc.).
• atlasPort is the port that Atlas uses on the host machine.
• atlasProtocol is the protocol (HTTP or HTTPS) that Nginx in Atlas listens on.
• atlasInterface is the IP of the network interface that Atlas listens to. The default 0.0.0.0 listens on all interfaces.
• atlasLogLevel is the log verbosity setting for the Atlas server.
• AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY are access credentials for the internal S3 datastore.

Configure Atlas password

The password used to login to the Atlas Account page, Studio, and other resources should be changed from the default. To change this,

1. Open the config file and set the desired password in the atlasUserPassword field.
2. Restart Atlas: ./atlas.sh stop && ./atlas.sh start.

Configure HTTPS

Atlas can be configured to communicate using HTTPS for increased security. In production use, the certificate file should be signed by a trusted Certificate Authority. For ease of testing, we also provide a script to generate a self-signed certificate.

1. Edit the configuration:
   1. Open ./config.
   2. Change atlasURL to start with the https protocol (e.g., https://localhost:2999).
   3. Change atlasProtocol to https.
   4. Save and close the file.
2. Save the certificate:
   • If you have a signed certificate:
     1. Save the private key (in PEM format) to ./ssl_cert/key.pem.
     2. Save the certificate (in PEM format) to ./ssl_cert/cert.pem.
   • If you want to create a self-signed certificate:
     1. cd to the Atlas root directory and run sudo ./atlas.sh create-certificate. This stores the necessary files in ./ssl_cert for you.
3. Restart Atlas: ./atlas.sh stop && ./atlas.sh start

For example, to configure HTTPS with a self-signed certificate, you would run:

sed -i .bak -e 's/atlasURL=http:/atlasURL=https:/;s/atlasProtocol=http$/atlasProtocol=https/' config
sudo ./atlas.sh create-certificate
./atlas.sh stop && ./atlas.sh start

Configure HTTP

Atlas can be configured to communicate by unencrypted HTTP if it has previously been configured with HTTPS.

1. Edit the configuration:
   1. Open ./config.
   2. Change atlasURL to start with the http protocol (e.g., http://localhost:2999).
   3. Change atlasProtocol to http.
   4. Save and close the file.
2. Restart Atlas: ./atlas.sh stop && ./atlas.sh start

For example, to configure HTTP, you would run:

sed -i .bak -e 's/atlasURL=https:/atlasURL=http:/;s/atlasProtocol=https$/atlasProtocol=http/' config
./atlas.sh stop && ./atlas.sh start

Run Atlas behind a reverse proxy

A common configuration is to have Atlas served over HTTP and behind an HTTPS reverse proxy. Because some services within Atlas advertise URIs to the client, you will need to configure Atlas to advertise the correct URI. These environment variables populate those URIs within the services.

1. Stop Atlas: ./atlas.sh stop
2. Edit ./config
3. Update the atlasURL variable with your public hostname or load balancer address.
4. Update atlasProtocol & atlasPort as needed.
5. Start Atlas: ./atlas.sh start

Run Atlas

Atlas is started and stopped from the command line.

Start the server

To start Atlas, run the following commands:

1. $ cd atlas-server
2. $ ./atlas.sh start
3. You will be able to follow startup progress in the console.

Stop the server

To stop Atlas, run the following commands:

1. $ cd atlas-server
2. $ ./atlas.sh stop
3. You will see a confirmation message once the Atlas server has stopped running.

Fix Redis THP and memory warnings

There are a few system requirements that Redis requires to run efficiently in a production environment. When you start Atlas, you may see the following Redis log warnings during startup:

redis           | 1:C 18 Nov 2019 19:09:59.300 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis           | 1:C 18 Nov 2019 19:09:59.301 # Redis version=5.0.5, bits=64, commit=00000000, modified=0, pid=1, just started
redis           | 1:C 18 Nov 2019 19:09:59.301 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
redis           | 1:M 18 Nov 2019 19:09:59.324 * Running mode=standalone, port=6379.
redis           | 1:M 18 Nov 2019 19:09:59.324 # Server initialized
redis           | 1:M 18 Nov 2019 19:09:59.324 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.

Enable vm.overcommit_memory (always)

To enable vm.overcommit_memory without checks, run the following:

$ echo "vm.overcommit_memory = 1" | sudo tee -a /etc/sysctl.conf

Disable THP

Typically, THP can be disabled via a kernel option and the process for that depends on your specific system. Here are some examples that work well for us. Once reconfigured, you'll need to restart the system and verify the changes have persisted.

To disable THP on CentOS 7 / RHEL 7

Add transparent_hugepage=never to the GRUB_CMDLINE_LINUX line in /etc/default/grub. Then the file should look something like:

$ cat /etc/default/grub
GRUB_TIMEOUT=1
GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
GRUB_DEFAULT=saved
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL="serial console"
GRUB_SERIAL_COMMAND="serial --speed=115200"
GRUB_CMDLINE_LINUX="transparent_hugepage=never console=tty0 crashkernel=auto console=ttyS0,115200"
GRUB_DISABLE_RECOVERY="true"

Then, rebuild grub.cfg

# Rebuild grub.cfg on a BIOS-based machine
$ sudo grub2-mkconfig -o /boot/grub2/grub.cfg

# Rebuild grub.cfg on a UEFI-based machine
$ sudo grub2-mkconfig -o /boot/efi/EFI/centos/grub.cfg

To disable THP on Ubuntu 18.04

Add transparent_hugepage=never to the GRUB_CMDLINE_LINUX line in /etc/default/grub. Then the file should look something like:

$ cat /etc/default/grub
GRUB_TIMEOUT=1
GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
GRUB_DEFAULT=saved
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL="serial console"
GRUB_SERIAL_COMMAND="serial --speed=115200"
GRUB_CMDLINE_LINUX="transparent_hugepage=never console=tty0 crashkernel=auto console=ttyS0,115200"
GRUB_DISABLE_RECOVERY="true"

Then, rebuild grub.cfg

$ sudo update-grub

To disable THP on Ubuntu 18.04 (Cloud Image)

$ sudo echo "GRUB_CMDLINE_LINUX_DEFAULT=\"\$GRUB_CMDLINE_LINUX_DEFAULT transparent_hugepage=never\"" > /etc/default/grub.d/99-transparent-hugepage.cfg'
$ update-grub

Reboot and verify

Reboot the machine to take effect. Next, verify the settings are correct:

$ sysctl vm.overcommit_memory
vm.overcommit_memory = 1

$ cat /sys/kernel/mm/transparent_hugepage/enabled
always madvise [never]

Troubleshoot your Atlas installation

If you have any issues while installing or running Atlas, the following troubleshooting strategies may help you diagnose and fix the problem.

Out-of-memory Docker errors

When developing locally with Atlas, you will sometimes get out-of-memory errors. Docker is conservative about garbage collection, meaning you will sometimes need to use the Docker prune command to clean up unused Docker assets. To learn more about how to use the prune command, visit the Docker documentation on pruning.

"No space left on device" errors

The Atlas installer setup command uses docker load. You need enough disk space in both /var/lib/docker and /tmp to run setup. We recommend at least 20 GB of space for both directories. If you receive the error message Error processing tar file(exit status 1): ... no space left on device, increase the disk space in either the /var/lib/docker or /tmp directories.

"Docker daemon is not running" errors

When the setup command is running, you might see the error: The Docker daemon is not running. Please start Docker before proceeding. If this occurs, reboot your instance with the command sudo reboot. Then you can run the setup command again.

View server logs

To view your Docker logs to diagnose issues, run the command docker-compose logs -f and pipe the output of this command into a new log file. The -f flag lets you follow the log output. To view other flag options for this command, see the docker-compose logs documentation.

For additional log output, set the atlasLogLevel setting to info in the Atlas configuration file.

Expired license error message

Your Account page will display the expiration date of your Atlas user license so that you know when to get the renewal process started.

If your Atlas license has expired, you will see an expiration error when you try to start Atlas. If you see this message, contact us to discuss renewing your Atlas license.

Cannot verify certificate

On certain older systems Mapbox's certificate issued by DigiCert is not recognized. You can download a certificate authority bundle and configure wget to trust the certificate by executing these commands:

$ mkdir ~/ca_certs
$ openssl s_client -showcerts -servername mapbox-atlas.s3.amazonaws.com \
  -connect mapbox-atlas.s3.amazonaws.com:443 </dev/null > ~/ca_certs/mapbox.crt
$ echo ca_directory=~/ca_certs >> ~/.wgetrc

Submit feedback about Atlas

If you have any technical issues with installing or running Atlas, send an email to support@mapbox.com with the following information:

• Operating system name and version
• Atlas version and architecture
• What were you doing when you ran into the problem? What specific commands did you run or actions did you take?
• What was the expected outcome?
• What was the actual outcome?
• Any relevant error logs in the browser console
• Any relevant application error logs

Upgrade Atlas

To upgrade to the latest version of Atlas, you will need access to the internet to download new data.

To upgrade when a new Atlas version is released, you will need to download new Docker images and data. Upgrading data (e.g., to add another tileset) will not require new Docker images. This guide applies to both scenarios.

Upgrade Atlas and data

Atlas should be updated as a fresh installation, where Atlas is newly installed and settings and data are copied. Always backup your data and settings before starting an upgrade.

Fresh install

To do a fresh install, first install Atlas with the latest data and then migrate custom data over.

1. Install the new version of Atlas using the instructions in Install Atlas and Configure Atlas. Copy any custom configuration settings from the old config file into the new config file.

2. Copy over your custom data:

   <atlas-server-new>/atlas.sh stop
   cp <atlas-server-old>/mbtiles/atlas-user.*.mbtiles <atlas-server-new>/mbtiles
   rm -rf <atlas-server-new>/dynalite-data/
   cp <atlas-server-old>/dynalite-data <atlas-server-new>/dynalite-data
   

3. Start the new Atlas installation:

   cd <atlas-server-old>
   ./atlas.sh stop
   ./atlas.sh clean
   cd <atlas-server-new>
   ./atlas.sh start
   

4. Connect to Atlas in your browser and verify the listed version is correct. Use Studio to verify expected data is correct.

5. Once you are satisfied the new installation is complete, you can uninstall the previous version of Atlas:

<atlas-server-old>/remove.sh <atlas-server-old>
rm -rf <atlas-server-old>

./atlas-installer remove <atlas-server-old>
rm -rf <atlas-server-old>

Upgrade to a specific version of Atlas

To upgrade to a specific version of Atlas, provide the version number when downloading the files. Use the atlas-installer list command to list available Atlas versions. You may need to download a different version of the installer if the Atlas version you need is not displayed.

$ ./atlas-installer download atlas-server-files/ --token <your-atlas-token> --docker --version <your-version>

# Set up Atlas with new images and tilesets
$ atlas-server-files/install.sh atlas-server/

$ ./atlas-installer download atlas-server-files/ --token <your-atlas-token> --docker --version <your-version>

# Set up Atlas with new images and tilesets
$ ./atlas-installer setup atlas-server-files/ atlas-server/

Upgrade data only

To upgrade the data used by your Atlas installation, follow these instructions. For example, this will be useful if you want to replace development tilesets with production tilesets or have an update for a tileset.

Update Boundaries data

Periodically, Boundaries data receives updates with additional and improved information. To upgrade Atlas with these updates, you will re-run the download and setup commands.

First verify that your download destination directory is empty. Once confirmed, run the following commands to update Boundaries data:

# Download updated Boundaries data
$ ./atlas-installer download atlas-server-files/ --token <your-atlas-token> --boundaries

# Overwrite the previous Boundaries data with the updated Boundaries data
$ atlas-server-files/install.sh atlas-server/

# Download updated Boundaries data
$ ./atlas-installer download atlas-server-files/ --token <your-atlas-token> --boundaries

# Overwrite the previous Boundaries data with the updated Boundaries data
$ ./atlas-installer setup atlas-server-files/ atlas-server/

Upgrade from development to production tilesets

If you initially installed Atlas with low-res development tilesets, at some point you may want to upgrade to full-zoom production tilesets. To do this, you will re-run the download command with the --tilesets flag.

First verify that your download destination directory is empty. Once confirmed, run the following commands to upgrade from development tilesets to production tilesets:

# Download production tilesets
$ ./atlas-installer download atlas-server-files/ --token <your-atlas-token> --tilesets all

# Overwrite the dev tilesets with the downloaded production tilesets
$ atlas-server-files/install.sh atlas-server/

# Download production tilesets
$ ./atlas-installer download atlas-server-files/ --token <your-atlas-token> --tilesets all

# Overwrite the dev tilesets with the downloaded production tilesets
$ ./atlas-installer setup atlas-server-files/ atlas-server/

You can select which tilesets to download and update. Run atlas-installer download --help to see available tileset options. For example, to download production Mapbox Satellite and Mapbox Terrain data:

./atlas-installer download atlas-server-files --tilesets satellite --tilesets terrain

Upgrade from development to production geocoding data

Like upgrading from development to production tilesets, to fully experience Atlas Search, you should use production search data. To do this, you will re-run the download command with the --geocoding flag.

First verify that your download destination directory is empty. Once confirmed, run the following commands to upgrade from development geocoding data to production geocoding data:

# Download production geocoding data
$ ./atlas-installer download atlas-server-files/ --token <your-atlas-token> --geocoding

# Overwrite the dev geocoding data with the downloaded production geocoding data
$ atlas-server-files/install.sh atlas-server/

# Download production geocoding data
$ ./atlas-installer download atlas-server-files/ --token <your-atlas-token> --geocoding

# Overwrite the dev geocoding data with the downloaded production geocoding data
$ ./atlas-installer setup atlas-server-files/ atlas-server/ --config <config.json>

Update your Atlas license

To update your Atlas license, update the atlasLicense setting in the Atlas root config file.

First, log in to your Mapbox.com account with an Atlas license. Then visit the Atlas installation page and copy the license key.

Next, open the config file and paste your updated license string:

atlasLicense=ey...

Save the file and restart Atlas. After Atlas restarts, visit the main Atlas landing page (e.g., http://localhost:2999). Verify the new license expiration date under the License heading on the right-hand side of the main landing page.

To update your Atlas license, you will need to update the license.json file, located in the same folder as atlas.sh. You can do this while Atlas is up and running.

First, grab your license from https://atlas.mapbox.com/install (make sure you're logged in as the account with the Atlas license) and copy your license key.

Next, open up ./config, and paste your updated license string:

{
  "license": "your-old-license-string-will-be-here----replace-it-with-your-new-one"
}

Save the file, and restart the Atlas application. Once this succeeds, you should be able to see your new license expiration date under the License heading on the right side of the main Atlas landing page (e.g., http://localhost:2999).

Production checklist

This checklist serves as a guide by providing recommended configuration options for securely deploying your Atlas instance to a production environment. It is a complement to implementation guides, best practices, and organizational policies for deploying production-grade web services.

Deploying with a load balancer

Regardless of the size of your Atlas deployment, a load balancer, CDN, or DDoS protection service can protect your Atlas server from excessive traffic. Any administrator of a public Atlas deployment should consider how their server will perform under higher-than-expected load.

When deploying Atlas with a load balancer, make sure the X-Forwarded-For header is set to allow Atlas to include the correct client IP address in system logs.

The rate limits in Atlas are not a substitute for a dedicated DDoS protection service. Any administrator of a public Atlas deployment should consider appropriate denial-of-service mitigation, such as limiting simultaneous sessions from an IP or block.

Firewall and network configuration

Atlas is designed to run in a variety of network environments, both on the Internet and in air-gapped enclaves.

Protocol

Under default configuration settings, Atlas will serve data over the unencrypted HTTP protocol. Most production deployments should encrypt traffic to and from the Atlas server, either using a load balancer, a reverse proxy, or by setting atlasProtocol to https in the config file.

Ports

Atlas listens on a single TCP port, set by atlasPort in the config file. By default this port is 2999, but we recommend that your public-facing endpoint use TCP port 443 (for HTTPS) or port 80 (for HTTP). Atlas does not require any outbound ports.

Listening interfaces

By default, Atlas will listen on the configured port on all network interfaces. If you want to listen on a particular interface, set the atlasInterface configuration option to the IP address of the appropriate interface, then restart Atlas.

Logging

Atlas containers send logs to stdout, which is routed through the logging configuration of the Docker daemon. To enable log rotation or to send logs to a centralized log repository, configure Docker logging appropriately.

Atlas logs contain information that can identify the cause of service failure. However, no particular information needs to be retained to expediently bring the application back online.

Data protection

Atlas APIs include rate limiting to mitigate Denial of Service and scraping of Atlas map content. You must not host the Atlas Software in a way that makes any Atlas content accessible to the public except through the Atlas APIs.

Atlas sets Cross-Origin Resource Sharing (CORS) headers to allow any of your sites to use its APIs. To limit access to a more restricted set of origins, change the CORS headers in a proxy or load balancer.

Atlas makes certain information about the running version, license validity, and default public token available on endpoints with the /api/ route prefix. These endpoints are required for operation of the Atlas Account Dashboard, Atlas Studio, and the kepler.gl plugin. You can block access to this data by configuring a reverse proxy or load balancer to deny access to endpoints with the /api/ prefix.

Updates

Atlas updates include new functionality and security patches. To make sure that you receive the latest security updates, check for new Atlas versions using the Atlas installer or the Atlas installer website at least once every 30 days.

Release notes

2.8.0-beta.2

Added

• Added support for the map-sessions endpoint to support GL JS v2
• Added the style-components volume to support new Studio features
• Added validation checks to the atlasURL setting
• Added support for the atlasURL http[s]://use-host-header value to allow a single instance to serve requests from multiple domains

Changed

• The user's Atlas license is now written to the config file instead of to license.json
• Running ./atlas.sh stop no longer removes Atlas containers after stopping them
• Renamed bundled memcached and redis containers to atlas-memcached and atlas-redis
• Moved static examples inside a container, so that they can now be downloaded and edited

Removed

• Removed api-maps container
• Removed support for deprecated Legacy Static API

2.7.0

Added

• Added geocoding data for Belarus, Cambodia, Canada, and France
• Added support for Atlas Navigation for preview customers
• Improved SELinux support for Docker volumes

Changed

• Replaced fake-s3 with MinIO for scalability improvements
• Bypass default proxy settings when generating public access token

Fixed

• Fixed set-user-limits command to allow authorized users to configure rate limits
• create-cert.sh utility script uses 4096 bits instead of 4086

2.6.0

Added

• Kepler.gl, a web application for visual exploration of large-scale geolocation data sets, added to dashboard
• Mapbox Boundaries v2 demos now accessible from the main landing page
• Maps cleared from Redis cache when associated MBTiles is changed
• Atlas start script includes create-certificate command to generate self-signed certificate for development use
• Atlas start script provides help text
• Atlas start script can disable starting api-geocoder with command ./atlas.sh start --disable-geocoding
• Atlas start script has improved status handling while starting containers and terminates with helpful messages if containers reach a state other than healthy or starting

Changed

• Updated Mapbox APIs to the latest versions
• Atlas configuration now documented inline and accessible at $ATLAS_ROOT/config
• Includes up to Mapbox GL JS v1.6.1
• Includes up to Mapbox.js v3.2.1
• Simplified environment variable-based configurations
• Renames atlas-redis and atlas-memcached to redis and memcached
• Docker container health checks are faster

Fixed

• CORS support added on /static/ for local development
• Permissions on pk_token.txt explicitly set
• Improved resiliency of start-up health checks

2.5.4

Added

• Mapbox Boundaries v2 support
• Upgraded Maps APIs to latest state as of 5/16/2019
• Upgraded Accounts API to latest state as of 5/29/2019
• Display current Atlas version on account dashboard
• Upgraded Mapbox GL JS up to v0.54.0 changelog
• Upgraded Mapbox GL Geocoder up to v4.3.0 changelog
• Large MBTiles file (> 25 GB) support

Changed

• Removed docker-compose.https.yml — HTTPS is now configured by environment variable
• Consolidated .env files in config

Security

• Upgraded images to use node 8/10 and Ubuntu 18.04