Advanced Vision Based MHE Location Tracking

Introduction

Warehouse material handling equipment (lift trucks) transport goods between bays and staging zones. Bays are long term storage locations while staging areas are used as temporary storage areas for loading and unloading. Dozens of trucks can be doing pick and drop tasks simultaneously at a WH location. BlipAI's computer vision (CV) based solution helps with always identifying the precise location where each vehicle is conducting these operations.

Solution Overview

BlipAI's vision based location tracking (VLT) can identify 1D and 2D barcode signs mounted above each bay location. This is achieved through a vehicle mounted device that can capture and analyze scenes in realtime. The analyzed data events are published using an industry standard lightweight protocol like MQTT. All interested parties can subscribe to this telemetry data to make realtime updates to WMS or downstream analytics

Requirements

CV models are difficult to train. The complexity increases further when the camera is in motion. Successful in-motion scene analysis depends on some critical aspects of the hardware like the optics, CMOS frame rate, AE/AF/AWB and ISO ranges. The realtime scene analysis requires capable CPUs & GPUs that can run the neural network (NN) model at an acceptable speed. 

Camera Hardware

Commercially available Android phone devices can be used to run BlipAI's VLT app. Devices tested to run the VLT solution are based on their optical, GPU, CMOS, and compute capabilities. Each MHE is equipped with an Android device that has a clear view of the overhead location barcodes (specification below).

Mounting Hardware

We recommend using a vibration free holder to attach the camera devices to the MHEs. Vibration has been found to be minimal with a bolted anchor type clamp mounted as close as possible to the safety light assembly.

Cooling Hardware

GPU bound workloads can generate heat. In WH environments where ambient temperatures can go above 90F, the Android devices can overheat and cause an automatic shutdown. A Peltier based cooling device is required for such conditions.

Power Requirements

A 5V/3A supply is sufficient to provide continuous power to the smartphone and the optional cooling hardware. The fuse tap must be connected to a continuous 12V supply on the MHE battery.  A buck converter and USB splitter are used to bring the 5V power to the mounted devices.

On-site Facilities

Bays and staging locations are marked b DataMatrix encoded signs. The distance and size of the signage is determined based on the on-site conditions. Good Wifi coverage is essential for the solution to work without interruptions or manual exception handling.

Software Overview

Blip's vision software works with a modified NN model trained to recognize 1D and 2D codes in moving scenes. The model is trained to take into account lighting conditions and tune the optics accordingly.  

Android App

The VLT app is the primary software component that utilizes CV algorithms to perform realtime object recognition, detection and decoding. High performance GPUs on commodity smartphones are utilized for computing location information with negligible lag allowing for realtime operations.

MQTT Broadcast

All on-site events captured by the VLT system are broadcast in realtime to a designated MQTT broker.  The event model is designed to be minimally chatty - i.e. no telemetry is sent when nothing is read and all relevant scan events are limited to a maximum of one per second per MHE. Downstream subscribers to these events can trigger various actions including live updates to WMS, YMS etc.

Scope & Outlook

We are in the early stage of buidling industrial CV applications using low cost Android devices. BlipAI is testing the combined use of vision with other onboard inputs like motion, BLE, and GPS to build powerful SaaS products for a wide range of logistics applications.

For API and sample code documentation please contact us