Internship Project @ Hanlun AI

This project investigates using a simple setup consisting of an embedded computing module (NVIDIA Jetson Nano) with only a monocular camera to avoid objects and detect collisions in real-time, serving as a cost-effective alternative to systems requiring depth cameras or lidar/radar.

We use dense optical flow in OpenCV to deduce the robot's motion and infer environment structure through motion parallax. The system builds a boundary graph of obstacles based on flow magnitude thresholds, allowing the vehicle to steer towards free space and back up upon collision detection.

Platform

Built on the Donkey Car platform, featuring an HSP 94186 RC car, IMX219 wide-angle camera, PCA9685 servo driver, and NVIDIA Jetson Nano. The software stack uses Python, OpenCV, and NVIDIA's Jetrac/Jetcam libraries.

Algorithm

The algorithm processes the optical flow field to make steering and speed decisions:

• Boundary Graph Construction: We compute the magnitude of flow vectors along each vertical strip of the image. By taking the normalized finite difference along these strips, we detect "spikes" in flow magnitude, which correspond to obstacle boundaries.
• Combined Control Metric: We integrate two signals:
  1. The "horizon" of the boundary graph (the lowest point where flow exceeds a threshold), representing free space.
  2. The sum of flow magnitudes in each vertical strip, which indicates overall proximity to obstacles due to motion parallax.
  A combined metric is derived by subtracting the normalized flow sum from the boundary height. The car steers towards the maximum of this combined graph, targeting areas that are both far away (high boundary) and have low overall flow (distant objects).
• Dynamic Speed Adjustment: The vehicle's speed is modulated based on the 85th percentile of the combined graph values. This allows the car to slow down when approaching complex environments or close obstacles, and speed up in open spaces.
• Collision Handling: If the flow magnitude drops below a threshold (indicating a stall) or collision counters exceed a limit, the car executes a recovery maneuver: backing up while steering away from the calculated obstacle direction before resuming forward motion.