UAV navigation in unknown complex environment

Survey

• A survey on vision-based UAV navigation, 2018
• Insights on obstacle avoidance for small unmanned aerial systems from a study of flying animal behavior, 2018
• A Review of Autonomous Obstacle Avoidance Technology for Multi-rotor UAVs, 2018
• A literature review of UAV 3D path planning, 2014
• Survey on computational-intelligence-based UAV path planning, 2018
• A Survey of Motion Planning Algorithms from the Perspective of Autonomous UAV Guidance, 2010
• Survey of Motion Planning Literature in the Presence of Uncertainty: Considerations for UAV Guidance, 2012

Traditional method

Multirotor

• First Results in Detecting and Avoiding Frontal Obstacles from a Monocular Camera for Micro Unmanned Aerial Vehicles, 2013
  • uses SURF feature matches in combination with template matching to compare relative obstacle sizes with different image spacing
  • 20 successful real flight experiments using Parrot AR.Drone

Fixed-wing

• Research on obstacle avoidance technology of fixed wing formation based on improved artificial potential field method with stereo vision, 2020
  • improved artificial potential field method

Learning-based method

• Survey

  • Drone deep reinforcement learning: A review, 2021
    • targeted the guidance, navigation, and control (GNC) of UAVs.
    • RL for UAV path planning
  • Deep reinforcement learning based mobile robot navigation: A review, 2021
    • Focus on mobile robot
  • A review of deep learning methods and applications for unmanned aerial vehicles, 2017
    • Focus on DL
    • Supervised and unsupervised learning
  • Deep Learning and Reinforcement Learning for Autonomous Unmanned Aerial Systems: Roadmap for Theory to Deployment, 2020
    • good paper
      • overview of DL and RL
      • How DL used on UAS
      • How RL used on UAS
      • directions to choose appropriate simulation suites and hardware platforms
      • open problems and challenges

• Multirotor

  • Reinforcement Learning for Autonomous UAV Navigation Using Function Approximation, 2018
    • Quadrotor，NAV，Sim and Real、RL（Q-Learning）
  • Path Planning for UAV Ground Target Tracking via Deep Reinforcement Learning, 2020
    • UAV target tracking and obstacle avoidance
    • Fixed-wing, range finder，not MAV
    • Simulation
    • DDPG
  • Memory-based Deep Reinforcement Learning for Obstacle Avoidance in UAV with Limited Environment Knowledge, 2018
  • Quadrotor，OA，DRQN+Attention
    • 特色：RNN+Temporal Attention
    • conditional generative adversarial network for depth estimation
    • 60 Hz on NVIDIA GeForce GTX 1050 mobile GPU
  • Autonomous UAV Navigation: A DDPG-Based Deep Reinforcement Learning Approach, 2020
    • DDPG,3D env, NAV
  • Obstacle Avoidance Drone by Deep Reinforcement Learning and Its Racing with Human Pilot, 2019
    • Quadrotor
    • using multi perception: RGB, Depth and RGB+Depth
    • using multi algorithms: DQN，Actor-Critic RL，TRPO, PPO
    • utilize diverse RL within two categories: (1) discrete action space and (2) continuous action space.
    • Results suggest that our best continuous algorithm easily outperformed the discrete ones and yet was similar to an expert pilot
  • Autonomous navigation of UAV by using real-time model-based reinforcement learning, 2016
    • model-based RL TEXPLORE
    • find an efficient route when it is constrained in battery life
    • simulated quadcopter UAV in ROS and Gazebo environment
    • our approach significantly outperforms Q-learning based method
  • UAV navigation in high dynamic environments: A deep reinforcement learning approach, 2020
    • high dynamic environments, LSTM
    • a clipped DRL loss function is proposed
    • propose a distributed DRL framework containing two sub-networks, namely the Avoid Network and the Acquire Network
    • 2D navigation, using range finders
  • NavREn-Rl: Learning to fly in real environment via end-to-end deep reinforcement learning using monocular images, 2019
    • DDQN
    • small number of expert data and knowledge based data aggregation is integrated into the RL process to aid convergence
    • Parrot AR drone real test
  • A Vision Based Deep Reinforcement Learning Algorithm for UAV Obstacle Avoidance, 2021
  • Deep-Reinforcement-Learning-Based Autonomous UAV Navigation with Sparse Rewards, 2020
    • adopt the sparse reward scheme
    • using prior policy (nonex- pert helper) that might be of poor performance is available to the learning agent.
  • Collision-free UAV navigation with a monocular camera using deep reinforcement learning, 2020
    • monocular camera
    • OBJECT DETECTION-ASSISTED DRL OD+DQN
    • proposed framework reduces flying times to- wards given destinations by 25%, and cuts down 50% of unnecessary turns.
  • Autonomous Navigation of UAVs in Large-Scale Complex Environments: A Deep Reinforcement Learning Approach, 2019
    • POMDP, LSTM
    • Fast-RDPG
    • 3D
  • Generalization through Simulation: Integrating Simulated and Real Data into Deep Reinforcement Learning for Vision-Based Autonomous Flight, 2019
    • Focus on generalization from sim to real
    • Evaluated with a nano aerial vehicle (NAV), Crazyflie 2.0
    • First, we train a deep neural network Q-function using deep reinforcement learning in a visually diverse set of simulated environments.
    • Then, we create the deep neural network action-conditioned reward prediction model, in which we use the perception layers from the simulation-trained Q- function to process the input image state.
    • Next, we train the action-conditioned reward prediction model using real-world data gathered by the robot; however, when training the model, we do not update the parameters of the perception layers.
  • A Deep Reinforcement Learning Framework for UAV Navigation in Indoor Environments, 2019
    • Formulate the UAV navigation problem using MDP and POMDP
    • separating the search problem into high-level planning and low-level action under uncertainty
  • Uncertainty-Aware Reinforcement Learning for Collision Avoidance, 2017
    • uncertainty-aware model-based learning algorithm
    • 16 by 16 grayscale image
  • Deep reinforcement learning for drone navigation using sensor data, 2021
    • PPO
    • incremental curriculum learning
    • LSTM
  • Deep Reinforcement Learning-based UAV Navigation and Control: A Soft Actor-Critic with Hindsight Experience Replay Approach, 2021
    • Focused on algorithm, propose SACHER (soft actor-critic(SAC) with hindsight experience replay (HER))
    • Out perform SAC and DDPG
  • Deep Reinforcement Learning for End-to-End Local Motion Planning of Autonomous Aerial Robots in Unknown Outdoor Environments: Real-Time Flight Experiments, 2021
    • Using laser range finder to autonomously navigate among obstacles and achieve a user-specified goal location in a GPS-denied environment
    • planning smooth forward linear velocity and heading rates
  • A Deep Reinforcement Learning Method with Action Switching for Autonomous Navigation, 2021
    • PPO with Action Switching
    • PID is used as baseline controller
  • Vision Based Drone Obstacle Avoidance by Deep Reinforcement Learning, 2021
    • Use the depth maps as input and combine SAC with a variational auto-encoder (VAE), compared with TD3
    • The output of the actor network has two values ranging from −1 to 1, representing the velocity of the drone in the y direction (left and right directions) and z direction (up and down directions)
  • Learning to Fly with Deep Reinforcement Learning, 2021
    • PPO-Clip
    • Navigation controller without obstacle avoidance

• Fixed-wing

  • End-to-End Deep Reinforcement Learning for Image-Based UAV Autonomous Control, 2021
    • Fixed wing vision-based landing
    • map the input image directly to the continuous actuator control command
    • a comparison with a nonlinear model predictive (NMPC) technique has been proposed in simulation
  • UAV Path Planning Based on Multi-Layer Reinforcement Learning Technique, 2021
    • multi-layer path planning algorithm
      • higher layer deals with the local information (short-term strategy)
      • lower layer deals with the global information (long-term strategy)
      • B-spline curve approach is applied for on-line path smoothing

• Flapping-wing

  • Vision-based obstacle avoidance for flapping-wing aerial vehicles, 2020
    • binocular vision for depth (640 × 480, 70 deg FoV)
    • Its total weight is 420 g, and the load weight is 170 g
    • single-shot multi-box detector based on deep learning (MobileNet architecture, 34.7Mb model)
    • Raspberry Pi 3b+ with Intel neural network acceleration bar
    • steering angle control using reactive method
    • 10 Hz but flight speed is only 1.5m/s

• Simulator

  • AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles, 2017
    • built on Unreal Engine that offers physically and visually realistic simulations for both of these goals
  • Air Learning: An AI Research Platform for Algorithm-Hardware Benchmarking of Autonomous Aerial Robots, 2019
    • an AI research platform for benchmarking algorithm-hardware performance and energy efficiency trade-offs.
    • quality-of-flight (QoF) metrics
    • AirSim for sim and crazyflie for real
  • RotorS---A Modular Gazebo MAV Simulator Framework, 2016
  • FlightGoogles, 2019
  • Flightmare: A Flexible Quadrotor Simulator, 2021
  • A Survey of UAV Simulation With Reinforcement Learning, blog

Flapping wing Obstacle avoidance

• Optical Flow on a Flapping Wing Robot, 2009
  • Optical Flow, indoor
  • 7 gram commercially available ornithopter airframe
  • on-board camera and CPU module with mass of 2.5 grams and 2.6 gram battery
  • black-and-white at 160x120
  • low resolution such as 18x13 is adequate
  • optic flow vectors are oscillating at around 11-12Hz
  • he pitch range induced by flapping is estimated to be ±5◦
• First Autonomous Multi-Room Exploration with an Insect-Inspired Flapping Wing Vehicle, 2018
  • 4 g stereo vision system
  • DelFly Explorer
• Stereo Vision for Flapping Wing MAVs: Design of an Obstacle Avoidance System, 2012
  • S. Tijmons, Master Thesis, TU Delft
  • literature in the field of computational stereo vision.
  • the first time it has been investigated what the requirements are for a stereo vision system to do successful stereo vision-based obstacle avoidance on FWMAVs
  • development of a systematical way to use the 3D information extracted by the stereo vision algorithm in order to find a guaranteed collision-free flight path.
  • giving an indication on the usefulness of stereo vision based on multiple experiments
  • The DelFly II successfully avoided the walls in an indoor office space of 7.3×8.2m for more than 72 seconds
• A Tailless Flapping Wing MAV Performing Monocular Visual Servoing Tasks, 2020
  • Monocular
  • ∼30-gram tailless flapping wing robot
• Obstacle Avoidance Strategy using Onboard Stereo Vision on a Flapping Wing MAV, 2017
  • stereo vision
  • 20-gram flapping wing MAV DelFly Explorer
• The ROBUR project: towards an autonomous flapping-wing animat, 2006