# RobLearn

**Repository Path**: hitall888/RobLearn

## Basic Information

- **Project Name**: RobLearn
- **Description**: No description available
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2020-09-10
- **Last Updated**: 2020-12-19

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# Roblearn
Deep learning of a mobile robot equipped with a laser scanner and a RGB-D camera to navigated in unknown environments.

![Alt text](images/RoblearnRoboterplattform.png?raw=true "Robot")

Deep Reinforcement Learning has been successfully applied in various computer games. But it is still rarely used in real world applications especially for the navigation and continuous control of real mobile robots. Previous approaches lack of safety and robustness and/or need environmental interventions. In this project, we present our proof of concept to learn robot navigation in an unknown environment for a real robot without a map or planer. The input for the robot is only the fused data from a 2D laser scanner and a RGB-D camera as well as the orientation to the goal. The map of the environment is unknown. The output actions of an Asynchronous Advantage Actor-Critic network (GA3C) are the linear and angular velocities for the robot. The  navigator/controller network is pretrained in a very fast, parallel, self implemented simulation environment to speed up the learning process and then deployed to the real robot. To avoid over fitting we train relative small networks, and we add random Gaussian noise to the input laser data. The sensor data fusion with the RGB-D camera allows the robot to navigate in real environments with real 3D obstacle avoidance and without environmental interventions. To further increase the robustness we use different levels of difficulties of the environment and train 32 of them simultaneously.

# Video
Youtube https://www.youtube.com/watch?v=KyA2uTIQfxw, https://www.youtube.com/watch?v=skpUU6ggbCo

# Paper (PDF)
https://arxiv.org/abs/2005.13857

# Installation
The software was developed under Ubuntu 16.04 with tensorflow (GPU and CPU works)

clone the Repo
```
git clone https://github.com/RoblabWh/RobLearn
```

and install dependencies

```
sudo apt-get install libeigen3-dev libboost-python-dev libboost-system-dev python3-dev inkscape python3-pip gnuplot-qt
```

For the 2D simulation do 

```
cd NeuronalNetwork 
bash build.sh
(or bash build_avx.sh if our processor support AVX)
```
For the installation of Cuda 9.0 see also cuda developer + libcudnn -> nvidia

```
sudo dpkg -i libcudnn7_7.3.1.20-1+cuda9.0_amd64.deb
sudo dpkg -i libcudnn7-dev_7.3.1.20-1+cuda9.0_amd64.deb 
```

```
sudo -H pip3 install tensorflow-gpu keras numpy
```

If you can run example_dqn.py it works!!!

```
python3 example_dqn.py
```

__Hint: Start the programm in a terminal since gnuplot blocked the desktop.__

# Training
```
./_ga3c_clean.sh &&
./_ga3c_train.sh
```
The repository contains also some intermediate step with different architectures and simulation environements (Gazebo, torse). Developers can check the directories. The directory ROS contains the software to run the trained network at a real robot e.g. turtle bot 2.

 # Dependencies
- tensorflow
- keras
- gnuplot-x11 (Visualierung)

![Alt text](images/simulation.png?raw=true "Simulation")

_LEVEL Modus_

The modes control different environments. See worlds for details.

- "test"       --> set_mode(Mode.ALL_COMBINATION, terminate_at_end=True) (easy)      -- Must work
- "diff_forms" --> set_mode(Mode.PAIR_COMBINATION, terminate_at_end=True)(easy)      -- Must work
- "roblab"     --> set_mode(Mode.ALL_COMBINATION, terminate_at_end=True) (medium)    -- Start is hard
- "room"       --> set_mode(Mode.ALL_RANDOM, terminate_at_end=False)     (hard)      -- Success would be very nice
- "four_rooms" --> set_mode(Mode.ALL_RANDOM, terminate_at_end=False)     (very hard) -- Challenger

__Issues:__
- For ubuntu 18.04 change python 3.5 to 3.6 and (python-py35 -> python.py36) in Simulation2d/CMakeLists.txt
- On some laptops sometimes gnuplot crashed (rarly).


# Citation
https://arxiv.org/abs/2005.13857

# Credits
Nvidia for GA3C, Google for tensorflow, Keras, Gnuplot, Ubuntu