Skip to content →

The Progress of DeepL2: Robot moved forward!

Hey, we made a progress!

It’s coord.e.

Forward Movement

This is the result of learning! We did this!

The method we took is far different from that of what I wrote in the previous post.

This time, I’m going to introduce how we came to be able to produce this good result.

Very Long Fighting with Gazebo&ROS&ODE

I wrote that we used Gazebo&ROS for simulation.  Arms were broken and sank under the ground then,,,

I used ‘bullet’ as a physics engine, however, I realized that ODE which is default of Gazebo may be better and changed to it.

Then,

sometimes unsuccessful, however

Since we adjusted physical parameter for long time, standing still has achieved like above.
However simulation still looks weird↓

_人人人人人人人人人_
> parallel transition <
 ̄Y^Y^Y^Y^Y^YY^Y^ ̄

We had been in trouble for two month because of this weird behavior.

Physical parameters got more unrealistic.

Schedules were so tight, and the time of presentation got closer and closer.

I once thought this won’t work anymore, but finally, I cleared out the factor.

It was the bug of Gazebo.

It was the known bug happens when integrating with ROS. This bug make simulation moon-like gravity.

Well, with the reason, It was easy to fix this.

I’ll dig into this bug in the another post.(The patch is already uploaded on github)

After my fix, realistic gravity shown on the right side of the movie below had came back.

Although the weird gravity had fixed, however, Its actions were unrealistic at all.

Use bullet directly

I had an idea not to use Gazebo, but to use another physics engine.

I chose pyBullet because it is easy to load URDF models and integrate with machine learning. (I also considered MuJoCo, but I felt loading URDF may be going to bother me)

And the simulation looks like this below:

Very lightweight, and realistic simulation. This was simply fantastic.

Also, I could find the reinforcement example in ‘examples’ directory. ↓

I wrote the program to perform learning and performed learning for over a night.

The result of it was that.

Reward moved like this. Looks like it learns to go forward properly isn’t it

moving-average graph

Summary

Yeah, YamaX in simulation went forward.

I pushed the source code of this onto GitHub:

https://github.com/Y-modify/deepl2-pybullet-locomotion

I’m going to post an article about this learning program.

Thank you for reading and See you again.

Published in 3D model DeepL2 YamaX Software Machine Learning apologize progress