Xavier Geerinck

In Part 1 I explained you how you can create a Robot Arm that can be mounted on an Xbox controller. In OpenAI Lunar Lander I then continued to see how we are able to train a lunar lander environment to land a lunar lander all by itself with a continuous action space.

For an upcoming blog post, I would like to have a robotic arm to land a Lunar Lander autonomously. In Part 1 I explained how we can build such a robotic arm already, but now we need to be able to go deeper into how we are able to train an environment in a simulation environment (before deploying it on a physical device).

Creating an Xbox Robot Arm is something that I've been wanting to do ever since I saw the post by Kevin Drouglazet who was able to utilize an Xbox Controller with an arm he created and was so friendly to publish the design files for (I am definitely not a hardware designer 😅) - thanks for that Kevin!

In my last article about ML-Agents I went through an introduction of what it was and how we can use it. Now I wanted to get some more hands-on experience and found this awesome course on Unity Learn that went in detail on how to create a Hummingbird. After following this course, I wanted to do my own and work on a more industrial focused example.

Seeing the rising popularity of the ML-Agents framework and them having achieved Release 2 - congratulations! - I decided to start by running one of the examples and documenting it.

In a previous blog post I introduced the concept of creating a Digital Twin representation utilizing Dapr's Virtual Actors. Now, this was a theoretical post on how feasible this could be. Of course anything theoretical should also be put in practice, which is what I have been doing this weekend 😉.