January 5, 2021 | ai ai-ml ai-rl
Implementing Deepmind's MuZero Algorithm with Python
Deepmind has achieved a huge milestone by publishing its latest paper around Reinforcement Learning in Nature - 23/DEC/2020. How they were able to train a Reinforcement Learning algorithm that masters Go, Chess, Shogi and Atari without needing to be told the rules.
October 21, 2020 | ai ai-ml
Auto scaling a HTTP triggered application in Kubernetes using Keda
Kubernetes is getting more popular everyday and it's no wonder why! When you are running applications on-premise or in-cloud, the possibility of having the applications in a portable way is a strong one! Removing the friction for scaling-out your application when you are ready for it, or even bursting scenarios.
October 20, 2020 | ai ai-ml
Creating a simple "Hello World" docker application with Express and NodeJS
For a lot of infrastructure related blog posts I would like to utilize a very basic application that spins up a web server and allows us to access it. Once we access it we simple see a welcome message but with a delay that we can configure!
September 20, 2020 | ai ai-ml
Monitoring the Kubernetes Nginx Ingress Controller with Prometheus and Grafana
In a previous article I explained how we can set-up an Nginx Kubernetes Ingress Controller, but how can we now monitor this? This is what I would like to tackle in this article, on how we are able to utilize Prometheus and Grafana to start visualizing what is happening on our Ingress Controller.
September 20, 2020 | ai ai-ml
Creating a Kubernetes Nginx Ingress Controller and create a rule to a sample application
Whenever you are creating an application that you want to expose to the outside world, it's always smart to control the flow towards the application behind it. That's why Kubernetes has something called Kubernetes Ingress. But what is it?
June 19, 2020 | coding ai ai-rl iot
Setting up Stable Baselines with Python 3.7 and WSL 2
I've been playing around a bit with Stable Baselines lately for an upcoming project. However while playing, I encountered some issues seeing that Stable Baselines currently requires Tensorflow <= 1.14 which is only supported through Python < 3.8. Due those difficulties, I thought it would be interesting to share how I set up my personal development environment from scratch to play around with Stable Baselines.
June 8, 2020 | coding ai ai-rl iot
Autonomously Landing a Lunar Lander with an Xbox Controller Robotic Arm - Part 2
In Part 1 I explained you how you can create a Robot Arm that can be mounted on an Xbox controller. In OpenAI Lunar Lander I then continued to see how we are able to train a lunar lander environment to land a lunar lander all by itself with a continuous action space.
June 7, 2020 | coding ai ai-rl iot
Training the Continuous Lunar Lander with Reinforcement Learning, RLLib and PPO
For an upcoming blog post, I would like to have a robotic arm to land a Lunar Lander autonomously. In Part 1 I explained how we can build such a robotic arm already, but now we need to be able to go deeper into how we are able to train an environment in a simulation environment (before deploying it on a physical device).
May 30, 2020 | coding ai ai-rl iot
Autonomously Landing a Lunar Lander with an Xbox Controller Robotic Arm - Part 1
Creating an Xbox Robot Arm is something that I've been wanting to do ever since I saw the post by Kevin Drouglazet who was able to utilize an Xbox Controller with an arm he created and was so friendly to publish the design files for (I am definitely not a hardware designer 😅) - thanks for that Kevin!
May 27, 2020 | ai ai-ml ai-rl
Unity ML-Agents - Creating a Robot Picker
In my last article about ML-Agents I went through an introduction of what it was and how we can use it. Now I wanted to get some more hands-on experience and found this awesome course on Unity Learn that went in detail on how to create a Hummingbird. After following this course, I wanted to do my own and work on a more industrial focused example.
May 5, 2020 | ai ai-ml ai-rl
Reinforcement Learning with the Bonsai Platform
The Bonsai Machine Teaching platform has been released! Promising an easy to use environment for end-to-end Reinforcement Learning projects, starting with simulator selection / integration to algorithm configuration and training.
April 15, 2020 | coding ai ai-rl iot
A Multi-Language Reinforcement Learning Digital Twin Environment
One of the ideas I have been playing around with the last couple of months is the combination of Digital Twins and Reinforcement Learning. This is an experimental idea where I would love to hear your opinions about it (feel free to comment below, send me an email or reach out to me on Social Media such as Twitter / LinkedIn), and that will be refined over the coming months.
February 24, 2020 | ai ai-ml
AI Battlecards - End to End Process for building and evaluating AI models
A couple of months ago it struck me that my personal knowledge in AI could be improved quite a bit. That's why I took the time to brush up on the different concepts ranging from data gathering until the evaluation of a deployed model.
January 3, 2020 | ai ai-ml
Artificial Intelligence - How to measure performance - Accuracy, Precision, Recall, F1, ROC, RMSE, F-Test and R-Squared
We currently see a lot of AI algorithms being created, but how can we actually measure the performance of these models? What are the terms we should look at to detect this?
October 30, 2019 | azure ai ai-rl ai-ml
Facebook ReAgent - An End-to-End Use Case
Facebook decided to release their end-to-end applied reinforcement learning platform called ReAgent, after reading their vision on this, I have to say that I am completely hooked! They are providing an excellent view of Reinforcement Learning and the future adoption of it. But why is this and how can we get started with it?
December 2, 2018 | ai ai-ml ai-rl
Facebook's Open-Source Reinforcement Learning Platform - A Deep Dive
Facebook decided to open-source the platform that they created to solve end-to-end Reinforcement Learning problems at the scale they are working on. So of course I just had to try this ;) Let's go through this together on how they installed it and what you should do to get this working yourself.
November 29, 2018 | ai ai-ml ai-rl
Reinforcement Learning - Terminology
August 10, 2018 | ai ai-ml ai-rl
Writing a C# SDK for the OpenAI Gym using .NET Core
When we take a look at the OpenAI Gym on Github (https://github.com/openai/gym-http-api), we see that it does not have bindings available for C#. Now since I am a firm believer of .NET Core and what it brings to developer ecosystem, I decided to write one myself (https://github.com/Xaviergeerinck/dotnetcore-sdk-openai). Using what I learned in my previous blog post How to write a SDK in dotnet Core I created one that looks like this for the main method:
June 10, 2018 | ai ai-ml ai-rl
OpenAI Gym Problems - Solving the CartPole Gym
In a previous post we set-up the OpenAI Gym to interface with our Javascript environment. Let's now look at how we can use this interface to run the CartPole example and solve it with the theory that we learned in previous blog posts.
May 24, 2018 | ai ai-ml ai-rl
Bellman Equations
Summary
May 7, 2018 | ai ai-ml ai-rl
Installing OpenAI Gym in a Windows Environment
Reinforcement learning does not only requires a lot of knowledge about the subject to get started, it also requires a lot of tools to help you test your ideas. Since this process is quite lengthy and hard, OpenAI helped us with this. By creating something called the OpenAI Gym, they allow you to get started developing and comparing reinforcement learning algorithms in an easy to use way.
May 3, 2018 | ai ai-ml ai-rl
Multi-armed bandit framework
To start solving the problem of exploration, we are going to introduce the Multi-armed bandits framework. But what exactly does this solve? Just think that you are executing a clinical trial with 4 pills. You know that the pills have a survival rate but you don't know what that survival rate is. Your goal: find the pill with the highest chance of survival in X trials.
May 1, 2018 | ai ai-ml ai-rl
An introduction to Reinforcement Learning (RL)
So as we learned in the intro to Machine Learning, Reinforcement Learning is this technique where we have an agent who will take specific actions on an environment to try to reach an optimal state. But how can we illustrate this? Take a look at the following picture.
April 29, 2018 | ai ai-ml
An introduction to Machine Learning (ML)
Every few decennia there is a new cool kid around the block that makes an impact on the current world, an impact that is so big that we will have to adapt. Starting with the introduction of mainframe computing in 1950, the pc in 1975, the internet in 1980 and the introduction of mobile phones in 2007, towards the current age of Big Data which started in 2012.