How to run OpenAI Gym on Windows and with Javascript

Reinforcement learning does not only requires a lot of knowledge about the subject to get started, it also requires a lot of tools to help you test your ideas. Since this process is quite lengthy and hard, OpenAI helped us with this. By creating something called the OpenAI Gym, they allow you to get started developing and comparing reinforcement learning algorithms in an easy to use way.

Now since setting up the OpenAI Gym with python is quite easy to do (just follow their tutorial), I decided to make things more difficult and want to run the OpenAI Gym using Javascript on a Windows machine. So let’s get started!


For this blog, we need several components installed upfront to make our lives easier:

Installing our dependencies

  1. Install Xming for Windows: https://sourceforge.net/projects/xming/

  2. Open up your WSL and run the following commands:

# Install Python + Dependencies
sudo apt-get install -y python-dev
sudo apt-get install -y python-pip
sudo apt-get install -y python-numpy python-dev cmake zlib1g-dev libjpeg-dev xvfb xorg-dev python-opengl libboost-all-dev libsdl2-dev swig

sudo pip install werkzeug
sudo pip install itsdangerous
sudo pip install click

# Export our display settings for XMing
export DISPLAY=localhost:0.0
echo 'export DISPLAY=localhost:0.0 ' >> ~/.bashrc

Setting up our OpenAI Gym

Once we are done installing the dependencies, we can continue by installing our OpenAI Gym. Since we are using Javascript here, we will be using the OpenAI Gym Http Bindings. For this, run the following commands to clone and set up our requirements for the Javascript Bindings.

# OpenAI Gym
git clone https://github.com/openai/gym-http-api
cd gym-http-api
sudo pip install -r requirements.txt
sudo pip install -e '.[all]'
cd binding-js
npm install

Running dev server

Once you installed everything correctly, you can start the OpenAI server through the following commands:

cd gym-http-api
python gym_http_server.py

and test it with the following script in a different terminal:

cd gym-http-api/bindings-js/dist/examples
node exampleAgent.js

This should open up an Xming display running a cartpole example.

Note: Make sure that your Xming display server is started on windows and that it is running in your taskbar!



Xavier Geerinck

Xavier works as a Cloud Solution Architect at Microsoft, helping its customer unlock the full potential of the cloud. Even though he is still considered a young graduate, he achieved his first success at the age 16, by creating and selling his first startup. He then took this knowledge to create and help more startups in different markets such as technology, social media, philanthropy and home care. While in the meantime gaining more enterprise insights at renowned enterprises such as Nokia, Cisco and now Microsoft.

Read More