Physics control tasks with Deep Reinforcement Learning

In this tutorial we will implement the paper Continuous Control with Deep Reinforcement Learning, published by Google DeepMind and presented as a conference paper at ICRL 2016. The networks will be implemented in PyTorch and using OpenAI gym. The algorithm combines Deep Learning and Reinforcement Learning techniques to deal with high-dimensional, i.e. continuous, action spaces.

This is a companion discussion topic for the original entry at
1 Like

Hi Antonio,
Great job man, it’s really helped me to get through the RL as I’m new to this concept and in general field of AI.
So, I faced some issue when tried to run the code on the CPU. Sadly, I don’t have an external GPU to run the process on. :expressionless:
Anyway, here is the output of the last block of code:

NotImplementedError                       Traceback (most recent call last)
<ipython-input-24-97b79fb15b62> in <module>
     31         a += noise()*max(0, epsilon)
     32         a = np.clip(a, -1., 1.)
---> 33         s2, reward, terminal, info = env.step(a)

~/anaconda3/envs/lstm/lib/python3.7/site-packages/gym/ in step(self, action)
    284     def step(self, action):
--> 285         return self.env.step(self.action(action))
    287     def action(self, action):

~/anaconda3/envs/lstm/lib/python3.7/site-packages/gym/ in action(self, action)
    287     def action(self, action):
--> 288         raise NotImplementedError
    290     def reverse_action(self, action):


I tried to adjust the code to run on CPU :grin: and succeeded to fix some of it but apparently there should be more to consider. :sweat_smile: Any suggestions highly appreciated. :v:

hello, I have the same issue with you about the NotImplementedError:, how did you fix it?

Hi Antonio,
I started reading your post and it started really good. Unfortunately, all the formulae are mnissing. All the stuff that is stored on… is obviously missing.

e. g.

See this screenshot:


Any idea?


Hi Antonio,
Thanks for your great job , but some important pictures are missing because of the 404 error, which are from the title “Reword function”
See this screenshot:

Thanks for sharing this awesome article ! I could apply DDPG to my research thanks to your simple and kind explanation.

For those who suffer from NotImpelementedError, one can simply fix this error by changing the names of two methods in the NormalizedEnv class. (This error comes from using different version of Gym)

  • _actionaction
  • _reverse_actionreverse_action
1 Like