Guidance on how to create a private container


#1

Hey there,
I have already seen this piece of documentation: https://paperspace.zendesk.com/hc/en-us/articles/360003415434-Private-containers-and-workspaces

But unfortunately I’m quite new to the Docker-game, so it feels overwhelming for me to mirror my environment in a Docker container. So it would be nice If anybody could give me some hints about how to:

  1. Derive a Docker-Image from nvidia-docker (the article above said this is a good practice)
  2. Install my necessary pip3 python dependencies there
  3. Make this Docker-Image accessible somewhere so I can use it with Paperspace.

And one extra question: Should I fetch my codebase & training-data either within this container or should I do this in a run.sh and leave my container free from code/data? What are some best practices here?

Btw I am on macOS if that matters.

Regards from Germany,
Dennis


#2

Hi @wottpal great questions. You are spot on with the steps above. For example, here is the dockerfile for our Fast.ai template: https://github.com/Paperspace/fastai-docker/blob/master/Dockerfile . You can see that we do steps like pip install, etc.

You could pull in your code using a git pull which is a totally fine practice, but I would not recommend pulling in the dataset in to the container generally (unless it is really small, say <1Gb). You should get your data in to the /storage directory where it will be auto-mounted on every job.

Let me know if that helps!