Julia using statement fails in Gradient notebook


#1

I am trying to run some Julia code on a Gradient notebook. I am using the Jupyter Notebook Data Science Stack container.

I use Pkg.add() to add needed packages to the base Julia installation (e.g., Flux, JuMP).

When I evaluate the statement “using [package name],” I get an error in the pre-compiling step.

For example, when I try “using Flux,” I get:

INFO: Precompiling module Flux.
ERROR: LoadError: LoadError: Failed to precompile MacroTools to /opt/julia/lib/v0.6/MacroTools.ji.
Stacktrace:
[1] compilecache(::String) at ./loading.jl:710
_[2] require(::Symbol) at ./loading.jl:463
[3] require(::Symbol) at ./loading.jl:405
[4] include_from_node1(::String) at ./loading.jl:576
[5] include(::String) at ./sysimg.jl:14
[6] include_from_node1(::String) at ./loading.jl:576
[7] include(::String) at ./sysimg.jl:14
[8] anonymous at ./:2
while loading /opt/julia/v0.6/Media/src/system.jl, in expression starting on line 1
while loading /opt/julia/v0.6/Media/src/Media.jl, in expression starting on line 6
ERROR: LoadError: Failed to precompile Media to /opt/julia/lib/v0.6/Media.ji.
Stacktrace:
[1] compilecache(::String) at ./loading.jl:710
_[2] require(::Symbol) at ./loading.jl:463
[3] require(::Symbol) at ./loading.jl:405
[4] include_from_node1(::String) at ./loading.jl:576
[5] include(::String) at ./sysimg.jl:14
[6] anonymous at ./:2
while loading /opt/julia/v0.6/Juno/src/Juno.jl, in expression starting on line 5
ERROR: LoadError: Failed to precompile Juno to /opt/julia/lib/v0.6/Juno.ji.
Stacktrace:
[1] compilecache(::String) at ./loading.jl:710
_[2] require(::Symbol) at ./loading.jl:463
[3] require(::Symbol) at ./loading.jl:405
[4] include_from_node1(::String) at ./loading.jl:576
[5] include(::String) at ./sysimg.jl:14
[6] anonymous at ./:2
while loading /opt/julia/v0.6/Flux/src/Flux.jl, in expression starting on line 7
Failed to precompile Flux to /opt/julia/lib/v0.6/Flux.ji.
Stacktrace:
[1] compilecache(::String) at ./loading.jl:710
_[2] require(::Symbol) at ./loading.jl:497
[3] require(::Symbol) at ./loading.jl:405

Trying to use other added packages leads to a similar output. Just to check, I also tried using a package that supposedly comes pre-added to the julia installation in the container (Gadfly). This also fails:

INFO: Precompiling module Gadfly.
ERROR: LoadError: Failed to precompile SortingAlgorithms to /opt/julia/lib/v0.6/SortingAlgorithms.ji.
Stacktrace:
_ [1] compilecache(::String) at ./loading.jl:710_
_ [2] require(::Symbol) at ./loading.jl:463
_ [3] require(::Symbol) at ./loading.jl:405_
_ [4] include_from_node1(::String) at ./loading.jl:576_
_ [5] include(::String) at ./sysimg.jl:14_
_ [6] anonymous at ./:2_
while loading /opt/julia/v0.6/StatsBase/src/StatsBase.jl, in expression starting on line 9
ERROR: LoadError: Failed to precompile StatsBase to /opt/julia/lib/v0.6/StatsBase.ji.
Stacktrace:
_ [1] compilecache(::String) at ./loading.jl:710_
_ [2] require(::Symbol) at ./loading.jl:463
_ [3] require(::Symbol) at ./loading.jl:405_
_ [4] include_from_node1(::String) at ./loading.jl:576_
_ [5] include(::String) at ./sysimg.jl:14_
_ [6] anonymous at ./:2_
while loading /opt/julia/v0.6/DataArrays/src/DataArrays.jl, in expression starting on line 6
ERROR: LoadError: Failed to precompile DataArrays to /opt/julia/lib/v0.6/DataArrays.ji.
Stacktrace:
_ [1] compilecache(::String) at ./loading.jl:710_
_ [2] require(::Symbol) at ./loading.jl:463
_ [3] require(::Symbol) at ./loading.jl:405_
_ [4] include_from_node1(::String) at ./loading.jl:576_
_ [5] include(::String) at ./sysimg.jl:14_
_ [6] anonymous at ./:2_
while loading /opt/julia/v0.6/Gadfly/src/Gadfly.jl, in expression starting on line 8
Failed to precompile Gadfly to /opt/julia/lib/v0.6/Gadfly.ji.

Stacktrace:
_ [1] compilecache(::String) at ./loading.jl:710_
_ [2] require(::Symbol) at ./loading.jl:497
_ [3] require(::Symbol) at ./loading.jl:405_

I brought this up with Paperspace support, and they asked me to check my code and post the issue here. If anyone can see an error in the code statement:

using Gadfly

Please let me know. Otherwise, it seems that there is a problem with Julia in the Jupyter Notebook Data Science Stack container.


#2

What type of machine are you running on? I tried on a K80 and a C7 instance and got the following both times:

It did take about 5 minutes to compile which is odd and the CPU was pegged at 100%. Not being a Julia expert I’m not sure if this is expected.

Confirmed a simple plot worked as well for my test:


#3

Hi, and thanks for the reply. I have confirmed it also works for me on a C7 instance. I was originally using a C2 instance, and I just tried it again, and I got similar errors. So maybe something is wrong with using a C2? I was hoping to use a low powered instance to work on code, and use the bigger instances when I needed to have some throughput.