Python and Julia in harmony.

Overview

PythonCall.jl logo
PythonCall & JuliaCall

Project Status: Active – The project has reached a stable, usable state and is being actively developed. Stable Documentation Dev Documentation Test Status Codecov

Bringing Python® and Julia together in seamless harmony:

  • Call Python code from Julia and Julia code from Python via a symmetric interface.
  • Simple syntax, so the Python code looks like Python and the Julia code looks like Julia.
  • Intuitive and flexible conversions between Julia and Python: anything can be converted, you are in control.
  • Fast non-copying conversion of numeric arrays in either direction: modify Python arrays (e.g. bytes, array.array, numpy.ndarray) from Julia or Julia arrays from Python.
  • Helpful wrappers: interpret Python sequences, dictionaries, arrays, dataframes and IO streams as their Julia couterparts, and vice versa.
  • Beautiful stack-traces.
  • Supports modern systems: tested on Windows, MacOS and Linux, 64-bit, Julia 1.4 upwards and Python 3.5 upwards.

If you like this, a GitHub star would be lovely thank you.

To get started, read the documentation.

Example 1: Calling Python from Julia

In this example, we use the Julia module PythonCall from a Pluto notebook to inspect the Iris dataset:

  • We load the Iris dataset as a Julia DataFrame using RDatasets.
  • We use pytable(df) to convert it to a Python Pandas DataFrame.
  • We use the Python package Seaborn to produce a pair-plot, which is automatically displayed.

Seaborn example screenshot

Example 2: Calling Julia from Python

In this example we use the Python module JuliaCall from an IPython notebook to train a simple neural network:

  • We generate some random training data using Python's Numpy.
  • We construct and train a neural network model using Julia's Flux.
  • We plot some sample output from the model using Python's MatPlotLib.

Flux example screenshot

What about PyCall?

The existing package PyCall is another similar interface to Python. Here we note some key differences, but a more detailed comparison is in the documentation.

  • PythonCall supports a wider range of conversions between Julia and Python, and the conversion mechanism is extensible.
  • PythonCall by default never copies mutable objects when converting, but instead directly wraps the mutable object. This means that modifying the converted object modifies the original, and conversion is faster.
  • PythonCall does not usually automatically convert results to Julia values, but leaves them as Python objects. This makes it easier to do Pythonic things with these objects (e.g. accessing methods) and is type-stable.
  • PythonCall installs dependencies into a separate conda environment for each Julia project. This means each Julia project can have an isolated set of Python dependencies.
  • PythonCall supports Julia 1.0+ and Python 3.5+ whereas PyCall supports Julia 0.7+ and Python 2.7+.
Comments
  • TagBot trigger issue

    TagBot trigger issue

    This issue is used to trigger TagBot; feel free to unsubscribe.

    If you haven't already, you should update your TagBot.yml to include issue comment triggers. Please see this post on Discourse for instructions and more details.

    If you'd like for me to do this for you, comment TagBot fix on this issue. I'll open a PR within a few hours, please be patient!

    opened by JuliaTagBot 25
  • LoadError: InitError during Package Build

    LoadError: InitError during Package Build

    Thanks again for this package!

    I'm currently trying to build my recently released package Anneal.jl and this error is showing up during precompilation. One must run ] precompile to see the message below:

    ERROR: LoadError: InitError: failed process: Process(`'C:\Users\pedroxavier\.julia\artifacts\d628738c351341856bc6e8abe94cd6c8d8a7bf66\Library\bin\micromamba.exe' -r 'C:\Users\pedroxavier\.julia\scratchspaces\0b3b1443-0f03-428d-bdfb-f27f9c1191ea\root' create -y -p 'C:\Users\pedroxavier\.julia\environments\v1.7\.CondaPkg\env' --override-channels --no-channel-priority pip 'python >=3.5,<4' 'python >=3.7,<4' -c anaconda -c conda-forge`, ProcessExited(3221226505)) [3221226505]
    

    A way to fix this is to plug the command between backticks in a Julia run call.

    run(`'C:\Users\pedroxavier\.julia\artifacts\d628738c351341856bc6e8abe94cd6c8d8a7bf66\Library\bin\micromamba.exe' -r 'C:\Users\pedroxavier\.julia\scratchspaces\0b3b1443-0f03-428d-bdfb-f27f9c1191ea\root' create -y -p 'C:\Users\pedroxavier\.julia\environments\v1.7\.CondaPkg\env' --override-channels --no-channel-priority pip 'python >=3.5,<4' 'python >=3.7,<4' -c anaconda -c conda-forge`)
    

    This way, micro mamba gets installed and everything works fine. The problem is that my package won't precompile directly without this trick. I guess it has something to do with privileges, chmod or something like that.

    To reproduce this issue, it is enough to run

    ] add Anneal
    ] precompile
    

    Anneal is available in the General Registry, so the first line will error after a little while and ] precompile will reveal the message above.

    bug priority 
    opened by pedromxavier 22
  • pyclass

    pyclass

    I'm trying to implement a Python class using PythonCall.pyclass.

    The following works as expected:

    using PythonCall
    
    Person = pyclass("Person", (); print_random = print ∘ rand)
    @pyexec (Person = Person) => """
    p1 = Person()
    p1.print_random() 
    """
    # prints a random number
    

    But any attempt to include methods that accept arguments referencing the instance of the class itself:

    Person = pyclass("Person", (); __init__ = (self) -> (self.state = 0))
    @pyexec (Person = Person) => """
    p1 = Person()
    """
    

    fails:

    ERROR: Python: TypeError: Julia: MethodError: no method matching (::var"#5#6")()
    Closest candidates are:
      (::var"#5#6")(!Matched::Any) at REPL[15]:1
    Python stacktrace:
     [1] __call__
       @ /home/yakir/.julia/packages/PythonCall/Z6DIG/src/jlwrap/any.jl:167:30
     [2] <module>
       @ REPL[16]:1:1
    Stacktrace:
     [1] pythrow()
       @ PythonCall ~/.julia/packages/PythonCall/Z6DIG/src/err.jl:94
     [2] errcheck
       @ ~/.julia/packages/PythonCall/Z6DIG/src/err.jl:10 [inlined]
     [3] pycallargs(f::Py, args::Py)
       @ PythonCall ~/.julia/packages/PythonCall/Z6DIG/src/abstract/object.jl:153
     [4] pycall(::Py, ::Py, ::Vararg{Py}; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
       @ PythonCall ~/.julia/packages/PythonCall/Z6DIG/src/abstract/object.jl:171
     [5] pycall
       @ ~/.julia/packages/PythonCall/Z6DIG/src/abstract/object.jl:161 [inlined]
     [6] #_#11
       @ ~/.julia/packages/PythonCall/Z6DIG/src/Py.jl:330 [inlined]
     [7] Py
       @ ~/.julia/packages/PythonCall/Z6DIG/src/Py.jl:330 [inlined]
     [8] pyexec(::Type{Nothing}, code::Py, globals::Module, locals::NamedTuple{(:Person,), Tuple{Py}})
       @ PythonCall ~/.julia/packages/PythonCall/Z6DIG/src/concrete/code.jl:98
     [9] top-level scope
       @ ~/.julia/packages/PythonCall/Z6DIG/src/concrete/code.jl:146
    

    More context

    I'm trying to use the picamera Python module and process each frame in Julia. The following MWE however segfaults:

    ENV["JULIA_PYTHONCALL_EXE"] = "/usr/bin/python3" # I have to include this due to https://github.com/cjdoris/PythonCall.jl/issues/120
    
    using PythonCall
    
    SideEffect = pyclass("SideEffect", (), 
                       __init__ = () -> nothing,
                       write = s -> println(rand()),
                       flush = () -> print("done")
                      )
    
    @pyexec (SideEffect = SideEffect) => """
    import picamera
    with picamera.PiCamera() as camera:
        camera.start_recording(SideEffect(), format='h264') # start_recording calls SideEffect.write for each frame and SideEffect.flush at the end
        camera.wait_recording(5)
        camera.stop_recording()
    """
    
    opened by yakir12 22
  • Add example of fast non-copying conversion of numeric arrays to docs

    Add example of fast non-copying conversion of numeric arrays to docs

    It would be useful to have an example of "Fast non-copying conversion of numeric arrays" somewhere in the documentation. For example, I've never been able to figure out how to do this. I imagine some others have not either.

    More specifically, I'd like to pass a numpy array from Python to a Julia function where the Julia function receives the data as an Array with no copying.

    opened by jlapeyre 13
  • julia thread hangs with multiple sevals

    julia thread hangs with multiple sevals

    Hi, I can reproduce a hang (no cpu activity) with the following code on 1.8rc1, 1.8rc3 & 1.9dev. I'm running on Ubuntu 21.10, but I can also reproduce it on Apple M1.

    Curiously, but only in 1.8rc1, the hang does not occur if I uncomment the println(42). I could not reproduce the hang directly on julia. On my real code I tried merging the 2 seval calls, but it still freezes with 1.

    It's very important to enable parallelism setting env vars JULIA_NUM_THREADS=6

    from juliacall import Main as jl
    
    jl.seval(
        """
        function worker()
                for i in 1:typemax(Int64)
                    a = Float64[]
                    push!(a, 0.42)
                    i % 1000 == 0 && println(i)
                end
        end
    """
    )
    jl.seval(
        """
    begin
    #println(42) #this fixes hang only on 1.8rc1
    t = Threads.@spawn worker()
    println("waiting")
    wait(t)
    end
    """
    )
    
    

    On my computer it hangs after printing 30200

    ....
    301000
    302000
    

    The code runs fine also with a single eval

    from juliacall import Main as jl
    
    jl.seval(
        """
    begin
        function worker()
                for i in 1:typemax(Int64)
                    a = Float64[]
                    push!(a, 0.42)
                    i % 1000 == 0 && println(i)
                end
        end
    
    #println(4) #this fixes hang only on 1.8rc1
    t = Threads.@spawn worker()
    println("waiting")
    wait(t)
    end
    """
    )
    
    

    thanks

    opened by dpinol 11
  • Error when using PythonCall attached to a previous Conda environment.

    Error when using PythonCall attached to a previous Conda environment.

    Hello, I would be very gratefull to understand what is the problem. I followed the instructions:

    julia > ENV["JULIA_CONDAPKG_BACKEND"] = "System"
    julia > ENV["JULIA_CONDAPKG_EXE"] = "/home/mmestre/.conda/envs/cosmos/bin/python3.9"
    

    Then I install PythonCall.

    (RiosDelCosmos) pkg> st
          Status `~/casa/work/2022/RiosDelCosmos/Project.toml`
      [6099a3de] PythonCall v0.9.6
    

    When I try to use it I get an error (I do have astropy installed in the cosmos environment):

    julia> using PythonCall
    julia> ast = pyimport("astropy")
    ERROR: Python: ModuleNotFoundError: No module named 'astropy'
    Python stacktrace: none
    Stacktrace:
     [1] pythrow()
       @ PythonCall ~/.julia/packages/PythonCall/DqZCE/src/err.jl:94
     [2] errcheck
       @ ~/.julia/packages/PythonCall/DqZCE/src/err.jl:10 [inlined]
     [3] pyimport(m::String)
       @ PythonCall ~/.julia/packages/PythonCall/DqZCE/src/concrete/import.jl:11
     [4] top-level scope
       @ REPL[5]:1
    
    

    I also tried with julia > ENV["JULIA_CONDAPKG_EXE"] = "/home/mmestre/.conda/envs/cosmos/bin/python" obtaining the same result. Thank you very much in advance.

    opened by martinmestre 10
  • How to pass a numpy array into a Julia function?

    How to pass a numpy array into a Julia function?

    Consider a dummy function inside my custom package EOS:

    function dummy(x::AbstractVector)
        @show typeof(x)
    end
    

    From Python, I call it as follows

    jl.EOS.dummy(np.array([1.1, 2.3]))
    

    and the following error occurs

    TypeError: Julia: MethodError: no method matching dummy(::PythonCall.PyIterable{PythonCall.Py})
    Closest candidates are:
      dummy(!Matched::AbstractVector{T} where T) at /home/shuhua/Work/opt/EOS/src/EOS.jl:26
    

    According to Conversion to Julia, I expect that a numpy array is converted to a Julia array automatically. Did I miss anything?

    opened by ShuhuaGao 9
  • Juliacall on Apple M1

    Juliacall on Apple M1

    Hi, thank you for PythonCall.jl/juliacall! I hope it's ok to just open an issue here. When trying to import juliacall on Apple M1 I am getting the following error:

    Python 3.9.12 (main, Apr  5 2022, 01:52:34) 
    [Clang 12.0.0 ] :: Anaconda, Inc. on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import juliacall
    fatal: error thrown and no exception handler available.
    ErrorException("could not load symbol "jl_n_threads":
    dlsym(RTLD_DEFAULT, jl_n_threads): symbol not found")
    ijl_errorf at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
    ijl_dlsym at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
    ijl_load_and_lookup at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
    nthreads at ./threadingconstructs.jl:23 [inlined]
    __preinit_threads__ at ./task.jl:706
    jfptr___preinit_threads___40244 at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/sys.dylib (unknown line)
    ijl_apply_generic at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
    _finish_julia_init at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
    julia_init at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
    ijl_init at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
    

    I would be really happy about any help with this. Thanks again! :)

    opened by nmheim 8
  • JuliaCall reinstalling PythonCall every time

    JuliaCall reinstalling PythonCall every time

    Hi,

    Two users of my Julia pkg via JuliaCall have encountered the same issue when multithreading on the Python side was involved. They were mainly Python users who haven't used Julia before. After they installed JuliaCall via pip install juliacall, they had no issue calling Julia in a serial program. However, when they used multithreading on the Python side, they told me Julia tried to repeatly install PythonCall everytime with lengthy error messages.

    The easy fix would be to manually install PythonCall in Julia. I haven't encountered this issue myself previously because I have all the pkgs already installed on the Julia side.

    Is this behaviour expected? Or should we mention this in the doc?

    opened by henry2004y 8
  • Using Python.jl and PyCall.jl together

    Using Python.jl and PyCall.jl together

    If I do:

    using Python
    using PyCall
    

    the latter segfaults (at least on 1.6-rc1). For the reverse order I do get an error.

    I don't personally need both to work, just curious if they could (probably not). That's not really the end-goal, but some packages do have PyCall as a dependency, e.g. PyPlot.jl and Pandas.jl. Is there a good way to support such, as it seems this package has a similar API as PyCall?

    opened by PallHaraldsson 8
  • libstdcxx-ng versioning

    libstdcxx-ng versioning

    Thanks for the excellent package! I recently ran into a situation where I tried to install a Python package using CondaPkg.jl that depended upon libstdcxx-ng v12 and was unable to install it due to incompatible dependencies. This resulted in some error about GLIBCXX_3.4.30 not being found. I think I could in principle resolve this by building Julia from source with a more recent version of libstdc++, but because of the lines below I don't think it would work even then. Would it be a reasonable idea to actually run the strings command and find the highest GLIBCXX version string in PythonCall?

    https://github.com/cjdoris/PythonCall.jl/blob/77ee463d149b418ed1e6b34036f50172429ae46e/src/cpython/context.jl#L67-L80

    Also, the link in the comment on L70 doesn't contain GLIBCXX_3.4.30, however I do see it mentioned here: https://gcc.gnu.org/onlinedocs/libstdc++/manual/abi.html

    opened by kllrak 7
  • Error when importing matplotlib and JuliaCall from Python

    Error when importing matplotlib and JuliaCall from Python

    I'm on Red Hat Enterprise Linux 8.6, running Python 3.9.7. I haven't installed Julia, but juliapkg appears to download and install Julia 1.8.4 just fine. I've created and activated a virtual environment, then installed matplotlib and juliacall:

    $ virtualenv -p `which python3.9` venv
    $ source ./venv/bin/activate
    $ pip install matplotlib juliacall
    

    In my working directory I have one Julia file:

    $ cat Foo.jl
    struct Foo end
    $
    

    and two Python files. The first Python file includes the Foo.jl file:

    $ cat foo.py
    import os
    
    # Create a new Julia module that will hold all the Julia code imported into this Python module.
    import juliacall; jl = juliacall.newmodule("FooModule")
    
    # Get the directory this file is in, then include the `Foo.jl` Julia source code.
    d = os.path.dirname(os.path.abspath(__file__))
    jl.include(os.path.join(d, "Foo.jl"))
    
    Foo = jl.Foo
    $
    

    And the second one imports the first foo.py file:

    $ cat run.py
    import matplotlib
    from foo import Foo
    $
    

    When I run the run.py file, I get this error:

    $ python run.py
    ERROR: `ccall` requires the compilerTraceback (most recent call last):
      File "/home/dingraha/projects/pythoncall_import_error/run.py", line 2, in <module>
        from foo import Foo
      File "/home/dingraha/projects/pythoncall_import_error/foo.py", line 4, in <module>
        import juliacall; jl = juliacall.newmodule("FooModule")
      File "/home/dingraha/projects/pythoncall_import_error/venv/lib/python3.9/site-packages/juliacall/__init__.py", line 218, in <module>
        init()
      File "/home/dingraha/projects/pythoncall_import_error/venv/lib/python3.9/site-packages/juliacall/__init__.py", line 214, in init
        raise Exception('PythonCall.jl did not start properly')
    Exception: PythonCall.jl did not start properly
    $
    

    Now, if I switch the order of the two import lines in run.py, there's no error. Also if I import juliacall first, i.e.:

    $ cat run-no_error.py
    import juliacall
    import matplotlib
    from foo import Foo
    $
    

    the error goes away.

    Any ideas?

    opened by dingraha 1
  • `isvalid` error with PythonCall.Utils.StaticString

    `isvalid` error with PythonCall.Utils.StaticString

    When converting to PyArray some numpy arrays with string elements, I end up with strings of type PythonCall.Utils.StaticString{UInt32, 44}.

    It seems that this type is not well supported though, and I get the following errors when indexing and computing the length:

    julia> s = PythonCall.Utils.StaticString{UInt32, 44}("ababababb")
    "ababababb"
    
    julia> s[1]
    ERROR: MethodError: no method matching isvalid(::PythonCall.Utils.StaticString{UInt32, 44}, ::Int64)
    Closest candidates are:
      isvalid(::AbstractString, ::Integer) at strings/basic.jl:143
      isvalid(::SubstitutionString, ::Integer) at regex.jl:566
      isvalid(::Type{T}, ::Integer) where T<:AbstractChar at strings/unicode.jl:58
      ...
    Stacktrace:
     [1] isvalid
       @ ./strings/basic.jl:143 [inlined]
     [2] getindex(s::PythonCall.Utils.StaticString{UInt32, 44}, i::Int64)
       @ Base ./strings/basic.jl:186
     [3] top-level scope
       @ REPL[5]:1
    
    julia> length(s)
    ERROR: MethodError: no method matching isvalid(::PythonCall.Utils.StaticString{UInt32, 44}, ::Int64)
    Closest candidates are:
      isvalid(::AbstractString, ::Integer) at strings/basic.jl:143
      isvalid(::SubstitutionString, ::Integer) at regex.jl:566
      isvalid(::Type{T}, ::Integer) where T<:AbstractChar at strings/unicode.jl:58
      ...
    Stacktrace:
     [1] isvalid
       @ ./strings/basic.jl:143 [inlined]
     [2] length
       @ ./strings/basic.jl:396 [inlined]
     [3] length(s::PythonCall.Utils.StaticString{UInt32, 44})
       @ Base ./strings/basic.jl:387
     [4] top-level scope
       @ REPL[6]:1
    
    julia> s isa AbstractString # in principle the type should implement the whole interface
    true
    
    opened by CarloLucibello 0
  • Drop matplotlib support?

    Drop matplotlib support?

    I can confirm new PythonPlot.jl (which uses matplotlib) works.

    Then having support here, even minimal seems redundant. IF it's slowing startup (a big if), then consider just dropping it and update major version (possible to drop more?).

    PythonPlot.jl should have downloaded matplotlib automatically. I guess I got an error because of non-clean environment, at least I could easily install it.

    Whatever is used for plotting (from Python or not), needs not (should not?) be in the package. And for users who rely on, can use older version of your package.

    If you don't want to drop it (or even if) then docs could be improved. I thought about improving them, or manage to follow it, then fix, but will not if it's redundant. At a minimum you may want to point to PythonPlot (not PyPlot).

    [It's ok to wait a few days to see if new PR on master helps with startup, before investigating this or other ways for improved startup.]

    Off-topic here (only related to new major version if you so choose):

    If you do consider a new breaking version change then may too:

    version = ">=3.11,<4"

    since you download automatically for users. It may not be needed since you get latest anyway 3.11 now. Maybe there's a downside to that, blocking use of some Python packages? At least if/when you users can use 3.11, it's faster and your users will be grateful, and being able to rely on new features:

    PEP 654 – Exception Groups and except*

    Is that something you need to support somehow from the Julia side?

    3.12 is still a year away but 3.12 alpha 2 is out with better error messages. It would be nice if you could easily opt into prereleases but I think not up to you, rather Conda, and I know I can install my own, bypassing Conda. Possibly documenting that such is possible (for e.g. better error messages) is helpful, though I guess mostly if you're developing on the Python side...

    Is there something you rely on or want to rely on (or want dropped) from Python?

    I see some modules have been dropped in the alpha, and:

    The deprecated wstr and wstr_length members of the C implementation of unicode objects were removed, per PEP 623.

    It's to pave way for UTF-8 internally in Python (I believe not yet there). I don't think the removal affects you (only Python extension developers), but it would be great if/when Julia and Python can share strings, send back and forth, keeping same encoding. At least then, likely you want a new version number.

    opened by PallHaraldsson 0
  • PythonCall not playing nicely with pyinstaller

    PythonCall not playing nicely with pyinstaller

    First of all I'd like to express my appreciation for all the great work being done here. I think this package is a great facilitator for moving Julia out of it's initial niche towards integration into real, practical applications.

    In that line, I've been trying to call Julia from a Python application that is then made into a (Windows) executable using pyinstaller. This did not work "out of the box". After a lot of digging, I discovered the bug was in the init_consts function, in particular in the evaluation of

    :(pycopy!(pybuiltins.help, pybuiltinsmodule.help)), 
    

    Commenting the :help in src/constants/consts.jl:66 gave a working application, much to my joy and that of my colleagues, but I don't know exactly why this helped, and if there's any downside to doing this. Even if my immediate problem has been resolved/avoided, I thought I should share what I found for the benefit of the community.

    opened by KeithWM 1
  • unable to install package datasets

    unable to install package datasets

    This is possible related to GLIBC, but I don't know for sure. I cannot reliably reproduce the error, but here are the problematic steps:

    pkg> conda channel_add conda-forge
    pkg> conda channel_add huggingface
    pkg> conda add datasets
    

    Sometimes this works, sometimes works but when I load my package importing datasets I get

    ERROR: InitError: Python: ImportError: /home/lucibello/.julia/juliaup/julia-1.8.2+0.x64/bin/../lib/julia/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /home/lucibello/.julia/dev/HuggingFaceDatasets/.CondaPkg/env/lib/python3.10/site-packages/pyarrow/../../../libarrow.so.600)
    

    and sometimes the installation itself fails with

    conda-forge/linux-64                                        Using cache
    conda-forge/noarch                                          Using cache
    Encountered problems while solving:
      - package datasets-2.7.0-py_0 requires pyarrow >=6.0.0, but none of the providers can be installed
    
    The environment can't be solved, aborting the operation
    error    libmamba Could not solve for environment specs
    critical libmamba UnsatisfiableError
    ERROR: LoadError: InitError: failed process: Process(`/home/runner/.julia/artifacts/8d5103d84a46e89c60d007a7d30b926037514616/bin/micromamba -r /home/runner/.julia/scratchspaces/0b3b1443-0f03-428d-bdfb-f27f9c1191ea/root create -y -p /tmp/jl_dCVni6/.CondaPkg/env --override-channels --no-channel-priority "datasets[version='>=2.7, <3']" "libstdcxx-ng[version='>=3.4,<11.4',channel='conda-forge']" "numpy[version='>=1.23, <2']" "pillow[version='>=9.2, <10']" "python[version='>=3.7,<4',channel='conda-forge',build='*cpython*']" -c conda-forge -c huggingface`, ProcessExited(1)) [1]
    

    in this CI run.

    Sorry but the behavior is very erratic and I cannot offer a stable way to trigger the issue.

    opened by CarloLucibello 5
  • unable to install impedancefitter

    unable to install impedancefitter

    I fail to install the PyPy-package ImpedanceFitter. I also tried in Julia RC v.1.8.3. Here my commands:

    import Pkg; Pkg.add("CondaPkg")
    import Pkg; Pkg.add("PythonCall")
    impfit = PythonCall.pyimport("impedancefitter")
    

    and here is the error output on my machine:

    The environment can't be solved, aborting the operation
    error    libmamba Could not solve for environment specs
    critical libmamba UnsatisfiableError
    ERROR: failed process: Process(`'C:\Users\stefanpofahl\.julia\artifacts\7950c161b9c76ecc6d58b2b6734696fee7d8e4a0\Library\bin\micromamba.exe' -r 'C:\Users\stefanpofahl\.julia\scratchspaces\0b3b1443-0f03-428d-bdfb-f27f9c1191ea\root' create -y -p 'C:\Users\stefanpofahl\.julia\environments\v1.9\.CondaPkg\env' --override-channels --no-channel-priority "impedancefitter[version='*']" "python[version='>=3.7,<4',channel='conda-forge',build='*cpython*']" -c conda-forge`, ProcessExited(1)) [1]
    
    Stacktrace:
      [1] pipeline_error
        @ .\process.jl:565 [inlined]
      [2] run(::Cmd; wait::Bool)
        @ Base .\process.jl:480
      [3] run(::Cmd)
        @ Base .\process.jl:477
      [4] _run(io::IO, cmd::Cmd, args::Any; flags::Any)
        @ CondaPkg C:\Users\stefanpofahl\.julia\packages\CondaPkg\RuSdR\src\resolve.jl:301
      [5] _resolve_conda_install(io::Any, conda_env::Any, specs::Any, channels::Any; create::Any)
        @ CondaPkg C:\Users\stefanpofahl\.julia\packages\CondaPkg\RuSdR\src\resolve.jl:202
      [6] resolve(; force::Bool, io::IO, interactive::Bool, dry_run::Bool)
        @ CondaPkg C:\Users\stefanpofahl\.julia\packages\CondaPkg\RuSdR\src\resolve.jl:452
      [7] resolve()
        @ CondaPkg C:\Users\stefanpofahl\.julia\packages\CondaPkg\RuSdR\src\resolve.jl:315
      [8] add(pkgs::AbstractVector; resolve::Any, file::Any, temp::Any)
        @ CondaPkg C:\Users\stefanpofahl\.julia\packages\CondaPkg\RuSdR\src\deps.jl:244
      [9] add(pkgs::AbstractVector)
        @ CondaPkg C:\Users\stefanpofahl\.julia\packages\CondaPkg\RuSdR\src\deps.jl:231
     [10] add(pkg::Union{CondaPkg.ChannelSpec, CondaPkg.PipPkgSpec, CondaPkg.PkgSpec}; kw::Base.Pairs{Symbol, V, Tuple{Vararg{Symbol, N}}, NamedTuple{names, T}} where {V, N, names, T<:Tuple{Vararg{Any, N}}})
        @ CondaPkg C:\Users\stefanpofahl\.julia\packages\CondaPkg\RuSdR\src\deps.jl:248
     [11] add(pkg::Union{CondaPkg.ChannelSpec, CondaPkg.PipPkgSpec, CondaPkg.PkgSpec})
        @ CondaPkg C:\Users\stefanpofahl\.julia\packages\CondaPkg\RuSdR\src\deps.jl:248
     [12] add(pkg::AbstractString; version::Any, channel::Any, build::Any, kw::Base.Pairs{Symbol, V, Tuple{Vararg{Symbol, N}}, NamedTuple{names, T}} where {V, N, names, T<:Tuple{Vararg{Any, N}}})
        @ CondaPkg C:\Users\stefanpofahl\.julia\packages\CondaPkg\RuSdR\src\deps.jl:375
     [13] add(pkg::AbstractString)
        @ CondaPkg C:\Users\stefanpofahl\.julia\packages\CondaPkg\RuSdR\src\deps.jl:375
     [14] top-level scope
        @ REPL[6]:1
    

    Do you have asuggestion?

    bug 
    opened by StefanPofahl 9
Releases(v0.9.10)
  • v0.9.10(Dec 2, 2022)

    PythonCall v0.9.10

    Diff since v0.9.9

    Closed issues:

    • Trouble Converting Collections from Python to Julia (#231)
    • Using HuggingFace Transformers pipeline? (#236)
    • libstdcxx-ng versioning (#237)
    • micromamba hash mismatch (#238)
    • Improving startup time (#239)
    • PythonCall in sysimage (#240)
    • PythonCall v0.9.9 cannot be loaded (#241)
    • Support alternative CPython e.g. register-based? (#242)
    • How to install julia 1.7.0 with juliacall 0.4.3? (#246)
    • Unable to import juliacall in Python (#250)

    Merged pull requests:

    • libstdcxx version-bounds (#243) (@samtkaplan)
    Source code(tar.gz)
    Source code(zip)
  • v0.9.9(Oct 20, 2022)

  • v0.9.8(Oct 18, 2022)

  • v0.9.7(Oct 11, 2022)

    PythonCall v0.9.7

    Diff since v0.9.6

    Closed issues:

    • Width & height info of the terminal is not passed to the Julia side? (#216)
    • python string wrapper? (#218)
    • segfault importing PyCall after importing Julia package with python dependencies / PythonCall.jl (#220)
    • JuliaCall doesn't work with pandas DataFrames that have categorical columns (#222)
    • Error when using PythonCall attached to a previous Conda environment. (#224)
    • Package with pip dep fails with Jupyter notebook on Windows (#226)
    • import PyCall, PythonCall crashes Julia (#228)
    • Security update (#229)
    • Security update (for all supported Python versions) (#230)

    Merged pull requests:

    • fix deprecation and possible faulty construction of pandas tables (#221) (@ExpandingMan)
    Source code(tar.gz)
    Source code(zip)
  • v0.9.6(Sep 9, 2022)

    PythonCall v0.9.6

    Diff since v0.9.5

    Closed issues:

    • convenient julia->python conversion of structured data (#200)
    • julia thread hangs with multiple sevals (#202)
    • Multiline evaluation (#210)
    • NamedTuple (#213)
    • __rmul__ did not act as expected (#214)
    • Heap corruption on Python when torch is imported before juliacall, but not the reverse (#215)
    • setindex! not defined for np arrays of non-conventional (isbits) types (#217)
    Source code(tar.gz)
    Source code(zip)
  • v0.9.5(Aug 19, 2022)

    PythonCall v0.9.5

    Diff since v0.9.4

    Closed issues:

    • find wrong Julia on M1 (#197)
    • https://cjdoris.github.io/PythonCall.jl not updated to 0.9.3? (#203)
    • printing MIME type is not handled correctly for graphs (#205)
    • juliacall and PythonCall versions can get out of sync (#206)
    • Segfault when indexing with SymbolServer (#207)
    • copy(::PyDict) crash (#208)
    • Resolve sometimes too eager to download package metadata and trigger updates (#209)
    • Error when trying to display a figure from GLMakie through juliacall (#212)
    Source code(tar.gz)
    Source code(zip)
  • v0.9.4(Jul 26, 2022)

    PythonCall v0.9.4

    Diff since v0.9.3

    Closed issues:

    • Bug when 1.6-DEV installed (#174)
    • sys.so not found if julia is a soft link (#196)
    • Can not use pip package although conda status correct (#198)

    Merged pull requests:

    • Sync docs (#195) (@PallHaraldsson)
    • Update index.md (#199) (@pitmonticone)
    Source code(tar.gz)
    Source code(zip)
  • v0.9.3(Jul 2, 2022)

  • v0.9.2(Jul 2, 2022)

    PythonCall v0.9.2

    Diff since v0.9.1

    Closed issues:

    • Add env var to use custom sysimage (#73)
    • Configure julia options (#184)
    • PythonCall.jl did not start properly when import juliacall (#188)
    • Fail to include Julia file in a python script through the Main.include function (#189)
    • support calling Julia with custom sysimage from Python (#190)

    Merged pull requests:

    • pass settings from -Xjuliacall_ to julia (#192) (@dpinol)
    Source code(tar.gz)
    Source code(zip)
  • v0.9.1(Jun 18, 2022)

    PythonCall v0.9.1

    Diff since v0.9.0

    Closed issues:

    • DataFrame conversion from pandas fails for non-string column names (#133)
    • How to convert a Pandas DataFrame to Julia one without copy? (#135)
    • LoadError: InitError during Package Build (#136)
    • UndefVarError when using include in a new module (#176)
    • Array conversion error? (#177)
    • Converting pytable to DataFrame fails on v0.9 (#180)
    • How to pass a Pandas DataFrame to Julia in 0.9? (#181)
    • juliacall.JuliaError: MethodError: Cannot convert an object of type PythonCall.Py to an object of type Tuple (#183)
    • Crash on Apple M1 with Julia 1.8-rc1 (#185)

    Merged pull requests:

    • fix: Ensure julia lib is always loaded as global (#186) (@gnuille)
    Source code(tar.gz)
    Source code(zip)
  • v0.9.0(May 27, 2022)

    PythonCall v0.9.0

    Diff since v0.8.0

    Closed issues:

    • Using PythonCall within Julia modules (#57)
    • Conversion rules for numpy [and ctypes] scalars (#62)
    • Error in "using PythonCall" (#132)
    • UndefVarError: y not defined for iteration in Python back to Julia (#134)
    • bib citation (#137)
    • Precompilation fails with both PyCall and PythonCall (#140)
    • fail to display inside vscode/jupyter notebook (#141)
    • JuliaCall reinstalling PythonCall every time (#144)
    • docstring method for Base.Docs.doc(::Py) (#147)
    • missing haskey, get, etcetera? (#148)
    • pyisnothing function? (#149)
    • ispynull (#151)
    • Docs: how to use CondaPkg from PythonCall? (#152)
    • How to define python function from Julia side (#156)
    • optimized hasproperty(o::Py, attr) = pyhasattr(o, attr) (#157)
    • using functions ending with ! in juliacall (#158)
    • support array(dtype=...) optional argument (#160)
    • missing .copy() method for Dict wrapper (#161)
    • Errors trying to use logomaker (#164)
    • first import of juliacall clobbers Project.toml (#166)
    • Using seval in a new module created from Python (#170)
    • missing docstring pyhasattr in docs (#173)
    • Juliacall on Apple M1 (#175)
    • module eval should be eval(m, expr) instead of eval(expr) in seval (#178)

    Merged pull requests:

    • define 2-arg Base.hash (#154) (@stevengj)
    • correct Julia minimum version to ≥ 1.6 (#155) (@stevengj)
    • Not support EOLed 3.6 or older (#162) (@PallHaraldsson)
    Source code(tar.gz)
    Source code(zip)
  • v0.8.0(Mar 17, 2022)

  • v0.7.0(Mar 17, 2022)

    PythonCall v0.7.0

    Diff since v0.6.1

    Closed issues:

    • Segmentation fault when importing python package (#115)
    • Can't manage to use juliacall in WSL (#116)
    • Failed install on Raspberry Pi (#120)
    • no version of Julia is compatible (#121)
    • plt.show() (#122)
    • pyclass (#124)
    • Document pyconvert_add_rule (#125)
    • Make pyconvert_add_rule and Py safe by default (#126)
    • LaTeXStrings (#127)
    • Environment reinitialization if failed (#130)
    • Error during Module initialization (#131)

    Merged pull requests:

    • crop whitespace around displayed figures (#123) (@marius311)
    Source code(tar.gz)
    Source code(zip)
  • v0.6.1(Feb 21, 2022)

    PythonCall v0.6.1

    Diff since v0.6.0

    Closed issues:

    • show stacktrace of Julia exceptions in Python (#83)
    • Docstring for dispatched methods (#107)
    • Support selective import in @py macro @py from <module> import ... (#117)

    Merged pull requests:

    • Expose PyCapsule type and methods (#114) (@pabloferz)
    Source code(tar.gz)
    Source code(zip)
  • v0.6.0(Feb 17, 2022)

    PythonCall v0.6.0

    Diff since v0.5.1

    Closed issues:

    • Registries in juliacalldeps.json (#74)
    • Function support? (#95)
    • UndefVarError triggered by err.jl (#100)
    • Display for pytype in Pluto notebooks (#101)
    • PyPlot equivalent? (#105)
    • Initialization fails for Julia with DEV in version (#106)
    • Any known threading/GIL issues? (#108)
    • A direct way to import Julia packages in Python? (#109)

    Merged pull requests:

    • small typo fix (#103) (@tlienart)
    Source code(tar.gz)
    Source code(zip)
  • v0.5.1(Jan 24, 2022)

    PythonCall v0.5.1

    Diff since v0.5.0

    Closed issues:

    • Test on earlier versions of Julia and Python (#54)
    • Make docs less PythonCall-centric (#56)
    • coming from PyCall section in docs (#59)
    • Hooks into startup (#72)
    • "Python exited unexpectedly" when using external conda environment (#85)
    • Install specific Python version (#87)
    • To do: fix multimedia bug (#89)
    • Imported juliacall.newmodule() namespace not updated (#92)
    • how to convert list of lists into matrix (#99)
    Source code(tar.gz)
    Source code(zip)
  • v0.5.0(Dec 21, 2021)

    PythonCall v0.5.0

    Diff since v0.4.3

    Closed issues:

    • use conda --freeze-installed in Deps.add() (#64)
    • Relative paths in dependencies (#65)
    • Rework Python dependencies (#66)
    • Startup code is slow (#67)
    • When using Pluto, output from pyprint is buffered (#68)
    • remove Python<3.10 restriction (#69)
    • Importing local modules (#70)
    • Support for using a custom system image? (#71)
    • hope support python f-string in julia (#75)
    • Is juliacall supporting python multiprocessing? (#77)
    • Can we use julia functions with ! postfix in python calls to julia functions (#78)
    • Using private Julia package SSH fails (#80)
    • Should PythonCall create separate environment for different projects? (#81)
    • error using development version of package (#82)
    Source code(tar.gz)
    Source code(zip)
  • v0.4.3(Nov 3, 2021)

  • v0.4.2(Oct 23, 2021)

  • v0.4.1(Oct 22, 2021)

  • v0.4.0(Oct 21, 2021)

    PythonCall v0.4.0

    Diff since v0.3.0

    Closed issues:

    • How to pass a numpy array into a Julia function? (#38)
    • Set Jill upstream when checking versions (#40)
    • Use timestamp of juliacallmeta.json (#41)
    • unsafe_pynext, pyisnull, pynext, pyprint, ... (#42)
    • pymacro next, print, int, ... (#43)
    • Run Aqua tests (#46)
    • Document unsafe API (#47)
    • Check all getptr calls (#48)
    • Announce on discourse (#52)
    • PyCall compat mode (#55)
    • "writing packages" section in docs (#58)
    • InitError (#60)
    • FYI: 3.10 is out (to test) and 3.11 alpha (#63)

    Merged pull requests:

    • Python setup (#39) (@ShuhuaGao)
    Source code(tar.gz)
    Source code(zip)
  • v0.3.0(Sep 21, 2021)

    PythonCall v0.3.0

    Diff since v0.2.1

    Closed issues:

    • Todo list (#1)
    • Precompile statements (and error) (#13)
    • Rely less on PyRef (#14)
    • How can we mix Python and Julia code without relying on finalizers? (#15)
    • Remove lazy functionality from PyObject (#16)
    • Reloading code (#17)
    • Faster Py_IncRef/Py_DecRef (#18)
    • Preload C API pointers (#19)
    • DSL avoid checking embedded Python objects (#20)
    • _type_lb and _type_ub are wrong (#21)
    • Make it so tryconvert rules can assume S is a DataType (#22)
    • Error when getting columns on PyPandasDataFrame (#23)
    • PythonCall/juliacall doesn't work in conda environment (#25)
    • Doesn't work on 1.7.0-beta2 or 1.8 (#26)
    • Error when calling Julia function defined in module (#27)
    • Don't understand how to pass numpy arrays to Julia function (#28)
    • Serialise NULL to Nothing (#33)
    • Serialising traceback objects (#34)
    • Is rewrite complete now ? (#37)
    Source code(tar.gz)
    Source code(zip)
  • v0.2.1(Mar 25, 2021)

  • v0.2.0(Mar 8, 2021)

  • v0.1.0(Mar 4, 2021)

    PythonCall v0.1.0

    Closed issues:

    • Possible legal issues with the logo (#3)

    Merged pull requests:

    • Big rewrite (#4) (@cjdoris)
    • adds julia.SetValue (#5) (@cjdoris)
    Source code(tar.gz)
    Source code(zip)
Owner
Christopher Rowley
Mathematics and data science researcher
Christopher Rowley
Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Scala, Go, Javascript and more

Apache MXNet (incubating) for Deep Learning Apache MXNet is a deep learning framework designed for both efficiency and flexibility. It allows you to m

The Apache Software Foundation 20.2k Jan 5, 2023
Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Scala, Go, Javascript and more

Apache MXNet (incubating) for Deep Learning Apache MXNet is a deep learning framework designed for both efficiency and flexibility. It allows you to m

The Apache Software Foundation 19.3k Feb 12, 2021
Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Scala, Go, Javascript and more

Apache MXNet (incubating) for Deep Learning Master Docs License Apache MXNet (incubating) is a deep learning framework designed for both efficiency an

ROCm Software Platform 29 Nov 16, 2022
Numba-accelerated Pythonic implementation of MPDATA with examples in Python, Julia and Matlab

PyMPDATA PyMPDATA is a high-performance Numba-accelerated Pythonic implementation of the MPDATA algorithm of Smolarkiewicz et al. used in geophysical

Atmospheric Cloud Simulation Group @ Jagiellonian University 15 Nov 23, 2022
Pythonic particle-based (super-droplet) warm-rain/aqueous-chemistry cloud microphysics package with box, parcel & 1D/2D prescribed-flow examples in Python, Julia and Matlab

PySDM PySDM is a package for simulating the dynamics of population of particles. It is intended to serve as a building block for simulation systems mo

Atmospheric Cloud Simulation Group @ Jagiellonian University 32 Oct 18, 2022
Calling Julia from Python - an experiment on data loading

Calling Julia from Python - an experiment on data loading See the slides. TLDR After reading Patrick's blog post, we decided to try to replace C++ wit

Abel Siqueira 8 Jun 7, 2022
MacroTools provides a library of tools for working with Julia code and expressions.

MacroTools.jl MacroTools provides a library of tools for working with Julia code and expressions. This includes a powerful template-matching system an

FluxML 278 Dec 11, 2022
A little software to generate and save Julia or Mandelbrot's Fractals.

Julia-Mandelbrot-s-Fractals A little software to generate and save Julia or Mandelbrot's Fractals. Dependencies : Python 3.7 or more. (Also possible t

Olivier 0 Jul 9, 2022
Perspective: Julia for Biologists

Perspective: Julia for Biologists 1. Examples Speed: Example 1 - Single cell data and network inference Domain: Single cell data Methodology: Network

Elisabeth Roesch 55 Dec 2, 2022
✔️ Visual, reactive testing library for Julia. Time machine included.

PlutoTest.jl (alpha release) Visual, reactive testing library for Julia A macro @test that you can use to verify your code's correctness. But instead

Pluto 68 Dec 20, 2022
Python-experiments - A Repository which contains python scripts to automate things and make your life easier with python

Python Experiments A Repository which contains python scripts to automate things

Vivek Kumar Singh 11 Sep 25, 2022
Scalable, Portable and Distributed Gradient Boosting (GBDT, GBRT or GBM) Library, for Python, R, Java, Scala, C++ and more. Runs on single machine, Hadoop, Spark, Dask, Flink and DataFlow

eXtreme Gradient Boosting Community | Documentation | Resources | Contributors | Release Notes XGBoost is an optimized distributed gradient boosting l

Distributed (Deep) Machine Learning Community 23.6k Dec 31, 2022
Python Library for learning (Structure and Parameter) and inference (Statistical and Causal) in Bayesian Networks.

pgmpy pgmpy is a python library for working with Probabilistic Graphical Models. Documentation and list of algorithms supported is at our official sit

pgmpy 2.2k Jan 3, 2023
High performance, easy-to-use, and scalable machine learning (ML) package, including linear model (LR), factorization machines (FM), and field-aware factorization machines (FFM) for Python and CLI interface.

What is xLearn? xLearn is a high performance, easy-to-use, and scalable machine learning package that contains linear model (LR), factorization machin

Chao Ma 3k Jan 3, 2023
This repository is related to an Arabic tutorial, within the tutorial we discuss the common data structure and algorithms and their worst and best case for each, then implement the code using Python.

Data Structure and Algorithms with Python This repository is related to the Arabic tutorial here, within the tutorial we discuss the common data struc

Mohamed Ayman 33 Dec 2, 2022