Python with the scientific stack, compiled to WebAssembly.

Related tags

python webassembly
Overview

Build Status Documentation Status

Python with the scientific stack, compiled to WebAssembly.

What is Pyodide?

Pyodide may be used in any context where you want to run Python inside a web browser.

Pyodide brings the Python 3.8 runtime to the browser via WebAssembly, along with the Python scientific stack including NumPy, Pandas, Matplotlib, SciPy, and scikit-learn. The packages directory lists over 75 packages which are currently available. In addition it's possible to install pure Python wheels from PyPi.

Pyodide provides transparent conversion of objects between Javascript and Python. When used inside a browser, Python has full access to the Web APIs.

Try Pyodide (no installation needed)

Try Pyodide in a REPL directly in your browser. For further information, see the documentation.

Getting Started

Pyodide offers three different ways to get started depending on your needs and technical resources. These include:

  • Use a hosted distribution of Pyodide: see the Getting Started documentation.
  • Download a version of Pyodide from the releases page and serve it with a web server.
  • Build Pyodide from source
    • Build natively with make: primarily for Linux users who want to experiment or contribute back to the project.
    • Use a Docker image: recommended for Windows and macOS users and for Linux users who prefer a Debian-based Docker image with the dependencies already installed.

History

Pyodide was created in 2018 by Michael Droettboom at Mozilla as part of the Iodide project. Iodide is an experimental web-based notebook environment for literate scientific computing and communication.

Iodide is no longer maintained. If you want to use Pyodide in an interactive client-side notebook, see Pyodide notebook environments.

Contributing

Please view the contributing guide for tips on filing issues, making changes, and submitting pull requests. Pyodide is an independent and community-driven open-source project. The decision making process is outlined in the Project governance.

Communication

License

Pyodide uses the Mozilla Public License Version 2.0.

Issues
  • Memory access out of bounds in Chrome

    Memory access out of bounds in Chrome

    I'm getting the following intermittent error running the develop version in Chrome, running from a web worker.

    Chrome Version 89.0.4389.90 (Official Build) (64-bit) Windows 10 1909 build 18363.1379

    pyodide.js:406 The cause of the fatal error was: RuntimeError: memory access out of bounds at PyArray_IntpFromIndexSequence (:wasm-function[991]:0x737a1) at PyArray_IntpConverter (:wasm-function[990]:0x73702) at byn$fpcast-emu$PyArray_IntpConverter (:wasm-function[3415]:0x15bfeb)

    Blocker bug 
    opened by dmondev 67
  • Emscripten 2.0.12 Upstream backend

    Emscripten 2.0.12 Upstream backend

    What it says. Emscripten 2.0.9 upstream (non-fastcomp, compiles straight to wasm), plus patches for binaryen and emscripten so that dynamic linking and passing of function pointers between side modules and main modules works.

    I'm part way through pull requests for binaryen and emscripten so that the patches there aren't needed.

    All other patches to emscripten and binaryen have been dropped - I'm not sure if they were all things that are fixed in the new backend - none of them worked with new backend anyway

    Closes #531 Closes #476

    opened by joemarshall 38
  • Allow to import python modules directly / remotePath feature

    Allow to import python modules directly / remotePath feature

    remotePath-feature for Pyodide

    This pull request implements the remotePath-feature. This feature makes Pyodide capable to fetch Python modules remotely from a set of given URLs. Both single Python py-files as well as Python-modules in sub-folders using __init__.py are supported.

    In the demonstrated example below, there is a module called xyz located at https://phorward.info/tmp/pyodide/xyz/, which in turn imports the modules x and y.

    Python console: python-console

    JavaScript console: image

    Any remotely fetched module is parsed for further imports which are imported as well, including pre-build packages from Pyodide. Due to this entirely changed behavior on how imports are resolved, packages from the pyodide packages.json take a higher precedence than remotely loaded packages.

    The feature is already tested well in a re-implementation of a Python-based HTML5-web-app that previously was compiled into JavaScript using PyJS, but now can directly be interpreted (further details here)


    [INITIAL MESSAGE OF THIS PULL REQUEST] Hi there!

    This is a work-in-progress draft to resolve my problem I reported in #481 to allow importing Python files directly, which are not pre-packaged with pyodide, but directly served. So this pull request now makes it possible to make a

    import a
    

    and a.py is fetched from the same source as pyodide is served from (baseURL). In case a.py imports further modules, let it be b.py and c.py, these are also fetched. This is done until all required packages are fetched, including imports to available packages, as it is the case before.

    Python files fetched from the server are stored into the browser cache using FS.writeFile, and then imported by Python by setting PYTHONPATH to /. I hope this won't be a security problem.

    A demo of this feature is available here: https://phorward.info/tmp/pyodide/

    • abc.html does the above import of a.py, b.py, c.py and the package html5
    • game.html just does an import game, loads game.py and html5 and executes.
    • console.html can also be used: just type import game or import a there to see what's happening (take a look into the JavaScript console for output)

    This is just a first draft, I want to enhance it also to import module directories served. Is this useful also for others? Does anyone have suggestions for improvements?

    opened by phorward 36
  • Configurable

    Configurable "import js"

    Add a pyodide.mountPackage("name", object) api that mounts object at name, so that import "name" imports object, and from name import field imports object.field as field.

    For example, the current js import would then be installed by saying pyodide.mountPackage("js", globalThis) though we would keep the current behavior as the default. We could get rid of import js by saying pyodide.mountPackage("js", undefined).

    opened by hoodmane 36
  • Question: How to sandboxise pyodide .

    Question: How to sandboxise pyodide .

    relative to #955 , i want a sandboxed python runtime, i face two question is that :

    1. how to limit the import function ? have any offical way todo this work ? i want limit the import with a denyList or a allowList, like the js i will deny it in a special pyodide.runPythonAsync call.
    2. how to pip out the stdio ( redirect the print output from python ) ? now i use a way is make a patch in the sys.stdout from here, but seems like it will failure after i import the numpy package , so strange .

    about the question (1) , i was try 2 way to do. ( sorry i never read the code of the pyodide , all the follow patch work are try on runtime console and then write to code . ) Summary: i run the pyodide in WebWorker to use the javascript sandboxie runtime to have a additional protection .

    1. let the debuger hard to direct find the pyodide ref from root ( Worker's self or globalThis )

    • in one side, i wrapper all the thing into a Typescript class , i named it class SClass ( mean : secure class)

    use follow code to hide it

    
    class SClass {
        constructor() {
            workerSelf.onmessage = this.onmessage.bind(this);
        }
    
        // .......
    
    const workerSelf: typeof WorkerSelf = self as any;
    workerSelf.aaaaaa = new SClass();
    workerSelf.aaaaaa.init();
    // https://github.com/Microsoft/TypeScript/issues/24587#issuecomment-412287117
    // we have the `workerSelf.onmessage` on the ref link, the `onmessage` are `onmessage.bind(this)`,
    //   so, the GC couldn't delete our object,
    //   so, we can safe delete the direct ref from root object,
    //   then, we implement the hidden all the ref target,
    //   no one can access our object from debug console.
    // @ts-ignore
    // workerSelf[Symbol() as any] = workerSelf.aaaaaa;
    delete workerSelf.aaaaaa;
    
    

    now , no one can access the SClass from self .

    • in other side, because the pyodide cannot be delete from globalThis, (it be set the non-configable flag), i cloned all the field and delete them from pyodide, only recover the pyodide._module to let it can use import js
    
        init() {
            return workerSelf.languagePluginLoader.then(async () => {
                // await workerSelf.pyodide.loadPackage(['numpy', 'pytz']);
    
                // delete root ref from self.root
                this.pyodide = clone(workerSelf.pyodide);
                // console.log(this.pyodide);
                // console.log(this.pyodide._module);
                // console.log(Object.getOwnPropertyNames(workerSelf.pyodide));
                Object.keys(workerSelf.pyodide).forEach(T => delete workerSelf.pyodide[T]);
    
                // delete packages lookup-table from pyodide
                // console.log('packages', workerSelf.pyodide._module.packages);
                // console.log('packages', this.pyodide._module.packages);
                workerSelf.pyodide._module = {};
                workerSelf.pyodide._module.packages = this.pyodide._module.packages;
                workerSelf.pyodide._module.packages = {
                    dependencies: {},
                    import_name_to_package_name: {},
                };
                // console.log('packages', workerSelf.pyodide._module.packages);
    
            })
    

    now, other than the js package cannot be import, all the try will throw error .

    .

    but if i only want a allowList or denyList, i cannot comlete it use follow code to recover the pyodide._module.packages . it's not work.

    
                // recover selected packages to pyodide lookup-table
                const recoverPack = (pName: string) => {
                    workerSelf.pyodide._module.packages.dependencies[pName] =
                        this.pyodide._module.packages.dependencies[pName];
                    workerSelf.pyodide._module.packages.import_name_to_package_name[pName] =
                        this.pyodide._module.packages.import_name_to_package_name[pName];
                };
                const allowList = [
                    'numpy',
                ];
                allowList.forEach(T => recoverPack(T));
                Object.getOwnPropertyNames(this.pyodide._module.packages.dependencies).forEach(T => recoverPack(T));
                // dont remember to recover the prototype
                if (allowList.length > 0) {
                    // console.log('this.pyodide._module.packages.prototype',
                    //     Object.getPrototypeOf(this.pyodide._module.packages));
                    Object.setPrototypeOf(workerSelf.pyodide._module.packages,
                        Object.getPrototypeOf(this.pyodide._module.packages));
                    Object.setPrototypeOf(workerSelf.pyodide._module.packages.dependencies,
                        Object.getPrototypeOf(this.pyodide._module.packages.dependencies));
                    Object.setPrototypeOf(workerSelf.pyodide._module.packages.import_name_to_package_name,
                        Object.getPrototypeOf(this.pyodide._module.packages.import_name_to_package_name));
                }
                console.log('packages', workerSelf.pyodide._module.packages);
    

    seems like some other important thing i deleted from the workerSelf.pyodide.

    in the end, i remove above the attempt that delete thing from pyodide.

    2. use pyodide.find_imports to check the code before run it . and simple not run it if find something not allowed.

    see pyodide.find_imports

    thsi is a easy way, but in the document i see some dangerous function can bypass this way. like pyodide.eval_code or the pyodide.open_url , and i not sure not have any other way can load code from remote or eval a string to code in runtime.


    BTW: the cursor params of pyodide.get_completions API seems like not receive {row, col} as params, i dont known how to use it in a large section code. so, now i run a second WebWorker to run the Jedi to do the code complete task. but seems like the jedi cannot get information from a imported package , like numpy .

    sorry for my knowledge, i'm good at C++/Typescript and software security/software architecture, the Python is in my weakness area.

    opened by Lyoko-Jeremie 36
  • Enable building sqlite3 into cpython

    Enable building sqlite3 into cpython

    I'm messing around with adding sqlite3, as mentioned in #345 .

    It's not working. For some reason, makesetup seems to be choking on the added sqlite3 line in Setup.local and then it emits a garbage line in the generated Makefile, resulting in an error:

    [email protected]:/src/cpython# make
    cp Setup.local /src/cpython/build/3.7.0/Python-3.7.0/Modules/
    cat pyconfig.undefs.h >> /src/cpython/build/3.7.0/Python-3.7.0/pyconfig.h
    ( \
    	cp build/3.7.0/host/lib/python3.7/`/src/cpython/build/3.7.0/host/bin/python3 -c "import sysconfig; print(sysconfig._get_sysconfigdata_name())"`.py build/3.7.0/Python-3.7.0/Lib/_sysconfigdata__emscripten_.py; \
    	cd /src/cpython/build/3.7.0/Python-3.7.0; \
    	emmake make HOSTPYTHON=/src/cpython/build/3.7.0/host/bin/python3 HOSTPGEN=/src/cpython/build/3.7.0/host/bin/pgen CROSS_COMPILE=yes libpython3.7.a \
    )
    make[1]: Entering directory '/src/cpython/build/3.7.0/Python-3.7.0'
    Makefile:271: *** missing separator.  Stop.
    make[1]: Leaving directory '/src/cpython/build/3.7.0/Python-3.7.0'
    Makefile:91: recipe for target '/src/cpython/build/3.7.0/Python-3.7.0/libpython3.7.a' failed
    make: *** [/src/cpython/build/3.7.0/Python-3.7.0/libpython3.7.a] Error 2
    
    opened by msabramo 32
  • Interactive console in pyodide-py

    Interactive console in pyodide-py

    This is the 1/3 part of the #875 splitting. It addresses:

    • sync behavior
    • full stdout/stderr support
    • clean namespace
    • tests
    • bigger font (in console.html)

    In a separate PR, I will address:

    • correct result representation
    • more tests

    And in a last PR, I will propose a completion system.

    opened by casatir 30
  • Await jsproxy

    Await jsproxy

    Implemented __await__ on JsProxy. This is a work in progress towards #245. In particular, this is related to a suggestion in @oeway's comment: https://github.com/iodide-project/pyodide/issues/245#issuecomment-744072568

    If this were in Python the definition would be:

    def __await__(self):
       x = yield self
       return x
    

    The code is a WIP still: it does not correctly handle the case where you await a JsProxy that is not a Promise. I'm not sure what the best way to handle this case is. Probably the best thing to do would be to add hiwire_is_promise using the Promise.resolve(x) === x check and raise an error if hiwire_is_promise returns false.

    opened by hoodmane 30
  • Flexible jsimport

    Flexible jsimport

    Edit: Removed resolution of #768 since it ran into #788 and fixing both will require a dedicated PR.

    This PR implements the feature proposed in #960 and ~resolves/ #768~ (and probably some other open issues with the same cause). The fix for #768 is to wrap the js object we are importing from (by default globalThis) in a JsProxy. Then dir and getattr have already been implemented on JsProxy. In particular, if the resulting object is a function, JsProxy.__getattr__ will wrap it in a JsBoundMethod. By using JsProxy internally to the JsImports modules, we get consistent behavior. I also implemented a method jsproxy that returns the wrapped jsproxy.

    I added two new APIS to the pyodide-js module: pyodide.mountPackage(name, object) and pyodide.dismountPackage(name). mountPackage(name, object) makes object a new Python module with name name. dismountPackage removes the python module with name name only if it was created with mountPackage.

    You can of course mountPackage an object on top of an existing python package that hasn't been imported yet. This is probably not a good idea but I see no good way to check for that. If you later dismountPackage(name) or del sys.modules[name] then you can import the real package.

    Also, to handle the problem of trash in the object namespace, if object is a javascript Map then getattr will use Map.get. I am going to encourage users to do it this way in the docs because I think it is strictly better: with the Map the namespace has exactly the things you put into it yourself and nothing more. (resolves #960)

    opened by hoodmane 29
  • WIP: Porting to LLVM upstream

    WIP: Porting to LLVM upstream

    Ref: https://github.com/iodide-project/pyodide/issues/476

    This doesn't work yet due to some bugs in emscripten. But hopefully this will be useful to others tackling the same problem.

    This is based on work started by @rth, but I had to rearrange the history and his original commits were lost. I intend to come back and give him authorship credit on those commits before this is finally merged.

    opened by mdboom 27
  • Update to Python 3.9.5

    Update to Python 3.9.5

    Closes https://github.com/pyodide/pyodide/issues/978

    opened by hoodmane 2
  • Improve lifetime management for PyProxy attributes

    Improve lifetime management for PyProxy attributes

    This implements the features proposed in #1617.

    opened by hoodmane 0
  • Finish InteractiveConsole revamp

    Finish InteractiveConsole revamp

    This sets up the InteractiveConsole by making subclasses of the classes from stdlib codeop that use our CodeRunner instead of the builtin compile and modifying the code from stdlib code to make a variant of code.InteractiveConsole that is designed to support top level await. Supporting top level await required a significant rearrangement of the API. In order to increase flexibility, rather than writing the result directly to stdout/stderr, we return them.

    So now runcode is an async function which takes both the source code and the CodeRunner as arguments and returns a dependent sum of:

    • ("success", result : Any) -- evaluation completed without error, the result is result, or
    • ("exception", message : str) -- an exception was returned, the formatted error message is message, the original error object is stored as sys.last_value.

    runsource requires the source string and returns a dependent sum of:

    • ("syntax_error", message : str) -- The source contains a syntax error, which was message. The original syntax error is stored as sys.last_value.
    • ("incomplete", None) -- the source is incomplete but has no syntax errors
    • ("complete", fut: Future) -- the source is complete and has no syntax errors, fut is a future which comes from using asyncio.ensure_future(self.runcode(source, code)).

    Getting the codeop completeness heuristic to work correctly requires parsing the code in "single" mode with a special undocumented compiler flag PyCF_DONT_IMPLY_DEDENT. I added an extra mode argument to CodeRunner to allow it to be used in a mode that is not exec and changed it to parse the code with compile and PyCF_ONLY_AST instead of ast.parse so that any compilation flags that affect parsing get treated correctly.

    opened by hoodmane 0
  • pillow jpg encoder error

    pillow jpg encoder error

    I try use PIL read/write in bmp, png, jpg, tif. jpg not work: image

    bug 
    opened by yxdragon 5
  • TST RecursionError maximum recursion in test_zarr

    TST RecursionError maximum recursion in test_zarr

    zarr/test_zarr.py is currently consistently failing consistently in Chrome with a recursion error.

    It might be a good occasion to compile Pyodide with -Os for smaller stack frame size https://github.com/pyodide/pyodide/issues/1541 and increasing the recursion limit a bit.

    opened by rth 3
  • install opencv-python with micropip error

    install opencv-python with micropip error

    I am recently working on a project that use ML and this library is truly a game changer for project in js... Any luck to have opencv-python included in the list of the python libraries? thank you in advance

    opened by Monder87 1
  • Pytorch support

    Pytorch support

    Does pyodide support Pytorch? Or does wasm support Pytorch?

    new package request 
    opened by JonathanSum 4
  • Cannot build MarkupSafe locally or in Docker

    Cannot build MarkupSafe locally or in Docker

    I've been unsuccessfully trying to build MarkupSafe both locally and in a container, both unsuccessful - here are the logs from running in a container: (I also tried most of the things in #884)

    Error building MarkupSafe. Printing build logs.
    running install
    running bdist_egg
    running egg_info
    writing src/MarkupSafe.egg-info/PKG-INFO
    writing dependency_links to src/MarkupSafe.egg-info/dependency_links.txt
    writing top-level names to src/MarkupSafe.egg-info/top_level.txt
    reading manifest file 'src/MarkupSafe.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    no previously-included directories found matching 'docs/_build'
    warning: no previously-included files matching '*.py[co]' found anywhere in distribution
    writing manifest file 'src/MarkupSafe.egg-info/SOURCES.txt'
    installing library code to build/bdist.linux-x86_64/egg
    running install_lib
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.8
    creating build/lib.linux-x86_64-3.8/markupsafe
    copying src/markupsafe/_constants.py -> build/lib.linux-x86_64-3.8/markupsafe
    copying src/markupsafe/_compat.py -> build/lib.linux-x86_64-3.8/markupsafe
    copying src/markupsafe/__init__.py -> build/lib.linux-x86_64-3.8/markupsafe
    copying src/markupsafe/_native.py -> build/lib.linux-x86_64-3.8/markupsafe
    copying src/markupsafe/_speedups.c -> build/lib.linux-x86_64-3.8/markupsafe
    running build_ext
    building 'markupsafe._speedups' extension
    creating build/temp.linux-x86_64-3.8
    creating build/temp.linux-x86_64-3.8/src
    creating build/temp.linux-x86_64-3.8/src/markupsafe
    cc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/usr/local/include/python3.8 -c src/markupsafe/_speedups.c -o build/temp.linux-x86_64-3.8/src/markupsafe/_speedups.o
    gcc -pthread -shared build/temp.linux-x86_64-3.8/src/markupsafe/_speedups.o -L/usr/local/lib -o build/lib.linux-x86_64-3.8/markupsafe/_speedups.cpython-38-x86_64-linux-gnu.so
    creating build/bdist.linux-x86_64
    creating build/bdist.linux-x86_64/egg
    creating build/bdist.linux-x86_64/egg/markupsafe
    copying build/lib.linux-x86_64-3.8/markupsafe/_speedups.cpython-38-x86_64-linux-gnu.so -> build/bdist.linux-x86_64/egg/markupsafe
    copying build/lib.linux-x86_64-3.8/markupsafe/_speedups.c -> build/bdist.linux-x86_64/egg/markupsafe
    copying build/lib.linux-x86_64-3.8/markupsafe/_constants.py -> build/bdist.linux-x86_64/egg/markupsafe
    copying build/lib.linux-x86_64-3.8/markupsafe/_compat.py -> build/bdist.linux-x86_64/egg/markupsafe
    copying build/lib.linux-x86_64-3.8/markupsafe/__init__.py -> build/bdist.linux-x86_64/egg/markupsafe
    copying build/lib.linux-x86_64-3.8/markupsafe/_native.py -> build/bdist.linux-x86_64/egg/markupsafe
    byte-compiling build/bdist.linux-x86_64/egg/markupsafe/_constants.py to _constants.cpython-38.pyc
    byte-compiling build/bdist.linux-x86_64/egg/markupsafe/_compat.py to _compat.cpython-38.pyc
    byte-compiling build/bdist.linux-x86_64/egg/markupsafe/__init__.py to __init__.cpython-38.pyc
    byte-compiling build/bdist.linux-x86_64/egg/markupsafe/_native.py to _native.cpython-38.pyc
    creating stub loader for markupsafe/_speedups.cpython-38-x86_64-linux-gnu.so
    byte-compiling build/bdist.linux-x86_64/egg/markupsafe/_speedups.py to _speedups.cpython-38.pyc
    creating build/bdist.linux-x86_64/egg/EGG-INFO
    copying src/MarkupSafe.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
    copying src/MarkupSafe.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
    copying src/MarkupSafe.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
    copying src/MarkupSafe.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
    writing build/bdist.linux-x86_64/egg/EGG-INFO/native_libs.txt
    zip_safe flag not set; analyzing archive contents...
    markupsafe.__pycache__._speedups.cpython-38: module references __file__
    creating dist
    creating 'dist/MarkupSafe-1.1.1-py3.8-linux-x86_64.egg' and adding 'build/bdist.linux-x86_64/egg' to it
    removing 'build/bdist.linux-x86_64/egg' (and everything under it)
    Processing MarkupSafe-1.1.1-py3.8-linux-x86_64.egg
    removing '/src/packages/.artifacts/lib/python/MarkupSafe-1.1.1-py3.8-linux-x86_64.egg' (and everything under it)
    creating /src/packages/.artifacts/lib/python/MarkupSafe-1.1.1-py3.8-linux-x86_64.egg
    Extracting MarkupSafe-1.1.1-py3.8-linux-x86_64.egg to /src/packages/.artifacts/lib/python
    MarkupSafe 1.1.1 is already the active version in easy-install.pth
    
    Installed /src/packages/.artifacts/lib/python/MarkupSafe-1.1.1-py3.8-linux-x86_64.egg
    Processing dependencies for MarkupSafe==1.1.1
    Finished processing dependencies for MarkupSafe==1.1.1
    cache:INFO: generating system asset: generated_struct_info.json... (this will be cached in "/src/emsdk/emsdk/upstream/emscripten/cache/generated_struct_info.json" for subsequent builds)
    /src/emsdk/emsdk/upstream/bin/wasm-opt: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by /src/emsdk/emsdk/upstream/bin/wasm-opt)
    /src/emsdk/emsdk/upstream/bin/wasm-opt: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by /src/emsdk/emsdk/upstream/bin/wasm-opt)
    emcc: error: error running binaryen executable (/src/emsdk/emsdk/upstream/bin/wasm-opt). Please check your binaryen installation
    FAIL: Compilation failed!: ['/src/emsdk/emsdk/upstream/emscripten/emcc', '-D_GNU_SOURCE', '-o', '/tmp/tmpgt8h23oz.js', '/tmp/tmpm9_3york.c', '-O0', '-Werror', '-Wno-format', '-nostdlib', '/src/emsdk/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/pic/libcompiler_rt.a', '-s', 'BOOTSTRAPPING_STRUCT_INFO=1', '-s', 'STRICT', '-s', 'SINGLE_FILE', '-Wno-error=version-check', '-Wno-deprecated']
    emcc -O2 -g -fPIC -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -Wall -fPIC -I/src/cpython/installs/python-3.8.2/include/python3.8 -c src/markupsafe/_speedups.c -o build/temp.linux-x86_64-3.8/src/markupsafe/_speedups.o
    emcc -s BINARYEN_EXTRA_PASSES="[email protected]" -O2 -s MODULARIZE=1 -s EMULATE_FUNCTION_POINTER_CASTS=1 -s LINKABLE=1 -s EXPORT_ALL=1 -s WASM=1 -std=c++14 -s LZ4=1 -s SIDE_MODULE=1 build/temp.linux-x86_64-3.8/src/markupsafe/_speedups.o -o build/lib.linux-x86_64-3.8/markupsafe/_speedups.cpython-38-x86_64-linux-gnu.so
    Traceback (most recent call last):
      File "/usr/local/lib/python3.8/runpy.py", line 193, in _run_module_as_main
        return _run_code(code, main_globals, None,
      File "/usr/local/lib/python3.8/runpy.py", line 86, in _run_code
        exec(code, run_globals)
      File "/src/pyodide-build/pyodide_build/__main__.py", line 50, in <module>
        main()
      File "/src/pyodide-build/pyodide_build/__main__.py", line 44, in main
        args.func(args)
      File "/src/pyodide-build/pyodide_build/buildpkg.py", line 344, in main
        build_package(path, args)
      File "/src/pyodide-build/pyodide_build/buildpkg.py", line 279, in build_package
        compile(path, srcpath, pkg, args)
      File "/src/pyodide-build/pyodide_build/buildpkg.py", line 147, in compile
        subprocess.run(
      File "/usr/local/lib/python3.8/subprocess.py", line 512, in run
        raise CalledProcessError(retcode, process.args,
    subprocess.CalledProcessError: Command '['/usr/local/bin/python3', '-m', 'pyodide_build', 'pywasmcross', '--cflags', '-O2 -g -fPIC ', '--cxxflags', ' ', '--ldflags', '-s BINARYEN_EXTRA_PASSES="[email protected]" -O2 -s MODULARIZE=1 -s EMULATE_FUNCTION_POINTER_CASTS=1 -s LINKABLE=1 -s EXPORT_ALL=1 -s WASM=1 -std=c++14 -s LZ4=1  -s SIDE_MODULE=1 ', '--target', '/src/cpython/installs/python-3.8.2', '--install-dir', '/src/packages/.artifacts', '--replace-libs', '']' returned non-zero exit status 1.
    [2021-06-03 23:40:38] Building package MarkupSafe...
    [2021-06-03 23:40:47] done building package MarkupSafe in 9.1 s.
    Traceback (most recent call last):
      File "/src/packages/.artifacts/bin/pyodide-build", line 33, in <module>
        sys.exit(load_entry_point('pyodide-build', 'console_scripts', 'pyodide-build')())
      File "/src/pyodide-build/pyodide_build/__main__.py", line 44, in main
        args.func(args)
      File "/src/pyodide-build/pyodide_build/buildall.py", line 360, in main
        build_packages(packages_dir, outputdir, args)
      File "/src/pyodide-build/pyodide_build/buildall.py", line 248, in build_packages
        build_from_graph(pkg_map, outputdir, args)
      File "/src/pyodide-build/pyodide_build/buildall.py", line 234, in build_from_graph
        raise pkg
      File "/src/pyodide-build/pyodide_build/buildall.py", line 217, in builder
        pkg.build(outputdir, args)
      File "/src/pyodide-build/pyodide_build/buildall.py", line 109, in build
        p.check_returncode()
      File "/usr/local/lib/python3.8/subprocess.py", line 444, in check_returncode
        raise CalledProcessError(self.returncode, self.args, self.stdout,
    subprocess.CalledProcessError: Command '['/usr/local/bin/python3', '-m', 'pyodide_build', 'buildpkg', '/src/packages/MarkupSafe/meta.yaml', '--cflags', '-O2 -g -fPIC', '--cxxflags', '', '--ldflags', '-s BINARYEN_EXTRA_PASSES="[email protected]" -O2 -s MODULARIZE=1 -s EMULATE_FUNCTION_POINTER_CASTS=1 -s LINKABLE=1 -s EXPORT_ALL=1 -s WASM=1 -std=c++14 -s LZ4=1  -s SIDE_MODULE=1', '--target', '/src/cpython/installs/python-3.8.2', '--install-dir', '/src/packages/.artifacts']' returned non-zero exit status 1.
    make: *** [Makefile:11: all] Error 1
    make: Leaving directory '/src/packages'
    
    opened by radu-matei 2
  • Web Worker postMessage failing using numpy

    Web Worker postMessage failing using numpy

    pyodide version: 0.17.0 i'm using pyodide in my react app. I use the off-the-shelf webworker provided from the example. It more or less works, however, there's one "interesting" caveat that I thought I'd bring to the attention of the maintainers:

    say I have the following python code:

    import pandas as pd
    pd.DataFrame([1,2,3])
    

    If I open up a python REPL or even use the Pyodide Terminal Emulator

    I get the output:

       0
    0  1
    1  2
    2  3
    

    However, whenever

    const results = await self.pyodide.runPythonAsync(python);
    self.postMessage({ results })
    

    gets called, I get the following:

    failed to execute 'postMessage' on 'DedicatedWorkerGlobalScope': [object Object] could not be cloned

    the googles and the overflowing stack lead me to believe it might have something to do with this but seems like a red herring.

    After banging my head on the keyboard for a while I realized you could sort of work around this by doing:

    self.postMessage({results: `${results}`})
    

    except if we change the initial example to:

    import pandas as pd
    pd.DataFrame([1,2,3])
    print("SOMETHING")
    

    now result is undefined. I can get the SOMETHING by redirecting stdout and returning that, but overall, this is kind of not an ideal situation.

    I tried splitting the statements line-wise and iterating (ala a repl), but that doesn't work either.

    I did a fair amount of diligence on this and realized this might be solved by https://github.com/pyodide/pyodide/pull/1563, but it'd be great if someone could provide a working example of how this should work or maybe point me in the direction of what i'm doing wrong as i'm basically doing exactly what the docs said and can't get this seemingly trivial example to work.

    maybe update the docs? happy to provide more context + an example.

    opened by enahs 0
  • Strange behavior with importing seaborn !

    Strange behavior with importing seaborn !

    Hello! Someone ran into issues importing seaborn through a starboard notebook and, at first it seemed like a simple issue of wheel availability and versions. But in trying to find a specific version that maybe works, something just doesn't seem right.

    Ultimately I think that micropip.install('seaborn==0.9.0') worked in pyodide 0.16 but it doesn't seem to work in 0.17, and I can't quite figure out why!

    The post + my original response is here, for further context: https://community.starboard.gg/t/issue-with-micropip-installing-packages/100

    opened by unhott 4
Releases(0.17.0)
This repository is an archive of emails that are sent by the awesome Quincy Larson every week.

Awesome Quincy Larson Email Archive This repository is an archive of emails that are sent by the awesome Quincy Larson every week. If you fi

Sourabh Joshi 382 Jun 13, 2021
Buffer Overflows

BOF Buffer Overflows 1. BOF tips Practice using mona.py Download vulnerable exe from Exploit DB.

Vinh Nguyễn 25 Mar 5, 2021
a sketch of what a zkvm could look like

We want to build a ZKP that validates an entire EVM block or as much of it as we can efficiently. Its okay to adjust the gas costs for every EVM opcode. Its also to exclude some opcodes for now if they are super expensive. Its okay to exclude precompiles.

null 14 Jun 7, 2021
Write complicated anonymous functions other than lambdas in Python.

lambdex allows you to write multi-line anonymous function expression (called a lambdex) in an idiomatic manner.

Xie Jingyi 52 May 18, 2021
A collection of full-stack resources for programmers.

A collection of full-stack resources for programmers.

Charles-Axel Dein 15.6k Jun 13, 2021
Simple python code for compile brainfuck program.

py-brainf*ck Just a basic compiled that compiles your brainf*ck codes and gives you informations about memory, used cells, dumped version, logs etc...

null 4 Jun 13, 2021
PyDy, short for Python Dynamics, is a tool kit written in the Python

PyDy, short for Python Dynamics, is a tool kit written in the Python programming language that utilizes an array of scientific programs to enable the study of multibody dynamics. The goal is to have a modular framework and eventually a physics abstraction layer which utilizes a variety of backends that can provide the user with their desired workflow

PyDy 250 Jun 14, 2021
A Python simple Dice Simulator just for fun

Dice Simulator ?? A Simple Python Dice Simulator ?? ?? ?? Description: That program make your RPG session more easy and simple. Roll the dice never be

Lauro Brant 15 Jun 1, 2021
Welcome to my pod transcript search webb app!

pod_transcript_search Welcome to the pod transcript search webb app! Tech stack used: Languages used: Python (for the back-end), JavaScript (for the f

null 3 Jun 5, 2021
200 LeetCode problems

LeetCode I classify 200 leetcode problems into some categories and upload my code to who concern WEEK 1 # Title Difficulty Array 15 3Sum Medium 1324 P

Hoang Cao Bao 18 Jun 10, 2021
Some ideas and tools to develop Python 3.8 plugins for GIMP 2.99.4

gimp-python-development Some ideas and tools to develop Python 3.8 plugins for GIMP 2.99.4. GIMP 2.99.4 is the latest unstable pre-release of GIMP 3.

Ismael Benito 45 May 29, 2021
Repo Home WPDrawBot - (Repo, Home, WP) A powerful programmatic 2D drawing application for MacOS X which generates graphics from Python scripts. (graphics, dev, mac)

DrawBot DrawBot is a powerful, free application for macOS that invites you to write Python scripts to generate two-dimensional graphics. The built-in

Frederik Berlaen 283 Jun 2, 2021
Participants of Bertelsmann Technology Scholarship created an awesome list of resources and they want to share it with the world, if you find illegal resources please report to us and we will remove.

Participants of Bertelsmann Technology Scholarship created an awesome list of resources and they want to share it with the world, if you find illegal

Wissem Marzouki 26 Apr 4, 2021
A Trace Explorer for Reverse Engineers

Tenet - A Trace Explorer for Reverse Engineers Overview Tenet is an IDA Pro plugin for exploring execution traces. The goal of this plugin is to provi

null 562 Jun 7, 2021