The Foundation for All Legate Libraries

Related tags

GraphQL legate.core
Overview

Legate

The Legate project endeavors to democratize computing by making it possible for all programmers to leverage the power of large clusters of CPUs and GPUs by running the same code that runs on a desktop or a laptop at scale. Using this technology, computational and data scientists can develop and test programs on moderately sized data sets on local machines and then immediately scale up to larger data sets deployed on many nodes in the cloud or on a supercomputer without any code modifications. In visual terms:

drawing

The Legate project is built upon two foundational principles:

  1. For end users, such as computational and data scientists, the programming model must be identical to programming a single sequential CPU on their laptop or desktop. All concerns relating to parallelism, data distribution, and synchronization must be implicit. The cloud or a supercomputer should appear as nothing more than a super-powerful CPU core.
  2. Software must be compositional and not just interoperable (i.e. functionally correct). Libraries developed in the Legate ecosystem must be able to exchange partitioned and distributed data without requiring "shuffles" or unnecessary blocking synchronization. Computations from different libraries should be able to use arbitrary data and still be reordered across abstraction boundaries to hide communication and synchronization latencies where the original sequential semantics of the program allow. This is essential for achieving speed-of-light performance on large scale machines.

The Legate project is still in its nascent stages of development, but much of the fundamental architecture is in place. We encourage development and contributions to existing Legate libraries, such as Legate NumPy and Legate Pandas, as well as the development of new Legate libraries. Pull requests are welcomed.

  1. Why Legate?
  2. What is the Legate Core?
  3. How Does Legate Work?
  4. How Do I Install Legate?
  5. How Do I Use Legate?
  6. Other FAQs
  7. Documentation
  8. Next Steps

Why Legate?

Computational problems today continue to grow both in their complexity as well as the scale of the data that they consume and generate. This is true both in traditional HPC domains as well as enterprise data analytics cases. Consequently, more and more users truly need the power of large clusters of both CPUs and GPUs to address their computational problems. Not everyone has the time or resources required to learn and deploy the advanced programming models and tools needed to target this class of hardware today. Legate aims to bridge this gap so that any programmer can run code on any scale machine without needing to be an expert in parallel programming and distributed systems, thereby allowing developers to bring the problem-solving power of large machines to bear on more kinds of challenging problems than ever before.

What is the Legate Core?

The Legate Core is our version of Apache Arrow. Apache Arrow has significantly improved composability of software libraries by making it possible for different libraries to share in-memory buffers of data without unnecessary copying. However, it falls short when it comes to meeting two of our primary requirements for Legate:

  1. Arrow only provides an API for describing a physical representation of data as a single memory allocation. There is no interface for describing cases where data has been partitioned and then capturing the logical relationships of those partitioned subsets of data.
  2. Arrow is mute on the subject of synchronization. Accelerators such as GPUs achieve significantly higher performance when computations are performed asynchronously with respect to other components of the system. When data is passed between libraries today, accelerators must be pessimistically synchronized to ensure that data dependences are satisfied across abstraction boundaries. This might result in tolerable overheads for single GPU systems, but can result in catastrophically poor performance when hundreds of GPUs are involved.

The Legate Core provides an API very similar to Arrow's interface with several important distinctions that provide stronger guarantees about data coherence and synchronization to aid library developers when building Legate libraries. These guarantees are the crux of how libraries in the Legate ecosystem are able to provide excellent composability.

The Legate Core API imports several important concepts from Arrow such that users that are familiar with Arrow already will find it unsurprising. We use the same type system representation as Arrow so libraries that have already adopted it do not need to learn or adapt to a new type system. We also reuse the concept of an Array from Arrow. The LegateArray class supports many of the same methods as the Arrow Array interface (we'll continue to add methods to improve compatibility). The main difference is that instead of obtaining Buffer objects from arrays to describe allocations of data that back the array, the Legate Core API introduces a new primitive called a LegateStore which provides a new interface for reasoning about partitioned and distributed data in asynchronous execution environments.

Any implementation of a LegateStore must maintain the following guarantees to clients of the Legate Core API (i.e. Legate library developers):

  1. The coherence of data contained in a LegateStore must be implicitly managed by the implementation of the Legate Core API. This means that no matter where data is requested to perform a computation in a machine, the most recent modifications to that data in program order must be reflected. It should never be clients responsibility to maintain this coherence.
  2. It should be possible to create arbitrary views onto LegateStore objects such that library developers can precisely describe the working sets of their computations. Modifications to views must be reflected onto all aliasing views data. This property must be maintained by the Legate Core API implementation such that it is never the concern of clients.
  3. Dependence management between uses of the LegateStore objects and their views is the responsibility of Legate Core API regardless of what (asynchronous) computations are performed on LegateStore objects or their views. This dependence analysis must be both sound and precise. It is illegal to over-approximate dependences. This dependence analysis must also be performed globally in scope. Any use of the LegateStore on any processor/node in the system must abide by the original sequential semantics of the program

Note that we do not specify exactly what the abstractions are that are needed for implementing LegateStore objects. Our goal is not prescribe what these abstractions are as they may be implementation dependent. Our only requirements are that they have these properties to ensure that incentives are aligned in such a way for Legate libraries to achieve a high degree of composability at any scale of machine. Indeed, these requirements shift many of the burdens that make implementing distributed and accelerated libraries hard off of the library developers and onto the implementation of the Legate Core API. This is by design as it allows the costs to be amortized across all libraries in the ecosystem and ensures that Legate library developers are more productive.

How Does Legate Work?

Our implementation of the Legate Core API is built on top of the Legion programming model and runtime system. Legion was originally designed for large HPC applications that target supercomputers and consequently applications written in the Legion programming model tend to both perform and scale well on large clusters of both CPUs and GPUs. Legion programs are also easy to port to new machines as they inherently decouple the machine-independent specification of computations from decisions about how that application is mapped to the target machine. Due to this abstract nature, many programmers find writing Legion programs challenging. By implementing the Legate Core API on top of Legion, we've made it easier to use Legion such that developers can still get access to the benefits of Legion without needing to learn all of the lowest-level interfaces.

The Legion programming model greatly aids in implementing the Legate Core API. Data types from libraries, such as arrays in Legate NumPy are mapped down onto LegateStore objects that wrap Legion data types such as logical regions or futures. In the case of regions, Legate application libraries rely heavily on Legion's support for partitioning of logical regions into arbitrary subregion views. Each library has its own heuristics for computing such partitions that take into consideration the computations that will access the data, the ideal sizes of data to be consumed by different processor kinds, and the available number of processors. Legion automatically manages the coherence of subregion views regardless of the scale of the machine.

Computations in Legate application libraries are described by Legion tasks. Tasks describe their data usage in terms of LegateStore objects, thereby allowing Legion to infer where dependences exist. Legion uses distributed bounding volume hierarchies, similar to a high performance ray-tracer, to soundly and precisely perform dependence analysis on logical regions and insert the necessary synchronization between tasks to maintain the original sequential semantics of a Legate program.

Each Legate application library also comes with its own custom Legion mapper that uses heuristics to determine the best choice of mapping for tasks (e.g. are they best run on a CPU or a GPU). All Legate tasks are currently implemented in native C or CUDA in order to achieve excellent performance on the target processor kind, but Legion has bindings in other languages such as Python, Fortran, and Lua for users that would prefer to use them. Importantly, by using Legion, Legate is able to control the placement of data in order to leave it in-place in fast memories like GPU framebuffers across tasks.

When running on large clusters, Legate leverages a novel technology provided by Legion called "control replication" to avoid the sequential bottleneck of having one node farm out work to all the nodes in the cluster. With control replication, Legate will actually replicate the Legate program and run it across all the nodes of the machine at the same time. These copies of the program all cooperate logically to appear to execute as one program. When communication is necessary between different computations, the Legion runtime's program analysis will automatically detect it and insert the necessary data movement and synchronization across nodes (or GPU framebuffers). This is the transformation that allows sequential programs to run efficiently at scale across large clusters as though they are running on a single processor.

How Do I Install Legate?

Dependencies

Legate has been tested on both Linux and Darwin operating systems, although only a few flavors of Linux such as Ubuntu have been thoroughly tested. There is currently no support for Windows.

The Legate Core currently requires Python >= 3.7 and the following packages:

  • pyarrow=1.0.1
  • numpy
  • CUDA >= 8.0
  • C++14 compatible compiler (g++, clang, or nvc++)

Installation

The Legate Core library comes with both a standard setup.py script and a custom install.py script in the top-level directory of the repository that will build and install the Legate Core library. Users can use either script to install Legate as they will produce the same effect. A simple single-node, CPU-only installation of Legate into targetdir will be performed by:

./setup.py --install-dir targetdir

To add GPU support for Legate simply use the --cuda flag. The first time you request GPU support you will need to use the --with-cuda flag to specify the location of your CUDA installation. For later invocations you do not need to use this flag again the installation scripts will remember the location of your CUDA installation until you tell it differently. You can also specify the name of the CUDA architecture you want to target with the --arch flag (the default is volta but you can also specify kepler, maxwell, pascal, or ampere).

./install.py --cuda --with-cuda /usr/local/cuda/ --arch volta

For multi-node support Legate uses GASNet which can be requested using the the --gasnet flag. If you have an existing GASNet installation then you can inform the install script with the --with-gasnet flag. The install.py script also requires you to specify the interconnect network of the target machine using the --conduit flag (current choices are one of ibv for Infiniband, or gemini or aries for Cray interconnects). For example this would be an installation for a DGX SuperPOD:

./install.py --gasnet --conduit ibv --cuda --arch ampere

Alternatively here is an install line for the Piz-Daint supercomputer:

./install.py --gasnet --conduit aries --cuda --arch pascal

To see all the options available for installing Legate, just run with the --help flag:

./install.py --help

Options passed to setup.py will automatically be forwarded to install.py so that users can use them interchangeably (this provides backwards compatibility for earlier versions of Legate when only install.py existed).

Python used by Legate

Legate discovers the Python library and version during build time, and then it builds all successive Legate libraries against that version of Python. The build system tries to detect the Python setup from the default Python interpreter, but sometimes it is unsuccessful or a different version of Python than the one in the environment may be desired. To use a different version of Python than the one available in the environment, the PYTHON_ROOT variable must be set to the base directory of the desired Python installation.

Sometimes, the search for the Python library may fail. In such situation, the build system generates a warning:

runtime.mk: cannot find libpython3.6*.so - falling back to using LD_LIBRARY_PATH

In this case, Legate will use the Python library that is available at runtime, if any. To explicitly specify the Python library to use, PYTHON_LIB should be set to the location of the library, and PYTHON_VERSION_MAJOR should be set to 3.

Toolchain selection

Legate relies on environment variables to select its toolchain and build flags (such as CXX, CC_FLAGS, LD_FLAGS, NVCC_FLAGS). Setting these environment variables prior to building and installing Legate will influence the build of any C++ and CUDA code in Legate.

How Do I Use Legate?

After installing the Legate Core library, the next step is to install a Legate application library such as Legate NumPy. The installation process for a Legate application library will require you to provide a pointer to the location of your Legate Core library installation as this will be used to configure the installation of the Legate application library. After you finish installing any Legate application libraries, you can then simply replace their import statements with the equivalent ones from any Legate application libraries you have installed. For example, you can change this:

import numpy as np

to this:

import legate.numpy as np

After this, you can use the legate driver script in the bin directory of your installation to run any Python program. For example, to run your script in the default configuration (4 CPUs cores and 4 GB of memory) just run:

installdir/bin/legate my_python_program.py [other args]

The legate script also allows you to control the amount of resources that Legate consumes when running on the machine. The --cpus and --gpus flags are used to specify how many CPU and GPU processors should be used on a node. The --sysmem flag can be used to specify how many MBs of DRAM Legate is allowed to use per node, while the --fbmem flag controls how many MBs of framebuffer memory Legate is allowed to use per GPU. For example, when running on a DGX station, you might run your application as follows:

installdir/bin/legate --cpus 16 --gpus 4 --sysmem 100000 -fbmem 15000 my_python_program.py

This will make 16 CPU processors and all 4 GPUs available for use by Legate. It will also allow Legate to consume up to 100 GB of DRAM memory and 15 GB of framebuffer memory per GPU for a total of 60 GB of GPU framebuffer memory. Note that you probably will not be able to make all the resources of the machine available for Legate as some will be used by the system or Legate itself for meta-work. Currently if you try to exceed these resources during execution then Legate will inform you that it had insufficient resources to complete the job given its current mapping heuristics. If you believe the job should fit within the assigned resources please let us know so we can improve our mapping heuristics. There are many other flags available for use in the legate driver script that you can use to communicate how Legate should view the available machine resources. You can see a list of them by running:

installdir/bin/legate --help

In addition to running Legate programs, you can also use Legate in an interactive mode by simply not passing any *py files on the command line. You can still request resources just as you would though with a normal file. Legate will still use all the resources available to it, including doing multi-node execution.

installdir/bin/legate --cpus 16 --gpus 4 --sysmem 100000 -fbmem 15000
Welcome to Legion Python interactive console
>>>

Note that Legate does not currently support multi-tenancy cases where different users are attempting to use the same hardware concurrently.

Distributed Launch

If legate is compiled with GASNet support (see the installation section), it can be run in parallel by using the --nodes option followed by the number of nodes to be used. Whenever the --nodes option is used, Legate will be launched using mpirun, even with --nodes 1. Without the --nodes option, no launcher will be used. Legate currently supports mpirun, srun, and jsrun as launchers and we are open to adding additional launcher kinds. You can select the target kind of launcher with --launcher.

Debugging and Profiling

Legate also comes with several tools that you can use to better understand your program both from a correctness and a performance standpoint. For correctness, Legate has facilities for constructing both dataflow and event graphs for the actual run of an application. These graphs require that you have an installation of GraphViz available on your machine. To generate a dataflow graph for your Legate program simply pass the --dataflow flag to the legate.py script and after your run is complete we will generate a dataflow_legate.pdf file containing the dataflow graph of your program. To generate a corresponding event graph you simply need to pass the --event flag to the legate.py script to generate a event_graph_legate.pdf file. These files can grow to be fairly large for non-trivial programs so we encourage you to keep your programs small when using these visualizations or invest in a robust PDF viewer.

For profiling all you need to do is pass the --profile flag to Legate and afterwards you will have a legate_prof directory containing a web page that can be viewed in any web browser that displays a timeline of your program's execution. You simply need to load the index.html page from a browser. You may have to enable local JavaScript execution if you are viewing the page from your local machine (depending on your browser).

We recommend that you do not mix debugging and profiling in the same run as some of the logging for the debugging features requires significant file I/O that can adversely effect the performance of the application.

Other FAQs

  • Does Legate only work on NVIDIA hardware? No, Legate will run on most kinds of hardware, anywhere that Legion and GASNet will run. This includes x86, ARM, and PowerPC CPUs. GASNet (and therefore Legate) also includes support for Infiniband, Cray, Omnipath, and (ROC-)Ethernet based interconnects.

  • What languages does the Legate Core API have bindings for? Currently the Legate Core bindings are only available in Python. Watch this space for new language bindings soon or make a pull request to contribute your own. Legion has a C API which should make it easy to develop bindings in any language with a foreign function interface.

  • Do I have to build drop-in replacement libraries? No! While we've chosen to provide drop-in replacement libraries for popular Python libraries to illustrate the benefits of Legate, you are both welcomed and encouraged to develop your own libraries on top of Legate. We promise that they will compose well with other existing Legate libraries.

  • What other libraries are you planning to release for the Legate ecosystem? We're still working on that. If you have thoughts about what is important please let us know so that we can get a feel for where best to put our time.

  • Can I use Legate with other Legion libraries? Yes! If you're willing to extract the Legion primitives from the LegateStore objects you should be able to pass them into other Legion libraries such as FlexFlow.

  • Does Legate interoperate with X? Yes, probably, but we don't recommend it. Our motivation for building Legate is to provide the bare minimum subset of functionality that we believe is essential for building truly composable software that can still run at scale. No other systems out there met our needs. Providing interoperability with those other systems will destroy the very essence of what Legate is and significantly dilute its benefits. All that being said, Legion does provide some means of doing stop-the-world exchanges with other runtime system running concurrently in the same processes. If you are interested in pursuing this approach please open an issue on the Legion github issue tracker as it will be almost entirely orthogonal to how you use Legate.

Documentation

A complete list of available features can is provided in the API reference.

Next Steps

We recommend starting by experimenting with at least one Legate application library to test out performance and see how Legate works. If you are interested in building your own Legate application library, we recommend that you investigate our Legate Hello World application library that provides a small example of how to get started developing your own drop-in replacement library on top of Legion using the Legate Core library.

Comments
  • Runtime warning when reusing arrays in pytest modules

    Runtime warning when reusing arrays in pytest modules

    When "reusing" a cunumeric array in a pytest module, all test can pass, but a runtime warning is issued at the end:

    Exception ignored in: <function RegionField.__del__ at 0x7f30c0594320>
    Traceback (most recent call last):
      File "/home/bryan/work/legate.core/install37/lib/python3.7/site-packages/legate/core/store.py", line 124, in __del__
      File "/home/bryan/work/legate.core/install37/lib/python3.7/site-packages/legate/core/store.py", line 251, in detach_external_allocation
      File "/home/bryan/work/legate.core/install37/lib/python3.7/site-packages/legate/core/runtime.py", line 447, in detach_external_allocation
      File "/home/bryan/work/legate.core/install37/lib/python3.7/site-packages/legate/core/runtime.py", line 436, in _remove_allocation
      File "/home/bryan/work/legate.core/install37/lib/python3.7/site-packages/legate/core/runtime.py", line 430, in _remove_attachment
    TypeError: argument of type 'NoneType' is not iterable
    Exception ignored in: <function RegionField.__del__ at 0x7f30c0594320>
    Traceback (most recent call last):
      File "/home/bryan/work/legate.core/install37/lib/python3.7/site-packages/legate/core/store.py", line 124, in __del__
      File "/home/bryan/work/legate.core/install37/lib/python3.7/site-packages/legate/core/store.py", line 251, in detach_external_allocation
      File "/home/bryan/work/legate.core/install37/lib/python3.7/site-packages/legate/core/runtime.py", line 447, in detach_external_allocation
      File "/home/bryan/work/legate.core/install37/lib/python3.7/site-packages/legate/core/runtime.py", line 436, in _remove_allocation
      File "/home/bryan/work/legate.core/install37/lib/python3.7/site-packages/legate/core/runtime.py", line 430, in _remove_attachment
    TypeError: argument of type 'NoneType' is not iterable
    

    The warning only appears when -cunumeric:test is specified.

    To reproduce, run the code below with legate mre.py -cunumeric:test

    # mre.py
    
    import sys
    import pytest
    import cunumeric as cnp
    
    x = cnp.array([1, 2, 3])
    
    def test_lt():
        y = x < 2
    
    def test_le():
        y = x <= 2
    
    pytest.main(sys.argv)
    
    opened by bryevdv 19
  • added docs for git, versions, and changelog

    added docs for git, versions, and changelog

    This describes a slightly more conservative approach than what we discussed, but this is taken from the RAPIDS team. I thought perhaps folks on the @nv-legate/core-team would find it more comfortable.

    opened by mmccarty 19
  • Add script to generate conda envs

    Add script to generate conda envs

    cc @manopapad

    This PR adds a script to generate a matrix of conda envs for

    {osx, linux} x {py version} x {ctk version}
    

    All of them are labeled "test" for their use, and reproduce the structure of the existing "test" env files.

    This can certainly be made more sophisticated (e.g. handling openmpi, or splitting different envs for different use scenarios). Currently generates the following files:

    ~/build_test/legate.core bv/driver_refactor*
    bldtest ❯ python scripts/generate-conda-envs.py 
    ------- environment-test-linux-py38-cuda-10.2.yaml
    ------- environment-test-linux-py38-cuda-11.0.yaml
    ------- environment-test-linux-py38-cuda-11.1.yaml
    ------- environment-test-linux-py38-cuda-11.2.yaml
    ------- environment-test-linux-py38-cuda-11.3.yaml
    ------- environment-test-linux-py38-cuda-11.4.yaml
    ------- environment-test-linux-py38-cuda-11.5.yaml
    ------- environment-test-linux-py38-cuda-11.6.yaml
    ------- environment-test-linux-py38-cuda-11.7.yaml
    ------- environment-test-linux-py38.yaml
    ------- environment-test-linux-py39-cuda-10.2.yaml
    ------- environment-test-linux-py39-cuda-11.0.yaml
    ------- environment-test-linux-py39-cuda-11.1.yaml
    ------- environment-test-linux-py39-cuda-11.2.yaml
    ------- environment-test-linux-py39-cuda-11.3.yaml
    ------- environment-test-linux-py39-cuda-11.4.yaml
    ------- environment-test-linux-py39-cuda-11.5.yaml
    ------- environment-test-linux-py39-cuda-11.6.yaml
    ------- environment-test-linux-py39-cuda-11.7.yaml
    ------- environment-test-linux-py39.yaml
    ------- environment-test-linux-py310-cuda-10.2.yaml
    ------- environment-test-linux-py310-cuda-11.0.yaml
    ------- environment-test-linux-py310-cuda-11.1.yaml
    ------- environment-test-linux-py310-cuda-11.2.yaml
    ------- environment-test-linux-py310-cuda-11.3.yaml
    ------- environment-test-linux-py310-cuda-11.4.yaml
    ------- environment-test-linux-py310-cuda-11.5.yaml
    ------- environment-test-linux-py310-cuda-11.6.yaml
    ------- environment-test-linux-py310-cuda-11.7.yaml
    ------- environment-test-linux-py310.yaml
    ------- environment-test-darwin-py38.yaml
    ------- environment-test-darwin-py39.yaml
    ------- environment-test-darwin-py310.yam
    

    A sample env looks like:

    name: legate-core-test
    channels:
      - conda-forge
    dependencies:
    
      - python=3.9
    
      # build
      - c-compiler
      - cmake>=3.24
      - cxx-compiler
      - gcc_linux-64 # [linux64]
      - git
      - make
      - ninja
      - openmpi
      - scikit-build>=0.14.1
      - setuptools>=61
      - sysroot_linux-64==2.17 # [linux64]
      - zlib
    
      # runtime
      - cffi
      - llvm-openmp
      - numpy>=1.22
      - opt_einsum
      - pyarrow>=5
      - scipy
      - typing_extensions
    
      # tests
      - clang-tools>=8
      - clang>=8
      - colorama
      - coverage
      - mock
      - mypy>=0.961
      - pre-commit
      - pytest-cov
      - pytest-lazy-fixture
      - pytest
      - types-docutils
    
      - pip
      - pip:
    
          # tests
          - tifffile
    
          # docs
          - jinja2
          - markdown<3.4.0
          - pydata-sphinx-theme
          - recommonmark
          - sphinx-copybutton
          - sphinx-markdown-tables
          - sphinx>=4.4.0
    
    category:improvement 
    opened by bryevdv 13
  • Allow launcher_extra to split quoted values

    Allow launcher_extra to split quoted values

    This PR makes the driver automatically split --launcher-extra values on whitespace internally, Doing so allows for simpler invocations with quoted strings like this:

    legate --launcher-extra="-H =g0002,g0003"
    

    rather than having to manually split across multiple --launcher-extra arguments as is currently required:

    legate --launcher-extra="-H" --launcher-extra="g0002,g0003"
    

    cc @lightsighter

    category:improvement 
    opened by bryevdv 11
  • Update python installation method

    Update python installation method

    Legion and legate are using easy_install to install the python packages to an arbitrary location. This mode is deprecated and will soon be removed. We should switch to following standard conventions (e.g. pip-based), which install directly into the currently active virtual environment (e.g. the one managed by conda).

    Here are some relevant messages we get during installation:

    DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
    DeprecationWarning: The distutils.sysconfig module is deprecated, use sysconfig instead
    SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
    EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
    SetuptoolsDeprecationWarning: Custom 'build_py' does not implement 'get_data_files_without_manifest'.
    Please extend command classes from setuptools instead of distutils.
    

    One thing I would like to know is how easy it would be to "uninstall" our packages in this mode, in case we want to do a clean build.

    Copying from my discussion with @bryevdv on alternatives:

    The issue probably comes down to whether "build cmd + package/install cmd" is acceptable or whether you want some one-shot to do everything. One option might be to have cmake build the binary artifacts and install them in the source tree (or somewhere), and then a pip install . just does a fairly normal "python package install". Maybe the cmake process even does the pip install . for you as the last step. If you want things "driven" from python then you still have a setup.py that invokes the C++ build. Since we can't call setup.py as a script anymore (that is deprecated) we'd have to rely on only env vars to control any conditional logic inside it when you call pip install . or build. There are evidently other tools that support custom steps, like the hatch tool mentioned in use by Jupyter.

    CC @trxcllnt for input

    opened by manopapad 11
  • cunumeric/legate eats the `-l` command line flag

    cunumeric/legate eats the `-l` command line flag

    The following program:

    import cunumeric
    import argparse
    
    parser = argparse.ArgumentParser()
    parser.add_argument("-l", type=int, default=None)
    args, _ = parser.parse_known_args()
    print(args.l)
    

    run as legate repro.py -l 5 will print None. However, if the import cunumeric is removed, then it will print 5. I don't see anywhere in the legate help messages etc why this might happen.

    Pinging @bryevdv since you probably know what's up here.

    opened by rohany 10
  • bin/legate doesn't find legion_top in non-editable installation

    bin/legate doesn't find legion_top in non-editable installation

    I installed legate.core using ./install.py --install-dir ../install --verbose --no-build-isolation, and when running bin/legate I see the error:

    [0 - 113f0adc0]    0.000081 {4}{threads}: reservation ('CPU proc 1d00000000000003') cannot be satisfied
    [0 - 700001ade000]    0.050624 {6}{python}: unable to import Python module legion_top
    ModuleNotFoundError: No module named 'legion_top'
    Assertion failed: (0), function find_or_import_function, file /Users/rohany/Documents/nvidia/legate.core/_skbuild/macosx-10.15-x86_64-3.9/cmake-build/_deps/legion-src/runtime/realm/python/python_module.cc, line 230.
    Signal 6 received by node 0, process 30628 (thread 700001ade000) - obtaining backtrace
    Signal 6 received by process 30628 (thread 700001ade000) at: stack trace: 0 frames
    
    opened by rohany 10
  • Cleanup / Initial pass for type annotations

    Cleanup / Initial pass for type annotations

    This PR makes some small cleanups:

    • Use new style-classes everywhere
    • Remove some obsolete idioms and imports

    Additionally, this PR makes an initial pass at adding type annotations, starting with the following modules:

    legate.core.legion
    legate.core.projection
    legate.core.types
    legate.core.utils
    

    For now, type checking is disabled globally, and files are added to an explicit allow-list. Eventually we can just enable type checking on the whole legate directory. No modifications to try to add mypy to CI or pre-commit were made here.

    Some notes about the approach:

    • from __future__ import annotations boilerplate was added everywhere. In my experience __future__ imports should be used everywhere, or nowhere, to prevent the possibility of different behaviours in different modules.
    • For now, had to resort to Any for ~5-10% of types. I intend to refine the more complicated types in future PRs. Alot of these are related to CFFI, which is a mess.
    • Very basic type stubs for pyarrow and legion_cffi were added under typings at the top level. The legion typings are were generated semi-automated, and are completely generic, but could be refined with a little tedious effort. (Obviously in the ideal case n the future, legion provides its own types)
    • Imported TypeAlias from typing_extensions, which seems to already be in my legate conda env, but should make the dependency explicit to be completely safe (or remove use of TypeAlias)
    • I generally only use Optional for parameters, not e.g. for returns or attributes.

    Hopefully getting the biggest legion.py module passing mypy will clear the way for other modules to be updated more smoothly. But I wanted to pause here for review and discussion of any concerns with the approach, etc. cc @magnatelee

    All cunumeric tests with --use-cuda pass for me. Mypy picked up a few small issues here and there, I will note them in inline comments below (apologies for the one giant file to review, in hindsight I wish I had worked to split up legion.py first, however commits are split up in case you want to review them individually)

    opened by bryevdv 9
  • Improvements to legate.jupyter

    Improvements to legate.jupyter

    This PR fixes the legate.jupyter functionality and also make some improvements:

    • stub typings added so that mypy can be employed

    • driver code is re-used to:

      • ensure consistency with standard legate invocations
      • reduce code (semi-)duplication
      • take advantage of existing extensive unit tests
    • config values for %legate_info are passed in kernel spec metadata, reducing filesystem operations

    • kernel name is included in the kernelspec env, so that:

      • legate.jupyter can create multiple differently named kernel configurations
      • %legate_info can report on the exact running kernel

      image

    Questions

    1. Currently the --legate-dir option is removed. Is this option needed? At present, legate.jupyter always uses the legate install for itself, which seems appropriate, especially now that installs always happen into python environments. This includes pointing the spec argv at a _legion_kernel.py module in the install, instead of actually copying files around.

    2. Currently --json (renamed --config) is a no-op. Is it actually needed? Just running python -m legate.jupyter uses exactly the same defaults as legate. These can be overridden with the standard legate command line options. Do we need to allow configuration of kernels from a file as well? If so I would propose splitting into two sub-commmands, one for using a file and one for using cmdline args, to avoid complicated config merges.

    1. I don't see any reason we couldn't easily support any other standard driver options (e.g. for profiling) easily now. Would it be useful for users to be able to install a debugging or profiling kernel alongside others?

    2. Do we want a real entry point? e.g. users run legate-jupyter instead of python -m legate.jupyter

    3. Currently have to explicitly run %load_ext legate.jupyter first before %legate_info was this avoided somehow before?

    4. %legate_info could report the computed legion_python invocation. Would that be useful for debugging? Or perhaps %legate info and %legate driver ? or similar

    Todo

    Once questions are resolved I intend to implement any appropriate changes in this PR, and then add a unit test suite.

    I would also like to push some improvements down into legion's version of this code, but I do think it will result in much less friction if legate's implementation remains self-contained and decoupled from legion (apart from the ultimate legion_python invocation)

    Notes:

    The changes under legate.driver are to facilitate increased code re-use, e.g the exact command line args can be re-used between different command line programs.

    Updated code never needs to create/install a fake spec, so there is no code to remove specs. If users give an existing name they get actionable error message:

    dev38 ❯ python -m legate.jupyter  --name leg3 -v       
    ERROR: kernel spec 'leg3' already exists. Remove it by running 'jupyter kernelspec uninstall 'leg3', or choose a new kernel name.
    

    Verbose mode has multiple levels of detail. -v gives basic info:

    dev38 ❯ python -m legate.jupyter  --name leg4 -v
    Wrote kernel spec file leg4/kernel.json
    
    Jupyter kernel spec leg4 (leg4) has been installed
    

    But --vv dumps the full spec that is installed

    dev38 ❯ python -m legate.jupyter  --name leg5 -vv
    Wrote kernel spec file leg5/kernel.json
    
    
    {
      "argv": [
        "/home/bryan/work/legate.core/_skbuild/linux-x86_64-3.8/cmake-build/_deps/legion-build/bin/legion_python",
        "-ll:py",
        "1",
        "-lg:local",
        "0",
        "-ll:cpu",
        "4",
        "-ll:util",
        "2",
        "-ll:csize",
        "4000",
        "-level",
        "openmp=5,",
        "-lg:eager_alloc_percentage",
        "50",
        "/home/bryan/work/legate.core/legate/jupyter/_legion_kernel.py",
        "-f",
        "{connection_file}"
      ],
      "display_name": "leg5",
      "env": {
        "GASNET_MPI_THREAD": "MPI_THREAD_MULTIPLE",
        "LD_LIBRARY_PATH": "/home/bryan/work/legate.core/_skbuild/linux-x86_64-3.8/cmake-build/_deps/legion-build/lib:/home/bryan/work/legate.core/build/lib",
        "LEGATE_MAX_DIM": "4",
        "LEGATE_MAX_FIELDS": "256",
        "NCCL_LAUNCH_MODE": "PARALLEL",
        "PYTHONDONTWRITEBYTECODE": "1",
        "PYTHONPATH": "/home/bryan/work/legate.core/_skbuild/linux-x86_64-3.8/cmake-build/_deps/legion-src/bindings/python/build/lib:/home/bryan/work/legate.core/_skbuild/linux-x86_64-3.8/cmake-build/_deps/legion-src/jupyter_notebook:/home/bryan/work/legate.core/legate",
        "REALM_BACKTRACE": "1",
        "__LEGATE_JUPYTER_KERNEL_SPEC__": "leg5"
      },
      "interrupt_mode": "signal",
      "language": "python",
      "metadata": {
        "legate": {
          "argv": [
            "--name",
            "leg5",
            "-vv"
          ],
          "core": {
            "cpus": 4,
            "gpus": 0,
            "ompthreads": 4,
            "openmp": 0,
            "utility": 2
          },
          "memory": {
            "eager_alloc": 50,
            "fbmem": 4000,
            "numamem": 0,
            "regmem": 0,
            "sysmem": 4000,
            "zcmem": 32
          },
          "multi_node": {
            "launcher": "none",
            "launcher_extra": [],
            "nodes": 1,
            "not_control_replicable": false,
            "ranks_per_node": 1
          }
        }
      }
    }
    
    Jupyter kernel spec leg5 (leg5) has been installed
    
    category:bug-fix 
    opened by bryevdv 8
  • Add script wrapper for zero code-change patching

    Add script wrapper for zero code-change patching

    This PR adds a wrapper for legate that can patch specified libraries and then execute a given script with those patches already in place.

    ~/work/legate.core bryanv/zcc
    dev38 ❯ lgpatch.py test.py -patch numpy -cunumeric:report:coverage
    cuNumeric API coverage: 4/4 (100.0%)
    

    See below for test.py example script that only imports numpy.

    Questions

    • Is scripts/lgpatch.py the best place/name for this? Should it be installed (as lgpatch?) in the install bin directory?
    • Is -patch:help suffient for help? Legate intercepts -h and --help before we can get to them so we can't use those.
    • This does not use the trivial cunumeric.patch() because I wanted to have more error checking. But we could move that to patch and simplify here if desired (but then every new lib would be responsible for checking individually in their patch()) If we did this the usage would also switch to -patch cunumeric instead of the current -patch numpy.
    • Alternatively, we could also just remove cunumeric.patch if is not needed after this.

    test.py

    import numpy as np
    
    def cholesky(n, dtype):
        input = np.eye(n, dtype=dtype)
        np.linalg.cholesky(input)
    
    if __name__ == "__main__":
        cholesky(10, np.float32)
    

    -patch:help

    dev38 ❯ ./scripts/lgpatch.py -patch:help                                    
    usage: lgpatch [-patch PATCH [PATCH ...]] PROG
    
    Patch existing libraries with legate equivalents.
    
    Currently the following patching can be applied:
    
        numpy -> cunumeric
    
    positional arguments:
      PROG                  The legate program to run
    
    optional arguments:
      -patch PATCH [PATCH ...]
                            Patch the specified libraries
    
    Any additional command line arguments are passed on to PROG as-is
    
    opened by bryevdv 8
  • Can't install legate.core with CUDA

    Can't install legate.core with CUDA

    I tried to install legate.core with CUDA in workstations with gaming GPUs (RTX 3090 and GTX1080) and did not succeed. I am getting errors like:

    Already on 'control_replication' /home/beka/opt/legate.core/legion/runtime/mathtypes/complex.h(126): error: more than one conversion function from "__half" to a built-in type applies: function "__half::operator float() const" /usr/local/cuda//include/cuda_fp16.hpp(204): here function "__half::operator short() const"

    I suspect these are due to the wrong GPU architecture is given to the compiler. When I indicate --arch pascal, arch=sm_60 appears in the compiler options.

    For my GPUs I need arch=sm_61 or arch=sm_86 How can I get it right?

    opened by BS-astronomer 8
  • spurious error messages on shutdown

    spurious error messages on shutdown

    I have an application that prints lots of messages like this on shutdown:

    Exception ignored in: <function RegionField.__del__ at 0x203236ae5790>
    Traceback (most recent call last):
      File "/autofs/nccs-svm1_home1/rohany/nvidia/legate.core/legate/core/store.py", line 126, in __del__
        self.detach_external_allocation(unordered=True, defer=True)
      File "/autofs/nccs-svm1_home1/rohany/nvidia/legate.core/legate/core/store.py", line 251, in detach_external_allocation
        attachment_manager.detach_external_allocation(
      File "/autofs/nccs-svm1_home1/rohany/nvidia/legate.core/legate/core/runtime.py", line 585, in detach_external_allocation
        self._remove_allocation(alloc)
      File "/autofs/nccs-svm1_home1/rohany/nvidia/legate.core/legate/core/runtime.py", line 570, in _remove_allocation
        self._remove_attachment(alloc)
      File "/autofs/nccs-svm1_home1/rohany/nvidia/legate.core/legate/core/runtime.py", line 565, in _remove_attachment
        raise RuntimeError("Unable to find attachment to remove")
    RuntimeError: Unable to find attachment to remove
    ```.
    opened by rohany 3
  • mapping: extremely long `map_task` calls by the `BaseMapper`

    mapping: extremely long `map_task` calls by the `BaseMapper`

    I have an application where map_task invocations for a particular task are taking on the order of 10's to 100's of MS.

    Profile: http://sapling.stanford.edu/~rohany/lassen-gmg-1-gpu-3000/?start=5208171.801858855&end=5565692.92951843&collapseAll=false&resolution=10.

    This should be reproducible by legate examples/gmg.py -n 3000 -m 100 --fbmem 13000 --zcmem 10000 --sysmem 25000 from within the legate.sparse repo.

    opened by rohany 0
  • Buliding by install.py fails with subprocess-exited-with-error

    Buliding by install.py fails with subprocess-exited-with-error

    It seems the main error is subprocess-exited-with-error. I also found the same issue. https://github.com/nv-legate/legate.core/issues/107#issue-1054642948 But in my case, I'm using python 3.9, so it should not be the problem about version of python. I also tried python 3.10, and the same error occurs.

    full log of output
    Verbose build is  off
    Using python lib and version: /home/DENSEQCD/luoxiao/.miniconda3/envs/legate/lib/libpython3.9.so, 3.9.13
    Performing a clean build to accommodate build isolation.
    Processing /home/DENSEQCD/luoxiao/downloads/legate.core
      Installing build dependencies ... error
      error: subprocess-exited-with-error
      
      × pip subprocess to install build dependencies did not run successfully.
      │ exit code: 1
      ╰─> [7 lines of output]
          WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x2ae99cfa7a90>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/wheel/
          WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x2ae99cfa7dc0>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/wheel/
          WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x2ae99cfa7e80>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/wheel/
          WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x2ae99cfcd3d0>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/wheel/
          WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x2ae99cfcd580>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/wheel/
          ERROR: Could not find a version that satisfies the requirement wheel (from versions: none)
          ERROR: No matching distribution found for wheel
          [end of output]
      
      note: This error originates from a subprocess, and is likely not a problem with pip.
    error: subprocess-exited-with-error
    
    × pip subprocess to install build dependencies did not run successfully.
    │ exit code: 1
    ╰─> See above for output.
    
    note: This error originates from a subprocess, and is likely not a problem with pip.
    Traceback (most recent call last):
      File "/home/DENSEQCD/luoxiao/downloads/legate.core/./install.py", line 751, in <module>
        driver()
      File "/home/DENSEQCD/luoxiao/downloads/legate.core/./install.py", line 747, in driver
        install(unknown=unknown, **vars(args))
      File "/home/DENSEQCD/luoxiao/downloads/legate.core/./install.py", line 474, in install
        execute_command(pip_install_cmd, verbose, cwd=legate_core_dir, env=cmd_env)
      File "/home/DENSEQCD/luoxiao/downloads/legate.core/./install.py", line 74, in execute_command
        subprocess.check_call(args, **kwargs)
      File "/home/DENSEQCD/luoxiao/.miniconda3/envs/legate/lib/python3.9/subprocess.py", line 373, in check_call
        raise CalledProcessError(retcode, cmd)
    subprocess.CalledProcessError: Command '['/home/DENSEQCD/luoxiao/.miniconda3/envs/legate/bin/python3', '-m', 'pip', 'install', '--root', '/', '--prefix', '/home/DENSEQCD/luoxiao/.miniconda3/envs/legate', '--upgrade', '.']' returned non-zero exit status 1.
    
    opened by LUOXIAO92 9
Releases(v22.10.00)
  • v22.10.00(Oct 13, 2022)

    Release 22.10 contains several improvements to memory management. Those changes are to recycle memory space from GC'ed Legate stores more eagerly for fresh ones. Another big change in this release is a new build infrastructure based on CMake and scikit-build for the Legate ecosystem, which is a big leap over the previous ad-hoc build system. The release also includes two useful debugging features: 1) provenance tracking for tasks and other operator kinds issued by client libraries and 2) detailed logging for client library mappers.

    Conda packages for this release are available at https://anaconda.org/legate/legate-core.

    What's Changed

    🐛 Bug Fixes

    • Fix target_memory setting for task futures by @manopapad in https://github.com/nv-legate/legate.core/pull/327
    • avoiding division by 0 in slicing by @ipdemes in https://github.com/nv-legate/legate.core/pull/319
    • Use the correct key when adding types to a library TypeSystem by @manopapad in https://github.com/nv-legate/legate.core/pull/333
    • fix partitioning of empty regions by @ipdemes in https://github.com/nv-legate/legate.core/pull/337
    • Correctly inline map 0d stores by @magnatelee in https://github.com/nv-legate/legate.core/pull/340
    • Fix error and warning message when --launcher is missing by @manopapad in https://github.com/nv-legate/legate.core/pull/341
    • make sure communicators are destroyed one by one by @eddy16112 in https://github.com/nv-legate/legate.core/pull/345
    • Specify an upper bound for return value sizes for core::extract_scalar by @magnatelee in https://github.com/nv-legate/legate.core/pull/346
    • Set an upper bound for allocations for the serdez type used in the core by @magnatelee in https://github.com/nv-legate/legate.core/pull/348
    • fix cpu communicator for omp by @eddy16112 in https://github.com/nv-legate/legate.core/pull/352
    • Use Legion primitives to coordinate accesses to the shared instance manager by @magnatelee in https://github.com/nv-legate/legate.core/pull/355
    • Synchronize instance manager accesses from mapper calls other than map_task by @magnatelee in https://github.com/nv-legate/legate.core/pull/358
    • Follow-on changes to #353 by @magnatelee in https://github.com/nv-legate/legate.core/pull/360
    • Scalar store fix by @magnatelee in https://github.com/nv-legate/legate.core/pull/365
    • InstanceManager segfault fixes by @manopapad in https://github.com/nv-legate/legate.core/pull/368
    • Fix typos in solver.py by @magnatelee in https://github.com/nv-legate/legate.core/pull/366
    • legate_core_cpp.cmake: add missing barrier header file in export by @rohany in https://github.com/nv-legate/legate.core/pull/389
    • Use Python GC to release Legion handles from destroyed RegionManagers by @magnatelee in https://github.com/nv-legate/legate.core/pull/391
    • legate/driver: fix driver legion_module path by @rohany in https://github.com/nv-legate/legate.core/pull/394
    • Legion bug WAR: don't instantiate futures on framebuffer by @manopapad in https://github.com/nv-legate/legate.core/pull/409
    • Revive dead region managers on field allocations by @magnatelee in https://github.com/nv-legate/legate.core/pull/418

    🚀 New Features

    • Support for mapper logging by @magnatelee in https://github.com/nv-legate/legate.core/pull/356
    • Provenance tracking by @magnatelee in https://github.com/nv-legate/legate.core/pull/370
    • Add Fill operation by @manopapad in https://github.com/nv-legate/legate.core/pull/369
    • add jupyter config for legate by @eddy16112 in https://github.com/nv-legate/legate.core/pull/309

    🛠️ Improvements

    • Make stores have an explicit bottom in their transform stacks by @magnatelee in https://github.com/nv-legate/legate.core/pull/320
    • Update conda env files to match cunumeric by @manopapad in https://github.com/nv-legate/legate.core/pull/324
    • An internal method to force initialize communicators by @magnatelee in https://github.com/nv-legate/legate.core/pull/328
    • Use empty buffers to create empty output stores by @magnatelee in https://github.com/nv-legate/legate.core/pull/330
    • Two improvements to error handling by @magnatelee in https://github.com/nv-legate/legate.core/pull/336
    • Skip conduit check when binding by @manopapad in https://github.com/nv-legate/legate.core/pull/342
    • Make numactl optional by @manopapad in https://github.com/nv-legate/legate.core/pull/343
    • Remove deprecated option --no-tensor by @manopapad in https://github.com/nv-legate/legate.core/pull/344
    • Silence shard registration warnings by @manopapad in https://github.com/nv-legate/legate.core/pull/347
    • Instance manager improvements by @magnatelee in https://github.com/nv-legate/legate.core/pull/350
    • A custom task wrapper for efficient handling of return values by @magnatelee in https://github.com/nv-legate/legate.core/pull/353
    • Refactoring to make the runtime object singleton by @magnatelee in https://github.com/nv-legate/legate.core/pull/363
    • Turn off the precise stacktrace capturing by default by @magnatelee in https://github.com/nv-legate/legate.core/pull/362
    • Add CMake build for C++ and scikit-build infrastructure for Python package installation by @jjwilke in https://github.com/nv-legate/legate.core/pull/323
    • Support building with GASNet-Ex and MPI backends by @manopapad in https://github.com/nv-legate/legate.core/pull/384
    • Better store management by @magnatelee in https://github.com/nv-legate/legate.core/pull/364
    • Modularize the legate driver by @bryevdv in https://github.com/nv-legate/legate.core/pull/371
    • Add a pool of region managers with LRU eviction by @magnatelee in https://github.com/nv-legate/legate.core/pull/392
    • Adjust consensus match frequency based on field sizes by @magnatelee in https://github.com/nv-legate/legate.core/pull/402
    • On mapping failure retry after tightening non-RO reqs by @manopapad in https://github.com/nv-legate/legate.core/pull/424

    📖 Documentation

    • Update docs to match cunumeric by @bryevdv in https://github.com/nv-legate/legate.core/pull/316

    New Contributors

    • @ipdemes made their first contribution in https://github.com/nv-legate/legate.core/pull/319

    Full Changelog: https://github.com/nv-legate/legate.core/compare/v22.08.00...v22.10.00

    Source code(tar.gz)
    Source code(zip)
  • v22.08.00(Aug 9, 2022)

    Release 22.08.00 includes two major features: exception support and a communicator library for CPU and OpenMP tasks. The exception support captures any exception raised by a Legate task and re-raises it as a Python exception. The communicator library allows CPU and OpenMP tasks to perform explicit communication.

    Conda packages for this release are available at https://anaconda.org/legate/legate-core.

    New Features

    • API to pop off the transformation stack by @magnatelee in #299
    • API to query the key partition of store by @magnatelee in #296
    • Exception support by @magnatelee in #258
    • Support for scaling on partition variables by @magnatelee in #270
    • Implement collective communicator for multi-node CPUs by @eddy16112 in #214

    Improvements

    • Add type annotations to helper functions for indirect copies by @magnatelee in #245
    • Split create_task by @bryevdv in #269
    • Minor env and mypy updates by @bryevdv in #273
    • let gasnet to handle mpi init and finalize by @eddy16112 in #275
    • Get rid of the LEGATE_MAX_CPU_COMMS by @eddy16112 in #280
    • src/core/comm: add simple implementation of pthread_barrier for mac by @rohany in #284
    • Add some types needed for work on cunumeric by @bryevdv in #287
    • Make the binding facility work on a single node by @magnatelee in #295
    • Eliminate null shift transforms by @magnatelee in #300
    • Pass the shape by default when creating an accessor by @magnatelee in #301
    • Store refactoring by @magnatelee in #302
    • Use bounding box to convert the store domain to a rect by @magnatelee in #303
    • Broaden add_scalar_arg type by @bryevdv in #306
    • Solver fix to correctly handle unaligned color shapes when no unbound stores exist by @magnatelee in #310
    • Check if the task returned a buffer to every unbound store by @magnatelee in #312
    • Note minimum CTK requirement on runtime requirements by @manopapad in #250
    • Remove some unused code in the driver by @manopapad in #254
    • Only overwrite installed legion_c_util.h if less recent by @manopapad in #256
    • Conda build doesn't need to mess with CUDA stubs by @manopapad in #253

    Bug Fixes

    • Use phase barriers to work around the CUDA driver bug by @magnatelee in #246
    • Add missing includes to aid intellisense providers by @trxcllnt in #249
    • Minor type fix by @bryevdv in #252
    • Update the launcher to initialize MPI with MPI_THREAD_MULTIPLE by @magnatelee in #264
    • Set the GASNet variable for MPI init unconditionally by @magnatelee in #265
    • Assign correct dimensions to unbound stores by @magnatelee in #267
    • Set cuda virtual package as hard run requirement for gpu conda package by @m3vaz in #266
    • Fixes for building with setup.py outside conda, primarily Mac (#260) by @jjwilke in #263
    • use mpicc to compile the code when gasnet is enabled by @eddy16112 in #274
    • fix the coll for non-mpi case. by @eddy16112 in #291
    • StoreTransform needs a virtual destructor by @magnatelee in #304
    • Correctly handle 0-D stores backed by region fields by @magnatelee in #315
    • src: switch to abolute include path for pthread_barrier.h by @rohany in #311
    • Temporarily drop support for UDP conduit by @manopapad in #305

    New Contributors

    • @trxcllnt made their first contribution in https://github.com/nv-legate/legate.core/pull/249
    • @eddy16112 made their first contribution in https://github.com/nv-legate/legate.core/pull/214
    • @jjwilke made their first contribution in https://github.com/nv-legate/legate.core/pull/263
    • @rohany made their first contribution in https://github.com/nv-legate/legate.core/pull/284

    Full Changelog: https://github.com/nv-legate/legate.core/compare/v22.05.03...v22.08.00

    Source code(tar.gz)
    Source code(zip)
  • v22.05.03(Jul 13, 2022)

    This is a hotfix release updating the Legion version.

    What's Changed

    • Update legion commit to the latest by @marcinz in https://github.com/nv-legate/legate.core/pull/297

    Full Changelog: https://github.com/nv-legate/legate.core/compare/v22.05.02...v22.05.03

    Source code(tar.gz)
    Source code(zip)
  • v22.05.02(Jun 21, 2022)

    This hotfix release fixes issues in conda recipes.

    What's Changed

    • Cherry pick: Freeze Conda Compiler Versions (#261) by @marcinz in https://github.com/nv-legate/legate.core/pull/276
    • Cherry pick: Set cuda virtual package as hard run requirement for conda gpu package ( #266) by @marcinz in https://github.com/nv-legate/legate.core/pull/277
    • Fix typo in conda run requirements by @m3vaz in https://github.com/nv-legate/legate.core/pull/281

    Full Changelog: https://github.com/nv-legate/legate.core/compare/v22.05.01...v22.05.02

    Source code(tar.gz)
    Source code(zip)
  • v22.05.01(Jun 16, 2022)

    This hotfix release updates the conda build recipe to use the right compiler versions.

    Full Changelog: https://github.com/nv-legate/legate.core/compare/v22.05.00...v22.05.01

    Source code(tar.gz)
    Source code(zip)
  • v22.05.00(Jun 7, 2022)

    Release 22.05 introduces the following new features: 1) multi-dimensional unbound Legate stores, 2) communicator support for Legate tasks with multi-dimensional launch domains, 3) zero-code patching support (lgpatch), 4) unified per-library command-line arguments handling, and 5) NVTX ranges for Legate tasks.

    Conda packages for this release are available at https://anaconda.org/legate/legate-core.

    New Features

    • Simple stream pool for all Legate libraries by @magnatelee in #208
    • NVTX integration by @magnatelee in #209
    • Adding conda build recipe files by @marcinz in #188
    • APIs to declare out-of-range indirects by @magnatelee in #221
    • Communicators for N-D task launches by @magnatelee in #182

    Improvements

    • Add install.py flag to feed -march to legion and legate by @manopapad in #170
    • Add support for checking out a commit by @marcinz in #172
    • Added allocator rebase pr by @mfoerste4 in #171
    • Split up legate/core/legion.py by @bryevdv in #163
    • Add script wrapper for zero code-change patching by @bryevdv in #236
    • Forward some more relevant env vars through mpirun -x by @manopapad in #180
    • Add more type annotations by @bryevdv in #181
    • Updates for N-D unbound stores by @magnatelee in #186
    • Variety of quality-of-life changes by @magnatelee in #190
    • Auto-detect current GPU architecture at install time by @manopapad in #184
    • More type annotations by @bryevdv in #189
    • Pass -std=c++14 to Legion explicitly by @manopapad in #194
    • Bryanv/arg protocol by @bryevdv in #195
    • More typing by @bryevdv in #196
    • Support for string scalar arguments by @magnatelee in #199
    • Flush scheduling window before tree reduction is performed by @magnatelee in #203
    • Bump up NumPy version by @magnatelee in #206
    • Start requiring C++17 for Legate libraries by @magnatelee in #207
    • Raise an exception if there's no usable processor by @magnatelee in #210
    • Use ordered sets in requirement analyzers for determinism by @magnatelee in #212
    • Type annotations for the rest of the core modules by @magnatelee in #213
    • Split ManualTask from other tasks by @bryevdv in #197
    • Use updated legion python cleanup API by @manopapad in #217
    • Create output directory if it doesn't exist by @manopapad in #218
    • Update the Python cleanup interface by @magnatelee in #222
    • Bump minpy to 3.8 by @bryevdv in #220
    • Update build.sh with specific CPU targeting by @m3vaz in #223
    • Use stdlib typing where possible by @bryevdv in #224
    • Compile everything using C++17 by @manopapad in #227
    • Centralize command line argument parsing in legate.core by @bryevdv in #232
    • Launcher flags to configure Nsight Systems by @magnatelee in #234
    • Conda recipes improvements by @marcinz in #233
    • Use 32-bit integers for type codes by @magnatelee in #239
    • Misc small updates by @bryevdv in #238
    • Add option to keep profiler/spy dumps by @manopapad in #240
    • Use read-write privilege for scatter targets by @magnatelee in #241
    • Set target memories for future-backed stores by @magnatelee in #242

    Bug Fixes

    • Fix for output region by @magnatelee in #175
    • Unbound store fix by @magnatelee in #179
    • Handle projections to 0D correctly by @magnatelee in #202
    • Fix for the constraint solver by @magnatelee in #204
    • Fix compile issue with cuda_help.h by @magnatelee in #211
    • Pull conda build number into a variable by @marcinz in #216
    • Fix unit tests for Legate stores by @magnatelee in #235
    • Fix unordered field destruction by @magnatelee in #237
    • Update the number of elements correctly in create_output_buffer by @magnatelee in #243
    • Use phase barriers to work around the CUDA driver bug (#246) by @marcinz in #247

    New Contributors

    • @mfoerste4 made their first contribution in https://github.com/nv-legate/legate.core/pull/171
    • @m3vaz made their first contribution in https://github.com/nv-legate/legate.core/pull/223

    Full Changelog: https://github.com/nv-legate/legate.core/compare/v22.03.00...v22.05.00

    Source code(tar.gz)
    Source code(zip)
  • v22.03.00(Apr 5, 2022)

    Release 22.03 adds communicator support for tasks, allowing intra-task communication to be performed in a safe way. The new task type for constructing task trees is useful for operations implementing set semantics, such as distributed value de-duplication. The build system is also extended such that CUDA tasks can use offline device linking. Finally, the auto-partitioner is revised and extended to handle distributed gather/scatter copies correctly.

    Conda packages for this release are available at https://anaconda.org/legate/legate-core.

    New Features

    • Support for unaligned color spaces by @magnatelee in #157
    • Support for uneven partitions by @magnatelee in #152
    • Support for offline device linking by @magnatelee in #145
    • Communicator support by @magnatelee in #140
    • APIs to query launch shape by @magnatelee in #139
    • Support for partial broadcasting by @magnatelee in #137

    Improvements

    • Add support for checking out a commit by @marcinz in #169
    • Turn off local function optimization for NCCL initialization by @magnatelee in #167
    • Allow users to explicitly redirect logs to file by @manopapad in #166
    • Cleanup / Initial pass for type annotations by @bryevdv in #158
    • Safety checks for copies by @magnatelee in #159
    • Support for reduction tree construction by @magnatelee in #151
    • Be a little more lenient in fake-dims check by @manopapad in #148
    • Update Legate copy by @magnatelee in #149
    • Build update by @magnatelee in #142

    Bug Fixes

    • Quickfix: Fix non-annotations on 3.7 by @bryevdv in #164
    • Solver fix by @magnatelee in #162
    • Mapper fix for tasks with no store arguments by @magnatelee in #161
    • Solver fix by @magnatelee in #160
    • Fix for #133 by @magnatelee in #155
    • Include ccbin when linking device code by @manopapad in #153
    • Some fixes for auto-partitioner by @magnatelee in #147

    New Contributors

    • @bryevdv made their first contribution in https://github.com/nv-legate/legate.core/pull/158

    Full Changelog: https://github.com/nv-legate/legate.core/compare/v22.01.01...v22.03.00

    Source code(tar.gz)
    Source code(zip)
  • v22.01.01(Feb 16, 2022)

    What's Changed

    This is a minor bug fix release, fixing the version of legion to a particular commit.

    Conda packages for this release are available at https://anaconda.org/legate/legate-core.

    Full Changelog: https://github.com/nv-legate/legate.core/compare/v22.01.00...v22.01.01

    Source code(tar.gz)
    Source code(zip)
  • v22.01.00(Feb 10, 2022)

    Release 22.01 introduces APIs for manual data partitioning and task parallelization to express distributed algorithms that are not data parallel, such as matrix decompositions.

    New Features

    • Support ingest of external distributed data by @manopapad in #81
    • Support for manual partitioning by @magnatelee in #120
    • Cyclic distribution by @magnatelee in #122
    • Make LEGATE_ABORT a statement by @magnatelee in #126
    • API for issuing fences by @magnatelee in #128

    Improvements

    • Support multi-rank mode by @manopapad in #80
    • Realm now sets GASNET_PHYSMEM_PROBE/MAX internally by @manopapad in #86
    • Replace math.prod with itertools.product by @magnatelee in #88
    • Print the command once even if externally launched by @manopapad in #87
    • Build system improvements by @manopapad in #101
    • Verify the python lib used by Realm matches what user is running by @manopapad in #110
    • Add has_fake_dims() check on Stores by @manopapad in #112
    • Remove CUDA resource management by @magnatelee in #111
    • Debug-printing functions for dense arrays by @manopapad in #115
    • No longer need to reserve one dim for reductions by @manopapad in #121
    • Flag to use empty tasks by @magnatelee in #129
    • Do python install on custom dir w/o eggs by @manopapad in #131

    Bug Fixes

    • Object cycle fix ported to branch-22.01 by @magnatelee in #85
    • Fix product computation by @manopapad in #89
    • Bug fix for base mapper (rebased) by @magnatelee in #93
    • Fix for Issue 77 (rebased) by @magnatelee in #92
    • Keep RegionField alive while any inline mapping remains by @manopapad in #94
    • Old versions of NVCC only supported -M, not -MMD, -MF, -MP by @manopapad in #95
    • Port legate.core#97 by @manopapad in #98
    • Fix for deferred detachments (back-ported) by @magnatelee in #100
    • Override keywords to appease Clang by @magnatelee in #104
    • Remove an overzealous assertion by @magnatelee in #108
    • Fix for set_storage by @magnatelee in #109
    • Fix dependency generation for .cu files by @manopapad in #114
    • Remove jsrun option not supported on lassen by @manopapad in #116
    • Minor bugfix in debugging code by @manopapad in #117
    • Pin setuptools version, to work around breaking change by @manopapad in #124
    • bug fix in install.py by @sunwayihep in #105
    • Don't apply python installation fix under setuptools<60 by @manopapad in #132

    Documentation

    • Added contributing policy by @marcinz in #119

    New Contributors

    • @sunwayihep made their first contribution in https://github.com/nv-legate/legate.core/pull/105

    Full Changelog: https://github.com/nv-legate/legate.core/compare/v21.11.00...v22.01.00

    Source code(tar.gz)
    Source code(zip)
  • v21.11.00(Nov 9, 2021)

    This is the initial public alpha release of Legate Core library. This release supports the initial release of cuNumeric.

    Conda packages for this release are available at https://anaconda.org/legate/legate-core.

    What's Changed

    • CI Infrastructure by @marcinz in https://github.com/nv-legate/legate.core/pull/18
    • Added ENVVARs for some launcher options. by @jefflarkin in https://github.com/nv-legate/legate.core/pull/20
    • Support initializing FieldSpace from existing handle by @manopapad in https://github.com/nv-legate/legate.core/pull/13
    • Development of CI by @marcinz in https://github.com/nv-legate/legate.core/pull/19
    • Add testing to CI by @marcinz in https://github.com/nv-legate/legate.core/pull/22
    • Update the core CI to match the other repositories by @marcinz in https://github.com/nv-legate/legate.core/pull/28
    • Force left alignment for pointers and references by @magnatelee in https://github.com/nv-legate/legate.core/pull/30
    • Invoke srun in pseudo-terminal mode when using (cuda-)gdb by @manopapad in https://github.com/nv-legate/legate.core/pull/32
    • A utility for deferred buffers by @magnatelee in https://github.com/nv-legate/legate.core/pull/34
    • Improve the auto-partitioner, part 1 by @magnatelee in https://github.com/nv-legate/legate.core/pull/33
    • Clean up install script by @manopapad in https://github.com/nv-legate/legate.core/pull/35
    • Missing virtual from destructor by @manopapad in https://github.com/nv-legate/legate.core/pull/36
    • Fix for delinearization's inversion function by @magnatelee in https://github.com/nv-legate/legate.core/pull/37
    • Fix overzealous checks on dimensions by @magnatelee in https://github.com/nv-legate/legate.core/pull/38
    • Suppress some -Wswitch warnings by @manopapad in https://github.com/nv-legate/legate.core/pull/39
    • When generating CI images, name them after the branch by @marcinz in https://github.com/nv-legate/legate.core/pull/31
    • Remove some operators that were defined in Legion by @manopapad in https://github.com/nv-legate/legate.core/pull/41
    • Explicit checks for underlying Legion handle equality by @manopapad in https://github.com/nv-legate/legate.core/pull/42
    • Cache index partitions created for index spaces by @magnatelee in https://github.com/nv-legate/legate.core/pull/43
    • Formalize "scalar" property on Stores by @manopapad in https://github.com/nv-legate/legate.core/pull/46
    • Clarify documentation on --python-lib install option by @manopapad in https://github.com/nv-legate/legate.core/pull/49
    • Process output in a separate step that is always executed by @marcinz in https://github.com/nv-legate/legate.core/pull/50
    • adding docs for git, versions, and changelog by @mmccarty in https://github.com/nv-legate/legate.core/pull/44
    • Make sure that images for pull request get named properly by @marcinz in https://github.com/nv-legate/legate.core/pull/51
    • Revert installation defaults to CUDA & OpenMP off by @manopapad in https://github.com/nv-legate/legate.core/pull/52
    • Switch the default Legion branch to control_replication by @magnatelee in https://github.com/nv-legate/legate.core/pull/55
    • Refactor the task wrapper for scalar outputs by @magnatelee in https://github.com/nv-legate/legate.core/pull/54
    • Stop jobs on forks by @marcinz in https://github.com/nv-legate/legate.core/pull/56
    • Use data structures with deterministic iteration order in the solver by @manopapad in https://github.com/nv-legate/legate.core/pull/58
    • Mapping API (part 1) by @magnatelee in https://github.com/nv-legate/legate.core/pull/47
    • Interface for registering and retrieving reduction operators by @magnatelee in https://github.com/nv-legate/legate.core/pull/59
    • Fixes for transformations by @magnatelee in https://github.com/nv-legate/legate.core/pull/60
    • Dead code elimination by @magnatelee in https://github.com/nv-legate/legate.core/pull/61
    • Reorganizing source files by @magnatelee in https://github.com/nv-legate/legate.core/pull/62
    • Fix cases where transpose treats axes in the wrong direction by @manopapad in https://github.com/nv-legate/legate.core/pull/64
    • Functions for debug-printing Transforms on the C++ side by @manopapad in https://github.com/nv-legate/legate.core/pull/65
    • Improve support for incremental builds by @manopapad in https://github.com/nv-legate/legate.core/pull/63
    • Update for changes to FutureMap functions in Legion C interface by @manopapad in https://github.com/nv-legate/legate.core/pull/66
    • Properly count subregion reference destruction towards root region unmapping by @manopapad in https://github.com/nv-legate/legate.core/pull/67
    • Support attaching onto existing Stores + related cleanups by @manopapad in https://github.com/nv-legate/legate.core/pull/68
    • Rearrange user flags by @magnatelee in https://github.com/nv-legate/legate.core/pull/69
    • Mapper fix by @magnatelee in https://github.com/nv-legate/legate.core/pull/70
    • Remove --no-interpreter by @magnatelee in https://github.com/nv-legate/legate.core/pull/71
    • Minor fix for launcher by @magnatelee in https://github.com/nv-legate/legate.core/pull/72
    • Allow scalars to own buffers by @magnatelee in https://github.com/nv-legate/legate.core/pull/73
    • Support for affine images and store colocation by @magnatelee in https://github.com/nv-legate/legate.core/pull/74
    • Lazy evaluation support by @magnatelee in https://github.com/nv-legate/legate.core/pull/75
    • Minor fix for scalar by @magnatelee in https://github.com/nv-legate/legate.core/pull/76
    • Minor fix for the cost function by @magnatelee in https://github.com/nv-legate/legate.core/pull/79
    • Field reuse tunables by @magnatelee in https://github.com/nv-legate/legate.core/pull/82
    • Object cycle fix by @magnatelee in https://github.com/nv-legate/legate.core/pull/84
    • Fix for Issue 77 by @lightsighter in https://github.com/nv-legate/legate.core/pull/78
    • Bug fix for base mapper by @magnatelee in https://github.com/nv-legate/legate.core/pull/91
    • Fix import of legion CFFI by @manopapad in https://github.com/nv-legate/legate.core/pull/97
    • Fix for deferred detachments by @magnatelee in https://github.com/nv-legate/legate.core/pull/99
    • Release 21.11.00 by @marcinz in https://github.com/nv-legate/legate.core/pull/102

    New Contributors

    • @marcinz made their first contribution in https://github.com/nv-legate/legate.core/pull/18
    • @jefflarkin made their first contribution in https://github.com/nv-legate/legate.core/pull/20
    • @mmccarty made their first contribution in https://github.com/nv-legate/legate.core/pull/44
    • @lightsighter made their first contribution in https://github.com/nv-legate/legate.core/pull/78

    Full Changelog: https://github.com/nv-legate/legate.core/commits/v21.11.00

    Source code(tar.gz)
    Source code(zip)
Owner
Legate
High Productivity High Performance Computing
Legate
Python script to like all posts from a selected Anilist profile

AniLiker: A python autoliker What's the point of this project? This project was a way to learn GraphQL, and also create a project that I've been inter

Iván Pérez 9 Nov 25, 2022
An open letter in support of Richard Matthew Stallman being reinstated by the Free Software Foundation

An open letter in support of RMS. To sign, click here and name the file <username>.yaml (replace <username> with your name) with the following content

null 2.4k Jan 7, 2023
Earth Vision Foundation

EVer - A Library for Earth Vision Researcher EVer is a Pytorch-based Python library to simplify the training and inference of the deep learning model.

Zhuo Zheng 34 Nov 26, 2022
OWASP Foundation Web Respository

WWWGrep OWASP Foundation Web Respository Author: Mark Deen & Aditi Mohan Introduction WWWGrep is a rapid search “grepping” mechanism that examines HTM

OWASP 34 Jun 15, 2022
A Django starter template with a sound foundation.

SOS Django Template SOS Django Tempalate is a Django starter template that has opinionated and good solutions while starting your new Django project.

Eray Erdin 19 Oct 30, 2022
OpenMMLab Computer Vision Foundation

English | 简体中文 Introduction MMCV is a foundational library for computer vision research and supports many research projects as below: MMCV: OpenMMLab

OpenMMLab 4.6k Jan 9, 2023
Foundation Auth Proxy is an abstraction on Foundations' authentication layer and is used to authenticate requests to Atlas's REST API.

foundations-auth-proxy Setup By default the server runs on http://0.0.0.0:5558. This can be changed via the arguments. Arguments: '-H' or '--host': ho

Dessa - Open Source 2 Jul 3, 2020
Simple Python script to scrape youtube channles of "Parity Technologies and Web3 Foundation" and translate them to well-known braille language or any language

Simple Python script to scrape youtube channles of "Parity Technologies and Web3 Foundation" and translate them to well-known braille language or any

Little Endian 1 Apr 28, 2022
Implementation of CoCa, Contrastive Captioners are Image-Text Foundation Models, in Pytorch

CoCa - Pytorch Implementation of CoCa, Contrastive Captioners are Image-Text Foundation Models, in Pytorch. They were able to elegantly fit in contras

Phil Wang 565 Dec 30, 2022
This is the remake of the program PYOBD. It works on Python3 and all new libraries. It was tested on Linux, Windows, and it should work on MAC too.

This is the remake of the program PYOBD. It works on Python3 and all new libraries. It was tested on Linux, Windows, and it should work on MAC too. You just need an ELM327 USB or bluetooth device and a PC(laptop preferably).

null 127 Jan 6, 2023
🌈Python cheatsheet for all standard libraries(Continuously Updated)

Python Standard Libraries Cheatsheet Depend on Python v3.9.8 All code snippets have been tested to ensure they work properly. Fork me on GitHub. 中文 En

nick 12 Dec 27, 2022
That is a example of a Book app on Python, made with support of all JS libraries on React framework

React+Python Books App You can use this repository whenever you want Used for a video Create the database: python -m dbutils Start the web server: pyt

Koma Human 1 Apr 20, 2022
Develop open-source Python Arabic NLP libraries that the Arab world will easily use in all Natural Language Processing applications

Develop open-source Python Arabic NLP libraries that the Arab world will easily use in all Natural Language Processing applications

BADER ALABDAN 2 Oct 22, 2022
Donatus Prince 6 Feb 25, 2022
A curated list of awesome Python asyncio frameworks, libraries, software and resources

Awesome asyncio A carefully curated list of awesome Python asyncio frameworks, libraries, software and resources. The Python asyncio module introduced

Timo Furrer 3.8k Jan 8, 2023
A curated list of awesome Jupyter projects, libraries and resources

Awesome Jupyter A curated list of awesome Jupyter projects, libraries and resources. Jupyter is an open-source web application that allows you to crea

Markus Schanta 3.1k Dec 30, 2022
Fast image augmentation library and easy to use wrapper around other libraries. Documentation: https://albumentations.ai/docs/ Paper about library: https://www.mdpi.com/2078-2489/11/2/125

Albumentations Albumentations is a Python library for image augmentation. Image augmentation is used in deep learning and computer vision tasks to inc

null 11.4k Jan 9, 2023
A library of extension and helper modules for Python's data analysis and machine learning libraries.

Mlxtend (machine learning extensions) is a Python library of useful tools for the day-to-day data science tasks. Sebastian Raschka 2014-2020 Links Doc

Sebastian Raschka 4.2k Jan 2, 2023
🏆 A ranked list of awesome python developer tools and libraries. Updated weekly.

Best-of Python Developer Tools ?? A ranked list of awesome python developer tools and libraries. Updated weekly. This curated list contains 250 awesom

Machine Learning Tooling 646 Jan 7, 2023
🏆 A ranked list of awesome python libraries for web development. Updated weekly.

Best-of Web Development with Python ?? A ranked list of awesome python libraries for web development. Updated weekly. This curated list contains 540 a

Machine Learning Tooling 1.8k Jan 4, 2023