nptsne is a numpy compatible python binary package that offers a number of APIs for fast tSNE calculation.

Overview

Build Status

Build status

Documentation Status

nptsne

nptsne is a numpy compatible python binary package that offers a number of APIs for fast tSNE calculation and HSNE modelling.

For more detail see the documentation for the current release - 1.1.0

Currently python 3.6, 3.7, and 3.8 are supported on Windows, Mac and Linux using cibuildwheel

Demo software using nptsne

Can be downloaded from DOI

Building

The requirements.txt and the pyproject.toml contain the list of build requirements.

  • Add the biovault conan remote (for prebuilt packages):
conan remote add conan-biovault http://cytosplore.lumc.nl:8081/artifactory/api/conan/conan-local

Development build & install using python

pip install -v -e .

This will automatically create a build subdirectory build the bindings and create an .egg-link file in the current python environment.

On Windows a _nptsne.sln file will be present under the build directory

Alternative manual Windows build

  • Make a build directory below the HDILib project root. For example: ./_build_release or ./_build_debug (when using conan the source directories are shared but separate build directories should be used for release and debug.)
  • In the python environment (with conan and cmake accessible) cd to the build directory and issue the following (for VisualStudio 2017):
cmake .. -G "Visual Studio 15 2017 Win64" -DCMAKE_BUILD_TYPE=Release -DNPTSNE_BUILD_WITH_CONAN=ON
(*Note: this assumes that the build dir is one level down from the project root.
The default of NPTSNE_BUILD_WITH_CONAN is OFF*)
  • If all goes well Conan will have installed the dependencies in its cache and created the required defines for the Cmake configuration. Open the .sln in VisualStudio and build ALL_BUILD for Release or Debug matching the CMAKE_BUILD_TYPE. On Windows the result of the build are three *.lib files
Comments
  • Which type of initialization does HSNE use?

    Which type of initialization does HSNE use?

    Hi, I was looking in the code to check how does HSNE initialize embeddings for dimensionality reduction.

    I understand that t-SNE is used at the end of the day but which type of initialization does it use? PCA, random?

    Checking the outputs and code I believe it is not random. Am I right?

    opened by wilsonjr 2
  • How to use the HSNE interpolation and project scale?

    How to use the HSNE interpolation and project scale?

    Hi, I was reading the HSNE article.

    1. How can I use the library to generate an interpolation based on a hierarchy level?
    2. How can I project an HSNE scale to plot, for example, using pyplot?

    Thank you!

    opened by wilsonjr 2
  • Can't import because of GLIBC requirements

    Can't import because of GLIBC requirements

    Hi nptsne team,

    I'm having some trouble getting nptsne to import (see below for error).

    This seems to be due to a requirement for a high version of GLIBC (2.29), which isn't directly available through apt-get for my system (the max version available currently is 2.24).

    If there's any way I might be able to work around this, that would be great, as manually updating GLIBC is difficult in my computing environment.

    Thanks! -Lucas

    lucasg@graybuck-ide:/home/lucas.graybuck/nptsne_leiden_testing$ miniconda/envs/r-reticulate-9530452876/bin/python
    Python 3.7.3 (default, Mar 27 2019, 22:11:17) 
    [GCC 7.3.0] :: Anaconda, Inc. on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import nptsne
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/home/lucas.graybuck/nptsne_leiden_testing/miniconda/envs/r-reticulate-9530452876/lib/python3.7/site-packages/nptsne/__init__.py", line 1, in <module>
        from .nptsne import *
    ImportError: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by /home/lucas.graybuck/nptsne_leiden_testing/miniconda/envs/r-reticulate-9530452876/lib/python3.7/site-packages/nptsne/./libpng16.so.16)
    >>> quit()
    lucasg@graybuck-ide:/home/lucas.graybuck/nptsne_leiden_testing$ sudo apt-cache policy libc6
    libc6:
      Installed: 2.24-11+deb9u4
      Candidate: 2.24-11+deb9u4
      Version table:
     *** 2.24-11+deb9u4 500
            500 http://deb.debian.org/debian stretch/main amd64 Packages
            100 /var/lib/dpkg/status
         2.24-11+deb9u1 500
            500 http://security.debian.org stretch/updates/main amd64 Packages
    
    opened by hypercompetent 2
  • Documentation links broken

    Documentation links broken

    Github readme still points to the github pages doc (biovault.github.io/nptsne) and the PyPI links old nptsne 1.0.0 doc (bldrvnlw.readthedocs.io) Update to https://nptsne.readthedocs.io/en/release-1.1.0

    opened by bldrvnlw 0
  • On 1.2.0 cmake fails with conan errors

    On 1.2.0 cmake fails with conan errors

    Hi, I'm trying to install the latest version (1.2.0, bb49bb6aa1bdde4c4e39652e1ae331fecdf2c782). When I run pip install -v -e . it fails with cmake exit status 1. The problem is in the following cmake call:

    cmake /home/vpetukhov/build/nptsne  -DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/home/vpetukhov/build/nptsne/src/nptsne/libs -DPYTHON_EXECUTABLE=/home/vpetukhov/local/miniconda3/envs/viaduct/bin/python -DCMAKE_BUILD_TYPE=Release -DTEMP_LIBS_DIR=/tmp/cibwlibsdir -DLIBCXX=libstdc++ -DNPTSNE_BUILD_WITH_CONAN=ON
    

    It fails with a certificate error:

    ERROR: HTTPSConnectionPool(host='lkeb-artifactory.lumc.nl', port=443): Max retries exceeded with url: /artifactory/api/conan/conan-local/v1/ping (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1123)')))

    And if I disable SSL validation with conan remote update conan-biovault https://lkeb-artifactory.lumc.nl/artifactory/api/conan/conan-local False, the new error is:

    ERROR: HDILib/1.2.4@lkeb/testing: option 'shared' doesn't exist Possible options are []

    Here are full cmake logs for enabled and disabled ssl.

    opened by VPetukhov 2
  • Support non-Euclidean distance for TextureTsne

    Support non-Euclidean distance for TextureTsne

    Hi, I work with high-dimensional data (hundreds of dimensions), and it requires using cosine distance. However, TextureTsne has Euclidean distance hardcoded. Could you please make it configurable?

    opened by VPetukhov 6
  • nptsne 1.1.0 missing latest HDILib features

    nptsne 1.1.0 missing latest HDILib features

    New features in HDILib (changing metrics and user generated pre-calculated metrics) are not accessible in nptsne 1.1.0 . Please make a new release soon.

    opened by bldrvnlw 0
  • Reports of issues installing on python 3.8.1 (conda)

    Reports of issues installing on python 3.8.1 (conda)

    Original report

    When creating a conda environment, 3.8.1 was the default, so at first I was wondering why it didn't work
    3:06
    Now, I'm not super familiar with the sub-versioning of python and package distribution, so I don't know if 3.8.1 should work when 3.8 is supported
    3:10
    Mhm, but I'm a little confused:
    3:12
    If I create a conda environment without specifying the python version, it defaults to 3.8.1 and pip install nptsne does not work (ERROR: Could not find a version that satisfies the requirement nptsne (from versions: none)
    ERROR: No matching distribution found for nptsne) but when I create an environment and explicitly define the python version to be 3.8.1, it works just fine
    
    opened by bldrvnlw 0
  • HSNELouvainDemo fails with OSError: [WinError 126]

    HSNELouvainDemo fails with OSError: [WinError 126]

    python testhsnelouvain.py

    Traceback (most recent call last): File "testhsnelouvain.py", line 5, in import community as community_louvain File "C:\Users\bvanlew\Anaconda3\envs\p37hsnedemo\lib\site-packages\community_init_.py", line 9, in from .community_louvain import ( File "C:\Users\bvanlew\Anaconda3\envs\p37hsnedemo\lib\site-packages\community\community_louvain.py", line 12, in import networkx as nx File "C:\Users\bvanlew\Anaconda3\envs\p37hsnedemo\lib\site-packages\networkx_init_.py", line 112, in import networkx.generators File "C:\Users\bvanlew\Anaconda3\envs\p37hsnedemo\lib\site-packages\networkx\generators_init_.py", line 8, in from networkx.generators.community import * File "C:\Users\bvanlew\Anaconda3\envs\p37hsnedemo\lib\site-packages\networkx\generators\community.py", line 20, in from scipy.special import zeta as zeta File "C:\Users\bvanlew\Anaconda3\envs\p37hsnedemo\lib\site-packages\scipy_init.py", line 136, in from . import distributor_init File "C:\Users\bvanlew\Anaconda3\envs\p37hsnedemo\lib\site-packages\scipy_distributor_init.py", line 61, in WinDLL(os.path.abspath(filename)) File "C:\Users\bvanlew\Anaconda3\envs\p37hsnedemo\lib\ctypes_init.py", line 364, in init self._handle = _dlopen(self._name, mode) OSError: [WinError 126] The specified module could not be found

    opened by bldrvnlw 2
Owner
Biomedical Visual Analytics Unit LUMC - TU Delft
Biomedical Visual Analytics Unit LUMC - TU Delft
649 Pokémon palettes as CSVs, with a Python lib to turn names/IDs into palettes, or MatPlotLib compatible ListedColormaps.

PokePalette 649 Pokémon, broken down into CSVs of their RGB colour palettes. Complete with a Python library to convert names or Pokédex IDs into eithe

null 11 Dec 5, 2022
This project is an Algorithm Visualizer where a user can visualize algorithms like Bubble Sort, Merge Sort, Quick Sort, Selection Sort, Linear Search and Binary Search.

Algo_Visualizer This project is an Algorithm Visualizer where a user can visualize common algorithms like "Bubble Sort", "Merge Sort", "Quick Sort", "

Rahul 4 Feb 7, 2022
The windML framework provides an easy-to-use access to wind data sources within the Python world, building upon numpy, scipy, sklearn, and matplotlib. Renewable Wind Energy, Forecasting, Prediction

windml Build status : The importance of wind in smart grids with a large number of renewable energy resources is increasing. With the growing infrastr

Computational Intelligence Group 125 Dec 24, 2022
A simple python script using Numpy and Matplotlib library to plot a Mohr's Circle when given a two-dimensional state of stress.

Mohr's Circle Calculator This is a really small personal project done for Department of Civil Engineering, Delhi Technological University (formerly, D

Agyeya Mishra 0 Jul 17, 2021
In-memory Graph Database and Knowledge Graph with Natural Language Interface, compatible with Pandas

CogniPy for Pandas - In-memory Graph Database and Knowledge Graph with Natural Language Interface Whats in the box Reasoning, exploration of RDF/OWL,

Cognitum Octopus 34 Dec 13, 2022
NumPy and Pandas interface to Big Data

Blaze translates a subset of modified NumPy and Pandas-like syntax to databases and other computing systems. Blaze allows Python users a familiar inte

Blaze 3.1k Jan 1, 2023
Python scripts to manage Chia plots and drive space, providing full reports. Also monitors the number of chia coins you have.

Chia Plot, Drive Manager & Coin Monitor (V0.5 - April 20th, 2021) Multi Server Chia Plot and Drive Management Solution Be sure to ⭐ my repo so you can

null 338 Nov 25, 2022
Visualizing weather changes across the world using third party APIs and Python.

WEATHER FORECASTING ACROSS THE WORLD Overview Python scripts were created to visualize the weather for over 500 cities across the world at varying di

G Johnson 0 Jun 12, 2021
Bar Chart of the number of Senators from each party who are up for election in the next three General Elections

Congress-Analysis Bar Chart of the number of Senators from each party who are up for election in the next three General Elections This bar chart shows

null 11 Oct 26, 2021
Yata is a fast, simple and easy Data Visulaization tool, running on python dash

Yata is a fast, simple and easy Data Visulaization tool, running on python dash. The main goal of Yata is to provide a easy way for persons with little programming knowledge to visualize their data easily.

Cybercreek 3 Jun 28, 2021
Simple and fast histogramming in Python accelerated with OpenMP.

pygram11 Simple and fast histogramming in Python accelerated with OpenMP with help from pybind11. pygram11 provides functions for very fast histogram

Doug Davis 28 Dec 14, 2022
A python package for animating plots build on matplotlib.

animatplot A python package for making interactive as well as animated plots with matplotlib. Requires Python >= 3.5 Matplotlib >= 2.2 (because slider

Tyler Makaro 394 Dec 18, 2022
A python package for animating plots build on matplotlib.

animatplot A python package for making interactive as well as animated plots with matplotlib. Requires Python >= 3.5 Matplotlib >= 2.2 (because slider

Tyler Makaro 356 Feb 16, 2021
Python package for hypergraph analysis and visualization.

The HyperNetX library provides classes and methods for the analysis and visualization of complex network data. HyperNetX uses data structures designed to represent set systems containing nested data and/or multi-way relationships. The library generalizes traditional graph metrics to hypergraphs.

Pacific Northwest National Laboratory 304 Dec 27, 2022
Python package that generates hardware pinout diagrams as SVG images

PinOut A Python package that generates hardware pinout diagrams as SVG images. The package is designed to be quite flexible and works well for general

null 336 Dec 20, 2022
A Python package that provides evaluation and visualization tools for the DexYCB dataset

DexYCB Toolkit DexYCB Toolkit is a Python package that provides evaluation and visualization tools for the DexYCB dataset. The dataset and results wer

NVIDIA Research Projects 107 Dec 26, 2022
A minimal Python package that produces slice plots through h5m DAGMC geometry files

A minimal Python package that produces slice plots through h5m DAGMC geometry files Installation pip install dagmc_geometry_slice_plotter Python API U

Fusion Energy 4 Dec 2, 2022
eoplatform is a Python package that aims to simplify Remote Sensing Earth Observation by providing actionable information on a wide swath of RS platforms and provide a simple API for downloading and visualizing RS imagery

An Earth Observation Platform Earth Observation made easy. Report Bug | Request Feature About eoplatform is a Python package that aims to simplify Rem

Matthew Tralka 4 Aug 11, 2022
Python Package for CanvasXpress JS Visualization Tools

CanvasXpress Python Library About CanvasXpress for Python CanvasXpress was developed as the core visualization component for bioinformatics and system

Dr. Todd C. Brett 5 Nov 7, 2022