HyperTag

HyperTag helps humans intuitively express how they think about their files using tags and machine learning. Represent how you think using tags. Find what you look for using semantic search for your text documents (yes, even PDF's) and images. Instead of introducing proprietary file formats like other existing file organization tools, HyperTag just smoothly layers on top of your existing files without any fuss.

Objective Function: Minimize time between a thought and access to all relevant files.

Accompanying blog post: https://blog.neotree.uber.space/posts/hypertag-file-organization-made-for-humans

Table of Contents

HyperTag

Install

Available on PyPI

$ pip install hypertag (supports both CPU only & CUDA accelerated execution!)

Community

Join the HyperTag matrix chat room to stay up to date on the latest developments or to ask for help.

Overview

HyperTag offers a slick CLI but more importantly it creates a directory called HyperTagFS which is a file system based representation of your files and tags using symbolic links and directories.

Directory Import: Import your existing directory hierarchies using $ hypertag import path/to/directory. HyperTag converts it automatically into a tag hierarchy using metatagging.

Semantic Text & Image Search (Experimental): Search for images (jpg, png) and text documents (yes, even PDF's) content with a simple text query. Text search is powered by the awesome Sentence Transformers library. Text to image search is powered by OpenAI's CLIP model. Currently only English queries are supported.

HyperTag Daemon (Experimental): Monitors HyperTagFS and directories added to the auto import list for user changes (see section "Start HyperTag Daemon" below). Also spawns the DaemonService which speeds up semantic search significantly (warning: daemon process is a RAM hog with ~2GB usage).

Fuzzy Matching Queries: HyperTag uses fuzzy matching to minimize friction in the unlikely case of a typo.

File Type Groups: HyperTag automatically creates folders containing common files (e.g. Images: jpg, png, etc., Documents: txt, pdf, etc., Source Code: py, js, etc.), which can be found in HyperTagFS.

HyperTag Graph: Quickly get an overview of your HyperTag Graph! HyperTag visualizes the metatag graph on every change and saves it at HyperTagFS/hypertag-graph.pdf.

CLI Functions

Import existing directory recursively

Import files with tags inferred from the existing directory hierarchy.

$ hypertag import path/to/directory

Add file/s or URL/s manually

$ hypertag add path/to/file https://github.com/SeanPedersen/HyperTag

Tag file/s (with values)

Manually tag files. Shortcut: $ hypertag t

$ hypertag tag humans/*.txt with human "Homo Sapiens"

Add a value to a file's tag:

$ hypertag tag sean.txt with name="Sean Pedersen"

Untag file/s

Manually remove tag/s from file/s.

$ hypertag untag humans/*.txt with human "Homo Sapiens"

Tag a tag

Metatag tag/s to create tag hierarchies. Shortcut: $ hypertag tt

$ hypertag metatag human with animal

Merge tags

Merge all associations (files & tags) of tag A into tag B.

$ hypertag merge human into "Homo Sapiens"

Query using Set Theory

Print file names of the resulting set matching the query. Queries are composed of tags (with values) and operands. Tags are fuzzy matched for convenience. Nesting is currently not supported, queries are evaluated from left to right.
Shortcut: $ hypertag q

Query with a value using a wildcard: $ hypertag query name="Sean*"
Print paths: $ hypertag query human --path
Print fuzzy matched tag: $ hypertag query man --verbose
Disable fuzzy matching: $ hypertag query human --fuzzy=0

Default operand is AND (intersection):
$ hypertag query human name="Sean*" is equivalent to $ hypertag query human and name="Sean*"

OR (union):
$ hypertag query human or "Homo Sapiens"

MINUS (difference):
$ hypertag query human minus "Homo Sapiens"

Index supported image and text files

Only indexed files can be searched.

$ hypertag index

To parse even unparseable PDF's, install tesseract: # pacman -S tesseract tesseract-data-eng

Index only image files: $ hypertag index --image
Index only text files: $ hypertag index --text

Semantic search for text files

A custom search algorithm combining semantic with token matching search. Print text file names sorted by matching score. Performance benefits greatly from running the HyperTag daemon.
Shortcut: $ hypertag s

$ hypertag search "your important text query" --path --score --top_k=10

Semantic search for image files

Print image file names sorted by matching score. Performance benefits greatly from running the HyperTag daemon.
Shortcut: $ hypertag si

Text to image: $ hypertag search_image "your image content description" --path --score --top_k=10

Image to image: $ hypertag search_image "path/to/image.jpg" --path --score --top_k=10

Start HyperTag Daemon

Start daemon process with triple functionality:

Watches HyperTagFS directory for user changes
- Maps file (symlink) and directory deletions into tag / metatag removal/s
- On directory creation: Interprets name as set theory tag query and automatically populates it with results
- On directory creation in Search Images or Search Texts: Interprets name as semantic search query (add top_k=42 to limit result size) and automatically populates it with results
Watches directories on the auto import list for user changes:
- Maps file changes (moves & renames) to DB
- On file creation: Adds new file/s with inferred tag/s and auto-indexes it (if supported file format).
Spawns DaemonService to load and expose models used for semantic search, speeding it up significantly

$ hypertag daemon

Print all tags of file/s

$ hypertag tags filename1 filename2

Print all metatags of tag/s

$ hypertag metatags tag1 tag2

Print all tags

$ hypertag show

Print all files

Print names: $ hypertag show files

Print paths: $ hypertag show files --path

Visualize HyperTag Graph

Visualize the metatag graph hierarchy (saved at HyperTagFS root).

$ hypertag graph

Specify layout algorithm (default: fruchterman_reingold):

$ hypertag graph --layout=kamada_kawai

Generate HyperTagFS

Generate file system based representation of your files and tags using symbolic links and directories.

$ hypertag mount

Add directory to auto import list

Directories added to the auto import list will be monitored by the daemon for new files or changes.

$ hypertag add_auto_import_dir path/to/directory

Set HyperTagFS directory path

Default is the user's home directory.

$ hypertag set_hypertagfs_dir path/to/directory

Architecture

Python and it's vibrant open-source community power HyperTag
Many other awesome open-source projects make HyperTag possible (listed in pyproject.toml)
SQLite3 serves as the meta data storage engine (located at ~/.config/hypertag/hypertag.db)
Added URLs are saved in ~/.config/hypertag/web_pages for websites, others in ~/.config/hypertag/downloads
Symbolic links are used to create the HyperTagFS directory structure
Semantic Search: boosted using hnswlib
- Text to text search is powered by the awesome DistilBERT
- Text to image & image to image search is powered by OpenAI's impressive CLIP model

Development

Find prioritized issues here: TODO List
Pick an issue and comment how you plan to tackle it before starting out, to make sure no dev time is wasted.
Clone repo: $ git clone https://github.com/SeanPedersen/HyperTag.git
$ cd HyperTag/
Install Poetry
Install dependencies: $ poetry install
Activate virtual environment: $ poetry shell
Run all tests: $ pytest -v
Run formatter: $ black hypertag/
Run linter: $ flake8
Run type checking: $ mypy **/*.py
Run security checking: $ bandit --exclude tests/ -r .
Codacy: Dashboard
Run HyperTag: $ python -m hypertag

Inspiration

What is the point of HyperTag's existence?
HyperTag offers many unique features such as the import, semantic search, graphing and fuzzy matching functions that make it very convenient to use. All while HyperTag's code base staying relatively tiny at <2000 LOC compared to similar projects like TMSU (>10,000 LOC in Go) and SuperTag (>25,000 LOC in Rust), making it easy to hack on.

running current master (d3d8300b195)

% python -m hypertag search info
/home/john/.cache/pypoetry/virtualenvs/hypertag-4QF0ZWLA-py3.9/lib/python3.9/site-packages/torch/cuda/__init__.py:52: UserWarning: CUDA initialization: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx (Triggered internally at  /pytorch/c10/cuda/CUDAFunctions.cpp:100.)
  return torch._C._cuda_getDeviceCount() > 0
[1]    11067 segmentation fault  python -m hypertag search info

gdb stack trace:

0x00007ffe6418a6d7 in Index<float, float>::knnQuery_return_numpy(pybind11::object, unsigned long, int)::{lambda(unsigned long, unsigned long)#2}::operator()(unsigned long, unsigned long) const (this=0x55555980ac90, row=row@entry=0, threadId=threadId@entry=0) at bindings.cpp:315
315     bindings.cpp: No such file or directory.
(gdb) bt full
#0  0x00007ffe6418a6d7 in Index<float, float>::knnQuery_return_numpy(pybind11::object, unsigned long, int)::{lambda(unsigned long, unsigned long)#2}::operator()(unsigned long, unsigned long) const (this=0x55555980ac90, row=row@entry=0, threadId=threadId@entry=0) at bindings.cpp:315
        data = <optimized out>
        start_idx = <optimized out>
        result = std::priority_queue wrapping: std::vector of length -192, capacity 25769803593 = {<error reading variable result (Cannot access memory at address 0xc30)>
        data_numpy_l = @0x7fffffffb210: 0x5555582e4060
        data_numpy_d = @0x7fffffffb218: 0x5555586abbc0
        k = @0x7fffffffb1f8: 40
        norm_array = std::vector of length 768, capacity 768 = {-0.0656438693, -0.00188240607, 0.024570806, -0.00132345792, 0.0250285137, -0.0298759732, 0.0306670982, 0.00316128298, -0.0284196995, -0.0187344141, -0.0242117085, 0.0363302045, -0.0213107802, -0.0186562091, -0.0209287778, -0.0191882774, 0.0233381856,
          0.00579207297, -0.0691136792, -0.00609853491, 0.0187803525, -0.0146395136, -0.0138068749, 0.0181932077, 0.0265010484, 0.0328834727, 0.0109599996, -0.0108995168, -0.0538588502, -0.038510941, -0.00958266016, -0.0123666301, -0.0104721515, 0.068536438, 0.00844705012, -0.0400565453, -0.0012298777,
          0.00155452534, 0.0183192324, 0.00129542616, 0.0504850112, 0.0635909587, 0.0709769726, 0.0390915535, -0.00420415169, -0.0351894051, -0.0472720265, 0.0298247505, -0.0904107764, -0.0165663566, -0.0187629294, 0.00447256444, 0.0280851889, 0.0292292573, -0.0141723091, 0.0162908006, -0.059703283,
          -0.00156370644, -0.00643661199, 0.049946405, 0.0334154293, 0.0287730508, 0.0163538437, -0.0425121821, 0.00693678297, 0.00168456219, 0.0312860757, -0.0207320303, 0.0186786968, -0.0284498483, 0.014462023, -0.050294999, -0.0786222368, -0.00467137992, -0.00730591174, -0.00373656908, -0.0207697768,
          0.036457371, 0.0311673228, 0.00110819482, -0.0183178894, -0.0189260878, -0.0155715616, -0.0187465418, 0.00147301028, -0.0545794182, -0.0671095252, 0.0195625871, -0.0236419607, 0.0429460928, 0.047442425, -0.0551435314, -0.014894818, 0.0460909568, -0.0436526127, -0.0701980069, -0.0238671843, -0.0181005225,
          -0.0107285921, 0.0023586622, 0.0245333388, -0.0149427503, 0.00258170348, -0.0166542511, -0.0349081531, 0.0157148689, 0.027221946, 0.000511900114, -0.0220854413, 0.00104241422, -0.0679998919, -0.0417860784, 0.0270387214, 0.0649867281, -0.00174358313, 0.0174567234, -0.00848075375, 0.0265734121,
          0.0357317738, -0.021921346, -0.055617284, 0.0899901837, 0.0105315251, 0.0751143992, 0.0247911327, 0.0716819167, -0.00638728356, 0.0364727117, 0.0457414016, -0.0330321081, 0.0582752153, -0.0101318816, -0.0288470928, -0.0558135808, 0.0357543379, 0.0200881418, 0.0377327949, 0.020218417, 0.0150590744,
          0.0103015713, 0.088244088, 0.0369234011, -0.0466240346, -0.0196729153, 0.0469583608, -0.0450488739, -0.0273035206, -0.0187784862, 0.0366531126, 0.0216387268, -0.00605851645, -0.0292723831, 0.0314363725, -0.0274906158, -0.0479848012, -0.0641067401, -0.0597888455, -0.0380994976, -0.0326780826,
          0.00785975717, 0.0481747016, 0.000950561138, -0.0176460147, -0.0181128122, -0.0411796272, 0.0198744182, -0.0466705188, 0.0194101408, 0.03275159, 0.0567777753, 0.0404897071, -0.0546920411, -0.0469799712, 0.071445778, -0.0140111465, -0.0112946713, 0.0324517675, -0.0191585794, -0.0126166344, -0.0474057607,
          -0.00157130254, -0.0483237244, 0.0341237411, -0.0373514146, -0.0388341397, 0.027352469, -0.00625597592, -0.0236225352, 0.0235540867, -0.0165337138, 0.0128682218, -0.0323442556, 0.0289525893, -0.0109649915, 0.0528527834, -0.0154767036, 0.0231210813, -0.00930580404, -0.0102893002, -0.0458446592...}
        this = 0x55555980ac90
        items = @0x7fffffffb208: {<pybind11::array> = {<pybind11::buffer> = {<pybind11::object> = {<pybind11::handle> = {<pybind11::detail::object_api<pybind11::handle>> = {<pybind11::detail::pyobject_tag> = {<No data fields>}, <No data fields>},
                  m_ptr = 0x7ffff3794530}, <No data fields>}, <No data fields>}, <No data fields>}, <No data fields>}
#1  0x00007ffe641a0237 in ParallelFor<Index<float, float>::knnQuery_return_numpy(pybind11::object, unsigned long, int)::{lambda(unsigned long, unsigned long)#2}>(unsigned long, unsigned long, unsigned long, Index<float, float>::knnQuery_return_numpy(pybind11::object, unsigned long, int)::{lambda(unsigned long, unsigned long)#2}) (fn=..., numThreads=<optimized out>, end=<optimized out>, start=0) at bindings.cpp:26
        id = 0
#2  Index<float, float>::knnQuery_return_numpy (this=0x55555980ac90, input=..., k=<optimized out>, num_threads=<optimized out>) at bindings.cpp:309
        norm_array = std::vector of length 768, capacity 768 = {-0.0656438693, -0.00188240607, 0.024570806, -0.00132345792, 0.0250285137, -0.0298759732, 0.0306670982, 0.00316128298, -0.0284196995, -0.0187344141, -0.0242117085, 0.0363302045, -0.0213107802, -0.0186562091, -0.0209287778, -0.0191882774, 0.0233381856,
          0.00579207297, -0.0691136792, -0.00609853491, 0.0187803525, -0.0146395136, -0.0138068749, 0.0181932077, 0.0265010484, 0.0328834727, 0.0109599996, -0.0108995168, -0.0538588502, -0.038510941, -0.00958266016, -0.0123666301, -0.0104721515, 0.068536438, 0.00844705012, -0.0400565453, -0.0012298777,
          0.00155452534, 0.0183192324, 0.00129542616, 0.0504850112, 0.0635909587, 0.0709769726, 0.0390915535, -0.00420415169, -0.0351894051, -0.0472720265, 0.0298247505, -0.0904107764, -0.0165663566, -0.0187629294, 0.00447256444, 0.0280851889, 0.0292292573, -0.0141723091, 0.0162908006, -0.059703283,
          -0.00156370644, -0.00643661199, 0.049946405, 0.0334154293, 0.0287730508, 0.0163538437, -0.0425121821, 0.00693678297, 0.00168456219, 0.0312860757, -0.0207320303, 0.0186786968, -0.0284498483, 0.014462023, -0.050294999, -0.0786222368, -0.00467137992, -0.00730591174, -0.00373656908, -0.0207697768,
          0.036457371, 0.0311673228, 0.00110819482, -0.0183178894, -0.0189260878, -0.0155715616, -0.0187465418, 0.00147301028, -0.0545794182, -0.0671095252, 0.0195625871, -0.0236419607, 0.0429460928, 0.047442425, -0.0551435314, -0.014894818, 0.0460909568, -0.0436526127, -0.0701980069, -0.0238671843, -0.0181005225,
          -0.0107285921, 0.0023586622, 0.0245333388, -0.0149427503, 0.00258170348, -0.0166542511, -0.0349081531, 0.0157148689, 0.027221946, 0.000511900114, -0.0220854413, 0.00104241422, -0.0679998919, -0.0417860784, 0.0270387214, 0.0649867281, -0.00174358313, 0.0174567234, -0.00848075375, 0.0265734121,
          0.0357317738, -0.021921346, -0.055617284, 0.0899901837, 0.0105315251, 0.0751143992, 0.0247911327, 0.0716819167, -0.00638728356, 0.0364727117, 0.0457414016, -0.0330321081, 0.0582752153, -0.0101318816, -0.0288470928, -0.0558135808, 0.0357543379, 0.0200881418, 0.0377327949, 0.020218417, 0.0150590744,
          0.0103015713, 0.088244088, 0.0369234011, -0.0466240346, -0.0196729153, 0.0469583608, -0.0450488739, -0.0273035206, -0.0187784862, 0.0366531126, 0.0216387268, -0.00605851645, -0.0292723831, 0.0314363725, -0.0274906158, -0.0479848012, -0.0641067401, -0.0597888455, -0.0380994976, -0.0326780826,
          0.00785975717, 0.0481747016, 0.000950561138, -0.0176460147, -0.0181128122, -0.0411796272, 0.0198744182, -0.0466705188, 0.0194101408, 0.03275159, 0.0567777753, 0.0404897071, -0.0546920411, -0.0469799712, 0.071445778, -0.0140111465, -0.0112946713, 0.0324517675, -0.0191585794, -0.0126166344, -0.0474057607,
          -0.00157130254, -0.0483237244, 0.0341237411, -0.0373514146, -0.0388341397, 0.027352469, -0.00625597592, -0.0236225352, 0.0235540867, -0.0165337138, 0.0128682218, -0.0323442556, 0.0289525893, -0.0109649915, 0.0528527834, -0.0154767036, 0.0231210813, -0.00930580404, -0.0102893002, -0.0458446592...}
        l = {tstate = 0x55555557f6d0, disassoc = false}
        items = {<pybind11::array> = {<pybind11::buffer> = {<pybind11::object> = {<pybind11::handle> = {<pybind11::detail::object_api<pybind11::handle>> = {<pybind11::detail::pyobject_tag> = {<No data fields>}, <No data fields>},
                  m_ptr = 0x7ffff3794530}, <No data fields>}, <No data fields>}, <No data fields>}, <No data fields>}
        buffer = {ptr = 0x555558f8e100, itemsize = 4, size = 768, format = "f", ndim = 1, shape = std::vector of length 1, capacity 1 = {768}, strides = std::vector of length 1, capacity 1 = {4}, readonly = false, m_view = 0x555558f77000, ownview = true}
        data_numpy_l = 0x5555582e4060
        data_numpy_d = 0x5555586abbc0
        rows = 1
        features = <optimized out>
        free_when_done_l = {<pybind11::object> = {<pybind11::handle> = {<pybind11::detail::object_api<pybind11::handle>> = {<pybind11::detail::pyobject_tag> = {<No data fields>}, <No data fields>}, m_ptr = 0x0}, <No data fields>}, <No data fields>}
        free_when_done_d = {<pybind11::object> = {<pybind11::handle> = {<pybind11::detail::object_api<pybind11::handle>> = {<pybind11::detail::pyobject_tag> = {<No data fields>}, <No data fields>}, m_ptr = 0x55555557f6d0}, <No data fields>}, <No data fields>}
#3  0x00007ffe641a16c4 in pybind11::cpp_function::cpp_function<pybind11::object, Index<float, float>, pybind11::object, unsigned long, int, pybind11::name, pybind11::is_method, pybind11::sibling, pybind11::arg, pybind11::arg_v, pybind11::arg_v>(pybind11::object (Index<float, float>::*)(pybind11::object, unsigned long, int), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&, pybind11::arg const&, pybind11::arg_v const&, pybind11::arg_v const&)::{lambda(Index<float, float>*, pybind11::object, unsigned long, int)#1}::operator()(Index<float, float>*, pybind11::object, unsigned long, int) const (
    this=<optimized out>, this=<optimized out>, args#2=<optimized out>, args#1=<optimized out>, args#0=..., c=<optimized out>) at /home/john/.cache/pypoetry/virtualenvs/hypertag-4QF0ZWLA-py3.9/lib/python3.9/site-packages/pybind11/include/pybind11/pybind11.h:84
        f = <optimized out>
        f = <optimized out>
#4  pybind11::detail::argument_loader<Index<float, float>*, pybind11::object, unsigned long, int>::call_impl<pybind11::object, pybind11::cpp_function::cpp_function<pybind11::object, Index<float, float>, pybind11::object, unsigned long, int, pybind11::name, pybind11::is_method, pybind11::sibling, pybind11::arg, pybind11::arg_v, pybind11::arg_v>(pybind11::object (Index<float, float>::*)(pybind11::object, unsigned long, int), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&, pybind11::arg const&, pybind11::arg_v const&, pybind11::arg_v const&)::{lambda(Index<float, float>*, pybind11::object, unsigned long, int)#1}&, 0ul, 1ul, 2ul, 3ul, pybind11::detail::void_type>(pybind11::cpp_function::cpp_function<pybind11::object, Index<float, float>, pybind11::object, unsigned long, int, pybind11::name, pybind11::is_method, pybind11::sibling, pybind11::arg, pybind11::arg_v, pybind11::arg_v>(pybind11::object (Index<float, float>::*)(pybind11::object, unsigned long, int), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&, pybind11::arg const&, pybind11::arg_v const&, pybind11::arg_v const&)::{lambda(Index<float, float>*, pybind11::object, unsigned long, int)#1}&, std::integer_sequence<unsigned long, 0ul, 1ul, 2ul, 3ul>, pybind11::detail::void_type&&) && (f=..., this=0x7fffffffb490) at /home/john/.cache/pypoetry/virtualenvs/hypertag-4QF0ZWLA-py3.9/lib/python3.9/site-packages/pybind11/include/pybind11/cast.h:2010
No locals.
#5  pybind11::detail::argument_loader<Index<float, float>*, pybind11::object, unsigned long, int>::call<pybind11::object, pybind11::detail::void_type, pybind11::cpp_function::cpp_function<pybind11::object, Index<float, float>, pybind11::object, unsigned long, int, pybind11::name, pybind11::is_method, pybind11::sibling, pybind11::arg, pybind11::arg_v, pybind11::arg_v>(pybind11::object (Index<float, float>::*)(pybind11::object, unsigned long, int), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&, pybind11::arg const&, pybind11::arg_v const&, pybind11::arg_v const&)::{lambda(Index<float, float>*, pybin--Type <RET> for more, q to quit, c to continue without paging--
d11::object, unsigned long, int)#1}&>(pybind11::cpp_function::cpp_function<pybind11::object, Index<float, float>, pybind11::object, unsigned long, int, pybind11::name, pybind11::is_method, pybind11::sibling, pybind11::arg, pybind11::arg_v, pybind11::arg_v>(pybind11::object (Index<float, float>::*)(pybind11::object, unsigned long, int), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&, pybind11::arg const&, pybind11::arg_v const&, pybind11::arg_v const&)::{lambda(Index<float, float>*, pybind11::object, unsigned long, int)#1}&) && (f=..., this=0x7fffffffb490)
    at /home/john/.cache/pypoetry/virtualenvs/hypertag-4QF0ZWLA-py3.9/lib/python3.9/site-packages/pybind11/include/pybind11/cast.h:1982
No locals.
#6  pybind11::cpp_function::initialize<pybind11::cpp_function::initialize<pybind11::object, Index<float, float>, pybind11::object, unsigned long, int, pybind11::name, pybind11::is_method, pybind11::sibling, pybind11::arg, pybind11::arg_v, pybind11::arg_v>(pybind11::object (Index<float, float>::*)(pybind11::object, unsigned long, int), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&, pybind11::arg const&, pybind11::arg_v const&, pybind11::arg_v const&)::{lambda(Index<float, float>*, pybind11::object, unsigned long, int)#1}, pybind11::object, Index<float, float>*, pybind11::object, unsigned long, int, pybind11::name, pybind11::is_method, pybind11::sibling, pybind11::arg, pybind11::arg_v, pybind11::arg_v>(pybind11::cpp_function::initialize<pybind11::object, Index<float, float>, pybind11::object, unsigned long, int, pybind11::name, pybind11::is_method, pybind11::sibling, pybind11::arg, pybind11::arg_v, pybind11::arg_v>(pybind11::object (Index<float, float>::*)(pybind11::object, unsigned long, int), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&, pybind11::arg const&, pybind11::arg_v const&, pybind11::arg_v const&)::{lambda(Index<float, float>*, pybind11::object, unsigned long, int)#1}&&, pybind11::object (*)(Index<float, float>*, pybind11::object, unsigned long, int), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&, pybind11::arg const&, pybind11::arg_v const&, pybind11::arg_v const&)::{lambda(pybind11::detail::function_call&)#3}::operator()(pybind11::detail::function_call&) const (this=0x0, call=...) at /home/john/.cache/pypoetry/virtualenvs/hypertag-4QF0ZWLA-py3.9/lib/python3.9/site-packages/pybind11/include/pybind11/pybind11.h:184
        args_converter = {static args_pos = 0, static kwargs_pos = 0, static args_kwargs_are_last = true, static has_kwargs = false, static has_args = false, static arg_names = 746399099, argcasters = std::tuple containing = {
            [1] = {<pybind11::detail::type_caster_base<Index<float, float> >> = {<pybind11::detail::type_caster_generic> = {typeinfo = 0x5555589befa0, cpptype = 0x7ffe641b53c0 <typeinfo for Index<float, float>>, value = 0x55555980ac90}, static name = -669581275}, <No data fields>},
            [2] = {<pybind11::detail::pyobject_caster<pybind11::object>> = {value = {<pybind11::handle> = {<pybind11::detail::object_api<pybind11::handle>> = {<pybind11::detail::pyobject_tag> = {<No data fields>}, <No data fields>}, m_ptr = 0x0}, <No data fields>}, static name = -114687963}, <No data fields>},
            [3] = {value = 40, static name = 7630441}, [4] = {value = -1, static name = 7630441}}}
        data = <optimized out>
        policy = <optimized out>
        cap = <optimized out>
        result = {<pybind11::detail::object_api<pybind11::handle>> = {<pybind11::detail::pyobject_tag> = {<No data fields>}, <No data fields>}, m_ptr = <optimized out>}
#7  pybind11::cpp_function::initialize<pybind11::cpp_function::initialize<pybind11::object, Index<float, float>, pybind11::object, unsigned long, int, pybind11::name, pybind11::is_method, pybind11::sibling, pybind11::arg, pybind11::arg_v, pybind11::arg_v>(pybind11::object (Index<float, float>::*)(pybind11::object, unsigned long, int), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&, pybind11::arg const&, pybind11::arg_v const&, pybind11::arg_v const&)::{lambda(Index<float, float>*, pybind11::object, unsigned long, int)#1}, pybind11::object, Index<float, float>*, pybind11::object, unsigned long, int, pybind11::name, pybind11::is_method, pybind11::sibling, pybind11::arg, pybind11::arg_v, pybind11::arg_v>(pybind11::cpp_function::initialize<pybind11::object, Index<float, float>, pybind11::object, unsigned long, int, pybind11::name, pybind11::is_method, pybind11::sibling, pybind11::arg, pybind11::arg_v, pybind11::arg_v>(pybind11::object (Index<float, float>::*)(pybind11::object, unsigned long, int), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&, pybind11::arg const&, pybind11::arg_v const&, pybind11::arg_v const&)::{lambda(Index<float, float>*, pybind11::object, unsigned long, int)#1}&&, pybind11::object (*)(Index<float, float>*, pybind11::object, unsigned long, int), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&, pybind11::arg const&, pybind11::arg_v const&, pybind11::arg_v const&)::{lambda(pybind11::detail::function_call&)#3}::_FUN(pybind11::detail::function_call&) ()
    at /home/john/.cache/pypoetry/virtualenvs/hypertag-4QF0ZWLA-py3.9/lib/python3.9/site-packages/pybind11/include/pybind11/pybind11.h:161
No locals.
#8  0x00007ffe64195e06 in pybind11::cpp_function::dispatcher (self=<optimized out>, args_in=0x7ffe5d492bc0, kwargs_in=0x7ffe635c8e80) at /home/john/.cache/pypoetry/virtualenvs/hypertag-4QF0ZWLA-py3.9/lib/python3.9/site-packages/pybind11/include/pybind11/pybind11.h:717
        guard = {<No data fields>}
        func = @0x555558113390: {name = 0x55555859b1d0 "knn_query", doc = 0x0, signature = 0x5555586d2bc0 "(self: hnswlib.Index, data: object, k: int = 1, num_threads: int = -1) -> object", args = std::vector of length 4, capacity 4 = {{name = 0x555558315980 "self", descr = 0x0,
              value = {<pybind11::detail::object_api<pybind11::handle>> = {<pybind11::detail::pyobject_tag> = {<No data fields>}, <No data fields>}, m_ptr = 0x0}, convert = true, none = false}, {name = 0x5555586c5e00 "data", descr = 0x0,
              value = {<pybind11::detail::object_api<pybind11::handle>> = {<pybind11::detail::pyobject_tag> = {<No data fields>}, <No data fields>}, m_ptr = 0x0}, convert = true, none = true}, {name = 0x5555586d2b10 "k", descr = 0x5555586d1000 "1",
              value = {<pybind11::detail::object_api<pybind11::handle>> = {<pybind11::detail::pyobject_tag> = {<No data fields>}, <No data fields>}, m_ptr = 0x7ffff7591930}, convert = true, none = true}, {name = 0x5555583212b0 "num_threads", descr = 0x5555586cb390 "-1",
              value = {<pybind11::detail::object_api<pybind11::handle>> = {<pybind11::detail::pyobject_tag> = {<No data fields>}, <No data fields>}, m_ptr = 0x7ffff75918f0}, convert = true, none = true}},
          impl = 0x7ffe641a1580 <pybind11::cpp_function::initialize<pybind11::cpp_function::initialize<pybind11::object, Index<float, float>, pybind11::object, unsigned long, int, pybind11::name, pybind11::is_method, pybind11::sibling, pybind11::arg, pybind11::arg_v, pybind11::arg_v>(pybind11::object (Index<float, float>::*)(pybind11::object, unsigned long, int), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&, pybind11::arg const&, pybind11::arg_v const&, pybind11::arg_v const&)::{lambda(Index<float, float>*, pybind11::object, unsigned long, int)#1}, pybind11::object, Index<float, float>*, pybind11::object, unsigned long, int, pybind11::name, pybind11::is_method, pybind11::sibling, pybind11::arg, pybind11::arg_v, pybind11::arg_v>(pybind11::cpp_function::initialize<pybind11::object, Index<float, float>, pybind11::object, unsigned long, int, pybind11::name, pybind11::is_method, pybind11::sibling, pybind11::arg, pybind11::arg_v, pybind11::arg_v>(pybind11::object (Index<float, float>::*)(pybind11::object, unsigned long, int), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&, pybind11::arg const&, pybind11::arg_v const&, pybind11::arg_v const&)::{lambda(Index<float, float>*, pybind11::object, unsigned long, int)#1}&&, pybind11::object (*)(Index<float, float>*, pybind11::object, unsigned long, int), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&, pybind11::arg const&, pybind11::arg_v const&, pybind11::arg_v const&)::{lambda(pybind11::detail::function_call&)#3}::_FUN(pybind11::detail::function_call&)>, data = {0x7ffe6419f250 <Index<float, float>::knnQuery_return_numpy(pybind11::object, unsigned long, int)>, 0x0, 0x0}, free_data = 0x0, policy = pybind11::return_value_policy::automatic, is_constructor = false, is_new_style_constructor = false, is_stateless = false, is_operator = false,
          is_method = true, has_args = false, has_kwargs = false, has_kw_only_args = false, prepend = false, nargs = 4, nargs_kw_only = 0, nargs_pos_only = 0, def = 0x5555580ec370, scope = {<pybind11::detail::object_api<pybind11::handle>> = {<pybind11::detail::pyobject_tag> = {<No data fields>}, <No data fields>},
            m_ptr = 0x555558b49750}, sibling = {<pybind11::detail::object_api<pybind11::handle>> = {<pybind11::detail::pyobject_tag> = {<No data fields>}, <No data fields>}, m_ptr = 0x7ffff7f92680 <_Py_NoneStruct>}, next = 0x0}
        num_args = <optimized out>
        pos_args = <optimized out>
        call = {func = @0x555558113390, args = std::vector of length 4, capacity 4 = {{<pybind11::detail::object_api<pybind11::handle>> = {<pybind11::detail::pyobject_tag> = {<No data fields>}, <No data fields>}, m_ptr = 0x7ffff37375b0},
            {<pybind11::detail::object_api<pybind11::handle>> = {<pybind11::detail::pyobject_tag> = {<No data fields>}, <No data fields>}, m_ptr = 0x7ffe6366fcc0}, {<pybind11::detail::object_api<pybind11::handle>> = {<pybind11::detail::pyobject_tag> = {<No data fields>}, <No data fields>}, m_ptr = 0x7ffff7591e10},
            {<pybind11::detail::object_api<pybind11::handle>> = {<pybind11::detail::pyobject_tag> = {<No data fields>}, <No data fields>}, m_ptr = 0x7ffff75918f0}}, args_convert = std::vector<bool> of length 4, capacity 64 = {true, true, true, true},
          args_ref = {<pybind11::handle> = {<pybind11::detail::object_api<pybind11::handle>> = {<pybind11::detail::pyobject_tag> = {<No data fields>}, <No data fields>}, m_ptr = 0x0}, <No data fields>},
          kwargs_ref = {<pybind11::handle> = {<pybind11::detail::object_api<pybind11::handle>> = {<pybind11::detail::pyobject_tag> = {<No data fields>}, <No data fields>}, m_ptr = 0x0}, <No data fields>},
          parent = {<pybind11::detail::object_api<pybind11::handle>> = {<pybind11::detail::pyobject_tag> = {<No data fields>}, <No data fields>}, m_ptr = 0x7ffff37375b0}, init_self = {<pybind11::detail::object_api<pybind11::handle>> = {<pybind11::detail::pyobject_tag> = {<No data fields>}, <No data fields>},
            m_ptr = 0x0}}
        args_to_copy = <optimized out>
        args_copied = <optimized out>
        bad_arg = false
        kwargs = {<pybind11::object> = {<pybind11::handle> = {<pybind11::detail::object_api<pybind11::handle>> = {<pybind11::detail::pyobject_tag> = {<No data fields>}, <No data fields>}, m_ptr = 0x7ffff3782d00}, <No data fields>}, <No data fields>}
        second_pass_convert = std::vector<bool> of length 0, capacity 0
        second_pass = std::vector of length 0, capacity 0
        overloaded = false
        overloads = 0x555558113390
        it = 0x555558113390
        n_args_in = 2
        parent = {<pybind11::detail::object_api<pybind11::handle>> = {<pybind11::detail::pyobject_tag> = {<No data fields>}, <No data fields>}, m_ptr = 0x7ffff37375b0}
        result = {<pybind11::detail::object_api<pybind11::handle>> = {<pybind11::detail::pyobject_tag> = {<No data fields>}, <No data fields>}, m_ptr = 0x1}
        self_value_and_holder = {inst = 0x0, index = 0, type = 0x0, vh = 0x0}
        append_note_if_missing_header_is_suspected = {<No data fields>}
#9  0x00007ffff7d43933 in ?? () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#10 0x00007ffff7d2957d in _PyObject_MakeTpCall () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#11 0x00007ffff7d42119 in ?? () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#12 0x00007ffff7d20fa4 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#13 0x00007ffff7d1ecbd in ?? () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#14 0x00007ffff7d313fe in _PyFunction_Vectorcall () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#15 0x00007ffff7d2041a in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#16 0x00007ffff7d1ecbd in ?? () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#17 0x00007ffff7d313fe in _PyFunction_Vectorcall () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#18 0x00007ffff7d41eb4 in ?? () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#19 0x00007ffff7d20fa4 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#20 0x00007ffff7d1ecbd in ?? () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#21 0x00007ffff7d313fe in _PyFunction_Vectorcall () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#22 0x00007ffff7d421f4 in ?? () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#23 0x00007ffff7d22abe in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#24 0x00007ffff7d1ecbd in ?? () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#25 0x00007ffff7d313fe in _PyFunction_Vectorcall () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#26 0x00007ffff7d20fa4 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#27 0x00007ffff7d1ecbd in ?? () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#28 0x00007ffff7d313fe in _PyFunction_Vectorcall () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#29 0x00007ffff7d201e6 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#30 0x00007ffff7d1ecbd in ?? () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#31 0x00007ffff7d313fe in _PyFunction_Vectorcall () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#32 0x00007ffff7d24821 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#33 0x00007ffff7d3125a in _PyFunction_Vectorcall () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#34 0x00007ffff7d201e6 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#35 0x00007ffff7d1ecbd in ?? () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#36 0x00007ffff7d1e681 in _PyEval_EvalCodeWithName () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#37 0x00007ffff7de2593 in PyEval_EvalCode () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#38 0x00007ffff7de985d in ?? () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#39 0x00007ffff7d31c31 in ?? () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#40 0x00007ffff7d201e6 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#41 0x00007ffff7d1ecbd in ?? () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#42 0x00007ffff7d313fe in _PyFunction_Vectorcall () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#43 0x00007ffff7d201e6 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#44 0x00007ffff7d1ecbd in ?? () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#45 0x00007ffff7d313fe in _PyFunction_Vectorcall () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#46 0x00007ffff7e0d958 in ?? () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#47 0x00007ffff7e04222 in Py_RunMain () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#48 0x00007ffff7dd5529 in Py_BytesMain () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#49 0x00007ffff7a50b25 in __libc_start_main () from /usr/lib/libc.so.6
No symbol table info available.
#50 0x000055555555504e in _start ()
No symbol table info available.

CLIP CPU

Hi,

Great project!

@shawwn converted the CLIP model weights to load on a cpu (https://github.com/shawwn/CLIP/blob/dev/clip_cpu.py). This works fine for HyperTag as long as the torch device is correctly set everywhere. However, urlretrieve() uses a Python user-agent to fetch the weights for which his server gives a 403 so the inline download fails. I'm not sure about weight hosting or licensing in general so I've not modified the UA.

Hopefully this is helpful.

opened by mszarski 3

Cannot install hypertag version 0.6.2.7

Hi,

I am trying to install and use hypertag==0.6.2.7 but it is giving me following error

ERROR: Could not find a version that satisfies the requirement torchvision<0.9.0,>=0.8.2 (from hypertag)
ERROR: No matching distribution found for torchvision<0.9.0,>=0.8.2

Full traceback error

(hypertag_env) PS C:\<some path>\Documents> pip install hypertag==0.6.2.7
Collecting hypertag==0.6.2.7
  Using cached hypertag-0.6.2.7-py3-none-any.whl (27 kB)
ERROR: Could not find a version that satisfies the requirement torchvision<0.9.0,>=0.8.2 (from hypertag)
ERROR: No matching distribution found for torchvision<0.9.0,>=0.8.2

I am using gcp windows instance. hypertag_env is just a virtual environment with following package

numpy==1.19.5
pandas==1.2.1
python-dateutil==2.8.1
pytz==2020.5
six==1.15.0

Thanks in advance for help.

opened by hiteshsom 1

Add a way to add web pages to HyperTag

Should work like this: $ hypertag add https://blog.neotree.uber.space/posts/hypertag-file-organization-made-for-humans

This should fetch all assets (HTML, CSS, JS, etc.) and budle them up for future offline access. And also store the URL.

opened by SeanPedersen 1
Add CPU / GPU toggle option

Currently things stop working if no CUDA GPU is available. This is bad. Make CUDA optional (allow CPU only usage). Looks like CLIP does not work without CUDA...

opened by SeanPedersen 1
Semantic search for text documents

Vectorize all text documents and let the user search them.

Related to #24 and #9

Just eyeballing: Glove model (average_word_embeddings_glove.6B.300d) seems to perform better than DistilBERT (stsb-distilbert-base), add some small benchmark tests with common and diverse papers and queries.

Models: https://docs.google.com/spreadsheets/d/14QplCdTCDwEmTqrn1LH4yrbKvdogK4oQvYO1K1aPR5M/edit#gid=0

opened by SeanPedersen 1
Add image search to HyperTagFS

Create a dedicated directory called "Search Images". All directories names created in "Search Images" are interpreted as search queries for image files and accordingly populated with the results.

opened by SeanPedersen 0
Add text search to HyperTagFS

Create a dedicated directory called "Search Texts". All directories names created in "Search Texts" are interpreted as search queries for text documents and accordingly populated with the results.

opened by SeanPedersen 0
Visualize the HyperTag graph
Candidates:

https://graph-tool.skewed.de/static/doc/quickstart.html

Pro: Performance (fast -> C++ wrapper)

Con: Size (big), no pip install (cuz C++)

https://github.com/networkx/networkx

Pro: well tested

https://github.com/igraph/python-igraph

Pro: Performance (fast -> C wrapper)

https://github.com/root-11/graph-theory

Pro: Size (tiny)

Con: Performance (slow?)
opened by SeanPedersen 0
HyperTagFS: Let user create directories with names as queries

Use Case: User creates a directory named: animal minus human -> directory should contain all files associated with animals minus human files.

Depends on #10 & #18

opened by SeanPedersen 0