Hello i'm trying to install the fs-onedatafs lib using anaconda, to install it inside a jupyter notebook

this is my dockerfile

FROM jupyter/minimal-notebook:ubuntu-18.04

USER $NB_UID

RUN conda install mamba -y --quiet -c conda-forge 
RUN mamba install -y  -c onedata fs.onedatafs

installation goes fine ( it installs fs.onedatafs 20.02.4 py37_0 )

but when i try to import it in a notebook i got

from fs.onedatafs import OnedataFS

ImportError                               Traceback (most recent call last)
<ipython-input-1-12d9be3ef961> in <module>
----> 1 from fs.onedatafs import OnedataFS

/opt/conda/lib/python3.7/site-packages/fs/onedatafs/__init__.py in <module>
     10 
     11 if "pytest" not in sys.modules:
---> 12     from ._onedatafs import OnedataFS, OnedataSubFS # noqa

/opt/conda/lib/python3.7/site-packages/fs/onedatafs/_onedatafs.py in <module>
     34 import six
     35 
---> 36 import onedatafs # noqa
     37 
     38 from ._util import ensure_unicode, info_to_stat, stat_to_permissions, to_ascii

/opt/conda/lib/python3.7/site-packages/onedatafs/__init__.py in <module>
----> 1 from .onedatafs import OnedataFS, Stat, Xattr

ImportError: libprotobuf.so.24: cannot open shared object file: No such file or directory

i see that this version of protobuf is installed when running the installation of the lib

libprotobuf 3.14.0 h780b84a_0 conda-forge/linux-64 2 MB

cheers Andrea

Hi,

we would like to use and push the usage of onedata in our community. For that, we wanted to integrate fs.onedatafs into our system. However, it is not clear what the preferred way is to install this library.

Installing it via PyPI leads to the following error on python3.8 but also on python3.7.

Python 3.7.10 | packaged by conda-forge | (default, Feb 19 2021, 16:07:37) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import fs.onedatafs
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/bag/miniconda3/envs/python37/lib/python3.7/site-packages/fs/onedatafs/__init__.py", line 12, in <module>
    from ._onedatafs import OnedataFS, OnedataSubFS # noqa
  File "/home/bag/miniconda3/envs/python37/lib/python3.7/site-packages/fs/onedatafs/_onedatafs.py", line 36, in <module>
    import onedatafs # noqa
ModuleNotFoundError: No module named 'onedatafs'

Is python>3.7 supported? The PyPI release is really outdated and the GitHub release is much newer. Is there a reason? We tried the latest github release but we got the same error.

Next, we tried the anaconda release, but this does not work because you don't pin protobuf or libtbb so this release is also not useable out of the box. I'm part of the conda-forge and bioconda community so we could help with the correct setup of conda builds if you like.

My general concern is that this project does not seem to be very active :( no issues, no PRs, the master branch is not updated and no CI infrastructure that runs tests. Is there any other preferred way to access OneData via python that I missed?

Thanks for OneData we like it a lot and would more tightly integrate with it but we would need to python library for that. Bjoern

Reading one large netCDF file using xarray results in errors due to wrong arguments to read:

ArgumentError: Python argument types in
    OnedataFileHandle.read(OnedataFileHandle, numpy.int64, int)
did not match C++ signature:
    read(OnedataFileHandle {lvalue}, long, unsigned long)

The self.pos value was updated before by a call to seek() with a numpy.int64.

Code to reproduce:

import numpy as np

from fs.onedatafs import OnedataFS
onedata_provider_host = "plg-cyfronet-01.datahub.egi.eu"
onedata_access_token = "MDA.."

# Create connection to Oneprovider
odfs = OnedataFS(onedata_provider_host, onedata_access_token)
h = np.int64(100)
f = odfs.open('/EGI Foundation/Untitled Folder/tas_1991_2016_NLD.csv', 'rb')
f.seek(h)
f.read(10)

The original code was doing this:

import xarray as xr
f = odfs.open('/EGI Foundation/Untitled Folder/data2/ERA5_rainfall_and_temperature.nc', 'rb')
ds = xr.open_dataset(f)