A Jupyter server based on FastAPI (Experimental)

Jupyter Server

Last update: Dec 27, 2022

Related tags

FastAPI Projects jupyverse

Overview

jupyverse

jupyverse is experimental and should not be used in place of jupyter-server, which is the official Jupyter server.

A set of FPS plugins implementing a Jupyter server.

Motivation for Experimental Server

For the motivations behind this project, please refer to this issue in the Jupyter server team compass.

Install

pip install jupyverse[jupyterlab]

Note: at this stage of development, it is preferable to install from sources (see below).

Development install

Clone this repository and install the needed plugins:

pip install -e .[jupyterlab] --no-deps
pip install -e plugins/jupyterlab
pip install -e plugins/contents
pip install -e plugins/kernels
pip install -e plugins/terminals
pip install -e plugins/nbconvert
pip install -e plugins/yjs
pip install -e plugins/auth

# you should also install the latest FPS:
pip install git+https://github.com/jupyter-server/fps

# if you want RetroLab instead of JupyterLab:
# pip install -e .[retrolab] --no-deps
# pip install -e plugins/retrolab
# ...

Usage

Without authentication

jupyverse --open-browser --authenticator.mode=noauth

This will open a browser at http://127.0.0.1:8000 by default, and load the JupyterLab front-end. You have full access to the API, without restriction.

With token authentication

jupyverse --open-browser --authenticator.mode=token

This is the default mode, and it corresponds to Jupyter Server's token-based authentication.

With user authentication

jupyverse --open-browser --authenticator.mode=user

We provide a JupyterLab extension for authentication, that you can install with:

pip install git+https://github.com/davidbrochart/jupyverse-auth

You can currently authenticate as an anonymous user, or using a GitHub account.

With collaborative editing

jupyverse --open-browser --JupyterLab.collaborative=true

This is especially interesting if you are "user-authenticated", since your will appear as the identity you chose for authentication.

Comments

Login page

Added a new plugin that registers a new endpoint for the login page.

The login page has three ways of login, GitHub, token or anonymous.

The GitHub login throws a server error when redirecting, looks like the error comes from FastAPI Users when it tries to retrieve the user information from GitHub. I'll debug tomorrow.

Regarding the login with the token, I'm not sure how it works.

opened by hbcarlos 36
Use Hatch Environments and Scripts
PR implements some of the Hatch features mentioned in #209, main ones are:

Matrix definitions for incompatible plugins

The dev environment is a matrix currently made up of the all combinations of frontends ['lab', 'retro'] and authentication plugins ['noauth', 'auth', 'fief]

This lets you have all plugins installed in their own environments, and allows for easy testing/switching between them

hatch run dev.lab-noauth:jupyverse spawns jupyverse with jupyterlab with noauth, hatch run dev.retro-fief:jupyverse is retro with fief auth, etc...

Docs environment and scripts allow for easy testing/building of docs, e.g. hatch run docs:serve

Docs were updated to explain the above.

Right now this isn't ideal as using context formatting to specify the plugins as dependencies for development didn't work, since Hatch doesn't have an easy mechanism for creating environments with editable dependency installations. So for now this is done with pre/post install commands running pip install on the required plugins.

I've started a discussion on the Hatch repo here https://github.com/pypa/hatch/discussions/516 to see if a PR allowing for the easy definition of editable local dependencies would be accepted. If it is then I'd like to wait for that. and remove the workaround calling pip install in the pre/post script sections.

Still to do:

[x] Wait for new Hatch release with https://github.com/pypa/hatch/pull/484 merged

[x] Update GitHub Workflows to use hatch environments during setup

[ ] Add Hatch template to make creation of new plugins easier/standardised

Closes https://github.com/jupyter-server/jupyverse/issues/209
opened by RobertRosca 14
Changing Build Backend (to Hatch?)
Problem

The main project and all plugins have pyproject.toml, setup.py, and setup.cfg to define various installation options, splitting up where options/dependencies can be configured

Development install instructions, CI commands, and (now) devcontainer install all explicitly call pip install ./plugins/... for each plugin, and all of them need to be updated as plugins are added/removed/renamed

Packaging requires individually building all pulgins and moving the files to dist/

Creating new plugins (at least if you want them to be consistent with the current layout) requires creating these files/copying them from an existing plugin and modifying them

Proposed Solution

Hatch has some features which would help:

pyproject.toml contains all the configurations and dependencies so they're no longer spread out

Optional project dependencies and local directory packages can be used to allow for installation with the standard optional syntax, e.g. pip install -e '.[plugins]', pip install -e '.[plugins,retro]', pip install -e '.[plugins,lab]'

Hatch has extensive build and publish configurations, which should be able to build all juypverse and plugins for a release with one command

Hatch has project templates, meaning a template can be provided which can be used to create new plugins, making it easier to set them up

In addition to that there are some other useful features:

Hatch lets you define environments with matrices, simplifying the process of testing plugins like retro/lab which cannot be installed together

Related to that, you can have scripts attached to environments, meaning that a lab and retro environment could be defined with their respective plugin s as dependencies and with serve scripts, then hatch run lab:serve and hatch run retro:serve could both be ran, making it easier to check incompatible plugins without having to run install/uninstall commands

There are a lot of other features too which could be useful, but off the top of my head those are the main ones.

Additional context

Jupyter Server, as well as a few other major projects, already use Hatch for their project management and build system.
enhancement
opened by RobertRosca 9
Use `fps_uvicorn` plugin
Description

https://github.com/jupyter-server/fps/pull/32 moved uvicorn and corresponding CLI to a dedicated plugin.

Use fps_uvicorn plugin:

use config moved from fps to fps_uvicorn

update dependencies

use logger to print link in console
opened by adriendelsalle 9

Clean new install does not work.

$ conda create -n jv
Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... done

## Package Plan ##

  environment location: /Users/bussonniermatthias/miniforge3/envs/jv



Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done

$ conda activate jv
$ conda install jupyverse fps-retrolab
Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... failed with initial frozen solve. Retrying with flexible solve.
Solving environment: ...working... failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done

## Package Plan ##

  environment location: ~/miniforge3/envs/jv

  added / updated specs:
    - fps-retrolab
    - jupyverse


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    cryptography-37.0.2        |  py310h94bb23d_0         1.2 MB  conda-forge
    libcxx-14.0.5              |       h04bba0f_1         1.3 MB  conda-forge
    libzlib-1.2.12             |       ha287fd2_1          48 KB  conda-forge
    openssl-1.1.1p             |       ha287fd2_0         1.8 MB  conda-forge
    python-3.10.5              |h71ab1a4_0_cpython        12.3 MB  conda-forge
    readline-8.1.2             |       h46ed386_0         263 KB  conda-forge
    setuptools-62.6.0          |  py310hbe9552e_0         1.3 MB  conda-forge
    zlib-1.2.12                |       ha287fd2_1          77 KB  conda-forge
    ------------------------------------------------------------
                                           Total:        18.3 MB

The following NEW packages will be INSTALLED:

  aiofiles           conda-forge/noarch::aiofiles-0.8.0-pyhd8ed1ab_0
  aiosqlite          conda-forge/noarch::aiosqlite-0.17.0-pyhd8ed1ab_0
  anyio              conda-forge/osx-arm64::anyio-3.6.1-py310hbe9552e_0
  appnope            conda-forge/noarch::appnope-0.1.3-pyhd8ed1ab_0
  argon2-cffi        conda-forge/noarch::argon2-cffi-21.3.0-pyhd8ed1ab_0
  argon2-cffi-bindi~ conda-forge/osx-arm64::argon2-cffi-bindings-21.2.0-py310hf8d0d8f_2
  asgiref            conda-forge/noarch::asgiref-3.5.2-pyhd8ed1ab_0
  asttokens          conda-forge/noarch::asttokens-2.0.5-pyhd8ed1ab_0
  attrs              conda-forge/noarch::attrs-21.4.0-pyhd8ed1ab_0
  babel              conda-forge/noarch::babel-2.10.3-pyhd8ed1ab_0
  backcall           conda-forge/noarch::backcall-0.2.0-pyh9f0ad1d_0
  backports          conda-forge/noarch::backports-1.0-py_2
  backports.functoo~ conda-forge/noarch::backports.functools_lru_cache-1.6.4-pyhd8ed1ab_0
  bcrypt             conda-forge/osx-arm64::bcrypt-3.2.2-py310h02f21da_0
  beautifulsoup4     conda-forge/noarch::beautifulsoup4-4.11.1-pyha770c72_0
  bleach             conda-forge/noarch::bleach-5.0.0-pyhd8ed1ab_0
  brotlipy           conda-forge/osx-arm64::brotlipy-0.7.0-py310hf8d0d8f_1004
  bzip2              conda-forge/osx-arm64::bzip2-1.0.8-h3422bc3_4
  ca-certificates    conda-forge/osx-arm64::ca-certificates-2022.6.15-h4653dfc_0
  certifi            conda-forge/osx-arm64::certifi-2022.6.15-py310hbe9552e_0
  cffi               conda-forge/osx-arm64::cffi-1.15.0-py310hf10583b_0
  charset-normalizer conda-forge/noarch::charset-normalizer-2.0.12-pyhd8ed1ab_0
  click              conda-forge/osx-arm64::click-8.1.3-py310hbe9552e_0
  colorama           conda-forge/noarch::colorama-0.4.5-pyhd8ed1ab_0
  cryptography       conda-forge/osx-arm64::cryptography-37.0.2-py310h94bb23d_0
  debugpy            conda-forge/osx-arm64::debugpy-1.6.0-py310h1105856_0
  decorator          conda-forge/noarch::decorator-5.1.1-pyhd8ed1ab_0
  defusedxml         conda-forge/noarch::defusedxml-0.7.1-pyhd8ed1ab_0
  dnspython          conda-forge/noarch::dnspython-2.2.1-pyhd8ed1ab_0
  email-validator    conda-forge/noarch::email-validator-1.1.3-pyhd8ed1ab_0
  entrypoints        conda-forge/noarch::entrypoints-0.4-pyhd8ed1ab_0
  executing          conda-forge/noarch::executing-0.8.3-pyhd8ed1ab_0
  fastapi            conda-forge/noarch::fastapi-0.75.2-pyhd8ed1ab_0
  fastapi-users      conda-forge/noarch::fastapi-users-10.1.1-pyhd8ed1ab_0
  fastapi-users-db-~ conda-forge/noarch::fastapi-users-db-sqlalchemy-4.0.3-pyhd8ed1ab_0
  flit-core          conda-forge/noarch::flit-core-3.7.1-pyhd8ed1ab_0
  fps                conda-forge/noarch::fps-0.0.9-pyhd8ed1ab_0
  fps-auth           conda-forge/noarch::fps-auth-0.0.33-pyhd8ed1ab_0
  fps-contents       conda-forge/noarch::fps-contents-0.0.33-pyhd8ed1ab_0
  fps-kernels        conda-forge/noarch::fps-kernels-0.0.33-pyhd8ed1ab_0
  fps-lab            conda-forge/noarch::fps-lab-0.0.33-pyhd8ed1ab_0
  fps-login          conda-forge/noarch::fps-login-0.0.33-pyhd8ed1ab_0
  fps-nbconvert      conda-forge/noarch::fps-nbconvert-0.0.33-pyhd8ed1ab_0
  fps-retrolab       conda-forge/noarch::fps-retrolab-0.0.33-pyhd8ed1ab_0
  fps-terminals      conda-forge/noarch::fps-terminals-0.0.30-pyhd8ed1ab_1
  fps-uvicorn        conda-forge/noarch::fps-uvicorn-0.0.3-pyhd8ed1ab_0
  fps-yjs            conda-forge/noarch::fps-yjs-0.0.33-pyhd8ed1ab_0
  greenlet           conda-forge/osx-arm64::greenlet-1.1.2-py310h1105856_2
  h11                conda-forge/noarch::h11-0.12.0-pyhd8ed1ab_0
  h2                 conda-forge/noarch::h2-4.1.0-pyhd8ed1ab_0
  hpack              conda-forge/noarch::hpack-4.0.0-pyh9f0ad1d_0
  httpcore           conda-forge/noarch::httpcore-0.14.7-pyhd8ed1ab_1
  httpx              conda-forge/osx-arm64::httpx-0.22.0-py310hbe9552e_0
  httpx-oauth        conda-forge/noarch::httpx-oauth-0.7.0-pyhd8ed1ab_0
  hyperframe         conda-forge/noarch::hyperframe-6.0.1-pyhd8ed1ab_0
  idna               conda-forge/noarch::idna-3.3-pyhd8ed1ab_0
  importlib-metadata conda-forge/osx-arm64::importlib-metadata-4.11.4-py310hbe9552e_0
  importlib_metadata conda-forge/noarch::importlib_metadata-4.11.4-hd8ed1ab_0
  importlib_resourc~ conda-forge/noarch::importlib_resources-5.8.0-pyhd8ed1ab_0
  ipykernel          conda-forge/noarch::ipykernel-6.15.0-pyh736e0ef_0
  ipython            conda-forge/osx-arm64::ipython-8.4.0-py310hbe9552e_0
  ipython_genutils   conda-forge/noarch::ipython_genutils-0.2.0-py_1
  jedi               conda-forge/osx-arm64::jedi-0.18.1-py310hbe9552e_1
  jinja2             conda-forge/noarch::jinja2-3.1.2-pyhd8ed1ab_1
  json5              conda-forge/noarch::json5-0.9.5-pyh9f0ad1d_0
  jsonschema         conda-forge/noarch::jsonschema-4.6.0-pyhd8ed1ab_0
  jupyter_client     conda-forge/noarch::jupyter_client-7.3.2-pyhd8ed1ab_0
  jupyter_core       conda-forge/osx-arm64::jupyter_core-4.10.0-py310hbe9552e_0
  jupyter_server     conda-forge/noarch::jupyter_server-1.18.0-pyhd8ed1ab_1
  jupyterlab         conda-forge/noarch::jupyterlab-3.4.3-pyhd8ed1ab_0
  jupyterlab_pygmen~ conda-forge/noarch::jupyterlab_pygments-0.2.2-pyhd8ed1ab_0
  jupyterlab_server  conda-forge/noarch::jupyterlab_server-2.14.0-pyhd8ed1ab_1
  jupyverse          conda-forge/noarch::jupyverse-0.0.33-pyhd8ed1ab_0
  libcxx             conda-forge/osx-arm64::libcxx-14.0.5-h04bba0f_1
  libffi             conda-forge/osx-arm64::libffi-3.4.2-h3422bc3_5
  libsodium          conda-forge/osx-arm64::libsodium-1.0.18-h27ca646_1
  libzlib            conda-forge/osx-arm64::libzlib-1.2.12-ha287fd2_1
  makefun            conda-forge/noarch::makefun-1.13.1-pyhd8ed1ab_0
  markupsafe         conda-forge/osx-arm64::markupsafe-2.1.1-py310hf8d0d8f_1
  matplotlib-inline  conda-forge/noarch::matplotlib-inline-0.1.3-pyhd8ed1ab_0
  mistune            conda-forge/osx-arm64::mistune-0.8.4-py310he2143c4_1005
  nbclassic          conda-forge/noarch::nbclassic-0.3.7-pyhd8ed1ab_0
  nbclient           conda-forge/noarch::nbclient-0.6.4-pyhd8ed1ab_1
  nbconvert          conda-forge/noarch::nbconvert-6.5.0-pyhd8ed1ab_0
  nbconvert-core     conda-forge/noarch::nbconvert-core-6.5.0-pyhd8ed1ab_0
  nbconvert-pandoc   conda-forge/noarch::nbconvert-pandoc-6.5.0-pyhd8ed1ab_0
  nbformat           conda-forge/noarch::nbformat-5.4.0-pyhd8ed1ab_0
  ncurses            conda-forge/osx-arm64::ncurses-6.3-h07bb92c_1
  nest-asyncio       conda-forge/noarch::nest-asyncio-1.5.5-pyhd8ed1ab_0
  notebook           conda-forge/noarch::notebook-6.4.12-pyha770c72_0
  notebook-shim      conda-forge/noarch::notebook-shim-0.1.0-pyhd8ed1ab_0
  openssl            conda-forge/osx-arm64::openssl-1.1.1p-ha287fd2_0
  packaging          conda-forge/noarch::packaging-21.3-pyhd8ed1ab_0
  pandoc             pkgs/main/osx-arm64::pandoc-2.12-hca03da5_0
  pandocfilters      conda-forge/noarch::pandocfilters-1.5.0-pyhd8ed1ab_0
  parso              conda-forge/noarch::parso-0.8.3-pyhd8ed1ab_0
  passlib            conda-forge/noarch::passlib-1.7.4-pyh9f0ad1d_0
  pexpect            conda-forge/noarch::pexpect-4.8.0-pyh9f0ad1d_2
  pickleshare        conda-forge/noarch::pickleshare-0.7.5-py_1003
  pip                conda-forge/noarch::pip-22.1.2-pyhd8ed1ab_0
  pluggy             conda-forge/osx-arm64::pluggy-1.0.0-py310hbe9552e_3
  prometheus_client  conda-forge/noarch::prometheus_client-0.14.1-pyhd8ed1ab_0
  prompt-toolkit     conda-forge/noarch::prompt-toolkit-3.0.29-pyha770c72_0
  psutil             conda-forge/osx-arm64::psutil-5.9.1-py310h02f21da_0
  ptyprocess         conda-forge/noarch::ptyprocess-0.7.0-pyhd3deb0d_0
  pure_eval          conda-forge/noarch::pure_eval-0.2.2-pyhd8ed1ab_0
  pycparser          conda-forge/noarch::pycparser-2.21-pyhd8ed1ab_0
  pydantic           conda-forge/osx-arm64::pydantic-1.9.1-py310h02f21da_0
  pygments           conda-forge/noarch::pygments-2.12.0-pyhd8ed1ab_0
  pyjwt              conda-forge/noarch::pyjwt-2.4.0-pyhd8ed1ab_0
  pyopenssl          conda-forge/noarch::pyopenssl-22.0.0-pyhd8ed1ab_0
  pyparsing          conda-forge/noarch::pyparsing-3.0.9-pyhd8ed1ab_0
  pyrsistent         conda-forge/osx-arm64::pyrsistent-0.18.1-py310hf8d0d8f_1
  pysocks            conda-forge/osx-arm64::pysocks-1.7.1-py310hbe9552e_5
  python             conda-forge/osx-arm64::python-3.10.5-h71ab1a4_0_cpython
  python-dateutil    conda-forge/noarch::python-dateutil-2.8.2-pyhd8ed1ab_0
  python-fastjsonsc~ conda-forge/noarch::python-fastjsonschema-2.15.3-pyhd8ed1ab_0
  python-multipart   conda-forge/noarch::python-multipart-0.0.5-py_0
  python_abi         conda-forge/osx-arm64::python_abi-3.10-2_cp310
  pytz               conda-forge/noarch::pytz-2022.1-pyhd8ed1ab_0
  pyzmq              conda-forge/osx-arm64::pyzmq-22.3.0-py310h5cfa1c3_2
  readline           conda-forge/osx-arm64::readline-8.1.2-h46ed386_0
  requests           conda-forge/noarch::requests-2.28.0-pyhd8ed1ab_1
  retrolab           conda-forge/noarch::retrolab-0.3.21-pyhd8ed1ab_0
  rfc3986            conda-forge/noarch::rfc3986-1.5.0-pyhd8ed1ab_0
  send2trash         conda-forge/noarch::send2trash-1.8.0-pyhd8ed1ab_0
  setuptools         conda-forge/osx-arm64::setuptools-62.6.0-py310hbe9552e_0
  shellingham        conda-forge/noarch::shellingham-1.4.0-pyh44b312d_0
  six                conda-forge/noarch::six-1.16.0-pyh6c4a22f_0
  sniffio            conda-forge/osx-arm64::sniffio-1.2.0-py310hbe9552e_3
  soupsieve          conda-forge/noarch::soupsieve-2.3.1-pyhd8ed1ab_0
  sqlalchemy         conda-forge/osx-arm64::sqlalchemy-1.4.39-py310h02f21da_0
  sqlite             conda-forge/osx-arm64::sqlite-3.38.5-h40dfcc0_0
  stack_data         conda-forge/noarch::stack_data-0.3.0-pyhd8ed1ab_0
  starlette          conda-forge/noarch::starlette-0.17.1-pyhd8ed1ab_0
  terminado          conda-forge/osx-arm64::terminado-0.15.0-py310hbe9552e_0
  tinycss2           conda-forge/noarch::tinycss2-1.1.1-pyhd8ed1ab_0
  tk                 conda-forge/osx-arm64::tk-8.6.12-he1e0b03_0
  toml               conda-forge/noarch::toml-0.10.2-pyhd8ed1ab_0
  tornado            conda-forge/osx-arm64::tornado-6.1-py310hf8d0d8f_3
  traitlets          conda-forge/noarch::traitlets-5.3.0-pyhd8ed1ab_0
  typer              conda-forge/noarch::typer-0.4.1-pyhd8ed1ab_0
  typing-extensions  conda-forge/noarch::typing-extensions-4.2.0-hd8ed1ab_1
  typing_extensions  conda-forge/noarch::typing_extensions-4.2.0-pyha770c72_1
  tzdata             conda-forge/noarch::tzdata-2022a-h191b570_0
  urllib3            conda-forge/noarch::urllib3-1.26.9-pyhd8ed1ab_0
  uvicorn            conda-forge/osx-arm64::uvicorn-0.17.6-py310hbe9552e_1
  wcwidth            conda-forge/noarch::wcwidth-0.2.5-pyh9f0ad1d_2
  webencodings       conda-forge/noarch::webencodings-0.5.1-py_1
  websocket-client   conda-forge/noarch::websocket-client-1.3.3-pyhd8ed1ab_0
  websockets         conda-forge/osx-arm64::websockets-10.3-py310hf8d0d8f_0
  wheel              conda-forge/noarch::wheel-0.37.1-pyhd8ed1ab_0
  xz                 conda-forge/osx-arm64::xz-5.2.5-h642e427_1
  zeromq             conda-forge/osx-arm64::zeromq-4.3.4-hbdafb3b_1
  zipp               conda-forge/noarch::zipp-3.8.0-pyhd8ed1ab_0
  zlib               conda-forge/osx-arm64::zlib-1.2.12-ha287fd2_1


Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done

$ jupyverse --open-browser --authenticator.mode=noauth
[W 2022-06-27 11:51:09 uvicorn.error] Current configuration will not reload as not all conditions are met,please refer to documentation.
[I 2022-06-27 11:51:09 fps] Loading server configuration
[I 2022-06-27 11:51:09 fps] Loading names from plugin package(s) {'fps_auth', 'fps_lab', 'fps_uvicorn', 'fps_retrolab'}
[I 2022-06-27 11:51:09 fps] Registering name 'Lab' for plugins package 'fps_lab'
[I 2022-06-27 11:51:09 fps] Registering name 'RetroLab' for plugins package 'fps_retrolab'
[I 2022-06-27 11:51:09 fps] Registering name 'authenticator' for plugins package 'fps_auth'
[I 2022-06-27 11:51:09 fps] Registering name 'uvicorn' for plugins package 'fps_uvicorn'
[I 2022-06-27 11:51:09 fps] Loading configurations from plugin package(s) {'fps_auth', 'fps_lab', 'fps_uvicorn', 'fps_retrolab'}
[I 2022-06-27 11:51:09 fps] Registering configuration model for 'Lab'
[I 2022-06-27 11:51:09 fps] Registering configuration model for 'RetroLab'
[I 2022-06-27 11:51:09 fps] Registering configuration model for 'authenticator'
[I 2022-06-27 11:51:09 fps] Registering configuration model for 'uvicorn'
Traceback (most recent call last):
  File "/Users/bussonniermatthias/miniforge3/envs/jv/bin/jupyverse", line 10, in <module>
    sys.exit(app())
  File "/Users/bussonniermatthias/miniforge3/envs/jv/lib/python3.10/site-packages/typer/main.py", line 214, in __call__
    return get_command(self)(*args, **kwargs)
  File "/Users/bussonniermatthias/miniforge3/envs/jv/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Users/bussonniermatthias/miniforge3/envs/jv/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Users/bussonniermatthias/miniforge3/envs/jv/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/bussonniermatthias/miniforge3/envs/jv/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users/bussonniermatthias/miniforge3/envs/jv/lib/python3.10/site-packages/typer/main.py", line 500, in wrapper
    return callback(**use_params)  # type: ignore
  File "/Users/bussonniermatthias/miniforge3/envs/jv/lib/python3.10/site-packages/fps_uvicorn/cli.py", line 133, in start
    uvicorn.run(
  File "/Users/bussonniermatthias/miniforge3/envs/jv/lib/python3.10/site-packages/uvicorn/main.py", line 463, in run
    server.run()
  File "/Users/bussonniermatthias/miniforge3/envs/jv/lib/python3.10/site-packages/uvicorn/server.py", line 60, in run
    return asyncio.run(self.serve(sockets=sockets))
  File "/Users/bussonniermatthias/miniforge3/envs/jv/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/Users/bussonniermatthias/miniforge3/envs/jv/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
  File "/Users/bussonniermatthias/miniforge3/envs/jv/lib/python3.10/site-packages/uvicorn/server.py", line 67, in serve
    config.load()
  File "/Users/bussonniermatthias/miniforge3/envs/jv/lib/python3.10/site-packages/uvicorn/config.py", line 458, in load
    self.loaded_app = import_from_string(self.app)
  File "/Users/bussonniermatthias/miniforge3/envs/jv/lib/python3.10/site-packages/uvicorn/importer.py", line 24, in import_from_string
    raise exc from None
  File "/Users/bussonniermatthias/miniforge3/envs/jv/lib/python3.10/site-packages/uvicorn/importer.py", line 21, in import_from_string
    module = importlib.import_module(module_str)
  File "/Users/bussonniermatthias/miniforge3/envs/jv/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/Users/bussonniermatthias/miniforge3/envs/jv/lib/python3.10/site-packages/fps/main.py", line 3, in <module>
    app = create_app()
  File "/Users/bussonniermatthias/miniforge3/envs/jv/lib/python3.10/site-packages/fps/app.py", line 284, in create_app
    _load_routers(app)
  File "/Users/bussonniermatthias/miniforge3/envs/jv/lib/python3.10/site-packages/fps/app.py", line 158, in _load_routers
    pm = _get_pluggin_manager(HookType.ROUTER)
  File "/Users/bussonniermatthias/miniforge3/envs/jv/lib/python3.10/site-packages/fps/app.py", line 24, in _get_pluggin_manager
    pm.load_setuptools_entrypoints(hook_type.value)
  File "/Users/bussonniermatthias/miniforge3/envs/jv/lib/python3.10/site-packages/pluggy/_manager.py", line 287, in load_setuptools_entrypoints
    plugin = ep.load()
  File "/Users/bussonniermatthias/miniforge3/envs/jv/lib/python3.10/importlib/metadata/__init__.py", line 171, in load
    module = import_module(match.group('module'))
  File "/Users/bussonniermatthias/miniforge3/envs/jv/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/Users/bussonniermatthias/miniforge3/envs/jv/lib/python3.10/site-packages/fps_terminals/routes.py", line 9, in <module>
    from fps_auth.models import User  # type: ignore
ImportError: cannot import name 'User' from 'fps_auth.models' (/Users/bussonniermatthias/miniforge3/envs/jv/lib/python3.10/site-packages/fps_auth/models.py)

bug

opened by Carreau 7

Await zmq sends
asyncio zmq sends return awaitables, even though they usually return a completed Future, due to optimizations.

Otherwise, we're creating and not awaiting Futures, which can cause ordering problems.

The layering means there are now two additional awaits for a likely already-done future. I don't know the performance cost of that. But if you care to optimize the awaits, you can do:

def send_message(sock) -> Awaitable[Any]: return sock.send(...) ... awaitable = send_message(sock) f = asyncio.ensure_future(awaitable) # no-op because it's already a Future, but could be a coroutine in the future if not f.done(): await f # most of the time, get here with zero awaits

An extra await can be avoided by changing the type to ->Awaitable[Any] and return sock.send..., if that's preferable.

Found while looking at #183
opened by minrk 6
Per-user settings/workspace

Problem

In Jupyverse, settings and workspace are stored alongside the user information, in a database. When jupyverse is launched with --authenticator.mode=noauth, there is no authentication and the whole API is unprotected. Naturally, we want settings and workspace to be stored with this unique, "global user", so that users get their settings back the next time they launch jupyverse. When jupyverse is launched with --authenticator.mode=token, a token is created and a user is registered in the DB with this token as its ID. When jupyverse is stopped, the user is removed from the DB. What this means is that this is a new user each time, and settings won't persist between jupyverse launches.

Proposed Solution

Maybe each token-user should inherit the unauthenticated, "global user" settings and workspace, by copying them when the token-user is registered? That would at least guarantee some persistence for settings and workspace. Similarly, settings and workspace could be copied back to the "global user" when the token-user jupyverse is stopped? If several unauthenticated-user or token-user jupyverse are launched at the same time, this is obviously a source of conflicts, but the main use-case of these two modes are really a single user launching a single server at a time.
enhancement

opened by davidbrochart 6
Align path format for windows.
This PR makes the following modifications.

overall:

change str(pathlib.Path) to pathlib.Path.as_posix()

~convert pathlib.Path / "a" / "b" / "c" to pathlib.Path / "a/b/c"~

replace parent.parent with parents[1]

for plugins/lab/fps_lab/utils.py:

use pathlib.Path.rglob to replace glob.glob(pathlib.Path, recursive=True)

check "jupyterlab" in package

With these modifications, the following issues on the windows platform solved:

no labextension is found.

in the file browser, the path is not separated (displayed as / a\b\c /, and can't enter the parent folder).

raise error the package.json without "jupyterlab"

however, the following errors still exist, and I'm not sure they're windows specific:

--root-path doesn't change the root folder, and the root folder is still .

the pwd is always the root folder and doesn't change following the editing file.

To sum up, the unix like path is more friendly even in the windows platform, but str(pahtlib.Path) will convert the path as the native format.
opened by specter119 5
Load/auto-save document from the back-end using y-py
This PR updates jupyverse with the following PRs in JupyterLab:

https://github.com/jupyterlab/jupyterlab/pull/11599

https://github.com/jupyterlab/jupyterlab/pull/12360
opened by davidbrochart 5
Remove Podman specific `runArgs`

After #207 was merged I went to check if codespaces were working and there was an error as I left a podman-only command run argument in the configuration.

In the docs it's recommended to use this configuration when running with rootless podman, but it isn't compatible with docker, which most people (and the online based codespaces) use, so this patch comments it out and adds an explanation.

opened by RobertRosca 4
Moves the frontend generic configuration to a new plugin

cc @davidbrochart

This PR moves the generic frontend configuration to a new plugin.

The idea is that another fps-base app could reuse the kernel plugin without depending on the fps-lab plugin, being able to implement a new frontend that reuses the kernel plugin.

opened by hbcarlos 3
[Yjs] Stateful file manager for yjs
Problem

All file changes are stored in the same db file, This can lead to performance problems (https://github.com/jupyter-server/jupyter_server_ydoc/issues/49)

Unedited files should be treated as static and previous collaboration records deleted (this means that the ydoc record for a file should be temporary), Otherwise, the user may rename the file and create a new one with the same name and then have problems

For example, if you create a new ipynb (Untitled.ipynb), write something and rename it, and then create another ipynb (Untitled.ipynb), the new Untitled.ipynb will not open on the front end...

Proposed Solution

A file open and close related management should be added, yjs websocket connection can be seen as a file open, the crdt algorithm to solve the conflict of multiple users editing at the same time, when no user open the file, the file should be returned to "non-collaborative state"

This means that the life cycle of the file editing db should be bound to the user's access.

Also, I haven't figured out how to handle it when the service restarts or crashes, so maybe it needs an elegant startup to go with...

Additional context

This problem may also be related to the horizontal scaling of the jupyverse
enhancement
opened by Wh1isper 3
FileID pluggin

Problem

The jupyter-server FileID extension is soon going to be used in JupyterLab.

Proposed Solution

Create the equivalent service as a Jupyverse plugin.
enhancement

opened by davidbrochart 0
Contents root_dir

Problem

Jupyverse currently serves contents relatively to the directory it was launched in. We should have a notion of root directory, which gives access to the file system in this directory and below.

Proposed Solution

Add root_dir to the contents plugin configuration.
enhancement

opened by davidbrochart 5
LSP support

Problem

LSP is now in JupyterLab core.

Proposed Solution

Jupyverse should support it. Should it be implemented here as a plugin, or in https://github.com/jupyter-lsp/jupyterlab-lsp/tree/master/python_packages/jupyter_lsp ?
enhancement

opened by davidbrochart 0
Jupyverse may take a different path than JupyterHub for a multi-user scenario?

From this comment (and below): https://github.com/jupyter-server/team-compass/issues/11#issuecomment-1193653603

We might scale Jupyverse differently

Interesting...

so that it remains the only server.

serving what... what would be the entity you are spawning? Kernels? Something else?

And @davidbrochart suggested continuing the conversation here :wink:

opened by damianavila 3
Non-blocking nbconvert handler

Problem

Currently, the nbconvert handler makes a blocking call, which can hurt the server performances.

Proposed Solution

We should run nbconvert in a separate thread, possibly using anyio.to_thread's run_sync, or wait for nbconvert to get an async API.
enhancement

opened by davidbrochart 0