This ~~WIP~~ PR (fixes #53) adds support for optionally adding LDAP into the login and registration procedure by looking the user up in the LDAP directory to decide whether to proceed with the login/registration.

While the login code is tested and working, the registration code is untested.

A question that was brought up in #53 is how to deal with a user that is logged in but whose access in LDAP is then revoked. I am unsure how other applications do this. My guess is that they wait until the session of the user expires and the user has to reauthenticate. But since tokens appear to live for ~30 days this might not work. The most elegant solution would probably be to cache the result of the user lookup for some time and use this on every authenticated API path (as suggested in #53).

~~Edit: Once this PR is approved, I will add documentation on how to use this PR.~~

Edit 2: Instructions for running this PR Edit 3: Further document the cache_ttl setting

Make sure you have python-ldap installed and can successfully import ldap. Then, if you use the easy config, add the following to your config:

; [...]
; Regular etesync configuration

[ldap]
bind_dn = <Your LDAP "user" to bind as. See Note 1>
bind_pw = <The password to authenticate as your bind user>
; Or if you have the password in a file:
; bind_pw_file = /path/to/the/file.txt
server = <The URL to your LDAP server>
search_base = <Your search base>
filter = <Your LDAP filter query. See Note 2>
; In case a cache TTL of 1 hour is too short for you, set `cache_ttl` to the preferred
; amount of hours a cache entry should be viewed as valid:
; cache_ttl = 5

With this config, I am able to make the EteSync server check with my LDAP server if a user should be able to login or register. Note that if a user is allowed to login or register, the password of the LDAP user will be ignored. This LDAP patch is nothing more than an additional check before the actual authentication.

A successful LDAP check will be cached, if not configured (correctly), for one hour, after which the LDAP query will be performed again.

Note 1: This PR only works with a bind user Note 2: The query must be specified. If an LDAP query returns more than one or no result, then the authentication fails. If your query needs to include the username that currently tries to perform a login or registration, you can use %%s, which will be subsituted for the used username.

Fixes #94.

I've temporarily pushed this image to djmitche/etesync-test-server:0.7.0. I'll put up a patch shortly to etebase-go to make its test use this image, to demonstrate that it works.

Etesync-server version from branch v0.3.0 works like a charm, but build from branch v0.5.0 cannot start and throws:

Watching for file changes with StatReloader
Performing system checks...

System check identified no issues (0 silenced).
Exception in thread django-main-thread:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 219, in ensure_connection
    self.connect()
  File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 200, in connect
    self.connection = self.get_new_connection(conn_params)
  File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/django/db/backends/sqlite3/base.py", line 207, in get_new_connection
    conn = Database.connect(**conn_params)
sqlite3.OperationalError: unable to open database file

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.8/site-packages/django/utils/autoreload.py", line 53, in wrapper
    fn(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/django/core/management/commands/runserver.py", line 121, in inner_run
    self.check_migrations()
  File "/usr/local/lib/python3.8/site-packages/django/core/management/base.py", line 459, in check_migrations
    executor = MigrationExecutor(connections[DEFAULT_DB_ALIAS])
  File "/usr/local/lib/python3.8/site-packages/django/db/migrations/executor.py", line 18, in __init__
    self.loader = MigrationLoader(self.connection)
  File "/usr/local/lib/python3.8/site-packages/django/db/migrations/loader.py", line 53, in __init__
    self.build_graph()
  File "/usr/local/lib/python3.8/site-packages/django/db/migrations/loader.py", line 216, in build_graph
    self.applied_migrations = recorder.applied_migrations()
  File "/usr/local/lib/python3.8/site-packages/django/db/migrations/recorder.py", line 77, in applied_migrations
    if self.has_table():
  File "/usr/local/lib/python3.8/site-packages/django/db/migrations/recorder.py", line 55, in has_table
    with self.connection.cursor() as cursor:
  File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 259, in cursor
    return self._cursor()
  File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 235, in _cursor
    self.ensure_connection()
  File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 219, in ensure_connection
    self.connect()
  File "/usr/local/lib/python3.8/site-packages/django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 219, in ensure_connection
    self.connect()
  File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 200, in connect
    self.connection = self.get_new_connection(conn_params)
  File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/django/db/backends/sqlite3/base.py", line 207, in get_new_connection
    conn = Database.connect(**conn_params)
django.db.utils.OperationalError: unable to open database file

What am I missing?

I have been trying to setup uwsgi however I run into a few issues.

I have set my etesync.ini as follows:

[uwsgi]
socket = /tmp/etesync.sock
chown-socket = ubuntu:www-data
chmod-socket = 660
vacuum = true

uid = ubuntu
chdir = /mnt/HDD/server-skeleton
home = %(chdir)/.venv
module = etesync_server.wsgi
master = true

The error I am getting right now in the log file when I restart uwsgi (sudo service uwsgi restart) is: Fatal Python error: Py_Initialize: Unable to get the locale encoding ImportError: No module named 'encodings' (I did google the error however I was not able to solve the issue)

I am also not sure, once I get the uwsgi working how do I start it? In my etesync venv home folder with: uwsgi --http :8000 --module etesync_server.wsgi?

Thanks for your help, it is much appreciated.

Hi, I'm setting up a new phone and the android app is not working, however on my current phone I don't get any kind of error when using the etesync application. Posting on the server repo because this is the error I'm getting on the server with the debug flag as true.

Not sure what's wrong, can you help with it?

Full log of one of the collections:

Internal Server Error: /api/v1/collection/tqw-R3i_3-Zz9xubJjB9gqv-u0CHLsNR/item/
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.9/site-packages/django/core/handlers/base.py", line 179, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
    return view_func(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/rest_framework/viewsets.py", line 114, in view
    return self.dispatch(request, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/rest_framework/views.py", line 505, in dispatch
    response = self.handle_exception(exc)
  File "/usr/local/lib/python3.9/site-packages/rest_framework/views.py", line 465, in handle_exception
    self.raise_uncaught_exception(exc)
  File "/usr/local/lib/python3.9/site-packages/rest_framework/views.py", line 476, in raise_uncaught_exception
    raise exc
  File "/usr/local/lib/python3.9/site-packages/rest_framework/views.py", line 502, in dispatch
    response = handler(request, *args, **kwargs)
  File "./django_etebase/views.py", line 310, in list
    "data": serializer.data,
  File "/usr/local/lib/python3.9/site-packages/rest_framework/serializers.py", line 760, in data
    ret = super().data
  File "/usr/local/lib/python3.9/site-packages/rest_framework/serializers.py", line 260, in data
    self._data = self.to_representation(self.instance)
  File "/usr/local/lib/python3.9/site-packages/rest_framework/serializers.py", line 677, in to_representation
    return [
  File "/usr/local/lib/python3.9/site-packages/rest_framework/serializers.py", line 678, in <listcomp>
    self.child.to_representation(item) for item in iterable
  File "/usr/local/lib/python3.9/site-packages/rest_framework/serializers.py", line 529, in to_representation
    ret[field.field_name] = field.to_representation(attribute)
  File "/usr/local/lib/python3.9/site-packages/rest_framework/serializers.py", line 529, in to_representation
    ret[field.field_name] = field.to_representation(attribute)
  File "/usr/local/lib/python3.9/site-packages/rest_framework/relations.py", line 533, in to_representation
    return [
  File "/usr/local/lib/python3.9/site-packages/rest_framework/relations.py", line 534, in <listcomp>
    self.child_relation.to_representation(value)
  File "./django_etebase/serializers.py", line 119, in to_representation
    with open(obj.chunkFile.path, "rb") as f:
FileNotFoundError: [Errno 2] No such file or directory: '/data/media/user_2/tqw-R3i_3-Zz9xubJjB9gqv-u0CHLsNR/si/vSh_iSBHJbUqqtEW1YrNJolBGQny0HWc97-yjgWhk'
[pid: 40|app: 0|req: 2/2] 172.17.0.1 () {42 vars in 721 bytes} [Mon Nov 30 23:52:04 2020] GET /api/v1/collection/tqw-R3i_3-Zz9xubJjB9gqv-u0CHLsNR/item/? => generated 14817 bytes in 361 msecs (HTTP/1.0 500) 6 headers in 202 bytes (1 switches on core 0)

I have an openLDAP instance on my server running against which most of my services authenticate. EteSync doesn't support this. As such, I patched the EteSync server to also authenticate against an LDAP server.

With this issue I wanted to ask whether there is demand for such a feature. If there is, I would create a PR with my changes.

This script will create an etesync user, install etesync to /opt/etesync and add a service to systemd while enabling it. It should be a good start but further configuration is recommended (for example, limiting the allowed hosts that etesync will be authorized on)

# Set needed variables
ETESYNC_HOME=/opt/etesync;
ETESYNC_GITHUB=https://github.com/etesync/server.git;
ETESYNC_APP_FOLDER=$ETESYNC_HOME/server;

# Setup etesync user 
sudo groupadd etesync;
sudo useradd -m -d $ETESYNC_HOME etesync -g etesync;
sudo mkdir -p $ETESYNC_HOME;
sudo chown -R etesync:etesync $ETESYNC_HOME;

# Install Etesync
sudo apt install -y python3-venv git;
sudo -u etesync bash << EOF
    git clone $ETESYNC_GITHUB $ETESYNC_APP_FOLDER;
    python3 -m venv $ETESYNC_APP_FOLDER/venv;
    . $ETESYNC_APP_FOLDER/venv/bin/activate;
    python3 -m pip install wheel;
    python3 -m pip install gunicorn;
    python3 -m pip install -r $ETESYNC_APP_FOLDER/requirements.txt;
    cp $ETESYNC_APP_FOLDER/etesync-server.ini.example $ETESYNC_APP_FOLDER/etesync-server.ini;
    sed -i 's/allowed_host1 = example.com/allowed_host1 = */g' $ETESYNC_APP_FOLDER/etesync-server.ini;
    sed -i "s@secret_file = secret.txt@secret_file = $ETESYNC_APP_FOLDER/secret.txt@g" $ETESYNC_APP_FOLDER/etesync-server.ini;
    python3 $ETESYNC_APP_FOLDER/manage.py migrate;
EOF

# Setup static files
sudo mkdir -p /srv/http/etesync_server/static;
sudo chown -R etesync /srv/http/etesync_server;
sudo -u etesync bash << EOF
    ln -s /srv/http/etesync_server/static $ETESYNC_APP_FOLDER/static;
    python3 $ETESYNC_APP_FOLDER/manage.py collectstatic;
EOF

# Create the Systemd Service
sudo tee /lib/systemd/system/etesync.service > /dev/null << EOF
[Unit]
Description=Etesync gunicorn server

[Service]
User=etesync
Group=etesync
WorkingDirectory=$ETESYNC_APP_FOLDER
ExecStart=$ETESYNC_APP_FOLDER/venv/bin/gunicorn -b 0.0.0.0 etesync_server.wsgi:application
#Restart=always
KillSignal=SIGQUIT
Type=notify
NotifyAccess=all

[Install]
WantedBy=multi-user.target

EOF

# Enable and Start the service
sudo systemctl daemon-reload;
sudo systemctl enable etesync.service;
sudo systemctl start etesync.service;

TODO: Create a pull request to merge it into the repo

TODO: Create an update script to complement this one as well

This is not a feature request but a hint for anyone else who might be in the same situation as me:

I recently wanted to run the master branch of the EteSync server on my NixOS unstable host. However, whenever I wanted to log in or register, I got returned a 422 Unprocessable Entity. After some digging, I found out that NixOS unstable ships FastAPI version 0.70.0, while the EteSync server requires 0.63.0. This means that FastAPI has made a change somewhere between 0.63.0 and 0.70.0 that break the EteSync server, which meant that I have to downgrade the FastAPI version on NixOS.

Hey folks,

Thanks for this great project, I'm excited to set up my own etebase on a Raspberry 3 B+ running 5.10.36-1-MANJARO-ARM! Followed the docs for setting it up running on port 8001 and managed to connect via the etesync app on my google-free phone.

The issue is that the app server doesn't seem to see the static path specified in the config file. It keeps logging errors such as

INFO:     192.168.178.xxx:51968 - "GET /static/admin/css/responsive.css HTTP/1.1" 404 Not Found
INFO:     192.168.178.xxx:51978 - "GET /static/admin/js/prepopulate_init.js HTTP/1.1" 404 Not Found
INFO:     192.168.178.xxx:51976 - "GET /static/admin/js/change_form.js HTTP/1.1" 404 Not Found
INFO:     192.168.178.xxx:51974 - "GET /static/admin/img/icon-addlink.svg HTTP/1.1" 404 Not Found

meaning that the admin page works but looks very black and white. However, the requested files are in the static folder which has read permissions for everyone. The entry in my etebase_nginx.conf reads

location /static/ {
        alias /home/user/etebase/static/ ; # Project's static files
    }

Any idea what could be the trouble? Let me know if you need more info.

Thanks and cheers!

Hi, I'm facing some issues making eteSync server working on my VPS. Installation worked well. Here my settings: Apache virtual host (ssl):

<VirtualHost *:443>

ServerAdmin [email protected] 
ServerName xxx.xxxxxxx.xx 

<Location "/">
ProxyPass        http://localhost:8000/ retry=0
ProxyPassReverse http://localhost:8000/
RequestHeader    set X-Script-Name /
</Location>

Include /etc/letsencrypt/options-ssl-apache.conf
SSLCertificateFile /etc/letsencrypt/live/xxx/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/xxx/privkey.pem

</VirtualHost>
</IfModule>

etesync-server.ini:

secret_file = secret.txt
debug = False

[allowed_hosts]
allowed_hosts = localhost

[database]
engine = django.db.backends.sqlite3
name = db.sqlite3

Then, when typing the command ./manage.py runserver localhost:8000 everything looks fine, but when I try to access to the web page xxx.xxxxxxxx.xx (the ServerName set up in my Apache's VirtualHost), I have a lovely "Bad Request (400)" error... I can however access to the "admin" part of it through the http://localhost:8000 address directly on my VPS (but with no css loaded...)

Any idea of where the mistake is ? Thanks a lot !

Romain

Any admin can change a user's password in the admin panel:

This is dangerous, as anyone with admin perms can gain access to user data (albeit noticeable for the end user.) Also AFAIK there's no other way to change it on self-hosted servers.

When editing contacts at pim.etesync.com, sometimes I see this error in the logs. The gui does not indicate a problem, but modified values are not persisted reliably.

ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/sqlite3/base.py", line 423, in execute
    return Database.Cursor.execute(self, query, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
sqlite3.OperationalError: database is locked

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 419, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 270, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 124, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/trustedhost.py", line 34, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 92, in __call__
    await self.simple_response(scope, receive, send, request_headers=headers)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 147, in simple_response
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "/usr/local/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
    raise e
  File "/usr/local/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 706, in __call__
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 66, in app
    response = await func(request)
               ^^^^^^^^^^^^^^^^^^^
  File "/etebase/etebase_server/fastapi/msgpack.py", line 74, in custom_route_handler
    return await route_handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 235, in app
    raw_response = await run_endpoint_function(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 163, in run_endpoint_function
    return await run_in_threadpool(dependant.call, **values)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/starlette/concurrency.py", line 41, in run_in_threadpool
    return await anyio.to_thread.run_sync(func, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/anyio/to_thread.py", line 31, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
    return await future
           ^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 867, in run
    result = context.run(func, *args)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/etebase/etebase_server/fastapi/db_hack.py", line 25, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/etebase/etebase_server/fastapi/db_hack.py", line 25, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/etebase/etebase_server/fastapi/routers/collection.py", line 588, in item_batch
    return item_bulk_common(data, user, stoken, collection_uid, validate_etag=False, background_tasks=background_tasks)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/etebase/etebase_server/fastapi/routers/collection.py", line 484, in item_bulk_common
    item_create(item, collection_object, validate_etag)
  File "/etebase/etebase_server/fastapi/routers/collection.py", line 402, in item_create
    current_revision.save()
  File "/usr/local/lib/python3.11/site-packages/django/db/models/base.py", line 739, in save
    self.save_base(using=using, force_insert=force_insert,
  File "/usr/local/lib/python3.11/site-packages/django/db/models/base.py", line 776, in save_base
    updated = self._save_table(
              ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/base.py", line 858, in _save_table
    updated = self._do_update(base_qs, using, pk_val, values, update_fields,
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/base.py", line 912, in _do_update
    return filtered._update(values) > 0
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 802, in _update
    return query.get_compiler(self.db).execute_sql(CURSOR)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1559, in execute_sql
    cursor = super().execute_sql(result_type)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1175, in execute_sql
    cursor.execute(sql, params)
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 66, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
    return executor(sql, params, many, context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 79, in _execute
    with self.db.wrap_database_errors:
  File "/usr/local/lib/python3.11/site-packages/django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/sqlite3/base.py", line 423, in execute
    return Database.Cursor.execute(self, query, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
django.db.utils.OperationalError: database is locked

I have two users, admin and an unprivileged user. Admin has no user data and is only for user management. Unprivleged user is logged in to my phone and the web ui. At some point I had two tabs open in the UI.

Here's what I have deployed:

  etebase:
    image: victorrds/etesync:alpine
    container_name: etebase
    networks:
      - web
    volumes:
      - etebase-data:/data
    environment:
      SERVER: http
      SUPER_USER: admin
      SUPER_PASS: ${ADMIN_PASS}
    restart: unless-stopped

REPOSITORY          TAG       IMAGE ID       CREATED         SIZE
victorrds/etesync   alpine    53c4c139033b   2 weeks ago     217MB

Hello,

I’ve noticed on update that now dev-python/watchgod is being masked. It will be removed on 2022-12⁻04. They were saying it was only used by dev-python/uvicorn and that they have replaced it by dev-python/watchfiles.

What should I do for etebase server? Should I add watchgod on my own overlay? Should we work on etebase switching on it?

They have linked a bug report for this #879515.

Their original comment about the package mask:

# Michał Górny <[email protected]> (2022-11-04)
# NIH version of dev-python/watchdog that was used by dev-python/uvicorn
# only, and was replaced by rustified dev-python/watchfiles.
# Removal on 2022-12-04.  Bug #879515.

If I sign out of the Android app, and attempt to sign back in, I receive endless file not found errors, and the Android Etesync App reports Calendar and Task sync failed.

INFO:     10.40.40.2:47438 - "GET /api/v1/collection/yC0aqctkmG1fP6L6WtgAoIBsCXKbopTD/item/ HTTP/1.1" 200 OK
INFO:     10.40.40.2:47436 - "GET /api/v1/collection/HgAHqCikBnmWDRFr3f9Wq1ict3joXG4W/item/ HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/opt/etebase/.venv/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 372, in run_asgi
    result = await app(self.scope, self.receive, self.send)
  File "/opt/etebase/.venv/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 75, in __call__
    return await self.app(scope, receive, send)
  File "/opt/etebase/.venv/lib/python3.10/site-packages/fastapi/applications.py", line 261, in __call__
    await super().__call__(scope, receive, send)
  File "/opt/etebase/.venv/lib/python3.10/site-packages/starlette/applications.py", line 112, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/opt/etebase/.venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 181, in __call__
    raise exc
  File "/opt/etebase/.venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 159, in __call__
    await self.app(scope, receive, _send)
  File "/opt/etebase/.venv/lib/python3.10/site-packages/starlette/middleware/trustedhost.py", line 51, in __call__
    await self.app(scope, receive, send)
  File "/opt/etebase/.venv/lib/python3.10/site-packages/starlette/middleware/cors.py", line 84, in __call__
    await self.app(scope, receive, send)
  File "/opt/etebase/.venv/lib/python3.10/site-packages/starlette/exceptions.py", line 82, in __call__
    raise exc
  File "/opt/etebase/.venv/lib/python3.10/site-packages/starlette/exceptions.py", line 71, in __call__
    await self.app(scope, receive, sender)
  File "/opt/etebase/.venv/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
    raise e
  File "/opt/etebase/.venv/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "/opt/etebase/.venv/lib/python3.10/site-packages/starlette/routing.py", line 656, in __call__
    await route.handle(scope, receive, send)
  File "/opt/etebase/.venv/lib/python3.10/site-packages/starlette/routing.py", line 259, in handle
    await self.app(scope, receive, send)
  File "/opt/etebase/.venv/lib/python3.10/site-packages/starlette/routing.py", line 61, in app
    response = await func(request)
  File "/opt/etebase/./etebase_server/fastapi/msgpack.py", line 74, in custom_route_handler
    return await route_handler(request)
  File "/opt/etebase/.venv/lib/python3.10/site-packages/fastapi/routing.py", line 227, in app
    raw_response = await run_endpoint_function(
  File "/opt/etebase/.venv/lib/python3.10/site-packages/fastapi/routing.py", line 162, in run_endpoint_function
    return await run_in_threadpool(dependant.call, **values)
  File "/opt/etebase/.venv/lib/python3.10/site-packages/starlette/concurrency.py", line 39, in run_in_threadpool
    return await anyio.to_thread.run_sync(func, *args)
  File "/opt/etebase/.venv/lib/python3.10/site-packages/anyio/to_thread.py", line 28, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(func, *args, cancellable=cancellable,
  File "/opt/etebase/.venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 818, in run_sync_in_worker_thread
    return await future
  File "/opt/etebase/.venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 754, in run
    result = context.run(func, *args)
  File "/opt/etebase/./etebase_server/fastapi/db_hack.py", line 25, in wrapper
    return func(*args, **kwargs)
  File "/opt/etebase/./etebase_server/fastapi/db_hack.py", line 25, in wrapper
    return func(*args, **kwargs)
  File "/opt/etebase/./etebase_server/fastapi/routers/collection.py", line 451, in item_list
    response = item_list_common(queryset, user, stoken, limit, prefetch)
  File "/opt/etebase/./etebase_server/fastapi/routers/collection.py", line 435, in item_list_common
    data: t.List[CollectionItemOut] = [CollectionItemOut.from_orm_context(item, context) for item in result]
  File "/opt/etebase/./etebase_server/fastapi/routers/collection.py", line 435, in <listcomp>
    data: t.List[CollectionItemOut] = [CollectionItemOut.from_orm_context(item, context) for item in result]
  File "/opt/etebase/./etebase_server/fastapi/routers/collection.py", line 90, in from_orm_context
    content=CollectionItemRevisionInOut.from_orm_context(obj.content, context),
  File "/opt/etebase/./etebase_server/fastapi/routers/collection.py", line 64, in from_orm_context
    with open(chunk_obj.chunkFile.path, "rb") as f:
FileNotFoundError: [Errno 2] No such file or directory: '/opt/etebase/media/user_3/HgAHqCikBnmWDRFr3f9Wq1ict3joXG4W/v-/UX-O64j5IbmNKdyRnubaguGEpp88Hobfej_BPLV_s'

If I mkdir -p and touch those files, the error messages just mention different files that are missing, endlessly.

Ubuntu 22.04.1, up to date. Any ideas?

I want to allow signup users in eteserver, but I do not want anyone else to allow this (I mean do not publicly allow, except my front end application ). What I have in mind is that I will create new custom sign up API, and use some signature to make sure request is coming from my front end application. After that verification, I want to signup user, I am not sure how to do this, I tried to explore etebase python sdk so that I can replicate functionality from there, but no luck.

Can anyone please guide, how I can achieve this thing. Thanks in advance -:)

Due to the permissions set on the requirements file which is owned by root, it is impossible to install the requirements without sudo. The etesync user which does not have the sudo command therefore cannot install the requirements. It would be a good idea to add the requirements installation in the docker initialization but for update etc it probably would be good to change the permissions as well.

Hello,

Note: Skip the next three bullets, which are ranting about the centralisation of calendar software, if you just want the description of the issue.

I'd like to discuss the hosted DAV adapter as seen on EteSync's paid hosted service - let me know if this is the right place for it. I have been trying to setup smart scheduling for a really long time now, but there are three problems:

• Universally all scheduling services (Calendly being the biggest) exclusively support Google, Microsoft, and occasionally Apple calendars.
• If you want to use any of these scheduling services without calendar sync, you'd hope that the scheduling service would default to send you email calendar invitations when someone books with you and you'll manually import it in your calendar, and same for the booking user. You'd be very wrong - none of these services do this - they just send you an email with info on when the meeting is scheduled, and you're expected to manually add it to your calendar (same for the booking user on the other side).
• The one service which supports CalDAV integrations is Harmonizely (or what looks like a clone of it, simplymeet.me). However, it seems like Etesync's hosted Dav adapter is causing trouble.

Now for the actual issue, the DAV adapter seems slow and unstable - when I synced it locally with thunderbird, it started syncing, sent some events over, then threw HTTP 500 on a request after 603ms. Later, it again syncs some events, and ends with another 500. Similarly, Harmonizely just returns a generic message saying something is wrong with the calendar.

Any clue? It'd be great if we can find a convenient way to do smart scheduling, which is a must for any professional usecases.