A CalDAV/CardDAV server

Following https://github.com/pimutils/vdirsyncer/issues/605, I tried to set collections = [["addressbook", null, "addressbook"]] (note that collections = null used to work one week ago), this fails with:

$ vdirsyncer -vdebug sync my_contacts
debug: Using 1 maximal workers.
Syncing my_contacts/addressbook
debug: Loaded status for my_contacts/addressbook
debug: PROPFIND http://localhost:8080/user/contacts/addressbook/
debug: {'Depth': '1', 'User-Agent': 'vdirsyncer', 'Content-Type': 'application/xml; charset=UTF-8'}
debug: <?xml version="1.0" encoding="utf-8" ?>
debug:             <D:propfind xmlns:D="DAV:">
debug:                 <D:prop>
debug:                     <D:resourcetype/>
debug:                     <D:getcontenttype/>
debug:                     <D:getetag/>
debug:                 </D:prop>
debug:             </D:propfind>
debug:             
debug: Sending request...
debug: 207
debug: {'Content-Type': 'text/xml; charset="utf-8"', 'Content-Length': '507'}
debug: <ns0:multistatus xmlns:ns0="DAV:" xmlns:ns1="urn:ietf:params:xml:ns:carddav"><ns0:response><ns0:href>/user/contacts/addressbook/</ns0:href><ns0:status>HTTP/1.1 200 OK</ns0:status><ns0:propstat><ns0:status>HTTP/1.1 200 OK</ns0:status><ns0:prop><ns0:resourcetype><ns0:collection /><ns1:addressbook /></ns0:resourcetype><ns0:getcontenttype>httpd/unix-directory</ns0:getcontenttype><ns0:getetag>"4b825dc642cb6eb9a060e54bf8d69288fbee4904"</ns0:getetag></ns0:prop></ns0:propstat></ns0:response></ns0:multistatus>
debug: Already normalized: '/user/contacts/addressbook/'
debug: Skipping '/user/contacts/addressbook/', is collection.
Copying (uploading) item ed0e2e72a72d08638d393e59ef322270e4e7b4050ce80f0cba97a17783f2f12c to my_contacts_remote/addressbook
debug: Normalized URL from 'ed0e2e72a72d08638d393e59ef322270e4e7b4050ce80f0cba97a17783f2f12c.vcf' to '/user/contacts/addressbook/ed0e2e72a72d08638d393e59ef322270e4e7b4050ce80f0cba97a17783f2f12c.vcf'
debug: PUT http://localhost:8080/user/contacts/addressbook/ed0e2e72a72d08638d393e59ef322270e4e7b4050ce80f0cba97a17783f2f12c.vcf
debug: {'If-None-Match': '*', 'User-Agent': 'vdirsyncer', 'Content-Type': 'text/vcard'}
debug: BEGIN:VCARD
debug: VERSION:3.0
debug: FN:Forest Gump
debug: N:Gump;Forest
debug: END:VCARD
debug: Sending request...
debug: 412
debug: {'Content-Type': 'text/plain; encoding="utf-8"', 'Content-Length': '0'}

error: Unknown error occured for my_contacts/addressbook: Precondition Failed
error: Use `-vdebug` to see the full traceback.
debug:   File "/home/user/.local/lib/python3.5/site-packages/vdirsyncer-0.15.1.dev24+ng8a1e7a8-py3.5.egg/vdirsyncer/cli/utils.py", line 74, in handle_cli_error
debug:     raise e
debug:   File "/home/user/.local/lib/python3.5/site-packages/vdirsyncer-0.15.1.dev24+ng8a1e7a8-py3.5.egg/vdirsyncer/sync.py", line 268, in sync
debug:     action.run(a_info, b_info, conflict_resolution, partial_sync)
debug:   File "/home/user/.local/lib/python3.5/site-packages/vdirsyncer-0.15.1.dev24+ng8a1e7a8-py3.5.egg/vdirsyncer/sync.py", line 299, in run
debug:     self._run_impl(a, b)
debug:   File "/home/user/.local/lib/python3.5/site-packages/vdirsyncer-0.15.1.dev24+ng8a1e7a8-py3.5.egg/vdirsyncer/sync.py", line 330, in _run_impl
debug:     href, etag = self.dest.storage.upload(self.item)
debug:   File "/home/user/.local/lib/python3.5/site-packages/vdirsyncer-0.15.1.dev24+ng8a1e7a8-py3.5.egg/vdirsyncer/storage/base.py", line 15, in inner
debug:     return f(self, *args, **kwargs)
debug:   File "/home/user/.local/lib/python3.5/site-packages/vdirsyncer-0.15.1.dev24+ng8a1e7a8-py3.5.egg/vdirsyncer/storage/base.py", line 15, in inner
debug:     return f(self, *args, **kwargs)
debug:   File "/home/user/.local/lib/python3.5/site-packages/vdirsyncer-0.15.1.dev24+ng8a1e7a8-py3.5.egg/vdirsyncer/storage/dav.py", line 528, in upload
debug:     return self._put(href, item, None)
debug:   File "/home/user/.local/lib/python3.5/site-packages/vdirsyncer-0.15.1.dev24+ng8a1e7a8-py3.5.egg/vdirsyncer/storage/dav.py", line 499, in _put
debug:     headers=headers
debug:   File "/home/user/.local/lib/python3.5/site-packages/vdirsyncer-0.15.1.dev24+ng8a1e7a8-py3.5.egg/vdirsyncer/storage/dav.py", line 358, in request
debug:     return http.request(method, url, session=self._session, **more)
debug:   File "/home/user/.local/lib/python3.5/site-packages/vdirsyncer-0.15.1.dev24+ng8a1e7a8-py3.5.egg/vdirsyncer/http.py", line 179, in request
debug:     raise exceptions.PreconditionFailed(r.reason)
error: 1 out of 2 tasks failed.

On xandikos' side I get:

[pid: 14421|app: 0|req: 1/1] 127.0.0.1 (user) {38 vars in 529 bytes} [Tue Mar 21 14:51:03 2017] PROPFIND /user/contacts/addressbook/ => generated 507 bytes in 5 msecs (HTTP/1.1 207) 2 headers in 91 bytes (1 switches on core 0)
[pid: 14421|app: 0|req: 2/2] 127.0.0.1 (user) {38 vars in 647 bytes} [Tue Mar 21 14:51:03 2017] PUT /user/contacts/addressbook/ed0e2e72a72d08638d393e59ef322270e4e7b4050ce80f0cba97a17783f2f12c.vcf => generated 0 bytes in 0 msecs (HTTP/1.1 412) 2 headers in 99 bytes (0 switches on core 0)

Now I have a calendar with those three events:

BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Example Corp.//CalDAV Client//EN
BEGIN:VEVENT
SUMMARY:Our Blissful Anniversary
DTSTART;VALUE=DATE:19971102
DTSTAMP:19970901T130000Z
UID:[email protected]
RRULE:FREQ=YEARLY
CATEGORIES:ANNIVERSARY,PERSONAL,SPECIAL OCCASION
CLASS:CONFIDENTIAL
TRANSP:TRANSPARENT
END:VEVENT
END:VCALENDAR

BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Example Corp.//CalDAV Client//EN
BEGIN:VEVENT
SUMMARY:Bastille Day Party
DTSTART:20060714T170000Z
DTEND:20060715T040000Z
DTSTAMP:20060712T182145Z
UID:[email protected]
END:VEVENT
END:VCALENDAR

BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Example Corp.//CalDAV Client//EN
BEGIN:VEVENT
SUMMARY:Bastille Day Jitsi Party
DTSTART:20210714T170000Z
DTEND:20210715T040000Z
DTSTAMP:20210712T182145Z
UID:[email protected]
END:VEVENT
END:VCALENDAR

... and doing this report query:

<C:calendar-query xmlns:D="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav">
  <D:prop>
    <C:calendar-data/>
  </D:prop>
  <C:filter>
    <C:comp-filter name="VCALENDAR">
      <C:comp-filter name="VEVENT">
        <C:prop-filter name="CATEGORIES">
          <C:text-match collation="i;octet">PERSONAL</C:text-match>
        </C:prop-filter>
      </C:comp-filter>
    </C:comp-filter>
  </C:filter>
</C:calendar-query>

... produces this traceback:

Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/aiohttp/web_protocol.py", line 422, in _handle_request
    resp = await self._request_handler(request)
  File "/usr/local/lib/python3.8/dist-packages/aiohttp/web_app.py", line 499, in _handle
    resp = await handler(request)
  File "/usr/local/lib/python3.8/dist-packages/xandikos-0.2.8-py3.8.egg/xandikos/web.py", line 1476, in xandikos_handler
    return await main_app.aiohttp_handler(request, options.route_prefix)
  File "/usr/local/lib/python3.8/dist-packages/xandikos-0.2.8-py3.8.egg/xandikos/webdav.py", line 2152, in aiohttp_handler
    response = await self._handle_request(request, environ)
  File "/usr/local/lib/python3.8/dist-packages/xandikos-0.2.8-py3.8.egg/xandikos/webdav.py", line 2108, in _handle_request
    return await do.handle(request, environ, self)
  File "/usr/local/lib/python3.8/dist-packages/xandikos-0.2.8-py3.8.egg/xandikos/webdav.py", line 1764, in handle
    return await reporter.report(
  File "/usr/local/lib/python3.8/dist-packages/xandikos-0.2.8-py3.8.egg/xandikos/webdav.py", line 353, in wrapper
    async for resp in req_fn(self, environ, *args, **kwargs):
  File "/usr/local/lib/python3.8/dist-packages/xandikos-0.2.8-py3.8.egg/xandikos/caldav.py", line 566, in report
    async for (href, resource) in webdav.traverse_resource(
  File "/usr/local/lib/python3.8/dist-packages/xandikos-0.2.8-py3.8.egg/xandikos/webdav.py", line 1205, in traverse_resource
    for (child_name, child_resource) in members_fn(resource):
  File "/usr/local/lib/python3.8/dist-packages/xandikos-0.2.8-py3.8.egg/xandikos/web.py", line 636, in calendar_query
    for (name, file, etag) in self.store.iter_with_filter(filter=filter):
  File "/usr/local/lib/python3.8/dist-packages/xandikos-0.2.8-py3.8.egg/xandikos/store/__init__.py", line 358, in _iter_with_filter_indexes
    if filter.check_from_indexes(name, file_values):
  File "/usr/local/lib/python3.8/dist-packages/xandikos-0.2.8-py3.8.egg/xandikos/icalendar.py", line 832, in check_from_indexes
    if not child_filter.match_indexes(indexes, self.tzify):
  File "/usr/local/lib/python3.8/dist-packages/xandikos-0.2.8-py3.8.egg/xandikos/icalendar.py", line 630, in match_indexes
    if not child.match_indexes(subindexes, tzify):
  File "/usr/local/lib/python3.8/dist-packages/xandikos-0.2.8-py3.8.egg/xandikos/icalendar.py", line 630, in match_indexes
    if not child.match_indexes(subindexes, tzify):
  File "/usr/local/lib/python3.8/dist-packages/xandikos-0.2.8-py3.8.egg/xandikos/icalendar.py", line 728, in match_indexes
    if not child.match_indexes(subindexes):
  File "/usr/local/lib/python3.8/dist-packages/xandikos-0.2.8-py3.8.egg/xandikos/icalendar.py", line 509, in match_indexes
    return any(
  File "/usr/local/lib/python3.8/dist-packages/xandikos-0.2.8-py3.8.egg/xandikos/icalendar.py", line 510, in <genexpr>
    self.match(self.type_fn.from_ical(k)) for k in indexes[None])
  File "/usr/local/lib/python3.8/dist-packages/icalendar/prop.py", line 283, in from_ical
    out = unescape_char(ical).split(',')
  File "/usr/local/lib/python3.8/dist-packages/icalendar/parser.py", line 34, in unescape_char
    assert isinstance(text, (compat.unicode_type, compat.bytes_type))
AssertionError
[18/Oct/2022:13:09:32 +0000] "REPORT /user/calendars/pythoncaldav-test/ HTTP/1.0" 500 231 "-" "Mozilla/5.0"

The weird thing here is that while I'm consistently able to reproduce this towards my existing calendar server running stand-alone on the latest HEAD of master (a43ba24a13ac5a8810cc13b031fe73d6505e1e95), I'm not able to reproduce it running the same test code but towards an "internal" xandikos server running the same revision.

I will do more research into this, though not sure if I will catch it today.

when creating collections via MKCOL, xandikos seems to use latin-1 encoding. However, when accessing collections, it uses utf-8.

% LC_ALL=C.UTF-8 ls
calendar test test @ foo ät bar-vdirsyncer-ci-5b7ed28d-be75-46e2-9daa-65e03d1f2f57 test @ foo ät bar-vdirsyncer-ci-ba578f29-0f5f-4772-a777-18ca3df8a2ba

(ä should actually be an a-umlaut)

Found by vdirsyncer's test_specialchars.

I'm currently implementing support for search by category and other stuff in the python caldav client library.

I have this test data in my calendar (an example from RFC5545):

BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Example Corp.//CalDAV Client//EN
BEGIN:VEVENT
UID:[email protected]
DTSTAMP:19970901T130000Z
DTSTART;VALUE=DATE:19971102
SUMMARY:Our Blissful Anniversary
TRANSP:TRANSPARENT
CLASS:CONFIDENTIAL
CATEGORIES:ANNIVERSARY,PERSONAL,SPECIAL OCCASION
RRULE:FREQ=YEARLY
END:VEVENT
END:VCALENDAR

And I'm doing a REPORT search like this:

<C:calendar-query xmlns:D="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav">
  <D:prop>
    <C:calendar-data/>
  </D:prop>
  <C:filter>
    <C:comp-filter name="VCALENDAR">
      <C:comp-filter name="VEVENT">
        <C:prop-filter name="CATEGORIES">
          <C:text-match collation="i;octet">PERSONAL</C:text-match>
        </C:prop-filter>
      </C:comp-filter>
    </C:comp-filter>
  </C:filter>
</C:calendar-query>

Which, if I've understood the RFCs correctly, should yield the example calendar object - but, no - an empty list is returned:

<multistatus xmlns="DAV:" />

I tested matching on the CLASS property, if I ask for a text match on CONFIDENTIAL it is returned. If I ask for a text match on CONF it is not returned. While that behaviour does make sense to me, at least in this context, I think the RFC states that it should match.

I'm looking to run xandikos in docker. I'm not sure from the readme which paths I need to supply to docker, what ports it exposes, how SSL termination works, etc, etc. Assuming I can figure this stuff out, are you open to a PR to expand on the docker docs? I might also include an example docker-compose file.

I am running into some problems while testing Xandikos. Akonadi (1.13.0) in form of KOrganizer(5.9.3) and KAddressBook(5.9.3) cant read from my Xandikos instance. Writing in form of creating new contacts and events works just fine and syncs to other devices. Automatic discovery works, and akonadi finds all the calendars and vcard groups on the server.

The "other devices" is DAVdroid (2.0.6-ose) on my phone and evolution (3.30.2) running on the same machine as KOrganizer. This is a fully updated Fedora 29 install.

Xandikos is installed in a virtenv on a fedora 28 server with the following packages

$ bin/python3 --version
Python 3.6.7
$ bin/pip3 list
Package         Version   
--------------- ----------
certifi         2018.10.15
defusedxml      0.5.0     
dulwich         0.19.9    
icalendar       4.0.3     
Jinja2          2.10      
MarkupSafe      1.1.0     
pip             18.1      
python-dateutil 2.6.1     
pytz            2018.7    
setuptools      40.6.2    
six             1.11.0    
urllib3         1.24.1    
uWSGI           2.0.17.1  
wheel           0.32.3    
xandikos        0.0.11  

I am starting xandikos as an uwsgi app as following:

/home/ochagenes/xandikos/bin/uwsgi /home/ochagenes/xandikos/uwsgi.ini

uwsgi.ini:

[uwsgi]
socket = 127.0.0.1:8001
uid = ochagenes
gid = ochagenes
master = true
cheaper = 0
processes = 1
plugin = python3
module = xandikos.wsgi:app
umask = 022
env = XANDIKOSPATH=/home/ochagenes/xandikos/collections
env = CURRENT_USER_PRINCIPAL=/user/
# Set AUTOCREATE to have Xandikos create default CalDAV/CardDAV
# collections if they don't yet exist. Possible values:
#  - principal: just create the current user principal
#  - defaults: create the principal and default calendar and contacts
#       collections. (recommended)
env = AUTOCREATE=defaults

and i'm using apache as uwsgi gateway, the relevant config snippet is:

<IfModule mod_proxy_uwsgi.c>
        <Location "/xandikos">
                AuthType      Digest
                AuthName      "Radicale"
                AuthUserFile  /etc/httpd/conf/passwd
                Require       user ochagenes

                ProxyPass uwsgi://127.0.0.1:8001/
        </Location>
</IfModule>

The reason for AuthName "Radicale" is so that it uses the same password as my existing radicale install.

The relevant logs are: Creating a new event on the phone and syncing with DAVdroid:

[pid: 32055|app: 0|req: 105/105] 10.10.30.40 (ochagenes) {74 vars in 1591 bytes} [Mon Dec 10 22:14:30 2018] PROPFIND /xandikos/user/calendars/calendar/ => generated 408 bytes in 4 msecs (HTTP/1.1 207) 2 headers in 91 bytes (2 switches on core 0)
[pid: 32055|app: 0|req: 106/106] 10.10.30.40 (ochagenes) {74 vars in 1591 bytes} [Mon Dec 10 22:14:30 2018] PROPFIND /xandikos/user/calendars/calendar/ => generated 844 bytes in 3 msecs (HTTP/1.1 207) 2 headers in 91 bytes (1 switches on core 0)
[pid: 32055|app: 0|req: 107/107] 10.10.30.40 (ochagenes) {74 vars in 1589 bytes} [Mon Dec 10 22:14:30 2018] REPORT /xandikos/user/calendars/calendar/ => generated 335 bytes in 32 msecs (HTTP/1.1 207) 2 headers in 91 bytes (1 switches on core 0)
[pid: 32055|app: 0|req: 108/108] 10.10.30.40 (ochagenes) {74 vars in 1832 bytes} [Mon Dec 10 22:14:30 2018] PUT /xandikos/user/calendars/calendar/b1626b52-1865-4ac8-9533-ba1cc3fa71ce.ics => generated 0 bytes in 158 msecs (HTTP/1.1 201) 1 headers in 74 bytes (0 switches on core 0)
[pid: 32055|app: 0|req: 109/109] 10.10.30.40 (ochagenes) {74 vars in 1591 bytes} [Mon Dec 10 22:14:30 2018] PROPFIND /xandikos/user/calendars/calendar/ => generated 408 bytes in 5 msecs (HTTP/1.1 207) 2 headers in 91 bytes (1 switches on core 0)
[pid: 32055|app: 0|req: 110/110] 10.10.30.40 (ochagenes) {74 vars in 1589 bytes} [Mon Dec 10 22:14:30 2018] REPORT /xandikos/user/calendars/calendar/ => generated 1953 bytes in 38 msecs (HTTP/1.1 207) 2 headers in 92 bytes (1 switches on core 0)
[pid: 32055|app: 0|req: 111/111] 10.10.30.40 (ochagenes) {74 vars in 1591 bytes} [Mon Dec 10 22:14:41 2018] PROPFIND /xandikos/user/calendars/calendar/ => generated 844 bytes in 4 msecs (HTTP/1.1 207) 2 headers in 91 bytes (2 switches on core 0)

Syncing with KOrganizer. The newly created event does not show up in KOrganizer.

[pid: 32055|app: 0|req: 113/113] 10.10.20.25 (ochagenes) {76 vars in 1561 bytes} [Mon Dec 10 22:14:54 2018] PROPFIND /xandikos/user/calendars/calendar/ => generated 335 bytes in 3 msecs (HTTP/1.1 207) 2 headers in 91 bytes (1 switches on core 0)
[pid: 32055|app: 0|req: 114/114] 10.10.20.25 (ochagenes) {76 vars in 1559 bytes} [Mon Dec 10 22:14:54 2018] REPORT /xandikos/user/calendars/calendar/ => generated 1953 bytes in 37 msecs (HTTP/1.1 207) 2 headers in 92 bytes (2 switches on core 0)
[pid: 32055|app: 0|req: 116/116] 10.10.20.25 (ochagenes) {74 vars in 1544 bytes} [Mon Dec 10 22:14:54 2018] REPORT /xandikos/user/calendars/calendar/ => generated 2537 bytes in 1 msecs (HTTP/1.1 207) 2 headers in 92 bytes (1 switches on core 0)
[pid: 32055|app: 0|req: 117/117] 10.10.20.25 (ochagenes) {76 vars in 1561 bytes} [Mon Dec 10 22:14:54 2018] PROPFIND /xandikos/user/calendars/calendar/ => generated 335 bytes in 1 msecs (HTTP/1.1 207) 2 headers in 91 bytes (1 switches on core 0)
[pid: 32055|app: 0|req: 118/118] 10.10.20.25 (ochagenes) {76 vars in 1559 bytes} [Mon Dec 10 22:14:54 2018] REPORT /xandikos/user/calendars/calendar/ => generated 1953 bytes in 37 msecs (HTTP/1.1 207) 2 headers in 92 bytes (1 switches on core 0)

Creating a new event in KOrganizer.

[pid: 32055|app: 0|req: 119/119] 10.10.20.25 (ochagenes) {82 vars in 1933 bytes} [Mon Dec 10 22:15:04 2018] PUT /xandikos/user/calendars/calendar/1544476504.R886.ics => generated 0 bytes in 203 msecs (HTTP/1.1 201) 1 headers in 74 bytes (0 switches on core 0)
[pid: 32055|app: 0|req: 120/120] 10.10.20.25 (ochagenes) {76 vars in 1761 bytes} [Mon Dec 10 22:15:04 2018] GET /xandikos/user/calendars/calendar/1544476504.R886.ics => generated 1205 bytes in 2 msecs (HTTP/1.1 200) 3 headers in 120 bytes (1 switches on core 0)

Syncing with DAVdroid, the event from KOrganizer shows up on the phone, as seen by the GET.

[pid: 32055|app: 0|req: 121/121] 10.10.30.40 (ochagenes) {74 vars in 1591 bytes} [Mon Dec 10 22:15:11 2018] PROPFIND /xandikos/user/calendars/calendar/ => generated 408 bytes in 2 msecs (HTTP/1.1 207) 2 headers in 91 bytes (1 switches on core 0)
[pid: 32055|app: 0|req: 122/122] 10.10.30.40 (ochagenes) {74 vars in 1591 bytes} [Mon Dec 10 22:15:11 2018] PROPFIND /xandikos/user/calendars/calendar/ => generated 844 bytes in 4 msecs (HTTP/1.1 207) 2 headers in 91 bytes (1 switches on core 0)
[pid: 32055|app: 0|req: 123/123] 10.10.30.40 (ochagenes) {74 vars in 1589 bytes} [Mon Dec 10 22:15:11 2018] REPORT /xandikos/user/calendars/calendar/ => generated 335 bytes in 53 msecs (HTTP/1.1 207) 2 headers in 91 bytes (1 switches on core 0)
[pid: 32055|app: 0|req: 124/124] 10.10.30.40 (ochagenes) {74 vars in 1589 bytes} [Mon Dec 10 22:15:11 2018] REPORT /xandikos/user/calendars/calendar/ => generated 2215 bytes in 54 msecs (HTTP/1.1 207) 2 headers in 92 bytes (1 switches on core 0)
[pid: 32055|app: 0|req: 125/125] 10.10.30.40 (ochagenes) {70 vars in 1650 bytes} [Mon Dec 10 22:15:11 2018] GET /xandikos/user/calendars/calendar/1544476504.R886.ics => generated 1205 bytes in 2 msecs (HTTP/1.1 200) 3 headers in 120 bytes (1 switches on core 0)
[pid: 32055|app: 0|req: 126/126] 10.10.30.40 (ochagenes) {74 vars in 1591 bytes} [Mon Dec 10 22:15:12 2018] PROPFIND /xandikos/user/calendars/calendar/ => generated 844 bytes in 4 msecs (HTTP/1.1 207) 2 headers in 91 bytes (2 switches on core 0)

Traceback (most recent call last):
  File "/nix/store/p37npidsgzbyqaryjz40brgkrcdvqczh-xandikos-0.0.6/bin/.xandikos-wrapped", line 30, in <module>
    from xandikos.web import main
  File "/nix/store/p37npidsgzbyqaryjz40brgkrcdvqczh-xandikos-0.0.6/lib/python2.7/site-packages/xandikos/web.py", line 37, in <module>
    from xandikos import (access, apache, caldav, carddav, quota, sync, webdav,
  File "/nix/store/p37npidsgzbyqaryjz40brgkrcdvqczh-xandikos-0.0.6/lib/python2.7/site-packages/xandikos/access.py", line 25, in <module>
    from xandikos import webdav
  File "/nix/store/p37npidsgzbyqaryjz40brgkrcdvqczh-xandikos-0.0.6/lib/python2.7/site-packages/xandikos/webdav.py", line 33, in <module>
    import urllib.parse
ImportError: No module named parse

I'm in the process of packaging xandikos for nixos (because I'm still using f**king owncloud to sync my contacts and calendars from two computers and a android device), but am running into this error.

Can you tell what's wrong?

The dependencies I pass into the package are:

• icalendar
• dulwich
• defusedxml
• jinja2

which should be all dependencies, right?

Another total weirdness ...

$ xandikos
Traceback (most recent call last):
  File "/usr/local/bin/xandikos", line 11, in <module>
    load_entry_point('xandikos==0.2.8', 'console_scripts', 'xandikos')()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 490, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2854, in load_entry_point
    return ep.load()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2445, in load
    return self.resolve()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2451, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/local/lib/python3.8/dist-packages/xandikos-0.2.8-py3.8.egg/xandikos/__main__.py", line 22, in <module>
    import asyncio
  File "/usr/local/lib/python3.8/dist-packages/asyncio/__init__.py", line 21, in <module>
    from .base_events import *
  File "/usr/local/lib/python3.8/dist-packages/asyncio/base_events.py", line 296
    future = tasks.async(future, loop=self)
                   ^
SyntaxError: invalid syntax

I tested to run a simple "import asyncio" from python shell, no problems. I tried running ./bin/xandikos, no problems. But the xandikos script that is installed by setup.py fails when installing master. Installing efe43705035f132bca3916fd562fc44935a9c0b2 and it works fine. I will do a bisect again ...

I've gone through some of the documentation (notes, examples, docs, readme) but could not really find an explanation. I'd like to know why is there a Git repository in the "calendars" folder as well as in "calendars/calendar". The one for a collection (calendar) makes sense but I don't really understand what gets written to the "calendars" folder.

There's a rationale for the collections: https://github.com/jelmer/xandikos/blob/master/notes/file-format.rst

A separate but related issue would be a quick hint on the best way to back up the data structure. The repository in the "calendars" and "contacts" are confusing me and I'm not sure what the best way would be to have a backup - either to add git remote repositories and push, or to copy the whole folder structure to a remote location.

I'd like to see these clarifications added to the docs so please suggest a topic and I'll see to submit a pull request.

Tests should not be installed, as they have no use once installed. However, it would be ideal to be able to run the tests prior to installation.

For this, apparently tests/ has to be moved out of xandikos/, amongst other things. See for example: https://github.com/sphinx-doc/sphinx/blob/5d2d4e04778c5cbdb89c29d62d5a30cc2c23e76d/setup.py#L214 https://github.com/sphinx-doc/sphinx/blob/5d2d4e04778c5cbdb89c29d62d5a30cc2c23e76d/MANIFEST.in#L25

When setting caching_threshold=0, I get this traceback ... changing the paranoid setting does not seem to change anything:

  File "/usr/lib/python3.10/site-packages/aiohttp/web_app.py", line 504, in _handle
    resp = await handler(request)
  File "/home/tobias/caldav/tests/test_caldav.py", line 2209, in xandikos_handler
  File "/usr/lib/python3.10/site-packages/xandikos-0.2.8-py3.10.egg/xandikos/webdav.py", line 2152, in aiohttp_handler
    response = await self._handle_request(request, environ)
  File "/usr/lib/python3.10/site-packages/xandikos-0.2.8-py3.10.egg/xandikos/webdav.py", line 2108, in _handle_request
    return await do.handle(request, environ, self)
  File "/usr/lib/python3.10/site-packages/xandikos-0.2.8-py3.10.egg/xandikos/webdav.py", line 1764, in handle
    return await reporter.report(
  File "/usr/lib/python3.10/site-packages/xandikos-0.2.8-py3.10.egg/xandikos/webdav.py", line 353, in wrapper
    async for resp in req_fn(self, environ, *args, **kwargs):
  File "/usr/lib/python3.10/site-packages/xandikos-0.2.8-py3.10.egg/xandikos/caldav.py", line 566, in report
    async for (href, resource) in webdav.traverse_resource(
  File "/usr/lib/python3.10/site-packages/xandikos-0.2.8-py3.10.egg/xandikos/webdav.py", line 1205, in traverse_resource
    for (child_name, child_resource) in members_fn(resource):
  File "/usr/lib/python3.10/site-packages/xandikos-0.2.8-py3.10.egg/xandikos/web.py", line 637, in calendar_query
    for (name, file, etag) in self.store.iter_with_filter(filter=filter):
  File "/usr/lib/python3.10/site-packages/xandikos-0.2.8-py3.10.egg/xandikos/store/__init__.py", line 338, in _iter_with_filter_indexes
    if filter.check_from_indexes(name, file_values):
  File "/usr/lib/python3.10/site-packages/xandikos-0.2.8-py3.10.egg/xandikos/icalendar.py", line 864, in check_from_indexes
    if not child_filter.match_indexes(
  File "/usr/lib/python3.10/site-packages/xandikos-0.2.8-py3.10.egg/xandikos/icalendar.py", line 659, in match_indexes
    if not child.match_indexes(subindexes, tzify):
  File "/usr/lib/python3.10/site-packages/xandikos-0.2.8-py3.10.egg/xandikos/icalendar.py", line 653, in match_indexes
    if self.time_range is not None and not self.time_range.match_indexes(
  File "/usr/lib/python3.10/site-packages/xandikos-0.2.8-py3.10.egg/xandikos/icalendar.py", line 479, in match_indexes
    d[field] = vDDDTypes(vDatetime.from_ical(
  File "/usr/lib/python3.10/site-packages/icalendar/prop.py", line 442, in from_ical
    raise ValueError('Wrong datetime format: %s' % ical)
ValueError: Wrong datetime format: 20070501

This is the report query:

<C:calendar-query xmlns:D="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav">
  <D:prop>
    <C:calendar-data/>
  </D:prop>
  <C:filter>
    <C:comp-filter name="VCALENDAR">
      <C:comp-filter name="VTODO">
        <C:time-range start="20250413T220000Z"/>
      </C:comp-filter>
    </C:comp-filter>
  </C:filter>
</C:calendar-query>

I have multiple tasks in the calendar, but none after 2025. I guess this one is the one causing the problem:

BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Example Corp.//CalDAV Client//EN
BEGIN:VTODO
CATEGORIES:FAMILY,FINANCE
CLASS:CONFIDENTIAL
DTSTAMP:20070313T123432Z
DUE;VALUE=DATE:20070501
STATUS:NEEDS-ACTION
SUMMARY:Submit Quebec Income Tax Return for 2006
UID:[email protected]
END:VTODO
END:VCALENDAR

I've turned on paranoid and index-threshold=0 in the caldav functional tests now ... this stops me from doing a release, so I will have to revert it unless we get this issue resolved relatively fast :-)

I get this traceback:

E           caldav.lib.error.ReportError: ReportError at '500 Internal Server Error
E
E           b'500 Internal Server Error\n\nServer got itself in trouble'', reason no reason

caldav/objects.py:194: ReportError
---------------------------------------------------------- Captured log call -----------------------------------------------------------
ERROR    aiohttp.server:web_protocol.py:403 Error handling request
Traceback (most recent call last):
  File "/usr/lib/python3.10/site-packages/aiohttp/web_protocol.py", line 433, in _handle_request
    resp = await request_handler(request)
  File "/usr/lib/python3.10/site-packages/aiohttp/web_app.py", line 504, in _handle
    resp = await handler(request)
  File "/home/tobias/caldav/tests/test_caldav.py", line 2199, in xandikos_handler
    return await mainapp.aiohttp_handler(request, "/")
  File "/usr/lib/python3.10/site-packages/xandikos-0.2.8-py3.10.egg/xandikos/webdav.py", line 2152, in aiohttp_handler
    response = await self._handle_request(request, environ)
  File "/usr/lib/python3.10/site-packages/xandikos-0.2.8-py3.10.egg/xandikos/webdav.py", line 2108, in _handle_request
    return await do.handle(request, environ, self)
  File "/usr/lib/python3.10/site-packages/xandikos-0.2.8-py3.10.egg/xandikos/webdav.py", line 1764, in handle
    return await reporter.report(
  File "/usr/lib/python3.10/site-packages/xandikos-0.2.8-py3.10.egg/xandikos/webdav.py", line 353, in wrapper
    async for resp in req_fn(self, environ, *args, **kwargs):
  File "/usr/lib/python3.10/site-packages/xandikos-0.2.8-py3.10.egg/xandikos/caldav.py", line 566, in report
    async for (href, resource) in webdav.traverse_resource(
  File "/usr/lib/python3.10/site-packages/xandikos-0.2.8-py3.10.egg/xandikos/webdav.py", line 1205, in traverse_resource
    for (child_name, child_resource) in members_fn(resource):
  File "/usr/lib/python3.10/site-packages/xandikos-0.2.8-py3.10.egg/xandikos/web.py", line 637, in calendar_query
    for (name, file, etag) in self.store.iter_with_filter(filter=filter):
  File "/usr/lib/python3.10/site-packages/xandikos-0.2.8-py3.10.egg/xandikos/store/__init__.py", line 351, in _iter_with_filter_indexes
    raise AssertionError(
AssertionError: index based filter not matching real file filter

All tests pass if I turn off paranoid or if I turn the index threshold high.

I realize that this is a bit too little information for further debugging, but ... I need to sleep now, will get back with more details tomorrow.

With the new aiohttp support in Dulwich, we could make the Xandikos repositories cloneable over HTTP. There are some caveats to watch out for here. For example, if we allow write-access, we should possibly reject anything that adds invalid files.

Sorry to pester you again :-)

I have this REPORT query (as per https://www.rfc-editor.org/rfc/rfc4791#section-7.8.5):

<C:calendar-query xmlns:D="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav">
  <D:prop>
    <C:calendar-data/>
  </D:prop>
  <C:filter>
    <C:comp-filter name="VCALENDAR">
      <C:comp-filter name="VEVENT">
        <C:comp-filter name="VALARM">
          <C:time-range start="20151010T060100Z" end="20151010T060700Z"/>
        </C:comp-filter>
      </C:comp-filter>
    </C:comp-filter>
  </C:filter>
</C:calendar-query>

And this event on the calendar:

BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//python-caldav//caldav//en_DK
BEGIN:VEVENT
SUMMARY:This is a test event
DTSTART;VALUE=DATE-TIME:20151010T080706
DTEND;VALUE=DATE-TIME:20161010T090807
DTSTAMP;VALUE=DATE-TIME:20221024T143714Z
UID:96d4e74a-5398-11ed-a1df-a0510ba5c65c
BEGIN:VALARM
ACTION:AUDIO
TRIGGER:-PT15M
END:VALARM
END:VEVENT
END:VCALENDAR

I see that no timezone information given for the DTSTART and DTEND, though I assume from the NotImplementedError that this is not the reason for the 500 ...

  File "/home/tobias/caldav/tests/test_caldav.py", line 2224, in xandikos_handler
    return await mainapp.aiohttp_handler(request, "/")
  File "/usr/lib/python3.10/site-packages/xandikos-0.2.8-py3.10.egg/xandikos/webdav.py", line 2152, in aiohttp_handler
    response = await self._handle_request(request, environ)
  File "/usr/lib/python3.10/site-packages/xandikos-0.2.8-py3.10.egg/xandikos/webdav.py", line 2108, in _handle_request
    return await do.handle(request, environ, self)
  File "/usr/lib/python3.10/site-packages/xandikos-0.2.8-py3.10.egg/xandikos/webdav.py", line 1764, in handle
    return await reporter.report(
  File "/usr/lib/python3.10/site-packages/xandikos-0.2.8-py3.10.egg/xandikos/webdav.py", line 353, in wrapper
    async for resp in req_fn(self, environ, *args, **kwargs):
  File "/usr/lib/python3.10/site-packages/xandikos-0.2.8-py3.10.egg/xandikos/caldav.py", line 566, in report
    async for (href, resource) in webdav.traverse_resource(
  File "/usr/lib/python3.10/site-packages/xandikos-0.2.8-py3.10.egg/xandikos/webdav.py", line 1205, in traverse_resource
    for (child_name, child_resource) in members_fn(resource):
  File "/usr/lib/python3.10/site-packages/xandikos-0.2.8-py3.10.egg/xandikos/web.py", line 637, in calendar_query
    for (name, file, etag) in self.store.iter_with_filter(filter=filter):
  File "/usr/lib/python3.10/site-packages/xandikos-0.2.8-py3.10.egg/xandikos/store/__init__.py", line 314, in _iter_with_filter_naive
    if filter.check(name, file):
  File "/usr/lib/python3.10/site-packages/xandikos-0.2.8-py3.10.egg/xandikos/icalendar.py", line 822, in check
    if not child_filter.match(file.calendar, self.tzify):
  File "/usr/lib/python3.10/site-packages/xandikos-0.2.8-py3.10.egg/xandikos/icalendar.py", line 603, in match
    if not any(child.match(c, tzify) for c in comp.subcomponents):
  File "/usr/lib/python3.10/site-packages/xandikos-0.2.8-py3.10.egg/xandikos/icalendar.py", line 603, in <genexpr>
    if not any(child.match(c, tzify) for c in comp.subcomponents):
  File "/usr/lib/python3.10/site-packages/xandikos-0.2.8-py3.10.egg/xandikos/icalendar.py", line 603, in match
    if not any(child.match(c, tzify) for c in comp.subcomponents):
  File "/usr/lib/python3.10/site-packages/xandikos-0.2.8-py3.10.egg/xandikos/icalendar.py", line 603, in <genexpr>
    if not any(child.match(c, tzify) for c in comp.subcomponents):
  File "/usr/lib/python3.10/site-packages/xandikos-0.2.8-py3.10.egg/xandikos/icalendar.py", line 596, in match
    and not self.time_range.match(comp, tzify)):
  File "/usr/lib/python3.10/site-packages/xandikos-0.2.8-py3.10.egg/xandikos/icalendar.py", line 453, in match
    return component_handler(self.start, self.end, comp, tzify)
  File "/usr/lib/python3.10/site-packages/xandikos-0.2.8-py3.10.egg/xandikos/icalendar.py", line 378, in apply_time_range_valarm
    raise NotImplementedError(apply_time_range_valarm)
NotImplementedError: <function apply_time_range_valarm at 0x7fccbb9bc940>

I'm not expecting this to get merged right now. Just putting it here for a place holder and feedback. Will begin working on get_resource() ACLS.

Also updates nginx example to pass $remote_user as HTTP_X_REMOTE_USER from a proxy.

On Calendar.app I have a problem renaming or deleting calendars.

• I delete the calendar and click View->Refresh calendars.
• Error appears: The request (CalDAVCalendarSplitDualTypeCalendarQueueableOperation) for account “Localhost” failed.
• Many duplicates of the calendar to be deleted appear in the calendar list for that account.

Originally posted by @walkertraylor in https://github.com/jelmer/xandikos/discussions/172