Hi

I needed to pass some arguments to the aiohttp request (i.e.: http basic auth), which is supported at the Resource level but not at the Attachment/Document/... level. This patch adds that, on each public method (except the ones i missed, i guess ;) )

I think it would be best to be able to set that this on the Server object and propagate it to the objects returned, but i don't really have the time to implement that, so here's an intermediate solution.

What do you think?

First off, thank you for this project, I'm building an app that should scale quite largely against couchdb and I'm looking forward to using the power of async :)

I had prototype code working perfectly against CouchDb (1.4) and recently we are trying to transition to Couchbase Sync Gateway for scalability issues (we expect thousands of concurrent feeds). After some setup pains, my script to write test data worked, and the web interface returns the list of documents, but all queries with aiocouchdb returned nothing.

I spent quite some time debugging this. And it seems to be a bug in the library, which may well be related to the problem in issue #8 curl http://localhost:5984/default/_all_docs?include_docs=true returns the proper information. In this case:

{"rows":[
{"key":"product:484d5edd-e8f3-4cbb-962f-c88b08898a4f","id":"product:484d5edd-e8f3-4cbb-962f-c88b08898a4f","value":{"rev":"1-2d78300a38c3f4e8db23e05d9ed20d1d"},"doc":{"_id":"product:484d5edd-e8f3-4cbb-962f-c88b08898a4f","_rev":"1-2d78300a38c3f4e8db23e05d9ed20d1d","created_at":"2016-01-21T14:19:02.627+0000","name":"Pasta","price":750,"type":"product","version":1}}
,{"key":"product:4ab9a3e1-2081-4159-9746-2b1cb29e0cb4","id":"product:4ab9a3e1-2081-4159-9746-2b1cb29e0cb4","value":{"rev":"1-39244db8fbf1150f18ef2b755487068d"},"doc":{"_id":"product:4ab9a3e1-2081-4159-9746-2b1cb29e0cb4","_rev":"1-39244db8fbf1150f18ef2b755487068d","created_at":"2016-01-21T14:01:02.279+0000","name":"Peanuts","price":150,"type":"product","version":1}}
,{"key":"product:7219f59b-3fce-4893-8f38-e2a1f38a82dc","id":"product:7219f59b-3fce-4893-8f38-e2a1f38a82dc","value":{"rev":"1-1bf8bbc9e5b5cc626361f76f14dc29bd"},"doc":{"_id":"product:7219f59b-3fce-4893-8f38-e2a1f38a82dc","_rev":"1-1bf8bbc9e5b5cc626361f76f14dc29bd","created_at":"2016-01-21T13:57:52.495+0000","name":"Beer","price":200,"type":"product","version":1}}
],
"total_rows":3,"update_seq":4}

My query code looks something like this:

        db = await ensure_db(server, db_name)
        docs = await db.all_docs(include_docs=True)
        with docs:
            rec = await docs.next()
            while rec:
                print_json(rec)
                rec = await docs.next()

As I said, this worked against CouchDB, but the first call to docs.next() now returns None....

Digging into the aiocouchdb code code, I put a breakpoint in ViewFeed.next (feeds.py:140) and found that chunk was the entire body of the response. Apparently Sync Gateway buffers the text, making one network IO call, while couchdb sends each chunk in it's own network IO call. And quoting from feeds.py:146-147:

        elif chunk.startswith(('{"rows"', ']}')):
            return (yield from self.next())

Thus, the chunk is skipped and just moved onto the next one... which doesn't exist, so the result is None, an empty iterator :(

For me, the solution is to no longer depend on the network IO caching behavior of the server, but use a more powerful parser to return one line at a time. (I think ijson (https://pypi.python.org/pypi/ijson) is a great solution for parsing streaming json, and with optional c libraries, very fast).

My work-around looks something like this:

in views.py, add to the top: from .client import HttpStreamResponse and on line 50, change: resp = yield from request(auth=auth, data=data, params=params) to: resp = yield from request(auth=auth, data=data, params=params, response_class=HttpStreamResponse)

in Feed._loop (feeds.py:59), change: chunk = yield from self._resp.content.read() to chunk = yield from self._resp.content.readline()

This mostly works and parses the rows correctly now, it raises an exception on parsing the remainder in ViewFeed.next (feeds.py:140-149). We have an empty line with ], but the code only checks for ]}. Also, we no longer have {"total_rows" with no trailing }, but rather "total_rows" with no leading {.

I could work on a patch here, but I would like some direction from the project maintainer.

• There don't seem to be tests for the feeds in unit test (understandably, as it usually requires a server to test), shall I try to mock something there, or is there a way to run tests against various servers.
• What variants need to be supported (CouchDB 1.4? CouchDB 2.0? Sync Gateway?)
• Is it acceptable to add requirements that provide more robust streaming json parsing?

Thanks again for the great package, and please let me know how I can contribute.

Hi.

This is not an issue, I'm abusing your issue tracker to ask a newbie question in the absence of a mailing list / IRC channel. Or if taken positively, maybe it can be taken as a hint that the doc could be more helpful to total asyncio newcomers; I'll be glad to propose such a doc patch if we end up feeling the need for it.

My humble objective here is to succeed at fetching and printing the 'Welcome!' Couch message using your server.info().

So after reading the doc, I run my localhost:5984 couch instance, create a python3.4 virtualenv, pip install aiocouchdb, touch a file, and here's what I try to run:

#! /usr/bin/env python3

import aiocouchdb

server = aiocouchdb.Server()
print('server:', server, '\n')

That works (and I'll omit the imports and server creation for now): it prints server: <aiocouchdb.v1.server.Server(http://localhost:5984) object at 0x7fcc064c0be0>. Now, let's try getting my server info. I see the doc mentioning Python doesn’t supports yield from in shell, but I don't know better, so I try it anyway:

yield from server.info()

Barf, SyntaxError: 'yield' outside function. Okay, I can fix that:

def sinfo():
    yield from server.info()

something = sinfo()
print(something)

Great, I have a <generator object sinfo at 0x7fe8e59c13f0>. Let's next() it:

def sinfo():
    yield from server.info()

gen = sinfo()
box = next(gen)
print(box)

Yay, a <Future pending>. How do I attach a callback to this thing? Thank you, python 3.4 doc on add_done_callback:

def sinfo():
    yield from server.info()

gen = sinfo()
fut = next(gen)
fut.add_done_callback(print)
print(fut)

And hey, there's even a nice __repr__ here to help me see this in my final print: <Future pending cb=[print()]>.

But sadly, nothing is printed.

• Am I exiting before receiving anything? Well no, sleep()ing for a while doesn't help, and more importantly...
• ... my couchdb log shows no incoming request :-/
• Am I failing to initialize something? I didn't see more required initialization in the doc :/ .Also, I don't see any mention of me needing to do event loop management, and as I see an asyncio.get_event_loop() call in client.py so I assume it's none of my business.

What am I doing terribly wrong? Sorry again for hijacking your issue tracker, and thanks for your work :)

I'm using this code

@asyncio.coroutine
def go():
    db = yield from couch_server.db('db)
    ddoc = yield from gobox_db.ddoc('ddoc')
    viewdoc = yield from ddoc.view('viewdoc')
    while True:
        res = yield from viewdoc.next()
        pprint.pprint(res['value'])

viewdoc.next() would return a JSONDecodeError exception on some rows. by adding print(chunk) on feeds.py I found on some rows the chunk ends in the middle of the json object.

The new code works well in line per line, and probably for couchdb 2.0. However, couchbase sync gateway has another peculiarity, in that the "total_rows": X, "offset": Y fields come AFTER the rows, with slightly different sequence of closing brackets :(

I made a small adjustment to the parsing code, so it can handle both variants. And tested on sync gateway and couchdb 1.4.

Still working on a more general solution at https://github.com/ethanfrey/aiojson ...

It seems that further development is frozen for now (latest commit was in Sep 12, 2016).

Is this project still maintained?

There are few PRs, but I'd be glad to help in near future with development. Can I count on code review from maintainers or should I fork it for my own?

Thanks in advance!

Commit https://github.com/KeepSafe/aiohttp/commit/66a27842fd4f7d11953694f3555ff92693befdb8 adds skip_auto_header as another arguments for update_body_from_data.

aiocouchdb currently fails with: TypeError: update_body_from_data() takes 2 positional arguments but 3 were given

the function call in https://github.com/aio-libs/aiocouchdb/blob/master/aiocouchdb/client.py#L187 and https://github.com/aio-libs/aiocouchdb/blob/master/aiocouchdb/client.py#L197 needs to be changed accordingly.

having _content as bytearray causes chardet to throw a ValueError. Here is an example of requesting a non-existing design doc.

Traceback (most recent call last):
  File "/home/vukasin/.local/lib64/python3.4/site-packages/aiohttp/server.py", line 272, in start
    yield from self.handle_request(message, payload)
  File "/home/vukasin/.local/lib64/python3.4/site-packages/aiohttp/web.py", line 85, in handle_request
    resp = yield from handler(request)
  File "/usr/lib64/python3.4/asyncio/tasks.py", line 105, in coro
    res = yield from res
  File "/home/vukasin/peach-2/peach/queue/__init__.py", line 43, in get
    q = yield from ddoc.view("queue", name)
  File "/home/vukasin/.local/lib64/python3.4/site-packages/aiocouchdb/v1/designdoc.py", line 332, in view
    params=params))
  File "/home/vukasin/.local/lib64/python3.4/site-packages/aiocouchdb/views.py", line 52, in request
    yield from resp.maybe_raise_error()
  File "/home/vukasin/.local/lib64/python3.4/site-packages/aiocouchdb/errors.py", line 140, in maybe_raise_error
    data = yield from resp.json()
  File "/home/vukasin/.local/lib64/python3.4/site-packages/aiohttp/client_reqrep.py", line 703, in json
    encoding = self._get_encoding()
  File "/home/vukasin/.local/lib64/python3.4/site-packages/aiohttp/client_reqrep.py", line 671, in _get_encoding
    encoding = chardet.detect(self._content)['encoding']
  File "/home/vukasin/.local/lib64/python3.4/site-packages/chardet/__init__.py", line 26, in detect
    raise ValueError('Expected a bytes object, not a unicode object')
ValueError: Expected a bytes object, not a unicode object

Fixes #13, and followup to discussion in https://github.com/aio-libs/aiocouchdb/issues/12#issuecomment-198662156 . I tried to keep minimal, and insist on reading the doc and getting help if needed.

Feel free to modify as desired. Thanks for aiocouchdb.

when i try to install aiocouchdb its uninstall my aiohttp-2.2.5 and install a lower version.

here is what happened: pip install aiocouchdb Collecting aiocouchdb Downloading aiocouchdb-0.9.1.tar.gz (63kB) 100% |████████████████████████████████| 71kB 523kB/s Collecting aiohttp==0.17.4 (from aiocouchdb) Downloading aiohttp-0.17.4.tar.gz (475kB) 100% |████████████████████████████████| 481kB 968kB/s Requirement already satisfied: chardet in /Users/veto/webs/merkuro/env/lib/python3.6/site-packages (from aiohttp==0.17.4->aiocouchdb) Installing collected packages: aiohttp, aiocouchdb Found existing installation: aiohttp 2.2.5 Uninstalling aiohttp-2.2.5: Successfully uninstalled aiohttp-2.2.5 Running setup.py install for aiohttp ... done Running setup.py install for aiocouchdb ... done Successfully installed aiocouchdb-0.9.1 aiohttp-0.17.4

(env)$ pip freeze aiocouchdb==0.9.1 aiodns==1.1.1 aiohttp==0.17.4

Currently, test suite is quite a overly complicated. It has two mode to run: "mock" and "couchdb". For the "mock" it mocks (obliviously!) HTTP request method and only test logic for some magically expected response. For "couchdb" mode it makes real requests to CouchDB instance and it makes sure that our imaginations of how things should work is close to real.

Initially, that was good idea: mock tests runs faster, require no service to run and, hell, why not? However, since those time it started to be bad idea because mocks hides real issues and keeping both modes to run the same test cases makes life harder.

Current plan is to completely move away from mock tests to integration ones. Despite all problems this will make sure that aiocouchdb really works with real service right.