I have tried to execute that example: https://github.com/ironport/shrapnel/blob/master/coro/http/demo/jsonrpc_server.py and got error:

$ python env/http_server.py FIX ber.py to use coro.asn1.ber Segmentation fault

Then I tried run in gdb:

(gdb) run Starting program: /str/development/projects/open-source/shrapnel/env/bin/python env/http_server.py warning: Could not load shared library symbols for linux-vdso.so.1. Do you need "set solib-search-path" or "set sysroot"? [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". FIX ber.py to use coro.asn1.ber

Program received signal SIGSEGV, Segmentation fault. 0x00007ffff675e875 in _wrap1 () from /str/development/projects/open-source/shrapnel/env/lib/python2.7/site-packages/coro-1.0.3_000-py2.7-linux-x86_64.egg/coro/_coro.so (gdb) c Continuing.

Program terminated with signal SIGSEGV, Segmentation fault. The program no longer exists. (gdb)

My hack that checks for llvm doesn't appear to work with pre-Lion gcc. That gcc reports itself as:

gcc --version i686-apple-darwin10-gcc-4.2.1 (GCC) 4.2.1 (Apple Inc. build 5666) (dot 3)

But requires the 16-byte offset, not the 8-byte offset. Perhaps it's not llvm that distinguishes the different offset, but the platform?

This leads to the immediate segfault. Anyone have a pre-Lion machine to test this on?

This is to fix socket._fileobject.write() method. It was failing because _fileobject.flush() that is eventually called after the _fileobject.write() passes data to sock.sendall() as a memoryview object.

Hello Guys,

I found that sslip module (wrapper for the OpenSSL library) is missing in the shrapnel code tree. Because of this fact the coroutine/coro_ssl.py module is broken and I cannot use SSL connections with shrapnel. If the module was forgotten when shrapnel went open source please add it. If it was decided not to post the sslip here for some reason please let me know.

Thanks.

The dependency on aplib causes a lot of complexity with installation. It would be best to remove the dependency, and the drawback of copying some stuff into shrapnel.

The ticks per second value was hard-coded in tsc_time.pyx module. This caused the time in shrapnel-based program to notably diverge from the system time. I implemented detection of actual ticks per second based on CPU frequency from /proc/cpuinfo.

I tried using stub_resolver yesterday and I noticed that when you do stub_resolver.install(), the_resolver doesn't change until reload(coro).

This is a python behavior but I am wondering if we can workaround it in a better way.

from coro.dns import stub_resolver stub_resolver.install(['8.8.8.8']) import coro coro.the_resolver <coro._coro.dummy_resolver object at 0x8015590b0> reload(coro) coro.the_resolver <coro.dns.stub_resolver.stub_resolver instance at 0x801a1ed40>

Note: this code will not work correctly until https://github.com/awslabs/s2n/issues/165 is addressed. If you would like to test it now, use samrushing/s2n to get the 3-line patch.

Only the most basic of testing has been done. Using s2n/test/{serve.py,openssl_client.py} I can open multiple client connections and funnel about 8MB/s through two clients simultaneously. No client testing from s2n has been tried yet.

Single-byte tags are limited to the range 0-30. When used in a protocol implementation with the APPLICATION flag set, this limits the number of different packet types that can be flagged in the protocol. While 31 types should be enough for most protocols, it precludes the ability to use a 'private' range of tags (e.g. tags > 100 indicate private protocol extensions).

Unfortunately the assumption that tag & flags fit into a single byte was pretty heavily built in to the codec, so fixing it required sweeping changes. I've placed this work into the multibyte-tag branch.

But I'd actually prefer to not use that code. I have a different code base that uses a nifty trick for encoding that will give much higher performance (by avoiding lots of copying). This different code base also has extensive auto-generated tests.

Python 2.7 Cython 0.20.1

$ python setup.py build

Error compiling Cython file: ... cdef int scope_id cdef addrinfo * ai if len(address) == 4: # as per python return value from getaddrinfo() and arg to connect() ip, port, flowinfo, scope_id = address sin6.sin6_flowinfo = htonl (flowinfo) ^ coro/socket.pyx:830:16: Object of type 'sockaddr_in6' has no attribute 'sin6_flowinfo'

and similar errors for 'sin6_scope_id' eventually terminating in:

building 'coro._coro' extension creating build/temp.linux-x86_64-2.7 creating build/temp.linux-x86_64-2.7/coro x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/home/venzen/sourcecode/shrapnel/. -I/home/venzen/sourcecode/shrapnel/include -I/usr/include/python2.7 -c coro/_coro.c -o build/temp.linux-x86_64-2.7/coro/_coro.o -Wno-unused-function coro/_coro.c:1:2: error: #error Do not use this file, it is the result of a failed Cython compilation. #error Do not use this file, it is the result of a failed Cython compilation. ^ error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

_Using Python 3.4 results in errors with building package distribute egg

The _coro.coro.TimeoutError class inherits from _coro.coro.Interrupted, which, in turn, inherits from BaseException. Simple Exception is not subclassed. Therefore, timeouts cannot be trapped so:

    try:
        coro.with_timeout(timeout, method)
    except Exception, unused:
        handle_exception()

Instead, one must use:

    try:
        coro.with_timeout(timeout, method)
    except (Exception, TimeoutError), unused:
        handle_exception()

According to documentation, BaseException is not to be directly inherited by user-defined classes. Likewise, BaseException is used in terminal situations which are not meant to be (normally) intercepted by the user. For example, GeneratorExit is a subclass of BaseException, while StopIteration subclasses Exception. Users are expected to handle StopIteration, while GeneratorExit is not often observed in userland.

Generally, I think timeout situation should not be considered as a terminal condition. If there is a possibility to impose an explicit timeout on a some operation, I think it should be possible to trap and handle the related exception on the same level, as anticipated, and not terminal, situation.

Hello, I am getting Build Error while compiling with Python3.7. openssl Extension was compiling properly with Python3.5 at least. currently i uninstall Python3.5 and install Python3.7, not sure if any thing i got miss .kindly help me to find the issue

Error: ^ coro/ssl/openssl.c: In function ‘__Pyx_GetException’: coro/ssl/openssl.c:25622:22: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’ tmp_type = tstate->exc_type; ^ coro/ssl/openssl.c:25623:23: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’ tmp_value = tstate->exc_value; ^ coro/ssl/openssl.c:25624:20: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’ tmp_tb = tstate->exc_traceback; ^ coro/ssl/openssl.c:25625:11: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’ tstate->exc_type = local_type; ^ coro/ssl/openssl.c:25626:11: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’ tstate->exc_value = local_value; ^ coro/ssl/openssl.c:25627:11: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’ tstate->exc_traceback = local_tb; ^ coro/ssl/openssl.c: In function ‘__Pyx_ExceptionSwap’: coro/ssl/openssl.c:25649:22: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’ tmp_type = tstate->exc_type; ^ coro/ssl/openssl.c:25650:23: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’ tmp_value = tstate->exc_value; ^ coro/ssl/openssl.c:25651:20: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’ tmp_tb = tstate->exc_traceback; ^ coro/ssl/openssl.c:25652:11: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’ tstate->exc_type = *type; ^ coro/ssl/openssl.c:25653:11: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’ tstate->exc_value = *value; ^ coro/ssl/openssl.c:25654:11: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’ tstate->exc_traceback = *tb; ^ coro/ssl/openssl.c: In function ‘__Pyx_ExceptionSave’: coro/ssl/openssl.c:25667:19: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’ *type = tstate->exc_type; ^ coro/ssl/openssl.c:25668:20: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’ *value = tstate->exc_value; ^ coro/ssl/openssl.c:25669:17: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’ *tb = tstate->exc_traceback; ^ coro/ssl/openssl.c: In function ‘__Pyx_ExceptionReset’: coro/ssl/openssl.c:25681:22: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’ tmp_type = tstate->exc_type; ^ coro/ssl/openssl.c:25682:23: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’ tmp_value = tstate->exc_value; ^ coro/ssl/openssl.c:25683:20: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’ tmp_tb = tstate->exc_traceback; ^ coro/ssl/openssl.c:25684:11: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’ tstate->exc_type = type; ^ coro/ssl/openssl.c:25685:11: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’ tstate->exc_value = value; ^ coro/ssl/openssl.c:25686:11: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’ tstate->exc_traceback = tb; ^ error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

It seems to have problem when building with cython 0.29.10 (currently installed by pip3). "pip3 install coro" failed as well as build from source by "python3 setup.py build. Platform: Ubuntu and MacOS (10.14.5) Python: 3.7.3

Error message is

Error compiling Cython file:
------------------------------------------------------------
...
    """Raise an OSError exception by errno.

    :Parameters:
        - `error_number`: The errno value to raise.
    """
    map_exception(OSError(error_number, libc.strerror(error_number)))
                                           ^
------------------------------------------------------------

coro/oserrors.pyx:54:44: cimported module has no attribute 'strerror'
building 'coro.oserrors' extension
gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python2.7 -c coro/oserrors.c -o build/temp.linux-x86_64-2.7/coro/oserrors.o
coro/oserrors.c:1:2: error: #error Do not use this file, it is the result of a failed Cython compilation.
 #error Do not use this file, it is the result of a failed Cython compilation.
  ^~~~~
error: command 'gcc' failed with exit status 1

chatty@ubuntu:~shrapnel/test$ python test_readv.py E

ERROR: test_readv (main.Test) Test readv.

Traceback (most recent call last): File "test_readv.py", line 113, in test_readv '01234567')) File "test_readv.py", line 77, in testit blocks = coro.with_timeout(5, s.readv, block_receives) File "coro/_coro.pyx", line 1111, in coro._coro.sched.with_timeout (coro/_coro.c:16399) TypeError: Argument 'size_list' has incorrect type (expected list, got tuple)

Ran 1 test in 0.001s

FAILED (errors=1) 7: Tue Dec 6 22:26:19 2016 thread 7 (<bound method TestSession.run of <main.TestSession instance at 0x7f1f252603f8>>): error '('coro/oserrors.pyx coro.oserrors.map_exception (coro/oserrors.c:874)|44', "<class 'coro.oserrors.EPIPE'>", '[Errno 32] Broken pipe', '[coro/_coro.pyx coro._coro._wrap1 (coro/_coro.c:12115)|772] [test_readv.py run|63] [coro/socket.pyx coro._coro.sock.send (coro/_coro.c:27626)|282] [coro/socket.pyx coro._coro.sock.send (coro/_coro.c:27500)|314] [coro/pyrex_helpers.pyx coro._coro.raise_oserror (coro/_coro.c:3879)|32] [coro/oserrors.pyx coro.oserrors.map_exception (coro/oserrors.c:874)|44]')' chatty@ubuntu:~shrapnel/test$

Ran into this issue using coro inside of an lxc container when cpu resources of the container were fixed to something other than the first few cores.

When you assign the cpus to the container - they report back sequentially starting at , 0,1,2,3,4...depending on the number of cores assigned to the container ( in the /proc/cpuinfo file.)

However when you get the current_cpu_number = sched_getcpu(), this returns the actual cpu on which the program is running. They don't match up, so cpu-found remains false - and the error below is produced.

see tsc_time.pyx, lines 254-280

import coro File "/root/traffic_gen/py27/local/lib/python2.7/site-packages/coro/init.py", line 25, in from coro._coro import * File "coro/clocks/tsc_time.pxd", line 25, in init coro._coro (coro/_coro.c:58161) File "coro/clocks/tsc_time.pyx", line 308, in init coro.clocks.tsc_time (coro/clocks/tsc_time.c:15190) File "coro/clocks/tsc_time.pyx", line 272, in coro.clocks.tsc_time.get_ticks_per_sec (coro/clocks/tsc_time.c:2119) RuntimeError: failed to detect CPU frequency