Environment

• pytest 4.0.0
• Python 3.7.1
• Annaconda
• Windows10

> pytest --version
This is pytest version 4.0.0, imported from c:\programdata\anaconda3\lib\site-packages\pytest.py
setuptools registered plugins:
  pytest-remotedata-0.3.0 at c:\programdata\anaconda3\lib\site-packages\pytest_remotedata\plugin.py
  pytest-openfiles-0.3.0 at c:\programdata\anaconda3\lib\site-packages\pytest_openfiles\plugin.py
  pytest-doctestplus-0.1.3 at c:\programdata\anaconda3\lib\site-packages\pytest_doctestplus\plugin.py
  pytest-arraydiff-0.2 at c:\programdata\anaconda3\lib\site-packages\pytest_arraydiff\plugin.py

Warning

DeprecationWarning causes in socks.py when pytest command is executed.

https://github.com/pytest-dev/pytest/issues/4769

> pytest tests/
============================= test session starts =============================
platform win32 -- Python 3.7.1, pytest-4.0.0, py-1.6.0, pluggy-0.7.1 -- c:\programdata\anaconda3\python.exe
cachedir: .pytest_cache
rootdir: C:\vagrant\synced_folder\annofab-api-access-python-commons, inifile: pytest.ini
plugins: remotedata-0.3.0, openfiles-0.3.0, doctestplus-0.1.3, arraydiff-0.2
collected 1 items

tests/test_annofab.py::test_account PASSED

============================== warnings summary ===============================
c:\programdata\anaconda3\lib\site-packages\socks.py:58
  c:\programdata\anaconda3\lib\site-packages\socks.py:58: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
    from collections import Callable

I'm trying to write a small test application to test if a Proxy supports UDP over SOCKS5.

Currently the application seems to be sending messages correctly but receiving is falling over when receiving the echoed data. File "E:\pacs\socks.py", line 330, in recvfrom peerhost, peerport = self.proxy_peername TypeError: 'NoneType' object is not iterable

I cannot see how self.proxy_peername is being initialized with socks5 proxy. My test environment is using proxy plus (free edition) with SOCKS activated and the following code

#! /usr/bin/env python

# Client and server for udp (datagram) echo.
#
# Usage: udpecho -s [port]            (to start a server)
# or:    udpecho -c host [port] <file (client)

import sys
import socks
from socket import *

ECHO_PORT = 50000 + 7
BUFSIZE = 1024

def main():
    if len(sys.argv) < 2:
        usage()
    if sys.argv[1] == '-s':
        server()
    elif sys.argv[1] == '-c':
        client()
    elif sys.argv[1] == '-ss':
        sockserver()
    elif sys.argv[1] == '-sc':
        sockclient()
    else:
        usage()

def usage():
    sys.stdout = sys.stderr
    print 'Usage: udpecho -s [port]            (server)'
    print 'or:    udpecho -c host [port] <file (client)'
    sys.exit(2)

def server():
    if len(sys.argv) > 2:
        port = eval(sys.argv[2])
    else:
        port = ECHO_PORT
    s = socket(AF_INET, SOCK_DGRAM)
    s.bind(('', port))
    print 'udp echo server ready'
    while 1:
        data, addr = s.recvfrom(BUFSIZE)
        print 'server received', `data`, 'from', `addr`
        s.sendto(data, addr)

def client():
    if len(sys.argv) < 3:
        usage()
    host = sys.argv[2]
    if len(sys.argv) > 3:
        port = eval(sys.argv[3])
    else:
        port = ECHO_PORT
    addr = host, port
    s = socket(AF_INET, SOCK_DGRAM)
    s.settimeout(100)
    s.bind(('', 0))
    print 'udp echo client ready, reading stdin'
    while 1:
        line = sys.stdin.readline()
        if not line:
            break
        s.sendto(line, addr)
        data, fromaddr = s.recvfrom(BUFSIZE)
        print 'client received', `data`, 'from', `fromaddr`

def sockserver():
    if len(sys.argv) > 2:
        port = eval(sys.argv[2])
    else:
        port = ECHO_PORT
    socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, "127.0.0.1", 1080)
    s = socks.socksocket(AF_INET, SOCK_DGRAM)
    s.bind(('', port))
    print 'udp echo server ready'
    while 1:
        data, addr = s.recvfrom(BUFSIZE)
        print 'server received', `data`, 'from', `addr`
        s.sendto(data, addr)

def sockclient():
    if len(sys.argv) < 3:
        usage()
    host = sys.argv[2]
    if len(sys.argv) > 3:
        port = eval(sys.argv[3])
    else:
        port = ECHO_PORT
    addr = host, port
    s = socks.socksocket(AF_INET, SOCK_DGRAM)
    s.set_proxy(socks.SOCKS5, "localhost")
    s.settimeout(100)
    s.bind(('', 0))

    print 'udp echo client ready, reading stdin'
    try:
        while 1:
            line = sys.stdin.readline()
            if not line:
                break
            print 'sending', line, 'to', addr
            s.sendto(line, addr)
            res = s.recvfrom(BUFSIZE)
            print '%r' % res
            print 'client received', `data`, 'from', `fromaddr`
    except KeyboardInterrupt:
        print 'Bye'
        pass

main()

In one command window I run: udpecho.py -s

In the another window: udpecho.py -sc localhost test

The send is working as the server sees udp echo server ready server received 'test\n' from ('127.0.0.1', 49165)

But the receive is failing.

when using urllib2 + gevent 1.0 and setting timeout on request i got error "send() got an unexpected keyword argument 'timeout'

backtrace [...
 ('/usr/lib64/python2.7/urllib2.py', 400, 'open', 'response = self._open(req, data)'), ('/usr/lib64/python2.7/urllib2.py', 418, '_open', "'_open', req)"), ('/usr/lib64/python2.7/urllib2.py', 378, '_call_chain', 'result = func(*args)'), ('./main/sockshandler.py', 49, 'http_open', 'return self.do_open(build, req)'), ('/usr/lib64/python2.7/urllib2.py', 1174, 'do_open', 'h.request(req.get_method(), req.get_selector(), req.data, headers)'), ('/usr/lib64/python2.7/httplib.py', 958, 'request', 'self._send_request(method, url, body, headers)'), ('/usr/lib64/python2.7/httplib.py', 992, '_send_request', 'self.endheaders(body)'), ('/usr/lib64/python2.7/httplib.py', 954, 'endheaders', 'self._send_output(message_body)'), ('/usr/lib64/python2.7/httplib.py', 814, '_send_output', 'self.send(msg)'), ('/usr/lib64/python2.7/httplib.py', 776, 'send', 'self.connect()'), 
('./main/sockshandler.py', 24, 'connect', 'self.sock.connect((self.host, self.port))'), ('.../v/lib/python2.7/site-packages/socks.py', 668, 'connect', 'negotiate(self, dest_addr, dest_port)'), ('.../v/lib/python2.7/site-packages/socks.py', 378, '_negotiate_SOCKS5', 'CONNECT, dest_addr)'), ('.../v/lib/python2.7/site-packages/socks.py', 460, '_SOCKS5_request', 'writer.close()'), ('/usr/lib64/python2.7/socket.py', 279, 'close', 'self.flush()'), 
('/usr/lib64/python2.7/socket.py', 303, 'flush', 'self._sock.sendall(view[write_offset:write_offset+buffer_size])'), 
('.../v/lib/python2.7/site-packages/gevent/socket.py', 464, 'sendall', 'data_sent += self.send(_get_memory(data, data_sent), flags, timeout=timeleft)')]```

This adds support for sending and receiving UDP packets with a SOCKS5 proxy.

I reworked the code a fair amount, so that I could reuse some of the existing SOCKS5 code. For instance, _recvall() is now _readall() and works on a socket.makefile() or BytesIO object rather than the raw socket. Let me know if you aren’t comfortable with this refactoring, and I could try to tone it down a bit.

Another point is that my UDP code only works with Python 3. Python 2’s socket class sets the sendto() etc methods as instance attributes (rather than class attributes), defeating the normal way of overriding methods. I’m not really motivated to make it work for Python 2 unless there is a really simple workaround.

I should add some tests, but haven’t got around to it. Would you be interested if I converted the test suite to use Python’s standard unittest framework? It should be more maintainable and flexible that way.

As you said

sockshandler.py is just a "recipe"/utility for easy integration with urllib2. So, I don't think there is a need for an__init__.py because socks.py is the only file intended for importion.

Why not? One can use PySocks just for urllib2 (like I do). So maybe it would be a nice choice to add sockshandler in pip package?

SOCKS5 adds support for sending literal IPv6 addresses. This patch takes advantage of that support to allow applications using PySocks to request connection to literal IPv6 addresses, and to allow local DNS resolution to resolve IPv6 addresses as well as IPv4 addresses.

Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working.

Preserves Python 2 compatibility.

Using requests with a socks5 proxy fails on: AttributeError: 'super' object has no attribute 'settimeout'

Reproduction Steps

pip install requests[socks]==2.14.2 (this installs PySocks v1.6.7)

run a socks proxy on port 9050. I use Tor for that matter.

Simplified code I use is:

import requests
proxies={'http': 'socks5://127.0.0.1:9050'}
url = 'http://www.whatismyip.org/'
requests.get(url, proxies=proxies)

System Information

Windows 7 (x64): 6.1.7601
Python 2.7.12 64bit

Full Traceback

File "c:\Python27\lib\site-packages\requests\api.py", line 72, in get return request('get', url, params=params, **kwargs) File "c:\Python27\lib\site-packages\requests\api.py", line 58, in request return session.request(method=method, url=url, **kwargs) File "c:\Python27\lib\site-packages\requests\sessions.py", line 518, in request resp = self.send(prep, **send_kwargs) File "c:\Python27\lib\site-packages\requests\sessions.py", line 639, in send r = adapter.send(request, **kwargs) File "c:\Python27\lib\site-packages\requests\adapters.py", line 438, in send timeout=timeout File "c:\Python27\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 600, in urlopen chunked=chunked) File "c:\Python27\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 356, in _make_request conn.request(method, url, **httplib_request_kw) File "c:\python27\Lib\httplib.py", line 1057,in request self._send_request(method, url, body, headers) File "c:\python27\Lib\httplib.py", line 1097, in _send_request self.endheaders(body) File "c:\python27\Lib\httplib.py",line 1053, in endheaders self._send_output(message_body) File "c:\python27\Lib\httplib.py", line 897, in _send_output self.send(msg) File "c:\python27\Lib\httplib.py", line 859,in send self.connect() File "c:\Python27\lib\site-packages\requests\packages\urllib3\connection.py", line 166, in connect conn = self._new_conn() File "c:\Python27\lib\site-packages\requests\packages\urllib3\contrib\socks.py", line 88, in _new_conn **extra_kw File "c:\Python27\lib\site-packages\socks.py", line 229, in create_connection sock.connect((remote_host, remote_port)) File "c:\Python27\lib\site-packages\socks.py", line 96, in wrapper return function(*args, **kwargs) File "c:\Python27\lib\site-packages\socks.py", line 781, in connect super(socksocket, self).settimeout(self._timeout) AttributeError: 'super' object has no attribute 'settimeout'

Extras

I want to suggest the closest issue I have found, that is on gevent: https://github.com/gevent/gevent/issues/937

Replacing all occurences of (in socks.py): super(socksocket, self).settimeout with: self.settimeout solves the issue, but I don't know if there are any other side effects

Tracking changes that might have caused issues in software using this library would be easier with a CHANGELOG, please include one : http://keepachangelog.com/en/0.3.0/.

This problem can be repro by following code:

import socket
import socks

BUF_SIZE = 1024

conn = socks.socksocket(socket.AF_INET, socket.SOCK_DGRAM, socket.SOL_UDP)
conn.set_proxy(socks.SOCKS5, "127.0.0.1", 1080)
conn.sendto(b'a' * BUF_SIZE, ('127.0.0.1', 8000))
data, _ = conn.recvfrom(BUF_SIZE)
# less exact 10 bytes than request
assert len(data) == BUF_SIZE - 10

This is caused by following lines in recvfrom:

        buf = BytesIO(_BaseSocket.recv(self, bufsize, flags))
        # remain BUF_SIZE - 2
        buf.seek(2, SEEK_CUR)
        # remain BUF_SIZE - 3
        frag = buf.read(1)
        if ord(frag):
            raise NotImplementedError("Received UDP packet fragment")
        # remain BUF_SIZE - 10
        fromhost, fromport = self._read_SOCKS5_address(buf)

Hi,

I need to make multiple calls in a script, with or without proxies.

For now, I do this:

if proxy:
            socks.set_default_proxy(socks.SOCKS5, proxy, port)
            socket.socket = socks.socksocket
handler = urlopen(url)

My problem is that I then need to reset socket.socket to prevent using SOCKS proxy for further calls. My method for now is to store socket.socket in a global var before and then to restore it when needed.

Is there a better way ?

Maybe calling socks.set_default_proxy() should remove all default proxies. What do you think about it ?

Thanks

This adds comma between DATA and OR PROFITS, adds WHETHER IN CONTRACT, STRICT LIABILITY, words, and fixes a typo (DAMANGE -> DAMAGE).

Reference: https://spdx.org/licenses/BSD-3-Clause.html

while trying to connect() , it takes some time and throw an Error msg -

TimeoutError: [Errno 60] Operation timed out

During handling of the above exception, another exception occurred:

socks.GeneralProxyError: Socket error: [Errno 60] Operation timed out

Solution please

Some applications publish SOCKS/HTTP proxy via unix domain socket. For example, TOR creates /run/tor/socks as a SOCKS proxy.

I think we should able to use PySocks like this:

import socks
s = socks.socksocket()
s.set_proxy(socks.SOCKS5, "/run/tor/socks")        # using unix domain socket /run/tor/socks as proxy

Hi there,

while monkeypatching a 3rd-party module, I observed a different behavior when calling socket.socket.recv(0) and socks.socksocket.recv(0). The former call will return b'' but the latter one will just stuck and never return.

The following code snippet reproduces this issue for me:

import socket
import socks

addr = ("127.0.0.1", 80)

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(addr)
r = s.recv(0)
print(r)

s1 = socks.socksocket(socket.AF_INET, socket.SOCK_STREAM)
s1.connect(addr)
r1 = s1.recv(0)  # never returns!
print(r1)

I would have expected similar behavior at least when no proxy is used. But I also do not understand what the idea of a recv(0) is, since I have no experience using sockets. So maybe this is as expected?!

(Python 3.9.12, PySocks 1.7.1)

Thanks in advance Thomas

Nowadays many proxy servers are authenticating the user by a token or key, instead username and password. Every request shall include the key, in a way similar to: http://myproxyserver.com:51337/?apiKey=bW3ZXxHJdjfodhsfodshfoisdofwe6wjCU

Is there a way to authenticate with pysocks using such keys?