Python Ssl Eof Occurred In Violation Of Protocol, Wantwriteerror, Zeroreturnerror

I'm running many celery tasks (20,000) using gevent for the pool (also monkey patching all). Each of these tasks hit 3rd party services like adwords to pull data. I keep having tas

Solution 1:

So let's break this down by each traceback block. The first ends with:

  File "/usr/lib/python2.7/urllib2.py", line 1178, in do_open
    raise URLError(err)
URLError: <urlopen error [Errno 8] _ssl.c:504: EOF occurred in violation of protocol>

This is coming from urllib2. The fact that this receives an EOF makes me think that the server closed the connection while you were waiting for that "thread" to read from the socket again. You might want to use more time.sleep(0) to yield to gevent.

The second traceback comes from requests:

  File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 382, in send
    raise SSLError(e, request=request)
SSLError: [Errno bad handshake] (-1, 'Unexpected EOF')

The [Errno bad handshake] would make me tend to think this is a problem establishing the connection which could be caused by an unexpected EOF. Is that caused by using gevent? I'm uncertain.

The final traceback is definitely from requests as well but it also is coming out of PyOpenSSL and isn't being caught by urllib3 or requests.

  File "/usr/local/lib/python2.7/dist-packages/OpenSSL/SSL.py", line 851, in _raise_ssl_error
    raise ZeroReturnError()
ZeroReturnError

I did some searching and found that "According to the pyOpenSSL docs ZeroReturnError means that the SSL connection has been closed cleanly." This says to me that the server again closed the connection because you took to long to read anything from the socket.

In short, I think you need to explicitly yield more often just to ensure that these socket problems don't arise. That's just a guess though, so take it with a grain of salt.