How To Implement A Retry Option In Pycurl

How could I implement a retry option within the PyCurl (libcurl) module in python? Something similar to the effect of: curl --retry 3 --retry-delay 5 'http://somesite.com/somefile

Solution 1:

Pycurl doesn't know how to re-initialize the consumer you supply through the WRITEDATA or WRITEFUNCTION options, so your code has to implement the retry logic:

retries_left = 3
delay_between_retries = 5 # seconds
success = False
c = pycurl.Curl()
c.setopt(c.URL, 'http://somesite.com/somefile')
while retries_left > 0:
  try:
    with open('output.txt', 'w') as f:
      c.setopt(c.WRITEFUNCTION, f.write)
      c.perform()
    success = True
    break
  except BaseException as e:
    retries_left -= 1
    time.sleep(delay_between_retries)
# check success