云烟成雨 发表于 2022-7-6 09:14

requests模块设置超时和重试好像无效

我的python代码如下:
reqSesssion = requests.Session()
reqSesssion.mount('http://', HTTPAdapter(max_retries=30))
reqSesssion.mount('https://', HTTPAdapter(max_retries=30))

resData = reqSesssion.post(url, data=data, headers=headers, timeout=30)

按理来说,我这里设置了超时时间为30秒,重试次数30次,应该过30*30=900秒才会报错,而实际情况是没有过那么长时间就报错了,这是为什么?
遇到过这个情况的老铁能否给个解释?{:301_1009:}
我现在的解决方案是又写了try catch来捕获,但是感觉有点多余啊,明明他带了超时和重试,我还要自己再写错误捕获去重试,那它存在的意义何在?

错误信息提示:
Traceback (most recent call last):
File "C:\Program Files\Python310\lib\site-packages\urllib3\connectionpool.py", line 449, in _make_request
    six.raise_from(e, None)
File "<string>", line 3, in raise_from
File "C:\Program Files\Python310\lib\site-packages\urllib3\connectionpool.py", line 444, in _make_request
    httplib_response = conn.getresponse()
File "C:\Program Files\Python310\lib\http\client.py", line 1374, in getresponse
    response.begin()
File "C:\Program Files\Python310\lib\http\client.py", line 318, in begin
    version, status, reason = self._read_status()
File "C:\Program Files\Python310\lib\http\client.py", line 279, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "C:\Program Files\Python310\lib\socket.py", line 705, in readinto
    return self._sock.recv_into(b)
File "C:\Program Files\Python310\lib\ssl.py", line 1274, in recv_into
    return self.read(nbytes, buffer)
File "C:\Program Files\Python310\lib\ssl.py", line 1130, in read
    return self._sslobj.read(len, buffer)
TimeoutError: The read operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Program Files\Python310\lib\site-packages\requests\adapters.py", line 489, in send
    resp = conn.urlopen(
File "C:\Program Files\Python310\lib\site-packages\urllib3\connectionpool.py", line 785, in urlopen
    retries = retries.increment(
File "C:\Program Files\Python310\lib\site-packages\urllib3\util\retry.py", line 550, in increment
    raise six.reraise(type(error), error, _stacktrace)
File "C:\Program Files\Python310\lib\site-packages\urllib3\packages\six.py", line 770, in reraise
    raise value
File "C:\Program Files\Python310\lib\site-packages\urllib3\connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
File "C:\Program Files\Python310\lib\site-packages\urllib3\connectionpool.py", line 451, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
File "C:\Program Files\Python310\lib\site-packages\urllib3\connectionpool.py", line 340, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='node.kg.qq.com', port=443): Read timed out. (read timeout=1800)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "E:\Projects\Others\my-python\quanMin\songRoomTownPropCard.py", line 315, in main
    begin(strRoomId, roomName, roomIndex=value['index'])
File "E:\Projects\Others\my-python\quanMin\songRoomTownPropCard.py", line 276, in begin
    getTaskList(strRoomId, roomName, roomIndex)
File "E:\Projects\Others\my-python\quanMin\songRoomTownPropCard.py", line 150, in getTaskList
    resData = reqSesssion.post(url, data=json.dumps(reqData), headers=headers, timeout=Timeout)
File "C:\Program Files\Python310\lib\site-packages\requests\sessions.py", line 635, in post
    return self.request("POST", url, data=data, json=json, **kwargs)
File "C:\Program Files\Python310\lib\site-packages\requests\sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
File "C:\Program Files\Python310\lib\site-packages\requests\sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
File "C:\Program Files\Python310\lib\site-packages\requests\adapters.py", line 578, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='node.kg.qq.com', port=443): Read timed out. (read timeout=1800)

截图:

云烟成雨 发表于 2022-7-6 10:07

一个timeout时间超过以后就直接报错了,并没有自动重试,好像只有连接超时的时候会重试,读取超时不会自动重试,还是要写捕获啊

getstr88 发表于 2022-7-6 10:25

他本来设计的也不是说,读超时自动重试。是得自己处理

云烟成雨 发表于 2022-7-6 11:11

getstr88 发表于 2022-7-6 10:25
他本来设计的也不是说,读超时自动重试。是得自己处理

那这设计感觉有点鸡肋{:301_973:}

getstr88 发表于 2022-7-6 11:16

云烟成雨 发表于 2022-7-6 11:11
那这设计感觉有点鸡肋

我觉得作为基本系统库,本来也不应该有太过于个性的需求

你看,你说读失败就要自动重写。但超时了,一定能保证下一次不超么?

我觉得基本库提供最基础的功能就对了

然后应该是开发者自己封装自己个性的需求。比如我做一个,可以预设超时后自动重试指定次数的。也无非是多写几行代码

但是如果基本库都把这个需求弄进去了。对于没有这种需求的多数人,平白无故加了一个参数

云烟成雨 发表于 2022-7-6 11:23

getstr88 发表于 2022-7-6 11:16
我觉得作为基本系统库,本来也不应该有太过于个性的需求

你看,你说读失败就要自动重写。但超时了,一 ...

那他设计重试次数的目的是什么,是什么场景下才会重试?

getstr88 发表于 2022-7-6 11:29

云烟成雨 发表于 2022-7-6 11:23
那他设计重试次数的目的是什么,是什么场景下才会重试?

他有好些参数需要指定

Retry(total=1, backoff_factor=0.1, status_forcelist=)

云烟成雨 发表于 2022-7-6 13:36

getstr88 发表于 2022-7-6 11:29
他有好些参数需要指定

Retry(total=1, backoff_factor=0.1, status_forcelist=)

这个是max_retries的元组格式参数,直接写个数字也是可以的,这个是多了间隔时间和错误码

getstr88 发表于 2022-7-6 14:09

云烟成雨 发表于 2022-7-6 13:36
这个是max_retries的元组格式参数,直接写个数字也是可以的,这个是多了间隔时间和错误码

对呀。你都不精细的确定重试原因么?如果是返回404,难道再试1000次,就不会404么?

云烟成雨 发表于 2022-7-6 14:22

getstr88 发表于 2022-7-6 14:09
对呀。你都不精细的确定重试原因么?如果是返回404,难道再试1000次,就不会404么?

没有重试,在超时就直接抛出错误了,这个错误可能是需要有响应才能重试
页: [1] 2
查看完整版本: requests模块设置超时和重试好像无效