遇到了一个奇怪的请求问题请高手能帮忙解惑
本帖最后由 thepoy 于 2022-11-6 10:10 编辑请求如下：
```
GET https://shopee.com.my/api/v4/item/get?itemid=20537120944&shopid=68461810
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:106.0) Gecko/20100101 Firefox/106.0
Host: shopee.com.my
Content-Type: application/json
af-ac-enc-dat: AAcyLjQuMS0yAAABhEYD7joAAAjHAcEAAAAAAAAAAOYyhFB7rjob26/8rq6jA0F3J6Kfg5aGEX+GYncix7fIyPghAefe3JS894jq/3nC9cJjpEn32HTqasIUhFkInWzoCOj1uSC5kl6LU06aSrm61kX/Ny1L5jzxFjDrS1IzPHwt9muZUbatRPTf42k24UXHBZsir4fwWxQLVKw5gDu5CyybpSVWFCd7OLsY30Hj1OjSKZvDNTpkAhYqvdOyLCTGr41kHyFGV3ZaoQ01NX1u6R9AnBG6X9s1ynZK6vnTBgzBIOKTNFS4j1VT8sOl1BEtObri8ZUW3OTOHeCO4vGDCq4gRJFmvwSm1BNdccjxAekgEx3xwroP6ZL6LO5bh9QSxuKGYkUmR84CcHLB6dmMPnXDUGkagca9MFiK8RmRsrN2vcLDNTpkAhYqvdOyLCTGr41kKUmjTinalW5/ctjHa7Lte+06J5ekdC078Iv4wrMjrvbzUjYNqi2Hdu8tLPGrNL/jmEfixe8rpESf8+9J+WOK8kusILDBjMDq/xa+8hI9GWbdxIdVmB5payUD+EtCC4BUkWOzjLDykZY2dhCO2aemlpFjs4yw8pGWNnYQjtmnppZDbeO6witi5K5LrYrVnhWzWCX7lKDZYje5tgIJeETgYw==
Accept: application/json
Accept-Language: zh-CN
X-Shopee-Language: zh-Hans
X-API-SOURCE: pc
```
上面是我用 rest client 发送的请求体，能得到正确的响应。
但同样的请求体我用代码发出，就得不到正确的响应，下面是代码构造的请求体：
```
GET /api/v4/item/get?itemid=20537120944&shopid=68461810 HTTP/1.1\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:106.0) Gecko/20100101 Firefox/106.0\r\nHost: shopee.com.my\r\nContent-Type: application/json\r\nX-Shopee-Language: zh-Hans\r\nX-API-SOURCE: pc\r\naf-ac-enc-dat: AAcyLjQuMS0yAAABhEYD7joAAAjHAcEAAAAAAAAAAOYyhFB7rjob26/8rq6jA0F3J6Kfg5aGEX+GYncix7fIyPghAefe3JS894jq/3nC9cJjpEn32HTqasIUhFkInWzoCOj1uSC5kl6LU06aSrm61kX/Ny1L5jzxFjDrS1IzPHwt9muZUbatRPTf42k24UXHBZsir4fwWxQLVKw5gDu5CyybpSVWFCd7OLsY30Hj1OjSKZvDNTpkAhYqvdOyLCTGr41kHyFGV3ZaoQ01NX1u6R9AnBG6X9s1ynZK6vnTBgzBIOKTNFS4j1VT8sOl1BEtObri8ZUW3OTOHeCO4vGDCq4gRJFmvwSm1BNdccjxAekgEx3xwroP6ZL6LO5bh9QSxuKGYkUmR84CcHLB6dmMPnXDUGkagca9MFiK8RmRsrN2vcLDNTpkAhYqvdOyLCTGr41kKUmjTinalW5/ctjHa7Lte+06J5ekdC078Iv4wrMjrvbzUjYNqi2Hdu8tLPGrNL/jmEfixe8rpESf8+9J+WOK8kusILDBjMDq/xa+8hI9GWbdxIdVmB5payUD+EtCC4BUkWOzjLDykZY2dhCO2aemlpFjs4yw8pGWNnYQjtmnppZDbeO6witi5K5LrYrVnhWzWCX7lKDZYje5tgIJeETgYw==\r\nAccept: application/json\r\nAccept-Language: zh-CN\r\n\r\n
```
实再是不知道问题出在哪里，明明是一模一样的请求体，浏览器和rest client能获取正常响应，go或python却不能。 检查一下看是不是http2.0 最简单办法，挂fd
https://img01-xusong.91q.com/AEB47220-2379-48DE-869D-52FD3F99763A.png
import requests
requests.packages.urllib3.disable_warnings()
def get_item(itemid, shopid):
url = 'http://shopee.com.my/api/v4/item/get'
params = {
'itemid': itemid,
'shopid': shopid
}
headers = {
'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:106.0) Gecko/20100101 Firefox/106.0',
'Content-Type': 'application/json',
'af-ac-enc-dat': 'AAcyLjQuMS0yAAABhEYD7joAAAjHAcEAAAAAAAAAAOYyhFB7rjob26/8rq6jA0F3J6Kfg5aGEX+GYncix7fIyPghAefe3JS894jq/3nC9cJjpEn32HTqasIUhFkInWzoCOj1uSC5kl6LU06aSrm61kX/Ny1L5jzxFjDrS1IzPHwt9muZUbatRPTf42k24UXHBZsir4fwWxQLVKw5gDu5CyybpSVWFCd7OLsY30Hj1OjSKZvDNTpkAhYqvdOyLCTGr41kHyFGV3ZaoQ01NX1u6R9AnBG6X9s1ynZK6vnTBgzBIOKTNFS4j1VT8sOl1BEtObri8ZUW3OTOHeCO4vGDCq4gRJFmvwSm1BNdccjxAekgEx3xwroP6ZL6LO5bh9QSxuKGYkUmR84CcHLB6dmMPnXDUGkagca9MFiK8RmRsrN2vcLDNTpkAhYqvdOyLCTGr41kKUmjTinalW5/ctjHa7Lte+06J5ekdC078Iv4wrMjrvbzUjYNqi2Hdu8tLPGrNL/jmEfixe8rpESf8+9J+WOK8kusILDBjMDq/xa+8hI9GWbdxIdVmB5payUD+EtCC4BUkWOzjLDykZY2dhCO2aemlpFjs4yw8pGWNnYQjtmnppZDbeO6witi5K5LrYrVnhWzWCX7lKDZYje5tgIJeETgYw==',
'Accept': 'application/json',
'Accept-Language': 'zh-CN',
'X-Shopee-Language': 'zh-Hans',
'X-API-SOURCE': 'pc'
}
#FD代{过}{滤}理端口
proxies = {
'http': 'http://127.0.0.1:8887',
'https': 'http://127.0.0.1:8887'
}
r = requests.get(url, params=params, headers=headers, proxies=proxies, verify=False)
print(r.json())
if __name__ == '__main__':
print(get_item(20537120944, 68461810))
https://img01-xusong.91q.com/EF462521-776A-4EAB-8F52-9B57E15E0153.png
https://img01-xusong.91q.com/0BE07446-AC66-42DB-981C-2625DD7702E7.png 有个重定向,需要处理一下 我是不会改名的 发表于 2022-11-6 00:15
最简单办法，挂fd
import requests
我写错地址了，本来就是https，不存在重定向的问题。
curl无需任何代{过}{滤}理都能获得正确响应，python却不行，这是我很好奇的点。
gorkys 发表于 2022-11-5 23:20
检查一下看是不是http2.0
不是2.0的问题 花了几十分钟帮你看了下，该网站有一定的反扒手段，初步判断请求不能太频繁。
requests方法确实不行。
这个能跑，和2楼一样，加上代{过}{滤}理请求成功的概率会大一些。
import http.client
import json
conn = http.client.HTTPSConnection("shopee.com.my")
payload = ''
headers = {
'sec-ch-ua': '"Google Chrome";v="107", "Chromium";v="107", "Not=A?Brand";v="24"',
'sz-token': 'Ra/NJSFhMbpYXlWXB/kMKw==|tbtAeRuQraLaHXP0PPLER62V4RREUxTHlM0sHpOnYMwbMMQeM9+qLgkomlUGpUVkjby5btMmGHCl9DFdTYOavUNJ3LuXv0sobg==|oZGZsgOMDe5oaMrE|06|3',
'sec-ch-ua-mobile': '?0',
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36',
'Content-Type': 'application/json',
'X-API-SOURCE': 'pc',
'Accept': 'application/json',
'X-Shopee-Language': 'zh-Hans',
'X-Requested-With': 'XMLHttpRequest',
'af-ac-enc-dat': 'AAcyLjQuMS0yAAABhE3Hy6UAAAtLAkIAAAAAAAAAAOYyhFAbVQMMpIKa2+dGIBkKaWUVkWOzjLDykZY2dhCO2aemll6zSawP7dqKio0tSXU8mzeKLk1kA/WyNZ3Ie5A5Nt4iJ1W9aNhMRfWeQ7v8d9m7rMmAzdpxzPCKsKZUt89X1xFf0TvL/NsDCZSiE3F0ERsJBPsctX1CYA4baakfSx+Mbik3yDbxzfoZ9uXVS6oq2etzh1XniTb6mLKOF6kwSZbxI4FnzTS4k+XtcD7yPyi9O9arrEYGHKDT1AgxhDj+TLRcJqaQNDvlDA7IiJvpFsgpeHQXx+lEECXlHknmoTTyj/Vn1dcMyFsj0rpApuO6LdYYfR0WvgE2pJEUSvIqWlCgTPpSIKBuWK+C0hzfGe1o4iEHKgzvRydrIs0DhQk1YcG18ylDmIpexxyWlfUaRjfn5DtSUG9QIVLUSI3R8+2Vi3kGkiVgGH5vlT424PFUtNdb+8K1kDEitmBAZRnJhX2FvN44sM3YZnxNQDhfDc5iLQNlRwi12gPtyiO0aJ0u0EHc6LpdriVstGniIcACAExdmmlMQ+JYTMHF6kF68iRD85aWYi9ro3Wl9b3oSySuEOrDBjHFr5rF050g933uBLbAAhpLfIO+hSPwGUPsJ9/+NzAlsFLDIzfdYGNDP9z6GA2N8G6rj84L/ADQ7yqJhWpZ4w6ktdU1QBJ19JM+JjxlMGGa5/CKeBxPVMGF0rhbVQGRY7OMsPKRljZ2EI7Zp6aWkWOzjLDykZY2dhCO2aemlhn+1ZL5DKAyhUWhwsV2anPVK717vjBfA5ZAs0C9mW0Z',
'X-CSRFToken': 'IDq3287GJnXS5JJj5c2p5NfuvByGNpst',
'sec-ch-ua-platform': '"Windows"',
'Sec-Fetch-Site': 'same-origin',
'Sec-Fetch-Mode': 'cors',
'Sec-Fetch-Dest': 'empty',
'host': 'shopee.com.my',
'Cookie': 'REC_T_ID=99d3d94f-5df0-11ed-b735-2cea7fa8daa9; SPC_F=w4AfWBkq1JrakV2jppFicIhTClRLblU7; SPC_R_T_ID=UXubTQvc/YPsM6xl+teVYis0dA2qLMfX6XkESPWMhmcIFaEi8LrMjdy7JotmJURfc8VtNc6ecvjQ5h0QxzshCQkSOwSRbEYbPzopwt27ggZispI0Ud4iKQQFka+NykFRJH0vM8M04lQUpSa5SxdHmdirR2k7laqgj3MTGLHCqTQ=; SPC_R_T_IV=R0Y1N1IwUUFYa1VqTjJRTw==; SPC_SI=m5xjYwAAAABKQ2FmemF6RlFAiQAAAAAARjNNa3hrdEw=; SPC_T_ID=UXubTQvc/YPsM6xl+teVYis0dA2qLMfX6XkESPWMhmcIFaEi8LrMjdy7JotmJURfc8VtNc6ecvjQ5h0QxzshCQkSOwSRbEYbPzopwt27ggZispI0Ud4iKQQFka+NykFRJH0vM8M04lQUpSa5SxdHmdirR2k7laqgj3MTGLHCqTQ=; SPC_T_IV=R0Y1N1IwUUFYa1VqTjJRTw=='
}
conn.request("GET", "/api/v4/item/get?itemid=9037931641&shopid=41985388", payload, headers)
res = conn.getresponse()
data = res.read()
print(data.decode("utf-8")) 建议你多用下postman或是fidder什么的，fidder可以全局抓，抓到你代码的结果，然后和你浏览器的结果比较。
返回结果此次都没报错。
页:
[1]
2