遇到了一个奇怪的请求问题请高手能帮忙解惑

thepoy

请求如下：

GET https://shopee.com.my/api/v4/item/get?itemid=20537120944&shopid=68461810
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:106.0) Gecko/20100101 Firefox/106.0
Host: shopee.com.my
Content-Type: application/json
af-ac-enc-dat: AAcyLjQuMS0yAAABhEYD7joAAAjHAcEAAAAAAAAAAOYyhFB7rjob26/8rq6jA0F3J6Kfg5aGEX+GYncix7fIyPghAefe3JS894jq/3nC9cJjpEn32HTqasIUhFkInWzoCOj1uSC5kl6LU06aSrm61kX/Ny1L5jzxFjDrS1IzPHwt9muZUbatRPTf42k24UXHBZsir4fwWxQLVKw5gDu5CyybpSVWFCd7OLsY30Hj1OjSKZvDNTpkAhYqvdOyLCTGr41kHyFGV3ZaoQ01NX1u6R9AnBG6X9s1ynZK6vnTBgzBIOKTNFS4j1VT8sOl1BEtObri8ZUW3OTOHeCO4vGDCq4gRJFmvwSm1BNdccjxAekgEx3xwroP6ZL6LO5bh9QSxuKGYkUmR84CcHLB6dmMPnXDUGkagca9MFiK8RmRsrN2vcLDNTpkAhYqvdOyLCTGr41kKUmjTinalW5/ctjHa7Lte+06J5ekdC078Iv4wrMjrvbzUjYNqi2Hdu8tLPGrNL/jmEfixe8rpESf8+9J+WOK8kusILDBjMDq/xa+8hI9GWbdxIdVmB5payUD+EtCC4BUkWOzjLDykZY2dhCO2aemlpFjs4yw8pGWNnYQjtmnppZDbeO6witi5K5LrYrVnhWzWCX7lKDZYje5tgIJeETgYw==
Accept: application/json
Accept-Language: zh-CN
X-Shopee-Language: zh-Hans
X-API-SOURCE: pc

上面是我用 rest client 发送的请求体，能得到正确的响应。

但同样的请求体我用代码发出，就得不到正确的响应，下面是代码构造的请求体：

GET /api/v4/item/get?itemid=20537120944&shopid=68461810 HTTP/1.1\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:106.0) Gecko/20100101 Firefox/106.0\r\nHost: shopee.com.my\r\nContent-Type: application/json\r\nX-Shopee-Language: zh-Hans\r\nX-API-SOURCE: pc\r\naf-ac-enc-dat: AAcyLjQuMS0yAAABhEYD7joAAAjHAcEAAAAAAAAAAOYyhFB7rjob26/8rq6jA0F3J6Kfg5aGEX+GYncix7fIyPghAefe3JS894jq/3nC9cJjpEn32HTqasIUhFkInWzoCOj1uSC5kl6LU06aSrm61kX/Ny1L5jzxFjDrS1IzPHwt9muZUbatRPTf42k24UXHBZsir4fwWxQLVKw5gDu5CyybpSVWFCd7OLsY30Hj1OjSKZvDNTpkAhYqvdOyLCTGr41kHyFGV3ZaoQ01NX1u6R9AnBG6X9s1ynZK6vnTBgzBIOKTNFS4j1VT8sOl1BEtObri8ZUW3OTOHeCO4vGDCq4gRJFmvwSm1BNdccjxAekgEx3xwroP6ZL6LO5bh9QSxuKGYkUmR84CcHLB6dmMPnXDUGkagca9MFiK8RmRsrN2vcLDNTpkAhYqvdOyLCTGr41kKUmjTinalW5/ctjHa7Lte+06J5ekdC078Iv4wrMjrvbzUjYNqi2Hdu8tLPGrNL/jmEfixe8rpESf8+9J+WOK8kusILDBjMDq/xa+8hI9GWbdxIdVmB5payUD+EtCC4BUkWOzjLDykZY2dhCO2aemlpFjs4yw8pGWNnYQjtmnppZDbeO6witi5K5LrYrVnhWzWCX7lKDZYje5tgIJeETgYw==\r\nAccept: application/json\r\nAccept-Language: zh-CN\r\n\r\n

实再是不知道问题出在哪里，明明是一模一样的请求体，浏览器和rest client能获取正常响应，go或python却不能。

gorkys

检查一下看是不是http2.0

我是不会改名的

最简单办法，挂fd

[Python] 纯文本查看 复制代码

import requests
requests.packages.urllib3.disable_warnings()
def get_item(itemid, shopid):
    url = 'http://shopee.com.my/api/v4/item/get'
    params = {
        'itemid': itemid,
        'shopid': shopid
    }
    headers = {
        'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:106.0) Gecko/20100101 Firefox/106.0',
        'Content-Type': 'application/json',
        'af-ac-enc-dat': 'AAcyLjQuMS0yAAABhEYD7joAAAjHAcEAAAAAAAAAAOYyhFB7rjob26/8rq6jA0F3J6Kfg5aGEX+GYncix7fIyPghAefe3JS894jq/3nC9cJjpEn32HTqasIUhFkInWzoCOj1uSC5kl6LU06aSrm61kX/Ny1L5jzxFjDrS1IzPHwt9muZUbatRPTf42k24UXHBZsir4fwWxQLVKw5gDu5CyybpSVWFCd7OLsY30Hj1OjSKZvDNTpkAhYqvdOyLCTGr41kHyFGV3ZaoQ01NX1u6R9AnBG6X9s1ynZK6vnTBgzBIOKTNFS4j1VT8sOl1BEtObri8ZUW3OTOHeCO4vGDCq4gRJFmvwSm1BNdccjxAekgEx3xwroP6ZL6LO5bh9QSxuKGYkUmR84CcHLB6dmMPnXDUGkagca9MFiK8RmRsrN2vcLDNTpkAhYqvdOyLCTGr41kKUmjTinalW5/ctjHa7Lte+06J5ekdC078Iv4wrMjrvbzUjYNqi2Hdu8tLPGrNL/jmEfixe8rpESf8+9J+WOK8kusILDBjMDq/xa+8hI9GWbdxIdVmB5payUD+EtCC4BUkWOzjLDykZY2dhCO2aemlpFjs4yw8pGWNnYQjtmnppZDbeO6witi5K5LrYrVnhWzWCX7lKDZYje5tgIJeETgYw==',
        'Accept': 'application/json',
        'Accept-Language': 'zh-CN',
        'X-Shopee-Language': 'zh-Hans',
        'X-API-SOURCE': 'pc'
    }
    #FD代{过}{滤}理端口
    proxies = {
        'http': 'http://127.0.0.1:8887',
        'https': 'http://127.0.0.1:8887'
    }
    r = requests.get(url, params=params, headers=headers, proxies=proxies, verify=False)
    print(r.json())
if __name__ == '__main__':
    print(get_item(20537120944, 68461810))

wg521125

有个重定向,需要处理一下

thepoy

我是不会改名的 发表于 2022-11-6 00:15
最简单办法，挂fd

[mw_shl_code=python,true]import requests

我写错地址了，本来就是https，不存在重定向的问题。
curl无需任何代{过}{滤}理都能获得正确响应，python却不行，这是我很好奇的点。

thepoy

gorkys 发表于 2022-11-5 23:20
检查一下看是不是http2.0

不是2.0的问题

lineulb

花了几十分钟帮你看了下，该网站有一定的反扒手段，初步判断请求不能太频繁。
requests方法确实不行。
这个能跑，和2楼一样，加上代{过}{滤}理请求成功的概率会大一些。

[Python] 纯文本查看 复制代码

import http.client
import json

conn = http.client.HTTPSConnection("shopee.com.my")
payload = ''
headers = {
  'sec-ch-ua': '"Google Chrome";v="107", "Chromium";v="107", "Not=A?Brand";v="24"',
  'sz-token': 'Ra/NJSFhMbpYXlWXB/kMKw==|tbtAeRuQraLaHXP0PPLER62V4RREUxTHlM0sHpOnYMwbMMQeM9+qLgkomlUGpUVkjby5btMmGHCl9DFdTYOavUNJ3LuXv0sobg==|oZGZsgOMDe5oaMrE|06|3',
  'sec-ch-ua-mobile': '?0',
  'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36',
  'Content-Type': 'application/json',
  'X-API-SOURCE': 'pc',
  'Accept': 'application/json',
  'X-Shopee-Language': 'zh-Hans',
  'X-Requested-With': 'XMLHttpRequest',
  'af-ac-enc-dat': 'AAcyLjQuMS0yAAABhE3Hy6UAAAtLAkIAAAAAAAAAAOYyhFAbVQMMpIKa2+dGIBkKaWUVkWOzjLDykZY2dhCO2aemll6zSawP7dqKio0tSXU8mzeKLk1kA/WyNZ3Ie5A5Nt4iJ1W9aNhMRfWeQ7v8d9m7rMmAzdpxzPCKsKZUt89X1xFf0TvL/NsDCZSiE3F0ERsJBPsctX1CYA4baakfSx+Mbik3yDbxzfoZ9uXVS6oq2etzh1XniTb6mLKOF6kwSZbxI4FnzTS4k+XtcD7yPyi9O9arrEYGHKDT1AgxhDj+TLRcJqaQNDvlDA7IiJvpFsgpeHQXx+lEECXlHknmoTTyj/Vn1dcMyFsj0rpApuO6LdYYfR0WvgE2pJEUSvIqWlCgTPpSIKBuWK+C0hzfGe1o4iEHKgzvRydrIs0DhQk1YcG18ylDmIpexxyWlfUaRjfn5DtSUG9QIVLUSI3R8+2Vi3kGkiVgGH5vlT424PFUtNdb+8K1kDEitmBAZRnJhX2FvN44sM3YZnxNQDhfDc5iLQNlRwi12gPtyiO0aJ0u0EHc6LpdriVstGniIcACAExdmmlMQ+JYTMHF6kF68iRD85aWYi9ro3Wl9b3oSySuEOrDBjHFr5rF050g933uBLbAAhpLfIO+hSPwGUPsJ9/+NzAlsFLDIzfdYGNDP9z6GA2N8G6rj84L/ADQ7yqJhWpZ4w6ktdU1QBJ19JM+JjxlMGGa5/CKeBxPVMGF0rhbVQGRY7OMsPKRljZ2EI7Zp6aWkWOzjLDykZY2dhCO2aemlhn+1ZL5DKAyhUWhwsV2anPVK717vjBfA5ZAs0C9mW0Z',
  'X-CSRFToken': 'IDq3287GJnXS5JJj5c2p5NfuvByGNpst',
  'sec-ch-ua-platform': '"Windows"',
  'Sec-Fetch-Site': 'same-origin',
  'Sec-Fetch-Mode': 'cors',
  'Sec-Fetch-Dest': 'empty',
  'host': 'shopee.com.my',
  'Cookie': 'REC_T_ID=99d3d94f-5df0-11ed-b735-2cea7fa8daa9; SPC_F=w4AfWBkq1JrakV2jppFicIhTClRLblU7; SPC_R_T_ID=UXubTQvc/YPsM6xl+teVYis0dA2qLMfX6XkESPWMhmcIFaEi8LrMjdy7JotmJURfc8VtNc6ecvjQ5h0QxzshCQkSOwSRbEYbPzopwt27ggZispI0Ud4iKQQFka+NykFRJH0vM8M04lQUpSa5SxdHmdirR2k7laqgj3MTGLHCqTQ=; SPC_R_T_IV=R0Y1N1IwUUFYa1VqTjJRTw==; SPC_SI=m5xjYwAAAABKQ2FmemF6RlFAiQAAAAAARjNNa3hrdEw=; SPC_T_ID=UXubTQvc/YPsM6xl+teVYis0dA2qLMfX6XkESPWMhmcIFaEi8LrMjdy7JotmJURfc8VtNc6ecvjQ5h0QxzshCQkSOwSRbEYbPzopwt27ggZispI0Ud4iKQQFka+NykFRJH0vM8M04lQUpSa5SxdHmdirR2k7laqgj3MTGLHCqTQ=; SPC_T_IV=R0Y1N1IwUUFYa1VqTjJRTw=='
}
conn.request("GET", "/api/v4/item/get?itemid=9037931641&shopid=41985388", payload, headers)
res = conn.getresponse()
data = res.read()
print(data.decode("utf-8"))

lineulb

建议你多用下postman或是fidder什么的，fidder可以全局抓，抓到你代码的结果，然后和你浏览器的结果比较。

lineulb

返回结果此次都没报错。