在练习爬虫，爬取b站的视频下载链接遇到了问题

thepoy

https://api.bilibili.com/x/player/playurl?platform=pc&otype=json&avid=76065109&cid=130115422&qn=80&type=

以上面的页面为例，有两个视频链接，一主一备用。

{"url": "http://upos-hz-mirrorkodou.acgvideo.com/upgcxcode/22/54/130115422/130115422_nb2-1-80.flv?e=ig8euxZM2rNcNbu17zUVhoMahWuBhwdEto8g5X10ugNcXBlqNxHxNEVE5XREto8KqJZHUa6m5J0SqE85tZvEuENvNo8g2ENvNo8i8o859r1qXg8xNEVE5XREto8GuFGv2U7SuxI72X6fTr859r1qXg8gNEVE5XREto8z5JZC2X2gkX5L5F1eTX1jkXlsTXHeux_f2o859IB_&uipk=5&nbs=1&deadline=1574255875&gen=playurl&os=kodou&oi=1862266156&trid=1270425084864a8fbfab8561b6679a13u&platform=pc&upsig=e2986681352bc65be19d0f9fcc6ef025&uparams=e,uipk,nbs,deadline,gen,os,oi,trid,platform&mid=359098024",
"backup_url": ["http://upos-hz-mirrorcosu.acgvideo.com/upgcxcode/22/54/130115422/130115422_nb2-1-80.flv?e=ig8euxZM2rNcNbu17zUVhoMahWuBhwdEto8g5X10ugNcXBlqNxHxNEVE5XREto8KqJZHUa6m5J0SqE85tZvEuENvNo8g2ENvNo8i8o859r1qXg8xNEVE5XREto8GuFGv2U7SuxI72X6fTr859r1qXg8gNEVE5XREto8z5JZC2X2gkX5L5F1eTX1jkXlsTXHeux_f2o859IB_&uipk=5&nbs=1&deadline=1574255875&gen=playurl&os=cosu&oi=1862266156&trid=1270425084864a8fbfab8561b6679a13u&platform=pc&upsig=cda45f63812e43c9125d63e210d13d52&uparams=e,uipk,nbs,deadline,gen,os,oi,trid,platform&mid=359098024"]}





在页面中点击链接能够正常下载视频，但是把视频链接复制出来后，下载器就报403了，请问是怎么回事，如何解决呢？

52loli

估计是Referer的问题，UA也要设置一下

wty1641

cookie 问题，下载的时候把请求时候的cookie加进去

thepoy

wty1641 发表于 2019-11-20 20:04
cookie 问题，下载的时候把请求时候的cookie加进去

下载怎么加cookie

thepoy

iPhone8Plus 发表于 2019-11-20 20:02
哔哩哔哩的视频挺难抓的  抓到也没办法把它给下载下来

能下载的，b站根据ip所在地分配视频服务器，此api网址用美国ip打开，获取到的是下载直链。一旦换回国内，同样的json里的视频链接换了个服务器，还不止换了服务器，连后端规则都不太一样

wxb519774317

[Python] 纯文本查看 复制代码

1、直接将Cookie写在header头部
# coding:utf-8
import requests
from bs4 import BeautifulSoup
cookie = '''cisession=19dfd70a27ec0eecf1fe3fc2e48b7f91c7c83c60;CNZZDATA1000201968=181584
6425-1478580135-https%253A%252F%252Fwww.baidu.com%252F%7C1483922031;Hm_lvt_f805f7762a9a2
37a0deac37015e9f6d9=1482722012,1483926313;Hm_lpvt_f805f7762a9a237a0deac37015e9f6d9=14839
26368'''
header = {
'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Geck
o) Chrome/53.0.2785.143 Safari/537.36',
'Connection': 'keep-alive',
'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8',
'Cookie': cookie}
url = 'https://kankandou.com/book/view/22353.html'
wbdata = requests.get(url,headers=header).text
soup = BeautifulSoup(wbdata,'lxml')
print(soup)
2、使用requests插入Cookie
# coding:utf-8
import requests
from bs4 import BeautifulSoup
cookie = {
"cisession":"19dfd70a27ec0eecf1fe3fc2e48b7f91c7c83c60",
"CNZZDATA100020196":"1815846425-1478580135-https%253A%252F%252Fwww.baidu.com%252F%7C1483
922031",
"Hm_lvt_f805f7762a9a237a0deac37015e9f6d9":"1482722012,1483926313",
"Hm_lpvt_f805f7762a9a237a0deac37015e9f6d9":"1483926368"
}
url = 'https://kankandou.com/book/view/22353.html'
wbdata = requests.get(url,cookies=cookie).text
soup = BeautifulSoup(wbdata,'lxml')
print(soup)
这样我们就轻松的使用Cookie获取到了需要登录验证后才能浏览的网页和资源了。上面的Cookie是从自己浏览的网页中复制粘贴得到的。

参考自：《python爬虫实战入门教程》
————————————————

零点0511

weiguan  围观大神开挂

thepoy

wxb519774317 发表于 2019-11-20 20:31
[mw_shl_code=python,true]1、直接将Cookie写在header头部
# coding:utf-8
import requests

视频的下载链接在api页面内，cookie只能正常访问api页面，和下载视频是两回事，cookie这样用一般爬取文字信息和图片信息是够了，下载视频可能还差点意思