网站 › 『编程语言区』 › 某视频网站抓取排行榜视频

aiyamaya 发表于 2022-5-5 18:13

某视频网站抓取排行榜视频

程序默认下载目录为D盘根目录，请指正

import requests
from lxml import etree
import time

url ='https://www.pearvideo.com/popular'
raw_url = 'https://www.pearvideo.com/'

respon = requests.get(url)
main_page = etree.HTML(respon.text)
name_list = main_page.xpath("/html/body/div/div/div/ul")

for i in name_list:# 各个榜单的名字和代码
    pop_name = i.xpath("./li/a/text()")
    print('榜单为：',pop_name)
    pop_num = i.xpath("./li/a/@href")
    # print(pop_num)
   

input_name = input('输入榜单名称：')
if input_name in pop_name:
    pop_page_url = raw_url+pop_num
else:
    pop_page_url = url
    print('将使用默认总榜！')

respon = requests.get(pop_page_url)
top_page = etree.HTML(respon.text)
top_list = top_page.xpath('//*[@id="popularList"]')
for j in top_list:# 榜单的视频id
    vedio_id = j.xpath("./li/div/div/span/@data-id")
    file_name = j.xpath('./li/div/a/h2/text()')

for k in vedio_id:
    url = f'https://www.pearvideo.com/video_{k}'
    XHR_url = f'https://www.pearvideo.com/videoStatus.jsp?contId={k}&mrd=0.12841592667885604'

    head = {'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:99.0) Gecko/20100101 Firefox/99.0',
            'Referer':url}
    respon = requests.get(XHR_url,headers=head)
    systemTime = respon.json()['systemTime']
    srcUrl = respon.json()['videoInfo']['videos']['srcUrl']
    srcUrl = srcUrl.replace(systemTime,f'cont-{k}')
    output_name = file_name
    f = open(f'd:/{output_name}.mp4',mode='wb')
    f.write(requests.get(srcUrl).content)
    f.close()
    print(output_name,'下载完毕！')
    time.sleep(2)

respon.close()

wangjj5741 发表于 2022-5-5 21:14

我不懂，进来看一下

nihaohello1 发表于 2022-5-5 21:42

给个大大的赞

周师傅 发表于 2022-5-5 22:19

必须给个赞，必须推荐！！！

ych13846701169 发表于 2022-5-5 22:31

了解了，谢谢分享

xi0729 发表于 2022-5-5 22:50

必须给个赞

lengyanwl 发表于 2022-5-5 23:02

看看~感谢分享

xingwenzhiyuan 发表于 2022-5-6 06:40

进来学习一下，谢谢你的分享

cy2931 发表于 2022-5-6 11:00

学习一下

basovusr 发表于 2022-5-16 16:57

感谢分享，学习了。

查看完整版本: 某视频网站抓取排行榜视频