Python 请教selenium的用法!?

hahawangzi · 发表于 2020-4-17 10:19

本帖最后由 hahawangzi 于 2020-5-12 15:51 编辑

import requests
from bs4 import BeautifulSoup
import re
import json

url = 'http://www.dm5.com/m11076/'
headers={
      'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36',
      'Referer':'http://www.dm5.com/'
         }
r = requests.get(url,headers=headers)
soup = BeautifulSoup(r.text,'html.parser')
img = soup.find_all('div',class_=re.compile('view-+$'),id="showimage")
for i in img:
print(i)
我用
from selenium import webdriver
driver=webdriver.Chrome()
driver.get("http://www.dm5.com/m987161/")
print(driver.page_source)
源码里面还是没有图片地址，难道只能一张张截图了吗？

pzx521521 · 发表于 2020-4-17 10:27

正常操作撒
后台控制的, 如果key对应不上就不返回图片数据
抓包前一般在先在浏览器分析一下哦
图片地址都是带cid 和key 这个没什么问题的~

Liu4529 · 发表于 2020-4-17 10:28

看了一下地址, 中间的大图是通过js动态渲染加载的你直接拿源码肯定没有

xiao智可以不帅 · 发表于 2020-4-17 10:45

你还是用Selenium

葫芦炒鸡蛋 · 发表于 2020-4-17 11:25

老老实实分析js代码或者直接用Selenium、PyQt5加载网页。
友情提示：Python中正则不用编译，看源码便知。

ixsec · 发表于 2020-4-17 11:41

网址有script 好好看看哪个里面有需要的组成页面的所有东西，就是顺序有问题而已~
需要你自己再分析一下

ixsec · 发表于 2020-4-17 11:43

[JavaScript] 纯文本查看 复制代码

cdndm5          99     com     piaoliujiaoshi     specials     manhua1001     http     61     50     174     1622b91090844e425ca2f2139c72971a     key     4B6E3D3668F4F28A1182A28EC8D94FAC45715C68E30FAEA4D121348B0C3CE11C     type     11076     uk     jpg     cid     
0219044747_94824     0219044747_59668     0219044747_61662     0219044747_82966     0219044747_87508     0219044747_32426     0219044747_45820     0219044747_21426     0219044747_19237     0219044747_98862     0219044747_86410     0219044747_75429     0219044747_42385     0219044747_22825     0219044747_45847     0219044747_56343     0219044747_76773     0219044747_15774     0219044747_69095     0219044747_12925     0219044747_36366     0219044747_11224     0219044747_72554     0219044747_21123     0219044747_96988     0219044646_29746     0219044646_16425     0219044646_54290     0219044646_46134     0219044646_96522     0219044646_48812     0219044646_11102     
newImgs     var      
0219044646_93046     0219044646_16913     0219044646_32871     0219044646_18518     0219044707_26863     0219044706_58124     0219044704_44364     0219044747_85866     0219044747_90858     0219044709_90688     0219044646_68028     0219044646_33164     0219044646_56920     0219044702_19913     0219044701_65280     0219044700_69955     0219044848_15330     0219044848_33781     0219044848_34215     0219044848_31538     0219044848_74807     0219044848_43216     0219044848_37838     0219044848_57522     0219044848_94199     0219044848_61486     0219044848_83234     0219044848_38220     0219044848_30939     0219044848_80132     0219044848_84322     0219044848_44403     0219044848_77554     0219044848_88810     0219044848_63075     0219044848_51192     0219044848_84146     0219044848_85505     0219044848_83355     0219044848_40746     0219044801_45398     0219044747_37408     0219044747_34471     0219044805_97807     0219044803_81456     0219044802_92227     0219044747_48049     0219044747_39608     0219044747_43225     0219044747_85597     0219044747_61200     0219044747_60526     0219044848_60668     0219044848_67618     0219044848_31458     0219044848_19159     0219044848_49450     0219044848_18199     0219044809_39333     0219044808_52623     0219044806_59493     0219044848_57728     0219044848_32267     0219044848_21429

下面这个是图片地址

[JavaScript] 纯文本查看 复制代码

http://manhua1001-61-174-50-99.cdndm5.com/specials/p/piaoliujiaoshi/0219044646_93046.jpg?cid=11076&key=1622b91090844e425ca2f2139c72971a&type=1&uk=4B6E3D3668F4F28A1182A28EC8D94FACF6264725111DA400397ABD0A187271BB

[img]http://manhua1001-61-174-50-99.cdndm5.com/specials/p/piaoliujiaoshi/0219044646_54290.jpg[/img]?cid=11076&key=1622b91090844e425ca2f2139c72971a&type=1&uk=4B6E3D3668F4F28A1182A28EC8D94FACF6264725111DA400D18C3D1A105B1CB2

辣丝丝小白菜 · 发表于 2020-4-17 11:56

[Python] 纯文本查看 复制代码

import execjs
import requests

with open('1.js','r') as fp:
	ctx = execjs.compile(fp.read())
a = ctx.call('dm5imagefun')

headers = {
	'Referer': 'http://www.dm5.com/m11076-p2/',
	'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.25 Safari/537.36 Core/1.70.3756.400 QQBrowser/10.5.4039.400'
}

for i,url in enumerate(a):
	res = requests.get(url, headers=headers)
	with open(f'{i}.jpg', 'wb') as fpp:
		fpp.write(res.content)

1.js 文件是http://www.dm5.com/m11076-p2/chapterfun.ashx?cid=*******这个请求获取的，这个请求在chapternew_v22.js中发送。
这样可以直接获取所有的链接url

hahawangzi · 发表于 2020-4-17 15:35

辣丝丝小白菜发表于 2020-4-17 11:56
[mw_shl_code=python,true]import execjs
import requests

老师，能在详细点吗？怎么请求这个http://www.dm5.com/m11076-p2/chapterfun.ashx?cid=******* 获得 1.js文件

hahawangzi · 发表于 2020-4-17 15:44

辣丝丝小白菜发表于 2020-4-17 11:56
[mw_shl_code=python,true]import execjs
import requests

老师，能在详细点吗？怎么请求这个http://www.dm5.com/m11076-p2/chapterfun.ashx?cid=******* 获得 1.js文件

帐号		自动登录	找回密码
密码			注册[Register]

[求助] Python 请教selenium的用法!?

本帖被以下淘专辑推荐: