本帖最后由 tuhaojun 于 2020-9-18 21:32 编辑
新手，学了两三天python，爬了个比较容易爬的小说网，笔趣阁有点坑。爬着爬着就505 不知道什么机制。
看了看别人学了两三天python跟我这个比起来 简直了
请求大佬指点。
[Python] 纯文本查看 复制代码 import requests
from bs4 import BeautifulSoup
import lxml
import re
#爬取天籁小说网小说
#url = 'https://www.23txt.com/files/article/html/60/60792/'
url = input('网址URL：\n本脚本仅支持天籁小说 \n类型：https://www.23txt.com/files/article/html/**/*****/ \n')
url_fix = input('类型：**/******/ \n')
req = requests.get (url)
req.encoding = 'GBK'
text = req.text
bf = BeautifulSoup (text,'lxml')
title = bf.find_all('div',id = 'info')
title = bf.find_all('h1')
title = title[0].text.replace ('h1',' ')
text_info = re.findall(r'<div id="list">.*?</div>',text,re.S)[0]
text_info_list = re.findall (r'<dd><a href="/files/article/html/%s(.*?)" >(.*?)</a>'% url_fix ,text_info)
for book in text_info_list:
text_url = book[0]
text_name = book[1]
text_download = url + text_url
reqs = requests.get(text_download)
reqs.encoding = 'GBK'
texts = reqs.text
texts_info = re.findall(r'<div id="content">(.*?)</div>',texts,re.S)[0]
texts_info = texts_info.replace (' ',' ')
texts_info = texts_info.replace (';',' \n')
texts_info = texts_info.replace ('<br>',' ')
with open ('%s.txt'% title,'a+',encoding = 'GBK')as f:
f.write ('\n\n')
f.write (text_name)
f.write (texts_info)
f.write ('\n')
print (text_name)
print ('下载完成')
| 
[复制链接]
发表于 2020-9-18 21:29
|
发表于 2020-9-22 20:56
