【Python】小说爬虫界面版(各种BUG已修复)

luoluoovo

之前写的那几篇爬虫就是最终实现GUI版的！终于大功告成！
(找了很久，好像没有见过这类的python写的爬虫GUI，分享一下！)
以前只是自己的笔记吧！今天有一个能和大家分享的东西了！用pyinstall打包成exe了！
界面很简陋！因为python的GUI太难搞了！(偷懒)
(GUI版本比.py下载速度慢很多)
(有个问题，就是下载时程序动一下就未响应，等下载好了就又恢复了，是BUG还是正常)
(有人提出来了问题了！感谢！这样才有接下来努力的动力，关于程序未响应，速度慢的问题，如果有人有能力可以帮助一下改进！谢谢)
(今天修改了许多BUG，界面颜色换了，这次带了控制台，可以查看下载情况了(下载到E盘的txt里))
源码如下:

[Python] 纯文本查看 复制代码

import tkinter as tk
import re
import urllib.request
import os
import time
from urllib.parse import quote
from tkinter import *
import tkinter.messagebox
def search1():
    search = entryUrl.get()
    search_Book = "https://www.qidian.com/search?kw=" + quote(search)
    if search=="":
        tkinter.messagebox.showinfo('错误', "请输入书名！")
    else:
        req = urllib.request.Request(search_Book)
        req.add_header = {
            'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:30.0) Gecko/20100101 Firefox/30.0'
        }
        page = urllib.request.urlopen(req).read()
        txt = page.decode('utf-8')
        filter_bookname = r'<h4><a href=".+?" target="_blank" data-eid=".+?" data-bid=".+?" data-algrid="0.0.0">(.+?)</h4>'
        book_name_nosub = re.findall(filter_bookname, txt, re.S)
        book_name_sub = re.sub(r'<cite class="red-kw">|</a>|</cite>', '', " ".join(book_name_nosub))
        book_name = book_name_sub.split(' ')
        filter_Chapter = r'<p class="update"><a href=".+?>(.+?)</a>'
        state_Chapter = re.findall(filter_Chapter, txt, re.S)
        filter_time = r'<em>·</em><span>(.+?)</span>'
        state_time = re.findall(filter_time, txt, re.S)
        list1 = ["   "] * 10
        list2 = ["   "] * 10
        number = ["0  ", "1  ", "2  ", "3  ", "4  ", "5  ", "6  ", "7  ", "8  ", "9  "]
        a = list(map(lambda c, x, a, y, b, z,: c + x + a + y + b + z, number, book_name, list1, state_Chapter, list2, state_time))
        b = "\n".join(a)
        var.set(b + "\n\n\n只限起点免费章节小说\n输入你要下载书的号数和章节(需5位,例;00100（第0项100章） --  书号+章节):")
    return search_Book
def mkdir(path):
    floder=os.path.exists(path)
    if not floder:
        os.makedirs(path)
        print("创建成功")
    else:
        print("文件已存在")
def download():
    img_path = "E:/txt/txt/"
    mkdir(img_path)
    search = entryUrl.get()
    search_Book = "https://www.qidian.com/search?kw=" + quote(search)
    req = urllib.request.Request(search_Book)
    req.add_header = {
        'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:30.0) Gecko/20100101 Firefox/30.0'
    }
    page = urllib.request.urlopen(req).read()
    txt = page.decode('utf-8')
    filter_book_id = r'<a class="red-btn" href="(.+?)"'
    book_id = re.findall(filter_book_id,txt, re.S)
    book_id_add = " http:".join(book_id)
    book_id_list = ("http:" + book_id_add).split(' ')
    book_id_love=book_id_list[int(str(entryUrl1.get()[0]))]
    page = urllib.request.urlopen(book_id_love).read()
    txt = page.decode('utf-8')
    filter_book_id = r'data-firstchapterjumpurl="(.+?)">'
    link = "http:" + re.findall(filter_book_id, txt, re.S)[0]
    z = 0
    url = []
    x=int(str(entryUrl1.get()[1])+str(entryUrl1.get()[2])+str(entryUrl1.get()[3])+str(entryUrl1.get()[4]))
    for read in (range(0, x)):
        url.append(link)
        req = urllib.request.Request(url[z])
        req.add_header = {
            'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:30.0) Gecko/20100101 Firefox/30.0'
        }
        page = urllib.request.urlopen(req, timeout=50).read().decode('UTF-8')
        filter_page = r'p>\u3000\u3000(.+?)<'  # 小说的文本  <p>　****<p>　\u3000 代表空格
        html = re.findall(filter_page, page, re.S)
        filter_bookname = r'60c;</em>(.+?)</a>'
        bookname = re.findall(filter_bookname, page, re.S)

        filter_chaptername = r'<h3 class="j_chapterName">(.+?)</h3>' 
        chaptername = re.findall(filter_chaptername, page, re.S)  
        i = 0
        for txt in html:
            line = html[i]
            f = open(img_path + chaptername[0] + ".txt", "a")
            f.write(line + "\n")
            f.close()
            i = i + 1
        entryUrl.insert('insert', "1")
        next = r'<a id="j_chapterNext".+?href="//(.+?)"'
        nextread = re.findall(next, page, re.S)
        b = ''
        link = "https://" + b.join(nextread)
        z = z + 1
        var.set( "下载完成")
    img_path2 = "E:/txt/" + bookname[0]
    if not os.path.exists(img_path2):
        os.rename(img_path, img_path2)
    else:
        path = "E:/txt/txt/"
        downloadtime = time.strftime("%Y%m%d%I%M%S", time.localtime())
        os.rename(img_path, img_path2 + downloadtime)
        print("文件名:" + bookname[0] + " 已存在,重命名为:" + bookname[0] + downloadtime + "\n" + "请勿重复操作")
window = tk.Tk()
window.title('小说爬虫——吾爱:luoluoovo')
var = tk.StringVar()
textLabel = Label(window, textvariable=var, bg='lightgreen', font=('Arial', 12), width=85,justify=LEFT,
             height=20)
textLabel.pack()
entryUrl = tk.Entry(window,width=37 )
entryUrl.place(x=3, y=372)
entryUrl1 = tk.Entry(window,width=37 )
entryUrl1.place(x=250, y=372)

b = tk.Button(window, text='下载', width=15,
               command=download)
c = tk.Button(window, text='搜索', width=15,
               command=search1)
window.minsize(750, 400)
window.maxsize(750,400)
b.pack(side=tk.RIGHT)
c.pack(side=tk.RIGHT)
window.mainloop()

已补链接 2018年7月5日17:15:01
界面版(修改版) 链接：https://pan.baidu.com/s/1xHdPN2XSzYKXwW0HqNQ5ig 密码：2adn
控制台(cmd)版  链接：https://pan.baidu.com/s/18UIkpbseRoOppWUFSpfOcg 密码：ebeu (无界面效率并且好用一点)
希望大家多多回复评分支持一下！(无教程，全手打！很累)

luoluoovo

qxzzxw 发表于 2018-7-3 22:55
谢谢楼主，不过小说txt都是放到一个txt里的，没有分开的。。这样分开的没人一个一个点开。然后就是只有免费 ...

因为前期做的是起点的网站，写到一半才发现这个问题，之后会继续写其他免费源的规则的！谢谢支持

luoluoovo

netCheney 发表于 2018-7-3 23:02
用pyinstall打包成EXE是亮点啊，之前写过，都是命令行版或者网页版的，没试过这个打包工具，弱弱的问一下楼 ...

不用的哦！好像是把库和命令打包成EXE，.py转换成exe很快的

凉米饭

学习下。谢谢楼主

凉米饭

学习下。谢谢楼主

jf950210

xuexixuexi

luoluoovo

w5645060 发表于 2018-7-3 21:25
绿色，辣眼睛。

绿色不是护眼吗

lunaaero

学习下。谢谢楼主

cosxiaobaitu

飘过一只书荒坛友  谢谢楼主分享

凉拌梨心儿

666，这个是界面用的tk？

zhaog1219

感谢楼主