本帖最后由 lovecloudssh 于 2020-3-10 19:37 编辑
看着大家都在分享，我也分享一个。还在学习中。。。。。。写的比较乱
爬取51job工作信息，包含工作、公司、发布时间、薪水、工作地点等，薪水。
[Python] 纯文本查看 复制代码 # coding:utf-8
import requests
import re
from lxml import etree
new_headers={"User-Agent":"Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36"}
fp=open("51job.txt",'w+',encoding="utf-8")
for i in range(0,15):
print("正在爬取第"+str(i)+"页")
url = "https://search.51job.com/list/090200,000000,0000,00,9,99,Python,2,"+str(i)+".html?lang=c&postchannel=0000&workyear=99&cotype=99°reefrom=99&jobterm=99&companysize=99&ord_field=0&dibiaoid=0&line=&welfare="
response=requests.get(url,headers=new_headers)
response.encoding='gbk'
html=etree.HTML(response.text)
jobname = html.xpath("//a[@onmousedown='']/@title")
jobsalary = html.xpath('//div[@class="el"]/span[@class="t4"]')
jobcompany = html.xpath("//span[@class='t2']/a/text()")
jobtime =html.xpath("//div[@class='el']/span[@class='t5']/text()")
jobaddress=html.xpath("//div[@class='el']/span[@class='t3']/text()")
for j in range(0,len(jobname)):
dic={'工作名称':jobname[j],'薪水':jobsalary[j].text,'工作地点':jobaddress[j],'公司':jobcompany[j],'发布时间':jobtime[j]}
fp.write(str(dic)+'\n')
fp.close() | 
发表于 2020-3-10 19:23
|
发表于 2020-3-11 17:09
