本帖最后由 完成大我 于 2023-2-7 15:58 编辑
续前贴【vue3】孩子一岁半了，压力很大，给她开发一个学习平台吧，持续更新
https://www.52pojie.cn/thread-1737171-1-1.html
(出处: 吾爱破解论坛)
楼主还沉浸在过年放假的氛围中，无心工作，这两天抽空写了古诗词模块+爬了一部分数据，把这部分码的代码分享一下。
1.古诗词模块需求背景
本来只打算做数学的，评论区看到一些建议，数学那块都是反复的工作量，正好换个心情，做点新东西。目前项目目录有点乱，东一块西一块，后面进度到一定程度后，重新整合一些目录结构。
| 需求编号 |
需求名称 |
需求描述 |
需求备注 |
| 1 |
古诗词 |
支持古诗词的增删改查；<br />列表点击详情，展示古诗词详情卡片，能够看到古诗词的所有详细信息；<br />详情内容还包含古诗词的视频讲解，视频采用第三方视频嵌入的方式；<br />需要爬取古诗词，及视频链接； |
|
2.技术方案
- 在之前制定的前后端技术框架的基础上编写本模块，数据考虑后期爬取的海量数据，需要使用mysql存取，通过后端服务实现增删改查。
- 古诗词有现成的库：https://gitee.com/mirrors/chinese-poetry/tree/master，是json文件的形式，只需要写脚本导到我的数据库里就可以
- 视频需要爬取，爬虫一般用python比较方便，库很全，以爬取bilibili视频为例，爬取时，使用诗词标题等关键词进行爬取，爬取的数据同样存为json文件，通过脚本导到数据库。
3.代码
前端代码
vue3很简单就能写好，视频播放组件试用<iframe>，src属性动态的绑定后端返回的视频链接；通过后端查询接口查询数据
[JavaScript] 纯文本查看 复制代码 <template>
<div>
<!-- 搜索条件 -->
<div>
<el-form :inline="true" :model="searchCondition" class="demo-form-inline">
<el-form-item label="古诗词名称">
<el-input v-model="searchCondition.title" placeholder="古诗词名称" />
</el-form-item>
<el-form-item label="作者">
<el-input v-model="searchCondition.author" placeholder="作者" />
</el-form-item>
<el-form-item label="标签">
<el-input v-model="searchCondition.tags" placeholder="标签" />
</el-form-item>
<el-form-item>
<el-button type="primary" @click="search">查询</el-button>
</el-form-item>
</el-form>
</div>
<!-- 列表 -->
<div>
<div>
<el-table :data="searchResult.record" stripe style="width: 100%"
:header-cell-style="{ 'background-color': '#F1F4FF', 'text-align': 'center' }"
:cell-style="{ 'text-align': 'center' }">
<el-table-column prop="title" label="诗名" width="240" />
<el-table-column prop="author" label="作者" width="240" />
<el-table-column label="标签" width="360">
<template #default="scope">
<el-tag v-for="tag in scope.row.tags" :key="tag" class="mx-1">
{{ tag }}
</el-tag>
</template>
</el-table-column>
<el-table-column prop="paragraphs[0]" label="诗词预览" width="360" />
<el-table-column fixed="right" label="操作">
<template #default="scope">
<el-button link type="primary" size="small" @click="viewClick(scope.row)">详情</el-button>
</template>
</el-table-column>
</el-table>
<el-pagination @current-change="currentChange" @size-change="sizeChange" background
layout="prev, pager, next" :total="searchResult.total" />
</div>
</div>
<!-- 详情弹框 -->
<el-dialog v-model="showStatus['detail-isshow']" :title="showStatus['title']" align-center>
<div class="detail_box">
<h2>{{ currentPoetry.title }}</h2>
<h3>{{ currentPoetry.author }}</h3>
<el-card class="box-card">
<div v-for="paragraph in currentPoetry.paragraphs" :key="paragraph" class="text item"
style="height:30px">{{ paragraph }}</div>
</el-card>
<div>
<iframe class="video" :src="currentPoetry.video"
scrolling="no" border="0" frameborder="no" framespacing="0" allowfullscreen="true" v-if="showStatus['detail-isshow']"> </iframe>
</div>
</div>
</el-dialog>
</div>
</template>
<script lang="ts">
import { defineComponent, reactive, toRefs, onMounted } from 'vue'
import { PoetryInterface, PoetryData } from '@/type/poetry';
import { getPoetryList } from '@/request/api'
import { requestResultNoti } from '@/request/resultNotification';
export default defineComponent({
setup() {
const data = reactive(new PoetryData())
const showStatus = reactive({
'detail-isshow': false,
'title': '诗词详情'
})
const search = () => {
getPoetryList(data.searchCondition).then(
res => {
requestResultNoti(res)
data.searchResult = res.data
console.log(data.searchResult);
}
)
}
const currentChange = (page: number) => {
data.searchCondition.pageNo = page
search()
}
const sizeChange = (pagesize: number) => {
data.searchCondition.pageSize = pagesize
search()
}
const viewClick = (row: PoetryInterface) => {
showStatus['detail-isshow'] = true
data.currentPoetry = row
}
onMounted(async () => {
//dom 挂载后
console.log("*******onMounted******")
search()
})
return {
...toRefs(data),
showStatus,
search,
currentChange,
sizeChange,
viewClick
}
}
})
</script>
<style lang="scss" scoped>
.mx-1 {
margin: 2px;
}
.detail_box {
text-align: center;
background-image: url('../assets/shuimo.png');
}
.box-card {
background-color: rgba(0, 0, 0, 0);
}
.video{
height: 480px;
width: 720px;
}
</style>
表结构
[SQL] 纯文本查看 复制代码 SET NAMES utf8mb4;
SET FOREIGN_KEY_CHECKS = 0;
-- ----------------------------
-- Table structure for poetry
-- ----------------------------
DROP TABLE IF EXISTS `poetry`;
CREATE TABLE `poetry` (
`id` varchar(32) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
`author` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT NULL,
`title` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT NULL,
`tags` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT NULL,
`paragraphs` varchar(2000) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT NULL,
`pexplain` varchar(1000) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT NULL,
`video` varchar(1000) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT NULL,
PRIMARY KEY (`id`) USING BTREE,
INDEX `index_tags`(`tags`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8 COLLATE = utf8_general_ci ROW_FORMAT = Dynamic;
SET FOREIGN_KEY_CHECKS = 1;
后端代码
现在后端很多公司都在推ddd，楼主比较懒还是用mvc结构吧
crud固定格式套路，这里只贴service的代码吧
[Java] 纯文本查看 复制代码 package com.guojian.student.home.service.impl;
import com.alibaba.fastjson.JSON;
import com.alibaba.fastjson.JSONArray;
import com.alibaba.fastjson.JSONObject;
import com.baomidou.mybatisplus.core.conditions.query.QueryWrapper;
import com.baomidou.mybatisplus.core.toolkit.StringUtils;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import com.github.houbb.opencc4j.util.ZhConverterUtil;
import com.guojian.student.home.common.PageResponse;
import com.guojian.student.home.model.PoetryDO;
import com.guojian.student.home.mapper.PoetryMapper;
import com.guojian.student.home.model.param.PoetrySearchParam;
import com.guojian.student.home.model.vo.PoetryVo;
import com.guojian.student.home.service.PoetryService;
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import org.apache.commons.io.FileUtils;
import org.springframework.stereotype.Service;
import java.io.File;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
/**
* <p>
* 服务实现类
* </p>
*
* @author guojian
* [url=home.php?mod=space&uid=441028]@Since[/url] 2023-01-31
*/
@Service
public class PoetryServiceImpl extends ServiceImpl<PoetryMapper, PoetryDO> implements PoetryService {
@Override
//数据源导入数据库
public void dataImport() throws IOException {
File file = new File("D:\\code\\chinese-poetry-master\\json\\唐诗三百首.json");
String content = FileUtils.readFileToString(file, "UTF-8");
JSONArray jsonArr = JSON.parseArray(content);
for (int i = 0; i < jsonArr.size(); i++) {
JSONObject jsonObj = jsonArr.getJSONObject(i);
PoetryDO poetryDO = new PoetryDO();
poetryDO.setAuthor(ZhConverterUtil.toSimple(jsonObj.getString("author")));
poetryDO.setParagraphs(ZhConverterUtil.toSimple(jsonObj.getString("paragraphs")));
poetryDO.setTitle(ZhConverterUtil.toSimple(jsonObj.getString("title")));
poetryDO.setTags(ZhConverterUtil.toSimple(jsonObj.getString("tags")));
saveOrUpdate(poetryDO);
}
}
@Override
public void videoUrlImport() throws IOException {
File file = new File("D:\\code\\chinese-poetry-master\\json\\唐诗三百首video.json");
String content = FileUtils.readFileToString(file, "UTF-8");
JSONArray jsonArr = JSON.parseArray(content);
for (int i = 0; i < jsonArr.size(); i++) {
JSONObject jsonObj = jsonArr.getJSONObject(i);
System.out.println(jsonObj);
String id = jsonObj.getString("id");
String video = jsonObj.getString("iframeUrl");
PoetryDO poetryDO = getById(id);
poetryDO.setVideo(video);
saveOrUpdate(poetryDO);
}
}
@Override
public PageResponse<List<PoetryVo>> getListByCondition(PoetrySearchParam param) {
QueryWrapper<PoetryDO> wrapper = new QueryWrapper<>();
wrapper.like(StringUtils.isNotBlank(param.getTitle()), "title", param.getTitle())
.like(StringUtils.isNotBlank(param.getAuthor()), "author", param.getAuthor())
.like(StringUtils.isNotBlank(param.getTags()), "tags", param.getTags());
Page page = page(new Page(param.getPageNo(), param.getPageSize()), wrapper);
PageResponse<List<PoetryVo>> result = new PageResponse<>();
List<PoetryDO> recordDOList = page.getRecords();
List<PoetryVo> recordReturn = new ArrayList<>();
for (PoetryDO poetry : recordDOList) {
recordReturn.add(poetry.convertToVo());
}
result.setRecord(recordReturn);
result.setTotal(page.getTotal());
result.setPageSize(page.getSize());
result.setPageNo(page.getCurrent());
return result;
}
}
爬取数据的代码
不太了解bilibili的接口，分析很麻烦，直接ui自动化爬数据，注释写的很清楚
把数据库里的古诗词数据导出成csv（视频链接为空），读取csv文件作为输入
[Python] 纯文本查看 复制代码 import json
from time import sleep
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.action_chains import ActionChains
import re
import tkinter as tk
import name_list_gen
def find_iframe_src_by_chrome(driver,key):
# 根据关键词搜索视频
driver.get('https://search.bilibili.com/all?keyword='+key+'&from_source=webtop_search&spm_id_from=333.1007&search_source=5')
# 获取第一个视频href并访问
videoelement = driver.find_elements(By.CLASS_NAME, 'bili-video-card__info--right')[0].find_element(By.TAG_NAME, 'a')
url = videoelement.get_attribute('href')
driver.get(url)
# 等待加载
sleep(3)
# 分享按钮悬停
el = driver.find_element(By.ID, 'share-btn-outer')
ActionChains(driver).move_to_element(el).perform()
# 定级复制代码按钮
driver.find_element(By.ID, 'share-btn-iframe').click()
# 获取剪切板内容
root = tk.Tk()
root.withdraw()
result_cp = root.clipboard_get()
# 正则获取src内容
result = re.findall(r'(?=src)src=\"(?P<src>[^\"]+)',result_cp)
return result[0]
if __name__ == '__main__':
list = name_list_gen.get_poetry_namejsonarr()
result = []
driver = webdriver.Chrome() # 开启一个Chrome窗口
driver.maximize_window() # 最大化窗口，充分展示元素控件
# 循环遍历需要搜索的视频名称信息，id是为了爬完数据后能够通过唯一标识找到原有记录，写入数据库。
for li in list:
result.append(
{
"id": li['id'],
"name": li['name'],
"iframeUrl": find_iframe_src_by_chrome(driver,li['name'])
}
)
# 输出成json文件
result_json = json.dumps(result, ensure_ascii=False)
print(result_json)
file = open('data.json', 'w')
file.write(result_json)
file.close()
4.页面效果
5.服务地址
楼主花了70块买了个垃圾云服务器，卡的呀批，终于把服务部署起来了，楼主都是利用工作之余写一下，目前比较简陋，还望海涵；
爬取的视频数据，可能因为关键词不准，有些古诗的视频对不上号，后面楼主会优化关键词重新爬取，目前只有唐诗300首，古诗也会继续丰富
http://1.15.179.113/
更新：
- 算术题导出功能；
- 字帖功能（字库包括1~4年级，随机汉字）
| 
[复制链接]
发表于 2023-2-2 18:13
|
发表于 2023-2-2 20:35
