huanghai
/
dsProject
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

'commit'

Browse Source
main
HuangHai 1 month ago
parent 68231fd5f8
commit 8579fcdbed
3 changed files with 79 additions and 114 deletions
Show Stats Download Patch File Download Diff File

BIN
dsRag/Config/__pycache__/Config.cpython-310.pyc
View File

Binary file not shown.

2
dsRag/T8_RAG.py
Unescape View File

@ -85,7 +85,7 @@ def process_query(query):
"""处理用户查询并生成报告"""
print(f"正在搜索与'{query}'相关的数据...")
context = search_related_data(query)
print(f"找到{len(context.split('\n\n'))}条相关数据")
print(f"找到{len(context.split(chr(10)+chr(10)))}条相关数据")
print("正在生成报告...")
report = generate_report(query, context)

191
dsRag/运行结果.txt
Unescape View File

@ -1,129 +1,94 @@
D:\anaconda3\envs\rag\python.exe D:\dsWork\dsProject\dsRag\T7_Query.py
D:\anaconda3\envs\rag\python.exe D:\dsWork\dsProject\dsRag\T8_RAG.py
D:\anaconda3\envs\rag\lib\site-packages\jieba\_compat.py:18: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
import pkg_resources
2025-06-23 20:03:12,025 - INFO - loading projection weights from D:\Tencent_AILab_ChineseEmbedding\Tencent_AILab_ChineseEmbedding.txt
2025-06-23 20:03:13,349 - INFO - KeyedVectors lifecycle event {'msg': 'loaded (10000, 200) matrix of type float32 from D:\\Tencent_AILab_ChineseEmbedding\\Tencent_AILab_ChineseEmbedding.txt', 'binary': False, 'encoding': 'utf8', 'datetime': '2025-06-23T20:03:13.329238', 'gensim': '4.3.3', 'python': '3.10.18 | packaged by conda-forge | (main, Jun 4 2025, 14:42:04) [MSC v.1943 64 bit (AMD64)]', 'platform': 'Windows-10-10.0.19044-SP0', 'event': 'load_word2vec_format'}
2025-06-23 20:03:13,349 - INFO - 模型加载成功，词向量维度: 200
2025-06-23 20:13:47,913 - INFO - loading projection weights from D:\Tencent_AILab_ChineseEmbedding\Tencent_AILab_ChineseEmbedding.txt
2025-06-23 20:13:49,382 - INFO - KeyedVectors lifecycle event {'msg': 'loaded (10000, 200) matrix of type float32 from D:\\Tencent_AILab_ChineseEmbedding\\Tencent_AILab_ChineseEmbedding.txt', 'binary': False, 'encoding': 'utf8', 'datetime': '2025-06-23T20:13:49.361378', 'gensim': '4.3.3', 'python': '3.10.18 | packaged by conda-forge | (main, Jun 4 2025, 14:42:04) [MSC v.1943 64 bit (AMD64)]', 'platform': 'Windows-10-10.0.19044-SP0', 'event': 'load_word2vec_format'}
2025-06-23 20:13:49,382 - INFO - 模型加载成功，词向量维度: 200
D:\anaconda3\envs\rag\lib\site-packages\elasticsearch\_sync\client\__init__.py:311: SecurityWarning: Connecting to 'https://10.10.14.206:9200' using TLS with verify_certs=False is insecure
_transport = transport_class(
正在搜索与'整理云南省初中在校生情况文档'相关的数据...
Building prefix dict from the default dictionary ...
2025-06-23 20:03:13,622 - DEBUG - Building prefix dict from the default dictionary ...
2025-06-23 20:13:50,520 - DEBUG - Building prefix dict from the default dictionary ...
Loading model from cache C:\Users\ADMINI~1\AppData\Local\Temp\jieba.cache
2025-06-23 20:03:13,622 - DEBUG - Loading model from cache C:\Users\ADMINI~1\AppData\Local\Temp\jieba.cache
测试句子: （万人） 教职工数需求
向量搜索结果:
Loading model cost 0.655 seconds.
2025-06-23 20:03:14,277 - DEBUG - Loading model cost 0.655 seconds.
2025-06-23 20:13:50,520 - DEBUG - Loading model from cache C:\Users\ADMINI~1\AppData\Local\Temp\jieba.cache
Loading model cost 0.648 seconds.
2025-06-23 20:13:51,168 - DEBUG - Loading model cost 0.648 seconds.
Prefix dict has been built successfully.
2025-06-23 20:03:14,278 - DEBUG - Prefix dict has been built successfully.
2025-06-23 20:03:14,278 - INFO - 文本: （万人） 教职工数需求, 分词结果: ['（', '万人', '）', '\t', '教职工', '数', '需求']
2025-06-23 20:03:14,278 - INFO - 有效词向量数量: 5
2025-06-23 20:03:14,278 - INFO - 生成的平均向量: [-0.0246012 -0.1393744 -0.0812634 0.22129479 -0.0203214 ]...
2025-06-23 20:13:51,169 - DEBUG - Prefix dict has been built successfully.
2025-06-23 20:13:51,169 - INFO - 文本: 整理云南省初中在校生情况文档, 分词结果: ['整理', '云南省', '初中', '在校生', '情况', '文档']
2025-06-23 20:13:51,169 - INFO - 有效词向量数量: 3
2025-06-23 20:13:51,169 - INFO - 生成的平均向量: [ 0.18687634 -0.19447033 -0.04333766 -0.05068566 0.09433967]...
D:\anaconda3\envs\rag\lib\site-packages\urllib3\connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host '10.10.14.206'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings
warnings.warn(
2025-06-23 20:03:14,384 - INFO - POST https://10.10.14.206:9200/knowledge_base/_search [status:200 duration:0.104s]
2025-06-23 20:13:51,222 - INFO - POST https://10.10.14.206:9200/knowledge_base/_search [status:200 duration:0.051s]
D:\anaconda3\envs\rag\lib\site-packages\urllib3\connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host '10.10.14.206'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings
warnings.warn(
- （万人） 教职工数需求
- （万人） 教职工数需求
- （万人） 专任教师数需求
- （万人） 教职工数需求
- （万人） 教职工需求
2025-06-23 20:13:51,231 - INFO - POST https://10.10.14.206:9200/raw_texts/_search [status:200 duration:0.009s]
找到10条相关数据
正在生成报告...
2025-06-23 20:13:56,582 - INFO - Retrying request to /chat/completions in 0.469314 seconds
2025-06-23 20:14:02,067 - INFO - Retrying request to /chat/completions in 0.762535 seconds
Traceback (most recent call last):
File "D:\anaconda3\envs\rag\lib\site-packages\httpx\_transports\default.py", line 101, in map_httpcore_exceptions
yield
File "D:\anaconda3\envs\rag\lib\site-packages\httpx\_transports\default.py", line 250, in handle_request
resp = self._pool.handle_request(req)
File "D:\anaconda3\envs\rag\lib\site-packages\httpcore\_sync\connection_pool.py", line 256, in handle_request
raise exc from None
File "D:\anaconda3\envs\rag\lib\site-packages\httpcore\_sync\connection_pool.py", line 236, in handle_request
response = connection.handle_request(
File "D:\anaconda3\envs\rag\lib\site-packages\httpcore\_sync\connection.py", line 101, in handle_request
raise exc
File "D:\anaconda3\envs\rag\lib\site-packages\httpcore\_sync\connection.py", line 78, in handle_request
stream = self._connect(request)
File "D:\anaconda3\envs\rag\lib\site-packages\httpcore\_sync\connection.py", line 124, in _connect
stream = self._network_backend.connect_tcp(**kwargs)
File "D:\anaconda3\envs\rag\lib\site-packages\httpcore\_backends\sync.py", line 207, in connect_tcp
with map_exceptions(exc_map):
File "D:\anaconda3\envs\rag\lib\contextlib.py", line 153, in __exit__
self.gen.throw(typ, value, traceback)
File "D:\anaconda3\envs\rag\lib\site-packages\httpcore\_exceptions.py", line 14, in map_exceptions
raise to_exc(exc) from exc
httpcore.ConnectTimeout: timed out
文本精确搜索结果:
2025-06-23 20:03:14,419 - INFO - POST https://10.10.14.206:9200/raw_texts/_search [status:200 duration:0.033s]
2025-06-23 20:03:14,420 - INFO - 文本: 表 6 高等教育规模及资源配置需求预测统计表, 分词结果: ['表', ' ', '6', ' ', '高等教育', '规模', '及', '资源配置', '需求预测', '统计表']
2025-06-23 20:03:14,420 - INFO - 有效词向量数量: 4
2025-06-23 20:03:14,421 - INFO - 生成的平均向量: [ 0.18066275 -0.12073625 -0.12693274 0.219709 0.00525575]...
D:\anaconda3\envs\rag\lib\site-packages\urllib3\connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host '10.10.14.206'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings
warnings.warn(
- （万人） 教职工数需求
- （万人） 教职工数需求
- （万人） 教职工数需求
==================================================
测试句子: 表 6 高等教育规模及资源配置需求预测统计表
向量搜索结果:
- 表 6 高等教育规模及资源配置需求预测统计表
- 表 1 学前教育规模及资源配置需求预测统计表
- 表 5 中等职业教育规模及资源配置需求预测统计表
- 表 4 普通高中教育规模及资源配置需求预测统计表
- 表 3 初中教育规模及资源配置需求预测统计表
The above exception was the direct cause of the following exception:
文本精确搜索结果:
- 表 6 高等教育规模及资源配置需求预测统计表
==================================================
测试句子: 指标
向量搜索结果:
2025-06-23 20:03:14,431 - INFO - POST https://10.10.14.206:9200/knowledge_base/_search [status:200 duration:0.008s]
D:\anaconda3\envs\rag\lib\site-packages\urllib3\connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host '10.10.14.206'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings
warnings.warn(
2025-06-23 20:03:14,440 - INFO - POST https://10.10.14.206:9200/raw_texts/_search [status:200 duration:0.008s]
2025-06-23 20:03:14,440 - INFO - 文本: 指标, 分词结果: ['指标']
2025-06-23 20:03:14,441 - INFO - 有效词向量数量: 1
2025-06-23 20:03:14,441 - INFO - 生成的平均向量: [-0.24731 -0.30286 -0.223835 0.217864 -0.162194]...
D:\anaconda3\envs\rag\lib\site-packages\urllib3\connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host '10.10.14.206'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings
warnings.warn(
2025-06-23 20:03:14,450 - INFO - POST https://10.10.14.206:9200/knowledge_base/_search [status:200 duration:0.007s]
D:\anaconda3\envs\rag\lib\site-packages\urllib3\connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host '10.10.14.206'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings
warnings.warn(
2025-06-23 20:03:14,456 - INFO - POST https://10.10.14.206:9200/raw_texts/_search [status:200 duration:0.005s]
2025-06-23 20:03:14,457 - INFO - 文本: 二是保障随迁子女受教育权利。健全以居住证为主要依据的农业转移人口随迁子女入学保障政策，以公办学校为主将随迁子女纳入流入地义务教育保障范围，保障享受基本公共教育服务。, 分词结果: ['二是', '保障', '随迁', '子女', '受', '教育', '权利', '。', '健全', '以', '居住证', '为', '主要', '依据', '的', '农业', '转移', '人口', '随迁', '子女', '入学', '保障', '政策', '，', '以', '公办', '学校', '为主', '将', '随迁', '子女', '纳入', '流入地', '义务教育', '保障', '范围', '，', '保障', '享受', '基本', '公共', '教育', '服务', '。']
2025-06-23 20:03:14,457 - INFO - 有效词向量数量: 32
2025-06-23 20:03:14,458 - INFO - 生成的平均向量: [ 0.18174155 -0.13902284 -0.13658895 0.17560197 0.04838478]...
D:\anaconda3\envs\rag\lib\site-packages\urllib3\connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host '10.10.14.206'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings
warnings.warn(
- 指标
- 指标
- 指标
- 指标
- 指标
文本精确搜索结果:
- 指标
- 指标
- 指标
- 指标
- 指标
- 指标
- 2001-2022年云南省人口指标统计表
- 各学段数据及人口指标统计表
- 1.2001-2022年云南省人口指标统计表 33
- 2.各学段数据及人口指标统计表 34
==================================================
测试句子: 二是保障随迁子女受教育权利。健全以居住证为主要依据的农业转移人口随迁子女入学保障政策，以公办学校为主将随迁子女纳入流入地义务教育保障范围，保障享受基本公共教育服务。
向量搜索结果:
- 健全以居住证为主要依据的农业转移人口随迁子女入学保障政策，以公办学校为主将随迁子女纳入流入地义务教育保障范围，保障享受基本公共教育服务
- 在学校布局、入学政策等方面守牢底线，既适应城乡人口变化趋势，又能提供公平的受教育机会，保障不同群体受教育权利
- 健全随迁子女入学保障政策，保障各类特殊群体受教育权利
- 健全随迁子女入学保障政策，保障各类特殊群体受教育权利
- 健全随迁子女入学保障政策，保障各类特殊群体受教育权利
Traceback (most recent call last):
File "D:\anaconda3\envs\rag\lib\site-packages\openai\_base_client.py", line 972, in request
response = self._client.send(
File "D:\anaconda3\envs\rag\lib\site-packages\httpx\_client.py", line 914, in send
response = self._send_handling_auth(
File "D:\anaconda3\envs\rag\lib\site-packages\httpx\_client.py", line 942, in _send_handling_auth
response = self._send_handling_redirects(
File "D:\anaconda3\envs\rag\lib\site-packages\httpx\_client.py", line 979, in _send_handling_redirects
response = self._send_single_request(request)
File "D:\anaconda3\envs\rag\lib\site-packages\httpx\_client.py", line 1014, in _send_single_request
response = transport.handle_request(request)
File "D:\anaconda3\envs\rag\lib\site-packages\httpx\_transports\default.py", line 249, in handle_request
with map_httpcore_exceptions():
File "D:\anaconda3\envs\rag\lib\contextlib.py", line 153, in __exit__
self.gen.throw(typ, value, traceback)
File "D:\anaconda3\envs\rag\lib\site-packages\httpx\_transports\default.py", line 118, in map_httpcore_exceptions
raise mapped_exc(message) from exc
httpx.ConnectTimeout: timed out
文本精确搜索结果:
2025-06-23 20:03:14,464 - INFO - POST https://10.10.14.206:9200/knowledge_base/_search [status:200 duration:0.005s]
D:\anaconda3\envs\rag\lib\site-packages\urllib3\connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host '10.10.14.206'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings
warnings.warn(
2025-06-23 20:03:14,475 - INFO - POST https://10.10.14.206:9200/raw_texts/_search [status:200 duration:0.010s]
2025-06-23 20:03:14,475 - INFO - 文本: （一）2012—2022年全省人口总体呈现“长增－突降”态势, 分词结果: ['（', '一', '）', '2012', '—', '2022', '年', '全省', '人口', '总体', '呈现', '“', '长增', '－', '突降', '”', '态势']
2025-06-23 20:03:14,476 - INFO - 有效词向量数量: 14
2025-06-23 20:03:14,476 - INFO - 生成的平均向量: [ 0.06827115 -0.212308 -0.07557671 0.21362822 0.12347364]...
D:\anaconda3\envs\rag\lib\site-packages\urllib3\connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host '10.10.14.206'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings
warnings.warn(
2025-06-23 20:03:14,482 - INFO - POST https://10.10.14.206:9200/knowledge_base/_search [status:200 duration:0.005s]
D:\anaconda3\envs\rag\lib\site-packages\urllib3\connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host '10.10.14.206'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings
warnings.warn(
==================================================
测试句子: （一）2012—2022年全省人口总体呈现“长增－突降”态势
向量搜索结果:
- （一）2012—2022年全省人口总体呈现“长增－突降”态势
- （一）2012—2022年全省人口总体呈现“长增－突降”态势 2
- 总的看，2012—2022年全省人口变化呈现先增、突降的态势（图1）
- 结合全国趋势看，预计我省2023—2035年新生人口数持续保持在较低水平（图2），从2024年起，全省人口总数呈缓慢下降趋势，与全国范围人口下降趋势基本同步（图1）
- 按照全省普通高中师生比标准（1：12.5）测算，2023—2035年全省普通高中教师需求量逐年增大，2033年达到峰值（10.31万人），与2022年实际专任教师数7.64万人比，存在2.67万人的缺口（图20）
The above exception was the direct cause of the following exception:
文本精确搜索结果:
- （一）2012—2022年全省人口总体呈现“长增－突降”态势
- （一）2012—2022年全省人口总体呈现“长增－突降”态势 2
==================================================
2025-06-23 20:03:14,488 - INFO - POST https://10.10.14.206:9200/raw_texts/_search [status:200 duration:0.005s]
Traceback (most recent call last):
File "D:\dsWork\dsProject\dsRag\T8_RAG.py", line 98, in <module>
report = process_query(user_query)
File "D:\dsWork\dsProject\dsRag\T8_RAG.py", line 91, in process_query
report = generate_report(query, context)
File "D:\dsWork\dsProject\dsRag\T8_RAG.py", line 73, in generate_report
response = client.chat.completions.create(
File "D:\anaconda3\envs\rag\lib\site-packages\openai\_utils\_utils.py", line 287, in wrapper
return func(*args, **kwargs)
File "D:\anaconda3\envs\rag\lib\site-packages\openai\resources\chat\completions\completions.py", line 925, in create
return self._post(
File "D:\anaconda3\envs\rag\lib\site-packages\openai\_base_client.py", line 1249, in post
return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
File "D:\anaconda3\envs\rag\lib\site-packages\openai\_base_client.py", line 990, in request
raise APITimeoutError(request=request) from err
openai.APITimeoutError: Request timed out.
进程已结束，退出代码为 0
进程已结束，退出代码为 1

Write Preview
Loading…
Cancel
Save