You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

3.8 KiB

This file contains invisible Unicode characters!

This file contains invisible Unicode characters that may be processed differently from what appears below. If your use case is intentional and legitimate, you can safely ignore this warning. Use the Escape button to reveal hidden characters.

This file contains ambiguous Unicode characters that may be confused with others in your current locale. If your use case is intentional and legitimate, you can safely ignore this warning. Use the Escape button to highlight these characters.

PaddleServing预测功能测试

PaddleServing预测功能测试的主程序为test_serving_infer_python.shtest_serving_infer_cpp.sh可以测试基于PaddleServing的部署功能。

1. 测试结论汇总

基于训练是否使用量化,进行本测试的模型可以分为正常模型量化模型这两类模型对应的Serving预测功能汇总如下

模型类型 device batchsize tensorrt mkldnn cpu多线程
正常模型 GPU 1/6 fp32/fp16 - -
正常模型 CPU 1/6 - fp32 支持
量化模型 GPU 1/6 int8 - -
量化模型 CPU 1/6 - int8 支持

2. 测试流程

运行环境配置请参考文档的内容配置TIPC的运行环境。

2.1 功能测试

python serving 先运行prepare.sh准备数据和模型,然后运行test_serving_infer_python.sh进行测试,最终在test_tipc/output/{model_name}/serving_infer/python目录下生成python_*.log后缀的日志文件。

bash test_tipc/prepare.sh ./test_tipc/configs/ch_PP-OCRv2/model_linux_gpu_normal_normal_serving_python_linux_gpu_cpu.txt "serving_infer"

# 用法:
bash test_tipc/test_serving_infer_python.sh ./test_tipc/configs/ch_PP-OCRv2/model_linux_gpu_normal_normal_serving_python_linux_gpu_cpu.txt "serving_infer"

cpp serving 先运行prepare.sh准备数据和模型,然后运行test_serving_infer_cpp.sh进行测试,最终在test_tipc/output/{model_name}/serving_infer/cpp目录下生成cpp_*.log后缀的日志文件。

bash test_tipc/prepare.sh ./test_tipc/configs/ch_PP-OCRv2/model_linux_gpu_normal_normal_serving_cpp_linux_gpu_cpu.txt "serving_infer"

# 用法:
bash test_tipc/test_serving_infer_cpp.sh ./test_tipc/configs/ch_PP-OCRv2/model_linux_gpu_normal_normal_serving_cpp_linux_gpu_cpu.txt "serving_infer"

运行结果

各测试的运行情况会打印在 test_tipc/output/{model_name}/serving_infer/python(cpp)/results_python(cpp)_serving.log 中: 运行成功时会输出:

Run successfully with command - ch_PP-OCRv2_rec - nohup python3.7 web_service_rec.py --config=config.yml --opt op.rec.concurrency="1" op.det.local_service_conf.devices= op.det.local_service_conf.use_mkldnn=False op.det.local_service_conf.thread_num=6 op.rec.local_service_conf.model_config=ppocr_rec_v2_serving > ./test_tipc/output/ch_PP-OCRv2_rec/serving_infer/python/python_server_cpu_usemkldnn_False_threads_6.log 2>&1 &!
Run successfully with command - ch_PP-OCRv2_rec - python3.7 pipeline_http_client.py --det=False --image_dir=../../inference/rec_inference > ./test_tipc/output/ch_PP-OCRv2_rec/serving_infer/python/python_client_cpu_pipeline_http_usemkldnn_False_threads_6_batchsize_1.log 2>&1 !
...

运行失败时会输出:

Run failed with command - ch_PP-OCRv2_rec - nohup python3.7 web_service_rec.py --config=config.yml --opt op.rec.concurrency="1" op.det.local_service_conf.devices= op.det.local_service_conf.use_mkldnn=False op.det.local_service_conf.thread_num=6 op.rec.local_service_conf.model_config=ppocr_rec_v2_serving > ./test_tipc/output/ch_PP-OCRv2_rec/serving_infer/python/python_server_cpu_usemkldnn_False_threads_6.log 2>&1 &!
Run failed with command - ch_PP-OCRv2_rec - python3.7 pipeline_http_client.py --det=False --image_dir=../../inference/rec_inference > ./test_tipc/output/ch_PP-OCRv2_rec/serving_infer/python/python_client_cpu_pipeline_http_usemkldnn_False_threads_6_batchsize_1.log 2>&1 !
...

详细的预测结果会存在 test_tipc/output/{model_name}/serving_infer/python(cpp)/ 文件夹下

3. 更多教程

本文档为功能测试用更详细的Serving预测使用教程请参考PPOCR 服务化部署