nli-MiniLM2-L6-H768保姆级：ONNX导出+TensorRT加速部署全流程-洪萨配资

nli-MiniLM2-L6-H768保姆级：ONNX导出+TensorRT加速部署全流程

1. 模型简介

nli-MiniLM2-L6-H768是一个专为自然语言推理(NLI)与零样本分类设计的轻量级交叉编码器(Cross-Encoder)模型。它在保持接近BERT-base精度的同时，通过精简架构实现了更高的效率：

精度表现：在NLI任务上接近BERT-base水平
速度/体积平衡：6层Transformer结构，768维隐藏层
开箱即用：支持直接零样本分类和句子对推理任务

2. 环境准备

2.1 硬件要求

NVIDIA GPU (推荐RTX 3060及以上)
CUDA 11.x 兼容驱动
至少4GB GPU显存

2.2 软件依赖

pip install torch transformers onnx onnxruntime-gpu tensorrt

2.3 模型下载

from transformers import AutoModelForSequenceClassification, AutoTokenizer model_name = "cross-encoder/nli-MiniLM2-L6-H768" model = AutoModelForSequenceClassification.from_pretrained(model_name) tokenizer = AutoTokenizer.from_pretrained(model_name)

3. ONNX模型导出

3.1 基础导出步骤

import torch dummy_input = tokenizer("This is a test", return_tensors="pt") torch.onnx.export( model, tuple(dummy_input.values()), "nli_minilm.onnx", input_names=["input_ids", "attention_mask"], output_names=["output"], dynamic_axes={ "input_ids": {0: "batch", 1: "sequence"}, "attention_mask": {0: "batch", 1: "sequence"}, "output": {0: "batch"} }, opset_version=13 )

3.2 导出优化技巧

序列长度固定：设置固定max_length可提升推理效率
精度选择：FP16导出可减少模型体积
算子验证：使用onnxruntime验证导出结果

4. TensorRT加速部署

4.1 转换ONNX到TensorRT

trtexec --onnx=nli_minilm.onnx \ --saveEngine=nli_minilm.trt \ --fp16 \ --workspace=2048

4.2 Python推理代码

import tensorrt as trt import pycuda.driver as cuda import pycuda.autoinit # 加载TensorRT引擎 with open("nli_minilm.trt", "rb") as f: runtime = trt.Runtime(trt.Logger(trt.Logger.WARNING)) engine = runtime.deserialize_cuda_engine(f.read()) # 创建执行上下文 context = engine.create_execution_context() # 分配输入输出缓冲区 inputs, outputs, bindings = [], [], [] stream = cuda.Stream() for binding in engine: size = trt.volume(engine.get_binding_shape(binding)) dtype = trt.nptype(engine.get_binding_dtype(binding)) host_mem = cuda.pagelocked_empty(size, dtype) device_mem = cuda.mem_alloc(host_mem.nbytes) bindings.append(int(device_mem)) if engine.binding_is_input(binding): inputs.append({'host': host_mem, 'device': device_mem}) else: outputs.append({'host': host_mem, 'device': device_mem}) # 推理函数 def infer(input_ids, attention_mask): # 拷贝输入数据 np.copyto(inputs[0]['host'], input_ids.ravel()) np.copyto(inputs[1]['host'], attention_mask.ravel()) # 数据传输 cuda.memcpy_htod_async(inputs[0]['device'], inputs[0]['host'], stream) cuda.memcpy_htod_async(inputs[1]['device'], inputs[1]['host'], stream) # 执行推理 context.execute_async_v2(bindings=bindings, stream_handle=stream.handle) # 取回结果 cuda.memcpy_dtoh_async(outputs[0]['host'], outputs[0]['device'], stream) stream.synchronize() return outputs[0]['host']

5. 性能对比测试

5.1 测试环境

GPU: NVIDIA RTX 3090
CPU: AMD Ryzen 9 5950X
测试数据: SNLI验证集(1000样本)

5.2 性能数据

框架	延迟(ms)	吞吐量(samples/s)	显存占用(MB)
PyTorch	15.2	65.8	1240
ONNX Runtime	8.7	114.9	980
TensorRT	4.3	232.6	820

6. 实际应用示例

6.1 零样本分类

def zero_shot_classification(text, labels): # 构造句子对 pairs = [(text, f"This example is about {label}") for label in labels] # 批量推理 inputs = tokenizer(pairs, padding=True, truncation=True, return_tensors="pt") outputs = infer(inputs["input_ids"], inputs["attention_mask"]) # 获取概率 probs = torch.softmax(torch.tensor(outputs), dim=1)[:, 1] return {label: float(prob) for label, prob in zip(labels, probs)}

6.2 NLI推理服务

from fastapi import FastAPI import uvicorn app = FastAPI() @app.post("/predict") async def predict_nli(premise: str, hypothesis: str): inputs = tokenizer(premise, hypothesis, return_tensors="pt") outputs = infer(inputs["input_ids"], inputs["attention_mask"]) probs = torch.softmax(torch.tensor(outputs), dim=1)[0] return { "entailment": float(probs[0]), "neutral": float(probs[1]), "contradiction": float(probs[2]) } if __name__ == "__main__": uvicorn.run(app, host="0.0.0.0", port=8000)