Pārlūkot izejas kodu

fix(aiclean): 导入失败日志统一走手动清洗弹框

- AI 清洗拒绝行同步写入 clean_error_log(文件/工作表/行号/列/原因/原值)
- 导入文件页取消 AI 专用明细弹框,统一打开「导入失败日志」
- 删除 AiRejectModal,补充统一弹框的 E2E 截图与说明
cc 1 nedēļu atpakaļ
vecāks
revīzija
25926870d6

+ 4 - 7
ai-electron/frontend/src/case/api/govern/governApi.ts

@@ -192,14 +192,11 @@ export const aiCleanDataPage = (params: {
  * 某个 AI 清洗任务的行级失败明细(分页)
  *
  * 每条记录包含 row_no / blocking_col_name / reason / errors_json(该行全部列错误,
- * 值为脱敏后的预览)。「导入失败」数字点击后用它解释「哪一行、哪一列、为什么没进去」。
+ * 值为完整原值预览)。导入文件页的「导入失败」统一查 clean_error_log,
+ * 此接口保留给排障与后续明细页使用。
  */
-export const aiCleanRejections = (params: {
-  jobId: ApiId | number;
-  reason?: string;
-  page?: number;
-  size?: number;
-}) => defHttp.get<Recordable>({ url: adminPath + '/aiClean/job/rejections', params });
+export const aiCleanRejections = (params: { jobId: ApiId | number; reason?: string; page?: number; size?: number }) =>
+  defHttp.get<Recordable>({ url: adminPath + '/aiClean/job/rejections', params });
 
 /**
  * 文件预处理(Electron 本地路径方式,保留兼容)

+ 0 - 222
ai-electron/frontend/src/case/views/data/components/AiRejectModal.vue

@@ -1,222 +0,0 @@
-<template>
-  <BasicModal
-    :open="open"
-    title="AI 清洗失败明细"
-    :width="1080"
-    :showOkBtn="false"
-    cancelText="关闭"
-    wrapClassName="case-ai-reject-modal"
-    @update:open="handleOpenChange"
-    @cancel="handleOpenChange(false)"
-  >
-    <div class="ai-reject-summary">
-      <span class="ai-reject-summary__file">
-        {{ fileName || '未命名文件' }}<template v-if="sheetName"> / {{ sheetName }}</template>
-      </span>
-      <span class="ai-reject-summary__ok">成功 {{ successNum }}</span>
-      <span class="ai-reject-summary__fail">失败 {{ failNum }}</span>
-      <span class="ai-reject-summary__hint">每一行失败都记录了阻塞列、原因与原值</span>
-    </div>
-
-    <Table
-      row-key="rowKey"
-      size="small"
-      :columns="columns"
-      :data-source="rows"
-      :loading="loading"
-      :pagination="pagination"
-      :scroll="{ y: 420 }"
-      @change="handleTableChange"
-    />
-  </BasicModal>
-</template>
-
-<script lang="ts" setup name="ViewsDataAiRejectModal">
-  import { computed, ref, watch } from 'vue';
-  import { Table } from 'ant-design-vue';
-  import type { TableColumnsType, TablePaginationConfig } from 'ant-design-vue';
-  import { BasicModal } from '@/core/components/Modal';
-  import { useMessage } from '@/core/hooks/web/useMessage';
-  import { aiCleanRejections } from '@/case/api/govern/governApi';
-
-  interface RejectRow {
-    rowKey: string;
-    rowNo: number;
-    column: string;
-    reason: string;
-    value: string;
-    others: string;
-  }
-
-  interface RawRejectRecord {
-    row_no?: number | string | null;
-    blocking_col_index?: number | string | null;
-    blocking_col_name?: string | null;
-    blocking_col_cn?: string | null;
-    reason?: string | null;
-    errors_json?: string | null;
-  }
-
-  const props = withDefaults(
-    defineProps<{
-      open: boolean;
-      jobId?: string | number | null;
-      fileName?: string;
-      sheetName?: string;
-      successNum?: number;
-      failNum?: number;
-    }>(),
-    {
-      jobId: null,
-      fileName: '',
-      sheetName: '',
-      successNum: 0,
-      failNum: 0,
-    },
-  );
-
-  const emit = defineEmits<{ 'update:open': [value: boolean] }>();
-  const { createMessage } = useMessage();
-
-  const rows = ref<RejectRow[]>([]);
-  const loading = ref(false);
-  const total = ref(0);
-  const page = ref(1);
-  const size = ref(20);
-
-  const REASON_LABELS: Record<string, string> = {
-    REQUIRED_EMPTY: '必填列为空',
-    REQUIRED_TYPE_INVALID: '必填列类型不合法',
-    REQUIRED_FUNC_ERROR: '必填列函数执行异常',
-    REQUIRED_LENGTH_OVERFLOW: '必填列超长/超精度',
-    OPTIONAL_TYPE_INVALID: '非必填列类型不合法(单元格置空)',
-    OPTIONAL_FUNC_ERROR: '非必填列函数执行异常(单元格置空)',
-    OPTIONAL_LENGTH_OVERFLOW: '非必填列超长/超精度(单元格置空)',
-    WRITER_ERROR: '写入数据库失败',
-  };
-
-  const reasonLabel = (reason?: string | null) => (reason ? REASON_LABELS[reason] || reason : '-');
-
-  const columns: TableColumnsType = [
-    { title: '行号', dataIndex: 'rowNo', key: 'rowNo', width: 90 },
-    { title: '阻塞列', dataIndex: 'column', key: 'column', width: 180, ellipsis: true },
-    { title: '原因', dataIndex: 'reason', key: 'reason', width: 200 },
-    { title: '原值预览', dataIndex: 'value', key: 'value', ellipsis: true },
-    { title: '该行其他错误', dataIndex: 'others', key: 'others', ellipsis: true },
-  ];
-
-  const pagination = computed<TablePaginationConfig>(() => ({
-    current: page.value,
-    pageSize: size.value,
-    total: total.value,
-    showSizeChanger: true,
-    pageSizeOptions: ['20', '50', '100'],
-    showTotal: (count: number) => `共 ${count} 行`,
-  }));
-
-  const parseErrors = (raw: RawRejectRecord) => {
-    try {
-      const list = JSON.parse(String(raw.errors_json || '[]'));
-      return Array.isArray(list) ? list : [];
-    } catch {
-      return [];
-    }
-  };
-
-  const toRow = (raw: RawRejectRecord): RejectRow => {
-    const errors = parseErrors(raw);
-    const blocking = errors.find((e: any) => e?.blocking) || errors[0] || {};
-    const others = errors
-      .filter((e: any) => e !== blocking)
-      .map((e: any) => {
-        const cn = String(e?.columnCn ?? '').trim();
-        const col = e?.column || '?';
-        return `${col}${cn ? `(${cn})` : ''}:${reasonLabel(e?.reason)}`;
-      })
-      .join(';');
-    const col = raw.blocking_col_name || blocking?.column || '';
-    // 优先展示中文表头:c1(身份证号);没有中文名时保留列序号
-    const cn = String(raw.blocking_col_cn ?? blocking?.columnCn ?? '').trim();
-    const idx = raw.blocking_col_index ?? blocking?.colIndex;
-    const column = !col ? '-' : cn ? `${col}(${cn})` : `${col}${idx != null && idx !== '' ? `(列 ${idx})` : ''}`;
-    return {
-      rowKey: `${raw.row_no ?? ''}-${idx ?? ''}-${raw.reason ?? ''}`,
-      rowNo: Number(raw.row_no ?? 0),
-      column,
-      reason: reasonLabel(raw.reason || blocking?.reason),
-      value: String(blocking?.value ?? '-'),
-      others: others || '-',
-    };
-  };
-
-  const load = async () => {
-    if (!props.jobId) {
-      rows.value = [];
-      total.value = 0;
-      return;
-    }
-    loading.value = true;
-    try {
-      const res = await aiCleanRejections({ jobId: props.jobId, page: page.value, size: size.value });
-      const records = Array.isArray(res?.records) ? (res.records as RawRejectRecord[]) : [];
-      rows.value = records.map(toRow);
-      total.value = Number(res?.total ?? 0);
-    } catch (error: any) {
-      createMessage.error(error?.message || '失败明细加载失败');
-      rows.value = [];
-      total.value = 0;
-    } finally {
-      loading.value = false;
-    }
-  };
-
-  const handleTableChange = (p: TablePaginationConfig) => {
-    page.value = Number(p.current || 1);
-    size.value = Number(p.pageSize || 20);
-    load();
-  };
-
-  const handleOpenChange = (value: boolean) => emit('update:open', value);
-
-  watch(
-    () => props.open,
-    (open) => {
-      if (open) {
-        page.value = 1;
-        size.value = 20;
-        load();
-      }
-    },
-  );
-</script>
-
-<style lang="less">
-  .case-ai-reject-modal {
-    .ai-reject-summary {
-      display: flex;
-      align-items: center;
-      gap: 16px;
-      flex-wrap: wrap;
-      margin-bottom: 12px;
-      color: #4e5969;
-
-      &__file {
-        font-weight: 600;
-        color: #1d2129;
-      }
-
-      &__ok {
-        color: #00a870;
-      }
-
-      &__fail {
-        color: #e34d59;
-      }
-
-      &__hint {
-        color: #86909c;
-        font-size: 12px;
-      }
-    }
-  }
-</style>

+ 2 - 21
ai-electron/frontend/src/case/views/data/importList.vue

@@ -35,15 +35,6 @@
       :file-infos="currentLogFileInfos"
       :initial-sheet-id="currentLogSheetId"
     />
-
-    <AiRejectModal
-      v-model:open="aiRejectOpen"
-      :job-id="aiRejectRecord?.aiJobId ?? null"
-      :file-name="String(aiRejectRecord?.fileName ?? '')"
-      :sheet-name="String(aiRejectRecord?.sheetName ?? '')"
-      :success-num="Number(aiRejectRecord?.successNum ?? 0)"
-      :fail-num="Number(aiRejectRecord?.failNum ?? 0)"
-    />
   </div>
 </template>
 <script lang="ts" setup name="ViewsSysEmpUserList">
@@ -65,7 +56,6 @@
   import { FileStatusEnum } from '@/case/types/enum';
   import componentSetting from '@/core/settings/componentSetting';
   import CleanLogModal from './components/CleanLogModal.vue';
-  import AiRejectModal from './components/AiRejectModal.vue';
   import { resetImportProgressSession, setCleanFlowSource } from './importProgressSession';
 
   const props = defineProps({
@@ -92,9 +82,6 @@
   const logModalOpen = ref(false);
   const currentLogFileInfos = ref<FileInfo[]>([]);
   const currentLogSheetId = ref<string>('');
-  /** AI 清洗失败明细:按 sheet 最新 job 查 ai_clean_reject */
-  const aiRejectOpen = ref(false);
-  const aiRejectRecord = ref<Partial<FileInfo> | null>(null);
   const hasLoadedOnActivate = ref(false);
   const recleaningFileIds = ref<string[]>([]);
   const paginationLoading = ref(false);
@@ -504,14 +491,8 @@
     event.preventDefault();
     event.stopPropagation();
 
-    // AI 清洗的失败原因在 ai_clean_reject(逐行:哪一列、什么原因、脱敏原值),
-    // 不再走 clean_error_log;即使失败数为 0 也允许打开看汇总。
-    if (record.cleanMode === 'AI清洗' && record.aiJobId != null) {
-      aiRejectRecord.value = record;
-      aiRejectOpen.value = true;
-      return;
-    }
-
+    // 手动清洗与 AI 清洗都写 clean_error_log,统一走「导入失败日志」弹框,
+    // 不再为 AI 清洗单独弹一个明细框。
     const failCount = Number(record.failNum ?? 0);
     if (!Number.isFinite(failCount) || failCount <= 0) {
       createMessage.info('当前文件暂无失败记录');

+ 14 - 0
ai-server/src/main/java/com/zsjz/ai/module/aiclean/enums/AiRejectReason.java

@@ -32,4 +32,18 @@ public enum AiRejectReason {
     public boolean blocking() {
         return this == WRITER_ERROR || name().startsWith("REQUIRED_");
     }
+
+    /** 写入 clean_error_log 时使用的中文原因,保持与界面文案一致 */
+    public String label() {
+        return switch (this) {
+            case REQUIRED_EMPTY -> "必填列为空";
+            case REQUIRED_TYPE_INVALID -> "必填列类型不合法";
+            case REQUIRED_FUNC_ERROR -> "必填列函数执行失败";
+            case REQUIRED_LENGTH_OVERFLOW -> "必填列超长或精度溢出";
+            case OPTIONAL_TYPE_INVALID -> "非必填列类型不合法(已置空)";
+            case OPTIONAL_FUNC_ERROR -> "非必填列函数执行失败(已置空)";
+            case OPTIONAL_LENGTH_OVERFLOW -> "非必填列超长或精度溢出(已置空)";
+            case WRITER_ERROR -> "写入失败";
+        };
+    }
 }

+ 86 - 0
ai-server/src/main/java/com/zsjz/ai/module/aiclean/service/AiCleanService.java

@@ -11,6 +11,7 @@ import com.zsjz.ai.common.context.CaseAccessGuard;
 import com.zsjz.ai.common.enums.FileCategoryEnum;
 import com.zsjz.ai.common.utils.Json;
 import com.zsjz.ai.common.model.dm.dto.TableRuleDTO;
+import com.zsjz.ai.common.model.dm.entity.CleanErrorLog;
 import com.zsjz.ai.common.model.dm.entity.FileInfo;
 import com.zsjz.ai.common.model.plat.entity.TableInfo;
 import com.zsjz.ai.module.aiclean.entity.AiCleanField;
@@ -19,6 +20,7 @@ import com.zsjz.ai.module.aiclean.entity.AiCleanRule;
 import com.zsjz.ai.module.aiclean.entity.AiCleanTemplate;
 import com.zsjz.ai.module.aiclean.enums.AiCleanExitEnum;
 import com.zsjz.ai.module.aiclean.enums.AiCleanStatusEnum;
+import com.zsjz.ai.module.aiclean.enums.AiRejectReason;
 import com.zsjz.ai.module.aiclean.exec.AiDuckDb;
 import com.zsjz.ai.module.aiclean.exec.AiRecordCleaner;
 import com.zsjz.ai.module.aiclean.exec.AiRejectCollector;
@@ -30,6 +32,7 @@ import com.zsjz.ai.module.aiclean.match.AiTemplateMatcher;
 import com.zsjz.ai.module.aiclean.model.AiMatchOutcome;
 import com.zsjz.ai.module.aiclean.model.AiRowCleanResult;
 import com.zsjz.ai.module.aiclean.model.AiRuleDraft;
+import com.zsjz.ai.module.aiclean.model.ColumnError;
 import com.zsjz.ai.module.aiclean.model.RuleBind;
 import com.zsjz.ai.module.aiclean.model.SheetEvidence;
 import com.zsjz.ai.module.aiclean.model.TemplateCandidate;
@@ -41,6 +44,7 @@ import com.zsjz.ai.module.aiclean.rule.CompiledColumn;
 import com.zsjz.ai.module.aiclean.store.AiCleanStore;
 import com.zsjz.ai.module.aiclean.support.HeaderFingerprint;
 import com.zsjz.ai.module.aiclean.verify.AiRuleVerifier;
+import com.zsjz.ai.module.dm.mapper.CleanErrorLogMapper;
 import com.zsjz.ai.module.dm.mapper.FileInfoMapper;
 import com.zsjz.ai.module.agent.llm.LlmResult;
 import lombok.RequiredArgsConstructor;
@@ -85,6 +89,8 @@ public class AiCleanService {
 
     private final FileInfoMapper fileInfoMapper;
 
+    private final CleanErrorLogMapper cleanErrorLogMapper;
+
     private final AiTemplateMatcher matcher;
 
     private final ExistingTemplateDispatcher dispatcher;
@@ -540,6 +546,8 @@ public class AiCleanService {
         AiRejectCollector rejects = new AiRejectCollector(job.getId(), job.getCaseId(), job.getFileId(),
                 job.getSheetId(), ev.primaryBlock().blockNo(), columnNameCn,
                 rejectLogSample, rejectLogAll);
+        // 与手动清洗共用 clean_error_log:导入文件页的「导入失败」弹框直接按批次/文件查这张表
+        List<CleanErrorLog> errorLogs = new ArrayList<>();
         long written = -1L;
         try (AiDuckDb.BatchWriter writer = AiDuckDb.openWriter(tpl.getTableNameEn(), cols)) {
             // 行数据优先取自上传预览表:不用为了入库再把整份源文件重新解析一遍。
@@ -550,6 +558,10 @@ public class AiCleanService {
                 AiRowCleanResult result = AiRecordCleaner.cleanRowDetailed(row, aiPlan.columns());
                 rejects.record(lineNo, result);
                 if (result.rejected()) {
+                    errorLogs.add(buildErrorLog(job, lineNo, result.errors(), columnNameCn));
+                    if (errorLogs.size() >= 500) {
+                        flushErrorLogs(errorLogs);
+                    }
                     return;
                 }
                 Map<String, Object> cleaned = new LinkedHashMap<>(result.values());
@@ -569,6 +581,7 @@ public class AiCleanService {
                     writer.add(cleaned);
                 } catch (RuntimeException e) {
                     rejects.recordWriterError(lineNo, e.getMessage());
+                    errorLogs.add(buildWriterErrorLog(job, lineNo, e.getMessage()));
                     throw e;
                 }
             };
@@ -594,10 +607,12 @@ public class AiCleanService {
                 AiDuckDb.dropTable(tpl.getTableNameEn());
             }
             rejects.flush();
+            flushErrorLogs(errorLogs);
             fillLoadStats(job, rejects, 0, true);
             return -1L;
         }
         rejects.flush();
+        flushErrorLogs(errorLogs);
         if (written <= 0) {
             // 一行没落库不能算成功(含「读到 0 行」):全自动链路没有人看日志,
             // 写成 0 行会被上层当成清洗完成。新建表删掉不留幽灵;复用表里还有
@@ -648,6 +663,77 @@ public class AiCleanService {
         return written;
     }
 
+    /**
+     * 构造手动清洗同款失败日志:写进 {@code clean_error_log},由导入文件页的
+     * 「导入失败」弹框统一展示,不再为 AI 清洗单独准备弹框。
+     */
+    private CleanErrorLog buildErrorLog(AiCleanJob job, long lineNo, List<ColumnError> errors,
+                                        Map<String, String> columnNameCn) {
+        CleanErrorLog log = new CleanErrorLog();
+        log.setFileId(job.getFileId());
+        log.setSheetId(job.getSheetId());
+        log.setBatchId(job.getBatchId());
+        log.setFileName(job.getFileName());
+        log.setSheetName(job.getSheetName());
+        log.setContent(formatRowErrors(lineNo, errors, columnNameCn));
+        log.setCreateDateTime(LocalDateTime.now());
+        return log;
+    }
+
+    private CleanErrorLog buildWriterErrorLog(AiCleanJob job, long lineNo, String message) {
+        CleanErrorLog log = new CleanErrorLog();
+        log.setFileId(job.getFileId());
+        log.setSheetId(job.getSheetId());
+        log.setBatchId(job.getBatchId());
+        log.setFileName(job.getFileName());
+        log.setSheetName(job.getSheetName());
+        log.setContent("第 " + lineNo + " 行:写入失败"
+                + (StrUtil.isBlank(message) ? "" : ":" + message));
+        log.setCreateDateTime(LocalDateTime.now());
+        return log;
+    }
+
+    /** 例:第 2 行:c1(身份证号)必填列类型不合法,原值 110105194912310021 */
+    private String formatRowErrors(long lineNo, List<ColumnError> errors, Map<String, String> columnNameCn) {
+        List<String> parts = new ArrayList<>();
+        if (errors != null) {
+            for (ColumnError error : errors) {
+                if (error == null) {
+                    continue;
+                }
+                String column = StrUtil.trim(error.columnName());
+                if (column == null) {
+                    column = "";
+                }
+                String columnCn = column.isEmpty() || columnNameCn == null ? null : columnNameCn.get(column);
+                String columnText = column.isEmpty() ? ""
+                        : StrUtil.isBlank(columnCn) ? column : column + "(" + columnCn + ")";
+                AiRejectReason reason = error.reason();
+                String reasonText = reason == null ? "未知原因" : reason.label();
+                String valueText = StrUtil.isBlank(error.valuePreview())
+                        ? "" : ",原值 " + error.valuePreview();
+                parts.add((columnText.isEmpty() ? "" : columnText + ":") + reasonText + valueText);
+            }
+        }
+        String detail = String.join(";", parts);
+        return "第 " + lineNo + " 行" + (detail.isEmpty() ? "" : ":" + detail);
+    }
+
+    /** 日志写失败不能影响入库,只记录应用日志。 */
+    private void flushErrorLogs(List<CleanErrorLog> logs) {
+        if (logs.isEmpty()) {
+            return;
+        }
+        try {
+            cleanErrorLogMapper.insert(logs);
+        } catch (Exception e) {
+            log.error("AI 清洗失败日志写入 clean_error_log 失败(不影响入库): size={}, err={}",
+                    logs.size(), e.getMessage());
+        } finally {
+            logs.clear();
+        }
+    }
+
     /**
      * 把行级统计写回 job,并合并进 {@code detail.load}。
      *

+ 10 - 11
docs/ai-clean-e2e/README.md

@@ -75,7 +75,7 @@
 - 操作:上传 `05`(20 行,第 1 行身份证 `12345`)→ 智能清洗(指纹命中同一 AI 模板,出口 B 复用)。
 - 预期:19 行入库、1 行拒绝;失败明细能看到行号/列/原因/原值。
 - 实测:`loadedRows=19, rejectedRows=1, unprocessedRows=0`;job `rowTotal=20, rowLoaded=19, rowRejected=1`。
-- 失败明细接口:
+- 行级审计(`ai_clean_reject`,保留用于排障):
 
   ```json
   {"row_no":2,"blocking_col_index":1,"blocking_col_name":"c1",
@@ -85,10 +85,10 @@
   ```
 
 - 失败日志:`AI 清洗拒绝: ... rowNo=2, col=c1(身份证号), reason=REQUIRED_TYPE_INVALID, value=12345`
-- 界面:导入文件列表 → 展开文件 → 工作表行「导入失败 1」→ 弹窗显示:
-  - 文件 / 工作表、**成功 19 / 失败 1**
-  - 行号 2、阻塞列 `c1(身份证号)`、原因「必填列类型不合法」、原值预览 `12345`
-- 截图:[ai-reject-modal.png](screenshots/ai-reject-modal.png)
+- 界面:导入文件列表 → 点「导入失败 1」→「导入失败日志」弹框(与手动清洗同一个弹框):
+  - 文件名称 / 工作表 / 日志内容 / 记录时间
+  - 日志内容:`第 2 行:c1(身份证号):必填列类型不合法,原值 12345`
+- 截图:[import-fail-log-ai.png](screenshots/import-fail-log-ai.png)
 - 后置:`required` 已重置为 0,避免影响后续样本。
 
 ### TC-03 CSV 脏数据:CSV 不再 Skip,可选脏值只置空(04)
@@ -147,12 +147,11 @@
 - 操作:上传 `07`(20 行,第 1 行身份证为校验位错误的完整 18 位 `110105194912310021`)→ 智能清洗。
 - 预期:19 行入库、1 行拒绝;失败明细完整显示 18 位原值(不脱敏、只做超长截断)。
 - 实测:`loadedRows=19, rejectedRows=1`;文件列表「导入成功 19 / 导入失败 1」。
-- 失败明细弹窗:
-  - 行号 `2`
-  - 阻塞列 `c1(身份证号)`(有中文名时括号展示)
-  - 原因「必填列类型不合法」
-  - 原值预览 `110105194912310021`(完整 18 位,无掩码)
-- 截图:[ai-reject-modal-id18.png](screenshots/ai-reject-modal-id18.png)
+- 导入失败日志弹窗(与手动清洗同一个弹框):
+  - 文件名称 / 工作表
+  - 日志内容:`第 2 行:c1(身份证号):必填列类型不合法,原值 110105194912310021`(完整 18 位,无掩码)
+  - 记录时间
+- 截图:[import-fail-log-ai.png](screenshots/import-fail-log-ai.png)
 - 后置:`required` 已重置为 0。
 - 任务日志:进度页内只保留内存日志,离开页面即清空;导入文件页不再提供「任务日志」入口,
   旧版本遗留的 `localStorage.case_clean_task_log` 会在进入进度页时清除。

BIN
docs/ai-clean-e2e/screenshots/ai-reject-modal-id18.png


BIN
docs/ai-clean-e2e/screenshots/ai-reject-modal.png


BIN
docs/ai-clean-e2e/screenshots/import-fail-log-ai.png