Compare commits

..
5 Commits
Author SHA1 Message Date
连龙刚andClaude 7ca5bd4290 feat(t17): 单聊同步+补拉对账+同步消息进分发;修复 get_roam_msg 60008
新增单聊(C2C)同步、补拉对账定时任务,同步消息纳入分发队列推送业务系统。
1.单聊同步④:user遍历→recentcontact/get_list收集C2C会话→会话对去重→
  get_roam_msg时间窗向前滚动全量→落库(conv_type=1/SYNC/进分发)
2.补拉对账⑤:SyncCheckTask定时(1h)+手动按钮,重跑群+单聊幂等补缺失
3.同步消息进分发:群/单聊distStatus=0入DistQueue;限速分批入队
  (next_retry_at按序号错开delay=seq/dispatchQps)
4.重构:enqueue抽到DispatchService公共方法,Pull/Sync共用

修复(get_roam_msg一直60008静默失败,C2C补拉从未工作):
-命令字 openim_admin/get_roam_msg → openim/admin_getroammsg
-字段 From/To/MaxTimeInterval → Operator_Account/Peer_Account/MaxTime
-新增 getRoamMsgRangeAs(MinTime/MaxTime+LastMsgKey续拉)+getRecentContactListAs

联调(sa 16用户):单聊链路跑通(get_list32+admin_getroammsg OK/Complete=1),
幂等正常,补拉定时工作,无死循环。enqueue实证待配callbackUrl+新消息。

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-10 11:21:49 +08:00
连龙刚andClaude 968a5f2e42 fix(t17): 数据同步联调修复——群消息死循环 + 字段名 + schema自动升级 + maxGroups精确
联调(新疆信创测试应用 1600134460,租户sa 67群)发现并修复4类问题:
1.群消息死循环:group_msg_get_simple 的 IsFinished=2 表"还有更旧可拉",占位消息群恒为2;
  原 oldestSeq=max(seq) 游标方向反,反复拉同一批致2217次/200s超时。
  改 min(seq) 向旧翻页 + 游标未推进即停兜底 + 过滤 IsPlaceMsg=1 占位消息(119次/15.7s)
2.get_group_info 请求体字段名 GroupId→GroupIdList(腾讯要求 GroupIdList:["id"] 字符串数组)
3.运行库缺列致 upserted=0:init.sql 仅新建库执行,新增 SchemaUpgradeBootstrap 启动幂等ALTER(6条DDL)
4.maxGroups 防护:break 由 for 外移入 for 内,精确限制避免单页海量群绕过上限

联调验证(租户sa 67群):群组67/67、群成员241/13、群消息119次/imported=3,死循环消除

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-10 09:55:48 +08:00
连龙刚andClaude fdbb7724fa feat(t17): 数据同步——只读拉取群组/群成员/群消息到本地表,按租户查看
主体(T17 数据同步,/admin/sync 三个独立按钮):
- 同步群组:get_appid_group_list 全量 Next 分页 + 逐群 get_group_info → group_mapping
- 同步群成员(→用户):遍历该租户群取 MemberList → user_mapping(腾讯无全量用户API,靠群成员反推)
- 同步群消息:遍历该租户群 getGroupMsg + IsFinished 滚动全量 → im_message(source=SYNC)
- DB: group_mapping 加 name/owner_account/member_count/last_synced_at;user_mapping 加 nick/last_synced_at
  (init.sql 建表 + ADD COLUMN IF NOT EXISTS 老库升级补丁,幂等)
- TencentImClient: 新增 getAppidGroupList(limit,next[,sdkAppId,secretKey])
- 已知限制:C2C单聊无全量会话API;超大群成员需换 get_group_member_info 分页

附带收尾此前未提交的改动:
- 回调字段名修正(FromAccount→From_Account 等腾讯标准字段) + pickMsgRandom/Time/Type 兼容字段差异
- 租户识别重构:前缀经 TenantService.getByPrefixCode 反查 tenantId(主键雪花化与前缀解耦)
- FreeMarker java.time ?string 坑修复(usage/queue Controller 预格式化) + 消息记录"全部"状态修复
- 新增项目 CLAUDE.md + .claude/memory 基建(gitignore 含密钥记忆,不进 git)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-10 08:40:46 +08:00
连龙刚andClaude b679d47adb fix(log): 控制台中文乱码——CONSOLE charset GBK→UTF-8(适配现代终端)
Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-09 17:22:31 +08:00
连龙刚andClaude 4ce030aa74 feat(t16): 消息记录查看(列表+筛选+分页+详情+分发链路)
- ImMessageMapper: +selectMessagePage/+countMessages(动态条件+LIMIT/OFFSET,带时间窗分区裁剪)
- DistQueueMapper: +selectByMsgKey(详情关联分发链路,LIMIT 100)
- AdminController: GET /admin/messages(列表+筛选+分页,默认近7天已分发)+ /admin/messages/detail
- messages.ftl/messages_detail.ftl: 列表+筛选+分页 / 详情+msg_body+dist_queue链路
- _macros.ftl: 侧边栏加消息记录导航
- 时间由Controller预格式化(FreeMarker对java.time支持不稳),msgTimeIso用纳秒精度匹配DB微秒

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-09 17:22:31 +08:00
29 changed files with 2134 additions and 99 deletions
+3
View File
@@ -38,3 +38,6 @@ logs/
# 本地配置覆盖(密钥等敏感信息) # 本地配置覆盖(密钥等敏感信息)
app-env.yml app-env.yml
# Claude 项目记忆(含密码/密钥等敏感信息,不进 git;基准在 .claude/memory/,由 /memory-sync 同步到会话目录)
.claude/memory/
+82
View File
@@ -0,0 +1,82 @@
<!-- Last updated: 2026-07-09 | Commit: b679d47 -->
# tencent-im-util
腾讯 IM 回调分发工具:接收腾讯 IM 的消息回调,落库后按租户/源应用分发。配套管理后台、用量统计、关系链/消息迁移等。
> 本文件为**项目级技术上下文**(行为准则见全局 `~/.claude/CLAUDE.md`)。密钥一律不写在此处,配置项以 `app.yml` 为准。
## 技术栈
- **框架**Solon`solon-parent`,非 Spring),Java 21`solon-web`
- **鉴权**Sa-Token`sa-token-solon-plugin`
- **持久层**MyBatis-Plus`mybatis-plus-extension-solon-plugin`+ PostgreSQL + HikariCP
- **缓存/队列**RedisJedis+ Caffeine 本地缓存
- **视图**FreeMarker`solon-view-freemarker`
- **日志**logback`solon-logging-logback-jakarta`
- **腾讯 IM**`tls-sig-api-v2`UserSig 签名)+ 自封装 `TencentImClient`REST API
- **定时**`solon-scheduling-simple`**工作线程**:自实现 `worker/DispatchWorker`
## 构建与运行
```bash
# 打包(离线模式,跳过测试)
mvn -o package -DskipTests
# 运行
java -jar target/tencent-im-util.jar
```
> ⚠️ **不要用 `mvn solon:solon`**(本项目下不可用,会失败)。详见 `.claude/memory/run-app-jar.md`。
> 端口 `8092`contextPath `/imutil`(即所有路由前缀 `/imutil`)。控制台中文需 UTF-8 终端(已在 `logback.xml` 修正 charset)。
## 目录结构(`src/main/java/com/imutil`
| 包 | 职责 |
|---|---|
| `controller` | `AdminController`(`/admin/*` 后台)、`CallbackController`(`/callback/im` 回调入口)、`HealthController``SigController`(UserSig 下发) |
| `service` + `service/impl` | 业务逻辑(接口 + 实现) |
| `entity` | MyBatis-Plus 实体:`Tenant`/`SourceApp`/`ImMessage`/`DistQueue`/`UsageStat`/`MigrateTask`/`UserMapping`/`GroupMapping`/`Recording`/`TrtcRoom`/`CrossTenantGrant`/`CrossTenantAudit`/`PullWatermark`/`ApiCallLog`/`AdminUser` |
| `mapper` | MyBatis 映射,含 `PartitionMapper`PG 分区管理) |
| `filter` | `AdminAuthFilter`(后台鉴权)、`TenantAuthFilter`(租户鉴权)、`GlobalExceptionFilter`(全局异常) |
| `task` | 定时任务:`DispatchRecoverTask`(死信重投)、`PartitionCreateTask`(分区创建)、`PullCheckTask``UsageStatTask` |
| `tencent` | `TencentImClient`REST API + 429 限速重试)、`TencentCallbackSign`(回调签名校验)、`UserSigUtil` |
| `worker` | `DispatchWorker`(消费分发队列) |
| `common` | 工具类:`PasswordUtil`(PBKDF2)、`RedisService``LocalCache``RateLimiter``Jsons``Httpx``Ids``MsgKeys``TenantContext``HealthService``BizException` |
| `model` | `Result`(统一响应体) |
## 核心调用链
1. **回调分发(主链路)**:腾讯服务器 → `CallbackController#callback`(`/imutil/callback/im`) → `CallbackServiceImpl`(验签 + 落 `ImMessage` + 入 `DistQueue`)→ `DispatchWorker` 消费 → 分发到目标租户/源应用;失败由 `DispatchRecoverTask` 重投。
2. **管理后台**`AdminController`Sa-Token 会话)+ FreeMarker 页面(`resources/templates/*.ftl`):login / home / tenant / sourceapp / grant / queue / usage / password / messages / migrate。
3. **数据迁移**`MigrateService`C2C 全量分页 + 群消息,`msgLookbackDays` 回溯天数,429 自动重试)。
4. **跨租户授权**`CrossTenantService``CrossTenantGrant` 授权 + `CrossTenantAudit` 审计)。
5. **分区与统计**`PartitionService`(按 `msg_time` 分区裁剪)+ `PartitionCreateTask``UsageStatService` + `UsageStatTask`
## 项目约定(务必遵守)
- **Long ID → String**:雪花 ID 等 `Long` 序列化给前端**必须** `String.valueOf()`,否则 JS 精度丢失(全局规则 10.1)。
- **FreeMarker + java.time 坑**ftl 里**禁止**对 `LocalDateTime`/`OffsetDateTime``?string(pattern)`(抛 `NonMethodException`)。改为 Controller 端 `DateTimeFormatter.format` 预格式化成 `xxxStr` 字段,ftl 用 `${obj.xxxStr!}`。详见 `.claude/memory/freemarker-java-time.md`
- **LocalDate / DateTimeFormatter**pattern 含 `H/m/s` 时只能用 `LocalDateTime`,禁用 `LocalDate.now()`(全局规则 10.2)。
- **中文注释**:所有注释用简体中文,标识符保持英文(全局规则 9)。
- **密码**`PasswordUtil` PBKDF2 哈希存 `admin_user.password_hash``iterations:salt:hash`);改密走 `/admin/password`
- **配置不提交**`app.yml` 含密钥(Redis 密码 / `tencent.secretKey` / admin 默认密码),长期处于未提交状态,**勿提交**;如需本地覆盖敏感值用 `app-env.yml`(已 gitignore)。
## 关键配置(`src/main/resources/app.yml`,值为准)
- `server.port` / `server.contextPath`
- `solon.dataSources.db1`PostgreSQL 连接)
- `imutil.redis`host/port/password/database
- `imutil.tencent``sdkAppId`/`secretKey`/`adminUserId`/`apiHost`/`usersigExpireDays`/`callbackToken`——`callbackToken` 留空则跳过回调签名校验,仅联调用)
- `imutil.admin``defaultUsername`/`defaultPassword`——仅首次初始化 `admin_user`
- `imutil.migrate``msgLookbackDays` 等)
- `sa-token.*`
## 文档与外部资源
- **实施记录与测试用例**Obsidian 库 `8 腾讯IM&音视频分发\实施记录与测试用例.md`(项目外,按 T1–T16 任务编号组织;**不进 git**)。
- **项目记忆**`.claude/memory/`(已 gitignore,含账号/密码/密钥提醒等;索引见其 `MEMORY.md`)。召回失效时跑 `/memory-sync` 重新同步到会话目录。
- **数据库操作**:本机无 `psql`,用 DBX MCP`192.168.10.118-pg`database `tencen_im`)查改数据。详见 `.claude/memory/pg-via-dbx.md`
- **回调测试**:端点 `/imutil/callback/im`,本地可用 curl 模拟(见 `.claude/memory/tencent-callback-test.md`)。
## 当前进度
核心功能 T1T15 已完成(T12/T13 TRTC 音视频按需暂缓),T14 消息迁移 V1–V4、T16 消息记录查看已完成。详见实施记录文档与 `.claude/memory/git-author-and-progress.md`
@@ -2,6 +2,7 @@ package com.imutil.controller;
import cn.dev33.satoken.stp.StpUtil; import cn.dev33.satoken.stp.StpUtil;
import com.baomidou.mybatisplus.core.toolkit.Wrappers; import com.baomidou.mybatisplus.core.toolkit.Wrappers;
import com.imutil.common.Ids;
import com.imutil.entity.AdminUser; import com.imutil.entity.AdminUser;
import com.imutil.entity.CrossTenantGrant; import com.imutil.entity.CrossTenantGrant;
import com.imutil.entity.DistQueue; import com.imutil.entity.DistQueue;
@@ -14,13 +15,23 @@ import com.imutil.mapper.TenantMapper;
import com.imutil.mapper.UsageStatMapper; import com.imutil.mapper.UsageStatMapper;
import com.imutil.service.AdminUserService; import com.imutil.service.AdminUserService;
import com.imutil.service.SourceAppService; import com.imutil.service.SourceAppService;
import com.imutil.service.TenantService;
import com.imutil.tencent.TencentImClient; import com.imutil.tencent.TencentImClient;
import com.imutil.entity.ImMessage;
import com.imutil.entity.MigrateTask; import com.imutil.entity.MigrateTask;
import com.imutil.service.MigrateService; import com.imutil.service.MigrateService;
import com.imutil.service.SyncService;
import com.imutil.mapper.ImMessageMapper;
import org.noear.solon.core.handle.UploadedFile; import org.noear.solon.core.handle.UploadedFile;
import java.nio.charset.StandardCharsets; import java.nio.charset.StandardCharsets;
import java.time.format.DateTimeFormatter;
import java.time.LocalDate;
import java.time.OffsetDateTime;
import java.time.ZoneOffset;
import java.util.ArrayList; import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List; import java.util.List;
import java.util.Map;
import lombok.extern.slf4j.Slf4j; import lombok.extern.slf4j.Slf4j;
import org.noear.solon.annotation.Controller; import org.noear.solon.annotation.Controller;
import org.noear.solon.annotation.Get; import org.noear.solon.annotation.Get;
@@ -53,12 +64,18 @@ public class AdminController {
@Inject @Inject
private TenantMapper tenantMapper; private TenantMapper tenantMapper;
@Inject
private TenantService tenantService;
@Inject @Inject
private CrossTenantGrantMapper grantMapper; private CrossTenantGrantMapper grantMapper;
@Inject @Inject
private DistQueueMapper distQueueMapper; private DistQueueMapper distQueueMapper;
@Inject
private ImMessageMapper imMessageMapper;
@Inject @Inject
private UsageStatMapper usageStatMapper; private UsageStatMapper usageStatMapper;
@@ -71,6 +88,9 @@ public class AdminController {
@Inject @Inject
private MigrateService migrateService; private MigrateService migrateService;
@Inject
private SyncService syncService;
// ==================== 登录 / 登出 ==================== // ==================== 登录 / 登出 ====================
@Get @Get
@@ -179,21 +199,30 @@ public class AdminController {
@Post @Post
@Mapping("/tenant/save") @Mapping("/tenant/save")
public void tenantSave(Tenant t, Context ctx) throws Throwable { public void tenantSave(Tenant t, Context ctx) throws Throwable {
if (t.getTenantId() == null || t.getTenantId().isEmpty()) { // tenantId 页面不可见:新增时雪花生成,编辑时由 hidden 携带(主键不可改)
ctx.redirect(basePath + "/admin/tenant?msg=tenant_id_required"); boolean isNew = (t.getTenantId() == null || t.getTenantId().isEmpty());
return; if (isNew) {
} // 前缀码(IM 账号前缀)必填且全局唯一
if (t.getPrefixCode() == null || t.getPrefixCode().isEmpty()) { if (t.getPrefixCode() == null || t.getPrefixCode().isEmpty()) {
t.setPrefixCode(t.getTenantId()); ctx.redirect(basePath + "/admin/tenant?msg=prefix_code_required");
} return;
if (t.getStatus() == null) { }
t.setStatus(1); Tenant exist = tenantMapper.selectOne(Wrappers.<Tenant>lambdaQuery()
} .eq(Tenant::getPrefixCode, t.getPrefixCode()).last("LIMIT 1"));
if (tenantMapper.selectById(t.getTenantId()) == null) { if (exist != null) {
ctx.redirect(basePath + "/admin/tenant?msg=prefix_code_duplicate");
return;
}
t.setTenantId(Ids.nextIdStr());
if (t.getStatus() == null) {
t.setStatus(1);
}
tenantMapper.insert(t); tenantMapper.insert(t);
} else { } else {
tenantMapper.updateById(t); tenantMapper.updateById(t);
} }
// 失效缓存,使后续按 id/appKey/prefix 查询拿到最新值
tenantService.evictCache(t.getTenantId());
ctx.redirect(basePath + "/admin/tenant?msg=saved"); ctx.redirect(basePath + "/admin/tenant?msg=saved");
} }
@@ -204,6 +233,7 @@ public class AdminController {
t.setTenantId(tenantId); t.setTenantId(tenantId);
t.setStatus(status == null ? 0 : status); t.setStatus(status == null ? 0 : status);
tenantMapper.updateById(t); tenantMapper.updateById(t);
tenantService.evictCache(tenantId);
ctx.redirect(basePath + "/admin/tenant?msg=toggled"); ctx.redirect(basePath + "/admin/tenant?msg=toggled");
} }
@@ -251,11 +281,26 @@ public class AdminController {
@Get @Get
@Mapping("/queue") @Mapping("/queue")
public Object queuePage(@Param(defaultValue = "") String msg) { public Object queuePage(@Param(defaultValue = "") String msg) {
ModelAndView mv = view("queue.ftl", "队列监控", "queue"); // nextRetryAt 为 OffsetDateTimeFreeMarker ?string 对 java.time 不支持,Controller 端预格式化
mv.put("pendings", distQueueMapper.selectList(Wrappers.<DistQueue>lambdaQuery() DateTimeFormatter fmt = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss");
List<DistQueue> rawPendings = distQueueMapper.selectList(Wrappers.<DistQueue>lambdaQuery()
.eq(DistQueue::getStatus, 0) .eq(DistQueue::getStatus, 0)
.orderByAsc(DistQueue::getId) .orderByAsc(DistQueue::getId)
.last("LIMIT 50"))); .last("LIMIT 50"));
List<Map<String, Object>> pendings = new ArrayList<>();
if (rawPendings != null) {
for (DistQueue q : rawPendings) {
Map<String, Object> v = new LinkedHashMap<>();
v.put("id", q.getId());
v.put("tenantId", q.getTenantId());
v.put("convId", q.getConvId());
v.put("retryCount", q.getRetryCount());
v.put("nextRetryAtStr", q.getNextRetryAt() == null ? "" : q.getNextRetryAt().format(fmt));
pendings.add(v);
}
}
ModelAndView mv = view("queue.ftl", "队列监控", "queue");
mv.put("pendings", pendings);
mv.put("deads", distQueueMapper.selectList(Wrappers.<DistQueue>lambdaQuery() mv.put("deads", distQueueMapper.selectList(Wrappers.<DistQueue>lambdaQuery()
.eq(DistQueue::getStatus, 3) .eq(DistQueue::getStatus, 3)
.orderByDesc(DistQueue::getId) .orderByDesc(DistQueue::getId)
@@ -278,10 +323,27 @@ public class AdminController {
@Get @Get
@Mapping("/usage") @Mapping("/usage")
public Object usagePage() { public Object usagePage() {
ModelAndView mv = view("usage.ftl", "用量报表", "usage"); // statTime 为 OffsetDateTimeFreeMarker ?string 对 java.time 不支持,Controller 端预格式化
mv.put("stats", usageStatMapper.selectList(Wrappers.<UsageStat>lambdaQuery() List<UsageStat> raw = usageStatMapper.selectList(Wrappers.<UsageStat>lambdaQuery()
.orderByDesc(UsageStat::getStatTime) .orderByDesc(UsageStat::getStatTime)
.last("LIMIT 100"))); .last("LIMIT 100"));
DateTimeFormatter fmt = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm");
List<Map<String, Object>> stats = new ArrayList<>();
if (raw != null) {
for (UsageStat s : raw) {
Map<String, Object> v = new LinkedHashMap<>();
v.put("tenantId", s.getTenantId());
v.put("statTimeStr", s.getStatTime() == null ? "" : s.getStatTime().format(fmt));
v.put("statLevel", s.getStatLevel());
v.put("imMsgCount", s.getImMsgCount());
v.put("imDau", s.getImDau());
v.put("apiCallCount", s.getApiCallCount());
v.put("trtcDurationSec", s.getTrtcDurationSec());
stats.add(v);
}
}
ModelAndView mv = view("usage.ftl", "用量报表", "usage");
mv.put("stats", stats);
return mv; return mv;
} }
@@ -511,6 +573,93 @@ public class AdminController {
} }
} }
// ==================== 数据同步(T17 ====================
/**
* 数据同步页:只读拉取腾讯 IM 群组/群成员/群消息到本地表,供按租户查看。
* 与 migrate(带建群/建号写回)不同,本功能在腾讯侧无任何写操作。
*/
@Get
@Mapping("/sync")
public Object syncPage(@Param(defaultValue = "") String msg) {
ModelAndView mv = view("sync.ftl", "数据同步", "sync");
mv.put("tasks", migrateService.listTasks());
mv.put("apps", sourceAppService.list());
mv.put("tenants", tenantMapper.selectList(null));
mv.put("msg", msg);
return mv;
}
/**
* 同步群组:sourceAppId 空=主应用(需选 tenantId);非空=老应用(密钥+租户取其配置)
*/
@Post
@Mapping("/sync/groups")
public void syncGroups(@Param(required = false) Long sourceAppId,
@Param(defaultValue = "") String tenantId,
Context ctx) throws Throwable {
if (sourceAppId == null && (tenantId == null || tenantId.isEmpty())) {
ctx.redirect(basePath + "/admin/sync?msg=syncopt");
return;
}
syncService.syncGroups(sourceAppId, tenantId);
ctx.redirect(basePath + "/admin/sync?msg=sgdone");
}
/** 同步群成员(→用户):需先同步群组 */
@Post
@Mapping("/sync/users")
public void syncUsers(@Param(required = false) Long sourceAppId,
@Param(defaultValue = "") String tenantId,
Context ctx) throws Throwable {
if (sourceAppId == null && (tenantId == null || tenantId.isEmpty())) {
ctx.redirect(basePath + "/admin/sync?msg=syncopt");
return;
}
syncService.syncGroupMembers(sourceAppId, tenantId);
ctx.redirect(basePath + "/admin/sync?msg=sudone");
}
/** 同步群消息:需先同步群组 */
@Post
@Mapping("/sync/groupmsg")
public void syncGroupMsg(@Param(required = false) Long sourceAppId,
@Param(defaultValue = "") String tenantId,
Context ctx) throws Throwable {
if (sourceAppId == null && (tenantId == null || tenantId.isEmpty())) {
ctx.redirect(basePath + "/admin/sync?msg=syncopt");
return;
}
syncService.syncGroupMessages(sourceAppId, tenantId);
ctx.redirect(basePath + "/admin/sync?msg=smdone");
}
/** 同步单聊消息:需先同步用户(群成员) */
@Post
@Mapping("/sync/c2cmsg")
public void syncC2CMsg(@Param(required = false) Long sourceAppId,
@Param(defaultValue = "") String tenantId,
Context ctx) throws Throwable {
if (sourceAppId == null && (tenantId == null || tenantId.isEmpty())) {
ctx.redirect(basePath + "/admin/sync?msg=syncopt");
return;
}
syncService.syncC2CMessages(sourceAppId, tenantId);
ctx.redirect(basePath + "/admin/sync?msg=scdone");
}
/** 补拉对账:重跑群+单聊(幂等补缺失),仅主应用 + 指定租户 */
@Post
@Mapping("/sync/check")
public void syncCheck(@Param String tenantId, Context ctx) throws Throwable {
if (tenantId == null || tenantId.isEmpty()) {
ctx.redirect(basePath + "/admin/sync?msg=syncopt");
return;
}
syncService.checkAndPull(tenantId);
ctx.redirect(basePath + "/admin/sync?msg=chkdone");
}
// ==================== 公共:构造页面模型 ==================== // ==================== 公共:构造页面模型 ====================
/** /**
@@ -520,6 +669,162 @@ public class AdminController {
* @param title 页面标题 * @param title 页面标题
* @param active 侧边栏高亮项(home/tenant/grant/queue/usage * @param active 侧边栏高亮项(home/tenant/grant/queue/usage
*/ */
// ==================== 消息记录 ====================
/**
* 消息记录列表(已分发消息查询)
* <p>
* 默认最近 7 天 + 已分发(dist_status=1);带 msg_time 范围走分区裁剪。
*/
@Get
@Mapping("/messages")
public Object messagesPage(@Param(defaultValue = "") String tenantId,
@Param(required = false) Integer convType,
@Param(required = false) String distStatus,
@Param(defaultValue = "") String dateFrom,
@Param(defaultValue = "") String dateTo,
@Param(defaultValue = "") String keyword,
@Param(defaultValue = "1") int page,
@Param(defaultValue = "20") int size) {
// 默认时间窗:最近 7 天(带 msg_time 范围触发分区裁剪,避免全分区扫描)
OffsetDateTime now = OffsetDateTime.now();
OffsetDateTime from = parseDateStart(dateFrom, now.minusDays(7));
OffsetDateTime to = parseDateEnd(dateTo, now);
int safePage = Math.max(1, page);
int safeSize = size <= 0 ? 20 : Math.min(size, 200);
int offset = (safePage - 1) * safeSize;
// distStatusnull=首次进入(默认已分发) / ""=用户选"全部" / "0,1,2"=对应状态
// 前端"全部"提交空串,Solon 对带空值的 Integer 参数会判为缺失(defaultValue 不兜底)
// 故改用 String 接收后手动解析,Mapper 以 null 表示不限分发状态。
Integer distStatusInt;
if (distStatus == null) {
distStatusInt = 1;
} else if (distStatus.isEmpty()) {
distStatusInt = null;
} else {
distStatusInt = Integer.valueOf(distStatus);
}
String tid = (tenantId == null || tenantId.isEmpty()) ? null : tenantId;
String kw = (keyword == null || keyword.isEmpty()) ? null : keyword;
List<ImMessage> rawRows = imMessageMapper.selectMessagePage(tid, convType, distStatusInt, from, to, kw, offset, safeSize);
long total = imMessageMapper.countMessages(tid, convType, distStatusInt, from, to, kw);
int totalPages = (int) ((total + safeSize - 1) / Math.max(1, safeSize));
// 转成 Map 并预格式化时间(FreeMarker 对 java.time OffsetDateTime 格式化支持不稳,Controller 端处理)
DateTimeFormatter fmtDisp = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss");
DateTimeFormatter fmtIso = DateTimeFormatter.ofPattern("yyyy-MM-dd'T'HH:mm:ss.SSSSSSSSSxxx");
List<Map<String, Object>> rows = new ArrayList<>();
if (rawRows != null) {
for (ImMessage r : rawRows) {
Map<String, Object> v = new LinkedHashMap<>();
v.put("msgKey", r.getMsgKey());
v.put("msgKeyEnc", r.getMsgKey() == null ? "" : java.net.URLEncoder.encode(r.getMsgKey(), StandardCharsets.UTF_8));
v.put("msgTimeStr", r.getMsgTime() == null ? "" : r.getMsgTime().format(fmtDisp));
v.put("msgTimeIso", r.getMsgTime() == null ? "" : java.net.URLEncoder.encode(r.getMsgTime().format(fmtIso), StandardCharsets.UTF_8));
v.put("tenantId", r.getTenantId());
v.put("convType", r.getConvType());
v.put("convId", r.getConvId());
v.put("fromAccount", r.getFromAccount());
v.put("toAccount", r.getToAccount());
v.put("groupId", r.getGroupId());
v.put("msgType", r.getMsgType());
v.put("source", r.getSource());
v.put("isCrossTenant", r.getIsCrossTenant());
v.put("distStatus", r.getDistStatus());
rows.add(v);
}
}
ModelAndView mv = view("messages.ftl", "消息记录", "messages");
mv.put("rows", rows);
mv.put("total", total);
mv.put("page", safePage);
mv.put("size", safeSize);
mv.put("totalPages", totalPages);
mv.put("tenantId", tenantId == null ? "" : tenantId);
mv.put("convType", convType);
mv.put("distStatus", distStatusInt);
mv.put("dateFrom", dateFrom == null ? "" : dateFrom);
mv.put("dateTo", dateTo == null ? "" : dateTo);
mv.put("keyword", keyword == null ? "" : keyword);
mv.put("tenants", tenantMapper.selectList(null));
return mv;
}
/**
* 消息详情:完整字段 + msg_body + 关联 dist_queue 分发链路
*/
@Get
@Mapping("/messages/detail")
public Object messageDetail(@Param String msgKey, @Param String msgTime) {
ModelAndView mv = view("messages_detail.ftl", "消息详情", "messages");
if (msgKey == null || msgKey.isEmpty() || msgTime == null || msgTime.isEmpty()) {
mv.put("msg", null);
mv.put("dists", java.util.Collections.emptyList());
return mv;
}
OffsetDateTime t;
try {
t = OffsetDateTime.parse(msgTime);
} catch (Exception e) {
mv.put("msg", null);
mv.put("dists", java.util.Collections.emptyList());
return mv;
}
ImMessage m = imMessageMapper.selectOne(Wrappers.<ImMessage>lambdaQuery()
.eq(ImMessage::getMsgKey, msgKey).eq(ImMessage::getMsgTime, t));
mv.put("msg", m);
mv.put("msgTimeStr", (m != null && m.getMsgTime() != null)
? m.getMsgTime().format(DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss.SSS")) : "");
mv.put("msgCreatedStr", (m != null && m.getCreatedAt() != null)
? m.getCreatedAt().format(DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss")) : "");
// dist_queue 分发链路,时间预格式化(同列表原因)
DateTimeFormatter fmtDisp = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss");
List<DistQueue> rawDists = distQueueMapper.selectByMsgKey(msgKey);
List<Map<String, Object>> dists = new ArrayList<>();
if (rawDists != null) {
for (DistQueue d : rawDists) {
Map<String, Object> v = new LinkedHashMap<>();
v.put("id", d.getId());
v.put("targetUrl", d.getTargetUrl());
v.put("tenantId", d.getTenantId());
v.put("status", d.getStatus());
v.put("retryCount", d.getRetryCount());
v.put("createdStr", d.getCreatedAt() == null ? "" : d.getCreatedAt().format(fmtDisp));
v.put("updatedStr", d.getUpdatedAt() == null ? "" : d.getUpdatedAt().format(fmtDisp));
dists.add(v);
}
}
mv.put("dists", dists);
return mv;
}
/** 解析 yyyy-MM-dd 为当天 0 点(+8),空或非法用默认值 */
private OffsetDateTime parseDateStart(String s, OffsetDateTime def) {
if (s == null || s.isEmpty()) {
return def;
}
try {
return LocalDate.parse(s).atStartOfDay(ZoneOffset.ofHours(8)).toOffsetDateTime();
} catch (Exception e) {
return def;
}
}
/** 解析 yyyy-MM-dd 为次日 0 点(+8,开区间覆盖整天),空或非法用默认值 */
private OffsetDateTime parseDateEnd(String s, OffsetDateTime def) {
if (s == null || s.isEmpty()) {
return def;
}
try {
return LocalDate.parse(s).plusDays(1).atStartOfDay(ZoneOffset.ofHours(8)).toOffsetDateTime();
} catch (Exception e) {
return def;
}
}
private ModelAndView view(String tpl, String title, String active) { private ModelAndView view(String tpl, String title, String active) {
ModelAndView mv = new ModelAndView(tpl); ModelAndView mv = new ModelAndView(tpl);
mv.put("basePath", basePath); mv.put("basePath", basePath);
@@ -42,6 +42,7 @@ public class CallbackController {
@Param(value = "RequestTime", required = false) String requestTime, @Param(value = "RequestTime", required = false) String requestTime,
@Param(value = "SdkAppid", required = false) String sdkAppid) throws Throwable { @Param(value = "SdkAppid", required = false) String sdkAppid) throws Throwable {
String body = ctx.body(); String body = ctx.body();
log.info("收到腾讯回调 command={} sdkAppid={} requestTime={} body={}", command, sdkAppid, requestTime, body);
// 1. 签名校验(未配置 Token 时跳过,便于联调,生产必须配置) // 1. 签名校验(未配置 Token 时跳过,便于联调,生产必须配置)
if (callbackToken != null && !callbackToken.isEmpty()) { if (callbackToken != null && !callbackToken.isEmpty()) {
@@ -62,6 +63,7 @@ public class CallbackController {
// 落库失败的消息由补拉服务(T10)兜底 // 落库失败的消息由补拉服务(T10)兜底
result = "{\"ActionStatus\":\"OK\",\"ErrorCode\":0,\"ErrorInfo\":\"\"}"; result = "{\"ActionStatus\":\"OK\",\"ErrorCode\":0,\"ErrorInfo\":\"\"}";
} }
log.info("回调处理完成 command={} result={}", command, result);
ctx.output(result); ctx.output(result);
} }
} }
@@ -30,5 +30,17 @@ public class GroupMapping {
/** 群类型:Public/Private/ChatRoom/AVChatRoom */ /** 群类型:Public/Private/ChatRoom/AVChatRoom */
private String groupType; private String groupType;
/** 群名称(同步拉取自腾讯 group_info.Name */
private String name;
/** 群主账号(同步拉取自腾讯 group_info.Owner_Account */
private String ownerAccount;
/** 群成员数(同步拉取自腾讯 group_info.MemberNum / 成员列表长度) */
private Integer memberCount;
/** 最近一次同步时间(数据同步功能刷新) */
private OffsetDateTime lastSyncedAt;
private OffsetDateTime createdAt; private OffsetDateTime createdAt;
} }
@@ -40,5 +40,11 @@ public class UserMapping {
/** 状态:1=正常 0=封禁 */ /** 状态:1=正常 0=封禁 */
private Integer status; private Integer status;
/** 用户昵称(同步拉取自腾讯群成员资料) */
private String nick;
/** 最近一次同步时间(数据同步功能刷新) */
private OffsetDateTime lastSyncedAt;
private OffsetDateTime createdAt; private OffsetDateTime createdAt;
} }
@@ -86,4 +86,17 @@ public interface DistQueueMapper extends BaseMapper<DistQueue> {
*/ */
@org.apache.ibatis.annotations.Select("SELECT status, count(*) AS cnt FROM dist_queue WHERE status IN (0, 1, 3) GROUP BY status") @org.apache.ibatis.annotations.Select("SELECT status, count(*) AS cnt FROM dist_queue WHERE status IN (0, 1, 3) GROUP BY status")
java.util.List<java.util.Map<String, Object>> countByStatus(); java.util.List<java.util.Map<String, Object>> countByStatus();
/**
* 按消息键查分发记录(消息详情关联分发链路)
* <p>
* msg_key 无独立索引,LIMIT 100 控制扫描范围;大数据量建议加 idx_dq_msgkey。
*
* @param msgKey 消息键
* @return 该消息的分发任务记录(按 id 倒序,最多 100 条)
*/
@org.apache.ibatis.annotations.Select("SELECT id, msg_key, tenant_id, conv_id, target_url, payload, status, retry_count, "
+ "next_retry_at, locked_by, locked_at, created_at, updated_at "
+ "FROM dist_queue WHERE msg_key = #{msgKey} ORDER BY id DESC LIMIT 100")
List<DistQueue> selectByMsgKey(@Param("msgKey") String msgKey);
} }
@@ -3,9 +3,11 @@ package com.imutil.mapper;
import com.baomidou.mybatisplus.core.mapper.BaseMapper; import com.baomidou.mybatisplus.core.mapper.BaseMapper;
import com.imutil.entity.ImMessage; import com.imutil.entity.ImMessage;
import org.apache.ibatis.annotations.Param; import org.apache.ibatis.annotations.Param;
import org.apache.ibatis.annotations.Select;
import org.apache.ibatis.annotations.Update; import org.apache.ibatis.annotations.Update;
import java.time.OffsetDateTime; import java.time.OffsetDateTime;
import java.util.List;
/** /**
* 消息主表 Mapper * 消息主表 Mapper
@@ -29,4 +31,62 @@ public interface ImMessageMapper extends BaseMapper<ImMessage> {
@Param("msgTime") OffsetDateTime msgTime, @Param("msgTime") OffsetDateTime msgTime,
@Param("fromStatus") int fromStatus, @Param("fromStatus") int fromStatus,
@Param("toStatus") int toStatus); @Param("toStatus") int toStatus);
/**
* 分页查询消息(管理后台消息记录列表)
* <p>
* 动态条件 + ORDER BY msg_time DESC + LIMIT/OFFSET。
* 必须带 msg_time 范围(dateFrom/dateTo)触发分区裁剪,避免全分区扫描。
*
* @param tenantId 租户(null/空=全部)
* @param convType 会话类型(null=全部,1=C2C 2=GROUP
* @param distStatus 分发状态(null=全部,0待发/1已发/2失败)
* @param dateFrom 起始时间(含)
* @param dateTo 结束时间(开区间,传次日 0 点以覆盖整天)
* @param keyword 模糊关键字(null/空=无,匹配 from/to/conv_id
* @param offset 偏移
* @param limit 条数
*/
@Select("<script>"
+ "SELECT msg_key, tenant_id, msg_time, conv_type, conv_id, from_account, to_account, group_id, "
+ "msg_type, source, is_cross_tenant, dist_status, created_at "
+ "FROM im_message "
+ "<where>"
+ "<if test='tenantId != null and tenantId != \"\"'>AND tenant_id = #{tenantId} </if>"
+ "<if test='convType != null'>AND conv_type = #{convType} </if>"
+ "<if test='distStatus != null'>AND dist_status = #{distStatus} </if>"
+ "<if test='dateFrom != null'>AND msg_time &gt;= #{dateFrom} </if>"
+ "<if test='dateTo != null'>AND msg_time &lt; #{dateTo} </if>"
+ "<if test='keyword != null and keyword != \"\"'>AND (from_account ILIKE CONCAT('%', #{keyword}, '%') OR to_account ILIKE CONCAT('%', #{keyword}, '%') OR conv_id ILIKE CONCAT('%', #{keyword}, '%')) </if>"
+ "</where>"
+ "ORDER BY msg_time DESC, msg_key DESC "
+ "LIMIT #{limit} OFFSET #{offset}</script>")
List<ImMessage> selectMessagePage(@Param("tenantId") String tenantId,
@Param("convType") Integer convType,
@Param("distStatus") Integer distStatus,
@Param("dateFrom") OffsetDateTime dateFrom,
@Param("dateTo") OffsetDateTime dateTo,
@Param("keyword") String keyword,
@Param("offset") int offset,
@Param("limit") int limit);
/**
* 计数(与 selectMessagePage 同条件,用于分页总数)
*/
@Select("<script>"
+ "SELECT COUNT(*) FROM im_message "
+ "<where>"
+ "<if test='tenantId != null and tenantId != \"\"'>AND tenant_id = #{tenantId} </if>"
+ "<if test='convType != null'>AND conv_type = #{convType} </if>"
+ "<if test='distStatus != null'>AND dist_status = #{distStatus} </if>"
+ "<if test='dateFrom != null'>AND msg_time &gt;= #{dateFrom} </if>"
+ "<if test='dateTo != null'>AND msg_time &lt; #{dateTo} </if>"
+ "<if test='keyword != null and keyword != \"\"'>AND (from_account ILIKE CONCAT('%', #{keyword}, '%') OR to_account ILIKE CONCAT('%', #{keyword}, '%') OR conv_id ILIKE CONCAT('%', #{keyword}, '%')) </if>"
+ "</where></script>")
long countMessages(@Param("tenantId") String tenantId,
@Param("convType") Integer convType,
@Param("distStatus") Integer distStatus,
@Param("dateFrom") OffsetDateTime dateFrom,
@Param("dateTo") OffsetDateTime dateTo,
@Param("keyword") String keyword);
} }
@@ -1,7 +1,9 @@
package com.imutil.service; package com.imutil.service;
import com.imutil.entity.DistQueue; import com.imutil.entity.DistQueue;
import com.imutil.entity.ImMessage;
import java.time.OffsetDateTime;
import java.util.List; import java.util.List;
/** /**
@@ -32,4 +34,24 @@ public interface DispatchService {
* @return 重置条数 * @return 重置条数
*/ */
int recoverStuck(); int recoverStuck();
/**
* 写入分发队列(默认 nextRetryAt=now,立即可被消费分发)
* <p>
* 取租户 callbackUrl 作为目标;租户未配置 callbackUrl 则跳过(不报错,本地留底仍有效)。
*
* @param tenantId 归属租户
* @param msg 已落库消息(payload 取其 msgKey/convType/convId/from/to/msgTime/msgType/msgBody/source
*/
void enqueue(String tenantId, ImMessage msg);
/**
* 写入分发队列并指定下次分发时间(限速平滑分发用)
* <p>
* 同步批量入队时按序号错开 nextRetryAtdelay = seq / dispatchQps 秒),
* DispatchWorker 按时间消费,避免历史消息瞬时打爆业务系统 callbackUrl。
*
* @param nextRetryAt 下次可分发时间
*/
void enqueue(String tenantId, ImMessage msg, OffsetDateTime nextRetryAt);
} }
@@ -0,0 +1,60 @@
package com.imutil.service;
import com.imutil.entity.MigrateTask;
/**
* 数据同步服务(T17
* <p>
* 与 {@link MigrateService}(老应用→主应用的"迁移",带建群/建号写回副作用)不同,
* 本服务定位为<b>只读拉取</b>:把指定腾讯 IM 应用里的群组/群成员/群消息拉到本地表,
* 供管理后台按租户查看,不在腾讯侧产生任何写操作。
* <p>
* 腾讯 IM 现实约束:无"全量用户列表"接口(用户靠群成员反推);C2C 单聊会话通过
* recentcontact/get_list 按"已同步用户"反查({@link #syncC2CMessages}),漫游消息用 get_roam_msg 全量翻页。
* 同步进来的消息(群/单聊)均入分发队列(dist_status=0)推送业务系统,本地同时留底。
*
* @author imutil
*/
public interface SyncService {
/**
* 同步群组:get_appid_group_list 全量 Next 分页 + 逐群 get_group_info 拉资料 → upsert group_mapping
*
* @param sourceAppId 数据源:null=主应用(全局密钥);非空=该老应用密钥
* @param tenantId 数据归属租户(主应用时必填;源应用时以其 tenant_id 为准)
* @return 同步任务记录(含 total/processed/校验结果)
*/
MigrateTask syncGroups(Long sourceAppId, String tenantId);
/**
* 同步群成员(→用户):遍历该租户已同步的群,逐群 get_group_info 取成员 → upsert user_mapping
*
* @see #syncGroups(Long, String) 先同步群组,群成员才有来源
*/
MigrateTask syncGroupMembers(Long sourceAppId, String tenantId);
/**
* 同步群消息:遍历该租户已同步的群,逐群 getGroupMsg + IsFinished 滚动全量 → 写 im_message(source=SYNC,进分发)
*
* @see #syncGroups(Long, String) 先同步群组,群消息才有目标群清单
*/
MigrateTask syncGroupMessages(Long sourceAppId, String tenantId);
/**
* 同步单聊(C2C)消息:遍历该租户已同步用户 → recentcontact/get_list 收集 C2C 会话 →
* 会话对去重 → get_roam_msg 时间窗向前滚动全量 → 写 im_message(conv_type=1, source=SYNC, 进分发)。
* <p>
* 腾讯无全量 C2C 会话列表接口,靠"已同步用户"反查其最近会话;未导入用户返回 50001 跳过。
*
* @see #syncGroupMembers(Long, String) 先同步用户(群成员),单聊才有用户清单可遍历
*/
MigrateTask syncC2CMessages(Long sourceAppId, String tenantId);
/**
* 补拉对账:重跑群消息 + 单聊同步(幂等,本地已有的跳过、缺失的补入),确保本系统数据最全。
* 定时任务 {@link com.imutil.task.SyncCheckTask} 驱动,也可手动触发。
*
* @param tenantId 归属租户(补拉对账仅对主应用归属租户有意义)
*/
MigrateTask checkAndPull(String tenantId);
}
@@ -22,6 +22,16 @@ public interface TenantService {
*/ */
Tenant getById(String tenantId); Tenant getById(String tenantId);
/**
* 根据 prefix_code(IM 账号前缀)查询租户(带缓存)
* <p>
* 回调识别租户时,账号前缀经此反查得 tenant_id。
*
* @param prefixCode IM 前缀码
* @return 租户实体,不存在返回 null
*/
Tenant getByPrefixCode(String prefixCode);
/** /**
* 失效租户缓存(租户变更时调用) * 失效租户缓存(租户变更时调用)
*/ */
@@ -83,7 +83,7 @@ public class CallbackServiceImpl implements CallbackService {
String tenantId = identifyTenant(callbackCommand, node); String tenantId = identifyTenant(callbackCommand, node);
if (tenantId == null) { if (tenantId == null) {
// 无法识别租户(如腾讯系统消息 administrator),不落库不分发 // 无法识别租户(如腾讯系统消息 administrator),不落库不分发
log.debug("回调无法识别租户,跳过 command={} from={}", callbackCommand, node.get("FromAccount").getString()); log.debug("回调无法识别租户,跳过 command={} from={}", callbackCommand, node.get("From_Account").getString());
return ok(); return ok();
} }
@@ -99,7 +99,8 @@ public class CallbackServiceImpl implements CallbackService {
// 跨租户标记:复查授权确认(before 已拦截未授权)。 // 跨租户标记:复查授权确认(before 已拦截未授权)。
// 若 before 未开启/失效,未授权消息会漏到 after,此处查不到授权即告警(兜底发现隔离异常)。 // 若 before 未开启/失效,未授权消息会漏到 after,此处查不到授权即告警(兜底发现隔离异常)。
if (msg.getConvType() != null && msg.getConvType() == 1) { if (msg.getConvType() != null && msg.getConvType() == 1) {
String toTenant = parsePrefix(msg.getConvId()); Tenant toT = tenantService.getByPrefixCode(parsePrefix(msg.getConvId()));
String toTenant = toT == null ? null : toT.getTenantId();
if (toTenant != null && !toTenant.equals(tenantId)) { if (toTenant != null && !toTenant.equals(tenantId)) {
if (crossTenantService.checkSendMsgGrant( if (crossTenantService.checkSendMsgGrant(
tenantId, msg.getFromAccount(), toTenant, msg.getConvId()) != null) { tenantId, msg.getFromAccount(), toTenant, msg.getConvId()) != null) {
@@ -138,13 +139,16 @@ public class CallbackServiceImpl implements CallbackService {
* @return 放行 ok() 或拦截 fail() * @return 放行 ok() 或拦截 fail()
*/ */
private String handleBeforeSendMsg(ONode node) { private String handleBeforeSendMsg(ONode node) {
String from = node.get("FromAccount").getString(); String from = node.get("From_Account").getString();
String to = node.get("ToAccount").getString(); String to = node.get("To_Account").getString();
if (to == null || to.isEmpty()) { if (to == null || to.isEmpty()) {
to = node.get("ToPlayerId").getString(); to = node.get("ToPlayerId").getString();
} }
String fromTenant = parsePrefix(from); // 前缀(prefixCode) → 反查 tenantId(主键雪花化后,授权记录按 tenant_id 存储)
String toTenant = parsePrefix(to); Tenant fromT = tenantService.getByPrefixCode(parsePrefix(from));
Tenant toT = tenantService.getByPrefixCode(parsePrefix(to));
String fromTenant = fromT == null ? null : fromT.getTenantId();
String toTenant = toT == null ? null : toT.getTenantId();
// 同租户或任一租户无法识别(如系统消息):不拦截,放行正常通讯 // 同租户或任一租户无法识别(如系统消息):不拦截,放行正常通讯
if (fromTenant == null || toTenant == null || fromTenant.equals(toTenant)) { if (fromTenant == null || toTenant == null || fromTenant.equals(toTenant)) {
return ok(); return ok();
@@ -160,7 +164,7 @@ public class CallbackServiceImpl implements CallbackService {
String msgKey = null; String msgKey = null;
try { try {
long msgSeq = node.get("MsgSeq").getLong(); long msgSeq = node.get("MsgSeq").getLong();
long msgRandom = node.get("MsgRandom").getLong(); long msgRandom = pickMsgRandom(node);
msgKey = MsgKeys.build(from, to, msgSeq, msgRandom); msgKey = MsgKeys.build(from, to, msgSeq, msgRandom);
} catch (Exception ignore) { } catch (Exception ignore) {
// 回调体缺字段无法算 msgKey,审计仍写(msgKey=null // 回调体缺字段无法算 msgKey,审计仍写(msgKey=null
@@ -215,14 +219,18 @@ public class CallbackServiceImpl implements CallbackService {
* @return 租户ID,识别失败返回 null * @return 租户ID,识别失败返回 null
*/ */
private String identifyTenant(String command, ONode node) { private String identifyTenant(String command, ONode node) {
String from = node.get("FromAccount").getString(); // 账号前缀即 prefixCode(短码),按 prefix_code 反查得 tenantId(主键已雪花化,与前缀解耦)
String from = node.get("From_Account").getString();
if (from != null && !from.isEmpty()) { if (from != null && !from.isEmpty()) {
String tenantId = parsePrefix(from); String prefix = parsePrefix(from);
if (tenantId != null && tenantService.getById(tenantId) != null) { if (prefix != null) {
return tenantId; Tenant t = tenantService.getByPrefixCode(prefix);
if (t != null) {
return t.getTenantId();
}
} }
} }
// 群回调:GroupId 查映射 // 群回调:GroupId 查映射group_mapping.tenant_id 即租户主键,雪花/短码皆正确)
String groupId = node.get("GroupId").getString(); String groupId = node.get("GroupId").getString();
if (groupId != null && !groupId.isEmpty()) { if (groupId != null && !groupId.isEmpty()) {
GroupMapping gm = groupMappingMapper.selectOne(Wrappers.<GroupMapping>lambdaQuery() GroupMapping gm = groupMappingMapper.selectOne(Wrappers.<GroupMapping>lambdaQuery()
@@ -230,11 +238,6 @@ public class CallbackServiceImpl implements CallbackService {
if (gm != null) { if (gm != null) {
return gm.getTenantId(); return gm.getTenantId();
} }
// 群ID也带前缀
String tid = parsePrefix(groupId);
if (tid != null && tenantService.getById(tid) != null) {
return tid;
}
} }
return null; return null;
} }
@@ -269,15 +272,15 @@ public class CallbackServiceImpl implements CallbackService {
* 解析消息体,构造 ImMessage(含幂等键 msg_key * 解析消息体,构造 ImMessage(含幂等键 msg_key
*/ */
private ImMessage parseMessage(String command, ONode node, String tenantId) { private ImMessage parseMessage(String command, ONode node, String tenantId) {
String from = node.get("FromAccount").getString(); String from = node.get("From_Account").getString();
String to = node.get("ToAccount").getString(); String to = node.get("To_Account").getString();
if (to == null || to.isEmpty()) { if (to == null || to.isEmpty()) {
to = node.get("ToPlayerId").getString(); to = node.get("ToPlayerId").getString();
} }
String groupId = node.get("GroupId").getString(); String groupId = node.get("GroupId").getString();
long msgSeq = node.get("MsgSeq").getLong(); long msgSeq = node.get("MsgSeq").getLong();
long msgRandom = node.get("MsgRandom").getLong(); long msgRandom = pickMsgRandom(node);
long msgTs = node.get("MsgTimeStamp").getLong(); long msgTs = pickMsgTime(node);
if (msgTs == 0) { if (msgTs == 0) {
msgTs = System.currentTimeMillis() / 1000L; msgTs = System.currentTimeMillis() / 1000L;
@@ -297,7 +300,7 @@ public class CallbackServiceImpl implements CallbackService {
msg.setFromAccount(from); msg.setFromAccount(from);
msg.setToAccount(groupId != null && !groupId.isEmpty() ? null : to); msg.setToAccount(groupId != null && !groupId.isEmpty() ? null : to);
msg.setGroupId(groupId); msg.setGroupId(groupId);
msg.setMsgType(node.get("MsgType").getString()); msg.setMsgType(pickMsgType(node));
msg.setMsgBody(node.get("MsgBody").toString()); msg.setMsgBody(node.get("MsgBody").toString());
msg.setSource("CALLBACK"); msg.setSource("CALLBACK");
msg.setIsCrossTenant(false); msg.setIsCrossTenant(false);
@@ -313,13 +316,43 @@ public class CallbackServiceImpl implements CallbackService {
if (groupId != null && !groupId.isEmpty()) { if (groupId != null && !groupId.isEmpty()) {
return groupId; return groupId;
} }
String to = node.get("ToAccount").getString(); String to = node.get("To_Account").getString();
if (to == null || to.isEmpty()) { if (to == null || to.isEmpty()) {
to = node.get("ToPlayerId").getString(); to = node.get("ToPlayerId").getString();
} }
return to; return to;
} }
/**
* 消息随机数:C2C=MsgRandom,群=Random,取到非 0 者为准(避免字段差异导致 msg_key 退化为 :0
*/
private long pickMsgRandom(ONode node) {
long v = node.get("MsgRandom").getLong();
return v != 0 ? v : node.get("Random").getLong();
}
/**
* 消息时间(秒):优先 MsgTime(腾讯标准),回退 MsgTimeStamp(兼容历史/异常回调体)
*/
private long pickMsgTime(ONode node) {
long v = node.get("MsgTime").getLong();
return v != 0 ? v : node.get("MsgTimeStamp").getLong();
}
/**
* 消息类型:优先 MsgBody[0].MsgType(群/C2C 均在数组元素内),回退顶层 MsgType
*/
private String pickMsgType(ONode node) {
ONode body = node.get("MsgBody");
if (body.isArray() && body.size() > 0) {
String t = body.get(0).get("MsgType").getString();
if (t != null && !t.isEmpty()) {
return t;
}
}
return node.get("MsgType").getString();
}
/** /**
* 返回腾讯回调成功响应 * 返回腾讯回调成功响应
*/ */
@@ -2,9 +2,13 @@ package com.imutil.service.impl;
import com.imutil.common.Httpx; import com.imutil.common.Httpx;
import com.imutil.entity.DistQueue; import com.imutil.entity.DistQueue;
import com.imutil.entity.ImMessage;
import com.imutil.entity.Tenant;
import com.imutil.mapper.DistQueueMapper; import com.imutil.mapper.DistQueueMapper;
import com.imutil.service.DispatchService; import com.imutil.service.DispatchService;
import com.imutil.service.TenantService;
import lombok.extern.slf4j.Slf4j; import lombok.extern.slf4j.Slf4j;
import org.noear.snack4.ONode;
import org.noear.solon.annotation.Component; import org.noear.solon.annotation.Component;
import org.noear.solon.annotation.Inject; import org.noear.solon.annotation.Inject;
import org.noear.solon.data.annotation.Tran; import org.noear.solon.data.annotation.Tran;
@@ -29,6 +33,9 @@ public class DispatchServiceImpl implements DispatchService {
@Inject @Inject
private DistQueueMapper distQueueMapper; private DistQueueMapper distQueueMapper;
@Inject
private TenantService tenantService;
@Inject("${imutil.dispatch.fetchBatch:50}") @Inject("${imutil.dispatch.fetchBatch:50}")
private int fetchBatch; private int fetchBatch;
@@ -91,4 +98,51 @@ public class DispatchServiceImpl implements DispatchService {
OffsetDateTime threshold = now.minusMinutes(lockTimeoutMin); OffsetDateTime threshold = now.minusMinutes(lockTimeoutMin);
return distQueueMapper.recoverStuck(threshold, now); return distQueueMapper.recoverStuck(threshold, now);
} }
@Override
public void enqueue(String tenantId, ImMessage msg) {
enqueue(tenantId, msg, OffsetDateTime.now());
}
/**
* 写入分发队列(复用回调分发链路,payload 为结构化 JSON
* <p>
* 取租户 callbackUrl 作为目标;未配置则跳过(不报错,本地留底仍有效)。
* nextRetryAt 由调用方控制:同步批量入队时按序号错开以限速平滑分发。
*/
@Override
public void enqueue(String tenantId, ImMessage msg, OffsetDateTime nextRetryAt) {
Tenant t = tenantService.getById(tenantId);
if (t == null || t.getCallbackUrl() == null || t.getCallbackUrl().isEmpty()) {
return;
}
ONode payload = ONode.ofJson("{}");
payload.set("source", msg.getSource());
payload.set("msgKey", msg.getMsgKey());
payload.set("tenantId", tenantId);
payload.set("convType", msg.getConvType());
payload.set("convId", msg.getConvId());
payload.set("fromAccount", msg.getFromAccount());
payload.set("toAccount", msg.getToAccount());
payload.set("groupId", msg.getGroupId());
payload.set("msgTime", msg.getMsgTime());
payload.set("msgType", msg.getMsgType());
if (msg.getMsgBody() != null) {
try {
payload.set("msgBody", ONode.ofJson(msg.getMsgBody()));
} catch (Exception e) {
payload.set("msgBody", msg.getMsgBody());
}
}
DistQueue q = new DistQueue();
q.setMsgKey(msg.getMsgKey());
q.setTenantId(tenantId);
q.setConvId(msg.getConvId());
q.setTargetUrl(t.getCallbackUrl());
q.setPayload(payload.toJson());
q.setStatus(0);
q.setRetryCount(0);
q.setNextRetryAt(nextRetryAt);
distQueueMapper.insert(q);
}
} }
@@ -10,6 +10,7 @@ import com.imutil.entity.Tenant;
import com.imutil.mapper.DistQueueMapper; import com.imutil.mapper.DistQueueMapper;
import com.imutil.mapper.ImMessageMapper; import com.imutil.mapper.ImMessageMapper;
import com.imutil.mapper.PullWatermarkMapper; import com.imutil.mapper.PullWatermarkMapper;
import com.imutil.service.DispatchService;
import com.imutil.service.PullService; import com.imutil.service.PullService;
import com.imutil.service.TenantService; import com.imutil.service.TenantService;
import com.imutil.tencent.TencentImClient; import com.imutil.tencent.TencentImClient;
@@ -56,6 +57,9 @@ public class PullServiceImpl implements PullService {
@Inject @Inject
private TenantService tenantService; private TenantService tenantService;
@Inject
private DispatchService dispatchService;
@Inject("${imutil.pull.convsPerRound:20}") @Inject("${imutil.pull.convsPerRound:20}")
private int convsPerRound; private int convsPerRound;
@@ -168,50 +172,12 @@ public class PullServiceImpl implements PullService {
imMessageMapper.insert(msg); imMessageMapper.insert(msg);
newCount++; newCount++;
// 仅新插入的消息入队分发 // 仅新插入的消息入队分发
enqueue(wm.getTenantId(), msg); dispatchService.enqueue(wm.getTenantId(), msg);
} }
advanceWatermark(wm, maxSeq, maxTime); advanceWatermark(wm, maxSeq, maxTime);
return newCount; return newCount;
} }
/**
* 写入分发队列(复用回调分发链路,payload 用结构化 JSON
*/
private void enqueue(String tenantId, ImMessage msg) {
Tenant t = tenantService.getById(tenantId);
if (t == null || t.getCallbackUrl() == null || t.getCallbackUrl().isEmpty()) {
return;
}
ONode payload = ONode.ofJson("{}");
payload.set("source", msg.getSource());
payload.set("msgKey", msg.getMsgKey());
payload.set("tenantId", tenantId);
payload.set("convType", msg.getConvType());
payload.set("convId", msg.getConvId());
payload.set("fromAccount", msg.getFromAccount());
payload.set("toAccount", msg.getToAccount());
payload.set("groupId", msg.getGroupId());
payload.set("msgTime", msg.getMsgTime());
payload.set("msgType", msg.getMsgType());
if (msg.getMsgBody() != null) {
try {
payload.set("msgBody", ONode.ofJson(msg.getMsgBody()));
} catch (Exception e) {
payload.set("msgBody", msg.getMsgBody());
}
}
DistQueue q = new DistQueue();
q.setMsgKey(msg.getMsgKey());
q.setTenantId(tenantId);
q.setConvId(msg.getConvId());
q.setTargetUrl(t.getCallbackUrl());
q.setPayload(payload.toJson());
q.setStatus(0);
q.setRetryCount(0);
q.setNextRetryAt(OffsetDateTime.now());
distQueueMapper.insert(q);
}
/** /**
* 解析群消息节点为 ImMessage * 解析群消息节点为 ImMessage
*/ */
@@ -0,0 +1,747 @@
package com.imutil.service.impl;
import com.baomidou.mybatisplus.core.toolkit.Wrappers;
import com.imutil.common.BizException;
import com.imutil.common.MsgKeys;
import com.imutil.entity.GroupMapping;
import com.imutil.entity.ImMessage;
import com.imutil.entity.MigrateTask;
import com.imutil.entity.SourceApp;
import com.imutil.entity.Tenant;
import com.imutil.entity.UserMapping;
import com.imutil.mapper.GroupMappingMapper;
import com.imutil.mapper.ImMessageMapper;
import com.imutil.mapper.MigrateTaskMapper;
import com.imutil.mapper.SourceAppMapper;
import com.imutil.mapper.TenantMapper;
import com.imutil.mapper.UserMappingMapper;
import com.imutil.service.DispatchService;
import com.imutil.service.SyncService;
import com.imutil.tencent.TencentImClient;
import lombok.extern.slf4j.Slf4j;
import org.noear.snack4.ONode;
import org.noear.solon.annotation.Component;
import org.noear.solon.annotation.Inject;
import java.time.Instant;
import java.time.OffsetDateTime;
import java.time.ZoneId;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
/**
* 数据同步服务实现(T17
* <p>
* 只读拉取腾讯 IM 数据到本地表,供后台按租户查看。与 MigrateService(带写回副作用)互补。
* <ul>
* <li>群组:get_appid_group_list 全量 Next 分页 + 逐群 get_group_info → upsert group_mapping</li>
* <li>群成员(→用户):遍历该租户已同步群,取 MemberList → upsert user_mapping(腾讯无全量用户接口,靠群成员反推)</li>
* <li>群消息:遍历该租户已同步群,getGroupMsg + IsFinished 滚动全量 → 写 im_message(source=SYNC)</li>
* </ul>
* 数据源密钥:sourceAppId=null 走主应用全局密钥(需指定归属租户);非空走 source_app 密钥(租户取其关联值)。
* 幂等可重跑:群/用户按 imGroupId/imUserId 唯一索引 upsert;消息按 msgKey 查重跳过。
*
* @author imutil
*/
@Slf4j
@Component
public class SyncServiceImpl implements SyncService {
@Inject
private SourceAppMapper sourceAppMapper;
@Inject
private TenantMapper tenantMapper;
@Inject
private GroupMappingMapper groupMappingMapper;
@Inject
private UserMappingMapper userMappingMapper;
@Inject
private ImMessageMapper imMessageMapper;
@Inject
private MigrateTaskMapper migrateTaskMapper;
@Inject
private TencentImClient tencentImClient;
/** 主应用全局密钥(sourceAppId=null 时用) */
@Inject("${imutil.tencent.sdkAppId:0}")
private long configSdkAppId;
@Inject("${imutil.tencent.secretKey:}")
private String configSecretKey;
/** 拉群列表每页数量(get_appid_group_list Limit */
@Inject("${imutil.sync.groupListPageSize:100}")
private int groupListPageSize;
/** 拉群消息每页条数(getGroupMsg reqMsgNumber */
@Inject("${imutil.sync.groupMsgPerPage:100}")
private int groupMsgPerPage;
/** 单群消息分页上限(防异常死循环) */
@Inject("${imutil.sync.groupMsgMaxPages:500}")
private int groupMsgMaxPages;
/** 单次同步群组数量上限(同步阻塞 HTTP,避免海量群卡死;如需全量调大) */
@Inject("${imutil.sync.maxGroups:500}")
private int maxGroups;
/** 同步入队分发限速(每秒入队条数,保护业务系统 callbackUrl;错开 next_retry_at 平滑分发) */
@Inject("${imutil.sync.dispatchQps:10}")
private int syncDispatchQps;
@Inject
private DispatchService dispatchService;
/** 单聊同步用户上限(防海量用户 N×M 爆配额;如需全量调大) */
@Inject("${imutil.sync.maxSyncUsers:200}")
private int maxSyncUsers;
/** 单聊漫游回溯天数(腾讯 C2C 漫游存储期,默认 7 天,套餐更长可调大) */
@Inject("${imutil.sync.c2cLookbackDays:7}")
private int c2cLookbackDays;
/** 数据源密钥解析结果 */
private static class SyncCtx {
long sdkAppId;
String secretKey;
String tenantId;
/** migrate_task.source_app_id;主应用记 0(该字段 NOT NULL,0 表示主应用) */
Long taskSourceAppId;
}
/**
* 解析数据源密钥 + 归属租户
*/
private SyncCtx resolveCtx(Long sourceAppId, String tenantId) {
SyncCtx ctx = new SyncCtx();
if (sourceAppId == null) {
if (tenantId == null || tenantId.isEmpty()) {
throw new BizException("主应用同步需指定归属租户");
}
Tenant t = tenantMapper.selectById(tenantId);
if (t == null) {
throw new BizException("租户不存在: " + tenantId);
}
ctx.sdkAppId = configSdkAppId;
ctx.secretKey = configSecretKey;
ctx.tenantId = tenantId;
ctx.taskSourceAppId = 0L;
} else {
SourceApp app = sourceAppMapper.selectById(sourceAppId);
if (app == null) {
throw new BizException("老应用不存在");
}
ctx.sdkAppId = Long.parseLong(app.getSdkAppId());
ctx.secretKey = app.getSecretKey();
ctx.tenantId = app.getTenantId();
ctx.taskSourceAppId = sourceAppId;
}
if (ctx.sdkAppId <= 0 || ctx.secretKey == null || ctx.secretKey.isEmpty()) {
throw new BizException("数据源密钥未配置");
}
return ctx;
}
private MigrateTask startTask(SyncCtx ctx, String objectType) {
MigrateTask task = new MigrateTask();
task.setSourceAppId(ctx.taskSourceAppId);
task.setObjectType(objectType);
task.setStatus(1);
task.setStartedAt(OffsetDateTime.now());
migrateTaskMapper.insert(task);
return task;
}
// ==================== 同步群组 ====================
@Override
public MigrateTask syncGroups(Long sourceAppId, String tenantId) {
SyncCtx ctx = resolveCtx(sourceAppId, tenantId);
MigrateTask task = startTask(ctx, "GROUP_SYNC");
try {
int total = 0;
int upserted = 0;
String next = "";
int guard = 0;
// Next 游标分页拉全量群,直到返回 Next 为空
while (guard++ < 100000) {
String resp = tencentImClient.getAppidGroupList(groupListPageSize, next, ctx.sdkAppId, ctx.secretKey);
ONode root = ONode.ofJson(resp);
if (!"OK".equalsIgnoreCase(root.get("ActionStatus").getString())) {
throw new BizException("拉群组列表失败: " + safeErr(root));
}
ONode groupIds = root.get("GroupIdList");
int n = groupIds.size();
for (int i = 0; i < n; i++) {
// 实测 GroupIdList 元素为对象 {GroupId:"..."},兼容字符串形态
ONode gidNode = groupIds.get(i);
String groupId = gidNode.isObject() ? gidNode.get("GroupId").getString() : gidNode.getString();
if (groupId == null || groupId.isEmpty()) {
continue;
}
total++;
try {
if (upsertGroupFromRemote(ctx, groupId)) {
upserted++;
}
} catch (Exception ge) {
log.warn("同步单群失败 {} : {}", groupId, ge.toString());
}
// 达上限停止(for 内精确限制,避免单页海量群卡死同步 HTTP)
if (total >= maxGroups) {
log.warn("群组同步达上限 {},停止(调大 imutil.sync.maxGroups 可继续)", maxGroups);
break;
}
}
task.setTotalCount((long) total);
task.setProcessedCount((long) upserted);
task.setPosCursor("page:" + guard);
migrateTaskMapper.updateById(task);
// 达上限停止(由 for 内触发并告警,此处仅跳出 while)
if (total >= maxGroups) {
break;
}
// Next 游标推进;为空或未变化则结束(防死循环)
String respNext = root.get("Next").getString();
if (respNext == null || respNext.isEmpty() || respNext.equals(next)) {
break;
}
next = respNext;
}
task.setStatus(3);
task.setFinishedAt(OffsetDateTime.now());
task.setVerifyResult(String.format("{\"total\":%d,\"upserted\":%d}", total, upserted));
migrateTaskMapper.updateById(task);
log.info("群组同步完成 tenant={} total={} upserted={}", ctx.tenantId, total, upserted);
return task;
} catch (Exception e) {
failTask(task, e);
log.error("群组同步失败 tenant={}", ctx.tenantId, e);
throw e;
}
}
/**
* 拉单群资料 → upsert group_mapping
*
* @return 是否成功处理(已记录为 true)
*/
private boolean upsertGroupFromRemote(SyncCtx ctx, String groupId) {
String resp = tencentImClient.getGroupInfo(groupId, ctx.sdkAppId, ctx.secretKey);
ONode root = ONode.ofJson(resp);
if (!"OK".equalsIgnoreCase(root.get("ActionStatus").getString())) {
return false;
}
ONode groupInfo = root.get("GroupInfo");
if (groupInfo.size() == 0) {
return false;
}
ONode g = groupInfo.get(0);
String name = g.get("Name").getString();
String type = g.get("Type").getString();
String owner = g.get("Owner_Account").getString();
long memberNum = g.get("MemberNum").getLong();
OffsetDateTime now = OffsetDateTime.now();
GroupMapping exist = groupMappingMapper.selectOne(Wrappers.<GroupMapping>lambdaQuery()
.eq(GroupMapping::getImGroupId, groupId).last("LIMIT 1"));
if (exist != null) {
exist.setTenantId(ctx.tenantId);
exist.setBizGroupId(groupId);
exist.setGroupType(type != null && !type.isEmpty() ? type : "Public");
exist.setName(name);
exist.setOwnerAccount(owner);
exist.setMemberCount((int) memberNum);
exist.setLastSyncedAt(now);
groupMappingMapper.updateById(exist);
} else {
GroupMapping gm = new GroupMapping();
gm.setTenantId(ctx.tenantId);
gm.setBizGroupId(groupId);
gm.setImGroupId(groupId);
gm.setGroupType(type != null && !type.isEmpty() ? type : "Public");
gm.setName(name);
gm.setOwnerAccount(owner);
gm.setMemberCount((int) memberNum);
gm.setLastSyncedAt(now);
groupMappingMapper.insert(gm);
}
return true;
}
// ==================== 同步群成员 → 用户 ====================
@Override
public MigrateTask syncGroupMembers(Long sourceAppId, String tenantId) {
SyncCtx ctx = resolveCtx(sourceAppId, tenantId);
MigrateTask task = startTask(ctx, "USER_SYNC");
try {
List<GroupMapping> groups = groupMappingMapper.selectList(Wrappers.<GroupMapping>lambdaQuery()
.eq(GroupMapping::getTenantId, ctx.tenantId));
task.setTotalCount((long) groups.size());
migrateTaskMapper.updateById(task);
int memberSeen = 0;
int upserted = 0;
for (int gi = 0; gi < groups.size(); gi++) {
String groupId = groups.get(gi).getImGroupId();
try {
String resp = tencentImClient.getGroupInfo(groupId, ctx.sdkAppId, ctx.secretKey);
ONode root = ONode.ofJson(resp);
if (!"OK".equalsIgnoreCase(root.get("ActionStatus").getString())) {
continue;
}
ONode groupInfo = root.get("GroupInfo");
if (groupInfo.size() == 0) {
continue;
}
ONode memberList = groupInfo.get(0).get("MemberList");
int mc = memberList.size();
for (int j = 0; j < mc; j++) {
ONode mem = memberList.get(j);
String accountId = mem.get("Member_Account").getString();
if (accountId == null || accountId.isEmpty()) {
continue;
}
memberSeen++;
String nick = mem.get("Name").getString();
if (upsertUser(ctx, accountId, nick)) {
upserted++;
}
}
} catch (Exception ge) {
log.warn("同步群成员失败 {} : {}", groupId, ge.getMessage());
}
task.setProcessedCount((long) (gi + 1));
task.setPosCursor("group:" + (gi + 1));
migrateTaskMapper.updateById(task);
}
task.setStatus(3);
task.setFinishedAt(OffsetDateTime.now());
task.setVerifyResult(String.format("{\"groups\":%d,\"memberSeen\":%d,\"upserted\":%d}",
groups.size(), memberSeen, upserted));
migrateTaskMapper.updateById(task);
log.info("群成员同步完成 tenant={} groups={} memberSeen={} upserted={}",
ctx.tenantId, groups.size(), memberSeen, upserted);
return task;
} catch (Exception e) {
failTask(task, e);
log.error("群成员同步失败 tenant={}", ctx.tenantId, e);
throw e;
}
}
/**
* upsert user_mapping(按 im_user_id 唯一索引)
*
* @return 是否为本次新增
*/
private boolean upsertUser(SyncCtx ctx, String imUserId, String nick) {
UserMapping exist = userMappingMapper.selectOne(Wrappers.<UserMapping>lambdaQuery()
.eq(UserMapping::getImUserId, imUserId).last("LIMIT 1"));
OffsetDateTime now = OffsetDateTime.now();
if (exist != null) {
exist.setNick(nick);
exist.setLastSyncedAt(now);
if (exist.getStatus() == null) {
exist.setStatus(1);
}
userMappingMapper.updateById(exist);
return false;
}
UserMapping um = new UserMapping();
um.setTenantId(ctx.tenantId);
// 同步场景无业务侧 ID,用 IM ID 兜底(biz_user_id 仅做本地标识)
um.setBizUserId(imUserId);
um.setImUserId(imUserId);
um.setIsDefault(false);
um.setIsGlobal(false);
um.setStatus(1);
um.setNick(nick);
um.setLastSyncedAt(now);
userMappingMapper.insert(um);
return true;
}
// ==================== 同步群消息 ====================
@Override
public MigrateTask syncGroupMessages(Long sourceAppId, String tenantId) {
SyncCtx ctx = resolveCtx(sourceAppId, tenantId);
MigrateTask task = startTask(ctx, "GMSG_SYNC");
try {
List<GroupMapping> groups = groupMappingMapper.selectList(Wrappers.<GroupMapping>lambdaQuery()
.eq(GroupMapping::getTenantId, ctx.tenantId));
task.setTotalCount((long) groups.size());
migrateTaskMapper.updateById(task);
int imported = 0;
OffsetDateTime syncBase = OffsetDateTime.now();
for (int gi = 0; gi < groups.size(); gi++) {
String groupId = groups.get(gi).getImGroupId();
try {
long reqSeq = 0;
int pages = 0;
// IsFinished 滚动全量:reqSeq 取本页最旧 seq 继续向前
while (pages++ < groupMsgMaxPages) {
String resp = tencentImClient.getGroupMsgAs(groupId, groupMsgPerPage, reqSeq, ctx.sdkAppId, ctx.secretKey);
ONode root = ONode.ofJson(resp);
if (!"OK".equalsIgnoreCase(root.get("ActionStatus").getString())) {
break;
}
ONode rspList = root.get("RspMsgList");
int n = rspList.size();
long oldestSeq = Long.MAX_VALUE; // 本页最小 seq(最旧)
for (int i = 0; i < n; i++) {
ONode m = rspList.get(i);
long seq = m.get("MsgSeq").getLong();
if (seq > 0 && seq < oldestSeq) {
oldestSeq = seq;
}
ImMessage msg = parseGroupMsgForSync(m, ctx.tenantId, groupId);
if (msg == null) {
continue;
}
// 幂等:本地已存在跳过
long cnt = imMessageMapper.selectCount(Wrappers.<ImMessage>lambdaQuery()
.eq(ImMessage::getMsgKey, msg.getMsgKey()));
if (cnt > 0) {
continue;
}
imMessageMapper.insert(msg);
imported++;
// 限速入队分发:按序号错开 next_retry_atDispatchWorker 按速率消费
dispatchService.enqueue(ctx.tenantId, msg, nextRetryAt(imported, syncBase));
}
boolean finished = root.get("IsFinished").getLong() == 1;
// 终止:已拉完 / 空页 / 本页无有效 seq
if (finished || n == 0 || oldestSeq == Long.MAX_VALUE) {
break;
}
// 游标未向更旧方向推进则停(占位消息群 IsFinished 恒为 2,靠此兜底防死循环)
if (reqSeq > 0 && oldestSeq >= reqSeq) {
break;
}
reqSeq = oldestSeq;
}
} catch (Exception ge) {
log.warn("同步群消息失败 {} : {}", groupId, ge.getMessage());
}
task.setProcessedCount((long) imported);
task.setPosCursor("group:" + (gi + 1));
migrateTaskMapper.updateById(task);
}
task.setStatus(3);
task.setFinishedAt(OffsetDateTime.now());
task.setVerifyResult(String.format("{\"groups\":%d,\"imported\":%d}", groups.size(), imported));
migrateTaskMapper.updateById(task);
log.info("群消息同步完成 tenant={} groups={} imported={}", ctx.tenantId, groups.size(), imported);
return task;
} catch (Exception e) {
failTask(task, e);
log.error("群消息同步失败 tenant={}", ctx.tenantId, e);
throw e;
}
}
/**
* 解析腾讯群消息节点为 ImMessagesource=SYNCdist_status=0 进分发)
*/
private ImMessage parseGroupMsgForSync(ONode m, String tenantId, String groupId) {
// 过滤占位/系统消息(IsPlaceMsg=1From_Account 与 MsgBody 均空,非真实消息,不入库)
if (m.get("IsPlaceMsg").getLong() == 1) {
return null;
}
long msgSeq = m.get("MsgSeq").getLong();
long msgRandom = m.get("MsgRandom").getLong();
long msgTs = m.get("MsgTimeStamp").getLong();
String from = m.get("From_Account").getString();
ONode body = m.get("MsgBody");
ImMessage msg = new ImMessage();
msg.setMsgKey(MsgKeys.build(from, groupId, msgSeq, msgRandom));
msg.setTenantId(tenantId);
msg.setMsgTime(toTime(msgTs));
msg.setConvType(2);
msg.setConvId(groupId);
msg.setFromAccount(from);
msg.setGroupId(groupId);
msg.setMsgType(firstBodyType(body));
msg.setMsgBody(body.toString());
msg.setSource("SYNC");
msg.setIsCrossTenant(false);
msg.setDistStatus(0); // 进分发
return msg;
}
// ==================== 同步单聊(C2C) ====================
/**
* 同步单聊消息:遍历已同步用户 → recentcontact/get_list 收集 C2C 会话 → 会话对去重 →
* get_roam_msg 时间窗向前滚动全量 → 落库(conv_type=1/source=SYNC/进分发)。
*/
@Override
public MigrateTask syncC2CMessages(Long sourceAppId, String tenantId) {
SyncCtx ctx = resolveCtx(sourceAppId, tenantId);
MigrateTask task = startTask(ctx, "C2C_SYNC");
try {
List<UserMapping> users = userMappingMapper.selectList(Wrappers.<UserMapping>lambdaQuery()
.eq(UserMapping::getTenantId, ctx.tenantId));
task.setTotalCount((long) users.size());
migrateTaskMapper.updateById(task);
Set<String> seenPairs = new HashSet<>();
int imported = 0;
OffsetDateTime syncBase = OffsetDateTime.now();
int userCnt = 0;
for (UserMapping u : users) {
if (++userCnt > maxSyncUsers) {
log.warn("单聊同步用户达上限 {},停止(调大 imutil.sync.maxSyncUsers 可继续)", maxSyncUsers);
break;
}
String self = u.getImUserId();
if (self == null || self.isEmpty()) {
continue;
}
try {
List<String> peers = collectC2CPeers(self, ctx);
for (String peer : peers) {
if (peer == null || peer.isEmpty() || peer.equals(self)) {
continue;
}
// 会话对去重:A↔B 只拉一次
if (!seenPairs.add(pairKey(self, peer))) {
continue;
}
imported += pullRoamMsgs(self, peer, ctx, syncBase, imported);
}
} catch (Exception ue) {
// 50001(用户未导入)等按用户级跳过
log.warn("单聊同步用户失败 {} : {}", self, ue.toString());
}
task.setProcessedCount((long) imported);
task.setPosCursor("user:" + userCnt);
migrateTaskMapper.updateById(task);
}
task.setStatus(3);
task.setFinishedAt(OffsetDateTime.now());
task.setVerifyResult(String.format("{\"users\":%d,\"pairs\":%d,\"imported\":%d}",
Math.min(userCnt, maxSyncUsers), seenPairs.size(), imported));
migrateTaskMapper.updateById(task);
log.info("单聊同步完成 tenant={} users={} pairs={} imported={}",
ctx.tenantId, Math.min(userCnt, maxSyncUsers), seenPairs.size(), imported);
return task;
} catch (Exception e) {
failTask(task, e);
log.error("单聊同步失败 tenant={}", ctx.tenantId, e);
throw e;
}
}
/**
* 分页拉取用户最近会话,收集 C2C(Type=1) 对端账号
*/
private List<String> collectC2CPeers(String self, SyncCtx ctx) {
List<String> peers = new ArrayList<>();
long timeStamp = 0;
long startIndex = 0;
int guard = 0;
while (guard++ < 1000) {
String resp = tencentImClient.getRecentContactListAs(self, timeStamp, startIndex,
ctx.sdkAppId, ctx.secretKey);
ONode root = ONode.ofJson(resp);
if (!"OK".equalsIgnoreCase(root.get("ActionStatus").getString())) {
break;
}
ONode items = root.get("SessionItem");
int n = items.size();
for (int i = 0; i < n; i++) {
ONode item = items.get(i);
if (item.get("Type").getLong() == 1) { // 1=C2C
String to = item.get("To_Account").getString();
if (to != null && !to.isEmpty()) {
peers.add(to);
}
}
}
long complete = root.get("CompleteFlag").getLong();
if (complete == 1 || n == 0) {
break;
}
long ts = root.get("TimeStamp").getLong();
long si = root.get("StartIndex").getLong();
if (ts == timeStamp && si == startIndex) {
break; // 游标未推进防死循环
}
timeStamp = ts;
startIndex = si;
}
return peers;
}
/**
* 全量拉取两人漫游消息(向旧翻页),落库 + 限速入队分发,返回新增条数
*/
private int pullRoamMsgs(String self, String peer, SyncCtx ctx, OffsetDateTime syncBase, int seqBase) {
long minTime = Instant.now().getEpochSecond() - c2cLookbackDays * 86400L;
long maxTime = Instant.now().getEpochSecond();
String lastMsgKey = null;
int imported = 0;
int pages = 0;
while (pages++ < groupMsgMaxPages) {
String resp = tencentImClient.getRoamMsgRangeAs(self, peer, groupMsgPerPage, minTime, maxTime,
lastMsgKey, ctx.sdkAppId, ctx.secretKey);
ONode root = ONode.ofJson(resp);
if (!"OK".equalsIgnoreCase(root.get("ActionStatus").getString())) {
break;
}
ONode list = root.get("MsgList");
int n = list.size();
for (int i = 0; i < n; i++) {
ONode m = list.get(i);
ImMessage msg = parseC2CMsgForSync(m, ctx.tenantId, self, peer);
if (msg == null) {
continue;
}
long cnt = imMessageMapper.selectCount(Wrappers.<ImMessage>lambdaQuery()
.eq(ImMessage::getMsgKey, msg.getMsgKey()));
if (cnt > 0) {
continue;
}
imMessageMapper.insert(msg);
imported++;
dispatchService.enqueue(ctx.tenantId, msg, nextRetryAt(seqBase + imported, syncBase));
}
long complete = root.get("Complete").getLong();
long lastMsgTime = root.get("LastMsgTime").getLong();
String nextLastMsgKey = root.get("LastMsgKey").getString();
// 终止:已拉完 / 空页 / 无 LastMsgTime
if (complete == 1 || n == 0 || lastMsgTime <= 0) {
break;
}
// 游标未推进防死循环(LastMsgTime 未向旧推进且 LastMsgKey 未变)
if (lastMsgTime >= maxTime && (nextLastMsgKey == null || nextLastMsgKey.equals(lastMsgKey))) {
break;
}
maxTime = lastMsgTime;
lastMsgKey = nextLastMsgKey;
}
return imported;
}
/**
* 解析腾讯单聊消息节点为 ImMessagesource=SYNC,进分发)
*/
private ImMessage parseC2CMsgForSync(ONode m, String tenantId, String from, String to) {
long msgSeq = m.get("MsgSeq").getLong();
long msgRandom = m.get("MsgRandom").getLong();
long msgTs = m.get("MsgTimeStamp").getLong();
String fromAcc = m.get("From_Account").getString();
if (fromAcc == null || fromAcc.isEmpty()) {
fromAcc = from;
}
ONode body = m.get("MsgBody");
ImMessage msg = new ImMessage();
msg.setMsgKey(MsgKeys.build(fromAcc, to, msgSeq, msgRandom));
msg.setTenantId(tenantId);
msg.setMsgTime(toTime(msgTs));
msg.setConvType(1);
msg.setConvId(to);
msg.setFromAccount(fromAcc);
msg.setToAccount(to);
msg.setMsgType(firstBodyType(body));
msg.setMsgBody(body.toString());
msg.setSource("SYNC");
msg.setIsCrossTenant(false);
msg.setDistStatus(0); // 进分发
return msg;
}
/** 会话对去重键:min#max,使 A↔B 只拉一次 */
private String pairKey(String a, String b) {
return (a.compareTo(b) <= 0) ? a + "#" + b : b + "#" + a;
}
// ==================== 补拉对账 ====================
/**
* 补拉对账:顺序重跑群消息 + 单聊同步(幂等,本地已有的跳过、缺失的补入)。
* 供定时任务 SyncCheckTask 与后台手动按钮调用,确保本系统数据最全。
*/
@Override
public MigrateTask checkAndPull(String tenantId) {
SyncCtx ctx = resolveCtx(null, tenantId);
MigrateTask task = startTask(ctx, "CHECK_PULL");
try {
task.setPosCursor("start");
migrateTaskMapper.updateById(task);
log.info("补拉对账开始 tenant={}", tenantId);
MigrateTask g = syncGroupMessages(null, tenantId);
MigrateTask c = syncC2CMessages(null, tenantId);
long gImp = g.getProcessedCount() == null ? 0 : g.getProcessedCount();
long cImp = c.getProcessedCount() == null ? 0 : c.getProcessedCount();
task.setStatus(3);
task.setFinishedAt(OffsetDateTime.now());
task.setVerifyResult(String.format("{\"groupImported\":%d,\"c2cImported\":%d}", gImp, cImp));
migrateTaskMapper.updateById(task);
log.info("补拉对账完成 tenant={} groupImported={} c2cImported={}", tenantId, gImp, cImp);
return task;
} catch (Exception e) {
failTask(task, e);
log.error("补拉对账失败 tenant={}", tenantId, e);
throw e;
}
}
// ==================== 工具方法 ====================
private void failTask(MigrateTask task, Exception e) {
task.setStatus(4);
task.setErrorMsg(e.getClass().getSimpleName() + ": " + e.getMessage());
task.setFinishedAt(OffsetDateTime.now());
migrateTaskMapper.updateById(task);
}
private String safeErr(ONode root) {
try {
String info = root.get("ErrorInfo").getString();
return info == null ? "" : info;
} catch (Exception e) {
return "";
}
}
private String firstBodyType(ONode body) {
if (body == null || !body.isArray() || body.size() == 0) {
return null;
}
return body.get(0).get("MsgType").getString();
}
private OffsetDateTime toTime(long epochSecond) {
if (epochSecond <= 0) {
return OffsetDateTime.now();
}
return OffsetDateTime.ofInstant(Instant.ofEpochSecond(epochSecond), ZoneId.systemDefault());
}
/**
* 计算限速入队的 next_retry_at:按序号错开(delay = seq / dispatchQps 秒),
* 使同步进来的历史消息按速率平滑分发,避免瞬时打爆业务系统 callbackUrl。
* dispatchQps&lt;=0 时不限速,立即 next_retry_at=now。
*/
private OffsetDateTime nextRetryAt(long seq, OffsetDateTime base) {
if (syncDispatchQps <= 0) {
return OffsetDateTime.now();
}
return base.plusSeconds(seq / syncDispatchQps);
}
}
@@ -24,6 +24,7 @@ public class TenantServiceImpl implements TenantService {
private static final String CACHE_KEY_BY_KEY = "imutil:tenant:bykey:"; private static final String CACHE_KEY_BY_KEY = "imutil:tenant:bykey:";
private static final String CACHE_KEY_BY_ID = "imutil:tenant:byid:"; private static final String CACHE_KEY_BY_ID = "imutil:tenant:byid:";
private static final String CACHE_KEY_BY_PREFIX = "imutil:tenant:byprefix:";
private static final long CACHE_TTL_SEC = 300; private static final long CACHE_TTL_SEC = 300;
@Inject @Inject
@@ -84,6 +85,33 @@ public class TenantServiceImpl implements TenantService {
return t; return t;
} }
@Override
public Tenant getByPrefixCode(String prefixCode) {
if (prefixCode == null || prefixCode.isEmpty()) {
return null;
}
String key = CACHE_KEY_BY_PREFIX + prefixCode;
// 1. 本地缓存
Tenant t = localCache.get(key, Tenant.class);
if (t != null) {
return t;
}
// 2. Redis
t = redisService.getJson(key, Tenant.class);
if (t == null) {
// 3. DB
t = tenantMapper.selectOne(Wrappers.<Tenant>lambdaQuery()
.eq(Tenant::getPrefixCode, prefixCode).last("LIMIT 1"));
if (t != null) {
redisService.setJson(key, t, CACHE_TTL_SEC, TimeUnit.SECONDS);
}
}
if (t != null) {
localCache.put(key, t);
}
return t;
}
@Override @Override
public void evictCache(String tenantId) { public void evictCache(String tenantId) {
Tenant t = tenantMapper.selectById(tenantId); Tenant t = tenantMapper.selectById(tenantId);
@@ -94,6 +122,9 @@ public class TenantServiceImpl implements TenantService {
String k2 = CACHE_KEY_BY_KEY + t.getAppKey(); String k2 = CACHE_KEY_BY_KEY + t.getAppKey();
redisService.del(k2); redisService.del(k2);
localCache.invalidate(k2); localCache.invalidate(k2);
String k3 = CACHE_KEY_BY_PREFIX + t.getPrefixCode();
redisService.del(k3);
localCache.invalidate(k3);
} }
} }
} }
@@ -0,0 +1,47 @@
package com.imutil.task;
import com.imutil.mapper.PartitionMapper;
import lombok.extern.slf4j.Slf4j;
import org.noear.solon.annotation.Component;
import org.noear.solon.annotation.Init;
import org.noear.solon.annotation.Inject;
/**
* 数据库 Schema 启动自动升级(幂等)
* <p>
* 解决"实体加字段但运行库未 ALTER"问题:init.sql 仅新建库时执行,老库不会自动加列,
* 导致 MyBatis insert/update 到新列时报 column 不存在。启动时对增量列执行
* ADD COLUMN IF NOT EXISTS,老库自动补列,新库重复执行也无副作用。
*
* @author imutil
*/
@Slf4j
@Component
public class SchemaUpgradeBootstrap {
@Inject
private PartitionMapper partitionMapper;
@Init
public void init() {
// T17: group_mapping/user_mapping 同步资料列
String[] ddls = {
"ALTER TABLE group_mapping ADD COLUMN IF NOT EXISTS name varchar(256)",
"ALTER TABLE group_mapping ADD COLUMN IF NOT EXISTS owner_account varchar(128)",
"ALTER TABLE group_mapping ADD COLUMN IF NOT EXISTS member_count int",
"ALTER TABLE group_mapping ADD COLUMN IF NOT EXISTS last_synced_at timestamptz",
"ALTER TABLE user_mapping ADD COLUMN IF NOT EXISTS nick varchar(128)",
"ALTER TABLE user_mapping ADD COLUMN IF NOT EXISTS last_synced_at timestamptz",
};
int ok = 0;
for (String sql : ddls) {
try {
partitionMapper.execute(sql);
ok++;
} catch (Exception e) {
log.warn("schema升级跳过 [{}] : {}", sql, e.getMessage());
}
}
log.info("启动 schema 升级完成({}/{} 条 DDL", ok, ddls.length);
}
}
@@ -0,0 +1,59 @@
package com.imutil.task;
import com.imutil.entity.Tenant;
import com.imutil.mapper.TenantMapper;
import com.imutil.service.SyncService;
import lombok.extern.slf4j.Slf4j;
import org.noear.solon.annotation.Component;
import org.noear.solon.annotation.Inject;
import org.noear.solon.scheduling.annotation.Scheduled;
import java.util.List;
/**
* 数据同步补拉对账定时任务
* <p>
* 定期重跑群消息 + 单聊同步(幂等,本地已有的跳过、缺失的补入),
* 兜底腾讯侧有而本系统暂无的消息,确保本系统数据最全、业务系统不漏收。
* 对应 app.yml 的 solon.scheduling.job.syncCheckJob。
* <p>
* 遍历所有租户,对每个用主应用密钥执行 checkAndPull(主应用数据为主)。
*
* @author imutil
*/
@Slf4j
@Component
public class SyncCheckTask {
@Inject
private SyncService syncService;
@Inject
private TenantMapper tenantMapper;
/**
* 由 app.yml syncCheckJob 驱动(默认 fixedDelay=1 小时)
*/
@Scheduled(name = "syncCheckJob")
public void run() {
List<Tenant> tenants = tenantMapper.selectList(null);
if (tenants == null || tenants.isEmpty()) {
return;
}
int ok = 0;
for (Tenant t : tenants) {
String tid = t.getTenantId();
if (tid == null || tid.isEmpty()) {
continue;
}
try {
syncService.checkAndPull(tid);
ok++;
} catch (Throwable e) {
// 单租户异常不影响其他租户
log.error("补拉对账定时任务异常 tenant={}", tid, e);
}
}
log.info("补拉对账定时任务完成 租户数={} 成功={}", tenants.size(), ok);
}
}
@@ -256,7 +256,7 @@ public class TencentImClient {
/** /**
* 查询单聊历史漫游消息(C2C) * 查询单聊历史漫游消息(C2C)
* <p> * <p>
* 命令字 openim_admin/get_roam_msg,按时间窗返回指定两人之间最近的 MaxCnt 条消息。 * 命令字 openim/admin_getroammsg,按时间窗返回指定两人之间最近的 MaxCnt 条消息。
* 补拉服务据此增量补全回调丢失的单聊消息。 * 补拉服务据此增量补全回调丢失的单聊消息。
* *
* @param fromAccount 发送方 IM 账号 * @param fromAccount 发送方 IM 账号
@@ -268,12 +268,13 @@ public class TencentImClient {
*/ */
public String getRoamMsg(String fromAccount, String toAccount, int maxCnt, long minTime, long maxInterval) { public String getRoamMsg(String fromAccount, String toAccount, int maxCnt, long minTime, long maxInterval) {
Map<String, Object> body = new HashMap<>(); Map<String, Object> body = new HashMap<>();
body.put("From_Account", fromAccount); // 腾讯 get_roam_msg 请求字段为 Operator_Account/Peer_Account + MinTime/MaxTime(无 MaxTimeInterval
body.put("To_Account", toAccount); body.put("Operator_Account", fromAccount);
body.put("Peer_Account", toAccount);
body.put("MaxCnt", maxCnt); body.put("MaxCnt", maxCnt);
body.put("MinTime", minTime); body.put("MinTime", minTime);
body.put("MaxTimeInterval", maxInterval); body.put("MaxTime", minTime + maxInterval);
return callApi("openim_admin/get_roam_msg", Jsons.stringify(body)); return callApi("openim/admin_getroammsg", Jsons.stringify(body));
} }
/** /**
@@ -301,12 +302,12 @@ public class TencentImClient {
public String getRoamMsgAs(String fromAccount, String toAccount, int maxCnt, long minTime, long maxInterval, public String getRoamMsgAs(String fromAccount, String toAccount, int maxCnt, long minTime, long maxInterval,
long srcSdkAppId, String srcSecretKey) { long srcSdkAppId, String srcSecretKey) {
Map<String, Object> body = new HashMap<>(); Map<String, Object> body = new HashMap<>();
body.put("From_Account", fromAccount); body.put("Operator_Account", fromAccount);
body.put("To_Account", toAccount); body.put("Peer_Account", toAccount);
body.put("MaxCnt", maxCnt); body.put("MaxCnt", maxCnt);
body.put("MinTime", minTime); body.put("MinTime", minTime);
body.put("MaxTimeInterval", maxInterval); body.put("MaxTime", minTime + maxInterval);
return callApiAs("openim_admin/get_roam_msg", Jsons.stringify(body), srcSdkAppId, srcSecretKey); return callApiAs("openim/admin_getroammsg", Jsons.stringify(body), srcSdkAppId, srcSecretKey);
} }
/** /**
@@ -325,6 +326,58 @@ public class TencentImClient {
return callApiAs("group_open_http_svc/group_msg_get_simple", Jsons.stringify(body), srcSdkAppId, srcSecretKey); return callApiAs("group_open_http_svc/group_msg_get_simple", Jsons.stringify(body), srcSdkAppId, srcSecretKey);
} }
/**
* 拉取用户最近会话列表(含 C2C 单聊会话)— 单聊同步用
* <p>
* 命令字 recentcontact/get_list,分页返回 From_Account 的会话 SessionItem
* Type=1 为 C2C(带 To_Account),Type=2 为群(带 GroupId)。
* 单聊同步据此发现用户的 C2C 会话对,再配 getRoamMsgRangeAs 全量拉取漫游消息。
*
* @param fromAccount 本端账号
* @param timeStamp 分页时间戳(首轮 0;后续取上轮返回的 TimeStamp
* @param startIndex 分页起始索引(首轮 0;后续取上轮返回的 StartIndex
* @return 腾讯响应原始 JSON(含 SessionItem[]/CompleteFlag/TimeStamp/StartIndex
*/
public String getRecentContactListAs(String fromAccount, long timeStamp, long startIndex,
long srcSdkAppId, String srcSecretKey) {
Map<String, Object> body = new HashMap<>();
body.put("From_Account", fromAccount);
body.put("TimeStamp", timeStamp);
body.put("StartIndex", startIndex);
body.put("TopTimeStamp", 0);
body.put("AssistFlags", 15);
return callApiAs("recentcontact/get_list", Jsons.stringify(body), srcSdkAppId, srcSecretKey);
}
/**
* 查询单聊漫游消息(按时间区间,全量续拉用)— 单聊同步用
* <p>
* 腾讯 openim/admin_getroammsg 请求字段为 Operator_Account/Peer_Account + MinTime/MaxTime
* 续拉:MaxTime 取上轮返回 LastMsgTime,并带 LastMsgKey,直到 Complete=1。
*
* @param operatorAccount 会话一方(本端)
* @param peerAccount 会话另一方
* @param maxCnt 单次条数上限
* @param minTime 区间下界(秒级 epoch)
* @param maxTime 区间上界(秒级 epoch;首轮 now,续拉取上轮 LastMsgTime
* @param lastMsgKey 上轮返回 LastMsgKey(首轮 null
* @return 腾讯响应原始 JSON(含 MsgList/Complete/LastMsgTime/LastMsgKey
*/
public String getRoamMsgRangeAs(String operatorAccount, String peerAccount, int maxCnt,
long minTime, long maxTime, String lastMsgKey,
long srcSdkAppId, String srcSecretKey) {
Map<String, Object> body = new HashMap<>();
body.put("Operator_Account", operatorAccount);
body.put("Peer_Account", peerAccount);
body.put("MaxCnt", maxCnt);
body.put("MinTime", minTime);
body.put("MaxTime", maxTime);
if (lastMsgKey != null && !lastMsgKey.isEmpty()) {
body.put("LastMsgKey", lastMsgKey);
}
return callApiAs("openim/admin_getroammsg", Jsons.stringify(body), srcSdkAppId, srcSecretKey);
}
// ==================== 群迁移相关(T14 S3 ==================== // ==================== 群迁移相关(T14 S3 ====================
/** /**
@@ -341,6 +394,36 @@ public class TencentImClient {
return callApiAs("group_open_http_svc/get_appid_groups", Jsons.stringify(body), srcSdkAppId, srcSecretKey); return callApiAs("group_open_http_svc/get_appid_groups", Jsons.stringify(body), srcSdkAppId, srcSecretKey);
} }
/**
* 全量分页列出 SDKAppId 下所有群(T17 数据同步,主应用密钥)
* <p>
* 命令字 group_open_http_svc/get_appid_group_list,支持 Next 游标分页:
* 首次 Next 传空串,循环把上次返回的 Next 再传入,直到返回 Next 为空。
* 与 {@link #getAppidGroups} 的区别:标准接口、支持分页、GroupIdList 元素为纯字符串。
*
* @param limit 每页数量(建议 100
* @param next 分页游标,首次传空串
* @return 腾讯响应原始 JSON(含 Next/Total/GroupIdList),由调用方解析
*/
public String getAppidGroupList(int limit, String next) {
Map<String, Object> body = new HashMap<>();
body.put("Limit", limit);
body.put("Next", next == null ? "" : next);
return callApi("group_open_http_svc/get_appid_group_list", Jsons.stringify(body));
}
/**
* 全量分页列出 SDKAppId 下所有群(T17 数据同步,指定应用密钥,用于老应用)
*
* @see #getAppidGroupList(int, String)
*/
public String getAppidGroupList(int limit, String next, long sdkAppId, String secretKey) {
Map<String, Object> body = new HashMap<>();
body.put("Limit", limit);
body.put("Next", next == null ? "" : next);
return callApiAs("group_open_http_svc/get_appid_group_list", Jsons.stringify(body), sdkAppId, secretKey);
}
/** /**
* 查群资料 + 成员列表(老应用拉群详情,用老密钥) * 查群资料 + 成员列表(老应用拉群详情,用老密钥)
* <p> * <p>
@@ -349,8 +432,9 @@ public class TencentImClient {
* @return 腾讯响应原始 JSON,由调用方解析 * @return 腾讯响应原始 JSON,由调用方解析
*/ */
public String getGroupInfo(String groupId, long srcSdkAppId, String srcSecretKey) { public String getGroupInfo(String groupId, long srcSdkAppId, String srcSecretKey) {
// 腾讯 get_group_info 要求 GroupIdList: ["groupId"](字符串数组,元素须为 string)
Map<String, Object> body = new HashMap<>(); Map<String, Object> body = new HashMap<>();
body.put("GroupId", List.of(groupId)); body.put("GroupIdList", List.of(groupId));
return callApiAs("group_open_http_svc/get_group_info", Jsons.stringify(body), srcSdkAppId, srcSecretKey); return callApiAs("group_open_http_svc/get_group_info", Jsons.stringify(body), srcSdkAppId, srcSecretKey);
} }
+15
View File
@@ -19,6 +19,7 @@ CREATE TABLE IF NOT EXISTS tenant (
CONSTRAINT pk_tenant PRIMARY KEY (tenant_id) CONSTRAINT pk_tenant PRIMARY KEY (tenant_id)
); );
CREATE UNIQUE INDEX IF NOT EXISTS uk_tenant_app_key ON tenant(app_key); CREATE UNIQUE INDEX IF NOT EXISTS uk_tenant_app_key ON tenant(app_key);
CREATE UNIQUE INDEX IF NOT EXISTS uk_tenant_prefix ON tenant(prefix_code);
-- ===== 2. 用户映射(业务用户 ↔ IM 用户) ===== -- ===== 2. 用户映射(业务用户 ↔ IM 用户) =====
CREATE TABLE IF NOT EXISTS user_mapping ( CREATE TABLE IF NOT EXISTS user_mapping (
@@ -29,6 +30,8 @@ CREATE TABLE IF NOT EXISTS user_mapping (
is_default boolean NOT NULL DEFAULT false, is_default boolean NOT NULL DEFAULT false,
is_global boolean NOT NULL DEFAULT false, is_global boolean NOT NULL DEFAULT false,
status smallint NOT NULL DEFAULT 1, -- 1=正常 0=封禁 status smallint NOT NULL DEFAULT 1, -- 1=正常 0=封禁
nick varchar(128), -- 用户昵称(T17 数据同步拉取自群成员资料)
last_synced_at timestamptz, -- 最近一次同步时间(T17
created_at timestamptz NOT NULL DEFAULT now(), created_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT pk_user_mapping PRIMARY KEY (id) CONSTRAINT pk_user_mapping PRIMARY KEY (id)
); );
@@ -42,6 +45,10 @@ CREATE TABLE IF NOT EXISTS group_mapping (
biz_group_id varchar(128) NOT NULL, biz_group_id varchar(128) NOT NULL,
im_group_id varchar(128) NOT NULL, im_group_id varchar(128) NOT NULL,
group_type varchar(16) NOT NULL DEFAULT 'Public', group_type varchar(16) NOT NULL DEFAULT 'Public',
name varchar(256), -- 群名称(T17 数据同步拉取)
owner_account varchar(128), -- 群主账号(T17 数据同步拉取)
member_count int, -- 群成员数(T17 数据同步拉取)
last_synced_at timestamptz, -- 最近一次同步时间(T17
created_at timestamptz NOT NULL DEFAULT now(), created_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT pk_group_mapping PRIMARY KEY (id) CONSTRAINT pk_group_mapping PRIMARY KEY (id)
); );
@@ -243,3 +250,11 @@ CREATE TABLE IF NOT EXISTS migrate_task (
CONSTRAINT pk_migrate_task PRIMARY KEY (id) CONSTRAINT pk_migrate_task PRIMARY KEY (id)
); );
CREATE INDEX IF NOT EXISTS idx_migrate_task_app ON migrate_task(source_app_id, object_type, status); CREATE INDEX IF NOT EXISTS idx_migrate_task_app ON migrate_task(source_app_id, object_type, status);
-- ===== T17 数据同步:老库升级补丁(ADD COLUMN IF NOT EXISTS,幂等可重跑) =====
ALTER TABLE group_mapping ADD COLUMN IF NOT EXISTS name varchar(256);
ALTER TABLE group_mapping ADD COLUMN IF NOT EXISTS owner_account varchar(128);
ALTER TABLE group_mapping ADD COLUMN IF NOT EXISTS member_count int;
ALTER TABLE group_mapping ADD COLUMN IF NOT EXISTS last_synced_at timestamptz;
ALTER TABLE user_mapping ADD COLUMN IF NOT EXISTS nick varchar(128);
ALTER TABLE user_mapping ADD COLUMN IF NOT EXISTS last_synced_at timestamptz;
+2 -2
View File
@@ -1,10 +1,10 @@
<?xml version="1.0" encoding="UTF-8"?> <?xml version="1.0" encoding="UTF-8"?>
<configuration scan="true" scanPeriod="60 seconds"> <configuration scan="true" scanPeriod="60 seconds">
<!-- 控制台输出(GBK,适配 Windows 终端 --> <!-- 控制台输出(UTF-8,适配现代终端;老 cmd.exe 需先 chcp 65001 -->
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender"> <appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder> <encoder>
<charset>GBK</charset> <charset>UTF-8</charset>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} %highlight(%-5level) %magenta(${PID:-}) --- %-15([%15.15thread]) %-56(%cyan(%-40.40logger{39})) : %msg%n</pattern> <pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} %highlight(%-5level) %magenta(${PID:-}) --- %-15([%15.15thread]) %-56(%cyan(%-40.40logger{39})) : %msg%n</pattern>
</encoder> </encoder>
</appender> </appender>
+2
View File
@@ -66,8 +66,10 @@
<a href="${basePath}/admin/grant" class="nav-item <#if active=='grant'>active</#if>">跨租户授权</a> <a href="${basePath}/admin/grant" class="nav-item <#if active=='grant'>active</#if>">跨租户授权</a>
<a href="${basePath}/admin/queue" class="nav-item <#if active=='queue'>active</#if>">队列监控</a> <a href="${basePath}/admin/queue" class="nav-item <#if active=='queue'>active</#if>">队列监控</a>
<a href="${basePath}/admin/usage" class="nav-item <#if active=='usage'>active</#if>">用量报表</a> <a href="${basePath}/admin/usage" class="nav-item <#if active=='usage'>active</#if>">用量报表</a>
<a href="${basePath}/admin/messages" class="nav-item <#if active=='messages'>active</#if>">消息记录</a>
<a href="${basePath}/admin/sourceapp" class="nav-item <#if active=='sourceapp'>active</#if>">老应用配置</a> <a href="${basePath}/admin/sourceapp" class="nav-item <#if active=='sourceapp'>active</#if>">老应用配置</a>
<a href="${basePath}/admin/migrate" class="nav-item <#if active=='migrate'>active</#if>">数据迁移</a> <a href="${basePath}/admin/migrate" class="nav-item <#if active=='migrate'>active</#if>">数据迁移</a>
<a href="${basePath}/admin/sync" class="nav-item <#if active=='sync'>active</#if>">数据同步</a>
</nav> </nav>
<div class="sidebar-footer"> <div class="sidebar-footer">
<a href="${basePath}/admin/password" class="nav-item" style="border-left:none">修改密码</a> <a href="${basePath}/admin/password" class="nav-item" style="border-left:none">修改密码</a>
+107
View File
@@ -0,0 +1,107 @@
<#--
消息记录列表页
数据源 im_message(按月分区),默认最近 7 天 + 已分发(dist_status=1)。
带时间窗走分区裁剪;时间已由 Controller 预格式化(msgTimeStr/msgTimeIso)。
-->
<#import "_macros.ftl" as m>
<@m.layout active="messages" title="消息记录">
<form method="get" action="${basePath}/admin/messages" style="background:#fff;padding:16px;border-radius:8px;margin-bottom:16px;display:flex;flex-wrap:wrap;gap:12px;align-items:flex-end">
<div style="margin:0">
<label style="display:block;font-size:12px;color:#666;margin-bottom:6px">租户</label>
<select name="tenantId" style="padding:7px;border:1px solid #ddd;border-radius:6px">
<option value="">全部租户</option>
<#list tenants as t>
<option value="${t.tenantId!}" <#if tenantId?? && tenantId==t.tenantId?string>selected</#if>>${t.tenantId!}${t.tenantName!}</option>
</#list>
</select>
</div>
<div style="margin:0">
<label style="display:block;font-size:12px;color:#666;margin-bottom:6px">会话类型</label>
<select name="convType" style="padding:7px;border:1px solid #ddd;border-radius:6px">
<option value="">全部</option>
<option value="1" <#if convType?? && convType==1>selected</#if>>C2C</option>
<option value="2" <#if convType?? && convType==2>selected</#if>>群</option>
</select>
</div>
<div style="margin:0">
<label style="display:block;font-size:12px;color:#666;margin-bottom:6px">分发状态</label>
<select name="distStatus" style="padding:7px;border:1px solid #ddd;border-radius:6px">
<option value="">全部</option>
<option value="1" <#if distStatus?? && distStatus==1>selected</#if>>已分发</option>
<option value="0" <#if distStatus?? && distStatus==0>selected</#if>>待分发</option>
<option value="2" <#if distStatus?? && distStatus==2>selected</#if>>失败</option>
</select>
</div>
<div style="margin:0">
<label style="display:block;font-size:12px;color:#666;margin-bottom:6px">起始日期</label>
<input type="date" name="dateFrom" value="${dateFrom!}" style="padding:7px;border:1px solid #ddd;border-radius:6px">
</div>
<div style="margin:0">
<label style="display:block;font-size:12px;color:#666;margin-bottom:6px">结束日期</label>
<input type="date" name="dateTo" value="${dateTo!}" style="padding:7px;border:1px solid #ddd;border-radius:6px">
</div>
<div style="margin:0">
<label style="display:block;font-size:12px;color:#666;margin-bottom:6px">关键字(账号/会话)</label>
<input type="text" name="keyword" value="${keyword!}" placeholder="from/to/conv_id" style="padding:7px;border:1px solid #ddd;border-radius:6px;width:180px">
</div>
<button type="submit" class="btn btn-primary">查询</button>
<a href="${basePath}/admin/messages" class="btn btn-secondary">重置</a>
</form>
<p class="muted" style="margin-bottom:10px">共 ${total!0} 条,第 ${page!1}/${totalPages!0} 页(默认最近 7 天 + 已分发)</p>
<#if rows?has_content>
<table class="data-table">
<thead>
<tr>
<th>时间</th><th>租户</th><th>类型</th><th>会话</th><th>发送方</th><th>接收方/群</th><th>消息类型</th><th>分发状态</th><th>来源</th><th>跨租户</th><th>操作</th>
</tr>
</thead>
<tbody>
<#list rows as r>
<tr>
<td>${r.msgTimeStr!}</td>
<td>${r.tenantId!}</td>
<td><#if r.convType?? && r.convType==1>C2C<#elseif r.convType?? && r.convType==2>群<#else>-</#if></td>
<td class="muted">${r.convId!}</td>
<td>${r.fromAccount!}</td>
<td><#if r.convType?? && r.convType==2>${r.groupId!}<#else>${r.toAccount!}</#if></td>
<td>${r.msgType!}</td>
<td>
<#if r.distStatus?? && r.distStatus==1><span class="badge badge-ok">已分发</span>
<#elseif r.distStatus?? && r.distStatus==0><span class="badge badge-warn">待分发</span>
<#elseif r.distStatus?? && r.distStatus==2><span class="badge badge-no">失败</span>
<#else>-</#if>
</td>
<td>${r.source!}</td>
<td><#if r.isCrossTenant?? && r.isCrossTenant>是<#else>-</#if></td>
<td>
<a class="btn btn-sm btn-primary" href="${basePath}/admin/messages/detail?msgKey=${r.msgKeyEnc!}&msgTime=${r.msgTimeIso!}">详情</a>
</td>
</tr>
</#list>
</tbody>
</table>
<form id="pageForm" method="get" action="${basePath}/admin/messages" style="display:inline">
<input type="hidden" name="tenantId" value="${tenantId!}">
<input type="hidden" name="convType" value="${convType!}">
<input type="hidden" name="distStatus" value="${distStatus!}">
<input type="hidden" name="dateFrom" value="${dateFrom!}">
<input type="hidden" name="dateTo" value="${dateTo!}">
<input type="hidden" name="keyword" value="${keyword!}">
<input type="hidden" name="page" id="pageInput" value="${page!1}">
</form>
<div style="margin-top:16px;display:flex;gap:8px;align-items:center">
<#if (page!1) gt 1>
<button type="button" class="btn btn-secondary btn-sm" onclick="document.getElementById('pageInput').value='${page-1}';document.getElementById('pageForm').submit()">上一页</button>
</#if>
<span class="muted">第 ${page!1} 页</span>
<#if (page!1) lt (totalPages!0)>
<button type="button" class="btn btn-secondary btn-sm" onclick="document.getElementById('pageInput').value='${page+1}';document.getElementById('pageForm').submit()">下一页</button>
</#if>
</div>
<#else>
<div class="empty">无符合条件的消息</div>
</#if>
</@m.layout>
@@ -0,0 +1,77 @@
<#--
消息详情页
基本信息(im_message 完整字段,时间由 Controller 预格式化)+ msg_body 原始 JSON + 分发链路(dist_queue
-->
<#import "_macros.ftl" as m>
<@m.layout active="messages" title="消息详情">
<div style="margin-bottom:16px">
<a href="${basePath}/admin/messages" class="btn btn-secondary btn-sm">← 返回列表</a>
</div>
<#if msg??>
<div style="background:#fff;border-radius:8px;padding:20px;box-shadow:0 1px 6px rgba(0,0,0,.07);margin-bottom:16px">
<h3 style="font-size:15px;margin-bottom:14px">基本信息</h3>
<table class="data-table" style="margin-bottom:0">
<tbody>
<tr><th style="width:140px">MsgKey</th><td>${msg.msgKey!}</td></tr>
<tr><th>消息时间</th><td>${msgTimeStr!}</td></tr>
<tr><th>租户</th><td>${msg.tenantId!}</td></tr>
<tr><th>会话类型</th><td><#if msg.convType?? && msg.convType==1>C2C<#elseif msg.convType?? && msg.convType==2>群<#else>-</#if></td></tr>
<tr><th>会话ID</th><td>${msg.convId!}</td></tr>
<tr><th>发送方</th><td>${msg.fromAccount!}</td></tr>
<tr><th>接收方</th><td>${msg.toAccount!}</td></tr>
<tr><th>群ID</th><td>${msg.groupId!}</td></tr>
<tr><th>消息类型</th><td>${msg.msgType!}</td></tr>
<tr><th>来源</th><td>${msg.source!}</td></tr>
<tr><th>跨租户</th><td><#if msg.isCrossTenant?? && msg.isCrossTenant>是<#else>否</#if></td></tr>
<tr><th>分发状态</th><td>
<#if msg.distStatus?? && msg.distStatus==1><span class="badge badge-ok">已分发</span>
<#elseif msg.distStatus?? && msg.distStatus==0><span class="badge badge-warn">待分发</span>
<#elseif msg.distStatus?? && msg.distStatus==2><span class="badge badge-no">失败</span>
<#else>-</#if>
</td></tr>
<tr><th>入库时间</th><td>${msgCreatedStr!}</td></tr>
</tbody>
</table>
</div>
<div style="background:#fff;border-radius:8px;padding:20px;box-shadow:0 1px 6px rgba(0,0,0,.07);margin-bottom:16px">
<h3 style="font-size:15px;margin-bottom:14px">消息体(msg_body</h3>
<pre style="background:#f6f8fa;padding:14px;border-radius:6px;overflow:auto;font-size:13px;max-height:400px;white-space:pre-wrap;word-break:break-all">${msg.msgBody!}</pre>
</div>
<div style="background:#fff;border-radius:8px;padding:20px;box-shadow:0 1px 6px rgba(0,0,0,.07)">
<h3 style="font-size:15px;margin-bottom:14px">分发链路(dist_queue,最多 100 条)</h3>
<#if dists?has_content>
<table class="data-table">
<thead>
<tr><th>ID</th><th>目标地址</th><th>租户</th><th>状态</th><th>重试</th><th>创建时间</th><th>更新时间</th></tr>
</thead>
<tbody>
<#list dists as d>
<tr>
<td>${d.id!}</td>
<td class="muted" style="max-width:320px;word-break:break-all">${d.targetUrl!}</td>
<td>${d.tenantId!}</td>
<td>
<#if d.status?? && d.status==0><span class="badge badge-warn">待分发</span>
<#elseif d.status?? && d.status==1><span class="badge badge-warn">处理中</span>
<#elseif d.status?? && d.status==2><span class="badge badge-ok">已分发</span>
<#elseif d.status?? && d.status==3><span class="badge badge-no">死信</span>
<#else>-</#if>
</td>
<td>${d.retryCount!}</td>
<td>${d.createdStr!}</td>
<td>${d.updatedStr!}</td>
</tr>
</#list>
</tbody>
</table>
<#else>
<div class="empty">无分发记录</div>
</#if>
</div>
<#else>
<div class="empty">消息不存在或参数缺失</div>
</#if>
</@m.layout>
+1 -1
View File
@@ -15,7 +15,7 @@
<td>${q.tenantId!}</td> <td>${q.tenantId!}</td>
<td>${q.convId!}</td> <td>${q.convId!}</td>
<td>${q.retryCount!0}</td> <td>${q.retryCount!0}</td>
<td><#if q.nextRetryAt??>${q.nextRetryAt?string('yyyy-MM-dd HH:mm:ss')}<#else>-</#if></td> <td>${(q.nextRetryAtStr)!}</td>
</tr> </tr>
</#list> </#list>
</tbody> </tbody>
+59 -2
View File
@@ -41,7 +41,16 @@
<span class="badge badge-no">停用</span> <span class="badge badge-no">停用</span>
</#if> </#if>
</td> </td>
<td> <td style="white-space:nowrap">
<button class="btn btn-sm btn-primary"
data-id="${a.id!}"
data-app-name="${(a.appName!'')?html}"
data-sdk-app-id="${(a.sdkAppId!'')?html}"
data-secret-key="${(a.secretKey!'')?html}"
data-tenant-id="${(a.tenantId!'')?html}"
data-remark="${(a.remark!'')?html}"
data-status="${(a.status!'')?html}"
onclick="openEditApp(this)">编辑</button>
<form method="post" action="${basePath}/admin/sourceapp/toggle" style="display:inline"> <form method="post" action="${basePath}/admin/sourceapp/toggle" style="display:inline">
<input type="hidden" name="id" value="${a.id!}"> <input type="hidden" name="id" value="${a.id!}">
<#if a.status?? && a.status == 1> <#if a.status?? && a.status == 1>
@@ -78,7 +87,7 @@
<select name="tenantId" required> <select name="tenantId" required>
<option value="">请选择租户</option> <option value="">请选择租户</option>
<#list tenants as t> <#list tenants as t>
<option value="${t.tenantId!}">${t.tenantId!}${t.tenantName!},前缀 ${t.prefixCode!}</option> <option value="${t.tenantId!}">${t.prefixCode!}${t.tenantName!}</option>
</#list> </#list>
</select> </select>
</div> </div>
@@ -91,4 +100,52 @@
</form> </form>
</div> </div>
</div> </div>
<#-- 编辑弹窗:主键 id 自增 hidden 携带;secretKey 回填明文(与租户 appSecret 一致) -->
<div id="editAppModal" class="modal" onclick="if(event.target===this)this.classList.remove('show')">
<div class="modal-content">
<div class="modal-title">编辑老应用</div>
<form method="post" action="${basePath}/admin/sourceapp/save">
<div class="modal-body">
<input type="hidden" name="id" id="eaId">
<div class="form-group"><label>应用名称</label><input name="appName" id="eaAppName" required></div>
<div class="form-group"><label>SDKAppID(老应用)</label><input name="sdkAppId" id="eaSdkAppId" required></div>
<div class="form-group"><label>SecretKey(老应用密钥)</label><input name="secretKey" id="eaSecretKey" required></div>
<div class="form-group">
<label>对应租户(决定 UserID 前缀)</label>
<select name="tenantId" id="eaTenantId" required>
<option value="">请选择租户</option>
<#list tenants as t>
<option value="${t.tenantId!}">${t.prefixCode!}${t.tenantName!}</option>
</#list>
</select>
</div>
<div class="form-group"><label>备注</label><input name="remark" id="eaRemark"></div>
<div class="form-group"><label>状态</label>
<select name="status" id="eaStatus">
<option value="1">启用</option>
<option value="0">停用</option>
</select>
</div>
</div>
<div class="modal-footer">
<button type="button" class="btn btn-secondary" onclick="document.getElementById('editAppModal').classList.remove('show')">取消</button>
<button type="submit" class="btn btn-primary">保存</button>
</div>
</form>
</div>
</div>
<script>
function openEditApp(b){
var d=b.dataset;
document.getElementById('eaId').value=d.id||'';
document.getElementById('eaAppName').value=d.appName||'';
document.getElementById('eaSdkAppId').value=d.sdkAppId||'';
document.getElementById('eaSecretKey').value=d.secretKey||'';
document.getElementById('eaTenantId').value=d.tenantId||'';
document.getElementById('eaRemark').value=d.remark||'';
document.getElementById('eaStatus').value=d.status||1;
document.getElementById('editAppModal').classList.add('show');
}
</script>
</@m.layout> </@m.layout>
+127
View File
@@ -0,0 +1,127 @@
<#import "_macros.ftl" as m>
<@m.layout active="sync" title="数据同步">
<#if msg?has_content>
<div class="alert">
<#if msg=='sgdone'>✅ 群组同步完成(详见任务记录)
<#elseif msg=='sudone'>✅ 群成员/用户同步完成(详见任务记录)
<#elseif msg=='smdone'>✅ 群消息同步完成(详见任务记录)
<#elseif msg=='scdone'>✅ 单聊消息同步完成(详见任务记录)
<#elseif msg=='chkdone'>✅ 补拉对账完成(详见任务记录)
<#elseif msg=='syncopt'>❌ 请选择数据源(主应用时须同时选归属租户)
<#else>${msg}
</#if>
</div>
</#if>
<div class="stat-card" style="margin-bottom:14px;padding:12px 18px">
<b>说明</b>
<span class="muted">:本页为<b>只读拉取</b>,把腾讯 IM 数据同步到本地表用于查看,不在腾讯侧建群/建号。
<b>群组</b>=get_appid_group_list 全量分页;<b>群成员(用户)</b>=遍历群取成员(腾讯无全量用户接口,靠群成员反推);<b>群消息</b>=逐群全量拉取;<b>单聊</b>=recentcontact/get_list 按已同步用户反查会话 + get_roam_msg 全量翻页。
<b>所有同步消息均入分发队列推送业务系统</b>(限速分批入队),本地同时留底。</span>
</div>
<#-- 公共:数据源 + 归属租户选择(三个表单各自携带) -->
<#macro srcSelect>
<div class="form-group">
<label>数据源</label>
<select name="sourceAppId">
<option value="">主应用(app.yml 当前配置)</option>
<#list apps as a>
<option value="${a.id!}">${a.appName!}SDK ${a.sdkAppId!} → 租户 ${a.tenantId!}</option>
</#list>
</select>
</div>
<div class="form-group">
<label>归属租户 <span class="muted">(选「主应用」时必选;选老应用时自动取其配置租户,此项忽略)</span></label>
<select name="tenantId">
<option value="">请选择租户</option>
<#list tenants as t>
<option value="${t.tenantId!}">${t.tenantName!}${t.tenantId!}</option>
</#list>
</select>
</div>
</#macro>
<div class="stat-card" style="margin-bottom:20px">
<div class="modal-title">① 同步群组</div>
<p class="muted">拉取数据源应用的所有群(get_appid_group_list 全量分页)+ 逐群 get_group_info(名/类型/群主/成员数)→ 写 group_mapping。幂等可重跑。</p>
<form method="post" action="${basePath}/admin/sync/groups" style="margin-top:14px">
<div class="modal-body"><@srcSelect/></div>
<button type="submit" class="btn btn-primary">同步群组</button>
</form>
</div>
<div class="stat-card" style="margin-bottom:20px">
<div class="modal-title">② 同步群成员(→ 用户)</div>
<p class="muted">遍历该租户已同步的群,逐群取成员列表 → 写 user_mapping。<b>请先执行「同步群组」</b>,否则无群可遍历。</p>
<form method="post" action="${basePath}/admin/sync/users" style="margin-top:14px">
<div class="modal-body"><@srcSelect/></div>
<button type="submit" class="btn btn-primary">同步群成员</button>
</form>
</div>
<div class="stat-card" style="margin-bottom:20px">
<div class="modal-title">③ 同步群消息</div>
<p class="muted">遍历该租户已同步的群,逐群 getGroupMsg + IsFinished 滚动全量 → 写 im_messagesource=SYNC<b>进分发队列</b>,限速分批)。<b>请先执行「同步群组」</b>。</p>
<form method="post" action="${basePath}/admin/sync/groupmsg" style="margin-top:14px">
<div class="modal-body"><@srcSelect/></div>
<button type="submit" class="btn btn-primary">同步群消息</button>
</form>
</div>
<div class="stat-card" style="margin-bottom:20px">
<div class="modal-title">④ 同步单聊消息</div>
<p class="muted">遍历该租户已同步用户 → recentcontact/get_list 收集 C2C 会话 → 会话对去重 → get_roam_msg 全量翻页 → 写 im_messageconv_type=1source=SYNC,进分发)。<b>请先执行「同步群成员」</b>。</p>
<form method="post" action="${basePath}/admin/sync/c2cmsg" style="margin-top:14px">
<div class="modal-body"><@srcSelect/></div>
<button type="submit" class="btn btn-primary">同步单聊消息</button>
</form>
</div>
<div class="stat-card" style="margin-bottom:20px">
<div class="modal-title">⑤ 补拉对账</div>
<p class="muted">重跑群消息 + 单聊同步(幂等:本地已有的跳过、缺失的补入),确保本系统数据最全。仅主应用 + 指定租户,可定时自动或手动触发。</p>
<form method="post" action="${basePath}/admin/sync/check" style="margin-top:14px">
<div class="form-group">
<label>归属租户 <span class="muted">(主应用,对账该租户)</span></label>
<select name="tenantId">
<option value="">请选择租户</option>
<#list tenants as t>
<option value="${t.tenantId!}">${t.tenantName!}${t.tenantId!}</option>
</#list>
</select>
</div>
<button type="submit" class="btn btn-primary">补拉对账</button>
</form>
</div>
<h3 style="margin:20px 0 10px;font-size:15px;color:#2c3e50">同步/迁移任务记录</h3>
<#if tasks?has_content>
<table class="data-table">
<thead>
<tr><th>ID</th><th>源应用</th><th>对象</th><th>进度(已处理/总数)</th><th>状态</th><th>校验结果</th><th>开始时间</th></tr>
</thead>
<tbody>
<#list tasks as t>
<tr>
<td>${t.id!}</td>
<td><#if t.sourceAppId?? && t.sourceAppId==0>主应用<#else>${t.sourceAppId!}</#if></td>
<td>${t.objectType!}</td>
<td>${t.processedCount!'0'}/${t.totalCount!'0'}</td>
<td>
<#if t.status?? && t.status==3><span class="badge badge-ok">完成</span>
<#elseif t.status?? && t.status==1><span class="badge badge-warn">运行</span>
<#elseif t.status?? && t.status==4><span class="badge badge-no">失败</span>
<#else><span class="badge">待</span>
</#if>
</td>
<td class="muted">${t.verifyResult!'-'}</td>
<td class="muted">${t.startedAt!}</td>
</tr>
</#list>
</tbody>
</table>
<#else>
<div class="empty">暂无任务</div>
</#if>
</@m.layout>
+58 -4
View File
@@ -10,14 +10,13 @@
<table class="data-table"> <table class="data-table">
<thead> <thead>
<tr> <tr>
<th>租户ID</th><th>名称</th><th>前缀</th><th>AppKey</th><th>AppSecret</th> <th>名称</th><th>前缀</th><th>AppKey</th><th>AppSecret</th>
<th>回调URL</th><th>IM QPS</th><th>状态</th><th>操作</th> <th>回调URL</th><th>IM QPS</th><th>状态</th><th>操作</th>
</tr> </tr>
</thead> </thead>
<tbody> <tbody>
<#list tenants as t> <#list tenants as t>
<tr> <tr>
<td>${t.tenantId!}</td>
<td>${t.tenantName!}</td> <td>${t.tenantName!}</td>
<td>${t.prefixCode!}</td> <td>${t.prefixCode!}</td>
<td>${t.appKey!}</td> <td>${t.appKey!}</td>
@@ -31,7 +30,18 @@
<span class="badge badge-no">停用</span> <span class="badge badge-no">停用</span>
</#if> </#if>
</td> </td>
<td> <td style="white-space:nowrap">
<button class="btn btn-sm btn-primary"
data-tenant-id="${(t.tenantId!'')?html}"
data-tenant-name="${(t.tenantName!'')?html}"
data-prefix-code="${(t.prefixCode!'')?html}"
data-app-key="${(t.appKey!'')?html}"
data-app-secret="${(t.appSecret!'')?html}"
data-callback-url="${(t.callbackUrl!'')?html}"
data-quota-im-qps="${(t.quotaImQps!'')?html}"
data-quota-trtc-concurrent="${(t.quotaTrtcConcurrent!'')?html}"
data-status="${(t.status!'')?html}"
onclick="openEdit(this)">编辑</button>
<form method="post" action="${basePath}/admin/tenant/toggle" style="display:inline"> <form method="post" action="${basePath}/admin/tenant/toggle" style="display:inline">
<input type="hidden" name="tenantId" value="${t.tenantId!}"> <input type="hidden" name="tenantId" value="${t.tenantId!}">
<#if t.status?? && t.status == 1> <#if t.status?? && t.status == 1>
@@ -56,7 +66,7 @@
<div class="modal-title">新增租户</div> <div class="modal-title">新增租户</div>
<form method="post" action="${basePath}/admin/tenant/save"> <form method="post" action="${basePath}/admin/tenant/save">
<div class="modal-body"> <div class="modal-body">
<div class="form-group"><label>租户ID(即前缀,如 sa</label><input name="tenantId" required></div> <div class="form-group"><label>前缀码(IM 账号前缀,如 sa,全局唯一</label><input name="prefixCode" required></div>
<div class="form-group"><label>租户名称</label><input name="tenantName"></div> <div class="form-group"><label>租户名称</label><input name="tenantName"></div>
<div class="form-group"><label>AppKey(业务系统调用凭证)</label><input name="appKey" required></div> <div class="form-group"><label>AppKey(业务系统调用凭证)</label><input name="appKey" required></div>
<div class="form-group"><label>AppSecret</label><input name="appSecret" required></div> <div class="form-group"><label>AppSecret</label><input name="appSecret" required></div>
@@ -71,4 +81,48 @@
</form> </form>
</div> </div>
</div> </div>
<#-- 编辑弹窗:主键 tenantId 只读(readonly 仍随表单提交,供 updateById 定位);其余字段全部放开 -->
<div id="editModal" class="modal" onclick="if(event.target===this)this.classList.remove('show')">
<div class="modal-content">
<div class="modal-title">编辑租户</div>
<form method="post" action="${basePath}/admin/tenant/save">
<div class="modal-body">
<input type="hidden" name="tenantId" id="eTenantId">
<div class="form-group"><label>租户名称</label><input name="tenantName" id="eTenantName"></div>
<div class="form-group"><label>前缀码</label><input name="prefixCode" id="ePrefixCode"></div>
<div class="form-group"><label>AppKey</label><input name="appKey" id="eAppKey" required></div>
<div class="form-group"><label>AppSecret</label><input name="appSecret" id="eAppSecret" required></div>
<div class="form-group"><label>回调URL</label><input name="callbackUrl" id="eCallbackUrl"></div>
<div class="form-group"><label>IM QPS 配额</label><input name="quotaImQps" type="number" id="eQuotaImQps"></div>
<div class="form-group"><label>TRTC 并发房间配额</label><input name="quotaTrtcConcurrent" type="number" id="eQuotaTrtc"></div>
<div class="form-group"><label>状态</label>
<select name="status" id="eStatus">
<option value="1">启用</option>
<option value="0">停用</option>
</select>
</div>
</div>
<div class="modal-footer">
<button type="button" class="btn btn-secondary" onclick="document.getElementById('editModal').classList.remove('show')">取消</button>
<button type="submit" class="btn btn-primary">保存</button>
</div>
</form>
</div>
</div>
<script>
function openEdit(b){
var d=b.dataset;
document.getElementById('eTenantId').value=d.tenantId||'';
document.getElementById('eTenantName').value=d.tenantName||'';
document.getElementById('ePrefixCode').value=d.prefixCode||'';
document.getElementById('eAppKey').value=d.appKey||'';
document.getElementById('eAppSecret').value=d.appSecret||'';
document.getElementById('eCallbackUrl').value=d.callbackUrl||'';
document.getElementById('eQuotaImQps').value=d.quotaImQps||50;
document.getElementById('eQuotaTrtc').value=d.quotaTrtcConcurrent||0;
document.getElementById('eStatus').value=d.status||1;
document.getElementById('editModal').classList.add('show');
}
</script>
</@m.layout> </@m.layout>
+1 -1
View File
@@ -9,7 +9,7 @@
<#list stats as s> <#list stats as s>
<tr> <tr>
<td>${s.tenantId!}</td> <td>${s.tenantId!}</td>
<td><#if s.statTime??>${s.statTime?string('yyyy-MM-dd HH:mm')}<#else>-</#if></td> <td>${(s.statTimeStr)!}</td>
<td><#if s.statLevel?? && s.statLevel == 1>小时<#else>天</#if></td> <td><#if s.statLevel?? && s.statLevel == 1>小时<#else>天</#if></td>
<td>${s.imMsgCount!0}</td> <td>${s.imMsgCount!0}</td>
<td>${s.imDau!0}</td> <td>${s.imDau!0}</td>