总览 · ← 返回运行 20261002-202908-search-t1-scr-B

节点 n4

local_ot: top-k 祖先截断 + 逐细胞位移收缩（λ_base=0.75）

方法说明?节点程序自带的 METHOD.md：这个程序做了什么、为什么。

来自 programs.git 51f90ccd33:solution/METHOD.md

调研员的计划

代码改动?这个节点的程序和父节点程序的逐行差别：绿色是新增，红色是删除。

对比：父节点版本 8204ac3d1a。改动的文件：solution/METHOD.md +24 −69、solution/run.py +43 −9

diff --git a/solution/METHOD.md b/solution/METHOD.mdindex ca7a3d2..ea4dc09 100644--- a/solution/METHOD.md+++ b/solution/METHOD.md@@ -1,81 +1,36 @@-# ot_moscot — Waddington-OT / moscot coupling, one-step displacement extrapolation+# local_ot: top-k 祖先截断 + 逐细胞位移收缩（λ_base=0.75） -Seed (2026-10-02) made from the G37 candidate `modeling/candidates/T1/ot_moscot/` (commit 227eeb2). Same method and-hyper-parameters. Changes for the seed contract only: the dev-only environment overrides (`G37_LAMBDA`, `G37_DECODE`,-`G37_GROWTH`, `G37_JAX_GPU`) and the unused `knn` decode branch are removed; JAX forced to CPU, torch threads fixed-at 8 (`EXECUTION.json {"gpu": false}`); `stage_pair` reads every input stage of the view the same way (no reference to-manifest `mode` / `source`). Output depends only on the view's data, the time differences between stages and `--seed`.+在父节点 ot_moscot 的解码段加入 PLAN 指定的 top-k 祖先截断与逐细胞收缩；实测该耦合本就接近 1对1，截断/熵收缩近乎惰性，真实增益来自把位移基础系数 λ_base 从 1 降到 0.75。 -Contract: `python run.py --data <view> --out <pred.h5ad> --seed <int>`; `g37_common.py` must stay next to `run.py`.+## 改动（只动 run.py 解码段，嵌入/生长/抽样/耦合不变） -## Method+- 每列耦合保留 top-k（G37_K，默认 30）个祖先，重归一后算重心 anc_j；+- 逐细胞置信 conf_j = 1 − H_j/log(k)（截断列的归一化熵），λ_j = λ_base·(conf_floor + (1−conf_floor)·conf_j)，conf_floor=0.2；+- λ_base 默认改为 0.75（原 1.0）；开关：G37_K=0 或 G37_CONF_FLOOR=1 → 复现父节点；G37_LAMBDA=0 → 关闭位移（生长重采样复制）。提交默认 = 机制打开。 -Two input stages `prev` (time t0) and `last` (t1); target time t2 (final view: E8.5, E9.5 -> E10.5).+## 机制生效证据（X3 视图，seed 0，G37_DIAG） -1. **Embedding.** Genes measured in both stages; top 2000 by variance (both stages pooled); z-score, clip at 10;-   PCA, 30 components (randomized, `random_state=seed`). Fitted on the input stages only.-2. **Growth prior (Waddington-OT).** Proliferation / apoptosis scores (`scanpy.tl.score_genes`, moscot's mouse-   gene lists) -> birth = generalised logistic(prolif; 1.7, 0.3, 0.25, 0.5), death = logistic(apopt; 1.7, 0.3, 0.1,-   0.2), per-day growth g = exp(birth - death) (Schiebinger 2019; same formula and defaults as moscot's-   `BirthDeathProblem.estimate_marginals`).-3. **Coupling.** `moscot.problems.time.TemporalProblem` prev -> last on the PCA (`joint_attr="X_pca"`),-   source marginal ∝ g^(t1-t0), target uniform; entropic unbalanced Sinkhorn, `epsilon=1e-3`, `tau_a=0.95`,-   `tau_b=1`, `scale_cost="mean"` (moscot tutorial settings). JAX on CPU unless `G37_JAX_GPU=1`.-4. **Output cells.** n = number of latest-stage cells clipped to `[min_cells, max_cells]` (as copy_last),-   drawn without replacement from the latest stage with probability ∝ g^(t2-t1) (the WOT birth-death model-   continued over the target interval).-5. **Displacement extrapolation.** For output cell j: ancestor mean a_j = Σ_i π_ij x_i / Σ_i π_ij (barycentric-   projection of the coupling, gene space, all panel genes); smoothed position s_j = mean expression of its 30-   nearest latest-stage cells in the PCA; step_j = λ (t2-t1)/(t1-t0) (s_j - a_j), λ = 1 (continue the last-   observed displacement at the same rate). The step is added **to the cell's non-zero entries only** and clipped-   at 0; genes not measured in both stages (external stages) get no step. Each cell keeps its own residual.+- 耦合本身极稀疏：每列非零祖先 min/中位/max = 1/2/13，exp(熵) 中位 ≈ 1.003。因此 k=30 截断对绝大多数细胞无操作，conf_j 中位 0.999，λ_j ∈ [0.70·λ_base, λ_base]，只有少数多祖先细胞被熵收缩（λ_j<0.98 约 40%，λ_j<0.6·λ_base 为 0）。**如实报告：PLAN 设想的“弥散耦合被局部化”在 X3 上不成立，熵收缩的实际作用很小**；分数变化主要来自 λ_base 的整体收缩。+- 位移非每型常向量：step 范数中位 ≈ 99，逐细胞不同（top-k 重心 + kNN 平滑 + 各自残差），与父节点相同的结构。 -**One input stage (proxy view, E8.5 only):** steps 1, 3, 5 need two stages; only steps 2 + 4 run, i.e. a-growth-weighted copy of the latest stage (g^(t2-t1) resampling). This is the only thing the proxy can test.+## 关闭机制对照（X3 A 半，vec-score，seed 0） -Sources:-- Schiebinger G. et al. Optimal-transport analysis of single-cell gene expression identifies developmental-  trajectories in reprogramming. *Cell* 176, 928–943 (2019). doi:10.1016/j.cell.2019.01.006 (WOT: unbalanced-  entropic OT between snapshots, growth from proliferation/apoptosis signatures, birth-death logistic).-- Klein D., Palla G., Lange M. et al. Mapping cells through time and space with moscot. *Nature* 638, 1065–1075-  (2025). doi:10.1038/s41586-024-08453-2 (TemporalProblem; code moscot 0.5.2, BSD-3).-- Cuturi M. Sinkhorn distances. NeurIPS 2013; Chizat L. et al. Scaling algorithms for unbalanced optimal-  transport problems. *Math. Comp.* 87, 2563–2609 (2018).-- Extrapolating the barycentric displacement one more step is our use of the coupling (WOT/moscot interpolate,-  they do not extrapolate); listed in agent/knowledge/T1_methods_landscape.md §1.+| 配置 | board | direction | de_recovery | cell_state | covariation |+|---|---|---|---|---|---|+| A: K=全列, conf_floor=1（≈父节点） | 50.41 | 49.03 | 50.00 | 49.88 | 53.45 |+| B: λ_base=0（位移关闭） | 50.52 | **50.46** | 46.49 | 54.10 | 50.25 |+| 机制开, λ_base=1 | 50.85 | 49.15 | 50.48 | 50.71 | 53.66 |+| λ_base=0.25 | 50.86 | 50.32 | 50.48 | 52.82 | 49.06 |+| λ_base=0.5 | 51.51 | 50.33 | 50.00 | 53.49 | 51.92 |+| **λ_base=0.75（提交默认）** | **52.37** | 49.86 | 51.46 | 53.80 | 54.50 | -## Data / knowledge used+对照 B 证实 PLAN 的怀疑：λ=1 的整步外推确实伤 direction（49.1 < 50.5）。λ_base 扫描显示 0.75 在保留位移带来的 de_recovery/covariation 增益的同时把 direction 拉回，是四个 λ 中最优。注意 A/B 半与种子噪声（T1≈2 分）：0.5 与 0.75 之差 (0.86) 在噪声内，0.75 为单次测量下最优，选它因四组分均不弱于父节点。 -Only the view's input stages. Generic knowledge: moscot's built-in mouse proliferation (97) and apoptosis (193)-gene lists (`moscot.utils.data`, from the WOT paper; stage-agnostic gene-function annotation). No held-out stage,-no information from (E9.5, E13.5], no pre-trained weights.+## 验证过 / 未验证 -## Hyper-parameters+- 验证：X3（两输入）上 k/conf_floor/λ_base 的作用与两个关闭对照；输出通过 vec-check；单输入退路（生长重采样复制）未改动、与父节点相同；seed 确定性（default_rng + 固定 random_state，无全局随机）。+- 未验证：k∈{15,60,100} 与 conf_floor∈{0,0.4} 的完整扫描（因熵收缩近乎惰性，预期差异 < 噪声，未花额度）；final 视图（E8.5+E9.5→E10.5，dt_out/dt_in=1，factor 不同）上 λ_base=0.75 是否同样最优；多 seed 复测。 -| Name | Value | Where it came from |-|---|---|---|-| `N_HVG`, `N_PCS`, `K_SMOOTH` | 2000, 30, 30 | a priori; not tuned |-| `EPSILON`, `TAU_A`, `TAU_B`, `scale_cost` | 1e-3, 0.95, 1, mean | a priori (moscot tutorial settings) |-| growth prior | on | a priori (WOT / moscot defaults); G37 saw it cost ~3 points on the old proxy, kept on |-| `LAMBDA` | 1 | a priori (continue the observed displacement at the same rate); G37 also ran 0.5 on X3 (50.4 vs 50.2), not changed |-| decode `addnz` | — | **chosen on the X3 ruler (G37)** against `add` (25.6) and `knn` (48.7): the dense step destroys the zero pattern. Also a first-principles choice (the scorer compares sparse log-expression), but the evidence that picked it was X3 |+## 知识来源 -No re-tuning for the seed.--## Resources (Spark, CPU)--Final view (16.8k x 17.1k coupling): 69 s, max RSS 6.7 GB; proxy 4 s / 1.6 GB; X3 ~ 8 s / 2.3 GB; proxy2 ~ 20 s / 4.3 GB.-The dense coupling (n_prev x n_last float32, ~1.1 GB on final) and the dense earlier stage (~2.2 GB) dominate memory.--## Findings (G37, local scorer: fast engine, truth half B, scorer seed = program seed)--- Dense gene-space step (`add`) is destructive: X3 25.6 at λ=1, still 32.9 at λ=0.25 (cell_state, covariation-  collapse) — the step makes every zero slightly positive. `addnz` fixes it (X3 50.2 at λ=1, 50.4 at λ=0.5);-  `knn` decode 48.7 (λ=1). Growth resampling has no effect on X3/proxy2 (all cells are kept there).-- proxy2 (E8.5 official -> Qiu E9.0 heart, then +0.5 d): every variant ≈ copy of the Qiu cells (~27.5), the-  cross-dataset step is batch effect.-- Full eval (seeds 0-2, half B): proxy 46.8 / 46.8 / 47.9 (copy_last 50.0 / 50.3 / 50.1) — the WOT growth-  resampling alone costs ~3 points on E8.5 -> E9.5 (direction, cell_state); X3 50.2 / 50.1 / 49.8 (copy_last 50.0,-  pseudobulk_shift 40.5-40.7); proxy2 27.7-27.9 (copy_last 27.4-27.6). `G37_GROWTH=0` turns the proxy into copy_last.-- Final-view prediction (`~/vec/scratch/g37/ot_moscot/final.h5ad`, seed 0): 5118 cells, 21 nearest-E9.5 types,-  composition within ±2 % of E9.5 (OFT/RV-CM -2.1 %); no new states (the method cannot create them).+无新增外部数据或文献；WOT/moscot 引用同父节点 METHOD.md。视图无关：仅依赖输入数据、阶段时间差与 seed；开关为环境变量，提交默认为常数。diff --git a/solution/run.py b/solution/run.pyindex 1411cb7..8d0c7da 100644--- a/solution/run.py+++ b/solution/run.py@@ -15,11 +15,11 @@ Two input stages (final: E8.5, E9.5): One input stage (proxy: E8.5 only): steps 1, 2, 4 need two stages; only the growth resampling (3) runs, i.e. a growth-weighted copy of the latest stage. -Seed version (agent/seeds/T1__val/ot_moscot, 2026-10-02) of modeling/candidates/T1/ot_moscot (G37): same method and-hyper-parameters; the dev-only environment overrides (G37_LAMBDA / G37_DECODE / G37_GROWTH / G37_JAX_GPU) and the-`knn` decode branch are removed; JAX and torch on CPU, fixed thread count (EXECUTION.json gpu false).-Parameter provenance (METHOD.md): DECODE = addnz was chosen on the X3 ruler (G37); LAMBDA = 1 a priori, also checked-on X3 (0.5 vs 1 within 0.2).+Seed version (agent/seeds/T1__val/ot_moscot, 2026-10-02) of modeling/candidates/T1/ot_moscot (G37), modified by+local_ot node 4: the decode step gains top-k ancestor truncation + per-cell entropy shrinkage and lambda_base drops+1.0 -> 0.75 (see METHOD.md for controls and the X3 lambda scan); dev switches G37_K / G37_LAMBDA / G37_CONF_FLOOR /+G37_DIAG, submitted defaults = mechanism ON. JAX and torch on CPU, fixed thread count (EXECUTION.json gpu false).+Parameter provenance: DECODE = addnz chosen on the X3 ruler (G37); lambda_base=0.75 chosen on X3 A-half (node 4). """  from __future__ import annotations@@ -46,10 +46,17 @@ EPSILON = 1e-3 TAU_A = 0.95 TAU_B = 1.0 K_SMOOTH = 30-LAMBDA = 1.0     # 1 = continue the observed displacement at full rate GROWTH = True    # resample output cells by g^dt_out (WOT birth-death model) N_THREADS = 8 +# local_ot mechanism (dev switches; submitted defaults below = mechanism ON):+#   G37_K            top-k ancestor truncation of each coupling column; 0 / >= n_prev = full column (off)+#   G37_LAMBDA       base displacement factor lambda_base; 0 = displacement fully off+#   G37_CONF_FLOOR   per-cell shrinkage floor: lambda_j = lambda_base*(floor + (1-floor)*conf_j)+K_ANCESTORS = int(os.environ.get("G37_K", "30"))+LAMBDA_BASE = float(os.environ.get("G37_LAMBDA", "0.75"))+CONF_FLOOR = float(os.environ.get("G37_CONF_FLOOR", "0.2"))+  def weighted_rows(w: np.ndarray, n: int, rng: np.random.Generator) -> np.ndarray:     p = w / w.sum()@@ -103,14 +110,41 @@ def main() -> None:     Pc = P[:, rows]     del P     Pc /= np.maximum(Pc.sum(axis=0, keepdims=True), 1e-30)-    factor = LAMBDA * dt_out / dt_in++    # top-k local ancestor truncation + per-column entropy confidence+    n_prev = Pc.shape[0]+    k = K_ANCESTORS if 0 < K_ANCESTORS < n_prev else n_prev+    truncate = k < n_prev+    conf = np.ones(Pc.shape[1], dtype=np.float64)+    if truncate:+        logk = np.log(k)+        for j in range(Pc.shape[1]):+            col = Pc[:, j]+            idx = np.argpartition(col, n_prev - k)[n_prev - k:]+            w = col[idx]+            s = float(w.sum())+            col[:] = 0.0+            if s > 1e-30:+                col[idx] = w / s+                p = col[idx]+                H = float(-np.sum(p * np.log(np.maximum(p, 1e-30))))+                conf[j] = min(max(1.0 - H / logk, 0.0), 1.0)+    lam = LAMBDA_BASE * (CONF_FLOOR + (1.0 - CONF_FLOOR) * conf)  # per-cell factor+    factor = (dt_out / dt_in) * lam+    if os.environ.get("G37_DIAG"):+        np.savez(os.environ["G37_DIAG"], lam=lam.astype(np.float32), conf=conf.astype(np.float32),+                 k=k, dt_in=dt_in, dt_out=dt_out)     torch.set_num_threads(N_THREADS)     Xp = torch.from_numpy(prev.X.toarray())-    anc = (torch.from_numpy(Pc).T @ Xp).numpy()  # barycentric ancestor mean, (n, n_genes)+    anc = (torch.from_numpy(Pc).T @ Xp).numpy()  # top-k barycentric ancestor mean, (n, n_genes)     del Xp, Pc     smooth = knn_mean(Zl, last.X, rows, K_SMOOTH)-    step = (smooth - anc) * factor+    step = (smooth - anc) * factor[:, None]     del smooth, anc+    if os.environ.get("G37_DIAG"):+        np.savez(os.path.dirname(os.environ["G37_DIAG"]) + "/diag_step.npz",+                 step_norm=np.linalg.norm(step, axis=1).astype(np.float32),+                 Zl_rows=Zl[rows])     step[:, ~mask] = 0.0     X = last.X[rows].toarray()     step *= X > 0  # "addnz": move only measured (non-zero) entries, keeps each cell's zero pattern

调研来源?调研员查到并用到的知识条目和文献检索结果（只列标题和编号）。

用到的知识库条目

分析结果?分析员写的 ANALYSIS.json：改了什么、各组分数怎么变、假设是否成立、经验和下一步建议。

对话摘要?每个角色和大模型对话的统计：轮数、工具调用、用时、token 数和最后的回答摘录；原始记录只给路径。

只给统计和最后回答的摘录；完整对话请到原始记录位置里列出的文件看。

分析员

ANALYSIS.json written

工程师

Done. Summary:

**做了什么**：按 PLAN 在父节点 ot_moscot 解码段实现 top-k(k=30) 祖先截断 + 逐细胞熵收缩（conf_floor=0.2），带三个环境开关（提交默认机制打开）。实测发现 X3 上的耦合本就接近 1对1（每列非零祖先中位=2，exp(熵)≈1.003），截断/熵收缩近乎惰性——已在 METHOD.md 如实报告；真实增益来自关闭对照 B（λ=0 时 direction 50.46 > λ=1 的 49.15）揭示的"整步外推伤 direction"，据此把 λ_base 降到 0.75。

**查分结果**（X3 A 半，seed 0，用 6/20 次）：对照 A(≈父) 50.41；对照 B(λ=0) 50.52；λ=1 机制开 50.85（父 50.65）；λ=0.25→50.86；λ=0.5→51.51；**λ=0.75→52.37（提交默认）**，direction 49.86、de_recovery 51.46、cell_state 53.80、covariation 54.50，四组分均不弱于父。vec-check 通过，确定性复现验证通过。

**下一步最值得试**：(1) λ_base 在 0.6–0.9 细扫 + 多 seed 复测（0.5/0.75 差在噪声内）；(2) 逐细胞收缩换用真正有变异的置信信号（如配对位移距离 d_j=‖smoothZ−ancZ‖），熵信号在此耦合下已退化；(3) 在 final 视图（dt_out/dt_in=1）确认 λ=0.75 仍优。

调研员

节点用全局熵正则耦合（epsilon=1e-3, scale_cost=mean），每个目标细胞的祖先重心是对整列所有源细胞的加权平均，耦合弥散→位移趋近全局/每型均值场，继续整步外推（factor=1）对不可靠的弥散细胞产生噪声方向，拉低 direction。",
 "approach": "只改 run.py 中耦合→位移的解码段，保持嵌入/生长/抽样不变。(1) 局部化：对传输矩阵 P 的每一列（每个输出细胞），仅保留耦合质量最大的 top-k 个祖先，重新归一化后再算祖先重心 anc_j（替换现在的全列重心）。k 初值30，搜索{15,30,60,100}。(2) 逐细胞收缩：对每列算归一化熵 conf_j=1-H_j/log(k)∈[0,1]，位移因子 λ_j=λ_base·(conf_floor+(1-conf_floor)·conf_j)，λ_base=1.0、conf_floor 初值0.2、搜索{0.0,0.2,0.4}。弥散/不确定的细胞自动小步，聚焦的细胞大步，直接减少伤方向的噪声位移。(3) 保留 addnz 解码与 clip≥0，保留每细胞残差。参数用 X3 的 A 半 vec-score 快速筛选：先按阶段各抽 2000–4000 细胞拟合耦合并查分，锁定 k 与 conf_floor 后再全量；重点读 direction 组分，同时盯 cell_state/covariation/de_recovery 是否因收缩而回吐增益。预算内最多约 6–8 次查分，留 2–3 次对最优配置复测以越过 T1≈2 分噪声。单输入阶段（无两阶段可拟合）退路与父节点一致：只跑生长重采样 g^dt_out 的最新阶段复制；该退路不作为本机制的证据，真实证据只来自两输入的 X3。",
 "expected_groups": ["direction"],
 "risks": "1) 若 covariation/de_recovery 增益主要来自 kNN 平滑而非耦合，收缩位移会回吐这部分分——Engineer 应逐配置读四组分，若 direction 升而 de_recovery/covariation 大跌，说明收缩过头，调大 conf_floor 或 k。2) k 太小→位移噪声大、方向更差；k 太大→退回全局、机制失效，需看 k 扫描曲线尽早发现。3) 低 epsilon/小 k 下 JAX CPU 求解可能变慢或不收敛，超时即回退 epsilon=1e-3 仅做截断+收缩。4) 提升可能<噪声(2分)，必须对最优配置重复查分确认，勿凭单次小涨下结论。",
 "family_id": "local_ot",
 "mechanism": "把每个输出细胞的祖先分布从全局熵正则耦合改为 top-k 局部祖先截断，并按该列耦合熵对位移做逐细胞收缩：耦合越聚焦的细胞位移越大，弥散不确定的细胞位移被压缩，从而让外推方向来自真正的局部谱系配对而非全局均值场。",
 "vs_constant_shift": "位移不是每型常向量，也不是全局重加权：每个细胞的位移来自它自己的截断局部耦合列的祖先重心，且乘以其自身的耦合置信度 λ_j，同簇内不同细胞的位移向量不同。会用按簇的位移离散度证明其未退化为常向量（见 mechanism_evidence）。",
 "mechanism_evidence": "Engineer 在 X3 上报告：(a) 逐细胞位移范数与收缩因子 λ_j 的分布（确认收缩在起作用、非恒定）；(b) 在输入阶段 PCA 上 k-means 聚成若干簇作为『型』的代理（仅用输入数据，合规），计算每簇内位移向量的离散度/方向离散度，并与父节点全列重心对比——若簇内离散度接近0即退化为每型常向量，机制未生效；(c) 四组分各自随 k 与 conf_floor 的变化，重点看 direction 是否升、以及 de_recovery/covariation 是否保持；(d) 被显著收缩(λ_j 明显<1)的细胞比例。",
 "mechanism_off_control": "同一程序两个关闭开关：A) K_ANCESTORS=全列 且 conf_floor=1（即收缩系数恒=1）→ 应近似复现父节点的全局重心位移；B) λ_base=0 → 位移完全关闭，输出退化为生长重采样的最新阶段（近似 copy_last 的生长版）。预期：B 的 direction 回落到 copy_last 基线(~49.2)附近，证明 direction 的变化确实来自本机制的位移而非抽样/平滑；A 与父节点接近则证明截断+收缩是增量来源。若关闭后输出与打开时完全相同，说明机制没在运行，须返工。",
 "sources": []}
```

审查员

REVIEW.json written