总览 · ← 返回运行 20261002-202907-search-t1-scr-A

节点 n3

local_ot：moscot 耦合位移场双侧 kNN(K=100) 平滑后外推；生长重采样与 addnz 解码不变

方法说明?节点程序自带的 METHOD.md：这个程序做了什么、为什么。

来自 programs.git 02f7c6e439:solution/METHOD.md

调研员的计划

代码改动?这个节点的程序和父节点程序的逐行差别：绿色是新增，红色是删除。

对比：父节点版本 ddaa69d516。改动的文件：solution/METHOD.md +45 −67、solution/run.py +89 −8

diff --git a/solution/METHOD.md b/solution/METHOD.mdindex ca7a3d2..692e131 100644--- a/solution/METHOD.md+++ b/solution/METHOD.md@@ -1,81 +1,59 @@-# ot_moscot — Waddington-OT / moscot coupling, one-step displacement extrapolation+# local_ot：moscot 耦合位移场双侧 kNN(K=100) 平滑后外推；生长重采样与 addnz 解码不变 -Seed (2026-10-02) made from the G37 candidate `modeling/candidates/T1/ot_moscot/` (commit 227eeb2). Same method and-hyper-parameters. Changes for the seed contract only: the dev-only environment overrides (`G37_LAMBDA`, `G37_DECODE`,-`G37_GROWTH`, `G37_JAX_GPU`) and the unused `knn` decode branch are removed; JAX forced to CPU, torch threads fixed-at 8 (`EXECUTION.json {"gpu": false}`); `stage_pair` reads every input stage of the view the same way (no reference to-manifest `mode` / `source`). Output depends only on the view's data, the time differences between stages and `--seed`.+方法族 local_ot（父节点 ot_moscot 的 improve）。流程仍是：两输入阶段联合 PCA(30) → WOT 生长率先验 → moscot+TemporalProblem 耦合（epsilon=1e-3, tau_a=0.95, scale_cost=mean，全部与父节点相同）→ 输出细胞 = 末阶段按+g^dt_out 加权重采样 → 每个输出细胞加一个位移 step = λ·(dt_out/dt_in)·(pos_j − anc_j)，仅作用于非零项（addnz），+夹到 ≥0。 -Contract: `python run.py --data <view> --out <pred.h5ad> --seed <int>`; `g37_common.py` must stay next to `run.py`.+## 本节点的改动（PLAN 检验结果 + 实际采用） -## Method+PLAN 指定两个机制改动，在 X3 上实测： -Two input stages `prev` (time t0) and `last` (t1); target time t2 (final view: E8.5, E9.5 -> E10.5).+1. **top-k 稀疏化耦合（k=10）：无效（no-op）**。k=10 与全耦合的预测逐元素差 max 2.4e-7（浮点级），因为+   epsilon=1e-3 的 Sinkhorn 解每列质量本来就已集中在 ≤10 个祖先上。这正是 PLAN 对照条款预言的情形。代码保留+   该步骤（`K_SPARSE=10`），如实报告它不改变输出。+2. **去掉 kNN 平滑、用细胞自身位置外推（step=2·(x_j−anc_j)）：灾难性**。X3 分 26.3（cell_state 6.1、+   covariation 8.1）：输出变成 3x−2anc，单细胞 dropout 噪声被 factor=dt_out/dt_in=2 放大，群体分布被摧毁。+   K=5 轻平滑也差（39.8）。结论：在这个数据上 per-cell 位移的信噪比太低，方向信息在平滑后才可用。 -1. **Embedding.** Genes measured in both stages; top 2000 by variance (both stages pooled); z-score, clip at 10;-   PCA, 30 components (randomized, `random_state=seed`). Fitted on the input stages only.-2. **Growth prior (Waddington-OT).** Proliferation / apoptosis scores (`scanpy.tl.score_genes`, moscot's mouse-   gene lists) -> birth = generalised logistic(prolif; 1.7, 0.3, 0.25, 0.5), death = logistic(apopt; 1.7, 0.3, 0.1,-   0.2), per-day growth g = exp(birth - death) (Schiebinger 2019; same formula and defaults as moscot's-   `BirthDeathProblem.estimate_marginals`).-3. **Coupling.** `moscot.problems.time.TemporalProblem` prev -> last on the PCA (`joint_attr="X_pca"`),-   source marginal ∝ g^(t1-t0), target uniform; entropic unbalanced Sinkhorn, `epsilon=1e-3`, `tau_a=0.95`,-   `tau_b=1`, `scale_cost="mean"` (moscot tutorial settings). JAX on CPU unless `G37_JAX_GPU=1`.-4. **Output cells.** n = number of latest-stage cells clipped to `[min_cells, max_cells]` (as copy_last),-   drawn without replacement from the latest stage with probability ∝ g^(t2-t1) (the WOT birth-death model-   continued over the target interval).-5. **Displacement extrapolation.** For output cell j: ancestor mean a_j = Σ_i π_ij x_i / Σ_i π_ij (barycentric-   projection of the coupling, gene space, all panel genes); smoothed position s_j = mean expression of its 30-   nearest latest-stage cells in the PCA; step_j = λ (t2-t1)/(t1-t0) (s_j - a_j), λ = 1 (continue the last-   observed displacement at the same rate). The step is added **to the cell's non-zero entries only** and clipped-   at 0; genes not measured in both stages (external stages) get no step. Each cell keeps its own residual.+**实际采用的结构修复（位移场表示/解码）**：把位移的两个端点用**同一 kNN 图（K=100）**平滑——+step_j = λ·(dt_out/dt_in)·(knn₁₀₀(x)_j − knn₁₀₀(anc)_j)，anc 为全耦合重心祖先均值。相比父节点（只对 pos 做+K=30 平滑、anc 不平滑），step 范数从 55 降到 10.7，去掉了祖先指派噪声，保留了经耦合的局部传输方向。 -**One input stage (proxy view, E8.5 only):** steps 1, 3, 5 need two stages; only steps 2 + 4 run, i.e. a-growth-weighted copy of the latest stage (g^(t2-t1) resampling). This is the only thing the proxy can test.+## 机制证据（stderr diag 行） -Sources:-- Schiebinger G. et al. Optimal-transport analysis of single-cell gene expression identifies developmental-  trajectories in reprogramming. *Cell* 176, 928–943 (2019). doi:10.1016/j.cell.2019.01.006 (WOT: unbalanced-  entropic OT between snapshots, growth from proliferation/apoptosis signatures, birth-death logistic).-- Klein D., Palla G., Lange M. et al. Mapping cells through time and space with moscot. *Nature* 638, 1065–1075-  (2025). doi:10.1038/s41586-024-08453-2 (TemporalProblem; code moscot 0.5.2, BSD-3).-- Cuturi M. Sinkhorn distances. NeurIPS 2013; Chizat L. et al. Scaling algorithms for unbalanced optimal-  transport problems. *Math. Comp.* 87, 2563–2609 (2018).-- Extrapolating the barycentric displacement one more step is our use of the coupling (WOT/moscot interpolate,-  they do not extrapolate); listed in agent/knowledge/T1_methods_landscape.md §1.+- 采用版：step_norm mean/med/max = 10.7/10.4/17.1；随机同批细胞对 step 余弦 = 0.077±0.088（方向分散，非常数位移）；+  100% 细胞的 step 方向偏离其类型均值 >30°（8 个类型）。位移逐细胞不同，由各自的局部祖先集合与邻域决定。+- 对照组（`G37_MECH_OFF=1`，全耦合 + K=30 单侧平滑 = 父节点计算）：step_norm 55.3/56.2/77.9。输出确实不同，+  机制在运行。 -## Data / knowledge used+## X3 查分（A 半，seed 0 除注明外） -Only the view's input stages. Generic knowledge: moscot's built-in mouse proliferation (97) and apoptosis (193)-gene lists (`moscot.utils.data`, from the WOT paper; stage-agnostic gene-function annotation). No held-out stage,-no information from (E9.5, E13.5], no pre-trained weights.+| 配置 | 分 | direction | cell_state | covariation | de_recovery |+|---|---|---|---|---|---|+| 父节点计算（off 对照） | 50.41（s1: 47.10） | 49.03 | 49.88 | 53.45 | 50.0 |+| PLAN 机制：own-pos + top10 | 26.32 | 49.04 | 6.14 | 8.07 | 42.4 |+| own-pos 变体 K=5 平滑 | 39.76 | 48.77 | 28.35 | 37.72 | 46.1 |+| 仅 pos 平滑 K=100 | 50.43 | 49.23 | 47.95 | 55.57 | 50.5 |+| **双侧平滑 K=100（采用）** | **51.21（s1: 47.71）** | 49.95 | 57.94 | 47.0 | 47.8 |+| 双侧平滑 K=30 | 50.75 | 49.80 | 57.90 | 45.0 | 47.8 |+| 双侧 K=100 + λ=2 | 51.15 | 49.82 | 60.13 | 43.6 | 47.8 |+| pos100/anc30 非对称 | 48.97 | 50.05 | 49.54 | 48.8 | 47.3 | -## Hyper-parameters+采用版在两个种子上都配对优于 off 对照（+0.80、+0.61），direction 首次超过 copy_last 的 49.20（PLAN 目标）；+但总提升 < 2 分的噪声阈，且以 covariation/de_recovery 换 cell_state/direction，B 半是否成立未知。种子间方差+很大（±3.5，来自 652 细胞重抽样）。 -| Name | Value | Where it came from |-|---|---|---|-| `N_HVG`, `N_PCS`, `K_SMOOTH` | 2000, 30, 30 | a priori; not tuned |-| `EPSILON`, `TAU_A`, `TAU_B`, `scale_cost` | 1e-3, 0.95, 1, mean | a priori (moscot tutorial settings) |-| growth prior | on | a priori (WOT / moscot defaults); G37 saw it cost ~3 points on the old proxy, kept on |-| `LAMBDA` | 1 | a priori (continue the observed displacement at the same rate); G37 also ran 0.5 on X3 (50.4 vs 50.2), not changed |-| decode `addnz` | — | **chosen on the X3 ruler (G37)** against `add` (25.6) and `knn` (48.7): the dense step destroys the zero pattern. Also a first-principles choice (the scorer compares sparse log-expression), but the evidence that picked it was X3 |+## 开关与验证 -No re-tuning for the seed.+- `G37_MECH_OFF=1`：恢复父节点精确计算（对照用）；默认关闭（机制开）。`G37_SMOOTH_K`/`G37_ANC_K`/`G37_LAMBDA`/+  `G37_K_SPARSE`/`G37_SELF_W` 为 dev 探索用 env，默认值 = 采用配置，harness 运行时不设 env。+- 验证过：X3 seed 0/1 均 vec-check ok；默认运行与最优配置逐字节一致（md5）；单输入阶段退路（生长加权重采样，+  与父节点相同）逻辑未动但未在本节点重跑；输出只依赖时间差（dt_in、dt_out），无绝对时间、无视图路径依赖。+- 数据/知识来源：仅视图内两个输入阶段 + moscot 内置小鼠增殖/凋亡基因表（WOT 论文，Schiebinger 2019, Cell；+  通用基因功能注释，不针对禁窗）。未用任何保留阶段信息、external/、prior/。 -## Resources (Spark, CPU)+## 下一步建议 -Final view (16.8k x 17.1k coupling): 69 s, max RSS 6.7 GB; proxy 4 s / 1.6 GB; X3 ~ 8 s / 2.3 GB; proxy2 ~ 20 s / 4.3 GB.-The dense coupling (n_prev x n_last float32, ~1.1 GB on final) and the dense earlier stage (~2.2 GB) dominate memory.--## Findings (G37, local scorer: fast engine, truth half B, scorer seed = program seed)--- Dense gene-space step (`add`) is destructive: X3 25.6 at λ=1, still 32.9 at λ=0.25 (cell_state, covariation-  collapse) — the step makes every zero slightly positive. `addnz` fixes it (X3 50.2 at λ=1, 50.4 at λ=0.5);-  `knn` decode 48.7 (λ=1). Growth resampling has no effect on X3/proxy2 (all cells are kept there).-- proxy2 (E8.5 official -> Qiu E9.0 heart, then +0.5 d): every variant ≈ copy of the Qiu cells (~27.5), the-  cross-dataset step is batch effect.-- Full eval (seeds 0-2, half B): proxy 46.8 / 46.8 / 47.9 (copy_last 50.0 / 50.3 / 50.1) — the WOT growth-  resampling alone costs ~3 points on E8.5 -> E9.5 (direction, cell_state); X3 50.2 / 50.1 / 49.8 (copy_last 50.0,-  pseudobulk_shift 40.5-40.7); proxy2 27.7-27.9 (copy_last 27.4-27.6). `G37_GROWTH=0` turns the proxy into copy_last.-- Final-view prediction (`~/vec/scratch/g37/ot_moscot/final.h5ad`, seed 0): 5118 cells, 21 nearest-E9.5 types,-  composition within ±2 % of E9.5 (OFT/RV-CM -2.1 %); no new states (the method cannot create them).+cell_state 与 covariation 呈明显跷跷板（step 幅度是主轴）：可试 λ∈[1,2] 与 K∈[100,300] 细扫，或把 step 限制在+HVG/通路基因子集上以少破坏全基因协变；也可试把 anc 平滑图与 pos 平滑图解耦为不同 K 的更细网格。diff --git a/solution/run.py b/solution/run.pyindex 1411cb7..b4eb96a 100644--- a/solution/run.py+++ b/solution/run.py@@ -34,6 +34,7 @@ os.environ.setdefault("XLA_PYTHON_CLIENT_MEM_FRACTION", "0.1") os.environ["JAX_PLATFORMS"] = "cpu"  # CPU only: deterministic, no GPU slot  import numpy as np+from scipy import sparse  sys.path.insert(0, str(Path(__file__).resolve().parent)) from g37_common import embed, growth_rates, knn_mean, stage_pair  # noqa: E402@@ -50,6 +51,17 @@ LAMBDA = 1.0     # 1 = continue the observed displacement at full rate GROWTH = True    # resample output cells by g^dt_out (WOT birth-death model) N_THREADS = 8 +# mechanism (local_ot): per output cell keep only its top-k coupling ancestors, and extrapolate the cell's OWN+# displacement (x_j - sparse ancestor mean), no kNN smoothing.+# G37_MECH_OFF=1 restores the parent computation exactly (full coupling + K=30 kNN smoothing) - control only.+K_SPARSE = int(os.environ.get("G37_K_SPARSE", "10"))+MECH_OFF = os.environ.get("G37_MECH_OFF", "0") == "1"+SMOOTH_K = int(os.environ.get("G37_SMOOTH_K", "100"))  # 0 = cell's own position (no smoothing)+SELF_W = float(os.environ.get("G37_SELF_W", "0.0"))    # blend: pos = (1-w)*knn + w*own+ANC_SMOOTH = os.environ.get("G37_ANC_SMOOTH", "1") == "1"  # kNN-smooth the ancestor mean too+LAMBDA = float(os.environ.get("G37_LAMBDA", str(LAMBDA)))+ANC_K = int(os.environ.get("G37_ANC_K", "0"))  # separate kNN K for ancestor smoothing; 0 = same as SMOOTH_K+  def weighted_rows(w: np.ndarray, n: int, rng: np.random.Generator) -> np.ndarray:     p = w / w.sum()@@ -101,23 +113,92 @@ def main() -> None:     rows = weighted_rows(g_last ** dt_out if GROWTH else np.ones(last.n_obs), n, rng)     print(f"diag: growth last min/median/max {g_last.min():.3f}/{np.median(g_last):.3f}/{g_last.max():.3f}", file=sys.stderr)     Pc = P[:, rows]-    del P-    Pc /= np.maximum(Pc.sum(axis=0, keepdims=True), 1e-30)+    n_kept = Pc.shape[0]+    if not MECH_OFF:+        k = min(K_SPARSE, Pc.shape[0])+        n_kept = k+        part = np.argpartition(Pc, Pc.shape[0] - k, axis=0)[Pc.shape[0] - k:]  # (k, n) top-k ancestors per cell+        cols = np.broadcast_to(np.arange(Pc.shape[1]), part.shape)+        vals = Pc[part, cols]+        Pc = np.zeros_like(Pc)+        Pc[part, cols] = vals     factor = LAMBDA * dt_out / dt_in     torch.set_num_threads(N_THREADS)     Xp = torch.from_numpy(prev.X.toarray())-    anc = (torch.from_numpy(Pc).T @ Xp).numpy()  # barycentric ancestor mean, (n, n_genes)-    del Xp, Pc-    smooth = knn_mean(Zl, last.X, rows, K_SMOOTH)-    step = (smooth - anc) * factor-    del smooth, anc-    step[:, ~mask] = 0.0     X = last.X[rows].toarray()+    sk = K_SMOOTH if MECH_OFF else SMOOTH_K+    if ANC_SMOOTH and not MECH_OFF:+        # smooth both sides of the displacement with the SAME kNN graph: step = knn(x) - knn(anc_all)[rows]+        Pn = P / np.maximum(P.sum(axis=0, keepdims=True), 1e-30)+        anc_all = (torch.from_numpy(Pn).T @ Xp).numpy()  # (n_last, n_genes)+        del Pn, P+        from sklearn.neighbors import NearestNeighbors+        nn = NearestNeighbors(n_neighbors=min(sk, Zl.shape[0])).fit(Zl)+        idx = nn.kneighbors(Zl[rows], return_distance=False)+        kk = idx.shape[1]+        W = sparse.csr_matrix((np.full(idx.size, 1.0 / kk, dtype=np.float32), idx.ravel(),+                               np.arange(0, idx.size + 1, kk)), shape=(len(rows), Zl.shape[0]))+        smooth = np.asarray((W @ last.X).todense(), dtype=np.float32)+        if ANC_K > 0 and ANC_K != sk:+            nn2 = NearestNeighbors(n_neighbors=min(ANC_K, Zl.shape[0])).fit(Zl)+            idx2 = nn2.kneighbors(Zl[rows], return_distance=False)+            k2 = idx2.shape[1]+            W2 = sparse.csr_matrix((np.full(idx2.size, 1.0 / k2, dtype=np.float32), idx2.ravel(),+                                    np.arange(0, idx2.size + 1, k2)), shape=(len(rows), Zl.shape[0]))+            anc = np.asarray((W2 @ anc_all), dtype=np.float32)+        else:+            anc = np.asarray(W @ anc_all, dtype=np.float32)+        del anc_all+        step = (smooth - anc) * factor+        del smooth+    else:+        del P+        Pc /= np.maximum(Pc.sum(axis=0, keepdims=True), 1e-30)+        anc = (torch.from_numpy(Pc).T @ Xp).numpy()  # barycentric ancestor mean, (n, n_genes)+        del Pc+        if sk > 0:+            smooth = knn_mean(Zl, last.X, rows, sk)+            w = 0.0 if MECH_OFF else SELF_W+            pos = X * w + smooth * (1.0 - w) if w > 0 else smooth+            del smooth+        else:+            pos = X+        step = (pos - anc) * factor+        del anc, pos+    del Xp+    step[:, ~mask] = 0.0     step *= X > 0  # "addnz": move only measured (non-zero) entries, keeps each cell's zero pattern+    _diag_steps(step, last.obs["celltype"].to_numpy()[rows], n_kept)     X += step     np.maximum(X, 0.0, out=X)     write_prediction(X, genes, args.out, seed=args.seed)  +def _diag_steps(step: np.ndarray, types: np.ndarray, n_kept: int) -> None:+    """Mechanism evidence (stderr only): step heterogeneity within cell types."""+    norms = np.linalg.norm(step, axis=1)+    keep = norms > 0+    S = step[keep]+    S = S / np.linalg.norm(S, axis=1, keepdims=True)+    rng = np.random.default_rng(0)+    m = min(len(S), 2000)+    sel = rng.choice(len(S), size=m, replace=False)+    pairs = rng.integers(0, m, size=(500, 2))+    cos = (S[sel[pairs[:, 0]]] * S[sel[pairs[:, 1]]]).sum(axis=1)+    uniq = np.unique(types[keep])+    dev = []+    for t in uniq:+        idx = np.where(types[keep] == t)[0]+        if len(idx) < 5:+            continue+        mu = S[idx].mean(axis=0)+        mu /= max(np.linalg.norm(mu), 1e-12)+        dev.append((S[idx] @ mu < np.cos(np.deg2rad(30))).mean())+    print(f"diag: mech_off={MECH_OFF} k_anc={n_kept} step_norm mean/med/max "+          f"{norms.mean():.3f}/{np.median(norms):.3f}/{norms.max():.3f} "+          f"rand_pair_cos mean/std {cos.mean():.3f}/{cos.std():.3f} "+          f"frac_>30deg_from_typemean {np.mean(dev):.3f} (n_types={len(dev)})", file=sys.stderr)++ if __name__ == "__main__":     main()

调研来源?调研员查到并用到的知识条目和文献检索结果（只列标题和编号）。

用到的知识库条目

分析结果?分析员写的 ANALYSIS.json：改了什么、各组分数怎么变、假设是否成立、经验和下一步建议。

对话摘要?每个角色和大模型对话的统计：轮数、工具调用、用时、token 数和最后的回答摘录；原始记录只给路径。

只给统计和最后回答的摘录；完整对话请到原始记录位置里列出的文件看。

分析员

ANALYSIS.json written

工程师

**做了什么**：按 PLAN 实现 local_ot 机制改动并实测——top-k 稀疏化耦合被证明是 no-op（耦合列本就集中在 ≤10 祖先，输出差 2e-7）；去掉 kNN 平滑用细胞自身位置外推是灾难（26.3，factor=2 放大 dropout 噪声）。实际采用的结构修复：位移场两端用同一 kNN(K=100) 图双侧平滑（knn(x)−knn(anc)），保留生长重采样与 addnz 解码，`G37_MECH_OFF=1` 可精确恢复父计算作对照。

**查分**（X3 A 半，10 次查询）：off 对照 50.41 ≈ 父 50.65；采用版 seed0 **51.21** / seed1 47.71，两种子均配对优于对照（+0.80/+0.61）；direction 49.95 首次超 copy_last 49.20，cell_state 57.9 大涨，但 covariation/de_recovery 回落，总提升在噪声阈内。两 seed 均过 vec-check，默认运行与最优配置逐字节一致。

**下一步最值得试**：step 幅度是 cell_state↔covariation 跷跷板主轴——细扫 λ∈[1,2]×K∈[100,300]，或把 step 限制在 HVG/通路基因子集上以保住全基因协变；也值得在 pos/anc 用不同 K 的解耦网格上找双优点。

调研员

-k sparsification with small k may make ancestor estimates unstable for cells with diffuse coupling (high entropy columns); check whether step norms correlate inversely with coupling entropy—if so, add confidence weighting w_j=1-H_j/H_max. 3) If direction still doesn't improve, the coupling itself may be too entropic; try epsilon=1e-4 as a secondary run. Discover early by running one small subset (2000 cells per stage) and checking step-vector cosine diversity before full run.",
  "family_id": "local_ot",
  "mechanism": "Sparsify the OT coupling to each cell's top-k ancestors, then extrapolate the cell's own observed displacement (own position minus sparse barycentric ancestor) forward—making the transport local per cell rather than smoothed over 30 neighbors.",
  "vs_constant_shift": "Each cell's displacement depends on its individual sparse ancestor set from the coupling; different cells in the same type get different directions and magnitudes. A constant shift would give one vector per type. Evidence: report std of step vectors within each cluster/type.",
  "mechanism_evidence": "1) Report per-cell step norms and within-cluster std of step vectors (should be substantially >0 and heterogeneous, not one vector per region). 2) Compare direction group score specifically: must exceed parent's 49.14. 3) Print cosine similarity matrix of steps for 100 random same-type pairs: should show spread, not all ≈1. 4) Report fraction of cells whose step direction differs from their cluster mean by >30°.",
  "mechanism_off_control": "Set k_sparsify = n_prev (use full coupling, no sparsification) AND restore K=30 kNN smoothing. This recovers the parent's exact computation. If output with mechanism 'off' equals output with mechanism 'on', the sparsification and direct-displacement changes had no effect and the mechanism is not running. Expected difference: off-control reproduces parent score ~50.65 with direction ~49.1; on-control should show direction ≥ 50.",
  "sources": []
}
```

审查员

REVIEW.json written