Virtual Embryo Challenge更新于 10-03 18:47(北京时间) / 每 5 分钟更新

总览 · ← 返回运行 20261003-093415-search-t1-r2-D-s1

节点 n5

OT 位移解码改为投影到前 22 个 PCA 程序方向(α=1,无基因激活),单输入阶段回退改为均匀复制(去掉生长重采样)。

运行?一次完整的自动搜索或 Agent 会话,有自己的锁定配置和证据包。20261003-093415-search-t1-r2-D-s1
父节点n2
子节点—
操作?种子:人写的起点;改进:在父节点上改;草稿:从头写;修复:修父节点的报错。改进
状态已打分
分数搜索目标分 52.82(+2.5) · X3 52.76(+2.1) · proxy10 52.94(+3.3) · 3 次复测均分 51.44
审查未审查
用时?从运行开始到结束(或到现在)的挂钟时间。20 分
程序版本a2b75251d98408641cac8da351017aef2f36e74d (programs.git)

方法说明?节点程序自带的 METHOD.md:这个程序做了什么、为什么。

来自 programs.git a2b75251d9:solution/METHOD.md

OT 位移解码改为投影到前 22 个 PCA 程序方向(α=1,无基因激活),单输入阶段回退改为均匀复制(去掉生长重采样)。

local_ot: program-projected displacement (improve over node 2 ot_moscot)

方法

两个输入阶段的路径与父节点完全相同(joint HVG/PCA → moscot TemporalProblem 耦合 → g^dt_out 重采样输出细胞 → raw_step = λ·dt_out/dt_in·(kNN 平滑位置 − 耦合加权祖先均值),λ=1,addnz 解码:step 只加到该细胞的非零项、夹到 ≥0)。 唯一改动在解码表示:

  1. PCA 程序投影(机制,默认开):把 raw_step 换到标准化 HVG 空间(除以 embed 已拟合的 sd),投影到前 K_PROJ=22 个 PCA 载荷张成的子空间,再变回基因空间:step = ((raw[:,hvg]/sd) @ V_K.T @ V_K) * sd; 非 HVG 基因的位移置 0。ALPHA=1.0(收缩系数保留为开关,数据选了 1)。逐基因噪声被去掉,只保留数据驱动的 程序级协同位移;每细胞的位移仍由它自己的 OT 耦合与局部邻域决定(非常数位移:投影后型内位移 std 为无约束 版本的 13.9%,K=15 时 6.1%,见下方证据与讨论)。
  2. 基因激活(已实现,默认关 M_ACT=0):按 PLAN 实现了"邻域检出率 − 祖先检出率 ≥ 0.15 且 |top-K 载荷| > 0.02 的零基因,按增幅×载荷排序取前 M 个,置为 30 近邻非零中位数 × 0.5"。X3 上 M=50 使四项全面下降 (47.15 vs 同解码无激活 50.41;mmd_u skill 0.50→0.45),故提交默认 M=0。
  3. 单阶段回退修复:只有一个输入阶段(proxy)时不再做 g^dt_out 生长重采样(父节点因此比 copy_last 低约 3 分), 改为均匀复制最后阶段(n = clip(n_obs, min_cells, max_cells);n ≥ n_obs 时按原序全取)。单阶段没有任何两阶段差 信息可用,这是诚实回退。

机制开关(环境变量,仅为跑对照,提交默认=机制开):N5_ALPHA / N5_KPROJ(0=不投影) / N5_MACT / N5_GROWTH1 (1=恢复父节点单阶段生长重采样)。

机制对照(PLAN mechanism_off_control,X3 视图,seed 0,A 半查分)

配置X3 榜分de_scorede_dirmmd_uvariogram
机制关(α=1, K=0, M=0 ≈ 父节点解码)50.410.5000.4900.4990.534
机制开(α=1, K=22, M=0,提交默认)52.780.5150.5140.5620.510
α=0.5, K=15, M=50(PLAN 初始默认)47.20.4490.5120.4900.424
α=1, K=0, M=50(只激活)47.150.4650.4900.4520.487
K 扫描(α=1):K=10 / 15 / 20 / 25 / 3051.41 / 52.51 / 52.80 / 52.85 / 52.40
α 扫描(K=15/20):α=0.551.86 / 51.78

机制生效证据:机制开与关的输出不同(mmd_u skill 0.499→0.562 为主要收益,de 两项同向上升);投影确实改变细胞 (型内位移 std 为无约束版的 6–14%,即保留了部分而非全部型内异质性;输出零占比变化 ≤ 0.01%)。收益主体在 cell_state(mmd_u),与 PLAN 预期一致。K=15 时型内 std 比例 6.1% 低于 PLAN 风险线 20%,按风险预案增大 K; K∈[15,30] 分数均高于 K=0,K=22 取在 20/25 之间。α=0.5 一致地低于 α=1(4 组对照同向),故收缩关闭。

proxy10(单阶段回退,均匀复制):53.19(父节点 49.67;同流程 copy_last 参考 52.75——回退路径输出即全量复制, 差值来自评分抽样)。预计节点分 ≈ (53.19 + 2×52.78)/3 ≈ 52.9 vs 父 50.33。

已验证 / 未验证

  • 已验证:X3(两外部阶段,dt_out/dt_in=2)上投影解码优于父解码约 2.4 分(A 半,噪声约 2 分,方向与 5 个 K 值 的一致性支持它);proxy 单阶段路径跑通、vec-check 通过;seed 0 复跑逐字节一致,seed 1 正常;X3 652 细胞 ~19 s / 内存小(<2 GB 级别),proxy ~2 s。
  • 未验证:final 视图(E8.5+E9.5→E10.5,16.8k×17.1k 耦合)未跑(不在本节点视图里);父节点在 final 上 69 s / 6.7 GB,本改动只增加 (n×2000) 矩阵乘,激活路径默认关,预计资源相近。dt_out/dt_in=1(final)时 factor=1, 投影幅度更小,方向性收益按 X3 外推但幅度未测。生长重采样(两阶段路径的 GROWTH=True)保持不变,其在 final 上的效果未测(父节点证据:X3/proxy2 上无影响)。基因激活只测了 M=50(阈值 0.15、值系数 0.5),更小的 M 未测。
  • 视图无关性:输出只依赖视图数据、时间差(dt_in/dt_out 平移不变)与 seed;无绝对时间、路径、manifest 排版依赖。

知识来源

与父节点相同:Waddington-OT(Schiebinger et al., Cell 2019, doi:10.1016/j.cell.2019.01.006)、moscot TemporalProblem(Klein et al., Nature 2025, doi:10.1038/s41586-024-08453-2)、moscot 内置小鼠增殖/凋亡基因表 (阶段无关的基因功能注释)。新增部分(PCA 程序投影)不引入任何外部生物学知识:投影方向完全由两个输入阶段 自己的数据拟合。未使用保留阶段/保留基因型的任何信息。

调研员的计划

名称local_ot: PCA-program-projected displacement with shrinkage + capped gene activation
动机Node 2 cell_state=49.21 is the weakest group; proxy10 mmd_u skill=0.475, variogram skill=0.470 are below floor. The addnz decode applies unconstrained displacement in full gene space (noise) and cannot activate new genes. On proxy10, growth resampling scores 49.67 vs copy_last 52.75 (−3.08). Node 3 (composition_trend, 53.69) shows composition changes help; node 2's unconstrained gene-space step likely degrades distribution structure. Direction library priority: fix 'genes can't turn on' and 'displacement unconstrained'.
做法Modify the decode step in three ways, keeping the OT coupling (steps 1–3) unchanged:

1. Program-constrained displacement: After computing raw_step = (smooth − anc) × factor, project onto top K=15 PCA loadings (from the already-fitted PCA): step_proj = (raw_step @ V_K) @ V_K.T. Apply shrinkage: step_final = α × step_proj, with α initial 0.5, search [0.3, 0.8]. This removes gene-level noise while preserving coherent program-level movement. K and α are the only new hyperparameters.

2. Capped gene activation: For each output cell j, identify candidate genes where: (a) expression is 0 in cell j, (b) the gene's detection rate in the 30 nearest later-stage neighbours (already computed for smooth) exceeds the detection rate in the cell's ancestor distribution by ≥0.15, (c) the gene has |loading| > 0.02 in at least one of the top K PCs. Activate the top M=50 candidates (ranked by detection-rate increase × max |loading|), setting each to the median non-zero value among those 30 neighbours × 0.5. This is sparse, module-supported, and preserves the cell's overall sparsity pattern for the vast majority of genes.

3. Single-stage fallback fix: When only one input stage exists (pro…
风险1. PCA projection may over-smooth real heterogeneous displacement, losing cell-type-specific signals → check within-type displacement std; if <20% of unconstrained, increase K. 2. Gene activation may introduce artifacts if detection-rate threshold too low → monitor zero-fraction change; should stay within ±2% of input. 3. α too low → displacement vanishes, output ≈ copy_last (score ~50); α too high → same as parent. Check de_direction and de_score to confirm directional signal preserved. 4. 30 min budget: PCA projection is cheap (matrix multiply); gene activation loop vectorized over cells. Total added compute <1 min. If moscot solve is the bottleneck, subsample to 3000 cells per stage.

代码改动?这个节点的程序和父节点程序的逐行差别:绿色是新增,红色是删除。

对比:父节点版本 25a98fa4a1。改动的文件:solution/METHOD.md +62 −81、solution/run.py +151 −32

diff --git a/solution/METHOD.md b/solution/METHOD.mdindex ca7a3d2..09a9e7b 100644--- a/solution/METHOD.md+++ b/solution/METHOD.md@@ -1,81 +1,62 @@-# ot_moscot — Waddington-OT / moscot coupling, one-step displacement extrapolation--Seed (2026-10-02) made from the G37 candidate `modeling/candidates/T1/ot_moscot/` (commit 227eeb2). Same method and-hyper-parameters. Changes for the seed contract only: the dev-only environment overrides (`G37_LAMBDA`, `G37_DECODE`,-`G37_GROWTH`, `G37_JAX_GPU`) and the unused `knn` decode branch are removed; JAX forced to CPU, torch threads fixed-at 8 (`EXECUTION.json {"gpu": false}`); `stage_pair` reads every input stage of the view the same way (no reference to-manifest `mode` / `source`). Output depends only on the view's data, the time differences between stages and `--seed`.--Contract: `python run.py --data <view> --out <pred.h5ad> --seed <int>`; `g37_common.py` must stay next to `run.py`.--## Method--Two input stages `prev` (time t0) and `last` (t1); target time t2 (final view: E8.5, E9.5 -> E10.5).--1. **Embedding.** Genes measured in both stages; top 2000 by variance (both stages pooled); z-score, clip at 10;-   PCA, 30 components (randomized, `random_state=seed`). Fitted on the input stages only.-2. **Growth prior (Waddington-OT).** Proliferation / apoptosis scores (`scanpy.tl.score_genes`, moscot's mouse-   gene lists) -> birth = generalised logistic(prolif; 1.7, 0.3, 0.25, 0.5), death = logistic(apopt; 1.7, 0.3, 0.1,-   0.2), per-day growth g = exp(birth - death) (Schiebinger 2019; same formula and defaults as moscot's-   `BirthDeathProblem.estimate_marginals`).-3. **Coupling.** `moscot.problems.time.TemporalProblem` prev -> last on the PCA (`joint_attr="X_pca"`),-   source marginal ∝ g^(t1-t0), target uniform; entropic unbalanced Sinkhorn, `epsilon=1e-3`, `tau_a=0.95`,-   `tau_b=1`, `scale_cost="mean"` (moscot tutorial settings). JAX on CPU unless `G37_JAX_GPU=1`.-4. **Output cells.** n = number of latest-stage cells clipped to `[min_cells, max_cells]` (as copy_last),-   drawn without replacement from the latest stage with probability ∝ g^(t2-t1) (the WOT birth-death model-   continued over the target interval).-5. **Displacement extrapolation.** For output cell j: ancestor mean a_j = Σ_i π_ij x_i / Σ_i π_ij (barycentric-   projection of the coupling, gene space, all panel genes); smoothed position s_j = mean expression of its 30-   nearest latest-stage cells in the PCA; step_j = λ (t2-t1)/(t1-t0) (s_j - a_j), λ = 1 (continue the last-   observed displacement at the same rate). The step is added **to the cell's non-zero entries only** and clipped-   at 0; genes not measured in both stages (external stages) get no step. Each cell keeps its own residual.--**One input stage (proxy view, E8.5 only):** steps 1, 3, 5 need two stages; only steps 2 + 4 run, i.e. a-growth-weighted copy of the latest stage (g^(t2-t1) resampling). This is the only thing the proxy can test.--Sources:-- Schiebinger G. et al. Optimal-transport analysis of single-cell gene expression identifies developmental-  trajectories in reprogramming. *Cell* 176, 928–943 (2019). doi:10.1016/j.cell.2019.01.006 (WOT: unbalanced-  entropic OT between snapshots, growth from proliferation/apoptosis signatures, birth-death logistic).-- Klein D., Palla G., Lange M. et al. Mapping cells through time and space with moscot. *Nature* 638, 1065–1075-  (2025). doi:10.1038/s41586-024-08453-2 (TemporalProblem; code moscot 0.5.2, BSD-3).-- Cuturi M. Sinkhorn distances. NeurIPS 2013; Chizat L. et al. Scaling algorithms for unbalanced optimal-  transport problems. *Math. Comp.* 87, 2563–2609 (2018).-- Extrapolating the barycentric displacement one more step is our use of the coupling (WOT/moscot interpolate,-  they do not extrapolate); listed in agent/knowledge/T1_methods_landscape.md §1.--## Data / knowledge used--Only the view's input stages. Generic knowledge: moscot's built-in mouse proliferation (97) and apoptosis (193)-gene lists (`moscot.utils.data`, from the WOT paper; stage-agnostic gene-function annotation). No held-out stage,-no information from (E9.5, E13.5], no pre-trained weights.--## Hyper-parameters--| Name | Value | Where it came from |-|---|---|---|-| `N_HVG`, `N_PCS`, `K_SMOOTH` | 2000, 30, 30 | a priori; not tuned |-| `EPSILON`, `TAU_A`, `TAU_B`, `scale_cost` | 1e-3, 0.95, 1, mean | a priori (moscot tutorial settings) |-| growth prior | on | a priori (WOT / moscot defaults); G37 saw it cost ~3 points on the old proxy, kept on |-| `LAMBDA` | 1 | a priori (continue the observed displacement at the same rate); G37 also ran 0.5 on X3 (50.4 vs 50.2), not changed |-| decode `addnz` | — | **chosen on the X3 ruler (G37)** against `add` (25.6) and `knn` (48.7): the dense step destroys the zero pattern. Also a first-principles choice (the scorer compares sparse log-expression), but the evidence that picked it was X3 |--No re-tuning for the seed.--## Resources (Spark, CPU)--Final view (16.8k x 17.1k coupling): 69 s, max RSS 6.7 GB; proxy 4 s / 1.6 GB; X3 ~ 8 s / 2.3 GB; proxy2 ~ 20 s / 4.3 GB.-The dense coupling (n_prev x n_last float32, ~1.1 GB on final) and the dense earlier stage (~2.2 GB) dominate memory.--## Findings (G37, local scorer: fast engine, truth half B, scorer seed = program seed)--- Dense gene-space step (`add`) is destructive: X3 25.6 at λ=1, still 32.9 at λ=0.25 (cell_state, covariation-  collapse) — the step makes every zero slightly positive. `addnz` fixes it (X3 50.2 at λ=1, 50.4 at λ=0.5);-  `knn` decode 48.7 (λ=1). Growth resampling has no effect on X3/proxy2 (all cells are kept there).-- proxy2 (E8.5 official -> Qiu E9.0 heart, then +0.5 d): every variant ≈ copy of the Qiu cells (~27.5), the-  cross-dataset step is batch effect.-- Full eval (seeds 0-2, half B): proxy 46.8 / 46.8 / 47.9 (copy_last 50.0 / 50.3 / 50.1) — the WOT growth-  resampling alone costs ~3 points on E8.5 -> E9.5 (direction, cell_state); X3 50.2 / 50.1 / 49.8 (copy_last 50.0,-  pseudobulk_shift 40.5-40.7); proxy2 27.7-27.9 (copy_last 27.4-27.6). `G37_GROWTH=0` turns the proxy into copy_last.-- Final-view prediction (`~/vec/scratch/g37/ot_moscot/final.h5ad`, seed 0): 5118 cells, 21 nearest-E9.5 types,-  composition within ±2 % of E9.5 (OFT/RV-CM -2.1 %); no new states (the method cannot create them).+OT 位移解码改为投影到前 22 个 PCA 程序方向(α=1,无基因激活),单输入阶段回退改为均匀复制(去掉生长重采样)。++# local_ot: program-projected displacement (improve over node 2 ot_moscot)++## 方法++两个输入阶段的路径与父节点完全相同(joint HVG/PCA → moscot TemporalProblem 耦合 → g^dt_out 重采样输出细胞 →+raw_step = λ·dt_out/dt_in·(kNN 平滑位置 − 耦合加权祖先均值),λ=1,addnz 解码:step 只加到该细胞的非零项、夹到 ≥0)。+唯一改动在**解码表示**:++1. **PCA 程序投影(机制,默认开)**:把 raw_step 换到标准化 HVG 空间(除以 embed 已拟合的 sd),投影到前+   `K_PROJ=22` 个 PCA 载荷张成的子空间,再变回基因空间:`step = ((raw[:,hvg]/sd) @ V_K.T @ V_K) * sd`;+   非 HVG 基因的位移置 0。`ALPHA=1.0`(收缩系数保留为开关,数据选了 1)。逐基因噪声被去掉,只保留数据驱动的+   程序级协同位移;每细胞的位移仍由它自己的 OT 耦合与局部邻域决定(非常数位移:投影后型内位移 std 为无约束+   版本的 13.9%,K=15 时 6.1%,见下方证据与讨论)。+2. **基因激活(已实现,默认关 M_ACT=0)**:按 PLAN 实现了"邻域检出率 − 祖先检出率 ≥ 0.15 且 |top-K 载荷| > 0.02+   的零基因,按增幅×载荷排序取前 M 个,置为 30 近邻非零中位数 × 0.5"。X3 上 M=50 使四项全面下降+   (47.15 vs 同解码无激活 50.41;mmd_u skill 0.50→0.45),故提交默认 M=0。+3. **单阶段回退修复**:只有一个输入阶段(proxy)时不再做 g^dt_out 生长重采样(父节点因此比 copy_last 低约 3 分),+   改为均匀复制最后阶段(n = clip(n_obs, min_cells, max_cells);n ≥ n_obs 时按原序全取)。单阶段没有任何两阶段差+   信息可用,这是诚实回退。++机制开关(环境变量,仅为跑对照,提交默认=机制开):`N5_ALPHA` / `N5_KPROJ`(0=不投影) / `N5_MACT` / `N5_GROWTH1`+(1=恢复父节点单阶段生长重采样)。++## 机制对照(PLAN mechanism_off_control,X3 视图,seed 0,A 半查分)++| 配置 | X3 榜分 | de_score | de_dir | mmd_u | variogram |+|---|---|---|---|---|---|+| 机制关(α=1, K=0, M=0 ≈ 父节点解码) | 50.41 | 0.500 | 0.490 | 0.499 | 0.534 |+| 机制开(α=1, K=22, M=0,提交默认) | **52.78** | 0.515 | 0.514 | **0.562** | 0.510 |+| α=0.5, K=15, M=50(PLAN 初始默认) | 47.2 | 0.449 | 0.512 | 0.490 | 0.424 |+| α=1, K=0, M=50(只激活) | 47.15 | 0.465 | 0.490 | 0.452 | 0.487 |+| K 扫描(α=1):K=10 / 15 / 20 / 25 / 30 | 51.41 / 52.51 / 52.80 / 52.85 / 52.40 | | | | |+| α 扫描(K=15/20):α=0.5 | 51.86 / 51.78 | | | | |++机制生效证据:机制开与关的输出不同(mmd_u skill 0.499→0.562 为主要收益,de 两项同向上升);投影确实改变细胞+(型内位移 std 为无约束版的 6–14%,即保留了部分而非全部型内异质性;输出零占比变化 ≤ 0.01%)。收益主体在+cell_state(mmd_u),与 PLAN 预期一致。K=15 时型内 std 比例 6.1% 低于 PLAN 风险线 20%,按风险预案增大 K;+K∈[15,30] 分数均高于 K=0,K=22 取在 20/25 之间。α=0.5 一致地低于 α=1(4 组对照同向),故收缩关闭。++proxy10(单阶段回退,均匀复制):53.19(父节点 49.67;同流程 copy_last 参考 52.75——回退路径输出即全量复制,+差值来自评分抽样)。预计节点分 ≈ (53.19 + 2×52.78)/3 ≈ 52.9 vs 父 50.33。++## 已验证 / 未验证++- 已验证:X3(两外部阶段,dt_out/dt_in=2)上投影解码优于父解码约 2.4 分(A 半,噪声约 2 分,方向与 5 个 K 值+  的一致性支持它);proxy 单阶段路径跑通、vec-check 通过;seed 0 复跑逐字节一致,seed 1 正常;X3 652 细胞+  ~19 s / 内存小(<2 GB 级别),proxy ~2 s。+- 未验证:final 视图(E8.5+E9.5→E10.5,16.8k×17.1k 耦合)未跑(不在本节点视图里);父节点在 final 上 69 s /+  6.7 GB,本改动只增加 (n×2000) 矩阵乘,激活路径默认关,预计资源相近。dt_out/dt_in=1(final)时 factor=1,+  投影幅度更小,方向性收益按 X3 外推但幅度未测。生长重采样(两阶段路径的 GROWTH=True)保持不变,其在 final+  上的效果未测(父节点证据:X3/proxy2 上无影响)。基因激活只测了 M=50(阈值 0.15、值系数 0.5),更小的 M+  未测。+- 视图无关性:输出只依赖视图数据、时间差(dt_in/dt_out 平移不变)与 seed;无绝对时间、路径、manifest 排版依赖。++## 知识来源++与父节点相同:Waddington-OT(Schiebinger et al., Cell 2019, doi:10.1016/j.cell.2019.01.006)、moscot+TemporalProblem(Klein et al., Nature 2025, doi:10.1038/s41586-024-08453-2)、moscot 内置小鼠增殖/凋亡基因表+(阶段无关的基因功能注释)。新增部分(PCA 程序投影)不引入任何外部生物学知识:投影方向完全由两个输入阶段+自己的数据拟合。未使用保留阶段/保留基因型的任何信息。diff --git a/solution/run.py b/solution/run.pyindex 1411cb7..ce49718 100644--- a/solution/run.py+++ b/solution/run.py@@ -1,25 +1,25 @@ #!/usr/bin/env python3-"""ot_moscot: Waddington-OT / moscot TemporalProblem coupling, extrapolated one step past the latest input stage.+"""local_ot: PCA-program-projected displacement with shrinkage + capped gene activation (improve over ot_moscot). -Two input stages (final: E8.5, E9.5):+Two input stages (X3: Qiu E8.75, E9.0 -> E9.5; final: E8.5, E9.5 -> E10.5):   1. joint PCA of both stages (HVG, z-score, 30 PCs; fitted on the inputs only);-  2. moscot TemporalProblem prev -> last on the PCA, source marginals from Waddington-OT prior growth-     (proliferation / apoptosis gene scores), entropic unbalanced Sinkhorn (epsilon 1e-3, tau_a 0.95);-  3. output cells = latest-stage cells resampled with weights g^dt_out (prior growth rate continued for the target-     interval, Waddington-OT birth-death model);-  4. each output cell j moves by LAMBDA * dt_out / dt_in * (kNN-mean(x_j) - ancestor_mean_j) in gene space, where-     ancestor_mean_j is the coupling-weighted (barycentric) mean of its ancestors in the earlier stage: the last-     observed displacement continued for the target interval; the cell keeps its own residual. The step is applied-     to the cell's non-zero entries only (DECODE "addnz": a dense step turns every zero into a small positive value-     and wrecks cell_state / covariation, see METHOD.md). Clipped at 0.-One input stage (proxy: E8.5 only): steps 1, 2, 4 need two stages; only the growth resampling (3) runs, i.e.-a growth-weighted copy of the latest stage.--Seed version (agent/seeds/T1__val/ot_moscot, 2026-10-02) of modeling/candidates/T1/ot_moscot (G37): same method and-hyper-parameters; the dev-only environment overrides (G37_LAMBDA / G37_DECODE / G37_GROWTH / G37_JAX_GPU) and the-`knn` decode branch are removed; JAX and torch on CPU, fixed thread count (EXECUTION.json gpu false).-Parameter provenance (METHOD.md): DECODE = addnz was chosen on the X3 ruler (G37); LAMBDA = 1 a priori, also checked-on X3 (0.5 vs 1 within 0.2).+  2. moscot TemporalProblem prev -> last on the PCA, WOT growth source marginals, entropic unbalanced Sinkhorn;+  3. output cells = latest-stage cells resampled with weights g^dt_out;+  4. raw displacement raw_step = LAMBDA * dt_out/dt_in * (kNN-smoothed position - coupling-weighted ancestor mean);+     NEW: projected onto the top K_PROJ PCA programs in the standardized HVG space (removes gene-level noise, keeps+     coherent program-level movement); ALPHA shrinkage kept as a switch, data chose ALPHA=1;+  5. decode "addnz": the step moves only the cell's non-zero entries, clipped at 0;+  6. capped gene activation (implemented, default OFF M_ACT=0: on X3 it lowered all four metric groups, see+     METHOD.md): up to M_ACT genes per cell that are 0 in the cell but whose detection rate in its 30 nearest+     latest-stage neighbours exceeds the detection rate in its OT ancestor distribution by >= DET_THRESH,+     restricted to genes with |loading| > LOAD_MIN in one of the top K PCs; value = median non-zero neighbour+     value * VAL_FRAC.+One input stage (proxy: E8.5 only): OT needs two stages; honest fallback = uniform copy of the latest stage+(NEW: the parent's growth-weighted resampling is removed here - it cost ~3 points on proxy10 with no signal to+estimate growth from a single stage).++Mechanism-off control (recovers the parent's decode): N5_ALPHA=1 N5_KPROJ=0 N5_MACT=0; single-stage growth+resampling restored with N5_GROWTH1=1. Submitted defaults keep the mechanism ON. """  from __future__ import annotations@@ -36,7 +36,7 @@ os.environ["JAX_PLATFORMS"] = "cpu"  # CPU only: deterministic, no GPU slot import numpy as np  sys.path.insert(0, str(Path(__file__).resolve().parent))-from g37_common import embed, growth_rates, knn_mean, stage_pair  # noqa: E402+from g37_common import embed, growth_rates, stage_pair  # noqa: E402  from src.task1_temporal.view_io import load_manifest, panel_genes, target_n_cells, write_prediction  # noqa: E402 @@ -47,15 +47,85 @@ TAU_A = 0.95 TAU_B = 1.0 K_SMOOTH = 30 LAMBDA = 1.0     # 1 = continue the observed displacement at full rate-GROWTH = True    # resample output cells by g^dt_out (WOT birth-death model)+GROWTH = True    # two-stage path: resample output cells by g^dt_out (WOT birth-death model) N_THREADS = 8 +# --- mechanism switches (defaults ON; env overrides are for the mechanism-off control only) ---+ALPHA = float(os.environ.get("N5_ALPHA", "1.0"))        # shrinkage of the program-projected step+K_PROJ = int(os.environ.get("N5_KPROJ", "22"))          # top-K PCA programs; 0 = no projection (control)+M_ACT = int(os.environ.get("N5_MACT", "0"))             # activated genes per cell; 0 = off (X3: harmful)+GROWTH1 = os.environ.get("N5_GROWTH1", "0") == "1"      # single-stage fallback: growth resampling (parent) if 1+DET_THRESH = float(os.environ.get("N5_DET", "0.15"))    # detection-rate increase required for activation+LOAD_MIN = float(os.environ.get("N5_LOAD", "0.02"))     # |PC loading| required for activation candidates+VAL_FRAC = float(os.environ.get("N5_VALFRAC", "0.5"))   # activated value = median non-zero neighbour value * this+  def weighted_rows(w: np.ndarray, n: int, rng: np.random.Generator) -> np.ndarray:     p = w / w.sum()     return np.sort(rng.choice(len(w), size=n, replace=n > len(w), p=p))  +def within_type_std(S: np.ndarray, ct: np.ndarray) -> float:+    tot = 0.0+    for t in np.unique(ct):+        A = S[ct == t]+        if len(A) > 1:+            tot += float(((A - A.mean(0)) ** 2).sum())+    return float(np.sqrt(tot / max(S.size, 1)))+++def activate_genes(X, idx, W, PcT, prev, last, gidx, lm, M, rng_free=None):+    """Capped gene activation (PLAN step 2). X: dense (n, n_genes) output being built; idx: (n, K) neighbour+    indices into last; W: (n, n_last) kNN mean operator; PcT: (n, n_prev) normalised coupling; gidx: candidate+    panel-gene indices; lm: their max |loading| over the top-K PCs. Returns per-cell activation counts."""+    from scipy import sparse+    import torch++    n = X.shape[0]+    Bl = last.X[:, gidx].copy()+    Bl.data[:] = 1.0+    det_near = np.asarray((W @ Bl).todense(), dtype=np.float32)          # (n, G)+    Bp = prev.X[:, gidx].toarray()+    Bp = Bp.astype(np.float32)+    Bp[Bp > 0] = 1.0+    torch.set_num_threads(N_THREADS)+    anc_det = (torch.from_numpy(PcT) @ torch.from_numpy(Bp)).numpy()     # (n, G)+    del Bp+    increase = det_near - anc_det+    elig = (increase >= DET_THRESH) & (X[:, gidx] == 0)+    score = np.where(elig, increase * lm[None, :], -np.inf).astype(np.float32)+    if score.shape[1] > M:+        part = np.argpartition(-score, M - 1, axis=1)[:, :M]+    else:+        part = np.broadcast_to(np.arange(score.shape[1]), (n, score.shape[1]))+    rowi = np.repeat(np.arange(n, dtype=np.int64), part.shape[1])+    coli = np.asarray(part).ravel().astype(np.int64)+    keep = np.isfinite(score[rowi, coli])+    rowi, coli = rowi[keep], coli[keep]+    order = np.argsort(coli, kind="stable")+    rowi, coli = rowi[order], coli[order]+    lastX = last.X[:, gidx].tocsc()+    counts = np.zeros(n, dtype=np.int64)+    bounds = np.searchsorted(coli, np.arange(lastX.shape[1] + 1))+    for c in range(lastX.shape[1]):+        lo, hi = bounds[c], bounds[c + 1]+        if lo == hi:+            continue+        r = rowi[lo:hi]+        vals = np.asarray(lastX[:, c].todense()).ravel()+        nb = vals[idx[r]]                       # (m, K) neighbour values+        nb = nb.astype(np.float32)+        nb[nb == 0] = np.nan+        with np.errstate(invalid="ignore"):+            med = np.nanmedian(nb, axis=1) * VAL_FRAC+        bad = ~np.isfinite(med)+        med[bad] = 0.0+        good = ~bad+        X[r[good], gidx[c]] = med[good]+        counts[r[good]] += 1+    return counts++ def main() -> None:     parser = argparse.ArgumentParser()     parser.add_argument("--data", required=True)@@ -71,17 +141,25 @@ def main() -> None:     n = target_n_cells(manifest, last.n_obs)      if prev is None:-        (g_last,) = growth_rates([last], genes, mask, args.seed)-        rows = weighted_rows(g_last ** dt_out if GROWTH else np.ones(last.n_obs), n, rng)+        if GROWTH1:+            (g_last,) = growth_rates([last], genes, mask, args.seed)+            rows = weighted_rows(g_last ** dt_out, n, rng)+        elif n >= last.n_obs:+            rows = np.arange(last.n_obs)+        else:+            rows = weighted_rows(np.ones(last.n_obs), n, rng)+        print(f"single-stage fallback: uniform copy (growth1={GROWTH1}), n={n}", file=sys.stderr)         write_prediction(last.X[rows], genes, args.out, seed=args.seed)         return      import anndata as ad     import pandas as pd     import torch+    from sklearn.neighbors import NearestNeighbors+    from scipy import sparse     from moscot.problems.time import TemporalProblem -    (Zp, Zl), _ = embed([prev.X, last.X], mask, N_HVG, N_PCS, args.seed)+    (Zp, Zl), info = embed([prev.X, last.X], mask, N_HVG, N_PCS, args.seed)     g_prev, g_last = growth_rates([prev, last], genes, mask, args.seed)      obs = pd.DataFrame({"time": np.r_[np.zeros(prev.n_obs), np.ones(last.n_obs)],@@ -91,7 +169,6 @@ def main() -> None:     small = ad.AnnData(X=np.zeros((len(obs), 1), dtype=np.float32), obs=obs)     small.obsm["X_pca"] = np.vstack([Zp, Zl])     tp = TemporalProblem(small)-    # source marginals = WOT prior growth over the input interval (normalised by moscot); target uniform     tp = tp.prepare(time_key="time", joint_attr="X_pca", a="growth")     tp = tp.solve(epsilon=EPSILON, tau_a=TAU_A, tau_b=TAU_B, scale_cost="mean")     sol = tp[(0.0, 1.0)].solution@@ -99,23 +176,65 @@ def main() -> None:     P = np.asarray(sol.transport_matrix, dtype=np.float32)  # (n_prev, n_last)      rows = weighted_rows(g_last ** dt_out if GROWTH else np.ones(last.n_obs), n, rng)-    print(f"diag: growth last min/median/max {g_last.min():.3f}/{np.median(g_last):.3f}/{g_last.max():.3f}", file=sys.stderr)     Pc = P[:, rows]     del P     Pc /= np.maximum(Pc.sum(axis=0, keepdims=True), 1e-30)     factor = LAMBDA * dt_out / dt_in     torch.set_num_threads(N_THREADS)     Xp = torch.from_numpy(prev.X.toarray())-    anc = (torch.from_numpy(Pc).T @ Xp).numpy()  # barycentric ancestor mean, (n, n_genes)-    del Xp, Pc-    smooth = knn_mean(Zl, last.X, rows, K_SMOOTH)-    step = (smooth - anc) * factor+    PcT = np.ascontiguousarray(Pc.T)                        # (n, n_prev)+    anc = (torch.from_numpy(PcT) @ Xp).numpy()              # barycentric ancestor mean, (n, n_genes)+    del Xp++    nn = NearestNeighbors(n_neighbors=K_SMOOTH).fit(Zl)+    idx = nn.kneighbors(Zl[rows], return_distance=False)    # (n, K)+    W = sparse.csr_matrix((np.full(idx.size, 1.0 / K_SMOOTH, dtype=np.float32), idx.ravel(),+                           np.arange(0, idx.size + 1, K_SMOOTH)), shape=(n, Zl.shape[0]))+    smooth = np.asarray((W @ last.X).todense(), dtype=np.float32)++    raw = (smooth - anc) * factor     del smooth, anc-    step[:, ~mask] = 0.0-    X = last.X[rows].toarray()+    raw[:, ~mask] = 0.0++    ct = last.obs["celltype"].astype(str).to_numpy()[rows]+    if K_PROJ > 0:+        K = min(K_PROJ, info["components"].shape[0])+        V = info["components"][:K]                          # (K, N_HVG) orthonormal rows, z-space+        hvg, sd = info["hvg"], info["sd"]+        z = raw[:, hvg] / sd+        z = (z @ V.T) @ V+        step = np.zeros_like(raw)+        step[:, hvg] = z * sd+        step *= ALPHA+        print(f"diag: within-type std ratio proj/raw = {within_type_std(step, ct) / max(within_type_std(raw, ct), 1e-12):.3f}",+              file=sys.stderr)+        top_var = np.argsort(((raw[:, hvg] / sd) @ V.T).var(0))[::-1][:5]+        print(f"diag: top-5 PCs by projected displacement variance: {top_var.tolist()}", file=sys.stderr)+    else:+        step = raw * ALPHA++    Xin = last.X[rows]+    zf_in = 1.0 - Xin.count_nonzero() / (Xin.shape[0] * Xin.shape[1])+    X = Xin.toarray()+    del Xin     step *= X > 0  # "addnz": move only measured (non-zero) entries, keeps each cell's zero pattern     X += step     np.maximum(X, 0.0, out=X)+    del step, raw++    if M_ACT > 0:+        K = min(max(K_PROJ, 1), info["components"].shape[0]) if K_PROJ > 0 else info["components"].shape[0]+        V_K = info["components"][:K]+        loadmax = np.abs(V_K).max(axis=0)                   # (N_HVG,)+        sel = loadmax > LOAD_MIN+        gidx = info["hvg"][sel]+        lm = loadmax[sel].astype(np.float32)+        counts = activate_genes(X, idx, W, PcT, prev, last, gidx, lm, M_ACT)+        print(f"diag: activation candidates={len(gidx)} activated/cell mean={counts.mean():.1f} "+              f"p90={np.percentile(counts, 90):.0f} max={counts.max()}", file=sys.stderr)+    zf_out = float((X == 0).sum()) / X.size+    print(f"diag: zero-fraction in={zf_in:.4f} out={zf_out:.4f} (change {zf_out - zf_in:+.4f})", file=sys.stderr)+     write_prediction(X, genes, args.out, seed=args.seed)  

调研来源?调研员查到并用到的知识条目和文献检索结果(只列标题和编号)。

用到的知识库条目

编号标题出处
k031Offline OT toolkit in the sandbox: moscot TemporalProblem, wot OTModel, POT, geomloss10.1038/s41586-024-08453-2 (moscot); 10.1016/j.cell.2019.01.006 (Waddington-OT)
k002moscot.time: scalable temporal OT with growth rates and low-rank solvers10.1038/s41586-024-08453-2
k001Waddington-OT: unbalanced entropic OT between snapshot time points10.1016/j.cell.2019.01.006

分析结果?分析员写的 ANALYSIS.json:改了什么、各组分数怎么变、假设是否成立、经验和下一步建议。

改了什么在父节点 ot_moscot 上改解码:两阶段路径把 raw 位移投影到前 K=22 个 PCA 程序方向(α=1,无收缩);基因激活已实现但 X3 上有害,默认关(M=0);单阶段回退(proxy10)去掉 g^dt_out 生长重采样,改为均匀复制最后阶段。
各组分数的变化cell_state:变好 +5.09(49.21→54.30),超出噪声,是主要收益:X3 mmd_u 0.03281→0.02794,skill 0.501→0.560,得分 +1.78;proxy10 mmd_u 0.04512→0.04135,skill 0.475→0.509,得分 +1.02(此项来自均匀复制回退,非投影机制)
covariation:噪声内 −0.51(50.70→50.18):X3 variogram 0.001415→0.001504 得分 −0.36,proxy10 +0.41,互有抵消
de_recovery:+1.24(50.79→52.03):proxy10 de_score −0.013→0.042 得分 +0.45,X3 0.039→0.065 得分 +0.24,两把尺子单独看都接近噪声,合计方向一致
direction:变好 +3.03(50.90→53.94):proxy10 de_direction 0.098→0.201,得分 +1.39,来自去掉生长重采样(回退修复,与投影无关);X3 de_direction −0.023→0.023,得分 +0.44,偏小
family_idlocal_ot
假设是否成立是
经验
  1. 在两阶段 OT 位移解码中,把位移投影到 top-K PCA 程序(K∈[15,30],α=1)在 X3 上把 mmd_u skill 从 0.50 提到 0.56(同尺子机制关对照 50.41 vs 机制开 52.78,K 扫描 5 个值全部高于 K=0),说明去除逐基因噪声、保留程序级协同位移对 cell_state 有效;但幅度收缩(α=0.5)在 4 组对照中一致更低,去噪有效不等于缩小位移有效。
  2. 稀疏基因激活(每细胞 50 个零基因置为 30 近邻非零中位数×0.5,检出率增幅阈值 0.15)在 X3 上使四项全面下降(47.15 vs 无激活 50.41,mmd_u skill 0.50→0.45):给零值基因填正值直接破坏细胞分布,与'把零值整体抬成小正数损害 mmd_u/variogram'的已知事实一致。
  3. 单输入阶段时,基于增殖/凋亡评分的 g^dt_out 生长重采样是纯噪声源:改为均匀复制使 proxy10 从 49.67 涨到 52.94(+3.27),de_direction、mmd_u 同时上升;没有两阶段差信息就不要用签名评分做重加权。
  4. PLAN 的机制证据线'型内位移 std ≥ 无约束版 20%'过严:实际 K=22 时型内 std 比例仅约 14%(K=15 时 6%)仍拿到 X3 +2.4,评分奖励的是分布层面的去噪,不要求保留大部分型内异质性。
  5. 本节点榜分 +2.49 中约六成来自 proxy10 的回退修复(+3.27,单阶段路径,投影机制在该尺子上根本没运行),X3 的投影收益 +2.10 接近 2 分噪声,仅靠机制关对照(+2.37)和 K 扫描一致性支持;读多尺子总分时要区分哪把尺子在测哪个机制。
mechanism_active是
下一步建议
  1. 针对 cell_state/direction:把本节点的 PCA 投影解码(K=22, α=1)与节点 3 的组成趋势重采样组合——组成改动负责细胞类型比例(mmd_u、de_direction),投影负责型内位移去噪,两者作用正交;在 X3 和 final 两把尺子上分别验证叠加性。
  2. 针对 final 视图(cell_state、covariation):先单跑投影解码验证 dt_out/dt_in=1 时收益是否保留(X3 的 factor=2,final 的 factor=1,位移幅度减半可能使去噪收益缩水),并在 final 上重扫 K∈{15,22,30},不要直接沿用 X3 选的 K=22。
  3. 放弃或大幅收紧基因激活方向:若再试,用 M≤10 且 DET_THRESH≥0.3,并只允许激活在 top-K PC 上高载荷的基因;以 mmd_u 不降为硬门槛,否则不做。

对话摘要?每个角色和大模型对话的统计:轮数、工具调用、用时、token 数和最后的回答摘录;原始记录只给路径。

只给统计和最后回答的摘录;完整对话请到原始记录位置里列出的文件看。

分析员

角色?调研员写计划、工程师改代码、分析员解读分数、审查员检查作弊。分析员 alibaba-token-plan-cn/qwen3.8-max
调用次数1 次
轮数?大模型一共回复了几轮(每轮可以调用多个工具)。3
工具调用?大模型调用读文件、执行命令等工具的次数,按工具名统计。共 2 次:bash 1、write 1
用时不到 1 分
token 数?输入 / 输出(含思考)的 token 数,来自接口返回的用量。输入 18,142 · 输出 1,430 · 思考 1,396
任务(第一行)分析节点 n5 的打分结果,写 ANALYSIS.json
最后的回答(摘录)
ANALYSIS.json written
原始记录位置?完整对话记录在 Spark 上的文件路径;看板不复制原始内容。/home/spark-longxinyang/vec/runs/formal/20261003-093415-search-t1-r2-D-s1/nodes/5/analyst.jsonl 7 KB
/home/spark-longxinyang/vec/runs/formal/20261003-093415-search-t1-r2-D-s1/nodes/5/analyst.stderr

工程师

角色工程师 alibaba-token-plan-cn/qwen3.8-max
调用次数1 次
轮数29
工具调用共 34 次:bash 26、edit 4、read 2、write 2
用时17 分
token 数输入 63,847 · 输出 11,142 · 思考 13,374
任务(第一行)按计划修改程序:local_ot: PCA-program-projected displacement with shrinkage + capped gene activation
最后的回答(摘录)
**做了什么**:在父节点 ot_moscot 上改解码(improve, PLAN local_ot):① 位移投影到前 K 个 PCA 程序方向(提交 K=22, α=1),② 基因激活已实现但 X3 上明显有害(−3~−5 分),默认 M=0,③ 单阶段回退去掉生长重采样改均匀复制。机制对照(K=0,M=0≈父解码 50.41 → 机制开 52.78)确认收益来自投影,主要在 mmd_u。

**查分**(A 半,用 15/20):X3 = **52.78**(父 50.65;K∈[15,30] 全部 >51.4,K=0 为 50.41,α=0.5 一致更低);proxy10 = **53.19**(父 49.67,≈copy_last)。预计节点分 ≈52.9 vs 父 50.33。seed 0 复跑逐字节一致,两视图 vec-check 通过。

**下一步最值得试**:final 视图上验证投影解码(dt_out/dt_in=1,X3 的 +2.4 未必等幅迁移);以及把节点 3 的组成趋势重加权与本投影解码组合(两者收益方向正交:组成→mmd/direction,投影→去噪)。
原始记录位置/home/spark-longxinyang/vec/runs/formal/20261003-093415-search-t1-r2-D-s1/nodes/5/engineer.jsonl 167 KB
/home/spark-longxinyang/vec/runs/formal/20261003-093415-search-t1-r2-D-s1/nodes/5/engineer.stderr

调研员

角色调研员 alibaba-token-plan-cn/qwen3.8-max
调用次数1 次
轮数1
工具调用共 0 次:—
用时1 分
token 数输入 15,175 · 输出 1,476 · 思考 2,848
任务(第一行)为节点 n5(改进,父节点 n2)写改进计划 PLAN.json
最后的回答(摘录)
too high → same as parent. Check de_direction and de_score to confirm directional signal preserved. 4. 30 min budget: PCA projection is cheap (matrix multiply); gene activation loop vectorized over cells. Total added compute <1 min. If moscot solve is the bottleneck, subsample to 3000 cells per stage.",
 "family_id": "local_ot",
 "mechanism": "OT coupling estimates per-cell ancestor distributions; displacement is projected onto top-K PCA programs with shrinkage factor α (removing gene-level noise), and module-supported genes are sparsely activated based on local detection-rate increases in coupled descendants.",
 "vs_constant_shift": "Each cell's displacement comes from its own coupling-weighted ancestor mean projected through data-driven PCA programs; the direction and magnitude vary per cell based on its local OT coupling, not a per-type constant vector. Gene activation is cell-specific (depends on its neighbours' detection rates). Report within-type displacement std to prove heterogeneity.",
 "mechanism_evidence": "1. Within-type std of displacement vectors: should be >20% of the unconstrained version (not collapsed to per-type constant). 2. Number of activated genes per cell (expect 10–50) and their top-K PC loading distribution. 3. Four metric groups separately: mmd_u/variogram should improve from shrinkage; de_direction/de_score should improve from activation. 4. Compare zero-fraction of output vs input (should change by <2%). 5. Print top-5 PCs by projected displacement variance to verify programs are coherent.",
 "mechanism_off_control": "Run same code with α=1.0, K=full rank (no projection), M=0 (no activation): this recovers the parent's unconstrained addnz displacement exactly. Additionally run with GROWTH=False on single-stage (uniform copy). If output with mechanism off equals parent output, mechanism was not running. Expected difference: mechanism-on should show lower mmd_u, higher de_direction, and non-zero activated gene counts.",
 "sources": []}
```
原始记录位置/home/spark-longxinyang/vec/runs/formal/20261003-093415-search-t1-r2-D-s1/nodes/5/researcher.jsonl 7 KB
/home/spark-longxinyang/vec/runs/formal/20261003-093415-search-t1-r2-D-s1/nodes/5/researcher.stderr