总览 · ← 返回运行 20261003-093415-search-t1-r2-D-s1
节点 n5
OT 位移解码改为投影到前 22 个 PCA 程序方向(α=1,无基因激活),单输入阶段回退改为均匀复制(去掉生长重采样)。
| 运行?一次完整的自动搜索或 Agent 会话,有自己的锁定配置和证据包。 | 20261003-093415-search-t1-r2-D-s1 |
|---|---|
| 父节点 | n2 |
| 子节点 | — |
| 操作?种子:人写的起点;改进:在父节点上改;草稿:从头写;修复:修父节点的报错。 | 改进 |
| 状态 | 已打分 |
| 分数 | 搜索目标分 52.82(+2.5) · X3 52.76(+2.1) · proxy10 52.94(+3.3) · 3 次复测均分 51.44 |
| 审查 | 未审查 |
| 用时?从运行开始到结束(或到现在)的挂钟时间。 | 20 分 |
| 程序版本 | a2b75251d98408641cac8da351017aef2f36e74d (programs.git) |
方法说明?节点程序自带的 METHOD.md:这个程序做了什么、为什么。
来自 programs.git a2b75251d9:solution/METHOD.md
OT 位移解码改为投影到前 22 个 PCA 程序方向(α=1,无基因激活),单输入阶段回退改为均匀复制(去掉生长重采样)。
local_ot: program-projected displacement (improve over node 2 ot_moscot)
方法
两个输入阶段的路径与父节点完全相同(joint HVG/PCA → moscot TemporalProblem 耦合 → g^dt_out 重采样输出细胞 → raw_step = λ·dt_out/dt_in·(kNN 平滑位置 − 耦合加权祖先均值),λ=1,addnz 解码:step 只加到该细胞的非零项、夹到 ≥0)。 唯一改动在解码表示:
- PCA 程序投影(机制,默认开):把 raw_step 换到标准化 HVG 空间(除以 embed 已拟合的 sd),投影到前
K_PROJ=22个 PCA 载荷张成的子空间,再变回基因空间:step = ((raw[:,hvg]/sd) @ V_K.T @ V_K) * sd; 非 HVG 基因的位移置 0。ALPHA=1.0(收缩系数保留为开关,数据选了 1)。逐基因噪声被去掉,只保留数据驱动的 程序级协同位移;每细胞的位移仍由它自己的 OT 耦合与局部邻域决定(非常数位移:投影后型内位移 std 为无约束 版本的 13.9%,K=15 时 6.1%,见下方证据与讨论)。 - 基因激活(已实现,默认关 M_ACT=0):按 PLAN 实现了"邻域检出率 − 祖先检出率 ≥ 0.15 且 |top-K 载荷| > 0.02 的零基因,按增幅×载荷排序取前 M 个,置为 30 近邻非零中位数 × 0.5"。X3 上 M=50 使四项全面下降 (47.15 vs 同解码无激活 50.41;mmd_u skill 0.50→0.45),故提交默认 M=0。
- 单阶段回退修复:只有一个输入阶段(proxy)时不再做 g^dt_out 生长重采样(父节点因此比 copy_last 低约 3 分), 改为均匀复制最后阶段(n = clip(n_obs, min_cells, max_cells);n ≥ n_obs 时按原序全取)。单阶段没有任何两阶段差 信息可用,这是诚实回退。
机制开关(环境变量,仅为跑对照,提交默认=机制开):N5_ALPHA / N5_KPROJ(0=不投影) / N5_MACT / N5_GROWTH1
(1=恢复父节点单阶段生长重采样)。
机制对照(PLAN mechanism_off_control,X3 视图,seed 0,A 半查分)
| 配置 | X3 榜分 | de_score | de_dir | mmd_u | variogram |
|---|---|---|---|---|---|
| 机制关(α=1, K=0, M=0 ≈ 父节点解码) | 50.41 | 0.500 | 0.490 | 0.499 | 0.534 |
| 机制开(α=1, K=22, M=0,提交默认) | 52.78 | 0.515 | 0.514 | 0.562 | 0.510 |
| α=0.5, K=15, M=50(PLAN 初始默认) | 47.2 | 0.449 | 0.512 | 0.490 | 0.424 |
| α=1, K=0, M=50(只激活) | 47.15 | 0.465 | 0.490 | 0.452 | 0.487 |
| K 扫描(α=1):K=10 / 15 / 20 / 25 / 30 | 51.41 / 52.51 / 52.80 / 52.85 / 52.40 | ||||
| α 扫描(K=15/20):α=0.5 | 51.86 / 51.78 |
机制生效证据:机制开与关的输出不同(mmd_u skill 0.499→0.562 为主要收益,de 两项同向上升);投影确实改变细胞 (型内位移 std 为无约束版的 6–14%,即保留了部分而非全部型内异质性;输出零占比变化 ≤ 0.01%)。收益主体在 cell_state(mmd_u),与 PLAN 预期一致。K=15 时型内 std 比例 6.1% 低于 PLAN 风险线 20%,按风险预案增大 K; K∈[15,30] 分数均高于 K=0,K=22 取在 20/25 之间。α=0.5 一致地低于 α=1(4 组对照同向),故收缩关闭。
proxy10(单阶段回退,均匀复制):53.19(父节点 49.67;同流程 copy_last 参考 52.75——回退路径输出即全量复制, 差值来自评分抽样)。预计节点分 ≈ (53.19 + 2×52.78)/3 ≈ 52.9 vs 父 50.33。
已验证 / 未验证
- 已验证:X3(两外部阶段,dt_out/dt_in=2)上投影解码优于父解码约 2.4 分(A 半,噪声约 2 分,方向与 5 个 K 值 的一致性支持它);proxy 单阶段路径跑通、vec-check 通过;seed 0 复跑逐字节一致,seed 1 正常;X3 652 细胞 ~19 s / 内存小(<2 GB 级别),proxy ~2 s。
- 未验证:final 视图(E8.5+E9.5→E10.5,16.8k×17.1k 耦合)未跑(不在本节点视图里);父节点在 final 上 69 s / 6.7 GB,本改动只增加 (n×2000) 矩阵乘,激活路径默认关,预计资源相近。dt_out/dt_in=1(final)时 factor=1, 投影幅度更小,方向性收益按 X3 外推但幅度未测。生长重采样(两阶段路径的 GROWTH=True)保持不变,其在 final 上的效果未测(父节点证据:X3/proxy2 上无影响)。基因激活只测了 M=50(阈值 0.15、值系数 0.5),更小的 M 未测。
- 视图无关性:输出只依赖视图数据、时间差(dt_in/dt_out 平移不变)与 seed;无绝对时间、路径、manifest 排版依赖。
知识来源
与父节点相同:Waddington-OT(Schiebinger et al., Cell 2019, doi:10.1016/j.cell.2019.01.006)、moscot TemporalProblem(Klein et al., Nature 2025, doi:10.1038/s41586-024-08453-2)、moscot 内置小鼠增殖/凋亡基因表 (阶段无关的基因功能注释)。新增部分(PCA 程序投影)不引入任何外部生物学知识:投影方向完全由两个输入阶段 自己的数据拟合。未使用保留阶段/保留基因型的任何信息。
调研员的计划
| 名称 | local_ot: PCA-program-projected displacement with shrinkage + capped gene activation |
|---|---|
| 动机 | Node 2 cell_state=49.21 is the weakest group; proxy10 mmd_u skill=0.475, variogram skill=0.470 are below floor. The addnz decode applies unconstrained displacement in full gene space (noise) and cannot activate new genes. On proxy10, growth resampling scores 49.67 vs copy_last 52.75 (−3.08). Node 3 (composition_trend, 53.69) shows composition changes help; node 2's unconstrained gene-space step likely degrades distribution structure. Direction library priority: fix 'genes can't turn on' and 'displacement unconstrained'. |
| 做法 | Modify the decode step in three ways, keeping the OT coupling (steps 1–3) unchanged: 1. Program-constrained displacement: After computing raw_step = (smooth − anc) × factor, project onto top K=15 PCA loadings (from the already-fitted PCA): step_proj = (raw_step @ V_K) @ V_K.T. Apply shrinkage: step_final = α × step_proj, with α initial 0.5, search [0.3, 0.8]. This removes gene-level noise while preserving coherent program-level movement. K and α are the only new hyperparameters. 2. Capped gene activation: For each output cell j, identify candidate genes where: (a) expression is 0 in cell j, (b) the gene's detection rate in the 30 nearest later-stage neighbours (already computed for smooth) exceeds the detection rate in the cell's ancestor distribution by ≥0.15, (c) the gene has |loading| > 0.02 in at least one of the top K PCs. Activate the top M=50 candidates (ranked by detection-rate increase × max |loading|), setting each to the median non-zero value among those 30 neighbours × 0.5. This is sparse, module-supported, and preserves the cell's overall sparsity pattern for the vast majority of genes. 3. Single-stage fallback fix: When only one input stage exists (pro… |
| 风险 | 1. PCA projection may over-smooth real heterogeneous displacement, losing cell-type-specific signals → check within-type displacement std; if <20% of unconstrained, increase K. 2. Gene activation may introduce artifacts if detection-rate threshold too low → monitor zero-fraction change; should stay within ±2% of input. 3. α too low → displacement vanishes, output ≈ copy_last (score ~50); α too high → same as parent. Check de_direction and de_score to confirm directional signal preserved. 4. 30 min budget: PCA projection is cheap (matrix multiply); gene activation loop vectorized over cells. Total added compute <1 min. If moscot solve is the bottleneck, subsample to 3000 cells per stage. |
代码改动?这个节点的程序和父节点程序的逐行差别:绿色是新增,红色是删除。
对比:父节点版本 25a98fa4a1。改动的文件:solution/METHOD.md +62 −81、solution/run.py +151 −32
diff --git a/solution/METHOD.md b/solution/METHOD.mdindex ca7a3d2..09a9e7b 100644--- a/solution/METHOD.md+++ b/solution/METHOD.md@@ -1,81 +1,62 @@-# ot_moscot — Waddington-OT / moscot coupling, one-step displacement extrapolation--Seed (2026-10-02) made from the G37 candidate `modeling/candidates/T1/ot_moscot/` (commit 227eeb2). Same method and-hyper-parameters. Changes for the seed contract only: the dev-only environment overrides (`G37_LAMBDA`, `G37_DECODE`,-`G37_GROWTH`, `G37_JAX_GPU`) and the unused `knn` decode branch are removed; JAX forced to CPU, torch threads fixed-at 8 (`EXECUTION.json {"gpu": false}`); `stage_pair` reads every input stage of the view the same way (no reference to-manifest `mode` / `source`). Output depends only on the view's data, the time differences between stages and `--seed`.--Contract: `python run.py --data <view> --out <pred.h5ad> --seed <int>`; `g37_common.py` must stay next to `run.py`.--## Method--Two input stages `prev` (time t0) and `last` (t1); target time t2 (final view: E8.5, E9.5 -> E10.5).--1. **Embedding.** Genes measured in both stages; top 2000 by variance (both stages pooled); z-score, clip at 10;- PCA, 30 components (randomized, `random_state=seed`). Fitted on the input stages only.-2. **Growth prior (Waddington-OT).** Proliferation / apoptosis scores (`scanpy.tl.score_genes`, moscot's mouse- gene lists) -> birth = generalised logistic(prolif; 1.7, 0.3, 0.25, 0.5), death = logistic(apopt; 1.7, 0.3, 0.1,- 0.2), per-day growth g = exp(birth - death) (Schiebinger 2019; same formula and defaults as moscot's- `BirthDeathProblem.estimate_marginals`).-3. **Coupling.** `moscot.problems.time.TemporalProblem` prev -> last on the PCA (`joint_attr="X_pca"`),- source marginal ∝ g^(t1-t0), target uniform; entropic unbalanced Sinkhorn, `epsilon=1e-3`, `tau_a=0.95`,- `tau_b=1`, `scale_cost="mean"` (moscot tutorial settings). JAX on CPU unless `G37_JAX_GPU=1`.-4. **Output cells.** n = number of latest-stage cells clipped to `[min_cells, max_cells]` (as copy_last),- drawn without replacement from the latest stage with probability ∝ g^(t2-t1) (the WOT birth-death model- continued over the target interval).-5. **Displacement extrapolation.** For output cell j: ancestor mean a_j = Σ_i π_ij x_i / Σ_i π_ij (barycentric- projection of the coupling, gene space, all panel genes); smoothed position s_j = mean expression of its 30- nearest latest-stage cells in the PCA; step_j = λ (t2-t1)/(t1-t0) (s_j - a_j), λ = 1 (continue the last- observed displacement at the same rate). The step is added **to the cell's non-zero entries only** and clipped- at 0; genes not measured in both stages (external stages) get no step. Each cell keeps its own residual.--**One input stage (proxy view, E8.5 only):** steps 1, 3, 5 need two stages; only steps 2 + 4 run, i.e. a-growth-weighted copy of the latest stage (g^(t2-t1) resampling). This is the only thing the proxy can test.--Sources:-- Schiebinger G. et al. Optimal-transport analysis of single-cell gene expression identifies developmental- trajectories in reprogramming. *Cell* 176, 928–943 (2019). doi:10.1016/j.cell.2019.01.006 (WOT: unbalanced- entropic OT between snapshots, growth from proliferation/apoptosis signatures, birth-death logistic).-- Klein D., Palla G., Lange M. et al. Mapping cells through time and space with moscot. *Nature* 638, 1065–1075- (2025). doi:10.1038/s41586-024-08453-2 (TemporalProblem; code moscot 0.5.2, BSD-3).-- Cuturi M. Sinkhorn distances. NeurIPS 2013; Chizat L. et al. Scaling algorithms for unbalanced optimal- transport problems. *Math. Comp.* 87, 2563–2609 (2018).-- Extrapolating the barycentric displacement one more step is our use of the coupling (WOT/moscot interpolate,- they do not extrapolate); listed in agent/knowledge/T1_methods_landscape.md §1.--## Data / knowledge used--Only the view's input stages. Generic knowledge: moscot's built-in mouse proliferation (97) and apoptosis (193)-gene lists (`moscot.utils.data`, from the WOT paper; stage-agnostic gene-function annotation). No held-out stage,-no information from (E9.5, E13.5], no pre-trained weights.--## Hyper-parameters--| Name | Value | Where it came from |-|---|---|---|-| `N_HVG`, `N_PCS`, `K_SMOOTH` | 2000, 30, 30 | a priori; not tuned |-| `EPSILON`, `TAU_A`, `TAU_B`, `scale_cost` | 1e-3, 0.95, 1, mean | a priori (moscot tutorial settings) |-| growth prior | on | a priori (WOT / moscot defaults); G37 saw it cost ~3 points on the old proxy, kept on |-| `LAMBDA` | 1 | a priori (continue the observed displacement at the same rate); G37 also ran 0.5 on X3 (50.4 vs 50.2), not changed |-| decode `addnz` | — | **chosen on the X3 ruler (G37)** against `add` (25.6) and `knn` (48.7): the dense step destroys the zero pattern. Also a first-principles choice (the scorer compares sparse log-expression), but the evidence that picked it was X3 |--No re-tuning for the seed.--## Resources (Spark, CPU)--Final view (16.8k x 17.1k coupling): 69 s, max RSS 6.7 GB; proxy 4 s / 1.6 GB; X3 ~ 8 s / 2.3 GB; proxy2 ~ 20 s / 4.3 GB.-The dense coupling (n_prev x n_last float32, ~1.1 GB on final) and the dense earlier stage (~2.2 GB) dominate memory.--## Findings (G37, local scorer: fast engine, truth half B, scorer seed = program seed)--- Dense gene-space step (`add`) is destructive: X3 25.6 at λ=1, still 32.9 at λ=0.25 (cell_state, covariation- collapse) — the step makes every zero slightly positive. `addnz` fixes it (X3 50.2 at λ=1, 50.4 at λ=0.5);- `knn` decode 48.7 (λ=1). Growth resampling has no effect on X3/proxy2 (all cells are kept there).-- proxy2 (E8.5 official -> Qiu E9.0 heart, then +0.5 d): every variant ≈ copy of the Qiu cells (~27.5), the- cross-dataset step is batch effect.-- Full eval (seeds 0-2, half B): proxy 46.8 / 46.8 / 47.9 (copy_last 50.0 / 50.3 / 50.1) — the WOT growth- resampling alone costs ~3 points on E8.5 -> E9.5 (direction, cell_state); X3 50.2 / 50.1 / 49.8 (copy_last 50.0,- pseudobulk_shift 40.5-40.7); proxy2 27.7-27.9 (copy_last 27.4-27.6). `G37_GROWTH=0` turns the proxy into copy_last.-- Final-view prediction (`~/vec/scratch/g37/ot_moscot/final.h5ad`, seed 0): 5118 cells, 21 nearest-E9.5 types,- composition within ±2 % of E9.5 (OFT/RV-CM -2.1 %); no new states (the method cannot create them).+OT 位移解码改为投影到前 22 个 PCA 程序方向(α=1,无基因激活),单输入阶段回退改为均匀复制(去掉生长重采样)。++# local_ot: program-projected displacement (improve over node 2 ot_moscot)++## 方法++两个输入阶段的路径与父节点完全相同(joint HVG/PCA → moscot TemporalProblem 耦合 → g^dt_out 重采样输出细胞 →+raw_step = λ·dt_out/dt_in·(kNN 平滑位置 − 耦合加权祖先均值),λ=1,addnz 解码:step 只加到该细胞的非零项、夹到 ≥0)。+唯一改动在**解码表示**:++1. **PCA 程序投影(机制,默认开)**:把 raw_step 换到标准化 HVG 空间(除以 embed 已拟合的 sd),投影到前+ `K_PROJ=22` 个 PCA 载荷张成的子空间,再变回基因空间:`step = ((raw[:,hvg]/sd) @ V_K.T @ V_K) * sd`;+ 非 HVG 基因的位移置 0。`ALPHA=1.0`(收缩系数保留为开关,数据选了 1)。逐基因噪声被去掉,只保留数据驱动的+ 程序级协同位移;每细胞的位移仍由它自己的 OT 耦合与局部邻域决定(非常数位移:投影后型内位移 std 为无约束+ 版本的 13.9%,K=15 时 6.1%,见下方证据与讨论)。+2. **基因激活(已实现,默认关 M_ACT=0)**:按 PLAN 实现了"邻域检出率 − 祖先检出率 ≥ 0.15 且 |top-K 载荷| > 0.02+ 的零基因,按增幅×载荷排序取前 M 个,置为 30 近邻非零中位数 × 0.5"。X3 上 M=50 使四项全面下降+ (47.15 vs 同解码无激活 50.41;mmd_u skill 0.50→0.45),故提交默认 M=0。+3. **单阶段回退修复**:只有一个输入阶段(proxy)时不再做 g^dt_out 生长重采样(父节点因此比 copy_last 低约 3 分),+ 改为均匀复制最后阶段(n = clip(n_obs, min_cells, max_cells);n ≥ n_obs 时按原序全取)。单阶段没有任何两阶段差+ 信息可用,这是诚实回退。++机制开关(环境变量,仅为跑对照,提交默认=机制开):`N5_ALPHA` / `N5_KPROJ`(0=不投影) / `N5_MACT` / `N5_GROWTH1`+(1=恢复父节点单阶段生长重采样)。++## 机制对照(PLAN mechanism_off_control,X3 视图,seed 0,A 半查分)++| 配置 | X3 榜分 | de_score | de_dir | mmd_u | variogram |+|---|---|---|---|---|---|+| 机制关(α=1, K=0, M=0 ≈ 父节点解码) | 50.41 | 0.500 | 0.490 | 0.499 | 0.534 |+| 机制开(α=1, K=22, M=0,提交默认) | **52.78** | 0.515 | 0.514 | **0.562** | 0.510 |+| α=0.5, K=15, M=50(PLAN 初始默认) | 47.2 | 0.449 | 0.512 | 0.490 | 0.424 |+| α=1, K=0, M=50(只激活) | 47.15 | 0.465 | 0.490 | 0.452 | 0.487 |+| K 扫描(α=1):K=10 / 15 / 20 / 25 / 30 | 51.41 / 52.51 / 52.80 / 52.85 / 52.40 | | | | |+| α 扫描(K=15/20):α=0.5 | 51.86 / 51.78 | | | | |++机制生效证据:机制开与关的输出不同(mmd_u skill 0.499→0.562 为主要收益,de 两项同向上升);投影确实改变细胞+(型内位移 std 为无约束版的 6–14%,即保留了部分而非全部型内异质性;输出零占比变化 ≤ 0.01%)。收益主体在+cell_state(mmd_u),与 PLAN 预期一致。K=15 时型内 std 比例 6.1% 低于 PLAN 风险线 20%,按风险预案增大 K;+K∈[15,30] 分数均高于 K=0,K=22 取在 20/25 之间。α=0.5 一致地低于 α=1(4 组对照同向),故收缩关闭。++proxy10(单阶段回退,均匀复制):53.19(父节点 49.67;同流程 copy_last 参考 52.75——回退路径输出即全量复制,+差值来自评分抽样)。预计节点分 ≈ (53.19 + 2×52.78)/3 ≈ 52.9 vs 父 50.33。++## 已验证 / 未验证++- 已验证:X3(两外部阶段,dt_out/dt_in=2)上投影解码优于父解码约 2.4 分(A 半,噪声约 2 分,方向与 5 个 K 值+ 的一致性支持它);proxy 单阶段路径跑通、vec-check 通过;seed 0 复跑逐字节一致,seed 1 正常;X3 652 细胞+ ~19 s / 内存小(<2 GB 级别),proxy ~2 s。+- 未验证:final 视图(E8.5+E9.5→E10.5,16.8k×17.1k 耦合)未跑(不在本节点视图里);父节点在 final 上 69 s /+ 6.7 GB,本改动只增加 (n×2000) 矩阵乘,激活路径默认关,预计资源相近。dt_out/dt_in=1(final)时 factor=1,+ 投影幅度更小,方向性收益按 X3 外推但幅度未测。生长重采样(两阶段路径的 GROWTH=True)保持不变,其在 final+ 上的效果未测(父节点证据:X3/proxy2 上无影响)。基因激活只测了 M=50(阈值 0.15、值系数 0.5),更小的 M+ 未测。+- 视图无关性:输出只依赖视图数据、时间差(dt_in/dt_out 平移不变)与 seed;无绝对时间、路径、manifest 排版依赖。++## 知识来源++与父节点相同:Waddington-OT(Schiebinger et al., Cell 2019, doi:10.1016/j.cell.2019.01.006)、moscot+TemporalProblem(Klein et al., Nature 2025, doi:10.1038/s41586-024-08453-2)、moscot 内置小鼠增殖/凋亡基因表+(阶段无关的基因功能注释)。新增部分(PCA 程序投影)不引入任何外部生物学知识:投影方向完全由两个输入阶段+自己的数据拟合。未使用保留阶段/保留基因型的任何信息。diff --git a/solution/run.py b/solution/run.pyindex 1411cb7..ce49718 100644--- a/solution/run.py+++ b/solution/run.py@@ -1,25 +1,25 @@ #!/usr/bin/env python3-"""ot_moscot: Waddington-OT / moscot TemporalProblem coupling, extrapolated one step past the latest input stage.+"""local_ot: PCA-program-projected displacement with shrinkage + capped gene activation (improve over ot_moscot). -Two input stages (final: E8.5, E9.5):+Two input stages (X3: Qiu E8.75, E9.0 -> E9.5; final: E8.5, E9.5 -> E10.5): 1. joint PCA of both stages (HVG, z-score, 30 PCs; fitted on the inputs only);- 2. moscot TemporalProblem prev -> last on the PCA, source marginals from Waddington-OT prior growth- (proliferation / apoptosis gene scores), entropic unbalanced Sinkhorn (epsilon 1e-3, tau_a 0.95);- 3. output cells = latest-stage cells resampled with weights g^dt_out (prior growth rate continued for the target- interval, Waddington-OT birth-death model);- 4. each output cell j moves by LAMBDA * dt_out / dt_in * (kNN-mean(x_j) - ancestor_mean_j) in gene space, where- ancestor_mean_j is the coupling-weighted (barycentric) mean of its ancestors in the earlier stage: the last- observed displacement continued for the target interval; the cell keeps its own residual. The step is applied- to the cell's non-zero entries only (DECODE "addnz": a dense step turns every zero into a small positive value- and wrecks cell_state / covariation, see METHOD.md). Clipped at 0.-One input stage (proxy: E8.5 only): steps 1, 2, 4 need two stages; only the growth resampling (3) runs, i.e.-a growth-weighted copy of the latest stage.--Seed version (agent/seeds/T1__val/ot_moscot, 2026-10-02) of modeling/candidates/T1/ot_moscot (G37): same method and-hyper-parameters; the dev-only environment overrides (G37_LAMBDA / G37_DECODE / G37_GROWTH / G37_JAX_GPU) and the-`knn` decode branch are removed; JAX and torch on CPU, fixed thread count (EXECUTION.json gpu false).-Parameter provenance (METHOD.md): DECODE = addnz was chosen on the X3 ruler (G37); LAMBDA = 1 a priori, also checked-on X3 (0.5 vs 1 within 0.2).+ 2. moscot TemporalProblem prev -> last on the PCA, WOT growth source marginals, entropic unbalanced Sinkhorn;+ 3. output cells = latest-stage cells resampled with weights g^dt_out;+ 4. raw displacement raw_step = LAMBDA * dt_out/dt_in * (kNN-smoothed position - coupling-weighted ancestor mean);+ NEW: projected onto the top K_PROJ PCA programs in the standardized HVG space (removes gene-level noise, keeps+ coherent program-level movement); ALPHA shrinkage kept as a switch, data chose ALPHA=1;+ 5. decode "addnz": the step moves only the cell's non-zero entries, clipped at 0;+ 6. capped gene activation (implemented, default OFF M_ACT=0: on X3 it lowered all four metric groups, see+ METHOD.md): up to M_ACT genes per cell that are 0 in the cell but whose detection rate in its 30 nearest+ latest-stage neighbours exceeds the detection rate in its OT ancestor distribution by >= DET_THRESH,+ restricted to genes with |loading| > LOAD_MIN in one of the top K PCs; value = median non-zero neighbour+ value * VAL_FRAC.+One input stage (proxy: E8.5 only): OT needs two stages; honest fallback = uniform copy of the latest stage+(NEW: the parent's growth-weighted resampling is removed here - it cost ~3 points on proxy10 with no signal to+estimate growth from a single stage).++Mechanism-off control (recovers the parent's decode): N5_ALPHA=1 N5_KPROJ=0 N5_MACT=0; single-stage growth+resampling restored with N5_GROWTH1=1. Submitted defaults keep the mechanism ON. """ from __future__ import annotations@@ -36,7 +36,7 @@ os.environ["JAX_PLATFORMS"] = "cpu" # CPU only: deterministic, no GPU slot import numpy as np sys.path.insert(0, str(Path(__file__).resolve().parent))-from g37_common import embed, growth_rates, knn_mean, stage_pair # noqa: E402+from g37_common import embed, growth_rates, stage_pair # noqa: E402 from src.task1_temporal.view_io import load_manifest, panel_genes, target_n_cells, write_prediction # noqa: E402 @@ -47,15 +47,85 @@ TAU_A = 0.95 TAU_B = 1.0 K_SMOOTH = 30 LAMBDA = 1.0 # 1 = continue the observed displacement at full rate-GROWTH = True # resample output cells by g^dt_out (WOT birth-death model)+GROWTH = True # two-stage path: resample output cells by g^dt_out (WOT birth-death model) N_THREADS = 8 +# --- mechanism switches (defaults ON; env overrides are for the mechanism-off control only) ---+ALPHA = float(os.environ.get("N5_ALPHA", "1.0")) # shrinkage of the program-projected step+K_PROJ = int(os.environ.get("N5_KPROJ", "22")) # top-K PCA programs; 0 = no projection (control)+M_ACT = int(os.environ.get("N5_MACT", "0")) # activated genes per cell; 0 = off (X3: harmful)+GROWTH1 = os.environ.get("N5_GROWTH1", "0") == "1" # single-stage fallback: growth resampling (parent) if 1+DET_THRESH = float(os.environ.get("N5_DET", "0.15")) # detection-rate increase required for activation+LOAD_MIN = float(os.environ.get("N5_LOAD", "0.02")) # |PC loading| required for activation candidates+VAL_FRAC = float(os.environ.get("N5_VALFRAC", "0.5")) # activated value = median non-zero neighbour value * this+ def weighted_rows(w: np.ndarray, n: int, rng: np.random.Generator) -> np.ndarray: p = w / w.sum() return np.sort(rng.choice(len(w), size=n, replace=n > len(w), p=p)) +def within_type_std(S: np.ndarray, ct: np.ndarray) -> float:+ tot = 0.0+ for t in np.unique(ct):+ A = S[ct == t]+ if len(A) > 1:+ tot += float(((A - A.mean(0)) ** 2).sum())+ return float(np.sqrt(tot / max(S.size, 1)))+++def activate_genes(X, idx, W, PcT, prev, last, gidx, lm, M, rng_free=None):+ """Capped gene activation (PLAN step 2). X: dense (n, n_genes) output being built; idx: (n, K) neighbour+ indices into last; W: (n, n_last) kNN mean operator; PcT: (n, n_prev) normalised coupling; gidx: candidate+ panel-gene indices; lm: their max |loading| over the top-K PCs. Returns per-cell activation counts."""+ from scipy import sparse+ import torch++ n = X.shape[0]+ Bl = last.X[:, gidx].copy()+ Bl.data[:] = 1.0+ det_near = np.asarray((W @ Bl).todense(), dtype=np.float32) # (n, G)+ Bp = prev.X[:, gidx].toarray()+ Bp = Bp.astype(np.float32)+ Bp[Bp > 0] = 1.0+ torch.set_num_threads(N_THREADS)+ anc_det = (torch.from_numpy(PcT) @ torch.from_numpy(Bp)).numpy() # (n, G)+ del Bp+ increase = det_near - anc_det+ elig = (increase >= DET_THRESH) & (X[:, gidx] == 0)+ score = np.where(elig, increase * lm[None, :], -np.inf).astype(np.float32)+ if score.shape[1] > M:+ part = np.argpartition(-score, M - 1, axis=1)[:, :M]+ else:+ part = np.broadcast_to(np.arange(score.shape[1]), (n, score.shape[1]))+ rowi = np.repeat(np.arange(n, dtype=np.int64), part.shape[1])+ coli = np.asarray(part).ravel().astype(np.int64)+ keep = np.isfinite(score[rowi, coli])+ rowi, coli = rowi[keep], coli[keep]+ order = np.argsort(coli, kind="stable")+ rowi, coli = rowi[order], coli[order]+ lastX = last.X[:, gidx].tocsc()+ counts = np.zeros(n, dtype=np.int64)+ bounds = np.searchsorted(coli, np.arange(lastX.shape[1] + 1))+ for c in range(lastX.shape[1]):+ lo, hi = bounds[c], bounds[c + 1]+ if lo == hi:+ continue+ r = rowi[lo:hi]+ vals = np.asarray(lastX[:, c].todense()).ravel()+ nb = vals[idx[r]] # (m, K) neighbour values+ nb = nb.astype(np.float32)+ nb[nb == 0] = np.nan+ with np.errstate(invalid="ignore"):+ med = np.nanmedian(nb, axis=1) * VAL_FRAC+ bad = ~np.isfinite(med)+ med[bad] = 0.0+ good = ~bad+ X[r[good], gidx[c]] = med[good]+ counts[r[good]] += 1+ return counts++ def main() -> None: parser = argparse.ArgumentParser() parser.add_argument("--data", required=True)@@ -71,17 +141,25 @@ def main() -> None: n = target_n_cells(manifest, last.n_obs) if prev is None:- (g_last,) = growth_rates([last], genes, mask, args.seed)- rows = weighted_rows(g_last ** dt_out if GROWTH else np.ones(last.n_obs), n, rng)+ if GROWTH1:+ (g_last,) = growth_rates([last], genes, mask, args.seed)+ rows = weighted_rows(g_last ** dt_out, n, rng)+ elif n >= last.n_obs:+ rows = np.arange(last.n_obs)+ else:+ rows = weighted_rows(np.ones(last.n_obs), n, rng)+ print(f"single-stage fallback: uniform copy (growth1={GROWTH1}), n={n}", file=sys.stderr) write_prediction(last.X[rows], genes, args.out, seed=args.seed) return import anndata as ad import pandas as pd import torch+ from sklearn.neighbors import NearestNeighbors+ from scipy import sparse from moscot.problems.time import TemporalProblem - (Zp, Zl), _ = embed([prev.X, last.X], mask, N_HVG, N_PCS, args.seed)+ (Zp, Zl), info = embed([prev.X, last.X], mask, N_HVG, N_PCS, args.seed) g_prev, g_last = growth_rates([prev, last], genes, mask, args.seed) obs = pd.DataFrame({"time": np.r_[np.zeros(prev.n_obs), np.ones(last.n_obs)],@@ -91,7 +169,6 @@ def main() -> None: small = ad.AnnData(X=np.zeros((len(obs), 1), dtype=np.float32), obs=obs) small.obsm["X_pca"] = np.vstack([Zp, Zl]) tp = TemporalProblem(small)- # source marginals = WOT prior growth over the input interval (normalised by moscot); target uniform tp = tp.prepare(time_key="time", joint_attr="X_pca", a="growth") tp = tp.solve(epsilon=EPSILON, tau_a=TAU_A, tau_b=TAU_B, scale_cost="mean") sol = tp[(0.0, 1.0)].solution@@ -99,23 +176,65 @@ def main() -> None: P = np.asarray(sol.transport_matrix, dtype=np.float32) # (n_prev, n_last) rows = weighted_rows(g_last ** dt_out if GROWTH else np.ones(last.n_obs), n, rng)- print(f"diag: growth last min/median/max {g_last.min():.3f}/{np.median(g_last):.3f}/{g_last.max():.3f}", file=sys.stderr) Pc = P[:, rows] del P Pc /= np.maximum(Pc.sum(axis=0, keepdims=True), 1e-30) factor = LAMBDA * dt_out / dt_in torch.set_num_threads(N_THREADS) Xp = torch.from_numpy(prev.X.toarray())- anc = (torch.from_numpy(Pc).T @ Xp).numpy() # barycentric ancestor mean, (n, n_genes)- del Xp, Pc- smooth = knn_mean(Zl, last.X, rows, K_SMOOTH)- step = (smooth - anc) * factor+ PcT = np.ascontiguousarray(Pc.T) # (n, n_prev)+ anc = (torch.from_numpy(PcT) @ Xp).numpy() # barycentric ancestor mean, (n, n_genes)+ del Xp++ nn = NearestNeighbors(n_neighbors=K_SMOOTH).fit(Zl)+ idx = nn.kneighbors(Zl[rows], return_distance=False) # (n, K)+ W = sparse.csr_matrix((np.full(idx.size, 1.0 / K_SMOOTH, dtype=np.float32), idx.ravel(),+ np.arange(0, idx.size + 1, K_SMOOTH)), shape=(n, Zl.shape[0]))+ smooth = np.asarray((W @ last.X).todense(), dtype=np.float32)++ raw = (smooth - anc) * factor del smooth, anc- step[:, ~mask] = 0.0- X = last.X[rows].toarray()+ raw[:, ~mask] = 0.0++ ct = last.obs["celltype"].astype(str).to_numpy()[rows]+ if K_PROJ > 0:+ K = min(K_PROJ, info["components"].shape[0])+ V = info["components"][:K] # (K, N_HVG) orthonormal rows, z-space+ hvg, sd = info["hvg"], info["sd"]+ z = raw[:, hvg] / sd+ z = (z @ V.T) @ V+ step = np.zeros_like(raw)+ step[:, hvg] = z * sd+ step *= ALPHA+ print(f"diag: within-type std ratio proj/raw = {within_type_std(step, ct) / max(within_type_std(raw, ct), 1e-12):.3f}",+ file=sys.stderr)+ top_var = np.argsort(((raw[:, hvg] / sd) @ V.T).var(0))[::-1][:5]+ print(f"diag: top-5 PCs by projected displacement variance: {top_var.tolist()}", file=sys.stderr)+ else:+ step = raw * ALPHA++ Xin = last.X[rows]+ zf_in = 1.0 - Xin.count_nonzero() / (Xin.shape[0] * Xin.shape[1])+ X = Xin.toarray()+ del Xin step *= X > 0 # "addnz": move only measured (non-zero) entries, keeps each cell's zero pattern X += step np.maximum(X, 0.0, out=X)+ del step, raw++ if M_ACT > 0:+ K = min(max(K_PROJ, 1), info["components"].shape[0]) if K_PROJ > 0 else info["components"].shape[0]+ V_K = info["components"][:K]+ loadmax = np.abs(V_K).max(axis=0) # (N_HVG,)+ sel = loadmax > LOAD_MIN+ gidx = info["hvg"][sel]+ lm = loadmax[sel].astype(np.float32)+ counts = activate_genes(X, idx, W, PcT, prev, last, gidx, lm, M_ACT)+ print(f"diag: activation candidates={len(gidx)} activated/cell mean={counts.mean():.1f} "+ f"p90={np.percentile(counts, 90):.0f} max={counts.max()}", file=sys.stderr)+ zf_out = float((X == 0).sum()) / X.size+ print(f"diag: zero-fraction in={zf_in:.4f} out={zf_out:.4f} (change {zf_out - zf_in:+.4f})", file=sys.stderr)+ write_prediction(X, genes, args.out, seed=args.seed)
调研来源?调研员查到并用到的知识条目和文献检索结果(只列标题和编号)。
用到的知识库条目
| 编号 | 标题 | 出处 |
|---|---|---|
| k031 | Offline OT toolkit in the sandbox: moscot TemporalProblem, wot OTModel, POT, geomloss | 10.1038/s41586-024-08453-2 (moscot); 10.1016/j.cell.2019.01.006 (Waddington-OT) |
| k002 | moscot.time: scalable temporal OT with growth rates and low-rank solvers | 10.1038/s41586-024-08453-2 |
| k001 | Waddington-OT: unbalanced entropic OT between snapshot time points | 10.1016/j.cell.2019.01.006 |
分析结果?分析员写的 ANALYSIS.json:改了什么、各组分数怎么变、假设是否成立、经验和下一步建议。
| 改了什么 | 在父节点 ot_moscot 上改解码:两阶段路径把 raw 位移投影到前 K=22 个 PCA 程序方向(α=1,无收缩);基因激活已实现但 X3 上有害,默认关(M=0);单阶段回退(proxy10)去掉 g^dt_out 生长重采样,改为均匀复制最后阶段。 |
|---|---|
| 各组分数的变化 | cell_state:变好 +5.09(49.21→54.30),超出噪声,是主要收益:X3 mmd_u 0.03281→0.02794,skill 0.501→0.560,得分 +1.78;proxy10 mmd_u 0.04512→0.04135,skill 0.475→0.509,得分 +1.02(此项来自均匀复制回退,非投影机制) covariation:噪声内 −0.51(50.70→50.18):X3 variogram 0.001415→0.001504 得分 −0.36,proxy10 +0.41,互有抵消 de_recovery:+1.24(50.79→52.03):proxy10 de_score −0.013→0.042 得分 +0.45,X3 0.039→0.065 得分 +0.24,两把尺子单独看都接近噪声,合计方向一致 direction:变好 +3.03(50.90→53.94):proxy10 de_direction 0.098→0.201,得分 +1.39,来自去掉生长重采样(回退修复,与投影无关);X3 de_direction −0.023→0.023,得分 +0.44,偏小 |
| family_id | local_ot |
| 假设是否成立 | 是 |
| 经验 |
|
| mechanism_active | 是 |
| 下一步建议 |
|
对话摘要?每个角色和大模型对话的统计:轮数、工具调用、用时、token 数和最后的回答摘录;原始记录只给路径。
只给统计和最后回答的摘录;完整对话请到原始记录位置里列出的文件看。
分析员
| 角色?调研员写计划、工程师改代码、分析员解读分数、审查员检查作弊。 | 分析员 alibaba-token-plan-cn/qwen3.8-max |
|---|---|
| 调用次数 | 1 次 |
| 轮数?大模型一共回复了几轮(每轮可以调用多个工具)。 | 3 |
| 工具调用?大模型调用读文件、执行命令等工具的次数,按工具名统计。 | 共 2 次:bash 1、write 1 |
| 用时 | 不到 1 分 |
| token 数?输入 / 输出(含思考)的 token 数,来自接口返回的用量。 | 输入 18,142 · 输出 1,430 · 思考 1,396 |
| 任务(第一行) | 分析节点 n5 的打分结果,写 ANALYSIS.json |
| 最后的回答(摘录) | ANALYSIS.json written |
| 原始记录位置?完整对话记录在 Spark 上的文件路径;看板不复制原始内容。 | /home/spark-longxinyang/vec/runs/formal/20261003-093415-search-t1-r2-D-s1/nodes/5/analyst.jsonl 7 KB /home/spark-longxinyang/vec/runs/formal/20261003-093415-search-t1-r2-D-s1/nodes/5/analyst.stderr |
工程师
| 角色 | 工程师 alibaba-token-plan-cn/qwen3.8-max |
|---|---|
| 调用次数 | 1 次 |
| 轮数 | 29 |
| 工具调用 | 共 34 次:bash 26、edit 4、read 2、write 2 |
| 用时 | 17 分 |
| token 数 | 输入 63,847 · 输出 11,142 · 思考 13,374 |
| 任务(第一行) | 按计划修改程序:local_ot: PCA-program-projected displacement with shrinkage + capped gene activation |
| 最后的回答(摘录) | **做了什么**:在父节点 ot_moscot 上改解码(improve, PLAN local_ot):① 位移投影到前 K 个 PCA 程序方向(提交 K=22, α=1),② 基因激活已实现但 X3 上明显有害(−3~−5 分),默认 M=0,③ 单阶段回退去掉生长重采样改均匀复制。机制对照(K=0,M=0≈父解码 50.41 → 机制开 52.78)确认收益来自投影,主要在 mmd_u。 **查分**(A 半,用 15/20):X3 = **52.78**(父 50.65;K∈[15,30] 全部 >51.4,K=0 为 50.41,α=0.5 一致更低);proxy10 = **53.19**(父 49.67,≈copy_last)。预计节点分 ≈52.9 vs 父 50.33。seed 0 复跑逐字节一致,两视图 vec-check 通过。 **下一步最值得试**:final 视图上验证投影解码(dt_out/dt_in=1,X3 的 +2.4 未必等幅迁移);以及把节点 3 的组成趋势重加权与本投影解码组合(两者收益方向正交:组成→mmd/direction,投影→去噪)。 |
| 原始记录位置 | /home/spark-longxinyang/vec/runs/formal/20261003-093415-search-t1-r2-D-s1/nodes/5/engineer.jsonl 167 KB /home/spark-longxinyang/vec/runs/formal/20261003-093415-search-t1-r2-D-s1/nodes/5/engineer.stderr |
调研员
| 角色 | 调研员 alibaba-token-plan-cn/qwen3.8-max |
|---|---|
| 调用次数 | 1 次 |
| 轮数 | 1 |
| 工具调用 | 共 0 次:— |
| 用时 | 1 分 |
| token 数 | 输入 15,175 · 输出 1,476 · 思考 2,848 |
| 任务(第一行) | 为节点 n5(改进,父节点 n2)写改进计划 PLAN.json |
| 最后的回答(摘录) | too high → same as parent. Check de_direction and de_score to confirm directional signal preserved. 4. 30 min budget: PCA projection is cheap (matrix multiply); gene activation loop vectorized over cells. Total added compute <1 min. If moscot solve is the bottleneck, subsample to 3000 cells per stage.", "family_id": "local_ot", "mechanism": "OT coupling estimates per-cell ancestor distributions; displacement is projected onto top-K PCA programs with shrinkage factor α (removing gene-level noise), and module-supported genes are sparsely activated based on local detection-rate increases in coupled descendants.", "vs_constant_shift": "Each cell's displacement comes from its own coupling-weighted ancestor mean projected through data-driven PCA programs; the direction and magnitude vary per cell based on its local OT coupling, not a per-type constant vector. Gene activation is cell-specific (depends on its neighbours' detection rates). Report within-type displacement std to prove heterogeneity.", "mechanism_evidence": "1. Within-type std of displacement vectors: should be >20% of the unconstrained version (not collapsed to per-type constant). 2. Number of activated genes per cell (expect 10–50) and their top-K PC loading distribution. 3. Four metric groups separately: mmd_u/variogram should improve from shrinkage; de_direction/de_score should improve from activation. 4. Compare zero-fraction of output vs input (should change by <2%). 5. Print top-5 PCs by projected displacement variance to verify programs are coherent.", "mechanism_off_control": "Run same code with α=1.0, K=full rank (no projection), M=0 (no activation): this recovers the parent's unconstrained addnz displacement exactly. Additionally run with GROWTH=False on single-stage (uniform copy). If output with mechanism off equals parent output, mechanism was not running. Expected difference: mechanism-on should show lower mmd_u, higher de_direction, and non-zero activated gene counts.", "sources": []} ``` |
| 原始记录位置 | /home/spark-longxinyang/vec/runs/formal/20261003-093415-search-t1-r2-D-s1/nodes/5/researcher.jsonl 7 KB /home/spark-longxinyang/vec/runs/formal/20261003-093415-search-t1-r2-D-s1/nodes/5/researcher.stderr |