总览 · ← 返回运行 20261002-202908-search-t1-scr-B
节点 n11 在终选来历上
local_ot: PCA 空间逐细胞位移 + 残差保留解码(moscot 耦合),λ=3、kNN=30、addnz
| 运行?一次完整的自动搜索或 Agent 会话,有自己的锁定配置和证据包。 | 20261002-202908-search-t1-scr-B |
|---|---|
| 父节点 | n2 |
| 子节点 | n14 |
| 操作?种子:人写的起点;改进:在父节点上改;草稿:从头写;修复:修父节点的报错。 | 改进 |
| 状态 | 已打分 |
| 分数 | 搜索目标分 62.24(+11.6) · X3 62.24(+11.6) · 3 次复测均分 60.51 |
| 审查 | 通过 1 越界读取:未发现问题——run.py/g37_common.py 仅通过 src.task1_temporal.view_io 的 load_manifest/read_stage/panel_genes 读视图内数据(run.py:42,74-76),无绝对路径、'..'、/mnt、data/raw、评分器或 src/common/evaluation 访问,无联网代码(grep 全源码无 open/np.load/urllib/requests/http 命中)。; 2 硬编码目标统计量:未发现问题——常量仅为超参(run.py:44-54 的 N_HVG/N_PCS/EPSILON… |
| 用时?从运行开始到结束(或到现在)的挂钟时间。 | 21 分 |
| 程序版本 | 016536be7cb44077062f2651cc285daa04c971e8 (programs.git) |
方法说明?节点程序自带的 METHOD.md:这个程序做了什么、为什么。
来自 programs.git 016536be7c:solution/METHOD.md
local_ot: PCA 空间逐细胞位移 + 残差保留解码(moscot 耦合),λ=3、kNN=30、addnz
在父节点 2(ot_moscot 基因空间位移)上按 PLAN 改为:耦合重心祖先与 kNN 平滑都在拟合耦合的 30 维 PCA 空间里计算,位移经 PCA 线性解码回基因空间并保留每细胞 PCA 正交残差;λ 与 K_SMOOTH 在 X3 上网格搜索定标。
方法
两输入阶段 prev(t0)、last(t1),目标 t2;单输入时退化为生长加权重采样(与父节点相同,已在合成单输入视图上验证跑通 + vec-check ok)。
- 嵌入:双阶段共测基因 → 2000 HVG → z-score(clip 10) → 30 PC(只在输入阶段上拟合,random_state=seed)。
- 生长先验 + moscot TemporalProblem 耦合(ε=1e-3, τ_a=0.95, scale_cost=mean):与父节点完全相同。
- 输出细胞 = last 阶段按 g^dt_out 加权重采样(与父节点相同,rng 消耗顺序不变,同 seed 下 rows 与父节点一致)。
- PCA 空间位移(改动 1):anc_Z_j = 耦合重心祖先的 PCA 均值(Pc.T @ Zp,Pc 为按 rows 归一化的耦合列);s_Z_j = Zl 中 K_SMOOTH=30 近邻(含自身)的 PCA 均值;step_Z_j = λ·(dt_out/dt_in)·(s_Z_j − anc_Z_j)。
- 残差保留解码(改动 2):x_new = pca.inverse(z_j + step_Z_j) + [x_j − pca.inverse(z_j)]。残差项解析抵消,等价于 x_new = x_j + (step_Z_j @ components)·sd,只作用于 HVG 列;非 HVG 面板基因不变。随后 addnz(步长只加到该细胞非零表达项,β_zero=0)并 clip≥0。addnz 与 clip 由父节点在 X3 上定标,本节点复验:β_zero=0.3 → 51.5,β_zero=1(稠密)→ 37.9,均大幅劣于 β_zero=0 → 61.5(covariation 23.2/11.5 vs 50.3)。
--lambda、--k-smooth、--beta-zero为 CLI 旗标,提交默认 λ=3.0、K=30、β=0;输出只依赖视图数据、时间差与 seed,不依赖绝对时间/路径/manifest 写法。
超参定标(X3 A 半,seed 0,13 次查分)
| 配置 | 榜分 | cell_state | covariation | de_recovery | direction |
|---|---|---|---|---|---|
| λ=0(机制关闭对照) | 50.52 | 54.10 | 50.25 | 46.49 | 50.46 |
| λ=1, k15 | 53.20 | 60.15 | 50.38 | 49.07 | 51.25 |
| λ=2, k15 | 58.84 | 79.00 | 50.77 | 48.62 | 51.33 |
| λ=2.5, k30 | 59.98 | 83.19 | 51.18 | 47.75 | 51.42 |
| λ=3, k10 / k15 / k30 / k50 | 60.81 / 61.10 / 61.47 / 61.11 | 88.1 / 89.2 / 89.2 / 87.2 | 49.2 / 49.5 / 50.3 / 50.8 | 46.9 / 46.5 / 47.3 / 47.8 | ~51.3 |
| λ=3.5, k30 | 61.06 | 89.28 | 49.04 | 46.49 | 51.37 |
| λ=4 / λ=5, k30 | 59.09 / 53.04 | 84.1 / 66.6 | 47.6 / 44.8 | 46.1 / 45.3 | ~51.2 |
| λ=3, k30, β_zero=0.3 / 1.0 | 51.55 / 37.90 | 75.0 / 39.3 | 23.2 / 11.5 | 46.5 / 44.9 | 51.2 / 50.4 |
λ=3、K=30 名义最优;K∈[10,50] 差异 < 噪声(T1 约 2 分),λ 峰值明确(3 附近,4 起下降)。
机制生效证据(λ=3, k30, seed 0,run.py stderr 诊断)
- 对照差:λ=0 → 50.52 vs λ=3 → 61.47(+10.9,主要来自 cell_state 54.1 → 89.2);λ=0 输出与 λ>0 不同(文件与分数均不同),机制确实在跑。
- mean|step|/mean|residual| 每细胞中位数 1.23(p10 0.94, p90 1.63)> 0.5:PCA 步长相对残差是主导信号,解码不过保守(PLAN 风险 1 排除)。
- 每细胞被移动基因数(|step|>0.01)中位 1982(min 1953, max 1995),逐细胞不同。
- step 簇内弥散:8 个 KMeans 簇内逐维 std 均值 0.059 vs 全体 0.061(比值 0.97):位移不是逐类型常数,簇内细胞间方向/幅度仍各异(每个细胞的步长取决于自身耦合重心 + 自身局部邻域)。
验证过 / 没验证
- 验证:X3 A 半 seed 0 全网格(上表);同 seed 逐比特确定性;vec-check ok;合成单输入视图 fallback 跑通;λ=0 对照。
- seed 稳健性:seed 1(X3 A 半)57.70(cell_state 80.7, cov 45.8, de_rec 46.9, dir 50.4)vs seed 0 61.47,种子间波动 ~3.8 分,仍显著高于对照 50.5 与父节点 50.65。
- 没验证:X3 B 半与 seed 2 复跑(正式分由系统跑);final/proxy 视图本地不可得(fallback 代码与父节点逐行相同,父节点已在 proxy 验证);伪装视图重跑(代码不读绝对时间/路径/board 字段,时间平移下 dt_in、dt_out 不变,预测应逐比特相同)。
- 相对节点 8(同族,λ=3 PCA 位移,62.24):本节点 covariation 50.3 略高、de_recovery 47.3 略低、cell_state 89.2 略低,A 半分差在噪声内;残差保留解码在本实现下与"直接加解码步长"数学等价,未带来 PLAN 预期的 de_recovery 修复(de_recovery 对 λ 增大单调下降,是位移外推本身的代价,不是解码丢残差造成)。
知识/数据来源
只用视图输入阶段 + 父节点已有组件:moscot(Klein et al., Nature 2025)、Waddington-OT 生长先验(Schiebinger et al., Cell 2019,moscot 内置小鼠增殖/凋亡基因列表,阶段无关的通用基因功能注释)。无保留阶段/保留基因型信息,无外部训练数据(X3 视图挂载的 external 未使用)。
资源
X3 视图全程 ~15 s / <2 GB(CPU,EXECUTION.json gpu:false,JAX/torch 固定 CPU、8 线程)。
调研员的计划
| 名称 | PCA-space displacement with residual-preserving decode and tighter kNN |
|---|---|
| 动机 | Node 8 (sibling, rank3 60.51) proved that moving displacement into PCA space boosts cell_state +40.88, but its rank-30 decode hurt covariation (-2.91) and de_recovery (-2.26) by discarding gene-specific variance orthogonal to the 30 PCs. Node 2's direction group is weakest (49.14), and K_SMOOTH=30 may over-average local directional signal. Node 4 confirmed coupling is already sparse (median 2 ancestors), so top-k truncation is a no-op; the structural fix must be in the decode and smoothing. |
| 做法 | Starting from parent node 2's code, make three changes: 1. PCA-space displacement (adopted from node 8's proven structure): compute anc_Z = Pc.T @ Zp (ancestor mean in 30-dim PCA), s_Z = kNN mean of last-stage cells in PCA space, step_Z = λ·(dt_out/dt_in)·(s_Z − anc_Z). 2. Residual-preserving decode (fixes node 8's covariation/de_recovery loss): For each output cell j with raw expression x_j (HVG subset): - z_j = pca.transform(x_j_HVG) - z_new_j = z_j + step_Z_j - x_recon_j = pca.inverse_transform(z_new_j) (using components_ × sd + mean_) - residual_j = x_j_HVG − pca.inverse_transform(z_j) - x_new_j_HVG = x_recon_j + residual_j This preserves each cell's gene-specific variation orthogonal to the 30 PCs. Apply non-zero mask and clip≥0 as before. Non-HVG genes in the panel remain unchanged (step=0). 3. Reduce K_SMOOTH from 30 to 15 (default), with a flag --k-smooth. Smaller K preserves local directional structure, addressing the weakest group (direction 49.14). Parameter search on X3 A-half (vec-score): - λ ∈ {1.0, 2.0, 3.0} (start with 2.0 as middle ground) - K_SMOOTH ∈ {10, 15, 30} - Total 9 configs; run small (first 2000 cells per stage) to f… |
| 风险 | 1. Residual-preserving decode may partially cancel the displacement effect if residuals are large relative to the PCA step, reducing cell_state gain vs node 8. Engineer should check: compare mean |step| to mean |residual| per cell; if ratio < 0.5, the decode is too conservative and λ should be increased. 2. K_SMOOTH=15 may introduce noise in sparse regions. Check: if direction doesn't improve or cell_state drops, revert to K=30. 3. The 30-min budget is tight if 9 configs need full runs. Mitigation: run all 9 on a 2000-cell subset first (each ~3s), pick top 2 by X3 A-half, then run full. 4. If residual preservation fully restores covariation/de_recovery but eliminates cell_state gain, the mechanism is not working as intended — the PCA step must be the dominant signal. |
代码改动?这个节点的程序和父节点程序的逐行差别:绿色是新增,红色是删除。
对比:父节点版本 8204ac3d1a。改动的文件:solution/METHOD.md +36 −66、solution/g37_common.py +2 −1、solution/run.py +82 −29
diff --git a/solution/METHOD.md b/solution/METHOD.mdindex ca7a3d2..8c91c0c 100644--- a/solution/METHOD.md+++ b/solution/METHOD.md@@ -1,81 +1,51 @@-# ot_moscot — Waddington-OT / moscot coupling, one-step displacement extrapolation+# local_ot: PCA 空间逐细胞位移 + 残差保留解码(moscot 耦合),λ=3、kNN=30、addnz -Seed (2026-10-02) made from the G37 candidate `modeling/candidates/T1/ot_moscot/` (commit 227eeb2). Same method and-hyper-parameters. Changes for the seed contract only: the dev-only environment overrides (`G37_LAMBDA`, `G37_DECODE`,-`G37_GROWTH`, `G37_JAX_GPU`) and the unused `knn` decode branch are removed; JAX forced to CPU, torch threads fixed-at 8 (`EXECUTION.json {"gpu": false}`); `stage_pair` reads every input stage of the view the same way (no reference to-manifest `mode` / `source`). Output depends only on the view's data, the time differences between stages and `--seed`.+在父节点 2(ot_moscot 基因空间位移)上按 PLAN 改为:耦合重心祖先与 kNN 平滑都在拟合耦合的 30 维 PCA 空间里计算,位移经 PCA 线性解码回基因空间并保留每细胞 PCA 正交残差;λ 与 K_SMOOTH 在 X3 上网格搜索定标。 -Contract: `python run.py --data <view> --out <pred.h5ad> --seed <int>`; `g37_common.py` must stay next to `run.py`.+## 方法 -## Method+两输入阶段 prev(t0)、last(t1),目标 t2;单输入时退化为生长加权重采样(与父节点相同,已在合成单输入视图上验证跑通 + vec-check ok)。 -Two input stages `prev` (time t0) and `last` (t1); target time t2 (final view: E8.5, E9.5 -> E10.5).+1. 嵌入:双阶段共测基因 → 2000 HVG → z-score(clip 10) → 30 PC(只在输入阶段上拟合,random_state=seed)。+2. 生长先验 + moscot TemporalProblem 耦合(ε=1e-3, τ_a=0.95, scale_cost=mean):与父节点完全相同。+3. 输出细胞 = last 阶段按 g^dt_out 加权重采样(与父节点相同,rng 消耗顺序不变,同 seed 下 rows 与父节点一致)。+4. **PCA 空间位移(改动 1)**:anc_Z_j = 耦合重心祖先的 PCA 均值(Pc.T @ Zp,Pc 为按 rows 归一化的耦合列);s_Z_j = Zl 中 K_SMOOTH=30 近邻(含自身)的 PCA 均值;step_Z_j = λ·(dt_out/dt_in)·(s_Z_j − anc_Z_j)。+5. **残差保留解码(改动 2)**:x_new = pca.inverse(z_j + step_Z_j) + [x_j − pca.inverse(z_j)]。残差项解析抵消,等价于 x_new = x_j + (step_Z_j @ components)·sd,只作用于 HVG 列;非 HVG 面板基因不变。随后 addnz(步长只加到该细胞非零表达项,β_zero=0)并 clip≥0。addnz 与 clip 由父节点在 X3 上定标,本节点复验:β_zero=0.3 → 51.5,β_zero=1(稠密)→ 37.9,均大幅劣于 β_zero=0 → 61.5(covariation 23.2/11.5 vs 50.3)。+6. `--lambda`、`--k-smooth`、`--beta-zero` 为 CLI 旗标,提交默认 λ=3.0、K=30、β=0;输出只依赖视图数据、时间差与 seed,不依赖绝对时间/路径/manifest 写法。 -1. **Embedding.** Genes measured in both stages; top 2000 by variance (both stages pooled); z-score, clip at 10;- PCA, 30 components (randomized, `random_state=seed`). Fitted on the input stages only.-2. **Growth prior (Waddington-OT).** Proliferation / apoptosis scores (`scanpy.tl.score_genes`, moscot's mouse- gene lists) -> birth = generalised logistic(prolif; 1.7, 0.3, 0.25, 0.5), death = logistic(apopt; 1.7, 0.3, 0.1,- 0.2), per-day growth g = exp(birth - death) (Schiebinger 2019; same formula and defaults as moscot's- `BirthDeathProblem.estimate_marginals`).-3. **Coupling.** `moscot.problems.time.TemporalProblem` prev -> last on the PCA (`joint_attr="X_pca"`),- source marginal ∝ g^(t1-t0), target uniform; entropic unbalanced Sinkhorn, `epsilon=1e-3`, `tau_a=0.95`,- `tau_b=1`, `scale_cost="mean"` (moscot tutorial settings). JAX on CPU unless `G37_JAX_GPU=1`.-4. **Output cells.** n = number of latest-stage cells clipped to `[min_cells, max_cells]` (as copy_last),- drawn without replacement from the latest stage with probability ∝ g^(t2-t1) (the WOT birth-death model- continued over the target interval).-5. **Displacement extrapolation.** For output cell j: ancestor mean a_j = Σ_i π_ij x_i / Σ_i π_ij (barycentric- projection of the coupling, gene space, all panel genes); smoothed position s_j = mean expression of its 30- nearest latest-stage cells in the PCA; step_j = λ (t2-t1)/(t1-t0) (s_j - a_j), λ = 1 (continue the last- observed displacement at the same rate). The step is added **to the cell's non-zero entries only** and clipped- at 0; genes not measured in both stages (external stages) get no step. Each cell keeps its own residual.+## 超参定标(X3 A 半,seed 0,13 次查分) -**One input stage (proxy view, E8.5 only):** steps 1, 3, 5 need two stages; only steps 2 + 4 run, i.e. a-growth-weighted copy of the latest stage (g^(t2-t1) resampling). This is the only thing the proxy can test.+| 配置 | 榜分 | cell_state | covariation | de_recovery | direction |+|---|---|---|---|---|---|+| λ=0(机制关闭对照) | 50.52 | 54.10 | 50.25 | 46.49 | 50.46 |+| λ=1, k15 | 53.20 | 60.15 | 50.38 | 49.07 | 51.25 |+| λ=2, k15 | 58.84 | 79.00 | 50.77 | 48.62 | 51.33 |+| λ=2.5, k30 | 59.98 | 83.19 | 51.18 | 47.75 | 51.42 |+| λ=3, k10 / k15 / **k30** / k50 | 60.81 / 61.10 / **61.47** / 61.11 | 88.1 / 89.2 / **89.2** / 87.2 | 49.2 / 49.5 / **50.3** / 50.8 | 46.9 / 46.5 / **47.3** / 47.8 | ~51.3 |+| λ=3.5, k30 | 61.06 | 89.28 | 49.04 | 46.49 | 51.37 |+| λ=4 / λ=5, k30 | 59.09 / 53.04 | 84.1 / 66.6 | 47.6 / 44.8 | 46.1 / 45.3 | ~51.2 |+| λ=3, k30, β_zero=0.3 / 1.0 | 51.55 / 37.90 | 75.0 / 39.3 | 23.2 / 11.5 | 46.5 / 44.9 | 51.2 / 50.4 | -Sources:-- Schiebinger G. et al. Optimal-transport analysis of single-cell gene expression identifies developmental- trajectories in reprogramming. *Cell* 176, 928–943 (2019). doi:10.1016/j.cell.2019.01.006 (WOT: unbalanced- entropic OT between snapshots, growth from proliferation/apoptosis signatures, birth-death logistic).-- Klein D., Palla G., Lange M. et al. Mapping cells through time and space with moscot. *Nature* 638, 1065–1075- (2025). doi:10.1038/s41586-024-08453-2 (TemporalProblem; code moscot 0.5.2, BSD-3).-- Cuturi M. Sinkhorn distances. NeurIPS 2013; Chizat L. et al. Scaling algorithms for unbalanced optimal- transport problems. *Math. Comp.* 87, 2563–2609 (2018).-- Extrapolating the barycentric displacement one more step is our use of the coupling (WOT/moscot interpolate,- they do not extrapolate); listed in agent/knowledge/T1_methods_landscape.md §1.+λ=3、K=30 名义最优;K∈[10,50] 差异 < 噪声(T1 约 2 分),λ 峰值明确(3 附近,4 起下降)。 -## Data / knowledge used+## 机制生效证据(λ=3, k30, seed 0,run.py stderr 诊断) -Only the view's input stages. Generic knowledge: moscot's built-in mouse proliferation (97) and apoptosis (193)-gene lists (`moscot.utils.data`, from the WOT paper; stage-agnostic gene-function annotation). No held-out stage,-no information from (E9.5, E13.5], no pre-trained weights.+- 对照差:λ=0 → 50.52 vs λ=3 → 61.47(+10.9,主要来自 cell_state 54.1 → 89.2);λ=0 输出与 λ>0 不同(文件与分数均不同),机制确实在跑。+- mean|step|/mean|residual| 每细胞中位数 1.23(p10 0.94, p90 1.63)> 0.5:PCA 步长相对残差是主导信号,解码不过保守(PLAN 风险 1 排除)。+- 每细胞被移动基因数(|step|>0.01)中位 1982(min 1953, max 1995),逐细胞不同。+- step 簇内弥散:8 个 KMeans 簇内逐维 std 均值 0.059 vs 全体 0.061(比值 0.97):位移不是逐类型常数,簇内细胞间方向/幅度仍各异(每个细胞的步长取决于自身耦合重心 + 自身局部邻域)。 -## Hyper-parameters+## 验证过 / 没验证 -| Name | Value | Where it came from |-|---|---|---|-| `N_HVG`, `N_PCS`, `K_SMOOTH` | 2000, 30, 30 | a priori; not tuned |-| `EPSILON`, `TAU_A`, `TAU_B`, `scale_cost` | 1e-3, 0.95, 1, mean | a priori (moscot tutorial settings) |-| growth prior | on | a priori (WOT / moscot defaults); G37 saw it cost ~3 points on the old proxy, kept on |-| `LAMBDA` | 1 | a priori (continue the observed displacement at the same rate); G37 also ran 0.5 on X3 (50.4 vs 50.2), not changed |-| decode `addnz` | — | **chosen on the X3 ruler (G37)** against `add` (25.6) and `knn` (48.7): the dense step destroys the zero pattern. Also a first-principles choice (the scorer compares sparse log-expression), but the evidence that picked it was X3 |+- 验证:X3 A 半 seed 0 全网格(上表);同 seed 逐比特确定性;vec-check ok;合成单输入视图 fallback 跑通;λ=0 对照。+- seed 稳健性:seed 1(X3 A 半)57.70(cell_state 80.7, cov 45.8, de_rec 46.9, dir 50.4)vs seed 0 61.47,种子间波动 ~3.8 分,仍显著高于对照 50.5 与父节点 50.65。+- 没验证:X3 B 半与 seed 2 复跑(正式分由系统跑);final/proxy 视图本地不可得(fallback 代码与父节点逐行相同,父节点已在 proxy 验证);伪装视图重跑(代码不读绝对时间/路径/board 字段,时间平移下 dt_in、dt_out 不变,预测应逐比特相同)。+- 相对节点 8(同族,λ=3 PCA 位移,62.24):本节点 covariation 50.3 略高、de_recovery 47.3 略低、cell_state 89.2 略低,A 半分差在噪声内;残差保留解码在本实现下与"直接加解码步长"数学等价,未带来 PLAN 预期的 de_recovery 修复(de_recovery 对 λ 增大单调下降,是位移外推本身的代价,不是解码丢残差造成)。 -No re-tuning for the seed.+## 知识/数据来源 -## Resources (Spark, CPU)+只用视图输入阶段 + 父节点已有组件:moscot(Klein et al., Nature 2025)、Waddington-OT 生长先验(Schiebinger et al., Cell 2019,moscot 内置小鼠增殖/凋亡基因列表,阶段无关的通用基因功能注释)。无保留阶段/保留基因型信息,无外部训练数据(X3 视图挂载的 external 未使用)。 -Final view (16.8k x 17.1k coupling): 69 s, max RSS 6.7 GB; proxy 4 s / 1.6 GB; X3 ~ 8 s / 2.3 GB; proxy2 ~ 20 s / 4.3 GB.-The dense coupling (n_prev x n_last float32, ~1.1 GB on final) and the dense earlier stage (~2.2 GB) dominate memory.+## 资源 -## Findings (G37, local scorer: fast engine, truth half B, scorer seed = program seed)--- Dense gene-space step (`add`) is destructive: X3 25.6 at λ=1, still 32.9 at λ=0.25 (cell_state, covariation- collapse) — the step makes every zero slightly positive. `addnz` fixes it (X3 50.2 at λ=1, 50.4 at λ=0.5);- `knn` decode 48.7 (λ=1). Growth resampling has no effect on X3/proxy2 (all cells are kept there).-- proxy2 (E8.5 official -> Qiu E9.0 heart, then +0.5 d): every variant ≈ copy of the Qiu cells (~27.5), the- cross-dataset step is batch effect.-- Full eval (seeds 0-2, half B): proxy 46.8 / 46.8 / 47.9 (copy_last 50.0 / 50.3 / 50.1) — the WOT growth- resampling alone costs ~3 points on E8.5 -> E9.5 (direction, cell_state); X3 50.2 / 50.1 / 49.8 (copy_last 50.0,- pseudobulk_shift 40.5-40.7); proxy2 27.7-27.9 (copy_last 27.4-27.6). `G37_GROWTH=0` turns the proxy into copy_last.-- Final-view prediction (`~/vec/scratch/g37/ot_moscot/final.h5ad`, seed 0): 5118 cells, 21 nearest-E9.5 types,- composition within ±2 % of E9.5 (OFT/RV-CM -2.1 %); no new states (the method cannot create them).+X3 视图全程 ~15 s / <2 GB(CPU,EXECUTION.json gpu:false,JAX/torch 固定 CPU、8 线程)。diff --git a/solution/g37_common.py b/solution/g37_common.pyindex 8c991b3..7676511 100644--- a/solution/g37_common.py+++ b/solution/g37_common.py@@ -59,7 +59,8 @@ def embed(mats: list, mask: np.ndarray, n_hvg: int, n_pcs: int, seed: int): for M in mats: out.append(Z[start:start + M.shape[0]]) start += M.shape[0]- return out, {"hvg": hvg, "mu": mu, "sd": sd, "components": pca.components_.astype(np.float32)}+ return out, {"hvg": hvg, "mu": mu, "sd": sd, "components": pca.components_.astype(np.float32),+ "pca_mean": pca.mean_.astype(np.float32)} def knn_mean(Z: np.ndarray, X: sparse.csr_matrix, rows: np.ndarray, k: int) -> np.ndarray:diff --git a/solution/run.py b/solution/run.pyindex 1411cb7..971f54e 100644--- a/solution/run.py+++ b/solution/run.py@@ -1,5 +1,7 @@ #!/usr/bin/env python3-"""ot_moscot: Waddington-OT / moscot TemporalProblem coupling, extrapolated one step past the latest input stage.+"""local_ot (node 11, from ot_moscot seed): moscot coupling, per-cell displacement computed and extrapolated in the+PCA subspace where the coupling was fitted, decoded back to gene space with each cell's PCA-orthogonal residual+preserved. Two input stages (final: E8.5, E9.5): 1. joint PCA of both stages (HVG, z-score, 30 PCs; fitted on the inputs only);@@ -7,19 +9,18 @@ Two input stages (final: E8.5, E9.5): (proliferation / apoptosis gene scores), entropic unbalanced Sinkhorn (epsilon 1e-3, tau_a 0.95); 3. output cells = latest-stage cells resampled with weights g^dt_out (prior growth rate continued for the target interval, Waddington-OT birth-death model);- 4. each output cell j moves by LAMBDA * dt_out / dt_in * (kNN-mean(x_j) - ancestor_mean_j) in gene space, where- ancestor_mean_j is the coupling-weighted (barycentric) mean of its ancestors in the earlier stage: the last- observed displacement continued for the target interval; the cell keeps its own residual. The step is applied- to the cell's non-zero entries only (DECODE "addnz": a dense step turns every zero into a small positive value- and wrecks cell_state / covariation, see METHOD.md). Clipped at 0.-One input stage (proxy: E8.5 only): steps 1, 2, 4 need two stages; only the growth resampling (3) runs, i.e.+ 4. displacement in PCA space: anc_Z_j = coupling-weighted (barycentric) mean of cell j's ancestors' PCA+ coordinates; s_Z_j = kNN mean (K_SMOOTH nearest latest-stage cells) of its PCA coordinates;+ step_Z_j = LAMBDA * dt_out / dt_in * (s_Z_j - anc_Z_j);+ 5. residual-preserving decode: x_new_j = x_j + (step_Z_j @ components) * sd on the HVG columns (equivalently+ pca.inverse(z_j + step_Z_j) + [x_j - pca.inverse(z_j)]; the residual cancels analytically), non-HVG panel+ genes unchanged. The step is applied to the cell's non-zero entries only (DECODE "addnz": a dense step turns+ every zero into a small positive value and wrecks cell_state / covariation, see METHOD.md). Clipped at 0.+One input stage (proxy: E8.5 only): steps 1, 2, 4, 5 need two stages; only the growth resampling (3) runs, i.e. a growth-weighted copy of the latest stage. -Seed version (agent/seeds/T1__val/ot_moscot, 2026-10-02) of modeling/candidates/T1/ot_moscot (G37): same method and-hyper-parameters; the dev-only environment overrides (G37_LAMBDA / G37_DECODE / G37_GROWTH / G37_JAX_GPU) and the-`knn` decode branch are removed; JAX and torch on CPU, fixed thread count (EXECUTION.json gpu false).-Parameter provenance (METHOD.md): DECODE = addnz was chosen on the X3 ruler (G37); LAMBDA = 1 a priori, also checked-on X3 (0.5 vs 1 within 0.2).+Mechanism-off control: --lambda 0 zeroes step_Z; the output is then exactly the growth-weighted resampling of the+latest stage (steps 2, 4, 5 skipped). JAX and torch on CPU, fixed thread count (EXECUTION.json gpu false). """ from __future__ import annotations@@ -36,7 +37,7 @@ os.environ["JAX_PLATFORMS"] = "cpu" # CPU only: deterministic, no GPU slot import numpy as np sys.path.insert(0, str(Path(__file__).resolve().parent))-from g37_common import embed, growth_rates, knn_mean, stage_pair # noqa: E402+from g37_common import embed, growth_rates, stage_pair # noqa: E402 from src.task1_temporal.view_io import load_manifest, panel_genes, target_n_cells, write_prediction # noqa: E402 @@ -45,9 +46,11 @@ N_PCS = 30 EPSILON = 1e-3 TAU_A = 0.95 TAU_B = 1.0-K_SMOOTH = 30-LAMBDA = 1.0 # 1 = continue the observed displacement at full rate+K_SMOOTH = 30 # kNN neighbourhood in PCA space (X3: k30 61.5 >= k15 61.1 >= k50 61.1 >= k10 60.8, within noise)+LAMBDA = 3.0 # extrapolation strength; X3 grid: 1 -> 53.2, 2 -> 58.8, 2.5 -> 60.0, 3 -> 61.5, 3.5 -> 61.1,+ # 4 -> 59.1, 5 -> 53.0; 0 = mechanism off (growth resampling only, X3 50.5) GROWTH = True # resample output cells by g^dt_out (WOT birth-death model)+BETA_ZERO = 0.0 # step weight on zero entries: 0 = "addnz" (keep each cell's zero pattern), 1 = dense N_THREADS = 8 @@ -61,6 +64,11 @@ def main() -> None: parser.add_argument("--data", required=True) parser.add_argument("--out", required=True) parser.add_argument("--seed", type=int, default=0)+ parser.add_argument("--lambda", dest="lam", type=float, default=LAMBDA,+ help="displacement extrapolation strength; 0 = mechanism off (growth resampling only)")+ parser.add_argument("--k-smooth", type=int, default=K_SMOOTH, help="kNN neighbourhood size in PCA space")+ parser.add_argument("--beta-zero", type=float, default=BETA_ZERO,+ help="fraction of the step applied to zero entries (0 = addnz, 1 = dense)") args = parser.parse_args() manifest = load_manifest(args.data)@@ -76,13 +84,21 @@ def main() -> None: write_prediction(last.X[rows], genes, args.out, seed=args.seed) return + (Zp, Zl), info = embed([prev.X, last.X], mask, N_HVG, N_PCS, args.seed)+ g_prev, g_last = growth_rates([prev, last], genes, mask, args.seed)+ rows = weighted_rows(g_last ** dt_out if GROWTH else np.ones(last.n_obs), n, rng)+ print(f"diag: growth last min/median/max {g_last.min():.3f}/{np.median(g_last):.3f}/{g_last.max():.3f}", file=sys.stderr)++ if args.lam == 0.0:+ # mechanism-off control: step_Z = 0, pure growth-weighted resampling of the latest stage+ write_prediction(last.X[rows], genes, args.out, seed=args.seed)+ return+ import anndata as ad import pandas as pd import torch from moscot.problems.time import TemporalProblem-- (Zp, Zl), _ = embed([prev.X, last.X], mask, N_HVG, N_PCS, args.seed)- g_prev, g_last = growth_rates([prev, last], genes, mask, args.seed)+ from sklearn.neighbors import NearestNeighbors obs = pd.DataFrame({"time": np.r_[np.zeros(prev.n_obs), np.ones(last.n_obs)], "growth": np.r_[g_prev ** dt_in, g_last ** dt_in]},@@ -98,26 +114,63 @@ def main() -> None: print(f"ot: converged={getattr(sol, 'converged', None)} cost={getattr(sol, 'cost', None)}", file=sys.stderr) P = np.asarray(sol.transport_matrix, dtype=np.float32) # (n_prev, n_last) - rows = weighted_rows(g_last ** dt_out if GROWTH else np.ones(last.n_obs), n, rng)- print(f"diag: growth last min/median/max {g_last.min():.3f}/{np.median(g_last):.3f}/{g_last.max():.3f}", file=sys.stderr) Pc = P[:, rows] del P Pc /= np.maximum(Pc.sum(axis=0, keepdims=True), 1e-30)- factor = LAMBDA * dt_out / dt_in torch.set_num_threads(N_THREADS)- Xp = torch.from_numpy(prev.X.toarray())- anc = (torch.from_numpy(Pc).T @ Xp).numpy() # barycentric ancestor mean, (n, n_genes)- del Xp, Pc- smooth = knn_mean(Zl, last.X, rows, K_SMOOTH)- step = (smooth - anc) * factor- del smooth, anc- step[:, ~mask] = 0.0+ anc_Z = (torch.from_numpy(Pc).T @ torch.from_numpy(Zp)).numpy() # barycentric ancestor mean in PCA space+ del Pc+ factor = args.lam * dt_out / dt_in++ k_smooth = min(args.k_smooth, Zl.shape[0])+ nn = NearestNeighbors(n_neighbors=k_smooth).fit(Zl)+ idx = nn.kneighbors(Zl[rows], return_distance=False)+ s_Z = Zl[idx].mean(axis=1) # smoothed position in PCA space+ step_Z = (s_Z - anc_Z).astype(np.float32) * factor++ # residual-preserving decode: x_new = pca.inverse(z + step_Z) + [x - pca.inverse(z)]+ # = x + (step_Z @ components) * sd on the HVG columns (residual cancels exactly);+ # non-HVG panel genes keep their raw value (step 0 there).+ hvg, sd, comp = info["hvg"], info["sd"], info["components"]+ step = np.zeros((n, len(genes)), dtype=np.float32)+ step[:, hvg] = (step_Z @ comp) * sd+ del step_Z X = last.X[rows].toarray()- step *= X > 0 # "addnz": move only measured (non-zero) entries, keeps each cell's zero pattern+ _diagnostics(step, X, Zl, rows, info)+ if args.beta_zero <= 0.0:+ step *= X > 0 # "addnz": move only measured (non-zero) entries, keeps each cell's zero pattern+ elif args.beta_zero < 1.0:+ step *= np.where(X > 0, 1.0, np.float32(args.beta_zero)) X += step np.maximum(X, 0.0, out=X) write_prediction(X, genes, args.out, seed=args.seed) +def _diagnostics(step: np.ndarray, X: np.ndarray, Zl: np.ndarray, rows: np.ndarray, info: dict) -> None:+ """Mechanism evidence (stderr only): per-cell step magnitude vs residual, within-cluster step spread,+ number of genes moved per cell. Never affects the output."""+ from sklearn.cluster import KMeans++ hvg, mu, sd, comp = info["hvg"], info["mu"], info["sd"], info["components"]+ xh = X[:, hvg]+ recon = (Zl[rows] @ comp + info["pca_mean"]) * sd + mu+ resid = np.abs(xh - recon).mean(axis=1)+ mstep = np.abs(step[:, hvg]).mean(axis=1)+ ratio = mstep / np.maximum(resid, 1e-9)+ n_moved = (np.abs(step[:, hvg]) > 0.01).sum(axis=1)+ km = KMeans(n_clusters=min(8, len(rows)), n_init=3, random_state=0).fit(Zl[rows])+ within, overall = [], step.std(axis=0)+ for c in range(km.cluster_centers_.shape[0]):+ m = km.labels_ == c+ if m.sum() > 2:+ within.append(step[m].std(axis=0).mean())+ print(f"diag: mean|step|/mean|resid| median={np.median(ratio):.3f} "+ f"p10={np.percentile(ratio, 10):.3f} p90={np.percentile(ratio, 90):.3f}", file=sys.stderr)+ print(f"diag: genes moved per cell (|step|>0.01) median={np.median(n_moved):.0f} "+ f"min={n_moved.min()} max={n_moved.max()}", file=sys.stderr)+ print(f"diag: step spread within-cluster mean={np.mean(within):.3f} overall per-dim std mean={overall.mean():.3f}"+ f" (within/overall={np.mean(within) / max(overall.mean(), 1e-9):.3f})", file=sys.stderr)++ if __name__ == "__main__": main()
调研来源?调研员查到并用到的知识条目和文献检索结果(只列标题和编号)。
用到的知识库条目
| 编号 | 标题 | 出处 |
|---|---|---|
| k031 | Offline OT toolkit in the sandbox: moscot TemporalProblem, wot OTModel, POT, geomloss | 10.1038/s41586-024-08453-2 (moscot); 10.1016/j.cell.2019.01.006 (Waddington-OT) |
| k002 | moscot.time: scalable temporal OT with growth rates and low-rank solvers | 10.1038/s41586-024-08453-2 |
| k001 | Waddington-OT: unbalanced entropic OT between snapshot time points | 10.1016/j.cell.2019.01.006 |
分析结果?分析员写的 ANALYSIS.json:改了什么、各组分数怎么变、假设是否成立、经验和下一步建议。
| 改了什么 | 把父节点 2(基因空间位移)改为在拟合耦合的 30 维 PCA 空间里计算逐细胞位移(耦合重心祖先 vs kNN 平滑位置),线性解码回基因空间(残差项解析抵消,等价于 x + step_Z@components·sd,addnz + clip≥0);λ 从 1 升到 3(X3 网格定标),K_SMOOTH 保持 30,新增 --lambda/--k-smooth/--beta-zero 旗标与诊断输出。 |
|---|---|
| 各组分数的变化 | cell_state:大幅变好:50.07 → 90.95(+40.88),远超噪声,是榜分提升的全部来源 covariation:略变坏:52.52 → 49.61(-2.91),超噪声约 1 分;PLAN 预期残差保留解码能护住该组,但该解码在本实现下与直接加步长数学等价,未起作用 de_recovery:略变坏:51.38 → 49.12(-2.26),刚超噪声;X3 网格显示 de_recovery 随 λ 单调下降,是外推本身的代价 direction:噪声内:49.14 → 51.02(+1.88,T1 噪声约 2 分),PLAN 预期 K=15 改善 direction 未实现(最终提交 K=30) |
| family_id | local_ot |
| 假设是否成立 | 是 |
| 经验 |
|
| mechanism_active | 是 |
| 下一步建议 |
|
对话摘要?每个角色和大模型对话的统计:轮数、工具调用、用时、token 数和最后的回答摘录;原始记录只给路径。
只给统计和最后回答的摘录;完整对话请到原始记录位置里列出的文件看。
分析员
| 角色?调研员写计划、工程师改代码、分析员解读分数、审查员检查作弊。 | 分析员 alibaba-token-plan-cn/qwen3.8-max |
|---|---|
| 调用次数 | 1 次 |
| 轮数?大模型一共回复了几轮(每轮可以调用多个工具)。 | 2 |
| 工具调用?大模型调用读文件、执行命令等工具的次数,按工具名统计。 | 共 1 次:write 1 |
| 用时 | 不到 1 分 |
| token 数?输入 / 输出(含思考)的 token 数,来自接口返回的用量。 | 输入 13,179 · 输出 996 · 思考 664 |
| 任务(第一行) | 分析节点 n11 的打分结果,写 ANALYSIS.json |
| 最后的回答(摘录) | ANALYSIS.json written |
| 原始记录位置?完整对话记录在 Spark 上的文件路径;看板不复制原始内容。 | /home/spark-longxinyang/vec/runs/formal/20261002-202908-search-t1-scr-B/nodes/11/analyst.jsonl 5 KB /home/spark-longxinyang/vec/runs/formal/20261002-202908-search-t1-scr-B/nodes/11/analyst.stderr |
工程师
| 角色 | 工程师 alibaba-token-plan-cn/qwen3.8-max |
|---|---|
| 调用次数 | 1 次 |
| 轮数 | 31 |
| 工具调用 | 共 38 次:bash 23、edit 10、read 3、todowrite 1、write 1 |
| 用时 | 18 分 |
| token 数 | 输入 59,871 · 输出 13,185 · 思考 12,158 |
| 任务(第一行) | 按计划修改程序:PCA-space displacement with residual-preserving decode and tighter kNN |
| 最后的回答(摘录) | **Done.** Implemented PLAN on parent 2: per-cell displacement moved into the 30-dim coupling PCA space (barycentric ancestor vs kNN-smoothed position), residual-preserving linear decode to gene space (addnz, clip≥0), CLI flags `--lambda/--k-smooth/--beta-zero`.
**Scores (X3 A-half, seed 0):** grid over λ∈{0,1,2,2.5,3,3.5,4,5}×K∈{10,15,30,50}: best **λ=3, K=30 → 61.47** (cell_state 89.2, cov 50.3, de_rec 47.3, dir 51.4); mechanism-off control λ=0 → 50.5; parent 50.65. Dense decode (β>0) strongly harmful (51.5/37.9). Seed 1: 57.7 (seed variance ~3.8). Diagnostics confirm per-cell, non-constant steps (|step|/|resid| median 1.23, within/cluster spread 0.97). Deterministic, vec-check ok, single-input fallback verified.
**Next best directions:** de_recovery falls monotonically with λ — try decoupling strengths (large λ for cell_state PCs, shrinkage on PCs aligned with DE genes), or per-PC damping instead of scalar λ; also test whether dropping growth-resampling (uniform rows) recovers covariation at seed variance level. |
| 原始记录位置 | /home/spark-longxinyang/vec/runs/formal/20261002-202908-search-t1-scr-B/nodes/11/engineer.jsonl 192 KB /home/spark-longxinyang/vec/runs/formal/20261002-202908-search-t1-scr-B/nodes/11/engineer.stderr |
调研员
| 角色 | 调研员 alibaba-token-plan-cn/qwen3.8-max |
|---|---|
| 调用次数 | 1 次 |
| 轮数 | 1 |
| 工具调用 | 共 0 次:— |
| 用时 | 1 分 |
| token 数 | 输入 14,698 · 输出 1,244 · 思考 4,005 |
| 任务(第一行) | 为节点 n11(改进,父节点 n2)写改进计划 PLAN.json |
| 最后的回答(摘录) | .\n2. K_SMOOTH=15 may introduce noise in sparse regions. Check: if direction doesn't improve or cell_state drops, revert to K=30.\n3. The 30-min budget is tight if 9 configs need full runs. Mitigation: run all 9 on a 2000-cell subset first (each ~3s), pick top 2 by X3 A-half, then run full.\n4. If residual preservation fully restores covariation/de_recovery but eliminates cell_state gain, the mechanism is not working as intended — the PCA step must be the dominant signal.", "family_id": "local_ot", "mechanism": "Per-cell displacement from OT coupling, computed and extrapolated in the PCA subspace where the coupling was fitted, then decoded back to gene space while preserving each cell's PCA-orthogonal residual to maintain gene-specific expression structure.", "vs_constant_shift": "Each cell's displacement vector is unique: it depends on its own barycentric ancestor (from the coupling) and its own local neighborhood mean in PCA space. The residual-preserving decode further ensures gene-specific variation differs per cell. A constant shift would move all cells identically regardless of their coupling or neighborhood.", "mechanism_evidence": "Engineer should report: (1) std of step_Z across cells within each coarse cluster (if std≈0, displacement degenerated to per-type constant); (2) mean |step_gene| vs mean |residual| ratio per cell (confirms PCA step is non-trivial); (3) per-group score changes vs parent (cell_state should rise, covariation/de_recovery should not drop more than 1 point); (4) number of genes with |step|>0.01 per cell (should be >>0 and vary across cells).", "mechanism_off_control": "Run with --lambda 0 (or equivalently --no-displace flag): step_Z is zeroed, output is pure growth-weighted resampling of the last stage (identical to copy_last with growth weights). Expected: cell_state drops to ~50, direction drops to ~49, covariation returns to ~52.5. If output with λ=0 is bit-identical to λ>0, the mechanism is not running.", "sources": [] } |
| 原始记录位置 | /home/spark-longxinyang/vec/runs/formal/20261002-202908-search-t1-scr-B/nodes/11/researcher.jsonl 5 KB /home/spark-longxinyang/vec/runs/formal/20261002-202908-search-t1-scr-B/nodes/11/researcher.stderr |
审查员
| 角色 | 审查员 alibaba-token-plan-cn/qwen3.8-max |
|---|---|
| 调用次数 | 1 次 |
| 轮数 | 7 |
| 工具调用 | 共 12 次:read 9、bash 2、write 1 |
| 用时 | 1 分 |
| token 数 | 输入 22,685 · 输出 1,635 · 思考 1,481 |
| 任务(第一行) | 审查节点 n11 的程序是否越界读取、写死目标、钻评分器漏洞 |
| 最后的回答(摘录) | REVIEW.json written |
| 原始记录位置 | /home/spark-longxinyang/vec/runs/formal/20261002-202908-search-t1-scr-B/nodes/11/reviewer.jsonl 104 KB /home/spark-longxinyang/vec/runs/formal/20261002-202908-search-t1-scr-B/nodes/11/reviewer.stderr |