总览 · ← 返回运行 20261002-135403-search-t1-x3-era-mechcheck

节点 n4

nnz_pseudobulk_shift：copy_last 抽样后，把每类型伪批量差值只加到该细胞原本非零的元素上（负值截 0），保留 dropout 稀疏结构。

方法说明?节点程序自带的 METHOD.md：这个程序做了什么、为什么。

来自 programs.git d6e593e5fc:solution/METHOD.md

调研员的计划

代码改动?这个节点的程序和父节点程序的逐行差别：绿色是新增，红色是删除。

对比：父节点版本 801b261cb2。改动的文件：solution/EXECUTION.json +1 −0、solution/METHOD.md +42 −0、solution/README.md +2 −3、solution/run.py +103 −10

diff --git a/solution/EXECUTION.json b/solution/EXECUTION.jsonnew file mode 100644index 0000000..9d5125c--- /dev/null+++ b/solution/EXECUTION.json@@ -0,0 +1 @@+{"gpu": false}diff --git a/solution/METHOD.md b/solution/METHOD.mdnew file mode 100644index 0000000..25a3bf0--- /dev/null+++ b/solution/METHOD.md@@ -0,0 +1,42 @@+nnz_pseudobulk_shift：copy_last 抽样后，把每类型伪批量差值只加到该细胞原本非零的元素上（负值截 0），保留 dropout 稀疏结构。++## 方法++- 读最后两个输入阶段（按 time 排序，视图无关），对同时出现在两阶段的每个细胞类型计算逐基因差值+  `delta_g = mean(last|type) - mean(prev|type)`。+- 对抽样输出的每个细胞，仅在其自身 `x_g != 0` 的位置执行 `x_g += alpha * delta_g`，随后按输出格式要求+  截到 ≥0（负值被格式检查拒绝，无法保留负数）。零元素（dropout）完全不动。+- 类型不在前一阶段 → 原样复制。单输入阶段（如 T1 proxy 只有 E8.5）→ 退化为 copy_last（同样抽样），不崩。+- 默认 `alpha=1.0`，全部有差值的基因都平移（`--z 0 --min-delta 0`），不按 z 分数筛基因。+- 纯 CPU、确定性（`np.random.default_rng(seed)`），X3 视图上 ~5s / <1GB。`EXECUTION.json` 声明 gpu=false。++## 为什么这样改（相对父节点 pseudobulk_shift）++父节点把差值稠密地加到所有元素再截 0：正差值基因的所有 dropout 位置变成非零常数，破坏稀疏+dropout 模式，X3 上 covariation 从 ~48 掉到 15.4，虽然 de_recovery 升到 47.5。本节点发现：+de_recovery 主要由「平移覆盖的基因数」决定，与幅度关系不大；covariation/cell_state 主要由+「是否改动零元素、值扭曲多大」决定。只动非零元素同时满足两边：de_recovery 46.9（接近父节点），+covariation 47.8、cell_state 48.6（接近 copy_last）。++## 验证过什么（X3 视图，vec-score A 半，seed 0）++- copy_last（本代码单阶段路径等效）≈ 47.6；父节点稠密平移 40.4。+- 只加非零元素、全基因、alpha=1.0：48.11（de 46.9 / dir 48.96 / cs 48.60 / cov 47.80）。+- 试过并放弃：稠密 add 平移（各种 z/min-delta/alpha 组合，46.9–47.6）；1/nnz 比例放大差值使均值+  完全对齐（放大伤害 cell_state/covariation，47.1–47.6）；alpha=0.75（48.12，与 1.0 持平）、+  alpha=1.5（47.86，更差）；z 筛选只平移高置信 DE 基因（de_recovery 上不去）。+- seed 0 两次运行输出 md5 相同；seed 0/1/2 均通过 vec-check。++## 没验证什么++- 未在 T1 proxy/proxy2/final 官方视图上跑分（本节点只有 X3 额度）；单输入退化路径按构造等于+  copy_last，逻辑上安全。+- alpha=1.0 在 X3（步长 0.25d、外推 0.5d）与 T1 final（步长 1d、外推 1d）含义不同；`--dt-scale`+  开关存在但默认关闭，X3 上 dt 缩放更差（47.4）。+- 差值截 0 只在负差值超过原值时发生，扭曲小；未系统比较其他 flooring。++## 知识来源++未使用任何阶段特异生物学先验（无标记基因、无类型清单、无禁窗信息）。唯一假设是通用机制性的：+相邻阶段间测得的伪批量差异是该发育方向变化的可用局部近似。全部统计量从 manifest 指定的输入+视图现场计算。diff --git a/solution/README.md b/solution/README.mdindex ba29577..b2c8edd 100644--- a/solution/README.md+++ b/solution/README.md@@ -1,4 +1,3 @@-# pseudobulk_shift+# nnz_pseudobulk_shift -最新阶段抽样后，每个细胞加上所属类型在最后一步的伪批量差值 mean(last|type) − mean(prev|type)，夹到 ≥0；前一阶段没有的类型原样复制。-T1 proxy 只有一个输入阶段，没有差值可取，退化成 copy_last（同样的抽样），所以 proxy 分 = copy_last（seed 0 实测 49.77）。final 才真正平移；官方在真实 T1 上报的常数位移是 48.6，低于地板。+最新阶段抽样后，每个细胞只在自身非零元素上加上所属类型的伪批量差值 mean(last|type) − mean(prev|type)（alpha=1.0，负值截 0），dropout 模式原样保留；前一阶段没有的类型原样复制。单输入阶段退化为 copy_last。详见 METHOD.md。diff --git a/solution/run.py b/solution/run.pyindex f3a0f25..ceaea06 100644--- a/solution/run.py+++ b/solution/run.py@@ -1,13 +1,20 @@ #!/usr/bin/env python3-"""pseudobulk_shift: latest stage + per-cell-type pseudobulk delta of the last step.+"""nnz_pseudobulk_shift: copy_last sampling + per-type pseudobulk delta applied+only to already-nonzero entries. -The delta is mean(last|type) - mean(prev|type) over the two latest inputs,-computed on the full stages and added once to a subsample of the latest stage-(clipped at 0). Types missing from the earlier stage are copied unchanged.+For every cell type present in both of the two latest input stages we compute+the per-gene delta mean(last|type) - mean(prev|type). Each sampled cell gets+x_g + alpha*delta_g added ONLY where its own x_g != 0 (then floored at 0, as+the format requires nonnegative values). Zero (dropout) entries are left+untouched, so the sparse pattern -- which carries most of the gene-gene+covariance structure -- is preserved, while per-gene means still move by+alpha*delta*nnzfrac in the direction of the observed developmental step.+Types missing from the earlier stage are copied unchanged. -With a single input stage (T1 proxy: E8.5 only) there is no step to take a-delta from, so this falls back to copy_last with the same sampling. The proxy-therefore cannot tell this seed from copy_last; that gap is expected.+With a single input stage there is no delta and this falls back to copy_last+with the same sampling. No stage-specific priors are used; the only knowledge+assumed is that a measured inter-stage pseudobulk difference is a usable+local direction of change. """  from __future__ import annotations@@ -15,8 +22,8 @@ from __future__ import annotations import argparse  import numpy as np+from scipy import sparse -from src.task1_temporal.baselines import shift_rows, type_deltas from src.task1_temporal.view_io import (     inputs_by_time,     labels_of,@@ -28,12 +35,70 @@ from src.task1_temporal.view_io import (     write_prediction, ) +EPS = 1e-8+++def col_stats(X: sparse.csr_matrix, idx: np.ndarray, chunk: int = 4096):+    """Column mean, variance and nonzero fraction of X[idx], in chunks."""+    n = len(idx)+    s = np.zeros(X.shape[1], dtype=np.float64)+    sq = np.zeros(X.shape[1], dtype=np.float64)+    nz = np.zeros(X.shape[1], dtype=np.float64)+    for a in range(0, n, chunk):+        c = X[idx[a : min(a + chunk, n)]]+        s += np.asarray(c.sum(axis=0)).ravel()+        sq += np.asarray(c.multiply(c).sum(axis=0)).ravel()+        nz += np.asarray(c.getnnz(axis=0)).ravel()+    m = s / n+    v = np.maximum(sq / n - m * m, 0.0)+    return m, v, nz / n+++def type_shifts(+    prev_X,+    prev_labels,+    last_X,+    last_labels,+    z_thresh: float,+    min_delta: float,+) -> dict[str, tuple[np.ndarray, np.ndarray]]:+    """Per type: (selected gene columns, delta on those columns)."""+    out: dict[str, tuple[np.ndarray, np.ndarray]] = {}+    prev_types = set(np.unique(prev_labels).tolist())+    for t in np.unique(last_labels):+        if t not in prev_types:+            continue+        il = np.flatnonzero(last_labels == t)+        ip = np.flatnonzero(prev_labels == t)+        if len(il) < 3 or len(ip) < 3:+            continue+        m_l, v_l, f_l = col_stats(last_X, il)+        m_p, v_p, _ = col_stats(prev_X, ip)+        d = (m_l - m_p).astype(np.float32)+        se = np.sqrt(v_l / len(il) + v_p / len(ip) + EPS)+        z = np.abs(d) / se+        sel = (z >= z_thresh) & (np.abs(d) >= min_delta)+        if sel.any():+            out[str(t)] = (np.flatnonzero(sel), d[sel], f_l[sel].astype(np.float32))+    return out+  def main() -> None:     parser = argparse.ArgumentParser()     parser.add_argument("--data", required=True)     parser.add_argument("--out", required=True)     parser.add_argument("--seed", type=int, default=0)+    parser.add_argument("--alpha", type=float, default=1.0)+    parser.add_argument("--z", type=float, default=0.0)+    parser.add_argument("--min-delta", type=float, default=0.0)+    parser.add_argument("--floor", type=float, default=0.0,+                        help="optional soft floor after shift (e.g. -0.1); None = no clip")+    parser.add_argument("--frac-floor", type=float, default=1.0)+    parser.add_argument("--dt-scale", type=int, default=0,+                        help="scale alpha by (t_target-t_last)/(t_last-t_prev)")+    parser.add_argument("--mode", choices=["add", "nnz"], default="nnz",+                        help="add: shift all entries; nnz: shift only nonzero entries, "+                             "rescaled by 1/nnz-fraction so the pseudobulk mean matches")     args = parser.parse_args()      manifest = load_manifest(args.data)@@ -43,11 +108,39 @@ def main() -> None:     rng = np.random.default_rng(args.seed)     rows = sample_rows(last.n_obs, target_n_cells(manifest, last.n_obs), rng)     X = last.X[rows]+    alpha = args.alpha+    if args.dt_scale and len(stages) >= 2:+        dt_in = float(stages[-1]["time"]) - float(stages[-2]["time"])+        dt_out = float(manifest["target"]["time"]) - float(stages[-1]["time"])+        if dt_in > 0:+            alpha *= max(0.0, dt_out / dt_in)     if len(stages) >= 2:         prev = read_stage(args.data, stages[-2], genes)-        deltas = type_deltas(prev.X, labels_of(prev), last.X, labels_of(last))+        shifts = type_shifts(+            prev.X, labels_of(prev), last.X, labels_of(last), args.z, args.min_delta+        )         del prev-        X = shift_rows(X, labels_of(last)[rows], deltas)+        if shifts:+            labs = labels_of(last)[rows]+            Xd = np.asarray(X.todense(), dtype=np.float32)+            for t, (cols, d, frac) in shifts.items():+                m = np.flatnonzero(labs == t)+                if not m.size:+                    continue+                if args.mode == "add":+                    Xd[np.ix_(m, cols)] += np.float32(alpha) * d+                else:+                    # shift only nonzero entries, inflated by 1/nnz-fraction+                    # (floor 0.25 -> at most 4x) so the per-gene mean moves by+                    # ~alpha*d while the dropout pattern is untouched+                    d_eff = np.float32(alpha) * d / np.maximum(frac, args.frac_floor)+                    sub = Xd[np.ix_(m, cols)]+                    nz = sub != 0+                    sub[nz] += np.broadcast_to(d_eff, sub.shape)[nz]+                    Xd[np.ix_(m, cols)] = sub+            if args.floor is not None:+                np.clip(Xd, args.floor, None, out=Xd)+            X = sparse.csr_matrix(Xd)     write_prediction(X, genes, args.out, seed=args.seed)  

调研来源?调研员查到并用到的知识条目和文献检索结果（只列标题和编号）。

用到的知识库条目

分析结果?分析员写的 ANALYSIS.json：改了什么、各组分数怎么变、假设是否成立、经验和下一步建议。

对话摘要?每个角色和大模型对话的统计：轮数、工具调用、用时、token 数和最后的回答摘录；原始记录只给路径。

只给统计和最后回答的摘录；完整对话请到原始记录位置里列出的文件看。

分析员

ANALYSIS.json written

工程师

Done. Final state verified: `solution/` contains run.py, METHOD.md, README.md, EXECUTION.json; default run is deterministic (md5-stable across repeats, seeds 0/1/2 pass vec-check), single-input fallback tested.

**做了什么**：父节点（稠密 pseudobulk_shift + 截 0）的 covariation 崩塌源于把差值加到所有 dropout 零元素上。新版本只把每类型伪批量差值加到细胞自身**非零**元素上（负值截 0，格式要求非负），全基因覆盖、alpha=1.0，不按 z 筛（实验证明 de_recovery 由覆盖基因数决定、与幅度/筛选关系小）。

**查分结果**（X3 A 半，seed 0，用了 13/20 次）：本方法 **48.11**（de 46.9 / dir 49.0 / cs 48.6 / cov 47.8），vs 父节点 40.43、copy_last ~47.6–47.9。放弃的变体：稠密 add（46.9–47.6）、1/nnz 比例放大（47.1–47.6）、alpha 0.75/1.5（48.12/47.86）、z 筛基因、dt-scale。

**下一步最值得试**：① 对高表达（nnz frac 高）基因用稠密 add、稀疏基因用 nnz-only 的混合平移，进一步抬 de_recovery 而不伤 covariation；② 差值先按基因表达水平做温和 shrinkage（如乘以 frac^β）；③ 在 T1 final 双官方输入上验证 alpha=1.0 是否仍是最优（X3 步长 0.25d 与 final 1d 不同）。

调研员

{"name":"Targeted DE-gene shrunk shift without hard clipping","motivation":"Parent node 2 covariation is 15.40 vs copy_last 48.44 (node 1). The uniform per-type delta applied to ALL genes then clipped at ≥0 destroys gene-gene covariance. Yet de_recovery gains +3.37 (47.46 vs 44.09), proving the delta carries real signal. The fix: shift only high-confidence DE genes with shrinkage, remove hard clip.","approach":"1) Compute per-type delta exactly as parent (mean(last|type)−mean(prev|type)). 2) Per-gene z-score: z_g = delta_g / sqrt(var_g_last/n_last + var_g_prev/n_prev). Only shift genes with |z_g|>2.0 (search [1.5, 3.0]); all others stay unchanged. 3) Apply x_new = x + alpha*delta_g for selected genes; alpha init 0.4, search [0.2, 0.6]. 4) Remove the clip-at-0 (or replace with soft floor at −0.5 in log space). 5) Types absent from prev stage: copy unchanged. 6) Single-input-stage fallback (T1 proxy): identical to copy_last (no delta computable). 7) Engineer should first run on a 500-cell subsample with vec-score to confirm covariation recovers above 35 before full run. If covariation still <30 at alpha=0.3, drop to alpha=0.15. Compare against node 1 copy_last baseline on each query.","expected_groups":["covariation","de_recovery"],"risks":"1) If most DE genes have low per-type cell counts, SE estimates are noisy and the z-threshold retains too few genes → de_recovery gain vanishes. Engineer should log the fraction of genes passing threshold per type; if <5%, lower threshold to 1.5. 2) Removing the clip may produce negative values the scorer handles oddly → check vec-score immediately after removing clip; if worse, use floor at −0.1 instead. 3) Score noise ~2 pts: need ≥2 queries at same config to confirm improvement over 40.43 is real. 4) If covariation recovers but cell_state drops further (parent already −5 vs copy_last), the net score may still be <47.92; in that case reduce alpha to 0.2 and re-check."}