总览 · ← 返回运行 20261001-233756-search-t1-abc-r0-A-era

节点 n3

以最新官方输入阶段的均匀抽样子本作预测（外部阶段不作底），按类型伪批量平移保留为 α=0 默认关闭——实测各尺子上一致有害。

方法说明?节点程序自带的 METHOD.md：这个程序做了什么、为什么。

来自 programs.git 0b78a5bd86:solution/METHOD.md

调研员的计划

代码改动?这个节点的程序和父节点程序的逐行差别：绿色是新增，红色是删除。

对比：父节点版本 3988fbfbda。改动的文件：solution/METHOD.md +33 −0、solution/README.md +2 −3、solution/run.py +125 −13

diff --git a/solution/METHOD.md b/solution/METHOD.mdnew file mode 100644index 0000000..7cf9528--- /dev/null+++ b/solution/METHOD.md@@ -0,0 +1,33 @@+以最新官方输入阶段的均匀抽样子本作预测（外部阶段不作底），按类型伪批量平移保留为 α=0 默认关闭——实测各尺子上一致有害。++# official_copy（节点 3，父：pseudobulk_shift 种子）++## 方法++1. **基座选择（本节点的核心修复）**：`inputs_by_time(manifest, include_external=False)`，只用官方阶段作预测基座。父节点在 proxy2 上把外部 Qiu E9.0（仅心脏 2174 细胞、另一平台）当全胚预测输出，proxy2 只有 27.43；改为复制官方 E8.5 后 proxy2 = 50.40（A 半，与 proxy 同一输出、同一真值）。X3 没有官方/外部之分，两个输入都是本测试题自己的阶段，照常使用。+2. **平移（默认关）**：保留父节点的两阶段按类型伪批量差值机器，加了 (a) 时间比例缩放 `clip(dt_target/dt_step, 0, cap)`，(b) 每基因方差收缩 `sigma_g/(sigma_g+median sigma)`，(c) 全局系数 `--alpha`。默认 `alpha=0`，即纯复制。+3. **抽样**：默认 `sample_rows` 均匀无放回（与父相同）。另实现了按类型分层抽样（`--stratified`），实测更差，见下。++## 查分证据（全为 A 半；节点正式分用 B 半）++- X3（E8.75+E9.0 → E9.5，差值可用）：α·scale = 2→39.24，1→41.69，0.5→43.62，0.25→45.14，**0（纯复制）→ 50.00**。单调：平移越重越差；且 de_score<0（−0.086），差值方向与真实 E9.0→E9.5 变化**反相关**——0.25 天步长的差值被批次/胚胎组成噪声主导。方法卡也记载官方常数位移在真实 T1 上（48.6）低于 copy_last。两处证据一致，故默认纯复制。+- proxy（E8.5→E9.5，单阶段，本来就退化为复制）：均匀 vs 分层，3 个成对种子：50.40/48.96、49.33/48.70、49.72/49.33，均匀均值 49.82 > 分层 49.00，故默认均匀。PLAN 里"分层抽样改善协变"的假设未成立（proxy 协变：均匀 51.2–52.0 vs 分层 50.7–51.1）。+- proxy2：50.40（seed 0，输出与 proxy 逐字节相同，已实测确认同分）。++本节点预期 (proxy+proxy2+X3)/3 ≈ (50.4+50.4+50.0)/3 ≈ 50.3（A 半，seed 0），父 39.34。++## 迁移性++- 真实 T1 final（E8.5+E9.5→E10.5）：默认路径 = 复制官方 E9.5，两阶段代码路径存在但 α=0 不触发；方法卡与 X3 都表明平移不如复制，方向一致。若后续节点想重试平移，入口是 `--alpha`（建议 ≤0.25 并保留收缩）。+- 任意视图都不崩：单阶段、双阶段、无官方阶段（回退用全部输入）、`celltype` 列缺失（回退均匀抽样）都有退路。输出基因始终按视图 `genes.txt`。+- 未用任何保留阶段/禁窗信息：程序只读视图内输入；外部阶段在 proxy2 中被显式排除出基座（不是因其内容，而是因其组织组成与全胚目标不符）。无硬编码统计量。++## 验证过 / 未验证++- 验证：三个视图 `vec-check` 通过；运行 ~5 s、峰值内存远低于限额；`--seed` 确定（rng 局部）。+- 未验证：真实 final 视图（不在本 workspace）；α>0 的任何设置对正式分的影响；分层抽样在 X3/final 上的表现。++## 下一步建议++1. proxy 的 cell_state（mmd≈0.018–0.020）是最弱项：E8.5 与 E9.5 的分布差是真实发育漂移，只用允许的先验（如 prior/ 里的通路注释、已发布早期阶段的趋势）构造一个"通用发育漂移"向量小幅平移，或许能把 de_recovery/direction 拉离中性 50；X3 的教训是任何数据驱动差值都要先在尺子上证明方向正确。+2. X3 上复制恰好 50.0（de 指标为 0 的中性点），提升需要真正预测 E9.0→E9.5 变化，可试同数据集内按 embryo 配对的差值以消批次噪声。diff --git a/solution/README.md b/solution/README.mdindex ba29577..c3f928b 100644--- a/solution/README.md+++ b/solution/README.md@@ -1,4 +1,3 @@-# pseudobulk_shift+# official_copy -最新阶段抽样后，每个细胞加上所属类型在最后一步的伪批量差值 mean(last|type) − mean(prev|type)，夹到 ≥0；前一阶段没有的类型原样复制。-T1 proxy 只有一个输入阶段，没有差值可取，退化成 copy_last（同样的抽样），所以 proxy 分 = copy_last（seed 0 实测 49.77）。final 才真正平移；官方在真实 T1 上报的常数位移是 48.6，低于地板。+复制最新**官方**输入阶段的均匀抽样子本作预测（`include_external=False`，外部阶段不作底，只在全为外部时回退）。按类型伪批量平移保留在 `--alpha`（默认 0=纯复制）：X3 实测平移单调有害（α·dt比 2→39.2，0.25→45.1，0→50.0），差值方向与真实变化反相关；方法卡亦载官方常数位移低于 copy_last。分层抽样实测比均匀低 ~1 分，留在 `--stratified`。proxy 50.40 / proxy2 50.40 / X3 50.00（A 半，seed 0）。详见 METHOD.md。diff --git a/solution/run.py b/solution/run.pyindex f3a0f25..4fab61d 100644--- a/solution/run.py+++ b/solution/run.py@@ -1,13 +1,29 @@ #!/usr/bin/env python3-"""pseudobulk_shift: latest stage + per-cell-type pseudobulk delta of the last step.+"""official_copy: uniform-sampled copy of the latest *official* input stage. -The delta is mean(last|type) - mean(prev|type) over the two latest inputs,-computed on the full stages and added once to a subsample of the latest stage-(clipped at 0). Types missing from the earlier stage are copied unchanged.+Rationale (measured on the search rulers): -With a single input stage (T1 proxy: E8.5 only) there is no step to take a-delta from, so this falls back to copy_last with the same sampling. The proxy-therefore cannot tell this seed from copy_last; that gap is expected.+* proxy2 (T1): the parent pseudobulk_shift copied the external Qiu E9.0+  heart-only stage (2174 cells, wrong tissue composition) as its whole-embryo+  prediction and scored 27.4. Using ``include_external=False`` the base is the+  official E8.5 stage; the same output as proxy, ~49.+* X3 (Qiu heart E8.75+E9.0 -> E9.5): applying the per-type pseudobulk delta of+  the only available step monotonically *hurts* as its scale grows+  (alpha 2 -> 39.2, 1 -> 41.7, 0.5 -> 43.6, 0.25 -> 45.1, 0 -> 50.0, A-half).+  The extrapolated delta is even anti-correlated with the true E9.0->E9.5+  change (de_score < 0), i.e. batch/embryo composition noise dominates the+  0.25 d step. The official T1 note also reports constant pseudobulk shift+  (48.6) *below* copy_last. So the delta machinery is kept behind ``--alpha``+  (default 0) with per-gene variance shrinkage and dt-ratio scaling, but the+  default is a pure copy.+* Sampling: uniform without replacement (the harness ``sample_rows``).+  Stratified per-type sampling was implemented and tested but scored ~1 point+  lower on proxy across 3 matched seeds (49.00 vs 49.82 mean, A-half), so it+  stays behind ``--stratified`` and is off by default.++With a single input stage there is no step, and the program is a copy anyway;+the code path is the same on proxy, proxy2, X3 and the final T1 view (never+crashes on one stage), and output genes always follow the view's genes.txt. """  from __future__ import annotations@@ -15,8 +31,9 @@ from __future__ import annotations import argparse  import numpy as np+from scipy import sparse -from src.task1_temporal.baselines import shift_rows, type_deltas+from src.task1_temporal.baselines import type_deltas from src.task1_temporal.view_io import (     inputs_by_time,     labels_of,@@ -29,25 +46,120 @@ from src.task1_temporal.view_io import ( )  +def stratified_rows(labels: np.ndarray, n: int, rng: np.random.Generator) -> np.ndarray:+    """Row indices whose per-type counts match the population proportions."""+    types, counts = np.unique(labels, return_counts=True)+    total = int(counts.sum())+    alloc = np.maximum(1, np.floor(counts / total * n).astype(np.int64))+    alloc = np.minimum(alloc, counts)+    diff = n - int(alloc.sum())+    order = np.argsort(-counts, kind="stable")+    guard = 0+    while diff > 0 and guard < 10 * len(order) + n:+        t = order[guard % len(order)]+        if alloc[t] < counts[t]:+            alloc[t] += 1+            diff -= 1+        guard += 1+    guard = 0+    while diff < 0 and guard < 10 * len(order) + n:+        t = order[guard % len(order)]+        if alloc[t] > 1:+            alloc[t] -= 1+            diff += 1+        guard += 1+    rows = []+    for t, ty in enumerate(types):+        idx = np.flatnonzero(labels == ty)+        k = int(alloc[t])+        if k <= len(idx):+            rows.append(np.sort(rng.choice(idx, size=k, replace=False)))+        else:+            rows.append(np.sort(rng.choice(idx, size=k, replace=True)))+    out = np.sort(np.concatenate(rows))+    if len(out) < n:  # tiny stage with many types: top up with replacement+        extra = rng.choice(len(labels), size=n - len(out), replace=True)+        out = np.sort(np.concatenate([out, extra]))+    return out+++def gene_sigma(X: sparse.csr_matrix) -> np.ndarray:+    """Per-gene std over cells (sparse-friendly, one pass)."""+    m1 = np.asarray(X.mean(axis=0)).ravel()+    m2 = np.asarray(X.multiply(X).mean(axis=0)).ravel()+    return np.sqrt(np.maximum(m2 - m1**2, 0.0)).astype(np.float32)+++def shrink_deltas(deltas: dict, sigma: np.ndarray) -> dict:+    """Scale each gene's delta by sigma_g / (sigma_g + lambda), lambda = median sigma."""+    expressed = sigma > 1e-6+    lam = float(np.median(sigma[expressed])) if expressed.any() else 1.0+    lam = max(lam, 1e-6)+    s = (sigma / (sigma + lam)).astype(np.float32)+    s[~expressed] = 0.0+    return {t: (d * s).astype(np.float32) for t, d in deltas.items()}+++def shift_rows_scaled(X, labels, deltas: dict, scale: float) -> sparse.csr_matrix:+    """clip(x + scale*delta[type], 0) per row; types without a delta copied."""+    blocks = []+    order = []+    for t in np.unique(labels):+        idx = np.flatnonzero(labels == t)+        order.append(idx)+        key = str(t)+        if key in deltas:+            dense = np.clip(X[idx].toarray() + scale * deltas[key], 0, None).astype(np.float32)+            blocks.append(sparse.csr_matrix(dense))+        else:+            blocks.append(X[idx])+    out = sparse.vstack(blocks, format="csr")+    inv = np.empty(X.shape[0], dtype=np.int64)+    inv[np.concatenate(order)] = np.arange(X.shape[0])+    return out[inv]++ def main() -> None:     parser = argparse.ArgumentParser()     parser.add_argument("--data", required=True)     parser.add_argument("--out", required=True)     parser.add_argument("--seed", type=int, default=0)+    parser.add_argument("--alpha", type=float, default=0.0,+                        help="global multiplier on the (shrunk, dt-scaled) type delta; 0 = pure copy")+    parser.add_argument("--scale-cap", type=float, default=2.0,+                        help="cap on dt_target/dt_step extrapolation factor")+    parser.add_argument("--no-shrink", action="store_true")+    parser.add_argument("--stratified", action="store_true",+                        help="stratified per-type sampling (uniform scored ~1 pt higher on proxy over 3 seeds)")     args = parser.parse_args()      manifest = load_manifest(args.data)     genes = panel_genes(args.data, manifest)-    stages = inputs_by_time(manifest)+    stages = inputs_by_time(manifest, include_external=False)+    if not stages:+        stages = inputs_by_time(manifest)     last = read_stage(args.data, stages[-1], genes)     rng = np.random.default_rng(args.seed)-    rows = sample_rows(last.n_obs, target_n_cells(manifest, last.n_obs), rng)+    has_types = "celltype" in last.obs.columns+    labels = labels_of(last) if has_types else np.array([""] * last.n_obs)+    n_target = target_n_cells(manifest, last.n_obs)+    if args.stratified and has_types:+        rows = stratified_rows(labels, n_target, rng)+    else:+        rows = sample_rows(last.n_obs, n_target, rng)     X = last.X[rows]-    if len(stages) >= 2:++    if args.alpha > 0 and len(stages) >= 2:         prev = read_stage(args.data, stages[-2], genes)-        deltas = type_deltas(prev.X, labels_of(prev), last.X, labels_of(last))+        deltas = type_deltas(prev.X, labels_of(prev), last.X, labels)         del prev-        X = shift_rows(X, labels_of(last)[rows], deltas)+        dt_step = float(stages[-1]["time"]) - float(stages[-2]["time"])+        dt_tgt = float(manifest["target"]["time"]) - float(stages[-1]["time"])+        scale = float(np.clip(dt_tgt / dt_step, 0.0, args.scale_cap)) if dt_step > 0 else 1.0+        if not args.no_shrink:+            deltas = shrink_deltas(deltas, gene_sigma(last.X))+        X = shift_rows_scaled(X, labels[rows], deltas, args.alpha * scale)+     write_prediction(X, genes, args.out, seed=args.seed)  

调研来源?调研员查到并用到的知识条目和文献检索结果（只列标题和编号）。

用到的知识库条目

分析结果?分析员写的 ANALYSIS.json：改了什么、各组分数怎么变、假设是否成立、经验和下一步建议。

对话摘要?每个角色和大模型对话的统计：轮数、工具调用、用时、token 数和最后的回答摘录；原始记录只给路径。

只给统计和最后回答的摘录；完整对话请到原始记录位置里列出的文件看。

分析员

ANALYSIS.json written

工程师

完成。节点 3 提交为 `official_copy`。

**做了什么**
- 核心修复：预测基座只取**官方**输入阶段（`include_external=False`）。父节点在 proxy2 上把外部 Qiu E9.0（仅心脏 2174 细胞）当全胚预测，是其 27.43 的直接原因。
- X3 上实测按类型伪批量差值单调有害（α·dt比 2→39.2、1→41.7、0.5→43.6、0.25→45.1、0→**50.0**，且 de_score<0 即差值方向与真实变化反相关），与方法卡"官方常数位移低于 copy_last"一致 → 默认 α=0 纯复制，平移机器（dt 缩放 + 每基因方差收缩）保留在 `--alpha` 后。
- PLAN 的分层抽样实现并实测：3 个成对种子均低于均匀抽样（49.00 vs 49.82），故默认均匀，留在 `--stratified`。

**查分**（A 半，seed 0，共用 14/20 次）：proxy 50.40、proxy2 50.40（实测确认）、X3 50.00；预期节点分 ≈50.3 vs 父 39.42。三视图 `vec-check` 通过，运行 ~5 s。

**下一步最值得试**：proxy 最弱项是 cell_state（mmd≈0.018）——用允许的先验（prior/ 通路注释、已发布早期阶段趋势）构造"通用发育漂移"小幅平移，把 de/direction 拉离中性 50；X3 上试按 embryo 配对消批次后的差值。

调研员

 is the within-type standard deviation of gene g in the last stage, and lambda is a global regularization. Initial lambda = median(sigma_g) across all genes and types; search range [0.25x, 4x] of that median. This ensures tightly-regulated (low-variance) genes are shifted less, preserving their correlation structure. High-variance genes still get nearly the full delta, preserving directional signal.\n3. Soft clip instead of hard clip: replace clip(x,0,None) with softplus-like x_new = log(1+exp(x)) applied only to genes that would go negative, or simply np.maximum(x, 0.01 * sigma_g) to avoid exact-zero pile-up that destroys variance.\nSingle-stage fallback (proxy): no delta exists, so steps 2-3 are skipped; only stratified sampling (step 1) applies, which should slightly improve covariation over uniform sampling by preserving type proportions. This is the only change visible on proxy; expect a small gain (1-2 points) — confirm with 2 vec-score queries.\nValidation order: first run on proxy (1 query) to confirm no regression from stratified sampling; then run on proxy2 (1 query) to measure delta-shrinkage effect on covariation. If proxy2 covariation improves by >3 points, sweep lambda at {0.5x, 1x, 2x} median sigma (3 more queries). Total ≤6 queries, well within budget of 20.",
  "expected_groups": ["covariation", "cell_state"],
  "risks": "1. Stratified sampling gain on proxy may be <2 points (noise level), making it undetectable in a single query — mitigate by running 2 seeds and comparing mean. 2. Lambda too large over-shrinks the delta, degrading direction/de_recovery scores; Engineer should check all four group scores, not just covariation. 3. The sigma_g computation requires one extra pass over the expression matrix; with 30-min budget and ~1.5 GB peak memory this is fine, but Engineer should verify runtime stays <30 s. 4. If cell types have very few cells (<5), stratified sampling may fail; fall back to uniform for types below threshold.",
  "sources": []
}
```