总览 · ← 返回运行 20261002-034201-search-t1-abc-r1-B-population

节点 n2

输出群体一律取最新官方阶段（proxy2 的 Qiu E9.0 心脏细胞只当参考、不作输出），位移改成保零的乘法式并按 α 收缩，实测 α=0 最优。

方法说明?节点程序自带的 METHOD.md：这个程序做了什么、为什么。

来自 programs.git 75214eb730:solution/METHOD.md

调研员的计划

代码改动?这个节点的程序和父节点程序的逐行差别：绿色是新增，红色是删除。

对比：父节点版本 15cb10398e。改动的文件：solution/METHOD.md +67 −0、solution/README.md +6 −3、solution/run.py +142 −14

diff --git a/solution/METHOD.md b/solution/METHOD.mdnew file mode 100644index 0000000..77f10a9--- /dev/null+++ b/solution/METHOD.md@@ -0,0 +1,67 @@+输出群体一律取最新**官方**阶段（proxy2 的 Qiu E9.0 心脏细胞只当参考、不作输出），位移改成保零的乘法式并按 α 收缩，实测 α=0 最优。++## 方法++1. **群体来源（本节点的主要修正）**+   父节点用 `inputs_by_time(manifest)`，在 proxy2 视图里返回的“最新阶段”是外部 Qiu E9.0+   心脏细胞（2174 个、只有心脏三个谱系、面板缺 4402 个基因由 E8.5 均值补齐）。它和官方+   E8.5 的细胞类型名没有任何交集，`type_deltas` 返回空字典，于是父节点在 proxy2 上直接把+   这群心脏细胞当预测输出 → 27.43。+   本节点显式用 `inputs_by_time(manifest, include_external=False)` 作为群体与位移的基准，+   只有当视图里根本没有官方输入时才退回全部输入。外部输入阶段不参与输出（CONTRACT 也要求+   不把外部细胞当目标阶段细胞直接输出）。++2. **位移形式**：`mode="mult"`，`x' = log1p(expm1(x) * exp(α·δ_type))`，只作用在已存储的非零+   元素上，稀疏模式（零膨胀结构）完全保留。父节点的 `clip(x + δ, 0)` 会把大量 0 变成正值，+   把共变结构打乱（X3 上 covariation 15.3 vs 保零乘法 49.2）。+   δ_type = mean(last|t) − mean(prev|t)，只在两个阶段都出现的类型上算；缺的类型原样复制。++3. **收缩**：`ALPHA=0.0`（当前值）。另有 `BETA`（每 (类型,基因) 的经验贝叶斯权重+   d²/(d²+β·se²)，se² = var_last/n_last + var_prev/n_prev）与 `TSCALE`（按+   (t_target−t_last)/(t_last−t_prev) 缩放步长，上限 `TSCALE_CAP`）两个开关，实测都不如把 α 压到 0。++4. 单官方输入（proxy：只有 E8.5）没有步长可取，退化成 copy_last，抽样与父节点完全一致。++## 查分记录（A 半，seed 0）++| 配置 | proxy | proxy2 | X3 |+|---|---|---|---|+| 父节点（加性 clip 位移，proxy2 输出外部心脏细胞） | 50.04 | 27.43 | 40.53 |+| 官方群体 + 加性 clip，α=1 | – | **50.40** | 40.33 |+| 官方群体 + 加性 clip，α=0.4 / 0.2 / 0.05 | – | – | 43.02 / 44.62 / 46.78 |+| 官方群体 + 加性 clip + 每基因 EB（α=1, β=1） | – | – | 41.78 |+| 官方群体 + 保零乘法，α=1 / 0.3 / 0.25 / 0.1 | – | – | 48.50 / 48.78 / 48.80 / 48.86 |+| **提交版：保零乘法，α=0（= copy_last 官方最新阶段）** | 50.40 | 50.40 | 50.00 |++节点分估计 ≈ (50.40 + 50.40 + 50.00)/3 ≈ 50.3（父 39.34）。用了 9 次查分。++## 验证过 / 没验证过++- 验证过：三个视图（proxy / proxy2 / X3）都能跑通并通过 `vec-check`；同一 seed 输出逐字节相同；+  单视图运行 1–2 s、峰值内存 ~1.5 GB，远低于 limits。+- 验证过（重要结论）：在 X3 上 de_direction 为 **−0.068**，即 E8.75→E9.0 这一步的伪批量差值+  方向与真值 E9.0→E9.5 的变化方向**反相关**；de_recovery / direction 对 α 是尺度不变的+  （α=1 与 α=0.05 得分完全相同），所以任何 α>0 都会在 X3 上扣掉约 1.2 分。这与方法卡里+  “官方常数位移在真实 T1 上 48.6 < copy_last”一致，因此选 α=0。+- 没验证过：final 视图（E8.5+E9.5 → E10.5，本次不打分）；α>0 在 final 上是否反而有益+  （两把尺子都指向无益，但不能排除）；把 Qiu E9.0 通过标记基因映射到官方心脏类型后做+  跨数据集位移（跨批次混淆，未实现）；任何改变细胞比例的做法（保留阶段比例不可用）。++## 用到的生物学知识++只用了 CONTRACT / 方法卡里已给的通用机制知识：外部输入阶段是另一数据集、另一技术+（sci-RNA-seq3）、只含心脏谱系、细胞类型命名与官方不同，因此不能作为全胚目标阶段的群体。+没有使用任何保留阶段（E10.5 / E12.5 / 禁窗 9.5 < E ≤ 13.5）或保留基因型的测量信息，+没有读取 `uns.celltype_palette`，没有硬编码任何比例、均值或类型清单——所有统计量都在运行时+从 manifest 指向的输入阶段现算。++## 下一步最值得试++1. proxy2 上把 Qiu E9.0 用起来但不作输出：按心脏标记基因（如 Nkx2-5、Tnnt2、Isl1、Kdr）+   在官方 E8.5 里筛出心脏谱系细胞，用 Qiu 与官方的**共同基因**做分位数/均值对齐后，+   只对这部分细胞施加很小的跨数据集位移；先在对齐质量上把关（批次混淆是主要风险）。+2. 既然“最新阶段的经验云 + 不位移”是三把尺子的共同上限附近，改进空间更可能在**群体组成**+   与**采样方式**：例如按类型分层重采样以匹配目标阶段应有的细胞数上限（不引入保留阶段比例，+   只用输入阶段自身的组成 + 通用谱系知识），或输出更多细胞以降低 MMD 的抽样噪声。+3. covariation 在 α=0 时只有 50–51，说明还有空间：可试保留输入阶段的高阶结构（如按类型+   做 PCA 子空间内的重采样 / 最近邻图上的局部混合），而不是加常数位移。diff --git a/solution/README.md b/solution/README.mdindex ba29577..1256d69 100644--- a/solution/README.md+++ b/solution/README.md@@ -1,4 +1,7 @@-# pseudobulk_shift+# official_population_shift -最新阶段抽样后，每个细胞加上所属类型在最后一步的伪批量差值 mean(last|type) − mean(prev|type)，夹到 ≥0；前一阶段没有的类型原样复制。-T1 proxy 只有一个输入阶段，没有差值可取，退化成 copy_last（同样的抽样），所以 proxy 分 = copy_last（seed 0 实测 49.77）。final 才真正平移；官方在真实 T1 上报的常数位移是 48.6，低于地板。+输出群体取最新的**官方**输入阶段（外部输入阶段如 proxy2 的 Qiu E9.0 心脏细胞不作输出）；+每个细胞按其类型乘以保零的乘法位移 `log1p(expm1(x)·exp(α·δ_type))`，δ 为最后一步的伪批量差值，+α=ALPHA 全局收缩（当前 0.0，即 copy_last）。开关：BETA（每基因经验贝叶斯权重）、+TSCALE/TSCALE_CAP（按时间间隔缩放步长）、MODE（"mult" 保零 / "add" 加性 clip）。+单官方输入时退化为 copy_last。详见 METHOD.md。diff --git a/solution/run.py b/solution/run.pyindex f3a0f25..e2c968a 100644--- a/solution/run.py+++ b/solution/run.py@@ -1,13 +1,29 @@ #!/usr/bin/env python3-"""pseudobulk_shift: latest stage + per-cell-type pseudobulk delta of the last step.+"""Latest official stage as the population + optional zero-preserving per-type pseudobulk shift. -The delta is mean(last|type) - mean(prev|type) over the two latest inputs,-computed on the full stages and added once to a subsample of the latest stage-(clipped at 0). Types missing from the earlier stage are copied unchanged.+Current setting: ALPHA = 0.0, i.e. the population is copied from the latest+official stage; the shift machinery below is kept (and measured) because it is+the knob the two-stage views (proxy2, final, X3) can use. -With a single input stage (T1 proxy: E8.5 only) there is no step to take a-delta from, so this falls back to copy_last with the same sampling. The proxy-therefore cannot tell this seed from copy_last; that gap is expected.+Base population: the latest *official* input stage (external input stages such+as the Qiu E9.0 heart cells in ``proxy2`` are never used as the output+population: they cover a single lineage and part of the panel, so copying them+throws away the whole-embryo composition).++Shift: for every cell type present in the two latest official stages,+``delta = mean(last|t) - mean(prev|t)``, then++* ``alpha``  - global shrinkage of the step (alpha=0 -> copy_last),+* ``beta``   - per (type, gene) empirical-Bayes weight ``d^2 / (d^2 + beta*se^2)``+  with ``se^2 = var_last/n_last + var_prev/n_prev``; genes whose step is not+  distinguishable from sampling noise are damped, which keeps gene-gene+  covariation from being scrambled by noise,+* ``tscale`` - the step is scaled by the ratio of the predicted interval to the+  observed interval, ``(t_target - t_last) / (t_last - t_prev)``, capped, so a+  short observed step is extrapolated instead of applied verbatim.++With a single official input stage (T1 proxy: E8.5 only) there is no step, and+the program reduces to copy_last with the same sampling as the seed baseline. """  from __future__ import annotations@@ -15,10 +31,11 @@ from __future__ import annotations import argparse  import numpy as np+from scipy import sparse -from src.task1_temporal.baselines import shift_rows, type_deltas from src.task1_temporal.view_io import (     inputs_by_time,+    is_external,     labels_of,     load_manifest,     panel_genes,@@ -28,6 +45,106 @@ from src.task1_temporal.view_io import (     write_prediction, ) +ALPHA = 0.0+BETA = 0.0+TSCALE = "none"+TSCALE_CAP = 2.0+MODE = "mult"+++def as_csr(X) -> sparse.csr_matrix:+    return X.tocsr() if sparse.issparse(X) else sparse.csr_matrix(np.asarray(X))+++def group_stats(X: sparse.csr_matrix, labels: np.ndarray) -> dict[str, tuple[np.ndarray, np.ndarray, int]]:+    """Per cell type: (mean, variance, n) over genes, from a sparse log-space matrix."""+    out = {}+    for t in np.unique(labels):+        idx = np.flatnonzero(labels == t)+        sub = X[idx]+        n = len(idx)+        s = np.asarray(sub.sum(axis=0), dtype=np.float64).ravel()+        sq = np.asarray(sub.multiply(sub).sum(axis=0), dtype=np.float64).ravel()+        mean = s / n+        var = np.maximum(sq / n - mean * mean, 0.0)+        out[str(t)] = (mean, var, n)+    return out+++def shrunk_deltas(prev_X, prev_labels, last_X, last_labels, alpha: float, beta: float) -> dict[str, np.ndarray]:+    """Noise-weighted, shrunk per-type pseudobulk delta of the last observed step."""+    prev_st = group_stats(as_csr(prev_X), prev_labels)+    last_st = group_stats(as_csr(last_X), last_labels)+    out: dict[str, np.ndarray] = {}+    for t, (m_last, v_last, n_last) in last_st.items():+        if t not in prev_st:+            continue+        m_prev, v_prev, n_prev = prev_st[t]+        d = (m_last - m_prev).astype(np.float32)+        if beta > 0:+            se2 = (v_last / max(n_last, 1) + v_prev / max(n_prev, 1)).astype(np.float32)+            d2 = d * d+            w = d2 / (d2 + np.float32(beta) * se2 + np.float32(1e-12))+            d = (d * w).astype(np.float32)+        out[t] = (np.float32(alpha) * d).astype(np.float32)+    return out+++def shift_rows(X, labels, deltas: dict[str, np.ndarray], mode: str = "add") -> sparse.csr_matrix:+    """Per-type shift of a subsample; types without a delta are copied.++    ``add``: ``clip(x + delta, 0)`` (densifies the block).+    ``mult``: ``log1p(expm1(x) * exp(delta))`` applied to stored entries only, so+    the sparsity pattern - and with it the zero-inflation that drives the+    covariation structure - is preserved exactly.+    """+    X = as_csr(X)+    if mode == "mult":+        types = [str(t) for t in np.unique(labels)]+        tpos = {t: k for k, t in enumerate(types)}+        n_genes = X.shape[1]+        D = np.zeros((len(types), n_genes), dtype=np.float32)+        for k, t in enumerate(types):+            d = deltas.get(t)+            if d is not None:+                D[k] = np.clip(d, -20.0, 20.0)+        row_t = np.array([tpos[str(t)] for t in labels], dtype=np.int64)+        out = X.copy()+        rows = np.repeat(np.arange(X.shape[0]), np.diff(out.indptr))+        cols = out.indices+        factor = np.exp(D[row_t[rows], cols], dtype=np.float32)+        vals = out.data+        newvals = np.expm1(vals).astype(np.float32) * factor+        out.data = np.log1p(np.maximum(newvals, 0.0)).astype(np.float32)+        return out+    blocks = []+    order = []+    for t in np.unique(labels):+        idx = np.flatnonzero(labels == t)+        order.append(idx)+        d = deltas.get(str(t))+        if d is not None and np.any(d != 0):+            dense = np.clip(X[idx].toarray() + d, 0, None).astype(np.float32)+            blocks.append(sparse.csr_matrix(dense))+        else:+            blocks.append(X[idx])+    out = sparse.vstack(blocks, format="csr")+    inv = np.empty(X.shape[0], dtype=np.int64)+    inv[np.concatenate(order)] = np.arange(X.shape[0])+    return out[inv]+++def step_scale(manifest: dict, base: list[dict], mode: str, cap: float) -> float:+    """Ratio of the interval to predict to the interval observed, capped."""+    if mode != "linear" or len(base) < 2:+        return 1.0+    t_prev, t_last = base[-2]["time"], base[-1]["time"]+    t_tgt = manifest.get("target", {}).get("time")+    if t_tgt is None or t_last <= t_prev:+        return 1.0+    r = (float(t_tgt) - float(t_last)) / (float(t_last) - float(t_prev))+    return float(np.clip(r, 0.0, cap))+  def main() -> None:     parser = argparse.ArgumentParser()@@ -38,16 +155,27 @@ def main() -> None:      manifest = load_manifest(args.data)     genes = panel_genes(args.data, manifest)-    stages = inputs_by_time(manifest)-    last = read_stage(args.data, stages[-1], genes)++    base = inputs_by_time(manifest, include_external=False)+    if not base:  # a view whose only inputs are external+        base = sorted(manifest["inputs"], key=lambda e: e["time"])++    last_entry = base[-1]+    last = read_stage(args.data, last_entry, genes)     rng = np.random.default_rng(args.seed)     rows = sample_rows(last.n_obs, target_n_cells(manifest, last.n_obs), rng)+    labels = labels_of(last)     X = last.X[rows]-    if len(stages) >= 2:-        prev = read_stage(args.data, stages[-2], genes)-        deltas = type_deltas(prev.X, labels_of(prev), last.X, labels_of(last))++    if ALPHA != 0.0 and len(base) >= 2:+        prev_entry = base[-2]+        prev = read_stage(args.data, prev_entry, genes, missing="error" if not is_external(prev_entry) else "fill")+        scale = ALPHA * step_scale(manifest, base, TSCALE, TSCALE_CAP)+        deltas = shrunk_deltas(prev.X, labels_of(prev), last.X, labels, scale, BETA)         del prev-        X = shift_rows(X, labels_of(last)[rows], deltas)+        if deltas:+            X = shift_rows(X, labels[rows], deltas, MODE)+     write_prediction(X, genes, args.out, seed=args.seed)  

调研来源?调研员查到并用到的知识条目和文献检索结果（只列标题和编号）。

用到的知识库条目

分析结果?分析员写的 ANALYSIS.json：改了什么、各组分数怎么变、假设是否成立、经验和下一步建议。

对话摘要?每个角色和大模型对话的统计：轮数、工具调用、用时、token 数和最后的回答摘录；原始记录只给路径。

只给统计和最后回答的摘录；完整对话请到原始记录位置里列出的文件看。

分析员

ANALYSIS.json written

工程师

提交完成。

**做了什么**：发现父节点在 proxy2 上把外部 Qiu E9.0 心脏细胞（2174 个、单谱系、缺 4402 基因）当成了输出群体——因为它的类型名与官方 E8.5 无交集，`type_deltas` 返回空，退化成复制外部数据。改为群体一律取最新**官方**阶段（`include_external=False`），并把加性 `clip(x+δ,0)` 位移换成保零的乘法位移 `log1p(expm1(x)·exp(α·δ))`，加 α 全局收缩、每基因 EB 权重（β）、时间间隔缩放三个开关。

**查分（9/20 次，A 半）**：proxy2 27.43→**50.40**；proxy 50.04→**50.40**；X3：加性 α=1 40.33 / α=0.4 43.02 / α=0.05 46.78，乘法 α=1 48.50 / α=0.25 48.80 / α=0 **50.00**。关键实测：X3 上 de_direction = −0.068（观测步长方向与真值反相关），且 de_recovery/direction 对 α 尺度不变，所以任何 α>0 都扣 ~1.2 分 → 提交版取 α=0，节点分估计 ≈50.3（父 39.34）。三视图均通过 `vec-check`，同 seed 输出逐字节一致，运行 ~2 s / 1.5 GB。

**下一步最值得试**：(1) 把 Qiu E9.0 用作心脏谱系的跨数据集先验而非输出——按共同基因做分位数对齐后只对该子群施加小位移；(2) 改进群体组成本身（分层重采样、增加输出细胞数降低 MMD 抽样噪声），因为 α=0 时 covariation 只有 50–51，说明上限不在位移而在分布结构。

调研员

k018 states full delta (alpha=1) scores below copy_last, so the unshrunk shift is over-aggressive. On proxy2 the delta spans two datasets/technologies (official E8.5 vs Qiu E9.0 heart), amplifying noise; the noisy per-gene deltas distort gene–gene correlations, explaining the low covariation. The 22.6-point gap between proxy and proxy2 is almost entirely caused by the unshrunk cross-dataset delta.",
  "approach": "1) Keep type_deltas and sampling from parent. 2) Add global shrinkage: delta *= alpha; initial alpha=0.3, grid {0.15, 0.3, 0.5, 0.7}. 3) Per-gene empirical-Bayes shrinkage on top: for gene g compute v_g = variance of delta_g across types; shrink factor s_g = v0 / (v0 + v_g) where v0 = median(v_g). Final delta_g *= alpha * s_g. This suppresses noisy genes that break covariation while keeping well-estimated genes. 4) x_new = clip(x + shrunk_delta, 0); types absent from prev stage copied unchanged (already in parent). 5) Single-input fallback (proxy, X3 if single-stage): copy_last, identical to parent—no regression risk. 6) Quick screen via vec-score: run alpha=0.3 on proxy2 first (the ruler where delta is active); if score > 29 (≥ 2 above 27.43, clearing T1 noise ≈ 2), test alpha=0.15 and 0.5; else try alpha=0.5, 0.7. Expect ≤ 6 queries. 7) For final run use the best alpha; seed 0.",
  "expected_groups": ["covariation", "direction"],
  "risks": "If alpha is too small the prediction collapses toward copy_last and proxy2 gains < 2 points; detect immediately by comparing vec-score at alpha=0.3 vs the parent's 27.43. Per-gene shrinkage may over-regularise if v0 is poorly chosen; Engineer should print the distribution of s_g and ensure it spans roughly [0.2, 1]. If proxy2's cross-dataset delta is fundamentally uninformative, no amount of shrinkage helps beyond copy_last; in that case fall back to alpha=0 (copy_last) for proxy2 and focus on not regressing proxy/X3. 30 min is sufficient: the code change is ~20 lines on top of existing type_deltas/shift_rows."
}
```