总览 · ← 返回运行 20261003-153851-search-t1-D-smoke2h

节点 n2 在终选来历上

ot_moscot — Waddington-OT / moscot coupling, one-step displacement extrapolation

方法说明?节点程序自带的 METHOD.md：这个程序做了什么、为什么。

来自 programs.git 3554b5ef18:solution/METHOD.md

调研员的计划

代码改动?这个节点的程序和父节点程序的逐行差别：绿色是新增，红色是删除。

对比：这个提交的上一版（种子程序：相对空仓库）。改动的文件：solution/EXECUTION.json +1 −0、solution/METHOD.md +81 −0、solution/README.md +4 −0、solution/g37_common.py +101 −0、solution/run.py +129 −0

diff --git a/solution/EXECUTION.json b/solution/EXECUTION.jsonnew file mode 100644index 0000000..9d5125c--- /dev/null+++ b/solution/EXECUTION.json@@ -0,0 +1 @@+{"gpu": false}diff --git a/solution/METHOD.md b/solution/METHOD.mdnew file mode 100644index 0000000..06c26ae--- /dev/null+++ b/solution/METHOD.md@@ -0,0 +1,81 @@+# ot_moscot — Waddington-OT / moscot coupling, one-step displacement extrapolation++Seed (2026-10-02) made from the G37 candidate `modeling/candidates/T1/ot_moscot/` (commit 227eeb2). Same method and+hyper-parameters. Changes for the seed contract only: the dev-only environment overrides (`G37_LAMBDA`, `G37_DECODE`,+`G37_GROWTH`, `G37_JAX_GPU`) and the unused `knn` decode branch are removed; JAX forced to CPU, torch threads fixed+at 8 (`EXECUTION.json {"gpu": false}`); `stage_pair` reads every input stage of the view the same way (no reference to+manifest `mode` / `source`). Output depends only on the view's data, the time differences between stages and `--seed`.++Contract: `python run.py --data <view> --out <pred.h5ad> --seed <int>`; `g37_common.py` must stay next to `run.py`.++## Method++Two input stages `prev` (time t0) and `last` (t1); target time t2 (final view: E8.5, E9.5 -> E10.5).++1. **Embedding.** Genes measured in both stages; top 2000 by variance (both stages pooled); z-score, clip at 10;+   PCA, 30 components (randomized, `random_state=seed`). Fitted on the input stages only.+2. **Growth prior (Waddington-OT).** Proliferation / apoptosis scores (`scanpy.tl.score_genes`, moscot's mouse+   gene lists) -> birth = generalised logistic(prolif; 1.7, 0.3, 0.25, 0.5), death = logistic(apopt; 1.7, 0.3, 0.1,+   0.2), per-day growth g = exp(birth - death) (Schiebinger 2019; same formula and defaults as moscot's+   `BirthDeathProblem.estimate_marginals`).+3. **Coupling.** `moscot.problems.time.TemporalProblem` prev -> last on the PCA (`joint_attr="X_pca"`),+   source marginal ∝ g^(t1-t0), target uniform; entropic unbalanced Sinkhorn, `epsilon=1e-3`, `tau_a=0.95`,+   `tau_b=1`, `scale_cost="mean"` (moscot tutorial settings). JAX on CPU unless `G37_JAX_GPU=1`.+4. **Output cells.** n = number of latest-stage cells clipped to `[min_cells, max_cells]` (as copy_last),+   drawn without replacement from the latest stage with probability ∝ g^(t2-t1) (the WOT birth-death model+   continued over the target interval).+5. **Displacement extrapolation.** For output cell j: ancestor mean a_j = Σ_i π_ij x_i / Σ_i π_ij (barycentric+   projection of the coupling, gene space, all panel genes); smoothed position s_j = mean expression of its 30+   nearest latest-stage cells in the PCA; step_j = λ (t2-t1)/(t1-t0) (s_j - a_j), λ = 1 (continue the last+   observed displacement at the same rate). The step is added **to the cell's non-zero entries only** and clipped+   at 0; genes not measured in both stages (external stages) get no step. Each cell keeps its own residual.++**One input stage (proxy view, E8.5 only):** steps 1, 3, 5 need two stages; only steps 2 + 4 run, i.e. a+growth-weighted copy of the latest stage (g^(t2-t1) resampling). This is the only thing the proxy can test.++Sources:+- Schiebinger G. et al. Optimal-transport analysis of single-cell gene expression identifies developmental+  trajectories in reprogramming. *Cell* 176, 928–943 (2019). doi:10.1016/j.cell.2019.01.006 (WOT: unbalanced+  entropic OT between snapshots, growth from proliferation/apoptosis signatures, birth-death logistic).+- Klein D., Palla G., Lange M. et al. Mapping cells through time and space with moscot. *Nature* 638, 1065–1075+  (2025). doi:10.1038/s41586-024-08453-2 (TemporalProblem; code moscot 0.5.2, BSD-3).+- Cuturi M. Sinkhorn distances. NeurIPS 2013; Chizat L. et al. Scaling algorithms for unbalanced optimal+  transport problems. *Math. Comp.* 87, 2563–2609 (2018).+- Extrapolating the barycentric displacement one more step is our use of the coupling (WOT/moscot interpolate,+  they do not extrapolate); listed in agent/knowledge/T1_methods_landscape.md §1.++## Data / knowledge used++Only the view's input stages. Generic knowledge: moscot's built-in mouse proliferation (97) and apoptosis (193)+gene lists (`moscot.utils.data`, from the WOT paper; stage-agnostic gene-function annotation). No held-out stage,+no information from (E9.5, E13.5], no pre-trained weights.++## Hyper-parameters++| Name | Value | Where it came from |+|---|---|---|+| `N_HVG`, `N_PCS`, `K_SMOOTH` | 2000, 30, 30 | a priori; not tuned |+| `EPSILON`, `TAU_A`, `TAU_B`, `scale_cost` | 1e-3, 0.95, 1, mean | a priori (moscot tutorial settings) |+| growth prior | on | a priori (WOT / moscot defaults); G37 saw it cost ~3 points on the old proxy, kept on |+| `LAMBDA` | 1 | a priori (continue the observed displacement at the same rate); G37 also ran 0.5 on X3 (50.4 vs 50.2), not changed |+| decode `addnz` | — | **chosen on the X3 ruler (G37)** against `add` (25.6) and `knn` (48.7): the dense step destroys the zero pattern. Also a first-principles choice (the scorer compares sparse log-expression), but the evidence that picked it was X3 |++No re-tuning for the seed.++## Resources (Spark, CPU)++Final view (16.8k x 17.1k coupling): 69 s, max RSS 6.7 GB; proxy 4 s / 1.6 GB; X3 ~ 8 s / 2.3 GB; proxy2 ~ 20 s / 4.3 GB.+The dense coupling (n_prev x n_last float32, ~1.1 GB on final) and the dense earlier stage (~2.2 GB) dominate memory.++## Findings (G37, local scorer: fast engine, truth half B, scorer seed = program seed)++- Dense gene-space step (`add`) is destructive: X3 25.6 at λ=1, still 32.9 at λ=0.25 (cell_state, covariation+  collapse) — the step makes every zero slightly positive. `addnz` fixes it (X3 50.2 at λ=1, 50.4 at λ=0.5);+  `knn` decode 48.7 (λ=1). Growth resampling has no effect on X3/proxy2 (all cells are kept there).+- proxy2 (E8.5 official -> Qiu E9.0 heart, then +0.5 d): every variant ≈ copy of the Qiu cells (~27.5), the+  cross-dataset step is batch effect.+- Full eval (seeds 0-2, half B): proxy 46.8 / 46.8 / 47.9 (copy_last 50.0 / 50.3 / 50.1) — the WOT growth+  resampling alone costs ~3 points on E8.5 -> E9.5 (direction, cell_state); X3 50.2 / 50.1 / 49.8 (copy_last 50.0,+  pseudobulk_shift 40.5-40.7); proxy2 27.7-27.9 (copy_last 27.4-27.6). `G37_GROWTH=0` turns the proxy into copy_last.+- Final-view prediction (`~/vec/scratch/g37/ot_moscot/final.h5ad`, seed 0): board-cap cells, only the last input's types,+  composition close to the last input's; no new states (the method cannot create them).diff --git a/solution/README.md b/solution/README.mdnew file mode 100644index 0000000..dc1cbaf--- /dev/null+++ b/solution/README.md@@ -0,0 +1,4 @@+# ot_moscot++Waddington-OT / moscot：两个最新输入阶段在联合 PCA 上做非平衡熵 OT 耦合（WOT 增殖/凋亡先验生长），最新阶段按 g^Δt 重抽样，每个细胞沿“kNN 平滑位置 − 耦合祖先均值”再走一个等比例步长，只加在非零基因上，夹到 ≥0。+来自 G37 候选 modeling/candidates/T1/ot_moscot；addnz 解码是在 X3 上选的（详见 METHOD.md）。纯 CPU，final 约 70 s、6.7 GB。diff --git a/solution/g37_common.py b/solution/g37_common.pynew file mode 100644index 0000000..8c991b3--- /dev/null+++ b/solution/g37_common.py@@ -0,0 +1,101 @@+"""Helpers of the ot_moscot seed (copy of modeling/candidates/T1/ot_moscot/g37_common.py, G37; stage_pair reads every+input stage of the view).++- ``stage_pair``: the two latest input stages (official, or external in proxy2 / test views) and the gene mask on+  which both are really measured (external stages lack part of the panel; filled columns must not drive dynamics);+- ``embed``: HVG -> z-score (clip 10) -> PCA, fitted on the input stages only (scanpy-style preprocessing);+- ``knn_mean``: per-cell kNN average in the embedding (smooths single-cell noise before taking displacements);+- ``growth_rates``: Waddington-OT / moscot prior growth from proliferation and apoptosis gene scores.+"""++from __future__ import annotations++import numpy as np+from scipy import sparse++from src.task1_temporal.view_io import covered_mask, inputs_by_time, read_stage+++def stage_pair(view, manifest: dict, genes: list[str]):+    """(prev, last, mask, dt_in) for the two latest inputs; prev None with a single input stage."""+    stages = inputs_by_time(manifest, include_external=True)  # every input stage, whatever its source+    last_e = stages[-1]+    last = read_stage(view, last_e, genes)+    mask = covered_mask(view, last_e, genes)+    if len(stages) < 2:+        return None, last, mask, None, last_e+    prev_e = stages[-2]+    prev = read_stage(view, prev_e, genes)+    mask &= covered_mask(view, prev_e, genes)+    return prev, last, mask, float(last_e["time"] - prev_e["time"]), last_e+++def _gene_moments(X: sparse.csr_matrix):+    n = X.shape[0]+    mean = np.asarray(X.mean(axis=0)).ravel()+    sq = np.asarray(X.multiply(X).sum(axis=0)).ravel() / n+    return mean, np.maximum(sq - mean ** 2, 0.0)+++def embed(mats: list, mask: np.ndarray, n_hvg: int, n_pcs: int, seed: int):+    """PCA coordinates of each matrix in ``mats`` (same order), fitted on all of them together.++    Returns (list of Z, info) with info = {hvg, mean, std, components} so displacements can be decoded."""+    from sklearn.decomposition import PCA++    allX = sparse.vstack(mats).tocsr()+    mean, var = _gene_moments(allX)+    var = np.where(mask, var, -1.0)+    hvg = np.sort(np.argsort(var)[::-1][:n_hvg])+    sub = allX[:, hvg].toarray().astype(np.float32)+    mu, sd = mean[hvg].astype(np.float32), np.sqrt(var[hvg]).astype(np.float32)+    sd[sd == 0] = 1.0+    sub -= mu+    sub /= sd+    np.clip(sub, -10, 10, out=sub)+    pca = PCA(n_components=n_pcs, svd_solver="randomized", random_state=seed)+    Z = pca.fit_transform(sub).astype(np.float32)+    out, start = [], 0+    for M in mats:+        out.append(Z[start:start + M.shape[0]])+        start += M.shape[0]+    return out, {"hvg": hvg, "mu": mu, "sd": sd, "components": pca.components_.astype(np.float32)}+++def knn_mean(Z: np.ndarray, X: sparse.csr_matrix, rows: np.ndarray, k: int) -> np.ndarray:+    """Dense (len(rows), n_genes): mean expression of the k nearest cells (in Z, the cell itself included)."""+    from sklearn.neighbors import NearestNeighbors++    nn = NearestNeighbors(n_neighbors=k).fit(Z)+    idx = nn.kneighbors(Z[rows], return_distance=False)+    W = sparse.csr_matrix((np.full(idx.size, 1.0 / k, dtype=np.float32), idx.ravel(),+                           np.arange(0, idx.size + 1, k)), shape=(len(rows), Z.shape[0]))+    return np.asarray((W @ X).todense(), dtype=np.float32)+++# Waddington-OT prior growth (Schiebinger et al. 2019, Cell; as implemented in moscot.base.problems.birth_death):+# birth = generalised logistic of the proliferation score, death = of the apoptosis score, g = exp(birth - death) / day.+def _gen_logistic(p, sup, inf, center, width):+    return inf + (sup - inf) / (1 + np.exp(-(p - center) / width))+++def growth_rates(adata_list: list, genes: list[str], mask: np.ndarray, seed: int) -> list[np.ndarray]:+    """Per-day growth rate of every cell of each AnnData (scores computed on all of them jointly)."""+    import anndata as ad+    import scanpy as sc+    from moscot.utils.data import apoptosis_markers, proliferation_markers++    ok = set(np.asarray(genes)[mask])+    joint = ad.concat(adata_list, join="inner")+    prolif = [g for g in proliferation_markers("mouse") if g in ok]+    apopt = [g for g in apoptosis_markers("mouse") if g in ok]+    sc.tl.score_genes(joint, prolif, score_name="proliferation", random_state=seed)+    sc.tl.score_genes(joint, apopt, score_name="apoptosis", random_state=seed)+    birth = _gen_logistic(joint.obs["proliferation"].to_numpy(float), 1.7, 0.3, 0.25, 0.5)+    death = _gen_logistic(joint.obs["apoptosis"].to_numpy(float), 1.7, 0.3, 0.1, 0.2)+    g = np.exp(birth - death)+    out, start = [], 0+    for a in adata_list:+        out.append(g[start:start + a.n_obs])+        start += a.n_obs+    return outdiff --git a/solution/run.py b/solution/run.pynew file mode 100644index 0000000..a8a575c--- /dev/null+++ b/solution/run.py@@ -0,0 +1,129 @@+#!/usr/bin/env python3+"""ot_moscot: Waddington-OT / moscot TemporalProblem coupling, extrapolated one step past the latest input stage.++Two input stages (final: E8.5, E9.5):+  1. joint PCA of both stages (HVG, z-score, 30 PCs; fitted on the inputs only);+  2. moscot TemporalProblem prev -> last on the PCA, source marginals from Waddington-OT prior growth+     (proliferation / apoptosis gene scores), entropic unbalanced Sinkhorn (epsilon 1e-3, tau_a 0.95);+  3. output cells = latest-stage cells resampled with weights g^dt_out (prior growth rate continued for the target+     interval, Waddington-OT birth-death model);+  4. each output cell j moves by LAMBDA * dt_out / dt_in * (kNN-mean(x_j) - ancestor_mean_j) in gene space, where+     ancestor_mean_j is the coupling-weighted (barycentric) mean of its ancestors in the earlier stage: the last+     observed displacement continued for the target interval; the cell keeps its own residual. The step is applied+     to the cell's non-zero entries only (DECODE "addnz": a dense step turns every zero into a small positive value+     and wrecks cell_state / covariation, see METHOD.md). Clipped at 0.+--ablate (G39.7): `growth` turns step 3 off (uniform resampling); any other name turns the OT displacement (steps 1, 2,+4) off, leaving the growth-weighted copy of the latest stage.+One input stage (proxy: E8.5 only): steps 1, 2, 4 need two stages; only the growth resampling (3) runs, i.e.+a growth-weighted copy of the latest stage.++Seed version (agent/seeds/T1__val/ot_moscot, 2026-10-02) of modeling/candidates/T1/ot_moscot (G37): same method and+hyper-parameters; the dev-only environment overrides (G37_LAMBDA / G37_DECODE / G37_GROWTH / G37_JAX_GPU) and the+`knn` decode branch are removed; JAX and torch on CPU, fixed thread count (EXECUTION.json gpu false).+Parameter provenance (METHOD.md): DECODE = addnz was chosen on the X3 ruler (G37); LAMBDA = 1 a priori, also checked+on X3 (0.5 vs 1 within 0.2).+"""++from __future__ import annotations++import argparse+import os+import sys+from pathlib import Path++os.environ.setdefault("XLA_PYTHON_CLIENT_PREALLOCATE", "false")+os.environ.setdefault("XLA_PYTHON_CLIENT_MEM_FRACTION", "0.1")+os.environ["JAX_PLATFORMS"] = "cpu"  # CPU only: deterministic, no GPU slot++import numpy as np++sys.path.insert(0, str(Path(__file__).resolve().parent))+from g37_common import embed, growth_rates, knn_mean, stage_pair  # noqa: E402++from src.task1_temporal.view_io import load_manifest, panel_genes, target_n_cells, write_prediction  # noqa: E402++N_HVG = 2000+N_PCS = 30+EPSILON = 1e-3+TAU_A = 0.95+TAU_B = 1.0+K_SMOOTH = 30+LAMBDA = 1.0     # 1 = continue the observed displacement at full rate+GROWTH = True    # resample output cells by g^dt_out (WOT birth-death model)+N_THREADS = 8+++def weighted_rows(w: np.ndarray, n: int, rng: np.random.Generator) -> np.ndarray:+    p = w / w.sum()+    return np.sort(rng.choice(len(w), size=n, replace=n > len(w), p=p))+++def main() -> None:+    parser = argparse.ArgumentParser()+    parser.add_argument("--data", required=True)+    parser.add_argument("--out", required=True)+    parser.add_argument("--seed", type=int, default=0)+    # G39.7 mechanism-off control: `growth` -> no growth resampling (uniform); any other name (the OT displacement,+    # e.g. transport / displacement / mechanism) -> no displacement step: the growth-weighted copy of the latest stage+    parser.add_argument("--ablate", default=None)+    args = parser.parse_args()++    manifest = load_manifest(args.data)+    genes = panel_genes(args.data, manifest)+    prev, last, mask, dt_in, last_e = stage_pair(args.data, manifest, genes)+    dt_out = float(manifest["target"]["time"] - last_e["time"])+    rng = np.random.default_rng(args.seed)+    n = target_n_cells(manifest, last.n_obs)+    growth_on = GROWTH and args.ablate != "growth"++    if prev is None or (args.ablate and args.ablate != "growth"):+        (g_last,) = growth_rates([last], genes, mask, args.seed)+        rows = weighted_rows(g_last ** dt_out if growth_on else np.ones(last.n_obs), n, rng)+        write_prediction(last.X[rows], genes, args.out, seed=args.seed)+        return++    import anndata as ad+    import pandas as pd+    import torch+    from moscot.problems.time import TemporalProblem++    (Zp, Zl), _ = embed([prev.X, last.X], mask, N_HVG, N_PCS, args.seed)+    g_prev, g_last = growth_rates([prev, last], genes, mask, args.seed)++    obs = pd.DataFrame({"time": np.r_[np.zeros(prev.n_obs), np.ones(last.n_obs)],+                        "growth": np.r_[g_prev ** dt_in, g_last ** dt_in]},+                       index=[f"p{i}" for i in range(prev.n_obs)] + [f"l{i}" for i in range(last.n_obs)])+    obs["time"] = obs["time"].astype(float)+    small = ad.AnnData(X=np.zeros((len(obs), 1), dtype=np.float32), obs=obs)+    small.obsm["X_pca"] = np.vstack([Zp, Zl])+    tp = TemporalProblem(small)+    # source marginals = WOT prior growth over the input interval (normalised by moscot); target uniform+    tp = tp.prepare(time_key="time", joint_attr="X_pca", a="growth")+    tp = tp.solve(epsilon=EPSILON, tau_a=TAU_A, tau_b=TAU_B, scale_cost="mean")+    sol = tp[(0.0, 1.0)].solution+    print(f"ot: converged={getattr(sol, 'converged', None)} cost={getattr(sol, 'cost', None)}", file=sys.stderr)+    P = np.asarray(sol.transport_matrix, dtype=np.float32)  # (n_prev, n_last)++    rows = weighted_rows(g_last ** dt_out if growth_on else np.ones(last.n_obs), n, rng)+    print(f"diag: growth last min/median/max {g_last.min():.3f}/{np.median(g_last):.3f}/{g_last.max():.3f}", file=sys.stderr)+    Pc = P[:, rows]+    del P+    Pc /= np.maximum(Pc.sum(axis=0, keepdims=True), 1e-30)+    factor = LAMBDA * dt_out / dt_in+    torch.set_num_threads(N_THREADS)+    Xp = torch.from_numpy(prev.X.toarray())+    anc = (torch.from_numpy(Pc).T @ Xp).numpy()  # barycentric ancestor mean, (n, n_genes)+    del Xp, Pc+    smooth = knn_mean(Zl, last.X, rows, K_SMOOTH)+    step = (smooth - anc) * factor+    del smooth, anc+    step[:, ~mask] = 0.0+    X = last.X[rows].toarray()+    step *= X > 0  # "addnz": move only measured (non-zero) entries, keeps each cell's zero pattern+    X += step+    np.maximum(X, 0.0, out=X)+    write_prediction(X, genes, args.out, seed=args.seed)+++if __name__ == "__main__":+    main()

调研来源?调研员查到并用到的知识条目和文献检索结果（只列标题和编号）。

没有记录调研来源。

分析结果?分析员写的 ANALYSIS.json：改了什么、各组分数怎么变、假设是否成立、经验和下一步建议。

没有分析结果（ANALYSIS.json）。

对话摘要?每个角色和大模型对话的统计：轮数、工具调用、用时、token 数和最后的回答摘录；原始记录只给路径。

只给统计和最后回答的摘录；完整对话请到原始记录位置?完整对话记录在 Spark 上的文件路径；看板不复制原始内容。里列出的文件看。

这个节点没有大模型对话记录。