总览 · ← 返回运行 20261003-093415-search-t1-r2-D-s1

节点 n18

CollecTRI+ULM 调控程序族：两阶段按细胞类型估 TF 活性差并经签名调控网解码回表达；X3 上解码不敌其乱网对照，故默认 λ=0（机制关闭，≈copy_last），负结果如实上报。

方法说明?节点程序自带的 METHOD.md：这个程序做了什么、为什么。

来自 programs.git 217811c9ea:solution/METHOD.md

调研员的计划

代码改动?这个节点的程序和父节点程序的逐行差别：绿色是新增，红色是删除。

对比：这个提交的上一版（种子程序：相对空仓库）。改动的文件：solution/EXECUTION.json +1 −0、solution/METHOD.md +39 −0、solution/run.py +229 −0

diff --git a/solution/EXECUTION.json b/solution/EXECUTION.jsonnew file mode 100644index 0000000..9d5125c--- /dev/null+++ b/solution/EXECUTION.json@@ -0,0 +1 @@+{"gpu": false}diff --git a/solution/METHOD.md b/solution/METHOD.mdnew file mode 100644index 0000000..150aea5--- /dev/null+++ b/solution/METHOD.md@@ -0,0 +1,39 @@+CollecTRI+ULM 调控程序族：两阶段按细胞类型估 TF 活性差并经签名调控网解码回表达；X3 上解码不敌其乱网对照，故默认 λ=0（机制关闭，≈copy_last），负结果如实上报。++## 方法（family: regulatory_program，按 PLAN 实现）++- 网络：`prior/tf_regulons/collectri_mouse.tsv.gz`，过滤到视图 `genes.txt` 内的 TF 与靶基因（1086 TF、37,105 条有符号边），source/target/weight(=sign(mor)) 去重。+- 活性：decoupler 2.2 `dc.mt.ulm(adata, net, tmin=5, raw=False)`（数据已是 log1p CP10k）。ULM 会按数据集 prune TF，两阶段的 TF 列取交集后再差分。+- **两阶段路径**（X3/final，≥2 输入）：对每个在最后两个输入阶段都出现的细胞类型，ΔA = mean(A_last) − mean(A_prev)，乘 scale = β·T，T=(t_target−t_last)/(t_last−t_prev) 截断到 ≤2（只用时间差，视图无关）。解码：Δexpr[type,g] = λ/n_reg(g) · Σ_tf sign(tf→g)·ΔA[type,tf]，仅 n_reg≥3 的基因（9.1% 面板），逐基因 clip ±0.5，且**只加在原本表达的条目上**（保住稀疏结构；未 mask 时 variogram skill 从 0.49 崩到 0.30）。输出 = 末阶段细胞均匀抽样 + 按类型加位移。+- **单输入路径**（proxy10）：均匀抽样复制末阶段。PLAN 的"调控偏差代表性细胞选择"（RP_SELECT=1 保留实现）实测把 proxy10 从 52.97 拉到 46.07（收窄分布伤 mmd_u/variogram），故默认关闭。+- **off 对照**：`--shuffle_regulon` 对 target 列做随机置换（保持每个 TF 出度、每个基因入度），其余步骤逐字节相同。++## 查分结果（A 半；vec-score）++| 变体 | X3 | X3 分组 (de/dir/state/cov) | proxy10 |+|---|---|---|---|+| copy_last 基线（node 1） | 47.92 | 46.2/52.4/50.4/48.8 | 52.75 |+| on λ=0.3 无 mask | 44.57 | 45.3/49.7/49.3/**30.3** | — |+| on λ=0.3 + mask | 48.53 | 46.1/49.0/50.6/48.0 | — |+| on λ=3.0 + mask | 48.64 | 46.1/49.0/51.2/47.5 | — |+| on λ=3.0 + 逐细胞偏差解码 γ=1 | 48.20 | 44.5/49.0/51.8/46.4 | — |+| **off（乱网）λ=3.0 + mask** | **49.73** | 46.9/49.7/52.8/48.7 | — |+| 单输入=选择（PLAN Step4 原版） | — | — | 46.07 |+| 单输入=均匀抽样（shipped） | — | — | 52.97 |++**机制生效证据（PLAN mechanism_evidence）**：(1) 每细胞修改基因数 = 表达且 n_reg≥3 的基因，占面板 9.1%（稀疏，>>0 且 <<全部）；(2) Δexpr 幅度 ∝ 1/n_reg，与调控覆盖直接挂钩；(3) 型内 Δexpr 跨基因 std > 0（mean|Δ|≈0.043@λ=3，非均匀）；(4) 四组分对比见上表；(5) n_reg≥3 基因占比 9.13%。++**但 off 对照判负**：乱网对照（49.73）≥ 真网（48.64），四组分全部持平或更好。按 PLAN 自己的判据，CollecTRI 结构在 X3（E8.75→E9.0→E9.5 心脏）上没有产生超过随机边配置的信号；位移只剩噪声，反而轻微伤害 cell_state/covariation。DE 两组（de_recovery 46.1、direction 49.0）与基线同在地板附近——PLAN 风险 1 应验：调控网覆盖/方向不足以捕捉该窗口真实变化的基因。因此**提交版本默认 λ=0**（RP_LAMBDA=0 即零位移，输出 = 末阶段均匀抽样，两视图 ≈ copy_last），机制代码完整保留、可用环境变量打开，不冒充成有效的位移方法。++## 验证过 / 没验证++- 验证：两视图跑通 + vec-check ok；seed 确定性（seed 7 两次逐元素相同）；视图无关（只用时间差与数据）；上表所有查分。+- 没验证：λ=0 下 shipped 预测的 vec-score（数学上零位移=均匀 copy，proxy 均匀版已测 52.97；X3 均匀版≈node 1 的 47.92±抽样噪声）；final（E8.5+E9.5→E10.5）上的行为（机制默认关，与 copy_last 同）；β、T_CAP、MIN_REG 的网格（λ=0 下不影响输出）。++## 知识来源++- CollecTRI（prior/tf_regulons，通用 TF–靶基因调控先验，不针对禁窗阶段）；decoupler ULM 标准用法。未使用任何保留阶段/保留基因型的测量信息，未读 external/、未用 uns 调色板。++## 下一步建议++调控解码在 X3 判负，regulatory_program 族按当前形态（型均值 ΔA + 线性签名解码）不值得继续调参；若保留该族，应改用型间差异基因的富集方向约束（而非全网络线性解码），或直接放弃该族转向 composition_trend（node 7/16）与 graph_fate（node 10）的组合。diff --git a/solution/run.py b/solution/run.pynew file mode 100644index 0000000..29a37fe--- /dev/null+++ b/solution/run.py@@ -0,0 +1,229 @@+"""regulatory_program: CollecTRI ULM activity transition + regulon-constrained decode.++Two-stage views (>=2 inputs): per-cell-type TF activity change between the last two+input stages, extrapolated to the target and decoded back to expression via signed+regulon weights (only genes with >=3 regulators; per-gene shrink by regulon size).+Single-stage views: select the most representative cells per type by smallest+regulatory deviation (ULM activities), no expression modification.++Off control: --shuffle_regulon permutes the target column of the network+(preserves each TF out-degree and each gene in-degree) -> uncoordinated decode.+"""++from __future__ import annotations++import argparse+import gzip+import os+from pathlib import Path++import numpy as np+import pandas as pd+import scipy.sparse as sp++from src.task1_temporal import view_io++BETA = float(os.environ.get("RP_BETA", "1.0"))+# λ=0 ships the honest negative result: the CollecTRI decode scored BELOW its+# own off-control on X3 (48.64 vs 49.73), so the mechanism is disabled by+# default and the two-stage path reduces to a uniform copy of the last stage.+# Set RP_LAMBDA>0 to re-enable the regulon decode.+LAMBDA_DECODE = float(os.environ.get("RP_LAMBDA", "0.0"))+CLIP = float(os.environ.get("RP_CLIP", "0.5"))+MIN_REG = int(os.environ.get("RP_MIN_REG", "3"))+KEEP_RATE = float(os.environ.get("RP_KEEP", "0.85"))+SEL_LAMBDA = float(os.environ.get("RP_SEL_LAMBDA", "0.15"))+GAMMA = float(os.environ.get("RP_GAMMA", "0.0"))+T_CAP = float(os.environ.get("RP_TCAP", "2.0"))+USE_T = os.environ.get("RP_USE_T", "1") == "1"+++def load_net(view, manifest, genes, shuffle: bool, rng) -> pd.DataFrame:+    entry = next(p for p in manifest["prior"] if p["id"] == "tf_regulons")+    path = Path(view) / entry["path"]+    files = list(path.iterdir()) if path.is_dir() else [path]+    f = files[0]+    opener = gzip.open if str(f).endswith(".gz") else open+    net = pd.read_csv(opener(f, "rt"), sep="\t")+    gset = set(genes)+    net = net[net["tf"].isin(gset) & net["target"].isin(gset)].copy()+    net = net.rename(columns={"tf": "source", "target": "target", "mor": "weight"})+    net = net[["source", "target", "weight"]].drop_duplicates(["source", "target"])+    net["weight"] = np.sign(net["weight"].astype(float))+    net = net[net["weight"] != 0].reset_index(drop=True)+    if shuffle:+        perm = rng.permutation(len(net))+        net["target"] = net["target"].to_numpy()[perm]+        net = net.drop_duplicates(["source", "target"]).reset_index(drop=True)+    return net+++def activities(adata, net) -> pd.DataFrame:+    import decoupler as dc++    res = dc.mt.ulm(adata, net, tmin=5, raw=False, verbose=False)+    if res is None:+        return adata.obsm["score_ulm"]+    return res[0]+++def main() -> None:+    ap = argparse.ArgumentParser()+    ap.add_argument("--data", required=True)+    ap.add_argument("--out", required=True)+    ap.add_argument("--seed", type=int, required=True)+    ap.add_argument("--shuffle_regulon", action="store_true")+    args = ap.parse_args()++    view = args.data+    rng = np.random.default_rng(args.seed)+    manifest = view_io.load_manifest(view)+    genes = view_io.panel_genes(view, manifest)++    net = load_net(view, manifest, genes, args.shuffle_regulon, rng)+    if net.empty:+        raise RuntimeError("empty regulon network after filtering")++    inputs = view_io.inputs_by_time(manifest)+    last_entry = inputs[-1]+    adata_last = view_io.read_stage(view, last_entry, genes, missing="fill")+    labels_last = view_io.labels_of(adata_last)++    stats = {"n_tf": int(net["source"].nunique()), "n_edges": int(len(net))}++    if len(inputs) >= 2:+        prev_entry = inputs[-2]+        adata_prev = view_io.read_stage(view, prev_entry, genes, missing="fill")+        labels_prev = view_io.labels_of(adata_prev)++        A_last = activities(adata_last, net)+        A_prev = activities(adata_prev, net)++        dt_target = float(manifest["target"]["time"]) - float(last_entry["time"])+        dt_train = float(last_entry["time"]) - float(prev_entry["time"])+        T = dt_target / dt_train if dt_train > 0 else 1.0+        if not USE_T:+            T = 1.0+        T = float(min(T, T_CAP))+        scale = BETA * T++        delta_rows, row_types = [], []+        # ULM prunes TFs per dataset; restrict to the TFs scored in both stages+        common_tfs = [tf for tf in A_last.columns if tf in set(A_prev.columns)]+        A_last = A_last[common_tfs]+        A_prev = A_prev[common_tfs]+        mean_prev = A_prev.groupby(pd.Series(labels_prev, index=A_prev.index)).mean()+        ct_series = pd.Series(labels_last, index=A_last.index)+        mean_last = A_last.groupby(ct_series).mean()+        shared = [ct for ct in mean_last.index if ct in mean_prev.index]+        for ct in shared:+            dA = (mean_last.loc[ct] - mean_prev.loc[ct]).to_numpy() * scale+            delta_rows.append(dA)+            row_types.append(ct)++        if delta_rows:+            DA = np.vstack(delta_rows)  # (n_types, n_tfs) in ULM column order+            tf_cols = list(A_last.columns)+            tf_idx = {tf: i for i, tf in enumerate(tf_cols)}+            net = net[net["source"].isin(tf_idx)]+            cols = net["target"].to_numpy()+            # gene <- tf signed matrix+            gene_pos = {g: i for i, g in enumerate(genes)}+            tfs = net["source"].to_numpy()+            src_i = np.array([tf_idx[t] for t in tfs])+            tgt_i = np.array([gene_pos[c] for c in cols])+            W = sp.csr_matrix((net["weight"].to_numpy(), (src_i, tgt_i)),+                              shape=(len(tf_idx), len(genes)))+            n_reg = np.asarray(W.getnnz(axis=0)).ravel()+            keep_genes = n_reg >= MIN_REG+            w_gene = np.zeros(len(genes))+            w_gene[keep_genes] = LAMBDA_DECODE / n_reg[keep_genes]+            # decode: dExpr[type, gene] = sum_tf DA[type,tf] * sign * w_gene+            DEX = (DA @ W.toarray()) if W.shape[0] <= 2000 else None+            if DEX is None:  # fallback chunked (never hit at current sizes)+                DEX = np.zeros((DA.shape[0], len(genes)))+                for s in range(0, DA.shape[0], 64):+                    DEX[s:s + 64] = DA[s:s + 64] @ W.toarray()+            DEX = DEX * w_gene[None, :]+            DEX = np.clip(DEX, -CLIP, CLIP)+            DEX[:, ~keep_genes] = 0.0++            type_of = {ct: i for i, ct in enumerate(row_types)}+            row_ids = np.array([type_of.get(ct, -1) for ct in labels_last])++            # per-cell decode: type-mean transition (DEX) plus the cell's own+            # centered regulatory deviation decoded through the same regulons,+            # so gene-gene coordination is regulatory, not gene-independent+            Wd = W.toarray()+            Ac = A_last.to_numpy()+            ct_s = pd.Series(labels_last, index=A_last.index)+            type_mean_full = A_last.groupby(ct_s).transform("mean").to_numpy()+            devA = Ac - type_mean_full+            DEX_cell = (devA @ Wd) * w_gene[None, :] * GAMMA+            DEX_cell = np.clip(DEX_cell, -CLIP, CLIP)+            DEX_cell[:, ~keep_genes] = 0.0++            n_out = view_io.target_n_cells(manifest, adata_last.n_obs)+            idx = view_io.sample_rows(adata_last.n_obs, n_out, rng)+            X = adata_last.X[idx].toarray()+            rid = row_ids[idx]+            has = rid >= 0+            # modify only already-expressed entries: keeps the sparse structure+            # (covariation) intact instead of lifting every zero gene+            mask = X > 0+            shift = np.zeros_like(X)+            shift[has] = DEX[rid[has]] + DEX_cell[idx[has]]+            np.clip(shift, -CLIP, CLIP, out=shift)+            shift *= mask+            X = X + shift+            np.clip(X, 0.0, None, out=X)+            stats["frac_genes_modified"] = float(keep_genes.mean())+            stats["mean_abs_dex"] = float(np.abs(shift[mask]).mean()) if mask.any() else 0.0+            stats["pb_shift_std"] = float(np.std(shift.mean(axis=0))) if has.any() else 0.0+            view_io.write_prediction(sp.csr_matrix(X.astype(np.float32)), genes,+                                     args.out, seed=args.seed)+            _log(stats)+            return++    # single-stage fallback: no reliable regulatory trend can be estimated from+    # one stage. Deviation-based selection (RP_SELECT=1) narrows the cell+    # distribution and measurably hurt mmd_u/variogram, so the default is a+    # uniform subsample of the last stage.+    n_out = view_io.target_n_cells(manifest, adata_last.n_obs)+    if os.environ.get("RP_SELECT", "0") == "1":+        A_last = activities(adata_last, net)+        ct_series = pd.Series(labels_last, index=A_last.index)+        type_mean = A_last.groupby(ct_series).transform("mean")+        dev = np.linalg.norm(A_last.to_numpy() - type_mean.to_numpy(), axis=1)+        chosen = []+        for ct in pd.unique(labels_last):+            m = labels_last == ct+            pool = np.flatnonzero(m)+            k = min(max(1, int(round(n_out * m.mean()))), len(pool))+            if k >= len(pool):+                chosen.append(pool)+                continue+            d = dev[pool]+            n_cand = max(k, int(np.ceil(KEEP_RATE * len(pool))))+            cand = pool[np.argsort(d)[:n_cand]]+            dc_ = dev[cand]+            s = dc_.std() if dc_.std() > 0 else 1.0+            logits = -SEL_LAMBDA * (dc_ - dc_.mean()) / s * np.sqrt(len(cand))+            w = np.exp(logits - logits.max())+            w = w / w.sum()+            take = rng.choice(cand, size=k, replace=False, p=w)+            chosen.append(np.sort(take))+        idx = np.sort(np.concatenate(chosen))[:n_out]+    else:+        idx = view_io.sample_rows(adata_last.n_obs, n_out, rng)+    stats["n_selected"] = int(len(idx))+    view_io.write_prediction(adata_last.X[idx], genes, args.out, seed=args.seed)+    _log(stats)+++def _log(stats: dict) -> None:+    print("[regulatory_program]", stats, flush=True)+++if __name__ == "__main__":+    main()

调研来源?调研员查到并用到的知识条目和文献检索结果（只列标题和编号）。

用到的知识库条目

分析结果?分析员写的 ANALYSIS.json：改了什么、各组分数怎么变、假设是否成立、经验和下一步建议。

对话摘要?每个角色和大模型对话的统计：轮数、工具调用、用时、token 数和最后的回答摘录；原始记录只给路径。

只给统计和最后回答的摘录；完整对话请到原始记录位置里列出的文件看。

分析员

ANALYSIS.json written

工程师

完成。总结：

**做了什么**：从零实现 PLAN 的 regulatory_program 族——CollecTRI 网络（1086 TF / 37k 边）+ decoupler ULM 活性；两阶段路径按细胞类型估 ΔA、乘时间外推因子 T、经签名调控网线性解码（λ/n_reg 收缩、±0.5 clip、仅 n_reg≥3 基因、只改已表达条目）；单输入路径按 PLAN 做调控偏差选择；off 对照 `--shuffle_regulon`（保度置换）。

**查分**（8/20 次）：X3 on λ=3 = 48.64，**off 乱网对照 = 49.73 ≥ on**，PLAN 判据判负（调控结构无信号，位移=噪声）；无 mask 的稠密小位移使 variogram skill 崩到 0.30（44.57）；单输入"代表性选择"把 proxy10 从 52.97 拉到 46.07。故提交版默认 λ=0（机制关闭，两视图 ≈ copy_last：proxy10 均匀版实测 52.97），机制代码完整保留、环境变量可开，METHOD.md 如实记录负结果。seed 确定性与 vec-check 均通过。

**下一步最值得试**：该族按当前形态（全网络线性解码）应放弃；建议把算力投向 composition_trend（node 7/16，rank3 56.41）与 graph_fate（node 10，X3 54.13）的组合——X3 权重 2 且 node 10 在 X3 上最强，二者机制正交（组成趋势 vs 细胞级后继抽样）。

调研员

gene matrix if >500 TFs × >20k genes.",
  "family_id": "regulatory_program",
  "mechanism": "Compute per-cell TF activities via CollecTRI+ULM; estimate activity transitions between stages (or per-type deviations in single-stage); decode activity changes back to expression via signed regulon weights, so expression shifts are coordinated by regulatory structure rather than gene-independent.",
  "vs_constant_shift": "Displacement is cell-type-specific AND gene-regulon-specific: each gene's change is determined by which TFs regulate it and by how much those TFs' activities changed. A constant per-type shift moves all genes equally regardless of regulatory membership. Here, only regulon-covered genes change, and their direction/magnitude is determined by TF activity × regulatory sign. Must demonstrate: (a) modified genes are enriched for regulon targets, (b) per-type displacement vectors are NOT proportional to a single scalar (show variance in Δexpr across genes within a type).",
  "mechanism_evidence": "Engineer should report: (1) number of genes modified per cell (should be >>0 and <<total genes, confirming sparsity); (2) correlation between per-gene Δexpr and regulon coverage (should be positive); (3) for top-5 most-changed cell types, show that Δexpr is NOT uniform across genes (std of Δexpr within type > 0.1); (4) four-group score breakdown comparing on vs off; (5) fraction of genes with ≥3 regulators in filtered network. If all genes shift by same amount within a type, mechanism collapsed to constant shift.",
  "mechanism_off_control": "Same program with --shuffle_regulon flag: randomly reassign target genes among TFs preserving each TF's out-degree and each gene's in-degree (configuration model shuffle). All other steps identical. Expected: shuffled version produces uncoordinated noise → de_score, de_direction, covariation should all be near copy_last baseline or worse; if shuffled ≈ unshuffled, the regulon structure is not driving any signal.",
  "sources": []
}