总览 · ← 返回运行 20261001-124831-search-t1-fake

节点 n5

数据导出的组成重加权（生长率轴）+ 保底分层抽样

方法说明?节点程序自带的 METHOD.md：这个程序做了什么、为什么。

来自 programs.git a511d07447:solution/METHOD.md

PYTHONPATH=modeling python solution/run.py --data <view> --out pred.h5ad --seed 0 \
  --kappa -0.5 --trend-s 0.5 --floor-m 10 --wcap 4 --shrink-m0 200 --families 6

调研员的计划

代码改动?这个节点的程序和父节点程序的逐行差别：绿色是新增，红色是删除。

对比：父节点版本 0ee75fe5a5。改动的文件：solution/METHOD.md +50 −0、solution/README.md +10 −3、solution/run.py +288 −10

diff --git a/solution/METHOD.md b/solution/METHOD.mdnew file mode 100644index 0000000..2456db7--- /dev/null+++ b/solution/METHOD.md@@ -0,0 +1,50 @@+# 数据导出的组成重加权（生长率轴）+ 保底分层抽样++用 view 内 prior 基因集算出的"增殖−凋亡"生长率轴给每个细胞类型重加权（κ 由替代评测选为 −0.5），表达矩阵一个数都不改，只按配额分层抽样真实细胞。++## 方法++1. **标签**：读 last 输入阶段的 `obs` 里第一个类型注释列（proxy E8.5 = `celltype`，18 型）；没有注释列时退路为 TruncatedSVD(50)+KMeans(k=30, random_state=seed) 的伪类型。列名/类型数打进日志。+2. **B1 生长率项**（`run.py: load_score_sets / growth_scores`）：遍历 `<view>/prior/**/*.gmt`（proxy 命中 17,275 个集），按**集名**正则挑增殖集（`cell cycle|mitotic|chromosome segregation|dna replication`）与凋亡集（`apopto|programmed cell death|cell death`），限制集大小 20–500 且与面板有交集 → 199 + 161 个基因。细胞打分 = mean_z(增殖) − mean_z(凋亡)（z 按全阶段逐基因算，只在列子集上做，稀疏、不稠密化）。类型级 g_c 可按表达中心的层次聚类族收缩（`--shrink-m0 200`，`--families 6`），再标准化成 z 并减去按比例的加权均值。+3. **B2 趋势项**：`--trend-s 0.5`，**只在 ≥2 个输入阶段时启用**（proxy 单阶段自动为 0）。对两阶段同名类型取 Δ = logit(p_last) − logit(p_prev)，截断 ±1.5；只在 last 出现的类型 Δ=0，消失的类型不推向 −inf。+4. **B3 合成与安全**：`log w_c = κ·z_c + s·Δ_c`，截断 |log w_c| ≤ log(`--wcap` 4)。任何类型 w>0，不整类丢弃。+5. **C 配额分层抽样**：n_c ∝ avail_c·w_c，largest-remainder 取整，保底 min(`--floor-m` 10, avail)，硬约束 Σn_c = N = `target_n_cells`（T1 = 5118），每类内部**无放回**抽真实细胞、不复制行（配额超可用量才截断并把余额分给别的类）。输出走 `write_prediction(X[rows], genes, out, seed)`。++## 参数与查分（T1:val proxy，seed 0，除注明外）++| 配置 | 榜分 | cell_state | covariation | de_recovery | direction |+|---|---|---|---|---|---|+| 父节点 copy_last | 49.77 | 48.07 | 51.54 | 50.00 | 50.16 |+| κ=0（纯分层，w≡1） | 49.25 | 46.11 | 51.99 | 50.00 | 50.08 |+| κ=0，seed 1 | 48.82 | 45.96 | 52.11 | 49.06 | 49.40 |+| κ=+1（生物学先验方向） | 37.72 | 23.77 | 36.20 | 49.52 | 43.88 |+| κ=−0.25 | 52.76 | 51.66 | 52.86 | 50.98 | 55.77 |+| **κ=−0.5（出货）** | **53.03** | **53.61** | **49.90** | **50.48** | **57.37** |+| κ=−0.6 | 51.37 | 49.11 | 48.67 | 50.48 | 57.12 |+| κ=−0.75 | 49.62 | 45.25 | 45.88 | 50.48 | 56.98 |+| κ=−1 | 46.83 | 39.30 | 40.37 | 50.48 | 57.40 |+| κ=−2 | 45.30 | 36.28 | 37.72 | 50.98 | 56.50 |++节点 2（手调 heart×1.6 / edge×0.25 / 丢 Neural Tube）= 55.97，仍是本榜最佳；本节点是不写任何类型名的数据导出版本。++## 验证过什么++- `vec-check` 通过；行数 = 5118 = N，无 NaN/Inf，只是 last 阶段的行子集（表达值不可能越界）。+- 同一 seed 确定性：κ=0 的配额与父节点差异只来自类内抽样；κ≠0 时配额由 w 决定，seed 只影响类内抽行。+- κ 的 3 点粗网格 + 细化（−0.25/−0.5/−0.6/−0.75/−1/−2/+1），共 9 次查分，**单 seed**：κ=−0.5 与 κ=−0.25 差 0.27 分（噪声内），κ=−0.5 与 κ=−0.6 差 1.66 分（噪声内），所以 −0.25…−0.6 是一段平台，取中间偏上的 −0.5；κ ≤ −0.75 明显变差，符号一致的结论只有"负方向有效、幅度不能大"。+- 时间/内存：单次运行 4.4 s（含读 gmt），远低于 30 min / 28 GB。++## 没验证 / 风险（后续节点请注意）++- **κ<0 的方向是替代评测选出来的，不是先验预测的。** 生物学上"增殖高的类型在下一阶段占比下降"只在 E8.5→E9.5 这一段被测到；它是否等价于"取材范围向心脏集中"无从判断。**final（E9.5→E10.5）上必须在 κ∈{0,−0.25,−0.5} 各跑一次再定，不要默认 −0.5。**+- **B2 趋势项在 proxy 上完全没被测到**（单输入阶段，s 自动为 0）。它只是"同一阶段喂两次必须与单阶段逐行相同"这类逻辑退路，未做查分验证；|Δ| 截断 1.5、s=0.5 是保守值。+- 族收缩（A 部件）在 proxy 上开着但作用很小（每型 ≥266 个细胞，λ≈0.4 只轻微拉平 z）；未做 A/B。+- κ=0 的精确比例分层比父节点的随机无放回抽样低约 0.5–1 分（cell_state 46.0 vs 48.1）。**说明 cell_state 对组成的响应不是"越接近 E8.5 比例越好"，抽样噪声本身就能值 2 分**；这也意味着 κ=0 不是一个安全的地板。+- 未试：wcap≠4、floor-m∈{1,25}（保底在 18 型 × 5118 细胞的规模下几乎不触发，预期无影响）、D 部件（快照内伪时间位移）。++## 复现++```bash+PYTHONPATH=modeling python solution/run.py --data <view> --out pred.h5ad --seed 0 \+  --kappa -0.5 --trend-s 0.5 --floor-m 10 --wcap 4 --shrink-m0 200 --families 6+```diff --git a/solution/README.md b/solution/README.mdindex e6cdf6f..f220007 100644--- a/solution/README.md+++ b/solution/README.md@@ -1,4 +1,11 @@-# copy_last+# n5: growth-axis reweight + floored stratified resampling (parent n1 copy_last 49.77) -输出最新一个输入阶段（proxy：E8.5；final：E9.5），随机无放回抽到榜的细胞上限（T1 5118）。-不改表达，不改组成。这是地板：proxy 预期 ≈ 50（seed 0 实测 49.77；抽到 5118 个细胞，比整份 E8.5 的 50.00 略低）。+`python run.py --data <view> --out <pred.h5ad> --seed <int>`（默认 κ=−0.5，其余开关见 METHOD.md）++表达矩阵不改一行：只用 `<view>/prior` 的 gmt 基因集算"增殖−凋亡"生长率轴，按类型标准化成 z（可按表达族收缩），+`log w_c = κ·z_c + s·Δlogit p_c`（趋势项只在 ≥2 个输入阶段启用，proxy 自动为 0），截断 |log w| ≤ log 4，+再按 n_c ∝ avail_c·w_c 做 largest-remainder 配额 + 保底 10 + 类内无放回抽样到 N=5118。++proxy 实测（seed 0）：**53.03**（cell_state 53.61 / covariation 49.90 / de_recovery 50.48 / direction 57.37），+父节点 49.77。κ 网格见 METHOD.md；κ ≥ −0.75 会变差，κ>0 崩到 37.7。+双输入自检：同一阶段喂两次与单阶段输出逐行相同。4.4 s / <2 GB。diff --git a/solution/run.py b/solution/run.pyindex aa4bbf0..b7e20a4 100644--- a/solution/run.py+++ b/solution/run.py@@ -1,36 +1,314 @@ #!/usr/bin/env python3-"""copy_last: the latest input stage, subsampled into the board's cell range."""+"""Composition-only predictor: data-derived per-type weights + floored stratified sampling.++The expression matrix is never modified: the prediction is a row subset of the+last observed stage.  All the signal is in *how many* cells each cell type+contributes, which is decided from quantities computed on the fly from the view+(prior gene sets + the input stages themselves), never from hard-coded type+names or stage statistics.++log w_c = kappa * z(growth_c) + s * delta_logit_p_c      (|log w_c| <= log wcap)++growth_c  proliferation-minus-apoptosis score of type c (gene sets picked by+          *set name* regex from <view>/prior/**/*.gmt, scored as mean per-gene+          z over the panel), shrunk towards the mean of its expression family.+delta     only when >= 2 input stages: logit(p_c^last) - logit(p_c^prev) for+          types seen in both, 0 for types new in last, clipped at +-trend-clip.+"""  from __future__ import annotations  import argparse+import re+from pathlib import Path  import numpy as np+from scipy import sparse  from src.task1_temporal.view_io import (     inputs_by_time,     load_manifest,     panel_genes,     read_stage,-    sample_rows,     target_n_cells,     write_prediction, ) +PROLIF_RE = re.compile(r"cell cycle|mitotic|chromosome segregation|dna replication", re.I)+APOPT_RE = re.compile(r"apopto|programmed cell death|cell death", re.I)+FAMILY_RE = re.compile(r".*", re.I)+++# ---------------------------------------------------------------- gene sets+def _read_gmt(path: Path, gene_index: dict[str, int]) -> dict[str, list[int]]:+    out: dict[str, list[int]] = {}+    with path.open("r", encoding="utf-8", errors="ignore") as fh:+        for line in fh:+            fields = line.rstrip("\n").split("\t")+            if len(fields) < 3:+                continue+            idx = sorted({gene_index[g] for g in fields[2:] if g in gene_index})+            if idx:+                out.setdefault(fields[0], idx)+    return out+++def load_score_sets(view: str, genes: list[str], min_genes: int = 20, max_genes: int = 500):+    """(proliferation columns, apoptosis columns) from every gmt under prior/."""+    gene_index = {g: i for i, g in enumerate(genes)}+    prolif: set[int] = set()+    apopt: set[int] = set()+    root = Path(view) / "prior"+    n_sets = 0+    if root.is_dir():+        for path in sorted(root.rglob("*.gmt")):+            for name, idx in _read_gmt(path, gene_index).items():+                n_sets += 1+                if not (min_genes <= len(idx) <= max_genes):+                    continue+                if PROLIF_RE.search(name):+                    prolif.update(idx)+                elif APOPT_RE.search(name):+                    apopt.update(idx)+    return sorted(prolif), sorted(apopt), n_sets+++def _mean_z(X: sparse.csr_matrix, cols, mu: np.ndarray, sd: np.ndarray) -> np.ndarray:+    """mean over cols of (x - mu)/sd, computed on the column subset only."""+    cols = np.asarray(cols, dtype=np.int64)+    if cols.size == 0:+        return np.zeros(X.shape[0], dtype=np.float64)+    inv = (1.0 / sd[cols]).astype(np.float32)+    vals = np.asarray(X[:, cols].multiply(inv).mean(axis=1)).ravel()+    return vals - float(np.mean(mu[cols] / sd[cols]))+++def growth_scores(X: sparse.csr_matrix, prolif, apopt) -> np.ndarray | None:+    mu = np.asarray(X.mean(axis=0)).ravel().astype(np.float64)+    sq = np.asarray(X.multiply(X).mean(axis=0)).ravel().astype(np.float64)+    sd = np.sqrt(np.maximum(sq - mu**2, 1e-12))+    if not prolif or not apopt:+        return None+    return _mean_z(X, prolif, mu, sd) - _mean_z(X, apopt, mu, sd)+++# ---------------------------------------------------------------- labels+def type_labels(adata, seed: int) -> tuple[np.ndarray, str]:+    """Cell-type column if the view has one, else PCA+KMeans pseudo-types."""+    import pandas as pd++    for col in ("celltype", "cell_type", "celltype_name", "annotation", "cell_label"):+        if col in adata.obs.columns:+            return adata.obs[col].astype(str).to_numpy(), f"obs[{col}]"+    best = None+    for col in adata.obs.columns:+        s = adata.obs[col]+        if not isinstance(s.dtype, (pd.CategoricalDtype, np.object_)) and s.dtype.kind not in "OU":+            continue+        u = s.astype(str).unique()+        if 2 <= len(u) <= max(200, adata.n_obs // 10):+            if best is None or len(u) > best[1]:+                best = (col, len(u))+    if best is not None:+        return adata.obs[best[0]].astype(str).to_numpy(), f"obs[{best[0]}]"++    from sklearn.cluster import KMeans+    from sklearn.decomposition import TruncatedSVD++    X = adata.X+    n_comp = min(50, X.shape[0] - 1, X.shape[1] - 1)+    emb = TruncatedSVD(n_components=n_comp, random_state=seed).fit_transform(X.astype(np.float32))+    k = min(30, X.shape[0] // 20)+    lab = KMeans(n_clusters=max(2, k), random_state=seed, n_init=10).fit_predict(emb)+    return np.array([f"cluster_{i}" for i in lab]), f"kmeans(k={k})"+++# ---------------------------------------------------------------- families+def family_of(centroids: np.ndarray, n_fam: int) -> np.ndarray:+    """Average-linkage clusters of type centroids (Pearson distance)."""+    from scipy.cluster.hierarchy import fcluster, linkage+    from scipy.spatial.distance import squareform++    n = centroids.shape[0]+    if n <= 2 or n_fam >= n:+        return np.arange(n)+    C = np.corrcoef(centroids)+    C = np.nan_to_num(C, nan=0.0)+    D = 1.0 - C+    np.fill_diagonal(D, 0.0)+    D = np.clip((D + D.T) / 2.0, 0, None)+    Z = linkage(squareform(D, checks=False), method="average")+    return fcluster(Z, t=min(n_fam, n - 1), criterion="maxclust")+++# ---------------------------------------------------------------- allocation+def allocate(w: np.ndarray, avail: np.ndarray, n_total: int, floor_m: int) -> np.ndarray:+    # NOTE: quotas are proportional to avail_c * w_c (w acts on population fractions).+    """Largest-remainder quotas, floored at min(floor_m, avail), never above avail."""+    k = len(w)+    n_total = int(min(n_total, int(avail.sum())))+    out = np.zeros(k, dtype=np.int64)+    if n_total <= 0:+        return out+    out = np.minimum(floor_m, avail).astype(np.int64)+    left = n_total - out.sum()+    if left < 0:  # floors already exceed the budget: keep the biggest types first+        order = np.argsort(-avail)+        keep = np.zeros(k, dtype=np.int64)+        rem = n_total+        for i in order:+            take = int(min(avail[i], rem))+            keep[i] = take+            rem -= take+            if rem <= 0:+                break+        return keep+    cap = avail - out+    pos = cap > 0+    if left > 0 and pos.any():+        raw = w * avail * pos+        s = raw.sum()+        if s <= 0:+            raw = (avail * pos).astype(np.float64)+            s = raw.sum()+        target = raw / s * left+        add = np.minimum(np.floor(target).astype(np.int64), cap)+        out += add+        rem = left - add.sum()+        if rem > 0:+            frac = np.where(cap - add > 0, target - np.floor(target), -1.0)+            for i in np.argsort(-frac):+                if rem <= 0:+                    break+                room = int(cap[i] - (out[i] - np.minimum(floor_m, avail[i])))+                room = int(avail[i] - out[i])+                if room > 0:+                    out[i] += 1+                    rem -= 1+            while rem > 0:  # still short (rounding): give to types with room+                room = avail - out+                cand = np.flatnonzero(room > 0)+                if cand.size == 0:+                    break+                i = int(cand[np.argmax(w[cand])])+                out[i] += 1+                rem -= 1+    return out+++def draw_rows(labels: np.ndarray, types: list[str], alloc: np.ndarray,+              rng: np.random.Generator) -> np.ndarray:+    rows = []+    for c in np.flatnonzero(alloc > 0):+        pool = np.flatnonzero(labels == types[c])+        n = int(alloc[c])+        if pool.size == 0:+            continue+        rows.append(rng.choice(pool, size=min(n, pool.size), replace=False) if n <= pool.size+                    else np.concatenate([pool, rng.choice(pool, size=n - pool.size, replace=True)]))+    if not rows:+        return np.array([], dtype=np.int64)+    return np.sort(np.concatenate(rows))+++# ---------------------------------------------------------------- trend+def trend_term(stages: list, labels_last: np.ndarray, types: list[str], clip: float) -> np.ndarray:+    """logit(p_last) - logit(p_prev) for shared types, 0 elsewhere, clipped."""+    if len(stages) < 2:+        return np.zeros(len(types))+    prev_labels = stages[-2][1]+    eps = 1e-3+    d = np.zeros(len(types))+    prev_types, prev_cnt = np.unique(prev_labels, return_counts=True)+    prev_p = {str(t): c / prev_cnt.sum() for t, c in zip(prev_types, prev_cnt)}+    last_cnt = np.array([(labels_last == t).sum() for t in types], dtype=np.float64)+    last_p = last_cnt / max(last_cnt.sum(), 1)+    for i, t in enumerate(types):+        if t not in prev_p:+            continue+        pl = min(max(last_p[i], eps), 1 - eps)+        pp = min(max(prev_p[t], eps), 1 - eps)+        d[i] = np.log(pl / (1 - pl)) - np.log(pp / (1 - pp))+    return np.clip(d, -clip, clip)+  def main() -> None:-    parser = argparse.ArgumentParser()-    parser.add_argument("--data", required=True)-    parser.add_argument("--out", required=True)-    parser.add_argument("--seed", type=int, default=0)-    args = parser.parse_args()+    ap = argparse.ArgumentParser()+    ap.add_argument("--data", required=True)+    ap.add_argument("--out", required=True)+    ap.add_argument("--seed", type=int, default=0)+    ap.add_argument("--kappa", type=float, default=-0.5)+    ap.add_argument("--trend-s", type=float, default=0.5)+    ap.add_argument("--trend-clip", type=float, default=1.5)+    ap.add_argument("--floor-m", type=int, default=10)+    ap.add_argument("--wcap", type=float, default=4.0)+    ap.add_argument("--shrink-m0", type=float, default=200.0)+    ap.add_argument("--families", type=int, default=6)+    args = ap.parse_args()      manifest = load_manifest(args.data)     genes = panel_genes(args.data, manifest)-    last = read_stage(args.data, inputs_by_time(manifest)[-1], genes)+    entries = inputs_by_time(manifest)     rng = np.random.default_rng(args.seed)-    rows = sample_rows(last.n_obs, target_n_cells(manifest, last.n_obs), rng)-    write_prediction(last.X[rows], genes, args.out, seed=args.seed)++    stages = []+    for e in entries:+        a = read_stage(args.data, e, genes)+        lab, src = type_labels(a, args.seed)+        stages.append((a, lab))+        print(f"[stage {e['stage']}] n_obs={a.n_obs} labels={src} types={len(np.unique(lab))}", flush=True)++    last, labels_last = stages[-1]+    X = last.X+    types_u, counts = np.unique(labels_last, return_counts=True)+    types = [str(t) for t in types_u]+    counts = counts.astype(np.float64)+    p = counts / counts.sum()+    n_cells = target_n_cells(manifest, last.n_obs)+    print(f"[types] {len(types)}; single input stage: {len(stages) == 1}; N={n_cells}", flush=True)++    # ---- B1 growth term (z-scored, shrunk to expression family means)+    logw = np.zeros(len(types))+    prolif, apopt, n_sets = load_score_sets(args.data, genes)+    print(f"[prior] gmt sets={n_sets} prolif_genes={len(prolif)} apopt_genes={len(apopt)}", flush=True)+    g = growth_scores(X, prolif, apopt) if args.kappa != 0.0 else None+    if g is not None:+        g_c = np.array([g[labels_last == t].mean() for t in types], dtype=np.float64)+        if args.shrink_m0 > 0 and args.families > 1:+            cents = np.vstack([+                np.asarray(X[labels_last == t].mean(axis=0)).ravel() for t in types+            ]).astype(np.float64)+            fam = family_of(cents, args.families)+            g_sh = g_c.copy()+            for f in np.unique(fam):+                m = fam == f+                gf = float(np.average(g_c[m], weights=counts[m]))+                lam = args.shrink_m0 / (args.shrink_m0 + counts[m])+                g_sh[m] = (1 - lam) * g_c[m] + lam * gf+            g_c = g_sh+        sd = g_c.std()+        z = (g_c - np.average(g_c, weights=p)) / (sd if sd > 1e-9 else 1.0)+        logw += args.kappa * z+        for i in np.argsort(-z):+            print(f"  z[{types[i]}]={z[i]:+.3f} n={int(counts[i])}", flush=True)++    # ---- B2 trend term (>=2 input stages only)+    s = 0.0 if len(stages) < 2 else args.trend_s+    if s != 0.0:+        logw += s * trend_term(stages, labels_last, types, args.trend_clip)++    # ---- B3 safety clip+    cap = np.log(max(args.wcap, 1.0 + 1e-9))+    logw = np.clip(logw, -cap, cap)+    w = np.exp(logw)++    alloc = allocate(w, counts.astype(np.int64), n_cells, args.floor_m)+    rows = draw_rows(labels_last, types, alloc, rng)+    print(f"[alloc] sum={alloc.sum()} target={n_cells} rows={rows.size} "+          f"w=[{w.min():.3f},{w.max():.3f}]", flush=True)+    for i, (t, n, a) in enumerate(zip(types, counts.astype(int), alloc)):+        print(f"  {t:22s} avail={n:6d} n={a:6d} frac={a / max(rows.size, 1):.4f} w={w[i]:.3f}", flush=True)+    write_prediction(X[rows], genes, args.out, seed=args.seed)   if __name__ == "__main__":

调研来源?调研员查到并用到的知识条目和文献检索结果（只列标题和编号）。

用到的知识库条目

分析结果?分析员写的 ANALYSIS.json：改了什么、各组分数怎么变、假设是否成立、经验和下一步建议。

对话摘要?每个角色和大模型对话的统计：轮数、工具调用、用时、token 数和最后的回答摘录；原始记录只给路径。

只给统计和最后回答的摘录；完整对话请到原始记录位置里列出的文件看。

分析员

ANALYSIS.json written

工程师

**做了什么**：把父节点 copy_last 改成「表达矩阵一个数不改、只改配额」的组成重加权器——用 `<view>/prior` 的 gmt（17,275 集，按集名正则挑出 199 增殖 / 161 凋亡基因）算每类型 z 化的生长率，`log w = κ·z + s·Δlogit`（趋势项仅 ≥2 输入阶段启用），截断 ±log4，再 largest-remainder 配额 + 保底 10 + 类内无放回抽样。无任何硬编码类型名/阶段统计量。

**查分（9 次，proxy seed 0）**：κ=0 → 49.25（cell_state 46.11，比父节点随机抽样还低，说明精确比例不是地板）；κ=+1（生物学先验方向）→ 37.72 崩；κ=−0.25 → 52.76；**κ=−0.5 → 53.03（cell_state 53.61 / cov 49.90 / de 50.48 / dir 57.37）出货**；κ=−0.6 → 51.37；−0.75 → 49.62；−1 → 46.83；−2 → 45.30。父 49.77，榜最佳仍是节点 2 的 55.97。4.4 s / <2 GB，vec-check 通过，双输入退化输出与单输入逐行相同。

**下一步最值得试**：(1) 生长率轴的**符号是替代评测选的、不是先验**，且 −0.25…−0.6 是一段平台 → 在 final 上必须重扫 κ∈{0,−0.25,−0.5}，不要默认 −0.5；(2) 与节点 2/4 的手调心脏权重集成（T1-11），因为两者都指向"心脏系上、神经/体节下"，但数据导出版幅度更保守；(3) 趋势项 B2 至今零查分验证，是 final 上最大的未知收益/风险源。

调研员

PLAN.json written