Virtual Embryo Challenge更新于 10-03 18:47(北京时间) / 每 5 分钟更新

运行档案 / T1:val / 10-01 15:21

G23 验收

先看终选决定,再沿节点查看方法、代码改动和对话证据。

已选出最终候选

以实际终选程序为结果;搜索中曾达到的最高分与最终交付分开记录。

与其他运行比较
终选程序?按该运行锁定的规则最终选出的程序;可能是候选节点,也可能由护栏回退到基线。
n2
本运行目标 43.94(种子 0 单次)
节点?搜索树里的每个候选程序。
82
已打分 56
官网分?官网 P2 返回的真实分数。
—
还没有提交或还没有分数
大模型调用次数?各角色(调研员、工程师、分析员、审查员)调用大模型的次数。
250
分析员 72、工程师 86、调研员 87、审查员 5

实际结果与评分依据

搜索目标、护栏和诊断分开阅读。不同真值分片或 seed 的同名尺子保留各自记录。

尺子用途分数评分上下文
X3搜索目标复测47.7真值 B 半 / 3 个 seed均值
proxy搜索目标复测56.6真值 B 半 / 3 个 seed均值
proxy2搜索目标复测27.5真值 B 半 / 3 个 seed均值
X1终检诊断46.8全部真值 / 1 个 seed单次

来历?终选程序从种子程序一步步改过来的路径,每一步都写了改了什么、分数变了多少。

括号里的分差只表示变化方向,不代表统计显著性。

n2 种子 终选程序
seed heart_jcf_peri
搜索目标分 43.50 · proxy 56.23 · proxy2 27.43 · X3 46.85 · 3 次复测均分 43.94
审查意见?审查员(Reviewer)检查程序是否越界读取、写死目标或钻评分器漏洞后的意见。 通过 seed (human-written, locked)

搜索树?每个点是一个节点,连线从父节点指向子节点;颜色表示结果。

蓝色粗线是来历路径;鼠标停在点上看节点,点一下打开节点页。

n1 种子,已打分,42.491n2 种子,已打分,43.502n3 种子,已打分,39.343n4 改进,没有改动n5 改进,已打分,43.29n6 改进,格式不合格n7 改进,已打分,43.53n8 改进,已打分,43.45n9 改进,已打分,43.23n10 改进,程序报错n11 改进,已打分,43.41n12 改进,没有改动n13 改进,已打分,43.28n14 改进,格式不合格n15 改进,已打分,43.29n16 修复,已打分,43.50n17 改进,已打分,43.28n18 改进,被中断n19 修复,已打分,43.41n20 改进,没有改动n21 改进,已打分,43.34n22 改进,被中断n23 改进,已打分,43.43n24 改进,已打分,43.47n25 改进,已打分,43.27n26 改进,程序报错n27 改进,已打分,43.39n28 改进,被中断n29 草稿,已打分,43.30n30 改进,格式不合格n31 改进,已打分,43.23n32 改进,已打分,43.29n33 改进,已打分,43.24n34 改进,程序报错n35 改进,已打分,43.40n36 修复,已打分,43.16n37 改进,已打分,43.48n38 改进,格式不合格n39 草稿,已打分,43.36n40 改进,已打分,43.20n41 改进,已打分,43.26n42 改进,程序报错n43 改进,已打分,43.28n44 草稿,没有改动n45 改进,已打分,43.28n46 改进,格式不合格n47 修复,已打分,43.59n48 改进,已打分,43.41n49 改进,已打分,43.33n50 修复,已打分,43.32n51 改进,已打分,43.45n52 修复,已打分,43.28n53 改进,已打分,43.12n54 改进,格式不合格n55 改进,已打分,43.22n56 草稿,已打分,43.42n57 改进,已打分,43.41n58 改进,程序报错n59 改进,已打分,43.25n60 草稿,没有改动n61 改进,已打分,43.36n62 改进,格式不合格n63 改进,已打分,43.33n64 改进,已打分,43.37n65 改进,已打分,43.42n66 改进,程序报错n67 改进,已打分,43.30n68 改进,没有改动n69 改进,已打分,43.24n70 改进,格式不合格n71 改进,已打分,43.45n72 改进,已打分,43.23n73 修复,已打分,43.39n74 修复,已打分,43.41n75 修复,已打分,43.32n76 改进,没有改动n77 改进,已打分,43.33n78 改进,格式不合格n79 草稿,已打分,43.50n80 改进,已打分,43.28n81 改进,已打分,43.36n82 改进,程序报错终选
已打分?程序正常跑完并拿到分数。程序报错?候选程序运行时出错。格式不合格?输出文件不符合提交格式(基因、细胞数、坐标等)。没有改动?这次会话没有对代码做任何修改。被中断?因重启或大模型接口故障中断,不计入搜索。
搜索分数的变化?横轴是从运行开始算起的分钟数,纵轴是到那时为止最好的搜索目标分。
42.54343.54444.50246810从开始算起的分钟数20261001-152125-search-t1-g23-fake-followup20261001-152125-search-t1-g23-fake-followup:0 分钟 43.5020261001-152125-search-t1-g23-fake-followup:1 分钟 43.5320261001-152125-search-t1-g23-fake-followup:11 分钟 43.59
全部节点与对话(82)
节点父节点操作?种子:人写的起点;改进:在父节点上改;草稿:从头写;修复:修父节点的报错。状态搜索目标分?搜索用来比较节点的分数(各把尺子按权重合成,种子 0 单次)。比父节点?这个节点的分数减去它父节点的分数;正数是变好。proxy?替代题:用手头已知阶段造的本地考题,T1 是用 E8.5 预测 E9.5。proxy2?两输入替代题:官方 E8.5 + 外部 Qiu E9.0 预测 E9.5,给需要两个时间点的方法一个公平分数。X3?用外部公开数据造的同类型题。3 次复测?换 3 个随机种子各跑一次取平均 ± 标准差,比单次分更稳。审查用时?从运行开始到结束(或到现在)的挂钟时间。方法说明?节点程序自带的 METHOD.md:这个程序做了什么、为什么。
n1—种子已打分42.5—50.027.450.042.5通过不到 1 分seed copy_last
n2—种子已打分43.5—56.227.446.843.9通过不到 1 分seed heart_jcf_peri
n3—种子已打分39.3—50.027.440.539.4通过不到 1 分seed pseudobulk_shift
n4n2改进没有改动——————未审查不到 1 分fake-plan-4
n5n1改进已打分43.3+0.855.627.446.843.8通过不到 1 分heart_jcf_peri reweighting, heart x2.0, edge x0.3 (fake Engineer, node 5)
n6n3改进格式不合格——————未审查不到 1 分heart_jcf_peri reweighting, heart x1.8, edge x0.2 (fake Engineer, node 6)
n7n2改进已打分43.5+0.056.327.446.843.7通过不到 1 分heart_jcf_peri reweighting, heart x1.6, edge x0.1 (fake Engineer, node 7)
n8n3改进已打分43.4+4.156.127.446.843.8通过不到 1 分heart_jcf_peri reweighting, heart x1.4, edge x0.35 (fake Engineer, node 8)
n9n5改进已打分43.2-0.155.427.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x1.2, edge x0.25 (fake Engineer, node 9)
n10n7改进程序报错——————未审查不到 1 分heart_jcf_peri reweighting, heart x1.9, edge x0.15 (fake Engineer, node 10)
n11n2改进已打分43.4-0.156.027.446.843.8通过不到 1 分heart_jcf_peri reweighting, heart x1.7, edge x0.4 (fake Engineer, node 11)
n12n7改进没有改动——————未审查不到 1 分fake-plan-12
n13n8改进已打分43.3-0.255.627.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x1.3, edge x0.2 (fake Engineer, node 13)
n14n2改进格式不合格——————未审查不到 1 分heart_jcf_peri reweighting, heart x2.0, edge x0.1 (fake Engineer, node 14)
n15n7改进已打分43.3-0.255.627.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x1.8, edge x0.35 (fake Engineer, node 15)
n16n14修复已打分43.5—56.227.446.843.9通过不到 1 分heart_jcf_peri reweighting, heart x1.6, edge x0.25 (fake Engineer, node 16)
n17n2改进已打分43.3-0.255.627.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x1.4, edge x0.15 (fake Engineer, node 17)
n18n16改进被中断——————未审查不到 1 分—
n19n14修复已打分43.4—55.927.446.8—未审查1 分heart_jcf_peri reweighting, heart x1.9, edge x0.3 (fake Engineer, node 19)
n20n2改进没有改动——————未审查1 分fake-plan-20
n21n16改进已打分43.3-0.255.727.446.8—未审查2 分heart_jcf_peri reweighting, heart x1.5, edge x0.1 (fake Engineer, node 21)
n22n2改进被中断——————未审查不到 1 分—
n23n7改进已打分43.4-0.156.027.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x2.0, edge x0.25 (fake Engineer, node 23)
n24n5改进已打分43.5+0.256.127.446.8—未审查2 分heart_jcf_peri reweighting, heart x1.8, edge x0.15 (fake Engineer, node 24)
n25n2改进已打分43.3-0.255.527.446.8—未审查1 分heart_jcf_peri reweighting, heart x1.6, edge x0.4 (fake Engineer, node 25)
n26n16改进程序报错——————未审查1 分heart_jcf_peri reweighting, heart x1.4, edge x0.3 (fake Engineer, node 26)
n27n8改进已打分43.4-0.155.927.446.8—未审查1 分heart_jcf_peri reweighting, heart x1.2, edge x0.2 (fake Engineer, node 27)
n28n11改进被中断——————未审查不到 1 分—
n29—草稿已打分43.3—55.627.446.8—未审查1 分heart_jcf_peri reweighting, heart x1.7, edge x0.35 (fake Engineer, node 29)
n30n11改进格式不合格——————未审查不到 1 分heart_jcf_peri reweighting, heart x1.5, edge x0.25 (fake Engineer, node 30)
n31n11改进已打分43.2-0.255.427.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x1.3, edge x0.15 (fake Engineer, node 31)
n32n2改进已打分43.3-0.255.627.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x2.0, edge x0.4 (fake Engineer, node 32)
n33n8改进已打分43.2-0.255.527.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x1.8, edge x0.3 (fake Engineer, node 33)
n34n16改进程序报错——————未审查不到 1 分heart_jcf_peri reweighting, heart x1.6, edge x0.2 (fake Engineer, node 34)
n35n11改进已打分43.4-0.055.927.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x1.4, edge x0.1 (fake Engineer, node 35)
n36n34修复已打分43.2—55.227.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x1.2, edge x0.35 (fake Engineer, node 36)
n37n16改进已打分43.5-0.056.227.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x1.9, edge x0.25 (fake Engineer, node 37)
n38n2改进格式不合格——————未审查不到 1 分heart_jcf_peri reweighting, heart x1.7, edge x0.15 (fake Engineer, node 38)
n39—草稿已打分43.4—55.827.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x1.5, edge x0.4 (fake Engineer, node 39)
n40n2改进已打分43.2-0.355.327.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x1.3, edge x0.3 (fake Engineer, node 40)
n41n16改进已打分43.3-0.255.527.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x2.0, edge x0.2 (fake Engineer, node 41)
n42n11改进程序报错——————未审查不到 1 分heart_jcf_peri reweighting, heart x1.8, edge x0.1 (fake Engineer, node 42)
n43n2改进已打分43.3-0.255.627.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x1.6, edge x0.35 (fake Engineer, node 43)
n44—草稿没有改动——————未审查不到 1 分fake-plan-44
n45n11改进已打分43.3-0.155.627.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x1.2, edge x0.15 (fake Engineer, node 45)
n46n16改进格式不合格——————未审查不到 1 分heart_jcf_peri reweighting, heart x1.9, edge x0.4 (fake Engineer, node 46)
n47n46修复已打分43.6—56.527.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x1.7, edge x0.3 (fake Engineer, node 47)
n48n16改进已打分43.4-0.156.027.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x1.5, edge x0.2 (fake Engineer, node 48)
n49n2改进已打分43.3-0.255.727.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x1.3, edge x0.1 (fake Engineer, node 49)
n50n46修复已打分43.3—55.727.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x2.0, edge x0.35 (fake Engineer, node 50)
n51n16改进已打分43.4-0.156.127.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x1.8, edge x0.25 (fake Engineer, node 51)
n52n42修复已打分43.3—55.627.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x1.6, edge x0.15 (fake Engineer, node 52)
n53n2改进已打分43.1-0.455.127.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x1.4, edge x0.4 (fake Engineer, node 53)
n54n47改进格式不合格——————未审查不到 1 分heart_jcf_peri reweighting, heart x1.2, edge x0.3 (fake Engineer, node 54)
n55n2改进已打分43.2-0.355.427.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x1.9, edge x0.2 (fake Engineer, node 55)
n56—草稿已打分43.4—56.027.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x1.7, edge x0.1 (fake Engineer, node 56)
n57n16改进已打分43.4-0.156.027.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x1.5, edge x0.35 (fake Engineer, node 57)
n58n2改进程序报错——————未审查不到 1 分heart_jcf_peri reweighting, heart x1.3, edge x0.25 (fake Engineer, node 58)
n59n11改进已打分43.3-0.255.527.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x2.0, edge x0.15 (fake Engineer, node 59)
n60—草稿没有改动——————未审查不到 1 分fake-plan-60
n61n2改进已打分43.4-0.155.827.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x1.6, edge x0.3 (fake Engineer, node 61)
n62n16改进格式不合格——————未审查不到 1 分heart_jcf_peri reweighting, heart x1.4, edge x0.2 (fake Engineer, node 62)
n63n8改进已打分43.3-0.155.727.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x1.2, edge x0.1 (fake Engineer, node 63)
n64n16改进已打分43.4-0.155.827.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x1.9, edge x0.35 (fake Engineer, node 64)
n65n2改进已打分43.4-0.156.027.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x1.7, edge x0.25 (fake Engineer, node 65)
n66n16改进程序报错——————未审查不到 1 分heart_jcf_peri reweighting, heart x1.5, edge x0.15 (fake Engineer, node 66)
n67n16改进已打分43.3-0.255.627.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x1.3, edge x0.4 (fake Engineer, node 67)
n68n2改进没有改动——————未审查不到 1 分fake-plan-68
n69n2改进已打分43.2-0.355.427.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x1.8, edge x0.2 (fake Engineer, node 69)
n70n37改进格式不合格——————未审查不到 1 分heart_jcf_peri reweighting, heart x1.6, edge x0.1 (fake Engineer, node 70)
n71n11改进已打分43.4+0.056.127.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x1.4, edge x0.35 (fake Engineer, node 71)
n72n16改进已打分43.2-0.355.427.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x1.2, edge x0.25 (fake Engineer, node 72)
n73n70修复已打分43.4—55.927.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x1.9, edge x0.15 (fake Engineer, node 73)
n74n66修复已打分43.4—56.027.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x1.7, edge x0.4 (fake Engineer, node 74)
n75n62修复已打分43.3—55.727.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x1.5, edge x0.3 (fake Engineer, node 75)
n76n16改进没有改动——————未审查不到 1 分fake-plan-76
n77n16改进已打分43.3-0.255.727.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x2.0, edge x0.1 (fake Engineer, node 77)
n78n2改进格式不合格——————未审查不到 1 分heart_jcf_peri reweighting, heart x1.8, edge x0.35 (fake Engineer, node 78)
n79—草稿已打分43.5—56.227.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x1.6, edge x0.25 (fake Engineer, node 79)
n80n16改进已打分43.3-0.255.627.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x1.4, edge x0.15 (fake Engineer, node 80)
n81n2改进已打分43.4-0.155.827.446.8—未审查不到 1 分heart_jcf_peri reweighting, heart x1.2, edge x0.4 (fake Engineer, node 81)
n82n79改进程序报错——————未审查不到 1 分heart_jcf_peri reweighting, heart x1.9, edge x0.3 (fake Engineer, node 82)
资源消耗与失败情况

格式不合格 9 没有改动 7 程序报错 7 被中断 3

角色?调研员写计划、工程师改代码、分析员解读分数、审查员检查作弊。大模型调用次数失败总用时输入 token 数?输入 / 输出(含思考)的 token 数,来自接口返回的用量。输出 token
分析员7251 分1,407335
工程师86107 分3,887941
调研员87101 分1,309308
审查员50不到 1 分6015
运行配置与来源
运行?一次完整的自动搜索或 Agent 会话,有自己的锁定配置和证据包。20261001-152125-search-t1-g23-fake-followup
10-01 15:21 · 自动搜索 · T1:val · G23 验收 · fake-followup
题目?比赛的哪道题、哪个阶段,例如 T1:val 是第 1 题的验证阶段。T1:val
方式?自动搜索:ERA 式搜索树,多个节点不断改进程序;单 Agent:一个 Agent 会话从头做到尾。自动搜索 测试
搜索目标?这次运行用哪几把尺子、按什么权重合成分数来比较节点;目标不同的运行,最佳分不能直接比。proxy+proxy2+X3 等权;终检 X1
状态已结束
开始 / 结束10-01 15:21 / 10-01 15:38
用时17 分 / 预算 24 分
所在副本?运行目录所在的代码副本(Spark 上 ~/vec 下的哪个目录)。g23_wt
目的?这次运行为了回答什么问题,写在配置文件里。[回填] TEST ONLY (G23 follow-ups): fake backend on the Spark. Scattered provider errors (every other LLM call between 150 and 450 s) must trip the rate breaker; the run must keep opening nodes until now + p80 node time > deadline - final reserve, and start the final selection when the last node is done (or at deadline - final reserve). harness.py launch --config agent/configs/search_t1_g23_fake_followup.

出处?这次运行用的代码、配置、数据和模型的指纹,靠它们可以原样找回并复现。

代码版本?运行锁定时代码仓库的 git 提交号;带 +dirty 表示当时有未提交的改动。a47b1afbd57f0a9d9e4ba69ff145ead3816d69e0 代码有未提交改动
版本标签?可提交运行在锁定的提交上打的 git 标签 run/<运行编号>,以后能原样找回代码。—
配置文件agent/configs/search_t1_g23_fake_followup.yaml
配置指纹 31739123e3eed7c8
锁定指纹?启动时把配置、代码、提示词等全部锁定后算出的总指纹。1a9c3f5365c6bd67 锁定格式 v7
运行专用代码副本?启动时为这次运行单独检出的一份只读代码;运行全程只读它,不受主副本更新影响。没有(这次运行早于运行专用代码副本功能,读的是启动时的代码副本)
打分口径指纹?打分器、数据视图、种子等合起来的指纹;指纹相同,分数才能直接比较。3c65ac0d77849967
数据视图指纹?程序能看到的输入数据文件的指纹,按视图汇总成一个短指纹。final:22 个文件,指纹 edb68fc794d79881
proxy:18 个文件,指纹 166d21c45cc35ede
proxy2:20 个文件,指纹 3a64c6f141374179
test:38 个文件,指纹 f706ad2858ee7264
大模型?每个角色用的大模型;括号里是接口实际报告的模型名。分析员:fake (接口报告 fake)
工程师:fake (接口报告 fake)
调研员:fake (接口报告 fake)
审查员:fake (接口报告 fake)
运行目录/home/spark-longxinyang/vec/g23_wt/agent/runs/20261001-152125-search-t1-g23-fake-followup 主机 spark-ad3f
提交账本?notes/changelogs/submissions.tsv:每次上传官网的记录和返回的官网分。这次运行没有提交过官网
迭代报告?运行结束后自动生成的中文复盘报告(notes/reports/runs)。还没有迭代报告