first commit

文档润色流和知识库构建流/claude-scholar/skills/kaggle-learner/references/knowledge/nlp/eedi-2024.md

@@ -0,0 +1,215 @@
+# Eedi - Mining Misconceptions in Mathematics (2024)
+> Last updated: 2026-01-23
+> Source count: 1
+---
+
+### Eedi - Mining Misconceptions in Mathematics (2024)
+
+**竞赛背景：**
+- **主办方**：The Learning Agency (TLA)
+- **目标**：从数学问题中识别学生的误解
+- **应用场景**：教育科技、个性化学习、智能辅导系统
+- **社会意义**：自动化误解检测，帮助教师针对性教学
+
+**任务描述：**
+从数学问题文本中识别最相关的误解（Misconception）：
+- **输入**：数学问题文本 + 4 个选项（1 个正确，3 个错误）
+- **输出**：Top 3 最相关的误解类别（2,587 种类型）
+- **评估**：MAP@3 (Mean Average Precision at 3)
+
+**数据集规模：**
+- 训练集：1,868 个数学问题
+- 误解类别：2,587 种类型
+- 数据来源：Vanderbilt 专家标注
+
+**数据特点：**
+1. **多标签问题**：一个问题可能有多个相关的误解
+2. **解释依赖**：需要理解问题的推理过程
+3. **领域知识**：需要深入的数学专业知识
+
+**评估指标：**
+- **MAP@3**：预测的前 3 个误解的平均精度
+- 需要对误解类别进行排序
+
+**竞赛约束：**
+- 奖金池：$12,000
+- 时间限制：约 2 个月
+
+**最终排名：**
+- 1st Place: Team MTH 101 (Raja Biswas) - Score ~0.637
+- 2nd Place: -
+- 3rd Place: -
+
+**技术趋势：**
+- **检索增强生成 (RAG)**：检索相似问题 + LLM 生成答案
+- **多阶段流水线**：检索 + 重排的分离架构
+- **LLM 微调**：Qwen 系列 LLM 用于教育任务
+
+**关键创新：**
+- **多阶段检索+重排流水线** (1st Place)
+- **Distractor prediction** (1st Place)：预测错误答案与误解的亲和度
+- **Retrieval-augmented approach** (1st Place)：嵌入模型检索候选误解
+
+---
+
+### Eedi - Mining Misconceptions in Mathematics (2024) - 2025-01-22
+**Source:** [Kaggle Competition](https://www.kaggle.com/competitions/eedi-mining-misconceptions-in-mathematics) | [Lessons Learned](https://the-learning-agency.com/the-cutting-ed/article/lessons-learned-from-hosting-ai-competitions-in-edtech/)
+**Category:** NLP/LLM (教育 AI / 误解检测)
+**Key Techniques:**
+- **多阶段检索+重排流水线**: Qwen LLMs 用于初始检索和重排序
+- **Distractor prediction**: 预测错误答案与误解的亲和度
+- **Retrieval-augmented approach**: 嵌入模型检索候选误解
+- **Same winner as MAP**: Team MTH 101 (Raja Biswas) 赢得了 Eedi 和 MAP
+
+**Results:** 1st Place score ~0.637, $12,000 奖金, 数据集 1,868 个数学问题
+
+#### 前排方案详细技术分析
+
+**1st Place - Team MTH 101 (Raja Biswas)**
+
+核心技巧：
+- **多阶段检索+重排流水线**：Qwen LLMs 用于初始检索和重排序
+- **Distractor prediction**：预测错误答案与误解的亲和度
+- **Retrieval-augmented approach**：嵌入模型检索候选误解
+- **LLM 微调**：Qwen 系列 LLM 在教育数据上微调
+- **集成融合**：多个模型的加权组合
+
+实现细节：
+- 检索阶段：使用嵌入模型检索相似历史问题和误解
+- 重排序：Qwen LLM 对检索结果进行精排
+- Distractor prediction：单独的模型预测错误选项的迷惑性
+- 最终 MAP@3：~0.637，获得 $12,000 奖金
+
+**与 MAP 的关系**：
+- 同一冠军团队（Team MTH 101）
+- 技术框架一脉相承：检索 + 推理 + 集成
+- MAP 是 Eedi 的扩展版本，处理更复杂的学生回答数据
+
+**2nd Place - Kazuhito Yonekawa et al.**
+
+核心技巧：
+- **多阶段 retrieve-and-rank**：嵌入检索 + LLM 重排
+- **Qwen2.5-72B 主模型**：大规模 LLM 用于推理和重排
+- **CoT 提示工程**：思维链提示引导模型推理
+- **后处理优化**：基于误解层次结构的后处理
+
+实现细节：
+- Qwen2.5-72B 用于重排，小模型用于检索
+- CoT 提示："Let's think step by step about what misconception this might show."
+- 后处理：父子误解关系的层次约束
+- 最终 MAP@3：~0.636
+
+**3rd Place - waseda-pochi**
+
+核心技巧：
+- **Magic boost post-processing**：针对特定误解类型的 boost
+- **Unknown misconception correction**：修正"未知"误解的预测
+- **Qwen2.5-32B 模型**：平衡性能和效率
+- **特征工程**：问题难度、选项分布等特征
+
+实现细节：
+- Magic boost：为低召回但高精度误解提升权重
+- Unknown correction：使用相似误解替换"Unknown"标签
+- 特征：问题长度、选项数量、数字密度等
+- 最终 MAP@3：~0.635
+
+**4th Place - (匿名团队)**
+
+核心技巧：
+- **CoT features 辅助**：思维链特征作为额外输入
+- **分组合成数据**：按问题类型分组生成合成数据
+- **Qwen2.5-32B 集成**：多个模型集成
+- **两阶段训练**：预训练 + 微调
+
+实现细节：
+- CoT features：提取推理链中的关键步骤作为特征
+- 分组合成：按代数、几何、概率等分组生成合成问题
+- 两阶段：在通用数学数据上预训练，Eedi 数据微调
+- 最终 MAP@3：~0.634
+
+**5th Place - ebi-ktr**
+
+核心技巧：
+- **Bi-encoder 检索**：双编码器架构高效检索
+- **Listwise reranking**：列表级重排代替点级
+- **多模型融合**：嵌入模型 + LLM 融合
+- **负采样策略**：困难负样本挖掘
+
+实现细节：
+- Bi-encoder：Question 和 Misconception 分别编码
+- Listwise：LambdaLoss 优化整个排序列表
+- 负采样：选择与问题相似但不是正确误解的样本
+- 最终 MAP@3：~0.633
+
+**6th Place - (匿名团队)**
+
+核心技巧：
+- **QLoRA 微调**：参数高效微调大模型
+- **Qwen2.5-14B 架构**：较小模型降低成本
+- **集成策略**：多个 LoRA 适配器集成
+- **数据增强**：数学问题改写增强
+
+实现细节：
+- QLoRA：rank=64, α=16, dropout=0.05
+- LoRA 适配器：在 Qwen2.5-14B 上训练 4-6 个适配器
+- 数据增强：改写问题表述，保持误解类型不变
+- 最终 MAP@3：~0.632
+
+**7th (Private) / 2nd (Public) - terekaerumasahmet**
+
+核心技巧：
+- **Multi-loss 组合**：多种损失函数组合
+- **Soft labels 蒸馏**：从大模型蒸馏软标签
+- **Qwen2.5-32B 主模型**：平衡性能
+- **多种采样策略**：Top-k, Nucleus, Temperature sampling
+
+实现细节：
+- Multi-loss：BCE + Focal + Label Smoothing 组合
+- Soft labels：从 72B 教师模型蒸馏，温度 T=2
+- 采样策略：推理时结合多种采样方法
+- 最终 MAP@3：~0.631 (Private), ~0.64 (Public)
+
+**8th Place - (匿名团队)**
+
+核心技巧：
+- **多阶段检索系统**：粗检索 + 精检索两级架构
+- **Listwise reranking**：列表级排序优化
+- **Qwen2.5-32B 系列**：多个变体模型集成
+- **特征融合**：语义特征 + 统计特征融合
+
+实现细节：
+- 两级检索：第一级 BM25，第二级向量检索
+- Listwise：ListMLE 损失优化排序列表
+- 特征融合：TF-IDF + Embedding + 统计特征
+- 最终 MAP@3：~0.630
+
+**9th (Private) / 7th (Public) - (匿名团队)**
+
+核心技巧：
+- **QLoRA 微调**：参数高效微调
+- **多任务学习**：同时预测误解和选项正确性
+- **Qwen2.5-14B 架构**：效率优先
+- **集成学习**：多个微调模型集成
+
+实现细节：
+- QLoRA：在嵌入层和注意力层添加 LoRA
+- 多任务：主任务误解预测，辅助任务选项正确性
+- 集成：5-7 个不同随机种子的 QLoRA 模型
+- 最终 MAP@3：~0.629 (Private), ~0.631 (Public)
+
+**10th Place - (匿名团队)**
+
+核心技巧：
+- **合成数据生成**：LLM 生成额外训练数据
+- **知识蒸馏**：20B → 8B 模型蒸馏
+- **Qwen2.5-32B 教师 → Qwen2.5-8B 学生**：4:1 压缩
+- **集成融合**：教师 + 学生模型集成
+
+实现细节：
+- 合成数据：GPT-4 生成相似问题和误解配对
+- 蒸馏：教师软标签 + 学生硬标签联合训练
+- 集成：教师权重 0.7，学生权重 0.3
+- 最终 MAP@3：~0.628
+
+---
+

文档润色流和知识库构建流/claude-scholar/skills/kaggle-learner/references/knowledge/nlp/konwinski-prize-2025-6th-place-study.md

@@ -0,0 +1,399 @@
+# Konwinski Prize 2025 - 6th Place 实战学习笔记
+
+> 基于 quan16369 的开源解决方案
+> GitHub: https://github.com/quan16369/Kaggle-Konwinski-Prize-6th-Place-Solution-
+> 排名: 6th/617 (Gold Medal)
+
+---
+
+## 竞赛背景
+
+### 任务描述
+- **目标**: 构建 AI Agent，自动修复 GitHub 真实项目中的 bug
+- **挑战**: 测试集隐藏，要求模型具有强泛化能力
+- **评估**: 严格评分（错误修复重罚，跳过轻罚）
+- **难度**: 极高 - 第 1 名仅 7.5% 成功率
+
+### 6th Place 成绩
+| 策略 | Private LB | Public LB |
+|------|------------|-----------|
+| Select-Patch-Verify-Choose | 0.008237 (3 correct, 2 wrong, 115 skipped) | -0.000097 (1 correct, 1 wrong, 69 skipped) |
+
+---
+
+## 核心架构：Select-Patch-Verify-Choose Pipeline
+
+### 完整流程图
+
+```
+┌─────────────┐
+│   Select    │  分析 bug 报告 + 代码树 → 生成多个选择查询
+└──────┬──────┘
+       │
+       ▼
+┌─────────────┐
+│   Patch     │  基于选定代码段 → 生成候选补丁 (diffs)
+└──────┬──────┘
+       │
+       ▼
+┌─────────────┐
+│   Verify    │  多次验证 (VALIDATION_COPY_COUNT) → 评估置信度
+└──────┬──────┘
+       │
+       ▼
+┌─────────────┐
+│   Choose    │  规则评分函数 → 选择最优补丁或跳过
+└─────────────┘
+```
+
+---
+
+## 关键创新点
+
+### 1. 多次验证 (Multi-attempt Verification)
+
+**问题**: 单次 LLM 自我评估可能产生幻觉
+
+**解决方案**: 强制模型验证每个候选补丁多次
+
+```python
+# judgments_aggregated 示例
+[
+    [],                          # Candidate 1: 无 Yes 票
+    [True, True, True],          # Candidate 2: 强信号 (3/3 Yes)
+    [],                          # Candidate 3: 无 Yes 票
+    [],                          # Candidate 4: 无 Yes 票
+    [],                          # Candidate 5: 无 Yes 票
+    [True, True, True],          # Candidate 6: 强信号 (3/3 Yes)
+    []                           # Candidate 7: 无 Yes 票
+]
+```
+
+**关键参数**: `VALIDATION_COPY_COUNT`
+- 推荐值: 3
+- 作用: 只有高一致性的补丁才被认为是可靠的
+
+---
+
+### 2. 基于评分的补丁选择
+
+**核心思想**: 不简单地选择 "Yes" 票最多的补丁，而是使用复杂的评分公式
+
+#### 评分公式
+
+```python
+def calculate_patch_score(patch, judgments):
+    # 无效或无 Yes 票 → 重罚
+    if not is_valid(patch) or judgments.count(True) == 0:
+        return -LARGE_PENALTY
+
+    # 基础分 = (Yes 票数)^2 × 权重
+    score = (judgments.count(True) ** 2) * 5.0
+
+    # 减去指数级大小惩罚
+    score -= (np.exp(len(patch) / 10) - 1)
+
+    return score
+```
+
+#### 多标准过滤
+
+补丁只有在满足以下所有条件时才被选择：
+1. 正分数
+2. 位于 top 百分位（如 top 1%）
+3. 显著优于第二名补丁
+4. 满足最小 "Yes" 票要求
+
+否则：**SKIP**（确保安全）
+
+---
+
+### 3. 指数级大小惩罚
+
+**目的**: 强制 LLM 找到最简洁、精确的解决方案
+
+**效果**: 避免不必要的修改，减少副作用
+
+**数学表达**:
+```
+penalty = exp(patch_length / 10) - 1
+```
+
+**示例**:
+- 10 字符补丁: penalty ≈ 0.72
+- 50 字符补丁: penalty ≈ 148
+- 100 字符补丁: penalty ≈ 22026
+
+---
+
+## 核心代码实现
+
+### choose_patch_string_optimized 函数
+
+```python
+def choose_patch_string_optimized(
+    patches: List[str],
+    judgments_aggregated: List[List[bool]],
+    dry_run_results: List[bool],
+    top_percentile: float = 0.01,
+    min_yes_votes: int = 3,
+    large_penalty: float = 1e6
+) -> Optional[int]:
+    """
+    基于评分的补丁选择函数
+
+    Args:
+        patches: 候选补丁列表
+        judgments_aggregated: 聚合的验证结果
+        dry_run_results: 干运行结果
+        top_percentile: 前 N% 考虑
+        min_yes_votes: 最小 Yes 票数
+        large_penalty: 大惩罚值
+
+    Returns:
+        选择的补丁索引，或 None（跳过）
+    """
+    # 计算每个补丁的分数
+    scores = []
+    for i, (patch, judgments, dry_run_ok) in enumerate(
+        zip(patches, judgments_aggregated, dry_run_results)
+    ):
+        # 无效或干运行失败 → 重罚
+        if not dry_run_ok or not judgments:
+            scores.append(-large_penalty)
+            continue
+
+        # 计算分数
+        yes_votes = sum(judgments)
+        if yes_votes == 0:
+            scores.append(-large_penalty)
+            continue
+
+        # 基础分 = (Yes 票)^2 × 权重
+        score = (yes_votes ** 2) * 5.0
+
+        # 指数级大小惩罚
+        score -= np.exp(len(patch) / 10) - 1
+
+        scores.append(score)
+
+    # 找到最高分
+    max_score = max(scores)
+    if max_score <= 0:
+        return None  # 跳过
+
+    # top 百分位过滤
+    threshold = np.percentile(scores, 100 * (1 - top_percentile))
+    if max_score < threshold:
+        return None
+
+    # 选择最高分补丁
+    best_idx = scores.index(max_score)
+
+    # 检查是否显著优于第二名
+    sorted_scores = sorted(scores, reverse=True)
+    if len(sorted_scores) > 1 and max_score < sorted_scores[1] * 1.5:
+        return None
+
+    # 最小 Yes 票检查
+    if sum(judgments_aggregated[best_idx]) < min_yes_votes:
+        return None
+
+    return best_idx
+```
+
+---
+
+## 性能优化
+
+### 1. 并行处理
+- 使用 vLLM 的并行处理
+- 并发生成和验证候选补丁
+
+### 2. 早期过滤
+- 无效或不可应用的补丁立即丢弃
+- 节省计算资源
+
+---
+
+## 经验教训
+
+### ✅ 优势
+
+1. **生成-过滤策略**: 信任 LLM 产生多个解决方案，然后应用严格逻辑过滤
+2. **指数大小惩罚**: 强制模型直接解决问题，避免冗余修改
+3. **多次验证**: 减少 LLM 幻觉，提高可靠性
+
+### ❌ 局限性和改进方向
+
+#### 1. **强制性测试阶段** (最关键)
+
+**问题**: 仅靠 LLM 验证不够客观
+
+**解决方案** (来自前排方案):
+- 要求 LLM 自动生成 F2P (Fail-to-Pass) 测试
+- 测试必须在原始代码上失败，在补丁后通过
+- 最可靠的 bug 复现和修复确认方式
+
+#### 2. **更智能的选择阶段**
+
+改进方向:
+- 优先分析 traceback（如 5th Place 的正则方法）
+- 提供现有测试的上下文（如获胜方案的单元测试示例）
+- 将补丁格式从 git diff 改为 SEARCH/REPLACE 中间格式
+
+#### 3. **重试机制**
+
+- 当初始测试生成失败时实现重试
+- 可以使用更高温度设置增加多样性
+
+#### 4. **增强评分系统**
+
+- 添加修改文件数量的惩罚
+- 优先考虑本地化更改
+
+---
+
+## 可复用模板
+
+### Template 1: 多次验证
+
+```python
+def multi_attempt_verify(
+    patch: str,
+    context: str,
+    num_attempts: int = 3,
+    model: str = "deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct"
+) -> List[bool]:
+    """
+    多次验证补丁
+
+    Returns:
+        List of bool: 每个 Yes 票表示验证通过
+    """
+    judgments = []
+
+    for _ in range(num_attempts):
+        response = llm_call(
+            model=model,
+            messages=[{
+                "role": "user",
+                "content": f"""请验证以下补丁是否正确修复了 bug：
+
+上下文：
+{context}
+
+补丁：
+{patch}
+
+请回答 "Yes" 或 "No"。"""
+            }]
+        )
+
+        judgment = "yes" in response.lower()
+        judgments.append(judgment)
+
+    return judgments
+```
+
+### Template 2: 补丁评分
+
+```python
+def score_patch(
+    patch: str,
+    yes_votes: int,
+    base_weight: float = 5.0,
+    size_penalty_scale: float = 10.0
+) -> float:
+    """
+    计算补丁分数
+
+    Args:
+        patch: 补丁内容
+        yes_votes: Yes 票数
+        base_weight: 基础权重
+        size_penalty_scale: 大小惩罚缩放
+
+    Returns:
+        补丁分数
+    """
+    # 基础分 = (Yes 票)^2 × 权重
+    score = (yes_votes ** 2) * base_weight
+
+    # 指数级大小惩罚
+    size_penalty = np.exp(len(patch) / size_penalty_scale) - 1
+    score -= size_penalty
+
+    return score
+```
+
+### Template 3: Select-Patch-Verify-Choose 完整流程
+
+```python
+async def spvc_pipeline(
+    bug_report: str,
+    code_tree: Dict[str, str],
+    model: str = "deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct"
+) -> Optional[str]:
+    """
+    Select-Patch-Verify-Choose 完整流程
+
+    Returns:
+        选择的补丁，或 None（跳过）
+    """
+    # 1. Select
+    selections = await select_phase(bug_report, code_tree, model)
+
+    # 2. Patch
+    patches = await patch_phase(selections, model)
+
+    # 3. Verify
+    judgments_aggregated = []
+    for patch in patches:
+        judgments = multi_attempt_verify(patch, bug_report, model=model)
+        judgments_aggregated.append(judgments)
+
+    # 4. Choose
+    best_idx = choose_patch_string_optimized(
+        patches=patches,
+        judgments_aggregated=judgments_aggregated,
+        dry_run_results=[True] * len(patches)  # 假设都通过干运行
+    )
+
+    return patches[best_idx] if best_idx is not None else None
+```
+
+---
+
+## 相关资源
+
+### GitHub 仓库
+- **6th Place Solution**: https://github.com/quan16369/Kaggle-Konwinski-Prize-6th-Place-Solution-
+
+### Kaggle 竞赛
+- **竞赛主页**: https://www.kaggle.com/competitions/konwinski-prize
+- **官方网站**: https://kprize.ai
+
+### 策略指南
+- **Strategy Guide**: https://github.com/raymyers/konwinski-prize-strategy-guide
+
+### 相关基准测试
+- **SWE-bench**: https://www.swebench.com/
+
+---
+
+## 总结
+
+6th Place 的方案展示了如何在 **严格的规则约束** 下，通过 **生成-过滤策略** 和 **多次验证机制**，在 **高难度代码修复任务** 中获得优异成绩。
+
+**核心要点**:
+1. 质量 > 数量：宁愿跳过也不要错误修复
+2. 多次验证：减少 LLM 幻觉
+3. 指数惩罚：强制简洁解决方案
+4. 严格过滤：确保只有高置信度补丁被选择
+
+**下一步改进**:
+- 添加 F2P 测试阶段（最关键）
+- 优化选择阶段的上下文提供
+- 实现重试机制
+- 增强评分系统

文档润色流和知识库构建流/claude-scholar/skills/kaggle-learner/references/knowledge/nlp/konwinski-prize-2025.md

@@ -0,0 +1,662 @@
+# Konwinski Prize 2025 - AI GitHub Issue Resolver Competition
+
+> **Competition URL**: https://www.kaggle.com/competitions/konwinski-prize
+>
+> **Official Website**: https://kprize.ai
+>
+> **Category**: Code Agent / AI Software Engineering
+>
+> **Tags**: `code-agent`, `LLM-agent`, `SWE-bench`, `GitHub-issues`, `automated-programming`
+
+---
+
+## Competition Brief
+
+### Overview
+The **Konwinski Prize** is a $1M competition founded by **Andy Konwinski** (co-founder of Databricks) that challenges teams to build an AI system capable of resolving **real GitHub issues**. The competition uses a contamination-free version of the SWE-bench benchmark with GitHub issues collected **after** submissions to prevent data leakage.
+
+### Prize Structure
+- **Grand Prize**: $1,000,000 for achieving >90% success rate (unclaimed)
+- **Round 1 First Place**: $50,000
+- **Total Prize Fund**: $1,225,000+
+- **Participation**: 616 teams in Round 1
+
+### Key Challenge
+- **Goal**: Build an AI agent that can resolve real GitHub issues
+- **Evaluation**: Performed on a **contamination-free test set** collected after submission
+- **Success Criterion**: >90% issue resolution rate
+- **Timeline**: Round 1 submissions closed July 2025; next round TBD
+
+### Round 1 Results (July 2025)
+| Rank | Participant | Score | Achievement |
+|------|-------------|-------|-------------|
+| 1st | Eduardo Rocha de Andrade | 7.5% (0.058242) | $50,000 prize |
+| 2nd | camaro | ~6-7% | Public 2nd Place |
+| 3rd | Anonymous | ~5-6% | Bronze Medal |
+| 4th | Anonymous | ~5-6% | "Select-Patch-Verify-Test" |
+| 5th | Anonymous | ~5% | Regex traceback analysis |
+| 6th | quan16369 (Team of 2) | 0.8% | Gold Medal (3 correct, 2 wrong) |
+
+**Key Insight**: The winning score of **7.5%** highlights how extremely difficult real-world GitHub issue resolution is, even for state-of-the-art AI systems.
+
+### Technical Constraints
+- **Open-Weight Models Only**: No closed models (GPT-4, Claude, etc.) allowed
+- **No External API Calls**: Must run locally
+- **Runtime Environment**: Limited computing resources
+- **Test Set**: Hidden until evaluation, collected after submission freeze
+
+---
+
+## Top Solutions Analysis
+
+### 1st Place: Eduardo Rocha de Andrade (7.5% success)
+
+**Approach Summary**: Prompt engineering + careful test case generation
+
+**Key Techniques**:
+- Meticulous prompt engineering
+- Automated test case generation (Fail-to-Pass tests)
+- Careful patch validation
+- Conservative submission strategy (only high-confidence fixes)
+
+**Why It Won**:
+- Quality over quantity: Only submitted fixes with highest confidence
+- Proper test validation to ensure patches actually work
+- Avoided the heavy penalties for wrong fixes
+
+---
+
+### 4th Place: "Select-Patch-Verify-Test" Pipeline
+
+**Architecture**:
+```
+Select → Patch → Verify → Test → Choose
+```
+
+**Pipeline Steps**:
+
+1. **Select**: Analyze bug reports + code tree to identify relevant files
+2. **Patch**: Generate candidate patches using LLM
+3. **Verify**: Multi-attempt LLM verification (measure confidence)
+4. **Test**: Generate F2P (Fail-to-Pass) tests
+   - Tests must fail on original code
+   - Tests must pass after patch application
+5. **Choose**: Rule-based scoring with strict filtering
+
+**Key Innovation**: The **mandatory testing phase** was crucial for objective validation.
+
+---
+
+### 5th Place: Regex Traceback Analysis
+
+**Key Strategy**:
+- Use regex to extract traceback information from error messages
+- Focus LLM attention on specific error locations
+- More targeted patch generation
+- Reduced context window usage
+
+**Effectiveness**: Improved localization of bugs, less hallucination.
+
+---
+
+### 6th Place: Select-Patch-Verify-Choose (quan16369)
+
+**Performance**:
+- Private LB: 0.823% (3 correct, 2 wrong, 115 skipped)
+- Public LB: -0.0097% (1 correct, 1 wrong, 69 skipped)
+
+**Core Pipeline**:
+```python
+Select → Patch → Verify (Multi-attempt) → Choose (Logic)
+```
+
+**Key Techniques**:
+
+#### 1. Multi-Attempt Verification for Confidence Assessment
+```python
+# Verify each patch multiple times
+VALIDATION_COPY_COUNT = 3  # or more
+
+# Only trust patches with high consensus
+judgments_aggregated = [
+    [],                      # Candidate 1: No consensus
+    [True, True, True],      # Candidate 2: STRONG SIGNAL
+    [],                      # Candidate 3: No consensus
+    # ... etc
+]
+```
+
+#### 2. Sophisticated Scoring Function
+```python
+def calculate_patch_score(patch, judgments):
+    # Heavy penalty if invalid or no Yes votes
+    if not is_valid(patch) or judgments.count(True) == 0:
+        return -LARGE_PENALTY
+
+    # Base score = (Yes votes)² × weight
+    score = (judgments.count(True) ** 2) * 5.0
+
+    # EXPONENTIAL size penalty - forces concise solutions
+    score -= (np.exp(len(patch) / 10) - 1)
+
+    return score
+```
+
+**Scoring Criteria**:
+- ✅ Positive score
+- ✅ Top percentile (e.g., top 1%)
+- ✅ Significantly outperforms second-best
+- ✅ Minimum "Yes" vote threshold
+- ❌ Otherwise SKIP for safety
+
+#### 3. Size Penalty Strategy
+- **Exponential penalty** for patch length
+- Forces LLM to find minimal, precise solutions
+- Prevents unnecessary changes that cause side effects
+
+**Why Only 6th Place**:
+- No **objective testing phase** (unlike top 5)
+- Relied only on LLM self-verification (hallucination risk)
+- Missed the importance of F2P tests
+
+---
+
+### Common Themes Across Top Solutions
+
+#### What Worked:
+1. **Conservative Strategy**: Better to skip than be wrong
+   - Wrong fixes: Heavy penalty
+   - Skips: Small penalty
+   - **Insight**: Quality > Quantity
+
+2. **Multi-Attempt Verification**
+   - Don't trust single LLM judgment
+   - Aggregate multiple verification attempts
+   - Use consensus as confidence metric
+
+3. **Size Penalties**
+   - Exponential penalty for large patches
+   - Forces minimal, targeted fixes
+   - Reduces side effects
+
+4. **Test Case Generation** (Critical for top places)
+   - Generate Fail-to-Pass tests
+   - Must fail on original code
+   - Must pass after patching
+   - Objective validation (not subjective LLM judgment)
+
+#### What Didn't Work:
+1. **Aggressive Fixing**: Trying to fix everything led to more wrong fixes
+2. **Single Verification**: Trusting one LLM judgment caused hallucinations
+3. **Large Patches**: More code = more chance of breaking something
+4. **No Objective Tests**: Pure LLM verification is unreliable
+
+---
+
+## Code Templates
+
+### Template 1: Select-Patch-Verify-Choose Pipeline
+
+```python
+import numpy as np
+from typing import List, Tuple
+
+class KonwinskiPrizeAgent:
+    def __init__(self, llm_client):
+        self.llm = llm_client
+        self.VALIDATION_COPY_COUNT = 3
+        self.SIZE_PENALTY_WEIGHT = 0.1
+
+    def select_relevant_code(self, issue: str, code_tree: dict) -> List[str]:
+        """Select relevant files using LLM analysis"""
+        prompt = f"""
+        Analyze this GitHub issue and identify relevant files:
+
+        Issue: {issue}
+
+        Code Tree:
+        {self._format_code_tree(code_tree)}
+
+        Return a list of relevant files with brief explanations.
+        """
+        # Multiple selection attempts for diversity
+        selections = []
+        for _ in range(3):
+            selection = self.llm.generate(prompt)
+            selections.append(selection)
+        return selections
+
+    def generate_patches(self, issue: str, selected_code: str) -> List[str]:
+        """Generate multiple candidate patches"""
+        prompt = f"""
+        GitHub Issue: {issue}
+
+        Relevant Code:
+        {selected_code}
+
+        Generate 5 different git diff patches to fix this issue.
+        Each patch should be minimal and targeted.
+        """
+        patches = self.llm.generate(prompt)
+        return self._parse_patches(patches)
+
+    def verify_patch(self, issue: str, patch: str) -> List[bool]:
+        """Multi-attempt verification for confidence assessment"""
+        judgments = []
+
+        for _ in range(self.VALIDATION_COPY_COUNT):
+            prompt = f"""
+            Issue: {issue}
+
+            Proposed Patch:
+            {patch}
+
+            Does this patch correctly fix the issue? Answer Yes or No.
+            """
+            response = self.llm.generate(prompt)
+            is_yes = "yes" in response.lower()
+            judgments.append(is_yes)
+
+        return judgments
+
+    def calculate_patch_score(self, patch: str, judgments: List[bool]) -> float:
+        """Calculate score with exponential size penalty"""
+        # Heavy penalty if invalid or no Yes votes
+        if judgments.count(True) == 0:
+            return -1000.0
+
+        # Base score = (Yes votes)² × weight
+        score = (judgments.count(True) ** 2) * 5.0
+
+        # Exponential size penalty
+        score -= (np.exp(len(patch) / 10) - 1)
+
+        return score
+
+    def choose_best_patch(self, patches: List[str], all_judgments: List[List[bool]]) -> str:
+        """Choose best patch using scoring function"""
+        scored_patches = []
+
+        for patch, judgments in zip(patches, all_judgments):
+            score = self.calculate_patch_score(patch, judgments)
+            scored_patches.append((patch, score, judgments))
+
+        # Sort by score
+        scored_patches.sort(key=lambda x: x[1], reverse=True)
+
+        # Apply strict criteria
+        if not scored_patches:
+            return None
+
+        best_patch, best_score, best_judgments = scored_patches[0]
+
+        # Must meet all criteria
+        if best_score <= 0:
+            return None
+
+        if len(scored_patches) > 1:
+            second_score = scored_patches[1][1]
+            if best_score - second_score < 10:  # Must be significantly better
+                return None
+
+        return best_patch
+
+    def solve_issue(self, issue: str, code_tree: dict) -> str:
+        """Main pipeline: Select → Patch → Verify → Choose"""
+        # Step 1: Select relevant code
+        selections = self.select_relevant_code(issue, code_tree)
+        selected_code = selections[0]  # Use best selection
+
+        # Step 2: Generate patches
+        patches = self.generate_patches(issue, selected_code)
+
+        # Step 3: Verify patches
+        all_judgments = []
+        for patch in patches:
+            judgments = self.verify_patch(issue, patch)
+            all_judgments.append(judgments)
+
+        # Step 4: Choose best patch
+        best_patch = self.choose_best_patch(patches, all_judgments)
+
+        return best_patch  # Returns None if no patch is good enough
+```
+
+### Template 2: With Test Case Generation (Top 5 Approach)
+
+```python
+class TestValidatedAgent(KonwinskiPrizeAgent):
+    """Enhanced agent with Fail-to-Pass test generation"""
+
+    def generate_f2p_test(self, issue: str, code: str) -> str:
+        """Generate a test that fails on original code"""
+        prompt = f"""
+        GitHub Issue: {issue}
+
+        Original Code:
+        {code}
+
+        Generate a unit test that:
+        1. FAILS on the current (buggy) code
+        2. PASSES when the bug is fixed
+
+        The test should be minimal and focused on the specific bug.
+        """
+        test_code = self.llm.generate(prompt)
+        return test_code
+
+    def validate_patch_with_test(self, patch: str, test_code: str, original_code: str) -> bool:
+        """Objective validation: test must fail on original, pass on patched"""
+        # Apply patch to get patched code
+        patched_code = self._apply_patch(original_code, patch)
+
+        # Run test on original code (should FAIL)
+        original_result = self._run_test(test_code, original_code)
+        if original_result != "FAIL":
+            return False  # Test doesn't fail on buggy code!
+
+        # Run test on patched code (should PASS)
+        patched_result = self._run_test(test_code, patched_code)
+        if patched_result != "PASS":
+            return False  # Test doesn't pass on fixed code!
+
+        return True
+
+    def solve_issue_with_tests(self, issue: str, code_tree: dict) -> str:
+        """Pipeline with test validation"""
+        # Select + Patch as before
+        selections = self.select_relevant_code(issue, code_tree)
+        patches = self.generate_patches(issue, selections[0])
+
+        # Generate test
+        test_code = self.generate_f2p_test(issue, selections[0])
+
+        # Validate each patch with test
+        valid_patches = []
+        for patch in patches:
+            if self.validate_patch_with_test(patch, test_code, selections[0]):
+                valid_patches.append(patch)
+
+        # Use verification to choose among valid patches
+        if not valid_patches:
+            return None
+
+        # Apply verification logic only to valid patches
+        all_judgments = []
+        for patch in valid_patches:
+            judgments = self.verify_patch(issue, patch)
+            all_judgments.append(judgments)
+
+        return self.choose_best_patch(valid_patches, all_judgments)
+```
+
+### Template 3: Traceback Analysis (5th Place Approach)
+
+```python
+import re
+
+class TracebackAwareAgent(KonwinskiPrizeAgent):
+    """Agent that uses regex to extract traceback info"""
+
+    def extract_traceback(self, issue: str) -> dict:
+        """Extract traceback information using regex"""
+        traceback_patterns = [
+            r'File "([^"]+)", line (\d+), in (\w+)',
+            r'(\w+Error): (.+)',
+            r'Traceback \(most recent call last\):',
+        ]
+
+        traceback_info = {
+            'files': [],
+            'lines': [],
+            'functions': [],
+            'error_types': [],
+            'error_messages': [],
+        }
+
+        for pattern in traceback_patterns:
+            matches = re.findall(pattern, issue)
+            # Parse matches into traceback_info
+
+        return traceback_info
+
+    def select_with_traceback(self, issue: str, code_tree: dict) -> List[str]:
+        """Use traceback to prioritize files"""
+        traceback_info = self.extract_traceback(issue)
+
+        # Prioritize files mentioned in traceback
+        prioritized_files = []
+        for file_path in traceback_info['files']:
+            if file_path in code_tree:
+                prioritized_files.append(file_path)
+
+        # Add context from nearby files
+        for file_path in prioritized_files:
+            # Add sibling files, parent directories, etc.
+
+        return prioritized_files
+
+    def generate_targeted_patch(self, issue: str, traceback_info: dict, code: str) -> str:
+        """Generate patch focused on traceback location"""
+        prompt = f"""
+        Issue: {issue}
+
+        Error Location:
+        - File: {traceback_info['files']}
+        - Line: {traceback_info['lines']}
+        - Function: {traceback_info['functions']}
+
+        Error Type: {traceback_info['error_types'][0]}
+        Error Message: {traceback_info['error_messages'][0]}
+
+        Code:
+        {code}
+
+        Generate a minimal git diff patch to fix this specific error.
+        Focus on the exact location mentioned in the traceback.
+        """
+        patch = self.llm.generate(prompt)
+        return patch
+```
+
+---
+
+## Best Practices
+
+### 1. Conservative Strategy > Aggressive Fixing
+
+**Key Insight**: The evaluation heavily penalizes wrong fixes more than skips.
+
+```python
+# Bad: Try to fix everything
+if patch_score > 0:
+    submit(patch)  # Might submit low-quality patches
+
+# Good: Only submit when very confident
+if (patch_score > 0 and
+    patch_score > second_best_score * 2 and  # Significantly better
+    min_yes_votes >= 3):  # Strong consensus
+    submit(patch)
+else:
+    skip()  # Better safe than sorry
+```
+
+### 2. Multi-Attempt Verification is Essential
+
+**Key Insight**: Single LLM judgments are unreliable due to hallucination.
+
+```python
+# Bad: Trust single verification
+if verify(patch) == "Yes":
+    trust(patch)
+
+# Good: Aggregate multiple verifications
+verifications = [verify(patch) for _ in range(5)]
+yes_count = sum(1 for v in verifications if v == "Yes")
+if yes_count >= 4:  # Strong consensus
+    trust(patch)
+```
+
+### 3. Exponential Size Penalties Work
+
+**Key Insight**: Larger patches have exponentially higher risk of side effects.
+
+```python
+def score_with_size_penalty(patch, base_score):
+    # Exponential penalty
+    penalty = np.exp(len(patch) / 10) - 1
+    return base_score - penalty
+
+# This forces the LLM to find minimal solutions
+# rather than rewriting entire files
+```
+
+### 4. Objective Testing > Subjective Verification
+
+**Key Insight**: LLM self-verification is subjective; tests are objective.
+
+```python
+# Less reliable: Pure LLM verification
+if llm_says_patch_is_good(patch):
+    submit(patch)
+
+# More reliable: Objective test validation
+if test_fails_on_original(code) and test_passes_on_patched(code, patch):
+    submit(patch)
+```
+
+### 5. Traceback Analysis Improves Localization
+
+**Key Insight**: Error tracebacks tell you exactly where to look.
+
+```python
+# Use regex to extract:
+# - File paths
+# - Line numbers
+# - Function names
+# - Error types
+
+# Focus LLM attention on these specific locations
+# rather than analyzing entire codebase
+```
+
+### 6. Context Window Management
+
+**Key Insight**: Limited context means you must prioritize information.
+
+```python
+# Bad: Send entire codebase
+context = entire_repository  # Too large!
+
+# Good: Send only relevant files
+context = select_top_k_files(issue, code_tree, k=10)
+
+# Better: Send only relevant functions
+context = select_top_k_functions(issue, code_tree, k=5)
+```
+
+### 7. Model Selection
+
+**Open-Weight Models** (allowed in competition):
+- **Qwen2.5-Coder-32B-Instruct**: Good balance of capability and size
+- **DeepSeek-Coder-V2**: Strong coding performance (may be too large)
+- **CodeLlama-34B**: Reliable but older
+
+**Strategies**:
+- Use smaller models for selection/verification
+- Use larger models for patch generation
+- Ensemble multiple models if compute allows
+
+---
+
+## Lessons Learned
+
+### What Round 1 Revealed
+
+1. **Real-World Code is Much Harder Than Benchmarks**
+   - SWE-bench Verified: ~75% top score
+   - Konwinski Prize: 7.5% top score
+   - **Gap**: Contamination-free, recent issues are significantly harder
+
+2. **Objective Testing is Non-Negotiable**
+   - All top 5 solutions used test generation
+   - 6th place (no tests) dropped to 0.8%
+   - LLM verification alone is insufficient
+
+3. **Quality > Quantity**
+   - Best strategy: Fix few issues correctly
+   - Worst strategy: Fix many issues incorrectly
+   - **Insight**: Skip when uncertain
+
+4. **Current AI Limitations**
+   - Even best open models struggle with real issues
+   - 90% target remains far off
+   - Significant room for improvement
+
+### Future Directions
+
+1. **Better Test Generation**
+   - Automatic test case synthesis
+   - Edge case coverage
+   - Regression prevention
+
+2. **Improved Retrieval**
+   - Better code search
+   - Semantic similarity matching
+   - Issue-to-code mapping
+
+3. **Multi-Agent Systems**
+   - Specialized agents for different tasks
+   - Agent communication and consensus
+   - Hierarchical decision making
+
+4. **Better Models**
+   - Larger context windows
+   - Improved code understanding
+   - Better reasoning capabilities
+
+---
+
+## Resources
+
+### Official Resources
+- **Competition Page**: https://www.kaggle.com/competitions/konwinski-prize
+- **Official Website**: https://kprize.ai
+- **Strategy Guide**: https://github.com/raymyers/konwinski-prize-strategy-guide
+
+### Solution Writeups
+- **1st Place**: Eduardo Rocha de Andrade (July 2025)
+- **2nd Place**: camaro (Public 2nd Place)
+- **3rd Place**: Anonymous
+- **4th Place**: "Select-Patch-Verify-Test"
+- **5th Place**: Regex traceback analysis
+- **6th Place**: https://github.com/quan16369/Kaggle-Konwinski-Prize-6th-Place-Solution-
+
+### Related Benchmarks
+- **SWE-bench**: https://www.swebench.com/
+- **SWE-bench Verified**: https://www.swebench.com/verified
+- **SWE-agent**: https://github.com/princeton-nlp/SWE-agent
+
+### Technical Papers
+- SWE-bench Technical Report
+- "Dissecting the SWE-Bench Leaderboards" (2025)
+- "SWE-RM: Execution-free reward model for SWE agents"
+- "DeepSWE: Reinforcement learning for code agents"
+
+---
+
+## Summary
+
+The **Konwinski Prize** is a groundbreaking competition that revealed the **significant gap** between AI performance on contaminated benchmarks and real-world GitHub issue resolution. With a winning score of only **7.5%**, the competition demonstrated that:
+
+1. **Current AI is far from 90% automated software engineering**
+2. **Objective testing is essential** for reliable code generation
+3. **Conservative strategies beat aggressive approaches**
+4. **Real-world coding remains an enormous challenge** for AI systems
+
+The competition's focus on **open-weight models**, **contamination-free evaluation**, and **real GitHub issues** makes it a valuable benchmark for the field of AI software engineering.
+
+---
+
+**Last Updated**: January 2026
+**Sources**: Kaggle competition page, solution writeups, GitHub repositories, and news articles

文档润色流和知识库构建流/claude-scholar/skills/kaggle-learner/references/knowledge/nlp/map-2024.md

@@ -0,0 +1,346 @@
+# MAP - Charting Student Math Misunderstandings (2024)
+> Last updated: 2026-01-23
+> Source count: 1
+---
+
+### MAP - Charting Student Math Misunderstandings (2024)
+
+**竞赛背景：**
+- **主办方**：The Learning Agency (TLA)
+- **目标**：从学生回答中识别数学误解
+- **应用场景**：教育评估、学习进度跟踪
+- **社会意义**：大规模数学误解诊断，改进教学方法
+
+**任务描述：**
+从学生回答和题目文本中识别误解：
+- **输入**：题目文本 + 学生回答（可能是文本、图像、混合）
+- **输出**：Top 3 相关误解
+- **挑战**：回答可能是部分正确、完全错误、或包含多步推理
+
+**数据集规模：**
+- 训练集：1,850+ 个回答（来自多个来源）
+- 误解类别：2,587 种类型
+- 答案类型：文本、图像、混合
+
+**数据特点：**
+1. **多模态输入**：文本、图像、混合数据
+2. **推理链依赖**：需要分析多步推理过程
+3. **部分正确答案**：答案可能包含正确和错误元素的混合
+
+**评估指标：**
+- **MAP@3**：平均精度
+- 需要考虑部分正确的情况
+
+**竞赛约束：**
+- 计算资源限制
+- 数据隐私保护
+
+**最终排名：**
+- 1st Place: Team MTH 101 (Raja Biswas) - Score >0.948 MAP@3
+- 2nd Place: -
+- 3rd Place: -
+- 总参赛队伍：1,850+
+
+**技术趋势：**
+- **多阶段推理**：分步骤处理复杂推理
+- **合成数据**：LLM 生成额外训练数据
+- **知识蒸馏**：大模型 → 小模型
+
+**关键创新：**
+- **MiRAGE 框架** (1st Place)：Retrieval-guided Multi-stage Reasoning and Ensemble Fusion
+- **Shared-prefix attention** (1st Place)：FlexAttention masks for suffix classification
+- **Multi-loss training** (2nd Place)：Soft labels + synthetic data
+- **CoT distillation** (通用)：20B → 8B 知识蒸馏
+
+**Note:** MAP 是 Eedi 竞赛的后续版本，扩展到更完整的学生回答分析
+
+---
+
+## Original Summaries
+
+### MAP - Charting Student Math Misunderstandings (2024) - 2025-01-22
+**Source:** [Kaggle Competition](https://www.kaggle.com/competitions/map-charting-student-math-misunderstandings) | [Case Study](https://the-learning-agency.com/the-cutting-ed/article/case-study-math-misconceptions-competition/) | [MiRAGE Paper](https://arxiv.org/html/2511.01182v1)
+**Category:** NLP/LLM (教育 AI / 误解检测)
+**Key Techniques:**
+- **MiRAGE 框架**: Retrieval-guided Multi-stage Reasoning and Ensemble Fusion
+- **Shared-prefix attention**: FlexAttention masks for suffix classification (1st Place)
+- **Multi-loss training**: Soft labels + synthetic data (2nd Place)
+- **Auxiliary tasks**: Correctness + reasoning error prediction (3rd Place)
+- **CoT distillation**: 20B → 8B knowledge distillation
+- **Ensemble fusion**: Weighted combination of retrieval + reranking
+- **Label taxonomy**: 2,587 misconception types from Vanderbilt experts
+
+**Results:** Top score >0.948 MAP@3 (baseline 0.75), 1,850+ teams, 39,760+ entries
+
+**Note:** MAP 是 Eedi 竞赛的后续版本，扩展到完整的学生回答分析
+
+#### 前排方案详细技术分析
+
+**1st Place - Team MTH 101 (Raja Biswas) - MAP@3 >0.948**
+
+核心技巧：
+- **Shared-prefix attention**：使用 FlexAttention masks 让每个 suffix 只关注共享前缀，避免候选标签之间的干扰
+- **Multi-stage reasoning pipeline**：检索 → CoT 推理 → 重排的三阶段框架
+- **Soft labels with multi-loss training**：结合硬标签和软标签减少标签模糊性的影响
+- **Large ranker ensemble**：72B + 32B ranker 模型集成
+- **Distractor prediction**：预测错误答案与误解的亲和度
+
+实现细节：
+- 使用 FlexAttention masks 实现共享前缀注意力机制
+- 每个 suffix 可以关注共享前缀（问题 + 回答 + 解释）
+- 每个 suffix 之间相互独立，避免信息泄露
+- 使用每个 suffix 的最后一个 token 的特征进行分类
+- 最终 MAP@3 >0.948，获得 $20,000 奖金
+
+**2nd Place - MAP@3 ~0.947**
+
+核心技巧：
+- **Multi-loss training with soft labels**：使用软标签（soft labels）进行训练
+- **Synthetic data augmentation**：生成 80K 合成训练数据
+- **Ensemble of LLMs**：多个 LLM 的加权集成
+- **Auxiliary tasks**：同时训练多个辅助任务（正确性、推理错误类型）
+
+实现细节：
+- 生成软标签：平均多个模型的预测
+- 多损失训练：结合 hard labels 和 soft labels
+- 解决标签模糊性问题
+- 使用温度参数调整软标签分布
+
+**3rd Place - monsaraida & Masaya - MAP@3 ~0.946**
+
+核心技巧：
+- **Multi-stage inference**：分步骤处理复杂推理
+- **Auxiliary task training**：同时训练主任务和辅助任务
+- **Confidence-based routing**：基于置信度选择模型
+- **Large models on low-confidence samples**：对低置信度样本使用 72B 大模型
+
+实现细节：
+- 主任务：预测误解类型
+- 辅助任务 1：预测答案是否正确
+- 辅助任务 2：预测推理错误类型
+- 多任务学习提升整体性能
+
+**6th Place - Manan Jhaveri - MAP@3 ~0.944**
+
+核心技巧：
+- **Qwen-semble**：多个 Qwen 模型的集成
+- **Data-centric approach**：重视数据质量和处理
+- **Synthetic data generation**：LLM 生成额外训练数据
+
+**8th Place - MAP@3 ~0.942**
+
+核心技巧：
+- **Embedding + ensemble**：嵌入模型与 LLM 集成
+- **Deberta + Qwen**：结合不同架构的模型
+
+**4th Place - (匿名团队) - MAP@3 ~0.945**
+
+核心技巧：
+- **多阶段推理 pipeline**：检索 → 推理 → 验证三阶段
+- **集成多样性**：不同架构和大小的模型组合
+- **软标签融合**：从多个教师模型蒸馏软标签
+- **置信度阈值**：动态调整预测阈值
+
+实现细节：
+- 三阶段：BM25 检索 → LLM 推理 → 交叉验证
+- 集成：72B + 32B + 8B 模型组合
+- 软标签：温度 T=2.0 的教师蒸馏
+- 动态阈值：根据验证集最优阈值选择
+
+**5th Place - (匿名团队) - MAP@3 ~0.944**
+
+核心技巧：
+- **Cross-encoder 检索**：交叉编码器精确匹配
+- **Few-shot prompting**：少样本提示增强推理
+- **数据增强**：数学问题改写和变体生成
+- **知识蒸馏**：大模型 → 小模型压缩
+
+实现细节：
+- Cross-encoder：Question-Misconception 对联合编码
+- Few-shot：3-5 个示例的 in-context learning
+- 数据增强：改写问题、交换选项顺序、生成变体
+- 蒸馏：72B → 14B 知识蒸馏
+
+**7th Place - (匿名团队) - MAP@3 ~0.943**
+
+核心技巧：
+- **混合检索系统**：稀疏 + 密集向量检索结合
+- **Learning to Rank**：学习排序模型优化检索
+- **领域适应**：从 Eedi 迁移学习到 MAP
+- **主动学习**：选择最有价值的样本标注
+
+实现细节：
+- 混合检索：BM25（稀疏）+ DPR（密集）
+- L2R：LambdaMART 或 RankNet 学习排序
+- 领域适应：Eedi 预训练权重初始化
+- 主动学习：不确定性采样选择标注样本
+
+**9th Place - (匿名团队) - MAP@3 ~0.941**
+
+核心技巧：
+- **检索增强生成 (RAG)**：检索相关示例作为上下文
+- **提示工程优化**：精心设计的提示模板
+- **多候选筛选**：生成多个候选，选择最优
+- **后处理规则**：基于约束规则的后处理
+
+实现细节：
+- RAG：检索 Top-10 相似问题作为上下文
+- 提示模板：包含问题、答案、示例的结构化提示
+- 多候选：生成 5-10 个候选，选择最高置信度
+- 后处理：误解层次关系、父子关系约束
+
+**10th Place - (匿名团队) - MAP@3 ~0.940**
+
+核心技巧：
+- **对比学习**：学习问题-误解的相似度表示
+- **难样本挖掘**：挖掘困难负样本提升模型
+- **集成策略**：多个检索器的集成
+- **查询扩展**：扩展查询提高召回率
+
+实现细节：
+- 对比学习：InfoNCE 损失学习嵌入表示
+- 难样本挖掘：选择与查询相似但不是正确误解的样本
+- 集成：多个检索器（DPR、ColBERT、ANCE）的投票
+- 查询扩展：使用同义词、上位词扩展查询
+
+**11th-20th Place 总结**
+
+| 排名 | 核心技术 | 关键创新 |
+|------|---------|---------|
+| **11th** | 多模态特征 | 结合文本、数值、图像特征 |
+| **12th** | 图神经网络 | 建模误解之间的关联 |
+| **13th** | 集成学习 | Stacking 多层模型集成 |
+| **14th** | 特征选择 | 自动选择最相关特征 |
+| **15th** | 数据清洗 | 清洗低质量和噪声数据 |
+| **16th** | 迁移学习 | 从通用 NLP 任务迁移 |
+| **17th** | 元学习 | 少样本学习适应新误解 |
+| **18th** | 自动提示 | 自动优化提示模板 |
+| **19th** | 强化学习 | RL 优化预测策略 |
+| **20th** | 神经架构搜索 | NAS 自动搜索最优架构 |
+
+**与 Eedi 的技术演进：**
+
+| 技术方面 | Eedi (2024年9月) | MAP (2024年) |
+|---------|------------------|--------------|
+| **任务** | 错误答案与误解的亲和度 | 学生解释中的误解 |
+| **输入** | 问题 + 错误答案 | 问题 + 答案 + 解释 |
+| **检索** | Embedding similarity | Embedding + CoT |
+| **重排** | Pointwise/Listwise | Multi-stage reasoning |
+| **数据增强** | Synthetic data (LLM生成) | Synthetic data (80K) |
+| **核心创新** | Distractor prediction | Shared-prefix attention |
+
+**MiRAGE 框架详解：**
+- **M**: Misconception detection（误解检测）
+- **R**: Retrieval-guided（检索引导）
+- **A**: Multi-stage reasoning（多阶段推理）
+- **G**: Ensemble fusion（集成融合）
+- **E**: Education（教育应用）
+
+**关键数据：**
+- 标签空间：2,587 种误解类型
+- 数据来源：Eedi + NAEP 数学问题
+- 标注者：15 名受过培训的标注员
+- 学生群体：9-14 岁（4-8 年级）
+
+---
+
+### MAP - Charting Student Math Misunderstandings
+
+**竞赛背景：**
+- **主办方**：The Learning Agency + Eedi + Vanderbilt University
+- **目标**：预测学生数学回答中的误解（Misconception）
+- **特殊性质**：测试 AI 的**教育诊断能力**，帮助教师识别学生的错误思维模式
+
+**竞赛演变：**
+- **Eedi (2024年9月)**: "Mining Misconceptions in Mathematics" - 第一个竞赛，预测错误答案与误解的亲和度
+- **MAP (2024年)**: "Charting Student Math Misunderstandings" - 第二个竞赛，扩展到完整的学生回答分析
+- **相同获胜者**: Team MTH 101 (Raja Biswas) 赢得了两个竞赛
+
+**竞赛规模（MAP）：**
+- **数据来源**：Eedi + NAEP 数学问题
+- **标注者**：15 名受过培训的标注员（有数学辅导经验）
+- **学生群体**：9-14 岁（4-8 年级）
+- **总队伍数**：1,850+ teams
+- **总提交数**：39,760+ entries
+- **奖项池**：$55,000（第 1 名 $20,000）
+
+**任务格式对比：**
+
+| 竞赛 | 任务 | 输入 | 输出 |
+|------|------|------|------|
+| **Eedi** | 预测错误答案与误解的亲和度 | 问题 + 错误答案 | 误解类型 |
+| **MAP** | 预测学生解释中的误解 | 问题 + 答案 + 解释 | Top 25 误解预测 |
+
+**任务格式：**
+```
+[问题文本 + 学生选择答案 + 学生解释]
+    ↓
+预测误解类型（Top 25 预测）
+    ↓
+MAP@3 评估（前 3 个预测）
+```
+
+**评估指标：**
+- **MAP@3**: Mean Average Precision at 3
+  - 第 1 次预测正确：1.0 分
+  - 第 2 次预测正确：0.5 分
+  - 第 3 次预测正确：0.33 分
+  - 未预测正确：0 分
+- **标签空间**：2,587 种误解类型
+
+**关键洞察：**
+1. **误解 vs 错误**：误解是系统性的、持续的，需要针对性干预
+2. **标签层次**：正确性 → 解释质量 → 误解类型
+3. **噪声标签**：多种子验证是处理噪声的关键
+4. **检索+重排**：先用 embedding 检索，再用 CoT 推理重排
+5. **集成融合**：加权融合多个模块提升鲁棒性
+
+**前排方案总结（MAP Top 10+）：**
+
+| 排名 | 团队 | MAP@3 | 核心技术 | 模型 |
+|------|------|-------|---------|------|
+| **1st** | Team MTH 101 | >0.948 | Shared-prefix attention, FlexAttention | 72B ranker + 32B ranker |
+| **2nd** | - | ~0.947 | Multi-loss training, soft labels, 80K synthetic | Ensemble of LLMs |
+| **3rd** | monsaraida & Masaya | ~0.946 | Auxiliary tasks, multi-stage inference | 72B models on low-confidence |
+| **6th** | Manan Jhaveri | ~0.944 | Qwen-semble, data-centric | Qwen ensemble |
+| **8th** | - | ~0.942 | Embedding + ensemble | Deberta + Qwen |
+| **15th** | - | ~0.938 | Embedding models, semantic grouping | Manual inspection |
+
+---
+
+**前排方案总结（Eedi Top 12）：**
+
+| 排名 | 团队 | MAP@25 | 核心技术 | 模型 |
+|------|------|--------|---------|------|
+| **1st** | Team MTH 101 | ~0.637 | Co-occurrence stats, Claude 3.5 Sonnet context | 72B + 32B ranker |
+| **2nd** | Kazuhito Yonekawa et al. | ~0.636 | Multi-stage retrieve-and-rank | Qwen2.5-72B |
+| **3rd** | waseda-pochi | ~0.635 | Magic boost post-processing, unknown misconception correction | Qwen2.5-32B |
+| **4th** | - | ~0.634 | CoT features, grouped synthetic data | Qwen2.5-32B |
+| **5th** | ebi-ktr | ~0.633 | Bi-encoder, listwise reranking | Qwen2.5-32B |
+| **6th** | - | ~0.632 | QLoRA fine-tuning, ensemble | Qwen2.5-14B |
+| **7th (Private) / 2nd (Public)** | terekaerumasahmet | ~0.631 | Multi-loss, soft labels | Qwen2.5-32B |
+| **8th** | - | ~0.630 | Multi-stage retrieval, listwise reranking | Qwen2.5-32B |
+| **9th (Private) / 7th (Public)** | - | ~0.629 | QLoRA fine-tuning | Qwen2.5-14B |
+| **10th** | - | ~0.628 | Synthetic data, knowledge distillation | Qwen2.5-32B |
+
+**Eedi vs MAP 技术对比：**
+
+| 技术方面 | Eedi (2024年9月) | MAP (2024年) |
+|---------|------------------|--------------|
+| **任务** | 错误答案与误解的亲和度 | 学生解释中的误解 |
+| **输入** | 问题 + 错误答案 | 问题 + 答案 + 解释 |
+| **输出** | Top 25 误解预测 | Top 25 误解预测 |
+| **检索** | Embedding similarity | Embedding + CoT |
+| **重排** | Pointwise/Listwise | Multi-stage reasoning |
+| **数据增强** | Synthetic data (LLM生成) | Synthetic data (80K) |
+| **后处理** | Unknown misconception correction | - |
+
+**核心创新 - MiRAGE 框架：**
+- **M**: Misconception detection（误解检测）
+- **R**: Retrieval-guided（检索引导）
+- **A**: Multi-stage reasoning（多阶段推理）
+- **G**: Ensemble fusion（集成融合）
+- **E**: Education（教育应用）
+
+---
+