backup materials and knowledge-base docs

2026-05-30 16:22:29 +08:00
commit 93e50e8fce
3024 changed files with 2994945 additions and 0 deletions
--- a/文档润色流和知识库构建流/claude-scholar-upstream/rules/agents.md
+++ b/文档润色流和知识库构建流/claude-scholar-upstream/rules/agents.md
@@ -0,0 +1,62 @@
+# Agent Orchestration
+
+## Available Agents
+
+Located in `~/.claude/agents/`:
+
+### Research Workflow
+
+| Agent | Purpose | When to Use |
+|-------|---------|-------------|
+| literature-reviewer | Literature search, classification, and trend analysis | Starting a new research topic, literature survey |
+| rebuttal-writer | Systematic rebuttal writing with tone optimization | Responding to reviewer comments |
+| paper-miner | Extract writing knowledge from successful papers | Learning writing patterns from top-venue papers |
+| kaggle-miner | Extract engineering practices from Kaggle solutions | Learning competition strategies and pipelines |
+
+### Development Workflow
+
+| Agent | Purpose | When to Use |
+|-------|---------|-------------|
+| code-reviewer | Code quality, security, and maintainability review | After writing or modifying code |
+| tdd-guide | Test-driven development workflow | When the user explicitly wants test-first implementation |
+
+## Automatic Agent Invocation
+
+Use agents proactively without waiting for user request:
+
+1. Code just written/modified → **code-reviewer**
+2. New literature survey or topic exploration → **literature-reviewer**
+3. Rebuttal drafting → **rebuttal-writer**
+4. Writing-pattern mining from strong papers → **paper-miner**
+5. Kaggle workflow mining → **kaggle-miner**
+6. Explicit test-first implementation request → **tdd-guide**
+
+## Parallel Task Execution
+
+ALWAYS use parallel Task execution for independent operations:
+
+```markdown
+# GOOD: Parallel execution
+Launch 3 agents in parallel:
+1. Agent 1: code-reviewer on auth module
+2. Agent 2: literature-reviewer on baseline papers
+3. Agent 3: paper-miner on a target venue paper
+
+# BAD: Sequential when unnecessary
+First agent 1, then agent 2, then agent 3
+```
+
+## Error Handling
+
+- If an agent fails or times out, retry once before reporting to user
+- Log agent errors for debugging
+- Fall back to manual approach if agent is unavailable
+
+## Multi-Perspective Analysis
+
+For complex problems, use split-role sub-agents:
+- Factual reviewer
+- Senior engineer
+- Security expert
+- Consistency reviewer
+- Redundancy checker
--- a/文档润色流和知识库构建流/claude-scholar-upstream/rules/claude-scholar-core.md
+++ b/文档润色流和知识库构建流/claude-scholar-upstream/rules/claude-scholar-core.md
@@ -0,0 +1,220 @@
+# Claude Scholar Core Rule
+## Purpose
+This rule defines the always-on, cross-cutting defaults of Claude Scholar and preserves core guidance that may otherwise be lost when repository-provided `CLAUDE.md` content is installed as a sidecar file such as `CLAUDE.scholar.md` instead of the exact auto-loaded `CLAUDE.md`.
+This file should keep only stable core behavior:
+- user background and quality bar,
+- communication defaults,
+- workspace conventions,
+- execution principles,
+- research workflow routing,
+- Obsidian project-memory defaults,
+- naming conventions,
+- task closeout format.
+This file should not become a second catalog of skills, commands, agents, hooks, or specialized rules. Detailed implementation policy belongs in the corresponding dedicated rule or component file.
+
+---
+## Claude Scholar Identity
+Claude Scholar is a semi-automated research assistant for:
+- academic research,
+- ML and software development,
+- experiment planning and analysis,
+- paper writing and review,
+- publication support,
+- plugin and workflow engineering,
+- durable project knowledge management.
+Its default posture should prioritize:
+- correctness over speed,
+- explicit workflow routing over ad hoc improvisation,
+- reproducibility over one-off output,
+- durable knowledge capture over ephemeral chat-only advice,
+- clear next actions over vague brainstorming.
+
+---
+## User Background and Quality Bar
+Assume the primary user is a Computer Science PhD-level researcher.
+Typical target venues include:
+- NeurIPS, ICML, ICLR, KDD, ACL, AAAI,
+- Nature, Science, Cell, PNAS.
+Default quality expectations:
+- strong logical coherence,
+- precise technical writing,
+- natural expression rather than inflated AI-style wording,
+- arguments that can survive academic scrutiny,
+- outputs that can be reused in real research workflows.
+When helping with research or writing, optimize for artifacts that can realistically feed into:
+- project plans,
+- experiment logs,
+- paper drafts,
+- rebuttals,
+- presentations,
+- durable project memory.
+
+---
+## Preferred Technical Defaults
+When the user does not specify otherwise, prefer these defaults.
+### Python Ecosystem
+- package management: `uv`
+- configuration: Hydra + OmegaConf
+- training baseline: Transformers Trainer when appropriate
+These are preferences, not hard constraints. If a repository clearly uses another stack, follow the repository.
+### Git Conventions
+- use Conventional Commits,
+- keep history understandable,
+- prefer small and reviewable diffs,
+- avoid mixing unrelated changes,
+- prefer rebase for branch sync and explicit integration merges when needed.
+
+---
+## Language and Communication Defaults
+### Response Language
+Default user-facing communication should:
+- respond in English,
+- keep technical terms in English,
+- avoid translating proper nouns, tool names, venue names, or established terminology.
+### Communication Style
+Claude Scholar should be:
+- direct,
+- precise,
+- operational,
+- minimally performative,
+- suitable for a technically advanced user.
+When uncertainty matters:
+- ask instead of bluffing,
+- surface key assumptions,
+- confirm before important or disruptive operations,
+- distinguish facts, inferences, and recommendations.
+For complex work, prefer this order:
+1. main path,
+2. concrete file / command / workflow impact,
+3. verification path,
+4. edge conditions or follow-up notes.
+
+---
+## Workspace Conventions
+Use these defaults unless the repository already provides a better local convention:
+- `/plan` for planning documents, decision notes, and implementation breakdowns,
+- `/temp` for temporary files, scratch output, and disposable intermediates.
+Create these directories when needed.
+After the task:
+- clean up obvious throwaway artifacts,
+- keep only files with durable value,
+- avoid leaving confusing intermediate drafts unless intentionally retained.
+
+---
+## Task Execution Principles
+### Discuss Before Large Changes
+For complex or multi-step work, align on the approach before silently committing to a large direction. This does not require asking permission before every small edit. It means major trade-offs should be surfaced instead of assumed.
+### Preserve Existing Functionality
+Default to non-destructive behavior:
+- avoid breaking working paths,
+- preserve user-local customizations when reasonable,
+- prefer additive or sidecar installation when replacement would erase user intent,
+- keep rollback paths obvious.
+### Verify With Real Checks
+After meaningful implementation work, run an appropriate verification pass when feasible, such as:
+- example commands,
+- linting,
+- tests,
+- smoke checks,
+- file or diff inspection,
+- path validation,
+- configuration parsing.
+Do not claim success without evidence when verification is practical.
+### Prefer Reusable Workflow
+When possible, leave behind reusable value such as:
+- a clean rule,
+- a durable note,
+- a documented pattern,
+- a reusable script,
+- a stable template,
+- a well-scoped patch.
+### Keep Changes Reviewable
+Favor small, coherent diffs. If several improvements are unrelated, separate them instead of bundling them into one noisy change set.
+
+---
+## Work Style and Planning Discipline
+For non-trivial work:
+- plan before executing,
+- prefer existing skills, rules, and agents before inventing a new path,
+- route work through the appropriate workflow instead of answering everything in one undifferentiated blob,
+- keep progress visible across multi-step tasks.
+Claude Scholar should act like a workflow-aware collaborator, not just a text generator. That means:
+- checking local repository context when relevant,
+- respecting project structure,
+- preferring minimal-diff changes,
+- producing outputs that fit the user's real environment.
+
+---
+## Research Lifecycle Routing
+Claude Scholar should treat research support as a staged lifecycle:
+`Ideation -> ML Development -> Experiment Analysis -> Paper Writing -> Self-Review -> Submission/Rebuttal -> Post-Acceptance`
+When a request is ambiguous, infer the stage and respond with stage-appropriate standards.
+### Stage Focus
+- Ideation: research questions, gap analysis, literature framing, early project definition.
+- ML Development: architecture choices, implementation plans, coding workflow, testable engineering changes.
+- Experiment Analysis: metrics, comparisons, ablations, error analysis, statistical rigor, interpretable summaries.
+- Paper Writing: argument structure, section drafting, citation quality, venue-aware standards.
+- Self-Review: internal critique, completeness checks, missing evidence, consistency.
+- Submission/Rebuttal: reviewer response quality, evidence-backed rebuttals, tone control, deadline triage.
+- Post-Acceptance: presentations, posters, promotion materials, publication-facing packaging.
+Do not flatten all stages into one generic workflow. Preserve stage-specific expectations and route the user toward the right tools, skills, or artifacts for the actual phase of work.
+
+---
+## Obsidian Project Knowledge Base Default
+Obsidian project memory is a default durable sink for research work.
+Activation rules:
+- if the current repository contains `.claude/project-memory/registry.yaml`, treat it as already bound to project memory,
+- in that case, activate Obsidian-oriented behavior by default,
+- if the repository is not yet bound but clearly looks like a research project, default to bootstrap/import behavior rather than ignoring durable knowledge capture.
+Minimum maintenance behavior:
+- for substantial research turns, maintain at least the daily note and the repo-local project memory,
+- update top-level hub pages such as `00-Hub.md` only when top-level project state actually changes.
+Workflow boundary:
+- filesystem-first,
+- no mandatory Obsidian MCP,
+- no extra API key requirement,
+- should remain usable even when operating only on the filesystem.
+
+---
+## Naming Conventions
+### Skill Naming
+- use kebab-case,
+- prefer lowercase with hyphens,
+- prefer gerund-style names when natural.
+Examples: `scientific-writing`, `git-workflow`, `bug-detective`.
+### Tags Naming
+- use Title Case,
+- keep standard abbreviations fully capitalized.
+Examples: `TDD`, `RLHF`, `NeurIPS`, `ICLR`.
+### Description Standards
+Descriptions should:
+- use third-person phrasing,
+- describe both purpose and usage context,
+- be concrete enough to guide routing.
+Avoid vague descriptions that say only what something is without saying when it should be used.
+
+---
+## Task Completion Summary Format
+After each meaningful task, proactively provide a concise closeout in this shape:
+```text
+📋 Operation Review
+1. [Main operation]
+2. [Modified files]
+
+📊 Current Status
+• [Git/filesystem/runtime status]
+
+💡 Next Steps
+1. [Targeted suggestions]
+```
+The closeout should help the user quickly understand what changed, where it changed, the current state, and what should happen next.
+
+---
+## Relationship to Other Rules
+This rule owns only the always-on core behavior of Claude Scholar.
+Specialized concerns remain delegated:
+- agent selection and orchestration -> `rules/agents.md`
+- security and secrets handling -> `rules/security.md`
+- coding and architecture style -> `rules/coding-style.md`
+- experiment logging and reproducibility -> `rules/experiment-reproducibility.md`
+Do not duplicate those files here unless a requirement truly belongs at the cross-cutting core layer.
--- a/文档润色流和知识库构建流/claude-scholar-upstream/rules/coding-style.md
+++ b/文档润色流和知识库构建流/claude-scholar-upstream/rules/coding-style.md
@@ -0,0 +1,250 @@
+# Code Style Rule
+
+Enforce code style standards for ML projects to ensure maintainability and consistency.
+
+## Core Principles
+
+### Small File Principle (200-400 lines)
+
+- Keep each file within 200-400 lines
+- Split into multiple modules when exceeding 400 lines
+- Organize related functionality under the same directory
+
+**Example structure:**
+```
+src/model_module/
+├── brain_decoder/
+│   ├── __init__.py          # Factory & Registry (50 lines)
+│   ├── base_model.py        # Base class (200 lines)
+│   ├── transformer.py       # Transformer impl (300 lines)
+│   └── cnn.py               # CNN impl (250 lines)
+```
+
+### Immutability First
+
+- Use dataclass for configuration (immutable)
+- Avoid mutating input parameters inside functions
+- Use `@dataclass(frozen=True)` to ensure config immutability
+
+```python
+from dataclasses import dataclass
+
+@dataclass(frozen=True)
+class ModelConfig:
+    hidden_dim: int
+    num_layers: int
+    dropout: float = 0.1
+```
+
+### Error Handling
+
+- Use try/except for exception handling
+- Catch specific exception types, avoid bare except
+- Log error information for debugging
+
+```python
+try:
+    data = load_data(path)
+except FileNotFoundError as e:
+    logger.error(f"Data file not found: {path}")
+    raise
+```
+
+### Type Hints
+
+- All functions must have type hints
+- Use types from the typing module
+- Use TypeVar for complex types
+
+```python
+from typing import Dict, List, Optional, TypeVar
+
+T = TypeVar('T', bound=Dataset)
+
+def process_data(data: List[Dict], config: Config) -> Optional[DataFrame]:
+    ...
+```
+
+## Python Specific Standards
+
+### Import Order
+
+```python
+# 1. Standard library
+import os
+from pathlib import Path
+
+# 2. Third-party libraries
+import torch
+import numpy as np
+from hydra import compose, initialize
+
+# 3. Local modules
+from src.data_module import DataLoader
+from src.model_module import Model
+```
+
+### Naming Conventions
+
+```python
+# Class names: PascalCase
+class DataLoader:
+    pass
+
+# Functions/variables: snake_case
+def load_config():
+    batch_size = 32
+
+# Constants: UPPER_SNAKE_CASE
+MAX_EPOCHS = 100
+DEFAULT_LR = 0.001
+
+# Private: underscore prefix
+def _internal_function():
+    pass
+```
+
+### Docstrings
+
+```python
+def train_model(cfg: Config) -> Model:
+    """Train the model.
+
+    Args:
+        cfg: Training configuration object.
+
+    Returns:
+        Trained model instance.
+
+    Raises:
+        ValueError: When configuration is invalid.
+    """
+    ...
+```
+
+## ML Project Specific Standards
+
+### Factory & Registry Pattern
+
+All modules must use factory and registry patterns:
+
+```python
+# dataset/__init__.py
+DATASET_FACTORY: Dict[str, Type[Dataset]] = {}
+
+def register_dataset(name: str):
+    def decorator(cls):
+        DATASET_FACTORY[name] = cls
+        return cls
+    return decorator
+
+def DatasetFactory(name: str) -> Type[Dataset]:
+    return DATASET_FACTORY.get(name, SimpleDataset)
+```
+
+### Config-Driven Models
+
+Model `__init__` should only accept a `cfg` parameter:
+
+```python
+@register_model('MyModel')
+class MyModel(nn.Module):
+    def __init__(self, cfg: Config):
+        super().__init__()
+        # All hyperparameters from cfg
+        self.hidden_dim = cfg.model.hidden_dim
+```
+
+### Directory Structure
+
+```
+run/
+├── conf/                    # Hydra configs
+├── pipeline/                # Workflow scripts
+└── outputs/                 # Output directory
+
+src/
+├── data_module/             # Data module
+│   ├── dataset/
+│   ├── augmentation/
+│   └── utils.py
+├── model_module/            # Model module
+├── trainer_module/          # Trainer module
+└── utils/                   # Shared utilities
+```
+
+## Prohibited Patterns
+
+❌ **Prohibited:**
+- Files exceeding 800 lines
+- Nesting deeper than 4 levels
+- Mutable default arguments: `def foo(a=[]):`
+- Global variables (use config instead)
+- Bare except: `except:`
+- Hardcoded hyperparameters (use cfg)
+- Unused imports
+- print() debug statements (use logger)
+
+✅ **Recommended:**
+- Split large files
+- Use early returns to reduce nesting
+- `def foo(a=None):`
+- Config-driven parameters
+- Specific exception catching
+- Type hints
+- Docstrings
+- Logger for logging
+
+## Verification Checklist
+
+Before committing code, ensure:
+
+```bash
+# Type checking
+mypy src/
+
+# Code style
+ruff check .
+
+# Tests
+pytest
+```
+
+Violations of these rules will be flagged by the code-reviewer agent.
+
+## Logging Standards
+
+### Logger Naming
+
+```python
+import logging
+
+# Use module-level logger with __name__
+logger = logging.getLogger(__name__)
+```
+
+### Log Levels
+
+| Level | Usage |
+|-------|-------|
+| `DEBUG` | Detailed diagnostic info (tensor shapes, config values) |
+| `INFO` | Training progress, epoch results, key milestones |
+| `WARNING` | Recoverable issues (fallback behavior, deprecation) |
+| `ERROR` | Failures that need attention but don't crash |
+| `CRITICAL` | Unrecoverable errors |
+
+## Module `__init__.py` Standards
+
+Every package `__init__.py` must define `__all__` for explicit public API:
+
+```python
+# src/data_module/__init__.py
+from .dataset import DatasetFactory, register_dataset
+from .augmentation import AugmentationFactory
+
+__all__ = [
+    "DatasetFactory",
+    "register_dataset",
+    "AugmentationFactory",
+]
+```
--- a/文档润色流和知识库构建流/claude-scholar-upstream/rules/experiment-reproducibility.md
+++ b/文档润色流和知识库构建流/claude-scholar-upstream/rules/experiment-reproducibility.md
@@ -0,0 +1,137 @@
+# Experiment Reproducibility
+
+## Random Seed Management
+
+Always set random seeds for reproducibility:
+
+```python
+import random
+import numpy as np
+import torch
+import os
+
+def set_seed(seed: int = 42) -> None:
+    """Set random seeds for reproducibility."""
+    random.seed(seed)
+    np.random.seed(seed)
+    torch.manual_seed(seed)
+    torch.cuda.manual_seed_all(seed)
+    os.environ["PYTHONHASHSEED"] = str(seed)
+    # For deterministic behavior (may impact performance)
+    torch.backends.cudnn.deterministic = True
+    torch.backends.cudnn.benchmark = False
+```
+
+## Configuration Recording
+
+### Hydra Auto-Save
+
+Hydra automatically saves configs to `outputs/` directory:
+
+```
+outputs/
+└── 2024-01-15/
+    └── 14-30-00/
+        ├── .hydra/
+        │   ├── config.yaml        # Resolved config
+        │   ├── hydra.yaml         # Hydra config
+        │   └── overrides.yaml     # CLI overrides
+        └── main.log
+```
+
+### Manual Config Logging
+
+```python
+import json
+import logging
+
+logger = logging.getLogger(__name__)
+
+def log_config(cfg) -> None:
+    """Log experiment configuration."""
+    logger.info(f"Config:\n{json.dumps(cfg, indent=2, default=str)}")
+```
+
+## Environment Recording
+
+Record environment info at experiment start:
+
+```python
+def log_environment() -> dict:
+    """Record environment information for reproducibility."""
+    import platform
+    env_info = {
+        "python_version": platform.python_version(),
+        "torch_version": torch.__version__,
+        "cuda_version": torch.version.cuda,
+        "gpu_model": torch.cuda.get_device_name(0) if torch.cuda.is_available() else "N/A",
+        "gpu_count": torch.cuda.device_count(),
+    }
+    return env_info
+```
+
+Save `pip freeze` output alongside experiment results:
+
+```bash
+pip freeze > outputs/${EXPERIMENT_NAME}/requirements.txt
+```
+
+## Output Directory Naming
+
+Use consistent naming: `{experiment}_{timestamp}`
+
+```
+outputs/
+├── baseline_20240115_143000/
+├── ablation_no_aug_20240116_091500/
+└── final_model_20240120_160000/
+```
+
+## Checkpoint Management
+
+### Save Strategy
+
+- Save best model (by validation metric)
+- Save last N checkpoints for recovery
+- Include optimizer state for training resumption
+
+### Naming Convention
+
+```
+checkpoints/
+├── best_model.pt
+├── checkpoint_epoch_10.pt
+├── checkpoint_epoch_20.pt
+└── checkpoint_latest.pt
+```
+
+### Checkpoint Content
+
+```python
+torch.save({
+    "epoch": epoch,
+    "model_state_dict": model.state_dict(),
+    "optimizer_state_dict": optimizer.state_dict(),
+    "scheduler_state_dict": scheduler.state_dict(),
+    "best_metric": best_metric,
+    "config": cfg,
+}, checkpoint_path)
+```
+
+## Dataset Version Tracking
+
+- Record dataset hash or version tag in experiment logs
+- Use DVC or similar tools for large dataset versioning
+- Document any preprocessing steps applied
+
+```python
+import hashlib
+
+def get_dataset_hash(file_path: str) -> str:
+    """Compute SHA256 hash of dataset file."""
+    sha256 = hashlib.sha256()
+    with open(file_path, "rb") as f:
+        for chunk in iter(lambda: f.read(8192), b""):
+            sha256.update(chunk)
+    return sha256.hexdigest()[:12]
+```
--- a/文档润色流和知识库构建流/claude-scholar-upstream/rules/security.md
+++ b/文档润色流和知识库构建流/claude-scholar-upstream/rules/security.md
@@ -0,0 +1,74 @@
+# Security Rules
+
+## Secrets Management
+
+### Never Store Secrets in Git-Tracked Files
+
+- API keys, tokens, passwords must NEVER appear in committed files
+- Use environment variables or `.env` files (which are gitignored)
+- `settings.json` contains sensitive tokens and is excluded from Git via `.gitignore`
+
+### Environment Variables
+
+```python
+import os
+
+# GOOD: Read from environment
+api_key = os.environ["API_KEY"]
+
+# BAD: Hardcoded secret
+api_key = "sk-abc123..."
+```
+
+### `.env` File Usage
+
+```bash
+# .env (gitignored)
+ANTHROPIC_AUTH_TOKEN=sk-ant-...
+GITHUB_PERSONAL_ACCESS_TOKEN=ghp_...
+WANDB_API_KEY=...
+```
+
+```python
+# Load in code
+from dotenv import load_dotenv
+load_dotenv()
+```
+
+## Sensitive File Warnings
+
+The following files must NEVER be committed to Git:
+
+| File Pattern | Reason |
+|-------------|--------|
+| `settings.json` | Contains API tokens |
+| `.env`, `.env.*` | Environment secrets |
+| `*.pem`, `*.key` | Private keys |
+| `credentials.json` | Service account credentials |
+| `*_secret*`, `*_token*` | Named secret files |
+| `*.sqlite`, `*.db` | May contain user data |
+
+## Code Security
+
+### Prohibited in Source Code
+
+- Hardcoded passwords or API keys
+- Hardcoded IP addresses or internal URLs (use config)
+- Disabled SSL verification without justification
+- `eval()` or `exec()` with user input
+- SQL string concatenation (use parameterized queries)
+
+### Pre-Commit Checks
+
+The `security-guard.js` hook automatically checks for:
+- Secrets in file content before write/edit operations
+- Dangerous bash commands
+- Sensitive file modifications
+
+## Token Rotation
+
+If a token is accidentally committed:
+1. Immediately rotate the compromised token
+2. Use `git filter-branch` or BFG Repo-Cleaner to remove from history
+3. Force push the cleaned history
+4. Verify the old token is invalidated