first commit

This commit is contained in:
2026-06-11 03:33:14 +08:00
commit 5f555bf342
599 changed files with 142347 additions and 0 deletions

View File

@@ -0,0 +1,199 @@
# arXiv Literature Search Guide
## Overview
This guide provides workflows for discovering and evaluating recent ML research papers on arXiv. Use this when conducting literature reviews, finding related work, or staying updated on recent publications.
---
## Search Strategies
### 1. Keyword-Based Search
**arXiv Search URL Pattern:**
```
https://arxiv.org/search/?searchtype=all&query=KEYWORDS&abstracts=show&order=-announced_date_first
```
**Common ML Search Keywords:**
- **General ML**: `machine learning`, `deep learning`, `neural networks`
- **Specific Areas**: `reinforcement learning`, `transformer`, `attention mechanism`, `graph neural networks`
- **Applications**: `computer vision`, `natural language processing`, `reinforcement learning`
- **Methods**: `self-supervised learning`, `contrastive learning`, `foundation models`
**Tips:**
- Combine keywords with `+` for AND operation
- Use `|` for OR operation
- Put phrases in quotes for exact matches: `"attention is all you need"`
### 2. Category-Based Search
**Recommended arXiv Categories for ML:**
- `cs.LG` (Machine Learning)
- `cs.AI` (Artificial Intelligence)
- `cs.CV` (Computer Vision and Pattern Recognition)
- `cs.CL` (Computation and Language)
- `cs.NE` (Neural and Evolutionary Computing)
- `stat.ML` (Machine Learning - Statistics)
**Category Filter URL:**
```
https://arxiv.org/search/?cat:cs.LG+OR+cat:cs.AI+AND+all:transformer&abstracts=show&order=-announced_date_first
```
### 3. Time-Based Filtering
**Recent Papers (Last 3 Months):**
- Use `order=-announced_date_first` for newest first
- Manually filter by submission date
- Check paper metadata for submission date
---
## Using Chrome MCP for arXiv Search
When available, use Chrome MCP tools for automated arXiv searching:
1. **Navigate to arXiv search** using Chrome MCP navigation
2. **Extract paper information** from search results:
- Paper title
- Authors
- arXiv ID
- Abstract preview
- Publication date
3. **Navigate to individual papers** for detailed review
---
## Paper Quality Evaluation
Evaluate papers using the 5-dimension criteria below:
| Dimension | Weight | Key Points |
|-----------|--------|------------|
| **Innovation** | 30% | Novelty of contribution |
| **Method Completeness** | 25% | Clarity and reproducibility |
| **Experimental Thoroughness** | 25% | Validation depth |
| **Writing Quality** | 10% | Clarity of expression |
| **Relevance & Impact** | 10% | Domain importance |
### Scoring Guidelines (1-5 scale)
**Innovation (30%):**
- 5: Breakthrough contribution, major impact
- 4: Significant improvement, new insights
- 3: Methodological innovation
- 2: Incremental improvement
- 1: Minor improvements
**Method Completeness (25%):**
- 5: Complete and rigorous, easily reproducible
- 4: Very detailed, mostly reproducible
- 3: Core method clear, basically reproducible
- 2: Lacks key details
- 1: Unclear description
**Experimental Thoroughness (25%):**
- 5: Comprehensive multi-dataset, ablation studies
- 4: Multiple datasets, reasonable ablations
- 3: Main experiments complete
- 2: Limited experiments
- 1: Minimal validation
**Writing Quality (10%):**
- 5: Excellent clarity and rigor
- 4: Clear and well-structured
- 3: Understandable
- 2: Some ambiguity
- 1: Confusing
**Relevance & Impact (10%):**
- 5: Solves important problem, wide impact
- 4: Important domain problem
- 3: Meaningful contribution
- 2: Niche problem
- 1: Limited impact
### Selection Process
1. **Screen by title/abstract** for relevance
2. **Navigate to full paper** for detailed review
3. **Score each dimension** (1-5)
4. **Calculate weighted total**
5. **Rank and select** top papers
---
## Extracting Paper Metadata
**From arXiv Abstract Page (`https://arxiv.org/abs/ARXIV_ID`):**
- Title (from `<h1>` tag)
- Authors (from `.authors` element)
- Abstract (from `blockquote.abstract`)
- Submission date (from `.dateline`)
- arXiv ID (from URL)
- Categories (from `.subjects`)
- Comments (if present)
- Code repository (check abstract for GitHub links)
---
## Integration with Citation Workflow
After finding relevant papers:
1. **Verify citations** using Semantic Scholar API (see `../citation-workflow.md`)
2. **Fetch BibTeX** programmatically via DOI
3. **Store in bibliography** with verification status
---
## Common Use Cases
### Finding Related Work
When writing a paper, use arXiv search to:
1. Find recent papers on your topic
2. Identify state-of-the-art methods
3. Discover competing approaches
4. Find baseline comparisons
### Staying Updated
Set up regular searches for:
- Your specific research area
- Competing labs/researchers
- New methods in your domain
- Conference proceedings (preprints)
### Literature Reviews
For comprehensive reviews:
1. Start with broad keyword searches
2. Filter by recent publications (last 1-3 years)
3. Use citation chaining (forward and backward)
4. Evaluate and select high-quality papers
5. Organize by theme/contribution
---
## Tips for Effective Searching
1. **Use specific keywords** rather than broad terms
2. **Combine techniques** (keywords + categories + time filters)
3. **Check code availability** (many arXiv papers link to GitHub)
4. **Look for citations** to understand impact
5. **Read abstracts carefully** before full papers
6. **Use paper metrics** (citation count, code stars) as indicators
---
## External Resources
- **arXiv**: https://arxiv.org/
- **Semantic Scholar**: https://www.semanticscholar.org/
- **Papers With Code**: https://paperswithcode.com/
- **Connected Papers**: https://www.connectedpapers.com/
- **arXiv API**: http://export.arxiv.org/api_help/

View File

@@ -0,0 +1,303 @@
# ML Paper Quality Evaluation Criteria
## Overview
Use these criteria to evaluate ML research papers found during literature search or when selecting papers for detailed review. The 5-dimension framework provides structured assessment for paper selection and comparison.
---
## Evaluation Dimensions
| Dimension | Weight | Description |
|-----------|--------|-------------|
| **Innovation** | 30% | Novelty and originality of contribution |
| **Method Completeness** | 25% | Clarity, rigor, and reproducibility |
| **Experimental Thoroughness** | 25% | Validation depth and analysis quality |
| **Writing Quality** | 10% | Clarity and presentation |
| **Relevance & Impact** | 10% | Domain importance and potential impact |
---
## Detailed Scoring Rubrics
### 1. Innovation (30%)
**Score 5 - Breakthrough:**
- Proposes entirely new paradigm or framework
- Solves long-standing open problem
- Major impact expected on the field
- Challenges fundamental assumptions
**Score 4 - Significant Innovation:**
- Substantial improvement over existing methods
- New insights or perspectives
- Novel combination of techniques
- Clear advancement over state-of-the-art
**Score 3 - Methodological Innovation:**
- New method or architecture proposed
- Some novelty but incremental
- Reasonable contribution
- Standard type of innovation
**Score 2 - Incremental Improvement:**
- Minor improvements to existing methods
- Limited novelty
- Small advancement
- Mostly derivative
**Score 1 - Trivial:**
- Minimal contribution
- Obvious extension
- No real innovation
- Known results
**Evaluation Questions:**
- Does this paper propose something genuinely new?
- Does it advance the state-of-the-art?
- Will this influence future work?
- Is the contribution significant or marginal?
---
### 2. Method Completeness (25%)
**Score 5 - Complete and Rigorous:**
- Full mathematical derivation
- All hyperparameters specified
- Complete algorithmic details
- Easily reproducible
- Code available
**Score 4 - Very Complete:**
- Detailed method description
- Most important details included
- Mostly reproducible
- Minor gaps in documentation
**Score 3 - Reproducible:**
- Core method clearly described
- Key details present
- Can be reproduced with effort
- Some ambiguity in details
**Score 2 - Lacks Details:**
- Key details missing
- Difficult to reproduce
- Incomplete description
- Ambiguous in important areas
**Score 1 - Unclear:**
- Method description unclear
- Missing critical information
- Cannot determine validity
- Poorly explained
**Evaluation Questions:**
- Can another researcher reproduce this work?
- Are all important details specified?
- Is mathematical derivation sound?
- Is code available and documented?
---
### 3. Experimental Thoroughness (25%)
**Score 5 - Comprehensive:**
- Multiple diverse datasets
- Extensive ablation studies
- Statistical significance testing
- Thorough analysis and discussion
- Comparison with strong baselines
**Score 4 - Very Thorough:**
- Multiple datasets
- Reasonable ablation studies
- Proper baseline comparisons
- Good analysis
**Score 3 - Adequate:**
- Main experiments complete
- Standard datasets
- Basic baselines
- Results are credible
**Score 2 - Limited:**
- Limited experiments
- Few datasets
- Weak baselines
- Minimal analysis
**Score 1 - Insufficient:**
- Minimal validation
- Toy examples only
- No meaningful comparisons
- Results not convincing
**Evaluation Questions:**
- Are experiments comprehensive?
- Are baselines strong and appropriate?
- Are statistical tests used?
- Is there ablation analysis?
- Are results on standard datasets?
---
### 4. Writing Quality (10%)
**Score 5 - Excellent:**
- Clear, precise, well-structured
- Logical flow throughout
- Professional presentation
- High-quality figures
- No ambiguity
**Score 4 - Very Good:**
- Clear and well-written
- Mostly logical structure
- Good presentation
- Minor issues
**Score 3 - Understandable:**
- Basically clear
- Some organizational issues
- Acceptable presentation
- Understandable with effort
**Score 2 - Fair:**
- Some confusing sections
- Organization problems
- Presentation issues
- Hard to follow at times
**Score 1 - Poor:**
- Unclear or confusing
- Poor organization
- Difficult to understand
- Major presentation problems
**Evaluation Questions:**
- Is the paper easy to understand?
- Is the structure logical?
- Are figures/tables clear?
- Is the writing professional?
---
### 5. Relevance & Impact (10%)
**Score 5 - High Impact:**
- Solves important problem
- Broad applicability
- Expected wide influence
- Addresses fundamental challenge
**Score 4 - Domain Important:**
- Important problem in field
- Significant potential impact
- Relevant to many researchers
**Score 3 - Meaningful:**
- Meaningful contribution
- Moderate impact expected
- Relevant to subset of field
**Score 2 - Niche:**
- Specialized problem
- Limited applicability
- Narrow impact
**Score 1 - Limited:**
- Very narrow problem
- Minimal impact expected
- Limited relevance
**Evaluation Questions:**
- Is this an important problem?
- Will this influence future work?
- Is it relevant to current research needs?
- Does it address a significant challenge?
---
## Scoring Calculation
**Weighted Total:**
```
Total = (Innovation × 0.30) + (Method × 0.25) + (Experiments × 0.25) + (Writing × 0.10) + (Impact × 0.10)
```
**Example Calculation:**
- Innovation: 4/5
- Method: 3/5
- Experiments: 4/5
- Writing: 3/5
- Impact: 4/5
```
Total = (4 × 0.30) + (3 × 0.25) + (4 × 0.25) + (3 × 0.10) + (4 × 0.10)
= 1.20 + 0.75 + 1.00 + 0.30 + 0.40
= 3.65 / 5.0
```
---
## Selection Process
### For Literature Reviews
1. **Screen papers** by title/abstract for relevance
2. **Full review** of potentially relevant papers
3. **Score each paper** using all 5 dimensions
4. **Rank by total score**
5. **Select top papers** for detailed review
### Quality Thresholds
- **Excellent**: 4.0+ (include definitely)
- **Good**: 3.5-3.9 (include if relevant)
- **Fair**: 3.0-3.4 (include if highly relevant)
- **Poor**: <3.0 (exclude unless essential)
---
## Quick Screening Indicators
Before detailed review, check:
**Positive Indicators:**
- Published at top venue (NeurIPS, ICML, ICLR)
- Citations in top papers
- Code available with stars
- Authors from top labs
- Clear novelty in abstract
**Negative Indicators:**
- Vague abstract
- Limited experiments mentioned
- No baselines mentioned
- Poor writing in abstract
- incremental claims only
---
## Integration with Paper Discovery
When using arXiv search (`arxiv-search-guide.md`):
1. **Search** for relevant papers
2. **Extract metadata** from arXiv pages
3. **Quick screen** by abstract/relevance
4. **Detailed review** of promising papers
5. **Score using** these criteria
6. **Rank and select** top candidates
---
## Notes
- These criteria are designed for ML papers specifically
- Adjust weights based on your specific needs
- Use scores as relative comparisons, not absolute judgments
- Consider venue reputation as additional signal
- Code availability is increasingly important for reproducibility