backup materials and knowledge-base docs
This commit is contained in:
admin
@@ -0,0 +1,47 @@
|
||||
# Chart Selection
|
||||
|
||||
## Task-to-chart mapping
|
||||
|
||||
| Communication task | Preferred forms | Notes |
|
||||
|---|---|---|
|
||||
| Benchmark comparison | grouped scatter, bar, companion table | use table when exact values matter most |
|
||||
| Ablation | grouped comparison, dumbbell, compact table | keep the dimension count small |
|
||||
| Calibration / evaluation | calibration, ROC, PR, Bland–Altman | choose what matches the evaluation claim |
|
||||
| Distribution | box, violin, raincloud, histogram, density, ECDF, QQ | choose by whether shape or exact quantiles matter |
|
||||
| Relationship | scatter, bubble, contour2d, hexbin | use hexbin/contour2d when overplotting is severe |
|
||||
| Trend | line, area | line is usually the safer default |
|
||||
| Diagnostic effect size | forest plot, volcano | match domain and inference style |
|
||||
| Set/composition | UpSet, stacked ratio, donut, radial hierarchy | avoid decorative complexity unless it helps interpretation |
|
||||
| Exact benchmark appendix | publication table | default to `pubtab` |
|
||||
|
||||
## Use X instead of Y
|
||||
|
||||
- Use **grouped scatter** or a **table** instead of a dense grouped bar when exact per-group values matter.
|
||||
- Use **line** instead of bar for ordered progression over time or scale.
|
||||
- Use **UpSet** instead of Venn-style thinking once the set count grows.
|
||||
- Use **forest plot** instead of overloaded textual effect summaries.
|
||||
- Use **table** instead of radar when precision and comparability matter more than shape.
|
||||
- Use **hexbin** or **contour2d** instead of raw scatter when point overlap hides structure.
|
||||
- Use **ECDF** when comparing cumulative distributions clearly is more important than showing a smoothed KDE.
|
||||
|
||||
## Anti-patterns
|
||||
|
||||
Avoid:
|
||||
|
||||
- pie/donut for exact quantitative comparison unless the composition story is primary and category count is small,
|
||||
- radar for many categories or when axes are not semantically comparable,
|
||||
- 3D effects,
|
||||
- decorative color ramps without semantic purpose,
|
||||
- overly dense legends that repeat axis information,
|
||||
- mixed chart types that make the evidence harder to read,
|
||||
- turning every result into a figure when a publication table would be cleaner.
|
||||
|
||||
## Selection heuristic
|
||||
|
||||
Ask in order:
|
||||
|
||||
1. What claim is the reader supposed to take away?
|
||||
2. Does the reader need pattern perception or exact value lookup?
|
||||
3. Are the groups ordered, categorical, repeated, hierarchical, or overlapping?
|
||||
4. Is the result single-panel or likely part of a multi-panel figure?
|
||||
5. Would a figure-only answer hide important exact values that should live in a table?
|
||||
@@ -0,0 +1,34 @@
|
||||
# Composite Assembly
|
||||
|
||||
## Principle
|
||||
|
||||
Composite assembly is a **secondary branch**, not the default workflow.
|
||||
|
||||
Use it when:
|
||||
|
||||
- the user explicitly wants a multi-panel paper figure,
|
||||
- panel-level maintenance matters,
|
||||
- or the final paper figure needs finishing beyond a single exported plot.
|
||||
|
||||
## Default stance
|
||||
|
||||
- Single-panel figure → stay in `pubfig` normal export mode.
|
||||
- Multi-panel figure with real assembly needs → use panel/composite export.
|
||||
- Figma is optional and should not be introduced unless it solves a real assembly problem.
|
||||
|
||||
## pubfig routes
|
||||
|
||||
Relevant `pubfig` capabilities:
|
||||
|
||||
- `export_panel(...)`
|
||||
- `export_panels(...)`
|
||||
- `batch_export(...)`
|
||||
- `save_figure(...)`
|
||||
|
||||
If the environment already uses a pubfig/Figma bridge workflow, keep `figure_id` stable across revisions.
|
||||
|
||||
## Practical rule
|
||||
|
||||
Escalate to composite assembly only after the panel content itself is strong.
|
||||
|
||||
Do not use Figma/composite assembly to hide weak chart choice, poor labels, or overloaded panels.
|
||||
@@ -0,0 +1,159 @@
|
||||
# Execution and Verification
|
||||
|
||||
## Goal
|
||||
|
||||
Turn a high-level publication figure/table request into a route that is actually runnable in the current environment.
|
||||
|
||||
## Minimum environment probe
|
||||
|
||||
Prefer the lightest useful checks.
|
||||
|
||||
### Preferred bundled probe
|
||||
|
||||
```bash
|
||||
python3 scripts/ensure_publication_tooling.py --require pubfig --json
|
||||
python3 scripts/ensure_publication_tooling.py --require pubtab --json
|
||||
```
|
||||
|
||||
The helper probes availability, force-installs missing dependencies into the active interpreter, and returns the post-install status.
|
||||
|
||||
### Equivalent manual checks
|
||||
|
||||
```bash
|
||||
python -c "import pubfig; print(pubfig.__version__)"
|
||||
python -c "import pubtab; print(pubtab.__version__)"
|
||||
pubtab --help
|
||||
```
|
||||
|
||||
Do not spend the whole turn on setup if the user primarily needs design guidance. Just identify whether the route is executable now or should degrade gracefully.
|
||||
|
||||
## Automatic installation policy
|
||||
|
||||
If a dependency is missing and the task requires real execution, install it automatically before continuing.
|
||||
|
||||
### Preferred bundled route
|
||||
|
||||
Use the bundled helper when it is present:
|
||||
|
||||
```bash
|
||||
python3 scripts/ensure_publication_tooling.py --require pubfig
|
||||
python3 scripts/ensure_publication_tooling.py --require pubtab
|
||||
```
|
||||
|
||||
The helper chooses `uv pip install --python <active-python>` when the project is clearly `uv`-managed, and otherwise falls back to `python -m pip install ...`.
|
||||
|
||||
### Equivalent manual install commands
|
||||
|
||||
```bash
|
||||
uv pip install --python "$VIRTUAL_ENV/bin/python" pubfig
|
||||
uv pip install --python "$VIRTUAL_ENV/bin/python" pubtab
|
||||
python -m pip install pubfig
|
||||
python -m pip install pubtab
|
||||
```
|
||||
|
||||
### Required follow-up
|
||||
|
||||
After installation:
|
||||
|
||||
1. re-run the availability probe,
|
||||
2. report the updated environment status,
|
||||
3. continue with the runnable figure/table workflow.
|
||||
|
||||
If installation fails, capture the exact error and then fall back to design/specification guidance.
|
||||
|
||||
## Route selection
|
||||
|
||||
### Use `pubfig` when
|
||||
|
||||
- the task is primarily a figure,
|
||||
- the user already has Python data structures,
|
||||
- the result is a plot family already covered by `pubfig`,
|
||||
- export quality matters immediately.
|
||||
|
||||
### Use `pubtab` when
|
||||
|
||||
- the task is primarily a publication table,
|
||||
- the input is an Excel workbook, a `.tex` table, or a file-driven workflow,
|
||||
- the reader needs exact values,
|
||||
- previewing the table before manuscript insertion matters.
|
||||
|
||||
### Use both when
|
||||
|
||||
- the figure carries the visual pattern,
|
||||
- the table preserves exact benchmark values,
|
||||
- the paper section benefits from one fast visual plus one exact-value artifact.
|
||||
|
||||
## First runnable verification
|
||||
|
||||
### `pubfig`
|
||||
|
||||
After generating a minimal figure route, the first useful verification is:
|
||||
|
||||
- can the code execute,
|
||||
- does `save_figure(...)` or `batch_export(...)` produce the expected files,
|
||||
- do output suffixes match the intended formats.
|
||||
|
||||
### `pubtab`
|
||||
|
||||
After generating a minimal table route, the first useful verification is:
|
||||
|
||||
- can `xlsx2tex` or `tex2xlsx` run,
|
||||
- can `preview` render PNG or PDF,
|
||||
- does the chosen backend (`tabular` or `tabularray`) match the manuscript need.
|
||||
|
||||
## Current practical notes
|
||||
|
||||
### `pubfig`
|
||||
|
||||
Useful export primitives include:
|
||||
|
||||
- `save_figure(...)`
|
||||
- `batch_export(...)`
|
||||
- `export_panel(...)`
|
||||
- `export_panels(...)`
|
||||
|
||||
Use panel export only when multi-panel assembly is truly needed.
|
||||
|
||||
### `pubtab`
|
||||
|
||||
Useful file-oriented routes include:
|
||||
|
||||
- `pubtab xlsx2tex ...`
|
||||
- `pubtab tex2xlsx ...`
|
||||
- `pubtab preview ...`
|
||||
|
||||
Remember:
|
||||
|
||||
- `xlsx2tex` exports all sheets by default when `--sheet` is not set,
|
||||
- `preview` can render PNG or PDF,
|
||||
- `--latex-backend tabularray` should be chosen only when the manuscript/backend requires `tblr`,
|
||||
- when preview reliability is the immediate priority, validate the table body first and add final `caption` / `label` in a separate manuscript-facing step if needed.
|
||||
|
||||
## Graceful degradation
|
||||
|
||||
If the tool is missing:
|
||||
|
||||
- first try the bundled auto-install helper,
|
||||
- if that route is unavailable, use the manual install commands above,
|
||||
- if installation still fails, provide:
|
||||
- the artifact recommendation,
|
||||
- the exact files the user should prepare,
|
||||
- a draft CLI or Python route,
|
||||
- the export targets,
|
||||
- and the publication QA checklist.
|
||||
|
||||
## Default output wording
|
||||
|
||||
When the route is runnable now, say:
|
||||
|
||||
- what to run,
|
||||
- what files should appear,
|
||||
- what to inspect next.
|
||||
|
||||
When the route is not runnable now, say:
|
||||
|
||||
- what is missing,
|
||||
- which helper command or install command was attempted,
|
||||
- whether the install succeeded or failed,
|
||||
- what the intended route will be after install,
|
||||
- and what design decision can already be locked in today.
|
||||
@@ -0,0 +1,104 @@
|
||||
# pubfig Recipes
|
||||
|
||||
`pubfig` is the default engine for scientific figures.
|
||||
|
||||
## Core route
|
||||
|
||||
Typical minimal workflow:
|
||||
|
||||
```python
|
||||
import pubfig as pf
|
||||
|
||||
fig = pf.line(data, x=x, series_names=["A", "B"])
|
||||
pf.save_figure(fig, "figure1.pdf")
|
||||
```
|
||||
|
||||
## Common figure families
|
||||
|
||||
| Need | Preferred `pubfig` functions |
|
||||
|---|---|
|
||||
| benchmark comparison | `bar_scatter`, `grouped_scatter`, `bar`, `line` |
|
||||
| ablation | `bar_scatter`, `dumbbell`, `paired`, `bar` |
|
||||
| distribution | `box`, `violin`, `raincloud`, `histogram`, `density`, `ecdf`, `qq` |
|
||||
| relationship | `scatter`, `bubble`, `contour2d`, `hexbin` |
|
||||
| trend | `line`, `area` |
|
||||
| diagnostic / evaluation | `calibration`, `forest_plot`, `bland_altman`, `volcano`, `roc`, `pr_curve` |
|
||||
| composition / hierarchy | `donut`, `upset`, `radial_hierarchy`, `circular_grouped_bar`, `circular_stacked_bar`, `stacked_ratio_barh` |
|
||||
| matrix / map | `heatmap`, `corr_matrix`, `clustermap` |
|
||||
|
||||
## Export defaults
|
||||
|
||||
For a normal first pass:
|
||||
|
||||
```python
|
||||
pf.save_figure(fig, "figure1.pdf")
|
||||
```
|
||||
|
||||
For multiple formats:
|
||||
|
||||
```python
|
||||
pf.batch_export(
|
||||
fig,
|
||||
"figure1",
|
||||
formats=("pdf", "svg", "png"),
|
||||
spec="nature",
|
||||
width="single",
|
||||
dpi=300,
|
||||
)
|
||||
```
|
||||
|
||||
## When to add export parameters
|
||||
|
||||
Only add more export controls when the task demands them:
|
||||
|
||||
- `spec` / `width` for venue-style export
|
||||
- explicit SVG for vector-first downstream editing
|
||||
- PNG for quick review or raster deliverables
|
||||
- panel export when the user truly needs composite assembly
|
||||
- `batch_export(...)` when the same figure needs several publication-style outputs
|
||||
|
||||
## Panel export branch
|
||||
|
||||
Use these only when multi-panel assembly is genuinely needed:
|
||||
|
||||
- `export_panel(...)`
|
||||
- `export_panels(...)`
|
||||
|
||||
Do not default to panel export for single figures.
|
||||
|
||||
## Minimal recipe patterns
|
||||
|
||||
### Benchmark comparison
|
||||
|
||||
```python
|
||||
fig = pf.grouped_scatter(values, category_names=category_names, group_names=model_names)
|
||||
pf.save_figure(fig, "benchmark.pdf")
|
||||
```
|
||||
|
||||
### Ablation
|
||||
|
||||
```python
|
||||
fig = pf.dumbbell(baseline, improved, category_names=labels)
|
||||
pf.save_figure(fig, "ablation.pdf")
|
||||
```
|
||||
|
||||
### Calibration
|
||||
|
||||
```python
|
||||
fig = pf.calibration(prob_true, prob_pred)
|
||||
pf.save_figure(fig, "calibration.pdf")
|
||||
```
|
||||
|
||||
### Forest plot
|
||||
|
||||
```python
|
||||
fig = pf.forest_plot(effect, lower, upper, labels=labels, reference=1.0)
|
||||
pf.save_figure(fig, "forest.pdf")
|
||||
```
|
||||
|
||||
### Heatmap
|
||||
|
||||
```python
|
||||
fig = pf.heatmap(matrix)
|
||||
pf.save_figure(fig, "heatmap.pdf")
|
||||
```
|
||||
@@ -0,0 +1,47 @@
|
||||
# Publication QA Checklist
|
||||
|
||||
## Figure QA
|
||||
|
||||
Check:
|
||||
|
||||
- Is the chart family appropriate for the scientific claim?
|
||||
- Are axes labeled clearly, with units when needed?
|
||||
- Is category ordering meaningful and stable?
|
||||
- Are color choices readable in grayscale or for color-deficient readers?
|
||||
- Are fonts, line weights, and marker sizes internally consistent?
|
||||
- Is the legend necessary, and if so, is it compact?
|
||||
- Does the figure stay readable after downscaling to likely paper width?
|
||||
- If multi-panel, are labels, spacing, and styling consistent across panels?
|
||||
- Does the caption need exact numbers that should instead live in a table?
|
||||
|
||||
## Table QA
|
||||
|
||||
Check:
|
||||
|
||||
- Is a table actually the right artifact, or should a companion figure exist?
|
||||
- Are row/column labels short but unambiguous?
|
||||
- Are important values easy to locate?
|
||||
- Is the caption specific enough to stand alone?
|
||||
- Is the table too dense for the main paper and better suited to appendix?
|
||||
- If two-column format matters, was width/span considered explicitly?
|
||||
- Was a preview rendered before treating the table as final?
|
||||
|
||||
## Mixed deliverable QA
|
||||
|
||||
If both a figure and table are present:
|
||||
|
||||
- Does each have a distinct role?
|
||||
- Is the figure carrying pattern perception?
|
||||
- Is the table preserving exact values?
|
||||
- Is there duplication that should be reduced?
|
||||
|
||||
## Revision questions
|
||||
|
||||
If the artifact is weak, ask:
|
||||
|
||||
- Should the chart family change?
|
||||
- Should exact values move into a table?
|
||||
- Should the artifact split into two panels?
|
||||
- Should labels or legends be simplified?
|
||||
- Should export width/format change?
|
||||
- Should the artifact move from figure-first to table-first, or the reverse?
|
||||
@@ -0,0 +1,133 @@
|
||||
# pubtab Recipes
|
||||
|
||||
`pubtab` is the default engine for publication-ready tables.
|
||||
|
||||
## Core routes
|
||||
|
||||
### Excel to LaTeX
|
||||
|
||||
```bash
|
||||
pubtab xlsx2tex results.xlsx -o results.tex
|
||||
```
|
||||
|
||||
### LaTeX to Excel
|
||||
|
||||
```bash
|
||||
pubtab tex2xlsx tables.tex -o tables.xlsx
|
||||
```
|
||||
|
||||
### Preview
|
||||
|
||||
```bash
|
||||
pubtab preview results.tex -o results.png --dpi 300
|
||||
pubtab preview results.tex --format pdf -o results.pdf
|
||||
```
|
||||
|
||||
## Python route
|
||||
|
||||
```python
|
||||
import pubtab
|
||||
|
||||
pubtab.xlsx2tex("results.xlsx", output="results.tex", theme="three_line")
|
||||
pubtab.preview("results.tex", output="results.png", dpi=300)
|
||||
```
|
||||
|
||||
## Route selection rule
|
||||
|
||||
Prefer the **CLI** when:
|
||||
|
||||
- the user already speaks in files,
|
||||
- the source is Excel or `.tex`,
|
||||
- the main need is export and preview.
|
||||
|
||||
Prefer the **Python API** when:
|
||||
|
||||
- the task already lives inside a notebook or script,
|
||||
- the table generation is part of a larger reproducible pipeline.
|
||||
|
||||
## Current practical notes
|
||||
|
||||
- `xlsx2tex` exports **all sheets by default** when `--sheet` is not set.
|
||||
- `preview` can render **PNG or PDF**.
|
||||
- `--latex-backend tabularray` is useful only when the manuscript/backend really requires `tblr`.
|
||||
- `preview` can auto-detect `tblr`, but explicit backend override is still fine when needed.
|
||||
- for a robust preview-first workflow, preview the table body first and add the final `caption` / `label` in the manuscript or in a final non-preview export step when needed.
|
||||
|
||||
## When to use `tabularray`
|
||||
|
||||
Use `--latex-backend tabularray` when:
|
||||
|
||||
- the user explicitly wants `tblr`,
|
||||
- the manuscript already uses `tabularray`,
|
||||
- or the backend must match an existing paper template.
|
||||
|
||||
Example:
|
||||
|
||||
```bash
|
||||
pubtab xlsx2tex results.xlsx -o results_tblr.tex --theme three_line --latex-backend tabularray
|
||||
```
|
||||
|
||||
## Common publication controls
|
||||
|
||||
Use these when they are justified:
|
||||
|
||||
- `--caption`
|
||||
- `--label`
|
||||
- `--span-columns`
|
||||
- `--preview`
|
||||
- `--latex-backend`
|
||||
- `--sheet`
|
||||
- `--with-resizebox`
|
||||
- `--without-resizebox`
|
||||
- `--resizebox-width`
|
||||
|
||||
## Default guidance
|
||||
|
||||
- start with the smallest `xlsx2tex` route,
|
||||
- preview before treating the table as final,
|
||||
- use a publication table when exact values matter more than quick pattern perception,
|
||||
- keep figure and table roles distinct in mixed deliverables.
|
||||
|
||||
## Minimal recipe patterns
|
||||
|
||||
### Benchmark table from Excel
|
||||
|
||||
```bash
|
||||
pubtab xlsx2tex benchmark.xlsx -o benchmark.tex --caption "Main benchmark results." --label "tab:benchmark"
|
||||
```
|
||||
|
||||
### Two-column table
|
||||
|
||||
```bash
|
||||
pubtab xlsx2tex benchmark.xlsx -o benchmark.tex --span-columns
|
||||
```
|
||||
|
||||
### Preview before submission
|
||||
|
||||
```bash
|
||||
pubtab xlsx2tex benchmark.xlsx -o benchmark_preview.tex
|
||||
pubtab preview benchmark_preview.tex -o benchmark.png --dpi 300
|
||||
```
|
||||
|
||||
Use this route when the immediate goal is a reliable visual check of the table body.
|
||||
Keep `caption` / `label` as a separate manuscript-facing step if the preview is the main verification target.
|
||||
|
||||
### Final manuscript-facing export
|
||||
|
||||
```bash
|
||||
pubtab xlsx2tex benchmark.xlsx -o benchmark.tex --caption "Main benchmark results." --label "tab:benchmark"
|
||||
```
|
||||
|
||||
### All-sheets export
|
||||
|
||||
```bash
|
||||
pubtab xlsx2tex benchmark.xlsx -o out/benchmark.tex
|
||||
```
|
||||
|
||||
### Native file-pipeline batch roundtrip
|
||||
|
||||
```bash
|
||||
pubtab tex2xlsx ./tables_tex -o ./out/xlsx
|
||||
pubtab xlsx2tex ./out/xlsx -o ./out/tex
|
||||
pubtab preview ./out/tex -o ./out/png --format png --dpi 300
|
||||
```
|
||||
@@ -0,0 +1,199 @@
|
||||
# pubfig API map (source-driven)
|
||||
|
||||
This guide maps the public `pubfig` API from `pubfig/src/pubfig/__init__.py` to the underlying modules.
|
||||
|
||||
## 1. Stable entrypoint
|
||||
|
||||
The public contract is defined by re-exports in `pubfig.__init__`.
|
||||
|
||||
For agents, this means:
|
||||
|
||||
- if a symbol is re-exported there, it is a good default public entrypoint;
|
||||
- if a helper only exists in deep internal modules, treat it as implementation detail unless there is a strong reason not to.
|
||||
|
||||
## 2. Public API groups
|
||||
|
||||
### Themes
|
||||
|
||||
Public re-exports:
|
||||
|
||||
- `set_default_theme`
|
||||
- `get_default_theme`
|
||||
- `get_theme`
|
||||
- `register_theme`
|
||||
|
||||
Use these when the task is really about reusable visual policy.
|
||||
Do not hardcode theme assumptions if a registry call is more appropriate.
|
||||
|
||||
### Colors and palettes
|
||||
|
||||
Public re-exports include:
|
||||
|
||||
- `DEFAULT`, `NATURE`, `SCIENCE`, `LANCET`, `JAMA`
|
||||
- `get_palette`
|
||||
- `register_palette`
|
||||
- `color_to_rgba`
|
||||
- `darken_color`
|
||||
- `show_palette`
|
||||
|
||||
Source fact:
|
||||
|
||||
- palette registration and palette inspection are exposed from the package root.
|
||||
|
||||
Operational implication:
|
||||
|
||||
- treat palette selection and palette registration as public API usage, not as deep internal customization.
|
||||
|
||||
### Export
|
||||
|
||||
Public re-exports:
|
||||
|
||||
- `save_figure`
|
||||
- `batch_export`
|
||||
- `PanelExportRecord`
|
||||
- `export_panel`
|
||||
- `export_panels`
|
||||
- `package_figma_bundle`
|
||||
- `validate_figma_bundle`
|
||||
- `inspect_figma_bundle`
|
||||
|
||||
Skill implication:
|
||||
|
||||
- normal paper figures should usually stop at `save_figure` or `batch_export`
|
||||
- panel workflows should use `export_panel(s)`
|
||||
- bundle helpers are for bridge/Figma handoff, not the default answer
|
||||
|
||||
### Figure specs
|
||||
|
||||
Public re-exports:
|
||||
|
||||
- `FigureSpec`
|
||||
- `get_figure_spec`
|
||||
- `register_figure_spec`
|
||||
- `list_figure_specs`
|
||||
|
||||
Use this layer whenever the user asks for venue-aware width, journal defaults, or a custom export profile.
|
||||
|
||||
### Plot families
|
||||
|
||||
The public plot surface is broad, but the source still clusters naturally.
|
||||
|
||||
#### Comparison / summary figures
|
||||
|
||||
Representative public calls:
|
||||
|
||||
- `bar`
|
||||
- `bar_scatter`
|
||||
- `stacked_bar`
|
||||
- `stacked_ratio_barh`
|
||||
- `donut`
|
||||
- `dumbbell`
|
||||
- `forest_plot`
|
||||
- `grouped_scatter`
|
||||
- `upset`
|
||||
|
||||
Use these for benchmark, ablation, summary, composition, and set-overlap tasks.
|
||||
|
||||
#### Distribution figures
|
||||
|
||||
Representative public calls:
|
||||
|
||||
- `ecdf`
|
||||
- `qq`
|
||||
- `box`
|
||||
- `density`
|
||||
- `hexbin`
|
||||
- `histogram`
|
||||
- `raincloud`
|
||||
- `strip`
|
||||
- `ridgeline`
|
||||
- `violin`
|
||||
|
||||
Use these when the scientific claim is about spread, calibration of assumptions, or cohort structure.
|
||||
|
||||
#### Trend / profile figures
|
||||
|
||||
Representative public calls:
|
||||
|
||||
- `line`
|
||||
- `area`
|
||||
- `parallel_coordinates`
|
||||
- `radar`
|
||||
- `radial_hierarchy`
|
||||
- `circular_stacked_bar`
|
||||
- `circular_grouped_bar`
|
||||
|
||||
Not all of these are equally strong for publication use. The skill should still apply chart-selection discipline before calling them.
|
||||
|
||||
#### Relationship / embedding figures
|
||||
|
||||
Representative public calls:
|
||||
|
||||
- `scatter`
|
||||
- `volcano`
|
||||
- `bubble`
|
||||
- `contour2d`
|
||||
- `paired`
|
||||
- `heatmap`
|
||||
- `corr_matrix`
|
||||
- `clustermap`
|
||||
- `dimreduce`
|
||||
- `pca_biplot`
|
||||
|
||||
Use these for association, error structure, feature layout, and representation views.
|
||||
|
||||
#### Evaluation figures
|
||||
|
||||
Representative public calls:
|
||||
|
||||
- `roc`
|
||||
- `pr_curve`
|
||||
- `calibration`
|
||||
- `bland_altman`
|
||||
|
||||
This cluster matters because the source gives them dedicated implementation in `plots/evaluation.py`, which is a sign that evaluation charts are a first-class use case.
|
||||
|
||||
## 3. Return-value contract
|
||||
|
||||
The source in `export/io.py` makes a subtle but important contract explicit:
|
||||
|
||||
- export functions accept a Matplotlib `Figure`,
|
||||
- or an `Axes`,
|
||||
- or any object exposing a `.figure` attribute that resolves to a `Figure`.
|
||||
|
||||
Source fact:
|
||||
|
||||
- the export layer accepts standard Matplotlib figure objects or figure-bearing wrappers.
|
||||
|
||||
Operational implication:
|
||||
|
||||
- keep the `Figure` handle available and route export through the standard Matplotlib-facing export path.
|
||||
|
||||
For the skill, the safest phrasing is:
|
||||
|
||||
- create the figure,
|
||||
- keep a handle to the `Figure`,
|
||||
- then export explicitly.
|
||||
|
||||
## 4. What is not the main stable plotting interface
|
||||
|
||||
From the source tree, the CLI is not where normal chart creation happens.
|
||||
It is mostly an operational layer for Figma bundle and bridge actions.
|
||||
|
||||
So if a user says “generate a paper-ready figure,” the skill should not default to a CLI answer.
|
||||
|
||||
## 5. Safe public usage pattern
|
||||
|
||||
The most source-faithful pattern is:
|
||||
|
||||
1. choose a public plot function from `pubfig`
|
||||
2. generate a `Figure`
|
||||
3. export via `save_figure(...)` or `batch_export(...)`
|
||||
4. only use panel/bundle helpers when composition is actually required
|
||||
|
||||
## 6. Source-guided caution points
|
||||
|
||||
- Do not mix up plot-time design sizing with export-time publication sizing.
|
||||
- Do not use `save_figure(...)` as a multi-format exporter; the source now pushes that role to `batch_export(...)`.
|
||||
- Do not route ordinary figure-generation tasks through the Figma bridge CLI.
|
||||
- Do not assume all public plot families are equally appropriate; the skill must still filter by scientific communication quality.
|
||||
@@ -0,0 +1,190 @@
|
||||
# pubfig architecture (source-driven)
|
||||
|
||||
This guide reads `pubfig` from the source tree rather than from high-level overview material.
|
||||
|
||||
Source root:
|
||||
|
||||
- `pubfig/src/pubfig/__init__.py`
|
||||
- `pubfig/src/pubfig/plots/`
|
||||
- `pubfig/src/pubfig/export/`
|
||||
- `pubfig/src/pubfig/specs.py`
|
||||
- `pubfig/src/pubfig/themes/`
|
||||
- `pubfig/src/pubfig/colors/`
|
||||
- `pubfig/src/pubfig/cli.py`
|
||||
|
||||
## 1. Start at `pubfig.__init__`
|
||||
|
||||
The stable user-facing surface is re-exported from `__init__.py`.
|
||||
|
||||
That file tells you the package is organized into five main layers:
|
||||
|
||||
1. **plot constructors** from `plots/`
|
||||
2. **export helpers** from `export/`
|
||||
3. **publication sizing** from `specs.py`
|
||||
4. **theme and palette registries** from `themes/` and `colors/`
|
||||
5. **Figma/bridge helpers** and related CLI support
|
||||
|
||||
For skill design, this is the most important architectural fact:
|
||||
|
||||
- figure generation lives in `plots/`
|
||||
- figure export lives in `export/`
|
||||
- venue sizing lives in `specs.py`
|
||||
- multi-panel/Figma handoff is a separate downstream layer
|
||||
|
||||
The default mental model is **plot first, export second, compose third**.
|
||||
|
||||
## 2. Package boundaries
|
||||
|
||||
### `plots/`
|
||||
|
||||
This is the core figure-construction layer.
|
||||
|
||||
Representative files:
|
||||
|
||||
- `plots/line.py`
|
||||
- `plots/comparison.py`
|
||||
- `plots/evaluation.py`
|
||||
- `plots/_grouped_scatter.py`
|
||||
|
||||
From the source, plot functions usually do the same sequence:
|
||||
|
||||
1. normalize/coerce input data,
|
||||
2. enter `theme_context(theme)`,
|
||||
3. resolve design-time size via `resolve_design_dpi(...)`,
|
||||
4. allocate figure/axes via `get_fig_ax(...)`,
|
||||
5. style axes/legends through helpers in `_style.py`,
|
||||
6. return a Matplotlib `Figure`.
|
||||
|
||||
Interpretation from source:
|
||||
|
||||
- `pubfig` behaves as a **Matplotlib-first figure factory layer**, not as a separate scene-graph runtime.
|
||||
|
||||
### `export/`
|
||||
|
||||
This is intentionally separated from plotting.
|
||||
|
||||
Important files:
|
||||
|
||||
- `export/io.py`
|
||||
- `export/panels.py`
|
||||
|
||||
`export/io.py` handles normal figure export:
|
||||
|
||||
- coerce Figure/Axes into a real `Figure`
|
||||
- enforce explicit suffixes
|
||||
- apply publication width/height rules
|
||||
- write vector or raster output
|
||||
|
||||
Current source implication:
|
||||
|
||||
- `batch_export(...)` now belongs to the same publication-aware export layer, rather than to a simple multi-format `savefig` wrapper.
|
||||
|
||||
`export/panels.py` handles panel-level export for composite or Figma-oriented workflows:
|
||||
|
||||
- one panel at a time or many panels together
|
||||
- optional publication-aware sizing
|
||||
- optional title stripping
|
||||
- metadata index generation (`panel-index.json`)
|
||||
|
||||
### `specs.py`
|
||||
|
||||
This file is the publication-sizing contract.
|
||||
|
||||
`FigureSpec` defines:
|
||||
|
||||
- `font_family`
|
||||
- `design_dpi`
|
||||
- `single_column_mm`
|
||||
- `double_column_mm`
|
||||
- `default_raster_dpi`
|
||||
- `background_color`
|
||||
|
||||
Built-in registry entries include:
|
||||
|
||||
- `nature`
|
||||
- `science`
|
||||
- `cell`
|
||||
|
||||
The source shows a strong split between:
|
||||
|
||||
- **design size** used when constructing interactive figures,
|
||||
- **physical export size** used when saving publication figures.
|
||||
|
||||
That split is why the skill should not treat `width` in plot calls and `width` in export calls as the same semantic layer.
|
||||
|
||||
### `themes/` and `colors/`
|
||||
|
||||
These are registries, not plain constants.
|
||||
|
||||
From `__init__.py`, the public surface includes:
|
||||
|
||||
- `get_theme`, `register_theme`, `set_default_theme`
|
||||
- `get_palette`, `register_palette`, `show_palette`
|
||||
|
||||
Operational implication:
|
||||
|
||||
- treat theme and palette selection as first-class API configuration rather than as hardcoded styling trivia.
|
||||
|
||||
### `cli.py`
|
||||
|
||||
The current CLI is not the main figure-generation interface.
|
||||
|
||||
From the source, `cli.py` is mainly about:
|
||||
|
||||
- Figma bridge serving
|
||||
- bundle packaging/inspection/validation
|
||||
- sync job submission and waiting
|
||||
- local bridge auto-start logic
|
||||
|
||||
So for this skill:
|
||||
|
||||
- **Python API is the primary route for figure generation**
|
||||
- `pubfig.cli` is a secondary operational layer for bridge/Figma workflows
|
||||
|
||||
## 3. Plotting architecture pattern
|
||||
|
||||
From `line.py`, `comparison.py`, and `evaluation.py`, the recurring internal pattern is:
|
||||
|
||||
- input normalization is local to the chart family,
|
||||
- shared visual behavior is delegated to internal helpers,
|
||||
- returned artifact is still a standard Matplotlib figure.
|
||||
|
||||
This is why the skill should map requests to a chart family first, instead of jumping directly to export or panel assembly.
|
||||
|
||||
Examples from source:
|
||||
|
||||
- `line.py` groups time/trend style plots
|
||||
- `comparison.py` groups comparison-style statistical displays like `dumbbell` and `forest_plot`
|
||||
- `evaluation.py` groups metric/evaluation plots like `roc`, `pr_curve`, and `calibration`
|
||||
- `_grouped_scatter.py` contains the more specialized placement/jitter/annotation logic behind grouped scatter layouts
|
||||
|
||||
## 4. Export architecture pattern
|
||||
|
||||
From `export/io.py` and `export/panels.py`, `pubfig` uses three distinct output modes:
|
||||
|
||||
1. **single explicit artifact** via `save_figure(...)`
|
||||
2. **publication-aware multi-format artifact set** via `batch_export(...)`
|
||||
3. **panel bundle workflow** via `export_panel(...)` / `export_panels(...)`
|
||||
|
||||
Those are different contracts, and the skill should keep them separate in its recommendations.
|
||||
|
||||
## 5. Reading order for deep debugging
|
||||
|
||||
When a skill or agent needs source-level certainty, use this order:
|
||||
|
||||
1. `pubfig/src/pubfig/__init__.py`
|
||||
2. relevant chart-family module in `plots/`
|
||||
3. `pubfig/src/pubfig/specs.py`
|
||||
4. `pubfig/src/pubfig/export/io.py`
|
||||
5. `pubfig/src/pubfig/export/panels.py`
|
||||
6. `pubfig/src/pubfig/cli.py` only if the task involves bridge/Figma sync
|
||||
|
||||
## 6. Implications for this skill
|
||||
|
||||
This source layout implies the skill should:
|
||||
|
||||
- default to **Python plot API + explicit export call**,
|
||||
- treat publication sizing as an export concern,
|
||||
- treat panel/Figma work as optional downstream composition,
|
||||
- avoid presenting the CLI as the main path for ordinary figure generation,
|
||||
- keep chart selection logically ahead of export tuning.
|
||||
@@ -0,0 +1,210 @@
|
||||
# pubfig export flow (source-driven)
|
||||
|
||||
This guide explains how `pubfig` moves from a generated Matplotlib figure to paper-ready files.
|
||||
|
||||
Primary source files:
|
||||
|
||||
- `pubfig/src/pubfig/export/io.py`
|
||||
- `pubfig/src/pubfig/export/panels.py`
|
||||
- `pubfig/src/pubfig/specs.py`
|
||||
|
||||
## 1. Core export contract
|
||||
|
||||
`export/io.py` separates two concerns:
|
||||
|
||||
- **coercing a valid figure object**
|
||||
- **writing explicit output files**
|
||||
|
||||
The helper `_coerce_mpl_figure(...)` accepts:
|
||||
|
||||
- a `Figure`
|
||||
- an `Axes`
|
||||
- an object with a `.figure` attribute pointing to a `Figure`
|
||||
|
||||
The export layer is standardized around Matplotlib figures, even if upstream plotting code returns a richer wrapper.
|
||||
|
||||
## 2. `save_figure(...)` is now single-target and suffix-explicit
|
||||
|
||||
A key source-level rule lives in `_resolve_save_figure_target(...)`:
|
||||
|
||||
- `save_figure(...)` now requires an explicit filename suffix,
|
||||
- supported examples include `.pdf`, `.svg`, `.png`, `.jpg`, `.tif`, `.eps`, `.ps`,
|
||||
- if there is no suffix, the function raises an error,
|
||||
- if multiple outputs are wanted, the source tells you to use `batch_export(...)`.
|
||||
|
||||
Skill implication:
|
||||
|
||||
- always write `results/figure1.pdf` rather than `results/figure1`
|
||||
- when you want several formats, recommend `batch_export(...)`, not legacy vector/raster format lists
|
||||
|
||||
## 3. Publication sizing path
|
||||
|
||||
`save_figure(...)` is publication-aware.
|
||||
|
||||
Internally it does the following:
|
||||
|
||||
1. load the chosen `FigureSpec` via `get_figure_spec(...)`
|
||||
2. resolve width through `resolve_width_mm(...)`
|
||||
3. resolve height through `resolve_height_mm(...)`
|
||||
4. set the Matplotlib figure size in inches using mm-to-inch conversion
|
||||
5. choose raster DPI from the spec unless overridden
|
||||
6. save the explicit target file
|
||||
7. restore original caller state afterward
|
||||
|
||||
Interpretation from source:
|
||||
|
||||
- export sizing is more than a file-write step; it can temporarily resize the figure to venue-oriented physical dimensions before output.
|
||||
|
||||
## 4. Width and height semantics
|
||||
|
||||
From `specs.py`:
|
||||
|
||||
- width can be `single`, `double`, or a numeric mm value
|
||||
- the built-in registry contains `nature`, `science`, and `cell`
|
||||
- height can be explicit `height_mm`
|
||||
- otherwise height is derived from `aspect_ratio`
|
||||
|
||||
That yields a clean rule for the skill:
|
||||
|
||||
- if the user asks for publication width, use `save_figure(..., spec=..., width=...)`
|
||||
- if the user only wants quick draft export, keep the recommendation minimal
|
||||
|
||||
## 5. `batch_export(...)`
|
||||
|
||||
`batch_export(...)` is the publication-aware multi-format lane.
|
||||
|
||||
Source behavior:
|
||||
|
||||
- it takes a `base_path`
|
||||
- it accepts publication export controls such as `spec`, `width`, `height_mm`, `aspect_ratio`, and `dpi`
|
||||
- appends each explicit suffix from `formats`
|
||||
- relayouts the figure through `_export_with_publication_layout(...)` for each target format
|
||||
- restores the original in-memory figure size/state after export
|
||||
|
||||
This is the right recommendation when the user needs, for example:
|
||||
|
||||
- `PDF` for manuscript submission
|
||||
- `SVG` for downstream editing
|
||||
- `PNG` for slides or issue threads
|
||||
|
||||
Operational implication:
|
||||
|
||||
- use `batch_export(...)` when the task needs multiple publication-style outputs from the same figure,
|
||||
- do not describe it as a plain `savefig` loop.
|
||||
|
||||
## 6. What `_save_basic_figure(...)` still does
|
||||
|
||||
`_save_basic_figure(...)` is still relevant, but it is no longer the main multi-format path for `batch_export(...)`.
|
||||
|
||||
It remains the lower-level path used for:
|
||||
|
||||
- direct basic export helpers,
|
||||
- size-preserving panel export in `export/panels.py`,
|
||||
- and internal single-target save operations that do not need publication relayout.
|
||||
|
||||
From the source, it also handles:
|
||||
|
||||
- output directory creation
|
||||
- vector-text rcParams (important for editable SVG/PDF text handling)
|
||||
- post-layout legend alignment
|
||||
- post-layout callbacks attached by plot code
|
||||
- trim/tight bbox behavior
|
||||
|
||||
So export quality is partially centralized in the export layer, not only inside plot modules.
|
||||
|
||||
## 7. Panel export lane
|
||||
|
||||
`export/panels.py` defines the multi-panel handoff path.
|
||||
|
||||
Key components:
|
||||
|
||||
- `PanelExportRecord`
|
||||
- `export_panel(...)`
|
||||
- `export_panels(...)`
|
||||
- `_write_panel_index(...)`
|
||||
|
||||
A `PanelExportRecord` stores:
|
||||
|
||||
- `panel_id`
|
||||
- `path`
|
||||
- `format`
|
||||
- `exported_at`
|
||||
- `figma_node_name`
|
||||
- `pubfig_version`
|
||||
- optional `title`
|
||||
- optional `label`
|
||||
|
||||
This shows that panel export is not only file emission. It also preserves minimal sync metadata.
|
||||
|
||||
## 8. Title stripping is intentional
|
||||
|
||||
One subtle but important source behavior:
|
||||
|
||||
- `_temporarily_strip_titles(...)` removes figure/axes titles during panel export by default unless `include_title=True`
|
||||
|
||||
Operational implication:
|
||||
|
||||
- panel-first composite assembly usually wants clean panel artwork,
|
||||
- whole-figure titles and layout labels are often handled later,
|
||||
- prefer exporting clean panel art first and adding whole-figure titles or layout labels downstream when needed.
|
||||
|
||||
## 9. Publication-aware vs size-preserving panel export
|
||||
|
||||
`export_panel(...)` has two modes:
|
||||
|
||||
### Publication-aware mode
|
||||
|
||||
Triggered when any of these are supplied:
|
||||
|
||||
- `spec`
|
||||
- `width`
|
||||
- `height_mm`
|
||||
|
||||
Then it delegates to `save_figure(...)`.
|
||||
|
||||
### Size-preserving mode
|
||||
|
||||
If none of those are supplied, it falls back to `_save_basic_figure(...)` and preserves the figure’s current size.
|
||||
|
||||
Skill implication:
|
||||
|
||||
- for reproducible paper panels, specify publication export parameters
|
||||
- for design review or quick composition, preserving current size may be acceptable
|
||||
|
||||
## 10. Multiple panel export
|
||||
|
||||
`export_panels(...)` does three main things:
|
||||
|
||||
1. normalize and validate panel ids,
|
||||
2. resolve labels for each panel,
|
||||
3. export each panel and optionally write `panel-index.json`.
|
||||
|
||||
Default recommendation:
|
||||
|
||||
- prefer this route when the user wants a structured panel directory rather than a single whole-figure asset.
|
||||
|
||||
## 11. Overwrite and safety behavior
|
||||
|
||||
From `_ensure_writable_target(...)`:
|
||||
|
||||
- an existing panel file raises unless `overwrite=True`
|
||||
|
||||
That is useful for skill guidance because it means refresh-in-place is an explicit decision.
|
||||
|
||||
## 12. Recommended source-faithful export patterns
|
||||
|
||||
### Single paper figure
|
||||
|
||||
- plot with `pubfig.<chart_family>(...)`
|
||||
- save with `save_figure(fig, 'out/figure1.pdf', spec='nature', width='single')`
|
||||
|
||||
### Same figure in several formats
|
||||
|
||||
- plot once
|
||||
- export with `batch_export(fig, 'out/figure1', formats=('pdf', 'svg', 'png'), spec='nature', width='single', dpi=300)`
|
||||
|
||||
### Multi-panel downstream assembly
|
||||
|
||||
- generate each panel as a separate `Figure`
|
||||
- export with `export_panels(...)`
|
||||
- use the index file for composite/Figma-aware downstream handling
|
||||
@@ -0,0 +1,216 @@
|
||||
# pubtab architecture (source-driven)
|
||||
|
||||
This guide explains `pubtab` from the actual package layout under `pubtab/src/pubtab`.
|
||||
|
||||
Core files:
|
||||
|
||||
- `__init__.py`
|
||||
- `cli.py`
|
||||
- `models.py`
|
||||
- `reader.py`
|
||||
- `renderer.py`
|
||||
- `config.py`
|
||||
- `_preview.py`
|
||||
- `tex_reader.py`
|
||||
- `backends/`
|
||||
- `themes/`
|
||||
|
||||
## 1. Start at `pubtab.__init__`
|
||||
|
||||
The public contract is defined in `__init__.py`.
|
||||
|
||||
Public exports are intentionally small:
|
||||
|
||||
- `xlsx2tex`
|
||||
- `preview`
|
||||
- `compile_pdf`
|
||||
- `tex_to_excel`
|
||||
- `SpacingConfig`
|
||||
|
||||
This is the key architectural signal: `pubtab` exposes a compact API, while the real complexity is pushed into reader/renderer/preview internals.
|
||||
|
||||
## 2. Core data model layer
|
||||
|
||||
`models.py` defines the structured table representation.
|
||||
|
||||
Important dataclasses:
|
||||
|
||||
- `CellStyle`
|
||||
- `Cell`
|
||||
- `TableData`
|
||||
- `SpacingConfig`
|
||||
- `ThemeConfig`
|
||||
- `BackendConfig`
|
||||
|
||||
Source fact:
|
||||
|
||||
- the shared structured representation is `TableData`, not raw Excel cells or raw LaTeX text.
|
||||
|
||||
Forward and reverse conversions both pass through this structured table model.
|
||||
|
||||
## 3. Forward pipeline: Excel to LaTeX
|
||||
|
||||
The main forward path is:
|
||||
|
||||
1. `xlsx2tex(...)` in `__init__.py`
|
||||
2. `read_excel(...)` in `reader.py`
|
||||
3. `render(...)` in `renderer.py`
|
||||
4. optional preview through `_preview.py`
|
||||
|
||||
This gives a clean source-driven decomposition:
|
||||
|
||||
- `reader.py` = file ingestion and structure recovery
|
||||
- `renderer.py` = LaTeX generation
|
||||
- `_preview.py` = compile and raster preview
|
||||
|
||||
## 4. Reverse pipeline: LaTeX to Excel
|
||||
|
||||
The reverse path is:
|
||||
|
||||
1. `tex_to_excel(...)` in `__init__.py`
|
||||
2. `read_tex_multi(...)` from `tex_reader.py`
|
||||
3. writer functions to `.xlsx`
|
||||
|
||||
So roundtrip support is not an afterthought. It is a real architecture branch.
|
||||
|
||||
## 5. `reader.py` is richer than a plain spreadsheet loader
|
||||
|
||||
From the source, `reader.py` does much more than “read cells”:
|
||||
|
||||
- supports `.xlsx` and `.xls`
|
||||
- extracts rich text segments
|
||||
- reconstructs merged cells and spans
|
||||
- reads styling information
|
||||
- trims only trailing globally empty columns
|
||||
- reads pubtab metadata sheets
|
||||
- restores group separators, multicolumn alignment hints, and math-script hints
|
||||
|
||||
Interpretation from source:
|
||||
|
||||
- `pubtab` is optimized for **publication table semantics**, rather than for plain tabular text dumping.
|
||||
|
||||
## 6. `renderer.py` is the central logic hub
|
||||
|
||||
`renderer.py` turns `TableData` into backend-specific LaTeX.
|
||||
|
||||
From the source, it handles:
|
||||
|
||||
- style/theme loading
|
||||
- backend template loading
|
||||
- spacing resolution
|
||||
- column spec construction or projection
|
||||
- tabular vs tabularray differences
|
||||
- merged cells, row/column spans, header rules, vertical rules
|
||||
- background colors and grouped separators
|
||||
- final template rendering
|
||||
|
||||
This file is where most of the difficult publication logic lives.
|
||||
|
||||
For skill design, that means:
|
||||
|
||||
- backend choice is not a cosmetic toggle
|
||||
- column spec and rule behavior are structural concerns
|
||||
- preview/render bugs usually require reading `renderer.py`
|
||||
|
||||
## 7. Theme vs backend is a real split
|
||||
|
||||
The codebase separates:
|
||||
|
||||
- **themes** in `themes/`
|
||||
- **LaTeX backends** in `backends/`
|
||||
|
||||
That is reflected in two dataclasses:
|
||||
|
||||
- `ThemeConfig`
|
||||
- `BackendConfig`
|
||||
|
||||
And in two loaders:
|
||||
|
||||
- `load_theme(...)`
|
||||
- `load_backend(...)`
|
||||
|
||||
This is a major architectural point for the skill:
|
||||
|
||||
- theme decides stylistic defaults,
|
||||
- backend decides LaTeX environment/template behavior.
|
||||
|
||||
Do not explain them as if they were the same thing.
|
||||
|
||||
## 8. Config precedence is explicit
|
||||
|
||||
In `xlsx2tex(...)`, the source implements a clear precedence order:
|
||||
|
||||
1. hardcoded defaults
|
||||
2. YAML config loaded by `config.py`
|
||||
3. explicit function kwargs
|
||||
4. in some roundtrip cases, values recovered from `TableData`
|
||||
|
||||
This is why user-facing guidance should say “CLI flags or function kwargs override YAML config.”
|
||||
|
||||
## 9. Multi-file and multi-sheet support are first-class
|
||||
|
||||
From `__init__.py`:
|
||||
|
||||
- directory input is supported for both forward and reverse paths
|
||||
- sheet enumeration is supported when `sheet=None`
|
||||
- multi-sheet export produces `*_sheetNN.tex`
|
||||
|
||||
The default skill guidance can therefore recommend batch/file-driven workflows, not only one-table-at-a-time usage.
|
||||
|
||||
## 10. Preview is a real compilation layer
|
||||
|
||||
`preview(...)` and `compile_pdf(...)` in `__init__.py` delegate into `_preview.py`.
|
||||
|
||||
That layer:
|
||||
|
||||
- finds or installs `pdflatex`
|
||||
- builds a standalone document
|
||||
- compiles the output
|
||||
- retries missing packages through `tlmgr`
|
||||
- converts PDF to PNG when requested
|
||||
|
||||
Preview is not a fake HTML-like snapshot. It is a real LaTeX compile pipeline.
|
||||
|
||||
## 11. `tex_reader.py` closes the roundtrip loop
|
||||
|
||||
`tex_reader.py` is substantial, not decorative.
|
||||
|
||||
From the source it supports parsing of:
|
||||
|
||||
- `tabular`
|
||||
- `tblr`
|
||||
- `longtblr`
|
||||
- `talltblr`
|
||||
|
||||
It also handles:
|
||||
|
||||
- color parsing
|
||||
- rule parsing
|
||||
- multirow/multicolumn reconstruction
|
||||
- metadata extraction
|
||||
- grouped rows and placeholder cleanup
|
||||
|
||||
This makes `pubtab` suitable for source-aware roundtrip and migration tasks, rather than only one-way Excel export.
|
||||
|
||||
## 12. Reading order for source debugging
|
||||
|
||||
When you need source-level certainty, use this order:
|
||||
|
||||
1. `pubtab/src/pubtab/__init__.py`
|
||||
2. `pubtab/src/pubtab/models.py`
|
||||
3. `pubtab/src/pubtab/reader.py`
|
||||
4. `pubtab/src/pubtab/renderer.py`
|
||||
5. `pubtab/src/pubtab/_preview.py`
|
||||
6. `pubtab/src/pubtab/tex_reader.py`
|
||||
7. `pubtab/src/pubtab/backends/` and `themes/`
|
||||
8. `pubtab/src/pubtab/cli.py` for flag-to-API mapping only
|
||||
|
||||
## 13. Implications for this skill
|
||||
|
||||
The source says the most faithful default workflow is:
|
||||
|
||||
- use `xlsx2tex(...)` or CLI `xlsx2tex` for forward generation,
|
||||
- use `preview(...)` to verify actual compile output,
|
||||
- use `tex_to_excel(...)` for roundtrip or migration tasks,
|
||||
- explain theme/backend separately,
|
||||
- escalate into renderer/source debugging only when table structure or LaTeX behavior is the real problem.
|
||||
@@ -0,0 +1,201 @@
|
||||
# pubtab backend and preview internals (source-driven)
|
||||
|
||||
This guide focuses on the two most important deeper layers in `pubtab`:
|
||||
|
||||
- backend/theme separation
|
||||
- real preview/compile execution
|
||||
|
||||
Primary source files:
|
||||
|
||||
- `pubtab/src/pubtab/themes/__init__.py`
|
||||
- `pubtab/src/pubtab/backends/__init__.py`
|
||||
- `pubtab/src/pubtab/renderer.py`
|
||||
- `pubtab/src/pubtab/_preview.py`
|
||||
|
||||
## 1. Theme and backend are different contracts
|
||||
|
||||
The source separates them cleanly.
|
||||
|
||||
### Theme layer
|
||||
|
||||
In `themes/__init__.py`, a theme resolves to `ThemeConfig`.
|
||||
|
||||
It carries style defaults such as:
|
||||
|
||||
- `column_sep`
|
||||
- `font_size`
|
||||
- `caption_position`
|
||||
- `SpacingConfig`
|
||||
|
||||
### Backend layer
|
||||
|
||||
In `backends/__init__.py`, a backend resolves to:
|
||||
|
||||
- `BackendConfig`
|
||||
- a Jinja template string
|
||||
|
||||
`BackendConfig` carries things like:
|
||||
|
||||
- package requirements
|
||||
- preamble hints
|
||||
- backend identity
|
||||
|
||||
Operational implication:
|
||||
|
||||
- theme = stylistic defaults
|
||||
- backend = LaTeX environment/template machinery
|
||||
|
||||
## 2. Legacy normalization exists, but it is not the conceptual model
|
||||
|
||||
The source still supports legacy composite theme names like `_tabularray` suffixed themes.
|
||||
|
||||
Normalization happens in places like:
|
||||
|
||||
- `_normalize_theme_backend_choice(...)`
|
||||
- `resolve_theme(...)`
|
||||
|
||||
But that compatibility layer should not define the skill’s main explanation.
|
||||
The current conceptual model is still **separate theme + backend**.
|
||||
|
||||
## 3. What `render(...)` actually does
|
||||
|
||||
`renderer.py` is the core place where theme/backend decisions become concrete LaTeX.
|
||||
|
||||
Inside `render(...)`, the source does roughly this:
|
||||
|
||||
1. normalize theme/backend choice
|
||||
2. load theme config
|
||||
3. load backend config and template
|
||||
4. merge default/theme/user spacing
|
||||
5. compute or project column specs
|
||||
6. branch into backend-specific row/cell rendering
|
||||
7. render through Jinja
|
||||
|
||||
Backend choice changes the internal rendering algorithm, not only the final environment name.
|
||||
|
||||
## 4. `tabular` vs `tabularray` from the source perspective
|
||||
|
||||
The source suggests this practical distinction:
|
||||
|
||||
### `tabular`
|
||||
|
||||
- more classic LaTeX path
|
||||
- column spec and rules are handled in the traditional environment
|
||||
- when source `column_spec` exists, renderer tries to preserve classic rule structure
|
||||
|
||||
### `tabularray`
|
||||
|
||||
- dedicated `tblr`-style rendering path
|
||||
- special handling for promoted vertical lines and grouped header boundaries
|
||||
- extra sanitization in preview compile path
|
||||
|
||||
Interpretation from source:
|
||||
|
||||
- `tabularray` is a distinct renderer path with its own structural handling, not merely a cosmetic wrapper around `tabular`.
|
||||
|
||||
## 5. Why backend choice sometimes matters a lot
|
||||
|
||||
From `renderer.py`, backend differences affect:
|
||||
|
||||
- how colspec is interpreted
|
||||
- how vertical rules are preserved or promoted
|
||||
- how merged cells are encoded
|
||||
- how row coloring and header boundaries are emitted
|
||||
|
||||
So if a table is structurally complex, the skill should not present backend choice as arbitrary.
|
||||
|
||||
## 6. Preview is a real LaTeX toolchain
|
||||
|
||||
`_preview.py` proves preview is a full execution pipeline.
|
||||
|
||||
Major steps:
|
||||
|
||||
1. locate `pdflatex`
|
||||
2. install TinyTeX if missing
|
||||
3. build a standalone document around the table
|
||||
4. compile with `pdflatex`
|
||||
5. auto-install missing LaTeX packages when possible
|
||||
6. return PDF or convert to PNG
|
||||
|
||||
Preview is therefore a genuine verification step.
|
||||
|
||||
## 7. `pdflatex` discovery and TinyTeX fallback
|
||||
|
||||
The source checks:
|
||||
|
||||
- system `PATH`
|
||||
- pubtab-managed TinyTeX under `~/.pubtab/TinyTeX`
|
||||
|
||||
If neither exists, it installs TinyTeX automatically.
|
||||
|
||||
This is why the skill can confidently describe preview as relatively self-healing, while still warning that first-run setup may download TeX assets.
|
||||
|
||||
## 8. Missing package retry
|
||||
|
||||
A particularly important source behavior:
|
||||
|
||||
- compile logs are scanned for missing `.sty`
|
||||
- missing style names are mapped to `tlmgr` package names when needed
|
||||
- `tlmgr install <pkg>` is run automatically
|
||||
- compilation is retried
|
||||
|
||||
This is strong evidence that the recommended workflow should include preview, because preview can repair part of the environment on the way.
|
||||
|
||||
## 9. Standalone preview wrapping
|
||||
|
||||
`_build_standalone(...)` does more than wrap text in a document.
|
||||
|
||||
It also:
|
||||
|
||||
- imports backend-required packages
|
||||
- preserves setup commands outside the `resizebox` body
|
||||
- wraps the body in a preview-friendly standalone/minipage layout
|
||||
- converts `\caption{...}` into `\captionof{table}{...}` during float stripping
|
||||
|
||||
This explains why preview output can differ from naive manual compilation if the user simply pastes a table fragment into a document incorrectly.
|
||||
|
||||
## 10. `tabularray` preview sanitization
|
||||
|
||||
Before compilation, `_sanitize_tblr_for_compile(...)` removes some commands that break inside `tblr` preview contexts, including certain row-color and `\cmidrule` forms.
|
||||
|
||||
So when debugging preview-vs-final-document differences, this source behavior matters.
|
||||
|
||||
## 11. PDF to PNG conversion fallback stack
|
||||
|
||||
For PNG previews, the source tries:
|
||||
|
||||
1. `pdf2image`
|
||||
2. fallback to PyMuPDF (`fitz`)
|
||||
|
||||
Preview generation remains resilient even after PDF compilation succeeds.
|
||||
|
||||
## 12. Practical skill guidance from the source
|
||||
|
||||
### Recommend `tabular` when
|
||||
|
||||
- the table is simple/classic,
|
||||
- the user wants conservative LaTeX output,
|
||||
- compatibility matters more than modern layout features.
|
||||
|
||||
### Recommend `tabularray` when
|
||||
|
||||
- grouped headers and more complex structural layouts matter,
|
||||
- the user is already targeting a `tblr`-capable workflow,
|
||||
- the roundtrip/source table has rule structure that benefits from the dedicated renderer path.
|
||||
|
||||
### Recommend preview almost always when
|
||||
|
||||
- the table is intended for publication,
|
||||
- the user is changing backend/theme/colspec,
|
||||
- the task involves debugging table layout,
|
||||
- the task depends on compile-time package correctness.
|
||||
|
||||
## 13. Failure triage order
|
||||
|
||||
When the generated table is wrong, debug in this order:
|
||||
|
||||
1. check theme/backend choice,
|
||||
2. inspect `render(...)` inputs (`TableData`, colspec, span settings),
|
||||
3. run preview/compile,
|
||||
4. inspect missing package or backend-specific compile issues,
|
||||
5. only then move into manuscript-level integration debugging.
|
||||
@@ -0,0 +1,178 @@
|
||||
# pubtab CLI and API flow (source-driven)
|
||||
|
||||
This guide follows the actual control flow from `pubtab/src/pubtab/cli.py` into the public API and then into the internal pipeline.
|
||||
|
||||
## 1. Architectural headline
|
||||
|
||||
`cli.py` is a **thin Click wrapper** over the public API in `pubtab.__init__`.
|
||||
|
||||
Operational implication:
|
||||
|
||||
- command-line behavior should usually match Python API behavior,
|
||||
- when docs disagree, the source of truth is `pubtab.__init__`, not CLI help text alone.
|
||||
|
||||
## 2. CLI commands exposed in `cli.py`
|
||||
|
||||
The main commands are:
|
||||
|
||||
- `pubtab xlsx2tex`
|
||||
- `pubtab themes`
|
||||
- `pubtab tex2xlsx`
|
||||
- `pubtab preview`
|
||||
|
||||
There is also a hidden backward-compatible alias:
|
||||
|
||||
- `convert` -> `xlsx2tex`
|
||||
|
||||
## 3. `xlsx2tex` command flow
|
||||
|
||||
CLI entrypoint:
|
||||
|
||||
- `xlsx2tex_cmd(...)` in `cli.py`
|
||||
|
||||
Control flow:
|
||||
|
||||
1. validate input/output shape
|
||||
2. coerce `--sheet` into int when possible
|
||||
3. build kwargs only for explicitly provided options
|
||||
4. call `pubtab.xlsx2tex(input_file, output, **kwargs)`
|
||||
5. print output summary based on sheet count and preview mode
|
||||
|
||||
This thin-wrapper design matters because the CLI does **not** reimplement conversion logic.
|
||||
|
||||
## 4. `xlsx2tex(...)` API flow
|
||||
|
||||
The public API in `__init__.py` adds the real orchestration:
|
||||
|
||||
### Input modes
|
||||
|
||||
- single Excel file
|
||||
- directory of Excel files
|
||||
- single sheet
|
||||
- all sheets (`sheet=None`)
|
||||
|
||||
### Output path behavior
|
||||
|
||||
- single-sheet export can target a direct `.tex` path
|
||||
- directory input must target a directory
|
||||
- multi-sheet export uses `*_sheetNN.tex`
|
||||
|
||||
This behavior is implemented by `_build_sheet_output_paths(...)` and directory iteration helpers.
|
||||
|
||||
## 5. Config precedence in the real API
|
||||
|
||||
Inside `xlsx2tex(...)`, the source builds parameters in this order:
|
||||
|
||||
1. defaults
|
||||
2. YAML config via `load_config(...)`
|
||||
3. explicit kwargs passed from CLI or Python
|
||||
4. roundtrip-restored values where relevant
|
||||
|
||||
Operational rule:
|
||||
|
||||
- YAML config sets baseline behavior,
|
||||
- CLI flags / Python kwargs override it.
|
||||
|
||||
## 6. Sheet expansion behavior
|
||||
|
||||
When `sheet is None`, the source does not simply choose the first sheet.
|
||||
It calls `list_excel_sheets(...)` and expands all sheet names into separate outputs.
|
||||
|
||||
That is why a single workbook can generate:
|
||||
|
||||
- `table_sheet01.tex`
|
||||
- `table_sheet02.tex`
|
||||
- ...
|
||||
|
||||
The skill should explicitly mention this when users want appendix exports or workbook-wide conversion.
|
||||
|
||||
## 7. Read -> render -> write flow
|
||||
|
||||
For each selected sheet, `xlsx2tex(...)` does:
|
||||
|
||||
1. `read_excel(...)`
|
||||
2. optional header or group-separator reconstruction
|
||||
3. `render(...)`
|
||||
4. write `.tex`
|
||||
5. optional preview generation to `.png`
|
||||
|
||||
Preview is downstream of actual `.tex` generation, not an alternate renderer.
|
||||
|
||||
## 8. `preview` command flow
|
||||
|
||||
CLI entrypoint:
|
||||
|
||||
- `preview_cmd(...)`
|
||||
|
||||
The CLI again mostly validates paths and forwards to `pubtab.preview(...)`.
|
||||
|
||||
The public `preview(...)` API supports:
|
||||
|
||||
- raw LaTeX content
|
||||
- a single `.tex` file
|
||||
- a directory of `.tex` files
|
||||
- `png` or `pdf` output
|
||||
|
||||
A key source detail: when backend is omitted, `preview(...)` may infer it from the LaTeX content using `_resolve_preview_inputs(...)`.
|
||||
|
||||
## 9. Backend inference path
|
||||
|
||||
`_infer_latex_backend(...)` checks for environments like:
|
||||
|
||||
- `tblr`
|
||||
- `longtblr`
|
||||
- `talltblr`
|
||||
|
||||
If found, backend becomes `tabularray`; otherwise `tabular`.
|
||||
|
||||
Operational implication:
|
||||
|
||||
- a preview or compile call can often resolve the correct backend without requiring an explicit `--latex-backend` flag.
|
||||
|
||||
## 10. `compile_pdf(...)` API flow
|
||||
|
||||
Public `compile_pdf(...)` in `__init__.py` does:
|
||||
|
||||
1. detect whether input is raw LaTeX or a file path,
|
||||
2. infer theme/backend if needed,
|
||||
3. delegate to `_preview.compile_pdf(...)`.
|
||||
|
||||
The compile path is still part of the public API, even though the heavy lifting is in `_preview.py`.
|
||||
|
||||
## 11. `tex2xlsx` command flow
|
||||
|
||||
CLI entrypoint:
|
||||
|
||||
- `tex2xlsx(...)` in `cli.py`
|
||||
|
||||
It forwards to `pubtab.tex_to_excel(...)`.
|
||||
|
||||
The public API then handles:
|
||||
|
||||
- single `.tex` file -> one `.xlsx`
|
||||
- multi-table `.tex` -> one workbook with multiple sheets
|
||||
- directory of `.tex` files -> one `.xlsx` per file
|
||||
|
||||
This keeps the reverse path operationally symmetric with the forward path.
|
||||
|
||||
## 12. Why the CLI should stay thin in this skill
|
||||
|
||||
Because the real logic is centralized in `pubtab.__init__`, the skill should:
|
||||
|
||||
- use CLI examples for file-driven shell workflows,
|
||||
- use Python API examples for notebooks or scripted pipelines,
|
||||
- avoid duplicating pseudo-logic that already exists in the library.
|
||||
|
||||
## 13. Recommended source-faithful routing
|
||||
|
||||
### Use CLI when
|
||||
|
||||
- the user already has Excel or `.tex` files on disk,
|
||||
- the task is batch conversion,
|
||||
- the user wants a terminal-first workflow.
|
||||
|
||||
### Use Python API when
|
||||
|
||||
- the user is in a notebook or script,
|
||||
- the table needs custom preprocessing before render,
|
||||
- the agent is composing a larger Python pipeline.
|
||||
@@ -0,0 +1,82 @@
|
||||
# Workflow
|
||||
|
||||
## Default decision order
|
||||
|
||||
1. Identify the scientific communication goal.
|
||||
2. Probe the environment and available assets lightly.
|
||||
3. Decide whether the evidence should be a figure, a table, or a paired figure+table deliverable.
|
||||
4. Choose the strongest representation family.
|
||||
5. Route to `pubfig`, `pubtab`, or both.
|
||||
6. Produce the smallest runnable implementation.
|
||||
7. Specify export outputs explicitly.
|
||||
8. Run publication QA.
|
||||
9. Propose revisions if the result is weak.
|
||||
|
||||
## Handoff checklist
|
||||
|
||||
For every task, try to make these explicit:
|
||||
|
||||
- claim the artifact is supposed to support
|
||||
- data shape and grouping structure
|
||||
- target audience or venue expectations
|
||||
- figure vs table role
|
||||
- exact output filenames and formats
|
||||
- whether the artifact is final, draft, or revision
|
||||
- whether the current environment can execute the proposed route immediately
|
||||
|
||||
## Delivery contract
|
||||
|
||||
A strong response should make clear:
|
||||
|
||||
- which artifact type was chosen,
|
||||
- why it was chosen,
|
||||
- which tool owns each artifact,
|
||||
- what the first runnable command/code path is,
|
||||
- what output files should be produced,
|
||||
- what still needs user input or upstream data.
|
||||
|
||||
## Default output priorities
|
||||
|
||||
Prioritize in this order:
|
||||
|
||||
1. clarity of claim
|
||||
2. correct artifact type
|
||||
3. minimal runnable implementation
|
||||
4. publication-ready export
|
||||
5. QA and revision guidance
|
||||
|
||||
## Graceful degradation when tools are missing
|
||||
|
||||
If `pubfig` or `pubtab` is not installed:
|
||||
|
||||
- keep the workflow going,
|
||||
- provide installation guidance,
|
||||
- provide pseudocode or draft commands,
|
||||
- specify the recommended artifact structure,
|
||||
- preserve the QA and revision guidance.
|
||||
|
||||
## Figure / table split rules
|
||||
|
||||
Use a **figure** when the reader needs to quickly perceive:
|
||||
|
||||
- trend
|
||||
- distribution shape
|
||||
- relationship
|
||||
- calibration or diagnostic behavior
|
||||
- composition or hierarchy
|
||||
- visual comparison across a moderate number of groups
|
||||
|
||||
Use a **table** when the reader needs:
|
||||
|
||||
- exact numbers
|
||||
- many metrics side by side
|
||||
- benchmark grids
|
||||
- ablation matrices
|
||||
- appendix-style detail
|
||||
- reproducible value lookup
|
||||
|
||||
Use **both** when:
|
||||
|
||||
- the figure carries the visual claim,
|
||||
- the table preserves exact values,
|
||||
- or the paper section benefits from a fast visual summary plus precise numeric evidence.
|
||||
Reference in New Issue
Block a user