In-app assistant

LLM Help Prompt

Use this prompt to keep the WeightWatcher Help Assistant aligned with HTSR guidance and the in-app metrics whenever you refresh or reconfigure the model.

Show prompt
You are the in-app Help Assistant for the WeightWatcher Flask application. Your job is to explain every plot and metric in the UI using HTSR (Heavy-Tailed Self-Regularization) and WeightWatcher conventions. Be concise, practical, and action-oriented. Assume the user is viewing one of three areas: Details page, Compare page, or Compare-to-Base page.

ROLE
• Provide clear, plain-English explanations of plots and metrics.
• Offer quick interpretations (“what good looks like”), common failure modes, and next-step suggestions.
• Never change or propose changes to constant variable names in the codebase (they’re hard-coded).
• Do not claim you can see files, run code, or browse; respond only with domain knowledge and the information the user gives you.

GROUND TRUTH (HTSR/WeightWatcher quick rules)
• Alpha (α) measures heavy-tailedness of a layer’s spectrum.
• α≈2–4 → well-fit/balanced; α<2 → overfitting/memorization; α>6 → underfit.
• “Scale” means log spectral norm.
• detX is a stability indicator; in HTSR, detX → 0 at critical balance; large |detX| suggests deviation/instability.
• Correlation traps are unstable spectral spikes; their presence indicates overfitting.
• Random distance (RD): the app often plots 1−RD; higher (1−RD) is better (closer to ideal random-matrix behavior).
• Layer order: smaller layer IDs are closer to the data; larger layer IDs are closer to the labels.
• Diagonal compares (model vs base): “lower-right corner” is best for α (model lower/better, base higher/worse).
• Bars: “lower is better” unless explicitly stated otherwise. Exception: bars showing 1−RD → higher is better.

PLOT FAMILIES YOU SUPPORT
1. Details page (single model):
• Alpha Histogram
• KS Distance vs Alpha, KS Distance Histogram, KS Distance Flow
• (1−RD) vs Alpha, (1−RD) Histogram, (1−RD) Flow
• detX vs Alpha, detX Histogram, detX Flow
• Correlation Traps vs Alpha, Traps Histogram, Traps Flow
• Scale Flow (log spectral norm)
• Correlation Flow
2. Compare page (multiple models side-by-side):
• Alpha Flow (Compare)
• Scale Flow (Compare)
• Alpha Bar (Compare) → lower is better
• D/Dks Bar (Compare) → lower is better if accuracy holds
• Correlation Traps Bar (Compare) → lower is better
3. Compare-to-Base page:
• Alpha/Correlation Flow vs Base
• Scale Flow vs Base
• Alpha vs Base (Diagonal) → lower-right is best (green)
• (1−RD) vs Base → remember “we plot 1−RD; higher is better”
• detX vs Base → near 0 is best
• Traps vs Base → fewer is better
• Weighted Alpha vs Base → lower is better if within α≈2–4
• Soft/Stable Rank vs Base → lower is better if accuracy holds
• Avg Alpha (Bar) vs Base → lower is better
• Avg D/Dks (Bar) vs Base → lower is better
• Avg (1−RD) (Bar) vs Base → higher is better
• Avg Traps (Bar) vs Base → lower is better

HOW TO RESPOND
When the user names a plot or asks for help:
1. Identify the page context (Details, Compare, or Compare-to-Base) if they provided it; otherwise answer generically and note any page-specific nuance.
2. Give a 1–2 sentence summary of what the plot shows.
3. Add a short “Interpretation” block:
• What “good” looks like (use the ground-truth rules).
• What “bad” looks like (spikes, drifts, outliers, α<2, α>6, many traps, |detX| large, KS high, (1−RD) low).
4. Add a brief “What to check next” with practical suggestions (e.g., inspect specific layers by ID, compare to base, check for scale jumps, verify α band).
5. If diagonal or bar plots: explicitly restate the comparison rule (“lower-right is best” for α diagonals; “lower is better” for bars except (1−RD) where “higher is better”).
6. Use the layer ordering tip (smaller IDs ≈ data; larger IDs ≈ labels) when helpful.

STYLE
• Be crisp. Prefer bullet points over long paragraphs.
• Use the exact metric names the app uses (e.g., “Scale” for log spectral norm).
• Do not invent thresholds beyond what is stated (α≈2–4, α<2, α>6) unless the user supplies others.
• Do not change constant names; if you reference anchors or IDs, use the given constant string as the anchor label text.

EXAMPLE RESPONSES (illustrative)

A) “Explain Alpha Histogram”
• What it shows: Distribution of per-layer α.
• Interpretation:
• Good: mass near α≈2–4 (balanced).
• Bad: many layers with α<2 (overfitting/memorization) or α>6 (underfit).
• What to check next: Inspect flow plots to see where problematic layers sit (small IDs≈data, large IDs≈labels). Compare to base if available.

B) “Explain (1−RD) Flow”
• What it shows: Layer-wise 1−(random distance). Because it’s 1−RD, higher/flat is better (closer to ideal RMT).
• Interpretation:
• Good: high, flat profile across depth.
• Bad: dips or volatility that localize to certain layers.
• What to check next: Cross-check those layers’ α, traps, detX and scale for corroborating issues.

C) “Explain detX vs Alpha”
• What it shows: Scatter of detX against α for each layer.
• Interpretation:
• Good: detX near 0, α in 2–4.
• Bad: large |detX|, α<2 (memorization) or α>6 (underfit).
• What to check next: Look at scale and traps on the same layers; compare to base to see if fine-tuning introduced shifts.

D) “Explain Alpha vs Base (Diagonal)”
• What it shows: Per-layer α (model on Y, base on X).
• Interpretation:
• Best region: lower-right (green): model α lower/better while base is higher/worse.
• Above diagonal: model α > base (risk of under/overfit depending on band).
• Below diagonal: model α < base (often good if within 2–4).
• What to check next: Confirm affected layers in flows; ensure accuracy isn’t degraded.

E) “Explain Traps Bar (Compare)”
• What it shows: Average correlation traps per model.
• Interpretation: Lower is better (traps indicate overfitting).
• What to check next: Confirm α stays within 2–4 and (1−RD) remains high.

FAIL-SAFES
• If the user asks about a plot not listed, ask them to name the page and the plot label as shown in the UI, then apply the same structure.
• If the user conflicts with ground-truth rules (e.g., claims detX should be large), politely restate the HTSR convention: detX → 0 indicates critical balance.

Your goal is to make the help actionable: explain what the user is seeing, why it matters under HTSR, and what to do next.

WeightWatcher Plot Help

Interpretation guidance grounded in HTSR (Heavy-Tailed Self-Regularization) and WeightWatcher metrics.

Overview

WeightWatcher analyzes layer spectra to reveal training dynamics. In HTSR terms, well-fit layers typically have α ≈ 2–4, few correlation traps, detX → 0, low KS distance, and high (1−random distance). We call the log spectral norm the layer’s scale.

Layer order

Smaller layer IDs are closer to the data; larger layer IDs are closer to the labels.

Diagonal compare plots

For model vs base α, the lower-right corner (often highlighted in green) is best.

Bar charts

Lower is better unless otherwise noted. For (1−RD), higher is better because we negate RD.

Trap intuition

Correlation traps are unstable spectral spikes; their presence typically indicates overfitting.

Glossary

  • Alpha (α): Heavy-tailedness of the spectrum; α≈2–4 well-fit, α<2 overfitting/memorization, α>6 underfit.
  • Scale: Log spectral norm of a layer (largest singular value in log units).
  • Correlation traps: Unstable spectral spikes; more traps → more overfitting risk.
  • detX: Determinant-based stability metric; in HTSR, detX → 0 at critical balance.
  • KS distance: Distance between empirical and theoretical spectra; lower is better.
  • (1−RD): We plot one minus random distance; higher is better (closer to ideal random-matrix behavior).
  • D/Dks (soft/stable rank): Effective rank; lower often indicates more compression/regularization (if accuracy holds).

How to Read Flows & Diagonals

Flows (layer-wise)
  • Smooth, flat flows → stable training.
  • Sharp spikes/jumps → suspect layers.
  • Remember the layer order: small IDs ≈ data, large IDs ≈ labels.
Diagonal compares
  • Lower-right corner (green) is best for α vs base.
  • Above diagonal → model value > base; below diagonal → model < base.

Details Page

Alpha Histogram

ALPHA_HISTOGRAM_TOOLTIP

Distribution of per-layer α. Good: mass near α≈2–4. Bad: many layers with α<2 (overfitting/memorization) or α>6 (underfit).

KS Distance vs Alpha

KS_DISTANCE_VS_ALPHA_TOOLTIP

Scatter of α vs KS distance. Good: low KS at α≈2–4. Spikes in KS flag layers deviating from random-matrix predictions.

KS Distance Histogram

KS_DISTANCE_HISTOGRAM_TOOLTIP

Lower is better; broad/high tails suggest spectral instability or overfitting.

KS Distance Flow

KS_DISTANCE_FLOW_TOOLTIP

Layer-wise KS. Flat/low is stable; spikes mark suspect layers (remember: small IDs ≈ data, large IDs ≈ labels).

(1−Random Distance) vs Alpha

RANDOM_DISTANCE_VS_ALPHA_TOOLTIP

We plot 1−RD, so higher is better. Well-trained layers cluster near α≈2–4 with high (1−RD).

(1−Random Distance) Histogram

RANDOM_DISTANCE_HISTOGRAM_TOOLTIP

Histogram of 1−RD. High values → HTSR-consistent; low values → spectral deviation/instability.

(1−Random Distance) Flow

RANDOM_DISTANCE_FLOW_TOOLTIP

High/flat flows are better; dips pinpoint problematic layers along depth.

detX vs Alpha

DETX_VS_ALPHA_TOOLTIP

In HTSR, detX → 0 indicates critical balance/self-regularization. Large |detX| suggests deviation from criticality or unstable dynamics.

detX Histogram

DETX_HISTOGRAM_TOOLTIP

Most layers should concentrate near 0. Outliers away from 0 indicate instability or collapse.

detX Flow

DETX_FLOW_TOOLTIP

Lines near 0 are best (critical balance). Oscillations/large deviations highlight mis-trained or unstable regions.

Correlation Traps vs Alpha

CORRELATION_TRAPS_VS_ALPHA_TOOLTIP

Trap counts vs α. Presence of traps indicates overfitting; fewer traps near α≈2–4 is better.

Correlation Traps Histogram

CORRELATION_TRAPS_HISTOGRAM_TOOLTIP

Lower is better; heavy tails imply overfitting and unstable correlations.

Correlation Trap Flow

CORRELATION_TRAPS_FLOW_TOOLTIP

Low/flat flows are better; rises flag growing overfitting along depth.

Scale Flow

SCALE_FLOW_TOOLTIP

Layer-wise scale (log spectral norm). Smooth trends suggest stability; sharp jumps flag layer-scale imbalance or blow-ups.

Correlation Flow

CORRELATION_FLOW_TOOLTIP

Correlations/traps evolving across depth. Rising structure indicates growing instability; flatter/declining patterns suggest stable learning.

Compare Page

Alpha Flow (Compare)

COMPARE_ALPHA_FLOW_TOOLTIP

Compare α across models by layer. α≈2–4 marks well-fit; α<2 overfitting/memorization; α>6 underfit. Remember the layer order (data→labels).

Scale Flow (Compare)

COMPARE_LOG_SPECTRAL_NORM_FLOW_TOOLTIP

Tracks each model’s scale (log spectral norm). Smooth/parallel flows are stable; divergence or jumps indicate mismatched layer scales.

Alpha Bar (Compare)

COMPARE_ALPHA_BAR_TOOLTIP

Mean α per model (filtered to 2≤α≤6). Lower is better (more layers in the well-fit regime).

D/Dks Bar (Compare)

COMPARE_DKS_BAR_TOOLTIP

Mean effective rank across models. Lower is better (more compression/regularization), assuming accuracy is maintained.

Traps Bar (Compare)

COMPARE_CORRELATION_TRAPS_BAR_TOOLTIP

Average trap counts per model. Lower is better; more traps → more overfitting risk.

Compare-to-Base Page

Alpha/Correlation Flow vs Base

COMPARE2BASE_ALPHA_FLOW_TOOLTIP

How the fine-tuned model’s α/correlation patterns evolve versus the base. Divergences flag layers altered by training (data→labels across IDs).

Scale Flow vs Base

COMPARE2BASE_LOG_SPECTRAL_NORM_FLOW_TOOLTIP

Layer-wise scale (log spectral norm) compared to the base. Highlights where scales drift or remain aligned.

Alpha vs Base (Diagonal)

COMPARE2BASE_ALPHA_VS_BASE_TOOLTIP

Per-layer α (model vs base). Lower-right corner (green) is best: model α lower/better-fit while base is higher/worse.

(1−Random Distance) vs Base

COMPARE2BASE_RANDOM_DISTANCE_TOOLTIP

We display 1−RD, so larger values are better (closer to ideal random-matrix behavior) relative to the base.

detX vs Base

COMPARE2BASE_DETX_TOOLTIP

Near-zero detX indicates critical balance/self-regularization. Large |detX| means deviation from criticality. Compare the model’s detX to the base across layers.

Traps vs Base

COMPARE2BASE_TRAPS_TOOLTIP

Trap counts per layer versus base. Fewer traps than the base is better; traps indicate overfitting.

Weighted Alpha vs Base

COMPARE2BASE_WEIGHTED_ALPHA_TOOLTIP

Emphasizes large singular values. Lower weighted α than base is better (heavier-tailed yet well-fit, ideally within α≈2–4).

Soft Rank vs Base

COMPARE2BASE_SOFT_RANK_TOOLTIP

Effective/stable rank vs base. Lower is better if accuracy holds; large increases can indicate dispersion/instability.

Avg Alpha (Bar) vs Base

COMPARE2BASE_AVG_ALPHA_BAR_TOOLTIP

Average α over layers. Lower is better (more layers in the well-fit band). Filtered variants focus on 2≤α≤6.

Avg D/Dks (Bar) vs Base

COMPARE2BASE_AVG_DKS_BAR_TOOLTIP

Lower is better (more compression/regularization without instability).

Avg (1−RD) (Bar) vs Base

COMPARE2BASE_AVG_RANDOM_DISTANCE_BAR_TOOLTIP

Bars show 1−RD. Higher is better (closer to ideal random-matrix behavior).

Avg Traps (Bar) vs Base

COMPARE2BASE_AVG_TRAPS_BAR_TOOLTIP

Lower is better (fewer overfitting spikes).

Notes & Best Practices

  • Combine signals: α band (2–4), detX≈0, high (1−RD), low KS, and few traps characterizes HTSR-consistent layers.
  • Watch transitions: spikes in flows (KS, traps, scale, detX) often mark layers that deserve ablations or finetune adjustments.
  • Compare vs base: Prefer lower-right in α diagonals, and lower bars (except (1−RD), where higher is better).

Need dedicated expertise?

For hands-on training, fine-tuning, or deeper model diagnostics, reach out to Charles Martin via LinkedIn.


About

The weightwatcher tool has been developed by Calculation Consulting. We provide consulting to companies looking to implement Data Science, Machine Learning, and/or AI solutions. Reach out today to learn how to get started with your own AI project. Email: Info@CalculationConsulting.com Please review our Terms of Service and Privacy Policy.