Every model in the dashboard traces back to one of four training recipes. This page is the exact hyperparameters, data splits, and loss formulations we used — read it alongside the data pipeline and the evaluation methodology.
Note: the T5 family was trained on a separate stratified split with its own pipeline (Camille's CS4248-project-AY2526S2 repo). Same epochs and LR as the BART pipeline but different batch shape, max length, and best-metric criterion.
BART-Base and BART-CE trained with cross-entropy on the sar-to-non splits. 5 epochs, early stopping on val BLEU, HuggingFace Seq2SeqTrainer.
T5-Joint, T5-Control, and the 6 ablations — 4 epochs, effective batch 16, max length 1248, fp16, eval_loss as best metric, SLURM orchestration.
Takes an SFT BART checkpoint as policy, uses a sarcasm classifier + ROUGE-L as reward, KL penalty against the frozen reference.
LLaMA 3.2 1B fine-tuned via low-rank adapters on 7 projection layers. Loss masked to the assistant response only.
Yang Zhi's BART pipeline — cross-entropy training with the HuggingFace Seq2SeqTrainer, early stopping on validation BLEU (patience 2), and a cosine warmup schedule. BART doesn't need a task prefix: it's pretrained with its own denoising objective, so the input is the raw sarcastic headline.
facebook/bart-base (140M)
Plain supervised fine-tuning on the 10,868 headline→rewrite pairs from the main sar-to-non split. Input is the raw sarcastic headline with no prefix — BART is pretrained with its own denoising objective and doesn't expect a task token.
facebook/bart-base (140M)
Same BART architecture and hyperparameters as BART-Base, but trained on the context-enhanced data split: each training pair is conditioned on the scraped article body in addition to the headline. Smaller split (8,258 pairs) because not every headline has a scrape-able article body.
Camille's T5 pipeline runs in a separate repo with its own stratified split, SLURM orchestration, and a longer max-sequence length to fit the combined input+target with the strategy token. Same HuggingFace Seq2SeqTrainer as the BART side, but different hyperparameters — the table below is the shared recipe; each model card then notes where it differs.
google-t5/t5-base (220M)
T5-base trained to do two things at once on every example: classify the sarcasm strategy AND rewrite the headline. The input is prefixed with 'rewrite to non-sarcastic and predict strategy: ' and the target is 'strategy: <type> rewrite: <headline>'. Forcing the model to emit the strategy token before the rewrite makes it decompose the task (identify what's sarcastic first, then remove it) and is why T5-Joint wins human eval on meaning preservation — 16.4% meaning change vs T5-Control's 25%.
google-t5/t5-base (220M)
Same T5 recipe and split as T5-Joint but the strategy token is stripped from both input and output — the model only sees 'rewrite to non-sarcastic: {sarcastic}' and outputs the plain rewrite. This isolates the contribution of the strategy-prefix trick: any difference on meaning preservation between T5-Joint and T5-Control is attributable to the joint objective, not data or backbone.
google-t5/t5-small (60M)
Earlier T5-small variant of the joint model using the default model flag in Camille's slurm script. Same training recipe as T5-Joint, smaller backbone — kept in the evaluation to show that the joint model's edge comes from the strategy prefix, not just capacity. Still listed in the dashboard under the original name 'joint'.
REINFORCE with a KL penalty against the frozen SFT reference — the recipe ViSP (arxiv 2507.09482) used for sarcasm generation, inverted here for sarcasm removal. Pure style reward saturates instantly because the SFT outputs already score ~1.0 on the classifier, so we blend in a ROUGE-L content-preservation term to keep the policy from collapsing to “delete everything”.
BART-Base SFT checkpoint
Takes the BART-Base SFT checkpoint as both policy and frozen reference. Generates outputs via sampling, scores them with a sarcasm classifier, and updates the policy with REINFORCE + a KL penalty to stop it drifting. The reward is a weighted sum of style (classifier) and content preservation (ROUGE-L against the reference).
BART-CE SFT checkpoint
Same RL recipe as BART-RL, but starts from the context-enhanced SFT checkpoint instead of the plain one. Intended to stack the benefits of article-aware supervision with the reward-driven polish.
Human eval shows BART-RL has a 40.7% meaning-change rate — more than double T5-Joint's 16.4%. The model learned that deleting sarcastic tokens reduces P(sarcastic) while still preserving enough ROUGE-L overlap to satisfy the content reward. Deletion optimizes the composite reward without actually rewriting. We document this in detail on the eval page.
LLaMA 3.2 1B is a decoder-only chat model, so the recipe looks nothing like the seq2seq pipeline. We use PEFT LoRA adapters on every attention + MLP projection, keeping the base weights frozen, and mask the loss to the assistant response. The system prompt and user message are encoded with the Llama 3 chat template so the model stays aligned with its instruction-tuned prior.
“You are a writing assistant. Rewrite sarcastic news headlines as neutral, factual equivalents that preserve the core meaning without irony or mockery. Respond with only the rewritten headline, no explanation.”
meta-llama/Llama-3.2-1B-Instruct
Instruction-tuned LLaMA fine-tuned via LoRA adapters — only ~6M of the 1.24B parameters are trained. Uses the chat template with a system prompt, loss is masked to the assistant response only (the prompt tokens are set to −100). The SFT checkpoint is then merged back into the base model and exported to GGUF so LMStudio can serve it on consumer hardware.
meta-llama/Llama-3.2-1B-Instruct
Same LoRA recipe as the base LLaMA variant, but the user message includes the scraped article body alongside the headline. This tests whether the model can use the article to ground its rewrite in the real event. Bumping max_length to 1024 to fit the article forces a smaller batch size.
Six retrainings of the T5 control recipe (not joint — the ablations use the plain rewrite to non-sarcastic: prefix and emit the plain rewrite), each with one sarcasm subtype dropped from the training data. To keep effective dataset size constant across the six variants, every ablation pool is stratified- sampled down to the minimum across all drops. The finding on the dashboard: the six ablations cluster within 0.005 of each other on similarity — the model learns generic sarcasm patterns that transfer across subtypes, so no single one is load-bearing.