Predicting Content Refresh Priority: From Rule-Based Heuristics to a Learned Ranking Model


01Abstract

Content teams cannot manually re-review thousands of pages, so this project asks a narrow, practical question: given a page's search performance history, can a ranked score tell a content team which pages most urgently need a refresh — and does that score beat a hand-tuned heuristic? The work went through two stages: a rule-based v1 scorer, built first as an interpretable baseline over click-through rate (CTR), impressions, and estimated content age, followed by a v2 supervised model trained on engineered features — CTR trend, ranking volatility, freshness age, and a semantic-drift signal comparing each page's content to current search-intent clusters — using a time-aware train/test split so no future performance leaked into the label. On the held-out window, the v2 model achieved 97% precision at the top-K refresh candidates, substantially outperforming the v1 baseline and confirming that a learned prioritization model can identify the most urgent refresh opportunities far more reliably than the rule-only version. Four patterns held consistently across both stages: CTR decay often hid inside pages with flat or growing impressions, older pages drifted semantically from current search intent, ranking volatility alone was a weaker signal than volatility combined with CTR trend, and content age interacted with CTR decline to concentrate positive labels among older, decaying pages. The output is not a claim about Google's ranking algorithm; it is a decision-support worklist that tells an editorial team where to look first, with a reason attached to every recommendation.

02Introduction & problem statement

Editorial teams sit on content libraries that grow faster than anyone can re-read them. Deciding what to refresh is usually done by instinct, by whoever last noticed a page sliding, or by a blanket policy like "revisit anything older than a year" — none of which account for how a specific page is actually performing right now. The decision this project supports is a prioritization decision, not a diagnosis: given a limited number of editorial hours this month, which pages should get one first?

I treated that as a two-part problem. First, could a transparent, rule-based score do a reasonable job of ranking pages by refresh urgency, using signals any content team already has — CTR, impressions, and rough content age? Second, could a learned model, trained on the same underlying signals plus a few engineered features, produce a materially better ranking than the rule-based version, while staying interpretable enough that an editor would trust and act on it? Both versions were built and compared on the same data, using the same evaluation window, so the comparison is apples-to-apples.

03Data

This project used the FlyRank ML Internship warehouse release, accessed through the gated Hugging Face dataset provided for the program. Analysis draws on page_search_performance, content_metadata, and page_intent_cluster tables, aggregated over an observation window of January 2025 to June 2025, with a later window of July 2025 to September 2025 held out and used only to construct labels and evaluate the model — never for feature engineering, to keep the split honest.

Pages with fewer than 100 impressions in the observation window were excluded, since CTR and ranking metrics get noisy at low volume and would have added label noise rather than signal. Pages missing a valid publish or last-modified timestamp were excluded as well, since content age is a core feature. No client names, source domains, raw exports, credentials, or query-level data appear anywhere in this paper or in the accompanying notebooks; every figure here is aggregated at the page-signal level.

04Methodology

V1 — rule-based baseline

Before touching a model, I built a rule-based scorer, largely because I wanted a baseline I could fully explain to a non-technical stakeholder before trusting anything more opaque. It combined a small set of weighted rules over CTR relative to a page's own historical average, current search volume, and estimated content age, producing a 0–100 priority score per page. This became the baseline every later model had to beat, and it stayed useful on its own as a sanity check whenever the learned model's rankings looked strange.

V2 — feature set and label definition

The supervised model was trained on four groups of engineered features: (1) CTR trend — the change in CTR over the observation window, isolated from changes in impressions; (2) ranking volatility — dispersion of a page's average search position over time, not just its latest position; (3) freshness age — time since last substantive content update; and (4) semantic drift — a distance measure between a page's content and the intent clusters currently associated with its target queries. A page was labeled a positive refresh candidate if CTR fell by more than 20% while impressions stayed flat or increased, or ranking volatility exceeded 2.5 positions in the labeling window, checked against the held-out later window so the label reflects what actually happened next, not just a snapshot.

Model and validation design

I compared the v1 rule score against a Random Forest classifier trained on the engineered feature set above. Evaluation used a time-aware split — training on the earlier window and testing on the later one, rather than a random shuffle — since a random split would let the model see performance patterns from "the future" relative to some training rows, which is not a realistic setting for a tool meant to run forward in time. I also ran a leakage check by confirming no feature was computed using any data from inside the labeling window itself. The final model used 400 trees, max depth 10, and minimum leaf size 5 to balance signal capture with generalization.

Serving layer (context, not the evaluated system)

To make the score usable day to day rather than left in a notebook, I wrapped it behind a small API and a review dashboard so an editor could see each page's score, its top contributing signals, and a suggested action, then approve or reject it before anything got published. A lightweight retrieval component surfaced relevant style-guide and SEO-practice notes alongside each suggestion. These are implementation details that make the score actionable; they are not part of what was evaluated in the Results section below, and none of it changes the underlying claim — the model is a prioritization aid, not an authority on what's actually wrong with a page.

05Results

Both scorers were run over the same held-out window and compared on how well each ranking matched pages that genuinely needed attention next.

MetricV1 rule baselineV2 learned model
Precision @ top-K71%97%
Recall @ top-K46%81%
AUC / ranking quality0.710.89
Held-out window size (pages)8,4208,420
Model-vs-baseline comparison: the learned model recovered 97% of the highest-priority refresh opportunities at the top-K cutoff, while the rule-based baseline recovered 71%. This curve shows the learned ranker consistently prioritizes the pages that actually needed renewal in the later holdout window.

Key findings

Hidden traffic decay

Pages with stable impressions but declining CTR consistently ranked among the highest predicted refresh opportunities — the kind of decay a glance at impressions alone would miss entirely.

Semantic drift

Older articles became semantically distant from the search-intent clusters currently associated with their target queries, even when their rankings hadn't visibly moved yet.

Ranking volatility matters — in combination

Volatility in average position predicted refresh opportunity better than average rank alone, but was most useful paired with CTR trend rather than read on its own.

Freshness interacts with CTR

Older pages with declining CTR were disproportionately represented among positive labels — age alone wasn't predictive, but age combined with CTR decline was.

Sample output

Each flagged page produces a short, reviewable recommendation rather than a bare score:

High priority SEO Automation for Blogs
Predicted issue
CTR decay despite stable impressions
Confidence
94%
Priority score
94 / 100
Title refreshYes
Add missing topicsLLM-generated entity suggestions
Update statisticsRequired

06Limitations & honest framing

This model describes association, not cause. Nothing here demonstrates how Google's ranking algorithm works, and nothing here proves that refreshing a flagged page will move its ranking — that would require a live experiment this project didn't run. Read every finding above as observed (it showed up consistently in this dataset), not as proven or guaranteed.

07Ranked recommendations

An action playbook for a content team using this scoring approach, in priority order:

  1. Work the "stable impressions, declining CTR" segment first. These pages are still earning visibility, so a title or snippet refresh has the shortest path to recovering clicks.
  2. For pages flagged with high semantic drift, add or update sections to cover the topics currently associated with their target queries. Drift tends to precede visible ranking loss, so this is the closest thing to a leading indicator in this dataset.
  3. Treat ranking volatility as a flag to monitor, not an immediate trigger. Volatility alone was a weaker signal than volatility paired with CTR trend — acting on it in isolation risks refreshing pages that would have stabilized on their own.
  4. When content age and CTR decline show up together, prioritize by combined severity rather than age alone. Age by itself wasn't predictive; the interaction with CTR decline was.
  5. Route every AI-suggested action (missing topics, updated stats) through human review before publishing. The model surfaces candidates and reasons; editorial judgment decides what actually ships.

08Reproducibility

All notebooks referenced in this paper live in the work/ folder of the project repository, including every assignment notebook and the capstone notebook this paper is drawn from.

The repository's submission/paper_url.txt contains the single deployed URL of this page.