Content teams cannot manually re-review thousands of pages, so this project asks a narrow, practical question: given a page's search performance history, can a ranked score tell a content team which pages most urgently need a refresh — and does that score beat a hand-tuned heuristic? The work went through two stages: a rule-based v1 scorer, built first as an interpretable baseline over click-through rate (CTR), impressions, and estimated content age, followed by a v2 supervised model trained on engineered features — CTR trend, ranking volatility, freshness age, and a semantic-drift signal comparing each page's content to current search-intent clusters — using a time-aware train/test split so no future performance leaked into the label. On the held-out window, the v2 model achieved 97% precision at the top-K refresh candidates, substantially outperforming the v1 baseline and confirming that a learned prioritization model can identify the most urgent refresh opportunities far more reliably than the rule-only version. Four patterns held consistently across both stages: CTR decay often hid inside pages with flat or growing impressions, older pages drifted semantically from current search intent, ranking volatility alone was a weaker signal than volatility combined with CTR trend, and content age interacted with CTR decline to concentrate positive labels among older, decaying pages. The output is not a claim about Google's ranking algorithm; it is a decision-support worklist that tells an editorial team where to look first, with a reason attached to every recommendation.
Editorial teams sit on content libraries that grow faster than anyone can re-read them. Deciding what to refresh is usually done by instinct, by whoever last noticed a page sliding, or by a blanket policy like "revisit anything older than a year" — none of which account for how a specific page is actually performing right now. The decision this project supports is a prioritization decision, not a diagnosis: given a limited number of editorial hours this month, which pages should get one first?
I treated that as a two-part problem. First, could a transparent, rule-based score do a reasonable job of ranking pages by refresh urgency, using signals any content team already has — CTR, impressions, and rough content age? Second, could a learned model, trained on the same underlying signals plus a few engineered features, produce a materially better ranking than the rule-based version, while staying interpretable enough that an editor would trust and act on it? Both versions were built and compared on the same data, using the same evaluation window, so the comparison is apples-to-apples.
This project used the FlyRank ML Internship warehouse release, accessed through the gated Hugging Face dataset provided for the program. Analysis draws on page_search_performance, content_metadata, and page_intent_cluster tables, aggregated over an observation window of January 2025 to June 2025, with a later window of July 2025 to September 2025 held out and used only to construct labels and evaluate the model — never for feature engineering, to keep the split honest.
Pages with fewer than 100 impressions in the observation window were excluded, since CTR and ranking metrics get noisy at low volume and would have added label noise rather than signal. Pages missing a valid publish or last-modified timestamp were excluded as well, since content age is a core feature. No client names, source domains, raw exports, credentials, or query-level data appear anywhere in this paper or in the accompanying notebooks; every figure here is aggregated at the page-signal level.
Before touching a model, I built a rule-based scorer, largely because I wanted a baseline I could fully explain to a non-technical stakeholder before trusting anything more opaque. It combined a small set of weighted rules over CTR relative to a page's own historical average, current search volume, and estimated content age, producing a 0–100 priority score per page. This became the baseline every later model had to beat, and it stayed useful on its own as a sanity check whenever the learned model's rankings looked strange.
The supervised model was trained on four groups of engineered features: (1) CTR trend — the change in CTR over the observation window, isolated from changes in impressions; (2) ranking volatility — dispersion of a page's average search position over time, not just its latest position; (3) freshness age — time since last substantive content update; and (4) semantic drift — a distance measure between a page's content and the intent clusters currently associated with its target queries. A page was labeled a positive refresh candidate if CTR fell by more than 20% while impressions stayed flat or increased, or ranking volatility exceeded 2.5 positions in the labeling window, checked against the held-out later window so the label reflects what actually happened next, not just a snapshot.
I compared the v1 rule score against a Random Forest classifier trained on the engineered feature set above. Evaluation used a time-aware split — training on the earlier window and testing on the later one, rather than a random shuffle — since a random split would let the model see performance patterns from "the future" relative to some training rows, which is not a realistic setting for a tool meant to run forward in time. I also ran a leakage check by confirming no feature was computed using any data from inside the labeling window itself. The final model used 400 trees, max depth 10, and minimum leaf size 5 to balance signal capture with generalization.
To make the score usable day to day rather than left in a notebook, I wrapped it behind a small API and a review dashboard so an editor could see each page's score, its top contributing signals, and a suggested action, then approve or reject it before anything got published. A lightweight retrieval component surfaced relevant style-guide and SEO-practice notes alongside each suggestion. These are implementation details that make the score actionable; they are not part of what was evaluated in the Results section below, and none of it changes the underlying claim — the model is a prioritization aid, not an authority on what's actually wrong with a page.
Both scorers were run over the same held-out window and compared on how well each ranking matched pages that genuinely needed attention next.
| Metric | V1 rule baseline | V2 learned model |
|---|---|---|
| Precision @ top-K | 71% | 97% |
| Recall @ top-K | 46% | 81% |
| AUC / ranking quality | 0.71 | 0.89 |
| Held-out window size (pages) | 8,420 | 8,420 |
Pages with stable impressions but declining CTR consistently ranked among the highest predicted refresh opportunities — the kind of decay a glance at impressions alone would miss entirely.
Older articles became semantically distant from the search-intent clusters currently associated with their target queries, even when their rankings hadn't visibly moved yet.
Volatility in average position predicted refresh opportunity better than average rank alone, but was most useful paired with CTR trend rather than read on its own.
Older pages with declining CTR were disproportionately represented among positive labels — age alone wasn't predictive, but age combined with CTR decline was.
Each flagged page produces a short, reviewable recommendation rather than a bare score:
| Title refresh | Yes |
| Add missing topics | LLM-generated entity suggestions |
| Update statistics | Required |
This model describes association, not cause. Nothing here demonstrates how Google's ranking algorithm works, and nothing here proves that refreshing a flagged page will move its ranking — that would require a live experiment this project didn't run. Read every finding above as observed (it showed up consistently in this dataset), not as proven or guaranteed.
An action playbook for a content team using this scoring approach, in priority order:
All notebooks referenced in this paper live in the work/ folder of the project repository, including every assignment notebook and the capstone notebook this paper is drawn from.
The repository's submission/paper_url.txt contains the single deployed URL of this page.