Hybrid Rank-MMR: Zero-shot Text Summarization via Weighted Feature Fusion based on Pre-trained Language Models

Extractive text summarization

2.1.

Extractive text summarization is the process of selecting the most representative sentences from a source text. Previous proposals relied on statistics or graphs to find important sentences, such as TextRank [8]. This algorithm represents the text as a graph: each sentence is a point (node), and the lines connecting the points represent the degree of similarity between sentences (e.g., TF-IDF score). While it is fast, this method only sees local connections and does not understand the flow of the entire text. To overcome this, feature-based systems [9] were developed. They scored sentences based on position, length, and the number of proper nouns. Thanks to this, the “lead bias” was discovered – that is, the main idea of the article is often found in the first few sentences. However, the weakness of this method is that when moving to a new field, one has to manually find these features.

Later, the explosion of deep learning brought the sequence model into the summarization problem. For example, SummaRuNNer [10] uses a recurrent neural network to score each sentence as keep-or-discard. A major advance came with the introduction of the Transformer architecture [11] and large language models. For example, BERTSum [12] used BERT to read and understand the two-dimensional context of a sentence, then classified whether the sentence was worth selecting. This sentence-by-sentence scoring method yielded very good results.

However, it scores each sentence independently, ignoring inter-sentence dependencies. To address this, MatchSum [5] takes a different angle. Rather than treating summarization as classification, MatchSum frames it as semantic matching: the system no longer scores each sentence in isolation. Instead, it groups sentences into candidate summaries and chooses the candidate whose meaning sits closest to the source text. This keeps the summary coherent and avoids the redundancy that sentence-level scoring tends to produce. Even so, we found that semantic matching alone is not enough on complex datasets, where the semantic signal is weak or runs against the natural structure of the text [13].

Vietnamese low-resource summarization

2.2.

Research on English text summarization has matured over the years. In contrast, Vietnamese is a low-resource language with distinct features, meaning it often faces severe data shortages. However, the field has recently made significant progress thanks to new large-scale datasets.

VNDS [6] is a dataset for single-document text summarization. This dataset has also shown the general writing style of news, where important sentences are usually at the beginning of the source text. Next, ViMs [14] is a dataset for multi-document text summarization. This dataset has a major challenge in information synthesis, for example, how to filter duplicate information between sentences and arrange events in the correct chronological order.

Most recently, AbMuSu [7] is a dataset built for the task of abstract text summarization of product reviews. User comments in this dataset are often disorganized and unstructured. For example, introductory sentences are often off-topic, while the main idea is scattered throughout the paragraph.

Re-ranking and explainability

2.3.

Re-ranking is an important step in improving summarization quality in many NLP systems. Maximal Marginal Relevance (MMR) [15] is a fundamental method for selecting diverse candidates for summarization. It helps balance the topic’s relevance with the sentence’s diversity.

Subsequently, deep learning models such as SummaReranker [16] and SimCSE [17] have significantly improved semantic-based sentence ranking through PLMs. However, most ranking methods select sentences without explaining why.

Explainable artificial intelligence (XAI) addresses this problem [18]. Traditional XAI tools such as LIME [19] and SHAP [20] are designed for classification tasks and are therefore not well-suited for text generation. Instead, recent studies have shown that large language models (LLMs) can perform well for evaluation [21] and inference [22].

Therefore, we use chain-of-thought prompting with LLMs to explain why certain sentences in the summary were selected. This approach increases user confidence in the model’s effectiveness.

Sustainable summarization

2.4.

Currently, large AI models require significant computational resources. NLP researchers are concerned about the negative environmental impact of their work. Therefore, some recent studies [23] have begun to call for a shift toward more energy-efficient designs, also known as the “Green AI” trend.

Discussions [24] show that although large language models (LLMs) can generate high-quality abstractive summaries, the energy they consume per run is many times higher than that of standard extractive models. This has led to a renewed interest in hybrid model architectures: using a compact model to select the main content first, and then using a large model to refine it at the final step [25,26].

Our research follows this direction by combining a BERT encoder with a feature-based re-ranking mechanism. This architecture offers a more computationally efficient alternative to end-to-end LLM-based summarization pipelines.

Candidate generation and pruning

3.1.

A core difficulty in extractive multi-document summarization is the combinatorial search space. Given a document cluster with $M$ sentences, selecting a summary of $k$ sentences induces $\binom{M}{k}$ possible candidates. For typical values (e.g., $M = 50$ and $k = 3$), this already yields over 19,600 combinations. To enable efficient decoding under a strict training-free setting, we adopt a coarse-to-fine strategy that prunes sentences using only frozen pre-trained representations. Concretely, we reuse the shared encoder $\Phi$ to pre-compute: ($i$) a sentence embedding $h_{s_{i}}$ for each sentence $s_{i}$, and (ii) a document-cluster embedding $h_{D}$ representing the overall semantic context. We then define a sentence-level salience proxy by cosine similarity (we use Cos(.) denote the cosine similarity function):

Intuitively, $\tilde{p}\left( s_{i} \right)$ measures how well a sentence aligns with the global topic of the cluster in the embedding space, providing a lightweight relevance signal without any supervision. We sort all sentences by $\tilde{p}\left( s_{i} \right)$ and retain the top $K\prime$ sentences, yielding a pruned sentence set

From $S_{pruned}$, we generate the candidate summary pool by enumerating all $k$-sentence combinations:

where each candidate $C_{j}\left( j = \overline{1,P} \right)$ is a set of $k$ sentences. This pruning step reduces the candidate space by orders of magnitude while remaining strictly training-free: we do not train an extractor, and we do not fine-tune the encoder.

Throughout the paper, we use $k = 3$ as the summary budget and set $K\prime = 15$ as the pruning threshold. The choice of $K\prime = 15$ balances computational efficiency against oracle coverage: with $K\prime = 15$, we retain $\binom{15}{3} = 455$ candidates. Empirical analysis indicates that this threshold retains over 90% of oracle-optimal sentences, while $K\prime \in \lbrack 12,20\rbrack$ yields comparable performance.

Semantic matching

3.2.

Once the candidate pool $C$ is constructed, we score each candidate summary against the document cluster in a shared embedding space. This follows a matching-based view of summarization popularized by MatchSum [5]: a high-quality extractive summary should be semantically close to its source documents while being substantially shorter.

Feature-aware re-ranking

3.3.

While $S_{sem}$ is very effective, it has a weakness: it ignores sentence position (it considers the whole text as just a bunch of unordered sentences). To address this limitation, we incorporate additional discourse-aware features.

MMR-based diversity selection

3.4.

The hybrid score finds salient sentences but says nothing about redundancy. Source documents in summarization often overlap, and a top relevance ranking can pick three sentences saying almost the same thing. With a 3-sentence budget, that means up to two-thirds of the summary repeats itself. The classic Maximal Marginal Relevance (MMR) framework [15] handles this: it adds a penalty that discounts candidates similar to what is already in the summary, mixing relevance with novelty.

Explainable AI

3.5.

To address transparency concerns inherent in neural systems, we generate post-hoc rationales for summary selection. We formulate this as a comparative verification task: the language model is prompted not to generate summaries (an abstractive task) but to justify the extractive model’s selection relative to a base model (e.g., Lead-3). The rationale generation procedure is formalized in Algorithm 3. This approach enables users to verify whether high scores correlate with actual information gain.

Datasets

4.1.

We evaluate on seven benchmark datasets spanning two languages.Table 1 summarizes the statistics.

CNN-DailyMail [27] is a single-document benchmark with strong lead bias. Multi-News [28] groups multiple articles into one summary, while WCEP [29] gathers short event summaries from Wikipedia. Scientific papers are the focus of XScience [30]. On the Vietnamese side, VNDS [6] is a news corpus that also leans on position; ViMs [14] covers multi-document Vietnamese news. AbMuSu [7] is the most loosely structured of the seven - product reviews with minimal positional cues.

All parameter estimations ($\alpha$, $\beta$, $\gamma$) are computed solely from source document statistics - specifically, sentence embeddings and their pairwise similarities. We do not use reference summaries for parameter tuning, nor do we perform grid search on validation sets. This design ensures that the framework can be applied to new datasets immediately without any labeled data.

Vietnamese preprocessing. For Vietnamese datasets, we apply word segmentation using underthesea¹ before encoding with PhoBERT, following the standard preprocessing pipeline for Vietnamese NLP. Sentence boundaries are detected using rule-based heuristics.

Basic models and implementation

4.2.

We implement two basic models: Lead-3 selects the first three sentences; MatchSum (ZS) applies the matching intuition of MatchSum but without fine-tuning, we only compute cosine similarity in the shared embedding space using a frozen BERT/PhoBERT encoder.

This revision adds six stronger baselines. Two are graph-based, TextRank [8] and LexRank [31]. One is the BERT-based zero-shot extractor BERTSum-ZS [12]. PacSum [32] is an unsupervised position-aware ranker. The last two are large language models: Gemini 2.5 Flash (commercial) and Qwen 2.5-7B (open-weights). None of these methods sees a labelled summary at any point, so the comparison stays apples-to-apples with our training-free pipeline.

We use a fixed extractive budget of $k = 3$ sentences for controlled comparison. Parameters $\alpha$, $\beta$, and $\gamma$ are estimated automatically from corpus statistics as described in Section 3.3.3. Evaluation employs ROUGE scores [33].

Main results

5.1.

Table 2 andTable 3 compare our method against the two basic models. We present the incremental contribution of our Hybrid scoring mechanism and the full system with adaptive MMR.

Performance analysis

5.2.

The results show improvements across all seven datasets. The ROUGE-1 (R-1) scores increase by +1.29 to +6.68 compared to the best baseline model.

To put these gains in context, a one-point ROUGE-1 increase corresponds to recovering roughly one additional informative unigram per reference tokens, which qualitatively manifests as better coverage of named entities, dates and key facts in the produced summaries (the case study in Section 6.4 makes this concrete on a CNN-DM article).

Ablation study

5.3.

To understand the effects of each component in our method, we gradually remove features from the Multi-News dataset (Table 5). The results show three findings: First, on this data set, the centrality score is three times more influential than the location score (+1.14 compared to +0.38 on R-1). This fits perfectly with our formula, automatically assigning a very high $\beta$ weight (0.90). Second, combining features achieves significant results. The total increase when combined (+1.77 on R-1) is even higher than the simple addition of each part (+1.52 on R-1). This shows that location and concentration complement each other and do not overlap. Third, adding MMR increases R-1 by +0.79, showing that even with good rankings, we still need to force the model to choose more diverse sentences.

Parameter sensitivity analysis

5.4.

A key property of the proposed method is the automatic calculation of parameters from data without manual adjustment.Table 6 shows that these numbers clearly represent the true nature of the text.

For newspaper data (CNN-DM, VNDS), the weight $\alpha$ is much larger than $\beta$. This is consistent with newspaper writing habits: the opening sentence summarizes the entire article. On the contrary, with unstructured or multi-document data, $\beta$ is larger than $\alpha$. Because it cannot rely on location, the system must rely on semantic focus to find the main idea. On the WCEP data set, the parameters are in the middle ($\alpha$=0.47, $\beta$=0.53). This can be explained by the fact that Wikipedia event summaries are both clearly structured and come from multiple sources, so both signals are equally important.

Validation of parameter estimation

5.5.

To verify the formula for automatically calculating parameters, we compare it with the grid search method, which scans all values from 0.0 to 1.0 for $\alpha$ and $\beta$. This is shown inFig. 2.

Fig. 2. Grid search validation for formula-estimated parameters. Top: CNN-DM. Bottom: ViMs. Circles indicate formula estimates

With CNN-Daily Mail, the formula automatically calculates $\alpha$=0.83, $\beta$=0.17 (reaching 42.68 on R-1). The optimal level of exhaustive scanning is $\alpha$=0.8, $\beta$=0.2 (reaching 42.70 on R-1). A difference of 0.02 is completely insignificant. Similar to ViMs, the formula yields $\alpha$=0.08, $\beta$=0.84 (35.93 on R-1), very consistent with the optimal level of exhaustion of $\alpha$=0.1, $\beta$=0.9 (35.95 on R-1). These results suggest that automatic estimation performs comparably to grid search, while reducing the parameter selection time from $O\left( n^{2} \right)$ to just $O(1)$.

Fig. 3 also verifies the accuracy of the diversity parameter $\gamma$. For example, on CNN-DM, the formula yields $\gamma$=0.51 (42.90 on R-1), which is only 0.01 points away from the result from the grid search. On the ViMs set, the level $\gamma$=0.48 calculated by the formula also matches the actual optimal result.

Fig. 3. Grid search validation for γ\gamma on CNN-DM and ViMs. Stars indicate grid search maxima; the circles indicate formula estimates

Efficiency analysis

5.6.

Section 2.4 raised the compute-cost question. We measured per-sample inference time on a single NVIDIA RTX 3090, 24 GB. The seven datasets all run on the same hardware. Each reported number is the mean number of sentences in documents.

One encoder per language does the work — BERT-Base at 110M parameters for English, PhoBERT at 135M parameters for Vietnamese. Position and centrality come from closed-form scalar functions of the same embedding matrix. No extra weights are learned, and no second model is loaded.

Fig. 4 plots ROUGE-1 against per-sample latency. Hybrid Rank-MMR sits on the upper-left of the Pareto frontier: higher mean ROUGE-1 than the LLM baselines (Gemini 2.5 Flash, Qwen 2.5-7B), and latency in the same band as classical encoder methods. Training cost is zero - no fine-tuning anywhere - against roughly 20 GPU-hours for supervised BERTSum [12]. Few methods in the benchmark match this combination of speed, quality, and no training time.

Fig. 4. Speed-quality Pareto. Mean ROUGE-1 across the seven datasets versus per-sample inference time (log scale). Hybrid Rank-MMR (red, ours) lies above and to the left of the LLM baselines and overlaps the latency range of the classical encoder methods

Feature weight interpretation

6.1.

The automatically estimated parameters accurately reflect the nature of each data set. Instead of being randomly chosen numbers, they are actually measures that help us analyze the text’s structure.

Adaptive diversity analysis

6.2.

The parameter $\gamma$ is used to balance between choosing sentences that are on topic and choosing diverse sentences. The optimal $\gamma$ value depends greatly on the homogeneity of the dataset. Our automatic formulation captured this association through the internal similarity score $\bar{\rho}$. In homogeneous data sets, sentences often have similar meanings ($\bar{\rho}$ is high). If you penalize too many duplicates, the model will eliminate much useful content because the article doesn’t have enough distinct ideas to keep. A typical example is the VNDS set ($\bar{\rho}$ = 0.71, $\gamma$ = 0.49): Vietnamese articles on the same topic often have many overlapping sentences, so applying a moderate diversity penalty is enough. On the contrary, unstructured datasets pose a different problem. AbMuSu ($\bar{\rho}$ = 0.38, $\gamma$ = 0.65) contains user comments with a variety of opinions, products, and styles. Without a strong duplicate penalty, the model can easily choose all the criticism and forget the praise (or vice versa). Therefore, pushing $\gamma$ higher will help the summary cover a broader perspective.

The problem of multi-document summarization is different. The ViMs set ($\gamma$ = 0.48) contains many articles about the same event, so the information is often repeated with minor changes. A moderate $\gamma$ value here helps balance two requirements: avoiding duplication of information between articles and retaining supporting details to make the summary richer. This automatic adjustment removes the need for manual parameter tuning while maintaining strong performance.

Error analysis

6.3.

Although the performance has clearly improved, when analyzing cases where the model runs incorrectly, we find that the extraction method still has certain weaknesses. First is the limit on sentence length. If the reference summary is only 10 words, but the sentence containing the shortest important information is 25 words long, the ROUGE score will be dragged down, no matter how good the sentence is. Second, there is a disadvantage in the preprocessing step. In Vietnamese data, we discovered many errors stemming from incorrect sentence segmentation. For example, in the ViMs collection, sometimes the article title is attached to the body of the article. This creates falsely important sentences that have high position scores but are not the summary sentence sought. For low-resource languages, focusing on considering preprocessing methods is also necessary to increase model efficiency. Third is when concentration is not effective. We hypothesize that a sentence with a high semantic level is a good sentence. However, this assumption is sometimes not appropriate. In some samples in the ViMs set, the label summary appears as distinct sentences rather than a general meaning sentence.

Qualitative analysis: XAI case study

6.4.

The case study draws on one real CNN-DM article. The 15-sentence story is a road accident in Nottingham: a £80,000 Porsche 911 was crushed by a builder’s lorry while both pulled aside for emergency services responding to a separate cyclist injury. The 3-sentence reference summary names three facts: the Porsche-lorry collision, the cause of the queue, and an onlooker’s quote about the driver.

Hybrid Rank-MMR picks sentences 0, 2, 3 - one for each reference fact. Sentence 0 introduces the incident; sentence 2 explains the cyclist-traffic context; sentence 3 reports the onlooker quote. MatchSum (ZS) goes 0, 1, 5 instead. Sentence 1 repeats the lead with different wording; sentence 5 is a bystander aside that the reference never mentions. Hybrid Rank-MMR covers all three reference facts, MatchSum (ZS) covers only one.Table 7 lays out the selection grid.Table 8 reports the feature z-scores for our three picks.

Sentences 0 and 3 come from complementary signals. Sentence 0 leads on position and semantics ($z_{pos}$ = +2.31, $z_{sem}$ = +1.61). Sentence 3 leads on centrality ($z_{cen}$ = +1.44). Sentence 2 is the interesting case. Alone, its semantic z-score is negative - the sentence looks less cluster-central than the lead. The MMR step changes that. Once sentence 0 enters the summary, the diversity term penalises near-duplicates of the collision fact, and sentence 2 - the cyclist-traffic explanation - wins the next slot because nothing in the running summary covers that ground yet. MatchSum (ZS) omits this step and reverts to the lead-style sentence 1. This is the feature-fusion and MMR behaviour described in Sections 3.3 and 3.4, in a single sample.

Quantitative XAI evaluation

6.5.

Section 6.4 shows what the explanation module does on a single sample. It does not show whether the rationales are faithful or whether they agree with off-the-shelf saliency methods. We add three quantitative checks on the dataset. The deletion protocol follows [34].

First, faithfulness via deletion (Fig. 5). We remove the $K$=3 sentences picked by the rationale and re-rank what remains. The drop in ROUGE-1 tells us how much those sentences mattered. Hybrid Rank-MMR’s rationale gives a positive drop on every dataset; a random-sentence control gives a negative drop on every dataset. The picks are therefore not arbitrary.

Fig. 5. Faithfulness via deletion. ROUGE-1 drop after removing the kk=3 sentences identified by Hybrid Rank-MMR’s rationale (red) versus a random-control rationale (grey), per dataset. Positive drops mean the rationale identifies critical content

Second, plausibility against LIME-style saliency. We run leave-one-sentence-out perturbation in the style of [29], then measure Jaccard agreement between LIME’s top-k and each method’s $k$=3 picks. Hybrid Rank-MMR beats MatchSum (ZS) on five of the seven datasets. On the other two, the gap is under 0.02. Absolute Jaccard stays in the 0.06-0.18 range, which is what we expect when top-k extractive selections compete.

Third, head-to-head preference from an external judge. We compare Hybrid Rank-MMR against MatchSum (ZS) pairwise, with the two summary positions swapped between runs to absorb order bias. Gemini 3 Flash acts as judge. Hybrid Rank-MMR is preferred on the multi-document and Vietnamese sets - Multi-News 51.5%, WCEP 53.0%, VNDS 52.5%, ViMs 57%, AbMuSu 55%, which are illustrated inTable 9 below.