This study was approved by the Clinical Research Ethics Committee of Shenzhen People's Hospital (Approval No. LL-KY-2025300-01). The requirement for informed consent was waived by the ethics committee because this retrospective study used existing histopathological images and clinical materials without direct patient contact or intervention. All clinical data and histopathological images were de-identified before analysis, and no personally identifiable information was used in this study. All data used in this research were handled in accordance with the ethical standards of the institutional and national research committees.
Framework overview
The overall architecture of the proposed UAD framework, RareCode, is illustrated in Figure 1. The framework employs a parallel dual-branch architecture. For each 512 x 512 input image, non-overlapping 32 x 32 and 64 x 64 patches are extracted as fine- and coarse-scale inputs, respectively. The 64 x 64 patches are resized to 32 x 32 before being passed to the network so that both branches share the same encoder-decoder input size while preserving different receptive fields. Encoders in each branch map the extracted features into a latent space, where the MHC module imposes discretization constraints. Subsequently, decoders reconstruct the image from quantized features to compute spatial-domain reconstruction errors. The CAR mechanism then measures the degree of semantic-level anomaly by profiling codebook activation frequencies. Finally, the model fuses reconstruction scores and CAR scores through a weighted combination to obtain image-level anomaly scores. The RareCode framework is trained end-to-end using only normal samples, requiring no anomaly annotations. The dual-branch design was used to capture both local cellular features and broader glandular structures. The MHC module was used to model these features with codebooks of different capacities.
Encoder-decoder architecture
RareCode constructs structurally independent encoders and decoders for the fine- and coarse-scale branches. Each branch uses the same encoder-decoder design but does not share parameters. The encoder consists of four convolutional blocks, each including a 3 x 3 convolution, batch normalization, ReLU activation, and dropout. The channel width increases from 64 to 128, then to 256, and the final feature map is flattened and projected into a 64-dimensional latent embedding. The decoder mirrors the encoder with a fully connected projection layer followed by four transposed convolution layers, reconstructing each patch to the original input size of the network. Reconstruction error is calculated as the mean squared error between the input and reconstructed patches. By combining this encoder-decoder structure with discrete codebook constraints, RareCode learns compact representations of normal tissue and measures deviations during inference.
Multi-scale hierarchical codebook
To capture pathological features at multiple levels of abstraction—from broad tissue patterns to localized cellular variations—the MHC module employs K discrete codebooks
with varying capacities n1, n2, ..., nK. In the final FourScales configuration, each branch contains four codebooks with 64, 128, 256, and 512 entries, respectively, and each codebook entry has a 64-dimensional embedding. For each encoded patch feature, the nearest codebook entry is selected according to Euclidean distance. The quantized outputs from different codebooks are then fused using learnable weights normalized across codebooks. Smaller codebooks, due to stronger compression constraints, are expected to encode coarse-grained prototypical patterns that abstract away local variations, whereas larger codebooks preserve finer-grained features that capture more specific structural characteristics. For each codebook
, where ek(i)
Rd denotes the i-th embedding vector in the k-th codebook, continuous features are mapped to the discrete codebook space via nearest neighbor lookup (Eq. 1 and 2):


To achieve adaptive fusion of multi-scale features, we introduce learnable weights . After softmax normalization, weighted summation is performed over the quantized outputs from each codebook (Eq. 3):

where σ(·) denotes the softmax function, ensuring that weights satisfy Σk σ(wk) = 1. This design enables the model to automatically adjust the contribution ratios of codebooks at different granularities based on the semantic content of the input features.
To optimize codebook learning, we adopt the standard VQ loss function (Eq. 4):

where sg[·] denotes the stop-gradient operation, and β = 0.25 is the commitment loss weight coefficient. The first term encourages codebook vectors to move toward encoder outputs, while the second term encourages encoder outputs to remain consistent with their corresponding codebook vectors. The complete mechanism of multi-scale VQ is illustrated in Figure 2.
Codebook activation rarity scoring
CAR measures semantic-level anomalies based on codebook activation statistics estimated from normal training samples. After model training, all normal training images were passed through the encoder and MHC module without data augmentation. For each branch and each codebook, the assignment index of every patch feature was recorded, and the number of assignments to each codebook entry was counted. The counts were then normalized to obtain the activation-frequency distribution for that codebook (Eq. 5):

During inference, each test image was processed through the same patch extraction, encoding, and nearest-neighbor codebook assignment procedure. For each assigned codebook entry, the rarity score is calculated as the negative logarithm of its activation frequency in the normal training set (Eq. 6):

where ε is a small constant used for numerical stability. A low-frequency codebook entry, therefore, receives a higher rarity score, suggesting that the corresponding patch feature is less consistent with the learned normal-tissue distribution. Patch-level rarity scores were averaged within each image and across the fine- and coarse-scale branches to obtain an image-level rarity response.
To complement activation-frequency rarity, we also compute a percentile-based quantization distance score (Eq. 7):

This score is derived from the Euclidean distance between each encoded feature vector and its nearest codebook entry. The distance distribution is estimated from normal training samples, and test-sample distances are converted into percentile scores. Larger percentile scores indicate that the encoded feature is more difficult to represent using the learned normal codebook prototypes.
The final CAR score combines activation rarity and quantization distance across all codebooks using the learnable codebook weights (Eq. 8):

The same normalized codebook weights used for multi-scale feature fusion are applied to score-level fusion, maintaining consistency between representation learning and anomaly scoring. The resulting CAR score is calculated at the image level and is then combined with the reconstruction score to obtain the final anomaly score. The overall CAR scoring mechanism is illustrated in Figure 3.
Final anomaly score computation
We fuse spatial-domain reconstruction errors with semantic-level CAR scores. Before fusion, both scores are separately normalized using percentile-based min-max normalization to reduce the influence of extreme outliers. The final image-level anomaly score is then defined as (Eq. 9):

where Srecon denotes the normalized mean squared reconstruction error, reflecting sample reconstruction quality in pixel space; SCAR is the CAR score described above, capturing degrees of semantic-level anomaly. The hyperparameter α
[0,1] controls the relative weighting between the two components, with optimal values determined on validation sets.
Training objective
The RareCode framework is trained end-to-end using only normal samples. The total training loss comprises the following components (Eq. 10):

Each term is defined as follows: Reconstruction Loss:
, measuring the mean squared error between input patches p and reconstructed patches D(zq) after quantization. This loss is applied separately to the 32 × 32 patch branch (
) and 64 × 64 patch branch (
) to ensure effective learning of features at both spatial scales.
Dataset description
To evaluate RareCode, experiments were conducted on a clinically annotated CRC histopathological image dataset comprising images of colorectal tissue sections stained with hematoxylin and eosin (H&E), obtained from Shenzhen People's Hospital. All images in this dataset originate from real clinical cases. Images were digitized at 40x magnification with an original resolution of 1024 × 1024 pixels. To ensure annotation reliability and quality, all sample category labels were jointly reviewed and confirmed by two or more senior professional pathologists following a double-blind protocol.
Based on histopathological characteristics, the dataset was categorized into three classes: normal tissue (1,226 images), cancer (1,215 images), and inflammation (1,214 images). In the binary UAD setting, cancer and inflammation samples were treated as anomalous, while normal samples served as the reference distribution. This formulation reflects the intended triage task of separating cases requiring further pathological review from those that are morphologically normal.
All H&E-stained images were resized to 512 × 512 pixels before being used as network input. Stratified 5-fold cross-validation was used for evaluation. In each fold, the normal samples were first divided into training, validation, and held-out test subsets. RareCode was trained only on the training normal samples. The validation subset, which also contained only normal samples, was used for model selection, hyperparameter tuning, and selection of the fusion weight α. The held-out test set, consisting of unseen normal samples and anomalous samples including cancer and inflammation, was used only for final performance evaluation and was not used for model training, hyperparameter selection, or α selection. The representative categories, dataset structure, and validation protocol are shown in Figure 4.
Implementation details
The proposed RareCode framework and comparison baselines were implemented using PyTorch. All experiments were conducted on a workstation equipped with a single NVIDIA GeForce RTX 4060 Ti GPU (16 GB). For network architecture configuration, to capture features at different spatial extents, the dual-branch architecture processes non-overlapping patches at two scales: 32 × 32 pixels (smaller receptive field, capturing local texture patterns) and 64 × 64 pixels (larger receptive field, capturing broader spatial context), respectively. The continuous feature embedding dimension produced by both branch encoders is uniformly set to 64. In the MHC module, we configure four codebooks with different capacities for each branch, with discrete prototype quantities of 64, 128, 256, and 512, respectively.
During optimization, the network was trained end-to-end for 50 epochs with a batch size of 8. We employ the AdamW optimizer for weight updates, with an initial learning rate of 5 × and weight decay of to prevent overfitting. Additionally, to ensure smooth training convergence, a cosine annealing learning rate schedule was used, enabling finer optimization in later training stages. Patch sizes and codebook sizes were selected to balance multi-scale representation and computational cost. The final FourScales configuration was supported by ablation analysis, and α was selected on the validation set before test-set evaluation.
For the decoder-complexity ablation, a complex-decoder variant was implemented based on the FourScales configuration. This variant used the same data split, codebook sizes, training epochs, optimizer, learning rate, validation-based α selection, and evaluation metrics as the FourScales model. The basic decoder was replaced with an EMCAD-style decoder containing multi-scale depthwise convolution blocks and Convolutional Block Attention Modules (CBAM). After each of the first three transposed-convolution upsampling stages, parallel 3 × 3, 5 × 5, and 7 × 7 depthwise convolutions were applied, followed by 1 × 1 pointwise fusion, batch normalization, ReLU activation, a residual connection, and CBAM attention. CBAM included channel attention based on average- and max-pooling descriptors and spatial attention using a 7 × 7 convolution. The final stage used transposed convolution and sigmoid activation to reconstruct 32 × 32 patches.