Building Segmentation Refinement via Local Rank-Based Calibration and Graph Cut

Abstract

Building segmentation in dense urban scenes remains challenging due to frequent under-segmentation, where adjacent buildings are erroneously merged into single objects. Conventional thresholding, morphological filtering, or context-based optimization methods such as Conditional Random Fields (CRFs) and Graph Cut alleviate this issue only partially, as soft boundaries between closely located buildings often lead to incorrect merging. To address this limitation, we propose a novel post-processing framework termed Local Rank-Based Calibration and Graph Cut for building segmentation refinement. The method introduces a percentile rank-based calibration applied to the softmax probability map before binarization. Instead of treating each pixel's absolute probability as a decision boundary, our calibration reinterprets the probability as a relative percentile within a local sliding window. Low-percentile pixels are penalized via an exponential weighting function, thereby suppressing spurious foreground responses in narrow gaps between buildings, while high-percentile pixels retain their original values. The calibrated map is subsequently refined using a Graph Cut optimization, which balances unary terms from the calibrated probabilities with pairwise smoothness terms to enforce globally consistent segmentation. Experiments were conducted on high-resolution aerial orthoimages of Suseo, Seoul, using building footprints from the National Geographic Information Institute (NGII) as ground truth. SegFormer was adopted as the baseline segmentation backbone, and multiple post-processing strategies were compared under identical conditions. Quantitative results show that our method reduces the under-segmentation rate from 28.47% (baseline) to 15.33%, while maintaining the lowest over-segmentation rate (0.49%) among all tested methods. Pixel-level metrics also improved, with Intersection over Union (IoU) reaching 0.8012 and F1-score 0.8896. Visual comparisons confirm that the proposed Local Rank-Based Calibration and Graph Cut method effectively separates adjacent buildings while preserving the continuity of individual building interiors. Sensitivity analysis further demonstrates the robustness of the method across a reasonable range of parameter values. Although the Graph Cutstep increases computational cost(2 hours 28 minutes for full-scene processing without GPU acceleration), the accuracy gains are significant for applications requiring reliable building-level delineation. The modular design allows our approach to be seamlessly integrated with various backbone models and combined with other post-processing methods such as morphology or CRF. In summary, this study presents a lightweight yet effective refinement strategy that substantially improves building boundary delineation in dense urban imagery. By combining local rank-based calibration with global Graph Cut optimization, the proposed method offers a generalizable and transferable solution for enhancing building segmentation in remote sensing applications.

1. Introduction

A central challenge in computer vision is scene understanding. Scene understanding refers to the process of discerning what objects exist in a given image and understanding their structure and relationships. This process begins by interpreting the meaning of each pixel, which lays the foundation for recognition and inference by organizing the scene into structured information. When perceiving a scene, humans infer the meaning of each pixel by synthesizing its surrounding context. This cognitive process allows for a comprehensive understanding of the entire scene. Computer vision has also evolved based on this approach, and detailed pixel-level interpretation plays a vital role in numerous fields, such as remote sensing and autonomous driving. Recently, deep learning-based segmentation models have significantly improved the accuracy of image understanding by automating pixel-level interpretation with vast training data. The segmentation technique, which enables object-level interpretation by dividing a scene into continuous regions, is now one of the most widely used methods in computer vision. A prominent application is building segmentation—the task of extracting individual buildings from aerial or drone imagery—where deep learning models can identify buildings rapidly and accurately even in complex urban scenes. However, in real-world urban areas, buildings are often situated in very close proximity to one another. This leads to a problem where adjacent buildings are not correctly separated as independent objects in segmentation results, but are instead merged into a single entity. Some pixels within the narrow gaps between buildings are easily misclassified by the model as belonging to a building rather than the background, because the majority of their surroundings are part of the building class. As a result, originally separate buildings become connected as one identical object in the segmentation mask. Although deep learning segmentation models generally demonstrate high performance in overall pixel accuracy, this phenomenon remains a frequent issue in urban imagery.

The objective of building segmentation is not merely to identify the presence of buildings, but to accurately reproduce the precise boundaries and shapes of each individual structure. Consequently, prior research has consistently emphasized the importance of correctly separating adjacent buildings into distinct objects. Zhao et al. (2018) reported that an instance segmentation approach is effective for separating closely-spaced buildings in urban areas. Larionov et al. (2020) highlighted the issue of nearby buildings merging into a single object in satellite imagery and proposed an auxiliary technique to explicitly delineate boundaries across narrow gaps. Such studies have explored various methods to mitigate the problem of buildings failing to be segmented as individual entities. More recently, the impact of this building merging phenomenon on the interpretation of results has been quantitatively analyzed. Chen et al. (2023) observed a tendency for adjacent buildings to merge into a single object in satellite imagery of high-density regions. As a result, buildings that should be recognized as individual instances are instead represented as a single, connected area in the predicted mask, leading to the detection of fewer buildings than are actually present. Crucially, this merging error may not be apparent when relying solely on pixel-level accuracy metrics. Myint et al. (2011) noted that when buildings are predicted as a large merged cluster, minor pixel errors along the boundary do not significantly impact overall accuracy metrics like Intersection over Union (IoU), potentially resulting in deceptively high scores. Therefore, enhancing the object separation capabilities of building segmentation models is an essential task for reliably identifying individual buildings in practice, going beyond merely improving performance metrics.

Various post-processing techniques and model design strategies have been proposed to address the erroneous segmentation of adjacent objects. One approach involves applying additional processing steps to the binarized segmentation mask in order to separate merged objects and refine their boundaries. Methods utilizing morphological operations improve results by removing small noise and refining boundaries in the prediction mask with pre-defined structural elements (Haralick et al., 1987). This is useful for rapidly eliminating spurious pixels or smoothing edges, and it has been reported that applying morphological operations to deep learning building segmentation results enhances performance (Neto and Dantas, 2024). Superpixel-based post-processing methods have also been proposed. The superpixel technique improves Boundary Adherence and visual consistency by clustering adjacent, similarly colored pixels into small, homogeneous regions. Efficient superpixel algorithms such as SLIC offer high computational efficiency, as they perform segmentation based on a simple distance metric within a constrained search space. These generated homogeneous superpixel units can be used to fill small holes and more precisely refine the results along the boundaries of the segmentation output. Achanta et al. (2012) reported that the SLIC-based superpixel method exhibits superior Boundary Adherence and computational speed compared to other conventional methods.

However, generating excessively large superpixels can lead to an over-simplification of the image’s fine details. This limitation eliminates narrow gaps that originally existed, causing adjacent objects to be merged. Watershed segmentation maps the pixel intensities of an image to topographical elevations to identify boundaries corresponding to ridges, and separates adjacent objects by expanding regions along these boundaries (Vincent and Soille, 1991). The marker-controlled Watershed, in particular, allows for more stable segmentation by pre-defining markers for the objects of interest, thereby preventing excessive over-segmentation. However, the final quality of this method is not only highly dependent on the accuracy of the user-defined markers but is also sensitive to factors such as shadows and surface reflections. Wu and Li (2022) noted that these weaknesses of the traditional Watershed algorithm make accurate segmentation in urban imagery challenging. Post-processing techniques applied to binarized segmentation outputs are advantageous as they are simple to implement directly on the resulting mask and do not require modification of the original segmentation process. However, because these operations are confined to the final segmentation mask, they struggle to fundamentally resolve the under-segmentation that occurs in the narrow gaps between buildings. Furthermore, the quality of the results is sensitive to parameter settings due to the inherent trade-off between noise removal and boundary preservation.

Furthermore, research has been actively pursued to enhance the boundary refinement capabilities of segmentation models. Approaches include adding a boundary branch to the model’s encoder-decoder architecture or applying a boundary-aware loss function that assigns greater weight to errors near the boundary. Additionally, methods have been proposed that leverage various multi-scale receptive-field techniques to design models capable of clearly distinguishing between boundaries and object interiors during training. Wang et al. (2025b) proposed the BEMS-UNetFormer model, which introduces a Boundary Awareness module to extract object contours from low-level features and a Boundary-Guided Fusion module to emphasize boundary information when integrating prediction results. This model reduced errors near the boundary and achieved a more distinct representation of boundaries in the prediction mask, thereby improving overall IoU performance. In another work, Wang et al. (2025a) proposed SAIP-Net, which combines Adaptive Frequency Filtering with Multi-scale Receptive Field Enhancement techniques. This model demonstrated superior segmentation performance over existing models by providing a sharper boundary representation while maintaining intra-class consistency.

In a related approach, Ma et al. (2023) proposed the Local and Global Class-Aware Network (LoGCAN) model, which hierarchically integrates global and local feature representations. This model sought to resolve the issue of blurred boundaries by reducing the misclassification of background pixels as foreground near boundaries and enhancing intra-class consistency. Experiments confirmed that the integration of global-local context reduced errors around the boundary and produced sharper boundary delineation. By learning to explicitly define object boundaries during the prediction phase, boundary-enhancement models can improve segmentation quality and object detection accuracy without relying on post-processing. Furthermore, they exhibit relatively stable generalization performance across diverse scenes. Despite these advances, boundary identification remains a challenge for even state-of-the-art models in scenarios with extremely narrow gaps between objects. When adjacent objects share similar colors, approaches based solely on edge detectors or Gaussian smoothing are often insufficient to clearly distinguish the boundary.

Meanwhile, Conditional Random Field (CRF) and Graph Cut algorithms have been widely used as traditional approaches to binarize segmentation results by leveraging spatial context, aiming to improve boundary accuracy. Both methods typically use per-pixel predictions as a unary term and incorporate a pairwise term that encourages adjacent pixels to take the same label, thereby ensuring local consistency in labeling decisions. In a grid-based CRF, for instance, pairs of neighboring pixels (4-or 8-connected) are assigned an interaction term, typically expressed in the form of a Potts model. This imposes a penalty when pixel labels change abruptly, helping to suppress noise and indistinct boundaries in the segmentation mask. A Fully-Connected CRF connects all pairs of pixels, leveraging long-range dependencies through Gaussian kernels and mean-field approximate inference (Krähenbühl and Koltun, 2011). Chen et al. (2018) demonstrated that applying a Fully-Connected CRF significantly improves segmentation accuracy along boundaries. Graph Cut represents pixels as a graph with source and sink nodes and globally optimizes an energy function by finding the minimum cut. It transforms the pixel labeling task into an energy optimization problem, minimizing a cost function that simultaneously considers both unary and pairwise terms. This process yields an optimal solution where boundaries are smooth and contiguous, and region interiors remain consistent.

Boykov and Jolly (2001) demonstrated that Graph Cut provides a globally optimal solution that balances the trade-off between a data term and a boundary term. These spatial context-based binarization methods move beyond the assumption of independent pixel classification, incorporating adjacency and positional similarity to produce more reliable labeling near boundaries. When the initial softmax output produces indistinct boundaries, applying CRF or Graph Cut can partially restore these faint edges, thereby improving overall boundary accuracy. However, if the pixel values in the gap between adjacent objects are not substantially different from those of the object interiors, the unary term can easily cause these pixels to be absorbed into one of the objects. When the smoothing effect of the pairwise term is added, this causes the narrow gap region to be merged into a single object. Consequently, achieving robust separation of closely spaced objects remains challenging when relying solely on traditional CRF or Graph Cut methods.

Existing boundary enhancement models and various binarization techniques still exhibit limitations in separating adjacent buildings. In such regions, the softmax probabilities in building interiors and the intervening gaps are often nearly indistinguishable, leading to the erroneous merging of two objects. When the absolute difference between these probabilities is small, a simple comparison is insufficient to guarantee consistent separation. To overcome these limitations and strengthen the delineation of boundaries between adjacent objects, this study proposes a novel procedure that calibrates the softmax map prior to binarization by reinterpreting its absolute values as percentile ranks within a local distribution. Instead of treating each pixel’s probability as an absolute magnitude, our approach evaluates it based on its relative position within a local window, thereby mitigating the problem of pixels in narrow gaps being misclassified as foreground and merging with adjacent objects. Specifically, the distribution within the window is examined: if a pixel falls within a lower percentile, its value is penalized, whereas if it falls within an upper percentile, its value is preserved, thereby adjusting the unary map. This calibrated map is then provided to a Graph Cut optimization, ensuring that the effect of the Local Rank-Based Prior Calibration is incorporated into the global optimization process. As a result, boundaries are accurately separated even in narrow gaps between adjacent buildings, yielding segmentation outputs where individual structures are clearly delineated. The proposed method does not require modifications to the model architecture or training process and can be applied as an additional module to the softmax output stage.

Therefore, it can be universally applied regardless of the segmentation model’s backbone or loss function, making it easy to integrate into and maintain within existing systems. Furthermore, it can be used in tandem with other post-processing techniques such as morphological operations or CRFs, providing significant flexibility for the overall segmentation pipeline.

The key contributions of this study are threefold:

• Proposal of a percentile-based prior calibration module: We propose a novel module that reinterprets the Softmap map as a local rank to calibrate pixel values prior to binarization, thereby systematically suppressing under-segmentation.

• Global optimization via Graph-Cut integration: We perform energy minimization using the calibrated unary map and Potts-style pairwise terms. This process sharply delineates adjacent objects while ensuring the interior of each object remains consistent.

• Modularity and pipeline flexibility: The proposed method can be applied as a plug-in module without requiring modifications to the existing backbone or training process. It can also be used in conjunction with other post-processing techniques, such as morphological operations and CRFs.

2. Materials and Methods

2.1. Orthoimage

The study area for this research is Suseo-dong, Gangnam-gu, Seoul, a dense urban region comprising both high-rise apartment complexes and low-rise residential zones. High-resolution aerial orthoimages captured in 2022 were used for the analysis. The imagery features a fine pixel size with a Ground Sampling Distance (GSD) of 12 cm. It is a true orthoimage, meaning that positional displacements caused by building relief have been corrected. This ensures that building footprints are projected vertically and align precisely with ground coordinates, making the imagery ideal for boundary extraction. Fig. 1 shows an aerial orthoimage of the study area, illustrating the dense distribution of high-rise buildings and surrounding residential houses.

Fig. 1. Suseo, Gangnam-gu, Seoul.

2.2. Ground Truth

To objectively evaluate the performance of the building segmentation results, the 2022 building digital map from the National Geographic Information Institute (NGII) was utilized as the ground truth. This digital map is a national standard vector dataset that provides building footprints in polygon format and is a highly reliable dataset, produced through aerial photo interpretation and rigorous quality control. Notably, this study minimized errors arising from temporal discrepancies by using a digital map from the same time period as the aerial imagery being analyzed. A preprocessing step was undertaken in which the orthoimages and the digital map were compared. Any missing or incorrect building polygons in the digital map were manually corrected, and positional errors were adjusted. This refined final vector dataset was used as the Ground Truth. Fig. 2 illustrates the building Ground Truth for the study area, represented as polygons.

Fig. 2. Building footprints from the National Geographic Information Institute (National Geographic Information Institute, 2025).

2.3. Model and Data Preparation

For the building segmentation model, we adopted SegFormer, a state-of-the-art Transformer-based model (Xie et al., 2021). SegFormer is known for its strong performance in boundary representation and its high inference efficiency, with superior IoU metrics reported across multiple benchmarks, as summarized in Table 1. This consistent performance advantage led to its selection as the baseline model for this study. The model was trained in two stages: pre-training and fine-tuning. In the pre-training stage, the model was trained on publicly available large-scale datasets to learn general representations of building shapes and patterns, using 4,500 satellite images from the WHU building dataset (Ji et al., 2019) and 2,500 aerial images from the Massachusetts building dataset (Mnih, 2013). Subsequently, to specialize the model for Korean urban imagery, it was fine-tuned using the domestic building image dataset provided by AI-Hub. During the inference stage, the aerial photographs of the study area were partitioned into 512 × 512 pixel tiles to match the model’s input size, with a 50% overlap between tiles to mitigate edge effects.

Table 1. Performance of building segmentation models

Each tile was processed through SegFormer to generate a Softmax map, and to reduce uncertainty at the tile boundaries, only the central 256 × 256 region of each tile was retained as the valid output. The resulting per-tile Softmax maps were then mosaicked to produce a complete Softmax map for the entire study area, which served as the input for the binarization stage of this research. Fig. 3 presents a visualization of the resulting Softmax map for the entire study area.

Fig. 3. Segformer softmax map for building segmentation.

2.4. Method Overview

In this study, we propose a binarization technique that improves the separation of building objects by applying an additional post-processing module to the Softmax map of the SegFormer model. Specifically, the method clearly delineates the boundaries between adjacent buildings by performing a Prior Calibration on the Softmax map, followed by the application of a Graph-Cut algorithm. The overall workflow of the proposed technique is illustrated in Fig. 4, and it is designed to be integrated as a modular component at the output stage of existing segmentation models. As such, the technique operates independently of a model’s backbone or training process, making it universally applicable to various segmentation models. Furthermore, it can be used in conjunction with other post-processing techniques, such as morphological operations or CRFs, providing flexibility in pipeline configuration for practical applications. Ultimately, the proposed method is expected to improve the object-level separation performance of existing segmentation results without the need for additional training.

Fig. 4. Flowchart of Local Rank-Based Prior Calibration and Graph-Cut applied to softmax map.

2.5. Local Rank-Based Prior Calibration

Simply applying a threshold to the raw Softmax map can lead to under-segmentation errors, where buildings in narrow gaps are merged. To mitigate this, we introduce a Local Rank-Based Prior Calibration, which adjusts the Softmax values of each pixel by interpreting it as a relative rank among its local neighbors. Specifically, for each pixel p, a window of size w × w is defined with p at its center, and the rank of p is calculated within this window based on its Softmax map. Here s(p), and s(q) denote the Softmax map values of the target pixel p and the pixels q located within the local window centered at p, respectively. This rank is then normalized to define a percentile(p), a value that is close to 0 if p has the lowest value in the window and close to 1 if it has the highest. Eq. (1) provides the formula for calculating the percentile of a pixel p. In this equation, the denominator w² denotes the total number of pixels within the window, representing the maximum possible rank. However, near the image borders, the effective number of pixels within the window can be smaller than w² due to the cropped window area. In such cases, the denominator is adjusted according to the actual number of valid pixels.

\(\begin{align}\operatorname{percentile}(p)=\frac{\mid\{q \in \text { window }(p) \mid s(q) \leq s(p)\} \mid}{w^{2}}\end{align}\)       (1)

Next, a weight w_p for pixel p is calculated by applying a power function to its percentile value. The weighting function, which uses α as the exponent, is defined in Eq. (2). Here, α is a hyperparameter that controls the strength of the penalty assigned to pixels with low percentile values. In this equation, \(\begin{align}\bar {s}10\%\end{align}\) represents the mean Softmax map value of the top 10% highest confidence pixels within the local window centered at p. A larger value of α causes the weight w_p to decrease more sharply for pixels with low percentiles, leading to a significant reduction in their Softmax values, whereas for pixels with high percentiles, w_p approaches 1, thereby effectively preserving their original values. Setting α to a value greater than 1 reduces under-segmentation errors, but it can also excessively suppress the values of pixels near boundaries, potentially causing small buildings to disappear or leading to over-segmentation. Conversely, as α approaches 0, the calibration effect diminishes, and the results show no significant difference from the baseline.

\(\begin{align}w_{p}=\left\{\begin{aligned} s(p), & \text { if }\left|\bar{s}_{10 \%}-s(p)\right|<\delta \\ (\operatorname{percentile}(p))^{\alpha}, & \text { otherwise }\end{aligned}\right.\end{align}\)       (2)

Finally, the calibrated Softmax map, s_cal(p), is obtained by multiplying the original Softmax value, s(p), by the weight w_p calculated in Eq. (2). This adjustment process is formulated in Eq. (3). This Local Rank-based calibration prevents the misclassification of pixels as foreground by incorporating the contextual significance of those located in narrow gaps or along boundaries. The new Softmax map generated after this calibration serves as the basis for constructing a more refined energy function in the subsequent Graph-Cut stage. Fig. 5 illustrates the process of this Prior Calibration.

s_cal(P) = w_p · s(p)       (3)

Fig. 5. Local Rank-Based Prior Calibration process, where (a) the local percentile map is first derived from the softmax result, (b) the weight map is then generated based on the local rank distribution, and (c) the final calibrated map is obtained after applying the rank-based weighting.

2.6. Binarization with Graph-Cut

The final segmentation mask is generated by applying a Graph-Cut algorithm to the map produced by the Local Rank-Based Prior Calibration. The Graph-Cut algorithm treats each pixel in the Softmax map as a graph node and frames the problem of assigning a foreground or background label to each pixel as an energy function optimization problem. In this study, following conventional Graph-Cut approaches, we define the energy function as the sum of a data term and a boundary term.

The data term represents the labeling cost for each pixel and is derived from the calibrated Softmax values. A cost of -log(s_cal(p)) is assigned if pixel p is labeled as foreground, and a cost of -log (1− s_cal(p)) is assigned if it is labeled as background. This ensures that the penalty for selecting a particular label is lower when the pixel’s predicted value for that label is higher. This data term is mathematically defined in Eq. (4). The boundary term is introduced as a penalty when the labels of two adjacent pixels differ, designed to enforce the smoothness of segmentation boundaries. For any two 8-connected neighboring pixels, p and q, a cost is added if their labels differ; the smaller the intensity difference between them, the higher the cost of placing a boundary cut, and vice versa. This mechanism strengthens meaningful boundaries while suppressing unnecessary ones. In Eq. (5), λ denotes the overall weight of the boundary term, where a larger value enforces a stronger tendency to reduce the total length of boundaries, while a smaller value permits finer boundary details. Additionally, σ reflects the intensity difference between neighboring pixels: when the difference is small, the likelihood that they belong to the same region increases, and therefore a larger cost is imposed. Finally, the energy function E(L) is formulated as the sum of the costs defined in Eqs. (4) and (5) over all pixels and their neighboring pairs, as expressed in Eq. (6).

\(\begin{align}R\left(p, l_{p}\right)=\left\{\begin{array}{r}-\log \left(1-s_{c a l}(\mathrm{p})\right), l_{p}=0 \\ -\log \left(s_{c a l}(p)\right), l_{p}=1\end{array}\right.\end{align}\)       (4)

\(\begin{align}V(p, q)=\left\{\begin{array}{r}0, l_{p}=l_{q} \\ \lambda \cdot \exp \left(-\frac{|I(p)-I(q)|^{2}}{2 \sigma^{2}}\right), l_{p} \neq l_{q}\end{array}\right.\end{align}\)       (5)

The Graph-Cut optimization is the process of finding the labeling L that minimizes the defined energy function E(L). In this study, we globally optimize the energy function using the max-flow/min-cut algorithm. The resulting labeling, obtained as the solution to this energy minimization problem, assigns each pixel to either the foreground or background, ultimately yielding a globally optimal segmentation mask. The final form of the energy function is given in Eq. (6).

\(\begin{align}E(L)=\sum_{p} R\left(p, l_{p}\right)+\sum_{(p, q) \in N} V(p, q)\end{align}\)       (6)

2.7. Performance Evaluation

This section describes the evaluation framework established to objectively and reproducibly assess the proposed method. All methods were tested under identical input imagery, Ground Truth data, and preprocessing settings to ensure fairness and consistency. Both qualitative and quantitative analyses were conducted. The qualitative evaluation visually compared the segmentation results of each method by overlaying them on the orthoimages, allowing examination of boundary representation and separation between adjacent buildings, which are difficult to capture through numerical metrics alone.

At the pixel level, segmentation accuracy and boundary consistency were quantified using Precision, Recall, F1-score, and IoU. Precision measures the proportion of correctly detected building pixels among all predicted building pixels, whereas Recall indicates the proportion of correctly detected building pixels among all reference building pixels. The F1-score, as the harmonic mean of Precision and Recall, represents the overall segmentation accuracy, and IoU measures the degree of overlap between the predicted and reference masks.

At the object level, structural detection errors were evaluated by defining under-segmentation as the merging of adjacent buildings into a single object and over-segmentation as the unnecessary division of a single building into multiple fragments. In addition, Average Precision (AP) and Average Recall (AR) were employed to assess object-level detection accuracy and completeness. AP was computed as the mean area under the Precision–Recall curve across IoU thresholds from 0.5 to 0.95 at 0.05 intervals, reflecting the consistency and reliability of detections under varying overlap criteria. AR was defined as the mean recall across the same IoU range, indicating the ability to comprehensively detect building objects without omission. Together, these metrics provide a balanced and comprehensive evaluation of both pixel-level accuracy and object-level tructural reliability in densely built urban areas.

2.8. Parameter Configuration

The proposed framework involves several adjustable hyperparameters that control the behavior of the Local Rank-Based Prior Calibration and Graph-Cut stages. The key parameters include the window size (w), the percentile exponent (α), the calibration-skip threshold (δ), and the boundary-regularization weight (λ). Their roles and rationale for selection are described below.

The parameter w defines the side length of the local window used to compute percentile ranks for each pixel in the Prior Calibration stage. A small w makes the local rank estimation unstable because it relies on too few neighboring pixels, whereas an excessively large w causes pixels within a single building to be compared with distant pixels from other structures, leading to erroneous rank distributions. To balance local reliability and inter-building discrimination, w = 9 was adopted in this study, as it effectively captures narrow gaps between adjacent buildings while preserving the internal variation within building regions.

The hyperparameter α acts as the exponent of the percentile-based weighting function and regulates the penalty applied to low-percentile pixels. Larger α values exponentially suppress low-percentile pixels, which reduces spurious foreground responses in building gaps but may overly weaken boundary pixels, increasing the risk of over-segmentation. Conversely, smaller α values lessen the calibration effect and may lead to residual merging between adjacent structures. Considering these trade-offs, α was set to 0.65 to achieve a balanced adjustment between under- and over-segmentation.

The threshold δ determines whether the per-pixel calibration is applied. Pixels whose confidence exceeds δ are excluded from calibration to maintain the stability of interior regions. When δ is too small, nearly all pixels are recalibrated, which may excessively widen the building boundaries; when δ is too large, boundary pixels are omitted from calibration, weakening the boundary-enhancement effect. To preserve the integrity of building interiors while strengthening the adjustment near boundaries, δ was fixed at 0.08.

The parameter λ in the Graph-Cut stage controls the relative weight of the boundary-length regularization term in the energy minimization. A larger λ enforces smoother and more compact regions by penalizing long or irregular boundaries, but an excessively high value can cause narrow gaps between adjacent buildings to remain merged. In contrast, a small λ weakens the penalty for boundary creation, increasing the likelihood of unnecessary or noisy boundaries. In this study, λ = 2 was chosen as an appropriate compromise between region smoothness and accurate boundary delineation.

Overall, the proposed method exhibited stable performance across a reasonable range of parameter values, indicating that it is relatively insensitive to minor parameter variations. The robustness of these parameter choices is further confirmed in the subsequent experimental results.

3. Results

3.1. Baseline Comparison

The proposed method (Ours) involves adjusting the Softmax map via Prior Calibration, followed by the application of Graph-Cut. This method effectively separates the narrow gaps between adjacent buildings while maintaining the continuity of individual building interiors. As observed in the visual comparison in Fig. 6, targets that were predicted as a single building by the baseline and other post-processing methods were clearly separated and detected as individual objects when our proposed method was applied. At the same time, our method effectively suppressed the excessive fragmentation of buildings that was occasionally produced by the standalone Graph-Cut approach, resulting in almost no cases of unnatural building splits. These characteristics are also reflected in the quantitative evaluation. The proposed method achieved an under-segmentation rate of 15.33%, lower than both the thresholding and CRF-based methods, demonstrating its effectiveness in suppressing the excessive merging of adjacent buildings. Its over-segmentation rate of 0.49% was the lowest among all compared methods, indicating that the unnecessary fragmentation of buildings was minimized. The method also achieved the highest scores in terms of pixel-level accuracy, with an IoU of 0.8012 and an F1-score of 0.8896. Furthermore, Precision (0.8949) and Recall (0.8844) remained well-balanced, confirming superior performance with few misclassified pixels and a low rate of missed detections. These findings are corroborated by the quantitative comparison in Table 2, supporting the conclusion that the proposed method effectively resolves the trade-off between boundary precision and object preservation by reducing under-segmentation errors while minimizing detection loss.

Fig. 6. Visual comparison between existing methods and the proposed method.

Table 2. Quantitative comparison between existing and proposed methods

The default output of the deep learning model used as the baseline is ArgMax binarization, which assigns each pixel to the class with the highest Softmax values. This method was unable to adequately separate the narrow boundaries between buildings, resulting in a high under-segmentation rate of 28.47%. To address this issue, the thresholding criterion was adjusted. With a threshold of 0.7, the under-segmentation rate decreased to 18.49%, allowing some previously merged buildings to be separated. However, several narrow gaps were still not recognized as background, leaving some buildings unseparated. Increasing the threshold further to 0.9 reduced the under-segmentation rate to 9.98%, successfully separating many adjacent buildings. At the expense of this improvement, fewer pixels were classified as buildings, leading to a decrease in Recall (0.8227) and an increase in Precision(0.9274). This also caused unnecessary fragmentation of some buildings, raising the over-segmentation rate to 1.70%.

These results demonstrate a clear trade-off: higher thresholds improve the separation of closely situated buildings but increase the omission of true building pixels. This trend was evident not only in the quantitative metrics but also in the visual results shown in Fig. 7, highlighting the limitation that a single threshold cannot simultaneously achieve high recall and precision. Additionally, Otsu’s method (Otsu, 1979), which automatically determines a global threshold by maximizing inter-class variance in the image histogram, was evaluated. For this dataset, the calculated threshold was 0.462745. As a single-threshold-based method, it exhibited the same limitations as manual thresholding, with an under-segmentation rate of 29.68%, similar to the baseline. It achieved an IoU of 0.7809 and an F1-score of 0.877, showing only limited improvement.

Fig. 7. Visual comparison of building segmentation results with different post-processing methods (green: TP, red: FP, blue: FN) under threshold-based strategies.

The Grid-CRF method considers pixel adjacency, encouraging pixels with similar values to be assigned the same label. In this study, the Grid-CRF was effective at removing small noisy regions, but it exhibited a tendency to merge building objects in narrow, low-contrast gaps. Quantitatively, the under-segmentation rate with Grid-CRF was 29.20%, which was almost identical to the baseline value of 28.47%, showing little improvement in reducing this type of error. The over-segmentation rate was 0.73%, remaining at the same level as the baseline. These results indicate that while CRF was effective in mitigating minor pixel noise, it contributed little to the critical task of separating building boundaries. This is likely because boundary pixels in narrow gaps had spectral values similar to their surroundings and were thus smoothed into the background during the CRF’s regularization process.

Graph-Cut assigns labels by simultaneously optimizing a data term and a boundary term. This process penalizes abrupt label changes between adjacent pixels, thereby ensuring contiguous region segmentation. Applying Graph-Cut to the Softmax maps significantly reduced the under-segmentation rate to 13.87%, less than half of that observed in the baseline, indicating that a substantial number of previously merged buildings were successfully separated. However, Graph-Cut alone still struggled to separate buildings across some of the narrowest gaps. Fig. 8 provides a direct visual comparison of the Grid-CRF and Graph-Cut results against the Ground Truth (GT), illustrating the respective strengths and limitations of each post-processing method.

Fig. 8. Visual comparison of building segmentation results with different post-processing methods (green: TP, red: FP, blue: FN) under graph-based strategies.

3.2. Parameter Sensitivity

We analyzed the sensitivity of the proposed method to variations in its main hyperparameters. The parameters were configured according to the setup of the proposed framework, and the resulting quantitative and qualitative outcomes are summarized in Table 3 and Fig. 9.

Table 3. Quantitative comparison between parameter settings of the proposed method (Ours parameters: w = 9, α = 0.65, δ = 0.08, γ = 2)

Fig. 9. Visual comparison between parameter settings of the proposed method.

As shown in Table 3, the proposed method exhibited stable performance across all parameter variations. For the window size w, both smaller (w = 7) and larger (w = 19) settings resulted in slightly lower AR and AP values compared with the final configuration (Ours). Similarly, changes in α (0.5 and 1.5), δ (0.06 and 0.10), and λ (1 and 7) led to only marginal fluctuations in overall accuracy, indicating that the segmentation results were largely unaffected by moderate parameter changes. The final configuration achieved the highest and most balanced scores in AR and AP, confirming that the selected parameter combination provides the most consistent performance.

Fig. 9 presents the qualitative results for different parameter settings. Even when parameters were varied to lower or higher values, the building boundaries and separations between adjacent structures remained visually stable, with no noticeable merging or fragmentation. Only minor differences in boundary smoothness appeared under extreme cases, while the overall spatial structure of the buildings was preserved. These results demonstrate that the proposed method maintains stable segmentation quality and high robustness across a reasonable range of w, α, δ, and λ values.

3.3. Processing Time

The experiments were conducted on a system running the Ubuntu 20.04.6 LTS operating system, equipped with two Intel Xeon Silver 4214 12-core processors. All post-processing methods were executed on the CPU without GPU acceleration to compare the computational speed between the different techniques. The input image was a very large-scale image with dimensions of 14,781 × 13,935 pixels. The processing time for each method was measured from the moment the image was fully loaded from the disk until the final binarized result was produced. The detailed specifications of this experimental environment are summarized in Table 4.

Table 4. Experimental environment, including system specifications, runtime settings, and measurement definition

The comparison of results showed that the simple thresholding-based methods were extremely fast. Global threshold binarization using Otsu’s method was completed in approximately 3 seconds, while the fixed-threshold baseline binarization took about 2 seconds. In contrast, the graph-based methods required significantly more computational time. The Grid-CRF method could be processed within a few minutes, taking approximately 5 minutes and 9 seconds, whereas the Graph-Cut algorithm required a much longer, hour-scale computation of 2 hours and 26 minutes. The proposed method, which adds the Prior Calibration preprocessing step to the Graph-Cut, took 2 hours and 28 minutes, approximately 2 minutes longer than the standard Graph-Cut. A comparison of the processing times for each method is presented in Table 5. These results demonstrate that graph-based approaches require substantially more computational time than threshold-based methods.

Table 5. Comparison of processing time for different post-processing methods (Measured from raster loading to binary output)

The reason the graph-based methods took several hours is that they are required to construct and optimize a large-scale grid graph with approximately 4.12 × 10⁸ edges, based on an 8 connectivity criterion. In our measurements, the graph construction stage accounted for the majority of the total computation time. Particularly for such high-resolution, large-scale images, a bottleneck was observed where increasing the number of CPU threads did not lead to a linear speed improvement, due to limitations in memory bandwidth and cache utilization. The proposed method introduces a slight overhead compared to the standard Graph-Cut due to the addition of the Local Rank-based weight calculation as a preprocessing step, and this difference is reflected in the final processing time. In the context of a speed-accuracy trade-off, this computational cost may be prohibitive for real-time applications or for processing entire large-scale datasets. However, our experiments confirm that for offline applications where maximizing segmentation accuracy is the primary goal, the performance gains provide sufficient value.

4. Discussion

The PR-curve in Fig. 10 illustrates the trade-off between Precision and Recall for the threshold-based methods. The typical pattern was observed, where increasing the threshold leads to a rise in Precision and a corresponding drop in Recall. It is difficult to achieve high performance on both metrics simultaneously by only adjusting a single threshold, a limitation that was also evident in our baseline results. In contrast, the proposed method achieves a balance of high Precision and high Recall that is difficult to attain using a single threshold. The proposed method also surpassed all other methods in pixel-level performance metrics. This is because the combination of Prior Calibration and Graph-Cut optimization successfully reduced the number of misclassified pixels while minimizing the loss of true building pixels. In effect, the proposed method adaptively calibrates the softmax probabilities before binarization, resulting in an outcome similar to a locally adaptive decision boundary rather than a fixed global threshold. By adjusting the probability distribution according to the local percentile context, each pixel is effectively assigned a context-dependent decision level, which enables precise separation of adjacent buildings while preserving interior consistency. Furthermore, Fig. 11 presents the object-level AR–AP comparison, where the proposed method consistently outperforms the others, confirming its effectiveness across both pixel-level and object-level evaluations.

Fig. 10. Precision-Recall curve of thresholding-based segmentation and performance of each method.

Fig. 11. AR–AP curve of object-based evaluation and performance of each method.

However, the “Local Rank-Based Prior Calibrated Softmax and Graph Cut” technique proposed in this study has limitations in terms of computational efficiency. The high computational cost of Graph-Cut-based algorithms imposes constraints on their use for large-scale data processing and real-time applications. This remains a challenge to be addressed in future work through large-scale parallelization or algorithmic streamlining. Additionally, since the proposed method is fundamentally dependent on the initial segmentation results, building objects that are not detected at all in the initial stage are likely to be missed. Therefore, the technique is limited in that it cannot provide a direct solution for objects that fall beyond the detection capabilities of the initial model.

5. Conclusions

This study presented a novel approach to address the problem of adjacent buildings being merged in dense urban imagery, and successfully achieved this research objective through the proposed method. By combining percentile-based Softmax calibration with Graph-Cut optimization, we addressed the structural limitations of conventional segmentation results. This approach improved both object-level accuracy and boundary quality by accurately separating individual buildings and reproducing fine boundary details. The proposed module is designed to be applied as an independent post-processing step to the output of existing models, without requiring any changes to the training process. Therefore, it is universally applicable to various segmentation backbone models and exhibits high portability and reproducibility in practical applications. The experimental results demonstrate that the proposed technique performs robustly even in complex, dense urban scenes, effectively identifying narrow boundaries that are poorly distinguished by conventional methods. Notably, the under-segmentation rate was reduced by nearly half, from approximately 28% to 15%. Furthermore, object-level metrics showed an improvement in AR from 0.46 to 0.55 and in AP from 0.59 to 0.66, indicating enhanced accuracy and reliability in building segmentation.

However, the high computational cost of the Graph-Cut-based algorithm remains a challenge, imposing constraints on its use for large-scale data processing and real-time applications. The method is also intrinsically dependent on the initial segmentation results, meaning that objects completely missed in the initial stage are likely to remain undetected. Future work will focus on improving computational efficiency by introducing large-scale parallelization or lightweight optimization techniques. We will also explore integrating the percentile calibration concept into the deep learning training process to achieve similar effects at a lower computational cost. Furthermore, we plan to extend this research by applying the technique to other object segmentation problems and data types to validate its generalizability, as well as optimizing fusion strategies with existing post-processing methods to broaden its range of applications.

Author Contributions

Conceptualization: Lee C, Lee IP; Data curation: Lee C, Lee JH, Lee IH; Formal analysis: Lee C, Bui NA; Investigation, Methodology: Lee C; Visualization: Lee C, Lee IH; Supervision: Cheon JW, Lee IP; Validation: All authors; Writing–original draft: Lee C; Writing–review & editing: All authors.

Conflicts of Interest

No potential conflict of interest relevant to this article was reported.

Funding

This work is supported by the Korea Agency for Infrastructure Technology Advancement(KAIA) grant R&D program of Digital Land Information Technology Development funded by the Ministry of Land, Infrastructure and Transport(MOLIT)(Grant RS-2022-00142501).

Data Availability Statement

The source code that supports the findings of this study is openly available on GitHub at https://github.com/2chong/inference_ postprocess.