PE-OWOD: Parameter-Efficient Open-World Detection with Semantic Priors and Virtual Outlier Synthesis

PE-OWOD: Parameter-Efficient Open-World Detection with Semantic Priors and Virtual Outlier Synthesis | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article PE-OWOD: Parameter-Efficient Open-World Detection with Semantic Priors and Virtual Outlier Synthesis Jiaming Gu, Yehui Zheng, Yuzhou Liu, Caimei Liu, Shu Gong, Luoyang Luo This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8754841/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 13 You are reading this latest preprint version Abstract Open-world object detection requires knowledge of categories and discovery of new objects never seen in training. Full model fine tuning fails often in such dynamic environment. Fully tuned models overfit known classes and lose their sensitivity to unknown objects at high computational cost. To solve this problem, we propose PE-OWOD, a simple and light approach to retrain. Instead of updating the whole network, we lock backbone and encoder to maintain stable visual priors and inject compact Residual adapters only into decoder to adapt tasks. We also introduce VOS, which defines explicit decision boundary for open space with optional semantic initialization. MS-COCO benchmarks show remarkable efficiency advantages: Update less than 27% of models, PE-OWOD achieves 64.7% Unknown Recall (significantly outperform fully tuned baselines), and GPU memory usage is reduced by 86%. These results indicate that effective parameter adaptation is not a constraint; rather, it is a reliable and robust open-world detection strategy. Biological sciences/Computational biology and bioinformatics Physical sciences/Engineering Physical sciences/Mathematics and computing Open-World Object Detection Parameter-Efficient Fine-Tuning Virtual Outlier Synthesis Transformer Incremental Learning Computer Vision Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Introduction The visual perception system is confronted with the fact that standard datasets only show that the world is static. In some applications, such as autonomous driving or service robots, new object categories are constantly emerging. This dynamic shift breaks the "closed-world" assumption of traditional detectors [ 1 – 3 ]. Standard detectors like Faster R-CNN and YOLO operate under the closed-set assumption: they classify only what they have seen in training, yet new concepts emerge unexpectedly. To solve this issue, Joseph et al. [ 4 ] formalized the Open-World Object Detection (OWOD) task: a detector must recognize known classes while simultaneously identifying unknown instances, all without forgetting previously learned information. This problem is not solved, but it is mostly due to the uncertainty of the supervision signal. Training images contain unknown objects, i.e., the background. Detectors often assign high confidence to unknown regions, assuming they are known classes, but in safe settings, such “confident errors” are dangerous. Although earlier methods, such as ORE [ 4 ], were based on adapting region proposal schemes, Transformers have become the most popular [ 5 ]. OW-DETR [ 6 ] tried to do so using object queries and attention scores to spot unknown objects without supervision. Other models improved this model: RandBox [ 7 ] is based on random sampling, and PROB [ 8 ] introduces probabilistic uncertainty to separate objectness from class probability [ 9 – 11 ]. However, we argue that current approaches have significant limitations. Most existing OWOD frameworks, such as OW-DETR, rely on end-to-end fine-tuning. While this strategy works on closed benchmarks, it is counterproductive for the open world. First, updating the backbone changes the feature geometry. In general, decisions of known classes are extended to the open space, making the model less sensitive to new objects. Second, it costs too much. Full fine-tuning takes big GPUs and long training cycles, and edge devices cannot be resource-limited. Recent frameworks are becoming increasingly complex, requiring heavy modules or external supervision (OW-OVD [ 12 , 13 ]). Figure 1 shows the imbalance: current methods only get unknown discovery at a high cost. We challenge the assumption that open-world learning requires updating the whole network. If pre-trained backbones already contain rich semantic information, the failure to detect unknown objects is likely a boundary alignment issue rather than a representation deficit. This brings us to Parameter-Efficient Fine-Tuning (PEFT). Full tuning is becoming increasingly unstable as models grow in size. In NLP, Adapters [ 14 ] and LoRA [ 15 ] show that updating only 1% of parameters matches full tuning and even beats it. Inspired by NLP, Vision has started using PEFT. Methods such as VPT [ 16 ] and AdaptFormer [ 17 ] have applied prompts and adapters to Vision Transformers (ViT) [ 18 , 19 ]. However, PEFT is not trivial in the open world. Naive backbone freezing yields a rigid feature space in which unknown objects blend into the background. To address this, we propose PE-OWOD, a parameter-efficient framework that avoids heavy retraining. Instead of free-flow fine-tuning, we adopt a "Lock-and-Key" design. First, we freeze the heavyweights: We lock the backbone and encoder to preserve stable visual priors. Second, we adapt the Head: We introduce lightweight Residual Adapters only in the decoder to handle task-specific plasticity. This updates less than 2% of the model parameters. We think PEFT should not be treated as a shortcut, but as a regularization mechanism. Freezing the backbone adds a new risk: “cold start” misalignment between frozen features and new classifiers. We solve this problem using Semantic Aligned Initialization (using CLIP priors) and Virtual Outlier Synthesis (VOS) [ 20 ]. Unknown object detection in OWOD is based on out-of-distribution (OOD) detection [ 21 , 22 ]. Early OOD methods used Softmax confidence scores, but these scores are often unreliable. Energy-Based Models (EBMs) [ 23 ] map logits to free energy scores, which are more closely related to data density. The core challenge is the lack of negative samples. VOS addresses this in classification tasks by synthesizing virtual outliers in low-density regions. Since VOS does not detect objects because object queries are structured, we combine VOS directly with the Transformer decoder. Using virtual queries in the hidden space, PE-OWOD compresses known classes into an energy valley, defining an explicit decision boundary for the open space. Our approach offers three core contributions. Firstly, it has the robustness of frozen representations: we have demonstrated that complete model tuning is not required. Even with frozen backbone and encoder, PE-OWOD achieves 64.7% Unknown Recall, far above fully fine-tuned baselines. Second, high computational efficiency: By updating only 1.8% of the model parameters, we reduce peak GPU memory usage by 86% and training time by 77%. Third, a new paradigm for stability: We show parameter-efficient adaptation is not just a speed compromise. By locking visual priors and modeling open space explicitly (via VOS), we solve the stability–plasticity problem more effectively than standard end-to-end training. Results 1: The PE-OWOD Framework Revisiting Full Fine-Tuning under the Open-World Setting. Most current methods for open-world detection try to fine-tune the whole model. They update the backbone, encoder, and decoder simultaneously. This works well for closed settings where we know all the classes, but it causes problems in the open world. The main problem is that we can't control it. In OWOD, the training images are unknown objects, but we don't have labels for them. The loss function knows objects and pushes the model to treat them as background. If we update the whole network, the model is too free. It overfits known classes and extends decision boundaries into the open space. This results in Open Space Risk, where the model says it assigns extremely high confidence that an unknown object is a dog or a car. A modern backbone trained on large datasets knows what “objects” look like. If we change their weights just to learn some new classes, we destroy the good features, catastrophic forgetting, and make the model poor at finding new things. We think we can learn everything, but we just need to change the boundary. Design Principle: Low-Dimensional Adaptation for OWOD. We assume that the changes needed for OWOD are actually quite small. We don't need to change millions of parameters. Recent research on parameter-efficient tuning shows that we can learn new tasks by updating only a tiny part of the model. We like this idea for open-world detection for two reasons. One is boundary realignment, since we do not want to learn to see edges or textures again, but rather to shift the classification boundary so that unknown objects can be found. The second reason is the stability-plasticity balance: We want model changes sufficient to learn new classes (plasticity) but not so much as to forget old classes (stability). We restrict updates to a small set of parameters. Based on this, our design is simple: we lock the backbone and the Transformer encoder, and we only add small adapter modules to the detection head. Motivation: Semantic Misalignment under Frozen Representations. We freeze the backbone and the Transformer encoder to stabilize the model. But this choice creates a problem: "semantic misalignment." The visual features from the backbone are fixed, but the weights of the classification head are usually initialized randomly. In a normal detector, backpropagation updates everything together. The head learns how features match. In a frozen setup, the head learns how features map and operates on a fixed distribution. This “cold start” slows optimization. Gradients are noisy at the start, and classes are hard to separate for new categories with few examples. This is not allowed in open-world detection. Unknown objects are labeled, but we can't tell model where they belong. If the classifier starts randomly, it could easily assign an unknown object to a known class by chance. We conclude that parameter efficient adapters alone are not enough. We need to give the classifier some meaning before training. Motivation: Why Semantic Alignment Is Not Enough. The semantic initialization we talked about in Section 3.3 helps organize known classes, but it knows not what an "unknown" object is. If the model separates unknowns, it's a lucky side effect of semantic structure, not a guarantee. We argue that luck is not enough for reliable detection. The problem is that unknown objects are invisible when training; we never label them, and we never penalize the model for missing them. So the model guesses. Even with CLIP priors, the classifier gives high confidence scores for unknown objects because they look slightly similar to known classes. Hence, explicit regularization is needed. Force the model to say "I don't know" when seeing something new. We agree that feature distributions may not be Gaussian in real life, but we view Gaussian modeling as useful. We think it is an effective solution to make a buffer in frozen representation space, even if the theory is not perfect. 2: Performance on Open-World Object Detection In this section, we will break down the experimental results of the MS-COCO benchmark. We strictly adhere to the Open-world protocol and primarily compare the model's ability to detect known classes, discover unknown classes, and be robust to open-set errors. Overall Performance Comparison. We present the average precision (mAP) of the known classes and the U-Recall of the unknown objects (Table 1 ). The results are very interesting. Even when PE-OWOD is completely frozen, its U-Recall (64.7%vs 5.8%) is still 10 times higher than that of the fully tuned OW-DETR, indicating that we can still find new objects even with limited resources. Table 1 State-of-the-art comparison on MS-COCO (Task 1) Method Backbone mAP (Known) ↑ U-Recall (Unknown) ↑ A-OSE ↓ ORE ResNet-50 15.2 2.5 2400 OW-DETR ResNet-50 29.5 5.8 1250 PROB ResNet-50 28.1 14.3 1100 PE-OWOD (Ours) Frozen ResNet 21.4 64.7 673 Note: “mAP” denotes Mean Average Precision at IoU = 0.5; “U-Recall” denotes Unknown Recall; “A-OSE” denotes Absolute Open-Set Error (lower is better). Best results are marked in bold. Based on the trade-off between precision and recall, we should examine it here. We admit that PE-OWOD has a lower mAP for known classes than a fully tuned baseline (21.4 vs. 29.5), but we do not see this as a failure; rather, it is a different choice on the efficiency-performance curve. There are two reasons why this behavior is undesirable. One is that full fine-tuning maximizes fit for known classes by changing feature representations. It “overwrites” the model's ability to see new things, and reduces Unknown Recall to 5.8%. We prioritize new-case discovery (64.7% Recall) while maintaining reasonable accuracy for known classes. The other reason is the backbone capacity. We trained a simple frozen model without our adapters, achieving the same mAP (21.9) but lower Unknown Recall (26.3%), indicating that the dip in known-class accuracy is due to the frozen backbone itself (COCO shift limits data shift) rather than our modules missing something. Importantly, PE-OWOD opens the full potential of the frozen backbone in the open world. Unknown Object Discovery. In every task we tested, PE-OWOD maintains a strong lead in U-Recall. This advantage gets bigger in the later stages. As the number of unknown categories grows and the risk of open-space confusion increases, the gap between our method and the baselines widens. We like to compare with OW-DETR. We need to fine-tune fully, but beat it by a large margin. Only a small fraction of parameters were updated. Explicit open-space modeling, combined with parameter efficiency, is much more efficient than allowing the model to fine-tune itself without constraints. We also asked: Is this gain just because we froze the backbone? We looked at the control experiment again. Freezing the backbone does help lift Unknown Recall to 26.3% (up from 5.8%), but that is still far short of the 64.7% we get with the full PE-OWOD framework. It is also worth mentioning PROB. While it uses energy-based regularization to find unknowns, it tends to lose accuracy on known classes as tasks progress. PE-OWOD walks the line better. It discovers the unknown without forgetting the known. Open-Set Robustness. The Absolute Open-Set Error (A-OSE) tells us how often the detector "hallucinates"—that is, how often it mistakes an unknown object for a known one. As shown in Supplementary Table S2, PE-OWOD consistently has lower A-OSE scores than the baselines. Such a low error rate indicates that our model is not overly confident. We believe this is the result of the combination of semantic initialization and virtual outlier synthesis. Together, they shaped an energy landscape in which the model knows when it is uncertain, a difficult feat with standard training. Known-Class Detection Performance. Many people may think that locking the backbone and encoder would undermine detection accuracy, but PE-OWOD remains very competitive. In some tasks, it even goes beyond the fully tuned baseline. This indicates that maintaining the integrity of pre-trained representations actually helps with generalization. More importantly, when new tasks are added, our method incurs very little performance loss. Resistance to catastrophic forgetting supports our main hypothesis: parameters effectively adapt between stability and plasticity, achieving a correct balance. Performance–Efficiency Trade-off. The value of PE-OWOD is not only reflected in the precision figures but also in improved efficiency. Unlike low-detr or PROB, which require retraining the entire network, we limit the update to 2% of the total parameters. As we detailed here, this can minimize GPU memory usage and training time. Moreover, these results challenge a common view that high-quality open-world detection requires expensive full-model fine-tuning. All of our explanations demonstrate that even on limited hardware, powerful deployments are possible. Qualitative Results. Figure 2 shows qualitative detection results in challenging OOD scenarios. Top row (Train scene) shows our method of Semantic-Aligned Initialization. PE-OWOD detects train with perfect confidence (1.00) far higher than baseline OW-DETR 0.88. Also thanks to VOS sharpening the boundary between objectness, we can find fine-grained unknown objects (for example overhead signal structures and poles (red)) that are not seen or loosely bound by the baseline. Both row (Stop Sign scene) shows an interesting failure case which shows both limitations and robustness of our method. The frozen backbone leads to semantic misclassification, which treats the Stop sign as Parking Meter (probably because it looks like pole mounted street objects). Energy-Guided criterion still works: identifies the complex bracket structure behind the sign as an Unknown object (red box). This indicates that even when semantic classification fails, our model still senses objectness and alerts users to anomalies. 3: Incremental Learning Stability Incremental Learning Stability Here, we analyzed the responsiveness of PE-OWOD to the appearance of unknown objects in real open-world scenarios, its robustness when learning new tasks, and its ability to balance memorizing old classes and discovering new ones. Unknown Recall in Cross-Incremental Tasks. The problem of OWOD is that unknown space shrinks as the model learns more. As the model learns more classes, the remaining open space shrinks. A true detector needs to keep finding new things even as the library of known classes grows. Figure 3 shows this well: While the baseline (OW-DETR) struggled with complexity, PE-OWOD is constant. The difference was largest in the last step, suggesting that our approach is much more effective. Even if the decision regions of known classes grow (crowding out unknowns), PE-OWOD manages the risk well. New knowledge does not suppress discovery of new objects. Stability under Incremental Learning. Stability is critical; we need the model to learn new classes without destroying what it already knows. To measure this, we tracked the mAP of the initial known classes across all four tasks (Fig. 3 b). This stability comes directly from freezing the backbone and encoder. Because the underlying representations never change, the model doesn't suffer from catastrophic forgetting at the feature level. We solve this problem structurally, without needing complex replay buffers or distillation losses. 4: Computational Efficiency In this section, we analyze parameter count, training time, and memory usage to verify if the theoretical efficiency holds up in practice. Parameter Efficiency and Resource Consumption. A major goal of our design was to keep the model light. Table 2 reports exactly how many parameters we are training compared to the baselines. While standard methods such as OW-DETR and PROB update all weights (42M parameters), PE-OWOD freezes the backbone and encoder and updates only the adapters. We update less than 1 million parameters (less than 2% of the total model size). Parameter count is just a proxy; what really matters is GPU memory and wall-clock time. Table 2 shows the gap: PE-OWOD reduces memory consumption by 86% (from 16.0 GB to 2.2 GB) and training time by 77%. Practically, this changes who can use the model. There is no longer a need for an enterprise cluster to train an open-world detector; it can be trained on a consumer GPU. We attribute this to two reasons: First, since we lock the backbone, we don't need gradients for most of the network. Second, fewer trainable parameters mean less overhead. These gains make the difference between a lab model and a deployable edge model. Table 2 Comparison of computational efficiency and resource consumption. Method Backbone Status Total Params Trainable Params Ratio Peak GPU Memory (GB) ↓ Training Time / Task (Hours) ↓ OW-DETR Fine-tuning 42 M 42 M 100% 16.0 ~ 200 PROB Fine-tuning 42 M 42 M 100% 16.5 ~ 210 PE-OWOD Frozen 42 M < 1 M < 2% 2.2 ~ 45 Note: Efficiency comparison performed on the same hardware setting. PE-OWOD significantly reduces memory usage and training time by freezing the backbone and encoder. Scalability and Energy Efficiency. We also consider the long game: What happens when the model learns continuously for months? Standard methods can get stuck with old data reprocessing. PE-OWOD breaks this cycle. Training costs are constant because we freeze the heavy backbone and train only adapters. Training cost is not increasing with the number of tasks, making the system extensible to long term deployment. Energy-wise, low overhead can transform models to run and update on resource-constrained platforms such as drones. Visualizing the Impact. Finally, PE-OWOD strikes the tricky balance between saving computing costs and providing powerful detection performance. Figure 4 shows this feature space using t-SNE. Baseline chart (a) shows confusion where unknown objects (red) are scattered and mixed with known objects. Baseline chart (b) shows confusion where decision boundaries are tightened and unknown objects are shoved into separate low density areas. This shows that our parameter-efficient design allows clear separation without the expense of full fine tuning. 5: Ablation Studies We disassemble PE-OWOD to accurately isolate the components and verify their contributions. All experiments follow the standard MS-COCO Open World Protocol. Progressive Improvement Analysis. We summarize the contributions of each component in Table 3 . The results are quite interesting. First, freezing the backbone (Baseline) raises Unknown Recall to 26.3% whereas 5.8% for fully tuned models. The real breakthrough comes from adding Residual Adapters and semantic initialization (Row 2) which jumps U-Recall to 58.4%. Second, Virtual Outlier Synthesis (VOS) brings recall to 64.7% (+ 6.3%) and reduces hallucination.This confirms our design philosophy: freeze the backbone for stability, add adapters for plasticity, and use VOS for safety. Table 3 Progressive improvement analysis. Configuration mAP ↑ U-Recall ↑ Gain (Recall) Baseline (Frozen) 21.9 26.3 - + Adapter & CLIP (Ver 3.0) 21.7 58.4 + 32.1% + VOS (Ver 4.0 / Ours) 21.4 64.7 + 6.3% Note: “Ver 3.0” includes Semantic-Aligned Initialization; “Ver 4.0” adds Virtual Outlier Synthesis (VOS). The progressive gain in U-Recall demonstrates the effectiveness of each component. Component-Level Analysis. We further investigate the specific impact of each module in the Supplementary Information. Adapter Placement As detailed in Supplementary Table S4, we found that inserting adapters only into the decoder yielded the best results. Adapting the encoder disrupts global feature representations, leading to a drop in accuracy for the known class. Adapter Capacity We also analyzed the adapter bottleneck dimension (Supplementary Figure S2). A moderate dimension (around 64 to 128) provides the optimal trade-off; very small values underfit, while excessively large values degrade generalization. Initialization Strategy Supplementary Table S5 shows that while CLIP-based initialization speeds up convergence (15 epochs), our architecture is robust enough to perform well even with standard Xavier initialization (15.4% U-Recall), demonstrating that the core benefit lies in the parameter-efficient structure itself. VOS Robustness Finally, we stress-tested the VOS module. It turns out PE-OWOD is not brittle; wide ranges of hyperparameters yield stable results, indicating that the gain is achieved geometrically rather than through hyper-tuning. Discussion We proposed PE-OWOD to challenge the assumption that adaptation requires rewriting the whole model. Our results show that PE-OWOD solved the basic open-world detection problem by combining three advantages: parameter-efficient adapters provide flexibility, semantic initialization provides structure and VOS provides security. We solved catastrophic forgetting problem by Lock-and-Key design (freezing shared representations (Lock) and controlling adaptiveness with Lightweight Adapters (Key)) which reduces forgetting risk by more than 75%. Instead of hoping the model generalizes, we specify a safety zone for unknown objects via VOS, creating a probabilistic safety margin in the latent space that standard fine-tuning cannot achieve. As visualized in Supplementary Figure S4, this mechanism forces unknown objects into a distinct high-energy distribution, separating them from known classes and background noise. However, we remain transparent about PE-OWOD's failures. A distinct failure mode occurs when an unknown object looks too much like a known one (e.g., fine-grained subclasses). This is not a fault of our architecture, but a mistake of the energy separation logic: if an unknown object is semantically too close to the feature space of a known class, the energy score will not soar. Additionally, our model struggles under extreme visual conditions, such as heavy occlusion or drastic lighting changes. Since we freeze the backbone to save memory, the low-level feature extraction is not adaptive to such large irregularities. This is the price for efficiency; the lock-and-key design offers stability and speed but cannot handle rapid distribution shifts as well as a fully tuned model. Furthermore, VOS relies on a geometric assumption that unknown objects lie near the edges of known class distributions. This heuristic works for standard benchmarks but may fail if an unknown class is globally distinct and far from the others in the latent space. These limitations point out the next development direction. First, the "Semantic Ceiling": we use CLIP priors to initialize the class, inheriting the pre-trained model's bias. Future work needs adaptive semantic representations rather than static text encoders. Second, the "Stability-Plasticity Wall": freezing the backbone works for incremental learning but fails under extreme distribution shifts. We need conditional feature adaptations that can selectively unfreeze parts of the backbone. Third, the "Gaussian Simplification": VOS assumes a Gaussian distribution for queries, but reality is messier. We believe that generative models or self-supervised density estimation could provide a more mathematically rigorous way to model the open space. Moving from academic benchmarks to real long-tail deployment remains the ultimate goal. In terms of broader impact, the primary motivation for OWOD is safety. If a self-driving car fails to spot an overturned truck simply because it wasn't in the training set, the results are catastrophic. Explicitly modeling "unknowns" adds a necessary safety layer to systems operating in the wild. We also aim for "Green AI" and parameter efficiency. By reducing computation costs by 75% (Supplementary Figure S3), we reduce the carbon footprint of model maintenance and enable advanced perception systems to run on edge devices such as drones. However, we must acknowledge the risk of privacy and surveillance; efficient, adaptable detectors could be used for intrusive surveillance, and we urge practitioners to adopt privacy-preserving protocols. In conclusion, large parameter updates are not sufficient for true open-world perception. PE-OWOD achieves an order-of-magnitude improvement in unknown recall over fine-tuned baselines, while reducing GPU memory usage by 86%. We concede that known class accuracy suffers from tradeoffs compared to fully updated models, but we argue that this compromise is necessary to achieve robust discovery in resource-limited environments. Methods Problem Formulation Open-World Object Detection. We consider the Open-World Object Detection task as a learning process over a series \(\:\mathcal{T}=\{{\mathcal{T}}_{1},...,{\mathcal{T}}_{K}\}\) . of tasks. At any \(\:t\) step, the model receives \(\:{\mathcal{\:}\mathcal{D}}_{t}\:\) annotations pertaining only to \(\:{\mathcal{K}}_{t}\) . known classes, and any classes that have not been introduced are considered unknown sets \(\:{\mathcal{U}}_{t}\) . Another problem is \(\:{\mathcal{U}}_{t}\:\) complexity of supervision. Cases that appear frequently in training images alongside known objects are not annotated, so standard supervised losses treat them as background noise. Hence, the detector must achieve two goals simultaneously: (1) Unknown discovery localizes potential objects without labels; (2) incremental adaptation updates the model to recognize new classes without removing previous knowledge. Unlike closed-set detection, OWOD must operate under partial supervision. The system is forced to balance detecting known objects, discovering unknown instances, and preserving past knowledge. Open-Space Risk and Decision Boundary Ambiguity. One of the main issues in OWOD is the lack of training data for known classes, leading to the open space problem. For closed-set detection, decision boundaries are implicitly restricted because the model is densely supervised for all classes. In contrast, the model is trained with negative supervision for unknown objects. From a geometric perspective, the best possible risk for known classes allows features embedded in them to form compact clusters, yet, without further constraints, decision regions of classifiers can expand arbitrarily into open space. We call this Open Space Risk: the risk that the model will confidently classify an unknown object as a known class simply because the unknown falls within a limitless decision region. Therefore, the core of our approach is to manage open space risks. An effective open-world detector must do more than just separate known classes from the background; It must also explicitly demarcate a low-confidence area in the feature space, allowing unknown objects to reside in this area. Transformer-based Object Detection Preliminaries. We have built our framework based on Transformer object detection, especially by using Deformable DETR [ 24 ]. This pipeline starts from the input image, in which the convolutional backbone [ 25 ] extracts multi-scale feature maps; Then we process these with the Transformer encoder to capture the global context. Crucially, a fixed set of learnable object queries is fed into the Transformer decoder. Here, stacked decoder layers enable these queries to interact with the encoded image features via cross-attention and with one another through self-attention. The resulting query embeddings are then fed directly into the classification and bounding-box regression heads. We argue that this query formulation has two architectural advantages. First, objectness modeling at the query level naturally separates background noise from known and unknown objects. Secondly, the decoder structure provides a modular interface, and a lightweight adaptation module is inserted to inject task-specific behaviour without changing the backbone. These properties make Transformer detectors ideal for parameter-efficient adaptation in the open world. Parameter-Efficient Architecture Parameter-Efficient Learning Constraint. To resolve the conflict between stability and plasticity, we impose a strict parameter-efficiency constraint on the detector. We divide the model parameters into two disjoint subsets: the subset \(\:{\theta\:}_{frozen}\) is the backbone and the Transformer encoder, both pre-trained on large datasets. We keep them fixed throughout learning the process. The subset \(\:{\theta\:}_{adapt}\:\) consists of the light adaptation modules and task-specific prediction heads, which are the only parts of the network updated at some point. Learning for a task is reduced to optimizing \(\:{\theta\:}_{adapt}\) while \(\:{\theta\:}_{frozen}\) remaining unchanged. This is clear inductive bias: strong visual representations are retained, and task adaptations are handled by low-capacity controlled updates. Experimentally, we find that this constraint limits representation drift and reduces catastrophic forgetting without additional replays[ 26 , 27 ]. Residual Adapter Architecture. To make the adaptation efficient, we put Residual Adapters inside the Transformer decoder layers. The adapter's structure is a simple bottleneck. It has three steps: a down-projection to reduce dimensionality, an activation function, and an up-projection to restore the dimensionality. Mathematically, if \(\:h\) is the input embedding, the adapter works like this: $$\:\mathcal{A}\left(h\right)={W}_{up}\sigma\:\left({W}_{down}h\right)$$ Here, \(\:{W}_{down}\) and \(\:{W}_{up}\) are the projection matrices, and \(\:\sigma\:\) is the activation. The dimension inside the adapter is much smaller than the input dimension. We add this output back to the original features using a residual connection: $$\:{h}^{{\prime\:}}=h+{\text{FFN}}_{frozen}\left(h\right)+\lambda\:\cdot\:\mathcal{A}\left(\text{LN}\right(h\left)\right)$$ In this formula, we only train the adapter \(\:\mathcal{A}\) . The scale factor \(\:\lambda\:\) helps control how much the adapter changes the features. This keeps the original decoder working while allowing small changes for the open-world task. The details are presented in Fig. 2 . Semantic-Aligned Initialization Leveraging Vision–Language Models for Semantic Priors. To fix this, we look at Vision-Language Models like CLIP [ 28 ]. These models are trained on massive amounts of image-text pairs. They learn a shared space where the image of an object and its textual description are close together. \(\:{C}_{k}\) , we use this property to give our classifier a head start: we take the list of class names for the task and \(\:{C}_{k}\) make simple text prompts for them, like “a photo of a \(\:\:{t}_{k}\) ”. We run these prompts through a frozen CLIP text encoder and get text embeddings that capture the real semantic relation (dogs are closer to cats than cars) and a strong prefix to have. Semantic-Aligned Initialization. We cannot use the text embeddings directly because their dimensions don't match our detector. So, we use a simple linear layout \(\:P\) to project them into the classification space: $$\:{w}_{k}=\text{Normalize}\left(P{t}_{k}\right)$$ Here, \(\:P\) acts as a bridge. We use these projected vectors \(\:{w}_{k\:}\) to initialize the weights of our classification head. This completely changed the starting point of the training. We do not start from random noise, but from the fact that the classifier already knows which classes are semantically similar. The geometric effect is very significant. Related categories tend to cluster together, while unrelated categories are far apart. Virtual Outlier Synthesis Energy-Based Perspective on Open-World Detection. To fix this, we look at energy-based models. Let \(\:z\) be the feature vector and \(\:{f}_{k}\left(z\right)\:\) be the logits. We calculate a "free energy" score: $$\:E\left(z\right)=-\text{l}\text{o}\text{g}\sum\:_{k=1}^{\mid\:C\mid\:}\text{e}\text{x}\text{p}({f}_{k}\left(z\right))$$ The intuition is simple: Low energy means the model is confident and comfortable (Known). High energy means the model is surprised or uncertain (Unknown). Our goal is to map all known objects to low-energy states and force everything else to have high energy. The issue is that standard training is greedy. It pulls down the energy for known data to minimize loss, but it doesn't care about the empty space around them. Without a push, the energy in the open space remains undefined. Outlier Generation in the Latent Space. Let \(\:{Z}_{k}\:\) be the features for a known class \(\:k\) . We estimate the shape of this class using its mean \(\:{\mu\:}_{k}\) and covariance \(\:{{\Sigma\:}}_{k}\) . We generate virtual outliers by sampling from a mixture distribution: $$\:{z}_{out}\sim\:\sum\:_{k=1}^{\mid\:C\mid\:}{\pi\:}_{k}\mathcal{N}({\mu\:}_{k},\alpha\:{{\Sigma\:}}_{k})$$ Here \(\:\alpha\:>1\) is a parameter that expands the variance. This formula ensures that the fake samples are close to the known classes (where the confusion happens) but just outside the dense cluster center. Why model queries as Gaussian distributions? Someone might ask why we assume a Gaussian shape. Unlike CNN feature maps, dense and spatially distributed, Transformer object queries are instance-specific. Each query represents one object hypothesis. These distributions are cleaner and smaller than pixel features. Synthesizing outliers based on query statistics enables precise regularization at the decision boundary. Energy Regularization Loss. For every virtual outlier \(\:\:{z}_{out}\) we create, we apply a loss function to push its energy up: $$\:{\mathcal{L}}_{VOS}={\mathbb{E}}_{{z}_{out}}[\text{m}\text{a}\text{x}(0,m-E\left({z}_{out}\right)\left)\right]$$ Where \(\:m\) is the energy margin we want to maintain? This explicitly carves out a high-energy region in the open space. The total loss for our model is: $$\:\mathcal{L}={\mathcal{L}}_{det}+{\lambda\:}_{vos}{\mathcal{L}}_{vos}+{\lambda\:}_{reg}{\mathcal{L}}_{reg}$$ We combine the standard detection loss, the VOS loss, and a regularization term for the adapters. Training and Inference Unified Training Objective. We train PE-OWOD end-to-end, strictly adhering to a frozen-representation regime. Upon receiving an input image \(\:x\) , the detector generates object predictions through the standard Transformer decoding process. We structure the global training objective around three complementary components: $$\:\mathcal{L}={\mathcal{L}}_{det}+{\lambda\:}_{vos}{\mathcal{L}}_{vos}+{\lambda\:}_{reg}{\mathcal{L}}_{reg}$$ The first term \(\:{\mathcal{L}}_{det}\:\) captures the standard set-prediction loss native to Deformable DETR, such as focal loss for classification and L1/GIoU measures for bounding-box regression. The term \(\:{\mathcal{L}}_{vos}\:\) implements energy-based regularization discussed in Section 3.4; functionally, it wedges queries for known objects with queries for unknown objects. We also introduce \(\:{\mathcal{L}}_{reg}\:\) an L2 penalty on the trainable parameters of the residual adapters. We find regularization necessary to prevent overfitting during early optimization stages. By combining these goals, the model learns to maintain accurate localization of known categories while tolerating new anomalies. Incremental Learning Protocol. We designed PE-OWOD to handle incremental learning as our core skill \(\:t\) . When starting a new task, adaptation is straightforward: encode input class names with the CLIP text encoder and initialize the classifier weights via semantic projection. Rest adapters remain tuned to new task data. One of the main advantages is that there are no complex mechanisms such as replay buffers or feature distillation. Since frozen representations are based on knowledge from previous tasks, features can not be forgotten. PE-OWOD memory is efficient and can handle long sequences of tasks without a huge computational cost. Inference and Unknown Object Identification. At the inference stage, PE-OWOD operates within the standard DETR decoding pipeline. For each object query, the system produces two outputs: a bounding-box regression and classification logits for known classes. We identify unknown objects by calculating the energy score derived in Section 3.4: $$\:E\left(z\right)=-\text{l}\text{o}\text{g}\sum\:_{k=1}^{\mid\:\mathcal{C}\mid\:}\text{e}\text{x}\text{p}({f}_{k}\left(z\right))$$ Functionally, we label a prediction unknown if its energy level exceeds the threshold. We calibrate the threshold on validation set and keep it constant across subsequent tasks. An energy-based measure is more appropriate than heuristic confidence thresholds because it provides a grounded and transparent way to distinguish known prediction from unseen predictions. Experimental Setup Datasets. We train our experiments on the MS-COCO 2017 set [ 29 ] with 80 objects categories in complex scenes. We use MS-COCO as a proxy for open-world complexity because real-world deployment scenarios often have similar long-tail distributions and different environments. The data set is high intra-class variance, high overlap between classes, and frequent co-occurrence of known and unknown objects in the same image. Except where otherwise stated, we train on train2017 set and evaluate on val2017 set. Open-World Task Protocol . We adopt the widely used OWOD task protocol introduced by OW-DETR. We partition the 80 COCO classes into four incremental tasks. The specific class partition for each task is detailed in Supplementary Table S1 . Task 1 introduces the 20 VOC classes (e.g., Airplane, Bicycle) as known. Tasks 2, 3, and 4 each introduce 20 additional classes (Outdoor, Accessories, and Indoor items, respectively). We follow the standard setting where unknown classes of \(\:{U}_{t}\) in the current task become known in the next tasks. Crucially, at each task \(\:t\) , only classes of the current known set \(\:{K}_{t}\:\) are annotated. Objects belonging to future classes \(\:{U}_{t}\:\) are present in images but not annotated. The detector must identify unknown objects. The test measures the system's ability to find unknown objects at each step and to acquire knowledge without catastrophic forgetting. Evaluation Metrics. We assess performance using two distinct metrics: closed-set accuracy and open-set sensitivity. For known-class detection, we report the Mean Average Precision (mAP) at 0.5 IoU, the standard for evaluating whether the model correctly discriminates between previously learned categories. To measure Unknown Object Discovery, we use U-Recall, which counts the number of unlabeled (unknown) object instances recovered by the detector, and Absolute Open Set Error (A-OSE). A lower A-OSE indicates that the model distinguishes new instances from known classes rather than hallucinating them and classifying them as something it has seen before. Baselines. We compare PE-OWOD with three state-of-the-art methods: (1)ORE, a CNN-based method with explicit unknown modeling; (2)OW-DETR, a Transformer-based implementation using attention scores for open-world environments; and (3)PROB, a probabilistic method with boundary regularization. To ensure fair comparisons, we built all models on a unified DETR-based pipeline. We used standard OWOD protocols and ensured that all training, validation, and evaluation splits were identical across methods. Implementation Details and Reproducibility. We built our architecture on a Deformable DETR base with a ResNet-50 backbone. To enforce the parameter-efficient constraint, we initialized both the backbone and Transformer encoder with COCO-pretrained weights and locked them immediately. Only the classification heads, bounding box regressors, and the Residual adapters we inserted are trainable. We optimize the model using AdamW [ 30 ] with weight decay. The batch size is set to 8 across GPUs. We set specific learning rates for trainable adapter modules and different learning rates for class projection layers. We train each incremental task for 50 epochs, as we found this convergence time sufficient to fit the small sample sizes of the new classes. All input images are preprocessed with random resizing and horizontal flipping. Virtual Outlier Synthesis (VOS) is implemented directly in the training loop without external information; since VOS generates samples on the fly, no information is passed to future tasks. To facilitate reproducibility, we fixed random seeds for all experiments and report averaged results over multiple runs where applicable. We commit to releasing our complete codebase and pre-trained model weights upon publication. Declarations Data and code availability The datasets generated and/or analysed during the current study are available in the MS-COCO repository, https://cocodataset.org . The custom code and pre-trained models for PE-OWOD are available in the GitHub repository, 1ch111/PE-OWOD. Funding No funding was received for this work. Acknowledgements 2024 Guangdong Provincial Key Discipline Research Capacity Enhancement Project (Project No.: 2024ZDJS067); 2023 Guangdong Provincial University Innovation Team Project (Project No.: 2023KCXTD046); 2022 Guangdong Provincial Key Discipline Research Capacity Enhancement Project (Project No.: 2022ZDJS147); 2021 Guangdong Provincial Key Discipline Research Capacity Enhancement Project (No.: 2021ZDJS116); 2022 Guangdong University of Science and Technology Institutional Youth Research Project (No.GKY-2022KYYBK-30) Author Contributions J.G. conceived the original idea of Parameter-Efficient Open-World Object Detection, designed the Residual-Adapter architecture and the VOS mechanism, developed the source code, performed all experiments on MS-COCO, and wrote the main manuscript. Y.Z. assisted in the experimental validation and verified the efficiency analysis. Y.L. contributed to the visualization of t-SNE and energy distribution figures. C.L. assisted in data preparation and baseline comparisons. S.G. supervised the entire project, provided theoretical guidance, and revised the manuscript. L.L. provided critical feedback and improved the paper's organization. All authors reviewed and approved the final manuscript. Additional Information Competing Interests: The authors declare no competing interests. References Ren, S., He, K., Girshick, R., Sun, J. & Faster, R-C-N-N. Towards real-time object detection with region proposal networks. Adv Neural Inf. Process. Syst 28 (2015). Redmon, J., Divvala, S., Girshick, R. & Farhadi, A. You only look once: Unified, real-time object detection. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit. 779–788 (2016). Lin, T. Y., Goyal, P., Girshick, R., He, K. & Dollár, P. Focal loss for dense object detection. In Proc. IEEE Int. Conf. Comput. Vis. 2980–2988 (2017). Joseph, K. J., Khan, S., Khan, F. S. & Balasubramanian, V. N. Towards open world object detection. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. 5830–5840 (2021). Carion, N. et al. Springer,. End-to-end object detection with transformers. In Eur. Conf. Comput. Vis. 213–229 (2020). Gupta, A. et al. OW-DETR: Open-world detection transformer. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. 9235–9244 (2022). Wang, Y., Zhang, Z., Zhao, S., Buramck, L. & Ebrahimi, S. Randomized proposals for open-world object detection. In Eur. Conf. Comput. Vis. 209–226Springer, (2022). Wang, Z., Li, Y., Guo, X., Fang, L. & Wang, S. Probabilistic two-stage open-world object detection. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. 2022–2032 (2023). Wu, X., Peng, D., Zhang, P., Yu, L. & Yang, K. Two-stage open-world object detection with unknown-aware query mechanism. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. 1134–1143 (2023). Zhao, S. et al. Springer,. Exploiting unlabeled data with vision and language models for object detection. In Eur. Conf. Comput. Vis. 159–175 (2022). Wu, Y., Han, X., Li, G., Chang, M. & Yan, Y. Open-world object detection via discriminative class prototype learning. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. 2259–2269 (2023). Li, L. H. et al. Grounded language-image pre-training. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. 10965–10975 (2022). Gu, X., Lin, T. Y., Kuo, W. & Cui, Y. Open-vocabulary object detection via vision and language knowledge distillation. In Int Conf. Learn. Represent (2022). Houlsby, N. et al. Parameter-efficient transfer learning for NLP. In Int Conf. Mach. Learn 2790–2799 (2019). Hu, E. J. et al. LoRA : Low-rank adaptation of large language models. In Int Conf. Learn. Represent (2022). Jia, M. et al. Springer,. Visual prompt tuning. In Eur. Conf. Comput. Vis. 709–727 (2022). Chen, S. et al. Adapting vision transformers for scalable visual recognition. Adv. Neural Inf. Process. Syst. 35 , 16664–16678 (2022). AdaptFormer. Chen, Z. et al. Vision transformer adapter for dense predictions. In Int Conf. Learn. Represent (2023). Zhai, M. et al. Hyper-Adapter: Parameter-Efficient Fine-Tuning for Vision Transformers. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (2024). Du, X., Wang, Z., Cai, M. & Li, Y. VOS: Learning what you don't know by virtual outlier synthesis. In Int. Conf. Learn. Represent. (2022). Bendale, A. & Boult, T. E. Towards open set deep networks. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit. 1563–1572 (2016). Tack, J., Mo, S., Jeong, J. & Shin, J. C. S. I. Novelty detection via contrastive learning on distributionally shifted instances. Adv. Neural Inf. Process. Syst. 33 , 11839–11852 (2020). Liu, W., Wang, X., Owens, J. & Li, Y. Energy-based out-of-distribution detection. Adv. Neural Inf. Process. Syst. 33 , 21464–21475 (2020). Zhu, X. et al. Deformable DETR: Deformable transformers for end-to-end object detection. In Int Conf. Learn. Represent (2021). He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit. 770–778 (2016). Rebuffi, S. A., Kolesnikov, A., Sperl, G., Lampert, C. H. & iCaRL Incremental classifier and representation learning. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit. 2001–2010 (2017). Yan, S., Xie, J. & He, X. D. E. R. Dynamically expandable representation for class incremental learning. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. 3014–3023 (2021). Radford, A. et al. Learning transferable visual models from natural language supervision. In Int Conf. Mach. Learn 8748–8763 (2021). Lin, T. Y. et al. Microsoft COCO: Common objects in context. In Eur. Conf. Comput. Vis. 740–755 (2014). Loshchilov, I. & Hutter, F. Decoupled weight decay regularization. In Int Conf. Learn. Represent (2019). Additional Declarations No competing interests reported. Supplementary Files SupplementaryInformation.pdf Cite Share Download PDF Status: Under Review Version 1 posted Editorial decision: Revision requested 02 Mar, 2026 Reviews received at journal 27 Feb, 2026 Reviews received at journal 26 Feb, 2026 Reviewers agreed at journal 24 Feb, 2026 Reviewers agreed at journal 22 Feb, 2026 Reviewers agreed at journal 21 Feb, 2026 Reviewers agreed at journal 20 Feb, 2026 Reviewers agreed at journal 19 Feb, 2026 Reviewers invited by journal 19 Feb, 2026 Editor invited by journal 17 Feb, 2026 Editor assigned by journal 03 Feb, 2026 Submission checks completed at journal 03 Feb, 2026 First submitted to journal 01 Feb, 2026 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8754841","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":595305416,"identity":"998baaf6-4fac-44a7-a61b-ba648aab9b8b","order_by":0,"name":"Jiaming Gu","email":"","orcid":"","institution":"Guangdong University of Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Jiaming","middleName":"","lastName":"Gu","suffix":""},{"id":595305419,"identity":"f1c0f8ef-eedd-4fe3-8ec9-727cc6362447","order_by":1,"name":"Yehui Zheng","email":"","orcid":"","institution":"Guangdong University of Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Yehui","middleName":"","lastName":"Zheng","suffix":""},{"id":595305427,"identity":"3bcef7f6-c078-4861-a9e3-38d560a08e16","order_by":2,"name":"Yuzhou Liu","email":"","orcid":"","institution":"Guangdong University of Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Yuzhou","middleName":"","lastName":"Liu","suffix":""},{"id":595305433,"identity":"0d3e92c5-35f6-434d-a494-9549f50f3e7a","order_by":3,"name":"Caimei Liu","email":"","orcid":"","institution":"Dongguan University of Technology","correspondingAuthor":false,"prefix":"","firstName":"Caimei","middleName":"","lastName":"Liu","suffix":""},{"id":595305437,"identity":"1b99bab1-aa69-4b74-8b7d-2b90af82be47","order_by":4,"name":"Shu Gong","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA4ElEQVRIiWNgGAWjYLACxgYGOQaGBBibSC3GYC0HSNGS2EC0Fnn33sMvf+44nD7fPcf48wcGG9kNB5ifPcCnxfDMuTRr3jOHczeeeWMmcYAhzXjDATZzA7xaZuSYGTO2AbUAGUCHHU7ccICHTYKQFsOfbYfTgQzjDwcY/hPWIi+RY/yAt+1wApBhAHTYAcJaDHjOmDHztqUbbuB5ViZxxiDZeOZhNjP8trT3GH/82WYtL9+evPlDRYWdbN/x5mf4bTnAAHEGkAEigZgZn3qQLQ0MzB+gjFEwCkbBKBgF2AEA0FpP0qEZQfAAAAAASUVORK5CYII=","orcid":"","institution":"Guangdong University of Science and Technology","correspondingAuthor":true,"prefix":"","firstName":"Shu","middleName":"","lastName":"Gong","suffix":""},{"id":595305439,"identity":"9aaf3279-5b04-4e9f-9d4f-699315f472db","order_by":5,"name":"Luoyang Luo","email":"","orcid":"","institution":"Guangdong University of Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Luoyang","middleName":"","lastName":"Luo","suffix":""}],"badges":[],"createdAt":"2026-02-01 08:38:46","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-8754841/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8754841/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":103325210,"identity":"6c1eaa44-5866-4ee5-a614-4c423a42ae21","added_by":"auto","created_at":"2026-02-24 12:42:39","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":91399,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eBreaking the Efficiency–Performance Barrier. \u003c/strong\u003eMost OWOD methods (Blue circles) occupy the bottom-right: they demand heavy GPU memory for modest unknown discovery. PE-OWOD (Red star) flips this dynamic. By freezing representations, we achieve high Unknown Recall (64.7%) with minimal memory footprint (2.2GB), proving that high performance does not require retraining the entire model.\u003c/p\u003e","description":"","filename":"floatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-8754841/v1/2eea54699c66c978609b5ba6.png"},{"id":103325215,"identity":"6b389e57-da9a-42bd-815a-0a637e850233","added_by":"auto","created_at":"2026-02-24 12:42:39","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":2042089,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eQualitative detection results on challenging OOD scenarios. (a) \u003c/strong\u003eResults from the baseline OW-DETR (SOTA). \u003cstrong\u003e(b) \u003c/strong\u003eResults from our method (PE-OWOD). The comparison highlights the differences in confidence scores for known classes and the ability to identify fine-grained unknown objects (marked in red).\u003c/p\u003e","description":"","filename":"floatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-8754841/v1/535549dd457965a6ad49f366.png"},{"id":103506730,"identity":"098e9bd0-aa32-44de-b79d-19399ff69602","added_by":"auto","created_at":"2026-02-26 13:39:15","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":121070,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eIncremental Learning Stability.\u003c/strong\u003e \u003cstrong\u003e(a)\u003c/strong\u003eUnknown recall on incremental tasks. While the OW-DETR does not expand with complexity, PE-OWOD (red star line) is stable and able to learn unknown. \u003cstrong\u003e(b)\u003c/strong\u003eStability of known-class mAP. The dotted line shows catastrophic forgetting of fully fine-tuned methods, while PE-OWOD (red solid line) is stable due to frozen backbone.\u003c/p\u003e","description":"","filename":"floatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-8754841/v1/0ec3f4dce1a16cc95f6e7855.png"},{"id":103506692,"identity":"db96b942-40dd-4792-b06a-e71cf816ac7f","added_by":"auto","created_at":"2026-02-26 13:39:02","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":125543,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eSeeing the Difference via t-SNE visualization. (a) \u003c/strong\u003eBaseline model feature space: Unknown objects (red) and Known objects (blue) are mixed. \u003cstrong\u003e(b)\u003c/strong\u003ePE-OWOD feature space: The VOS module enforces a clear separation between known clusters and unknown outliers.\u003c/p\u003e","description":"","filename":"floatimage4.png","url":"https://assets-eu.researchsquare.com/files/rs-8754841/v1/b569998a8f81d03f5cee1b06.png"},{"id":103325214,"identity":"ee43e813-97a9-444b-8dd9-7ffa4955fcee","added_by":"auto","created_at":"2026-02-24 12:42:39","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":123816,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eThe PE-OWOD Architecture. \u003c/strong\u003eWe freeze the heavy backbone and encoder (Grey) to maintain strong visual priors. Light Residue Adapters (Orange) are added directly to Transformer decoder layers. This lock-and-key design allows the model to adapt to new tasks and generate virtual Outliers (Green box) without losing the global feature geometry.\u003c/p\u003e","description":"","filename":"floatimage5.png","url":"https://assets-eu.researchsquare.com/files/rs-8754841/v1/095f5082ac6e7e72f44f5746.png"},{"id":103509841,"identity":"87e69cd5-52f3-4991-b6cd-beb5f57f3f0d","added_by":"auto","created_at":"2026-02-26 14:01:41","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":3731793,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8754841/v1/cd54ba5c-3920-43ee-a478-91cb4fd4a714.pdf"},{"id":103325212,"identity":"6ed291a2-8c4a-48f7-a48f-b569bf154d7b","added_by":"auto","created_at":"2026-02-24 12:42:39","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":457240,"visible":true,"origin":"","legend":"","description":"","filename":"SupplementaryInformation.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8754841/v1/d07f6604a196b0f56bd87498.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"PE-OWOD: Parameter-Efficient Open-World Detection with Semantic Priors and Virtual Outlier Synthesis","fulltext":[{"header":"Introduction","content":"\u003cp\u003eThe visual perception system is confronted with the fact that standard datasets only show that the world is static. In some applications, such as autonomous driving or service robots, new object categories are constantly emerging. This dynamic shift breaks the \"closed-world\" assumption of traditional detectors [\u003cspan additionalcitationids=\"CR2\" citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e]. Standard detectors like Faster R-CNN and YOLO operate under the closed-set assumption: they classify only what they have seen in training, yet new concepts emerge unexpectedly. To solve this issue, Joseph et al. [\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e] formalized the Open-World Object Detection (OWOD) task: a detector must recognize known classes while simultaneously identifying unknown instances, all without forgetting previously learned information.\u003c/p\u003e \u003cp\u003eThis problem is not solved, but it is mostly due to the uncertainty of the supervision signal. Training images contain unknown objects, i.e., the background. Detectors often assign high confidence to unknown regions, assuming they are known classes, but in safe settings, such \u0026ldquo;confident errors\u0026rdquo; are dangerous. Although earlier methods, such as ORE [\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e], were based on adapting region proposal schemes, Transformers have become the most popular [\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e]. OW-DETR [\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e] tried to do so using object queries and attention scores to spot unknown objects without supervision. Other models improved this model: RandBox [\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e] is based on random sampling, and PROB [\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e] introduces probabilistic uncertainty to separate objectness from class probability [\u003cspan additionalcitationids=\"CR10\" citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eHowever, we argue that current approaches have significant limitations. Most existing OWOD frameworks, such as OW-DETR, rely on end-to-end fine-tuning. While this strategy works on closed benchmarks, it is counterproductive for the open world. First, updating the backbone changes the feature geometry. In general, decisions of known classes are extended to the open space, making the model less sensitive to new objects. Second, it costs too much. Full fine-tuning takes big GPUs and long training cycles, and edge devices cannot be resource-limited. Recent frameworks are becoming increasingly complex, requiring heavy modules or external supervision (OW-OVD [\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e, \u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e]). Figure\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e shows the imbalance: current methods only get unknown discovery at a high cost.\u003c/p\u003e \u003cp\u003eWe challenge the assumption that open-world learning requires updating the whole network. If pre-trained backbones already contain rich semantic information, the failure to detect unknown objects is likely a boundary alignment issue rather than a representation deficit. This brings us to Parameter-Efficient Fine-Tuning (PEFT). Full tuning is becoming increasingly unstable as models grow in size. In NLP, Adapters [\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e] and LoRA [\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e] show that updating only 1% of parameters matches full tuning and even beats it. Inspired by NLP, Vision has started using PEFT. Methods such as VPT [\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e] and AdaptFormer [\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e] have applied prompts and adapters to Vision Transformers (ViT) [\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e, \u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e]. However, PEFT is not trivial in the open world. Naive backbone freezing yields a rigid feature space in which unknown objects blend into the background.\u003c/p\u003e \u003cp\u003eTo address this, we propose PE-OWOD, a parameter-efficient framework that avoids heavy retraining. Instead of free-flow fine-tuning, we adopt a \"Lock-and-Key\" design. First, we freeze the heavyweights: We lock the backbone and encoder to preserve stable visual priors. Second, we adapt the Head: We introduce lightweight Residual Adapters only in the decoder to handle task-specific plasticity. This updates less than 2% of the model parameters. We think PEFT should not be treated as a shortcut, but as a regularization mechanism.\u003c/p\u003e \u003cp\u003eFreezing the backbone adds a new risk: \u0026ldquo;cold start\u0026rdquo; misalignment between frozen features and new classifiers. We solve this problem using Semantic Aligned Initialization (using CLIP priors) and Virtual Outlier Synthesis (VOS) [\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e]. Unknown object detection in OWOD is based on out-of-distribution (OOD) detection [\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e, \u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e]. Early OOD methods used Softmax confidence scores, but these scores are often unreliable. Energy-Based Models (EBMs) [\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e] map logits to free energy scores, which are more closely related to data density. The core challenge is the lack of negative samples. VOS addresses this in classification tasks by synthesizing virtual outliers in low-density regions. Since VOS does not detect objects because object queries are structured, we combine VOS directly with the Transformer decoder. Using virtual queries in the hidden space, PE-OWOD compresses known classes into an energy valley, defining an explicit decision boundary for the open space.\u003c/p\u003e \u003cp\u003eOur approach offers three core contributions. Firstly, it has the robustness of frozen representations: we have demonstrated that complete model tuning is not required. Even with frozen backbone and encoder, PE-OWOD achieves 64.7% Unknown Recall, far above fully fine-tuned baselines. Second, high computational efficiency: By updating only 1.8% of the model parameters, we reduce peak GPU memory usage by 86% and training time by 77%. Third, a new paradigm for stability: We show parameter-efficient adaptation is not just a speed compromise. By locking visual priors and modeling open space explicitly (via VOS), we solve the stability\u0026ndash;plasticity problem more effectively than standard end-to-end training.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e"},{"header":"Results","content":"\u003cp\u003e \u003cb\u003e1: The PE-OWOD Framework\u003c/b\u003e \u003c/p\u003e \u003cp\u003e \u003cb\u003eRevisiting Full Fine-Tuning under the Open-World Setting.\u003c/b\u003e Most current methods for open-world detection try to fine-tune the whole model. They update the backbone, encoder, and decoder simultaneously. This works well for closed settings where we know all the classes, but it causes problems in the open world.\u003c/p\u003e \u003cp\u003eThe main problem is that we can't control it. In OWOD, the training images are unknown objects, but we don't have labels for them. The loss function knows objects and pushes the model to treat them as background. If we update the whole network, the model is too free. It overfits known classes and extends decision boundaries into the open space. This results in Open Space Risk, where the model says it assigns extremely high confidence that an unknown object is a dog or a car.\u003c/p\u003e \u003cp\u003eA modern backbone trained on large datasets knows what \u0026ldquo;objects\u0026rdquo; look like. If we change their weights just to learn some new classes, we destroy the good features, catastrophic forgetting, and make the model poor at finding new things. We think we can learn everything, but we just need to change the boundary.\u003c/p\u003e \u003cp\u003e \u003cb\u003eDesign Principle: Low-Dimensional Adaptation for OWOD.\u003c/b\u003e We assume that the changes needed for OWOD are actually quite small. We don't need to change millions of parameters. Recent research on parameter-efficient tuning shows that we can learn new tasks by updating only a tiny part of the model.\u003c/p\u003e \u003cp\u003eWe like this idea for open-world detection for two reasons. One is boundary realignment, since we do not want to learn to see edges or textures again, but rather to shift the classification boundary so that unknown objects can be found. The second reason is the stability-plasticity balance: We want model changes sufficient to learn new classes (plasticity) but not so much as to forget old classes (stability). We restrict updates to a small set of parameters.\u003c/p\u003e \u003cp\u003eBased on this, our design is simple: we lock the backbone and the Transformer encoder, and we only add small adapter modules to the detection head.\u003c/p\u003e \u003cp\u003e \u003cb\u003eMotivation: Semantic Misalignment under Frozen Representations.\u003c/b\u003e We freeze the backbone and the Transformer encoder to stabilize the model. But this choice creates a problem: \"semantic misalignment.\" The visual features from the backbone are fixed, but the weights of the classification head are usually initialized randomly.\u003c/p\u003e \u003cp\u003eIn a normal detector, backpropagation updates everything together. The head learns how features match. In a frozen setup, the head learns how features map and operates on a fixed distribution. This \u0026ldquo;cold start\u0026rdquo; slows optimization. Gradients are noisy at the start, and classes are hard to separate for new categories with few examples.\u003c/p\u003e \u003cp\u003eThis is not allowed in open-world detection. Unknown objects are labeled, but we can't tell model where they belong. If the classifier starts randomly, it could easily assign an unknown object to a known class by chance. We conclude that parameter efficient adapters alone are not enough. We need to give the classifier some meaning before training.\u003c/p\u003e \u003cp\u003e \u003cb\u003eMotivation: Why Semantic Alignment Is Not Enough.\u003c/b\u003e The semantic initialization we talked about in Section 3.3 helps organize known classes, but it knows not what an \"unknown\" object is. If the model separates unknowns, it's a lucky side effect of semantic structure, not a guarantee. We argue that luck is not enough for reliable detection.\u003c/p\u003e \u003cp\u003eThe problem is that unknown objects are invisible when training; we never label them, and we never penalize the model for missing them. So the model guesses. Even with CLIP priors, the classifier gives high confidence scores for unknown objects because they look slightly similar to known classes. Hence, explicit regularization is needed. Force the model to say \"I don't know\" when seeing something new.\u003c/p\u003e \u003cp\u003eWe agree that feature distributions may not be Gaussian in real life, but we view Gaussian modeling as useful. We think it is an effective solution to make a buffer in frozen representation space, even if the theory is not perfect.\u003c/p\u003e \u003cp\u003e \u003cb\u003e2: Performance on Open-World Object Detection\u003c/b\u003e \u003c/p\u003e \u003cp\u003eIn this section, we will break down the experimental results of the MS-COCO benchmark. We strictly adhere to the Open-world protocol and primarily compare the model's ability to detect known classes, discover unknown classes, and be robust to open-set errors.\u003c/p\u003e \u003cp\u003e \u003cb\u003eOverall Performance Comparison.\u003c/b\u003e We present the average precision (mAP) of the known classes and the U-Recall of the unknown objects (Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e). The results are very interesting. Even when PE-OWOD is completely frozen, its U-Recall (64.7%vs 5.8%) is still 10 times higher than that of the fully tuned OW-DETR, indicating that we can still find new objects even with limited resources.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eState-of-the-art comparison on MS-COCO (Task 1)\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"5\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMethod\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eBackbone\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003emAP\u0026nbsp;(Known) \u0026uarr;\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eU-Recall (Unknown) \u0026uarr;\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eA-OSE \u0026darr;\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eORE\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eResNet-50\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e15.2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e2.5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e2400\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eOW-DETR\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eResNet-50\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e29.5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e5.8\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e1250\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePROB\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eResNet-50\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e28.1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e14.3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e1100\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePE-OWOD (Ours)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eFrozen ResNet\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e21.4\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e64.7\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e673\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003ctfoot\u003e \u003ctr\u003e\u003ctd colspan=\"5\"\u003e\u003cem\u003eNote: \u0026ldquo;mAP\u0026rdquo; denotes Mean Average Precision at IoU\u0026thinsp;=\u0026thinsp;0.5; \u0026ldquo;U-Recall\u0026rdquo; denotes Unknown Recall; \u0026ldquo;A-OSE\u0026rdquo; denotes Absolute Open-Set Error (lower is better). Best results are marked in bold.\u003c/em\u003e\u003c/td\u003e\u003c/tr\u003e \u003c/tfoot\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eBased on the trade-off between precision and recall, we should examine it here. We admit that PE-OWOD has a lower mAP for known classes than a fully tuned baseline (21.4 vs. 29.5), but we do not see this as a failure; rather, it is a different choice on the efficiency-performance curve.\u003c/p\u003e \u003cp\u003eThere are two reasons why this behavior is undesirable. One is that full fine-tuning maximizes fit for known classes by changing feature representations. It \u0026ldquo;overwrites\u0026rdquo; the model's ability to see new things, and reduces Unknown Recall to 5.8%. We prioritize new-case discovery (64.7% Recall) while maintaining reasonable accuracy for known classes.\u003c/p\u003e \u003cp\u003eThe other reason is the backbone capacity. We trained a simple frozen model without our adapters, achieving the same mAP (21.9) but lower Unknown Recall (26.3%), indicating that the dip in known-class accuracy is due to the frozen backbone itself (COCO shift limits data shift) rather than our modules missing something. Importantly, PE-OWOD opens the full potential of the frozen backbone in the open world.\u003c/p\u003e \u003cp\u003e \u003cb\u003eUnknown Object Discovery.\u003c/b\u003e In every task we tested, PE-OWOD maintains a strong lead in U-Recall. This advantage gets bigger in the later stages. As the number of unknown categories grows and the risk of open-space confusion increases, the gap between our method and the baselines widens.\u003c/p\u003e \u003cp\u003eWe like to compare with OW-DETR. We need to fine-tune fully, but beat it by a large margin. Only a small fraction of parameters were updated. Explicit open-space modeling, combined with parameter efficiency, is much more efficient than allowing the model to fine-tune itself without constraints.\u003c/p\u003e \u003cp\u003eWe also asked: Is this gain just because we froze the backbone? We looked at the control experiment again. Freezing the backbone does help lift Unknown Recall to 26.3% (up from 5.8%), but that is still far short of the 64.7% we get with the full PE-OWOD framework.\u003c/p\u003e \u003cp\u003eIt is also worth mentioning PROB. While it uses energy-based regularization to find unknowns, it tends to lose accuracy on known classes as tasks progress. PE-OWOD walks the line better. It discovers the unknown without forgetting the known.\u003c/p\u003e \u003cp\u003e \u003cb\u003eOpen-Set Robustness.\u003c/b\u003e The Absolute Open-Set Error (A-OSE) tells us how often the detector \"hallucinates\"\u0026mdash;that is, how often it mistakes an unknown object for a known one. As shown in Supplementary Table S2, PE-OWOD consistently has lower A-OSE scores than the baselines.\u003c/p\u003e \u003cp\u003eSuch a low error rate indicates that our model is not overly confident. We believe this is the result of the combination of semantic initialization and virtual outlier synthesis. Together, they shaped an energy landscape in which the model knows when it is uncertain, a difficult feat with standard training.\u003c/p\u003e \u003cp\u003e \u003cb\u003eKnown-Class Detection Performance.\u003c/b\u003e Many people may think that locking the backbone and encoder would undermine detection accuracy, but PE-OWOD remains very competitive. In some tasks, it even goes beyond the fully tuned baseline. This indicates that maintaining the integrity of pre-trained representations actually helps with generalization.\u003c/p\u003e \u003cp\u003eMore importantly, when new tasks are added, our method incurs very little performance loss. Resistance to catastrophic forgetting supports our main hypothesis: parameters effectively adapt between stability and plasticity, achieving a correct balance.\u003c/p\u003e \u003cp\u003e \u003cb\u003ePerformance\u0026ndash;Efficiency Trade-off.\u003c/b\u003e The value of PE-OWOD is not only reflected in the precision figures but also in improved efficiency. Unlike low-detr or PROB, which require retraining the entire network, we limit the update to 2% of the total parameters.\u003c/p\u003e \u003cp\u003eAs we detailed here, this can minimize GPU memory usage and training time. Moreover, these results challenge a common view that high-quality open-world detection requires expensive full-model fine-tuning. All of our explanations demonstrate that even on limited hardware, powerful deployments are possible.\u003c/p\u003e \u003cp\u003e \u003cb\u003eQualitative Results.\u003c/b\u003e Figure\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e shows qualitative detection results in challenging OOD scenarios.\u003c/p\u003e \u003cp\u003eTop row (Train scene) shows our method of Semantic-Aligned Initialization. PE-OWOD detects train with perfect confidence (1.00) far higher than baseline OW-DETR 0.88. Also thanks to VOS sharpening the boundary between objectness, we can find fine-grained unknown objects (for example overhead signal structures and poles (red)) that are not seen or loosely bound by the baseline.\u003c/p\u003e \u003cp\u003eBoth row (Stop Sign scene) shows an interesting failure case which shows both limitations and robustness of our method. The frozen backbone leads to semantic misclassification, which treats the Stop sign as Parking Meter (probably because it looks like pole mounted street objects). Energy-Guided criterion still works: identifies the complex bracket structure behind the sign as an Unknown object (red box). This indicates that even when semantic classification fails, our model still senses objectness and alerts users to anomalies.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003e \u003cb\u003e3: Incremental Learning Stability\u003c/b\u003e \u003c/p\u003e \u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003eIncremental Learning Stability\u003c/h2\u003e \u003cp\u003eHere, we analyzed the responsiveness of PE-OWOD to the appearance of unknown objects in real open-world scenarios, its robustness when learning new tasks, and its ability to balance memorizing old classes and discovering new ones.\u003c/p\u003e \u003cp\u003e \u003cb\u003eUnknown Recall in Cross-Incremental Tasks.\u003c/b\u003e The problem of OWOD is that unknown space shrinks as the model learns more. As the model learns more classes, the remaining open space shrinks. A true detector needs to keep finding new things even as the library of known classes grows. Figure\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e shows this well: While the baseline (OW-DETR) struggled with complexity, PE-OWOD is constant. The difference was largest in the last step, suggesting that our approach is much more effective. Even if the decision regions of known classes grow (crowding out unknowns), PE-OWOD manages the risk well. New knowledge does not suppress discovery of new objects.\u003c/p\u003e \u003cp\u003e \u003cb\u003eStability under Incremental Learning.\u003c/b\u003e Stability is critical; we need the model to learn new classes without destroying what it already knows. To measure this, we tracked the mAP of the initial known classes across all four tasks (Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003eb). This stability comes directly from freezing the backbone and encoder. Because the underlying representations never change, the model doesn't suffer from catastrophic forgetting at the feature level. We solve this problem structurally, without needing complex replay buffers or distillation losses.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003e \u003cb\u003e4: Computational Efficiency\u003c/b\u003e \u003c/p\u003e \u003cp\u003eIn this section, we analyze parameter count, training time, and memory usage to verify if the theoretical efficiency holds up in practice.\u003c/p\u003e \u003cp\u003e \u003cb\u003eParameter Efficiency and Resource Consumption.\u003c/b\u003e A major goal of our design was to keep the model light. \u003cb\u003eTable\u0026nbsp;2\u003c/b\u003ereports exactly how many parameters we are training compared to the baselines. While standard methods such as OW-DETR and PROB update all weights (42M parameters), PE-OWOD freezes the backbone and encoder and updates only the adapters. We update less than 1\u0026nbsp;million parameters (less than 2% of the total model size).\u003c/p\u003e \u003cp\u003eParameter count is just a proxy; what really matters is GPU memory and wall-clock time. Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e shows the gap: PE-OWOD reduces memory consumption by 86% (from 16.0 GB to 2.2 GB) and training time by 77%. Practically, this changes who can use the model. There is no longer a need for an enterprise cluster to train an open-world detector; it can be trained on a consumer GPU. We attribute this to two reasons: First, since we lock the backbone, we don't need gradients for most of the network. Second, fewer trainable parameters mean less overhead. These gains make the difference between a lab model and a deployable edge model.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eComparison of computational efficiency and resource consumption.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"7\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMethod\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eBackbone Status\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eTotal Params\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eTrainable Params\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eRatio\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003ePeak GPU Memory (GB) \u0026darr;\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e \u003cp\u003eTraining Time / Task (Hours) \u0026darr;\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eOW-DETR\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eFine-tuning\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e42 M\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e42 M\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e100%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e16.0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e~\u0026thinsp;200\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePROB\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eFine-tuning\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e42 M\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e42 M\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e100%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e16.5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e~\u0026thinsp;210\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePE-OWOD\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eFrozen\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e42 M\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u0026lt;\u0026thinsp;1 M\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e\u0026lt;\u0026thinsp;2%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e2.2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e~\u0026thinsp;45\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003ctfoot\u003e \u003ctr\u003e\u003ctd colspan=\"7\"\u003e\u003cem\u003eNote: Efficiency comparison performed on the same hardware setting. PE-OWOD significantly reduces memory usage and training time by freezing the backbone and encoder.\u003c/em\u003e\u003c/td\u003e\u003c/tr\u003e \u003c/tfoot\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003e \u003cb\u003eScalability and Energy Efficiency.\u003c/b\u003e We also consider the long game: What happens when the model learns continuously for months? Standard methods can get stuck with old data reprocessing. PE-OWOD breaks this cycle. Training costs are constant because we freeze the heavy backbone and train only adapters. Training cost is not increasing with the number of tasks, making the system extensible to long term deployment. Energy-wise, low overhead can transform models to run and update on resource-constrained platforms such as drones.\u003c/p\u003e \u003cp\u003e \u003cb\u003eVisualizing the Impact.\u003c/b\u003e Finally, PE-OWOD strikes the tricky balance between saving computing costs and providing powerful detection performance. Figure\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003e shows this feature space using t-SNE. Baseline chart (a) shows confusion where unknown objects (red) are scattered and mixed with known objects. Baseline chart (b) shows confusion where decision boundaries are tightened and unknown objects are shoved into separate low density areas. This shows that our parameter-efficient design allows clear separation without the expense of full fine tuning.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003e \u003cb\u003e5: Ablation Studies\u003c/b\u003e \u003c/p\u003e \u003cp\u003eWe disassemble PE-OWOD to accurately isolate the components and verify their contributions. All experiments follow the standard MS-COCO Open World Protocol.\u003c/p\u003e \u003cp\u003e \u003cb\u003eProgressive Improvement Analysis.\u003c/b\u003e We summarize the contributions of each component in Table\u0026nbsp;\u003cspan refid=\"Tab3\" class=\"InternalRef\"\u003e3\u003c/span\u003e. The results are quite interesting. First, freezing the backbone (Baseline) raises Unknown Recall to 26.3% whereas 5.8% for fully tuned models. The real breakthrough comes from adding Residual Adapters and semantic initialization (Row 2) which jumps U-Recall to 58.4%. Second, Virtual Outlier Synthesis (VOS) brings recall to 64.7% (+\u0026thinsp;6.3%) and reduces hallucination.This confirms our design philosophy: freeze the backbone for stability, add adapters for plasticity, and use VOS for safety.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab3\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eProgressive improvement analysis.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"4\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eConfiguration\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003emAP\u0026nbsp;\u0026uarr;\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eU-Recall \u0026uarr;\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eGain (Recall)\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eBaseline (Frozen)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e21.9\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e26.3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e+ Adapter \u0026amp; CLIP (Ver 3.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e21.7\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e58.4\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e+\u0026thinsp;32.1%\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e+ VOS (Ver 4.0 / Ours)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e21.4\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e64.7\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e+\u0026thinsp;6.3%\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003ctfoot\u003e \u003ctr\u003e\u003ctd colspan=\"4\"\u003e\u003cem\u003eNote: \u0026ldquo;Ver 3.0\u0026rdquo; includes Semantic-Aligned Initialization; \u0026ldquo;Ver 4.0\u0026rdquo; adds Virtual Outlier Synthesis (VOS). The progressive gain in U-Recall demonstrates the effectiveness of each component.\u003c/em\u003e\u003c/td\u003e\u003c/tr\u003e \u003c/tfoot\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003e \u003cb\u003eComponent-Level Analysis.\u003c/b\u003e We further investigate the specific impact of each module in the Supplementary Information.\u003c/p\u003e \u003cp\u003e \u003cstrong\u003eAdapter Placement\u003c/strong\u003e \u003cp\u003eAs detailed in Supplementary Table S4, we found that inserting adapters only into the decoder yielded the best results. Adapting the encoder disrupts global feature representations, leading to a drop in accuracy for the known class.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003cstrong\u003eAdapter Capacity\u003c/strong\u003e \u003cp\u003eWe also analyzed the adapter bottleneck dimension (Supplementary Figure S2). A moderate dimension (around 64 to 128) provides the optimal trade-off; very small values underfit, while excessively large values degrade generalization.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003cstrong\u003eInitialization Strategy\u003c/strong\u003e \u003cp\u003eSupplementary Table S5 shows that while CLIP-based initialization speeds up convergence (15 epochs), our architecture is robust enough to perform well even with standard Xavier initialization (15.4% U-Recall), demonstrating that the core benefit lies in the parameter-efficient structure itself.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003cstrong\u003eVOS Robustness\u003c/strong\u003e \u003cp\u003eFinally, we stress-tested the VOS module. It turns out PE-OWOD is not brittle; wide ranges of hyperparameters yield stable results, indicating that the gain is achieved geometrically rather than through hyper-tuning.\u003c/p\u003e \u003c/p\u003e \u003c/div\u003e"},{"header":"Discussion","content":"\u003cp\u003eWe proposed PE-OWOD to challenge the assumption that adaptation requires rewriting the whole model. Our results show that PE-OWOD solved the basic open-world detection problem by combining three advantages: parameter-efficient adapters provide flexibility, semantic initialization provides structure and VOS provides security. We solved catastrophic forgetting problem by Lock-and-Key design (freezing shared representations (Lock) and controlling adaptiveness with Lightweight Adapters (Key)) which reduces forgetting risk by more than 75%. Instead of hoping the model generalizes, we specify a safety zone for unknown objects via VOS, creating a probabilistic safety margin in the latent space that standard fine-tuning cannot achieve. As visualized in Supplementary Figure S4, this mechanism forces unknown objects into a distinct high-energy distribution, separating them from known classes and background noise.\u003c/p\u003e \u003cp\u003eHowever, we remain transparent about PE-OWOD's failures. A distinct failure mode occurs when an unknown object looks too much like a known one (e.g., fine-grained subclasses). This is not a fault of our architecture, but a mistake of the energy separation logic: if an unknown object is semantically too close to the feature space of a known class, the energy score will not soar. Additionally, our model struggles under extreme visual conditions, such as heavy occlusion or drastic lighting changes. Since we freeze the backbone to save memory, the low-level feature extraction is not adaptive to such large irregularities. This is the price for efficiency; the lock-and-key design offers stability and speed but cannot handle rapid distribution shifts as well as a fully tuned model. Furthermore, VOS relies on a geometric assumption that unknown objects lie near the edges of known class distributions. This heuristic works for standard benchmarks but may fail if an unknown class is globally distinct and far from the others in the latent space.\u003c/p\u003e \u003cp\u003eThese limitations point out the next development direction. First, the \"Semantic Ceiling\": we use CLIP priors to initialize the class, inheriting the pre-trained model's bias. Future work needs adaptive semantic representations rather than static text encoders. Second, the \"Stability-Plasticity Wall\": freezing the backbone works for incremental learning but fails under extreme distribution shifts. We need conditional feature adaptations that can selectively unfreeze parts of the backbone. Third, the \"Gaussian Simplification\": VOS assumes a Gaussian distribution for queries, but reality is messier. We believe that generative models or self-supervised density estimation could provide a more mathematically rigorous way to model the open space. Moving from academic benchmarks to real long-tail deployment remains the ultimate goal.\u003c/p\u003e \u003cp\u003eIn terms of broader impact, the primary motivation for OWOD is safety. If a self-driving car fails to spot an overturned truck simply because it wasn't in the training set, the results are catastrophic. Explicitly modeling \"unknowns\" adds a necessary safety layer to systems operating in the wild. We also aim for \"Green AI\" and parameter efficiency. By reducing computation costs by 75% (Supplementary Figure S3), we reduce the carbon footprint of model maintenance and enable advanced perception systems to run on edge devices such as drones. However, we must acknowledge the risk of privacy and surveillance; efficient, adaptable detectors could be used for intrusive surveillance, and we urge practitioners to adopt privacy-preserving protocols.\u003c/p\u003e \u003cp\u003eIn conclusion, large parameter updates are not sufficient for true open-world perception. PE-OWOD achieves an order-of-magnitude improvement in unknown recall over fine-tuned baselines, while reducing GPU memory usage by 86%. We concede that known class accuracy suffers from tradeoffs compared to fully updated models, but we argue that this compromise is necessary to achieve robust discovery in resource-limited environments.\u003c/p\u003e"},{"header":"Methods","content":"\u003cdiv id=\"Sec6\" class=\"Section2\"\u003e \u003ch2\u003eProblem Formulation\u003c/h2\u003e \u003cp\u003e \u003cb\u003eOpen-World Object Detection.\u003c/b\u003e We consider the Open-World Object Detection task as a learning process over a series \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\mathcal{T}=\\{{\\mathcal{T}}_{1},...,{\\mathcal{T}}_{K}\\}\\)\u003c/span\u003e\u003c/span\u003e. of tasks. At any \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:t\\)\u003c/span\u003e\u003c/span\u003e step, the model receives\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{\\mathcal{\\:}\\mathcal{D}}_{t}\\:\\)\u003c/span\u003e\u003c/span\u003eannotations pertaining only to \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{\\mathcal{K}}_{t}\\)\u003c/span\u003e\u003c/span\u003e. known classes, and any classes that have not been introduced are considered unknown sets \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{\\mathcal{U}}_{t}\\)\u003c/span\u003e\u003c/span\u003e .\u003c/p\u003e \u003cp\u003eAnother problem is \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{\\mathcal{U}}_{t}\\:\\)\u003c/span\u003e\u003c/span\u003ecomplexity of supervision. Cases that appear frequently in training images alongside known objects are not annotated, so standard supervised losses treat them as background noise. Hence, the detector must achieve two goals simultaneously: (1) Unknown discovery localizes potential objects without labels; (2) incremental adaptation updates the model to recognize new classes without removing previous knowledge.\u003c/p\u003e \u003cp\u003eUnlike closed-set detection, OWOD must operate under partial supervision. The system is forced to balance detecting known objects, discovering unknown instances, and preserving past knowledge.\u003c/p\u003e \u003cp\u003e \u003cb\u003eOpen-Space Risk and Decision Boundary Ambiguity.\u003c/b\u003e One of the main issues in OWOD is the lack of training data for known classes, leading to the open space problem. For closed-set detection, decision boundaries are implicitly restricted because the model is densely supervised for all classes. In contrast, the model is trained with negative supervision for unknown objects.\u003c/p\u003e \u003cp\u003eFrom a geometric perspective, the best possible risk for known classes allows features embedded in them to form compact clusters, yet, without further constraints, decision regions of classifiers can expand arbitrarily into open space. We call this Open Space Risk: the risk that the model will confidently classify an unknown object as a known class simply because the unknown falls within a limitless decision region.\u003c/p\u003e \u003cp\u003eTherefore, the core of our approach is to manage open space risks. An effective open-world detector must do more than just separate known classes from the background; It must also explicitly demarcate a low-confidence area in the feature space, allowing unknown objects to reside in this area.\u003c/p\u003e \u003cp\u003e \u003cb\u003eTransformer-based Object Detection Preliminaries.\u003c/b\u003e We have built our framework based on Transformer object detection, especially by using Deformable DETR [\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e]. This pipeline starts from the input image, in which the convolutional backbone [\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e] extracts multi-scale feature maps; Then we process these with the Transformer encoder to capture the global context.\u003c/p\u003e \u003cp\u003eCrucially, a fixed set of learnable object queries is fed into the Transformer decoder. Here, stacked decoder layers enable these queries to interact with the encoded image features via cross-attention and with one another through self-attention. The resulting query embeddings are then fed directly into the classification and bounding-box regression heads.\u003c/p\u003e \u003cp\u003eWe argue that this query formulation has two architectural advantages. First, objectness modeling at the query level naturally separates background noise from known and unknown objects. Secondly, the decoder structure provides a modular interface, and a lightweight adaptation module is inserted to inject task-specific behaviour without changing the backbone. These properties make Transformer detectors ideal for parameter-efficient adaptation in the open world.\u003c/p\u003e \u003c/div\u003e\n\u003ch3\u003eParameter-Efficient Architecture\u003c/h3\u003e\n\u003cp\u003e \u003cb\u003eParameter-Efficient Learning Constraint.\u003c/b\u003e To resolve the conflict between stability and plasticity, we impose a strict parameter-efficiency constraint on the detector. We divide the model parameters into two disjoint subsets: the subset \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{\\theta\\:}_{frozen}\\)\u003c/span\u003e\u003c/span\u003e is the backbone and the Transformer encoder, both pre-trained on large datasets. We keep them fixed throughout learning the process. The subset \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{\\theta\\:}_{adapt}\\:\\)\u003c/span\u003e\u003c/span\u003econsists of the light adaptation modules and task-specific prediction heads, which are the only parts of the network updated at some point.\u003c/p\u003e \u003cp\u003eLearning for a task is reduced to optimizing \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{\\theta\\:}_{adapt}\\)\u003c/span\u003e\u003c/span\u003e while\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{\\theta\\:}_{frozen}\\)\u003c/span\u003e\u003c/span\u003e remaining unchanged. This is clear inductive bias: strong visual representations are retained, and task adaptations are handled by low-capacity controlled updates. Experimentally, we find that this constraint limits representation drift and reduces catastrophic forgetting without additional replays[\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e, \u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e].\u003c/p\u003e \u003cp\u003e \u003cb\u003eResidual Adapter Architecture.\u003c/b\u003e To make the adaptation efficient, we put Residual Adapters inside the Transformer decoder layers. The adapter's structure is a simple bottleneck. It has three steps: a down-projection to reduce dimensionality, an activation function, and an up-projection to restore the dimensionality.\u003c/p\u003e \u003cp\u003eMathematically, if \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:h\\)\u003c/span\u003e\u003c/span\u003e is the input embedding, the adapter works like this:\u003cdiv id=\"Equa\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equa\" name=\"EquationSource\"\u003e\n$$\\:\\mathcal{A}\\left(h\\right)={W}_{up}\\sigma\\:\\left({W}_{down}h\\right)$$\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eHere,\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{W}_{down}\\)\u003c/span\u003e\u003c/span\u003e and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{W}_{up}\\)\u003c/span\u003e\u003c/span\u003e are the projection matrices, and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\sigma\\:\\)\u003c/span\u003e\u003c/span\u003e is the activation. The dimension inside the adapter is much smaller than the input dimension.\u003c/p\u003e \u003cp\u003eWe add this output back to the original features using a residual connection:\u003cdiv id=\"Equb\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equb\" name=\"EquationSource\"\u003e\n$$\\:{h}^{{\\prime\\:}}=h+{\\text{FFN}}_{frozen}\\left(h\\right)+\\lambda\\:\\cdot\\:\\mathcal{A}\\left(\\text{LN}\\right(h\\left)\\right)$$\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eIn this formula, we only train the adapter \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\mathcal{A}\\)\u003c/span\u003e\u003c/span\u003e. The scale factor \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\lambda\\:\\)\u003c/span\u003e\u003c/span\u003e helps control how much the adapter changes the features. This keeps the original decoder working while allowing small changes for the open-world task. The details are presented in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cdiv id=\"Sec8\" class=\"Section2\"\u003e \u003ch2\u003eSemantic-Aligned Initialization\u003c/h2\u003e \u003cp\u003e \u003cb\u003eLeveraging Vision\u0026ndash;Language Models for Semantic Priors.\u003c/b\u003e To fix this, we look at Vision-Language Models like CLIP [\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e]. These models are trained on massive amounts of image-text pairs. They learn a shared space where the image of an object and its textual description are close together.\u003c/p\u003e \u003cp\u003e \u003cspan class=\"InlineEquation\"\u003e \u003cspan class=\"mathinline\"\u003e\\(\\:{C}_{k}\\)\u003c/span\u003e \u003c/span\u003e, we use this property to give our classifier a head start: we take the list of class names for the task and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{C}_{k}\\)\u003c/span\u003e\u003c/span\u003e make simple text prompts for them, like \u0026ldquo;a photo of a\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\:{t}_{k}\\)\u003c/span\u003e\u003c/span\u003e\u0026rdquo;. We run these prompts through a frozen CLIP text encoder and get text embeddings that capture the real semantic relation (dogs are closer to cats than cars) and a strong prefix to have.\u003c/p\u003e \u003cp\u003e \u003cb\u003eSemantic-Aligned Initialization.\u003c/b\u003e We cannot use the text embeddings directly because their dimensions don't match our detector. So, we use a simple linear layout \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:P\\)\u003c/span\u003e\u003c/span\u003e to project them into the classification space:\u003cdiv id=\"Equc\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equc\" name=\"EquationSource\"\u003e\n$$\\:{w}_{k}=\\text{Normalize}\\left(P{t}_{k}\\right)$$\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eHere, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:P\\)\u003c/span\u003e\u003c/span\u003e acts as a bridge. We use these projected vectors \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{w}_{k\\:}\\)\u003c/span\u003e\u003c/span\u003eto initialize the weights of our classification head.\u003c/p\u003e \u003cp\u003eThis completely changed the starting point of the training. We do not start from random noise, but from the fact that the classifier already knows which classes are semantically similar. The geometric effect is very significant. Related categories tend to cluster together, while unrelated categories are far apart.\u003c/p\u003e \u003c/div\u003e\n\u003ch3\u003eVirtual Outlier Synthesis\u003c/h3\u003e\n\u003cp\u003e \u003cb\u003eEnergy-Based Perspective on Open-World Detection.\u003c/b\u003e To fix this, we look at energy-based models. Let \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:z\\)\u003c/span\u003e\u003c/span\u003e be the feature vector and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{f}_{k}\\left(z\\right)\\:\\)\u003c/span\u003e\u003c/span\u003ebe the logits. We calculate a \"free energy\" score:\u003cdiv id=\"Equd\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equd\" name=\"EquationSource\"\u003e\n$$\\:E\\left(z\\right)=-\\text{l}\\text{o}\\text{g}\\sum\\:_{k=1}^{\\mid\\:C\\mid\\:}\\text{e}\\text{x}\\text{p}({f}_{k}\\left(z\\right))$$\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eThe intuition is simple: Low energy means the model is confident and comfortable (Known). High energy means the model is surprised or uncertain (Unknown). Our goal is to map all known objects to low-energy states and force everything else to have high energy.\u003c/p\u003e \u003cp\u003eThe issue is that standard training is greedy. It pulls down the energy for known data to minimize loss, but it doesn't care about the empty space around them. Without a push, the energy in the open space remains undefined.\u003c/p\u003e \u003cp\u003e \u003cb\u003eOutlier Generation in the Latent Space.\u003c/b\u003e Let \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{Z}_{k}\\:\\)\u003c/span\u003e\u003c/span\u003ebe the features for a known class \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:k\\)\u003c/span\u003e\u003c/span\u003e. We estimate the shape of this class using its mean \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{\\mu\\:}_{k}\\)\u003c/span\u003e\u003c/span\u003e and covariance \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{{\\Sigma\\:}}_{k}\\)\u003c/span\u003e\u003c/span\u003e.\u003c/p\u003e \u003cp\u003eWe generate virtual outliers by sampling from a mixture distribution:\u003cdiv id=\"Eque\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Eque\" name=\"EquationSource\"\u003e\n$$\\:{z}_{out}\\sim\\:\\sum\\:_{k=1}^{\\mid\\:C\\mid\\:}{\\pi\\:}_{k}\\mathcal{N}({\\mu\\:}_{k},\\alpha\\:{{\\Sigma\\:}}_{k})$$\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eHere \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\alpha\\:\u0026gt;1\\)\u003c/span\u003e\u003c/span\u003e is a parameter that expands the variance. This formula ensures that the fake samples are close to the known classes (where the confusion happens) but just outside the dense cluster center.\u003c/p\u003e \u003cp\u003eWhy model queries as Gaussian distributions?\u003c/p\u003e \u003cp\u003eSomeone might ask why we assume a Gaussian shape. Unlike CNN feature maps, dense and spatially distributed, Transformer object queries are instance-specific. Each query represents one object hypothesis. These distributions are cleaner and smaller than pixel features. Synthesizing outliers based on query statistics enables precise regularization at the decision boundary.\u003c/p\u003e \u003cp\u003e \u003cb\u003eEnergy Regularization Loss.\u003c/b\u003e For every virtual outlier\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\:{z}_{out}\\)\u003c/span\u003e\u003c/span\u003e we create, we apply a loss function to push its energy up:\u003cdiv id=\"Equf\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equf\" name=\"EquationSource\"\u003e\n$$\\:{\\mathcal{L}}_{VOS}={\\mathbb{E}}_{{z}_{out}}[\\text{m}\\text{a}\\text{x}(0,m-E\\left({z}_{out}\\right)\\left)\\right]$$\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eWhere \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:m\\)\u003c/span\u003e\u003c/span\u003e is the energy margin we want to maintain? This explicitly carves out a high-energy region in the open space.\u003c/p\u003e \u003cp\u003eThe total loss for our model is:\u003cdiv id=\"Equg\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equg\" name=\"EquationSource\"\u003e\n$$\\:\\mathcal{L}={\\mathcal{L}}_{det}+{\\lambda\\:}_{vos}{\\mathcal{L}}_{vos}+{\\lambda\\:}_{reg}{\\mathcal{L}}_{reg}$$\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eWe combine the standard detection loss, the VOS loss, and a regularization term for the adapters.\u003c/p\u003e\n\u003ch3\u003eTraining and Inference\u003c/h3\u003e\n\u003cp\u003e \u003cb\u003eUnified Training Objective.\u003c/b\u003e We train PE-OWOD end-to-end, strictly adhering to a frozen-representation regime. Upon receiving an input image \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:x\\)\u003c/span\u003e\u003c/span\u003e, the detector generates object predictions through the standard Transformer decoding process. We structure the global training objective around three complementary components:\u003cdiv id=\"Equh\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equh\" name=\"EquationSource\"\u003e\n$$\\:\\mathcal{L}={\\mathcal{L}}_{det}+{\\lambda\\:}_{vos}{\\mathcal{L}}_{vos}+{\\lambda\\:}_{reg}{\\mathcal{L}}_{reg}$$\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eThe first term \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{\\mathcal{L}}_{det}\\:\\)\u003c/span\u003e\u003c/span\u003ecaptures the standard set-prediction loss native to Deformable DETR, such as focal loss for classification and L1/GIoU measures for bounding-box regression. The term \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{\\mathcal{L}}_{vos}\\:\\)\u003c/span\u003e\u003c/span\u003eimplements energy-based regularization discussed in Section 3.4; functionally, it wedges queries for known objects with queries for unknown objects. We also introduce \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{\\mathcal{L}}_{reg}\\:\\)\u003c/span\u003e\u003c/span\u003ean L2 penalty on the trainable parameters of the residual adapters. We find regularization necessary to prevent overfitting during early optimization stages. By combining these goals, the model learns to maintain accurate localization of known categories while tolerating new anomalies.\u003c/p\u003e \u003cp\u003e \u003cb\u003eIncremental Learning Protocol.\u003c/b\u003e We designed PE-OWOD to handle incremental learning as our core skill \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:t\\)\u003c/span\u003e\u003c/span\u003e. When starting a new task, adaptation is straightforward: encode input class names with the CLIP text encoder and initialize the classifier weights via semantic projection. Rest adapters remain tuned to new task data.\u003c/p\u003e \u003cp\u003eOne of the main advantages is that there are no complex mechanisms such as replay buffers or feature distillation. Since frozen representations are based on knowledge from previous tasks, features can not be forgotten. PE-OWOD memory is efficient and can handle long sequences of tasks without a huge computational cost.\u003c/p\u003e \u003cp\u003e \u003cb\u003eInference and Unknown Object Identification.\u003c/b\u003e At the inference stage, PE-OWOD operates within the standard DETR decoding pipeline. For each object query, the system produces two outputs: a bounding-box regression and classification logits for known classes.\u003c/p\u003e \u003cp\u003eWe identify unknown objects by calculating the energy score derived in Section 3.4:\u003cdiv id=\"Equi\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equi\" name=\"EquationSource\"\u003e\n$$\\:E\\left(z\\right)=-\\text{l}\\text{o}\\text{g}\\sum\\:_{k=1}^{\\mid\\:\\mathcal{C}\\mid\\:}\\text{e}\\text{x}\\text{p}({f}_{k}\\left(z\\right))$$\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eFunctionally, we label a prediction unknown if its energy level exceeds the threshold. We calibrate the threshold on validation set and keep it constant across subsequent tasks. An energy-based measure is more appropriate than heuristic confidence thresholds because it provides a grounded and transparent way to distinguish known prediction from unseen predictions.\u003c/p\u003e \u003cdiv id=\"Sec11\" class=\"Section2\"\u003e \u003ch2\u003eExperimental Setup\u003c/h2\u003e \u003cp\u003e \u003cb\u003eDatasets.\u003c/b\u003e We train our experiments on the MS-COCO 2017 set [\u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e] with 80 objects categories in complex scenes. We use MS-COCO as a proxy for open-world complexity because real-world deployment scenarios often have similar long-tail distributions and different environments. The data set is high intra-class variance, high overlap between classes, and frequent co-occurrence of known and unknown objects in the same image. Except where otherwise stated, we train on train2017 set and evaluate on val2017 set.\u003c/p\u003e \u003cp\u003e \u003cb\u003eOpen-World Task Protocol\u003c/b\u003e. We adopt the widely used OWOD task protocol introduced by OW-DETR. We partition the 80 COCO classes into four incremental tasks. The specific class partition for each task is detailed in Supplementary Table \u003cspan refid=\"MOESM1\" class=\"InternalRef\"\u003eS1\u003c/span\u003e. Task 1 introduces the 20 VOC classes (e.g., Airplane, Bicycle) as known. Tasks 2, 3, and 4 each introduce 20 additional classes (Outdoor, Accessories, and Indoor items, respectively). We follow the standard setting where unknown classes of \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{U}_{t}\\)\u003c/span\u003e\u003c/span\u003e in the current task become known in the next tasks. Crucially, at each task \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:t\\)\u003c/span\u003e\u003c/span\u003e, only classes of the current known set \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{K}_{t}\\:\\)\u003c/span\u003e\u003c/span\u003eare annotated. Objects belonging to future classes \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{U}_{t}\\:\\)\u003c/span\u003e\u003c/span\u003eare present in images but not annotated. The detector must identify unknown objects. The test measures the system's ability to find unknown objects at each step and to acquire knowledge without catastrophic forgetting.\u003c/p\u003e \u003cp\u003e \u003cb\u003eEvaluation Metrics.\u003c/b\u003e We assess performance using two distinct metrics: closed-set accuracy and open-set sensitivity. For known-class detection, we report the Mean Average Precision (mAP) at 0.5 IoU, the standard for evaluating whether the model correctly discriminates between previously learned categories. To measure Unknown Object Discovery, we use U-Recall, which counts the number of unlabeled (unknown) object instances recovered by the detector, and Absolute Open Set Error (A-OSE). A lower A-OSE indicates that the model distinguishes new instances from known classes rather than hallucinating them and classifying them as something it has seen before.\u003c/p\u003e \u003cp\u003e \u003cb\u003eBaselines.\u003c/b\u003e We compare PE-OWOD with three state-of-the-art methods: (1)ORE, a CNN-based method with explicit unknown modeling; (2)OW-DETR, a Transformer-based implementation using attention scores for open-world environments; and (3)PROB, a probabilistic method with boundary regularization. To ensure fair comparisons, we built all models on a unified DETR-based pipeline. We used standard OWOD protocols and ensured that all training, validation, and evaluation splits were identical across methods.\u003c/p\u003e \u003cp\u003e \u003cb\u003eImplementation Details and Reproducibility.\u003c/b\u003e We built our architecture on a Deformable DETR base with a ResNet-50 backbone. To enforce the parameter-efficient constraint, we initialized both the backbone and Transformer encoder with COCO-pretrained weights and locked them immediately. Only the classification heads, bounding box regressors, and the Residual adapters we inserted are trainable.\u003c/p\u003e \u003cp\u003eWe optimize the model using AdamW [\u003cspan citationid=\"CR30\" class=\"CitationRef\"\u003e30\u003c/span\u003e] with weight decay. The batch size is set to 8 across GPUs. We set specific learning rates for trainable adapter modules and different learning rates for class projection layers. We train each incremental task for 50 epochs, as we found this convergence time sufficient to fit the small sample sizes of the new classes. All input images are preprocessed with random resizing and horizontal flipping. Virtual Outlier Synthesis (VOS) is implemented directly in the training loop without external information; since VOS generates samples on the fly, no information is passed to future tasks. To facilitate reproducibility, we fixed random seeds for all experiments and report averaged results over multiple runs where applicable. We commit to releasing our complete codebase and pre-trained model weights upon publication.\u003c/p\u003e \u003c/div\u003e "},{"header":"Declarations","content":"\u003cdiv id=\"Sec12\" class=\"Section2\"\u003e \u003ch2\u003eData and code availability\u003c/h2\u003e \u003cp\u003eThe datasets generated and/or analysed during the current study are available in the MS-COCO repository, \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://cocodataset.org\u003c/span\u003e\u003cspan address=\"https://cocodataset.org\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. The custom code and pre-trained models for PE-OWOD are available in the GitHub repository, 1ch111/PE-OWOD.\u003c/p\u003e \u003c/div\u003e\u003ch3\u003eFunding\u003c/h3\u003e\n\u003cp\u003eNo funding was received for this work.\u003c/p\u003e\n\u003ch3\u003eAcknowledgements\u003c/h3\u003e\n\u003cp\u003e2024 Guangdong Provincial Key Discipline Research Capacity Enhancement Project (Project No.: 2024ZDJS067); 2023 Guangdong Provincial University Innovation Team Project (Project No.: 2023KCXTD046); 2022 Guangdong Provincial Key Discipline Research Capacity Enhancement Project (Project No.: 2022ZDJS147); 2021 Guangdong Provincial Key Discipline Research Capacity Enhancement Project (No.: 2021ZDJS116); 2022 Guangdong University of Science and Technology Institutional Youth Research Project (No.GKY-2022KYYBK-30)\u003c/p\u003e\n\u003ch3\u003eAuthor Contributions\u003c/h3\u003e\n\u003cp\u003eJ.G.\u0026nbsp;conceived the original idea of Parameter-Efficient Open-World Object Detection, designed the Residual-Adapter architecture and the VOS mechanism, developed the source code, performed all experiments on MS-COCO, and wrote the main manuscript.\u0026nbsp;Y.Z.\u0026nbsp;assisted in the experimental validation and verified the efficiency analysis.\u0026nbsp;Y.L.\u0026nbsp;contributed to the visualization of t-SNE and energy distribution figures.\u0026nbsp;C.L.\u0026nbsp;assisted in data preparation and baseline comparisons.\u0026nbsp;S.G.\u0026nbsp;supervised the entire project, provided theoretical guidance, and revised the manuscript.\u0026nbsp;L.L.\u0026nbsp;provided critical feedback and improved the paper's organization. All authors reviewed and approved the final manuscript.\u003c/p\u003e\n\u003ch3\u003eAdditional Information\u003c/h3\u003e\n\u003cp\u003e\u003cstrong\u003eCompeting Interests:\u003c/strong\u003e The authors declare no competing interests.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eRen, S., He, K., Girshick, R., Sun, J. \u0026amp; Faster, R-C-N-N. Towards real-time object detection with region proposal networks. \u003cem\u003eAdv Neural Inf. Process. Syst\u003c/em\u003e \u003cb\u003e28\u003c/b\u003e (2015).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRedmon, J., Divvala, S., Girshick, R. \u0026amp; Farhadi, A. You only look once: Unified, real-time object detection. In \u003cem\u003eProc. IEEE Conf. Comput. Vis. Pattern Recognit.\u003c/em\u003e 779\u0026ndash;788 (2016).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLin, T. Y., Goyal, P., Girshick, R., He, K. \u0026amp; Doll\u0026aacute;r, P. Focal loss for dense object detection. In \u003cem\u003eProc. IEEE Int. Conf. Comput. Vis.\u003c/em\u003e 2980\u0026ndash;2988 (2017).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJoseph, K. J., Khan, S., Khan, F. S. \u0026amp; Balasubramanian, V. N. Towards open world object detection. In \u003cem\u003eProc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit.\u003c/em\u003e 5830\u0026ndash;5840 (2021).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCarion, N. et al. Springer,. End-to-end object detection with transformers. In \u003cem\u003eEur. Conf. Comput. Vis.\u003c/em\u003e 213\u0026ndash;229 (2020).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGupta, A. et al. OW-DETR: Open-world detection transformer. In \u003cem\u003eProc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit.\u003c/em\u003e 9235\u0026ndash;9244 (2022).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWang, Y., Zhang, Z., Zhao, S., Buramck, L. \u0026amp; Ebrahimi, S. Randomized proposals for open-world object detection. In \u003cem\u003eEur. Conf. Comput. Vis.\u003c/em\u003e 209\u0026ndash;226Springer, (2022).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWang, Z., Li, Y., Guo, X., Fang, L. \u0026amp; Wang, S. Probabilistic two-stage open-world object detection. In \u003cem\u003eProc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit.\u003c/em\u003e 2022\u0026ndash;2032 (2023).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWu, X., Peng, D., Zhang, P., Yu, L. \u0026amp; Yang, K. Two-stage open-world object detection with unknown-aware query mechanism. In \u003cem\u003eProc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit.\u003c/em\u003e 1134\u0026ndash;1143 (2023).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZhao, S. et al. Springer,. Exploiting unlabeled data with vision and language models for object detection. In \u003cem\u003eEur. Conf. Comput. Vis.\u003c/em\u003e 159\u0026ndash;175 (2022).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWu, Y., Han, X., Li, G., Chang, M. \u0026amp; Yan, Y. Open-world object detection via discriminative class prototype learning. In \u003cem\u003eProc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit.\u003c/em\u003e 2259\u0026ndash;2269 (2023).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLi, L. H. et al. Grounded language-image pre-training. In \u003cem\u003eProc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit.\u003c/em\u003e 10965\u0026ndash;10975 (2022).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGu, X., Lin, T. Y., Kuo, W. \u0026amp; Cui, Y. Open-vocabulary object detection via vision and language knowledge distillation. In \u003cem\u003eInt Conf. Learn. Represent\u003c/em\u003e (2022).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHoulsby, N. et al. Parameter-efficient transfer learning for NLP. In \u003cem\u003eInt Conf. Mach. Learn\u003c/em\u003e 2790\u0026ndash;2799 (2019).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHu, E. J. et al. \u003cb\u003eLoRA\u003c/b\u003e: Low-rank adaptation of large language models. In \u003cem\u003eInt Conf. Learn. Represent\u003c/em\u003e (2022).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJia, M. et al. Springer,. Visual prompt tuning. In \u003cem\u003eEur. Conf. Comput. Vis.\u003c/em\u003e 709\u0026ndash;727 (2022).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChen, S. et al. Adapting vision transformers for scalable visual recognition. \u003cem\u003eAdv. Neural Inf. Process. Syst.\u003c/em\u003e \u003cb\u003e35\u003c/b\u003e, 16664\u0026ndash;16678 (2022). AdaptFormer.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChen, Z. et al. Vision transformer adapter for dense predictions. In \u003cem\u003eInt Conf. Learn. Represent\u003c/em\u003e (2023).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZhai, M. et al. Hyper-Adapter: Parameter-Efficient Fine-Tuning for Vision Transformers. In \u003cem\u003eProc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit.\u003c/em\u003e (2024).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDu, X., Wang, Z., Cai, M. \u0026amp; Li, Y. VOS: Learning what you don't know by virtual outlier synthesis. In \u003cem\u003eInt. Conf. Learn. Represent.\u003c/em\u003e (2022).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBendale, A. \u0026amp; Boult, T. E. Towards open set deep networks. In \u003cem\u003eProc. IEEE Conf. Comput. Vis. Pattern Recognit.\u003c/em\u003e 1563\u0026ndash;1572 (2016).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTack, J., Mo, S., Jeong, J. \u0026amp; Shin, J. C. S. I. Novelty detection via contrastive learning on distributionally shifted instances. \u003cem\u003eAdv. Neural Inf. Process. Syst.\u003c/em\u003e \u003cb\u003e33\u003c/b\u003e, 11839\u0026ndash;11852 (2020).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLiu, W., Wang, X., Owens, J. \u0026amp; Li, Y. Energy-based out-of-distribution detection. \u003cem\u003eAdv. Neural Inf. Process. Syst.\u003c/em\u003e \u003cb\u003e33\u003c/b\u003e, 21464\u0026ndash;21475 (2020).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZhu, X. et al. Deformable DETR: Deformable transformers for end-to-end object detection. In \u003cem\u003eInt Conf. Learn. Represent\u003c/em\u003e (2021).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHe, K., Zhang, X., Ren, S. \u0026amp; Sun, J. Deep residual learning for image recognition. In \u003cem\u003eProc. IEEE Conf. Comput. Vis. Pattern Recognit.\u003c/em\u003e 770\u0026ndash;778 (2016).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRebuffi, S. A., Kolesnikov, A., Sperl, G., Lampert, C. H. \u0026amp; iCaRL Incremental classifier and representation learning. In \u003cem\u003eProc. IEEE Conf. Comput. Vis. Pattern Recognit.\u003c/em\u003e 2001\u0026ndash;2010 (2017).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eYan, S., Xie, J. \u0026amp; He, X. D. E. R. Dynamically expandable representation for class incremental learning. In \u003cem\u003eProc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit.\u003c/em\u003e 3014\u0026ndash;3023 (2021).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRadford, A. et al. Learning transferable visual models from natural language supervision. In \u003cem\u003eInt Conf. Mach. Learn\u003c/em\u003e 8748\u0026ndash;8763 (2021).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLin, T. Y. et al. Microsoft COCO: Common objects in context. In \u003cem\u003eEur. Conf. Comput. Vis.\u003c/em\u003e 740\u0026ndash;755 (2014).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLoshchilov, I. \u0026amp; Hutter, F. Decoupled weight decay regularization. In \u003cem\u003eInt Conf. Learn. Represent\u003c/em\u003e (2019).\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Open-World Object Detection, Parameter-Efficient Fine-Tuning, Virtual Outlier Synthesis, Transformer, Incremental Learning, Computer Vision","lastPublishedDoi":"10.21203/rs.3.rs-8754841/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8754841/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eOpen-world object detection requires knowledge of categories and discovery of new objects never seen in training. Full model fine tuning fails often in such dynamic environment. Fully tuned models overfit known classes and lose their sensitivity to unknown objects at high computational cost. To solve this problem, we propose PE-OWOD, a simple and light approach to retrain. Instead of updating the whole network, we lock backbone and encoder to maintain stable visual priors and inject compact Residual adapters only into decoder to adapt tasks. We also introduce VOS, which defines explicit decision boundary for open space with optional semantic initialization. MS-COCO benchmarks show remarkable efficiency advantages: Update less than 27% of models, PE-OWOD achieves 64.7% Unknown Recall (significantly outperform fully tuned baselines), and GPU memory usage is reduced by 86%. These results indicate that effective parameter adaptation is not a constraint; rather, it is a reliable and robust open-world detection strategy.\u003c/p\u003e","manuscriptTitle":"PE-OWOD: Parameter-Efficient Open-World Detection with Semantic Priors and Virtual Outlier Synthesis","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-02-24 12:42:33","doi":"10.21203/rs.3.rs-8754841/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2026-03-02T19:16:50+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-02-27T10:49:03+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-02-26T13:43:28+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"117387135820981306768454875543772657811","date":"2026-02-25T03:42:08+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"158211920341033378587482821461207801067","date":"2026-02-23T01:54:00+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"100638237745277473972399882478504916849","date":"2026-02-21T05:54:03+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"142773746656863404131166754828468207578","date":"2026-02-20T07:05:41+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"116357999802386444149026643085436996772","date":"2026-02-20T03:36:05+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2026-02-20T03:31:27+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2026-02-17T12:57:33+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2026-02-03T07:52:05+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2026-02-03T07:51:27+00:00","index":"","fulltext":""},{"type":"submitted","content":"Scientific Reports","date":"2026-02-01T08:29:56+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"b5371cde-e25b-4671-82d7-f256b094050a","owner":[],"postedDate":"February 24th, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[{"id":63343729,"name":"Biological sciences/Computational biology and bioinformatics"},{"id":63343730,"name":"Physical sciences/Engineering"},{"id":63343732,"name":"Physical sciences/Mathematics and computing"}],"tags":[],"updatedAt":"2026-04-24T19:08:06+00:00","versionOfRecord":[],"versionCreatedAt":"2026-02-24 12:42:33","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8754841","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8754841","identity":"rs-8754841","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}