BullySense: A Approach for Campus Bullying Detection Leveraging LLava Foundation Model

BullySense: A Approach for Campus Bullying Detection Leveraging LLava Foundation Model | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article BullySense: A Approach for Campus Bullying Detection Leveraging LLava Foundation Model Houming Gong, Tao Li, Cong Chen, Wei Lu, Wei Qu, Ruiqi Du This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-5668231/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Campus bullying has become a significant social issue, with profound impacts on the mental and physical well-being of students. Timely and effective detection is crucial for addressing this problem. To tackle these challenges, this study introduces BullySense, a campus bullying detection model built on the vision language model (VLM) framework. Utilizing the pre-trained LLaVA-1.5 model, BullySense is fine-tuned with a curated dataset of bullying and normal campus activity images, employing Low-Rank Adaptation (LoRA) to enhance task-specific performance. Experimental results show that BullySense outperforms traditional models, achieving precision of 0.982 and F1 scores of 0.977. While challenges remain, such as handling low-quality images and extending to multimodal data, this work demonstrates the potential of AI for improving campus safety and lays the groundwork for future advancements. Campus Bullying Violence Recognition VLMs Educational Institution Safety Figures Figure 1 Figure 2 1 Introduction In recent years, incidents of campus bullying have become alarmingly frequent, drawing widespread attention from various sectors of society. The term "mobbing" was coined by Lorenz [ 1 ] to describe instances where a group of animals collectively attacked a single victim. In studies of classroom behavior in Norway during the 1990s, Olweus [ 2 ] adopted the term "bullying" to describe situations where victims experienced repeated aggression over an extended period. Peter et al. [ 3 ] further categorized bullying into five distinct types: physical abuse, relational abuse, verbal abuse, cyberbullying, and sexual harassment. The roots of campus bullying are complex and multifaceted, encompassing individual, familial, and societal dimensions [ 4 , 5 ]. Timely identification and intervention are crucial for effectively addressing this issue. However, teachers and parents cannot continuously monitor students' well-being, and victims often hesitate to report their experiences due to fear or shame. This delay in addressing bullying allows such behaviors to escalate, severely impacting the mental and physical health of the victims [ 6 – 8 ]. This study focuses on leveraging surveillance systems and visual content, such as images, to detect campus bullying and violence promptly. With the rapid advancement of artificial intelligence, the vision language models, such as Qwen-VL and LLaMA, have demonstrated exceptional capabilities in image understanding and natural language processing [ 9 , 10 ]. These models can analyze image information, conduct sentiment analysis, and recognize behaviors, offering new approaches for detecting campus bullying. By developing detection algorithms based on VLMs, this research aims to identify bullying behaviors effectively, assisting educators and mental health professionals in providing timely support and intervention. The significance of this study lies not only in enhancing campus safety but also in offering innovative methodologies for future research and technological applications. This study presents three main contributions: Development of a Campus Bullying Detection Dataset We curated a comprehensive dataset specifically designed for campus bullying detection. This dataset includes diverse image data representing typical bullying scenarios, such as physical violence and group intimidation, providing extensive training resources for constructing robust LLM-based models. Proposal of an VLM-Based Monitoring Framework A regulatory model grounded in VLMs was developed to accurately identify campus bullying behaviors. By learning the behavioral patterns associated with bullying, the model exhibits enhanced robustness in complex real-world settings, effectively distinguishing between playful interactions and actual bullying incidents. Experimental Validation and Comparative Analysis Extensive experiments were conducted to validate the effectiveness of the proposed approach in detecting campus bullying. Results demonstrate significant advantages in terms of detection accuracy and inference capabilities compared to traditional methods, underscoring the proposed framework’s precision and reliability as a solution for campus bullying detection. 2 Related Work With the advancement of artificial intelligence, numerous researchers have explored using deep neural networks for violence detection to achieve real-time identification of bullying behaviors. Aldehim et al. [ 11 ] proposed a method called TSODL VD, designed for accurate and efficient recognition of violence in surveillance videos. By employing the TSO protocol as a hyperparameter enhancer for the residual DenseNet model, their approach improved violence detection performance. Garcia-Cobo et al. [ 12 ] introduced an advanced architecture for violence detection in surveillance videos that integrates human posture extraction and motion variation detection to autonomously identify violent events. Huszár et al. [ 13 ] proposed two innovative architectures for classifying violence in video clips using action recognition features from the Kinetics 400 dataset. The FT model optimized X3D M parameters pre-trained on Kinetics 400, while the TL model extracted spatial features without modifying these parameters and added multiple fully connected layers for training. Mahareek et al. [ 14 ] introduced a novel model for anomaly detection in surveillance videos by seamlessly integrating 3DCNN and ConvLSTM architectures, demonstrating superior performance compared to standalone 3DCNN configurations. Hsairi et al. [ 15 ] employed widely recognized models such as sequential CNNs, MobileNetV2, and VGG-16 to evaluate performance on a large annotated dataset containing violent and non-violent images across eight categories. Techniques like data augmentation, transfer learning, and fine-tuning were utilized to enhance model performance. Ullah et al. [ 16 ] developed a violence detection system for surveillance videos using computer vision and AI technologies to enhance campus security. While deep neural networks show technical promise for bullying detection, they face notable challenges. These include the complexity of campus environments, the diversity of student behaviors, and background interference, which can lead to recognition errors. Additionally, understanding contextual semantics and non-verbal behaviors (e.g., body language) remains a significant limitation. The development of large language models (LLMs) has significantly advanced natural language processing applications. VLMs have further expanded these capabilities by integrating visual data, enabling the seamless understanding of textual and image-based information. This integration not only improves sentiment analysis but also broadens the potential for image content understanding, enhancing adaptability and practicality. Zhao et al. [ 17 ] introduced the Multimodal Image Relationship Benchmark (MIRB) to evaluate the ability of vision language models to compare, analyze, and reason across multiple images. Jin et al. [ 18 ] presented Chat-UniVi, a unified vision language model employing dynamic visual tokens to represent both images and videos. This framework efficiently captures spatial details for images and temporal relationships for videos. Zhu et al. [ 19 ] proposed MiniGPT-4, which aligns a frozen visual encoder with the advanced LLM Vicuna using a projection layer. This work demonstrated that correctly aligning visual features with LLMs could unlock many advanced multimodal capabilities showcased by GPT-4. Carneros-Prado et al. [ 20 ] conducted a comparative analysis of sentiment detection using GPT-3.5 and IBM Watson on a dataset of 30,000 tweets related to the COVID-19 pandemic. Results indicated that GPT-3.5 outperformed Watson in detecting nuanced emotions, such as sarcasm. Yang et al. [ 21 ] addressed the poor performance of LLMs in sentiment recognition with limited data by proposing a novel approach that combines LLM knowledge with contrastive prompting. This method augments training samples using LLM commonsense knowledge and enhances semantic representation by training on unlabeled data with contrastive embeddings. Nadeem et al. [ 22 ] compared fine-tuned deep learning models and general-purpose LLMs for image-related tasks, highlighting that GPT-4 performed better on smaller datasets, emphasizing the practicality of foundational LLMs in data-scarce scenarios. Zanella et al. [ 23 ] introduced LAVAD, a language-based video anomaly detection framework that transforms pre-trained LLMs and VLMs into effective video anomaly detectors in a zero-shot paradigm. Building on these studies, leveraging the image understanding and semantic reasoning capabilities of VLMs can effectively identify campus bullying incidents, addressing challenges faced by deep neural network-based approaches. Our experimental results validate this conclusion, demonstrating the efficacy of large models in detecting campus bullying and violence. 3 Methodology 3.1 Data Collection and Preprocessing Currently, widely-used violent video datasets include the Hockey Dataset [ 24 ], Movie Dataset [ 24 ], Violent Flow Dataset [ 25 ], Real-Life Violence Situations (RLVS) [ 26 ], and UCF-Crime [ 27 ]. However, datasets specific to campus violence remain limited, largely due to the sensitivity of such incidents and privacy concerns involving minors. To address this gap, our research team created a custom dataset to ensure sufficient samples for model training and testing. The data, sourced from diverse materials depicting bullying scenarios, strictly adhered to ethical guidelines and privacy standards, ensuring no personally identifiable information was included. Image Collection In the process of constructing a campus bullying behavior dataset, we utilized diverse search phrases such as "physical aggression," "campus violence," "bullying," and "school fight" to gather video materials from the internet. These resources, often sourced from news reports, surveillance footage, and social media platforms, ensured coverage of various scenarios. Key frames representing bullying behaviors were manually extracted, focusing on critical moments that showcased physical conflict, facial expressions, and environmental context. Additionally, normal campus activity images, such as sports and classroom interactions, were collected to help the model differentiate bullying from routine behaviors. Figure 1 illustrates representative examples. Image Annotation In the initial stages of dataset construction, our goal was to provide detailed annotations for each image, describing specific actions and environmental contexts to help the model comprehensively understand campus bullying behaviors. However, this approach did not yield the expected accuracy, likely due to the diverse and context-dependent nature of bullying behaviors. To address this, we simplified the annotation strategy by classifying images as either "bullying" or "normal behavior." This adjustment allowed the model to autonomously learn distinguishing features, improving its ability to generalize across various scenarios while enhancing recognition precision. Collecting bullying content was a significant challenge. In total, we gathered 1022 images, including 550 depicting campus bullying incidents and 472 showing normal campus life. For the training phase, 440 bullying images and 360 normal images were used. Additionally, we retained 110 bullying images and 112 normal images as a test set to evaluate the model's performance. 3.2 Model Selection and Training To construct BullySense, we evaluated several pretrained VLMs to assess their applicability in campus safety detection tasks [ 19 , 28 , 29 ]. Among these, the pretrained LLaVA model demonstrated reliable detection responses and robust zero-shot performance, indicating a foundational understanding of campus safety recognition [ 30 ]. Consequently, BullySense was built upon the pretrained LLaVA-1.5 model, with fine-tuning specifically aimed at identifying campus bullying. Through fine-tuning, our goal was to enhance the model’s capacity to recognize bullying images and apply it to campus safety monitoring tasks. The training process employed LoRA [ 31 ] to fine-tune the visual-language components of LLaVA-1.5. This approach targeted task-specific datasets emphasizing bullying behavior detection. By leveraging LoRA, we introduced task-specific adaptations without retraining the entire model, preserving the original model's generalization capabilities. Specifically, LoRA modifies trainable parameters by aligning the visual feature projection layers with language embeddings while incorporating low-rank matrices into the attention layers for efficient parameter updates. These matrices enable targeted updates to the attention mechanisms, thereby improving the model's ability to interpret and recognize content related to campus bullying. By selectively fine-tuning only these parameters, we significantly reduced computational costs and training time while enhancing task-specific performance. The fine-tuning focused on improving the model's capability to generate contextually aligned image descriptions, ensuring accurate identification and labeling of bullying content. This approach optimized the model for practical application in monitoring campus environments effectively. 4 Experiments BullySense, capitalizing on the sophisticated capabilities of VLMs in the realms of image comprehension and affective computing, offers a detection solution for campus bullying that is both more accurate and encompassing than conventional approaches. To augment the VLM's ability to discern and interpret characteristics indicative of campus bullying, BullySense has been developed by leveraging the foundational strengths of pre-trained VLMs. We have amassed and meticulously annotated an array of image datasets sourced from the internet, encompassing imagery of both bullying incidents and typical campus activities. It is upon this curated dataset that BullySense has been fine-tuned. The empirical findings from our study demonstrate that BullySense surpasses current models in its ability to identify acts of campus bullying, offering not only enhanced detection accuracy but also more nuanced analysis and more dependable forecasting. 4.1 Experimental Setup To optimize deployment costs, we selected the smallest model available for fine-tuning. Training was conducted on a single machine equipped with two RTX 3090 GPUs. Due to memory constraints, the batch size was set to one, and the learning rate was configured at 2e-4. The model was trained for one epoch to enable a quick evaluation of its performance. Based on these initial findings, future iterations may involve extended training over multiple epochs to further enhance the model's performance. 4.2 Evaluation Metric This research employs the F1 score as the primary evaluation metric, emphasizing its role in balancing precision and recall while considering accuracy for comprehensive assessment. Precision measures the proportion of true positive predictions among all positive predictions, reflecting prediction specificity. Recall calculates the proportion of true positives identified out of all actual positives, assessing the model's sensitivity. Accuracy evaluates the overall correctness of predictions, encompassing both positive and negative cases. The F1 score, as the harmonic mean of precision and recall, effectively balances these aspects for robust performance evaluation. The following are the definitions of these essential metrics: \(\:Precision=\:\frac{TP}{TP+FP}\) (1) \(\:Recall=\:\frac{TP}{TP+FN}\) (2) \(\:Accuracy=\:\frac{TP+TN}{TP+FP+TN+FN}\) (3) In the formula, TP (True Positives) is the number of samples correctly predicted as positive, TN (True Negatives) is the number of samples correctly predicted as negative, FN (False Negatives) is the number of positive samples incorrectly predicted as negative, FP (False Positives) is the number of samples incorrectly predicted as positive. $$\:{F}_{1}=2\times\:\frac{Precision\times\:Recall}{Precision+Recall}$$ 4 The F1 score provides a comprehensive metric that considers both precision and recall, especially useful in cases of class imbalance, which is critical in content safety tasks where both false positives and false negatives can lead to significant consequences. While accuracy is commonly viewed as a general measure of model performance, it may not fully capture the model's ability to differentiate between safe and unsafe content, particularly in datasets with imbalanced class distributions, where a high accuracy could be achieved by predominantly predicting the majority class. 4.3 Baseline Comparison Currently, research on detecting campus bullying behaviors using VLMs is relatively scarce, as the application of multimodal VLMs in this field remains exploratory. To validate the advantages of VLM-based methods, this study compares the performance of four violence detection models based on deep neural networks. By contrasting traditional deep learning models with VLMs, the research highlights the multimodal VLMs' accuracy, robustness, and adaptability to complex scenarios. These findings offer novel perspectives and solutions for bullying detection and potential applications in broader social safety and child protection contexts. Table 1 Introduction to using the baseline model Study Approach Description Rathi et al.,2023 CNN implemented with MobileNetV2 CNN algorithm utilizing MobileNetV2 and OpenCV for violence detection, trained on a balanced dataset of real-life videos dataset. Kozhamkulova et. al., 2023 MoveNet based physical bullying detection MoveNet evaluate body position, artificial neural network to determine whether the scene contains violent situations. AlDahoul et. al., 2021 CNN-LSTM based model CNN learn spatial features from video's frames, LSTM for video classification into violence/non-violence classes. Deepak et. al., 2020 Gradients based violence detection Extract autocorrelation of gradient features from the input videos. A discriminative classifier is then used to recognize violent actions in videos. CNN with MobileNetV2 [ 26 ]: Uses MobileNetV2 and OpenCV for training on real-life videos containing equal amounts of violent and non-violent scenes to recognize violence. MoveNet-based Detection [ 32 ]: Combines body position features from photo sequences with neural network-based activity classification to identify hostile or violent actions. CNN-LSTM Model [ 33 ]: A novel end-to-end architecture where CNN extracts spatial features, and LSTM classifies video frames as violent or non-violent. Gradient-Based Detection [ 34 ]: In the proposed work, first extract autocorrelation of gradient features from the input videos. Then the auto-correlation feature-based representation is fed as an input to discriminative classifiers such as Support Vector Machine (SVM) to classify different normal and abnormal activities. These baseline methods offer a robust framework for evaluating the effectiveness of our proposed BullySense model. 4.4 The Impact of Dataset Size In this study, we focused on the impact of fine-tuning on model performance and conducted an in-depth analysis of how the size of the dataset during fine-tuning affects the model's performance. Through this analysis, we aimed to explore the specific impact of datasets of varying sizes on the effectiveness of model training and to seek effective strategies for optimizing model performance by adjusting the amount of data. The results presented in Fig. 2 clearly demonstrate the significant influence of dataset size on model performance. By comparing and analyzing datasets of different sizes, we found that the scale of the dataset has a notable impact on the model's effectiveness. Specifically, we observed that when the dataset is larger (for example, containing 800 images), there is a significant improvement in the model's accuracy and F1 score, indicating that increasing the number of samples can enhance the model's recognition accuracy and robustness. This enhancement may be since a larger dataset provides a richer representation of features and more comprehensive coverage, enabling the model to learn more generalized characteristics. However, we are also aware that collecting large-scale datasets in practical applications faces numerous challenges, including the difficulty, cost, and time required for data acquisition. Furthermore, our research found that beyond a certain critical point of dataset size, the improvement in performance may gradually diminish, possibly due to overfitting or other factors leading to model performance saturation. This suggests that when designing and implementing data collection strategies, we should consider the diversity and quality of the data, not just the quantity, to ensure that the model performs optimally in real-world application scenarios. 4.5 Performance Analysis Table 2 compares the performance of BullySense with other baseline models across key evaluation metrics, with higher scores indicating better performance. According to the results, BullySense demonstrates superior capabilities in campus bullying detection tasks. Notably, it achieves a Precision of 97.4% and an F1 Score of 97.7%, showcasing robust performance in accurately identifying bullying behaviors. These results highlight BullySense's strong detection accuracy and resilience compared to baseline models, making it a highly effective tool for addressing campus bullying scenarios. Table 2 The performance comparison of the proposed model with other models Approach Precision Recall Accuracy F1 Score CNN implemented with MobileNetV2 0.945 0.936 0.941 0.941 MoveNet based physical bullying detection 0.936 0.927 0.932 0.932 CNN-LSTM based model 0.731 0.791 0.752 0.760 Gradients based violence detection 0.906 0.873 0.892 0.889 The proposed approach 0.982 0.973 0.977 0.977 Despite the impressive performance demonstrated by BullySense, the model encountered several challenges during practical testing. Specifically, there were instances where the model misclassified bullying behavior as normal interactions, resulting in false negatives (FN). A thorough analysis of these misclassification cases revealed several key influencing factors. Environment Context The complexity of the environment in which bullying occurs is a significant factor. For example, in open spaces like school playgrounds, physical contact between students is quite common, making it challenging to accurately determine whether bullying behavior is present based solely on a single image. Image Quality The clarity of the images plays a crucial role. In cases where the images are blurred or of low quality, even advanced visual recognition models struggle to capture subtle expressions and movements of the students, thereby increasing the difficulty of identifying bullying behavior. To address these challenges, future work could focus on several directions: First, enhancing the model's ability to recognize bullying behavior in complex environments may require the incorporation of additional contextual information and environmental features. Secondly, improving the quality of image and video data collection, including resolution and clarity, is essential to reduce misclassifications caused by image quality issues. Furthermore, the introduction of multimodal data, such as integrating audio and textual information, could provide a more comprehensive approach to bullying behavior recognition. 4.6 Generalization Analysis To comprehensively assess the generalization capability of the proposed model, we selected three representative datasets for performance testing: The Hockey dataset, Violent-Flow dataset, and RLVS violence behavior dataset. These datasets span a range of scenarios from sports competitions to everyday life, thoroughly evaluating the model's adaptability and accuracy in diverse environments. The test results presented in Table 3 provide significant insights into the model's performance. From the experimental outcomes, our model demonstrated high precision in detecting violent behaviors across other datasets, indicating its ability to accurately identify genuine acts of violence and reducing the misclassification of normal behaviors as violent. This is particularly evident in The Hockey dataset and Violent-Flow dataset, suggesting the model's strong capability in recognizing violence within these specific domains. However, the model's recall performance was less satisfactory, implying that the model falls short in identifying all actual violent incidents, failing to capture every violent event. This performance discrepancy may stem from the model's limited capacity to recognize the complexity and diversity of violent behaviors, especially when confronted with the complex real-life scenarios included in the RLVS dataset, where the model's precision was at its lowest. This further highlights the challenges the model faces in identifying violent behaviors in complex and variable environments. Analyzing the reasons for this performance variation, we attribute it to several factors: firstly, the manifestations of violent behaviors across different datasets vary, and the model may not have sufficiently learned these differences; secondly, the samples in the test datasets may exhibit considerable disparities in lighting, angle, and occlusion, affecting the model's recognition capabilities; and finally, the model may have focused too heavily on certain specific features, neglecting other cues that are equally important for the identification of violent behaviors. Table 3 Generalization Analysis of the Proposed Model Dataset Precision Recall Accuracy F1 Score The Hockey Dataset 0.995 0.772 0.884 0.869 Violent-Flow Dataset 0.926 0.590 0.770 0.721 Real-Life Violence Situations 0.786 0.633 0.719 0.701 5 Limitations When discussing the limitations of this paper, we recognize that the current model has several key limitations in detecting campus bullying. Firstly, the model's performance may be affected when dealing with images of low quality or blurry resolution, leading to biases in identification results. It is necessary to increase the diversity of the training dataset to enhance the model's adaptability to various image conditions. Secondly, the model primarily relies on image data for bullying behavior recognition, which limits its application capabilities in audio, text, or other modalities. In the field of multimodal learning, combining visual, audio, and textual information can provide a more comprehensive understanding of the context, potentially improving the accuracy and robustness of detection. In summary, although the BullySense model has made certain progress in detecting campus bullying, further research and improvement are still needed in multimodal data processing, model robustness, and computational efficiency. We look forward to developing more comprehensive and accurate tools for campus bullying detection, contributing to the creation of safer campus environments. 6 Conclusion This study presents BullySense, a model designed to detect campus bullying behaviors using advanced vision language mode. By fine-tuning the pre-trained LLaVA-1.5 model, BullySense achieves robust and accurate performance in identifying bullying incidents from visual data. Through an innovative dataset focusing on diverse bullying scenarios and a systematic training process employing LoRA, the model successfully bridges the gap between theoretical advancements in AI and practical applications in campus safety. The experimental results demonstrate that BullySense outperforms traditional deep learning models across key metrics such as precision and F1 score, showcasing its capability to handle the complexities of real-world bullying detection. Despite its strengths, challenges persist, including difficulties in handling ambiguous or low-quality images and a reliance on single-modality data, which can limit performance in nuanced scenarios. This research underscores the importance of integrating multimodal data and diversifying datasets to further improve model reliability and accuracy. Additionally, reducing computational overhead and validating the model in practical deployment scenarios remain crucial for broad adoption. In conclusion, BullySense represents a significant step forward in the field of campus safety, combining cutting-edge AI techniques with practical considerations. Its development and performance provide valuable insights for future research on using advanced AI tools to address complex social issues, ultimately contributing to safer campus environments and better outcomes for students. Declarations Author Contribution Houming Gong: Involved in the research conception and design, performed data analysis or statistical processing, drafted the initial and final manuscripts, and made critical revisions and improvements to the paper.Tao Li: Involved in the research conception and design, supervised the implementation and management of the research project.Cong Chen: Responsible for experimental data collection and assisted in the analysis or statistical processing of experimental results.Wei Lu: Involved in the research conception and design, conducted the literature review, and provided the research background and theoretical foundation.Wei Qu: Involved in the research conception and design, and discussed the experimental results.Ruiqi Du: Responsible for data collection. References Lorenz K. Sobre la agresión: el pretendido mal. Siglo xxi; 1998. Olweus D, Jimerson SR, Swearer SM, Espelage DLJoJ, SR, Swearer, S. M, Espelage DNYR. Handbook of bullying in schools: An international perspective. 2010:9–33. Peter K SJJoCP, Psychiatry. Commentary: Types of bullying, types of intervention: reflections on Arseneault. 2018;59(4):422–3. Li J, Hesketh TJFip. Experiences and perspectives of traditional bullying and cyberbullying among adolescents in Mainland China-implications for policy. 2021;12:672223. Zapf D, Einarsen SV. Individual antecedents of bullying: Personality, motives and competencies of victims and perpetrators. Bullying and Harassment in the Workplace. CRC press; 2020. p. 269–303. Arslan G, Allen K-A, Tanhan AJCIR. School bullying, mental health, and wellbeing in adolescents: Mediating impact of positive psychological orientations. 2021;14(3):1007–26. Eyuboglu M, Eyuboglu D, Pala SC, Oktar D, Demirtas Z, Arslantas D, et al. Traditional school bullying and cyberbullying: Prevalence, the effect on mental health problems and self-harm behavior. 2021;297:113730. Luo X, Zheng R, Xiao P, Xie X, Liu Q, Zhu K, et al. Relationship between school bullying and mental health status of adolescent students in China: a nationwide cross-sectional study. 2022;70:103043. Li X, Wu Y, Jiang X, Guo Z, Gong M, Cao H, et al. Enhancing visual document understanding with contrastive learning in large visual-language models. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition2024. p. 15546-55. Tong S, Liu Z, Zhai Y, Ma Y, LeCun Y, Xie S. Eyes wide shut? exploring the visual shortcomings of multimodal llms. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition2024. p. 9568-78. Aldehim G, Asiri MM, Aljebreen M, Mohamed A, Assiri M, Ibrahim SSJIA. Tuna swarm algorithm with deep learning enabled violence detection in smart video surveillance systems. 2023. Garcia-Cobo G, SanMiguel JCJCV, Understanding I. Human skeletons and change detection for efficient violence detection in surveillance videos. 2023;233:103739. Huszar VD, Adhikarla VK, Négyesi I, Krasznay CJIA. Toward fast and accurate violence detection for automated video surveillance applications. 2023;11:18772–93. Mahareek EA, El-Sayed EK, El-Desouky NM, El-Dahshan KA. Detecting anomalies in security cameras with 3DCNN and ConvLSTM. 2023. Hsairi L, Alosaimi SM, Alharaz GAJAJfS, Engineering. Violence Detection Using Deep Learning. 2024:1–11. Ullah FUM, Obaidat MS, Ullah A, Muhammad K, Hijji M, Baik SWJACS. A comprehensive review on vision-based violence detection in surveillance videos. 2023;55(10):1–44. Zhao B, Zong Y, Zhang L, Hospedales TJapa. Benchmarking Multi-Image Understanding in Vision and Language Models: Perception, Knowledge, Reasoning, and Multi-Hop Reasoning. 2024. Jin P, Takanobu R, Zhang W, Cao X, Yuan L. Chat-univi: Unified visual representation empowers large language models with image and video understanding. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition2024. p. 13700-10. Zhu D, Chen J, Shen X, Li X, Elhoseiny MJapa. Minigpt-4: Enhancing vision-language understanding with advanced large language models. 2023. Carneros-Prado D, Villa L, Johnson E, Dobrescu CC, Barragán A, García-Martínez B. Comparative study of large language models as emotion and sentiment analysis systems: A case-specific analysis of GPT vs. IBM Watson. International Conference on Ubiquitous Computing and Ambient Intelligence: Springer; 2023. p. 229 – 39. Yang H, Zi Y, Qin H, Zheng H, Hu YJJoCS, Applications S. Advancing emotional analysis with large language models. 2024;4(3):8–15. Nadeem M, Javed L, Cambria E, Hussain A. Are Foundation Models the Next-Generation Social Media Content Moderators? Zanella L, Menapace W, Mancini M, Wang Y, Ricci E. Harnessing Large Language Models for Training-free Video Anomaly Detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition2024. p. 18527-36. Bermejo Nievas E, Deniz Suarez O, Bueno García G, Sukthankar R. Violence detection in video using computer vision techniques. Computer Analysis of Images and Patterns: 14th International Conference, CAIP 2011, Seville, Spain, August 29–31, 2011, Proceedings, Part II 14: Springer; 2011. p. 332-9. Hassner T, Itcher Y, Kliper-Gross O. Violent flows: Real-time detection of violent crowd behavior. 2012 IEEE computer society conference on computer vision and pattern recognition workshops: IEEE; 2012. p. 1–6. Rathi S, Sharma S, Ojha S, Kumar K. Violence Recognition from Videos Using Deep Learning. Proceedings of International Conference on Recent Trends in Computing: ICRTC 2022: Springer; 2023. p. 69–77. Sultani W, Chen C, Shah M. Real-world anomaly detection in surveillance videos. Proceedings of the IEEE conference on computer vision and pattern recognition2018. p. 6479-88. Gallifant J, Fiske A, Levites Strekalova YA, Osorio-Valencia JS, Parke R, Mwavu R, et al. Peer review of GPT-4 technical report and systems card. 2024;3(1):e0000417. Wang W, Lv Q, Yu W, Hong W, Qi J, Wang Y, et al. Cogvlm: Visual expert for pretrained language models. 2023. Liu H, Li C, Wu Q, Lee YJJAinips. Visual instruction tuning. 2024;36. Hu EJ, Shen Y, Wallis P, Allen-Zhu Z, Li Y, Wang S, et al. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:210609685. 2021. Kozhamkulova Z, Kirgizbayeva B, Sembina G, Smailova U, Suleimenova M, Keneskanova A, et al. MoveNET Enabled Neural Network for Fast Detection of Physical Bullying in Educational Institutions. 2023;14(5). AlDahoul N, Karim HA, Datta R, Gupta S, Agrawal K, Albunni A. Convolutional neural network-long short term memory based IOT node for violence detection. 2021 IEEE International Conference on Artificial Intelligence in Engineering and Technology (IICAIET): IEEE; 2021. p. 1–6. Deepak K, Vignesh L, Chandrakala S. Autocorrelation of gradients based violence detection in surveillance videos. ICT Express. 2020;6(3):155–9. Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-5668231","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":391858792,"identity":"f4e527c1-05e4-4904-bd89-5456ee19b597","order_by":0,"name":"Houming Gong","email":"","orcid":"","institution":"Hunan University of Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Houming","middleName":"","lastName":"Gong","suffix":""},{"id":391858793,"identity":"0b9d6bb7-1474-40b9-a491-c96cc0ae92a3","order_by":1,"name":"Tao Li","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAwUlEQVRIiWNgGAWjYDACdiD+2ADl8BCjg4eZgYFxJkgLGylamHlJ0mLPzHzwse2OO4nz5zcwPnjbxiBvTtgWtmTj3DPPEjccY2A2nNvGYLizgaAWHjPp3LbDiRvYGNikedsYEgwOENTC//23JVDL/DYG9t9EauFhY2YEamk4xsDGTJyWw2zGkr1th403HEtslpxzTsJwAyEt7O3NDz/8bDssO7/58MEPb8ps5AnaggQYG4CEBPHqR8EoGAWjYBTgBgAYtTpuvoCAWwAAAABJRU5ErkJggg==","orcid":"","institution":"Hunan University of Science and Technology","correspondingAuthor":true,"prefix":"","firstName":"Tao","middleName":"","lastName":"Li","suffix":""},{"id":391858794,"identity":"1291a697-73e8-4944-9fe1-7ffa096fc032","order_by":2,"name":"Cong Chen","email":"","orcid":"","institution":"Hunan University of Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Cong","middleName":"","lastName":"Chen","suffix":""},{"id":391858795,"identity":"0759beea-b992-4496-a1e7-2d03dd17cc02","order_by":3,"name":"Wei Lu","email":"","orcid":"","institution":"Hunan University of Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Wei","middleName":"","lastName":"Lu","suffix":""},{"id":391858796,"identity":"023b47bb-f0d7-4053-a1ec-630da2087e6c","order_by":4,"name":"Wei Qu","email":"","orcid":"","institution":"Hunan University of Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Wei","middleName":"","lastName":"Qu","suffix":""},{"id":391858797,"identity":"9d150ee9-c027-42a9-b136-588f616427d7","order_by":5,"name":"Ruiqi Du","email":"","orcid":"","institution":"Hunan University of Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Ruiqi","middleName":"","lastName":"Du","suffix":""}],"badges":[],"createdAt":"2024-12-18 09:53:06","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-5668231/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-5668231/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":72416089,"identity":"ecc5669f-eeb4-4baf-9a73-6586f1d4e4a3","added_by":"auto","created_at":"2024-12-26 20:35:13","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":425414,"visible":true,"origin":"","legend":"\u003cp\u003eSamples images from the collected dataset\u003c/p\u003e","description":"","filename":"1.png","url":"https://assets-eu.researchsquare.com/files/rs-5668231/v1/f6b8bc28096c298596136ec1.png"},{"id":72416086,"identity":"9da4db3a-bd93-44c5-9245-b8a71c665ea0","added_by":"auto","created_at":"2024-12-26 20:35:12","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":38140,"visible":true,"origin":"","legend":"\u003cp\u003eThe effect of dataset size on model performance\u003c/p\u003e","description":"","filename":"2.png","url":"https://assets-eu.researchsquare.com/files/rs-5668231/v1/398fbcfd569d4e3a1a09279a.png"},{"id":72416560,"identity":"a1c3b11f-16a9-465b-abf8-0e877a28e965","added_by":"auto","created_at":"2024-12-26 20:59:16","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1003320,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-5668231/v1/bbc75ff6-9176-4234-8add-783b85ea48ee.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"BullySense: A Approach for Campus Bullying Detection Leveraging LLava Foundation Model","fulltext":[{"header":"1 Introduction","content":"\u003cp\u003eIn recent years, incidents of campus bullying have become alarmingly frequent, drawing widespread attention from various sectors of society. The term \"mobbing\" was coined by Lorenz [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e] to describe instances where a group of animals collectively attacked a single victim. In studies of classroom behavior in Norway during the 1990s, Olweus [\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e] adopted the term \"bullying\" to describe situations where victims experienced repeated aggression over an extended period. Peter et al. [\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e] further categorized bullying into five distinct types: physical abuse, relational abuse, verbal abuse, cyberbullying, and sexual harassment.\u003c/p\u003e \u003cp\u003eThe roots of campus bullying are complex and multifaceted, encompassing individual, familial, and societal dimensions [\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e, \u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e]. Timely identification and intervention are crucial for effectively addressing this issue. However, teachers and parents cannot continuously monitor students' well-being, and victims often hesitate to report their experiences due to fear or shame. This delay in addressing bullying allows such behaviors to escalate, severely impacting the mental and physical health of the victims [\u003cspan additionalcitationids=\"CR7\" citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e]. This study focuses on leveraging surveillance systems and visual content, such as images, to detect campus bullying and violence promptly.\u003c/p\u003e \u003cp\u003eWith the rapid advancement of artificial intelligence, the vision language models, such as Qwen-VL and LLaMA, have demonstrated exceptional capabilities in image understanding and natural language processing [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e, \u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e]. These models can analyze image information, conduct sentiment analysis, and recognize behaviors, offering new approaches for detecting campus bullying. By developing detection algorithms based on VLMs, this research aims to identify bullying behaviors effectively, assisting educators and mental health professionals in providing timely support and intervention. The significance of this study lies not only in enhancing campus safety but also in offering innovative methodologies for future research and technological applications. This study presents three main contributions:\u003c/p\u003e \u003cp\u003e \u003cstrong\u003eDevelopment of a Campus Bullying Detection Dataset\u003c/strong\u003e \u003cp\u003eWe curated a comprehensive dataset specifically designed for campus bullying detection. This dataset includes diverse image data representing typical bullying scenarios, such as physical violence and group intimidation, providing extensive training resources for constructing robust LLM-based models.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003cstrong\u003eProposal of an VLM-Based Monitoring Framework\u003c/strong\u003e \u003cp\u003eA regulatory model grounded in VLMs was developed to accurately identify campus bullying behaviors. By learning the behavioral patterns associated with bullying, the model exhibits enhanced robustness in complex real-world settings, effectively distinguishing between playful interactions and actual bullying incidents.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003cstrong\u003eExperimental Validation and Comparative Analysis\u003c/strong\u003e \u003cp\u003eExtensive experiments were conducted to validate the effectiveness of the proposed approach in detecting campus bullying. Results demonstrate significant advantages in terms of detection accuracy and inference capabilities compared to traditional methods, underscoring the proposed framework\u0026rsquo;s precision and reliability as a solution for campus bullying detection.\u003c/p\u003e \u003c/p\u003e"},{"header":"2 Related Work","content":"\u003cp\u003eWith the advancement of artificial intelligence, numerous researchers have explored using deep neural networks for violence detection to achieve real-time identification of bullying behaviors. Aldehim et al. [\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e] proposed a method called TSODL VD, designed for accurate and efficient recognition of violence in surveillance videos. By employing the TSO protocol as a hyperparameter enhancer for the residual DenseNet model, their approach improved violence detection performance. Garcia-Cobo et al. [\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e] introduced an advanced architecture for violence detection in surveillance videos that integrates human posture extraction and motion variation detection to autonomously identify violent events. Husz\u0026aacute;r et al. [\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e] proposed two innovative architectures for classifying violence in video clips using action recognition features from the Kinetics 400 dataset. The FT model optimized X3D M parameters pre-trained on Kinetics 400, while the TL model extracted spatial features without modifying these parameters and added multiple fully connected layers for training. Mahareek et al. [\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e] introduced a novel model for anomaly detection in surveillance videos by seamlessly integrating 3DCNN and ConvLSTM architectures, demonstrating superior performance compared to standalone 3DCNN configurations. Hsairi et al. [\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e] employed widely recognized models such as sequential CNNs, MobileNetV2, and VGG-16 to evaluate performance on a large annotated dataset containing violent and non-violent images across eight categories. Techniques like data augmentation, transfer learning, and fine-tuning were utilized to enhance model performance. Ullah et al. [\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e] developed a violence detection system for surveillance videos using computer vision and AI technologies to enhance campus security.\u003c/p\u003e \u003cp\u003eWhile deep neural networks show technical promise for bullying detection, they face notable challenges. These include the complexity of campus environments, the diversity of student behaviors, and background interference, which can lead to recognition errors. Additionally, understanding contextual semantics and non-verbal behaviors (e.g., body language) remains a significant limitation.\u003c/p\u003e \u003cp\u003eThe development of large language models (LLMs) has significantly advanced natural language processing applications. VLMs have further expanded these capabilities by integrating visual data, enabling the seamless understanding of textual and image-based information. This integration not only improves sentiment analysis but also broadens the potential for image content understanding, enhancing adaptability and practicality. Zhao et al. [\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e] introduced the Multimodal Image Relationship Benchmark (MIRB) to evaluate the ability of vision language models to compare, analyze, and reason across multiple images. Jin et al. [\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e] presented Chat-UniVi, a unified vision language model employing dynamic visual tokens to represent both images and videos. This framework efficiently captures spatial details for images and temporal relationships for videos. Zhu et al. [\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e] proposed MiniGPT-4, which aligns a frozen visual encoder with the advanced LLM Vicuna using a projection layer. This work demonstrated that correctly aligning visual features with LLMs could unlock many advanced multimodal capabilities showcased by GPT-4. Carneros-Prado et al. [\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e] conducted a comparative analysis of sentiment detection using GPT-3.5 and IBM Watson on a dataset of 30,000 tweets related to the COVID-19 pandemic. Results indicated that GPT-3.5 outperformed Watson in detecting nuanced emotions, such as sarcasm. Yang et al. [\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e] addressed the poor performance of LLMs in sentiment recognition with limited data by proposing a novel approach that combines LLM knowledge with contrastive prompting. This method augments training samples using LLM commonsense knowledge and enhances semantic representation by training on unlabeled data with contrastive embeddings. Nadeem et al. [\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e] compared fine-tuned deep learning models and general-purpose LLMs for image-related tasks, highlighting that GPT-4 performed better on smaller datasets, emphasizing the practicality of foundational LLMs in data-scarce scenarios. Zanella et al. [\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e] introduced LAVAD, a language-based video anomaly detection framework that transforms pre-trained LLMs and VLMs into effective video anomaly detectors in a zero-shot paradigm.\u003c/p\u003e \u003cp\u003eBuilding on these studies, leveraging the image understanding and semantic reasoning capabilities of VLMs can effectively identify campus bullying incidents, addressing challenges faced by deep neural network-based approaches. Our experimental results validate this conclusion, demonstrating the efficacy of large models in detecting campus bullying and violence.\u003c/p\u003e"},{"header":"3 Methodology","content":"\u003cdiv id=\"Sec4\" class=\"Section2\"\u003e \u003ch2\u003e3.1 Data Collection and Preprocessing\u003c/h2\u003e \u003cp\u003eCurrently, widely-used violent video datasets include the Hockey Dataset [\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e], Movie Dataset [\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e], Violent Flow Dataset [\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e], Real-Life Violence Situations (RLVS) [\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e], and UCF-Crime [\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e]. However, datasets specific to campus violence remain limited, largely due to the sensitivity of such incidents and privacy concerns involving minors. To address this gap, our research team created a custom dataset to ensure sufficient samples for model training and testing. The data, sourced from diverse materials depicting bullying scenarios, strictly adhered to ethical guidelines and privacy standards, ensuring no personally identifiable information was included.\u003c/p\u003e \u003cp\u003e \u003cstrong\u003eImage Collection\u003c/strong\u003e \u003cp\u003eIn the process of constructing a campus bullying behavior dataset, we utilized diverse search phrases such as \"physical aggression,\" \"campus violence,\" \"bullying,\" and \"school fight\" to gather video materials from the internet. These resources, often sourced from news reports, surveillance footage, and social media platforms, ensured coverage of various scenarios. Key frames representing bullying behaviors were manually extracted, focusing on critical moments that showcased physical conflict, facial expressions, and environmental context. Additionally, normal campus activity images, such as sports and classroom interactions, were collected to help the model differentiate bullying from routine behaviors. Figure\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e illustrates representative examples.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003cstrong\u003eImage Annotation\u003c/strong\u003e \u003cp\u003eIn the initial stages of dataset construction, our goal was to provide detailed annotations for each image, describing specific actions and environmental contexts to help the model comprehensively understand campus bullying behaviors. However, this approach did not yield the expected accuracy, likely due to the diverse and context-dependent nature of bullying behaviors. To address this, we simplified the annotation strategy by classifying images as either \"bullying\" or \"normal behavior.\" This adjustment allowed the model to autonomously learn distinguishing features, improving its ability to generalize across various scenarios while enhancing recognition precision.\u003c/p\u003e \u003c/p\u003e \u003cp\u003eCollecting bullying content was a significant challenge. In total, we gathered 1022 images, including 550 depicting campus bullying incidents and 472 showing normal campus life. For the training phase, 440 bullying images and 360 normal images were used. Additionally, we retained 110 bullying images and 112 normal images as a test set to evaluate the model's performance.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec5\" class=\"Section2\"\u003e \u003ch2\u003e3.2 Model Selection and Training\u003c/h2\u003e \u003cp\u003eTo construct BullySense, we evaluated several pretrained VLMs to assess their applicability in campus safety detection tasks [\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e, \u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e, \u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e]. Among these, the pretrained LLaVA model demonstrated reliable detection responses and robust zero-shot performance, indicating a foundational understanding of campus safety recognition [\u003cspan citationid=\"CR30\" class=\"CitationRef\"\u003e30\u003c/span\u003e]. Consequently, BullySense was built upon the pretrained LLaVA-1.5 model, with fine-tuning specifically aimed at identifying campus bullying.\u003c/p\u003e \u003cp\u003eThrough fine-tuning, our goal was to enhance the model\u0026rsquo;s capacity to recognize bullying images and apply it to campus safety monitoring tasks. The training process employed LoRA [\u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e] to fine-tune the visual-language components of LLaVA-1.5. This approach targeted task-specific datasets emphasizing bullying behavior detection. By leveraging LoRA, we introduced task-specific adaptations without retraining the entire model, preserving the original model's generalization capabilities.\u003c/p\u003e \u003cp\u003eSpecifically, LoRA modifies trainable parameters by aligning the visual feature projection layers with language embeddings while incorporating low-rank matrices into the attention layers for efficient parameter updates. These matrices enable targeted updates to the attention mechanisms, thereby improving the model's ability to interpret and recognize content related to campus bullying. By selectively fine-tuning only these parameters, we significantly reduced computational costs and training time while enhancing task-specific performance.\u003c/p\u003e \u003cp\u003eThe fine-tuning focused on improving the model's capability to generate contextually aligned image descriptions, ensuring accurate identification and labeling of bullying content. This approach optimized the model for practical application in monitoring campus environments effectively.\u003c/p\u003e \u003c/div\u003e"},{"header":"4 Experiments","content":"\u003cp\u003eBullySense, capitalizing on the sophisticated capabilities of VLMs in the realms of image comprehension and affective computing, offers a detection solution for campus bullying that is both more accurate and encompassing than conventional approaches. To augment the VLM's ability to discern and interpret characteristics indicative of campus bullying, BullySense has been developed by leveraging the foundational strengths of pre-trained VLMs. We have amassed and meticulously annotated an array of image datasets sourced from the internet, encompassing imagery of both bullying incidents and typical campus activities. It is upon this curated dataset that BullySense has been fine-tuned. The empirical findings from our study demonstrate that BullySense surpasses current models in its ability to identify acts of campus bullying, offering not only enhanced detection accuracy but also more nuanced analysis and more dependable forecasting.\u003c/p\u003e \u003cdiv id=\"Sec7\" class=\"Section2\"\u003e \u003ch2\u003e4.1 Experimental Setup\u003c/h2\u003e \u003cp\u003eTo optimize deployment costs, we selected the smallest model available for fine-tuning. Training was conducted on a single machine equipped with two RTX 3090 GPUs. Due to memory constraints, the batch size was set to one, and the learning rate was configured at 2e-4. The model was trained for one epoch to enable a quick evaluation of its performance. Based on these initial findings, future iterations may involve extended training over multiple epochs to further enhance the model's performance.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec8\" class=\"Section2\"\u003e \u003ch2\u003e4.2 Evaluation Metric\u003c/h2\u003e \u003cp\u003eThis research employs the F1 score as the primary evaluation metric, emphasizing its role in balancing precision and recall while considering accuracy for comprehensive assessment. Precision measures the proportion of true positive predictions among all positive predictions, reflecting prediction specificity. Recall calculates the proportion of true positives identified out of all actual positives, assessing the model's sensitivity. Accuracy evaluates the overall correctness of predictions, encompassing both positive and negative cases. The F1 score, as the harmonic mean of precision and recall, effectively balances these aspects for robust performance evaluation. The following are the definitions of these essential metrics:\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"No\" id=\"Taba\" border=\"1\"\u003e \u003ccolgroup cols=\"2\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:Precision=\\:\\frac{TP}{TP+FP}\\)\u003c/span\u003e\u003c/span\u003e\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003e(1)\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:Recall=\\:\\frac{TP}{TP+FN}\\)\u003c/span\u003e\u003c/span\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e(2)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:Accuracy=\\:\\frac{TP+TN}{TP+FP+TN+FN}\\)\u003c/span\u003e\u003c/span\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e(3)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eIn the formula, TP (True Positives) is the number of samples correctly predicted as positive, TN (True Negatives) is the number of samples correctly predicted as negative, FN (False Negatives) is the number of positive samples incorrectly predicted as negative, FP (False Positives) is the number of samples incorrectly predicted as positive.\u003cdiv id=\"Equ1\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ1\" name=\"EquationSource\"\u003e\n$$\\:{F}_{1}=2\\times\\:\\frac{Precision\\times\\:Recall}{Precision+Recall}$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e4\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eThe F1 score provides a comprehensive metric that considers both precision and recall, especially useful in cases of class imbalance, which is critical in content safety tasks where both false positives and false negatives can lead to significant consequences. While accuracy is commonly viewed as a general measure of model performance, it may not fully capture the model's ability to differentiate between safe and unsafe content, particularly in datasets with imbalanced class distributions, where a high accuracy could be achieved by predominantly predicting the majority class.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec9\" class=\"Section2\"\u003e \u003ch2\u003e4.3 Baseline Comparison\u003c/h2\u003e \u003cp\u003eCurrently, research on detecting campus bullying behaviors using VLMs is relatively scarce, as the application of multimodal VLMs in this field remains exploratory. To validate the advantages of VLM-based methods, this study compares the performance of four violence detection models based on deep neural networks. By contrasting traditional deep learning models with VLMs, the research highlights the multimodal VLMs' accuracy, robustness, and adaptability to complex scenarios. These findings offer novel perspectives and solutions for bullying detection and potential applications in broader social safety and child protection contexts.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eIntroduction to using the baseline model\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"3\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eStudy\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eApproach\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eDescription\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eRathi et al.,2023\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eCNN implemented with MobileNetV2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eCNN algorithm utilizing MobileNetV2 and OpenCV for violence detection, trained on a balanced dataset of real-life videos dataset.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eKozhamkulova et. al., 2023\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eMoveNet based physical bullying detection\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eMoveNet evaluate body position, artificial neural network to determine whether the scene contains violent situations.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAlDahoul et. al., 2021\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eCNN-LSTM based model\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eCNN learn spatial features from video's frames, LSTM for video classification into violence/non-violence classes.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDeepak et. al., 2020\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eGradients based violence detection\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eExtract autocorrelation of gradient features from the input videos. A discriminative classifier is then used to recognize violent actions in videos.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eCNN with MobileNetV2 [\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e]: Uses MobileNetV2 and OpenCV for training on real-life videos containing equal amounts of violent and non-violent scenes to recognize violence.\u003c/p\u003e \u003cp\u003eMoveNet-based Detection [\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e]: Combines body position features from photo sequences with neural network-based activity classification to identify hostile or violent actions.\u003c/p\u003e \u003cp\u003eCNN-LSTM Model [\u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e]: A novel end-to-end architecture where CNN extracts spatial features, and LSTM classifies video frames as violent or non-violent.\u003c/p\u003e \u003cp\u003eGradient-Based Detection [\u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e]: In the proposed work, first extract autocorrelation of gradient features from the input videos. Then the auto-correlation feature-based representation is fed as an input to discriminative classifiers such as Support Vector Machine (SVM) to classify different normal and abnormal activities.\u003c/p\u003e \u003cp\u003eThese baseline methods offer a robust framework for evaluating the effectiveness of our proposed BullySense model.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec10\" class=\"Section2\"\u003e \u003ch2\u003e4.4 The Impact of Dataset Size\u003c/h2\u003e \u003cp\u003eIn this study, we focused on the impact of fine-tuning on model performance and conducted an in-depth analysis of how the size of the dataset during fine-tuning affects the model's performance. Through this analysis, we aimed to explore the specific impact of datasets of varying sizes on the effectiveness of model training and to seek effective strategies for optimizing model performance by adjusting the amount of data.\u003c/p\u003e \u003cp\u003eThe results presented in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e clearly demonstrate the significant influence of dataset size on model performance. By comparing and analyzing datasets of different sizes, we found that the scale of the dataset has a notable impact on the model's effectiveness. Specifically, we observed that when the dataset is larger (for example, containing 800 images), there is a significant improvement in the model's accuracy and F1 score, indicating that increasing the number of samples can enhance the model's recognition accuracy and robustness. This enhancement may be since a larger dataset provides a richer representation of features and more comprehensive coverage, enabling the model to learn more generalized characteristics.\u003c/p\u003e \u003cp\u003eHowever, we are also aware that collecting large-scale datasets in practical applications faces numerous challenges, including the difficulty, cost, and time required for data acquisition. Furthermore, our research found that beyond a certain critical point of dataset size, the improvement in performance may gradually diminish, possibly due to overfitting or other factors leading to model performance saturation. This suggests that when designing and implementing data collection strategies, we should consider the diversity and quality of the data, not just the quantity, to ensure that the model performs optimally in real-world application scenarios.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec11\" class=\"Section2\"\u003e \u003ch2\u003e4.5 Performance Analysis\u003c/h2\u003e \u003cp\u003eTable\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e compares the performance of BullySense with other baseline models across key evaluation metrics, with higher scores indicating better performance. According to the results, BullySense demonstrates superior capabilities in campus bullying detection tasks. Notably, it achieves a Precision of 97.4% and an F1 Score of 97.7%, showcasing robust performance in accurately identifying bullying behaviors. These results highlight BullySense's strong detection accuracy and resilience compared to baseline models, making it a highly effective tool for addressing campus bullying scenarios.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eThe performance comparison of the proposed model with other models\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"5\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eApproach\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003ePrecision\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eRecall\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eAccuracy\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eF1 Score\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCNN implemented with MobileNetV2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.945\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.936\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.941\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.941\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMoveNet based physical bullying detection\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.936\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.927\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.932\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.932\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCNN-LSTM based model\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.731\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.791\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.752\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.760\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eGradients based violence detection\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.906\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.873\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.892\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.889\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eThe proposed approach\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e\u003cb\u003e0.982\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e\u003cb\u003e0.973\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e\u003cb\u003e0.977\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e\u003cb\u003e0.977\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eDespite the impressive performance demonstrated by BullySense, the model encountered several challenges during practical testing. Specifically, there were instances where the model misclassified bullying behavior as normal interactions, resulting in false negatives (FN). A thorough analysis of these misclassification cases revealed several key influencing factors.\u003c/p\u003e \u003cp\u003e \u003cstrong\u003eEnvironment Context\u003c/strong\u003e \u003cp\u003eThe complexity of the environment in which bullying occurs is a significant factor. For example, in open spaces like school playgrounds, physical contact between students is quite common, making it challenging to accurately determine whether bullying behavior is present based solely on a single image.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003cstrong\u003eImage Quality\u003c/strong\u003e \u003cp\u003eThe clarity of the images plays a crucial role. In cases where the images are blurred or of low quality, even advanced visual recognition models struggle to capture subtle expressions and movements of the students, thereby increasing the difficulty of identifying bullying behavior.\u003c/p\u003e \u003c/p\u003e \u003cp\u003eTo address these challenges, future work could focus on several directions: First, enhancing the model's ability to recognize bullying behavior in complex environments may require the incorporation of additional contextual information and environmental features. Secondly, improving the quality of image and video data collection, including resolution and clarity, is essential to reduce misclassifications caused by image quality issues. Furthermore, the introduction of multimodal data, such as integrating audio and textual information, could provide a more comprehensive approach to bullying behavior recognition.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec12\" class=\"Section2\"\u003e \u003ch2\u003e4.6 Generalization Analysis\u003c/h2\u003e \u003cp\u003eTo comprehensively assess the generalization capability of the proposed model, we selected three representative datasets for performance testing: The Hockey dataset, Violent-Flow dataset, and RLVS violence behavior dataset. These datasets span a range of scenarios from sports competitions to everyday life, thoroughly evaluating the model's adaptability and accuracy in diverse environments. The test results presented in Table\u0026nbsp;\u003cspan refid=\"Tab3\" class=\"InternalRef\"\u003e3\u003c/span\u003e provide significant insights into the model's performance.\u003c/p\u003e \u003cp\u003eFrom the experimental outcomes, our model demonstrated high precision in detecting violent behaviors across other datasets, indicating its ability to accurately identify genuine acts of violence and reducing the misclassification of normal behaviors as violent. This is particularly evident in The Hockey dataset and Violent-Flow dataset, suggesting the model's strong capability in recognizing violence within these specific domains.\u003c/p\u003e \u003cp\u003eHowever, the model's recall performance was less satisfactory, implying that the model falls short in identifying all actual violent incidents, failing to capture every violent event. This performance discrepancy may stem from the model's limited capacity to recognize the complexity and diversity of violent behaviors, especially when confronted with the complex real-life scenarios included in the RLVS dataset, where the model's precision was at its lowest. This further highlights the challenges the model faces in identifying violent behaviors in complex and variable environments.\u003c/p\u003e \u003cp\u003eAnalyzing the reasons for this performance variation, we attribute it to several factors: firstly, the manifestations of violent behaviors across different datasets vary, and the model may not have sufficiently learned these differences; secondly, the samples in the test datasets may exhibit considerable disparities in lighting, angle, and occlusion, affecting the model's recognition capabilities; and finally, the model may have focused too heavily on certain specific features, neglecting other cues that are equally important for the identification of violent behaviors.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab3\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eGeneralization Analysis of the Proposed Model\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"5\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDataset\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003ePrecision\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eRecall\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eAccuracy\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eF1 Score\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eThe Hockey Dataset\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.995\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.772\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.884\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.869\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eViolent-Flow Dataset\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.926\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.590\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.770\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.721\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eReal-Life Violence Situations\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.786\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.633\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.719\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.701\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003c/div\u003e"},{"header":"5 Limitations","content":"\u003cp\u003eWhen discussing the limitations of this paper, we recognize that the current model has several key limitations in detecting campus bullying. Firstly, the model's performance may be affected when dealing with images of low quality or blurry resolution, leading to biases in identification results. It is necessary to increase the diversity of the training dataset to enhance the model's adaptability to various image conditions. Secondly, the model primarily relies on image data for bullying behavior recognition, which limits its application capabilities in audio, text, or other modalities. In the field of multimodal learning, combining visual, audio, and textual information can provide a more comprehensive understanding of the context, potentially improving the accuracy and robustness of detection.\u003c/p\u003e \u003cp\u003eIn summary, although the BullySense model has made certain progress in detecting campus bullying, further research and improvement are still needed in multimodal data processing, model robustness, and computational efficiency. We look forward to developing more comprehensive and accurate tools for campus bullying detection, contributing to the creation of safer campus environments.\u003c/p\u003e"},{"header":"6 Conclusion","content":"\u003cp\u003eThis study presents BullySense, a model designed to detect campus bullying behaviors using advanced vision language mode. By fine-tuning the pre-trained LLaVA-1.5 model, BullySense achieves robust and accurate performance in identifying bullying incidents from visual data. Through an innovative dataset focusing on diverse bullying scenarios and a systematic training process employing LoRA, the model successfully bridges the gap between theoretical advancements in AI and practical applications in campus safety.\u003c/p\u003e \u003cp\u003eThe experimental results demonstrate that BullySense outperforms traditional deep learning models across key metrics such as precision and F1 score, showcasing its capability to handle the complexities of real-world bullying detection. Despite its strengths, challenges persist, including difficulties in handling ambiguous or low-quality images and a reliance on single-modality data, which can limit performance in nuanced scenarios.\u003c/p\u003e \u003cp\u003eThis research underscores the importance of integrating multimodal data and diversifying datasets to further improve model reliability and accuracy. Additionally, reducing computational overhead and validating the model in practical deployment scenarios remain crucial for broad adoption.\u003c/p\u003e \u003cp\u003eIn conclusion, BullySense represents a significant step forward in the field of campus safety, combining cutting-edge AI techniques with practical considerations. Its development and performance provide valuable insights for future research on using advanced AI tools to address complex social issues, ultimately contributing to safer campus environments and better outcomes for students.\u003c/p\u003e"},{"header":"Declarations","content":"\u003ch2\u003eAuthor Contribution\u003c/h2\u003e\u003cp\u003eHouming Gong: Involved in the research conception and design, performed data analysis or statistical processing, drafted the initial and final manuscripts, and made critical revisions and improvements to the paper.Tao Li: Involved in the research conception and design, supervised the implementation and management of the research project.Cong Chen: Responsible for experimental data collection and assisted in the analysis or statistical processing of experimental results.Wei Lu: Involved in the research conception and design, conducted the literature review, and provided the research background and theoretical foundation.Wei Qu: Involved in the research conception and design, and discussed the experimental results.Ruiqi Du: Responsible for data collection.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eLorenz K. Sobre la agresi\u0026oacute;n: el pretendido mal. Siglo xxi; 1998.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eOlweus D, Jimerson SR, Swearer SM, Espelage DLJoJ, SR, Swearer, S. M, Espelage DNYR. Handbook of bullying in schools: An international perspective. 2010:9\u0026ndash;33.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePeter K SJJoCP, Psychiatry. Commentary: Types of bullying, types of intervention: reflections on Arseneault. 2018;59(4):422\u0026ndash;3.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLi J, Hesketh TJFip. Experiences and perspectives of traditional bullying and cyberbullying among adolescents in Mainland China-implications for policy. 2021;12:672223.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZapf D, Einarsen SV. Individual antecedents of bullying: Personality, motives and competencies of victims and perpetrators. Bullying and Harassment in the Workplace. CRC press; 2020. p. 269\u0026ndash;303.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eArslan G, Allen K-A, Tanhan AJCIR. School bullying, mental health, and wellbeing in adolescents: Mediating impact of positive psychological orientations. 2021;14(3):1007\u0026ndash;26.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eEyuboglu M, Eyuboglu D, Pala SC, Oktar D, Demirtas Z, Arslantas D, et al. Traditional school bullying and cyberbullying: Prevalence, the effect on mental health problems and self-harm behavior. 2021;297:113730.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLuo X, Zheng R, Xiao P, Xie X, Liu Q, Zhu K, et al. Relationship between school bullying and mental health status of adolescent students in China: a nationwide cross-sectional study. 2022;70:103043.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLi X, Wu Y, Jiang X, Guo Z, Gong M, Cao H, et al. Enhancing visual document understanding with contrastive learning in large visual-language models. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition2024. p. 15546-55.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTong S, Liu Z, Zhai Y, Ma Y, LeCun Y, Xie S. Eyes wide shut? exploring the visual shortcomings of multimodal llms. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition2024. p. 9568-78.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAldehim G, Asiri MM, Aljebreen M, Mohamed A, Assiri M, Ibrahim SSJIA. Tuna swarm algorithm with deep learning enabled violence detection in smart video surveillance systems. 2023.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGarcia-Cobo G, SanMiguel JCJCV, Understanding I. Human skeletons and change detection for efficient violence detection in surveillance videos. 2023;233:103739.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHuszar VD, Adhikarla VK, N\u0026eacute;gyesi I, Krasznay CJIA. Toward fast and accurate violence detection for automated video surveillance applications. 2023;11:18772\u0026ndash;93.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMahareek EA, El-Sayed EK, El-Desouky NM, El-Dahshan KA. Detecting anomalies in security cameras with 3DCNN and ConvLSTM. 2023.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHsairi L, Alosaimi SM, Alharaz GAJAJfS, Engineering. Violence Detection Using Deep Learning. 2024:1\u0026ndash;11.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eUllah FUM, Obaidat MS, Ullah A, Muhammad K, Hijji M, Baik SWJACS. A comprehensive review on vision-based violence detection in surveillance videos. 2023;55(10):1\u0026ndash;44.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZhao B, Zong Y, Zhang L, Hospedales TJapa. Benchmarking Multi-Image Understanding in Vision and Language Models: Perception, Knowledge, Reasoning, and Multi-Hop Reasoning. 2024.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJin P, Takanobu R, Zhang W, Cao X, Yuan L. Chat-univi: Unified visual representation empowers large language models with image and video understanding. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition2024. p. 13700-10.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZhu D, Chen J, Shen X, Li X, Elhoseiny MJapa. Minigpt-4: Enhancing vision-language understanding with advanced large language models. 2023.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCarneros-Prado D, Villa L, Johnson E, Dobrescu CC, Barrag\u0026aacute;n A, Garc\u0026iacute;a-Mart\u0026iacute;nez B. Comparative study of large language models as emotion and sentiment analysis systems: A case-specific analysis of GPT vs. IBM Watson. International Conference on Ubiquitous Computing and Ambient Intelligence: Springer; 2023. p. 229\u0026thinsp;\u0026ndash;\u0026thinsp;39.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eYang H, Zi Y, Qin H, Zheng H, Hu YJJoCS, Applications S. Advancing emotional analysis with large language models. 2024;4(3):8\u0026ndash;15.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNadeem M, Javed L, Cambria E, Hussain A. Are Foundation Models the Next-Generation Social Media Content Moderators?\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZanella L, Menapace W, Mancini M, Wang Y, Ricci E. Harnessing Large Language Models for Training-free Video Anomaly Detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition2024. p. 18527-36.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBermejo Nievas E, Deniz Suarez O, Bueno Garc\u0026iacute;a G, Sukthankar R. Violence detection in video using computer vision techniques. Computer Analysis of Images and Patterns: 14th International Conference, CAIP 2011, Seville, Spain, August 29\u0026ndash;31, 2011, Proceedings, Part II 14: Springer; 2011. p. 332-9.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHassner T, Itcher Y, Kliper-Gross O. Violent flows: Real-time detection of violent crowd behavior. 2012 IEEE computer society conference on computer vision and pattern recognition workshops: IEEE; 2012. p. 1\u0026ndash;6.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRathi S, Sharma S, Ojha S, Kumar K. Violence Recognition from Videos Using Deep Learning. Proceedings of International Conference on Recent Trends in Computing: ICRTC 2022: Springer; 2023. p. 69\u0026ndash;77.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSultani W, Chen C, Shah M. Real-world anomaly detection in surveillance videos. Proceedings of the IEEE conference on computer vision and pattern recognition2018. p. 6479-88.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGallifant J, Fiske A, Levites Strekalova YA, Osorio-Valencia JS, Parke R, Mwavu R, et al. Peer review of GPT-4 technical report and systems card. 2024;3(1):e0000417.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWang W, Lv Q, Yu W, Hong W, Qi J, Wang Y, et al. Cogvlm: Visual expert for pretrained language models. 2023.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLiu H, Li C, Wu Q, Lee YJJAinips. Visual instruction tuning. 2024;36.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHu EJ, Shen Y, Wallis P, Allen-Zhu Z, Li Y, Wang S, et al. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:210609685. 2021.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKozhamkulova Z, Kirgizbayeva B, Sembina G, Smailova U, Suleimenova M, Keneskanova A, et al. MoveNET Enabled Neural Network for Fast Detection of Physical Bullying in Educational Institutions. 2023;14(5).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAlDahoul N, Karim HA, Datta R, Gupta S, Agrawal K, Albunni A. Convolutional neural network-long short term memory based IOT node for violence detection. 2021 IEEE International Conference on Artificial Intelligence in Engineering and Technology (IICAIET): IEEE; 2021. p. 1\u0026ndash;6.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDeepak K, Vignesh L, Chandrakala S. Autocorrelation of gradients based violence detection in surveillance videos. ICT Express. 2020;6(3):155\u0026ndash;9.\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Campus Bullying, Violence Recognition, VLMs, Educational Institution Safety","lastPublishedDoi":"10.21203/rs.3.rs-5668231/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-5668231/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eCampus bullying has become a significant social issue, with profound impacts on the mental and physical well-being of students. Timely and effective detection is crucial for addressing this problem. To tackle these challenges, this study introduces BullySense, a campus bullying detection model built on the vision language model (VLM) framework. Utilizing the pre-trained LLaVA-1.5 model, BullySense is fine-tuned with a curated dataset of bullying and normal campus activity images, employing Low-Rank Adaptation (LoRA) to enhance task-specific performance. Experimental results show that BullySense outperforms traditional models, achieving precision of 0.982 and F1 scores of 0.977. While challenges remain, such as handling low-quality images and extending to multimodal data, this work demonstrates the potential of AI for improving campus safety and lays the groundwork for future advancements.\u003c/p\u003e","manuscriptTitle":"BullySense: A Approach for Campus Bullying Detection Leveraging LLava Foundation Model","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-12-26 20:35:08","doi":"10.21203/rs.3.rs-5668231/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"2461f361-add1-4d7b-9d8d-04a853da55f6","owner":[],"postedDate":"December 26th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2024-12-30T01:38:18+00:00","versionOfRecord":[],"versionCreatedAt":"2024-12-26 20:35:08","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-5668231","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-5668231","identity":"rs-5668231","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}