Evaluation of the use of box size priors for 6D plane segment tracking from point clouds with applications in cargo packing

Evaluation of the use of box size priors for 6D plane segment tracking from point clouds with applications in cargo packing | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Evaluation of the use of box size priors for 6D plane segment tracking from point clouds with applications in cargo packing Guillermo Alberto Camacho Muñoz, Sandra Esperanza Nope-Rodríguez, and 3 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-3918980/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 06 Aug, 2024 Read the published version in EURASIP Journal on Image and Video Processing → Version 1 posted 4 You are reading this latest preprint version Abstract Available solutions to assist human operators in cargo packing processes offer alternatives to maximize the spatial occupancy of containers used in intralogistics. However, these solutions consist of sequential instructions for picking each box and positioning it in the containers, making it challenging for an operator to interpret and requiring them to alternate between reading the instructions and executing the task. A potential solution to these issues lies in a tool that naturally communicates each box's initial and final location in the desired sequence to the operator. While 6D visual object tracking systems have demonstrated good performance, they have yet to be evaluated in real-world scenarios of manual box packing. They also need to use the available prior knowledge of the packing operation, such as the number of boxes, box size, and physical packing sequence. This study explores the inclusion of box size priors in 6D plane segment tracking systems driven by images from moving cameras and quantifies their contribution in terms of tracker performance when assessed in manual box packing operations. To do this, it compares the performance of a plane segment tracking system, considering variations in the tracking algorithm and camera speed (onboard the packing operator) during the mapping of a manual cargo packing process. The tracking algorithm varies at two levels: algorithm ( A wpk ), which integrates prior knowledge of box sizes in the scene, and algorithm ( A woutpk ), which assumes ignorance of box properties. Camera speed is also evaluated at two levels: low speed ( S low ) and high speed ( S high ). This study analyzes the impact of these factors on the precision, recall, and F1-score of the plane segment tracking system. ANOVA analysis was applied to the precision and F1-score results, which allows determining that neither the camera speed-algorithm interactions nor the camera speed are significant in the precision of the tracking system. The factor that presented a significant effect is the tracking algorithm. Tukey's pairwise comparisons concluded that the precision and F1-score of each algorithm level are significantly different, with algorithm A wpk being superior in each evaluation. This superiority reaches its maximum in the tracking of top plane segments: 22 and 14 percentage units for precision and F1-score metrics, respectively. However, the results on the recall metric remain similar with and without the addition of prior knowledge. The contribution of including prior knowledge of box sizes in ( 6 D ) plane segment tracking algorithms is identified in reducing false positives. This reduction is associated with significant increases in the tracking system's precision and F1-score metrics. Future work will investigate whether the identified benefits propagate to the tracking problem on objects composed of plane segments, such as cubes or boxes. Visual 6D tracking plane tracking manual packing of cargo 6D object detection visual tracking on dynamic environment multi object tracking integration of size priors Full Text Cite Share Download PDF Status: Published Journal Publication published 06 Aug, 2024 Read the published version in EURASIP Journal on Image and Video Processing → Version 1 posted Reviewers agreed at journal 20 Feb, 2024 Reviewers invited by journal 16 Feb, 2024 Editor assigned by journal 04 Feb, 2024 First submitted to journal 02 Feb, 2024 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-3918980","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":273380018,"identity":"c1a8cf52-75e8-4e45-baa1-d7d2d2ffaa82","order_by":0,"name":"Guillermo Alberto Camacho Muñoz","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABZElEQVRIie3RMUvDQBQH8CuFZDns+kKl+QoXAqVg6ecQXHIE0iURoSAVCl4JxKXY1eKXqIuTw5VApn6ASEUaAk4dioMISvVSa0jV7g73n95x97vj3UNIRuZfptTn88LSQirLt0R10sgqsk0Yt7YI5kVC4DdB6AeB4vIPoh/a4pXu41EFlZPnuNc6rmhpkp7dNWtk5gfwSqCGVP8WUK+5IUZMBZl2vBFTzKob2R3t2jGN6ZNjkodJoA0ImAhHp4Ai55tcCUIDyxtzjKquwul45ioa4yEdx9QnmABl4NahxMItslqT8pu7EuR+mpGPNTHeCZwzfVEkOmSErYlS9QJBYpyRrKD9VLwiPgQXCcGJ6CWyvJGv1A+8S5uOBqIXxm1Ty8g+ASPATqdh5b3oF+1kvuxZ3lD105n70qJDNUzENa3aXtyeTxbdpl5Rw5t4mf8YyedWRruifM0rnwvbeVJGRkZGZpNPGNSSkdny020AAAAASUVORK5CYII=","orcid":"https://orcid.org/0000-0003-0858-7814","institution":"Universidad del Valle","correspondingAuthor":true,"prefix":"","firstName":"Guillermo","middleName":"Alberto Camacho","lastName":"Muñoz","suffix":""},{"id":273380019,"identity":"1cd47749-fd37-450e-b74d-157f9ef91e2c","order_by":1,"name":"Sandra Esperanza Nope-Rodríguez","email":"","orcid":"https://orcid.org/0000-0003-0245-1086","institution":"Universidad del Valle","correspondingAuthor":false,"prefix":"","firstName":"Sandra","middleName":"Esperanza","lastName":"Nope-Rodríguez","suffix":""},{"id":273380020,"identity":"3bc365f1-6c0d-4d61-b304-fe5ee8566238","order_by":2,"name":"Humberto Loaiza-Correa","email":"","orcid":"","institution":"Universidad del Valle","correspondingAuthor":false,"prefix":"","firstName":"Humberto","middleName":"","lastName":"Loaiza-Correa","suffix":""},{"id":273380021,"identity":"9d03d250-eed2-48f0-b523-42e59f7286ca","order_by":3,"name":"João Paulo Silva do Monte Lima","email":"","orcid":"","institution":"Universidade Federal Rural de Pernambuco","correspondingAuthor":false,"prefix":"","firstName":"João","middleName":"Paulo Silva do Monte","lastName":"Lima","suffix":""},{"id":273380022,"identity":"24ce6040-c567-421b-8b0e-d52607e7ffca","order_by":4,"name":"Rafael Alves Roberto","email":"","orcid":"","institution":"Universidade Federal de Pernambuco Centro de Informatica","correspondingAuthor":false,"prefix":"","firstName":"Rafael","middleName":"Alves","lastName":"Roberto","suffix":""}],"badges":[],"createdAt":"2024-02-01 22:54:19","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-3918980/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-3918980/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1186/s13640-024-00636-1","type":"published","date":"2024-08-06T15:58:14+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":62298575,"identity":"f9b24288-25b8-4d85-a7af-a581309acd67","added_by":"auto","created_at":"2024-08-12 16:14:47","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1491829,"visible":true,"origin":"","legend":"","description":"","filename":"Manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-3918980/v1_covered_9ee8751f-9909-44a5-828e-f58eae7d2275.pdf"}],"financialInterests":"","formattedTitle":"Evaluation of the use of box size priors for 6D plane segment tracking from point clouds with applications in cargo packing","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":true,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"eurasip-journal-on-image-and-video-processing","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"jivp","sideBox":"Learn more about [EURASIP Journal on Image and Video Processing](http://jivp-eurasipjournals.springeropen.com)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/jivp/default.aspx","title":"EURASIP Journal on Image and Video Processing","twitterHandle":"@SpringerEng","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Visual 6D tracking, plane tracking, manual packing of cargo, 6D object detection, visual tracking on dynamic environment, multi object tracking, integration of size priors","lastPublishedDoi":"10.21203/rs.3.rs-3918980/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-3918980/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eAvailable solutions to assist human operators in cargo packing processes offer alternatives to maximize the spatial occupancy of containers used in intralogistics. However, these solutions consist of sequential instructions for picking each box and positioning it in the containers, making it challenging for an operator to interpret and requiring them to alternate between reading the instructions and executing the task. A potential solution to these issues lies in a tool that naturally communicates each box's initial and final location in the desired sequence to the operator. While 6D visual object tracking systems have demonstrated good performance, they have yet to be evaluated in real-world scenarios of manual box packing. They also need to use the available prior knowledge of the packing operation, such as the number of boxes, box size, and physical packing sequence. This study explores the inclusion of box size priors in 6D plane segment tracking systems driven by images from moving cameras and quantifies their contribution in terms of tracker performance when assessed in manual box packing operations. To do this, it compares the performance of a plane segment tracking system, considering variations in the tracking algorithm and camera speed (onboard the packing operator) during the mapping of a manual cargo packing process. The tracking algorithm varies at two levels: algorithm (\u003cem\u003e\u003cstrong\u003eA\u003c/strong\u003e\u003c/em\u003e\u003csub\u003e\u003cem\u003e\u003cstrong\u003ewpk\u003c/strong\u003e\u003c/em\u003e\u003c/sub\u003e), which integrates prior knowledge of box sizes in the scene, and algorithm (\u003cem\u003e\u003cstrong\u003eA\u003c/strong\u003e\u003c/em\u003e\u003csub\u003e\u003cem\u003e\u003cstrong\u003ewoutpk\u003c/strong\u003e\u003c/em\u003e\u003c/sub\u003e), which assumes ignorance of box properties. Camera speed is also evaluated at two levels: low speed (\u003cem\u003e\u003cstrong\u003eS\u003c/strong\u003e\u003c/em\u003e\u003csub\u003e\u003cem\u003e\u003cstrong\u003elow\u003c/strong\u003e\u003c/em\u003e\u003c/sub\u003e) and high speed (\u003cem\u003e\u003cstrong\u003eS\u003c/strong\u003e\u003c/em\u003e\u003csub\u003e\u003cem\u003e\u003cstrong\u003ehigh\u003c/strong\u003e\u003c/em\u003e\u003c/sub\u003e). This study analyzes the impact of these factors on the precision, recall, and F1-score of the plane segment tracking system. ANOVA analysis was applied to the precision and F1-score results, which allows determining that neither the camera speed-algorithm interactions nor the camera speed are significant in the precision of the tracking system. The factor that presented a significant effect is the tracking algorithm. Tukey's pairwise comparisons concluded that the precision and F1-score of each algorithm level are significantly different, with algorithm \u003cem\u003e\u003cstrong\u003eA\u003c/strong\u003e\u003c/em\u003e\u003csub\u003e\u003cem\u003e\u003cstrong\u003ewpk\u003c/strong\u003e\u003c/em\u003e\u003c/sub\u003e being superior in each evaluation. This superiority reaches its maximum in the tracking of top plane segments: \u003cem\u003e\u003cstrong\u003e22\u003c/strong\u003e\u003c/em\u003e and \u003cem\u003e\u003cstrong\u003e14\u003c/strong\u003e\u003c/em\u003e percentage units for precision and F1-score metrics, respectively. However, the results on the recall metric remain similar with and without the addition of prior knowledge. The contribution of including prior knowledge of box sizes in (\u003cstrong\u003e6\u003c/strong\u003e\u003cem\u003e\u003cstrong\u003eD\u003c/strong\u003e\u003c/em\u003e) plane segment tracking algorithms is identified in reducing false positives. This reduction is associated with significant increases in the tracking system's precision and F1-score metrics. Future work will investigate whether the identified benefits propagate to the tracking problem on objects composed of plane segments, such as cubes or boxes.\u003c/p\u003e","manuscriptTitle":"Evaluation of the use of box size priors for 6D plane segment tracking from point clouds with applications in cargo packing","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-02-20 05:04:14","doi":"10.21203/rs.3.rs-3918980/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"reviewerAgreed","content":"","date":"2024-02-21T01:38:53+00:00","index":0,"fulltext":""},{"type":"reviewersInvited","content":"","date":"2024-02-16T10:37:19+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2024-02-05T04:57:18+00:00","index":"","fulltext":""},{"type":"submitted","content":"EURASIP Journal on Image and Video Processing","date":"2024-02-02T10:53:18+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"eurasip-journal-on-image-and-video-processing","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"jivp","sideBox":"Learn more about [EURASIP Journal on Image and Video Processing](http://jivp-eurasipjournals.springeropen.com)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/jivp/default.aspx","title":"EURASIP Journal on Image and Video Processing","twitterHandle":"@SpringerEng","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"93277aa4-2a19-47c7-822b-b655bd315217","owner":[],"postedDate":"February 20th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2024-08-12T16:06:42+00:00","versionOfRecord":{"articleIdentity":"rs-3918980","link":"https://doi.org/10.1186/s13640-024-00636-1","journal":{"identity":"eurasip-journal-on-image-and-video-processing","isVorOnly":false,"title":"EURASIP Journal on Image and Video Processing"},"publishedOn":"2024-08-06 15:58:14","publishedOnDateReadable":"August 6th, 2024"},"versionCreatedAt":"2024-02-20 05:04:14","video":"","vorDoi":"10.1186/s13640-024-00636-1","vorDoiUrl":"https://doi.org/10.1186/s13640-024-00636-1","workflowStages":[]},"version":"v1","identity":"rs-3918980","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-3918980","identity":"rs-3918980","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}