{"paper_id":"2c6ef39b-a783-4331-996d-64402ff1d020","body_text":"Condensed Reasoning Prompting: Efficient Strategies, Evaluations, and Trade Offs in Large Language Model Reasoning | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Condensed Reasoning Prompting: Efficient Strategies, Evaluations, and Trade Offs in Large Language Model Reasoning Gautam Mehra, Danish Khan This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6170708/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Recent advancements in large language models (LLMs) have demonstrated that explicitly prompting for intermediate reasoning steps significantly improves performance in complex tasks. Traditional chain of thought (CoT) prompting, however, can result in verbose outputs that increase both latency and computational cost. Condensed Reasoning Prompting (CRP) addresses this trade-off by encouraging more concise reasoning traces while maintaining high accuracy. In this paper, we systematically evaluate three prompting strategies: Chain Of Thought (CoT), Chain of Draft (CoD) and Condensed Reasoning across multiple datasets, including MMLU, Big Bench, arithmetic, and symbolic reasoning tasks. We report accuracy, average tokens per question, and a token effectiveness metric (accuracy divided by token count). Our experiments are conducted in a zero-shot setting, without specific system instructions to \"skip reasoning,\" providing a more realistic assessment of model capabilities. Results indicate that condensed prompts often match or exceed chain of thought accuracy while reducing token usage, thus offering significant gains in efficiency. We discuss the implications for real-world deployments, highlighting how CRP can enable more efficient LLM applications without compromising performance. Figures Figure 1 1. Introduction Large language models (LLMs) have demonstrated remarkable capabilities in tasks ranging from question answering to code generation and creative writing. A key advancement in improving LLM performance on complex, multi-step tasks has been chain of thought (CoT) prompting, which encourages the model to articulate its intermediate reasoning steps rather than proceeding directly to an answer. Empirical evidence shows that CoT prompting enhances accuracy on arithmetic, logical reasoning, and multi-hop question answering by effectively providing the model with a \"scratchpad\" to decompose problems. However, this benefit often comes with increased verbosity. A fully articulated chain of thought can be quite extensive, particularly for tasks requiring multiple inferential steps. Verbosity increases computational overhead, leads to higher latency in interactive settings, and increases API costs when models are accessed via token-based billing. These issues become significant when deploying LLMs at scale. In response, researchers have begun exploring methods to reduce the token footprint of chain of thought while preserving its accuracy advantages. Condensed Reasoning Prompting (CRP) is one such approach, directing the model to produce succinct intermediate reasoning. By eliminating redundant text and focusing on essential logic, CRP can maintain the performance gains of CoT while substantially reducing token usage. In this paper, we: Review theoretical foundations of stepwise inference in LLMs and recent prompting strategies (e.g., Chain of Draft, Least to Most, Self-Ask). Present a comprehensive evaluation of three prompting strategies: Chain Of Thought (CoT), Chain of Draft (CoD) and Condensed Reasoning across multiple benchmarks: MMLU, Big Bench, arithmetic reasoning, and symbolic reasoning. Introduce a token effectiveness metric, defined as (accuracy ÷ average tokens per question), to quantify the efficiency–accuracy trade-off. Highlight that our experiments are conducted in a zero-shot manner without special system instructions to suppress reasoning for base models, thus reflecting a more natural usage scenario. Our findings indicate that condensed prompts can substantially reduce token usage while maintaining strong performance, underscoring the practical importance of CRP for real-world AI deployments. 2. Background and Related Work 2.1 Stepwise Reasoning in LLMs LLMs generate tokens autoregressively, with each token conditioned on the prompt and previously generated tokens. Chain of Thought (CoT) prompting encourages the model to write down each intermediate step explicitly, which often improves accuracy on tasks requiring multi-step reasoning. The theoretical rationale is that the model's \"internal scratchpad\" becomes externally visible, allowing it to refer to intermediate results and reducing the burden of maintaining everything in hidden states. 2.2 Trade-Off: Accuracy vs. Verbosity While CoT prompts can significantly improve performance, they may be excessively verbose. This verbosity: Increases Latency: More tokens to generate means longer inference time. Raises Costs: Many commercial APIs charge by token usage. Challenges Context Windows: Very long responses can approach context length limits in certain models. 2.3 Condensed Reasoning Prompting To address verbosity, Condensed Reasoning Prompting (CRP) and similar methods (e.g., Chain of Draft) instruct the model to keep reasoning steps minimal. For instance, rather than restating the entire problem scenario, the model is guided to write only the key equations or short logic statements needed to reach the conclusion. Empirical results in prior work have shown that this can reduce token usage by over 70% compared to standard CoT, with minimal accuracy impact. 2.4 Other Prompting Strategies Least to Most Prompting decomposes a complex question into simpler sub-questions, solving them in ascending order of difficulty. Self-Ask Prompting has the model generate its own clarifying sub-questions before arriving at the final answer. ReAct interleaves reasoning with external actions (e.g., search queries) to ground the chain of thought in factual data. 3. Theoretical Underpinnings of Efficient Reasoning 3.1 Cognitive Decomposition When a model breaks down a question into smaller, logically coherent steps, it reduces the likelihood of overlooking crucial details. Each step's output is fed back into the model's context, forming a transparent chain of sub-results. This approach is analogous to how humans solve problems in a methodical, stepwise fashion. 3.2 Transparency and Error Diagnosis Verbose reasoning is often valued for being transparent, users can identify where the logic failed if the final answer is incorrect. However, condensed reasoning can maintain transparency if it concisely highlights the essential steps. In practice, a short, high-signal reasoning trace can be more interpretable than a lengthy explanation. 3.3 Token Efficiency We introduce a token effectiveness metric to capture how efficiently a model's reasoning yields correct answers. Specifically, Token Effectiveness = Accuracy / Average Tokens per Question High token effectiveness indicates that the model is using fewer tokens to achieve higher accuracy. 4. Experimental Setup We evaluate four variants of a GPT-4–style model: base_gpt4o Zero-shot usage of our model without specialized instructions to skip intermediate reasoning. Reflects the model's natural reasoning style. CoT_gpt4o A standard chain of thought approach prompting the model to articulate each step in detail. CoD_gpt4o A chain of draft prompting, instructing the model to produce minimal reasoning steps. crp_gpt4o Our proposed CRP method, which explicitly guides the model to “sketch” the reasoning and keep it succinct while preserving essential logic. 4.2 Zero-Shot Configuration All tests are zero-shot: we do not provide multiple examples or additional system messages to artificially constrain the model's output. Notably, we do not instruct the model to skip reasoning. This choice offers an unbiased measure of how each prompting method naturally balances accuracy and verbosity. 4.3 Datasets We evaluate on four core tasks: MMLU (Massive Multitask Language Understanding): A challenging benchmark covering multiple academic subjects at varying difficulty levels. Big Bench: A collection of tasks spanning common sense, logic, and knowledge. GSM8k: Multi-step math word problems requiring explicit numeric manipulation. Symbolic Reasoning: Classic coin flip problems. For each dataset, we measure: Accuracy: Percentage of questions answered correctly. Average Tokens per Question: The mean number of tokens generated by the model in its response. Token Effectiveness: Accuracy ÷ Average Tokens per Question. 5. Results and Analysis This section presents our main findings. We first provide a summary table comparing MMLU, Big Bench, Arithmetic Reasoning, and Symbolic Reasoning performance for each model, followed by detailed sub-sections for each benchmark. 5.1 Summary of Key Benchmarks Model MMLU Big Bench Arithmetic Symbolic Token Effectiveness CoD_gpt4o 0.72 0.95 0.94 0.68 0.47 CoT_gpt4o 0.79 0.95 0.96 0.79 0.24 base_gpt4o 0.66 0.95 0.98 0.72 0.48 crp_gpt4o 0.75 0.99 0.93 0.74 0.53 From this high-level comparison, we observe: CoT_gpt4o often performs well in certain reasoning tasks (e.g., MMLU at 79%, Symbolic at 79%) but at the cost of lower \"Token Effectiveness\" (0.24). CoD_gpt4o typically achieves a good balance between accuracy and token usage (Token Effectiveness of 0.47). base_gpt4o achieves a high score of 0.98 on arithmetic tasks but shows lower performance on MMLU (0.66). crp_gpt4o demonstrates consistent performance across datasets, with particularly strong results on Big Bench (0.99) and the highest \"Token Effectiveness\" measure (0.53) among the approaches. Below, we provide more detailed metrics for each dataset: mean accuracy, average tokens per question, and token effectiveness. 5.2 MMLU Results Model Mean Accuracy Avg Tokens/Qn Token Effectiveness base_gpt4o 65.5% 194.8 0.336 CoT_gpt4o 78.6% 517.9 0.151 CoD_gpt4o 72.4% 282.7 0.256 crp_gpt4o 75.2% 242.8 0.310 Accuracy: CoT_gpt4o leads with 78.6% on MMLU, surpassing crp_gpt4o (75.2%) by 3.4 percentage points. Token Usage: CoT_gpt4o is the most verbose (517.9 tokens/question), while base_gpt4o uses the fewest tokens (194.8) but has lower accuracy (65.5%). Token Effectiveness: crp_gpt4o (0.31) approaches base_gpt4o (0.34) while achieving notably higher accuracy (75.2% vs. 65.5%). This demonstrates the efficiency of condensed prompting for MMLU tasks. Analysis MMLU tasks span various academic subjects. The results suggest that detailed reasoning (as in CoT_gpt4o) helps achieve higher accuracy but requires significantly more tokens. CRP is more efficient, maintaining competitive performance while generating fewer tokens. 5.3 Big Bench Results Model Mean Accuracy Avg Tokens/Qn Token Effectiveness base_gpt4o 95.2% 134.5 0.7071 CoT_gpt4o 94.8% 355.5 0.267 CoD_gpt4o 95.2% 103.0 0.923 crp_gpt4o 98.6% 111.8 0.882 Accuracy: crp_gpt4o achieves the highest accuracy (98.6%), followed by base_gpt4o and CoD_gpt4o (both at 95.2%). Token Usage: CoD_gpt4o is the most efficient in raw token count (103.07 tokens/question), while CoT_gpt4o is the most verbose (355.55). Token Effectiveness: CoD_gpt4o scores the highest (0.92), with crp_gpt4o close behind (0.88). Analysis Big Bench includes various tasks, from basic knowledge to more logic-intensive questions. The results indicate that condensed prompts can lead to both high accuracy and high token effectiveness. CoT_gpt4o's detailed reasoning does not translate into superior accuracy in this context, suggesting that longer reasoning traces are not always beneficial. 5.4 (GSM8K) Arithmetic Reasoning Results Model Mean Accuracy Avg Tokens/Qn Token Effectiveness base_gpt4o 98% 252.8 0.389 CoT_gpt4o 96% 356.1 0.270 CoD_gpt4o 94% 178.7 0.527 crp_gpt4o 94% 167.0 0.562 Accuracy: base_gpt4o attains the highest accuracy (98%), despite not being specifically prompted for multi-step mathematical reasoning. Token Usage: CoT_gpt4o is the most verbose (356.07 tokens/question). CoD_gpt4o and crp_gpt4o both use fewer than 180 tokens/question. Token Effectiveness: crp_gpt4o leads with 0.56, indicating that it delivers strong accuracy with fewer tokens. CoD_gpt4o follows closely (0.53). Analysis These results highlight that zero-shot base_gpt4o can perform well in arithmetic tasks, potentially due to robust underlying numerical capabilities. However, from an efficiency perspective, condensed prompts remain advantageous because they achieve comparable accuracy (93–94%) with significantly fewer tokens. 5.5 Symbolic Reasoning Results Model Mean Accuracy Avg Tokens/Qn Token Effectiveness base_gpt4o 72% 121.862069 0.594 CoT_gpt4o 79% 247.9655172 0.320 CoD_gpt4o 68% 122.6896552 0.556 crp_gpt4o 74% 126.5172414 0.586 Accuracy: CoT_gpt4o achieves the highest accuracy (79%) but uses approximately twice the tokens compared to base_gpt4o or CoD_gpt4o. Token Usage: base_gpt4o and CoD_gpt4o both use approximately 122 tokens/question, while crp_gpt4o is slightly higher (126.52). Token Effectiveness: base_gpt4o and crp_gpt4o both achieve around 0.59, with CoD_gpt4o at 0.56. CoT_gpt4o lags at 0.32 due to its higher token count. Analysis Symbolic reasoning often benefits from detailed stepwise thinking. However, the data suggests that longer explanations (CoT_gpt4o) are not always the most efficient approach. Condensed reasoning provides a balanced method, offering strong accuracy with relatively minimal token requirements. 5.6 Overall Observations CoT_gpt4o typically achieves high accuracy but is notably verbose, resulting in lower token effectiveness scores. CoD_gpt4o excels in token minimization, it’s often at the cost of accuracy. crp_gpt4o offers a good compromise, frequently ranking near the top in both accuracy and token effectiveness. Collectively, these results indicate that while CoT can be effective, there is significant value in guiding models to be concise. This not only reduces computational costs but can also preserve—and sometimes enhance—accuracy by focusing the model's attention on the essential steps. 6. Discussion 6.1 Zero-Shot vs. Special Instructions A notable contribution of this study is our decision to use base_gpt4o in a zero-shot manner, without instructing it to skip intermediate reasoning. In many prior works, prompts are carefully crafted to minimize reasoning text, which we have observed can affect accuracy. By allowing base_gpt4o to produce its natural reasoning steps, we obtain a more authentic view of how each prompting strategy (CoT, chain_of_draft, condensed reasoning) can balance verbosity with accuracy. 6.2 Arithmetic Performance and Natural Reasoning Our data shows that base_gpt4o achieves strong arithmetic performance (98%) despite not being optimized for multi-step mathematical reasoning. There is a suspicion that these base models might be “recalling” rather than reasoning for problems that are in public datasets like GSM8K, since a lot of the LLM models are trained on such datasets. 6.3 Token Effectiveness as a Practical Metric We introduce token effectiveness to measure the trade-off between accuracy and token usage. While high accuracy remains essential, generating three to five times more tokens can significantly increase inference costs. Token effectiveness is thus particularly relevant for large-scale applications, where each additional token has practical cost and latency implications. 6.4 Real-World Deployment Considerations Cost and Latency: In commercial settings (e.g., customer service bots), each call to an LLM may be billed by token usage. Strategies like CRP can reduce AI expenditure and improve throughput. User Experience: More concise answers can enhance user satisfaction by avoiding overly lengthy responses. Transparency vs. Brevity: Some applications (e.g., education) might still require more detailed solutions. CRP can be calibrated to balance these needs—offering concise but logically sound steps that remain interpretable. For example, the “2–4 words” can be made “4–6 words” and/or the target can be expanded to ≤ 40 or condensed to ≤ 20, depending on need for increased accuracies or increased time/cost efficiency. 7. Applications and Implications 7.1 Customer Service and Support Customer support systems benefit from concise, to-the-point resolutions. Token-efficient reasoning reduces response latency and operational costs. A condensed chain of reasoning ensures the system provides clear explanations without excessive verbosity, potentially improving user satisfaction. 7.2 Technical Troubleshooting Systems diagnosing technical issues can use a condensed approach to generate essential diagnostic steps. This helps maintain clarity, especially in conversational contexts where multiple follow-up questions may be required. 7.3 Educational Tutoring In educational settings, stepwise explanations are valuable for learning. However, excessively verbose explanations may confuse students. CRP can provide structured yet concise explanations, balancing thoroughness with clarity. 7.4 Business Intelligence and Research Organizations frequently query LLMs to analyze reports or summarize data. Token-efficient methods can reduce operational costs, particularly when queries occur at high volume. Maintaining accuracy while minimizing token usage ensures that insights are both correct and cost-effective. 7.5 Large-Scale AI Deployment When scaling LLMs to many users (e.g., in personal assistants or widespread chat services), even modest reductions in tokens per response yield significant aggregate savings in computational resources. CRP approaches help maintain system responsiveness and economic viability. 8. Conclusion Condensed Reasoning Prompting (CRP) offers a promising approach to preserving the accuracy benefits of a chain of thought reasoning without incurring its often substantial token costs. Our comprehensive evaluation of base_gpt4o, CoT_gpt4o, CoD_gpt4o, and crp_gpt4o across MMLU, Big Bench, arithmetic, and symbolic benchmarks demonstrate that condensed prompts can match or exceed standard CoT accuracy while using significantly fewer tokens. The token effectiveness metric underscores the importance of balancing performance with token usage, especially for real-world AI deployments where latency and cost are significant considerations. By conducting our tests in a zero-shot setting without instructing models to skip intermediate reasoning, we present a more realistic assessment of how LLMs behave under typical usage. While specialized prompts can yield excellent results on tasks like arithmetic, our findings show that a well-designed condensed reasoning approach can generalize effectively across various domains. Looking ahead, we envision further refinements that combine the strengths of multiple prompting strategies—such as integrating Least-to-Most or Self-Ask approaches with token-efficient instructions. Future work might also investigate adaptive prompting strategies that adjust reasoning verbosity based on task complexity, context length, or user preferences. As LLMs continue to evolve, condensed reasoning represents an important technique for delivering high-quality, interpretable, and cost-effective AI solutions at scale. Declarations Author Contribution G.M. - discovering the core prompting technique and running the evaluationsG.M.- Preparing the main paperD.K. - Reviewing the manuscript, creating the diagram and final data tables. Data Availability I have attached the eval jsons in related files. References J. Wei , X. Wang, D. Schuurmans, et al. , “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models,” in Advances in Neural Information Processing Systems (NeurIPS) , vol. 35, 2022, pp. 24824–24837 arxiv.org arxiv.org T. Kojima , S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa, “Large Language Models are Zero-Shot Reasoners,” in NeurIPS 2022 (Workshops), 2022 arxiv.org S. Xu , W. Xie, L. Zhao, and P. He, “Chain of Draft: Thinking Faster by Writing Less,” arXiv preprint arXiv:2502.18600, 2025 arxiv.org arxiv.org D. Zhou , N. Schärli, L. Hou, et al. , “Least-to-Most Prompting Enables Complex Reasoning in Large Language Models,” in Proc. of the 11th Int. Conf. on Learning Representations (ICLR) , 2023 arxiv.org O. Press , M. Zhang, S. Min, L. Schmidt, N. A. Smith, and M. Lewis, “Measuring and Narrowing the Compositionality Gap in Language Models,” in Findings of ACL: EMNLP 2023 , Dec. 2023 aclanthology.org S. Yao , J. Zhao, D. Yu, et al. , “ReAct: Synergizing Reasoning and Acting in Language Models,” in Proc. of ICLR 2023 , 2023 arxiv.org arxiv.org X. Wang , J. Wei, D. Schuurmans, et al. , “Self-Consistency Improves Chain-of-Thought Reasoning in Language Models,” in Proc. of ICLR 2023 , 2023 arxiv.org T. Han , Z. Wang, C. Fang, et al. , “Token-Budget-Aware LLM Reasoning,” arXiv:2412.18547, 2024 arxiv.org arxiv.org J. Cheng and B. Van Durme, “Compressed Chain-of-Thought: Efficient Reasoning Through Dense Representations,” arXiv:2412.13171, 2024 arxiv.org arxiv.org H. Xia , Y. Li, C. T. Leong, W. Wang, and W. Li, “TokenSkip: Controllable Chain-of-Thought Compression in LLMs,” arXiv:2502.12067, 2025 github.com Z. Yu , L. He, Z. Wu, X. Dai, and J. Chen, “Towards Better Chain-of-Thought Prompting Strategies: A Survey,” arXiv:2310.04959, 2023 arxiv.org Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {\"props\":{\"pageProps\":{\"initialData\":{\"identity\":\"rs-6170708\",\"acceptedTermsAndConditions\":true,\"allowDirectSubmit\":true,\"archivedVersions\":[],\"articleType\":\"Research Article\",\"associatedPublications\":[],\"authors\":[{\"id\":425083615,\"identity\":\"e4eccd78-efbf-4b93-8af9-686d0a2ce33d\",\"order_by\":0,\"name\":\"Gautam Mehra\",\"email\":\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA6UlEQVRIiWNgGAWjYFADZubGB0CKh48ItYwNEC2MzQYgLWzEa2FgbJMAUQS1yLufMX/wM4chmr+dsa3ya46dDBsD88NHN/BoMTyTY9jYu40hd8ZhxrbbstuSgQ5jMzbOwaelIcewgReopQGkRXIbM1ALD5s0Xi39bwwb/wK1zAdqKZbcVk9Yi7xEjmEzyJYNQC2MH7cdJqzFQOJZ4WzZbRK5Gw8zNkszbjvOw8ZMwC/y/ckbPr7dZpM77/zhgx9/bqu252dvfvgYry0HwBQ4RhiYecAkHuVgWxqQOIw/CKgeBaNgFIyCkQkAMppG9itqjNAAAAAASUVORK5CYII=\",\"orcid\":\"\",\"institution\":\"\",\"correspondingAuthor\":true,\"prefix\":\"\",\"firstName\":\"Gautam\",\"middleName\":\"\",\"lastName\":\"Mehra\",\"suffix\":\"\"},{\"id\":425083617,\"identity\":\"c69bd998-32b8-4968-bdeb-bec5ead9cf4b\",\"order_by\":1,\"name\":\"Danish Khan\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Danish\",\"middleName\":\"\",\"lastName\":\"Khan\",\"suffix\":\"\"}],\"badges\":[],\"createdAt\":\"2025-03-06 12:53:14\",\"currentVersionCode\":1,\"declarations\":\"\",\"doi\":\"10.21203/rs.3.rs-6170708/v1\",\"doiUrl\":\"https://doi.org/10.21203/rs.3.rs-6170708/v1\",\"draftVersion\":[],\"editorialEvents\":[],\"editorialNote\":\"\",\"failedWorkflow\":false,\"files\":[{\"id\":77971282,\"identity\":\"d0c87a5b-7c9b-4d63-8c36-08ac778afeec\",\"added_by\":\"auto\",\"created_at\":\"2025-03-07 10:55:36\",\"extension\":\"png\",\"order_by\":1,\"title\":\"Figure 1\",\"display\":\"\",\"copyAsset\":false,\"role\":\"figure\",\"size\":71573,\"visible\":true,\"origin\":\"\",\"legend\":\"\\u003cp\\u003eUnnumbered image in the Experimental Setup section.\\u003c/p\\u003e\",\"description\":\"\",\"filename\":\"floatimage1.png\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-6170708/v1/5e1ac6a605a800937e459727.png\"},{\"id\":78401674,\"identity\":\"1e4a62af-b9be-46c1-8bfb-f844f6da02c8\",\"added_by\":\"auto\",\"created_at\":\"2025-03-13 00:31:15\",\"extension\":\"pdf\",\"order_by\":0,\"title\":\"\",\"display\":\"\",\"copyAsset\":false,\"role\":\"manuscript-pdf\",\"size\":957106,\"visible\":true,\"origin\":\"\",\"legend\":\"\",\"description\":\"\",\"filename\":\"manuscript.pdf\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-6170708/v1/941cb7ef-4381-4903-b9d6-f10e2cc88561.pdf\"}],\"financialInterests\":\"No competing interests reported.\",\"formattedTitle\":\"Condensed Reasoning Prompting: Efficient Strategies, Evaluations, and Trade Offs in Large Language Model Reasoning\",\"fulltext\":[{\"header\":\"1. Introduction\",\"content\":\"\\u003cp\\u003eLarge language models (LLMs) have demonstrated remarkable capabilities in tasks ranging from question answering to code generation and creative writing. A key advancement in improving LLM performance on complex, multi-step tasks has been chain of thought (CoT) prompting, which encourages the model to articulate its intermediate reasoning steps rather than proceeding directly to an answer. Empirical evidence shows that CoT prompting enhances accuracy on arithmetic, logical reasoning, and multi-hop question answering by effectively providing the model with a \\u0026quot;scratchpad\\u0026quot; to decompose problems.\\u003c/p\\u003e\\n\\u003cp\\u003eHowever, this benefit often comes with increased verbosity. A fully articulated chain of thought can be quite extensive, particularly for tasks requiring multiple inferential steps. Verbosity increases computational overhead, leads to higher latency in interactive settings, and increases API costs when models are accessed via token-based billing. These issues become significant when deploying LLMs at scale.\\u003c/p\\u003e\\n\\u003cp\\u003eIn response, researchers have begun exploring methods to reduce the token footprint of chain of thought while preserving its accuracy advantages. Condensed Reasoning Prompting (CRP) is one such approach, directing the model to produce succinct intermediate reasoning. By eliminating redundant text and focusing on essential logic, CRP can maintain the performance gains of CoT while substantially reducing token usage.\\u003c/p\\u003e\\u003cp\\u003eIn this paper, we:\\u003c/p\\u003e\\n\\u003col start=\\\"1\\\" type=\\\"1\\\"\\u003e\\n \\u003cli\\u003eReview theoretical foundations of stepwise inference in LLMs and recent prompting strategies (e.g., Chain of Draft, Least to Most, Self-Ask).\\u003c/li\\u003e\\n \\u003cli\\u003ePresent a comprehensive evaluation of three prompting strategies: Chain Of Thought (CoT), Chain of Draft (CoD) and Condensed Reasoning across multiple benchmarks: MMLU, Big Bench, arithmetic reasoning, and symbolic reasoning.\\u003c/li\\u003e\\n \\u003cli\\u003eIntroduce a token effectiveness metric, defined as (accuracy \\u0026divide; average tokens per question), to quantify the efficiency\\u0026ndash;accuracy trade-off.\\u003c/li\\u003e\\n \\u003cli\\u003eHighlight that our experiments are conducted in a zero-shot manner without special system instructions to suppress reasoning for base models, thus reflecting a more natural usage scenario.\\u003c/li\\u003e\\n\\u003c/ol\\u003e\\n\\u003cp\\u003eOur findings indicate that condensed prompts can substantially reduce token usage while maintaining strong performance, underscoring the practical importance of CRP for real-world AI deployments.\\u003c/p\\u003e\"},{\"header\":\"2. Background and Related Work\",\"content\":\"\\u003cdiv id=\\\"Sec2\\\" class=\\\"Section2\\\"\\u003e \\u003ch2\\u003e2.1 Stepwise Reasoning in LLMs\\u003c/h2\\u003e \\u003cp\\u003eLLMs generate tokens autoregressively, with each token conditioned on the prompt and previously generated tokens. Chain of Thought (CoT) prompting encourages the model to write down each intermediate step explicitly, which often improves accuracy on tasks requiring multi-step reasoning. The theoretical rationale is that the model's \\\"internal scratchpad\\\" becomes externally visible, allowing it to refer to intermediate results and reducing the burden of maintaining everything in hidden states.\\u003c/p\\u003e \\u003c/div\\u003e \\u003cdiv id=\\\"Sec3\\\" class=\\\"Section2\\\"\\u003e \\u003ch2\\u003e2.2 Trade-Off: Accuracy vs. Verbosity\\u003c/h2\\u003e \\u003cp\\u003eWhile CoT prompts can significantly improve performance, they may be excessively verbose. This verbosity:\\u003c/p\\u003e \\u003cp\\u003e \\u003cul\\u003e \\u003cli\\u003e \\u003cp\\u003eIncreases Latency: More tokens to generate means longer inference time.\\u003c/p\\u003e \\u003c/li\\u003e \\u003cli\\u003e \\u003cp\\u003eRaises Costs: Many commercial APIs charge by token usage.\\u003c/p\\u003e \\u003c/li\\u003e \\u003cli\\u003e \\u003cp\\u003eChallenges Context Windows: Very long responses can approach context length limits in certain models.\\u003c/p\\u003e \\u003c/li\\u003e \\u003c/ul\\u003e \\u003c/p\\u003e \\u003c/div\\u003e \\u003cdiv id=\\\"Sec4\\\" class=\\\"Section2\\\"\\u003e \\u003ch2\\u003e2.3 Condensed Reasoning Prompting\\u003c/h2\\u003e \\u003cp\\u003eTo address verbosity, Condensed Reasoning Prompting (CRP) and similar methods (e.g., Chain of Draft) instruct the model to keep reasoning steps minimal. For instance, rather than restating the entire problem scenario, the model is guided to write only the key equations or short logic statements needed to reach the conclusion. Empirical results in prior work have shown that this can reduce token usage by over 70% compared to standard CoT, with minimal accuracy impact.\\u003c/p\\u003e \\u003cp\\u003e \\u003cb\\u003e2.4 Other Prompting Strategies\\u003c/b\\u003e \\u003c/p\\u003e \\u003cp\\u003e \\u003cul\\u003e \\u003cli\\u003e \\u003cp\\u003eLeast to Most Prompting decomposes a complex question into simpler sub-questions, solving them in ascending order of difficulty.\\u003c/p\\u003e \\u003c/li\\u003e \\u003cli\\u003e \\u003cp\\u003eSelf-Ask Prompting has the model generate its own clarifying sub-questions before arriving at the final answer.\\u003c/p\\u003e \\u003c/li\\u003e \\u003cli\\u003e \\u003cp\\u003eReAct interleaves reasoning with external actions (e.g., search queries) to ground the chain of thought in factual data.\\u003c/p\\u003e \\u003c/li\\u003e \\u003c/ul\\u003e \\u003c/p\\u003e \\u003c/div\\u003e\"},{\"header\":\"3. Theoretical Underpinnings of Efficient Reasoning\",\"content\":\"\\u003cdiv id=\\\"Sec6\\\" class=\\\"Section2\\\"\\u003e \\u003ch2\\u003e3.1 Cognitive Decomposition\\u003c/h2\\u003e \\u003cp\\u003eWhen a model breaks down a question into smaller, logically coherent steps, it reduces the likelihood of overlooking crucial details. Each step's output is fed back into the model's context, forming a transparent chain of sub-results. This approach is analogous to how humans solve problems in a methodical, stepwise fashion.\\u003c/p\\u003e \\u003c/div\\u003e \\u003cdiv id=\\\"Sec7\\\" class=\\\"Section2\\\"\\u003e \\u003ch2\\u003e3.2 Transparency and Error Diagnosis\\u003c/h2\\u003e \\u003cp\\u003eVerbose reasoning is often valued for being transparent, users can identify where the logic failed if the final answer is incorrect. However, condensed reasoning can maintain transparency if it concisely highlights the essential steps. In practice, a short, high-signal reasoning trace can be more interpretable than a lengthy explanation.\\u003c/p\\u003e \\u003c/div\\u003e \\u003cdiv id=\\\"Sec8\\\" class=\\\"Section2\\\"\\u003e \\u003ch2\\u003e3.3 Token Efficiency\\u003c/h2\\u003e \\u003cp\\u003eWe introduce a token effectiveness metric to capture how efficiently a model's reasoning yields correct answers. Specifically,\\u003c/p\\u003e \\u003cp\\u003eToken Effectiveness\\u0026thinsp;=\\u0026thinsp;Accuracy / Average Tokens per Question\\u003c/p\\u003e \\u003cp\\u003eHigh token effectiveness indicates that the model is using fewer tokens to achieve higher accuracy.\\u003c/p\\u003e \\u003c/div\\u003e\"},{\"header\":\"4. Experimental Setup\",\"content\":\"\\u003cp\\u003eWe evaluate four variants of a GPT-4\\u0026ndash;style model:\\u003c/p\\u003e\\n\\u003col start=\\\"1\\\" type=\\\"1\\\"\\u003e\\n \\u003cli\\u003ebase_gpt4o\\u0026nbsp;\\u003cul type=\\\"circle\\\"\\u003e\\n \\u003cli\\u003eZero-shot usage of our model without specialized instructions to skip intermediate reasoning.\\u003c/li\\u003e\\n \\u003cli\\u003eReflects the model\\u0026apos;s natural reasoning style.\\u003c/li\\u003e\\n \\u003c/ul\\u003e\\n \\u003c/li\\u003e\\n \\u003cli\\u003eCoT_gpt4o\\u0026nbsp;\\u003cul type=\\\"circle\\\"\\u003e\\n \\u003cli\\u003eA standard chain of thought approach prompting the model to articulate each step in detail.\\u003c/li\\u003e\\n \\u003c/ul\\u003e\\n \\u003c/li\\u003e\\n \\u003cli\\u003eCoD_gpt4o\\u0026nbsp;\\u003cul type=\\\"circle\\\"\\u003e\\n \\u003cli\\u003eA chain of draft prompting, instructing the model to produce minimal reasoning steps.\\u003c/li\\u003e\\n \\u003c/ul\\u003e\\n \\u003c/li\\u003e\\n \\u003cli\\u003ecrp_gpt4o\\u0026nbsp;\\u003cul type=\\\"circle\\\"\\u003e\\n \\u003cli\\u003eOur proposed CRP method, which explicitly guides the model to \\u0026ldquo;sketch\\u0026rdquo; the reasoning and keep it succinct while preserving essential logic.\\u003c/li\\u003e\\n \\u003c/ul\\u003e\\n \\u003c/li\\u003e\\n\\u003c/ol\\u003e\\n\\u003cp\\u003e\\u003cstrong\\u003e4.2 Zero-Shot Configuration\\u003c/strong\\u003e\\u003c/p\\u003e\\n\\u003cp\\u003eAll tests are zero-shot: we do not provide multiple examples or additional system messages to artificially constrain the model\\u0026apos;s output. Notably, we do not instruct the model to skip reasoning. This choice offers an unbiased measure of how each prompting method naturally balances accuracy and verbosity.\\u003c/p\\u003e\\n\\u003cp\\u003e\\u003cstrong\\u003e4.3 Datasets\\u003c/strong\\u003e\\u003c/p\\u003e\\n\\u003cp\\u003eWe evaluate on four core tasks:\\u003c/p\\u003e\\n\\u003col start=\\\"1\\\" type=\\\"1\\\"\\u003e\\n \\u003cli\\u003eMMLU (Massive Multitask Language Understanding): A challenging benchmark covering multiple academic subjects at varying difficulty levels.\\u003c/li\\u003e\\n \\u003cli\\u003eBig Bench: A collection of tasks spanning common sense, logic, and knowledge.\\u003c/li\\u003e\\n \\u003cli\\u003eGSM8k: Multi-step math word problems requiring explicit numeric manipulation.\\u003c/li\\u003e\\n \\u003cli\\u003eSymbolic Reasoning: Classic coin flip problems.\\u003c/li\\u003e\\n\\u003c/ol\\u003e\\n\\u003cp\\u003eFor each dataset, we measure:\\u003c/p\\u003e\\n\\u003cul type=\\\"disc\\\"\\u003e\\n \\u003cli\\u003eAccuracy: Percentage of questions answered correctly.\\u003c/li\\u003e\\n \\u003cli\\u003eAverage Tokens per Question: The mean number of tokens generated by the model in its response.\\u003c/li\\u003e\\n \\u003cli\\u003eToken Effectiveness: Accuracy \\u0026divide; Average Tokens per Question.\\u003c/li\\u003e\\n\\u003c/ul\\u003e\"},{\"header\":\"5. Results and Analysis\",\"content\":\"\\u003cp\\u003eThis section presents our main findings. We first provide a summary table comparing MMLU, Big Bench, Arithmetic Reasoning, and Symbolic Reasoning performance for each model, followed by detailed sub-sections for each benchmark.\\u003c/p\\u003e \\u003cdiv id=\\\"Sec15\\\" class=\\\"Section2\\\"\\u003e \\u003ch2\\u003e5.1 Summary of Key Benchmarks\\u003c/h2\\u003e \\u003cp\\u003e \\u003cdiv class=\\\"gridtable\\\"\\u003e\\u003ctable float=\\\"No\\\" id=\\\"Taba\\\" border=\\\"1\\\"\\u003e \\u003ccolgroup cols=\\\"6\\\"\\u003e \\u003cdiv align=\\\"left\\\" class=\\\"colspec\\\" colname=\\\"c1\\\" colnum=\\\"1\\\"\\u003e\\u003c/div\\u003e \\u003cdiv align=\\\"char\\\" char=\\\".\\\" class=\\\"colspec\\\" colname=\\\"c2\\\" colnum=\\\"2\\\"\\u003e\\u003c/div\\u003e \\u003cdiv align=\\\"char\\\" char=\\\".\\\" class=\\\"colspec\\\" colname=\\\"c3\\\" colnum=\\\"3\\\"\\u003e\\u003c/div\\u003e \\u003cdiv align=\\\"char\\\" char=\\\".\\\" class=\\\"colspec\\\" colname=\\\"c4\\\" colnum=\\\"4\\\"\\u003e\\u003c/div\\u003e \\u003cdiv align=\\\"char\\\" char=\\\".\\\" class=\\\"colspec\\\" colname=\\\"c5\\\" colnum=\\\"5\\\"\\u003e\\u003c/div\\u003e \\u003cdiv align=\\\"char\\\" char=\\\".\\\" class=\\\"colspec\\\" colname=\\\"c6\\\" colnum=\\\"6\\\"\\u003e\\u003c/div\\u003e \\u003cthead\\u003e \\u003ctr\\u003e \\u003cth align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003eModel\\u003c/p\\u003e \\u003c/th\\u003e \\u003cth align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003eMMLU\\u003c/p\\u003e \\u003c/th\\u003e \\u003cth align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eBig Bench\\u003c/p\\u003e \\u003c/th\\u003e \\u003cth align=\\\"left\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003eArithmetic\\u003c/p\\u003e \\u003c/th\\u003e \\u003cth align=\\\"left\\\" colname=\\\"c5\\\"\\u003e \\u003cp\\u003eSymbolic\\u003c/p\\u003e \\u003c/th\\u003e \\u003cth align=\\\"left\\\" colname=\\\"c6\\\"\\u003e \\u003cp\\u003eToken Effectiveness\\u003c/p\\u003e \\u003c/th\\u003e \\u003c/tr\\u003e \\u003c/thead\\u003e \\u003ctbody\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003eCoD_gpt4o\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e0.72\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003e0.95\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003e0.94\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c5\\\"\\u003e \\u003cp\\u003e0.68\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c6\\\"\\u003e \\u003cp\\u003e0.47\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003eCoT_gpt4o\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e0.79\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003e0.95\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003e0.96\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c5\\\"\\u003e \\u003cp\\u003e0.79\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c6\\\"\\u003e \\u003cp\\u003e0.24\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003ebase_gpt4o\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e0.66\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003e0.95\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003e0.98\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c5\\\"\\u003e \\u003cp\\u003e0.72\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c6\\\"\\u003e \\u003cp\\u003e0.48\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003ecrp_gpt4o\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e0.75\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003e0.99\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003e0.93\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c5\\\"\\u003e \\u003cp\\u003e0.74\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c6\\\"\\u003e \\u003cp\\u003e0.53\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003c/tbody\\u003e \\u003c/colgroup\\u003e \\u003c/table\\u003e\\u003c/div\\u003e \\u003c/p\\u003e \\u003cp\\u003eFrom this high-level comparison, we observe:\\u003c/p\\u003e \\u003cp\\u003e \\u003cul\\u003e \\u003cli\\u003e \\u003cp\\u003eCoT_gpt4o often performs well in certain reasoning tasks (e.g., MMLU at 79%, Symbolic at 79%) but at the cost of lower \\\"Token Effectiveness\\\" (0.24).\\u003c/p\\u003e \\u003c/li\\u003e \\u003cli\\u003e \\u003cp\\u003eCoD_gpt4o typically achieves a good balance between accuracy and token usage (Token Effectiveness of 0.47).\\u003c/p\\u003e \\u003c/li\\u003e \\u003cli\\u003e \\u003cp\\u003ebase_gpt4o achieves a high score of 0.98 on arithmetic tasks but shows lower performance on MMLU (0.66).\\u003c/p\\u003e \\u003c/li\\u003e \\u003cli\\u003e \\u003cp\\u003ecrp_gpt4o demonstrates consistent performance across datasets, with particularly strong results on Big Bench (0.99) and the highest \\\"Token Effectiveness\\\" measure (0.53) among the approaches.\\u003c/p\\u003e \\u003c/li\\u003e \\u003c/ul\\u003e \\u003c/p\\u003e \\u003cp\\u003eBelow, we provide more detailed metrics for each dataset: mean accuracy, average tokens per question, and token effectiveness.\\u003c/p\\u003e \\u003cp\\u003e \\u003cb\\u003e5.2 MMLU Results\\u003c/b\\u003e \\u003c/p\\u003e \\u003cp\\u003e \\u003cdiv class=\\\"gridtable\\\"\\u003e\\u003ctable float=\\\"No\\\" id=\\\"Tabb\\\" border=\\\"1\\\"\\u003e \\u003ccolgroup cols=\\\"4\\\"\\u003e \\u003cdiv align=\\\"left\\\" class=\\\"colspec\\\" colname=\\\"c1\\\" colnum=\\\"1\\\"\\u003e\\u003c/div\\u003e \\u003cdiv align=\\\"char\\\" char=\\\".\\\" class=\\\"colspec\\\" colname=\\\"c2\\\" colnum=\\\"2\\\"\\u003e\\u003c/div\\u003e \\u003cdiv align=\\\"char\\\" char=\\\".\\\" class=\\\"colspec\\\" colname=\\\"c3\\\" colnum=\\\"3\\\"\\u003e\\u003c/div\\u003e \\u003cdiv align=\\\"char\\\" char=\\\".\\\" class=\\\"colspec\\\" colname=\\\"c4\\\" colnum=\\\"4\\\"\\u003e\\u003c/div\\u003e \\u003cthead\\u003e \\u003ctr\\u003e \\u003cth align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003eModel\\u003c/p\\u003e \\u003c/th\\u003e \\u003cth align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003eMean Accuracy\\u003c/p\\u003e \\u003c/th\\u003e \\u003cth align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eAvg Tokens/Qn\\u003c/p\\u003e \\u003c/th\\u003e \\u003cth align=\\\"left\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003eToken Effectiveness\\u003c/p\\u003e \\u003c/th\\u003e \\u003c/tr\\u003e \\u003c/thead\\u003e \\u003ctbody\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003ebase_gpt4o\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e65.5%\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003e194.8\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003e0.336\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003eCoT_gpt4o\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e78.6%\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003e517.9\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003e0.151\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003eCoD_gpt4o\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e72.4%\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003e282.7\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003e0.256\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003ecrp_gpt4o\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e75.2%\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003e242.8\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003e0.310\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003c/tbody\\u003e \\u003c/colgroup\\u003e \\u003c/table\\u003e\\u003c/div\\u003e \\u003c/p\\u003e \\u003cp\\u003e \\u003cul\\u003e \\u003cli\\u003e \\u003cp\\u003eAccuracy: CoT_gpt4o leads with 78.6% on MMLU, surpassing crp_gpt4o (75.2%) by 3.4 percentage points.\\u003c/p\\u003e \\u003c/li\\u003e \\u003cli\\u003e \\u003cp\\u003eToken Usage: CoT_gpt4o is the most verbose (517.9 tokens/question), while base_gpt4o uses the fewest tokens (194.8) but has lower accuracy (65.5%).\\u003c/p\\u003e \\u003c/li\\u003e \\u003cli\\u003e \\u003cp\\u003eToken Effectiveness: crp_gpt4o (0.31) approaches base_gpt4o (0.34) while achieving notably higher accuracy (75.2% vs. 65.5%). This demonstrates the efficiency of condensed prompting for MMLU tasks.\\u003c/p\\u003e \\u003c/li\\u003e \\u003c/ul\\u003e \\u003c/p\\u003e \\u003cp\\u003e \\u003cb\\u003eAnalysis\\u003c/b\\u003e \\u003c/p\\u003e \\u003cp\\u003eMMLU tasks span various academic subjects. The results suggest that detailed reasoning (as in CoT_gpt4o) helps achieve higher accuracy but requires significantly more tokens. CRP is more efficient, maintaining competitive performance while generating fewer tokens.\\u003c/p\\u003e \\u003cp\\u003e \\u003cb\\u003e5.3 Big Bench Results\\u003c/b\\u003e \\u003c/p\\u003e \\u003cp\\u003e \\u003cdiv class=\\\"gridtable\\\"\\u003e\\u003ctable float=\\\"No\\\" id=\\\"Tabc\\\" border=\\\"1\\\"\\u003e \\u003ccolgroup cols=\\\"4\\\"\\u003e \\u003cdiv align=\\\"left\\\" class=\\\"colspec\\\" colname=\\\"c1\\\" colnum=\\\"1\\\"\\u003e\\u003c/div\\u003e \\u003cdiv align=\\\"char\\\" char=\\\".\\\" class=\\\"colspec\\\" colname=\\\"c2\\\" colnum=\\\"2\\\"\\u003e\\u003c/div\\u003e \\u003cdiv align=\\\"char\\\" char=\\\".\\\" class=\\\"colspec\\\" colname=\\\"c3\\\" colnum=\\\"3\\\"\\u003e\\u003c/div\\u003e \\u003cdiv align=\\\"char\\\" char=\\\".\\\" class=\\\"colspec\\\" colname=\\\"c4\\\" colnum=\\\"4\\\"\\u003e\\u003c/div\\u003e \\u003cthead\\u003e \\u003ctr\\u003e \\u003cth align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003eModel\\u003c/p\\u003e \\u003c/th\\u003e \\u003cth align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003eMean Accuracy\\u003c/p\\u003e \\u003c/th\\u003e \\u003cth align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eAvg Tokens/Qn\\u003c/p\\u003e \\u003c/th\\u003e \\u003cth align=\\\"left\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003eToken Effectiveness\\u003c/p\\u003e \\u003c/th\\u003e \\u003c/tr\\u003e \\u003c/thead\\u003e \\u003ctbody\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003ebase_gpt4o\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e95.2%\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003e134.5\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003e0.7071\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003eCoT_gpt4o\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e94.8%\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003e355.5\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003e0.267\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003eCoD_gpt4o\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e95.2%\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003e103.0\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003e0.923\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003ecrp_gpt4o\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e98.6%\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003e111.8\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003e0.882\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003c/tbody\\u003e \\u003c/colgroup\\u003e \\u003c/table\\u003e\\u003c/div\\u003e \\u003c/p\\u003e \\u003cp\\u003e \\u003cul\\u003e \\u003cli\\u003e \\u003cp\\u003eAccuracy: crp_gpt4o achieves the highest accuracy (98.6%), followed by base_gpt4o and CoD_gpt4o (both at 95.2%).\\u003c/p\\u003e \\u003c/li\\u003e \\u003cli\\u003e \\u003cp\\u003eToken Usage: CoD_gpt4o is the most efficient in raw token count (103.07 tokens/question), while CoT_gpt4o is the most verbose (355.55).\\u003c/p\\u003e \\u003c/li\\u003e \\u003cli\\u003e \\u003cp\\u003eToken Effectiveness: CoD_gpt4o scores the highest (0.92), with crp_gpt4o close behind (0.88).\\u003c/p\\u003e \\u003c/li\\u003e \\u003c/ul\\u003e \\u003c/p\\u003e \\u003cp\\u003e \\u003cb\\u003eAnalysis\\u003c/b\\u003e \\u003c/p\\u003e \\u003cp\\u003eBig Bench includes various tasks, from basic knowledge to more logic-intensive questions. The results indicate that condensed prompts can lead to both high accuracy and high token effectiveness. CoT_gpt4o's detailed reasoning does not translate into superior accuracy in this context, suggesting that longer reasoning traces are not always beneficial.\\u003c/p\\u003e \\u003cp\\u003e \\u003cb\\u003e5.4 (GSM8K) Arithmetic Reasoning Results\\u003c/b\\u003e \\u003c/p\\u003e \\u003cp\\u003e \\u003cdiv class=\\\"gridtable\\\"\\u003e\\u003ctable float=\\\"No\\\" id=\\\"Tabd\\\" border=\\\"1\\\"\\u003e \\u003ccolgroup cols=\\\"4\\\"\\u003e \\u003cdiv align=\\\"left\\\" class=\\\"colspec\\\" colname=\\\"c1\\\" colnum=\\\"1\\\"\\u003e\\u003c/div\\u003e \\u003cdiv align=\\\"left\\\" class=\\\"colspec\\\" colname=\\\"c2\\\" colnum=\\\"2\\\"\\u003e\\u003c/div\\u003e \\u003cdiv align=\\\"char\\\" char=\\\".\\\" class=\\\"colspec\\\" colname=\\\"c3\\\" colnum=\\\"3\\\"\\u003e\\u003c/div\\u003e \\u003cdiv align=\\\"char\\\" char=\\\".\\\" class=\\\"colspec\\\" colname=\\\"c4\\\" colnum=\\\"4\\\"\\u003e\\u003c/div\\u003e \\u003cthead\\u003e \\u003ctr\\u003e \\u003cth align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003eModel\\u003c/p\\u003e \\u003c/th\\u003e \\u003cth align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003eMean Accuracy\\u003c/p\\u003e \\u003c/th\\u003e \\u003cth align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eAvg Tokens/Qn\\u003c/p\\u003e \\u003c/th\\u003e \\u003cth align=\\\"left\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003eToken Effectiveness\\u003c/p\\u003e \\u003c/th\\u003e \\u003c/tr\\u003e \\u003c/thead\\u003e \\u003ctbody\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003ebase_gpt4o\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e98%\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003e252.8\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003e0.389\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003eCoT_gpt4o\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e96%\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003e356.1\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003e0.270\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003eCoD_gpt4o\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e94%\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003e178.7\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003e0.527\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003ecrp_gpt4o\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e94%\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003e167.0\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003e0.562\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003c/tbody\\u003e \\u003c/colgroup\\u003e \\u003c/table\\u003e\\u003c/div\\u003e \\u003c/p\\u003e \\u003cp\\u003e \\u003cul\\u003e \\u003cli\\u003e \\u003cp\\u003eAccuracy: base_gpt4o attains the highest accuracy (98%), despite not being specifically prompted for multi-step mathematical reasoning.\\u003c/p\\u003e \\u003c/li\\u003e \\u003cli\\u003e \\u003cp\\u003eToken Usage: CoT_gpt4o is the most verbose (356.07 tokens/question). CoD_gpt4o and crp_gpt4o both use fewer than 180 tokens/question.\\u003c/p\\u003e \\u003c/li\\u003e \\u003cli\\u003e \\u003cp\\u003eToken Effectiveness: crp_gpt4o leads with 0.56, indicating that it delivers strong accuracy with fewer tokens. CoD_gpt4o follows closely (0.53).\\u003c/p\\u003e \\u003c/li\\u003e \\u003c/ul\\u003e \\u003c/p\\u003e \\u003cp\\u003e \\u003cb\\u003eAnalysis\\u003c/b\\u003e \\u003c/p\\u003e \\u003cp\\u003eThese results highlight that zero-shot base_gpt4o can perform well in arithmetic tasks, potentially due to robust underlying numerical capabilities. However, from an efficiency perspective, condensed prompts remain advantageous because they achieve comparable accuracy (93\\u0026ndash;94%) with significantly fewer tokens.\\u003c/p\\u003e \\u003cp\\u003e \\u003cb\\u003e5.5 Symbolic Reasoning Results\\u003c/b\\u003e \\u003c/p\\u003e \\u003cp\\u003e \\u003cdiv class=\\\"gridtable\\\"\\u003e\\u003ctable float=\\\"No\\\" id=\\\"Tabe\\\" border=\\\"1\\\"\\u003e \\u003ccolgroup cols=\\\"4\\\"\\u003e \\u003cdiv align=\\\"left\\\" class=\\\"colspec\\\" colname=\\\"c1\\\" colnum=\\\"1\\\"\\u003e\\u003c/div\\u003e \\u003cdiv align=\\\"left\\\" class=\\\"colspec\\\" colname=\\\"c2\\\" colnum=\\\"2\\\"\\u003e\\u003c/div\\u003e \\u003cdiv align=\\\"char\\\" char=\\\".\\\" class=\\\"colspec\\\" colname=\\\"c3\\\" colnum=\\\"3\\\"\\u003e\\u003c/div\\u003e \\u003cdiv align=\\\"char\\\" char=\\\".\\\" class=\\\"colspec\\\" colname=\\\"c4\\\" colnum=\\\"4\\\"\\u003e\\u003c/div\\u003e \\u003cthead\\u003e \\u003ctr\\u003e \\u003cth align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003eModel\\u003c/p\\u003e \\u003c/th\\u003e \\u003cth align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003eMean Accuracy\\u003c/p\\u003e \\u003c/th\\u003e \\u003cth align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eAvg Tokens/Qn\\u003c/p\\u003e \\u003c/th\\u003e \\u003cth align=\\\"left\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003eToken Effectiveness\\u003c/p\\u003e \\u003c/th\\u003e \\u003c/tr\\u003e \\u003c/thead\\u003e \\u003ctbody\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003ebase_gpt4o\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e72%\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003e121.862069\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003e0.594\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003eCoT_gpt4o\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e79%\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003e247.9655172\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003e0.320\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003eCoD_gpt4o\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e68%\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003e122.6896552\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003e0.556\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003ecrp_gpt4o\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e74%\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003e126.5172414\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003e0.586\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003c/tbody\\u003e \\u003c/colgroup\\u003e \\u003c/table\\u003e\\u003c/div\\u003e \\u003c/p\\u003e \\u003cp\\u003e \\u003cul\\u003e \\u003cli\\u003e \\u003cp\\u003eAccuracy: CoT_gpt4o achieves the highest accuracy (79%) but uses approximately twice the tokens compared to base_gpt4o or CoD_gpt4o.\\u003c/p\\u003e \\u003c/li\\u003e \\u003cli\\u003e \\u003cp\\u003eToken Usage: base_gpt4o and CoD_gpt4o both use approximately 122 tokens/question, while crp_gpt4o is slightly higher (126.52).\\u003c/p\\u003e \\u003c/li\\u003e \\u003cli\\u003e \\u003cp\\u003eToken Effectiveness: base_gpt4o and crp_gpt4o both achieve around 0.59, with CoD_gpt4o at 0.56. CoT_gpt4o lags at 0.32 due to its higher token count.\\u003c/p\\u003e \\u003c/li\\u003e \\u003c/ul\\u003e \\u003c/p\\u003e \\u003cp\\u003e \\u003cb\\u003eAnalysis\\u003c/b\\u003e \\u003c/p\\u003e \\u003cp\\u003eSymbolic reasoning often benefits from detailed stepwise thinking. However, the data suggests that longer explanations (CoT_gpt4o) are not always the most efficient approach. Condensed reasoning provides a balanced method, offering strong accuracy with relatively minimal token requirements.\\u003c/p\\u003e \\u003c/div\\u003e \\u003cdiv id=\\\"Sec16\\\" class=\\\"Section2\\\"\\u003e \\u003ch2\\u003e5.6 Overall Observations\\u003c/h2\\u003e \\u003cp\\u003e \\u003col\\u003e \\u003cspan\\u003e \\u003cli\\u003e \\u003cp\\u003eCoT_gpt4o typically achieves high accuracy but is notably verbose, resulting in lower token effectiveness scores.\\u003c/p\\u003e \\u003c/li\\u003e \\u003c/span\\u003e \\u003cspan\\u003e \\u003cli\\u003e \\u003cp\\u003eCoD_gpt4o excels in token minimization, it\\u0026rsquo;s often at the cost of accuracy.\\u003c/p\\u003e \\u003c/li\\u003e \\u003c/span\\u003e \\u003cspan\\u003e \\u003cli\\u003e \\u003cp\\u003ecrp_gpt4o offers a good compromise, frequently ranking near the top in both accuracy and token effectiveness.\\u003c/p\\u003e \\u003c/li\\u003e \\u003c/span\\u003e \\u003c/ol\\u003e \\u003c/p\\u003e \\u003cp\\u003eCollectively, these results indicate that while CoT can be effective, there is significant value in guiding models to be concise. This not only reduces computational costs but can also preserve\\u0026mdash;and sometimes enhance\\u0026mdash;accuracy by focusing the model's attention on the essential steps.\\u003c/p\\u003e \\u003c/div\\u003e\"},{\"header\":\"6. Discussion\",\"content\":\"\\u003cdiv id=\\\"Sec18\\\" class=\\\"Section2\\\"\\u003e \\u003ch2\\u003e6.1 Zero-Shot vs. Special Instructions\\u003c/h2\\u003e \\u003cp\\u003eA notable contribution of this study is our decision to use base_gpt4o in a zero-shot manner, without instructing it to skip intermediate reasoning. In many prior works, prompts are carefully crafted to minimize reasoning text, which we have observed can affect accuracy. By allowing base_gpt4o to produce its natural reasoning steps, we obtain a more authentic view of how each prompting strategy (CoT, chain_of_draft, condensed reasoning) can balance verbosity with accuracy.\\u003c/p\\u003e \\u003c/div\\u003e \\u003cdiv id=\\\"Sec19\\\" class=\\\"Section2\\\"\\u003e \\u003ch2\\u003e6.2 Arithmetic Performance and Natural Reasoning\\u003c/h2\\u003e \\u003cp\\u003eOur data shows that base_gpt4o achieves strong arithmetic performance (98%) despite not being optimized for multi-step mathematical reasoning. There is a suspicion that these base models might be “recalling” rather than reasoning for problems that are in public datasets like GSM8K, since a lot of the LLM models are trained on such datasets.\\u003c/p\\u003e \\u003c/div\\u003e \\u003cdiv id=\\\"Sec20\\\" class=\\\"Section2\\\"\\u003e \\u003ch2\\u003e6.3 Token Effectiveness as a Practical Metric\\u003c/h2\\u003e \\u003cp\\u003eWe introduce token effectiveness to measure the trade-off between accuracy and token usage. While high accuracy remains essential, generating three to five times more tokens can significantly increase inference costs. Token effectiveness is thus particularly relevant for large-scale applications, where each additional token has practical cost and latency implications.\\u003c/p\\u003e \\u003c/div\\u003e \\u003cdiv id=\\\"Sec21\\\" class=\\\"Section2\\\"\\u003e \\u003ch2\\u003e6.4 Real-World Deployment Considerations\\u003c/h2\\u003e \\u003cp\\u003e \\u003c/p\\u003e\\u003col\\u003e \\u003cspan\\u003e \\u003cli\\u003e \\u003cp\\u003eCost and Latency: In commercial settings (e.g., customer service bots), each call to an LLM may be billed by token usage. Strategies like CRP can reduce AI expenditure and improve throughput.\\u003c/p\\u003e \\u003c/li\\u003e \\u003c/span\\u003e \\u003cspan\\u003e \\u003cli\\u003e \\u003cp\\u003eUser Experience: More concise answers can enhance user satisfaction by avoiding overly lengthy responses.\\u003c/p\\u003e \\u003c/li\\u003e \\u003c/span\\u003e \\u003cspan\\u003e \\u003cli\\u003e \\u003cp\\u003eTransparency vs. Brevity: Some applications (e.g., education) might still require more detailed solutions. CRP can be calibrated to balance these needs—offering concise but logically sound steps that remain interpretable. For example, the “2–4 words” can be made “4–6 words” and/or the target can be expanded to ≤ 40 or condensed to ≤ 20, depending on need for increased accuracies or increased time/cost efficiency.\\u003c/p\\u003e \\u003c/li\\u003e \\u003c/span\\u003e \\u003c/ol\\u003e \\u003cp\\u003e\\u003c/p\\u003e \\u003c/div\\u003e \"},{\"header\":\"7. Applications and Implications\",\"content\":\"\\u003ch2\\u003e7.1 Customer Service and Support\\u003c/h2\\u003e\\u003cp\\u003eCustomer support systems benefit from concise, to-the-point resolutions. Token-efficient reasoning reduces response latency and operational costs. A condensed chain of reasoning ensures the system provides clear explanations without excessive verbosity, potentially improving user satisfaction.\\u003c/p\\u003e\\u003ch2\\u003e7.2 Technical Troubleshooting\\u003c/h2\\u003e\\u003cp\\u003eSystems diagnosing technical issues can use a condensed approach to generate essential diagnostic steps. This helps maintain clarity, especially in conversational contexts where multiple follow-up questions may be required.\\u003c/p\\u003e\\u003ch2\\u003e7.3 Educational Tutoring\\u003c/h2\\u003e\\u003cp\\u003eIn educational settings, stepwise explanations are valuable for learning. However, excessively verbose explanations may confuse students. CRP can provide structured yet concise explanations, balancing thoroughness with clarity.\\u003c/p\\u003e\\u003ch2\\u003e7.4 Business Intelligence and Research\\u003c/h2\\u003e\\u003cp\\u003eOrganizations frequently query LLMs to analyze reports or summarize data. Token-efficient methods can reduce operational costs, particularly when queries occur at high volume. Maintaining accuracy while minimizing token usage ensures that insights are both correct and cost-effective.\\u003c/p\\u003e\\u003ch2\\u003e7.5 Large-Scale AI Deployment\\u003c/h2\\u003e\\u003cp\\u003eWhen scaling LLMs to many users (e.g., in personal assistants or widespread chat services), even modest reductions in tokens per response yield significant aggregate savings in computational resources. CRP approaches help maintain system responsiveness and economic viability.\\u003c/p\\u003e\"},{\"header\":\"8. Conclusion\",\"content\":\"\\u003cp\\u003eCondensed Reasoning Prompting (CRP) offers a promising approach to preserving the accuracy benefits of a chain of thought reasoning without incurring its often substantial token costs. Our comprehensive evaluation of base_gpt4o, CoT_gpt4o, CoD_gpt4o, and crp_gpt4o across MMLU, Big Bench, arithmetic, and symbolic benchmarks demonstrate that condensed prompts can match or exceed standard CoT accuracy while using significantly fewer tokens. The token effectiveness metric underscores the importance of balancing performance with token usage, especially for real-world AI deployments where latency and cost are significant considerations.\\u003c/p\\u003e \\u003cp\\u003eBy conducting our tests in a zero-shot setting without instructing models to skip intermediate reasoning, we present a more realistic assessment of how LLMs behave under typical usage. While specialized prompts can yield excellent results on tasks like arithmetic, our findings show that a well-designed condensed reasoning approach can generalize effectively across various domains.\\u003c/p\\u003e \\u003cp\\u003eLooking ahead, we envision further refinements that combine the strengths of multiple prompting strategies\\u0026mdash;such as integrating Least-to-Most or Self-Ask approaches with token-efficient instructions. Future work might also investigate adaptive prompting strategies that adjust reasoning verbosity based on task complexity, context length, or user preferences. As LLMs continue to evolve, condensed reasoning represents an important technique for delivering high-quality, interpretable, and cost-effective AI solutions at scale.\\u003c/p\\u003e\"},{\"header\":\"Declarations\",\"content\":\"\\u003ch2\\u003eAuthor Contribution\\u003c/h2\\u003e\\u003cp\\u003eG.M. - discovering the core prompting technique and running the evaluationsG.M.- Preparing the main paperD.K. - Reviewing the manuscript, creating the diagram and final data tables.\\u003c/p\\u003e\\u003ch2\\u003eData Availability\\u003c/h2\\u003e\\u003cp\\u003eI have attached the eval jsons in related files.\\u003c/p\\u003e\"},{\"header\":\"References\",\"content\":\"\\u003col\\u003e\\n\\u003cli\\u003e\\u003cstrong\\u003eJ. Wei\\u003c/strong\\u003e, X. Wang, D. Schuurmans, \\u003cem\\u003eet al.\\u003c/em\\u003e, \\u0026ldquo;Chain-of-Thought Prompting Elicits Reasoning in Large Language Models,\\u0026rdquo; in \\u003cem\\u003eAdvances in Neural Information Processing Systems (NeurIPS)\\u003c/em\\u003e, vol. 35, 2022, pp. 24824\\u0026ndash;24837 arxiv.org arxiv.org\\u003c/li\\u003e\\n\\u003cli\\u003e\\u003cstrong\\u003eT. Kojima\\u003c/strong\\u003e, S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa, \\u0026ldquo;Large Language Models are Zero-Shot Reasoners,\\u0026rdquo; in \\u003cem\\u003eNeurIPS 2022\\u003c/em\\u003e (Workshops), 2022 arxiv.org\\u003c/li\\u003e\\n\\u003cli\\u003e\\u003cstrong\\u003eS. Xu\\u003c/strong\\u003e, W. Xie, L. Zhao, and P. He, \\u0026ldquo;Chain of Draft: Thinking Faster by Writing Less,\\u0026rdquo; arXiv preprint arXiv:2502.18600, 2025 arxiv.org arxiv.org\\u003c/li\\u003e\\n\\u003cli\\u003e\\u003cstrong\\u003eD. Zhou\\u003c/strong\\u003e, N. Sch\\u0026auml;rli, L. Hou, \\u003cem\\u003eet al.\\u003c/em\\u003e, \\u0026ldquo;Least-to-Most Prompting Enables Complex Reasoning in Large Language Models,\\u0026rdquo; in \\u003cem\\u003eProc. of the 11th Int. Conf. on Learning Representations (ICLR)\\u003c/em\\u003e, 2023 arxiv.org\\u003c/li\\u003e\\n\\u003cli\\u003e\\u003cstrong\\u003eO. Press\\u003c/strong\\u003e, M. Zhang, S. Min, L. Schmidt, N. A. Smith, and M. Lewis, \\u0026ldquo;Measuring and Narrowing the Compositionality Gap in Language Models,\\u0026rdquo; in \\u003cem\\u003eFindings of ACL: EMNLP 2023\\u003c/em\\u003e, Dec. 2023 aclanthology.org\\u003c/li\\u003e\\n\\u003cli\\u003e\\u003cstrong\\u003eS. Yao\\u003c/strong\\u003e, J. Zhao, D. Yu, \\u003cem\\u003eet al.\\u003c/em\\u003e, \\u0026ldquo;ReAct: Synergizing Reasoning and Acting in Language Models,\\u0026rdquo; in \\u003cem\\u003eProc. of ICLR 2023\\u003c/em\\u003e, 2023 arxiv.org arxiv.org\\u003c/li\\u003e\\n\\u003cli\\u003e\\u003cstrong\\u003eX. Wang\\u003c/strong\\u003e, J. Wei, D. Schuurmans, \\u003cem\\u003eet al.\\u003c/em\\u003e, \\u0026ldquo;Self-Consistency Improves Chain-of-Thought Reasoning in Language Models,\\u0026rdquo; in \\u003cem\\u003eProc. of ICLR 2023\\u003c/em\\u003e, 2023 arxiv.org\\u003c/li\\u003e\\n\\u003cli\\u003e\\u003cstrong\\u003eT. Han\\u003c/strong\\u003e, Z. Wang, C. Fang, \\u003cem\\u003eet al.\\u003c/em\\u003e, \\u0026ldquo;Token-Budget-Aware LLM Reasoning,\\u0026rdquo; arXiv:2412.18547, 2024 arxiv.org arxiv.org\\u003c/li\\u003e\\n\\u003cli\\u003e\\u003cstrong\\u003eJ. Cheng\\u003c/strong\\u003e and B. Van Durme, \\u0026ldquo;Compressed Chain-of-Thought: Efficient Reasoning Through Dense Representations,\\u0026rdquo; arXiv:2412.13171, 2024 arxiv.org arxiv.org\\u003c/li\\u003e\\n\\u003cli\\u003e\\u003cstrong\\u003eH. Xia\\u003c/strong\\u003e, Y. Li, C. T. Leong, W. Wang, and W. Li, \\u0026ldquo;TokenSkip: Controllable Chain-of-Thought Compression in LLMs,\\u0026rdquo; arXiv:2502.12067, 2025 github.com\\u003c/li\\u003e\\n\\u003cli\\u003e\\u003cstrong\\u003eZ. Yu\\u003c/strong\\u003e, L. He, Z. Wu, X. Dai, and J. Chen, \\u0026ldquo;Towards Better Chain-of-Thought Prompting Strategies: A Survey,\\u0026rdquo; arXiv:2310.04959, 2023 arxiv.org\\u003c/li\\u003e\\n\\u003c/ol\\u003e\"}],\"fulltextSource\":\"\",\"fullText\":\"\",\"funders\":[],\"hasAdminPriorityOnWorkflow\":false,\"hasManuscriptDocX\":true,\"hasOptedInToPreprint\":true,\"hasPassedJournalQc\":\"\",\"hasAnyPriority\":true,\"hideJournal\":true,\"highlight\":\"\",\"institution\":\"\",\"isAcceptedByJournal\":false,\"isAuthorSuppliedPdf\":false,\"isDeskRejected\":\"\",\"isHiddenFromSearch\":false,\"isInQc\":false,\"isInWorkflow\":false,\"isPdf\":false,\"isPdfUpToDate\":true,\"isWithdrawnOrRetracted\":false,\"journal\":{\"display\":true,\"email\":\"info@researchsquare.com\",\"identity\":\"researchsquare\",\"isNatureJournal\":false,\"hasQc\":true,\"allowDirectSubmit\":true,\"externalIdentity\":\"\",\"sideBox\":\"\",\"snPcode\":\"\",\"submissionUrl\":\"/submission\",\"title\":\"Research Square\",\"twitterHandle\":\"researchsquare\",\"acdcEnabled\":true,\"dfaEnabled\":false,\"editorialSystem\":\"\",\"reportingPortfolio\":\"\",\"inReviewEnabled\":false,\"inReviewRevisionsEnabled\":true},\"keywords\":\"\",\"lastPublishedDoi\":\"10.21203/rs.3.rs-6170708/v1\",\"lastPublishedDoiUrl\":\"https://doi.org/10.21203/rs.3.rs-6170708/v1\",\"license\":{\"name\":\"CC BY 4.0\",\"url\":\"https://creativecommons.org/licenses/by/4.0/\"},\"manuscriptAbstract\":\"\\u003cp\\u003eRecent advancements in large language models (LLMs) have demonstrated that explicitly prompting for intermediate reasoning steps significantly improves performance in complex tasks. Traditional chain of thought (CoT) prompting, however, can result in verbose outputs that increase both latency and computational cost. Condensed Reasoning Prompting (CRP) addresses this trade-off by encouraging more concise reasoning traces while maintaining high accuracy. In this paper, we systematically evaluate three prompting strategies: Chain Of Thought (CoT), Chain of Draft (CoD) and Condensed Reasoning across multiple datasets, including MMLU, Big Bench, arithmetic, and symbolic reasoning tasks. We report accuracy, average tokens per question, and a token effectiveness metric (accuracy divided by token count). Our experiments are conducted in a zero-shot setting, without specific system instructions to \\\"skip reasoning,\\\" providing a more realistic assessment of model capabilities. Results indicate that condensed prompts often match or exceed chain of thought accuracy while reducing token usage, thus offering significant gains in efficiency. We discuss the implications for real-world deployments, highlighting how CRP can enable more efficient LLM applications without compromising performance.\\u003c/p\\u003e\",\"manuscriptTitle\":\"Condensed Reasoning Prompting: Efficient Strategies, Evaluations, and Trade Offs in Large Language Model Reasoning\",\"msid\":\"\",\"msnumber\":\"\",\"nonDraftVersions\":[{\"code\":1,\"date\":\"2025-03-07 10:55:32\",\"doi\":\"10.21203/rs.3.rs-6170708/v1\",\"editorialEvents\":[{\"type\":\"communityComments\",\"content\":0}],\"status\":\"published\",\"journal\":{\"display\":true,\"email\":\"info@researchsquare.com\",\"identity\":\"researchsquare\",\"isNatureJournal\":false,\"hasQc\":true,\"allowDirectSubmit\":true,\"externalIdentity\":\"\",\"sideBox\":\"\",\"snPcode\":\"\",\"submissionUrl\":\"/submission\",\"title\":\"Research Square\",\"twitterHandle\":\"researchsquare\",\"acdcEnabled\":true,\"dfaEnabled\":false,\"editorialSystem\":\"\",\"reportingPortfolio\":\"\",\"inReviewEnabled\":false,\"inReviewRevisionsEnabled\":true}}],\"origin\":\"\",\"ownerIdentity\":\"831a1f27-9601-4a18-8339-101c44ddbc16\",\"owner\":[],\"postedDate\":\"March 7th, 2025\",\"published\":true,\"recentEditorialEvents\":[],\"rejectedJournal\":[],\"revision\":\"\",\"amendment\":\"\",\"status\":\"posted\",\"subjectAreas\":[],\"tags\":[],\"updatedAt\":\"2025-03-13T00:23:06+00:00\",\"versionOfRecord\":[],\"versionCreatedAt\":\"2025-03-07 10:55:32\",\"video\":\"\",\"vorDoi\":\"\",\"vorDoiUrl\":\"\",\"workflowStages\":[]},\"version\":\"v1\",\"identity\":\"rs-6170708\",\"journalConfig\":\"researchsquare\"},\"__N_SSP\":true},\"page\":\"/article/[identity]/[[...version]]\",\"query\":{\"redirect\":\"/article/rs-6170708\",\"identity\":\"rs-6170708\",\"version\":[\"v1\"]},\"buildId\":\"8U1c8b4HqxoKbykW_rLl7\",\"isFallback\":false,\"isExperimentalCompile\":false,\"dynamicIds\":[84888],\"gssp\":true,\"scriptLoader\":[]}","source_license":"CC-BY-4.0","license_restricted":false}