GPU based High-efficiency PSO Algorithm with Initialization strategy and Thread Slef-adaption

GPU based High-efficiency PSO Algorithm with Initialization strategy and Thread Slef-adaption | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article GPU based High-efficiency PSO Algorithm with Initialization strategy and Thread Slef-adaption Ye Liu, Jia Wu, Hui Ren, Shuopeng Yang, Fuqiang Zhang, Jie Cao This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-2640747/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Particle Swarm Optimization (PSO) is one of the most commonly heuristics-based methods that has been used to solve various optimization problems due to its simplicity and robustness. However, when comes to practical applications, it requires a huge computational cost. With the development of parallel computing and Graphics Processing Unit (GPU) calculating, many researchers have tried taking these techniques to break down the obstacle of computational efficiency. It is a challenging problem for the long-term application of PSO. In this paper, we propose a HEPSO algorithm that focuses on the procedure optimization of PSO in GPU-based architecture. It optimizes the GPU computation process from two following aspects: 1) Migrate the data initialization procedure from CPUs to GPUs to reduce the huge IO loss caused by repeating migration while the computing process. 2) Employ a self-adaptive thread management strategy to improve the algorithm execution efficiency. Moreover, we use four benchmark optimization functions to test the efficiency of our HEPSO. The experiment results show that the time speedup ratio between HEPSO and GPU-PSO can exceed 6 times. Meanwhile, when we evaluate the performance of HEPSO with the time consumption for functions converge, HEPSO only needs 1/3 time of GPU-PSO in most cases. PSO GPU Initization strategy Thread self-adaption. Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Figure 7 1. Introduction Particle Swarm Optimization (PSO) is a stochastic optimization algorithm based on swarm intelligence proposed by J. Kennedy and R. C. Eberhart in 1995 [ 1 ] , which has the characteristics of simple implementation, few adjustment parameters, a fast convergence rate, and solid robustness. However, the traditional PSO algorithm has some inherent defects, which are mainly reflected in the problem of easily trapping into local optimum and premature convergence speed. In order to overcome these flaws, researchers have first tried to optimize the performance of PSO by improving the algorithm itself. They changed the relevant parameters [ 2 ][ 3 ] or topology structure [ 4 ][ 5 ] of PSO, optimized the particle learning strategy [ 6 ][ 7 ] , and mixed the PSO with other algorithms [ 8 ][ 9 ] to improve the efficiency of PSO. These methods all have achieved positive results, which efficiently improved the convergence speed and accuracy of PSO. Nonetheless, when facing large-scale and highly complex cyclic optimization problems, the convergence speed of the PSO will decrease dramatically as the number of particles increases, such that the efficiency of the optimization procedure is unsatisfactory. Delving into the optimization process of the PSO algorithm, the particles are mutual independence and synergistic cooperation. So researchers consider that the parallelism of particles can be exploited to effectively accelerate the PSO. At present, there are three main strategies of PSO by parallel computation: 1) hardware environment-based parallel PSO algorithms, which are generally implemented through using hardware architectures [ 10 ][ 11 ] such as controllers [ 12 ] , Field Programmable Gate Array (FPGA) [ 13 ][ 14 ] , etc. to effectively improve the optimization speed of PSO algorithm, 2) CPU-based parallel PSO algorithms, which usually adopt multi-threading techniques [ 15 ][ 16 ] or multi-core processors [ 17 ] to express the independence of particles, making full use of parallelization to enhance the efficiency of PSO algorithm, 3) GPU-based parallel PSO algorithms, which flexibly use the architecture of GPU to synchronize the parallel optimization process of particles. However, each strategy has its advantages and disadvantages like the parallel hardware environment are accessible to deploy and implement but depends on the cluster node scale. And CPU-based multi-core parallelism only requires a PC but is greatly limited by the CPU structure and suffers from the complexity of the communication and management of inter-processes. At this time, applying the GPU with low hardware cost and strong computing power to realize the parallelism of PSO algorithm shows significant advantages. With the rapid promotion of GPU parallel calculating capability and the development of Compute Unified Device Architecture (CUDA), many procedures have been realized to take advantage of the availability, parallelism, and unified computing features of GPU to improve the program execution efficiency. Meanwhile, how to combine the advantages of GPU to accelerate the PSO execution has attracted some attention as mentioned above, and a large number of research results have been produced constantly to improve the speedup. However, in recent years, due to the traditional methods suffering from the numerous thread management of GPU and data IO costs, the speedup ratio of PSO has reached a bottleneck. Further research is turning to dig the GPU potential from multiple perspectives such as hardware, algorithms, problem scale et al. 1.1 Object and innovation In summary, while solving increasingly complex practical problems, how to better combine the independence of particle behavior with the parallel architecture of CUDA programming model is the key to achieving higher efficiency of the PSO algorithm. Accordingly, we propose a high-efficiency PSO (HEPSO) based on GPU initialization and thread adaption. Based on the asynchronous model of the PSO algorithm [ 18 ] , HEPSO regards the loop iteration behavior of each particle as an independent optimization process and adopts a coarse-grained parallel method to make the particles correspond to the threads one by one. At the same time, we optimize the PSO algorithm in GPU from the following two perspectives: (1) reduce unnecessary data IO by maximizing data throughput to optimize memory usage, (2) maximize parallel execution and leverage GPU multi-core advantages by designing a thread scheduling scheme to optimize thread usage. The main contributions of the paper are summarized as follows. A high-efficiency PSO arithmetic (HEPSO) with GPU initialization and thread adaption strategy is designed by CUDA. This method can fully exploit the concurrency potential of the PSO and improve the running efficiency of the algorithm from two aspects of memory optimization and thread utilization. HEPSO migrates the data initialization of PSO from CPUs to GPUs to reduce unnecessary costs and employ thread adaption and multiplexing strategy to further improve the optimization speed. We design some experiments to proven the proposed algorithm can remarkably enhance the execution efficiency of the PSO algorithm. The results show that compared with CPU-PSO, the “ \({\text{S}}_{\text{i}\text{t}\text{e}\text{r}\text{a}\text{t}\text{i}\text{o}\text{n}}\) ” of HEPSO can reach up to 580, and is 6 times faster than GPU-PSO. The overall average “ \({\text{S}}_{\text{c}\text{o}\text{n}\text{v}\text{e}\text{r}\text{g}\text{e}\text{n}\text{c}\text{e}}\) ” between the GPU-PSO and the HEPSO is more than 3 times. The remaining paper is organized as follows. Section 2 presents the background on various PSO algorithms and CUDA computing architecture. Then Section 3 describes the detail of the proposed HEPSO algorithm and emphasizes its innovations. In section 4, several experiments are designed and implemented by 4 benchmarks to test the improvement in the efficiency of the HEPSO. Finally, we provide the summarization and discussion of our work in section 5. 2. Background 2.1 Standard PSO The PSO is a stochastic optimization algorithm that simulates group foraging activities and swarms intelligence [ 19 ] . It randomly generates particle swarms of a specific size as efficient solutions of the function in the problem space, and then iteratively searches to obtain an optimal result [ 20 ] . Each particle has its own velocity and position, while the value of the function corresponding to the position of the particle is the valid solution it finds [ 21 ] . In the process of iterative updating, the historical optimal solution of the individual and the global optimal solution of the swarm is used to calculate the initial velocity and position of the particle in the next iteration. The particle looks constantly in the search space until finding the global optimal solution or a feasible solution that satisfies the requirements. The way to update the particle velocity and position is as follows $${V}_{i}^{d+1} = \omega {V}_{i}^{d} + {c}_{1}{r}_{1}\left( {P}_{i}^{d} - {X}_{i}^{d}\right) + {c}_{2}{r}_{2}\left({P}_{g}^{d} - {X}_{i}^{d}\right)$$ 1 $${X}_{i}^{d+1} = {X}_{i}^{d} + \alpha {V}_{i}^{d+1}$$ 2 In which, i = 1, 2…, M, d = 1, 2…, N. \(\omega\) is a non-negative number called the inertia factor, which plays a great role in the convergence effect of the algorithm. The larger its value, the wider the leap range of the particle. \({P}_{i}^{}\) is the local optimal position and \({P}_{g}^{}\) is the global optimal position. The learning factors \({c}_{1}\) and \({c}_{2}\) are non-negative constants, which are parameters for adjusting the weights of local and global optimal values. In practical application, the most appropriate parameter value could be found through multiple experimental adjustments. \({r}_{1}\) , \({r}_{2}\) are arbitrary numbers in the range of [0, 1]. \(\alpha\) is a constraint factor to control the weight of the velocity. In the implementation of standard PSO, the entire calculation of the algorithm is completed on the CPUs. Note that \({X}_{i}^{d+1}\) and \({V}_{i}^{d+1}\) are the current position and velocity respectively, \({X}_{i}^{d}\) and \({V}_{i}^{d}\) are the previous position and velocity respectively. 2.2 Traditional GPU-based PSO In recent years, many researchers have taken great efforts to improve the efficiency of PSO by combining the parallelism of the GPU architecture, and many achievements have been obtained. In spite of the attention, there was an endless challenge for improving efficiency of GPU-based PSO algorithms. R. M. Calazan et al. [ 22 ] proposed an algorithm Parallel Dimension PSO (PDPSO) based on the GPU architecture to massively parallelize PSO programs in which each particle is implemented as a block and each dimension is mapped into a distinct thread. Their experimental results are up to 85 times faster than the serial implementation on the CPU. You Zhou et al. [ 23 ] proposed a variational PSO algorithm, which makes particles evolve in a beneficial direction to accelerate the convergence speed of the process. They proved that the variant PSO algorithm has a more excellent running speed by several benchmark functions, and when the population size is larger, the program can at least achieve a speedup of 25 times after being accelerated by GPU. Cai Yong et al. [ 24 ] designed a coarse-grained synchronous parallel PSO algorithm, which is based on the coarse-grained parallelism idea of making threads correspond to particles one-to-one. This method creates a large number of threads to deserialize the particle search process and makes full use of various mathematical function calculation libraries from CUDA to ensure the reliability and usability of PSO. Through the experiments of three optimization functions, they demonstrate that compared with the traditional CPU-PSO, their algorithm is able to acquire a computational speedup of up to 90 times under the condition of function convergence. Xicheng Fu et al. [ 17 ] proposed a local PSOA based on medical image registration GPU. Their method has obvious advantages in the optimization of a high dimensional objective function, with the maximum acceleration ratio reaching 95 times. The total average difference in the number of iterations when the serial implementation and parallel implementation of the algorithm stop running under the premise of satisfying the accuracy is 17 times. 2.3 Thread allocation strategy based on CUDA parallel architecture The parallel computing model of CUDA adopts a multi-level memory architecture, which is divided into three levels: Thread, Block, and Grid. There are multiple threads and blocks in a block and a grid, respectively. In practice, Block is divided into smaller thread bundles (Warp), which is the basic unit of scheduling and execution for Streaming Multiprocessor (SM). At present, there are three main thread allocation schemes suitable for the CUDA parallel architecture model [ 26 ] : (1) coarse-grained parallelism where particles correspond to threads one by one [ 24 ] , (2) fine-grained parallelism where particles correspond to blocks one by one and the dimensions of the particles correspond to threads one by one [ 22 ][ 25 ] , (3) adaptive thread bundle parallelism where particles correspond to one or more warps, the dimensions of the particles correspond to threads one by one, and one or more particles correspond to block [ 26 ] . In a concrete implementation, by expanding the number of threads in each block as much as possible, and setting the block size to a multiple of warp, we can ensure the task balance among SMs and improve the running efficiency of the algorithm. 3. Methodology 3.1 Algorithm flow The algorithm flow of HEPSO is shown in Fig. 3 . The steps of HEPSO are as follows. (1) The CPUs initialize the relevant variables on the CPUs, and copy them to the GPUs by function cuda.to_device(), (2) According to the problem scale and swarm size, the CPUs adaptively specify the execute configuration [GridDim, BlockDim], (3) The CPUs invoke the kernel1() to initialize the particle swarm on the GPUs, and obtain initial velocity and position to calculate the particle local optimum and the swarm global optimum respectively, (4) The CPUs invoke the kernel2() to cyclically update velocity and position on the GPUs to calculate the fitness value of the particle while gaining the particle local optimum and global swarm optimum through comparison, (5) The GPU loops through step (4) until the end condition is met (reaching the target accuracy or the specified number of iterations), (6) The GPUs terminate the loop and copy the solution of the global swarm optimum to CPUs by function cuda.copy_to_host(). Among the above steps, steps (3) and (4) are the core of the HEPSO algorithm, the pseudo-code is as follow. Algorithm 1 kernel1() : Initialization Map all the threads to S particles one-to-one //Do operations to thread i (i = 1,…,S) synchronously: for i = 1 to S do for j = 1 to N do initialize the position Xi and velocity Vi of particle[i] compute the local optimum PF[i] of particle[i] compute the global optimum PB[i] of Swarm In Algorithm 1, as well as in the remainder of this paper, we denote by PB[i] the best global fitness particle i has achieved so far and PBX[i] the coordinates of the position that yields it. As well, we denote PF[i] as the best local fitness particle i has achieved so far and PFX[i] as the coordinates of the position that yields it. In addition, S represents the particle swarm size and N represents the problem size. Algorithm 2 kernel2() : Update Map all the threads to S particles one-to-one //Do operations to thread i (i = 1,…,S) synchronously: repeat for i = 1 to S do for j = 1 to N do update position Xi and velocity Vi of particle[i] compute fitness F[i] if F[i] \(\le\) PF[i] then update PF[i] and PFX[i] using position of particle[i] If PF[i] \(\le\) PB[i] then update PB[i] and PBX[i] using the PF[i] and PFX[i] Until reaching the stop conditions Return PB[i] and PBX[i] During the execution of the iterative loop, the algorithm requires a large number of random numbers for updating velocity. We use the random number generation algorithm numba.cuda.random.create_xoroshiro128p_states() for the CUDA. 3.2 GPU initialization When data is initialized, the traditional way is to perform the initialization operations on CPUs, and then copy the data to GPUs for further computation. However, this conventional IO process will waste a lot of invalid time. Based on the experimental environment shown in Table 1 , we test the time consumption by the initialization of the data on the CPUs and the GPUs respectively. The experimental results as shown in Fig. 4 . we can notice that when the scale of the problem reaches the million level, one IO operation will cause an average time loss of 6 seconds, which greatly limits the execution efficiency of the algorithm. HEPSO we proposed only performs the necessary initialization on CPUs, including the population dimension and the number of particles. Meanwhile, all the rest of data initialization, such as velocity, position, and fitness by creating some empty arrays to take over, and the initialization of concrete assignments are done by GPUs. In this way, most of the initializations, all subsequent iterations, and comparisons of data can be wholly undertaken by the GPU core, without consuming IO resources. 3.3 Thread self-adaptive strategy The execute configuration (Kernel<<>>) are set by the kernel function which needs to be determined after debugging sessions in CUDA call time. In the Single-Instruction Multiple-Thread (SIMT) architecture used by SM, Warp is the smallest unit of scheduling and execution. If the number of threads contained in a block is not set properly, some inactive threads will remain in the underutilized warp and also consume SM resources. Therefore, in order to avoid wasting memory resources, the number of threads in a block is recommended as an integer multiple of the warp size. In our experiments, we proposed a thread-adaptive coarse-grained parallel algorithm to make full use of shared memory and maximize the degree of parallel execution. It can automatically adjust the parameters according to the specific particle numbers and problem dimensions when calling the kernel function to allow the particles to correspond with the threads one by one. The self-adaptive formula are as shown in Eq. ( 3 ). $$\text{K}\text{e}\text{r}\text{n}\text{e}\text{l} \left( \text{G}\text{r}\text{i}\text{d}\text{D}\text{i}\text{m}, \text{B}\text{l}\text{o}\text{c}\text{k}\text{D}\text{i}\text{m} \right)=$$ $$\text{K}\text{e}\text{r}\text{n}\text{e}\text{l} \left( \right(\text{S} / \text{N} / \text{B}\text{l}\text{o}\text{c}\text{k}\text{D}\text{i}\text{m} , \text{N}), (\text{n}, \text{W}\text{a}\text{r}\text{p}\text{S}\text{i}\text{z}\text{e}\left) \right)$$ 3 Among them, GridDim is the size of a grid, that is, the number of blocks contained in it; BlockDim is the size of a block, that is, the number of threads contained in it. We set the maximum size of BlockDim is set to 512. WarpSize represents the size of a warp, which is normally calculated by the GPU in HalfWarp units, so WarpSize is defined as 16. This self-adaptive strategy allows the kernel function to expand the number of threads in each block as much as possible and set the BlockDim to a multiple of warp [ 26 ] . It dynamically adjusts the number of threads to match both the problem scale and particle size to avoid the waste of memory resources and improve program performance. 3.4 Thread multiplexing There is overhead in the creation and destruction of threads in CUDA. In order to process a large amount of data, CUDA needs to create threads larger than the number of calculations. If each thread only performs an operation once and then is destroyed, the more data, the cost will be greater. For the sake of saving this overhead, we adopt the way of grid stride which adds a loop for each thread to realize thread multiplexing, so that a created thread can be used repeatedly. The grid stride is shown in Fig. 5 . In Fig. 3 , the data size to process in parallel is 12. We create a grid that can start 4<<>> threads in parallel and set the stride size to 4 after adding a loop. That means, the 1st, 5th, and 9th data can share thread 1. Such a procedure allows each thread to be utilized multiple times, which reduces the thread overhead of repeatedly starting and destroying and permits CUDA to parallelly handle large-scale problems. 4. Experiment 4.1 Experimental environment The computing environment based on our experiments is shown in Table 1 . Table 1 Computing environment Name Version CPU Intel(R) Core (TM) i7-6700 CPU @ 3.40GHz 3.41 GHz GPU NVIDIA Tesla T4 Operating System Centos7 Development Environment Anaconda 3, CUDA 10.2 4.2 Optimization function In practice, the performance of optimization algorithms is usually determined by the function evaluation value. In this paper, performance comparisons were conducted based on four benchmark test functions as listed in Table 2 . Table 2 Benchmark functions F(x) Domain Optimum Solution \({f}1\left({{X}}_{{i}}\right)= \sum _{{i}=1}^{{n}-1}{(100\left({{X}}_{{i}+1}- {{X}}_{{i}}^{2}\right)}^{2}+ {({{X}}_{{i}}- 1)}^{2})\) \(-10.0\le {{X}}_{{i}}\le 10.0\) \({{f}}_{{m}{i}{n}}={f} (\text{1,1},\dots ,1) = 0\) \({f}2\left({{X}}_{{i}}\right)= \sum _{{i}=1}^{{n}}{{X}}_{{i}}^{2}\) \(-100.0\le {{X}}_{{i}}\le 100.0\) \({{f}}_{{m}{i}{n}}={f} (\text{0,0},\dots ,0) = 0\) \({f}3\left({{X}}_{{i}}\right)=10{n}+ \sum _{{i}=1}^{{n}}({{X}}_{{i}}^{2}- 10\mathbf{c}\mathbf{o}\mathbf{s}\left(2{\pi }{{X}}_{{i}}\right))\) \(-5.12\le {{X}}_{{i}}\le 5.12\) \({{f}}_{{m}{i}{n}}={f} (\text{0,0},\dots ,0) = 0\) \({f}4\left({{X}}_{{i}}\right)= \sum _{{i}=1}^{{n}}\frac{{{X}}_{{i}}^{2}}{4000} - \prod _{{i}=1}^{{n}}\mathbf{cos}\left(\frac{{{X}}_{{i}}}{\sqrt{{i}}}\right)+1\) \(-600.0\le {{X}}_{{i}}\le 600.0\) \({{f}}_{{m}{i}{n}}={f} (\text{0,0},\dots ,0) = 0\) The inertia weight ω was taken as 0.32, the learning factors C1 and C2 were both taken as 2.0, and the constraint factor α was taken as 0.5. 4.3 Evaluation assessment To quantitate access to the performance of different algorithms, an evaluation index is required. In this subsection, two index computing methods are introduced. They are all based on the time cost but with different finish conditions. 4.3.1 The assessment of time-ratio to stop iterating The very first one is the time ratio to stop iterating which indicates the time difference between different algorithms under the condition of the same iterations. It can evaluate the pure computational capability of an algorithm. Under the condition that the numbers of particles and iterations are specified to be the same, we define " \({\text{T}}_{\text{R}\text{e}\text{f}}\) " and " \({\text{T}}_{\text{O}\text{b}\text{j}}\) " as the running time of reference and object algorithm programs respectively. The time ratio to stop iteration index is \(\text{“}{\text{S}}_{\text{i}\text{t}\text{e}\text{r}\text{a}\text{t}\text{i}\text{o}\text{n}}\text{”}\) is set as follows. $${\text{S}}_{\text{i}\text{t}\text{e}\text{r}\text{a}\text{t}\text{i}\text{o}\text{n}}= \frac{{\text{T}}_{\text{R}\text{e}\text{f}}}{{\text{T}}_{\text{O}\text{b}\text{j}}}$$ 4 4.3.2 The assessment of time-ratio after functions convergence On the other hand, the above assessment index ignores the convergence difference within the optimization solver which may turn up a condition of one party stopping iterating while the other party still running. In consequence, we consider testing the overall average difference in execution time after functions convergence between different algorithms. We define the " \(\text{min}\left({ \text{T}}_{\text{R}\text{e}\text{f}} \right)\) " and " \({ \text{m}\text{i}\text{n}(\text{T}}_{\text{O}\text{b}\text{j} })\) " as the minimum running time under the condition of function converges to a particular precision of reference and object algorithm, respectively. This index \(\text{“}{\text{S}}_{\text{c}\text{o}\text{n}\text{v}\text{e}\text{r}\text{g}\text{e}\text{n}\text{c}\text{e}}\text{”}\) is set as follows. $${\text{S}}_{\text{c}\text{o}\text{n}\text{v}\text{e}\text{r}\text{g}\text{e}\text{n}\text{c}\text{e}}= \frac{\text{m}\text{i}\text{n}\left({ \text{T}}_{\text{R}\text{e}\text{f}} \right)}{{\text{m}\text{i}\text{n}（ \text{T}}_{\text{O}\text{b}\text{j} }）}$$ 5 4.4 Experimental results 4.4.1 time-ratio to stop iterating The initial number of particles and iterations were set to 128 and 10000 respectively, and the dimension N of all four benchmark functions was 16. In the experiments, we increased the number of particles and dynamically decreased the number of iterations to reduce the execution time on CPUs. The number of particles and iterations were respectively range from 128 to 131072 and from 10000 to 200. Each experiment was run until the maximum number of iterations has been reached. The optimization were repeated 20 times with different seeds. The average results are shown in Fig. 6 . – Fig. 7 . and Table 3 - Table 6 in the appendix [ 10 , 11 ] respectively. Table 3 Execution time and “ \({\text{S}}_{\text{i}\text{t}\text{e}\text{r}\text{a}\text{t}\text{i}\text{o}\text{n}}\) ” of f1 Swarmsize Iterations Execution Time/s \({\mathbf{S}}_{\mathbf{i}\mathbf{t}\mathbf{e}\mathbf{r}\mathbf{a}\mathbf{t}\mathbf{i}\mathbf{o}\mathbf{n}}\) CPU GPU HEPSO CPU/HEPSO GPU/HEPSO 128 10000 49.951 16.988 4.547 10.99 3.74 256 9000 93.823 15.789 4.674 20.07 3.38 512 7500 154.924 14.183 4.625 33.49 3.07 1024 6000 245.898 12.409 4.003 61.43 3.10 2048 5000 414.184 12.149 4.150 99.80 2.93 4096 3000 501.445 10.190 3.757 133.48 2.71 8192 2000 640.905 9.168 4.073 157.37 2.25 16384 1200 792.098 8.093 2.860 276.92 2.83 32768 700 899.922 7.989 2.096 429.27 3.81 65536 380 998.583 8.903 2.023 493.72 4.40 131072 200 1018.983 11.666 1.746 583.66 6.68 Table 4 Execution time and “ \({\text{S}}_{\text{i}\text{t}\text{e}\text{r}\text{a}\text{t}\text{i}\text{o}\text{n}}\) ” of f2 Swarmsize Iterations Execution Time/s \({\mathbf{S}}_{\mathbf{i}\mathbf{t}\mathbf{e}\mathbf{r}\mathbf{a}\mathbf{t}\mathbf{i}\mathbf{o}\mathbf{n}}\) CPU GPU HEPSO CPU/HEPSO GPU/HEPSO 128 10000 16.726 16.771 4.122 4.06 4.07 256 9000 29.223 15.371 4.022 7.27 3.82 512 7500 49.287 13.509 3.862 12.76 3.50 1024 6000 76.568 11.722 3.725 20.55 3.15 2048 5000 127.971 11.284 3.482 36.75 3.24 4096 3000 154.396 9.116 2.788 55.38 3.27 8192 2000 204.227 8.777 2.671 76.47 3.29 16384 1200 249.169 7.555 2.025 123.03 3.73 32768 700 287.210 6.548 1.533 187.35 4.27 65536 380 317.406 6.584 1.537 206.49 4.28 131072 200 334.853 7.511 1.485 225.44 5.06 Table 5 Execution time and “ \({\text{S}}_{\text{i}\text{t}\text{e}\text{r}\text{a}\text{t}\text{i}\text{o}\text{n}}\) ” of f3 Swarmsize Iterations Execution Time/s \({\mathbf{S}}_{\mathbf{i}\mathbf{t}\mathbf{e}\mathbf{r}\mathbf{a}\mathbf{t}\mathbf{i}\mathbf{o}\mathbf{n}}\) CPU GPU HEPSO CPU/HEPSO GPU/HEPSO 128 10000 21.692 16.958 5.615 3.86 3.02 256 9000 38.740 15.977 5.374 7.21 2.97 512 7500 62.691 14.707 5.278 11.88 2.79 1024 6000 100.296 13.686 5.899 17.00 2.32 2048 5000 169.697 14.410 7.718 21.99 1.87 4096 3000 200.803 10.991 8.142 24.66 1.35 8192 2000 268.514 9.401 10.024 26.79 0.94 16384 1200 322.508 9.465 6.443 50.05 1.47 32768 700 379.388 9.337 4.107 92.38 2.27 65536 380 420.789 10.006 2.768 152.00 3.61 131072 200 444.381 10.892 2.807 158.30 3.88 Table 6 Execution time and “ \({\text{S}}_{\text{i}\text{t}\text{e}\text{r}\text{a}\text{t}\text{i}\text{o}\text{n}}\) ” of f4 Swarmsize Iterations Execution Time/s \({\mathbf{S}}_{\mathbf{i}\mathbf{t}\mathbf{e}\mathbf{r}\mathbf{a}\mathbf{t}\mathbf{i}\mathbf{o}\mathbf{n}}\) CPU GPU HEPSO CPU/HEPSO GPU/HEPSO 128 10000 32.419 17.266 5.906 5.49 2.92 256 9000 56.345 16.707 6.261 8.99 2.67 512 7500 93.533 15.741 7.373 12.69 2.13 1024 6000 151.712 15.508 9.803 15.48 1.58 2048 5000 253.252 15.085 14.304 17.70 1.05 4096 3000 300.272 10.960 15.499 19.37 0.71 8192 2000 387.161 11.188 19.817 19.54 0.56 16384 1200 470.528 11.740 11.640 40.42 1.01 32768 700 549.339 12.207 7.097 77.40 1.72 65536 380 614.538 13.327 4.360 140.94 3.06 131072 200 649.055 15.094 4.408 147.25 3.42 The “ \({\text{S}}_{\text{i}\text{t}\text{e}\text{r}\text{a}\text{t}\text{i}\text{o}\text{n}}\) ” of 4 benchmark functions are shown in Fig. 5 . The analysis of the experimental results in Fig. 6 . – Fig. 7 . and Table 3 - Table 6 in the appendix [ 10 , 11 ] shows that the HEPSO algorithm on f1, f2, f3, and f4 has achieved the maximum “ \({\text{S}}_{\text{i}\text{t}\text{e}\text{r}\text{a}\text{t}\text{i}\text{o}\text{n}}\) ” of 583.6, 225.4, 158.3, and 147.2 compared to CPU-PSO. Meanwhile, for HEPSO to GPU-PSO, it achieved the maximum “ \({\text{S}}_{\text{i}\text{t}\text{e}\text{r}\text{a}\text{t}\text{i}\text{o}\text{n}}\) ” of 6.68, 5.06, 3.88, and 3.42. Several conclusions can be drawn as follows. From the above experiments, we can conclude that the PSO executed on the CPU is only suitable for a simple functions optimization problem (the number of particles is small, usually below 300). When facing a more complex and higher dimensional problem, with requires particle number enlarging, the load on the CPU expands accordingly and the optimization time increases significantly. As a whole, the execution time of HEPSO is less than GPU-PSO. When the number of particles is greater than 16384, the time consumption difference represents a rapidly widening trend. With the support of the parallel ability of GPU, the computation time is shorter with particle number. That means one can deploy more particles to accelerate the optimization search procedure. For complex functions f3 and f4, when the number of particles is less than 8192, the execution time of GPU-PSO keeps decreasing while the execution time of HEPSO keeps increasing, which is because the advantage of local parallelism outweighs the time required for thread synchronization. When the number of particles increases to more than 8192, the benefits brought by local parallelism gradually fails to offset the large IO loss. That causes a significant decrease in the efficiency of GPU-PSO to about 1/2. This indicates that the HEPSO we proposed needs to generate a large number of particles to accelerate the optimization process with more complicated functions. Among several test functions, HEPSO has achieved the best speedup in the f1. It has achieved 580 times “ \({\text{S}}_{\text{i}\text{t}\text{e}\text{r}\text{a}\text{t}\text{i}\text{o}\text{n}}\) ” compared with CPU-PSO and has also achieved 6 times “ \({\text{S}}_{\text{i}\text{t}\text{e}\text{r}\text{a}\text{t}\text{i}\text{o}\text{n}}\) ” compared with GPU-PSO. Therefore, HEPSO can be proven to be suitable for solving large-scale optimization problems with complex functions. 4.4.2 time-ratio after functions convergence Considering the problem mentioned in 4.3.2, we design another experiment with the iteration termination condition set to be the convergence error of 10 − 4 . In this experiment, the dimension N of all four benchmark functions is 16. The optimization were repeated 50 times with different seeds. The average results we obtained are shown in Table 7 . Table 7 “ \({\text{S}}_{\text{c}\text{o}\text{n}\text{v}\text{e}\text{r}\text{g}\text{e}\text{n}\text{c}\text{e}}\) ” of f1, f2, f3, and f4 F(x) Swarmsize (GPU) Swarmsize (HEPSO) Accuracy(GPU) Accuracy (HEPSO) GPU-time HEPSO-time \({\mathbf{S}}_{\mathbf{c}\mathbf{o}\mathbf{n}\mathbf{v}\mathbf{e}\mathbf{r}\mathbf{g}\mathbf{e}\mathbf{n}\mathbf{c}\mathbf{e}}\) (GPU/HEPSO) f1 65536 524288 8.2E-05 2.9E-05 8.57 2.25 3.81 f2 32768 65536 7E-05 9.5E-05 3.53 1.32 2.67 f3 32768 65536 9E-05 1.4E-05 3.91 1.67 2.34 f4 32768 131072 2.2E-06 1.3E-06 4.51 2.24 2.01 From the data in Table 7 , it can be seen that the HEPSO achieved the maximum \(\text{“}{\text{S}}_{\text{c}\text{o}\text{n}\text{v}\text{e}\text{r}\text{g}\text{e}\text{n}\text{c}\text{e}}\text{”}\) of 3.81, 2.67, 2.34, and 2.01 for the 4 benchmark functions of f1, f2, f3, and f4. Compared with the above “ \({\text{S}}_{\text{i}\text{t}\text{e}\text{r}\text{a}\text{t}\text{i}\text{o}\text{n}}\) ”, the “ \({\text{S}}_{\text{c}\text{o}\text{n}\text{v}\text{e}\text{r}\text{g}\text{e}\text{n}\text{c}\text{e}}\) ” is more in line with the actual application situation. 5. Conclusion This paper proposes a high-efficiency PSO arithmetic based on GPU initialization and thread self-adaptive. Firstly, the initialization of the particle process in the PSO is migrated from CPUs to GPUs, which enormously improves the computational efficiency of the algorithm by reducing its IO cost. In addition, two strategies of thread adaption and thread multiplexing strategy are added to enhance the efficiency of PSO. In this paper, with 4 benchmark functions, HEPSO is tested by calculating two sets of evaluation indexes: “ \({\text{S}}_{\text{i}\text{t}\text{e}\text{r}\text{a}\text{t}\text{i}\text{o}\text{n}}\) ” and “ \({\text{S}}_{\text{c}\text{o}\text{n}\text{v}\text{e}\text{r}\text{g}\text{e}\text{n}\text{c}\text{e}}\) ”. Among them, (1) The speedup of CPU-PSO, GPU-PSO, and HEPSO is quantitatively tested and analyzed at different particle numbers and iteration times. The results show that when the number of particles reaches 131072, HEPSO can achieve 580-times “ \({\text{S}}_{\text{i}\text{t}\text{e}\text{r}\text{a}\text{t}\text{i}\text{o}\text{n}}\) ” compared to the CPU-PSO, and over 6 times higher than GPU-PSO. (2) HEPSO was tested from the perspective of practical application by contrasting the overall average difference of time after functions convergence. In all 4 related tests, the HEPSO obtains a “ \({\text{S}}_{\text{c}\text{o}\text{n}\text{v}\text{e}\text{r}\text{g}\text{e}\text{n}\text{c}\text{e}}\) ” of more than 3 times. From the theoretical and experimental analysis, our HEPSO shows many significant advantages. Compared with other methods, especially in large-scale and complex optimization problems, the efficiency of PSO is improved to a higher level. This can help PSO overcome the efficiency limitations to face practical application challenges. Declarations Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Acknowledgments This work was supported bythe Natural Science Foundation of Shaanxi Province ( 2022JM-301), and the Postgraduate Innovation and Practice Ability Development Fund of Xi’an Shiyou University. Author contributions The corresponding author Ye Liu conceived and designed the work that led to the submission, is responsible for the crucial parts in interpreting the results. Jia Wu and Hui Ren process the dataset and help to analysis the basic data. Shuopeng Yang, Fuqiang Zhang, and Jie Cao process the data and revised the manscript. References Wang D, Tan D, Liu L. Particle swarm optimization algorithm: an overview, J. Soft computing, 2018, 22(2): 387-408. Chen K, Zhou F, Liu A. Chaotic dynamic weight particle swarm optimization for numerical function optimization, J. Knowledge-Based Systems, 2018, 139: 23-40. Goudarzi A, Li Y, Xiang J. A hybrid non-linear time-varying double-weighted particle swarm optimization for solving non-convex combined environmental economic dispatch problem, J. Applied Soft Computing, 2020, 86: 105894. Lim W H, Isa N A M. Particle swarm optimization with increasing topology connectivity, J. Engineering Applications of Artificial Intelligence, 2014, 27: 80-102. Lin A, Sun W, Yu H, et al. Global genetic learning particle swarm optimization with diversity enhancement by ring topology, J. Swarm and evolutionary computation, 2019, 44: 571-583. Lynn N, Suganthan P N. Heterogeneous comprehensive learning particle swarm optimization with enhanced exploration and exploitation, J. Swarm and Evolutionary Computation, 2015, 24: 11-24. Xu G, Cui Q, Shi X, et al. Particle swarm optimization based on dimensional learning strategy, J. Swarm and Evolutionary Computation, 2019, 45: 33-51. Jindal V, Bedi P. An improved hybrid ant particle optimization (IHAPO) algorithm for reducing travel time in VANETs, J. Applied Soft Computing, 2018, 64: 526-535. Laskar N M, Guha K, Chatterjee I, et al. HWPSO: A new hybrid whale-particle swarm optimization algorithm and its application in electronic design optimization problems, J. Applied Intelligence, 2019, 49: 265-291. Tewolde G S, Hanna D M, Haskell R E. Multi-swarm parallel PSO: Hardware implementation, C. 2009 IEEE Swarm Intelligence Symposium. IEEE, 2009: 60-66. Damaj I, Elshafei M, El-Abd M, et al. An analytical framework for high-speed hardware particle swarm optimization, J. Microprocessors and Microsystems, 2020, 72: 102949. Suzuki R, Kawai F, Nakazawa C, et al. Parameter optimization of model predictive control by PSO, J. Electrical Engineering in Japan, 2012, 178(1): 40-49. Da Costa A L X, Silva C A D, Torquato M F, et al. Parallel implementation of particle swarm optimization on FPGA, J. IEEE Transactions on Circuits and Systems II: Express Briefs, 2019, 66(11): 1875-1879. Zhao Y, Yu X, Wu H, et al. A Fast 2-D Otsu lung tissue image segmentation algorithm based on improved PSO, J. Microprocessors and Microsystems, 2021, 80: 103527. Martinez-Rios F, Murillo-Suarez A. A new swarm algorithm for global optimization of multimodal functions over multi-threading architecture hybridized with simulating annealing, J. Procedia Computer Science, 2018, 135: 449-456. S.Thongkrairat and V.Chutchavong, "A Time Improvement PSO Base Algorithm Using Multithread Programming," 2019 4th International Conference on Communication and Information Systems (ICCIS), 2019, pp. 212-216. E. A. Abdullah, I. Ahmed Saleh and O. I. Al Saif, "Performance Evaluation of Parallel Particle Swarm Optimization for Multicore Environment," 2018 International Conference on Advanced Science and Engineering (ICOASE), 2018, pp. 81-86. Jingchao L. The Research and Application of Parallel Particle Swarm Optimization Algorithm Based on CUDA, D. Guangdong University of Technology, 2014. Verma A, Kaushal S. A hybrid multi-objective particle swarm optimization for scientific workflow scheduling, J. Parallel Computing, 2017, 62: 1-19. Li B, Wada K. Communication latency tolerant parallel algorithm for particle swarm optimization, J. Parallel Computing, 2011, 37(1): 1-10. Hussain M M, Fujimoto N. GPU-based parallel multi-objective particle swarm optimization for large swarms and high dimensional problems, J. Parallel Computing, 2020, 92: 102589. Calazan R M, Nedjah N, de Macedo Mourelle L. Parallel GPU-based implementation of high dimension particle swarm optimizations, C. 2013 IEEE 4th Latin American Symposium on Circuits and Systems (LASCAS). IEEE, 2013: 1-4. Zhou Y, Tan Y. Particle swarm optimization with triggered mutation and its implementation based on GPU, C. Proceedings of the 12th annual conference on Genetic and evolutionary computation. 2010: 1-8. Cai Y, Li G Y, Wang H. Research and implementation of parallel particle swarm optimization based on CUDA, J. Application Research of Computers, 2013, 30(8): 2415-2418. Fu X, Ma S Q, Yun D W, et al. GPU Local PSO Algorithm at Dimension Level-Based Medical Image Registration, M. Fuzzy Information and Engineering-2019. Springer, Singapore, 2020: 133-144. ZHANG S, HE F, ZHOU Y, et al. GPU parallel particle swarm optimization algorithm based on adaptive warp, J. Journal of Computer Applications, 2016, 36(12): 3274. Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-2640747","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":183901033,"identity":"087d4e55-0b74-40df-996b-a002814bc870","order_by":0,"name":"Ye Liu","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAxklEQVRIiWNgGAWjYFAC5oYDDBVQNg9xWhiBWs6QqoWBsY0ULQbHGxsP8847bG9wI4Hxwds2BnlzglrOHGw4zLstLXHDjQRmw7ltDIY7GwhpuZEI0mKTALSFTZq3jSHB4AAhLfcfArXMkQA5jP03cVpuMAK1NNgwAh3GxkyUFskziQ0H5xxLS5x55mGz5JxzEoYbCGnhO3748Ic3NYft+Y4nH/zwpsxGnqAtCgfgDGAEMTBIEFAPBPIN6IxRMApGwSgYBegAACS/Rxvgigq0AAAAAElFTkSuQmCC","orcid":"","institution":"Xi'an Shiyou University","correspondingAuthor":true,"prefix":"","firstName":"Ye","middleName":"","lastName":"Liu","suffix":""},{"id":183901034,"identity":"8a4b4780-effa-478e-a7a4-92f3200cb24e","order_by":1,"name":"Jia Wu","email":"","orcid":"","institution":"Xi'an Shiyou University","correspondingAuthor":false,"prefix":"","firstName":"Jia","middleName":"","lastName":"Wu","suffix":""},{"id":183901035,"identity":"fb355ba7-0445-4f2a-887f-f554afd446c6","order_by":2,"name":"Hui Ren","email":"","orcid":"","institution":"Xi'an Shiyou University","correspondingAuthor":false,"prefix":"","firstName":"Hui","middleName":"","lastName":"Ren","suffix":""},{"id":183901036,"identity":"4e4c14b8-1d07-4a5c-a535-cf931dc11371","order_by":3,"name":"Shuopeng Yang","email":"","orcid":"","institution":"Xi'an Shiyou University","correspondingAuthor":false,"prefix":"","firstName":"Shuopeng","middleName":"","lastName":"Yang","suffix":""},{"id":183901037,"identity":"a390af49-114a-450e-87cf-028dd9431af8","order_by":4,"name":"Fuqiang Zhang","email":"","orcid":"","institution":"Xi'an Shiyou University","correspondingAuthor":false,"prefix":"","firstName":"Fuqiang","middleName":"","lastName":"Zhang","suffix":""},{"id":183901038,"identity":"e3c8f83a-7535-4652-97d5-9b10f0d3ab47","order_by":5,"name":"Jie Cao","email":"","orcid":"","institution":"eDrilling AS","correspondingAuthor":false,"prefix":"","firstName":"Jie","middleName":"","lastName":"Cao","suffix":""}],"badges":[],"createdAt":"2023-03-01 01:29:18","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-2640747/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-2640747/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":34528295,"identity":"5c141f19-b412-49f7-b787-addcca72e966","added_by":"auto","created_at":"2023-03-20 15:09:42","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":111778,"visible":true,"origin":"","legend":"\u003cp\u003eTraditional GPU-PSO\u003c/p\u003e","description":"","filename":"floatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-2640747/v1/02565be8d9ed49351220516f.png"},{"id":34530555,"identity":"4a898e6a-2317-4e4d-bfc4-638b7611d35a","added_by":"auto","created_at":"2023-03-20 15:25:42","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":153775,"visible":true,"origin":"","legend":"\u003cp\u003eCUDA parallel architecture\u003c/p\u003e","description":"","filename":"floatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-2640747/v1/2397652c7aec0b719c4f62ff.png"},{"id":34529701,"identity":"6d93e5da-3c26-447b-b55e-a1df0fcbc31a","added_by":"auto","created_at":"2023-03-20 15:17:42","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":191763,"visible":true,"origin":"","legend":"\u003cp\u003eHigh-efficiency PSO\u003c/p\u003e","description":"","filename":"floatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-2640747/v1/1ed6717413f42bb01afeaa1e.png"},{"id":34528297,"identity":"33a923f8-017f-4017-be55-8893dc1c13cc","added_by":"auto","created_at":"2023-03-20 15:09:42","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":27169,"visible":true,"origin":"","legend":"\u003cp\u003eA contrast test about date initialization\u003c/p\u003e","description":"","filename":"floatimage4.png","url":"https://assets-eu.researchsquare.com/files/rs-2640747/v1/84dbaba486cb84025f80bd50.png"},{"id":34528300,"identity":"f15956a1-bd90-4512-8579-7409da06f8bc","added_by":"auto","created_at":"2023-03-20 15:09:42","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":70572,"visible":true,"origin":"","legend":"\u003cp\u003eGrid stride\u003c/p\u003e","description":"","filename":"floatimage5.png","url":"https://assets-eu.researchsquare.com/files/rs-2640747/v1/c8d5a6bde9ee52c8428bb3d3.png"},{"id":34528296,"identity":"091c565e-e4aa-4995-8fcc-cabe7fbb7264","added_by":"auto","created_at":"2023-03-20 15:09:42","extension":"png","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":40704,"visible":true,"origin":"","legend":"\u003cp\u003eExecution time comparison of f1, f2, f3, and f4\u003c/p\u003e","description":"","filename":"floatimage6.png","url":"https://assets-eu.researchsquare.com/files/rs-2640747/v1/b33038b899554b456c630755.png"},{"id":34528299,"identity":"c08d174b-1c64-49e1-a128-bbc2f651ec63","added_by":"auto","created_at":"2023-03-20 15:09:42","extension":"png","order_by":7,"title":"Figure 7","display":"","copyAsset":false,"role":"figure","size":44867,"visible":true,"origin":"","legend":"\u003cp\u003e“S\u003csub\u003eiteration\u003c/sub\u003e” of f1, f2, f3, and f4\u003c/p\u003e","description":"","filename":"floatimage7.png","url":"https://assets-eu.researchsquare.com/files/rs-2640747/v1/9c2a60b5c07db8fae7abc1d7.png"},{"id":34819381,"identity":"20f42513-cae7-4b43-a6ea-c63127d88e72","added_by":"auto","created_at":"2023-03-26 00:44:32","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1148010,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-2640747/v1/f0162a5f-ffd1-4e30-837d-f286694bc991.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"GPU based High-efficiency PSO Algorithm with Initialization strategy and Thread Slef-adaption","fulltext":[{"header":"1.\tIntroduction","content":"\u003cp\u003eParticle Swarm Optimization (PSO) is a stochastic optimization algorithm based on swarm intelligence proposed by J. Kennedy and R. C. Eberhart in 1995\u003csup\u003e[\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e]\u003c/sup\u003e, which has the characteristics of simple implementation, few adjustment parameters, a fast convergence rate, and solid robustness.\u003c/p\u003e\u003cp\u003eHowever, the traditional PSO algorithm has some inherent defects, which are mainly reflected in the problem of easily trapping into local optimum and premature convergence speed. In order to overcome these flaws, researchers have first tried to optimize the performance of PSO by improving the algorithm itself. They changed the relevant parameters\u003csup\u003e[\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e][\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e]\u003c/sup\u003e or topology structure\u003csup\u003e[\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e][\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e]\u003c/sup\u003e of PSO, optimized the particle learning strategy\u003csup\u003e[\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e][\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e]\u003c/sup\u003e, and mixed the PSO with other algorithms\u003csup\u003e[\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e][\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e]\u003c/sup\u003e to improve the efficiency of PSO. These methods all have achieved positive results, which efficiently improved the convergence speed and accuracy of PSO. Nonetheless, when facing large-scale and highly complex cyclic optimization problems, the convergence speed of the PSO will decrease dramatically as the number of particles increases, such that the efficiency of the optimization procedure is unsatisfactory.\u003c/p\u003e\u003cp\u003eDelving into the optimization process of the PSO algorithm, the particles are mutual independence and synergistic cooperation. So researchers consider that the parallelism of particles can be exploited to effectively accelerate the PSO. At present, there are three main strategies of PSO by parallel computation: 1) hardware environment-based parallel PSO algorithms, which are generally implemented through using hardware architectures\u003csup\u003e[\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e][\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e]\u003c/sup\u003e such as controllers\u003csup\u003e[\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e]\u003c/sup\u003e, Field Programmable Gate Array (FPGA)\u003csup\u003e[\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e][\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e]\u003c/sup\u003e, etc. to effectively improve the optimization speed of PSO algorithm, 2) CPU-based parallel PSO algorithms, which usually adopt multi-threading techniques\u003csup\u003e[\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e][\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e]\u003c/sup\u003e or multi-core processors\u003csup\u003e[\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e]\u003c/sup\u003e to express the independence of particles, making full use of parallelization to enhance the efficiency of PSO algorithm, 3) GPU-based parallel PSO algorithms, which flexibly use the architecture of GPU to synchronize the parallel optimization process of particles. However, each strategy has its advantages and disadvantages like the parallel hardware environment are accessible to deploy and implement but depends on the cluster node scale. And CPU-based multi-core parallelism only requires a PC but is greatly limited by the CPU structure and suffers from the complexity of the communication and management of inter-processes. At this time, applying the GPU with low hardware cost and strong computing power to realize the parallelism of PSO algorithm shows significant advantages.\u003c/p\u003e\u003cp\u003eWith the rapid promotion of GPU parallel calculating capability and the development of Compute Unified Device Architecture (CUDA), many procedures have been realized to take advantage of the availability, parallelism, and unified computing features of GPU to improve the program execution efficiency. Meanwhile, how to combine the advantages of GPU to accelerate the PSO execution has attracted some attention as mentioned above, and a large number of research results have been produced constantly to improve the speedup. However, in recent years, due to the traditional methods suffering from the numerous thread management of GPU and data IO costs, the speedup ratio of PSO has reached a bottleneck. Further research is turning to dig the GPU potential from multiple perspectives such as hardware, algorithms, problem scale et al.\u003c/p\u003e\u003cp\u003e1.1 Object and innovation\u003c/p\u003e\u003cp\u003eIn summary, while solving increasingly complex practical problems, how to better combine the independence of particle behavior with the parallel architecture of CUDA programming model is the key to achieving higher efficiency of the PSO algorithm. Accordingly, we propose a high-efficiency PSO (HEPSO) based on GPU initialization and thread adaption. Based on the asynchronous model of the PSO algorithm\u003csup\u003e[\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e]\u003c/sup\u003e, HEPSO regards the loop iteration behavior of each particle as an independent optimization process and adopts a coarse-grained parallel method to make the particles correspond to the threads one by one. At the same time, we optimize the PSO algorithm in GPU from the following two perspectives: (1) reduce unnecessary data IO by maximizing data throughput to optimize memory usage, (2) maximize parallel execution and leverage GPU multi-core advantages by designing a thread scheduling scheme to optimize thread usage. The main contributions of the paper are summarized as follows.\u003c/p\u003e\u003cp\u003e\u003col\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003eA high-efficiency PSO arithmetic (HEPSO) with GPU initialization and thread adaption strategy is designed by CUDA. This method can fully exploit the concurrency potential of the PSO and improve the running efficiency of the algorithm from two aspects of memory optimization and thread utilization.\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003eHEPSO migrates the data initialization of PSO from CPUs to GPUs to reduce unnecessary costs and employ thread adaption and multiplexing strategy to further improve the optimization speed.\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003eWe design some experiments to proven the proposed algorithm can remarkably enhance the execution efficiency of the PSO algorithm. The results show that compared with CPU-PSO, the \u0026ldquo;\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\text{S}}_{\\text{i}\\text{t}\\text{e}\\text{r}\\text{a}\\text{t}\\text{i}\\text{o}\\text{n}}\\)\u003c/span\u003e\u003c/span\u003e\u0026rdquo; of HEPSO can reach up to 580, and is 6 times faster than GPU-PSO. The overall average \u0026ldquo;\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\text{S}}_{\\text{c}\\text{o}\\text{n}\\text{v}\\text{e}\\text{r}\\text{g}\\text{e}\\text{n}\\text{c}\\text{e}}\\)\u003c/span\u003e\u003c/span\u003e\u0026rdquo; between the GPU-PSO and the HEPSO is more than 3 times.\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003c/ol\u003e\u003c/p\u003e\u003cp\u003eThe remaining paper is organized as follows. Section 2 presents the background on various PSO algorithms and CUDA computing architecture. Then Section 3 describes the detail of the proposed HEPSO algorithm and emphasizes its innovations. In section 4, several experiments are designed and implemented by 4 benchmarks to test the improvement in the efficiency of the HEPSO. Finally, we provide the summarization and discussion of our work in section 5.\u003c/p\u003e"},{"header":"2. Background","content":"\u003cdiv id=\"Sec2\" class=\"Section2\"\u003e \u003ch2\u003e2.1 Standard PSO\u003c/h2\u003e \u003cp\u003eThe PSO is a stochastic optimization algorithm that simulates group foraging activities and swarms intelligence\u003csup\u003e[\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e]\u003c/sup\u003e. It randomly generates particle swarms of a specific size as efficient solutions of the function in the problem space, and then iteratively searches to obtain an optimal result\u003csup\u003e[\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e]\u003c/sup\u003e. Each particle has its own velocity and position, while the value of the function corresponding to the position of the particle is the valid solution it finds\u003csup\u003e[\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e]\u003c/sup\u003e. In the process of iterative updating, the historical optimal solution of the individual and the global optimal solution of the swarm is used to calculate the initial velocity and position of the particle in the next iteration. The particle looks constantly in the search space until finding the global optimal solution or a feasible solution that satisfies the requirements. The way to update the particle velocity and position is as follows\u003cdiv id=\"Equ1\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ1\" name=\"EquationSource\"\u003e\n$${V}_{i}^{d+1} = \\omega {V}_{i}^{d} + {c}_{1}{r}_{1}\\left( {P}_{i}^{d} - {X}_{i}^{d}\\right) + {c}_{2}{r}_{2}\\left({P}_{g}^{d} - {X}_{i}^{d}\\right)$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e1\u003c/div\u003e\u003c/div\u003e\u003cdiv id=\"Equ2\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ2\" name=\"EquationSource\"\u003e\n$${X}_{i}^{d+1} = {X}_{i}^{d} + \\alpha {V}_{i}^{d+1}$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e2\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eIn which, i\u0026thinsp;=\u0026thinsp;1, 2\u0026hellip;, M, d\u0026thinsp;=\u0026thinsp;1, 2\u0026hellip;, N. \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\omega\\)\u003c/span\u003e\u003c/span\u003e is a non-negative number called the inertia factor, which plays a great role in the convergence effect of the algorithm. The larger its value, the wider the leap range of the particle. \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({P}_{i}^{}\\)\u003c/span\u003e\u003c/span\u003e is the local optimal position and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({P}_{g}^{}\\)\u003c/span\u003e\u003c/span\u003e is the global optimal position. The learning factors \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({c}_{1}\\)\u003c/span\u003e\u003c/span\u003e and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({c}_{2}\\)\u003c/span\u003e\u003c/span\u003e are non-negative constants, which are parameters for adjusting the weights of local and global optimal values. In practical application, the most appropriate parameter value could be found through multiple experimental adjustments. \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({r}_{1}\\)\u003c/span\u003e\u003c/span\u003e, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({r}_{2}\\)\u003c/span\u003e\u003c/span\u003e are arbitrary numbers in the range of [0, 1]. \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\alpha\\)\u003c/span\u003e\u003c/span\u003e is a constraint factor to control the weight of the velocity. In the implementation of standard PSO, the entire calculation of the algorithm is completed on the CPUs. Note that \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({X}_{i}^{d+1}\\)\u003c/span\u003e\u003c/span\u003e and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({V}_{i}^{d+1}\\)\u003c/span\u003e\u003c/span\u003e are the current position and velocity respectively, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({X}_{i}^{d}\\)\u003c/span\u003e\u003c/span\u003e and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({V}_{i}^{d}\\)\u003c/span\u003e\u003c/span\u003e are the previous position and velocity respectively.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003e2.2 Traditional GPU-based PSO\u003c/h2\u003e \u003cp\u003eIn recent years, many researchers have taken great efforts to improve the efficiency of PSO by combining the parallelism of the GPU architecture, and many achievements have been obtained. In spite of the attention, there was an endless challenge for improving efficiency of GPU-based PSO algorithms.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eR. M. Calazan et al.\u003csup\u003e[\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e]\u003c/sup\u003e proposed an algorithm Parallel Dimension PSO (PDPSO) based on the GPU architecture to massively parallelize PSO programs in which each particle is implemented as a block and each dimension is mapped into a distinct thread. Their experimental results are up to 85 times faster than the serial implementation on the CPU. You Zhou et al.\u003csup\u003e[\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e]\u003c/sup\u003e proposed a variational PSO algorithm, which makes particles evolve in a beneficial direction to accelerate the convergence speed of the process. They proved that the variant PSO algorithm has a more excellent running speed by several benchmark functions, and when the population size is larger, the program can at least achieve a speedup of 25 times after being accelerated by GPU. Cai Yong et al.\u003csup\u003e[\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e]\u003c/sup\u003e designed a coarse-grained synchronous parallel PSO algorithm, which is based on the coarse-grained parallelism idea of making threads correspond to particles one-to-one. This method creates a large number of threads to deserialize the particle search process and makes full use of various mathematical function calculation libraries from CUDA to ensure the reliability and usability of PSO. Through the experiments of three optimization functions, they demonstrate that compared with the traditional CPU-PSO, their algorithm is able to acquire a computational speedup of up to 90 times under the condition of function convergence. Xicheng Fu et al.\u003csup\u003e[\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e]\u003c/sup\u003e proposed a local PSOA based on medical image registration GPU. Their method has obvious advantages in the optimization of a high dimensional objective function, with the maximum acceleration ratio reaching 95 times. The total average difference in the number of iterations when the serial implementation and parallel implementation of the algorithm stop running under the premise of satisfying the accuracy is 17 times.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec4\" class=\"Section2\"\u003e \u003ch2\u003e2.3 Thread allocation strategy based on CUDA parallel architecture\u003c/h2\u003e \u003cp\u003eThe parallel computing model of CUDA adopts a multi-level memory architecture, which is divided into three levels: Thread, Block, and Grid. There are multiple threads and blocks in a block and a grid, respectively. In practice, Block is divided into smaller thread bundles (Warp), which is the basic unit of scheduling and execution for Streaming Multiprocessor (SM).\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eAt present, there are three main thread allocation schemes suitable for the CUDA parallel architecture model\u003csup\u003e[\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e]\u003c/sup\u003e: (1) coarse-grained parallelism where particles correspond to threads one by one\u003csup\u003e[\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e]\u003c/sup\u003e, (2) fine-grained parallelism where particles correspond to blocks one by one and the dimensions of the particles correspond to threads one by one\u003csup\u003e[\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e][\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e]\u003c/sup\u003e, (3) adaptive thread bundle parallelism where particles correspond to one or more warps, the dimensions of the particles correspond to threads one by one, and one or more particles correspond to block\u003csup\u003e[\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e]\u003c/sup\u003e. In a concrete implementation, by expanding the number of threads in each block as much as possible, and setting the block size to a multiple of warp, we can ensure the task balance among SMs and improve the running efficiency of the algorithm.\u003c/p\u003e \u003c/div\u003e"},{"header":"3. Methodology","content":"\u003cdiv id=\"Sec6\" class=\"Section2\"\u003e \u003ch2\u003e3.1 Algorithm flow\u003c/h2\u003e \u003cp\u003eThe algorithm flow of HEPSO is shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eThe steps of HEPSO are as follows.\u003c/p\u003e \u003cp\u003e(1) The CPUs initialize the relevant variables on the CPUs, and copy them to the GPUs by function cuda.to_device(),\u003c/p\u003e \u003cp\u003e(2) According to the problem scale and swarm size, the CPUs adaptively specify the execute configuration [GridDim, BlockDim],\u003c/p\u003e \u003cp\u003e(3) The CPUs invoke the kernel1() to initialize the particle swarm on the GPUs, and obtain initial velocity and position to calculate the particle local optimum and the swarm global optimum respectively,\u003c/p\u003e \u003cp\u003e(4) The CPUs invoke the kernel2() to cyclically update velocity and position on the GPUs to calculate the fitness value of the particle while gaining the particle local optimum and global swarm optimum through comparison,\u003c/p\u003e \u003cp\u003e(5) The GPU loops through step (4) until the end condition is met (reaching the target accuracy or the specified number of iterations),\u003c/p\u003e \u003cp\u003e(6) The GPUs terminate the loop and copy the solution of the global swarm optimum to CPUs by function cuda.copy_to_host().\u003c/p\u003e \u003cp\u003eAmong the above steps, steps (3) and (4) are the core of the HEPSO algorithm, the pseudo-code is as follow.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"No\" id=\"Taba\" border=\"1\"\u003e \u003ccolgroup cols=\"1\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAlgorithm 1 kernel1() : Initialization\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMap all the threads to S particles one-to-one\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e//Do operations to thread i (i\u0026thinsp;=\u0026thinsp;1,\u0026hellip;,S) synchronously:\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003efor\u003c/b\u003e i\u0026thinsp;=\u0026thinsp;1 to S \u003cb\u003edo\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003efor\u003c/b\u003e j\u0026thinsp;=\u0026thinsp;1 to N \u003cb\u003edo\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003einitialize\u003c/b\u003e the position Xi and velocity Vi of particle[i]\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003ecompute\u003c/b\u003e the local optimum PF[i] of particle[i]\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003ecompute\u003c/b\u003e the global optimum PB[i] of Swarm\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eIn Algorithm 1, as well as in the remainder of this paper, we denote by \u003cem\u003ePB[i]\u003c/em\u003e the best global fitness particle \u003cem\u003ei\u003c/em\u003e has achieved so far and \u003cem\u003ePBX[i]\u003c/em\u003e the coordinates of the position that yields it. As well, we denote \u003cem\u003ePF[i]\u003c/em\u003e as the best local fitness particle \u003cem\u003ei\u003c/em\u003e has achieved so far and \u003cem\u003ePFX[i]\u003c/em\u003e as the coordinates of the position that yields it. In addition, \u003cem\u003eS\u003c/em\u003e represents the particle swarm size and \u003cem\u003eN\u003c/em\u003e represents the problem size.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"No\" id=\"Tabb\" border=\"1\"\u003e \u003ccolgroup cols=\"1\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAlgorithm 2 kernel2() : Update\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMap all the threads to S particles one-to-one\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e//Do operations to thread i (i\u0026thinsp;=\u0026thinsp;1,\u0026hellip;,S) synchronously:\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003erepeat\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003efor\u003c/b\u003e i\u0026thinsp;=\u0026thinsp;1 to S \u003cb\u003edo\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003efor\u003c/b\u003e j\u0026thinsp;=\u0026thinsp;1 to N \u003cb\u003edo\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eupdate\u003c/b\u003e position Xi and velocity Vi of particle[i]\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003ecompute\u003c/b\u003e fitness F[i]\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eif\u003c/b\u003e F[i] \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\le\\)\u003c/span\u003e\u003c/span\u003e PF[i] \u003cb\u003ethen\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eupdate\u003c/b\u003e PF[i] and PFX[i] using position of particle[i]\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eIf\u003c/b\u003e PF[i] \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\le\\)\u003c/span\u003e\u003c/span\u003e PB[i] \u003cb\u003ethen\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eupdate\u003c/b\u003e PB[i] and PBX[i] using the PF[i] and PFX[i]\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eUntil\u003c/b\u003e reaching the stop conditions\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eReturn\u003c/b\u003e PB[i] and PBX[i]\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eDuring the execution of the iterative loop, the algorithm requires a large number of random numbers for updating velocity. We use the random number generation algorithm numba.cuda.random.create_xoroshiro128p_states() for the CUDA.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec7\" class=\"Section2\"\u003e \u003ch2\u003e3.2 GPU initialization\u003c/h2\u003e \u003cp\u003eWhen data is initialized, the traditional way is to perform the initialization operations on CPUs, and then copy the data to GPUs for further computation. However, this conventional IO process will waste a lot of invalid time.\u003c/p\u003e \u003cp\u003eBased on the experimental environment shown in Table \u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e, we test the time consumption by the initialization of the data on the CPUs and the GPUs respectively. The experimental results as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003ewe can notice that when the scale of the problem reaches the million level, one IO operation will cause an average time loss of 6 seconds, which greatly limits the execution efficiency of the algorithm. HEPSO we proposed only performs the necessary initialization on CPUs, including the population dimension and the number of particles. Meanwhile, all the rest of data initialization, such as velocity, position, and fitness by creating some empty arrays to take over, and the initialization of concrete assignments are done by GPUs. In this way, most of the initializations, all subsequent iterations, and comparisons of data can be wholly undertaken by the GPU core, without consuming IO resources.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec8\" class=\"Section2\"\u003e \u003ch2\u003e3.3 Thread self-adaptive strategy\u003c/h2\u003e \u003cp\u003eThe execute configuration (Kernel\u0026lt;\u0026lt;\u0026lt;GridDim, BlockDim\u0026gt;\u0026gt;\u0026gt;) are set by the kernel function which needs to be determined after debugging sessions in CUDA call time. In the Single-Instruction Multiple-Thread (SIMT) architecture used by SM, Warp is the smallest unit of scheduling and execution. If the number of threads contained in a block is not set properly, some inactive threads will remain in the underutilized warp and also consume SM resources. Therefore, in order to avoid wasting memory resources, the number of threads in a block is recommended as an integer multiple of the warp size. In our experiments, we proposed a thread-adaptive coarse-grained parallel algorithm to make full use of shared memory and maximize the degree of parallel execution. It can automatically adjust the parameters according to the specific particle numbers and problem dimensions when calling the kernel function to allow the particles to correspond with the threads one by one. The self-adaptive formula are as shown in Eq.\u0026nbsp;(\u003cspan refid=\"Equ3\" class=\"InternalRef\"\u003e3\u003c/span\u003e).\u003cdiv id=\"Equa\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equa\" name=\"EquationSource\"\u003e\n$$\\text{K}\\text{e}\\text{r}\\text{n}\\text{e}\\text{l} \\left( \\text{G}\\text{r}\\text{i}\\text{d}\\text{D}\\text{i}\\text{m}, \\text{B}\\text{l}\\text{o}\\text{c}\\text{k}\\text{D}\\text{i}\\text{m} \\right)=$$\u003c/div\u003e\u003c/div\u003e\u003cdiv id=\"Equ3\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ3\" name=\"EquationSource\"\u003e\n$$\\text{K}\\text{e}\\text{r}\\text{n}\\text{e}\\text{l} \\left( \\right(\\text{S} / \\text{N} / \\text{B}\\text{l}\\text{o}\\text{c}\\text{k}\\text{D}\\text{i}\\text{m} , \\text{N}), (\\text{n}, \\text{W}\\text{a}\\text{r}\\text{p}\\text{S}\\text{i}\\text{z}\\text{e}\\left) \\right)$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e3\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eAmong them, GridDim is the size of a grid, that is, the number of blocks contained in it; BlockDim is the size of a block, that is, the number of threads contained in it. We set the maximum size of BlockDim is set to 512. WarpSize represents the size of a warp, which is normally calculated by the GPU in HalfWarp units, so WarpSize is defined as 16. This self-adaptive strategy allows the kernel function to expand the number of threads in each block as much as possible and set the BlockDim to a multiple of warp\u003csup\u003e[\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e]\u003c/sup\u003e. It dynamically adjusts the number of threads to match both the problem scale and particle size to avoid the waste of memory resources and improve program performance.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec9\" class=\"Section2\"\u003e \u003ch2\u003e3.4 Thread multiplexing\u003c/h2\u003e \u003cp\u003eThere is overhead in the creation and destruction of threads in CUDA. In order to process a large amount of data, CUDA needs to create threads larger than the number of calculations. If each thread only performs an operation once and then is destroyed, the more data, the cost will be greater. For the sake of saving this overhead, we adopt the way of grid stride which adds a loop for each thread to realize thread multiplexing, so that a created thread can be used repeatedly. The grid stride is shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eIn Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e, the data size to process in parallel is 12. We create a grid that can start 4\u0026lt;\u0026lt;\u0026lt;2,2\u0026gt;\u0026gt;\u0026gt; threads in parallel and set the stride size to 4 after adding a loop. That means, the 1st, 5th, and 9th data can share thread 1. Such a procedure allows each thread to be utilized multiple times, which reduces the thread overhead of repeatedly starting and destroying and permits CUDA to parallelly handle large-scale problems.\u003c/p\u003e \u003c/div\u003e"},{"header":"4. Experiment","content":"\u003cdiv class=\"Section2\" id=\"Sec11\"\u003e\n \u003ch2\u003e4.1 Experimental environment\u003c/h2\u003e\n \u003cp\u003eThe computing environment based on our experiments is shown in Table \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e.\u003c/p\u003e\u0026nbsp;\u003ctable border=\"1\" id=\"Tab1\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eComputing environment\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eName\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eVersion\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eCPU\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eIntel(R) Core (TM) i7-6700 CPU @ 3.40GHz 3.41 GHz\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eGPU\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNVIDIA Tesla T4\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eOperating System\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eCentos7\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eDevelopment Environment\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAnaconda 3, CUDA 10.2\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n\u003c/div\u003e\n\u003cdiv class=\"Section2\" id=\"Sec12\"\u003e\n \u003ch2\u003e4.2 Optimization function\u003c/h2\u003e\n \u003cp\u003eIn practice, the performance of optimization algorithms is usually determined by the function evaluation value. In this paper, performance comparisons were conducted based on four benchmark test functions as listed in Table \u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003e.\u003c/p\u003e\u0026nbsp;\u003ctable border=\"1\" id=\"Tab2\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eBenchmark functions\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eF(x)\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eDomain\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eOptimum Solution\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({f}1\\left({{X}}_{{i}}\\right)= \\sum _{{i}=1}^{{n}-1}{(100\\left({{X}}_{{i}+1}- {{X}}_{{i}}^{2}\\right)}^{2}+ {({{X}}_{{i}}- 1)}^{2})\\)\u003c/span\u003e\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(-10.0\\le {{X}}_{{i}}\\le 10.0\\)\u003c/span\u003e\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({{f}}_{{m}{i}{n}}={f} (\\text{1,1},\\dots ,1) = 0\\)\u003c/span\u003e\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({f}2\\left({{X}}_{{i}}\\right)= \\sum _{{i}=1}^{{n}}{{X}}_{{i}}^{2}\\)\u003c/span\u003e\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(-100.0\\le {{X}}_{{i}}\\le 100.0\\)\u003c/span\u003e\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({{f}}_{{m}{i}{n}}={f} (\\text{0,0},\\dots ,0) = 0\\)\u003c/span\u003e\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({f}3\\left({{X}}_{{i}}\\right)=10{n}+ \\sum _{{i}=1}^{{n}}({{X}}_{{i}}^{2}- 10\\mathbf{c}\\mathbf{o}\\mathbf{s}\\left(2{\\pi }{{X}}_{{i}}\\right))\\)\u003c/span\u003e\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(-5.12\\le {{X}}_{{i}}\\le 5.12\\)\u003c/span\u003e\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({{f}}_{{m}{i}{n}}={f} (\\text{0,0},\\dots ,0) = 0\\)\u003c/span\u003e\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({f}4\\left({{X}}_{{i}}\\right)= \\sum _{{i}=1}^{{n}}\\frac{{{X}}_{{i}}^{2}}{4000} - \\prod _{{i}=1}^{{n}}\\mathbf{cos}\\left(\\frac{{{X}}_{{i}}}{\\sqrt{{i}}}\\right)+1\\)\u003c/span\u003e\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(-600.0\\le {{X}}_{{i}}\\le 600.0\\)\u003c/span\u003e\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({{f}}_{{m}{i}{n}}={f} (\\text{0,0},\\dots ,0) = 0\\)\u003c/span\u003e\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003cp\u003e\u003cbr\u003e\u003c/p\u003e\n \u003cp\u003eThe inertia weight \u0026omega; was taken as 0.32, the learning factors C1 and C2 were both taken as 2.0, and the constraint factor \u0026alpha; was taken as 0.5.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv class=\"Section2\" id=\"Sec13\"\u003e\n \u003ch2\u003e4.3 Evaluation assessment\u003c/h2\u003e\n \u003cp\u003eTo quantitate access to the performance of different algorithms, an evaluation index is required. In this subsection, two index computing methods are introduced. They are all based on the time cost but with different finish conditions.\u003c/p\u003e\n \u003cdiv class=\"Section3\" id=\"Sec14\"\u003e\n \u003ch2\u003e4.3.1 The assessment of time-ratio to stop iterating\u003c/h2\u003e\n \u003cp\u003eThe very first one is the time ratio to stop iterating which indicates the time difference between different algorithms under the condition of the same iterations. It can evaluate the pure computational capability of an algorithm.\u003c/p\u003e\n \u003cp\u003eUnder the condition that the numbers of particles and iterations are specified to be the same, we define \u0026quot;\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\text{T}}_{\\text{R}\\text{e}\\text{f}}\\)\u003c/span\u003e\u003c/span\u003e\u0026quot; and \u0026quot;\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\text{T}}_{\\text{O}\\text{b}\\text{j}}\\)\u003c/span\u003e\u003c/span\u003e\u0026quot; as the running time of reference and object algorithm programs respectively. The time ratio to stop iteration index is \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\text{\u0026ldquo;}{\\text{S}}_{\\text{i}\\text{t}\\text{e}\\text{r}\\text{a}\\text{t}\\text{i}\\text{o}\\text{n}}\\text{\u0026rdquo;}\\)\u003c/span\u003e\u003c/span\u003e is set as follows.\u003c/p\u003e\n \u003cdiv class=\"Equation\" id=\"Equ4\"\u003e\n \u003cdiv class=\"mathdisplay\" id=\"FileID_Equ4\" name=\"EquationSource\"\u003e$${\\text{S}}_{\\text{i}\\text{t}\\text{e}\\text{r}\\text{a}\\text{t}\\text{i}\\text{o}\\text{n}}= \\frac{{\\text{T}}_{\\text{R}\\text{e}\\text{f}}}{{\\text{T}}_{\\text{O}\\text{b}\\text{j}}}$$\u003c/div\u003e\n \u003cdiv class=\"EquationNumber\"\u003e4\u003c/div\u003e\n \u003c/div\u003e\n \u003c/div\u003e\n \u003cdiv class=\"Section3\" id=\"Sec15\"\u003e\n \u003ch2\u003e4.3.2 The assessment of time-ratio after functions convergence\u003c/h2\u003e\n \u003cp\u003eOn the other hand, the above assessment index ignores the convergence difference within the optimization solver which may turn up a condition of one party stopping iterating while the other party still running. In consequence, we consider testing the overall average difference in execution time after functions convergence between different algorithms.\u003c/p\u003e\n \u003cp\u003eWe define the \u0026quot;\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\text{min}\\left({ \\text{T}}_{\\text{R}\\text{e}\\text{f}} \\right)\\)\u003c/span\u003e\u003c/span\u003e\u0026quot; and \u0026quot;\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({ \\text{m}\\text{i}\\text{n}(\\text{T}}_{\\text{O}\\text{b}\\text{j} })\\)\u003c/span\u003e\u003c/span\u003e\u0026quot; as the minimum running time under the condition of function converges to a particular precision of reference and object algorithm, respectively. This index \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\text{\u0026ldquo;}{\\text{S}}_{\\text{c}\\text{o}\\text{n}\\text{v}\\text{e}\\text{r}\\text{g}\\text{e}\\text{n}\\text{c}\\text{e}}\\text{\u0026rdquo;}\\)\u003c/span\u003e\u003c/span\u003e is set as follows.\u003c/p\u003e\n \u003cdiv class=\"Equation\" id=\"Equ5\"\u003e\n \u003cdiv class=\"mathdisplay\" id=\"FileID_Equ5\" name=\"EquationSource\"\u003e$${\\text{S}}_{\\text{c}\\text{o}\\text{n}\\text{v}\\text{e}\\text{r}\\text{g}\\text{e}\\text{n}\\text{c}\\text{e}}= \\frac{\\text{m}\\text{i}\\text{n}\\left({ \\text{T}}_{\\text{R}\\text{e}\\text{f}} \\right)}{{\\text{m}\\text{i}\\text{n}（ \\text{T}}_{\\text{O}\\text{b}\\text{j} }）}$$\u003c/div\u003e\n \u003cdiv class=\"EquationNumber\"\u003e5\u003c/div\u003e\n \u003c/div\u003e\n \u003c/div\u003e\n\u003c/div\u003e\n\u003cdiv class=\"Section2\" id=\"Sec16\"\u003e\n \u003ch2\u003e4.4 Experimental results\u003c/h2\u003e\n \u003cdiv class=\"Section3\" id=\"Sec17\"\u003e\n \u003ch2\u003e4.4.1 time-ratio to stop iterating\u003c/h2\u003e\n \u003cp\u003eThe initial number of particles and iterations were set to 128 and 10000 respectively, and the dimension N of all four benchmark functions was 16. In the experiments, we increased the number of particles and dynamically decreased the number of iterations to reduce the execution time on CPUs. The number of particles and iterations were respectively range from 128 to 131072 and from 10000 to 200. Each experiment was run until the maximum number of iterations has been reached. The optimization were repeated 20 times with different seeds. The average results are shown in Fig. \u003cspan class=\"InternalRef\"\u003e6\u003c/span\u003e. \u0026ndash; Fig. \u003cspan class=\"InternalRef\"\u003e7\u003c/span\u003e. and Table \u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e - Table \u003cspan class=\"InternalRef\"\u003e6\u003c/span\u003e in the appendix\u003csup\u003e[\u003cspan class=\"CitationRef\"\u003e10\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e11\u003c/span\u003e]\u003c/sup\u003e respectively.\u003c/p\u003e\n \u003ctable border=\"1\" id=\"Tab3\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eExecution time and \u0026ldquo;\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\text{S}}_{\\text{i}\\text{t}\\text{e}\\text{r}\\text{a}\\text{t}\\text{i}\\text{o}\\text{n}}\\)\u003c/span\u003e\u003c/span\u003e\u0026rdquo; of f1\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\" rowspan=\"2\" style=\"width: 6.7508%;\"\u003e\n \u003cp\u003eSwarmsize\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" rowspan=\"2\" style=\"width: 6.1526%;\"\u003e\n \u003cp\u003eIterations\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" colspan=\"3\" style=\"width: 15.0398%;\"\u003e\n \u003cp\u003eExecution Time/s\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" colspan=\"2\" style=\"width: 59.0738%;\"\u003e\n \u003cp\u003e\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\mathbf{S}}_{\\mathbf{i}\\mathbf{t}\\mathbf{e}\\mathbf{r}\\mathbf{a}\\mathbf{t}\\mathbf{i}\\mathbf{o}\\mathbf{n}}\\)\u003c/span\u003e\u003c/span\u003e\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003cth align=\"left\" style=\"width: 5.5545%;\"\u003e\n \u003cp\u003eCPU\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" style=\"width: 4.1872%;\"\u003e\n \u003cp\u003eGPU\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" style=\"width: 5.0417%;\"\u003e\n \u003cp\u003eHEPSO\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" style=\"width: 32.1304%;\"\u003e\n \u003cp\u003eCPU/HEPSO\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" style=\"width: 32.3868%;\"\u003e\n \u003cp\u003eGPU/HEPSO\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 6.7508%;\"\u003e\n \u003cp\u003e128\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 6.1526%;\"\u003e\n \u003cp\u003e10000\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 5.5545%;\"\u003e\n \u003cp\u003e49.951\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 4.1872%;\"\u003e\n \u003cp\u003e16.988\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 5.0417%;\"\u003e\n \u003cp\u003e4.547\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 32.1304%;\"\u003e\n \u003cp\u003e10.99\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 32.3868%;\"\u003e\n \u003cp\u003e3.74\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 6.7508%;\"\u003e\n \u003cp\u003e256\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 6.1526%;\"\u003e\n \u003cp\u003e9000\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 5.5545%;\"\u003e\n \u003cp\u003e93.823\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 4.1872%;\"\u003e\n \u003cp\u003e15.789\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 5.0417%;\"\u003e\n \u003cp\u003e4.674\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 32.1304%;\"\u003e\n \u003cp\u003e20.07\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 32.3868%;\"\u003e\n \u003cp\u003e3.38\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 6.7508%;\"\u003e\n \u003cp\u003e512\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 6.1526%;\"\u003e\n \u003cp\u003e7500\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 5.5545%;\"\u003e\n \u003cp\u003e154.924\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 4.1872%;\"\u003e\n \u003cp\u003e14.183\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 5.0417%;\"\u003e\n \u003cp\u003e4.625\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 32.1304%;\"\u003e\n \u003cp\u003e33.49\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 32.3868%;\"\u003e\n \u003cp\u003e3.07\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 6.7508%;\"\u003e\n \u003cp\u003e1024\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 6.1526%;\"\u003e\n \u003cp\u003e6000\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 5.5545%;\"\u003e\n \u003cp\u003e245.898\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 4.1872%;\"\u003e\n \u003cp\u003e12.409\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 5.0417%;\"\u003e\n \u003cp\u003e4.003\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 32.1304%;\"\u003e\n \u003cp\u003e61.43\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 32.3868%;\"\u003e\n \u003cp\u003e3.10\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 6.7508%;\"\u003e\n \u003cp\u003e2048\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 6.1526%;\"\u003e\n \u003cp\u003e5000\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 5.5545%;\"\u003e\n \u003cp\u003e414.184\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 4.1872%;\"\u003e\n \u003cp\u003e12.149\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 5.0417%;\"\u003e\n \u003cp\u003e4.150\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 32.1304%;\"\u003e\n \u003cp\u003e99.80\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 32.3868%;\"\u003e\n \u003cp\u003e2.93\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 6.7508%;\"\u003e\n \u003cp\u003e4096\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 6.1526%;\"\u003e\n \u003cp\u003e3000\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 5.5545%;\"\u003e\n \u003cp\u003e501.445\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 4.1872%;\"\u003e\n \u003cp\u003e10.190\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 5.0417%;\"\u003e\n \u003cp\u003e3.757\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 32.1304%;\"\u003e\n \u003cp\u003e133.48\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 32.3868%;\"\u003e\n \u003cp\u003e2.71\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 6.7508%;\"\u003e\n \u003cp\u003e8192\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 6.1526%;\"\u003e\n \u003cp\u003e2000\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 5.5545%;\"\u003e\n \u003cp\u003e640.905\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 4.1872%;\"\u003e\n \u003cp\u003e9.168\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 5.0417%;\"\u003e\n \u003cp\u003e4.073\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 32.1304%;\"\u003e\n \u003cp\u003e157.37\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 32.3868%;\"\u003e\n \u003cp\u003e2.25\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 6.7508%;\"\u003e\n \u003cp\u003e16384\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 6.1526%;\"\u003e\n \u003cp\u003e1200\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 5.5545%;\"\u003e\n \u003cp\u003e792.098\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 4.1872%;\"\u003e\n \u003cp\u003e8.093\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 5.0417%;\"\u003e\n \u003cp\u003e2.860\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 32.1304%;\"\u003e\n \u003cp\u003e276.92\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 32.3868%;\"\u003e\n \u003cp\u003e2.83\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 6.7508%;\"\u003e\n \u003cp\u003e32768\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 6.1526%;\"\u003e\n \u003cp\u003e700\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 5.5545%;\"\u003e\n \u003cp\u003e899.922\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 4.1872%;\"\u003e\n \u003cp\u003e7.989\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 5.0417%;\"\u003e\n \u003cp\u003e2.096\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 32.1304%;\"\u003e\n \u003cp\u003e429.27\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 32.3868%;\"\u003e\n \u003cp\u003e3.81\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 6.7508%;\"\u003e\n \u003cp\u003e65536\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 6.1526%;\"\u003e\n \u003cp\u003e380\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 5.5545%;\"\u003e\n \u003cp\u003e998.583\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 4.1872%;\"\u003e\n \u003cp\u003e8.903\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 5.0417%;\"\u003e\n \u003cp\u003e2.023\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 32.1304%;\"\u003e\n \u003cp\u003e493.72\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 32.3868%;\"\u003e\n \u003cp\u003e4.40\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 6.7508%;\"\u003e\n \u003cp\u003e131072\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 6.1526%;\"\u003e\n \u003cp\u003e200\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 5.5545%;\"\u003e\n \u003cp\u003e1018.983\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 4.1872%;\"\u003e\n \u003cp\u003e11.666\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 5.0417%;\"\u003e\n \u003cp\u003e1.746\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 32.1304%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e583.66\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" style=\"width: 32.3868%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e6.68\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003ctable border=\"1\" id=\"Tab4\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 4\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eExecution time and \u0026ldquo;\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\text{S}}_{\\text{i}\\text{t}\\text{e}\\text{r}\\text{a}\\text{t}\\text{i}\\text{o}\\text{n}}\\)\u003c/span\u003e\u003c/span\u003e\u0026rdquo; of f2\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\" rowspan=\"2\"\u003e\n \u003cp\u003eSwarmsize\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" rowspan=\"2\"\u003e\n \u003cp\u003eIterations\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" colspan=\"3\"\u003e\n \u003cp\u003eExecution Time/s\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\mathbf{S}}_{\\mathbf{i}\\mathbf{t}\\mathbf{e}\\mathbf{r}\\mathbf{a}\\mathbf{t}\\mathbf{i}\\mathbf{o}\\mathbf{n}}\\)\u003c/span\u003e\u003c/span\u003e\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eCPU\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eGPU\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eHEPSO\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eCPU/HEPSO\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eGPU/HEPSO\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e128\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e10000\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e16.726\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e16.771\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e4.122\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e4.06\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e4.07\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e256\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e9000\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e29.223\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e15.371\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e4.022\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e7.27\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e3.82\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e512\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e7500\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e49.287\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e13.509\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e3.862\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e12.76\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e3.50\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e1024\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e6000\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e76.568\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e11.722\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e3.725\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e20.55\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e3.15\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2048\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e5000\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e127.971\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e11.284\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e3.482\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e36.75\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e3.24\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4096\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e3000\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e154.396\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e9.116\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.788\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e55.38\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e3.27\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e8192\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2000\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e204.227\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e8.777\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.671\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e76.47\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e3.29\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e16384\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1200\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e249.169\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e7.555\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.025\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e123.03\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e3.73\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e32768\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e700\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e287.210\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e6.548\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.533\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e187.35\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e4.27\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e65536\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e380\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e317.406\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e6.584\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.537\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e206.49\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e4.28\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e131072\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e200\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e334.853\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e7.511\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.485\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e\u003cstrong\u003e225.44\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e\u003cstrong\u003e5.06\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003ctable border=\"1\" id=\"Tab5\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 5\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eExecution time and \u0026ldquo;\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\text{S}}_{\\text{i}\\text{t}\\text{e}\\text{r}\\text{a}\\text{t}\\text{i}\\text{o}\\text{n}}\\)\u003c/span\u003e\u003c/span\u003e\u0026rdquo; of f3\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\" rowspan=\"2\"\u003e\n \u003cp\u003eSwarmsize\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" rowspan=\"2\"\u003e\n \u003cp\u003eIterations\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" colspan=\"3\"\u003e\n \u003cp\u003eExecution Time/s\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\mathbf{S}}_{\\mathbf{i}\\mathbf{t}\\mathbf{e}\\mathbf{r}\\mathbf{a}\\mathbf{t}\\mathbf{i}\\mathbf{o}\\mathbf{n}}\\)\u003c/span\u003e\u003c/span\u003e\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" colspan=\"1\"\u003e\u0026nbsp;\u003c/th\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eCPU\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eGPU\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eHEPSO\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eCPU/HEPSO\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003eGPU/HEPSO\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e128\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e10000\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e21.692\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e16.958\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e5.615\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.86\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e3.02\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e256\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e9000\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e38.740\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e15.977\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e5.374\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e7.21\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e2.97\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e512\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e7500\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e62.691\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e14.707\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e5.278\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e11.88\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e2.79\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e1024\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e6000\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e100.296\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e13.686\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e5.899\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e17.00\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e2.32\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2048\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e5000\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e169.697\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e14.410\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e7.718\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e21.99\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e1.87\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4096\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3000\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e200.803\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e10.991\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e8.142\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e24.66\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e1.35\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e8192\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2000\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e268.514\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e9.401\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e10.024\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e26.79\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e0.94\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e16384\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e1200\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e322.508\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e9.465\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e6.443\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e50.05\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e1.47\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e32768\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e700\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e379.388\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e9.337\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.107\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e92.38\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e2.27\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e65536\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e380\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e420.789\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e10.006\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2.768\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e152.00\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e3.61\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e131072\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e200\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e444.381\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e10.892\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2.807\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003e158.30\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e\u003cstrong\u003e3.88\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003ctable border=\"1\" id=\"Tab6\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 6\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eExecution time and \u0026ldquo;\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\text{S}}_{\\text{i}\\text{t}\\text{e}\\text{r}\\text{a}\\text{t}\\text{i}\\text{o}\\text{n}}\\)\u003c/span\u003e\u003c/span\u003e\u0026rdquo; of f4\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\" rowspan=\"2\"\u003e\n \u003cp\u003eSwarmsize\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" rowspan=\"2\"\u003e\n \u003cp\u003eIterations\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" colspan=\"3\"\u003e\n \u003cp\u003eExecution Time/s\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\mathbf{S}}_{\\mathbf{i}\\mathbf{t}\\mathbf{e}\\mathbf{r}\\mathbf{a}\\mathbf{t}\\mathbf{i}\\mathbf{o}\\mathbf{n}}\\)\u003c/span\u003e\u003c/span\u003e\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eCPU\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eGPU\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eHEPSO\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eCPU/HEPSO\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eGPU/HEPSO\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e128\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e10000\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e32.419\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e17.266\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e5.906\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e5.49\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2.92\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e256\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e9000\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e56.345\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e16.707\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e6.261\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e8.99\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2.67\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e512\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e7500\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e93.533\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e15.741\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e7.373\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e12.69\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2.13\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e1024\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e6000\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e151.712\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e15.508\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e9.803\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e15.48\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e1.58\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2048\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e5000\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e253.252\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e15.085\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e14.304\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e17.70\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e1.05\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4096\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3000\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e300.272\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e10.960\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e15.499\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e19.37\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.71\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e8192\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2000\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e387.161\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e11.188\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e19.817\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e19.54\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.56\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e16384\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e1200\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e470.528\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e11.740\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e11.640\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e40.42\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e1.01\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e32768\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e700\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e549.339\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e12.207\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e7.097\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e77.40\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e1.72\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e65536\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e380\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e614.538\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e13.327\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.360\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e140.94\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.06\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e131072\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e200\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e649.055\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e15.094\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.408\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003e147.25\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003e3.42\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003cp\u003eThe \u0026ldquo;\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\text{S}}_{\\text{i}\\text{t}\\text{e}\\text{r}\\text{a}\\text{t}\\text{i}\\text{o}\\text{n}}\\)\u003c/span\u003e\u003c/span\u003e\u0026rdquo; of 4 benchmark functions are shown in Fig. \u003cspan class=\"InternalRef\"\u003e5\u003c/span\u003e.\u003c/p\u003e\n \u003cp\u003eThe analysis of the experimental results in Fig. \u003cspan class=\"InternalRef\"\u003e6\u003c/span\u003e. \u0026ndash; Fig. \u003cspan class=\"InternalRef\"\u003e7\u003c/span\u003e. and Table \u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e - Table \u003cspan class=\"InternalRef\"\u003e6\u003c/span\u003e in the appendix\u003csup\u003e[\u003cspan class=\"CitationRef\"\u003e10\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e11\u003c/span\u003e]\u003c/sup\u003e shows that the HEPSO algorithm on f1, f2, f3, and f4 has achieved the maximum \u0026ldquo;\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\text{S}}_{\\text{i}\\text{t}\\text{e}\\text{r}\\text{a}\\text{t}\\text{i}\\text{o}\\text{n}}\\)\u003c/span\u003e\u003c/span\u003e\u0026rdquo; of 583.6, 225.4, 158.3, and 147.2 compared to CPU-PSO. Meanwhile, for HEPSO to GPU-PSO, it achieved the maximum \u0026ldquo;\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\text{S}}_{\\text{i}\\text{t}\\text{e}\\text{r}\\text{a}\\text{t}\\text{i}\\text{o}\\text{n}}\\)\u003c/span\u003e\u003c/span\u003e\u0026rdquo; of 6.68, 5.06, 3.88, and 3.42. Several conclusions can be drawn as follows.\u003c/p\u003e\n \u003cp\u003e\u003cbr\u003e\u003c/p\u003e\n \u003col\u003e\n \u003cli\u003e\n \u003cp\u003eFrom the above experiments, we can conclude that the PSO executed on the CPU is only suitable for a simple functions optimization problem (the number of particles is small, usually below 300). When facing a more complex and higher dimensional problem, with requires particle number enlarging, the load on the CPU expands accordingly and the optimization time increases significantly.\u003c/p\u003e\n \u003c/li\u003e\n \u003cli\u003e\n \u003cp\u003eAs a whole, the execution time of HEPSO is less than GPU-PSO. When the number of particles is greater than 16384, the time consumption difference represents a rapidly widening trend. With the support of the parallel ability of GPU, the computation time is shorter with particle number. That means one can deploy more particles to accelerate the optimization search procedure.\u003c/p\u003e\n \u003c/li\u003e\n \u003cli\u003e\n \u003cp\u003eFor complex functions f3 and f4, when the number of particles is less than 8192, the execution time of GPU-PSO keeps decreasing while the execution time of HEPSO keeps increasing, which is because the advantage of local parallelism outweighs the time required for thread synchronization. When the number of particles increases to more than 8192, the benefits brought by local parallelism gradually fails to offset the large IO loss. That causes a significant decrease in the efficiency of GPU-PSO to about 1/2. This indicates that the HEPSO we proposed needs to generate a large number of particles to accelerate the optimization process with more complicated functions.\u003c/p\u003e\n \u003c/li\u003e\n \u003cli\u003e\n \u003cp\u003eAmong several test functions, HEPSO has achieved the best speedup in the f1. It has achieved 580 times \u0026ldquo;\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\text{S}}_{\\text{i}\\text{t}\\text{e}\\text{r}\\text{a}\\text{t}\\text{i}\\text{o}\\text{n}}\\)\u003c/span\u003e\u003c/span\u003e\u0026rdquo; compared with CPU-PSO and has also achieved 6 times \u0026ldquo;\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\text{S}}_{\\text{i}\\text{t}\\text{e}\\text{r}\\text{a}\\text{t}\\text{i}\\text{o}\\text{n}}\\)\u003c/span\u003e\u003c/span\u003e\u0026rdquo; compared with GPU-PSO. Therefore, HEPSO can be proven to be suitable for solving large-scale optimization problems with complex functions.\u003c/p\u003e\n \u003c/li\u003e\n \u003c/ol\u003e\n \u003cp\u003e\u003cbr\u003e\u003c/p\u003e\n \u003c/div\u003e\n \u003cdiv class=\"Section3\" id=\"Sec18\"\u003e\n \u003ch2\u003e4.4.2 time-ratio after functions convergence\u003c/h2\u003e\n \u003cp\u003eConsidering the problem mentioned in 4.3.2, we design another experiment with the iteration termination condition set to be the convergence error of 10\u003csup\u003e\u0026minus;\u0026thinsp;4\u003c/sup\u003e. In this experiment, the dimension N of all four benchmark functions is 16. The optimization were repeated 50 times with different seeds. The average results we obtained are shown in Table \u003cspan class=\"InternalRef\"\u003e7\u003c/span\u003e.\u003c/p\u003e\n \u003ctable border=\"1\" id=\"Tab7\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 7\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003e\u0026ldquo;\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\text{S}}_{\\text{c}\\text{o}\\text{n}\\text{v}\\text{e}\\text{r}\\text{g}\\text{e}\\text{n}\\text{c}\\text{e}}\\)\u003c/span\u003e\u003c/span\u003e\u0026rdquo; of f1, f2, f3, and f4\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eF(x)\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eSwarmsize\u003c/p\u003e\n \u003cp\u003e(GPU)\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eSwarmsize\u003c/p\u003e\n \u003cp\u003e(HEPSO)\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eAccuracy(GPU)\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eAccuracy\u003c/p\u003e\n \u003cp\u003e(HEPSO)\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eGPU-time\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eHEPSO-time\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003e\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\mathbf{S}}_{\\mathbf{c}\\mathbf{o}\\mathbf{n}\\mathbf{v}\\mathbf{e}\\mathbf{r}\\mathbf{g}\\mathbf{e}\\mathbf{n}\\mathbf{c}\\mathbf{e}}\\)\u003c/span\u003e\u003c/span\u003e\u003c/p\u003e\n \u003cp\u003e(GPU/HEPSO)\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003ef1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e65536\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e524288\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e8.2E-05\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2.9E-05\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e8.57\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.25\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e\u003cstrong\u003e3.81\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003ef2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e32768\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e65536\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e7E-05\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e9.5E-05\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e3.53\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.32\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.67\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003ef3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e32768\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e65536\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e9E-05\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e1.4E-05\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e3.91\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.67\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.34\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003ef4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e32768\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e131072\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2.2E-06\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e1.3E-06\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e4.51\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.24\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.01\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003cp\u003eFrom the data in Table \u003cspan class=\"InternalRef\"\u003e7\u003c/span\u003e, it can be seen that the HEPSO achieved the maximum \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\text{\u0026ldquo;}{\\text{S}}_{\\text{c}\\text{o}\\text{n}\\text{v}\\text{e}\\text{r}\\text{g}\\text{e}\\text{n}\\text{c}\\text{e}}\\text{\u0026rdquo;}\\)\u003c/span\u003e\u003c/span\u003e of 3.81, 2.67, 2.34, and 2.01 for the 4 benchmark functions of f1, f2, f3, and f4. Compared with the above \u0026ldquo;\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\text{S}}_{\\text{i}\\text{t}\\text{e}\\text{r}\\text{a}\\text{t}\\text{i}\\text{o}\\text{n}}\\)\u003c/span\u003e\u003c/span\u003e\u0026rdquo;, the \u0026ldquo;\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\text{S}}_{\\text{c}\\text{o}\\text{n}\\text{v}\\text{e}\\text{r}\\text{g}\\text{e}\\text{n}\\text{c}\\text{e}}\\)\u003c/span\u003e\u003c/span\u003e\u0026rdquo; is more in line with the actual application situation.\u003c/p\u003e\n \u003c/div\u003e\n\u003c/div\u003e"},{"header":"5. Conclusion","content":"\u003cp\u003eThis paper proposes a high-efficiency PSO arithmetic based on GPU initialization and thread self-adaptive. Firstly, the initialization of the particle process in the PSO is migrated from CPUs to GPUs, which enormously improves the computational efficiency of the algorithm by reducing its IO cost. In addition, two strategies of thread adaption and thread multiplexing strategy are added to enhance the efficiency of PSO.\u003c/p\u003e \u003cp\u003eIn this paper, with 4 benchmark functions, HEPSO is tested by calculating two sets of evaluation indexes: \u0026ldquo;\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\text{S}}_{\\text{i}\\text{t}\\text{e}\\text{r}\\text{a}\\text{t}\\text{i}\\text{o}\\text{n}}\\)\u003c/span\u003e\u003c/span\u003e\u0026rdquo; and \u0026ldquo;\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\text{S}}_{\\text{c}\\text{o}\\text{n}\\text{v}\\text{e}\\text{r}\\text{g}\\text{e}\\text{n}\\text{c}\\text{e}}\\)\u003c/span\u003e\u003c/span\u003e\u0026rdquo;. Among them, (1) The speedup of CPU-PSO, GPU-PSO, and HEPSO is quantitatively tested and analyzed at different particle numbers and iteration times. The results show that when the number of particles reaches 131072, HEPSO can achieve 580-times \u0026ldquo;\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\text{S}}_{\\text{i}\\text{t}\\text{e}\\text{r}\\text{a}\\text{t}\\text{i}\\text{o}\\text{n}}\\)\u003c/span\u003e\u003c/span\u003e\u0026rdquo; compared to the CPU-PSO, and over 6 times higher than GPU-PSO. (2) HEPSO was tested from the perspective of practical application by contrasting the overall average difference of time after functions convergence. In all 4 related tests, the HEPSO obtains a \u0026ldquo;\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\text{S}}_{\\text{c}\\text{o}\\text{n}\\text{v}\\text{e}\\text{r}\\text{g}\\text{e}\\text{n}\\text{c}\\text{e}}\\)\u003c/span\u003e\u003c/span\u003e\u0026rdquo; of more than 3 times. From the theoretical and experimental analysis, our HEPSO shows many significant advantages. Compared with other methods, especially in large-scale and complex optimization problems, the efficiency of PSO is improved to a higher level. This can help PSO overcome the efficiency limitations to face practical application challenges.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eDeclaration of competing interest\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAcknowledgments\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis work was supported bythe Natural Science Foundation of Shaanxi Province ( 2022JM-301), and the \u0026nbsp;Postgraduate Innovation and Practice Ability Development Fund of Xi\u0026rsquo;an Shiyou University.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthor contributions\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe corresponding author Ye Liu conceived and designed the work that led to the submission, is responsible for the crucial parts in interpreting the results.\u003c/p\u003e\n\u003cp\u003eJia Wu and Hui Ren process the dataset and help to analysis the basic data.\u003c/p\u003e\n\u003cp\u003eShuopeng Yang, Fuqiang Zhang, and Jie Cao process the data and revised the manscript.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eWang D, Tan D, Liu L. Particle swarm optimization algorithm: an overview, J. Soft computing, 2018, 22(2): 387-408. \u003c/li\u003e\n\u003cli\u003eChen K, Zhou F, Liu A. Chaotic dynamic weight particle swarm optimization for numerical function optimization, J. Knowledge-Based Systems, 2018, 139: 23-40.\u003c/li\u003e\n\u003cli\u003eGoudarzi A, Li Y, Xiang J. A hybrid non-linear time-varying double-weighted particle swarm optimization for solving non-convex combined environmental economic dispatch problem, J. Applied Soft Computing, 2020, 86: 105894.\u003c/li\u003e\n\u003cli\u003eLim W H, Isa N A M. Particle swarm optimization with increasing topology connectivity, J. Engineering Applications of Artificial Intelligence, 2014, 27: 80-102.\u003c/li\u003e\n\u003cli\u003eLin A, Sun W, Yu H, et al. Global genetic learning particle swarm optimization with diversity enhancement by ring topology, J. Swarm and evolutionary computation, 2019, 44: 571-583.\u003c/li\u003e\n\u003cli\u003eLynn N, Suganthan P N. Heterogeneous comprehensive learning particle swarm optimization with enhanced exploration and exploitation, J. Swarm and Evolutionary Computation, 2015, 24: 11-24.\u003c/li\u003e\n\u003cli\u003eXu G, Cui Q, Shi X, et al. Particle swarm optimization based on dimensional learning strategy, J. Swarm and Evolutionary Computation, 2019, 45: 33-51.\u003c/li\u003e\n\u003cli\u003eJindal V, Bedi P. An improved hybrid ant particle optimization (IHAPO) algorithm for reducing travel time in VANETs, J. Applied Soft Computing, 2018, 64: 526-535.\u003c/li\u003e\n\u003cli\u003eLaskar N M, Guha K, Chatterjee I, et al. HWPSO: A new hybrid whale-particle swarm optimization algorithm and its application in electronic design optimization problems, J. Applied Intelligence, 2019, 49: 265-291.\u003c/li\u003e\n\u003cli\u003eTewolde G S, Hanna D M, Haskell R E. Multi-swarm parallel PSO: Hardware implementation, C. 2009 IEEE Swarm Intelligence Symposium. IEEE, 2009: 60-66.\u003c/li\u003e\n\u003cli\u003eDamaj I, Elshafei M, El-Abd M, et al. An analytical framework for high-speed hardware particle swarm optimization, J. Microprocessors and Microsystems, 2020, 72: 102949. \u003c/li\u003e\n\u003cli\u003eSuzuki R, Kawai F, Nakazawa C, et al. Parameter optimization of model predictive control by PSO, J. Electrical Engineering in Japan, 2012, 178(1): 40-49. \u003c/li\u003e\n\u003cli\u003eDa Costa A L X, Silva C A D, Torquato M F, et al. Parallel implementation of particle swarm optimization on FPGA, J. IEEE Transactions on Circuits and Systems II: Express Briefs, 2019, 66(11): 1875-1879. \u003c/li\u003e\n\u003cli\u003eZhao Y, Yu X, Wu H, et al. A Fast 2-D Otsu lung tissue image segmentation algorithm based on improved PSO, J. Microprocessors and Microsystems, 2021, 80: 103527.\u003c/li\u003e\n\u003cli\u003eMartinez-Rios F, Murillo-Suarez A. A new swarm algorithm for global optimization of multimodal functions over multi-threading architecture hybridized with simulating annealing, J. Procedia Computer Science, 2018, 135: 449-456. \u003c/li\u003e\n\u003cli\u003eS.Thongkrairat and V.Chutchavong, \u0026quot;A Time Improvement PSO Base Algorithm Using Multithread Programming,\u0026quot; 2019 4th International Conference on Communication and Information Systems (ICCIS), 2019, pp. 212-216.\u003c/li\u003e\n\u003cli\u003eE. A. Abdullah, I. Ahmed Saleh and O. I. Al Saif, \u0026quot;Performance Evaluation of Parallel Particle Swarm Optimization for Multicore Environment,\u0026quot; 2018 International Conference on Advanced Science and Engineering (ICOASE), 2018, pp. 81-86.\u003c/li\u003e\n\u003cli\u003eJingchao L. The Research and Application of Parallel Particle Swarm Optimization Algorithm Based on CUDA, D. Guangdong University of Technology, 2014.\u003c/li\u003e\n\u003cli\u003eVerma A, Kaushal S. A hybrid multi-objective particle swarm optimization for scientific workflow scheduling, J. Parallel Computing, 2017, 62: 1-19. \u003c/li\u003e\n\u003cli\u003eLi B, Wada K. Communication latency tolerant parallel algorithm for particle swarm optimization, J. Parallel Computing, 2011, 37(1): 1-10. \u003c/li\u003e\n\u003cli\u003eHussain M M, Fujimoto N. GPU-based parallel multi-objective particle swarm optimization for large swarms and high dimensional problems, J. Parallel Computing, 2020, 92: 102589. \u003c/li\u003e\n\u003cli\u003eCalazan R M, Nedjah N, de Macedo Mourelle L. Parallel GPU-based implementation of high dimension particle swarm optimizations, C. 2013 IEEE 4th Latin American Symposium on Circuits and Systems (LASCAS). IEEE, 2013: 1-4. \u003c/li\u003e\n\u003cli\u003eZhou Y, Tan Y. Particle swarm optimization with triggered mutation and its implementation based on GPU, C. Proceedings of the 12th annual conference on Genetic and evolutionary computation. 2010: 1-8. \u003c/li\u003e\n\u003cli\u003eCai Y, Li G Y, Wang H. Research and implementation of parallel particle swarm optimization based on CUDA, J. Application Research of Computers, 2013, 30(8): 2415-2418.\u003c/li\u003e\n\u003cli\u003eFu X, Ma S Q, Yun D W, et al. GPU Local PSO Algorithm at Dimension Level-Based Medical Image Registration, M. Fuzzy Information and Engineering-2019. Springer, Singapore, 2020: 133-144.\u003c/li\u003e\n\u003cli\u003eZHANG S, HE F, ZHOU Y, et al. GPU parallel particle swarm optimization algorithm based on adaptive warp, J. Journal of Computer Applications, 2016, 36(12): 3274. \u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"PSO, GPU, Initization strategy, Thread self-adaption.","lastPublishedDoi":"10.21203/rs.3.rs-2640747/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-2640747/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eParticle Swarm Optimization (PSO) is one of the most commonly heuristics-based methods that has been used to solve various optimization problems due to its simplicity and robustness. However, when comes to practical applications, it requires a huge computational cost. With the development of parallel computing and Graphics Processing Unit (GPU) calculating, many researchers have tried taking these techniques to break down the obstacle of computational efficiency. It is a challenging problem for the long-term application of PSO. In this paper, we propose a HEPSO algorithm that focuses on the procedure optimization of PSO in GPU-based architecture. It optimizes the GPU computation process from two following aspects: 1) Migrate the data initialization procedure from CPUs to GPUs to reduce the huge IO loss caused by repeating migration while the computing process. 2) Employ a self-adaptive thread management strategy to improve the algorithm execution efficiency. Moreover, we use four benchmark optimization functions to test the efficiency of our HEPSO. The experiment results show that the time speedup ratio between HEPSO and GPU-PSO can exceed 6 times. Meanwhile, when we evaluate the performance of HEPSO with the time consumption for functions converge, HEPSO only needs 1/3 time of GPU-PSO in most cases.\u003c/p\u003e","manuscriptTitle":"GPU based High-efficiency PSO Algorithm with Initialization strategy and Thread Slef-adaption","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2023-03-20 15:09:37","doi":"10.21203/rs.3.rs-2640747/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"31b83661-b01d-4675-8bbc-c2e48153ba3e","owner":[],"postedDate":"March 20th, 2023","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2023-03-26T00:44:17+00:00","versionOfRecord":[],"versionCreatedAt":"2023-03-20 15:09:37","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-2640747","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-2640747","identity":"rs-2640747","version":["v1"]},"buildId":"_2-kVJe1T_tPrBINL-cwx","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}