TY - GEN
T1 - Measuring optimiser performance on a conical barrier tree benchmark
AU - Tkach, Itshak
AU - Blackwell, Tim
N1 - Publisher Copyright:
© 2022 ACM.
PY - 2022/7/8
Y1 - 2022/7/8
N2 - The common method for testing metaheuristic optimisation algorithms is to benchmark against problem test suites. However, existing benchmark problems limit the ability to analyse algorithm performance due to their inherent complexity. This paper proposes a novel benchmark, BTB, whose member functions have known geometric properties and critical point topologies. A given function in the benchmark is a realisation of a specified barrier tree in which funnel and basin geometries, and values and locations of all critical points are predetermined. We investigate the behaviour of two metaheuristics, PSO and DE, on the simplest manifestations of the framework, ONECONE and TWOCONES, and relate algorithm performance to a downhill walker reference algorithm. We study success rate, defined as the probability of optimal basin attainment, and inter-basin mobility. We find that local PSO is the slowest optimiser on the unimodal ONECONE but surpasses global PSO in all TWOCONES problems instances below 70 dimensions. DE is the best optimiser when basin difference depths are large but performance degrades as the differences become smaller. LPSO is the superior algorithm in the more difficult case where basins have similar depth. DE consistently finds the optimum basin when the basins have equal size and a large depth difference in all dimensions below 100D; the performance of LPSO falls away abruptly beyond 70D.
AB - The common method for testing metaheuristic optimisation algorithms is to benchmark against problem test suites. However, existing benchmark problems limit the ability to analyse algorithm performance due to their inherent complexity. This paper proposes a novel benchmark, BTB, whose member functions have known geometric properties and critical point topologies. A given function in the benchmark is a realisation of a specified barrier tree in which funnel and basin geometries, and values and locations of all critical points are predetermined. We investigate the behaviour of two metaheuristics, PSO and DE, on the simplest manifestations of the framework, ONECONE and TWOCONES, and relate algorithm performance to a downhill walker reference algorithm. We study success rate, defined as the probability of optimal basin attainment, and inter-basin mobility. We find that local PSO is the slowest optimiser on the unimodal ONECONE but surpasses global PSO in all TWOCONES problems instances below 70 dimensions. DE is the best optimiser when basin difference depths are large but performance degrades as the differences become smaller. LPSO is the superior algorithm in the more difficult case where basins have similar depth. DE consistently finds the optimum basin when the basins have equal size and a large depth difference in all dimensions below 100D; the performance of LPSO falls away abruptly beyond 70D.
KW - DE
KW - Differential evolution
KW - Optimization
KW - PSO
KW - Particle swarm optimization
KW - Problem benchmarks
UR - http://www.scopus.com/inward/record.url?scp=85135229570&partnerID=8YFLogxK
U2 - 10.1145/3512290.3528842
DO - 10.1145/3512290.3528842
M3 - Conference contribution
AN - SCOPUS:85135229570
T3 - GECCO 2022 - Proceedings of the 2022 Genetic and Evolutionary Computation Conference
SP - 22
EP - 30
BT - GECCO 2022 - Proceedings of the 2022 Genetic and Evolutionary Computation Conference
PB - Association for Computing Machinery, Inc
T2 - 2022 Genetic and Evolutionary Computation Conference, GECCO 2022
Y2 - 9 July 2022 through 13 July 2022
ER -