Benchmarks

Starting from version 0.2.0 of OrbisChessEngine this page shows benchmark results for perft for various depths, which can be used to compare performance with older versions, as well as an estimate of Orbis's playing strength.

Benchmark Results for Perft

All benchmarks below are using a single thread. Perft uses the Board stuct from OrbisChessEngine, which means it computes zobrist hash, and evaluation score at each node. Thus, it mimics the search process more closely than a pure move generator perft.

using OrbisChessEngine
using BenchmarkTools
b = Board()
perft(b, 5) # warm up
@benchmark perft($b, 5)

BenchmarkTools.Trial: 13 samples with 1 evaluation per sample.
 Range (min … max):  410.429 ms … 414.399 ms  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     410.872 ms               ┊ GC (median):    0.00%
 Time  (mean ± σ):   411.371 ms ±   1.125 ms  ┊ GC (mean ± σ):  0.00% ± 0.00%

  █ ▁ ▁▁█    ▁  ▁ ▁    ▁              ▁                       ▁  
  █▁█▁███▁▁▁▁█▁▁█▁█▁▁▁▁█▁▁▁▁▁▁▁▁▁▁▁▁▁▁█▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁█ ▁
  410 ms           Histogram: frequency by time          414 ms <

 Memory estimate: 19.34 KiB, allocs estimate: 16.

Using perft_bishop_magic which uses magic bitboards for bishop move generation:

using OrbisChessEngine
using BenchmarkTools
b = Board()
perft_bishop_magic(b, 5) # warm up
@benchmark perft_bishop_magic($b, 5)

BenchmarkTools.Trial: 13 samples with 1 evaluation per sample.
 Range (min … max):  409.276 ms … 416.628 ms  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     409.682 ms               ┊ GC (median):    0.00%
 Time  (mean ± σ):   410.423 ms ±   2.074 ms  ┊ GC (mean ± σ):  0.00% ± 0.00%

  █ ▁▁                                                           
  █▁██▆▆▁▁▁▁▆▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▆▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▆ ▁
  409 ms           Histogram: frequency by time          417 ms <

 Memory estimate: 19.34 KiB, allocs estimate: 16.

Seems to be barely affect performance.

Playing Strength

Starting from version 0.3.1 we estimate Orbis's playing strength directly by playing it against Stockfish at reduced strength (via the UCI_LimitStrength/UCI_Elo options), using cutechess-cli as the match manager.

Based on 40 games vs Stockfish 2000 Elo 1s/move it scored 19-14-7 (W/L/D) giving an estimate of 2043 ± 101.2 Elo.