| - | TCEC Grand Champion | Runner-up | - | Stockfish | AllieStein | - | 54.5 (14) | 45.5 (5) | - | TCEC Season 17 → |
The 16th season of the Top Chess Engine Championship began on 15 July 2019 and ended on 13 Oct 2019.
The season showed how fast neural network-based engines were progressing relative to traditional ones, with neural network-based engines making up half of Premier Division for the first time. Furthermore, AllieStein, a neural network-based engine that had reached Premier Division last season, made its first appearance in the superfinal after finishing second in Premier Division. Nonetheless, a traditional engine – Stockfish – won both Premier Division and the superfinal.
TCEC changed its format for this season again, with the aim of allowing more chances for new engines to climb the rankings. This is in turn necessary because of the fast pace of development in computer chess, with numerous very strong engines emerging quickly. The superfinal and Premier Division were kept, but the lower divisions were replaced by three leagues: a Qualification League, League Two, and League One. The top six engines from the Qualification League promote to League Two (where the bottom six relegate), while the top four engines from League Two promote to League One (where the bottom four relegate). Finally, the top four engines in League One contest a playoff, with the top two finishers advancing to Premier Division.[1] The top two engines in Premier Division then contest the superfinal. All three lower-division leagues are single double-round robins, while Premier Division is 3x double-round robin, and the superfinal remains a 100-game match.
The qualification league comprised 18 engines; however, one engine - Leela Chess Zero's CPU version - would only be used as a benchmark, and its scores zeroed out to determine final standings.
Right from the start it became apparent that two engines - the new engine Stoofvlees and ScorpioNN - were significantly stronger than everyone else. ScorpioNN had been plagued by crashes last season, but those issues had been fixed, and it had further improved rapidly. It defeated ten of its seventeen rivals 2-0, and did not drop a match. Stoofvlees turned in a similarly dominant performance, also defeating ten of its seventeen rivals 2-0, but it dropped a game to 3rd-place Marvin. Both engines comfortably claimed the first two spots, six points ahead of Marvin. The remaining promotion slots were more contested. Marvin and Wasp were able to distance themselves from the chasing pack, but Asymptote and Winter were able to keep up with Topple and chess22k up until the final rounds, eventually finishing two points off the pace.[2]
Fresh from promotion and with an update before the start of League Two, Stoofvlees comfortably placed first with 22.5/30, two points ahead of ScorpioNN. Ominously however, it showed erratic play and lost three games. Second-placed ScorpioNN did not lose a game and promoted smoothly as well. In contrast, the remaining two promotion spots were closely contested. Pedone was whitewashed by Stoofvlees, but was able to defeat six engines 2-0. This eventually allowed it to finish third, one point ahead of Arasan and Fritz. Arasan and Fritz both finished with 18/30. Like ScorpioNN, Fritz did not lose a game, but it wasn't able to score many wins either. This allowed Arasan - who had narrowly missed promotion last season - to claim the final promotion spot on the third tiebreak (number of wins).
In contrast to the dominating performances in the previous leagues, League One was closely contested, with every engine losing games. Facing their strongest opponents yet, Stoofvlees and ScorpioNN again managed to promote, although Stoofvlees again played erratically and lost three games. ScorpioNN, which had not lost any games in the previous divisions, finally dropped points to Stoofvlees and last season's Premier Division engine, Ethereal. Eventual second-placed finisher Laser crashed once against Ginkgo, but still mustered enough wins to finish half a point ahead of ScorpioNN and Ethereal, qualifying for the playoffs. The final results were marred by crashes: Booot crashed three times and was disqualified, while Ginkgo came close by crashing twice. The possibility of Ginkgo crashing a third time kept fifth-placed Xiphos in contention up until the final round, when it needed to beat Stoofvlees and hope for a Ginkgo crash, in which case it would qualify for the playoffs instead of Ethereal. Nonetheless, Ginkgo did not crash, and Ethereal finished third, just ahead of ScorpioNN on tiebreak.
At the other end of the table, Fire, which had been relegated from Premier Division together with Ethereal last season, submitted a drastically different neural-network based version for this season, and was soundly outmatched. It finished last with 6.5/30, scoring only one win by forfeit against Booot.
The League One playoffs was very close, and each engine lost games. After 12 games, only two points separated first-placed Stoofvlees from last-placed Laser. Stoofvlees, which had won League 1 comfortably, was defeated by ScorpioNN in the very first game of the playoffs, but recovered with three wins to finish first. Laser started with a win against Ethereal, but lost consecutive games to Stoofvlees and ScorpioNN to finish last. Ethereal was within striking distance of ScorpioNN up until the final games, and even won a head-to-head game, but it was defeated by Stoofvlees twice and finished one point behind ScorpioNN.
After the straightforward Premier Division of TCEC S15, the S16 Premier Division turned out to be a roller-coaster. As in the previous season, multiple-time champion and S15 runner-up Stockfish cruised to a dominating lead. After three rounds, it had scored wins against every engine other than S15 champion Leela Chess Zero and third-place finisher Komodo, while soundly defeating newcomers Stoofvlees and ScorpioNN 2.5-0.5. Meanwhile Lc0 didn't manage to win nearly as many games as Stockfish, but did score a head-to-head win, and was second. Komodo had started brightly with two wins, but went down to AllieStein, who in turn was defeated by Stockfish.
However, Stockfish dramatically crashed once in the second round against Houdini, and then again in the third round against AllieStein, both times in probably-drawn positions. Although Stockfish had built up such a lead that it was still joint first after the crashes, the crashes did mean that one more crash would lead to Stockfish being disqualified (per the rules). Stockfish's developers eventually came to the conclusion that the crashes were caused by a bug in MinGW used in compiling Stockfish, which included a faulty .dll file.[3] Although the source of the crashes was due to third-party code and not Stockfish itself, by TCEC rules, engines could not be updated during divisions. Nonetheless, in Season 8, Stockfish had been allowed to update during a division to correct a non-critical bug, which set a precedent to allow updates if all participating teams agreed.[4] Tournament director Anton Mihailov therefore organized an anonymous vote among all competitors to see if Stockfish should be allowed to replace the .dll file.[5] Stockfish would be allowed to update only if the vote was unanimously in favor. Most of the participating teams voted "yes", but one team voted "no". The "no" vote served as a veto and meant that Stockfish would not be allowed to update the .dll file.[6]
In the rounds after the vote, AllieStein surged to win after win to seize first place, defeating ScorpioNN, Houdini, Komodo MCTS and Stoofvlees one after another. By the last round, AllieStein had amassed 14 wins, including 7 with Black – more Black wins than the rest of the division combined. This large number of wins gave it the tiebreaker advantage over Lc0. Stockfish, with two crashes, had the worst tiebreakers of the three rivals and needed to finish ahead in points. It managed to defeat AllieStein a further two times, which gave it an advantage over Lc0, who was still struggling to score wins. Nonetheless, with the specter of two crashes hanging over Stockfish, the race was not over. In the final round, Lc0 needed to beat Stoofvlees with Black while hoping for Stockfish to either crash or lose to Komodo MCTS with White. When Stockfish–Komodo MCTS ended in a draw, Lc0 was eliminated. It became one of three engines in TCEC to finish a season undefeated (the other two are Stockfish in Season 10, and Fritz in Season 16). The draw also meant Stockfish narrowly won the division, 0.5 points ahead of AllieStein, in spite of two crashes.
For the other competitors, Komodo turned in a credible performance, conceding only two losses: to AllieStein, which it avenged in their fifth head-to-head encounter, and to Leela. This allowed it to finish clear fourth, 3.5 points behind Lc0, but 3 points ahead of Stoofvlees. Stoofvlees continued to play erratically (see diagram), but was one of only two engines to hold Leela to all draws. Houdini, which had not been updated for two years, showed it was still strong enough to compete, finishing sixth. Komodo MCTS had been competitive with Houdini last season, but performed poorly this season, losing ten games and scoring only one win against ScorpioNN. Finally, newcomer ScorpioNN was outmatched and finished last without scoring a win.
With three head-to-head wins in Premier Division (and neglecting the one loss due to crash), Stockfish began the superfinal as the favorite. Nonetheless, it was AllieStein who drew first blood when it won game 3, smoothly outplaying Stockfish in a King's Indian Defense. After this initial loss, however, Stockfish quickly took the lead. It first equalized by winning game 6, and then won a further three games before AllieStein notched a second win, in game 27. The two traded wins in games 32 and 39, but then Stockfish notched six wins in a row without reply to take an 8-point lead. AllieStein won game 75, but when Stockfish won game 80 after AllieStein went kamikaze with its king (diagram) the superfinal was all but over. Stockfish scored another two wins in the final 20 games for one loss, winning the superfinal by 9 points (+14 -5 =81).
In an article published a few weeks after the end of the superfinal, GM Matthew Sadler wrote that although clashes between traditional and neural-network engines were always interesting because of the difference in playstyle, it was also frustrating to miss out on the tension of a Lc0–Stockfish match. Nonetheless, there were excellent games by both engines in the superfinal.[7]
In this King's Indian Defense position, AllieStein demonstrated superior understanding of the pawn structure after 18...e4. The point is that this pawn structure is only playable for Black if he has control of the d4-square and if White is unable to mount a queenside breakthrough. In this position, White not only has access to the d4-square, he can play a4-a5 at will. In the game, AllieStein made full use of the d4 square to break apart Black's position, scoring a victory.
Game 3: AllieStein–Stockfish (King's Indian)
1. d4 Nf6 2. c4 g6 3. Nc3 Bg7 4. e4 d6 5. Nf3 O-O 6. Be2 e5 7. O-O Nc6 8. d5 Ne7 9. Ne1 Nd7 10. b4 a5 11. bxa5 Rxa5 12. a4 Ra8 13. f3 f5 14. Nd3 Kh8 15. Bd2 b6 16. Ra3 Ng8 17. exf5 gxf5 18. f4 e4 19. Nb4 Ne7 20. Be3 Nc5 21. Qd2 Bd7 22. Rfa1 Qe8 23. R1a2 Bf6 24. Nb5 Qd8 25. Nc2 Nb7 26. g3 h6 27. Qd1 Kh7 28. Ncd4 Nc5 29. Kf1 Rg8 30. Bf2 Nb7 31. Kg2 Rc8 32. Kh1 Ra8 33. Be1 Qc8 34. Bd2 Na5 35. Bc3 Ng6 36. Qf1 Qd8 37. Qh3 Rf8 38. Ra1 Nb7 39. Bh5 Bxd4 40. Bxd4 Nc5 41. Be2 Rf7 42. Qf1 Rf8 43. Qf2 Ne7 44. h3 Rg8 45. Kh2 Ng6 46. Qe3 Nb7 47. Bc3 Ne7 48. Bh5 Ng6 49. Nd4 Na5 50. Be2 Qe7 51. Bb2 Qf7 52. Nb3 Nxb3 53. Qxb3 Ra5 54. Qd1 Rga8 55. Bc3 R5a7 56. Qd4 Ne7 57. R3a2 Qf8 58. a5 Be8 59. a6 Bg6 60. Ra3 Qf7 61. Bb2 Rc8 62. Rg1 Rg8 63. Kh1 Re8 64. g4 Rg8 65. Rg2 Qf8 66. Qe3 Be8 67. Qf2 c5 68. g5 Ra8 69. Bf6 Rg6 70. Qh4 Ng8 71. Bh5 Rxf6 72. gxf6 Qxf6 73. Qxf6 Nxf6 74. Bxe8 Rxe8 75. Rb2 Nd7 76. a7 Ra8 77. Kg1 Kg6 78. Ra6 Kf7 79. Rba2 Kg6 80. Rg2+ Kf7 81. h4 h5 82. Kf1 Nf8 83. Rb2 Nd7 84. Ke1 Ke7 1-0
Stockfish won this game pair (each engine plays both sides of the same opening) after drawing game 5 with a fantastic tactical defense, then winning the reverse game with a maneuver that AlphaZero had employed against it in their 2018 match.
Game 5: AllieStein–Stockfish (Sicilian Defense)
1. e4 c5 2. Nf3 e6 3. d4 cxd4 4. Nxd4 Nc6 5. Nc3 Qc7 6. Be3 a6 7. a3 Nf6 8. f4 d6 9. Qf3 Be7 10. O-O-O O-O 11. Nxc6 bxc6 12. g4 d5 13. g5 Nd7 14. h4 Rb8 15. h5 Qa5 16. Bf2 Rd8 17. g6 Rxb2 18. Kxb2 Qxa3+ 19. Kb1 Qb4+ 20. Ka2 Qa3+ 21. Kb1 Qb4+ 22. Kc1 Qa3+ 23. Kd2 Ne5 24. gxf7+ Kf8 25. Qg3 Nf3+ 26. Ke2 Qxc3 27. e5 Qc4+ 28. Kxf3 Qe4+ 29. Kg4 Qf5+ 30. Kf3 Qe4+ 31. Kg4 Qf5+ 32. Kf3 Qe4+ 1/2-1/2
Game 6: Stockfish–AllieStein (Sicilian Defense)
1. e4 c5 2. Nf3 e6 3. d4 cxd4 4. Nxd4 Nc6 5. Nc3 Qc7 6. Be3 a6 7. a3 Nf6 8. f4 d6 9. Qf3 Be7 10. O-O-O O-O 11. Nxc6 bxc6 12. g4 Nd7 13. h4 Rd8 14. h5 Rb8 15. Be2 d5 16. h6 g6 17. Rhf1 Qa5 18. Nb1 Qc7 19. Nc3 Qa5 20. Bd4 c5 21. Bg7 d4 22. Nb1 Qb6 23. b3 a5 24. Nd2 Ba6 25. Nc4 Qc6 26. a4 Rb4 27. Ne5 Nxe5 28. fxe5 Bg5+ 29. Kb1 f6 30. Bxf6 Rf8 31. Bxa6 Qxa6 32. Bxg5 Rxf3 33. Rxf3 Qa7 34. Rdf1 Rb8 35. Bf6 g5 36. Bg7 Qa8 37. Kc1 Re8 38. Kd2 c4 39. bxc4 Qd8 40. c5 Qe7 41. c6 Qb4+ 1-0
In this French Winawer, AllieStein played 12...Na6, a rare variation that has also been played by AlphaZero. The game played out similarly to AlphaZero's game, but although AllieStein held the balance well, it missed a relatively simple tactic late in the game, and its position crumbled.
Game 14: Stockfish–AllieStein (French Winawer)
1. e4 e6 2. d4 d5 3. Nc3 Bb4 4. e5 c5 5. a3 Bxc3+ 6. bxc3 Ne7 7. Qg4 Qc7 8. Qxg7 Rg8 9. Qxh7 cxd4 10. Ne2 dxc3 11. f4 Bd7 12. Qd3 Na6 13. Nd4 Qa5 14. Nb3 Qa4 15. g3 Rc8 16. Be3 b6 17. Nd4 Nc5 18. Qh7 Rg6 19. Bh3 Ne4 20. O-O Nf5 21. Bxf5 exf5 22. Rad1 Nd2 23. Rf2 Rc4 24. e6 Bxe6 25. Qh5 Ne4 26. Nxe6 Rxe6 27. Rxd5 Nd6 28. Rd3 Ke7 29. g4 Rce4 30. gxf5 Rf6 31. Re2 Nxf5 32. Qh8 Rg6+ 33. Kf1 Qc6 34. Bc5+ Nd6 35. Bxd6+ Rxd6 36. Rxd6 Kxd6 37. Qh6+ Re6 38. Qf8+ Kd5 39. Qxf7 1-0
This opening was another that had featured in the AlphaZero–Stockfish match. After 11 moves (diagram), AlphaZero demonstrated an exceptionally strong plan that involved a Rook lift and kingside advance. The plan was strong enough that GM Sadler opined that it more or less made this line unplayable for Black. However, neither AllieStein nor Stockfish played the idea. AllieStein played a more sedate version with g4, which failed to break through. Stockfish played the more traditional plan of pushing central pawns that however led to a very non-traditional king hunt (diagrams).
Game 26: Stockfish–AllieStein (Queen's Indian)
1. d4 Nf6 2. c4 e6 3. Nf3 b6 4. g3 Bb7 5. Bg2 Bb4+ 6. Bd2 Be7 7. Nc3 O-O 8. Qc2 Na6 9. a3 c5 10. d5 exd5 11. Ng5 Nc7 12. Nxd5 Ncxd5 13. cxd5 d6 14. e4 b5 15. f4 h6 16. h4 b4 17. axb4 cxb4 18. e5 dxe5 19. fxe5 Re8 20. Kd1 hxg5 21. hxg5 Nh5 22. g6 Bxd5 23. Rxh5 Qd7 24. gxf7+ Kxf7 25. Ra6 Qg4+ 26. Kc1 Qxh5 27. Bxd5+ Kf8 28. Rh6 g6 29. Rxh5 Rec8 30. Rh8+ Kg7 31. Rxc8 Rxc8 32. Qxc8 Bg5 33. Qg8+ Kh6 34. Qh8# 1-0
The opening for this game pair featured another of AlphaZero's more outlandish openings (diagram). GM Sadler wrote that although he had seriously studied the 3. f3 line in the Grunfeld, he could not have imagined this idea; in fact for a while afterwards he wondered if it was a case of AlphaZero hating its position and going all-in, or if there was something of substance concealed. In this game, Stockfish deviated from the line it played against AlphaZero after 11. Bxf4 Nb4 (provoking a queenside weakness) 12. a3 Nc6 with 13. g3. This time Stockfish avoided the queenside counterplay that AlphaZero had achieved, and although AllieStein was able to regain the sacrificed pawn, White was able to erect a wall around the g7-bishop that constituted a long-term problem for Black. Stockfish exploited the advantage convincingly to win in 64 moves.
Stockfish–AllieStein, game 42 (Neo-Gruenfeld defence)1. d4 Nf6 2. c4 g6 3. f3 d5 4. cxd5 Nxd5 5. e4 Nb6 6. Nc3 Bg7 7. Be3 O-O 8. Qd2 Nc6 9. O-O-O f5 10. e5 f4 11. Bxf4 Nb4 12. a3 Nc6 13. g3 Bf5 14. Ne4 Qd5 15. Ne2 Rad8 16. Qc2 h6 17. h4 Qa2 18. N2c3 Qf7 19. Be3 Na5 20. Bd3 Nb3+ 21. Kb1 Nxd4 22. Bxd4 Rxd4 23. f4 Rfd8 24. Rhe1 Nc4 25. Nc5 Bxd3 26. Rxd3 Rxd3 27. Nxd3 c6 28. Ne4 a5 29. a4 Rd4 30. Ndc5 Qd5 31. Nb3 Rd3 32. Nc3 Qf3 33. Ka2 b5 34. Ne4 Kf7 35. Nbc5 Rd5 36. Qc1 Bf8 37. f5 Kg8 38. fxg6 Qf5 39. g4 Qxg6 40. h5 Qg7 41. axb5 cxb5 42. Rf1 Rxe5 43. Rf4 Rxe4 44. Nxe4 Qe5 45. b3 Nd6 46. Nxd6 Qxd6 47. Rf2 a4 48. Qe3 axb3+ 49. Qxb3+ e6 50. Re2 Kf7 51. Qxb5 Kf6 52. Rf2+ Kg7 53. Qb7+ Be7 54. Qf3 Qd8 55. Rb2 Kg8 56. Rb7 Qd4 57. Kb3 Bf6 58. Rb8+ Kf7 59. Rb4 Qd6 60. Rb7+ Kg8 61. Qe4 Be7 62. Qg6+ Kf8 63. Qxh6+ Kg8 64. Qg6+ Kf8 1-0
In this game, Stockfish drifted into a position which it had previously had trouble understanding; indeed, AlphaZero had won several games from this kind of position. AllieStein exploited the advantage in typical fashion, first with a kingside advance that planted a pawn on h6 (fixing a long-term weakness on h7), and then with an attack on the opposite side of the board.
AllieStein–Stockfish, game 75 (Pirc defence)1. e4 d6 2. d4 Nf6 3. Nc3 g6 4. Be3 c6 5. h3 Bg7 6. f4 O-O 7. Nf3 Nbd7 8. e5 Nd5 9. Nxd5 cxd5 10. Bd3 Qb6 11. Qc1 dxe5 12. dxe5 Qc7 13. Bd4 Nc5 14. Qe3 b6 15. O-O Qc6 16. b4 Nxd3 17. cxd3 Qa4 18. a3 Be6 19. g4 a5 20. Rfb1 Bd7 21. Kf2 Rfc8 22. Kg3 e6 23. h4 axb4 24. Rxb4 Qa5 25. h5 b5 26. Rb2 Qd8 27. Rh2 g5 28. fxg5 Ra4 29. Rhh1 Be8 30. Rhc1 Raa8 31. Bc5 Rxc5 32. Qxc5 Bf8 33. Qc7 Be7 34. Qxd8 Bxd8 35. h6 Kf8 36. Rcb1 Ba5 37. Rb3 Bd7 38. g6 hxg6 39. g5 Kg8 40. Kh4 d4 41. Nxd4 Bc7 42. Nf3 Ra4+ 43. Kg3 Ra8 44. Rc1 Ba5 45. d4 Rd8 46. Rd3 Bc8 47. Rb3 Ba6 1-0
The final two games of the superfinal had both engines winning with White in a Sicilian Dragon. Game 100 was Stockfish's win, in which it played an attack featuring a highly geometrical Queen maneuver (diagrams). White's Queen circles the board several times in 20 moves, eventually catching Black's King in a mating net.
Stockfish–AllieStein, game 100 (Sicilian Dragon)
1. e4 c5 2. Nc3 Nc6 3. Nf3 g6 4. d4 cxd4 5. Nxd4 Bg7 6. Be3 Nf6 7. Bc4 O-O 8. Bb3 d6 9. f3 Bd7 10. Qd2 Rc8 11. O-O-O Ne5 12. Kb1 a6 13. h4 h5 14. g4 hxg4 15. h5 Nxh5 16. Nd5 Rc5 17. Bg5 Re8 18. fxg4 Bxg4 19. Rdg1 Qd7 20. Qf2 Rxd5 21. Bxd5 e6 22. Bb3 d5 23. exd5 exd5 24. Re1 Rc8 25. Rh4 Nc6 26. Nxc6 bxc6 27. Ka1 Qf5 28. Qg1 Nf6 29. Bd2 g5 30. Rh2 Ne4 31. c3 Bf3 32. Bd1 Bxd1 33. Rxd1 Re8 34. Bc1 f6 35. Re1 Re5 36. Rhe2 Re7 37. Qb6 Qc8 38. c4 Qb7 39. Qa5 Kf7 40. Qa3 g4 41. Rf1 Kg6 42. cxd5 cxd5 43. Rg2 f5 44. Qh3 Qb5 45. Rh1 Kf6 46. Qh7 Ke6 47. Qg6+ Bf6 48. Rh5 Qf1 49. Qg8+ Ke5 50. Rc2 g3 51. Qb8+ Kd4 52. Rh6 Bg5 53. Qb4+ Ke5 54. Rg6 Qb5 55. Qa3 Bh4 56. Rcc6 g2 57. Qf3 Qf1 58. Qh5 Re8 59. Rxg2 Re6 60. Qh8+ Rf6 61. Qb8+ Kd4 62. Qb4+ Ke5 63. Qe7+ 1-0