Analyzing TCEC Season 14 games with lc0 10520 + TB #290
Comments
|
As at TCEC games are adjudicated with tablebases, as soon as Lc0 picks a tablebase move, game would be stopped. At TCEC that would surely help. |
|
I updated the original post with 10,000 visits and pgn to check lichess TB. Looking at positions where 10520 + tb would have not played the game ending move: Q for e4f4 is close to draw, and TB says it’s a draw. All these top 3 rook moves seem to have pretty good win rates, but 7-man TB says they’re all draws, so avoiding the capture and extending the game might lead to odd behavior given that the eval is so off. Moving the king instead of capturing with the knight is still a draw, and the Q is somewhat close to draw. All moves here are draw. Instead of capturing for a TB hit loss, it tries a different move, but all moves are losses anyway, so not changing the outcome here. No escaping the draw, but the other moves Q are pretty close to draw anyway. It thinks it has a decent chance to win by avoiding the TB draw, but those moves turns out to be draws anyway. Avoiding the capture for TB draw attempting to win but these are draws. Overall, looks like when lc0 + TB plays away from the TB move in these cases, it would have drawn or lost anyway. However, there can be quite a difference in the play-away-from-TB-draw move win rate, so it’s giving up a capture to extend the game attempting a win, but these positions aren’t winnable, so potentially it opens up the opportunity to blunder especially that avoiding the move means the opponent has an extra piece now. So guessing the behavior several moves ahead, these "avoid TB" leaf moves could mean search will be directed towards these positions that are drawn or lost instead of finding the better winning move. I guess we'll see in a bit if the 50-move training data fix will clean up these wrong evals or if something like #237 with a lower temperature will be needed. Edit: To be clear, these positions are those that TCEC adjudicated with SyzygyTB, so most likely this means the two engines disagreed that it was a draw or a win/loss. There are plenty of positions where lc0 + TB would play towards the winning move, but perhaps those positions are those that both engines would agree that one side is winning. |
@jjoshua2 Are these the expected behavior with 6-man TB? Here's the games with at most 100 moves that reached 6 pieces. I ran the position just before the 26th capture with 10,000 visits using 10520 to see what search thought of the played move with and without TB including the top 3 most visited moves.
In
16.3 Tucano 7.05 vs Ivanhoe 999946h, 10520 thought the move heavily favored the opponent and avoids the move, but with TB, it would have played it to draw. Amusingly in23.2 LCZero 16.10161 vs Senpai 2.0, lc0 thinks its played move is the least losing, but with TB, it would have felt better knowing it was a draw. And in26.4 Tucano 7.05 vs Senpai 2.0for both with and without TB, the most visited move tries to win instead of drawing.In
13.4 Pedone 1.8 vs Arasan TCEC13, lc0 thought it found an amazing winning move out of other losing moves, but TB would have clarified it was just a draw. In14.1 lc0 16.10520 vs Pedone 1.8, lc0 itself normally would have played the draw TB move, but with TB, it would play a different move as it believes it's winning. Similarly15.2 Ethereal 10.81 vs DeusX 1.0, the highest prior move is for the TB draw move, but with TB, it finds a different move to continue the game.Generally in most positions, 10520 would play the same move that ended the game, but even then, the Q could be quite different, and some positions lc0+tb would play a completely different move, although unclear if it would have changed the final outcome.
The text was updated successfully, but these errors were encountered: