I have run UnicornFork for a few weeks now and have some statistics. I gave the LLM tools to check lines etc, and then each response is fact checked, e.g., if I detect "24...Be5 25.dxe5 Ke8 26.exf6+ Kd7 27.Ne5+" in the response then I check that this would be a valid continuation from the current position.
Some stats:
GPT-6 Luna 337 answers which included 20 false claims, which all were corrected.
Nemotron 3.5 364 answers with 53 false claims, where 13 were not be corrected.
I have run UnicornFork for a few weeks now and have some statistics. I gave the LLM tools to check lines etc, and then each response is fact checked, e.g., if I detect "24...Be5 25.dxe5 Ke8 26.exf6+ Kd7 27.Ne5+" in the response then I check that this would be a valid continuation from the current position.
Some stats: GPT-6 Luna 337 answers which included 20 false claims, which all were corrected. Nemotron 3.5 364 answers with 53 false claims, where 13 were not be corrected.