Chess Players! - A New Chess Engine!

English Open Chat
#1

Some of you may already know that the final round of the Computer Chess Championship is going on right now.
As might be expected of computer players and unlike human tournaments, the event has been played practically non-stop 24hrs a day, 7 days a week for several months now, and probably still has about a week or two more to go.

Stockfish, which is available in our openSUSE repos as a free download is the reigning champion and is currently leading the field, in fact was on something like a 140 game undefeated streak going into this final round and has been crushing foes one after another. Then, the unexpected. The engine who was faring last in this final round suddenly took Stockfish’s scalp in a very unexpected win. Then, a few games later and only a couple hours ago, lc0 which is the actual subject of this post also beat Stockfish.

This new engine, lc0 (Leela) is a new engine which in its current incarnation has been competing only very recently, and it grabbed my attention for a couple reasons… It doesn’t work very much like any previous chess engine, and if one looks at its games, you’d be struck by how differently it values initiative and seems to evaluate positional sacrifices, one of the most difficult things to master where you give away material and arrive at an objective position that cannot be measured by traditional piece values… That’s the magic of chess, that wooden pieces which are normally assigned static values can be imbued with extraordinary powers when conditions are right… And, most players computer and human don’t or can’t recognize those situations.

The other unique thing about lc0 which I still want to investigate is that supposedly this new incarnation doesn’t require instruction how to play the game. Supposedly you just feed it thousands of games, and it teaches itself the rules of the game, and because of this the program doesn’t just play world class chess. It also plays world class go and shojei without modifying the program. Wow. That’s some kind of machine learning.

Anyway…
Any of you interested in trying out this lc0 chess engine on openSUSE?

I’ve just created a Pull request for the lc0 project containing instructions for installing on openSUSE.
Don’t know how long it will take for approval, but anyone running openSUSE is welcome to install by accessing the guide from my github site

https://github.com/putztzu/lc0/blob/putztzu-patch-1/openSUSE_install.md

The main project site
https://github.com/LeelaChessZero/lc0

And,
For any of you who also are interested in writing BASH install scripts, you might find the included lc0 install script interesting…
It detects Tumbleweed vs LEAP and modifies the “Add Repo” install accordingly, installs a C++ build enviornment and executes the build that’s already on the site.

If anyone runs into problems or questions, the usual place to post would be in the Applications Technical Help forum…

Enjoy…
TSU

#2

Hi
https://build.opensuse.org/package/show/home:malcolmlewis:TESTING/lc0

Options enabled, gtest, openblas and OpenCL…

#3

Cool!

Depending on how things go, I’ll update the Project Info.
Note that my script points to the lc0 Development branch,
Your RPM may want to point to a Release branch instead… unless a Maintainer is willing to pay constant attention to re-building.

TSU

#4

Hi
It’s all automated, changelog building etc, minimal user interaction with the following command :wink:


osc service remoterun [PROJECT PACKAGE]
#5

Malcolm,
Your the one-click install for your LEAP package fails because it thinks it doesn’t have root permissions. Have tested 3x and never seen that error elsewhere, so I’m pretty sure it’s something specific to the spec (Is it possible to hard code the root password credential?)

The install proceeds up to where you enter the root password, but when it’s entered correctly YaST continues and then displays the results and the reason for failure (says root permissions required).

TSU

#6

Hi
Likely one-click and those mechanisms, I would download and install, it
will ask about the repo key. You would need to add the repo or import
manually.

So it will work with OpenCL (AMD gpu) else blas and use the cpu.

If wanting nvidia then would need to download the src rpm, ensure your
system has the nvidia bits installed and rebuild the rpm (this also
changes the license…).

Just upgrading to the latest Tumbleweed, will reboot and try the one-click.

#7

On Fri 11 Jan 2019 12:06:03 AM CST, tsu2 wrote:

Malcolm,
Your the one-click install for your LEAP package fails because it thinks
it doesn’t have root permissions. Have tested 3x and never seen that
error elsewhere, so I’m pretty sure it’s something specific to the spec
(Is it possible to hard code the root password credential?)

The install proceeds up to where you enter the root password, but when
it’s entered correctly YaST continues and then displays the results and
the reason for failure (says root permissions required).

TSU

Hi
Likely one-click and those mechanisms, I would download and install, it
will ask about the repo key. You would need to add the repo or import
manually.

So it will work with OpenCL (AMD gpu) else blas and use the cpu.

If wanting nvidia then would need to download the src rpm, ensure your
system has the nvidia bits installed and rebuild the rpm (this also
changes the license…).

–
Cheers Malcolm °¿° SUSE Knowledge Partner (Linux Counter #276890)
SLES 15 | GNOME Shell 3.26.2 | 4.12.14-25.25-default
If you find this post helpful and are logged into the web interface,
please show your appreciation and click on the star below… Thanks!

#8

Hi
Hmmm, worked fine here on Tumbleweed, selected not to remain subscribed, asked for root password and installed…

#9

Hi
So doesn’t pick up the discrete GPU to use if kicked off with DRI_PRIME=1 other apps are working fine switching


DRI_PRIME=1 glmark2
=======================================================
    glmark2 2017.07
=======================================================
    OpenGL Information
    GL_VENDOR:     X.Org
    GL_RENDERER:   AMD Radeon (TM) R7 M340 (ICELAND, DRM 3.27.0, 4.19.12-1-default, LLVM 7.0.0)
    GL_VERSION:    4.5 (Compatibility Profile) Mesa 18.3.1
=======================================================

glmark2
=======================================================
    glmark2 2017.07
=======================================================
    OpenGL Information
    GL_VENDOR:     X.Org
    GL_RENDERER:   AMD CARRIZO (DRM 3.27.0, 4.19.12-1-default, LLVM 7.0.0)
    GL_VERSION:    4.5 (Compatibility Profile) Mesa 18.3.1
=======================================================

Benchmark results for lc0;


lc0 benchmark --weights=.weights/weights
       _
|   _ | |
|_ |_ |_| v0.21.0-dev built Jan 10 2019
Loading weights file from: .weights/weights
Creating backend [opencl]...
OpenCL, maximum batch size set to 16.
Initializing OpenCL.
Detected 1 OpenCL platforms.
Platform version: OpenCL 1.1 Mesa 18.3.1
Platform profile: FULL_PROFILE
Platform name:    Clover
Platform vendor:  Mesa
Device ID:      0
Device name:    AMD CARRIZO (DRM 3.27.0, 4.19.12-1-default, LLVM 7.0.0)
Device type:    GPU
Device vendor:  AMD
Device driver:  18.3.1
Device speed:   720 MHZ
Device cores:   6 CU
Device score:   1111
Device ID:      1
Device name:    AMD Radeon (TM) R7 M340 (ICELAND, DRM 3.27.0, 4.19.12-1-default, LLVM 7.0.0)
Device type:    GPU
Device vendor:  AMD
Device driver:  18.3.1
Device speed:   1021 MHZ
Device cores:   5 CU
Device score:   1111
Selected platform: Clover
Selected device: AMD CARRIZO (DRM 3.27.0, 4.19.12-1-default, LLVM 7.0.0)
with OpenCL 1.1 capability.
Started OpenCL SGEMM tuner with batch size 16.
Will try 578 valid configurations.
(1/578) KWG=32 KWI=2 MDIMA=8 MDIMC=8 MWG=16 NDIMB=8 NDIMC=8 NWG=16 SA=0 SB=0 STRM=0 STRN=0 VWM=1 VWN=1 20677.0 us (26.0 GFLOPS)
....
(491/578) KWG=32 KWI=2 MDIMA=32 MDIMC=32 MWG=64 NDIMB=8 NDIMC=8 NWG=64 SA=1 SB=1 STRM=0 STRN=0 VWM=2 VWN=2 3778.9 us (142.1 GFLOPS)
Wavefront/Warp size: 64

Max workgroup size: 256
Max workgroup dimensions: 256 256 256
Benchmark time 490ms, 2 nodes, 4 nps, move e2e4
....
Benchmark time 183506ms, 19764 nodes, 107 nps, move e2e4
bestmove e2e4
Benchmark final time 187.364s calculating 107.705 nodes per second.

cat leelaz_opencl_tuning

0;XgemmBatched;256;256;256;16; -DKWG=32 -DKWI=2 -DMDIMA=32 -DMDIMC=32 -DMWG=64 -DNDIMB=8 -DNDIMC=8 -DNWG=64 -DSA=1 -DSB=1 -DSTRM=0 -DSTRN=0 -DVWM=2 -DVWN=2;OpenCL: AMD AMD CARRIZO (DRM 3.27.0, 4.19.12-1-default, LLVM 7.0.0) @ 720MHz
#10

Hi
Using the blas backend… way way slower :wink:


lc0 benchmark --weights=.weights/weights -b blas
       _
|   _ | |
|_ |_ |_| v0.21.0-dev built Jan 10 2019
Loading weights file from: .weights/weights
Creating backend [blas]...
BLAS, maximum batch size set to 256
BLAS vendor: OpenBlas.
OpenBlas [OpenBLAS 0.3.4 DYNAMIC_ARCH NO_AFFINITY Excavator MAX_THREADS=64].
OpenBlas found 4 Excavator core(s).
OpenBLAS using 1 core(s) for this backend.
BLAS max batch size is 256.
Benchmark time 1787ms, 2 nodes, 1 nps, move e2e4
....
Benchmark time 674791ms, 19729 nodes, 29 nps, move e2e4
bestmove e2e4
Benchmark final time 689.464s calculating 29.3054 nodes per second.
#11

Your scores regarding GPU vs CPU are expected…
And, that’s the other thing… AFAIK lc0 might be the only chess engine designed for GPU computing as its preferred configuration.

Here are the World Computer Chess Championship rules and the hardware competitors are running on… The GPU configuration got a CPU (not GPU part) upgrade today so is even beefier…

https://www.chess.com/news/view/announcing-the-new-computer-chess-championship

Believe the “Basic” package on the following page is the GPU hardware it’s running on… And in addition to this, there are also 2 CPUs, about $3000 each…

https://lambdalabs.com/servers/hyperplane/customize

TSU

#12

Hi
Yes, they mention cpu vs gpu… anyway, pushed a small fix upstream, if that’s accepted, will submit to the games development repository.

#13

Re the RPM…

Although the one-click installs don’t work,
I was able to download the LEAP RPM package and install locally.

Users should know that only OpenCL works if you’re hooked up to Arena (That means this will work only if you have an AMD GPU).
Am leaning right now to a problem with Arena and not with lc0 but that’s pure speculation right now.

Although executing the console debug tests directly against the binary work fine,
I can’t seem to get the Arena chessboard to recognize the backend argument… I tried specifying “-b blas” and then “–backend=blas” both in the command and separately in the “Command Parameters” input field, but neither has an effect… When the runtime log is viewed, only the OpenCL backend is run.

So,
Although I’ll update documentation to mention the availability of your RPM, I’ll also have to mention that at the moment it doesn’t seem to be configurable for any other than the OpenCL backend when hooked up to Arena.

The problem does not exist with the binary built by my script but on the other hand as my documentation describes, it does not support any other backend than BLAS so I can’t test for a possible compile difference… It may simply be that for some unknown reason Arena won’t support the backend selection and there is no fault in lc0…

I’ll have to try another chessboard…

TSU

#14

The problem is likely Arena…

I was able to install Cutechess, point to the binary from the RPM and specify the blas backend…

https://github.com/cutechess/cutechess/releases

For those installing the Cutechess GUI on openSUSE, install the main RPM (first listed) and install.
The install will complain about not finding a sufficiently late version of Qt-Base, ignore the dependency and select option 2, cutechess will work fine with what openSUSE provides today.

Don’t forget to have a Networks file in the same folder as the lc0 binary, it doesn’t have to be renamed and doesn’t need to be explicitly named as a command parameter unless you have multiple files and want to specify one (A default can be found automatically).

Configure the command to point to the lc0 binary with “-b blas” as the command argument unless you want to specify something else (The binary currently is compiled to support only OpenCL and BLAS, not CUDA or any other backend).

Set up your chess game with lc0 as a player and start playing!

TSU

#15

Hi
I see you pushed an update to your openSUSE install to the lc0 git repo. A couple of comments, the lc0 binary from the rpm is /usr/bin/lc0. There should be no need to move the binary, the user should look at command line options to point at the network file --weights=/path/to/file?

Why the additional repos in the install script, all files required are present in the openSUSE Leap/Tumbleweed default release repositories (this is how the rpm is built)? It should be built with python3 in mind, hence my simple patch for checkdir.py using python3 rather than python. Not sure how/why python3-abseil is installed?

#16

Yes,
I’ve submitted a Pull Request to the main lc0 project.
For now, people can see the preview in my own repo(link below)
Comments and corrections are welcome.
BTW - I just checked used the Tumbleweed RPM and the blas backend is working, now wondering unless you’ve made recent changes if the non-working blas backend is only the LEAP binary.
Moving the binary is not required, but a convenience specific to the Arena frontend so that it can be automatically found by the Arena Engine Installation wizard.
If only one weights file exists and is placed in the same directory as the binary, it will be found automatically no matter the name and no need for a command line parameter.

https://github.com/putztzu/lc0/blob/master/openSUSE_install.md

Re the additional repos, they’re not needed and actually won’t work in TW which is why they’re commented out in the script. But they’re needed building in LEAP. Did not remove altogether because they might be needed sometime (so why remove something that might need plenty of research and constructing the entries again?). Unless I made a mistake, IIRC all should point to python 3.

TSU

#17

Hi
Well the Build Service is only using the default repositories (oss), so should not need any of the development ones… perhaps this is introducing issues at your end with leap 15.0 and the blas backend?

I’ve tested on Leap 15 and use the backend switch as well as the demux backend and batching for blas all seems to do it’s thing.

I’ve also played with building just a blas default, but no changes so rolled back.

#18

Actually, I don’t have any problems with the blas backend when lc0 is generated by my script, and using your Tumbleweed RPM. The problem is using your LEAP RPM, and only with the Arena frontend (no problem with Cute Chess board). This almost certainly means that the issue is specific to the Arena frontend (one of, if not the primary board used by serious players today) and you won’t see the problem testing only lc0 or with other frontends. Unless I modify my script to compile OpenCL as well as blas, I can’t know if the problem is only with Arena or if your RPM is at fault.

It’s hard for me to remember exactly what I encountered earlier on when I was first trying to get things to work, but I just ran a check just now running my script, modified to not add the LEAP repos and without python3-abseil.

Result is that although the build completed without error, when I hooked it up to Arena, protobuf refused to run, so lc0 wouldn’t start.

So, for now I’ll stay with what works… although I probably will find some spare time to pin down exactly what is minimally required.

TSU

#19

On Sat 12 Jan 2019 10:46:03 PM CST, tsu2 wrote:

malcolmlewis;2891397 Wrote:
> Hi
> Well the Build Service is only using the default repositories (oss),
> so should not need any of the development ones… perhaps this is
> introducing issues at your end with leap 15.0 and the blas backend?
>
> I’ve tested on Leap 15 and use the backend switch as well as the demux
> backend and batching for blas all seems to do it’s thing.
>
> I’ve also played with building just a blas default, but no changes so
> rolled back.

Actually, I don’t have any problems with the blas backend when lc0 is
generated by my script, and using your Tumbleweed RPM. The problem is
using your LEAP RPM, and only with the Arena frontend (no problem with
Cute Chess board). This almost certainly means that the issue is
specific to the Arena frontend (one of, if not the primary board used by
serious players today) and you won’t see the problem testing only lc0 or
with other frontends. Unless I modify my script to compile OpenCL as
well as blas, I can’t know if the problem is only with Arena or if your
RPM is at fault.

It’s hard for me to remember exactly what I encountered earlier on when
I was first trying to get things to work, but I just ran a check just
now running my script, modified to not add the LEAP repos and without
python3-abseil.

Result is that although the build completed without error, when I hooked
it up to Arena, protobuf refused to run, so lc0 wouldn’t start.

So, for now I’ll stay with what works… although I probably will find
some spare time to pin down exactly what is minimally required.

TSU

Hi
Full list of build requires in addition to standard OBS build
requirements;


BuildRequires:  clang
BuildRequires:  gtest
BuildRequires:  meson
BuildRequires:  ninja
BuildRequires:  pkg-config
BuildRequires:  openblas-devel
BuildRequires:  pkgconfig(libprofiler)
BuildRequires:  pkgconfig(protobuf)
BuildRequires:  pkgconfig(zlib)

For OpenCL add
BuildRequires:  opencl-headers
BuildRequires:  pkgconfig(OpenCL)

–
Cheers Malcolm °¿° SUSE Knowledge Partner (Linux Counter #276890)
SLES 15 | GNOME Shell 3.26.2 | 4.12.14-25.25-default
If you find this post helpful and are logged into the web interface,
please show your appreciation and click on the star below… Thanks!

#20

Testing by chessplayers needed
Particularly using the RPM which builds without error but sometimes might have a problem hooking up to a chessboard.
The early feedback you can provide might promote openSUSE as a platform of choice by future players and developers!

And of course I’m also looking to improve the installation instructions.
I don’t know if non-chessplayers will understand enough of the app to analyze and evaluate functionality, though.

Your additional benefit is that you’ll have an Engine that’s among the strongest today, is cutting edge and is the only Engine capable of GPU computing (although currently only AMD support is compiled for now)

I would consider current status of the instructions “late Beta,” ie next to Final Release quality.

Visit the link in the above quote to get started.

TIA,
TSU