Little Go 1.4.0 and 1.4.1 released

Little Go 1.4.0 has been published on January 15 2019 on the App Store, followed-up two days later by the 1.4.1 update that contained a high-prio bugfix. Version 1.4.0 is a maintenance and bugfix release that mainly focuses on updating the project to the newest development environment, Xcode 10.1 and iOS 12.1 SDK.

Although the bugfixes are quite substantial, the amount of time and work that went into them was dwarfed by what I had to invest into just getting the project to run again under Xcode 10.1! This is why I am dreading each new Xcode release: Apple's relentless push for technical renewal is sometimes disheartening for me because I do not code for the platform for profit, and my willingness to spend large amounts of free time on this project has certainly diminished over the years. Ah well...

There are a few things related to the Xcode upgrade that I want to highlight, and a short outlook on a future version 1.5.0. Read on after the break. Or check out the App Store update notes and the GitHub release page.

Code signing

Apple changed the way how code signing works. Again?!?

I groaned when I first read about this in the Xcode release notes, because code signing has always been a major PITA in the past, and whenever I had to upgrade Xcode to a new version I could be certain that afterwards I wouldn't be able to upload my app bundle to App Store Connect (formerly iTunes Connect) without jumping through hoops. This time the change was especially unwelcome because for version 1.3.0 I had worked long on a code signing scheme that was supposed to work even after an Xcode upgrade - but of course this scheme didn't (couldn't) protect against Apple changing something about the process. So I simply knew that with Xcode 10.1 my code signing scheme would fall to pieces again.

Well, I was right about that, but at the same time I discovered that Apple also got it right this time! In Xcode there's now a new checkbox in the "General" tab of a target's settings which is labelled "Automatically manage signing", and I was pleasantly surprised to find that checking this actually works. I had to clean up the overall project settings a bit (e.g. remove UUIDs that reference a specific provisioning profile), but afterwards I didn't have a single thing to do with code signing when it came to submitting to the App Store.

Well done, Apple!

Switch to libc++

A change that I had anticipated was the switch from libstdc++ (the GNU project's implementation of the C++ standard library) to libc++ (the LLVM project's implementation). Apple has deprecated libstdc++ a long time ago, I believe it was in 2013. Being a good boy I wanted to make the switch as soon as possible after I learned of the deprecation, but an initial attempt didn't work out for reasons that I find obscure today (cf. GitHub issue 181).

Unfortunately I then decided to let things go, but as you know problems seldom go away if you ignore them.

With the release of Xcode 10.0, Apple finally followed up their deprecation warning and completely removed support for libstdc++. Well, you can't fault them really, they gave us 5-6 years warning, and under normal circumstances this should not have been a big problem, after all it's only a simple compiler switch that changes: -stdlib=libc++ instead of -stdlib=libstdc++.

Right?

Wrong! To my horror I found out that Little Go was affected by a bug in libc++ that breaks the way how Little Go communicates with Fuego over named pipes. You can read the details in GitHub issue 316. To be honest, I had already found out about this in November 2017, but again I had let things go in the hope that the libc++ bug would be fixed "soon". It wasn't, and so I had to finally come up with a solution.

The result, of which I am quite proud, is the PipeStreamBuffer class (GitHub repo link), which is a reusable custom I/O stream buffer that acts as an in-memory pipe. Other uses may be possible, but PipeStreamBuffer is designed to enable two threads within the same process to communicate with each other via a text-based protocol. Guess what? Just what I need for communicating over the text-based GTP protocol ☺.

Two PipeStreamBuffer objects now replace the two filesystem-based named pipes, freeing the project of the crippling libc++ bug with only minimal code changes necessary in Little Go and fuego-on-ios. The two main difficulties that I faced in designing and implementing PipeStreamBuffer were:

  1. To find out how the I/O stream buffer interface in the C++ Standard Library is supposed to be extended (outlined in this StackOverflow answer), and
  2. To make PipeStreamBuffer fully thread-safe.

For me the second task was much less difficult than finding out about the interface in the first place, because with sound analysis you always find a solution for multi-threading problems, whereas a lack of (understandable) documentation can be frustrating and cost much more time.

Anyway, as mentioned, the resulting PipeStreamBuffer C++ class is fully reusable and is not tied to the Little Go project at all.

An undiscovered bug in Fuego

Updating to Xcode 10.1 finally brought a so far undiscovered bug in Fuego into broad daylight. Unfortunately I didn't find out about this bug until I had already released 1.4.0 to the App Store, and then it was only thanks to two users who notified me within a few hours: Mark Spurlock who created the issue on GitHub, and Rob Wildschut who emailed me and then patiently answered my subsequent barrage of questions with many screenshots. Rob's help was invaluable in letting me reproduce the problem after only a short time.

It then took me a good many hours to find the root cause, because naturally I assumed at first that the bug must be in Little Go's code - after all that's where the changes for a new release are happening. When I read the initial problem description I was afraid that the bug might be hidden in the newly developed PipeStreamBuffer class (see previous section), but fortunately it turned out soon that the problem lies elsewhere: It was the genmove GTP command that did not execute successfully, the error message in the GTP log looked something like this: "GoUctPlayer generated illegal move: 3 B D12 (occupied)".

So this was the first inkling that something in Fuego might be going wrong.

The next problem I had to overcome is, in my opinion, a flaw in Xcode: How do you debug code in a file that is not part of your project but part of a library your project is using? In my case, how can I set a breakpoint in one of Fuego's source code files (I wanted to start with the place where the genmove GTP command is processed)? If you just open an external file and set the breakpoint, Xcode will happily ignore it because the file is not in the project. In the end I had to use a cryptic GDB command in Xcode's debug console to set the breakpoint: 

_regexp-break GoGtpEngine.cpp:461

Fortunately the debug console helps with "code" completion when you start typing the filename, and you also don't have to specify the full file path, only the name. The new breakpoint does not appear in Xcode's list of breakpoints, so you can't delete it in the UI once you no longer need it. To do that you have to resort to the debug console once more: First list all breakpoints that currently exist, note the number of the breakpoint to delete, then issue the command to delete. 

breakpoint list
[...]
breakpoint delete

Debugging soon revealed the code that generated the illegal move, but it took me a few stupid hours more to realize why the surrounding checks that should have prevented that a stone is placed on an occupied intersection did not work. In the end it was obvious, but at first I was blinded by the fact that Fuego's code is generally of a high quality and so I couldn't/wouldn't believe what was staring me in the eye. Also, at 2am my thought processes might have become a bit sluggish ☺.

To make a long story short, I fixed the issue and then worked as fast as possible to prepare another release, upload the binaries to App Store Connect and finally submit them to Apple for review. It was an all-nighter, my first for some time, but it was also kind of a satisfying experience to once more have "beaten the forces" playing against me ☺.

You can check out the bugfix in the fuego-on-ios project's GitHub issue 20 or, if you like a puzzle, try out whether you can spot what's wrong with the following code. You don't need to know any details about Fuego-specific classes, all you need is intermediate C++ knowledge and how C++ iterators usually work (big hint! ☺). 

bool IsRectEmpty(const GoBoard& bd, int left, int right, int top, int bottom)
{
    for (SgRectIterator it(SgRect(left, right, top, bottom)); it; ++it)
        if (! bd.IsEmpty(*it))
            return false;
    return true;
}

One might argue: Is this really a bug in Fuego? Isn't this a Clang compiler bug? After all the code worked flawlessly for many years. To be honest, I have no idea. Someone more experienced with the C++ standard's scoping rules probably can provide an answer, but in my opinion these questions lead the discussion in the wrong direction. I think it's safe to say that - SPOILER ALERT - using a temporary object in the way that the code above does is fishy at best. To make an analogy: Would you create, for instance, a temporary std::vector in the constructor call of the iterator that is supposed to iterate over the very same std::vector? I think not. So even if Clang's scoping rules are buggy, I think the code above is simply asking for trouble and should be cleaned up to make it clear and unambigous.

By the way, the change that I implemented in the fuego-on-ios project has meanwhile been integrated upstream.

Onwards to 1.5.0

A bit of good news is that in 1.4.0 I have managed to implement one new feature: Support for loading and saving .sgf files that contain stone and/or player setup nodes. This kind of .sgf files is frequently used for sharing board positions that teach how to play best in certain game situations, or that are puzzles to be solved.

I started work on this because I received one or two bug reports from people who tried to load such .sgf files, but were rewarded with a message such as this: "Fuego rejected the move B15 played by me. The reason given was: illegal move: 53 B B15 (occupied)." So although my main motivation for implementing the feature was to reduce the amount of work for me, the end result is a hopefully better product for the end user.

What's more, this feature is only the first half of a bigger plan: For the next version 1.5.0 I want to add UI support for seting up arbitrary game positions before the first move. I don't know how much people will appreciate this feature, maybe they would prefer some other feature to be implemented first. But on the other hand this is now pretty low-hanging fruit: The underlying model classes already support board and player setup, so adding some UI stuff on top of it shouldn't be such a big deal.

Famous last words? We will see. As usual you can track progress in the GitHub issue.