When Daleron and I decided that we wanted to hold a limited-access Beta Test for Zulu Hotel, we considered a number of different approaches, and in this post I will compare and contrast the three main potential courses of action. But first, a quick look at the requirements of any solution.

Our priorities for Beta Testing are, in no particular order:

  1. Limit access to a reasonable number of testers so that it's easier to manage.
  2. Avoid unnecessary complexity which can lead to an increase in attack surface size or number of potential vulnerabilities, and adds a time/opportunity cost if it breaks.
  3. Get a diverse group of testers involved, to avoid groupthink and tunnel-vision.
  4. Minimize ongoing work requirements, i.e. we wanted a "set it and forget it"-type of solution.

It turns out that there aren't a lot of solutions out there that tick all of those boxes, but here are a few we looked at.

The first and most straightforward is to manually hand out access, in the form of account credentials. The admin command can create accounts with easy temporary passwords, and there's a password command for players to secure their accounts after the fact. However this solution doesn't really scale well. Quite frankly, each of us have better things to do than copy-pasting account credentials into IRC or an email (like, say, developing. Or our actual jobs).

So that option went right out the window. The next thing we considered was, well, maybe we can create a web-based account request form where you register with a username/password combo and it requires a special code to complete the registration. I liked this idea but the more we discussed it the more it seemed like it would balloon out of control (criterion #2) and we'd still have to find a way to distribute beta invite keys on reddit or other venues (criterion 4).

On top of that, UO shards have a history of—shall we say—brittle connections with web-based tools. Web-based account creation modules used to be an easy way to launch denial-of-service attacks on POL-based shards, and frankly I'd rather have fewer web-facing gateways anyway.

We toyed briefly with maybe hacking the client to take a third credential (i.e. username, password, and access code) but that would be an enormous amount of work, and so on a random weeknight back in August I decided. Fuck it. We'll leave auto-account on, and delete all the starting locations at character generation time, so that you can only start in The Lobby, which is just a tiny little island with nothing on it off the coast of Trinsic. That's it. There's nothing else and you can't do anything there. But this is where the fun begins: A player who's already in the Beta doesn't start in the Lobby - they start in Serpent's Hold, and from Serpent's Hold you can access the Beta's single dungeon (the Fire dungeon of course, but with different spawns). The little island attached to Serpent's Hold has some "nature"-type spawns as well for training Animal Taming, Herding, and the other related skills. But every single spawn has a small chance to drop a new item, called a Fiery Moonstone which, when double-clicked, will evaporate. When it does, you will hear the spirits whisper the words of power to you.

The words of power are drawn at random from the Britannian phonetic alphabet:

That string of words you receive from the evaporating stone is the invitation code. If you as a player in-game issue the beta command you will be presented with a gump asking for your invite code. A successful code will allow one character (and only one) to leave the Lobby and join the rest of the Beta Testers. (You can see here I made a typo).

On the technical side, what's happening is that the invite stone item creates a random string from the above list and stores it into a key-value hash table (a Dictionary in C# parlance) where the string is the key, and a Boolean is the value, defaulting to false. When someone correctly enters an existing key it gets marked as true in the hash table, meaning it has been used. Could it be brute-forced or guessed? Absolutely. But even using just four words in a beta key means we have about 0.0002% chance of guessing correctly, and to brute-force all 457000 permutations in hopes of finding a key that's been generated but not used would take about 4 years. And frankly, if someone will go through all that just to get into the beta test, then that's the kind of tester I want!

The more we thought about this solution the more it makes sense: It's low-maintenance, it encourages play (friend wants an invite code? Go literally hunt for one!) as opposed to AFKing, it doesn't expose any additional attack surface, and when we're done Beta I'll just simply set Core.Beta = false; and the items will stop dropping in loot. Minimal, mostly in-game driven, etc.

Stay tuned - next I'll be writing about our new combat minigame.