UFO: Enemy Unknown (XCOM 1994) Splunked

From w.mfi
Jump to navigation Jump to search
Danial's cool explosion skull, lifted directly from XCOM 1994.

After having loved XCOM back in the 90s (creepy, engaging, and sometimes thrilling), I revisited it again mid 2005, then found Ufopaedia. Starting in September 2005 I did tons of work with others at Ufopaedia.org.

This is far and away the largest amount of work I have done on any game.

This was in the mid 2000s, long before the 2012 refresh was a twinkle in anyone's eye. XCOM seemed largely forgotten then.

Metrics on Contributions

From September 2005 to December 2016, I contributed to XcomUfopedia.com:

Characters added to Ufopaedia.Org by me, per quarter year
  1,272 edits
    571 character per edit on average
 on 292 pages (including 54 File: and 67 Talk: pages) for
726,184 net characters added to the site via edits in total 
        (not including uploaded files)

For a summary sorted by the pages I edited the most, see this.

As the graphic shows, most of the work was done late 2005 and 2006, then trailed off to a pittance by 2009, with a resurge for 50k characters in late 2009. Otherwise I have only popped in for a rare reply or edit.

In all, 62 original files were uploaded, 40 of them graphics:

     20 gif
     11 png
      9 jpg
      7 xls
      8 zip
      4 DAT (XCOM-specific data files)
      3 pdf

Colleagues in fun

BombBloke's super cool damage-displaying terrain tiles.
  • Zombie was my main partner in crime. We spent months (years!) interacting heavily, bouncing ideas off of each other and reaching eureka moments. Untold little stories of exploration, false leads, and revelations are now buried under many polished, final Ufopaedia pages. He's done ~3500 edits since starting in Oct. 2005! He's been an admin at Strategy Core for many years now. StratCore is a great place for XCOM talk, especially for the original game, and fan-based offshoots and related offerings.
  • BombBloke is super skilled with XCOM files and made priceless aids that Zombie, I, and others used to perform work such as explosives and shot damage. (Follow links in this section for a taste of it.) His incredible "self diagnosing" tile maps were amazing (see inset) - the tile itself shows a number corresponding to how much damage it received. He set up the "kill tile" (fall-back tile) for them all in a huge sequence corresponding to a little image of how much damage the tile had received. (If it took 51 damage, it would fall back through tiles saying 0 to 51 and stop there.) It doesn't get any better than that! But he did far more, as seen on his User page.
  • Danial, a bright bloke from down under who did some cool 3D modelling of XCOM alien ships which you could walk through. He worked with Zombie and I and contributed quite a bit in those days.
  • Ethereal Cereal started later than the core guys then, but still had tons of insights and additions, as his Contributions pages show. I count 1,010 from 2006 to 2011!
  • NKF tinkered with XCOM files, checking things and also uncovering important aspects of encoded data that others then ran with, enabling a lot of great work. I also remember he was always doing supporting things around Ufopaedia that made you think, Nice, NKF! Thanks! I count going on 3,000 contributions. I think he first instituted templates there.

There were many, many others of note, including Pi Masta, Seb76, Spike, Arrow_Quivershaft, BladeFireLight with his XcomUtil, Hobbes, Sfnhltb, and Zaimoni. These are folks who stood out for me personally; there were others doing great work (graphics, page organization, etc), which wasn't directly involved with game discovery.

I hope I haven't forgotten anyone I should have put here!

Also, I hope I don't sound like I am responsible for much of Ufopaedia. No way! It's huge! I've only done a tiny percent. But I have done some very important work for advancing the understanding of how the 1994 game worked.

Deep insights accompany complete understanding.
Pile those insights so high and deep you'll need a snorkel.

Experience and Skill Training

Ufopaedia's page on Experience was what got my attention and brought me to Ufopaedia in the first place. When I first arrived, this page was simple and vague, mainly describing what was already obvious. But I felt sure it would be very useful to know exactly how Experience works, so my soldiers could advance faster. So I began taking it apart with tons of notes.

From Sept. 6 to Oct. 13, 2005, I made the page four times as large (5,565 to 20,334 characters, +14,769), packing it densely with hard numbers on my brand new concepts of primary and secondary stats sets, showing the exact amounts of training needed for all the precise levels of skills I had discovered, detailing what the experience caps were, and what each increase in level accomplishes.

It also included over 4,000 additional words on "behind the scenes" for Experience Testing. I did almost all of this work all by myself.

All these concepts were completely novel insights which immediately translated into the Experience Training and better understanding of how every single skill worked. These experience-training precepts are now considered so basic to hardcore players that they are taken for granted. But not a whit of it existed before my work.

In the course of 38 days, Ufopaedia's Experience page went from a simple brochure to a textbook with manifold appendices.

And this was just the start of my Ufopaedia work.

Explosions and Damage

Explosions and Damage modelling was another huge aspect of the work we did. This was a big team effort with all the players listed in the Colleagues section here, although Zombie and I were the main protagonists. This covered so many aspects and details, it's hard to cover them all:

  • Explosions themselves, of course. While many of us contributed, I wrote quite a bit of this page. All the "ASCII" tables and accompanying text are mine. (It's a good way to pack info into a small space. Wiki tables tend to be too relaxed for very tight packing.) Danial made that nice map object table.
  • Damage to units and terrain from explosives and regular armaments.
  • Extensive work on Destroying Terrain in the game.
  • There were so many smaller topics impacted. Compare the See Also list here.
  • Check out Playing With Fire: Hacking High Explosives. Pure Mike Fay nerdical gloria in excelsis.
Equipment in XCOM. How easily can it be destroyed?

This was a vast sprawling effort with innumerable findings leading to other findings, as can be seen reading through Explosions. BombBloke's super-cool numeric tiles (inset) were so helpful to this effort.

Psionic Equations

Topo map of distance versus success rate for psionic control of a muton target

The Psionic Equations were a personal crowning achievement. Zombie and I compared notes and ideas initially, but then I did pretty much all the concrete work of that page... figured out exact calculation boundaries, then wrote up all of the equations, tables, and text. And you can see that I looked at the issue every which way, as usual (attack strength, defense strength, aliens' abilities, XCOM versus alien, alien versus XCOM, even how the official guide compared to my findings). This work bundled up understanding of this important aspect of gameplay.

Groundwork for understanding how psi skill works. Done by me.

That said, it did not reveal breakthrough insights or wrinkles. Everybody knew that ones' soldiers strength mattered, that some aliens were stronger than others, and that distance makes a difference. And it is interesting to know exactly how it works. But it doesn't particularly change how you play the game, except that you might feel more confident that you're safe with particular distances versus particular alien species.

This is as good a place as any to add that I set up the groundwork for this when I fleshed Experience Training all the way out to Psionic Skill. I performed deep sampling of how Psi Lab training works by VBA code that read results of soldiers' training directly from savegames. Only in this way did I have the high quantity of data which in turn made skill advancement crystal clear.

More. Much more.

My PDF of tables on Manufacturing Profitability.

Manufacturing Profitability

This is a topic that's near and dear to most commanders' hearts - how can I turn my bases into more funding?

When I arrived, the Ufopaedia page on manufacturing profitability had tables of numbers, but there was little explanation. For how many engineers? How many workshops? What about the cost of other components? How could we be sure the math was right, with no explanation? It was a listing of relevant (but simple) values, instead of a holistic evaluation.

I posted a serious evaluation (spreadsheet and PDF of the same) December 2005, and revisited the wiki page a number of times. While many players have contributed to this page over the years, I would like to think that my multi-faceted spreadsheet is a mainstay of this very useful page. It shows everyone every single datum and formula going into profitability calculations.

My recruit stats with bonus graphic.

Recruit Stats and Starting Countries

I made a page of statistics on new recruits to help commanders know what they're dealing with. This topic had been covered before by others (see Zombie's good work), but there wasn't much on Ufopaedia yet. I independently sampled an enormous number of savegames of raw recruits. It nailed down info for all to see... the similarity of names and apparent countries, how frequent women are, and of course the range of values for starting skills.

This had the interesting spin-off of all the name sets for all six nationalities. It was amusing to find out later that what I had considered as two sets of English names turned out to be an American and a British set. It had gone right over my head. I guess you had to be non-American to realize the difference.

Kill Modelling

Using everything we knew about weapon and explosive damage, I modelled how many hits it took to kill an alien with a given weapon. As its Intro says:

Kill Model Table 1 (partial)
This table shows how many hits it takes to kill a target with a particular weapon. For example, it takes an average of 1.65 Laser Pistol hits to kill a newbie soldier (30 Health) in coveralls (12 Front armor, 8 Side armor). These results are from computer modelling, not in-game testing. However, we think that all aspects of damage are entirely understood (fingers crossed). Each number represents an average of one million modelled kills. Across several runs of a million trials each, the averages only showed variability at the fourth or fifth significant figure.

This was intended to answer the question of which weapon is best for killing aliens slowly. You want to shoot and hit them as many times as possible because you get experience for each hit, not each kill. (First you use mind control to get them to drop their weapons.)

There were 206 output rows, each one representing a million trials in order to generate statistics modelling how likely damage will occur. So that's a fifth of a billion virtual aliens dead. All in the name of science.

My game science. :)

Research Progress Rate versus Efficiency of Scientists

Research Details

A lot of people have posted research data. I was a latecomer to this party. But I put together a nice table of Research information, and tried to make good overviews, such as the Progress Versus Efficiency graphic and estimates of total research time (and thus, how many labs you need).

Extensive SOLDIER.DAT and UNITREF.DAT work

I am not hardly the only person to work on these files, but still, I sunk tons of time into them. This included relating and showing how all the other experience and skill counters worked in these files, once we knew how they worked.

It's Too Bad She Won't Live *

Here are items that were never completed. And probably never shall be, except perhaps in some wonderful(?) alternate reality (where they probably already have been):

  • Firing Accuracy Testing, a very important issue to players. Exactly how effective are weapons? At particular distances? Versus particular aliens? Ultimately this proved to be more complex (time consuming) than I wanted to deal with, despite some valiant attempts by BombBloke to set up semi-automated scenarios for testing. It would have involved countless weapon firings, distances to target, weapon types, alien target types, do partial obstacles block?, countless savegame imports, endless data reviewing. So much work.
If you read between the lines, that Testing page shows how we're only scratching the surface, and numerous complexities and interpretations are already arising. I also seem to recall BombBloke and I had a fundamental disagreement on how accuracy must work. It just goes to show how tough a topic it is.
UPDATE 3/30/17: In a great thread on StratCore, Bomb Bloke pointed out how he went on to do considerably more on Firing Accuracy. Wow, some amazing work! Plus see THIS for EU 2012!
As he says on StratCore, the programmers probably never figured things out like this. They just made something decent that worked good enough (like everything else in the game). After that, it was others' choice to spend countless hours trying to describe it mathematically. :)
  • Kill Modelling: After I had performed the extensive kill modelling described above, and moved on to other things, comments from others showed me that I had missed something that would have been really useful: To run the model again, but this time with an emphasis on killing aliens instead of preserving them. (In the model, this means hitting the same armor facing, instead of deliberately hitting multiple faces.)
At the time I ran the model, I could have done this very easily. But I was focussed on experience training then, seeing as how I had long before mastered the game. By the time I realized I should have also run the other model, I had moved on.
There is one consolation. The results will probably not be greatly different from what they are now. It's still the same weapons, aliens, and formulas. But one still wants to have the clarity of the best modelling.
  • WORLD.DAT: Toward the end of my days, a number of us focussed on WORLD.DAT, the data which defines the earth (the rotatable globe shown on the XCOM geoscape). This included some really fantastic work by Pi Masta and Zaimoni, as I recall.
I got involved because Zombie asked me ca. September 2008 if I could determine the area of each terrain type and/or continents, as defined in the file. In theory it was straightforward because land on the globe was a series of three or four-sided figures bounded by coordinates. But ultimately it become very complicated because of a lot of different transformations. You can see many of them discussed on Talk:WORLD.DAT, such as Map Wrap problem with WORLD.DAT and World Area and Cylinder vs. Sphere.
Ultimately, although I sank some real time into it, I gave up. Sorry, Zombie my friend. If anyone can figure it out, please do!
Mvgulik's cool regional zone Geoscape map

Final Thoughts

As with some other games (Civ, Stalker), I have played XCOM so much I'm not sure I can go back. It is haunting and bittersweet to review it all now. It's like a tour through a childhood town, decades later. You once knew it so well... once were king there. At least, in your own little world. But now it's... sigh. I'm not sure you can ever go back.

*"It's too bad she won't live" is from Blade Runner. It suggests things that seem not done... but actually are. Or vice versa? The final scene of Blade Runner is here. One of my favorite scenes.