Commodore 64 'ReM' – The lost art of source code archeology

ReM

(If you are reading this from Commodore.Ninja we are aware of text formatting problems – Please access http://www.vitno.org/commodore-64-rem-the-lost-art-of-source-code-archeology/ to read the article well-formatted)

A recent article tells the story of archeologists who have discovered a prehistoric village in the Jordan Valley that appears to have been occupied by both Paleolithic foraging peoples and early Neolithic farmers around 12,000 years ago. It is only after such discovery that the real work will start where these careful professionals will dig, clean, reconstruct and catalog every tiny spec of a piece that they can find to tell an accurate story about the habits of the people that lived there so long ago.

With retro computing and retro gaming, the history is not that different. A lot of the source code written between the late 70s, and throughout the 80s only got lost never to be found again. Some of them probably still exist inside mega corporations’ vaults like Activision, which was present at the dawn of the personal computing and throughout the years had acquired an uncountable number of game studios too small to survive, or because they went out of business.

In rare cases, the developers involved in the creation of these games just happen to find old disks in their attic and with some effort, digitize then and make them available to our eyes. Jordan Mechner’s Prince of Persia for the Apple II is a well-known case or Star Raiders for the Atari 8-bit which recently found its way out from old boxes to the Internet Archive.

Most of the time, however, the only thing available is the game itself in its final binary format, perfect for us to play but no good to understand how it was made. To circumvent these situations, another type of work has been performed by a small group of people, dedicated to the Commodore 64 ReM scene.

If you visit CSDb (Commodore Scene Database) you might have seen a few game releases with source code included and, although many homebrew games were recently developed and released including their source code, some of them were designed and commercialized in the past and now are showing up with full and commented source code and build instructions.

One example of the latter is “Boulder Dashes ReM”, released in 2016 by DrHonz. This version includes not one, but many Boulder Dashes games with source code. There is also a Construction Kit for those who want to create their version of the iconic video game.

The acronym ReM stands for Re-engineer and Modify, which means that, other words, the game was carefully decompiled and the source code originated from it was analyzed and re-worked in a way that ensures it can be compiled back to the same binary, even after all the modifications it went through.

DrHonz, known in real life as Hans-Georg Busse is a what we could call a ‘digital archeologist’ who has worked relentlessly to bring back the source code of many games like Miner 2049’er, Lode Runner and the already mentioned Boulder Dash.

Although there are tools that perform the translation from binary to assembler code, the process cannot be fully automated. Sometimes the assembler source code generated is not 100% accurate. For example, if the binary has some data in it that is not executable code but, let’s say, game settings, a disassembler tool can think it is code and generate instructions where there aren’t any.

When that happens, the developer doing the ReM has to understand what the code surrounding the area does and interpret that correctly.

DrHonz explains that the process to bring binaries back to its source code form can take from two to three months, sometimes more. “Along this path I constantly assemble/disassemble the sources and compare the disassembled result with the original version just to be sure that my work is still equal with the first disassembly of the game”, adds DrHonz.

The work is considered done when DrHonz can compile his source code back to the binary, and all code is thoroughly commented and the build system is setup. The objective is that anyone who downloads the zip file containing his work can just compile the game running a simple script.

You might be thinking that is a lot of work, and indeed, it is. I asked DrHonz what motivate him to endure such task. He explains to me that everything had started since his early days with the C64 when he wanted to be able to add his own sauce in some unused memory of his favorite games. Disassembling it was just the means to achieve that.

Doing it in an actual Commodore 64 was a nightmare, but nowadays he uses a cross development environment comprising of UltraEdit, a modified version of the D64 Editor, Quick 64 to simplify the process to test the binaries and WinVice emulator to see the results. The disassembler process itself is performed by an Excel sheet that also assemble it back in a single step. To trigger all the tools he uses UltraEdit, which also is useful to compare the original with the source being re-worked.

The final step is to identify all the tables/split instructions in the code resulted from the Excel sheet process. After this is all done, the clean code is commented, packed and published.

As many other C64 demo scene initiatives, ReM is a work of love and commitment which is also an excellent source of information for people looking to start or improve their knowledge about C64 programming techniques.

If you want to know more about ReM or just drop a word to Hans, you can find him on CSDb as DrHonz and his group, Reengine & Mod. Who knows if deep inside there is a dormant archeologist inside you and soon everybody will be able to know more about how your favorite game was made!

Feel free to comment and share this article. We’d love to hear what you think!

Author: Paulo Garcia

4 thoughts on “Commodore 64 'ReM' – The lost art of source code archeology

  1. This is, easily, the _best single article_ that Commodore Is Awesome has published! Well done, Paulo – your piece of work is investigative journalism at its very finest (which is something you certainly don’t see a lot today, where most sites covering scene news just publish press-releases).

    Again, _great_ work – please keep it up, and please don’t hesitate writing a follow-up article on the art of restoring source code for the Commodore 8-bit platforms; I’d love to read more about it, and how the ReM project is coming along. In the meantime, and once again – thank you!

    1. Thanks, you just made my day! I am very happy how the article turned out to be, a lot thanks to DrHonz patience to answer my questions.

      I will try to bring more of these more often. Thanks a lot for the kind words!

  2. moin Paulo,

    it’s a major honor for me to find my work worth to be reviewed in an article on your very nice and interesting web site – wunderbar – i will definitely pass by here quite often in the future to see what you will dig out next for the public

    all the best
    hans-georg

    1. Hi Hans, it is a pleasure for me to write about your work. The reception was more than positive, with many views. Also many other websites have referred to the article. Maybe we can write a follow up in a near future! Cheers!

Leave a Reply

Your email address will not be published. Required fields are marked *