Full Motion Video on Pac-Man Hardware

Scott Lawrence

July 2007

Introduction

The goal of this project is to do something that doesn't really seem possible on completely obsolete computer hardware. I intend to play back video, even if just a second or two, on 3 MHz Z80 Pac-Man video arcade game hardware from 1980.

By using the background tile memory, which is usually used for drawing the Pac-Man maze, we can get a 32x28 bitmap area to display video data. We can also compress the video in a way that it's easy for the Z80 to uncrunch it, while saving precious ROM space, since we're limited to around 20k of ROM.

The Target Hardware

Obviously, Pac isn't an export option from Quicktime Pro, ffmpeg, or any of those fancy programs, so we need to do it ourselves. First, we should look at our target screen:

Figure 1: Screen RAM layout¹

If you look at Figure 1 above, adapted from this page¹, you can see a diagram of how the memory is layed out for the screen. As you can see, we basically have a background field that is 28 characters wide by 36 characters tall, seen in the blue, center section. We're going to stick to this region of the screen to make it easier to work with.

The way that this is handled in memory is that there is a chunk RAM, "Video RAM", which is the screen. Each byte of this ram specifies which character in the 256 available characters is to be placed on the screen. These characters can be letters, numbers, images, fruit, etc. They're all 8x8 pixels, and can have up to four colors in them, from a predetermined palette.

Each character is 8x8 pixels. We can do nifty tricks like, have some special characters that give us sub-character resolution, but we'll stick with the blocky 8x8 version. It gives us images like this:

Figure 2: Example 8x8 picture

Figure 2 shows a comic picture of myself which I rendered on Pac hardware for another project². Due to the layout of the screen memory, it is much easier to pseudo-letterbox the image and drop the top two and bottom two lines. As a quick aside, those two are addressed horizontally, right to left. Usually, this is where the score, number of lives, and such are displayed. The main block of the screen in the center is addressed vertically, top to bottom, right to left.

Since we're dropping these four lines, we now have a target screen resolution of 28x32. If we were to use sub-character 4x4 pixels, we could see a resolution of 56x64. This however would complicate the process significantly due to the way color is handled.

Color RAM for the screen is arranged in the same way as the character RAM above. Character RAM starts at 0x4000, while Color RAM starts at 0x4400 in the Z80's address space.

It should be noted that the target hardware for this is a 19" arcade game monitor. Some basic geometric math shows us that this means that the horizontal span of the monitor is 11.4", meaning that each of the above 8x8 pixels is roughly 0.407" square... or almost half-an-inch square.

RLE Encoding

In order to save space for our process, we well only store the color data for the image. We'll set the character RAM to a single character, and then change the color on it to determine the frame's image. We will also do some basic Run-Length Encoding³, to save some space, while making the video data very easy and quick to decompress to the screen, since we have limited CPU power (Z80 running at 3MHz) and want to run at 30 frames per second.

Past experience has shown that a single full frame of RLE encoded image data takes roughly 700 bytes of ROM space. Since I'm basing this on the BleuLlama Game Kernel² I have already used up about 2k bytes of ROM space. Standard Pac-Man has 16k ROM space total. Ms. Pac bootlegs have either 24k or 32k.

This means that we have 14k, 22k, or 30k of space available for image data, which works out to roughly 20, 32, or 43 frames, which is .66, 1.06 or 1.43 seconds of video footage. I can drop the frame rate down to 15 fps to double these times, but I'd rather have fluid motion.

Past experience has shown that any one of: Framerate, Color depth, or resolution can help make up for the other ones. It's my hope that with a very low color depth, and very low resolution, I can make up for it with a high framerate to get decent results.

The Conversion Process

The first step is to find a clip. I'm using a segment from "the best music video ever"⁴, as seen in Figure 3 below.

Figure 3: A Music Video in Quicktime Pro

Since we can only store a couple of seconds of video, select a small region of the video clip, and export it via Quicktime Pro as "Movie to Image Sequence". This will generate files along the lines of: "Journey01.bmp" "Journey02.bmp" and so on.

Next, we'll create a file with a representative palette whcih we'll use for all of the following images of the clip. The following does a lot of the same conversions we'll do on the bulk of the video, but we're exporting it as palette.bmp.

	% convert Journey01.bmp -colors 8 -geometry 48x32 -crop 28x32+7+0
                  -contrast -contrast palette.bmp

Next, we need to reduce the pixel count and colors to something more managable. We'll use Image Magick to do a bunch of these operations simultaneously. The following commands will be applied to all of the images, with the results dumped in a new directory named "cropped". They will be the target image size (28x32) and they will be using the above generated palette colors.

	% mkdir cropped
	% foreach i ( Journey*bmp )
	foreach? convert $i -colors 8 -geometry 48x32 -crop 28x32+7+0
		 -contrast -contrast cropped/$i
	foreach? convert cropped/$i -map palette.bmp -type TrueColor cropped/$i
	foreach? echo $i done.
	foreach? end

Next, we need to figure out the palette remapping from these images to the Pac palette that is available to us. We can have Image Magick do this for us, but it will be imprecise, and it will dither the imagery for us. This gives us much better control on how the colors map. For example, you might want to map skin tones to yellow or orange, while Image Magick will want to map them to white or some other color.

We will use the "bmp2asm" command to do this remapping and such for us. The source code ⁷ to this is in the Tools directory of the included source package. NOTE: This software only works on 24 bit TrueColor images.

	% foreach i ( *bmp )
	foreach? bmp2asm SCAN $i
	foreach? end

It will dump out a lot of garbage you can ignore. It will also generate a "palette.html" file. It will actually load this in and append any new colors it encounters for each file it loads. Hopefully, it should be only 8 colors as we specified above. If you open this in your web browser, it will look something like this:

The above table, Figure 4 consists of:
[color number] : [red] [green] [blue] : [Pac color] :

By referencing the following table in Figure 5, we can come up with a decent remapping. Edit the "palette.html" file, and set the final column to have the color code for the remapping. The following shows the "Color-1" values that we should care about. (Character 161, which we're using as our "pixel" is a solid block of "color-1".)

Based on the colors in Figure 5, we'll adjust the remap to be as seen in the following Figure 6.

The only thing left to do from here is to generate the .asm file that we can include in our project. We use the "bmp2asm" command again, this time, it will generate the code we need. This will load in the above palette file.

	% rm full.asm
	% foreach i ( *bmp )
	foreach? bmp2asm ASM $i >> full.asm
	foreach? end

The current version of bmp2asm does not do any RLE compression on the video, so the romset included below⁸ only contains 8 frames of animation in it.

Results

Due to the fact that I'm not compressing anything at all yet, I can only fit 8 frames in the "mspacmab" romset. ⁸ You can see this running in the movie seen below in Figure 7.

For reference, the results of the Image Magick steps above can be seen in Figure 8 below. As you can see, there are reduced, consistent colors, and it is in the correct pixel size for the Pac screen.

As you can see, the results are promising. From here, either a simple RLE encoding will save some memory. We could also store two colors per byte in ROM, doubling the number of frames we can store.

Original Results

The first version of the conversion procedure consisted of using Quicktime to do the image resizing and color remapping. Since there was little to no control over this process, the original results were poor.

Figure 9: Initial result

The result shown above in Figure 9 was a result of misunderstanding the BMP file format. I had the red and blue components of the colors swapped. The cyan background turned yellow. The artists' fleshtones tunrned cyan. A fixed version can be seen in Figure 10 below.

Next Steps, Future Enhancements

The items to try out next include:

• Implement CourtC's double-nibble-LUT compression
• Implement RLE compression
• Implement RLE-Nibble compression
• Lower framerates to improve duration

References, Footnotes and Downloads

1. This table was adapted from David Caldwell's Ms. Pac-Man Memory Map.
2. BleuLlama Game Kernel is a library of Z80 routines and a menu interface to easily have a collection of new games and programs on Pac-Man hardware.
3. Wikipedia's entry on Run-length encoding does a better job at explaining RLE than I could ever do.
4. Journey - "Separate Ways"⁵
5. Seven Oh Five - "Separate Ways"
6. Current romset 162k zip file
7. Source Code 476k tar-gz file