Scott's Nearly Full Motion Video Manifesto

Hackers don't write documentation, or plan out their programming, instead they write Manifestos.

This isn't really a manifesto, but rather, a collection of notes.

Newer items are at the bottom of the page


25 May 1999

The FMV Pipeline


Flowchart of the compressor

26 May 1999

Let it be known. I know little to nothing about doing this.. But I feel that that's the best way to walk into a project... I'll be bringing in a new viewpoint that may not have been done before... I certainly won't be thinking along the lines of "well, Y person did it this way, so thats how I should do it." Granted my method might be vastly inferior, but it's mine. :]

Here are links to my old notes from "the demo" (mid 1997). First, my Concept. Next, my e-mailings with Virne from the demo group "Coma", who coded the FMV sequeces in their "Washingmachine", Agent 42, and many other demos.. Then my sanity communications with Pat Fleck. And some preliminary design forfile block headers.
Some of the info there is okay, and some is just plain wrong. :P But, it's a good starting point. I guess i need to do some research into the proper proportions to create the Y component.

Just found the following info from http://disney.ctr.columbia.edu/jrsthesis/thesissmall.html (section 2.7.2) and http://ourworld.compuserve.com/homepages/compuphase/cmetric.htm.

YUV, YIQ, YCrCb Color Transforms

Y =
0.299
0.587
0.114
   R'
U -0.147 -0.289 0.463 G'
V
0.615
-0.515
-0.100
 B'
The RGB->YUV color transform
Used for PAL & SECAM


Y =
0.299
0.587
0.114
   R'
I
 0.596 -0.274 -0.322 G'
Q
0.211
-0.523
0.312
 B'
The RGB->YIQ color transform
Used for NTSC


Y
 =
0.2990
0.5870
0.1140
   R'
Cr 0.5000 -0.4187 -0.0813 G'
Cb
-0.1687
-0.3313
0.5000
 B'
The RGB->YCrCb color transform
Used for JPEG

I'm not sure which of these i'm gonna be going with... I'll need a good test application to find out for sure. I'll probably go with a simplified YUV method or something..

28 May 1999

My gameplan / work order

  1. DOS DJGPP/Allegro test/demo application
  2. Simple video playback engine of just Luminance
  3. Video playback engine of all components
  4. Audio streaming
  5. Audio compression


29 May 1999

More formula like things...


Y = (R * .30) + (G * .59) + (B * .11)
U = Y - R
V = Y - B

R = Y - U
B = Y - V
G = 3Y - R - B

This might work. the G = 3Y - R - B assumes that the Y component was computed using Y = (R+G+B)/3, but I'll be testing it out to see how it fares... who knows... it might look fine.


26 May 2000

some samples & sample code

Almost a year since I last touched this, and i had an urge to try out the above formulas. All of these images have been converted to .jpg format with very little compression. The artifacts seen in the U channel are actually from the original. I thought this image looked good. Shows ya how much our eye can be faked out. hehe. The source code that generated this can be found in this .tar archive.

It should be noted that if you encode with one of the Y methods, then decode with the alternate Y method you get clipped or out of range values. So a matching pair is needed. Not that it seems it matters. there are very slight differences between the results and the original. So slight, they probably won't be visible here. Contrast is slightly different, as well as the colors are a little more "rich". IMO, it looks fine.

Here's the interesting one. On a whim i did a divide by 4 in the encode, and a multiply by 4 in the decode to see how it works. The Y ended up with a smaller contrast range, while the U & V have a larger range. HOWEVER, this math is a lot easier to do. Instead of a multiply by 3 (and a divide by 3) there is now a shift right by 2 bits ">>2" (and shift left by 2 bits "<< 2".).

Description Y = (R * .30) + (G * .59) + (B * .11) Y = (R+G+B)/3 Y = (R+G+B)>>2 (divide by 4)
Originals
Results
Y component
U component
V component


|| Main Home || CSH Home ||