Scott's Nearly Full Motion Video Manifesto
Hackers don't write documentation, or plan out their programming,
instead they write Manifestos.
This isn't really a manifesto, but rather, a collection of notes.
Newer items are at the bottom of the page
25 May 1999
The FMV Pipeline
Flowchart of the compressor
|
|
26 May 1999
Let it be known. I know little to nothing about doing this.. But I feel that
that's the best way to walk into a project... I'll be bringing in a new
viewpoint that may not have been done before... I certainly won't be thinking
along the lines of "well, Y person did it this way, so thats how I should
do it." Granted my method might be vastly inferior, but it's mine. :]
Here are links to my old notes from "the demo" (mid 1997).
First, my Concept.
Next, my e-mailings with Virne from the demo group
"Coma", who coded the FMV sequeces in their "Washingmachine", Agent 42, and
many other demos..
Then my sanity communications with Pat Fleck.
And some preliminary design forfile block headers.
Some of the info there is okay, and some is
just plain wrong. :P But, it's a good starting point. I guess i need to
do some research into the proper proportions to create the Y component.
Just found the following info from
http://disney.ctr.columbia.edu/jrsthesis/thesissmall.html (section 2.7.2)
and
http://ourworld.compuserve.com/homepages/compuphase/cmetric.htm.
YUV, YIQ, YCrCb Color Transforms
 |
Y |
 |
= |
 |
0.299 |
0.587 |
0.114 |
 |
  |
 |
R' |
 |
U |
-0.147 |
-0.289 |
0.463 |
G' |
V |
0.615 |
-0.515 |
-0.100 |
B' |
The RGB->YUV color transform
Used for PAL & SECAM
 |
Y |
 |
= |
 |
0.299 |
0.587 |
0.114 |
 |
  |
 |
R' |
 |
I |
0.596 |
-0.274 |
-0.322 |
G' |
Q |
0.211 |
-0.523 |
0.312 |
B' |
The RGB->YIQ color transform
Used for NTSC
 |
Y |
 |
= |
 |
0.2990 |
0.5870 |
0.1140 |
 |
  |
 |
R' |
 |
Cr |
0.5000 |
-0.4187 |
-0.0813 |
G' |
Cb |
-0.1687 |
-0.3313 |
0.5000 |
B' |
The RGB->YCrCb color transform
Used for JPEG
I'm not sure which of these i'm gonna be going with... I'll need a good
test application to find out for sure. I'll probably go with a simplified
YUV method or something..
28 May 1999
My gameplan / work order
- DOS DJGPP/Allegro test/demo application
- selectable YUV / YIQ / YCrCb encoding/decoding
- view each component seperately
- adjust levels of components
- adjust resolution of components
- Simple video playback engine of just Luminance
- nail down file format
- Simple encoder
- Simple decoder (256 color paletted)
- add in artifical throttling to simulate processor load
- Video playback engine of all components
- 16 bit depth playback
- full component encoder
- full component decoder
- Audio streaming
- wave file playback (acts as timing mechanism as well)
- non-compressed
- Audio compression
- wave file compression routines
- wave file decompression routines
29 May 1999
More formula like things...
Y = (R * .30) + (G * .59) + (B * .11)
U = Y - R
V = Y - B
R = Y - U
B = Y - V
G = 3Y - R - B
This might work. the G = 3Y - R - B assumes that the Y component was
computed using Y = (R+G+B)/3, but I'll be testing it out to see how
it fares... who knows... it might look fine.
26 May 2000
some samples & sample code
Almost a year since I last touched this, and i had an urge to try out
the above formulas. All of these images have been converted to .jpg
format with very little compression. The artifacts seen in the U
channel are actually from the original. I thought this image looked
good. Shows ya how much our eye can be faked out. hehe. The source
code that generated this can be found in this
.tar archive.
It should be noted that if you encode with one of the Y methods, then
decode with the alternate Y method you get clipped or out of range values.
So a matching pair is needed. Not that it seems it matters. there are
very slight differences between the results and the original. So slight,
they probably won't be visible here. Contrast is slightly different, as
well as the colors are a little more "rich". IMO, it looks fine.
Here's the interesting one. On a whim i did a divide by 4 in the encode,
and a multiply by 4 in the decode to see how it works. The Y ended up with
a smaller contrast range, while the U & V have a larger range. HOWEVER,
this math is a lot easier to do. Instead of a multiply by 3 (and a divide
by 3) there is now a shift right by 2 bits ">>2" (and shift left
by 2 bits "<< 2".).
Description |
Y = (R * .30) + (G * .59) + (B * .11) |
Y = (R+G+B)/3 |
Y = (R+G+B)>>2 (divide by 4) |
Originals |
 |
 |
 |
Results |
 |
 |
 |
Y component |
 |
 |
 |
U component |
 |
 |
 |
V component |
 |
 |
 |
||
Main Home
||
CSH Home
||