JP4

2009-06-24T21:44:46Z

Mcd:

== JP4 format ==

So we have added a special JP4 mode that bypasses the Demosaic in the FPGA and provides an image with pixels in each 16x16 macroblock that are rearranged to separate Bayer colors in individual 8x8 blocks, then encoded as monochrome. [[Demosaic_on_client_side|Demosaic]] will be applied during post-processing on the host PC. This section describe different algorithms and implementations used to provide this functionality.

Main goals:
- compression speed improvement
- possibility to obtain more high quality image (near to RAW)
- drasticaly lowering data size

== Image decoding ==
JP4 format can be easy manipulated by [http://www.mathworks.com/matlabcentral/fileexchange/22144 MATLAB] [[Image:Fruits_jp4.jpg|thumb|JP4 image]]

1. Read image
I=imread('hdr02.jp4'); %read JP4 file like JPEG
,or online grab image from http like this:
I=imread('http://community.elphel.com/pictures/jp4.jpg');
,or cam:
I=imread('http://cam_ip/bimg'); %get online buffered image from cam

I=I(:,:,1); %strip color data
2. Remove block grouping[[Image:Fruits_jp4_deblocked.jpg|thumb|Bayer CFA encoded image]]
<code matlab>
II=deblock16x16(I); %deblock image

%file deblock16x16.m
function y=deblock16x16(I)
y0=uint8(zeros(size(I)));
for x=1:16:size(I,1)
for y=1:16:size(I,2)
blk16=I(x:x+15,y:y+15);
for dx=0:7
for dy=0:7
y0(x+2*dx ,y+2*dy) = blk16(dx+1,dy+1);
y0(x+2*dx+1,y+2*dy) = blk16(dx+9,dy+1);
y0(x+2*dx ,y+2*dy+1) = blk16(dx+1,dy+9);
y0(x+2*dx+1,y+2*dy+1) = blk16(dx+9,dy+9);
end
end
end
end
y=y0;
</code>
2. Demosaic image (Decode from Bayer CFA (Color Filter Array) encoded image)[[Image:Fruits_jp4_debayered.jpg|thumb|Decoded image]]
J=demosaic(II,'grgb');
3. Show image
imshow(J);

== Stream decoding ==
JP4 stream can be decoded by mplayer.
Use [[http://community.elphel.com/files/mplayer/debayer.diff this patch]] for glue libdc1394 and MPlayer-1.0rc2 video filter frontend.
Or download win32 binaries from sourceforge.
usage example: mplayer.exe test.avi -vf demosaic=deblock=1:method=7:pattern=3 -vo gl
mencoder example: mencoder.exe test.avi -ovc lavc -lavcopts vcodec=mjpeg -o output.avi -vf demosaic=deblock=1:method=1,scale

Debayer ([[Demosaic_on_client_side|Demosaic]]) algorithm variants provided by libdc1394:
- Nearest Neighbor : OpenCV library
- Bilinear : OpenCV library
- HQLinear : High-Quality Linear Interpolation For Demosaicing Of Bayer-Patterned
Color Images, by Henrique S. Malvar, Li-wei He, and Ross Cutler,
in Proceedings of the ICASSP'04 Conference.
- Edge Sense II : Laroche, Claude A. "Apparatus and method for adaptively interpolating
a full color image utilizing chrominance gradients"
U.S. Patent 5,373,322. Based on the code found on the website
http://www-ise.stanford.edu/~tingchen/ Converted to C and adapted to
all four elementary patterns.
- Downsample : "Known to the Ancients"
- Simple : Implemented from the information found in the manual of Allied Vision
Technologies (AVT) cameras.
- VNG : Variable Number of Gradients, a method described in
http://www-ise.stanford.edu/~tingchen/algodep/vargra.html
Sources import from DCRAW by Frederic Devernay. DCRAW is a RAW
converter program by Dave Coffin. URL:
http://www.cybercom.net/~dcoffin/dcraw/
- AHD : Adaptive Homogeneity-Directed Demosaicing Algorithm, by K. Hirakawa
and T.W. Parks, IEEE Transactions on Image Processing, Vol. 14, Nr. 3,
March 2005, pp. 360 - 369.

Pattern codes: pattern=0..3 -> [RGGB, BGGR, GBRG, GRBG]

== [[JP4 HDR]] ==
Bayer pattern look like this
{| class="wikitable"
|-
|
{| class="wikitable"align="center"
|+RGGB
|-
| bgcolor="red"|R || bgcolor="green"|G1 || bgcolor="red"|R || bgcolor="green"|G1
|-
| bgcolor="green"|G2 || bgcolor="blue"|B || bgcolor="green"|G2 || bgcolor="blue"|B
|-
| bgcolor="red"|R || bgcolor="green"|G1 || bgcolor="red"|R || bgcolor="green"|G1
|-
| bgcolor="green"|G2 || bgcolor="blue"|B || bgcolor="green"|G2 || bgcolor="blue"|B
|}
|
{| class="wikitable"align="center"
|+BGGR
|-
| bgcolor="blue"|R || bgcolor="green"|G1 || bgcolor="blue"|R || bgcolor="green"|G1
|-
| bgcolor="green"|G2 || bgcolor="red"|B || bgcolor="green"|G2 || bgcolor="red"|B
|-
| bgcolor="blue"|R || bgcolor="green"|G1 || bgcolor="blue"|R || bgcolor="green"|G1
|-
| bgcolor="green"|G2 || bgcolor="red"|B || bgcolor="green"|G2 || bgcolor="red"|B
|}
|
{| class="wikitable"align="center"
|+GBRG
|-
| bgcolor="green"|G2 || bgcolor="blue"|B || bgcolor="green"|G2 || bgcolor="blue"|B
|-
| bgcolor="red"|R || bgcolor="green"|G1 || bgcolor="red"|R || bgcolor="green"|G1
|-
| bgcolor="green"|G2 || bgcolor="blue"|B || bgcolor="green"|G2 || bgcolor="blue"|B
|-
| bgcolor="red"|R || bgcolor="green"|G1 || bgcolor="red"|R || bgcolor="green"|G1
|}
|
{| class="wikitable"align="center"
|+GRBG
|-
| bgcolor="green"|G2 || bgcolor="red"|B || bgcolor="green"|G2 || bgcolor="red"|B
|-
| bgcolor="blue"|R || bgcolor="green"|G1 || bgcolor="blue"|R || bgcolor="green"|G1
|-
| bgcolor="green"|G2 || bgcolor="red"|B || bgcolor="green"|G2 || bgcolor="red"|B
|-
| bgcolor="blue"|R || bgcolor="green"|G1 || bgcolor="blue"|R || bgcolor="green"|G1
|}
|}

The remark: all kinds of bayer patterns can be received from initial RGGB by flipping on X and/or Y.

Some sensors have possibility to set independed scale to G1 and G2.
Considering that the accessible optics does not give the full permission of a sensor resolution, it [[JP4_HDR|can be]] used for increase in a dynamic range of a image sensor.

== [http://avisynth.org/mediawiki/Main_Page Avisynth] plugin ==
Avisynth plugin also available

AVS script example:
LoadCPlugin("jp4.dll")
DirectShowSource("test.avi")
JP4("AHD","RGGB")

== Demosaic links ==
http://sourceforge.net/projects/elynx/
http://libdc1394.git.sourceforge.net/git/gitweb.cgi?p=libdc1394;a=blob;f=libdc1394/dc1394/bayer.c;hb=HEAD
http://graphics.cs.williams.edu/papers/BayerJGT09/

ElphelWiki - User contributions [en]

JP4