JP4
Note: the JP4 mode described here is referred as "JP46" in current 8.0 firwmare
Contents
JP4 Format
We have added a special JP4 mode that bypasses the Demosaic in the FPGA and provides an image with pixels in each 16x16 macroblock that are rearranged to separate Bayer colors in individual 8x8 blocks, then encoded as monochrome. Demosaic will be applied during post-processing on the host PC. This section describe different algorithms and implementations used to provide this functionality.
Main goals:
- compression speed improvement - possibility to obtain more high quality image (near to RAW) - drasticaly lowering data size
Different JP4 Modes in 8.X Software
only modes 0-2 can be processed with standard libjpeg:
- 0 - mono6, monochrome (color YCbCr 4:2:0 with zeroed out color componets)
- 1 - color, YCbCr 4:2:0, 3x3 pixels
- 2 - jp46 - original JP4 (from 7.X software), encoded as 4:2:0 with zeroed color components
- 3 - jp46dc, modified jp46 so each color component uses individual DC diffenential encoding
- 4 - reserved for color with 5x5 conversion (not yet implemented)
- 5 - jp4 with ommitted color components (4:0:0)
- 6 - jp4dc, similar to jp46dc encoded as 4:0:0
- 7 - jp4diff, differential where (R-G), G, (G2-G) and (B-G) components are encoded as 4:0:0
- 8 - jp4hdr, (R-G), G, G2,(B-G) are encoded so G2 can be used with high gain
- 9 - jp4fiff2, (R-G)/2, G,(G2-G)/2, (B-G)/2 to avoid possible overflow in compressed values
- 10 - jp4hdr2, (R-G)/2, G,G2,(B-G)/2
- 14 - mono, monochrome with ommitted color components (4:0:0)
Image decoding
JP4 format can be easy manipulated by MATLAB
1. Read image
I=imread('hdr02.jp4'); %read JP4 file like JPEG
,or online grab image from http like this:
I=imread('http://community.elphel.com/pictures/jp4.jpg');
,or cam:
I=imread('http://cam_ip/bimg'); %get online buffered image from cam
I=I(:,:,1); %strip color data2. Remove block grouping
II=deblock16x16(I); %deblock image
%file deblock16x16.m
function y=deblock16x16(I)
y0=uint8(zeros(size(I)));
for x=1:16:size(I,1)
for y=1:16:size(I,2)
blk16=I(x:x+15,y:y+15);
for dx=0:7
for dy=0:7
y0(x+2*dx ,y+2*dy) = blk16(dx+1,dy+1);
y0(x+2*dx+1,y+2*dy) = blk16(dx+9,dy+1);
y0(x+2*dx ,y+2*dy+1) = blk16(dx+1,dy+9);
y0(x+2*dx+1,y+2*dy+1) = blk16(dx+9,dy+9);
end
end
end
end
y=y0;
2. Demosaic image (Decode from Bayer CFA (Color Filter Array) encoded image)
J=demosaic(II,'gbrg');
3. Show image
imshow(J);
Stream decoding
JP4 stream can be decoded by mplayer. Use this [this patch]
patch . -p0 < debayer.diff in the mplayer source dir
Or download win32 binaries from sourceforge.
usage example: mplayer.exe test.avi -vf demosaic=deblock=1:method=7:pattern=3 -vo gl mencoder example: mencoder.exe test.avi -ovc lavc -lavcopts vcodec=mjpeg -o output.avi -vf demosaic=deblock=1:method=1,scale
Debayer (Demosaic) algorithm variants provided by libdc1394: - Nearest Neighbor : OpenCV library - Bilinear : OpenCV library - HQLinear : High-Quality Linear Interpolation For Demosaicing Of Bayer-Patterned Color Images, by Henrique S. Malvar, Li-wei He, and Ross Cutler, in Proceedings of the ICASSP'04 Conference. - Edge Sense II : Laroche, Claude A. "Apparatus and method for adaptively interpolating a full color image utilizing chrominance gradients" U.S. Patent 5,373,322. Based on the code found on the website http://www-ise.stanford.edu/~tingchen/ Converted to C and adapted to all four elementary patterns. - Downsample : "Known to the Ancients" - Simple : Implemented from the information found in the manual of Allied Vision Technologies (AVT) cameras. - VNG : Variable Number of Gradients, a method described in http://www-ise.stanford.edu/~tingchen/algodep/vargra.html Sources import from DCRAW by Frederic Devernay. DCRAW is a RAW converter program by Dave Coffin. URL: http://www.cybercom.net/~dcoffin/dcraw/ - AHD : Adaptive Homogeneity-Directed Demosaicing Algorithm, by K. Hirakawa and T.W. Parks, IEEE Transactions on Image Processing, Vol. 14, Nr. 3, March 2005, pp. 360 - 369.
Pattern codes: pattern=0..3 -> [RGGB, BGGR, GBRG, GRBG]
JP4 HDR
Bayer pattern look like this
|
|
|
|
The remark: all kinds of bayer patterns can be received from initial RGGB by flipping on X and/or Y.
Some sensors have possibility to set independed scale to G1 and G2. Considering that the accessible optics does not give the full permission of a sensor resolution, it can be used for increase in a dynamic range of a image sensor.
Avisynth plugin
Avisynth plugin also available
AVS script example:
LoadCPlugin("jp4.dll")
DirectShowSource("test.avi")
JP4("AHD","RGGB")
Demosaicing/debayering links
A Study of Spatial Color Interpolation Algorithms for Single-Detector Digital Cameras. Ting Chen / Stanford University
Source code:
eLynx Image Processing SDK and Lab libdc1394 Efficient, high-quality Bayer demosaic filtering on GPUs http://svn2.assembla.com/svn/ge/libgedrawing/trunk/src/ImageBayer.cpp
JP4 to DNG conversion
Credits: Dave Coffin
This Linux comanndline tool allows conversion of JP4 files into a DNGs that dcraw and Adobe Photoshop can open.
Download LibTIFF v3.8.2 a
- extract the tar.gz (this should create a new folder called "tiff-3.8.2")
Apply this patch: in terminal (first cd to path of libtiff.patch):
patch -p0 < libtiff.patch
build LibTIFF:
cd tiff-3.8.2 ./configure make sudo make install
Then compile this C program with:
gcc -o elphel_dng elphel_dng.c -ltiff
Then use the created application:
Usage: ./elphel_dng "gamma" "input.jpg" "output.dng" Example: ./elphel_dng 100 example_JP4.jpeg example.dng
Example files:
See also:
