Demosaic on client side
Contents
Background
Our actual FPGA code on the camera use very simple algorithm to calculate YCbCr from the Bayer pixels. It use just 3x3 block of neighbors. The other point is that this algorithm is time consuming and with the 5MPix sensor the FPGA became the bottleneck. So we have added a special JP4 mode what bypass the Demosaic in the FPGA and provide an image with pixels in each 16x16 macroblock what are rearranged to separate Bayer colors in individual 8x8 blocks, then encoded as monochrome. Demosaic will be applied during post-processing on the host PC. This page describe different algorithms and implementations used to provide this functionality.
Cf. wikipedia for more info on Demosaicing.
The goals
Still image processing
First reasonable goal is to be able to process still images. We need to be able to specify input and output files as parameters and also be able to use STDIN and STDOUT.
Video processing
Next step is to be able to embed this algorithm into MPlayer, VLC, FFmpeg and GSTreamer for video processing. The goal is to get the maximum sensor FPS processed on the computer. So at 5MPix it's 15 FPS, at 1920x1088 it's 30 FPS.
Algorithm
Existing
There are several algorithms that provide good results with less artifacts (see Wikipedia article) and these detailed descriptions:
Implemented
We actually did implement the Variable Number of Gradients algorithm.
Implementation
Several implementation with different numbers of software & hardware dependencies are possible.
