Demosaic on client side - Revision history

Polto: /* Gstreamer implementation */

2010-01-15T08:59:45Z

Gstreamer implementation

← Older revision		Revision as of 08:59, 15 January 2010
Line 58:		Line 58:
	== Gstreamer implementation ==		== Gstreamer implementation ==

	~~Gstreamer already offers:~~		[http://Ubicast.eu Ubicast] made a GStremer implementation also based on libdc1394. See how to use it [http://wiki.elphel.com/index.php?title=JP4#GStreamer_plugins_for_Elphel_JP4_image_and_video_processing.C2.A0 here]
	* a jpegdec element
	* a [http://~~webcvs.freedesktop.org/gstreamer/gst-plugins-bad/gst/bayer/gstbayer2rgb~~.~~c?view=markup bayer2rgb~~] ~~component, which decodes raw camera bayer (fourcc BA81) to RGB~~

	An implementation based on ~~bayer2rgb is currently in development, using~~ [http://~~libdc1394~~.~~git~~.~~sourceforge.net~~/~~git/gitweb~~.~~cgi~~?p=~~libdc1394;a=blob;f=libdc1394/dc1394/bayer~~.~~c;hb=HEAD libdc1394~~] ~~for the demosaicing~~

	~~It will be used like this:~~
	~~gst-launch rtspsrc ... ! rtpjpegdepay ! jpegdec ! bayer2rgb2 jp4=true ! xvimagesink~~

Polto: /* Implementation */

2009-07-26T15:38:53Z

Implementation

← Older revision		Revision as of 15:38, 26 July 2009
Line 52:		Line 52:
	Several implementation with different numbers of software & hardware dependencies are possible.		Several implementation with different numbers of software & hardware dependencies are possible.

	== ~~OpenCV implementation~~ ==		== AviSynth ==
			Two experimental implementations are available:
	~~== libjpeg implementation ==~~		* http://community.elphel.com/Apertus/Avisynth_DebayerFilter_forElphelCamera.zip by Steven Mingam.
			* http://community.elphel.com/files/avisynth/ by Konstantin Kim.
	== Gstreamer implementation ==		== Gstreamer implementation ==

Fthiery: /* Gstreamer implementation */

2009-07-23T15:04:12Z

Gstreamer implementation

← Older revision		Revision as of 15:04, 23 July 2009
Line 63:		Line 63:

	An implementation based on bayer2rgb is currently in development, using [http://libdc1394.git.sourceforge.net/git/gitweb.cgi?p=libdc1394;a=blob;f=libdc1394/dc1394/bayer.c;hb=HEAD libdc1394] for the demosaicing		An implementation based on bayer2rgb is currently in development, using [http://libdc1394.git.sourceforge.net/git/gitweb.cgi?p=libdc1394;a=blob;f=libdc1394/dc1394/bayer.c;hb=HEAD libdc1394] for the demosaicing

			It will be used like this:
			gst-launch rtspsrc ... ! rtpjpegdepay ! jpegdec ! bayer2rgb2 jp4=true ! xvimagesink

Fthiery: /* Gstreamer implementation */

2009-07-23T14:50:50Z

Gstreamer implementation

← Older revision		Revision as of 14:50, 23 July 2009
Line 62:		Line 62:
	* a [http://webcvs.freedesktop.org/gstreamer/gst-plugins-bad/gst/bayer/gstbayer2rgb.c?view=markup bayer2rgb] component, which decodes raw camera bayer (fourcc BA81) to RGB		* a [http://webcvs.freedesktop.org/gstreamer/gst-plugins-bad/gst/bayer/gstbayer2rgb.c?view=markup bayer2rgb] component, which decodes raw camera bayer (fourcc BA81) to RGB

	~~These may be starting points for implementing the forementioned method~~. ~~However, it might be better to stick to the standards and provide raw data contained in e~~.g. ~~YUV4MPEG4 container~~ ?		An implementation based on bayer2rgb is currently in development, using [http://libdc1394.git.sourceforge.net/git/gitweb.cgi?p=libdc1394;a=blob;f=libdc1394/dc1394/bayer.c;hb=HEAD libdc1394] for the demosaicing

Kimstik: /* Existing */

2008-11-19T15:08:43Z

Existing

← Older revision		Revision as of 15:08, 19 November 2008
Line 44:		Line 44:
	* [http://scien.stanford.edu/class/psych221/projects/99/tingchen/algodep/vargra.html Variable Number of Gradients]		* [http://scien.stanford.edu/class/psych221/projects/99/tingchen/algodep/vargra.html Variable Number of Gradients]
	* [http://web.cecs.pdx.edu/~cklin/demosaic/ Pixel Grouping]		* [http://web.cecs.pdx.edu/~cklin/demosaic/ Pixel Grouping]
			* [http://www.dei.unipd.it/~menond/pub/dfapd/dfapd.html Demosaicing with Directional Filtering and a Posteriori Decision]

	== Implemented ==		== Implemented ==
	We actually did implement the Variable Number of Gradients algorithm.		We actually did implement the Variable Number of Gradients algorithm.

Kimstik: /* Background */

2008-11-18T14:08:13Z

Background

← Older revision		Revision as of 14:08, 18 November 2008
Line 1:		Line 1:
	= Background =		= Background =
	Our actual FPGA code on the camera use very simple algorithm to calculate YCbCr from the Bayer pixels. It use just 3x3 block of neighbors. The other point is that this algorithm is time consuming and with the 5MPix sensor the FPGA became the bottleneck. So we have added a special JP4 mode what bypass the Demosaic in the FPGA and provide an image with pixels in each 16x16 macroblock what are rearranged to separate Bayer colors in individual 8x8 blocks, then encoded as monochrome. Demosaic will be applied during post-processing on the host PC. This page describe different algorithms and implementations used to provide this functionality.		Our actual FPGA code on the camera use very simple algorithm to calculate YCbCr from the Bayer pixels. It use just 3x3 block of neighbors. The other point is that this algorithm is time consuming and with the 5MPix sensor the FPGA became the bottleneck. So we have added a special [[JP4]] mode what bypass the Demosaic in the FPGA and provide an image with pixels in each 16x16 macroblock what are rearranged to separate Bayer colors in individual 8x8 blocks, then encoded as monochrome. Demosaic will be applied during post-processing on the host PC. This page describe different algorithms and implementations used to provide this functionality.

	Cf. wikipedia for more info on [http://en.wikipedia.org/wiki/Demosaicing Demosaicing].		Cf. wikipedia for more info on [http://en.wikipedia.org/wiki/Demosaicing Demosaicing].

OneArtPlease: /* Comparing different demosaicing algorithms */

2008-08-29T07:35:39Z

Comparing different demosaicing algorithms

← Older revision		Revision as of 07:35, 29 August 2008
Line 27:		Line 27:

	Detail comparison:		Detail comparison:

			[[Image:Detail01.jpg\|300% Zoom]]

			[[Image:Detail02.jpg\|300% Zoom]]

			'''Conclusion''':
			For some reason the elphel internal algorithm which should actually be identical to the UFRaw VNG falls behind all UFRaw created methods. The elphel processed image lacks sharpness and richness in detail and is similar to the bilinear interpolation. AHD, VNG and PPG algorithms processed in UFRaw show very sharp results with clearly more visible details. Noise is a bit stronger in luminosity but less in chroma. VNG has less color artifacts than AHD and PPG.

	= The goals =		= The goals =

OneArtPlease: /* Comparing different demosaicing algorithms */

2008-08-29T07:01:41Z

Comparing different demosaicing algorithms

← Older revision		Revision as of 07:01, 29 August 2008
Line 19:		Line 19:
	{\| border="0" cellpadding="2"		{\| border="0" cellpadding="2"
	\|-		\|-
	\| [[Image:1080 jpg 99.jpeg\|thumb\|JPEG processed within a Elphel 353 at 99% JPEG quality - camera uses the Variable Number of Gradients algorithm]]\|\|[[Image:1080 UFRaw Bilinear.jpg\|thumb\|UFRaw processed using the fast "Bilinear" algorithm]]\|\|[[Image:1080 UFRaw VNG.jpg\|thumb\|UFRaw processed using the ~~fast "~~VNG" algorithm]]\|\|[[Image:1080 UFRaw PPG.jpg\|thumb\|UFRaw processed using the ~~fast "~~PPG" algorithm]]\|\|[[Image:1080 UFRaw AHD.jpg\|thumb\|UFRaw processed using the ~~fast "PPG"~~ algorithm]]		\| [[Image:1080 jpg 99.jpeg\|thumb\|JPEG processed within a Elphel 353 at 99% JPEG quality - camera uses the Variable Number of Gradients algorithm]]\|\|[[Image:1080 UFRaw Bilinear.jpg\|thumb\|UFRaw processed using the fast "Bilinear" algorithm]]\|\|[[Image:1080 UFRaw VNG.jpg\|thumb\|UFRaw processed using the VNG (Variable Number of Gradients interpolation) algorithm]]\|\|[[Image:1080 UFRaw PPG.jpg\|thumb\|UFRaw processed using the PPG (Patterned Pixel Grouping) algorithm]]\|\|[[Image:1080 UFRaw AHD.jpg\|thumb\|UFRaw processed using the AHD (Adaptive Homogeneity-Directed interpolation) algorithm]]
	\|-		\|-
	\|}		\|}


	~~These~~ RAW file were ~~all~~ generated by converting JP4 encoded JPEGs to DNG and where then processed in UFRaw. (sorry for the slight difference in brightness and saturation, it was the closest I could get in UFRaw to what the camera does internally)		All RAW file were generated by converting JP4 encoded JPEGs to DNG and where then processed in UFRaw. (sorry for the slight difference in brightness and saturation, it was the closest I could get in UFRaw to what the camera does internally)

	~~'''Conclusion'''~~: Every algorithm has it's strong and weak points. AHD and VNG for example generate a sharper image with much more visible details compared to the camera internal Variable Number of Gradients but also makes noise a little more visible or adds new artifacts.		Detail comparison:

	= The goals =		= The goals =

OneArtPlease: /* Working with RAW image data */

2008-08-28T21:46:23Z

Working with RAW image data

@@ Line 11: / Line 11: @@
 * Step 3: We use the executable we just built to convert our jp4 jpeg. Usage "./elphel_dng gamma infile outfile" for example "./elphel_dng 100 jp4.jpg test.dng"
 * Done. You should now have a DNG that you can load in any raw editing software like photoshop, lightroom, aperture or GUIs for dcraw.
 = The goals =

Fthiery at 13:26, 14 August 2008

2008-08-14T13:26:36Z

← Older revision		Revision as of 13:26, 14 August 2008
Line 30:		Line 30:

	== libjpeg implementation ==		== libjpeg implementation ==

			== Gstreamer implementation ==

			Gstreamer already offers:
			* a jpegdec element
			* a [http://webcvs.freedesktop.org/gstreamer/gst-plugins-bad/gst/bayer/gstbayer2rgb.c?view=markup bayer2rgb] component, which decodes raw camera bayer (fourcc BA81) to RGB

			These may be starting points for implementing the forementioned method. However, it might be better to stick to the standards and provide raw data contained in e.g. YUV4MPEG4 container ?