Difference between revisions of "Elphel Camera Remote Audio using GStreamer"

Revision as of 08:36, 21 July 2008

You can browse a better looking version of this how-to at: http://luxigo.tiddlyspot.com/#%5B%5BElphel%20Camera%20Remote%20Audio%20HOW-TO%5D%5D

----------------------------------------------------------------------------
HOW TO ADD AUDIO TO YOUR ELPHEL CAMERA VIDEO STREAM USING GSTREAMER:
----------------------------------------------------------------------------

# bug here: oggmux doesnt recognize the stream oggdemux just demuxed,
# you MUST transcode :( tried to contact oggmux author for it but
# had no success with it yet... use ogmmerge instead or transcode
# in the meanwhile:)

 In two words and three lines we have to:

 - demultiplex (demux) the incoming ogg stream to read the mjpeg stream,
 - grab the sound from an audio source, 
 - multiplex (mux) the mjpeg and the audio streams into an ogg container.

 Paste the script below in a new file or download the full version here,
 change the settings, save it as "soundmux.sh", and make it executable with:

	 chmod +x soundmux.sh

 then run it with:

	ElphelOgm | soundmux.sh


 Simplified soundmux.sh: (download the full version here: http://luxigo.ath.cx/elphel/gstreamer/soundmux.sh)
 
 #!/bin/sh

 # audio settings
 AUDIO_DEVICE="hw:0,0"
 AUDIO_BITRATE=48000

 # output file name
 OUTFILE="out.ogg"

 DEBUG="-v -m"

 gst-launch-0.10 $DEBUG \
 \
	oggmux name=mux \
	! filesink location="$OUTFILE" \
 \
	fdsrc fd=0 \
	! queue \
	! oggparse \
	! oggdemux \
	! ogmvideoparse \
	! jpegdec \
	! theoraenc \
 	! mux. \
 \
	alsasrc \
		device="$AUDIO_DEVICE" \
	! queue \
	! audioconvert \
	! vorbisenc bitrate=$AUDIO_BITRATE \
	! mux.

 # Description of the gst-launch command above:

 # - the muxed stream (the output from the oggmux element) will be
 #   stored in a file (filesink)

 # - The video comes from ElphelOgm through stdin (fdsrc fd=0).
 # - The ogg stream is parsed for security (optional?) and demuxed.
 # - The ogmvideo stream is also parsed for security (optional?).
 # - The resulting mjpeg stream is sent to the ogg muxer through the
 #    "mux." element (defined with "oggmux name=mux"). 
 #   (This syntax could also be used for oggdemux but here there's only
 #   a single stream in the incoming ogg container so we dont need to
 #   complicate things here) 

 # - The sound comes from an alsa device through alsasrc,
 # - The sound is converted in a suitable format then encoded in vorbis,
 #   before being muxed with the video stream.

----------------------------------------------------------------------------
BROADCAST THE VIDEO MULTIPLEXED WITH AUDIO USING ICECAST 2
----------------------------------------------------------------------------
 In the same time you add audio, you can resize and transcode the video stream
 in theora in order to broadcast the resulting stream on your icecast server
 (if your machine can handle the frame size at the given framerate).

 Paste the script below in a new file or download the full version here,
 change the default settings if required, then save it as "oggmicecast.sh"
 and make it executable with:

	 chmod +x oggmicecast.sh

 Then run it with:

	 ElphelOgm | oggmicecast.sh

 Finally you can use mplayer to watch the stream:

	gmplayer http://icecast:8000/elphel.ogg.m3u


 Simplified oggmicecast.sh: (download the full version here:http://luxigo.ath.cx/elphel/gstreamer/oggmicecast.sh) 

 #!/bin/sh

 # video settings
 VIDEO_QUALITY=51	# between 0 and 63 (51 is ~80%)
 WIDTH=480		# 4/3: ... 512x384, 480x360, 353x264, ...
 HEIGHT=360		# 16/9: ... 640x360, 512x288, 384x216, ... 

 # audio settings
 AUDIO_BITRATE=48000	# ..., 32000, 48000, 64000, ...
 AUDIO_DEVICE="hw:0,0"	# ALSA notation

 # icecast settings
 ICECAST_SERVER=icecast
 ICECAST_SERVER_PORT=8000
 ICECAST_SERVER_PW="hackme"
 ICECAST_STREAMNAME="Elphel Camera"
 ICECAST_MOUNTPOINT="/elphel.ogg"
 DEBUG="-v -m"

 # starting gstreamer

 gst-launch-0.10 $DEBUG \
 \
	oggmux name=mux \
        ! queue \
	! shout2send \
		ip=$ICECAST_SERVER \
		port=$ICECAST_SERVER_PORT \
		password="$ICECAST_SERVER_PW" \
		streamname="$ICECAST_STREAMNAME" \
		protocol=3 \
		sync=true \
		mount="$ICECAST_MOUNTPOINT" \
 \
	fdsrc fd=0 \
	! queue \
	! oggdemux \
	! ogmvideoparse \
	! jpegdec \
	! queue \
	! videoscale \
		method=1 ! video/x-raw-yuv,width=$WIDTH height=$HEIGHT \
	! theoraenc quality=$VIDEO_QUALITY \
	! mux. \
 \
	alsasrc \
		device="$AUDIO_DEVICE" \
	! audioconvert \
	! vorbisenc bitrate=$AUDIO_BITRATE \
	! mux.

----------------------------------------------------------------------------
REMOTE MICROPHONE:
----------------------------------------------------------------------------
 The microphone can be located on another machine, far from the camera.
 Replace the "alsasrc" paragraph in oggmicecast.sh on the server with:

	tcpserversrc \
		host=local_interface_ip \
		port=9003 \
		protocol=1 \
	! mux.

 and run the script below (micsend.sh) on the client machine where you
 plugged the microphone, giving the ip you set for the tcpserversrc host
 on the other side as last argument:

	micsend.sh <hostname>


 Simplified micsend.sh: (download the full version here:http://luxigo.ath.cx/elphel/gstreamer/micsend.sh)

 # audio settings
 AUDIO_DEVICE="hw:0,0"
 AUDIO_BITRATE=48000

 # network settings
 TCP_SERVER_PORT=9003
 TCP_SERVER=host_running_oggmicecast

 DEBUG="-v -m"

 gst-launch-0.10 $DEBUG \
 \
	alsasrc \
		device=$AUDIO_DEVICE \
	! audioconvert \
	! queue \
	! vorbisenc bitrate=$AUDIO_BITRATE \
	! tcpclientsink \
		host=$TCP_SERVER \
		port=$TCP_SERVER_PORT \
		max-lateness=150000000 \
		sync=true \
		protocol=1


----------------------------------------------------------------------------
GSTREAMER HINTS:
----------------------------------------------------------------------------
 - Use "!" pipe the output from the previous gstreamer element in the next one.

 - Try "gst-inspect-0.10" to list available gst-launch elements (commands)
   and "gst-inspect-0.10 <command>" to display syntax details for a single
   command.

 - An empty line in the formatted gst-launch command line layout used in this
   page denotes a distinct pipe.

----------------------------------------------------------------------------
KNOWN PROBLEMS:
----------------------------------------------------------------------------
 Trying to use a different thread to recompress video (using gst-launch
 brace syntax), we were not able to synchronize audio with video. 

 Using ElphelOgm-0.9.7 in unicast mode and gstreamer-0.10, we experienced
 systematic crash in oggdemux (but i cant say for sure it never worked in
 the past or if it is an ElphelOgm encapsulation problem or a gstreamer
 oggdemux problem).  

 So actually we are using ElphelOgm 0.9.9 in multicast mode, but..

 Using ElphelOgm in multicast mode, icecast server 2.3.1, and finally
 vlc 0.8.6 to watch the stream from the icecast server, we experienced
 image freeze in vlc after some time of playback, with all subsequent
 frames dropped. It was not the case using ElphelOgm-0.9.7 in unicast mode.
 (maybe vlc had problems decoding theora because the network was overloaded
 by many other streams and some packets were lost)

 However MPlayer1.0rc1-0ubuntu9 did play the same stream through the same
 overloaded switch without any problem ;)

 Maybe you must recompile your icecast server so that it supports theora:
 Try "ldd /usr/bin/icecast" (/usr/bin/icecast2 in ubuntu) to see if yours
 is compiled with theora support enabled.

----------------------------------------------------------------------------
LINKS:
----------------------------------------------------------------------------
 http://gstreamer.freedesktop.org/ (see GSTREAMER HINTS)
 http://www.icecast.org/
 http://www.mplayerhq.hu/
 http://www.videolan.org/ (see KNOWN PROBLEMS)