Difference between revisions of "Elphel Camera Remote Audio using GStreamer"

From ElphelWiki
Jump to: navigation, search
(bug fix)
 
Line 1: Line 1:
You can browse a better looking version of this how-to at: http://luxigo.tiddlyspot.com/#%5B%5BElphel%20Camera%20Remote%20Audio%20HOW-TO%5D%5D
+
Here is a simple gst-pipeline to record camera live stream and PC's default audio input to a Theora / Vorbis OGG file:
  
<pre>
+
  gst-launch-0.10 -e -m rtspsrc location=rtsp://192.168.0.9:554 latency=40 ! rtpjpegdepay ! queue \
----------------------------------------------------------------------------
+
  ! jpegdec ! queue ! theoraenc quality=60 name=venc  pulsesrc ! audio/x-raw-int,rate=16000,channels=1,depth=16 \
HOW TO ADD AUDIO TO YOUR ELPHEL CAMERA VIDEO STREAM USING GSTREAMER:
+
  ! audioconvert ! queue ! vorbisenc quality=0.9 name=aenc oggmux name=mux ! filesink location=test.ogg aenc. \
----------------------------------------------------------------------------
+
  ! mux. venc. ! mux.
 
 
# bug here: oggmux doesnt recognize the stream oggdemux just demuxed,
 
# you MUST transcode :( tried to contact oggmux author for it but
 
# had no success with it yet... use ogmmerge instead or transcode
 
# in the meanwhile:)
 
 
 
In two words and three lines we have to:
 
 
 
- demultiplex (demux) the incoming ogg stream to read the mjpeg stream,
 
- grab the sound from an audio source,
 
- multiplex (mux) the mjpeg and the audio streams into an ogg container.
 
 
 
Paste the script below in a new file or download the full version here,
 
change the settings, save it as "soundmux.sh", and make it executable with:
 
 
 
chmod +x soundmux.sh
 
 
 
then run it with:
 
 
 
ElphelOgm | soundmux.sh
 
 
 
 
 
Simplified soundmux.sh: (download the full version here: http://luxigo.ath.cx/elphel/gstreamer/soundmux.sh)
 
 
#!/bin/sh
 
 
 
# audio settings
 
AUDIO_DEVICE="hw:0,0"
 
AUDIO_BITRATE=48000
 
 
 
# output file name
 
OUTFILE="out.ogg"
 
 
 
DEBUG="-v -m"
 
 
 
  gst-launch-0.10 $DEBUG \
 
\
 
oggmux name=mux \
 
! filesink location="$OUTFILE" \
 
\
 
fdsrc fd=0 \
 
! queue \
 
! oggparse \
 
! oggdemux \
 
! ogmvideoparse \
 
! jpegdec \
 
! theoraenc \
 
! mux. \
 
\
 
alsasrc \
 
device="$AUDIO_DEVICE" \
 
! queue \
 
! audioconvert \
 
! vorbisenc bitrate=$AUDIO_BITRATE \
 
! mux.
 
 
 
# Description of the gst-launch command above:
 
 
 
# - the muxed stream (the output from the oggmux element) will be
 
  #  stored in a file (filesink)
 
 
 
# - The video comes from ElphelOgm through stdin (fdsrc fd=0).
 
# - The ogg stream is parsed for security (optional?) and demuxed.
 
# - The ogmvideo stream is also parsed for security (optional?).
 
# - The resulting mjpeg stream is sent to the ogg muxer through the
 
#    "mux." element (defined with "oggmux name=mux").
 
#  (This syntax could also be used for oggdemux but here there's only
 
#  a single stream in the incoming ogg container so we dont need to
 
#  complicate things here)
 
 
 
# - The sound comes from an alsa device through alsasrc,
 
# - The sound is converted in a suitable format then encoded in vorbis,
 
#  before being muxed with the video stream.
 
 
 
----------------------------------------------------------------------------
 
BROADCAST THE VIDEO MULTIPLEXED WITH AUDIO USING ICECAST 2
 
----------------------------------------------------------------------------
 
In the same time you add audio, you can resize and transcode the video stream
 
in theora in order to broadcast the resulting stream on your icecast server
 
(if your machine can handle the frame size at the given framerate).
 
 
 
Paste the script below in a new file or download the full version here,
 
change the default settings if required, then save it as "oggmicecast.sh"
 
and make it executable with:
 
 
 
chmod +x oggmicecast.sh
 
 
 
Then run it with:
 
 
 
ElphelOgm | oggmicecast.sh
 
 
 
Finally you can use mplayer to watch the stream:
 
 
 
gmplayer http://icecast:8000/elphel.ogg.m3u
 
 
 
 
 
Simplified oggmicecast.sh: (download the full version here:http://luxigo.ath.cx/elphel/gstreamer/oggmicecast.sh)
 
 
 
#!/bin/sh
 
 
 
# video settings
 
VIDEO_QUALITY=51 # between 0 and 63 (51 is ~80%)
 
WIDTH=480 # 4/3: ... 512x384, 480x360, 353x264, ...
 
HEIGHT=360 # 16/9: ... 640x360, 512x288, 384x216, ...
 
 
 
# audio settings
 
AUDIO_BITRATE=48000 # ..., 32000, 48000, 64000, ...
 
AUDIO_DEVICE="hw:0,0" # ALSA notation
 
 
 
# icecast settings
 
ICECAST_SERVER=icecast
 
ICECAST_SERVER_PORT=8000
 
ICECAST_SERVER_PW="hackme"
 
ICECAST_STREAMNAME="Elphel Camera"
 
ICECAST_MOUNTPOINT="/elphel.ogg"
 
DEBUG="-v -m"
 
 
 
# starting gstreamer
 
 
 
gst-launch-0.10 $DEBUG \
 
\
 
oggmux name=mux \
 
        ! queue \
 
! shout2send \
 
ip=$ICECAST_SERVER \
 
port=$ICECAST_SERVER_PORT \
 
password="$ICECAST_SERVER_PW" \
 
streamname="$ICECAST_STREAMNAME" \
 
protocol=3 \
 
sync=true \
 
mount="$ICECAST_MOUNTPOINT" \
 
  \
 
fdsrc fd=0 \
 
! queue \
 
! oggdemux \
 
! ogmvideoparse \
 
! jpegdec \
 
! queue \
 
! videoscale \
 
method=1 ! video/x-raw-yuv,width=$WIDTH height=$HEIGHT \
 
! theoraenc quality=$VIDEO_QUALITY \
 
! mux. \
 
\
 
alsasrc \
 
device="$AUDIO_DEVICE" \
 
! audioconvert \
 
! vorbisenc bitrate=$AUDIO_BITRATE \
 
! mux.
 
 
 
----------------------------------------------------------------------------
 
REMOTE MICROPHONE:
 
----------------------------------------------------------------------------
 
The microphone can be located on another machine, far from the camera.
 
Replace the "alsasrc" paragraph in oggmicecast.sh on the server with:
 
 
 
tcpserversrc \
 
host=local_interface_ip \
 
port=9003 \
 
protocol=1 \
 
! mux.
 
 
 
and run the script below (micsend.sh) on the client machine where you
 
plugged the microphone, giving the ip you set for the tcpserversrc host
 
on the other side as last argument:
 
 
 
micsend.sh <hostname>
 
 
 
 
 
Simplified micsend.sh: (download the full version here:http://luxigo.ath.cx/elphel/gstreamer/micsend.sh)
 
 
 
# audio settings
 
AUDIO_DEVICE="hw:0,0"
 
AUDIO_BITRATE=48000
 
 
 
# network settings
 
TCP_SERVER_PORT=9003
 
TCP_SERVER=host_running_oggmicecast
 
 
 
DEBUG="-v -m"
 
 
 
gst-launch-0.10 $DEBUG \
 
  \
 
alsasrc \
 
device=$AUDIO_DEVICE \
 
! audioconvert \
 
! queue \
 
! vorbisenc bitrate=$AUDIO_BITRATE \
 
! tcpclientsink \
 
host=$TCP_SERVER \
 
port=$TCP_SERVER_PORT \
 
max-lateness=150000000 \
 
sync=true \
 
protocol=1
 
 
 
 
 
----------------------------------------------------------------------------
 
GSTREAMER HINTS:
 
----------------------------------------------------------------------------
 
  - Use "!" pipe the output from the previous gstreamer element in the next one.
 
 
 
- Try "gst-inspect-0.10" to list available gst-launch elements (commands)
 
  and "gst-inspect-0.10 <command>" to display syntax details for a single
 
  command.
 
 
 
- An empty line in the formatted gst-launch command line layout used in this
 
  page denotes a distinct pipe.
 
 
 
----------------------------------------------------------------------------
 
KNOWN PROBLEMS:
 
----------------------------------------------------------------------------
 
Trying to use a different thread to recompress video (using gst-launch
 
brace syntax), we were not able to synchronize audio with video.
 
 
 
Using ElphelOgm-0.9.7 in unicast mode and gstreamer-0.10, we experienced
 
systematic crash in oggdemux (but i cant say for sure it never worked in
 
the past or if it is an ElphelOgm encapsulation problem or a gstreamer
 
oggdemux problem). 
 
 
 
So actually we are using ElphelOgm 0.9.9 in multicast mode, but..
 
 
 
Using ElphelOgm in multicast mode, icecast server 2.3.1, and finally
 
vlc 0.8.6 to watch the stream from the icecast server, we experienced
 
image freeze in vlc after some time of playback, with all subsequent
 
frames dropped. It was not the case using ElphelOgm-0.9.7 in unicast mode.
 
(maybe vlc had problems decoding theora because the network was overloaded
 
by many other streams and some packets were lost)
 
 
 
However MPlayer1.0rc1-0ubuntu9 did play the same stream through the same
 
overloaded switch without any problem ;)
 
 
 
Maybe you must recompile your icecast server so that it supports theora:
 
Try "ldd /usr/bin/icecast" (/usr/bin/icecast2 in ubuntu) to see if yours
 
is compiled with theora support enabled.
 
 
 
----------------------------------------------------------------------------
 
LINKS:
 
----------------------------------------------------------------------------
 
http://gstreamer.freedesktop.org/ (see GSTREAMER HINTS)
 
http://www.icecast.org/
 
http://www.mplayerhq.hu/
 
http://www.videolan.org/ (see KNOWN PROBLEMS)
 
</pre>
 

Latest revision as of 10:33, 24 January 2010

Here is a simple gst-pipeline to record camera live stream and PC's default audio input to a Theora / Vorbis OGG file:

gst-launch-0.10 -e -m  rtspsrc location=rtsp://192.168.0.9:554 latency=40 ! rtpjpegdepay ! queue \
! jpegdec ! queue ! theoraenc quality=60 name=venc  pulsesrc ! audio/x-raw-int,rate=16000,channels=1,depth=16 \
! audioconvert ! queue ! vorbisenc quality=0.9 name=aenc oggmux name=mux ! filesink location=test.ogg aenc. \
! mux. venc. ! mux.