Latest revision as of 11:33, 13 March 2019

GStreamer is a modular node based player as well as encoder in a single application. It is possible to create chains of so called elements with a wide range of different plugins.

GStreamer

GStreamer is a library for constructing graphs of media-handling components. The applications it supports range from simple Ogg/Vorbis playback, audio/video streaming to complex audio (mixing) and video (non-linear editing) processing.

Applications can take advantage of advances in codec and filter technology transparently. Developers can add new codecs and filters by writing a simple plugin with a clean, generic interface.

Installing GStreamer

NicoLargo made a script to automatically install all the required packages for GStreamer on Ubuntu 10.04. Should work for up to (K)Ubuntu 16.04 as well. Or:

sudo apt-get install gstreamer1.0-tools gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly

Tips

How to get 25 FPS ?

Depends on the sensor and the exposure:

In the sensor's datasheet find resolutions that allow the required FPS.
You won't get 25 fps if autoexposure is on and local brightness not high enough: the camera will automatically lower framerate for keeping clear picture. Either lighten up, or play with image settings (notably, gain).

GStreamer plugins

All of the plugins are listed here.
Alternative way of learning a plugin's parameters is:
- version 1.0

gst-inspect-1.0 <name>

- version 0.10

gst-inspect-0.1 <name>

Command line examples

Note: replace width and height accordingly to your camera setup and your computer's horsepower :P)

GStreamer 1.0 (10393 & 10353)

Display

Multipart JPEG (mjpeg), 1 channel

gst-launch-1.0 souphttpsrc is-live=true location=http://192.168.0.8:2323/mimg ! jpegdec ! xvimagesink

Notes:

10393 ports are 2323-2326
10353 port is 8081

Multipart JPEGs (mjpeg), 4 channels in 1 window

Each source is resized to 640x480
Text overlay and borders added

display4in1.sh:

#!/bin/sh
LOC1="http://192.168.0.8:2323/mimg"
LOC2="http://192.168.0.8:2324/mimg"
LOC3="http://192.168.0.8:2325/mimg"
LOC4="http://192.168.0.8:2326/mimg"

LABEL1="CHN1"
LABEL2="CHN2"
LABEL3="CHN3"
LABEL4="CHN4"

#souphttpsrc
SOUP="souphttpsrc is-live=true"
#image parameters
PI="image/jpeg,width=1,height=1,framerate=1000/1,pixel-aspect-ratio=1/1"
#video parameters
PV="videoscale ! video/x-raw,width=640,height=480"
#videobox parameters
PVB="videobox fill=Black top=-4 bottom=-2 left=-4 right=-2 border-alpha=0.5"
#textoverlay parameters
PTO="textoverlay font-desc=\"Sans 24\" shaded-background=true valignment=top halignment=left"

gst-launch-1.0 -ve videomixer name=mix \
    sink_0::alpha=1 sink_0::xpos=0   sink_0::ypos=0 \
    sink_1::alpha=1 sink_1::xpos=646 sink_1::ypos=0 \
    sink_2::alpha=1 sink_2::xpos=0   sink_2::ypos=486 \
    sink_3::alpha=1 sink_3::xpos=646 sink_3::ypos=486 \
    ! \
    xvimagesink sync=false \
    $SOUP location=$LOC1 ! $PI ! jpegdec ! $PV ! $PVB ! $PTO text=$LABEL1 ! mix. \
    $SOUP location=$LOC2 ! $PI ! jpegdec ! $PV ! $PVB ! $PTO text=$LABEL2 ! mix. \
    $SOUP location=$LOC3 ! $PI ! jpegdec ! $PV ! $PVB ! $PTO text=$LABEL3 ! mix. \
    $SOUP location=$LOC4 ! $PI ! jpegdec ! $PV ! $PVB ! $PTO text=$LABEL4 ! mix.

Live RTSP stream

Notes:

careful with streams at higher resolution than 1920x1088
10353 - works
10393 - works

native resolution

gst-launch-1.0 rtspsrc location=rtsp://192.168.0.9:554 ! rtpjpegdepay ! jpegdec ! xvimagesink sync=false

resized, 640x480

gst-launch-1.0 rtspsrc location=rtsp://192.168.0.9:554 ! rtpjpegdepay ! jpegdec ! videoscale ! video/x-raw,width=640,height=480 ! xvimagesink sync=false

GStreamer 0.10 (10353)

Display

Notes:

Careful with streams at higher resolution than 1920x1088

Live RTSP stream

native resolution

gst-launch-0.10 rtspsrc location=rtsp://192.168.0.9:554 ! rtpjpegdepay ! jpegdec ! queue ! ffmpegcolorspace ! xvimagesink sync=false

resized, fit in window

gst-launch-0.10 rtspsrc location=rtsp://192.168.0.9:554 ! rtpjpegdepay ! jpegdec ! queue ! ffmpegcolorspace ! videoscale ! xvimagesink sync=false

Dumping

mjpeg dumping

gst-launch -v rtspsrc location=rtsp://192.168.0.9:554 ! queue ! rtpjpegdepay ! \
    videorate ! capsfilter caps = "image/jpeg, framerate=(fraction)25/1, width=1024, height=768" ! \
    queue ! matroskamux ! filesink location=/tmp/test.mkv

YUV Dumping

gst-launch -v rtspsrc location=rtsp://192.168.0.9:554 ! queue ! rtpjpegdepay ! queue ! jpegdec ! queue ! \
    videorate ! capsfilter caps="video/x-raw-yuv, format=(fourcc)I420, width=(int)1024, height=(int)768, framerate (fraction)25/1" ! \
    queue ! avimux ! filesink location=/tmp/test.avi

Dump transcoding example

gst-launch filesrc location=test.mkv ! matroskademux ! queue ! jpegdec ! queue ! theoraenc bitrate=4000 ! queue ! oggmux ! filesink location=test.ogg

Live encoding

gst-launch -v rtspsrc location=rtsp://192.168.0.9:554 ! queue ! rtpjpegdepay ! queue ! jpegdec ! queue ! \
    videorate ! capsfilter caps="video/x-raw-yuv, format=(fourcc)I420, width=(int)1024, height=(int)768, framerate=(fraction)25/1" ! \
    queue ! theoraenc bitrate=4000 ! queue ! oggmux ! filesink location=/tmp/test1024.ogg

Notes:

I did some benchmarks; a Core 2 Quad Q6600 (2.4 Ghz) is not powerful enough for h264 encoding @fullHD resolution (using 4 treads).

Useful links

Some other examples

Simply playing stream

gst-launch-1.0 -v playbin uri=rtsp://192.168.0.9:554 uridecodebin0::source::latency=0

Receive rtsp stream and re-stream as rtp:

gst-launch-1.0 -e rtspsrc location='rtsp://192.168.0.9:554' latency=0 ! udpsink host=127.0.0.1 port=5000

Receive rtsp stream then re-stream as rtp and display at the same time

gst-launch-1.0 -e rtspsrc location='rtsp://192.168.0.37:554' latency=0 ! rtpjpegdepay ! jpegdec ! timeoverlay ! tee name=t ! queue ! xvimagesink t. ! jpegenc ! rtpjpegpay ! udpsink host=127.0.0.1 port=5000

To play that rtp with vlc one would need an sdp file (somefile.sdp):

v=0
m=video 5000 RTP/AVP 26
c=IN IP4 127.0.0.1
a=rtpmap:26 JPEG/90000

To play with vlc:

vlc somefile.sdp --network-caching=0

To play with gstreamer (sdp is not required):

gst-launch-1.0 udpsrc uri=udp://127.0.0.1:5000 ! rtpjpegdepay ! jpegdec ! autovideosink

Latency tests

Test 1: vlc relative to gstreamer

Camera:

elphel393~# /usr/bin/str

PC, console 1 - restream and display at the same time:

~$ gst-launch-1.0 -e rtspsrc location='rtsp://192.168.0.9:554' latency=0 ! rtpjpegdepay ! jpegdec ! timeoverlay ! tee name=t ! queue ! xvimagesink t. ! jpegenc ! rtpjpegpay ! udpsink host=127.0.0.1 port=5000

PC, console 2 - run vlc:

~$ vlc test.sdp --network-caching=0

where test.sdp:

v=0
m=video 5000 RTP/AVP 26
c=IN IP4 127.0.0.1
a=rtpmap:26 JPEG/90000

@@ Line 1: / Line 1: @@
-= Gstreamer and live video processing over the network =
+GStreamer is a modular node based player as well as encoder in a single application. It is possible to create chains of so called elements with a wide range of different plugins.
-Gstreamer is very suited to live video and audio processing, notably for live decoding/encoding, audio muxing.
+= GStreamer =
-= Requirements =
+GStreamer is a library for constructing graphs of media-handling components. The applications it supports range from simple Ogg/Vorbis playback, audio/video streaming to complex audio (mixing) and video (non-linear editing) processing.
-You will need
+Applications can take advantage of advances in codec and filter technology transparently. Developers can add new codecs and filters by writing a simple plugin with a clean, generic interface.
-* a fairly recent gstreamer distribution, with the rtpjpegdepay plugin
-* a relatively powerful computer to decode real time
-= Limitations =
+= Installing GStreamer =
+* [http://blog.nicolargo.com/ NicoLargo] made [http://blog.nicolargo.com/2010/05/installation-de-tout-les-composants-gstreamer-sous-ubuntu.html a script] to automatically install all the required packages for GStreamer on Ubuntu 10.04. Should work for up to (K)Ubuntu 16.04 as well. Or:
+<font size='2'>
+ sudo apt-get install gstreamer1.0-tools gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly
+</font>
-You will not (at this time) be able to decode the elphel 353 cameras on resolutions > 1920x1080 (which is the highest resolution you can reach with 25 fps anyway), because of the RTP payloading limits, with 25 fps anyway.
+= Tips =
+== How to get 25 FPS ? ==
+Depends on the sensor and the exposure:
+* In the sensor's datasheet find resolutions that allow the required FPS.
+* You won't get 25 fps if autoexposure is on and local brightness not high enough: the camera will automatically lower framerate for keeping clear picture. Either lighten up, or play with image settings (notably, gain).
-If you don't, you'll just have to build it yourself (see [http://www.pitivi.org/wiki/GStreamer_CVS_Setup_Page this guide from PiTiVi]).
+== GStreamer plugins ==
+* All of the plugins are listed [https://gstreamer.freedesktop.org/documentation/plugins.html here].
+* Alternative way of learning a plugin's parameters is:
+** version 1.0
+<font size='2'>
+ gst-inspect-1.0 <name>
+</font>
+** version 0.10
+<font size='2'>
+ gst-inspect-0.1 <name>
+</font>
-= Tips =
+= Command line examples =
+<b>Note: replace width and height accordingly to your camera setup and your computer's horsepower :P)</b>
+== GStreamer 1.0 (10393 & 10353)==
+===Display===
+====Multipart JPEG (mjpeg), 1 channel====
+<font size='2'>
+ gst-launch-1.0 souphttpsrc is-live=true location=http://192.168.0.8:2323/mimg ! jpegdec ! xvimagesink
+</font>
+Notes:
+* 10393 ports are 2323-2326
+* 10353 port is 8081
+====Multipart JPEGs (mjpeg), 4 channels in 1 window====
+* Each source is resized to 640x480
+* Text overlay and borders added
+''display4in1.sh'':
+<font size='2'>
+ #!/bin/sh
+ LOC1="http://192.168.0.8:2323/mimg"
+ LOC2="http://192.168.0.8:2324/mimg"
+ LOC3="http://192.168.0.8:2325/mimg"
+ LOC4="http://192.168.0.8:2326/mimg"
+ LABEL1="CHN1"
+ LABEL2="CHN2"
+ LABEL3="CHN3"
+ LABEL4="CHN4"
+ #souphttpsrc
+ SOUP="souphttpsrc is-live=true"
+ #image parameters
+ PI="image/jpeg,width=1,height=1,framerate=1000/1,pixel-aspect-ratio=1/1"
+ #video parameters
+ PV="videoscale ! video/x-raw,width=640,height=480"
+ #videobox parameters
+ PVB="videobox fill=Black top=-4 bottom=-2 left=-4 right=-2 border-alpha=0.5"
+ #textoverlay parameters
+ PTO="textoverlay font-desc=\"Sans 24\" shaded-background=true valignment=top halignment=left"
+ gst-launch-1.0 -ve videomixer name=mix \
+     sink_0::alpha=1 sink_0::xpos=0   sink_0::ypos=0 \
+     sink_1::alpha=1 sink_1::xpos=646 sink_1::ypos=0 \
+     sink_2::alpha=1 sink_2::xpos=0   sink_2::ypos=486 \
+     sink_3::alpha=1 sink_3::xpos=646 sink_3::ypos=486 \
+     ! \
+     xvimagesink sync=false \
+     $SOUP location=$LOC1 ! $PI ! jpegdec ! $PV ! $PVB ! $PTO text=$LABEL1 ! mix. \
+     $SOUP location=$LOC2 ! $PI ! jpegdec ! $PV ! $PVB ! $PTO text=$LABEL2 ! mix. \
+     $SOUP location=$LOC3 ! $PI ! jpegdec ! $PV ! $PVB ! $PTO text=$LABEL3 ! mix. \
+     $SOUP location=$LOC4 ! $PI ! jpegdec ! $PV ! $PVB ! $PTO text=$LABEL4 ! mix.
+</font>
+====Live RTSP stream====
+Notes:
+* careful with streams at higher resolution than 1920x1088
+* 10353 - works
+* <b>10393 - works</b>
+=====native resolution=====
+<font size='2'>
+ gst-launch-1.0 rtspsrc location=rtsp://192.168.0.9:554 ! rtpjpegdepay ! jpegdec ! xvimagesink sync=false
+</font>
+=====resized, 640x480=====
+<font size='2'>
+ gst-launch-1.0 rtspsrc location=rtsp://192.168.0.9:554 ! rtpjpegdepay ! jpegdec ! videoscale ! video/x-raw,width=640,height=480 ! xvimagesink sync=false
+</font>
+== GStreamer 0.10 (10353)==
+=== Display ===
+Notes:
+* Careful with streams at higher resolution than 1920x1088
-You won't get 25 fps if autoexposure is on and local brightness not high enough: the camera will automatically lower framerate for keeping clear picture. Either lighten up, or play with image settings (notably, gain).
+====Live RTSP stream====
+=====native resolution=====
+<font size='2'>
+ gst-launch-0.10 rtspsrc location=rtsp://192.168.0.9:554 ! rtpjpegdepay ! jpegdec ! queue ! ffmpegcolorspace ! xvimagesink sync=false
+</font>
-= Command line =
+=====resized, fit in window=====
-== Displaying ==
+<font size='2'>
+ gst-launch-0.10 rtspsrc location=rtsp://192.168.0.9:554 ! rtpjpegdepay ! jpegdec ! queue ! ffmpegcolorspace ! videoscale ! xvimagesink sync=false
+</font>
-gst-launch -v rtspsrc location=rtsp://elphel:554 ! queue ! rtpjpegdepay !  queue ! jpegdec ! queue ! xvimagesink sync=false
+=== Dumping ===
+==== mjpeg dumping ====
+<font size='2'>
+ gst-launch -v rtspsrc location=rtsp://192.168.0.9:554 ! queue ! rtpjpegdepay ! \
+     videorate ! capsfilter caps = "image/jpeg, framerate=(fraction)25/1, width=1024, height=768" ! \
+     queue ! matroskamux ! filesink location=/tmp/test.mkv
+</font>
-== Dumping ==
+==== YUV Dumping ====
-=== mjpeg dumping ===
+<font size='2'>
+ gst-launch -v rtspsrc location=rtsp://192.168.0.9:554 ! queue ! rtpjpegdepay ! queue ! jpegdec ! queue ! \
+     videorate ! capsfilter caps="video/x-raw-yuv, format=(fourcc)I420, width=(int)1024, height=(int)768, framerate (fraction)25/1" ! \
+     queue ! avimux ! filesink location=/tmp/test.avi
+</font>
-gst-launch -v rtspsrc location=rtsp://elphel:554 ! queue ! rtpjpegdepay ! videorate ! capsfilter caps = "image/jpeg, framerate=(fraction)25/1, width=1024, height=768" ! queue ! matroskamux ! filesink location=/tmp/test.mkv
+=== Dump transcoding example ===
+<font size='2'>
+ gst-launch filesrc location=test.mkv ! matroskademux ! queue ! jpegdec ! queue ! theoraenc bitrate=4000 ! queue ! oggmux ! filesink location=test.ogg
+</font>
+=== Live encoding ===
+<font size='2'>
+ gst-launch -v rtspsrc location=rtsp://192.168.0.9:554 ! queue ! rtpjpegdepay ! queue ! jpegdec ! queue ! \
+     videorate ! capsfilter caps="video/x-raw-yuv, format=(fourcc)I420, width=(int)1024, height=(int)768, framerate=(fraction)25/1" ! \
+     queue ! theoraenc bitrate=4000 ! queue ! oggmux ! filesink location=/tmp/test1024.ogg
+</font>
+Notes:
+* I did some benchmarks; a Core 2 Quad Q6600 (2.4 Ghz) is not powerful enough for h264 encoding @fullHD resolution (using 4 treads).
-=== YUV Dumping ===
+=Useful links=
+* [http://wiki.oz9aec.net/index.php/Gstreamer_cheat_sheet GStreamer cheat sheet]
+* [http://processors.wiki.ti.com/index.php/Example_GStreamer_Pipelines Example GStreamer pipelines]
+* [https://coaxion.net/blog/2014/01/gstreamer-dynamic-pipelines/ GStreamer dynamic pipelines]
+* [https://gstreamer.freedesktop.org/data/doc/gstreamer/head/manual/html/section-dynamic-pipelines.html Dynamic pipelines]
-gst-launch -v rtspsrc location=rtsp://elphel:554 ! queue ! rtpjpegdepay ! queue ! jpegdec ! queue ! videorate ! capsfilter caps="video/x-raw-yuv, format=(fourcc)I420, width=(int)1024, height=(int)768, framerate=(fraction)25/1" ! queue ! avimux ! filesink location=/tmp/test.avi
+=Some other examples=
+* Simply playing stream
+ gst-launch-1.0 -v playbin uri=rtsp://192.168.0.9:554 uridecodebin0::source::latency=0
+* Receive rtsp stream and re-stream as rtp:
+ gst-launch-1.0 -e rtspsrc location='rtsp://192.168.0.9:554' latency=0 ! udpsink host=127.0.0.1 port=5000
+* Receive rtsp stream then re-stream as rtp and display at the same time
+ gst-launch-1.0 -e rtspsrc location='rtsp://192.168.0.37:554' latency=0 ! rtpjpegdepay ! jpegdec ! timeoverlay ! tee name=t ! queue ! xvimagesink t. ! jpegenc ! rtpjpegpay ! udpsink host=127.0.0.1 port=5000
+To play that rtp with vlc one would need an sdp file (somefile.sdp):
+ v=0
+ m=video 5000 RTP/AVP 26
+ c=IN IP4 127.0.0.1
+ a=rtpmap:26 JPEG/90000
-== Encoding ==
+To play with vlc:
+ vlc somefile.sdp --network-caching=0
+To play with gstreamer (sdp is not required):
+ gst-launch-1.0 udpsrc uri=udp://127.0.0.1:5000 ! rtpjpegdepay ! jpegdec ! autovideosink
-gst-launch -v rtspsrc location=rtsp://elphel:554 ! queue ! rtpjpegdepay ! queue ! jpegdec ! queue ! videorate ! capsfilter caps="video/x-raw-yuv, format=(fourcc)I420, width=(int)1024, height=(int)768, framerate=(fraction)25/1" ! queue ! theoraenc bitrate=4000 ! queue ! oggmux ! filesink location=/tmp/test1024.ogg
+=Latency tests=
+==Test 1: vlc relative to gstreamer==
+* Camera:
+ <font size=2>elphel393~# /usr/bin/str</font>
+* PC, console 1 - restream and display at the same time:
+ <font size=2>~$ gst-launch-1.0 -e rtspsrc location='rtsp://192.168.0.9:554' latency=0 ! rtpjpegdepay ! jpegdec ! timeoverlay ! tee name=t ! queue ! xvimagesink t. ! jpegenc ! rtpjpegpay ! udpsink host=127.0.0.1 port=5000</font>
+* PC, console 2 - run vlc:
+ <font size=2>~$ vlc test.sdp --network-caching=0</font>
+where ''test.sdp'':
+ <font size=2>v=0
+ m=video 5000 RTP/AVP 26
+ c=IN IP4 127.0.0.1
+ a=rtpmap:26 JPEG/90000</font>
-(Note: replace width and height accordingly to your camera setup and your computer's horsepower :p).
+[[Category:393]]
+[[Category:353]]

Difference between revisions of "Using gstreamer"

Latest revision as of 11:33, 13 March 2019

Contents

GStreamer

Installing GStreamer

Tips

How to get 25 FPS ?

GStreamer plugins

Command line examples

GStreamer 1.0 (10393 & 10353)

Display

Multipart JPEG (mjpeg), 1 channel

Multipart JPEGs (mjpeg), 4 channels in 1 window

Live RTSP stream

native resolution

resized, 640x480

GStreamer 0.10 (10353)

Display

Live RTSP stream

native resolution

resized, fit in window

Dumping

mjpeg dumping

YUV Dumping

Dump transcoding example

Live encoding

Useful links

Some other examples

Latency tests

Test 1: vlc relative to gstreamer

Navigation menu

Personal tools

Namespaces

Variants

Views

More

Search

Navigation

Tools