Difference between revisions of "Using gstreamer"
OneArtPlease (talk | contribs) (→Displaying) |
(→Live RTSP stream) |
||
(51 intermediate revisions by 3 users not shown) | |||
Line 1: | Line 1: | ||
− | + | GStreamer is a modular node based player as well as encoder in a single application. It is possible to create chains of so called elements with a wide range of different plugins. | |
− | + | = GStreamer = | |
− | + | GStreamer is a library for constructing graphs of media-handling components. The applications it supports range from simple Ogg/Vorbis playback, audio/video streaming to complex audio (mixing) and video (non-linear editing) processing. | |
− | + | Applications can take advantage of advances in codec and filter technology transparently. Developers can add new codecs and filters by writing a simple plugin with a clean, generic interface. | |
− | = | + | = Installing GStreamer = |
+ | * [http://blog.nicolargo.com/ NicoLargo] made [http://blog.nicolargo.com/2010/05/installation-de-tout-les-composants-gstreamer-sous-ubuntu.html a script] to automatically install all the required packages for GStreamer on Ubuntu 10.04. Should work for up to (K)Ubuntu 16.04 as well. Or: | ||
+ | <font size='2'> | ||
+ | sudo apt-get install gstreamer1.0-tools gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly | ||
+ | </font> | ||
− | + | = Tips = | |
− | * | + | == How to get 25 FPS ? == |
+ | Depends on the sensor and the exposure: | ||
+ | * In the sensor's datasheet find resolutions that allow the required FPS. | ||
+ | * You won't get 25 fps if autoexposure is on and local brightness not high enough: the camera will automatically lower framerate for keeping clear picture. Either lighten up, or play with image settings (notably, gain). | ||
− | = | + | == GStreamer plugins == |
+ | * All of the plugins are listed [https://gstreamer.freedesktop.org/documentation/plugins.html here]. | ||
+ | * Alternative way of learning a plugin's parameters is: | ||
+ | ** version 1.0 | ||
+ | <font size='2'> | ||
+ | gst-inspect-1.0 <name> | ||
+ | </font> | ||
+ | ** version 0.10 | ||
+ | <font size='2'> | ||
+ | gst-inspect-0.1 <name> | ||
+ | </font> | ||
− | + | = Command line examples = | |
− | + | <b>Note: replace width and height accordingly to your camera setup and your computer's horsepower :P)</b> | |
− | = | + | == GStreamer 1.0 (10393 & 10353)== |
+ | |||
+ | ===Display=== | ||
+ | ====Multipart JPEG (mjpeg), 1 channel==== | ||
+ | <font size='2'> | ||
+ | gst-launch-1.0 souphttpsrc is-live=true location=http://192.168.0.8:2323/mimg ! jpegdec ! xvimagesink | ||
+ | </font> | ||
+ | Notes: | ||
+ | * 10393 ports are 2323-2326 | ||
+ | * 10353 port is 8081 | ||
− | + | ====Multipart JPEGs (mjpeg), 4 channels in 1 window==== | |
+ | * Each source is resized to 640x480 | ||
+ | * Text overlay and borders added | ||
+ | ''display4in1.sh'': | ||
+ | <font size='2'> | ||
+ | #!/bin/sh | ||
+ | LOC1="http://192.168.0.8:2323/mimg" | ||
+ | LOC2="http://192.168.0.8:2324/mimg" | ||
+ | LOC3="http://192.168.0.8:2325/mimg" | ||
+ | LOC4="http://192.168.0.8:2326/mimg" | ||
+ | |||
+ | LABEL1="CHN1" | ||
+ | LABEL2="CHN2" | ||
+ | LABEL3="CHN3" | ||
+ | LABEL4="CHN4" | ||
+ | |||
+ | #souphttpsrc | ||
+ | SOUP="souphttpsrc is-live=true" | ||
+ | #image parameters | ||
+ | PI="image/jpeg,width=1,height=1,framerate=1000/1,pixel-aspect-ratio=1/1" | ||
+ | #video parameters | ||
+ | PV="videoscale ! video/x-raw,width=640,height=480" | ||
+ | #videobox parameters | ||
+ | PVB="videobox fill=Black top=-4 bottom=-2 left=-4 right=-2 border-alpha=0.5" | ||
+ | #textoverlay parameters | ||
+ | PTO="textoverlay font-desc=\"Sans 24\" shaded-background=true valignment=top halignment=left" | ||
+ | |||
+ | gst-launch-1.0 -ve videomixer name=mix \ | ||
+ | sink_0::alpha=1 sink_0::xpos=0 sink_0::ypos=0 \ | ||
+ | sink_1::alpha=1 sink_1::xpos=646 sink_1::ypos=0 \ | ||
+ | sink_2::alpha=1 sink_2::xpos=0 sink_2::ypos=486 \ | ||
+ | sink_3::alpha=1 sink_3::xpos=646 sink_3::ypos=486 \ | ||
+ | ! \ | ||
+ | xvimagesink sync=false \ | ||
+ | $SOUP location=$LOC1 ! $PI ! jpegdec ! $PV ! $PVB ! $PTO text=$LABEL1 ! mix. \ | ||
+ | $SOUP location=$LOC2 ! $PI ! jpegdec ! $PV ! $PVB ! $PTO text=$LABEL2 ! mix. \ | ||
+ | $SOUP location=$LOC3 ! $PI ! jpegdec ! $PV ! $PVB ! $PTO text=$LABEL3 ! mix. \ | ||
+ | $SOUP location=$LOC4 ! $PI ! jpegdec ! $PV ! $PVB ! $PTO text=$LABEL4 ! mix. | ||
+ | |||
+ | </font> | ||
− | = | + | ====Live RTSP stream==== |
+ | Notes: | ||
+ | * careful with streams at higher resolution than 1920x1088 | ||
+ | * 10353 - works | ||
+ | * <b>10393 - works</b> | ||
+ | =====native resolution===== | ||
+ | <font size='2'> | ||
+ | gst-launch-1.0 rtspsrc location=rtsp://192.168.0.9:554 ! rtpjpegdepay ! jpegdec ! xvimagesink sync=false | ||
+ | </font> | ||
− | + | =====resized, 640x480===== | |
+ | <font size='2'> | ||
+ | gst-launch-1.0 rtspsrc location=rtsp://192.168.0.9:554 ! rtpjpegdepay ! jpegdec ! videoscale ! video/x-raw,width=640,height=480 ! xvimagesink sync=false | ||
+ | </font> | ||
− | == | + | == GStreamer 0.10 (10353)== |
+ | === Display === | ||
+ | Notes: | ||
+ | * Careful with streams at higher resolution than 1920x1088 | ||
− | gst-launch-0.10 rtspsrc location=rtsp:// | + | ====Live RTSP stream==== |
+ | =====native resolution===== | ||
+ | <font size='2'> | ||
+ | gst-launch-0.10 rtspsrc location=rtsp://192.168.0.9:554 ! rtpjpegdepay ! jpegdec ! queue ! ffmpegcolorspace ! xvimagesink sync=false | ||
+ | </font> | ||
− | == | + | =====resized, fit in window===== |
− | === | + | <font size='2'> |
+ | gst-launch-0.10 rtspsrc location=rtsp://192.168.0.9:554 ! rtpjpegdepay ! jpegdec ! queue ! ffmpegcolorspace ! videoscale ! xvimagesink sync=false | ||
+ | </font> | ||
− | gst-launch -v rtspsrc location=rtsp:// | + | === Dumping === |
+ | ==== mjpeg dumping ==== | ||
+ | <font size='2'> | ||
+ | gst-launch -v rtspsrc location=rtsp://192.168.0.9:554 ! queue ! rtpjpegdepay ! \ | ||
+ | videorate ! capsfilter caps = "image/jpeg, framerate=(fraction)25/1, width=1024, height=768" ! \ | ||
+ | queue ! matroskamux ! filesink location=/tmp/test.mkv | ||
+ | </font> | ||
− | === YUV Dumping === | + | ==== YUV Dumping ==== |
+ | <font size='2'> | ||
+ | gst-launch -v rtspsrc location=rtsp://192.168.0.9:554 ! queue ! rtpjpegdepay ! queue ! jpegdec ! queue ! \ | ||
+ | videorate ! capsfilter caps="video/x-raw-yuv, format=(fourcc)I420, width=(int)1024, height=(int)768, framerate (fraction)25/1" ! \ | ||
+ | queue ! avimux ! filesink location=/tmp/test.avi | ||
+ | </font> | ||
− | gst-launch -v rtspsrc location=rtsp:// | + | === Dump transcoding example === |
+ | <font size='2'> | ||
+ | gst-launch filesrc location=test.mkv ! matroskademux ! queue ! jpegdec ! queue ! theoraenc bitrate=4000 ! queue ! oggmux ! filesink location=test.ogg | ||
+ | </font> | ||
+ | === Live encoding === | ||
+ | <font size='2'> | ||
+ | gst-launch -v rtspsrc location=rtsp://192.168.0.9:554 ! queue ! rtpjpegdepay ! queue ! jpegdec ! queue ! \ | ||
+ | videorate ! capsfilter caps="video/x-raw-yuv, format=(fourcc)I420, width=(int)1024, height=(int)768, framerate=(fraction)25/1" ! \ | ||
+ | queue ! theoraenc bitrate=4000 ! queue ! oggmux ! filesink location=/tmp/test1024.ogg | ||
+ | </font> | ||
+ | Notes: | ||
+ | * I did some benchmarks; a Core 2 Quad Q6600 (2.4 Ghz) is not powerful enough for h264 encoding @fullHD resolution (using 4 treads). | ||
− | == | + | =Useful links= |
+ | * [http://wiki.oz9aec.net/index.php/Gstreamer_cheat_sheet GStreamer cheat sheet] | ||
+ | * [http://processors.wiki.ti.com/index.php/Example_GStreamer_Pipelines Example GStreamer pipelines] | ||
+ | * [https://coaxion.net/blog/2014/01/gstreamer-dynamic-pipelines/ GStreamer dynamic pipelines] | ||
+ | * [https://gstreamer.freedesktop.org/data/doc/gstreamer/head/manual/html/section-dynamic-pipelines.html Dynamic pipelines] | ||
− | gst-launch | + | =Some other examples= |
+ | * Simply playing stream | ||
+ | gst-launch-1.0 -v playbin uri=rtsp://192.168.0.9:554 uridecodebin0::source::latency=0 | ||
+ | * Receive rtsp stream and re-stream as rtp: | ||
+ | gst-launch-1.0 -e rtspsrc location='rtsp://192.168.0.9:554' latency=0 ! udpsink host=127.0.0.1 port=5000 | ||
+ | * Receive rtsp stream then re-stream as rtp and display at the same time | ||
+ | gst-launch-1.0 -e rtspsrc location='rtsp://192.168.0.37:554' latency=0 ! rtpjpegdepay ! jpegdec ! timeoverlay ! tee name=t ! queue ! xvimagesink t. ! jpegenc ! rtpjpegpay ! udpsink host=127.0.0.1 port=5000 | ||
+ | To play that rtp with vlc one would need an sdp file (somefile.sdp): | ||
+ | v=0 | ||
+ | m=video 5000 RTP/AVP 26 | ||
+ | c=IN IP4 127.0.0.1 | ||
+ | a=rtpmap:26 JPEG/90000 | ||
− | == | + | To play with vlc: |
+ | vlc somefile.sdp --network-caching=0 | ||
+ | To play with gstreamer (sdp is not required): | ||
+ | gst-launch-1.0 udpsrc uri=udp://127.0.0.1:5000 ! rtpjpegdepay ! jpegdec ! autovideosink | ||
− | gst-launch - | + | =Latency tests= |
+ | ==Test 1: vlc relative to gstreamer== | ||
+ | * Camera: | ||
+ | <font size=2>elphel393~# /usr/bin/str</font> | ||
+ | * PC, console 1 - restream and display at the same time: | ||
+ | <font size=2>~$ gst-launch-1.0 -e rtspsrc location='rtsp://192.168.0.9:554' latency=0 ! rtpjpegdepay ! jpegdec ! timeoverlay ! tee name=t ! queue ! xvimagesink t. ! jpegenc ! rtpjpegpay ! udpsink host=127.0.0.1 port=5000</font> | ||
+ | * PC, console 2 - run vlc: | ||
+ | <font size=2>~$ vlc test.sdp --network-caching=0</font> | ||
+ | where ''test.sdp'': | ||
+ | <font size=2>v=0 | ||
+ | m=video 5000 RTP/AVP 26 | ||
+ | c=IN IP4 127.0.0.1 | ||
+ | a=rtpmap:26 JPEG/90000</font> | ||
− | + | [[Category:393]] | |
+ | [[Category:353]] |
Latest revision as of 11:33, 13 March 2019
GStreamer is a modular node based player as well as encoder in a single application. It is possible to create chains of so called elements with a wide range of different plugins.
Contents
GStreamer
GStreamer is a library for constructing graphs of media-handling components. The applications it supports range from simple Ogg/Vorbis playback, audio/video streaming to complex audio (mixing) and video (non-linear editing) processing.
Applications can take advantage of advances in codec and filter technology transparently. Developers can add new codecs and filters by writing a simple plugin with a clean, generic interface.
Installing GStreamer
- NicoLargo made a script to automatically install all the required packages for GStreamer on Ubuntu 10.04. Should work for up to (K)Ubuntu 16.04 as well. Or:
sudo apt-get install gstreamer1.0-tools gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly
Tips
How to get 25 FPS ?
Depends on the sensor and the exposure:
- In the sensor's datasheet find resolutions that allow the required FPS.
- You won't get 25 fps if autoexposure is on and local brightness not high enough: the camera will automatically lower framerate for keeping clear picture. Either lighten up, or play with image settings (notably, gain).
GStreamer plugins
- All of the plugins are listed here.
- Alternative way of learning a plugin's parameters is:
- version 1.0
gst-inspect-1.0 <name>
- version 0.10
gst-inspect-0.1 <name>
Command line examples
Note: replace width and height accordingly to your camera setup and your computer's horsepower :P)
GStreamer 1.0 (10393 & 10353)
Display
Multipart JPEG (mjpeg), 1 channel
gst-launch-1.0 souphttpsrc is-live=true location=http://192.168.0.8:2323/mimg ! jpegdec ! xvimagesink
Notes:
- 10393 ports are 2323-2326
- 10353 port is 8081
Multipart JPEGs (mjpeg), 4 channels in 1 window
- Each source is resized to 640x480
- Text overlay and borders added
display4in1.sh:
#!/bin/sh LOC1="http://192.168.0.8:2323/mimg" LOC2="http://192.168.0.8:2324/mimg" LOC3="http://192.168.0.8:2325/mimg" LOC4="http://192.168.0.8:2326/mimg" LABEL1="CHN1" LABEL2="CHN2" LABEL3="CHN3" LABEL4="CHN4" #souphttpsrc SOUP="souphttpsrc is-live=true" #image parameters PI="image/jpeg,width=1,height=1,framerate=1000/1,pixel-aspect-ratio=1/1" #video parameters PV="videoscale ! video/x-raw,width=640,height=480" #videobox parameters PVB="videobox fill=Black top=-4 bottom=-2 left=-4 right=-2 border-alpha=0.5" #textoverlay parameters PTO="textoverlay font-desc=\"Sans 24\" shaded-background=true valignment=top halignment=left" gst-launch-1.0 -ve videomixer name=mix \ sink_0::alpha=1 sink_0::xpos=0 sink_0::ypos=0 \ sink_1::alpha=1 sink_1::xpos=646 sink_1::ypos=0 \ sink_2::alpha=1 sink_2::xpos=0 sink_2::ypos=486 \ sink_3::alpha=1 sink_3::xpos=646 sink_3::ypos=486 \ ! \ xvimagesink sync=false \ $SOUP location=$LOC1 ! $PI ! jpegdec ! $PV ! $PVB ! $PTO text=$LABEL1 ! mix. \ $SOUP location=$LOC2 ! $PI ! jpegdec ! $PV ! $PVB ! $PTO text=$LABEL2 ! mix. \ $SOUP location=$LOC3 ! $PI ! jpegdec ! $PV ! $PVB ! $PTO text=$LABEL3 ! mix. \ $SOUP location=$LOC4 ! $PI ! jpegdec ! $PV ! $PVB ! $PTO text=$LABEL4 ! mix.
Live RTSP stream
Notes:
- careful with streams at higher resolution than 1920x1088
- 10353 - works
- 10393 - works
native resolution
gst-launch-1.0 rtspsrc location=rtsp://192.168.0.9:554 ! rtpjpegdepay ! jpegdec ! xvimagesink sync=false
resized, 640x480
gst-launch-1.0 rtspsrc location=rtsp://192.168.0.9:554 ! rtpjpegdepay ! jpegdec ! videoscale ! video/x-raw,width=640,height=480 ! xvimagesink sync=false
GStreamer 0.10 (10353)
Display
Notes:
- Careful with streams at higher resolution than 1920x1088
Live RTSP stream
native resolution
gst-launch-0.10 rtspsrc location=rtsp://192.168.0.9:554 ! rtpjpegdepay ! jpegdec ! queue ! ffmpegcolorspace ! xvimagesink sync=false
resized, fit in window
gst-launch-0.10 rtspsrc location=rtsp://192.168.0.9:554 ! rtpjpegdepay ! jpegdec ! queue ! ffmpegcolorspace ! videoscale ! xvimagesink sync=false
Dumping
mjpeg dumping
gst-launch -v rtspsrc location=rtsp://192.168.0.9:554 ! queue ! rtpjpegdepay ! \ videorate ! capsfilter caps = "image/jpeg, framerate=(fraction)25/1, width=1024, height=768" ! \ queue ! matroskamux ! filesink location=/tmp/test.mkv
YUV Dumping
gst-launch -v rtspsrc location=rtsp://192.168.0.9:554 ! queue ! rtpjpegdepay ! queue ! jpegdec ! queue ! \ videorate ! capsfilter caps="video/x-raw-yuv, format=(fourcc)I420, width=(int)1024, height=(int)768, framerate (fraction)25/1" ! \ queue ! avimux ! filesink location=/tmp/test.avi
Dump transcoding example
gst-launch filesrc location=test.mkv ! matroskademux ! queue ! jpegdec ! queue ! theoraenc bitrate=4000 ! queue ! oggmux ! filesink location=test.ogg
Live encoding
gst-launch -v rtspsrc location=rtsp://192.168.0.9:554 ! queue ! rtpjpegdepay ! queue ! jpegdec ! queue ! \ videorate ! capsfilter caps="video/x-raw-yuv, format=(fourcc)I420, width=(int)1024, height=(int)768, framerate=(fraction)25/1" ! \ queue ! theoraenc bitrate=4000 ! queue ! oggmux ! filesink location=/tmp/test1024.ogg
Notes:
- I did some benchmarks; a Core 2 Quad Q6600 (2.4 Ghz) is not powerful enough for h264 encoding @fullHD resolution (using 4 treads).
Useful links
Some other examples
- Simply playing stream
gst-launch-1.0 -v playbin uri=rtsp://192.168.0.9:554 uridecodebin0::source::latency=0
- Receive rtsp stream and re-stream as rtp:
gst-launch-1.0 -e rtspsrc location='rtsp://192.168.0.9:554' latency=0 ! udpsink host=127.0.0.1 port=5000
- Receive rtsp stream then re-stream as rtp and display at the same time
gst-launch-1.0 -e rtspsrc location='rtsp://192.168.0.37:554' latency=0 ! rtpjpegdepay ! jpegdec ! timeoverlay ! tee name=t ! queue ! xvimagesink t. ! jpegenc ! rtpjpegpay ! udpsink host=127.0.0.1 port=5000
To play that rtp with vlc one would need an sdp file (somefile.sdp):
v=0 m=video 5000 RTP/AVP 26 c=IN IP4 127.0.0.1 a=rtpmap:26 JPEG/90000
To play with vlc:
vlc somefile.sdp --network-caching=0
To play with gstreamer (sdp is not required):
gst-launch-1.0 udpsrc uri=udp://127.0.0.1:5000 ! rtpjpegdepay ! jpegdec ! autovideosink
Latency tests
Test 1: vlc relative to gstreamer
- Camera:
elphel393~# /usr/bin/str
- PC, console 1 - restream and display at the same time:
~$ gst-launch-1.0 -e rtspsrc location='rtsp://192.168.0.9:554' latency=0 ! rtpjpegdepay ! jpegdec ! timeoverlay ! tee name=t ! queue ! xvimagesink t. ! jpegenc ! rtpjpegpay ! udpsink host=127.0.0.1 port=5000
- PC, console 2 - run vlc:
~$ vlc test.sdp --network-caching=0
where test.sdp:
v=0 m=video 5000 RTP/AVP 26 c=IN IP4 127.0.0.1 a=rtpmap:26 JPEG/90000