gstreamer

From Noah.org
Revision as of 22:05, 4 October 2014 by Root (talk | contribs) (→‎Gstreamer)
Jump to navigationJump to search


Gstreamer

Gstreamer is one of the best tools in Linux for handling video. It comes with a command-line tool that allows you to build almost any time of video processing stream that you could with the gstreamer API.

See also this Gstreamer cheat sheet.

This will display the camera view in a window:

gst-launch v4l2src ! ffmpegcolorspace ! autovideosink
# or
gst-launch v4l2src ! ffmpegcolorspace ! xvimagesink

This will display the camera view in a window with a specified size (320x240) and framerate (30 fps). Note that the device is explicitly specified. The ffmpegcolorspace elements appears to be optional. I am not sure if removing this element will cause the command to fail in some environments.

gst-launch v4l2src device=/dev/video0 ! 'video/x-raw-yuv,width=320,height=240,framerate=30/1' ! ffmpegcolorspace ! xvimagesink
# or
gst-launch v4l2src device=/dev/video0 ! 'video/x-raw-yuv,width=320,height=240,framerate=30/1' ! xvimagesink

Timecode overlay.

gst-launch v4l2src device=/dev/video0 ! 'video/x-raw-yuv,width=320,height=240,framerate=30/1' ! timeoverlay ! ffmpegcolorspace ! autovideosink

Record video stream to a file using Motion JPEG encoding (MJPEG):

gst-launch --eos-on-shutdown v4l2src ! ffmpegcolorspace ! jpegenc ! avimux ! filesink location=video.avi

Record with fixed image size and frame rate.

gst-launch --eos-on-shutdown v4l2src device="/dev/video0" ! video/x-raw-yuv,width=640,height=480 ! ffmpegcolorspace ! jpegenc ! avimux ! filesink location=video.avi

1920x1080: This will have a low frame-rate since it will essentially max-out the USB bandwidth. It's useful for capturing individual frames without compression.

gst-launch v4l2src device=/dev/video0 ! 'video/x-raw-yuv,width=1920,height=1080' ! xvimagesink

This will capture at a higher framerate, but it will undersample the pixels so you will get an effective 320x240 resolution.

gst-launch v4l2src device=/dev/video0 ! 'video/x-raw-yuv,width=640,height=480,framerate=60/1' ! xvimagesink

Record video and display the stream at the same time requires tee to split the stream. Notice how the name parameter of tee refers to a label defined at the end of the command-line. Also notice how queue is necessary so that xvimagesink will run in parallel with the filesink. Without this you would record video but you would see a window with a single frozen frame of video.

gst-launch --eos-on-shutdown v4l2src ! ffmpegcolorspace ! tee name=my_videosink ! jpegenc ! avimux ! filesink location=video.avi my_videosink. ! queue ! xvimagesink

Video test pattern:

gst-launch videotestsrc ! ffmpegcolorspace ! xvimagesink

capture video with transparent overlay (picture in picture)

This will display the camera view with a semi-transparent overlay of random snow.

gst-launch \
   videomixer name=mix sink_0::zorder=1 \
       sink_1::xpos=160 sink_1::ypos=120 sink_1::alpha=0.2 sink_1::zorder=2 ! \
       ffmpegcolorspace ! xvimagesink \
   v4l2src device=/dev/video0 ! video/x-raw-yuv,width=640 ! ffmpegcolorspace ! mix.sink_0 \
   videotestsrc pattern="snow" ! video/x-raw-yuv, framerate=10/1, width=320, height=240 ! mix.sink_1 \

capture video with transparent overlay (picture in picture) version 2

This will display a static PNG image in the foreground. Anywhere the static PNG image has alpha transparency the live video will show through behind the static image.

The TV.png file below will work, or any PNG image with transparency (an alpha channel) will work. The still image on the right shows what the video display window looks like after running this command.

TV.png noah on TV.png

gst-launch \
   videomixer name=mix sink_0::zorder=1 sink_1::alpha=1.0 sink_1::zorder=2 ! ffmpegcolorspace ! xvimagesink \
   v4l2src device=/dev/video0 ! video/x-raw-yuv,width=640,height=480,framerate=30/1 ! ffmpegcolorspace ! mix.sink_0 \
   filesrc location=TV.png ! pngdec ! alphacolor ! ffmpegcolorspace ! imagefreeze ! mix.sink_1

record two video cameras into side-by-side stereo video

gst-launch v4l2src device=/dev/video1 ! videoscale ! ffmpegcolorspace ! video/x-raw-yuv, width=320, height=240 ! videobox border-alpha=0 left=-320 ! videomixer name=mixme ! ffmpegcolorspace ! jpegenc ! avimux ! filesink location=sbs-3d-video.mov v4l2src device=/dev/video2 ! videoscale ! ffmpegcolorspace ! video/x-raw-yuv, width=320, height=240 ! videobox right=-320 ! mixme.

Using mplayer you can play side-by-side stereo video as anaglyphic stereo video.

mplayer -vf stereo3d,scale -idx sbs-3d-video.mov -loop 0

record video frames into separate files

record from device to separate PNG files

gst-launch --eos-on-shutdown v4l2src device=/dev/video1 ! video/x-raw-yuv,format=\(fourcc\)YUY2,width=640,height=480,framerate=30/1 ! ffmpegcolorspace ! videorate ! video/x-raw-rgb,framerate=30/1 ! ffmpegcolorspace ! pngenc snapshot=false ! multifilesink location="frame%05d.png"

record from video file to separate PNG files

mkdir test
gst-launch -v filesrc location=test.avi ! avidemux ! ffdec_huffyuv ! ffmpegcolorspace ! pngenc snapshot=false ! multifilesink location="frame%05d.png"

record desktop

gst-launch -v ximagesrc startx=0 starty=0 endx=1279 endy=1023 ! ffmpegcolorspace ! "video/x-raw-yuv,width=1280,height=1024,framerate=10/1" ! v4l2sink device=/dev/video1

playback a collection of jpeg or PNG sill image files as a video (slideshow)

The most annoying issue is that you can't use * wildcard.

gst-launch multifilesrc location="frame%05d.png" ! image/png,framerate=5/1 ! pngdec ! videorate ! video/x-raw-rgb,framerate=5/1 ! ffmpegcolorspace ! xvimagesink

The following works great for JPEG images. This is very robust. It will skip errors when trying to decode a broken JPEG image. Then makes it easy to just dump in a whole bunch of unrelated images without having the stream die whenever it hits a bad image.

gst-launch multifilesrc location="image%05d.jpg" ! jpegdec max-errors=-1 ! videoscale ! ffmpegcolorspace ! autovideosink

mix two video sources into one (side-by-side)

gst-launch v4l2src device=/dev/video1 ! videoscale ! ffmpegcolorspace ! video/x-raw-yuv, width=320, height=240 ! videobox border-alpha=0 left=-320 ! videomixer name=mix ! ffmpegcolorspace ! xvimagesink v4l2src device=/dev/video2 ! videoscale ! ffmpegcolorspace ! video/x-raw-yuv, width=320, height=240 ! videobox right=-320 ! mix.

date and time stamps

The timeoverlay filter adds the frame buffer time to each video frame. The clockoverlay filter adds the date and time to each video frame.

gst-launch -e v4l2src device=/dev/video1 ! video/x-raw-yuv,format=\(fourcc\)YUY2,width=640,height=480,framerate=30/1 ! ffmpegcolorspace ! timeoverlay shadow=false halignment=right valignment=bottom font-desc="sanserif 10" ypad=5 xpad=5 ! clockoverlay shadow=false halignment=left valignment=bottom time-format="%Y-%m-%d %H:%M:%S" font-desc="sanserif 10" ypad=5 xpad=5 ! videorate ! video/x-raw-rgb,framerate=30/1 ! ffmpegcolorspace ! xvimagesink

interesting filters

fpsdisplaysink

gstreamer plugin documentation

Many Gstreamer plugins lack good documentation. You can find internal descriptions of plugins and their properties by using gst-inspect. For examples:

gst-inspect timeoverlay
gst-inspect clockoverlay

gstreamer flip video

This is handy if the camera is mounted upside-down or sideways. Upside-down:

gst-launch v4l2src device=/dev/video0 ! video/x-raw-yuv,width=320,height=240,framerate=30/1 ! videoflip method=clockwise ! xvimagesink

Sideways:

gst-launch v4l2src device=/dev/video1 ! video/x-raw-yuv,width=320,height=240,framerate=30/1 ! videoflip method=clockwise ! xvimagesink
# or
gst-launch v4l2src device=/dev/video1 ! video/x-raw-yuv,width=320,height=240,framerate=30/1 ! videoflip method=counterclockwise ! xvimagesink

gstreamer fbdevsink "ERROR: Pipeline doesn't want to pause."

If you are trying to use the framebuffer device for video playback then you may get an error like the one below. This is a permissions problem. Try adding sudo in front of the pipeline, or run the command as root.

$ gst-launch videotestsrc ! ffmpegcolorspace ! fbdevsink
Setting pipeline to PAUSED ...
ERROR: Pipeline doesn't want to pause.
Setting pipeline to NULL ...
Freeing pipeline ...

gstreamer xvimagesink "ERROR: ... Could not initialize Xv output"

This happens if your X11 installation does not support Xv. This is a common problem when working with a virtual machine. Try using ximagesink instead of xvimagesink.

playback video on framebuffer (/dev/fbdev0)

sudo gst-launch uridecodebin uri=file:///home/noah/Videos/ct_scan_sample.flv ! ffmpegcolorspace ! fbdevsink

You can also use mplayer to play video on a framebuffer device. Be sure to specify fbdev2 if you want color.

sudo mplayer -vo fbdev2 ct_scan_sample.flv