April 27, 2014

Catatan GStreamer

Inilah catatan seputar utak-atik GStreamer, mengetahui bagaimana cara bekerjanya dan bisa digunakan untuk apa. Misalnya: bagaimana cara mengambil raw video dari webcam, lalu dilakukan konversi, dan ditampilkan ke komputer lain. Karena ini hanyalah catatan lepas, maka tidak disusun dengan rapi.

Memainkan lagu

Source filesrc adalah corelement GStreamer. Elemen ini membaca file lagu /home/ardhyw/Music/Random\ Access\ Memories/03\ -\ Giorgio\ by\ Moroder.mp3

decodebin adalah elemen yang melakukan auto-detect data format dan melakukan decoding data. Informasi bahwa ini adalah file MP3 tidak perlu diberikan. Src dari filesrc disambungkan ke sink elemen decodebin, hasil decode akan dikeluarkan ke port src dari decodebin.

pulsesink adalah elemen yang berfungsi untuk mengolah data menjadi suara menggunakan driver Pulse Audio. Port src dari decodebin dihubungkan dengan port sink dari pulsesink

gst-launch-1.0 -evvv filesrc location=/home/ardhyw/Music/Random\ Access\ Memories/03\ -\ Giorgio\ by\ Moroder.mp3 ! decodebin ! pulsesink
Log-nya
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstTypeFindElement:typefind.GstPad:src: caps = application/x-id3
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstTypeFindElement:typefind.GstPad:src: caps = NULL
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstID3Demux:id3demux0.GstPad:src: caps = NULL
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstID3Demux:id3demux0.GstPad:src: caps = NULL
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMpegAudioParse:mpegaudioparse0.GstPad:src: caps = audio/mpeg, mpegversion=(int)1, mpegaudioversion=(int)1, layer=(int)3, rate=(int)44100, channels=(int)2, parsed=(boolean)true
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMad:mad0.GstPad:sink: caps = audio/mpeg, mpegversion=(int)1, mpegaudioversion=(int)1, layer=(int)3, rate=(int)44100, channels=(int)2, parsed=(boolean)true
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMad:mad0.GstPad:src: caps = audio/x-raw, format=(string)S32LE, layout=(string)interleaved, rate=(int)44100, channels=(int)2, channel-mask=(bitmask)0x0000000000000003
/GstPipeline:pipeline0/GstPulseSink:pulsesink0.GstPad:sink: caps = audio/x-raw, format=(string)S32LE, layout=(string)interleaved, rate=(int)44100, channels=(int)2, channel-mask=(bitmask)0x0000000000000003
/GstPipeline:pipeline0/GstDecodeBin:decodebin0.GstDecodePad:src_0.GstProxyPad:proxypad1: caps = audio/x-raw, format=(string)S32LE, layout=(string)interleaved, rate=(int)44100, channels=(int)2, channel-mask=(bitmask)0x0000000000000003
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstPulseSinkClock
/GstPipeline:pipeline0/GstPulseSink:pulsesink0: volume = 1
/GstPipeline:pipeline0/GstPulseSink:pulsesink0: mute = false
/GstPipeline:pipeline0/GstPulseSink:pulsesink0: current-device = alsa_output.pci-0000_00_1b.0.analog-stereo
/GstPipeline:pipeline0/GstPulseSink:pulsesink0: volume = 1
/GstPipeline:pipeline0/GstPulseSink:pulsesink0: mute = false
/GstPipeline:pipeline0/GstPulseSink:pulsesink0: current-device = alsa_output.pci-0000_00_1b.0.analog-stereo
/GstPipeline:pipeline0/GstPulseSink:pulsesink0: volume = 1
/GstPipeline:pipeline0/GstPulseSink:pulsesink0: mute = false
/GstPipeline:pipeline0/GstPulseSink:pulsesink0: current-device = alsa_output.pci-0000_00_1b.0.analog-stereo
/GstPipeline:pipeline0/GstPulseSink:pulsesink0: volume = 1
/GstPipeline:pipeline0/GstPulseSink:pulsesink0: mute = false
/GstPipeline:pipeline0/GstPulseSink:pulsesink0: current-device = alsa_output.pci-0000_00_1b.0.analog-stereo

Capture H.264 dari Webcam dan Stream via RTP

gst-launch-1.0 -v v4l2src device=/dev/video1 ! h264parse ! rtph264pay ! udpsink host=127.0.0.1 port=4000
Log-nya
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-h264, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:sink: caps = video/x-h264, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:src: caps = video/x-h264, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1, parsed=(boolean)true, stream-format=(string)avc, alignment=(string)au, codec_data=(buffer)01424028ffe1002667424028bb403c0112f2e02890000003001000000303c6040002dc6c002dc7bdef85e1108d4001000468ce3880
/GstPipeline:pipeline0/GstRtpH264Pay:rtph264pay0.GstPad:src: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, sprop-parameter-sets=(string)"Z0JAKLtAPAES8uAokAAAAwAQAAADA8YEAALcbAAtx73vheEQjUA\=\,aM44gA\=\=", payload=(int)96, ssrc=(uint)2396222928, timestamp-offset=(uint)249955673, seqnum-offset=(uint)23288
/GstPipeline:pipeline0/GstUDPSink:udpsink0.GstPad:sink: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, sprop-parameter-sets=(string)"Z0JAKLtAPAES8uAokAAAAwAQAAADA8YEAALcbAAtx73vheEQjUA\=\,aM44gA\=\=", payload=(int)96, ssrc=(uint)2396222928, timestamp-offset=(uint)249955673, seqnum-offset=(uint)23288
/GstPipeline:pipeline0/GstRtpH264Pay:rtph264pay0.GstPad:sink: caps = video/x-h264, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1, parsed=(boolean)true, stream-format=(string)avc, alignment=(string)au, codec_data=(buffer)01424028ffe1002667424028bb403c0112f2e02890000003001000000303c6040002dc6c002dc7bdef85e1108d4001000468ce3880
/GstPipeline:pipeline0/GstRtpH264Pay:rtph264pay0: timestamp = 249983424
/GstPipeline:pipeline0/GstRtpH264Pay:rtph264pay0: seqnum = 23288

Terima H.264 dari RTP

gst-launch-0.10 -v udpsrc port=4000 caps='application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264' ! rtph264depay ! ffdec_h264 ! xvimagesink sync=false
Log-nya
Setting pipeline to PAUSED ...
/GstPipeline:pipeline0/GstUDPSrc:udpsrc0.GstPad:src: caps = application/x-rtp, media=(string)video, payload=(int)96, clock-rate=(int)90000, encoding-name=(string)H264
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstRtpH264Depay:rtph264depay0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)nal
/GstPipeline:pipeline0/GstRtpH264Depay:rtph264depay0.GstPad:sink: caps = application/x-rtp, media=(string)video, payload=(int)96, clock-rate=(int)90000, encoding-name=(string)H264
/GstPipeline:pipeline0/ffdec_h264:ffdec_h2640.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)nal
/GstPipeline:pipeline0/ffdec_h264:ffdec_h2640.GstPad:src: caps = video/x-raw-yuv, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, format=(fourcc)I420, interlaced=(boolean)false, pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstXvImageSink:xvimagesink0.GstPad:sink: caps = video/x-raw-yuv, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, format=(fourcc)I420, interlaced=(boolean)false, pixel-aspect-ratio=(fraction)1/1

CPU Load

Mantaps! Setelah dilihat, proses capture memakan CPU cuma 1%, padahal output-nya H.264 lho. Sedangkan untuk receiver-nya makan CPU 8-10% karena harus melakukan decode H.264

Upate 29/April: Ternyata decode H.264 pake gstreamer-1.0 utilisasi CPU cuma 5-6%

Streaming Suara

Ini yang capture → resample → encode jadi speex → kirim via UDP

gst-launch-0.10 -v alsasrc ! audioconvert ! audioresample ! 'audio/x-raw-int,rate=16000,width=16,channels=1' ! speexenc ! rtpspeexpay ! udpsink host=127.0.0.1 port=60000

Ini yang terima dan decode.

gst-launch-0.10 udpsrc port=60000 caps="application/x-rtp, media=(string)audio, clock-rate=(int)16000, encoding-name=(string)SPEEX, encoding-params=(string)1, payload=(int)110" ! gstrtpjitterbuffer ! rtpspeexdepay ! speexdec ! audioconvert ! audioresample ! alsasink

No comments:

Post a Comment