oldcpu's meandering thoughts on Computers, GNU/Linux and openSUSE

The example I gave above was with pulse audio installed. Of course, one can bypass pulse and run the ffmpeg command receive input direct from the device (albeit one has argueably less control over the device input) . The command, for example, for this would be:

Code:

ffmpeg -f alsa -ac 2 -i hw:1,0 -f x11grab -r 30 -s 1920x1200 -i :0.0 -acodec pcm_s16le -vcodec libx264 -vpre lossless_ultrafast -threads 0 output.avi

where in my example here I have assumed the microphone is device " hw:1,0".

Of course that begs the question, how does now the device (mic) is hw:1,0 ?

In my case I first ran :

Code:

arecord -l

which gave me:

Code:

oldcpu@core-i7:~> arecord -l
**** List of CAPTURE Hardware Devices ****
card 0: Intel [HDA Intel], device 0: AD198x Analog [AD198x Analog]
  Subdevices: 3/3
  Subdevice #0: subdevice #0
  Subdevice #1: subdevice #1
  Subdevice #2: subdevice #2
card 1: U0x46d0x821 [USB Device 0x46d:0x821], device 0: USB Audio [USB Audio]
  Subdevices: 1/1
  Subdevice #0: subdevice #0

and determined my USB (webcam) mic was HW:1,0 (ie card-1, subdevice 0). Since I wanted to use the mic in my webcam to record, I input hw:1,0 in the ffmpeg command. Which happens to my USB webcam mic.

I think the rest is intuitively obvious ?

The ffmpeg arguments have changed since my last blog post in this thread.

The ffmpeg developers are constantly improving ffmpeg, but in the process of doing so they continually modify the User Interface (UI) which can result in breakage. In this case their change means I need to modify my screen capture command.

It turns out the code " -vpre lossless_ultrafast " is depreciated. What works instead now is "-preset ultrafast" .

Also, in my example above to use my webcam I specified " -i hw:1,0 " ... what works also, is to specify my webcam as the default mic in pulse audio and thus use " -i pulse ".

So here is an example that captures my entire desktop (1920x1200 resolution), at 25 frames per second ( -r 25 ) :

Code:

ffmpeg -f alsa -ac 2 -i pulse -f x11grab -r 25 -s 1920x1200 -i :0.0 -acodec pcm_s16le -vcodec libx264 -preset ultrafast -threads 0 output.avi

I replaced 'pulse' with 'default' in ffmpeg -i value as it appears the ffmpeg arguments have changed again (!) since my last post.

I previous used this:

Code:

ffmpeg -f alsa -ac 2 -i pulse -f x11grab -r 25 -s 1920x1200 -i :0.0 -acodec pcm_s16le -vcodec libx264 -preset ultrafast -threads 0 output.avi

and it did not work any more. I gave me the error:

Code:

[alsa @ 0x638680] cannot open audio device pulse (No such file or directory)
pulse: Input/output error

For some reason, the device 'pulse' is no longer recognized.

I then ran

Code:

arecord -L

to see what devices are listed on my system. That gave me a large output, but the only part in that output that mentioned pulse audio was at the start of the 'arecord -L' output, which stated:

Code:

null
    Discard all samples (playback) or generate zero samples (capture)
default
    Default ALSA Output (currently PulseAudio Sound Server)

... so it appears, that the device is called 'default'. ... So I then tried, replacing 'pulse' with 'default' which worked:

Code:

ffmpeg -f alsa -ac 2 -i default -f x11grab -r 25 -s 1920x1200 -i :0.0 -acodec pcm_s16le -vcodec libx264 -preset ultrafast -threads 0 output.avi

and as noted the screen capture with audio worked. I was able to record my desktop, and capture the audio in the same 'output.avi' video file, using pulse audio.

This is very useful, as one can do some interesting things with pulse audio wrt the audio.

I noted the audio and video was desync'd in my capture.

Plus a further note - I was obtaining an error "Incompatible pixel format 'bgra' for codec 'libx264', auto-selecting format 'yuv420p'" which was a bit surprising. Anyway, I decided to specify the libx264 'pix_fmt' with the following:

Code:

ffmpeg -f pulse -async -25 -i default -f x11grab -r 25 -s 1920x1200 -i :0.0 -acodec pcm_s16le -vcodec libx264 -pix_fmt yuv422p -threads 0 output.avi

which got rid of the error.

One thing I noted was the audio / video was out of sync. I don't know why. Surfing to see if I could find a solution I noted there are some interesting websites on ffmpeg:

• Fixing out-of-sync audio and video with ffmpeg | Internet Services from KKoncepts
• ffmpeg audio/video manipulation

so as a work around, until I can figure out how to have this in sync at the start, I took the output video 'out.avi' and resync'd it with the command

Code:

ffmpeg -i output.avi -itsoffset 0.1 -i output.avi -map 1:0 -map 0:1 -vcodec libx264 -pix_fmt yuv422p -ar 22050 outfixed.avi

where the file 'outfixed.avi' is back with the video and audio in sync.

There probably is a better way to do this, but it is working better now for me with this.

Desktop capture and two audio sources

pavucontrol and ffmpeg running with simultaneous computer audio and mic audio recording is a neat feature of pavucontrol allowing one to have a neat recording of one's desktop, with multimedia playing and one talking on their Mic.

I successfully performed this on my 64-bit openSUSE-12.1, recording my desktop which was playing a video (with audio) and at the same time recording what I was saying in my external mic. ie ffmpeg recorded from two separate audio inputs at the same time (using pulse audio) and recorded the desktop video at a resolution of 1920x1280. Note this was on a > 3-year old Core i7 920 computer. Users with slower CPUs may wish to use a lower resolution. Also 1920x1280 is far too high a resolution for a video tutorial.

Caution

Before I explain how I did this, first when doing this for the first few times with ffmpeg and pavucontrol, do not have any unnecessary applications open (that may use multimedia) as they may confuse one wrt the many options that pavucontrol will provide. For example, a browser is also a multimedia app, so do not have any browsers open initially, until you learn enough about pavucontrol use such that you can recognize a browser entry in it.

My biggest complaint about pavucontrol is it does not provide sufficient detail in its labels to avoid ambiguity. I find that unfortunately pavucontrol's labels are not always as clear as one might want, which is another reason why it is important not to have too many applications running when doing this screen capture (until one can recognize the various apps in pauvcontrol).

Overall Method

The way I performed this, was to have the audio from the computer and the audio from the mic sent to an audio sink. And I had ffmpeg record the audio sink in addition to record the desktop video. It was that basic. The controls were not quite so basic to set this up.

Create an Audio Sink in pavucontrol

Now we need to create a sink to mix the captured audio. Pulse audio will refer to this sink by the term 'null'. To do so we need load the module 'null sink'. We also we need to redirect the capture of the two capture streams (from the mic and from the internal computer audio) into this new 'null sink' so we load two instances of the loopback module so as to redirect (loopback) those two captures into the 'null' (sink).

The command to create the audio sink (as a regular user):

Code:

pactl load-module module-null-sink sink_name=mixcapture

and the commands (as a regular user) to create two instances of a loop back (so to direct the computer audio and the mic input to the audio sink are)

Code:

pactl load-module module-loopback sink=mixcapture
pactl load-module module-loopback sink=mixcapture

ie I simply ran the same command twice.

Note the above will only last until the terminal in which they are executed is closed, and will not survive the 1st reboot. The advantage here is a reboot will clean up everything, so it is perfectly safe to mess with this (although after a reboot you may need to go into pavucontrol and fix any strange settings you applied).

Start recording:

So lets start recording with (where the following is all one line):

Code:

ffmpeg -f pulse -i default -f x11grab -r 15 -s 1920x1200 -i :0.0 -acodec pcm_s16le -vcodec libx264 -pix_fmt yuv422p -threads 0 output00.avi

That is a very high resolution 1920x1280. One may instead wish to reduce the resolution/capture size and tune the " -i :0.0 " setting to a limited area of the desktop. But I won't get into that tuning here.

So now the desktop is recording, I went into 'pavucontrol' and tuned its settings:

pavucontrol settings

Now lets look at my pavucontrol settings.

pavucontrol Configuration Tab

First here is my configuration tab.

[click on image for a larger view]

You can see two entries:

• Internal Audio - This is set to "Analog Surround 4.1 Output + Analog Stero Input" which is my motherboard surround sound and an input audio (my external mic)
• HD Webcam C910 - This is set to "Analog Stereo Input" and it is my webcam mic - which I did not use for this example

pavucontrol Input Devices Tab

Next here is the Input Devices Tab.

[click on image for a larger view]

Note at the bottom of that above image, I have 'Show' set to 'All Input Devices'. That is important to ease one's understanding. Here you can see:

• Internal Audio Analog Stereo - That is my external mic, which in this case is plugged into the PC's front microphone jack (so I selected Front Microphone)
• HD Webcam C910 Analog Stereo - That is my webcam mic. For this example I did not use that mic.
• Monitor of Internal Audio Analog Surround 4.1 - That is the computer audio
• Monitor of Null Output - That is the Audio 'sink' that I created above. One can direct audio input from applications/devices into this sink

pavucontrol Output Devices Tab

Next here is the Output Devices Tab.

[click on image for a larger view]

Note at the bottom of that above image I have 'Show' set to 'All Output Devices'. That is important to ease one's understanding. Here you can see:

• Internal Audio Analog Surround-4.1 - this is the computer audio which can have an output
• Null Output - this is the Audio 'sink' that I created above, where this Audio 'sink' can also have an output

pavucontrol Recording Tab

Next here is the Recording Tab.

[click on image for a larger view]

Note at the bottom of the above iamge I have 'Show' set to 'All Streams'. That is important to ease one's understanding. Here you can see:

• Loopback to Null Output from - Here I have 'Monitor of Internal Audio Analog Surround 4.1' selected. This means that the 'Computer audio' is being directed into the Null Output, which is the 'Audio' sink
• Loopback to Internal Audio Analog Surround 4.1 from - Here I have 'Internal Audio Analog Stero' selected. If you recall, 'Internal Audio Analog Stereo' is my External (front) Mic. So this means the External Mic output is being sent to join the audio in the 'Computer Audio'.
• Lavf53.32.100: record from - Here I have 'Monitor of Null Output' selected. Lavf53.31.100 is the application ffmpeg. And 'Monitor of Null Output' is the output of the Audio 'Sink'. So this tells ffmpeg to record what ever is being sent to the Audio 'sink'. And from above we can see that the 'External mic' output is being sent to the 'Computer Audio' which in turn is being sent to the Audio 'sink'. Hence ffmpeg by recording the 'Null Output' is in fact recording the 'Computer Audio' which has the External Mic input also inside.

... continued ... My next post will provide detail on the Playback settings I had in place.

recording desktop and 2 audio input continued ....

pavucontrol Playback Tab

Next here is the Playback Tab.

[click on above image for a larger view]

Note at the bottom I have 'Show' set to 'All Streams'. That is important to ease one's understanding. The playback is important to control where the audio from the multimedia application is sent , and also important to control what goes to the speakers so one can listen to what one is trying to record (ie provide the user some audio feedback that this is all working properly).

So here you can see:

• System Sounds - I did not use this setting in my desktop recording
• Loopback of Monitor of Internal Audio Analog Surround 4.1 on - here I have this set to 'Null Output'. Again, recall that 'Internal Audio Analog Surround 4.1' is the Computer audio. For this setting, in addition to the speakers playing any Computer audio, I am also looping it to a 'Null Output' (to the Audio 'sink').
• Loopback of Internal Analog Stereo on - here I have this set to 'Internal Audio Analog Surround 4.1'. Again, recall that 'Internal Analog Stero' is my external mic and recall that 'Internal Audio Analog Suround 4.1' is the computer audio. So here I am telling pavucontrol to take the Mic input and send it to 'Computer Audio', which will then be played in my speakers while I am recording (which may be desireable). Without this setting I would not hear myself talk on my speakers while I was recording. So as noted, if instead this had been set to 'Null Output' then I would not hear my voice on my speakers, but my voice would still be recorded. It is really a matter of preference here. Do you wish to hear your own voice on the speakers when recording your desktop ?
• ALSA plug-in [mplayer]: ALSA Playback on - here I have this set to 'Internal Audio Analog Surround 4.1'. Again, recall that 'Internal Audio Analog Surround 4.1' is the Computer Audio. In fact I was playing smplayer, but smplayer is just a front end to mplayer, so pavucontrol only can see that mplayer is playing. With this setting I am able to hear the output of the smplayer application as I record.

Resynchronise audio and video if out of sync

When I played back my video, I found the audio and video desyncronised. I resynchronized it with this command:

Code:

ffmpeg -i output00.avi -itsoffset 0.5 -i output.avi -map 1:0 -map 0:1 -vcodec libx264 -pix_fmt yuv422p -ar 22050 outputresyncd.avi

where the argument ' -itsoffset 0.5 ' can be used to tune the delay between the audio and video. One might find a value of ' 0.1 ' is better than '0.5' or instead find that a value of ' 4.627 ' (for example) is better than ' 0.5 ' . Try different values until the audio/video is properly synchronized.

Now this can be a bit confusing, but if one plays with it enough, it becomes easier.

Other applications

Note for example, I could have selected 'Skype' instead of 'smplayer' and recorded myself chatting on Skype with a friend. Or instead of 'ffmpeg' I could have used another desktop recording application, which might be useful if that other desktop recording application uses pulse audio but is not nominally configured to record from multiple sources.

Permanent audio sink and loop back in support of desktop capture

Further to my above post, rather than have to create the audio sink and two loop back instances each time I want to capture my desktop with audio from 2 different sources, I created a permanent audio sink and 2 permanent loop backs.

I did this by the following:

1. with root permissions edited the file /etc/pulse/default.pa
   
   Code:

   load-module module-null-sink sink_name=audiomixer

   where 'audiomixer' is an arbitrary name picked by me. Pick your own here.
   .
2. and also with root permissions, add two (identical) loopback sinks to the /etc/pulse/default.pa file for the audio sink 'audiomixer'
   
   Code:

   load-module module-loopback latency_msec=5 sink=audiomixer

   Code:

   load-module module-loopback latency_msec=5 sink=audiomixer

   I don't know if the 'latency_msec=5' is necessary. I suspect I should either remove it, or put a much much bigger value in place.
   
   And again I have 2 loopbacks, one for audio passing through motherboard and one for audio coming from my Front Mic. If one wanted to add another Mic (such as webcam mic) would could add another identical loopback module load entry.

Then I rebooted and tested and it worked. I did note my audio/video had a large desynchronisation, that I was able to fix with another ffmpeg instances (and different arguments) as noted above in my previous blog post.

This is discouraging. So far the instructions here are the best thing I have found to record my desktop. I have tried freeseer which does not generate a file. I have tried xvidcap which freezes, and I have tried other applications which don’t work and seem to be abandoned. None of those have audio support.

This method I get scratchy faint sound, and a flashing picture.The desktop flashes through any window I have open along the top. I have slowed down the capture rate to 8fps and it looks jerky, but the flashing stops - mostly.

I am using Suse 12.1, a 64 bit machine, 4GB of ram, running at 3.2 Mhz on a Core 2 Duo.

Is there something else? Do I have enough memory?