This project has moved. For the latest updates, please go here.

NVENC Encoding Failures

Jun 10, 2016 at 2:07 AM

I have been using NVENC encoding for quite a while now, really happy with it - but recently, it seems to be failing more often than running. I have updated to the latest (Early Access) release, but no joy.

I will upload a couple logs tonight, to see if that helps - very odd that something has gone south. Perhaps the latest NVidia drivers?

Jun 10, 2016 at 3:19 AM
Check if you updated your driver. Failure has more to do with drivers than MCEBuddy with hardware encoding.

Jun 10, 2016 at 12:10 PM

Yep, my driver is always up to date ... in fact, using the driver from just a couple days ago now.

Jun 10, 2016 at 12:30 PM
Yes that's what I was saying. Latest is not always greatest. Try rolling back to an older drivers. Often new drivers can reduce performance or break stuff. With drivers if all is good I usually never update them

Jun 11, 2016 at 3:26 AM
OK, been playing with this a bit - and if I manually convert or change the crop (in either case to 1280x720) -> the file converts without error (though crop of course messes things up a bit, it was just a test).

Any ideas why the (output) resolution would cause such issues?
Jun 11, 2016 at 12:34 PM
Same reason I guess. Buggy drivers

Jun 11, 2016 at 3:43 PM
It may be, but it looks like it's ffmpeg that is actually breaking. I tried two conversions this morning ... one worked fine, the other failed (same settings). But for the one that failed, here is the error message,

2016-06-11T07:53:50 MCEBuddy.AppWrapper.FFmpeg --> Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height

And the intended output parameters don't look right, as below,
2016-06-11T07:53:50 MCEBuddy.AppWrapper.FFmpeg --> Stream #0:0: Video: h264, none, q=2-31, 128 kb/s, SAR 267:268 DAR 0:0, 29.97 fps

128 kb/s? Seems very odd to me. For the one that ran fine, the same ffmpeg output is quite different (below),
2016-06-11T06:31:49 MCEBuddy.AppWrapper.FFmpeg --> Stream #0:0: Video: h264 (nvenc) ([33][0][0][0] / 0x0021), yuv420p, 1280x712 [SAR 267:268 DAR 120:67], q=-1--1, 2488 kb/s, 29.97 fps, 30k tbn, 29.97 tbc

That bit rate makes more sense to me ... :-). Perhaps it's one of the parameters in the call? It could be drivers, just trying to make sure something else isn't triggering it.

Jun 11, 2016 at 6:59 PM
Do you have nvidia hardware?
Do you have cropping enabled?

Jun 11, 2016 at 7:53 PM
Yep, Nvidia GeForce GTX 750.

Cropping ... err, not sure how to check ... :-(. Sorry - that's my dumb. Let me poke around, see how to figure that out.

Jun 11, 2016 at 10:20 PM
Start with turning of hardware encoding. ffmpeg is a wrapper for your driver. It's your driver that's failing.

Cropping and hardware encoding options are in the expert settings page. Simple solution, if it working some time ago and not now go back to the driver when it was working.

Jun 12, 2016 at 5:03 PM
Makes sense - will play with it.

As for versions - I changed the driver and MCEBuddy both, was just trying to make sure of the issue (but agreed, I believe it's the driver als). That said ... is there a way to force QuickSync (instead of NVENC)? And also, from the logs ... how do I set a fallback, so if NVENC fails, then it runs the SW codec? HW is preferred (big delta in load), but if not - at least convert.

Jun 12, 2016 at 6:19 PM

A bit of perusing through the log file - a couple notes for you (may help with future updates ... or may not ... LOL),

1) Check for NVidia HW (which exists),
2016-06-12T13:08:40 MCEBuddy.AppWrapper.NVidiaQuery --> Launching process C:\Program Files\MCEBuddy2x\nvidia\nvidiaQuery.exe
2016-06-12T13:08:40 MCEBuddy.AppWrapper.NVidiaQuery --> Process arguments
2016-06-12T13:08:40 MCEBuddy.AppWrapper.NVidiaQuery --> UI Session Admin Process : True
2016-06-12T13:08:40 MCEBuddy.AppWrapper.NVidiaQuery --> Starting process as a UISession process with Admin privileges. This requires atleast 1 user to be logged into the system (remote desktop or locally)
2016-06-12T13:08:40 MCEBuddy.AppWrapper.NVidiaQuery --> Setting process priority to Normal
2016-06-12T13:08:41 MCEBuddy.AppWrapper.NVidiaQuery --> C:\Program Files\MCEBuddy2x\nvidia\nvidiaQuery.exe Starting...
2016-06-12T13:08:41 MCEBuddy.AppWrapper.NVidiaQuery --> CUDA Device Query (Runtime API) version (CUDART static linking)
2016-06-12T13:08:41 MCEBuddy.AppWrapper.NVidiaQuery --> cudaGetDeviceCount returned 30
2016-06-12T13:08:41 MCEBuddy.AppWrapper.NVidiaQuery --> -> unknown error
2016-06-12T13:08:41 MCEBuddy.AppWrapper.NVidiaQuery --> Result = FAIL
--> Process exited with code 1
WARNING> 2016-06-12T13:08:43 MCEBuddy.AppWrapper.NVidiaQuery --> nvENC NVidia driver not detected or driver is too old
WARNING> --> Error detecting NVidia Hardware Encoder Capabilites

2) Check for QSV (which exists)
2016-06-12T13:08:43 MCEBuddy.AppWrapper.Handbrake --> Launching process C:\Program Files\MCEBuddy2x\handbrake\handbrakecli.exe
2016-06-12T13:08:43 MCEBuddy.AppWrapper.Handbrake --> Process arguments -i null -o null
2016-06-12T13:08:43 MCEBuddy.AppWrapper.Handbrake --> UI Session Admin Process : True
2016-06-12T13:08:43 MCEBuddy.AppWrapper.Handbrake --> Starting process as a UISession process with Admin privileges. This requires atleast 1 user to be logged into the system (remote desktop or locally)
2016-06-12T13:08:43 MCEBuddy.AppWrapper.Handbrake --> Setting process priority to Normal
2016-06-12T13:08:43 MCEBuddy.AppWrapper.Handbrake --> [13:08:43] hb_init: starting libhb thread
2016-06-12T13:08:43 MCEBuddy.AppWrapper.Handbrake --> [13:08:43] thread 250cb50 started ("libhb")
2016-06-12T13:08:43 MCEBuddy.AppWrapper.Handbrake --> HandBrake 20151008180809-f9c5593-master (2015100901) - MinGW x86_64 -
2016-06-12T13:08:43 MCEBuddy.AppWrapper.Handbrake --> 4 CPUs detected
2016-06-12T13:08:43 MCEBuddy.AppWrapper.Handbrake --> Opening null...
2016-06-12T13:08:43 MCEBuddy.AppWrapper.Handbrake --> [13:08:43] CPU: Intel(R) Core(TM) i5-4590 CPU @ 3.30GHz
2016-06-12T13:08:43 MCEBuddy.AppWrapper.Handbrake --> [13:08:43] - Intel microarchitecture Haswell
2016-06-12T13:08:43 MCEBuddy.AppWrapper.Handbrake --> [13:08:43] - logical processor count: 4
2016-06-12T13:08:43 MCEBuddy.AppWrapper.Handbrake --> [13:08:43] Intel Quick Sync Video support: no
2016-06-12T13:08:43 MCEBuddy.AppWrapper.Handbrake --> [13:08:43] hb_scan: path=null, title_index=1
2016-06-12T13:08:43 MCEBuddy.AppWrapper.Handbrake --> src/libbluray/disc/disc.c:251: failed opening UDF image null
2016-06-12T13:08:43 MCEBuddy.AppWrapper.Handbrake --> src/libbluray/disc/disc.c:332: error opening file BDMV\index.bdmv
2016-06-12T13:08:43 MCEBuddy.AppWrapper.Handbrake --> src/libbluray/disc/disc.c:332: error opening file BDMV\BACKUP\index.bdmv
2016-06-12T13:08:43 MCEBuddy.AppWrapper.Handbrake --> [13:08:43] bd: not a bd - trying as a stream/file instead
2016-06-12T13:08:43 MCEBuddy.AppWrapper.Handbrake --> libdvdnav: Using dvdnav version 5.0.1
2016-06-12T13:08:43 MCEBuddy.AppWrapper.Handbrake --> libdvdread: Encrypted DVD support unavailable.
2016-06-12T13:08:43 MCEBuddy.AppWrapper.Handbrake --> libdvdread: Can't stat null
2016-06-12T13:08:43 MCEBuddy.AppWrapper.Handbrake --> No such file or directory
2016-06-12T13:08:43 MCEBuddy.AppWrapper.Handbrake --> libdvdread: Could not open null
2016-06-12T13:08:43 MCEBuddy.AppWrapper.Handbrake --> libdvdnav: vm: failed to open/read the DVD
2016-06-12T13:08:43 MCEBuddy.AppWrapper.Handbrake --> [13:08:43] dvd: not a dvd - trying as a stream/file instead
2016-06-12T13:08:43 MCEBuddy.AppWrapper.Handbrake --> [13:08:43] hb_stream_open: open null failed
2016-06-12T13:08:43 MCEBuddy.AppWrapper.Handbrake --> [13:08:43] scan: unrecognized file type
2016-06-12T13:08:43 MCEBuddy.AppWrapper.Handbrake --> [13:08:43] libhb: scan thread found 0 valid title(s)
2016-06-12T13:08:43 MCEBuddy.AppWrapper.Handbrake --> No title found.
2016-06-12T13:08:43 MCEBuddy.AppWrapper.Handbrake --> HandBrake has exited.
--> Process exited with code 2
WARNING> 2016-06-12T13:08:45 MCEBuddy.AppWrapper.Handbrake --> Handbrake failed, non 0 return code
INFORMATION> 2016-06-12T13:08:45 MCEBuddy.AppWrapper.Handbrake --> QuickSync encoding supported available -> False
Jun 12, 2016 at 9:57 PM
If it's using hardware encoding and it fails it'll fall back to software encoding.
Ffmpeg uses NVIDIA and handbrake uses QuickSync.

Most likely try disabling cropping and it should solve your issue.

Jun 13, 2016 at 12:53 AM
I did disable cropping - sorry, thought I mentioned that above. No joy unfortunately. The logs seem to be saying that it's not detecting either HW accelerator, no?

Jun 13, 2016 at 1:47 AM
I think it's an invalid SAR. Upload the logs with the cropping turned off

Jun 13, 2016 at 1:59 AM
You bet, will do! But one other interesting note (probing around ... LOL).

The note above, the error for CUDA detect (cudaGetDeviceCount returned 30). If I try to run nvidiaQuery.exe over RDP, I get this same error - which matches what I found before ... CUDA and the driver are not being used for RDP. So if I go to the machine itself, run the same command, I get the following,

nvidiaQuery.exe Starting...

CUDA Device Query (Runtime API) version (CUDART static linking)

Detected 1 CUDA Capable device(s)

Device 0: "GeForce GTX 750"
CUDA Driver Version / Runtime Version 7.5 / 7.5
CUDA Capability Major/Minor version number: 5.0
Total amount of global memory: 1024 MBytes (1073741824 bytes)
( 4) Multiprocessors, (128) CUDA Cores/MP: 512 CUDA Cores
GPU Max Clock rate: 1137 MHz (1.14 GHz)
Memory Clock rate: 2505 Mhz
Memory Bus Width: 128-bit
L2 Cache Size: 2097152 bytes
Maximum Texture Dimension Size (x,y,z) 1D=(65536), 2D=(65536, 65536), 3D=(4096, 4096, 4096)
Maximum Layered 1D Texture Size, (num) layers 1D=(16384), 2048 layers
Maximum Layered 2D Texture Size, (num) layers 2D=(16384, 16384), 2048 layers
Total amount of constant memory: 65536 bytes
Total amount of shared memory per block: 49152 bytes
Total number of registers available per block: 65536
Warp size: 32
Maximum number of threads per multiprocessor: 2048
Maximum number of threads per block: 1024
Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
Max dimension size of a grid size (x,y,z): (2147483647, 65535, 65535)
Maximum memory pitch: 2147483647 bytes
Texture alignment: 512 bytes
Concurrent copy and kernel execution: Yes with 1 copy engine(s)
Run time limit on kernels: Yes
Integrated GPU sharing Host Memory: No
Support host page-locked memory mapping: Yes
Alignment requirement for Surfaces: Yes
Device has ECC support: Disabled
CUDA Device Driver Mode (TCC or WDDM): WDDM (Windows Display Driver Model)
Device supports Unified Addressing (UVA): Yes
Device PCI Domain ID / Bus ID / location ID: 0 / 1 / 0
Compute Mode:
 < Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >
deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 7.5, CUDA Runtime Version = 7.5, NumDevs = 1, Device0 = GeForce GTX 750
Result = PASS

Now, this all makes sense => CUDA is detected only locally, when the driver is in use ... but why is the log above showing the same thing? Odd ... no?

For the log (cropping off) ... run the NVENC profile, right?
Jun 13, 2016 at 12:21 PM
That's right. You need to be physically logged in for hardware encoding to work. Windows design, it needs a UI interactive sessions

Jun 13, 2016 at 10:07 PM
Yep, that makes sense - absolutely agreed! But why then is the MCEBuddy log showing the same detection failure? It's running on that same machine (and running locally)?

Jun 13, 2016 at 11:24 PM
Are you using Windows server? Did you upload the log

Jun 14, 2016 at 1:08 AM

There were two logs from before - but to try to make it easier (not as much to wade through), I just did another quick recording (1-2 min), and uploaded that log file as well.

And no, not Windows Server. Windows 10, 64 bit.

Jun 18, 2016 at 6:04 PM
Can you try today's 2.4.5 BETA build with the same quick recording and upload the logs if it's still failing. (it has a new build of ffmpeg with NvENC improvements), also see if you can spot any difference in the NvEnc encoding performance.

Your log showed it failed to initialize nvenc
[nvenc @ 0000020ddb860520] >> dl_fn->cu_init(0) - failed with error code 0x3e7

Jun 20, 2016 at 11:52 AM
Absolutely, will do - but later this week. Sorry for the delay, out of the country on travel right now ... :-(.

Jun 23, 2016 at 11:28 PM
OK, testing with 2.4.5 here, but I admit - getting a bit confused. I think part of it is that HW vs. SW encoding is available even within some of the profiles ... so not really forcing an approach it seems. That said, on a smaller test file (to limit time spent trying different options), a few observations,
  • HW encoding seemed to work when setting the max resolution to 1280 ... but the output failed due to "Moving converted file to destination failed". Very odd, as after that, this same error didn't happen.
  • HW encoding failed when trying to convert to a higher resolution (1920), but the log file shows the following. Perhaps incorrect parameters?
    Stream #0:0: Unknown: none, SAR 1:1 DAR 0:0 (was the following for 1280 above, Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p, 1280x712 [SAR 267:268 DAR 120:67], 2535 kb/s, 29.97 fps, 29.97 tbr, 30k tbn, 59.94 tbc (default))
    Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
  • SW encoding passed ... but only because the HW detection failed (not above, but with the "SW profile"?),
    nvENC NVidia driver not detected or driver is too old => but nothing changed from above!
So, some oddities, and I admit - confused overall now ... as SW vs. HW is not being "forced". Perhaps test with "fixed" configurations, to force HW or SW?

And BTW, I am now seeing "green" overlay video with HW - that may be a driver or parameter thing ... not sure. Is there a way to manually convert, with minimal parameters?

Jun 25, 2016 at 5:44 AM
Green stuff is a bad graphics driver. Revert to a older stable version.

The HW failure is also due to a graphics driver. Try disabling cropping in the setting and it may solve the issue. This is because the buggy driver isn't able to handle odd resolutions. Some shitty drivers need a resolution with a multiple of 16 or square pixels so with cropping enabled sometimes it doesn't end up that way. Turn it off and you should be good. Notice the SAR/DAR ratios, very odd funky numbers. Not all encoders can handle it.

MCEBuddy will always use HW is the options is enabled in the configuration. It it's disabled it won't (but will still try to detect it so ignore that).

The move failure is likely due to your antivirus or some software which had a lock on the file preventing it from moving.

Jun 25, 2016 at 1:11 PM
OK, makes sense - thanks! Just a couple questions then, trying to get this working (as when I did, I found NVENC to be 2-3x faster than SW, and much lower CPU!),
1) Do you have a recommended NVidia driver to use? I keep it current, would have thought that would work.
2) I also have QSV in my CPU, but that never seems to get detected or used ... is there a way to force that to work?

Thanks for all the help!
Jun 25, 2016 at 3:50 PM
OK, got it working! I reinstalled the CUDA v7.5 SDK (downgrading driver, etc. in the process) - and that fixed it. It seems there is an issue or compatibility problem with v8 - just so you know, and also for others (this is how to get back to a stable driver version).

So then - I ran my NVENC (forced) encode, works fine. But I wanted to compare ... tried "MP4 Normal", and even though that profile seems to say to use Handbrake, it used ffmpeg and NVENC. Is it supposed to?

Do you have profiles to force SW encoding, and also QSV - to compare CPU load and encoding time? I'm trying it here, but obviously I don't understand the profile format, as I don't see why "MP4 Normal" us using ffmpeg and NVENC ... :(.

Jun 27, 2016 at 10:03 PM
See the FAQ thread on hardware encoding. If enable hardware encoding is checked for a conversion task it will always try hardware first.

Jun 27, 2016 at 10:04 PM
What version of the nvidia driver are you using?

Jun 28, 2016 at 12:43 AM
OK, makes sense on the logic - didn't know that part. Is there a way to force QSV over NVENC (for testing)?

On the driver - the latest version (with CUDA v8) wasn't working ... this was 368.39. I backed up to what comes with SDK v7.5 (v353.90), and that works.

Jun 28, 2016 at 12:53 AM
See the hardware encoding FAQ thread

Jun 28, 2016 at 12:55 AM
I did check - first place I looked. I may have missed it though, likely me. Will give it a go again.

Jun 28, 2016 at 3:45 AM
There is a section on how to enable Intel QSV using the dummy monitor trick

Jun 28, 2016 at 10:51 PM
Jul 8, 2016 at 3:37 AM
I also continue to struggle with using Nvidia drivers (using the latest donate beta version). Setting up a new rig with an I5-3570K, 16GB ram and a Nvidia GTX 970. I tested the rig without the GPU first and was reasonably pleased with pretty quick MP4 normal conversions of around 160 FPS. So I thought for sure once I added the GTX 970, things would be blazing fast. NOPE! the exact opposite, GPU utilization was practically nil, CPU utilization was high and the FPS was somewhere around only 50-60 FPS. And yes I installed the latest driver from Nvidia and that could very well be the same issue as described above.

I find the MCEBuddy software truly amazing! Still, despite all my searching and reading through the posts here, I struggle to wrap my head around the correct way to set up a Nvidia card. Are there any posts (despite the dozens I have read) that explain the process of configuring and using MCEBuddy with a NVidia GPU card? I can't believe that the integrated HD 4000 outperforms the GTX 970 when doing conversions. But the logs don't lie. 8-)

I'd be glad to post a log for review Rboy if you have the time and can provide the link once again... (sorry, I have lost the post where it was given...)
Jul 8, 2016 at 4:55 AM
See the read me before posting thread

Jul 9, 2016 at 1:34 PM
Thanks Rboy... Right there is the READ ME sticky ... uh DUH. I have uploaded my log and descriptive text file. in a folder named Nvidia Conversions.

Please have look see and steer me in the right direction, please. Also should I start a new thread and perhaps repost my issue?

Jul 26, 2016 at 3:45 PM
Okay I looked at your conversion logs, MCEBuddy has detected NVIDIA and has enabled it. In your case your driver is converting at 2.1x (about 60fps). I understand your Intel 4000 is giving your 160 FPS, these are hardware and driver limitations. Intel is highly tuned for video conversion (hardware and drivers), nvidia, I suspect the driver, has some way to go. Also another user tested is v353.90 which is part of NVIDIA SDK v7.5