Commercial skipping without removing from file.

Feb 2, 2013 at 3:41 PM
So I have been playing with comskip for commercial skipping. It works sometimes, and I can tune it for some of the series I am recording, but for general recording using it, and have MCEBuddy do the cuts is too much of a risk that it will remove parts of the show. I did find that with the DVRMStoolbox plugin for WMC, if you have the XML file that comskip generates, WMC will skip the parts marked as commercials, but let you scan backwards if it made an error. So when it gets it right it is hands off, otherwise you just do a little rewinding.

The problem is, MCEBuddy only has the option to keep the EDL files if you don't do the commercial cutting and it places them with the converted video. DVMRStoolbox needs the XML files in a common directory, which seems like a bad design to me, but that is what it is, and I guess one of the reasons I use MCEBuddy :-). Does anybody know of a plugin for WMC that will work with the EDL files to provide the same functionality of having WMC skip the parts of the files with the marked commercials?

Only other option would be to ask for a new MCEBuddy feature to support moving/renaming the XML files after running comskip.
Feb 2, 2013 at 7:12 PM
I would be interested in the techniques you used to tune comskip.

I was just starting to play with commercial skip in 2.3.11 when I switched to 2.3.12 Beta and noticed comskip.ini had changed and it did a better job. Which version are you using?

I'm still running with archiving on so I can check against the original if I have questions, but have found only one program in dozens so far with any noticeable portion was skipped. I do find that sometimes it seems like it cuts away abruptly as if it cut too soon, perhaps missing the last few seconds before commercial, but when I take the time to review the original it is never more than a few frames or spot on.

More often the error is failing to remove a commercial or two, which I can live with. I can ball park it by checking the total length of the processed program, a 1/2 hour program should be about 20 minutes and an hour program should be about 40 minutes, more often they are a few minutes longer and there are a few uncut commercials.

A question I have, if I wanted to modify the MCEBuddy configuration so that it left in a couple of seconds of overlap with the first commercial in a series of commercials, how would I do this? I've been thinking this would be an easy way to verify and build a confidence level.

You bring up an important topic as we need to trust commercial skip as arguably MCEBuddy's important feature.
Coordinator
Feb 2, 2013 at 10:53 PM
Extensively documented, refer to the documentation advanced commands. You can specify custom comskip.ini files for each conversion task
Feb 3, 2013 at 2:40 AM
lightman2: I am using the non-beta version, the tuning just amounts to looking at the characteristics of the show to see where it might have trouble (found some channels that don't put much fade to black between commercials, so comskip thinks they are all one commercial, so I had to tune the max commercial length). As for adding a couple seconds in, I think the padding config parameter will do what you are asking (I believe it is in seconds).

rboy1: Not sure whose question you were answering. If it was mine, I do know about using different comskip.ini's for conversion tasks, I already do that. My issue is that for general recording, where I can't easily tune comskip (since my family could records anything), I'd rather be able to keep the XML file that comskip can generate, and put it in a "commercial" directory. This would allow WMC to automatically skip the commercials during playback (but if they are wrong, you can still rewind to the part marked as a commercial, since the file itself is not altered). Since MCEbuddy has the option to keep the EDL file and not cut commercials, I figured that was put in place to provide the same functionality, but I can't find anything that allows EDL files to perform that function in WMC.
Coordinator
Feb 3, 2013 at 2:41 PM
@SportsWagon, the comskip note was for @lightman2. Regarding your XML in the next BETA build dated today I will include support for MCEBuddy to copy any XML files generated during the comskip (or conversion process) to the destination diretory. You'll have to configure comskip.ini to generate the XML file however. MCEBuddy will only look for an XML file with the same name and then copy it over if it finds it.
Feb 3, 2013 at 10:39 PM
After reading your comment indicating comskip tuning was extensively documented, I went back to be sure I did not miss this, but I do not think I did. I re-read, I found two paragraphs in the advanced commands and a reference to a more detailed page 'A short guide to tuning Comskip'. My original comment was intended to elicit more practical information as to how to go about tuning comskip which I could not find in the documentation which talks about how to specify a comskip.ini file and what the parameters mean but not how to go about the process.

For instance @sportwagon, you mention that you noticed that some stations don't put much fade to black between commercials, I would be interested in how you figured that out. The short guide references a debug window, I am thinking that this tuning has to be done outside MCEBuddy to preserve the generated comskip files and then using these files to start the debug window?

The option that I've found so far that might do what I asked is a comskip padding option that looks like it makes commercials smaller by taking some off the beginning and end, assuming I am understanding the documentation.
Feb 5, 2013 at 2:35 AM
So how I discovered there wasn't much fade to black on some commercials was observing comskips behavior. I noticed that it properly detected the start of a commercial block on the series in question, but then oddly always seemed to stop short of removing the entire block. A quick review of the parameter file showed that it was stopping because of the max commercial length parameter, (or that was my guess since it was the only parameter that matched closely to the section length cut). A quick few minutes spent watching the commercials in question and it was pretty obvious. There was a clear break before the block, but almost nothing between each commercial.

And yes I think padding will do what you said which is why I had suggested it before.
Feb 5, 2013 at 12:50 PM
So I tested the new beta last night, and indeed it nicely saves the xml file if it is generated. Thanks!

One thing I did notice, is that it copies the xml file no matter what (whether MCEbuddy is set to actually do the commercial removal or not after running comskip). For me I can easily deal with this, but for logical constancy, it should probably only be copied when the user selects "skip commercial removal" option (e.g. when the EDL file is copied). Otherwise, when the user selects having MCEbuddy do commercial removal and an XML file is generated, you end up with a video file without commercials, and a pretty useless XML file (since it indicates commercial locations that no longer exist), but if post processing from other programs try and use that XML file it will hack out parts of the show :-)
Feb 5, 2013 at 12:55 PM
And just another note of thanks for all the support you give. I saw the changes you made for another user with regards to customcommand parameters, and that change I believe will let me handle moving the XML file to where I need it, without having to use another application. Fantastic!
Coordinator
Feb 5, 2013 at 2:09 PM
MCEBuddy looks for an XML file (could be created from any 3rd party app). The question is if you aren't using comskip (ie. commercial removal) then comskip shouldn't run and there shouldn't be any XML file. Where is teh XML file coming from?
Feb 5, 2013 at 9:26 PM
Sorry my description wasn't clear.

So there are some shows that comskip doesn't reliably remove commercials for. So for those (and any random recordings family members setup) I have a profile that runs comskip, but has the "do not remove commercials" option checked in the advanced options section. It is for these shows that I want to save the XML file, since there is a plugin for WMC that will use the XML file of commercial locations, along with the uncut video and just "jump" over the indicated commercials (but you can ff/rew into the skipped portions if there is an error in the commercial location file).

I have other shows that comskip works well for, and for space reasons, I have MCEbuddy run comskip and do the commercial removal and save the shortened file. It is these that now get an XML file saved with them with the new feature you added (since I am using the same comskip profile). Now if I would use the XML file with the now truncated video file, the plugin will now skip what it thinks is a commercial, but in fact is regular content.

Now as I said, I can easily work around this (multiple ways to get rid of the XML files, without too much effort), but, if you are going to make this a standard feature, it seemed most logical to only save the XML file when you save the EDL file, e.g. when commercial skipping detection is run, but the commercials themselves are not removed. Otherwise (when MCEbuddy removes the commercials) the XML file is not really useful.
Feb 6, 2013 at 12:21 AM
Sorry I missed the reference to padding the first time but it works. I have it set to 2 seconds and it is reassuring to see the commercial briefly to be sure the cut is in the correct place.

At this point the majority of programs are being edited satisfactorily (some spot on!) by comskip other than some on ABC which seem erratic and one show, Private Practice, on that channel that seems to have some of the show scenes cut with some consistency.

Looks like I'll have to work with comskip at some point, still do not have a good feel as to how to go about it, but understand the parameters available just not sure how to determine where the problems are to know what to change.

Great project, support and discussion board!
Feb 6, 2013 at 3:04 AM
Yeah, it is a bit of a guessing game for some shows. For some, like Phineas and Ferb, it was always cutting the introductory song. Looking at it, I realized it was probably a lost cause to get comskip to figure this one out. The opening is about 30 seconds long, with fade to black before and after it. In addition it has no channel logos. So there is nothing to really differentiate it from a commercial at all! In fact, the more I look at the shows that comskip gets right, the more I realize it is those hated logos that probably allow it to do so well. Food channel, discovery channel, all have consistent, very obvious (even to a computer analyzer) logos.

One feature I wish comskip had was a way to say that a show has a certain format to the show/commercial pattern. If it did, Phineas and Ferb would be dead easy, as the it is always the opening + the first cartoon, then about 200 seconds of commercials, then the 2nd cartoon, another 200 seconds of commercials and the final 30 seconds of credits/ending joke. The show parts are always the same length, so if I could tell comskip those lengths, I'm sure it could do a perfect job. Oh well, in the case of cartoons it isn't so bad, since those compress so well they don't take up very much hard disk space anyway.
Coordinator
Feb 7, 2013 at 3:13 AM
sportwagon wrote:
Sorry my description wasn't clear.

So there are some shows that comskip doesn't reliably remove commercials for. So for those (and any random recordings family members setup) I have a profile that runs comskip, but has the "do not remove commercials" option checked in the advanced options section. It is for these shows that I want to save the XML file, since there is a plugin for WMC that will use the XML file of commercial locations, along with the uncut video and just "jump" over the indicated commercials (but you can ff/rew into the skipped portions if there is an error in the commercial location file).

I have other shows that comskip works well for, and for space reasons, I have MCEbuddy run comskip and do the commercial removal and save the shortened file. It is these that now get an XML file saved with them with the new feature you added (since I am using the same comskip profile). Now if I would use the XML file with the now truncated video file, the plugin will now skip what it thinks is a commercial, but in fact is regular content.

Now as I said, I can easily work around this (multiple ways to get rid of the XML files, without too much effort), but, if you are going to make this a standard feature, it seemed most logical to only save the XML file when you save the EDL file, e.g. when commercial skipping detection is run, but the commercials themselves are not removed. Otherwise (when MCEbuddy removes the commercials) the XML file is not really useful.
Aah so the root cause here is that you're using the same comskip profile. I would suggest you leave the default comskip profile without the XML generation. Then using the same base, create a new comskip.ini with the XML generation enabled, for the convresion task where you've checked Skip Commercial Cutting, in the comskip.ini box point it to the one enabled with XML.

Thus no extra XML files.
Feb 7, 2013 at 8:25 PM
Yes I did say this was easy to get around, and in fact I already did what you wrote above. I was just thinking from a design consistency standpoint, and especially if you make this a general feature, it makes more sense to only copy when you copy the EDL files. This might result in less questions by others if they try and exploit the feature in the future. But in the end it is your code to maintain/support, and the feature you add meets my needs, so I am happy. Thanks!
Coordinator
Feb 7, 2013 at 11:26 PM
From a design point, this is actually appropriate since today it's comskip tomorrow it may be some other software which may be customized. If MCEBuddy find a SRT, EDL, XML or NFO file it copies it to the desctination.
Feb 8, 2013 at 1:28 PM
Actually that was my point. EDL files are only copied when the "skip removing commercials" box is checked (which is the documented behavior), EDL files are generated anytime comskip runs, so today MCEbuddy doesn't copy EDL files if it performs the commercial removal. I thought it would make more sense to follow that pattern with XML files so it was consistent.
Coordinator
Feb 8, 2013 at 1:40 PM
Yes I understand your point, however Comskip is not the ONLY one creating XML files. XML files could be created for metadata or other purpose by 3rd Party apps.
So the behavior (I'll document it) of MCEBuddy is that if finds XML, SRT, EDL and NFO files with the source videos or if finds XML, SRT and NFO files during the conversion process it copies them to the output.
Feb 8, 2013 at 2:36 PM
ah ok now I see. You are distinguishing actions between what you find with the source and what you find during conversion process. Makes sense as well.
Feb 20, 2013 at 2:07 AM
@sportwagon: I uploaded some custom comskip.ini files to the MCEBuddy FTP site. See if they help. I would also be interested to see your tweaks to the ini's.

I have a question about using the comskip xml files created by comskip during a MCEBuddy conversion. Do you have to have dvrmstoolbox installed on your pc for wmc to recognize the xml file and allow the skip button to skip to the end of the commercial break? Does it work with recordings that are changed to MKV format or do the conversions have to be wtv files? Does it matter what directory they are in?

I may be in for some hands on dvrmstoolbox learning!

As of right now I use Recorded TV HD (WMC plugin that replaces the non-configurable default recorded tv) to watch my recorded TV original wtv files as well as my recordings that MCEBuddy converts to MKV. It is a great program because it seamlessly integrates and displays my wtv files in d:\recorded tv\ as well as all the mkv files that are in e:\show name\season#\ all in one place, with backdrops and folder art.
Feb 22, 2013 at 4:56 AM
I have the xml files being moved to the output directory along with the converted mkv's using 2.3.12. When I play the mkv in WMC I see the red segments of the video timeline (representing commercial breaks).

Image

2 Questions:

1) How do I get WMC to skip to the end of each red segment? When I hit the 'skip fwd' button on my remote I just get the normal 30 second skip ahead.

2) Is there a way to only move xml files to the output directory and not edl files also? I don't need both.
Feb 24, 2013 at 5:33 PM
Installing DVRMSToolbox and DTBAddin answered question 1.

Any thoughts on just moving xml and not edl to output directory?
Coordinator
Feb 24, 2013 at 5:42 PM
Wilky13 wrote:
Installing DVRMSToolbox and DTBAddin answered question 1.

Any thoughts on just moving xml and not edl to output directory?
Modify the comskip.ini (or use a custom one) and disable EDL output
Feb 25, 2013 at 3:04 AM
rboy1 wrote:
Wilky13 wrote:
Installing DVRMSToolbox and DTBAddin answered question 1.

Any thoughts on just moving xml and not edl to output directory?
Modify the comskip.ini (or use a custom one) and disable EDL output
Oh duh, me!! Sometimes you don't see the forest through the trees, haha!

Thanks, rboy1