This project has moved. For the latest updates, please go here.

Keep database of everything

Feb 4, 2014 at 10:31 AM
Edited Feb 4, 2014 at 10:36 AM
Hello Rboy,

I had an idea for great functionality over the weekend when i found many 'orphaned' converted files in my destination folder.

I am using feature that allows MCEBuddy to keep Converted and Original files in sync. This means if I delete file in WMC, converted file gets deleted as well.

This is awesome feature but as is implied in the readme file, deletion attempt happens only once. This means in case drive is not mounted, or NAS is just rebooting, MCEBuddy will not attempt to delete the file again.

Would it be possible to add a timer to attempt to perform unsuccessful jobs on regular basis ??? ( same as searching for files ( currently set to every 3 mins ) is currently used ). Keep it in the different "history" file so that we can clean up this queue on demand.

Also, same feature as described above could be added for "sorting". Right now, if destination is unavailable, MCEBuddy dumps the file to designated folder and forgets about it. It would be cool if on same timer as above, MCEBuddy attempted to sort those recordings multiple times until it succeeds (or via custom settings auto clean up queue for jobs older than X days/weeks/months )

I know it sounds complicated but GUI for the functionality would pretty much stay the same. It could be called "Keep Converted files in sync" with one extra button to "clean sync" to delete previous history log. Timer could be set to default value (same as searching for new files in monitored folders) and changeable by user via advanced settings or mcebuddy.conf file.

Thanks
Coordinator
Feb 5, 2014 at 12:59 PM
Nice idea. How can you offer entire between the file has been deleted and the file doesn't exist ?

Let say the network is disconnected for some reason. There is no way to tell whether the function deleted or could not delete.

Hence the timer would keep trying to deleted already deleted file and get into an finite loop. Did I miss something here?

Feb 5, 2014 at 3:22 PM
Edited Feb 5, 2014 at 3:39 PM
How can you offer entire between the file has been deleted and the file doesn't exist ?
-> I dont quite understand.

Let say the network is disconnected for some reason. There is no way to tell whether the function deleted or could not delete.
-> This is true.

Hence the timer would keep trying to deleted already deleted file and get into an finite loop. Did I miss something here?
->Single run would run same as it does now. Attempt to delete the file - if file isn't present - skip and move on.
The difference between proposed idea and current solution is that 3 minutes later, another attempt would be made to delete the file. If it's unsuccessful again, skip and move on. And on and on. Like I said, there would be a button in the GUI to clear out the history file to avoid leaving these jobs in the queue forever. So if you move files around outside of MCEBUddy or monitored folders are renamed or mapped drive changes letter, user would just use "Clean history file" button and all records for attempts will be deleted.

Plus automated mechanism could be added later to clean up the queue of attempts in the history file. Just like now there is a GUI for monitored folders and how old the file has to be to be picked up by MCEBuddy, there could be same GUI for how OLD the entry in the history file has to be for attempts to be made. So for example, if set up "keep trying for 7 days" for 7 days attempts will be made, and after 7 days MCEBuddy would ignore entries older than 7 days in the history file.

Workflow would be:

1, User delete source file -> MCEBuddy attempt to delete destination file
2a,File found and delete -> nothing is written to history file.
2b, File is NOT found therefore not delete -> write date, path and instruction to the history file.
3, X minutes later, history file is parsed and instruction is attempted
3a, if unsuccessful = do nothing and try again in X minutes
3b, if successful = delete entry from history file so operation is not made again.

4, Users presses "Clean history file" button = all entries in the history file are deleted.

Advanced workflow would be the same except ...
........
2b, File is NOT found therefore not delete -> write date, path and instruction to the history file.
3, X minutes later, history file is parsed and instruction is attempted IF it's within date range set up by the user
3a, if unsuccessful = do nothing and try again in X minutes IF it's within date range set up by the user.
3b, if successful = delete entry from history file so operation is not made again.

4, Users presses "Clean history file" button = all entries in the history file are deleted.

I hope this make sense. I've put alot of thought into this as you can tell :-) This way user can have VERY much automated iTunes/Plex/XBMC library of recordings or low/high quality of video clips. Once source is delete - destination will be guaranteed to be deleted ( of course unless user hasnt messed up path - which is imposible to code for )
Coordinator
Feb 5, 2014 at 4:41 PM
The key assumption you mentioned below is :

If it's unsuccessful again, skip and move on

There is way to tell right now if the delete command was successful or not. A limitation of the way Microsoft has Designed .NET

You see the issue here?


Feb 5, 2014 at 5:44 PM
rboy1 wrote:
The key assumption you mentioned below is : If it's unsuccessful again, skip and move on There is way to tell right now if the delete command was successful or not. A limitation of the way Microsoft has Designed .NET You see the issue here?
You mean there is NO way to tell if command was successful or not ? If so , no I did not know that. I've not touched Windows machine (besides HTPC) for decade.

I'm sure there is a simple work around though, no ?

First part of the delete function would be to verify whether the file ( or path to the file) exists (dir command on Windows, i suppose) and if it does, issue delete command.
This way you can reach 99% certainty that file WILL be deleted at the end of the function.

If path to the file does NOT exist, skip delete command and log it into the history file to be attempted again.
Coordinator
Feb 5, 2014 at 6:04 PM
you'd think that Windows would return a simple way to verify that right?

the command used to delete a file is File.Delete, that function does not return any notification of whether the file exists or not.

File.Exists is used to check the presence of a file.

the funny part is that both use the same underlying API's to manage files. In your case File.Delete does not work because the network if offline. Here File.Exists will also fail since the network if offline.

Either way it doesn't solve your issue.

The only way to get this logic to work is to figure out *if* the network offline - I don't know of any simple way to do that.

My question is more fundamental - if you're using a LAN/Home Network - WHY is your network offline? It's a pretty basic assumption that network should be connected all the time.


Feb 5, 2014 at 7:51 PM
Personally, I convert ALL recorded TV in our house for Plex and iTunes. Network is connected at all times but sometimes it can happen that storage server is being rebooted, not mapped to HTPC (MCEBuddy machine) or just down some one reason or another.
File.Exists is used to check the presence of a file.
Either way it doesn't solve your issue.
Doesnt File.Exists actually solve the issue ?

If File.Exists = true, go to File.Delete or else log date, time, path of the file into history file and attempt again.
Coordinator
Feb 5, 2014 at 9:38 PM
If the file has been deleted. File.exists will return false and it will go into an infinite loop (say you delete it accidentally)

Feb 6, 2014 at 8:38 AM
If File.Exists returns false, attempt will be made next time function is called. If file is deleted accidental or outside of MCEBuddy function, user has two options to remove it from "infinite loop".

1, (Manual) Clean database button to clean up all of these queued up jobs in the history file.
2, (Automatic) Clean up all entries in the history file older than X hours/days (X is to be defined by the user, same as currently monitored folder is )
Coordinator
Feb 6, 2014 at 3:49 PM
With the next build MCEbuddy will keep retrying to delete the converted files if the source file does not exist and there is a conevrted entry in the history file


Marked as answer by rboy1 on 2/7/2014 at 1:03 PM
Feb 7, 2014 at 8:27 AM
Wow, thanks. I will definitely test this. Can you let me know how it works and what is the default timer and how long will MCEBUddy re-try deletions ?

Or will it be in the release notes ? I think many people who like to keep their MCE and iTunes library in sync will appreciate this feature.

Thank you once again.
Coordinator
Feb 7, 2014 at 12:48 PM
Tries indefinitely every time it scans for new files (time is synced with poll period)

Marked as answer by rboy1 on 2/7/2014 at 1:03 PM