special notice from Schedules Direct: Re - mythfilldatabase

[m_theredhead]

Hello,

I just received this notice.

Code: Select all

Our logs indicate you are one of the people downloading data from the 
SD Data Direct service around the top of a midnight hour.

You were probably trying to be nice and pick an underutilized time for 
both you and us, but guess what...  many people had the same idea.  Our
top-of-the-hour load counts for U.S. midnight times are about 40 times 
our normal load!

In addition, it hurts you too, because that's when our data is the
least fresh. 

There is also no point in downloading multiple times a day.
There are no longer any mid-day updates.

The best thing to do is use your application's auto-schedule function 
(if it has it... MythTV does).  That uses an API to obtain 
a suggested time to download.

If you do want to pick a new time for a cron job, you can view the
output of that function at http://schedulesdirect.org/getack

To make it easier for you, here's a random time for your cron job:  
16:16 (hour is 00-24 not am/pm)

Please update your scheduled job.  The worst thing you can do is 
keep it the way it is.

If you any questions, you can contact us via the forum or at 
admin@schedulesdirect.org

I looked and indeed my last mythfilldatabase run was at midnight. However, I don't understand why it is doing this.

I have mythbackend running mythfilldatabase. In the 'Program Schedule Downloading Options' page, I have
[*] Automatically update program listings
[*]Guide data program execution start: 0
[*]Guid data program execution end: 23
[*]Run guid data program at time suggested by the grabber

I have also looked and I have no cron jobs that even run at midnight.

I am at a loss why mythtv would be pulling updates right at midnight and not using the suggested time.

Any thoughts or suggestions?

[rmeden]

Yea, there have been lots of reports about this.

Looking at http://www.schedulesdirect.org/getack I always get a good suggested time.

Someone replied via email and said that if the suggested time is less than 24 hours away, MythTV will will wait for 24 hours and then poll.

It's also possibly MythTV is treating the UTC as a local time.

I've asked some folks to check the MythTV code. Waiting for a response.

Robert

[cheetah]

I'm in a similar boat. I have both MythTV and xmltv usage, so I end up with two grabber runs per day (sorry?). The xmltv runs around noon local time, but MythTV seems to be running around 2 am local time.

I dug in the live mythtv settings, and I find this information:
Settings:
mythfilldatabaseLastRunStart = 2015-04-01T06:02:35Z
MythFillSuggestedRunTime = 2015-04-01T20:50:17Z
MythFillGrabberSuggestsTime = 1
Housekeeping:
MythFillDB lastrun = 2015-04-01 06:01:00 (UTC)

I looked at my mysql database backup from just before this run, and the prior values are:
mythfilldatabaseLastRunStart = 2015-03-31T06:00:28Z
MythFillSuggestedRunTime = 2015-03-31T20:18:30Z
MythFillGrabberSuggestsTime = 1
MythFillDB lastrun = 2015-03-31 06:00:00 (UTC)

So it looks like MythTV is ignoring the suggested grabber run time?

Edit: posts crossed in transit
I'll try fudging the values in the database so that it thinks the suggested runtime is more than 24 hours since the last run and can see if that makes it run at the suggested time instead of near midnight.

[rmeden]

I've bumped suggested times by a day... yea, a lot of folks may miss a day of data, but I should see a big difference in my logs.

Based on what others have noted via email, it seems the next scheduled run is being set correctly, but then around midnight a housekeeping job runs that reschedules it to run immediately!

Research still going on.

Robert

[ChristopherNeufeld]

I've seen this, and I believe I can explain it.

The MythTV backend will not run mythfilldatabase twice in the same calendar day (midnight to midnight local time). If it gets a suggestion to run a second time on the same day, it will postpone running until midnight, then it will immediately run.

The problem is that if mythfilldatabase runs shortly after midnight, schedulesdirect invariably supplies a suggestion to perform the next load later on the same day. It is this suggestion that is causing the bottleneck. If schedulesdirect were to ensure that it never suggested a time in the same calendar day as the most recently completed run, then much of the midnight congestion would likely vanish.

This behaviour of the MythTV backend is fairly recent, I suspect it was added to avoid overloading the schedulesdirect servers if they started issuing closely-spaced next-run suggestions. However, the decision to run as soon as the constraint was no longer met, and the constraint being the same local calendar day, seem to interact with the schedulesdirect suggestion algorithm in a non-optimal way.

I suspect it's easier to change on the schedulesdirect end than to wait for a bugfix to propagate out to MythTV installations.

[rmeden]

Yea, bad implementation... if SD says it's ok to poll at time x, then it's ok to poll at time x. (IMHO). When we had mid-day updates, that could have been useful. What they did here was basically tell everyone to download a midnight... oops.

I'll work to pick a random time *after* 24 hours. Since the best time for folks to poll is 0900-1800 central (so they have fresh data for east coast prime time) sometimes folks will just skip a day.

Even if MythTV would fix the algorithm folks probably wouldn't upgrade fast enough.

I'll work on a better algorithm on my end for distribution tomorrow.

Thanks for the explanation!

Robert

[kpatznh]

Here's a workaround for MythTV:

From a shell, run the command sudo crontab -e -u mythtv and enter your password when prompted.

You may be prompted to choose an editor. Pick nano or vim.basic (if you're familiar with the Unix "vi" editor).

Add this line to the bottom of the crontab file. The first two numbers (I chose 17 and 13) are the local minute and hour when the job will run (13:17 local time, or 1:17 PM). Choose a different time that isn't at the top of the hour.

17 13 * * * /usr/bin/mythfilldatabase --syslog --loglevel info >/dev/null 2>&1

Save the file and exit. If using vim, type :wq <enter> or ZZ to save and exit. If using Nano, hit Ctrl-O, enter, Ctrl-X to save and exit. The crontab will be automatically added for the mythtv user.

You should also go into mythtv-setup and tell the backend to not run the grabber itself. Uncheck "Automatically update program listings" in "Program Schedule Downloading Options." It's the last page in the General options.

Now your schedule download should occur daily at the time you specified in the crontab file.

[BrianH]

I got the same email from Schedules Direct... I'm more than happy to help them out by
altering my schedule download time, but I checked and I believe that I'm configured
to auto download at the time suggested by the grabber.

In the MythTV back-end setup menu I see that the following:

[X]- Automatically update program listings
Guide data program execution starts: [ 2 ]
Guide data program execution end: [ 5 ]
(These settings only allow integers! , not Hour:Minute as one might expect)

[X] - Run guide data program at the time suggested by the grabber.
(This setting is selected and should ensure that I run at the recommended time)

I looked on the mythweb status page and saw this:

Last mythfilldatabase run started on Wed Apr 1, 2:02 AM and ended on Wed Apr 1, 2:03 AM. Successful.
There's guide data until 2015-04-14 02:00:00 (12 days).

So it looks as if 2AM (approx) is the problem (I assume that's 2AM my time (east coast) which is
midnight in Mountain Time (MT). So why is Mythtv ignoring the suggested time?

I'm sure others have similar setting issues that cause the problem.

(Oh and by the way, I have no extra crontab job forcing a schedule download)

( mythbackend version: fixes/0.27 [v0.27-193-g8ee257c] )

[fphillips]

@BrianH
@cheetah

There is a bug in 0.27 that causes it to ignore suggested runtimes that are outside the window set, even though the window shouldn't be a consideration in your case. It is fixed for 0.28. In the meantime, you can set your window to 0-23 so it doesn't miss any runtimes.

If you will do me a favor, run the script at viewtopic.php?f=5&t=2541#p7623 and pastebin the contents of /tmp/health, I would appreciate it.

[gtb]

Yea, bad implementation... if SD says it's ok to poll at time x, then it's ok to poll at time x. (IMHO).

I would agree, but only with the requirement of some "sanity" checking (you do not want to enable a server DoS attack on clients by allowing a server to reply "(re)try now, (re)try now, (re)try now" again and again and again). And a broken server could self-DoS itself by replying foolishly to everyone to repeatedly suggesting to (re)try yesterday due to broken clock synchronization (yes, it does happen). A robust client implementation would insure that the time was sane (if it was before current time, report to admin and try again in no less than a few hours, and had a delay of (at least) a few hours, and the delay of no more than a few days). And a robust server does not trust blindly trust gmtime() (it goes out and checks with a few well known internet services (such as NIST) where the date can be verified as sane (if more than (say) 24 hours out of sync, do something rational)). As with all else, such code does not tunnel from another universe, and has to be written.