Only getting 2 days of data from SD

Forum rules
Get Help using the Schedules Direct service. NOTE: application issues are probably better resolved via the application's support methods. If you post here, at least include your application name!
japaget
Posts: 10
Joined: Sat Aug 18, 2007 7:41 am

Only getting 2 days of data from SD

Post by japaget »

I'm running XMLTV 0.5.48 under Windows XP Professional SP2 to grab 14 days of listing data as follows, but I'm getting an error. (It worked OK on August 20.)

Code: Select all

F:\tv>xmltv tv_grab_na_dd --dropbadchar --share .xmltv --days 14 --debug > tv.xml
Timezone is -0700
using config filename ./.xmltv/tv_grab_na_dd.conf
Using TZ=<+0000> offset=<0>
dd_start: 2007082400:00:00,2007-08-24T00:00:00Z
dd_stop : 2007090700:00:00,2007-09-07T00:00:00Z
Fetching from Schedules Direct
    dd_data is in C:\DOCUME~1\Jim\LOCALS~1\Temp\tv_grab_na_dd_LsSw.tmp
Filtering Bad Characters
    Fetched 1622 k/bytes in 4 seconds
loading data: ##################################################
NOTE: The endTime you specified :00Z is not in a valid format, and has been corrected to a default value.
NOTE: The startTime you specified was invalid and has been corrected to 2007-08-24T00:00:00Z.
NOTE: The endTime you specified was invalid and has been corrected to 2007-08-25T23:59:59Z.
NOTE: Your subscription will expire: 2007-11-27T20:39:09Z
Writing schedule: #################################################
Downloaded 2172 programs in 30 seconds

F:\tv>
My tv_grab_na_dd.conf file:

Code: Select all

username: (redacted)
password: (redacted)
timezone: +0000
lineup:   CA04885:-
channel: 2 KCBS
channel: 4 KNBC
(33 channels not listed)
channel: 96 WGNSAT
I'm not sure if the problem is with SD or with XMLTV 0.5.48. Please fix or report to TMS. Thanks!
(Using 0.5.47 with Zap2it Labs still gives me the full 14 days.)

rmeden
SD Board Member
Posts: 1576
Joined: Tue Aug 14, 2007 2:31 pm
Location: Cedar Hill, TX
Contact:

Re: Only getting 2 days of data from SD

Post by rmeden »

I think it has to do with the TMS profile, time of day, and grabbing 14 days.

I see it too. (and am capturing info for TMS)

Try 13 days and see if it works.

japaget
Posts: 10
Joined: Sat Aug 18, 2007 7:41 am

Re: Only getting 2 days of data from SD

Post by japaget »

rmeden wrote:Try 13 days and see if it works.
I tried 13 days and then removed the "--days 13" effectively giving me 7 days. I still got the same result. Perhaps the TMS DD SOAP interface doesn't like something about the formatting of the dd_stop value it is receiving?

rmeden
SD Board Member
Posts: 1576
Joined: Tue Aug 14, 2007 2:31 pm
Location: Cedar Hill, TX
Contact:

Re: Only getting 2 days of data from SD

Post by rmeden »

grrrr :x

I get the error when I use --days 14, but don't with --days 13.

I checked and the _na_dd change that fixed the bad dd_stop thing is in there.

Did you have problem with Z2L and bad stop times?

Try a version of xmltv.exe before SD and see if that works. I'll put one up on http://xmltv.org/alpha

If that doesn't work, I may have to capture a sniffer trace with Ethereal.

japaget
Posts: 10
Joined: Sat Aug 18, 2007 7:41 am

Re: Only getting 2 days of data from SD

Post by japaget »

"--days 14" is now working for me once again.
rmeden wrote:Did you have problem with Z2L and bad stop times?
Never. Occasionally it would adjust my stop time so I go only 13 1/2 days of data instead of 14, but Z2L never gave me only 2 days' worth of data or returned the error message

Code: Select all

NOTE: The endTime you specified :00Z is not in a valid format, and has been corrected to a default value.
rmeden wrote:Try a version of xmltv.exe before SD and see if that works. I'll put one up on http://xmltv.org/alpha

Try with with Z2L or SD? How can a pre-SD version be tried out with SD? I don't understand.
rmeden wrote:If that doesn't work, I may have to capture a sniffer trace with Ethereal.
Ethereal is no longer being supported and has been replaced by Wireshark.

rmeden
SD Board Member
Posts: 1576
Joined: Tue Aug 14, 2007 2:31 pm
Location: Cedar Hill, TX
Contact:

Re: Only getting 2 days of data from SD

Post by rmeden »

Glad to hear your working again. If you can get a sniffer trace with the capture, they may help. Strange that your account failed and mine worked.

I know about the new name for Etherreal... I still call it the old one... same with GAIM/Pidgin. :)

I was suggesting you try your old Z2L account (old grabber and old account) to see if it worked when the SD stuff didn't... too late now as it's working again.

Robert

hdtvincr
Posts: 7
Joined: Fri Aug 17, 2007 12:41 pm
Location: Cedar Rapids, IA
Contact:

Re: Only getting 2 days of data from SD

Post by hdtvincr »

I recall this popping up several months ago for our CW_EPG app. I don't recall all of the details, but we just changed our app to only do 13 FULL days. This ensures that when timezones are corrected, you have a full days worth of data.

By trying to fetch day 14 data and adding the timezone correction, if the call was made later in the day it would kick you over the 14 day limit and resort to the default like you mentioned. Of course a check could be implemented to check for that, but you would still only receive a pertial days worth of data.

I always thought it would be nice for Zap to resort to the very latest possible data time instead of kicking you back to 2 days.

japaget
Posts: 10
Joined: Sat Aug 18, 2007 7:41 am

Re: Only getting 2 days of data from SD

Post by japaget »

rmeden wrote:I was suggesting you try your old Z2L account (old grabber and old account) to see if it worked when the SD stuff didn't... too late now as it's working again.
Actually it isn't too late. I'm running Z2L and SD in parallel, so yesterday I also ran my Z2L with the old account and old grabber. Z2L gave me 14 days of data.

I suspect that this problem may recur whenever the GMT time is past midnight but the local (California) time isn't. I'll try again this evening around 10 PM local time, 0500 Zulu.

Robert, I really appreciate all of the hard work you have been putting into this project to make it a success!

-- Jim

japaget
Posts: 10
Joined: Sat Aug 18, 2007 7:41 am

Re: Only getting 2 days of data from SD

Post by japaget »

As expected, the download gave me only two days after 5 PM local/0000 Zulu. Setting "--days 12" did not fix the problem. Could XMLTV be sending a SOAP request that was acceptable to Z2L but not the new TMS DD?

rmeden
SD Board Member
Posts: 1576
Joined: Tue Aug 14, 2007 2:31 pm
Location: Cedar Hill, TX
Contact:

Re: Only getting 2 days of data from SD

Post by rmeden »

japaget wrote:As expected, the download gave me only two days after 5 PM local/0000 Zulu. Setting "--days 12" did not fix the problem. Could XMLTV be sending a SOAP request that was acceptable to Z2L but not the new TMS DD?
Great.. you figured out a reproducible pattern. You can compare the two times in Wireshark... it's in a HTTP POST message, or send me a capture and I'll take a look.

I'm also seeing similar results, except 13 days still works for me.

I haven't changed the date math at all between Z2L and SD. What time zone is in your config file?

Post Reply