Only getting 2 days of data from SD

Forum rules
Get Help using the Schedules Direct service. NOTE: application issues are probably better resolved via the application's support methods. If you post here, at least include your application name!
japaget
Posts: 10
Joined: Sat Aug 18, 2007 7:41 am

Re: Only getting 2 days of data from SD

Post by japaget »

I've run Wireshark and have captured the packets going to or from the TMS servers. How do I attach the file? ("Upload attachment" in the message editor doesn't work.) I'm guessing the problem is with packets 35 and 36, where a SOAP request is split across two packets. Packet 36 begins with a header followed by ":00Z", which appeared in the error message. Here's a hex dump of the packets in question:

Code: Select all

removed by moderator since it contains some personal info
BTW, it's after midnight local time and now I am getting only one day's worth of data from SD. I'll try again later today.

japaget
Posts: 10
Joined: Sat Aug 18, 2007 7:41 am

Re: Only getting 2 days of data from SD

Post by japaget »

I found an option that was recently removed from XMLTV that might have been helpful. It was called "--padd I<n>" and did the following:
tv_grab_na_dd.in source code from July 10, 2007 wrote: =item --padd I<n>

Add <n> spaces to the front of the start date. This is normally not needed,
but can be helpful in working around a Zap2IT problem when the request packet
spans TCP packets. Recommended initial value is "20". This is only needed if you get
"invalid start time" messages. If this helps, please post results to the list.

wepprop
Posts: 4
Joined: Sun Aug 19, 2007 3:40 pm

Re: Only getting 2 days of data from SD

Post by wepprop »

I've run Wireshark and have captured the packets going to or from the TMS servers. How do I attach the file? ("Upload attachment" in the message editor doesn't work.) I'm guessing the problem is with packets 35 and 36, where a SOAP request is split across two packets. Packet 36 begins with a header followed by ":00Z", which appeared in the error message. Here's a hex dump of the packets in question:
Ah, yes. "The endTime you specified is not in a valid format." Chuckle. I remember having that problem back in the first days of DD. It never impacted enough ppl for Z2L to fix it, and I was able to work around it by changing the MTU of my uplink. Maybe now that we're paying for the service they actually will fix it.

japaget
Posts: 10
Joined: Sat Aug 18, 2007 7:41 am

Re: Only getting 2 days of data from SD

Post by japaget »

I just checked my Windows XP registry and somehow my MTU is set to 1300 instead of the recommended value of 1492 or 1500. I will reset the values and report the results of running XMLTV sometime this afternoon.

rmeden
SD Board Member
Posts: 1561
Joined: Tue Aug 14, 2007 2:31 pm
Location: Cedar Hill, TX
Contact:

Re: Only getting 2 days of data from SD

Post by rmeden »

Accoridng to the sniffer trace, it looks like fixing the MTU will solve the problem. I personally set mine to 1400 instead of 1498 (higher than that wouldn't make any difference due to the max packet size for 802.11) due to some common router bugs with VPN software.

What's strange is why this is time of day related. I do think there's an issue around Midnight where you can't pull 14 days but 13 works.

I removed the --pad parameter because I was pretty sure it wasn't needed any more and it was truly a "hack".

The problem occurs when start/end time data spans a packet. The Tribune side *should* rebuild the stream properly but it doesn't. The problem originally surfaced when I upgraded Perl to a version that added additional stuff to the "envelope" pushing the start/stop time to the edge of the packet. The solution was to remove that stuff from the header.

I'll report this issue to Tribune.

Robert

ferrans@knology.net
Posts: 2
Joined: Sun Aug 26, 2007 7:36 pm

Re: Only getting 2 days of data from SD

Post by ferrans@knology.net »

I'm having the same problem, but I'm on GBPVR, and I'm in the central time zone - United States. I get anywhere from ONE HOUR to about 11 hours of TV listings, but never more than that.

Note the setting for 12 days.....
-=-=-=
2007-08-26 21:31:26.500 VERBOSE [1] getValue: /settings/Zap2itListingDays : 12
2007-08-26 21:31:31.062 INFO [3] Updating EPG for capture source: 1
2007-08-26 21:31:31.062 VERBOSE [3] getValue cached value: /settings/Zap2itListingDays : 12
2007-08-26 21:31:31.062 VERBOSE [3] getValue cached value: /settings/CompleteEPGReload : true
2007-08-26 21:31:31.062 VERBOSE [3] Zap2it full EPG update required.
2007-08-26 21:31:31.312 VERBOSE [3] Processing Zap2it/ScedulesDirect channels using SAX parser
2007-08-26 21:31:31.343 VERBOSE [3] Requesting listings for the period: 2007-08-26T00:00:00Z to 2007-09-07T23:59:59Z
2007-08-26 21:31:31.359 VERBOSE [3] Notify()
2007-08-26 21:31:31.359 VERBOSE [3] getValue() loading new key/value into cache: /settings/Zap2itPreferGzip
2007-08-26 21:31:31.359 VERBOSE [3] getValue: /settings/Zap2itPreferGzip : true
2007-08-26 21:31:31.453 VERBOSE [3] getValue() loading new key/value into cache: /settings/SchedulesDirectURL
2007-08-26 21:31:31.453 VERBOSE [3] getValue: /settings/SchedulesDirectURL : http://webservices.schedulesdirect.tmsd ... tvdService
2007-08-26 21:31:48.062 VERBOSE [1] labelAction.Text = Requesting listing information from Zap2it.com
2007-08-26 21:31:48.468 VERBOSE [3] Got zap2it response
2007-08-26 21:31:48.468 VERBOSE [3] Response header: Content-Encoding=gzip
2007-08-26 21:31:48.468 VERBOSE [3] Response header: Transfer-Encoding=chunked
2007-08-26 21:31:48.468 VERBOSE [3] Response header: Content-Type=text/xml;charset=utf-8
2007-08-26 21:31:48.468 VERBOSE [3] Response header: Date=Mon, 27 Aug 2007 02:31:45 GMT
2007-08-26 21:31:48.468 VERBOSE [3] Response header: Server=Apache-Coyote/1.1
2007-08-26 21:31:48.468 VERBOSE [3] Response header: X-Powered-By=Servlet 2.4; JBoss-4.0.4.GA (build: CVSTag=JBoss_4_0_4_GA date=200605151000)/Tomcat-5.5
2007-08-26 21:31:48.468 VERBOSE [3] Received compressed stream from Zap2it/ScedulesDirect
2007-08-26 21:31:48.500 VERBOSE [3] getValue() loading new key/value into cache: /settings/SaveZap2itXML
2007-08-26 21:31:48.500 VERBOSE [3] getValue: /settings/SaveZap2itXML : false
2007-08-26 21:31:48.500 VERBOSE [3] Notify(Requesting listing information from Zap2it.com)
2007-08-26 21:31:48.500 VERBOSE [3] About to parse listings
<b>
2007-08-26 21:31:48.515 VERBOSE [3] zap2it: The endTime you specified T23:59:59Z is not in a valid format, and has been corrected to a default value.
2007-08-26 21:31:48.515 VERBOSE [3] zap2it: The startTime you specified was invalid and has been corrected to 2007-08-26T00:00:00Z.
2007-08-26 21:31:48.515 VERBOSE [3] zap2it: The endTime you specified was invalid and has been corrected to 2007-08-27T23:59:59Z.
2007-08-26 21:31:48.515 VERBOSE [3] zap2it: Your subscription will expire: 2007-12-01T12:34:38Z
</b>

-=-=-=-=-=-=
So - any solution you guys come up with, could you e-mail it to me, too?

Post Reply