DataDirect replacement service going live!

Forum rules
Get Help using the Schedules Direct service. NOTE: application issues are probably better resolved via the application's support methods. If you post here, at least include your application name!
rmeden
SD Board Member
Posts: 1521
Joined: Tue Aug 14, 2007 2:31 pm
Location: Cedar Hill, TX
Contact:

Re: DataDirect replacement service going live!

Post by rmeden » Thu Oct 16, 2014 8:26 pm

skst wrote:I'm using my SD account user name and password. My user name is "skstskst".

I'm using my own application, unchanged from when it was TMS's DD feed. I've modified the hosts file as instructed in the first post to redirect it to the SD servers (just in case TMS's servers get fixed and I can switch back to them). I thought TMS's servers weren't going down until Nov 1.

Oh, yes, I've tried multiple times, and the same errors continue to occur.
If it's your own application, why did you edit your hosts file and just not change the URL you use to connect?

Are you forcing digest authentication? The new server is using BASIC authentication at the moment. (we need the actual password to get your TMS lineup). Most apps provide the authentication the server requests. (I'm seeing some "client used wrong authentication scheme" messages in the logs)

Yes, TMS's servers will be going down shortly after Nov 1. I've asked them to update their DNS to send us half the traffic. I figure it's better for folks to fail half the time now (so we can fix the problems) rather than 100% of the time Nov 1.

Robert

rmeden
SD Board Member
Posts: 1521
Joined: Tue Aug 14, 2007 2:31 pm
Location: Cedar Hill, TX
Contact:

Re: DataDirect replacement service going live!

Post by rmeden » Thu Oct 16, 2014 9:26 pm

Lineup support now live.. changes should now move to the new server.

skst
Posts: 49
Joined: Fri Aug 17, 2007 8:13 am

Re: DataDirect replacement service going live!

Post by skst » Thu Oct 16, 2014 9:34 pm

My apps are failing 100% of the time though, so I'm not very confident in that 50% figure. (I just tried five more times and every one of them failed.)

As I said, I edited my hosts file rather than my app because I thought TMS would fix the redirect and I didn't want to have to re-build my app again. I didn't realize this switch was intentional.

I'm using the same authentication method I was using before. Since it's changed from digest to BASIC, how can SD expect old apps to work with the redirected service? I suppose my app will have to remain broken until I finish my current project and get time to go back to it and code up BASIC authentication.

On a meta-note, I'm very disappointed in how this entire change from Web services to JSON has (not) been communicated to the SD users. I only found out about this whole thing by accident a couple of weeks ago, and I still haven't received an official e-mail notification from SD. I only know what's on these forums. Even that is not well organized. There are seemingly random threads, no documentation, no sample code, no test server, random PMs with unclear policies (a two-month dev account?). This was just a side project for me, and there isn't anywhere near enough time for me to update my own app. I'd hate to go back to using plain ol' XMLTV, so I'm considering dropping my use of TV listings altogether, even though I just renewed my SD subscription.
USA-IL58819-X
WOW Chicago Suburbs - Digital
Naperville USA 60005

rmeden
SD Board Member
Posts: 1521
Joined: Tue Aug 14, 2007 2:31 pm
Location: Cedar Hill, TX
Contact:

Re: DataDirect replacement service going live!

Post by rmeden » Thu Oct 16, 2014 10:50 pm

skst wrote:My apps are failing 100% of the time though, so I'm not very confident in that 50% figure. (I just tried five more times and every one of them failed.)
You can edit your hosts file and use Tribune's IP (get it from nslookup or dig.. it's the 144.* one). You should continue to work until they shut down their servers after Nov 1.

Most apps use standard libraries and don't implement HTTP directly. The library handles the authentication request.

The JSON API doesn't have anything to do with SD-DD or the Tribune shutdown.... it's been in development for quite some time. RobertK has it documented over at github... and I beleive it's quite through, but to be honest I haven't looked at it.

We didn't email the general SD membership about JSON because most members use apps, not develop them and won't care. You are right, I need to send an email blast (I think our first in 5+ years?) telling folks about the changes are coming. It's on my todo list.

Robert

ajlill
Posts: 9
Joined: Thu Sep 13, 2007 1:40 pm

Re: DataDirect replacement service going live!

Post by ajlill » Fri Oct 17, 2014 9:48 am

It looks like I can't connect to the old service either. Serious fail!

Code: Select all

--2014-10-17 06:38:44--  http://webservices.schedulesdirect.tmsdatadirect.com/schedulesdirect/tvlistings/xtvdService
Resolving webservices.schedulesdirect.tmsdatadirect.com... 144.142.232.53, 54.85.117.227
Connecting to webservices.schedulesdirect.tmsdatadirect.com|144.142.232.53|:80... connected.
HTTP request sent, awaiting response... 401 Unauthorized
Connecting to webservices.schedulesdirect.tmsdatadirect.com|144.142.232.53|:80... connected.
HTTP request sent, awaiting response... 401 Unauthorized
Authorization failed.
2014-10-17 06:38:45.003 DataDirect, Error: Failed to save DD cache! redownloading data...
--2014-10-17 06:38:45--  http://webservices.schedulesdirect.tmsdatadirect.com/schedulesdirect/tvlistings/xtvdService
Resolving webservices.schedulesdirect.tmsdatadirect.com... 144.142.232.53, 54.85.117.227
Connecting to webservices.schedulesdirect.tmsdatadirect.com|144.142.232.53|:80... connected.
HTTP request sent, awaiting response... 401 Unauthorized
Connecting to webservices.schedulesdirect.tmsdatadirect.com|144.142.232.53|:80... connected.
HTTP request sent, awaiting response... 401 Unauthorized
Authorization failed.
2014-10-17 06:38:45.372 DataDirect, Error: DataDirect XML failed to properly parse, downloaded listings were probably corrupt.

rmeden
SD Board Member
Posts: 1521
Joined: Tue Aug 14, 2007 2:31 pm
Location: Cedar Hill, TX
Contact:

Re: DataDirect replacement service going live!

Post by rmeden » Fri Oct 17, 2014 9:56 am

ajlill wrote:It looks like I can't connect to the old service either. Serious fail!
Maybe your account is expired? What's your SD username?

rmeden
SD Board Member
Posts: 1521
Joined: Tue Aug 14, 2007 2:31 pm
Location: Cedar Hill, TX
Contact:

Re: DataDirect replacement service going live!

Post by rmeden » Fri Oct 17, 2014 10:00 am

added a note to known issues saying that excessive hits to the server will cause a 1-day block.

For example: Tribune's servers were seeings over 10k download requests a day from two people with expired accounts.
There were many cases where people were sending over 5k download requests per day.

If your application gets an error, and you retry, be sure to back off before the retry (maybe an exponential retry).

Robert

ajlill
Posts: 9
Joined: Thu Sep 13, 2007 1:40 pm

Re: DataDirect replacement service going live!

Post by ajlill » Fri Oct 17, 2014 12:08 pm

My account is ajlill@ajlc.xxxxx and it doesn't expire till next june.
The problem started a 5am EST today, and was a connect to the replacement IP. My previous attempt was at 10am EST yesterday, so I doubt I was banned for excessive connections!

genbap

Re: DataDirect replacement service going live!

Post by genbap » Fri Oct 17, 2014 2:18 pm

My user name is genbap and I am using xmltv gui. As of this morning I cannot in any way using my app. Lost all channels and cannot select channels through interface. Is not working at all. Thought you should know.

skst
Posts: 49
Joined: Fri Aug 17, 2007 8:13 am

Re: DataDirect replacement service going live!

Post by skst » Fri Oct 17, 2014 2:24 pm

Isn't the hosts file format in the parent post incorrect?

At least on Windows, each host needs its own line, so it'd be like this:

Code: Select all

54.85.117.227  docs.tms.tribune.com
54.85.117.227  webservices.schedulesdirect.tmsdatadirect.com
You can edit your hosts file and use Tribune's IP (get it from nslookup or dig.. it's the 144.* one). You should continue to work until they shut down their servers after Nov 1.
Ah. I tried this and it works. It failed the first time, but maybe I didn't give the system a chance to read the new hosts file, or maybe I finally fell into the other 50%. We'll see how future requests go. Thanks!

Code: Select all

144.142.232.42  docs.tms.tribune.com
144.142.232.53  webservices.schedulesdirect.tmsdatadirect.com
Most apps use standard libraries and don't implement HTTP directly. The library handles the authentication request.
I'm using the built-in .NET HTTP classes, but no, AFAIK they don't automatically do digest or BASIC. I have to code whatever authentication request I want. Maybe someone who knows more about the .NET library's HTTP services can correct me. Networking isn't my bailiwick.
The JSON API doesn't have anything to do with SD-DD or the Tribune shutdown.... it's been in development for quite some time.
Unfortunately, there was no notice and I had no idea or else I might've had a chance to re-work my app before this all came down.
Last edited by skst on Fri Oct 17, 2014 3:31 pm, edited 1 time in total.
USA-IL58819-X
WOW Chicago Suburbs - Digital
Naperville USA 60005

Post Reply