RFE: API token for (long lived) data retrieval purposes

Post Reply
gtb
Posts: 87
Joined: Thu Oct 02, 2014 2:07 pm

RFE: API token for (long lived) data retrieval purposes

Post by gtb » Mon Mar 14, 2016 6:21 pm

RFE: API token for (long lived/never expiring) data retrieval purposes

The ability to obtain and use a long lived APIKEY (uuidgen -r?) that can only be used to retrieve data (not add/delete lineups, which would still need a short lived TOKEN).

The GET/POST methods used to retrieve data would be enhanced to allow a APIKEY in addition to a TOKEN header (for backwards compatibility).

In my ideal scenario, the APIKEY would be able to be created/regenerated/retrieved from the Schedules Direct site. (Alternative method (to website) to acquire existing/new APIKEY via a new method would just work as well for the usually one-time setups [GET (with TOKEN) to retrieve existing APIKEY associated with account (to possibly be stored for later use), DELETE (with TOKEN) to delete existing APIKEY in account, POST (with TOKEN) to get new APIKEY for account])

Adv: The ability to store a "retrieve EPG only" APIKEY in a place which might not be fully secure without the stored value used for automated retrieval being able to be used to add/delete lineups in the account (the sha1hash might not be your password, but it has all the power of your password).

Adv: Having an APIKEY which was not related to your Schedules Direct password would allow one to change ones Schedules Direct password independently of the APIKEY (allowing the option of good password and APIKEY management at independent time-frames).

Adv: The APIKEY could be used by multiple parallel applications accessing different lineups in your account without the problem of App A starts running and gets a token, App B (perhaps on another system) starts running a gets a token, invalidates token for App A, App A restarts, gets new token, invalidates App B token, which restarts, and gets new token.... and so on.

Dis: Having both a password and APIKEY might be confusing for some users (although any app could decide how to manage that process, passwords would still be supported in this proposal).

Dis: Clearly an additional lookup (and complexity) would be required (at some point) to determine which account is associated with what APIKEY.


This capability appears that it *could* be back-ported to the 20141201 API if accepted (there would appear to be no incompatible features added).

It should be noted that APIKEYs are used in some alternative offerings.

Thanks for the consideration (yes, I fully understand that even if accepted this is likely not to get implemented for quite some time).

rmeden
SD Board Member
Posts: 1526
Joined: Tue Aug 14, 2007 2:31 pm
Location: Cedar Hill, TX
Contact:

Re: RFE: API token for (long lived) data retrieval purposes

Post by rmeden » Mon Mar 14, 2016 9:22 pm

So, you want to share your SD-JSON access with others?

Yes, it gets around redistribution restriction, but using your account outside your household is not allowed.

Even if that's not your intent, it certainly makes it easier.

Robert

gtb
Posts: 87
Joined: Thu Oct 02, 2014 2:07 pm

Re: RFE: API token for (long lived) data retrieval purposes

Post by gtb » Tue Mar 15, 2016 10:12 am

rmeden wrote:So, you want to share your SD-JSON access with others?

Yes, it gets around redistribution restriction, but using your account outside your household is not allowed.
You are making an assumption without facts, and I am more than just a little unhappy that an official of the org would make such an accusation towards a subscriber/member on a public list.

Even if that was not your intent, your words were, at very least, sloppy. I will await your apology on list.


No, my real initial issue was that I have two different systems on different isolated networks (I perform network isolation for testing) in my location trying to use the refresh the data at "random" times (well, not so "random" for the test cases), and the token invalidation that occurred caused unnecessary breakage that I did not wish to add in substantial code complexity to deal with.

And I was thinking (but no more, clearly the organization is not interested in supporting such use cases) about an mobile app that would display the guide information that was running on my different devices wherever I am in the residence. Which meant refreshing the data wherever/whenever members of the household are, and that meant that random "token invalidation" errors were not going to be desirable/acceptable as continuous retries to get a new token caused different issues (lockouts/suspensions).

Of course, I understand that Schedules Direct may not be interested in assisting its members to use its data in their household the way they want to use the data. So be it. No organization is likely to be able to be fully responsive to all of the requests of their subscribers/members.
Even if that's not your intent, it certainly makes it easier.
I do not see that it makes it substantial easier, as anyone could share the sha1hash today, although I do grant it makes it easier to avoid having to code some additional retry variations. No one has provided evidence that people who are sharing care about sharing their sha1hash, although I can imagine that they do not like coding retry logic any more than I would.

But, for that matter, anyone can share the data downloaded after the fact if that was, indeed, their intent.

And if Schedules Direct is so concerned about data sharing why does it (in a trivial test I just ran) allow concurrent downloading of the DD transformed data? This RFE is (essentially) to allow what can be done today (concurrent data retrieval).

In any case, if you are going to be so concerned about data sharing, you need to think differently. There are many (well known) ways to perform analysis on the data to suggest fraud or abuse. Go there, rather than claiming/expecting that all your subscribers/members intentions are not to use the data legitimately.

rmeden
SD Board Member
Posts: 1526
Joined: Tue Aug 14, 2007 2:31 pm
Location: Cedar Hill, TX
Contact:

Re: RFE: API token for (long lived) data retrieval purposes

Post by rmeden » Tue Mar 15, 2016 10:27 pm

I would say a home network with your isolation rules are pretty rare. I'm not sure what fire you're playing with that's ok for the SD-JSON key (password) but not your SD login password (used to get the key). But it's your setup, go for it. Your SD password really doesn't protect any sensitive information.... we have your email address, everything else is for convenience and can be made up (and often is). We never see any CC or billing information (nor do we want to).

We all make assumptions. There's nothing wrong with that. I don't beleive I was disrespectful in my reply. While your reply separated the important next line of my response, I did accept that I could be wrong.

Since I offended you, I'm sorry. That was not my intent.

jagsd
Posts: 42
Joined: Sun Sep 09, 2007 10:52 am

Re: RFE: API token for (long lived) data retrieval purposes

Post by jagsd » Sat Mar 26, 2016 7:06 pm

Wow, this got needlessly hostile rather quickly.

It seems the principle problem is the invalidation of a token when a new one is requested. The current rules are that a token is invalidated either 24 hours after first granted or a new token is requested, which ever occurs first.

What is the motivation for these rules? Why not have get token return the currently valid token unless there is not one in which case generate a new one. Invalidate a token when last use was more than some specified time say an hour or 24 hours. It would still require valid credentials to do get token. And care must be taken to avoid race conditions.

This way two separate applications could run without collision and there would be the same level of security as the current system provides.

Of course all changes such as changing lineups would have to be atomic.

And I think this would be backwards compatible.

Would this work?

rmeden
SD Board Member
Posts: 1526
Joined: Tue Aug 14, 2007 2:31 pm
Location: Cedar Hill, TX
Contact:

Re: RFE: API token for (long lived) data retrieval purposes

Post by rmeden » Sun Mar 27, 2016 9:54 pm

valid point.. I don't see an issue with multiple devices hitting SD_JSON concurrently.. I can see that happening.

RobertK?

Post Reply