Request for comments: API 20150501

rkulagow
SD Staff
Posts: 915
Joined: Tue Aug 14, 2007 3:15 pm

Request for comments: API 20150501

Post by rkulagow » Sun Apr 26, 2015 6:42 pm

Keep the existing "batch" structure, or switch to one-item-per-call, with etags being used to indicate whether something has changed?

For example, if the URI was something like

/schedule/20454/2015-04-27

And using etags to determine if the schedule has been updated versus the scheme that we have now.

The negative is that you're now potentially making a few thousand requests (number of stations * number of days) to determine which stations need to be downloaded. And downloading programs would also be one-at-a-time potentially.

Anyway, soliciting ideas.

GameGod
Posts: 60
Joined: Fri Aug 17, 2007 12:26 pm

Re: Request for comments: API 20150501

Post by GameGod » Sun Apr 26, 2015 7:31 pm

I, for one, would like to keep the API as stable as possible. This is quite a radical change, and unless there's an enormous benefit to either other users or to you (bandwidth, for example) I would rather not have this change. I am not opposed to new features, but every time the API changes, it means I have to spend time on understanding all the breaking changes and making sure I upgrade my client to keep it working.

Thanks.

rkulagow
SD Staff
Posts: 915
Joined: Tue Aug 14, 2007 3:15 pm

Re: Request for comments: API 20150501

Post by rkulagow » Sun Apr 26, 2015 7:41 pm

I try to be as developer-friendly as possible, so noted. The "Good Idea Fairy" came by and hit me with her magic wand, but that doesn't mean that it's going to happen.

GameGod
Posts: 60
Joined: Fri Aug 17, 2007 12:26 pm

Re: Request for comments: API 20150501

Post by GameGod » Sun Apr 26, 2015 7:56 pm

Understood, and thank you for that :)

From a purely "idea" standpoint, it seems like it would be very resource intensive to make lots of calls to get relatively very small amounts of data back. This would be true for any client that doesn't have any cached data. It also assumes that clients implement data caching correctly and use it to reduce traffic. It would only take one (or a few) badly coded clients to offset any potential savings on the server side.

However, I could see this as an "add on" API so one could switch to this if the client was only doing a delta update, for example. Speaking for myself, I run the my client twice a day, once in the mid-morning and once in the late afternoon to catch any last minute changes to prime time schedules. I could see myself using this delta API for the afternoon update, as I know that there's very little chance of large changes coming thru.

Slugger
Posts: 77
Joined: Sun Sep 18, 2011 1:22 pm

Re: Request for comments: API 20150501

Post by Slugger » Mon Apr 27, 2015 7:04 pm

Along the same lines, what I'm most interested in at this point is maintaining back levels of the api instead of retiring them as new ones come online. This eases the burden of developers having to keep up with the latest and greatest and also gives you more flexibility to experiment/add/modify without breaking existing clients because existing clients can just stay on the existing api level and need not upgrade.

I'm all for new features/enhancements, etc. but really only if the existing api level is kept long term. If I have absolutely no need for any of the new features in the works and the current api level does everything I need it to do then I don't want to have to fix/update/reimplement my client for the new api version just because you're retiring the current one I'm happily using.

If my users are happy with what the 20141201 api provides then I'm hoping they can run on it forever (well, close to forever). If maintaining back versions of the api is not in your plans at all then I'm much more interested in maintaining the status quo as much as possible and only adding things to the data formats as opposed to removing or modifying data (in a client breaking way). Client breaking changes to the data structures returned by the endpoints are not fun, especially when your users are dead in the water if you don't update your client when you retire old versions of the api.

I don't think it's reasonable to ask that every api version be maintained but at least one level back seems reasonable. And if not forever then at least a long enough overlap that I don't have to rush to update things when my users don't need it. I'm thinking like at least 6 months of overlap, preferably longer (and really I'd prefer it just be kept forever).

Edit: I think going to this single request with etags is going to slow things down. You'd have to test it, but my gut tells me multiple single requests are going to be too much overhead vs. the current bulk approach. Probably less CPU intensive on your side, but going to cost more in bandwidth (the overhead of HTTP is going to be paid once per day per channel requested whereas the overhead is only paid once for bulk request and I guess most users only need to do one bulk request to download their schedules, 2 at most). For me, to get all my schedules by this method would be 105 channels * ~14 days (most channels have 20+ days now it seems, but let's use 14) == 1470 requests to the serer vs. 1 to get all the schedules for my 105 channels.

achuk
Posts: 17
Joined: Mon May 19, 2014 7:48 pm

Re: Request for comments: API 20150501

Post by achuk » Sun May 17, 2015 3:07 pm

As mentioned on Github by someone else, the server should not return 400 (Bad Request) for a /lineups request where the user has not subscribed to any lineups. The server should return an empty array. 400 should be reserved for 'true' bad requests to the server.

I am also concerned with the rate at which these APIs are being retired. 20140530 is less than a year old. I am not clear why backwards compatibility cannot be maintained by simply adding extra information to the JSON objects. Clients that properly parse JSON will simply ignore the new fields. Clients can be updated to use the new information at the developers' discretion. I understand breaking changes are occasionally required, but those releases should be less frequent.

What is the minimum amount of time that 20141201 will be supported? How long will 20150501 be supported once introduced? This is information developers require to plan their donated time.

jagsd
Posts: 42
Joined: Sun Sep 09, 2007 10:52 am

Re: Request for comments: API 20150501

Post by jagsd » Mon May 25, 2015 7:13 pm

In 20141201, the post data for both /schedules and /schedules/md5 are the same; namely, an array of stationID and date array pairs. I expected the response to both requests to be similar but they are not. I much prefer the md5 response structure; an object keyed by stationID whose value is an object keyed by date for the md5/lastModified data. I expected the /schedules response to be structured similarly with the value of the date key being an object with a programs key and a metadata key.

Is there some reason the /schedules data instead returns as an array of objects organized by date with keys stationID, programs, and metadata and the date found in the metadata? This does not lend itself to lookup by the keys which inform the request post data.

I understand this cannot change for 20141201 but is it worth considering for 20150501?

jagsd
Posts: 42
Joined: Sun Sep 09, 2007 10:52 am

Re: Request for comments: API 20150501

Post by jagsd » Mon Jun 01, 2015 11:02 am

Similar to my request May 25 just above, I request that /programs return an object keyed by programids rather than an array of programs. Since programids are supposed to be unique in the request, one cannot rely on ordering in the array to any useful purpose but reading the JSON in as an object provides easy lookup. In my javascript code I simply iterate the array and create such an object but it is wasteful to have read the array in the first place.

rkulagow
SD Staff
Posts: 915
Joined: Tue Aug 14, 2007 3:15 pm

Re: Request for comments: API 20150501

Post by rkulagow » Fri Jun 05, 2015 10:03 pm

Both of your ideas are something that I'm considering for API 20150501; I'm trying to make the parser required on the client side as generic as possible, within reason.

rkulagow
SD Staff
Posts: 915
Joined: Tue Aug 14, 2007 3:15 pm

Re: Request for comments: API 20150501

Post by rkulagow » Fri Jul 31, 2015 7:04 am

Next up for consideration: HTTP response codes. When should a 200 be returned, versus 400, versus 403.

Should all responses be 200 as long as the request made was syntactically valid JSON, with the body of the response indicating the "true" error using the "code" and the resulting short text / human-readable text?

Post Reply