There is a "bunch" of useful data available only in the generic SHow program that an EPisode program is part of. A schedule only provides a pointer to the one program, and the program hash, and not to the additional generic SHow program (nor its hash).
Items found (in some programs) which have been found to only be in the SHow and not the EPisode include actors roles, and total seasons. I have not done a more extensive check. But those are enough.
So, for something to amass the full information for an EPisode program, it is necessary for obtain all related unique program SHows that one has an EPisode for, and wants to process. As there is currently no hash for the (generic) SHow program provided, that means either requesting every SHow every time, or producing inaccurate data because the data is old and obsolete (how old or obsolete cannot be know until you fetch it) or presenting data which is not as complete as one can should be able to present.
In one test lineup, that is 50,000 SHows that need to be fetched each and every time the guide data is processed (just to give one an order of magnitude of the potential impact).
It is clearly desirable that apps be able to provide the best data available all the time.
As I understand it, the intent of the JSON api is to allow apps to use the hashing to minimize to load on the SD servers. To continue to do so, some changes are needed.
Here are a couple of possible solutions (that would not require an JSON API level change).
(sorry for the code formatting, but trying to use bbcode list is just not worth it at the moment)
- Code: Select all
* At original data load time at SD, build any EPisode program
entries with data from any generic SHow information, and
with the EPisode specific data presented in the schedule
reference. As is currently, one schedule entry points to
one program (with a hash).
- This means you get one (and only one) program
to deal with in the schedule, and makes the apps
easier to write.
- This (potentially) means a lot more preprocessing by SD
* Provide in the schedule data an additional (generic)
SHow program id and hash (in addition to the EPisode
program id and hash) [obviously the id can be calculated,
but it might be more flexible to provide it in case the calculation
might change with future data formats]. If the field does not
exist, there is no generic info available.
- This means apps need to (optionally) pull in additional
program entries, and do (essentially) what proposal 1
would do, except every app gets to do it.
- This allows apps to cache the SHows correctly because
there are hashes available to validate.
* Provide in the EPisode program an additional "generic" field
for the SHow and hash (essentially the previous reference is
moved from the schedule to the program elements)
- This means the apps need to (optional) pull in additional
program entries, and do (essentially) what proposal 1 would
do, except every app has to do it.
- This allows apps to cache the SHows correctly because
there are hashes available to validate
* Provide a new API call to return the hash(es) of a provided
list of programs.
- The app must calculate the SHow ids of interest (not a
great burden).
- This allows an app to validate the local hash for program
SHows that are downloaded for caching.
- The app will still need to do combining of the SHow data
with the EPisode data and do what proposal 1 would do,
except that every app has to do it.
- it seems this would be easy for SD to do?
* Do nothing (on the SD end), and apps can pull down all
needed (50,000?) generic SHow programs every guide refresh.
- This seems to go against the intent of a "cache if you can"
approach.
- This will result in a lot more load on the SD server
- The app will still need to do combining of the SHow data
with the EPisode data and do what proposal 1 would do,
except that every app has to do it.
* Do nothing (on the SD end) and apps can decide to cache
SHow data which means some of the time the data will be
wrong.
- Presumably apps will still need to randomly invalidate
the cache and generate the load(s).
- The app will still need to do combining of the SHow data
with the EPisode data and do what proposal 1 would do,
except that every app has to do it.
* Do nothing (on the SD end) and apps can ignore the issue,
and not provide full guide info to their users.
- Which is where some apps likely are today. It is where
my app is, as I was not aware that (for example) actor roles
were only in the SHow, not reliably presented in the EPisode
(sometimes they are in the EPisode program, and sometimes
not. I only checked a few during testing. Apparently I chose
badly)
No matter the choice, I would request to better document the solution/issue in the JSON api docs.
Obviously as a consumer of the data, the first approach is appealing to me (minimal to no changes on "my" end). But except for the last three (the do nothing on the SD end) any would be a solution I can live with as far as I am concerned. The only disadvantage for the second to fifth approachs is that each app has to do the programming to do the combining of the data, rather than SD doing it for everyone. But the counter argument is that some apps will not combine at all, and that each app might choose to combine differently (which likely has other issues, but ....)
So, does the issue make sense, or do I need to try to re-explain it?
And if it makes sense, comments, alternatives?
I recognize that this is going to be a slightly longer term discussion, and any resolution will take time. But picking an approach will allow (some) work to proceed in parallel (especially if the apps are expected to do the combining of the SHow and EPisode program).
Thanks for the consideration.