Accuracy of EPisode program guide data

Accuracy of EPisode program guide data

Postby gtb » Tue Jun 28, 2016 9:19 pm

This is intended to start a discussion about the guide data available in EPisodes vs generic SHows of the same series.

There is a "bunch" of useful data available only in the generic SHow program that an EPisode program is part of. A schedule only provides a pointer to the one program, and the program hash, and not to the additional generic SHow program (nor its hash).

Items found (in some programs) which have been found to only be in the SHow and not the EPisode include actors roles, and total seasons. I have not done a more extensive check. But those are enough.

So, for something to amass the full information for an EPisode program, it is necessary for obtain all related unique program SHows that one has an EPisode for, and wants to process. As there is currently no hash for the (generic) SHow program provided, that means either requesting every SHow every time, or producing inaccurate data because the data is old and obsolete (how old or obsolete cannot be know until you fetch it) or presenting data which is not as complete as one can should be able to present.

In one test lineup, that is 50,000 SHows that need to be fetched each and every time the guide data is processed (just to give one an order of magnitude of the potential impact).

It is clearly desirable that apps be able to provide the best data available all the time.

As I understand it, the intent of the JSON api is to allow apps to use the hashing to minimize to load on the SD servers. To continue to do so, some changes are needed.

Here are a couple of possible solutions (that would not require an JSON API level change).
(sorry for the code formatting, but trying to use bbcode list is just not worth it at the moment)

Code: Select all
 * At original data load time at SD, build any EPisode program
   entries with data from any generic SHow information, and
   with the EPisode specific data presented in the schedule
   reference.  As is currently, one schedule entry points to
   one program (with a hash).

  - This means you get one (and only one) program
    to deal with in the schedule, and makes the apps
    easier to write.

  - This (potentially) means a lot more preprocessing by SD

 * Provide in the schedule data an additional (generic)
   SHow program id and hash   (in addition to the EPisode
   program id and hash) [obviously the id can be calculated,
   but it might be more flexible to provide it in case the calculation
   might change with future data formats].  If the field does not
   exist, there is no generic info available.

  - This means apps need to (optionally) pull in additional
    program entries, and do (essentially)  what proposal 1
    would do, except every app gets to do it.

  - This allows apps to cache the SHows correctly because
    there are hashes available to validate.

 * Provide in the EPisode program an additional "generic" field
   for the SHow and hash (essentially the previous reference is
   moved from the schedule to the program elements)

  - This means the apps need to (optional) pull in additional
    program entries, and do (essentially) what proposal 1 would
   do, except every app has to do it.

  - This allows apps to cache the SHows correctly because
    there are hashes available to validate

 * Provide a new API call to return the hash(es) of a provided
   list of programs.

  - The app must calculate the SHow ids of interest (not a
    great burden).

  -  This allows an app to validate the local hash for program
     SHows that are downloaded for caching.

  - The app will still need to do combining of the SHow data
    with the EPisode data and do what proposal 1 would do,
    except that every app has to do it.

  - it seems this would be easy for SD to do?

 * Do nothing (on the SD end), and apps can pull down all
   needed (50,000?) generic SHow programs every guide refresh.

  - This seems to go against the intent of a "cache if you can"
    approach.

  - This will result in a lot more load on the SD server

  - The app will still need to do combining of the SHow data
    with the EPisode data and do what proposal 1 would do,
    except that every app has to do it.

 * Do nothing (on the SD end) and apps can decide to cache
   SHow data which means some of the time the data will be
   wrong.

  - Presumably apps will still need to randomly invalidate
    the cache and generate the load(s).

  - The app will still need to do combining of the SHow data
    with the EPisode data and do what proposal 1 would do,
    except that every app has to do it.

 * Do nothing (on the SD end) and apps can ignore the issue,
   and not provide full guide info to their users.

  - Which is where some apps likely are today.  It is where
    my app is, as I was not aware that (for example) actor roles
    were only in the SHow, not reliably presented in the EPisode
    (sometimes they are in the EPisode program, and sometimes
    not.  I only checked a few during testing.  Apparently I chose
    badly)



No matter the choice, I would request to better document the solution/issue in the JSON api docs.

Obviously as a consumer of the data, the first approach is appealing to me (minimal to no changes on "my" end). But except for the last three (the do nothing on the SD end) any would be a solution I can live with as far as I am concerned. The only disadvantage for the second to fifth approachs is that each app has to do the programming to do the combining of the data, rather than SD doing it for everyone. But the counter argument is that some apps will not combine at all, and that each app might choose to combine differently (which likely has other issues, but ....)

So, does the issue make sense, or do I need to try to re-explain it?

And if it makes sense, comments, alternatives?


I recognize that this is going to be a slightly longer term discussion, and any resolution will take time. But picking an approach will allow (some) work to proceed in parallel (especially if the apps are expected to do the combining of the SHow and EPisode program).

Thanks for the consideration.
gtb
 
Posts: 50
Joined: Thu Oct 02, 2014 2:07 pm

Return to General Discussions

Who is online

Users browsing this forum: No registered users and 1 guest

cron