Documentation??

Discussion about Schedules Direct grabber code and data formats.
johnsonsmythe
Posts: 13
Joined: Thu Sep 06, 2007 2:21 am

Documentation??

Post by johnsonsmythe »

I am playing around with a home recording project and was looking at this data for my personal use ... but where can i see the documentation for using this data as I have a question:

In the Lineup node (for DIRECTV, Zip 90210), there are multiple stations. Now sometimes these stations are duplicated with multiple values:

Code: Select all

<map station='11331' channel='380'/>
<map station='11331' channel='880' from='2006-01-03'/>
Which one of the above should be used?

Code: Select all

<map station='19548' channel='509' from='2000-09-11'/>
<map station='19548' channel='70' from='2004-09-08'/>
Which one of the above should be used?

Basically the duplicates sometimes both have 'from' dates, sometimes they dont have any dates and sometimes only one of them has a date. Which one takes precedence? And why on earth are there duplicates in the first place, cant it just have the current station id/epg node ... seems terribly confusing and superfluous.

Thanking you for your time.

GLCamp
Posts: 16
Joined: Sun Aug 19, 2007 7:27 pm

Excerpt from the old schema

Post by GLCamp »

This is what the Zap2It Labs documentation had ->

xtvd/lineups/lineup/map/@station
Field length: minimum 1, maximum 12 (e.g. 11259)
Station id number. This references the unique station id number in the xtvd/stations element, but a given station can be mapped to multiple channel numbers over multiple lineups.

xtvd/lineups/lineup/map/@channel
Field length: maximum 5 (e.g. 081, 1013, G5-13, C3101)
Physical location of a station or a cable network on a lineup. If this field is blank, it is channel 000.
Note: C-BAND lineup - Digits 1 and 2 are the satellite position, digit 3 is a "-", and digits 4 and 5 are the channel position. 4DTV lineup - Digits 1 and 2 are the satellite position, and digits 3, 4 and 5 are the channel position. For over the air digital channels, this is the Channel Major As described in ATSC Doc. A/65.

xtvd/lineups/lineup/map/@from
Field length: maximum 8 (format YYYY-MM-DD)
Optional. The date from which the station is effective on this lineup.

xtvd/lineups/lineup/map/@to
Field length: maximum 8 (format YYYY-MM-DD)
Optional. The date from which the station has expired on this lineup.


I believe the answer to your question lies in the phrase "a given station can be mapped to multiple channel numbers over multiple lineups."

I don't have DirectTV so I don't know if the same station is on multiple channels or not.
I know that Dish Networks has quite a few stations that appear on multiple channels.

Hope this helps.

johnsonsmythe
Posts: 13
Joined: Thu Sep 06, 2007 2:21 am

Re: Documentation??

Post by johnsonsmythe »

Thanks for the docs except.

I have a couple of issues with what was posted:
* There is no "to" attribute on 'any' station
* The stations appear within the lineup node, so there can only be one for that provider lineup. Of course you can have multiple lineups with the same station on each lineup with a different epg, but thats not the problem ... I have one lineup and the one station appears multiple times with different epg numbers. My example shows the two different cases I have seen in my one lineup.

I think the problem is simple, we are missing the "to" attribute which would tell me (hopefully) which ones to pick.

rmeden
SD Board Member
Posts: 1563
Joined: Tue Aug 14, 2007 2:31 pm
Location: Cedar Hill, TX
Contact:

Re: Documentation??

Post by rmeden »

from/to are optional.. they don't have to appear. (and typically don't)

A <station> appears in <stations>. (not <lineup>)
A <map> appears in a <lineup>. A station can map any number of times in a lineup (including zero)

I'm not clear what you're seeing..

Robert

johnsonsmythe
Posts: 13
Joined: Thu Sep 06, 2007 2:21 am

Re: Documentation??

Post by johnsonsmythe »

My example was very clear, you will see (ok maps) stations in a lineup. These are the stations for that lineup (obviously) and only that lineup complete with their EPG number.

Now, you cant have a station on a lineup more than once, it would be plain daft to have a channel on your cable box accessibly from twp EPG numbers.

Given that, then its clear that if there are more than one 'mapped' station in the lineup node then only one is to be used.

But which one?

I have given two examples, neither of which have a 'to' field so you have no idea which one is the station to use for that lineup.

I am at a loss as to how to explain this any easier I am afraid. Perhaps if you looked at the data file yourself for zip 90210 and DirectTV you will see what i mean.

To help, here is an excerpt of the lineup data:

Code: Select all

<lineups>
   <lineup id='DITV:-' name='DIRECTV' location='USA' type='Satellite' postalCode='90210'>
      <map station='11259' channel='386'/>

      <map station='11331' channel='380'/>
      <map station='11331' channel='880' from='2006-01-03'/>

      <map station='11514' channel='321' from='2003-12-22'/>
      <map station='19521' channel='2175' from='2006-11-28'/>

      <map station='19548' channel='509' from='2000-09-11'/>
      <map station='19548' channel='70' from='2004-09-08'/>

      <map station='19567' channel='81' from='2004-02-03'/>
   </lineup>
</lineups>
As you can see, station 'maps' for '11331' and '19548' appear twice with no TO field so which one to use, they are all valid given the FROM date; but for this lineup (provider) you can obviously only see the station once (as can be confirmed at http://www.directv.com/DTVAPP/epg/theGuide.jsp)

rmeden
SD Board Member
Posts: 1563
Joined: Tue Aug 14, 2007 2:31 pm
Location: Cedar Hill, TX
Contact:

Re: Documentation??

Post by rmeden »

<map station='11331' channel='380'/>
<map station='11331' channel='880' from='2006-01-03'/>
I interpret this to mean that since 2006-01-03, station 11331 is available on both channels 380 and 880.

Is that not correct?

It's not unusual for a station to be more than once on a lineup, but generally not in that numeric range (typically analog + digital cable)

Looking at the DirectTV link, channel 880 didn't seem to be listed. If it really doesn't exist (please try it), then it's a lineup issue we can pass to TMS.

Robert

johnsonsmythe
Posts: 13
Joined: Thu Sep 06, 2007 2:21 am

Re: Documentation??

Post by johnsonsmythe »

Looks wrong to me, looks like it should have a "to" date on it. Also the HBOHD looks more wrong. Can you pass to TMS please to check.

the_tom
Posts: 15
Joined: Mon Sep 10, 2007 4:19 pm

Re: Documentation??

Post by the_tom »

johnsonsmythe wrote:Looks wrong to me, looks like it should have a "to" date on it. Also the HBOHD looks more wrong. Can you pass to TMS please to check.
I read through this thread and I felt concerned about what seemed to me to be a certain "attitude" expressed {"My example was very clear" in response to a post that said he was not clear on your issue; "I have a couple of issues with what was posted" when what was posted was just the doc that was requested}, and the extent to which this discussion exercised the volunteer board member over a desire for help in understanding how to use the data.

Then I looked at some of your other posts and it seems pretty clear that you are hoping to find some way to make money on this. If you stay strictly within the Terms, I guess that's your business. Maybe pure donate-if-you-wish is OK, although with so much sw out there your stuff would have to be awfully good, and the donation stream probably awfully dilute. But if you stray outside of the Terms, it will become everybody's concern. For instance if someone's approach is "donate and I'll turn on these advanced features" they are out of bounds - that's a commercial application.

Robert told you what you need to do if you think there's an error in your lineup - pick up your Directv remote and enter 880; if that's not the channel the download say it is, it's evidently an error and that can/should be reported to TMS. And doing that would also give a key piece of additional info to help understand how to use the available TMS data. Just a wild guess here - you're probably not a DirecTV subscriber living in Beverly Hills, so you can't do that, right?

As the doc says, and the schema says, and Robert says: the "to" date is optional. Your above post seems to indicate that you want SD and TMS to research a question on a lineup that is (presumably) not actually available in your home (otherwise you could check it yourself in less time than posting it), and also to change their systems and databases to fill in an optional data item, expecting that would make it easier to figure out what to do with the data. I for one would not ask them to do those things, nor expect them to do them if asked.

I have tried to give this somewhat negative feedback without getting personal about it, and certainly without intending to - I'm telling about how I react to the words posted, not expressing any kind of judgment of you as a person. If I have mis-read your words, I apologize in advance. I certainly do not mean to suggest that you should not write your own software for your own home use and/or for free distribution. I would suggest that starting with your own cable or satellite system would make it a lot easier to work out for yourself how to make sense of the data that is available. That's what I did; and by eyeballing the data in the debugger, I was able to write my own sw quite quickly and without needing to ask anyone about the meaning of any of the data; and I've used it for over a year with minimal problems. Anyway the field is so large and dynamic, there will always be some errors and omissions in the data - sw has to deal with that.

If you suffered through the above, maybe this will be helpful. What's the answer to "which one do I use"? Both (all), if they are listed without "to" dates. I'm guessing that you want to "denormalize" the data and store the program data combined with a channel number in one record. As the tms data structures make clear, that's not a good representation of the data: it's more complicated than that. Even if these entries in the 90210 lineup are wrong, having multiple channels per station is not in general wrong - it is not just possible, it is common. So I would suggest keeping the channels/stations mapping in one table, and storing the station id, not the channel number, in the program data. To access program data by channel, read the channels/stations mapping table by channel number and then read the program data by station id mapped for that channel. That is a simple database join operation - any database can do it quickly and efficiently. And if a channel that shows up in the download is not really on the user's (um, your) system, you can just de-select it in the web page for the lineup - problem goes away on the next download!

johnsonsmythe
Posts: 13
Joined: Thu Sep 06, 2007 2:21 am

Re: Documentation??

Post by johnsonsmythe »

Thanks for your input (and this isnt a personal reply) but it looks like you havent read or understood the problem either. Perhaps if someone who comments could actually look at the source data rather than pontificating about it that would help.

I can confirm the channel shouldnt exist on my box, the channel shouldnt be in the lineup but is, its a TMS error, but as I am paying Schedules Direct and not TMS then I am reporting it to Schedules Direct. Its fairly clear where the error is, whether the document says its optional or not, means it should be there sometimes, its is *never* there so it looks like a simple bug.

Whether I make money or not (which I am clearly not otherwise I wouldnt have wanted clarification on it in an earlier post, to clear this up I am NOT distributing something that I am using personally and therefore legitimately as part of the SD Agreement) then that is completely irrelevant to this post.

the_tom
Posts: 15
Joined: Mon Sep 10, 2007 4:19 pm

Re: Documentation??

Post by the_tom »

Reference:
rmeden wrote:... If it really doesn't exist (please try it), then it's a lineup issue we can pass to TMS.
Robert
johnsonsmythe wrote:Thanks for your input (and this isnt a personal reply) but it looks like you havent read or understood the problem either. Perhaps if someone who comments could actually look at the source data rather than pontificating about it that would help.

I can confirm the channel shouldnt exist on my box, the channel shouldnt be in the lineup but is, its a TMS error, but as I am paying Schedules Direct and not TMS then I am reporting it to Schedules Direct.
...

Whether I make money or not (which I am clearly not otherwise I wouldn't have wanted clarification on it in an earlier post, to clear this up I am NOT distributing something that I am using personally and therefore legitimately as part of the SD Agreement) then that is completely irrelevant to this post.
Again, I apologize for apparently misunderstanding your intentions; and I appreciate the civility of you reply.

But I did read your data. I did understand it. I'll agree I did not understand the problem, but IMO that's only because until your last post you didn't respond to Robert's question about whether the channel was real or not. OK, so the actual problem is that there's a channel in your lineup that isn't real. That is a data problem, in the TMS data, and I quite agree you should report it to SD and not directly to TMS (I see that I seem to have suggested that, but that's not what I intended).
johnsonsmythe wrote:...Its fairly clear where the error is, whether the document says its optional or not, means it should be there sometimes, its is *never* there so it looks like a simple bug.
..
I clearly understand, and I understood earlier, that you are defining the problem as being, there is never a value in the "to" date in your channel lineup, and that's a bug in the software or an error in the data and it should be fixed. I was just disagreeing with you on that. This is a sprawling database and the data comes from a wide variety of feeds. I expect that to mean that "optional" data items are going to be found on an "as available" or "as economical to collect" basis. The schema is a "contract" with software with respect to document structure, but not a contract with the consumer to fill in all the fields.

But I guess I was not being entirely objective. I checked - there are no "to" dates in my lineup either. I don't care - I wouldn't use it anyway; but I have to allow that AFAIK there really could be a simple bug somewhere, where they have the data but don't emit it into the output. I would, and do, support the suggestion that the server code should be checked to see if there is such a bug.

But if it's just that the data is sparse/absent, I would expect TMS to laugh at any suggestion that someone should go track down those "to" dates and put them in the database so that the data makes more sense to someone.

In any case, you really should be able to proceed with your app - especially as it's really for your personal use, just go on the web page and deselect any channel that isn't real!

-tom-

PS. I'm working hard to try to be clear and thorough. Sometimes that may make me a bit pedantic. I find the term "pontificating" quite offensive. But as I said, I do appreciate the civility of your reply.

Post Reply