tv_grab_zz_sdjson not including new episode info?

Use this forum to discuss issues related to the SD-JSON service.

tv_grab_zz_sdjson not including new episode info?

Postby rykr » Fri Sep 29, 2017 1:31 pm

Runniing tvheadend and using the tv_grab_zz_sdjson grabber with my sd account. Was getting unexpected results on new episodes so ran the grabber myself to get the data and I see zero info on a show that I know to be new to indicate that it is new.

I'm wondering what I'm missing. Is this a problem with the grabber?
rykr
 
Posts: 3
Joined: Sat Jun 10, 2017 6:21 am

Re: tv_grab_zz_sdjson not including new episode info?

Postby rmeden » Sat Sep 30, 2017 10:40 pm

XMLTV has a <previously shown> element.

From: notes on "new" vs "repeat"

tv_grab_zz_sdjson may do something similar to tv_grab_na_dd

Old versions of TV_GRAB_NA_DD simply created an empty previously-shown tag to flag a repeat. Now we place "original-air-date" as a start attribute. <previously-shown start=original-air-date>. Applications are encouraged to use previously-shown/start to augment repeat detection. In TV_CHECK, I consider "new" any episode less than 2 weeks old.
rmeden
SD Board Member
 
Posts: 1486
Joined: Tue Aug 14, 2007 2:31 pm
Location: Cedar Hill, TX

Re: tv_grab_zz_sdjson not including new episode info?

Postby rykr » Mon Oct 02, 2017 8:09 am

rmeden wrote:XMLTV has a <previously shown> element.

From: notes on "new" vs "repeat"

tv_grab_zz_sdjson may do something similar to tv_grab_na_dd

Old versions of TV_GRAB_NA_DD simply created an empty previously-shown tag to flag a repeat. Now we place "original-air-date" as a start attribute. <previously-shown start=original-air-date>. Applications are encouraged to use previously-shown/start to augment repeat detection. In TV_CHECK, I consider "new" any episode less than 2 weeks old.


Thanks for the reply. It seems tv_grab_zz_sdjson is not doing that. :(
rykr
 
Posts: 3
Joined: Sat Jun 10, 2017 6:21 am

Re: tv_grab_zz_sdjson not including new episode info?

Postby rpcameron » Mon Oct 02, 2017 1:46 pm

If I remember correctly, the problem is Tvheadend does not treat the program as new if there is a previously-shown element. Also, because of the way that dates/times are ambiguously defined in the XMLTV DTD, sometimes you may end up with a previously-shown date that occurs after the program's air date.

This is compounded by XMLTV treating the "new" element as meaning "this program has been aired on a different channel, and this is the first time it is being aired on this channel". However, Tvheadend treats the new element as one might expect: this is the first broadcast of this program on this channel.

Over at the Tvheadend forums I've posted patches for both tv_grab_zz_sdjson and tv_grab_zz_sdjson_sqlite that fixes these issues. What I've done is suppressed the generation of the previously-shown element if that airing of the program is marked "new" in th schedule.

I also filed a bug report about this with the sqlite version of the grabber, but the author has indicated that he is not going to change it because he wishes to closely follow the XMLTV DTD. While I understand his perspective, it makes it difficult to use with Tvheadend.

The forum posting with my patches is here: tv_grab_zz_sdjson patches. I also have an updated patch for the current version of the sqlite grabber that I have yet to post on that thread, but will do so later today.
rpcameron
 
Posts: 3
Joined: Mon Oct 02, 2017 1:19 pm

Re: tv_grab_zz_sdjson not including new episode info?

Postby rpcameron » Tue Oct 03, 2017 10:24 am

Here are my tv_grab_zz_sdjson and tv_grab_zz_sdjson_sqlite patches. The first is for the version of the grabber that is included with the xmltv-utils package. The second is for the current (21 July 2017, v1.33) version from the author's GitHub repo.

tv_grab_zz_sdjson.patch:
Code: Select all
--- a/tv_grab_zz_sdjson   2017-02-03 15:19:59.000000000 -0800
+++ b/tv_grab_zz_sdjson   2017-05-16 11:55:37.426905237 -0700
@@ -1339,9 +1339,14 @@
 # The xmltv docs state this field is "When and where the programme was last shown".
 # However mythtv expects the original air date to be in this field.
 sub get_program_previously_shown {
-   my ($details) = @_;
+   my ($program, $details) = @_;
    my %previously_shown;
 
+   my $new = $program->{'new'};
+   if($new) {
+      return undef;
+   }
+
    my $date = $details->{'originalAirDate'};
    if($date) {
       my $dt = parse_original_airdate($date);
@@ -1457,7 +1462,7 @@
          'episode-num'      => get_program_episode($program, $details),
          'video'            => get_program_video($program),
          'audio'            => get_program_audio($program),
-         'previously-shown' => get_program_previously_shown($details),
+         'previously-shown' => get_program_previously_shown($program, $details),
          'premiere'         => get_program_premiere($program),
 #         'last-chance'      => undef,
          'new'              => get_program_new($program),


tv_grab_zz_sdjson_sqlite.patch:
Code: Select all
--- a/tv_grab_zz_sdjson_sqlite   2017-10-03 09:57:08.576347223 -0700
+++ b/tv_grab_zz_sdjson_sqlite   2017-09-22 13:22:00.512171274 -0700
@@ -1777,7 +1777,13 @@
 
         # XMLTV uses their standardized dates, while Schedules
         # Direct uses YYYY-MM-DD
-        if (defined($programDetails->{'originalAirDate'}))
+        # If "new", then do not generate previously-shown
+        if (defined($scheduleDetails->{'new'}))
+          {
+            my $new = $scheduleDetails->{'new'};
+            $w->emptyTag('new');
+          }
+        elsif (defined($programDetails->{'originalAirDate'}))
           {
             my $originalAirDate = $programDetails->{'originalAirDate'};
             my $offset = ' +0000';
Attachments
tv_grab_zz_sdjson.txt
tv_grab_zz_sdjson.patch
(1.04 KiB) Downloaded 9 times
tv_grab_zz_sdjson_sqlite.txt
tv_grab_zz_sdjson_sqlite.patch
(731 Bytes) Downloaded 10 times
rpcameron
 
Posts: 3
Joined: Mon Oct 02, 2017 1:19 pm

Re: tv_grab_zz_sdjson not including new episode info?

Postby rkulagow » Sat Oct 07, 2017 8:23 pm

You may want to post XMLTV-specific patches / code / discussion on the various XMLTV lists.

https://sourceforge.net/p/xmltv/mailman ... nth=201710

Would be a good starting point.
rkulagow
SD Staff
 
Posts: 884
Joined: Tue Aug 14, 2007 3:15 pm

Re: tv_grab_zz_sdjson not including new episode info?

Postby rpcameron » Sun Oct 08, 2017 6:56 am

rkulagow wrote:You may want to post XMLTV-specific patches / code / discussion on the various XMLTV lists.

https://sourceforge.net/p/xmltv/mailman ... nth=201710

Would be a good starting point.

The problem is that there isn't a good 1:1 mapping of of the data to XMLTV elements; also, the use of the elements that seem to match up is different in some cases. The problem isn't necessarily with the XMLTV, but that presently it is the only way to get SD data into Tvheadend, and in doing so things get "lost in translation".

I've posted them on the Tvheadend forum where they are more relevant, as the grabber author did not feel this was a real issue as it is slightly contrary to the XMLTV spec.
rpcameron
 
Posts: 3
Joined: Mon Oct 02, 2017 1:19 pm


Return to SD-JSON service

Who is online

Users browsing this forum: No registered users and 1 guest

cron