Developer Wishlish for new JSON API

rkulagow
SD Staff
Posts: 915
Joined: Tue Aug 14, 2007 3:15 pm

Re: Developer Wishlish for new JSON API

Post by rkulagow » Mon Feb 25, 2013 6:24 am

I have no intent on changing the API at this time. I will take a second look at REST when I have some spare cycles. For the time being, HTTP response codes will remain as they are.

rkulagow
SD Staff
Posts: 915
Joined: Tue Aug 14, 2007 3:15 pm

Re: Developer Wishlish for new JSON API

Post by rkulagow » Mon Feb 25, 2013 9:03 am

hall5714 wrote:Posted this on the other thread, so I'll summarize here:
(4) Sending zips should be consistent
If our command is successful, it throws a zip file at as, if it's not, it throws a json response string at us. Instead of sending a zip, it should send a json string with the "data" object being the zip bytes, or a temp "link" to grab the data. The temp link is nice because it only has to be active ~15 minutes, so if the client has some sort of error during transmission, we can download it again without the server having to rebuild the zip file (although you technically may be doing that server-side anyway).
I've changed the "get" "metadata" response on the test server:

{"randhash":"60610a5d8d41de88450a4196a0f7b9dd","object":"metadata","action":"get","api":20130224}
Response from server:

{"response":"OK","code":200,"serverID":"AWS-micro.1","message":"file available at URL.","URL":"https:\/\/s3.amazonaws.com\/schedulesdirect\/8f09d0c474199449ca2397fdb30b0d5c.zip","datetime":"2013-02-25T15:56:46Z"}

I've added a "URL" parameter; if you get the URL you will get the metadata for today. The filename is specifically non-deducible. While we're still testing things, the file will auto-purge in a day (since metadata would be generated once a day). This may be tweaked in the future.

Should I add another parameter to the response, like "filename" so that the client will know that this is the "1361807245.metadata.json.txt" file? It may be useful to include the filename if I move more things to S3.

Let me know if "message" needs tweaking or if there's something better that you'd like to see in that field. (Or if you want it at all if it's a "OK")

hall5714
Posts: 20
Joined: Thu Feb 07, 2013 4:34 pm

Re: Developer Wishlish for new JSON API

Post by hall5714 » Mon Feb 25, 2013 9:12 am

Slugger wrote:
I don't think much of any of this looks like a RESTful service. If it did, I'd be inclined to agree with you. However, this whole service works on sending requests to request.php and getting back responses. And request.php (the lone resource) always exists so unless the Apache server is ill, Apache should always return status 200. Now if the requests were changed to be RESTful then it would make sense to have the http status code reflect the status of the reqeusted resource.

With that said, it's, of course, Robert's call. Making this change will break the Java client.
Well, I didn't mean to imply only REST services use it:

http://www.jsonrpc.org/historical/json- ... onse-codes

Bitcoin is a great example of this implementation. And I guess we have to agree to disagree about request.php being the lone resource... but IMHO, by definition this API requests a resource, and since that resource is in the HTTP chain, there's no reason not to give it an HTTP response.
rkulagow wrote:I have no intent on changing the API at this time. I will take a second look at REST when I have some spare cycles. For the time being, HTTP response codes will remain as they are.
No problem. If I ever make a recommendation, the "if you have time to look at it" is forever implied. If/when you do have time REST or JSON-RPC are both worthy of taking a look at. If for no other reason than the mountain of existing implementations for both. Many PHP implementations are abandoned, but JSON-RPC isn't exactly a "changing" standard. From a server-side standpoint, I think JSON-RPC is easier to implement, since you're effectively just exposing a bunch of methods in a class.

I don't know if a massive context switch would work at this point, but I think long-term JSON-RPC, XML-RPC or REST are going to be FAR more manageable. Offloading the work of properly encoding/decoding/error responses etc to some library is always nice :).

hall5714
Posts: 20
Joined: Thu Feb 07, 2013 4:34 pm

Re: Developer Wishlish for new JSON API

Post by hall5714 » Mon Feb 25, 2013 9:15 am

rkulagow wrote:
hall5714 wrote:Posted this on the other thread, so I'll summarize here:
I've changed the "get" "metadata" response on the test server:

{"randhash":"60610a5d8d41de88450a4196a0f7b9dd","object":"metadata","action":"get","api":20130224}
Response from server:

{"response":"OK","code":200,"serverID":"AWS-micro.1","message":"file available at URL.","URL":"https:\/\/s3.amazonaws.com\/schedulesdirect\/8f09d0c474199449ca2397fdb30b0d5c.zip","datetime":"2013-02-25T15:56:46Z"}

I've added a "URL" parameter; if you get the URL you will get the metadata for today. The filename is specifically non-deducible. While we're still testing things, the file will auto-purge in a day (since metadata would be generated once a day). This may be tweaked in the future.

Should I add another parameter to the response, like "filename" so that the client will know that this is the "1361807245.metadata.json.txt" file? It may be useful to include the filename if I move more things to S3.

Let me know if "message" needs tweaking or if there's something better that you'd like to see in that field. (Or if you want it at all if it's a "OK")
Either a filename in the HTTP headers or in the response would work well, especially if this is the end-game route since for headends, for example, the file name is somewhat important :).

rkulagow
SD Staff
Posts: 915
Joined: Tue Aug 14, 2007 3:15 pm

Re: Developer Wishlish for new JSON API

Post by rkulagow » Mon Feb 25, 2013 9:24 am

{"response":"OK","code":200,"serverID":"AWS-micro.1","message":"file available at URL.","filename":"1361807245.metadata.json.txt.zip","URL":"https:\/\/s3.amazonaws.com\/schedulesdirect\/8f09d0c474199449ca2397fdb30b0d5c.zip","datetime":"2013-02-25T16:23:07Z"}

"filename" is now included.

rkulagow
SD Staff
Posts: 915
Joined: Tue Aug 14, 2007 3:15 pm

Re: Developer Wishlish for new JSON API

Post by rkulagow » Mon Feb 25, 2013 4:25 pm

I will also consider adding "expires" if that would be useful. However, I would like to lock down the API and move it to production shortly.

Slugger
Posts: 77
Joined: Sun Sep 18, 2011 1:22 pm

Re: Developer Wishlish for new JSON API

Post by Slugger » Mon Feb 25, 2013 4:52 pm

Will you be moving the other zip file response to the same process (i.e. get a link to s3 via json response then go download the zip from s3)?

rkulagow
SD Staff
Posts: 915
Joined: Tue Aug 14, 2007 3:15 pm

Re: Developer Wishlish for new JSON API

Post by rkulagow » Mon Feb 25, 2013 5:01 pm

Slugger wrote:Will you be moving the other zip file response to the same process (i.e. get a link to s3 via json response then go download the zip from s3)?
Now that I've figured out how to do it with the metadata, I can probably do it for everything. Would that be better? The API version thing isn't hard-and-fast; as long as external clients are using it yet, then it's just a "number".

Obviously I'd like it to be "done" at some point so that I can finish the new hopefully robust data import routines.

It would probably take another day or so to get everything converted over to S3.

hall5714
Posts: 20
Joined: Thu Feb 07, 2013 4:34 pm

Re: Developer Wishlish for new JSON API

Post by hall5714 » Mon Feb 25, 2013 5:31 pm

rkulagow wrote: Now that I've figured out how to do it with the metadata, I can probably do it for everything. Would that be better? The API version thing isn't hard-and-fast; as long as external clients are using it yet, then it's just a "number".

Obviously I'd like it to be "done" at some point so that I can finish the new hopefully robust data import routines.

It would probably take another day or so to get everything converted over to S3.
I think that would be ideal... it makes error checking easier, basically if we don't get json, something is broke.

Slugger
Posts: 77
Joined: Sun Sep 18, 2011 1:22 pm

Re: Developer Wishlish for new JSON API

Post by Slugger » Mon Feb 25, 2013 5:46 pm

hall5714 wrote:
rkulagow wrote: Now that I've figured out how to do it with the metadata, I can probably do it for everything. Would that be better? The API version thing isn't hard-and-fast; as long as external clients are using it yet, then it's just a "number".

Obviously I'd like it to be "done" at some point so that I can finish the new hopefully robust data import routines.

It would probably take another day or so to get everything converted over to S3.
I think that would be ideal... it makes error checking easier, basically if we don't get json, something is broke.
Yeah, I agree. Getting a json response to every request is more consistent. Removes a lot of branching in the client code.

Post Reply