Page 1 of 1

Trying to scrape one of our product reports w/ JSONP...

Posted: Wed Jun 29, 2016 2:55 pm
by Wolf_22
I'm trying to execute a JSONP AJAX request to scrape data from a report one of our products produces (it's a .NET product in an IIS environment and it lacks effective web output, hence the scraping). The only issue with all this is that I have to use something like JSONP due to the cross-domain circumstance, which is okay but I just can't get it to work.

Here's the script that I'm currently trying to get to work:

Code: Select all

function test(data, status, jqXHR){
    console.log('inside test()');
    console.log('data:'+data);
    console.log('status:'+status);
    console.log('jqXHR:'+jqXHR);
}


//Grab contents from the Argos report...
$.ajax({
    url: "https://mydomain/x?report=DP5BZWOYD5LJU4TMD4YPV2L7EQK53Y2VI7BVCUBXTP3OQGVYIBT7VHHQ6GWCBDCERA53J6G4YBIEM",
    jsonpCallback: test,
    contentType: 'application/json',
    processData: false,
    dataType:"jsonp",
    jsonp: false,
    success: function(data){
        console.log(data);
    },
    error: function(a, b, c){
        console.log(a);
        console.log(b);
        console.log(c);
    }
});
When I execute this, Firebug outputs the following (in order):
inside test()
data:undefined
status:undefined
jqXHR:undefined
Object { readyState=4, status=200, statusText="success", more...}
parsererror
Error: undefined was not called
SyntaxError: missing ; before statement
I don't understand how I'm getting a 200 response with nothing to show for it or the part about "undefined was not called" when the console.log() calls in test() print everything as expected, minus the actual data response I'm needing. It's all just kinda weird.

Can someone here shine some light on this? For whatever it's worth, I posted about this over on Sitepoint, so if someone over there helps me get this working, I'll update this with whatever I find out over there.

Thanks in advance.

Re: Trying to scrape one of our product reports w/ JSONP...

Posted: Wed Jun 29, 2016 6:17 pm
by requinix
"undefined was not called"? Sounds weird. But you told it to call test() with the callback? Or maybe... you didn't? What does the documentation say?
jsonpCallback
Type: String or Function()
Specify the callback function name for a JSONP request. This value will be used instead of the random name automatically generated by jQuery. It is preferable to let jQuery generate a unique name as it'll make it easier to manage the requests and provide callbacks and error handling. You may want to specify the callback when you want to enable better browser caching of GET requests. As of jQuery 1.5, you can also use a function for this setting, in which case the value of jsonpCallback is set to the return value of that function.
That last sentence is particularly important.

Also, I'm not entirely sure you know how JSONP works. So let's get one thing straight first: does the remote server support JSONP for that URL? And yes, it does have to explicitly support JSONP - you can't just make cross-domain calls work without support from the remote server.

Re: Trying to scrape one of our product reports w/ JSONP...

Posted: Wed Jun 29, 2016 6:38 pm
by Wolf_22
Thanks for the feedback, requinix, and you're right: I'm not entirely sure how JSONP works. I've only used it a few times and that was only until I had better options. I never realized that JSONP had to be explicitly supported (configured) on the server. For giggles, how does one verify that anyway? This particular app server is an IIS environment, so how would one verify that on there?

And by-the-way, I was able to get all this working when testing ON the app server (and not from my workstation, which was the different domain). Without a doubt, it's got to be the reason here. Sorry for the confusion.

Re: Trying to scrape one of our product reports w/ JSONP...

Posted: Wed Jun 29, 2016 6:57 pm
by requinix
Verifying, well... You'd have to ask the people who own the site. Or look for documentation. Odds are they don't - JSONP support isn't very common. Or they might support CORS, which you can check for by looking at response headers. That's also uncommon.

The alternative to this is using your website to proxy the information. Like you make a script that does a cURL request to the site and then outputs what the site returns. But you would need to check that their site actually allows for that sort of use, like through a ToS.

By the way, JSONP:

So there's regular JSON AJAX where you make a request and get back a JSON-encoded blob of data. Like

Code: Select all

{"fname": "John", "lname": "Smith"}
Your code can't make calls to another website because of security policies, but their code could make calls to yours. JSONP is about returning Javascript code instead of just data:

Code: Select all

jsonpcallback({"fname":"John","lname":"Smith"})
That will call a function. What function? You tell them, like

Code: Select all

https://www.example.com/remoteservice/user/lookup?callback=jsonpcallback
Then they output the above Javascript, your webpage executes the code, and your jsonpcallback function gets called with the data you want. The key to this is (a) you define a function to receive data and (b) they output Javascript code which calls your function. That's why the remote server has to support it explicitly: the whole scheme depends on their output.

Re: Trying to scrape one of our product reports w/ JSONP...

Posted: Thu Jun 30, 2016 7:26 am
by Wolf_22
requinix wrote:Verifying, well...
Thanks again for the insights, requinix. I think that's what it boils down to: JSONP just isn't enabled on the server. Since it's IIS, I'd like to think that adding it or turning it on wouldn't be that big of a deal but since I have verified that my process works when I upload it to the server, I figure it's something I can disregard for now--it just would've been nice to be able to fully test and develop from my workstation instead of having to RDP into the app server to do testing and development from there. (I have all my tools, etc. installed on my workstation.) I guess I can just do the dependent parts from the app server... I guess all is well. :)

Re: Trying to scrape one of our product reports w/ JSONP...

Posted: Thu Jun 30, 2016 8:17 am
by requinix
Wolf_22 wrote:I think that's what it boils down to: JSONP just isn't enabled on the server. Since it's IIS, I'd like to think that adding it or turning it on wouldn't be that big of a deal
Unless IIS supports something pretty cool to handle this automatically, it's not a server thing. It's a website coding thing.
Like they have code that does whatever JSON stuff - that would need to be changed to support JSONP. Meaning the code would need to be rewritten so that it could do either the normal JSON output or it could do the Javascript code/JSONP output. Which probably wouldn't be that difficult to do, actually.

Re: Trying to scrape one of our product reports w/ JSONP...

Posted: Thu Jun 30, 2016 9:44 am
by Wolf_22
Ah, okay. So it's not a server thing but an actual coding thing. That makes sense. Thanks for the clarification. :)