This blog post is targeted at beginners to JavaScript, NodeJS, and HTTP requests.
NodeJS is wonderful for writing scripts that take advantage of JSON API endpoints. Because the payloads both to and from your script are already in the format of its native language. NodeJS is also awesome at writing its own API endpoints as well, but that article is for another time. More reading on that for the curious.
NodeJS lets you write JavaScript and run it without any need for a browser. (like reading and writing to a file).
JSON is a way to package up a bunch of JavaScript data (It literally stands for JavaScript Object Notation). So I could send this to a site that supported it:
{ "name" : "Jim" }
Which simply put, is just an object with a name key, and “Jim” as its value (read like name=Jim), the curly brackets mark the beginning and end of an object, the colon separates they key from its value. For fun, paste that code snippet into JSONBlob.com on the left side. On the right side you can edit the object by clicking the little box and play around with the data and see how the JSON is actually affected.
An API endpoint is a place that can accept that kind of data. So you could send that data at: http://www.example.com/add-users. The code that runs on that server will listen for requests to its endpoint (/add-users), parse the data in the request (the name=Jim object), and most likely store it somewhere like in a database. In addition to sending, many APIs have GET endpoints like /get-users/1 which might be documented like /[entity(s)]/[id]. NOTE: most endpoints DO NOT have the name of the HTTP method in them. So generally, a POST to /users will probably add a Jim as a new user, an HTTP GET request on /users/1 will get a user with ID of 1.
Time to get a little more complex…
So how do we actually send data to an endpoint?
Like many other languages, Node has a bunch of built-in libraries (node calls these “modules”) to help make common tasks easier. In fact, Node excels strongly in the HTTP department. It has a server, a client, ways to parse messages, etc. Many of these can be very verbose however, or lacking in features such. node-fetch has some cool features which allow you to substitute its promise library, or decode modern web encoding like gzip/deflate, it has .json() (explained later). So different people have built different modules that wrap the built-in ones and make tasks easier. Enter, node-fetch.
node-fetch is one of my favorite libraries for doing server-side HTTP requests. The main reason being is that it works almost exactly like window.fetch. window.fetch is the new and improved way to fetch resources. This website shows you current browser adoption of window.fetch.
Making a request for JSON.
Building a bare-bones module:
To create our sample project, we are going to create a new directory called node-fetch-example initialize it as a module, create our entry point (index.js), and then download node-fetch using the node package manager. When running npm init, just hit enter for every single thing, none of that is important for now.
mkdir node-fetch-example
touch index.js
cd node-fetch-example
npm init
npm install node-fetch --save
Now open index.js with your favorite editor. Insode we need to include all the functions provided for us by node-fetch. To do this, we require the module.
var fetch = require('node-fetch');
Now, require goes and gets the module and stores whatever it chooses to export into your new variable called “fetch”. Now we need an endpoint; The people at jsonplaceholder.typicode.com have graciously hosted a test API for education purposes just as this. Here are a few they have listed on their site:
| Endpoint |
| /posts |
| /posts/1 |
| /posts/1/comments |
For this example, we will use /posts/1 since the data in the body of the response will be smaller.
Sending the request:
Simply call fetch.
fetch('http://jsonplaceholder.typicode.com/posts/1')
At this point, we can run our application and see the request fly live. To run your new module, all you have to do is call node and show it where your script is.
node index.js
Now lets watch what happens:
Here I am running a big long Linux command that simply shows me all GET requests out to port 80. You can see my GET to /posts/1 over HTTP version 1.1!
Wheres my data?
node-fetch uses promises. This basically means it’s saying: “I am going to go out and get this data for you, but I am not sure when ill be back. It could be 5 minutes, it could be 1 millisecond”. I am sure at this point you can see the implications of this. This means that if you want your script to keep going, you would have to run this on its own thread (asynchronous), OR if you want your script to wait for it to complete (most likely scenario) you might get stuck nesting a bunch of functions. So we have been given a chain-able function called “then()”. Lets see how it works. In your code, replace the fetch code with this addition:
fetch('http://jsonplaceholder.typicode.com/posts/1')
.then(function(response){
return response.json();
})
Notice I call .then() against the Promise that is returned by fetch(). .then() will take a function which the promise will call when it resolves and then it will pass that the value resolved by the promise to it. So in simple speak when fetch is done, it sends whatever it got to your function, so you need a variable inside said function to get to it.
The other new thing here is I am calling .json() on the response. This is important because the response is jam packed full of a TON of stuff. Everything that is related to the actual HTTP call is in there. Like headers, options, ect. So .json() is a convince function provided to us by the node-fetch module that will hunt through the payload sent back to us, and convert it nicely to JSON and return it. The one catch here is, it also returns a promise. So how do we handle that? First we return the promise that .json() returns and we chain on another .then();
fetch('http://jsonplaceholder.typicode.com/posts/1')
.then(function(response){
return response.json();
})
.then(function(json){
console.log(json);
});
Pretty obvious at this point, since .json() returned a promise, it promised to call the next .then(function) that it found, and shove the resolved value into its first parameter.
If all goes well, you get this:
Happy coding!