Node for Web
refer from jscomplete.com, author Samer Buna
The Basic Streaming HTTP Server
HTTP is a first class citizen in Node. In fact, node started as a web server and evolved into the much more generalized framework it is today. Node’s HTTP module is designed with streaming and low latency in mind. Node is today a very popular tool to create and run web servers. Let’s look at the typical hello-world example for Node’s http module. We create a server using the HTTP module create server method, which gives us, no surprise there, an event emitter object. That event emitter has many events, one of which is the request event, and this event happens every time a client connects to this http server. It exposes a request object and a response object. We can use the response object to modify the response Node will be using for that request. In here, we respond with 200 ok, content-type text, and the text "hello world." The server runs on port 8000. When we run this script, node will not exit, because it has a listener handler and it will respond to any http request on port 8000. If we inspect the headers this simple http server is sending, we’ll see http 1. 1, this is the current most recent version of http that’s supported in browsers. Response code is 200 ok, and the content-type is what we set it to, and we have a connection keep-alive and transfer-encoding is chunked. Keep-alive means that the connection to the web server will be persisted. The tcp connection will not be killed after a requester receives a response, so that they can send multiple requests on the same connection. Transfer-encoding chunked is used to send a variable length response text. It basically means that the response is being streamed! Node is ok with us sending partial chunked responses, because the response object is a writable stream. There is no response length value being sent. After Node sends the 200 ok here, it can do many other things before terminating the response, and instead of inefficiently buffering everything it wants to write in memory and then write it at once, it can just stream parts of the response as they’re ready. So this very simple http server can be used, for example, to stream video files out of the box. But remember that the connection is not terminated here, so the browser knows that the content is done through the http protocol. Http 1.1 has a way of terminating a message, and it’s what happens when we use the response. end function. So if we don’t actually use the end method, if we use the write method instead, in this case, when a client connects, they get the “hello world, “ but the response is not terminated, because basically Node is still streaming. In fact, in here, we can define a timer function. Let’s make that fire after one second. And inside this function, we’ll go ahead and write another message. And how about we write yet another message after two seconds? Let’s try it. Run the server. Connect. And you’ll see “hello world. “ After one second you’ll see another message, and after two seconds you’ll see the third message. And the server is still streaming. We have not instructed http to terminate the response object. Let’s make the timeout periods here a little bit longer. Let’s do 10 seconds and 20 seconds and test that. Initiate an http request. I see the "hello world." Node is not really sleeping, it’s just idling, and it can actually handle other client requests during this idling phase, thanks to the event loop. Both of these requests are being handled concurrently with the same node process. Please note, however, that terminating the response object with a call to the end method is not optional, you have to do it for every request. If you don’t, the request will timeout after the default timeout period, which is set to two minutes. As you can see, this right line did not happen, because it’s delayed till after the default server timeout. We can actually control the server timeout using the timeout property, so this line will make the timeout one second, and you’ll see now how the server is going to timeout right away.
const server = require('http').createServer();
server.on('request', (req, res) => {
res.writeHead(200, { 'content-type': 'text/plain' });
res.write('Hello world\n');
setTimeout(function () {
res.write('Another Hello world\n');
}, 10000);
setTimeout(function () {
res.write('Yet Another Hello world\n');
}, 20000);
});
server.listen(8000);
Working with HTTPS
HTTPS is the HTTP protocol over TLS/SSL. Node has a separate module to work with HTTPS, but it’s very similar to the HTTP module. To convert the basic HTTP server example to work with HTTPS, all we need to do is require https instead and provide the createServer method with an options object. This object can be used to configure multiple things. Usually we can pass key and certificate. Those can be buffers or strings to represent a private key and the certificate. We can also pass a pfx option to combine them if you want. I’ll just use key insert and let’s walk through a complete example. We first need to generate a certificate. We can use the openssl toolkit for that, which will allow us to first generate a private key. We can have it encrypted or not encrypted. And it will allow us to generate a certificate signing request (CSR), and then self-sign this certificate to test it. Of course, the browsers will not trust our self-signed certificate, but it’s good for testing purposes. We can actually combine all these steps with one command that will output a private key and a certificate file for us. It will ask for some information, but since it’s all just a test, you can use the default answers here. When this command is done, you should see a key.pem file, and a cert.pem file in the working directory. Now that we have these files, we just need to pass them here to the create server method. We can use the fs module to do that. We’ll do readFileSync. Since these files are to be read once, and used for creating a server, this is okay here. We’ll add key:pem file, and same thing for the cert: pem, and of course we need to require the fs module, and the last thing we need to do is to change this port to 443, which is the default port for https communication. To test all that, we need to execute the script. In my environment I need to sudo this command to get access to the 443 port. So sudo node https. js and we have an https server. So we can go to https localhost and the browser is going to warn us about the self-signed certificate. It simply means that the browser does not recognize this certificate, but we can trust it manually. And there you go, very simple.
const fs = require('fs');
const server = require('https')
.createServer({
key: fs.readFileSync('./key.pem'),
cert: fs.readFileSync('./cert.pem'),
});
server.on('request', (req, res) => {
res.writeHead(200, { 'content-type': 'text/plain' });
res.end('Hello world\n');
});
server.listen(443);
Requesting HTTP/HTTPS Data
Node can also be used as a client for requesting http and https data. There are five major classes of objects in Node’s HTTP module. The Server class is what we use to create the basic server, it inherits from net. Server, so it’s an EventEmitter. A ServerResponse object gets created internally by an HTTP server. The http. Agent class is used to manage pooling sockets used in HTTP client requests. Node uses a global agent by default, but we can create a different agent with different options when we need to. When we initiate an HTTP request, we will be working with a ClientRequest object. This is different from the request object we’ve seen in the server example. That request object is actually an IncomingMessage object, which we’re gonna see in a little bit. Both clientRequest and serverResponse implement the writable stream interface. IncomingMessage objects implement the readable stream interface, and all three objects are event emitters. We’ve seen the basic server example. Let’s see a basic request example. Let’s request the HTML page at google.com with Node. We can use the http.request method. This method takes an option argument and it gives us access to a callback with a response for the host that we’re gonna request. We can specify many options here. To request google. com, we need hostname is google. com, and we can use multiple other options here. For example, the default method is get, but we can use a method like POST if we need to post information to a hostname. I’ll leave it default, and in here we can console log the response, which is going to be the html at google. com. You’ll notice one thing about the handler that we define in the second argument here, it doesn’t have an error argument. It’s simply because that this handler gets registered as an event listener and we also handle the error with an event listener here. So this http request method returns an object, and that object is an event emitter. So we can register a handler for the error event. So we can do something like request. on error, and handle the error in that case. So we can do something like this. Since this request object is a writable stream, we can do things like write, for example, if we’re writing with a post method. But for get requests, we don’t need that. We just need to terminate the stream, so we do. end here and that should do it. Let’s test. So node request and I’ll pipe it on less and you’ll see the response object here is an incoming message. So we can read information on this response object like status code. We can also read response. headers, for example, and this response object is an event emitter. It emits a data event when it receives data from the hostname. This data event gives us a callback, and that callback’s argument is a buffer. So, for example, if we want to see the actual HTML, we do toString on it, so let’s actually take a look at that. You’ll see the status code 200, the headers object, and you’ll see the actual html for ‘google.com’. If we’re not writing any information to the headers request or if we’re not posting or putting or deleting data, we could actually just use the get method here. This get method is simpler. Its first argument is just a string with the URL that we want to read, so we need to add the http protocol in here, and we don’t need to do a request. end on this. That will be done for us. So this should be equivalent, and this request is done using the global http agent. So the http module has a globalAgent, which node uses to manage the sockets here. It has some pre-configured options here. We can see that agent information here if we do a request. agent. So in here you’ll first see the agent information and then you’ll see the printed information from the response. And all this interface is exactly the same if we want to work with https instead of http. So if we want to fetch https google. com, all we need to is replace http with https, and things will work exactly the same, but it’s now communicating the request over https. So let me now show you where to identify these objects in the examples we’ve seen so far. In this example, the request object here is from the class clientRequest. This response object is of type incomingMessage. And the agent that was used for the request is of type http Agent. In the server example, the server object is from the http server class, the request object inside the request listener is from the incomingMessage class, and the response object is from the server response class.
// request
const https = require('https');
// req: http.ClientRequest
const req = https.get(
'https://www.google.com',
(res) => {
// res: http.IncomingMessage
console.log(res.statusCode);
console.log(res.headers);
res.on('data', (data) => {
console.log(data.toString());
});
}
);
req.on('error', (e) => console.log(e));
console.log(req.agent); // http.Agent
// server: http.Server
const server = require('http').createServer();
server.on('request', (req, res) => {
// req: http.IncomingMessage
// res: http.ServerResponse
res.writeHead(200, { 'Content-Type': 'text/plain' });
res.end('Hello world\n');
});
server.listen(8000);
Working with Routes
Let’s see how we can work with routes using Node’s http module. We have a basic server here, and what I want to do I want to support some routes. Let’s try to support a home route, an api route, and see how we can respond to different things in there. So, to support that, we need to first read the URL information, and this is easy. We can use req. url to do so. So now, when I request localhost, the req/url is slash, and anything I put here I will be able to read it. So, to handle routes, all we need to do is we need a switch statement for this req. url. So, for example, what should we do when it’s slash? Or, what should we do when it’s home, and so on? So let’s assume that we have a /home.html file, very basic template HTML, and we want to display this file when we go to slash home. All we need to do is write the header, which is this case text HTML, and then in here we want to read home that HTML and write it to the response. So we can use fs. readfileSync ./home.html. And of course, we need fs module and let’s try it. There you go. Excellent. What if we have an about. html and now we also want to support /about. So it’s exact same code as this portion. So we can actually make it dynamic. Let’s make it support both home and about, and let’s make it support dynamic, because this is the exact req. url part. So first convert this into a template string and then this part becomes variable req. url, just like that. Let’s test. We can now see the about page and the home page. So what should we do on the root route? How about we redirect the user to /home With Node, we just write a header for that. So it’s writeHead, and we’ll just use 301 here to indicate a permanent move, and the header that we want to write here is the location header. So, this is permanently moved to /home. And we still need to end the response, so now when we request the slash route, it will tell us that it was moved permanently to slash home. By the way, this first number here is the http status code. We can actually take a look at all the status codes using http. status_codes. So you’ll see all of them here. 301 is moved permanently. If you need to work with json data, say that we have a route here. We’ll make it /api, so this route is gonna respond with some data. Let’s assume that we have the data here is a constant. This data is some kind of object and we want to respond with this data json, so we need to do two things. First, the content type in this case is application/json. And the information that we want to write here is a stringified version of the JSON object. So it’s json. stringify the data variable. And we can test that. So in here if we go to /api, we’ll get back a json response. Now what happens if we request something that does not exist? Right now, the server does not respond with anything and it will eventually timeout. So this would be this default case here, so we should probably respond with a 404, in this case. So 404 means not found. We should still end the response here, so let’s try it now. It will give me a 404 not found.
const fs = require('fs');
const server = require('http').createServer();
const data = {};
server.on('request', (req, res) => {
switch (req.url) {
case '/api':
res.writeHead(200, { 'Content-Type': 'application/json' });
res.end(JSON.stringify(data));
break;
case '/home':
case '/about':
res.writeHead(200, { 'Content-Type': 'text/html' });
res.end(fs.readFileSync(`.${req.url}.html`));
break;
case '/':
res.writeHead(301, { 'Location': '/home' });
res.end();
break;
default:
res.writeHead(404);
res.end();
}
});
server.listen(8000);
Parsing URLs and Query Strings
If you need to work with URLs, especially if you need to parse URL, Node has this URL module that you can use. It has many methods, but the most important one is parse. The format method is also useful. So let me show you how to work with that. Before we do, let’s take a look at the url api documentation from the nodeJS. org website. This diagram here has details on all the elements of a URL. This example is manually parsing this URL in here. So you can see we have the protocol, which is HTTP. There is the authentication part, which is username and password in case we have those, and then there is hostname and port, and both of them together is a host. And there’s the path name, which is what comes after the host and before the query string. And then with a query string, the question mark including the query itself we call it search, but without the question mark we call it query. And pathname + search is called path. And if we have a hash location here it will be called hash. These are all the elements in any URL object. So if we have the URL as a string and we need to parse it into those elements, we can use the URL parse method. For example, I can use the url. parse method to parse this URL that I just grabbed from Pluralsight search, and this call is going to give me all the elements that I have in that URL. We have an HTTPS protocol. We don’t have any authentication data. The host is plurlasight. com, and since the port is null the host and hostname is the same, there’s no hash, there’s a search part query part hash name path and the of itself, which is the full URL. We can actually also specify a second argument here true to parse the query string itself, so if we do that the query string will be parsed, and reading information from the query string is as easy as doing. query. queue, for example. This will give me exactly the search query for that URL. If, on the other hand, you have the inverse situation, if you have an object will all these elements detailed and you want to format this object into a URL, you can use the url. format method. And this will give you back a string with all these URL object properties concatenated in the right way. If you only care about the query string, then you can use the query string module, which has many methods, as you see, but the most important ones are the parse method and this stringify method. So if we have an object like this one and we want to convert this object into a query string portion, all we need to do is call queryString.stringify on that object, and it will give me an actual string I can use in any url query. And you can see how this stringify method escaped some special characters by default, which is great. If you have the inverse situation, if you have a query string and you want to parse it into an object, what you need here is queryString.parse, and you give the string that you want to parse, and this will give you back an object parsed from that query string.
const querystring = require('querystring');
console.log(
querystring.stringify({
name: 'Samer Buna',
website: 'jsComplete.com/samer-buna'
})
);
console.log(
querystring.parse('name=Samer%20Buna&website=jsComplete.com%2Fsamer-buna')
);
//url.js
const url = require('url');
const urlString =
'https://www.pluralsight.com/search?q=buna';
console.log(
url.parse(urlString, true)
);
const urlObject = {
protocol: 'https',
host: 'www.pluralsight.com',
search: '?q=buna',
pathname: '/search',
};
console.log(
url.format(urlObject)
);
Summary
In this post, we talked about the basic Node’s HTTP server and how it’s non-blocking and streaming-ready out of the box. We saw how to create an HTTPS server with a test self-signed certificate that we created using the OpenSSL kit. We reviewed the five major http module classes and identified them in the examples we used. We saw how to use Node for requesting http and https data from the web. We also talked about how to support basic web routes using Node’s HTTP module. We used that to respond with HTML files, JSON data, and to handle redirects and not-found URLs. Finally, we explored how to parse and format URLs and query strings using the native Node modules provided for them.
