Node’s Common Built-in Libraries
refer from jscomplete.com, author Samer Buna
Working with the Operating System
Let’s explore some of the common built-in modules that come with Node. Node provides a number of utilities to access information directly from the operating system. We can use the os module for that. We can read information about CPUs, like their model, speed, and times. We can read information about network interfaces. We can read their IP addresses. We can also read the mac addresses, netmasks, and families. We can read information about total and free memory, where is the OS temp directory, and most importantly, what Operating System was Node compiled for. The type method will return Linux or Windows_NT, or Darwin for OS X. We can use that method, for example, to write code specific to an Operating System. We can also read the release version of the Operating System with the release method. The userInfo method is a handy one. It returns an object with information about the current user: username, uid, gid, shell, and home directory. On Windows, the shell attribute is null and both uid and gid are -1. Os.constants return an object with all the operating system error codes and process signals. The signals list is a handy, quick way to see a reference of all process signals available in the underlying OS.
const os = require('os');
/*
os.EOL
os.arch()
os.constants
os.cpus()
os.endianness()
os.freemem()
os.homedir()
os.hostname()
os.loadavg()
os.networkInterfaces()
os.platform()
os.release()
os.tmpdir()
os.totalmem()
os.type()
os.uptime()
os.userInfo()
*/
some demo output for windows shell
Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.
Try the new cross-platform PowerShell https://aka.ms/pscore6
PS C:\project> node
Welcome to Node.js v12.20.1.
Type ".help" for more information.
> os
os
> os
os
> os.EOL
'\r\n'
> os.a
undefined
> os.arch
[Function: arch] { [Symbol(Symbol.toPrimitive)]: [Function] }
> os.arch()
'x64'
> os.const
undefined
> os.constants
[Object: null prototype] {
UV_UDP_REUSEADDR: 4,
dlopen: [Object: null prototype] {},
errno: [Object: null prototype] {
E2BIG: 7,
EACCES: 13,
EADDRINUSE: 100,
EADDRNOTAVAIL: 101,
EAFNOSUPPORT: 102,
EAGAIN: 11,
EALREADY: 103,
EBADF: 9,
EBADMSG: 104,
EBUSY: 16,
ECANCELED: 105,
ECHILD: 10,
ECONNABORTED: 106,
ECONNREFUSED: 107,
ECONNRESET: 108,
EDEADLK: 36,
EDESTADDRREQ: 109,
EDOM: 33,
EEXIST: 17,
EFAULT: 14,
EFBIG: 27,
EHOSTUNREACH: 110,
EIDRM: 111,
EILSEQ: 42,
EINPROGRESS: 112,
EINTR: 4,
EINVAL: 22,
EIO: 5,
EISCONN: 113,
EISDIR: 21,
ELOOP: 114,
EMFILE: 24,
EMLINK: 31,
EMSGSIZE: 115,
ENAMETOOLONG: 38,
ENETDOWN: 116,
ENETRESET: 117,
ENETUNREACH: 118,
ENFILE: 23,
ENOBUFS: 119,
ENODATA: 120,
ENODEV: 19,
ENOENT: 2,
ENOEXEC: 8,
ENOLCK: 39,
ENOLINK: 121,
ENOMEM: 12,
ENOMSG: 122,
ENOPROTOOPT: 123,
ENOSPC: 28,
ENOSR: 124,
ENOSTR: 125,
ENOSYS: 40,
ENOTCONN: 126,
ENOTDIR: 20,
ENOTEMPTY: 41,
ENOTSOCK: 128,
ENOTSUP: 129,
ENOTTY: 25,
ENXIO: 6,
EOPNOTSUPP: 130,
EOVERFLOW: 132,
EPERM: 1,
EPIPE: 32,
EPROTO: 134,
EPROTONOSUPPORT: 135,
EPROTOTYPE: 136,
ERANGE: 34,
EROFS: 30,
ESPIPE: 29,
ESRCH: 3,
ETIME: 137,
ETIMEDOUT: 138,
ETXTBSY: 139,
EWOULDBLOCK: 140,
EXDEV: 18,
WSAEINTR: 10004,
WSAEBADF: 10009,
WSAEACCES: 10013,
WSAEFAULT: 10014,
WSAENETUNREACH: 10051,
WSAENETRESET: 10052,
WSAECONNABORTED: 10053,
...
},
signals: [Object: null prototype] {
SIGHUP: 1,
SIGINT: 2,
SIGILL: 4,
SIGABRT: 22,
SIGFPE: 8,
SIGKILL: 9,
SIGSEGV: 11,
SIGTERM: 15,
SIGBREAK: 21,
SIGWINCH: 28
},
priority: [Object: null prototype] {
PRIORITY_LOW: 19,
PRIORITY_BELOW_NORMAL: 10,
PRIORITY_NORMAL: 0,
PRIORITY_ABOVE_NORMAL: -7,
PRIORITY_HIGH: -14,
PRIORITY_HIGHEST: -20
}
}
> os.cpus()
[
{
model: 'Intel(R) Core(TM) i5-8365U CPU @ 1.60GHz',
speed: 1896,
times: { user: 361984, nice: 0, sys: 422953, idle: 272906, irq: 13265 }
},
{
model: 'Intel(R) Core(TM) i5-8365U CPU @ 1.60GHz',
speed: 1896,
times: { user: 323296, nice: 0, sys: 351640, idle: 382500, irq: 2515 }
},
{
model: 'Intel(R) Core(TM) i5-8365U CPU @ 1.60GHz',
speed: 1896,
times: { user: 364515, nice: 0, sys: 386859, idle: 306062, irq: 1765 }
},
{
model: 'Intel(R) Core(TM) i5-8365U CPU @ 1.60GHz',
speed: 1896,
times: { user: 376437, nice: 0, sys: 356031, idle: 324968, irq: 1625 }
},
{
model: 'Intel(R) Core(TM) i5-8365U CPU @ 1.60GHz',
speed: 1896,
times: { user: 353312, nice: 0, sys: 390765, idle: 313359, irq: 2046 }
},
{
model: 'Intel(R) Core(TM) i5-8365U CPU @ 1.60GHz',
speed: 1896,
times: { user: 347250, nice: 0, sys: 383781, idle: 326406, irq: 1625 }
},
{
model: 'Intel(R) Core(TM) i5-8365U CPU @ 1.60GHz',
speed: 1896,
times: { user: 353015, nice: 0, sys: 391765, idle: 312656, irq: 1343 }
},
{
model: 'Intel(R) Core(TM) i5-8365U CPU @ 1.60GHz',
speed: 1896,
times: { user: 392671, nice: 0, sys: 412468, idle: 252296, irq: 1218 }
}
]
> os.endianness()
'LE'
> os.freemem()
5060804608
> os.homedir()
'C:\\Users\\I543007'
> os.hostname()
'W-PF2B3Q7D'
> os.loadavg()
[ 0, 0, 0 ]
> os.networkInterfaces()
{
'_Common_Full-Network-Access - connectsin.sap.com': [
],
'VirtualBox Host-Only Network': [
{
address:
netmask:
family: 'IPv6',
mac:
internal:
cidr:
scopeid: 8
}
],
'Wi-Fi': [
],
'vEthernet (Default Switch)': [
{
],
'vEthernet (WSL)': [
]
}
> os.platform()
'win32'
> os.release()
'10.0.19042'
> os.tmpdir()
'C:\\Users\\I543007\\AppData\\Local\\Temp'
> os.totalmem() /1024 /1024 /1024
15.722610473632812
> os.type()
'Windows_NT'
> os.uptime()
1225
> os.userInfo()
{
uid: -1,
gid: -1,
username: 'I543007',
homedir: 'C:\\Users\\I543007',
shell: null
}
>
Working with the File System
he fs module provides simple File System I/O functions to use with Node. All of the fs module functions have asynchronous and synchronous forms. You can pick either form depending on your code logic. For example, if you’re reading a file during a server initialization process, readFileSync is probably ok, but if you’re reading a file every time a user requests something from that server, you should probably stick with the asynchronous form. Other than being synchronous or asynchronous, those forms handle exceptions differently. The async functions pass any encountered errors normally as the first argument in the callback, while the synchronous functions immediately throw any errors. When using the synchronous functions, if you don’t want the errors to bubble up, you’ll need to use a try/catch statement to handle them. The readFile method, by the way, gives back a buffer if we don’t specify a character encoding, which we can do in the second argument if needed. I usually default to working with buffers, unless I have specific needs to convert the raw data into a string. . To explore the multiple capabilities of the fs module, I thought we’d go through some realistic tasks and demonstrate how to do them with the fs module.
- Example 1 you’re tasked to write a Node script to fix that, basically to truncate each file to half its content. I have prepared some bad data-doubled files for you in this task1/files directory. As you can see, the https js file, which was our https example, is duplicate here, and all the other files in this directory have the same problem.
To fix a list of files, we first need to read that list. The readdir method can do that for us. I’m using the sync version of this method here, because basically this script has nothing to do unless we have a list of files. Once we have the list of files, this files constant will be an array of file names. Just the names, not the full path, so we’ll use the path module to get the actual full path. Don’t use string concatenation here. Always use path.join when working with file paths to make your code platform-agnostic. For every filePath, the goal is to truncate it to the right size, which is half of its current size, so we need to read each file size. We don’t need to read the file content, we just need its size, so instead of using readFile here, we can use the stat method, which gives us only meta information about the file, including its size. Note how here we used the asynchronous versions of the methods, because we have multiple files to process. Once we have the size, we can use the truncate method with exactly half the size of the file.
module info
│───solution.js
│
└───files
http.js
https.js
index.js
index2.js
net.js
// solution.js
const fs = require('fs');
const path = require('path');
const dirname = path.join(__dirname, 'files');
const files = fs.readdirSync(dirname);
files.forEach(file => {
const filePath = path.join(dirname, file);
fs.stat(filePath, (err, stats) => {
if (err) throw err;
fs.truncate(filePath, stats.size/2, (err) => {
if (err) throw err;
});
});
});
- Example 2
you have a directory with many log files, and you’re tasked to write a Node script that is to be run every day on that directory and it should delete all files older than 7 days. So just keep 1 week worth of logs in that directory. You don’t have access to that directory, so your task includes generating test data to make sure your cleaning script is working correctly. Below, with a simple for loop, we’ll create 10 sample files, using the writeFile method, and then for each file we create, we use the utimes method to change its timestamp. This method takes two arguments after the file path, the access time and the modified time. We’re just using the same value here. These two arguments expect a Unix timestamp in seconds. I wanted every file to have a different timestamp starting from the current date and going back to 9 days in the past, so I subtracted 1 day milliseconds multiplied by the current file index value, which is 0 through 9. I then divided the whole thing by 1000 to suit the arguments for the utimes function. The result of running this script is a new files directory with 10 files, each with a different timestamp. File0 is current date, file 1 is one day old, and so on. Now that we have our test data, let’s write a script to clean any files older than 7 days. Basically our script should remove files 7, 8, and 9.
//seed.js
// to generate some sample files for test
const fs = require('fs');
const path = require('path');
const dirname = path.join(__dirname, 'files');
fs.mkdirSync(dirname);
const ms1Day = 24*60*60*1000;
for (let i=0; i < 10; i++) {
const filePath = path.join(dirname, `file${i}`);
fs.writeFile(filePath, i, (err) => {
if (err) throw err;
const time = (Date.now() - i*ms1Day)/1000;
// modify the file latest access time, last modified time
fs.utimes(filePath, time, time, (err) => {
if (err) throw err;
});
});
}
// solution.js
const fs = require('fs');
const path = require('path');
const dirname = path.join(__dirname, 'files');
const files = fs.readdirSync(dirname);
const ms1Day = 24*60*60*1000;
files.forEach(file => {
const filePath = path.join(dirname, file);
fs.stat(filePath, (err, stats) => {
if (err) throw err;
if ((Date.now() - stats.mtime.getTime() > 7*ms1Day)) {
fs.unlink(filePath, (err) => {
if (err) throw err;
console.log(`deleted ${filePath}`);
});
}
});
});
- Example 3
You’re tasked to write a script that watches 1 directory and log 3 type of events on that directory. When a file is added, removed, or changed in the directory, your script should output a timestamped message about that event. We can simply use the fs. watch for this task, but fs watch does not give us enough details to account for all 3 events. Both the add and delete events are reported under the watch method as rename eventType. Here’s what I did to work around that. For testing, the files directory is what we’ll watch, and it has 3 empty files. We first read the names of the current files in the directory to be watched. Then, we start the watch listener. If the eventType is rename, it means a file was either added or deleted from the directory. So if the file exists in the currentFiles array, it means the event is a remove event, and we update our currentFiles array to remove the removed file. The logWithTime method just outputs the message with a UTC timestamp. If we get to this point in the code, it means a file is being added, so we’ll update the currentFiles array and log that message. If the eventType is not rename, the other eventType supported in this listener is the change event, so in this case, the file is being changed. The fs watch API is actually not completely consistent on different environments, so be sure to test any code written with it thoroughly first. To test this code, we simply run this file, and then we’ll try to do some changes on our test files. We’ll go ahead and change file0 and we get
file0was changed message. And let’s go ahead and try to deletefile0, and we’ll get file0 was removed. And we’ll create a new file and we get the file0 was added message.
C:.
│ solution.js
│
└───files
file0
file1
file2
const fs = require('fs');
const path = require('path');
const dirname = path.join(__dirname, 'files');
const currentFiles = fs.readdirSync(dirname);
const logWithTime = (message) =>
console.log(`${new Date().toUTCString()}: ${message}`);
fs.watch(dirname, (eventType, filename) => {
if (eventType === 'rename') { // add or delete
const index = currentFiles.indexOf(filename);
if (index >= 0) {
currentFiles.splice(index, 1);
logWithTime(`${filename} was removed`);
return;
}
currentFiles.push(filename);
logWithTime(`${filename} was added`);
return;
}
logWithTime(`${filename} was changed`);
});
Console and Utilities
The console module has some interesting less-known parts that I want to make sure that you know about. The console module is designed to match the console object provided by web browsers. In Node.js, there is a Console class that we can use to write to any Node.js stream, and there is a global console object already configured to write to stdout and stderr. Those are two different things. So, for example, you want to use the console methods but instead of writing to stdout and stderr, you want to write to say a socket or a file, all you need to do is instantiate a different console object from the Console class and pass the desired output and error streams as arguments. When we run this code, for example, it’ll create an out.log and err.log files and write to both of them every five seconds. Console.log uses the util module under the hood to format and output a message with a new line. We can use printf formatting elements in the message, or we can simply use multiple arguments to print multiple messages on the same line. Popular formatting elements are %d for a number and %s for a string. There is also %j for a json object. If you want to use the printf substitutions without console logging it, you can use the util. format method. This just returns the formatted string. We can console log objects and Node will use util. inspect method to print a string representations of those objects. Util. inspect has a second options argument that we can use to control the output. For example, to only use the first level of an object, we can use a depth: 0 option. Util.inspect just returns the string. If you want to use the inspect second argument and still print to stdout, you can use the console.dir function. It will pass the second argument option to util.inspect and print out the result. Console. info is just an alias to console. log. Console. error behaves exactly like console. log, but writes to stderr instead of stdout. Console.warn is an alias for console.error. The console object has an assert function for simple assertions to test if the argument is true. It will throw an AssertionError if it’s not. The built-in assert module has a lot more features for assertions, and I usually use it for quick assertions. It’s not perfect, but it’s good enough for simple cases. The ifError function on the assert module is a particularly interesting one. It throws value if value is truthy, which is exactly what we usually do on any errors argument in callbacks. Console. trace behaves just like console. error, but it also prints the call stack at the point where it is placed, which is handy when debugging problems. You can use console. time and console. timeEnd to start and stop timers and report the duration of an operation. The argument you pass to them should be a unique label for that operation. The util module has a few more handy functions. There is debuglog if you want to conditionally write debug messages to stderr based on the existence of the NODE_DEBUG environment variable. This debug line here will only be reported if this script was executed with NODE_DEBUG = web. The util. deprecate function can be used to wrap a function, say before you export it, to mark that function as deprecated. Users of that function will get a deprecation warning the first time they use that function. Finally, the util. inherits method was heavily used before the introduction of ES6 classes to inherit the prototype methods from one constructor to another. However, with ES6 classes and the extends keyword, the use of this util. inherits method is no longer needed or recommended, but you’ll probably see it a lot in older examples of Node code.
const fs = require('fs');
const out = fs.createWriteStream('./out.log');
const err = fs.createWriteStream('./err.log');
const console2 = new console.Console(out, err);
setInterval(function () {
console2.log(new Date());
console2.error(new Error('Whoops'));
}, 5000);
// **********************
const util = require('util');
const debuglog = util.debuglog('web');
const server = require('http').createServer();
server.on('request', (req, res) => {
debuglog('HTTP Request: %s', req.url);
res.writeHead(200, { 'content-type': 'text/plain' });
res.end('Hello world\n');
});
server.listen(8000);
// **********************
const util = require('util');
module.exports.puts = util.deprecate(function() {
for (var i = 0, len = arguments.length; i < len; ++i) {
process.stdout.write(arguments[i] + '\n');
}
}, 'puts: Use console.log instead');
// **********************
const util = require('util');
const EventEmitter = require('events');
// ***** The old way
function CustomEmitter() {}
util.inherits(CustomEmitter, EventEmitter);
CustomEmitter.prototype.write = function(data) {
this.emit('data', data);
};
// *****
// ***** The new way
class CustomEmitter extends EventEmitter {
constructor() {
super();
}
write(data) {
this.emit('data', data);
}
}
// *****
const stream = new CustomEmitter();
Debugging Node.js Applications
ode comes with a built-in debugger that supports single-stepping, breakpoints, watchers, and much more. In here, I have a simple function that I expect to calculate the negative sum of all its arguments, so when we call it with 1, 5, and 10, I expect the result to be 0 - 1 -5 -10, which is -16. The implementation is simple. We reduce the arguments starting from 0 and every time we just subtract the argument from the running total. However, the script is not working as expected, so I’m going to use the built-in debugger to step through this code and try to find the problem. We activate the debugger by simply adding the debug argument to the command line. When we do that, you’ll notice how the debugger is listening on this port. Node basically communicates with the debugger on this port. You can type help to see what you can do. Familiar debugging commands like continue, next, step are all available. Node starts the debugger session in a break state, so we always need to continue execution. If we type in continue now, the script will simply run and output the wrong answer and be done. We don’t have any breakpoints. We can actually place a debugger line anywhere in the script and when the debugger reaches that line, it will halt, allowing us to inspect that point in code and issue other commands. Since we reached the end of this script and did not really step into anything, let’s start over using the restart command. This time, before continuing the execution, we’ll set a breakpoint on line two using the sb command. This is equivalent to adding a debugger line on that point. Now that we have a breakpoint, we can continue the execution with the continue command. The debugger will break at that point. We can now inspect anything accessible to the script at that point, using the REPL command. We can, for example, inspect the args variable, which appears to be correct, so it seems our problem might be inside the reduce callback, so let’s create a breakpoint in there, line 3. Now, the debugger should break three times there, because we’re looping over three arguments, but instead of manually inspecting the variables on every break, we can define watch expressions using the watch command. I’d like to watch the arg variable and the total variable. We now have two watchers that will be reported when the debugger breaks. When we continue, the watchers report arg to be 0 and total to be 1, which is actually flipped. I am expecting total to start with 0 and arg to be the first argument in the array, which is 1. So to verify, let’s continue again. Total is now getting the 5 value, which is the second argument in the array. And if we continue again, it’s getting the 10. So I definitely got the order of the callback arguments wrong. It should be total first and arg second. Easy fix. If while debugging you cleared your screen (I can do that with CTRL L here), and you can’t remember where exactly the debugger is currently breaking, you can use the list command, which takes an argument of how many lines it should list before and after the current breakpoint. This debugger client is not perfect, but it’s good enough for a quick interactive debugging session. But it gets better. Recent Node versions support an experimental feature to integrate the Chrome dev tools to debug any script. All we need to do is use the –inspect argument with the script. But this feature does not break by default, so we can use –debug-brk argument to make it break before the first execution. When we add the –inspect argument, we get a URL to open in Chrome and when we open that URL in Chrome, dev tools will open our Node script and we can use all the familiar fully featured tools here to inspect the script. We can step into and over functions. For example, to redo our debugging session here, I can simply step into negativeSum, and while I’m stepping over the three reduce calls, I can see the argument values right there without any need to watch or set breakpoints. But if I need to set breakpoints, I can simply do that by clicking on the line numbers here. I can also use the console to inspect any variables or expressions and I can even hover over variables to see their current values. So all the features that we know and love in Chrome devtools can now be used with Node scripts, and the best thing about it is that it’s all built-in. I don’t need to install any packages to get this value. When I resume the execution of this script, the console. log call on line seven will be reported here.
// this is not so popular, but still can have a look.
function negativeSum(...args) {
return args.reduce((arg, total) => {
return total-arg;
}, 0);
}
console.log(
negativeSum(1, 5, 10)
);
Summary
More practice, more patience.
