INTRODUCTION

As a relatively old Programmer, I’ve learned lots of different technologies and languages over the years. In addition to Games/VR, I also developed many commercial database systems in the past. I was a Senior Analyst at quite a few different companies, doing analysis, design and implementation (from dBase era to Clipper to Access to C++ MFC to PHP to .NET and Java). Lately I’ve been developing a new database application with NodeJS (https://nodejs.org). This new application will be publicly accessible most probably next month.

Grossly speaking, NodeJS is nothing more than a console application which runs Javascript. I am not telling that to criticize, but instead, to make it easier for newcomers to understand what it really is. Also it does basically the same as Asp.net and Java have been doing for more than a decade already, *but* it does it in a different way, using Javascript in both sides of the application, which ends up being really cool.

Although I’m writing this in 2018 while it has been around for a few years already, in my opinion, only recently it became a solid option against the big brothers. Javascript improved a lot, there are now good IDEs around and the entire thing took a somewhat clear shape. It is maturing. The NodeJS environment is still lightweight, easy to install and portable (you can install the NodeJS environment and develop servers even on an Android phone, for example). So, if done correctly, it becomes an interesting platform to develop with.

I have found a lot of confusion on the net though, and because of that I decided to write this post in the hope to help other people who probably know how to program already, but are finding it hard to enter the new bandwagon, because of all the confusion. Most random tutorials out there make it seem that you need to install half the internet to write a Hello World, and that is not true.

What I mean is: NodeJS itself is very simple. Really. What makes it confusing is the plethora of external libraries and helpers that everyone seems to need so desperately. To make a long story really short, you don’t actually *need* anything else (although some of them will indeed help to accelerate development). There seems to exist a phenomenon that exaggerates the argument “not reinvent the wheel”, which already led to absurds like this: 11 Lines Of NodeJS Code Almost Broke The Internet — seriously, that many people don’t know how to left-pad a string and need a package for that? That can be done on a single line, for God’s sake! I could write a lot about that matter, but OK, I won’t dive into that, at least not on this post.

NodeJS is basically Google V8 engine (https://developers.google.com/v8). Google V8 is used on Chrome to run Javascript code on the web. One of the nice things about Google V8 is that it is cross-platform, so it runs on many different Operating Systems.

What NodeJS made was — grossly speaking — compile V8 as a standalone program, and let it execute Javascript from the command-line. You create a Javascript file, then run it with NodeJS. Very grossly speaking, that is basically it. To make it more interesting, it provides some native system access — built-in and through plugins, so Javascript becomes more powerful than the sand-boxed Javascript from browsers. You can read/write local files, open network sockets, and have access to a number of cool native features — but still, cross-platform.

Now, as that made it so easy to write things and modules for it, these days there are thousands and thousands of plugins, libraries (in the form of packages) and code snippets out there, which can really make things too confusing for those who are just coming to the platform — but if you start from the start, the platform itself is very simple.

Again, grossly speaking, it is nothing more than a console application which runs Javascript.

A FIRST EXAMPLE

Before we dive into code examples, please note that I’m not actually teaching how to code. You should either already know Javascript, or have enough programming experience in other languages and client-server architectures to infer the logic from the code posted ahead. This is not a Javascript tutorial, this is a NodeJS architecture tutorial. If you want to learn Javascript, there are many tutorials, but I would suggest you to start here: https://www.google.com/search?q=learn+javascript.

Let’s suppose for example that you wanted Javascript to sum 2+2 and show the result. Traditionally, on a web-browser, that would require you to design a simple web-page in HTML, putting some lines on it, and running it inside a web-browser. Something like this:

<html>
	<head>
		<script>
			function sum(a, b)
			{
				return a + b;
			}
		</script>
	</head>
	<body>

		2 + 2 = <script>document.write(sum(2, 2))</script>

	</body>
</html>

You put the above inside an “index.html” and open it on your browser. And you see “2 + 2 = 4” on the page.

Now, with NodeJS, you don’t need an actual browser. You just create a file with any name, anywhere — let’say: “sum.js” in your current directory — and run it. Something like this:

console.log("2 + 2 = " + sum(2, 2));

function sum(a, b)
{
	return a + b;
}

Then on a console window, run with

node sum.js

And that is essentially the very same thing, except that you did not need a web-browser for that — well, again grossly, NodeJS *is* the browser running the Javascript for you, but on the console. And that is all the essence of NodeJS, really. As simple as it looks.

A BUILT-IN FEATURE EXAMPLE

But then things get more interesting, of course, because it actually provides some built-in and transparent native access for the server-side Javascript (the client-side Javascript which will run on the browser will still be sand-boxed for obvious security reasons, of course) — please check the NodeJS API to know all them; for example, it provides a built-in http module which allows you to quickly create a simple http server in Javascript, in just a few lines:

var http = require('http');
var port = 3000;

// create an http server that answers with "Hello World" when accessed
const server = http.createServer(function (req, res)
{
	res.writeHead(200, {'Content-Type': 'text/plain'});
	res.write('Hello World!');
	res.end();
});

// start listening for connections
server.listen(port, () =>
{
	console.log("Server running on port " + port);
});

Please note that on the above example, I used a built-in feature of NodeJS. You don’t need anything else except NodeJS installed. Just create a file anywhere — let’s say, “http.js” — then type:

node http.js

And you’re done. That is a working “Hello World” web-server, and you can test it by opening http://localhost:3000 on your browser. Very simple I must say, but if you previously searched for some tutorials, you probably found people adding half the internet to do stupidly simple things like that, and that is exactly what makes it look so confusing and unnecessarily complex. As I’m saying, just start from the start, and soon you’ll dominate it.

Here you will notice that you have both server and client already. The server is ran by “node http.js”, the client is ran with a browser, by simply opening the url. Of course you could write more than “Hello World!” as the return from the server — you could certainly write a complex web-page, including Javascript which would then be executed by the browser. Do you see that? The above is like a micro-apache server, let’s say. It runs as a console application, but when you connect to it through a browser, it returns html to the browser. Here the server returned just a string “Hello World!”, but it could return anything to the browser (which is, in fact, the client). You could easily extend the above bare-bones http server example by reading files from the hdd and writing them back to the pipeline, effectively creating a feature complete web-server. Just with NodeJS alone.

This is very important that you visualize correctly what is server and what is client in the example above. In the above case, the server returns things to a client — which does not really need to be a browser, although that is the most used architecture currently. You run a NodeJS server, and you connect to it using a browser.

ADDING PACKAGES WITH NPM

OK, so, as you start to write more and more complex code in Javascript for NodeJS, you will naturally remember that you’ve seen people downloading half the internet to build their applications. Now with that I can partially agree, there are indeed quite a few useful packages out there, which will save you precious development hours. Just use them wisely, and you’ll be fine and productive.

For example, you could replace the above bare-bones http server with something more powerful, and avoid writing a lot of http web-server handling code yourself. NodeJS comes with npm, a Package Manager which have a large database of packages (libraries), including ExpressJS for example, probably the most used http server these days. As I insist to say, though, ExpressJS is *not* a requirement, it’s just a way of accelerating development, but it’s better to first understand that it’s not really a requirement, so things don’t become too confusing so quickly. Not saying that you should, but you *could* very well write something similar yourself. And there are certainly times when it’s better to write your very own lib than to fight with something that will just bring bloat and confusion to your project. Keep that in mind.

Because now we’re going to use external packages downloaded from the internet to help our productivity, you will notice that now we have to actually create a directory for our project, and initialize npm there. Does not look so confusing now, right? Things are starting to make a bit more sense, hopefully. OK, so let’s rewrite our http web-server to use ExpressJS then:

mkdir web-server
cd web-server
npm init
(press enter to all questions, this is just a simple test anyway)
npm install express

You will notice that npm created a “node_modules” directory, with many sub-directories and files in there. Those are dependencies of ExpressJS, downloaded and installed by npm. Oh well, at least now that makes some sense. =)

OK so, in the same directory, create a “server.js” with this:

// bare-bones express web-server

const express = require('express');
const app = express();

var port = 3000;
app.get('/', (req, res) => res.send('Hello World!'));
app.listen(port, () => console.log('Listening on port ' + port));

And run with

node server.js

The server starts listening for http connections on port 3000. Just point your browser to http://localhost:3000 and you’ll see the “Hello World” returned by the ExpressJS web-server. I won’t dive into the additional features of ExpressJS here, please refer to its homepage (https://expressjs.com) to find more. I just wanted to progressively show to you WHY and HOW additional packages are added to a NodeJS project.

Please note that, although we improved the complexity a bit, we did not mention other things like ReactJS and Webpack yet. They are not really *needed* as well, but if you continue improving, you’ll eventually find that they start to make sense. Hmm well, for some cases, of course, not all. My criticism again comes to the inefficiency of adding half the internet to create stupidly simple things which are possibly built-in already, or could be written in just a few lines of code, instead of adding megabytes of code dependencies.

Soon you realize that the possibilities are really huge. You can add a MySQL package, and have a simple console app that access a database, or going further, access a database and then send the query results down the web pipeline through the web-server. Add a UI package and have beautiful client rendering of those database queries served through the web. And so on.

Let’s create a simple console app that lists people’s names by querying MySQL. This first MySQL example won’t have a web-server, it’ll just list the query results on console. We will use the MySQL package for that.

mkdir mysql-console
cd mysql-console
npm init
(press enter until npm finishes initialization)
npm install mysql

Now we will assume that we have a database “node_tut” with a table “people” and this table has only “id” and “name” fields, which we want to list on console. If you need to learn MySQL, please head to https://www.google.com/search?q=learn+mysql – as with everything else, you’ll find many free resources to study.

Create a file called “mysql-console.js” on the project’s root directory:

// table is: id and name

const mysql = require('mysql');

var con = mysql.createConnection
(
	{
		host: "localhost",
		user: "some_user",
		password: "some_password",
		database: "node_tut"
	}
);

con.connect(
	function(err)
	{
		if (err) throw err;
		console.log("Connected!");

		console.log("Querying...");
		var query = con.query("select id,name from people");

		query.on('error', function(err)
		{
			throw err;
		});

		query.on('result', function(row)
		{
			console.log("row: " + row.id + " - " + row.name);
		});

		con.end();
		console.log("Done.");
	}
);

Run it with

node mysql-console.js

And you should immediately see a list of people that are registered on that fictional database. Directly on the console.

Now, we want to go further and access that people’s list from the web. First, let’s add ExpressJS to the packages list of that application:

npm install express

ExpressJS should be available in our code now, so let’s add support for it — edit the file mysql-console.js and replace the code with this:

// table is: id and name

const express = require('express')
const mysql = require('mysql')
const app = express()

var port = 3000;


// queries de db and sends the result down the web
function list(res)
{
	var list = "";

	var con = mysql.createConnection
	(
		{
			host: "localhost",
			user: "some_user",
			password: "some_password",
			database: "node_tut"
		}
	);

	con.connect(
		function(err)
		{
			if (err) throw err;
			console.log("Connected!");

			console.log("Querying...");
			list = "<h3>PEOPLE:</h3><br>";

			var query = con.query("select id,name from people", (err, rows) =>
			{
				if (err) throw err;

				rows.forEach( (row) =>
				{
					list += row.id + " - " + row.name + "<br>";
				});

				res.send(list);
			});

			con.end();

			console.log("Done.");
		}
	);
}

// express web server
app.get('/', (req, res) => res.send("<a href='/list'>List People</a>"));
app.get('/list', (req, res) => list(res));
app.listen(port, () => console.log('Example app listening on port ' + port));

Run it again with

node mysql-console.js

Note that although the name is the same, this is now a web-server application. It does not list people in the console anymore, but on the web. The web-server will actually return a link when the homepage “/” is accessed, and the link will get another route (as it’s commonly called these days) “/list”, and that “/list” route will call the list() function to query the database, and return the list of people as html. Access it with http://localhost:3000 and see that you can see the people from the database on a browser.

Of course, that is all bare-bones, but it hopefully shows the concept in full, with minimal code and installed packages, while keeping clear distinction of parts.

CONCLUSION

I think that this post is too big already, so I’ll stop here for now. I will probably make another post in the near future, demystifying ReactJS and Webpack. I really want to write a continuation talking about them. Later, when time permits.

To get a solid initial grip on NodeJS, and expand on it, I suggest that you start by visiting its API documentation: https://nodejs.org/dist/latest-v9.x/docs/api — please note that this URL will certainly change over time, when newer versions arrive. Maybe you will prefer to simply start from the main site: https://nodejs.org and find the API docs from there.

After you get a solid understanding of the basis and built-in features, then you’ll be much less confused by the myriad of available packages. Start small and grow solid, it’s not as hard as it seems.

I hope that this tutorial was useful in some way. Thanks for reading, and good luck.