How Backend Works : Node.js

Node.js has revolutionized backend development, enabling developers to leverage JavaScript for server-side programming. In this comprehensive blog, we’ll delve into the core elements of Node.js, exploring its JavaScript engine, the fs module for file system operations, and how it manages HTTP requests to create robust backend servers. Additionally, we’ll cover key topics such as URLs in Node.js, HTTP methods, Express framework, REST API development, middleware usage, HTTP headers, HTTP status codes, database integration, and the MVC design pattern.

JavaScript Engine and Node.js

JavaScript Execution: JavaScript is traditionally associated with web browsers, where it’s used to make web pages interactive. Each browser comes with its own JavaScript engine responsible for interpreting and executing JavaScript code. For example, Chrome uses the V8 engine, Firefox uses SpiderMonkey, and Safari uses JavaScriptCore.

Node.js and JavaScript Execution: Node.js was created by Ryan Dahl to allow JavaScript code to be executed outside of the browser environment. Node.js uses the V8 JavaScript engine from Chrome, along with C++ to provide a runtime environment for executing JavaScript code on servers or local machines. This means that JavaScript can now be used to build web servers, desktop applications, and even IoT devices.

Node.js as a Runtime Environment: Node.js provides a runtime environment for JavaScript, similar to how a browser provides a runtime environment for JavaScript in web pages. However, Node.js does not include the DOM (Document Object Model) or the window object, which are specific to browsers. Instead, Node.js focuses on providing core functionalities needed for server-side development, such as file system access, networking capabilities, and the ability to create web servers.

File System Module (fs)

The fs module in Node.js, which stands for “file system,” provides functionality for interacting with the computer’s file system. It allows you to perform various operations on files and directories, including:

Creating Files: You can create new files using methods like fs.writeFile() or fs.appendFile()

fs.writeFileSync('./test.txt', 'this is test file'); // Creates test.txt file with 'this is test file' string data

Reading Files: The fs module provides methods like fs.readFile() to read the contents of a file.

fs.readFile('./test.txt', 'utf-8', (err, result) => {
    if (err) {
        console.log(err);
    } else {
        console.log(result);
    }
});

Updating Files: You can update existing files by writing new data to them using fs.writeFile() or fs.appendFile()

Deleting Files: The fs.unlink() function allows you to delete files.

fs.unlinkSync('./test.txt'); // Deletes test.txt file

Renaming Files: You can rename files using the fs.rename() function.

fs.renameSync('./old.txt', './new.txt'); // Renames old.txt to new.txt

Creating Directories: Methods like fs.mkdirSync() and fs.mkdir() can be used to create new directories.

fs.mkdirSync('new-folder'); // Creates a new directory named 'new-folder'

Getting Information about Files: The fs module offers functions like fs.statSync() to retrieve information about a file's size, creation time, and other properties.

console.log(fs.statSync('./test.txt')); // Shows all stats about the file

Note: Synchronous methods like fs.readFileSync() and fs.statSync() block the execution of the code until the operation is complete, while asynchronous methods like fs.readFile() and fs.stat() do not block the code and require a callback function to handle the result or error.

Node.js Server

Handling Client Requests: When a client sends a request to a Node.js server, the request is added to an event queue. The event loop in Node.js picks up these requests one by one and processes them.

Event Loop and Request Processing: The event loop processes requests asynchronously, meaning that it handles non-blocking operations without waiting for them to complete. Non-blocking operations are preferred in Node.js to prevent blocking the event loop, which could lead to performance issues.

Blocking vs. Non-blocking Requests: If a request is non-blocking, the event loop processes it and sends a response without delay. However, if a request is blocking (synchronous), it is sent to a thread pool to be handled. The thread pool manages a limited number of threads/workers (typically 4 by default) to handle blocking operations.

Threat Pool and Blocking Operations: The thread pool is responsible for fulfilling blocking operations. If a blocking operation requires a thread and all threads are busy, the request is queued until a thread becomes available. It’s important to note that excessive blocking operations can lead to performance degradation due to thread pool limitations. So the good practice is always have non-blocking operations

Example of Blocking and Non-blocking Operations:

// Blocking (Synchronous) Operation
// Reading a file synchronously
const data = fs.readFileSync('example.txt', 'utf8');
console.log('File content:', data);
console.log('This message will be printed after the file content is read.');


// Non-blocking (Asynchronous) Operation
// Reading a file asynchronously
fs.readFile('example.txt', 'utf8', (err, data) => {
  if (err) {
    console.error(err);
    return;
  }
  console.log('File content:', data);
});
console.log('This message will be printed before the file content is read.');

Thread Pool Size: The default size of the thread pool in Node.js is typically 4, but it can vary depending on the machine’s CPU cores. For example, if your CPU has 8 cores, the max thread pool size could be 8. You can check your CPU’s core count using the os module:

const os = require('os');
console.log(os.cpus().length);

Note: Node.js provides a powerful and efficient runtime environment for building scalable and non-blocking server-side applications. Understanding the event loop, blocking, and non-blocking operations is crucial for writing performant Node.js applications.

Workflow/Architecture

HTTP Server in Node.js

Including the HTTP Module: The http module in Node.js allows you to create an HTTP server and handle HTTP requests and responses. You include this module in your code using require('http')

Creating an HTTP Server: To create an HTTP server, you use the http.createServer() method. This method takes a callback function as an argument, which is called whenever the server receives a request. The callback function takes two arguments: req (the request object) and res (the response object)

Handling Requests: Inside the callback function, you can handle the incoming request (req) and send a response (res). In the example, the server logs a message when a new request is received and sends a simple "hello from server" response to the client using res.end()

Logging Request Information: You can also log information about the request by logging the req object. This object contains details such as the request method, URL, headers, and more. This can be useful for debugging and understanding the incoming requests

Starting the Server: After creating the server, you need to start it by calling the listen() method on the server object. This method takes a port number (e.g., 8000) and an optional callback function that is called once the server starts listening for requests

Accessing the Server: Once the server is running, you can access it using a web browser or tools like cURL by navigating to http://localhost:8000/ (assuming you're running the server locally on port 8000)

Example:

const http = require('http')

const myServer = http.createServer((req, res) => {
    console.log('new req recieved')
    res.end('hello from server')
    console.log(req)  //you can access all info about req
})

myServer.listen(8000, () => console.log('server started!')) 

Node.js’s http module provides a simple yet powerful way to create HTTP servers, handle incoming requests, and send responses. This example demonstrates the basic structure of an HTTP server in Node.js and how to handle requests and responses.

URLs in Node.js

Uniform Resource Locators (URLs) are used to specify the location of resources on the web. In Node.js, the url module provides utilities for working with URLs, including parsing and formatting them. Here's a breakdown of important URL components and how they are handled in Node.js:

Protocol (https):

• The protocol indicates the protocol used to transfer data, such as http or https.
• In Node.js, you can parse a URL string to extract the protocol using url.parse(urlString).protocol.

Domain (www.example.com):

• The domain is the human-readable name of the server hosting the resource.
• You can parse a URL string to extract the domain using url.parse(urlString).hostname.

Path (/search):

• The path specifies the location of the resource on the server’s file system.
• In Node.js, you can parse a URL string to extract the path using url.parse(urlString).pathname.

Query Parameters (?q=nodejs&page=1):

• Query parameters are used to send additional data to the server as key-value pairs.
• You can parse a URL string to extract query parameters using url.parse(urlString, true).query.

URL Module in Node.js:

• The url module in Node.js provides utilities for URL resolution and parsing.
• You can use url.parse() to parse a URL string into its various components and url.format() to format URL components into a complete URL string.
• Additionally, url.resolve() can be used to resolve a relative URL against a base URL to get an absolute URL.

Example:

const url = require('url');
const urlString = 'https://www.example.com/search?q=nodejs&page=1';
const parsedUrl = url.parse(urlString, true);

console.log(parsedUrl.hostname); // Output: 'www.example.com'
console.log(parsedUrl.pathname); // Output: '/search'
console.log(parsedUrl.query.q);   // Output: 'nodejs'

HTTP Methods in Node.js

GET Method:

• The GET method is used to retrieve data from the server.
• It should only retrieve data and should not have any other effect on the server.
• Example: Retrieving a list of products from a server.

POST Method:

• The POST method is used to submit data to the server.
• It is often used when submitting form data or uploading files.
• It can also be used to create new resources on the server.
• Example: Submitting a form to create a new user account.

PUT Method:

• The PUT method is used to update or replace an existing resource on the server.
• The entire resource is replaced with the new data sent in the request.
• Example: Updating a user’s profile information.

PATCH Method:

• The PATCH method is used to partially update an existing resource on the server.
• It is typically used when you want to update only specific fields of a resource.
• Example: Updating the status of an order without modifying other order details.

DELETE Method:

• The DELETE method is used to delete a resource on the server.
• It removes the specified resource from the server.
• Example: Deleting a user account from the server.

HEAD Method:

• The HEAD method is similar to the GET method, but it only retrieves the headers of the response without the body.
• It is often used to check the status of a resource without downloading the entire content.
• Example: Checking if a file exists on the server without downloading it.

Server Implementation Example:

const http = require('http');

const server = http.createServer((req, res) => {
  if (req.method === 'GET') {
    // Handle GET request
    res.writeHead(200, { 'Content-Type': 'text/plain' });
    res.end('GET request received');
  } else if (req.method === 'POST') {
    // Handle POST request
    res.writeHead(200, { 'Content-Type': 'text/plain' });
    res.end('POST request received');
  } else {
    // Handle other HTTP methods
    res.writeHead(405, { 'Content-Type': 'text/plain' });
    res.end('Method Not Allowed');
  }
});

server.listen(3000, () => {
  console.log('Server running at http://localhost:3000/');
});

Express (a node.js web application framework)

Simplifies Node.js Development:

• Express.js is a web application framework for Node.js that simplifies the process of building web applications.
• It provides a set of features for web and mobile applications, including routing, middleware support, and template engines.
• Express.js saves time by reducing the amount of common boilerplate code required in Node.js applications.

Cleaner and More Organized Code:

• Express.js helps in organizing code by providing a structured way to define routes, middleware, and error handling.
• It encourages best practices in code organization, making it easier to maintain and scale applications.

Large Community and Ecosystem:

• Express.js has a large and active community of developers who contribute to its development and provide support.
• There are many third-party middleware and plugins available for Express.js, allowing developers to add extra features to their applications easily.

Single Language for Frontend and Backend:

• With Express.js, you can use JavaScript for both frontend and backend development.
• This reduces the need to context switch between different languages and frameworks, making development more efficient.

Express Example:

const express = require('express');
const app = express();

app.get('/', (req, res) => {
  res.send('Hello, World!');
});

app.listen(3000, () => {
  console.log('Server running at http://localhost:3000/');
});

REST API in Express.js

What is REST API?

• REST (Representational State Transfer) is an architectural style for designing networked applications.
• It uses a client-server model where the client sends requests to the server, and the server responds with the requested data.

Key Concepts of REST:

• Resources: Resources are the key data entities in a RESTful API. They are typically represented by URIs (Uniform Resource Identifiers).
• HTTP Methods: RESTful APIs use standard HTTP methods (GET, POST, PUT, PATCH, DELETE) to perform operations on resources.
• Stateless: REST APIs are stateless, meaning each request from a client must contain all the information necessary to process the request.
• Response Formats: REST APIs can return data in various formats such as JSON, XML, HTML, etc.

Example Express.js REST API:

• In the provided Express.js example, we have a simple REST API for managing user data stored in a JSON file (MOCK_DATA.json).
• The API has routes for listing all users, getting a specific user by ID, creating a new user, updating an existing user, and deleting a user.

Using Express.js for REST API:

• Express.js simplifies the creation of RESTful APIs by providing a set of features for routing, middleware support, and request handling.
• It allows developers to define routes for different HTTP methods and handle requests and responses easily.

Example API Endpoints:

• GET /users: Returns a list of all users.
• GET /api/users/:id: Returns the user with the specified ID.
• POST /api/users: Creates a new user.
• PATCH /api/users/:id: Updates an existing user with the specified ID.
• DELETE /api/users/:id: Deletes the user with the specified ID.

Using Middleware:

• Middleware like express.urlencoded({ extended: false }) is used to parse incoming request bodies.
• This middleware parses form data and makes it available in req.body.

Handling Data:

• Data is stored in a JSON file (MOCK_DATA.json) and is manipulated using standard JavaScript array methods like findIndex and splice.

Example:

const express = require('express');
const users = require('./MOCK_DATA.json'); // Sample user data
const fs = require('fs');

const app = express();
const PORT = 8000;

// Middleware to parse incoming request bodies
app.use(express.urlencoded({ extended: false }));
app.use(express.json());

// Get all users
app.get('/users', (req, res) => {
  res.json(users);
});

// Get a specific user by ID
app.get('/api/users/:id', (req, res) => {
  const id = Number(req.params.id);
  const user = users.find((user) => user.id === id);
  if (user) {
    res.json(user);
  } else {
    res.status(404).json({ error: 'User not found' });
  }
});

// Create a new user
app.post('/api/users', (req, res) => {
  const newUser = req.body;
  newUser.id = users.length + 1; // Generate new ID
  users.push(newUser);
  fs.writeFile('./MOCK_DATA.json', JSON.stringify(users), (err) => {
    if (err) {
      console.error(err);
      res.status(500).json({ error: 'Failed to create user' });
    } else {
      res.status(201).json({ status: 'success', user: newUser });
    }
  });
});

// Update an existing user by ID
app.patch('/api/users/:id', (req, res) => {
  const id = Number(req.params.id);
  const updateUser = req.body;
  const index = users.findIndex((user) => user.id === id);
  if (index !== -1) {
    users[index] = { ...users[index], ...updateUser };
    fs.writeFile('./MOCK_DATA.json', JSON.stringify(users), (err) => {
      if (err) {
        console.error(err);
        res.status(500).json({ error: 'Failed to update user' });
      } else {
        res.json({ status: 'success', user: users[index] });
      }
    });
  } else {
    res.status(404).json({ error: 'User not found' });
  }
});

// Delete a user by ID
app.delete('/api/users/:id', (req, res) => {
  const id = Number(req.params.id);
  const index = users.findIndex((user) => user.id === id);
  if (index !== -1) {
    users.splice(index, 1);
    fs.writeFile('./MOCK_DATA.json', JSON.stringify(users), (err) => {
      if (err) {
        console.error(err);
        res.status(500).json({ error: 'Failed to delete user' });
      } else {
        res.json({ status: 'success', deletedUserId: id });
      }
    });
  } else {
    res.status(404).json({ error: 'User not found' });
  }
});

// Start the server
app.listen(PORT, () => {
  console.log(`Server running at http://localhost:${PORT}`);
});

What is Middleware?

Middleware functions in Express.js are functions that have access to the request object (req), the response object (res), and the next middleware function in the application’s request-response cycle. They can perform a variety of tasks, such as:

• Executing any code
• Modifying the request and response objects
• Ending the request-response cycle
• Passing control to the next middleware function

Middleware functions can be used for a wide range of purposes, such as logging, authentication, error handling, and more.

Get Shivam Shende’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Using Middleware in Express.js

In Express.js, middleware can be added to your application using the app.use() method. Middleware functions are executed sequentially in the order they are added to the application, and each middleware function can either end the request-response cycle or pass control to the next middleware function by calling the next() function.

const express = require('express');
const app = express();

// Middleware function 1
app.use((req, res, next) => {
    console.log('Middleware function 1');
    req.data = {
        message: 'Data from the first middleware',
        number: 42
    };
    next(); // Pass control to the next middleware function
});

// Middleware function 2
app.use((req, res, next) => {
    console.log('Middleware function 2');
    console.log('Message from the first middleware:', req.data.message);
    console.log('Number from the first middleware:', req.data.number);
    next();
});

// Route handler
app.get('/', (req, res) => {
    res.send('Hello World!');
});

app.listen(3000, () => {
    console.log('Server is running on port 3000');
});

Types of Middleware in Express.js

Express.js supports several types of middleware, including:

• Application-level middleware
• Router-level middleware
• Error-handling middleware
• Built-in middleware
• Third-party middleware

Each type of middleware serves a specific purpose and can be used to enhance the functionality of your Express.js applications.

HTTP Headers

HTTP headers are crucial components of Hypertext Transfer Protocol (HTTP) messages, providing additional information about the data being transmitted. While they are not part of the main content of a webpage, they play a significant role in defining how web browsers and servers communicate. Let’s delve deeper into HTTP headers to understand their format, types, and common examples.

Format of HTTP Headers:

HTTP headers are sent as key-value pairs, separated by a colon (:). For example, a header specifying the type of data being sent might look like this:

Content-Type: text/html

Types of HTTP Headers:

There are two main categories of HTTP headers: request headers and response headers.

1. Request Headers: These headers are sent by the browser to the server and contain information about the requested resource. Examples include Host, User-Agent, Accept, and Authorization.
2. Response Headers: These headers are sent back from the server to the client, providing details about the response. Examples include Content-Type, Content-Length, Location, and Server.

Common Types of HTTP Heade

1. General Headers: Apply to both request and response messages and provide general information. Examples include Cache-Control, Connection, Date, and Transfer-Encoding.
2. Request Headers: Provide additional information about the request. Examples include Host, User-Agent, Accept, and Authorization.
3. Response Headers: Provide additional information about the response. Examples include Content-Type, Content-Length, Location, and Server.
4. Entity Headers: Provide information about the entity (body) of the request or response. Examples include Content-Encoding, Content-Language, Content-Disposition, and Content-Range.
5. Custom Headers: Not standardized by the HTTP specification but used for application-specific purposes. Examples include X-Custom-Header

Visibility of HTTP Headers

HTTP headers are typically invisible to the user and are handled in the background by the browser and server. They play a critical role in facilitating efficient and secure communication between clients and servers on the web.

HTTP Status Codes

HTTP status codes are standard response codes used by web servers to indicate the outcome of a client’s request. They are three-digit integers where the first digit indicates the class of the response, while the last two digits have no categorization role. HTTP status codes are divided into five categories, each conveying a different meaning:

1. Informational responses (100–199): These status codes indicate that the server has received the request and is continuing the process. They are informational and not typically seen in normal web browsing.
2. Successful responses (200–299): These status codes indicate that the request was successful. The most common code in this category is 200 OK, which means the request was successful and the server is returning the requested data.
3. Redirection messages (300–399): These status codes indicate that the client must take additional action to complete the request. For example, 301 Moved Permanently indicates that the requested resource has been permanently moved to a new location.
4. Client error responses (400–499): These status codes indicate that there was an error on the client side of the request. For example, 404 Not Found indicates that the requested resource could not be found on the server.
5. Server error responses (500–599): These status codes indicate that there was an error on the server side of the request. For example, 500 Internal Server Error indicates that there was an unexpected condition encountered on the server.

Some common HTTP status codes include:

• 200 OK: The request was successful.
• 201 Created: The request has been fulfilled, and a new resource has been created.
• 400 Bad Request: The server could not understand the request due to invalid syntax.
• 401 Unauthorized: The request requires user authentication.
• 403 Forbidden: The server understood the request but refuses to authorize it.
• 404 Not Found: The requested resource could not be found on the server.
• 500 Internal Server Error: A generic error message returned when an unexpected condition was encountered on the server.

Databases

A database is a structured collection of data that is organized in a way that enables efficient storage, retrieval, and management of data. It is designed to handle large volumes of data and provides mechanisms for ensuring data integrity, security, and reliability.

Types of Databases:

There are several types of databases, each designed for specific use cases and requirements. Some common types include:

1. Relational Databases: Relational databases store data in tables, where each table represents a different entity, and rows represent individual records. Examples include MySQL, PostgreSQL, and SQLite.
2. NoSQL Databases: NoSQL databases are non-relational databases that can store and manage unstructured or semi-structured data. Examples include MongoDB, Cassandra, and Redis.
3. Graph Databases: Graph databases are designed to store and manage graph data structures, making them ideal for applications that require complex relationship mapping. Examples include Neo4j and Amazon Neptune.
4. In-Memory Databases: In-memory databases store data in system memory (RAM) instead of on disk, allowing for faster data retrieval and processing. Examples include Redis and Memcached

Role of Databases in Software Developement

Databases play a crucial role in software development by providing a reliable and efficient way to store and manage data. Some key roles of databases in software development include:

1. Data Storage: Databases provide a centralized repository for storing application data, ensuring data consistency and integrity.
2. Data Retrieval: Databases allow developers to retrieve data quickly and efficiently using query languages like SQL (Structured Query Language) or NoSQL query languages.
3. Data Manipulation: Databases support operations for adding, updating, and deleting data, enabling developers to modify data as needed.
4. Concurrency Control: Databases manage concurrent access to data, ensuring that multiple users or applications can access and modify data without conflicting with each other.
5. Data Security: Databases provide mechanisms for securing data, including access control, encryption, and authentication, to protect sensitive information from unauthorized access.

Example (MySql):

const mysql = require('mysql');

// Create a connection to the MySQL server
const connection = mysql.createConnection({
  host: 'localhost',
  user: 'root',
  password: 'shiv',
  database: 'new'
});

// Connect to the MySQL server
connection.connect((err) => {
  if (err) {
    console.error('Error connecting to MySQL server: ' + err.stack);
    return;
  }
  console.log('Connected to MySQL server as id ' + connection.threadId);

  // Create a new database
  connection.query('CREATE DATABASE IF NOT EXISTS new', (err, result) => {
    if (err) {
      console.error('Error creating database: ' + err.stack);
      return;                  //here return use to exit callback function(or any other function also) it stop exicuting of function
    }
    console.log('Database created successfully');

    // Switch to the new database
    connection.query('USE new', (err) => {
      if (err) {
        console.error('Error switching to database: ' + err.stack);
        return;
      }

      // Create the newTable table
      connection.query(`CREATE TABLE IF NOT EXISTS newTable (
        id INT AUTO_INCREMENT PRIMARY KEY,
        name VARCHAR(255) NOT NULL,
        email VARCHAR(255) NOT NULL,
        department VARCHAR(255) NOT NULL,
        position VARCHAR(255) NOT NULL,
        salary DECIMAL(10, 2) NOT NULL
      )`, (err) => {
        if (err) {
          console.error('Error creating table: ' + err.stack);
          return;
        }
        console.log('Table created successfully');

        // Close the connection
        connection.end((err) => {
          if (err) {
            console.error('Error closing connection: ' + err.stack);
            return;
          }
          console.log('Connection closed');
        });
      });
    });
  });
});

MVC Patterns

The MVC pattern helps in separating the concerns of an application, making it easier to manage and maintain. It promotes code reusability, as the same model and views can be used across different controllers. It also allows for easier testing, as each component can be tested independently of the others.

Here’s a breakdown of each component in the MVC pattern:

1. Model: The model represents the data and business logic of the application. It is responsible for managing the data, validating it, and performing the necessary operations on it. In a web application, the model interacts with the database to fetch or store data. For example, in a blog application, the model would handle operations such as fetching blog posts, creating new posts, and updating existing posts.
2. View: The view is responsible for the presentation layer of the application. It renders the data from the model into a user interface that the user can interact with. Views are typically HTML templates with embedded code (e.g., in a templating language like EJS or Handlebars) that allows dynamic content to be displayed. In the blog application example, the view would be responsible for rendering the blog posts in a readable format on the webpage.
3. Controller: The controller acts as an intermediary between the model and the view. It receives user input (e.g., a request from a web browser), processes it, and interacts with the model to perform the necessary actions. Based on the result of the model interaction, the controller selects the appropriate view to render. In the blog application, the controller would handle requests to view, create, or update blog posts, and then render the appropriate view based on the request.

Node.js has transformed server-side development by allowing JavaScript to be used for both client-side and server-side programming. Its event-driven, non-blocking architecture makes it perfect for building scalable and high-performance applications. By understanding how Node.js works, you can leverage its capabilities to create robust server-side solutions.

“If you think math hard, try web-development”