Retro counter
Did you have a chance to see the Internet in the late 90s or early 2000s? If you weren’t around to see it by yourself I can assure you that it was different from the Internet you saw this morning in your smartphone. And when I say different I mean it. That was the most unusual place where many controversial and opposite things were merged together to form something truly magical — the Web.
Many sites were covered with flashy banners of all imaginable sizes, relentlessly seducing visitors into clicking on them. Oh, those little animated bastards. If you looked at them long enough, you might either get a headache or end up in a hospital. It wasn’t an uncommon practice to disable image loading to get rid of the banners and to gain a speed boost while surfing as the bandwidth was rather limited those days.
Another element of the bygone era is the guestbook. Some sort of a decentralized social network! Any visitor could leave a message for the site owner or other visitors. Typically, next to the message, there was some contact information of a person who wrote it, allowing you to either reply to the author with another message in the book or to continue the conversation in private. And you didn’t have to create an account to do that!
There were many other cool things that might surprise you today but I would prefer to leave them aside and focus on one particular thing that no longer exists in its original form. It was replaced by more sophisticated modern tools, but its mechanics and basic principles are still valid today. I’m talking about the visitor counter.
These small fellas looked like this:
Basically, it’s an image generated by the server on each page load. We shouldn’t forget that JavaScript usage was rather limited back then, the term AJAX hadn’t yet become common and almost all the content on the page was static. That suggests some limitations: all visitors get the same copy of the page and see the same page structure in their browsers. If you want to change something on the page, you’d need to edit the code manually, redeploy a new version of the page on the server and voilà — the banner on the main page no longer shows a car rental ad.
What if, despite these restrictions, we want to show a different image for each visitor? The URL of the image can’t be changed. It’s static. What should we do?
The HTTP protocol already provides a solution for this problem. With each HTTP request, the browser sends a list of all cookie values related to the requested domain to the server. Thus, a unique ID in the cookie can help our server identify a user and send him a personalized image.
But how do we set this cookie in the browser? Actually we don’t have to. The server will happily do this for us. When it sends back an HTTP response, it may also send new cookie values to the browser. When a client requests the image for the first time, the server does not have any information about him, so it generates a new ID for that user, saves it in some internal storage, and then sends back a “default image” with the new cookie value. For users who already exist in the storage, it simply returns an updated image. Here is a diagram of this process:
This diagram suggests that we should create several modules in our program: a storage module and an image generator module.
ID Storage module
Probably a more appropriate name for this module would be a “session storage module”, which is more common in larger and more complex applications. I can assure you that our program is anything but complex, so I prefer to stick to simpler naming conventions. Let’s call it a simple “ID storage module” then. The main purpose of this module is to provide input data for the image that should be sent to the user. As we’ve already seen earlier, each user has a unique ID that he sends to the server with each HTTP request. The storage module collects these IDs, updates user data, and maintains a list of currently active users. That’s about all it does. It’s simple and straightforward.
Though there are two things that we should think of before we go any further.
- Where should we store the data?
If we want to reduce image loading time, then we should look in the direction of in-memory solutions. In such case, all the data will be placed in the server’s RAM which theoretically gives us the best read/write timings. Of course, it comes with a “small” trade-off — we lose our data whenever the server goes down or is restarted. For simple demo apps this isn’t an issue, but for production-ready apps one must come up with a more robust solution. For example, use a hybrid storage setup where the data is loaded into RAM and regularly saved to persistent storage, like a hard drive. For this app I‘ll use a regular JS variable as in-memory storage. - How to store it?
The easiest way is to look it up in an existing application. Let’s check Express.js — the most popular application server for Node.js. Express.js (or EJS for short) provides a session storage module called MemoryStore out of the box. By definition, the default session storage may not be the best option, but it gives us something to work with from the start. Be aware that this is not a production-ready solution. It should be replaced with something more viable before deployment on live servers. But we are not looking for the best solution, we are aiming for a simple one :)
So, let’s see what is inside this MemoryStore. Here is the code from the memory.js file.
...
function MemoryStore() {
Store.call(this)
this.sessions = Object.create(null)
}
...
MemoryStore.prototype.get = function get(sessionId, callback) {
defer(callback, null, getSession.call(this, sessionId))
}
...
MemoryStore.prototype.set = function set(sessionId, session, callback) {
this.sessions[sessionId] = JSON.stringify(session)
callback && defer(callback)
}
...
function getSession(sessionId) {
var sess = this.sessions[sessionId]
if (!sess) {
return
}
...
return sess
}As we can see here, MemoryStore uses a simple JS object as a key-value store, where a session id is the key and a serialized version of the session object is the value. It’s very tempting to do the same thing, but a simple object comes with some overhead cost, such as the lack of built-in CRUD functions, which would need to be written manually. Also, note the time of the first commit — it’s 2014. I assume there were no real alternatives to a simple object at that time. A Map object was already available then, but it was new and only starting to gain traction. Using it in a large-scale project like EJS would have been slightly unwise. Fortunately for us, it’s 2024 now, and in most cases, Map is a very good choice for creating a key-value structure. Surely, there are cases where a simple object has an advantage, like a serialization for instance, but we won’t be using it so I choose a Map object.
To sum up, the ID storage module will use an in-memory Map object. Let’s see how this translates into code.
/*
We initialize the storage with an empty Map object.
*/
const activeSessions = new Map();
...
/*
On each get request we repeat the following routine
*/
// Create a new client info object
let clientInfo = {
sid: null,
views: 1,
visitors: 1,
expTime: Date.now() + cookieMaxAgeSec * 1000 + 1000
};
// Get client cookies
const clientCookies = parsedCookies(request);
let userSession = null;
// Check if user session ID is already exist in the storage
if (clientCookies?.sid) {
userSession = activeSessions.get(clientCookies.sid);
}
// If there is a match we update the clientInfo object
// that was created earlier with the data from the storage
if (userSession) {
clientInfo = {
...userSession,
views: userSession.views + 1,
visitors: totalVisitors
};
} else { // If no match found then generate a new session id
clientInfo.sid = uid.sync(18);
clientInfo.visitors = ++totalVisitors;
}
// Generate a new image based on the clientInfo object
const imgData = await getCounterImage(clientInfo);
...
// Save clientInfo data into the storage
activeSessions.set(clientInfo.sid, {...clientInfo});Image generator module
There is an infinite number of counters out there. Each one may have a unique design, size or other aspects, whatever the reason may be. But despite all these differences, we can break them into two groups: a text-based group and a tile-based group.
In text-based counters, we add counter value as a string. If the rendering engine supports it, we can also apply some basic text transformations: change size, color, weight, or font-family. In the final image it will appear as regular text.
In tile-based counters, we create a counter value from a set of predefined images (tiles), where each image represents one digit.
Though the tiles offer more artistic freedom and are probably way more performant then text rendering, I find them more labor-intensive. The lazy person inside me votes for text-based rendering instead :)
I use Sharp library for image rendering. It’s quite popular and provides all the tools I need to create a counter image.
The code for the image generator module looks as follows.
// create an arrayBuffer for a background image
const imgPath = path.join(import.meta.dirname, '..', 'images', 'counter.png');
const initCounterImgBuffer = readFileSync(imgPath);
// create a buffer for a counter value
function inputTextBuffer(text) {
const width = 200;
const height = 18;
const svgImage = `
<svg width="${width}" height="${height}">
<style>
.title { fill: #000; font-family: 'Arial'; font-size: 14px; }
</style>
<text x="0" y="98%" class="title">${text}</text>
</svg>
`;
return Buffer.from(svgImage);
}
// merge texts into the background image and return result as a buffer
export function getCounterImage({views, visitors}) {
return sharp(initCounterImgBuffer)
.composite([
{
input: inputTextBuffer(views || 0),
top: 17,
left: 145
},
{
input: inputTextBuffer(visitors || 0),
top: 34,
left: 145
}
])
.png()
.toBuffer();
}We’ve already seen the call of the getCounterImage function in the previous code snippet. It’s the only function that the image generator module exposes to other modules. It takes text counter values as an input and returns an arrayBuffer of the final counter image.
I had to cheat a little in the inputTextBuffer function. The Sharp library is designed to work primarily with images, so I had to use SVG code to convert the counter’s text value into a vector image first and then merge it into a bitmap file.
Get Maxim Dyuzhinov’s stories in your inbox
Join Medium for free to get updates from this writer.
Now we have two main modules ready, but we need one more element, the final piece of the puzzle to glue everything together — a web server that will handle HTTP requests and send back the counter image.
Web server
When we’re looking for a Node.js web server, the first choice would be Express or one of its derivatives. Indeed, Express is a flexible and powerful framework that, in most cases, is exactly what you are looking for. It will do the job, but for small projects like this one, I encourage you to explore some alternatives or, if possible, to write your own solution. I’ll create a simple web server using the Node.js API.
Oh, and one more thing: we need not one, but two different servers. One will serve a static web page that shows the counter, and the second will be our counter server. The counter server is an independent service provider that counts users on the website. For each incoming request it sends back an image with the counter value.
In this diagram we have only two websites, but in the real world, there may be dozens of sites with thousands of users on each of them. Plus, all these users don’t stay still, they constantly jump from one page to another. This can generate a high number of requests to the counter server. It should be able to handle such a load and respond with minimal delay. Optimization is very vast and interesting subject, but I’ll deliberately avoid it, as it’s not a priority in our case. Instead, I’ll take a different approach: if the code is simple enough, it won’t break and it will be pretty fast.
import http from 'node:http';
import { open } from 'node:fs/promises';
import path from 'node:path';
const indexHtmlPath = path.join(import.meta.dirname, '..', 'front', 'index.html');
http
.createServer(async (request, response) => {
request.on('error', err => {
console.error(err);
response.statusCode = 400;
response.end();
});
response.on('error', err => {
console.error(err);
});
if (request.method === 'GET') {
const fd = await open(indexHtmlPath);
const fileStream = fd.createReadStream();
response.statusCode = 200;
response.setHeader('Content-Type', 'text/html');
fileStream.pipe(response);
} else {
response.statusCode = 404;
response.end();
}
})
.listen(8081);This is the entire code for the web server. It listens on port 8081 on localhost. For each GET request it sends back the content of the index.html file, and if something goes wrong, it returns an error code (404 or 400). We can say that this server generates a static content.
Note how easy it is to work with files in a Node.js environment. There’s no need to divide our file into chunks before sending. All we need to do is to create a stream (fileStream) and “pipe” it into the response object, Node.js handles the rest automatically.
<!DOCTYPE html>
<html>
<head>
<title>Site counter</title>
</head>
<body>
<style>
body {
display: flex;
justify-content: center;
align-items: center;
height: 100vh;
margin: 0;
}
</style>
<img src="http://localhost:8080/counter"/>
</body>
</html>On the other end of the line the browser will receive this HTML page from the web server, it will start to parse it and, eventually, when it reaches the <img> tag it will send a GET request to the counter server.
import http from 'node:http';
import { getCounterImage } from './image.js';
import { parsedCookies, asyncCleanSessionStorage } from './utils.js';
import { Readable } from 'node:stream';
import uid from 'uid-safe';
// inmemory session storage
const activeSessions = new Map();
setInterval(
() => {
// remove expired sessions once in 5 mins
asyncCleanSessionStorage(250, activeSessions);
},
5 * 60 * 1000
);
// global values
const cookieMaxAgeSec = 3*60;
let totalVisitors = 0;
http
.createServer(async (request, response) => {
request.on('error', err => {
console.error(err);
response.statusCode = 400;
response.end();
});
response.on('error', err => {
console.error(err);
});
// one endpoint per consumer (i.e. web site)
if (request.method === 'GET' && request.url === '/counter') {
let clientInfo = {
sid: null,
views: 1,
visitors: 1,
expTime: Date.now() + cookieMaxAgeSec * 1000 + 1000
};
const clientCookies = parsedCookies(request);
let userSession = null;
if (clientCookies?.sid) {
userSession = activeSessions.get(clientCookies.sid);
}
if (userSession) {
clientInfo = {
...userSession,
views: userSession.views + 1,
visitors: totalVisitors
};
} else {
clientInfo.sid = uid.sync(18);
clientInfo.visitors = ++totalVisitors;
}
const imgData = await getCounterImage(clientInfo);
const imgStream = Readable.from(imgData);
response.statusCode = 200;
response.setHeader('Content-Type', 'image/png');
if (clientInfo.sid) {
// update session storage + set client cookie
activeSessions.set(clientInfo.sid, {...clientInfo});
response.setHeader('Set-Cookie',[`sid=${clientInfo.sid};max-age=${cookieMaxAgeSec};HttpOnly`]);
}
imgStream.pipe(response);
} else {
response.statusCode = 404;
response.end();
}
})
.listen(8080);We’ve already seen fragments of this code earlier when we were discussing session management. The counter server does more or less the same thing as our web server does, though there are a few important differences.
First of all, it listens for a GET request on a specific URL path. Only users who know this unique URL can get the counter image. Of cause, regular site users are not aware of such details. They use the URLs provided by the web server. Site administrators should set up this URL for them, which is a relatively simple thing to do. We register our domain name on the site of a counter service provider, it creates a unique image URL for a given website. The link may look like this: https://counter-service.com/my-domain-com/counter. Then, we publish the URL on the web site, and that’s it. The counter is ready to count visitors.
For demo purposes, I don’t need to go this far, so I have only one URL. However, if you want, you can modify the code to create multiple end points.
Another difference is the type of the content returned. The server generates counter images in real time. There is nothing wrong with that, but it’s not what Node.js web servers are typically supposed to do. They are more focused on serving text data. Even when they need to serve an image, they treat it as static content and act more like a proxy: reading a file from the disk and sending it over the wire. When for each incoming request we serve a new text or image, we generate dynamic content. The big problem with such content is that it often leads to high CPU usage, which can have a dramatic impact on response time. Systems with good architecture try to address this issue by adding various caching and optimization techniques. Unfortunately, caching does not work well with dynamic content. The only viable strategy is to distribute the load across many servers and use more efficient algorithms.
One more point to note here is the way we handle cookies. A regular web server sets cookies when it serves the main HTML document. In contrast, the counter server only sends an image, so I use the image request to set and update cookies on the client side.
Demo
After getting through the server code, we can finally run it. If you want to run the code on your machine, follow these steps:
- Download the code from this Github link.
- Install all the dependencies by running
npm i. - Ensure you have a relatively recent version of Node.js installed. I’m using ECMAScript modules with some features that require at least version 20.11.
- Start the counter server:
npm run counter. - Open another terminal and start the web server:
npm run page. - Once both servers are up and running you can open the page with the counter:
http://locahlost:8081.
Here we have the same page open in two different browsers. Each browser has a cookie with a unique SID value, which was added by the counter server. As long as this cookie is present, I assume that there is an active visitor. So, I count all unique cookies issued by the server, which gives me two visitors. We can see this number in the “visitors” line.
Reload the page and the counter of visitors won’t change, but there will be a new number in the “views” line. Note that this is not the number of views for this particular page, but the total number of page views made by a given active user.
If a user doesn’t change or refresh the page within 3 minutes, his cookie will expire and he will be treated as a new user.
In this screenshot we can see both outgoing and incoming cookies. The browser sends the current SID and receives the same SID back with the updated expiration time.
That’s about it. The counter seems to be working as it was designed to. Of cause, it’s far from being perfect and there is a gigantic leap between this tiny counter and the sophisticated counters of the past, but I hope that you managed to get it running in your browser, and may be, for a brief moment, you had a glimpse of the internet from the late 90s.
Last but not least, it’s worth noting that this counter is powered by JavaScript. Who would have thought back then that it would be possible? :)
Sum up
First of all, I had a really good time working on this pet project. It was an exciting and rewarding experience. I’m glad that it has finally come to an end and I can turn towards a new project.
Looking back on what has been done, would I change anything or do differently now? Probably. There are a few things that could be revised:
- I would try to explore a tile-based approach for image generation that doesn’t depend on any external image library. I believe it would be relatively simple to implement and could produce better results from a performance point of view then the current implementation does.
- I would add persistent storage for sessions. Redis would be a good starting point here.
- I would rewrite the whole project in another language and then compare the performance results. Ideally, I would choose a language without automatic memory management, like C or Rust. For obvious reasons, C or Rust would outperform JavaScript on all accounts, but I’m curious how large the gap would be between them.
Thank you for your time. Stay curious and don’t hesitate to start new small projects to get more expertise.
