feat(web): add web generator #285

avivkeller · 2025-05-28T21:53:14Z

Fixes #7.

This PR adds the web generator.

Tasks / Issues

P1 – Must Complete Before Merge

Add more items (anyone can do this as they review¹)

P2 – Must complete before migration

ChangeHistory component doesn’t render Markdown correctly. (See entry.changes aren't converted to HTML #326)
Add more items (anyone can do this as they review¹)

P3 – Can Be Done in a Follow-up

Remove mustache dependency
Add more items (anyone can do this as they review¹)

Get a preview

node bin/cli.mjs generate -t orama-db -i "/node/doc/api/*.md" -o "./out" # If you want search functionality
node bin/cli.mjs generate -t web -i "/node/doc/api/*.md" -o "./out" --index "/node/doc/api/index.md" # Specifying `--index` is optional, but recommended
npx serve out # You can serve the output, or just open one of the files in your browser. Serving is required for using search.

Add things as they appear, or leave review comments. ↩ ↩² ↩³

codecov-commenter · 2025-05-28T21:53:59Z

Codecov Report

Attention: Patch coverage is 82.23529% with 302 lines in your changes missing coverage. Please review.

Project coverage is 73.57%. Comparing base (385f0c3) to head (0372cf1).

✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
src/generators/jsx-ast/utils/buildContent.mjs	44.50%	96 Missing ⚠️
...nerators/jsx-ast/utils/createSignatureElements.mjs	65.70%	94 Missing and 1 partial ⚠️
src/generators/jsx-ast/index.mjs	10.52%	34 Missing ⚠️
src/generators/web/index.mjs	79.03%	26 Missing ⚠️
src/generators/orama-db/index.mjs	57.14%	18 Missing ⚠️
src/generators/web/build/bundle.mjs	91.02%	7 Missing ⚠️
src/generators/jsx-ast/utils/buildBarProps.mjs	50.00%	5 Missing and 1 partial ⚠️
src/generators/web/build/plugins.mjs	95.18%	4 Missing ⚠️
src/utils/remark.mjs	50.00%	4 Missing ⚠️
src/generators/legacy-json/types.d.ts	0.00%	2 Missing ⚠️
... and 8 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #285      +/-   ##
==========================================
+ Coverage   71.51%   73.57%   +2.06%     
==========================================
  Files         117      132      +15     
  Lines        9721    11134    +1413     
  Branches      590      718     +128     
==========================================
+ Hits         6952     8192    +1240     
- Misses       2766     2937     +171     
- Partials        3        5       +2

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

src/generators/web/components/SearchBox/constants.mjs

AugustinMauroy · 2025-05-29T18:42:20Z

src/generators/web/components/CodeBox.jsx

+  const language = matches?.groups?.language ?? '';
+  const [copyText, setCopyText] = useState('Copy to clipboard');
+
+  const handleCopy = async text => {


Why not including useCopyToClipord in ui lib ?

IIRC I didn't include it since it didn't fall under the "UI Component" category, it's a hook, however, if you disagree, I won't block.

src/generators/legacy-html/utils/tableOfContents.mjs

src/generators/web/components/NavBar.jsx

ovflowd · 2025-05-29T19:51:14Z

package.json

    "estree-util-value-to-estree": "^3.4.0",
    "estree-util-visit": "^2.0.0",
    "github-slugger": "^2.0.0",
    "glob": "^11.0.2",
    "hast-util-to-string": "^3.0.1",
    "hastscript": "^9.0.1",
    "html-minifier-terser": "^7.2.0",
+    "mustache": "^4.2.0",


I tried to use __<VARIABLE>__ like in the other generators, however, the JavaScript I was trying to add to the HTML broke the structure of the file. Using a library like Mustache properly escapes anything injected.

export const TEMPLATE = ({...props}) => dedent` <html> ${props.foo} </html> `;

this can work

Sorry if I wasn’t clear, the compiled JavaScript broke the HTML structure because it contained characters that need to be escaped to be in an HTML script.

For example, in your scenario, if props.foo contains a </html>, it’ll escape the sandbox.

Mustache performs all the needed escaping.

I don't like that we're using yet another layer of whatever freakery here. Mustache is great, but it feels like you're throing a nuke at an ant's sized problem. Not to mention adding yet another engine here will make things even more complex and costly-performance wise. Let's try to avoid throwing npm packages to any problem we encounter, shall we?

avivkeller · 2025-05-30T14:54:47Z

🎉 The code now dehydrates to the client so it can render without JavaScript!

AugustinMauroy · 2025-05-30T15:28:54Z

🎉 The code now dehydrates to the client so it run without JavaScript!

Wow 😵‍💫 and what about codetab

avivkeller · 2025-05-30T15:43:36Z

It rehydrates and runs with JS, but if you don't have JS, you can still view the docs. I used React's SSRing

src/constants.mjs

avivkeller · 2025-05-31T21:54:29Z

@AugustinMauroy and I got search to finally work 🎉

src/generators/web/client/components/MetaBar.jsx

src/generators/web/index.mjs

src/generators/web/client/components/MetaBar.jsx

src/generators/web/client/components/SearchBox/index.jsx

avivkeller · 2025-06-03T23:05:06Z

@nodejs/nodejs-website @nodejs/web-infra ChangeHistory and SideBar aren't implemented yet, so this is still a draft, but it's ready for you to take a look, so feel free to review :-)

avivkeller · 2025-06-24T20:50:45Z

@ovflowd I've done a potential mock with modified <details/> tags (so no JS!), wdyt¹:

These are mocks, and would still need to be implemented in our ui-components, and might not be realistic. The data shown is placeholder, and doesn't actually reflect the docs ↩

ovflowd · 2025-06-24T22:05:00Z

I was more thinking of what I shared before. We can reconstruct what would be the method structure:

methodName(arg1, arg1, arg3,...): ReturnType1
methodName(arg1, argh): ReturnType2

(Think the above as the method having overloads if any). These would be within a code snippet as if they were a type definition of sort (see the link to next.js docs I've attached); We do have the structure of the method which is generated by the JSON Generator (and I suppose also resides on the AST?)

And then below this we would have a table with the argument names, description, etc...

That IMO would look the cleanest.

avivkeller · 2025-06-24T22:09:50Z

Ahh I didn't see the link reference, I understand now. Thanks!

dario-piotrowicz · 2025-06-24T23:57:04Z

the left aside nav links don't have a hover effect, is that intentional?

It's the same on https://nodejs.org, so any changes on that front should be done in @node-core/ui-components.

the links on the right aside (table of contents, etc...) do have a hover effect but the difference is practically non-existent (in dark mode is #fff vs #e9edf0)

Ditto, see @node-core/ui-components.

Yeah, I see 👀 , anyways I do feel that these are not alright, but I agree that then it is out of scope here 🙂👍

there are some elements that have the exact same style in light mode (but they look to be styled for dark mode for me)

the search element on the top

Fixed in the latest commit.

Looks great! Thanks! 😄

code snippets

It's the same on https://nodejs.org. IIRC we had a light-mode design, but it's in the Figma graveyard.

ok then again out of scope but ideally something to follow up on 🙂👍

I personally think the stability alter boxes should include the word stability (right now I only see something like 0 Experimental which I understand because of my familiarity with the nodejs api docs, however I don't think that this is new-comer friendly, ideally it should be, more similar to the current one where it says Stability 0: Experimental)

In our Figma design, the word "Stability" is not included, however, would you prefer one of the designs below?

The second one for sure for me! 😄

claudio already mentioned the version picker, I would also like to add the the width of the element makes the left aside have a very ugly/unnecessary horizontal scrollbar on smaller (but not mobile) window widths)

As I mentioned above, "Would you prefer we hide the scrollbar, like we do on the right side?"

I can see that you've hidden the scrollbar... no that's not the right solution in my opinion, I think that the version picker should have a max-width so that it doesn't overflow its container (causing the horizontal scrollbar not to appear at all since it won't be needed)

the Toggle navigation menu element is not very accessible, it has a role of button and an aria-label, but it doesn't seem like it is interactable via keyboard (tab navigation + enter)

That's an issue for @node-core/ui-components, it's the same behavior on https://nodejs.org.

the history selector's focus-visible style is barely noticeable in light mode

Ditto, see @node-core/ui-components.

Ok, sorry for all of these @node-core/ui-components comments here 😓

for some reason the whole document body is focusable via keyboard (/tab) navigation

I just tab-ed through the entire page, and didn't see a point where the entire body was highlighted.

Yes... I am not seeing that anymore... I 100% saw that though... I'll try to see if it happens only after certain conditions 😕

Out of curiosity I also run an accessibility audit using axe DevTools and it has found 5 serious issues

Fixed the HTML language issue in the latest commit, see nodejs/nodejs.org#7890 for the contrast issue, other issues are inherited from @node-core/ui-components, i.e. the list not containing only li.

👍

avivkeller · 2025-06-25T00:01:50Z

@dario-piotrowicz for your @node-core/ui-components concerns, do you mind opening a catch all issue in nodejs.org, so we can follow up?

dario-piotrowicz · 2025-06-25T00:01:59Z

Now there's an evident flickering on the search element (also on the theme toggle button) in dark mode when I refresh the page:

simplescreenrecorder.mp4

(PS: they flicker in light mode too but not as evidently)

dario-piotrowicz · 2025-06-25T00:04:13Z

@dario-piotrowicz for your @node-core/ui-components concerns, do you mind opening a catch all issue in nodejs.org, so we can follow up?

do you mind if I open self isolated issues? (as they might be easier to deal with?)

or maybe an issue with sub-issues?

avivkeller · 2025-06-25T00:05:05Z

do you mind if I open self isolated issues? (as they might be easier to deal with?)

or maybe an issue with sub-issues?

Same difference, just figured catch all might be less work for you, but that's even better :-)

dario-piotrowicz · 2025-06-25T00:10:28Z

do you mind if I open self isolated issues? (as they might be easier to deal with?)
or maybe an issue with sub-issues?

Same difference, just figured catch all might be less work for you, but that's even better :-)

Ok I'll open isolated issues there 🙂👍

avivkeller · 2025-06-25T00:10:47Z

(PS: they flicker in light mode too but not as evidently)

It's actually the Orama component loading in (which moves the toggle). Once Orama fixes their components, it'll be partially statically dehydrated, filling in the space

dario-piotrowicz · 2025-06-25T00:12:33Z

In order for search to work, you need to run:
node bin/cli.mjs generate -t orama-db -i "/node/doc/api/*.md" -o "./out"
To generate the database (orama-db.json)

Sorry for not following the PR instructions earlier 🙂

Yes I did run the orama-db generation and that does enable search for me! 😃

avivkeller · 2025-06-25T00:36:50Z

And then below this we would have a table with the argument names, description, etc...

I'm just concerned what happens when there is a lot of nesting, for example,

Parameter	Type	Description
`hostname`	`<string>`	The hostname to resolve.
`options`	`<integer>` \| `<Object>`	Optional flags or configuration object.
└─ `family`	`<integer>` \| `<string>`	The record family: `4`, `6`, or `0`. `'IPv4'` and `'IPv6'` are interpreted as `4` and `6`. `0` allows either IPv4 or IPv6. With `{ all: true }`, returns one or both address types depending on the system. Default: `0`.
└─ `hints`	`<number>`	One or more `getaddrinfo` flags, bitwise ORed if multiple.
└─ `all`	`<boolean>`	If `true`, returns all resolved addresses in an array. Otherwise, returns a single address. Default: `false`.
└─ `order`	`<string>`	Controls sorting of addresses: `verbatim`, `ipv4first`, or `ipv6first`. Default: `verbatim`. Configurable via `dns.setDefaultResultOrder()` or `--dns-result-order`.
└─ `verbatim`	`<boolean>`	If `true`, returns addresses in the order provided by the DNS resolver. If `false`, IPv4 addresses precede IPv6. Deprecated in favor of `order`. If both are set, `order` takes precedence. Default: `true`.
`callback`	`<Function>`	Callback function for the resolution.
└─ `err`	`<Error>`	Error object if resolution fails.
└─ `address`	`<string>`	The resolved IPv4 or IPv6 address.
└─ `family`	`<integer>`	`4` or `6`, representing the address family. `0` indicates a possible bug in the name resolution system.

That's going to be quite a table, no?

avivkeller · 2025-06-25T02:52:09Z

As a personal design preference, I quite like the expandable function signatures, I think it gives a really cool way of displaying the parameters.

For events, I was thinking of it showing on("[name]", (...) => {}) rather than the current layout; this also gives users a way to see how to access events (if they don't already know)

ovflowd · 2025-06-25T12:43:09Z

And then below this we would have a table with the argument names, description, etc...

I'm just concerned what happens when there is a lot of nesting, for example,

Parameter Type Description
hostname <string> The hostname to resolve.
options <integer> | <Object> Optional flags or configuration object.
└─ family <integer> | <string> The record family: 4, 6, or 0. 'IPv4' and 'IPv6' are interpreted as 4 and 6. 0 allows either IPv4 or IPv6. With { all: true }, returns one or both address types depending on the system. Default: 0.
└─ hints <number> One or more getaddrinfo flags, bitwise ORed if multiple.
└─ all <boolean> If true, returns all resolved addresses in an array. Otherwise, returns a single address. Default: false.
└─ order <string> Controls sorting of addresses: verbatim, ipv4first, or ipv6first. Default: verbatim. Configurable via dns.setDefaultResultOrder() or --dns-result-order.
└─ verbatim <boolean> If true, returns addresses in the order provided by the DNS resolver. If false, IPv4 addresses precede IPv6. Deprecated in favor of order. If both are set, order takes precedence. Default: true.
callback <Function> Callback function for the resolution.
└─ err <Error> Error object if resolution fails.
└─ address <string> The resolved IPv4 or IPv6 address.
└─ family <integer> 4 or 6, representing the address family. 0 indicates a possible bug in the name resolution system.
That's going to be quite a table, no?

I mean sure, but any other design would also be quite the thing. I guess we could refine. For example if you have 2 variations we could have:

methodName(arg1, arg1, arg3,...): ReturnType1

TABLE CONTAINING VARIATION ONE

methodName(arg1, argh): ReturnType2

TABLE CONTAINING VARIATION TWO?

ovflowd · 2025-06-25T12:46:47Z

To be honest, I like this design:

But it doesn't speak our design language and it is a bit complex visually speaking. The one from the example of Next.js docs is simple and easy to visualize. With variation we can adopt many different paths and experiment.

avivkeller · 2025-06-25T12:57:09Z

Okay, I'll program the table, and we'll see how it looks.

avivkeller · 2025-06-25T16:40:32Z

@ovflowd (Note: This is a first draft to show what a table would look it more-or-less, i.e. Property will be changed to Parameter, etc)

ovflowd · 2025-06-25T17:56:04Z

Nice, I still think we need to add that snippet of method signature that I mentioned. This would allow us to rename the header section to be simply the method name or class name. It also helps on Search Indexing

avivkeller · 2025-06-25T18:10:05Z

Nice, I still think we need to add that snippet of method signature that I mentioned. This would allow us to rename the header section to be simply the method name or class name. It also helps on Search Indexing

Yes, the draft was just for the table, but I plan to use parseSignature (or a variation of) for that.

avivkeller · 2025-06-27T14:18:30Z

Tables:

Combined signatures:

avivkeller · 2025-06-27T16:33:02Z

avivkeller · 2025-06-27T18:56:33Z

With the speed improvements we've made, this generator takes roughly a minute and a half. It's not amazing, but it's better than the 2 minutes we were seeing earlier. The hottest code path is PostCSS, so once we preprocess the CSS/JSX on the ui-components side, it'll speed up more.

All-in-all, the slowest code, in order of speed (slowest 1, fastest 4):

PostCSS (by a large margin)
Dehydration
Terser
ESTree to Babel

avivkeller marked this pull request as ready for review May 28, 2025 22:13

avivkeller requested a review from a team as a code owner May 28, 2025 22:13

avivkeller commented May 29, 2025

View reviewed changes

src/generators/web/components/SearchBox/constants.mjs Outdated Show resolved Hide resolved

avivkeller marked this pull request as draft May 29, 2025 16:36

AugustinMauroy reviewed May 29, 2025

View reviewed changes

ovflowd reviewed May 29, 2025

View reviewed changes

avivkeller force-pushed the feat/web/gen branch from 1eb144a to 492a696 Compare May 29, 2025 20:30

avivkeller force-pushed the feat/web/gen branch from d8ab24c to 9ebcb95 Compare May 30, 2025 14:57

avivkeller mentioned this pull request May 30, 2025

chore(ui): simplify CodeBox-like UI components nodejs/nodejs.org#7793

Merged

avivkeller added the enhancement New feature or request label May 30, 2025

avivkeller added this to Node.js API Documentation Tooling May 31, 2025

avivkeller moved this to In Progress in Node.js API Documentation Tooling May 31, 2025

avivkeller removed this from Node.js API Documentation Tooling May 31, 2025

AugustinMauroy reviewed May 31, 2025

View reviewed changes

src/constants.mjs Outdated Show resolved Hide resolved

flakey5 reviewed May 31, 2025

View reviewed changes

src/generators/web/client/components/MetaBar.jsx Outdated Show resolved Hide resolved

avivkeller force-pushed the feat/web/gen branch from 705f9e2 to 76ef14a Compare May 31, 2025 22:18

AugustinMauroy reviewed Jun 1, 2025

View reviewed changes

src/generators/web/index.mjs Show resolved Hide resolved

AugustinMauroy reviewed Jun 1, 2025

View reviewed changes

src/generators/web/client/components/MetaBar.jsx Outdated Show resolved Hide resolved

AugustinMauroy reviewed Jun 1, 2025

View reviewed changes

src/generators/web/client/components/SearchBox/index.jsx Outdated Show resolved Hide resolved

avivkeller force-pushed the feat/web/gen branch 2 times, most recently from 024bbea to dbfe55d Compare June 3, 2025 21:40

avivkeller force-pushed the feat/web/gen branch from e814300 to ba07721 Compare June 5, 2025 20:27

avivkeller linked an issue Jun 6, 2025 that may be closed by this pull request

Write React Components #10

Closed

avivkeller force-pushed the feat/web/gen branch from ca72ed0 to 99176ba Compare June 6, 2025 20:33

avivkeller force-pushed the feat/web/gen branch from dfae938 to e771607 Compare June 27, 2025 14:17

avivkeller force-pushed the feat/web/gen branch from ee44087 to beb629e Compare June 27, 2025 16:52

feat(web): add web generator

e4d7134

use fast shiki

0372cf1

avivkeller force-pushed the feat/web/gen branch from beb629e to 0372cf1 Compare June 27, 2025 18:57

feat(web): add web generator #285

Are you sure you want to change the base?

feat(web): add web generator #285

Conversation

avivkeller commented May 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Tasks / Issues

P1 – Must Complete Before Merge

P2 – Must complete before migration

P3 – Can Be Done in a Follow-up

Get a preview

Footnotes

Uh oh!

codecov-commenter commented May 28, 2025 • edited by codecov bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

AugustinMauroy May 29, 2025

Choose a reason for hiding this comment

Uh oh!

avivkeller Jun 6, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

ovflowd May 29, 2025

Choose a reason for hiding this comment

Uh oh!

avivkeller May 29, 2025

Choose a reason for hiding this comment

Uh oh!

AugustinMauroy May 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

avivkeller May 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ovflowd May 30, 2025

Choose a reason for hiding this comment

Uh oh!

avivkeller commented May 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AugustinMauroy commented May 30, 2025

Uh oh!

avivkeller commented May 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

avivkeller commented May 31, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

avivkeller commented Jun 3, 2025

Uh oh!

avivkeller commented Jun 24, 2025

Footnotes

Uh oh!

ovflowd commented Jun 24, 2025

Uh oh!

avivkeller commented Jun 24, 2025

Uh oh!

dario-piotrowicz commented Jun 24, 2025

Uh oh!

avivkeller commented Jun 25, 2025

Uh oh!

dario-piotrowicz commented Jun 25, 2025

Uh oh!

dario-piotrowicz commented Jun 25, 2025

Uh oh!

avivkeller commented Jun 25, 2025

Uh oh!

dario-piotrowicz commented Jun 25, 2025

Uh oh!

avivkeller commented Jun 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

avivkeller commented May 28, 2025 •

edited

Loading

codecov-commenter commented May 28, 2025 •

edited by codecov bot

Loading

AugustinMauroy May 29, 2025 •

edited

Loading

avivkeller May 30, 2025 •

edited

Loading

avivkeller commented May 30, 2025 •

edited

Loading

avivkeller commented May 30, 2025 •

edited

Loading

avivkeller commented Jun 25, 2025 •

edited

Loading

avivkeller commented Jun 27, 2025 •

edited

Loading

avivkeller commented Jun 27, 2025 •

edited

Loading