Exploring Better Monorepo and Faster Build Support for ReScript

[cristianoc]

Title: Exploring Better Monorepo and Faster Build Support for ReScript

Description:
There has been a discussion about improving monorepo support and faster build times for ReScript projects. A Rust-based tool called rewatch has been developed by @rolandpeelen and @jaap to address these issues in their own monorepo setup. It bypasses ninja and calls bsc directly, which provides a 2x to 3x improvement in build times and reduces stale builds. Some participants in the discussion considered the benefits and challenges of writing the tool in OCaml instead of Rust, while others discussed the possibility of cross-compilation and improving CI support for various platforms. The goal is to improve monorepo support and build times for a wider range of ReScript projects.

Key points from the discussion:

• rewatch is not a ninja generator, but does clever stuff to avoid rebuilding things that have already been built.
• The tool is currently tailored for a specific monorepo setup and needs further work to support the full bsconfig spec.
• The main reason for faster build times is putting all compiler assets in a single folder, which reduces the number of include paths for bsc and speeds up lookups.
• The tool compiles the entire monorepo as if it's a single project, which makes monorepo builds more consistent.
• It watches for changes in the monorepo as a whole and efficiently calculates the files that need to be recompiled.
• Dependency tracking is done globally, providing a more efficient and accurate tracking compared to bsb.
• Some potential improvements were identified, such as passing the bsconfig location as a parameter to save IO lookups.
• Rust was chosen over OCaml for rewatch due to its advantages in file watching, efficient OS system calls, and cross-platform support. However, certain aspects of the implementation might still require binding to C-libraries if done in OCaml.
• Participants discussed the possibility of integrating the Rust tool directly into the compiler to avoid the overhead of spawning processes and calling the compiler separately.

Actions:

• Document the current monorepo setup and how rewatch has improved the build process for it.
• Investigate how other monorepo setups can benefit from these improvements.
• Consider the benefits and challenges of writing the tool in OCaml for better integration with ReScript's ecosystem.
• Continue working on supporting the full bsconfig spec and fixing any remaining bugs.
• Explore cross-compilation options and improve CI support for various platforms, including statically linked Linux binaries.

[fhammerschmidt]

Referencing #3521 here, so that we don't forget about symlink support on the way.

[jfrolich]

@fhammerschmidt I think this is supported by rewatch currently?

[fhammerschmidt]

Yes.

I did not think of rewatch when I made that comment, as it was not as far is it is today.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[cknitt]

Adopting rewatch as the new build system.