Add an experimental tools-typescript package for better react-native …

…builds with typescript (#3503)

* create tools-typescript experimental package

* copy over metro-plugin-typescript code to use as a base

* start moving adapting FURN prototype code to rnx-kit

* add sanitization of ts inputs

* finish file porting and adaptation

* fix platform multiplexing and add tests

* add tracing support and some basic hookup to builds

* more in-flight changes

* clean up testing only changes

* more tool cleanup

* fix some issues with platform build splitting

* remove bogus test files

* fix test for batch writer

* more platform fixes

* code cleanup for module resolution issues

* fix builds for react-native platforms

* update readme, and add automic generation

* docs(changeset): Initial creation of an experimental typescript build tool

* remove unused dependency in tools-typescript

* switch from classes to interfaces, fix documentation

* export project creation, update readme file

* respond to code review feedback

* Apply suggestions from code review

Apply some of the PR feedback directly

Co-authored-by: Tommy Nguyen <4123478+tido64@users.noreply.github.com>

* apply some code review feedback

* switch allSuffixes to Sets

* clean up error handling in file writer

* respond to PR feedback

* use @rnx-kit/console logging helpers

* remove .d.tsx as a parsing option

* remove extra comment and leverage identity function

* get rid of promise queue in BatchWriter, also add robustness for restarts

* avoid unnecessary closures

---------

Co-authored-by: Tommy Nguyen <4123478+tido64@users.noreply.github.com>

.changeset/hot-bobcats-play.md

@@ -0,0 +1,5 @@
+---
+"@rnx-kit/tools-typescript": minor
+---
+
+Initial creation of an experimental typescript build tool

incubator/tools-typescript/README.md

@@ -0,0 +1,79 @@
+# @rnx-kit/tools-typescript
+
+[![Build](https://github.com/microsoft/rnx-kit/actions/workflows/build.yml/badge.svg)](https://github.com/microsoft/rnx-kit/actions/workflows/build.yml)
+[![npm version](https://img.shields.io/npm/v/@rnx-kit/tools-typescript)](https://www.npmjs.com/package/@rnx-kit/tools-typescript)
+
+🚧🚧🚧🚧🚧🚧🚧🚧🚧🚧🚧
+
+### THIS TOOL IS EXPERIMENTAL — USE WITH CAUTION
+
+🚧🚧🚧🚧🚧🚧🚧🚧🚧🚧🚧
+
+A package that helps with building typescript, particularly for react-native. It
+leverages the @rnx-kit/typescript-service package to build using the language
+service rather than using the compiler directly. This allows for custom
+resolution strategies (among other things). The compilation and type-checking
+are still done by typescript but this drives some convenient custom
+configurations.
+
+In particular the `buildTypeScript` command can do things like:
+
+- multiplex a build in a directory, to build for multiple react-native platforms
+  at the same time. The files will be split such that the minimal build can be
+  performed.
+- detect which react-native platforms should be built based on rnx-kit bundle
+  configs or react-native.config.js settings.
+- successfully build with the module suffixes without throwing up a ton of bogus
+  unresolved reference errors.
+- share caches where possible to speed up compilations within the same node
+  process.
+
+## Motivation
+
+The current story for building typescript for react-native is sub-par, and
+typescript itself is particularly restrictive with module resolution. This
+addresses the ability to build react-native better right now, but also creates a
+framework for experimenting with different resolvers.
+
+## Things to do
+
+- Look at watch mode behavior, add invalidation for caching layers
+- Look at routing host creation in metro-plugin-typescript through here
+- Evaluate alternative resolvers
+
+## Installation
+
+```sh
+yarn add @rnx-kit/tools-typescript --dev
+```
+
+or if you're using npm
+
+```sh
+npm add --save-dev @rnx-kit/tools-typescript
+```
+
+## Usage
+
+<!-- The following table can be updated by running `yarn update-readme` -->
+<!-- @rnx-kit/api start -->
+
+| Category | Type Name      | Description                                                                                                                                                                                                                |
+| -------- | -------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| types    | AsyncThrottler | Interface for capping the max number of async operations that happen at any given time. This allows for multiple AsyncWriter instances to be used in parallel while still limiting the max number of concurrent operations |
+| types    | AsyncWriter    | Interface for handling async file writing. There is a synchronous write function then a finish function that will wait for all writes to complete                                                                          |
+| types    | BuildOptions   | Options that control how the buildTypeScript command should be configured                                                                                                                                                  |
+| types    | PlatformInfo   | Information about each available platform                                                                                                                                                                                  |
+| types    | Reporter       | Interface for reporting logging and timing information                                                                                                                                                                     |
+
+| Category  | Function                                                   | Description                                                                                                                                                                                                                            |
+| --------- | ---------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| build     | `buildTypeScript(options)`                                 | Execute a build (or just typechecking) for the given package. This can be configured with standard typescript options, but can also be directed to split builds to build and type-check multiple platforms as efficiently as possible. |
+| files     | `createAsyncThrottler(maxActive, rebalanceAt)`             | Creates an AsyncThrottler that can be used to limit the number of concurrent operations that happen at any given time.                                                                                                                 |
+| files     | `createAsyncWriter(root, throttler, reporter)`             | Create an AsyncWriter that can be used to write files asynchronously and wait on the results                                                                                                                                           |
+| host      | `openProject(context)`                                     | Open a typescript project for the given context. Can handle react-native specific projects and will share cache information where possible given the configuration                                                                     |
+| platforms | `loadPkgPlatformInfo(pkgRoot, manifest, platformOverride)` | Load platform info for available platforms for a given package                                                                                                                                                                         |
+| platforms | `parseSourceFileDetails(file, ignoreSuffix)`               | Take a file path and return the base file name, the platform extension if one exists, and the file extension.                                                                                                                          |
+| reporter  | `createReporter(name, logging, tracing)`                   | Create a reporter that can log, time, and report errors                                                                                                                                                                                |
+
+<!-- @rnx-kit/api end -->

incubator/tools-typescript/eslint.config.js

@@ -0,0 +1 @@
+module.exports = require("@rnx-kit/eslint-config");

incubator/tools-typescript/package.json

@@ -0,0 +1,55 @@
+{
+  "name": "@rnx-kit/tools-typescript",
+  "version": "0.0.1",
+  "description": "EXPERIMENTAL - USE WITH CAUTION - tools-typescript",
+  "homepage": "https://github.com/microsoft/rnx-kit/tree/main/incubator/tools-typescript#readme",
+  "license": "MIT",
+  "author": {
+    "name": "Microsoft Open Source",
+    "email": "microsoftopensource@users.noreply.github.com"
+  },
+  "files": [
+    "lib/**/*.d.ts",
+    "lib/**/*.js"
+  ],
+  "main": "lib/index.js",
+  "types": "lib/index.d.ts",
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/microsoft/rnx-kit",
+    "directory": "incubator/tools-typescript"
+  },
+  "engines": {
+    "node": ">=16.17"
+  },
+  "scripts": {
+    "build": "rnx-kit-scripts build",
+    "format": "rnx-kit-scripts format",
+    "lint": "rnx-kit-scripts lint",
+    "test": "rnx-kit-scripts test"
+  },
+  "dependencies": {
+    "@rnx-kit/config": "^0.7.0",
+    "@rnx-kit/console": "^2.0.0",
+    "@rnx-kit/tools-node": "^3.0.0",
+    "@rnx-kit/tools-react-native": "^2.0.0",
+    "@rnx-kit/typescript-service": "^2.0.0"
+  },
+  "devDependencies": {
+    "@rnx-kit/eslint-config": "*",
+    "@rnx-kit/jest-preset": "*",
+    "@rnx-kit/scripts": "*",
+    "@rnx-kit/tsconfig": "*",
+    "eslint": "^9.0.0",
+    "jest": "^29.2.1",
+    "prettier": "^3.0.0",
+    "typescript": "^5.0.0"
+  },
+  "peerDependencies": {
+    "typescript": ">=4.7.0"
+  },
+  "jest": {
+    "preset": "@rnx-kit/jest-preset/private"
+  },
+  "experimental": true
+}

incubator/tools-typescript/src/build.ts

@@ -0,0 +1,111 @@
+import { findPackage, readPackage } from "@rnx-kit/tools-node";
+import type { AllPlatforms } from "@rnx-kit/tools-react-native";
+import { findConfigFile, readConfigFile } from "@rnx-kit/typescript-service";
+import path from "node:path";
+import ts from "typescript";
+import { loadPackagePlatformInfo } from "./platforms";
+import { createReporter } from "./reporter";
+import { createBuildTasks } from "./task";
+import type { BuildContext, BuildOptions, PlatformInfo } from "./types";
+
+/**
+ * Load the tsconfig.json file for the package
+ * @param pkgRoot the root directory of the package
+ * @param args the command line arguments to be passed to typescript
+ * @returns the parsed tsconfig.json file, if found
+ */
+function loadTypeScriptConfig(
+  pkgRoot: string,
+  options: ts.CompilerOptions = {}
+): ts.ParsedCommandLine {
+  // find the tsconfig.json, overriding with project if it is set
+  const configPath =
+    options.project ?? findConfigFile(pkgRoot, "tsconfig.json");
+
+  if (!configPath) {
+    throw new Error("Unable to find tsconfig.json");
+  }
+
+  // now load the config, mixing in the command line options
+  const config = readConfigFile(configPath, options);
+
+  if (!config) {
+    throw new Error(`Unable to parse 'tsconfig.json': ${pkgRoot}`);
+  }
+  return config;
+}
+
+/**
+ * Execute a build (or just typechecking) for the given package. This can be configured
+ * with standard typescript options, but can also be directed to split builds to build
+ * and type-check multiple platforms as efficiently as possible.
+ *
+ * @param options - options for the build
+ */
+export async function buildTypeScript(options: BuildOptions) {
+  // load the base package json
+  const pkgJsonPath = findPackage(options.target);
+  if (!pkgJsonPath) {
+    throw new Error("Unable to find package.json for " + options.target);
+  }
+  const manifest = readPackage(pkgJsonPath);
+  const root = path.dirname(pkgJsonPath);
+  const reporter = createReporter(
+    manifest.name,
+    options.verbose,
+    options.trace
+  );
+
+  // set up the typescript options and load the config file
+  const mergedOptions = {
+    ...options.options,
+    ...ts.parseCommandLine(options.args || []).options,
+  };
+  const cmdLine = loadTypeScriptConfig(root, mergedOptions);
+
+  // load/detect the platforms
+  let targetPlatforms: PlatformInfo[] | undefined = undefined;
+  if (options.platforms || options.reactNative) {
+    const platformInfo = loadPackagePlatformInfo(
+      root,
+      manifest,
+      options.platforms
+    );
+    const platforms = Object.keys(platformInfo);
+    if (platforms.length > 0) {
+      options.platforms = platforms as AllPlatforms[];
+      targetPlatforms = platforms.map((name) => platformInfo[name]);
+    }
+  }
+
+  if (options.verbose) {
+    const module = cmdLine.options.module;
+    const moduleStr =
+      module === ts.ModuleKind.CommonJS
+        ? "cjs"
+        : module === ts.ModuleKind.ESNext
+          ? "esm"
+          : "none";
+    const output = cmdLine.options.noEmit ? "noEmit" : cmdLine.options.outDir;
+    const platforms = options.platforms
+      ? ` [${options.platforms.join(", ")}]`
+      : "";
+    reporter.log(`Starting build (${moduleStr} -> ${output})${platforms}`);
+  }
+
+  // turn the parsed command line and options into a build context
+  const context: BuildContext = {
+    cmdLine,
+    root,
+    reporter,
+    build: [],
+    check: [],
+  };
+
+  // create the set of tasks to run then resolve all the tasks
+  try {
+    await Promise.all(createBuildTasks(options, context, targetPlatforms));
+  } catch (e) {
+    throw new Error(`${manifest.name}: Build failed. ${e}`);
+  }
+}