Ts-referent NPM

Typescript project-reference builder for monorepos focused on "cutting relations". To understand what "cutting relations" are please read One Thing Nobody Explained To You About TypeScript In short - this solution creates multiple tsconfigs for every package.

While other solutions are focused on Infering project references from common monorepo patterns / tools this one is trying to manage actually project references, not package.

😅🫠👨‍🔬 Let me be honest - project references gave me quite the miserable experience. Everything blew up and I still not sure am I happy or not...

official caveats can be found at typescript package references page
types are no longer "real time", as derived d.ts are used instead
- you have to update types as you go (see details below)
- affects only "other" projects, not the one you currently work with
types are not emitted in presence of any error - issue, another issue
- this is more a "feature" than a bug - only totally correct projects generates output
- this is not how you were able to "build" a package before
- it enforces you first to fix the package, then fix package consumers
  - by extracting tests and storybooks to a separate kinds you can restore the "old" behavior by minimizing " self-checks" in the package itself
typescript-eslint` does not support project references. You need to give some another config to it and not all things can work with "one config for all"
- known to break @typescript-eslint/no-unsafe-call and @typescript-eslint/no-unsafe-member-access
- see Support for Project References
build produces at least the same (at max double) of files you already had. That is a lot of files
- consider adding .referent directory containing all generated configs and typescript output files into .gitignore - really a recommendation, but this will delay "spin up" of repo in local or CI as everything has to be build first

API

Solution works for many package manager, but defined only for monorepos. Examples will be given using yarn

yarn add --dev ts-referent

CLI

Project references

ts-referent build - creates tsconfigs for every package in the monorepo
- ⚠️be sure to run this command on postinstall hook to keep tsconfig references and package.json dependencies in sync
ts-referent glossary tsconfig.packages.json - creates a "global" tsconfig referencing all packages in the monorepo
- 💡consider generating this file on demand. It also does not have to be committed. Only a "global type check" needs it.
- there are two available filters --filter-by-name and --filter-by-folder, both accepting globs to generate references not to "all" packages
  - you might need this command in rare situations when you refer to a file/project which is not a part of package dependencies. This might happen with some autogenerated "temporal" files in modular monolith.

Optional

ts-referent paths tscofig.paths.json - creates tsconfigs "aliases" you might want to extend your "base" one from, as it contains all links to all local packages and helps with auto-imports and other stuff.
- supports extra option --extends to configure configuration it should extend itself from. Extension is not required for TS5 as it supports multiple inheritance for configuration.

you might need configure entrypointResolver ONLY if you use this feature

Configuration

The most important moments

⚠️ your base tsconfig.json should explicitly have types:[] in compilerOptions. That will disable automated @types import and this is a feature you want.
⚠️ never put glossary into tsconfig.json, use tsconfig.projects.json. Otherwise, WebStorm TypeScript server will hang.
- tsc -b tsconfig.projects.json will build stuff for you
⚠️ keep include all your code in the top level tsconfig. Worry not - the nested tsconfig will override this setting, but "showing" your code to TypeScript will enable cross-package auto imports. Without it auto-import capability will be deeply limited
- issue, another issue
- expected to be improved in TS 5
⚠️regenerate references on postinstall hook to reflect changes in package.json
- 💩 that is not working for yarn (issue), you need to use a plugin (yarn 2+)
- automated npm and pnpm solutions under investigation...

IDE configuration recommendations

you need to constantly compile TS->JS or your changes will not be "reflected"
- 🫠 you actually dont need to do that since TS3.7, unless you have disableSourceOfProjectReferenceRedirect enabled, but there are examples when it's working only described mode, and actually that makes sence
- Importing modules from a referenced project will instead load its output declaration file (.d.ts)
  - declarations should be kept up to date
- for small projects you can use tsc -b --watch
  - for large projects that is not possible
- 👉 for WebStorm enable Recompile on changes in TypeScript settings. With Project References it will only speedup things.
- 👉 VSC should handle project references out of the box

Defining "cutting relations"

Different packages can be broken down into different "kinds". Think: sources, tests, cypress-tests:

sources are your main code. Only it can be referenced by other projects.
tests are internal to your code. Nobody can import them, no matter what.
cypress are given as an example, due to cypress definition clashing with jest ones, so you cannot have them both in "tests" slice and need to isolate from eachother.

Kinds

Kind is an include pattern and a slice of your code you can reference
- and every include pattern can specify exclude pattern as well
Every kind can specify
- which external definitions it should include
- which other kinds it should address
  - this enables configuration when source cannot import tests. Just cannot. For other restriction related configuration see eslint-plugin-relations

Specifying kinds

Kinds can be specified via tsconfig.referent.js file you can place at any folder affecting all packages "below".

note configuration file name - it is designed to blend with your tsconfig.json

Every such file can define 3 entities - extends, entrypointResolver and kinds

exports.baseConfig = "tsconfig you should extends from"
// only if you use them
exports.entrypointResolver = (packageJSON, dir) => [string, string][]

// the kinds
exports.kinds = {
    kindName: {
        includes: ['glob'],
        excludes: ['glob'],
        types: ['jest'],

    }
};

// or - a single entrypoint
/** @type {import('ts-referent').ConfigurationFile} */
module.exports/*: ConfigrationFile*/ = {
    baseConfig,
    entrypointResolver,
    kinds
}

//or

import {configure} from 'ts-referent';

export default configure({baseConfig, kinds});

Using ESM or Typescript as configuration

Just call ts-referent via ts-node, tsm, or others. Or the configuration file will not be found.

Depending on your package manager

node -r tsm ts-referent build > node -r tsm $(yarn bin ts-referent) build

import type {EntrypointResolver, Kinds} from "ts-referent";

export const baseConfig = "tsconfig you should extends from"

export const entrypointResolver: EntrypointResolver = (packageJSON, dir) => [string, string][]

// the kinds
export const kinds: Kinds = {
    kindName: {
        includes: ['glob'],
        excludes: ['glob'],
        types: ['jest'],
    }
};

import { configure } from 'ts-referent';

export default configure({
  baseConfig: require.resolve('tsconfig.json'),
  entrypointResolver: (packageJSON, dir) => [],
  kinds: {
    base: {
      include: ['**/*'],
    },
  },
});

Supporting `export` field

In order to support package export field one need to configure entrypointResolver

const pickExport = (entry) => {
  if (typeof entry === 'string') {
    return entry;
  }
  return entry['import'] || entry['require'];
}
export default configure({
  baseConfig: require.resolve('tsconfig.json'),
  // ⬇️
  entrypointResolver: (packageJSON, dir) => {
      if(!packageJSON.exports){
          // fallback to defaults (main field)
          return [];
      }
      return Object.entries(pkg.exports).map(([relativeName, pointsTo]) => {
        const name = relativeName.substring(2);
        // './entrypoint' -> `entrypoint` -> `/entrypoint`
        return [name ? `/${name}` : '', pickExport(pointsTo)]
      })
  },

As you might see - the following code will support only flat export map with minimal conditions. We do recommend using resolve.exports for anything more complex.

Publishing packages

In order to use ts-referent to publish packages considering creating two kinds for cjs and esm

If all packages are public

export default configure({
  ...,
  kids: {
      cjs: {
        include: ['**/*'],
        exclude: ['**/*.spec.*'], // dont forget to "forget" the tests
        compilerOptions: {
          target: 'es5',
          module: 'commonjs', // that's it
          verbatimModuleSyntax: false, // quite likely you need to disable it
        },
        outputDirectory: 'dist/cjs',
        focusOnDirectory:'src', // "focus" on src, so `dist` will not contain it
      },
      esm:{
        include: ['**/*'],
        exclude: ['**/*.spec.*'], // dont forget to "forget" the tests
        outputDirectory: 'dist/esm',
        focusOnDirectory:'src' // "focus" on src, so `dist` will not contain it
      },
  }
}

If some packages are public

The better idea would be to put them together in some directory and use alter

import { alter } from 'ts-referent';

export default alter((_, kinds) => ({
  base: {
    outputDirectory: 'dist/esm',
    focusOnDirectory: 'src',
  },
  'base-cjs': {
    // allow create new kinds
    expectExtension: true,
    //
    ...kinds['base'],
    outputDirectory: 'dist/cjs',
    focusOnDirectory: 'src',
    compilerOptions: {
      target: 'es5',
      module: 'commonjs',
    },
  },
}));

Configuring eslint

As long as project references are not directly supported by eslint there is only one, but good, way to handle this.

➡️ Use the same file names pattern matching you've used for kinds

In the eslintrc add an override section working only for a specific files

{
  rules: {
    //    someRules
  },
  overrides: [
    // kind 1
    {
      files: ['include-pattern'],
      excludedFiles: ['exclude-pattern'],
      parserOptions: {
        // ⚠️ tsconfig "repeating" kind configuration
        project: './tsconfig.kind.json',
      },
      // overrides of this sort are required ONLY for advanced typescript-eslint rules
      extends: ['plugin:@typescript-eslint/recommended-requiring-type-checking'],
    },
  ],
}

Right now you will have to create tsconfig.kind.json and keep in in sync with kinds configuration. We are working on automating this moment.

Advanced

Kinds configuration can be nested and also can be based on functions to derive new configuration from the previous one

note: configuration returned from a local function will be merged with the one above. In order to remove kind you need or 1) enable:false 2) or assign null 3) or use disableUnmatchedKinds from alter

Note that kinds can be defined using a function letting you specify different configuration for different packages. This is how you can slice and dice different kinds for different packages.

export default configure({
  // yes, that could be a function
  kinds: ({ base, ...otherKindsDefinedAbove }, currentPackage) => ({
    ...otherKindsDefinedAbove,
    base: {
      ...base,
      // "wire" externals defined in package json as "extra" references to a given package
      externals: currentPackage.packageJson.externals,
      types: [...(base.types || []), 'node'],
      exclude: ['**/*.spec.*'],
    },
    tests: {
      include: ['**/*.spec.*'],
      // tests can "access" base. base cannot access tests
      references: ['base'],
    },
  }),
});

Altering kinds

Just yesterday you were able to put whatever you need to any tsconfig you want. This is no longer possible. While it might sound as a good idea to preserve some settings from the original config, "slicing" everything into the pieces has to follow different logic

This is why, as you might see in the example above - in order to alter config you have to alter an applied kind. This is relatively rare operation, still worth a few handy tools.

// packages/some/package/tsconfig.referent.ts
import { alter } from 'ts-referent';

export default alter((currentPackage) => ({
  base: {
    // is equal to the implicit logic in the example above
    externals: currentPackage.packageJson.externals,
    types: ['node'],
    exclude: ['**/*.spec.*'],
  },
  tests: {
    include: ['**/*.spec.*'],
    // tests can "access" base. base cannot access tests
    references: ['base'],
  },
}));

The last example is a good demonstration of the essence of project references, the one from the official documentation.

alter accepts a second argument with options:

disableUnmatchedKinds - removes all unmodified kinds

export default alter(
  (currentPackage) => ({
    base: {
      // is equal to the implicit logic in the example above
      externals: currentPackage.packageJson.externals,
      types: ['node'],
      exclude: ['**/*.spec.*'],
    },
    tests: {}, // just keep tests
  }),
  {
    // only base and test will be predefined for folders below
    disableUnmatchedKinds: true,
  }
);

Type augmentation

In some cases you might need to work with non standard package.jsons, still willing to be typesafe. Note in the example above the extra field externals which is not a part of package.json standard. There could be many more fields you might find useful - entrypoint, client/server/workers, dev/prod - to affect available types and relations. To make them "visible" and "accepted" by ts-referent one can use typescript declaration merging

declare module 'ts-referent' {
    interface PackageJSON {
        // "extend" by a new field
        externals?: ReadonlyArray<string>;
    }
}
export default alter((currentPackage) => ({
    base: {
        // the new field is now a part of packageJson
        externals: currentPackage.packageJson.externals,
    }
});

Note on declaration merging and project references

Project references do affect module augmentation due the way d.ts is generated from the source files. If augmentation is no longer working for you please check related issue and (long story short) write d.ts manually.

Isolation

Project references works in two different ways:

for top level builds and checks, where you can reference different projects to compile
from bottom level where IDE is trying to find the nearest matching tsconfig to use

By default, both variants are supported, but this lead to suboptimal situation when package's tsconfig references all used kinds, meaning that any other package referencing a given one also references not only it's source, but also it tests and all other really not public pieces.

There are 2 ways to improve this moment, however there is no proof that this improvement anything (like speed) except the purity of the solution.

per kind: isolatedInDirectory - will put kind's configuration inside a nested directory - cypress, __tests__, examples - making it really a "local" configuration
- other kinds of the same package can still access it by specifying references at own kind configuration
- to reference hidden directory from workspace one can use relationMapper at own kind configuration. Do that on your own risk
- kind will not be created if if directory does not exists
global: isolatedMode at the root tsconfig.referent.js flag will activate package isolation mode
- every package will produce two configs - tsconfig.json for the IDE and tsconfig.public.json for external references
- internal per kind configuration property will remove kind from the public interface, thus separating private sources(tests) with the "real" ones(source)
  - there is no way to reference internal kind from the workspace, only isolatedInDirectory one
  - isolatedInDirectory are "private by default", but can became public via internal=false setting