fix: Allow ignore patterns to apply to absolute path instead of relative path #656

kingston · 2024-09-24T20:34:02Z

Upgrade chokidar to v4 for a smaller footprint
Add picomatch for glob matching
Add test for parent directory ignores

Addresses #221

…ive path + Upgrade chokidar to v4 for a smaller footprint + Add picomatch for glob matching + Add test for parent directory ignores

kingston · 2024-09-24T20:53:17Z

@privatenumber just kicking this off as an initial draft PR so we can comment on raw code vs. ideas. Right now all tests pass except one (--include with glob patterns). This is because chokidar v4 removes all glob support including include patterns. This is a bit trickier to resolve since we'd need to find all the files that match that glob which gets a lot messier/more complex. We can either stay with chokidar v3 or do something a bit more complex. Looking at chokidar v3, it seems like when they encounter a glob pattern they just take the glob-parent and watch the highest parent of the glob e.g. for /blah/foo/**/test - they watch /blah/foo and filter by **/test. So we can try to mimic this behavior manually.

We could potentially do two separate watchers:

Executed file + dependencies watcher (basically what we have today with this PR)
If --include is present, includes-only watcher. Watch all glob-parents and apply --include globs to filter out only the files we care about. (would coincidentally also address fix: Drop default excludes that are exist in includes #643)

I don't think it adds too much of an overhead and simplifies the logic.

On a side note, I realized the docs were out of date with --exclude / --include (https://github.com/privatenumber/tsx/blob/master/docs/watch-mode.md) but can be a separate issue to resolve.

privatenumber · 2024-09-25T06:01:19Z

That sounds good to me! Thanks @kingston

Will try to update the docs soon.

kingston · 2024-09-25T10:41:45Z

@privatenumber I've updated the PR to now pass all tests including the include tests. I introduced 2 additional libraries to get the glob parent and check if it is a glob or not (otherwise glob parent would remove the ending).

Overall, I think it should be good to go. However, there's one scenario where I'm unclear what the behavior should be.

If you pass a directory to --exclude without any glob patterns, should it automatically match all its children? e.g. --exclude ./foo will match ./foo/? If so, we'd probably need to re-work the logic to create a separate matcher for the exact path and path with / appended to it for non-glob patterns.

Also left a small update the docs to make it more accurate.

…erences

kingston · 2024-09-26T22:40:56Z

@privatenumber I don't have access to a windows instance but just uploaded a fix that should address the failing test.

kingston · 2024-09-27T12:31:12Z

Hmm.. that didn't seem to work. I'll try to see if I can get a windows machine to test it on since I have a Mac.

privatenumber · 2024-09-27T12:49:52Z

I actually don't have a Windows machine either. It's not the most ergonomic environment but I just make a separate branch, update the GitHub Actions CI to only run on Windows, and debug there 😅

I'll have more time this weekend so I can take a closer look/help out.

…o UNIX paths

kingston · 2024-09-30T11:20:20Z

@privatenumber OK, I spun up a Windows VM on my Mac and was able to debug more ergonomically :). It looks like globs and Windows paths are complicated since globs are technically not allowed to have \ in them so I used a library normalize-paths (used by chokidar v3 as well) to normalize the paths to / since chokidar v4 also does that when passing to ignored. This makes all tests pass on my Windows 11 VM :).

Fun side-note: I was unable to commit to the repo because of the lint pre-commit hook since the lint command doesn't work on Windows so I had to skip the hooks when committing.

C:\...\tsx\docs\getting-started.md
  62:14  error  Parsing error: Unexpected token :

C:\...\tsx\docs\typescript.md
  108:14  error  Parsing error: Unexpected token :
  123:14  error  Parsing error: Unexpected token :

My guess is that the glob pattern for ignoring files also doesn't work on Windows :P

lintroll --node --cache --ignore-pattern 'docs/*.md' .

kingston · 2024-10-09T21:39:07Z

@privatenumber just wanted to follow up - are there any other action items for this PR?

privatenumber · 2024-10-10T07:47:14Z

Sorry @kingston been swamped lately. I'll try to take a look Sunday or next week.

kingston · 2024-10-10T14:41:14Z

Sure no problem - I get it.

privatenumber · 2024-10-16T03:47:25Z

src/watch/index.ts

+					|| isOptionsExclude(file)
+				)
+			),
+		}).on('all', reRun);


Is it possible to add to the same watcher above instead of making a new one?

I created a separate one since the logic of the ignore is different for explicit includes compared to default includes (we want to include things that might be by default excluded e.g. node_modules) and I think under the hood the chokidar would just create separate watchers for the different glob parents resulting in no performance impact.

privatenumber · 2024-10-16T03:53:14Z

src/watch/utils.ts

+		return normalizePath(pattern);
+	}
+	return normalizePath(path.join(process.cwd(), pattern));
+};


This only seems used to normalize the path slashes in user provided paths—do they need need to be normalized?

Without normalization, Windows style separators break the glob logic. Normalizing the paths make all our paths we use consistent and easier to handle (e.g. we can match our globs based on Unix-style separators instead of having to figure out which it is)

privatenumber · 2024-10-16T03:58:09Z

src/watch/index.ts

 		},
 	).on('all', reRun);

+	if (resolvedIncludes.length > 0) {
+		const globParents = resolvedIncludes.map(pattern => (
+			isGlob(pattern)


This seems like a pretty hefty function to run only to check if it's a glob. What if we just run globParent on all inputs regardless of whether they're a glob?

I tried that but a simple include like test/foo/hello becomes test/foo when run through glob-parent. We could proactively append a separator at the end but that may result in weirder behavior potentially. Coincidentally glob-parent uses isGlob under the hood.

privatenumber · 2024-10-16T03:59:03Z

src/watch/index.ts

@@ -4,7 +4,10 @@ import { constants as osConstants } from 'node:os';
 import path from 'node:path';
 import { command } from 'cleye';
 import { watch } from 'chokidar';
+import picomatch from 'picomatch';


Picomatch is actually the largest dependency now (bundle size):
https://pkg-size.dev/chokidar%20glob-parent%20is-glob%20normalize-path%20picomatch

Is there a lighter alternative?

Hmm. this is the smallest library I have for globbing that I can find. Micromatch depends on picomatch and minimatch is about 5 times as large as picomatch.

…gnore-absolute-patterns

privatenumber · 2024-10-16T04:22:13Z

I pulled out your docs changes into master so we can get it out faster. Thanks!

kingston

Sorry for the delay - had a bit of a busy past month.

kingston · 2024-11-13T11:55:51Z

src/watch/index.ts

@@ -4,7 +4,10 @@ import { constants as osConstants } from 'node:os';
 import path from 'node:path';
 import { command } from 'cleye';
 import { watch } from 'chokidar';
+import picomatch from 'picomatch';


Hmm. this is the smallest library I have for globbing that I can find. Micromatch depends on picomatch and minimatch is about 5 times as large as picomatch.

kingston · 2024-11-13T12:24:04Z

src/watch/index.ts

 		},
 	).on('all', reRun);

+	if (resolvedIncludes.length > 0) {
+		const globParents = resolvedIncludes.map(pattern => (
+			isGlob(pattern)


I tried that but a simple include like test/foo/hello becomes test/foo when run through glob-parent. We could proactively append a separator at the end but that may result in weirder behavior potentially. Coincidentally glob-parent uses isGlob under the hood.

kingston · 2024-11-13T12:26:21Z

src/watch/index.ts

+					|| isOptionsExclude(file)
+				)
+			),
+		}).on('all', reRun);


I created a separate one since the logic of the ignore is different for explicit includes compared to default includes (we want to include things that might be by default excluded e.g. node_modules) and I think under the hood the chokidar would just create separate watchers for the different glob parents resulting in no performance impact.

kingston · 2024-11-13T12:28:26Z

src/watch/utils.ts

+		return normalizePath(pattern);
+	}
+	return normalizePath(path.join(process.cwd(), pattern));
+};


Without normalization, Windows style separators break the glob logic. Normalizing the paths make all our paths we use consistent and easier to handle (e.g. we can match our globs based on Unix-style separators instead of having to figure out which it is)

fix: Allow ignore patterns to apply to absolute path instead of relat…

ba6a3a0

…ive path + Upgrade chokidar to v4 for a smaller footprint + Add picomatch for glob matching + Add test for parent directory ignores

vercel bot deployed to Preview September 24, 2024 20:34 View deployment

kingston marked this pull request as draft September 24, 2024 20:34

kingston mentioned this pull request Sep 24, 2024

TSX is not ignoring symlinked dependencies in monorepo #221

Open

1 task

Create separate watcher for include patterns to support glob includes

29f53d4

vercel bot deployed to Preview September 25, 2024 10:29 View deployment

Update documentation

9112e93

kingston marked this pull request as ready for review September 25, 2024 10:33

vercel bot deployed to Preview September 25, 2024 10:33 View deployment

Use picomatch for matching glob parent to prevent cross-platform diff…

910604a

…erences

vercel bot deployed to Preview September 26, 2024 22:40 View deployment

Fix Windows compatability by using normalize-path to normalize path t…

70b9d2e

…o UNIX paths

vercel bot deployed to Preview September 30, 2024 11:11 View deployment

Merge branch 'master' into kingston/ignore-absolute-patterns

7e2e2c3

vercel bot deployed to Preview October 1, 2024 16:18 View deployment

privatenumber added 3 commits October 16, 2024 10:23

wip

2470fca

Merge branch 'master' into kingston/ignore-absolute-patterns

bcb9d6b

Merge branch 'master' into kingston/ignore-absolute-patterns

950e55d

vercel bot deployed to Preview October 16, 2024 03:45 View deployment

privatenumber reviewed Oct 16, 2024

View reviewed changes

wip

5bb6d3e

vercel bot deployed to Preview October 16, 2024 04:09 View deployment

Merge branch 'master' of github.com:privatenumber/tsx into kingston/i…

2d3dc2e

…gnore-absolute-patterns

vercel bot deployed to Preview October 16, 2024 04:22 View deployment

kingston commented Nov 13, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: Allow ignore patterns to apply to absolute path instead of relative path #656

fix: Allow ignore patterns to apply to absolute path instead of relative path #656

kingston commented Sep 24, 2024 •

edited

Loading

kingston commented Sep 24, 2024

privatenumber commented Sep 25, 2024 •

edited

Loading

kingston commented Sep 25, 2024

kingston commented Sep 26, 2024

kingston commented Sep 27, 2024

privatenumber commented Sep 27, 2024

kingston commented Sep 30, 2024

kingston commented Oct 9, 2024

privatenumber commented Oct 10, 2024

kingston commented Oct 10, 2024

privatenumber Oct 16, 2024

kingston Nov 13, 2024

privatenumber Oct 16, 2024

kingston Nov 13, 2024

privatenumber Oct 16, 2024

kingston Nov 13, 2024

privatenumber Oct 16, 2024 •

edited

Loading

kingston Nov 13, 2024

privatenumber commented Oct 16, 2024

kingston left a comment

kingston Nov 13, 2024

kingston Nov 13, 2024

kingston Nov 13, 2024

kingston Nov 13, 2024

fix: Allow ignore patterns to apply to absolute path instead of relative path #656

Are you sure you want to change the base?

fix: Allow ignore patterns to apply to absolute path instead of relative path #656

Conversation

kingston commented Sep 24, 2024 • edited Loading

kingston commented Sep 24, 2024

privatenumber commented Sep 25, 2024 • edited Loading

kingston commented Sep 25, 2024

kingston commented Sep 26, 2024

kingston commented Sep 27, 2024

privatenumber commented Sep 27, 2024

kingston commented Sep 30, 2024

kingston commented Oct 9, 2024

privatenumber commented Oct 10, 2024

kingston commented Oct 10, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

privatenumber Oct 16, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

privatenumber commented Oct 16, 2024

kingston left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kingston commented Sep 24, 2024 •

edited

Loading

privatenumber commented Sep 25, 2024 •

edited

Loading

privatenumber Oct 16, 2024 •

edited

Loading