Add local file path & Raw HTML string support.

[timothycarambat]

Added support for loading of local HTML files for parsing and LLM generation.

Requires new local HTML parser node-html-parser

resolves #15
resolves #10

[luminous8]

🙏 man you're fast!

Quick question, is it possible with your update to pass raw html not stored in fs
Doing something like

const response = await fetch('https://news.ycombinator.com/');
const rawHtml = await response.json();

// Run the scraper
const pages = await scraper.runFiles([rawHtml], {
  model: 'gpt-4-turbo',
  schema,
  mode: 'html',
  closeOnFinish: true,
})

instead of

// Grab today's HN front page to run the example
await fetch('https://news.ycombinator.com/')
  .then((res) => res.text())
  .then((html) => writeFileSync(HNRawHtmlPath, html, { encoding: 'utf-8', flag: 'w' }))
  .catch((e) => {
    console.error("Failed to fetch content from Hackernews", e)
  })

// Local file paths to scrape - will be loaded from local filepaths.
const filePaths = [HNRawHtmlPath]

// Run the scraper
const pages = await scraper.runFiles(filePaths, {
  model: 'gpt-4-turbo',
  schema,
  mode: 'html',
  closeOnFinish: true,
})

Thanks

[timothycarambat]

@luminous8 - added. IDK if maintainer will merge this, but ed45843 accomplishes that

[luminous8]

Awesome! 🤞

[mishushakov]

Hey, thank you so much for your contribution 🙏
Unfortunately, I cannot merge this at a moment as I'm planning making the library independent of Playwright.

For bigger contributions like this, we should discuss the implementation first (in an issue) and then proceed to pull request
Still appreciate and very thankful for all the effort you put into making this 😄