Complex pages are slow to start

[mounaiban]

Second Edition
NOTE: This is a completely re-written summary of the issue. Please check the post history under the "Edited" menu to review the original.

When printing a multi-page job, captdriver pauses before moving on to the next. This has been a known issue and has been tolerated on simple jobs. However, when complex pages that contain a lot of graphical details and gradient fills are involved, delays can become bothersome.

The printer can stall for upwards of a minute in such cases, making it feel like the driver has crashed. When printing multiple copies of the same page, the delays will happen multiple times.

Reproducing the problem

1. Prepare the test PDF as described in this test procedure from the studycapt repository.

   • You should end up with a three-page PDF.

   • The first and last pages contain simple solid-filled shapes, while the second page should filled with many small radial gradient-filled ellipses, and render more slowly.

2. Print the document with a PDF viewer.

   • If you are using Mozilla Firefox's built-in PDF viewer, pdf.js, you may get blurred pages if you attempt to print before the pages are ready.

The first page should print normally, but the second page stalls the job. The job should then complete within a few minutes.

Potential solutions
The following actions have been suggested, in order of the easiest to the hardest:

• Cache all compressed pages before the job starts

  • The biggest gains in speed are expected on jobs that involve printing multiple copies of complex pages.

  • Memory use per job will increase

• Discover and enable lower resolution modes. Currently, only 600dpi mode is supported.

  • Preliminary testing with GhostScript suggests that 300dpi saves about half the page rendering time compared to 600dpi
  • With captdriver, CUPS uses GhostScript to render pages, which seems to have linear time complexity with respect to raster pixel count

• Write a GhostScript device driver that compresses the raster during instead of after the rendering process.

[k25neo]

LBP 3010 very slow to start. to print 1 page of a pdf document, it takes a long time before starting. choosing in the settings from 600 dpi to 400 dpi does not solve the problem.

[mounaiban]

From what I know, we're using a very inefficient pipeline:
📄 PDF ➡️ CUPS raw raster ➡️ HiSCoA 🖨️

A more efficient pipeline would be:
📄 PDF ➡️ HiSCoA 🖨️

But bypassing the slow CUPS raw raster stage requires writing a GhostScript device from what I know. This allows rendering PDFs straight to HiSCoA. GhostScript devices for HiSCoA and SCoA are part of the plan, but it is going to take a while for me. The GhostScript API isn't looking easy to understand to a beginner... 😅

[mounaiban]

I should have mentioned this earlier: I created a branch of the driver that uses a binary search to find the bit length of integers in src/hiscoa-compress.c, 0.1.4.1-bisect-find-msb. The master branch uses a slower linear search method.

Pardon my mixing of tabs and spaces.

Using bisecting bit length measurement did not speed up printing for me, despite making the compression technically faster, leading me to the conviction that the rasterisation is the tightest bottleneck. Maybe I was wrong; you can test it if it is faster for you.

Finding the bit length of a number is a central part of HiSCoA compression as it uses a custom form of Elias coding for small numbers.

[saper]

I tried this with 6271924 and I didn't notice it being VERY slow, just "acceptable slow". GhostScript did run for some time at 99% CPU but it went relatively fast with some short breaks between pages.

What wonders me it managed to print page 1 and then started to print many copies of page 2 (interrupted it after page 9). Also page 1 has margins truncated, and page 2 has some overlapping ball muster at the top. Do we respect printer margins? How come do I get so many copies of page 2?

edit the logfile says

E [25/Dec/2022:23:43:39 +0100] [Job 400] CAPT: bad reply from printer, expected A2 E1 xx xx xx xx, got A8 A0 58 00 02 8E

This might be the problem with page 2

[mounaiban]

@saper Host OS/Host device/Printer?

Raspberry Pi OS on an RPi is known to be a problem target, normally the pages don't even print, either due to the commands not making it to the printer or the printer's response falling on libusb (0.1)'s deaf ears.

If not, I hope this isn't a new regression caused by recent changes in libusb and CUPS, or differences between GNU and BSD libraries... I'll investigate when I get the chance.

I haven't had page issues on LBP3000 hooked up onto an Ubuntu 21 system, aside from inaccurate content size and positioning. On my low-end PC, the second page in the test procedure has taken well over a minute to render.

EDIT: just read your pull requests, thanks for testing on FreeBSD

[saper]

Host OS/Host device/Printer?

FreeBSD on an ancient Sony Vaio laptop (amd64) with LBP3100

I have observed the USB timeout issue as well later (but not reliably reproducible). usbdump (FreeBSD's own USB tracing utility) shows that we are getting USB stall errors.

FreeBSD uses its own USB stack (originally from NetBSD) and offers libusb-compatible userland interface, but the implementation not libusb as known in the Linux world.

Some random things: Disconnecting the USB interface for a moment usually helps to recover. Maybe the stalls are caused by the printer - like some memory overflow, too much data sent maybe? and have nothing to do with the local USB stack.

[mounaiban]

@saper From what I know, the USB-related problems we have aren't the fault of libusb or any USB stacks, but the exact way CUPS uses it to implement the back and side channel functions like cupsBackChannelRead(), cupsSideChannelDoRequest(), and so on.

We're not the only ones affected by CUPS' backend issues, the cups-carps driver is reportedly affected as well, just not as badly as our driver.