Warn the user if biber needs to be invoked

[pkgw]

According to a comment in #35, if you process a biblatex/biber document with Tectonic, you get something with unprocessed references but no messages indicating that there's been a problem. Solving #35 will take a while, but in the meantime, we should add some hacks to detect documents that require biber and warn the user that we can't do it.

I confess that I am not fully clear as to whether the standard practice is that the user or driver manually executes biber after the first TeX engine run, or whether some shell-escape magic can be enabled that makes it happen automatically during processing.

[teras]

So, to understand, is there a way to support biber, by manually launching it, or something like that?

Thanks

[pkgw]

@teras I'm actually not sure! The following workflow is definitely possible, if inconvenient:

• Run tectonic once with the right command line arguments to generate the .aux file and/or whatever other files are needed (e.g. --outfmt=aux)
• Run biber with whatever args it needs
• Run tectonic again to finish off the compilation.

However, depending on what files are needed, what arguments biber needs, and how biblatex works, the above pattern might not suffice to get things to work 😢

This has come up often as a limitation for prospective Tectonic users, so if you get a chance to investigate this, I would love to hear what you find.

[teras]

I tried running the usual way biber, and I found that it needs PROJECT.bcf file.
How can I say to tectonic to delete all but this file?

Anyway, I created this file through xelatex, delete everything, run tectonic --outfmt=aux, run biber, and run tectonic once more. (Note, the first time I tried to run tectonic it tried 6 times to rebuild the file, and aborted with the message
warning: TeX rerun seems needed, but stopping at 6 passes

This is what I got:

... MORE JUNK ...


*************************************************
* fontspec warning: "icu-feature-not-exist-in-font"
* 
* OpenType feature 'Variant=01' (+ss01) not available for font 'Arial/BI' with
* script '' and language ''.
*************************************************
(marginnote.sty) (sectsty.sty) (ulem.sty) (textpos.sty
Package: textpos 2014/01/03 1.7j, absolute positioning of text on the page
(everyshi.sty)
Grid set 16 x 16 = 37.34424pt x 52.81541pt
TextBlockOrigin set to 0pt x 0pt
) (hyperref.sty (hobsub-hyperref.sty (hobsub-generic.sty)) (auxhook.sty)
(kvoptions.sty) (pd1enc.def) (hyperref.cfg)

Package hyperref Warning: XeTeX driver only supports unicode.
(hyperref)                Enabling option `unicode'.

(puenc.def) (url.sty))

Package hyperref Message: Driver (autodetected): hxetex.

(hxetex.def (stringenc.sty) (rerunfilecheck.sty)) (marvosym.sty) (url.tex)
(lang.tex (greek.tex)) (biblatex.sty (iftex.sty) (etoolbox.sty) (logreq.sty
(logreq.def)) (ifthen.sty) (xstring.sty (xstring.tex)) (blx-dm.def)
(blx-compat.def) (biblatex.def) (alphabetic.bbx (standard.bbx)) (alphabetic.cbx
) (biblatex.cfg)) (sections.tex) (english.lbx) (test_file.aux) (t3cmr.fd)
*geometry* driver: auto-detecting
*geometry* detected driver: xetex
ABD: EveryShipout initializing macros (nameref.sty (gettitlestring.sty))

Package biblatex Warning: File 'test_file.bbl' is wrong format version - e
xpected 2.7.

(test_file.bbl
! Undefined control sequence.
l.21   \datalist
                [entry]{ydnt/global//global/global}
No pages of output.
Transcript written on test_file.log.
===============================================================================

error: halted on potentially-recoverable error as specified

[malramsay64]

The version of biblatex included with tectonic requires biber 2.5 to work which is downloadable from sourceforge

Running the commands

$ tectonic --keep-intermediates --reruns 0 file.tex
$ biber2.5 file
$ tectonic file.tex

When using an output directory, tectonic looks for the .bbl file in the current working directory so the commands needed are

$ tectonic --keep-intermediates --reruns 0 -o output file.tex
$ biber2.5 --input-directory output file
$ tectonic -o output file.tex

Edit:

The latest version of tectonic 0.1.10 now requires biber 2.11, which at the time of writing is the latest version meaning it shouldn't require a special installation. This current version is also downloadable from sourceforge or using conda

conda install malramsay::biber

[hugobuddel]

The latest version of tectonic 0.1.10 now requires biber 2.11, which at the time of writing is the latest version meaning it shouldn't require a special installation. This current version is also downloadable from sourceforge or using conda

conda install malramsay::biber

That biber is 2.12 now, which results in this error for tectonic 0.1.15 (the latest as of this writing):

ERROR - Error: Found biblatex control file version 3.4, expected version 3.5.
This means that your biber (2.12) and biblatex (3.11) versions are incompatible.

This can be prevented by explicitly requesting biber 2.11:

conda install "malramsay::biber==2.11"

[pkgw]

FYI, I am working on updating the default bundle to TeXLive 2020, which may mean that it may be possible / necessary to update biber to 2.12?

[hugobuddel]

FYI, I am working on updating the default bundle to TeXLive 2020, which may mean that it may be possible / necessary to update biber to 2.12?

Thanks for the headsup @pkgw ! Apparently an even newer version of biber is required, because biber 2.12 complains that

ERROR - Error: Found biblatex control file version 3.7, expected version 3.5.
This means that your biber (2.12) and biblatex (3.14) versions are incompatible.
See compat matrix in biblatex or biber PDF documentation.

Biber 2.14 according to that compatibility matrix. Apparently they synchronized their minor version numbers since 2.7 and 3.7. But biber 2.14 is not yet on conda, only 2.12.

For now I've reverted to the 2018.1r0 bundle (with -w https://tectonic.newton.cx/bundles/tlextras-2018.1r0/bundle.tar) and biber 2.11, so at least everything still works.

This isn't really your problem, but it is good to keep this information here for reference.

[pkgw]

@hugobuddel Thanks for sharing your findings!

[malramsay64]

@hugobuddel I have updated the version of biber to 2.14 on my channel which you linked.

I am not sure whether it makes sense to propose biber for addition to conda forge. It believe it would require some work getting biber to compile, since my recipe (https://github.com/malramsay64/biber-conda) currently just downloads the release from sourceforge and uploads to anaconda.

[hugobuddel]

Thanks @malramsay64, I did not realize you were subscribed to this issue.

I forked your package to either give you a PR or upload a new version myself, but ran in some unrelated issues (conda/conda-build#3991). Getting biber to compile properly would be the best, but this is an okay-enough stopgap; thanks @pkgw.

(Or well, incorporating biber into tectonic would be even nicer, but that seems even more complicated, given the enormous task it seemed to have been to include bibtex; another thanks to that @pkgw.)

(Both conda and tectonic are trying to make package management and dependencies easier, and with big and small steps it seems we are getting there.)

[pkgw]

OK, as of the 0.7 release, Tectonic now supports execution of biber as an external tool. The shell-escape work set up the technical pieces needed, and my overall feeling now is that it's better to probably compile a document correctly as opposed to definitely compile it incorrectly.

With that in place, I think it makes sense to close this particular issue.

I'm a core member of conda-forge, and would be happy to help anyone who'd like to try to prepare a biber recipe! I don't think that I have the bandwidth to do it myself, though.

[hugobuddel]

Great, thank you @pkgw .

There is a biber recipe in https://github.com/malramsay64/biber-conda , which just copies the binaries. I care about this, so maybe I should take it upon me to create a proper conda-forge recipe from that. Not sure whether I have the bandwidth for that, but it shouldn't be too hard.

Is it possible to release tectonic 0.7 on conda-forge without having biber already there? That would make it easier to test the biber recipe. (And would probably force me to take some action because the current biber package might be too old.)

[pkgw]

@hugobuddel Yes, the conda-forge package of tectonic >= 0.7 shouldn't depend on any biber package, IMO, so it can be updated unilaterally. Normally there's a bot that does the updates, but it looks like it hasn't been working recently ...