BBC: Blead Breaks JSON::DWIW

[cjg-cguevara]

This is a bug report for perl from "Carlos Guevara" [email protected],
generated with the help of perlbug 1.43 running under perl 5.41.6.

---

BBC: Blead Breaks JSON::DWIW

Please see http://fast-matrix.cpantesters.org/?dist=JSON::DWIW

---

Flags

• category=core
• severity=low

---

Perl configuration

Site configuration information for perl 5.41.6:

Configured by cpan at Sun Oct 20 22:36:51 EDT 2024.

Summary of my perl5 (revision 5 version 41 subversion 6) configuration:
  Commit id: cb2baa70118e364c1428dc77cb8f9c614060c26e
  Platform:
    osname=linux
    osvers=5.14.0-427.40.1.el9_4.x86_64
    archname=x86_64-linux
    uname='linux cjg-rhel9 5.14.0-427.40.1.el9_4.x86_64 #1 smp preempt_dynamic fri oct 4 15:22:45 edt 2024 x86_64 x86_64 x86_64 gnulinux '
    config_args='-des -Dprefix=/home/cpan/bin/perl -Dscriptdir=/home/cpan/bin/perl/bin -Dusedevel -Duse64bitall'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=undef
    usemultiplicity=undef
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
  Compiler:
    cc='cc'
    ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2'
    optimize='-O2'
    cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
    ccversion=''
    gccversion='11.4.1 20231218 (Red Hat 11.4.1-3)'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='cc'
    ldflags =' -fstack-protector-strong -L/usr/local/lib'
    libpth=/usr/local/lib /usr/lib /usr/lib64 /usr/local/lib64
    libs=-lpthread -ldb -ldl -lm -lcrypt -lutil -lc
    perllibs=-lpthread -ldl -lm -lcrypt -lutil -lc
    libc=/lib/../lib64/libc.so.6
    so=so
    useshrplib=false
    libperl=libperl.a
    gnulibc_version='2.34'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=so
    d_dlsymun=undef
    ccdlflags='-Wl,-E'
    cccdlflags='-fPIC'
    lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong'


---
@INC for perl 5.41.6:
    /home/cpan/bin/perl/lib/site_perl/5.41.6/x86_64-linux
    /home/cpan/bin/perl/lib/site_perl/5.41.6
    /home/cpan/bin/perl/lib/5.41.6/x86_64-linux
    /home/cpan/bin/perl/lib/5.41.6

---
Environment for perl 5.41.6:
    HOME=/home/cpan
    LANG=en_US.UTF-8
    LANGUAGE (unset)
    LC_ALL=C
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/cpan/bin/perl/bin:/home/cpan/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin
    PERL_BADLANG (unset)
    SHELL=/bin/bash

[haarg]

This, along with #22686, #22687, and #22688, were broken by #22647.

@iabyn can you take a look?

[mauke]

Error message:

Unparseable XSUB parameter: 'SV *' in DWIW.xs, line 1775

[iabyn]

On Sun, Oct 20, 2024 at 09:51:05PM -0700, mauke wrote: Error message: ``` Unparseable XSUB parameter: 'SV *' in DWIW.xs, line 1775

The line in the XS file is: code_point_to_utf8_str(SV *, SV * code_point_sv) It appears to be trying to skip assigning the first arg (the object) to a local variable. But as far as I can tell, this isn't documented as allowable XS syntax. So the author seems to be relying on a bug. My recent changes made such an unparseable parameter declaration an error; previously the declaration was silently skipped (although the argument number was still incremented). Note that the XS parser doesn't understand or parse C type syntax. It just expects a parameter declaration to be of the general form: (IN/OUT/etc)? (unparsed line noise)? var_name (= (unparsed line noise))? It assumes that the first chunk of line noise is the C type, and the second chunk of line noise is the default expression, if any. It can't know that 'SV *' is a valid C type syntax, just that it is something where there isn't a var name at the end. Or to put it another way, the XS parser can't distinguish between these two parameter declarations: SV* SV* foo+ To us humans, the first is obviously a var-less C type declaration, while the second is obviously a typo. The XS parser would see both those parameter declarations as just unparsebale. If we allow the first, we have to allow the second too. The three options seem to be: 1) Revert to 5.40 behaviour: any unparsebale parameter declaration is treated as "silently skip the next arg". This would include typos such as 'SV* foo+'. 2) As 5.40, but add a warning that the parameter declaration is unparseable and is being skipped. 3) Keep the new behaviour which makes it an error. I favour 2 or 3, with leaning towards 3. But if more distros are affected, then I would lean towards 2. Note that JSON::DWIW hasn't been updated since 2010, and the author hasn't been active on CPAN since 2012. So getting a fixed release might not be straightforward. I have no idea of the popularity of otherwise of this distro.

…

-- The Enterprise successfully ferries an alien VIP from one place to another without serious incident. -- Things That Never Happen in "Star Trek" #7

[jkeenan]

The breaking commit:

commit a399a4923c7f15aa1aa5bdb38a5dccf59ef51678
Author:     David Mitchell <[email protected]>
AuthorDate: Mon Sep 23 18:52:11 2024 +0100
Commit:     David Mitchell <[email protected]>
CommitDate: Fri Oct 18 11:12:17 2024 +0100

    ParseXS: refactor: XSUB signature-processing

[iabyn]

On Mon, Oct 21, 2024 at 03:33:41AM -0700, David Mitchell wrote: The three options seem to be: 1) Revert to 5.40 behaviour: any unparsebale parameter declaration is treated as "silently skip the next arg". This would include typos such as 'SV* foo+'. 2) As 5.40, but add a warning that the parameter declaration is unparseable and is being skipped. 3) Keep the new behaviour which makes it an error. I favour 2 or 3, with leaning towards 3. But if more distros are affected, then I would lean towards 2.

Or I've just thought of a fourth option. We could officially sanction 'SV*' as the text for a placeholder parameter i.e. the XS equivalent of perl's ($a, undef, $b) = (...). Then that would work (by accident) on older versions, and would work officially on newer releases. Any other unrecognised text would continue to warn or die depending on which option we choose.

…

-- "I do not resent criticism, even when, for the sake of emphasis, it parts for the time with reality". -- Winston Churchill, House of Commons, 22nd Jan 1941.

[tonycoz]

We could officially sanction 'SV*' as the text for a placeholder parameter
i.e. the XS equivalent of perl's ($a, undef, $b) = (...).

I've done that with a name but no defined type in the past:

Imager::IO
io_new_bufchain(class)
    CODE:
	RETVAL = io_new_bufchain();
    OUTPUT:
        RETVAL

[iabyn]

I've just created PR #22761 with two commits.

The first just adds tests to confirm that using a parameter without a type acts as a placeholder (as it always has).

The second commit addresses this ticket and also ticket #22686, which are both concerned with using 'SV *' as a placeholder. Traditionally, XS silently ignored any parameter declaration it couldn't parse: so acting as a placeholder as a side-effect. My recent work made unparseable parameter declarations an error, which broke these two distributions. This commit makes an explicit exception for 'SV *', which now acts as a placeholder again. Any other unrecognised parameter declaration is still a syntax error.