in reply to Issues with first module build

G'day Leitz,

Firstly, please put code, data, and any error messages within <code>...</code> tags. That way, we can see what you're seeing and get a verbatim copy via the [download] link.

The first line of MANIFEST.SKIP contains a link: it looks to us like \bbookbot-\d\.\_+; it probably really (i.e. to you) looks like \bbookbot-[\d\.\_]+.

\bbookbot-[\d\.\_]+ actually matches all of your files. \bbookbot-.*\* does not match any of your files. A simpler regex to match all of your files is /bookbot-.+?/.

Because (my guess about) the first regex should match, it's possible you have some rogue (maybe whitespace) character in MANIFEST.SKIP. Please run the following command and post the result:

cat -vet MANIFEST.SKIP

That will highlight any rogue characters. It may present an immediate solution to you.

The following (in the spoiler) is my analysis. You may want to modify that to test other regexes or different input files. Note that I trimmed the leading /home/leam/lang/git from all pathnames.

— Ken

Replies are listed 'Best First'.
Re^2: Issues with first module build
by Leitz (Scribe) on Jan 05, 2021 at 08:47 UTC

    Ken, thanks! I've put the code into code tags. As an experiment, I created a totally new module and it seems to have the same issue issue with the MANIFEST.SKIP.

    module-starter --version #module-starter v1.77 module-starter -mb --module test_module_starter

    MANIFEST.SKIP.bak

    cat MANIFEST.SKIP.bak

    I then

    cd test_module_starter/ cp MANIFEST.SKIP.bak MANIFEST.SKIP ./Build disttest

    Perl Info

    perl -V
    Summary of my perl5 (revision 5 version 30 subversion 0) configuration +: Platform: osname=linux osvers=4.19.0 archname=x86_64-linux-gnu-thread-multi uname='linux localhost 4.19.0 #1 smp debian 4.19.0 x86_64 gnulinux + ' config_args='-Dmksymlinks -Dusethreads -Duselargefiles -Dcc=x86_64 +-linux-gnu-gcc -Dcpp=x86_64-linux-gnu-cpp -Dld=x86_64-linux-gnu-gcc - +Dccflags=-DDEBIAN -Wdate-time -D_FORTIFY_SOURCE=2 -g -O2 -fdebug-pref +ix-map=/build/perl-Wfb2Cd/perl-5.30.0=. -fstack-protector-strong -Wfo +rmat -Werror=format-security -Dldflags= -Wl,-Bsymbolic-functions -Wl, +-z,relro -Dlddlflags=-shared -Wl,-Bsymbolic-functions -Wl,-z,relro -D +cccdlflags=-fPIC -Darchname=x86_64-linux-gnu -Dprefix=/usr -Dprivlib= +/usr/share/perl/5.30 -Darchlib=/usr/lib/x86_64-linux-gnu/perl/5.30 -D +vendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/ +x86_64-linux-gnu/perl5/5.30 -Dsiteprefix=/usr/local -Dsitelib=/usr/lo +cal/share/perl/5.30.0 -Dsitearch=/usr/local/lib/x86_64-linux-gnu/perl +/5.30.0 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 - +Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3 -D +use64bitint -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pag +er -Uafs -Ud_csh -Ud_ualarm -Uusesfio -Uusenm -Ui_libutil -Ui_xlocale + -Uversiononly -DDEBUGGING=-g -Doptimize=-O2 -dEs -Duseshrplib -Dlibp +erl=libperl.so.5.30.0' hint=recommended useposix=true d_sigaction=define useithreads=define usemultiplicity=define use64bitint=define use64bitall=define uselongdouble=undef usemymalloc=n default_inc_excludes_dot=define bincompat5005=undef Compiler: cc='x86_64-linux-gnu-gcc' ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv -fno-strict- +aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSE +T_BITS=64' optimize='-O2 -g' cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv -fno-strict- +aliasing -pipe -I/usr/local/include' ccversion='' gccversion='9.3.0' gccosandvers='' intsize=4 longsize=8 ptrsize=8 doublesize=8 byteorder=12345678 doublekind=3 d_longlong=define longlongsize=8 d_longdbl=define longdblsize=16 longdblkind=3 ivtype='long' ivsize=8 nvtype='double' nvsize=8 Off_t='off_t' lseeksize=8 alignbytes=8 prototype=define Linker and Libraries: ld='x86_64-linux-gnu-gcc' ldflags =' -fstack-protector-strong -L/usr/local/lib' libpth=/usr/local/lib /usr/include/x86_64-linux-gnu /usr/lib /lib/ +x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../li +b /lib libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt perllibs=-ldl -lm -lpthread -lc -lcrypt libc=libc-2.31.so so=so useshrplib=true libperl=libperl.so.5.30 gnulibc_version='2.31' Dynamic Linking: dlsrc=dl_dlopen.xs dlext=so d_dlsymun=undef ccdlflags='-Wl,-E' cccdlflags='-fPIC' lddlflags='-shared -L/usr/local/lib -fstack-protector-strong' Characteristics of this binary (from libperl): Compile-time options: HAS_TIMES MULTIPLICITY PERLIO_LAYERS PERL_COPY_ON_WRITE PERL_DONT_CREATE_GVSV PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP PERL_OP_PARENT PERL_PRESERVE_IVUV USE_64_BIT_ALL USE_64_BIT_INT USE_ITHREADS USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_PERLIO USE_PERL_ATOF USE_REENTRANT_API USE_THREAD_SAFE_LOCALE Locally applied patches: DEBPKG:debian/cpan_definstalldirs - Provide a sensible INSTALLDIRS + default for modules installed from CPAN. DEBPKG:debian/db_file_ver - https://bugs.debian.org/340047 Remove +overly restrictive DB_File version check. DEBPKG:debian/doc_info - Replace generic man(1) instructions with +Debian-specific information. DEBPKG:debian/enc2xs_inc - https://bugs.debian.org/290336 Tweak en +c2xs to follow symlinks and ignore missing @INC directories. DEBPKG:debian/errno_ver - https://bugs.debian.org/343351 Remove Er +rno version check due to upgrade problems with long-running processes +. DEBPKG:debian/libperl_embed_doc - https://bugs.debian.org/186778 N +ote that libperl-dev package is required for embedded linking DEBPKG:fixes/respect_umask - Respect umask during installation DEBPKG:debian/writable_site_dirs - Set umask approproately for sit +e install directories DEBPKG:debian/extutils_set_libperl_path - EU:MM: set location of l +ibperl.a under /usr/lib DEBPKG:debian/no_packlist_perllocal - Don't install .packlist or p +erllocal.pod for perl or vendor DEBPKG:debian/fakeroot - Postpone LD_LIBRARY_PATH evaluation to th +e binary targets. DEBPKG:debian/instmodsh_doc - Debian policy doesn't install .packl +ist files for core or vendor. DEBPKG:debian/ld_run_path - Remove standard libs from LD_RUN_PATH +as per Debian policy. DEBPKG:debian/libnet_config_path - Set location of libnet.cfg to / +etc/perl/Net as /usr may not be writable. DEBPKG:debian/perlivp - https://bugs.debian.org/510895 Make perliv +p skip include directories in /usr/local DEBPKG:debian/squelch-locale-warnings - https://bugs.debian.org/50 +8764 Squelch locale warnings in Debian package maintainer scripts DEBPKG:debian/patchlevel - https://bugs.debian.org/567489 List pac +kaged patches for 5.30.0-9ubuntu0.2 in patchlevel.h DEBPKG:fixes/document_makemaker_ccflags - https://bugs.debian.org/ +628522 [rt.cpan.org #68613] Document that CCFLAGS should include $Con +fig{ccflags} DEBPKG:debian/find_html2text - https://bugs.debian.org/640479 Conf +igure CPAN::Distribution with correct name of html2text DEBPKG:debian/perl5db-x-terminal-emulator.patch - https://bugs.deb +ian.org/668490 Invoke x-terminal-emulator rather than xterm in perl5d +b.pl DEBPKG:debian/cpan-missing-site-dirs - https://bugs.debian.org/688 +842 Fix CPAN::FirstTime defaults with nonexisting site dirs if a pare +nt is writable DEBPKG:fixes/memoize_storable_nstore - [rt.cpan.org #77790] https: +//bugs.debian.org/587650 Memoize::Storable: respect 'nstore' option n +ot respected DEBPKG:debian/makemaker-pasthru - https://bugs.debian.org/758471 P +ass LD settings through to subdirectories DEBPKG:debian/makemaker-manext - https://bugs.debian.org/247370 Ma +ke EU::MakeMaker honour MANnEXT settings in generated manpage headers DEBPKG:debian/kfreebsd-softupdates - https://bugs.debian.org/79679 +8 Work around Debian Bug#796798 DEBPKG:fixes/autodie-scope - https://bugs.debian.org/798096 Fix a +scoping issue with "no autodie" and the "system" sub DEBPKG:fixes/memoize-pod - [rt.cpan.org #89441] Fix POD errors in +Memoize DEBPKG:debian/hurd-softupdates - https://bugs.debian.org/822735 Fi +x t/op/stat.t failures on hurd DEBPKG:fixes/math_complex_doc_great_circle - https://bugs.debian.o +rg/697567 [rt.cpan.org #114104] Math::Trig: clarify definition of gre +at_circle_midpoint DEBPKG:fixes/math_complex_doc_see_also - https://bugs.debian.org/6 +97568 [rt.cpan.org #114105] Math::Trig: add missing SEE ALSO DEBPKG:fixes/math_complex_doc_angle_units - https://bugs.debian.or +g/731505 [rt.cpan.org #114106] Math::Trig: document angle units DEBPKG:fixes/cpan_web_link - https://bugs.debian.org/367291 CPAN: +Add link to main CPAN web site DEBPKG:debian/hppa_op_optimize_workaround - https://bugs.debian.or +g/838613 Temporarily lower the optimization of op.c on hppa due to gc +c-6 problems DEBPKG:debian/installman-utf8 - https://bugs.debian.org/840211 Gen +erate man pages with UTF-8 characters DEBPKG:fixes/getopt-long-4 - https://bugs.debian.org/864544 [rt.cp +an.org #122068] Fix issue #122068. DEBPKG:debian/hppa_opmini_optimize_workaround - https://bugs.debia +n.org/869122 Lower the optimization level of opmini.c on hppa DEBPKG:debian/sh4_op_optimize_workaround - https://bugs.debian.org +/869373 Also lower the optimization level of op.c and opmini.c on sh4 DEBPKG:debian/perldoc-pager - https://bugs.debian.org/870340 [rt.c +pan.org #120229] Fix perldoc terminal escapes when sensible-pager is +less DEBPKG:debian/prune_libs - https://bugs.debian.org/128355 Prune th +e list of libraries wanted to what we actually need. DEBPKG:debian/mod_paths - Tweak @INC ordering for Debian DEBPKG:debian/configure-regen - https://bugs.debian.org/762638 Reg +enerate Configure et al. after probe unit changes DEBPKG:debian/deprecate-with-apt - https://bugs.debian.org/747628 +Point users to Debian packages of deprecated core modules DEBPKG:debian/disable-stack-check - https://bugs.debian.org/902779 + [perl #133327] Disable debugperl stack extension checks for binary c +ompatibility with perl DEBPKG:fixes/eumm-usrmerge - https://bugs.debian.org/913637 Avoid +mangling /bin non-perl shebangs on merged-/usr systems DEBPKG:debian/perlbug-editor - https://bugs.debian.org/922609 Use +"editor" as the default perlbug editor, as per Debian policy DEBPKG:fixes/gid-parsing - [79e302e] https://bugs.debian.org/94198 +5 [perl #134169] (perl #134169) mg.c reset endptr after use DEBPKG:fixes/CVE-2020-10543.patch - [PATCH v530 1/4] regcomp.c: Pr +event integer overflow from nested regex quantifiers. DEBPKG:fixes/CVE-2020-10878-1.patch - [PATCH v530 2/4] study_chunk +: extract rck_elide_nothing DEBPKG:fixes/CVE-2020-10878-2.patch - [PATCH v530 3/4] regcomp: us +e long jumps if there is any possibility of overflow DEBPKG:fixes/CVE-2020-12723.patch - [PATCH v530 4/4] study_chunk: +avoid mutating regexp program within GOSUB Built under linux Compiled at Oct 19 2020 10:56:54 @INC: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.30.0 /usr/local/share/perl/5.30.0 /usr/lib/x86_64-linux-gnu/perl5/5.30 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.30 /usr/share/perl/5.30 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base

    Chronicler: The Domici War (domiciwar.net)

    General Ne'er-do-well (github.com/LeamHall)

      I use Module::Starter, via the module-starter script, but I don't use Module::Build (I use ExtUtils::MakeMaker). I became very curious about all of these anomalies, so I decided to try the commands that you've shown for myself.

      I upgraded modules, where necessary, to the latest versions:

      $ perl -E 'use Module::Starter; say $Module::Starter::VERSION' 1.77 $ perl -E 'use Module::Build; say $Module::Build::VERSION' 0.4231

      I pared down my ~/.module-starter/config to an absolute barebones version:

      ken@titan ~/.module-starter $ cat config author: Ken Cotterill email: kcott@cpan.org

      I created a fresh directory, ~/tmp/pm_11126324, for my tests. I then ran the commands you show.

      ken@titan ~/tmp/pm_11126324 $ module-starter --version module-starter v1.77 ken@titan ~/tmp/pm_11126324 $ module-starter -mb --module test_module_starter Added to MANIFEST: Build.PL Added to MANIFEST: Changes Added to MANIFEST: lib/test_module_starter.pm Added to MANIFEST: MANIFEST Added to MANIFEST: README Added to MANIFEST: t/00-load.t Added to MANIFEST: t/manifest.t Added to MANIFEST: t/pod-coverage.t Added to MANIFEST: t/pod.t Added to MANIFEST: xt/boilerplate.t Created starter directories and files ken@titan ~/tmp/pm_11126324 $

      So, I don't see any of the initial lines that you show:

      Created test_module_starter Created test_module_starter/lib Created test_module_starter/lib/test_module_starter.pm Created test_module_starter/t Created test_module_starter/t/manifest.t Created test_module_starter/t/00-load.t Created test_module_starter/t/pod.t Created test_module_starter/t/pod-coverage.t Created test_module_starter/xt Created test_module_starter/xt/boilerplate.t Created test_module_starter/Build.PL Created test_module_starter/Changes Created test_module_starter/README Regenerating MANIFEST Can't find dist packages without a MANIFEST file Run 'Build manifest' to generate one WARNING: Possible missing or corrupt 'MANIFEST' file. Nothing to enter for 'provides' field in metafile. Created MYMETA.yml and MYMETA.json Creating new 'Build' script for 'test_module_starter' version '0.01' File 'MANIFEST.SKIP' does not exist: Creating a temporary 'MANIFEST.SK +IP'

      There would appear to be a bug in Module::Build in that it reports "Added to MANIFEST: xt/boilerplate.t" but either that didn't happen or that line was subsequently removed:

      ken@titan ~/tmp/pm_11126324 $ cat test_module_starter/MANIFEST Build.PL Changes lib/test_module_starter.pm MANIFEST This list of files README t/00-load.t t/manifest.t t/pod-coverage.t t/pod.t

      I had a look in "Active bugs for Module-Build" but couldn't see anything related to this. There are multiple screenfuls of active bugs, so I may have missed it; also, that many active bugs does not fill me with confidence about this module in general.

      It may be that you're attempting to create your module in a directory that's filled with files from previous attempts. That could account for all the extra messages that you reported. Try creating a fresh, empty directory and start again from scratch.

      Also look in your ~/.module-starter/config and see what's there that could be causing issues. Do you, perhaps, have a template_dir: line which is causing your directory to be populated with problem files.

      Your cp MANIFEST.SKIP.bak MANIFEST.SKIP command looks wrong to me. Normally, you copy a file to a backup version; not the other way around.

      I looked at the "Module::Build: ACTIONS: test" documentation. It does not appear to recursively work through the t/ directory:

      "Tests can be defined in the standard places: a file called test.pl in the top-level directory, or several files ending with .t in a t/ directory."

      You may be able to use the --test_files argument to do what you want. I haven't tested this.

      Also look at "Module::Build: SEE ALSO". There are a number of links there that might be useful for you (e.g. the Cookbook). I didn't look at any of these.

      Finally, you might consider using ExtUtils::MakeMaker instead of Module::Build. That's what I use for both personal projects and $work: I've never had any of the sort of issues that you're reporting.

      — Ken

        Ken, I tried a different solution; creating the module as a totally different user. That changes the results greatly, so my bet is that in some previous digging into module building I read some page and modified things I've long forgotten about.

        I'll give ExtUtils::MakeMaker a look.

        Chronicler: The Domici War (domiciwar.net)

        General Ne'er-do-well (github.com/LeamHall)

      Thanks for adding the code tags.

      The reason I asked you to use -vet with cat, was to locate characters that were not immediately obvious but could affect the regex.

      Compare

      $ cat rogue_char_test qwerty asdfgh zxcvbn

      with

      $ cat -vet rogue_char_test qwerty $ asdfgh^I$ zxcvbn^N$

      On a different note, your MANIFEST.SKIP.bak file has regexes for MSWin-style files (e.g. *.bat and *.com) but your perl -V indicates a Linux(Debian) installation. Did you perhaps copy files from another OS onto your Linux OS? Were the modules you have on your Linux OS installed on that OS (with package manager, cpan or similar utility, or manual build from source)?

      — Ken

      Change into the distribution's root directory, then post here (in code tags) the output of perl t/manifest.t.

      Those path names shouldn't be absolute, and if they're being interpreted as such for some reason, the word boundary of the regex will definitely not match.

        Not a lot of info:

        perl t/manifest.t 1..0 # SKIP Author tests not required for installation

        Addendum: More info

        export RELEASE_TESTING=1; perl t/manifest.t 1..1 not ok 1 # Failed test at /usr/local/share/perl/5.30.0/Test/CheckManifest.pm +line 190. # got: 0 # expected: 1 # The following files are not named in the MANIFEST file: /home/leam/t +mp/test_module_starter/MANIFEST.SKIP, /home/leam/tmp/test_module_star +ter/MANIFEST.SKIP.bak # MANIFEST: /home/leam/tmp/test_module_starter/MANIFEST # Looks like you failed 1 test of 1.

        Chronicler: The Domici War (domiciwar.net)

        General Ne'er-do-well (github.com/LeamHall)