Beefy Boxes and Bandwidth Generously Provided by pair Networks
Welcome to the Monastery
 
PerlMonks  

Re^13: CPAN failed install

by haukex (Archbishop)
on Nov 15, 2016 at 09:44 UTC ( [id://1175935]=note: print w/replies, xml ) Need Help??


in reply to Re^12: CPAN failed install
in thread CPAN failed install

Hi Alexander,

Thanks, that's very interesting, and the differences are good to know. I played around with things like busybox a few years back, but have to admit that now that embedded systems like the RPi can run full Linux distros, I've obviously found those easier to develop for. In a message, hippo pointed me to the controversy surrounding systemd, it seems the only two distros that haven't adopted it as the default yet are Slackware and Gentoo. I guess I should start keeping an eye on those and other distros as well.

Regards,
-- Hauke D

Replies are listed 'Best First'.
Re^14: CPAN failed install
by afoken (Chancellor) on Nov 18, 2016 at 00:04 UTC
    It seems the only two distros that haven't adopted it as the default yet are Slackware and Gentoo.

    At work, we use a third-party, ARM-based embedded computer featuring a touch screen for one of our products. It runs Yocto Linux, customized by the hardware vendor, further customized by us. It is just a really big pain to change anything in the distributed code. There are literally thousands of different configuration files, all depend on another, include, extend or override each other. The build process starts by building a big database of all that junk, then builds gcc, binutils, bash and friends a few times (LFS and T2 have to do that, too), builds qemu, one or two kernels, and finally builds native binaries inside qemu. And yes, you may have guessed it, a complete compile cycle takes about a day.

    I've burned a few days trying to customize really tiny things, like getting rid of audio or bluetooth drivers and applications that won't ever be able run on the hardware we use, because it lacks both audio and bluetooth hardware. Or trying to simply disable a service run from the init scripts. I didn't even try to integrate our own programs. In the end, I just used the vendor-supplied tar file and added, deleted and changed files during the installation from a shell script. Our programs are build using Makefiles, shell and perl scripts, and the vendor's "SDK", a toolchain hacked out of the Yocto build process, with lots of references to directories that only exist on the vendor's build server.

    It's not pretty, neither the toolchain nor my hack to get the system running, but it works and runs absolutely stable. And it does not use systemd.

    Yocto is not a distribution in the classic sense, its a distribution building kit like T2 or LFS, or buildroot. But it's the most complex one I've ever seen, and I think it takes a few weeks or months to learn how to use it. In https://xkcd.com/1343/, Yocto is about 1 m outside the screen, on the right-hand side.


    It is still possible to get rid of systemd. Systemd is nothing new, it's just a tons of more or less useful utilities stuffed into one ultra-fat binary nightmare that runs as the most privileged userspace process that absolutely must not die. The idea of starting several services in parallel, with automatic dependency resolving is years older, and has been implemented at least three times. All three times, the init process is reduced to nearly the bare minimum, as the name promises. In all of the three implementations, init just starts a monitoring process. The monitoring process starts one supervisor per service, and the supervisors start the services. System logging, filesystem mounting and almost all other tasks are handled as services.

    How?

    Why init should be small and have as few features as possible, in one four-word sentence: init must not die. If init dies, the kernel will panic. All unsaved data is lost, the machine is down. Paul Jarc has explained init's main jobs in Running svscan as process 1:

    • Start up other processes (services, logging) - svscan, svscanboot from daemontools is an ideal candidate to be started by init.
    • wait() for children not reaped by other processes. It's really just that. Call wait(), over and over again. Discard the results.
    • Handle signals, mainly as interface to the kernel. Well, that's nearly handled by wait(), just set up a few signal handlers before.

    Everything else can and should be handled in different processes, started by init or one of the processes started by init. This way, init needs only very little code, and it can (ideally) completely be audited.

    Systemd puts inetd, syslog, and tons of other services into process 1. Do you want a system where a malformed network package or a malformed log message kills init (not in the sense of sending a signal, but in the sense of terminating the process)? Well, some people accept that risk for a system that boots in 10 instead of 20 seconds. Imagine that. Half of the boot time, you can reboot your shiny new machine twice as often as with a traditional init.


    I don't really care about boot times, anything less than five minutes is ok. Servers and firewalls run 24/7, so boot time is absolutely irrelevant. Workstations are started when I enter the office, and shut down several hours later. During system boot, I walk around the office, greet the co-workers, fetch a new bootle of water, find out what needs to be done that day, perhaps try to guess a solution for a bizarre problem at a customer site that may have a hardware or a software cause. That takes from 3 min to hours, depending on what has happened since the last work day. So again, boot time is irrelevant.

    What if I really need instant access to a computer? I use an always-on machine, like an X terminal running on my main server, or my smartphone, or I simply use a computer of a co-worker to get a browser or a PDF viewer.

    Alexander

    --
    Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)

      Hi Alexander,

      Wow, Yocto Linux sure sounds like "fun" :-) I've heard some similar stories about getting Linux onto embedded ARM systems (and now that you mention it, I remember playing around with buildroot a while back), but I guess I am lucky in that I went from writing my own code and drivers for embedded systems without OSes (like TI's MSP430), skipped over getting Linux to run on embedded systems on my own, and now can develop directly on Raspbian on an RPi. If I had to design another embedded system, I'd probably use something that can run a full Linux distro directly.

      All the arguments against systemd do make a very strong case against it. In your other post you mention Devuan, but personally I'd be cautious with a distro that is a fork that changes just one thing about the system - I'd worry about whether or not it will be maintained well over time. Although I admit I'm not yet ready to switch right now because my setup works for me for now, but since I've been doing a lot of work on RPis lately, I'll have to start considering whether systemd is the right thing for such an embedded system.

      In terms of boot time, I can imagine there are situations where it might be important, for example if there's some kind of power saving necessary - I once developed a system with an FPGA that had to "boot" within a few milliseconds so that it could only be turned on for a fraction of a second to conserve power. But I assume that's rare, and I don't think systemd is the solution for that kind of a hypothetical situation ;-)

      Regards,
      -- Hauke D

        In terms of boot time, I can imagine there are situations where it might be important

        Sure, like in the product I mentioned before. It combines the ARM Linux system for the user interface and an ARM-based microcontroller for the product's main functions. The microcontroller has run through its bootloader and the power on self test and is ready to work before the kernel passes control to /sbin/init (no RAM-disk involved, the system boots directly from flash).

        I would have liked to strip some unneeded features out of the kernel (why should it search and wait for unused or non-existing hardware), but alas, Yocto won't let me do this with a sane amount of work. Well, I could build my own kernel out of the Yocto tree and replace the vendor-provided one. But again, this takes some time. Likewise, my initial idea was to get rid of all init scripts, instead use some tiny init and daemontools to start all required software - essentially, X11, the main application, and during development, network, ssh, and a serial console. It could be done, and I'm sure it would reduce the boot time by a few seconds. The existing boot time is good enough, not as fast as it could be, and nobody complained. So it is unlikely that we'll invest time (and money) to reduce the boot time.

        While the main product is powered, but the user interface is not needed, we can switch the entire display module to a deep-sleep mode. It keeps its memory content, but stops the CPU and cuts power to the LCD and most other peripherals, saving significant amounts of battery power. Waking up the display module takes about one second. When running on mains power, we don't even use the deep-sleep mode, but just switch off the LCD. Switching it back on is almost instantaneous. Sleep and display off modes are the common case, not booting. So the entire product appears to start within about one second or less most of the times, and in perhaps five or ten seconds when it was really switched off.

        Not perfect, but good enough.

        Alexander

        --
        Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)
Re^14: CPAN failed install
by afoken (Chancellor) on Nov 19, 2016 at 12:58 UTC
    it seems the only two distros that haven't adopted it as the default yet are Slackware and Gentoo

    There is a debian fork called Devuan. The reason for the fork was the decision to use systemd instead of a sane init. (via heise.de)

    While searching that, I found http://without-systemd.org/. You may guess what content is hosted there. ;.-)


    Update:

    From there, I followed a link to A history of modern init systems featuring a really compact init implementation in less than 100 lines:

    # MIT license. #include <sys/types.h> #include <sys/wait.h> #include <signal.h> #include <stdio.h> #include <stdlib.h> #include <unistd.h> #define LEN(x) (sizeof (x) / sizeof *(x)) static void sigpoweroff(void); static void sigreap(void); static void sigreboot(void); static void spawn(char *const []); static struct { int sig; void (*handler)(void); } sigmap[] = { { SIGUSR1, sigpoweroff }, { SIGCHLD, sigreap }, { SIGINT, sigreboot }, }; static char *const rcinitcmd[] = { "/bin/rc.init", NULL }; static char *const rcrebootcmd[] = { "/bin/rc.shutdown", "reboot", N +ULL }; static char *const rcpoweroffcmd[] = { "/bin/rc.shutdown", "poweroff", + NULL }; static sigset_t set; int main(void) { int sig; size_t i; if (getpid() != 1) return 1; chdir("/"); sigfillset(&set); sigprocmask(SIG_BLOCK, &set, NULL); spawn(rcinitcmd); while (1) { sigwait(&set, &sig); for (i = 0; i < LEN(sigmap); i++) { if (sigmap[i].sig == sig) { sigmap[i].handler(); break; } } } /* not reachable */ return 0; } static void sigpoweroff(void) { spawn(rcpoweroffcmd); } static void sigreap(void) { while (waitpid(-1, NULL, WNOHANG) > 0) ; } static void sigreboot(void) { spawn(rcrebootcmd); } static void spawn(char *const argv[]) { pid_t pid; pid = fork(); if (pid < 0) { perror("fork"); } else if (pid == 0) { sigprocmask(SIG_UNBLOCK, &set, NULL); setsid(); execvp(argv[0], argv); perror("execvp"); _exit(1); } }

    I think it looks quite good, there are only two things I'm missing here:

    • Sanitising of the output: This init relies on the kernel to provide a sane set of stdin, stdout, stderr. Technically, setting up the standard handles could be done in rc.init and rc.shutdown, but errors this init wants to report via perror() are written to whatever the kernel has set up as handle 2. Compare with busybox init: It calls console_init() and set_sane_term() as first actions.
    • Handling for all kernel signals. Keyboard request (usually mapped to Alt+UpArrow) results in a SIGWINCH signal sent to init. Easy to add to the sigmap[] variable, add a new handler that starts yet another program.

    rc.shutdown is started by two signals, you have to look at the command line parameters to get the difference. I would have prefered one program per signal. For the reboot and shutdown signals, I could either use two different tiny scripts that would execute the same shutdown script with the two command lines, or I could just symlink one to the other and use argv[0] to decide what to do.

    Everything else could be done by other programs, as explained in the second part of Re^14: CPAN failed install.

    Alexander

    --
    Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1175935]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others lurking in the Monastery: (5)
As of 2024-03-29 12:05 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found