in reply to Re^5: "When code reuse turns ugly"
in thread "When code reuse turns ugly"

On the other side, a search for "npm local repository" (google, ddg) shows interesting items that seem to exist longer than the few days since the incident, probably they get more traction now … :-)

Possibly. But that raises another potential issue that arises from the latest, greatest buzzword fad that's been all over the IT news of late, and comes under various names: Continuous Life Cycle; Continuous Integration; DevOPs; Continuous Delivery etc.

A big part of this, as evidenced here and elsewhere, is the idea that every project automatically incorporates every change from all its dependencies several times a day.

To me, this is total insanity. And I am quite sure that history will prove me right again.

In theory, the idea of having "everything, up-to-date, all the time" sounds wonderful. Until you do the math.

You take a smallish project with say a dozen dependencies; each of which has half-a-dozen dependencies; each of which has 2 or 3.

Someone way down stream publishes a change that breaks something (or several somethings) higher up in a subtle way; the upstreamers go into frantic overload to write workarounds and publish; their upstreamers do the same; and your project falls in a heap because of it.

Now the developer of the breaking change realises his mistake -- whether through his own processes or because he took heat from a few dozen of his upstreamers -- and backs out the change; and so the whole chain reaction fires again.

I'm not a mathematician, and I know little about Chaos Theory beyond its name; but the above scenario seems as good a real-world definition of it as a layman could hope to find.

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)
In the absence of evidence, opinion is indistinguishable from prejudice.

Replies are listed 'Best First'.
Re^7: "When code reuse turns ugly"
by soonix (Chancellor) on Mar 31, 2016 at 11:16 UTC

    Ideally, you'd have that "Continous Whatever" in your development, but somewhere between there and production, you want tests and/or other kind of auditing anyway. Wanting "up-to-date, all the time" in production is asking for free lunch.

    Of course, security issues have to be fixed, but usually, in production we have a different notion of "continuous", don't we?
[reply]
      in production we have a different notion of "continuous", don't we?

      If you look again at my scenario, nowhere did it mention production; at least not at the top level (or bottom level; depending on what you see as up and down!).

      Changes do not have to make it into your production, for them to have had an immediate and potentially costly affect upon your project.

      It is enough for them to have made it into your dependency's production for them to be integrated at your development level, and thus cause your development to stall pending a fix.

      Ditto, for those dependencies of your dependencies; and so on back down the line.

      At the bottom level, the change may eventually be determined erroneous; but if it passes the tests -- which if they are written by the same developer that wrote the erroneous code per TTD, they will -- then that change will pass into that bottom level's production.

      Then its dependee's have no choice -- by the dictates of CI -- but to accommodate that change into their development and once it passes tests, pass it though to their production; and so on up (down; sideways; along) the chain until it reaches you.

      With hourly builds and automated promotion; a 3 or 4 level dependency chain is affecting you within half a day. Throw in a post Friday-lunchtime-celebration coding session, followed by a Bank Holiday Monday in the country of that dependency; and it could be 4 or 5 days before they discover their error. Meanwhile, its knock-on effects and (wasted) development efforts have proceeded through the chain to you. And when they discover it and back it out; the chain repeats.

      Basically, automated inclusion of dependency changes is a nightmare waiting to happen.

      Incorporating changes from dependencies into your development cycle should only be done when they are known to be needed.

      Or when a stable product has been moved to production and you started a new cycle; but even then, only if the dependency's changes are seen to be either critical or beneficial.

      Anything else is madness. (One man's opinion; history will be the judge :)

      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)
      In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
Re^7: "When code reuse turns ugly"
by hippo (Archbishop) on Mar 31, 2016 at 11:35 UTC
    Someone way down stream publishes a change that breaks something (or several somethings) higher up in a subtle way; the upstreamers go into frantic overload to write workarounds and publish; their upstreamers do the same; and your project falls in a heap because of it.

    I think you have your ups and downs reversed - easy to do if not used to the metaphor. NEILB has written quite a bit on this, see The River of CPAN for an introduction. There's been some work to mitigate the problem which you've highlighted but so long as there are dependencies it will never entirely vanish.

    The point about Continuous Integration is that it requires 100% adherence to strict testing. This means that you have a build process which includes a full test cycle before the newest deps get anywhere near production. CI sounds good but it's a lot of work to get any robustness into it as hopefully should be obvious to most.

[reply]
      The point about Continuous Integration is that it requires 100% adherence to strict testing.

      Which is fine and dandy if your dependency supplier's are as rigorous as you are. You have that confidence?

      The point is that dependency changes don't have to make it through your testing to affect your development.

      And, with programmers writing the tests that verify their own code, if a dependency programmer gets the wrong end of the stick; or has a bad day, his code probably will satisfy his tests and make it through to his production branch without error. And it may not break the intermediaries test -- they haven't written any tests for the specific change and nothing breaks in their code, so it gets waved through by the automated processes.

      Its only when it hits you that things break. What do you do? Back out the automated change and hope it'll go away? Or try to write a fix?

      If you back out the change; what happened to Continuous X? And how long do you wait before trying again?

      And if you look for a workaround; how much effort should you expend given the possibility that you'll have to revert it all next Wednesday?

      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)
      In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]

In Section Perl News