in reply to Re^3: CP6AN should be a database
in thread Notion: CP6AN <strike>should</strike> could be a database

Brian. You don't like the idea. I get that.

If I proposed storing financial data with sales in one flat file. Customer numbers in another. Orders in yet another. And an index to all those files in yet another, you would be amongst the first people to advocate moving that data into a proper database.

The same holds true if it was an airline reservation system. Or a stock control application. Or almost any other set of data that had dependancies and relationships.

Why would I store that in something just to pull it out again? What structure is going to make that more simple?
The rest of the distribution looks the same, and the at the user level most people will not notice a difference.

You have read way to much into what I wrote--I made no proposals for what schema should be used, and you are showing no imagination for what could be done.

How about if the compiled bytecode was also stored the first time the module is used and subsequently that was returned.

How about if modules recorded the maximum or average sizes of arrays created by different calling applications and used that information to preextend their internals arrays accordingly on a per application basis.

How about ...


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco.
Rule 1 has a caveat! -- Who broke the cabal?

Replies are listed 'Best First'.
Re^5: CP6AN should be a database
by brian_d_foy (Abbot) on Apr 09, 2005 at 00:42 UTC

    You posted a directory structure that you said is not simple. I said I thought it was simple. I might not like your idea right now, but maybe you have something that will make things simpler. So far, I haven't seen anything specific, or how anything you are saying will improve on the structure you say is complex.

    You posted that directory tree saying it was "moderately complex" and "anything but simple", and responding to a post saying that flat files were simple. I disagreed, but I didn't read too much into it.

    If we were talking about an airline reservation system or stock control application, we wouldn't be talking about something that anyone can download and install anywhere. We're talking about CPAN and Perl here, and those have already been amazingly successful at running almost everywhere.

    I can't even export QuickBooks from my Mac to work with the QuickBooks on my accountant's PC. Surely you aren't advocating that situation. Operating systems can deal with filesystems pretty well. Add something on top of that and you have to make sure it works on all the operating systems, and do it without simple commands on simple files.

    Now you have your idea, let's see some implementation. :)

    --
    brian d foy <brian@stonehenge.com>
      Now you have your idea, let's see some implementation. :)

      I generally only expend energies where I have a fairly strong reason to believe that they are going to be useful.

      The purpose in mentioning the idea here, rather than say on P5i (or P6meta if that was still active), is to get a feel for whether the idea has any merit. And if it has, then perhaps gather ideas about how it could be used and what form it should take.

      My original thought was that using (something like) SQLite which is already ported to a wide variety of platforms and small, and legally unencumbered enough that it could be distributed as a part of the core. As well as providing for this particular usage, having a reasonably capable SQL database as a part of the distribution would itself encourage the use of DBI--which would not be a bad thing I think. It would be great for trying out ideas, prototyping, pilots etc. with the move to a more capable DB once things are working.

      I perhaps did not put enough emphasis upon the other part of the notion. That of having a module generic but platform specific module build process. Much of the complexity of the EU::MM/M::B generated makefiles revolves around need to generate makfiles that can copy/move/rename etc in a platform independant way. These modules are already capable of genrating this type of platform independant makefiles for all the platforms where Perl runs.

      The problem comes when things don't quite work. Because of the need for these makefiles and the generating process to be platform agnostic, they are (necessarialy) very complex. It makes tracking back through the makefile and the generation process very difficult, and means that changes required by one platform can have undesired side effects upon others.

      A platform specific makefile can

      1. use that platforms shell commands directly--level 1 reduction in complexity.
      2. A platform specific, built at Perl(6/Parrot) install-time makefile would have ready access to all the information used to build the installation. No 'discovery' would be require.--Level 2 complexity reduction.
      3. If everything lives in the database, much of the copying/renaming etc would be avoided.

        The current practice of downlooading a gz to a packages directory, Unzipping and untarring to a set of working directories (blib). Building, testing, and then copying everything from the build directories into the installation would (mostly) disappear.

        The sources would be replicated directly into the local DB. The generic makefile would be invoked to perform any compilation required, placing the results back into the DB the first time the package was invoked--which would be when the test scripts (make test) is automatically invoked as a result of the transfer completing.

      4. In fact, the vast majority of packages could be built using an appropriate set of inference rules with source and destination being the DB, though the need for a conventional make/makefile might well be eliminated entirely for many packages.

        Can it be harder to write a single generic makefile per platform, than it is to write a single tool capable of generating a generic makefile for every platform?

      Yes, it is a blue-sky notion that is ill thought through, but that is why I described it as a notion.

      I'd like to go further. I'd like all my application code within the DB also. Id like the compiled bytecode stored there. I'd like to be able to query the DB and discover what applications are using what modules. How often they are loaded. What there memory usage is on a per application/per run basis. That's real blue sky, but doable if you have a DB.

      Anyway, the thought was fleeting, and the resistance nearly universal on the basis of what I've seen so far, so like many a good idea, I'll allow it to drift off to the bit-bucket.

      Thanks for taking the time to consider it and respond.


      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      Lingua non convalesco, consenesco et abolesco.
      Rule 1 has a caveat! -- Who broke the cabal?