Re^2: CP6AN should be a database

A simple file is beautiful.

I agree, there is nothing more beautiful than simplicity, when simplicity is all that is required.

However, take a good look at a moderatly complex distribution and note that modules are anything but simple. Look the not untypical set of ancillary files involved:

ANNOUNCE    
ChangeLog   
Event.h     
Event.xs    
INSTALL     
MANIFEST    
MANIFEST.SKIP
META.yml    
Makefile.PL 
README      
TODO        
Tutorial.pdf
c/
    ev.c       
    generic.c  
    group.c    
    hook.c     
    idle.c     
    io.c       
    queue.c    
    signal.c   
    tied.c     
    timeable.c 
    timer.c    
    typemap.c  
    unix.c     
    var.c      
    watcher.c             
demo/ 
    echo.t                  
    group.t                
    msg.pm                 
    perlqt.t               
    process.pm             
    queue_pending.t        
    rand_interval.t        
    readline.t             
    repeat.t               
    semaphore.pm     
lib/        
    Event.pm     
    Event.pod   
    Event/    
        EventAPI.h               
        MakeMaker.pm            
        Watcher.pm              
        generic.pm              
        generic.pod             
        group.pm                
        idle.pm                 
        io.pm                   
        signal.pm               
        timer.pm                
        type.pm                 
        typemap                 
        var.pm       
ppport.h    
t/  
    attach_to.t   
    bored.t       
    callback.t    
    data.t        
    delete.t      
    eval.t        
    fifo.t        
    generic.t     
    group.t       
    hook.t        
    hup.t         
    idle.t        
    idle2.t       
    io.t          
    leak.t        
    leak2.t       
    loop.t        
    now.t         
    reenter.t     
    signal.t      
    timeout_cb.t  
    timer.t       
    unconfigured.t
    var.t                    
util/ 
    bench.pl         
    filehandle.txt
[download]

I can't think of any other situation where developers would not reach for the benefits of a database to manage access to this amount of data and meta information.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

Lingua non convalesco, consenesco et abolesco.

Rule 1 has a caveat! -- Who broke the cabal?

Comment on Re^2: CP6AN should be a database Download Code

Replies are listed 'Best First'.
Re^3: CP6AN should be a database by brian_d_foy (Abbot) on Apr 08, 2005 at 22:56 UTC
What's not simple about that directory tree? Files are segregated into directories by their purpose or type, and we can look at all the files. MANIFEST is already a tiny database. Why would I store that in something just to pull it out again? MANIFEST.SKIP is similar. What structure is going to make that more simple? At least with files I can list them all, grep them, edit them, or anything else I might have to do. Your proposal doesn't make that situation any better. From what I have read, you want to replace Makefile.PL with some other file. The rest of the distribution looks the same, and the at the user level most people will not notice a difference. -- brian d foy <brian@stonehenge.com>	[reply]
Re^4: CP6AN should be a database by BrowserUk (Patriarch) on Apr 08, 2005 at 23:08 UTC
Brian. You don't like the idea. I get that. If I proposed storing financial data with sales in one flat file. Customer numbers in another. Orders in yet another. And an index to all those files in yet another, you would be amongst the first people to advocate moving that data into a proper database. The same holds true if it was an airline reservation system. Or a stock control application. Or almost any other set of data that had dependancies and relationships. Why would I store that in something just to pull it out again? What structure is going to make that more simple? The rest of the distribution looks the same, and the at the user level most people will not notice a difference. You have read way to much into what I wrote--I made no proposals for what schema should be used, and you are showing no imagination for what could be done. How about if the compiled bytecode was also stored the first time the module is used and subsequently that was returned. How about if modules recorded the maximum or average sizes of arrays created by different calling applications and used that information to preextend their internals arrays accordingly on a per application basis. How about ... Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. Lingua non convalesco, consenesco et abolesco. Rule 1 has a caveat! -- Who broke the cabal?	[reply]
Re^5: CP6AN should be a database by brian_d_foy (Abbot) on Apr 09, 2005 at 00:42 UTC
You posted a directory structure that you said is not simple. I said I thought it was simple. I might not like your idea right now, but maybe you have something that will make things simpler. So far, I haven't seen anything specific, or how anything you are saying will improve on the structure you say is complex. You posted that directory tree saying it was "moderately complex" and "anything but simple", and responding to a post saying that flat files were simple. I disagreed, but I didn't read too much into it. If we were talking about an airline reservation system or stock control application, we wouldn't be talking about something that anyone can download and install anywhere. We're talking about CPAN and Perl here, and those have already been amazingly successful at running almost everywhere. I can't even export QuickBooks from my Mac to work with the QuickBooks on my accountant's PC. Surely you aren't advocating that situation. Operating systems can deal with filesystems pretty well. Add something on top of that and you have to make sure it works on all the operating systems, and do it without simple commands on simple files. Now you have your idea, let's see some implementation. :) -- brian d foy <brian@stonehenge.com>	[reply]
Re^6: CP6AN should be a database by BrowserUk (Patriarch) on Apr 09, 2005 at 02:49 UTC
Re^3: CP6AN should be a database by rir (Vicar) on Apr 11, 2005 at 17:28 UTC
I can't think of any other situation where developers would not reach for the benefits of a database to manage access to this amount of data and meta information. Given that the information is pretty much static there is little need to add complexity. If a database system is used it will be another barrier to contribution to C6PAN (and, I think, CPAN, for they must become one). As a module user anyone can jump into the code in $PERL5LIB and try to figure out what was going wrong/right. Put that into a database and you exclude those that have not learned to deal with your database. In this I have used the term database as I think you mean it but I also view `/usr/lib/perl/` as a database. Be well, rir	[reply] [d/l]