There are a few reasons I think that puting the code in the database is a bad idea, and I speak from experience on this, having started my serious programmimg career with a system that stored all code in a database. Principally, there are lots of great tools for working with files (CVS, grep, diff, emacs, rsync, etc.) and all of them have to be reinvented when the code is in the database instead. At the very least, you need something to import/export the code from the database. Writing test scripts becomes much more difficult. Getting a working copy of the current code becomes a chore.

You mention the difficulty of updating. This is not a hard problem to solve. Simply using Apache::Reload will avoid having to restart the server. However, you'll shred your copy-on-write shared memory this way, which must already be happening with the current system. That hurts the scalability of the site, since it lowers the number of processes you can run without going into swap.

Keeping code in CVS (or Subversion or whatever) is the expected standard, and there's a good reason for it. It allows you to do things like branches, which are not possible with a simple revision system. It also allows people who are familiar with other open source projects to get started quickly. Creating RCS-like functionality in PM itself is not a substitute for the full power of source control. I also don't really buy into the idea of this being more object-oriented. Looking at the node you linked to, I see no POD, no easy way to write a test script, no easy way to run perltidy on it, no way to perldoc it if there was POD, etc.

The bottom line is that putting all the code in the database is non-standard and unnecessary. It makes things much harder for new people who are familiar with other open source projects and are interested in getting involved with PM. It obviously has not prevented work from going on, but I consider it a clear negative, and I suspect many others would agree with me.

Well, color me unconvinced for this case. And I just had an example of why. tyes funky html error catcher, nesting enforcement thingee doesnt handle </br> in a way that I think it should. IMO such tags that arent matched by an opening br tag should be silently "fixed" and not marked as an error. (This is not a complaint tye, its a feature request. :-) Anyway, being pmdev etc I figured instead of annoying tye id just hack in the appropriate changes. So first i searched for the patches that created the changes, I found User Settings used certain keywords that i could do a code search from. So i did the search, and found that in fact the code used to do it is burried in a PM module. The consequence is that no patch is possible within the time that I have to donate.

The simple act of putting that code in a PM module (which afaict is due to historical reasons) means that in order for me to patch that code i need to do the following: 1. download the PM module. 2. Patch the PM module. 3. Upload the PM module to two seperate machines. 4. Stop the httpd's on those machines (since we aren't using Apache::Reload) and then 5. restart them (er, i guess i could wait for the periodic restart, but then i wouldnt necessarily be around to see the code go into production and respond to any issues it raised.)

Thats a lot of work to effect what i suspect would be only a few line of code. Now had that subroutine been an htmlcode node I could have done the following: 1. find the node, 2. patch the node just as you would write a note, 3. Push the patch to pmdev, 4. Apply it. 5. Assuming it worked, apply it on PM propper. So all told becuase its in a module the application and nondevelopment work required to get the patch in production would outweigh by many times the actual development time involved. This isnt a good use of my time, and frankly it is unlikely I will _ever_ do this.

Now I think the problem here is a methodology clash. PM is a production enviorment desgined to allow in use/inline development. All of the techniques and tools that you mention are oriented towards a far less dynamic, develop-test-rollout-repeat methodology that is slow and requires vastly more investment in development infrastructure. Sure developers are more comfortable working with their normal tools. But frankly im not in the slightest bit convinced that said tools would make development easier. In fact im of the opinion that if that were the case there would be virtually NO development on PM at all. Certainly there would be no opportunity to have a pmdev group interacting in a communal forum.

Now I recognize that your arguments have considerable merit. But I have to say that they have merit in other contexts. They have merit for the Everything Engine itself. They have merit for larger projects where the code will be duplicated and deployed in multiple sites with no reference to each others implementation other than the fact they share the core engine. They have merit when there are a considerable number of developers who can spend signifigant amounts of time managing the development and rollout process (ie, being the Perl Pumpking is a full time job practically, Linus and his leutenants represent multiple full time jobs) But in this context its rare that a god or pmdever will have more than a few hours at a time to work on the site.

So all in all I think that you are arguing that the methodology of one type of development should be used in an utterly different context, a context so different that i think the methodology would kill all development outright.

Now, id like to address some of your other points. Testing is a good example. I see no reason why PM cannot use the exact same testing methodology as is normal in Perl. Insofar as functionality is written in htmlcode nodes it is trivial to create a new htmlcode node that is called by a superdoc that will run the tests and report any issues directly. In fact the PM DB driven framework offers a large number of opportunities in this area that i cannot envision being effective in a conventional development enviornment. (Such as tightly binding test code to the code it tests, such as tightly binding documentation to the code it documents, such as allowing multiple people to simultaneously work on different parts of a system in a collaborative fashion without wildly forking the build.)

You mentioned that you worked on a DB based code system before. You didnt expand very much on it so i dont know what type of enviornment it was, but i think that you have to consider PM in an exceptional light. All of the problems you outlined have already been solved (uploading code, extracting code, etc). The site itself is in essence a huge collaborative development and work space on its own. I see only one real advantage to the approach you outline, and that is that it would increase the comfort of new developers with similar background as you (and many others :-). For instance I have never used CVS in my life, I'm a Win32 developer who uses SourceSafe instead, I've never used Emacs beyond the "why the **** don't the keys do NORMAL freaking things!!! *&$%$%$*@@!" point. As for POD, we could easily implement a "node POD view" for pmdevers that would pass any node through pod2html before rendering the code. Ditto for Perltidy, a link in pmdevnodelet could automatically pass code through Perltidy. The perldoc comment is superfluous as we have a host of more powerful ways to search and scan code and documentation as part of the site.

As a last point I think the only real thing stoping experienced programmers from getting involved in pmdev development was the lack of test/dev enviorment, and a paucity of gods willing and able to do code reviews and code application. Both of these problems are now resolved. So the only thing stopping folks from digging in is a lack of creativity (I'm a good programmer, but by no means brilliant, if I could figure out ways to develop and test off site _before_ the pmdev server existed then I know others can too) and an unwillingness to think outside of the box. (As in the box that represents "normal" development methodology.) Another point is that in order to do useful stuff on PM you need to understand the architecture. IMO it would be almost impossible to grok said architecture without a live site to play with.

Anyway, thanks for your kind words. :-)



---
demerphq


First they ignore you, then they laugh at you, then they fight you, then you win.
-- Gandhi

In my previous comments, you should read "emacs" as "your favorite editor", the point being that a TEXTAREA in a browser in not anyone's favorite editor.

It sounds to me like the reason you consider changing the .pm files difficult is that it's outside the way of doing things that you have adjusted to. The steps are basically the same. When I build a cluster, I typically share the code on NFS so that I don't need to copy it to multiple machines. I think loadng code without restarting should be discouraged because of the memory problems it causes, but Apache::Reload would allow you to update that stuff in the same way you currently update the database.

You certainly could write a web interface to perltidy and a POD viewer, and a modified test harness, etc., but to me it all looks like wasted effort. Those things (and many other handy tools) already exist for working with files.

There is a larger issue with scalability. With only a few people submitting patches, it will work to have an RCS-like system for simple revisions. Other people may simultaneously work on a patch that breaks because of an interaction with a patch you submit, but it's unlikely. If more people were working on the code, the chance of conflicts would increase. That is what a develop => test => release cycle prevents. You also can't make large changes that affect many parts of the system without effectively halting all other development. That's what branches are for.

I should take another look, because it sounds like things have improved a lot in terms of test environment and that was what stopped me cold before.

tyes funky html error catcher, nesting enforcement thingee doesnt handle </br> in a way that I think it should. IMO such tags that arent matched by an opening br tag should be silently "fixed" and not marked as an error.

As for "silently": User settings, "HTML error reporting level". The default is to silently fix everything with the user having several levels of optional error reporting.

It was quite a while before I 'got' part of what you meant here. You want </br> to be changed to <br /> if there is no opening <br>? Do you want to do this only for empty tags like BR and HR?

I'm not sure that is a good idea in either case. <br> already becomes <br /> and isn't tracked as far as nesting since it doesn't nest. I don't think I want <br></br> to become <br /><br />, but then I'm not sure how common these types of usage are. If some popular browsers already treat </br> as <br />, then that is likely an indication (and perhaps a cause) of how common that particular mistake is.

And I don't think it's the right idea for tags that nest either.

When HTML nesting enforcement becomes manditory, then everyone will be shown such errors when they preview and they can fix such ambiguous errors which will always be ignored as far as formatting goes.

- tye