BastardOperator has asked for the wisdom of the Perl Monks concerning the following question:

I'm having a bit of a problem with a web based application that I've created. Anyone that's visited my homenode may have seen something called SICKnotes being advertised. Hindsight has set in and I'm now thinking that I need to rewrite this thing, and I was wondering if someone could give me some insight before I set off on this adventure.

Here's the problem, SICKnotes is meant for systems administrators and the like as basically a bbs which instead of having general topics, has a system definition (i.e. ip address, model, etc). You put all of your systems in there and then you can get an inventory list and post notes to the system about things that you've changed and whatnot. I released this under the GPL on freshmeat and of course a bunch of people have written wondering what it would take to be able to do routers or other things. Looking back, I now realize that that should have been my goal in the first place, to provide a framework for shops to customize this. But, I didn't, I created it to scratch my itch and it does.

So, here's the real question, how do I make this so that people can customize it? I've got basically 2 thoughts on how I can go about doing this (although neither is perfect or completely clear to me yet). The first would be to basically create "modules" for each "type" that you want to support, if you want to track systems, you write a module in perl defining how to display the form to add a new system, how to edit a system, how to view it, including the necessary sql and html. This seems a little rough to me, both because I haven't worked out all of the details and because it assumes or expects that the person using it can/will program it. So much of this stuff is shop specific that you'd have to, as well as create database tables and whatnot.

The other method that I could see puts all of the work on me. I somehow create a simple language which does the same thing as mentioned above (how to add/edit/view), but in a very simplistic way.
PULLDOWN table:SOMETABLE,id,topic
TEXT "Default" NAME:foo SHOWAS:"System name"

[download]
or something like that (I'm far from having anything worked out for that).

Can anone tell me how to do this the right way? Are there any modules that would help me solve this problem? Please, any insight is appreciated, I'm lost and frustrated and not looking forward to a full rewrite.

Replies are listed 'Best First'.
Re: expect users to program or define a simple definition language?
by merlyn (Sage) on Oct 23, 2000 at 17:58 UTC
    Don't reinvent Yet Another Markup Language. Consider three tiers of interface:
    1. stupidly simple handles 80% style markup language
    2. XML for complete access
    3. Perl for the powertools dudes
    It's tough to come up with the first two, but if you do it right, you can advertise your site as "configurable without any code". XML is preferred for a complete language, because it seems to be the data-representation language of choice these days, or at least gets a lot of buzz.

    -- Randal L. Schwartz, Perl hacker

[reply]
RE: expect users to program or define a simple definition language?
by princepawn (Parson) on Oct 23, 2000 at 18:16 UTC
    Well I think what BastardOperator is touching on is exactly the pain I have felt: Because of all the funny characters and syntax and semantics in Perl, it is difficult to create easy console access to Perl-Module functionality.

    For example, based on my experiences with Net::FTP, and having had to write 5 or 6 lines of code for many common actions, I figured it was time to develop Net::FTP::Common, which provides one-line access to many common tasks that took 5-6 lines of Net::FTP (e.g., grepping a listing for a file, uploading a file, downloading a file).

    Then, I wanted to be able to perform all of my Net::FTP::Common tasks from the shell, but since, it would be very difficult and traumatizing to new users to try to and access Net::FTP::Common using Perl syntax, so I was forced to write Net::FTP::Shell.

    In languages with very few funny characters, ie where common datatypes are actually datatypes (in Perl a plethora of everyday things (dates, email, ip #'s, hostnames, URLS) are actually strings instead of types and require object-oriented syntax to access and use), the programming language and console access are equally easy to use.

    The language I have found that allows me access to my programmed functionality as shell access without having to write my own shell parser everytime is REBOL, with TCL/Lisp a close second, except that it again does not export real-word data as first-class types, but instead as strings.

[reply]
Re: expect users to program or define a simple definition language?
by Malkavian (Friar) on Oct 23, 2000 at 18:48 UTC
    By far the best solution is to have a config script driven solution.
    If you feel that the re-write necessary to do this is too gargantuan, have you thought about trying to obtain more coding colleagues, and have it as an ongoing open source project?
    This way, you open up what sounds like a very useful tool to the largest possible audience, without taking too much of the load on your own shoulders.
    As per Merlyn's post above, there are Perl modules around that happily process XML, which could take a lot of work from you too.
    I tend to use XML, as most people feel pretty at home with the style of it, and it's very easily extensible.
    Anyway, that's just my tuppence worth.

    Malk

    *I lost my .sig. Do you have a spare?*
[reply]
      Well, it is an open source project. Unfortunately, it's currently written in a different language *cough*python*cough*, so I can't imagine getting help from anyone here. To tell you the truth, my whole open source experience has been frustrating. I've received really no help, but many requests for this capability and others. One guy wanted to be able to upload images. I don't know if I'm just bitching right now or not, but frankly this thing has done as much as I really care for it to do. The whole "defining types" problem isn't really a problem to me. I more or less figured that people would download this and if they wanted different features, they would take the time to hack them themselves. I'd like for it to be able to do such neat things, but I'm not trying to fool anyone, I'm not a _real_ programmer, I'm a SysAdmin. :)

      Part of the reason that I'm asking about this here is this, I originally did this in Python for 2 reasons, first to learn it, second because I figured it would be an easier language for people to hack on. Number one is done, I learned it (enough for me at least), and number two is worthless if nobody is hacking it. Perl has _many_ more modules and people using it, so I'm thinking why not rewrite it in perl where I don't have to reinvent the wheel quite so often and I have more chance of getting people involved. I need to rethink my architecture in the process though and that's where I'm at now. I dread the thought of having to take time out to learn XML, I assume DTD's would be the way to go for this, but I don't know enough about it to be sure. 4,500 lines of code down the drain.... oi veh
[reply]
        You have learned well, grasshopper. {grin}

        Just because some open-source project can be hacked doesn't mean it will be hacked. Perhaps your code didn't get anyone else's job done easier. Perhaps reinventing Everything wasn't the right way to go. Perhaps Python is less fun to program. {grin}

        -- Randal L. Schwartz, Perl hacker

[reply]
Re: expect users to program or define a simple definition language?
by c-era (Curate) on Oct 23, 2000 at 18:09 UTC
    I think people would be more responsive to config type files. You can have a directory where users can put in a file that defines how a system is accessed. This way other people can write the config files and many people can use these files. You will probably find that if you do the first method you will be writing most of the mods yourself. While the second way is more work now, you'll save yourself the work later. The most important part is to come up with a good syntax. If you would like more help with that I would gladly help you out.
[reply]
Re: expect users to program or define a simple definition language?
by AgentM (Curate) on Oct 24, 2000 at 04:44 UTC
    I'm also writing a large web app and, even though it's basically a monolithic CGI (don't worry- FastCGI!), I also wanted to integrate some method of expansion. I am accomplishing this with a sort of menu page where various functionalities can be chosen or decactivated and a further page where each "program module" (Not necessarily perl module) has some preferences/ configuration page (or new window?). Each "program module" is supposed to provide a different filter or different editing functionality for the data set provided. I have not yet documented how one builds such a module since it involves modifying some hashes in the main CGI script. But the basic way to link in a module so that it is recognized is through my /etc/myprog.conf file which I parse at script startup (easy enough in perl!) with keywords like 'UseModule' and 'ModulePath'. After checking that the file exists, I import it (it's an actual perl module) and call a standard function which links it's hashes with the main CGI's hashes which in turn will be displayed in a select menu on the final HTML page. While this in no way simplifies the manner in which a user can come up with his module (until i cough up some documentation), this simplifies the manner in which I am able to manage each module individually and expand at will.

    This method of expansion gives me quite a few more advantages over your "create-a-new-markup-lanuage" method:

    • I don't need to redesign a language, deciding on the most important parts and perhaps extending the language in a future version.
    • I automatically use a fully-featured language which allows any module to do anything within it's permission set.
    • I don't have to come up with a parser.
    • No one needs to learn a new language (unless he doesn't know Perl) and only a quick EXPORT and a standard function is needed in the module. The rest is left up to the programmer.
    • Modules can be added or disabled in various users' configurations very easily without keeping track of a whole bunch of files.
    • Modules can affect virtually any part of the program as opposed to a limited language.
    • Virtually no chance of big bugs in the parser since you're using Perl!
    • speed
    Also, as your program progresses in version, the perl modules can be morphed to C, Pie-thon, or any other language users wish to see (can you say Rexx?). If you are able to come up with a strong and basic core to your web app, the functionality can be built out of shared modules (shared among the instances of the CGI) and a version upgrade would simply entail a module upgrade. Errors and bugs can be easily eradicated by isolating and repairing single modules! I thought it through for a while and I came up with a different solution but decided the aforementioned idea would be more efficient: I considered writing a daemon that would manage a bunch of mini-CGIs that would in turn be responsible for the dbengine calls. the daemon would handle user info and the config file. But then I just defaulted to fastCGI. You might also look into some current so-called "web objects servers". Have fun with GPL!
    AgentM Systems nor Nasca Enterprises nor Bone::Easy nor Macperl is responsible for the comments made by AgentM. Remember, you can build any logical system with NOR.
[reply]

Back to Seekers of Perl Wisdom