Up to a point, Lord Copper. As a Perl newbie, it's very difficult for me to know whether to put a question in SoPW or CQ&A. CQ&A is also very unstructured. What I would like to see is CQ&A in more of a tree form. To give an example, there are 14 entries in GUI programming, in no apparent order. It would be more helpful to me to have them categorised further into Tk, MessWare etc., although I accept that with only 14 it's probably redundant. However, there are lots of posts in SoPW that never seem to make it into CQ&A. I suspect that the amount of work involved is the main issue.

So, let me make a modest proposal. Someone with the time and knowledge enables CQ&A to operate on a tree basis, and creates a "Spreadsheets" top level category. Provided that the maintenance tools aren't too complicated, I hereby volunteer to populate and maintain links to spreadsheet and Perl related content. I say "links", because I don't see any great disadvantage to driving everything internal with node pointers, regardless of where the original node appeared. And yes, I would - unless I was instructed not to - link to nodes that are already in the tutorials section.

Obviously, the first question must be: is my proposal possible? The second: what tools are (or can be made) available? Oh yes, and would anyone else be prepared to maintain other categories? This last isn't essential, although I would prefer not to be a lone voice in the wilderness.

The reason I'm talking about spreadsheets rather than anything else is that I do know my way around them quite well, even if I don't know much Perl. I've also started to build up a library of links for my own use, and formalising the process would not be too much like work.

Let the flames commence!

Regards,

John Davies

Well Mr. Salter, but this goes a bit beyond the OP. When I read it, I immediately thought about Categorized Questions and Answers and the Perl Cookbook, which share a similar structure IMHO.

I don't believe much in deep tree structures, they tend to confuse and to hide stuff. I prefer http://search.cpan.org much more than http://www.cpan.org, to make an example; in a deep structure you could end up putting the solution to a more general problem inside a particular leaf which could match poorly the needs of others.

Flavio (perl -e 'print(scalar(reverse("\nti.xittelop\@oivalf")))')

Don't fool yourself.

Personally I am happiest when I can find well organized documentation (the online user-annotated mysql manual is nice, and the Mac developer documentation was too, haven't seen OS X stuff yet). Perldoc.com is kind of nice. Maybe someone has other better examples of good online knowledge bases.

How about this?

We need a collaborative tool built into PM that allows us to input which leaves of a tree a given thread/node should be linked to. Viewing a thread/node will show which leaves it has been registered to, potentially with voting involved.

I've thought of building something myself and hosting it for use by monks but it would probably be best if supported by a nodelet in PM.

So the tree structure would be editable and annotated to allow addition of more refined topic areas, plus adding an unlimited number of well-written faqs, articles, code snippets, and link lists, possibly each could include at the end a section including entries automatically generated by scanning nearby leaves. A map of crosslinks indicating related topic branches/leaves and entries in perldoc or PM library might be good too.

An ontology of topics and where they fit in a tree might be something to consider, as a guide to topics to create when needed, but I am thinking it would be more useful to make this work like a faceted metadata engine (look at for example in Flamenco which works like this. The point here being that you never see an empty topic in the tree, and the tree is presented showing the number of entries per branch or leaf. And that you may use it in connection with a keyword search, but the main idea of faceted metadata engines is that you never end up with "no matches" or "a zillion matches", and you are not bound to navigation up or down in an hierarchy but you can always see where all information is in relation to where you are in the knowledge base (user-centric navigation). Also I think it might be useful to have a history of things you have looked up and maybe show all those for today in a split screen.

Editors could move bits around and prune, perhaps making better use of their time. Recommendations can be made for writeups on topics.

Newbies would then read the library and know right away where to go and drill down. You could publish a selection in book form to make money (books on how to remane sane in a company, what strategies to use, poetry and meditations, etc.) You could start building it right away with some things loaded into it marked for consideration/editing, and an automatic search listing could show you other nodes to consider adding, with the idea that the knowledge base could hold every node but there should be a well-defined division for a given leaf/branch between highly edited answers (like a wikipedia entry), annotations, and then all the other nodes that could match the topic. Then you could browse the KB with or without the distraction of everything but the kitchen sink. Maybe also sort annotations chronologically or by XP? Finally a good use for XP?

Well I hope this helps. I just thought this up now since my original idea (for the past year or two) has been to start with an empty wikipedia like KB tree to which monks would add writeups, and not try to make it a reflection of the massive amount of data in all of the PM database. The above suggestion though includes more community and maybe more interesting.

Finally I think it would be very interesting if, unlike PM, an API (Soap?) was provided so that people could make on their own servers tools to work with the knowledge base and also with the PM database, to experiment with tools for annotation, searching or navigation, and display. This would greatly accelerate development of the PM knowledge base and make it practical for the community to get involved where currently it seems you have to be a God working on Everything.

The brief description I provided above on faceted metadata engines outlines a large project that could best be done by people not on the PM server I think, and could maybe even take advantage of software not developed by a monk.

This idea I've had is part of a concept I've been working out for a long time on how to focus information contributed in an ad hoc fashion to solve a problem (the initial case being how to assemble goods to help people in an earthquake when Internet is the only communications medium available). It has to be easy for lots of people to add information, and actively work toward gathering and organizing information, keeping people involved in single threads for a long time, which seems to oppose the web's "constantly surfing elsewhere" paradigm. I suppose this concept embodies parts of metadata engines, collaborative definition of an ontology, annotation and wiki-like knowledge bases. I have no idea if it will really work though if it does, it will be pretty quick I'd imagine. Currently I'm working on a wxPerl based application that may eventually be able to do some of this resource focussing in a distributed fashion. Well if I ever get anywhere on these things that might be neat but please consider some subset of the above suggestion as potential points for discussion.

Interested to hear your ideas!

Before diving headfirst into a tree-based ontology, please (re)read this essay by Clay Shirky: Ontology is overrated, in which he makes the case that ontological classificationis is a bad strategy when your domain has a:
• Large corpus
• No formal categories
• Unstable entities
• Unrestricted entities
• No clear edges

and your participants are:

• Uncoordinated users
• Amateur users
• Naive catalogers
• No Authority
In such cases a del.icio.us-like tagging system would make more sense, he argues.

Thank you very much for your reply. Yes, I think your point is correct. I think I was arguing to use a tag-like (well I said faceted metadata) system instead of an ontology, but for this specific project I was more interested in having an ad hoc approach where people could add categories if nothing appropriate existed, and people could edit categories in a wiki-like manner. Well I didn't have that totally worked out for a large group, which is what I wanted to support, and certainly the del.icio.us way is very interesting.

I am curious about how useful tagging would be for both my original system (intended to focus solutions to sub-problems for use in helping earthquake survivors) and the PM knowledge base which certainly seems to have some solid categories.

But as I think about it now it does seem that the key here is that a plurality of people will be involved in an editing or knowledge focusing process, and since they share the same vision of focusing knowledge and using perl, then the tags they select will be relatively focused as well. In which case a separate set of "narrative threads" would then need to be added by people, for example in PM they might indicate a favored order in which to view a set of nodes for use as an introductory text. In my ad hoc disaster aid system I think it also needs an engine or a human editor who drives forward the overarching requirement that we find a concrete answer to a problem within a given time frame (like who will bring what, where, for someone to drive to the disaster site). There can be all kinds of side discussions going on but the main thing I wanted was to solve a problem.

I had a vision of a very simple user interface with a small text entry box and a pull-down menu with subject names to select from, or you could add your own subject (or tag I guess). If you start up a new project to solve a different problem, you would have a different set of starting tags. I thought I would have a person periodically reorganizing things and reducing the number of tags as much as possible, or adding new ones (developing themes) to keep everybody headed toward resolution of the problem in a finite amount of time. This is not quite as important for PM perhaps.

Well, thank you for a really interesting note, it has gotten me thinking in new directions since I had not realized everything del.icio.us could do until now, and it seems to lead the way toward better architectures.

In the PM KB, it would seem then that you could derive a lot of information (metadata) from nodes (if you had access to the database) like who posted what when, how many votes the thread and each answering node got, and linked nodes and sites, which could provide a lot of initial tagging information.

I think the most interesting thing here is that user XP, which has been said to be useless, and votes per node, may in fact be the most useful tools we have to determine who our most influential editors are and what threads and nodes have served monks the best. Perhaps we already have sorting by votes built in to Super Search or something else? Best nodes this month seems also to be a good source of informaiton.

To get farther on this I think we need a corpus to look at, in other words start messing with WWW::Mechanize or something else to examine PM in these terms, or obtain readonly access to the database.

So thanks again and looking forward to hearing more from you if you have time. Yes, I agree with you about ontologies but would like to experiment with adding structure (albeit changing and minimal) to tags, and to add tags that have meaning in terms of an overall objective (i.e. solving a problem, or explaining perl and good solutions well). Great!

When I look for answers to my own questions, I don't browse categories. I just type the right stuff into a search box somewhere. I don't think categorization will help people find answers.

Most of the time, people asking FAQs either don't care that the answer is in the docs (Perl already comes with 9 perlfaq docs) or they don't know that the docs exists. Creating a new section of PerlMonks or spending a lot of time categorizing previous answers doesn't solve either of those.

Even though every six hours I post a part of the perlfaq to comp.lang.perl.misc, people still ask those questions there. I don't see technology or library science ever solving it. :)

Whenever I hear a question of the form: "Is anyone (else) doing anything with regard to x?"

My response is always: "I dunno. Why don't you?"

So, are you volunteering?

It sounds like a big job, but it seems mattr is also interested... Why don't you two put your heads together and work up a proposal?

planetscape

My response is always: "I dunno. Why don't you?"

Would you have replied the same if he had said: "Has anyone written a module that does X or is anyone working on that? I'd love to get my hands on it."?

Isn't it good that he asks about it first? If you don't know, why do you ask him to possibly redo work?

Why don't you two put your heads together and work up a proposal?

That's what this thread has done; brought up proposals, arguments, and counter-arguments. That's an effect of he asking whether it exists and saying that he'd like it. He inspired discussion. You don't have to be willing to volunteer in order to ask about something or bring up an idea. There may be several reasons why someone can't do it himself, but that shouldn't stop him from discussing and spreading the idea. There's no harm in asking and discussing.

We should welcome ideas. That's what leads evolution.

ihb

See perltoc if you don't know which perldoc to read!

Would you have replied the same if he had said: "Has anyone written a module that does X or is anyone working on that? I'd love to get my hands on it."?

Absolutely. My response would have also included: "And take care not to run me down in your haste to get started."

However, what the OP actually said was:

... I'd like to get my hands on a book called something like "Wisdoms of the perl Monastery, volume 3", "More Pe(a)rl for monks", etc.

Not "write a book", but "get my hands on a book", which (to my thinking) implied the OP had in mind someone else writing the book.

This distinction undoubtedly colored both my reaction while reading the OP and my response to it. It would seem I am not the only one who thought it sounded that way, or my response would have already been flamed/downvoted into obscurity.

Perhaps that it not at all what the OP intended. Perhaps the OP is a significant contributor to the inner workings of PM in his/her own right. Perhaps I should have inquired further to determine what exactly the OP did intend. I did not, and in the absence of being able to read the OP's mind, my response was what it was.

BTW, I am all in favor of not reinventing wheels. I always begin any problem-solving venture by seeing what solutions already exist. I do welcome ideas and recognize that it is through asking, discussing, and doing, that we learn.

I also try never to ask questions of the general form: "Is anyone (else) doing anything with regard to x?" where x is a site maintained by volunteers which benefits a larger majority who do not necessarily volunteer, unless I'm prepared to pick up my own shovel and start moving earth.

Lest anyone accuse me of hypocrisy, a word of "full disclosure": In CB recently I observed that it would be nice to have a Message Outbox to complement the Message Inbox so that I could refresh my memory when I'd forgotten what I'd said and then later did not understand the response. I also noted that what would be even nicer would be if I had enough Perl (and Everything) knowledge to actively contribute code towards a solution. I deeply regret I don't, yet. But I am learning - which is why I'm here.

And when I know enough to be "dangerous", I'm gonna grab that shovel.

planetscape