I don't think you need to pull out the heavy artillery.

"Defining a workflow" is a good way to put the solution. What you do is define what data a single task requires, and keep track of the data in a session object or something of the sort.

In your case, that would be:

• Product data
• Contact
  • name
  • data
• Contacts listed completely

The last item is simply a flag; it is set when the user says he is done adding contacts. Then you define a series of forms that collect various bits of this data; f.ex, if the user enters a known contact name, you can pull out the corresponding data immediately (or just set a flag that it's there). Then your logic simply looks at the blanks in the session data and branches to the form for the first unfulfilled prerequisite.

If you completely decouple that last step - deciding where to send the user based on the blanks in the data - from the code that collects the user's response and fills out whichever blanks it can accordingly, the desired behaviour of leading the user through all the steps will just fall out naturally. What's better, as you add or remove bits from the definition of the session, the framework will automagically send the user to the right forms.

Makeshifts last the longest.

As Aristotle said, you probably don't need to pull out the heavy artillery here. When I think of heavy artillery, I think of transactions based on EDI or XML events, millions a day. Folks hurl a lot of money at the problem and names like Peoplesoft, Siebel, PaperFree, Foresight (Edisim), etc, start making appearances and many many specialists storm the facility for implementation or upgrades.

At their core, however, we're talking about some pretty simple processes such as adding a contact or initiating a billing event, munging data into a different format, etc. The only thing that makes them scary is the amount of money riding on the transaction sets.

One thing you will find in this sort of business logic is a lot of attention to dealing with events such as cancellations that might occur mid-way through. Rolling back changes can sometimes introduce lots of complexity into a problem which started out as simple.

I'm not sure how transient your "products" and "contacts" are during the data-entry phase -- it sounds as though there's an intrinsic dependency in that you do not want a product without a contact (but it's probably okay to have a contact with no products?) I'd probably introduce a transaction key that will roll back changes if a cancellation is initiated or if the process is never finished after a certain time period (they close their browser mid-way through and never come back). Depending on the type of session, you could have multiple transaction keys -- each key for a single transaction set and the required events composing that transaction set. One of the "transaction" keys could just be a more familiar session key that helps with your lateral browsing across the form space of your application.

Matt

The first thing that comes to mind is, what do you do with incompletely entered data? Let's say your "add a product" screen has 10 fields, plus the "assign contacts" area. I fill in all the fields, then realize my contact doesn't exist yet. So I jump to the "add a contact" screen.

In a desktop application, this is fine; the contact entry screen appears "on top" and once I'm done adding the contact, I return to the product entry screen, with the state of everything (including whatever I typed in those 10 fields) fully preserved.

But in a CGI context, what happens? Do you create a new product record in the database, with my (perhaps partially entered) product information? If so, there are obvious potential problems. (What if I realize I don't know all the contact information yet, and it's time to go home, and I'll deal with it tomorrow?) But if not, then when you return me to the product entry screen, I have to re-enter those 10 fields.

I'm just thinking out loud here... haven't really thought it through. But I wonder if you could use popup windows and/or frames, combined with JavaScript/DHTML for updating, to get the same effect you have in a desktop app where a new mode can be pushed onto the stack without touching the state of anything beneath it.

Sorry if this was less than lucid. I'll try to elaborate and/or clarify when I have a chance.

Cheers,
--Kevin

But I wonder if you could use popup windows and/or frames, combined with JavaScript/DHTML for updating, to get the same effect you have in a desktop app where a new mode can be pushed onto the stack without touching the state of anything beneath

I would be leery of that. A great temptation with web apps is overdesign and non-standard behavior, and using frames with DHTML is just asking for trouble. You'll end up having to support ten versions of your code for different browser/OS configuratons.

One good idea is to spend some time on sites like Expedia, JetBlue, Amazon, Southwest Airlines and other web apps that have to maintain a lot of state and allow for users to do lateral navigation. Steal ideas from them, notice how they do error handling, UI design, and so on.

When that is done, mock up your HTML pages. Then write code using fictitious subroutines / objects / whatever you like, at as high a level of abstraction as you can. Once you are satisfied with the overall architecture, move down a level of abstraction, using stub objects / subroutines, and then move down another level, until finally you have coded the whole thing out.

Of course the process is never so pat and linear, but I've found it's a good model to work from. The most important part is looking at lots of other web applications, preferably ones run by companies that have poured money into usability testing.

I would be leery of that. A great temptation with web apps is overdesign and non-standard behavior, and using frames with DHTML is just asking for trouble.

You're absolutely right. I wrote that node in haste and now I can repent at leisure. But let me clarify. I certainly didn't mean that Ovid should blithely use every DHTML trick in the book. What I did mean was that if you have a page on which the user is entering data, and you want to update some subsidiary data on a different page, it might pay to put up the entry form for that subsidiary data in a popup window. And once the user has entered that data, how do you communicate that fact to the parent window, and how does the parent window display the newly entered info to the user? JavaScript and DHTML. That's really all I was trying to say.

You'll end up having to support ten versions of your code for different browser/OS configuratons.

That's certainly an issue when you're writing a web app for the general public, but when you're working on an intranet-based app that a strictly limited group of users will access, you probably have a certain amount of control over which OS and browser the users will be using. I don't think it's unreasonable to write platform-dependent code in such a case.

One good idea is to spend some time on sites like Expedia, JetBlue, Amazon, Southwest Airlines and other web apps that have to maintain a lot of state and allow for users to do lateral navigation. Steal ideas from them, notice how they do error handling, UI design, and so on.

Absolutely. But also pay attention to where they still screw up. Some of the most annoying experiences of my online life have come when trying to price airfares. If Expedia and Travelocity and their kin work smoothly now, it's only after YEARS of sucking, and falling prey to some of the worst pitfalls of CGI-context state maintenance.

That's what I addressed. You have to define what a complete set of data is and keep track of it on the server. Then, by looking at the blanks left in that set of data, you send the user to the pages he needs to fill out. Finally, you have accumulated what appears to be a complete set of data; then and only then do you actually commit any changes to the actual database.

It really is quite simple - sort of a reduced model/view/controller design.

Makeshifts last the longest.

I would think that a general implementation of such a state machine, complete with HTML for the forms, could be implemented on top of a database for storage. The tricky part for such an approach, would be in the transitions, e.g. data validation: that must be pure (Perl) code, and I wouldn't go as far as to store that source code in a database. Hardly.

The project involved Case Studies of some sort, each Study consisting of some data (description, location etc) and an unspecifed number of contact adresses, URLs and file attachments.

If you insert a new Case Studie, the application has to go the following different steps:

1. Generate a form to insert the generic date (title, description, ..)
2. Save this date, thereby generating a new object in the DB
3. Insert Contact Adresses
4. Insert Links
5. Upload Files
6. done

1. /casestudy/insert/1
   generate form, action of the form-tag is '2'
2. /casestudy/insert/2
   generate new object in DB - now we knoe the ID of the object and redirect to
3. /casestudy/case0001.html/insert/3
   Two different things happen here:
   • List all linked contacts (including buttons to unlink them etc)
   • Add new contatcs:
     Call /contact/insert/1?link_with=case0001 and thus start inserting a new contact (or selecting an existing one), link it with the Case Studie and the redirect to /casestudy/case0001.html/insert/3 to repeat the process
   Additionally, there is a button called "finished with contatcs" that takes you to:
4. /casestudy/case0001.html/insert/4
   same as above, only with links
5. /casestudy/case0001.html/insert/5
   now fileupload
6. done.

All this is done with some mod_perl handlers, but you could just do it with CGI and "regular" parameter passing (i.e.: /cgi-bin/insert.pl?type=casestudy&step=3&id=case0001 )

You can take a look at a (German) implementation here (with News, not Case Studies, but it is basically the same..), but please don't enter too much junk, because somebody will have to weed it out afterwards...

--
#!/usr/bin/perl
for(ref bless{},just'another'perl'hacker){s-:+-$"-g&&print$_.$/}
[download]

sub workflow {

  get workflow step from button or link selected.

  if (not step1) {
    %existing_detail = get stored details
                     identified by unique session id of user
    %existing_detail .= new details
    $self->store(\%existing_detail)
  }

  my $page = $self->draw_page(page => workflowstep, 
                              params =>\%existing_detail);

  return $page

}

sub draw_page {

  my $wrkflwObj = Workflow->new(step=>$step, param=>\%existing_details
+);
  

  return $wrkflwObj->output;  #return the html

}
[download]

So, the logic of your workflow's belong in your templates. You get to aggreate all the information the user supplies, and know which step the user wants to go to next. If the user wants to add a product, and the contact doesnt exist, you can take them to a step in the contact workflow, then return them to back to where they need to go.

Flexible, as you can extend the workflow in arbitrary ways.

draw_page could potentially call a library with the method being the step in the workflow, the library would then know exactly what information is needed then return back the html page.

Not really a down side, but you'd have to be careful how you manage the expiry of each set of \%existing_detail, ie if someone doesnt click the cancel button and walks away, then comes back a day later - should their day-old data be restored?

Potential downside, you'll have to be carefull how you manage multiple incomplete "actions" per user, ie if they are 1/2 way thru' the add product action and they want to go to the modify product workflow, what do you do?