Need some advice on project design

jdlev has asked for the wisdom of the Perl Monks concerning the following question:

Hey Monks!

I'm looking for some advice on project design. I'm a newb perl programmer, so please keep the discussion elementary, though I am getting better.

Here's what I want to do. Our clients can login to our message management system and view messages. I have a client who needs a file formatted in a specific way, and he wants to download the file and save it rather than just view it. It is supposed to be saved as a .vcf file. The file is not in this format to begin with. The user would click a link on the current window, and start the program. The first thing the program would have to do is scrape the current window that is open. I'm not sure if it is html or not, but you can view the source code on it and scrape the information from there. The next thing the program will do is compile the information into a vcf file. Once that is complete, it will offer the user the option to save the .vcf file. I've never used perl with a web application before, so I'm not sure where I should get started?

I love it when a program comes together - jdhannibal

Comment on Need some advice on project design

Replies are listed 'Best First'.
Re: Need some advice on project design by mr_mischief (Monsignor) on Feb 13, 2009 at 19:13 UTC
You really need to break your design into functional units and figure out how to implement each unit. Converting from one file format to another is a good, handy, and even reusable part of your project. So is the login system, which from reading your node it seems already exists but I'm not sure. You mention this is a web application. Your code on the server won't have a way to scrape what's in the browser window, because HTTP just doesn't offer that level of integration between client and server. It was never meant to, so that's not a failing of the protocol. Yet it is a limitation of the environment you'll be using. Your best bet is to get the data from the same place on the server that the current page did and convert that. If you want to use JavaScript in the browser and communicate back to the server, scraping what's already there is possible but in this situation is likely to be far more complex than just grabbing the data on the server again. Allowing the user to save a file rather than viewing it is partly the browser's issue, because some browsers can be configured to do one or the other with a particular file type no matter how it's sent. However, you can usually have an influence on those options by using the Content-Disposition header. Getting a description of what's needed is the first step. Figuring out what pieces you'll need is the next step. Getting each piece of the puzzle to work is after that, and finally you can fit the pieces together into a usable whole. Don't forget to test early and often, preferably using automated testing tools. Perl (just like several other languages) is quite capable of taking some data, putting it into a particular file format, and presenting the file to the user. It's done with spreadsheets, graphics files, PDFs, and more all the time. As for VCF files, there are a few different types of files that typically get that extension. vCard files are a breeze to process and there are lots of tools available such as Text::vCard. For some others I'm not sure what tools there are to convert to or from them.	[reply]

Replies are listed 'Best First'.

Re: Need some advice on project design
by mr_mischief (Monsignor) on Feb 13, 2009 at 19:13 UTC

You mention this is a web application. Your code on the server won't have a way to scrape what's in the browser window, because HTTP just doesn't offer that level of integration between client and server. It was never meant to, so that's not a failing of the protocol. Yet it is a limitation of the environment you'll be using. Your best bet is to get the data from the same place on the server that the current page did and convert that. If you want to use JavaScript in the browser and communicate back to the server, scraping what's already there is possible but in this situation is likely to be far more complex than just grabbing the data on the server again.

Allowing the user to save a file rather than viewing it is partly the browser's issue, because some browsers can be configured to do one or the other with a particular file type no matter how it's sent. However, you can usually have an influence on those options by using the Content-Disposition header.

Getting a description of what's needed is the first step. Figuring out what pieces you'll need is the next step. Getting each piece of the puzzle to work is after that, and finally you can fit the pieces together into a usable whole. Don't forget to test early and often, preferably using automated testing tools.

Perl (just like several other languages) is quite capable of taking some data, putting it into a particular file format, and presenting the file to the user. It's done with spreadsheets, graphics files, PDFs, and more all the time.

As for VCF files, there are a few different types of files that typically get that extension. vCard files are a breeze to process and there are lots of tools available such as Text::vCard. For some others I'm not sure what tools there are to convert to or from them.

[reply]