RFC: Package Shipping Modules

In my reply to this thread The sad state of Perl + UPS Rate Requests, I mentioned that it would make sense to have a common Perl API for package shipping services (Fedex, UPS etc), along the lines of the DBI/DBD modules. Since it seems that there may be some interest, I dug out my notes for such a project that never really got started.

Although the various shipping carriers have online APIs for their services, each uses their own proprietary system. While access to documentation for these API's often have various degrees of restrictions, I have been able to gain access to some of them. In studying documents from UPS, Fedex, DHL and USPS, I found is that there is much in common between them:

They use XML for request and response messages
Mesages are sent via HTTP/HTTPS
All have a similar set of basic services
All maintain a separate server for testing

It seems very feasible that we can have a common module with a standard API that does the heavy lifting, and a separate backend module for each shipping carrier. The carrier modules would probably need a minimum of actual coded logic, and would mostly consist of XML templates, and tables to translate field names and request/response data between the Perl API and the carrier's API.

Here is my very basic concept of what this might look like. ShipAPI is the common module and ShipAPI::carrier (substitute Fedex, UPS etc for carrier) is for whichever shipping carrier you use.

                                User           
                    Shipment    Service           Initialize
                    Info        Request           Module   
                                                              
                      |  A         |                 |         
                  . . |. |. .  . . |. . . . . . . . .|. . . . . . . . 
+.
                  .   |  |         |                 |     ShipAPI.pm 
+. 
                  .   V  |         |                 V                
+. 
                  .                |                                  
+. 
                  . initialize     |            _  shipper            
+. 
                  . object         |           |   config info        
+. 
                  .          ______|           |                      
+. 
                  .         |           . . . .|. . . . . . . . . . . 
+. 
                  .         |           .      |                 
                  .         V           .     .V. . . . . . . . . . . 
+. 
                  .                     .     .   ShipAPI::carrier.pm 
+.
                  .      parse          .     .                       
+.
                  .      request   <------------>   Available         
+.
                  .                     .     .     Services          
+.
                  .         |           .     .                       
+.
                  .         |           .     .                       
+.
                  .         V           .     .                       
+.
                  .                     .     .                       
+. 
                  .      translate      .     .     field             
+. 
                  .      & validate  <---------->   translation       
+. 
                  .      fields         .     .     tables            
+. 
                  .                     .     .                       
+. 
                  .         |           .     .                       
+. 
                  .         |           .     .                       
+. 
                  .         V           .     .                       
+. 
                  .                     .     .                       
+. 
                  .      fill XML       .     .                       
+. 
                  .      template    <---------->   XML template      
+. 
                  .                     .     .                       
+. 
                  .         |           .     .                       
+. 
                  .         |           .     .                       
+. 
                  .         V           .     .                       
+. 
                  .                     .     .                       
+. 
    to carrier    .      transmit       .     .     URI, Auth         
+. 
    server      <------  HTTPS       <---------->   & access          
+. 
                  .      request        .     .     info              
+. 
                  .                     .     .                       
+. 
                  .                     .     .                       
+. 
    from          .      receive        .     .                       
+. 
    carrier     -------> HTTPS          .     .                       
+. 
    server        .      response       .     .                       
+. 
                  .                     .     .                       
+. 
                  .         |           .     .                       
+.
                  .         |           .     .                       
+. 
                  .         V           .     .                       
+. 
                  .                     .     .                       
+. 
                  .      parse XML      .     .                       
+. 
                  .                     .     .                       
+. 
                  .         |           .     .                       
+. 
                  .         |           .     .                       
+. 
                  .         V           .     .                       
+. 
                  .                     .     .                       
+. 
                  .       translate     .     .     response          
+. 
                  .       response    <--------->   translation       
+. 
                  .       data          .     .     tables            
+. 
                  .                     .     .                       
+. 
                  .         |           .     .                       
+. 
                  . .  .  . | . . . . . .     . . . . . . . . . . . . 
+. 
                            |                                        
                            V                                        
                                                                     
                         Asynchronous                                
                         User Response                               
                        (via callback?)
[download]

The idea is to make it as trivial as possible to add new carrier modules, or to update a carrier module when their API changes. There are many other candidates for carrier modules (one person mentioned Australia Post) that could be added with a minimal effort. A Perl API that could communicate to any of the carrier services might look something like this:

# definitions:
#   shipper - sends the shipment
#   carrier - transports the shipment
#   recipient - recieves the shipment

# configuration info
%conf = (
    weightUnits     =>
    dimensionUnits  =>
    currency        =>
    shipper         => {
        name            =>
        address         =>
        ... etc         =>
    }
    carriers        => [
        {
            carrier     =>
            shipperId   =>
            serverUrl   =>
            authent     =>
            ... etc     =>
        },{
            carrier     =>
            ... etc   =>
        }
    ]
);

# configure the module
ShipAPI->configure(\%conf);

# create a shipping object w/recipient info
$pkg = ShipAPI->shipment(
    carrier    =>
    name       =>
    company    =>
    address    =>
    ...etc     => );

# validate recipient address
$rv = $pkg->validate();

# add package(s) info    
$rv = $pkg->addPackage(
    weight     =>
    dimensions => [$h, $w, $d]
    type       => );

# get shipping services available(ground, next day etc)
$rv = $pkg->availableShipping();

# get transit time
$rv = $pkg->transit(
    service => );

# get add-on services available for this shipment
$rv = $pkg->availableAddon()

# add add-on services (COD, pickup, etc)
$rv = $pkg->addService(
    type    =>
    param   =>
    ...etc  => );

# get service rates
$rv = $pkg->rates(
    service => );

# add billing info if not billed to shipper
$rv = $pkg->billing(
    account =>
    zipcode => );

# submit for shipment
$rv = $pkg->ship(
    service  => );

# cancel shipment
$rv = $pkg->cancel();

# cancel shipment
$rv = ShipAPI->cancel(
    carrier  =>
    tracknum => );

# track shipment
$rv = $pkg->track();

# track shipment
$rv = ShipAPI->track(
    carrier  =>
    tracknum => );

# response callbacks:

shippingRates

trackingResults

transitTime

addressValidation

shipmentAcknowledge
[download]

Remember, these are all ideas from some preliminary notes, there is no working code to go with it. This would be a rather ambitious project for me, and I'm not certain my coding skills are up to it without getting over some learning curves. So, I'm looking for feedback: What do you all think of the concept? Would this be a worthwhile & useful project? Any ideas to contribute? Anyone want to tackle this? Or help out?

Comment on RFC: Package Shipping Modules Select or Download Code

Replies are listed 'Best First'.
Re: RFC: Package Shipping Modules by ELISHEVA (Prior) on Mar 31, 2009 at 09:04 UTC
First, I want to say ++ for the amount of thought you have put into this, both on this post and your previous one. Would this be a worthwhile & useful project? Depends on your goal. If it is a learning project and you want hands on experience understanding just how terribly hard it is to build multi-anything support that really does make life easier, then go for it. This problem is sufficiently complex that it is likely to be a good learning experience. If the goal is to actually use the software, then "worthwhile" means something very different. Something is always worthwhile if there is a market. I'll take it as a given that people need to support multiple carriers in their information systems. The question is who and where are the people that would use ShipAPI? You? Start-ups who need to support multiple carriers and don't yet have the IT infrastructure for it? Software vendors selling systems to start-ups who are sick and tired of their current multi-carrier implementation? Enterprises and outsourcing shops that need to update their infrastructure? Before you invest a lot of time, be sure you have a market - even if that market is you. My rule of thumb for markets of one is this: decide how many carriers/data formats/etc I need now (and in the near future). Calculate the time needed to write and test N almost alike implementations for those carriers. Now compare that with the time to implement, test, configure, and test configurations of a generic solution. Whichever is less wins. Any ideas to contribute? If you really want to understand what makes carriers similar, focus real hard on the data. Processes change much faster than data. Process and data are always, to some degree, related. But in most cases, you can change how you do your job much more easily than you can change what you need to know to do your job. UPS, DHL, etc have been using the same data for years. XML, REST, SOAP - they are merely the favored processes and representation of the day. They change. You've talked about similarities, but to make this work, you really need to focus on the differences. It is always the differences that make these problems hard to solve well. Anyone want to tackle this? Or help out? The most likely people to help will be people that want to share your learning experience or are are also facing the problem of supporting multiple carriers. The common bond at PM is software in general and Perl in particular. You'll find people happy to use a Perl solution and maybe even learn with you, but not necessarily people that have clients/are themselves dealing with a multiple carrier support problem. To find people that need to support multiple carriers, you'll need to brainstorm a bit about who is interested the specific issue of supporting multiple carriers and are unhappy with existing solutions - enough to donate time or money to the issue. Then you'll need to figure out where to reach them - on or off line. Hope that helps a bit, beth	[reply]
Re^2: RFC: Package Shipping Modules by hangon (Deacon) on Mar 31, 2009 at 20:53 UTC
Thanks for insight ELISHEVA. While supporting multiple carriers would be useful, my intent was to also address the following. A developer would only need a single API to write applications for different shipping carriers. If a developer writes an application using this module, and later the client decides to use a different carrier, it would not require a major rewrite of the application, just installation of the appropriate carrier module. By incorporating the heavy lifting in the core module, a carrier module could be easily written for even the most obscure shipping company or postal service. When carriers change their API, only the carrier module would need to be updated. You bring up a good point about processes. Though it appears that XML is the protocol du jour, there are probably others in use. My guess is that, for instance, UPS will not arbitrarily make major changes that affect their customer's ability to do business with them, but that's no guarantee. So perhaps another part of the project would need to be a set of plug-in protocol handlers. By worthwhile I am trying to find out if there is an interest in having something like this available to the Perl community. It seems it could be especially useful to web developers, and would be another excuse to write e-commerce applications in Perl. Since I find this problem interesting, in a perfect world (where I had all the time needed to do whatever I want) I would work on this thing as long as my attention span lasted. But since the world is not perfect, I'm throwing the idea out there to see what happens. If this project has enough merit, maybe a group will coalesce to collaborate on developing it, and would let me participate. update: added another item to list	[reply]