If you don't need immediate client-side access to the data in the cookies you're much better off using sessions: see for example CGI::Session.

Store the data server side and use the cookie as a key to find it. CGI::Session pretty much wraps it all up for you.

There are too many limitations when you use cookies to store data. There are size limits, it reduces the responsiveness of the client and it is just not secure.

That is why people are going to tell you over and over that you need to use CGI::Session. Once you learn it you will be eternally grateful for it.

I'm already using a local variant of CGI::Session to store data server-side. The problem is that there are two machines which run the program, and a user could be directed to either of the two (the sys admin says that each user is usually sent to the machine they were previously on, but it can't be guaranteed every time). The other thing is that the volume of users is such that it is not feasible to keep the session data for every person, and it would be better for users to be able to have their settings persist between sessions.

I just re-checked the maximum size of the data structure as the original estimate had extra unneeded info in it; it's 402 bytes (as a file), and most user settings would probably be half that.

If you are using CGI::Session with a database, having multiple machines would not be a problem. The reason for this is that the user would just a Session ID cookie. The session ID would correspond with a primary key on a 'sessions' table in the database. The database would contain all user settings and can store whatever information that they need for the duration of the session.

All you would have to do is point both web services to the same database where the session data is stored. Now it would not matter which server you go to and you can add a third or a fourth server, etc... and you would not have to change anything.

This would also apply to data as well. You can expand the number of user settings from your 402 bytes to 3 or 4 KB and you not have to change anything.

The other thing is that the volume of users is such that it is not feasible to keep the session data for every person

You have to keep the data somewhere. You stated that currently that the settings take up 402 bytes. Than means 1 MB can store settings for 2600 users. That is not much storage space at all.

, and it would be better for users to be able to have their settings persist between sessions.

If the data is persistent and is not needed for a particular session then store it in a table. You need someway of identifying the user, create a 'user' table and put everything in there.

Databases are not a limitation, they can manage thousands and even millions of records and gigs of data.

Cookies are what are limiting, move away from using them so much.

                        /- prog 1  -\
client - load balancer -            - database
                        \- prog 2  -/

use Compress::Zlib ;
use Storable ;

my %hash = &some_data ;
my $data = pack( "u*", 
  Compress::Zlib::MemGzip( freeze \%hash) ) ;

# now you can do as you please with $data
# to unthaw:
my $hashref = thaw( Compress::Zlib::MemGunzip(
  unpack( "u", $data ) ) ) ;
[download]

Thanks for the advice so far. The data that I want to store isn't anything critical - there are five filters that users can set up for searching / viewing data, and it's these settings that I would like to put in the cookie. The cookie data would be compared against the list of valid parameters, so if users did have the inclination to tinker with the settings, the worst they could do would be to set the maximum possible filters, which you can do validly by setting those filters using the cgi.

One possibility if the data structure is too big might be to have a couple of cookies, as there are some filters which only apply to certain aspects of the program. I don't really want to be setting huge numbers of cookies if I can help it, though.

I would like to store some user settings in a cookie

Don't do that.

Sooner or later, someone will figure out how to decode the cookie contents and give themselves an enhanced status that you were probably not willing to concede.

Use the cookie as, well, a cookie. That is, a random string of bits that has no intrinsic meaning. When the client returns to your site, you read the cookie and go look up the settings that are keyed to that user in some table. Think of it as a the 100 dollar note torn in half and shared between two spies. Alone, either half is worthless: it's only when you put them together that they mean anything.

A cardinal rule of web programming is to never trust the user. Don't put any semantic meaning into a cookie. Keep all that information on your server. You'll also keep you're cookie size down. 256 bits encoded in Base-64 takes 11 bytes, and I doubt you'll hit a collision before the heat death of the universe.