But back to the question. To "edit" a file like this you basically have to load the entire thing into memory, make your changes, and then write the whole thing back out. And you'll have to lock the file while you're doing it to make sure you don't get two editors working at the same time. The reason it's not possible to edit the file in-place is because the lines are of variable length, so editing means changing the length of the file and the position of lines lower down.

So, you'll start by loading all the data into a structure in memory. Something like (untested):

  open (DB,"ext.log") or die("There was a problem opening the file.");
  my @data;
  while (my $entry=<DB>) {
    my @fields = split(/    /,$entry);
    push @data, \@fields;
  }
  close DB;
[download]

Now you'll make a change:

  $data[5][1] = "New value!";

Then write it all back out again:

  open(DB, ">", "ext.log") or die $!;
  foreach my $row (@data) {
    my $line = join('    ', @$row);
    print DB $line, "\n";
  }
  close DB;
[download]

If your file is it at all big this is going to totally suck. In that case you might look at using something like Tie::File. That will help with memory usage, but there's no helping the fact that making random edits to a sequential file will be slow.

Now doesn't this make you want to do it right? Good! You should be using a database of some kind to hold the data. If you have a real DB like MySQL or Postgres already available then use that. If not you might look at something like DBD::SQLite to save you the trouble of getting a real DB installed. Then create a table, load your log data into it, and suddenly editing lines will be cake! And who doesn't like cake?

-sam

Hi Sam,
That's what I meant by right -- having a "real" DB and the use to tools/mods but I'm stuck with what I have. As for locking people out, I was going to use a C-style flock() but not sure yet. I know this will totally suck, that I can agree with! I will say after the dataset is complete there will be ZERO chance to simultaneous edits to the DB, so locking isn't a huge concern at this point. I'll try to make what you have there work, but like I said -- I'm not really much of a developer, I just try to help when I can. Thank you sir!

lakeTrout

You could write two programs. One takes what you have in the flatfile and inserts it into the database. The second does what you need to do out of the database.

Before you object, consider that it's effectively the same amount of work as what you're doing now. The difference is that it'll be much easier to work with data when it's in a database. (You'll have to figure out how to map back and forth between Perl and your flatfile, which is what the flatfile-to-database converter has to do anyway.)

"Stuck" huh? That sounds unpleasant. Should we call the police? Are you being held against your will?

I guess I'm just an asshole, but I can't imagine anyone successfully preventing me from using a technique I knew was superior and which wouldn't take any additional time to implement! What's the worst they can do? Fire you? It sounds like they'd be doing you a favor!

-sam

Sam is actually a very gracious gentleman.

Listen to the lads. Is this a linux server you are on?

Use a db or something db-like to store and retrieve your data. Do not edit a log file. Extract data from a log file to some kind of store, then edit that. Log files are records that in most cases should remain records of activity, and not edited directly.

You will benefit more from heading in the right direction than not. Try to learn skills that will lead you to greater confidence. It looks like you need to help your users define and service their needs. The more skill you develop the more authority you will have to help them. Wisdom appears to be a little wanting in your realm. It is up to you to help.

The tools are free, there are ridiculously over-skilled people like sam out here trying to give free software and enlightenment away, and who cannot afford it?

Thank you Doc

I've been doing a ton of reading and looked at the suggestions above. I am very close to a solution -- when it's done, I'll post it for those of you curious. I appreciate your help and time greatly. Not having an admin or direct/su access to the server has made my work limited, but as said above, pushing a little and knowing a little more and presto -- I have gained more power as a result. Thank you all.

lakeTrout