moddingforfun has asked for the wisdom of the Perl Monks concerning the following question:

Hello,

Have a game modding problem and think Perl may be able to solve it. Bash shell scripting failed. No experience with Perl however.

( Yes, this is legal. )

Goal : reorder, in a specific order, in each file alone, the lines of text. And avoid having to hand edit approximately 12,000 ANSI text files.

These ANSI files are in a very loose CSV format. Key pairs / variable = value, for example. Be warned that the "value" is often 'the-rest-of-the-line'.

Started reading 'perldoc intro' ... did not go well.

I have only shell scripting experience and am unable to comprehend/learn high level programming languages ( ASSEMBLY / BASIC = yes, PASCAL / C = no ).

Thank you for your time and consideration.

I have put together the following files. A file name list 'filelist.txt', and a template containing the reordered lines of text ( by first word of the line ) called 'template.txt'.

I can supply an 'unedited' file as well as a 're-ordered' file. The files have an .odf extension but are really renamed ANSI .txt files.

Sample of 'filelist.txt'

bor_adaptorC.odf bor_adaptor_EE_M.odf bor_adaptor_EE.odf bor_adaptor_EO.odf bor_adaptor_OO.odf bor_adaptor_RE.odf bor_adaptor_RO.odf bor_adaptor_TE.odf bor_adaptor_TO.odf bor_adaptor_TR.odf bor_assemblerY.odf bor_assemblerYS.odf bor_assemblerZ.odf bor_assemblerZS.odf bor_assimilator_AAA.odf bor_assimilator_BAA.odf bor_assimilator_BRA.odf bor_assimilator_BTA.odf bor_assimilator_BTR.odf bor_assimilator_BTT.odf

Sample of 'template.txt'

unitname tooltip verbosetooltip race buildtime officercost crewcost dilithiumcost latinumcost metalcost biomattercost maxHealth curHealth healthrate maxshield curShield shieldrate maxSpecialEnergy specialEnergyRate specialenergydisplaymode

Replies are listed 'Best First'.
Re: re-ordering lines of text in a file? (updated)
by haukex (Archbishop) on Nov 29, 2016 at 19:35 UTC

    Hi moddingforfun,

    PerlMonks is mostly a place to learn, and usually* not a code writing service (there are other websites where you can get Perl coders to write things for you for not too much money if you are so inclined).

    * Sometimes, if you're lucky, some monks will help with working code. But you'll get much more help if you show your own efforts - of course we'd be happy to welcome another Perl coder here, so perhaps you should give perlintro another go :-) If you can write BASIC, that's a start - that's what I first started with many years ago, and if you know Bash, there are some similarities to Perl there. Try writing a few lines of code, we'll be happy to help.

    Also, we can point you in the right direction. For example, in your question, you don't specify how this reordering is supposed to happen. Have a look at How do I post a question effectively?, at the very least you'd have to give some short sample input, describe how the reordering is supposed to happen, and show the corresponding expected output.

    For listing the files in a directory, there's e.g. the glob function. For accessing and reordering the lines in each file, perhaps the module Tie::File can help - it provides mostly transparent access to the lines of a file via a Perl array.

    Update 2016-11-30: I think I may have made it sound too much like you won't get any code here. Of course that's not the case, but as I said it's best if you first show your efforts (= good questions + your code) - at least those are my favorite questions and that's what often motivates me to want to help.

    Hope this helps,
    -- Hauke D

Re: re-ordering lines of text in a file?
by Laurent_R (Canon) on Nov 29, 2016 at 20:16 UTC
    Yes, it is quite easy to do that in Perl. Probably not much more than a dozen code lines.

    One way to do it would be first to read the template file and store its contents in memory (probably as an array), so that you can then read it in order for each of your data files.

    Then read your file list line by line, and, for each line, read the corresponding file content into a hash. Then scan the array, lookup the corresponding file content, put the file content into the right order, and dump it to a file.

    One thing to take into account is that you normally can't edit a file in place (well, you can with a work around, but it's better and safer not to do it). So it is probably easier to output your result into files having the same names as the input files, but in another directory. Then you remove the old files only when you're confident that they have been successfully converted. This is much safer.

    We would really need a sample of the input file to help further.

      In response to your request for better samples and further info...

      Game Website http://www.fleetops.net

      Further explanation of the goal .... When you open a book to read, you expect to find certain things in a certain order. Preface / Forward first. Table of Contents next. After that, the chapters of the book, starting with Chapter 1. etc. That in essence is what I am trying to achieve with these files.

      The template is the ?skeleton?framework?blueprint?form? for the desired end result.

      Un-formated/non-reordered files :

      The full template file :

      Hand Edited File : rom_generix_beamr6.odf

      Look at the "value" of the following variable's in the samples.

      possiblecraftnames = hulltargethardpoints = weaponhardpoints1 = weaponhardpoints2 = etc.

      In attempts to use a bash shell script ( while read / grep AND/OR sed / >> to different directories ) the "value" part of the key pair turned out to be the downfall.

Re: re-ordering lines of text in a file?
by Marshall (Canon) on Nov 30, 2016 at 00:06 UTC
    With Perl it is possible to re-order (sort) any list of stuff according to a set of sorting rules. Of course things like "a"<"b" are the easiest (letter a is less than letter b). But this can be arbitrarily complex.

    I took your Template as the "sorting rules". Shuffled that array, then applied the sorting rules to arrive back at the input array. I have no idea of what you are trying to do. Your description was not understandable to me. But Perl can re-order (order) anything as long as you can describe what the ordering rules are.

    #!/usr/bin/perl use strict; use warnings; use Data::Dumper; use List::Util qw(shuffle); my $sort_order=<<END; unitname tooltip verbosetooltip race buildtime officercost crewcost dilithiumcost latinumcost metalcost biomattercost maxHealth curHealth healthrate maxshield curShield shieldrate maxSpecialEnergy specialEnergyRate specialenergydisplaymode END #generate sorting rules... my $i=0; my %sort_order = map{$_ => $i++}split ' ',$sort_order; my @keys = shuffle keys %sort_order; # shuffle not really needed, # but adds some credibility as to # the random nature of output print "SHUFFLED ARRAY...\n"; # randomized array print join("\n",@keys),"\n"; print "\n","RE-ORDERED ARRAY BY SORTING RULES\n"; print "THIS IS NOT A COPY OF THE INPUT ARRAY\n"; @keys = sort{$sort_order{$a} <=> $sort_order{$b}}@keys; print join("\n",@keys),"\n"; __END__ SHUFFLED ARRAY... shieldrate officercost maxHealth race curHealth metalcost curShield verbosetooltip latinumcost healthrate biomattercost unitname dilithiumcost maxSpecialEnergy specialenergydisplaymode tooltip specialEnergyRate buildtime crewcost maxshield RE-ORDERED ARRAY BY SORTING RULES THIS IS NOT A COPY OF THE INPUT ARRAY unitname tooltip verbosetooltip race buildtime officercost crewcost dilithiumcost latinumcost metalcost biomattercost maxHealth curHealth healthrate maxshield curShield shieldrate maxSpecialEnergy specialEnergyRate specialenergydisplaymode Process completed successfully
Re: re-ordering lines of text in a file?
by Not_a_Number (Prior) on Nov 29, 2016 at 19:30 UTC

    I think we need a sample of what one of the .odf files contains, in order to help...

Re: re-ordering lines of text in a file?
by perldigious (Priest) on Nov 29, 2016 at 22:23 UTC

    Hmm... looking at some of the lines of 'template.txt'. You are attempting to mod some sort of Star Trek video game I presume? If so you may just want to tell us which game and what you are trying to do exactly. If any Monk here happens to be familiar with or even have the game they can probably go look at the files themselves and see exactly what you are talking about.

    Beyond that, I know you probably hate the idea of putting in the time to learn the rudiments of a computer language, but Perl isn't so difficult to pick up, and even something like this very short tutorial can teach you everything you need and more to do what you are asking. The far greater upside of sticking it out and learning Perl yourself would be that if you are doing this sort of thing a lot, you will be amazed at how much more adept and proficient you can be at it with the Swiss Army Chainsaw that is Perl in your toolbox. :-)

    Just another Perl hooker - will code for food

      In response to your request ...

      The website is http://www.fleetops.net

      The tutorial ... a question ... The "arrays" ... input is a fixed expression? By that I mean 'pattern match'. grep for '^green' in a file, that is exactly what you get. The result is the first 'green' starting a line in that file. The problem I face, is the rest of the line. The rest of the line has the "value" of the key-pair.

      I posted more info/lengthy examples in another response on this same ?node?thread?
        I am puzzled how this could ... be input without major problems

        If you need to create a separate record for each value then try

        #!perl use strict; use DBI; use Text::ParseWords 'quotewords'; open OUT,'>','output.txt' or die "$!"; my @infiles = glob("*.odf"); for my $infile (@infiles){ open IN,'<',$infile or die "Could not open $infile : $!"; print "Reading $infile.."; my $count=0; while (<IN>){ chomp; ++$count; my ($key,$string) = split /\s*=\s*/,$_; my @values = quotewords('\s+',0,$string); for my $i (1..@values){ printf OUT "%-30s %-35s %-35s %5d\n", $infile,$key,$values[$i-1],$i; } } print "$count lines read\n"; }
        poj
Re: re-ordering lines of text in a file?
by moddingforfun (Initiate) on Dec 02, 2016 at 14:34 UTC

    Thanks to all Posters.

    Your comments got the old mental gears turning. I figured out *why* the shell script wasn't working, due to comments made here.

    Hint : replaced grep 'term-goes-here' with grep 'term-goes-here.*' .... that gets the entire line of text.

    Learned something anyways ... Thanks to All.