in reply to Re^3: n-dimensional statistical analysis of DNA sequences (or text, or ...)
in thread n-dimensional statistical analysis of DNA sequences (or text, or ...)

The link is right as you just posted it

When you download the sequence file you need to uncompress it: gunzip hs_ref_GRCh38.p12_chr20.fa.gz

Then, the command in the brief usage section of my original post analyse_DNA_sequence.pl --input-fasta hs_ref_GRCh38.p12_chr20.fa --ngram-length 3 --output-state hs_ref_GRCh38.p12_chr20.fa.3.state --output-stats stats.txt means the following:

If you want to remember your findings from the first sequence file when you process a second and append new data to it, then you use the previously output ".state" file as input state (--input-state) for your new run. This is to allow processing gigabytes of sequences incrementaly in small batches.

bliako

Replies are listed 'Best First'.
Re^5: n-dimensional statistical analysis of DNA sequences (or text, or ...)
by Aldebaran (Curate) on Feb 07, 2019 at 00:00 UTC

    I've tried to replicate and extend the first results I got, but I don't seem to be calling this properly. The logic for Getopt::Long seems pretty opaque to me now. Do you see why this is fundamentally different than what worked before?

    $ ./2.analyse_text.pl --input-corpus 84-0.txt  --ngram-length 4 --outp
+ut-state 4.shelley.state
args are --input-corpus 84-0.txt --ngram-length 4 --output-state 4.she
+lley.state
Unknown option: input-corpus
Unknown option: ngram-length
Unknown option: output-state
Usage : ./2.analyse_text.pl <options>


Something wrong with command-line parameters...
$ cat 2.analyse_text.pl 
#!/usr/bin/env perl

# FILE: analyse_text.pl
# by bliako

use 5.011;
use warnings;
use Getopt::Long;
use Data::Dump qw/dump/;
use lib '.';

use Markov::Ndimensional;

my @args = @ARGV;
say "args are @args";

my $input_corpus_filename = undef;
my $input_state_filename = undef;
my $output_state_filename = undef;
my $output_stats_filename = undef;
my $separator = '\s';
my $internal_separator = '|';
my $seed = undef;
my $num_iterations = 100;
if( ! Getopt::Long::GetOptions(
    'input-state=s' => \$input_state_filename,
    'separator=s' => \$separator,
    'num-iterations=i' => $num_iterations,
    'seed=s' => \$seed,
    'help|h' => sub { print STDERR usage($0); exit(0) }
) ){ print STDERR usage($0) . "\n\nSomething wrong with command-line p
+arameters...\n"; exit(1); }
...

    [download]
[reply]
[d/l]

      yes. You call analyse_text.pl correctly but the Getopt::Long parameters in the code you are pasting does not belong to that script. It belongs to predict_text.pl

      let me know if you need more help

[reply]
[d/l]
[select]
        let me know if you need more help

        Thx, bliako, I want to understand ngrams better now that I'm calling the scripts correctly again. Your example in for Getopt::Long in my recent thread Re: chunking up texts correctly for online translation got me through the bog and back on pavement. These scripts have abundant output, which I've edited down. I will always post the source that created it. I do have questions too, but the whole enchilada definitely calls for readmore tags:

        Read more... (24 kB)

        Just another perl meditation....

[reply]
[d/l]
[select]

In Section Meditations