|
Re: Converting Files
by jZed (Prior) on May 30, 2007 at 13:41 UTC
|
The module Text::CSV_XS will help you create the CSV file. As for converting the "binary" file - how could we possibly suggest anything without knowing what format the file is in? Please describe the format as best you can and give a *small* data sample from it. | [reply] |
|
|
| [reply] |
Re: Converting Files
by samizdat (Vicar) on May 30, 2007 at 14:27 UTC
|
Courtney, the key to solving your problem is to understand the format of the binary file. If it's a common numerical analysis format, you just might get lucky and find a module on CPAN that will do the work for you.
If not, then you'll need to read the file in and deal with it byte by byte.
#!/usr/bin/perl
use warnings;
use strict;
my $buf;
my @barray;
my $byt;
my $acc = 0;
# file 'datafile.bin' must exist and be in same directory as program f
+ile.
open BFILE, "<./datafile.bin" or die "Can't find or open binary data f
+ile.\n";
# not necessary on UNIX but helps with line-delineated files on Doze.
binmode BFILE;
# program will create 'tfile.csv' in same directory
open OFILE, ">./tfile.csv" or die "Can't create output file.\n";
while ( read BFILE, $buf, 64 )
{
my $bindex = 0;
@barray = unpack('C*', $buf);
for ( my $batch = 0; $batch < 16; $batch++ )
{
$acc = 0;
for ( my $i = 0; $i < 4; $i++ )
{
if ( exists( $barray[ $bindex ] ) )
{
$byt = $barray[ $bindex++ ];
$acc *= 256;
$acc += $byt;
}
}
printf OFILE '"' . $acc . "\",";
}
printf OFILE "\n";
}
close BFILE;
close OFILE;
I didn't fully test it beyond sanity check, and if you use this in production, you'll want to add little niceties like not writing the very last comma and ignoring null bytes, but this should take each four bytes of the file, convert them to an integer, and write them out in groups of 16 per line. A Windows-based system might have a little different invocation; this is done on Linux:
din@foobar
$ emacs tfile.pl &
din@foobar
$ chmod +x tfile.pl
din@foobar
$ ./tfile.pl
din@foobar
$
Good luck, and welcome to the Monastery! :D
UPDATE: added caveat about null bytes, and comments.
UPDATE2: Changed $b to $byt (as per Albannach) so as not to use special sorting variable $b.
Don Wilde
"There's more than one level to any answer."
| [reply]
[d/l]
[select] |
|
|
You are exactly right! There may be some special cases that will throw your
unpack('C*', $buf);
off. Mainly, when you have a binary file there can be BCD (Binary Coded Decimal) contained within. This would generally be used for a float data type (money or fractional data). There also may be encoded date fields (generally they consist of 2 bytes).
Also watch out for multiple byte hex numbers. i.e. you have 2 bytes 0x01 and 0x02, but a multiple byte hex number will append those two and convert to decimal. (0001 0010) == (18)
Finally some binary files will contain pointers (a.k.a memory locations) to other locations in the same file or other files. If this is the case you will need to be able to handle that pointer so that you do not lose any data.
I would suggest first, parsing the file as samizdat has suggested. Most likely you will find garbled fields after you are done. These garbled fields will probably fall into one of the categories I have mentioned. From there on out, you become Sherlock Holmes and try to determine what each one of them is.
Some tools to help you look at the binary data are hexdump or a combination of dd + hexdump (if you know the block-size of each record).
i.e. (dd if=binaryfile bs=blocksize | hexdump -c | more)
If you are using a *nix variant that is. If you do not have a *nix OS, then install cygWin and use it for these command line tools.
| [reply]
[d/l]
[select] |
Re: Converting Files
by Corion (Patriarch) on May 30, 2007 at 13:56 UTC
|
For the input side, I've had good success in using the unpack function (see also pack) for unpacking the data. As you haven't told us what format your data is in, we can't help you there. Most likely you will want to read binmode too.
| [reply] |
Re: Converting Files
by Fletch (Bishop) on May 30, 2007 at 13:43 UTC
|
Guess what? Neither will anyone here. You've given no specifics about what this 'binary file' contains, nor what this 'Vector Signal Processor' expects as input. You've not provided any of what you've tried so far either.
Aside from a pointer to something like Text::CSV_XS you're probably not going to get much help without being more forthcoming.
| [reply] |
Re: Converting Files
by Courtney1306 (Initiate) on May 31, 2007 at 14:54 UTC
|
I've said all of the information I have. I don't know if I need to 'dump'. As I said before I'm completely new to this programming language, so I have no specifics. According to management, that should be enough information between the two posts I have left. | [reply] |
|
|
| [reply] |
Re: Converting Files
by Anonymous Monk on May 30, 2007 at 23:54 UTC
|
Courtney, I too am new to perl & am trying to us a binary file.
If you don't mind, I'd like to message you my email address so we can compare notes. I just now created a user name of Aim9b.
I used my work email so I won't have my password until tomorrow. You can read my anonymous post by searching for bits on or about 5/25. These folks have been very helpful.
Thank you. Aim9b.
| [reply] |
|
|
The basic format:
Proprietary binary data file composed of several header fields of unsigned binary data followed by a list of 2 byte signed data items.
I want to take the data items and convert them into a CSV file.
| [reply] |
|
|
The basic format: Proprietary binary data file composed of several header fields of unsigned binary data followed by a list of 2 byte signed data items.
Still not clear enough: how 'bout a dump? Are you under *NIX (e.g. Linux)? If so, then you should have od(1) at your hand and surely ports are available for other osen, including Windows as well as other tools.
| [reply]
[d/l] |