... I have tried this using

perl -pe's/\x08//g' <20150625163139.xml >20150625163139.xml

But this gives error: Bad name after g' at <script_name>.pl line 13.

How are you invoking the above command? A Perl command invoked from the OS command line via the  -e switch should have a "script name" of "-e" and exactly one line. (Well, one line in Windows, anyway — what is your OS?) Also, you are still redirecting I/O to and from the same file at the same time, and this is still a bad idea (see hippo's previous reply).

An error message like "Code point \u0016 is not a valid character ..." suggests a source file character encoding problem: what is the encoding of 20150625163139.xml? Proper specification of MODE in the
    open FILEHANDLE,MODE,EXPR;
command (see open) should insure proper reading of any Unicode encoding from its file, but I have no idea if XML::Writer handles Unicode strings properly.

Either your sample data doesn't contain one of the 'bad' lines, or something else is wrong. I just installed the most recent XML modules on perl 5.18.2 and I don't get any errors or issues.

Make sure you've posted at least one of the lines with the problem, and if you have, try upgrading your modules and see if that helps.

-stevieb

some_proeccess < some.file > some.file
[download]

For another, when you see something like \u0016, that's a hexadecimal value, You can "grep" for that using a perl one-line like the following:

perl -CS -ne 'print if /\x{0016}/' < some.file
[download]

perl -ne 'print if /\x16/' some.file
[download]

You can look up how valid vs. invalid XML characters are enumerated (many people use regexes - e.g. on stackoverflow), and run a diagnosis on your file(s) before feeding them to your script (or just add code to your script to filter out the bad characters, if you're sure that just deleting them is The Right Thing To Do).

Thank you for your response graff. Finally I was able to remove the character that was causing this error "Code point \u0016 is not a valid character in XML".

I just added some code before

my $out = IO::File->new()

The code I added is as below:

`strings $20150625163139.txt > Temp.txt`;

my $temp_file = "Temp.txt";
open($fh, ">", $20150625163139.txt) or die "Could not open file '$file
+name' $!";
open(my $fh1, "<", $temp_file) or die "Could not open file 'Temp.txt' 
+$!";

while(<$fh1>){
    print $fh $_;
}
close $fh;
close $fh1;
unlink $temp_file;
[download]

So "strings $filename > Temp.txt" removes that character causing error and like many occasions UNIX came to my help this time as well :-)

Thanks you all for your inputs! If there is any way to mark a node as closed, please let me know and I'll mark this one as closed.

The usual way is to update the original question's title with (Solved).
لսႽ† ᥲᥒ⚪⟊Ⴙᘓᖇ Ꮅᘓᖇ⎱ Ⴙᥲ𝇋ƙᘓᖇ

Context: [Solved]:What regex changes are needed while creating xml

Alexander

Thanks for all your inputs. @stevieb: I think the problem is at line 50 in the file which I am guessing from the error I see after adding :encoding(UTF-8) in filehandle at line 17 which now looks like open(my $fh, "<:encoding(UTF-8)", "20150626102938.txt")

(Thanks to AnomalousMonk for sugestion)

I am on perl v5.10.1 at present and will see if can get it upgraded but it'll be a while before I can get it done.

@AnomalousMonk: I was trying to do this

perl -pe's/\x08//g' <20150625163139.xml >20150625163139.xml

from the code. And I tried using two different file names as well. I just kept getting same error so just left it there. But if I can get it working, then I'll ensure to use two different files there.

I am sorry for causing confusion about 20150625163139.xml. It is just an existing text file (generated at run time) which I have to use to generate xml. I can change it's extension to .txt though.

I tried adding

:encoding(UTF-8)

in filehandle at line 17 which now looks like

open(my $fh, "<:encoding(UTF-8)", "20150626102938.txt")

Before adding the output of script was:

Code point \u0016 is not a valid character in XML at ./Call_to_snmpwal
+k_V_1.pl line 34
[download]

After adding the output of script was:

utf8 "\xB8" does not map to Unicode at ./Call_to_snmpwalk_V_1.pl line 
+35, <$fh> line 50.
utf8 "\xF3" does not map to Unicode at ./Call_to_snmpwalk_V_1.pl line 
+35, <$fh> line 50.
Code point \u0016 is not a valid character in XML at ./Call_to_snmpwal
+k_V_1.pl line 34
[download]

@sundialsvc4: I am sorry for confusion caused by 20150625163139.xml It's just text file with no <?xml version="1.0" encoding="UTF-16" standalone="no"?> at the top I tried adding that tag manually at the top in file 20150625163139.xml just to give it a try but it still gives same error with and without :encoding(UTF-8) added in filehandle

@afoken: Thanks for adding the context link.

Hi all,

I got the line in the input file which is causing this error:

Code point \u0016 is not a valid character in XML at ./Call_to_snmpwalk_V_1.pl line 35

The line is:

GI-eSTB-MIB-NPH::eSTBOobNetworkAddress.0 = STRING: ¸^V^Y(^Xó

I made a few changes so as to print the generated xml in xml file instead of console and can see that the file generations stops at run time at the above line. That generated xml looks like:

<doc>
    <GI-eSTB-MIB-NPH>
        <eSTBOobNetworkAddress>
            <0>
[download]

Updated code now is:

#!/usr/bin/perl

use strict;
use warnings;
use XML::Writer;
use XML::Simple;
use XML::LibXML;
use IO::File;

my $out = IO::File->new(">output.xml");
my $xml = XML::Writer->new(OUTPUT => $out, DATA_MODE => 1, DATA_INDENT
+ => 4);

$xml->xmlDecl();
$xml->startTag('doc');

my $check_1 = 0;

open(my $fh, "<", "20150626161859.txt")
    or die "Failed to open file: $!\n";
while(<$fh>) {
    chomp;
    next if !length;

    my ($string1, $string2, $subscript_name, $subscript_value) = /
        ^(.*?)::
        ([^\s]+)
        \.([^\s]+)\s+=
        \s(.*)
        /x;
    if ( $check_1 == 0 ) {
        $xml->startTag($string1);
        $check_1 += 1;
    }
    $xml->startTag($string2);
    $xml->dataElement($subscript_name => $subscript_value);
    $xml->endTag();
}

$xml->endTag();
$xml->endTag();
$xml->end();

close $fh;
$out->close();
[download]

At least I know what exact line/characters are causing trouble now. I trying to find how to avoid this and will update here if I finally get a solution for it.

That file doesn’t look terribly like XML to me.   Nevertheless, what about the <?XML ... line at the very top?   Does it include an encoding= clause?

e.g.:   <?xml version="1.0" encoding="UTF-16" standalone="no"?>

It certainly appears that the file does contain UTF ... but does the file declare, in its header, that it does?   If it does not, then that erroneous omission certainly could fool many XML parsers.