\x{bf} or U+00BF is the inverted question mark and U+00F3 is “ó”, so AFAICT that looks correct, doesn't it?

Yes it looks correct in my browser.

Ok, so do I understand correctly that you've tested the code you showed within the script I posted?

Yes.

(BTW, why not use DBI to connect directly to the database instead of going through a file?)

Working on that! The tool I'm currently using doesn't allow it, so I'm running a system command to execute the query. Researching direct DBI access but there are access issues within my org...

So are you still seeing the same problem as before, or is it working now?

Sadly problem still exists, yes. To summarize:

foreach my $row (@sorted_urls){
 $csv->parse($row);

 my @els = $csv->fields;          #using Text::CSV
 my(@splits) = split('\|',$row);  #using split on same $row

 print $q->p($els[0]);            #prints wrong
 print %q->p(splits[0]);          #prints right
}
[download]

Honestly this is not worth wasting a lot of time on, though I really appreciate your help... thank you.

So here it gets interesting. Is it possible to get us that list of url's online somewhere so I/we could test on them?

If not, would it be possible to install Data::Peek and show me/us the output of

foreach my $row (@sorted_urls) {
    DPeek ($row);
    $csv->parse ($row);

    my @csv = $csv->fields;         #using Text::CSV
    my @row = split m/\|/ => $row;  #using split on same $row

    DPeek "CSV:   $csv[0]";
    DPeek "SPLIT: $row[0]";
    }
[download]

And I also have no idea what $q->p () has as influence on the output and I also guess that %q->p is a typo.


Enjoy, Have FUN! H.Merijn

I can't really give you the whole shebang but here are a couple of URLs, including the first one which has the spanish characters.

https://www.ibm.com/support/knowledgecenter/es/search/¿Cuales son las 
+partes de una cadena de conexión??scope=SSGU8G_12.1.0|https://www.ibm
+.com/support/knowledgecenter/es/SSGU8G_12.1.0/com.ibm.jdbc_pg.doc/ids
+_jdbc_011.htm|0|1|1|0
https://www.ibm.com/support/knowledgecenter/search/onsmsync?scope=SSGU
+8G_12.1.0|https://www.ibm.com/support/knowledgecenter/SSGU8G_12.1.0/c
+om.ibm.sec.doc/ids_lb_002.htm|1|1|1|1
[download]

Thanks!

Sadly problem still exists, yes. To summarize:

That's strange. Could you please show a complete example of the code that doesn't work, i.e. the full script along with its output? Also, to play it safe, try upgrading your installations of Text::CSV and Text::CSV_XS.

Hi @haukex sorry I can't give you the whole script due to security and privacy concerns, but I can give you the salient parts of it.

#execute the query using Aginity Workbench; output saved to flat filfe
my($res) = system($cmd);

### read the output
my(@urls);
my($header);
open my $fh, "<:encoding(utf8)", "$resultsFile" || die("cannot open re
+sults file $resultsFile for reading.\n cmd: $cmd");
my($c)=0;   # just here for counting
my($d)=0;   # just here for counting
while(<$fh>){
 $c++;
 if($c == 1) {    # get header row
  $header = $_;
 }
 if ($_ =~ /\/search\//){
  push(@urls, $_);
 }
 else{
  $d++;
 }
}
close($fh);

# sort @urls based on the search string 
# e.g. https://www.ibm.com/support/knowledgecenter/es/search/¿Cuales s
+on las partes de una cadena de conexión??scope=SSGU8G_12.1.0|https://
+www.ibm.com/support/knowledgecenter/es/SSGU8G_12.1.0/com.ibm.jdbc_pg.
+doc/ids_jdbc_011.htm|0|1|1|0

my @sorted_urls =
  map  { $_->[0] }
  sort { $a->[1] cmp $b->[1] }
  map  { m|/search/\s*([^\?]+)\?|; [$_, $1] }
  @urls;

# parse and print
print $q->header(-charset    => 'utf-8');
print $q->start_html( -title      => 'SearchME',
                -style=>{'src'=>$stylesheet});

print $q->start_table();
foreach my $row (@sorted_urls){
# print TEMP $row;
 $csv->parse($row);
 print "<tr>";
 $count++;


 my @els = $csv->fields;

 my(@splits) = split('\|',$row);

 $els[0] =~ /\/search\/(.+)\?scope=/i;
 my($term) = $1;

 my($link) = $els[0];

 print "<td>";
 # print $link;
 print $q->a({-href=>$link,-target=>'_blank'},$term);
 print "</td>";

# print other @fields here inside <td></td>
}
print $q->end_table,
$q->end_html;
[download]

oh and I reinstalled both modules.