encoding iso 8859 issues within a dataset of more than 10000 lines - ho

with the Mechanize i get a dataset with the following set:

see a datachunk:

Loosdorftown Ledochowskastra�e 4 3382 Loosdorftown Telefonnummer: 02754 6257 FAX-Nummer: 02754 6257-4

linux-wyee:/home/martin/perl #
the script below gives back result like this one;
Loosdorf
Ledochowskastraße
3382 Loostown
Telefonnummer: 0002754 6257
FAX-Nummer: 0002754 6257-4

Well - we have following options here:

to print to a file instead of printing at the screen, we just have to change:

say $text;

to:

print $OUT_FILE $text;

Some explanations: where $OUT_FILE will be a filehandle for the output file that we will have to open before getting into the so called “for loop”.

This would work for the code as it is, but it might be different if we are using the Text:CSV module which has probably dedicated functions or methods for printing CSV lines to a file (Well to be frank i don’t use this module and don’t know it, although I should probably change this because I am using CSV files from time to time . Well i try to describe more in details what we want to have: Which output file to look like. Well i want the comma to separate the fields of the addresses, or the records?

if we take this for example: katholisch.at

we have the following dataset:

well i want to have seperated each datset into these bits - in other words: if i have a dataset that delimiters and seperates the lines that are given like that

Loosdorf Ledochowskastra�e 4 3382 Loosdorf Telefonnummer: 02754 6257 FAX-Nummer: 02754 6257-4

i would be very very happy. Note: there also a Encoding issues is: see the Ledochowskastra�e - there is a sign in it “ß” so we have to take care for the iso 8859 encoding dont we!?

Well i love if you can give some hints and helping hands. That would be very very supportive. Note;: this is a great gerat chance f or me to learn alot about Perl, and the options and power of Mechanize.

see more results:
Marias Neustift Neustifttown 28 4443 Marias Neussstift Telefonnummer: 007250/204 FAX-Nummer: 07250/204-4 E-Mail: prre.inmarianeustift@dioezese-linz.at
Marias Puchheim Gmundnertown Stra�e 1b 4800 Attnanger-Puchheim Telefonnummer: 007674/62334 FAX-Nummer: 07674/62334-4 E-Mail: prre.inmariapuchheim@dioezese-linz.at
Marias Scharten Schartenstown 1 4612 Schartensbook Telefonnummer: 007272/5210
Marias Schmolln Maria Schmollntown 2 5241 Maria Schmolln Telefonnummer: 007743/2209-12 FAX-Nummer: 07743/2209-17 E-Mail: prre.inmariaschmolln@dioezese-linz.at
Mattighofen R�merstra�e 12 5230 Mattighofentown Telefonnummer: 007742/2273 0676/87765221 FAX-Nummer: 07742/2273-22 E-Mail: peipfarre.inmattighofen@dioezese-linz.at
Mauerkirchens Pfarrhofstra�e 4 5270 Mauerkirchentown Telefonnummer: 007724/2262

well you see - we ve have a encoding iso 8859 issue here.

what can we do!? At the end of the day - i have to get all in a CVS formate

**btw: **a friend also suggested me using Text::CSV which will load up Text::CSV_XS or,

Well at the moment all the results will only print the data to stdout (console) im sure that i can modify it… :slight_smile:

i just installed the** Text::CSV_XS
**
took it from here: Text::CSV_XS - search.cpan.org

love to hear from you

greetings

i can try the Text::CSV module too…

The Text::CSV module provides functions for both parsing and producing CSV data. However, we’ll focus on the parsing functionality here. The following code sample opens the prospects.csv file and parses each line in turn, printing out all the fields it finds.

 #!/usr/bin/perl     use strict;     use warnings;     use Text::CSV;      my $file = 'prospects.csv';      my $csv = Text::CSV->new();      open (CSV, "<", $file) or die $!;      while (<CSV>) {         if ($csv->parse($_)) {             my @columns = $csv->fields();             print "@columns

"; } else { my $err = $csv->error_input; print “Failed to parse line: $err”; } } close CSV;

Running the code produces the following output:

 Name Address Floors Donated last year Contact     Charlotte French Cakes 1179 Glenhuntly Rd 1 Y John     Glenhuntly Pharmacy 1181 Glenhuntly Rd 1 Y Paul     Dick Wicks Magnetic Pain Relief 1183-1185 Glenhuntly Rd 1 Y George     Gilmour's Shoes 1187 Glenhuntly Rd 1 Y Ringo 

And by replacing the line:

      print "@columns

";

with:

         print "Name: $columns[0]
Contact: $columns[4]

";

[/QUOTE]
we can get more particular about which fields we want to output. And while we’re at it, let’s skip past the first line of our csv file, since it’s only a list of column names.

#!/usr/bin/perl use strict; use warnings; use Text::CSV; my $file = ‘prospects.csv’; my $csv = Text::CSV->new(); open (CSV, “<”, $file) or die $!; while (<CSV>) { next if ($. == 1); if ($csv->parse($_)) { my @columns = $csv->fields(); print "Name: $columns[0]
Contact: $columns[4]
"; } else { my $err = $csv->error_input; print “Failed to parse line: $err”; } } close CSV;

Running this code will give us the following output:

Name: Charlotte French Cakes             Contact: John     Name: Glenhuntly Pharmacy             Contact: Paul     Name: Dick Wicks Magnetic Pain Relief             Contact: George&lt;br&gt;      Name: Gilmour's Shoes             Contact: Ringo 

well i can get some analogies what do you think!

It doesn’t look like there are so many Perl experts visiting this forum at the moment who would be able to help you, and I noticed that none of your other posts has received an answer yet. Why not asking for help in a Perl forum?

hiello dear please try again

thx for the hints

i will digg deeper into the stuff of manipulating text to cvs with perl - and yes: i will have a closer look at the other posts - and also a t perl-forums.

thx again
greets martin

well i can do the job with split - the function split too