python data consumption

qbxk has asked for the wisdom of the Perl Monks concerning the following question:

I've found myself holding a beautiful data structure (parsed rss using the excellent Universal Feed Parser) but with no way to manipulate it since I don't really know python and that's a python module (not some handy script written in python)...

so after exploring a multitude of ways of coercing this data structure into a perl data structure i've decided it's time to stop and ask for help. so far, attempts have included:

YAML, my first choice - but PyYAML won't dump_safe and so the output isn't useful to me (since it's not standard YAML, d'oh) .. i could subclass python's yaml module and teach it how to dump these, but i don't know python, and don't really have time to learn it properly for this project
Python's Pickling routines - sounds like their version of Storable (pickles store well..), but Perl's unpickler doesn't work so well, 'specially for these "custom objects" ... IOW, fails for the same reason as YAML
pyperl - some mysterious set of tools that allow you embed perl code in a python script, and access it's native data structures as well... but i can't get it to compile ... we'll not go into that in this post.

the thing that kills me is that if you just print the object using python's native printer (which sort of behaves like Data::Dumper apparently) the output looks very similar to a perl hash, but not quite. doing some regex's and then eval'ing the resultant string is .. sub-par, but so tempting at this point. before i damn myself that way,

i put to the clergy: how have you coerced your python data structres into perl?

It's not what you look like, when you're doin' what you’re doin'.
It's what you’re doin' when you’re doin' what you look like you’re doin'!
- Charles Wright & the Watts 103rd Street Rhythm Band, Express yourself

Comment on python data consumption

Replies are listed 'Best First'.
Re: python data consumption by snoopy (Curate) on Jul 03, 2006 at 04:29 UTC
Have you considered Inline::Python? This module sets up an in-process Python interpretor, then directly examines the Python symbol table to establish Perl bindings. From the doco: `print "9 + 16 = ", add(9, 16), "\n"; print "9 - 16 = ", subtract(9, 16), "\n"; use Inline Python => <<'END_OF_PYTHON_CODE'; def add(x,y): return x + y def subtract(x,y): return x - y END_OF_PYTHON_CODE` [download]	[reply] [d/l]
Re: python data consumption by Arunbear (Prior) on Jul 02, 2006 at 21:54 UTC
Have you considered just parsing the RSS with Perl e.g. see XML::RSS::Parser. On the other hand Python is one of the easiest languages to learn - if you are comfortable with Perl, it should only take a few hours to pick up enough Python to happily manipulate your data structure.	[reply]
Re^2: python data consumption by qbxk (Friar) on Jul 02, 2006 at 23:47 UTC
believe me, i exhausted all rss options involving perl before finding the Universal Feed Parser which is truly the end-all, be-all of rss parsers. nothing we perl has even comes close... maybe in theory, (XML::RAI, XML::RSS ..and many more) but not in practice, rss is a jungle; you thought html/css browser compatibility was bad? It's not what you look like, when you're doin' what you’re doin'. It's what you’re doin' when you’re doin' what you look like you’re doin'! - Charles Wright & the Watts 103rd Street Rhythm Band, Express yourself	[reply]
Re^3: python data consumption by clinton (Priest) on Jul 03, 2006 at 08:36 UTC
I am interested to hear what you say about XML::RSS/RAI. I use these modules quite a lot, and (after adding in some encoding guessing routines), they have handled pretty much all of my requirements. Why is the Universal Feed Parser so much better (I've never used it)? And, having taken a brief look at the code, it looks like it'd be reasonably easy to translate into Perl (all 2500 lines), wouldn't it? Again, no experience of Python code either, but the syntax looks pretty similar and the modules that they use are also available on CPAN. Not that I'm suggesting that that is what you should do in order to meet a deadline..., but if it really is so good, it'd be great to have it available in perl. And the 4000 tests that they provide would be a great way to prove the perl version.	[reply]
Re: python data consumption by davorg (Chancellor) on Jul 03, 2006 at 14:29 UTC
You could raise a bug report against Perl::Serialise::Pickle - tho' it looks like the author is already aware of some problems. The more people that report the bug, the more chance there is that it'll get fixed. -- <http://dave.org.uk> "The first rule of Perl club is you do not talk about Perl club." -- Chip Salzenberg	[reply]
Re: python data consumption by Moron (Curate) on Jul 03, 2006 at 14:17 UTC
As I recall, python output data structures use lots of nested brackets. So how about getting eval to unravel them: `use Data::Dumper; my $python = '[1,[2,3]]'; my $perl = eval( $python . ';' ); print Dumper( $perl );` [download] result: `$VAR1 = [ 1, [ 2, 3 ] ];` [download] -M Free your mind	[reply] [d/l] [select]
Re: python data consumption by Anonymous Monk on Jul 03, 2006 at 17:30 UTC
perl also have a 'kind of' universal parser http://search.cpan.org/dist/XML-Feed/lib/XML/Feed.pm	[reply]
Re: python data consumption by hobbs (Monk) on Jul 06, 2006 at 01:04 UTC
If you can, get the python app to output YAML -- the code is available for that, judging by google. Then just use your favorite Perl module (YAML or YAML::Syck) to read that in. Odds are the data will make it completely unscathed.	[reply]