Creating Storable objects from the C

zwon has asked for the wisdom of the Perl Monks concerning the following question:

Replies are listed 'Best First'.
Re: Creating Storable objects from the C by BrowserUk (Patriarch) on Dec 24, 2008 at 22:32 UTC
You're looking at the wrong tool. Storable serialises Perl data structures. Even if a C-callable Storable library existed, you'd have to convert your C-structure into a Perl data structure first before you could use it. Most relative simple C-structs--ie. those that do not contain pointers--are most easily and efficiently accessed from Perl using unpack. If the structure uses pointers, then serialising it to some text format, is probably far easier than converting it to Perl's internal format. Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. "Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."	[reply]
Re: Creating Storable objects from the C by tilly (Archbishop) on Dec 24, 2008 at 22:13 UTC
Writing a Storable file from C does not sound like an easy solution to me. Here are alternate ways to solve this problem. I am assuming throughout that there is some C library you have that provides an interface into this data structure. If it doesn't exist yet, you should be able to build one fairly easily. Use a standard data exchange format such as XML or YAML. As you have discovered, this can be slow. Come up with a format for the data and use pack/unpack to pass data between C and Perl (great if it works but it won't always work). Write an XS interface for a C library that has the complete specification for the data structure then access your C library directly from Perl. This is a lot of work but should be very fast. Use swig to generate an interface into your C library. This actually builds an XS interface. I don't hear about people doing this much for Perl, and in my one experience with such an interface we found some performance silliness that we cleaned up, but it may save you some work. Use Inline::C in Perl to access your data. This is simpler to set up than the other versions, is fast (except the first time you call it), but requires a C compiler and some (fairly minimal) C knowledge from your Perl developers. From your question I suspect that you're using Storable. Be warned that the direct C solutions will not necessarily play well with Storable if your Perl developers are using that. The solution to that is to build STORABLE_freeze and STORABLE_thaw hooks that serialize and deserialize your C data structures into a string that Storable can use.	[reply]
Re^2: Creating Storable objects from the C by zwon (Abbot) on Dec 24, 2008 at 22:43 UTC
Thanks, I suspected that there's no possibility to create Storable from C, but just wondered if it is. Yes, XML and YAML are too slow, but ikegamy just pointed me to JSON and it looks better, so maybe I will use it. Also I thought about XS, but this will require to design some format to transmit data and implement encoder/decoder, so I fear it would be just another *ML, maybe even not faster ;).	[reply]
Re: Creating Storable objects from the C by ikegami (Patriarch) on Dec 24, 2008 at 21:41 UTC
There are already C libraries for JSON and YAML, I believe. Implementing Storable in C sounds like a lot of unnecessary work if it hasn't been done already.	[reply]
Re^2: Creating Storable objects from the C by Your Mother (Archbishop) on Dec 24, 2008 at 22:44 UTC
Just adding to this JSON::XS was faster than Storable in the last benchmark I saw/ran. Benchmarks can be misleading (size, data, versions) but even if it's not faster it's probably roughly as fast and if there are C libs available it would be easy to see if it worked well for you.	[reply]
Re^3: Creating Storable objects from the C by diotalevi (Canon) on Dec 25, 2008 at 08:49 UTC
In the last benchmark I ran, Storable was faster than JSON::XS, fwiw. ⠤⠤ ⠙⠊⠕⠞⠁⠇⠑⠧⠊	[reply]
Re^4: Creating Storable objects from the C by zwon (Abbot) on Dec 25, 2008 at 09:34 UTC