That is O.K. But i'm still convinced that JSON is the way to go now. On the other hand decades ago i "endorsed" XML in the companies i was with. Yet another "reach into the toilet".

Yeah, XML sucks. There is absolutely no way to encode the NUL character ("\0"), it is simply illegal. You have to resort to base64 or some custom escaping. It has lots of redundancy, whitespace handling can drive you insane (CDATA, PCDATA ...), and a few lines of XML can kill your parser (Billion laughs attack). But at least, you can have comments.

JSON does not allow comments. And no, whatever garbage JSON::XS accepts with '#' followed by junk followed by a CR or LF is not JSON. JSON is defined as a subset of Javascript. Why on earth would one resort to Unix shell scripts for a comment syntax? Javascript has well-defined comments, so the most natural extension to JSON would be Javascript comments, not shell comments. (See also Re^3: Freezing a regular expression qr().)


Now, JSON instead of CSV. Feel free to propagate JSON, especially as a replacement for CSV. I'm fine with that. You should perhaps start by sending in some patches for various projects:

Gnuplot reads data from CSV files, but can't read JSON. It's a great tool for debugging the differences between real-world physics and the concept's idea of physics. Have your embedded system spit out some numbers of what it senses from the real world, feed them through Gnuplot, and compare the graphs with what is drawn in the concept.

LibreOffice Calc also lacks JSON import, and probably also JSON export. Calc can be used instead of Gnuplot, or to preprocess data for Gnuplot, when debugging physics. Most likely, the same is true for OpenOffice.

When you're done there, complain at Microsoft that Excel does not support JSON like Calc will then do.

Some databases already support JSON as a data type (i.e. content of a table cell). But can they dump out views, tables, query results as JSON instead of CSV? And can they read JSON back like they can read back CSV? Start with patching PostgreSQL and MariaDB / MySQL. Both are very widely used, open source, and both lack JSON import and JSON export.

When you're done there, start nagging Microsoft (SQL Server, Access), Oracle, and IBM (DB/2).

What's the use case for databases? Well, DB admins in general don't want third parties to access their precious databases. Even if they don't look like Gollum, my precious. During my professional life as a software developer, I have never met a DB admin that simply allowed access to his precious. Don't even think about write access, even highly restricted read access either takes ages or requires escalation to management. If you are lucky, you can get a CSV dump every day, every hour, or on change of relevant data. Yes, this is far from optimal, but often sufficient.

Oh, and do you have implementations of cut, paste, join, sort, and uniq that work on JSON. No? Send patches ...

Don't forget that /etc/passwd, /etc/groups, /etc/shadow, /etc/gshadow are also CSV files. Send patches to make them JSON. It's 2019, after all, Unix is nearly half a century old now, time to get rid of old habits. Perhaps start with the systemd people, they don't understand the basics of Unix at all, so they should be easy to persuade.

Alexander

--
Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)

In reply to Re^10: Complex file manipulation challenge by afoken
in thread Complex file manipulation challenge by jdporter

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.