in reply to Calculating the average on a timeslice of data

All things considered, you might wish to take a long, hard look at SQLite.   This is, believe it or not, a public domain database system which stores everything in a single file and does not involve a server.   So, you get the goodness of “a flat file,” and the power of SQL.   You also have very nice interactive utilities available.   I find myself using it almost exclusively these days, in situations where I would normally use a flat file, and I almost always find the project taking a positive shift into a new direction because I can push more of the hard work off to SQL queries.

The only thing to bear in mind when using it ... but it is a big thing ... is the importance of “transactions.”   If you are updating or inserting something, you need to be doing it within a transaction.