My immediate thought is that you should load these data into an SQLite database (file), then use a real statistics package – such as R – or even a spreadsheet to do the rest of the work. The statistical processes that you describe are familiar ones and the number-of-observations is actually considered very small. Custom-programming is a very inefficient and largely unnecessary way to approach such a task: you're building a "one-off solution" to a universal task.