mla12 has asked for the wisdom of the Perl Monks concerning the following question:

We had a case recently where a database view (PostgreSQL) was "optimized." And in fact, it made quite a bit of difference in the normal case. But in one particular situation, the planner ended up with a much worse plan. We didn't catch the problem until it was in production.

Is there any way to keep track of the planner statistics (in development environments) and raise an assertion if the plan cost exceeds some specified threshold?

IOW, if we know a query plan cost has an upper limit of, say, 37033 normally, I'd like to be alerted if that planner result changes materially (e.g., if the cost jumps to 60k).

I'm looking for an approach that's largely automatic, so we don't need to run "EXPLAIN" queries manually. I'm familiar with DBI's trace stuff, but I didn't see anything there that would help with this.

Thanks.

  • Comment on DBI and keeping track of planner statistics

Replies are listed 'Best First'.
Re: DBI and keeping track of planner statistics
by erix (Prior) on Jan 03, 2017 at 21:45 UTC

    In the main postgres configuration file, postgresql.conf, you could set log_min_duration_statement to some value larger than zero. The value is expressed in milliseconds, so for instance setting it to 1000 makes the server log all SQL statements that take longer than 1 second. It may or may not fit your case. (You'd also need some process monitoring that log, I guess.)

    The default setting of this parameter is -1 (=disabled):

    log_min_duration_statement = -1 # -1 is disabled, 0 logs all statem +ents # and their durations, > 0 logs only # statements running at least this number # of milliseconds

    And, by the way, you don't need to run "EXPLAIN" queries manually. EXPLAIN has nice machine-readable output formats like json and yaml. For example:

    echo " create table t(n integer primary key); insert into t select * from generate_series(1,10000); analyze t; --> gather table statistics explain (analyze, verbose, format yaml) select count(*) from t where n + between 10 and 20; " | psql -qtAX

    The output:

    - Plan: Node Type: "Aggregate" Strategy: "Plain" Partial Mode: "Simple" Parallel Aware: false Startup Cost: 8.51 Total Cost: 8.52 Plan Rows: 1 Plan Width: 8 Actual Startup Time: 0.040 Actual Total Time: 0.040 Actual Rows: 1 Actual Loops: 1 Output: - "count(*)" Plans: - Node Type: "Index Only Scan" Parent Relationship: "Outer" Parallel Aware: false Scan Direction: "Forward" Index Name: "t_pkey" Relation Name: "t" Schema: "public" Alias: "t" Startup Cost: 0.29 Total Cost: 8.48 Plan Rows: 10 Plan Width: 0 Actual Startup Time: 0.022 Actual Total Time: 0.029 Actual Rows: 11 Actual Loops: 1 Output: - "n" Index Cond: "((t.n >= 10) AND (t.n <= 20))" Rows Removed by Index Recheck: 0 Heap Fetches: 11 Planning Time: 0.481 Triggers: Execution Time: 0.149
Re: DBI and keeping track of planner statistics
by Anonymous Monk on Jan 08, 2017 at 21:00 UTC
    As a database engineer, you're expected to run EXPLAIN.