postgresql.conf Dosyası - Log Ayarları

log_min_duration_statement

Açıklaması şöyle

log_min_duration_statement: just logging the statements whose duration is more than this value in milliseconds. This is very useful for detecting slow queries.

Açıklaması şöyle

It sets the minimum execution time in milliseconds (ms) above which all statements will be logged.

The default value for the log_min_duration_statement parameter is -1, which disables logging statements.

Setting the PostgreSQL parameter log_min_duration_statement to 0 will print all statements' durations.

Örnek

Şöyle yaparız

# set log_min_duration_statement=1;
SET

# show log_min_duration_statement;
 log_min_duration_statement
----------------------------
 1ms
(1 row)

PostgreSQL log dosyalarındaki statement log cümleleri şöyledir

LOG:  duration: 5.477 ms  statement: insert into abc values(1);
LOG:  duration: 8.656 ms  statement: insert into abc values(2);

log_statement

ALL, MOD,DDL değerlerini alabilir

Açıklaması şöyle

Most fintech companies, who are required to log all user and application activities, set the most verbose option log_statement=all. This is the easiest way to bloat the log files and invite storage issues if storage consumption is not monitored. Pg_audit can be a smarter way of logging user activities, where you can specify which class of activities you want to log, such as READ, WRITE, DDL, or FUNCTION. 

log_statement_sample_rate

Açıklama yaz

log_destination

Bu alanlar için Audit Trails yazısına bakabilirsiniz

log_destination Alanı

PostgreSQL 15 için açıklama şöyle

PostgreSQL supports several methods for logging server messages, including stderr (default), csvlog, and syslog. With this release, jsonlog is also supported which is convenient for exporting logs and debugging. In order to enable this, add the jsonlog under the log_destination inside the postgresql.conf file.

This will be a nice feature to have with exports to logging and monitoring tools like HoneyComb and DataDog etc.

Historical Table

Giriş

Yedek yani backup amaçlı değildir. Sadece eski veriyi farklı bir tabloya taşır

Örnek

Şöyle yaparız

> create table main_table_historical_data
as select * from main_table where create_date < '01-Jan-2020';
> delete from main_table where create_date < '01-Jan-2020';

pg_createsubscriber

Giriş

Açıklaması şöyle

The pg_createsubscriber command allows for easy transformation of physical replicas into logical replicas, making high-availability setups more manageable.

Best Practice: For large retail applications, using pg_createsubscriber helps in maintaining fail-safe backups without overhauling the infrastructure, providing continuous service even during maintenance.

pg_cron Extension

Giriş

Açıklaması şöyle

pg_cron is a PostgreSQL extension that allows scheduling of SQL queries to run at specific times, similar to cron jobs in Linux.

Örnek

Şöyle yaparız

SELECT cron.schedule('0 0 * * *', $$DELETE 

FROM cron.job_run_details 

WHERE end_time < now() - interval '10 days'$$);

pg_combinebackup komutu - Incremental Backup İçindir

Örnek

Önce full backup yapılır. Şöyle yaparız

pg_combinebackup /backups/fullbackup/ /backups/incr_backup1/ -o /combinebackup/

Açıklaması şöyle

In this command:

- /backups/fullbackup/ is the directory containing the full backup.

- /backups/incr_backup1/ is the first incremental backup directory.

- -o specifies the output directory where the combined backup will be stored.

Daha sonra PostgreSQL Service durdurulur. Şöyle yaparız

/usr/pgsql-17/bin/pg_ctl -D /var/lib/pgsql/17/data/ -l logfile stop

Dizine okuma ve yazma hakkı veriririz. Şöyle yaparız

chmod 700 /combinebackup

PostgreSQL Service tekrar başlatılır. Şöyle yaparız

/usr/pgsql-17/bin/pg_ctl -D /combinebackup/ -l /combinebackup/logfile start

PostgreSQL Service tarafından kullanılan dizini kontrol etmek için şöyle yaparız

SHOW data_directory;
 data_directory
----------------
 /combinebackup
(1 row)

pg_basebackup komutu - Incremental Backup İçindir

Örnek

Önce full backup yapılır. Şöyle yaparız

/usr/pgsql-17/bin/pg_basebackup -D /backups/fullbackup

/* 
The `-D` flag specifies the destination directory for the backup. 
Ensure that the directory exists and has the necessary permissions.
*/

Sonra incremental backup yapılır. Şöyle yaparız

pg_basebackup - incremental=/backups/fullbackup/backup_manifest -D /backups/incr_backup1/

/*
The ` - incremental` flag specifies the location of the full backup's 
manifest file, and `-D` is again used to specify the directory for the 
incremental backup.
*/

PostgreSQL Shared Buffer

Giriş

Açıklaması şöyle

When you are sending a read or write request to Postgres, you are never interacting with the files directly. In order to read something, first those pages need to be loaded in shared_buffers! 

Proper Amount for Shared Buffer

Açıklaması şöyle

It is recommended in the docs: If (RAM > 1GB) shared_buffers = 25%

Official Docs: Values larger than 40% of RAM might NOT Help

Açıklaması şöyle

Pro tip: If you are setting a PostgreSQL or MySQL, don't use the default database settings because those are meant for personal computers or notebooks.

One example is the Buffer Pool size:

- For MySQL, increase the innodb_buffer_pool_size

- For PostgreSQL, increase shared_buffers and effective_cache_size to match your OS cache size

Örnek

postgresql.conf dosyasında şöyle yaparız

shared_buffers = 256MB
effective_cache_size = 1GB

Örnek

Şöyle

What's the worst PostgreSQL configuration mistake you've seen in production?

I'll start: shared_buffers = 128 MB on a 64 GB server.

Default PostgreSQL configuration. Running in production. For two years. On a database serving 50,000 queries per second.

The database was using 128 MB of its own cache and relying entirely on the operating system's page cache for everything else. It worked — PostgreSQL is remarkably resilient — but it was leaving enormous performance on the table.

Changed shared_buffers to 16 GB (25% of RAM), adjusted effective_cache_size to 48 GB, and query response times dropped 40% overnight.

Other configuration horrors I've seen:

• 𝗺𝗮𝘅_𝗰𝗼𝗻𝗻𝗲𝗰𝘁𝗶𝗼𝗻𝘀 = 𝟭𝟬𝟬𝟬 with no connection pooler. Each connection using 10 MB of RAM. Server running out of memory under load.

• 𝗿𝗮𝗻𝗱𝗼𝗺_𝗽𝗮𝗴𝗲_𝗰𝗼𝘀𝘁 = 𝟰.𝟬 on an NVMe SSD. The planner thought random reads were 4x more expensive than sequential reads, avoiding index scans that would have been faster. Should have been 1.1.

• 𝘄𝗼𝗿𝗸_𝗺𝗲𝗺 = 𝟰 𝗠𝗕 on an analytics workload. Every complex query spilling to disk for sorts and hash joins. Changed to 256 MB and query times dropped from minutes to seconds.

• 𝗮𝘂𝘁𝗼𝘃𝗮𝗰𝘂𝘂𝗺 = 𝗼𝗳𝗳. Yes, someone turned it off. The table bloat was spectacular.

The PostgreSQL defaults are intentionally conservative. They're designed to run on a Raspberry Pi, not to perform well on your production server. Five settings — shared_buffers, effective_cache_size, work_mem, maintenance_work_mem, random_page_cost — capture 80% of the performance gains.

PG_BUFFERCACHE Extension

Shared Buffer kullanımını görmek için kullanılabilir