Skip to content

Commit 1c7074d

Browse files
author
Commitfest Bot
committed
[CF 5938] v39 - Enhancing Memory Context Statistics Reporting
This branch was automatically generated by a robot using patches from an email thread registered at: https://commitfest.postgresql.org/patch/5938 The branch will be overwritten each time a new patch version is posted to the thread, and also periodically to check for bitrot caused by changes on the master branch. Patch(es): https://www.postgresql.org/message-id/CAH2L28t=c_iBj0+LoKX0TSTrNJB2nTgH6MPD7QgRb8kSBZ7HVg@mail.gmail.com Author(s): Rahila Syed
2 parents 3479a0f + 643be11 commit 1c7074d

File tree

31 files changed

+1484
-24
lines changed

31 files changed

+1484
-24
lines changed

‎doc/src/sgml/func/func-admin.sgml‎

Lines changed: 157 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -251,6 +251,130 @@
251251
<literal>false</literal> is returned.
252252
</para></entry>
253253
</row>
254+
255+
<row>
256+
<entry role="func_table_entry"><para role="func_signature">
257+
<indexterm>
258+
<primary>pg_get_process_memory_contexts</primary>
259+
</indexterm>
260+
<function>pg_get_process_memory_contexts</function> ( <parameter>pid</parameter> <type>integer</type>, <parameter>summary</parameter> <type>boolean</type> )
261+
<returnvalue>setof record</returnvalue>
262+
( <parameter>name</parameter> <type>text</type>,
263+
<parameter>ident</parameter> <type>text</type>,
264+
<parameter>type</parameter> <type>text</type>,
265+
<parameter>level</parameter> <type>integer</type>,
266+
<parameter>path</parameter> <type>integer[]</type>,
267+
<parameter>total_bytes</parameter> <type>bigint</type>,
268+
<parameter>total_nblocks</parameter> <type>bigint</type>,
269+
<parameter>free_bytes</parameter> <type>bigint</type>,
270+
<parameter>free_chunks</parameter> <type>bigint</type>,
271+
<parameter>used_bytes</parameter> <type>bigint</type>,
272+
<parameter>num_agg_contexts</parameter> <type>integer</type> )
273+
</para>
274+
<para>
275+
This function handles requests to display the memory contexts of a
276+
<productname>PostgreSQL</productname> process with the specified
277+
process ID. The function can be used to send requests to backends as
278+
well as <glossterm linkend="glossary-auxiliary-proc">auxiliary processes</glossterm>.
279+
</para>
280+
<para>
281+
The returned record contains extended statistics per each memory
282+
context:
283+
<itemizedlist spacing="compact">
284+
<listitem>
285+
<para>
286+
<parameter>name</parameter> - The name of the memory context.
287+
</para>
288+
</listitem>
289+
<listitem>
290+
<para>
291+
<parameter>ident</parameter> - Memory context ID (if any).
292+
</para>
293+
</listitem>
294+
<listitem>
295+
<para>
296+
<parameter>type</parameter> - The type of memory context, possible
297+
values are: AllocSet, Generation, Slab and Bump.
298+
</para>
299+
</listitem>
300+
<listitem>
301+
<para>
302+
<parameter>level</parameter> - The level in the tree of the current
303+
memory context.
304+
</para>
305+
</listitem>
306+
<listitem>
307+
<para>
308+
<parameter>path</parameter> - Memory contexts are organized in a
309+
tree model with TopMemoryContext as the root, and all other memory
310+
contexts as nodes in the tree. The <parameter>path</parameter>
311+
displays the path from the root to the current memory context. The
312+
path is limited to 100 children per node, which each node limited
313+
to a max depth of 100, to preserve memory during reporting. The
314+
printed path will also be limited to 100 nodes counting from the
315+
TopMemoryContext.
316+
</para>
317+
</listitem>
318+
<listitem>
319+
<para>
320+
<parameter>total_bytes</parameter> - The total number of bytes
321+
allocated to this memory context.
322+
</para>
323+
</listitem>
324+
<listitem>
325+
<para>
326+
<parameter>total_nblocks</parameter> - The total number of blocks
327+
used for the allocated memory.
328+
</para>
329+
</listitem>
330+
<listitem>
331+
<para>
332+
<parameter>free_bytes</parameter> - The amount of free memory in
333+
this memory context.
334+
</para>
335+
</listitem>
336+
<listitem>
337+
<para>
338+
<parameter>free_chunks</parameter> - The number of chunks that
339+
<parameter>free_bytes</parameter> corresponds to.
340+
</para>
341+
</listitem>
342+
<listitem>
343+
<para>
344+
<parameter>used_bytes</parameter> - The total number of bytes
345+
currently occupied.
346+
</para>
347+
</listitem>
348+
<listitem>
349+
<para>
350+
<parameter>num_agg_contexts</parameter> - The number of memory
351+
contexts aggregated in the displayed statistics.
352+
</para>
353+
</listitem>
354+
</itemizedlist>
355+
</para>
356+
<para>
357+
When <parameter>summary</parameter> is <literal>true</literal>, statistics
358+
for memory contexts at levels 1 and 2 are displayed, with level 1
359+
representing the root node (i.e., <literal>TopMemoryContext</literal>).
360+
Statistics for contexts on level 2 and below are aggregates of all
361+
child contexts' statistics, where <literal>num_agg_contexts</literal>
362+
indicate the number aggregated child contexts. When
363+
<parameter>summary</parameter> is <literal>false</literal>,
364+
<literal>the num_agg_contexts</literal> value is <literal>1</literal>,
365+
indicating that individual statistics are being displayed.
366+
</para>
367+
<para>
368+
After receiving memory context statistics from the target process, it
369+
returns the results as one row per context. If all the contexts don't
370+
fit within the pre-determined size limit, the remaining context
371+
statistics are aggregated and a cumulative total is displayed. The
372+
<literal>num_agg_contexts</literal> column indicates the number of
373+
contexts aggregated in the displayed statistics. When
374+
<literal>num_agg_contexts</literal> is <literal>1</literal> it means
375+
that the context statistics are displayed separately.
376+
</para></entry>
377+
</row>
254378
</tbody>
255379
</tgroup>
256380
</table>
@@ -302,6 +426,39 @@ LOG: Grand total: 1651920 bytes in 201 blocks; 622360 free (88 chunks); 1029560
302426
because it may generate a large number of log messages.
303427
</para>
304428

429+
<para>
430+
<function>pg_get_process_memory_contexts</function> can be used to request
431+
memory contexts statistics of any <productname>PostgreSQL</productname>
432+
process. For example:
433+
<programlisting>
434+
postgres=# SELECT * FROM pg_get_process_memory_contexts(
435+
(SELECT pid FROM pg_stat_activity
436+
WHERE backend_type = 'checkpointer'),
437+
false) LIMIT 1;
438+
-[ RECORD 1 ]----+------------------------------
439+
name | TopMemoryContext
440+
ident |
441+
type | AllocSet
442+
level | 1
443+
path | {1}
444+
total_bytes | 90304
445+
total_nblocks | 3
446+
free_bytes | 2880
447+
free_chunks | 1
448+
used_bytes | 87424
449+
num_agg_contexts | 1
450+
</programlisting>
451+
<note>
452+
<para>
453+
While <function>pg_get_process_memory_contexts</function> can be used to
454+
query memory contexts of the local backend,
455+
<structname>pg_backend_memory_contexts</structname>
456+
(see <xref linkend="view-pg-backend-memory-contexts"/> for more details)
457+
will be less resource intensive when only the local backend is of interest.
458+
</para>
459+
</note>
460+
</para>
461+
305462
</sect2>
306463

307464
<sect2 id="functions-admin-backup">

‎src/backend/catalog/system_views.sql‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -692,6 +692,11 @@ GRANT SELECT ON pg_backend_memory_contexts TO pg_read_all_stats;
692692
REVOKE EXECUTE ON FUNCTION pg_get_backend_memory_contexts() FROM PUBLIC;
693693
GRANT EXECUTE ON FUNCTION pg_get_backend_memory_contexts() TO pg_read_all_stats;
694694

695+
REVOKE EXECUTE ON FUNCTION
696+
pg_get_process_memory_contexts(integer, boolean) FROM PUBLIC;
697+
GRANT EXECUTE ON FUNCTION
698+
pg_get_process_memory_contexts(integer, boolean) TO pg_read_all_stats;
699+
695700
-- Statistics views
696701

697702
CREATE VIEW pg_stat_all_tables AS

‎src/backend/postmaster/autovacuum.c‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -791,6 +791,10 @@ ProcessAutoVacLauncherInterrupts(void)
791791
if (LogMemoryContextPending)
792792
ProcessLogMemoryContextInterrupt();
793793

794+
/* Publish memory contexts of this process */
795+
if (PublishMemoryContextPending)
796+
ProcessGetMemoryContextInterrupt();
797+
794798
/* Process sinval catchup interrupts that happened while sleeping */
795799
ProcessCatchupInterrupt();
796800
}

‎src/backend/postmaster/checkpointer.c‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -679,6 +679,10 @@ ProcessCheckpointerInterrupts(void)
679679
/* Perform logging of memory contexts of this process */
680680
if (LogMemoryContextPending)
681681
ProcessLogMemoryContextInterrupt();
682+
683+
/* Publish memory contexts of this process */
684+
if (PublishMemoryContextPending)
685+
ProcessGetMemoryContextInterrupt();
682686
}
683687

684688
/*

‎src/backend/postmaster/interrupt.c‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,10 @@ ProcessMainLoopInterrupts(void)
4848
/* Perform logging of memory contexts of this process */
4949
if (LogMemoryContextPending)
5050
ProcessLogMemoryContextInterrupt();
51+
52+
/* Publish memory contexts of this process */
53+
if (PublishMemoryContextPending)
54+
ProcessGetMemoryContextInterrupt();
5155
}
5256

5357
/*

‎src/backend/postmaster/pgarch.c‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -871,6 +871,10 @@ ProcessPgArchInterrupts(void)
871871
if (LogMemoryContextPending)
872872
ProcessLogMemoryContextInterrupt();
873873

874+
/* Publish memory contexts of this process */
875+
if (PublishMemoryContextPending)
876+
ProcessGetMemoryContextInterrupt();
877+
874878
if (ConfigReloadPending)
875879
{
876880
char *archiveLib = pstrdup(XLogArchiveLibrary);

‎src/backend/postmaster/startup.c‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -192,6 +192,10 @@ ProcessStartupProcInterrupts(void)
192192
/* Perform logging of memory contexts of this process */
193193
if (LogMemoryContextPending)
194194
ProcessLogMemoryContextInterrupt();
195+
196+
/* Publish memory contexts of this process */
197+
if (PublishMemoryContextPending)
198+
ProcessGetMemoryContextInterrupt();
195199
}
196200

197201

‎src/backend/postmaster/walsummarizer.c‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -879,6 +879,10 @@ ProcessWalSummarizerInterrupts(void)
879879
/* Perform logging of memory contexts of this process */
880880
if (LogMemoryContextPending)
881881
ProcessLogMemoryContextInterrupt();
882+
883+
/* Publish memory contexts of this process */
884+
if (PublishMemoryContextPending)
885+
ProcessGetMemoryContextInterrupt();
882886
}
883887

884888
/*

‎src/backend/storage/ipc/ipci.c‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,7 @@
5151
#include "storage/sinvaladt.h"
5252
#include "utils/guc.h"
5353
#include "utils/injection_point.h"
54+
#include "utils/memutils.h"
5455

5556
/* GUCs */
5657
int shared_memory_type = DEFAULT_SHARED_MEMORY_TYPE;
@@ -150,6 +151,7 @@ CalculateShmemSize(int *num_semaphores)
150151
size = add_size(size, InjectionPointShmemSize());
151152
size = add_size(size, SlotSyncShmemSize());
152153
size = add_size(size, AioShmemSize());
154+
size = add_size(size, MemoryContextKeysShmemSize() + sizeof(LWLockPadded));
153155

154156
/* include additional requested shmem from preload libraries */
155157
size = add_size(size, total_addin_request);
@@ -343,6 +345,7 @@ CreateOrAttachShmemStructs(void)
343345
WaitEventCustomShmemInit();
344346
InjectionPointShmemInit();
345347
AioShmemInit();
348+
MemoryContextKeysShmemInit();
346349
}
347350

348351
/*

‎src/backend/storage/ipc/procsignal.c‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -691,6 +691,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
691691
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
692692
HandleLogMemoryContextInterrupt();
693693

694+
if (CheckProcSignal(PROCSIG_GET_MEMORY_CONTEXT))
695+
HandleGetMemoryContextInterrupt();
696+
694697
if (CheckProcSignal(PROCSIG_PARALLEL_APPLY_MESSAGE))
695698
HandleParallelApplyMessageInterrupt();
696699

0 commit comments

Comments
 (0)