Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit 7aea8e4

Browse files
robertmhaasrobertmhaas
committed
Determine whether it's safe to attempt a parallel plan for a query.
Commit 924bcf4 introduced a framework for parallel computation in PostgreSQL that makes most but not all built-in functions safe to execute in parallel mode. In order to have parallel query, we'll need to be able to determine whether that query contains functions (either built-in or user-defined) that cannot be safely executed in parallel mode. This requires those functions to be labeled, so this patch introduces an infrastructure for that. Some functions currently labeled as safe may need to be revised depending on how pending issues related to heavyweight locking under paralllelism are resolved. Parallel plans can't be used except for the case where the query will run to completion. If portal execution were suspended, the parallel mode restrictions would need to remain in effect during that time, but that might make other queries fail. Therefore, this patch introduces a framework that enables consideration of parallel plans only when it is known that the plan will be run to completion. This probably needs some refinement; for example, at bind time, we do not know whether a query run via the extended protocol will be execution to completion or run with a limited fetch count. Having the client indicate its intentions at bind time would constitute a wire protocol break. Some contexts in which parallel mode would be safe are not adjusted by this patch; the default is not to try parallel plans except from call sites that have been updated to say that such plans are OK. This commit doesn't introduce any parallel paths or plans; it just provides a way to determine whether they could potentially be used. I'm committing it on the theory that the remaining parallel sequential scan patches will also get committed to this release, hopefully in the not-too-distant future. Robert Haas and Amit Kapila. Reviewed (in earlier versions) by Noah Misch.
1 parent b44d92b commit 7aea8e4

31 files changed

+3162
-2781
lines changed

‎doc/src/sgml/catalogs.sgml

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4994,6 +4994,23 @@
49944994
</entry>
49954995
</row>
49964996

4997+
<row>
4998+
<entry><structfield>proparallel</structfield></entry>
4999+
<entry><type>char</type></entry>
5000+
<entry></entry>
5001+
<entry>
5002+
<structfield>proparallel</structfield> tells whether the function
5003+
can be safely run in parallel mode.
5004+
It is <literal>s</literal> for functions which are safe to run in
5005+
parallel mode without restriction.
5006+
It is <literal>r</literal> for functions which can be run in parallel
5007+
mode, but their execution is restricted to the parallel group leader;
5008+
parallel worker processes cannot invoke these functions.
5009+
It is <literal>u</literal> for functions which are unsafe in parallel
5010+
mode; the presence of such a function forces a serial execution plan.
5011+
</entry>
5012+
</row>
5013+
49975014
<row>
49985015
<entry><structfield>pronargs</structfield></entry>
49995016
<entry><type>int2</type></entry>

‎doc/src/sgml/ref/alter_function.sgml

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,7 @@ ALTER FUNCTION <replaceable>name</replaceable> ( [ [ <replaceable class="paramet
3535
CALLED ON NULL INPUT | RETURNS NULL ON NULL INPUT | STRICT
3636
IMMUTABLE | STABLE | VOLATILE | [ NOT ] LEAKPROOF
3737
[ EXTERNAL ] SECURITY INVOKER | [ EXTERNAL ] SECURITY DEFINER
38+
PARALLEL { UNSAFE | RESTRICTED | SAFE }
3839
COST <replaceable class="parameter">execution_cost</replaceable>
3940
ROWS <replaceable class="parameter">result_rows</replaceable>
4041
SET <replaceable class="parameter">configuration_parameter</replaceable> { TO | = } { <replaceable class="parameter">value</replaceable> | DEFAULT }
@@ -191,6 +192,17 @@ ALTER FUNCTION <replaceable>name</replaceable> ( [ [ <replaceable class="paramet
191192
</listitem>
192193
</varlistentry>
193194

195+
<varlistentry>
196+
<term><literal>PARALLEL</literal></term>
197+
198+
<listitem>
199+
<para>
200+
Change whether the function is deemed safe for parallelism.
201+
See <xref linkend="sql-createfunction"> for details.
202+
</para>
203+
</listitem>
204+
</varlistentry>
205+
194206
<varlistentry>
195207
<term><literal>LEAKPROOF</literal></term>
196208
<listitem>

‎doc/src/sgml/ref/create_function.sgml

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ CREATE [ OR REPLACE ] FUNCTION
3030
| IMMUTABLE | STABLE | VOLATILE | [ NOT ] LEAKPROOF
3131
| CALLED ON NULL INPUT | RETURNS NULL ON NULL INPUT | STRICT
3232
| [ EXTERNAL ] SECURITY INVOKER | [ EXTERNAL ] SECURITY DEFINER
33+
| PARALLEL { UNSAFE | RESTRICTED | SAFE }
3334
| COST <replaceable class="parameter">execution_cost</replaceable>
3435
| ROWS <replaceable class="parameter">result_rows</replaceable>
3536
| SET <replaceable class="parameter">configuration_parameter</replaceable> { TO <replaceable class="parameter">value</replaceable> | = <replaceable class="parameter">value</replaceable> | FROM CURRENT }
@@ -411,6 +412,43 @@ CREATE [ OR REPLACE ] FUNCTION
411412
</listitem>
412413
</varlistentry>
413414

415+
<varlistentry>
416+
<term><literal>PARALLEL</literal></term>
417+
418+
<listitem>
419+
<para><literal>PARALLEL UNSAFE</literal> indicates that the function
420+
can't be executed in parallel mode and the presence of such a
421+
function in an SQL statement forces a serial execution plan. This is
422+
the default. <literal>PARALLEL RESTRICTED</literal> indicates that
423+
the function can be executed in parallel mode, but the execution is
424+
restricted to parallel group leader. <literal>PARALLEL SAFE</literal>
425+
indicates that the function is safe to run in parallel mode without
426+
restriction.
427+
</para>
428+
429+
<para>
430+
Functions should be labeled parallel unsafe if they modify any database
431+
state, or if they make changes to the transaction such as using
432+
sub-transactions, or if they access sequences or attempt to make
433+
persistent changes to settings (e.g. <literal>setval</>). They should
434+
be labeled as parallel restricted if they access temporary tables,
435+
client connection state, cursors, prepared statements, or miscellaneous
436+
backend-local state which the system cannot synchronize in parallel mode
437+
(e.g. <literal>setseed</> cannot be executed other than by the group
438+
leader because a change made by another process would not be reflected
439+
in the leader). In general, if a function is labeled as being safe when
440+
it is restricted or unsafe, or if it is labeled as being restricted when
441+
it is in fact unsafe, it may throw errors or produce wrong answers
442+
when used in a parallel query. C-language functions could in theory
443+
exhibit totally undefined behavior if mislabeled, since there is no way
444+
for the system to protect itself against arbitrary C code, but in most
445+
likely cases the result will be no worse than for any other function.
446+
If in doubt, functions should be labeled as <literal>UNSAFE</>, which is
447+
the default.
448+
</para>
449+
</listitem>
450+
</varlistentry>
451+
414452
<varlistentry>
415453
<term><replaceable class="parameter">execution_cost</replaceable></term>
416454

‎src/backend/catalog/pg_aggregate.c

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -540,6 +540,7 @@ AggregateCreate(const char *aggName,
540540
false, /* isStrict (not needed for agg) */
541541
PROVOLATILE_IMMUTABLE, /* volatility (not
542542
* needed for agg) */
543+
PROPARALLEL_UNSAFE,
543544
parameterTypes, /* paramTypes */
544545
allParameterTypes, /* allParamTypes */
545546
parameterModes, /* parameterModes */

‎src/backend/catalog/pg_proc.c

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -83,6 +83,7 @@ ProcedureCreate(const char *procedureName,
8383
bool isLeakProof,
8484
bool isStrict,
8585
char volatility,
86+
char parallel,
8687
oidvector *parameterTypes,
8788
Datum allParameterTypes,
8889
Datum parameterModes,
@@ -344,6 +345,7 @@ ProcedureCreate(const char *procedureName,
344345
values[Anum_pg_proc_proisstrict - 1] = BoolGetDatum(isStrict);
345346
values[Anum_pg_proc_proretset - 1] = BoolGetDatum(returnsSet);
346347
values[Anum_pg_proc_provolatile - 1] = CharGetDatum(volatility);
348+
values[Anum_pg_proc_proparallel - 1] = CharGetDatum(parallel);
347349
values[Anum_pg_proc_pronargs - 1] = UInt16GetDatum(parameterCount);
348350
values[Anum_pg_proc_pronargdefaults - 1] = UInt16GetDatum(list_length(parameterDefaults));
349351
values[Anum_pg_proc_prorettype - 1] = ObjectIdGetDatum(returnType);

‎src/backend/commands/explain.c

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -348,7 +348,7 @@ ExplainOneQuery(Query *query, IntoClause *into, ExplainState *es,
348348
INSTR_TIME_SET_CURRENT(planstart);
349349

350350
/* plan the query */
351-
plan = pg_plan_query(query, 0, params);
351+
plan = pg_plan_query(query, CURSOR_OPT_PARALLEL_OK, params);
352352

353353
INSTR_TIME_SET_CURRENT(planduration);
354354
INSTR_TIME_SUBTRACT(planduration, planstart);

‎src/backend/commands/extension.c

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -707,7 +707,7 @@ execute_sql_string(const char *sql, const char *filename)
707707
sql,
708708
NULL,
709709
0);
710-
stmt_list = pg_plan_queries(stmt_list, 0, NULL);
710+
stmt_list = pg_plan_queries(stmt_list, CURSOR_OPT_PARALLEL_OK, NULL);
711711

712712
foreach(lc2, stmt_list)
713713
{

‎src/backend/commands/functioncmds.c

Lines changed: 46 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -465,7 +465,8 @@ compute_common_attribute(DefElem *defel,
465465
DefElem **leakproof_item,
466466
List **set_items,
467467
DefElem **cost_item,
468-
DefElem **rows_item)
468+
DefElem **rows_item,
469+
DefElem **parallel_item)
469470
{
470471
if (strcmp(defel->defname, "volatility") == 0)
471472
{
@@ -513,6 +514,13 @@ compute_common_attribute(DefElem *defel,
513514

514515
*rows_item = defel;
515516
}
517+
else if (strcmp(defel->defname, "parallel") == 0)
518+
{
519+
if (*parallel_item)
520+
goto duplicate_error;
521+
522+
*parallel_item = defel;
523+
}
516524
else
517525
return false;
518526

@@ -544,6 +552,27 @@ interpret_func_volatility(DefElem *defel)
544552
}
545553
}
546554

555+
static char
556+
interpret_func_parallel(DefElem *defel)
557+
{
558+
char *str = strVal(defel->arg);
559+
560+
if (strcmp(str, "safe") == 0)
561+
return PROPARALLEL_SAFE;
562+
else if (strcmp(str, "unsafe") == 0)
563+
return PROPARALLEL_UNSAFE;
564+
else if (strcmp(str, "restricted") == 0)
565+
return PROPARALLEL_RESTRICTED;
566+
else
567+
{
568+
ereport(ERROR,
569+
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
570+
errmsg("parallel option \"%s\" not recognized",
571+
str)));
572+
return PROPARALLEL_UNSAFE; /* keep compiler quiet */
573+
}
574+
}
575+
547576
/*
548577
* Update a proconfig value according to a list of VariableSetStmt items.
549578
*
@@ -592,7 +621,8 @@ compute_attributes_sql_style(List *options,
592621
bool *leakproof_p,
593622
ArrayType **proconfig,
594623
float4 *procost,
595-
float4 *prorows)
624+
float4 *prorows,
625+
char *parallel_p)
596626
{
597627
ListCell *option;
598628
DefElem *as_item = NULL;
@@ -606,6 +636,7 @@ compute_attributes_sql_style(List *options,
606636
List *set_items = NIL;
607637
DefElem *cost_item = NULL;
608638
DefElem *rows_item = NULL;
639+
DefElem *parallel_item = NULL;
609640

610641
foreach(option, options)
611642
{
@@ -650,7 +681,8 @@ compute_attributes_sql_style(List *options,
650681
&leakproof_item,
651682
&set_items,
652683
&cost_item,
653-
&rows_item))
684+
&rows_item,
685+
&parallel_item))
654686
{
655687
/* recognized common option */
656688
continue;
@@ -712,6 +744,8 @@ compute_attributes_sql_style(List *options,
712744
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
713745
errmsg("ROWS must be positive")));
714746
}
747+
if (parallel_item)
748+
*parallel_p = interpret_func_parallel(parallel_item);
715749
}
716750

717751

@@ -858,6 +892,7 @@ CreateFunction(CreateFunctionStmt *stmt, const char *queryString)
858892
HeapTuple languageTuple;
859893
Form_pg_language languageStruct;
860894
List *as_clause;
895+
char parallel;
861896

862897
/* Convert list of names to a name and namespace */
863898
namespaceId = QualifiedNameGetCreationNamespace(stmt->funcname,
@@ -878,13 +913,14 @@ CreateFunction(CreateFunctionStmt *stmt, const char *queryString)
878913
proconfig = NULL;
879914
procost = -1; /* indicates not set */
880915
prorows = -1; /* indicates not set */
916+
parallel = PROPARALLEL_UNSAFE;
881917

882918
/* override attributes from explicit list */
883919
compute_attributes_sql_style(stmt->options,
884920
&as_clause, &language, &transformDefElem,
885921
&isWindowFunc, &volatility,
886922
&isStrict, &security, &isLeakProof,
887-
&proconfig, &procost, &prorows);
923+
&proconfig, &procost, &prorows, &parallel);
888924

889925
/* Look up the language and validate permissions */
890926
languageTuple = SearchSysCache1(LANGNAME, PointerGetDatum(language));
@@ -1061,6 +1097,7 @@ CreateFunction(CreateFunctionStmt *stmt, const char *queryString)
10611097
isLeakProof,
10621098
isStrict,
10631099
volatility,
1100+
parallel,
10641101
parameterTypes,
10651102
PointerGetDatum(allParameterTypes),
10661103
PointerGetDatum(parameterModes),
@@ -1141,6 +1178,7 @@ AlterFunction(AlterFunctionStmt *stmt)
11411178
List *set_items = NIL;
11421179
DefElem *cost_item = NULL;
11431180
DefElem *rows_item = NULL;
1181+
DefElem *parallel_item = NULL;
11441182
ObjectAddress address;
11451183

11461184
rel = heap_open(ProcedureRelationId, RowExclusiveLock);
@@ -1178,7 +1216,8 @@ AlterFunction(AlterFunctionStmt *stmt)
11781216
&leakproof_item,
11791217
&set_items,
11801218
&cost_item,
1181-
&rows_item) == false)
1219+
&rows_item,
1220+
&parallel_item) == false)
11821221
elog(ERROR, "option \"%s\" not recognized", defel->defname);
11831222
}
11841223

@@ -1250,6 +1289,8 @@ AlterFunction(AlterFunctionStmt *stmt)
12501289
tup = heap_modify_tuple(tup, RelationGetDescr(rel),
12511290
repl_val, repl_null, repl_repl);
12521291
}
1292+
if (parallel_item)
1293+
procForm->proparallel = interpret_func_parallel(parallel_item);
12531294

12541295
/* Do the update */
12551296
simple_heap_update(rel, &tup->t_self, tup);

‎src/backend/commands/proclang.c

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -135,6 +135,7 @@ CreateProceduralLanguage(CreatePLangStmt *stmt)
135135
false, /* isLeakProof */
136136
false, /* isStrict */
137137
PROVOLATILE_VOLATILE,
138+
PROPARALLEL_UNSAFE,
138139
buildoidvector(funcargtypes, 0),
139140
PointerGetDatum(NULL),
140141
PointerGetDatum(NULL),
@@ -174,6 +175,7 @@ CreateProceduralLanguage(CreatePLangStmt *stmt)
174175
false, /* isLeakProof */
175176
true, /* isStrict */
176177
PROVOLATILE_VOLATILE,
178+
PROPARALLEL_UNSAFE,
177179
buildoidvector(funcargtypes, 1),
178180
PointerGetDatum(NULL),
179181
PointerGetDatum(NULL),
@@ -216,6 +218,7 @@ CreateProceduralLanguage(CreatePLangStmt *stmt)
216218
false, /* isLeakProof */
217219
true, /* isStrict */
218220
PROVOLATILE_VOLATILE,
221+
PROPARALLEL_UNSAFE,
219222
buildoidvector(funcargtypes, 1),
220223
PointerGetDatum(NULL),
221224
PointerGetDatum(NULL),

‎src/backend/commands/typecmds.c

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1611,6 +1611,7 @@ makeRangeConstructors(const char *name, Oid namespace,
16111611
false, /* leakproof */
16121612
false, /* isStrict */
16131613
PROVOLATILE_IMMUTABLE, /* volatility */
1614+
PROPARALLEL_SAFE, /* parallel safety */
16141615
constructorArgTypesVector, /* parameterTypes */
16151616
PointerGetDatum(NULL), /* allParameterTypes */
16161617
PointerGetDatum(NULL), /* parameterModes */

‎src/backend/executor/execMain.c

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -243,6 +243,11 @@ standard_ExecutorStart(QueryDesc *queryDesc, int eflags)
243243
if (!(eflags & (EXEC_FLAG_SKIP_TRIGGERS | EXEC_FLAG_EXPLAIN_ONLY)))
244244
AfterTriggerBeginQuery();
245245

246+
/* Enter parallel mode, if required by the query. */
247+
if (queryDesc->plannedstmt->parallelModeNeeded &&
248+
!(eflags & EXEC_FLAG_EXPLAIN_ONLY))
249+
EnterParallelMode();
250+
246251
MemoryContextSwitchTo(oldcontext);
247252
}
248253

@@ -474,6 +479,11 @@ standard_ExecutorEnd(QueryDesc *queryDesc)
474479
*/
475480
MemoryContextSwitchTo(oldcontext);
476481

482+
/* Exit parallel mode, if it was required by the query. */
483+
if (queryDesc->plannedstmt->parallelModeNeeded &&
484+
!(estate->es_top_eflags & EXEC_FLAG_EXPLAIN_ONLY))
485+
ExitParallelMode();
486+
477487
/*
478488
* Release EState and per-query memory context. This should release
479489
* everything the executor has allocated.

‎src/backend/executor/functions.c

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -496,7 +496,9 @@ init_execution_state(List *queryTree_list,
496496
if (queryTree->commandType == CMD_UTILITY)
497497
stmt = queryTree->utilityStmt;
498498
else
499-
stmt = (Node *) pg_plan_query(queryTree, 0, NULL);
499+
stmt = (Node *) pg_plan_query(queryTree,
500+
fcache->readonly_func ? CURSOR_OPT_PARALLEL_OK : 0,
501+
NULL);
500502

501503
/* Precheck all commands for validity in a function */
502504
if (IsA(stmt, TransactionStmt))

‎src/backend/nodes/copyfuncs.c

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -94,6 +94,7 @@ _copyPlannedStmt(const PlannedStmt *from)
9494
COPY_NODE_FIELD(invalItems);
9595
COPY_SCALAR_FIELD(nParamExec);
9696
COPY_SCALAR_FIELD(hasRowSecurity);
97+
COPY_SCALAR_FIELD(parallelModeNeeded);
9798

9899
return newnode;
99100
}

‎src/backend/nodes/outfuncs.c

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -256,6 +256,7 @@ _outPlannedStmt(StringInfo str, const PlannedStmt *node)
256256
WRITE_NODE_FIELD(invalItems);
257257
WRITE_INT_FIELD(nParamExec);
258258
WRITE_BOOL_FIELD(hasRowSecurity);
259+
WRITE_BOOL_FIELD(parallelModeNeeded);
259260
}
260261

261262
/*
@@ -1787,6 +1788,8 @@ _outPlannerGlobal(StringInfo str, const PlannerGlobal *node)
17871788
WRITE_UINT_FIELD(lastRowMarkId);
17881789
WRITE_BOOL_FIELD(transientPlan);
17891790
WRITE_BOOL_FIELD(hasRowSecurity);
1791+
WRITE_BOOL_FIELD(parallelModeOK);
1792+
WRITE_BOOL_FIELD(parallelModeNeeded);
17901793
}
17911794

17921795
static void

0 commit comments

Comments
 (0)