Writing a Custom Scan Provider {#custom-scan}
==============================

custom scan provider

handler for

PostgreSQL supports a set of experimental facilities which are intended
to allow extension modules to add new scan types to the system. Unlike a
[foreign data wrapper](#fdwhandler), which is only responsible for
knowing how to scan its own foreign tables, a custom scan provider can
provide an alternative method of scanning any relation in the system.
Typically, the motivation for writing a custom scan provider will be to
allow the use of some optimization not supported by the core system,
such as caching or some form of hardware acceleration. This chapter
outlines how to write a new custom scan provider.

Implementing a new type of custom scan is a three-step process. First,
during planning, it is necessary to generate access paths representing a
scan using the proposed strategy. Second, if one of those access paths
is selected by the planner as the optimal strategy for scanning a
particular relation, the access path must be converted to a plan.
Finally, it must be possible to execute the plan and generate the same
results that would have been generated for any other access path
targeting the same relation.

Creating Custom Scan Paths {#custom-scan-path}
==========================

A custom scan provider will typically add paths for a base relation by
setting the following hook, which is called after the core code has
generated all the access paths it can for the relation (except for
Gather paths, which are made after this call so that they can use
partial paths added by the hook):

    typedef void (*set_rel_pathlist_hook_type) (PlannerInfo *root,
                                                RelOptInfo *rel,
                                                Index rti,
                                                RangeTblEntry *rte);
    extern PGDLLIMPORT set_rel_pathlist_hook_type set_rel_pathlist_hook;

Although this hook function can be used to examine, modify, or remove
paths generated by the core system, a custom scan provider will
typically confine itself to generating CustomPath objects and adding
them to `rel` using `add_path`. The custom scan provider is responsible
for initializing the CustomPath object, which is declared like this:

    typedef struct CustomPath
    {
        Path      path;
        uint32    flags;
        List     *custom_paths;
        List     *custom_private;
        const CustomPathMethods *methods;
    } CustomPath;

path must be initialized as for any other path, including the row-count
estimate, start and total cost, and sort ordering provided by this path.
flags is a bit mask, which specifies whether the scan provider can
support certain optional capabilities. flags should include
`CUSTOMPATH_SUPPORT_BACKWARD_SCAN` if the custom path can support a
backward scan, `CUSTOMPATH_SUPPORT_MARK_RESTORE` if it can support mark
and restore, and `CUSTOMPATH_SUPPORT_PROJECTION` if it can perform
projections. (If `CUSTOMPATH_SUPPORT_PROJECTION` is not set, the scan
node will only be asked to produce Vars of the scanned relation; while
if that flag is set, the scan node must be able to evaluate scalar
expressions over these Vars.) An optional custom\_paths is a list of
Path nodes used by this custom-path node; these will be transformed into
Plan nodes by planner. custom\_private can be used to store the custom
path\'s private data. Private data should be stored in a form that can
be handled by `nodeToString`, so that debugging routines that attempt to
print the custom path will work as designed. methods must point to a
(usually statically allocated) object implementing the required custom
path methods, of which there is currently only one.

A custom scan provider can also provide join paths. Just as for base
relations, such a path must produce the same output as would normally be
produced by the join it replaces. To do this, the join provider should
set the following hook, and then within the hook function, create
CustomPath path(s) for the join relation.

    typedef void (*set_join_pathlist_hook_type) (PlannerInfo *root,
                                                 RelOptInfo *joinrel,
                                                 RelOptInfo *outerrel,
                                                 RelOptInfo *innerrel,
                                                 JoinType jointype,
                                                 JoinPathExtraData *extra);
    extern PGDLLIMPORT set_join_pathlist_hook_type set_join_pathlist_hook;

This hook will be invoked repeatedly for the same join relation, with
different combinations of inner and outer relations; it is the
responsibility of the hook to minimize duplicated work.

Custom Scan Path Callbacks
--------------------------

    Plan *(*PlanCustomPath) (PlannerInfo *root,
                             RelOptInfo *rel,
                             CustomPath *best_path,
                             List *tlist,
                             List *clauses,
                             List *custom_plans);

Convert a custom path to a finished plan. The return value will
generally be a `CustomScan` object, which the callback must allocate and
initialize. See [Creating Custom Scan Plans](#custom-scan-plan) for more
details.

Creating Custom Scan Plans {#custom-scan-plan}
==========================

A custom scan is represented in a finished plan tree using the following
structure:

    typedef struct CustomScan
    {
        Scan      scan;
        uint32    flags;
        List     *custom_plans;
        List     *custom_exprs;
        List     *custom_private;
        List     *custom_scan_tlist;
        Bitmapset *custom_relids;
        const CustomScanMethods *methods;
    } CustomScan;

scan must be initialized as for any other scan, including estimated
costs, target lists, qualifications, and so on. flags is a bit mask with
the same meaning as in CustomPath. custom\_plans can be used to store
child Plan nodes. custom\_exprs should be used to store expression trees
that will need to be fixed up by `setrefs.c` and `subselect.c`, while
custom\_private should be used to store other private data that is only
used by the custom scan provider itself. custom\_scan\_tlist can be NIL
when scanning a base relation, indicating that the custom scan returns
scan tuples that match the base relation\'s row type. Otherwise it is a
target list describing the actual scan tuples. custom\_scan\_tlist must
be provided for joins, and could be provided for scans if the custom
scan provider can compute some non-Var expressions. custom\_relids is
set by the core code to the set of relations (range table indexes) that
this scan node handles; except when this scan is replacing a join, it
will have only one member. methods must point to a (usually statically
allocated) object implementing the required custom scan methods, which
are further detailed below.

When a CustomScan scans a single relation, scan.scanrelid must be the
range table index of the table to be scanned. When it replaces a join,
scan.scanrelid should be zero.

Plan trees must be able to be duplicated using `copyObject`, so all the
data stored within the "custom" fields must consist of nodes that that
function can handle. Furthermore, custom scan providers cannot
substitute a larger structure that embeds a CustomScan for the structure
itself, as would be possible for a CustomPath or CustomScanState.

Custom Scan Plan Callbacks
--------------------------

    Node *(*CreateCustomScanState) (CustomScan *cscan);

Allocate a CustomScanState for this CustomScan. The actual allocation
will often be larger than required for an ordinary CustomScanState,
because many providers will wish to embed that as the first field of a
larger structure. The value returned must have the node tag and methods
set appropriately, but other fields should be left as zeroes at this
stage; after `ExecInitCustomScan` performs basic initialization, the
`BeginCustomScan` callback will be invoked to give the custom scan
provider a chance to do whatever else is needed.

Executing Custom Scans {#custom-scan-execution}
======================

When a CustomScan is executed, its execution state is represented by a
CustomScanState, which is declared as follows:

    typedef struct CustomScanState
    {
        ScanState ss;
        uint32    flags;
        const CustomExecMethods *methods;
    } CustomScanState;

ss is initialized as for any other scan state, except that if the scan
is for a join rather than a base relation, `ss.ss_currentRelation` is
left NULL. flags is a bit mask with the same meaning as in CustomPath
and CustomScan. methods must point to a (usually statically allocated)
object implementing the required custom scan state methods, which are
further detailed below. Typically, a CustomScanState, which need not
support `copyObject`, will actually be a larger structure embedding the
above as its first member.

Custom Scan Execution Callbacks
-------------------------------

    void (*BeginCustomScan) (CustomScanState *node,
                             EState *estate,
                             int eflags);

Complete initialization of the supplied CustomScanState. Standard fields
have been initialized by `ExecInitCustomScan`, but any private fields
should be initialized here.

    TupleTableSlot *(*ExecCustomScan) (CustomScanState *node);

Fetch the next scan tuple. If any tuples remain, it should fill
`ps_ResultTupleSlot` with the next tuple in the current scan direction,
and then return the tuple slot. If not, `NULL` or an empty slot should
be returned.

    void (*EndCustomScan) (CustomScanState *node);

Clean up any private data associated with the `CustomScanState`. This
method is required, but it does not need to do anything if there is no
associated data or it will be cleaned up automatically.

    void (*ReScanCustomScan) (CustomScanState *node);

Rewind the current scan to the beginning and prepare to rescan the
relation.

    void (*MarkPosCustomScan) (CustomScanState *node);

Save the current scan position so that it can subsequently be restored
by the `RestrPosCustomScan` callback. This callback is optional, and
need only be supplied if the `CUSTOMPATH_SUPPORT_MARK_RESTORE` flag is
set.

    void (*RestrPosCustomScan) (CustomScanState *node);

Restore the previous scan position as saved by the `MarkPosCustomScan`
callback. This callback is optional, and need only be supplied if the
`CUSTOMPATH_SUPPORT_MARK_RESTORE` flag is set.

    Size (*EstimateDSMCustomScan) (CustomScanState *node,
                                   ParallelContext *pcxt);

Estimate the amount of dynamic shared memory that will be required for
parallel operation. This may be higher than the amount that will
actually be used, but it must not be lower. The return value is in
bytes. This callback is optional, and need only be supplied if this
custom scan provider supports parallel execution.

    void (*InitializeDSMCustomScan) (CustomScanState *node,
                                     ParallelContext *pcxt,
                                     void *coordinate);

Initialize the dynamic shared memory that will be required for parallel
operation. `coordinate` points to a shared memory area of size equal to
the return value of `EstimateDSMCustomScan`. This callback is optional,
and need only be supplied if this custom scan provider supports parallel
execution.

    void (*ReInitializeDSMCustomScan) (CustomScanState *node,
                                       ParallelContext *pcxt,
                                       void *coordinate);

Re-initialize the dynamic shared memory required for parallel operation
when the custom-scan plan node is about to be re-scanned. This callback
is optional, and need only be supplied if this custom scan provider
supports parallel execution. Recommended practice is that this callback
reset only shared state, while the `ReScanCustomScan` callback resets
only local state. Currently, this callback will be called before
`ReScanCustomScan`, but it\'s best not to rely on that ordering.

    void (*InitializeWorkerCustomScan) (CustomScanState *node,
                                        shm_toc *toc,
                                        void *coordinate);

Initialize a parallel worker\'s local state based on the shared state
set up by the leader during `InitializeDSMCustomScan`. This callback is
optional, and need only be supplied if this custom scan provider
supports parallel execution.

    void (*ShutdownCustomScan) (CustomScanState *node);

Release resources when it is anticipated the node will not be executed
to completion. This is not called in all cases; sometimes,
`EndCustomScan` may be called without this function having been called
first. Since the DSM segment used by parallel query is destroyed just
after this callback is invoked, custom scan providers that wish to take
some action before the DSM segment goes away should implement this
method.

    void (*ExplainCustomScan) (CustomScanState *node,
                               List *ancestors,
                               ExplainState *es);

Output additional information for `EXPLAIN` of a custom-scan plan node.
This callback is optional. Common data stored in the ScanState, such as
the target list and scan relation, will be shown even without this
callback, but the callback allows the display of additional, private
state.
