ikiwiki/doc/plugins/write/external.mdwn

External plugins are standalone, executable programs, that can be written
in any language. When ikiwiki starts up, it runs the program, and
communicates with it using XML RPC. If you want to [[write]] an external
plugin, read on..

ikiwiki contains one sample external plugin, named `externaldemo`. This is
written in perl, but is intended to be an example of how to write an
external plugin in your favorite programming language. Wow us at how much
easier you can do the same thing in your favorite language. ;-)

There's now a second external plugin, the [[rst]] plugin, written in
python. It uses a `proxy.py`, a helper library for ikiwiki python plugins.

[[toc ]]

## How external plugins use XML RPC

While XML RPC is typically used over http, ikiwiki doesn't do that.
Instead, the external plugin reads XML RPC data from stdin, and writes it
to stdout. To ease parsing, each separate XML RPC request or response must
start at the beginning of a line, and end with a newline. When outputting
XML RPC to stdout, be _sure_ to flush stdout. Failure to do so will result
in deadlock!

An external plugin should operate in a loop. First, read a command from
stdin, using XML RPC. Dispatch the command, and return its result to
stdout, also using XML RPC. After reading a command, and before returning
the result, the plugin can output XML RPC requests of its own, calling
functions in ikiwiki. Note: *Never* make an XML RPC request at any other
time. Ikiwiki won't be listening for it, and you will deadlock.

When ikiwiki starts up an external plugin, the first RPC it will make
is to call the plugin's `import()` function. That function typically makes
an RPC to ikiwiki's `hook()` function, registering a callback.

An external plugin can use XML RPC to call any of the exported functions
documented in the [[plugin_interface_documentation|write]]. It can also
actually call any non-exported IkiWiki function, but doing so is a good way
to break your plugin when ikiwiki changes. There is currently no versioned
interface like there is for perl plugins, but external plugins were first
supported in ikiwiki version 2.6.

## Accessing data structures

Ikiwiki has a few global data structures such as `%config`, which holds
its configuration. External plugins can use the `getvar` and `setvar` RPCs
to access any such global hash. To get the "url" configuration value,
call `getvar("config", "url")`. To set it, call 
`setvar("config", "url", "http://example.com/)`.

The `%pagestate` is a special hash with a more complex format. To access
it, external plugins can use the `getstate` and `setstate` RPCs. To access
stored state, call `getstate("page", "id", "key")`, and to store state,
call `setstate("page", "id", "key", "value")`.

To access ikiwiki's ARGV array, call `getargv()`. To change its ARGV, call
`setargv(array)`.

## Notes on function parameters

The [[plugin_interface_documentation|write]] talks about functions that take
"named parameters". When such a function is called over XML RPC, such named
parameters look like a list of keys and values:

	page, foo, destpage, bar, magnify, 1

If a name is repeated in the list, the later value overrides the earlier
one:

	name, Bob, age, 20, name, Sally, gender, female

In perl, boiling this down to an associative array of named parameters is
very easy:

	sub foo {
		my %params=@list;

Other languages might not find it so easy. If not, it might be a good idea
to convert these named parameters into something more natural for the
language as part of their XML RPC interface.

## undef

XML RPC has a limitation that it does not have a way to pass
undef/NULL/None. There is an extension to the protocol that supports this,
but it is not yet available in the [[cpan XML::RPC]] library used by
ikiwiki.

Until the extension is available, ikiwiki allows undef to be communicated
over XML RPC by passing a sentinal value, a hash with a single key "null"
with a value of an empty string. External plugins that need to communicate
null values to or from ikiwiki will have to translate between undef and
the sentinal.

## Function injection

Some parts of ikiwiki are extensible by adding functions. For example, the
RCS interface relies on plugins providing several IkiWiki::rcs_* functions.
It's actually possible to do this from an external plugin too. 

To make your external plugin provide an `IkiWiki::rcs_update` function, for
example, make an RPC call to `inject`. Pass it named parameters "name" and
"call", where "name" is the name of the function to inject into perl (here
"Ikiwiki::rcs_update" and "call" is the RPC call ikiwiki will make whenever
that function is run.

If the RPC call is memoizable, you can also pass a "memoize" parameter, set
to 1.

## Limitations of XML RPC

Since XML RPC can't pass around references to objects, it can't be used
with functions that take or return such references. That means you can't
100% use XML RPC for `cgi` or `formbuilder` hooks (which are passed CGI and
FormBuilder perl objects), or use it to call `template()` (which returns a
perl HTML::Template object).

## Performance issues

Since each external plugin is a separate process, when ikiwiki is
configured to use lots of external plugins, it will start up slower, and
use more resources. One or two should not be a problem though.

There is some overhead in using XML RPC for function calls. Most plugins
should find it to be pretty minimal though. In one benchmark, ikiwiki was
able to perform 10000 simple XML RPC calls in 11 seconds -- 900 per second.

Using external plugins for hooks such as `sanitize` and `format`, which
pass around entire pages, and are run for each page rendered, will cause
more XML RPC overhead than usual, due to the larger number of calls, and the
large quantity of data conversion going on. In contrast, `preprocess` hooks
are called generally rarely, and pass around minimal data.

External plugins should avoid making RPC calls unnecessarily (ie, in a loop).
Memoizing the results of appropriate RPC calls is one good way to minimise the
number of calls.

Injecting a replacement for a commonly called ikiwiki function
could result in a lot more RPC calls than expected and slow
eveything down. `pagetitle`, for instance, is called about 100 times
per page build. Whenever possible, you should tell ikiwiki to memoize
injected functions.

In general, use common sense, and your external plugin will probably
perform ok.