inline has a format hook that is an optimisation hack. Until this hook
runs, the inlined content is not present on the page. This can prevent
other format hooks, that process that content, from acting on inlined
content. In bug ##509710, we discovered this happened commonly for the
embed plugin, but it could in theory happen for many other plugins (color,
cutpaste, etc) that use format to fill in special html after sanitization.
The ordering was essentially random (hash key order). That's kinda a good
thing, because hooks should be independent of other hooks and able to run
in any order. But for things like inline, that just doesn't work.
To fix the immediate problem, let's make hooks able to be registered as
running "first". There was already the ability to make them run "last".
Now, this simple first/middle/last ordering is obviously not going to work
if a lot of things need to run first, or last, since then we'll be back to
being unable to specify ordering inside those sets. But before worrying about
that too much, and considering dependency ordering, etc, observe how few
plugins use last ordering: Exactly one needs it. And, so far, exactly one
needs first ordering. So for now, KISS.
Another implementation note: I could have sorted the plugins with
first/last/middle as the primary key, and plugin name secondary, to get a
guaranteed stable order. Instead, I chose to preserve hash order. Two
opposing things pulled me toward that decision:
1. Since has order is randomish, it will ensure that no accidental
ordering assumptions are made.
2. Assume for a minute that ordering matters a lot more than expected.
Drastically changing the order a particular configuration uses could
result in a lot of subtle bugs cropping up. (I hope this assumption is
false, partly due to #1, but can't rule it out.)
A malformed pagespec will cause $@ to be set when translated, but if
it is used a second time, the memoization will defeat that check. Better to
check for the result not being defined.
This avoids constructing urls like "./../foo/".
The leading "../" avoids any colon confusion already.
I noticed in my logs that certain badly written web spiders (hello again,
Yahoo!) fail to follow urls like ikiwiki was constructing to the right
place (instead ending up at "./foo/")
Since ikiwiki uses open :utf8, perl assumes that files contain valid utf-8.
If it turns out to be malformed it may later crash while processing strings
read from them, with 'Malformed UTF-8 character (fatal)'.
As at least a quick fix, use utf8::valid as soon as data is read, and if
it's not valid, call encode_utf8 on the string, thus clearing the utf-8
flag. This may cause follow-on encoding problems, but will avoid this
crash, and the input file was broken anyway, so GIGO is a reasonable
response. (I looked at calling decode_utf8 after, but it seemed to cause
more trouble than it was worth. BTW, use open ':encoding(utf8)' avaoids
this problem, but the corrupted data later causes Storable to crash when
writing the index.)
This is a quick fix, clearly imperfect:
- It might be better to explicitly call decode_utf8 when reading files,
rather than using the IO layer.
- Data read other than by readfile() can still sneak in bad utf-8. While
ikiwiki does very little file input not using it, stdin for the CGI
would be one way.
This is necessary so that things that fork to the background,
like pinger, and inline ping, don't block other cgis from running.
Note that websetup also calls unlockwiki, before refreshing / rebuilding
the wiki. It makes perfect sense for that not to block other cgis.
* Stop busy-waiting in lockwiki, as this could delay ikiwiki from waking up
for up to one second. The bailout code is no longer needed.
* Remove support for unused optional wait parameter from lockwiki.
This fixes a problem exposed by the recent change to tags
(a2839de936). That recorded tag links as
absolute by including a leading slash in the link. The same could also be
done with an absolute wikilink.
In either case, link() would not match such links, unless the leading slash
was included in the link to match. But that's not right, because pagespecs
match absolute by default. So strip the leading slash.
Note that to keep any existing `link(/foo)` pagespecs working after this
change, the leading slash is removed from there, too.
Add an inject function, that can be used by plugins that want to replace
one of ikiwiki's functions with their own version. (This is a scary thing
that grubs through the symbol table, and replaces all exported occurances
of a function with the injected version.)
external: RPC functions can be injected to replace exported functions.
Removed the stupid displaytime hook, and use injection instead.
Need to use a hook because an exported function cannot be reliably
overridden. The replacement verstion was actually only affecting plugins
loaded after it.
formattime doesn't need a hook, since there's no reason to export it.
This is the easy part of supporting foo/index.mdwn sources for page foo.
Note that if foo.mdwn exists too, there will be a warning about multiple
sources for the same page, and which is used is indeterminate.
indexpages should also cause web based editing to create index source pages
by default; this and other fallout of the option not yet implemented.
Upgrades to the new index format should be transparent.
The version field is 3, because 1 was the old textual index, 2 was the
pre-versioned format.
This also includes some efficiency improvements to index loading, by
not copying a hash and using a reference.
Setting the values in the setup file by default makes it annoyingly hard to
override them at the command line (since setup files override the command
line).
Both default to off if no value is set.
Move rcs plugin load to loadplugins; move duplicate rcs detection logic out
of individual plugins and into loadplugins. Avoids checkconfig failing when
run twice.
The fix for colons involved adding "./" to some urls. Due to the weird way
inline called urlto, these snuck into feed urls and permalinks. Fix it by
adding an optional third parameter to urlto.
* Renamed to parentlinks every single variable or function called
pedigree
* Removed the parentlinks function from Render.pm
* Enabled the new parentlinks plugin by default
* Adapted testsuite and documentation to reflate the above facts
Signed-off-by: intrigeri <intrigeri@boum.org>
This reverts commit e25c3a0a7c.
I didn't think that through far enough; beautify_url may be called on full
urls, which don't start with "/"!
Conflicts:
debian/changelog
So the problem is that ikiwiki would generate a relative link like
href="colon:problem", which web browsers treat as being in the "colon:"
uri scheme.
The best fix seems to be to make url beautification fix this, by slapping
a "./" in front.
* The editpage form now uses the raw page name, not the page title, in its
'page' cgi parameter. Using the title was ambiguous and made it
impossible to tell between some pages, like "foo/bar" and "foo__47__bar",
sometimes causing the wrong page to be edited.
* This change means that some edit links need to be updated.
Force a rebuild on upgrade to this version.
* Above change also allowed really fixing escaped slashes from the blogpost
form.
This special case crops up when generating the parentlink to the toplevel
index page. urlto("") had been generating a link to "./" (or "../" etc)
for that, which is fine, if the web server redirects that to the toplevel
index.html. It's less fine if there is no web server.
I actually ran into the problem first when using gopher. (Yes, yes, don't
laugh.. see upcoming tip.) But it also crops up when browsing local wiki
files.
Of course, the index.html is stripped back off if usedirs is enabled.
Because the search plugin needed it, also because it's one of the few
plugins that didn't already have it.
I also considered adding it to htmlize, but I really cannot imagine caring
what the destpage is when htmlizing. (I'll probably be poven wrong later.)
<mxey> I have a weird problem with Ikiwiki on FreeBSD: When rendering this page <http://rafb.net/p/zmfROX86.html>, I get "ikiwiki.setup: Modification of a read-only value attempted at /usr/local/lib/perl5/site_perl/5.8.8/IkiWiki.pm line 1300.".
And converting the constructors to not operate on the parameters directly
but use lexical variables fixed it. Not sure exactly when this occurs.
This manifested as wikis with no locked pages treating them all as locked.
The bug was introduced in version 2.41.
Medium urgency upload due to above fix.
During refresh of a wiki with 800 files, loadindex was using more total
time than any other function, and saveindex was also in the top ten.
Rewriting them to use Storable makes them three times as fast.
0.7 seconds is saved on my laptop in profiling mode.
About 12% of ikiwiki runtime was spent in pagespec_match. It was evaling
the same pagespec code over and over again. This changes pagespec_translate
to return memoized, precompiled functions that can be called to match against
a given pagespec.
This also allows getting rid of the weird variable scoping trick that had
to be in effect for pagespec_translate to be called -- the variables are
now just fed into the function it returns.
On my laptop, this drops build time for the docwiki from about 60 to 50
seconds.
license, and copyright. This can be used to create custom RecentChanges.
* meta: To support the pagespec functions, metadata about pages has to be
retained as pagestate.
* Fix encoding bug when pagestate values contained spaces.
This makes it a lot quicker to deal with lots of recentchanges pages
appearing and disappearing. It avoids needing to clutter up pagespecs with
exclusions for those pages, by making normal pagespecs not match them.
Can't use htmllink in userlink due to it being called from recentchanges
in cases where there is no configured url. It seems easist, at least for now,
to not linkify user names in this case. I don't think I've ever clicked on such
a link anyway.
Might revisit this later.
Add a prefix_directives option to the setup file to turn this syntax
on; currently defaults to false, for backward compatibility. Support
optional '!' prefix even with prefix_directives off, and use that in
the underlay to support either setting of prefix_directives. Add NEWS
entry with migration information.
Patch by Daniel Burrows. Closes: #451729
* When usedirs is disabled, link direct to index.html files, not to
directories, to improve browsing of file:// urls.
Patch by Daniel Burrows. Closes: #451728
htmllink needs to do it because it uses the basename of the link as the link
text
bestlink needs to do it because it is not always called from htmllink
are not included in the map. Include special styling for such pages.
* map: Remove common prefixes and don't over-indent.
* Add class option to htmllink().
* Plugins can add new directories to the search path with the add_underlay
function.
* Split out smiley underlay files into a separate underlay, so if the plugin
isn't used, the wiki isn't bloated with all those files.
old files.
* Change where the img plugin puts scaled images. It's better to make the
scaled images subpages of the page that embeds them, rather than putting
them alongside the original image, since if two pages scale the same image
the same way, this prevents complications in dealing with two pages
creating the same file. The move will be handled transparently, though you
might want to rebuild your wiki to make it occur in one step.
until the wiki is building and already locked, unless it's aggregating.
When aggregating, it does not wait for the lock if it cannot get it, and
instead exits, to prevent aggregating processes from piling up.
(Get a good message when a PageSpec fails due to a negated success by
creating success objects with a reason string, which morph into failure
objects when negated.)
scalar context, evaluates to a reason why the match failed.
* Add testpagespec plugin, which might be useful to see why a pagespec isn't
matching something.
for extended pagespecs. The old calling convention will still work for
back-compat for now.
* The calling convention for functions in the IkiWiki::PageSpec namespace
has changed so they are passed named parameters.
* Plugin interface version increased to 2.00 since I don't anticipate any
more interface changes before 2.0.
on and supported creating it (especially Tumov). This adds a "usedirs"
option that makes ikiwiki use foo/index.html instead of foo.html as
output page names. It is not yet enabled by default.
* Renamed %oldpagemtime to a more accurately named %pagemtime and fix it to
actually store pages' mtimes.
* Add "mtime" sort parameter to inline plugin.
that given link points based on the page doing the linking. Note that this
could make such PageSpecs match different things than before, if you
relied on the old behavior of them only matching the raw link text.
* This required changing the match_* interface, adding a third parameter.
* Allow link() PageSpecs to match relative, as is allowed with globs.a
* Add postform option to inline plugin.
* Add an bug tracker to the softwaresite example.
were titlepage escaped in the urls, and then doubly escaped by the CGI
when editing. To fix this, I removed the titlepage escaping in the edit
urls.
* That means that *every edit link* on the wiki is potentially changed.
Rebuilding wikis on upgrade to this version therefore necessary; enabled
that in postinst.
previous ugly hack used to avoid writing rss feeds in previews.
* Fix the img plugin to avoid overwriting images in previews. Instead it
does all the work to make sure the resizing works, and dummys up a resized
image using width and height attributes.
* Also fixes img preview display, the links were wrong in preview before.
commit hook, it was possible for one CGI to race another one and "win"
the commit of both their files. This race has been fixed by adding a new
commitlock, which when locked by the CGI, disables the commit hook
(except for commit mails). The CGI then takes care of the updates the
commit hook would have done.
parameters remain the same, but additional options are now passed in using
named parameters.
* Change plugin interface version to 1.02 to reflect this change.
* Add a new anchor option to htmllink. Thanks Ben for the idea.
* Support anchors in wikilinks.
* Add a "more" plugin based on one contributed by Ben to allow implementing
those dreaded "Read more" links in blogs.
including out of disk space situations. ikiwiki should never leave
truncated files, and if the error occurs during a web-based file edit,
the user will be given an opportunity to retry.
Inspired by the many ways Moin Moin destroys itself when out of disk. :-)
* Fix syslogging of errors.
* Add a "conditional" plugin, which allows displaying text if a condition
is true. It is enabled by default so conditional can be used in the
basewiki.
* Use conditionals in the template for plugins, so that plugin pages
say if they're currently enabled or not, and in various other places
in the wiki.
edited.
* Move code forcing signing before edit to a new "signinedit" plugin, and
code checking for locked pages into a new "lockedit" plugin. Both are
enabled by default.
* Remove the anonok config setting. This is now implemented by a new
"anonok" plugin. Anyone with a wiki allowing anonymous edits should
change their configs to enable this new plugin.
* Add an opendiscussion plugin that allows anonymous users to edit
discussion pages, on a wiki that is otherwise wouldn't allow it.
* Lots of CGI code reorg and cleanup.
* Fix code to make absolute urls for rss feeds, was missing some urls.
* Fix double-escaping of html entities in titles etc in rss feeds
that occured if escaped characters were present in the page filename.
manipulate.
* Only exclude rss and atom files from processing if the inline plugin
is enabled and that feed type is enabled. Else it's just a copyable file
type.
* Move rss and atom option handling code into the inline plugin.
* Applied a rather old patch from Recai to fix the "pruning is too strict"
issue. Now you can have wiki source directories inside dotdirs and the
like, if you want.
* Introduce the nicebundle. This is a kind of plugin, that just enables
many other plugins. It's an easy way to boost ikiwiki from its default,
basic wiki, to a full-featured wiki, without manually picking the right
set of plugins. New plugins will be added to the nicebundle from time to
time.
nothing more spohisticated will be needed.
* Add formbuilder_setup and formbuilder hooks.
* Split out a passwordauth module, that holds all the traditional password
based authentication etc code. It's enabled by default, but can be disabled
if you want only openid or some other auth method.
* Web commits by OpenID users will record the full OpenID url for the user,
but in recentchanges, these urls will be converted to a simplified display
form+link.
* Modified svn, git, tla backends to recognise such web commits.
* Use precalculated backlinks info when determining if files need an update
due to a page they link to being added/removed. Mostly significant if
there are lots of pages.
* Remove duplicate link info when saving index. In some cases it could
pile up rather badly. (Probably not the best way to deal with this
problem.)
8x.
* Add "scan" parameter to hook(), which is used to make the hook be called
during the scanning pass, as well as the render pass. The meta and tag
plugins need to use the new scan parameter, so will any others that modify
%links.
* Now that links are calculated in a separate pass, it can also
precalculate backlinks in one pass, which is O(N^2) instead of the
previous code that was O(N^3). A very nice speedup for wikis with lots
(thousands) of pages.
* Add a shortcut plugin, inspired by Victor Moral's contributed shortcuts
plugin, but featuring a more ikiwiki-ish syntax and with shortcuts that
can be configured using a page in wiki.
* Add feeds=no option to inline preprocessor directive to turn off all types
of feeds. feeds=rss will still work, and feeds=atom was also added, for
fine control.
* $IkiWiki::version now holds the program version, and is accessible to
plugins.
source file, to allow tracking of extra rendered files like rss feeds.
* Note that plugins that accessed this variable will need to be updated!
The plugin interface has been increased to version 1.01 for this change.
* Add will_render function to the plugin interface, used to register that a
page renders a destination file, and do some security checks.
* Use will_render in the inline and linkmap plugins.
* Previously but no longer rendered files will be cleaned up.
* You will need to rebuild your wiki on upgrade to this version.
using HTTP Authentication instead of ikiwiki's built in authentication.
Useful for eg, large sites with their own previously existing user auth
setup. Closes: #384534
- Plugins should not need to load IkiWiki::Render to get commonly
used functions, so moved some functions from there to IkiWiki.
- Picked out the set of functions and variables that most plugins
use, documented them, and made IkiWiki export them by default,
like a proper perl module should.
- Use the other functions at your own risk.
- This is not quite complete, I still have to decide whether to
export some other things.
* Changed all plugins included in ikiwiki to not use "IkiWiki::" when
referring to stuff now exported by the IkiWiki module.
* Anyone with a third-party ikiwiki plugin is strongly enrouraged
to make like changes to it and avoid use of non-exported symboles from
"IkiWiki::".
* Link debian/changelog and debian/news to NEWS and CHANGELOG.
* Support hyperestradier version 1.4.2, which adds a new required phraseform
setting.
cookies to only be sent over ssl connections to avoid interception.
* Factor out the cgi header printing code into a new function.
* Fix preferences page on anonok wikis; still need to sign in to get
to the preferences page.
pages.
* Change how the stylesheet url is determined in the templates: Remove
STYLEURL and add BASEURL to all templates (some already had it). This
new more general variable can be used to link to other things (eg, images)
from the template, as well as stylesheets.
* If a page links to itself, mark up the link text in a span with
class="selflink" so that it can be styled. I don't have a useful style
defined for that though.
* Add --version.
* Man page format fixups.
* Add a %pagecase which maps lower-case page names to the actual case
used in the filename. Use this in bestlinks calculation instead of
forcing the link to lowercase.
* Also use %pagecase in various other places that want to check if a page
with a given name exists.
* This means that links to pages with mixed case names will now work,
even if the link is in some other case mixture, and mixed case pages
should be fully supported throughout ikiwiki.
* Recommend rebuilding wikis on upgrade to this version.
them without annoyances.
* Suck in the page content when generating an inline archive page, to work
around the meta plugin's lack of caching of title metadata across runs.
* Add permalink and author support to meta plugin, affecting RSS feeds
and blog pages.
* Change titlepage() to encode utf-8 alnum characters. This is necessary
to avoid UTF-8 creeping into filenames in urls. (There are still
some other ways that it can get in.)
* PageSpecs can now include nested parens, "and", and "or". This remains
backwards compatible to the old GlobList format. It's implemented by
treating the GlobList as a very limited microlanguage that is transformed
to perl code that does the matching.
* The old GlobList format is deprecated, and I encourage users to switch to
using the new PageSpec format. Compatability with the old format will be
removed at some point, possibly by 2.0.
* Wiki rebuild needed on upgrade to this version due to PageSpec change.
* Add support for creation_month and creation_year to PageSpec.
Closes: #380680
* Changes to index file encoding.
plugin, so it's possible to implement a Planet using ikiwiki!
* --setup --refresh no longer rebuilds wrappers. Use --setup --refresh
--wrappers to do that.
* Add %IkiWiki::forcerebuild to provide a way for plugins like aggregate
to update pages that haven't changed on disk.
- Adds a locale setting to setup files.
- Proper local time, if the locale configuration option is used.
- Support for UTF-8 (or ISO-8859-X) filenames in SVN. Before this patch,
commiting (or even rcs_updating) on repositories with UTF-8 filenames was
impossible.
is a savings of 2 pointless fork/execs per link calculation, which
results in ~25% speedup of ikiwiki building its own doc wiki, and
about 35% speedup displaying RecentChanges!
layer, which led to lots of problems; make it force read files as utf-8.
Closes: #373203
* writefile() likewise needs to use the utf8 output layer.
* Remove the -CSD from ikiwiki's hashbang since it's useless to have it
there.
* Revert some of the decode_utf8 changes in CGI.pm that seem unnecessary
given the readfile fix.
* Add utf-8 testcases for readfile and htmlize.
for some handy stuff like:
- Using links as a kind of tag; creating blog pages that list all pages
containing a given tag/link or not containing some other tag.
- Subscribing to mail notifications whenever a change is made to a page
that is a backlink of page foo. Ie, "Please notify me of changes in all
pages that link to my home page in the wiki"
- Locking any pages that are linked to from a particular page, so that
lists of locks can be exposed in the wiki.
* When inlining a page in another one, links from the inlined page are now
expanded the same as they are when rendering the inlined page as a
standalone page. So rather than being expanded from the POV of the
inlining page, they are expanded from the POV of the inlined page.
For example, a link from blog/foo to "bar" will now link to blog/bar
if it exists. Previously this needed to be a link explicitly to
"blog/bar"; such links will also continue to work.
(This was slightly complex to do as the link still has to be constructed
relative to the inlining page.)
<span>, so pages can use <h1> internally instead of needing to use <h2>.
* Updated all of ikiwiki's own wiki pages for that.
* Add pagetemplate hook, which can be used by plugins that want to mess
around with adding new stuff to the page template.
* Remove headercontent; the search plugin now adds the search box to the
header by registering a pagetemplate hook, and other plugins should do
similarly.
for other types of search engine plugins if wanted, and also opening up a
lot of new possibilities for other kinds of plugins later
some notable changes along the way:
- lots of new hook types: cgi, render, delete
- wrapper files fixed to support config strings with newlines in them
- HEADERCONTENT in page template useful for plugins. Probably needs to be
expanded to more such for other places plugins might want to add content.
- remove unnecessary wrappers field from config info stored in wrappers
to pave the way for adding other RCS support. This also changes the
setup files, where before they had svn => 1 or svn => 0, now they have
rcs => "svn" or rcs => "".
* Add a debian/NEWS file with upgrade notes.
* Load whatever rcs module is specified, so new ones can be just dropped in
as plugins and work.
* Add some basic docs about writing RCS modules.
Joey Hess
Simon McVittie
Joey Hess
intrigeri
Josh Triplett
Joey Hess
joey
joshtriplett