This new method for determining when links on pages
have changed should be more efficient, since it avoids
double calculation of the bestlinks.
It also allows collecting data about the old links, before
the scan pass, so the data is accurate when pages move around
and bestlinks change.
Also got some code back to a saner indent level.
Involved some code refactoring so that same code that detects
link changes for backlinks updating can be used for link dependency
checking. The nice thing is that link dep checking is thus
comopletly free!
Preliminary support, anyway.
If a dependency only includes DEPEND_EXISTS, then only changes that
involved adding or deleting a page can trigger it.
This is complicated by internal pages, since the code did not previously
differentiate between add, delete, and change of internal pages.
Now it tracks change separately from add+delete, so DEPEND_EXISTS pagespecs
that actually match internal pages (which will probably be quite rare in
practice) should work.
As soon as a change happens, we know we will need to rescan all
dependencies from the start, so bail out of the current scan partway to
avoid doing redundant work.
Only problem with this is that ikiwiki sometimes ends up printing out
dependencies that, while correct, are not obvious. Before:
building B, which depends on A
building C, which depends on A
building D, which depends on A
After:
building B, which depends on A
building C, which depends on B
building D, which depends on C
It's not "exact" since case munging has to be done, and I think
"simple" captures the optimisation better.</pedant>
With apologies to smcv, who probably has to rebuild his wiki now.
Let E be the number of dependencies per page of the form "A depends on B and
nothing else", let D be the number of other dependencies per page,
let P be the total number of pages, and let C be the number of changed
pages in a refresh.
This patch should speed up a refresh from O(E*C*P + D*C*P) to
O(C + E*P + D*C*P), assuming that hash lookups are O(1).
In practice, plugins like inline and map produce a lot of these very simple
dependencies, and my album plugin's combination of inline with a large
number of pages causes it to suffer particularly badly.
In testing on a wiki with about 7000 objects (3500 full pages, 3500
images), a full rebuild continued to take about 5:30, and a refresh
after touching about 350 pages and 350 images reduced from 5:30 to 1:30.
As with my previous optimizations, this change will result in downgrades not
working correctly until the wiki is rebuilt.
This should be more efficient than pagespec_match_list since it short-circuits
after the first match is found.
The other problem with using pagespec_match_list here is it may throw an
error if a bad or failing pagespec somehow got into the dependencies.
On a large wiki you can spend a lot of time reading through large lists
of dependencies to see whether files need to be rebuilt (album, with its
one-page-per-photo arrangement, suffers particularly badly from this).
The dependency list is currently a single pagespec, but it's not used like
a normal pagespec - in practice, it's a list of pagespecs joined with the
"or" operator.
Accordingly, change it to be stored as a list of pagespecs. On a wiki
with many tagged photo albums, this reduces the time to refresh after
`touch tags/*.mdwn` from about 31 to 25 seconds.
Getting the benefit of this change on an existing wiki requires a rebuild.
During backlink calulation, all links are examined and broken links can
be detected for free, so store a list of broken links and have brokenlinks
use it.
Exposing the %brokenlinks structure is a bit ugly, but the speedup seems
worth it: Around 1 second for wikis the size of the doc wiki that use
brokenlinks.
By adding this setting, we get both more configurability, and a minor
optimisation too, since gettext does not need to be called continually
to get the Discussion value.
Previously, if a page changed its type but not its mtime
(e.g. mv page.txt page.mdwn), then it would not be rebuilt.
Now, check if the source of a page has changed,
in which case force a rebuild of that page.
(cherry picked from commit b6a3b8a683fed7a7f6d77a5b3f2dfbd14c849843)
can be used to avoid a security check that is a good safe default, but
problimatic overkill in some situations.
I decided to underdocument this, because the option looks ugly, and I don't
want people randomly turning it on because it looks like a good idea. So if
you need it, you'll get an error message mentioning how to fix it.
* Renamed to parentlinks every single variable or function called
pedigree
* Removed the parentlinks function from Render.pm
* Enabled the new parentlinks plugin by default
* Adapted testsuite and documentation to reflate the above facts
Signed-off-by: intrigeri <intrigeri@boum.org>
If hardlinks are enabled, it would hardlink files from the underlay. That
was sorta annoying if you tried to edit by hand for some reason, so let's
not. Files that are hardlinked should be rare enough that a few extra stats
won't hurt.
* The editpage form now uses the raw page name, not the page title, in its
'page' cgi parameter. Using the title was ambiguous and made it
impossible to tell between some pages, like "foo/bar" and "foo__47__bar",
sometimes causing the wrong page to be edited.
* This change means that some edit links need to be updated.
Force a rebuild on upgrade to this version.
* Above change also allowed really fixing escaped slashes from the blogpost
form.
Because the search plugin needed it, also because it's one of the few
plugins that didn't already have it.
I also considered adding it to htmlize, but I really cannot imagine caring
what the destpage is when htmlizing. (I'll probably be poven wrong later.)
since this leads to too many problems with web caching, especially with
inlined pages. Properly solving this would involve tracking every page
that contributes to a page's content and using the youngest of them all,
as well as special cases for things like the version plugin, and it's just
too complex to do.
license, and copyright. This can be used to create custom RecentChanges.
* meta: To support the pagespec functions, metadata about pages has to be
retained as pagestate.
* Fix encoding bug when pagestate values contained spaces.
I kept it to a simple global configuration, rather than using the
preprocessor directive for recentchanges, because that had chicken and egg
problems and seemed overcomplicated. This should work reasonably well,
though it would be good to add some more metadata so that more customised
recentchanges pages can be made.
This makes it a lot quicker to deal with lots of recentchanges pages
appearing and disappearing. It avoids needing to clutter up pagespecs with
exclusions for those pages, by making normal pagespecs not match them.
This is important to do because until will_render is called, ikiwiki doesn't
know that the page exists. This avoids recentchanges re-writing every change
page every run.
* Plugins can add new directories to the search path with the add_underlay
function.
* Split out smiley underlay files into a separate underlay, so if the plugin
isn't used, the wiki isn't bloated with all those files.
for extended pagespecs. The old calling convention will still work for
back-compat for now.
* The calling convention for functions in the IkiWiki::PageSpec namespace
has changed so they are passed named parameters.
* Plugin interface version increased to 2.00 since I don't anticipate any
more interface changes before 2.0.
on and supported creating it (especially Tumov). This adds a "usedirs"
option that makes ikiwiki use foo/index.html instead of foo.html as
output page names. It is not yet enabled by default.
* Renamed %oldpagemtime to a more accurately named %pagemtime and fix it to
actually store pages' mtimes.
* Add "mtime" sort parameter to inline plugin.
were titlepage escaped in the urls, and then doubly escaped by the CGI
when editing. To fix this, I removed the titlepage escaping in the edit
urls.
* That means that *every edit link* on the wiki is potentially changed.
Rebuilding wikis on upgrade to this version therefore necessary; enabled
that in postinst.
parameters remain the same, but additional options are now passed in using
named parameters.
* Change plugin interface version to 1.02 to reflect this change.
* Add a new anchor option to htmllink. Thanks Ben for the idea.
* Support anchors in wikilinks.
* Add a "more" plugin based on one contributed by Ben to allow implementing
those dreaded "Read more" links in blogs.
including out of disk space situations. ikiwiki should never leave
truncated files, and if the error occurs during a web-based file edit,
the user will be given an opportunity to retry.
Inspired by the many ways Moin Moin destroys itself when out of disk. :-)
* Fix syslogging of errors.
manipulate.
* Only exclude rss and atom files from processing if the inline plugin
is enabled and that feed type is enabled. Else it's just a copyable file
type.
* Move rss and atom option handling code into the inline plugin.
* Applied a rather old patch from Recai to fix the "pruning is too strict"
issue. Now you can have wiki source directories inside dotdirs and the
like, if you want.
* Use precalculated backlinks info when determining if files need an update
due to a page they link to being added/removed. Mostly significant if
there are lots of pages.
* Remove duplicate link info when saving index. In some cases it could
pile up rather badly. (Probably not the best way to deal with this
problem.)
8x.
* Add "scan" parameter to hook(), which is used to make the hook be called
during the scanning pass, as well as the render pass. The meta and tag
plugins need to use the new scan parameter, so will any others that modify
%links.
* Now that links are calculated in a separate pass, it can also
precalculate backlinks in one pass, which is O(N^2) instead of the
previous code that was O(N^3). A very nice speedup for wikis with lots
(thousands) of pages.
instead of over and over. This is up to 8 times faster than before!
(This could have introduced some subtle bugs, so it needs to be tested
extensively.)
source file, to allow tracking of extra rendered files like rss feeds.
* Note that plugins that accessed this variable will need to be updated!
The plugin interface has been increased to version 1.01 for this change.
* Add will_render function to the plugin interface, used to register that a
page renders a destination file, and do some security checks.
* Use will_render in the inline and linkmap plugins.
* Previously but no longer rendered files will be cleaned up.
* You will need to rebuild your wiki on upgrade to this version.
- Plugins should not need to load IkiWiki::Render to get commonly
used functions, so moved some functions from there to IkiWiki.
- Picked out the set of functions and variables that most plugins
use, documented them, and made IkiWiki export them by default,
like a proper perl module should.
- Use the other functions at your own risk.
- This is not quite complete, I still have to decide whether to
export some other things.
* Changed all plugins included in ikiwiki to not use "IkiWiki::" when
referring to stuff now exported by the IkiWiki module.
* Anyone with a third-party ikiwiki plugin is strongly enrouraged
to make like changes to it and avoid use of non-exported symboles from
"IkiWiki::".
* Link debian/changelog and debian/news to NEWS and CHANGELOG.
* Support hyperestradier version 1.4.2, which adds a new required phraseform
setting.
This allows passing a wikilink inside a parameter to a preprocessor
directive without it being expanded to html, and leaking out of the
parameter, which had required some non-obvious use of triple-quoting
to avoid. Note that any preprocessor plugins that output something
that looks like a wikilink will now have it treated as such; AFAIK
this doesn't change any behavior though except for the template plugin.
* Enable preprocessor directives when previewing an edit.
text blocks, for easy nesting of quotes inside.
* Add a template plugin.
* Use the template plugin to add infoboxes to each plugin page listing basic
info about the plugin.
pages.
* Change how the stylesheet url is determined in the templates: Remove
STYLEURL and add BASEURL to all templates (some already had it). This
new more general variable can be used to link to other things (eg, images)
from the template, as well as stylesheets.
* Add --version.
* Man page format fixups.
* Add a %pagecase which maps lower-case page names to the actual case
used in the filename. Use this in bestlinks calculation instead of
forcing the link to lowercase.
* Also use %pagecase in various other places that want to check if a page
with a given name exists.
* This means that links to pages with mixed case names will now work,
even if the link is in some other case mixture, and mixed case pages
should be fully supported throughout ikiwiki.
* Recommend rebuilding wikis on upgrade to this version.
Run it with -asxhtml instead, so it will output well-formed xhtml no
matter what the input.
* Disable tidy warnings too.
* Add a new format hook, and make tidy use it, since tidy can really only
operate on and output complete html documents, not the body chunks
that sanitise gets.
* PageSpecs can now include nested parens, "and", and "or". This remains
backwards compatible to the old GlobList format. It's implemented by
treating the GlobList as a very limited microlanguage that is transformed
to perl code that does the matching.
* The old GlobList format is deprecated, and I encourage users to switch to
using the new PageSpec format. Compatability with the old format will be
removed at some point, possibly by 2.0.
* Wiki rebuild needed on upgrade to this version due to PageSpec change.
* Add support for creation_month and creation_year to PageSpec.
Closes: #380680
* Changes to index file encoding.
plugin, so it's possible to implement a Planet using ikiwiki!
* --setup --refresh no longer rebuilds wrappers. Use --setup --refresh
--wrappers to do that.
* Add %IkiWiki::forcerebuild to provide a way for plugins like aggregate
to update pages that haven't changed on disk.
the wikiformat plugin works. Thanks, Emanuele Aina.
* Patch from Alessandro Dotti Contra to clean up backlinks and navbar
display, running them through pagetitle. This removes ugly underscores and
other gunk from the display, much nicer. You may want to rebuild your wiki
when upgrading to get the cleanup globally.
* Pass a "destpage" parameter to preprocessor and pagetemplate hooks.
This will be the page that a source page will be part of, which is
different than the source page for inlined pages.
* Audited all plugins to endure they pass page, destpage to htmllink
appropriatly. This means inlining of various plugins will not work
properly, with correct links generated.
- allows preprocessor directives to have parameters with no specified
value
- fixes preprocessor directive parameter parsing so that
foo=bar baz now means "foo=bar" and a "baz" with no value
- Add a tag plugin that allows more easily tagging pages.
The meta plugin can also still be used for this.
* Noticed a bug in the wikitext markup plugin -- it made CamelCase links the
default throughout the wiki, not only on wikitext pages. Decided to call
this a feature, and split the camelcase support out into a separate plugin
that is independant of wikitext.
is a savings of 2 pointless fork/execs per link calculation, which
results in ~25% speedup of ikiwiki building its own doc wiki, and
about 35% speedup displaying RecentChanges!
This allows adding or removing plugins w/o overriding the whole list of
default plugins, which makes it easier to upgrade when new default plugins
are added.
flagged string even if the locale causes it to generate utf8 output,
so make sure to let perl know it should be handled as utf8. Also,
the optimised version used for standard time formats won't work if the
user has changed locale, so drop it. Thanks, Faidon Liambotis.
* Fix re-encoding of the comments field to utf8 if a commit fails
due to a conflict. Thanks, Faidon Liambotis.
* Let svn know that commits have utf8 commit messages. Thanks, Faidon
Liambotis.
* Add insane double encode/decode to utf8 around call to markdown.
This works around a truely strange bug, which is apparently a bug in
perl, which I lack space to describe here (see t/crazy-badass-perl-bug.t)
Joey Hess
Amitai Schlair
Simon McVittie
Jonas Smedegaard
Joey Hess
Gabriel McManus
intrigeri
Joey Hess
joey