During refresh of a wiki with 800 files, loadindex was using more total
time than any other function, and saveindex was also in the top ten.
Rewriting them to use Storable makes them three times as fast.
0.7 seconds is saved on my laptop in profiling mode.
About 12% of ikiwiki runtime was spent in pagespec_match. It was evaling
the same pagespec code over and over again. This changes pagespec_translate
to return memoized, precompiled functions that can be called to match against
a given pagespec.
This also allows getting rid of the weird variable scoping trick that had
to be in effect for pagespec_translate to be called -- the variables are
now just fed into the function it returns.
On my laptop, this drops build time for the docwiki from about 60 to 50
seconds.
Markdown is slow. Especially if it has to process an enormous page. The
most common enormous page is currently the recentchanges page, which gets
processed a lot, and contains very little actual markdown. Most of it is a
big <div>, which markdown skips ... slowly.
This is a rather sick optimisation to work around markdown's speed issues.
Now inline inserts a small, dummy div, allows markdown to quickly render
the actual page content, then replaces the dummy with the actual inlined
pages later.
Results: Rendering just a recentchanges page, with diffs included, dropped
from 4.5 seconds to 2.7 seconds on my laptop. Building the entire wiki
dropped from 46.6 seconds to 39.5 seconds.
(It would be better if inline were a *post*-processor directive.)
set to the destination page. This avoids need for hacks to munge the urls
in preview mode, which fixes several bugs.
* Several destpage fixes in plugins.
correct, and it's certianly not correct now, since the wiki is locked
before rcs_commit is ever called, and should not be unlocked by
rcs_commit either.
Markdown is such a splintered mess.. The current debian package provides
only Text::Markdown::Markdown, while all versions of Text::Markdown support
Text::Markdown::markdown, and old versions also support the capitalised version,
while new ones don't.
It's getting to the point where `grep /markdown/i %symbol_table` is the only
sane way to figure out what function to call..
* rcs_diff is a new function that rcs modules should implement.
* Implemented rcs_diff for git, svn, and tla (tla version untested).
Mercurial and monotone still todo.
Add special handling for <meta name="robots" ...> which needs not be
scrubbed as it's harmless.
Signed-off-by: martin f. krafft <madduck@madduck.net>
(cherry picked from commit b15d0299a7f7b147e89d8a202d6cca1c21491af2)
As was already done for linkfication, links generated in a prevew page
are relative to the top of the wiki, so it has to be told that the destpage
is there.
I was using "" to indicate this, but that may confuse some preprocessor
plugins, which treat parameters with an empry value specially (sparkline is one
such). Instead, use "/", which is more accurate anyway and works just as well.
which forced a scan of the page to make available metadata that
appeared after the inline directive. Problem is that scan made it forget
about any other files rendered due to the page. The scan also turns out
to be unnecessary now, since meta persistently stores state and it's
always available. So it was just removed.
(as preserving the full list across preview would be tricky). Userdirs
were still being offered as an option there, remove them.
* Fix a bug where user A created a page concurrently with user B, and
when B previewed it would redirect B to A's new page, losing B's work.
Instead, don't redirect and let conflict handling resolve it.
containing ikiwiki.cgi, but this should not change the urls to the style
sheets etc. Add a new forcebareurl parameter to misctemplate to allow
it to do that.
of XML::RPC's default of us-ascii. Allows interoperation with
python's xmlrpc library, which threw invalid encoding exceptions and
caused the rst plugin to hang.
Some browsers interpret about: URIs like a limited version of data:
URIs. In particular, some versions of Internet Explorer interpret
arbitrary HTML content in about: URIs.
just avoid actually writing the files. This is necessary because ikiwiki
saves state after a preview (in case it actually *did* write files),
and if will_render isn't called its security checks will get upset
when the page is saved. Thanks to Edward Betts for his help tracking this
tricky bug down.
- On commits, replace "mtn sync" bidirectional with "mtn push" single
direction. No need to pull changes when doing a commit. mtn sync
is still called in rcs_update.
- Support for viewing differences via patches using viewmtn.
Now aggregation will not lock the wiki. Any changes made during aggregaton are
merged in with the changed state accumulated while aggregating. A separate
lock file prevents multiple concurrent aggregators. Garbage collection
of orphaned guids is much improved. loadstate() is only called once
per process, so tricky support for reloading wiki state is not needed.
(Tested fairly thuroughly.)
since this leads to too many problems with web caching, especially with
inlined pages. Properly solving this would involve tracking every page
that contributes to a page's content and using the youngest of them all,
as well as special cases for things like the version plugin, and it's just
too complex to do.
license, and copyright. This can be used to create custom RecentChanges.
* meta: To support the pagespec functions, metadata about pages has to be
retained as pagestate.
* Fix encoding bug when pagestate values contained spaces.
This makes it a lot quicker to deal with lots of recentchanges pages
appearing and disappearing. It avoids needing to clutter up pagespecs with
exclusions for those pages, by making normal pagespecs not match them.
Add a prefix_directives option to the setup file to turn this syntax
on; currently defaults to false, for backward compatibility. Support
optional '!' prefix even with prefix_directives off, and use that in
the underlay to support either setting of prefix_directives. Add NEWS
entry with migration information.
the code, since that process can change internal state as needed, and
it will automatically be cleaned up for the parent process, which proceeds
to render the changes.
remove the enclosing paragraph and newline markdown wraps it in.
This allows removing several hacks around this markdown behavior from
other plugins that htmlize fragements of pages.
returned (and not run in some cases) rather than the plugins directly
forcing a user to log in.
* opendiscussion: allow editing of the toplevel discussion page,
and, indirectly, allow creating new discussion pages.
* decode_form_utf8 only fixed the utf-8 encoding for fields that were
registered at the time it was called, which was before the
formbuilder_setup hook. Fields added by the hook didn't get decoded.
But it can't be put after the hook either, since plugins using the hook
need to be able to use form values. To fix this dilemma, it's been changed
to a decode_cgi_utf8, which is called on the cgi query object, before the
form is set up, and decodes *all* cgi parameters.
the needsbuild hook. This resulted in feeds not being removed when pages
were updated, and probably other bugs.
* aggregate: Avoid uninitialised value warning when removing a feed that
has an expired guid.
links required meta to be run during scan, which complicated its data
storage, since it had to clear data stored during the scan pass to avoid
duplicating it during the normal preprocessing pass.
* If you used "meta link", you should switch to either "meta openid" (for
openid delegations), or tags (for internal, invisible links). I assume
that nobody really used "meta link" for external, non-openid links, since
the htmlscrubber ate those. (Tell me differently and I'll consider bringing
back that support.)
* meta: Improved data storage.
* meta: Drop the hackish filter hook that was used to clear
stored data before preprocessing, this hack was ugly, and broken (cf:
liw's disappearing openids).
* aggregate: Convert filter hook to a needsbuild hook.
inserting them into the html template. This ensures that markdown
acts on them, even if the value is expanded inside a block-level html
element in the html template. Closes: #454058
* Use a div in the note template rather than a span.
so that more than one plugin can use this hook.
I believe this is a safe change, since only passwordauth uses this hook.
(If some other plugin already used it, it would have broken passwordauth!)
It would be better if it were a formbuilder hook. But the formbuilder hook
is wacked.. I may need to change how that hook works, which would mean
changing the only current user of it, passwordauth).
and forces rebuilds of the pages that contain calendars. So
running ikiwiki --refresh at midnight is now enough, no need for a full
wiki rebuild each midnight.
* calendar: Work around block html parsing bug in markdown 1.0.1 by
enclosing the calendar in an extra div.
which has been reported to cause encoding problems (though I haven't
reproduced them), just catch a failure of markdown, and retry.
(The crazy perl bug magically disappears on the retry.)
Closes: #449379
and importing existing content for svn, git, and mercurial. This makes
the setup process much simpler.
* Reorganised git documentation.
* Actually install the ikiwiki-update-wikilist program.
Patch by Daniel Burrows. Closes: #451729
* When usedirs is disabled, link direct to index.html files, not to
directories, to improve browsing of file:// urls.
Patch by Daniel Burrows. Closes: #451728
to be created owned by some group other than the default. Useful
then there's a shared repository with access controlled by a group,
to let ikiwiki run setgid to that group.
* ikiwiki-mass-rebuild: Run build with the user in all their groups.
It needs to be setgid if the master repo is a shared repository, so
that pushes into the working copy repository happen as the same group,
avoiding permissions problems.
page name to be expired and reused for several distinct guids. When this
happened, the expiry code counted each past guid that had used that page
name as a currently existing page, and thus expired too many pages.
* Reformat calendar plugin to ikiwiki conventions.
* The calendar plugin made *every* page depend on every other page,
which seemed a wee tiny little bit overkill. Fixed the dependency
calculations (I hope.)
* Removed manual ctime statting code, and just have the calendar plugin use
%pagectime.
ikiwiki via XML RPC. This should be much faster than the old plugin that
had to fork python for every rst page render. Note that if you use
the rst plugin, you now need to have the RPC::XML perl module installed.
showed up where a web edit that added a page caused a near-concurrent
web edit to fail in will_render. While it would be hard to reproduce this,
my analysis is that the failing cgi started first, loaded the index file
(prior to locking) then the other cgi created the new page and rendered
it, and then the failing cgi choked on the new file when _it_ tried to
render it. Ensuring that the index file is loaded after taking the lock
will avoid this bug.
files in some situations, and this is appropriate in some cases, such as
the teximg plugin's error log file.
Such files will be automatically cleaned up at an appopriate later time.
are not included in the map. Include special styling for such pages.
* map: Remove common prefixes and don't over-indent.
* Add class option to htmllink().
* table plugin: The previous version broke WikiLinks inside quoted values.
Fix this by linkifying CSV data after parsing it, while DSV data is still
linkified before parsing.
in the wikilink looked like a table field separator. Avoid this ambiguity
by linkifying the data before parsing it as a table.
* Turn on allow_loose_quotes in the table plugin's Text::CSV object,
so that links from wikilinks don't confuse the parser.
* Plugins can add new directories to the search path with the add_underlay
function.
* Split out smiley underlay files into a separate underlay, so if the plugin
isn't used, the wiki isn't bloated with all those files.
calendar, and youtube. Normally, the htmlsanitiser eats these since they
use unsafe tags, the embed plugin overrides it for trusted sites.
* The googlecalendar plugin is now deprecated, and will be removed
eventually. Please switch to using the embed plugin.
- add a title to the editpage form;
- pass a reference to the list of buttons to the formbuilder_setup
hooks, so we can add ours;
- relax asumption about the possible submit values (use "Save Page"
explicitly);
- de-hardcode the submit buttons from the editpage template
(This was needed for compatability with a bug in CGI::FormBuilder
3.0401, but ikiwiki already needs a newer version.)
* Pass buttons to all other formbuilder_setup hooks too.
atom feeds, and also changing the publication time for a feed to the
newest modiciation time (was newest creation time).
* The patch also adds dcterms:creator to rss items that have a known author.
* Use type= not style= in html for alternate stylesheets, which is more
correct (but in my testing both epiphany and iceweasel work ok with
style=text/css).
* Support building on systems that lack asprintf.
* mercurial getctime is currently broken, apparently by some change in
mercurial version 0.9.4. Turn the failing test case into a TODO test case.
most web sites serve ikiwiki xhtml files as text/html and mozilla browsers
get confused. So it's best for ikiwiki to follow the compatability
recommendations in appendix C of the XHTML spec. Closes: #432045
This turns out to have occured if the cgi wrapper was created by an
ikiwiki invocation that included --rebuild. Thanks to Carl Worth for
tracking that down.
ESCAPE=HTML for titles in the templates for these feeds, and instead
escape the title going in to the template. Previously, the title was
sometimes double-escaped in a feed (if set via meta title), and sometimes
not (if set from the page filename).
* In the meta plugin, when a title is set, encode the html entities in it
numerically. This works better in the current landscape of a rss spec that
doesn't specify encoding, and variously broken feed consumers, according
to <http://www.rssboard.org/rss-profile#data-types-characterdata>.
old files.
* Change where the img plugin puts scaled images. It's better to make the
scaled images subpages of the page that embeds them, rather than putting
them alongside the original image, since if two pages scale the same image
the same way, this prevents complications in dealing with two pages
creating the same file. The move will be handled transparently, though you
might want to rebuild your wiki to make it occur in one step.
until the wiki is building and already locked, unless it's aggregating.
When aggregating, it does not wait for the lock if it cannot get it, and
instead exits, to prevent aggregating processes from piling up.
and style sheet updates, and unless you're using customised versions,
you'll want to rebuild wikis on upgrade to this version to avoid
inconsistencies.
* Allow WIKINAME to to used in footers, as an example of something to put
there.
passwordauth page to the basewiki describing password
authentication; like openid, it uses conditional to check which
forms of authentication the wiki allows. Add conditional cross-
links between the openid and passwordauth pages, to help the user
understand how they can log in.
using the mercurial backend. Not 100% sure why it failed w/o the full
path, but this still passes the test suite, and indeed, is how the test
suite calls hg add.
(Get a good message when a PageSpec fails due to a negated success by
creating success objects with a reason string, which morph into failure
objects when negated.)
scalar context, evaluates to a reason why the match failed.
* Add testpagespec plugin, which might be useful to see why a pagespec isn't
matching something.
for extended pagespecs. The old calling convention will still work for
back-compat for now.
* The calling convention for functions in the IkiWiki::PageSpec namespace
has changed so they are passed named parameters.
* Plugin interface version increased to 2.00 since I don't anticipate any
more interface changes before 2.0.
* Note that some plugins have additional dependencies, found among the
Recommends and Suggests.
* ikiwiki supports both RSS and Atom feeds.
* Fix typos.
on and supported creating it (especially Tumov). This adds a "usedirs"
option that makes ikiwiki use foo/index.html instead of foo.html as
output page names. It is not yet enabled by default.
available templates. Rename the old templates page to wikitemplates.
* Include the note template in the basewiki.
* Add a popup template in the basewiki. CSS based on some by Martin Krafft.
* Make the note, popup, and plugin templates detect missing variables and be
self-documenting, listing the available variables.
and RCS server (or a push mirror). With this the RCS server can have a
real post-commit hook that sshs back to the web server, and runs ikiwiki
in post-commit mode to update the wiki.
* Renamed %oldpagemtime to a more accurately named %pagemtime and fix it to
actually store pages' mtimes.
* Add "mtime" sort parameter to inline plugin.
that given link points based on the page doing the linking. Note that this
could make such PageSpecs match different things than before, if you
relied on the old behavior of them only matching the raw link text.
* This required changing the match_* interface, adding a third parameter.
* Allow link() PageSpecs to match relative, as is allowed with globs.a
* Add postform option to inline plugin.
* Add an bug tracker to the softwaresite example.
plugins's support for inserting html link and meta tags. Now such content
is passed through the htmlscrubber like everything else.
* Unfortunatly, that means that some valid uses of those tags are no longer
usable, and special case methods needed to be added for including
stylesheets, and for doing openid delegation. If you use either of these
in your wiki, it will need to be modified. See the meta plugin docs
for details.
were titlepage escaped in the urls, and then doubly escaped by the CGI
when editing. To fix this, I removed the titlepage escaping in the edit
urls.
* That means that *every edit link* on the wiki is potentially changed.
Rebuilding wikis on upgrade to this version therefore necessary; enabled
that in postinst.
since it ended up being double-escaped. Instead, just remove slashes.
* Fix some nasty issues with page name escaping during previewing
(introduced in 1.44).
previous ugly hack used to avoid writing rss feeds in previews.
* Fix the img plugin to avoid overwriting images in previews. Instead it
does all the work to make sure the resizing works, and dummys up a resized
image using width and height attributes.
* Also fixes img preview display, the links were wrong in preview before.
commit hook, it was possible for one CGI to race another one and "win"
the commit of both their files. This race has been fixed by adding a new
commitlock, which when locked by the CGI, disables the commit hook
(except for commit mails). The CGI then takes care of the updates the
commit hook would have done.
parameters remain the same, but additional options are now passed in using
named parameters.
* Change plugin interface version to 1.02 to reflect this change.
* Add a new anchor option to htmllink. Thanks Ben for the idea.
* Support anchors in wikilinks.
* Add a "more" plugin based on one contributed by Ben to allow implementing
those dreaded "Read more" links in blogs.
OpenID login form rather than linking to a remote logo, to avoid various
issues. Since there is not yet a license for the actual OpenID logo, this
file is currently a blank image. Users who want to can copy
http://openid.net/login-bg.gif into their wiki.
Joey Hess
martin f. krafft
Adeodato Simó
Josh Triplett
Joey Hess
joey
joshtriplett
