Thanks to the new rename hook behaviour, the whole renaming work is now done
by the rename plugin, and we don't need to remember which pages were renamed.
inline has a format hook that is an optimisation hack. Until this hook
runs, the inlined content is not present on the page. This can prevent
other format hooks, that process that content, from acting on inlined
content. In bug ##509710, we discovered this happened commonly for the
embed plugin, but it could in theory happen for many other plugins (color,
cutpaste, etc) that use format to fill in special html after sanitization.
The ordering was essentially random (hash key order). That's kinda a good
thing, because hooks should be independent of other hooks and able to run
in any order. But for things like inline, that just doesn't work.
To fix the immediate problem, let's make hooks able to be registered as
running "first". There was already the ability to make them run "last".
Now, this simple first/middle/last ordering is obviously not going to work
if a lot of things need to run first, or last, since then we'll be back to
being unable to specify ordering inside those sets. But before worrying about
that too much, and considering dependency ordering, etc, observe how few
plugins use last ordering: Exactly one needs it. And, so far, exactly one
needs first ordering. So for now, KISS.
Another implementation note: I could have sorted the plugins with
first/last/middle as the primary key, and plugin name secondary, to get a
guaranteed stable order. Instead, I chose to preserve hash order. Two
opposing things pulled me toward that decision:
1. Since has order is randomish, it will ensure that no accidental
ordering assumptions are made.
2. Assume for a minute that ordering matters a lot more than expected.
Drastically changing the order a particular configuration uses could
result in a lot of subtle bugs cropping up. (I hope this assumption is
false, partly due to #1, but can't rule it out.)
People seem to be able to expect to enter www.foo.com and get away with it.
The resulting my.wiki/www.foo.com link was not ideal.
To fix it, use URI::Heuristic to expand such things into a real url. It
even looks up hostnames in the DNS if necessary.
A new ikiwiki-transition moveprefs subcommand can pull the old data out of
the userdb and inject it into the setup file.
Note that it leaves the old values behind in the userdb too. I did this
because I didn't want to lose data if it fails writing the setup file for
some reason, and the old data in the userdb will only use a small amount of
space. Running the command multiple times will mostly not change anything.
This leads to better display for OpenIDs like smcv.pseudorandom.co.uk
and thm.id.fedoraproject.org (to take a couple of examples from the
IkiWiki commit history).
None of the comment state needs to be stored through the a later run of
ikiwiki, so move it all from pagestate to a more transient storage.
This is assuming that we'll never want to add pagespecs to search against
the comment state. Pagespecs like author() are why the meta plugin does
store its meta data in pagestate -- the data can be needed later to match
against.
The thinking here is that having both a Discussion page and comments for
the same page is redundant, and certianly not what you want if you enable
comments for a page. At first I considered making configurable via pagespec
what pages got discussion links. But that would mean testing a new pagespec
for every page, and a redundant config setting to keep in sync. So intead,
take a lead from my previous change to make inlined pages have a comments
link, and change the discussion link at the top of regular pages to link to
their comments.
(Implementation is a bit optimised to avoid redundant pagespec checking.)
Jumping to the just posted comment was the imputus, but I killed a number
of birds here.
Added a INLINEPAGE template variable, which can be used to add anchors to
any inline template.
To keep that sufficiently general, it is the full page name, so the
comment anchors and links changed form.
Got rid of the FIXMEd hardcoded html anchor div.
More importantly, the anchor is now to the very top of the comment, not the
text below. So you can see the title, and how it attributes you.
Avoid changing the permalink of pages that are not really comments, but
happen to contain the _comment directive. I think that behavior was a bug,
though not a likely one to occur since _comment should only really be used
on comment pages.
I think it is clearer to have one pagespec that controls all pages with
comments, and a separate pagespec that can be used to close new comments on
a subset of those pages.
Not compacting whitespace is the most important one: now that we run
sanitize hooks on individual posted comments in the comments plugin,
whitespace that is significant to Markdown (but not HTML) is lost.
(cherry picked from commit cb5aaa3cee)
The [[!_comment]] directive is a serialization format, not something for
presentation to users, so we should use the least ambiguous possible
representation.
This delays all comment formatting until the last possible time, allows
us to set metadata without worrying that commenters may be able to evade
it, and means that changes to how a comment is saved can be handled
gracefully. It also gives us somewhere to put the commenter's username
or IP address for later reference.
Not compacting whitespace is the most important one: now that we run
sanitize hooks on individual posted comments in the comments plugin,
whitespace that is significant to Markdown (but not HTML) is lost.
This should ensure that users can't "break out" from the enclosing
<div>, making it impossible to forge comments (assuming htmlscrubber
is enabled, and so is either htmlbalance or htmltidy).
wikilinks are harmless, so we might as well allow them.
Access control for this plugin is a bit odd, since we specifically
don't want to allow comments to be edited - so the check is whether the
user is allowed to edit a deliberately invalid page name,
page/commented/on[smcvpostcomment]. You can put smcvpostcomment(*)
or smcvpostcomment(some/subdir/*) in $config{anonok_pagespec}
or the opposite in $config{locked_pages} to allow "editing" (really
just posting) comments.
I wanted this nearer to the top, but decided to put it after the
add_depends. Reasoning: It's possible with a combinaton of feedpages and
show options to make @list and @feedlist contain completly differing sets
of pages. We want to add_depends all pages in both sets. We could combine
the two lists and add_depends that, but it's slightly more efficient to
defer reducing @feedlist, and add_depends whichever list is longer.
This is a skeleton that does nothing yet.
See the comments in the code for an overview of the issue that arises, due to
the renamepage hook never being called globally.
Signed-off-by: intrigeri <intrigeri@boum.org>
Not implemented yet, 'cos the renamepage hook has to come first.
Else translations would be deleted on rename, what a shame.
Signed-off-by: intrigeri <intrigeri@boum.org>
... instead of already existing ones.
This fixes the "missing otherlanguages links on master pages just created via
the CGI" bug.
Signed-off-by: intrigeri <intrigeri@boum.org>
And enjoy a 10% rebuild time enhancement on a complex wiki full of maps and
other pseudo-dynamic content, with some other costly plugins enabled. So it
could well mean 20% on a more usual wiki.
Signed-off-by: intrigeri <intrigeri@boum.org>
This way, the po plugin will not appropriate PO files it is not responsible for,
and PO files existing before this plugin was enabled can coexist peacefully with
our own ones.
Signed-off-by: intrigeri <intrigeri@boum.org>
(I just removed in istranslation and _istranslation the dependency on
istranslatable... which broke things in a subtle way, hard to see at the first
glance.)
Signed-off-by: intrigeri <intrigeri@boum.org>
This is necessary so that things that fork to the background,
like pinger, and inline ping, don't block other cgis from running.
Note that websetup also calls unlockwiki, before refreshing / rebuilding
the wiki. It makes perfect sense for that not to block other cgis.
Fixed by making the cgi wrapper wait on a cgilock.
If you had to set apache's MaxClients low to avoid ikiwiki thrashing
your server, you can now turn it up to a high value.
The downside to this is that a cgi call that doesn't need to call lockwiki
will be serialised by this so only one can run at a time. (For example,
do=search.) There are few such calls, and all of them call loadindex,
so each still eats gobs of memory, so serialising them still seems ok.
It has grown up incrementally and new helper functions were added right in the
middle of the hooks, most often near the place they were used, which is
practical when doing initial development, but quite ugly afterwards, when helper
functions are useful to separate logic and implementation details.
Today's refactoring commits have brought the code to a much more maintainable
state, IMHO.
Signed-off-by: intrigeri <intrigeri@boum.org>
This is not needed now that tagpage returns a page name starting with a
slash.
(Also fixes a minor bug that the edit links started with double slashes due
to the hack.)
It is now more elegant IMHO, and the output is now sorted according to the
language name (instead of code).
Signed-off-by: intrigeri <intrigeri@boum.org>
The very same code was repeated at dozens of places.
NB: the real work is now done is _istranslation(), which is memoized,
so the additional function calls overhead should be compensated.
Signed-off-by: intrigeri <intrigeri@boum.org>
... to prevent the use of Encode::Guess::guess_encoding() in
Locale::Po4a::Transtractor (just a minor security measure, dependent on po4a
internals, but we have no reason to think Encode::Guess is not safe).
Signed-off-by: intrigeri <intrigeri@boum.org>
... by wrapping IkiWiki::urlto in order to workaround hard-coded
/index.$config{htmlext}, which is wrong when usedirs=0 and po_link_to=current
and translatable homepage
Signed-off-by: intrigeri <intrigeri@boum.org>
Now use the change hook to update these files, check them into VCS, and trigger
IkiWiki::refresh as needed. The needsbuild hook's help was required to prevent
infinite looping.
This more rigorous way of doing this fixes recentchanges (that was previously
not updated in some cases), and probably is a better long-term solution than the
two previously tested ones.
Signed-off-by: intrigeri <intrigeri@boum.org>
warnings are displayed if it is set to an invalid or incompatible value
(e.g. po_link_to=negotiated and disabled usedirs)
Signed-off-by: intrigeri <intrigeri@boum.org>
Only change of note is quoting some strings in a regexp, just in case
(also avoids the . matching any character!)
Mostly whitespace changes of no consequence.
... in a way compatible with various File::Temp versions.
The result is far from being perfect (see comments in the code for details),
but it does work.
Signed-off-by: intrigeri <intrigeri@boum.org>
manually triggering IkiWiki::refresh() was at least dubious, and more or less
buggy (it randomly broke the whole backlinks feature); thinking a bit more to
add the necessary bits to @needsbuild seems like a better way. don't play with
ikiwiki's internals if not absolutely needed.
Signed-off-by: intrigeri <intrigeri@boum.org>
As a trick, use editcontent hook to mark the page as unfiltered, to force our
filter() sub's to proceed again.
Signed-off-by: intrigeri <intrigeri@boum.org>
This reverts commits d9b9022c13 and
39d44d443d. This functionality should now be
achieved using the new inject() function.
Signed-off-by: intrigeri <intrigeri@boum.org>
It makes some test cases cry once every two tries; this may be related to the
artificial way the testsuite is run, or not. In the meantime, stop memoizing
this function.
Signed-off-by: intrigeri <intrigeri@boum.org>
This speeds up web commits by 1/4th of a second or so, since perl does
not have to start up for the post commit hook.
perl's locking is completly FuBar, since it's impossible to tell what perl
flock() really does, and thus difficult to write code in other languages
that interoperates with perl's locking. (Let alone interoperating with
existing fcntl locking from perl...)
In this particular case, I think I was able to find a way to avoid the
insanity, mostly. The C code does a true flock(2), and if perl is using an
incompatable lock method that does not use the same locking primative at
the kernel level, then the C code's test will fail, and it will go ahead
and run the perl code. Then the perl code's test will test the right thing.
On Debian, at least lately, perl's flock() does a true flock(2), so the
optimisation does work.
Add an inject function, that can be used by plugins that want to replace
one of ikiwiki's functions with their own version. (This is a scary thing
that grubs through the symbol table, and replaces all exported occurances
of a function with the injected version.)
external: RPC functions can be injected to replace exported functions.
Removed the stupid displaytime hook, and use injection instead.
The html links already went there, but internally the links were not
recorded as absolute, which could cause confusing backlinks etc.
For example, with tagbase=tags, if blog/tags/bar existed and blog/foo was
tagged bar, it would link to /tags/bar. But, the link would be recorded
simply as a link to tags/bar, and so later blog/tags/bar would appear to
have the backlink.
Need to use a hook because an exported function cannot be reliably
overridden. The replacement verstion was actually only affecting plugins
loaded after it.
formattime doesn't need a hook, since there's no reason to export it.
... else we can fall into some kind of nasty infinite loop, when two ikiwiki
instances don't store their working copy of the repository at the same place:
every POT file update in one repository would trigger an update of the same POT
file in the others repository, and so on.
Signed-off-by: intrigeri <intrigeri@boum.org>
This is not needed yet, but when newly created POT/PO files are added to
%pagesources and other data structures, we'll need this.
Signed-off-by: intrigeri <intrigeri@boum.org>
Both functions are called very often, and:
- istranslatable has no side effect
- _istranslation is the helper function, without any side effect, for the
istranslation function
Signed-off-by: intrigeri <intrigeri@boum.org>
Only the first filter function call on a given {page,destpage} must convert it
from the PO file, subsequent calls must leave the passed $content unmodified.
Else, preprocessing loops are the rule.
Signed-off-by: intrigeri <intrigeri@boum.org>
Replaced [[!translatable]] directive with po_translatable_pages setting.
Moved istranslatable/istranslation code to helper functions leaving place for
future caching and/or memoization. The PageSpec functions still work.
Signed-off-by: intrigeri <intrigeri@boum.org>
Apache's content negotiation transparently redirects any old URL (page.html) to
the new one, depending on the client preferred language (i.e. a German browser
will be fed with page.html.de). Transition to this naming convention is then
really smooth.
This naming convention allows one to deliberately display the master page, even
if her browser is configured for another language.
Signed-off-by: intrigeri <intrigeri@boum.org>
- renamed po_supported_languages to po_slave_languages
- added po_master_language option, which will soon be useful
Signed-off-by: intrigeri <intrigeri@boum.org>
- .po is a new supported wiki page type
- PO files are rendered verbatim into HTML
- override targetpage to ease Content Negotiation
Signed-off-by: intrigeri <intrigeri@boum.org>
* Add an underlay for javascript, and add ikiwiki.js containing some utility
code.
* toggle: Stop embedding the full toggle code on each page using it, and
move it to toggle.js in the javascript underlay.
Google allows has a nice feature, sitesearch, that allows anyone to
limit search results to a specific site. Obviously, this feature can be
used to provide a search engine for the local ikiwiki site without the
need to install any additional software. Just enable the 'google' plugin
and make sure that --url uses the proper hostname. Thanks to Joey for
helping to get the Perl implementation right.
Whenever the edit form is submitted, but not saved, the page location
select should reduce to the currently selected value. This was only done
when previewing before, but is also needed in order to support the case of
adding an attachment to a page that is just being created.
Before this change, the attachment plugin would get a weird value in
$form->field("page"), that did not reflect the actual page location.
newpagefile.
Note that newpagefile is not used here (or in recentchanges) because
the internal use pages they generate are transient and unlikely to
benefit from being put each in their own subdir.
I noticed that ikiwiki/formatting was beilg rebuilt when any page changed.
This turned out to be because it contained a complex conditional
"enabled(foo) or enabled(bar)", and the conditional plugin did not notice
that this consisted only of enabled() tests, and copied it unchanged into
add_depends. Thus, the page's dependencies were satisfied by any page
change.
The fix is to beef up the parser so that it can handle that and more
complex conditionals, and detect if they consist only of such tests.
To handle this, avoid populating %renderedfiles in preview,
and in expiry, check if the file is in %renderedfiles, if it is
do not delete it since it was saved.
Upgrades to the new index format should be transparent.
The version field is 3, because 1 was the old textual index, 2 was the
pre-versioned format.
This also includes some efficiency improvements to index loading, by
not copying a hash and using a reference.
* htmltidy: Avoid returning undef if tidy fails. Also avoid returning the
untidied content if tidy crashes. In either case, it seems best to tidy
the content to nothing.
* htmltidy: Avoid spewing tidy errors to stderr.
Conflicts:
IkiWiki/Plugin/recentchanges.pm
Note that smcv's approach of using urlto also gets the url right when
redirecting to a non-html file, which is a better approach than my recent
fix to recentchanges
Seems that the problem is that once the \nnn coming from git is converted
to a single character, decode_utf8 decides that this is a standalone
character, and not part of a multibyte utf-8 sequence, and so does nothing.
I tried playing with the utf-8 flag, but that didn't work. Instead, use
decode("utf8"), which doesn't have the same qualms, and successfully
decodes the octets into a utf-8 character.
Rant:
Think for a minute about fact that any and every program that parses git-log,
or git-show, etc output to figure out what files were in a commit needs to
contain this snippet of code, to convert from git-log's wacky output to a
regular character set:
if ($file =~ m/^"(.*)"$/) {
($file=$1) =~ s/\\([0-7]{1,3})/chr(oct($1))/eg;
}
(And it's only that "simple" if you don't care about filenames with
embedded \n or \t or other control characters.)
Does that strike anyone else as putting the parsing and conversion in the
wrong place (ie, in gitweb, ikiwiki, etc, etc)? Doesn't anyone who actually
uses git with utf-8 filenames get a bit pissed off at seeing \xxx\xxx
instead of the utf-8 in git-commit and other output?
I saw this in the wild, apparently a page was not present on disk, but was
in the aggregate db, and not marked as expired either. Not sure how that
happened, but such pages should get marked as expired since they have an
effectively zero ctime.
* edittemplate: Default new page file type to the same type as the template.
(willu)
* edittemplate: Add "silent" parameter. (Willu)
* edittemplate: Link to template, to allow creating it. (Willu)
* goodstuff: Remove otl plugin from the bundle since it needs a significant
external dependency and is not commonly used. If you use otl, make sure
you explicitly enable it now.
* goodstuff: Add more, progress, and table plugins to the bundle.
intrigeri
Joey Hess
Simon McVittie
Joey Hess
Peter Simons