strftime is a C function, it does not return decoded utf8.
Several places in ikiwiki manually decoded it, but at least two
forgot to.
Also, strftime might not return even encoded utf8, if LC_TIME is set
to a non-utf8 value. Went ahead and supported decoding whatever encoding
it uses.
The remaining direct calls to strftime() are all ones that first set
LC_TIME=C, in order to get times that are not for human display.
https://rt.cpan.org/Ticket/Display.html?id=74487
Gave up trying to support multiple YAML backends. The XS one requires ugly
manual encoding to get unicode right, and doesn't allow dumping yaml
fragments w/o the yaml header, but at least it doesn't randomly crash
on import like YAML::Mo has started to.
mdwn: Can use the discount markdown library, via the
Text::Markdown::Discount perl module.
This is preferred if available since it's the fastest currently supported
markdown library, speeding up markdown rendering by a factor of 40.
That is to say, when only rendering a lot of markdown, discount is 40x
faster. When building a ikiwiki site, ikiwiki's other overhead gets in the
way, but I still see significant speedups. Building the ikiwiki docwiki
dropped from 62 to 45 seconds, for example.
However, when multimarkdown is enabled, Text::Markdown::Multimarkdown is
still used.
While discount contains some nonstandard markdown extensions,
including tables and footnotes, AFAICS most of them are not
enabled by default in the perl bindings.
I consider sticking to non-extended markdown a desirable thing, since this
is probably not the last markdown engine. In particular, sundown is waiting
in the wings to get packaged and get a perl binding.
----
Reviewing all the showdown extensions, here are the ones that are enabled:
centered paragraphs:
->centered<-
image sizes: [dust mite](http://dust.mite =150x150)
<style>..</style> blocks are eaten. The perl binding does not provide
access to the gathered CSS. This is not legal html anyway, so unlikely
to cause breakage.
We had a weird problem where, after moving to a new, faster server,
"git push" would sometimes fail like this:
Unpacking objects: 100% (3/3), done.
fatal: The remote end hung up unexpectedly
fatal: The remote end hung up unexpectedly
What turned out to be going on was that git-receive-pack was dying due
to an uncaught SIGPIPE. The SIGPIPE occurred when it tried to write to
the pre-receive hook's stdin. The pre-receive hook, in this case, was
able to do all the checks it needed to do without the input, and so did
exit(0) without consuming it.
Apparently that causes a race. Most of the time, git forks the hook,
writes output to the hook, and then the hook runs, ignores it, and exits.
But sometimes, on our new faster server, git forked the hook, and it
ran, and exited, before git got around to writing to it, resulting in
the SIGPIPE.
write(7, "c9f98c67d70a1cfeba382ec27d87644a"..., 100) = -1 EPIPE (Broken
pipe)
--- SIGPIPE (Broken pipe) @ 0 (0) ---
I think git should ignore SIGPIPE when writing to hooks. Otherwise,
hooks may have to go out of their way to consume all input, and as I've
seen, the races when they fail to do this can lurk undiscovered.
I have written to the git mailing list about this.
As a workaround, consume all stdin before exiting.
Also, I let preview mode write real files, rather than using data: uri.
Which is ok these days, since ikiwiki tracks files created during
previewing, and cleans them up later.
In 875d550f12 I for some reason
made $page be changed when creating a discussion page, which
broke the link on the edit page. Changing page seems unnecessary,
so reverted that part of the change.
Involved dropping some checks for .svn which didn't add anything, since if
svn is enabled and you point it at a non-svn checkout, you get both pieces.
The tricky part is add and rename, in both cases the new file can be in
some subdirectory that is not added to svn.
For add, turns out svn has a --parents that will deal with this by adding
the intermediate directories to svn as well.
For rename though, --parents fails if the directories exist but are not
yet in svn -- which is exactly the case, since ikiwiki makes them
by calling prep_writefile. So instead, svn add the parent directory,
recursively.
tldr; svn made a reasonable change in dropping the .svn directories from
everywhere, but the semantics of other svn commands, particularly their
pickiness about whether parent directories are in svn or not, means
that without the easy crutch of checking for those .svn directories,
code has to tiptoe around svn to avoid pissing it off.
There's a nice message if the plugin is loaded and used and highlight is
not available, and a nice fallback. So no need for this other warning,
which can happen any time all plugins are loaded to generate a setup file.
This is such a pity. smcv had these great dates, but squeeze's Date::Parse
cannot parse them.
Oh well, at least it makes for a great bug closure title.
This kind of change is scary, but this particular lock is very simply
used and so it seems ok to make it even just for better portability to
SunOS. (People still use that?)
* Add unminified jquery js and css files to source.
* Update to jquery 1.6.2, and jquery-ui 1.8.14.
The full files are included in the source but not the binary.
I'm not minifying the files as part of build because I don't want ikiwiki
to build depend on a javascript minifier. (Let alone need one at runtime).
Nor do I want to deal with any breakage caused by the minifier. These
files were taken from the debian packages.
The jquery-tmpl full file was taken from revision
66bb852217c49ae8c9a8f2522150354ae80463de of its git repository, which
matches the minified file I already had. I did not want to deal with possible
breakage in newer versions; this thing claims to need an ancient version of
jquery (1.4.2), and is perhaps only working by luck with the newer versions
as it is.
* mercurial: openid nicknames are now used when committing. (Daniel Andersson)
* mercurial: implement rcs_commit_staged so comments, attachments, etc
can be used. (Daniel Andersson)
* mercurial: fix viewing of a diff containing non-utf8 changes.
(Daniel Andersson)
* rename: Fix logic error that broke renaming pages when the attachment
plugin was disabled.
* rename: Fix logic error that bypassed the usual pagespec checks.
Imagemagick does not generate svg images very well, but it can convert
them to png quite well.
For browsers that don't yet support displaying svg, this also provides a
workaround; just scale the svg down to get a png. But the workaround is
partial, since scaling the image larger, or leaving it the same size will
cause the original svg to be displayed. Since browsers are actively
improving svg support, this is good enough for me.
Needed for attachment to return json when requested.
I think some browsers send Accept: * , so I made sure to check that json
was explicitly listed as to be accepted, as well as having a high
priority.
Two problems fixed:
1. Files are written with a .ikiwiki-new suffix, which has to be taken into
account.
2. Need to count length of bytes, not of unicode characters.
Arguably, the real bug is in the interface to add_autofile, but since
that does take a filename, not a page name, it cannot really do case
handling on its own. The only other users of add_autofile in ikiwiki proper
is autoindex, and it always uses one case. Other third party plugins might
also need to add similar workarounds though.
* comments: Add avatar picture of comment author, using Libravatar::URL
when available. The avatar is looked up based on (Thanks, Francois Marier)
* Recommend libgravatar-url-perl, which contains Libravatar::URL.
This has been a while coming. It turns out that non-excutable setup files
have a number of benefits. Also, I find YAML setup files easier to edit
myself, and I suspect many users will prefer not needing to deal with
perl syntax.
Problem was this: websetup loads all plugins, but does not checkconfig
them. So, htmltidy's recently added configurable command setting was unset;
this resulted in its sanitize hook failing; the sanitize hook is called
when a sidebar was enabled, and this caused the sidebar to not display.
I put in a fix, but the underlying problem is that websetup loads all
plugins but leaves them in an unconfigured and possibly broken state while
trying to display its forms.
Probably the long-term fix is to have it cache the original hook states from
before loading the plugins, and restore it after getting their configuration.
Or, even to get the configuration using a subprocess, as plugins may do things
outside the hook system.
Now that page.tmpl is used for cgi, the parentlinks are able to be
displayed even when creating or editing a page. So it's redundant to
include the path to the page in the title, remove it.
There seems no need to allow selecting a location when creating a page this
way; the user should always want it to appear in the inline whose form they
submitted.
and support all elements that HTML::Tagset knows about.
(Which doesn't include html5 just yet, but then the old version didn't either.)
Bonus: 4 times faster than old regexp method.
So formbuilder has an annoying glitch, that setting the value of a
checkbox, even without force, will override the value currently on the
form. Thus the guards against changing checkbox values when a form has been
submitted.
But those guards also prevented the checkboxes for advanced items getting
the right value when going into advanced mode.
Note that if the user makes changes to advanced mode stuff and leaves
advanced mode, those changes are lost. That seems reasonable so I didn't
change it -- and it made this fix simple.
With a relative xrds-location, the openid perl client module will fail.
I haven't checked the specs to see if it needs to be absolute, but all
examples I've seen are absolute, so it seems a very good idea.
I also tried setting RPC::XML::ENCODING but that did not prevent the crash,
and it seems that blogspam.net doesn't like getting xml encoded in unicode,
since it mis-flagged comments as spammy that way that are normally allowed
through.
The rss spec says that unless the attribute is set, guid elements *are*
permalinks. The problem with that is that if [[meta permalink=]] is used,
as is done with aggregated posts, that goes into the link element, and
apparently some rss readers prefer the not-really-permalink in the guid
element when linking to the post.
Without meta permalink, the link and guid elements have the same content,
so it should be ok, in that case too for the guid to not be a permalink.
(Checked and this does not flood aggregators.)
The idea here is that <meta name="foo" description="bar">
can be written like [[!meta name="foo" description="bar">.
Of course, [[!meta foo=bar]] is still supported; this new feature
provides some DWIM when trying to directly convert a meta tag into
a meta directive.
template_depends was adding a dependency on the source filename,
instead of on the page name when a template is a page. Such a
dependency doesn't work.
Since misctemplate is called with a page context, the comments plugin
thinks it should add that, as well as the comment link in the actionbar.
I kept the comment link because a quick link back to the comments to a page
is sorta useful.
Cleanly fixed case where destdir file failed to be written because there
was a directory with the same name. This can be detected with no extra
system calls, and dealt with by finding all pages that wrote files
inside the directory, and removing them and the directory.
The other, inverse case would be expensive to detect in will_render,
since it would need to check each parent directory of the file to see
if the directory is really a conflicting file. But prep_writefile
already does a similar scan for symlinks in the path, so I added code
there to remove the conflicting file. This fix assumes that the file
is written using writefile, and not some other means (but using other means
would be a security hole too, so hopefully nothing does).
Renamed usershort => nickname.
Note that this means existing user login sessions will not have the nickname
recorded, and so it won't be used for those.
There was some confusion about whether the filename was
relative to srcdir or not. Some test cases, and the bzr
plugin assumed it was relative to the srcdir. Most everything else
assumed it was absolute.
Changed it to relative, for consistency with the rest
of the rcs_ functions.
Using named parameters for these is overdue. Passing the session in a
parameter instead of passing username and IP separately will later allow
storing other session info, like username or part of the email.
Note that these functions are not part of the exported API,
and the prototype change will catch (most) skew, so I am not changing
API versions. Any third-party plugins that call them will need updated
though.
Now the git plugin supports commits with author fields that look like:
Author: http://my.openid/ <me@web>
Then in recentchanges, the short username will be displayed, linking
to the openid.
Particularly useful for the horrible google openids, of course.
A short story:
Once there was a unicode string, let's call him Srcdir.
Along came a crufy old File::Find, who went through a tree and pasted each
of the leaves in turn onto Srcdir. But this 90's relic didn't decode the
leaves -- despite some of them using unicode! Poor Srcdir, with these
leaves stuck on him, tainted them with his nice unicode-ness. They didn't
look like leaves at all, but instead garbage.
In other words, perl's unicode support sucks mightily, and drives
us all to drink and bad storytelling. But we knew that..
So, srcdir is not normally flagged as unicode, because typically it's pure
ascii. And in that case, things work ok; File::Find finds filenames, which
are not yet decoded to unicode, and appends them to the srcdir, and then
decode_utf8 happily converts the whole thing.
But, if the srcdir does contain utf8 characters, that breaks. Or, if a Yaml
setup file is used, Yaml::Syck's implicitunicode sets the unicode flag of
*all* strings, even those containing only ascii. In either case, srcdir
has the unicode flag set; a non-decoded filename is appended, and
decode_utf8 sees the flag and does *nothing*. The result is that the
filename is not decoded, so looks valid and gets skipped.
File::Find only sticks the directory and filenames together in no_chdir
mode .. but we need that mode for security. In order to retain the
security, and avoid the problem, I made it not pass srcdir to File::Find.
Instead, chdir to the srcdir, and pass ".". Since "." is ascii, the problem
is avoided.
Note that it takes care to chdir back to the starting location. Because
the user may have specified relative paths and so staying in the srcdir
might break. A relative path could even be specifed for an underlay dir, so
it chdirs back after each.
Joey Hess