ikiwiki/doc/todo/should_optimise_pagespecs.mdwn

109 lines
5.0 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters!

This file contains ambiguous Unicode characters that may be confused with others in your current locale. If your use case is intentional and legitimate, you can safely ignore this warning. Use the Escape button to highlight these characters.

I think there is a problem in my "dependency graph". As an example,
[here](http://poivron.org/~nil/misc/ikiwiki_buggy_index) is the index
ikiwiki generated for [my site](http://poivron.org/~nil/misc/ikiwiki_buggy_index)
(note that the site changed since this index was generated).
Some **HUGE** dependencies appear, clearly non optimal, like
depends = A B | A | C | A | D | A | E | A | F | A | G | ....
or
depends= A | B | C | D | A | B | C | D | A | B | C | D | ....
Couldn't isolate the cause, but some sources for this problem may be:
* related to the img module
* easily observable in my sire because one of my pages includes 80 resized images
Other special things in my templates and site:
* a sidebar with \[[!include pages="notes/\*" template=foo]] while notes.mdwn has
a \[[!include pages="notes/*"]] and uses the sidebar; removed it, doesn't change
* a template (biblio.tmpl) calling the "img" plugin with a template parameter as the
image filename; removed it, doesn't change
* some strange games with tags whose page calls a "map" directive to show other tags
shile tags are also used in tagclouds (in the sidebar and in the main pages)
* ...
I observed these problems (same *kind*, I didn't check in details) on
* ikiwiki 2.00gpa1 + v5.8.4 + Debian 3.1
* ikiwiki 2.3 + v5.8.8 + Ubuntu 7.04
I can think about reducung the size of my wiki source and making it available online for analysis.
-- NicolasLimare
> As long as these dependencies don't grow over time (ie, when a page is
> edited and nothing changed that should add a dependency), I wouldn't
> worry about them. There are many things that can cause non-optimal
> dependencies to be recorded. For one thing, if you inline something, ikiwiki
> creates a dependency like:
>
> (PageSpec) or (file1 or file2 or file3 ...)
>
> Where fileN are all the files that the PageSpec currently matches. (This
> is ncessary to detect when a currently inlined file is deleted, and know
> the inlining page needs an update.) Now consider what it does if you have
> a single page with two inline statements, that inline the same set of
> stuff twice:
>
> ((PageSpec) or (file1 or file2 or file3 ...) or (PageSpec) or (file1 or file2 or file3 ...)
>
> Clearly non-optimal, indeed.
>
> Ikiwiki doesn't bother to simplify complex PageSpecs
> because it's difficult to do, and because all they use is some disk
> space. Consider what ikiwiki uses these dependencies for.
> All it wants to know is: does the PageSpec for this page it's considering
> rebuilding match any of the pages that have changed? Determining this is
> a simple operation -- the PageSpec is converted to perl code. The perl
> code is run.
>
> So the total impact of an ugly dependency like this is:
>
> 1. Some extra data read/written to disk.
> 2. Some extra space in memory.
> 3. A bit more data for the PageSpec translation code to handle. But that
> code is quite fast.
> 4. Typically one extra function call when the generated perl code is run.
> Ie, when the expression on the left-hand side fails, which typically
> happens after one (inexpensive) function call, it has to check
> the identical expression on the right hand side.
>
> So this is at best a wishlist todo item, not a bug. A PageSpec simplifier
> (or improved `pagespec_merge()` function) could be written and improve
> ikiwiki's memory and disk usage, but would it actually speed it up any?
> We'd have to see the code to the simplifier to know.
>
> --[[Joey]]
[[!template id=gitbranch branch=smcv/optimize-depends author="[[smcv]]"]]
>> I've been looking at optimizing ikiwiki for a site using
>> [[plugins/contrib/album]] (which produces a lot of pages) and it seems
>> that checking which pages depend on which pages does take a significant
>> amount of time. The optimize-depends branch in my git repository
>> avoids using `pagespec_merge()` for this (indeed it's no longer used
>> at all), and instead represents dependencies as a list of pagespecs
>> rather than a single pagespec. This does turn out to be faster, although
>> not as much as I'd like. --[[smcv]]
>>> I just wanted to note that there is a whole long discussion of dependencies and pagespecs on the [[todo/tracking_bugs_with_dependencies]] page. -- [[Will]]
>>>> Yeah, I had a look at that (as the only other mention of `pagespec_merge`).
>>>> I think I might have solved some of the problems mentioned there,
>>>> actually - `pagespec_merge` no longer needs to exist in my branch (although
>>>> I haven't actually deleted it), because the "or" operation is now done in
>>>> the Perl code, rather than by merging pagespecs and translating. --[[smcv]]
[[!template id=gitbranch branch=smcv/remove-pagespec-merge author="[[smcv]]"]]
>>>>> I've now added a patch to the end of that branch that deletes
>>>>> `pagespec_merge` almost entirely (we do need to keep a copy around, in
>>>>> ikiwiki-transition, but that copy doesn't have to be optimal or support
>>>>> future features like [[tracking_bugs_with_dependencies]]). --[[smcv]]
[[!tag wishlist patch patch/core]]