Commit Graph

120 Commits (26dae8f04955915f4203fab4bd5301c959f7771a)

Author SHA1 Message Date
Joey Hess 9151346687 aggregate: Avoid resetting ctime when an item md5 changes.
Besides being wrong to do, this could lead to the wrong item
being expired, as follows: If B is added and at the same time
A is changed, then A's ctime may be set to the current time,
while B's is set to its creation time. Thus the new item, A,
is incorrectly removed as older.

(This interacted especially badly with the bug fixed by
90b4d079605b72bb50d1da41402d994960e10937.)
2009-06-12 19:31:23 -04:00
Joey Hess 90b4d07960 aggregate: Fix storing of changed md5.
The aggregate state merge code neglected to merge changes to the md5
field of an item. Therefore, if an item's md5 changed after initial
aggregation, it would be updated, and rewritten, each time thereafter.
This was wasteful and indirectly led to some expire problems.
2009-06-12 19:24:47 -04:00
Simon McVittie dd862b4639 Work around XML::Atom strangeness that results in double-encoded posts
See [[bugs/Aggregated_Atom_feeds_are_double-encoded]]. By default,
XML::Atom outputs strings of UTF-8 bytes with the Perl UTF8 flag stripped
off, which IkiWiki assumes to be Latin-1 and re-encodes as UTF-8 on
output. XML::Feed  does not currently (0.41-1) set the magic variable to
change this behaviour (I've filed a bug on CPAN), but IkiWiki can
usefully set the same variable as a workaround.
2009-02-03 19:48:55 +00:00
Joey Hess 678d467a40 finalise version 3.00 of the plugin api 2008-12-23 16:34:19 -05:00
Joey Hess 14a9e704c1 enable aggregate_internal by default 2008-12-23 16:08:21 -05:00
Joey Hess 61f8ea7f1d avoid uninitialized value
This can happen when a new field,
such as the new lasttry, is added.
2008-12-19 17:57:09 -05:00
Joey Hess f3bec255e4 aggregate: If a feed fails to be downloaded, try again immediatly next time aggregation is run, even if the usual time has not passed. Closes: #508622 (Michael Gold) 2008-12-17 21:27:28 -05:00
Joey Hess bb93fccf06 Coding style change: Remove explcit vim folding markers. 2008-12-17 15:22:16 -05:00
Joey Hess a990afd2f7 avoid uninitialized value warning 2008-11-18 13:46:03 -05:00
Joey Hess 15269fed64 improve escaping of wikilinks and preprocessor directives
The old method failed for '[' x 3.
2008-11-18 02:43:17 -05:00
Joey Hess 75f262f44d call decode_utf8 inside eval
holger reported that decode_utf8 was crashing with perl 5.8.8. Earlier, I
thought that passing 0 to the function avoided this with old perls, but
that was apparently not enough, it still crashes. So, put it inside the
eval, so we can at least recover from it crashing.
2008-11-17 15:56:15 -05:00
Joey Hess ecf2399f4f aggregate: Try to query XML::Feed for the base url when derelevatising links. Since this needs the just released XML::Feed 0.3, as well as a not yet released XML::RSS, it will fall back to the old method if no xml:base info is available. 2008-11-06 16:05:10 -05:00
Joey Hess 2b569f99d9 fix relativedate timezone inclusion
The machine parseable date needs to include a timezone.

Also, simplified the interface for date display.
2008-10-19 19:21:44 -04:00
Joey Hess 2321906cd7 mark up date so relativedate will work 2008-10-19 15:17:00 -04:00
Joey Hess f36080ae02 aggregate: Avoid bug that caused immediate expiration of items with a date in the future. 2008-10-16 18:20:16 -04:00
Joey Hess 121efb246f Removed the pagefile function, which was confusingly close in name to
newpagefile.

Note that newpagefile is not used here (or in recentchanges) because
the internal use pages they generate are transient and unlikely to
benefit from being put each in their own subdir.
2008-09-29 18:59:20 -04:00
Joey Hess 9ec9d6901d Export pagetitle, titlepage, linkpage. 2008-09-27 14:27:42 -04:00
Joey Hess f181962dfc typo 2008-09-27 12:50:39 -04:00
Joey Hess 093e815440 aggregate: Avoid uninitialized value warnings for pages with no recorded ctime.
I saw this in the wild, apparently a page was not present on disk, but was
in the aggregate db, and not marked as expired either. Not sure how that
happened, but such pages should get marked as expired since they have an
effectively zero ctime.
2008-09-25 15:38:51 -04:00
Joey Hess 914c92c4e1 fix check for nonexistant content 2008-09-17 18:19:52 -04:00
Joey Hess 89953e3e19 store ctime for use by expiry code
The expiry code does need to make sure to sort in ctime order, even if
expiring by count, so it expires the right ones.
2008-09-17 18:11:22 -04:00
Joey Hess 1cb4154bc8 aggregate: Support atom feeds with only a summary element, and no content elements. 2008-09-17 15:56:58 -04:00
Joey Hess b540b263de aggregate: Allow expirecount to work on the first pass. (expireage still needs to wait for the pages to be rendered though) 2008-09-17 14:27:31 -04:00
Joey Hess 903213e63f add plugin safe/rebuild info (part 1 of 2)
too many plugins.. brain exploding..
2008-08-03 16:40:12 -04:00
Joey Hess 42ac4ec009 remove default values in getsetup
They were a bit confusing, since they did not actually set the default, and
example values are sufficient.
2008-07-26 21:07:15 -04:00
Joey Hess 49e86d2562 turn booleans into questions 2008-07-26 14:43:47 -04:00
Joey Hess 1f8b0460c3 added getsetup hooks for all plugins up to recentchanges 2008-07-25 18:05:55 -04:00
Joey Hess 2696e0c6db fixes 2008-07-14 23:00:29 -04:00
Joey Hess 76bda69a92 use prune to delete rendered files
This handles deleting empty directories too.
2008-07-14 22:45:12 -04:00
Simon McVittie b29d11b3c6 Fix aggregateinternal migration so it skips expired entries, and deletes output 2008-07-15 03:24:05 +01:00
Joey Hess af48316b1d Merge commit 'smcv/aggregateinternal' into aggregateinternal
Conflicts:

	IkiWiki/Plugin/aggregate.pm
2008-07-14 20:15:08 -04:00
Simon McVittie 302bb836f2 Use _aggregated as extension for internalized aggregation 2008-07-15 01:04:29 +01:00
Joey Hess 51bf0ea529 changes to debug message printing 2008-07-14 19:43:45 -04:00
Simon McVittie 50a5ab3c4e Add aggregateinternal mode to ikiwiki-transition.
Usage:
1. Update all pagespecs that use aggregated pages to use internal()
2. ikiwiki-transition aggregateinternal $srcdir $htmlext
   (where $srcdir and $htmlext are the srcdir and htmlext options in
   your .setup file)
3. Add aggregateinternal to your .setup file
4. Rebuild the wiki
2008-07-14 23:39:22 +01:00
Simon McVittie 18b3e970ff Merge commit 'origin/master' into aggregateinternal 2008-07-14 23:34:07 +01:00
Joey Hess ffc99f5904 switch preprocess hooks to use error function 2008-07-13 15:05:34 -04:00
Simon McVittie 5c0896e674 Make aggregated items be internal pages if --aggregateinternal is used.
This addresses <http://ikiwiki.info/todo/aggregate_to_internal_pages/>
in a simple way. With this approach, a flag day is required, on which all
users of aggregated pages start to inline them using the internal() pagespec;
after that, the aggregateinternal option can safely be switched on in the
setup file (and the old aggregated pages can be deleted by hand).
2008-07-12 18:15:11 +01:00
Bernd Zeimetz b16c43a440 `template` option for aggregate.pm.
Allows to specify the template file which is used to
create the html pages.
2008-06-21 16:23:39 -04:00
Joey Hess d5d56a24bd When calling decode_utf8 on known-problimatic content in aggregate, explicitly pass 0 (FB_DEFAULT) as the second parameter. Apparently perl 5.8 needs this to avoid crashing on malformed utf-8, despite its docs saying it is the default. 2008-05-28 15:38:04 -04:00
Joey Hess 1f88cad3a2 aggregate: Add support for web-based triggering of aggregation for people stuck on shared hosting without cron. (Sheesh.) Enabled via the `aggregate_webtrigger` configuration optiom. 2008-05-05 20:20:45 -04:00
Joey Hess de8c34df59 aggregate: Correct a mistake in the code that dummy up a guid for feeds lacking one. 2008-04-03 02:36:01 -04:00
Joey Hess 16299dbae8 load HTML::Entities at top
Used in several subs, not all of which load it on demand, this seems simpler.
2008-03-14 18:43:54 -04:00
Joey Hess bc2671082b no need to use HTML::Entities, as it's loaded on demand by code below 2008-03-12 18:52:33 -04:00
Joey Hess 9d54cc4659 implement aggregate_locking design
Now aggregation will not lock the wiki. Any changes made during aggregaton are
merged in with the changed state accumulated while aggregating. A separate
lock file prevents multiple concurrent aggregators. Garbage collection
of orphaned guids is much improved. loadstate() is only called once
per process, so tricky support for reloading wiki state is not needed.

(Tested fairly thuroughly.)
2008-02-03 16:48:26 -05:00
Joey Hess 38affb0c1c add aggregate locking functions 2008-02-03 15:17:15 -05:00
Joey Hess 1f6591f0a6 * aggregate: Revert use of forking to not save state, that was not the right
approach.
2008-02-03 03:04:19 -05:00
Joey Hess 0779013683 * aggregate: Forking a child broke the one state that mattered: Forcing
the aggregating page to be rebuilt. Fix this.
2008-02-02 23:56:13 -05:00
Joey Hess 3803266b8f merged the recentchanges branch
misc fixes
2008-01-29 17:50:11 -05:00
Joey Hess 7a40bcab9a add missing test to avoid uninitialised value when a page with metadata is removed 2008-01-29 17:36:25 -05:00
Josh Triplett fafb2edaa7 aggregate.pm: Replace Unicode apostrophe with ASCII to unbreak build
gettext choked on a Unicode apostrophe in the aggregate plugin, which
appeared in a new error message in commit
4f872b5633.  Replace it with an ASCII
apostrophe.
2008-01-26 23:12:29 -08:00