Commit Graph

77 Commits (59379d0205fdbdb90553d1f5cef666e7e72a8927)

Author SHA1 Message Date
Joey Hess 9d54cc4659 implement aggregate_locking design
Now aggregation will not lock the wiki. Any changes made during aggregaton are
merged in with the changed state accumulated while aggregating. A separate
lock file prevents multiple concurrent aggregators. Garbage collection
of orphaned guids is much improved. loadstate() is only called once
per process, so tricky support for reloading wiki state is not needed.

(Tested fairly thuroughly.)
2008-02-03 16:48:26 -05:00
Joey Hess 38affb0c1c add aggregate locking functions 2008-02-03 15:17:15 -05:00
Joey Hess 1f6591f0a6 * aggregate: Revert use of forking to not save state, that was not the right
approach.
2008-02-03 03:04:19 -05:00
Joey Hess 0779013683 * aggregate: Forking a child broke the one state that mattered: Forcing
the aggregating page to be rebuilt. Fix this.
2008-02-02 23:56:13 -05:00
Joey Hess 3803266b8f merged the recentchanges branch
misc fixes
2008-01-29 17:50:11 -05:00
Joey Hess 7a40bcab9a add missing test to avoid uninitialised value when a page with metadata is removed 2008-01-29 17:36:25 -05:00
Josh Triplett fafb2edaa7 aggregate.pm: Replace Unicode apostrophe with ASCII to unbreak build
gettext choked on a Unicode apostrophe in the aggregate plugin, which
appeared in a new error message in commit
4f872b5633.  Replace it with an ASCII
apostrophe.
2008-01-26 23:12:29 -08:00
Joey Hess 4f872b5633 * aggregate: Fork a child process to handle the aggregation. This simplifies
the code, since that process can change internal state as needed, and
  it will automatically be cleaned up for the parent process, which proceeds
  to render the changes.
2008-01-24 17:22:39 -05:00
Joey Hess ba6b8e5e3e * aggregate: Don't let feeds set creation times for pages in the future. 2008-01-16 02:01:00 -05:00
Joey Hess 86781fc43e * aggregate: Include copyright statements from rss feed as meta copyright
directives.
* aggregate: Yet another state saving fix (sigh).
* aggregate: Add hack to support feeds with invalidly escaped html entities.
2008-01-08 20:41:25 -05:00
Joey Hess 76e941a480 * aggregate: Only save state if it was already loaded. 2008-01-05 01:26:09 -05:00
Joey Hess 41234a3f0f round days old message 2008-01-02 23:38:45 -05:00
Joey Hess c3eec7fdf0 I *think* that this comment is out of date. 2007-12-30 14:57:33 -05:00
Joey Hess f0f52e602d * aggregate: Fix stupid mistake introduced when converting it to use
the needsbuild hook. This resulted in feeds not being removed when pages
  were updated, and probably other bugs.
* aggregate: Avoid uninitialised value warning when removing a feed that
  has an expired guid.
2007-12-30 14:49:25 -05:00
Joey Hess f373b09bf9 loadstate needs to be called even if no pages are being built 2007-12-16 19:40:50 -05:00
Joey Hess 96817b0032 * meta: Drop support for "meta link", since supporting this for internal
links required meta to be run during scan, which complicated its data
  storage, since it had to clear data stored during the scan pass to avoid
  duplicating it during the normal preprocessing pass.
* If you used "meta link", you should switch to either "meta openid" (for
  openid delegations), or tags (for internal, invisible links). I assume
  that nobody really used "meta link" for external, non-openid links, since
  the htmlscrubber ate those. (Tell me differently and I'll consider bringing
  back that support.)
* meta: Improved data storage.
* meta: Drop the hackish filter hook that was used to clear
  stored data before preprocessing, this hack was ugly, and broken (cf:
  liw's disappearing openids).
* aggregate: Convert filter hook to a needsbuild hook.
2007-12-16 15:56:09 -05:00
Joey Hess 2a6e353c20 improve use of seen hash 2007-10-30 22:56:13 -04:00
Joey Hess 5cbd86235e * Fix an aggregate plugin expiry bug. Over time, it's possible for the same
page name to be expired and reused for several distinct guids. When this
  happened, the expiry code counted each past guid that had used that page
  name as a currently existing page, and thus expired too many pages.
2007-10-30 22:50:44 -04:00
joey c0f8808232 * Various minor bug fixes for silly mistakes in the code, thanks to the
various reviewers.
2007-07-28 21:01:56 +00:00
joey 1612dce6bb * Add a htmlext config option, this can be used to make ikiwiki generate
html files with a different extension.
2007-07-25 01:16:53 +00:00
joey ee843c7523 improve messages 2007-05-21 03:00:45 +00:00
joey 10017feaec Clear state after performing aggregation, since it could hypothetically
change after the lock is dropped, and before the lock is regained by the
build process. The state will now be reloaded by the build process.
2007-05-21 02:59:06 +00:00
joey 82ac63d798 * Change the aggregate plugin's locking strategy. Now it defers loading state
until the wiki is building and already locked, unless it's aggregating.
  When aggregating, it does not wait for the lock if it cannot get it, and
  instead exits, to prevent aggregating processes from piling up.
2007-05-21 02:52:51 +00:00
joey ee1ad53c4c * pagespec_match() has changed to take named parameters, to better allow
for extended pagespecs. The old calling convention will still work for
  back-compat for now.
* The calling convention for functions in the IkiWiki::PageSpec namespace
  has changed so they are passed named parameters.
* Plugin interface version increased to 2.00 since I don't anticipate any
  more interface changes before 2.0.
2007-04-27 02:55:52 +00:00
joey 7ea3bfdb70 improve message 2007-04-23 19:32:21 +00:00
joey 71d2d93027 * Fix aggregator to not warn when a feed contains no body content at all.
* Work around bug #420636 by, if XML::Parser crashes, running the feed
  content though Encode::decode_utf8.
2007-04-23 18:36:44 +00:00
joey 13c7a51f1f * Optimise displaytime, trimming maybe 6% off the build time. 2007-04-10 01:49:16 +00:00
joey ae249feb07 another htmlpage call that is no longer appropriate 2007-04-01 20:55:39 +00:00
joey 02b9f69ba5 * Finally apply the index.html patch, with thanks to everyone who worked
on and supported creating it (especially Tumov). This adds a "usedirs"
  option that makes ikiwiki use foo/index.html instead of foo.html as
  output page names. It is not yet enabled by default.
2007-04-01 19:59:42 +00:00
joey 3a958f7196 fix error message to include url 2007-03-31 03:26:43 +00:00
joey 5b796b0bdb typo 2007-03-08 11:13:16 +00:00
joey 66a65b5ddd * Correct a deadlock that could occur in post-commit if the aggregate plugin
was enabled and tried to lock the already locked wiki.
2007-03-08 11:04:59 +00:00
joey 4d8b2d84d5 more i18n improvements 2007-03-07 12:04:45 +00:00
joey d4c61b7281 * Many changes to make ikiwiki very resistant to write failures
including out of disk space situations. ikiwiki should never leave
  truncated files, and if the error occurs during a web-based file edit,
  the user will be given an opportunity to retry.
  Inspired by the many ways Moin Moin destroys itself when out of disk. :-)
* Fix syslogging of errors.
2007-02-15 02:22:08 +00:00
joey 1379fa7402 feh 2007-01-14 05:01:46 +00:00
joey d253cef179 fix broken format string, manual unfuzzy 2007-01-14 04:28:50 +00:00
joey 5608aa078e * In the aggregator, check for and deal with post filenames that are
longer than the maximum file length.
2007-01-14 04:17:53 +00:00
joey f94e1dc11b fix whitespace 2007-01-03 04:19:51 +00:00
joey 912521ef07 * Initial work on internationalization of the program code. po/ikiwiki.pot
is available for translation.
* Export gettext() from IkiWiki module.
2006-12-29 04:38:40 +00:00
joey 35ee7e44a6 * Make sure to check for errors from every eval. 2006-11-08 21:03:33 +00:00
joey cb3f89f429 * Enable utf8 file IO in aggregate plugin.
* Fix some issues with the new registration form.
2006-11-08 20:13:59 +00:00
joey cc018082d0 typo 2006-11-06 04:27:29 +00:00
joey b78e93098b * Fix issue with aggregate plugin updating expired pages. 2006-11-04 19:20:24 +00:00
joey 5a82103526 * Implemented expiry options for aggregate plugin.
* Use precalculated backlinks info when determining if files need an update
  due to a page they link to being added/removed. Mostly significant if
  there are lots of pages.
* Remove duplicate link info when saving index. In some cases it could
  pile up rather badly. (Probably not the best way to deal with this
  problem.)
2006-11-01 05:41:37 +00:00
joey 9256360490 * Make the aggregate plugin emphasize error messages. 2006-10-13 18:31:18 +00:00
joey dae0f48e91 * Work on firming up the plugin interface:
- Plugins should not need to load IkiWiki::Render to get commonly
    used functions, so moved some functions from there to IkiWiki.
  - Picked out the set of functions and variables that most plugins
    use, documented them, and made IkiWiki export them by default,
    like a proper perl module should.
  - Use the other functions at your own risk.
  - This is not quite complete, I still have to decide whether to
    export some other things.
* Changed all plugins included in ikiwiki to not use "IkiWiki::" when
  referring to stuff now exported by the IkiWiki module.
* Anyone with a third-party ikiwiki plugin is strongly enrouraged
  to make like changes to it and avoid use of non-exported symboles from
  "IkiWiki::".
* Link debian/changelog and debian/news to NEWS and CHANGELOG.
* Support hyperestradier version 1.4.2, which adds a new required phraseform
  setting.
2006-09-09 22:50:27 +00:00
joey 6e273d60a6 shorten further 2006-09-07 23:55:21 +00:00
joey 2fd2646e66 remove the "0 new", it's not very useful 2006-09-07 23:54:37 +00:00
joey bd4cd357a1 * Add processed date to aggregate preprocessor directive status output. 2006-09-02 02:07:44 +00:00
joey e5b323a633 bugnum 2006-08-28 04:40:00 +00:00