541 lines
19 KiB
Markdown
541 lines
19 KiB
Markdown
This is an idea from [[JoshTriplett]]. --[[Joey]]
|
|
|
|
Some uses of ikiwiki, such as for a bug-tracking system (BTS), move a bit away from the wiki end
|
|
of the spectrum, and toward storing structured data about a page or instead
|
|
of a page.
|
|
|
|
For example, in a bug report you might want to choose a severity from a
|
|
list, enter a version number, and have a bug submitter or owner recorded,
|
|
etc. When editing online, it would be nice if these were separate fields on
|
|
the form, rather than the data being edited in the big edit form.
|
|
|
|
There's a tension here between remaining a wiki with human-editable source
|
|
files, containing freeform markup, and more structured data storage. I
|
|
think that it would be best to include the structured data in the page,
|
|
using a directive. Something like:
|
|
|
|
part of page content
|
|
\[[data yaml="<arbitrary yaml here>"]]
|
|
rest of page content
|
|
|
|
As long as the position of the directive is not significant, it could be
|
|
stripped out when web editing, the yaml used to generate/populate form fields,
|
|
and then on save, the directive regenerated and inserted at top/bottom of
|
|
the page.
|
|
|
|
Josh thinks that yaml is probably a good choice, but the source could be a
|
|
`.yaml` file that contains no directives, and just yaml. An addition
|
|
complication in this scenario is, if the yaml included wiki page formatted content,
|
|
ikiwiki would have to guess or be told what markup language it used.
|
|
|
|
Either way, the yaml on the page would encode fields and their current content.
|
|
Information about data types would be encoded elsewhere, probably on a
|
|
parent page (using a separate directive). That way, all child pages could
|
|
be forced to have the same fields.
|
|
|
|
There would be some simple types like select, boolean, multiselect, string, wiki markup.
|
|
Probably lists of these (ie, list of strings). Possibly more complex data
|
|
structures.
|
|
|
|
It should also be possible for plugins to define new types, and the type
|
|
definitions should include validation of entered data, and how to prompt
|
|
the user for the data.
|
|
|
|
This seems conceptually straightforward, if possibly quite internally
|
|
complex to handle the more complicated types and validation.
|
|
|
|
One implementation wrinkle is how to build the html form. The editpage.tmpl
|
|
currently overrides the standard [[!cpan CGI::FormBuilder]] generated form,
|
|
which was done to make the edit page be laid out in a nice way. This,
|
|
however, means that new fields cannot be easily added to it using
|
|
[[!cpan CGI::FormBuilder]]. The attachment plugin uses the hack of bouilding
|
|
up html by hand and dumping it into the form via a template variable.
|
|
|
|
It would be nice if the type implementation code could just use
|
|
FormBuilder, since its automatic form generation, and nice field validation
|
|
model is a perfect match for structured data. But this problem with
|
|
editpage.tmpl would have to be sorted out to allow that.
|
|
|
|
Additional tie-ins:
|
|
|
|
* Pagespecs that can select pages with a field with a given value, etc.
|
|
This should use a pagespec function like field(fieldname, value). The
|
|
semantics of this will depend on the type of the field; text fields will
|
|
match value against the text, and link fields will check for a link
|
|
matching the pagespec value.
|
|
* The search plugin could allow searching for specific fields with specific
|
|
content. (xapian term search is a good fit).
|
|
|
|
See also:
|
|
|
|
[[tracking_bugs_with_dependencies]]
|
|
|
|
> I was also thinking about this for bug tracking. I'm not sure what
|
|
> sort of structured data is wanted in a page, so I decided to brainstorm
|
|
> use cases:
|
|
>
|
|
> * You just want the page to be pretty.
|
|
> * You want to access the data from another page. This would be almost like
|
|
> like a database lookup, or the OpenOffice Calc [VLookup](http://wiki.services.openoffice.org/wiki/Documentation/How_Tos/Calc:_VLOOKUP_function) function.
|
|
> * You want to make a pagespec depend upon the data. This could be used
|
|
> for dependancy tracking - you could match against pages listed as dependencies,
|
|
> rather than all pages linked from a given page.
|
|
>
|
|
>The first use case is handled by having a template in the page creation. You could
|
|
>have some type of form to edit the data, but that's just sugar on top of the template.
|
|
>If you were going to have a web form to edit the data, I can imagine a few ways to do it:
|
|
>
|
|
> * Have a special page type which gets compiled into the form. The page type would
|
|
> need to define the form as well as hold the stored data.
|
|
> * Have special directives that allow you to insert form elements into a normal page.
|
|
>
|
|
>I'm happy with template based page creation as a first pass...
|
|
>
|
|
>The second use case could be handled by a regular expression directive. eg:
|
|
>
|
|
> \[[regex spec="myBug" regex="Depends: ([^\s]+)"]]
|
|
>
|
|
> The directive would be replaced with the match from the regex on the 'myBug' page... or something.
|
|
>
|
|
>The third use case requires a pagespec function. One that matched a regex in the page might work.
|
|
>Otherwise, another option would be to annotate links with a type, and then check the type of links in
|
|
>a pagespec. e.g. you could have `depends` links and normal links.
|
|
>
|
|
>Anyway, I just wanted to list the thoughts. In none of these use cases is straight yaml or json the
|
|
>obvious answer. -- [[Will]]
|
|
|
|
>> Okie. I've had a play with this. A 'form' plugin is included inline below, but it is only a rough first pass to
|
|
>> get a feel for the design space.
|
|
>>
|
|
>> The current design defines a new type of page - a 'form'. The type of page holds YAML data
|
|
>> defining a FormBuilder form. For example, if we add a file to the wiki source `test.form`:
|
|
|
|
---
|
|
fields:
|
|
age:
|
|
comment: This is a test
|
|
validate: INT
|
|
value: 15
|
|
|
|
>> The YAML content is a series of nested hashes. The outer hash is currently checked for two keys:
|
|
>> 'template', which specifies a parameter to pass to the FromBuilder as the template for the
|
|
>> form, and 'fields', which specifies the data for the fields on the form.
|
|
>> each 'field' is itself a hash. The keys and values are arguments to the formbuilder form method.
|
|
>> The most important one is 'value', which specifies the value of that field.
|
|
>>
|
|
>> Using this, the plugin below can output a form when asked to generate HTML. The Formbuilder
|
|
>> arguments are sanitized (need a thorough security audit here - I'm sure I've missed a bunch of
|
|
>> holes). The form is generated with default values as supplied in the YAML data. It also has an
|
|
>> 'Update Form' button at the bottom.
|
|
>>
|
|
>> The 'Update Form' button in the generated HTML submits changed values back to IkiWiki. The
|
|
>> plugin captures these new values, updates the YAML and writes it out again. The form is
|
|
>> validated when edited using this method. This method can only edit the values in the form.
|
|
>> You cannot add new fields this way.
|
|
>>
|
|
>> It is still possible to edit the YAML directly using the 'edit' button. This allows adding new fields
|
|
>> to the form, or adding other formbuilder data to change how the form is displayed.
|
|
>>
|
|
>> One final part of the plugin is a new pagespec function. `form_eq()` is a pagespec function that
|
|
>> takes two arguments (separated by a ','). The first argument is a field name, the second argument
|
|
>> a value for that field. The function matches forms (and not other page types) where the named
|
|
>> field exists and holds the value given in the second argument. For example:
|
|
|
|
\[[!inline pages="form_eq(age,15)" archive="yes"]]
|
|
|
|
>> will include a link to the page generated above.
|
|
|
|
>>> Okie, I've just made another plugin to try and do things in a different way.
|
|
>>> This approach adds a 'data' directive. There are two arguments, `key` and `value`.
|
|
>>> The directive is replaced by the value. There is also a match function, which is similar
|
|
>>> to the one above. It also takes two arguments, a key and a value. It returns true if the
|
|
>>> page has that key/value pair in a data directive. e.g.:
|
|
|
|
\[[!data key="age" value="15"]]
|
|
|
|
>>> then, in another page:
|
|
|
|
\[[!inline pages="data_eq(age,15)" archive="yes"]]
|
|
|
|
>>> I expect that we could have more match functions for each type of structured data,
|
|
>>> I just wanted to implement a rough prototype to get a feel for how it behaves. -- [[Will]]
|
|
|
|
>> Anyway, here are the plugins. As noted above these are only preliminary, exploratory, attempts. -- [[Will]]
|
|
|
|
>>>> I've just updated the second of the two patches below. The two patches are not mutually
|
|
>>>> exclusive, but I'm leaning towards the second as more useful (for the things I'm doing). -- [[Will]]
|
|
|
|
#!/usr/bin/perl
|
|
# Interpret YAML data to make a web form
|
|
package IkiWiki::Plugin::form;
|
|
|
|
use warnings;
|
|
use strict;
|
|
use CGI::FormBuilder;
|
|
use IkiWiki 2.00;
|
|
|
|
sub import { #{{{
|
|
hook(type => "getsetup", id => "form", call => \&getsetup);
|
|
hook(type => "htmlize", id => "form", call => \&htmlize);
|
|
hook(type => "sessioncgi", id => "form", call => \&cgi_submit);
|
|
} # }}}
|
|
|
|
sub getsetup () { #{{{
|
|
return
|
|
plugin => {
|
|
safe => 1,
|
|
rebuild => 1, # format plugin
|
|
},
|
|
} #}}}
|
|
|
|
sub makeFormFromYAML ($$$) { #{{{
|
|
my $page = shift;
|
|
my $YAMLString = shift;
|
|
my $q = shift;
|
|
|
|
eval q{use YAML};
|
|
error($@) if $@;
|
|
eval q{use CGI::FormBuilder};
|
|
error($@) if $@;
|
|
|
|
my ($dataHashRef) = YAML::Load($YAMLString);
|
|
|
|
my @fields = keys %{ $dataHashRef->{fields} };
|
|
|
|
unshift(@fields, 'do');
|
|
unshift(@fields, 'page');
|
|
unshift(@fields, 'rcsinfo');
|
|
|
|
# print STDERR "Fields: @fields\n";
|
|
|
|
my $submittedPage;
|
|
|
|
$submittedPage = $q->param('page') if defined $q;
|
|
|
|
if (defined $q && defined $submittedPage && ! ($submittedPage eq $page)) {
|
|
error("Submitted page doensn't match current page: $page, $submittedPage");
|
|
}
|
|
|
|
error("Page not backed by file") unless defined $pagesources{$page};
|
|
my $file = $pagesources{$page};
|
|
|
|
my $template;
|
|
|
|
if (defined $dataHashRef->{template}) {
|
|
$template = $dataHashRef->{template};
|
|
} else {
|
|
$template = "form.tmpl";
|
|
}
|
|
|
|
my $form = CGI::FormBuilder->new(
|
|
fields => \@fields,
|
|
charset => "utf-8",
|
|
method => 'POST',
|
|
required => [qw{page}],
|
|
params => $q,
|
|
action => $config{cgiurl},
|
|
template => scalar IkiWiki::template_params($template),
|
|
wikiname => $config{wikiname},
|
|
header => 0,
|
|
javascript => 0,
|
|
keepextras => 0,
|
|
title => $page,
|
|
);
|
|
|
|
$form->field(name => 'do', value => 'Update Form', required => 1, force => 1, type => 'hidden');
|
|
$form->field(name => 'page', value => $page, required => 1, force => 1, type => 'hidden');
|
|
$form->field(name => 'rcsinfo', value => IkiWiki::rcs_prepedit($file), required => 1, force => 0, type => 'hidden');
|
|
|
|
my %validkey;
|
|
foreach my $x (qw{label type multiple value fieldset growable message other required validate cleanopts columns comment disabled linebreaks class}) {
|
|
$validkey{$x} = 1;
|
|
}
|
|
|
|
while ( my ($name, $data) = each(%{ $dataHashRef->{fields} }) ) {
|
|
next if $name eq 'page';
|
|
next if $name eq 'rcsinfo';
|
|
|
|
while ( my ($key, $value) = each(%{ $data }) ) {
|
|
next unless $validkey{$key};
|
|
next if $key eq 'validate' && !($value =~ /^[\w\s]+$/);
|
|
|
|
# print STDERR "Adding to field $name: $key => $value\n";
|
|
$form->field(name => $name, $key => $value);
|
|
}
|
|
}
|
|
|
|
# IkiWiki::decode_form_utf8($form);
|
|
|
|
return $form;
|
|
} #}}}
|
|
|
|
sub htmlize (@) { #{{{
|
|
my %params=@_;
|
|
my $content = $params{content};
|
|
my $page = $params{page};
|
|
|
|
my $form = makeFormFromYAML($page, $content, undef);
|
|
|
|
return $form->render(submit => 'Update Form');
|
|
} # }}}
|
|
|
|
sub cgi_submit ($$) { #{{{
|
|
my $q=shift;
|
|
my $session=shift;
|
|
|
|
my $do=$q->param('do');
|
|
return unless $do eq 'Update Form';
|
|
IkiWiki::decode_cgi_utf8($q);
|
|
|
|
eval q{use YAML};
|
|
error($@) if $@;
|
|
eval q{use CGI::FormBuilder};
|
|
error($@) if $@;
|
|
|
|
my $page = $q->param('page');
|
|
|
|
return unless exists $pagesources{$page};
|
|
|
|
return unless $pagesources{$page} =~ m/\.form$/ ;
|
|
|
|
return unless IkiWiki::check_canedit($page, $q, $session);
|
|
|
|
my $file = $pagesources{$page};
|
|
my $YAMLString = readfile(IkiWiki::srcfile($file));
|
|
my $form = makeFormFromYAML($page, $YAMLString, $q);
|
|
|
|
my ($dataHashRef) = YAML::Load($YAMLString);
|
|
|
|
if ($form->submitted eq 'Update Form' && $form->validate) {
|
|
|
|
#first update our data structure
|
|
|
|
while ( my ($name, $data) = each(%{ $dataHashRef->{fields} }) ) {
|
|
next if $name eq 'page';
|
|
next if $name eq 'rcs-data';
|
|
|
|
if (defined $q->param($name)) {
|
|
$data->{value} = $q->param($name);
|
|
}
|
|
}
|
|
|
|
# now write / commit the data
|
|
|
|
writefile($file, $config{srcdir}, YAML::Dump($dataHashRef));
|
|
|
|
my $message = "Web form submission";
|
|
|
|
IkiWiki::disable_commit_hook();
|
|
my $conflict=IkiWiki::rcs_commit($file, $message,
|
|
$form->field("rcsinfo"),
|
|
$session->param("name"), $ENV{REMOTE_ADDR});
|
|
IkiWiki::enable_commit_hook();
|
|
IkiWiki::rcs_update();
|
|
|
|
require IkiWiki::Render;
|
|
IkiWiki::refresh();
|
|
|
|
IkiWiki::redirect($q, "$config{url}/".htmlpage($page)."?updated");
|
|
|
|
} else {
|
|
error("Invalid data!");
|
|
}
|
|
|
|
exit;
|
|
} #}}}
|
|
|
|
package IkiWiki::PageSpec;
|
|
|
|
sub match_form_eq ($$;@) { #{{{
|
|
my $page=shift;
|
|
my $argSet=shift;
|
|
my @args=split(/,/, $argSet);
|
|
my $field=shift @args;
|
|
my $value=shift @args;
|
|
|
|
my $file = $IkiWiki::pagesources{$page};
|
|
|
|
if ($file !~ m/\.form$/) {
|
|
return IkiWiki::FailReason->new("page is not a form");
|
|
}
|
|
|
|
my $YAMLString = IkiWiki::readfile(IkiWiki::srcfile($file));
|
|
|
|
eval q{use YAML};
|
|
error($@) if $@;
|
|
|
|
my ($dataHashRef) = YAML::Load($YAMLString);
|
|
|
|
if (! defined $dataHashRef->{fields}->{$field}) {
|
|
return IkiWiki::FailReason->new("field '$field' not defined in page");
|
|
}
|
|
|
|
my $formVal = $dataHashRef->{fields}->{$field}->{value};
|
|
|
|
if ($formVal eq $value) {
|
|
return IkiWiki::SuccessReason->new("field value matches");
|
|
} else {
|
|
return IkiWiki::FailReason->new("field value does not match");
|
|
}
|
|
} #}}}
|
|
|
|
1
|
|
|
|
----
|
|
|
|
#!/usr/bin/perl
|
|
# Allow data embedded in a page to be checked for
|
|
package IkiWiki::Plugin::data;
|
|
|
|
use warnings;
|
|
use strict;
|
|
use IkiWiki 2.00;
|
|
|
|
my $inTable = 0;
|
|
|
|
sub import { #{{{
|
|
hook(type => "getsetup", id => "data", call => \&getsetup);
|
|
hook(type => "needsbuild", id => "data", call => \&needsbuild);
|
|
hook(type => "preprocess", id => "data", call => \&preprocess, scan => 1);
|
|
hook(type => "preprocess", id => "datatable", call => \&preprocess_table, scan => 1); # does this need scan?
|
|
} # }}}
|
|
|
|
sub getsetup () { #{{{
|
|
return
|
|
plugin => {
|
|
safe => 1,
|
|
rebuild => 1, # format plugin
|
|
},
|
|
} #}}}
|
|
|
|
sub needsbuild (@) { #{{{
|
|
my $needsbuild=shift;
|
|
foreach my $page (keys %pagestate) {
|
|
if (exists $pagestate{$page}{data}) {
|
|
if (exists $pagesources{$page} &&
|
|
grep { $_ eq $pagesources{$page} } @$needsbuild) {
|
|
# remove state, it will be re-added
|
|
# if the preprocessor directive is still
|
|
# there during the rebuild
|
|
delete $pagestate{$page}{data};
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
sub preprocess (@) { #{{{
|
|
my @argslist = @_;
|
|
my %params=@argslist;
|
|
|
|
my $html = '';
|
|
my $class = defined $params{class}
|
|
? 'class="'.$params{class}.'"'
|
|
: '';
|
|
|
|
if ($inTable) {
|
|
$html = "<th $class >$params{key}:</th><td $class >";
|
|
} else {
|
|
$html = "<span $class >$params{key}:";
|
|
}
|
|
|
|
while (scalar(@argslist) > 1) {
|
|
my $type = shift @argslist;
|
|
my $data = shift @argslist;
|
|
if ($type eq 'link') {
|
|
# store links raw
|
|
$pagestate{$params{page}}{data}{$params{key}}{link}{$data} = 1;
|
|
my $link=IkiWiki::linkpage($data);
|
|
add_depends($params{page}, $link);
|
|
$html .= ' ' . htmllink($params{page}, $params{destpage}, $link);
|
|
} elsif ($type eq 'data') {
|
|
$data = IkiWiki::preprocess($params{page}, $params{destpage},
|
|
IkiWiki::filter($params{page}, $params{destpage}, $data));
|
|
$html .= ' ' . $data;
|
|
# store data after processing - allows pagecounts to be stored, etc.
|
|
$pagestate{$params{page}}{data}{$params{key}}{data}{$data} = 1;
|
|
}
|
|
}
|
|
|
|
if ($inTable) {
|
|
$html .= "</td>";
|
|
} else {
|
|
$html .= "</span>";
|
|
}
|
|
|
|
return $html;
|
|
} # }}}
|
|
|
|
sub preprocess_table (@) { #{{{
|
|
my %params=@_;
|
|
|
|
my @lines;
|
|
push @lines, defined $params{class}
|
|
? "<table class=\"".$params{class}.'">'
|
|
: '<table>';
|
|
|
|
$inTable = 1;
|
|
|
|
foreach my $line (split(/\n/, $params{datalist})) {
|
|
push @lines, "<tr>" . IkiWiki::preprocess($params{page}, $params{destpage},
|
|
IkiWiki::filter($params{page}, $params{destpage}, $line)) . "</tr>";
|
|
}
|
|
|
|
$inTable = 0;
|
|
|
|
push @lines, '</table>';
|
|
|
|
return join("\n", @lines);
|
|
} #}}}
|
|
|
|
package IkiWiki::PageSpec;
|
|
|
|
sub match_data_eq ($$;@) { #{{{
|
|
my $page=shift;
|
|
my $argSet=shift;
|
|
my @args=split(/,/, $argSet);
|
|
my $key=shift @args;
|
|
my $value=shift @args;
|
|
|
|
if (! exists $IkiWiki::pagestate{$page}{data}) {
|
|
return IkiWiki::FailReason->new("page does not contain any data directives");
|
|
}
|
|
|
|
if (! exists $IkiWiki::pagestate{$page}{data}{$key}) {
|
|
return IkiWiki::FailReason->new("page does not contain data key '$key'");
|
|
}
|
|
|
|
if ($IkiWiki::pagestate{$page}{data}{$key}{data}{$value}) {
|
|
return IkiWiki::SuccessReason->new("value matches");
|
|
} else {
|
|
return IkiWiki::FailReason->new("value does not match");
|
|
}
|
|
} #}}}
|
|
|
|
sub match_data_link ($$;@) { #{{{
|
|
my $page=shift;
|
|
my $argSet=shift;
|
|
my @params=@_;
|
|
my @args=split(/,/, $argSet);
|
|
my $key=shift @args;
|
|
my $value=shift @args;
|
|
|
|
if (! exists $IkiWiki::pagestate{$page}{data}) {
|
|
return IkiWiki::FailReason->new("page $page does not contain any data directives and so cannot match a link");
|
|
}
|
|
|
|
if (! exists $IkiWiki::pagestate{$page}{data}{$key}) {
|
|
return IkiWiki::FailReason->new("page $page does not contain data key '$key'");
|
|
}
|
|
|
|
foreach my $link (keys %{ $IkiWiki::pagestate{$page}{data}{$key}{link} }) {
|
|
# print STDERR "Checking if $link matches glob $value\n";
|
|
if (match_glob($link, $value, @params)) {
|
|
return IkiWiki::SuccessReason->new("Data link on page $page with key $key matches glob $value: $link");
|
|
}
|
|
}
|
|
|
|
return IkiWiki::FailReason->new("No data link on page $page with key $key matches glob $value");
|
|
} #}}}
|
|
|
|
1
|