It seems that rebuild a wiki (`ikiwiki --rebuild`) after changing the `underlaydir` config option doesn't remove the pages coming from the previous underlaydir.
I've noticed this with the debian package version 3.20100102.3~bpo50+1.
Perhaps it is possible to improve this or mention it in the manual page?