Camen Design Forum
----
As mentioned in previous thread, I'm currently developing a wiki engine based around NNF philosophy. There is not much of the specific functionality built in at the moment, and I'd like to hear about what should and could be done regarding its structure and the way it'll work. So far I've developed the markup parser and the read / store class; along with that I've adapted a TOC builder and a diff method for use within the application.
1. How are you storing data / what format?
One of the principles of NNF was to not duplicate data. I could have opted for sqlite or even text files, but it seemed pointless to have one text file, one RSS file and one HTML file for the same data. If I could have had my way then I would have used XSLT on the RSS feed and done away with the PHP templating entirely. :P
2. What markup have you gone with?
If I made a Wiki I would probably use a subset of ReMarkable because not enough Wikis (read: none) have syntax for definition lists, one of my favourite HTML constructs. What syntax are you using for interlinking? This is the hard one to solve elegantly.
Regarding the format, I'm using the Atom format for storage, so that the commit also works as a subscription channel should anyone want to subscribe to the changes made to a particular article. A revision is marked with its ISO8601 timestamp, which is the format Atom uses. Additionally, instances of <category> tag within the feed represent the article categories, whereas instances of <author> within entries indicate the authors of separate revisions.
Regarding the markup, it doesn't do much currently. All it does is replace double new lines into paragraphs, and it parses * and _ around bits of text as <strong> and <em> tags. For the linking, I implemented the syntax used by other wiki engines, where [[page_name]] links to a page and [url] links to an external page.
Really nice with the storage format! That's how I would have done it. For a Wiki though the feed could soon get to big (several megs) if you had a large article with many revisions. Perhaps you could include a config option to remove old revisions from the feed once the feed reaches a certain file size.
BTW, why would an article need categories? Is there something different that you're doing about the hierarchy of the Wiki? (I used categories to mark if the thread is locked and if posts are deleted because there was no other way to store this metadata without needlessly extending the RSS spec)
Discussion on Wikis simply suck. At least not to begin with, but something to consider is attaching a NNF-style comment thread to articles (or a talk page).
Just a small feature request, the ability to use the same trip codes from an NNF install.
To save space I wonder if Arom supports something like just logging changes? Somewhere along the line of how SFW[1] seems to log changes per paragraph and changes in paragraph order instead of saving the complete contents of the page.
Martijn added on
Just one quick idea, as it is more common to see the latest revision and rarely does someone see previous revisions (and mostly for moderation/administration), I think storing the most recent version and logging the changes in relation to next younger versoin would save the most space, although it'd require more and more processing the older the revision was.
@kroc
“For a Wiki though the feed could soon get to big (several megs) if you had a large article with many revisions.”
I have an article that is 7 paragraphs long which I use to test the markup parser and the table of contents thing. So far it has ~20 revisions and hasn't crossed 100kb. I guess that is a really long article, but still, for even longer ones, there can be a significant increase in size with every revision. (especially if editors do dumb things like "typo fix" and all they change is an apostrophe)
I was actually thinking about a "purge" option, where the user would be presented with following:
1) ability to delete revisions before a specific date (on article base)
2) ability to delete all revisions besides the most recent one (for whole wiki, and for specific articles)
“BTW, why would an article need categories? Is there something different that you're doing about the hierarchy of the Wiki?”
I thought about having categories for grouping related articles (e.g. all articles about physics go under "physics" category) but I rethought it and decided not to include categories after all. Automatic rebuilding would require the application to re-read *all* articles on every load. Solving it with folders would limit the possibility of one article belonging into multiple categories without duplicating content. Therefore, I think it would be the best if categories are used for protecting pages from edits rather than categorizing articles.
“Discussion on Wikis simply suck. At least not to begin with, but something to consider is attaching a NNF-style comment thread to articles (or a talk page).”
I have actually considered to include this feature in the second version of the application. The plan for now is to get a plain and simple wiki engine with only bare functionality, and to build upon that for future releases.
----
@simon
“Just a small feature request, the ability to use the same trip codes from an NNF install.”
If you are referring to the authentication system -- that is the idea. :)
----
@martijn and jose
That isn't a bad idea, but I am not sure how it would work out. Seems to me that it'd require much work on the script end so that it could track changes and build the article through time. I think the purging system can do much more good here.
“Automatic rebuilding would require the application to re-read *all* articles on every load. Solving it with folders would limit the possibility of one article belonging into multiple categories without duplicating content.”
Here’s a wild idea; why not use a sitemap to index the files in a category?
A sitemap lists a load of related feeds (articles in this case). NoNonsense Forum uses a sitemap to group together all the sub-folders so that Google indexes them all.
https://github.com/Kroc/NoNonsenseForum/blob/master/lib/functions.php#L289
Using a sitemap to manage the list of articles that go into a category would also provide you a per-category file that doubles up as perfect SEO! When a category is created, you create a site map file, and then you have a index of what files are in the category without having to parse every file in the wiki!
---
“If you are referring to the authentication system -- that is the idea. :)”
What simon means is that if an NNF forum and your Wiki could share the same name and passwords and vice-versa; that is, use the same `users` folder :)
“Here’s a wild idea; why not use a sitemap to index the files in a category?”
Never thought about that, great idea! I'll try to rework the current code to generate sitemaps with categories. The only problem then is URL scheme for categories -- how to differentiate between articles and categories?
“What simon means is that if an NNF forum and your Wiki could share the same name and passwords and vice-versa; that is, use the same
`users`folder :)”
If it has the same authentication scheme then it should work. ;)
Nikolai added on
^ but then there should be an option to specify where the `users` folder is located.
“how to differentiate between articles and categories?”
Sitemaps should be .xml files and articles .rss files. Categories therefore could also have an article page, with a list of participating articles appended to the end. That way, people can add a description to the top of category lists.
@Kroc
I meant within the URL scheme; for example, if both articles and categories share the "/cleaned_up_name" scheme there will be a clash if an article is named like a category as the system won't know if "/thermodynamics" is an article about thermodynamics or a category that groups the pages within the area.
I was thinking about using the namespaced scheme that MediaWiki uses, e.g. "/thermodynamics" for articles, "/category:thermodynamics" for categories, "/discuss:thermodynamics" for article discussion pages and so forth.
Ask yourself this: why would you want an article and a category with different names? Don't answer that :) Such a thing is a pointless complexity and not fit for a simple wiki!
If a user goes to `/thermodynamics` then it shows the thermodynamics article, if this is also a category then the list of articles is automatically shown at the bottom of the article. Why do this? So that categories can have a description and introuction text, allowing you to easily organise things.
For example, you could have the chapters of a story as articles, and use the category to act as a table of contents; the category article would then allow you to provide the book's name and information and thus be able to house everything neatly within one category and one URL.
I'm sorry for the lack of updates about this project. Other things, including the end-of-first-semester exams and the recent hard drive failure (due to which all progress on this and other projects as well was gone) have prevented me from going further about the wiki. Hopefully by the end of the week I'll manage to get back on the track with the development.
Sorry to hear that Nikolai :( Loosing code really sucks, but I always find you can come back stronger as you will have managed to sort out a lot of overall design issues which you won't need to tackle now and can just plow right in.
Are you developing this product for commercial or private use? If not, have you considered using a service like GitHub? That way your code (and it's entire history) is backed up online and makes it available for others to contribute to.
I imagined it to be open and free, so yes, it will find its way to GitHub; the reason I didn't put it up earlier is because it was ugly for the most part and I didn't want to have it online until it was at least decent.
On the topic, the last two days have been really productive and I managed to restore most of what was lost.
“the reason I didn't put it up earlier is because it was ugly”
An important lesson to learn is that all code is ugly to begin with, and it's better to "release early, release often" than it is to lock it away forever polishing it.
NNF only really exists because I set myself the explicit goal to produce something as quick and dirty as possible (about three days and 700 lines).
Glad to hear though that you've made good progress and look forward to examining the code design.
Your friendly neighbourhood moderators: Kroc, Impressed, theraje, Martijn