Managing Structured data in TWiki
I keep working with twiki. The ideas keep coming at fast pace.
These days I am working on making structured data manageable in twiki (and in general, wiki) framework. The Topic names provide a uniform resource locator (URL; yes, sometimes you need to spell it to highlight the significance!) which act as achors to specific sets of data you are interested in. This data could be a simple list, a set of rows of a table (for which, say, you are administratively responsible), and so on. Due to plug-in nature, the data could come from a Database, and yet be available in format we choose to keep them within the topic.
So first part is: How do you keep structured data in twiki, and yet make it editable? TWiki chooses to use simple lists with capitalized words in cases such as Access controls, groups etc. I am trying to use for the same purpose. The reasonings are:
So, I have chosen it for specifying any structured content in TWiki topic. (Typically wrapped with some markup so the plugins can identify that what follows is structured data.)
The next step is to be able to display this data nicely - in variety of ways. I now have a plugin to do exactly this. Currently it works with table data. My idea is to couple this with variety of CSS styles so user can choose how he would like to see the table.
How do we add the content to this table? Either handedit the topic. Or use forms interface. I have this done already. However, we have to think and see what are other options. For example, SVG is an emerging standard. XForms is also another standard.
And then we require integration with Databases. It is no good if my table is part of topic. Some people feel that databases are end-all of all structured data. Though I don't quite agree, we should anyway respect the fact that they are extremely well supported, and provide good integration with other tools in enterprise.
It is easy if my data has to reside only within a database. A plug-in can then act as intermediary. The topic only contains table name, database name and may be the set of columns. (Or may be the SQL query.)
But this is not acceptable. Why?
So, it becomes clear that twiki topics should have an internal cache, and mechanisms to update it. There is already a Cache plugiin. But this is not sufficient; we require good control over caching behaviour. I plan to do a plugin that allows you to do "Render and Cache" i.e. it has two arguments: the content to render (typically INCLUDE or SEARCH), and second argument is actual cache. It either shows only contents of cache, or it updates the cache with rendered twiki markup, and then shows the page.
More on this design later.