Translation management

With the holidays and such, I hadn’t found time yet to blog about the new translation management features colleague Karel and myself have been adding to Daisy, so let’s have a look at this now.

There are lots of aspects to managing multi-lingual content, here are some of them, not everything mentioned already exists in Daisy:

• repository and document structure related: storing multiple language variants of a document (supported in Daisy since version 1.3), content which is shared by all language variants such as non-translatable fields (not yet in Daisy), language-specific fulltext-indexing behavior (not yet in Daisy)
• management related: keeping track of the translation status, being able to query outstanding translation work (this is what we added now).
• content related: markup for indicating non-translatable content or other hints towards translators (not specifically present in Daisy, though could be based on HTML element classes). See also ITS.
• editing related: updating translation status (this is supported in Daisy, remainder not yet), split-screen editing, language-aware spell-checking, integration with terminology systems and translation memories, segment-based editing, …
• publishing related: deciding what language to show
• workflow related: external: export content and provide it to a translation agency (this we added now), internal: use of the Daisy-integrated jBPM workflow.

The main areas we have currently worked on are keeping track of the translation status, and extending the export/import tool with a format suitable for exchange with translation agencies (also known as localization providers).

Document model changes for translation management

A document in Daisy can exist in a number of languge variants, we call the reference variant the variant in which the original content is written, and the translated variants are translations of the reference variant.

To keep track of which version of a translated variant corresponds to which version of the reference variant, we added a ’synced with’ property to versions. The ’synced with’ property is a link to a version of another language variant. If new versions have been added to the reference variant, than the translated variants based on that reference variant will likely also require updating.

However, not all changes to the reference variant require updating the translated variants, for example this is the case when you only fix a small typo in the reference variant. For this purpose, we’ve added a property ‘change type’ to versions, which can have the value major or minor. Major changes invalidate the translations, while minor changes do not. This major/minor distinction is, for translation management, only important on the reference variant. On other variants, or when you’re not making use of translation management, you can use this property as you desire.

To indicate what the reference variant is for a document, a property ‘reference variant’ has been added to documents. Setting the reference variant for a document is also an indication that the document is considered to be under translation management.

While adding the ’synced with’ and ‘change type’ properties to versions, we’ve also added a ‘change comment’ property, which can contain a short description of the changes in that version. This is somewhat similar to the commit message in SVN/CVS.

All these version properties are still editable after version creation, in contrast with the actual content of a version, which cannot be modified after version creation.

Here is how the new properties are presented in the editor. The ’synced with’ is only shown when relevant: when the reference language is set, and when the current variant is not the reference language.

While at it, we also improved the top part of the document editor, to take less space, and show the document ID, type, branch and language:

Querying for translation status

With the new document model properties, especially the ’synced with’ field, we can now do various translation status searches. There are some new query language constructs for this, and to make it easy for users, we’ve made the commonly needed queries available via a new page accessible through Tools -> Translation Management.

The most useful is the overview query, which shows the translation status for all variants of a set of documents, as shown in the screenshot below.

In case you’re wondering: green means the translation is up to date, white means the variant does not exist, “not in sync” means the ’synced with’ link does not point to the last version with major changes of the reference language, “not synced” means the ’synced with’ link is not set at all, “not synced with ref variant” means the synced-with link points to a variant which is not the reference variant.

Translation import/export

A common need is to provide content stored in Daisy to localization providers. These people usually won’t edit content in Daisy, but have their own tool chain in place with content segmentation, translation memories, etc. So we only need a way to provide the content to them and load the returned result into Daisy. We already have an import/export tool to exchange content between repositories, but its format is not very well suited for this purpose: the content of one document is spread over multiple files (one for the meta data, one for each part), there is one directory per document rather than one file per document, … Also, for translation purposes, the content exported from one language variant needs to be imported into another language variant after translation, rather than the same language variant.

Therefore, we’ve extended the current import/export tool with the ability to import/export a new format which is better suited. All content of a document is embedded into one XML file (of course, this only includes text-based formats). There’s a lot more to say about this, if interested just go read the documentation.

Other changes

While at it, we’ve introduced some other improvements:

• the daisy-wiki-add-site tool now has the ability to do a multi-language setup: it will automatically set up multiple sites for each desired language, with the navigation documents and home page documents being variants of the same documents.
• the version list screen now allows editing the new version properties, as well as allows easy diffing between arbitrary versions.
• the document-info popup has been redesigned to show more information, and is quicker accessible through an icon in the menu.

We’re thinking of a 2.2 Milestone release in the next weeks, so that those who don’t dare to build Daisy from source will be able to try out this new stuff.

More information on the multi-language features can be found in the documentation.