New ODF testing method uncovers flaws in office suites

Content

OpenDoc Society held its 12th ODF Plugtest on 16 and 17 November 2017, hosted in Paris by the French Ministry of Finance and POSS. Delegates came from Australia, Canada, France, Germany, Italy, Russia, UK, Ukraine, Taiwan, The Netherlands, Luxemburg, United States and Belgium. They represented application vendors, standards bodies, governments, the European Commission and the European Parliament.

The ODF Plugfest aims to improve interoperability among applications that use the Open Document Format (ODF), which is an open standard created by OASIS that is supported by most productivity tools in the market today;and is mandatory for use by governments throughout the world. It is also published as an international standard, as ISO/IEC 26300. This edition of the ODF Plugfest featured a new testing infrastructure for the creation and excution of ODF autotests, partly financed by the Dutch government. Prior to the plugfest, developers had already submitted a significant set of tests to the server. The ODF Plugfest focused on triaging these tests, running them, analysing their results, creating new tests, identifying additional features to be tested and improving the test methodology.

One of the tests revealed that a number of established office suites lose heading hierarchy when opening and saving a document in ODF format, due to flaws in the alignment of their own legacy formats and the ODF standard. Loss of hierarchical structure makes a document far less accessible to disabled users who rely upon assistive technologies such as reading software or braille outputs to navigate through documents. The uncovered flaw damages common functionality for all users, such as the generation of tables of content but also influences automated content classification and information retrieval.

"The flaws discovered are a testament to the importance and effectiveness of the new testing methodology we have adopted", says Michiel Leenaars of OpenDoc Society. "Losing document information which is so critical to people with disabilities should be considered a major shortcoming of any application. One key aspect of moving from vendor-specific legacy file formats to ODF as a global office standard is that we are able to uncover this kind of significant data loss. Now the three vendors involved can get to work and improve the implementation in their products."

The testers also found other issues, for example with the way that applications treat Chinese, Japanese, Korean and Vietnamese language with vertical writing, or how applications process colours in different and sometimes unexpected ways.

"These results show that even digitally born documents are not safe for future generations as long as we don't modernise the tools we work with", says Jos van den Oever, co-chair of the ODF Technical Committee and member of the UNESCO Persist Technical Taskforce. Van den Oever, an expert on document formats employed by the Netherlands government, is the architect behind the new testing framework. "The fact shows that we have to be careful which tools we adopt, if we care about our information in the long term. The fact that the visual appearance is consistent between the original documents and the damaged documents makes these particular flaws even more treacherous. Merely opening and saving in the wrong office application is enough to cause permanent damage to user documents, and the end user cannot actually see the difference."

The complete report and presentations of the 12th ODF Plugfest are available at http://plugfest.opendocumentformat.org/2016-paris/report.html (externe link)