How To Test Documentation

 

[Got a testing tip? Why not share it?]

This is a follow up to the previous article - Documentation and The Tester.

  • Proof Read for Spelling and Grammar - Some may sneer at the notion of a technical writer making simple errors. That’s silly. The hard part is organizing information and presenting it cogently. Give me that, and I am more than happy to look for typos.
  • Spelling & Grammar Trick –  If you are reviewing documentation in a tool or viewer or format which does not offer native spell and grammar checking, considering importing the text into a tool which does. MS Word, Open Office Writer, Google Docs, a browser add-on – even cat the text file into “spell” on Unix. Take advantage of the tools at hand. But you still have to read it and check the rest manually.
  • Check for Accuracy – errors of commission. Keep the Development Spec and your test notes handy.
  • Check for Completeness – errors of omission. Again, use the Dev Spec and test notes for references.
  • Consider Organization and Approach – compare with the Marketing Req. Spec.
  • Test all Examples! – Do you (or your company) want to look dumb? Include an example in the doc that doesn’t work. I speak from experience here. So don’t just read the examples. Perform the exact steps documented.
  • Look for changes in one area that might affect another - And make sure that screenshots are up-to-date.

Author Profile – Sean Morley

Sean Morley is a Principle Quality Engineer for Dassault Systemes. Having worked in a variety of roles on software tools for electronic design and test, he is currently focused on the testing of engineering software. Sean participates in test related forums and blogs at www.testyengineer.com in a concerted effort to improve his knowledge of the field and share his experiences. He appreciates all feedback, recommendations, and challenges as opportunities for growth, and can be contacted at seansync@gmail.com.

 

7 Responses to How To Test Documentation

  1. @halperinko - Kobi Halperin January 2, 2012 at 11:27 am #

    Similar method applies for quick testing of SW GUI Labels spelling:
    If the application is properly written, you most likely to find that majority of labels and messages are not hard-coded (any which does = Bug), but rather translated using Locale files (which allows for easy maintenance, as well as internationalization).
    Finding these files, running quick filtering of the relevant data (either just the values, or the label_name + value), and running in MS-Word speller, can find majority of bugs, even if you test Chinese (not as mother tongue).

    You can take it further, by keeping private dictionaries with relevant technical arena phrases,
    You can run with default dictionary to identify all items which might need to reside in abbreviations table.
    You can even run some simple scripts, to identify differences in length of specific words or labels in comparison to translation in other languages – to quickly spot labels which might be longer than the designated object size.

    BTW – Spelling and Grammar might still miss some items which does exist in the language, but are illogical in current context.

    Kobi

  2. David Ramsay - @daver22 January 2, 2012 at 1:56 pm #

    You also need to know your own limitations /subtleties here are 3 examples;

    F-Test – Simple
    “Finished files are the result of years of scientific study combined with the experience of years.”

    F-Test – Complex
    “In our workshops, participants are shown a 60-second video during which they are asked to count the number of times two soccer balls are passed between a group of people. At the end of the exercise, there is usually a variety of opinion about the number of times the ball was passed. We also ask if anyone saw anything unusual in the video. About 20% of viewers will admit to having seen a gorilla. Everyone else snickers and wags their heads in disbelief that their fellow participants could be so mistaken. As we watch the video a second time, everyone sees a person dressed in a gorilla costume slowly walk into the scene, stop, face the camera, and beat his chest before calmly walking out of the scene. Those who did not see the gorilla during the first viewing sit gap-jawed upon realizing that they missed it altogether. The purpose of the exercise is then explained: Humans are not very good at noticing things. We can be fooled our eyes can deceive us. Optical illusions work for a reason. Humans are not good at inspection. We often miss large things (like a gorilla) when we are focused on anything else.”

    Cognition and the Human Brain

    This is a fascinating example of the compute power of the human brain.

    Aoccdrnig to rscheearch at Cmabrigde Uinervtisy, it deosn’t mttaer in waht oredr the ltteers in a wrod are, the olny iprmoetnt tihng is taht the frist and lsat ltteer be at the rghit pclae. The rset can be a total mses and you can sitll raed it wouthit porbelm. Tihs is bcuseae the huamn mnid deos not raed ervey lteter by istlef, but the wrod as a wlohe.

  3. Jeff Lucas January 2, 2012 at 5:12 pm #

    I just ran across a related subject in the book “Tacit and Explicit Knowledge” by Harry Collins. In there, he suggests separating reviews of grammatical errors (which use an “inscripted” area of the brain) from reviews of the meaning (which use a “translational” area of the brain). I had heard that previously, but never understood the reason before. In a previous job, it was the product manager’s review where they simply read the documentation for flow and meaning (many times finding problems with the flow that were previously undiscovered).

    Nice Post!

  4. Sean Morley January 6, 2012 at 5:58 am #

    Kobi – Thank you, that really belongs on this list.

    David – Interesting points, and I love that last example. I laughed at the fact that I could actually read that gibberish.

    Jeff – Thanks for the explanation. I often find I need to review text twice (at least). The first time through I get hung up on spelling and grammar and can’t pay enough attention to meaning and flow. I thought that was just my own hangup. Now I can accept it as process!

    Sean

  5. David Greenlees January 10, 2012 at 10:36 pm #

    Nice Sean.
    I recently finished proofing a novel and Jeff’s point above rings true. I wrote about the experience here…

    http://davidstest.posterous.com/proofreading-a-form-of-testing

    I particularly like your point about testing all documented examples. Like you, I have seen this play out badly first hand… Not pretty having a government minister working through an example that really doesn’t (nor should it) work. Ouch!

    Cheers.

  6. Eugenia Yakhnin January 21, 2012 at 11:03 pm #

    Technical Writers must test Documentation.

  7. Sean Morley January 29, 2012 at 6:56 am #

    In reply to Eugenia – “Technical Writers must test Documentation.”

    That’s like saying Developers must test software.
    Sure, a first pass; you might call it Unit Test. But then it falls to testers/QE/QA. Similarly, the Technical Writer should check their work before handoff. But then it should be reviewed for technical accuracy and proofread by someone other than the author. I feel this is the tester’s responsibility, but the developer will often review it as well.

Leave a Reply