NUT Quality Assurance
=====================

Recognizing the critical nature of NUT, the NUT Quality Assurance (NQA) effort
has been established to improve NUT where necessary, and to maintain software
quality as high as it should be.

NQA is present in many aspects and areas of NUT.

Documentation
-------------

The documentation toolchain uses
link:https://asciidoc.org/[AsciiDoc] to output both HTML pages and
manual pages (troff). This single point of control fills many gaps,
suppresses many redundancies, and optimizes documentation management in
general.

- The NUT website and HTML documentation are tested for W3C XHTML 1.1 and CSS
compliance. This can be counter verified by clicking the W3C XHTML 1.1 and CSS
icons, at the bottom of each page.

////////////////////////////////////////////////////////////////////////////////
- the manual pages conformance is tested with
link:http://catb.org/~esr/doclifter/index.html[doclifter] (outdated)
////////////////////////////////////////////////////////////////////////////////

- Documentation source files are spell checked, using
link:http://aspell.net[Aspell], both interactively (using
'make spellcheck-interactive') and automatically in Buildbot (using
'make spellcheck').

NOTE: A NUT dictionary is also available (docs/nut.dict), providing a
glossary of terms related to power devices and management, as well as
partial terms, technical jargon and author names.

Source code
-----------

Use of standards
~~~~~~~~~~~~~~~~

NUT promotes and uses many standards, such as:

- the variable names standard used in NUT,
- the coding rules and best practices for developers,
- the use of a software architecture limiting developments to the very minimum,
- the use of standard Free and OpenSource Software components, like:
  * the USB library,
  * the Net SNMP project,
  * the Neon library,
  * the OpenSSL library (to be replaced by NSS, which is more license compliant
    with NUT and can be FIPS 140 certified),
  * the TCP Wrappers library.

QA tools and metrics
~~~~~~~~~~~~~~~~~~~~

NUT's quality is constantly monitored using many tools, like:

- a Revision Control System (link:https://github.com/networkupstools/nut[Git]) to
track development and ease regression fixes.

////////////////////////////////////////////////////////////////////////////////
Any modification on the NUT source
trees are reported on the link:http://lists.alioth.debian.org/mailman/listinfo/nut-commits[NUT Commits]
mailing list.
////////////////////////////////////////////////////////////////////////////////

- link:http://buildbot.networkupstools.org/public/nut/[Buildbot]
  for older take on multi-platform builds, and the

- new dedicated Jenkins incarnation of the NUT CI Farm with "legacy UI"
  for link:https://ci.networkupstools.org/job/nut/job/nut/job/master/[main branch]
  and link:https://ci.networkupstools.org/job/nut/job/nut/view/change-requests/[PRs],
  also accessible at the slower but slicker-looking Blue Ocean user interface for
  link:https://ci.networkupstools.org/blue/organizations/jenkins/nut%2Fnut/activity/[activity],
  link:https://ci.networkupstools.org/blue/organizations/jenkins/nut%2Fnut/branches/[branches]
  and link:https://ci.networkupstools.org/blue/organizations/jenkins/nut%2Fnut/pr/[PRs],
  are all used to automate the compile/test cycle, using numerous platforms,
  target distributions, C/C++ revisions, compiler toolkits and make program
  implementations. Any build failure is caught early, and fixed quickly.
  Also we get to see if incoming pull requests (as well as Git branch HEADs)
  do not have code (or recipe) that is instantly faulty and can not build on
  one of the platforms we track even with relaxed warnings.

////////////////////////////////////////////////////////////////////////////////
 reported through the
link:http://lists.alioth.debian.org/mailman/listinfo/nut-commits[NUT Commits]
mailing list, and fixed quickly.
////////////////////////////////////////////////////////////////////////////////

- a project portal with trackers for bugs, feature request, patches and tasks

- LGTM.COM automatically checking C/C++ and Python code

- Static code analysis:
  * link:https://scan.coverity.com/projects/networkupstools-nut[Coverity Scan overview of NUT]
  * status: image:https://scan.coverity.com/projects/8829/badge.svg[Coverity Scan Build Status]
  * cppcheck as part of NUT CI farm builds and reports

NUT QA also relies on external tools and trackers, like:

////////////////////////////////////////////////////////////////////////////////
FIXME (POST):
- integrate static code analysis
	- consider splint, Frama-C, BLAST and Clang, and choose one.
	- only use coverity?!
- point other distro BTS (use Launchpad as an aggregator?!)

////////////////////////////////////////////////////////////////////////////////

- Clang
- the Debian QA tools, available through the link:https://tracker.debian.org/pkg/nut[NUT Package Tracking System]:

   * Lintian general QA checks,
   * link:http://piuparts.debian.org/sid/source/n/nut.html[piuparts] automates
     the installation, upgrade and removal testing processes.

- a runtime testing suite, which automates the inter-layer communication testing
  (driver -- upsd -- upsmon / clients), that is part of Ubuntu.
  link:https://git.launchpad.net/ubuntu/+source/nut/tree/debian/tests/test-nut.py[The NUT testing script]
  is available in the link:https://code.edge.launchpad.net/qa-regression-testing[Ubuntu QA Regression Testing suite].
+
  It installs NUT packages, configures it with the dummy-ups driver, changes
  a few data points and checks that these are well propagated with upsc.

- similar approach is explored in NIT (NUT Integration Testing) suite,
  which is part of the codebase and automated with `make check-NIT`;
  it can also be added to default `make check` activities by running
  `configure --enable-check-NIT`

  * Note that developers updating components which directly impact NIT runs
    may benefit from `make check-NIT-devel` target, to rebuild the `upsd`,
    `dummy-ups`, `cppnit` and other programs used in the test as they iterate.

- link:https://bugzilla.redhat.com/buglist.cgi?component=nut[Redhat / Fedora Bug tracker]

- link:https://www.openhub.net/p/nut[Black Duck Open Hub] (formerly Ohloh.net)
  provides metrics on NUT source code base and activity.

Runtime quality
~~~~~~~~~~~~~~~

- NUT provides many link:user-manual.html#NUT_Security[security features] to
ensure a maximum runtime security level.

- Packages use several link:http://wiki.debian.org/Hardening[Hardening methods]
to protect NUT binaries.

////////////////////////////////////////////////////////////////////////////////
FIXME (POST):

- write a code conformance checker (nut-lint)
- write a § on driver maintenance status
- consider using [http://forge.novell.com/modules/xfmod/project/?opensuse OpenSUSE Build Service tools]
- provide software metrics and evolution over the time

  * [http://www.flossmetrics.org/ FlossMetrics]
  * [http://en.wikipedia.org/wiki/Software_metric Wikipedia]
  * [http://cccc.sourceforge.net/ CCCC - C and C++ Code Counter]
  * [http://open.ncsu.edu/se/tutorials/metrics/ Metrics with Eclipse]

- Code documentation, for the core architecture (client and drivers are already
documented)

  * [http://doxygen.org/ Doxygen]
  * [http://naturaldocs.org/ NaturalDocs]

////////////////////////////////////////////////////////////////////////////////