My Work: Python Project Documentation Using Sphinx

Introduction

In this article we will look at how to uses Sphinx documentation builder for documenting python project.

Background

Code document is essential for a project .It increases the usability of code are make it appealing for use to wide range of users.It is a neccessary tool for developer working in opensource or enterprise software projects.

reStructuredText is an easy-to-read, plaintext markup syntax and parser system.The primary goal of reStructuredText is to define and implement a markup syntax for use variety of documentation domains.The advantage is that markup language is readable and simple, yet powerful enough to provide the ability to convert reStructuredText documents into useful structured data formats .

Sphinx uses reStructuredText as its markup language, and is a tool to translate a set of reStructuredText source files into various output formats like HTML and LaTeX.

The current standard tool for documenting python documents is by using Sphinx.

Sphinx supports automatic generation of module or package documentation based on parsing function headers and extracting doc strings.Such doc strings are typically referred to as API documentation

There are two principal steps in making API documentation. First, write doc strings in all key classes, methods, and functions using the formatting options defined by Sphinx and second is configuring and using Sphinx for document generation.

Python Docstrings

Python documentation strings (or docstrings) provide a convenient way of associating documentation with Python modules, functions, classes, and methods.

It is a good programming practice to Write docstrings for all public modules, functions, classes, and methods.Docstrings for private modules can be skipped ,however including comments or docstrings as developer notes is always helpful.

A lot of references can be found online describing the format of generic python docstring and good programming and documentation practice like http://legacy.python.org/dev/peps/pep-0008/#documentation-strings or http://legacy.python.org/dev/peps/pep-0257/

Sphinx uses numpy docstring format.A standard guide to numpy/scipy docstring format can be found at https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txt

NumPy, SciPy, and the scikits follow a common convention for docstrings that provides for consistency, while also allowing the toolchain to produce well-formatted reference guides.The docstring standard uses re-structured text (reST) syntax and is rendered using Sphinx

The are numerous docstring,here we mention only the important once which are freuently used in numpy/scipy documentation

Function Docstring

A documentation string (docstring) is a string that describes a module, function, class, or method definition. The docstring is a special attribute of the object that can be accessed by (object.__doc__) and is surrounded by triple double quotes .

For example

def compute(self,input):

""" function computes the output of the hidden layer for input matrix """

Parameters Docstring

Parameter docstring contains the description of the function arguments, keywords and their respective types.For example

Parameters

----------

input : numpy array,shape=(N,n_in)

:math:`h_{i-1}(x)` is the `input`

The input variable and associated type and desciption are mentioned.

Returns Docstring

Contains the explanation of the returned values and their types.Name of return variable is optional

Returns

-----------

output : numpy array,shape=(N,n_out)

:math:`f(b_k + w_k^T h_{i-1}(x))`

Latex Math

numpydoc docstring format which is used by Sphix allows us to use latex math expression

An way to include inline math is shown in examples in Parameter and Returns Docstring sections.It contains a specifier ":math:" followed by the math expression in back-uotes.

To include math equations in seperate line we can use

:math:`f(b_k + w_k^T h_{i-1}(x))`

.. math::
    f(b_k + w_k^T h_{i-1}(x))

There are various other docstring like Raises,See Also,Notes,References,Examples,warning.Refer the following link to get more information on the same https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txt

The sample output generated by sphinx for the function is shown below

 def compute(self,input):


        """function computes the output of the hidden layer for input matrix

        Parameters

        ---------

        input   :   numpy array,shape=(N,n_in)

                    :math:`h_{i-1}(x)` is the `input`

        Returns

        -----------

        output  : numpy array,shape=(N,n_out)

                    :math:`f(b_k + w_k^T h_{i-1}(x))

        """               
        #performs affine transformation over input vector        
        linout=numpy.dot(self.W,input.T)+np.reshape(self.b,(self.b.shape[0],1));     
        #applies non linear activation function over computed linear transformation

        self.output=self.activation(linout).T;               
        return self.output;

compute(input)[source]

function computes the output of the hidden layer for input matrix

Parameters:	input : numpy array,shape=(N,n_in) hi−1(x) is the input
Returns:	output : numpy array,shape=(N,n_out) f(bk+wTkhi−1(x))

Installation and Configuration

Since Sphinx is written in the Python language,Thus atleast python version 2.6 is required.

For Ubuntu OS

Easiest way to Install python,numpy,scipy packages are synaptic package manager
NumpyDocs :easy_install numpydoc
Pillow : pip install Pillow
sudo apt-get install python-matplotlib
apt-get install python-sphinx or easy_install sphinx

To generate documentation run sphinx-apidoc utility

sphinx-apidoc -o docs project_dir -F

will generate the files required for document generation in the docs directory.The source files are referred from the project_dir path.

cd docs

make html

the make command will generate the html documentation in the _build/html directory.The main page can be accessed by opening the index.html file .

Configuration

To enable numpydoc format docstring as well as latex math euations some additional configurations needs to be performed.

Go to the docs directory and open the config.py file in any document editor

Add or Modify the command to contain the path of the source directory.

sys.path.insert(0, os.path.abspath('project_dir'))

If the above command is not present,the sphix document builder will be unable to find the python file and generate blank documentation.

The next change reuired is to add extensions.Locate the extensions keyword in the document and modify it as below to add extensions for math,numpydocs ,autodoc,doctest etc

extensions = [
              'sphinx.ext.autodoc',
              'sphinx.ext.viewcode',
              'matplotlib.sphinxext.mathmpl',
              'matplotlib.sphinxext.only_directives',
              'matplotlib.sphinxext.plot_directive',
              'sphinx.ext.mathjax',
              'numpydoc',
              'sphinx.ext.autodoc',
              'sphinx.ext.doctest',
              
              
              
]

The file also contains the information like "project name" ,author name ,copyright information which can be modified

project = 'PyVision'
copyright = u'2014, pi19404'

version = '0.0.1'

release = '0.0.1'

# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'colorful'

There are also a host of options for html formatting

just run make html in the docs directory for changes to reflect and regenerate all the documentation
A html themes directory is taken from scikit-learn source just for demonstration purposes
The following documentation is used for pyVision - python machine learning library project.

Code

The source code for the project can be found at github repository www.github.com/pi19404/pyVision
The doc/api directory contains the code documentation

To generate the documentation for the project go to the api directory and run the command "make html"

you can access the documentation by opening the pyVision/docs/api/_build/html/index.html file in the browser

The html template files have been taken from scikit-learn documentation directory . Documentation of some of the files may be incomplete.

The entire project source can be found at below link