debianmemberportfolio/docs/source/devdocs.rst

Development of Debian Member Portfolio Service
==============================================

The Debian Member Portfolio Service is implemented in `Python 3
<https://www.python.org>`_ using the `Flask <http://flask.pocoo.org/>`_ web
application framework.

The following sections describe how to setup a local development environment
for the Debian Member Portfolio Service.

All instructions assume that you work on a Debian system. You should use Python
3 for development.

Setup of a local development
----------------------------

To start working on the source code you need to have `git`_ installed::

  sudo aptitude install git

.. _git: http://www.git-scm.com/

The canonical git repository for the Debian Member Portfolio Service is
available at https://debianstuff.dittberner.info/git/debianmemberportfolio.git.
To get a clone of the source code you change to a directory of your choice and
invoke git clone::

  cd ~/src
  git clone https://debianstuff.dittberner.info/git/debianmemberportfolio.git

You should use `venv`_ to separate the development environment from your
system wide Python installation. You can install virtualenv using::

  sudo aptitude install python3-venv

.. _venv: https://docs.python.org/3/library/venv.html

When you have :command:`pyvenv` installed you should create a virtual
environment for Debian Member Portfolio Service development and install the
requirements using `pip <https://pypi.python.org/pypi/pip>`_::

  mkdir ~/.virtualenvs
  pyvenv ~/.virtualenvs/dmportfolio
  . ~/.virtualenvs/dmportfolio/bin/activate
  cd ~/src/debianmemberportfolio
  pip install -r stretchreq.pip

.. note::

  The Debian Member Portfolio Service instance at http://portfolio.debian.net/
  is running on a Debian Stretch server, therefore :file:`stretchreq.pip`
  contains dependency versions matching that Debian release.

The dependency download and installation into the virtual environment takes
some time.

Debian Member Portfolio Service needs the JQuery JavaScript library to function
properly. The JQuery library is not included in the git clone and must be
copied into the subdirectory
:file:`debianmemberportfolio/static/javascript/jquery`. On Debian systems you
can install the package libjs-jquery and place a symlink to the directory
:file:`/usr/share/javascript` into :file:`debianmemberportfolio/static`: ::

  sudo aptitude install libjs-jquery
  ln -s /usr/share/javascript debianmemberportfolio/static

Prepare for first startup
~~~~~~~~~~~~~~~~~~~~~~~~~

The Debian Member Portfolio Service uses data from the Debian keyring to get
information regarding OpenPGP keys and names related to email addresses. Before you
can run the service you need to fetch a copy of the keyring and prepare it for
use by the code.

.. note::

  You need rsync and gnupg for these tasks::

    sudo aptitude install rsync gnupg

When you have both installed you can run::

  . ~/.virtualenvs/dmportfolio/bin/activate
  ./synckeyrings.sh
  python3 debianmemberportfolio/model/keyringanalyzer.py

The first synchronizes the keyrings in :file:`$HOME/debian/keyring.debian.org`
with files on the `keyring.debian.org <http://keyring.debian.org>`_ host. And
the second generates a key/value database in
:file:`debianmemberportfolio/model/keyringcache.db` that is used by the code.

Run a development server
~~~~~~~~~~~~~~~~~~~~~~~~

You can run a development server using::

  python3 run.py

The output of this command should look like the following::

  * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
  * Restarting with stat

You can now access your development server at the URL that is printed by the command.

If you want to stop the development server press :kbd:`Ctrl + C`.

Common development tasks
------------------------

Add new URL
~~~~~~~~~~~

Debian Member Portfolio Service uses a ini style configuration file
:file:`debianmemberportfolio/model/portfolio.ini` to configure the generated URL
patterns. The actual URL generation is done in
:py:func:`~debianmemberportfolio.views.urllist`.

If you want to add a new URL type you have to add a line in
:file:`portfolio.ini` and an entry in :py:mod:`~debianmemberportfolio.views`'s
:py:attr:`~debianmemberportfolio.views._LABELS` dictionary. The top level
dictionary keys correspond to sections in the ini file. The dictionary values
are dictionaries themselves that contain a special key ``label`` that defines
the label of the section in the output and keys for each entry to be rendered
in that section. The values in these sub-dictionaries are strings marked for
translation using the :py:func:`~flask_babel.lazy_gettext` function from
:py:mod:`flask_babel`.

The patterns in :file:`portfolio.ini` can contain the following placeholders
that are filled at runtime:

================== ========================================
Placeholder        Replacement
================== ========================================
%(salsausername)s  user name on `salsa.debian.org`_
%(email)s          email address (URL encoded)
%(emailnoq)s       email address
%(firstchar)s      first character of the email address
%(forumsid)s       forum user id
%(openpgpfp)s      OpenPGP key fingerprint
%(name)s           full name (i.e. John Smith)
%(username)s       Debian user name
%(wikihomepage)s   full name in camel case (i.e. JohnSmith)
================== ========================================

.. _salsa.debian.org: https://salsa.debian.org/

The replacement of placeholders is performed in the
:py:func:`~debianmemberportfolio.views.urllist` function. And uses data from
the Debian keyring. Access to the pre-parsed keyring data is performed using
the :py:func:`~debianmemberportfolio.model.dddatabuilder.build_data` function
of the module :py:mod:`debianmemberportfolio.model.dddatabuilder`, which uses
several helper functions from :py:mod:`debianmemberportfolio.model.keyfinder`
to access the key information.