Add PAGURE_PLUGINS_CONFIG setting in pagure configuration file.

Plugins are imported from a separate configuration file that is loaded either
by setting a PAGURE_PLUGIN environment variable, or with the --plugins flag of
the runserver.py script.
This commit introduces a new variable called PAGURE_PLUGINS_CONFIG that
replaces PAGURE_PLUGIN. The new variable preserves the behavior of the old
variable, and additionally allows to specify the plugins configuration file
directly from the pagure main configuration.
A new "Plugins" section has also been added to the pagure documentation for
documenting the new changes.

doc/configuration.rst

@@ -492,7 +492,6 @@ running.
          below)
 
 
-
 Web-hooks notifications
 -----------------------
 
@@ -510,6 +509,7 @@ Defaults to: ``False``.
 
 .. _redis-section:
 
+
 Redis options
 -------------
 
@@ -538,7 +538,6 @@ communicating with the EventSource server.
 Defaults to: ``0``.
 
 
-
 Authentication options
 ----------------------
 
@@ -604,7 +603,6 @@ the content posted in the user interface.
 Defaults to: ``None``.
 
 
-
 Stomp Options
 -------------
 
@@ -2086,6 +2084,13 @@ notifications on commits made on all projects in a pagure instance.
 Defaults to: ``False``.
 
 
+PAGURE_PLUGINS_CONFIG
+~~~~~~~~~~~~~~~~~~~~~~
+
+This option can be used to specify the configuration file used for loading
+plugins. It is not set by default, instead if must be declared explicitly.
+Also see the documentation on plugins at :ref:`plugins`.
+
 
 Deprecated configuration keys
 -----------------------------
@@ -2188,3 +2193,12 @@ GITOLITE_BACKEND
 This configuration key allowed specifying the gitolite backend.
 This has now been replaced by GIT_AUTH_BACKEND, please see that option
 for information on valid values.
+
+PAGURE_PLUGIN
+~~~~~~~~~~~~~
+
+This configuration key allows to specify the path to the plugins configuration
+file. It is set as an environment variable. It has been replaced by
+PAGURE_PLUGINS_CONFIG. The new variable does not modify the behavior of the old
+variable, however unlike PAGURE_PLUGIN it can be set in the main Pagure
+configuration.

doc/index.rst

@@ -40,6 +40,7 @@ Contents:
    install_pagure_logcom
    install_crons
    configuration
+   plugins
    custom_gitolite_conf
    development
    contributing

doc/plugins.rst

@@ -0,0 +1,43 @@
+.. _plugins:
+
+Plugins
+=======
+
+Pagure provides a mechanism for loading 3rd party plugins in the form of Flask
+Blueprints. The plugins are loaded from a separate configuration file that is
+specified using the PAGURE_PLUGINS_CONFIG option. There are at least two
+reasons for keeping plugins initialization outside the main pagure
+configuration file:
+
+#. avoid circular dependencies errors. For example if the pagure configuration
+   imports a plugin, which in turn imports the pagure configuration, the plugin
+   could try to read a configuration option that has not been imported yet and
+   thus raise an error
+#. the pagure configuration is also loaded by other processes such as Celery
+   workers. The Celery tasks might only be interested in reading the
+   configuration settings without having to load any external plugin
+
+
+Loading the configuration
+-------------------------
+
+The configuration file can be loaded by setting the variable
+``PAGURE_PLUGINS_CONFIG`` inside the pagure main configuration file, for
+example inside ``/etc/pagure/pagure.cfg``. Alternatively, it is also possible
+to set the environment variable ``PAGURE_PLUGINS_CONFIG`` before starting the
+pagure server. If both variables are set, the environment variable takes
+precedence over the configuration file.
+
+
+The configuration file
+----------------------
+
+After Pagure has imported the configuration file defined in
+PAGURE_PLUGINS_CONFIG it will look for Flask Blueprints in a variable called
+``PLUGINS`` defined in the same file, for example
+``PLUGINS = [ plugin1.blueprint, plugin2.blueprint, ... ]``. Pagure will then
+proceed to register any Blueprint into the main Flask app, in the same order as
+they are listed in ``PLUGINS``.
+
+An example configuration can be seen in ``files/plugins.cfg.sample`` inside
+the Pagure repository.

files/pagure.cfg.sample

@@ -241,3 +241,7 @@ REPOSPANNER_ADMIN_MIGRATION = False
 #             'push_cert': {'cert': '',
 #                           'key': ''}}
 REPOSPANNER_REGIONS = {}
+
+# Path to the plugins configuration file that is used to load plugins. Please
+# look at files/plugins.cfg.sample for a configuration example.
+# PAGURE_PLUGINS_CONFIG = "/etc/pagure/plugins.cfg"

files/plugins.cfg.sample

@@ -0,0 +1,35 @@
+# This file demonstrates how to load plugins in pagure.
+# Pagure uses Flask Blueprints as plugins, so what we need to do is import all
+# the Blueprints into a variable called PLUGINS.
+# See the "Plugins" section in the pagure documentation for more information.
+###############################################################################
+
+
+import os
+import sys
+
+
+# For plugins that are already available in sys.path, for example packages that
+# have been installed on the system, we simply import them
+import plugin1
+import plugin2
+...
+
+
+# For any other custom plugin that is *not* in sys.path we need to add our
+# folder to sys.path first
+PLUGINS_PATH = "/path/to/plugins/folder/"
+if PLUGINS_PATH not in sys.path:
+    sys.path.append(PLUGINS_PATH)
+
+
+# Then we can import all the plugins
+import myplugin1
+import myplugin2
+...
+
+
+# Finally, create the PLUGINS list of Blueprints that we want pagure to register
+PLUGINS = [ plugin1.APP,
+            myplugin2.APP,
+            ... ]

pagure/flask_app.py

@@ -16,6 +16,7 @@ import logging
 import string
 import time
 import os
+import warnings
 
 import flask
 import pygit2
@@ -132,7 +133,30 @@ def create_app(config=None):
     # Import 3rd party blueprints
     plugin_config = flask.config.Config("")
     if "PAGURE_PLUGIN" in os.environ:
+        # Warn the user about deprecated variable (defaults to stderr)
+        warnings.warn(
+            "The environment variable PAGURE_PLUGIN is deprecated and will be "
+            "removed in future releases of Pagure. Please replace it with "
+            "PAGURE_PLUGINS_CONFIG instead.",
+            FutureWarning,
+        )
+
+        # Log usage of deprecated variable
+        logger.warning(
+            "Using deprecated variable PAGURE_PLUGIN. "
+            "You should use PAGURE_PLUGINS_CONFIG instead."
+        )
+
         plugin_config.from_envvar("PAGURE_PLUGIN")
+
+    elif "PAGURE_PLUGINS_CONFIG" in os.environ:
+        plugin_config.from_envvar("PAGURE_PLUGINS_CONFIG")
+
+    elif "PAGURE_PLUGINS_CONFIG" in app.config:
+        # If the os.environ["PAGURE_PLUGINS_CONFIG"] is not set, we try to load
+        # it from the pagure config file.
+        plugin_config.from_pyfile(app.config.get("PAGURE_PLUGINS_CONFIG"))
+
     for blueprint in plugin_config.get("PLUGINS") or []:
         logger.info("Loading blueprint: %s", blueprint.name)
         app.register_blueprint(blueprint)

runserver.py

@@ -1,12 +1,17 @@
 #!/usr/bin/env python
 
+###############################################################################
+# Please note that this script is only used for development purposes.         #
+# Do not start the Flask app with this script in a production environment,    #
+# use files/pagure.wsgi instead!                                              #
+###############################################################################
+
 from __future__ import unicode_literals, absolute_import
 
 import argparse
 import sys
 import os
 
-
 parser = argparse.ArgumentParser(description="Run the Pagure app")
 parser.add_argument(
     "--config",
@@ -15,7 +20,12 @@ parser.add_argument(
     help="Configuration file to use for pagure.",
 )
 parser.add_argument(
-    "--plugins", dest="plugins", help="Configuration file for pagure plugin."
+    "--plugins",
+    dest="plugins",
+    help="Configuration file for pagure plugins. This argument takes "
+    "precedence over the environment variable PAGURE_PLUGINS_CONFIG, which in "
+    "turn takes precedence over the configuration variable "
+    "PAGURE_PLUGINS_CONFIG.",
 )
 parser.add_argument(
     "--debug",
@@ -65,7 +75,12 @@ if args.plugins:
     if not config.startswith("/"):
         here = os.path.join(os.path.dirname(os.path.abspath(__file__)))
         config = os.path.join(here, config)
-    os.environ["PAGURE_PLUGIN"] = config
+    os.environ["PAGURE_PLUGINS_CONFIG"] = config
+    
+    # If this script is ran with the deprecated env. variable PAGURE_PLUGIN
+    # and --plugins, we need to override it too.
+    if "PAGURE_PLUGIN" in os.environ:
+        os.environ["PAGURE_PLUGIN"] = config
 
 if args.perfverbose:
     os.environ["PAGURE_PERFREPO"] = "true"