README.rst 9.7 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298
  1. Pagure
  2. ======
  3. :Author: Pierre-Yves Chibon <pingou@pingoured.fr>
  4. Pagure is a git-centered forge, python based using pygit2.
  5. With pagure you can host your project with its documentation, let your users
  6. report issues or request enhancements using the ticketing system and build your
  7. community of contributors by allowing them to fork your projects and contribute
  8. to it via the now-popular pull-request mechanism.
  9. Homepage: https://pagure.io/pagure
  10. See it at work: https://pagure.io
  11. Playground version: https://stg.pagure.io
  12. If you have any questions or just would like to discuss about pagure,
  13. feel free to drop by `our Matrix room <https://matrix.to/#/#pagure:fedora.im>`_.
  14. About its name
  15. ==============
  16. The name Pagure is taken from the French word 'pagure'. Pagure in French is used as the
  17. common name for the crustaceans from the `Paguroidea <https://en.wikipedia.org/wiki/Hermit_crab>`_
  18. superfamily, which is basically the family of the Hermit crabs.
  19. Originating from French it is pronounced with a strong 'g' as you can hear
  20. on `this recording <https://pagure.io/how-do-you-pronounce-pagure/raw/master/f/pingou.ogg>`_.
  21. Get it running
  22. ==============
  23. There are several options when it comes to a development environment.
  24. They are: Docker Compose, Vagrant, and manual. Choose an option below.
  25. Docker Compose
  26. ^^^^^^^^^^^^^^
  27. Docker Compose will provide you with a container which you can develop on.
  28. Install it `with these instructions <https://docs.docker.com/compose/install/>`_.
  29. For more information about docker-compose cli, see: https://docs.docker.com/compose/reference/.
  30. To build and run the containers, use the following command::
  31. $ ./dev/docker-start.sh
  32. Once all the containers have started, you can access pagure on http://localhost:5000.
  33. To stop the containers, press Ctrl+C.
  34. Once the containers are up and running, run this command to populate the
  35. container with test data and create a new account ::
  36. $ docker-compose -f dev/docker-compose.yml exec web python3 dev-data.py --all
  37. You can then log in with any of the created users, by example:
  38. - username: pingou
  39. - password: testing123
  40. Vagrant
  41. ^^^^^^^
  42. For a more thorough introduction to Vagrant, see
  43. https://fedoraproject.org/wiki/Vagrant.
  44. An example Vagrantfile is provided as ``Vagrantfile.example``. To use it,
  45. just copy it and install Vagrant. Instructions for Fedora::
  46. $ cp dev/Vagrantfile.example Vagrantfile
  47. $ sudo dnf install ansible libvirt vagrant-libvirt vagrant-sshfs vagrant-hostmanager
  48. $ vagrant up
  49. On Ubuntu, install Vagrant directly `from the website <https://www.vagrantup.com/downloads.html>`_
  50. then run these commands instead::
  51. $ cp dev/Vagrantfile.example Vagrantfile
  52. $ sudo add-apt-repository ppa:ansible/ansible
  53. $ sudo apt update
  54. $ sudo apt install ansible libvirt0 openssh-server qemu libvirt-bin ebtables dnsmasq libxslt-dev libxml2-dev libvirt-dev zlib1g-dev ruby-dev
  55. $ vagrant plugin install vagrant-libvirt vagrant-sshfs vagrant-hostmanager
  56. If you get this error ``Block in synced_folders: Internal error. Invalid: sshfs``,
  57. when you run ``vagrant up`` , you need to install vagrant sshfs plugin, which can be done by::
  58. $ vagrant plugin install vagrant--sshfs
  59. and then::
  60. $ vagrant up
  61. The default ``Vagrantfile`` forwards ports from the host to the guest,
  62. so you can interact with the application as if it were running on your
  63. host machine.
  64. .. note::
  65. ``vagrant-hostmanager`` will automatically maintain /etc/hosts for you so you
  66. can access the development environment from the host using its hostname, which
  67. by default is ``pagure-dev.example.com``. You can choose not to use this
  68. functionality by simply not installing the ``vagrant-hostmanager`` plugin, but
  69. if you want Pagure to provide valid URLs in the UI for git repositories, you
  70. will need to adjust Pagure's configuration found in ~/pagure.cfg on the guest.
  71. When the vagrant VM is up and running, connect to it with::
  72. $ vagrant ssh
  73. This will log you into the VM as the user ``vagrant`` which has a couple of aliases
  74. preconfigured::
  75. $ pstart # Starts pagure, the workers and other tasks
  76. $ pstop # Stops all those tasks again
  77. $ pstatus # Shows pagure status
  78. The Vagrant pagure doesn't have its own log file, use ``journalctl -f`` to
  79. show the pagure output. The verbosity can be configured in the pagure config file
  80. with the ``LOGGING`` parameter.
  81. Running the unit-tests in container
  82. ***********************************
  83. To run the unit-tests, there are containers available with all the dependencies needed.
  84. First you will need to have podman and git installed on your workstation. ::
  85. $ sudo dnf install podman git
  86. Use the following command to run all tests on all container images, if the images not exist on your system, they will be build ::
  87. $ ./dev/run-tests-container.py
  88. If you wish to execute the test suite on a centos based container run the following command ::
  89. $ ./dev/run-tests-container.py --centos
  90. Container images are separated from the pagure source that will be tested.
  91. Therefore they will only automatically build if they not exist.
  92. A manual rebuild should be done from time to time to include new package versions.
  93. Also if you work on any changes in the pagure spec file, the tox config or any requirements.txt file,
  94. perform a rebuild to ensure your changed will taken into account. ::
  95. $ ./dev/run-tests-container.py --rebuild # all base and code container
  96. $ ./dev/run-tests-container.py --rebuild-code # code container only
  97. You can also run a single test case ::
  98. $ ./dev/run-tests-container.py tests/test_pagure_flask_ui_priorities.py
  99. Or a single test ::
  100. $ ./dev/run-tests-container.py tests/test_pagure_flask_ui_priorities.py:PagureFlaskPrioritiestests.test_ticket_with_no_priority
  101. You can also get ``run-tests-container`` help ::
  102. $ ./dev/run-tests-container.py --help
  103. By default, tests run against the git repo and the active branch in the current folder.
  104. To override this behavior and run the tests on your remote development branch in your fork ::
  105. $ ./dev/run-tests-container.py --repo https://pagure.io/forks/<username>/pagure.git --branch <name of branch to test>
  106. .. note:: All build, test and shell activities executed via ``run-tests-container`` will automatically be logged.
  107. Every container has it's own ``dev/results_<test-container-name>`` folder, every run creates separate
  108. files with the current unix timestamp as prefix. You should cleanup this folder from time to time.
  109. Running the unit-tests in tox
  110. *****************************
  111. You can run the tests using tox. This allows you to run the tests on local version of the code.
  112. .. note:: This way of running tests could help you test your local changes,
  113. but the output could be different then from the containerized tests.
  114. Always check your branch after push with containerized tests as well.
  115. * Install the needed system libraries::
  116. sudo dnf install libgit2-devel redis gcc tox python-alembic
  117. .. note:: You can also install any missing python interpreter.
  118. For example `sudo dnf install python35`
  119. * Run the whole test suite::
  120. tox
  121. * Or just single environment::
  122. tox -e py39
  123. * Or single module::
  124. tox tests/test_style.py
  125. Manually
  126. ^^^^^^^^
  127. * Install the needed system libraries::
  128. sudo dnf install git python3 python3-devel libgit2-devel redis \
  129. libjpeg-devel gcc libffi-devel redhat-rpm-config
  130. .. note:: Do note the version of libgit2 that you install, for example
  131. in ``libgit2-0.26.8-1`` you need to keep in mind the ``0.26``
  132. .. note:: On RHEL and derivative (CentOS, Scientific Linux) there is no
  133. `python3` package. Just `python36` or `python34` available in
  134. EPEL 7 (EPEL 6 only has `python34`). Choose the one you prefer
  135. (3.6 is newer and generally a better choice).
  136. * Retrieve the sources::
  137. git clone https://pagure.io/pagure.git
  138. cd pagure
  139. * Install dependencies
  140. * create the virtual environment (use `python3.X` explicitly on EPEL)::
  141. python3 -m venv pagure_env
  142. source ./pagure_env/bin/activate
  143. * Install the correct version of pygit2::
  144. pip install pygit2==<version of libgit2 found>.*
  145. So in our example::
  146. pip install pygit2==0.26.*
  147. * Install the rest of the dependencies::
  148. pip install -r requirements.txt
  149. * Create the folder that will receive the projects, forks, docs, requests and
  150. tickets' git repo::
  151. mkdir -p lcl/{repos,remotes,attachments,releases}
  152. * Copy and edit the alembic.ini file (especially the ``script_location`` key)::
  153. cp files/alembic.ini .
  154. vim alembic.ini
  155. * Set the ``script_location`` to ``alembic``, ie: the folder where the revisions
  156. are stored, relative to the location of the ``alembic.ini`` file.
  157. * Create the inital database scheme::
  158. python createdb.py --initial alembic.ini
  159. * Enable and start redis server::
  160. sudo systemctl enable redis
  161. sudo systemctl start redis
  162. * Start a worker, in one terminal::
  163. ./runworker.py
  164. * Run the application, in another terminal::
  165. ./runserver.py
  166. * To get some profiling information you can also run it as::
  167. ./runserver.py --profile
  168. This will launch the application at http://127.0.0.1:5000
  169. * To run unit-tests on pagure
  170. * Install the dependencies::
  171. pip install -r requirements-testing.txt
  172. * Run it::
  173. pytest tests/
  174. .. note:: While testing for worker tasks, pagure uses celery in /usr/bin/
  175. Celery then looks for eventlet (which we use for testing only) at
  176. system level and not in virtual environment. You will need to
  177. install eventlet outside of your virtual environment if you are
  178. using one.
  179. .. note:: This will also work in vagrant.