From: Matthew Fernandez Date: Mon, 15 Aug 2022 03:01:14 +0000 (-0700) Subject: DEVELOPERS.md: move how to make a release to the bottom X-Git-Tag: 5.0.1~5^2~3 X-Git-Url: https://granicus.if.org/sourcecode?a=commitdiff_plain;h=6ccfa224f1fb51b5697f877881a203832ead8660;p=graphviz DEVELOPERS.md: move how to make a release to the bottom This guide was sort of back-to-front, in that it prioritized readers who were maintainers looking at cutting an upcoming release. In reality, this document is read far more often by drive-by contributors seeking how to build and test their changes. --- diff --git a/DEVELOPERS.md b/DEVELOPERS.md index e187693e1..e425e397d 100644 --- a/DEVELOPERS.md +++ b/DEVELOPERS.md @@ -4,6 +4,174 @@ [[_TOC_]] +## Building + +Instructions for building Graphviz from source are available +[on the public website](https://graphviz.org/download/source/). However, if you +are building Graphviz from a repository clone for the purpose of testing changes +you are making, you will want to follow different steps. + +### Autotools build system + +```sh +# generate the configure script for setting up the build +./autogen.sh + +# you probably do not want to install your development version of Graphviz over +# the top of your system binaries/libraries, so create a temporary directory as +# an install destination +PREFIX=$(mktemp -d) + +# configure the build system +./configure --prefix=${PREFIX} + +# if this is the first time you are building Graphviz from source, you should +# inspect the output of ./configure and make sure it is what you expected + +# compile Graphviz binaries and libraries +make + +# install everything to the temporary directory +make install +``` + +### CMake build system + +Note that Graphviz’ CMake build system is incomplete. It only builds a subset +of the binaries and libraries. For most code-related changes, you will probably +want to test using Autotools or MSBuild instead. + +However, if you do want to use CMake: + +```sh +# make a scratch directory to store build artifacts +mkdir build +cd build + +# you probably do not want to install your development version of Graphviz over +# the top of your system binaries/libraries, so create a temporary directory as +# an install destination +PREFIX=$(mktemp -d) + +# configure the build system +cmake -DCMAKE_INSTALL_PREFIX=${PREFIX} .. + +# compile Graphviz binaries and libraries +make + +# install everything to the temporary directory +make install +``` + +### Microsoft Build Engine + +TODO + +## Testing + +The Graphviz test suite uses [pytest](https://pytest.org/). This is not because +of any Python-related specifics in Graphviz, but rather because pytest is a +convenient framework. + +If you have compiled Graphviz and installed to a custom location, as described +above, then you can run the test suite from the root of a Graphviz checkout as +follows: + +```sh +# run the Graphviz test suite, making the temporary installation visible to it +env PATH=${PREFIX}/bin:${PATH} C_INCLUDE_PATH=${PREFIX}/include \ + LD_LIBRARY_PATH=${PREFIX}/lib LIBRARY_PATH=${PREFIX}/lib \ + PYTHONPATH=${PREFIX}/lib/graphviz/python3 \ + python3 -m pytest tests rtest --verbose +``` + +On macOS, use the same command except replacing `LD_LIBRARY_PATH` with +`DYLD_LIBRARY_PATH`. + +*TODO: on Windows, you probably need to override different environment variables?* + +## Performance and profiling + +The runtime and memory usage of Graphviz is dependent on the user’s graph. It is +easy to create “expensive” graphs without realizing it using only moderately +sized input. For this reason, users regularly encounter performance bottlenecks +that they need help with. This situation is likely to persist even with hardware +advances, as the size and complexity of the graphs users construct will expand +as well. + +This section documents how to build performance-optimized Graphviz binaries and +how to identify performance bottlenecks. Note that this information assumes you +are working in a Linux environment. + +### Building an optimized Graphviz + +The first step to getting an optimized build is to make sure you are using a +recent compiler. If you have not upgraded your C and C++ compilers for a while, +make sure you do this first. + +The simplest way to change flags used during compilation is by setting the +`CFLAGS` and `CXXFLAGS` environment variables: + +```sh +env CFLAGS="..." CXXFLAGS="..." ./configure +``` + +You should use the maximum optimization level for your compiler. E.g. `-O3` for +GCC and Clang. If your toolchain supports it, it is recommended to also enable +link-time optimization (`-flto`). You should also disable runtime assertions +with `-DNDEBUG`. + +You can further optimize compilation for the machine you are building on with +`-march=native -mtune=native`. Be aware that the resulting binaries will no +longer be portable (may not run if copied to another machine). These flags are +also not recommended if you are debugging a user issue, because you will end up +profiling and optimizing different code than what may execute on their machine. + +Most profilers need a symbol table and/or debugging metadata to give you useful +feedback. You can enable this on GCC and Clang with `-g`. + +Putting this all together: + +```sh +env CFLAGS="-O3 -flto -DNDEBUG -march=native -mtune=native -g" \ + CXXFLAGS="-O3 -flto -DNDEBUG -march=native -mtune=native -g" ./configure +``` + +### Profiling + +#### [Callgrind](https://valgrind.org/docs/manual/cl-manual.html) + +Callgrind is a tool of [Valgrind](https://valgrind.org/) that can measure how +many times a function is called and how expensive the execution of a function is +compared to overall runtime. When you have built an optimized binary according +to the above instructions, you can run it under Callgrind: + +```sh +valgrind --tool=callgrind dot -Tsvg test.dot +``` + +This will produce a file like callgrind.out.2534 in the current directory. You +can use [Kcachegrind](https://kcachegrind.github.io/) to view the results by +running it in the same directory with no arguments: + +```sh +kcachegrind +``` + +If you have multiple trace results in the current directory, Kcachegrind will +load all of them and even let you compare them to each other. See the +Kcachegrind documentation for more information about how to use this tool. + +Be aware that execution under Callgrind will be a lot slower than a normal run. +If you need to see instruction-level execution costs, you can pass +`--dump-instr=yes` to Valgrind, but this will further slow execution and is +usually not necessary. To profile with less overhead, you can use a statistical +profiler like Linux Perf. + +#### [Linux Perf](https://perf.wiki.kernel.org/index.php/Main_Page) + +TODO + ## How to make a release ### Downstream packagers/consumers @@ -197,174 +365,6 @@ See [`gen_version.py`](https://gitlab.com/graphviz/graphviz/-/blob/main/gen_vers 1. Merge the merge request -## Building - -Instructions for building Graphviz from source are available -[on the public website](https://graphviz.org/download/source/). However, if you -are building Graphviz from a repository clone for the purpose of testing changes -you are making, you will want to follow different steps. - -### Autotools build system - -```sh -# generate the configure script for setting up the build -./autogen.sh - -# you probably do not want to install your development version of Graphviz over -# the top of your system binaries/libraries, so create a temporary directory as -# an install destination -PREFIX=$(mktemp -d) - -# configure the build system -./configure --prefix=${PREFIX} - -# if this is the first time you are building Graphviz from source, you should -# inspect the output of ./configure and make sure it is what you expected - -# compile Graphviz binaries and libraries -make - -# install everything to the temporary directory -make install -``` - -### CMake build system - -Note that Graphviz’ CMake build system is incomplete. It only builds a subset -of the binaries and libraries. For most code-related changes, you will probably -want to test using Autotools or MSBuild instead. - -However, if you do want to use CMake: - -```sh -# make a scratch directory to store build artifacts -mkdir build -cd build - -# you probably do not want to install your development version of Graphviz over -# the top of your system binaries/libraries, so create a temporary directory as -# an install destination -PREFIX=$(mktemp -d) - -# configure the build system -cmake -DCMAKE_INSTALL_PREFIX=${PREFIX} .. - -# compile Graphviz binaries and libraries -make - -# install everything to the temporary directory -make install -``` - -### Microsoft Build Engine - -TODO - -## Testing - -The Graphviz test suite uses [pytest](https://pytest.org/). This is not because -of any Python-related specifics in Graphviz, but rather because pytest is a -convenient framework. - -If you have compiled Graphviz and installed to a custom location, as described -above, then you can run the test suite from the root of a Graphviz checkout as -follows: - -```sh -# run the Graphviz test suite, making the temporary installation visible to it -env PATH=${PREFIX}/bin:${PATH} C_INCLUDE_PATH=${PREFIX}/include \ - LD_LIBRARY_PATH=${PREFIX}/lib LIBRARY_PATH=${PREFIX}/lib \ - PYTHONPATH=${PREFIX}/lib/graphviz/python3 \ - python3 -m pytest tests rtest --verbose -``` - -On macOS, use the same command except replacing `LD_LIBRARY_PATH` with -`DYLD_LIBRARY_PATH`. - -*TODO: on Windows, you probably need to override different environment variables?* - -## Performance and profiling - -The runtime and memory usage of Graphviz is dependent on the user’s graph. It is -easy to create “expensive” graphs without realizing it using only moderately -sized input. For this reason, users regularly encounter performance bottlenecks -that they need help with. This situation is likely to persist even with hardware -advances, as the size and complexity of the graphs users construct will expand -as well. - -This section documents how to build performance-optimized Graphviz binaries and -how to identify performance bottlenecks. Note that this information assumes you -are working in a Linux environment. - -### Building an optimized Graphviz - -The first step to getting an optimized build is to make sure you are using a -recent compiler. If you have not upgraded your C and C++ compilers for a while, -make sure you do this first. - -The simplest way to change flags used during compilation is by setting the -`CFLAGS` and `CXXFLAGS` environment variables: - -```sh -env CFLAGS="..." CXXFLAGS="..." ./configure -``` - -You should use the maximum optimization level for your compiler. E.g. `-O3` for -GCC and Clang. If your toolchain supports it, it is recommended to also enable -link-time optimization (`-flto`). You should also disable runtime assertions -with `-DNDEBUG`. - -You can further optimize compilation for the machine you are building on with -`-march=native -mtune=native`. Be aware that the resulting binaries will no -longer be portable (may not run if copied to another machine). These flags are -also not recommended if you are debugging a user issue, because you will end up -profiling and optimizing different code than what may execute on their machine. - -Most profilers need a symbol table and/or debugging metadata to give you useful -feedback. You can enable this on GCC and Clang with `-g`. - -Putting this all together: - -```sh -env CFLAGS="-O3 -flto -DNDEBUG -march=native -mtune=native -g" \ - CXXFLAGS="-O3 -flto -DNDEBUG -march=native -mtune=native -g" ./configure -``` - -### Profiling - -#### [Callgrind](https://valgrind.org/docs/manual/cl-manual.html) - -Callgrind is a tool of [Valgrind](https://valgrind.org/) that can measure how -many times a function is called and how expensive the execution of a function is -compared to overall runtime. When you have built an optimized binary according -to the above instructions, you can run it under Callgrind: - -```sh -valgrind --tool=callgrind dot -Tsvg test.dot -``` - -This will produce a file like callgrind.out.2534 in the current directory. You -can use [Kcachegrind](https://kcachegrind.github.io/) to view the results by -running it in the same directory with no arguments: - -```sh -kcachegrind -``` - -If you have multiple trace results in the current directory, Kcachegrind will -load all of them and even let you compare them to each other. See the -Kcachegrind documentation for more information about how to use this tool. - -Be aware that execution under Callgrind will be a lot slower than a normal run. -If you need to see instruction-level execution costs, you can pass -`--dump-instr=yes` to Valgrind, but this will further slow execution and is -usually not necessary. To profile with less overhead, you can use a statistical -profiler like Linux Perf. - -#### [Linux Perf](https://perf.wiki.kernel.org/index.php/Main_Page) - -TODO - ## How to update this guide ### Markdown flavor used