With this article I want to provide a detailed description of creating a Sourcetrail setup for C and C++ projects, that is easier to read and more in-depth than our documentation. I will also try to explain why Sourcetrail needs to know the information it requires and what’s going on behind the scenes. This guide is up-to-date for Sourcetrail 2017.4.46. I’ll start the article with an introduction about the WHY and get to the HOW as quickly as I can. If that’s too slow for you, you can always skip to the HOW directly.
Edit: New Video on Project Setup for C++
We provide some of this information now also in the form of a video. It shows how to create C++ projects using setup from Clang Compilation Database as well as empty source group setup. It includes some best practices as well as tips on how to fix errors.
Why Do We Need a Project Configuration
Wouldn’t it be great if we could just fire up Sourcetrail, pass it a bunch of source code and header files and start asking it questions about them? Maybe something basic like:
“Where is the class used that is defined inside the file I’m currently looking at?”
or maybe something more advanced…
“What are the types actually used as template arguments for std::unique_ptr? So what kind of std::unique_ptr instances are there?”
Why doesn’t that work? Why do we have to specify all kind of additional information in our project setup? Why does it have to be so complicated? The answer to those questions is quite simple. Sourcetrail heavily relies on the Clang compiler frontend to tackle the issue of “understanding” source code. But instead of building a binary like a real compiler does, Sourcetrail quickly steps in and takes over control just before any code generation and optimization would take place.
So the problem with compilers is that you can’t just tell them “Hey, go and build that code!” they’ll probably just respond with “How?” That’s why most projects offer some kind of build configuration files like Makefiles, Visual Studio projects or even meta formats like CMake. Those files specify exactly how the compiler should build the source code into a binary. And it is because Sourcetrail’s code analysis is no different from the initial steps of an ordinary compiler, that it needs to know the same stuff.
Different Ways to Create a Sourcetrail Project
With Sourcetrail we didn’t really want to reinvent the wheel on project configuration, so we made Sourcetrail re-use some of the most commonly used build configurations to extract the required data.
- As a Clang based tool Sourcetrail already works out of the box with the Clang build-file format called “Compilation Database”. If you are using CMake you can easily generate a compilation database from your CMakeLists file.
- If you are working on a Visual Studio project, you can use our Visual Studio extension to export a compilation database for your Visual Studio solution. Hint: You can also use the generated compilation database to run other Clang based tools like include-what-you-use to automatically clean up your includes or Clang-Tidy to check your source code for typical programming errors.
- If you are building your project using a makefile configuration you can use tools like Bear to generate a compilation database to feed into Sourcetrail.
However, if you don’t have one of these build configurations for your project but still want to use Sourcetrail, we offer a manual project setup as a fallback.
If you are the one person that configures all your company’s build setups, you can probably skip this article and master the manual Sourcetrail project setup all by yourself. However, most C and C++ programmers are probably not that familiar with setting up build configuration for their compilers, but trust me, it is not as scarry as it may sound at first.
The Project Setup
After choosing to create a new project, Sourcetrail needs to know two basic things: The project’s name and a file system location where it should save the project configuration (
.srctrlprj file) you are currently creating. This is for Sourcetrail’s internal reference only, so you can choose what you want here. However, please keep in mind that all relative paths that you use further on, need to be specified relative to your Sourcetrail project location.
Adding a Source Group
Your Sourcetrail project references the actual source code via source groups. A source group specifies which source files should be analyzed by Sourcetrail and includes all the parameters required to analyze those source files. A Sourcetrail project requires at least one source group but may also contain multiple source groups. This may be necessary if you want to analyze source files from different projects that do not share the same parameters, or even the same programming language. I’d recommend you set up a source group for the core of the project you are currently working on. Later on it’s still possible to add another source group for a different part of your project (e.g unit tests) or for a dependency that you want to inspect in depth. Once you are done with the project setup the indexed data for all your source groups will be merged into a single graph that you can view and navigate seamlessly.
Choosing the Kind of Source Group
Sourcetrail lets you choose between different kinds of source groups. Most of them allow for automatic project setup, but in the scope of this article we’ll mostly cover the “Empty C++ Source Group”. After picking this source group the project wizard starts asking you questions about the source code you want to add.
Language Standard and Cross Compilation Options
First, pick a Language Standard for your project. For most projects you won’t need to change this option because the latest available standard is already pre-selected. There are two versions available for each standard, a basic version and one with GNU extensions. Next, you may choose to enable Cross Compilation options for your project. You can leave this option disabled if your source code can be build into a binary that runs on your current machine. If your soure code only builds for a different architecture (e.g. if you are cross compiling for an embedded system) you can use these options to specify the target architecture of the provided source code. Even though Sourcetrail will not generate a target binary, providing these options will affect which header files the indexer will be looking for while analyzing your source code. If you enable cross compilation, but you are not sure which value to pick for a certain option, just choose "unknown" and Sourcetrail will try to guess the correct value.
Adding the Source Code
In this next step you can add all the Files & Directories to Index. Sourcetrail will recursively go through all the paths you entered and start the indexing process for all source files that match the Source File Extensions you specified. While processing the Files & Directories to Index Sourcetrail will skip all contents of the Excluded Files & Directories.
Please note that you don’t need to enter any header file extensions to the Source File Extensions list. Header files of your project will be known to Sourcetrail because the indexing process will encounter them while checking the
#include directives in your source files. However, Sourcetrail will skip indexing the content of header files if they are not located inside the Files & Directories to Index paths. So usually you will not add the paths to your system includes here because you just want to see where something like
std::vector is used, but you are not interested in the internals of this class. It would still be possible to add those paths but keep in mind that this will slow down the indexing process a lot.
Specifying Include Paths
In this step you need to tell Sourcetrail where it should look for header files whenever it encounters an
#include directive within your source code. This applies for both, your project internal included files as well as to external header files.
For example if your source code contains an
#include directive like
#include "boost/date_time.hpp" you need to add some path like
c:/dev/boost_1_65_1/ to header search paths because this path contains a folder called
boost that contains a file
Whether you add these header search paths to the Include Paths or to the Global Include Paths does not matter to Sourcetrail’s indexer. Include Paths only apply for your current source group while the Global Include Paths are shared between all your projects and source groups.
While indexing your source code you may encounter some FATAL ERRORs that inform you of some include that could not be found. In this case you would need to add the respective path to your source group’s Include Paths. However, indexing your project may take some time and getting all these header search paths right can be an iterative process. To speed things up, we added the validate include directives button that runs a really quick check to tell you which
#include directives could not be resolved. This check doesn’t account for any
#ifdef regions so there may be false positives that require you to think whether this will be really an issue while indexing, because the indexer accounts for
Providing Additional Compiler Flags
This last step allows you to add additional Clang-style Compiler Flags to your source group. Internally Sourcetrail converts all the options you specified in the previous steps to compiler flags as well, but here you can add the more advanced information directly. This means that you could also leave the Include Paths on the previous page empty and add those paths to the Compiler Flags list using
This list is probably used mostly to add preprocessor definitions to the project. For example you can use
-D RELEASE to add
#define RELEASE. In case you are working on Windows, the Microsoft Visual Studio compatibility flags might be important to add. If Sourcetrail detected a Visual Studio installation on your system, these flags will be prefilled to this list automatically.
Indexing Your Source Code
After you provided all this information Sourcetrai will go ahead to build a full index for your project and store the generated data to a database (
.srctrldb file). You can always choose to abort the indexing process and continue later on by clicking the refresh button (or using the refresh shortcut). If you refresh your fully indexed project and Sourcetrail detects that some files have changed, it will stick to an incremental indexing policy by only re-indexing the files that may be affected by those changes.
That’s it for now. I tried to keep this guide as short as possible while providing as much detailed information as necessary. I hope it helps you getting started with Sourcetrail for your C and C++ project. If anything remains unclear, please leave a comment and I’am happy to clarify.