Linking unrelated objects with each other

Sometimes it is necessary to build links between objects whose datatypes are not related via a OneToOneRelation or a OneToManyRelation. These external relations are called Links in podio, and they are implemented as a templated version of the code that would be generated by the following yaml snippet (in this case between generic FromT and ToT datatypes):

Link:
  Description: "A weighted link between a FromT and a ToT"
  Author: "P. O. Dio"
  Members:
    - float weight // the weight of the link
  OneToOneRelations:
    - FromT from // reference to the FromT
    - ToT to     // reference to the ToT

Link basics

Links are implemented as templated classes forming a similar structure as other podio generated classes, with several layers of which users only ever interact with the User layer. This layer has the following basic classes

/// The collection class that forms the basis of I/O and also is the main entry point
template<typename FromT, typename ToT>
class LinkCollection;

/// The default (immutable) class that one gets after reading a collection
template<typename FromT, typename ToT>
class Link;

/// The mutable class for creating links before writing them
template<typename FromT, typename ToT>
class MutableLink;

Although the names of the template parameters, FromT and ToT imply a direction of the link, from a technical point of view nothing actually enforces this direction, unless FromT and ToT are both of the same type. Hence, links can effectively be treated as bi-directional, and one combination of FromT and ToT should be enough for all use cases (see also the usage section).

For a more detailed explanation of the internals and the actual implementation see the implementation details.

How to use Links

Using Links is quite simple. The most straight forward way is to simply declare them as part of the datamodel, as described here. That will result in code generation that effectively does what is described below here. However, it’s not strictly necessary to do that in case non-generated code is preferred. In line with other datatypes that are generated by podio all the functionality can be gained by including the corresponding Collection header. After that it is generally recommended to introduce a type alias for easier usage. As a general rule Links need to be declared with the default (immutable) types. Trying to instantiate them with Mutable types will result in a compilation error.

#include "podio/LinkCollection.h"

#include "edm4hep/MCParticleCollection.h"
#include "edm4hep/ReconstructedParticleCollection.h"

// declare a new link type
using MCRecoParticleLinkCollection = podio::LinkCollection<edm4hep::MCParticle,
                                                           edm4hep::ReconstructedParticle>;

This can now be used exactly as any other podio generated collection, i.e.

edm4hep::MCParticle mcParticle{};
edm4hep::ReconstructedParticle recoParticle{};

auto mcRecoLinks = MCRecoParticleLinkCollection{};
auto link = mcRecoLinks.create(); // create an link;
link.setFrom(mcParticle);
link.setTo(recoParticle);
link.setWeight(1.0); // This is also the default value!

and similar for getting the linked objects

auto mcP = link.getFrom();
auto recoP = link.getTo();
auto weight = link.getWeight();

In the above examples the From and To in the method names imply a direction, but it is also possible to use a templated get and set method to retrieve the linked objects via their type:

link.set(mcParticle);
link.set(recoParticle);

auto mcP = link.get<edm4hep::MCParticle>();
auto recoP = link.get<edm4hep::ReconstructedParticle>();
auto weight = link.getWeight();

It is also possible to access the elements of a link via an index based get (similar to std::tuple). In this case 0 corresponds to getFrom, 1 corresponds to getTo and 2 corresponds to the weight. The main purpose of this feature is to enable structured bindings:

const auto& [mcP, recoP, weight] = link;

The above three examples are three equivalent ways of retrieving the same things from an Link. The templated get and set methods are only available if FromT and ToT are not the same type and will lead to a compilation error otherwise.

Enabling I/O capabilities for Links

Links do not have I/O support out of the box. This has to be enabled via the PODIO_DECLARE_LINK macro (defined in the LinkCollection.h header). If you simply want to be able to read / write Links in a standalone executable, it is enough to use this macro somewhere in the executable, e.g. to enable I/O capabilities for the MCRecoParticleLinks used above this would look like:

PODIO_DECLARE_LINK(edm4hep::MCParticle, edm4hep::ReconstructedParticle)

The macro will also enable SIO support if the PODIO_ENABLE_SIO=1 is passed to the compiler. This is done by default when linking against the podio::podioSioIO library in CMake.

For enabling I/O support for shared datamodel libraries, it is necessary to have all the necessary combinations of types declared via PODIO_DECLARE_LINK and have that compiled into the library. This is necessary if you want to use the python bindings, since they rely on dynamically loading the datamodel libraries.

The LinkNavigator utility

podio::LinkCollections store each link separately even if a given object is present in several links. Additionally, they don’t offer any really easy way to look up objects that are linked (apart from manually looping and comparing elements). To alleviate these issues, we provide the podio::LinkNavigator utility class that facilitates navigating links and lookups. It can be constructed from any podio::LinkCollection and can then be used to retrieve linked objects. E.g.

const auto& recoMcLinks = event.get<edm4hep::RecoMCParticleLinkCollection>("RecoMCLinks");
const auto linkNavigator = podio::LinkNavigator(recoMcLinks);

// For podio::LinkCollections with disparate types just use getLinked
const auto linkedRecs = linkNavigator.getLinked(mcParticle);

If you want to be explicit about the lookup direction, e.g. in case you have a link that has the same From and To type, you can use the overloads that take a second tag argument:

const auto linkedMCs = linkNavigator.getLinked(recoParticle, podio::ReturnTo);

The return type of all methods is a std::vector<WeightedObject>, where the WeightedObject is a simple template class that wraps the object and its weight. It supports structured bindings, so you can e.g. do the following

for (const auto& [reco, weight] : linkedRecs) {
  // do something with the reco particle and its weight
}

Alternatively, you can access the object via the o member and the weight via the weight member.

Implementation details

In order to give a slightly easier entry to the details of the implementation and also to make it easier to find where things in the generated documentation, we give a brief description of the main ideas and design choices here. With those it should be possible to dive deeper if necessary or to understand the template structure that is visible in the documentation, but should be fairly invisible in usage. We will focus mainly on the user facing classes, as those deal with the most complexity, the underlying layers are more or less what could be obtained by generating them via the yaml snippet above and sprinkling some <FromT, ToT> templates where necessary.

File structure

The user facing "podio/LinkCollection.h" header essentially just defines the PODIO_DECLARE_LINK macro (depending on whether SIO support is desired and possible or not). All the actual implementation is done in the following files:

• "podio/detail/LinkCollectionImpl.h": for the collection functionality

• "podio/detail/Link.h": for the functionality of single link

• "podio/detail/LinkCollectionIterator.h": for the collection iterator functionality

• "podio/detail/LinkObj.h": for the object layer functionality

• "podio/detail/LinkCollectionData.h": for the collection data functionality

• "podio/detail/LinkFwd.h": for some type helper functionality and some forward declarations that are used throughout the other headers

• "podio/detail/LinkSIOBlock.h": for defining the SIOBlocks that are necessary to use SIO

As is visible from this structure, we did not introduce an LinkData class, since that would effectively just be a float wrapped inside a struct.

Default and Mutable Link classes

A quick look into the LinkFwd.h header will reveal that the default and Mutable Link classes are in fact just partial specialization of the LinkT class that takes a bool Mutable as third template argument. The same approach is also followed by the LinkCollectionIterators:

template<typename FromT, typename ToT, bool Mutable>
class LinkT;

template <typename FromT, typename ToT>
using Link = LinkT<FromT, ToT, false>;

template <typename FromT, typename ToT>
using MutableLink = LinkT<FromT, ToT, true>;

Throughout the implementation it is assumed that FromT and ToT always are the default handle types. This is ensured through static_asserts in the LinkCollection to make sure it can only be instantiated with those. The GetDefaultHandleType helper templates are used to retrieve the correct type from any FromT regardless of whether it is a mutable or a default handle type With this in mind, effectively all mutating operations on Links are defined using the following template structure (taking here setFrom as an example)

template <typename FromU>
  requires(Mutable && std::is_same_v<detail::GetDefaultHandleType<FromU>, FromT> &&
           detail::isDefaultHandleType<FromU>)
void setFrom(FromU value);

Compilation will fail unless the following conditions are met

• The object this method is called on has to be Mutable.

• The passed in value is either a Mutable or default class of type FromT.

In some cases the template signature looks like this

template <bool Mut = Mutable>
  requires(Mut && Mutable)
void setWeight(float value) {
  m_obj->data.weight = value;
}

The reason to have a defaulted bool template parameter here is the same as the one for having a typename FromU template parameter above: SFINAE only works with deduced types. Using Mut && Mutable in the std::enable_if makes sure that users cannot bypass the immutability by specifying a template parameter themselves.