# Summary of build issues in >1.5G compilation units
## Overview of recurring problems
- The "ordered sublist" variant used for measurements is very costly
* For fitter examples, accounts for ~50% of template processing time
* The reason it's so costly is that every method code and std::visit
lambda is duplicated 63x.
* Paul think we can get this down to 6x, which would be helpful.
* See https://github.com/acts-project/acts/issues/350 .
* Quantitative impact on CKF tests:
- 3.0s of Source time (eager InstantiateFunction)
- 0.9s of ParseClass time (InstantiateClass)
- 0.2s of eager InstantiateClass time
- 0.4s of early CodeGen Function time (DebugType)
- 1.5s of MeasurementHelpers::getSurface (std::visit)
- 1.6s of `~_Variant_storage` (visit impl)
* Extra 100ms comes from instantiating multiple operator== of matrix
- 1.8s of `vector::emplace_back`
* 1.2s of allocator_traits::construct, 0.6s of `_M_realloc_insert`
- Some ~100ms of misc overhead from std::variant that includes it
- 0.7s of `std::visit` at CKFSourceLinkSelector
* 0.3s of which is instantiating various Matrix classes, will stay
- 0.3s of `std::visit` at CombinatorialKalmanFilter
* If we sum all of that up, we get 10.1s of overhead which is directly
attributable to the use of that large `std::variant`. For comparison,
total frontend time is 30s, of which 26s is clearly spent instantiating
stuff.
- The `debugLog` mechanism used throughout Propagator seems surprisingly costly
* Hard to quantify exact impact as it's spread out across lots of tiny
`std::function` constructor instantiations.
* `debugLog` impl itself is ~0.1s per instance when using boost string algs.
* See https://github.com/acts-project/acts/issues/347 for details.
* Paul is on it: https://github.com/acts-project/acts/pull/348 .
- Some Eigen geometry methods must be unbloated like matrix methods were
* These methods show up in every compilation profile:
- `Transform::rotation` (0.7s in CKF tests)
- `Transform::inverse` (0.7s in CKF tests)
- `Transform::operator*(Transform)` (0.3s in CKF tests)
- `Transform::operator*(Matrix)` (0.3s in CKF tests)
* These ones are a bit less frequent, but still common
- `Quaternion::operator*` (9/24) (0.2s in CKF tests)
* There ones are rare, but super-expensive
- `Quaternion::setFromTwoVectors` (4.2s in BoundingBoxTests)
* Based on JacobiSVD, so will also need to reimplement that one.
* These ones are rare and mildly expensive
- `Transform::pretranslate` (0.3s)
- `Translation::operator*(AngleAxis)` (0.3s)
- `Transform::prerotate` (0.2s)
- `Transform::operator*(AngleAxis)` (0.1s)
- More Eigen matrix operations could also use a reimplementation
* Unfortunately, more complex than a matrix sum/product...
* These operations show up in lots of compilation profiles:
- `Matrix::inverse` (11/24) (1.3s in CKF tests)
- `LDLT` aka Cholesky (6/24) (0.7s in CKF tests)
- `Matrix::determinant`, common in vertexing (0.6s)
* These operations show up sometimes, but rarely and remain quite cheap
- `JacobiSVD` (0.2s in CKF tests)
- Identity, setIdentity (0.2s in CKF tests)
- Norm, normalized, lpnorm (0.1s in CKF tests)
- Matrix constructors (0.1s in CKF tests)
- Matrix type instantiations & debug info (0.1s in CKF tests)
- The use of Map and Array in MultiTrajectory is not free (0.9s in CKF tests)
## Per compilation unit breakdown
- CKF Tests (4.3G)
* Why it's using lots of RAM: Code bloat from lots of small functions
* Main culprits from time profile:
- Measurement "ordered sublist" variant (top priority, ~40% of time
spent instantiating stuff aka 10.1s)
* Not just creating it: everyone who uses it suffers, detail above.
- Eigen geometry implementation (total: 2.2s)
* `Transform::rotation` (0.7s)
* `Transform::inverse` (0.7s)
* `Transform::operator*(Transform)` (0.3s)
* `Transform::operator*(Matrix)` (0.3s)
* `Quaternion::operator*` (0.2s)
- Eigen matrix transformations (total: 2.2s)
* `Matrix::inverse` (1.3s)
* `LDLT` aka Cholesky (0.7s)
* `JacobiSVD` (0.2s)
- Map and Array in MultiTrajectory reistantiate a lot (total: 0.9s)
- Misc Eigen matrix ops (total : 0.5s)
* Identity, setIdentity (0.2s in CKF tests)
* Norm, normalized, lpnorm (0.1s in CKF tests)
* Matrix constructors (0.1s in CKF tests)
* Matrix type instantiations & debug info (0.1s in CKF tests)
- KalmanFitterTests (4.1G)
* Mostly the same as CKF tests
- FittingAlgorithmFitterFunction (3.8G)
* Mostly the same as CKF tests
- EventDataViewTests (3.8G)
* Mostly the same as CKF tests
- TrackFindingAlgorithmTrackFinderFunction (3.7G)
* Mostly the same as CKF tests
- PropagationTests (2.5G)
* Why it's using lots of RAM: Code bloat from lots of small functions
* Main culprits from time profile:
- Overuse of std::function in logging (top priority)
* Lots and losts of std::functions everywhere in test
* Precise impact hard to estimate, but omnipresence is suspicious
* Also makes rest of the profile hard to read
- Eigen geometry overheads: As in CKF, plus...
* `Transform::pretranslate` (0.3s)
* `Transform::prerotate` (0.2s)
- boost::unit_test
* Test data generator produces gigantic complex types
* Internally uses other costly stuff like std::tuple and std::bind
- Maybe use of boost/string
* Header takes a suspicious 0.9s to parse
* Some traces in the profile, but not huge
- Overuse of std::variant here and there
- Traces of interesting Eigen stuff, but hard to see due to the
omnipresent std::function overhead.
- PropagationExampleBase (2.4G)
* Mostly the same as PropagationTests + overuse of std::variant for BField
- FatrasSimulationBase (2.4G)
* Mostly the same as PropagationTests, but also...
- Use of Eagen::applyOnTheLeft (0.1s)
- Use of boost::container::flat_multiset (0.1s)
- BoundingBoxTests (2.3G)
* Why it's using lots of RAM: Code bloat from lots of small functions
* Main culprits from time profile:
- Eigen geometry implementation
* `Quaternion::setFromTwoVectors` (4.2s, top priority)
- This thing is huge, it takes 4.2s to build on its own
- Reason is that it calls JacobiSVD, and Eigen decompositions
are a huge source of code bloat.
* `Transform::rotation` (0.6s)
* `Transform::inverse` (0.4s)
* `Transform::operator*(Matrix)` (0.4s)
* `Translation::operator*(AngleAxis)` (0.3s)
* `Transform::operator*` (0.2s)
* `Transform::operator*=(AngleAxis)` (0.2s)
* `Transform::translate` (0.2s)
* `Quaternion::operator*` (0.2s)
* `Transform::operator=(AngleAxis)` (0.2s)
* `Rotation2D::operator*` (0.1s)
* `Transform::operator=(Rotation2D)` (0.1s)
- Use of Eigen::Array in AxisAlignedBoundingBox::intersect (0.1s)
- MultiTrajectoryTests (2.1G)
* Mostly a subset of CKF overhead, including
- Measurement variant overhead
- Eigen geometry overhead
- Map and Array overhead
- About 0.6s of Boost.Test overhead coming from use of BOOST_TEST macro
* Solution: Don't use BOOST_TEST, its equality semantics are a mess
- GainMatrixUpdaterTests (2.1G)
* Mostly a subset of CKF overhead, including
- Measurement variant overhead
- Eigen geometry overhead
- Matrix inversion overhead
- MaterialValidationBase (2.0G)
* Profile is very similar to PropagationTests, fixing it will likely help
- AdaptiveMultiVertexFinderTests (2.0G)
* Some overheads similar to CKF
- Eigen geometry
- Matrix::inverse
* Some overheads similar to Propagator (e.g. debugLog)
* Large contribution from `Matrix::determinant` (0.6s)
* Use of `std::regex` in VertexingDataHelpers is a bit expensive (0.5s)
- This function really doesn't need to be inline, move it to cpp file.
- MeasurementHelpersTests (1.9G)
* Mostly a subset of CKF overhead, including
- Measurement variant overhead, obviously (3.3s worth of it)
- Eigen geometry overhead
- Map and Array overhead, amplified by equality w/o eval (1.5s)
- MaterialCollectionTests (1.8G)
* Similar "big things" as in AdaptiveMultiVertexFinderTests, but no regex.
* A bit more geometry overhead
- `Translation::operator*(AngleAxis)` (0.3s)
- `Transform::operator*(AngleAxis)` (0.1s)
- IterativeVertexFinderTests (1.7G)
* Profile is quite similar to AdaptiveMultiVertexFinder tests
- GainMatrixSmootherTests (1.7G)
* Mostly a subset of CKF overhead, including
- Measurement variant overhead
- Eigen geometry overhead
- `LDLT` and `JacobiSVD` overhead
- TrackingVolume (1.6G)
* Mostly a subset of BoundingBoxTests overhead, also bound by the
`Quaternion::setFromTwoVectors` monster.
- KalmanExtrapolatorTests (1.6G)
* Mostly a subset of PropagatorTests overhead, including
- debugLog mechanism overhead
- Eigen geometry overhead
- FullBilloirVertexFitterTests (1.6G)
* Profile is quite similar to AdaptiveMultiVertexFinder tests, mostly bound
by determinants, inverse, and Eigen geometry. But no Propagator here.
- FatrasSimulationTests (1.6G)
* Mostly a subset of PropagationTests overhead
- ExtrapolatorTests (1.6G)
* Mostly a subset of PropagationTests overhead
- DirectNavigatorTests (1.6G)
* Mostly a subset of PropagationTests overhead
* A bit more geometry overhead
- `Translation::operator*(AngleAxis)` (0.3s)
- `Transform::operator*(AngleAxis)` (0.1s)
- AdaptiveMultiVertexFitterTests (1.6G)
* Similar to FullBilloirVertexFitterTests