When working on a software library which is targeting 3rd-party consumers, it is rather of self-interest to make the library uncomplicated to use. Unless perhaps there is opportunity seen in having to help out in the process And to know if it is uncomplicated to use, best before learning otherwise the hard way, one has to take the position of the consumers. Even better if this can be automated, after all that is why people developed machines, like computers, to pass on the dull part.Many things of a library can be tested during the build, based on the sources and build artifacts. Like functionality of methods, proper export/visibility of their public entry points or the code any special pre-processor macros are resolved to. Here CMake’s VERIFY_INTERFACE_HEADER_SETS also offers a solution to verify that headers can be included on their own, though being bound to the build interface of the library.Now, the consumer will only see the installed files, and there are for one things that can go wrong when installing even already tested files, as are there files which only work in the installation location or might work differently there, so can be only properly tested there.Public Header FailsWhen about to use a feature of a library, one would see to apply the documented needed include statement. E.g. when writing code dealing with the class KContacts::Picture of the KDE Frameworks module KContacts, the docs hint to use the following include to have any needed declarations available:#include So what could go wrong with such header file (e.g. Picture in the example above):the header file is not installedthe header file is not placed in the relative location as by the documented include statement and the official include search directoriesthe header file includes another header from the same library that is not installed (e.g. for test utils)the header file includes another header from the same library which is placed in a relative location not resolvable by the official include search directoriesthe header file includes another header from a private dependency, so not resolvable by the inherited official include search directoriesthe header file uses language features unconditionally of a standard not enforced in the public build flagsthe header file relies on macro definitions not provided correctly, either missing or with wrong valuethe header file is not self-contained with the variants of the headers part of the installation (violating C++ Core Guidelines, SF.11: Header files should be self-contained)(… possibly more, please mention in comments)All these can be mainly checked by simply creating a C++ source file which only includes the very header file, by the documented include statement, and testing for its successful compilation/code parsing, applying the official include search directories and build flags (and the inherited ones of public dependencies).CMake Config FailsWhen about to use a library at all, one would see to use the meta data blobs which also integrate with one’s (meta) build system. E.g. in case of using CMake and as in the example above KF module KContacts, one would for setting up building against the library do as documented, relying on the CMake config files provided:find_package(KF6 REQUIRED COMPONENTS Contacts)target_link_libraries(MySoftwareProject PRIVATE KF6::Contacts)So what could go wrong with those CMake config files:the CMake config files are not installedthe CMake config files are not installed by the correct name and in the correct locationthe library target is not provided by the CMake config files at allthe library target is not provided by the documented name and the optional namespacethe library target does not have the correct include search directories setthe library target does not have the correct build flags setthe library target misses some documented propertiesthe library target misses a dependency in the public link interface, to inherit as needed include search directories and build flagsthe library target has an item in the public link interface not checked for by the CMake config filesother documented utility targets or variables are not defined or have wrong values(… possibly more, please mention in comments)Most of these issues can be simply checked by creating a CMake project which searches for the library and links some dummy software to it, and testing for its successful configuration and build.Other FailsThere are further library-related artifacts which should see checks after installation.Any resources used from a library which are installed as separate files, like translation catalogs, images, sounds, or domain-specific data, could again be missed to be installed, installed by a wrong name or into the wrong location, or otherwise be badly processed before or during the installation process. Those are not specific to libraries though, thus not looked at more here.Other artifacts are some which themselves are to be built against the library, such as project templates (like the KAppTemplate ones, e.g. for plugins) or examples for using the library. Those are not critical to the direct usage of a library though, thus also not looked at more here.Continuous Integration As Limited Safe GuardSoftware libraries are usually not developed in a void, but based on needs by other software. Changes in the public interface are made together with the respective changes in at least one consumer. So many issues would be detected in the process.Then the library and many of its consumers, not just the ones a changes was made for, might be part of the same CI setup, where all these consumers would see at least some regular rebuild against the latest version of a library and thus would catch further regressions before the next release of the library.But not all API of a library might be used by those known consumers, instead only by 3rd-party without any integration into the library development. And some issues of API while used can be shadowed by the usage pattern. So for a more complete check some generic own solution is needed.Some Automation: ECMInstalledLibraryCheckCombining the simple check ideas for headers and CMake config files above, some addition to KDE’s Extra CMake Modules is proposed in this merge request: ECMInstalledLibraryCheck. That module provides a macro to add checks for the installed library artifacts, from consumer POV. Currently it covers the public headers (includability) & CMake config files (proper data delivery needed for compiling). It adds one global target all_installed_library_check, to run the checks for all libraries, and a target per library, _installed_library_check. Those targets would be used after installation only, obviously.For the example used above, KF’s KContacts library, the check would be deployed like this:# use the module from ECMinclude(ECMInstalledLibraryCheck)# for the library target KF6Contacts set up the checkecm_add_installed_library_check(KF6Contacts PACKAGE_NAME "KF6Contacts" PACKAGE_VERSION ${KCONTACTS_VERSION} PACKAGE_TARGET_NAMESPACE "KF6::")# register the official include statements for the checkecm_installed_library_check_include_strings(KF6Contacts HEADERS ${KContacts_CamelCase_HEADERS} PREFIX KContacts)Given the library target KF6Contacts thus this adds the target named KF6Contacts_installed_library_check. This target generates and builds a separate CMake-based project for a dummy library linking against the installed library, whose sources are one source file per passed include string, with a single code line #include . The check is considered passed if the builds completes without an error.Additionally the checks can be extended to also cover CMake variables or preprocessor defines, by functions like ecm_installed_library_check_cmake_variable(), ecm_installed_library_check_compile_definition() and ecm_installed_library_check_preprocessor_macro(). The last also has some convenience variant ecm_installed_library_check_version_preprocessor_macros(), in our example deployed like this:# check all includes also have the usual version macros defined afterwards, only report errorsecm_installed_library_check_version_preprocessor_macros(KF6Contacts PREFIX KCONTACTS VERSION ${KCONTACTS_VERSION} SILENT)The current state of the prototype has already found a number of existing issues in a number of libraries, which so far had not yet been uncovered by consumers’ usage pattern, and first fixes have been rolled out to prevent consumers hitting them in the future. It also has exposed a principal issues with projects like some KF modules that provide multiple libraries, yet only a single CMake config file for their module. Such file would need to ensure the combined dependencies of all libraries, which then adds unneeded burden to consumers of just one library. Currently the dependency checks in KF CMake config files are rather pragmatically added, when existing consumers stumble over something missing.Hurdles To Adopting The Check AutomationChecks can only return value for the cost of integrating them if they are also routinely applied and their results considered. Worse, if not used they run chance of bit-rot and actually clutter a project.Typically a library has a suite of auto-tests, covering all the features. While developers are encouraged to use them on their local system during development, they are at least used on the central CI system, against as many platforms and configurations as possible.A first challenge with merging post-installation checks to the other checks and tests is that auto-tests are usually designed or at least prepared to be run pre-installation. KDE has had some “Running apps uninstalled” initiative, which resulted at least for auto-tests of KDE Frameworks to be also run on CI before installation. CMake only has the concept of one suite of tests, without an idea of pre-installation and post-installation test times. So one would have e.g. to create some custom system on top, e.g. by using CMake/CTest test labels. And for CI do extra work to run and process the two sets of tests separately.Another challenge is that like all tests the ECMInstalledLibraryCheck checks come at a cost. While their creation is quickly done (generating a buildsystem file), the execution is taking some time. Because it mainly consists of a full compilation per each tested include statement, which despite just being a simple include line still results in lot of processing due to all the implicit overhead, even more when transitively including many template-ridden headers. And being a new system with some people the cost will scale the normal “don’t need this, never done that, no changes wanted” reaction.Collaboration WantedThe author is currently done when it comes to own immediate desires for the features of ECMInstalledLibraryCheck and uses it in personal projects as is. It has been though developed in the (KDE) public with the intent to share it, to have it be useful to others like KDE projects and gain from common usage and feedback.Not being a CI system person, the author now is mainly looking for someone who can work on integrating the execution of ECMInstalledLibraryCheck checks with KDE CI (or also their own projects’ CI). The idea would be that the check of the installed library artifacts is as useful as any unit test, to catch issues with new code and regressions immediately, so should be run along the normal unit tests, just in its own post-installation stage. This stage could also later see the addition of post-install checks for templates and examples as well as other things which are found useful to test post-install. Ideally the check result would be integrated into the test result display, but for a start just successfully completing the build target should be already useful as indication. Possibly projects would opt in to the test using some configuration flag, or in case of KF could default to it. Perhaps CI integration will also require certain things from the macros, so ideally the CI deployment can be sorted out before ECMInstalledLibraryCheck gets merged and released. So if you find this kind of check useful and are experienced with (KDE) CI setup, please get in contact. Then post-installation check of library artifacts might have further constraints or desires with some. Would people need doing parallel test of the installed library with different configurations or usage flags? Would people like to use this in cross-compilation (so far no generated code is executed, so in theory should be usable) and does that need further support? Are there other aspects of the installed artifacts you would like to see covered? Please give the current state of the check module a try for your own library projects and get in contact.Try ECMInstalledLibraryCheck: Does Your Library Pass?The file ECMInstalledLibraryCheck.cmake should only depend on CMake, so can be just copied into your own project:download the raw file from https://invent.kde.org/frameworks/extra-cmake-modules/-/blob/work/kossebau/ECMInstalledLibraryCheck/modules/ECMInstalledLibraryCheck.cmakecreate subdir cmake/ecmcopy in your project, place ECMInstalledLibraryCheck.cmake insidemake subdir known to CMake: list(APPEND CMAKE_MODULE_PATH ${CMAKE_SOURCE_DIR}/cmake/ecmcopy)include module: include(ECMInstalledLibraryCheck)setup check for your library targets, see docs at begin of module file how to do thatrun the installation target, then the target all_installed_library_checksee how per library a project “InstalledLibraryCheck” is generated and build, the build log serves as success reportif the check build fails, try to fix it, install (easy to forget ) and run the check target againif curious, peek into the build directory and look at the generated file ${library_target}_ECMInstalledLibraryCheck/CMakeLists.txt, it holds the complete check and can be also played directly withThe author has done this with many projects already, in KDE for all of KF modules as well as some graphics, games, multimedia, utility and education software libraries.Now go and check your own library, and report back how useful ECMInstalledLibraryCheck is for you, what issues it discovered and what your further needs are here