What is CMake execute_process

Introduction

The execute_process() command in CMake provides a powerful way to execute external commands during the configuration stage, before the build system generation. This makes it ideal for tasks such as dynamically generating files, modifying the project structure, or retrieving information necessary for configuration. Unlike add_custom_command(), which runs during the build stage, execute_process() ensures immediate execution and can handle complex tasks that need to be completed before CMake generates the build system.

This post dives into the usage of execute_process(), demonstrating how to run commands, capture output and error streams, handle multiple commands, and understand its differences from add_custom_command().

Explanation

During configuration stage, as CMake walks through CMakeLists.txt files, the moment it sees an execute_process, it starts running its command(s). This happens before build system generation. The command can be as simple as

cmake_minimum_required(VERSION 3.23)
project(example LANGUAGES CXX)

execute_process(COMMAND ${CMAKE_COMMAND} -E echo hello)

If you run cmake for this CMakeLists.txt file, you notice hello is not printed in the terminal as echo is run in another process. To capture the standard output of that process we can set a variable for it:

execute_process(
    COMMAND ${CMAKE_COMMAND} -E echo hello
    OUTPUT_VARIABLE logVar1
)
message("logVar="${logVar}) # will display hello 

Here, I used ${CMake_Command} which is cmake executable in a cross-platform way, besides echo, there are many other tools like copy and make_directory, see here.. You can also use any available executable.

Run Python script

Let’s check another detailed example. File code.py has standard output and standard error:

import sys

print("Python Script: Message for standard output")
print("Python Script: Message for standard error!", file=sys. stderr)

# Uncomment for failed exit code
#sys.exit(1) 

We want to run it during CMake configuration, so we have this in our CMakeLists.txt:

execute_process(
  COMMAND python3 code.py
  WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}
#  OUTPUT_FILE log.txt        # capture stdout in a file
  OUTPUT_VARIABLE logVar      # Capture stdout in a variable
  ERROR_VARIABLE errorVar     # Capture stderr in a variable
  RESULT_VARIABLE resultVar   # Capture exit code or error string 
  COMMAND_ERROR_IS_FATAL ANY  # stop CMake if failed
)

message("logVar="${logVar})
message("errorVar="${errorVar})
message("resultVar="${resultVar})

Let’s explain each line:

• COMMAND : runs the python script
• WORKING_DIRECTORY: where the command is run, here, the directory of code.py
• OUTPUT_FILE: if uncomment this, the stdout is written in log.txt file
• OUTPUT_VARIABLE: writes stdout into logVar. Choose this or OUTPUT_FILE not both.
• ERROR_VARIABLE: to store stderr into errorVar
• RESULT_VARIABLE: captures the exit code or an error string in resultVar. If 0, the command is successful.
• COMMAND_ERROR_IS_FATAL: if any command fails, stop CMake.

Instead of COMMAND_ERROR_IS_FATAL, we can also use result variable as:

if(resultVar AND NOT resultVar EQUAL 0)
    message(FATAL_ERROR "Failed: ${resultVar}")
endif()

To see this behavior, in code.py uncomment line #sys.exit(1).

Multiple commands

To run different commands in a sequential order use different multiple execute_process() where each contains one command. When CMake finishes the first execute_process(), it moves to the next one.

You can have multiple commands in one execute_process() to run concurrently. However, it is only to mimic pipe in linux terminal, where standard output of the first command is the standard input of the second, and the standard output of the second is passed to the third, and so on. Therefore, if pipe is not your goal (probably in most cases), stick to one command per execute_process().

One easy approach to run complex commands, sequential or parallel, in a cross-platform way is to write a Python script and use one execute_process(). Python os and sys libraries have many useful commands which work on different platforms. For example, python script, code2.py that writes its two arguments is:

import sys
print("Arg1=",sys.argv[1])
print("Arg2=",sys.argv[2])

We can run it as

set(var1 "First")
set(var2 "Second")

execute_process(
  COMMAND python3 code2.py ${var1} ${var2}
  WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}
  OUTPUT_VARIABLE logVar      # Capture stdout in a variable
)

message(logVar=${logVar})

The output is

logVar=Arg1= First
Arg2= Second

If you are only interested in windows batch file, you can use cmd as

execute_process(
  COMMAND cmd /C myscript.bat
)

for Linux script:

execute_process(
    COMMAND bash mapping_string.sh 
)

execute_process() vs add_custom_command()

I explained add_custom_command in this post. The comparison is:

add_custom_command:

• Run stage: build stage.
• Dependency control: can be dependent to a file. It is only run if a target depends on it.
• Parallel run: Can make use of parallel generators like ninja at build time.
• Run frequency: Only run if it is requested by a target and its dependency file is out of date.

execute_process:

• Run stage: configuration stage.
• Dependency control: No.
• Parallel run: Possible but a headache.
• Run frequency: Runs everytime CMake configuration stage is run. It is not run at all during build.

Therefore, I recommand using add_Custom_command() as much as possible. However, there are actions that must be done during configuration for instance changing source directory structure, then we use execute_process(). An example is copying source directory of an external library like Google Test into a subdirectory of our project. After that CMake configuration makes the external library become a subdirectory and its targets will be available in our project:

execute_process(
  COMMAND ${CMAKE_COMMAND} -E copy_directory C:/from/somewhere/googletest ./googletest
  WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}
)

add_subdirectory(googletest)

Note that, the example above is to show the place of execute_process(), if you want to include an external project, the systematic way is to use FetchContent_Declare().

Comments

0 comment