Trampolines for pointers to internal procedures.

Overview

subroutine host()
  integer :: local
  local = 10
  call internal()
  return

  contains
  subroutine internal()
    print *, local
  end subroutine internal
end subroutine host

Procedure code generated for subprogram internal() must have access to the scope of its host procedure, e.g. to access local variable. Flang achieves this by passing an extra argument to internal() that is a tuple of references to all variables used via host association inside internal(). We will call this extra argument a static chain link.

Fortran standard 2008 allowed using internal procedures as actual arguments for procedure pointer targets:

Fortran 2008 contains several extensions to Fortran 2003; some of these are listed below.

• An internal procedure can be used as an actual argument or procedure pointer target.

NOTE 12.18

An internal procedure cannot be invoked using a procedure pointer from either Fortran or C after the host instance completes execution, because the pointer is then undefined. While the host instance is active, however, the internal procedure may be invoked from outside of the host procedure scoping unit if that internal procedure was passed as an actual argument or is the target of a procedure pointer.

Special handling is required for the internal procedures that might be invoked via an argument association or via pointer. This document describes Flang implementation to support it.

NOTE: in some languages/extensions the static chain may contain links to more than one stack frame, while Fortra’s static chain only ever has a link to a single host procedure.

Flang current implementation

Examples

Internal procedure as procedure pointer target:

module other
  abstract interface
     function callback()
       integer :: callback
     end function callback
  end interface
  contains
  subroutine foo(fptr)
    procedure(callback), pointer :: fptr
    ! `fptr` is pointing to `callee`, which needs the static chain link.
    print *, fptr()
  end subroutine foo
end module other

subroutine host(local)
  use other
  integer :: local
  procedure(callback), pointer :: fptr
  fptr => callee
  call foo(fptr)
  return

  contains

  function callee()
    integer :: callee
    callee = local
  end function callee
end subroutine host

program main
  call host(10)
end program main

Internal procedure as actual argument (F90 style):

module other
  contains
  subroutine foo(fptr)
    interface
      integer function fptr()
      end function
    end interface
    ! `fptr` is pointing to `callee`, which needs the static chain link.
    print *, fptr()
  end subroutine foo
end module other

subroutine host(local)
  use other
  integer :: local
  call foo(callee)
  return

  contains

  function callee()
    integer :: callee
    callee = local
  end function callee
end subroutine host

program main
  call host(10)
end program main

Internal procedure as actual argument (F77 style):

module other
  contains
  subroutine foo(fptr)
    integer :: fptr
    ! `fptr` is pointing to `callee`, which needs the static chain link.
    print *, fptr()
  end subroutine foo
end module other

subroutine host(local)
  use other
  integer :: local
  call foo(callee)
  return

  contains

  function callee()
    integer :: callee
    callee = local
  end function callee
end subroutine host

program main
  call host(10)
end program main

In all cases, the call sequence implementing fptr() call site inside foo() must pass the stack chain link to the actual function callee().

Usage of trampolines in Flang

BoxedProcedure pass recognizes fir.emboxproc operations that embox a subroutine address together with the static chain link, and transforms them into a sequence of operations that replace the result of fir.emboxproc with an address of a trampoline. Eventually, it is the address of the trampoline that is passed as an actual argument to foo().

The trampoline has the following structure:

callee_trampoline:
  MOV static-chain-address, R#
  JMP callee-address

Where:

• callee-address is the address of function callee().

• static-chain-address - the address of the static chain object created inside host().

• R# is a target specific register.

With the default stack-trampoline implementation, the replacement in the MLIR LLVM dialect looks like this:

    llvm.call @llvm.init.trampoline(%8, %9, %7) : (!llvm.ptr<i8>, !llvm.ptr<i8>, !llvm.ptr<i8>) -> ()
    %10 = llvm.call @llvm.adjust.trampoline(%8) : (!llvm.ptr<i8>) -> !llvm.ptr<i8>
    %11 = llvm.bitcast %10 : !llvm.ptr<i8> to !llvm.ptr<func<void ()>>
    llvm.call @_QMotherPfoo(%11) {fastmathFlags = #llvm.fastmath<fast>} : (!llvm.ptr<func<void ()>>) -> ()

When -fsafe-trampoline is enabled on a supported target, the pass instead uses the runtime API described below.

So any call of fptr inside foo() will result in invocation of the trampoline. The trampoline will setup R# register and jump to callee() directly.

The ABI of callee() is adjusted using llvm.nest call argument attribute, so that the target code generator assumes the static chain argument is passed to callee() in R#:

  llvm.func @_QFhostPcallee(%arg0: !llvm.ptr<struct<(ptr<i32>)>> {fir.host_assoc, llvm.nest}) -> i32 attributes {fir.internal_proc} {

Default stack-trampoline handling

The default path uses the llvm.init.trampoline intrinsic, which expects that the memory for the trampoline content is passed to it as the first argument. The memory has to be writeable at the point of the intrinsic call, and it has to be executable at any point where callee() might be ivoked via the trampoline.

@llvm.init.trampoline intrinsic initializes the trampoline area in a target-specific manner so that being executed: the trampoline sets a target-specific register to be equal to the third argument (which is a static chain address), and then calls the function defined by the second argument.

Some targets may perform additional actions to guarantee the readiness of the trampoline for execution, e.g. call __clear_cache or do something else.

For each internal procedure a trampoline may be initialized once per the host invocation.

The target-specific address of the new trampoline function must be taken via another intrinsic call:

%p = call i8* @llvm.adjust.trampoline(i8* %trampoline_address)

Note that value of %p is equal to %tramp1 in most cases, but this is not a requirement - this is partly why the second intrinsic was introduced:

By the way an example of adjust_trampoline is ARM, which or's a 1 into the address of the trampoline.  When the pointer is called the processor sees the 1 and puts itself into thumb mode.

By default, the trampolines are allocated on the stack of host() subroutine, so that they are available throughout the life span of host() and are automatically deallocated at the end of host() invocation. Unfortunately, this requires the program stack to be writeable and executable at the same time, which might be a security concern.

NOTE: LLVM’s AArch64 backend supports nest attribute, but it requires the compiler-rt runtime selected via the -rtlib=compiler-rt flag.

Opt-in runtime trampoline pool

On x86-64 and AArch64 targets, -fsafe-trampoline selects a Flang runtime implementation instead of the LLVM stack-trampoline intrinsics. In this implementation, the runtime manages a global, fixed-capacity pool. Trampoline code is generated in a separate region that is executable but not writeable after initialization, while per-slot data remains writeable but not executable:

trampoline0:
  MOV (TDATA[0].static_chain_address), R#
  JMP (TDATA[0].callee_address)
trampoline1:
  MOV (TDATA[1].static_chain_address), R#
  JMP (TDATA[1].callee_address)
...

Each TDATA entry stores the callee and static-chain addresses for one trampoline. The generated stub loads both values, places the static-chain address in the target-specific register, and jumps to the callee.

Implementation characteristics

• The pool contains 1024 slots by default. FLANG_TRAMPOLINE_POOL_SIZE can set a different capacity, as described in the runtime environment documentation.

• Allocation from a full pool terminates the program with a diagnostic. The pool does not grow dynamically.

• TrampolineFree returns a slot to the synchronized global pool for reuse. The implementation does not use a dynamic trampoline area per thread.

• Each trampoline invocation loads the static chain and callee addresses from its paired data entry.

Fortran runtime API

The BoxedProcedure pass uses these runtime APIs:

/**
 * \brief Initializes a new trampoline and returns its internal handle.
 *
 * Initializes a new trampoline with the given \p callee_address
 * and \p static_chain_address, and returns the trampoline's
 * internal handle. The compiler calls this method once per host
 * invocation for each internal procedure that will need its address
 * passed around.
 *
 * \p scratch is reserved and currently ignored. The lowering passes
 * a null pointer; this argument does not select the default
 * stack-trampoline implementation.
 */
void *TrampolineInit(void *scratch, const void *callee_address,
                     const void *static_chain_address);

/**
 * \brief Returns the trampoline's address for the given handle.
 *
 * \p handle is a value returned by TrampolineInit().
 * The result of TrampolineAdjust() is the actual callable
 * trampoline's address.
 */
void *TrampolineAdjust(void *handle);

/**
 * \brief Frees internal resources occupied for the given trampoline.
 *
 * The compiler must call this API at every exit from the host function.
 */
void TrampolineFree(void *handle);

TrampolineInit initializes the pool on first use, reserves an available slot, stores the callee and static chain addresses in its data entry, and returns an opaque handle. TrampolineAdjust returns the executable code address for that slot. TrampolineFree invalidates the data entry and returns the slot to the pool.

Sample IR

    // Init the trampoline once per host procedure invocation
    // (i.e. when the procedure address is emboxed).
    %handle = llvm.call @_FortranATrampolineInit(%nullptr, %9, %7) : (!llvm.ptr<i8>, !llvm.ptr<i8>, !llvm.ptr<i8>) -> !llvm.ptr<i8>
    // Get the actual internal procedure address once per host procedure invocation.
    %10 = llvm.call @_FortranATrampolineAdjust(%handle) : (!llvm.ptr<i8>) -> !llvm.ptr<i8>
    %11 = llvm.bitcast %10 : !llvm.ptr<i8> to !llvm.ptr<func<void ()>>
    llvm.call @_QMotherPfoo(%11) {fastmathFlags = #llvm.fastmath<fast>} : (!llvm.ptr<func<void ()>>) -> ()
    // The trampoline deallocation must be done only at the exits from the host procedure.
    llvm.call @_FortranATrampolineFree(%handle) : (!llvm.ptr<i8>) -> ()

The current implementation is self-contained in the Flang runtime. Because it needs to support only the Fortran/C interoperable calling convention, the implementation may reduce trampoline overhead by clobbering ABI-permitted scratch registers rather than saving and restoring them.

Implementations that were considered

Alternative implementations were considered, but not pursued:

• Reusing the libffi implementation for static trampolines.

• Extracting the static-trampoline implementation from libffi into a separate library (e.g. libstatictramp, as mentioned here).