• NestedFunctions

Nested functions

Nested functions are functions within another function, for example:

int
foo (const char *s)
{
  auto int iterator (const char *filename);
  int iterator (const char *filename)
  {
    grub_printf ("%s: %s\n", s, filename);
  }
 
  grub_iterate_foo_files (iterator);
}

In the example above the nested function is passed as a function pointer to another function AND it uses variables only available in the scope of its parent. In this case and in this case only trampolines are used.

Executable stack

When trampolines are used, an executable stack is required. When an executable stack is not available this code does not execute.

Problems when the stack is not executable

Some operating systems and GNU/Linux distributions don't allow executable stacks for security reasons. Sometimes binaries do not have an executable stack by default, but the executable stack can be enabled again by tagging the binary. On some other operating systems a function is available to make the stack executable again; this function can be called by the program or automatically by libgcc when it is required.

Some GNU/Linux distributions not consider re-enabling execution on the stack as an option because of their policies.

The GCC developers don't want to remove nested functions just because of this. Perhaps this it is possible to implement trampolines in another way.

This problem is filed in GCC Bugzilla Bug 27702.

GRUB

For GRUB we don't want to remove nested functions. It's a well documented feature and we like it a lot. We also don't have the same policies as redhat has. But if there is a way to make it possible not to require an executable stack without changing the existing code and without many changes, we should do so.

Solutions to the current problem

We have to deal with non-executable stacks. There are several solutions:

• Making the stack executable again.
• Adding a new stack which is executable.
• Change GCC. GCC has an ENABLE_EXECUTE_STACK macro which is used on some operating systems, but not Linux (?).
• Don't use function pointers to nested functions that use variables with a local scope.
• Remove the nested functions.

The last option is nota real opion. It removes an useful feature to fix things. In that case the second last option is better because in some cases nested functions can still be used, while the problem is fixed. Even that option should be evaded if possible because it means some code has to be changed, this will in some cases result in very ugly code. To me (MarcoGerards) the second option seems the best for now.

Apple

Apple made some changes to GCC so their GCC does not match the one distributed by the GNU project. They completely removed support for nested functions, regardless of if it is required or not (not in every case, as described above, trampolines are used). Therefore we unfortunately can not support compiling GRUB on the compiler distributed by Apple.

In 10.4, both gcc-4 and gcc-3.3 support nested functions.

CategoryDeveloper