Internal IR for compiler args by dcbaker · Pull Request #14748 · mesonbuild/meson

dcbaker · 2025-06-25T23:03:31Z

This is set to draft because it's very much a Work in Progress.

The way we handle compiler arguments in Meson is to manipulate raw strings. This works fine when the majority of your compilers are the same, or at least attempt to be compatible. Unfortunately, Meson has to deal with many different toolchains, which often have incompatible argument syntax.

This manifests in a number of different ways:

It makes de-duplicating or removing arguments difficult. Consider the high complexity of the CompileArgs class
It makes checking for a kind of argument difficult. Ie, we have a check for -Wl,-rlink or -rlink inside of build.py, but that could miss -rlink for many compilers.
It means when converting a compiler argument from one language to another, like when proxying linker arguments, we need to determine the format the argument is already in and then attempt to convert.
It makes it more difficult to set fields in the XCode and VisualStudio backends, which generally do not expect features to be set by compiler arguments, but by setting specific values. It is very easy to see this in the VS backend code, which has long if blocks converting string command line arguments to XML fields

In order to address this I propose moving to an abstract IR for these arguments. This abstract IR would simplify de-duplicating and cleaning arguments, as well as converting their forms. Each Compiler provides a mechanism to convert arguments from their format into the abstract IR, and a method to convert the abstract IR back into concrete arguments in their expected format. We then manipulate the abstract IR more easily. Consider something like this:

[Define('X'), Opaque('-ffoobar'), Warning('foo'), Warning('foo'), Warning('foo', enable=False), Define('X')]

In this format it's pretty easy to look and see that we have two copies of -DX and two copies of -Wfoo along with -Wno-foo, cancelling both of those out.

Below is a very trivial implementation of this idea, with just enough implemented for GCC or Clang to compile the 1 trivial test (run directly). This also is making use of python features we currently cant rely on, I'll rewrite those later.

Known TODO items:

Handle arguments that can be split but not required to be split (such as -D and -U with GCC/Clang)
Handle arguments that must be split (like --cfg with Rust)

dnicolodi · 2025-06-27T09:44:37Z

FWIW, I like this!

dnicolodi · 2025-06-27T09:45:22Z

mesonbuild/arguments.py

+    :param enable: If true then enable the warning, otherwise suppress it.
+    """
+
+    target: str


I think an enable member went missing here.

Oops, I the enable docstring is a copy-paste error. There is not -Wno-error=... in any compiler AFAIK, so no need to represent it.

s/any/every/ ?

So I guess yeah, if it can't be universally represented it might as well be opaque whenever it appears.

Originally I had Warning() with both enable and error fields, but realized that you could end up with the invalid state of error disabled. I guess we could leave it and have some kind of validation pass later, but I prefer to not be able to model invalid state if possible.

-Wno-error is valid at least on gcc and clang. Given f.c:

int main() { puts("aa"); }

you have:

$ gcc f.c -Wno-error=implicit-function-declaration -std=c99 f.c: In function ‘main’: f.c:3:9: warning: implicit declaration of function ‘puts’ [-Wimplicit-function-declaration] $ clang f.c -Wno-error=implicit-function-declaration -std=c99 f.c:3:2: warning: call to undeclared function 'puts'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]

Without the flag you get an error with both compilers.

Interesting, GCC does not document that on their warnings page, just -W, -Wnoand-Werror=`

Since it does exist I'll fold it.

?

-Werror= Make the specified warning into an error. The specifier for a warning is appended; for example -Werror=switch turns the warnings controlled by -Wswitch into errors. This switch takes a negative form, to be used to negate -Werror for specific warnings; for example -Wno-error=switch makes -Wswitch warnings not be errors, even when -Werror is in effect.

Maybe you could have an enum DISABLED, DEFAULT_LEVEL, WARNING, ERROR corresponding to -Wno-, -W, -Wno-error=, -Werror=.

Ah, I was looking at the htmlized version of the info page, It's probably there but not readily obvious

mesonbuild/compilers/mixins/gnu.py

Which I've somehow failed to add all of these years.

The goal is to be able to convert arguments from raw arguments that are passed in via the `c_args` and friends methods, as well as from options and the environment, into an abstract IR. This IR is then passed around internally, and the backend is responsible for lowering it into the format that it wants. I'm hoping that this will simplify some amount of passing arguments between languages (like C arguments being passed to non-c compilers, and linker arguments between languages) as well as simplify the non-ninja backends which generally use a different mechanism than raw compiler arguments.

…arguments These will be used to convert the arguments in many cases.

This is only for dynamic linkers.

dcbaker · 2025-07-07T20:13:30Z

@dnicolodi thanks for the reviews! I've fixed the things you pointed out.

Also in the latest update, linker arguments are now handled, along with a this the check for manually specified -rpath arguments has gotten better, as it no longer needs to check for strings, but can instead just see if there are any instances of Rpath() in the list.

There's still a lot of plumbing work to do here, apart from implementing all of this for various compilers.

eli-schwartz · 2025-07-07T21:31:35Z

mesonbuild/compilers/mixins/gnu.py

+            elif arg.startswith('-U'):
+                ret.append(arguments.Undefine(arg.removeprefix('-U')))
+            elif arg.startswith('-l'):
+                ret.append(arguments.LinkerSearch(arg.removeprefix('-l')))


eli-schwartz · 2025-07-07T21:34:24Z

mesonbuild/arguments.py

+
+
+@dataclasses.dataclass
+class Undefine:


For warnings we have enable=True, why not the same for Define?

In fact since (EDIT: fixed) -Dfoo and -Dfoo=1 are the same, could Undefine('foo') be replaced by Define('foo', value=None)?

They are NOT the same, one sets it to implicitly 1 and the other, explicitly the null string.

For both, ifdef says "yes it is defined" and redefining may result in compiler warnings.

Yeah, I really want to make it impossible to represent invalid state. So we basically have three distinct cases:
set define to value
set define to nothing
remove define

So... would representing it as:

class Define: name: str value: str | bool

work correctly?

You'd have value = False mean -Ufoo value = True mean -Dfoo and value = mean -Dfoo=<str>?

You're right, sorry. I meant that -Dfoo and -Dfoo=1 could be canonicalized to a single form (Define('foo', '1') in the abstract form, whereas the concrete form could be either) and value=None could be used for undefining.

Maybe, but would that mean the user sets -Dfoo=1 and we translate to -Dfoo (or vice versa)?

Yes... OTOH you do want to collapse -Dfoo -Dfoo=1 to just one of them. Setting value='1' would also make __eq__ easier to implement.

Maybe some kind of have_equals: bool could be added to make round-trip conversion possible, and it would be ignored by __eq__.

I don't know that we have to guarantee we pass exactly the same arguments as long as we pass equivalent arguments. Especially since we already attempt to do some level of de-duplication. I'm perfectly fine with -Dfoo= becoming -Dfoo=1.

dnicolodi · 2025-07-18T23:13:07Z

mesonbuild/compilers/mixins/gnu.py

+                    k, v = arg.split('=')
+                else:
+                    k, v = arg, None
+                ret.append(arguments.Define(k, v))


Defines can be spelled on the command line as -D KEY=value (with the space after the -D switch) meaning that they are seen as two arguments, not as one. Thus, I think that arguments parsing needs to be more sophisticated. Maybe using something like argparse may make the code simpler.

It's possible we will have to do something more sophisticated before this can land, right now my biggest concern is getting to the point that i can prove that the IR can do everything we currently do with the POSIX style string parsing, and solve some of the difficult problems we can't solve easily. So, for the moment I'd rather not get hung up here, but I will leave these opened to keep them in mind.

This is also an issue we'll need to resolve for Rust because --cfgs must be passed with a space in them.

dnicolodi · 2025-07-18T23:13:19Z

mesonbuild/compilers/mixins/gnu.py

+                    k, v = arg, None
+                ret.append(arguments.Define(k, v))
+            elif arg.startswith('-U'):
+                ret.append(arguments.Undefine(arg.removeprefix('-U')))


The same is true for -U.

bonzini · 2026-02-23T14:10:34Z

#15577 should be revisited once this work is complete.

dcbaker added the enhancement label Jun 25, 2025

dnicolodi reviewed Jun 27, 2025

View reviewed changes

mesonbuild/compilers/mixins/gnu.py Outdated Show resolved Hide resolved

dnicolodi reviewed Jun 27, 2025

View reviewed changes

mesonbuild/compilers/mixins/gnu.py Outdated Show resolved Hide resolved

dcbaker added 8 commits July 7, 2025 13:02

build: Add Intel copyright

7f519f1

Which I've somehow failed to add all of these years.

WIP: compilers: Add methods to convert between abstract and concrete …

e38e4fb

…arguments These will be used to convert the arguments in many cases.

WIP: compilers: implement arguments methods for GNU-like compilers

d03773a

WIP: linkers: Add interfaces for converting arguments

f5c35c7

This is only for dynamic linkers.

WIP: linkers: Add basic argument conversion for Posix Linkers

8243a7c

WIP: build: use arguments instead of raw strings

45aff63

WIP: backend/ninja: use Arguments

fd3bc13

dcbaker force-pushed the submit/compile-args-abstraction branch from bdccf01 to fd3bc13 Compare July 7, 2025 20:08

eli-schwartz reviewed Jul 7, 2025

View reviewed changes

dnicolodi reviewed Jul 18, 2025

View reviewed changes

jpakkane force-pushed the master branch from 00d6953 to 6447832 Compare August 19, 2025 20:51

Uh oh!

Conversation

dcbaker commented Jun 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dnicolodi commented Jun 27, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

eli-schwartz Jul 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dcbaker Jul 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dcbaker Jul 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

dcbaker commented Jul 7, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bonzini Jul 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bonzini Jul 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bonzini commented Feb 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

dcbaker commented Jun 25, 2025 •

edited

Loading

eli-schwartz Jul 7, 2025 •

edited

Loading

dcbaker Jul 7, 2025 •

edited

Loading

dcbaker Jul 7, 2025 •

edited

Loading

bonzini Jul 7, 2025 •

edited

Loading

bonzini Jul 7, 2025 •

edited

Loading