This patch refactors the `FrontendAction` class. It merely moves code
around so that re-using it is easier. No new functionality is
introduced.
1. Three new member methods are introduced: `RunPrescan`, `RunParse`,
`RunSemanticChecks`.
2. The following free functions are re-implemented as member methods:
* `reportFatalSemanticErrors`
* `reportFatalScanningErrors`
* `reportFatalParsingErrors`
* `reportFatalErrors`
`reportFatalSemanticErrors` is updated to resemble the other error
reporting functions and to make the API more consistent.
3. The `BeginSourceFileAction` methods are simplified and the unused
input argument is deleted.
Differential Revision: https://reviews.llvm.org/D108130
`CompilerInstance` is a more appropriate place for a key component of
the frontend like `Semantics`.
This change opens a path for us to introduce new frontend actions that
will also run semantics, but for which inheriting from
`PrescanAndSemaAction` wouldn't make much sense. For example, for
code-gen actions we plan to introduce a dedicate hierarchy of action
classes.
I've also added a doxyment for `CompilerInstance` to add a bit of
context for this change (and also make future refactoring more informed).
As `CompilerInstance` in Flang has been inspired by its counterpart in
Clang, this comment is roughly a verbatim copy of the comment in Clang
(with some adjustments from me). Credits to Daniel Dunbar for the great
design and the original comment.
Differential Revision: https://reviews.llvm.org/D108035
Introducing a plugin API and a simple HelloWorld Plugin example.
This patch adds the `-load` and `-plugin` flags to frontend driver and
the code around using custom frontend actions from within a plugin
shared library object.
It also adds to the Driver-help test to check the help option with the
updated driver flags.
Additionally, the patch creates a plugin-example test to check the
HelloWorld plugin example runs correctly. As part of this, a new CMake
flag (`FLANG_BUILD_EXAMPLES`) is added to allow the example to be built
and for the test to run.
This Plugin API has only been tested on Linux.
Reviewed By: awarzynski
Differential Revision: https://reviews.llvm.org/D106137
Rename the current -E option to "-E -Xflang -fno-reformat".
Add a new Parsing::EmitPreprocessedSource() routine to convert the
cooked character stream output of the prescanner back to something
more closely resembling output from a traditional preprocessor;
call this new routine when -E appears.
The new -E output is suitable for use as fixed form Fortran source to
compilation by (one hopes) any Fortran compiler. If the original
top-level source file had been free form source, the output will be
suitable for use as free form source as well; otherwise there may be
diagnostics about missing spaces if they were indeed absent in the
original fixed form source.
Unless the -P option appears, #line directives are interspersed
with the output (but be advised, f18 will ignore these if presented
with them in a later compilation).
An effort has been made to preserve original alphabetic character case
and source indentation.
Add -P and -fno-reformat to the new drivers.
Tweak test options to avoid confusion with prior -E output; use
-fno-reformat where needed, but prefer to keep -E, sometimes
in concert with -P, on most, updating expected results accordingly.
Differential Revision: https://reviews.llvm.org/D106727
Until now, `f18` would:
1. Use Flang to unparse the input files
2. Call an external Fortran compiler to compile the unparsed source
files (generated in step 1)
With this patch, `f18` will stop after unparsing the input source files,
i.e. step 1 above. The `flang` bash script will take care of step 2,
i.e. calling an external Fortran compiler driver to compile them. This
way:
* the functionality of `f18` is reduced - it will only drive Flang (as
opposed to delegating code-generation to an external tool on top of
this)
* we will able to switch between `f18` and `flang-new` for unparsing before
an external Fortran compiler is called for code-generation
The updated `flang` bash script needs to specify the output file when
using the `-fdebug-unparse` action. Both `f18` and `flang-new` have been
updated accordingly.
These changes were discussed in [1] as a requirement for replacing `f18`
with `flang-new`.
[1] https://lists.llvm.org/pipermail/flang-dev/2021-April/000677.html
Differential Revision: https://reviews.llvm.org/D103177
This patch adds a new option for the new Flang driver:
`-fno-analyzed-objects-for-unparse`. The semantics are similar to
`-funparse-typed-exprs-to-f18-fc` from `f18`. For consistency, the
latter is replaced with `-fno-analyzed-objects-for-unparse`.
The new option controls the behaviour of the unparser (i.e. the action
corresponding to `-fdebug-unparse`). The default behaviour is to use the
analyzed objects when unparsing. The new flag can be used to turn this
off, so that the original parse-tree objects are used. The analyzed
objects are generated during the semantic checks [1].
This patch also updates the semantics of
`-fno-analyzed-objects-for-unparse`/`-funparse-typed-exprs-to-f18-fc`
in `f18`, so that this flag is always taken into account when `Unparse`
is used (this way the semantics in `f18` and `flang-new` are identical).
The added test file is based on example from Peter Steinfeld.
[1]
https://github.com/llvm/llvm-project/blob/main/flang/docs/Semantics.md
Differential Revision: https://reviews.llvm.org/D103612
The new option will run the semantic checks and then dump the parse tree
and all the symbols. This is equivalent to running the driver twice,
once with `-fdebug-dump-parse-tree` and then with
the `-fdebug-dump-symbols` action flag.
Currently we wouldn't be able to achieve the same by simply running:
```
flang-new -fc1 -fdebug-dump-parse-tree -fdebug-dump-symbols <input-file>
```
That's because the new driver will only run one frontend action per
invocation (both of the flags used here are action flags). Diverging
from this design would lead to costly compromises and it's best avoided.
We may want to consider re-designing our debugging actions (and action
options) in the future so that there's more code re-use. For now, I'm
focusing on making sure that we support all the major cases requested by
our users.
Differential Revision: https://reviews.llvm.org/D104305
Adding the `-init-only` option and corresponding frontend action to
generate a diagnostic.
`-init-only` vs `-test-io`:
`-init-only` ignores the input (it never calls the prescanner)
`-test-io` is similar to `-init-only`, but does read and print the input
without calling the prescanner.
This patch also adds a Driver test to check this action.
Reviewed By: awarzynski, AMDChirag
Differential Revision: https://reviews.llvm.org/D102849
In error recovery situations, the mappings from source locations
to scopes were failing in a way that tripped some asserts.
Specifically, FindPureProcedureContaining() wasn't coping well
when starting at the global scope. (And since the global scope
no longer has a source range, clean up the Semantics constructor
to avoid confusion.)
Differential Revision: https://reviews.llvm.org/D103567
When generating output for `-fdebug-dump-symbols`, make sure that
BuildRuntimeDerivedTypeTables is also run. This change is needed in
order to make the implementation of `-fdebug-dump-symbols` in
`flang-new` consistent with `f18`. It also allows us to port more tests
to use the new driver whenever it is enabled.
Differential Revision: https://reviews.llvm.org/D100649
This patch adds `-fget-definition` to `flang-new`. The semantics of this
option are identical in both drivers. The error message in the
"throwaway" driver is updated so that it matches the one from
`flang-new` (which is auto-generated and cannot be changed easily).
Tests are updated accordingly. A dedicated test for error handling was
added: get-definition.f90 (for the sake of simplicity,
getdefinition01.f90 no longer tests for errors).
The `ParseFrontendArgs` function is updated so that it can return
errors. This change is required in order to report invalid values
following `-fget-definition`.
The actual implementation of `GetDefinitionAction::ExecuteAction()` was
extracted from f18.cpp (i.e. the bit that deals with
`-fget-definition`).
Depends on: https://reviews.llvm.org/D100556
Differential Revision: https://reviews.llvm.org/D100558
This patch adds two debugging options in the new Flang driver
(flang-new):
*fdebug-unparse-no-sema
*fdebug-dump-parse-tree-no-sema
Each of these options combines two options from the "throwaway" driver
(left: f18, right: flang-new):
* `-fdebug-uparse -fdebug-no-semantics` --> `-fdebug-unparse-no-sema`
* `-fdebug-dump-parse-tree -fdebug-no-semantics` -->
`-fdebug-dump-parse-tree-no-sema`
There are no plans to implement `-fdebug-no-semantics` in the new
driver. Such option would be too powerful. Also, it would only make
sense when combined with specific frontend actions (`-fdebug-unparse`
and `-fdebug-dump-parse-tree`). Instead, this patch adds 2 specialised
options listed above. Each of these is implemented through a dedicated
FrontendAction (also added).
The new frontend actions are implemented in terms of a new abstract base
action: `PrescanAndSemaAction`. This new base class was required so that
we can have finer control over what steps within the frontend are
executed:
* `PrescanAction`: run the _prescanner_
* `PrescanAndSemaAction`: run the _prescanner_ and the _parser_ (new
in this patch)
* `PrescanAndSemaAction`: run the _prescanner_, _parser_ and run the
_semantic checks_
This patch introduces `PrescanAndParseAction::BeginSourceFileAction`.
Apart from the semantic checks removed at the end, it is similar to
`PrescanAndSemaAction::BeginSourceFileAction`.
Differential Revision: https://reviews.llvm.org/D99645
This patch adds support for the `-cpp` and `-nocpp` flags. The
implemented semantics match f18 (i.e. the "throwaway" driver), but are
different to gfortran. In Flang the preprocessor is always run. Instead,
`-cpp/-nocpp` are used to control whether predefined and command-line
preprocessor macro definitions are enabled or not. In practice this is
sufficient to model gfortran`s `-cpp/-nocpp`.
In the absence of `-cpp/-nocpp`, the driver will use the extension of
the input file to decide whether to include the standard macro
predefinitions. gfortran's documentation [1] was used to decide which
file extension to use for this.
The logic mentioned above was added in FrontendAction::BeginSourceFile.
That's relatively late in the driver set-up, but this roughly where the
name of the input file becomes available. The logic for deciding between
fixed and free form works in a similar way and was also moved to
FrontendAction::BeginSourceFile for consistency (and to reduce
code-duplication).
The `-cpp/-nocpp` flags are respected also when the input is read from
stdin. This is different to:
* gfortran (behaves as if `-cpp` was used)
* f18 (behaves as if `-nocpp` was used)
Starting with this patch, file extensions are significant and some test
files had to be renamed to reflect that. Where possible, preprocessor
tests were updated so that they can be shared between `f18` and
`flang-new`. This was implemented on top of adding new test for
`-cpp/-nocpp`.
[1] https://gcc.gnu.org/onlinedocs/gcc/Overall-Options.html
Reviewed By: kiranchandramohan
Differential Revision: https://reviews.llvm.org/D99292
Adds support for `-fget-symbols-sources` in the new Flang driver. All
relevant tests are updated to use the new driver when
`FLANG_BUILD_NEW_DRIVER` is set.
`RUN` lines in tests are updated so `-fsyntax-only`
comes before `-fget-symbols-sources`. That's because:
* both `-fsyntax-only` and `-fget-symbols-sources` are
action flags, and
* the new driver, flang-new, will only consider the right-most
action flag.
In other words, this change is needed so that the tests work with both
`f18` (requires both flags) and `flang-new` (only considers the last
action flag).
Differential Revision: https://reviews.llvm.org/D98191
This patch adds `-fdebug-dump-parsing-log` in the new driver. This option is
semantically identical to `-fdebug-instrumented-parse` in `f18` (the
former is added as an alias in `f18`).
As dumping the parsing log makes only sense for instrumented parses, we
set Fortran::parser::Options::instrumentedParse to `True` when
`-fdebug-dump-parsing-log` is used. This is consistent with `f18`.
To facilitate tweaking the configuration of the frontend based on the
action being requested, `setUpFrontendBasedOnAction` is introduced in
CompilerInvocation.cpp.
Differential Revision: https://reviews.llvm.org/D97457
Add the following options:
* -fdebug-measure-parse-tree
* -fdebug-pre-fir-tree
Summary of changes:
- Add 2 new frontend actions: DebugMeasureParseTreeAction and DebugPreFIRTreeAction
- Add MeasurementVisitor to FrontendActions.h
- Make reportFatalSemanticErrors return true if there are any fatal errors
- Port most of the `-fdebug-pre-fir-tree` tests to use the new driver if built, otherwise use f18.
Differential Revision: https://reviews.llvm.org/D96884
Add the following options:
* -fdebug-dump-symbols
* -fdebug-dump-parse-tree
* -fdebug-dump-provenance
Summary of changes:
- Add 3 new frontend actions: DebugDumpSymbolsAction, DebugDumpParseTreeAction and DebugDumpProvenanceAction
- Add a unique pointer to the Semantics instance created in PrescanAndSemaAction
- Move fatal semantic error reporting to its own method, FrontendActions#reportFatalSemanticErrors
- Port most tests using `-fdebug-dump-symbols` and `-fdebug-dump-parse-tree` to the new driver if built, otherwise default to f18
Differential Revision: https://reviews.llvm.org/D96716
This patch adds the following compiler frontend driver options:
* -fdebug-unparse (f18 spelling: -funparse)
* -fdebug-unparse-with-symbols (f18 spelling: -funparse-with-symbols)
The new driver will only accept the new spelling. `f18` will accept both
the original and the new spelling.
A new base class for frontend actions is added: `PrescanAndSemaAction`.
This is added to reduce code duplication that otherwise these new
options would lead to. Implementation from
* `ParseSyntaxOnlyAction::ExecutionAction`
is moved to:
* `PrescanAndSemaAction::BeginSourceFileAction`
This implementation is now shared between:
* PrescanAndSemaAction
* ParseSyntaxOnlyAction
* DebugUnparseAction
* DebugUnparseWithSymbolsAction
All tests that don't require other yet unimplemented options are
updated. This way `flang-new -fc1` is used instead of `f18` when
`FLANG_BUILD_NEW_DRIVER` is set to `On`. In order to facilitate this,
`%flang_fc1` is added in the LIT configuration (lit.cfg.py).
`asFortran` from f18.cpp is duplicated as `getBasicAsFortran` in
FrontendOptions.cpp. At this stage it's hard to find a good place to
share this method. I suggest that we revisit this once a switch from
`f18` to `flang-new` is complete.
Differential Revision: https://reviews.llvm.org/D96483
Add support for the following options:
* -fopenmp
* -fopenacc
Update OpenMP and OpenACC semantics tests to use the new driver if it is built, otherwise use f18.
OpenMP tests that include `use omp_lib` or run `test_symbols.sh` have not been updated as they require options `-intrinsic-module-directory` and `-funparse-with-symbols` which are currently not implemented in the new driver.
Similarly OpenACC tests that run `test_symbols.sh` have not been updated.
This patch also moves semanticsContext to CompilerInvocation and creates it in CompilerInvocation#setSemanticsOpts so that the semantics context can use Fortran::parser::Options#features.
Summary of changes:
- Move semanticsContext to CompilerInvocation.h
- Update OpenMP and OpenACC semantics tests that do not rely on `-intrinsic-module-directory` and `-funparse-with-symbols` to use %flang
Differential Revision: https://reviews.llvm.org/D96032
This patch adds logic in the InputOutputTestAction frontend action for
reading input from stdin. Without this patch the following fails:
```
flang-new -fc1 -test-io -
```
The implementation of `InputOutputTestAction` is cleaned-up and a test
for reading from stdin is added.
Note that there's a difference between `-test-io` and e.g. `-E` in terms
of file I/O. The frontend action for the former handles all file I/O on
it's own. Conversely, the action corresponding to -E relies on the
prescanner API to handle this.
Currently we can't test reading from stdin for `flang-new -`. In this
case `libclangDriver` assumes `-x -c`. This in turn leads to `flang-new
-cc1`, which is not supported.
Add support for option -J/-module-dir in the new Flang driver. This
will allow for including module files in other directories, as the
default search path is currently the working folder. This also provides
an option of storing the output module in the specified folder.
Differential Revision: https://reviews.llvm.org/D95448
This new action encapsulates all actions that require the prescanner to
be run before proceeding with other processing. By adding this new
action, we are better equipped to control which actions _do_ run the
prescanner and which _do not_.
The following actions that require the prescanner are refactored to
inherit from `PrescanAction`:
* `PrintPreprocessedAction`
* `ParseSyntaxOnlyAction` .
New virtual method is introduced to facilitate all this:
* `BeginSourceFileAction`
Like in Clang, this method is run inside `BeginSourceFile`. In other
words, it is invoked before `ExecuteAction` for the corresponding
frontend action is run. This method allows us to:
* carry out any processing that is always required by the action (e.g.
run the prescanner)
* fine tune the settings/options on a file-by-file basis (e.g. to
decide between fixed-form and free-form based on file extension)
This patch implements non-functional-changes.
Reviewed By: FarisRehman
Differential Revision: https://reviews.llvm.org/D95464
Make the #include "file" preprocessing directive begin its
search in the same directory as the file containing the directive,
as other preprocessors and our Fortran INCLUDE statement do.
Avoid current working directory for all source files except the original.
Resolve tests.
Differential Revision: https://reviews.llvm.org/D95481
Ensure diagnostics from the prescanner are reported when running `flang-new -fsyntax-only` (i.e. only syntax parsing).
This keeps the diagnostics output of flang-new consistent with `f18 -fparse-only` when running the syntax parsing action, ParseSyntaxOnlyAction.
Summary of changes:
- Modify ParseSyntaxOnlyAction::ExecuteAction to report diagnostics
Differential Revision: https://reviews.llvm.org/D95220
This patch makes sure that diagnostics from the prescanner are reported
when running `flang-new -E` (i.e. only the preprocessor phase is
requested). More specifically, the `PrintPreprocessedAction` action is
updated.
With this patch we make sure that the `f18` and `flang-new` provide
identical output when running the preprocessor and the prescanner
generates diagnostics.
Differential Revision: https://reviews.llvm.org/D94782
This patch adds a frontend action for emitting object files. While Flang
does not support code-generation, this action remains a placeholder.
This patch simply provides glue-code to connect the compiler driver
with the appropriate frontend action.
The new action is triggered with the `-c` compiler driver flag, i.e.
`flang-new -c`. This is then translated to `flang-new -fc1 -emit-obj`,
so `-emit-obj` has to be marked as supported as well.
As code-generation is not available yet, `flang-new -c` results in a
driver error:
```
error: code-generation is not available yet
```
Hopefully this will help communicating the level of available
functionality within Flang.
The definition of `emit-obj` is updated so that it can be shared between
Clang and Flang. As the original definition was enclosed within a
Clang-specific TableGen `let` statement, it is extracted into a new `let`
statement. That felt like the cleanest option.
I also commented out `-triple` in Flang::ConstructJob and updated some
comments there. This is similar to https://reviews.llvm.org/D93027. I
wanted to make sure that it's clear that we can't support `-triple`
until we have code-generation. However, once code-generation is
available we _will need_ `-triple`.
As this patch adds `-emit-obj`, the emit-obj.f90 becomes irrelevant and
is deleted. Instead, phases.f90 is added to demonstrate that users can
control compilation phases (indeed, `-c` is a phase control flag).
Reviewed By: SouraVX, clementval
Differential Revision: https://reviews.llvm.org/D93301
As per Flang's coding guidelines
(flang/docs/C++style.md#error-messages):
```
Messages should start with a capital letter.
```
This patch updates error messages in the driver (new and old) so that
they conform with the guideline above.
This change was suggested in one of the recent reviews:
https://reviews.llvm.org/D93712. It felt like this deserved a dedicated
patch, so sending it separately.
If either `Prescan` or `Parse` generate any fatal errors, the new driver
will:
* report it (i.e. issue an error diagnostic)
* exit early
* return non-zero exit code
This behaviour is consistent with f18 (i.e. the old driver).
Reviewed By: sameeranjoshi
Differential Revision: https://reviews.llvm.org/D93712
The behaviour triggered with this flag is consistent with `-fparse-only`
in `flang` (i.e. the throwaway driver). This new spelling is consistent
with Clang and gfortran, and was proposed and agreed on for the new
driver in [1].
This patch also adds some minimal logic to communicate whether the
semantic checks have failed or not. When semantic checks fail, a
frontend driver error is generated. The return code from the frontend
driver is then determined by checking the driver diagnostics - the
presence of driver errors means that the compilation has failed. This
logic is consistent with `clang -cc1`.
[1] http://lists.llvm.org/pipermail/flang-dev/2020-November/000588.html
Differential Revision: https://reviews.llvm.org/D92854
This patch implements the first frontend action for the Flang parser (i.e.
Fortran::parser). This action runs the preprocessor and is invoked with the
`-E` flag. (i.e. `flang-new -E <input-file>). The generated output is printed
to either stdout or the output file (specified with `-` or `-o <output-file>`).
Note that currently there is no mechanism to map options for the
frontend driver (i.e. Fortran::frontend::FrontendOptions) to options for
the parser (i.e. Fortran::parser::Options). Instead,
Frotran::parser::options are hard-coded to:
```
std::vector<std::string> searchDirectories{"."s};
searchDirectories = searchDirectories;
isFixedForm = false;
_encoding(Fortran::parser::Encoding::UTF_8);
```
These default settings are compatible with the current Flang driver. Further
work is required in order for CompilerInvocation to read and map
clang::driver::options to Fortran::parser::options.
Co-authored-by: Andrzej Warzynski <andrzej.warzynski@arm.com>
Differential Revision: https://reviews.llvm.org/D88381
As per point 3 in [1]:
```
Accessor member functions are named with the non-public data member's
name, less the trailing underscore. Mutator member functions are named
set_...
```
Originally we just followed the LLVM's style, which is incompatible with
Flang. This patch renames the accessors and mutators accordingly.
`getDiagnostics` and `GetDiagnostics` are replaced with one accessor:
`diagnostics`. `SetDiagnostics` was neither implemented nor used, so
it's deleted.
[1] https://github.com/llvm/llvm-project/blob/master/flang/docs/C++style.md#naming
Differential Revision: https://reviews.llvm.org/D90300
This patch introduces the dependencies required to read and manage input files
provided by the command line option. It also adds the infrastructure to create
and write to output files. The output is sent to either stdout or a file
(specified with the `-o` flag).
Separately, in order to be able to test the code for file I/O, it adds
infrastructure to create frontend actions. As a basic testable example, it adds
the `InputOutputTest` FrontendAction. The sole purpose of this action is to
read a file from the command line and print it either to stdout or the output
file. This action is run by using the `-test-io` flag also introduced in this
patch (available for `flang-new` and `flang-new -fc1`). With this patch:
```
flang-new -test-io input-file.f90
```
will read input-file.f90 and print it in the output file.
The `InputOutputTest` frontend action has been introduced primarily to
facilitate testing. It is hidden from users (i.e. it's only displayed with
`--help-hidden`). Currently Clang doesn’t have an equivalent action.
`-test-io` is used to trigger the InputOutputTest action in the Flang frontend
driver. This patch makes sure that “flang-new” forwards it to “flang-new -fc1"
by creating a preprocessor job. However, in Flang.cpp, `-test-io` is passed to
“flang-new -fc1” without `-E`. This way we make sure that the preprocessor is
_not_ run in the frontend driver. This is the desired behaviour: `-test-io`
should only read the input file and print it to the output stream.
co-authored-by: Andrzej Warzynski <andrzej.warzynski@arm.com>
Differential Revision: https://reviews.llvm.org/D87989
Andrzej Warzynski