The parser combinator withMessage("error message"_err_en_US, PARSER) is meant
to run the parser PARSER and, if it fails, override its error messages if
it failed silently or it was unable to recognize any tokens at all. This
gives the parser a way to avoid emitting some confusing or missing error
messages. Unfortunately, the implementation could sometimes lose track of
whether any tokens had been recognized, leading to problems with outer usage
of withMessage() and also -- more seriously -- with ParseState::CombineFailedParses().
That's a utility that determines which error messages to retain when two
or more parsers have been attempted at the same starting point and none
of them succceed. Its policy is to retain the state from the parser that
consumed the most input text before failing, so long as it had recognized at
least one token.
So anyway, fix up withMessage(), adjust the tests, and add a test of the
original motivating confusing error situation, in which a syntax error in
a COMMON statement was being diagnosed as a problem with a statement function
definition because withMessage() had lost the fact that the parse of the
COMMON statement had recognized some tokens, and the last attempted parse
later was a failed attempt to parse a statement function.
Differential Revision: https://reviews.llvm.org/D135216
A generic-spec can appear on a module accessibility control statement
even if it has not been declared as a generic interface, because there's
nothing else that it could be.
While here, simplify the parse tree and parser for AccessId, since
one of its alternatives is ambiguous with the other.
Differential Revision: https://reviews.llvm.org/D134471
This patch adds support for the device clause on `Target` directive.
Device clause was added in OpenMP specification version 4.5 to
create a device data environment for the extent of a region. On
target construct, the device expression be either be `ancestor`
(taking after the parent) or assign a new `device_num`.
Reviewed By: kiranchandramohan
Differential Revision: https://reviews.llvm.org/D126441
The options -f{no-}color-diagnostics have been supported in driver. This
supports the behaviors in scanning, parsing, and semantics, and the
behaviors are exactly the same as the driver.
To illustrate the added behaviour, consider the following input file:
```! file.f90
program m
integer :: i = k
end
```
In the following invocations, "error: Must be a constant value" _will be_
formatted:
```
$ flang-new file.f90
error: Semantic errors in file.f90
./file.f90:2:18: error: Must be a constant value
integer :: i = k
```
Note that "error: Semantic errors in file.f90" is also formatted, which
is supported in https://reviews.llvm.org/D126164.
Also note that only "error", "warning" and "portability" are formatted.
Check the following input file:
```! file2.f90
program m
integer :: i =
end
```
```
$ flang-new test2.f90
error: Could not parse test2.f90
./test2.f90:2:11: error: expected '('
integer :: i =
^
./test2.f90:2:3: in the context: statement function definition
integer :: i =
^
...
```
The "error: Could not parse test2.f90" and "error: expected '('" are
formatted. Others such as "in the context" are not formatted yet, which
may or may not be supported.
Reviewed By: awarzynski
Differential Revision: https://reviews.llvm.org/D126166
A type-param-inquiry of %KIND or %LEN applies to a designator, and
so must also be allowed for a substring. F18 presently (mis)parses
instances of a type-param-inquiry as structure component references
and then fixes them in expression semantics when types are known and
we can distinguish them. But when the base of a type-param-inquiry is
a substring of an array element, as in "charArray(i)(j:k)%len",
parsing fails.
Adjust the grammar to parse these cases, and extend expression semantics
to process the new production.
Differential Revision: https://reviews.llvm.org/D130375
This patch introduce an automatic generation of the clause parser from the TableGen
information.
New information can be stored directly in the TableGen file:
- The different aliases that a clause support.
- prefix before a value.
- whether a prefix is optional or not.
Makes it easier to add new clauses and also avoid some error (`write` clause incorrect until now).
This patch is updating only the OpenACC part. A patch with a modification of the OpenMP clause parser will follow.
Reviewed By: kiranchandramohan
Differential Revision: https://reviews.llvm.org/D106968
Move the device_type parser to a separate parser AccDeviceTypeExprList. Preparatory work for D106968.
Reviewed By: kiranchandramohan
Differential Revision: https://reviews.llvm.org/D106967
Set the isOptional flag for the self clause. Move the optional and parenthesis part of the parser. Update the rest of the code to deal with the optional value.
Preparatory work for D106968.
Reviewed By: kiranchandramohan
Differential Revision: https://reviews.llvm.org/D106965
In free form source, pedantic mode will elicit portability warnings
about missing spaces when a token string ends with a character that
can be in an identifier and there is no space between that last token
character and a following character that can also be part of an identifier.
This behavior doesn't really work well for the token strings that are
parsed for edit descriptors in FORMAT statements. For example, the
'F' in FORMAT(F7.3) is followed by a digit, but obviously no space is
necessary. Free form or not, FORMATs are their own odd little world.
This patch adds trailing blanks to these FORMAT edit descriptor token
parsers to disable the missing space check, and extends the documentation
for token string parsing to explain why this technique works.
Differential Revision: https://reviews.llvm.org/D129023
OpenMP 5.0 adds a new clause `in_reduction` on OpenMP directives.
This patch adds parser support for the same.
Reviewed By: kiranchandramohan
Differential Revision: https://reviews.llvm.org/D124156
In the USE statements that f18 emits to module files, ensure that symbols
from intrinsic modules are marked as such on their USE statements. And
ensure that the current working directory (".") cannot override the intrinsic
module search path when trying to locate an intrinsic module.
Differential Revision: https://reviews.llvm.org/D127019
If a line is over 72 characters long, flang's preprocessor cuts it there
and continues on the next line.
For this purpose it uses the standard way of continuing line with & on each line.
However, it doesn't work with long compiler directives, like OpenMP or OpenACC ones.
The line that continues the directive also has to
contain the corresponding sentinel at the beginning.
This change implements the described functionality.
Also, some code was refactored in order to simplify and reuse existing code.
Reviewed By: klausler
Differential Revision: https://reviews.llvm.org/D126301
F(X)=Y may be initially parsed as a statement function definition; an
existing pass will detect statement functions that should be rewritten
into assignment statemets with array element references as their
left-hand side variables. However, F() may also be a reference to a
function that returns a data pointer, and f18 did not handle this
case correctly.
The right fix is to rewrite the parse tree for F(X)=Y into an assignment
to a function reference result. The cases that are actually assignments
to array elements -- including all of the cases previously handled --
will have their left-hand sides converted to array element references
later by another existing rewriting step.
Differential Revision: https://reviews.llvm.org/D124299
Adds flang/include/flang/Common/log2-visit.h, which defines
a Fortran::common::visit() template function that is a drop-in
replacement for std::visit(). Modifies most use sites in
the front-end and runtime to use common::visit().
The C++ standard mandates that std::visit() have O(1) execution
time, which forces implementations to build dispatch tables.
This new common::visit() is O(log2 N) in the number of alternatives
in a variant<>, but that N tends to be small and so this change
produces a fairly significant improvement in compiler build
memory requirements, a 5-10% improvement in compiler build time,
and a small improvement in compiler execution time.
Building with -DFLANG_USE_STD_VISIT causes common::visit()
to be an alias for std::visit().
Calls to common::visit() with multiple variant arguments
are referred to std::visit(), pending further work.
This change is enabled only for GCC builds with GCC >= 9;
an earlier attempt (D122441) ran into bugs in some versions of
clang and was reverted rather than simply disabled; and it is
not well tested with MSVC. In non-GCC and older GCC builds,
common::visit() is simply an alias for std::visit().
For parameterized derived type component initializers whose
expressions' types depend on parameter values, f18's current
scheme of analyzing the initialization expression once during
name resolution fails. For example,
type :: pdt(k)
integer, kind :: k
real :: component = real(0.0, kind=k)
end type
To handle such cases, it is necessary to re-analyze the parse
trees of these initialization expressions once for each distinct
initialization of the type.
This patch adds code to wipe an expression parse tree of its
typed expressions, and update those of its symbol table pointers
that reference type parameters, and then re-analyze that parse
tree to generate the properly typed component initializers.
Differential Revision: https://reviews.llvm.org/D123728
To avoid clashing with names of user derived types, the redundant
syntax TYPE(intrinsic type spec) must be interpreted as a monomorphic
derived type when "intrinsic type spec" is a single word. This
affects TYPE(BYTE) and TYPE(DOUBLECOMPLEX), but not TYPE(DOUBLE COMPLEX)
in free form source.
Differential Revision: https://reviews.llvm.org/D123724
Error messages can have a list of attachments; these are used to point
to related source locations, supply additional information, and to
encapsulate error messages that were *not* emitted in a given context
to explain why a warning was justified.
This patch adds a message severity ("Because") for that last case,
and extends to AttachTo() API to provide a means for overriding
the severity of an attached message.
Some existing message attachments had their severities adjusted,
now that we're printing them. And operator==() for Message was
cleaned up while debugging after I noticed that it was recursively
O(N**2) and subject to returning a false positive.
Differential Revision: https://reviews.llvm.org/D123710
When including debug lines as code, the `D` should be considered as
a white space. Currently an error was raised about bad labels because
it the `D` remained a `D` when considering the source line as code.
Differential Revision: https://reviews.llvm.org/D122711
Adds flang/include/flang/Common/visit.h, which defines
a Fortran::common::visit() template function that is a drop-in
replacement for std::visit(). Modifies most use sites in
the front-end and runtime to use common::visit().
The C++ standard mandates that std::visit() have O(1) execution
time, which forces implementations to build dispatch tables.
This new common::visit() is O(log2 N) in the number of alternatives
in a variant<>, but that N tends to be small and so this change
produces a fairly significant improvement in compiler build
memory requirements, a 5-10% improvement in compiler build time,
and a small improvement in compiler execution time.
Building with -DFLANG_USE_STD_VISIT causes common::visit()
to be an alias for std::visit().
Calls to common::visit() with multiple variant arguments
are referred to std::visit(), pending further work.
Differential Revision: https://reviews.llvm.org/D122441
Extend "extension<LanguageFeature>()" to incorporate an explanatory
message better than the current generic "nonstandard usage:".
Differential Revision: https://reviews.llvm.org/D122035
This patch adds parser nodes for each indivudual section in sections
construct. This should help with the translation to FIR. `!$omp section`
was not recognized as a construct and hence needed special handling.
`OpenMPSectionsConstruct` contains a list of `OpenMPConstruct`. Each
such `OpenMPConstruct` wraps an `OpenMPSectionConstruct`
(section, not sections). An `OpenMPSectionConstruct` is a wrapper around
a `Block`.
Reviewed By: kiranchandramohan, peixin
Differential Revision: https://reviews.llvm.org/D121680
Using recently established message severity codes, upgrade
non-fatal messages to usage and portability warnings as
appropriate.
Differential Revision: https://reviews.llvm.org/D121246
F18 presently has fatal and non-fatal diagnostic messages. We'd like
to make non-fatal warnings stand out better in the output of the compiler.
This will turn out to be a large change that affects many files.
This patch is just the first part. It converts a Boolean isFatal_ data
member of the message classes into a severity code, and defines four
of these codes (Error, Warning, Portability, and a catch-all Other).
Later patches will result from sweeping over the parser and semantics,
changing most non-fatal diagnostic messages into warnings and portability
notes.
Differential Revision: https://reviews.llvm.org/D121228
When a contiguous range of a cooked character stream is being
mapped to a range of source provenance, the code was assuming
that the "end()" position of the input range -- being the character
immediately after the range -- would also follow the range's
source provenance. This isn't always the case.
Modify the code to work with the true last character of the
input range (at end()-1) and to also cope with cases when that
last position truly maps to an earlier provenance, which can happen
when the prescanner has inserted a space into the cooked character
stream.
Differential Revision: https://reviews.llvm.org/D121124
EQUIVALENCE storage association of objects whose types are not
both default-kind numeric storage sequences, or not both default-kind
character storage sequences, are not standard conformant.
However, most Fortran compilers admit such usage, with warnings
in strict conformance mode. This patch allos EQUIVALENCE of objects
that have sequence types that are either identical, both numeric
sequences (of default kind or not), or both character sequences.
Non-sequence types, and sequences types that are not homogeneously
numeric or character, remain errors.
Differential Revision: https://reviews.llvm.org/D119848
For "USE, INTRINSIC", search only for intrinsic modules;
for "USE, NON_INTRINSIC", do not recognize intrinsic modules.
Allow modules of both kinds with the same name to be used in
the same source file (but not in the same scoping unit, a
constraint of the standard that is now enforced).
The symbol table's scope tree now has a single instance of
a scope with a new kind, IntrinsicModules, whose children are
the USE'd intrinsic modules (explicit or not). This separate
"top-level" scope is a child of the single global scope and
it allows both intrinsic and non-intrinsic modules of the same
name to exist in the symbol table. Intrinsic modules' scopes'
symbols now have the INTRINSIC attribute set.
The search path directories need to make a distinction between
regular directories and the one(s) that point(s) to intrinsic
modules. I allow for multiple intrinsic module directories in
the second search path, although only one is needed today.
Differential Revision: https://reviews.llvm.org/D118631
Implements part of the legacy "DEC structures" feature from
VMS Fortran. STRUCTUREs are processed as if they were derived
types with SEQUENCE. DATA-like object entity initialization
is supported as well (e.g., INTEGER FOO/666/) since it was used
for default component initialization in structures. Anonymous
components (named %FILL) are also supported.
These features, and UNION/MAP, were already being parsed.
An omission in the collection of structure field names in the
case of nested structures with entity declarations was fixed
in the parser.
Structures are supported in modules, but this is mostly for
testing purposes. The names of fields in structures accessed
via USE association cannot appear with dot notation in client
code (at least not yet). DEC structures antedate Fortran 90,
so their actual use in applications should not involve modules.
This patch does not implement UNION/MAP, since that feature
would impose difficulties later in lowering them to MLIR types.
In the meantime, if they appear, semantics will issue a
"not yet implemented" error message.
Differential Revision: https://reviews.llvm.org/D117151
When preprocessing "# ARG" in function-like macro expansion,
the preprocessor needs to pop the previously-pushed '#' token
from the end of the resulting token sequence after detecting the
argument name. The code to do this was just wrong in a couple of
ways.
Differential Revision: https://reviews.llvm.org/D117148
As reported in https://bugs.llvm.org/show_bug.cgi?id=48145, name resolution for omp critical construct was failing. This patch adds functionality to help that name resolution as well as implementation to catch name mismatches.
The following semantic restrictions are therefore handled here:
- If a name is specified on a critical directive, the same name must also be specified on the end critical directive
- If no name appears on the critical directive, no name can appear on the end critical directive
- If a name appears on either the start critical directive or the end critical directive
Reviewed By: kiranchandramohan
Differential Revision: https://reviews.llvm.org/D110502
Source lines with mismatched parentheses are hard cases for error
recovery in parsing, and the best error message (viz.,
"here's an unmatched parenthesis") can be emitted from the
prescanner.
Differential Revision: https://reviews.llvm.org/D111254#3046173
The THEN keyword in the "ELSE IF (test) THEN" statement is useless
syntactically, and to omit it is a common error (at least for me!)
that has poor error recovery. This patch changes the parser to
cough up a simple "expected 'THEN'" and still recognize the rest of
the IF construct.
Differential Revision: https://reviews.llvm.org/D110952
From subclause 6.3.3.5: a program unit END statement cannot be
continued in fixed form, and other statements cannot have initial
lines that look like program unit END statements. I think this
is to avoid violating assumptions that are important to legacy
compilers' statement classification routines.
Differential Revision: https://reviews.llvm.org/D109933
This patch adds parsing support for the nontemporal clause. Also adds a couple of test cases.
Reviewed By: clementval
Differential Revision: https://reviews.llvm.org/D106896
Ticking off a Parser TODO: Preprocessor::Directive()'s Prescanner
argument should be a reference, not a pointer.
Differential Revision: https://reviews.llvm.org/D109094
Flang uses positional arguments for `messages::say()`, such as "%1$s" which is only supported in MS Compilers with the `_*printf_p` form of the function. This uses a conditional macro to convert the existing `vsnprintf` used to the one needed in MS-World.
7 tests in D107575 rely on this change.
Reviewed By: Meinersbur
Differential Revision: https://reviews.llvm.org/D107654
Rename the current -E option to "-E -Xflang -fno-reformat".
Add a new Parsing::EmitPreprocessedSource() routine to convert the
cooked character stream output of the prescanner back to something
more closely resembling output from a traditional preprocessor;
call this new routine when -E appears.
The new -E output is suitable for use as fixed form Fortran source to
compilation by (one hopes) any Fortran compiler. If the original
top-level source file had been free form source, the output will be
suitable for use as free form source as well; otherwise there may be
diagnostics about missing spaces if they were indeed absent in the
original fixed form source.
Unless the -P option appears, #line directives are interspersed
with the output (but be advised, f18 will ignore these if presented
with them in a later compilation).
An effort has been made to preserve original alphabetic character case
and source indentation.
Add -P and -fno-reformat to the new drivers.
Tweak test options to avoid confusion with prior -E output; use
-fno-reformat where needed, but prefer to keep -E, sometimes
in concert with -P, on most, updating expected results accordingly.
Differential Revision: https://reviews.llvm.org/D106727
We sometimes unroll an ac-implied-do of an array constructor into a flat list
of values. We then re-analyze the array constructor that contains the
resulting list of expressions. Such a list may or may not contain errors.
But when processing an array constructor with an unrolled ac-implied-do, the
compiler was building an expression to represent the extent of the resulting
array constructor containing the list of values. The number of operands
in this extent expression was based on the number of elements in the
unrolled list of values. For very large lists, this created an
expression so large that it could not be evaluated by the compiler
without overflowing the stack.
I fixed this by continuously folding the extent expression as each operand is
added to it. I added the test .../flang/test/Semantics/array-constr-big.f90
that will cause the compiler to seg fault without this change.
Also, when the unrolled ac-implied-do expression contains errors, we were
repeating the same error message referencing the same source line for every
instance of the erroneous expression in the unrolled list. This potentially
resulted in a very long list of messages for a single error in the source code.
I fixed this by comparing the message being emitted to the previously emitted
message. If they are the same, I do not emit the message. This change is also
tested by the new test array-constr-big.f90.
Several of the existing tests had duplicate error messages for the same source
line, and this change caused differences in their output. So I adjusted the
tests to match the new message emitting behavior.
Differential Revision: https://reviews.llvm.org/D102210
Replace semantics::SymbolSet with alternatives that clarify
whether the set should order its contents by source position
or not. This matters because positionally-ordered sets must
not be used for Symbols that might be subjected to name
replacement during name resolution, and address-ordered
sets must not be used (without sorting) in circumstances
where the order of their contents affects the output of the
compiler.
All set<> and map<> instances in the compiler that are keyed
by Symbols now have explicit Compare types in their template
instantiations. Symbol::operator< is no more.
Differential Revision: https://reviews.llvm.org/D98878
In parser::AllCookedSources, implement a map from CharBlocks to
the CookedSource instances that they cover. This permits a fast
Find() operation based on std::map::equal_range to map a CharBlock
to its enclosing CookedSource instance.
Add a creation order number to each CookedSource. This allows
AllCookedSources to provide a Precedes(x,y) predicate that is a
true source stream ordering between two CharBlocks -- x is less
than y if it is in an earlier CookedSource, or in the same
CookedSource at an earlier position.
Add a reference to the singleton SemanticsContext to each Scope.
All of this allows operator< to be implemented on Symbols by
means of a true source ordering. From a Symbol, we get to
its Scope, then to the SemanticsContext, and then use its
AllCookedSources reference to call Precedes().
Differential Revision: https://reviews.llvm.org/D98743
`parser::AllocateObject` and `parser::PointerObject` can be represented
as typed expressions once analyzed. This simplifies the work for parse-tree
consumers that work with typed expressions to deal with allocatable and
pointer objects such as lowering.
This change also makes it easier to add typedExpr in the future by
automatically handling nodes that have this member when possible.
Changes:
- Add a `mutable TypedExpr typedExpr` field to `parser::PointerObject` and `parser::AllocateObject`.
- Add a `parser::HasTypedExpr<T>` helper to better share code relating to typedExpr in the parse tree.
- Add hooks in `semantics::ExprChecker` for AllocateObject and PointerObject nodes, and use
ExprOrVariable on it to analyze and set the tyedExpr field during
expression analysis. This required adding overloads for `AssumedTypeDummy`.
- Update check-nullify.cpp and check-deallocate.cpp to not re-analyze the StructureComponent but to
use the typedExpr field instead.
- Update dump/unparse to use HasTypedExpr and use the typedExpr when there is one.
Differential Revision: https://reviews.llvm.org/D98256
This patch adds the following compiler frontend driver options:
* -fdebug-unparse (f18 spelling: -funparse)
* -fdebug-unparse-with-symbols (f18 spelling: -funparse-with-symbols)
The new driver will only accept the new spelling. `f18` will accept both
the original and the new spelling.
A new base class for frontend actions is added: `PrescanAndSemaAction`.
This is added to reduce code duplication that otherwise these new
options would lead to. Implementation from
* `ParseSyntaxOnlyAction::ExecutionAction`
is moved to:
* `PrescanAndSemaAction::BeginSourceFileAction`
This implementation is now shared between:
* PrescanAndSemaAction
* ParseSyntaxOnlyAction
* DebugUnparseAction
* DebugUnparseWithSymbolsAction
All tests that don't require other yet unimplemented options are
updated. This way `flang-new -fc1` is used instead of `f18` when
`FLANG_BUILD_NEW_DRIVER` is set to `On`. In order to facilitate this,
`%flang_fc1` is added in the LIT configuration (lit.cfg.py).
`asFortran` from f18.cpp is duplicated as `getBasicAsFortran` in
FrontendOptions.cpp. At this stage it's hard to find a good place to
share this method. I suggest that we revisit this once a switch from
`f18` to `flang-new` is complete.
Differential Revision: https://reviews.llvm.org/D96483
Avoid spurious and confusing macro replacements from things like
-DPIC on Fortran source files whose suffixes indicate that preprocessing
is not expected.
Add gfortran-like "-cpp" and "-nocpp" flags to f18 to force predefinition
of macros independent of the source file suffix.
Differential Revision: https://reviews.llvm.org/D96464
The parsing of I/O units uses look-ahead to discriminate between
keywords, variables and expressions as part of distinguishing internal
from external I/O. The look-ahead was inaccurate for variables that
appear as the initial parts of expressions.
Differential Revision: https://reviews.llvm.org/D95743
Peter Klausler