Commit Graph

12 Commits

Author SHA1 Message Date
Haojian Wu e935f7fd0c [pseudo] Fix a bug in checking the duplicated grammar rules. 2022-08-11 13:16:01 +02:00
Sam McCall 6bdb15fe84 [pseudo] Reorganize CXX.h enums
- Place rules under rule::lhs::rhs__rhs__rhs
- Change mangling of keywords to ALL_CAPS (needed to turn keywords that appear
  alone on RHS into valid identifiers)
- Make enums implicitly convertible to underlying type (though still scoped,
  using alias tricks)

In principle this lets us exhaustively write a switch over all rules of a NT:
  switch ((rule::declarator)N->rule()) {
    case rule::declarator::noptr_declarator:
      ...
  }
In practice we don't do this anywhere yet as we're often switching over multiple
nonterminal kinds at once.

Differential Revision: https://reviews.llvm.org/D130414
2022-07-27 09:03:29 +02:00
Sam McCall 3132e9cd7c [pseudo] Key guards by RuleID, add guards to literals (and 0).
After this, NUMERIC_CONSTANT and strings should parse only one way.

There are 8 types of literals, and 24 valid (literal, TokenKind) pairs.
This means adding 8 new named guards (or 24, if we want to assert the token).

It seems fairly clear to me at this point that the guard names are unneccesary
indirection: the guards are in fact coupled to the rule signature.

(Also add the zero guard I forgot in the previous patch.)

Differential Revision: https://reviews.llvm.org/D130066
2022-07-21 22:42:31 +02:00
Haojian Wu 2315358906 [pseudo] Generate an enum type for identifying grammar rules.
The Rule enum type enables us to identify a grammar rule within C++'s
type system.

Differential Revision: https://reviews.llvm.org/D129359
2022-07-15 15:09:31 +02:00
Sam McCall 3f028c02ba [pseudo] Grammar::parseBNF returns Grammar not unique_ptr. NFC 2022-06-28 16:34:21 +02:00
Haojian Wu c70aeaad2b [pseudo] Move grammar-related headers to a separate dir, NFC.
We did that for .cpp, but forgot the headers.

Differential Revision: https://reviews.llvm.org/D127388
2022-06-09 14:58:05 +02:00
Haojian Wu f1ac00c9b0 [pseudo] Add grammar annotations support.
Add annotation handling ([key=value]) in the BNF grammar parser, which
will be used in the conditional reduction, and error recovery.

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D126536
2022-06-09 12:06:22 +02:00
Haojian Wu 1a65c491be [pseudo] Support parsing variant target symbols.
With this patch, we're able to parse smaller chunks of C++ code (statement,
declaration), rather than translation-unit.

The start symbol is listed in the grammar in a form of `_ :=
statement`, each start symbol has a dedicated state (`_ := • statement`).
We create and track all these separate states in the LRTable. When we
start parsing, we lookup the corresponding state to start the parser.

LR pasing table changes with this patch:
- number of states: 1467 -> 1471
- number of actions: 82891 -> 83578
- size of the table (bytes): 334248 -> 336996

Differential Revision: https://reviews.llvm.org/D125006
2022-05-16 10:38:16 +02:00
Haojian Wu f383b88d82 [pseudo] Sort nonterminals based on their reduction order.
Reductions need to be performed in a careful order in GLR parser, to
make sure we gather all alternatives before creating an ambigous forest
node.

This patch encodes the nonterminal order into the rule id, so that we
can efficiently to determinal ordering of reductions in GLR parser.

This patch also abstracts to a TestGrammar, which is shared among tests.

This is a part of the GLR parser, https://reviews.llvm.org/D121368,
https://reviews.llvm.org/D121150

Differential Revision: https://reviews.llvm.org/D122303
2022-03-24 14:30:12 +01:00
Sam McCall 89cd86bbc5 Reapply [pseudo] Move pseudoparser from clang to clang-tools-extra"
This reverts commit 049f4e4eab.

The problem was a stray dependency in CLANG_TEST_DEPS which caused cmake
to fail if clang-pseudo wasn't built. This is now removed.
2022-03-16 01:10:55 +01:00
Sam McCall 049f4e4eab Revert "[pseudo] Move pseudoparser from clang to clang-tools-extra"
This reverts commit b97856c4cf.

Breaks a bunch of bots:
https://lab.llvm.org/buildbot/#/builders/193/builds/8513
2022-03-16 01:06:24 +01:00
Sam McCall b97856c4cf [pseudo] Move pseudoparser from clang to clang-tools-extra
This should make clearer that:
 - it's not part of clang proper
 - there's no expectation to update it along with clang (beyond green tests)
 - clang should not depend on it

This is intended to be expose a library, so unlike other tools has a split
between include/ and lib/.

The main renames are:
  clang/lib/Tooling/Syntax/Pseudo/*           => clang-tools-extra/pseudo/lib/*
  clang/include/clang/Tooling/Syntax/Pseudo/* => clang-tools-extra/pseudo/include/clang-pseudo/*
  clang/tools/clang/pseudo/*                  => clang-tools-extra/pseudo/tool/*
  clang/test/Syntax/*                         => clang-tools-extra/pseudo/test/*
  clang/unittests/Tooling/Syntax/Pseudo/*     => clang-tools-extra/pseudo/unittests/*
  #include "clang/Tooling/Syntax/Pseudo/*"    => #include "clang-pseudo/*"
  namespace clang::syntax::pseudo             => namespace clang::pseudo
  check-clang                                 => check-clang-pseudo
  clangToolingSyntaxPseudo                    => clangPseudo
The clang-pseudo and ClangPseudoTests binaries are not renamed.

See discussion around:
https://discourse.llvm.org/t/rfc-a-c-pseudo-parser-for-tooling/59217/50

Differential Revision: https://reviews.llvm.org/D121233
2022-03-16 00:14:11 +01:00