Commit Graph

69 Commits

Author SHA1 Message Date
Chris Lattner 2b15cf705b simplify the lexer a bit, pulling stuff out of the default case.
llvm-svn: 45540
2008-01-03 17:58:54 +00:00
Chris Lattner 5b12ab8c93 Don't attribute in file headers anymore. See llvmdev for the
discussion of this change.

llvm-svn: 45410
2007-12-29 19:59:25 +00:00
Chris Lattner 67671ed4b7 add a helper method.
llvm-svn: 44976
2007-12-13 01:59:49 +00:00
Chris Lattner 4431a1b19b start partitioning the diagnostics into two classes: those
that are builtin and those that are aren't.  This is a bunch
of API refactoring that will make this possible, but there is
no functionality change yet.

llvm-svn: 44473
2007-11-30 22:53:43 +00:00
Hartmut Kaiser c8107e54d5 Silenced a couple of VC++ warnings.
llvm-svn: 43125
2007-10-18 12:47:01 +00:00
Chris Lattner 8e129c23c8 Move token length calculation out of the diagnostics machinery into
the lexer, where it can be shared.

llvm-svn: 43090
2007-10-17 21:18:47 +00:00
Chris Lattner 02b436a05a Add a new type of lexer: a raw lexer, which does not require a preprocessor
object in order to do its thing.

llvm-svn: 43084
2007-10-17 20:41:00 +00:00
Chris Lattner 1f1b0dbc28 Make a significant change to invert the control flow handling
predefined macros.  Previously, these were handled by the driver,
now they are handled by the preprocessor.

Some fallout of this:

1. Instead of preprocessing two buffers (the predefines, then the 
   main source file)  we now start preprocessing the main source 
   file and inject the predefines as a "psuedo #include" from the
   main source file.
2. #1 allows us to nuke the Lexer::IsMainFile flag and simplify
   Preprocessor::isInPrimaryFile.
3. The driver doesn't have to know about standard #defines, the
   preprocessor knows, which is nice for people wanting to define
   their own drivers.
4. This allows us to put normal tokens in the predefine buffer,
   for example a definition for __builtin_va_list that is 
   target-specific, and a typedef for id in objc.

llvm-svn: 42818
2007-10-09 22:10:18 +00:00
Chris Lattner 98c1f7cfde Switch lexer/pp over to new Token::is/isNot api
llvm-svn: 42799
2007-10-09 18:02:16 +00:00
Chris Lattner 4894f485c7 implement the Token class in the Lexer.cpp file instead of IdentifierInfo.cpp
llvm-svn: 42728
2007-10-07 08:47:24 +00:00
Chris Lattner 78b6221ff9 Eliminate some VC++ warnings, patch by Hartmut Kaiser!
llvm-svn: 41685
2007-09-03 18:28:41 +00:00
Chris Lattner 4c4a245475 Use a smallstring instead of an std::string in FileChanged to avoid some malloc traffic.
This speeds up -E on xalancbmk by 2.4%

llvm-svn: 40461
2007-07-24 06:57:14 +00:00
Chris Lattner 5d1c02748f Change hte lexer to start a start pointer to the underlying
memorybuffer instead of a pointer to the memorybuffer itself.  This
reduces coupling and eliminates a pointer dereference on a hot path.
This speeds up -Eonly on 483.xalancbmk by 2.1%

llvm-svn: 40394
2007-07-22 18:44:36 +00:00
Chris Lattner 619c174561 split the slow path out of Lexer::getSourceLocation and do not let the
compiler inline it.  This speeds up -Eonly on 483.xalancbmk by about 1%

llvm-svn: 40393
2007-07-22 18:38:25 +00:00
Chris Lattner b9b8597c23 avoid recursion between SkipBCPLComment and SkipWhitespace. In cases like this:
// foo
   // bar
   // baz

we'd get two levels of call (bcpl & whitespace) for each line, leading to some
seriously deep stacks in some cases.

llvm-svn: 40384
2007-07-22 06:29:05 +00:00
Chris Lattner c850ad6ee1 Fix a lexer bug where we incorrectly rejected
int i = /*/ */ 1;

Thanks to Neil for pointing this out.

llvm-svn: 40379
2007-07-21 23:43:37 +00:00
Chris Lattner 146762e7a4 At one point there were going to be lexer and parser tokens.
Since that point is now long gone, we should rename LexerToken to
Token, as it is the only kind of token we have.

llvm-svn: 40105
2007-07-20 16:59:19 +00:00
Chris Lattner 77e9de50a1 simplify the lexer ctor to take a SLoc instead of a sloc and a redundant buffer*.
llvm-svn: 40104
2007-07-20 16:52:03 +00:00
Chris Lattner dc5c055fd1 Reimplement SourceLocation. Instead of having a
fileid/offset pair, it now contains a bit discriminating between
mapped locations and file locations.  This separates the tables for
macros and files in SourceManager, and allows better separation of
concepts in the rest of the compiler.  This allows us to have *many*
macro instantiations before running out of 'addressing space'.

This is also more efficient, because testing whether something is a
macro expansion is now a bit test instead of a table lookup (which
also used to require having a srcmgr around, now it doesn't).

This is fully functional, but there are several refinements and
optimizations left.

llvm-svn: 40103
2007-07-20 16:37:10 +00:00
Chris Lattner 23b7eb677d Finally bite the bullet and make the major change: split the clang namespace
out of the llvm namespace.  This makes the clang namespace be a sibling of
llvm instead of being a child.

The good thing about this is that it makes many things unambiguous.  The
bad things is that many things in the llvm namespace (notably data structures
like smallvector) now require an llvm:: qualifier.  IMO, libsystem and libsupport
should be split out of llvm into their own namespace in the future, which will fix
this issue.

llvm-svn: 39659
2007-06-15 23:05:46 +00:00
Chris Lattner ff591e24eb Don't warn about escaped newlines in // comments if the next line is also
a // comment, this reduces noise in the llvm testsuite.

llvm-svn: 39636
2007-06-09 06:07:22 +00:00
Bill Wendling 5a85f9104d Bug #:
Submitted by: Bill Wendling
Reviewed by: Chris Lattner

- Removed unneeded <iostream> header.

llvm-svn: 39484
2007-05-23 08:05:58 +00:00
Chris Lattner 739e739b81 Remove the clang::SourceBuffer class, switch to the llvm::MemoryBuffer class.
llvm-svn: 39426
2007-04-29 07:12:06 +00:00
Chris Lattner b352e3edb5 Change KeepComments/KeepMacroComments modes to be facets of the preprocessor
state, not aspects of the language standard being parsed.

llvm-svn: 39209
2006-11-21 06:17:10 +00:00
Chris Lattner 9f6604fa60 Add altivec version of block comment skipping code.
llvm-svn: 39093
2006-10-30 20:01:22 +00:00
Chris Lattner c07ba1fe2f Refactor the paths used for checking and getting the spelling of #include
filenames (and also '#pragma GCC dependency' of course).  Now, assuming
no cleaning is needed, we can go all the way from lexing the filename to
doing filename lookups with no mallocs.  This speeds up user PP time from
0.077 to 0.075s for Cocoa.h (2.6%).

llvm-svn: 39092
2006-10-30 05:58:32 +00:00
Chris Lattner 9916c5ca7e Remove GNU C++ min/max operator extension support, they have been removed
from gcc mainline.

llvm-svn: 39067
2006-10-27 05:24:37 +00:00
Chris Lattner 089e51923e remove todo
llvm-svn: 39061
2006-10-27 04:53:38 +00:00
Chris Lattner aded4a977d Implement an sse2 version of the block comment scanner.
llvm-svn: 39060
2006-10-27 04:42:31 +00:00
Chris Lattner 6cc3e36cd7 Speed up block comment skipping by 35%.
llvm-svn: 39059
2006-10-27 04:12:35 +00:00
Chris Lattner dd0b7cbf0b Oversight: fixes test/Lexer/number.c
llvm-svn: 38998
2006-10-17 02:53:51 +00:00
Chris Lattner 8c2048710d Rename LexerToken methods to be more consistent.
llvm-svn: 38969
2006-10-14 05:19:21 +00:00
Chris Lattner d3e9895b9a Initial support for semantic analysis and AST building for StringExpr nodes.
llvm-svn: 38960
2006-10-06 05:22:26 +00:00
Chris Lattner 457fc15bc5 Implement comment saving mode: the -C and -CC options.
llvm-svn: 38783
2006-07-29 06:30:25 +00:00
Chris Lattner a5f4c882e2 disable malformed string/character errors when in raw mode. This fixes
test/Lexer/badstring_in_if0.c

llvm-svn: 38751
2006-07-20 06:08:47 +00:00
Chris Lattner 5a78a02ea2 If an invalid string or character is read, return the invalid part as a tok::unknown token.
llvm-svn: 38749
2006-07-20 06:02:19 +00:00
Chris Lattner 538d7f3c27 Simplify "raw lexing mode" even further. Now the preprocessor is only called
into when a hard error is found.  This simplifies logic and eliminates the need
for the preprocessor to know about raw mode.

llvm-svn: 38746
2006-07-20 04:31:52 +00:00
Chris Lattner 0f1f50517b Simplify identifier lookup in raw mode, implementing Preprocessor/macro_fn_lparen_scan2.c.
llvm-svn: 38744
2006-07-20 04:16:23 +00:00
Chris Lattner 30a2fa14ae Move LexingRawMode handling of file EOF out of the preprocessor into the
lexer.  This makes more logical sense and also unbreaks the case when the
lexer hasn't been pushed onto the PP include stack.  This is the case when
pasting identifiers.  This patch implements macro_paste_bcpl_comment.c.

llvm-svn: 38736
2006-07-19 06:31:49 +00:00
Chris Lattner 2183a6e8f4 Make end-of-file handling much less recursive. This reduces the worst case
stack depth sampled by shark from ~34 to ~17 frames when preprocessing <iostream>.

llvm-svn: 38726
2006-07-18 06:36:12 +00:00
Chris Lattner 2b271db205 Lex the microsoft 'charize' extension.
llvm-svn: 38711
2006-07-15 05:41:09 +00:00
Chris Lattner ecc39e9325 Change Lexer::Stringify to not add ""'s around the string.
llvm-svn: 38708
2006-07-15 05:23:31 +00:00
Chris Lattner 041bef8b32 The lexer should not warn about stray characters, it should just return
tok::unknown tokens.  This fixes test/Lexer/unknown-char.c

llvm-svn: 38703
2006-07-11 05:52:53 +00:00
Chris Lattner 678c880a69 Move Preprocessor::isNextPPTokenLParen to Lexer::isNextPPTokenLParen, where
it more rightly belongs.

llvm-svn: 38702
2006-07-11 05:46:12 +00:00
Chris Lattner 3ebcf4e2cd Change Preprocessor::SkippingContents into Lexer::LexingRawMode. Raw mode
is an intra-lexer property, not a inter-lexer property, so it makes sense
for it to be define here.  It also makes no sense for macros, and allows us
to define it more carefully in the header.

While I'm at it, improve comments and structuring in Lexer.h

llvm-svn: 38701
2006-07-11 05:39:23 +00:00
Chris Lattner d8aee0e81b Implement "lparen scanning" for lexer buffers, by making "skipping lexing"
completely reversible.  This implements tests 3/4 of
test/Preprocessor/macro_fn_lparen_scan.c

llvm-svn: 38699
2006-07-11 05:04:55 +00:00
Chris Lattner eb54b5973e Add simple optimization: check for (and skip) spaces and tabs immediately
before lexing a token.  This speeds the common case where tokens are
separated by small amount of whitespace.  This makes a slight but
reproducible positive effect lexing a preprocessed carbon.h.

llvm-svn: 38691
2006-07-10 06:34:27 +00:00
Chris Lattner cefc768f5b Start reading/validating the argument list for a function-like macro.
llvm-svn: 38681
2006-07-08 08:28:12 +00:00
Chris Lattner 27746e476a Fix Preprocessor/macro_space.c, which failed because the ! token had
identifier info incorrectly set.

llvm-svn: 38674
2006-07-05 00:07:54 +00:00
Chris Lattner ef9eae1c44 Change the Preprocessor::getSpelling interface to let it be zero-copy in
the common case.

llvm-svn: 38671
2006-07-04 22:33:12 +00:00