Previously, only the string contents were stored in lexerState::token_str (i.e.
not including the delimeters). Now, the delimeters are stored as well, thus
preserving them. This also simplifies the code a bit.
A new function is added to handle the character storage, which is also now
applied for normal identifiers. To that end, the MAX_STRING_LENGTH was boosted
to 256 so that all reasonably sized identifiers may fit.
Previously, things like:
struct Foo
{
#[bar]
baz: int
}
or
struct Foo
{
#![bar]
baz: int
}
would horribly confuse the parser and prevent proper parsing of the rest of the
file.
Fix a race initialization leading to incorrect handling of preprocessor
directives on the first input line.
Test case contributed by Adam Hirst, thanks.
To do this we change the internal types to only use types matched by
symbols_get_current_scope(). This is a bit of a hack, but the tag
types were already questionable and simply mapped to ones the
TagManager know.
This however merges Functions and Subroutines under the same top-level
item in the symbol list.
Fortran 2003 allows '[' and ']' as array constructors in addition to
the legacy '(/' and '/)'. Fortran 2008 allows '[]' as part of a
CODIMENSION specifier, which can either be `codimension[]` in
the type-spec; or as either `variable[]` or `variable()[]` in
the entry-spec.
Credit to Colomban Wendling for the skipOverSquares logic, and
treating `codimension` as a special-case.
Last part of bug #1023.
This fixes 'Go to Tag definition' for parsed Python imports as before the import
statement was chosen as the definition while we prefer the class definition as
target.
Mitigate parse error with generics like `foo<X<Y> x;` by avoiding
matching past a semicolon (";") or open curly brace ("{"), which
can't be part of the generic. This isn't a proper fix, but mitigates
the impact of such constructs.
fileEOF() actually returns TRUE anywhere after the last newline when
using the file*() API, which includes a last line without newline.
This is an implementation detail which makes this function not usable
with the rest of the file*() API. fileEOF() should actually probably
be called iFileEOF(), but that's outside the scope of this parser.
However, even if fileEOF() did work properly with fileGetc(), the very
last byte in the input would have been omitted because we actually
read ahead by one byte, which means the actual read reaches EOF one
byte before our "current byte" does.
Checking whether we reached EOF by checking whether our current byte
isn't equal to `EOF` fixes it, and isn't worse since it's actually how
fileEOF() is implemented anyway.
When allocation with g_malloc(), the memory should be freed using
g_free(), not plain free().
Also, use g_try_malloc() instead of g_malloc() where the code carefully
handles allocation failures itself.
tm_tags_find() relies on a sorted tags array to be passed in but in
tm_source_file_set_tag_arglist() we don't have a sorted array yet and
sorting it on demand seems more heavy than the alternative:
make tm_tags_find() search the array linear if the new flag is set.
This fixes a bug in the Python parser when assigning the argument list
of __init__() methods to their class' argument list which annoyed me
for years already.
Also add a test case for this.
Even though PHP doesn't handle those very well (it emits warnings about
"unexpected character"), it still counts them as whitespaces, so
properly handle them as such.
