This fixes parsing indented titles and titles with embedded delimiter
characters, and adds support for title nesting information.
Syntax: http://txt2tags.org/rules.html
Closes [feature-requests:#690].
This also makes recent GCC shut up about indexing below the bounds as
it detected the code checked for a negative index yet didn't guard the
actual access. For now GCC doesn't understand the more comprehensive
check, but it might come back if GCC becomes smart enough.
Anyway, this makes the Assert() more correct, and addition of the
explicit kinds array size makes sure any future kind addition won't
miss its entry.
Rust now allows CRLF line endings in source files, which doesn't actually
change any lexing here but the comment was wrong.
Also, the hashbang initial comment has a special case where #![ doesn't count
as a comment (but instead an inner attribute that just happens to be on the
first line).
Previously, only the string contents were stored in lexerState::token_str (i.e.
not including the delimeters). Now, the delimeters are stored as well, thus
preserving them. This also simplifies the code a bit.
A new function is added to handle the character storage, which is also now
applied for normal identifiers. To that end, the MAX_STRING_LENGTH was boosted
to 256 so that all reasonably sized identifiers may fit.
Previously, things like:
struct Foo
{
#[bar]
baz: int
}
or
struct Foo
{
#![bar]
baz: int
}
would horribly confuse the parser and prevent proper parsing of the rest of the
file.
Fix a race initialization leading to incorrect handling of preprocessor
directives on the first input line.
Test case contributed by Adam Hirst, thanks.
To do this we change the internal types to only use types matched by
symbols_get_current_scope(). This is a bit of a hack, but the tag
types were already questionable and simply mapped to ones the
TagManager know.
This however merges Functions and Subroutines under the same top-level
item in the symbol list.
Fortran 2003 allows '[' and ']' as array constructors in addition to
the legacy '(/' and '/)'. Fortran 2008 allows '[]' as part of a
CODIMENSION specifier, which can either be `codimension[]` in
the type-spec; or as either `variable[]` or `variable()[]` in
the entry-spec.
Credit to Colomban Wendling for the skipOverSquares logic, and
treating `codimension` as a special-case.
Last part of bug #1023.
This fixes 'Go to Tag definition' for parsed Python imports as before the import
statement was chosen as the definition while we prefer the class definition as
target.
Mitigate parse error with generics like `foo<X<Y> x;` by avoiding
matching past a semicolon (";") or open curly brace ("{"), which
can't be part of the generic. This isn't a proper fix, but mitigates
the impact of such constructs.
fileEOF() actually returns TRUE anywhere after the last newline when
using the file*() API, which includes a last line without newline.
This is an implementation detail which makes this function not usable
with the rest of the file*() API. fileEOF() should actually probably
be called iFileEOF(), but that's outside the scope of this parser.
However, even if fileEOF() did work properly with fileGetc(), the very
last byte in the input would have been omitted because we actually
read ahead by one byte, which means the actual read reaches EOF one
byte before our "current byte" does.
Checking whether we reached EOF by checking whether our current byte
isn't equal to `EOF` fixes it, and isn't worse since it's actually how
fileEOF() is implemented anyway.
