1. Don't include the newline itself in a line continuation construct.
This fixes generation of e.g. properties with embedded line
continuations.
2. Don't continue parsing strings past an unescaped newline (as naked
newlines are invalid inside strings). This avoids parsing the whole
remaining file as a string in case of broken input. It is both
useful to better support partly written files and to avoid loading a
whole malformed file in memory while reading it as a string.
See section 7.8.4 "String Literals" of ECMA-262:
http://www.ecma-international.org/publications/files/ECMA-ST/Ecma-262.pdf
Make go one of the builtin filetypes, add the parser and update the related
source and config files. While there, remove Rust from [Groups] in
filetype_extensions.conf because it's already a builtin filetype as well.
The parser itself is stolen from the fishman/ctags repo.
`let` is not yet part of the current ECMAScript standard but is part of
the ECMAScript 6 draft and is supported by Mozilla, and people already
use it in some contexts.
Also, the current ECMAScript standard marks `let` as a "considered
reserved word" (7.6.1.2), so it is already a reserved keyword in strict
mode.
See https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/let
Add support for implicit semicolons so many unterminated statements'
end are properly recognized.
The implementation doesn't follow the ECMAScript standard because doing
so requires to recognize precise grammar of all constructs, and the
parser doesn't currently work this way. So instead it uses some
heuristics that should work most of the time and only consider implicit
semicolons where they would be explicitly relevant to avoid most false-
positives. See the extensive comment in `readTokenFull()` for details.
In practice, this mostly fixes handling of files using unterminated
variable assignations like the following:
var v1 = 0
var v2 = 1
// ...
function f1() {
// ...
}
In such situations the parser used not to be able to really tell where
the variable assignation would end and would not recognize any
statement before the next semicolon or closing curly brace at the same
level. In practice, it wouldn't have emitted any tag for this example,
not even `v1` as it generates tags when reaching the statement's end.
Properly match open curly braces when parsing a statement not to
possibly get fooled by unexpected nested blocks, e.g. after a
`switch`'s `case` or a label.
This mostly reverts c54c3ad5e815d16e3b48f3c477465627808aadee and
replaces it with a more correct and complete solution.
Some highlights:
* Fixes handling of comments
* Adds support for attribute and namespace selectors
* Adds support for @supports blocks
* Fixes tag type for many selectors
* Adds support for pseudo-classes with arguments
By loading e.g. a huge DB dump into memory we could run out of memory.
Check the size of the file and determine whether to use file parsing
or buffer parsing.
Give up early if LANG_IGNORE set - there's no need to read the file in
this case.
Use the same (or compatible) sorting criteria everywhere.
Add tm_tag_attr_line_t to sort options so even after merging file tags
into workspace tags, the same tags defined at different lines are preserved
and not removed as duplicates.
Sort type before scope because it's cheaper to compare (string vs int comparison).
For some reason, the above changes make the sorting performance worse.
Simplify the tag comparison function a bit and reorder the case statements
in the switch to match the sort order. This (again not sure why), makes the
performance like before.
Previously, after finishing the while loop TRUE was returned - this is
wrong because the while was running because parsing was unsuccessful.
Make it work the same way as in ctags (parser() always succeeds,
parser2() returns whether to retry or not).
(The return value actually isn't used, it's just to make sure we know
what we are doing.)
With the previous TMWorkspace API it was possible to make the workspace
inconsistent by e.g. removing source files and forgetting to update
workspace. This could lead to non-obvious and not immediately visible
crashes.
The new set of the public (but also Geany private) API calls always
updates the workspace accordingly and neither of the calls can lead
to an inconsistent state of the workspace.
In addition, perform some minor cleanups and simplifications - unify
parsing from buffer and from file, support "parsing" of 0-sized buffers
and improve documentation.
Manage the list the same way as workspace tags_array by the fast tag removal
and merge. Thanks to this, typename tags don't have to be extracted from
tags_array periodically, which speeds up editing.
Even though the binary search requires expensive string comparisons,
there are just log(n) of them to find the tag in the workspace array
and the result is much faster than scanning the array linearly (this
of course works only under the condition that
len(source_file->tags_array) << len(workspace_array)
This is however satisfied for big projects (and doesn't matter for small
projects).
Also make the tm_tags_find() function more user friendly by returning
tagCount 0 when no tags found.
Instead of qsort() it's possible to use g_ptr_array_sort_with_data() with
similar performance. Unfortunately it seems there's no bsearch_with_data()
anywhere so the patch uses a modified bsearch() implementation from libc
(still probably cleaner than passing arguments using static variables).
Lazy initializing various member pointers doesn't bring any real performance
improvement but it requires lots of additional NULL checks - get rid of
this.
Make some more cleanups on the way.
In addition, remove success/failure return values from tm_workspace_add_source_file()
and tm_workspace_remove_source_file() which have no real use.
Avoid "utility" parameters like do_free for which we already have API calls
and which actually don't perform any free if the source file isn't
in TM. Clarify when to set the update_workspace parameter.
The placement of this function in tm_source_file is not right - by moving
it to the workspace we can make the source file unaware of the existence
of the workspace (no inclusion of tm_workspace.h in tm_source_file any
more). Also change tm_source_file_new() so it doesn't offer the source file
update.
After this change
* TMWorkspace knows TMSourceFile and TMTag
* TMSourceFile knows TMTag
* TMTag knows TMSourceFile
