X-Git-Url: https://git.hcoop.net/bpt/coccinelle.git/blobdiff_plain/1be43e1299fc61538d62349ca012514b28f8734f..8f657093d128c6436330659d273c2762ac9cbf79:/changes.txt diff --git a/changes.txt b/changes.txt index 0a0013d..25b4ce4 100644 --- a/changes.txt +++ b/changes.txt @@ -1,17 +1,658 @@ -*- org -*- +* upcoming/planned/todo +** Language: +- Semantic patch for other languages +- Support metavariables, virtual rules and virtual identifiers + in initialize and finalize rules +- Support for matching #ifdef, etc. +- function-typed metavariables + +** Features: +- New scripting languages + +** Bugfix: +- Add more information in documentation, man pages and wiki + +* 1.0.0 +** Language: +- Addition of initializer list metavariables. Thanks to Michael Stefaniuc + for noticing the need for them. +- Allow multiple position variables per token + +** Features: +- Preserve spacing before // comments when not at the beginning of a line + +** Bugfix: +- Corrected parsing of script strings delimited by a single quote. Thanks + to Eugeniy Meshcheryakov for reporting the problem. +- Improved indentation of added ifdefs. +- Allow added comments and #ifdefs to precede other added code. Thanks to + Jani Monoses for noticing the problem. + +* 0.2.5 +** Language: +- Make a very small attempt to parse C++ code, amounting to accepting + identifiers containing ::, tilde, and template invocations. Use the + option -c++. This is not likely to be very useful in practice. +- Added metavariable metavariable type. +- Add disjunctions on identifiers in some contexts +- Pretend that & indicates a pointer in -c++ mode +- Support for new and delete +- Allow arrays in smpl parameter declarations. Thanks to Jesper Louis + Andersen for pointing out the problem. +- Field list metavariables +- Add the ability to add __attributes__ (NB, no matching on attributes) +- Slightly improved the error message for transformation on multiple + variable declaration. Thanks to Jonathan Nieder for pointing out the + problem. + +** Features: +- support transformations on variables (only) in declarations that declare + multiple variables +- allow #endif XXX in C code +- relax_include_path now applied to non local includes too, in which case + it tries to find a unique file with a suffix of the provided name. + this is useful for directories that are intended to be symbolic links. +- support matching and removing #undef +- support for iteration in ocaml, requires use of -no_show_diff +- calls to likely and unlikely propagate test expression status to their + arguments +- reuse typedefs and macros from complete parsing when reparsing the + transformed code +- better explanation when presenting glimpse tokens +- optimization for an if branch that is just { ... } +- spatch -control_flow_to_file file.c generates a file file.dot +- include files with names ending in .cpp if -c++ option is given +- removed use of the no longer supported sexplib function Conv.hashtbl_of_sexp +- add information about which Fedora packages are needed + +** Bugfix: +- improved parsing of expressions to allow ... to the right of operators in + more places +- Fix check_config for Python 2.7 on Fedora 14 (Reported-by: Michael Stefaniuc) +- Check for ocamlfind in configure (Reported-by: Paul E. McKenney) +- Postpone use of ocamlfind at runtime to report fewer errors +- Add support for Python 2.4 binding with the provided pycaml library +- Allow '@' in script code. Thanks to Laurent Reveillere for noticing the + problem. +- Remove an optimization of x or true to true in asttoctl2 for the case + where x may make a modification, eg the case of -thing ?-thing +- Allow adding comments before a function. +- Introduction of newlines in function arguments needs to take into account + that the code added by SmPL can include newlines. Thanks to Thomas + Gleixner for finding the problem. + +* 0.2.4 +** Language: +- scripts now have names and can declare metavariables that are seen by + SmPL as identifier metavariables, see demos/{python,ocaml}tococci.cocci +- declaration (match variable declarations) and field (match structure + field declarations) metavariables +- matching of array initializations and of enum declarations + +** Features: +- compatability with Python 2.7. Thanks to Richard W.M. Jones for pointing + out the problem +- python and ocaml string representations of parameter lists and expression + lists +- try simpler patterns on glimpse failure +- graceful and immediate failure when a virtual rule is defined on the + command line, but not in the rule +- spatch returns -1 when a virtual rule is requested that is not supported +- no transformation when using * with the option -no_show_diff +- expanded +++ file name for match output +- struct *^* and enum *^* metavariable types +- allow an assignment to match a variable initialization, even if multiple + variables are declared at the same time. +- add the option -recursive_includes +- added the option -use_idutils. The requires the previous use of the + script idutils_index.sh + +** Bugfix: +- detect used after metavariables in simple statement metavariable match +- allow inheriting position variables over rules that make transformations + but depend on virtual rules that are not defined on the command line +- treat top-level tokens in decl visitor. Thanks to Peter Tummeltshammer + for noticing a resulting problem. +- improve typing of the result of an assignment expression +- eliminate some recursive calls in C code parsing to avoid stack overflows +- better pretty printing of #define when the whole line is removed +- better pretty printing of asm code containing + "::[input]"r"(&coherence_data[i])" +- allow pretty printing of C code with nameless bit fields +- avoid adding an error message to standard output if ocamlfind is not + found +- struct/union/enum-typed metavariables with a metavariable type name +- better management of whitespace between deleted lines. Thanks to + Andriy Gapon for noticing the problem. +- improved parsing recovery for badly parsed struct/enum typedefs +- drop expanded tokens in sgrep output. Thanks to Andriy Gapon for + noticing the problem. +- constant strings have array type, not pointer type. Thanks to Vasiliy + Kulikov for noticing the problem. +- improve indentation when adding code after a function call with indented + arguments. +- return type of sizeof converted to unsigned long. Thanks to Vasiliy + Kulikov for noticing the problem. +- improve spacing when adding structure initializers. Thanks to Vasiliy + Kulikov for noticing the problem. + +* 0.2.3 +** Language: +- <= constraints relating an expression metavariable to one or more + inherited expression metavariables that it should be a subexpression of +- the -inplace and -outplace options have been renamed -in_place and + -out_place. -in_place no longer makes a backup. A backup can be + requested using the option -backup_suffix, eg -backup_suffix .bk +- identifiers can be constrained to be different from an inherited + identifier metavariable, or from a set containing both concrete + identifiers and inherited identifier metavariables. +- support for ocaml scripting +- ast available in ocaml scripts + +** Features: +- drop inconsistent paths check when only one node was matched +- allow #define id with no definition +- attempt to add newlines in the generated code when function calls in the + smpl cause passing column 80 +- allow - in front of statement level nests, if everything inside the nest + is also - +- 'configure' looks for some system libraries: menhirLib, sexplib, pycaml. + Not available libraries are replaced by a copy provided with the sources. +- added static for a function goes just before what is specified in the + semantic patch, not before any comments etc. +- Add a new option, -ignore_unknown_options, to ease the integration + of Coccinelle as a checker in a toolchain. +- error in python code causes immediate abort of spatch +- use the same algorithm for collecting grep tokens as for collecting + glimpse tokens +- Add scripts/spatch.bash_completion for automatic completion of common + options under the bash shell. +- Print rule name when a script rule crashes +- Allow the declaration of an expression list or parameter list + metavariable to specify the number of expressions or parameters, + respectively. + +** Bugfix: +- drop inconsistent paths check when only one node was matched +- better take into account virtual rules when selecting glimpse tokens +- print diff when only a comment is added +- the type of the C code !x should be int regardless of the type of x +- allow python code at the beginning of an included file +- better adjustment of whitespace when deleting adjacent lines. Thanks to + Wolfram Sang for pointing out the problem. +- allow metavariables to be inherited from included files +- allow ? on goto +- more graceful failure on finding a strange character in a macro parameter + list +- support Parameter metavariables +- add space after the last comma in an added portion of an argument list +- SmPL nameless struct should only match a nameless struct, not a nameless + union. Thanks to Peter Tummeltshammer for pointing out the problem. +- adjustments to storage don't modify inline as well +- matching and transformation allowed on inline +- allow removed field between two ... in structure initialization +- allow removing an entire structure initialization +- manage labels within do while 0 macros +- added space after : in printing a conditional expression. Thanks to Josh + Triplett for noticing the problem. +- dropped trailing whitespace when code at the end of a line is deleted. + Thanks to Josh Triplett for noticing the problem. +- Fixed environment management when script rule uses cache. Thanks to + Bissyande for finding the problem. +- when false matches if with no else +- ensure { ... } with whencode checks to the end of the function. Thanks to + Rene Rydhof Hansen for finding the problem. + +* 0.2.2 +** Language: +- Added ToTestExpression to iso language, see standard.iso for an + explanation +- Added depends on to initialize and finalize script code + +** Features: +- Update Emacs mode (cocci.el) to support 'virtual' rules and fix other + keywords sush as 'depends on', 'using', 'disable' +- better treatment of != 0 in isos, communtativity for ==/!= for all + constants +- allow adding // comments and blank lines (even after cocci + code) +- Add support for multiple -I options + +** Bugfix: +- correct interaction between virtual rules and included .cocci files +- improvement in treatment of ! in isos, to avoid duplicating + code +- improvement in treatment of metavars as isos, to avoid duplicating + code + between toplevel and variable instantiation +- test expression of smpl conditional, etc no longer assumed to have type + int +- correct + line numbers in the patch produced when using * +- iso constant metavariable matches an identifier whose name is all capital + letters, eg NULL +- allow / at the end of the name of a directory (-dir) or patch prefix + (-patch) +- dropped space in + code after the binding of a type metavariable that is + a pointer type +- better handling of . or .. in -dir name +- allow keywords and metavariable names in identifier constraints (not sure + keywords is very useful, though) +- no lubtype on arguments of && and || in SmPL +- allow unknown as type for array indices in SmPL +- support matching of static annotation on functions that are both static + and inline +- support ENOTDIR error in Common.lfile_exists, to allow for the case where + an include file is in a subdirectory that exists but is an ordinary file, + not a directory. +- better management of unbound position variables that appear in + constraints +- cause python parser to skip over // comments, hoping that // is not + meaningful inside python +- require + on every line of a multiline comment +- correct calculation of line numbers when there is script code +- always annotate arguments of && || and ! as test expressions in C code. + +* 0.2.1 +** Language: +- Add virtual identifiers +- Add coccilib.report and coccilib.trac Python modules +- coccilib.org and coccilib.report are imported by default but not loaded + in the current scope. + +** Features: +- Parse_error_msg now more helpful. New option -verbose_parsing for even + more information. +- Improve Python import handling. They are imported once during script + initialization. + +** Bugfix: +- correct treatment of depends on with || for virtual rules + +* 0.2.0 +** Language: + +** Features: +- Remove duplicated code in disjunctions +- Better error message when grep finds nothing relevant. Thanks to Joe + Perches for the suggestion. +- added -keep_comments option for the unparsing of the transformed code +- Option "-version" now also gives information about built-in + Python binding support. +- slightly faster environment manipulation in pathological cases +- hack added to accept well-formed #define after function header + +** Bugfix: +- Proper consideration of #define macro arguments in checking for the use + of metavariables and in computing the line numbers of complex terms +- Better parsing of included .cocci files +- Put included .cocci files in the right order +- Bind position variables only once for #include +- Fix bug in include_match that caused everything to halt when all matches + were discarded +- Merge unlikely/likely iso rules under a iso rule named unlikely +- Some fixes to coccicheck rules, thanks to Andrew Lunn +- Support groups in regular expression, thanks to Michael Stefaniuc + +* 0.1.11 + +** Language: +- Meta-identifier/function and constant could be filtered from SmPL by + regular expressions using the "~=" and "!~=" operators followed by + an OCaml regular expression (see man Str) in double quote. +- Virtual rules, which can be referenced in depends, and set and unset + using the command-line options -D +- ++ for multiple additions + +** Features: +- coccicheck: a framework to check a series of SmPL files on a project + see scripts/coccicheck for more information + +** Bugfix: +- bind position variables to the correct starting position in the case of a + complex statement such as an if or while. Thanks to Derek Jones for + pointing out the problem. +- checking for non mentioned case lines in switch should be unchecked. +- space should be printed after sizeof when there are no parentheses around + the argument. Thanks to Daniel Walker for pointing out the problem. +- avoid introducing sharing in propagating ! over () in treatment of isos +- save_excursion has to handle and rethrow exceptions +- eliminate unnecessary consideration of CVS strings. Thanks to David + Young for pointing out the problem. +- completely new treatment of statement metavariables +- better type checking for macro definitions +- drop regression testing in -parse_c + +* 0.1.10 + +** Language: +- declarations allowed in switch, suggested by Derek M. Jones + +** Features: +- use interval timer for timeouts. Thanks to Derek M. Jones for the + implementation. +- more flexible env.sh, thanks to Derek Jones. +- faster Python invocation +- simplify unparsing in the sgrep case + +** Bugfix: +- for glimpse there is no point to create a pattern containing a numeric + constant because glimpse doesn't index them +- add spaces after commas in function calls and function headers +- made python integration more like the ubuntu version + http://patches.ubuntu.com/by-release/extracted/debian/c/coccinelle/0.1.5dbs-2/01-system-pycaml + this fixes some memory management problems with None, True, and False +- correct labels associated with a switch in the CFG so that a statement + metavariable can match a switch. Thanks to Derek Jones for pointing out + the problem. +- keep switch pattern within switch body. Thanks to Derek Jones for + pointing out the problem. +- Allow expanded tokens to be stored in metavariables, as long as they are + not removed. But this does currently allow them in + code, which will + produce the expansion. Thanks to Ali-Erdem Ozcan for pointing out the + problem. +- improved adjustment of spacing when code removed at the beginning of a line + +* 0.1.9 + +** Language: +- allow fresh identifiers to be declared using ## such that the value mixes + both strings and previously declared metavariables + +** Features: +- better handling of expanded code containing ##. Now compute the + result. +- more precise warning message for the "'\' outside define". + Thanks to Nicholas Mc Guire for pointing out the problem. +- more precise warning message related to ifdefization. + Thanks to Derek Jones for pointing out the problem. +- we don't create any more certain files in /tmp (they may be a cause + of security problems). + Thanks to Eugeniy Meshcheryakov for pointing out the problem. +- More optimization for the case of just deleting a complete function. + Allows this to happen without tracing through all the control-flow + paths. Thanks to Francois Bissyande for pointing out the problem. +- prevent code from being added to the beginning or end of a disjunction +- more information about why a script is not applied when using -debug option +- added -no_safe_expressions option +- added -no_loops option. Ignores back edges derived from looping + constructs. This is unsafe, but perhaps useful for bug finding, as it can + be more efficient. +- for semantic matches, allow "minus" on same code with multiple + environments +- better error message for mismatch of parenthesis in column 0 with normal + parenthesis. Thanks to Derek Jones for pointing out the problem. +- allow disjunctions on function return types. Thanks to Pierre Habouzit + for pointing this out. + +** Bugfix: +- keep disjunction in the proper order for structure initialization fields +- variables declared in different places should not seem to match each + other +- drop complaints about label metavariables not being used +- drop test information from the type of an expression when the expression + is bound to a metavariable +- nests should not extend beyond the before and after code, even if the + before and after code matches the nest code +- nests should extend into conditionals that end in error exit +- take into account metavariables on "else". Thanks to Derek Jones for + pointing out the problem. +- print single quotes on generated character constants +- better typedef handling in the initialisation/affectation builtin + isomorphism, cf -test init_affect_typedef +- support disjunction of types on variable declaration +- allow @ within strings in script code. ignore // comment lines in script + code. +- don't drop + code placed after the transformed code +- drop spaces produced by removing code before semicolons +- adjusted spacing within generated code +- less verbose -sp. Thanks to Derek Jones for pointing out the problem. +- accept multiple type names in a SmPL typedef declaration. + +* 0.1.8 + +** Language: + +** Features: +- Metavariables now capture the cpp code contained within their definitions +- When - fragments are separated by ... or nest boundaries in the semantic + patch, but end up matching adjacent source code, the comments, cpp code + and whitespace that are between them are not deleted. +- better parsing of C: do expansion of macros only when needed when + have actually a parse error and also leverage the definition of macros + in the parsed file (or in a optional_standard.h file passed as a parameter). + This should reduce the need for many hardcoded definitions in standard.h +- new semantics for the -macro_file option, by default now expand macros + only when necessary. To force use the -macro_file_builtins option instead. +- a new -extract_macros command line action to help the parser. Works with + the -macro_file option. e.g. + $ ./spatch -extract_macros ~/linux > /tmp/alldefs.h + $ ./spatch -macro_file /tmp/alldefs.h -sp_file foo.cocci -dir ~/linux +- removed -D macro_file option, not consistent with what -D usually means +- reattempt to be more efficient for statement metavariables that are just + placeholders (ie, no modification, no reuse) +- triples now returned from ctl in sorted order. The main key is the + state. On the other hand, the state order does not always agree with the + order of appearance in the code. +- spatch is now less verbose on the things it does yet handle. Less confusing + for new users. +- slightly better error report. Thanks to Derek Jones for the suggestion. +- added the options -linux_spacing and -smpl_spacing. -linux_spacing + causes spatch to follow the spacing conventions of Linux, while + -smpl_spacing causes spatch to follow the spacing in the semantic patch. + -linux_spacing is the default. +- more informative error reporting for the already tagged token case. + Thanks to Erik Hovland for the suggestion. + +** Bugfix: +- better parsing of declare macro at toplevel and in structure. + cf -text xfield +- allowing back typedef names for fieldname +- better printing of else in generated code +- slightly better type inference for binary operators. +- clear out declarer names and iterator names between SmPL files (for + -testall) +- better parsing and type checking of macro type aliases. Cf -test macro_int16. + Thanks to Li Yingtong for pointing out the problem. +- make insert_virtual_positions tail rec, avoid stack overflow pb. + Thanks to Diego Liziero for pointing out the + problem. +- Better type inference for arithmetic binary operators + Thanks to Li Yingtong for pointing out the problem. +- Better type inference for constants + Thanks to Li Yingtong for pointing out the problem. +- move computing of adjacency information for semantic patches to after + application of isomorphisms, because isomorphisms can introduce "..." +- compute adjacency information for negated ... +- record with each transformation site the set of indices of the witness + trees that caused the transformation site to come about. Whitespace and + comments between remove tokens associated with disjoint witness trees is + not removed. +- correct treatment of function pointer typed parameters in the SmPL ast0 + visitor. +- better parsing error message and error recovery when comments are not ended, + when some macros have a weird body, and when some switch have a weird + Body. + Thanks to Derek Jones for pointing out the problem. +- better detection and passing of "dangerous" ifdefs, cf -test + double_switch. +- dropped the separation of decls and body in Seq. This gives better + positioning of the bindings of metavariables shared between them. + Thanks to Erik Hovland for an example that shows the problem. + +** Internals: +- supress warning in compiling ocamlsexp (warning caused by a new behavior + of cpp used internally in processing files in ocamlsexp/) + +* 0.1.7 + +** Language: +- initialize and finalize script code, cf demos/initial_final.cocci + +** Features: +- -iso_limit option to limit the depth of isomorphism application +- with the dir option, the include path is implicitly set to the "include" + subdirectory of the specified directory, if the option -I is not used. +- give a seed for the name of a fresh identifier +- better handling of cpp "constructed" identifiers as in a##b, that in + the futur will make it easier to match over those idents. + cf tests/pb_parsing_macro.c. Thanks to Ali-Erdem Ozcan for pointing + out the problem. A new "parsing hack hint" is also available: + YACFE_IDENT_BUILDER, cf standard.h. + +** Bugfix: +- drop excessive "optimization" in ctl generation for while and for loops +- allow . as the name of the directory +- for type inference for an assignment, take the type of the right-hand + side expression, not the type of the assigned variable +- allow for with a declartion in the first header element, as in C++ + (partial support) +- allow for matching against variable declarations that include only + storage, eg static, but no type at all. +- allow for matching against types that contain both short/long and int +- allow the type metavariable in the SmPL code "unsigned T" to match a T + that is a type consisting of more than one word, eg long int. +- -ifdef_to_if option made to process nested ifdefs + (partial support) + +** Internals: +- improve and fix installation process (usable on BSD) +- improve and fix testing process +- apply patches from Eugeniy Meshcheryakov +- reorganize the way we parse C identifiers, especially concatenated cpp + identifiers as in a##b. This may lead to some regressions as we may + not parse as much code as before. +- removed popl/ and popl09/ and popl related stuff from official distrib. + +* 0.1.6 + +** Language: +- the ability to add comments + +** Features: +- grouping of generated rules with -hrule option +- handling of special coccinelle comments + /* {{coccinelle:skip_start}} */ and + /* {{coccinelle:skip_end}} */ + allowing to give more hints to the C parser. + Thanks to Flavien@lebarbe.net for the idea. +- the ability to print the values of more (but not all) kinds of + metavariables from python +- new vim SmPL mode. + Thanks to Alexander Faroy. + +** Bugfix: +- consider the ident tokens also in the 2 lines before the error line for the + 10-most-problematic-parsing-errors diagnostic. +- SmPL parser allows cast as the argument of a pointer +- SmPL type checker allows enum as an array index +- Better generation of fresh metavariables names in hrule +- no more warnings about things that should be metavariables when there is + a disjunction in a function position +- bugfix in parser, better error message. + Thanks to Ali-Erdem OZCAN for the bug report. + +** Internals: + +* 0.1.5 + +** Language: +- added initialiser metavariable +- added sequences of designators in structures + +** Features: +- improved printing of the C code corresponding to metavariables +- improved printing when code (eg declarations) is removed at the beginning + of a block, and then is followed by a blank line +- slightly less verbose error reporting in parsing_hacks + +** Bugfix: +- fixed some problems with parsing SmPL code where a nest appears after a | +- better treatment of { }, form in macros wrt unparse_c +- less quiet for -parse_c +- improve parsing heuristics regarding macro statement + +** Internals: + +* 0.1.4 + +** Language: +- long long added to SmPL + +** Documentation: +- add grammar reference and spatch command line options reference + +** Features: +- can match patterns of the form unsigned T or signed T, where T is a + metavariable +- dropped the sizeof_parens isomorphism, which was made redundant by the + paren isomorphism +- simple rule generation + +** Bugfix: +- trailing , ; and ) no longer left on a line by themselves +- better treatment of error exit when the searched for code matches the + error exit condition. +- fix incorrect treatment of parentheses in test_exps that could allow + + code to be added twice +- don't ask whether iterator names and declarer names should be declared as + metavariables. +- slightly better support for expression list metavariables. +- short and long allowed for array index types in SmPL +- more restrictions on type inference for pointer arithmetic in SmPL +- allow isomorphisms to apply when + code is anywhere within all - terms +- changed order of printing const and volatile +- allow eg ... <... in plus code +- better formatting of generated if/while/etc. code +- better parse error reporting when the problem is at the end of the file + +** Internals: +- isomorphisms don't apply under signed/unsigned, to prevent the creation + of things like unsigned signed int. Need a better solution for this. + +* 0.1.3 + +** Features: +- help in building the configuration macro file. The -parse_c action + now returns the 10 most frequent parsing errors. This give useful + hints to extend standard.h. + +** Bugfix: +- positions no longer allowed on \(, \|, and \) +- improved propagation of negation for isos in the presence of parens +- convert Todos in flow graph construction to recoverable errors +- fixed bug in treatment of when != true and when != false, to allow more + than one of them per ... +- improve parsing of typedef of function pointer. +- improve typing. +- parsing and typing support for old style C function declaration. +- consider position variables as modifications when optimizing the + translation into CTL of function definitions + +** Internals: + +* 0.1.2 + +** Bugfix: +- better handling of ifdef on statements in control flow graph. +- transform files even if they do not end in .c (thanks to Vegard Nossum) + +** Internals: +- merge code of yacfe + * 0.1.1 -** better support for initializer, cf -test substruct +** Language: + - support for initializer at toplevel, cf -test substruct * 0.1 ** first public release of the source code: -** Features +** Language: - embeded python scripting - position +** Features + * beta ** first public release of the binary