@c -*-texinfo-*-
@c This is part of the GNU Emacs Lisp Reference Manual.
-@c Copyright (C) 1990, 1991, 1992, 1993, 1994, 1995, 1998, 1999, 2004
-@c Free Software Foundation, Inc.
+@c Copyright (C) 1990, 1991, 1992, 1993, 1994, 1995, 1998, 1999, 2001,
+@c 2002, 2003, 2004, 2005, 2006, 2007 Free Software Foundation, Inc.
@c See the file elisp.texi for copying conditions.
@setfilename ../info/syntax
@node Syntax Tables, Abbrevs, Searching and Matching, Top
@chapter Syntax Tables
-@cindex parsing
+@cindex parsing buffer text
@cindex syntax table
@cindex text parsing
@node Syntax Descriptors
@section Syntax Descriptors
-@cindex syntax classes
+@cindex syntax class
This section describes the syntax classes and flags that denote the
syntax of a character, and how they are represented as a @dfn{syntax
@dfn{Punctuation characters} (designated by @samp{.}) are those
characters that are used as punctuation in English, or are used in some
way in a programming language to separate symbols from one another.
-Most programming language modes, including Emacs Lisp mode, have no
+Some programming language modes, such as Emacs Lisp mode, have no
characters in this class since the few characters that are not symbol or
-word constituents all have other uses.
+word constituents all have other uses. Other programming language modes,
+such as C mode, use punctuation syntax for operators.
@end deffn
@deffn {Syntax class} @w{open parenthesis character}
other characters in the quotation.
@end deffn
-@deffn {Syntax class} @w{escape}
+@deffn {Syntax class} @w{escape-syntax character}
An @dfn{escape character} (designated by @samp{\}) starts an escape
sequence such as is used in C string and character constants. The
character @samp{\} belongs to this class in both C and Lisp. (In C, it
(@samp{;}) starts a comment and a newline or formfeed ends one.
@end deffn
-@deffn {Syntax class} @w{inherit}
+@deffn {Syntax class} @w{inherit standard syntax}
This syntax class does not specify a particular syntax. It says to look
in the standard syntax table to find the syntax of this character. The
-designator for this syntax code is @samp{@@}.
+designator for this syntax class is @samp{@@}.
@end deffn
@deffn {Syntax class} @w{generic comment delimiter}
table. This is for the sake of C++. Each style of comment syntax has
its own comment-start sequence and its own comment-end sequence. Each
comment must stick to one style or the other; thus, if it starts with
-the comment-start sequence of style ``b'', it must also end with the
-comment-end sequence of style ``b''.
+the comment-start sequence of style ``b,'' it must also end with the
+comment-end sequence of style ``b.''
The two comment-start sequences must begin with the same character; only
the second character may differ. Mark the second character of the
@samp{p} identifies an additional ``prefix character'' for Lisp syntax.
These characters are treated as whitespace when they appear between
expressions. When they appear within an expression, they are handled
-according to their usual syntax codes.
+according to their usual syntax classes.
The function @code{backward-prefix-chars} moves back over these
characters, as well as over characters whose primary syntax class is
@code{char-syntax}.
@end defun
-@defun syntax-after pos
-This function returns a description of the syntax of the character in
-the buffer after position @var{pos}, taking account of syntax
-properties as well as the syntax table.
-@end defun
-
@defun set-syntax-table table
This function makes @var{table} the syntax table for the current buffer.
It returns @var{table}.
the current buffer.
@end defun
-@defmac with-syntax-table @var{table} @var{body}...
-@tindex with-syntax-table
+@defmac with-syntax-table @var{table} @var{body}@dots{}
This macro executes @var{body} using @var{table} as the current syntax
table. It returns the value of the last form in @var{body}, after
restoring the old current syntax table.
@section Syntax Properties
@kindex syntax-table @r{(text property)}
-When the syntax table is not flexible enough to specify the syntax of a
-language, you can use @code{syntax-table} text properties to override
-the syntax table for specific character occurrences in the buffer.
-@xref{Text Properties}.
+When the syntax table is not flexible enough to specify the syntax of
+a language, you can use @code{syntax-table} text properties to
+override the syntax table for specific character occurrences in the
+buffer. @xref{Text Properties}. You can use Font Lock mode to set
+@code{syntax-table} text properties. @xref{Setting Syntax
+Properties}.
The valid values of @code{syntax-table} text property are:
@defun skip-syntax-forward syntaxes &optional limit
This function moves point forward across characters having syntax
-classes mentioned in @var{syntaxes} (a string of syntax code
+classes mentioned in @var{syntaxes} (a string of syntax class
characters). It stops when it encounters the end of the buffer, or
position @var{limit} (if specified), or a character it is not supposed
to skip.
@end defun
@node Parsing Expressions
-@section Parsing Balanced Expressions
+@section Parsing Expressions
- Here are several functions for parsing and scanning balanced
+ This section describes functions for parsing and scanning balanced
expressions, also known as @dfn{sexps}. Basically, a sexp is either a
-balanced parenthetical grouping, or a symbol name (a sequence of
-characters whose syntax is either word constituent or symbol
-constituent). However, characters whose syntax is expression prefix
-are treated as part of the sexp if they appear next to it.
+balanced parenthetical grouping, a string, or a symbol name (a
+sequence of characters whose syntax is either word constituent or
+symbol constituent). However, characters whose syntax is expression
+prefix are treated as part of the sexp if they appear next to it.
The syntax table controls the interpretation of characters, so these
functions can be used for Lisp expressions when in Lisp mode and for C
expressions when in C mode. @xref{List Motion}, for convenient
higher-level functions for moving over balanced expressions.
- A syntax table only describes how each character changes the state
-of the parser, rather than describing the state itself. For example,
-a string delimiter character toggles the parser state between
-``in-string'' and ``in-code'' but the characters inside the string do
-not have any particular syntax to identify them as such. For example
-(note that 15 is the syntax code for generic string delimiters),
+ A character's syntax controls how it changes the state of the
+parser, rather than describing the state itself. For example, a
+string delimiter character toggles the parser state between
+``in-string'' and ``in-code,'' but the syntax of characters does not
+directly say whether they are inside a string. For example (note that
+15 is the syntax code for generic string delimiters),
@example
(put-text-property 1 9 'syntax-table '(15 . nil))
are a string, but rather that they are all string delimiters. As a
result, Emacs treats them as four consecutive empty string constants.
- Every time you use the parser, you specify it a starting state as
-well as a starting position. If you omit the starting state, the
-default is ``top level in parenthesis structure,'' as it would be at
-the beginning of a function definition. (This is the case for
-@code{forward-sexp}, which blindly assumes that the starting point is
-in such a state.)
+@menu
+* Motion via Parsing:: Motion functions that work by parsing.
+* Position Parse:: Determining the syntactic state of a position.
+* Parser State:: How Emacs represents a syntactic state.
+* Low-Level Parsing:: Parsing across a specified region.
+* Control Parsing:: Parameters that affect parsing.
+@end menu
-@defun parse-partial-sexp start limit &optional target-depth stop-before state stop-comment
-This function parses a sexp in the current buffer starting at
-@var{start}, not scanning past @var{limit}. It stops at position
-@var{limit} or when certain criteria described below are met, and sets
-point to the location where parsing stops. It returns a value
-describing the status of the parse at the point where it stops.
+@node Motion via Parsing
+@subsection Motion Commands Based on Parsing
-If @var{state} is @code{nil}, @var{start} is assumed to be at the top
-level of parenthesis structure, such as the beginning of a function
-definition. Alternatively, you might wish to resume parsing in the
-middle of the structure. To do this, you must provide a @var{state}
-argument that describes the initial status of parsing.
+ This section describes simple point-motion functions that operate
+based on parsing expressions.
-@cindex parenthesis depth
-If the third argument @var{target-depth} is non-@code{nil}, parsing
-stops if the depth in parentheses becomes equal to @var{target-depth}.
-The depth starts at 0, or at whatever is given in @var{state}.
+@defun scan-lists from count depth
+This function scans forward @var{count} balanced parenthetical groupings
+from position @var{from}. It returns the position where the scan stops.
+If @var{count} is negative, the scan moves backwards.
-If the fourth argument @var{stop-before} is non-@code{nil}, parsing
-stops when it comes to any character that starts a sexp. If
-@var{stop-comment} is non-@code{nil}, parsing stops when it comes to the
-start of a comment. If @var{stop-comment} is the symbol
-@code{syntax-table}, parsing stops after the start of a comment or a
-string, or the end of a comment or a string, whichever comes first.
+If @var{depth} is nonzero, parenthesis depth counting begins from that
+value. The only candidates for stopping are places where the depth in
+parentheses becomes zero; @code{scan-lists} counts @var{count} such
+places and then stops. Thus, a positive value for @var{depth} means go
+out @var{depth} levels of parenthesis.
+
+Scanning ignores comments if @code{parse-sexp-ignore-comments} is
+non-@code{nil}.
+
+If the scan reaches the beginning or end of the buffer (or its
+accessible portion), and the depth is not zero, an error is signaled.
+If the depth is zero but the count is not used up, @code{nil} is
+returned.
+@end defun
-@cindex parse state
-The fifth argument @var{state} is a nine-element list of the same form
-as the value of this function, described below. (It is OK to omit the
-last element of the nine.) The return value of one call may be used to
-initialize the state of the parse on another call to
-@code{parse-partial-sexp}.
+@defun scan-sexps from count
+This function scans forward @var{count} sexps from position @var{from}.
+It returns the position where the scan stops. If @var{count} is
+negative, the scan moves backwards.
+
+Scanning ignores comments if @code{parse-sexp-ignore-comments} is
+non-@code{nil}.
+
+If the scan reaches the beginning or end of (the accessible part of) the
+buffer while in the middle of a parenthetical grouping, an error is
+signaled. If it reaches the beginning or end between groupings but
+before count is used up, @code{nil} is returned.
+@end defun
+
+@defun forward-comment count
+This function moves point forward across @var{count} complete comments
+ (that is, including the starting delimiter and the terminating
+delimiter if any), plus any whitespace encountered on the way. It
+moves backward if @var{count} is negative. If it encounters anything
+other than a comment or whitespace, it stops, leaving point at the
+place where it stopped. This includes (for instance) finding the end
+of a comment when moving forward and expecting the beginning of one.
+The function also stops immediately after moving over the specified
+number of complete comments. If @var{count} comments are found as
+expected, with nothing except whitespace between them, it returns
+@code{t}; otherwise it returns @code{nil}.
+
+This function cannot tell whether the ``comments'' it traverses are
+embedded within a string. If they look like comments, it treats them
+as comments.
+@end defun
+
+To move forward over all comments and whitespace following point, use
+@code{(forward-comment (buffer-size))}. @code{(buffer-size)} is a good
+argument to use, because the number of comments in the buffer cannot
+exceed that many.
+
+@node Position Parse
+@subsection Finding the Parse State for a Position
+
+ For syntactic analysis, such as in indentation, often the useful
+thing is to compute the syntactic state corresponding to a given buffer
+position. This function does that conveniently.
+
+@defun syntax-ppss &optional pos
+This function returns the parser state (see next section) that the
+parser would reach at position @var{pos} starting from the beginning
+of the buffer. This is equivalent to @code{(parse-partial-sexp
+(point-min) @var{pos})}, except that @code{syntax-ppss} uses a cache
+to speed up the computation. Due to this optimization, the 2nd value
+(previous complete subexpression) and 6th value (minimum parenthesis
+depth) of the returned parser state are not meaningful.
+@end defun
+
+ @code{syntax-ppss} automatically hooks itself to
+@code{before-change-functions} to keep its cache consistent. But
+updating can fail if @code{syntax-ppss} is called while
+@code{before-change-functions} is temporarily let-bound, or if the
+buffer is modified without obeying the hook, such as when using
+@code{inhibit-modification-hooks}. For this reason, it is sometimes
+necessary to flush the cache manually.
+
+@defun syntax-ppss-flush-cache beg
+This function flushes the cache used by @code{syntax-ppss}, starting at
+position @var{beg}.
+@end defun
+
+ Major modes can make @code{syntax-ppss} run faster by specifying
+where it needs to start parsing.
+
+@defvar syntax-begin-function
+If this is non-@code{nil}, it should be a function that moves to an
+earlier buffer position where the parser state is equivalent to
+@code{nil}---in other words, a position outside of any comment,
+string, or parenthesis. @code{syntax-ppss} uses it to further
+optimize its computations, when the cache gives no help.
+@end defvar
-The result is a list of nine elements describing the final state of
-the parse:
+@node Parser State
+@subsection Parser State
+@cindex parser state
+
+ A @dfn{parser state} is a list of ten elements describing the final
+state of parsing text syntactically as part of an expression. The
+parsing functions in the following sections return a parser state as
+the value, and in some cases accept one as an argument also, so that
+you can resume parsing after it stops. Here are the meanings of the
+elements of the parser state:
@enumerate 0
@item
-The depth in parentheses, counting from 0.
+The depth in parentheses, counting from 0. @strong{Warning:} this can
+be negative if there are more close parens than open parens between
+the start of the defun and point.
@item
@cindex innermost containing parentheses
@item
What kind of comment is active: @code{nil} for a comment of style
``a'' or when not inside a comment, @code{t} for a comment of style
-``b'', and @code{syntax-table} for a comment that should be ended by a
+``b,'' and @code{syntax-table} for a comment that should be ended by a
generic comment delimiter character.
@item
the position where the comment began; while inside a string, this is the
position where the string began. When outside of strings and comments,
this element is @code{nil}.
+
+@item
+Internal data for continuing the parsing. The meaning of this
+data is subject to change; it is used if you pass this list
+as the @var{state} argument to another call.
@end enumerate
-Elements 0, 3, 4, 5 and 7 are significant in the argument @var{state}.
+ Elements 1, 2, and 6 are ignored in a state which you pass as an
+argument to continue parsing, and elements 8 and 9 are used only in
+trivial cases. Those elements serve primarily to convey information
+to the Lisp program which does the parsing.
-Actually, the return value is currently a list of ten, rather than
-nine, elements and @var{state} is allowed to be a list of ten elements
-as well. However, the meaning of the tenth element is subject to
-change and only the first eight elements of @var{state} need to be
-specified.
+ One additional piece of useful information is available from a
+parser state using this function:
-@cindex indenting with parentheses
-This function is most often used to compute indentation for languages
-that have nested parentheses.
+@defun syntax-ppss-toplevel-pos state
+This function extracts, from parser state @var{state}, the last
+position scanned in the parse which was at top level in grammatical
+structure. ``At top level'' means outside of any parentheses,
+comments, or strings.
+
+The value is @code{nil} if @var{state} represents a parse which has
+arrived at a top level position.
@end defun
-@defun scan-lists from count depth
-This function scans forward @var{count} balanced parenthetical groupings
-from position @var{from}. It returns the position where the scan stops.
-If @var{count} is negative, the scan moves backwards.
+ We have provided this access function rather than document how the
+data is represented in the state, because we plan to change the
+representation in the future.
-If @var{depth} is nonzero, parenthesis depth counting begins from that
-value. The only candidates for stopping are places where the depth in
-parentheses becomes zero; @code{scan-lists} counts @var{count} such
-places and then stops. Thus, a positive value for @var{depth} means go
-out @var{depth} levels of parenthesis.
+@node Low-Level Parsing
+@subsection Low-Level Parsing
-Scanning ignores comments if @code{parse-sexp-ignore-comments} is
-non-@code{nil}.
+ The most basic way to use the expression parser is to tell it
+to start at a given position with a certain state, and parse up to
+a specified end position.
-If the scan reaches the beginning or end of the buffer (or its
-accessible portion), and the depth is not zero, an error is signaled.
-If the depth is zero but the count is not used up, @code{nil} is
-returned.
-@end defun
+@defun parse-partial-sexp start limit &optional target-depth stop-before state stop-comment
+This function parses a sexp in the current buffer starting at
+@var{start}, not scanning past @var{limit}. It stops at position
+@var{limit} or when certain criteria described below are met, and sets
+point to the location where parsing stops. It returns a parser state
+describing the status of the parse at the point where it stops.
-@defun scan-sexps from count
-This function scans forward @var{count} sexps from position @var{from}.
-It returns the position where the scan stops. If @var{count} is
-negative, the scan moves backwards.
+@cindex parenthesis depth
+If the third argument @var{target-depth} is non-@code{nil}, parsing
+stops if the depth in parentheses becomes equal to @var{target-depth}.
+The depth starts at 0, or at whatever is given in @var{state}.
-Scanning ignores comments if @code{parse-sexp-ignore-comments} is
-non-@code{nil}.
+If the fourth argument @var{stop-before} is non-@code{nil}, parsing
+stops when it comes to any character that starts a sexp. If
+@var{stop-comment} is non-@code{nil}, parsing stops when it comes to the
+start of a comment. If @var{stop-comment} is the symbol
+@code{syntax-table}, parsing stops after the start of a comment or a
+string, or the end of a comment or a string, whichever comes first.
-If the scan reaches the beginning or end of (the accessible part of) the
-buffer while in the middle of a parenthetical grouping, an error is
-signaled. If it reaches the beginning or end between groupings but
-before count is used up, @code{nil} is returned.
+If @var{state} is @code{nil}, @var{start} is assumed to be at the top
+level of parenthesis structure, such as the beginning of a function
+definition. Alternatively, you might wish to resume parsing in the
+middle of the structure. To do this, you must provide a @var{state}
+argument that describes the initial status of parsing. The value
+returned by a previous call to @code{parse-partial-sexp} will do
+nicely.
@end defun
+@node Control Parsing
+@subsection Parameters to Control Parsing
+
@defvar multibyte-syntax-as-symbol
-@tindex multibyte-syntax-as-symbol
If this variable is non-@code{nil}, @code{scan-sexps} treats all
non-@acronym{ASCII} characters as symbol constituents regardless
of what the syntax table says about them. (However, text properties
@defopt parse-sexp-ignore-comments
@cindex skipping comments
If the value is non-@code{nil}, then comments are treated as
-whitespace by the functions in this section and by @code{forward-sexp}.
+whitespace by the functions in this section and by @code{forward-sexp},
+@code{scan-lists} and @code{scan-sexps}.
@end defopt
@vindex parse-sexp-lookup-properties
-The behaviour of @code{parse-partial-sexp} is also affected by
+The behavior of @code{parse-partial-sexp} is also affected by
@code{parse-sexp-lookup-properties} (@pxref{Syntax Properties}).
You can use @code{forward-comment} to move forward or backward over
one comment or several comments.
-@defun forward-comment count
-This function moves point forward across @var{count} complete comments
-(that is, including the starting delimiter and the terminating
-delimiter if any), plus any whitespace encountered on the way. It
-moves backward if @var{count} is negative. If it encounters anything
-other than a comment or whitespace, it stops, leaving point at the
-place where it stopped. This includes (for instance) finding the end
-of a comment when moving forward and expecting the beginning of one.
-The function also stops immediately after moving over the specified
-number of complete comments. If @var{count} comments are found as
-expected, with nothing except whitespace between them, it returns
-@code{t}; otherwise it returns @code{nil}.
-
-This function cannot tell whether the ``comments'' it traverses are
-embedded within a string. If they look like comments, it treats them
-as comments.
-@end defun
-
-To move forward over all comments and whitespace following point, use
-@code{(forward-comment (buffer-size))}. @code{(buffer-size)} is a good
-argument to use, because the number of comments in the buffer cannot
-exceed that many.
-
@node Standard Syntax Tables
@section Some Standard Syntax Tables
This table gives the value of @var{syntax-code} which corresponds
to each syntactic type.
-@multitable @columnfractions .05 .3 .3 .3
+@multitable @columnfractions .05 .3 .3 .31
@item
@tab
@i{Integer} @i{Class}
@end multitable
@defun string-to-syntax @var{desc}
-This function returns the internal form @code{(@var{syntax-code} .
-@var{matching-char})} corresponding to the syntax descriptor @var{desc}.
+This function returns the internal form corresponding to the syntax
+descriptor @var{desc}, a cons cell @code{(@var{syntax-code}
+. @var{matching-char})}.
+@end defun
+
+@defun syntax-after pos
+This function returns the syntax code of the character in the buffer
+after position @var{pos}, taking account of syntax properties as well
+as the syntax table. If @var{pos} is outside the buffer's accessible
+portion (@pxref{Narrowing, accessible portion}), this function returns
+@code{nil}.
+@end defun
+
+@defun syntax-class syntax
+This function returns the syntax class of the syntax code
+@var{syntax}. (It masks off the high 16 bits that hold the flags
+encoded in the syntax descriptor.) If @var{syntax} is @code{nil}, it
+returns @code{nil}; this is so evaluating the expression
+
+@example
+(syntax-class (syntax-after pos))
+@end example
+
+@noindent
+where @code{pos} is outside the buffer's accessible portion, will
+yield @code{nil} without throwing errors or producing wrong syntax
+class codes.
@end defun
@node Categories
@section Categories
@cindex categories of characters
+@cindex character categories
@dfn{Categories} provide an alternate way of classifying characters
syntactically. You can define several categories as needed, then
syntax classes, categories are not mutually exclusive; it is normal for
one character to belong to several categories.
+@cindex category table
Each buffer has a @dfn{category table} which records which categories
are defined and also which characters belong to each category. Each
category table defines its own categories, but normally these are
@defun define-category char docstring &optional table
This function defines a new category, with name @var{char} and
-documentation @var{docstring}, for the category table @var{table},
+documentation @var{docstring}, for the category table @var{table}.
@end defun
@defun category-docstring category &optional table
@end defun
@defun make-category-table
-@tindex make-category-table
This creates and returns an empty category table. In an empty category
table, no categories have been allocated, and no characters belong to
any categories.
HCoop / bpt / emacs.git / blobdiff