Backport from trunk: doc fix.
[bpt/emacs.git] / doc / lispref / processes.texi
CommitLineData
b8d4c8d0
GM
1@c -*-texinfo-*-
2@c This is part of the GNU Emacs Lisp Reference Manual.
3@c Copyright (C) 1990, 1991, 1992, 1993, 1994, 1995, 1998, 1999, 2001,
114f9c96 4@c 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010 Free Software Foundation, Inc.
b8d4c8d0 5@c See the file elisp.texi for copying conditions.
6336d8c3 6@setfilename ../../info/processes
b8d4c8d0
GM
7@node Processes, Display, Abbrevs, Top
8@chapter Processes
9@cindex child process
10@cindex parent process
11@cindex subprocess
12@cindex process
13
14 In the terminology of operating systems, a @dfn{process} is a space in
15which a program can execute. Emacs runs in a process. Emacs Lisp
16programs can invoke other programs in processes of their own. These are
17called @dfn{subprocesses} or @dfn{child processes} of the Emacs process,
18which is their @dfn{parent process}.
19
20 A subprocess of Emacs may be @dfn{synchronous} or @dfn{asynchronous},
21depending on how it is created. When you create a synchronous
22subprocess, the Lisp program waits for the subprocess to terminate
23before continuing execution. When you create an asynchronous
24subprocess, it can run in parallel with the Lisp program. This kind of
25subprocess is represented within Emacs by a Lisp object which is also
26called a ``process.'' Lisp programs can use this object to communicate
27with the subprocess or to control it. For example, you can send
28signals, obtain status information, receive output from the process, or
29send input to it.
30
31@defun processp object
23dd4ecd
EZ
32This function returns @code{t} if @var{object} represents an Emacs
33subprocess, @code{nil} otherwise.
b8d4c8d0
GM
34@end defun
35
23dd4ecd
EZ
36 In addition to subprocesses of the current Emacs session, you can
37also access other processes running on your machine. @xref{System
38Processes}.
39
b8d4c8d0
GM
40@menu
41* Subprocess Creation:: Functions that start subprocesses.
42* Shell Arguments:: Quoting an argument to pass it to a shell.
43* Synchronous Processes:: Details of using synchronous subprocesses.
44* Asynchronous Processes:: Starting up an asynchronous subprocess.
45* Deleting Processes:: Eliminating an asynchronous subprocess.
46* Process Information:: Accessing run-status and other attributes.
47* Input to Processes:: Sending input to an asynchronous subprocess.
48* Signals to Processes:: Stopping, continuing or interrupting
49 an asynchronous subprocess.
50* Output from Processes:: Collecting output from an asynchronous subprocess.
51* Sentinels:: Sentinels run when process run-status changes.
52* Query Before Exit:: Whether to query if exiting will kill a process.
23dd4ecd 53* System Processes:: Accessing other processes running on your system.
b8d4c8d0
GM
54* Transaction Queues:: Transaction-based communication with subprocesses.
55* Network:: Opening network connections.
56* Network Servers:: Network servers let Emacs accept net connections.
57* Datagrams:: UDP network connections.
58* Low-Level Network:: Lower-level but more general function
59 to create connections and servers.
60* Misc Network:: Additional relevant functions for network connections.
c73e02fa 61* Serial Ports:: Communicating with serial ports.
b8d4c8d0
GM
62* Byte Packing:: Using bindat to pack and unpack binary data.
63@end menu
64
65@node Subprocess Creation
66@section Functions that Create Subprocesses
67
583d8b3c 68 There are three primitives that create a new subprocess in which to run
b8d4c8d0
GM
69a program. One of them, @code{start-process}, creates an asynchronous
70process and returns a process object (@pxref{Asynchronous Processes}).
71The other two, @code{call-process} and @code{call-process-region},
72create a synchronous process and do not return a process object
73(@pxref{Synchronous Processes}).
74
75 Synchronous and asynchronous processes are explained in the following
76sections. Since the three functions are all called in a similar
77fashion, their common arguments are described here.
78
79@cindex execute program
80@cindex @code{PATH} environment variable
81@cindex @code{HOME} environment variable
82 In all cases, the function's @var{program} argument specifies the
83program to be run. An error is signaled if the file is not found or
84cannot be executed. If the file name is relative, the variable
85@code{exec-path} contains a list of directories to search. Emacs
86initializes @code{exec-path} when it starts up, based on the value of
87the environment variable @code{PATH}. The standard file name
88constructs, @samp{~}, @samp{.}, and @samp{..}, are interpreted as
89usual in @code{exec-path}, but environment variable substitutions
90(@samp{$HOME}, etc.) are not recognized; use
91@code{substitute-in-file-name} to perform them (@pxref{File Name
92Expansion}). @code{nil} in this list refers to
93@code{default-directory}.
94
95 Executing a program can also try adding suffixes to the specified
96name:
97
98@defvar exec-suffixes
99This variable is a list of suffixes (strings) to try adding to the
100specified program file name. The list should include @code{""} if you
101want the name to be tried exactly as specified. The default value is
102system-dependent.
103@end defvar
104
105 @strong{Please note:} The argument @var{program} contains only the
106name of the program; it may not contain any command-line arguments. You
107must use @var{args} to provide those.
108
109 Each of the subprocess-creating functions has a @var{buffer-or-name}
110argument which specifies where the standard output from the program will
111go. It should be a buffer or a buffer name; if it is a buffer name,
112that will create the buffer if it does not already exist. It can also
113be @code{nil}, which says to discard the output unless a filter function
114handles it. (@xref{Filter Functions}, and @ref{Read and Print}.)
115Normally, you should avoid having multiple processes send output to the
116same buffer because their output would be intermixed randomly.
117
118@cindex program arguments
119 All three of the subprocess-creating functions have a @code{&rest}
120argument, @var{args}. The @var{args} must all be strings, and they are
121supplied to @var{program} as separate command line arguments. Wildcard
122characters and other shell constructs have no special meanings in these
123strings, since the strings are passed directly to the specified program.
124
125 The subprocess gets its current directory from the value of
126@code{default-directory} (@pxref{File Name Expansion}).
127
128@cindex environment variables, subprocesses
129 The subprocess inherits its environment from Emacs, but you can
130specify overrides for it with @code{process-environment}. @xref{System
131Environment}.
132
133@defvar exec-directory
134@pindex movemail
135The value of this variable is a string, the name of a directory that
136contains programs that come with GNU Emacs, programs intended for Emacs
137to invoke. The program @code{movemail} is an example of such a program;
138Rmail uses it to fetch new mail from an inbox.
139@end defvar
140
141@defopt exec-path
142The value of this variable is a list of directories to search for
143programs to run in subprocesses. Each element is either the name of a
144directory (i.e., a string), or @code{nil}, which stands for the default
145directory (which is the value of @code{default-directory}).
146@cindex program directories
147
148The value of @code{exec-path} is used by @code{call-process} and
149@code{start-process} when the @var{program} argument is not an absolute
150file name.
151@end defopt
152
153@node Shell Arguments
154@section Shell Arguments
155@cindex arguments for shell commands
156@cindex shell command arguments
157
158 Lisp programs sometimes need to run a shell and give it a command
159that contains file names that were specified by the user. These
160programs ought to be able to support any valid file name. But the shell
161gives special treatment to certain characters, and if these characters
162occur in the file name, they will confuse the shell. To handle these
163characters, use the function @code{shell-quote-argument}:
164
165@defun shell-quote-argument argument
166This function returns a string which represents, in shell syntax,
167an argument whose actual contents are @var{argument}. It should
168work reliably to concatenate the return value into a shell command
169and then pass it to a shell for execution.
170
171Precisely what this function does depends on your operating system. The
172function is designed to work with the syntax of your system's standard
173shell; if you use an unusual shell, you will need to redefine this
174function.
175
176@example
177;; @r{This example shows the behavior on GNU and Unix systems.}
178(shell-quote-argument "foo > bar")
179 @result{} "foo\\ \\>\\ bar"
180
181;; @r{This example shows the behavior on MS-DOS and MS-Windows.}
182(shell-quote-argument "foo > bar")
183 @result{} "\"foo > bar\""
184@end example
185
186Here's an example of using @code{shell-quote-argument} to construct
187a shell command:
188
189@example
190(concat "diff -c "
191 (shell-quote-argument oldfile)
192 " "
193 (shell-quote-argument newfile))
194@end example
195@end defun
196
a873ee3d 197@cindex quoting and unquoting shell command line
0999039c
CY
198 The following two functions are useful for creating shell commands
199from individual argument strings, and taking shell command lines apart
200into individual arguments.
a873ee3d
EZ
201
202@defun split-string-and-unquote string &optional separators
203This function splits @var{string} into substrings at matches for the
204regular expression @var{separators}, like @code{split-string} does
0999039c
CY
205(@pxref{Creating Strings}); in addition, it removes quoting from the
206substrings. It then makes a list of the substrings and returns it.
a873ee3d 207
4185820c
CY
208If @var{separators} is omitted or @code{nil}, it defaults to
209@code{"\\s-+"}, which is a regular expression that matches one or more
210characters with whitespace syntax (@pxref{Syntax Class Table}).
a873ee3d 211
0999039c
CY
212This function performs two types of quoting: enclosing a whole string
213in double quotes @code{"@dots{}"}, and quoting individual characters
214with a backslash escape @samp{\}. The latter is also used in Lisp
215strings, so this function can handle those as well.
a873ee3d
EZ
216@end defun
217
218@defun combine-and-quote-strings list-of-strings &optional separator
219This function concatenates @var{list-of-strings} into a single string,
0999039c
CY
220quoting each string as necessary. It also sticks the @var{separator}
221string between each pair of strings; if @var{separator} is omitted or
222@code{nil}, it defaults to @code{" "}. The return value is the
223resulting string.
a873ee3d
EZ
224
225The strings in @var{list-of-strings} that need quoting are those that
226include @var{separator} as their substring. Quoting a string encloses
227it in double quotes @code{"@dots{}"}. In the simplest case, if you
228are consing a shell command from the individual command-line
229arguments, every argument that includes embedded blanks will be
230quoted.
231@end defun
232
b8d4c8d0
GM
233@node Synchronous Processes
234@section Creating a Synchronous Process
235@cindex synchronous subprocess
236
237 After a @dfn{synchronous process} is created, Emacs waits for the
238process to terminate before continuing. Starting Dired on GNU or
239Unix@footnote{On other systems, Emacs uses a Lisp emulation of
240@code{ls}; see @ref{Contents of Directories}.} is an example of this: it
241runs @code{ls} in a synchronous process, then modifies the output
242slightly. Because the process is synchronous, the entire directory
243listing arrives in the buffer before Emacs tries to do anything with it.
244
245 While Emacs waits for the synchronous subprocess to terminate, the
246user can quit by typing @kbd{C-g}. The first @kbd{C-g} tries to kill
247the subprocess with a @code{SIGINT} signal; but it waits until the
248subprocess actually terminates before quitting. If during that time the
249user types another @kbd{C-g}, that kills the subprocess instantly with
250@code{SIGKILL} and quits immediately (except on MS-DOS, where killing
251other processes doesn't work). @xref{Quitting}.
252
253 The synchronous subprocess functions return an indication of how the
254process terminated.
255
256 The output from a synchronous subprocess is generally decoded using a
257coding system, much like text read from a file. The input sent to a
258subprocess by @code{call-process-region} is encoded using a coding
259system, much like text written into a file. @xref{Coding Systems}.
260
261@defun call-process program &optional infile destination display &rest args
262This function calls @var{program} in a separate process and waits for
263it to finish.
264
265The standard input for the process comes from file @var{infile} if
266@var{infile} is not @code{nil}, and from the null device otherwise.
267The argument @var{destination} says where to put the process output.
268Here are the possibilities:
269
270@table @asis
271@item a buffer
272Insert the output in that buffer, before point. This includes both the
273standard output stream and the standard error stream of the process.
274
275@item a string
276Insert the output in a buffer with that name, before point.
277
278@item @code{t}
279Insert the output in the current buffer, before point.
280
281@item @code{nil}
282Discard the output.
283
284@item 0
285Discard the output, and return @code{nil} immediately without waiting
286for the subprocess to finish.
287
288In this case, the process is not truly synchronous, since it can run in
289parallel with Emacs; but you can think of it as synchronous in that
290Emacs is essentially finished with the subprocess as soon as this
291function returns.
292
293MS-DOS doesn't support asynchronous subprocesses, so this option doesn't
294work there.
295
296@item @code{(@var{real-destination} @var{error-destination})}
297Keep the standard output stream separate from the standard error stream;
298deal with the ordinary output as specified by @var{real-destination},
299and dispose of the error output according to @var{error-destination}.
300If @var{error-destination} is @code{nil}, that means to discard the
301error output, @code{t} means mix it with the ordinary output, and a
302string specifies a file name to redirect error output into.
303
304You can't directly specify a buffer to put the error output in; that is
305too difficult to implement. But you can achieve this result by sending
306the error output to a temporary file and then inserting the file into a
307buffer.
308@end table
309
310If @var{display} is non-@code{nil}, then @code{call-process} redisplays
311the buffer as output is inserted. (However, if the coding system chosen
312for decoding output is @code{undecided}, meaning deduce the encoding
313from the actual data, then redisplay sometimes cannot continue once
314non-@acronym{ASCII} characters are encountered. There are fundamental
315reasons why it is hard to fix this; see @ref{Output from Processes}.)
316
317Otherwise the function @code{call-process} does no redisplay, and the
318results become visible on the screen only when Emacs redisplays that
319buffer in the normal course of events.
320
321The remaining arguments, @var{args}, are strings that specify command
322line arguments for the program.
323
324The value returned by @code{call-process} (unless you told it not to
325wait) indicates the reason for process termination. A number gives the
326exit status of the subprocess; 0 means success, and any other value
327means failure. If the process terminated with a signal,
328@code{call-process} returns a string describing the signal.
329
330In the examples below, the buffer @samp{foo} is current.
331
332@smallexample
333@group
334(call-process "pwd" nil t)
335 @result{} 0
336
337---------- Buffer: foo ----------
338/usr/user/lewis/manual
339---------- Buffer: foo ----------
340@end group
341
342@group
343(call-process "grep" nil "bar" nil "lewis" "/etc/passwd")
344 @result{} 0
345
346---------- Buffer: bar ----------
347lewis:5LTsHm66CSWKg:398:21:Bil Lewis:/user/lewis:/bin/csh
348
349---------- Buffer: bar ----------
350@end group
351@end smallexample
352
353Here is a good example of the use of @code{call-process}, which used to
354be found in the definition of @code{insert-directory}:
355
356@smallexample
357@group
358(call-process insert-directory-program nil t nil @var{switches}
359 (if full-directory-p
360 (concat (file-name-as-directory file) ".")
361 file))
362@end group
363@end smallexample
364@end defun
365
366@defun process-file program &optional infile buffer display &rest args
367This function processes files synchronously in a separate process. It
368is similar to @code{call-process} but may invoke a file handler based
369on the value of the variable @code{default-directory}. The current
370working directory of the subprocess is @code{default-directory}.
371
372The arguments are handled in almost the same way as for
373@code{call-process}, with the following differences:
374
375Some file handlers may not support all combinations and forms of the
376arguments @var{infile}, @var{buffer}, and @var{display}. For example,
377some file handlers might behave as if @var{display} were @code{nil},
378regardless of the value actually passed. As another example, some
379file handlers might not support separating standard output and error
380output by way of the @var{buffer} argument.
381
382If a file handler is invoked, it determines the program to run based
383on the first argument @var{program}. For instance, consider that a
384handler for remote files is invoked. Then the path that is used for
385searching the program might be different than @code{exec-path}.
386
387The second argument @var{infile} may invoke a file handler. The file
388handler could be different from the handler chosen for the
389@code{process-file} function itself. (For example,
390@code{default-directory} could be on a remote host, whereas
391@var{infile} is on another remote host. Or @code{default-directory}
392could be non-special, whereas @var{infile} is on a remote host.)
393
394If @var{buffer} is a list of the form @code{(@var{real-destination}
395@var{error-destination})}, and @var{error-destination} names a file,
396then the same remarks as for @var{infile} apply.
397
398The remaining arguments (@var{args}) will be passed to the process
399verbatim. Emacs is not involved in processing file names that are
400present in @var{args}. To avoid confusion, it may be best to avoid
401absolute file names in @var{args}, but rather to specify all file
402names as relative to @code{default-directory}. The function
403@code{file-relative-name} is useful for constructing such relative
404file names.
405@end defun
406
3691a0a6
MA
407@defvar process-file-side-effects
408This variable indicates, whether a call of @code{process-file} changes
409remote files.
410
411Per default, this variable is always set to @code{t}, meaning that a
412call of @code{process-file} could potentially change any file on a
413remote host. When set to @code{nil}, a file handler could optimize
414its behaviour with respect to remote file attributes caching.
415
416This variable should never be changed by @code{setq}. Instead of, it
417shall be set only by let-binding.
418@end defvar
419
b8d4c8d0
GM
420@defun call-process-region start end program &optional delete destination display &rest args
421This function sends the text from @var{start} to @var{end} as
422standard input to a process running @var{program}. It deletes the text
423sent if @var{delete} is non-@code{nil}; this is useful when
424@var{destination} is @code{t}, to insert the output in the current
425buffer in place of the input.
426
427The arguments @var{destination} and @var{display} control what to do
428with the output from the subprocess, and whether to update the display
429as it comes in. For details, see the description of
430@code{call-process}, above. If @var{destination} is the integer 0,
431@code{call-process-region} discards the output and returns @code{nil}
432immediately, without waiting for the subprocess to finish (this only
433works if asynchronous subprocesses are supported).
434
435The remaining arguments, @var{args}, are strings that specify command
436line arguments for the program.
437
438The return value of @code{call-process-region} is just like that of
439@code{call-process}: @code{nil} if you told it to return without
440waiting; otherwise, a number or string which indicates how the
441subprocess terminated.
442
443In the following example, we use @code{call-process-region} to run the
444@code{cat} utility, with standard input being the first five characters
445in buffer @samp{foo} (the word @samp{input}). @code{cat} copies its
446standard input into its standard output. Since the argument
447@var{destination} is @code{t}, this output is inserted in the current
448buffer.
449
450@smallexample
451@group
452---------- Buffer: foo ----------
453input@point{}
454---------- Buffer: foo ----------
455@end group
456
457@group
458(call-process-region 1 6 "cat" nil t)
459 @result{} 0
460
461---------- Buffer: foo ----------
462inputinput@point{}
463---------- Buffer: foo ----------
464@end group
465@end smallexample
466
467 The @code{shell-command-on-region} command uses
468@code{call-process-region} like this:
469
470@smallexample
471@group
472(call-process-region
473 start end
474 shell-file-name ; @r{Name of program.}
475 nil ; @r{Do not delete region.}
476 buffer ; @r{Send output to @code{buffer}.}
477 nil ; @r{No redisplay during output.}
478 "-c" command) ; @r{Arguments for the shell.}
479@end group
480@end smallexample
481@end defun
482
483@defun call-process-shell-command command &optional infile destination display &rest args
484This function executes the shell command @var{command} synchronously
485in a separate process. The final arguments @var{args} are additional
486arguments to add at the end of @var{command}. The other arguments
487are handled as in @code{call-process}.
488@end defun
489
490@defun process-file-shell-command command &optional infile destination display &rest args
491This function is like @code{call-process-shell-command}, but uses
492@code{process-file} internally. Depending on @code{default-directory},
493@var{command} can be executed also on remote hosts.
494@end defun
495
496@defun shell-command-to-string command
497This function executes @var{command} (a string) as a shell command,
498then returns the command's output as a string.
499@end defun
500
583d8b3c
EZ
501@defun process-lines program &rest args
502This function runs @var{program} in a separate process, waits for it
503to finish, and returns its output as a list of strings. Each string
504in the list holds a single line of text output by the program; the
505end-of-line characters are stripped from each line. The arguments
506beyond @var{program}, @var{args}, are strings that specify
507command-line arguments with which to run the program.
508
509If @var{program} exits with a non-zero exit status, this function
510signals an error.
511
512This function works by calling @code{call-process}, so program output
513is decoded in the same way as for @code{call-process}.
514@end defun
515
b8d4c8d0
GM
516@node Asynchronous Processes
517@section Creating an Asynchronous Process
518@cindex asynchronous subprocess
519
520 After an @dfn{asynchronous process} is created, Emacs and the subprocess
521both continue running immediately. The process thereafter runs
522in parallel with Emacs, and the two can communicate with each other
523using the functions described in the following sections. However,
524communication is only partially asynchronous: Emacs sends data to the
525process only when certain functions are called, and Emacs accepts data
526from the process only when Emacs is waiting for input or for a time
527delay.
528
529 Here we describe how to create an asynchronous process.
530
531@defun start-process name buffer-or-name program &rest args
532This function creates a new asynchronous subprocess and starts the
533program @var{program} running in it. It returns a process object that
534stands for the new subprocess in Lisp. The argument @var{name}
535specifies the name for the process object; if a process with this name
536already exists, then @var{name} is modified (by appending @samp{<1>},
537etc.) to be unique. The buffer @var{buffer-or-name} is the buffer to
538associate with the process.
539
540The remaining arguments, @var{args}, are strings that specify command
541line arguments for the program.
542
543In the example below, the first process is started and runs (rather,
544sleeps) for 100 seconds. Meanwhile, the second process is started, and
545given the name @samp{my-process<1>} for the sake of uniqueness. It
546inserts the directory listing at the end of the buffer @samp{foo},
547before the first process finishes. Then it finishes, and a message to
548that effect is inserted in the buffer. Much later, the first process
549finishes, and another message is inserted in the buffer for it.
550
551@smallexample
552@group
553(start-process "my-process" "foo" "sleep" "100")
554 @result{} #<process my-process>
555@end group
556
557@group
558(start-process "my-process" "foo" "ls" "-l" "/user/lewis/bin")
559 @result{} #<process my-process<1>>
560
561---------- Buffer: foo ----------
562total 2
563lrwxrwxrwx 1 lewis 14 Jul 22 10:12 gnuemacs --> /emacs
564-rwxrwxrwx 1 lewis 19 Jul 30 21:02 lemon
565
566Process my-process<1> finished
567
568Process my-process finished
569---------- Buffer: foo ----------
570@end group
571@end smallexample
572@end defun
573
574@defun start-file-process name buffer-or-name program &rest args
575Like @code{start-process}, this function starts a new asynchronous
576subprocess running @var{program} in it, and returns its process
577object---when @code{default-directory} is not a magic file name.
578
579If @code{default-directory} is magic, the function invokes its file
580handler instead. This handler ought to run @var{program}, perhaps on
581the local host, perhaps on a remote host that corresponds to
582@code{default-directory}. In the latter case, the local part of
583@code{default-directory} becomes the working directory of the process.
584
585This function does not try to invoke file name handlers for
586@var{program} or for the @var{program-args}.
587
588Depending on the implementation of the file handler, it might not be
589possible to apply @code{process-filter} or @code{process-sentinel} to
590the resulting process object (@pxref{Filter Functions}, @pxref{Sentinels}).
591
592Some file handlers may not support @code{start-file-process} (for
593example @code{ange-ftp-hook-function}). In such cases, the function
594does nothing and returns @code{nil}.
595@end defun
596
03a74b84 597@defun start-process-shell-command name buffer-or-name command
b8d4c8d0
GM
598This function is like @code{start-process} except that it uses a shell
599to execute the specified command. The argument @var{command} is a shell
03a74b84 600command name. The variable @code{shell-file-name} specifies which shell to
b8d4c8d0
GM
601use.
602
603The point of running a program through the shell, rather than directly
604with @code{start-process}, is so that you can employ shell features such
605as wildcards in the arguments. It follows that if you include an
606arbitrary user-specified arguments in the command, you should quote it
607with @code{shell-quote-argument} first, so that any special shell
608characters do @emph{not} have their special shell meanings. @xref{Shell
609Arguments}.
610@end defun
611
03a74b84 612@defun start-file-process-shell-command name buffer-or-name command
b8d4c8d0
GM
613This function is like @code{start-process-shell-command}, but uses
614@code{start-file-process} internally. By this, @var{command} can be
615executed also on remote hosts, depending on @code{default-directory}.
616@end defun
617
618@defvar process-connection-type
619@cindex pipes
620@cindex @acronym{PTY}s
621This variable controls the type of device used to communicate with
622asynchronous subprocesses. If it is non-@code{nil}, then @acronym{PTY}s are
623used, when available. Otherwise, pipes are used.
624
625@acronym{PTY}s are usually preferable for processes visible to the user, as
626in Shell mode, because they allow job control (@kbd{C-c}, @kbd{C-z},
627etc.) to work between the process and its children, whereas pipes do
628not. For subprocesses used for internal purposes by programs, it is
629often better to use a pipe, because they are more efficient. In
630addition, the total number of @acronym{PTY}s is limited on many systems and
631it is good not to waste them.
632
633The value of @code{process-connection-type} takes effect when
634@code{start-process} is called. So you can specify how to communicate
635with one subprocess by binding the variable around the call to
636@code{start-process}.
637
638@smallexample
639@group
640(let ((process-connection-type nil)) ; @r{Use a pipe.}
641 (start-process @dots{}))
642@end group
643@end smallexample
644
645To determine whether a given subprocess actually got a pipe or a
646@acronym{PTY}, use the function @code{process-tty-name} (@pxref{Process
647Information}).
648@end defvar
649
650@node Deleting Processes
651@section Deleting Processes
652@cindex deleting processes
653
654 @dfn{Deleting a process} disconnects Emacs immediately from the
655subprocess. Processes are deleted automatically after they terminate,
656but not necessarily right away. You can delete a process explicitly
657at any time. If you delete a terminated process explicitly before it
658is deleted automatically, no harm results. Deleting a running
659process sends a signal to terminate it (and its child processes if
660any), and calls the process sentinel if it has one. @xref{Sentinels}.
661
662 When a process is deleted, the process object itself continues to
663exist as long as other Lisp objects point to it. All the Lisp
664primitives that work on process objects accept deleted processes, but
665those that do I/O or send signals will report an error. The process
666mark continues to point to the same place as before, usually into a
667buffer where output from the process was being inserted.
668
669@defopt delete-exited-processes
670This variable controls automatic deletion of processes that have
671terminated (due to calling @code{exit} or to a signal). If it is
672@code{nil}, then they continue to exist until the user runs
673@code{list-processes}. Otherwise, they are deleted immediately after
674they exit.
675@end defopt
676
677@defun delete-process process
678This function deletes a process, killing it with a @code{SIGKILL}
679signal. The argument may be a process, the name of a process, a
680buffer, or the name of a buffer. (A buffer or buffer-name stands for
681the process that @code{get-buffer-process} returns.) Calling
682@code{delete-process} on a running process terminates it, updates the
683process status, and runs the sentinel (if any) immediately. If the
684process has already terminated, calling @code{delete-process} has no
685effect on its status, or on the running of its sentinel (which will
686happen sooner or later).
687
688@smallexample
689@group
690(delete-process "*shell*")
691 @result{} nil
692@end group
693@end smallexample
694@end defun
695
696@node Process Information
697@section Process Information
698
699 Several functions return information about processes.
700@code{list-processes} is provided for interactive use.
701
702@deffn Command list-processes &optional query-only
703This command displays a listing of all living processes. In addition,
704it finally deletes any process whose status was @samp{Exited} or
705@samp{Signaled}. It returns @code{nil}.
706
707If @var{query-only} is non-@code{nil} then it lists only processes
708whose query flag is non-@code{nil}. @xref{Query Before Exit}.
709@end deffn
710
711@defun process-list
712This function returns a list of all processes that have not been deleted.
713
714@smallexample
715@group
716(process-list)
717 @result{} (#<process display-time> #<process shell>)
718@end group
719@end smallexample
720@end defun
721
722@defun get-process name
723This function returns the process named @var{name}, or @code{nil} if
724there is none. An error is signaled if @var{name} is not a string.
725
726@smallexample
727@group
728(get-process "shell")
729 @result{} #<process shell>
730@end group
731@end smallexample
732@end defun
733
734@defun process-command process
735This function returns the command that was executed to start
736@var{process}. This is a list of strings, the first string being the
737program executed and the rest of the strings being the arguments that
738were given to the program.
739
740@smallexample
741@group
742(process-command (get-process "shell"))
743 @result{} ("/bin/csh" "-i")
744@end group
745@end smallexample
746@end defun
747
c73e02fa
GM
748@defun process-contact process &optional key
749
750This function returns information about how a network or serial
751process was set up. For a network process, when @var{key} is
752@code{nil}, it returns @code{(@var{hostname} @var{service})} which
753specifies what you connected to. For a serial process, when @var{key}
754is @code{nil}, it returns @code{(@var{port} @var{speed})}. For an
755ordinary child process, this function always returns @code{t}.
756
757If @var{key} is @code{t}, the value is the complete status information
758for the connection, server, or serial port; that is, the list of
759keywords and values specified in @code{make-network-process} or
760@code{make-serial-process}, except that some of the values represent
761the current status instead of what you specified.
762
763For a network process:
764
765@table @code
766@item :buffer
767The associated value is the process buffer.
768@item :filter
769The associated value is the process filter function.
770@item :sentinel
771The associated value is the process sentinel function.
772@item :remote
773In a connection, the address in internal format of the remote peer.
774@item :local
775The local address, in internal format.
776@item :service
777In a server, if you specified @code{t} for @var{service},
778this value is the actual port number.
779@end table
780
781@code{:local} and @code{:remote} are included even if they were not
782specified explicitly in @code{make-network-process}.
783
784For a serial process, see @code{make-serial-process} and
785@code{serial-process-configure} for a list of keys.
786
787If @var{key} is a keyword, the function returns the value corresponding
788to that keyword.
789@end defun
790
b8d4c8d0
GM
791@defun process-id process
792This function returns the @acronym{PID} of @var{process}. This is an
793integer that distinguishes the process @var{process} from all other
794processes running on the same computer at the current time. The
795@acronym{PID} of a process is chosen by the operating system kernel when the
796process is started and remains constant as long as the process exists.
797@end defun
798
799@defun process-name process
800This function returns the name of @var{process}.
801@end defun
802
803@defun process-status process-name
804This function returns the status of @var{process-name} as a symbol.
170ddd97
CY
805The argument @var{process-name} must be a process, a buffer, or a
806process name (a string).
b8d4c8d0
GM
807
808The possible values for an actual subprocess are:
809
810@table @code
811@item run
812for a process that is running.
813@item stop
814for a process that is stopped but continuable.
815@item exit
816for a process that has exited.
817@item signal
818for a process that has received a fatal signal.
819@item open
820for a network connection that is open.
821@item closed
822for a network connection that is closed. Once a connection
823is closed, you cannot reopen it, though you might be able to open
824a new connection to the same place.
825@item connect
826for a non-blocking connection that is waiting to complete.
827@item failed
828for a non-blocking connection that has failed to complete.
829@item listen
830for a network server that is listening.
831@item nil
832if @var{process-name} is not the name of an existing process.
833@end table
834
835@smallexample
836@group
b8d4c8d0
GM
837(process-status (get-buffer "*shell*"))
838 @result{} run
839@end group
840@group
841x
842 @result{} #<process xx<1>>
843(process-status x)
844 @result{} exit
845@end group
846@end smallexample
847
848For a network connection, @code{process-status} returns one of the symbols
849@code{open} or @code{closed}. The latter means that the other side
850closed the connection, or Emacs did @code{delete-process}.
851@end defun
852
c73e02fa
GM
853@defun process-type process
854This function returns the symbol @code{network} for a network
855connection or server, @code{serial} for a serial port connection, or
856@code{real} for a real subprocess.
857@end defun
858
b8d4c8d0
GM
859@defun process-exit-status process
860This function returns the exit status of @var{process} or the signal
861number that killed it. (Use the result of @code{process-status} to
862determine which of those it is.) If @var{process} has not yet
863terminated, the value is 0.
864@end defun
865
866@defun process-tty-name process
867This function returns the terminal name that @var{process} is using for
868its communication with Emacs---or @code{nil} if it is using pipes
869instead of a terminal (see @code{process-connection-type} in
870@ref{Asynchronous Processes}).
871@end defun
872
873@defun process-coding-system process
874@anchor{Coding systems for a subprocess}
875This function returns a cons cell describing the coding systems in use
876for decoding output from @var{process} and for encoding input to
877@var{process} (@pxref{Coding Systems}). The value has this form:
878
879@example
880(@var{coding-system-for-decoding} . @var{coding-system-for-encoding})
881@end example
882@end defun
883
884@defun set-process-coding-system process &optional decoding-system encoding-system
885This function specifies the coding systems to use for subsequent output
886from and input to @var{process}. It will use @var{decoding-system} to
887decode subprocess output, and @var{encoding-system} to encode subprocess
888input.
889@end defun
890
891 Every process also has a property list that you can use to store
892miscellaneous values associated with the process.
893
894@defun process-get process propname
895This function returns the value of the @var{propname} property
896of @var{process}.
897@end defun
898
899@defun process-put process propname value
900This function sets the value of the @var{propname} property
901of @var{process} to @var{value}.
902@end defun
903
904@defun process-plist process
905This function returns the process plist of @var{process}.
906@end defun
907
908@defun set-process-plist process plist
909This function sets the process plist of @var{process} to @var{plist}.
910@end defun
911
912@node Input to Processes
913@section Sending Input to Processes
914@cindex process input
915
916 Asynchronous subprocesses receive input when it is sent to them by
917Emacs, which is done with the functions in this section. You must
918specify the process to send input to, and the input data to send. The
919data appears on the ``standard input'' of the subprocess.
920
921 Some operating systems have limited space for buffered input in a
922@acronym{PTY}. On these systems, Emacs sends an @acronym{EOF}
923periodically amidst the other characters, to force them through. For
924most programs, these @acronym{EOF}s do no harm.
925
926 Subprocess input is normally encoded using a coding system before the
927subprocess receives it, much like text written into a file. You can use
928@code{set-process-coding-system} to specify which coding system to use
929(@pxref{Process Information}). Otherwise, the coding system comes from
930@code{coding-system-for-write}, if that is non-@code{nil}; or else from
931the defaulting mechanism (@pxref{Default Coding Systems}).
932
933 Sometimes the system is unable to accept input for that process,
934because the input buffer is full. When this happens, the send functions
935wait a short while, accepting output from subprocesses, and then try
936again. This gives the subprocess a chance to read more of its pending
937input and make space in the buffer. It also allows filters, sentinels
938and timers to run---so take account of that in writing your code.
939
940 In these functions, the @var{process} argument can be a process or
941the name of a process, or a buffer or buffer name (which stands
942for a process via @code{get-buffer-process}). @code{nil} means
943the current buffer's process.
944
945@defun process-send-string process string
946This function sends @var{process} the contents of @var{string} as
947standard input. If it is @code{nil}, the current buffer's process is used.
948
949 The function returns @code{nil}.
950
951@smallexample
952@group
953(process-send-string "shell<1>" "ls\n")
954 @result{} nil
955@end group
956
957
958@group
959---------- Buffer: *shell* ----------
960...
961introduction.texi syntax-tables.texi~
962introduction.texi~ text.texi
963introduction.txt text.texi~
964...
965---------- Buffer: *shell* ----------
966@end group
967@end smallexample
968@end defun
969
970@defun process-send-region process start end
971This function sends the text in the region defined by @var{start} and
972@var{end} as standard input to @var{process}.
973
974An error is signaled unless both @var{start} and @var{end} are
975integers or markers that indicate positions in the current buffer. (It
976is unimportant which number is larger.)
977@end defun
978
979@defun process-send-eof &optional process
980This function makes @var{process} see an end-of-file in its
981input. The @acronym{EOF} comes after any text already sent to it.
982
983The function returns @var{process}.
984
985@smallexample
986@group
987(process-send-eof "shell")
988 @result{} "shell"
989@end group
990@end smallexample
991@end defun
992
106e6894
CY
993@defun process-running-child-p &optional process
994This function will tell you whether a @var{process} has given control of
b8d4c8d0
GM
995its terminal to its own child process. The value is @code{t} if this is
996true, or if Emacs cannot tell; it is @code{nil} if Emacs can be certain
997that this is not so.
998@end defun
999
1000@node Signals to Processes
1001@section Sending Signals to Processes
1002@cindex process signals
1003@cindex sending signals
1004@cindex signals
1005
1006 @dfn{Sending a signal} to a subprocess is a way of interrupting its
1007activities. There are several different signals, each with its own
1008meaning. The set of signals and their names is defined by the operating
1009system. For example, the signal @code{SIGINT} means that the user has
1010typed @kbd{C-c}, or that some analogous thing has happened.
1011
1012 Each signal has a standard effect on the subprocess. Most signals
1013kill the subprocess, but some stop or resume execution instead. Most
1014signals can optionally be handled by programs; if the program handles
1015the signal, then we can say nothing in general about its effects.
1016
1017 You can send signals explicitly by calling the functions in this
1018section. Emacs also sends signals automatically at certain times:
1019killing a buffer sends a @code{SIGHUP} signal to all its associated
1020processes; killing Emacs sends a @code{SIGHUP} signal to all remaining
1021processes. (@code{SIGHUP} is a signal that usually indicates that the
1022user hung up the phone.)
1023
1024 Each of the signal-sending functions takes two optional arguments:
1025@var{process} and @var{current-group}.
1026
1027 The argument @var{process} must be either a process, a process
1028name, a buffer, a buffer name, or @code{nil}. A buffer or buffer name
1029stands for a process through @code{get-buffer-process}. @code{nil}
1030stands for the process associated with the current buffer. An error
1031is signaled if @var{process} does not identify a process.
1032
1033 The argument @var{current-group} is a flag that makes a difference
1034when you are running a job-control shell as an Emacs subprocess. If it
1035is non-@code{nil}, then the signal is sent to the current process-group
1036of the terminal that Emacs uses to communicate with the subprocess. If
1037the process is a job-control shell, this means the shell's current
1038subjob. If it is @code{nil}, the signal is sent to the process group of
1039the immediate subprocess of Emacs. If the subprocess is a job-control
1040shell, this is the shell itself.
1041
1042 The flag @var{current-group} has no effect when a pipe is used to
1043communicate with the subprocess, because the operating system does not
1044support the distinction in the case of pipes. For the same reason,
1045job-control shells won't work when a pipe is used. See
1046@code{process-connection-type} in @ref{Asynchronous Processes}.
1047
1048@defun interrupt-process &optional process current-group
1049This function interrupts the process @var{process} by sending the
1050signal @code{SIGINT}. Outside of Emacs, typing the ``interrupt
1051character'' (normally @kbd{C-c} on some systems, and @code{DEL} on
1052others) sends this signal. When the argument @var{current-group} is
1053non-@code{nil}, you can think of this function as ``typing @kbd{C-c}''
1054on the terminal by which Emacs talks to the subprocess.
1055@end defun
1056
1057@defun kill-process &optional process current-group
1058This function kills the process @var{process} by sending the
1059signal @code{SIGKILL}. This signal kills the subprocess immediately,
1060and cannot be handled by the subprocess.
1061@end defun
1062
1063@defun quit-process &optional process current-group
1064This function sends the signal @code{SIGQUIT} to the process
1065@var{process}. This signal is the one sent by the ``quit
1066character'' (usually @kbd{C-b} or @kbd{C-\}) when you are not inside
1067Emacs.
1068@end defun
1069
1070@defun stop-process &optional process current-group
1071This function stops the process @var{process} by sending the
1072signal @code{SIGTSTP}. Use @code{continue-process} to resume its
1073execution.
1074
1075Outside of Emacs, on systems with job control, the ``stop character''
1076(usually @kbd{C-z}) normally sends this signal. When
1077@var{current-group} is non-@code{nil}, you can think of this function as
1078``typing @kbd{C-z}'' on the terminal Emacs uses to communicate with the
1079subprocess.
1080@end defun
1081
1082@defun continue-process &optional process current-group
1083This function resumes execution of the process @var{process} by sending
1084it the signal @code{SIGCONT}. This presumes that @var{process} was
1085stopped previously.
1086@end defun
1087
b8d4c8d0
GM
1088@defun signal-process process signal
1089This function sends a signal to process @var{process}. The argument
1090@var{signal} specifies which signal to send; it should be an integer.
1091
1092The @var{process} argument can be a system process @acronym{ID}; that
1093allows you to send signals to processes that are not children of
23dd4ecd 1094Emacs. @xref{System Processes}.
b8d4c8d0
GM
1095@end defun
1096
1097@node Output from Processes
1098@section Receiving Output from Processes
1099@cindex process output
1100@cindex output from processes
1101
1102 There are two ways to receive the output that a subprocess writes to
1103its standard output stream. The output can be inserted in a buffer,
1104which is called the associated buffer of the process, or a function
1105called the @dfn{filter function} can be called to act on the output. If
1106the process has no buffer and no filter function, its output is
1107discarded.
1108
1109 When a subprocess terminates, Emacs reads any pending output,
1110then stops reading output from that subprocess. Therefore, if the
1111subprocess has children that are still live and still producing
1112output, Emacs won't receive that output.
1113
1114 Output from a subprocess can arrive only while Emacs is waiting: when
1115reading terminal input, in @code{sit-for} and @code{sleep-for}
1116(@pxref{Waiting}), and in @code{accept-process-output} (@pxref{Accepting
1117Output}). This minimizes the problem of timing errors that usually
1118plague parallel programming. For example, you can safely create a
1119process and only then specify its buffer or filter function; no output
1120can arrive before you finish, if the code in between does not call any
1121primitive that waits.
1122
1123@defvar process-adaptive-read-buffering
1124On some systems, when Emacs reads the output from a subprocess, the
1125output data is read in very small blocks, potentially resulting in
1126very poor performance. This behavior can be remedied to some extent
1127by setting the variable @var{process-adaptive-read-buffering} to a
1128non-@code{nil} value (the default), as it will automatically delay reading
1129from such processes, thus allowing them to produce more output before
1130Emacs tries to read it.
1131@end defvar
1132
1133 It is impossible to separate the standard output and standard error
1134streams of the subprocess, because Emacs normally spawns the subprocess
1135inside a pseudo-TTY, and a pseudo-TTY has only one output channel. If
1136you want to keep the output to those streams separate, you should
1137redirect one of them to a file---for example, by using an appropriate
1138shell command.
1139
1140@menu
1141* Process Buffers:: If no filter, output is put in a buffer.
1142* Filter Functions:: Filter functions accept output from the process.
1143* Decoding Output:: Filters can get unibyte or multibyte strings.
1144* Accepting Output:: How to wait until process output arrives.
1145@end menu
1146
1147@node Process Buffers
1148@subsection Process Buffers
1149
1150 A process can (and usually does) have an @dfn{associated buffer},
1151which is an ordinary Emacs buffer that is used for two purposes: storing
1152the output from the process, and deciding when to kill the process. You
1153can also use the buffer to identify a process to operate on, since in
1154normal practice only one process is associated with any given buffer.
1155Many applications of processes also use the buffer for editing input to
1156be sent to the process, but this is not built into Emacs Lisp.
1157
1158 Unless the process has a filter function (@pxref{Filter Functions}),
1159its output is inserted in the associated buffer. The position to insert
1160the output is determined by the @code{process-mark}, which is then
1161updated to point to the end of the text just inserted. Usually, but not
1162always, the @code{process-mark} is at the end of the buffer.
1163
16d1ff5f
CY
1164@findex process-kill-buffer-query-function
1165 Killing the associated buffer of a process also kills the process.
1166Emacs asks for confirmation first, if the process's
1167@code{process-query-on-exit-flag} is non-@code{nil} (@pxref{Query
1168Before Exit}). This confirmation is done by the function
1169@code{process-kill-buffer-query-function}, which is run from
1170@code{kill-buffer-query-functions} (@pxref{Killing Buffers}).
1171
b8d4c8d0
GM
1172@defun process-buffer process
1173This function returns the associated buffer of the process
1174@var{process}.
1175
1176@smallexample
1177@group
1178(process-buffer (get-process "shell"))
1179 @result{} #<buffer *shell*>
1180@end group
1181@end smallexample
1182@end defun
1183
1184@defun process-mark process
1185This function returns the process marker for @var{process}, which is the
1186marker that says where to insert output from the process.
1187
1188If @var{process} does not have a buffer, @code{process-mark} returns a
1189marker that points nowhere.
1190
1191Insertion of process output in a buffer uses this marker to decide where
1192to insert, and updates it to point after the inserted text. That is why
1193successive batches of output are inserted consecutively.
1194
1195Filter functions normally should use this marker in the same fashion
1196as is done by direct insertion of output in the buffer. A good
1197example of a filter function that uses @code{process-mark} is found at
1198the end of the following section.
1199
1200When the user is expected to enter input in the process buffer for
1201transmission to the process, the process marker separates the new input
1202from previous output.
1203@end defun
1204
1205@defun set-process-buffer process buffer
1206This function sets the buffer associated with @var{process} to
1207@var{buffer}. If @var{buffer} is @code{nil}, the process becomes
1208associated with no buffer.
1209@end defun
1210
1211@defun get-buffer-process buffer-or-name
1212This function returns a nondeleted process associated with the buffer
1213specified by @var{buffer-or-name}. If there are several processes
1214associated with it, this function chooses one (currently, the one most
1215recently created, but don't count on that). Deletion of a process
1216(see @code{delete-process}) makes it ineligible for this function to
1217return.
1218
1219It is usually a bad idea to have more than one process associated with
1220the same buffer.
1221
1222@smallexample
1223@group
1224(get-buffer-process "*shell*")
1225 @result{} #<process shell>
1226@end group
1227@end smallexample
1228
1229Killing the process's buffer deletes the process, which kills the
1230subprocess with a @code{SIGHUP} signal (@pxref{Signals to Processes}).
1231@end defun
1232
1233@node Filter Functions
1234@subsection Process Filter Functions
1235@cindex filter function
1236@cindex process filter
1237
1238 A process @dfn{filter function} is a function that receives the
1239standard output from the associated process. If a process has a filter,
1240then @emph{all} output from that process is passed to the filter. The
1241process buffer is used directly for output from the process only when
1242there is no filter.
1243
1244 The filter function can only be called when Emacs is waiting for
1245something, because process output arrives only at such times. Emacs
1246waits when reading terminal input, in @code{sit-for} and
1247@code{sleep-for} (@pxref{Waiting}), and in @code{accept-process-output}
1248(@pxref{Accepting Output}).
1249
1250 A filter function must accept two arguments: the associated process
1251and a string, which is output just received from it. The function is
1252then free to do whatever it chooses with the output.
1253
1254 Quitting is normally inhibited within a filter function---otherwise,
1255the effect of typing @kbd{C-g} at command level or to quit a user
1256command would be unpredictable. If you want to permit quitting inside
1257a filter function, bind @code{inhibit-quit} to @code{nil}. In most
1258cases, the right way to do this is with the macro
1259@code{with-local-quit}. @xref{Quitting}.
1260
1261 If an error happens during execution of a filter function, it is
1262caught automatically, so that it doesn't stop the execution of whatever
1263program was running when the filter function was started. However, if
1264@code{debug-on-error} is non-@code{nil}, the error-catching is turned
1265off. This makes it possible to use the Lisp debugger to debug the
1266filter function. @xref{Debugger}.
1267
1268 Many filter functions sometimes or always insert the text in the
1269process's buffer, mimicking the actions of Emacs when there is no
1270filter. Such filter functions need to use @code{set-buffer} in order to
1271be sure to insert in that buffer. To avoid setting the current buffer
1272semipermanently, these filter functions must save and restore the
1273current buffer. They should also update the process marker, and in some
1274cases update the value of point. Here is how to do these things:
1275
1276@smallexample
1277@group
1278(defun ordinary-insertion-filter (proc string)
1279 (with-current-buffer (process-buffer proc)
1280 (let ((moving (= (point) (process-mark proc))))
1281@end group
1282@group
1283 (save-excursion
1284 ;; @r{Insert the text, advancing the process marker.}
1285 (goto-char (process-mark proc))
1286 (insert string)
1287 (set-marker (process-mark proc) (point)))
1288 (if moving (goto-char (process-mark proc))))))
1289@end group
1290@end smallexample
1291
1292@noindent
1293The reason to use @code{with-current-buffer}, rather than using
1294@code{save-excursion} to save and restore the current buffer, is so as
1295to preserve the change in point made by the second call to
1296@code{goto-char}.
1297
1298 To make the filter force the process buffer to be visible whenever new
1299text arrives, insert the following line just before the
1300@code{with-current-buffer} construct:
1301
1302@smallexample
1303(display-buffer (process-buffer proc))
1304@end smallexample
1305
1306 To force point to the end of the new output, no matter where it was
1307previously, eliminate the variable @code{moving} and call
1308@code{goto-char} unconditionally.
1309
1310 In earlier Emacs versions, every filter function that did regular
1311expression searching or matching had to explicitly save and restore the
1312match data. Now Emacs does this automatically for filter functions;
1313they never need to do it explicitly. @xref{Match Data}.
1314
1315 A filter function that writes the output into the buffer of the
1316process should check whether the buffer is still alive. If it tries to
1317insert into a dead buffer, it will get an error. The expression
1318@code{(buffer-name (process-buffer @var{process}))} returns @code{nil}
1319if the buffer is dead.
1320
1321 The output to the function may come in chunks of any size. A program
1322that produces the same output twice in a row may send it as one batch of
1323200 characters one time, and five batches of 40 characters the next. If
1324the filter looks for certain text strings in the subprocess output, make
1325sure to handle the case where one of these strings is split across two
fd211f0b
CY
1326or more batches of output; one way to do this is to insert the
1327received text into a temporary buffer, which can then be searched.
b8d4c8d0
GM
1328
1329@defun set-process-filter process filter
1330This function gives @var{process} the filter function @var{filter}. If
1331@var{filter} is @code{nil}, it gives the process no filter.
1332@end defun
1333
1334@defun process-filter process
1335This function returns the filter function of @var{process}, or @code{nil}
1336if it has none.
1337@end defun
1338
1339 Here is an example of use of a filter function:
1340
1341@smallexample
1342@group
1343(defun keep-output (process output)
1344 (setq kept (cons output kept)))
1345 @result{} keep-output
1346@end group
1347@group
1348(setq kept nil)
1349 @result{} nil
1350@end group
1351@group
1352(set-process-filter (get-process "shell") 'keep-output)
1353 @result{} keep-output
1354@end group
1355@group
1356(process-send-string "shell" "ls ~/other\n")
1357 @result{} nil
1358kept
1359 @result{} ("lewis@@slug[8] % "
1360@end group
1361@group
1362"FINAL-W87-SHORT.MSS backup.otl kolstad.mss~
1363address.txt backup.psf kolstad.psf
1364backup.bib~ david.mss resume-Dec-86.mss~
1365backup.err david.psf resume-Dec.psf
1366backup.mss dland syllabus.mss
1367"
1368"#backups.mss# backup.mss~ kolstad.mss
1369")
1370@end group
1371@end smallexample
1372
1373@ignore @c The code in this example doesn't show the right way to do things.
1374Here is another, more realistic example, which demonstrates how to use
1375the process mark to do insertion in the same fashion as is done when
1376there is no filter function:
1377
1378@smallexample
1379@group
1380;; @r{Insert input in the buffer specified by @code{my-shell-buffer}}
1381;; @r{and make sure that buffer is shown in some window.}
1382(defun my-process-filter (proc str)
1383 (let ((cur (selected-window))
1384 (pop-up-windows t))
1385 (pop-to-buffer my-shell-buffer)
1386@end group
1387@group
1388 (goto-char (point-max))
1389 (insert str)
1390 (set-marker (process-mark proc) (point-max))
1391 (select-window cur)))
1392@end group
1393@end smallexample
1394@end ignore
1395
1396@node Decoding Output
1397@subsection Decoding Process Output
1398@cindex decode process output
1399
1400 When Emacs writes process output directly into a multibyte buffer,
1401it decodes the output according to the process output coding system.
1402If the coding system is @code{raw-text} or @code{no-conversion}, Emacs
1403converts the unibyte output to multibyte using
1404@code{string-to-multibyte}, and inserts the resulting multibyte text.
1405
1406 You can use @code{set-process-coding-system} to specify which coding
1407system to use (@pxref{Process Information}). Otherwise, the coding
1408system comes from @code{coding-system-for-read}, if that is
1409non-@code{nil}; or else from the defaulting mechanism (@pxref{Default
3355f04d
EZ
1410Coding Systems}). If the text output by a process contains null
1411bytes, Emacs by default uses @code{no-conversion} for it; see
1412@ref{Lisp and Coding Systems, inhibit-null-byte-detection}, for how to
1413control this behavior.
b8d4c8d0
GM
1414
1415 @strong{Warning:} Coding systems such as @code{undecided} which
1416determine the coding system from the data do not work entirely
1417reliably with asynchronous subprocess output. This is because Emacs
1418has to process asynchronous subprocess output in batches, as it
1419arrives. Emacs must try to detect the proper coding system from one
1420batch at a time, and this does not always work. Therefore, if at all
1421possible, specify a coding system that determines both the character
1422code conversion and the end of line conversion---that is, one like
1423@code{latin-1-unix}, rather than @code{undecided} or @code{latin-1}.
1424
4972c361
SM
1425@c Let's keep the index entries that were there for
1426@c set-process-filter-multibyte and process-filter-multibyte-p,
b8d4c8d0
GM
1427@cindex filter multibyte flag, of process
1428@cindex process filter multibyte flag
1429 When Emacs calls a process filter function, it provides the process
1430output as a multibyte string or as a unibyte string according to the
4972c361
SM
1431process's filter coding system. Emacs
1432decodes the output according to the process output coding system,
1433which usually produces a multibyte string, except for coding systems
1434such as @code{binary} and @code{raw-text}
b8d4c8d0
GM
1435
1436@node Accepting Output
1437@subsection Accepting Output from Processes
1438@cindex accept input from processes
1439
1440 Output from asynchronous subprocesses normally arrives only while
1441Emacs is waiting for some sort of external event, such as elapsed time
1442or terminal input. Occasionally it is useful in a Lisp program to
1443explicitly permit output to arrive at a specific point, or even to wait
1444until output arrives from a process.
1445
1446@defun accept-process-output &optional process seconds millisec just-this-one
1447This function allows Emacs to read pending output from processes. The
1448output is inserted in the associated buffers or given to their filter
1449functions. If @var{process} is non-@code{nil} then this function does
1450not return until some output has been received from @var{process}.
1451
1452@c Emacs 19 feature
1453The arguments @var{seconds} and @var{millisec} let you specify timeout
1454periods. The former specifies a period measured in seconds and the
1455latter specifies one measured in milliseconds. The two time periods
1456thus specified are added together, and @code{accept-process-output}
1457returns after that much time, whether or not there has been any
1458subprocess output.
1459
1460The argument @var{millisec} is semi-obsolete nowadays because
1461@var{seconds} can be a floating point number to specify waiting a
1462fractional number of seconds. If @var{seconds} is 0, the function
1463accepts whatever output is pending but does not wait.
1464
1465@c Emacs 22.1 feature
1466If @var{process} is a process, and the argument @var{just-this-one} is
1467non-@code{nil}, only output from that process is handled, suspending output
1468from other processes until some output has been received from that
1469process or the timeout expires. If @var{just-this-one} is an integer,
1470also inhibit running timers. This feature is generally not
1471recommended, but may be necessary for specific applications, such as
1472speech synthesis.
1473
1474The function @code{accept-process-output} returns non-@code{nil} if it
1475did get some output, or @code{nil} if the timeout expired before output
1476arrived.
1477@end defun
1478
1479@node Sentinels
1480@section Sentinels: Detecting Process Status Changes
1481@cindex process sentinel
1482@cindex sentinel (of process)
1483
1484 A @dfn{process sentinel} is a function that is called whenever the
1485associated process changes status for any reason, including signals
1486(whether sent by Emacs or caused by the process's own actions) that
1487terminate, stop, or continue the process. The process sentinel is
1488also called if the process exits. The sentinel receives two
1489arguments: the process for which the event occurred, and a string
1490describing the type of event.
1491
1492 The string describing the event looks like one of the following:
1493
1494@itemize @bullet
1495@item
1496@code{"finished\n"}.
1497
1498@item
1499@code{"exited abnormally with code @var{exitcode}\n"}.
1500
1501@item
1502@code{"@var{name-of-signal}\n"}.
1503
1504@item
1505@code{"@var{name-of-signal} (core dumped)\n"}.
1506@end itemize
1507
1508 A sentinel runs only while Emacs is waiting (e.g., for terminal
1509input, or for time to elapse, or for process output). This avoids the
1510timing errors that could result from running them at random places in
1511the middle of other Lisp programs. A program can wait, so that
1512sentinels will run, by calling @code{sit-for} or @code{sleep-for}
1513(@pxref{Waiting}), or @code{accept-process-output} (@pxref{Accepting
1514Output}). Emacs also allows sentinels to run when the command loop is
1515reading input. @code{delete-process} calls the sentinel when it
1516terminates a running process.
1517
1518 Emacs does not keep a queue of multiple reasons to call the sentinel
1519of one process; it records just the current status and the fact that
1520there has been a change. Therefore two changes in status, coming in
1521quick succession, can call the sentinel just once. However, process
1522termination will always run the sentinel exactly once. This is
1523because the process status can't change again after termination.
1524
1525 Emacs explicitly checks for output from the process before running
1526the process sentinel. Once the sentinel runs due to process
1527termination, no further output can arrive from the process.
1528
1529 A sentinel that writes the output into the buffer of the process
1530should check whether the buffer is still alive. If it tries to insert
1531into a dead buffer, it will get an error. If the buffer is dead,
1532@code{(buffer-name (process-buffer @var{process}))} returns @code{nil}.
1533
1534 Quitting is normally inhibited within a sentinel---otherwise, the
1535effect of typing @kbd{C-g} at command level or to quit a user command
1536would be unpredictable. If you want to permit quitting inside a
1537sentinel, bind @code{inhibit-quit} to @code{nil}. In most cases, the
1538right way to do this is with the macro @code{with-local-quit}.
1539@xref{Quitting}.
1540
1541 If an error happens during execution of a sentinel, it is caught
1542automatically, so that it doesn't stop the execution of whatever
1543programs was running when the sentinel was started. However, if
1544@code{debug-on-error} is non-@code{nil}, the error-catching is turned
1545off. This makes it possible to use the Lisp debugger to debug the
1546sentinel. @xref{Debugger}.
1547
1548 While a sentinel is running, the process sentinel is temporarily
1549set to @code{nil} so that the sentinel won't run recursively.
1550For this reason it is not possible for a sentinel to specify
1551a new sentinel.
1552
1553 In earlier Emacs versions, every sentinel that did regular expression
1554searching or matching had to explicitly save and restore the match data.
1555Now Emacs does this automatically for sentinels; they never need to do
1556it explicitly. @xref{Match Data}.
1557
1558@defun set-process-sentinel process sentinel
1559This function associates @var{sentinel} with @var{process}. If
1560@var{sentinel} is @code{nil}, then the process will have no sentinel.
1561The default behavior when there is no sentinel is to insert a message in
1562the process's buffer when the process status changes.
1563
1564Changes in process sentinel take effect immediately---if the sentinel
1565is slated to be run but has not been called yet, and you specify a new
1566sentinel, the eventual call to the sentinel will use the new one.
1567
1568@smallexample
1569@group
1570(defun msg-me (process event)
1571 (princ
1572 (format "Process: %s had the event `%s'" process event)))
1573(set-process-sentinel (get-process "shell") 'msg-me)
1574 @result{} msg-me
1575@end group
1576@group
1577(kill-process (get-process "shell"))
1578 @print{} Process: #<process shell> had the event `killed'
1579 @result{} #<process shell>
1580@end group
1581@end smallexample
1582@end defun
1583
1584@defun process-sentinel process
1585This function returns the sentinel of @var{process}, or @code{nil} if it
1586has none.
1587@end defun
1588
1589@defun waiting-for-user-input-p
1590While a sentinel or filter function is running, this function returns
1591non-@code{nil} if Emacs was waiting for keyboard input from the user at
1592the time the sentinel or filter function was called, @code{nil} if it
1593was not.
1594@end defun
1595
1596@node Query Before Exit
1597@section Querying Before Exit
1598
1599 When Emacs exits, it terminates all its subprocesses by sending them
1600the @code{SIGHUP} signal. Because subprocesses may be doing
1601valuable work, Emacs normally asks the user to confirm that it is ok
1602to terminate them. Each process has a query flag which, if
1603non-@code{nil}, says that Emacs should ask for confirmation before
1604exiting and thus killing that process. The default for the query flag
1605is @code{t}, meaning @emph{do} query.
1606
1607@defun process-query-on-exit-flag process
1608This returns the query flag of @var{process}.
1609@end defun
1610
1611@defun set-process-query-on-exit-flag process flag
1612This function sets the query flag of @var{process} to @var{flag}. It
1613returns @var{flag}.
1614
1615@smallexample
1616@group
1617;; @r{Don't query about the shell process}
1618(set-process-query-on-exit-flag (get-process "shell") nil)
1619 @result{} t
1620@end group
1621@end smallexample
1622@end defun
1623
1624@defun process-kill-without-query process &optional do-query
1625This function clears the query flag of @var{process}, so that
1626Emacs will not query the user on account of that process.
1627
1628Actually, the function does more than that: it returns the old value of
1629the process's query flag, and sets the query flag to @var{do-query}.
1630Please don't use this function to do those things any more---please
1631use the newer, cleaner functions @code{process-query-on-exit-flag} and
1632@code{set-process-query-on-exit-flag} in all but the simplest cases.
1633The only way you should use @code{process-kill-without-query} nowadays
1634is like this:
1635
1636@smallexample
1637@group
1638;; @r{Don't query about the shell process}
1639(process-kill-without-query (get-process "shell"))
1640@end group
1641@end smallexample
1642@end defun
1643
23dd4ecd
EZ
1644@node System Processes
1645@section Accessing Other Processes
1646@cindex system processes
1647
1648 In addition to accessing and manipulating processes that are
1649subprocesses of the current Emacs session, Emacs Lisp programs can
1650also access other processes running on the same machine. We call
1651these @dfn{system processes}, to distinguish between them and Emacs
1652subprocesses.
1653
1654 Emacs provides several primitives for accessing system processes.
1655Not all platforms support these primitives; on those which don't,
1656these primitives return @code{nil}.
1657
1658@defun list-system-processes
1659This function returns a list of all the processes running on the
1660system. Each process is identified by its @acronym{PID}, a numerical
1661process ID that is assigned by the OS and distinguishes the process
1662from all the other processes running on the same machine at the same
1663time.
1664@end defun
1665
a20878b6 1666@defun process-attributes pid
23dd4ecd
EZ
1667This function returns an alist of attributes for the process specified
1668by its process ID @var{pid}. Each association in the alist is of the
1669form @code{(@var{key} . @var{value})}, where @var{key} designates the
1670attribute and @var{value} is the value of that attribute. The various
1671attribute @var{key}'s that this function can return are listed below.
1672Not all platforms support all of these attributes; if an attribute is
1673not supported, its association will not appear in the returned alist.
1674Values that are numbers can be either integer or floating-point,
1675depending on the magnitude of the value.
1676
1677@table @code
1678@item euid
1679The effective user ID of the user who invoked the process. The
1680corresponding @var{value} is a number. If the process was invoked by
1681the same user who runs the current Emacs session, the value is
1682identical to what @code{user-uid} returns (@pxref{User
1683Identification}).
1684
1685@item user
1686User name corresponding to the process's effective user ID, a string.
1687
1688@item egid
1689The group ID of the effective user ID, a number.
1690
1691@item group
1692Group name corresponding to the effective user's group ID, a string.
1693
1694@item comm
1695The name of the command that runs in the process. This is a string
1696that usually specifies the name of the executable file of the process,
1697without the leading directories. However, some special system
1698processes can report strings that do not correspond to an executable
1699file of a program.
1700
1701@item state
1702The state code of the process. This is a short string that encodes
1703the scheduling state of the process. Here's a list of the most
1704frequently seen codes:
1705
1706@table @code
1dca458f 1707@item "D"
23dd4ecd 1708uninterruptible sleep (usually I/O)
1dca458f 1709@item "R"
23dd4ecd 1710running
1dca458f 1711@item "S"
23dd4ecd 1712interruptible sleep (waiting for some event)
1dca458f 1713@item "T"
23dd4ecd 1714stopped, e.g., by a job control signal
1dca458f
EZ
1715@item "Z"
1716``zombie'': a process that terminated, but was not reaped by its parent
23dd4ecd
EZ
1717@end table
1718
1719@noindent
1720For the full list of the possible states, see the manual page of the
1721@command{ps} command.
1722
1723@item ppid
1724The process ID of the parent process, a number.
1725
1726@item pgrp
1727The process group ID of the process, a number.
1728
1729@item sess
1730The session ID of the process. This is a number that is the process
1731ID of the process's @dfn{session leader}.
1732
1733@item ttname
1734A string that is the name of the process's controlling terminal. On
1735Unix and GNU systems, this is normally the file name of the
1736corresponding terminal device, such as @file{/dev/pts65}.
1737
1738@item tpgid
1739The numerical process group ID of the foreground process group that
1740uses the process's terminal.
1741
1742@item minflt
1743The number of minor page faults caused by the process since its
1744beginning. (Minor page faults are those that don't involve reading
1745from disk.)
1746
1747@item majflt
1748The number of major page faults caused by the process since its
1749beginning. (Major page faults require a disk to be read, and are thus
1750more expensive than minor page faults.)
1751
1752@item cminflt
1753@itemx cmajflt
1754Like @code{minflt} and @code{majflt}, but include the number of page
1755faults for all the child processes of the given process.
1756
1757@item utime
1758Time spent by the process in the user context, for running the
1759application's code. The corresponding @var{value} is in the
1760@w{@code{(@var{high} @var{low} @var{microsec})}} format, the same
1761format used by functions @code{current-time} (@pxref{Time of Day,
1762current-time}) and @code{file-attributes} (@pxref{File Attributes}).
1763
1764@item stime
1765Time spent by the process in the system (kernel) context, for
1766processing system calls. The corresponding @var{value} is in the same
1767format as for @code{utime}.
1768
af34ad36
EZ
1769@item time
1770The sum of @code{utime} and @code{stime}. The corresponding
1771@var{value} is in the same format as for @code{utime}.
1772
23dd4ecd
EZ
1773@item cutime
1774@itemx cstime
af34ad36
EZ
1775@itemx ctime
1776Like @code{utime}, @code{stime}, and @code{time}, but include the
1777times of all the child processes of the given process.
23dd4ecd
EZ
1778
1779@item pri
1780The numerical priority of the process.
1781
1782@item nice
1dca458f
EZ
1783The @dfn{nice value} of the process, a number. (Processes with smaller
1784nice values get scheduled more favorably.)
23dd4ecd
EZ
1785
1786@item thcount
1787The number of threads in the process.
1788
1789@item start
1790The time the process was started, in the @w{@code{(@var{high}
1791@var{low} @var{microsec})}} format used by @code{current-time} and
1792@code{file-attributes}.
1793
1794@item etime
1795The time elapsed since the process started, in the @w{@code{(@var{high}
1796@var{low} @var{microsec})}} format.
1797
1798@item vsize
1799The virtual memory size of the process, measured in kilobytes.
1800
1801@item rss
1802The size of the process's @dfn{resident set}, the number of kilobytes
1803occupied by the process in the machine's physical memory.
1804
1805@item pcpu
1806The percentage of the CPU time used by the process since it started.
1807The corresponding @var{value} is a floating-point number between 0 and
1808100.
1809
1810@item pmem
1811The percentage of the total physical memory installed on the machine
1812used by the process's resident set. The value is a floating-point
1813number between 0 and 100.
1814
1815@item args
1816The command-line with which the process was invoked. This is a string
1817in which individual command-line arguments are separated by blanks;
1818whitespace characters that are embedded in the arguments are quoted as
1819appropriate for the system's shell: escaped by backslash characters on
1820GNU and Unix, and enclosed in double quote characters on Windows.
1821Thus, this command-line string can be directly used in primitives such
1822as @code{shell-command}.
1823@end table
1824
1825@end defun
1826
1827
b8d4c8d0
GM
1828@node Transaction Queues
1829@section Transaction Queues
1830@cindex transaction queue
1831
1832You can use a @dfn{transaction queue} to communicate with a subprocess
1833using transactions. First use @code{tq-create} to create a transaction
1834queue communicating with a specified process. Then you can call
1835@code{tq-enqueue} to send a transaction.
1836
1837@defun tq-create process
1838This function creates and returns a transaction queue communicating with
1839@var{process}. The argument @var{process} should be a subprocess
1840capable of sending and receiving streams of bytes. It may be a child
1841process, or it may be a TCP connection to a server, possibly on another
1842machine.
1843@end defun
1844
1845@defun tq-enqueue queue question regexp closure fn &optional delay-question
1846This function sends a transaction to queue @var{queue}. Specifying the
1847queue has the effect of specifying the subprocess to talk to.
1848
1849The argument @var{question} is the outgoing message that starts the
1850transaction. The argument @var{fn} is the function to call when the
1851corresponding answer comes back; it is called with two arguments:
1852@var{closure}, and the answer received.
1853
1854The argument @var{regexp} is a regular expression that should match
1855text at the end of the entire answer, but nothing before; that's how
1856@code{tq-enqueue} determines where the answer ends.
1857
4185820c
CY
1858If the argument @var{delay-question} is non-@code{nil}, delay sending
1859this question until the process has finished replying to any previous
b8d4c8d0
GM
1860questions. This produces more reliable results with some processes.
1861
1862The return value of @code{tq-enqueue} itself is not meaningful.
1863@end defun
1864
1865@defun tq-close queue
1866Shut down transaction queue @var{queue}, waiting for all pending transactions
1867to complete, and then terminate the connection or child process.
1868@end defun
1869
1870Transaction queues are implemented by means of a filter function.
1871@xref{Filter Functions}.
1872
1873@node Network
1874@section Network Connections
1875@cindex network connection
1876@cindex TCP
1877@cindex UDP
1878
1879 Emacs Lisp programs can open stream (TCP) and datagram (UDP) network
1880connections to other processes on the same machine or other machines.
1881A network connection is handled by Lisp much like a subprocess, and is
1882represented by a process object. However, the process you are
1883communicating with is not a child of the Emacs process, so it has no
1884process @acronym{ID}, and you can't kill it or send it signals. All you
1885can do is send and receive data. @code{delete-process} closes the
1886connection, but does not kill the program at the other end; that
1887program must decide what to do about closure of the connection.
1888
1889 Lisp programs can listen for connections by creating network
1890servers. A network server is also represented by a kind of process
1891object, but unlike a network connection, the network server never
1892transfers data itself. When it receives a connection request, it
1893creates a new network connection to represent the connection just
1894made. (The network connection inherits certain information, including
1895the process plist, from the server.) The network server then goes
1896back to listening for more connection requests.
1897
1898 Network connections and servers are created by calling
1899@code{make-network-process} with an argument list consisting of
1900keyword/argument pairs, for example @code{:server t} to create a
1901server process, or @code{:type 'datagram} to create a datagram
1902connection. @xref{Low-Level Network}, for details. You can also use
1903the @code{open-network-stream} function described below.
1904
c73e02fa
GM
1905 To distinguish the different types of processes, the
1906@code{process-type} function returns the symbol @code{network} for a
1907network connection or server, @code{serial} for a serial port
1908connection, or @code{real} for a real subprocess.
1909
1910 The @code{process-status} function returns @code{open},
1911@code{closed}, @code{connect}, and @code{failed} for network
1912connections. For a network server, the status is always
b8d4c8d0
GM
1913@code{listen}. None of those values is possible for a real
1914subprocess. @xref{Process Information}.
1915
1916 You can stop and resume operation of a network process by calling
1917@code{stop-process} and @code{continue-process}. For a server
1918process, being stopped means not accepting new connections. (Up to 5
1919connection requests will be queued for when you resume the server; you
1920can increase this limit, unless it is imposed by the operating
1921system.) For a network stream connection, being stopped means not
1922processing input (any arriving input waits until you resume the
1923connection). For a datagram connection, some number of packets may be
1924queued but input may be lost. You can use the function
1925@code{process-command} to determine whether a network connection or
1926server is stopped; a non-@code{nil} value means yes.
1927
1928@defun open-network-stream name buffer-or-name host service
1929This function opens a TCP connection, and returns a process object
1930that represents the connection.
1931
1932The @var{name} argument specifies the name for the process object. It
1933is modified as necessary to make it unique.
1934
1935The @var{buffer-or-name} argument is the buffer to associate with the
1936connection. Output from the connection is inserted in the buffer,
1937unless you specify a filter function to handle the output. If
1938@var{buffer-or-name} is @code{nil}, it means that the connection is not
1939associated with any buffer.
1940
1941The arguments @var{host} and @var{service} specify where to connect to;
1942@var{host} is the host name (a string), and @var{service} is the name of
1943a defined network service (a string) or a port number (an integer).
1944@end defun
1945
b8d4c8d0
GM
1946@node Network Servers
1947@section Network Servers
1948@cindex network servers
1949
1950 You create a server by calling @code{make-network-process} with
1951@code{:server t}. The server will listen for connection requests from
1952clients. When it accepts a client connection request, that creates a
1953new network connection, itself a process object, with the following
1954parameters:
1955
1956@itemize @bullet
1957@item
1958The connection's process name is constructed by concatenating the
1959server process' @var{name} with a client identification string. The
1960client identification string for an IPv4 connection looks like
1961@samp{<@var{a}.@var{b}.@var{c}.@var{d}:@var{p}>}. Otherwise, it is a
1962unique number in brackets, as in @samp{<@var{nnn}>}. The number
1963is unique for each connection in the Emacs session.
1964
1965@item
1966If the server's filter is non-@code{nil}, the connection process does
1967not get a separate process buffer; otherwise, Emacs creates a new
1968buffer for the purpose. The buffer name is the server's buffer name
1969or process name, concatenated with the client identification string.
1970
1971The server's process buffer value is never used directly by Emacs, but
1972it is passed to the log function, which can log connections by
1973inserting text there.
1974
1975@item
1976The communication type and the process filter and sentinel are
1977inherited from those of the server. The server never directly
1978uses its filter and sentinel; their sole purpose is to initialize
1979connections made to the server.
1980
1981@item
1982The connection's process contact info is set according to the client's
1983addressing information (typically an IP address and a port number).
1984This information is associated with the @code{process-contact}
1985keywords @code{:host}, @code{:service}, @code{:remote}.
1986
1987@item
1988The connection's local address is set up according to the port
1989number used for the connection.
1990
1991@item
1992The client process' plist is initialized from the server's plist.
1993@end itemize
1994
1995@node Datagrams
1996@section Datagrams
1997@cindex datagrams
1998
1999 A datagram connection communicates with individual packets rather
2000than streams of data. Each call to @code{process-send} sends one
2001datagram packet (@pxref{Input to Processes}), and each datagram
2002received results in one call to the filter function.
2003
2004 The datagram connection doesn't have to talk with the same remote
2005peer all the time. It has a @dfn{remote peer address} which specifies
2006where to send datagrams to. Each time an incoming datagram is passed
2007to the filter function, the peer address is set to the address that
2008datagram came from; that way, if the filter function sends a datagram,
2009it will go back to that place. You can specify the remote peer
2010address when you create the datagram connection using the
2011@code{:remote} keyword. You can change it later on by calling
2012@code{set-process-datagram-address}.
2013
2014@defun process-datagram-address process
2015If @var{process} is a datagram connection or server, this function
2016returns its remote peer address.
2017@end defun
2018
2019@defun set-process-datagram-address process address
2020If @var{process} is a datagram connection or server, this function
2021sets its remote peer address to @var{address}.
2022@end defun
2023
2024@node Low-Level Network
2025@section Low-Level Network Access
2026
2027 You can also create network connections by operating at a lower
2028level than that of @code{open-network-stream}, using
2029@code{make-network-process}.
2030
2031@menu
2032* Proc: Network Processes. Using @code{make-network-process}.
2033* Options: Network Options. Further control over network connections.
2034* Features: Network Feature Testing.
2035 Determining which network features work on
2036 the machine you are using.
2037@end menu
2038
2039@node Network Processes
2040@subsection @code{make-network-process}
2041
2042 The basic function for creating network connections and network
2043servers is @code{make-network-process}. It can do either of those
2044jobs, depending on the arguments you give it.
2045
2046@defun make-network-process &rest args
2047This function creates a network connection or server and returns the
2048process object that represents it. The arguments @var{args} are a
2049list of keyword/argument pairs. Omitting a keyword is always
2050equivalent to specifying it with value @code{nil}, except for
2051@code{:coding}, @code{:filter-multibyte}, and @code{:reuseaddr}. Here
2052are the meaningful keywords:
2053
2054@table @asis
2055@item :name @var{name}
2056Use the string @var{name} as the process name. It is modified if
2057necessary to make it unique.
2058
2059@item :type @var{type}
2060Specify the communication type. A value of @code{nil} specifies a
2061stream connection (the default); @code{datagram} specifies a datagram
f7704e26
CY
2062connection; @code{seqpacket} specifies a ``sequenced packet stream''
2063connection. Both connections and servers can be of these types.
b8d4c8d0
GM
2064
2065@item :server @var{server-flag}
2066If @var{server-flag} is non-@code{nil}, create a server. Otherwise,
2067create a connection. For a stream type server, @var{server-flag} may
2068be an integer which then specifies the length of the queue of pending
2069connections to the server. The default queue length is 5.
2070
2071@item :host @var{host}
2072Specify the host to connect to. @var{host} should be a host name or
2073Internet address, as a string, or the symbol @code{local} to specify
2074the local host. If you specify @var{host} for a server, it must
2075specify a valid address for the local host, and only clients
2076connecting to that address will be accepted.
2077
2078@item :service @var{service}
2079@var{service} specifies a port number to connect to, or, for a server,
2080the port number to listen on. It should be a service name that
2081translates to a port number, or an integer specifying the port number
2082directly. For a server, it can also be @code{t}, which means to let
2083the system select an unused port number.
2084
2085@item :family @var{family}
2086@var{family} specifies the address (and protocol) family for
2087communication. @code{nil} means determine the proper address family
2088automatically for the given @var{host} and @var{service}.
2089@code{local} specifies a Unix socket, in which case @var{host} is
2090ignored. @code{ipv4} and @code{ipv6} specify to use IPv4 and IPv6
2091respectively.
2092
2093@item :local @var{local-address}
2094For a server process, @var{local-address} is the address to listen on.
2095It overrides @var{family}, @var{host} and @var{service}, and you
2096may as well not specify them.
2097
2098@item :remote @var{remote-address}
2099For a connection, @var{remote-address} is the address to connect to.
2100It overrides @var{family}, @var{host} and @var{service}, and you
2101may as well not specify them.
2102
2103For a datagram server, @var{remote-address} specifies the initial
2104setting of the remote datagram address.
2105
2106The format of @var{local-address} or @var{remote-address} depends on
2107the address family:
2108
2109@itemize -
2110@item
2111An IPv4 address is represented as a five-element vector of four 8-bit
2112integers and one 16-bit integer
2113@code{[@var{a} @var{b} @var{c} @var{d} @var{p}]} corresponding to
2114numeric IPv4 address @var{a}.@var{b}.@var{c}.@var{d} and port number
2115@var{p}.
2116
2117@item
2118An IPv6 address is represented as a nine-element vector of 16-bit
2119integers @code{[@var{a} @var{b} @var{c} @var{d} @var{e} @var{f}
2120@var{g} @var{h} @var{p}]} corresponding to numeric IPv6 address
2121@var{a}:@var{b}:@var{c}:@var{d}:@var{e}:@var{f}:@var{g}:@var{h} and
2122port number @var{p}.
2123
2124@item
2125A local address is represented as a string which specifies the address
2126in the local address space.
2127
2128@item
2129An ``unsupported family'' address is represented by a cons
2130@code{(@var{f} . @var{av})}, where @var{f} is the family number and
2131@var{av} is a vector specifying the socket address using one element
2132per address data byte. Do not rely on this format in portable code,
2133as it may depend on implementation defined constants, data sizes, and
2134data structure alignment.
2135@end itemize
2136
2137@item :nowait @var{bool}
2138If @var{bool} is non-@code{nil} for a stream connection, return
2139without waiting for the connection to complete. When the connection
2140succeeds or fails, Emacs will call the sentinel function, with a
2141second argument matching @code{"open"} (if successful) or
2142@code{"failed"}. The default is to block, so that
2143@code{make-network-process} does not return until the connection
2144has succeeded or failed.
2145
2146@item :stop @var{stopped}
2147Start the network connection or server in the `stopped' state if
2148@var{stopped} is non-@code{nil}.
2149
2150@item :buffer @var{buffer}
2151Use @var{buffer} as the process buffer.
2152
2153@item :coding @var{coding}
2154Use @var{coding} as the coding system for this process. To specify
2155different coding systems for decoding data from the connection and for
2156encoding data sent to it, specify @code{(@var{decoding} .
2157@var{encoding})} for @var{coding}.
2158
2159If you don't specify this keyword at all, the default
2160is to determine the coding systems from the data.
2161
2162@item :noquery @var{query-flag}
2163Initialize the process query flag to @var{query-flag}.
2164@xref{Query Before Exit}.
2165
2166@item :filter @var{filter}
2167Initialize the process filter to @var{filter}.
2168
b8d4c8d0
GM
2169@item :sentinel @var{sentinel}
2170Initialize the process sentinel to @var{sentinel}.
2171
2172@item :log @var{log}
2173Initialize the log function of a server process to @var{log}. The log
2174function is called each time the server accepts a network connection
2175from a client. The arguments passed to the log function are
2176@var{server}, @var{connection}, and @var{message}, where @var{server}
2177is the server process, @var{connection} is the new process for the
2178connection, and @var{message} is a string describing what has
2179happened.
2180
2181@item :plist @var{plist}
2182Initialize the process plist to @var{plist}.
2183@end table
2184
2185The original argument list, modified with the actual connection
2186information, is available via the @code{process-contact} function.
2187@end defun
2188
2189@node Network Options
2190@subsection Network Options
2191
2192 The following network options can be specified when you create a
2193network process. Except for @code{:reuseaddr}, you can also set or
2194modify these options later, using @code{set-network-process-option}.
2195
2196 For a server process, the options specified with
2197@code{make-network-process} are not inherited by the client
2198connections, so you will need to set the necessary options for each
2199child connection as it is created.
2200
2201@table @asis
2202@item :bindtodevice @var{device-name}
2203If @var{device-name} is a non-empty string identifying a network
2204interface name (see @code{network-interface-list}), only handle
2205packets received on that interface. If @var{device-name} is @code{nil}
2206(the default), handle packets received on any interface.
2207
2208Using this option may require special privileges on some systems.
2209
2210@item :broadcast @var{broadcast-flag}
2211If @var{broadcast-flag} is non-@code{nil} for a datagram process, the
2212process will receive datagram packet sent to a broadcast address, and
2213be able to send packets to a broadcast address. Ignored for a stream
2214connection.
2215
2216@item :dontroute @var{dontroute-flag}
2217If @var{dontroute-flag} is non-@code{nil}, the process can only send
2218to hosts on the same network as the local host.
2219
2220@item :keepalive @var{keepalive-flag}
2221If @var{keepalive-flag} is non-@code{nil} for a stream connection,
2222enable exchange of low-level keep-alive messages.
2223
2224@item :linger @var{linger-arg}
2225If @var{linger-arg} is non-@code{nil}, wait for successful
2226transmission of all queued packets on the connection before it is
2227deleted (see @code{delete-process}). If @var{linger-arg} is an
2228integer, it specifies the maximum time in seconds to wait for queued
2229packets to be sent before closing the connection. Default is
2230@code{nil} which means to discard unsent queued packets when the
2231process is deleted.
2232
2233@item :oobinline @var{oobinline-flag}
2234If @var{oobinline-flag} is non-@code{nil} for a stream connection,
2235receive out-of-band data in the normal data stream. Otherwise, ignore
2236out-of-band data.
2237
2238@item :priority @var{priority}
2239Set the priority for packets sent on this connection to the integer
2240@var{priority}. The interpretation of this number is protocol
2241specific, such as setting the TOS (type of service) field on IP
2242packets sent on this connection. It may also have system dependent
2243effects, such as selecting a specific output queue on the network
2244interface.
2245
2246@item :reuseaddr @var{reuseaddr-flag}
2247If @var{reuseaddr-flag} is non-@code{nil} (the default) for a stream
2248server process, allow this server to reuse a specific port number (see
2249@code{:service}) unless another process on this host is already
2250listening on that port. If @var{reuseaddr-flag} is @code{nil}, there
2251may be a period of time after the last use of that port (by any
2252process on the host), where it is not possible to make a new server on
2253that port.
2254@end table
2255
106e6894 2256@defun set-network-process-option process option value &optional no-error
b8d4c8d0
GM
2257This function sets or modifies a network option for network process
2258@var{process}. See @code{make-network-process} for details of options
106e6894
CY
2259@var{option} and their corresponding values @var{value}. If
2260@var{no-error} is non-@code{nil}, this function returns @code{nil}
2261instead of signaling an error if @var{option} is not a supported
2262option. If the function successfully completes, it returns @code{t}.
b8d4c8d0
GM
2263
2264The current setting of an option is available via the
2265@code{process-contact} function.
2266@end defun
2267
2268@node Network Feature Testing
2269@subsection Testing Availability of Network Features
2270
2271 To test for the availability of a given network feature, use
2272@code{featurep} like this:
2273
2274@example
2275(featurep 'make-network-process '(@var{keyword} @var{value}))
2276@end example
2277
2278@noindent
2279The result of the first form is @code{t} if it works to specify
2280@var{keyword} with value @var{value} in @code{make-network-process}.
2281The result of the second form is @code{t} if @var{keyword} is
2282supported by @code{make-network-process}. Here are some of the
2283@var{keyword}---@var{value} pairs you can test in
2284this way.
2285
2286@table @code
2287@item (:nowait t)
2288Non-@code{nil} if non-blocking connect is supported.
2289@item (:type datagram)
2290Non-@code{nil} if datagrams are supported.
2291@item (:family local)
2292Non-@code{nil} if local (a.k.a.@: ``UNIX domain'') sockets are supported.
2293@item (:family ipv6)
2294Non-@code{nil} if IPv6 is supported.
2295@item (:service t)
2296Non-@code{nil} if the system can select the port for a server.
2297@end table
2298
2299 To test for the availability of a given network option, use
2300@code{featurep} like this:
2301
2302@example
2303(featurep 'make-network-process '@var{keyword})
2304@end example
2305
2306@noindent
2307Here are some of the options you can test in this way.
2308
2309@table @code
2310@item :bindtodevice
2311@itemx :broadcast
2312@itemx :dontroute
2313@itemx :keepalive
2314@itemx :linger
2315@itemx :oobinline
2316@itemx :priority
2317@itemx :reuseaddr
2318That particular network option is supported by
2319@code{make-network-process} and @code{set-network-process-option}.
2320@end table
2321
2322@node Misc Network
2323@section Misc Network Facilities
2324
2325 These additional functions are useful for creating and operating
305a7ef2
EZ
2326on network connections. Note that they are supported only on some
2327systems.
b8d4c8d0
GM
2328
2329@defun network-interface-list
2330This function returns a list describing the network interfaces
2331of the machine you are using. The value is an alist whose
2332elements have the form @code{(@var{name} . @var{address})}.
2333@var{address} has the same form as the @var{local-address}
2334and @var{remote-address} arguments to @code{make-network-process}.
2335@end defun
2336
2337@defun network-interface-info ifname
2338This function returns information about the network interface named
2339@var{ifname}. The value is a list of the form
2340@code{(@var{addr} @var{bcast} @var{netmask} @var{hwaddr} @var{flags})}.
2341
2342@table @var
2343@item addr
2344The Internet protocol address.
2345@item bcast
2346The broadcast address.
2347@item netmask
2348The network mask.
2349@item hwaddr
2350The layer 2 address (Ethernet MAC address, for instance).
2351@item flags
2352The current flags of the interface.
2353@end table
2354@end defun
2355
2356@defun format-network-address address &optional omit-port
2357This function converts the Lisp representation of a network address to
2358a string.
2359
2360A five-element vector @code{[@var{a} @var{b} @var{c} @var{d} @var{p}]}
2361represents an IPv4 address @var{a}.@var{b}.@var{c}.@var{d} and port
2362number @var{p}. @code{format-network-address} converts that to the
2363string @code{"@var{a}.@var{b}.@var{c}.@var{d}:@var{p}"}.
2364
2365A nine-element vector @code{[@var{a} @var{b} @var{c} @var{d} @var{e}
2366@var{f} @var{g} @var{h} @var{p}]} represents an IPv6 address along
2367with a port number. @code{format-network-address} converts that to
2368the string
2369@code{"[@var{a}:@var{b}:@var{c}:@var{d}:@var{e}:@var{f}:@var{g}:@var{h}]:@var{p}"}.
2370
2371If the vector does not include the port number, @var{p}, or if
2372@var{omit-port} is non-@code{nil}, the result does not include the
2373@code{:@var{p}} suffix.
2374@end defun
2375
c73e02fa
GM
2376@node Serial Ports
2377@section Communicating with Serial Ports
2378@cindex @file{/dev/tty}
2379@cindex @file{COM1}
545c2782 2380@cindex serial connections
c73e02fa
GM
2381
2382 Emacs can communicate with serial ports. For interactive use,
2383@kbd{M-x serial-term} opens a terminal window. In a Lisp program,
2384@code{make-serial-process} creates a process object.
2385
2386 The serial port can be configured at run-time, without having to
2387close and re-open it. The function @code{serial-process-configure}
2388lets you change the speed, bytesize, and other parameters. In a
2389terminal window created by @code{serial-term}, you can click on the
2390mode line for configuration.
2391
2392 A serial connection is represented by a process object which can be
2393used similar to a subprocess or network process. You can send and
2394receive data and configure the serial port. A serial process object
80e6b6df
EZ
2395has no process ID, you can't send signals to it, and the status codes
2396are different from other types of processes.
c73e02fa
GM
2397@code{delete-process} on the process object or @code{kill-buffer} on
2398the process buffer close the connection, but this does not affect the
2399device connected to the serial port.
2400
2401 The function @code{process-type} returns the symbol @code{serial}
80e6b6df 2402for a process object representing a serial port connection.
c73e02fa
GM
2403
2404 Serial ports are available on GNU/Linux, Unix, and Windows systems.
2405
80e6b6df 2406@deffn Command serial-term port speed
c73e02fa 2407Start a terminal-emulator for a serial port in a new buffer.
80e6b6df
EZ
2408@var{port} is the name of the serial port to which to connect. For
2409example, this could be @file{/dev/ttyS0} on Unix. On Windows, this
2410could be @file{COM1}, or @file{\\.\COM10} (double the backslashes in
2411Lisp strings).
c73e02fa
GM
2412
2413@var{speed} is the speed of the serial port in bits per second. 9600
80e6b6df
EZ
2414is a common value. The buffer is in Term mode; see @ref{Term Mode,,,
2415emacs, The GNU Emacs Manual}, for the commands to use in that buffer.
2416You can change the speed and the configuration in the mode line menu.
2417@end deffn
c73e02fa
GM
2418
2419@defun make-serial-process &rest args
80e6b6df
EZ
2420This function creates a process and a buffer. Arguments are specified
2421as keyword/argument pairs. Here's the list of the meaningful keywords:
c73e02fa
GM
2422
2423@table @code
80e6b6df
EZ
2424@item :port @var{port}@r{ (mandatory)}
2425This is the name of the serial port. On Unix and GNU systems, this is
2426a file name such as @file{/dev/ttyS0}. On Windows, this could be
2427@file{COM1}, or @file{\\.\COM10} for ports higher than @file{COM9}
2428(double the backslashes in Lisp strings).
2429
2430@item :speed @var{speed}@r{ (mandatory)}
2431The speed of the serial port in bits per second. This function calls
2432@code{serial-process-configure} to handle the speed.
2433
2434@item :name @var{name}
2435The name of the process. If @var{name} is not given, @var{port} will
2436serve as the process name as well.
2437
2438@item :buffer @var{buffer}
2439The buffer to associate with the process. The value could be either a
2440buffer or a string that names a buffer. Process output goes at the
2441end of that buffer, unless you specify an output stream or filter
2442function to handle the output. If @var{buffer} is not given, the
2443process buffer's name is taken from the value of the @code{:name}
2444keyword.
2445
2446@item :coding @var{coding}
c73e02fa
GM
2447If @var{coding} is a symbol, it specifies the coding system used for
2448both reading and writing for this process. If @var{coding} is a cons
2449@code{(decoding . encoding)}, @var{decoding} is used for reading, and
80e6b6df
EZ
2450@var{encoding} is used for writing. If not specified, the default is
2451to determine the coding systems from data itself.
c73e02fa 2452
80e6b6df
EZ
2453@item :noquery @var{query-flag}
2454Initialize the process query flag to @var{query-flag}. @xref{Query
2455Before Exit}. The flags defaults to @code{nil} if unspecified.
c73e02fa 2456
80e6b6df 2457@item :stop @var{bool}
c73e02fa
GM
2458Start process in the @code{stopped} state if @var{bool} is
2459non-@code{nil}. In the stopped state, a serial process does not
2460accept incoming data, but you can send outgoing data. The stopped
2461state is cleared by @code{continue-process} and set by
2462@code{stop-process}.
2463
80e6b6df 2464@item :filter @var{filter}
c73e02fa
GM
2465Install @var{filter} as the process filter.
2466
80e6b6df 2467@item :sentinel @var{sentinel}
c73e02fa
GM
2468Install @var{sentinel} as the process sentinel.
2469
80e6b6df 2470@item :plist @var{plist}
c73e02fa
GM
2471Install @var{plist} as the initial plist of the process.
2472
2473@item :speed
2474@itemx :bytesize
2475@itemx :parity
2476@itemx :stopbits
2477@itemx :flowcontrol
2478These arguments are handled by @code{serial-process-configure}, which
2479is called by @code{make-serial-process}.
2480@end table
2481
2482The original argument list, possibly modified by later configuration,
2483is available via the function @code{process-contact}.
2484
2485Examples:
2486
2487@example
2488(make-serial-process :port "/dev/ttyS0" :speed 9600)
2489
2490(make-serial-process :port "COM1" :speed 115200 :stopbits 2)
2491
80e6b6df
EZ
2492(make-serial-process :port "\\\\.\\COM13" :speed 1200
2493 :bytesize 7 :parity 'odd)
c73e02fa 2494
57eedec1
GM
2495(make-serial-process :port "/dev/tty.BlueConsole-SPP-1"
2496 :speed nil)
c73e02fa
GM
2497@end example
2498@end defun
2499
2500@defun serial-process-configure &rest args
80e6b6df
EZ
2501@cindex baud, in serial connections
2502@cindex bytesize, in serial connections
2503@cindex parity, in serial connections
2504@cindex stopbits, in serial connections
2505@cindex flowcontrol, in serial connections
2506
2507This functions configures a serial port connection. Arguments are
2508specified as keyword/argument pairs. Attributes that are not given
2509are re-initialized from the process's current configuration (available
2510via the function @code{process-contact}) or set to reasonable default
2511values. The following arguments are defined:
c73e02fa
GM
2512
2513@table @code
80e6b6df
EZ
2514@item :process @var{process}
2515@itemx :name @var{name}
2516@itemx :buffer @var{buffer}
2517@itemx :port @var{port}
c73e02fa
GM
2518Any of these arguments can be given to identify the process that is to
2519be configured. If none of these arguments is given, the current
2520buffer's process is used.
2521
2522@item :speed @var{speed}
545c2782
EZ
2523The speed of the serial port in bits per second, a.k.a.@: @dfn{baud
2524rate}. The value can be any number, but most serial ports work only
2525at a few defined values between 1200 and 115200, with 9600 being the
2526most common value. If @var{speed} is @code{nil}, the function ignores
2527all other arguments and does not configure the port. This may be
2528useful for special serial ports such as Bluetooth-to-serial converters
2529which can only be configured through AT commands sent through the
538395d9
EZ
2530connection. The value of @code{nil} for @var{speed} is valid only for
2531connections that were already opened by a previous call to
80e6b6df 2532@code{make-serial-process} or @code{serial-term}.
c73e02fa
GM
2533
2534@item :bytesize @var{bytesize}
80e6b6df
EZ
2535The number of bits per byte, which can be 7 or 8. If @var{bytesize}
2536is not given or @code{nil}, it defaults to 8.
c73e02fa
GM
2537
2538@item :parity @var{parity}
80e6b6df 2539The value can be @code{nil} (don't use parity), the symbol
c73e02fa 2540@code{odd} (use odd parity), or the symbol @code{even} (use even
80e6b6df 2541parity). If @var{parity} is not given, it defaults to no parity.
c73e02fa
GM
2542
2543@item :stopbits @var{stopbits}
80e6b6df
EZ
2544The number of stopbits used to terminate a transmission
2545of each byte. @var{stopbits} can be 1 or 2. If @var{stopbits} is not
2546given or @code{nil}, it defaults to 1.
c73e02fa
GM
2547
2548@item :flowcontrol @var{flowcontrol}
80e6b6df
EZ
2549The type of flow control to use for this connection, which is either
2550@code{nil} (don't use flow control), the symbol @code{hw} (use RTS/CTS
2551hardware flow control), or the symbol @code{sw} (use XON/XOFF software
2552flow control). If @var{flowcontrol} is not given, it defaults to no
2553flow control.
c73e02fa
GM
2554@end table
2555
2556@code{serial-process-configure} is called by @code{make-serial-process} for the
2557initial configuration of the serial port.
2558
2559Examples:
2560
2561@example
2562(serial-process-configure :process "/dev/ttyS0" :speed 1200)
2563
57eedec1
GM
2564(serial-process-configure :buffer "COM1" :stopbits 1
2565 :parity 'odd :flowcontrol 'hw)
c73e02fa
GM
2566
2567(serial-process-configure :port "\\\\.\\COM13" :bytesize 7)
2568@end example
2569@end defun
2570
b8d4c8d0
GM
2571@node Byte Packing
2572@section Packing and Unpacking Byte Arrays
2573@cindex byte packing and unpacking
2574
2575 This section describes how to pack and unpack arrays of bytes,
2576usually for binary network protocols. These functions convert byte arrays
2577to alists, and vice versa. The byte array can be represented as a
2578unibyte string or as a vector of integers, while the alist associates
2579symbols either with fixed-size objects or with recursive sub-alists.
2580
2581@cindex serializing
2582@cindex deserializing
2583@cindex packing
2584@cindex unpacking
2585 Conversion from byte arrays to nested alists is also known as
2586@dfn{deserializing} or @dfn{unpacking}, while going in the opposite
2587direction is also known as @dfn{serializing} or @dfn{packing}.
2588
2589@menu
2590* Bindat Spec:: Describing data layout.
2591* Bindat Functions:: Doing the unpacking and packing.
2592* Bindat Examples:: Samples of what bindat.el can do for you!
2593@end menu
2594
2595@node Bindat Spec
2596@subsection Describing Data Layout
2597
2598 To control unpacking and packing, you write a @dfn{data layout
2599specification}, a special nested list describing named and typed
2600@dfn{fields}. This specification controls length of each field to be
2601processed, and how to pack or unpack it. We normally keep bindat specs
2602in variables whose names end in @samp{-bindat-spec}; that kind of name
2603is automatically recognized as ``risky.''
2604
2605@cindex endianness
2606@cindex big endian
2607@cindex little endian
2608@cindex network byte ordering
2609 A field's @dfn{type} describes the size (in bytes) of the object
2610that the field represents and, in the case of multibyte fields, how
2611the bytes are ordered within the field. The two possible orderings
2612are ``big endian'' (also known as ``network byte ordering'') and
2613``little endian.'' For instance, the number @code{#x23cd} (decimal
26149165) in big endian would be the two bytes @code{#x23} @code{#xcd};
2615and in little endian, @code{#xcd} @code{#x23}. Here are the possible
2616type values:
2617
2618@table @code
2619@item u8
2620@itemx byte
2621Unsigned byte, with length 1.
2622
2623@item u16
2624@itemx word
2625@itemx short
2626Unsigned integer in network byte order, with length 2.
2627
2628@item u24
2629Unsigned integer in network byte order, with length 3.
2630
2631@item u32
2632@itemx dword
2633@itemx long
2634Unsigned integer in network byte order, with length 4.
2635Note: These values may be limited by Emacs' integer implementation limits.
2636
2637@item u16r
2638@itemx u24r
2639@itemx u32r
2640Unsigned integer in little endian order, with length 2, 3 and 4, respectively.
2641
2642@item str @var{len}
2643String of length @var{len}.
2644
2645@item strz @var{len}
2646Zero-terminated string, in a fixed-size field with length @var{len}.
2647
2648@item vec @var{len} [@var{type}]
2649Vector of @var{len} elements of type @var{type}, or bytes if not
2650@var{type} is specified.
2651The @var{type} is any of the simple types above, or another vector
2652specified as a list @code{(vec @var{len} [@var{type}])}.
2653
2654@item ip
2655Four-byte vector representing an Internet address. For example:
2656@code{[127 0 0 1]} for localhost.
2657
2658@item bits @var{len}
2659List of set bits in @var{len} bytes. The bytes are taken in big
2660endian order and the bits are numbered starting with @code{8 *
2661@var{len} @minus{} 1} and ending with zero. For example: @code{bits
26622} unpacks @code{#x28} @code{#x1c} to @code{(2 3 4 11 13)} and
2663@code{#x1c} @code{#x28} to @code{(3 5 10 11 12)}.
2664
2665@item (eval @var{form})
2666@var{form} is a Lisp expression evaluated at the moment the field is
2667unpacked or packed. The result of the evaluation should be one of the
2668above-listed type specifications.
2669@end table
2670
2671For a fixed-size field, the length @var{len} is given as an integer
2672specifying the number of bytes in the field.
2673
2674When the length of a field is not fixed, it typically depends on the
2675value of a preceding field. In this case, the length @var{len} can be
2676given either as a list @code{(@var{name} ...)} identifying a
2677@dfn{field name} in the format specified for @code{bindat-get-field}
2678below, or by an expression @code{(eval @var{form})} where @var{form}
2679should evaluate to an integer, specifying the field length.
2680
2681A field specification generally has the form @code{([@var{name}]
2682@var{handler})}. The square braces indicate that @var{name} is
2683optional. (Don't use names that are symbols meaningful as type
2684specifications (above) or handler specifications (below), since that
2685would be ambiguous.) @var{name} can be a symbol or the expression
2686@code{(eval @var{form})}, in which case @var{form} should evaluate to
2687a symbol.
2688
2689@var{handler} describes how to unpack or pack the field and can be one
2690of the following:
2691
2692@table @code
2693@item @var{type}
2694Unpack/pack this field according to the type specification @var{type}.
2695
2696@item eval @var{form}
2697Evaluate @var{form}, a Lisp expression, for side-effect only. If the
2698field name is specified, the value is bound to that field name.
2699
2700@item fill @var{len}
2701Skip @var{len} bytes. In packing, this leaves them unchanged,
2702which normally means they remain zero. In unpacking, this means
2703they are ignored.
2704
2705@item align @var{len}
2706Skip to the next multiple of @var{len} bytes.
2707
2708@item struct @var{spec-name}
2709Process @var{spec-name} as a sub-specification. This describes a
2710structure nested within another structure.
2711
2712@item union @var{form} (@var{tag} @var{spec})@dots{}
2713@c ??? I don't see how one would actually use this.
2714@c ??? what kind of expression would be useful for @var{form}?
2715Evaluate @var{form}, a Lisp expression, find the first @var{tag}
2716that matches it, and process its associated data layout specification
2717@var{spec}. Matching can occur in one of three ways:
2718
2719@itemize
2720@item
2721If a @var{tag} has the form @code{(eval @var{expr})}, evaluate
2722@var{expr} with the variable @code{tag} dynamically bound to the value
2723of @var{form}. A non-@code{nil} result indicates a match.
2724
2725@item
2726@var{tag} matches if it is @code{equal} to the value of @var{form}.
2727
2728@item
2729@var{tag} matches unconditionally if it is @code{t}.
2730@end itemize
2731
2732@item repeat @var{count} @var{field-specs}@dots{}
2733Process the @var{field-specs} recursively, in order, then repeat
2734starting from the first one, processing all the specs @var{count}
2735times overall. The @var{count} is given using the same formats as a
2736field length---if an @code{eval} form is used, it is evaluated just once.
2737For correct operation, each spec in @var{field-specs} must include a name.
2738@end table
2739
2740For the @code{(eval @var{form})} forms used in a bindat specification,
2741the @var{form} can access and update these dynamically bound variables
2742during evaluation:
2743
2744@table @code
2745@item last
2746Value of the last field processed.
2747
2748@item bindat-raw
2749The data as a byte array.
2750
2751@item bindat-idx
2752Current index (within @code{bindat-raw}) for unpacking or packing.
2753
2754@item struct
2755The alist containing the structured data that have been unpacked so
2756far, or the entire structure being packed. You can use
2757@code{bindat-get-field} to access specific fields of this structure.
2758
2759@item count
2760@itemx index
2761Inside a @code{repeat} block, these contain the maximum number of
2762repetitions (as specified by the @var{count} parameter), and the
2763current repetition number (counting from 0). Setting @code{count} to
2764zero will terminate the inner-most repeat block after the current
2765repetition has completed.
2766@end table
2767
2768@node Bindat Functions
2769@subsection Functions to Unpack and Pack Bytes
2770
2771 In the following documentation, @var{spec} refers to a data layout
2772specification, @code{bindat-raw} to a byte array, and @var{struct} to an
2773alist representing unpacked field data.
2774
2775@defun bindat-unpack spec bindat-raw &optional bindat-idx
2776This function unpacks data from the unibyte string or byte
2777array @code{bindat-raw}
2778according to @var{spec}. Normally this starts unpacking at the
2779beginning of the byte array, but if @var{bindat-idx} is non-@code{nil}, it
2780specifies a zero-based starting position to use instead.
2781
2782The value is an alist or nested alist in which each element describes
2783one unpacked field.
2784@end defun
2785
2786@defun bindat-get-field struct &rest name
2787This function selects a field's data from the nested alist
2788@var{struct}. Usually @var{struct} was returned by
2789@code{bindat-unpack}. If @var{name} corresponds to just one argument,
2790that means to extract a top-level field value. Multiple @var{name}
2791arguments specify repeated lookup of sub-structures. An integer name
2792acts as an array index.
2793
2794For example, if @var{name} is @code{(a b 2 c)}, that means to find
2795field @code{c} in the third element of subfield @code{b} of field
2796@code{a}. (This corresponds to @code{struct.a.b[2].c} in C.)
2797@end defun
2798
2799 Although packing and unpacking operations change the organization of
2800data (in memory), they preserve the data's @dfn{total length}, which is
2801the sum of all the fields' lengths, in bytes. This value is not
2802generally inherent in either the specification or alist alone; instead,
2803both pieces of information contribute to its calculation. Likewise, the
2804length of a string or array being unpacked may be longer than the data's
2805total length as described by the specification.
2806
2807@defun bindat-length spec struct
2808This function returns the total length of the data in @var{struct},
2809according to @var{spec}.
2810@end defun
2811
2812@defun bindat-pack spec struct &optional bindat-raw bindat-idx
2813This function returns a byte array packed according to @var{spec} from
2814the data in the alist @var{struct}. Normally it creates and fills a
2815new byte array starting at the beginning. However, if @var{bindat-raw}
2816is non-@code{nil}, it specifies a pre-allocated unibyte string or vector to
2817pack into. If @var{bindat-idx} is non-@code{nil}, it specifies the starting
2818offset for packing into @code{bindat-raw}.
2819
2820When pre-allocating, you should make sure @code{(length @var{bindat-raw})}
2821meets or exceeds the total length to avoid an out-of-range error.
2822@end defun
2823
2824@defun bindat-ip-to-string ip
2825Convert the Internet address vector @var{ip} to a string in the usual
2826dotted notation.
2827
2828@example
2829(bindat-ip-to-string [127 0 0 1])
2830 @result{} "127.0.0.1"
2831@end example
2832@end defun
2833
2834@node Bindat Examples
2835@subsection Examples of Byte Unpacking and Packing
2836
2837 Here is a complete example of byte unpacking and packing:
2838
2839@lisp
2840(defvar fcookie-index-spec
2841 '((:version u32)
2842 (:count u32)
2843 (:longest u32)
2844 (:shortest u32)
2845 (:flags u32)
2846 (:delim u8)
2847 (:ignored fill 3)
2848 (:offset repeat (:count)
2849 (:foo u32)))
2850 "Description of a fortune cookie index file's contents.")
2851
2852(defun fcookie (cookies &optional index)
2853 "Display a random fortune cookie from file COOKIES.
2854Optional second arg INDEX specifies the associated index
2855filename, which is by default constructed by appending
2856\".dat\" to COOKIES. Display cookie text in possibly
2857new buffer \"*Fortune Cookie: BASENAME*\" where BASENAME
2858is COOKIES without the directory part."
2859 (interactive "fCookies file: ")
2860 (let* ((info (with-temp-buffer
2861 (insert-file-contents-literally
2862 (or index (concat cookies ".dat")))
2863 (bindat-unpack fcookie-index-spec
2864 (buffer-string))))
2865 (sel (random (bindat-get-field info :count)))
2866 (beg (cdar (bindat-get-field info :offset sel)))
2867 (end (or (cdar (bindat-get-field info
2868 :offset (1+ sel)))
2869 (nth 7 (file-attributes cookies)))))
2870 (switch-to-buffer
2871 (get-buffer-create
2872 (format "*Fortune Cookie: %s*"
2873 (file-name-nondirectory cookies))))
2874 (erase-buffer)
2875 (insert-file-contents-literally
2876 cookies nil beg (- end 3))))
2877
2878(defun fcookie-create-index (cookies &optional index delim)
2879 "Scan file COOKIES, and write out its index file.
2880Optional second arg INDEX specifies the index filename,
2881which is by default constructed by appending \".dat\" to
2882COOKIES. Optional third arg DELIM specifies the unibyte
2883character which, when found on a line of its own in
2884COOKIES, indicates the border between entries."
2885 (interactive "fCookies file: ")
2886 (setq delim (or delim ?%))
2887 (let ((delim-line (format "\n%c\n" delim))
2888 (count 0)
2889 (max 0)
2890 min p q len offsets)
2891 (unless (= 3 (string-bytes delim-line))
2892 (error "Delimiter cannot be represented in one byte"))
2893 (with-temp-buffer
2894 (insert-file-contents-literally cookies)
2895 (while (and (setq p (point))
2896 (search-forward delim-line (point-max) t)
2897 (setq len (- (point) 3 p)))
2898 (setq count (1+ count)
2899 max (max max len)
2900 min (min (or min max) len)
2901 offsets (cons (1- p) offsets))))
2902 (with-temp-buffer
2903 (set-buffer-multibyte nil)
2904 (insert
2905 (bindat-pack
2906 fcookie-index-spec
2907 `((:version . 2)
2908 (:count . ,count)
2909 (:longest . ,max)
2910 (:shortest . ,min)
2911 (:flags . 0)
2912 (:delim . ,delim)
2913 (:offset . ,(mapcar (lambda (o)
2914 (list (cons :foo o)))
2915 (nreverse offsets))))))
2916 (let ((coding-system-for-write 'raw-text-unix))
2917 (write-file (or index (concat cookies ".dat")))))))
2918@end lisp
2919
2920Following is an example of defining and unpacking a complex structure.
2921Consider the following C structures:
2922
2923@example
2924struct header @{
2925 unsigned long dest_ip;
2926 unsigned long src_ip;
2927 unsigned short dest_port;
2928 unsigned short src_port;
2929@};
2930
2931struct data @{
2932 unsigned char type;
2933 unsigned char opcode;
2934 unsigned short length; /* In network byte order */
2935 unsigned char id[8]; /* null-terminated string */
2936 unsigned char data[/* (length + 3) & ~3 */];
2937@};
2938
2939struct packet @{
2940 struct header header;
2941 unsigned long counters[2]; /* In little endian order */
2942 unsigned char items;
2943 unsigned char filler[3];
2944 struct data item[/* items */];
2945
2946@};
2947@end example
2948
2949The corresponding data layout specification:
2950
2951@lisp
2952(setq header-spec
2953 '((dest-ip ip)
2954 (src-ip ip)
2955 (dest-port u16)
2956 (src-port u16)))
2957
2958(setq data-spec
2959 '((type u8)
2960 (opcode u8)
2961 (length u16) ;; network byte order
2962 (id strz 8)
2963 (data vec (length))
2964 (align 4)))
2965
2966(setq packet-spec
2967 '((header struct header-spec)
2968 (counters vec 2 u32r) ;; little endian order
2969 (items u8)
2970 (fill 3)
2971 (item repeat (items)
2972 (struct data-spec))))
2973@end lisp
2974
2975A binary data representation:
2976
2977@lisp
2978(setq binary-data
2979 [ 192 168 1 100 192 168 1 101 01 28 21 32
2980 160 134 1 0 5 1 0 0 2 0 0 0
2981 2 3 0 5 ?A ?B ?C ?D ?E ?F 0 0 1 2 3 4 5 0 0 0
2982 1 4 0 7 ?B ?C ?D ?E ?F ?G 0 0 6 7 8 9 10 11 12 0 ])
2983@end lisp
2984
2985The corresponding decoded structure:
2986
2987@lisp
2988(setq decoded (bindat-unpack packet-spec binary-data))
2989 @result{}
2990((header
2991 (dest-ip . [192 168 1 100])
2992 (src-ip . [192 168 1 101])
2993 (dest-port . 284)
2994 (src-port . 5408))
2995 (counters . [100000 261])
2996 (items . 2)
2997 (item ((data . [1 2 3 4 5])
2998 (id . "ABCDEF")
2999 (length . 5)
3000 (opcode . 3)
3001 (type . 2))
3002 ((data . [6 7 8 9 10 11 12])
3003 (id . "BCDEFG")
3004 (length . 7)
3005 (opcode . 4)
3006 (type . 1))))
3007@end lisp
3008
3009Fetching data from this structure:
3010
3011@lisp
3012(bindat-get-field decoded 'item 1 'id)
3013 @result{} "BCDEFG"
3014@end lisp
3015
3016@ignore
3017 arch-tag: ba9da253-e65f-4e7f-b727-08fba0a1df7a
3018@end ignore