mail/sendmail.el (mail-encode-header): Avoid double encoding by let-binding rfc2047...
[bpt/emacs.git] / doc / lispref / processes.texi
CommitLineData
b8d4c8d0
GM
1@c -*-texinfo-*-
2@c This is part of the GNU Emacs Lisp Reference Manual.
3@c Copyright (C) 1990, 1991, 1992, 1993, 1994, 1995, 1998, 1999, 2001,
5df4f04c 4@c 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011
d24880de 5@c Free Software Foundation, Inc.
b8d4c8d0 6@c See the file elisp.texi for copying conditions.
6336d8c3 7@setfilename ../../info/processes
b8d4c8d0
GM
8@node Processes, Display, Abbrevs, Top
9@chapter Processes
10@cindex child process
11@cindex parent process
12@cindex subprocess
13@cindex process
14
15 In the terminology of operating systems, a @dfn{process} is a space in
16which a program can execute. Emacs runs in a process. Emacs Lisp
17programs can invoke other programs in processes of their own. These are
18called @dfn{subprocesses} or @dfn{child processes} of the Emacs process,
19which is their @dfn{parent process}.
20
21 A subprocess of Emacs may be @dfn{synchronous} or @dfn{asynchronous},
22depending on how it is created. When you create a synchronous
23subprocess, the Lisp program waits for the subprocess to terminate
24before continuing execution. When you create an asynchronous
25subprocess, it can run in parallel with the Lisp program. This kind of
26subprocess is represented within Emacs by a Lisp object which is also
27called a ``process.'' Lisp programs can use this object to communicate
28with the subprocess or to control it. For example, you can send
29signals, obtain status information, receive output from the process, or
30send input to it.
31
32@defun processp object
23dd4ecd
EZ
33This function returns @code{t} if @var{object} represents an Emacs
34subprocess, @code{nil} otherwise.
b8d4c8d0
GM
35@end defun
36
23dd4ecd
EZ
37 In addition to subprocesses of the current Emacs session, you can
38also access other processes running on your machine. @xref{System
39Processes}.
40
b8d4c8d0
GM
41@menu
42* Subprocess Creation:: Functions that start subprocesses.
43* Shell Arguments:: Quoting an argument to pass it to a shell.
44* Synchronous Processes:: Details of using synchronous subprocesses.
45* Asynchronous Processes:: Starting up an asynchronous subprocess.
46* Deleting Processes:: Eliminating an asynchronous subprocess.
47* Process Information:: Accessing run-status and other attributes.
48* Input to Processes:: Sending input to an asynchronous subprocess.
49* Signals to Processes:: Stopping, continuing or interrupting
50 an asynchronous subprocess.
51* Output from Processes:: Collecting output from an asynchronous subprocess.
52* Sentinels:: Sentinels run when process run-status changes.
53* Query Before Exit:: Whether to query if exiting will kill a process.
23dd4ecd 54* System Processes:: Accessing other processes running on your system.
d24880de 55* Transaction Queues:: Transaction-based communication with subprocesses.
b8d4c8d0
GM
56* Network:: Opening network connections.
57* Network Servers:: Network servers let Emacs accept net connections.
58* Datagrams:: UDP network connections.
59* Low-Level Network:: Lower-level but more general function
60 to create connections and servers.
d24880de 61* Misc Network:: Additional relevant functions for net connections.
c73e02fa 62* Serial Ports:: Communicating with serial ports.
b8d4c8d0
GM
63* Byte Packing:: Using bindat to pack and unpack binary data.
64@end menu
65
66@node Subprocess Creation
67@section Functions that Create Subprocesses
68
583d8b3c 69 There are three primitives that create a new subprocess in which to run
b8d4c8d0
GM
70a program. One of them, @code{start-process}, creates an asynchronous
71process and returns a process object (@pxref{Asynchronous Processes}).
72The other two, @code{call-process} and @code{call-process-region},
73create a synchronous process and do not return a process object
74(@pxref{Synchronous Processes}).
75
76 Synchronous and asynchronous processes are explained in the following
77sections. Since the three functions are all called in a similar
78fashion, their common arguments are described here.
79
80@cindex execute program
81@cindex @code{PATH} environment variable
82@cindex @code{HOME} environment variable
83 In all cases, the function's @var{program} argument specifies the
84program to be run. An error is signaled if the file is not found or
85cannot be executed. If the file name is relative, the variable
86@code{exec-path} contains a list of directories to search. Emacs
87initializes @code{exec-path} when it starts up, based on the value of
88the environment variable @code{PATH}. The standard file name
89constructs, @samp{~}, @samp{.}, and @samp{..}, are interpreted as
90usual in @code{exec-path}, but environment variable substitutions
91(@samp{$HOME}, etc.) are not recognized; use
92@code{substitute-in-file-name} to perform them (@pxref{File Name
93Expansion}). @code{nil} in this list refers to
94@code{default-directory}.
95
96 Executing a program can also try adding suffixes to the specified
97name:
98
99@defvar exec-suffixes
100This variable is a list of suffixes (strings) to try adding to the
101specified program file name. The list should include @code{""} if you
102want the name to be tried exactly as specified. The default value is
103system-dependent.
104@end defvar
105
106 @strong{Please note:} The argument @var{program} contains only the
107name of the program; it may not contain any command-line arguments. You
108must use @var{args} to provide those.
109
110 Each of the subprocess-creating functions has a @var{buffer-or-name}
111argument which specifies where the standard output from the program will
112go. It should be a buffer or a buffer name; if it is a buffer name,
113that will create the buffer if it does not already exist. It can also
114be @code{nil}, which says to discard the output unless a filter function
115handles it. (@xref{Filter Functions}, and @ref{Read and Print}.)
116Normally, you should avoid having multiple processes send output to the
117same buffer because their output would be intermixed randomly.
118
119@cindex program arguments
120 All three of the subprocess-creating functions have a @code{&rest}
121argument, @var{args}. The @var{args} must all be strings, and they are
122supplied to @var{program} as separate command line arguments. Wildcard
123characters and other shell constructs have no special meanings in these
124strings, since the strings are passed directly to the specified program.
125
126 The subprocess gets its current directory from the value of
127@code{default-directory} (@pxref{File Name Expansion}).
128
129@cindex environment variables, subprocesses
130 The subprocess inherits its environment from Emacs, but you can
131specify overrides for it with @code{process-environment}. @xref{System
132Environment}.
133
134@defvar exec-directory
135@pindex movemail
136The value of this variable is a string, the name of a directory that
137contains programs that come with GNU Emacs, programs intended for Emacs
138to invoke. The program @code{movemail} is an example of such a program;
139Rmail uses it to fetch new mail from an inbox.
140@end defvar
141
142@defopt exec-path
143The value of this variable is a list of directories to search for
144programs to run in subprocesses. Each element is either the name of a
145directory (i.e., a string), or @code{nil}, which stands for the default
146directory (which is the value of @code{default-directory}).
147@cindex program directories
148
149The value of @code{exec-path} is used by @code{call-process} and
150@code{start-process} when the @var{program} argument is not an absolute
151file name.
152@end defopt
153
154@node Shell Arguments
155@section Shell Arguments
156@cindex arguments for shell commands
157@cindex shell command arguments
158
159 Lisp programs sometimes need to run a shell and give it a command
160that contains file names that were specified by the user. These
161programs ought to be able to support any valid file name. But the shell
162gives special treatment to certain characters, and if these characters
163occur in the file name, they will confuse the shell. To handle these
164characters, use the function @code{shell-quote-argument}:
165
166@defun shell-quote-argument argument
167This function returns a string which represents, in shell syntax,
168an argument whose actual contents are @var{argument}. It should
169work reliably to concatenate the return value into a shell command
170and then pass it to a shell for execution.
171
172Precisely what this function does depends on your operating system. The
173function is designed to work with the syntax of your system's standard
174shell; if you use an unusual shell, you will need to redefine this
175function.
176
177@example
178;; @r{This example shows the behavior on GNU and Unix systems.}
179(shell-quote-argument "foo > bar")
180 @result{} "foo\\ \\>\\ bar"
181
182;; @r{This example shows the behavior on MS-DOS and MS-Windows.}
183(shell-quote-argument "foo > bar")
184 @result{} "\"foo > bar\""
185@end example
186
187Here's an example of using @code{shell-quote-argument} to construct
188a shell command:
189
190@example
191(concat "diff -c "
192 (shell-quote-argument oldfile)
193 " "
194 (shell-quote-argument newfile))
195@end example
196@end defun
197
3c73e30e
EZ
198@cindex quoting and unquoting command-line arguments
199@cindex minibuffer input, and command-line arguments
200@cindex @code{call-process}, command-line arguments from minibuffer
201@cindex @code{start-process}, command-line arguments from minibuffer
4bb49a92
EZ
202 The following two functions are useful for combining a list of
203individual command-line argument strings into a single string, and
204taking a string apart into a list of individual command-line
205arguments. These functions are mainly intended to be used for
206converting user input in the minibuffer, a Lisp string, into a list of
207string arguments to be passed to @code{call-process} or
3c73e30e
EZ
208@code{start-process}, or for the converting such lists of arguments in
209a single Lisp string to be presented in the minibuffer or echo area.
a873ee3d
EZ
210
211@defun split-string-and-unquote string &optional separators
212This function splits @var{string} into substrings at matches for the
213regular expression @var{separators}, like @code{split-string} does
0999039c
CY
214(@pxref{Creating Strings}); in addition, it removes quoting from the
215substrings. It then makes a list of the substrings and returns it.
a873ee3d 216
4185820c
CY
217If @var{separators} is omitted or @code{nil}, it defaults to
218@code{"\\s-+"}, which is a regular expression that matches one or more
219characters with whitespace syntax (@pxref{Syntax Class Table}).
a873ee3d 220
3c73e30e 221This function supports two types of quoting: enclosing a whole string
0999039c
CY
222in double quotes @code{"@dots{}"}, and quoting individual characters
223with a backslash escape @samp{\}. The latter is also used in Lisp
224strings, so this function can handle those as well.
a873ee3d
EZ
225@end defun
226
227@defun combine-and-quote-strings list-of-strings &optional separator
228This function concatenates @var{list-of-strings} into a single string,
0999039c
CY
229quoting each string as necessary. It also sticks the @var{separator}
230string between each pair of strings; if @var{separator} is omitted or
231@code{nil}, it defaults to @code{" "}. The return value is the
232resulting string.
a873ee3d
EZ
233
234The strings in @var{list-of-strings} that need quoting are those that
235include @var{separator} as their substring. Quoting a string encloses
236it in double quotes @code{"@dots{}"}. In the simplest case, if you
4bb49a92
EZ
237are consing a command from the individual command-line arguments,
238every argument that includes embedded blanks will be quoted.
a873ee3d
EZ
239@end defun
240
b8d4c8d0
GM
241@node Synchronous Processes
242@section Creating a Synchronous Process
243@cindex synchronous subprocess
244
245 After a @dfn{synchronous process} is created, Emacs waits for the
246process to terminate before continuing. Starting Dired on GNU or
247Unix@footnote{On other systems, Emacs uses a Lisp emulation of
248@code{ls}; see @ref{Contents of Directories}.} is an example of this: it
249runs @code{ls} in a synchronous process, then modifies the output
250slightly. Because the process is synchronous, the entire directory
251listing arrives in the buffer before Emacs tries to do anything with it.
252
253 While Emacs waits for the synchronous subprocess to terminate, the
254user can quit by typing @kbd{C-g}. The first @kbd{C-g} tries to kill
255the subprocess with a @code{SIGINT} signal; but it waits until the
256subprocess actually terminates before quitting. If during that time the
257user types another @kbd{C-g}, that kills the subprocess instantly with
258@code{SIGKILL} and quits immediately (except on MS-DOS, where killing
259other processes doesn't work). @xref{Quitting}.
260
261 The synchronous subprocess functions return an indication of how the
262process terminated.
263
264 The output from a synchronous subprocess is generally decoded using a
265coding system, much like text read from a file. The input sent to a
266subprocess by @code{call-process-region} is encoded using a coding
267system, much like text written into a file. @xref{Coding Systems}.
268
269@defun call-process program &optional infile destination display &rest args
270This function calls @var{program} in a separate process and waits for
271it to finish.
272
273The standard input for the process comes from file @var{infile} if
274@var{infile} is not @code{nil}, and from the null device otherwise.
275The argument @var{destination} says where to put the process output.
276Here are the possibilities:
277
278@table @asis
279@item a buffer
280Insert the output in that buffer, before point. This includes both the
281standard output stream and the standard error stream of the process.
282
283@item a string
284Insert the output in a buffer with that name, before point.
285
286@item @code{t}
287Insert the output in the current buffer, before point.
288
289@item @code{nil}
290Discard the output.
291
292@item 0
293Discard the output, and return @code{nil} immediately without waiting
294for the subprocess to finish.
295
296In this case, the process is not truly synchronous, since it can run in
297parallel with Emacs; but you can think of it as synchronous in that
298Emacs is essentially finished with the subprocess as soon as this
299function returns.
300
301MS-DOS doesn't support asynchronous subprocesses, so this option doesn't
302work there.
303
304@item @code{(@var{real-destination} @var{error-destination})}
305Keep the standard output stream separate from the standard error stream;
306deal with the ordinary output as specified by @var{real-destination},
307and dispose of the error output according to @var{error-destination}.
308If @var{error-destination} is @code{nil}, that means to discard the
309error output, @code{t} means mix it with the ordinary output, and a
310string specifies a file name to redirect error output into.
311
312You can't directly specify a buffer to put the error output in; that is
313too difficult to implement. But you can achieve this result by sending
314the error output to a temporary file and then inserting the file into a
315buffer.
316@end table
317
318If @var{display} is non-@code{nil}, then @code{call-process} redisplays
319the buffer as output is inserted. (However, if the coding system chosen
320for decoding output is @code{undecided}, meaning deduce the encoding
321from the actual data, then redisplay sometimes cannot continue once
322non-@acronym{ASCII} characters are encountered. There are fundamental
323reasons why it is hard to fix this; see @ref{Output from Processes}.)
324
325Otherwise the function @code{call-process} does no redisplay, and the
326results become visible on the screen only when Emacs redisplays that
327buffer in the normal course of events.
328
329The remaining arguments, @var{args}, are strings that specify command
330line arguments for the program.
331
332The value returned by @code{call-process} (unless you told it not to
333wait) indicates the reason for process termination. A number gives the
334exit status of the subprocess; 0 means success, and any other value
335means failure. If the process terminated with a signal,
336@code{call-process} returns a string describing the signal.
337
338In the examples below, the buffer @samp{foo} is current.
339
340@smallexample
341@group
342(call-process "pwd" nil t)
343 @result{} 0
344
345---------- Buffer: foo ----------
346/usr/user/lewis/manual
347---------- Buffer: foo ----------
348@end group
349
350@group
351(call-process "grep" nil "bar" nil "lewis" "/etc/passwd")
352 @result{} 0
353
354---------- Buffer: bar ----------
355lewis:5LTsHm66CSWKg:398:21:Bil Lewis:/user/lewis:/bin/csh
356
357---------- Buffer: bar ----------
358@end group
359@end smallexample
360
361Here is a good example of the use of @code{call-process}, which used to
362be found in the definition of @code{insert-directory}:
363
364@smallexample
365@group
366(call-process insert-directory-program nil t nil @var{switches}
367 (if full-directory-p
368 (concat (file-name-as-directory file) ".")
369 file))
370@end group
371@end smallexample
372@end defun
373
374@defun process-file program &optional infile buffer display &rest args
375This function processes files synchronously in a separate process. It
376is similar to @code{call-process} but may invoke a file handler based
377on the value of the variable @code{default-directory}. The current
378working directory of the subprocess is @code{default-directory}.
379
380The arguments are handled in almost the same way as for
381@code{call-process}, with the following differences:
382
383Some file handlers may not support all combinations and forms of the
384arguments @var{infile}, @var{buffer}, and @var{display}. For example,
385some file handlers might behave as if @var{display} were @code{nil},
386regardless of the value actually passed. As another example, some
387file handlers might not support separating standard output and error
388output by way of the @var{buffer} argument.
389
390If a file handler is invoked, it determines the program to run based
391on the first argument @var{program}. For instance, consider that a
392handler for remote files is invoked. Then the path that is used for
393searching the program might be different than @code{exec-path}.
394
395The second argument @var{infile} may invoke a file handler. The file
396handler could be different from the handler chosen for the
397@code{process-file} function itself. (For example,
398@code{default-directory} could be on a remote host, whereas
399@var{infile} is on another remote host. Or @code{default-directory}
400could be non-special, whereas @var{infile} is on a remote host.)
401
402If @var{buffer} is a list of the form @code{(@var{real-destination}
403@var{error-destination})}, and @var{error-destination} names a file,
404then the same remarks as for @var{infile} apply.
405
406The remaining arguments (@var{args}) will be passed to the process
407verbatim. Emacs is not involved in processing file names that are
408present in @var{args}. To avoid confusion, it may be best to avoid
409absolute file names in @var{args}, but rather to specify all file
410names as relative to @code{default-directory}. The function
411@code{file-relative-name} is useful for constructing such relative
412file names.
413@end defun
414
3691a0a6
MA
415@defvar process-file-side-effects
416This variable indicates, whether a call of @code{process-file} changes
417remote files.
418
419Per default, this variable is always set to @code{t}, meaning that a
420call of @code{process-file} could potentially change any file on a
421remote host. When set to @code{nil}, a file handler could optimize
422its behaviour with respect to remote file attributes caching.
423
424This variable should never be changed by @code{setq}. Instead of, it
425shall be set only by let-binding.
426@end defvar
427
b8d4c8d0
GM
428@defun call-process-region start end program &optional delete destination display &rest args
429This function sends the text from @var{start} to @var{end} as
430standard input to a process running @var{program}. It deletes the text
431sent if @var{delete} is non-@code{nil}; this is useful when
432@var{destination} is @code{t}, to insert the output in the current
433buffer in place of the input.
434
435The arguments @var{destination} and @var{display} control what to do
436with the output from the subprocess, and whether to update the display
437as it comes in. For details, see the description of
438@code{call-process}, above. If @var{destination} is the integer 0,
439@code{call-process-region} discards the output and returns @code{nil}
440immediately, without waiting for the subprocess to finish (this only
441works if asynchronous subprocesses are supported).
442
443The remaining arguments, @var{args}, are strings that specify command
444line arguments for the program.
445
446The return value of @code{call-process-region} is just like that of
447@code{call-process}: @code{nil} if you told it to return without
448waiting; otherwise, a number or string which indicates how the
449subprocess terminated.
450
451In the following example, we use @code{call-process-region} to run the
452@code{cat} utility, with standard input being the first five characters
453in buffer @samp{foo} (the word @samp{input}). @code{cat} copies its
454standard input into its standard output. Since the argument
455@var{destination} is @code{t}, this output is inserted in the current
456buffer.
457
458@smallexample
459@group
460---------- Buffer: foo ----------
461input@point{}
462---------- Buffer: foo ----------
463@end group
464
465@group
466(call-process-region 1 6 "cat" nil t)
467 @result{} 0
468
469---------- Buffer: foo ----------
470inputinput@point{}
471---------- Buffer: foo ----------
472@end group
473@end smallexample
474
475 The @code{shell-command-on-region} command uses
476@code{call-process-region} like this:
477
478@smallexample
479@group
480(call-process-region
481 start end
482 shell-file-name ; @r{Name of program.}
483 nil ; @r{Do not delete region.}
484 buffer ; @r{Send output to @code{buffer}.}
485 nil ; @r{No redisplay during output.}
486 "-c" command) ; @r{Arguments for the shell.}
487@end group
488@end smallexample
489@end defun
490
491@defun call-process-shell-command command &optional infile destination display &rest args
492This function executes the shell command @var{command} synchronously
493in a separate process. The final arguments @var{args} are additional
494arguments to add at the end of @var{command}. The other arguments
495are handled as in @code{call-process}.
496@end defun
497
498@defun process-file-shell-command command &optional infile destination display &rest args
499This function is like @code{call-process-shell-command}, but uses
500@code{process-file} internally. Depending on @code{default-directory},
501@var{command} can be executed also on remote hosts.
502@end defun
503
504@defun shell-command-to-string command
505This function executes @var{command} (a string) as a shell command,
506then returns the command's output as a string.
507@end defun
508
583d8b3c
EZ
509@defun process-lines program &rest args
510This function runs @var{program} in a separate process, waits for it
511to finish, and returns its output as a list of strings. Each string
512in the list holds a single line of text output by the program; the
513end-of-line characters are stripped from each line. The arguments
514beyond @var{program}, @var{args}, are strings that specify
515command-line arguments with which to run the program.
516
517If @var{program} exits with a non-zero exit status, this function
518signals an error.
519
520This function works by calling @code{call-process}, so program output
521is decoded in the same way as for @code{call-process}.
522@end defun
523
b8d4c8d0
GM
524@node Asynchronous Processes
525@section Creating an Asynchronous Process
526@cindex asynchronous subprocess
527
528 After an @dfn{asynchronous process} is created, Emacs and the subprocess
529both continue running immediately. The process thereafter runs
530in parallel with Emacs, and the two can communicate with each other
531using the functions described in the following sections. However,
532communication is only partially asynchronous: Emacs sends data to the
533process only when certain functions are called, and Emacs accepts data
534from the process only when Emacs is waiting for input or for a time
535delay.
536
537 Here we describe how to create an asynchronous process.
538
539@defun start-process name buffer-or-name program &rest args
540This function creates a new asynchronous subprocess and starts the
541program @var{program} running in it. It returns a process object that
542stands for the new subprocess in Lisp. The argument @var{name}
543specifies the name for the process object; if a process with this name
544already exists, then @var{name} is modified (by appending @samp{<1>},
545etc.) to be unique. The buffer @var{buffer-or-name} is the buffer to
546associate with the process.
547
548The remaining arguments, @var{args}, are strings that specify command
549line arguments for the program.
550
551In the example below, the first process is started and runs (rather,
552sleeps) for 100 seconds. Meanwhile, the second process is started, and
553given the name @samp{my-process<1>} for the sake of uniqueness. It
554inserts the directory listing at the end of the buffer @samp{foo},
555before the first process finishes. Then it finishes, and a message to
556that effect is inserted in the buffer. Much later, the first process
557finishes, and another message is inserted in the buffer for it.
558
559@smallexample
560@group
561(start-process "my-process" "foo" "sleep" "100")
562 @result{} #<process my-process>
563@end group
564
565@group
566(start-process "my-process" "foo" "ls" "-l" "/user/lewis/bin")
567 @result{} #<process my-process<1>>
568
569---------- Buffer: foo ----------
570total 2
571lrwxrwxrwx 1 lewis 14 Jul 22 10:12 gnuemacs --> /emacs
572-rwxrwxrwx 1 lewis 19 Jul 30 21:02 lemon
573
574Process my-process<1> finished
575
576Process my-process finished
577---------- Buffer: foo ----------
578@end group
579@end smallexample
580@end defun
581
582@defun start-file-process name buffer-or-name program &rest args
583Like @code{start-process}, this function starts a new asynchronous
584subprocess running @var{program} in it, and returns its process
585object---when @code{default-directory} is not a magic file name.
586
587If @code{default-directory} is magic, the function invokes its file
588handler instead. This handler ought to run @var{program}, perhaps on
589the local host, perhaps on a remote host that corresponds to
590@code{default-directory}. In the latter case, the local part of
591@code{default-directory} becomes the working directory of the process.
592
593This function does not try to invoke file name handlers for
594@var{program} or for the @var{program-args}.
595
596Depending on the implementation of the file handler, it might not be
597possible to apply @code{process-filter} or @code{process-sentinel} to
598the resulting process object (@pxref{Filter Functions}, @pxref{Sentinels}).
599
600Some file handlers may not support @code{start-file-process} (for
601example @code{ange-ftp-hook-function}). In such cases, the function
602does nothing and returns @code{nil}.
603@end defun
604
03a74b84 605@defun start-process-shell-command name buffer-or-name command
b8d4c8d0
GM
606This function is like @code{start-process} except that it uses a shell
607to execute the specified command. The argument @var{command} is a shell
03a74b84 608command name. The variable @code{shell-file-name} specifies which shell to
b8d4c8d0
GM
609use.
610
611The point of running a program through the shell, rather than directly
612with @code{start-process}, is so that you can employ shell features such
613as wildcards in the arguments. It follows that if you include an
614arbitrary user-specified arguments in the command, you should quote it
615with @code{shell-quote-argument} first, so that any special shell
616characters do @emph{not} have their special shell meanings. @xref{Shell
617Arguments}.
618@end defun
619
03a74b84 620@defun start-file-process-shell-command name buffer-or-name command
b8d4c8d0
GM
621This function is like @code{start-process-shell-command}, but uses
622@code{start-file-process} internally. By this, @var{command} can be
623executed also on remote hosts, depending on @code{default-directory}.
624@end defun
625
626@defvar process-connection-type
627@cindex pipes
628@cindex @acronym{PTY}s
629This variable controls the type of device used to communicate with
630asynchronous subprocesses. If it is non-@code{nil}, then @acronym{PTY}s are
631used, when available. Otherwise, pipes are used.
632
633@acronym{PTY}s are usually preferable for processes visible to the user, as
634in Shell mode, because they allow job control (@kbd{C-c}, @kbd{C-z},
635etc.) to work between the process and its children, whereas pipes do
636not. For subprocesses used for internal purposes by programs, it is
637often better to use a pipe, because they are more efficient. In
638addition, the total number of @acronym{PTY}s is limited on many systems and
639it is good not to waste them.
640
641The value of @code{process-connection-type} takes effect when
642@code{start-process} is called. So you can specify how to communicate
643with one subprocess by binding the variable around the call to
644@code{start-process}.
645
646@smallexample
647@group
648(let ((process-connection-type nil)) ; @r{Use a pipe.}
649 (start-process @dots{}))
650@end group
651@end smallexample
652
653To determine whether a given subprocess actually got a pipe or a
654@acronym{PTY}, use the function @code{process-tty-name} (@pxref{Process
655Information}).
656@end defvar
657
658@node Deleting Processes
659@section Deleting Processes
660@cindex deleting processes
661
662 @dfn{Deleting a process} disconnects Emacs immediately from the
663subprocess. Processes are deleted automatically after they terminate,
664but not necessarily right away. You can delete a process explicitly
665at any time. If you delete a terminated process explicitly before it
666is deleted automatically, no harm results. Deleting a running
667process sends a signal to terminate it (and its child processes if
668any), and calls the process sentinel if it has one. @xref{Sentinels}.
669
670 When a process is deleted, the process object itself continues to
671exist as long as other Lisp objects point to it. All the Lisp
672primitives that work on process objects accept deleted processes, but
673those that do I/O or send signals will report an error. The process
674mark continues to point to the same place as before, usually into a
675buffer where output from the process was being inserted.
676
677@defopt delete-exited-processes
678This variable controls automatic deletion of processes that have
679terminated (due to calling @code{exit} or to a signal). If it is
680@code{nil}, then they continue to exist until the user runs
681@code{list-processes}. Otherwise, they are deleted immediately after
682they exit.
683@end defopt
684
685@defun delete-process process
686This function deletes a process, killing it with a @code{SIGKILL}
687signal. The argument may be a process, the name of a process, a
688buffer, or the name of a buffer. (A buffer or buffer-name stands for
689the process that @code{get-buffer-process} returns.) Calling
690@code{delete-process} on a running process terminates it, updates the
691process status, and runs the sentinel (if any) immediately. If the
692process has already terminated, calling @code{delete-process} has no
693effect on its status, or on the running of its sentinel (which will
694happen sooner or later).
695
696@smallexample
697@group
698(delete-process "*shell*")
699 @result{} nil
700@end group
701@end smallexample
702@end defun
703
704@node Process Information
705@section Process Information
706
707 Several functions return information about processes.
708@code{list-processes} is provided for interactive use.
709
710@deffn Command list-processes &optional query-only
711This command displays a listing of all living processes. In addition,
712it finally deletes any process whose status was @samp{Exited} or
713@samp{Signaled}. It returns @code{nil}.
714
715If @var{query-only} is non-@code{nil} then it lists only processes
716whose query flag is non-@code{nil}. @xref{Query Before Exit}.
717@end deffn
718
719@defun process-list
720This function returns a list of all processes that have not been deleted.
721
722@smallexample
723@group
724(process-list)
725 @result{} (#<process display-time> #<process shell>)
726@end group
727@end smallexample
728@end defun
729
730@defun get-process name
731This function returns the process named @var{name}, or @code{nil} if
732there is none. An error is signaled if @var{name} is not a string.
733
734@smallexample
735@group
736(get-process "shell")
737 @result{} #<process shell>
738@end group
739@end smallexample
740@end defun
741
742@defun process-command process
743This function returns the command that was executed to start
744@var{process}. This is a list of strings, the first string being the
745program executed and the rest of the strings being the arguments that
746were given to the program.
747
748@smallexample
749@group
750(process-command (get-process "shell"))
751 @result{} ("/bin/csh" "-i")
752@end group
753@end smallexample
754@end defun
755
c73e02fa
GM
756@defun process-contact process &optional key
757
758This function returns information about how a network or serial
759process was set up. For a network process, when @var{key} is
760@code{nil}, it returns @code{(@var{hostname} @var{service})} which
761specifies what you connected to. For a serial process, when @var{key}
762is @code{nil}, it returns @code{(@var{port} @var{speed})}. For an
763ordinary child process, this function always returns @code{t}.
764
765If @var{key} is @code{t}, the value is the complete status information
766for the connection, server, or serial port; that is, the list of
767keywords and values specified in @code{make-network-process} or
768@code{make-serial-process}, except that some of the values represent
769the current status instead of what you specified.
770
771For a network process:
772
773@table @code
774@item :buffer
775The associated value is the process buffer.
776@item :filter
777The associated value is the process filter function.
778@item :sentinel
779The associated value is the process sentinel function.
780@item :remote
781In a connection, the address in internal format of the remote peer.
782@item :local
783The local address, in internal format.
784@item :service
785In a server, if you specified @code{t} for @var{service},
786this value is the actual port number.
787@end table
788
789@code{:local} and @code{:remote} are included even if they were not
790specified explicitly in @code{make-network-process}.
791
792For a serial process, see @code{make-serial-process} and
793@code{serial-process-configure} for a list of keys.
794
795If @var{key} is a keyword, the function returns the value corresponding
796to that keyword.
797@end defun
798
b8d4c8d0
GM
799@defun process-id process
800This function returns the @acronym{PID} of @var{process}. This is an
801integer that distinguishes the process @var{process} from all other
802processes running on the same computer at the current time. The
803@acronym{PID} of a process is chosen by the operating system kernel when the
804process is started and remains constant as long as the process exists.
805@end defun
806
807@defun process-name process
808This function returns the name of @var{process}.
809@end defun
810
811@defun process-status process-name
812This function returns the status of @var{process-name} as a symbol.
170ddd97
CY
813The argument @var{process-name} must be a process, a buffer, or a
814process name (a string).
b8d4c8d0
GM
815
816The possible values for an actual subprocess are:
817
818@table @code
819@item run
820for a process that is running.
821@item stop
822for a process that is stopped but continuable.
823@item exit
824for a process that has exited.
825@item signal
826for a process that has received a fatal signal.
827@item open
828for a network connection that is open.
829@item closed
830for a network connection that is closed. Once a connection
831is closed, you cannot reopen it, though you might be able to open
832a new connection to the same place.
833@item connect
834for a non-blocking connection that is waiting to complete.
835@item failed
836for a non-blocking connection that has failed to complete.
837@item listen
838for a network server that is listening.
839@item nil
840if @var{process-name} is not the name of an existing process.
841@end table
842
843@smallexample
844@group
b8d4c8d0
GM
845(process-status (get-buffer "*shell*"))
846 @result{} run
847@end group
848@group
849x
850 @result{} #<process xx<1>>
851(process-status x)
852 @result{} exit
853@end group
854@end smallexample
855
856For a network connection, @code{process-status} returns one of the symbols
857@code{open} or @code{closed}. The latter means that the other side
858closed the connection, or Emacs did @code{delete-process}.
859@end defun
860
c73e02fa
GM
861@defun process-type process
862This function returns the symbol @code{network} for a network
863connection or server, @code{serial} for a serial port connection, or
864@code{real} for a real subprocess.
865@end defun
866
b8d4c8d0
GM
867@defun process-exit-status process
868This function returns the exit status of @var{process} or the signal
869number that killed it. (Use the result of @code{process-status} to
870determine which of those it is.) If @var{process} has not yet
871terminated, the value is 0.
872@end defun
873
874@defun process-tty-name process
875This function returns the terminal name that @var{process} is using for
876its communication with Emacs---or @code{nil} if it is using pipes
877instead of a terminal (see @code{process-connection-type} in
878@ref{Asynchronous Processes}).
879@end defun
880
881@defun process-coding-system process
882@anchor{Coding systems for a subprocess}
883This function returns a cons cell describing the coding systems in use
884for decoding output from @var{process} and for encoding input to
885@var{process} (@pxref{Coding Systems}). The value has this form:
886
887@example
888(@var{coding-system-for-decoding} . @var{coding-system-for-encoding})
889@end example
890@end defun
891
892@defun set-process-coding-system process &optional decoding-system encoding-system
893This function specifies the coding systems to use for subsequent output
894from and input to @var{process}. It will use @var{decoding-system} to
895decode subprocess output, and @var{encoding-system} to encode subprocess
896input.
897@end defun
898
899 Every process also has a property list that you can use to store
900miscellaneous values associated with the process.
901
902@defun process-get process propname
903This function returns the value of the @var{propname} property
904of @var{process}.
905@end defun
906
907@defun process-put process propname value
908This function sets the value of the @var{propname} property
909of @var{process} to @var{value}.
910@end defun
911
912@defun process-plist process
913This function returns the process plist of @var{process}.
914@end defun
915
916@defun set-process-plist process plist
917This function sets the process plist of @var{process} to @var{plist}.
918@end defun
919
920@node Input to Processes
921@section Sending Input to Processes
922@cindex process input
923
924 Asynchronous subprocesses receive input when it is sent to them by
925Emacs, which is done with the functions in this section. You must
926specify the process to send input to, and the input data to send. The
927data appears on the ``standard input'' of the subprocess.
928
929 Some operating systems have limited space for buffered input in a
930@acronym{PTY}. On these systems, Emacs sends an @acronym{EOF}
931periodically amidst the other characters, to force them through. For
932most programs, these @acronym{EOF}s do no harm.
933
934 Subprocess input is normally encoded using a coding system before the
935subprocess receives it, much like text written into a file. You can use
936@code{set-process-coding-system} to specify which coding system to use
937(@pxref{Process Information}). Otherwise, the coding system comes from
938@code{coding-system-for-write}, if that is non-@code{nil}; or else from
939the defaulting mechanism (@pxref{Default Coding Systems}).
940
941 Sometimes the system is unable to accept input for that process,
942because the input buffer is full. When this happens, the send functions
943wait a short while, accepting output from subprocesses, and then try
944again. This gives the subprocess a chance to read more of its pending
945input and make space in the buffer. It also allows filters, sentinels
946and timers to run---so take account of that in writing your code.
947
948 In these functions, the @var{process} argument can be a process or
949the name of a process, or a buffer or buffer name (which stands
950for a process via @code{get-buffer-process}). @code{nil} means
951the current buffer's process.
952
953@defun process-send-string process string
954This function sends @var{process} the contents of @var{string} as
955standard input. If it is @code{nil}, the current buffer's process is used.
956
957 The function returns @code{nil}.
958
959@smallexample
960@group
961(process-send-string "shell<1>" "ls\n")
962 @result{} nil
963@end group
964
965
966@group
967---------- Buffer: *shell* ----------
968...
969introduction.texi syntax-tables.texi~
970introduction.texi~ text.texi
971introduction.txt text.texi~
972...
973---------- Buffer: *shell* ----------
974@end group
975@end smallexample
976@end defun
977
978@defun process-send-region process start end
979This function sends the text in the region defined by @var{start} and
980@var{end} as standard input to @var{process}.
981
982An error is signaled unless both @var{start} and @var{end} are
983integers or markers that indicate positions in the current buffer. (It
984is unimportant which number is larger.)
985@end defun
986
987@defun process-send-eof &optional process
988This function makes @var{process} see an end-of-file in its
989input. The @acronym{EOF} comes after any text already sent to it.
990
991The function returns @var{process}.
992
993@smallexample
994@group
995(process-send-eof "shell")
996 @result{} "shell"
997@end group
998@end smallexample
999@end defun
1000
106e6894
CY
1001@defun process-running-child-p &optional process
1002This function will tell you whether a @var{process} has given control of
b8d4c8d0
GM
1003its terminal to its own child process. The value is @code{t} if this is
1004true, or if Emacs cannot tell; it is @code{nil} if Emacs can be certain
1005that this is not so.
1006@end defun
1007
1008@node Signals to Processes
1009@section Sending Signals to Processes
1010@cindex process signals
1011@cindex sending signals
1012@cindex signals
1013
1014 @dfn{Sending a signal} to a subprocess is a way of interrupting its
1015activities. There are several different signals, each with its own
1016meaning. The set of signals and their names is defined by the operating
1017system. For example, the signal @code{SIGINT} means that the user has
1018typed @kbd{C-c}, or that some analogous thing has happened.
1019
1020 Each signal has a standard effect on the subprocess. Most signals
1021kill the subprocess, but some stop or resume execution instead. Most
1022signals can optionally be handled by programs; if the program handles
1023the signal, then we can say nothing in general about its effects.
1024
1025 You can send signals explicitly by calling the functions in this
1026section. Emacs also sends signals automatically at certain times:
1027killing a buffer sends a @code{SIGHUP} signal to all its associated
1028processes; killing Emacs sends a @code{SIGHUP} signal to all remaining
1029processes. (@code{SIGHUP} is a signal that usually indicates that the
1030user hung up the phone.)
1031
1032 Each of the signal-sending functions takes two optional arguments:
1033@var{process} and @var{current-group}.
1034
1035 The argument @var{process} must be either a process, a process
1036name, a buffer, a buffer name, or @code{nil}. A buffer or buffer name
1037stands for a process through @code{get-buffer-process}. @code{nil}
1038stands for the process associated with the current buffer. An error
1039is signaled if @var{process} does not identify a process.
1040
1041 The argument @var{current-group} is a flag that makes a difference
1042when you are running a job-control shell as an Emacs subprocess. If it
1043is non-@code{nil}, then the signal is sent to the current process-group
1044of the terminal that Emacs uses to communicate with the subprocess. If
1045the process is a job-control shell, this means the shell's current
1046subjob. If it is @code{nil}, the signal is sent to the process group of
1047the immediate subprocess of Emacs. If the subprocess is a job-control
1048shell, this is the shell itself.
1049
1050 The flag @var{current-group} has no effect when a pipe is used to
1051communicate with the subprocess, because the operating system does not
1052support the distinction in the case of pipes. For the same reason,
1053job-control shells won't work when a pipe is used. See
1054@code{process-connection-type} in @ref{Asynchronous Processes}.
1055
1056@defun interrupt-process &optional process current-group
1057This function interrupts the process @var{process} by sending the
1058signal @code{SIGINT}. Outside of Emacs, typing the ``interrupt
1059character'' (normally @kbd{C-c} on some systems, and @code{DEL} on
1060others) sends this signal. When the argument @var{current-group} is
1061non-@code{nil}, you can think of this function as ``typing @kbd{C-c}''
1062on the terminal by which Emacs talks to the subprocess.
1063@end defun
1064
1065@defun kill-process &optional process current-group
1066This function kills the process @var{process} by sending the
1067signal @code{SIGKILL}. This signal kills the subprocess immediately,
1068and cannot be handled by the subprocess.
1069@end defun
1070
1071@defun quit-process &optional process current-group
1072This function sends the signal @code{SIGQUIT} to the process
1073@var{process}. This signal is the one sent by the ``quit
1074character'' (usually @kbd{C-b} or @kbd{C-\}) when you are not inside
1075Emacs.
1076@end defun
1077
1078@defun stop-process &optional process current-group
1079This function stops the process @var{process} by sending the
1080signal @code{SIGTSTP}. Use @code{continue-process} to resume its
1081execution.
1082
1083Outside of Emacs, on systems with job control, the ``stop character''
1084(usually @kbd{C-z}) normally sends this signal. When
1085@var{current-group} is non-@code{nil}, you can think of this function as
1086``typing @kbd{C-z}'' on the terminal Emacs uses to communicate with the
1087subprocess.
1088@end defun
1089
1090@defun continue-process &optional process current-group
1091This function resumes execution of the process @var{process} by sending
1092it the signal @code{SIGCONT}. This presumes that @var{process} was
1093stopped previously.
1094@end defun
1095
b8d4c8d0
GM
1096@defun signal-process process signal
1097This function sends a signal to process @var{process}. The argument
1098@var{signal} specifies which signal to send; it should be an integer.
1099
1100The @var{process} argument can be a system process @acronym{ID}; that
1101allows you to send signals to processes that are not children of
23dd4ecd 1102Emacs. @xref{System Processes}.
b8d4c8d0
GM
1103@end defun
1104
1105@node Output from Processes
1106@section Receiving Output from Processes
1107@cindex process output
1108@cindex output from processes
1109
1110 There are two ways to receive the output that a subprocess writes to
1111its standard output stream. The output can be inserted in a buffer,
1112which is called the associated buffer of the process, or a function
1113called the @dfn{filter function} can be called to act on the output. If
1114the process has no buffer and no filter function, its output is
1115discarded.
1116
1117 When a subprocess terminates, Emacs reads any pending output,
1118then stops reading output from that subprocess. Therefore, if the
1119subprocess has children that are still live and still producing
1120output, Emacs won't receive that output.
1121
1122 Output from a subprocess can arrive only while Emacs is waiting: when
1123reading terminal input, in @code{sit-for} and @code{sleep-for}
1124(@pxref{Waiting}), and in @code{accept-process-output} (@pxref{Accepting
1125Output}). This minimizes the problem of timing errors that usually
1126plague parallel programming. For example, you can safely create a
1127process and only then specify its buffer or filter function; no output
1128can arrive before you finish, if the code in between does not call any
1129primitive that waits.
1130
1131@defvar process-adaptive-read-buffering
1132On some systems, when Emacs reads the output from a subprocess, the
1133output data is read in very small blocks, potentially resulting in
1134very poor performance. This behavior can be remedied to some extent
1135by setting the variable @var{process-adaptive-read-buffering} to a
1136non-@code{nil} value (the default), as it will automatically delay reading
1137from such processes, thus allowing them to produce more output before
1138Emacs tries to read it.
1139@end defvar
1140
1141 It is impossible to separate the standard output and standard error
1142streams of the subprocess, because Emacs normally spawns the subprocess
1143inside a pseudo-TTY, and a pseudo-TTY has only one output channel. If
1144you want to keep the output to those streams separate, you should
1145redirect one of them to a file---for example, by using an appropriate
1146shell command.
1147
1148@menu
1149* Process Buffers:: If no filter, output is put in a buffer.
1150* Filter Functions:: Filter functions accept output from the process.
1151* Decoding Output:: Filters can get unibyte or multibyte strings.
1152* Accepting Output:: How to wait until process output arrives.
1153@end menu
1154
1155@node Process Buffers
1156@subsection Process Buffers
1157
1158 A process can (and usually does) have an @dfn{associated buffer},
1159which is an ordinary Emacs buffer that is used for two purposes: storing
1160the output from the process, and deciding when to kill the process. You
1161can also use the buffer to identify a process to operate on, since in
1162normal practice only one process is associated with any given buffer.
1163Many applications of processes also use the buffer for editing input to
1164be sent to the process, but this is not built into Emacs Lisp.
1165
1166 Unless the process has a filter function (@pxref{Filter Functions}),
1167its output is inserted in the associated buffer. The position to insert
1168the output is determined by the @code{process-mark}, which is then
1169updated to point to the end of the text just inserted. Usually, but not
1170always, the @code{process-mark} is at the end of the buffer.
1171
16d1ff5f
CY
1172@findex process-kill-buffer-query-function
1173 Killing the associated buffer of a process also kills the process.
1174Emacs asks for confirmation first, if the process's
1175@code{process-query-on-exit-flag} is non-@code{nil} (@pxref{Query
1176Before Exit}). This confirmation is done by the function
1177@code{process-kill-buffer-query-function}, which is run from
1178@code{kill-buffer-query-functions} (@pxref{Killing Buffers}).
1179
b8d4c8d0
GM
1180@defun process-buffer process
1181This function returns the associated buffer of the process
1182@var{process}.
1183
1184@smallexample
1185@group
1186(process-buffer (get-process "shell"))
1187 @result{} #<buffer *shell*>
1188@end group
1189@end smallexample
1190@end defun
1191
1192@defun process-mark process
1193This function returns the process marker for @var{process}, which is the
1194marker that says where to insert output from the process.
1195
1196If @var{process} does not have a buffer, @code{process-mark} returns a
1197marker that points nowhere.
1198
1199Insertion of process output in a buffer uses this marker to decide where
1200to insert, and updates it to point after the inserted text. That is why
1201successive batches of output are inserted consecutively.
1202
1203Filter functions normally should use this marker in the same fashion
1204as is done by direct insertion of output in the buffer. A good
1205example of a filter function that uses @code{process-mark} is found at
1206the end of the following section.
1207
1208When the user is expected to enter input in the process buffer for
1209transmission to the process, the process marker separates the new input
1210from previous output.
1211@end defun
1212
1213@defun set-process-buffer process buffer
1214This function sets the buffer associated with @var{process} to
1215@var{buffer}. If @var{buffer} is @code{nil}, the process becomes
1216associated with no buffer.
1217@end defun
1218
1219@defun get-buffer-process buffer-or-name
1220This function returns a nondeleted process associated with the buffer
1221specified by @var{buffer-or-name}. If there are several processes
1222associated with it, this function chooses one (currently, the one most
1223recently created, but don't count on that). Deletion of a process
1224(see @code{delete-process}) makes it ineligible for this function to
1225return.
1226
1227It is usually a bad idea to have more than one process associated with
1228the same buffer.
1229
1230@smallexample
1231@group
1232(get-buffer-process "*shell*")
1233 @result{} #<process shell>
1234@end group
1235@end smallexample
1236
1237Killing the process's buffer deletes the process, which kills the
1238subprocess with a @code{SIGHUP} signal (@pxref{Signals to Processes}).
1239@end defun
1240
1241@node Filter Functions
1242@subsection Process Filter Functions
1243@cindex filter function
1244@cindex process filter
1245
1246 A process @dfn{filter function} is a function that receives the
1247standard output from the associated process. If a process has a filter,
1248then @emph{all} output from that process is passed to the filter. The
1249process buffer is used directly for output from the process only when
1250there is no filter.
1251
1252 The filter function can only be called when Emacs is waiting for
1253something, because process output arrives only at such times. Emacs
1254waits when reading terminal input, in @code{sit-for} and
1255@code{sleep-for} (@pxref{Waiting}), and in @code{accept-process-output}
1256(@pxref{Accepting Output}).
1257
1258 A filter function must accept two arguments: the associated process
1259and a string, which is output just received from it. The function is
1260then free to do whatever it chooses with the output.
1261
1262 Quitting is normally inhibited within a filter function---otherwise,
1263the effect of typing @kbd{C-g} at command level or to quit a user
1264command would be unpredictable. If you want to permit quitting inside
1265a filter function, bind @code{inhibit-quit} to @code{nil}. In most
1266cases, the right way to do this is with the macro
1267@code{with-local-quit}. @xref{Quitting}.
1268
1269 If an error happens during execution of a filter function, it is
1270caught automatically, so that it doesn't stop the execution of whatever
1271program was running when the filter function was started. However, if
1272@code{debug-on-error} is non-@code{nil}, the error-catching is turned
1273off. This makes it possible to use the Lisp debugger to debug the
1274filter function. @xref{Debugger}.
1275
1276 Many filter functions sometimes or always insert the text in the
1277process's buffer, mimicking the actions of Emacs when there is no
1278filter. Such filter functions need to use @code{set-buffer} in order to
1279be sure to insert in that buffer. To avoid setting the current buffer
1280semipermanently, these filter functions must save and restore the
1281current buffer. They should also update the process marker, and in some
1282cases update the value of point. Here is how to do these things:
1283
1284@smallexample
1285@group
1286(defun ordinary-insertion-filter (proc string)
1287 (with-current-buffer (process-buffer proc)
1288 (let ((moving (= (point) (process-mark proc))))
1289@end group
1290@group
1291 (save-excursion
1292 ;; @r{Insert the text, advancing the process marker.}
1293 (goto-char (process-mark proc))
1294 (insert string)
1295 (set-marker (process-mark proc) (point)))
1296 (if moving (goto-char (process-mark proc))))))
1297@end group
1298@end smallexample
1299
1300@noindent
1301The reason to use @code{with-current-buffer}, rather than using
1302@code{save-excursion} to save and restore the current buffer, is so as
1303to preserve the change in point made by the second call to
1304@code{goto-char}.
1305
1306 To make the filter force the process buffer to be visible whenever new
1307text arrives, insert the following line just before the
1308@code{with-current-buffer} construct:
1309
1310@smallexample
1311(display-buffer (process-buffer proc))
1312@end smallexample
1313
1314 To force point to the end of the new output, no matter where it was
1315previously, eliminate the variable @code{moving} and call
1316@code{goto-char} unconditionally.
1317
1318 In earlier Emacs versions, every filter function that did regular
1319expression searching or matching had to explicitly save and restore the
1320match data. Now Emacs does this automatically for filter functions;
1321they never need to do it explicitly. @xref{Match Data}.
1322
1323 A filter function that writes the output into the buffer of the
1324process should check whether the buffer is still alive. If it tries to
1325insert into a dead buffer, it will get an error. The expression
1326@code{(buffer-name (process-buffer @var{process}))} returns @code{nil}
1327if the buffer is dead.
1328
1329 The output to the function may come in chunks of any size. A program
1330that produces the same output twice in a row may send it as one batch of
1331200 characters one time, and five batches of 40 characters the next. If
1332the filter looks for certain text strings in the subprocess output, make
1333sure to handle the case where one of these strings is split across two
fd211f0b
CY
1334or more batches of output; one way to do this is to insert the
1335received text into a temporary buffer, which can then be searched.
b8d4c8d0
GM
1336
1337@defun set-process-filter process filter
1338This function gives @var{process} the filter function @var{filter}. If
1339@var{filter} is @code{nil}, it gives the process no filter.
1340@end defun
1341
1342@defun process-filter process
1343This function returns the filter function of @var{process}, or @code{nil}
1344if it has none.
1345@end defun
1346
1347 Here is an example of use of a filter function:
1348
1349@smallexample
1350@group
1351(defun keep-output (process output)
1352 (setq kept (cons output kept)))
1353 @result{} keep-output
1354@end group
1355@group
1356(setq kept nil)
1357 @result{} nil
1358@end group
1359@group
1360(set-process-filter (get-process "shell") 'keep-output)
1361 @result{} keep-output
1362@end group
1363@group
1364(process-send-string "shell" "ls ~/other\n")
1365 @result{} nil
1366kept
1367 @result{} ("lewis@@slug[8] % "
1368@end group
1369@group
1370"FINAL-W87-SHORT.MSS backup.otl kolstad.mss~
1371address.txt backup.psf kolstad.psf
1372backup.bib~ david.mss resume-Dec-86.mss~
1373backup.err david.psf resume-Dec.psf
1374backup.mss dland syllabus.mss
1375"
1376"#backups.mss# backup.mss~ kolstad.mss
1377")
1378@end group
1379@end smallexample
1380
1381@ignore @c The code in this example doesn't show the right way to do things.
1382Here is another, more realistic example, which demonstrates how to use
1383the process mark to do insertion in the same fashion as is done when
1384there is no filter function:
1385
1386@smallexample
1387@group
1388;; @r{Insert input in the buffer specified by @code{my-shell-buffer}}
1389;; @r{and make sure that buffer is shown in some window.}
1390(defun my-process-filter (proc str)
1391 (let ((cur (selected-window))
1392 (pop-up-windows t))
1393 (pop-to-buffer my-shell-buffer)
1394@end group
1395@group
1396 (goto-char (point-max))
1397 (insert str)
1398 (set-marker (process-mark proc) (point-max))
1399 (select-window cur)))
1400@end group
1401@end smallexample
1402@end ignore
1403
1404@node Decoding Output
1405@subsection Decoding Process Output
1406@cindex decode process output
1407
1408 When Emacs writes process output directly into a multibyte buffer,
1409it decodes the output according to the process output coding system.
1410If the coding system is @code{raw-text} or @code{no-conversion}, Emacs
1411converts the unibyte output to multibyte using
1412@code{string-to-multibyte}, and inserts the resulting multibyte text.
1413
1414 You can use @code{set-process-coding-system} to specify which coding
1415system to use (@pxref{Process Information}). Otherwise, the coding
1416system comes from @code{coding-system-for-read}, if that is
1417non-@code{nil}; or else from the defaulting mechanism (@pxref{Default
3355f04d
EZ
1418Coding Systems}). If the text output by a process contains null
1419bytes, Emacs by default uses @code{no-conversion} for it; see
1420@ref{Lisp and Coding Systems, inhibit-null-byte-detection}, for how to
1421control this behavior.
b8d4c8d0
GM
1422
1423 @strong{Warning:} Coding systems such as @code{undecided} which
1424determine the coding system from the data do not work entirely
1425reliably with asynchronous subprocess output. This is because Emacs
1426has to process asynchronous subprocess output in batches, as it
1427arrives. Emacs must try to detect the proper coding system from one
1428batch at a time, and this does not always work. Therefore, if at all
1429possible, specify a coding system that determines both the character
1430code conversion and the end of line conversion---that is, one like
1431@code{latin-1-unix}, rather than @code{undecided} or @code{latin-1}.
1432
4972c361
SM
1433@c Let's keep the index entries that were there for
1434@c set-process-filter-multibyte and process-filter-multibyte-p,
b8d4c8d0
GM
1435@cindex filter multibyte flag, of process
1436@cindex process filter multibyte flag
1437 When Emacs calls a process filter function, it provides the process
1438output as a multibyte string or as a unibyte string according to the
4972c361
SM
1439process's filter coding system. Emacs
1440decodes the output according to the process output coding system,
1441which usually produces a multibyte string, except for coding systems
1442such as @code{binary} and @code{raw-text}
b8d4c8d0
GM
1443
1444@node Accepting Output
1445@subsection Accepting Output from Processes
1446@cindex accept input from processes
1447
1448 Output from asynchronous subprocesses normally arrives only while
1449Emacs is waiting for some sort of external event, such as elapsed time
1450or terminal input. Occasionally it is useful in a Lisp program to
1451explicitly permit output to arrive at a specific point, or even to wait
1452until output arrives from a process.
1453
1454@defun accept-process-output &optional process seconds millisec just-this-one
1455This function allows Emacs to read pending output from processes. The
1456output is inserted in the associated buffers or given to their filter
1457functions. If @var{process} is non-@code{nil} then this function does
1458not return until some output has been received from @var{process}.
1459
1460@c Emacs 19 feature
1461The arguments @var{seconds} and @var{millisec} let you specify timeout
1462periods. The former specifies a period measured in seconds and the
1463latter specifies one measured in milliseconds. The two time periods
1464thus specified are added together, and @code{accept-process-output}
1465returns after that much time, whether or not there has been any
1466subprocess output.
1467
1468The argument @var{millisec} is semi-obsolete nowadays because
1469@var{seconds} can be a floating point number to specify waiting a
1470fractional number of seconds. If @var{seconds} is 0, the function
1471accepts whatever output is pending but does not wait.
1472
1473@c Emacs 22.1 feature
1474If @var{process} is a process, and the argument @var{just-this-one} is
1475non-@code{nil}, only output from that process is handled, suspending output
1476from other processes until some output has been received from that
1477process or the timeout expires. If @var{just-this-one} is an integer,
1478also inhibit running timers. This feature is generally not
1479recommended, but may be necessary for specific applications, such as
1480speech synthesis.
1481
1482The function @code{accept-process-output} returns non-@code{nil} if it
1483did get some output, or @code{nil} if the timeout expired before output
1484arrived.
1485@end defun
1486
1487@node Sentinels
1488@section Sentinels: Detecting Process Status Changes
1489@cindex process sentinel
1490@cindex sentinel (of process)
1491
1492 A @dfn{process sentinel} is a function that is called whenever the
1493associated process changes status for any reason, including signals
1494(whether sent by Emacs or caused by the process's own actions) that
1495terminate, stop, or continue the process. The process sentinel is
1496also called if the process exits. The sentinel receives two
1497arguments: the process for which the event occurred, and a string
1498describing the type of event.
1499
1500 The string describing the event looks like one of the following:
1501
1502@itemize @bullet
1503@item
1504@code{"finished\n"}.
1505
1506@item
1507@code{"exited abnormally with code @var{exitcode}\n"}.
1508
1509@item
1510@code{"@var{name-of-signal}\n"}.
1511
1512@item
1513@code{"@var{name-of-signal} (core dumped)\n"}.
1514@end itemize
1515
1516 A sentinel runs only while Emacs is waiting (e.g., for terminal
1517input, or for time to elapse, or for process output). This avoids the
1518timing errors that could result from running them at random places in
1519the middle of other Lisp programs. A program can wait, so that
1520sentinels will run, by calling @code{sit-for} or @code{sleep-for}
1521(@pxref{Waiting}), or @code{accept-process-output} (@pxref{Accepting
1522Output}). Emacs also allows sentinels to run when the command loop is
1523reading input. @code{delete-process} calls the sentinel when it
1524terminates a running process.
1525
1526 Emacs does not keep a queue of multiple reasons to call the sentinel
1527of one process; it records just the current status and the fact that
1528there has been a change. Therefore two changes in status, coming in
1529quick succession, can call the sentinel just once. However, process
1530termination will always run the sentinel exactly once. This is
1531because the process status can't change again after termination.
1532
1533 Emacs explicitly checks for output from the process before running
1534the process sentinel. Once the sentinel runs due to process
1535termination, no further output can arrive from the process.
1536
1537 A sentinel that writes the output into the buffer of the process
1538should check whether the buffer is still alive. If it tries to insert
1539into a dead buffer, it will get an error. If the buffer is dead,
1540@code{(buffer-name (process-buffer @var{process}))} returns @code{nil}.
1541
1542 Quitting is normally inhibited within a sentinel---otherwise, the
1543effect of typing @kbd{C-g} at command level or to quit a user command
1544would be unpredictable. If you want to permit quitting inside a
1545sentinel, bind @code{inhibit-quit} to @code{nil}. In most cases, the
1546right way to do this is with the macro @code{with-local-quit}.
1547@xref{Quitting}.
1548
1549 If an error happens during execution of a sentinel, it is caught
1550automatically, so that it doesn't stop the execution of whatever
1551programs was running when the sentinel was started. However, if
1552@code{debug-on-error} is non-@code{nil}, the error-catching is turned
1553off. This makes it possible to use the Lisp debugger to debug the
1554sentinel. @xref{Debugger}.
1555
1556 While a sentinel is running, the process sentinel is temporarily
1557set to @code{nil} so that the sentinel won't run recursively.
1558For this reason it is not possible for a sentinel to specify
1559a new sentinel.
1560
1561 In earlier Emacs versions, every sentinel that did regular expression
1562searching or matching had to explicitly save and restore the match data.
1563Now Emacs does this automatically for sentinels; they never need to do
1564it explicitly. @xref{Match Data}.
1565
1566@defun set-process-sentinel process sentinel
1567This function associates @var{sentinel} with @var{process}. If
1568@var{sentinel} is @code{nil}, then the process will have no sentinel.
1569The default behavior when there is no sentinel is to insert a message in
1570the process's buffer when the process status changes.
1571
1572Changes in process sentinel take effect immediately---if the sentinel
1573is slated to be run but has not been called yet, and you specify a new
1574sentinel, the eventual call to the sentinel will use the new one.
1575
1576@smallexample
1577@group
1578(defun msg-me (process event)
1579 (princ
1580 (format "Process: %s had the event `%s'" process event)))
1581(set-process-sentinel (get-process "shell") 'msg-me)
1582 @result{} msg-me
1583@end group
1584@group
1585(kill-process (get-process "shell"))
1586 @print{} Process: #<process shell> had the event `killed'
1587 @result{} #<process shell>
1588@end group
1589@end smallexample
1590@end defun
1591
1592@defun process-sentinel process
1593This function returns the sentinel of @var{process}, or @code{nil} if it
1594has none.
1595@end defun
1596
1597@defun waiting-for-user-input-p
1598While a sentinel or filter function is running, this function returns
1599non-@code{nil} if Emacs was waiting for keyboard input from the user at
1600the time the sentinel or filter function was called, @code{nil} if it
1601was not.
1602@end defun
1603
1604@node Query Before Exit
1605@section Querying Before Exit
1606
1607 When Emacs exits, it terminates all its subprocesses by sending them
1608the @code{SIGHUP} signal. Because subprocesses may be doing
1609valuable work, Emacs normally asks the user to confirm that it is ok
1610to terminate them. Each process has a query flag which, if
1611non-@code{nil}, says that Emacs should ask for confirmation before
1612exiting and thus killing that process. The default for the query flag
1613is @code{t}, meaning @emph{do} query.
1614
1615@defun process-query-on-exit-flag process
1616This returns the query flag of @var{process}.
1617@end defun
1618
1619@defun set-process-query-on-exit-flag process flag
1620This function sets the query flag of @var{process} to @var{flag}. It
1621returns @var{flag}.
1622
1623@smallexample
1624@group
1625;; @r{Don't query about the shell process}
1626(set-process-query-on-exit-flag (get-process "shell") nil)
1627 @result{} t
1628@end group
1629@end smallexample
1630@end defun
1631
1632@defun process-kill-without-query process &optional do-query
1633This function clears the query flag of @var{process}, so that
1634Emacs will not query the user on account of that process.
1635
1636Actually, the function does more than that: it returns the old value of
1637the process's query flag, and sets the query flag to @var{do-query}.
1638Please don't use this function to do those things any more---please
1639use the newer, cleaner functions @code{process-query-on-exit-flag} and
1640@code{set-process-query-on-exit-flag} in all but the simplest cases.
1641The only way you should use @code{process-kill-without-query} nowadays
1642is like this:
1643
1644@smallexample
1645@group
1646;; @r{Don't query about the shell process}
1647(process-kill-without-query (get-process "shell"))
1648@end group
1649@end smallexample
1650@end defun
1651
23dd4ecd
EZ
1652@node System Processes
1653@section Accessing Other Processes
1654@cindex system processes
1655
1656 In addition to accessing and manipulating processes that are
1657subprocesses of the current Emacs session, Emacs Lisp programs can
1658also access other processes running on the same machine. We call
1659these @dfn{system processes}, to distinguish between them and Emacs
1660subprocesses.
1661
1662 Emacs provides several primitives for accessing system processes.
1663Not all platforms support these primitives; on those which don't,
1664these primitives return @code{nil}.
1665
1666@defun list-system-processes
1667This function returns a list of all the processes running on the
1668system. Each process is identified by its @acronym{PID}, a numerical
1669process ID that is assigned by the OS and distinguishes the process
1670from all the other processes running on the same machine at the same
1671time.
1672@end defun
1673
a20878b6 1674@defun process-attributes pid
23dd4ecd
EZ
1675This function returns an alist of attributes for the process specified
1676by its process ID @var{pid}. Each association in the alist is of the
1677form @code{(@var{key} . @var{value})}, where @var{key} designates the
1678attribute and @var{value} is the value of that attribute. The various
1679attribute @var{key}'s that this function can return are listed below.
1680Not all platforms support all of these attributes; if an attribute is
1681not supported, its association will not appear in the returned alist.
1682Values that are numbers can be either integer or floating-point,
1683depending on the magnitude of the value.
1684
1685@table @code
1686@item euid
1687The effective user ID of the user who invoked the process. The
1688corresponding @var{value} is a number. If the process was invoked by
1689the same user who runs the current Emacs session, the value is
1690identical to what @code{user-uid} returns (@pxref{User
1691Identification}).
1692
1693@item user
1694User name corresponding to the process's effective user ID, a string.
1695
1696@item egid
1697The group ID of the effective user ID, a number.
1698
1699@item group
1700Group name corresponding to the effective user's group ID, a string.
1701
1702@item comm
1703The name of the command that runs in the process. This is a string
1704that usually specifies the name of the executable file of the process,
1705without the leading directories. However, some special system
1706processes can report strings that do not correspond to an executable
1707file of a program.
1708
1709@item state
1710The state code of the process. This is a short string that encodes
1711the scheduling state of the process. Here's a list of the most
1712frequently seen codes:
1713
1714@table @code
1dca458f 1715@item "D"
23dd4ecd 1716uninterruptible sleep (usually I/O)
1dca458f 1717@item "R"
23dd4ecd 1718running
1dca458f 1719@item "S"
23dd4ecd 1720interruptible sleep (waiting for some event)
1dca458f 1721@item "T"
23dd4ecd 1722stopped, e.g., by a job control signal
1dca458f
EZ
1723@item "Z"
1724``zombie'': a process that terminated, but was not reaped by its parent
23dd4ecd
EZ
1725@end table
1726
1727@noindent
1728For the full list of the possible states, see the manual page of the
1729@command{ps} command.
1730
1731@item ppid
1732The process ID of the parent process, a number.
1733
1734@item pgrp
1735The process group ID of the process, a number.
1736
1737@item sess
1738The session ID of the process. This is a number that is the process
1739ID of the process's @dfn{session leader}.
1740
1741@item ttname
1742A string that is the name of the process's controlling terminal. On
1743Unix and GNU systems, this is normally the file name of the
1744corresponding terminal device, such as @file{/dev/pts65}.
1745
1746@item tpgid
1747The numerical process group ID of the foreground process group that
1748uses the process's terminal.
1749
1750@item minflt
1751The number of minor page faults caused by the process since its
1752beginning. (Minor page faults are those that don't involve reading
1753from disk.)
1754
1755@item majflt
1756The number of major page faults caused by the process since its
1757beginning. (Major page faults require a disk to be read, and are thus
1758more expensive than minor page faults.)
1759
1760@item cminflt
1761@itemx cmajflt
1762Like @code{minflt} and @code{majflt}, but include the number of page
1763faults for all the child processes of the given process.
1764
1765@item utime
1766Time spent by the process in the user context, for running the
1767application's code. The corresponding @var{value} is in the
1768@w{@code{(@var{high} @var{low} @var{microsec})}} format, the same
1769format used by functions @code{current-time} (@pxref{Time of Day,
1770current-time}) and @code{file-attributes} (@pxref{File Attributes}).
1771
1772@item stime
1773Time spent by the process in the system (kernel) context, for
1774processing system calls. The corresponding @var{value} is in the same
1775format as for @code{utime}.
1776
af34ad36
EZ
1777@item time
1778The sum of @code{utime} and @code{stime}. The corresponding
1779@var{value} is in the same format as for @code{utime}.
1780
23dd4ecd
EZ
1781@item cutime
1782@itemx cstime
af34ad36
EZ
1783@itemx ctime
1784Like @code{utime}, @code{stime}, and @code{time}, but include the
1785times of all the child processes of the given process.
23dd4ecd
EZ
1786
1787@item pri
1788The numerical priority of the process.
1789
1790@item nice
1dca458f
EZ
1791The @dfn{nice value} of the process, a number. (Processes with smaller
1792nice values get scheduled more favorably.)
23dd4ecd
EZ
1793
1794@item thcount
1795The number of threads in the process.
1796
1797@item start
1798The time the process was started, in the @w{@code{(@var{high}
1799@var{low} @var{microsec})}} format used by @code{current-time} and
1800@code{file-attributes}.
1801
1802@item etime
1803The time elapsed since the process started, in the @w{@code{(@var{high}
1804@var{low} @var{microsec})}} format.
1805
1806@item vsize
1807The virtual memory size of the process, measured in kilobytes.
1808
1809@item rss
1810The size of the process's @dfn{resident set}, the number of kilobytes
1811occupied by the process in the machine's physical memory.
1812
1813@item pcpu
1814The percentage of the CPU time used by the process since it started.
1815The corresponding @var{value} is a floating-point number between 0 and
1816100.
1817
1818@item pmem
1819The percentage of the total physical memory installed on the machine
1820used by the process's resident set. The value is a floating-point
1821number between 0 and 100.
1822
1823@item args
1824The command-line with which the process was invoked. This is a string
1825in which individual command-line arguments are separated by blanks;
1826whitespace characters that are embedded in the arguments are quoted as
1827appropriate for the system's shell: escaped by backslash characters on
1828GNU and Unix, and enclosed in double quote characters on Windows.
1829Thus, this command-line string can be directly used in primitives such
1830as @code{shell-command}.
1831@end table
1832
1833@end defun
1834
1835
b8d4c8d0
GM
1836@node Transaction Queues
1837@section Transaction Queues
1838@cindex transaction queue
1839
1840You can use a @dfn{transaction queue} to communicate with a subprocess
1841using transactions. First use @code{tq-create} to create a transaction
1842queue communicating with a specified process. Then you can call
1843@code{tq-enqueue} to send a transaction.
1844
1845@defun tq-create process
1846This function creates and returns a transaction queue communicating with
1847@var{process}. The argument @var{process} should be a subprocess
1848capable of sending and receiving streams of bytes. It may be a child
1849process, or it may be a TCP connection to a server, possibly on another
1850machine.
1851@end defun
1852
1853@defun tq-enqueue queue question regexp closure fn &optional delay-question
1854This function sends a transaction to queue @var{queue}. Specifying the
1855queue has the effect of specifying the subprocess to talk to.
1856
1857The argument @var{question} is the outgoing message that starts the
1858transaction. The argument @var{fn} is the function to call when the
1859corresponding answer comes back; it is called with two arguments:
1860@var{closure}, and the answer received.
1861
1862The argument @var{regexp} is a regular expression that should match
1863text at the end of the entire answer, but nothing before; that's how
1864@code{tq-enqueue} determines where the answer ends.
1865
4185820c
CY
1866If the argument @var{delay-question} is non-@code{nil}, delay sending
1867this question until the process has finished replying to any previous
b8d4c8d0
GM
1868questions. This produces more reliable results with some processes.
1869
1870The return value of @code{tq-enqueue} itself is not meaningful.
1871@end defun
1872
1873@defun tq-close queue
1874Shut down transaction queue @var{queue}, waiting for all pending transactions
1875to complete, and then terminate the connection or child process.
1876@end defun
1877
1878Transaction queues are implemented by means of a filter function.
1879@xref{Filter Functions}.
1880
1881@node Network
1882@section Network Connections
1883@cindex network connection
1884@cindex TCP
1885@cindex UDP
1886
1887 Emacs Lisp programs can open stream (TCP) and datagram (UDP) network
1888connections to other processes on the same machine or other machines.
1889A network connection is handled by Lisp much like a subprocess, and is
1890represented by a process object. However, the process you are
1891communicating with is not a child of the Emacs process, so it has no
1892process @acronym{ID}, and you can't kill it or send it signals. All you
1893can do is send and receive data. @code{delete-process} closes the
1894connection, but does not kill the program at the other end; that
1895program must decide what to do about closure of the connection.
1896
1897 Lisp programs can listen for connections by creating network
1898servers. A network server is also represented by a kind of process
1899object, but unlike a network connection, the network server never
1900transfers data itself. When it receives a connection request, it
1901creates a new network connection to represent the connection just
1902made. (The network connection inherits certain information, including
1903the process plist, from the server.) The network server then goes
1904back to listening for more connection requests.
1905
1906 Network connections and servers are created by calling
1907@code{make-network-process} with an argument list consisting of
1908keyword/argument pairs, for example @code{:server t} to create a
1909server process, or @code{:type 'datagram} to create a datagram
1910connection. @xref{Low-Level Network}, for details. You can also use
1911the @code{open-network-stream} function described below.
1912
c73e02fa
GM
1913 To distinguish the different types of processes, the
1914@code{process-type} function returns the symbol @code{network} for a
1915network connection or server, @code{serial} for a serial port
1916connection, or @code{real} for a real subprocess.
1917
1918 The @code{process-status} function returns @code{open},
1919@code{closed}, @code{connect}, and @code{failed} for network
1920connections. For a network server, the status is always
b8d4c8d0
GM
1921@code{listen}. None of those values is possible for a real
1922subprocess. @xref{Process Information}.
1923
1924 You can stop and resume operation of a network process by calling
1925@code{stop-process} and @code{continue-process}. For a server
1926process, being stopped means not accepting new connections. (Up to 5
1927connection requests will be queued for when you resume the server; you
1928can increase this limit, unless it is imposed by the operating
1929system.) For a network stream connection, being stopped means not
1930processing input (any arriving input waits until you resume the
1931connection). For a datagram connection, some number of packets may be
1932queued but input may be lost. You can use the function
1933@code{process-command} to determine whether a network connection or
1934server is stopped; a non-@code{nil} value means yes.
1935
1936@defun open-network-stream name buffer-or-name host service
1937This function opens a TCP connection, and returns a process object
1938that represents the connection.
1939
1940The @var{name} argument specifies the name for the process object. It
1941is modified as necessary to make it unique.
1942
1943The @var{buffer-or-name} argument is the buffer to associate with the
1944connection. Output from the connection is inserted in the buffer,
1945unless you specify a filter function to handle the output. If
1946@var{buffer-or-name} is @code{nil}, it means that the connection is not
1947associated with any buffer.
1948
1949The arguments @var{host} and @var{service} specify where to connect to;
1950@var{host} is the host name (a string), and @var{service} is the name of
1951a defined network service (a string) or a port number (an integer).
1952@end defun
1953
b8d4c8d0
GM
1954@node Network Servers
1955@section Network Servers
1956@cindex network servers
1957
1958 You create a server by calling @code{make-network-process} with
1959@code{:server t}. The server will listen for connection requests from
1960clients. When it accepts a client connection request, that creates a
1961new network connection, itself a process object, with the following
1962parameters:
1963
1964@itemize @bullet
1965@item
1966The connection's process name is constructed by concatenating the
1967server process' @var{name} with a client identification string. The
1968client identification string for an IPv4 connection looks like
1969@samp{<@var{a}.@var{b}.@var{c}.@var{d}:@var{p}>}. Otherwise, it is a
1970unique number in brackets, as in @samp{<@var{nnn}>}. The number
1971is unique for each connection in the Emacs session.
1972
1973@item
1974If the server's filter is non-@code{nil}, the connection process does
1975not get a separate process buffer; otherwise, Emacs creates a new
1976buffer for the purpose. The buffer name is the server's buffer name
1977or process name, concatenated with the client identification string.
1978
1979The server's process buffer value is never used directly by Emacs, but
1980it is passed to the log function, which can log connections by
1981inserting text there.
1982
1983@item
1984The communication type and the process filter and sentinel are
1985inherited from those of the server. The server never directly
1986uses its filter and sentinel; their sole purpose is to initialize
1987connections made to the server.
1988
1989@item
1990The connection's process contact info is set according to the client's
1991addressing information (typically an IP address and a port number).
1992This information is associated with the @code{process-contact}
1993keywords @code{:host}, @code{:service}, @code{:remote}.
1994
1995@item
1996The connection's local address is set up according to the port
1997number used for the connection.
1998
1999@item
2000The client process' plist is initialized from the server's plist.
2001@end itemize
2002
2003@node Datagrams
2004@section Datagrams
2005@cindex datagrams
2006
2007 A datagram connection communicates with individual packets rather
2008than streams of data. Each call to @code{process-send} sends one
2009datagram packet (@pxref{Input to Processes}), and each datagram
2010received results in one call to the filter function.
2011
2012 The datagram connection doesn't have to talk with the same remote
2013peer all the time. It has a @dfn{remote peer address} which specifies
2014where to send datagrams to. Each time an incoming datagram is passed
2015to the filter function, the peer address is set to the address that
2016datagram came from; that way, if the filter function sends a datagram,
2017it will go back to that place. You can specify the remote peer
2018address when you create the datagram connection using the
2019@code{:remote} keyword. You can change it later on by calling
2020@code{set-process-datagram-address}.
2021
2022@defun process-datagram-address process
2023If @var{process} is a datagram connection or server, this function
2024returns its remote peer address.
2025@end defun
2026
2027@defun set-process-datagram-address process address
2028If @var{process} is a datagram connection or server, this function
2029sets its remote peer address to @var{address}.
2030@end defun
2031
2032@node Low-Level Network
2033@section Low-Level Network Access
2034
2035 You can also create network connections by operating at a lower
2036level than that of @code{open-network-stream}, using
2037@code{make-network-process}.
2038
2039@menu
2040* Proc: Network Processes. Using @code{make-network-process}.
2041* Options: Network Options. Further control over network connections.
2042* Features: Network Feature Testing.
2043 Determining which network features work on
2044 the machine you are using.
2045@end menu
2046
2047@node Network Processes
2048@subsection @code{make-network-process}
2049
2050 The basic function for creating network connections and network
2051servers is @code{make-network-process}. It can do either of those
2052jobs, depending on the arguments you give it.
2053
2054@defun make-network-process &rest args
2055This function creates a network connection or server and returns the
2056process object that represents it. The arguments @var{args} are a
2057list of keyword/argument pairs. Omitting a keyword is always
2058equivalent to specifying it with value @code{nil}, except for
2059@code{:coding}, @code{:filter-multibyte}, and @code{:reuseaddr}. Here
2060are the meaningful keywords:
2061
2062@table @asis
2063@item :name @var{name}
2064Use the string @var{name} as the process name. It is modified if
2065necessary to make it unique.
2066
2067@item :type @var{type}
2068Specify the communication type. A value of @code{nil} specifies a
2069stream connection (the default); @code{datagram} specifies a datagram
f7704e26
CY
2070connection; @code{seqpacket} specifies a ``sequenced packet stream''
2071connection. Both connections and servers can be of these types.
b8d4c8d0
GM
2072
2073@item :server @var{server-flag}
2074If @var{server-flag} is non-@code{nil}, create a server. Otherwise,
2075create a connection. For a stream type server, @var{server-flag} may
2076be an integer which then specifies the length of the queue of pending
2077connections to the server. The default queue length is 5.
2078
2079@item :host @var{host}
2080Specify the host to connect to. @var{host} should be a host name or
2081Internet address, as a string, or the symbol @code{local} to specify
2082the local host. If you specify @var{host} for a server, it must
2083specify a valid address for the local host, and only clients
2084connecting to that address will be accepted.
2085
2086@item :service @var{service}
2087@var{service} specifies a port number to connect to, or, for a server,
2088the port number to listen on. It should be a service name that
2089translates to a port number, or an integer specifying the port number
2090directly. For a server, it can also be @code{t}, which means to let
2091the system select an unused port number.
2092
2093@item :family @var{family}
2094@var{family} specifies the address (and protocol) family for
2095communication. @code{nil} means determine the proper address family
2096automatically for the given @var{host} and @var{service}.
2097@code{local} specifies a Unix socket, in which case @var{host} is
2098ignored. @code{ipv4} and @code{ipv6} specify to use IPv4 and IPv6
2099respectively.
2100
2101@item :local @var{local-address}
2102For a server process, @var{local-address} is the address to listen on.
2103It overrides @var{family}, @var{host} and @var{service}, and you
2104may as well not specify them.
2105
2106@item :remote @var{remote-address}
2107For a connection, @var{remote-address} is the address to connect to.
2108It overrides @var{family}, @var{host} and @var{service}, and you
2109may as well not specify them.
2110
2111For a datagram server, @var{remote-address} specifies the initial
2112setting of the remote datagram address.
2113
2114The format of @var{local-address} or @var{remote-address} depends on
2115the address family:
2116
2117@itemize -
2118@item
2119An IPv4 address is represented as a five-element vector of four 8-bit
2120integers and one 16-bit integer
2121@code{[@var{a} @var{b} @var{c} @var{d} @var{p}]} corresponding to
2122numeric IPv4 address @var{a}.@var{b}.@var{c}.@var{d} and port number
2123@var{p}.
2124
2125@item
2126An IPv6 address is represented as a nine-element vector of 16-bit
2127integers @code{[@var{a} @var{b} @var{c} @var{d} @var{e} @var{f}
2128@var{g} @var{h} @var{p}]} corresponding to numeric IPv6 address
2129@var{a}:@var{b}:@var{c}:@var{d}:@var{e}:@var{f}:@var{g}:@var{h} and
2130port number @var{p}.
2131
2132@item
2133A local address is represented as a string which specifies the address
2134in the local address space.
2135
2136@item
2137An ``unsupported family'' address is represented by a cons
2138@code{(@var{f} . @var{av})}, where @var{f} is the family number and
2139@var{av} is a vector specifying the socket address using one element
2140per address data byte. Do not rely on this format in portable code,
2141as it may depend on implementation defined constants, data sizes, and
2142data structure alignment.
2143@end itemize
2144
2145@item :nowait @var{bool}
2146If @var{bool} is non-@code{nil} for a stream connection, return
2147without waiting for the connection to complete. When the connection
2148succeeds or fails, Emacs will call the sentinel function, with a
2149second argument matching @code{"open"} (if successful) or
2150@code{"failed"}. The default is to block, so that
2151@code{make-network-process} does not return until the connection
2152has succeeded or failed.
2153
2154@item :stop @var{stopped}
2155Start the network connection or server in the `stopped' state if
2156@var{stopped} is non-@code{nil}.
2157
2158@item :buffer @var{buffer}
2159Use @var{buffer} as the process buffer.
2160
2161@item :coding @var{coding}
2162Use @var{coding} as the coding system for this process. To specify
2163different coding systems for decoding data from the connection and for
2164encoding data sent to it, specify @code{(@var{decoding} .
2165@var{encoding})} for @var{coding}.
2166
2167If you don't specify this keyword at all, the default
2168is to determine the coding systems from the data.
2169
2170@item :noquery @var{query-flag}
2171Initialize the process query flag to @var{query-flag}.
2172@xref{Query Before Exit}.
2173
2174@item :filter @var{filter}
2175Initialize the process filter to @var{filter}.
2176
b8d4c8d0
GM
2177@item :sentinel @var{sentinel}
2178Initialize the process sentinel to @var{sentinel}.
2179
2180@item :log @var{log}
2181Initialize the log function of a server process to @var{log}. The log
2182function is called each time the server accepts a network connection
2183from a client. The arguments passed to the log function are
2184@var{server}, @var{connection}, and @var{message}, where @var{server}
2185is the server process, @var{connection} is the new process for the
2186connection, and @var{message} is a string describing what has
2187happened.
2188
2189@item :plist @var{plist}
2190Initialize the process plist to @var{plist}.
2191@end table
2192
2193The original argument list, modified with the actual connection
2194information, is available via the @code{process-contact} function.
2195@end defun
2196
2197@node Network Options
2198@subsection Network Options
2199
2200 The following network options can be specified when you create a
2201network process. Except for @code{:reuseaddr}, you can also set or
2202modify these options later, using @code{set-network-process-option}.
2203
2204 For a server process, the options specified with
2205@code{make-network-process} are not inherited by the client
2206connections, so you will need to set the necessary options for each
2207child connection as it is created.
2208
2209@table @asis
2210@item :bindtodevice @var{device-name}
2211If @var{device-name} is a non-empty string identifying a network
2212interface name (see @code{network-interface-list}), only handle
2213packets received on that interface. If @var{device-name} is @code{nil}
2214(the default), handle packets received on any interface.
2215
2216Using this option may require special privileges on some systems.
2217
2218@item :broadcast @var{broadcast-flag}
2219If @var{broadcast-flag} is non-@code{nil} for a datagram process, the
2220process will receive datagram packet sent to a broadcast address, and
2221be able to send packets to a broadcast address. Ignored for a stream
2222connection.
2223
2224@item :dontroute @var{dontroute-flag}
2225If @var{dontroute-flag} is non-@code{nil}, the process can only send
2226to hosts on the same network as the local host.
2227
2228@item :keepalive @var{keepalive-flag}
2229If @var{keepalive-flag} is non-@code{nil} for a stream connection,
2230enable exchange of low-level keep-alive messages.
2231
2232@item :linger @var{linger-arg}
2233If @var{linger-arg} is non-@code{nil}, wait for successful
2234transmission of all queued packets on the connection before it is
2235deleted (see @code{delete-process}). If @var{linger-arg} is an
2236integer, it specifies the maximum time in seconds to wait for queued
2237packets to be sent before closing the connection. Default is
2238@code{nil} which means to discard unsent queued packets when the
2239process is deleted.
2240
2241@item :oobinline @var{oobinline-flag}
2242If @var{oobinline-flag} is non-@code{nil} for a stream connection,
2243receive out-of-band data in the normal data stream. Otherwise, ignore
2244out-of-band data.
2245
2246@item :priority @var{priority}
2247Set the priority for packets sent on this connection to the integer
2248@var{priority}. The interpretation of this number is protocol
2249specific, such as setting the TOS (type of service) field on IP
2250packets sent on this connection. It may also have system dependent
2251effects, such as selecting a specific output queue on the network
2252interface.
2253
2254@item :reuseaddr @var{reuseaddr-flag}
2255If @var{reuseaddr-flag} is non-@code{nil} (the default) for a stream
2256server process, allow this server to reuse a specific port number (see
2257@code{:service}) unless another process on this host is already
2258listening on that port. If @var{reuseaddr-flag} is @code{nil}, there
2259may be a period of time after the last use of that port (by any
2260process on the host), where it is not possible to make a new server on
2261that port.
2262@end table
2263
106e6894 2264@defun set-network-process-option process option value &optional no-error
b8d4c8d0
GM
2265This function sets or modifies a network option for network process
2266@var{process}. See @code{make-network-process} for details of options
106e6894
CY
2267@var{option} and their corresponding values @var{value}. If
2268@var{no-error} is non-@code{nil}, this function returns @code{nil}
2269instead of signaling an error if @var{option} is not a supported
2270option. If the function successfully completes, it returns @code{t}.
b8d4c8d0
GM
2271
2272The current setting of an option is available via the
2273@code{process-contact} function.
2274@end defun
2275
2276@node Network Feature Testing
2277@subsection Testing Availability of Network Features
2278
2279 To test for the availability of a given network feature, use
2280@code{featurep} like this:
2281
2282@example
2283(featurep 'make-network-process '(@var{keyword} @var{value}))
2284@end example
2285
2286@noindent
2287The result of the first form is @code{t} if it works to specify
2288@var{keyword} with value @var{value} in @code{make-network-process}.
2289The result of the second form is @code{t} if @var{keyword} is
2290supported by @code{make-network-process}. Here are some of the
2291@var{keyword}---@var{value} pairs you can test in
2292this way.
2293
2294@table @code
2295@item (:nowait t)
2296Non-@code{nil} if non-blocking connect is supported.
2297@item (:type datagram)
2298Non-@code{nil} if datagrams are supported.
2299@item (:family local)
2300Non-@code{nil} if local (a.k.a.@: ``UNIX domain'') sockets are supported.
2301@item (:family ipv6)
2302Non-@code{nil} if IPv6 is supported.
2303@item (:service t)
2304Non-@code{nil} if the system can select the port for a server.
2305@end table
2306
2307 To test for the availability of a given network option, use
2308@code{featurep} like this:
2309
2310@example
2311(featurep 'make-network-process '@var{keyword})
2312@end example
2313
2314@noindent
2315Here are some of the options you can test in this way.
2316
2317@table @code
2318@item :bindtodevice
2319@itemx :broadcast
2320@itemx :dontroute
2321@itemx :keepalive
2322@itemx :linger
2323@itemx :oobinline
2324@itemx :priority
2325@itemx :reuseaddr
2326That particular network option is supported by
2327@code{make-network-process} and @code{set-network-process-option}.
2328@end table
2329
2330@node Misc Network
2331@section Misc Network Facilities
2332
2333 These additional functions are useful for creating and operating
305a7ef2
EZ
2334on network connections. Note that they are supported only on some
2335systems.
b8d4c8d0
GM
2336
2337@defun network-interface-list
2338This function returns a list describing the network interfaces
2339of the machine you are using. The value is an alist whose
2340elements have the form @code{(@var{name} . @var{address})}.
2341@var{address} has the same form as the @var{local-address}
2342and @var{remote-address} arguments to @code{make-network-process}.
2343@end defun
2344
2345@defun network-interface-info ifname
2346This function returns information about the network interface named
2347@var{ifname}. The value is a list of the form
2348@code{(@var{addr} @var{bcast} @var{netmask} @var{hwaddr} @var{flags})}.
2349
2350@table @var
2351@item addr
2352The Internet protocol address.
2353@item bcast
2354The broadcast address.
2355@item netmask
2356The network mask.
2357@item hwaddr
2358The layer 2 address (Ethernet MAC address, for instance).
2359@item flags
2360The current flags of the interface.
2361@end table
2362@end defun
2363
2364@defun format-network-address address &optional omit-port
2365This function converts the Lisp representation of a network address to
2366a string.
2367
2368A five-element vector @code{[@var{a} @var{b} @var{c} @var{d} @var{p}]}
2369represents an IPv4 address @var{a}.@var{b}.@var{c}.@var{d} and port
2370number @var{p}. @code{format-network-address} converts that to the
2371string @code{"@var{a}.@var{b}.@var{c}.@var{d}:@var{p}"}.
2372
2373A nine-element vector @code{[@var{a} @var{b} @var{c} @var{d} @var{e}
2374@var{f} @var{g} @var{h} @var{p}]} represents an IPv6 address along
2375with a port number. @code{format-network-address} converts that to
2376the string
2377@code{"[@var{a}:@var{b}:@var{c}:@var{d}:@var{e}:@var{f}:@var{g}:@var{h}]:@var{p}"}.
2378
2379If the vector does not include the port number, @var{p}, or if
2380@var{omit-port} is non-@code{nil}, the result does not include the
2381@code{:@var{p}} suffix.
2382@end defun
2383
c73e02fa
GM
2384@node Serial Ports
2385@section Communicating with Serial Ports
2386@cindex @file{/dev/tty}
2387@cindex @file{COM1}
545c2782 2388@cindex serial connections
c73e02fa
GM
2389
2390 Emacs can communicate with serial ports. For interactive use,
2391@kbd{M-x serial-term} opens a terminal window. In a Lisp program,
2392@code{make-serial-process} creates a process object.
2393
2394 The serial port can be configured at run-time, without having to
2395close and re-open it. The function @code{serial-process-configure}
2396lets you change the speed, bytesize, and other parameters. In a
2397terminal window created by @code{serial-term}, you can click on the
2398mode line for configuration.
2399
2400 A serial connection is represented by a process object which can be
2401used similar to a subprocess or network process. You can send and
2402receive data and configure the serial port. A serial process object
80e6b6df
EZ
2403has no process ID, you can't send signals to it, and the status codes
2404are different from other types of processes.
c73e02fa
GM
2405@code{delete-process} on the process object or @code{kill-buffer} on
2406the process buffer close the connection, but this does not affect the
2407device connected to the serial port.
2408
2409 The function @code{process-type} returns the symbol @code{serial}
80e6b6df 2410for a process object representing a serial port connection.
c73e02fa
GM
2411
2412 Serial ports are available on GNU/Linux, Unix, and Windows systems.
2413
80e6b6df 2414@deffn Command serial-term port speed
c73e02fa 2415Start a terminal-emulator for a serial port in a new buffer.
80e6b6df
EZ
2416@var{port} is the name of the serial port to which to connect. For
2417example, this could be @file{/dev/ttyS0} on Unix. On Windows, this
2418could be @file{COM1}, or @file{\\.\COM10} (double the backslashes in
2419Lisp strings).
c73e02fa
GM
2420
2421@var{speed} is the speed of the serial port in bits per second. 9600
80e6b6df
EZ
2422is a common value. The buffer is in Term mode; see @ref{Term Mode,,,
2423emacs, The GNU Emacs Manual}, for the commands to use in that buffer.
2424You can change the speed and the configuration in the mode line menu.
2425@end deffn
c73e02fa
GM
2426
2427@defun make-serial-process &rest args
80e6b6df
EZ
2428This function creates a process and a buffer. Arguments are specified
2429as keyword/argument pairs. Here's the list of the meaningful keywords:
c73e02fa
GM
2430
2431@table @code
80e6b6df
EZ
2432@item :port @var{port}@r{ (mandatory)}
2433This is the name of the serial port. On Unix and GNU systems, this is
2434a file name such as @file{/dev/ttyS0}. On Windows, this could be
2435@file{COM1}, or @file{\\.\COM10} for ports higher than @file{COM9}
2436(double the backslashes in Lisp strings).
2437
2438@item :speed @var{speed}@r{ (mandatory)}
2439The speed of the serial port in bits per second. This function calls
2440@code{serial-process-configure} to handle the speed.
2441
2442@item :name @var{name}
2443The name of the process. If @var{name} is not given, @var{port} will
2444serve as the process name as well.
2445
2446@item :buffer @var{buffer}
2447The buffer to associate with the process. The value could be either a
2448buffer or a string that names a buffer. Process output goes at the
2449end of that buffer, unless you specify an output stream or filter
2450function to handle the output. If @var{buffer} is not given, the
2451process buffer's name is taken from the value of the @code{:name}
2452keyword.
2453
2454@item :coding @var{coding}
c73e02fa
GM
2455If @var{coding} is a symbol, it specifies the coding system used for
2456both reading and writing for this process. If @var{coding} is a cons
2457@code{(decoding . encoding)}, @var{decoding} is used for reading, and
80e6b6df
EZ
2458@var{encoding} is used for writing. If not specified, the default is
2459to determine the coding systems from data itself.
c73e02fa 2460
80e6b6df
EZ
2461@item :noquery @var{query-flag}
2462Initialize the process query flag to @var{query-flag}. @xref{Query
2463Before Exit}. The flags defaults to @code{nil} if unspecified.
c73e02fa 2464
80e6b6df 2465@item :stop @var{bool}
c73e02fa
GM
2466Start process in the @code{stopped} state if @var{bool} is
2467non-@code{nil}. In the stopped state, a serial process does not
2468accept incoming data, but you can send outgoing data. The stopped
2469state is cleared by @code{continue-process} and set by
2470@code{stop-process}.
2471
80e6b6df 2472@item :filter @var{filter}
c73e02fa
GM
2473Install @var{filter} as the process filter.
2474
80e6b6df 2475@item :sentinel @var{sentinel}
c73e02fa
GM
2476Install @var{sentinel} as the process sentinel.
2477
80e6b6df 2478@item :plist @var{plist}
c73e02fa
GM
2479Install @var{plist} as the initial plist of the process.
2480
2481@item :speed
2482@itemx :bytesize
2483@itemx :parity
2484@itemx :stopbits
2485@itemx :flowcontrol
2486These arguments are handled by @code{serial-process-configure}, which
2487is called by @code{make-serial-process}.
2488@end table
2489
2490The original argument list, possibly modified by later configuration,
2491is available via the function @code{process-contact}.
2492
2493Examples:
2494
2495@example
2496(make-serial-process :port "/dev/ttyS0" :speed 9600)
2497
2498(make-serial-process :port "COM1" :speed 115200 :stopbits 2)
2499
80e6b6df
EZ
2500(make-serial-process :port "\\\\.\\COM13" :speed 1200
2501 :bytesize 7 :parity 'odd)
c73e02fa 2502
57eedec1
GM
2503(make-serial-process :port "/dev/tty.BlueConsole-SPP-1"
2504 :speed nil)
c73e02fa
GM
2505@end example
2506@end defun
2507
2508@defun serial-process-configure &rest args
80e6b6df
EZ
2509@cindex baud, in serial connections
2510@cindex bytesize, in serial connections
2511@cindex parity, in serial connections
2512@cindex stopbits, in serial connections
2513@cindex flowcontrol, in serial connections
2514
2515This functions configures a serial port connection. Arguments are
2516specified as keyword/argument pairs. Attributes that are not given
2517are re-initialized from the process's current configuration (available
2518via the function @code{process-contact}) or set to reasonable default
2519values. The following arguments are defined:
c73e02fa
GM
2520
2521@table @code
80e6b6df
EZ
2522@item :process @var{process}
2523@itemx :name @var{name}
2524@itemx :buffer @var{buffer}
2525@itemx :port @var{port}
c73e02fa
GM
2526Any of these arguments can be given to identify the process that is to
2527be configured. If none of these arguments is given, the current
2528buffer's process is used.
2529
2530@item :speed @var{speed}
545c2782
EZ
2531The speed of the serial port in bits per second, a.k.a.@: @dfn{baud
2532rate}. The value can be any number, but most serial ports work only
2533at a few defined values between 1200 and 115200, with 9600 being the
2534most common value. If @var{speed} is @code{nil}, the function ignores
2535all other arguments and does not configure the port. This may be
2536useful for special serial ports such as Bluetooth-to-serial converters
2537which can only be configured through AT commands sent through the
538395d9
EZ
2538connection. The value of @code{nil} for @var{speed} is valid only for
2539connections that were already opened by a previous call to
80e6b6df 2540@code{make-serial-process} or @code{serial-term}.
c73e02fa
GM
2541
2542@item :bytesize @var{bytesize}
80e6b6df
EZ
2543The number of bits per byte, which can be 7 or 8. If @var{bytesize}
2544is not given or @code{nil}, it defaults to 8.
c73e02fa
GM
2545
2546@item :parity @var{parity}
80e6b6df 2547The value can be @code{nil} (don't use parity), the symbol
c73e02fa 2548@code{odd} (use odd parity), or the symbol @code{even} (use even
80e6b6df 2549parity). If @var{parity} is not given, it defaults to no parity.
c73e02fa
GM
2550
2551@item :stopbits @var{stopbits}
80e6b6df
EZ
2552The number of stopbits used to terminate a transmission
2553of each byte. @var{stopbits} can be 1 or 2. If @var{stopbits} is not
2554given or @code{nil}, it defaults to 1.
c73e02fa
GM
2555
2556@item :flowcontrol @var{flowcontrol}
80e6b6df
EZ
2557The type of flow control to use for this connection, which is either
2558@code{nil} (don't use flow control), the symbol @code{hw} (use RTS/CTS
2559hardware flow control), or the symbol @code{sw} (use XON/XOFF software
2560flow control). If @var{flowcontrol} is not given, it defaults to no
2561flow control.
c73e02fa
GM
2562@end table
2563
2564@code{serial-process-configure} is called by @code{make-serial-process} for the
2565initial configuration of the serial port.
2566
2567Examples:
2568
2569@example
2570(serial-process-configure :process "/dev/ttyS0" :speed 1200)
2571
57eedec1
GM
2572(serial-process-configure :buffer "COM1" :stopbits 1
2573 :parity 'odd :flowcontrol 'hw)
c73e02fa
GM
2574
2575(serial-process-configure :port "\\\\.\\COM13" :bytesize 7)
2576@end example
2577@end defun
2578
b8d4c8d0
GM
2579@node Byte Packing
2580@section Packing and Unpacking Byte Arrays
2581@cindex byte packing and unpacking
2582
2583 This section describes how to pack and unpack arrays of bytes,
2584usually for binary network protocols. These functions convert byte arrays
2585to alists, and vice versa. The byte array can be represented as a
2586unibyte string or as a vector of integers, while the alist associates
2587symbols either with fixed-size objects or with recursive sub-alists.
2588
2589@cindex serializing
2590@cindex deserializing
2591@cindex packing
2592@cindex unpacking
2593 Conversion from byte arrays to nested alists is also known as
2594@dfn{deserializing} or @dfn{unpacking}, while going in the opposite
2595direction is also known as @dfn{serializing} or @dfn{packing}.
2596
2597@menu
2598* Bindat Spec:: Describing data layout.
2599* Bindat Functions:: Doing the unpacking and packing.
2600* Bindat Examples:: Samples of what bindat.el can do for you!
2601@end menu
2602
2603@node Bindat Spec
2604@subsection Describing Data Layout
2605
2606 To control unpacking and packing, you write a @dfn{data layout
2607specification}, a special nested list describing named and typed
2608@dfn{fields}. This specification controls length of each field to be
2609processed, and how to pack or unpack it. We normally keep bindat specs
2610in variables whose names end in @samp{-bindat-spec}; that kind of name
2611is automatically recognized as ``risky.''
2612
2613@cindex endianness
2614@cindex big endian
2615@cindex little endian
2616@cindex network byte ordering
2617 A field's @dfn{type} describes the size (in bytes) of the object
2618that the field represents and, in the case of multibyte fields, how
2619the bytes are ordered within the field. The two possible orderings
2620are ``big endian'' (also known as ``network byte ordering'') and
2621``little endian.'' For instance, the number @code{#x23cd} (decimal
26229165) in big endian would be the two bytes @code{#x23} @code{#xcd};
2623and in little endian, @code{#xcd} @code{#x23}. Here are the possible
2624type values:
2625
2626@table @code
2627@item u8
2628@itemx byte
2629Unsigned byte, with length 1.
2630
2631@item u16
2632@itemx word
2633@itemx short
2634Unsigned integer in network byte order, with length 2.
2635
2636@item u24
2637Unsigned integer in network byte order, with length 3.
2638
2639@item u32
2640@itemx dword
2641@itemx long
2642Unsigned integer in network byte order, with length 4.
2643Note: These values may be limited by Emacs' integer implementation limits.
2644
2645@item u16r
2646@itemx u24r
2647@itemx u32r
2648Unsigned integer in little endian order, with length 2, 3 and 4, respectively.
2649
2650@item str @var{len}
2651String of length @var{len}.
2652
2653@item strz @var{len}
2654Zero-terminated string, in a fixed-size field with length @var{len}.
2655
2656@item vec @var{len} [@var{type}]
2657Vector of @var{len} elements of type @var{type}, or bytes if not
2658@var{type} is specified.
2659The @var{type} is any of the simple types above, or another vector
2660specified as a list @code{(vec @var{len} [@var{type}])}.
2661
2662@item ip
2663Four-byte vector representing an Internet address. For example:
2664@code{[127 0 0 1]} for localhost.
2665
2666@item bits @var{len}
2667List of set bits in @var{len} bytes. The bytes are taken in big
2668endian order and the bits are numbered starting with @code{8 *
2669@var{len} @minus{} 1} and ending with zero. For example: @code{bits
26702} unpacks @code{#x28} @code{#x1c} to @code{(2 3 4 11 13)} and
2671@code{#x1c} @code{#x28} to @code{(3 5 10 11 12)}.
2672
2673@item (eval @var{form})
2674@var{form} is a Lisp expression evaluated at the moment the field is
2675unpacked or packed. The result of the evaluation should be one of the
2676above-listed type specifications.
2677@end table
2678
2679For a fixed-size field, the length @var{len} is given as an integer
2680specifying the number of bytes in the field.
2681
2682When the length of a field is not fixed, it typically depends on the
2683value of a preceding field. In this case, the length @var{len} can be
2684given either as a list @code{(@var{name} ...)} identifying a
2685@dfn{field name} in the format specified for @code{bindat-get-field}
2686below, or by an expression @code{(eval @var{form})} where @var{form}
2687should evaluate to an integer, specifying the field length.
2688
2689A field specification generally has the form @code{([@var{name}]
2690@var{handler})}. The square braces indicate that @var{name} is
2691optional. (Don't use names that are symbols meaningful as type
2692specifications (above) or handler specifications (below), since that
2693would be ambiguous.) @var{name} can be a symbol or the expression
2694@code{(eval @var{form})}, in which case @var{form} should evaluate to
2695a symbol.
2696
2697@var{handler} describes how to unpack or pack the field and can be one
2698of the following:
2699
2700@table @code
2701@item @var{type}
2702Unpack/pack this field according to the type specification @var{type}.
2703
2704@item eval @var{form}
2705Evaluate @var{form}, a Lisp expression, for side-effect only. If the
2706field name is specified, the value is bound to that field name.
2707
2708@item fill @var{len}
2709Skip @var{len} bytes. In packing, this leaves them unchanged,
2710which normally means they remain zero. In unpacking, this means
2711they are ignored.
2712
2713@item align @var{len}
2714Skip to the next multiple of @var{len} bytes.
2715
2716@item struct @var{spec-name}
2717Process @var{spec-name} as a sub-specification. This describes a
2718structure nested within another structure.
2719
2720@item union @var{form} (@var{tag} @var{spec})@dots{}
2721@c ??? I don't see how one would actually use this.
2722@c ??? what kind of expression would be useful for @var{form}?
2723Evaluate @var{form}, a Lisp expression, find the first @var{tag}
2724that matches it, and process its associated data layout specification
2725@var{spec}. Matching can occur in one of three ways:
2726
2727@itemize
2728@item
2729If a @var{tag} has the form @code{(eval @var{expr})}, evaluate
2730@var{expr} with the variable @code{tag} dynamically bound to the value
2731of @var{form}. A non-@code{nil} result indicates a match.
2732
2733@item
2734@var{tag} matches if it is @code{equal} to the value of @var{form}.
2735
2736@item
2737@var{tag} matches unconditionally if it is @code{t}.
2738@end itemize
2739
2740@item repeat @var{count} @var{field-specs}@dots{}
2741Process the @var{field-specs} recursively, in order, then repeat
2742starting from the first one, processing all the specs @var{count}
2743times overall. The @var{count} is given using the same formats as a
2744field length---if an @code{eval} form is used, it is evaluated just once.
2745For correct operation, each spec in @var{field-specs} must include a name.
2746@end table
2747
2748For the @code{(eval @var{form})} forms used in a bindat specification,
2749the @var{form} can access and update these dynamically bound variables
2750during evaluation:
2751
2752@table @code
2753@item last
2754Value of the last field processed.
2755
2756@item bindat-raw
2757The data as a byte array.
2758
2759@item bindat-idx
2760Current index (within @code{bindat-raw}) for unpacking or packing.
2761
2762@item struct
2763The alist containing the structured data that have been unpacked so
2764far, or the entire structure being packed. You can use
2765@code{bindat-get-field} to access specific fields of this structure.
2766
2767@item count
2768@itemx index
2769Inside a @code{repeat} block, these contain the maximum number of
2770repetitions (as specified by the @var{count} parameter), and the
2771current repetition number (counting from 0). Setting @code{count} to
2772zero will terminate the inner-most repeat block after the current
2773repetition has completed.
2774@end table
2775
2776@node Bindat Functions
2777@subsection Functions to Unpack and Pack Bytes
2778
2779 In the following documentation, @var{spec} refers to a data layout
2780specification, @code{bindat-raw} to a byte array, and @var{struct} to an
2781alist representing unpacked field data.
2782
2783@defun bindat-unpack spec bindat-raw &optional bindat-idx
2784This function unpacks data from the unibyte string or byte
2785array @code{bindat-raw}
2786according to @var{spec}. Normally this starts unpacking at the
2787beginning of the byte array, but if @var{bindat-idx} is non-@code{nil}, it
2788specifies a zero-based starting position to use instead.
2789
2790The value is an alist or nested alist in which each element describes
2791one unpacked field.
2792@end defun
2793
2794@defun bindat-get-field struct &rest name
2795This function selects a field's data from the nested alist
2796@var{struct}. Usually @var{struct} was returned by
2797@code{bindat-unpack}. If @var{name} corresponds to just one argument,
2798that means to extract a top-level field value. Multiple @var{name}
2799arguments specify repeated lookup of sub-structures. An integer name
2800acts as an array index.
2801
2802For example, if @var{name} is @code{(a b 2 c)}, that means to find
2803field @code{c} in the third element of subfield @code{b} of field
2804@code{a}. (This corresponds to @code{struct.a.b[2].c} in C.)
2805@end defun
2806
2807 Although packing and unpacking operations change the organization of
2808data (in memory), they preserve the data's @dfn{total length}, which is
2809the sum of all the fields' lengths, in bytes. This value is not
2810generally inherent in either the specification or alist alone; instead,
2811both pieces of information contribute to its calculation. Likewise, the
2812length of a string or array being unpacked may be longer than the data's
2813total length as described by the specification.
2814
2815@defun bindat-length spec struct
2816This function returns the total length of the data in @var{struct},
2817according to @var{spec}.
2818@end defun
2819
2820@defun bindat-pack spec struct &optional bindat-raw bindat-idx
2821This function returns a byte array packed according to @var{spec} from
2822the data in the alist @var{struct}. Normally it creates and fills a
2823new byte array starting at the beginning. However, if @var{bindat-raw}
2824is non-@code{nil}, it specifies a pre-allocated unibyte string or vector to
2825pack into. If @var{bindat-idx} is non-@code{nil}, it specifies the starting
2826offset for packing into @code{bindat-raw}.
2827
2828When pre-allocating, you should make sure @code{(length @var{bindat-raw})}
2829meets or exceeds the total length to avoid an out-of-range error.
2830@end defun
2831
2832@defun bindat-ip-to-string ip
2833Convert the Internet address vector @var{ip} to a string in the usual
2834dotted notation.
2835
2836@example
2837(bindat-ip-to-string [127 0 0 1])
2838 @result{} "127.0.0.1"
2839@end example
2840@end defun
2841
2842@node Bindat Examples
2843@subsection Examples of Byte Unpacking and Packing
2844
2845 Here is a complete example of byte unpacking and packing:
2846
2847@lisp
2848(defvar fcookie-index-spec
2849 '((:version u32)
2850 (:count u32)
2851 (:longest u32)
2852 (:shortest u32)
2853 (:flags u32)
2854 (:delim u8)
2855 (:ignored fill 3)
2856 (:offset repeat (:count)
2857 (:foo u32)))
2858 "Description of a fortune cookie index file's contents.")
2859
2860(defun fcookie (cookies &optional index)
2861 "Display a random fortune cookie from file COOKIES.
2862Optional second arg INDEX specifies the associated index
2863filename, which is by default constructed by appending
2864\".dat\" to COOKIES. Display cookie text in possibly
2865new buffer \"*Fortune Cookie: BASENAME*\" where BASENAME
2866is COOKIES without the directory part."
2867 (interactive "fCookies file: ")
2868 (let* ((info (with-temp-buffer
2869 (insert-file-contents-literally
2870 (or index (concat cookies ".dat")))
2871 (bindat-unpack fcookie-index-spec
2872 (buffer-string))))
2873 (sel (random (bindat-get-field info :count)))
2874 (beg (cdar (bindat-get-field info :offset sel)))
2875 (end (or (cdar (bindat-get-field info
2876 :offset (1+ sel)))
2877 (nth 7 (file-attributes cookies)))))
2878 (switch-to-buffer
2879 (get-buffer-create
2880 (format "*Fortune Cookie: %s*"
2881 (file-name-nondirectory cookies))))
2882 (erase-buffer)
2883 (insert-file-contents-literally
2884 cookies nil beg (- end 3))))
2885
2886(defun fcookie-create-index (cookies &optional index delim)
2887 "Scan file COOKIES, and write out its index file.
2888Optional second arg INDEX specifies the index filename,
2889which is by default constructed by appending \".dat\" to
2890COOKIES. Optional third arg DELIM specifies the unibyte
2891character which, when found on a line of its own in
2892COOKIES, indicates the border between entries."
2893 (interactive "fCookies file: ")
2894 (setq delim (or delim ?%))
2895 (let ((delim-line (format "\n%c\n" delim))
2896 (count 0)
2897 (max 0)
2898 min p q len offsets)
2899 (unless (= 3 (string-bytes delim-line))
2900 (error "Delimiter cannot be represented in one byte"))
2901 (with-temp-buffer
2902 (insert-file-contents-literally cookies)
2903 (while (and (setq p (point))
2904 (search-forward delim-line (point-max) t)
2905 (setq len (- (point) 3 p)))
2906 (setq count (1+ count)
2907 max (max max len)
2908 min (min (or min max) len)
2909 offsets (cons (1- p) offsets))))
2910 (with-temp-buffer
2911 (set-buffer-multibyte nil)
2912 (insert
2913 (bindat-pack
2914 fcookie-index-spec
2915 `((:version . 2)
2916 (:count . ,count)
2917 (:longest . ,max)
2918 (:shortest . ,min)
2919 (:flags . 0)
2920 (:delim . ,delim)
2921 (:offset . ,(mapcar (lambda (o)
2922 (list (cons :foo o)))
2923 (nreverse offsets))))))
2924 (let ((coding-system-for-write 'raw-text-unix))
2925 (write-file (or index (concat cookies ".dat")))))))
2926@end lisp
2927
2928Following is an example of defining and unpacking a complex structure.
2929Consider the following C structures:
2930
2931@example
2932struct header @{
2933 unsigned long dest_ip;
2934 unsigned long src_ip;
2935 unsigned short dest_port;
2936 unsigned short src_port;
2937@};
2938
2939struct data @{
2940 unsigned char type;
2941 unsigned char opcode;
2942 unsigned short length; /* In network byte order */
2943 unsigned char id[8]; /* null-terminated string */
2944 unsigned char data[/* (length + 3) & ~3 */];
2945@};
2946
2947struct packet @{
2948 struct header header;
2949 unsigned long counters[2]; /* In little endian order */
2950 unsigned char items;
2951 unsigned char filler[3];
2952 struct data item[/* items */];
2953
2954@};
2955@end example
2956
2957The corresponding data layout specification:
2958
2959@lisp
2960(setq header-spec
2961 '((dest-ip ip)
2962 (src-ip ip)
2963 (dest-port u16)
2964 (src-port u16)))
2965
2966(setq data-spec
2967 '((type u8)
2968 (opcode u8)
2969 (length u16) ;; network byte order
2970 (id strz 8)
2971 (data vec (length))
2972 (align 4)))
2973
2974(setq packet-spec
2975 '((header struct header-spec)
2976 (counters vec 2 u32r) ;; little endian order
2977 (items u8)
2978 (fill 3)
2979 (item repeat (items)
2980 (struct data-spec))))
2981@end lisp
2982
2983A binary data representation:
2984
2985@lisp
2986(setq binary-data
2987 [ 192 168 1 100 192 168 1 101 01 28 21 32
2988 160 134 1 0 5 1 0 0 2 0 0 0
2989 2 3 0 5 ?A ?B ?C ?D ?E ?F 0 0 1 2 3 4 5 0 0 0
2990 1 4 0 7 ?B ?C ?D ?E ?F ?G 0 0 6 7 8 9 10 11 12 0 ])
2991@end lisp
2992
2993The corresponding decoded structure:
2994
2995@lisp
2996(setq decoded (bindat-unpack packet-spec binary-data))
2997 @result{}
2998((header
2999 (dest-ip . [192 168 1 100])
3000 (src-ip . [192 168 1 101])
3001 (dest-port . 284)
3002 (src-port . 5408))
3003 (counters . [100000 261])
3004 (items . 2)
3005 (item ((data . [1 2 3 4 5])
3006 (id . "ABCDEF")
3007 (length . 5)
3008 (opcode . 3)
3009 (type . 2))
3010 ((data . [6 7 8 9 10 11 12])
3011 (id . "BCDEFG")
3012 (length . 7)
3013 (opcode . 4)
3014 (type . 1))))
3015@end lisp
3016
3017Fetching data from this structure:
3018
3019@lisp
3020(bindat-get-field decoded 'item 1 'id)
3021 @result{} "BCDEFG"
3022@end lisp
3023
3024@ignore
3025 arch-tag: ba9da253-e65f-4e7f-b727-08fba0a1df7a
3026@end ignore