Digital PDFs

AA-MF08A-TE

May 1988

432 pages

Original

90MB

view download

OCR Version

57MB

view download

Document:	ULTRIX-32 Supplementary Documents System Manager
Order Number:	AA-MF08A-TE
Revision:	0
Pages:	432
Original Filename:

OCR Text

ULTRIX-32 Supplementary Documents

System Manager
Order No. AA-MFO8A-TE

ULTRIX-32 Operating System, Version 3.0

Digital Equipment Corporation

The information in this document is subject to change without notice and
should not be construed as a commitment by Digital Equipment Corporation.
Digital Equipment Corporation assumes no responsibility for any errors that
may appear in this document.

The software described in this document is furnished under a license and may
be used or copied only in accordance with the terms of such license.
No responsibility is assumed for the use or reliability of software on equipment that is not supplied by DIGITAL or its affiliated companies.

The following are trademarks of Digital Equipment Corporation:
DEC
DECUS
MASSBUS
PDP

ULTRIX-11

ULTRIX

ULTRIX-32
UNIBUS
VAX
VMS

éTflgfluan "

UNIX is a trademark of AT&T Bell Laboratories.

Information herein is derived from copyrighted material as permitted under a
license agreement with AT&T Bell Laboratories.

This software and documentation is based in part on the Fourth Berkeley
Software Distribution under license from the Regents of the University of
California. We acknowledge the Electrical Engineering and Computer Science
Departments at the Berkeley Campus of the University of California for their
role in its development.

iii

This software and documentation is based in part on the Fourth Berkeley Software Distribution under
license from The Regents of the University of California. Digital Equipment Corporation acknowledges
the following individuals and institutions for their role in its development:

"The UNIX Time-Sharing System”:

reprinted by permission. This is a revised version of an article that appeared in Communications of the

ACM, 17, No. 7 (July 1974), pp. 365-375. That article was a revised version of a paper presnted at the
Fourth ACM Symposium on Operating Systems Principles, IBM Thomas J. Watson Research Center,
Yorktown Heights, New York, October 15-17, 1973.

Acknowledgements:

for their help and support,

R.H. Canaday, R. Morris, M.D. Mcllroy, and J.F. Ossanna.

” Advanced Editing on UNIX” acknowledgement: Ted Dolotta for his ideas and assistance.
” An Introduction to the UNIX Shell” acknowledgements: Dennis Ritchie, John Mashey and Joe Maranzano for their help and support.

"LEARN - Computer-Aided Instruction on UNIX” acknowledgements: for their help and support, M.E.
Bittrich, J.L. Blue, S.I. Feldman, P.A. Fox, M.J. McAlpin, E.Z. Rothkopf, Don Jackowski, and Tom
Plum.

"A System for Typesetting Mathematics” acknowledgements: J.F. Ossanna, A.V. Aho, and S.C. Johnson,
for their ideas and assistance.

A TROFF Tutorial” acknowledgements: J. F. Ossanna, Jim Blinn, Ted Dolotta, Doug Mcllroy, Mike
Lesk and Joel Sturman, for their help and support.

The document "The C Programming Language - Reference Manual” is reprinted, with minor changes,

from "The C Programming Language, by Brian W. Kernighan and Dennis M. Ritchie, Prentice-Hall,
Inc., 1978.

"Make - A Program for Maintaining Computer Programs” ackowledgements:

S.C.

Johnson, and H.

Gajewska, for their ideas and assistance.

"YACC: Yet Another Compiler-Compiler” acknowledgements: B.W. Kernighan, P.J. Plauger, S.I. Feldman, C. Imagna, M.E. Lesk, A. Snyder, C.B. Haley, D.M. Ritchie, M.O. Harris and Al Aho, for their
ideas and assistance.

"Lex - A Lexical Analyzer Generator” acknowledgements: S.C. Johnson, A.V. Aho, and Eric Schmidt, for
their help as originators of much of Lex, as well as debuggers of it.

The document "RATFOR - A Preprocessor for a Rational Fortran” is a revised and expanded version of

the one published in Software - Practice and Experience, October 1975. The Ratfor described here is
the one in use on UNIX and GCOS at A T & T Bell Laboratories. Acknowledgements: Dennis Ritchie,
and Stuart Feldman, for their ideas and assistance.

"The M4 Macro Processor” acknowledgements: Rick Becker, John Chambers, Doug Mcllroy, and Jim
Weythman, for the help and support.

"BC - An Arbitrary Precision Desk-Calculator Language” acknowledgement: The compiler is written in
YACC,; its original version was written by S.C. Johnson.

”A Dial-Up Network of UNIX TM Systems” acknowledgements: G.L. Chesson, A.S.

Cohen, J. Lions,

and P.F. Long, for their suggestions and assistance.

Copyright © 1979, 1980 Regents of the University of California. Permission to copy these documents or
any portion thereof as necessary for licensed use of the software is granted to licensees of this software,
provided this copyright notice and statement of permission are included.

The document *Writing Tools - The STYLE and DICTION Programs” is copyrighted © 1979 by A T &
T Bell Laboratories. Holders of a UNIX TM/32V software license are permitted to copy this document,
or any portion of it, as necessary for licensed use of the software, provided this copyright notice and
statement of permission are included.

The document "The Programming Language EFL” is copyrighted © 1979 by A T & T Bell Laboratories.
EFL has been approved for general release, so that one may copy it subject only to the restriction of giving proper acknowledgement to A T & T Bell Laboratories.

The documents A Portable Fortran 77 Compiler” and "Fsck - The UNIX File System Check Program”
are modifications of earlier documents which are copyrighted © 1979 by A T & T Bell Laboratories.
Holders of a UNIX TM/32V software license are permitted to copy these documents, or any portion of
them, as necessary for licensed use of the software, provided this copyright notice and statement of permission are included. This manual reflects system enhancements made at Berkeley and sponsored in
part by NSF Grants MCS-7807291, MCS-8005144, and MCS-74-07644-A04; DOE Contract DE-AT0376SF00034 and Project Agreement DE-AS03-79ER10358; and by Defense Advanced Research Projects
Agency (DoD) ARPA Order No. 4031, monitored by Naval Electronics Systems Command under Contract No. N00039-80-K-0649.

"Ex Reference Manual” acknowledgements: Chuck Haley contributed greatly to the early development
of ex. Bruce Englar encouraged the redesign which led to ex version 1. Bill Joy wrote versions 1 and 2.0
through 2.7, and created the framework that users see in the present editor. Mark Horton added macros
and other features and made the editor work on a large number of terminals and UNIX systems.

?A Guide to the Dungeons of Doom” acknowledgements: Rogue was originally conceived by Glenn Wichman and Michael Toy. Ken Arnold and Michael Toy then smoothed out the user interface, and added
many new features. We would like to thank Bob Arnold, Michelle Busch, Andy Hatcher, Kipp Hickman,
Mark Horton, Daniel Jensen, Bill Joy, Joe Kalash, Steve Maurer, Marty McNary, Jan Miller, and Scott
Nelson for their ideas and assistance.

The document "The FRANZ LISP Manual” is copyrighted © 1980, 1981, 1983 by the Regents of the
University of California. (exceptions: Chapters 13, 14 (first half), 15 and 16 have separate copyrights, as
indicated. These are reproduced by permission of the copyright holders.) Permission to copy without
fee all or part of this material is granted provided that the copies are not made or distributed for direct
commercial advantage, and the copyright notice of the Regents, University of California, is given. All
rights reserved. Work reported herein was supported in part by the U.S. Department of Energy, Contract DE-AT03-76SF00034, Project Agreement DE-AS03-79ER10358, and the National Science Foundation under Grant No. MCS 7807291. MC68000 is a trademark of Motorola Semiconductor Products, Inc.
"The FRANZ LISP Manual” acknowledgements: Richard Fateman, Mike Curry, John Breedlove, Jeff
Levinsky, Bill Rowan, Tom London, Keith Sklower, Kipp Hickman, Charles Koester, Mitch Marcus,
Don Cohen, John Foderaro, and Kevin Layer.

The document "Berkeley Pascal User’s Manual” is copyrighted © 1977, 1979, 1980, 1983 by W.N. Joy,
S.L. Graham, C.B. Haley, M.K. McKusick, P.B. Kessler. The financial support of the first and second
authors’ work by the National Science Foundation under grants MCS74-07644-A04, MCS78-07291, and
MCS80-05144, and the first author’s work by an IBM Graduate Fellowship are gratefully acknowledged.

"Introduction to the f77 I/O Library” acknowledgement: Peter J. Weinberger originally wrote the 1/0/
Library at A T & T Bell Laboratories.

"Writing Papers with NROFF Using -ME”, and ”-ME Reference Manual” acknowledgements: Bob
Epstein, Bill Joy, Larry Rowe, Ricki Blau, Pamela Humphrey, and Jim Joyce, for their ideas and assistance. UNIX, NROFF, and TROFF are trademarks of A T & T Bell Laboratories.
"Refer - A Bibliography System” acknowledgements: Mike Lesk of A T & T Bell Laboratories wrote the
original refer software, including the indexing programs. Al Stanberger of the Forestry Department
wrote the first version of addbib, then called bibin. Greg Shenaut of the Linguistics Department wrote
the original versions of sortbib and roffbib.

"Screen Updating and Cursor Movement Optimization: A Library Package” acknowledgements: For
their help and support, Bill Joy, Doug Merritt, Kurt Shoens, Ken Abrams, Alan Char, Mark Horton, and
Joe Kalash.

"Disc Quotas in a UNIX Environment” acknowledgements: Sam Lefller and Kirk McKusick, for their

work on the quota code.

The current disc quota system is loosely based on a very early scheme imple-

mented at the University of New South Wales and Syndey University.
The document, "Fsck - The UNIX File System Check Program”, is a revision by Marshall Kirk
McKusick; T.J. Kowalski wrote the original paper.

For their help and support, we thank Bill Joy, Sam

- Leffler, Robert Elz, Dennis Ritchie, Robert Henry, Larry A. Wehr, and Rick B. Brandt.

Our sponsors

were the National Science Foundation under grant MCS80-05144, and the Defense Advance Research
Projects Agency (DoD) under Arpa Order No. 4031 monitored by Naval Electronic System Command

under Contract No. N00039-82-C-0235.
”A Fast File System for UNIX” acknowledgements: William N. Joy, Samuel J. Leflier, Robert S. Fabry,

Marshall Kirk McKusick, Robert Elz, Michael Powell, Peter Kessler, Rober Henry, and Dennis Ritchie.
This work was done under grants from the National Science Foundation under grant MCS80-05144, and

the Defense Advance Research Projects Agency (DoD) under ARPA No. 4031 monitored by Naval Electronic System Command under Contract No. N00039-82-C-0235.
”4.2BSD Networking Implementation Notes” acknowledgements: The internal structure of the system is

patterned after the Xerox PUP architecture [Boggs79]. The use of software interrupts for process invocation is based on similar facilities found in the VMS operating system.

Many of the ideas are based on

Rob Gurwitz’s TCP/IP implementation for the 4.1BSD version of UNIX on the VAX [Gurwitz81]. Greg
Chesson explained his use of trailer encapsulations in Datakit, instigating their use in our system.
*SENDMAIL - An Internetwork Mail Router” acknowledgements:

For their ideas and assistance, Kurt

Shoens, Bill Joy, Mark Horton, Erick Schmidt, Kirk McKusick, Marvin Solomon, Mike Stonebraker, and
Bob Epstein.

A considerable part of this work was done while under the employ of the INGRES Project

at the University of California at Berkeley.

vil

BEFORE YOU START

This is the third volume of ULTRIX Supplementary Documents, a three volume set that contains articles describing the ULTRIX-32 system. The authors are computer scientists and
program developers at Bell Laboratories and the University of California at Berkeley. The
articles explain the software tools and utilities available on your ULTRIX-32 system. They
constitute most of the lore that enriches this operating system; topics range from getting
started to the details of screen updating and cursor movement facilities.
Each volume in this set contains several parts, and each part begins with an introduction.
The introduction to each part serves as a map that will help you find your way around in the
documentation, allowing you to select articles that relate to your interest. Each introduction
gives an overview of the material covered in the part and a description of the articles included.
Most readers will not need to read all articles, since many articles cover parallel topics.
These articles provide authoritative and accurate information that is unavailable elsewhere.
However, you should be aware that some of the information in some articles 1s dated. We
include those articles because many of the concepts they develop are still current and important.

At the end of each volume in this set, you will find a master index identifying topics in all

three volumes.

Topics in Volume III

The articles in this third volume are written for people responsible for the installation,
administration, and daily maintenance of the ULTRIX-32 system. “Bug Fixes and Changes in
4.2BSD,” in Part 1, lists changes in directories, libraries, and utilities between the 4.1BSD
software and ULTRIX-32 (based on 4.2BSD).

“A Fast File System for UNIX,” by McKusick, Joy, Lefller, and Fabry, compares the new file
system used in ULTRIX-32 with the original UNIX file system. The new system is faster and
more reliable, and the block size is adjustable. The article also explains considerations and
procedures that will help you take full advantage of these improvements.

The articles in Part 2, Maintenance and Administration, deal with disk quotas, fixing corrupted file systems, and management of the sendmail utility. The quota utility enables the
system manager to limit the number of blocks and the number of files available to each user.

Fsck, the file system check program, lets you examine the integrity of the file system and

repair any inconsistencies. The sendmail utility lets users send messages between computer
systems that are connected to different networks.

Articles in Part 3, Communication, explain the interprocess communication software. Articles

in Part 4, Security Considerations, offer a variety of tips on how you can protect your system

against crashes and unauthorized access. And Part 5, Supporting Documents, provides information on software changes new to this release.

Table of Contents

BEFORE YOU START

PART 1:

OPERATING SYSTEM CHANGES

BUG FIXES AND CHANGES IN 4.2BSD

e e e e 1-3
e e
e e e
e o
o
NOTABLE IMPROVEMENTS. . . . . . o
e e e e e e 1-5
e
e
e
Section 1. . . . 0
e e e e e e 1-10
e
e
e e e
e
Section 2. . . .
e e 1-14
e
e
e
e
e e
e
Section 3. . . .
e e e 1-15
e
e
e
e
e
e
e
e
e
Section 4. . . . o o e
e e 1-16
e
e
e
e
e
e
e
e
e
Section 5. . . .
1-17
P
Section 6. . . . . . . . ..oS
1-17
e
e
e
e
e
e
e
e
e
e
Section 7. . . o o o
e e e e 1-18
e
e
e e
Section 8. . . .
A FAST FILE SYSTEM FOR UNIX

e
s e e e
d e e e
e s
INTRODUCTION . . . . .
e
e
e
e
e
e
e
e
e
e
s
e
e
o
o
.
.
.
.
OLD FILE SYSTEM .
e
e
e
e
o0
o
o
o
o
.
.
.
.
.
.
.
.
.
ON
ORGANIZATI
NEW FILE SYSTEM
Optimizing Storage Utilization . . . . . . . . . . .« . o
File System Parameterization . . . . . . . . . . . . ..o
Layout Policies. . . . . . . . . . . oL
e e
e e e
e e e
e e e e e
PERFORMANCE . . . . . .
. o ...
.
.
.
.
.
.
.
.
.
.
.
.
.
NTS
FILE SYSTEM FUNCTIONAL ENHANCEME
e
e
Long File Names . . . . . . . . . . . . . .
e
e
e
File Locking . . . . . . . . . .
Symbolic Links. . . . . . . . . . Lo
e s e
Rename . . . . . . . . . . . . .. e
e e
e
QUOLAS . . . v v v e e e e e e e e e

SOFTWARE ENGINEERING . . . . . . o

PART 2:

e e e e e e

e e e

1-23
1-25
1-26
1-26
1-28
1-29
1-31
1-33
1-33
1-33
1-34
1-35
1-35

1-36

MAINTENANCE AND ADMINISTRATION

DISC QUOTAS IN A UNIX ENVIRONMENT
e 2-3
e e e
e
USERS’ VIEW OF DISC QUOTAS. . . . . . . o« o o
Surviving When Quota Limit Is Reached. . . . . . . . . . . . . . . . ... .. .. 2-3
o o o oo 2-4
ADMINISTERING THE QUOTA SYSTEM . . . . . . . . . .
SOME IMPLEMENTATION DETAIL . . . . . . . . . . . .. S 2-5

x Table of Contents

FSCK - THE UNIX FILE SYSTEM CHECK PROGRAM
INTRODUCTION .

OVERVIEW OF THE FILE SYSTEM .

Superblock .

Fragments .

FIXING CORRUPTED FILE SYSTEMS .

.«

Detecting and Correcting Corruption.

Super-Block Checking

. 2-8

.o
.

s 2-9

Lo 2-9
oo

..o

2-10

2-11
2-11

.
e e e e

Checking the Data Associated with anInode.
File System Connectivity .

2-10

s, 2-8

...

2-12

...

2-14

.
.

...

Initialization .

...

e e,

2-14

Phase 1 - Check Blocks and Sizes .

2-16

Phase 2 - Check Pathnames.

2-12

FSCK ERROR CONDITIONS.

Conventions

e 2-7

Checking the Inode State .

APPENDIX A:

Inode Links

e
e

Inode Data Size

e, 2-8

Updates to the File System .

Free Block Checking

s 2-8

Cylinder Groups
.

Summary Information
.

...

2-18

Phase 3 — Check Connectivity .

..o

2-21

2-22

Phase 4 - Check Reference Counts.

.«

2-24

Phase 6 — Salvage Cylinder Groups

...

2-25

Phase 5 - Check Cyl Groups
Cleanup .

.
.

...
...

SENDMAIL INSTALLATION AND OPERATION GUIDE
BASIC INSTALLATION.

Off-the-Shelf Configurations.

2-27
2-28

...

Installation Using the Makefile

...

2-28

Installation by Hand .

lib/libsys.a

2-28

2-29

fusr/lib/sendmail.cf
fusr/ucb/newaliases
fusr/lib/sendmail.cf
fusr/spool/mqueue .

.
.
.
.

. .
. .
. .
..

.
.
.
L.

.
.
.
L

. L,
L L L,
L.
L L
e
e,

2-29
2-29
2-29
2-29

fusr/lib/aliases . . . . . . . . L L,
Jusr/lib/sendmail.fc . . . . . . . . . L,
JECITC. . . . s,
fusr/lib/sendmail.hf . . . . . . . . .. .,

2-29
2-30
2-30
2-30

fusr/lib/sendmail.st
fete/syslog. . . . L

2-30
2-30

fusr/ucb/newaliases

fusr/lib/sendmail.

Jusr/ucb/mailq.

.
.

L
L

2-31

Table of Contents

NORMAL OPERATIONS .

Quick Configuration Startup
Format .

..o

2-31

.
e
e

2-31

e
e e
e
e e

2-31

Printing the Queue

Format of Queue Files .

Forcing the Queue .

2-31
2-32

2-33

Rebuilding the Alias Database .

Potential Problems

List Owners .

Per-User Forwarding (.forward Files).

Return-Receipt-To:

Errors-To:.

2-34

...

2-34

2-35

2-34

...

2-35
2-35

2-35

e
e
e

2-35

s
s

2-35

e
s e
e s e

2-35

Apparently-To:

Special Header Lines .

e
e

The Alias Database.

ARGUMENTS

2-31

The Mail Queue

The System Log
CLevels.

Queue Interval .

e
e

2-36

Daemon Mode .

2-36

Forcing the Queue

2-36

Debugging .

Trying a Different Configuration File

Timeouts.

... e

e
e

2-36

...

2-36

e
s e

2-37

.
.

Queue Interval

2-37

Read Timeouts

2-37

Delivery Mode .

Log Level

Message Timeouts.

2-36

...

Changing the Values of Options .
TUNING .

e
.

.
e e

2-37

e
s

2-38

File Modes .

2-38

To Suid or Not To Suid?.

2-38

.. S

2-38

Should My Alias Database Be Writable?

...

2-38

THE WHOLE SCOOP ON THE CONFIGURATION FILE .

2-39

L
e e

2-39

Temporary File Modes.

The Syntax

R and S - Rewriting Rules .

D - Define Macro .

C and F - Define Classes.

..o

e
.

2-39

2-40

..o

2-40

M - Define Mailer.

2-40

H - Define Header.

2-40

P - Precedence Definitions .

O - Set Option

T - Define Trusted Users
~

e
.

2-41

xii Table of Contents

SENDMAIL INSTALLATION AND OPERATION GUIDE (continued)
The Semantics .

Special Macros, Conditionals.

2-41

L
e
e,

2-43

The Left Hand Side .

The Right Hand Side

Semantics of Rewriting Rule Sets.

2-41

Special Classes

...

2-43

...

2-44

The “Error” Mailer

Building a Configuration File From Scratch

2-45

...

2-46

Mailer Flags Etc.

What You Are Trying ToDo .
Philosophy

Relevant Issues

.
.

How To Proceed.

2-45

...

2-47

2-45

2-46

...

2-47

2-48

...

2-48

...

2-48

.
.

s
.

2-51

2-52

2-54

2-56

L0000,

2-56

2-57

2-60

..o

e
e e

...

APPENDIX D: OTHER CONFIGURATION

Parameters in md/config.m4.

APPENDIX A: COMMAND LINE FLAGS .

APPENDIX B: CONFIGURATION OPTIONS
.

APPENDIX C: MAILER FLAGS.

PART 3:

APPENDIX E:

...

Building Mailer Descriptions .

Configuration in src/conf.c

...

Testing the Rewriting Rules - The -bt Flag.

Parameters in src/conf.h

Small Site - Complete Information

...

2-45

Large Site, Many Hosts - Minimum Information.

Single Host.

SUMMARY OF SUPPORT FILES.

...

COMMUNICATIONS

A 4.2BSD INTERPROCESS COMMUNICATION PRIMER
INTRODUCTION .

BASICS.

e
s, 3-5

s, 3-6

Socket Types.

Socket Creation

Binding Names.

.
e, 3-T7

Connection Establishment.

Discarding Sockets .

Connectionless Sockets .

Input/Output Multiplexing .

Data Transfer

NETWORK LIBRARY ROUTINES

Host Names

e, 3-6
3-7

3-8
3-9
e

3-10

3-11

oo,

3-12

Network Names

Protocol Names.

.
e,

Service Names .

Miscellaneous

3-13
3-13

3-14
3-14

Table of Contents

xiii

3-17

e e s e e e e
e e e
e s e s
e e
e
e e

3-17
3-19
3-20

3-23

e
e e
. .o
. . . ..o oo
e e e
. . ..o oL
. . . . . . O
e e

3-23
3-23
3-24
3-24
3-27
3-27

e
SEIVEIS. + + v v v e e e e e
e
Clients. . . . . . o o
e
Connectionless Servers . . . . . . . . . . .
e e e e e e

ADVANCED TOPICS . . . . .

Out of Band Data . . . . . . . . . . .
Signals and Process Groups . . . . . . .
Pseudo Terminals. . . . . . . . . . .
Internet Address Binding . . . . . . . .
Broadcasting and Datagram Sockets. . .
Signals. . . . . . . . .

e e

CLIENT/SERVER MODEL . . . . . . .

e e e

4.2BSD NETWORKING IMPLEMENTATION NOTES

3-29
3-30
3-31
3-32
3-33
3-35

3-35

e e e e e
Socket State. . . . . . . . .
Socket Data Queues . . . . . . . . . .. Lo
e
o
Socket Connection Queueing . . . . . . . . . . . . . oo

3-36
3-36
3-37

o
e
Protocol Layer(s) . . . . . . . . . . . ..
Network-Interface Layer . . . . . . . . . . . . . . . o

3-37
3-39

o o o

3-40

e e
e e
e
e
INTRODUCTION . . . . .
e e e e e e e e
OVERVIEW. . . . . .
e e e
e e
GOALS . . . o
. . .
.
.
INTERNAL ADDRESS REPRESENTATION. . . . . .
o
MEMORY MANAGEMENT. . . . . . . . . . . ..
e e
e e
INTERNAL LAYERING. . . . . . . .

Socket Layer.

e
s s s
e e
e
. . ...
.

e e s e

s s
e

. . . . . . . .

UNIBUS Interfaces . . . . . . . . . . .«

SOCKET/PROTOCOL INTERFACE . . . . . . . . . .

oo 3-42
o oo oo

3-45

..
e
e e e
e
e
e
.
. L L L
. . . .. e

3-45
3-45
3-45
3-46

. . . . . . . . . . . . .. . ...

3-47

Packet Transmission . . . . . . . . . .« . . o
s
Packet Reception. . . . . . s

3-47
3-47

3-48

Routing Tables. . . . . . . . . . . . . e
Routing Table Interface. . . . . . . . . . . . . . . . . . ... [
User Level Routing Policies . . . . . . . . . . . . . . . 000

3-48
3-49
3-49

e e

3-51

e
e
Control Blocks . . . . . . . . . e
e
e
Input Processing . . . . . . . . . . . .o
e e
Output Processing . . . . . . . . . . . ..

3-51
3-51
3-52

PROTOCOL/PROTOCOL INTERFACE . . . . . . . . .
Pr_output . .
Pr_input . .
prctlinput .
pr—ctloutput.

.
.
.
.

PROTOCOL/NETWORK-INTERFACE INTERFACE.

GATEWAYS AND ROUTING ISSUES.

RAW SOCKETS . . . . . . o

. . . . . . . . .

e e

e e e

e e

xiv Table of Contents
4.2BSD NETWORKING IMPLEMENTATION NOTES (continued)
BUFFERING AND CONGESTION CONTROL.

Memory Management. . .
Protocol Buffering Policies

.
.

.
.
.
.

Queue Limiting. . .
Packet Forwarding .

OUT OF BAND DATA . .
TRAILER PROTOCOLS. .

.
.

.
.
.
.

.
.

...

3-53

L
.

000
e e
e e e
e
. . . . oo
e
e e
e
e e e e
e

3-53
3-53
3-53
3-54

s
o

s
s
s e

e
e e,

3-55
3-56

. . e
e e e e e e
e
e e e e e e e

3-60

SENDMAIL: AN INTERNETWORK MAIL ROUTER
DESIGN GOALS .

OVERVIEW.

System Organization . . . . . .
Interfaces to the Outside World .

.
.

e
. . .

.
SMTP Over Pipes . . . . . . .
SMTP Over an IPC Connection

.
.

.
..
. .

.
.
.

Argument Vector/Exit Status.

Operational Description.

Argument Processing and Address Parsing
Message Collection

Message Delivery . . . . . .
Queueing for Retransmission .
Return To Sender . . . . . .

.
.
.

.
. .
. .

L
. .
. .

.
.

. .
.

Message Header Editing
Configuration File . . .

.
.

.
.
. .

e
.00

3-61
3-61

..o oo
.
. . . . . . .. T

3-62

e
.

.
.

e
.

.
.

3-61

3-62
3-62

3-62

..o

3-62

e
e
e
. . .. L Lo oo
oo
e

3-63
3-63
3-63

oo
e

3-63
3-63

USAGE AND IMPLEMENTATION . . . . . . . . . . . . o

3-63

e e e e
000
0000000

3-63
3-64
3-64

e
e e
e
oo e
e e
e o

3-64
3-64
3-65

Arguments . . . L L L L L L
Mail to Files and Programs . . . .
Aliasing, Forwarding, Inclusion . .
Aliasing . . .
Forwarding .
Inclusion . .

.
.

. . . . .
Lo

.
.

.
.
.

L
. . . . . . . . . .
. . L L

Message Collection .
Message Delivery . .
Queued Messages. .

.
.
.

.
.

Macros . . . . .
Header Declarations . . .
Mailer Declarations . . .
Address Rewriting Rules .
Option Setting. . . . . .

.
.
.
.

Configuration.

.
.
.

.
.
L

e
. .
. .

.
.

.
Lo
L
e
e

e
e
e e

3-65
3-65
3-65

. . . . . . . . . . .. e I

3-65

.
.
.
.

.
.
.
0

e
e
e e e e
e e
. . . . . ..
. . . oo
. . . . . . ... 0oL
.
e
e e
e e
e

3-66
3-66
3-66
3-66
3-67

COMPARISON WITH OTHER MAILERS .

...

3-67

. . . . . . . e
e e e e e
MMDEFE . e
e e e e
Message Processing Module . . . . . . . . . .. .
.00 Lo

3-67

.
.
.
.

Delivermail.

EVALUATIONS AND FUTURE PLANS .

. .

3-67
3-68
3-68

Table of Contents

PART 4:

SECURITY CONSIDERATIONS

ON THE SECURITY OF UNIX

PASSWORD SECURITY: A CASE HISTORY
e s e e e
e
INTRODUCTION . . . .
e e e e e
e e
PROLOGUE . . . . o
e e e
THE FIRST SCHEME . . . . . .
ATTACKS ON THE FIRST APPROACH . . . . . . .
e e
e e
AN ANECDOTE . . . .
IMPROVEMENTS TO THE FIRST APPROACH. . . .

e e e
e
e d e
. o 0
e e e
. . .

e
e
e e e e
e e e e e e
e
e e e e e s e e
e e e
o oo v o e
e e
e
e
e
e
e e
o o
. . . . . o

e 4-7
e e e 4-8
e e 4-8
e e e 4-8
e e 4-10
o .. 4-10

e
e e
e e e
e
o
Slower Encryption . . . . . . . . . .
e e e
e e e e
Less Predictable Passwords . . . . . . . .« .« . . .
e e e e e e e e e e e
e e
Salted Passwords . . . . . . . . o
o
o« o o
The Threat of the DES Chip . . . . . . .« .«
e e e
e
e
e
A Subtle Paint . . . . . . . e
e e
e
e s
s e e s
CONCLUSIONS . .

PART 5:

4-10
4-10
4-11
4-11
4-11

4-12

SUPPORTING DOCUMENTS

CHANGES TO THE KERNEL IN 4.2BSD
e 5-3
o o
CARRYING OVER LOCAL SOFTWARE. . . . . . . . . .«
e e b-4
e
e
e
oo
ORGANIZATIONAL CHANGES. . . . .
e 5-5
e
e
e
BUG FIXES AND CHANGES . . . . s
e 5-5
e
Jsys/R L e
5-7
e
e e e e e e
JSYS/SYS v e e e e e
e e e e e e 5-8
e
Initialization Code. . . . . . . .« . L
e 5-8
e
Kernel-Level Support . . . . . . . . . . o
Disk Quotas. . . . . . . . .. o 5-9
e e e e e e 5-9
e e
e e
General Subroutines . . . . . . . . . .
System Level Support . . . . . . . . . L 5-9
e 5-9
e
Terminal Handling . . . . . . . . . . o . .
e
File System Support. . . . . . . . . . .o 5-9
Interprocess Communication . . . . . . . . . . .. Lo 5-10
e 5-10
e e e
Virtual Memory Support .. . . . . . . . . . . .
e
e
e
e e
e
e e
[SYS/CONE . . . .
e
/sys/vaxuba. . .. L . L e
e e
e e e
e e e e e e e e e
[SYS/VAX .« o
e
e
fsys/vaxmba . .. L . L
e e e ..
s e e e e e e
o o
STANDALONE SUPPORT . . . . .

5-11
5-12
5-13
5-14

b-15

e e 5-15
e e
e e
Disk Formatting . . . . . . .« . . . o
5-15
o
.
.
.
.
.
.
.
.
.
.
.
.
Library
I/O
Standalone
e 5-15
System Bootstrapping. . . . . . . . . . . . oo oo

xvi Table of Contents

INSTALLING AND OPERATING 4.2BSD ON THE VAX
INTRODUCTION .

5-17

Hardware Supported .

b-17

Distribution Format

5-18

UNIX Devices: Block and Raw

...

5-22

Step 2: Copying the Mini-Root File System

5-24

Step 3: Booting from the Mini-Root File System .

Step 4: Restoring the Root File System

VAX Hardware Terminology

UNIX Device Naming

BOOTSTRAP PROCEDURE.

Step 1: Formatting the Disk.

5-19

..o,

5-20

5-22

...

5-25

...

5-26

Step 5: Creating a Boot Floppy or Cassette

...

5-217

Rebooting the Completed Root File System

...

5-27

...

Step 7: Setting Up the /usr File System .

...

5-28

Additional Software.

UPGRADING A 4BSD SYSTEM.

Step 1: What To Save
Step 2: Merging

5-31

5-32

Step 3: Converting File Systems.

5-32
e

5-33

5-34

Bootstrapping Language Processors

...

5-34

s
s,

5-35

SYSTEM SETUP .

Making a UNIX Boot Floppy .

Making a UNIX Boot Cassette

5-35

Kernel Configuration .

5-36

...

5-36

5-37

...

5-37

Kernel Organization .

Devices and Device Drivers.
Building New System Images.

Disk Configuration .

Initializing /etc/fstab.

Disk Naming and Divisions

Space Available .

.
.

5-35
.

...

5-38

...

5-38

..., ..

539

Layout Considerations .

5-38

File System Parameters

5-40

Implementing a Layout

5-42

Configuring Terminals .

5-42
5-43

Adding Users.

Site Tailoring

...

5-44

Setting Up the Mail System.

5-44

Setting Up a UUCP Connection

...

5-45

b-47

NETWORK SETUP.

System Configuration.
Network Data Bases

Jetc/relocal .

Jete/ftpusers.

...

.«

5-47

5-48

Regenerating /etc/hosts and /ete/networks .
Jete/hosts.equiv .

5-43

Setting Up the Line Printer System .

.
.

5-48

5-49

...

5-49

5-50

SYSTEM OPERATION .

.
.

xvii

5-52

5-53

Bootstrap and Shutdown Procedures.
Device Errors and Diagnostics.

Table of Contents

File System Checks, Backups and Disaster Recovery .

5-53

Moving Filesystem Data

0oL

5-54

Monitoring System Performance. . . . . . . . . . . . . . .00
Recompiling and Reinstalling System Software. . . . . . . . . . . . . . . . . . ..
Making Local Modifications. . . . . . . . . . . . . . Lo
Accounting . . . . . . . L
L L
e
e
Resource Control . . . . . . . .
L e
e
Network Troubleshooting . . . . . . . . . . . . . . . .. ..

5-54

5-57

Files Which Need Periodic Attention

APPENDIX A: BOOTSTRAP DETAILS .

.
.

...

5-59
H-64

Generic System Control Status Register Addresses .

5-64

APPENDIX B: LOADING THE TAPE MONITOR . . . .
APPENDIX C: INSTALLATION TROUBLESHOOTING .

.
.

..
..

.
.

...
...

5-65
5-69

. . .
e
e

.
. ...
e
e

5-69

Contents of the Distribution Tapes
Control Status Register Addresses .

.
.

Using the Distribution Console Medium .

Booting the Generic System. .
Building Console Cassettes . .

.
.

.
.
.

.
.

5-55
5-56
5-56

5-56

...

5-57

5-59

...

5-70
5-T71

BUILDING 4.2BSD SYSTEMS WITH CONFIG
INTRODUCTION .

CONFIGURATION FILE CONTENTS.
Machine Type .

Cpu Type

5-T73

5-74

e
e
e e e

5-T4

System Identification .

5-T4

Timezone

.
.

5-74

Hardware Devices

oL
e

H-T4

5-T4

5-T75

Maximum Number of Users.
Root File System Location

Creating a Configuration File .

Optional Items .

SYSTEM BUILDING PROCESS.

e
s

b-15
e

5-75

oo,

5-T76

...

5-76

Constructing Source Code Dependencies .

...,

5-T17

Building the System

5-T7

5-T17

5-78

CONFIGURATION FILE SYNTAX

s
e e
.

5-79

Sharing Object Modules
Building Profiled Systems.

Global Configuration Parameters

System Image Parameters.

.o,
.

.00

5-79

Lo
e

5-80

.. e

Device Specifications .

5-81

Pseudo-Devices.

5-82

xviii Table of Contents

BUILDING 4.2BSD SYSTEMS WITH CONFIG (continued)
. . . . . . . .

SAMPLE CONFIGURATION FILES.

VAX-11/780 System . . . . .« . o
VAX-11/750 with Network Support .
Miscellaneous Comments . . . . . .

o
.
.

v
.
.

.
.

e
.
.

. . . . .

ADDING NEW SYSTEM SOFTWARE.

Modifying System Code. . . . . . . . .
Adding Device Drivers to 4.2BSD . . . .
Autoconfiguration on the VAX . . . . .
UNIBUS Resource Management Routines
Autoconfiguration Requirements. . . . .

.
.
.
.
.

e
. . . . . .
e e
.

o oo

.o
. . . . o000
.o 000000
. . . . . . . . . . . .. . .. . ...
.

. . . ..

. . . . . . . . .. . .

APPENDIX A: CONFIGURATION FILE GRAMMAR
.

e e e
e
oo
e e e e e e e

. . . . . . . . . . . . . . ...

Adding Nonstandard System Facilities.
Lexical Conventions

. . .

oo oo

APPENDIX B: RULES FOR DEFAULTING SYSTEM DEVICES . . . . . . . .. .. ..
...

APPENDIX C: SAMPLE CONFIGURATION FILES . . . . . . . . . . . . . . .

APPENDIX D: VAX KERNEL DATA STRUCTURE SIZING RULES.
.

System Size Limitations

.«
. . .

o
e e
e
e e e e
e
e
v

.«

Compile Time Rules . .
Run-Time Calculations .

«
.

. . . . . . . . ..

SETTING UP VERSION 1.0 OF UNIX/32V OPERATING SYSTEM
. .

NEW USERS . . . . .
MULTIPLE USERS . .
FILE SYSTEM HEALTH .

. .

SPECIAL FILES . . .
TIME CONVERSION .
DISK LAYOUT . . . .

MAKING A DISK FROM TAPE.
BOOTING UNIX . . . .
RECONFIGURATION. .

e
oo
e
e
e e
b b
. . . . o
e
e e e
o
o
o
.
e e e e e
e
e
e
o
. . . . .
.

CONVERTING SIXTH EDITION FILESYSTEMS . . . . . . . . . . . . . . .
ODDS AND ENDS .

REGENERATING SYSTEM SOFTWARE FOR UNIX/32V
INTRODUCTION .

WHERE COMMANDS AND SUBROUTINES LIVE . . . . . . . . o

o
COMMANDS . .
THE ASSEMBLER .
THE C COMPILER .

.«

OTHER LIBRARIES
SYSTEM TUNING .

.
.

o
.. e

e
e

Table of Contents

xix

A DIAL-UP NETWORK OF UNIX SYSTEMS
PURPOSE . . . . .
DESIGN GOALS . . .

PROCESSING . . .

e
e
e

e
e e

e e 5-123
e e e s s
e e e 5-123
e e
e e

e e s

s e e 5-125

5-125
e e
e
e
e
File Copy . . . . o o o
e 5-125
e e e e e e
e
Scan for Work . . . . . . .
e e e e e e 5-125
Call Remote System . . . . . . . . . . . . .o
e 5-126
e e e e
e
Line Protocol Selection . . . . . . . . . . . . .
e e 5-126
Work Processing . . . . . . . . . . . ..oe
e e e e e e e e 5-126
Conversation Termination. . . . . . . . . .« o« v v e

e 5-126
e
e e e e e e e
e
e
PRESENT USES . . . . .
e e 5-127
e e e e e s e
e
e e
e
PERFORMANCE . . . . .
s e 5-128
e e e e s s s e s
e
FURTHER GOALS . . . . .
s e e 5-128
e e e e e e
LESSONS . . . e

UUCP IMPLEMENTATION DESCRIPTION
UUCP - UNIX TO UNIX FILE COPY . . . . . . . . . . .« oo 5-131
oo o oo oo 5-133
UUX - UNIX TO UNIX EXECUTION. . . . . . . . . .
e 5-134
e
oo
.«
.
.
.
.
.
.
.
UUCICO - COPY IN, COPY OUT
5-137
oo«
.
.
.
.
.
.
.
.
.
.
.
EXECUTION.
UUXQT - UUCP COMMAND
5-137
oo
oo
o
.
.
.
.
.
.
.
.
UULOG - UUCP LOG INQUIRY
UUCLEAN - UUCP SPOOL DIRECTORY CLEANUP . . . . . . . . . . . . .. . .. 5-137
oo oo S 5-138
SECURITY . . . . . . . . . o
e 5-138
e e e e
e
o
. . o o o
.
.
.
.
.
UUCP INSTALLATION
5-142
e
e
e
e
e
o
o
o
o
o
.
.
.
.
.
ADMINISTRATION.

Introduction

1-1

PART 1: OPERATING SYSTEM CHANGES

The two articles in Part 1 deal with the differences between the 4.2BSD UNIX system (the
ULTRIX-32 system is based on 4.2BSD) and earlier versions of UNIX. “Bug Fixes and
Changes in 4.2BSD” gives a comprehensive list of improvements to the system including:
e

A new set of interprocess communication facilities

A new signal package

Support for advisory locking on files

Per-user and per-file system disk quotas

A new symbolic debugger, dbx, for C and Fortran programs

A new internetwork mail router, sendmail

Changes to specific software tools and commands are listed alphabetically.

“A Fast File System for UNIX,” by McKusick, Joy, Lefller, and Fabry, explains the new file
system in detail. This file system is specifically designed for the VAX hardware; it is available
only on 4.2BSD and the ULTRIX-32 system. The article is essential to people responsible for
management and administration of ULTRIX-32 systems.

The new file system, unlike the original UNIX file system, allows you to select a block size.
The block size can be 4096 bytes or any power of 2 greater than 4096; you must choose the
size when you create the file system. You can optimize the disk usage and file transfer rates
on your ULTRIX-32 system by choosing a block size that:
o

Matches the physical characteristics of your disk drives

Is appropriate for your applications

The article also explains use of:

A file-locking facility that allows cooperating programs to apply advisory locks on
files

Symbolic links that allow references across separate physical file systems

A rename facility that replaces three system calls with one

A quota utility that allows the system administrator to set limits on the number of
blocks and the number of files available to each user

You can find more detailed information on the quota utility in “Disk Quotas in a UNIX
Environment” in Part 2 of this volume.

Bug Fixes and Changes in 4.2BSD 1-3

Bug fixes and changes in 4.2BSD
Samuel J. Leffler

Computer Systems Research Group

Department of Electrical Engineering and Computer Science
University of California, Berkeley
Berkeley, California 94720
(415) 642-7780

Notable improvements

The file system organization has been redesigned to provide at least an order of magnitude improvement in disk bandwidth.

The system now provides full support for the DOD Standard TCP/IP network communication protocols. This support has been integrated into the system in a manner which
allows the development and concurrent use of other communication protocols. Hardware
support and routing have been isolated from the protocols to allow sharing between
varying network architectures. Software support is provided for 10 different hardware
devices including 3 different 10 Mb/s Ethernet modules.

A new set of interprocess communication facilities has replaced the old multiplexed file
mechanism. These new facilities allow unrelated processes to exchange messages in
either a connection-oriented or connection-less manner. The interprocess communication facilities have been integrated with the networking facilities (described above) to
provide a single user interface which may be used in constructing applications which
operate on one or more machines.

A new signal package which closely models the hardware interrupt facilities found on the
VAX replaces the old signals and jobs library of 4.1BSD. The new signal package provides for automatic masking of signals, sophisticated signal stack management, and reliable protection of critical regions.

File names are now almost arbitrary length (up to 255 characters) and a new file type,
symbolic link, has been added. Symbolic links provide a “symbolic referencing” mechanism similar to that found in Multics. They are interpolated during pathname expansion
and allow users to create links to files and directories which span file systems.
The system supports advisory locking on files. Files can have “shared” or “exclusive”
locks applied by processes. Multiple processes may apply shared locks, but only one process at any time may have an exclusive lock on a file. Further, when an exclusive lock is
present on a file, shared locks are disallowed. Locking requests normally block a process
until they can be completed, or they may be indicated as “non-blocking” in which case
an error is returned if the lock can not be immediately obtained.
The group identifier notion has been extended to a “group set”. When users log in to
the system they are placed in all their groups. Access control is now done based on the
group set rather than just a single group id. This has obviated the need for the newgrp
command.

Per-user, per-filesystem disk quotas are now part of the system. Soft and hard limits
may be specified on a per user and per filesystem basis to control the number of files and

1-4 Bug Fixes and Changes in 4.2BSD
amount of disk space allocated to a user.

Users who exceed a soft limit are warned and

if, after three login sessions, their disk usage has not dropped below the soft

soft limit is treated as a hard limit.
reporting of disk quotas.

limit, their
Utilities exist for the creation, maintenance, and

System time is now available in microsecond precision and millisecond accuracy.
Users
are provided with 3 high-resolution timers which may be set up to automatica
lly reload
on expiration. The timers operate in real time, user time, and process
virtual time (for
profiling). All statistics and times returned to users are now given in a standard
format
with seconds and microseconds separated. This eliminates program dependenc
e on the
line clock frequency.

A new system call to rename files in the same file system has been added.
This call
eliminates many of the anomalies which could occur in older versions of the
system due
to lack of atomicity in removing and renaming files.

A new system call to truncate files to a specific length has been added.
improves the performance of the Fortran I/O library.

Swap space configuration has been improved by allowing multiple swap partition
of
These partitions are sized at boot time to minimize
configuration dependencies.

This call

varying sizes to be interleaved.

The Fortran 77 compiler and associated I/0O library have undergone extensive
improve reliability and performance.

changes to
Compilation may, optionally, include optimization

phases to improve code density and decrease execution time.
o

A new symbolic debugger, dbx, replaces the old symbolic debugger sdb.
Dbx works on
both C and Fortran 77 programs and allows users to set break points and trace
execution
by source code line numbers, references to memory locations, procedure entry, etc.
Dbx
allows users to reference structured and local variables using the program’s
programming
language syntax.

The delivermail program has been replaced by sendmail. Sendmail provides
full internetwork routing, domain style naming as defined in the DARPA Request
For Comments

document #833, and eliminates the compiled in configuration database previously
used

by delivermail.

Further, sendmail uses the DARPA standard Simple Mail Transfer Protocol (SMTP) for mail delivery.
o

The system contains a new line printer system.

Multiple line printers and spooling

queues are supported through a printer database file.

Printers on serial lines, raster

printing devices, and laser printers are supported through a series of
filter programs
which interface to the standard line printer “core programs”. A line printer
control program, lpc, allows printers and printer queues to be manipulated. Spooling
to remote
printers is supported in a transparent fashion.

Cu has been replaced by a new program tip. Tip supports a number of
auto-call units
and allows destination sites to be specified by name rather than phone number.
Tip also

supports file transfer to non-UNIX machines and can be used with sites which
require

half-duplex and/or odd-even parity.
o

Uucp now supports many auto-call units other than the DN11. Spooling
has been reorganized into multiple directories to cut down on system overhead. Several
new utilities
and shell scripts exist for use in adminstrating uucp traffic. Operation has been
greatly
improved by numerous bug fixes.

Bug Fixes and Changes in 4.2BSD 1-5

Bug fixes and changes
Section 1
adb

Support has been added for interpreting kernel data structures on a running system and in post mortem crash dumps created by savecore. A —k option causes
adb to map addresses according to the system and current process page tables. A
new command, $p, can be used to switch between process contexts. Many scripts
are available for symbolically displaying kernel data structures, searching for a
process’ context by process ID, etc. A new document, “Using ADB to Debug the
UNIX Kernel”, supplies hints in the use of adb with system crash dumps.

addbib

Is a new utility for creating and extending bibliographic data bases for use with
refer.

apply
ar

Is a new program which may be used to apply a command to a set of arguments.

Has a new key, ‘o’, for preserving a file’s modification time when it is extracted
from an archive.

Supports the additional symbol information used by dbx. The old symbol information, used by the defunct sdb debugger, is available by specifying the —go flag.

A new flag, —pg, creates executable programs which collect profiling information
to be interpreted by the new gprof program. A bug in the C preprocessor, which
caused line numbers to be incorrect for macros with formal parameters with

embedded newlines has been fixed. The C preprocessor now properly handles
hexadecimal constants in “#if” constructs and checks for missing “#endif”’ statements.

chfn
chgrp

Now works interactively in changing a user’s information field in the password file.
Is now in section 1 and may be executed by anyone. Users other than the superuser may change group ownership of a file they own to any group in their group
access list.

csh

Now has a —r flag to copy recursively down a file system tree.

A bug which caused backquoted commands to wedge the terminal when interrupted has been fixed. Job identifiers are now globbed. A bug which caused the
“wait” command to uninteruptible in certain cases has been fixed. History may

now be saved and restored across terminal sessions with the savehist variable.

The newgrp command has been deleted due to the new group facilities.
ctags

Now handles C typedefs.

Exists only in the form of a “compatible front-end” to the new tip program.

dbx

Is a new symbolic debugger replacing sdb. Dbx handles C and Fortran programs.

delivermail

Has been replaced by the new sendmail program.

Understands the new file system organization and reports all disk space totals in
kilobytes.

Now reports disk usage in kilobytes and uses the new field in the inode structure

which contains the actual number of blocks allocated to a file to increase accuracy
of calculations.

dump

Has been moved to section 8.

error

Has been taught about the error message formats of troff.

eyace

A bug which caused the generated parser to not recognize valid null statements
has been fixed.

1-6

Bug Fixes and Changes in 4.2BSD

Has undergone major changes.

The i/o library has been extensively tested and debugged.

Sequential files are

now opened at the BEGINNING by default; previously they were opened at the

end.

Compilation of data statements has been substantially sped up.

Significant new

optimization is optionally available (this is still a bit buggy and should be used
with caution).

Even without optimization, however, single precision computations

execute much faster.

The new debugger, dbx, has replaced sdb for debugging Fortran programs; sdb is
no longer supported.

Files with “.F” suffixes are preprocessed by the C preprocessor. This allows C-

style “#include” and “#define” constructs to be used.

The compiler has been

modified to print error messages with sensible line numbers.

stands the “.F”’ suffix.

Make also under-

Note that when using the C preprocessor, the 72 column

convention is not followed.

The —I option for specifying short integers has been changed to —i.

The —I

option is now used to specify directory search paths for “#include” statements.

—pg option for creating executable images which collect profiling information for

gprof has been added.

fed

Is a font editor of dubious value.

file

Now understands symbolic links.

find

Has a new —type value, ‘I’, for finding symbolic links.

fp
fpr

Is a new compiler/interpreter for the Functional Programming language.

A sup-

porting document is present in Volume 2C of the UNIX Programmer’s Manual.
Is a new program for printing Fortran files with embedded Fortran carriage con-

trols.
fsplit

Is a new program for splitting a multi-function Fortran file into individual files.

ftp

Is a new program which supports the ARPA standard File Transfer Protocol.

gcore

Is a new program which creates a core dump of a running process.

gprof

Is a new profiling tool which displays execution time for the dynamic call graph of

a program.

Gprof works on C, Fortran, and Pascal programs compiled with the

—pg option.
operating

Gprof may also be used in creating a call graph profile for the

system.

supporting

document,

“gprof:

Call

Graph

Execution

Profiler” is included in Volume 2C of the UNIX Programmer’s Manual.
groups

hostid

Is a new program which displays a user’s group access list.

Is a new program which displays the system’s unique identifier as returned by the
new gethostid system call.

The super-user uses this program to set the host

identifier at boot time.

hostname Is a new program which displays the system’s name as returned by the new
gethostname system call.

The super-user uses this program to set the host name

at boot time.
indent

Is a new program for formatting C program source.

install

Is a shell script used in installing software.

iostat

Now reports kilobytes per second transferred for each disk.

This is useful as the

unit of information transferred is no longer a constant one kilobytes.

last

Now displays the remote host from which a user logged in (when accessing a
machine across a network).

The pseudo user “ftp” may be specified to find out

information about FTP file transfer sessions.

Bug Fixes and Changes in 4.2BSD

1-7

lastcomm Now displays flags for each command indicating if the program dumped core, used

PDP-11 mode, executed with a set-user-ID, or was created as the result of a fork
(with no following exec).

learn

Now has lessons for vi (this is user contributed software which is not part of the
standard system).

lint

Has a new —C flag for creating lint libraries from C source code. Has improved

type checking on static variables.
lisp

Has been ported to several 68000 UNIX systems, the relevant code is included in
the distribution. A new vector data type and a form of “closure” have been
added.

Has a new flag, —s, for creating symbolic links.

Has been extensively modified for use with the rlogind and telnetd network
SEervers.

Ipq

Is totally new, see lpr.

Ipr

And its related programs are totally new. The line printer system supports multi-

ple printers of many dififerent characteristics. A master data base, /etc/printcap,
describes both local printers and printers accessable across a network. A document describing the line printer system is now part of Volume 2C of the UNIX
Programmer’s Manual.

Iprm

Is totally new, see lpr.

Has been rewritten for the new directory format. It understands symbolic links
and uses the new inode field which contains the actual number of blocks allocated
to a file when the —s flag is supplied. Many rarely used options have been
deleted.

A bug which caused m4 to dump core when keywords were undefined then
redefined has been fixed.

Mail

Now supports mail folders in the style of the Rand MH system. Has been
reworked to cooperate with sendmail in understanding the new mail address for-

mats. Allows users to defined message header fields which are not be displayed

when a messages is viewed. Many other changes are described in a revised version
of the user manual.
make

Understands not to unlink directories when interrupted. Understands the new
“ F” guffix for Fortran source files which are to be run through the C preprocessor.
Has a new predefined macro MFLAGS which contains the flags supplied to make
on the command line (useful in creating hierarchies of makefiles).

mkdir

Now uses the mkdir system call to run faster.

Has a new command, status, which shows the current state of a tape drive.

Has been rewritten to use the new rename system call. As a result, multiple directories may now be moved in a single command, the restrictions on having “..” in a
pathname are no longer present, and everything runs faster.
66

net

And all related Berknet programs are no longer part of the standard distribution.
These programs live on in /usr/src/old for those who can not do without them.

netstat

Is a new program which displays network statistics and active connections.

oldcsh

No longer exists.

Has gobs of new formats options.

pagesize

Is a new program which prints the system page size for use in constructing portable shell scripts.

1-8 Bug Fixes and Changes in 4.2BSD

passwd

Now reliably interlocks with chsh, chfn, and vipw, in guarding against concurrent
updates to the password file.

pc/pi

For loops are now done according to the standard.

Files may now be dynamically

allocated

records

and

disposed.

Records

and

variant

are

now

aligned

correspond to C structures and unions (this was falsely claimed before).

obscure bugs involving formal routines have been fixed.

Several

Three new library rou-

tines support random access file i/0, see /usr/include/pascal for details.
pc

For loop variables and with pointers are now allocated to registers.

Separate

compilation type checking can now be done without reference to the source file;
this permits movement (including distribution) of .o files and creation of libraries.

Display entries are saved only when needed (a speed optimization).
pdx

Is a new debugger for use with pi.

Pdx is invoked automatically by the inter-

preter if a run-time error is encountered. Future work is planned to extend the

new dbx debugger to understand code generated by the Pascal compiler pc.
PSs

Has been changed to work with the new kernel and is no longer dependent on system page size. All process segment sizes are now shown in kilobytes.

Understands

that the old ‘““using new signal facilities” bit in the process structure now means
“using old 4.1BSD signal facilities”.
pwd
rcp

Now simply calls the getwd (3) routine.

Is a new program for copying files across a network.

The complete syntax of cp is

supported, including recursive directory copying.

refer

Has had many bugs fixed in it and the associated —ms macro package support
made to work.

reset

Now resets all the special characters to the system defaults specified in the

include file <sys/ttychars.h>.
rlogin

Is a new program for logging in to a machine across a network.

Rlogin uses the

files /etc/hosts.equiv and .rhosts in the users login directory to allow logins to be
performed without a password.

Rlogin supports proper handling of "S/°Q and

flushing of output when an interrupt is typed at the terminal.

Its ¢’ escape

sequences are reminiscent of the old cu program (as it is based on the same source

code).
rmdir

Now uses the rmdir system call to run more efficiently and not require root
privileges.

Unfortunately, this means arguments which end in one or more “/”

characters are no longer legal.
roffbib

Is a new program for running off bibiliographic databases.

rsh

Is a new program which supports remote command execution across a network.

ruptime

new

program

which

displays

system

status

information

for

clusters

machines attached to a local area network.
rwho

Is a new program which displays users logged in on clusters of machines attached

to a local area network.

script

Has been rewritten to use pseudo-terminals.

This allows the C shell job control

facilities (among other things) to be used while scripting.

A side effect of this

change is that scripts now contain everything typed at a terminal.
sdb

Has been replaced by dbx; it still lives on in /usr/src/old for those with a personal
attachment.

sendbug

Is a new command for submitting bug reports on 4.2BSD in a standard format

suitable for automatic filing by the bugfiler program.

No longer has a newgrp command due to the new groups facilities.

Bug Fixes and Changes in 4.2BSD 1-9
sortbib
strip
stty

Is a new command for sorting bibliographic databases.

Has been made blindingly fast by using the new truncate system call (thereby

eliminating the old method of copying the file).

The default system erase, kill, and interrupt characters have been made the DEC
standard values of DEL (7)), “U’, and “*C’. This is not expected to gain much
popularity, but was done in the interest of compatibility with many other standard operating systems.

Has been changed to do a “full login” when starting up the subshell. A new flag,
~f, does a “fast” su for when a system is heavily loaded. Extra arguments sup-

plied to su are now treated as a command line and executed directly instead of
creating an interactive shell.

sysline

Is a new program for maintaining system status information on terminals which
support a “status line”’; a poor man’s alternative to a window manager (or emacs).

tail

Has a larger buffer so that “tail —r” and similar show more.

talk

Is a new program which provides a screen-oriented write facility. Users may be
“talked to”across a network, though satellite response times have indicated overseas conversations are still best done by phone. Can be very obnoxious when
engaged in important work.

tar

telnet
tip

Now allocates its internal buffers dynamically so that the block size can be
specified to be very large for streaming tape drives. Also, now avoids many corecore copy operations. Has a new —C option for forcing chdir operations in the
middle of operation (thereby allowing multiple disjoint subtrees to be easily
placed in a single file, each with short relative pathnames). Has a new flag, ‘B’,
for forcing 20 block records to be read and written; useful in joining two tar commands with a remote shell to transfer large amounts of data across a network.
Is a new program which supports the ARPA standard Telnet protocol.

Replaces cu as the standard mechanism for connecting to machines across a phone
line or through a hardwired connection. Tip uses a database of system descriptions, supports many different auto-call units, and understands many nuances
required to talk to non-UNIX systems. Files may be transferred to and from
non-UNIX systems in a simple fashion.

uucp

A bug which sometimes caused an extra blank line to be printed after reaching
end of file has been fixed.

And related programs have been extensively enhanced to support many different
auto-call units and multiple spooling directories (among other things). A large
number of bugs and performance enhancements have been made.

Is a new program which gives a snap-shot of the uucp spooling area.
vfontinfo Is a program used to inspect and print information about fonts.
Now uses a regular expression language to describe formatting. A —f flag forces
vgrind

uusnap

vgrind to act as a filter, generating output suitable for inclusion in troff and/or
nroff documents. Language descriptions exist for C, Pascal, Model, C shell,
Bourne shell, Ratfor, and Icon programs.

A bug which caused the "B command to place the cursor on the wrong line has

been fixed. A bug which caused vi to believe a file had been modified when an i/o

error occurred has been fixed. A bug which allowed “hardtabs” to be set to 0
causing a divide by zero fault has been fixed.

vip
vimstat

Is a new program for pretty printing Lisp programs.

Has had one new piece of information added to —s summary, the number of fast
page reciaims performed. The fields related to paging activity are now all given in
kilobytes.

1-10

Bug Fixes and Changes in 4.2BSD

vpr

And associated programs for spooling and printing files on Varian and Versatec
printers are now shell scripts which use the new line printer support.

vwidth

Is a new program for making troff width tables for a font.

wWC

Is once again identical to the version 7 program.

That is, the —v, —t, —b, —s, and

—u flags have been deleted.

whereis

Understands the new directory organization for the source code.

which

Now understands how to handle aliases.

who

Now displays the remote machine from which a user is logged in.

Section 2.
The most important change in section 2 is that the documentation has been significantly

improved.

Manual page entries now indicate the possible error codes which may be returned

and how to interpret them.

The introduction to section 2 now includes a glossary of terms

used throughout the section.

The terminology and formatting have been made consistent.

Many manual pages now have “NOTES” or “CAVEATS” providing useful information heretofore left out for the sake of brevity.

As always the manual pages are still for the program-

mer; they are terse and extremely concise.

The “4.2BSD System Manual” is likewise concise,

but a bit more verbose in providing an overall picture of the system facilities.
With regard to changes in the facilities, these fall into three major categories: interprocess communication, signals, and file system related calls.

The interprocess communication

facilities center around the socket mechanism described in the “A 4.2BSD Interprocess Communication Primer”.

The new signals do not have an accompanying document, so the manual

pages should be studied carefully.

The new file system calls pretty much stand on their own,

with a late section of the document “A Fast File System for UNIX” supplying a quick overview of the most important new file system facilities.

Finally, it should be noted that the job

control facilities introduced in 4.1BSD have been adopted as a standard part of 4.2BSD.

special distinction is given to these calls (in 4.1BSD they were earmarked “2J”).
Many of the new system calls have both a “set” and a “get” form.
are indicated below.
intro

Only the “get” forms

Consult the manual for details on the “set” form.

Has been updated to reflect the new list of possible error codes.

Now includes a

glossary of terminology used in section 2.

access

Now has symbolic definitions for the mode parameter defined in <sys/file.h>.

bind

Is a new interprocess communication system call for binding names to sockets.

connect

Is a new interprocess communication system call for establishing a connection

between two sockets.
creat

Has been obsoleted by the new open interface.

fchmod

Is a new system call which does a chmod operation given a file descriptor; useful
in conjunction with the new advisory locking facilities.

fchown

Is a new system call which does a chown operation given a file descriptor; useful in
conjunction with the new advisory locking facilities.

fentl

Is a new system call which is useful in controlling how i/o is performed on a file

descriptor (non-blocking i/o, signal drive i/o).

This interface is compatible with

the System III fcntl interface.

flock

Is a new system call for manipulating advisory locks on files.

Locks may be

shared or exclusive and locking operations may be indicated as being nonblocking, in which case a process is not blocked if the requested lock is currently
in use.

Bug Fixes and Changes in 4.2BSD 1-11

fstat

fsyne

Now returns a larger stat buffer; see below under stat.

Is a new system call for synchronizing a file’s in-core state with that on disk. Its
intended use is in building transaction oriented facilities.

ftruncate Is a new system call which does a truncate operation given a file descriptor; useful
in conjunction with the new advisory locking facilities.

getdtablesize

Is a new system call which returns the size of the descriptor table.

getgroupsIs a new system call which returns the group access list for the caller.
gethostid Is a new system call which returns the unique (hopefully) identifier for the current
|
host.
gethostname

Is a new system call which returns the name of the current host.

getitimer Is a new system call which gets the current value for an interval timer.
getpagesize

Is a new system call which returns the system page size.

getpriorityls a new system call which returns the current scheduling priority for a specific

process, a group of processes, or all processes owned by a user. In the latter two
cases, the priority returned is the highest (lowest numerical value) enjoyed by any
of the specified processes.

getrlimit Is a new system call which returns information about a resource limit. The
getrlimit and setrlimit calls replace the old vlimit call from 4.1BSD.
getrusagels a new system call which returns information about resource utilitization of a
child process or the caller. This call replaces the vtimes call of 4.1BSD.

getsockoptls a new interprocess communication system call which returns the current
options present on a socket.

gettimeofday

Is a new system call which returns the current Greenwich date and time, and the
current timezone in which the machine is opervating. Time is returned in seconds
and microseconds since January 1, 1970.

ioctl

Has been changed to encode the size of parameters and whether they are to be

copied in, out, or in and out of the user address space in the request. The symbolic names for the various ioctl requests remain the same, only the numeric
values have changed. A number of new ioctls exist for use with sockets and the
network facilities. The old LINTRUP request has been replaced by a call to fentl
and the SIGIO signal.

killpg

Has now been made a system call; in 4.1BSD it was a library routine.

listen

Is a new interprocess communication system call used to indicate a socket will be

Iseek

Now has symbolic definitions for its whence parameter defined in <sys/file.h>.

mkdir

Is a new system call which creates a directory.

used to listen for incoming connection requests.

mpx

The multiplexed file facilities are no longer part of the system. They have been

open

Is different, now taking an optional third parameter and supporting file creation,
automatic truncation, automatic append on write, and “exclusive” opens. The
open interface has been made compatible with System III with the exception that

replaced by the socket, and related, system calls.

non-blocking opens on terminal lines requiring carrier are not supported.

profil

Now returns statistical information based on a 100 hz clock rate.

1-12 Bug Fixes and Changes in 4.2BSD

quota

Is a new system call which is part of the disk quota facilities.

Quota is used to

manipulate disk quotas for a specific user, as well as perform certain random

chores such as syncing quotas to disk.

read

Now automatically restarts when a read on a terminal is interrupted by a signal
before any data is read.

readv

Is a new system call which supports scattering of read data into (possibly) disjoint
areas of memory.

readlink

Is a new system call for reading the value of a symbolic link.

recv

Is a new interprocess communication system call used to receive a message on a
connected socket.

recvfrom Is a new interprocess communication system call used to receive a message on a
(possibly) unconnected socket.
recvmsg

Is a new interprocess communication system call used to receive a message on a
(possibly) unconnected socket which may have access rights included. When using
on-machine communication, recvimsg and sendmsg may be used to pass file
descriptors between processes.

rename

Is a new system call which changes the name of an entry in the file system (plain
file, directory, character special file, etc.). Rename has an important property in
that it guarantees the target will always exist, even if the system crashes in the
middle of the operation. Rename only works with source and destination in the
same file system.

rmdir

Is a new system call for removing a directory.

select

Is a new system call (mainly for interprocess communication) which provides facility for synchronous i/o multiplexing. Sets of file descriptors may be queried for
readability, writability, and if any exceptional conditions are present (such as out
of band data on a socket). An optional timeout may also be supplied in which
case the select operation will return after a specified period of time should no
descriptor satisfy the requests.

send

Is a new interprocess communication system call for sending a message on a connected socket.

sendto

Is a new interprocess communication system call for sending a message on a (possibly) unconnected socket.

sendmsg

Is a new interprocess communication system call for sending a message on a (possibly) unconnected socket which may included access rights.

setquota

Is a new system call for enabling or disabling disk quotas on a file system.

setregid

Is a new system call which replaces the 4.1BSD setgid system call. Setregid allows
the real and effective group ID’s of a process to be set separately.

setreuid

Is a new system call which replaces the 4.1BSD setuid system call.
allows the real and effective user ID’s of a process to be set separately.

Setreuid

shutdown Is a new interprocess communication system call for shutting down part or all of
full-duplex connection.

sigblock

Is a new system call for blocking signals during a critical section of code.

sigpause

Is a new system call for blocking a set of signals and then pausing indefinitely for
a signal to arrive.

sigsetmask
Is a new system call for setting the set of signals which are currently blocked from
delivery to a process.

sigstack

Is a new system call for defining an alternate stack on which signals are to be pro-

Bug Fixes and Changes in 4.2BSD

sigsys

1-13

Is no longer supported. The new signal facilities are a superset of those which sig-

—

sys provided.

sigvec

Is the new system call interface for defining signal actions. For each signal
(except SIGSTOP and SIGKILL), sigvec allows a “signal vector” to be defined.
The signal vector is comprised of a handler, a mask of signals to be blocked while
the handler executes, and an indication of whether or not the handler should execute on a signal stack defined by a sigstack call. The old signal interface is provided as a library routine with several important caveats. First, signal actions are
no longer reset to their default value after a signal is delivered to a process.
Second, while a signal handler is executing the signal which is being processed is
blocked until the handler returns. To simulate the old signal interface, the user
must explicitly reset the signal action to be the default value and unblock the signal being processed.

Four new signals have been added for the interprocess communication and interval timer facilities. SIGIO is delivered to a process when an fentl call enables signal driven i/o and input is present on a terminal (and a signal handler is defined).
SIGURG is delivered when an urgent condition arises on a socket (and a handler

is defined). SIGPROF and SIGVTALRM are associated with the ITIMER PROF

and ITIMER VIRTUAL interval timers, and delivered to a process when such a
timer expires (the SIGALRM signal is used for the ITIMER REAL interval
timer). The old SIGTINT signal is replaced by SIGIO.

socket

Is a new interprocess communication system call for creating a socket.

socketpairls a new interprocess communication system call for creatlng a pair of connected
and unnamed sockets.

stat

Now returns a larger structure. New fields are present indicating the optimal
blocking factor in which i/o should be performed (for disk files the block size of
the underlying file system) and the actual number of disk blocks allocated to the
file. Inode numbers are now 32-bit quantities. Several spare fields have been allocated for future expansion. These include space for 64-bit file sizes and 64-bit
time stamps. Two new file types may be returned, S IFLNK for symbolic links,
and S IFSOCK for sockets residing in the file system.

swapon

Has been renamed from the vswapon call of 4.1BSD.

symlink

Is a new system call for creating a symbolic link.

truncate

Is a new system call for truncating a file to a specific size.

unlink

Should no longer be used for removing directories. Directories should only be
created with mkdir and removed with rmdir. Creating hard links to directories
can cause disastrous results.

utime

Is defunct, replaced by utimes.

utimes

Is a new system call which uses the new time format in setting the accessed and
updated times on a file.

vfork

Is still present, but definitely on its way out. Future plans include implementing
fork with a scheme in which pages are initially shared read-only. On the first
attempt by a process to write on a page the parent and child would receive
separate writable copies of the page.

viimit

Is no longer supported. Vlimit is replaced by the getrlimit and setrlimit calls.

vread

Is no longer supported in the system.

vswapon

Has been renamed swapon.

vtimes

Is no longer supported. Vtimes is replaced by the getrusage call.

Is no longer supported in the system.

vwrite

JUNSUUP

1-14 Bug Fixes and Changes in 4.2BSD

wait

Now is automatically restarted when interrupted by a signal before status could
be returned.
|

wait3

Returns resource usage in a different format than that which was returned in

4,1BSD. This structure is compatible with the new getrusage system call. Wait3
is now automatically restarted when interrupted by a signal before status could be
returned.

write

Now is automatically restarted when writing on a terminal and interrupted by a
signal before any i/o was completed.

writev

Is a new version of the write system call which supports gathering of data in (possibly) discontiguous areas of memory

Section 3
The section 3 documentation has been reorganized to group manual entries by library.
Introductory sections for each logical and physical library contain lists of the entry points in
the library.

A number of routines which had been system calls under 4.1BSD are now user-level
library routines in 4.2BSD. These routines have been grouped under section “3C” headings,
“C” for compatibility. Further, certain routines present in the standard C run-time library
which do not easily categorize as part of one of the standard libraries, have been group under
“3X” headings.

curses

A number of bug fixes have been incorporated, and the documentation has been
revised.

stdio

The standard i/o library has been modified to block i/o operations to disk files
according to the block size of the underlying file system. This is accomplished
using the new st blksize value returned by fstat. The resultant performance
improvement is significant as the old 1 kilobyte buffer size often resulted in 7
memory-to-memory copy operations by the system on 8 kilobyte block file systems.

End-of-file marks now “stick”. That is, all input requests on a stdio channel after
encountering end-of-file will return end-of-file until a clearerr call is made. This
has implications for programs which use stdio to read from a terminal and do not
process end-of-file as a terminating keystroke.

Two new functions may be used to control i/o buffering. The setlinebuf routine is
used to change stdout or stderr from block buffered or unbuffered to line
buffered. The setbuffer routine is an alternate form of setbuf which can be used

after a stream has been opened, but before it is read or wfitten.

bstring

Three new routines, bcmp, bcopy, and bzero have been added to the library.
These routines use the VAX string instructions to manipulate binary byte strings
of a known size.

ctime

Now uses the gettimeofday system call and supports time conversion in six
different time zones. Daylight savings calculations are also performed in each
time zone when appropriate.
|

isprint

Now considers space a printing character; as the manual page has always indicated.

directory Is a new directory interface package which provides a portable interface to reading
directories. A version of this library which operates under 4.1BSD is also available.

getpass

Now properly handles being unable to open /dev/tty.

Bug Fixes and Changes in 4.2BSD

getwd

perror

Has been moved from the old jobs library to the standard C run-time library. It
now returns an error string rather than printing on the standard error when
unable to decipher the current working directory.
Now uses the writev system call to pass all its arguments to the system in a single
system call. This has profound effects on programs which transmit error messages
|

across a network.
psignal

1-15

And sys siglist are routines for printing signal names in an equivalent manner to
perror.

qsort

Has been greatly sped up by choosing a random element with which to apply its
divide and conquer algorithm.

random

Is a successor to rand which generates much better random numbers. The old
rand routine is still available and most programs have not been switched over to
‘random as doing so would make certain facilities such encrypted mail unable to
operate on existing data files.

setjmp

And longjmp now save and restore the signal mask so that non-local exit from a
signal handler is transparent. The old semantics are available with setjmp and
longjmp.

net

Is a new set of routines for accessing database files for the DARPA Internet. Four
databases exist: one for host names, one for network names, one for protocol
numbers, and one for network services. The latter returns an Internet port and
protocol to be used in accessing a given network service.

An additional collection of routines, all prefaced with “inet ” may be used to
manipulate Internet addresses, and interpret and convert between Internet
addresses and ASCII representations in the Internet standard “dot” notation.
Finally, routines are available for converting 16 and 32 bit quantities between host

and network order (on high-ender machines these routines are defined to be
noops).

fstab

The routines for manipulating /etc/fstab have been rewritten to return arbitrary
length null-terminated strings.

Section 4

The system now supports the 11/730, the new 64Kbit RAM memory controllers for the
11/750 and 11/780, and the second UNIBUS adapter for the 11/750. Several new character
and/or block device drivers have been added, as well as support for many hardware devices
which are accessible only through the network facilities. Each new piece of hardware sup-

ported is listed below.

New manual entries in section 4 have been created to describe communications protocols, and network architectures supported. At present the only network architecture fully supported is the DARPA Internet with the TCP, IP, UDP, and ICMP protocols.
ace

A network driver for the ACC LH/DH IMP interface.

A driver for the Data Translation A/D converter.

arp

The Address Resolution Protocol for dynamically mapping betwee 32-bit DARPA
Internet addresses and 48-bit Xerox 10Mb/s Ethernet addresses.

CSS

A network driver for the DEC IMP-11A LH/DH IMP interface.

dme

A network interface driver for the DEC DMC-11/DMR-11 point-to-point communications device.

A network interface driver for the 3Com 10Mb/s Ethernet controller.

1-16 Bug Fixes and Changes in 4.2BSD
en

A network interface driver for the Xerox 3Mb/s experimental Ethernet controller.

A network interface driver for the Network Systems Hyperchannel Adapter.

A driver for an Ikonas frame buffer graphics device interface.

A network interface driver for the Interlan 10Mb/s Ethernet interface.

imp

A network interface driver for the standard 1822 interface to an IMP; used in conjunction with either acc or css hardware.

A driver for a KL-11/DL-11W used as an alternate real time clock source for gath-

A software loopback network interface forsprotocol testing and performance

ering kernel statistics and profiling information.
analysis.

pcl

A network interface driver for the DEC PCL-11B communications controller.

A driver for an Evans and Sutherland Picture System 2 graphics device connected
with a DMA interface.

pty

Now includes a simple packet protocol to support flow controlled operation with
mechanisms for flushing data to be read and/or written.

A driver for the DEC dual RX02 floppy disk unit.

Now supports TUS80 tape drives.

The VAX-11/750 console cassette interface has been made somewhat usable when
operating in single-user mode.

The device driver employs assembly language

pseudo-dma code for the reception of incoming packets from the cassette.
uda

Now supports RA81, RA80, and RA60 disk drives.

A network interface driver for an Ungermann-Bass network interface unit connected to the host via a DR-11W.

Now supports ECC correction and bad sector handling.

Also has improved logic

for recognizing many different kinds of disk drives automatically at boot time.

A driver for DEC dual TUSS8 tape cartridges connected via a DL-11W interface.

The Varian driver has been rewritten so that it may coexist on the same UNIBUS
with devices which require exclusive use of the bus; i.e. RK07’s.

A network interface driver for the Proteon proNET 10Mb/s ring network controller.

Section 5
dir

disktab

Reflects the new directory format.
Is a new file for maintaining disk geometry information.

This is a temporary

scheme until the information stored in this file for each disk is recorded on the
disk pack itself.
dump

Is a superset of that used in 4.1BSD.

Reflects the new file system organization.

gettytab

Is a new file which idescribes terminal characteristics.

Each entry in the file

describes one of the possible arguments to the getty program.
hosts

Is a database for mapping between host names and DARPA Internet hos
addresses.

mtab

Has been modified to include a “type” field indicating whether the file system is
mounted read-only, read-write, or read-write with disk quotas enabled.

networks Is a database for mapping between network names and DARPA standard network

numbers.

Bug Fixes and Changes in 4.2BSD 1-17
phones

Is a phone number data base for tip.

printcap

Is a termcap clone for configuring printers.

protocols Is a database for mapping between protocol names and DARPA Internetwork

standard protocol numbers.
remote

services

Is a database of remote hosts for use with tip.
Is a database in which DARPA Internet services are recorded. The information
contained in this file indicates the name of the service, the protocol which is

required to access it, and the port number at which a client should connect to
utilize the service.
tar

Is a new entry describing the format of a tar tape.

utmp

Has been augmented to include a remote host from which a login session originates. The wtmp file is also used to record FTP sessions.

vgrindefs Is a file describing how to vgrind programs written in many languages.
Section 6

aardvark Does not work because it requires the “Dungeon Definition Language” processor
which is a binary image requiring 4.1BSD compatibility mode; the DDL source is
still present.

aliens

The aliens have returned home, the game is no longer included in the distribution.

backgammon

Is now screen oriented. A new program, teachgammon, instructs the new back-

gammon player. The old version is now called btlgammon.

canfield

Is a new game which plays a brand of the popular game of solitaire. Betting is

ching

Now pipes its output through more. Thus the hacker placates the seekers.

chase

No longer exists because the binary does not work under 4.2BSD.

factor

Is a rewrite in C of the old version 7 assembly language program which finds the

included, the program cfscores may be used to find out your current debt.

prime factors of a number.

fortune

Has yet more adages.

hangman Is now screen oriented.
mille

Now plays more intelligently.

primes

Is a rewrite in C of the old version 7 assembly language program which finds

rogue

Has been made more of a scoundrel. The supplementary document “A Guide to
the Dungeons of Doom”, has been updated as well, and is now part of Volume 2C

prime numbers within a specified range.

of the programmer’s manual.

sail

Is a new game which simulates sea battles of yore. The manual page is large

trek

The original trek has returned; trekies rejoice.

enough to be a separate document and so has been left in its source directory.

Section 7
hier

Has been updated to reflect the reorganization to the user and system source.

mailaddr Is a new entry describing mail addressing syntax under sendmail (possibly too
Berkeley specific).

1-18 Bug Fixes and Changes in 4.2BSD
The —ms macros have been extended to allow automatic creation of a table of

contents.

Support for the refer preprocessor is improved.

Several bugs related to

multi-column output and floating keeps have been fixed. Extensions to the accent
mark string set are available by including the .AM macro.

Footnotes can now be

automatically numbered (in superscript) by —ms and referenced in the text with a
** string register. The manual page includes a summary of important number
and string registers.

A new document “Changes to —ms” is included in Volume

2C of the programmer’s manual.

Section 8
Major changes affecting system operations include:

The system now supports disk quotas. These allow system administrators to control users’
disk space and file allocation on a per-file system basis. Utilities in this section exist for
fixing, summarizing, and editing disk quota summary files.
File systems are now made with a new program, newfs, which acts as front end to the old
mkfs program. There no longer is a need to remember disk partition sizes, as newfs gets

this information automatically from the /etc/disktab file. In addition, newfs attempts to
lay out file systems according to the characteristics of the underlying disk drive (taking
into account disk geometry information).

DEC standard bad block forwarding is now supported on the RP06 and second vendor
UNIBUS storage module disks. The bad144 program can now be used to mark sectors bad
on many disks, though inclusion in the bad sector table is still somewhat risky due to
requirements in the ordering of entries in the table.
A new program, format, should be used to initialize all non-DEC storage modules before

creating file systems.

Format formats the sector headers and creates a bad sector table

which is used in normal system operation. Format runs in a standalone mode.

Getty has been rewritten to use a description file, /etc/gettytab. This allows sites to tailor
terminal operation and configuration without making modifications to getty.
The line printer system is totally new. A program to administer the operation of printers,
Ipc, is supplied, and printer accounting has been consolidated into a single program, pac.
The program used to restore files from dump tapes is now called restore.

This name

change was done to reinforce the fact that it is completely rewritten and operates in a very

different way than the old restor program. Restore operates on mounted file systems and
uses only normal file system operations to restore files. Versions of both dump and restore
which operate across a network are included as rdump and rrestore. Dump and restore
(and their network oriented counterparts) now perform so efficiently (mostly because of the
new file system), that disk to disk backups should no longer be an attractive alternative.

arff

No longer asks if you want to clobber the floppy when manipulating archives
which are not on the floppy.

| badl144

Has been modified to use the /etc/disktab file.

Can be used to create bad sector

tables for the DEC RPO06 and several new Winchester disk drives.

Consult the

source code for details and use with extreme care.

badsect

Has been modified to work with the new file system and now must interact with
fsck to perform its duties. Consult the manual page for more information.

bugfiler

Is a new program for automatic filing and acknowledgement of bug reports submitted by the sendbug program. Intended to operate with the Rand MH software
which is part of the user contributed software. Used at Berkeley to process bug
reports on 4.2BSD.

Bug Fixes and Changes in 4.2BSD 1-19
chgrp
comsat

Has been moved to section 1.

Has been changed to filter the noise lines in message headers when displaying
incoming mail. No longer uses a second process watchdog as it uses the more reliable socket facilities instead of the old mpx facilities.

config

diskpart

drtest

dump

dumpfs
edquota

Has been extensively modified to handle the new root and swap device
specification syntax. A new document, “Configuring 4.2BSD UNIX Systems with
Config”, describes its use, as well as other important information needed in
configuring system images; this is part of Volume 2C of the programmer’s manual.
Is a new program which may be used to generate disk partition tables according to
the rules used at Berkeley. Can automatically generate entries required for device
drivers and for the /etc/diskpart file. (Does not handle the new DEC DSA style
drives properly because it tries to reserve space for the bad sector table.)
Is a new standalone program which is useful in testing standalone disk device
drivers and for pinpointing bad sectors on a disk.

Has been modified for the new file system organization. Mainly due to the new
file system, it runs virtually at tape speed. Properly handles locking on the dumpdates file when multiple dumps are performed concurrently on the same machine.

Is a new program for dumping out information about a file system such as the
block size and disk layout information.

Is a new program for editing user quotas. Operates by invoking your favorite editor on an ASCII representation of the information stored in the binary quota files.
Edquota also has a “replication” mode whereby a quota template may be used to
create quotas for a group of users.

fastboot

Is a new shell script which reboots the system without checking the file systems;

fasthalt

Is a new script which is similar to fastboot.

format

fsck

ftpd

should be used with extreme care.

Is a new standalone program for formatting non-DEC storage modules and creating the appropriate bad sector table on the disk.

Has been changed for the new file system. Fsck is more paranoid then ever In
checking the disks, and has been sped up significantly. The accompanying
Volume 2C document has been updated to reflect the new file system organization.
Is the DARPA File Transfer Protocol server program. It supports C shell style
globbing of arguments and a large set of the commands in the specification
(except the ABORT command!).

gettable

Is a new program which can be used in aquiring up to date DARPA Internet host
database files.

getly

Has been rewritten to use a terminal description database, /etc/gettytab. Consult

icheck

Has been modified for the new file system.

init
kgmon

the manual entries for getty (8) and gettytab (5) for more information.

Has been significantly modified to use the new signal facilities. In doing so,
several race conditions related to signal delivery have been fixed.
Is a new program for controlling running systems which have been created with
kernel profiling. Using kgmon, profiling can be turned on or off and internal
profiling buffers can dumped into a gmon.out file suiitable for interpretation by
gprof.

Ipc

Is a new program controlling line printers and their associated spooling queues.
Lpc can be used to enable and disable printers and/or their spooling queues. Lpc
can also be used to rearrange existing jobs in a queue.

1-20 Bug Fixes and Changes in 4.2BSD
Ipd

Has been rewritten and now runs as a “server”, using the interprocess communication facilities to service print requests. A supplementary document describing the
line printer system is now part of Volume 2C of the programmer’s manual.

MAKEDEV

|
Is a new shell script which resides in /dev and is used to create special files there.

MAKEDEV keeps commands for creating and manipulating local devices in a
,

separate file MAKEDEV.local.

mkfs

Has been virtually rewritten for the new file system. The arguments supplied are
very different. For the most part, users now use the newfs program when creating
file systems. Mkfs now automatically creates the lost+found directory.

mount

Now indicates file systems which are mounted read-only or have disk quotas

enabled.
newfs

Is a new front-end to the mkfs program.

Newfs figures out the appropriate

parameters to supply to mkfs, invokes it, and then, if necessary, installs the boot

blocks necessary to bootstrap UNIX on 11/750’s.

pac

Is a new program which can be used to do printer accounting on any printer.

subsumes the vpac program.

quot

Now uses the information in the inode of each file to find out how many blocks

are allocated to it.
quotacheck

Is a new program which performs consistency checks on disk quota files. Quotacheck is normally run from the /etc/rc.local file after a system is rebooted, though

it can also be run on mounted on file systems which are not in use.
quotaon

Is a new program which enables disk quotas on file systems.
named quotaoff, is used to disable disk quotas on file systems.

A link to quotaon,

pstat

Has been modified to understand new kernel data structures.

Has had system dependent startup commands moved to /etc/rc.local.

rdump

Is a new program to dump file systems across a network.

renice

Has been rewritten to use the new setpriority system call.
now renice users and process groups.

As a result, you can

repquota Is a new program which summarizes disk quotas on one or more file systems.
restor

No longer exists.

restore

Replaces restor. Restore operates on mounted file systems; it contains an interactive mode and can be used to restore files by name. Restore has become almost as

A new program, restore, is its successor.

flexible to use as tar in retrieving files from tape.
rexecd

Is a network server for the rexec (3X) library routine. Supports remote command
execution where authentication is performed using user accounts and passwords.

rlogind

Is a network server for the rlogin (1C) command. Supports remote login sessions
where authentication is performed using privileged port numbers and two files,

/etc/hosts.equiv and .rhosts (in each users home directory).
rmt

Is a program used by rrestore and rdump for doing remote tape operations.

route

Is a program for manually manipulating network routing tables.

routed

Is a routing daemon which uses a variant of the Xerox Routing Information Protocol to automatically maintain up to date routing tables.

rrestore

Is a version of restore which works across a network.

rshd

Is a server for the rsh(1C) command.

It supports remote command execution

using privileged port numbers and the /etc/hosts.equiv and .rhosts files in users b

home directories.

Bug Fixes and Changes in 4.2BSD 1-21
rwhod

Is a server which generates and listens for host status information on local networks. The information stored by rwhod is used by the rwho(1C)
andruptime (1C) programs.

rxformat

Is a program for formatting floppy disks (this uses the rx device driver, not the

savecore

Has been modified to get many pieces of information from the running system

sendmail

console floppy interface).

and crash dump to avoid compiled in constants.

Is a new program replacing delivermail; it provides fully internetwork mail forwarding capabilities. Sendmail uses the DARPA standard SMTP protocol to send
and receive mail. Sendmail uses a configuration file to control its operation, eliminating the compiled in description used in delivermail.

setifaddr

Is a new program used to set a network interface’s address. Calls to this program
are normally placed in the /etc/rc.local file to configure the network hardware
present on a machine.

syslog

Is a server which receives system logging messages. Currently, only the sendmail
program uses this server.

telnetd
tftpd

Is a server for the DARPA standard TELNET protocol.
Is a server for the DARPA Trivial File Transfer Protocol.

trpt

Is a program used in debugging TCP. Trpt transliterates protocol trace informa-

tunefs

Is a program for modifying certain parameters in the super block of file systems.

vipw

tion recorded by TCP in a circular buffer in kernel memory.

Is no longer a shell script and propérly interacts with passwd, chsh, and chfn in
locking the password file.

Fast File System 1-23

A Fast File System for UNIX*
Revised July 27, 1983
Marshall Kirk McKusick, William N. JoyT,
r S. Fabry
{ ,Robert
Samuel J. Leffle
Computer Systems Research Group
Computer Science Division

Department of Electrical Engineering and Computer Science
University of California, Berkeley
Berkeley, CA 94720

1. Introduction

This paper describes the changes from the original 512 byte UNIX file system to the new
one released with the 4.2 Berkeley Software Distribution. It presents the motivations for the
changes, the methods used to affect these changes, the rationale behind the design decisions,
and a description of the new implementation. This discussion is followed by a summary of
the results that have been obtained, directions for future work, and the additions and changes
that have been made to the user visible facilities. The paper concludes with a history of the
software engineering of the project.

The original UNIX system that runs on the PDP-117 has simple and elegant file system
facilities. File system input/output is buffered by the kernel;, there are no alignment constraints on data transfers and all operations are made to appear synchronous. All transfers to

the disk are in 512 byte blocks, which can be placed arbitrarily within the data area of the file
system. No constraints other than available disk space are placed on file growth [Ritchie74],
[Thompson79].

When used on the VAX-11 together with other UNIX enhancements, the original 512
file system is incapable of providing the data throughput rates that many applicaUNIX
byte
For example, applications that need to do a small amount of processing on a
require.
tions
of data such as VLSI design and image processing, need to have a high
quantities
large
the file system. High throughput rates are also needed by programs with
from
throughput
large address spaces that are constructed by mapping files from the file system into virtual
memory. Paging data in and out of the file system is likely to occur frequently. This requires
a file system providing higher bandwidth than the original 512 byte UNIX one which provides
only about two percent of the maximum disk bandwidth or about 20 kilobytes per second per
arm [White80], [Smith81b].

Modifications have been made to the UNIX file system to improve its performance.

Since the UNIX file system interface is well understood and not inherently slow, this development retained the abstraction and simply changed the underlying implementation to increase
its throughput. Consequently users of the system have not been faced with massive software
conversion.

* UNIX is a trademark of Bell Laboratories.

+William N. Joy is currently employed by: Sun Microsystems, Inc, 2550 Garcia Avenue, Mountain View, CA
$Samuel J. LefHler is currently employed by: Lucasfilm Ltd., PO Box 2009, San Rafael, CA 94912
94043
This work was done under grants from the National Science Foundation under grant MCS80-05144, and the
Defense Advance Research Projects Agency (DoD) under Arpa Order No. 4031 monitored by Naval Electronic System Command under Contract No. N00039-82-C-0235.

+ DEC, PDP, VAX, MASSBUS, and UNIBUS are trademarks of Digital Equipment Corporation.
urs &
L8\

1-24

Fast File System

Problems with file system performance have been dealt with extensively in the literature;
see [Smith8la] for a survey. The UNIX operating system drew many of its ideas from Multics, a large, high performance operating system [Feiertag71]. Other work includes Hydra

[Almes78], Spice [Thompson80], and a file system for a lisp environment [Symbolics81a].

A major goal of this project has been to build a file system that is extensible into a

networked environment [Holler73]. Other work on network file systems describe centralized
file servers [Accetta80], distributed file servers [Dion80], [Luniewski77], [Porcar82],
and protocols to reduce the amount of information that must be transferred across a network

[Symbolics81b], [Sturgis80].

Fast File System 1-25

2. Old File System

In the old file system developed at Bell Laboratories each disk drive contains one or

more file systems.t A file system is described by its super-block, which contains the basic
parameters of the file system. These include the number of data blocks in the file system, a
count of the maximum number of files, and a pointer to a list of free blocks. All the free
blocks in the system are chained together in a linked list. Within the file system are files.
Certain files are distinguished as directories and contain pointers to files that may themselves
be directories. Every file has a descriptor associated with it called an inode. The inode contains information describing ownership of the file, time stamps marking last modification and
access times for the file, and an array of indices that point to the data blocks for the file. For
the purposes of this section, we assume that the first 8 blocks of the file are directly referenced by values stored in the inode structure itself*. The inode structure may also contain
references to indirect blocks containing further data block indices. In a file system with a 512
byte block size, a singly indirect block contains 128 further block addresses, a doubly indirect
block contains 128 addresses of further single indirect blocks, and a triply indirect block contains 128 addresses of further doubly indirect blocks.

A traditional 150 megabyte UNIX file system consists of 4 megabytes of inodes followed
by 146 megabytes of data. This organization segregates the inode information from the data;
thus accessing a file normally incurs a long seek from its inode to its data. Files in a single
directory are not typically allocated slots in consecutive locations in the 4 megabytes of
inodes, causing many non-consecutive blocks to be accessed when executing operations on all
the files in a directory.

The allocation of data blocks to files is also suboptimum. The traditional file system
never transfers more than 512 bytes per disk transaction and often finds that the next sequen-

tial data block is not on the same cylinder, forcing seeks between 512 byte transfers. The
combination of the small block size, limited read-ahead in the system, and many seeks
severely limits file system throughput.

The first work at Berkeley on the UNIX file system attempted to improve both reliability and throughput. The reliability was improved by changing the file system so that all
modifications of critical information were staged so that they could either be completed or
repaired cleanly by a program after a crash [Kowalski78]. The file system performance was
improved by a factor of more than two by changing the basic block size from 512 to 1024
bytes. The increase was because of two factors; each disk transfer accessed twice as much
data, and most files could be described without need to access through any indirect blocks
since the direct blocks contained twice as much data. The file system with these changes will
henceforth be referred to as the old file system.

This performance improvement gave a strong indication that increasing the block size

was a good method for improving throughput. Although the throughput had doubled, the old
file system was still using only about four percent of the disk bandwidth. The main problem
was that although the free list was initially ordered for optimal access, it quickly became
scrambled as files were created and removed. Eventually the free list became entirely random
causing files to have their blocks allocated randomly over the disk. This forced the disk to
seek before every block access. Although old file systems provided transfer rates of up to 175
kilobytes per second when they were first created, this rate deteriorated to 30 kilobytes per
second after a few weeks of moderate use because of randomization of their free block list.
There was no way of restoring the performance an old file system except to dump, rebuild,
and restore the file system. Another possibility would be to have a process that periodically
reorganized the data on the disk to restore locality as suggested by [Maruyama76].
T A file system always resides on a single drive.

* The actual number may vary from system to system, but is usually in the range 5-13.

1-26 Fast File System

3. New file system organization
As in the old file system organization each disk drive contains one or more file systems.
A file system is described by its super-block, that is located at the beginning of its disk parti-

tion.

Because the super-block contains critical data it is replicated to protect against catas-

trophic loss.

This is done at the time that the file system is created; since the super-block

data does not change, the copies need not be referenced unless a head crash or other hard disk
error causes the default super-block to be unusable.
To insure that it is possible to create files as large as 232

bytes with only two levels of

indirection, the minimum size of a file system block is 4096 bytes.

blocks can be any power of two greater than or equal to 4096.

The size of file system

The block size of the file sys-

tem is maintained in the super-block so it is possible for file systems with different block sizes

to be accessible simultaneously on the same system. The block size must be decided at the
time that the file system is created; it cannot be subsequently changed without rebuilding the
file system.

The new file system organization partitions the disk into one or more areas called
cylinder groups.

disk.

A cylinder group is comprised of one or more consecutive cylinders on a

Associated with each cylinder group is some bookkeeping information that includes a

redundant copy of the super-block, space .for inodes, a bit map describing available blocks in

the cylinder group, and summary information describing the usage of data blocks within the
cylinder group.

For each cylinder group a static number of inodes is allocated at file system

creation time.

The current policy is to allocate one inode for each 2048 bytes of disk space,

expecting this to be far more than will ever be needed.
All the cylinder group bookkeeping information could be placed at the beginning of each

cylinder group.

However if this approach were used, all the redundant information would be

on the top platter.

Thus a single hardware failure that destroyed the top platter could cause

the loss of all copies of the redundant super-blocks.

Thus the cylinder group bookkeeping

information begins at a floating offset from the beginning of the cylinder group.

The offset for

each successive cylinder group is calculated to be about one track further from the beginning

of the cylinder group.

In this way the redundant information spirals down into the pack so

that any single track, cylinder, or platter can be lost without losing all copies of the super-

blocks.

Except for the first cylinder group, the space between the beginning of the cylinder

group and the beginning of the cylinder group information is used for data blocks.T

3.1. Optimizing storage utilization
Data is laid out so that larger blocks can be transferred in a single disk transfer, greatly
increasing file system throughput.

As an example, consider a file in the new file system com-

posed of 4096 byte data blocks.

In the old file system this file would be composed of 1024

byte blocks.

By increasing the block size, disk accesses in the new file system may transfer up

to four times as much information per disk transaction.

In large files, several 4096 byte blocks

may be allocated from the same cylinder so that even larger data transfers are possible before
initiating a seek.

The main problem with bigger blocks is that most UNIX file systems are composed of
many small files.

A uniformly large block size wastes space.

Table 1 shows the effect of file

system block size on the amount of wasted space in the file system. The machine measured to

obtain these figures is one of our time sharing systems that has roughly 1.2 Gigabyte of on-line
storage.

The measurements are based on the active user file systems containing about 920

megabytes of formated space.

The space wasted is measured as the percentage of space on

the disk not containing user data.

As the block size on the disk increases, the waste rises

quickly, to an intolerable 45.6
% waste with 4096 byte file system blocks.
T While it appears that the first cylinder group could be laid out with its super-block at the “known” location, this would not work for file systems with blocks sizes of 16K or greater, because of the requirement
that the cylinder group information must begin at a block boundary.

Fast File System 1-27

Space used | % waste |Organization

775.2 Mb

0.0

Data only, no separation between files

828.7 Mb
866.5 Mb
948.5 Mb
1128.3 Mb

6.9
11.8
22.4
45.6

512 byte block UNIX file system
1024 byte block UNIX file system
2048 byte block UNIX file system
4096 byte block UNIX file system

807.8 Mb

4.2

Data only, each file starts on 512 byte boundary

Table 1 — Amount of wasted space as a function of block size.

more
To be able to use large blocks without undue waste, small files must be stored in aa single
of
division
the
efficient way. The new file system accomplishes this goal by allowing t size is specified at the
file system block into one or more fragments. The file system fragmen ly broken into 2, 4,
time that the file system is created; each file system block can be optional
fragments
or 8 fragments, each of which is addressable. The lower bound on the size of these
with each
ed
associat
map
block
is constrained by the disk sector size, typically 512 bytes. The
availablock
ne
determi
to
level;
t
cylinder group records the space availability at the fragmen

bility, aligned fragments are examined. Figure 1 shows a piece of a map from a 4096/1024 file
system.

Bits in map

Fragment numbers
Block numbers

XXXX
0-3
0

XX00
4-7
1

00XX
8-11
2

0000
12-15
3

Figure 1 — Example layout of blocks and fragments in a 4096/1024 file system.

fragment is in
Each bit in the map records the status of a fragment; an “X” shows that the example
, fragthis
In
n.
allocatio
for
e
use, while a “O” shows that the fragment is availabl
ts of
Fragmen
free.
are
12—15
and
6—9,
ts
ments 0—5, 10, and 11 are in use, while fragmen
,
example
this
In
enough.
large
are
they
if
adjoining blocks cannot be used as a block, even
allocafor
available
are
12—15
s
fragment
fragments 6—9 cannot be coalesced into a block; only
tion as a block.

1024 bytes, a file
On a file system with a block size of 4096 bytes and a fragment size aofsingle
fragmented

is represented by zero or more 4096 byte blocks of data, and possibly
amount of data,
block. If a file system block must be fragmented to obtain space for a small an
example conAs
files.
other
to
on
allocati
for
e
the remainder of the block is made availabl
uses two full
would
file
This
system.
file
byte
24
sider an 11000 byte file stored on a 4096/10
the time the
at
e
availabl
are
ts
fragmen
byte
3072
no
If
size blocks and a 3072 byte fragment.
and an
fragment
byte
3072
y
necessar
the
yielding
split
file is created, a full size block is
file as
another
to
d
allocate
be
can
t
fragmen
ng
remaini
unused 1024 byte fragment. This
needed.

to a file,
The granularity of allocation is the write system call. Each time data is written
the new
hold
to
needs
file
the
If
d*.
increase
has
file
the
of
size
the system checks to see if the
data, one of three conditions exists:

There is enough space left in an already allocated block to hold the new data. The new

Nothing has been allocated. If the new data contains more than 4096 bytes, a 4096 byteis
block is allocated and the first 4096 bytes of new data is written there. This process
to be
repeated until less than 4096 bytes of new data remain. If the remaining newt isdata
located,
fragmen
ted
written will fit in three or fewer 1024 byte pieces, an unalloca

data is written into the available space in the block.

* A program may be overwriting data in the middle of an existing file in which case space will already be al-

1-28 Fast File System

otherwise a 4096 byte block is located. The new data is written into the located piece.
3)

A fragment has been allocated.

If the number of bytes in the new data plus the number

of bytes already in the fragment exceeds 4096 bytes, a 4096 byte block is allocated. The
contents of the fragment is copied to the beginning of the block and the remainder of

the block is filled with the new data. The process then continues as in (2) above.

If the

number of bytes in the new data plus the number of bytes already in the fragment will
fit in three or fewer 1024 byte pieces, an unallocated fragment is located, otherwise a

4096 byte block is located.

The contents of the previous fragment appended with the

new data is written into the allocated piece.
The problem with allowing only a single fragment on a 4096/1024 byte file system is that

data may be potentially copied up to three times as its requirements grow from a 1024 byte
fragment to a 2048 byte fragment, then a 3072 byte fragment, and finally a 4096 byte block.
The fragment reallocation can be avoided if the user program writes a full block at a time,

except for a partial block at the end of the file.

Because file systems with different block sizes

may coexist on the same system, the file system interface been extended to provide the ability

to determine the optimal size for a read or write.

For files the optimal size is the block size of

the file system on which the file is being accessed.

For other objects, such as pipes and sock-

ets, the optimal size is the underlying buffer size.

This feature is used by the Standard

Input/Output Library, a package used by most user programs.

This feature is also used by

certain system utilities such as archivers and loaders that do their own input and output
management and need the highest possible file system bandwidth.

The space overhead in the 4096/1024 byte new file system organization is empirically

observed to be about the same as in the 1024 byte old file system organization. A file system
with 4096 byte blocks and 512 byte fragments has about the same amount of space overhead

as the 512 byte block UNIX file system.

The new file system is more space efficient than the

512 byte or 1024 byte file systems in that it uses the same amount of space for small files
while requiring less indexing information for large files.

This savings is offset by the need to

use more space for keeping track of available free blocks.

The net result is about the same

disk utilization when the new file systems fragment size equals the old file systems block size.
In order for the layout policies to be effective, the disk cannot be kept completely full.
Each file system maintains a parameter that gives the minimum acceptable percentage of file

system blocks that can be free.

If the the number of free blocks drops below this level only

the system administrator can continue to allocate blocks.

The value of this parameter can be

changed at any time, even when the file system is mounted and active.

The transfer rates to

be given in section 4 were measured on file systems kept less than 90% full.

If the reserve of

free blocks is set to zero, the file system throughput rate tends to be cut in half, because of
the inability of the file system to localize the blocks in a file.

If the performance is impaired

because of overfilling, it may be restored by removing enough files to obtain 10% free space.
Access speed for files created during periods of little free space can be restored by recreating
them once enough space is available.

The amount of free space maintained must be added to

the percentage of waste when comparing the organizations given in Table 1.

Thus, a site run-

ning the old 1024 byte UNIX file system wastes 11.8% of the space and one could expect to
fit the same amount of data into a 4096/512 byte new file system with 5% free space, since a

512 byte old file system wasted 6.9% of the space.
3.2. File system parameterization
Except for the initial creation of the free list, the old file system ignores the parameters

of the underlying hardware.

It has no information about either the physical characteristics of

the mass storage device, or the hardware that interacts with it.

A goal of the new file system

is to parameterize the processor capabilities and mass storage characteristics so that blocks
can be allocated in an optimum configuration dependent way. Parameters used include the

speed of the processor, the hardware support for mass storage transfers, and the characteristics

of the mass storage devices.

Disk technology is constantly improving and a given

Fast File System

1-29

installation can have several different disk technologies running on a single processor. Each
file system is parameterized so that it can adapt to the characteristics of the disk on which it
is placed.
For mass storage devices such as disks, the new file system tries to allocate new blocks

on the same cylinder as the previous block in the same file. Optimally, these new blocks will
also be well positioned rotationally. The distance between “rotationally optimal” blocks varies
greatly; it can be a consecutive block or a rotationally delayed block depending on system
characteristics. On a processor with a channel that does not require any processor intervention between mass storage transfer requests, two consecutive disk blocks often can be accessed
without suffering lost time because of an intervening disk revolution. For processors without
such channels, the main processor must field an interrupt and prepare for a new disk transfer.
The expected time to service this interrupt and schedule a new disk transfer depends on the
speed of the main processor.

The physical characteristics of each disk include the number of blocks per track and the
rate at which the disk spins.

The allocation policy routines use this information to calculate

the number of milliseconds required to skip over a block. The characteristics of the processor
include the expected time to schedule an interrupt. Given the previous block allocated to a
file, the allocation routines calculate the number of blocks to skip over so that the next block
in a file will be coming into position under the disk head in the expected amount of time that
it takes to start a new disk transfer operation.

For programs that sequentially access large

amounts of data, this strategy minimizes the amount of time spent waiting for the disk to
position itself.

To ease the calculation of finding rotationally optimal blocks, the cylinder group summary information includes a count of the availability of blocks at different rotational posi-

tions. Eight rotational positions are distinguished, so the resolution of the summary information is 2 milliseconds for a typical 3600 revolution per minute drive.
The parameter that defines the minimum number of milliseconds between the comple-

tion of a data transfer and the initiation of another data transfer on the same cylinder can be
changed at any time, even when the file system is mounted and active. If a file system is
parameterized to lay out blocks with rotational separation of 2 milliseconds, and the disk pack
is then moved to a system that has a processor requiring 4 milliseconds to schedule a disk

operation, the throughput will drop precipitously because of lost disk revolutions on nearly
every block. If the eventual target machine is known, the file system can be parameterized for
it even though it is initially created on a different processor.

Even if the move is not known

in advance, the rotational layout delay can be reconfigured after the disk is moved so that all
further allocation is done based on the characteristics of the new host.

3.3. Layout policies

The file system policies are divided into two distinct parts.

At the top level are global

policies that use file system wide summary information to make decisions regarding the placement of new inodes and data blocks. These routines are responsible for deciding the placement of new directories and files. They also calculate rotationally optimal block layouts, and
decide when to force a long seek to a new cylinder group because there are insufficient blocks
left in the current cylinder group to do reasonable layouts. Below the global policy routines
are the local allocation routines that use a locally optimal scheme to lay out data blocks.
Two methods for improving file system performance are to increase the locality of refer-

ence to minimize seek latency as described by [Trivedi80], and to improve the layout of data

to make larger transfers possible as described by [Nevalainen77].

The global layout policies

try to improve performance by clustering related information. They cannot attempt to localize all data references, but must also try to spread unrelated data among different cylinder
groups.

If too much localization is attempted, the local cylinder group may run out of space

forcing the data to be scattered to non-local cylinder groups. Taken to an extreme, total localization can result in a single huge cluster of data resembling the old file system.

The global

1-30 Fast File System

policies try to balance the two conflicting goals of localizing data that is concurrently accessed
while spreading out unrelated data.

One allocatable resource is inodes. Inodes are used to describe both files and directories.
Files in a directory are frequently accessed together. For example the “list directory” command often accesses the inode for each file in a directory. The layout policy tries to place all
the files in a directory in the same cylinder group. To ensure that files are allocated
throughout the disk, a different policy is used for directory allocation. A new directory is
placed in the cylinder group that has a greater than average number of free inodes, and the
fewest number of directories in it already. The intent of this policy is to allow the file clustering policy to succeed most of the time. The allocation of inodes within a cylinder group is
done using a next free strategy. Although this allocates the inodes randomly within a cylinder
group, all the inodes for each cylinder group can be read with 4 to 8 disk transfers. This puts
a small and constant upper bound on the number of disk transfers required to access all the
inodes for all the files in a directory as compared to the old file system where typically, one
disk transfer is needed to get the inode for each file in a directory.
The other major resource is the data blocks. Since data blocks for a file are typically
accessed together, the policy routines try to place all the data blocks for a file in the same
cylinder group, preferably rotationally optimally on the same cylinder. The problem with allocating all the data blocks in the same cylinder group is that large files will quickly use up
available space in the cylinder group, forcing a spill over to other areas. Using up all the
space in a cylinder group has the added drawback that future allocations for any file in the
cylinder group will also spill to other areas. Ideally none of the cylinder groups should ever
become completely full. The solution devised is to redirect block allocation to a newly chosen
cylinder group when a file exceeds 32 kilobytes, and at every megabyte thereafter. The newly
chosen cylinder group is selected from those cylinder groups that have a greater than average
number of free blocks left. Although big files tend to be spread out over the disk, a megabyte
of data is typically accessible before a long seek must be performed, and the cost of one long
seek per megabyte is small.

The global policy routines call local allocation routines with requests for specific blocks.
The local allocation routines will always allocate the requested block if it is free. If the
requested block is not available, the allocator allocates a free block of the requested size that
is rotationally closest to the requested block. If the global layout policies had complete information, they could always request unused blocks and the allocation routines would be reduced
to simple bookkeeping. However, maintaining complete information is costly; thus the implementation of the global layout policy uses heuristic guesses based on partial information.
If a requested block is not available the local allocator uses a four level allocation strategy:

1)
2)

Use the available block rotationally closest to the requested block on the same cylinder.
If there are no blocks available on the same cylinder, use a block within the same
cylinder group.

If the cylinder group is entirely full, quadratically rehash among the cylinder groups
looking for a free block.

Finally if the rehash fails, apply an exhaustive search.

The use of quadratic rehash is prompted by studies of symbol table strategies used in
programming languages. File systems that are parameterized to maintain at least 10% free
space almost never use this strategy; file systems that are run without maintaining any free
space typically have so few free blocks that almost any allocation is random. Consequently
the most important characteristic of the strategy used when the file system is low on space is
that it be fast.

Fast File System 1-31
4. Performance

Ultimately, the proof of the effectiveness of the algorithms described in the previous sec-

tion is the long term performance of the new file system.

Our empiric studies have shown that the inode layout policy has been effective. When
running the “list directory” command on a large directory that itself contains many directories, the number of disk accesses for inodes is cut by a factor of two. The improvements are
even more dramatic for large directories containing only files, disk accesses for inodes being
cut by a factor of eight. This is most encouraging for programs such as spooling daemons that
access many small files, since these programs tend to flood the disk request queue on the old
file system.

Table 2 summarizes the measured throughput of the new file system. Several comments
need to be made about the conditions under which these tests were run. The test programs
measure the rate that user programs can transfer data to or from a file without performing

any processing on it. These programs must write enough data to insure that buffering in the
operating system does not affect the results. They should also be run at least three times in
succession; the first to get the system into a known state and the second two to insure that the
experiment has stabilized and is repeatable. The methodology and test results are discussed
in detail in [Kridle83]f. The systems were running multi-user but were otherwise quiescent.
There was no contention for either the cpu or the disk arm. The only difference between the
UNIBUS and MASSBUS tests was the controller. All tests used an Ampex Capricorn 330
Megabyte Winchester disk. As Table 2 shows, all file system test runs were on a VAX 11/750.
All file systems had been in production use for at least a month before being measured.
Type of

File System
old 1024
new 4096/1024
new 8192/1024
new 4096/1024
new 8192/1024

Read

Processor and

Bandwidth
Speed
Bus Measured
29/1100 3%
29 Kbytes/sec
750/UNIBUS
221/1100 20%
221 Kbytes/sec
750/UNIBUS
233/1100 21%
233 Kbytes/sec
750/UNIBUS
750/MASSBUS | 466 Kbytes/sec = 466/1200 39%
466/1200 39%
750/MASSBUS | 466 Kbytes/sec

% CPU
11%
%
43
%
29
%
73
%
54

Table 2a — Reading rates of the old and new UNIX file systems.

Processor and
Bus Measured

Type of
_File System

old 1024
new 4096/1024
new 8192/1024
new 4096/1024
new 8192/1024

Speed

48 Kbytes/sec
750/UNIBUS
Kbytes/sec
142
750/UNIBUS
Kbytes/sec
215
|
750/UNIBUS
Kbytes/sec
323
|
750/MASSBUS
Kbytes/sec
466
|
S
750/MASSBU

Write
Bandwidth

% CPU

48/1100 4%
142/1100 13%
215/1100 19%
323/1200 27%
466/1200 39%

29 %
43 %
%
46
%
94
95 %

Table 2b — Writing rates of the old and new UNIX file systems.

Unlike the old file system, the transfer rates for the new file system do not appear to
change over time. The throughput rate is tied much more strongly to the amount of free

space that is maintained. The measurements in Table 2 were based on a file system run with
10% free space. Synthetic work loads suggest the performance deteriorates to about half the
throughput rates given in Table 2 when no free space is maintained.
The percentage of bandwidth given in Table 2 is a measure of the effective utilization of
the disk by the file system. An upper bound on the transfer rate from the disk is measured by
doing 65536* byte reads from contiguous tracks on the disk. The bandwidth is calculated by
ABLS
B command
P
+‘ FaA UNIX

eight Megabytes long.

hat is similar to the reading test that we used is, “cp file /dev/null”, where “file” is

vaa

* This number, 65536, is the maximal I/O size supported by the VAX hardware; it is a remnant of the

1-32 Fast File System

comparing the data rates the file system is able to achieve as a percentage of this rate. Using
this metric, the old file system is only able to use about 3-4% of the disk bandwidth, while the
new file system uses up to 39% of the bandwidth.
In the new file system, the reading rate is always at least as fast as the writing rate.
This is to be expected since the kernel must do more work when allocating blocks than when
simply reading them. Note that the write rates are about the same as the read rates in the
8192 byte block file system; the write rates are slower than the read rates in the 4096 byte
block file system. The slower write rates occur because the kernel has to do twice as many
disk allocations per second, and the processor is unable to keep up with the disk transfer rate.

In contrast the old file system is about 50% faster at writing files than reading them.
This is because the write system call is asynchronous and the kernel can generate disk
transfer requests much faster than they can be serviced, hence disk transfers build up in the
disk buffer cache. Because the disk buffer cache is sorted by minimum seek order, the average
seek between the scheduled disk writes is much less than they would be if the data blocks are
written out in the order in which they are generated. However when the file is read, the read
system call is processed synchronously so the disk blocks must be retrieved from the disk in
the order in which they are allocated. This forces the disk scheduler to do long seeks resulting
in a lower throughput rate.

The performance of the new file system is currently limited by a memory to memory
copy operation because it transfers data from the disk into buffers in the kernel address space
and then spends 40% of the processor cycles copying these buffers to user address space. If
the buffers in both address spaces are properly aligned, this transfer can be affected without
copying by using the VAX virtual memory management hardware. This is especially desirable
when large amounts of data are to be transferred. We did not implement this because it
would change the semantics of the file system in two major ways; user programs would be
required to allocate buffers on page boundaries, and data would disappear from buffers after
being written.

Greater disk throughput could be achieved by rewriting the disk drivers to chain
together kernel buffers. This would allow files to be allocated to contiguous disk blocks that
could be read in a single disk transaction. Most disks contain either 32 or 48 512 byte sectors
per track. The inability to use contiguous disk blocks effectively limits the performance on
these disks to less than fifty percent of the available bandwidth. Since each track has a multiple of sixteen sectors it holds exactly two or three 8192 byte file system blocks, or four or six
4096 byte file system blocks. If the the next block for a file cannot be laid out contiguously,
then the minimum spacing to the next allocatable block on any platter is between a sixth and
a half a revolution. The implication of this is that the best possible layout without contiguous
blocks uses only half of the bandwidth of any given track. If each track contains an odd
number of sectors, then it is possible to resolve the rotational delay to any number of sectors
by finding a block that begins at the desired rotational position on another track. The reason
that block chaining has not been implemented is because it would require rewriting all the
disk drivers in the system, and the current throughput rates are already limited by the speed
of the available processors.

Currently only one block is allocated to a file at a time. A technique used by the
DEMOS file system when it finds that a file is growing rapidly, is to preallocate several blocks
at once, releasing them when the file is closed if they remain unused. By batching up the allocation the system can reduce the overhead of allocating at each write, and it can cut down on
the number of disk writes needed to keep the block pointers on the disk synchronized with
the block allocation [Powell79].

system’s PDP-11 ancestry.

Fast File System

1-33

5. File system functional enhancements

The speed enhancements to the UNIX file system did not require any changes to the
semantics or data structures viewed by the users. However several changes have been generally desired for some time but have not been introduced because they would require users to
dump and restore all their file systems. Since the new file system already requires that all
existing file systems be dumped and restored, these functional enhancements have been introduced at this time.

5.1. Long file names

File names can now be of nearly arbitrary length. The only user programs affected by
this change are those that access directories. To maintain portability among UNIX systems
that are not running the new file system, a set of directory access routines have been introduced that provide a uniform interface to directories on both old and new systems.
Directories are allocated in units of 512 bytes. This size is chosen so that each allocation
can be transferred to disk in a single atomic operation. KEach allocation unit contains
variable-length directory entries. Each entry is wholly contained in a single allocation unit.
The first three fields of a directory entry are fixed and contain an inode number, the length of
the entry, and the length of the name contained in the entry. Following this fixed size information is the null terminated name, padded to a 4 byte boundary. The maximum length of a
name in a directory is currently 255 characters.

Free space in a directory is held by entries that have a record length that exceeds the
space required by the directory entry itself. All the bytes in a directory unit are claimed by
the directory entries. This normally results in the last entry in a directory being large. When
entries are deleted from a directory, the space is returned to the previous entry in the same
directory unit by increasing its length. If the first entry of a directory unit is free, then its
inode number is set to zero to show that it is unallocated.
5.2. File locking

The old file system had no provision for locking files. Processes that needed to synchronize the updates of a file had to create a separate “lock” file to synchronize their updates.
A process would try to create a “lock” file. If the creation succeeded, then it could proceed
with its update; if the creation failed, then it would wait, and try again. This mechanism had
three drawbacks. Processes consumed CPU time, by looping over attempts to create locks.
Locks were left lying around following system crashes and had to be cleaned up by hand.
Finally, processes running as system administrator are always permitted to create files, so they
had to use a different mechanism. While it is possible to get around all these problems, the
solutions are not straight-forward, so a mechanism for locking files has been added.
The most general schemes allow processes to concurrently update a file. Several of these
techniques are discussed in [Peterson83]. A simpler technique is to simply serialize access
with locks. To attain reasonable efficiency, certain applications require the ability to lock
pieces of a file. Locking down to the byte level has been implemented in the Onyx file system
by [Bass81]. However, for the applications that currently run on the system, a mechanism
that locks at the granularity of a file is sufficient.

Locking schemes fall into two classes, those using hard locks and those using advisory
locks. The primary difference between advisory locks and hard locks is the decision of when
to override them. A hard lock is always enforced whenever a program tries to access a file; an
advisory lock is only applied when it is requested by a program. Thus advisory locks are only
effective when all programs accessing a file use the locking scheme. With hard locks there
must be some override policy implemented in the kernel, with advisory locks the policy is
implemented by the user programs. In the UNIX system, programs with system administrator privilege can override any protection scheme. Because many of the programs that need to
use locks run as system administrators, we chose to implement advisory locks rather than
create a protection scheme that was contrary to the UNIX philosophy or could not be used by

1-34 Fast File System

system administration programs.
The file locking facilities allow cooperating programs to apply advisory shared
exclusive locks on files.

Only one process has an exclusive lock on a file while multiple shared

locks may be present. Both shared and exclusive locks cannot be present on a file at the same
time.

If any lock is requested when another process holds an exclusive lock, or an exclusive

lock is requested when another process holds any lock, the open will block until the lock can

be gained.

Because shared and exclusive locks are advisory only, even if a process has

obtained a lock on a file, another process can override the lock by opening the same file
without a lock.
Locks can be applied or removed on open files, so that locks can be manipulated without

needing to close and reopen the file.

This is useful, for example, when a process wishes to

open a file with a shared lock to read some information, to determine whether an update is

required.

It can then get an exclusive lock so that it can do a read, modify, and write to

update the file in a consistent manner.
A request for a lock will cause the process to block if the lock can not be immediately

obtained.

In certain instances this is unsatisfactory.

For example, a process that wants only

to check if a lock is present would require a separate mechanism to find out this information.

Consequently, a process may specify that its locking request should return with an error if a
lock can not be immediately obtained.

Being able to poll for a lock is useful to “daemon”

processes that wish to service a spooling area.

If the first instance of the daemon locks the

directory where spooling takes place, later daemon processes can easily check to see if an
active daemon exists.

Since the lock is removed when the process exits or the system crashes,

there is no problem with unintentional locks files that must be cleared by hand.
Almost no deadlock detection is attempted.

The only deadlock detection made by the

system is that the file descriptor to which a lock is applied does not currently have a lock of

the same type (i.e. the second of two successive calls to apply a lock of the same type will fail).
Thus a process can deadlock itself by requesting locks on two separate file descriptors for the
same object.

5.3. Symbolic links
The 512 byte UNIX file system allows multiple directory entries in the same file system
to reference a single file.

The link concept is fundamental; files do not live in directories, but

exist separately and are referenced by links.

cated.

When all the links are removed, the file is deallo-

This style of links does not allow references across physical file systems, nor does it

support inter-machine linkage. To avoid these limitations symbolic links have been added

similar to the scheme used by Multics [Feiertag71].

A symbolic link is implemented as a file that contains a pathname.

When the system

encounters a symbolic link while interpreting a component of a pathname, the contents of the

symbolic link is prepended to the rest of the pathname, and this name is interpreted to yield
the resulting pathname.

If the symbolic link contains an absolute pathname, the absolute

pathname is used, otherwise the contents of the symbolic link is evaluated relative to the location of the link in the file hierarchy.

Normally programs do not want to be aware that there is a symbolic link in a pathname
that they are using.
symbolic links.

However certain system utilities must be able to detect and manipulate

Three new system calls provide the ability to detect, read, and write symbolic

links, and seven system utilities were modified to use these calls.
In future Berkeley software distributions it will be possible to mount file systems from

other machines within a local file system.

bolic links that span machines.

When this occurs, it will be possible to create sym-

Fast File System

1-35

5.4. Rename

Programs that create new versions of data files typically create the new version as a temporary file and then rename the temporary file with the original name of the data file. In the
old UNIX file systems the renaming required three calls to the system. If the program were
interrupted or the system crashed between these calls, the data file could be left with only its
temporary name. To eliminate this possibility a single system call has been added that performs the rename in an atomic fashion to guarantee the existence of the original name.

In addition, the rename facility allows directories to be moved around in the directory
tree hierarchy. The rename system call performs special validation checks to insure that the
directory tree structure is not corrupted by the creation of loops or inaccessible directories.
Such corruption would occur if a parent directory were moved into one of its descendants.
The validation check requires tracing the ancestry of the target directory to insure that it does
not include the directory being moved.
5.5. Quotas

The UNIX system has traditionally attempted to share all available resources to the
greatest extent possible. Thus any single user can allocate all the available space in the file
system. In certain environments this is unacceptable. Consequently, a quota mechanism has
been added for restricting the amount of file system resources that a user can obtain. The
quota mechanism sets limits on both the number of files and the number of disk blocks that a
user may allocate. A separate quota can be set for each user on each file system. KEach
resource is given both a hard and a soft limit. When a program exceeds a soft limit, a warning
is printed on the users terminal; the offending program is not terminated unless it exceeds its
hard limit. The idea is that users should stay below their soft limit between login sessions,
but they may use more space while they are actively working. To encourage this behavior,
users are warned when logging in if they are over any of their soft limits. If they fail to
correct the problem for too many login sessions, they are eventually reprimanded by having
their soft limit enforced as their hard limit.

1-36 Fast File System

Software engineering
The preliminary design was done by Bill Joy in late 1980; he presented the design at

The USENIX Conference held in San Francisco in January 1981. The implementation of his

design was done by Kirk McKusick in the summer of 1981. Most of the new system calls were
The code for enforcing quotas was implemented by Robert Elz
at the University of Melbourne.

1implemented by Sam Leffler.

To understand how the project was done it is necessary to understand the interfaces that
the UNIX system provides to the hardware mass storage systems. At the lowest level is a raw
disk. This interface provides access to the disk as a linear array of sectors. Normally this
interface is only used by programs that need to do disk to disk copies or that wish to dump

file systems. However, user programs with proper access rights can also access this interface.
A disk is usually formated with a file system that is interpreted by the UNIX system to pro-

vide a directory hierarchy and files.

The UNIX system interprets and multiplexes requests
from user programs to create, read, write, and delete files by allocating and freeing inodes and

data blocks.
themselves.

The interpretation of the data on the disk could be done by the user programs
The reason that it is done by the UNIX system is to synchronize the user

requests, so that two processes do not attempt to allocate or modify the same resource simul-

taneously.

It also allows access to be restricted at the file level rather than at the disk level
and allows the common file system routines to be shared between processes.

The implementation of the new file system amounted to using a different scheme for for-

mating and interpreting the disk.

Since the synchronization and disk access routines themselves were not being changed, the changes to the file system could be developed by moving

the file system interpretation routines out of the kernel and into a user program.

Thus, the
first step was to extract the file system code for the old file system from the UNIX kernel and

change its requests to the disk driver to accesses to a raw disk. This produced a library of
routines that mapped what would normally be system calls into read or write operations on

the raw disk.

This library was then debugged by linking it into the system utilities that copy,

remove, archive, and restore files.

A new cross file system utility was written that copied files from the simulated file system to the one implemented by the kernel. This was accomplished by calling the simulation
library to do a read, and then writing the resultant data by using the conventional write sys-

tem call.

A similar utility copied data from the kernel to the simulated file system by doing a
conventional read system call and then writing the resultant data using the simulated file sys-

tem library.

The second step was to rewrite the file system simulation library to interpret the new file

- system.

By linking the new simulation library into the cross file system copying utility, it was
possible to easily copy files from the old file system into the new one and from the new one to

the old one.

Having the file system interpretation implemented in user code had several
These included being able to use the standard system tools such as the

major benefits.

debuggers to set breakpoints and single step through the code. When bugs were discovered,
the offending problem could be fixed and tested without the need to reboot the machine.
There was never a period where it was necessary to maintain two concurrent file systems in

the kernel.

Finally it was not necessary to dedicate a machine entirely to file system develop-

ment, except for a brief period while the new file system was boot strapped.

The final step was to merge the new file system back into the UNIX kernel. This was
done in less than two weeks, since the only bugs remaining were those that involved interfacing to the synchronization routines that could not be tested in the simulated system.

Again
the simulation system proved useful since it enabled files to be easily copied between old and

new file systems regardless of which file system was running in the kernel.
reduced the number of times that the system had to be rebooted.

This greatly

The total design and debug time took about one man year. Most of the work was done
on the file system utilities, and changing all the user programs to use the new facilities. The

code changes in the kernel were minor, involving the addition of only about 800 lines of code
(including comments).

Fast File System

1-37

Acknowledgements

We thank Robert Elz for his ongoing interest in the new file system, and for adding disk
quotas in a rational and efficient manner. We also acknowledge Dennis Ritchie for his suggestions on the appropriate modifications to the user interface. We appreciate Michael Powell’s
explanations on how the DEMOS file system worked; many of his ideas were used in this
implementation. Special commendation goes to Peter Kessler and Robert Henry for acting
like real users during the early debugging stage when files were less stable than they should

have been. Finally we thank our sponsors, the National Science Foundation under grant
MCS80-05144, and the Defense Advance Research Projects Agency (DoD) under Arpa Order
No. 4031 monitored by Naval Electronic System Command under Contract No. N00039-82-C0235.

References
[Accetta80]

[Almes78]

Accetta, M., Robertson, G., Satyanarayanan, M., and Thompson, M.
"The Design of a Network-Based Central File System”, Carnegie-Mellon
University, Dept of Computer Science Tech Report, #CMU-CS-80-134
Almes, G., and Robertson, G. ”An Extensible File System for Hydra”
Proceedings of the Third International Conference on Software
Engineering, IEEE, May 1978.

[Bass81]

Bass, J. ”"Implementation Description for File Locking”, Onyx Systems

[Dion8&0]

Dion, J. "The Cambridge File Server”, Operating Systems Review, 14, 4.

Inc, 73 E. Trimble Rd, San Jose, CA 95131 Jan 1981.
Oct 1980. pp 26-35

[Eswaran74]

Eswaran, K. "Placement of records in a file and file allocation in a com-

[Holler73]

Holler, J. ”Files in Computer Networks”, First European Workshop on

[Feiertag71]

Feiertag, R. J. and Organick, E. 1., "The Multics Input-Output System”,
Proceedings of the Third Symposium on Operating Systems Principles,

puter network”, Proceedings IFIPS, 1974. pp 304-307

Computer Networks, April 1973. pp 381-396

ACM, Oct 1971. pp 35-41
[Kridle83]

Kridle, R., and McKusick, M., "Performance Effects of Disk Subsystem
Choices for VAX Systems Running 4.2BSD UNIX”, Computer Systems
Research Group, Dept of EECS, Berkeley, CA 94720, Technical Report
#8.

[Kowalski78]

[Luniewski77]

[Maruyama’76]

Kowalski, T.

”"FSCK - The UNIX System Check Program”, Bell

Laboratory, Murray Hill, NJ 07974. March 1978

Luniewski, A. ”File Allocation in a Distributed System”, MIT Labora|

tory for Computer Science, Dec 1977.

Maruyama, K., and Smith, S. ”Optimal reorganization of Distributed
Space Disk Files”, Communications of the ACM, 19, 11. Nov 1976. pp
634-642

[Nevalainen77]

Nevalainen, O., Vesterinen, M. ”Determining Blocking Factors for
Sequential Files by Heuristic Methods”, The Computer Journal, 20, 3.
Aug 1977. pp 245-247

[Peterson83]

Peterson, G. ”Concurrent Reading While Writing”, ACM Transactions
on Programming Languages and Systems, ACM, 5, 1. Jan 1983. pp 46-55

Powell, M. "The DEMOS File System”, Proceedings of the Sixth Symposium on Operating Systems Principles, ACM, Nov 1977. pp 33-42

1-38 Fast File System

[Porcar82]

Porcar, J. ”File Migration in Distributed Computer Systems”, Ph.D.
Thesis, Lawrence Berkeley Laboratory Tech Report #LBL-14763.

[Ritchie74]

Ritchie, D. M. and Thompson, K., ”The UNIX Time-Sharing System?”,

[Smith81a]

CACM 17, 7. July 1974. pp 365-375
Smith, A. ”"Input/Output Optimization and Disk Architectures: A Survey”, Performance and Evaluation 1. Jan 1981. pp 104-117

[Smith81b]

Smith, A. ”Bibliography on File and I/O System Optimization and
Related Topics”, Operating Systems Review, 15, 4. Oct 1981. pp 39-54

[Sturgis80]

Sturgis, H., Mitchell, J., and Israel, J. "Issues in the Design and Use of
a Distributed File System”, Operating Systems Review, 14, 3. pp 55-79

[Symbolics81a]

"Symbolics File System”, Symbolics Inc, 9600 DeSoto Ave, Chatsworth,
CA 91311 Aug 1981.

[Symbolics81b]

"Chaosnet FILE Protocol”.
worth, CA 91311 Sept 1981.

Symbolics Inc, 9600 DeSoto Ave, Chats-

[Thompson79]

Thompson, K. "UNIX Implementation”, Section 31, Volume 2B, UNIX
Programmers Manual, Bell Laboratory, Murray Hill, NJ 07974. Jan 1979

[Thompson80]

Thompson, M.

[Trivedi80]

Trivedi, K.

[White80]

”Spice File System”, Carnegie-Mellon University, Dept
of Computer Science Tech Report, #CMU-CS-80-22?
”Optimal Selection of CPU Speed, Device Capabilities, and
File Assignments”, Journal of the ACM, 27, 3. July 1980. pp 457-473

White, R. M.

August 1980.

”Disk Storage Technology”, Scientific American, 243(2),

Introduction 2-1

PART 2: MAINTENANCE AND ADMINISTRATION

The three articles in this part describe system administration utilities on ULTRIX-32. Two
of the utilities, quota and the file system check program (fsck), will help you keep your system
running efficiently. The third utility, sendmail, makes possible communication between users
on computers that use different networking software.

Disk Quotas

The ULTRIX-32 system allows the system manager to impose limits on the amount of disk
space and the number of files available to each user. Each category (disk space and the maximum number of files) has a hard limit and a soft limit. The hard limit for a user sets an
absolute maximum that cannot be exceeded. The soft limit is a guideline: the number of
blocks or files that the user should try not to exceed. The quota utility warns any user who
exceeds his or her soft limit. If the user consistently ignores the warnings, the soft limit
becomes a hard limit after a set number of warnings.

The article, “Disc Quotas in a UNIX Environment,” by Elz, tells how the system manager can
establish, disable, or check the limits and the number of warnings for any user. Elz also

explains how a user can exit without loss from an editing session in which writing the edited
material to a file would exceed one of the hard limits.

Fixing Corrupted File Systems

The ULTRIX-32 system includes a file system check program called fsck. You can use this
utility to determine whether your file system is corrupted and to fix any inconsistencies you
find.

Fsck runs in two modes: noninteractive and interactive. Normally the boot procedure calls
fsck to run noninteractively after booting the operating system. In this mode, the utility
checks for inconsistencies and corrects only those that it can handle without help from an
operator. In general, these are problems associated with a system crash or improper shutdown
procedure. When the utility finds a problem it can’t deal with, it notifies the operator and
stops. The operator can then run fsck interactively, deciding between the alternative measures presented by the utility.

The article by McKusick, “Fsck - The UNIX File System Check Program,” gives an overview
of the file system, the kinds of corruption that can occur, and the methods that fsck uses to
check for inconsistencies. An appendix provides a comprehensive list of error messages
together with explanations and appropriate responses. Fsck is essential to proper maintenance
of the ULTRIX-32 system, and this article is essential to proper use of fsck.

2-2 Introduction

Managing the Sendmail Utility

Sendmail is an internetwork mail utility transparent to most users. Once it is installed and
running, you can send mail to users on foreign network systems in the same way that you send
mail to users on the local network. The sendmail utility handles the protocol and messagerouting differences between networks automatically.
The “Sendmail Installation and Operation Guide,” by Allman, tells what you need to know to
start up the utility and to keep it running correctly from day to day. A second article, “Sendmail - An Internetwork Mail Router,” in Part 3 of this volume, gives background information
that tells how sendmail works. Read the background article before using the installation and
operating information included in this part.

The installation information in the “Sendmail Installation and Operation Guide” explains:
e

How to use either of two off-the-shelf configuration files supplied with the software

How to use a makefile to install sendmail automatically

How to install sendmail by hand by building your own configuration file and setting
up the sendmail startup procedure on your ULTRIX-32 system

The day-to-day sendmail operations explained include:

Use of the system log for records and debugging

Mail queue processing

Treatment of address aliases

The mail-forwarding feature

Special headers for return receipts and error situations

The article describes parameters you can adjust to tune sendmail to suit a specific site. If you
must build your own configuration file, you will find the list of configuration file rules and
hints to be helpful. And for expert system managers, the appendixes list detailed sendmail
information in five categories:

Command line flags

Configuration options

Mailer flags

Compilation options (other configuration)

Support files

Disk Quotas

2-3

Disc Quotas in a UNIX" Environment
Robert Elz

Department of Computer Science
University of Melbourne,
Parkville,
Victoria,
Australia.

1. Users’ view of disc quotas

To most users, disc quotas will either be of no concern, or a fact of life that cannot be
avoided. The quota (1) command will provide information on any disc quotas that may have
been imposed upon a user.

There are two individual possible quotas that may be imposed, usually if one is, both
will be. A limit can be set on the amount of space a user can occupy, and there may be a
limit on the number of files (inodes) he can own.

Quota provides information on the quotas that have been set by the system administrators, in each of these areas, and current usage.

There are four numbers for each limit, the current usage, soft limit (quota), hard limit,
and number of remaining login warnings. The soft limit is the number of 1K blocks (or files)
that the user is expected to remain below. Each time the user’s usage goes past this limit, he
will be warned. The hard limit cannot be exceeded. If a user’s usage reaches this number,
further requests for space (or attempts to create a file) will fail with an EDQUOT error, and
the first time this occurs, a message will be written to the user’s terminal. Only one message
will be output, until space occupied is reduced below the limit, and reaches it again, in order
to avoid continual noise from those programs that ignore write errors.

Whenever a user logs in with a usage greater than his soft limit, he will be warned, and
his login warning count decremented. When he logs in under quota, the counter is reset to its
maximum value (which is a system configuration parameter, that is typically 3). If the warn-

ing count should ever reach zero (caused by three successive logins over quota), the particular

limit that has been exceeded will be treated as if the hard limit has been reached, and no
more resources will be allocated to the user. The only way to reset this condition is to reduce
usage below quota, then log in again.

1.1. Surviving when quota limit is reached

In most cases, the only way to recover from over quota conditions, is to abort whatever
activity was in progress on the filesystem that has reached its limit, remove sufficient files to

bring the limit back below quota, and retry the failed program.

However, if you are in the editor and a write fails because of an over quota situation,
that is not a suitable course of action, as it is most likely that initially attempting to write the
file will have truncated its previous contents, so should the editor be aborted without correctly
writing the file not only will the recent changes be lost, but possibly much, or even all, of the
data that previously existed.

* UNIX is a trademark of Bell Laboratories.

2-4 Disk Quotas
There are several possible safe exits for a user caught in this situation.

He may use the

editor ! shell escape command to examine his file space, and remove surplus files.

Alterna-

tively, using csh, he may suspend the editor, remove some files, then resume it. A third possibility, is to write the file to some other filesystem (perhaps to a file on /tmp) where the user’s
quota has not been exceeded.

Then after rectifying the quota situation, the file can be moved

back to the filesystem it belongs on.
2. Administering the quota system
To set up and establish the disc quota system, there are several steps necessary to be
performed by the system administrator.
First, the system must be configured to include the disc quota sub-system.

This is done

by including the line:
options

QUOTA

in the system configuration file, then running config (8) followed by a system configuration”.

Second, a decision as to what filesystems need to have quotas applied needs to be made.
Usually, only filesystems that house users’ home directories, or other user files, will need to be

subjected to the quota system, though it may also prove useful to also include /usr.

If possi-

ble, /tmp should usually be free of quotas.
Having decided on which filesystems quotas need to be set upon, the administrator

should then allocate the available space amongst the competing needs. How this should be
done is (way) beyond the scope of this document.
Then, the edquota (8) command can be used to actually set the limits desired upon each
user. Where a number of users are to be given the same quotas (a common occurrence) the —p

switch to edquota will allow this to be easily accomplished.

Once the quotas are set, ready to operate, the system must be informed to enforce quotas on the desired filesystems.

This is accomplished with the quotaon (8) command.

Quotaon

will either enable quotas for a particular filesystem, or with the —a switch, will enable quotas
for each filesystem indicated in /etc/fstab as using quotas.
sites using the quota system, will include the line

See fstab(5) for details.

Most

/ete/quotaon -a
in /ete/relocal.

Should quotas need to be disabled, the quotaoff(8) command will do that, however,
should the filesystem be about to be dismounted, the umount (8) command will disable quotas
immediately before the filesystem is unmounted. This is actually an effect of the umount (2)
system call, and it guarantees that the quota system will not be disabled if the umount would

fail because the filesystem is not idle.
Periodically (certainly after each reboot, and when quotas are first enabled for a filesys-

tem), the records retained in the quota file should be checked for consistency with the actual
number of blocks and files allocated to the user.
accomplish this.

The quotachk (8) command can be used to

It is not necessary to dismount the filesystem, or disable the quota system to

run this command, though on active filesystems inaccurate results may occur.

This does no

real harm in most cases, another run of quotachk when the filesystem is idle will certainly
correct any inaccuracy.

The super-user may use the quota (1) command to examine the usage and quotas of any

user, and the repquota (8) command may be used to check the usages and limits for all users
on a filesystem.

* See also the document “Building 4.2BSD UNIX Systems with Config”.

Disk Quotas 2-5

3. Some implementation detail

are
Disc quota usage and information is stored in a file on the filesystem that the quotas
While this

filesystem.
to be applied to. Conventionally, this file is quotas in the root of the utilities
"know” it, and
level
name is not known to the system in any way, several of the user
choosing any other name would not be wise.

re for
The data in the file comprises an array of structures, indexed by uid, one structu
uid
the
If
not).
or
em
filesyst
this
on
each user on the system (whether the user has a quota
best
is
it
so
copying,
by
lost
be
would
space is sparse, then the file may have holes in it, which
to avoid this.

call.
The system is informed of the existence of the quota file by the setquota(2) system
by
owned
open
files
any
for
then
active,
y
currentl
It then reads the quota entries for each user
be
will
em,
filesyst
the
on
file
a
of
open
ent
users who are not currently active. Each subsequ
be
will
ion
informat
this
cases
most
In
ion.
informat
accompanied by a pairing with its quota
because
process,
some
running
is
file
the
owns
who
retained in core, either because the user
this one) was
other files are open owned by the same user, or because some file (perhaps and
filesystem,
user-id
by
hashed
kept
is
tion
informa
recently accessed. In memory, the quota
tion
Informa
d.
reclaime
easily
be
can
data
released
and retained in an LRU chain so recently
way.
this
in
d
retaine
also
is
ted
termina
y
recentl
has
about those users whose last process
freed,
Each time a block is accessed or released, and each time an inode is allocated ornity
to
the quota system gets told about it, and in the case of allocations, gets the opportu
object.

Measurements have shown that the quota code uses a very small percentage of the sys-

tem cpu time consumed in writing a new block to disc.
4. Acknowledgments

at
The current disc quota system is loosely based upon a very early scheme implemented
system
That
70’s.
mid
the University of New South Wales, and Sydney University in the
implemented a single combined limit for both files and blocks on all filesystems.

but was
A later system was implemented at the University of Melbourne by the author,
to a file other

not kept highly accurately, eg: chown’s (etc) did not affect quotas, nor did i/o

- than one owned by the instigator.

82
The current system has been running (with only minor modifications) since January
scheme,
control
resource
broader
much
a
at Melbourne. It is actually just a small part of
which is capable of controlling almost anything that is usually uncontrolled in unix. The rest

of this is, as yet, still in a state where it is far too subject to change to be considered for distribution.

ate
For the 4.2BSD release, much work has been done to clean up and sanelyiaincorpor
at Berke-

the quota code by Sam Leffler and Kirk McKusick at The University of Californ
ley.

Fsck 2-7

Fsck — The UNIXT File System Check Program
Revised July 28, 1983
Marshall Kirk McKusick

Computer Systems Research Group
Computer Science Division
Department of Electrical Engineering and Computer Science
University of California, Berkeley

Berkeley, CA 94720
T. J. Kowalskt
Bell Laboratories

Murray Hill, New Jersey 07974

1. Introduction

This document reflects the use of fsck with the 4.2BSD file system organization. This is

a revision of the original paper written by T. J. Kowalski.
When a UNIX operating system is brought up, a consistency check of the file systems
should always be performed.

This precautionary measure helps to insure a reliable environ-

ment for file storage on disk.

If an inconsistency is discovered, corrective action must be

taken.

Fsck runs in two modes.

Normally it is run non-interactively by the system after a

normal boot. When running in this mode, it will only make changes to the file system that are
known to always be correct.

If an unexpected inconsistency is found fsck will exit with a

non-zero exit status, leaving the system running single-user.

Typically the operator then runs

fsck interactively.

When running in this mode, each problem is listed followed by a suggested

corrective action.

The operator must decide whether or not the suggested correction should

be made.

The purpose of this memo is to dispel the mystique surrounding file system inconsisten-

cies.

It first describes the updating of the file system (the calm before the storm) and then

describes file system corruption (the storm).

Finally, the set of deterministic corrective

actions used by fsck (the Coast Guard to the rescue) is presented.

UNIX is a trademark of Bell Laboratories.
This work was done under grants from the National Science Foundation under grant MCS80-05144, and the
Defense Advance Research Projects Agency (DoD) under Arpa Order No. 4031 monitored by Naval Electronic System Command under Contract No. N00039-82-C-0235.

2-8 Fsck

2. Overview of the file system

The file system is discussed in detail in [Mckusick83]; this section gives a brief overview.
2.1. Superblock

A file system is described by its super-block. The super-block is built when the file system is created (newfs(8)) and never changes. The super-block contains the basic parameters
of the file system, such as the number of data blocks it contains and a count of the maximum
number of files. Because the super-block contains critical data, newfs replicates it to protect
against catastrophic loss. The default super block always resides at a fixed offset from the
beginning of the file system’s disk partition. The redundant super blocks are not referenced
unless a head crash or other hard disk error causes the default super-block to be unusable.
The redundant blocks are sprinkled throughout the disk partition.

Within the file system are files. Certain files are distinguished as directories and contain
collections of pointers to files that may themselves be directories. Every file has a descriptor
associated with it called an inode. The inode contains information describing ownership of
the file, time stamps indicating modification and access times for the file, and an array of

indices pointing to the data blocks for the file. In this section, we assume that the first 12

blocks of the file are directly referenced by values stored in the inode structure itselff. The
inode structure may also contain references to indirect blocks containing further data block
indices. In a file system with a 4096 byte block size, a singly indirect block contains 1024
further block addresses, a doubly indirect block contains 1024 addresses of further single
indirect blocks, and a triply indirect block contains 1024 addresses of further doubly indirect
blocks.

In order to create files with up to 2132 bytes, using only two levels of indirection, the
minimum size of a file system block is 4096 bytes. The size of file system blocks can be any
power of two greater than or equal to 4096. The block size of the file system is maintained in
the super-block, so it is possible for file systems of different block sizes to be accessible simultaneously on the same system. The block size must be decided when newfs creates the file
system; the block size cannot be subsequently changed without rebuilding the file system.
2.2. Summary information

Associated with the super block is non replicated summary information. The summary
information changes as the file system is modified. The summary information contains the

number of blocks, fragments, inodes and directories in the file system.
2.3. Cylinder groups

The file system partitions the disk into one or more areas called cylinder groups. A
cylinder group is comprised of one or more consecutive cylinders on a disk. Each cylinder
group includes inode slots for files, a block map describing available blocks in the cylinder
group, and summary information describing the usage of data blocks within the cylinder
group. A fixed number of inodes is allocated for each cylinder group when the file system is
created. The current policy is to allocate one inode for each 2048 bytes of disk space; this is
expected to be far more inodes than will ever be needed.

All the cylinder group bookkeeping information could be placed at the beginning of each
cylinder group. However if this approach were used, all the redundant information would be
on the top platter. A single hardware failure that destroyed the top platter could cause the
loss of all copies of the redundant super-blocks. Thus the cylinder group bookkeeping information begins at a floating offset from the beginning of the cylinder group. The offset for the
i+1st cylinder group is about one track further from the beginning of the cylinder group than
it was for the ith cylinder group. In this way, the redundant information spirals down into
mber may vary from system to system, but is usually in the range 5-13.

Fsck 2-9

the pack; any single track, cylinder, or platter can be lost without losing all copies of the
super-blocks. Except for the first cylinder group, the space between the beginning of the
cylinder group and the beginning of the cylinder group information stores data.
2.4. Fragments

To avoid waste in storing small files, the file system space allocator divides a single file
system block into one or more fragments. The fragmentation of the file system is specified
when the file system is created; each file system block can be optionally broken into 2, 4, or 8
addressable fragments. The lower bound on the size of these fragments is constrained by the
disk sector size; typically 512 bytes is the lower bound on fragment size. The block map associated with each cylinder group records the space availability at the fragment level. Aligned
fragments are examined to determine block availability.

On a file system with a block size of 4096 bytes and a fragment size of 1024 bytes, a file
is represented by zero or more 4096 byte blocks of data, and possibly a single fragmented
block. If a file system block must be fragmented to obtain space for a small amount of data,
the remainder of the block is made available for allocation to other files. For example, consider an 11000 byte file stored on a 4096/1024 byte file system. This file uses two full size
blocks and a 3072 byte fragment. If no fragments with at least 3072 bytes are available when
the file is created, a full size block is split yielding the necessary 3072 byte fragment and an
unused 1024 byte fragment. This remaining fragment can be allocated to another file, as
needed.

2.5. Updates to the file system

Every working day hundreds of files are created, modified, and removed. Every time a
file is modified, the operating system performs a series of file system updates. These updates,
when written on disk, yield a consistent file system. The file system stages all modifications of
critical information; modification can either be completed or cleanly backed out after a crash.
Knowing the information that is first written to the file system, deterministic procedures can
be developed to repair a corrupted file system. To understand this process, the order that the
update requests were being honored must first be understood.
When a user program does an operation to change the file system, such as a write, the
data to be written is copied into an internal in-core buffer in the kernel. Normally, the disk
update is handled asynchronously; the user process is allowed to proceed even though the data
has not yet been written to the disk. The data, along with the inode information reflecting
the change, is eventually written out to disk. The real disk write may not happen until long
after the write system call has returned. Thus at any given time, the file system, as it resides
on the disk, lags the state of the file system represented by the in-core information.
The disk information is updated to reflect the in-core information when the buffer is

required for another use, when a sync(2) is done (at 30 second intervals) by /etc/update (8), or
by manual operator intervention with the sync(8) command. If the system is halted without
writing out the in-core information, the file system on the disk will be in an inconsistent state.
If all updates are done asynchronously, several serious inconsistencies can arise. One
inconsistency is that a block may be claimed by two inodes. Such an inconsistency can occur

when the system is halted before the pointer to the block in the old inode has been cleared in
the copy of the old inode on the disk, and after the pointer to the block in the new inode has
been written out to the copy of the new inode on the disk. Here, there is no deterministic
method for deciding which inode should really claim the block. A similar problem can arise
with a multiply claimed inode.

The problem with asynchronous inode updates can be avoided by doing all inode deallocations synchronously. Consequently, inodes and indirect blocks are written to the disk synchronously (i.e. the process blocks until the information is really written to disk) when they
are being deallocated. Similarly inodes are kept consistent by synchronously deleting, adding,
or changing directory entries.

2-10 Fsck

3. Fixing corrupted file systems

A file system can become corrupted in several ways. The most common of these ways

are improper shutdown procedures and hardware failures.

File systems may become corrupted during an unclean halt. This happens when proper
shutdown procedures are not observed, physically write-protecting a mounted file system, or a
mounted file system is taken off-line. The most common operator procedural failure is forget-

ting to sync the system before halting the CPU.

File systems may become further corrupted if proper startup procedures are not
observed, e.g., not checking a file system for inconsistencies, and not repairing inconsistencies.
Allowing a corrupted file system to be used (and, thus, to be modified further) can be disastrous.

Any piece of hardware can fail at any time. Failures can be as subtle as a bad block on a

disk pack, or as blatant as a non-functional disk-controller.
3.1. Detecting and correcting corruption

Normally fsck is run non-interactively. In this mode it will only fix corruptions that are
expected to occur from an unclean halt. These actions are a proper subset of the actions that
fsck will take when it is running interactively. Throughout this paper we assume that fsck is
being run interactively, and all possible errors can be encountered. When an inconsistency is
discovered in this mode, fsck reports the inconsistency for the operator to chose a corrective
action.

A quiescent} file system may be checked for structural integrity by performing consistency checks on the redundant data intrinsic to a file system. The redundant data is either
read from the file system, or computed from other known values. The file system must be in
a quiescent state when fsck is run, since fsck is a multi-pass program.

In the following sections, we discuss methods to discover inconsistencies and possible
corrective actions for the cylinder group blocks, the inodes, the indirect blocks, and the data
blocks containing directory entries.
3.2. Super-block checking

The most commonly corrupted item in a file system is the summary information associated with the super-block. The summary information is prone to corruption because it is
modified with every change to the file system’s blocks or inodes, and is usually corrupted after
an unclean halt.

The super-block is checked for inconsistencies involving file-system size, number of
inodes, free-block count, and the free-inode count. The file-system size must be larger than
the number of blocks used by the super-block and the number of blocks used by the list of
inodes. The file-system size and layout information are the most critical pieces of information
for fsck. While there is no way to actually check these sizes, since they are statically determined by newfs, fsck can check that these sizes are within reasonable bounds. All other file
system checks require that these sizes be correct. If fsck detects corruption in the static
parameters of the default super-block, fsck requests the operator to specify the location of an
alternate super-block.

3.3. Free block checking

Fsck checks that all the blocks marked as free in the cylinder group block maps are not
claimed by any files. When all the blocks have been initially accounted for, fsck checks that
the number of free blocks plus the number of blocks claimed by the inodes equals the total
number of blocks in the file system.
4 I.e., unmounted and not bei ng wriitten on.

WTl

Fsck 2-11

If anything is wrong with the block allocation maps, fsck will rebuild them, based on the

list it has computed of allocated blocks.

of
The summary information associated with the super-block counts the total number
it
blocks
free
of
number
the
free blocks within the file system. Fsck compares this count to
t
incorrec
the
replaces
fsck
found within the file system. If the two counts do not agree, then
count in the summary information by the actual free-block count.

The summary information counts the total number of free inodes within the file system.
If the

Fsck compares this count to the number of free inodes it found within the file system.
two counts do not agree, then fsck replaces the incorrect count in the summary information
by the actual free-inode count.

3.4. Checking the inode state

An individual inode is not as likely to be corrupted as the allocation information. Howd.

ever, because of the great number of active inodes, a few of the inodes are usually corrupte

0
The list of inodes in the file system is checked sequentially starting with inode 2 (inode
last
the
through
ing
progress
and
ons)
generati
future
marks unused inodes; inode 1 is saved for
inode in the file system. The state of each inode is checked for inconsistencies involving format and type, link count, duplicate blocks, bad blocks, and inode size.

the
Each inode contains a mode word. This mode word describes the type and state of
inode,
link
c
symboli
inode,
y
director
inode,
regular
types:
six
inode. Inodes must be one of
one of
special block inode, special character inode, or socket inode. Inodes may be found. inThis
last
allocated
nor
ted
unalloca
neither
and
,
allocated
ed,
three allocation states: unallocat
is
data
bad
if
state
this
in
get
can
inode
An
inode.
state suggests an incorrectly formated
inode.
the
clear
to
is
fsck
for
is
action
e
correctiv
possible
only
written into the inode list. The
3.5. Inode links

Each inode counts the total number of directory entries linked to the inode. Fsck
ing
verifies the link count of each inode by starting at the root of the file system, anded descend
the
during
through the directory structure. The actual link count for each inode is calculat
descent.

If the stored link count is non-zero and the actual link count is zero, then no directory
the
entry appears for the inode. If this happens, fsck will place the disconnected filea indirecunequal,
and
lost+found directory. If the stored and actual link counts are non-zero
tory entry may have been added or removed without the inode being updated. If this happens, fsck replaces the incorrect stored link count by the actual link count.

Each inode contains a list, or pointers to lists (indirect blocks), of all the blocks claimed
by the inode. Since indirect blocks are owned by an inode, inconsistencies in indirect blocks
directly affect the inode that owns it.

Fsck compares each block number claimed by an inode against a list of already allocateda
to
blocks. If another inode already claims a block number, then the block number is added
block
the
include
to
updated
is
blocks
list of duplicate blocks. Otherwise, the list of allocated
number.

If there are any duplicate blocks, fsck will perform a partial second pass over the inode
exalist to find the inode of the duplicated block. The second pass is needed, since withoution
is
informat
enough
not
content,
correct
for
mining the files associated with these inodes
does
n
conditio
this
If
cleared.
be
should
and
d
available to determine which inode is corrupte
is usuarise (only hardware failure will cause it), then the inode with the earliest modify time
both
clear
to
operator
the
prompts
fsck
happens,
ally incorrect, and should be cleared. If this
cleared.
be
should
one
which
and
kept
be
should
inodes. The operator must decide which one
Fsck checks the range of each block number claimed by an inode. If the block number
is lower than the first data block in the file system, or greater than the last data block, then

2-12 Fsck

the block number is a bad block number. Many bad blocks in an inode are usually caused by
an indirect block that was not written to the file system, a condition which can only occur if
there has been a hardware failure. If an inode contains bad block numbers, fsck prompts the
operator to clear it.

3.6. Inode data size
Each inode contains a count of the number of data blocks that it contains.

The number

of actual data blocks is the sum of the allocated data blocks and the indirect blocks. Fsck
computes the actual number of data blocks and compares that block count against the actual
number of blocks the inode claims. If an inode contains an incorrect count fsck prompts the
operator to fix it.

Each inode contains a thirty-two bit size field.

The size is the number of data bytes in

the file associated with the inode. The consistency of the byte size field is roughly checked by
computing from the size field the maximum number of blocks that should be associated with
the inode, and comparing that expected block count against the actual number of blocks the
inode claims.

3.7. Checking the data associated with an inode
An inode can directly or indirectly reference three kinds of data blocks.

All referenced

blocks must be the same kind. The three types of data blocks are: plain data blocks, symbolic
link data blocks, and directory data blocks.

Plain data blocks contain the information stored

in a file; symbolic link data blocks contain the path name stored in a link.

Directory data

blocks contain directory entries. Fsck can only check the validity of directory data blocks.
Each directory data block is checked for several types of inconsistencies.

These incon-

sistencies include directory inode numbers pointing to unallocated inodes, directory inode
numbers that are greater than the number of inodes in the file system, incorrect directory
inode numbers for “.” and “..”, and directories that are not attached to the file system.

If the

inode number in a directory data block references an unallocated inode, then fsck will remove
that directory entry.

Again, this condition can only arise when there has been a hardware

failure.

If a directory entry inode number references outside the inode list, then fsck will remove

that directory entry. This condition occurs if bad data is written into a directory data block.
The directory inode number entry for “.” must be the first entry in the directory data

block. The inode number for “.” must reference itself; e.g., it must equal the inode number
for the directory data block. The directory inode number entry for “..” must be the second

entry in the directory data block. Its value must equal the inode number for the parent of the
directory entry (or the inode number of the directory data block if the directory is the root
directory). If the directory inode numbers are incorrect, fsck will replace them with the
correct values.

3.8. File system connectivity

Fsck checks the general connectivity of the file system.

If directories are not linked into

the file system, then fsck links the directory back into the file system in the lost+found directory. This condition only occurs when there has been a hardware failure.

Fsck 2-13

Acknowledgements

I thank Bill Joy, Sam Leffler, Robert Elz and Dennis Ritchie for their suggestions and
help in implementing the new file system. Thanks also to Robert Henry for his editorial input
to get this document together. Finally we thank our sponsors, the National Science Foundation under grant MCS80-05144, and the Defense Advance Research Projects Agency (DoD)
under Arpa Order No. 4031 monitored by Naval Electronic System Command under Contract
No. N00039-82-C-0235. (Kirk McKusick, July 1983)

I would like to thank Larry A. Wehr for advice that lead to the first version of fsck and
Rick B. Brandt for adapting fsck to UNIX/TS. (T. Kowalski, July 1979)

References

[Dolotta78]

Dolotta, T. A., and Olsson, S. B. eds., UNIX User’s Manual, Edition

[Joy83]

Joy, W., Cooper, E., Fabry, R., Leffler, S., McKusick, M., and Mosher,

1.1 (January 1978).

D. 4.2BSD System Manual, University of California at Berkeley, Computer Systems Research Group Technical Report #4, 1982.

[McKusick83]

McKusick, M., Joy, W., Leffler, S., and Fabry, R. A Fast File System

for UNIX, University of California at Berkeley, Computer Systems
Research Group Technical Report #7, 1982.

[Ritchie78]

Ritchie, D. M., and Thompson, K., The UNIX Time-Sharing System,
The Bell System Technical Journal 57, 6 (July-August 1978, Part 2),

pp. 1905-29.

[Thompson78]

Thompson, K., UNIX Implementation, The Bell System Technical

Journal 57, 6 (July-August 1978, Part 2), pp. 1931-46.

2-14 Fsck

4. Appendix A — Fsck Error Conditions
4.1. Conventions

Fsck 1s a multi-pass file system check program. Each file system pass invokes a different

Phase of the fsck program.

After the initial setup, fsck performs successive Phases over each
file system, checking blocks and sizes, path-names, connectivity, reference counts, and the
map of free blocks, (possibly rebuilding it), and performs some cleanup.

Normally fsck is run non-interactively to preen the file systems after an unclean halt. While
preen’ing a file system, it will only fix corruptions that are expected to occur from an unclean

halt.

These actions are a proper subset of the actions that fsck will take when it is running
Throughout this appendix many errors have several options that the operator

interactively.

can take. When an inconsistency is detected, fsck reports the error condition to the operator.
If a response is required, fsck prints a prompt message and waits for a response. When
preen’ing most errors are fatal. For those that are expected, the response taken is noted.
This appendix explains the meaning of each error condition, the possible responses, and the
related error conditions.
The error conditions are organized by the Phase of the fsck program in which they can occur.
The error conditions that may occur in more than one Phase will be discussed in initialization.
4.2.

Initialization

Before a file system check can be performed, certain tables have to be set up and certain

files opened.

tables.

This section concerns itself with the opening of files and the initialization of
This section lists error conditions resulting from command line options, memory

requests, opening of files, status of files, file system size checks, and creation of the scratch

file.

All of the initialization errors are fatal when the file system is being preen’ed.

C option?
C is not a legal option to fsck; legal options are —b, —y, —n, and —p. Fsck terminates on this
error condition. See the fsck (8) manual entry for further detail.
cannot alloc

NNN bytes for blockmap

cannot alloc NNN bytes for freemap
cannot alloc NNN bytes for statemap

cannot alloc NNN bytes for Incntp

Fsck’s request for memory for its virtual memory tables failed.
Fsck terminates on this error condition.

This should never happen.

See a guru.

Can’t open checklist file: F
The file system checklist file F' (usually /etc/fstab) can not be opened for reading.
minates on this error condition. Check access modes of F.

Can’t stat root
Fsck’s request for statistics about the root directory “/” failed.
Fsck terminates on this error condition. See a guru.

Fsck ter-

|
This should never happen.

Can’t stat F
Can’t make sense out of name F
Fsck’s request for statistics about the file system F failed.

When running manually, it ignores
this file system and continues checking the next file system given. Check access modes of F.

Can’t open F
Fsck’s request attempt to open the file system F failed.

|
When running manually, it ignores

Fsck 2-15

this file system and continues checking the next file system given. Check access modes of F.
F: (NO WRITE)

Either the —n flag was specified or fsck’s attempt to open the file system F' for writing failed.
When running manually, all the diagnostics are printed out, but no modifications are
attempted to fix them.

file is not a block or character device; OK
You have given fsck a regular file name by mistake. Check the type of the file specified.
Possible responses to the OK prompt are:
YES Ignore this error condition.

ignore this file system and continues checking the next file system given.

One of the following messages will appear:
MAGIC NUMBER WRONG
NCG OUT OF RANGE
CPG OUT OF RANGE
NCYL DOES NOT JIVE WITH NCG*CPG
SIZE PREPOSTEROUSLY LARGE

TRASHED VALUES IN SUPER BLOCK
and will be followed by the message:
F: BAD SUPER BLOCK: B

USE -b OPTION TO FSCK TO SPECIFY LOCATION OF AN ALTERNATE
SUPER-BLOCK TO SUPPLY NEEDED INFORMATION; SEE fsck(8).
The super block has been corrupted. An alternative super block must be selected from among
those listed by newfs (8) when the file system was created. For file systems with a blocksize
less than 32K, specifying —b 32 is a good first choice.
INTERNAL INCONSISTENCY: M

Fsck’s has had an internal panic, whose message is specified as M. This should never happen.
See a guru.

CAN NOT SEEK: BLK B (CONTINUE)

Fsck’s request for moving to a specified block number B in the file system failed. This should
never happen. See a guru.

Possible responses to the CONTINUE prompt are:

YES attempt to continue to run the file system check. Often, however the problem will persist. This error condition will not allow a complete check of the file system. A second
run of fsck should be made to re-check this file system. If the block was part of the virtual memory buffer cache, fsck will terminate with the message “Fatal I/O error”.
NO

terminate the program.

CAN NOT READ: BLK B (CONTINUE)

Fsck’s request for reading a specified block number B in the file system failed. This should
never happen. See a guru.

Possible responses to the CONTINUE prompt are:

YES attempt to continue to run the file system check. Often, however, the problem will persist. This error condition will not allow a complete check of the file system. A second
run of fsck should be made to re-check this file system. If the block was part of the virtual memory buffer cache, fsck will terminate with the message “Fatal 1/O error”.

2-16 Fsck

terminate the program.

CAN NOT WRITE: BLK B (CONTINUE)

Fsck’s request for writing a specified block number B in the file system failed. The disk is
write-protected. See a guru.

Possible responses to the CONTINUE prompt are:

terminate the program.

4.3. Phase 1 — Check Blocks and Sizes

This phase concerns itself with the inode list. This section lists error conditions resulting from checking inode types, seiting up the zero-link-count table, examining inode block
numbers for bad or duplicate blocks, checking inode size, and checking inode format. All
errors in this phase except INCORRECT BLOCK COUNT are fatal if the file system 1is
being preen’ed,

CG C: BAD MAGIC NUMBER The magic number of cylinder group C is wrong. This usually indicates that the cylinder group maps have been destroyed. When running manually the
cylinder group is marked as needing to be reconstructed.
UNKNOWN FILE TYPE I=] (CLEAR) The mode word of the inode I indicates that the
inode is not a special block inode, special character inode, socket inode, regular inode, symbolic link, or directory inode.
Possible responses to the CLEAR prompt are:

YES de-allocate inode I by zeroing its contents. This will always invoke the UNALLOCATED error condition in Phase 2 for each directory entry pointing to this inode.
NO

ignore this error condition.

LINK COUNT TABLE OVERFLOW (CONTINUE)
An internal table for fsck containing allocated inodes with a link count of zero has no more
room. Recompile fsck with a larger value of MAXLNCNT.
Possible responses to the CONTINUE prompt are:

YES continue with the program. This error condition will not allow a complete check of the
file system. A second run of fsck should be made to re-check this file system. If another
allocated inode with a zero link count is found, this error condition is repeated.
NO

terminate the program.

B BAD I=1

Inode I contains block number B with a number lower than the number of the first data block
in the file system or greater than the number of the last block in the file system. This error
condition may invoke the EXCESSIVE BAD BLKS error condition in Phase 1 if inode I
has too many block numbers outside the file system range. This error condition will always
invoke the BAD/DUP error condition in Phase 2 and Phase 4.
EXCESSIVE BAD BLKS I=] (CONTINUE)

There is more than a tolerable number (usually 10) of blocks with a number lower than the
number of the first data block in the file system or greater than the number of last block in

Fsck 2-17

the file system associated with inode I.
Possible responses to the CONTINUE prompt are:

YES ignore the rest of the blocks in this inode and continue checking with the next inode in
the file system. This error condition will not allow a complete check of the file system.
A second run of fsck should be made to re-check this file system.
NO

terminate the program.

B DUP I=I]

Inode I contains block number B which is already claimed by another inode. This error condition may invoke the EXCESSIVE DUP BLKS error condition in Phase 1 if inode I has
too many block numbers claimed by other inodes. This error condition will always invoke
Phase 1b and the BAD/DUP error condition in Phase 2 and Phase 4.
EXCESSIVE DUP BLKS I=] (CONTINUE)
There is more than a tolerable number (usually 10) of blocks claimed by other inodes.
Possible responses to the CONTINUE prompt are:

terminate the program.

DUP TABLE OVERFLOW (CONTINUE)

An internal table in fsck containing duplicate block numbers has no more room. Recompile

fsck with a larger value of DUPTBLSIZE.

Possible responses to the CONTINUE prompt are:

YES continue with the program. This error condition will not allow a complete check of the
file system. A second run of fsck should be made to re-check this file system. If another
duplicate block is found, this error condition will repeat.
NO

terminate the program.

PARTIALLY ALLOCATED INODE I=] (CLEAR)
Inode I is neither allocated nor unallocated.
Possible responses to the CLEAR prompt are:
YES de-allocate inode I by zeroing its contents.

ignore this error condition.

INCORRECT BLOCK COUNT I=I (X should be Y) (CORRECT)
The block count for inode I is X blocks, but should be Y blocks. When preen’ing the count is
corrected.

Possible responses to the CORRECT prompt are:

YES replace the block count of inode I with Y.
NO

ignore this error condition.

4.4. Phase 1B: Rescan for More Dups

When a duplicate block is found in the file system, the file system is rescanned to find
the inode which previously claimed that block. This section lists the error condition when the
duplicate block is found.

2-18 Fsck
B DUP I=1]

Inode I contains block number B that is already claimed by another inode.
tion will always invoke the BAD/DUP error condition in Phase 2.

This error condi-

You can determine which

inodes have overlapping blocks by examining this error condition and the DUP error condition
in Phase 1.

4.5. Phase 2 — Check Pathnames
This phase concerns itself with removing directory entries pointing to error conditioned
inodes from Phase 1 and Phase 1b.

This section lists error conditions resulting from root

inode mode and status, directory inode pointers in range, and directory entries pointing to bad
inodes.

All errors in this phase are fatal if the file system is being preen’ed.

ROOT INODE UNALLOCATED. TERMINATING.
The root inode (usually inode number 2) has no allocate mode bits.

This should never hap-

pen. The program will terminate.
NAME TOO LONG F

An excessively long path name has been found. This is usually indicative of loops in the file
system name space.

This can occur if the super user has made circular links to directories.

The offending links must be removed (by a guru).
ROOT INODE NOT DIRECTORY (FIX)
The root inode (usually inode number 2) is not directory inode type.
Possible responses to the FIX prompt are:
YES replace the root inode’s type to be a directory.

If the root inode’s data blocks are not

directory blocks, a VERY large number of error conditions will be produced.
NO

terminate the program.

DUPS/BAD IN ROOT INODE (CONTINUE)
Phase 1 or Phase 1b have found duplicate blocks or bad blocks in the root inode (usually
inode number 2) for the file system.
Possible responses to the CONTINUE prompt are:

YES ignore the DUPS/BAD error condition in the root inode and attempt to continue to run
the file system check.

If the root inode is not correct, then this may result in a large

number of other error conditions.

terminate the program.

I OUT OF RANGE I=] NAME=F (REMOVE)
A directory entry F has an inode number I which is greater than the end of the inode list.
Possible responses to the REMOVE prompt are:

YES the directory entry F is removed.
NO

ignore this error condition.

UNALLOCATED

I=I

OWNER=0

MODE=M

SIZE=S

MTIME=T

DIR=F

(REMOVE)
A directory entry F has a directory inode I without allocate mode bits.
M, size S, modify time T, and directory name F are printed.

Possible responses to the REMOVE prompt are:

The owner O, mode

Fsck 2-19

YES the directory entry F' is removed.
NO

ignore this error condition.

UNALLOCATED

I=I

MODE=M

OWNER=0

SIZE=S

MTIME=T

FILE=F

(REMOVE)

A directory entry F has an inode I without allocate mode bits. The owner O, mode M, size S,
modify time T, and file name F are printed.

Possible responses to the REMOVE prompt are:

YES the directory entry F is removed.
NO

ignore this error condition.

DUP/BAD I=] OWNER=0 MODE=M SIZE=S MTIME=T DIR=F (REMOVE)
Phase 1 or Phase 1b have found duplicate blocks or bad blocks associated with directory entry
F, directory inode I. The owner O, mode M, size S, modify time T, and directory name F are
printed.

Possible responses to the REMOVE prompt are:

YES the directory entry F is removed.
NO

ignore this error condition.

DUP/BAD I=] OWNER=0 MODE=M SIZE=S MTIME=T FILE=F (REMOVE)
Phase 1 or Phase 1b have found duplicate blocks or bad blocks associated with directory entry
F, inode I. The owner O, mode M, size S, modify time T, and file name F are printed.
Possible responses to the REMOVE prompt are:

YES the directory entry F is removed.
NO

ignore this error condition.

ZERO LENGTH DIRECTORY I=] OWNER=0 MODE=M SIZE=S MTIME=T

DIR=F (REMOVE)

A directory entry F has a size S that is zero. The owner O, mode M, size S, modify time T,
and directory name F are printed.

Possible responses to the REMOVE prompt are:

YES the directory entry F is removed; this will always invoke the BAD/DUP error condition
in Phase 4.

ignore this error condition.

DIRECTORY TOO SHORT I=] OWNER=0 MODE=M SIZE=S MTIME=T DIR=F
(FIX)

A directory F has been found whose size S is less than the minimum size directory. The
owner O, mode M, size S, modify time T, and directory name F are printed.
Possible responses to the FIX prompt are:

YES increase the size of the directory to the minimum directory size.
NO

ignore this directory.

DIRECTORY CORRUPTED I=] OWNER=0 MODE=M SIZE=S MTIME=T DIR=F
(SALVAGE)

A directory with an inconsistent internal state has been found.
Possible responses to the FIX prompt are:
g o

won e

o wm

2-20 Fsck

YES throw away all entries up to the next 512-byte boundary. This rather drastic action can
throw away up to 42 entries, and should be taken only after other recovery efforts have
failed.

Skip up to the next 512-byte boundary and resume reading, but do not modify the directory.

BAD INODE NUMBER FOR ¢’ I=I OWNER=0 MODE=M SIZE=S MTIME=T
DIR=F (FIX)

A directory I has been found whose inode number for ‘.” does does not equal 1.
Possible responses to the FIX prompt are:

YES change the inode number for ‘.’ to be equal to I.

leave the inode number for ‘.’ unchanged.

MISSING ¢’ I=1 OWNER=0 MODE=M SIZE=S MTIME=T DIR=F (FIX)
A directory I has been found whose first entry is unallocated.
Possible responses to the FIX prompt are:

YES make an entry for ‘.’ with inode number equal to I.

leave the directory unchanged.

MISSING ¢’ I=1 OWNER=0 MODE=M SIZE=S MTIME=T DIR=F
CANNOT FIX, FIRST ENTRY IN DIRECTORY CONTAINS F
A directory I has been found whose first entry is F. Fsck cannot resolve this problem. The
file system should be mounted and the offending entry F moved elsewhere. The file system
should then be unmounted and fsck should be run again.
MISSING ¢’ I=1 OWNER=0 MODE=M SIZE=S MTIME=T DIR=F

CANNOT FIX, INSUFFICIENT SPACE TO ADD
A directory I has been found whose first entry is not ‘.’. Fsck cannot resolve this problem as
it should never happen. See a guru.

EXTRA ‘ ENTRY I=] OWNER=0 MODE=M SIZE=S MTIME=T DIR=F (FIX)
A directory I has been found that has more than one entry for ‘. .
Possible responses to the FIX prompt are:
YES remove the extra entry for ‘.’.

leave the directory unchanged.

BAD INODE NUMBER FOR ‘.’ I=I OWNER=0 MODE=M SIZE=S MTIME=T
DIR=F (FIX)

A directory I has been found whose inode number for ‘..” does does not equal the parent of I.
Possible responses to the FIX prompt are:

YES change the inode number for ‘.." to be equal to the parent of I.

leave the inode number for ‘..” unchanged.

MISSING ‘.. I=] OWNER=0 MODE=M SIZE=S MTIME=T DIR=F (FIX)
A directory I has been found whose second entry is unallocated.
Possible responses to the FIX prompt are:

YES make an entry for ‘..” with inode number equal to the parent of I.

Fsck

2-21

leave the directory unchanged.

MISSING .. I=I OWNER=0 MODE=M SIZE=S MTIME=T DIR=F
CANNOT FIX, SECOND ENTRY IN DIRECTORY CONTAINS F
A directory I has been found whose second entry is F. Fsck cannot resolve this problem. The
file system should be mounted and the offending entry F' moved elsewhere. The file system
should then be unmounted and fsck should be run again.

MISSING ¢ I=] OWNER=0 MODE=M SIZE=S MTIME=T DIR=F
CANNOT FIX, INSUFFICIENT SPACE TO ADD °.

A directory I has been found whose second entry is not ‘..". Fsck cannot resolve this problem
as it should never happen. See a guru.

EXTRA . ENTRY 1=/ OWNER=0 MODE=M SIZE=S MTIME=T DIR=F (FIX)

A directory I has been found that has more than one entry for °..".
Possible responses to the FIX prompt are:

YES remove the extra entry for °..".

leave the directory unchanged.

4.6. Phase 3 — Check Connectivity

This phase concerns itself with the directory connectivity seen in Phase 2. This section
lists error conditions resulting from unreferenced directories, and missing or full lost+found
directories.

UNREF DIR I=] OWNER=0 MODE=M SIZE=S MTIME=T (RECONNECT)

The directory inode I was not connected to a directory entry when the file system was

traversed. The owner O, mode M, size S, and modify time T of directory inode I are printed.
When preen’ing, the directory is reconnected if its size is non-zero, otherwise it is cleared.
Possible responses to the RECONNECT prompt are:

YES reconnect directory inode I to the file system in the directory for lost files (usually
lost+found). This may invoke the lost+found error condition in Phase 3 if there are
problems connecting directory inode I to lost+found. This may also invoke the CONNECTED error condition in Phase 3 if the link was successful.

ignore this error condition. This will always invoke the UNREF error condition in Phase
4.

SORRY. NO lost+found DIRECTORY

There is no lost+found directory in the root directory of the file system; fsck ignores the
request to link a directory in lost+found. This will always invoke the UNREF error condition
in Phase 4. Check access modes of lost+found. See fsck(8) manual entry for further detail.
This error is fatal if the file system is being preen’ed.

SORRY. NO SPACE IN lost+found DIRECTORY

There is no space to add another entry to the lost+found directory in the root directory of
the file system; fsck ignores the request to link a directory in lost+found. This will always
invoke the UNREF error condition in Phase 4. Clean out unnecessary entries in lost+found
or make lost+found larger. See fsck(8) manual entry for further detail. This error is fatal if
the file system is being preen’ed.

2-22

Fsck

DIR I=11 CONNECTED. PARENT WAS I=12
This is an advisory message indicating a directory inode /1 was successfully connected to the
lost+found directory.

The parent inode I2 of the directory inode /1 is replaced by the inode

number of the lost+found directory.
4.7. Phase 4 — Check Reference Counts
This phase concerns itself with the link count information seen in Phase 2 and Phase 3.
This section lists error conditions resulting from unreferenced files, missing or full lost+found
directory, incorrect link counts for files, directories, symbolic links, or special files, unrefer-

enced files, symbolic links, and directories, bad and duplicate blocks in files, symbolic links,
and directories, and incorrect total free-inode counts.

All errors in this phase are correctable

if the file system is being preen’ed except running out of space in the lost+found directory.
UNREF FILE I=] OWNER=0 MODE=M SIZE=S MTIME=T (RECONNECT)
Inode I was not connected to a directory entry when the file system was traversed.

O, mode M, size S, and modify time T of inode I are printed.

The owner

When preen’ing the file is

cleared if either its size or its link count is zero, otherwise it is reconnected.
Possible responses to the RECONNECT prompt are:

YES reconnect inode I to the file system in the directory for lost files (usually lost+found).
This may invoke the lost+found error condition in Phase 4 if there are problems con-

necting inode I to lost+found.

ignore this error condition.

This will always invoke the CLEAR error condition in Phase

(CLEAR)
The inode mentioned in the immediately previous error condition can not be reconnected.
This cannot occur if the file system is being preen’ed, since lack of space to reconnect files is a
fatal error.

Possible responses to the CLEAR prompt are:

YES de-allocate the inode mentioned in the immediately previous error condition by zeroing
its contents.

ignore this error condition.

SORRY. NO lost+found DIRECTORY

There is no lost+found directory in the root directory of the file system; fsck ignores the
request to link a file in lost+found. This will always invoke the CLEAR error condition in
Phase 4.

Check access modes of lost+found.

This error is fatal if the file system is being

preen’ed.

SORRY. NO SPACE IN lost+found DIRECTORY
There is no space to add another entry to the lost+found directory in the root directory of
the file system; fsck ignores the request to link a file in lost+found. This will always invoke

the CLEAR error condition in Phase 4.

Check size and contents of lost+found. This error is

fatal if the file system is being preen’ed.

LINK COUNT FILE I=I OWNER=0 MODE=M SIZE=S MTIME=T COUNT=X

SHOULD BE Y (ADJUST)
The link count for inode I which is a file, is X but should be Y.

S, and modify time T are printed.
'

The owner O, mode M, size

When preen’ing the link count is adjusted.
t

Fsck 2-23

YES replace the link count of file inode I with Y.
NO

ignore this error condition.

LINK

COUNT

DIR I=] OWNER=0 MODE=M SIZE=S MTIME=T COUNT=X

SHOULD BE Y (ADJUST)
The link count for inode I which is a directory, is X but should be Y.
size S, and modify time T of directory inode I are printed.

The owner O, mode M,

When preen’ing the link count is

adjusted.
Possible responses to the ADJUST prompt are:
YES replace the link count of directory inode I with Y.

ignore this error condition.

LINK

COUNT

I=I

OWNER=0

MODE=M

SIZE=S

MTIME=T

COUNT=X

SHOULD BE Y (ADJUST)
The link count for F inode I is X but should be Y.

The name F, owner O, mode M, size S,

and modify time T are printed. When preen’ing the link count is adjusted.
Possible responses to the ADJUST prompt are:
YES replace the link count of inode I with Y.

ignore this error condition.

UNREF FILE I=] OWNER=0 MODE=M SIZE=S MTIME=T (CLEAR)
Inode I which is a file, was not connected to a directory entry when the file system was
traversed.

The owner O, mode M, size S, and modify time T of inode I are printed.

When

preen’ing, this is a file that was not connected because its size or link count was zero, hence it

is cleared.

Possible responses to the CLEAR prompt are:
YES de-allocate inode I by zeroing its contents.

ignore this error condition.

UNREF DIR I=] OWNER=0 MODE=M SIZE=S MTIME=T (CLEAR)
Inode I which is a directory, was not connected to a directory entry when the file system was

traversed.

The owner O, mode M, size S, and modify time T of inode I are printed.

When

preen’ing, this is a directory that was not connected because its size or link count was zero,
hence it is cleared.

Possible responses to the CLEAR prompt are:
YES de-allocate inode I by zeroing its contents.

ignore this error condition.

BAD/DUP FILE I=] OWNER=0 MODE=M SIZE=S MTIME=T (CLEAR)
Phase 1 or Phase 1b have found duplicate blocks or bad blocks associated with file inode 1.
The owner O, mode M, size S, and modify time T of inode I are printed.

This error cannot

arise when the file system is being preen’ed, as it would have caused a fatal error earlier.
Possible responses to the CLEAR prompt are:
YES de-allocate inode I by zeroing its contents.

ignore this error condition.

NRATYDYITD IR
APLBES] A/ %/ E

- A QT7
N =0 MTIME
=T (T EAR)

KSR

Phase 1 or Phase

have found duplicate

blocks or bad

VIS

BIVEEY=

VAU

MEN)

blocks associated with directory inode

2-24 Fsck

I. The owner O, mode M, size S, and modify time T of inode I are printed. This error cannot

arise when the file system is being preen’ed, as it would have caused a fatal error earlier.
Possible responses to the CLEAR prompt are:

YES de-allocate inode I by zeroing its contents.

ignore this error condition.

FREE INODE COUNT WRONG IN SUPERBLK (FIX)
The actual count of the free inodes does not match the count in the super-block of the file

system. When preen’ing, the count is fixed.
Possible responses to the FIX prompt are:
YES replace the count in the super-block by the actual count.

ignore this error condition.

4.8. Phase 5 - Check Cyl groups

This phase concerns itself with the free-block maps.

This section lists error conditions

resulting from allocated blocks in the free-block maps, free blocks missing from free-block
maps, and the total free-block count incorrect.

EXCESSIVE BAD BLKS IN BIT MAPS (CONTINUE)
An inode contains more than a tolerable number (usually 10) of blocks claimed by other
inodes or that are out of the legal range for the file system. This error is fatal if the file system is being preen’ed.
Possible responses to the CONTINUE prompt are:
YES ignore the rest of the free-block maps and continue the execution of fsck.
NO

terminate the program.

SUMMARY INFORMATION T BAD
where T is one or more of:
(INODE FREE)

(BLOCK OFFSETS)
(FRAG SUMMARIES)

(SUPER BLOCK SUMMARIES)
The indicated summary information was found to be incorrect.

This error condition will

always invoke the BAD CYLINDER GROUPS condition in Phase 6. When preen’ing, the
summary information is recomputed.
X BLK(S) MISSING

X blocks unused by the file system were not found in the free-block maps. This error condition will always invoke the BAD CYLINDER GROUPS condition in Phase 6.

When

preen’ing, the block maps are rebuilt.
BAD CYLINDER GROUPS (SALVAGE)

Phase 5 has found bad blocks in the free-block maps, duplicate blocks in the free-block maps,
or blocks missing from the file system. When preen’ing, the cylinder groups are reconstructed.

Fsck 2-25

Possible responses to the SALVAGE prompt are:

YES replace the actual free-block maps with a new free-block maps.
NO

ignore this error condition.

FREE BLK COUNT WRONG IN SUPERBLOCK (FIX)
The actual count of free blocks does not match the count in the super-block of the file system.
When preen’ing, the counts are fixed.
Possible responses to the FIX prompt are:

YES replace the count in the super-block by the actual count.
NO

ignore this error condition.

4.9. Phase 6 - Salvage Cylinder Groups
This phase concerns itself with the free-block maps reconstruction.

No error messages

are produced.
4.10. Cleanup

Once a file system has been checked, a few cleanup functions are performed. This section lists advisory messages about the file system and modify status of the file system.
V files, W used, X free (Y frags, Z blocks)
This is an advisory message indicating that the file system checked contained V files using W
fragment sized blocks leaving X fragment sized blocks free in the file system. The numbers in
parenthesis breaks the free count down into Y free fragments and Z free full sized blocks.
¥kt REBOOT UNIX Hk**

This is an advisory message indicating that the root file system has been modified by fsck. If
UNIX is not rebooted immediately, the work done by fsck may be undone by the in-core
copies of tables UNIX keeps. When preen’ing, fsck will exit with a code of 4. The autoreboot script interprets an exit code of 4 by issuing a reboot system call.
¥k FILE SYSTEM WAS MODIFIED #*¥#*%*
This is an advisory message indicating that the current file system was modified by fsck. If
this file system is mounted or is the current root file system, fsck should be halted and UNIX
rebooted. If UNIX is not rebooted immediately, the work done by fsck may be undone by the
in-core copies of tables UNIX keeps.

SENDMAIL
INSTALLATION AND OPERATION GUIDE
Eric Allman
Britton-Lee, Inc.

Version 4.2

Sendmail implements a general purpose internetwork mail routing facility under the
UNIX* operating system. It is not tied to any one transport protocol — its function may be
likened to a crossbar switch, relaying messages from one domain into another. In the process,
it can do a limited amount of message header editing to put the message into a format that is
appropriate for the receiving domain. All of this is done under the control of a configuration
file.

Due to the requirements of flexibility for sendmail, the configuration file can seem somewhat unapproachable. However, there are only a few basic configurations for most sites, for
which standard configuration files have been supplied. Most other configurations can be built
by adjusting an existing configuration files incrementally.
Although sendmail is intended to run without the need for monitoring, it has a number
of features that may be used to monitor or adjust the operation under unusual circumstances.
These features are described.

Section one describes how to do a basic sendmail installation. Section two explains the
day-to-day information you should know to maintain your mail system. If you have a relatively normal site, these two sections should contain sufficient information for you to install
sendmail and keep it happy. Section three describes some parameters that may be safely
tweaked. Section four has information regarding the command line arguments. Section five
contains the nitty-gritty information about the configuration file. This section is for masochists and people who must write their own configuration file. The appendixes give a brief
but detailed explanation of a number of features not described in the rest of the paper.
The references in this paper are actually found in the companion paper Sendmail — An
Internetwork Mail Router. This other paper should be read before this manual to gain a
basic understanding of how the pieces fit together.
1. BASIC INSTALLATION

There are two basic steps to installing sendmail. The hard part is to build the
configuration table. This is a file that sendmail reads when it starts up that describes the
mailers it knows about, how to parse addresses, how to rewrite the message header, and the
settings of wvarious options. Although the configuration table is quite complex, a
configuration can usually be built by adjusting an existing off-the-shelf configuration. The
second part is actually doing the installation, i.e., creating the necessary files, etc.
The remainder of this section will describe the installation of sendmail assuming you
can use one of the existing configurations and that the standard installation parameters are
*UNIX is a trademark of Bell Laboratories.

Sendmail Installation and Operation Guide 2-27

2-28

Sendmail Installation and Operation Guide

acceptable. All pathnames and examples are given from the root of the sendmail subtree.
1.1. Off-The-Shelf Configurations
The configuration files are all in the subdirectory c¢f of the sendmail directory.

The ones used at Berkeley are in m4 (1) format; files with names ending “.m4” are m4

include files, while files with names ending “.mc” are the master files.

Files with names

ending “.cf” are the m4 processed versions of the corresponding “.mc” file.
Two off the shelf configuration files are supplied to handle the basic cases:

cf/arpaproto.cf for Arpanet (TCP) sites and cf/uucpproto.cf for UUCP sites. These are
not in m4 format.

The file you need should be copied to a file with the same name as

your system, e.g.,

cp uucpproto.cf ucsfegl.cf

This file is now ready for installation as /usr/lib/sendmail.cf.
1.2.

Installation Using the Makefile

A makefile exists in the root of the sendmail directory that will do all of these
steps for a 4.2BSD system.

It may have to be slightly tailored for use on other systems.

Before using this makefile, you should already have created your configuration file

and left it in the file “cf/system.cf” where system is the name of your system (i.e., what
is returned by hostname(1)).

If you do not have hostname you can use the declaration

“HOST=system” on the make(l) command line.

You should also examine the file

md/config.m4 and change the m4 macros there to reflect any libraries and compilation
flags you may need.
The basic installation procedure is to type:

make
make install
in the root directory of the sendmail distribution.
install them in the standard places.

This will make all binaries and

The second make command must be executed as

the superuser (root).
1.3. Installation by Hand
Along with building a configuration file, you will have to install the sendmail
startup into your UNIX system.

If you are doing this installation in conjunction with a

regular Berkeley UNIX install, these steps will already be complete.

Many of these

steps will have to be executed as the superuser (root).
1.3.1. lib/libsys.a

The library in lib/libsys.a contains some routines that should in some sense be
part of the system library.

These are the system logging routines and the new direc-

tory access routines (if required).

If you are not running the new 4.2BSD directory

code and do not have the compatibility routines installed in your system library, you
should execute the commands:

cd lib
make ndir
This will compile and install the 4.2 compatibility routines in the library.

should then type:

You

Sendmail Installation and Operation Guide 2-29

cd lib

# if required

make

This will recompile and fill the library.
1.8.2. /usr/lib/sendmail

The binary for sendmail is located in /usr/lib. There is a version available in

the source directory that is probably inadequate for your system. You should plan
on recompiling and installing the entire system:
cd src
rm —f *.0

make

cp sendmail /usr/lib
1.3.3. /usr/lib/sendmail.cf

The configuration
/usr/lib/sendmail.cf:

file

that you

created

earlier

should

installed

cp cf/system.cf /usr/lib/sendmail.cf
1.3.4. /usr/ucb/newaliases

- If you are running delivermail, it is critical that the newaliases command be
replaced. This can just be a link to sendmail:
|
rm —f /usr/ucb/newaliases
In /usr/lib/sendmail /usr/uchb/newaliases

1.3.5. /usr/lib/sendmail.cf

The configuration file must be installed in /usr/lib. This is described above.
1.3.6. /usr/spool/mqueue

The directory /usr/spool/mqueue should be created to hold the mail queue.
This directory should be mode 777 unless sendmail is run setuid, when mqueue
should be owned by the sendmail owner and mode 755.
1.3.7. /usr/lib/aliases*

The system aliases are held in three files. The file “/usr/lib/aliases” is the
master copy. A sample is given in “lib/aliases” which includes some aliases which
must be defined:

cp lib/aliases /usr/lib/aliases
You should extend this file with any aliases that are apropos to your system.

Normally sendmail looks at a version of these files maintained by the dbm (3)
routines. These are stored in “/usr/lib/aliases.dir” and “/usr/lib/aliases.pag.” These
can initially be created as empty files, but they will have to be initialized promptly.
These should be mode 666 if you are running a reasonably relaxed system:

cp /dev/null /usr/lib/aliases.dir
cp /dev/null /usr/lib/aliases.pag
chmod 666 /usr/lib/aliases.*
newaliases

2-30

Sendmail Installation and Operation Guide

1.3.8. /usr/lib/sendmail.fc
If you intend to install the frozen version of the configuration file (for quick
startup) you should create the file /usr/lib/sendmail.fc and initialize it.

This step

may be safely skipped.

cp /dev/null /usr/lib/sendmail.fc
/usr/lib/sendmail —bz
1.3.9. /etc/re

It will be necessary to start up the sendmail daemon when your system

reboots.

This daemon performs two functions: it listens on the SMTP socket for

connections (to receive mail from a remote system) and it processes the queue
periodically to insure that mail gets delivered when hosts come up.

Add the following lines to “/etc/rc” (or “/etc/rc.local” as appropriate) in the
area where it is starting up the daemons:

if [ —f /usr/lib/sendmail ]; then

(cd /usr/spool/mqueue; rm —f [Inx]f*)
/usr/lib/sendmail —bd —q30m &
echo —n ’ sendmail’ >/dev/console
fi

The “cd” and “rm” commands insure that all lock files have been removed; extrane-

ous lock files may be left around if the system goes down in the middle of processing
a message.

The line that actually invokes sendmail has two flags: “—bd” causes it to

listen on the SMTP port, and “—q30m” causes it to run the queue every half hour.
If you are not running a version of UNIX that supports Berkeley TCP/IP, do
not include the —bd flag.
1.3.10. /usr/lib/sendmail.hf

This is the help file used by the SMTP HELP command.

from “lib/sendmail.hf”:

It should be copied

cp lib/sendmail.hf /usr/lib
1.3.11. /usr/lib/sendmail.st

If you wish to collect statistics about your mail traffic, you should create the

file “/usr/lib/sendmail.st”:

cp /dev/null /usr/lib/sendmail.st
chmod 666 /usr/lib/sendmail.st
This file does not grow.

It is printed with the program “aux/mailstats.”

1.3.12. /etc/syslog

You may want to run the syslog program (to collect log information about
This program normally resides in /etc/syslog, with support files

sendmail).

/etc/syslog.conf and /etc/syslog.pid. The program is located in the aux subdirectory
of the sendmail distribution. The file /etc/syslog.conf describes the file(s) that

sendmail will log in.

For a complete description of syslog, see the manual page for

syslog (8) (located in sendmail/doc on the distribution).

Sendmail Installation and Operation Guide 2-31

1.3.13. /usr/ucb/newaliases
If sendmail is invoked as “newaliases,”

it will simulate the —bi flag (i.e., will

rebuild the alias database; see below). This should be a link to /usr/lib/sendmail.
1.3.14. /usr/ucb/mailq

If sendmail is invoked as “mailq,” it will simulate the —bp flag (i.e., sendmail
will print the contents of the mail queue; see below).

This should be a link to

/usr/lib/sendmail.
2. NORMAL OPERATIONS
2.1. Quick Configuration Startup
A fast version of the configuration file may be set up by using the —bz flag:

/usr/lib/sendmail —bz
This creates the file /usr/lib/sendmail.fc (“frozen configuration”).

This file is an image

of sendmail’s data space after reading in the configuration file.

If this file exists, it is

used instead of /usr/lib/sendmail.cf sendmail.fc must be rebuilt manually every time
sendmail.cf is changed.
The frozen configuration file will be ignored if a —C flag is specified or if sendmail

detects that it is out of date.

However, the heuristics are not strong so this should not

be trusted.
2.2. The System Log
The system log is supported by the syslog (8) program.
2.2.1. Format

Each line in the system log consists of a timestamp, the name of the machine

that generated it (for logging from several machines over the ethernet), the word

“sendmail:”’, and a message.
2.2.2. Levels
If you have syslog (8) or an equivalent installed, you will be able to do logging.
There is a large amount of information that can be logged.

The log is arranged as a

succession of levels. At the lowest level only extremely strange situations are logged.
At the highest level, even the most mundane and uninteresting events are recorded
for posterity.

As a convention, log levels under ten are considered ‘“useful;” log lev-

els above ten are usually for debugging purposes.

A complete description of the log levels is given in section 4.3.
2.3. The Mail Queue
The mail queue should be processed transparently.
manual intervention is sometimes necessary.

However, you may find that

For example, if a major host is down for a

period of time the queue may become clogged.

Although sendmail ought to recover

gracefully when the host comes up, you may find performance unacceptably bad in the
meantime.

2.3.1. Printing the queue

The contents of the queue can be printed using the mailg command (or by

specifying the —bp flag to sendmail):

Sendmail Installation and Operation Guide

mailq
This will produce a listing of the queue id’s, the size of the message, the date the
message entered the queue, and the sender and recipients.
2.3.2. Format of queue files

All queue files have the form xfAA99999 where AA99999 is the id for this file

and the x is a type. The types are:
d

The data file. The message body (excluding the header) is kept in this file.

The lock file.

If this file exists, the job is currently being processed, and a

queue run will not process the file.

For that reason, an extraneous If file can

cause a job to apparently disappear (it will not even time out!).

This file is created when an id is being created.

It is a separate file to insure

that no mail can ever be destroyed due to a race condition.

It should exist for

no more than a few milliseconds at any given time.
q

The queue control file.

This file contains the information necessary to process

the job.
t

A temporary file. These are an image of the gf file when it is being rebuilt.

should be renamed to a qf file very quickly.
X

A transcript file, existing during the life of a session showing everything that
happens during that session.

The qf file is structured as a series of lines each beginning with a code letter.
The lines are as follows:
D

The name of the data file. There may only be one of these lines.

A header definition.

There may be any number of these lines.

important: they represent the order in the final message.

The order is

These use the same

syntax as header definitions in the configuration file.
R

A recipient address.

This will normally be completely aliased, but is actually

realiased when the job is processed. There will be one line for each recipient.

The sender address. There may only be one of these lines.

2-32

The job creation time. This is used to compute when to time out the job.

The current message priority.

This

numbers mean lower priorities.

The priority increases as the message sits in

the queue.

is used to order the queue.

Higher

The initial priority depends on the message class and the size of

the message.
M

A message.

This line is printed by the mailg command, and is generally used

to store status information.

It can contain any text.

As an example, the following is a queue file sent to “mckusick@calder” and
13

39,

wnj

Sendmail Installation and Operation Guide 2-33

DdfA13557

Seric
T404261372
P132

Rmckusick@calder

Rwnyj
H?D?date: 23-Oct-82 15:49:32-PDT (Sat)

H?F?from: eric (Eric Allman)
H?x?full-name: Eric Allman
Hsubject: this is an example message

Hmessage-id: <8209232249.13557@UCBARPA.BERKELEY.ARPA>

Hreceived: by UCBARPA.BERKELEY.ARPA (3.227 [10/22/82])
id A13557; 23-Oct-82 15:49:32-PDT (Sat)

Hphone: (415) 548-3211
HTo: mckusick(@calder, wnj

This shows the name of the data file, the person who sent the message, the submis-

sion time (in seconds since January 1, 1970), the message priority, the message class,
the recipients, and the headers for the message.
2.3.3. Forcing the queue

Sendmail should run the queue automatically at intervals.

The algorithm is to

read and sort the queue, and then to attempt to process all jobs in order.

When it

attempts to run the job, sendmail first checks to see if the job is locked.

If so, it

ignores the job.

There is no attempt to insure that only one queue processor exists at any time,

since there is no guarantee that a job cannot take forever to process.

Due to the

locking algorithm, it is impossible for one job to freeze the queue.

However, an

uncooperative recipient host or a program recipient that never returns can accumu-

late many processes in your system.

Unfortunately, there is no way to resolve this

without violating the protocol.
In some cases, you may find that a major host going down for a couple of days
may create a prohibitively large queue.

This will result in sendmail spending an

inordinate amount of time sorting the queue.

This situation can be fixed by moving

the queue to a temporary place and creating a new queue.

The old queue can be

run later when the offending host returns to service.

To do this, it is acceptable to move the entire queue directory:

cd /usr/spool
mv mqgueue omqueue; mkdir mqueue; chmod 777 mqueue

You should then kill the existing daemon (since it will still be processing in the old

queue directory) and create a new daemon.
To run the old mail queue, run the following command:

/usr/lib/sendmail —o0Q/usr/spool/omqueue —q
The —o0Q flag specifies an alternate queue directory and the —q flag says to just run

every job in the queue.

If you have a tendency toward voyeurism, you can use the

—v flag to watch what is going on.
When the queue is finally emptied, you can remove the directory:

rmdir /usr/spool/omqueue

2-34 Sendmail Installation and Operation Guide

2.4. The Alias Database
The alias database exists in two forms.

Jusr/lib/aliases.

One is a text form, maintained in the file

The aliases are of the form

name: namel, name2, ...

Only local names may be aliased; e.g.,
eric@mit-xx: eric@berkeley
will not have the desired effect. Aliases may be continued by starting any continuation
lines with a space or a tab. Blank lines and lines beginning with a sharp sign (“#”) are
comments.

The second form is processed by the dbm (3) library.

fusr/lib/aliases.dir and /usr/lib/aliases.pag.

This form is in the files

This is the form that sendmail actually

uses to resolve aliases. This technique is used to improve performance.
2.4.1.

Rebuilding the alias database

The DBM version of the database may be rebuilt explicitly by executing the

command
newaliases

This is equivalent to giving sendmail the —bi flag:

/usr/lib/sendmail —bi
If the “D” option is specified in the configuration, sendmail will rebuild the

alias database automatically if possible when it is out of date.

The conditions under

which it will do this are:
(1)

The DBM version of the database is mode 666.

(2)

Sendmail is running setuid to root.

-or-

Auto-rebuild can be dangerous on heavily loaded machines with large alias files; if it
might take more than five minutes to rebuild the database, there is a chance that
several processes will start the rebuild process simultaneously.
2.4.2. Potential problems

There are a number of problems that can occur with the alias database.

They

all result from a sendmail process accessing the DBM version while it is only partially built.

This can happen under two circumstances: One process accesses the

database while another process is rebuilding it, or the process rebuilding the database dies (due to being killed or a system crash) before completing the rebuild.
Sendmail has two techniques to try to relieve these problems.

First, it ignores

interrupts while rebuilding the database; this avoids the problem of someone aborting the process leaving a partially rebuilt database.

Second, at the end of the

rebuild it adds an alias of the form

@: @
(which is not normally legal). Before sendmail will access the database, it checks to

insure that this entry exists!. It will wait up to five minutes for this entry to appear,
at which point it will force a rebuild itself?.
'The “a” option is required in the configuration for this action to occur. This should normally be specified unless you are running delivermail in parallel with sendmail.

?Note: the “D” option must be specified in the configuration file for this operation to occur.

Sendmail Installation and Operation Guide 2-35

2.4.3.

List owners

If an error occurs on sending to a certain address, say “x”, sendmail will look
for an alias of the form “owner-x”’ to receive the errors. This is typically useful for a
mailing list where the submitter of the list has no control over the maintanence of
the list itself; in this case the list maintainer would be the owner of the list. For
example:

unix-wizards: eric@ucbarpa, wnj@monet, nosuchuser,
sam(@matisse

owner-unix-wizards: eric@ucbarpa

would cause “eric@ucbarpa” to get the error that will occur when someone sends to
unix-wizards due to the inclusion of “nosuchuser” on the list.
2.5. Per-User Forwarding (.forward Files)

As an alternative to the alias database, any user may put a file with the name
“ forward” in his or her home directory. If this file exists, sendmail redirects mail for
that user to the list of addresses listed in the .forward file. For example, if the home
directory for user “mckusick” has a .forward file with contents:
mckusick@ernie
kirk(@calder

then any mail arriving for “mckusick” will be redirected to the specified accounts.
2.6. Special Header Lines

Several header lines have special interpretations defined by the configuration file.
Others have interpretations built into sendmail that cannot be changed without changing the code. These builtins are described here.
2.6.1. Return-Receipt-To:

If this header is sent, a message will be sent to any specified addresses when
the final delivery is complete. if the mailer has the 1 flag (local delivery) set in the
mailer descriptor.
2.6.2. Errors-To:

If errors occur anywhere during processing, this header will cause error mes-

sages to go to the listed addresses rather than to the sender. This is intended for
mailing lists.

2.6.3. Apparently-To:

If a message comes in with no recipients listed in the message (in a To:, Cc:, or
Bec: line) then sendmail will add an “Apparently-To:” header line for any recipients

it is aware of. This is not put in as a standard recipient line to warn any recipients

that the list is not complete.

At least one recipient line is required under RFC 822.
3. ARGUMENTS

The complete list of arguments to sendmail is described in detail in Appendix A.

Some important arguments are described here.

2-36

Sendmail Installation and Operation Guide

3.1. Queue Interval
The amount of time between forking a process to run through the queue is

defined by the —q flag.

If you run in mode f or a this can be relatively large, since it

will only be relevant when a host that was down comes back up.

If you run in q mode

it should be relatively short, since it defines the maximum amount of time that a message may sit in the queue.

3.2. Daemon Mode
If you allow incoming mail over an IPC connection, you should have a daemon

running.

This should be set by your /etc/rc file using the —bd flag.

The —bd flag and

the —q flag may be combinedin one call:

/usr/lib/sendmail —bd —q30m
3.3. Forcing the Queue
In some cases you may find that the queue has gotten clogged for some reason.
You can force a queue run using the —q flag (with no value).

It is entertaining to use

the —v flag (verbose) when this is done to watch what happens:

fusr/lib/sendmail —q —v
3.4. Debugging
There are a fairly large number of debug flags built into sendmail.

Each debug

flag has a number and a level, where higher levels means to print out more information.

The convention is that levels greater than nine are “absurd,” i.e., they print out so
much information that you wouldn’t normally want to see them except for debugging

that particular piece of code.

debug-flag:

Debug flags are set using the —d option; the syntax is:

—d debug-list

debug-list:
debug-option [ , debug-option ]
debug-option: debug-range [ . debug-level ]

debug-range:

integer| integer — integer

debug-level:

integer

where spaces are for reading ease only.

~d12

Set flag 12 to level 1

—d12.3

Set flag 12 to level 3

For example,

—-d3-17

Set flags 3 through 17 to level 1

—d3-17.4

Set flags 3 through 17 to level 4

For a complete list of the available debug flags you will have to look at the code (they

are too dynamic to keep this documentation up to date).

3.5. Trying a Different Configuration File
An alternative configuration file can be specified using the —C flag; for example,

/usr/lib/sendmail —Ctest.cf
uses the configuration file test.cf instead of the default /usr/lib/sendmail.cf.
flag has no value it defaults to sendmail.cf in the current directory.

3.6. Changing the Values of Options
Options can be overridden using the —o flag. For example,

If the —C

Sendmail Installation and Operation Guide 2-37

/usr/lib/sendmail —o0T2m

sets the T (timeout) option to two minutes for this run only.
4. TUNING

There are a number of configuration parameters you may want to change, depending
on the requirements of your site. Most of these are set using an option in the
configuration file. For example, the line “OT3d” sets option “T” to the value “3d” (three
days).
4.1. Timeouts

All time intervals are set using a scaled syntax.

For example, “10m” represents

ten minutes, whereas “2h30m” represents two and a half hours. The full set of scales
IS

seconds

minutes

hours

days

weeks

4.1.1. Queue interval

The argument to the —q flag specifies how often a subdaemon will run the
queue. This is typically set to between five minutes and one half hour.
4.1.2. Read timeouts

It is possible to time out when reading the standard input or when reading
from a remote SMTP server. Technically, this is not acceptable within the published protocols. However, it might be appropriate to set it to something large in
certain environments (such as an hour). This will reduce the chance of large
numbers of idle daemons piling up on your system. This timeout is set using the r
option in the configuration file.
4.1.3. Message timeouts

After sitting in the queue for a few days, a message will time out. This is to
insure that at least the sender is aware of the inability to send a message. The
timeout is typically set to three days. This timeout is set using the T option in the
configuration file.

The time of submission is set in the queue, rather than the amount of time left
until timeout. As a result, you can flush messages that have been hanging for a
short period by running the queue with a short message timeout. For example,
/usr/lib/sendmail —oT1d —q

will run the queue and flush anything that is one day old.
4.2. Delivery Mode

There are a number of delivery modes that sendmail can operate in, set by the
“d” configuration option. These modes specify how quickly mail will be delivered.
Legal modes are:

2-38 Sendmail Installation and Operation Guide

deliver interactively (synchronously)

deliver in background (asynchronously)

queue only (don’t deliver)

There are tradeoffs.

Mode “i” passes the maximum amount of information to the

sender, but is hardly ever necessary.

Mode “q” puts the minimum load on your

machine, but means that delivery may be delayed for up to the queue interval.

“b” is probably a good compromise.

Mode

However, this mode can cause large numbers of

processes if you have a mailer that takes a long time to deliver a message.
4.3.

Log Level

The level of logging can be set for sendmail.

The default using a standard

O
=

Major problems only.

Message collections and failed deliveries.

Successful deliveries.

Messages being defered (due to a host being down, etc.).

Normal message queueups.
Unusual but benign incidents, e.g., trying to process a locked queue file.

No logging.

configuration table is level 9. The levels are as follows:

Log internal queue id to external message id mappings.

This can be useful for

tracing a message as it travels between several hosts.
12

Several messages that are basically only of interest when debugging.

Verbose information regarding the queue.

4.4. File Modes
There are a number of files that may have a number of modes.

The modes

depend on what functionality you want and the level of security you require.
4.4.1. To suid or not to suid?

Sendmail can safely be made setuid to root.

At the point where it is about to

exec(2) a mailer, it checks to see if the userid is zero; if so, it resets the userid and
groupid to a default (set by the u and g options).

(This can be overridden by set-

ting the S flag to the mailer for mailers that are trusted and must be called as root.)
However, this will cause mail processing to be accounted (using sa (8)) to root rather

than to the user sending the mail.
4.4.2. Temporary file modes

The mode of all temporary files that sendmail creates is determined by the

“F” option.

Reasonable values for this option are 0600 and 0644.

If the more per-

missive mode is selected, it will not be necessary to run sendmail as root at all (even

when running the queue).
4.4.3. Should my alias database be writable?
At Berkeley we have the alias database (/usr/lib/aliases*) mode 666.

There are

some dangers inherent in this approach: any user can add him-/her-self to any list,
or can “steal” any other user’s mail.

However, we have found users to be basically

trustworthy, and the cost of having a read-only database greater than the expense of
finding and eradicating the rare nasty person.

Sendmail Installation and Operation Guide 2-39

The database that sendmail actually used is represented by the two files
aliases.dir and aliases.pag (both in /usr/lib). The mode on these files should match
the mode on /usr/lib/aliases. If aliases is writable and the DBM files (aliases.dir
and aliases.pag) are not, users will be unable to reflect their desired changes through
to the actual database. However, if aliases is read-only and the DBM files are writable, a slightly sophisticated user can arrange to steal mail anyway.
If your DBM files are not writable by the world or you do not have autorebuild enabled (with the “D” option), then you must be careful to reconstruct the
alias database each time you change the text version:
newaliases

If this step is ignored or forgotten any intended changes will also be ignored or forgotten.

5. THE WHOLE SCOOP ON THE CONFIGURATION FILE
This section describes the configuration file in detail, including hints on how to write
one of your own if you have to.

There is one point that should be made clear immediately: the syntax of the
configuration file is designed to be reasonably easy to parse, since this is done every time
sendmail starts up, rather than easy for a human to read or write. On the “future project”
list is a configuration-file compiler.

An overview of the configuration file is given first, followed by details of the semantics.

5.1. The Syntax

The configuration file is organized as a series of lines, each of which begins with a
single character defining the semantics for the rest of the line. Lines beginning with a
space or a tab are continuation lines (although the semantics are not well defined in
many places). Blank lines and lines beginning with a sharp symbol (‘#’) are comments.
5.1.1. R and S — rewriting rules

The core of address parsing are the rewriting rules. These are an ordered production system. Sendmail scans through the set of rewriting rules looking for a
match on the left hand side (LHS) of the rule. When a rule matches, the address is
replaced by the right hand side (RHS) of the rule.

There are several sets of rewriting rules. Some of the rewriting sets are used
internally and must have specific semantics. Other rewriting sets do not have
specifically assigned semantics, and may be referenced by the mailer definitions or
by other rewriting sets.

The syntax of these two commands are:
Sn

Sets the current ruleset being collected to n. If you begin a ruleset more than once
it deletes the old definition.
Rlhs rhs comments

The fields must be separated by at least one tab character; there may be embedded
spaces in the fields. The lhs is a pattern that is applied to the input. If it matches,
the input is rewritten to the rhs. The comments are ignored.

2-40 Sendmail Installation and Operation Guide

5.1.2. D — define macro
Macros are named with a single character.

These may be selected from the

entire ASCII set, but user-defined macros should be selected from the set of upper
case letters only. Lower case letters and special symbols are used internally.
The syntax for macro definitions is:
Dxval

where x 1s the name of the macro and val is the value it should have. Macros can be
interpolated in most places using the escape sequence $x.
5.1.3. C and F — define classes
Classes of words may be defined to match on the left hand side of rewriting
rules.

For example a class of all local names for this site might be created so that

attempts to send to oneself can be eliminated.

These can either be defined directly

in the configuration file or read in from another file.
from the set of upper case letters.

reserved for system use.

Classes may be given names

Lower case letters and special characters are

The syntax is:

Ccwordl word?2...
Fcfile [ format ]

The first form defines the class ¢ to match any of the named words.

It is permissi-

ble to split them among multiple lines; for example, the two forms:
CHmonet ucbmonet
and

CHmonet
CHucbmonet

are equivalent.

The second form reads the elements of the class ¢ from the named

file; the format is a scanf(3) pattern that should produce a single string.
5.1.4. M — define mailer
Programs and interfaces to mailers are defined in this line. The format is:

Mname, {field=value }*
where name is the name of the mailer (used internally only) and the “field=name”
pairs define attributes of the mailer. Fields are:
Path

The pathname of the mailer

Flags

Special flags for this mailer

Sender

A rewriting set for sender addresses

Recipient
Eol

A rewriting set for recipient addresses
An argument vector to pass to this mailer
The end-of-line string for this mailer

Maxsize

The maximum message length to this mailer

Argv

Only the first character of the field name is checked.
5.1.5. H — define header
The format of the header lines that sendmail inserts into the message are
defined by the H line. The syntax of this line is:

Sendmail Installation and Operation Guide 2-41

Continuation lines in this spec are reflected directly into the outgoing message. The
htemplate is macro expanded before insertion into the message. If the mflags (surrounded by question marks) are specified, at least one of the specified flags must be
stated in the mailer definition for this header to be automatically output. If one of
these headers is in the input it is reflected to the output regardless of these flags.
Some headers have special semantics that will be described below.
5.1.6. O — set option

There are a number of “random” options that can be set from a configuration
file. Options are represented by single characters. The syntax of this line is:
Oovalue

This sets option o to be value. Depending on the option, value may be a string, an
integer, a boolean (with legal values “t”, “T”, “f”, or “F”; the default is TRUE), or a
time interval.

5.1.7. T — define trusted users

Trusted users are those users who are permitted to override the sender address
using the —f flag. These typically are “root,” “uucp,” and “network,” but on some
users it may be convenient to extend this list to include other users, perhaps to support a separate UUCP login for each host. The syntax of this line is:
Tuserl user?2...

There may be more than one of these lines.

5.1.8. P — precedence definitions

Values for the “Precedence:” field may be defined using the P control line.
The syntax of this field is:
Pname=num

When the name is found in a “Precedence:” field, the message class is set to num.
Higher numbers mean higher precedence. Numbers less than zero have the special
property that error messages will not be returned. The default precedence is zero.
For example, our list of precedences is:
Pfirst-class=0
Pspecial-delivery=100
Pjunk=-100

5.2. The Semantics

This section describes the semantics of the configuration file.
5.2.1. Special macros, conditionals

Macros are interpolated using the construct $x, where x is the name of the
macro to be interpolated. In particular, lower case letters are reserved to have special semantics, used to pass information in or out of sendmail, and some special
characters are reserved to provide conditionals, etc.
The following macros must be defined to transmit information into sendmatl:

Sendmail Installation and Operation Guide

The SMTP entry message

The “official” domain name for this site
The format of the UNIX from line
The name of the daemon (for error messages)

The set of “operators” in addresses

default format of sender address

The $e macro is printed out when SMTP starts up.

The first word must be the $j

macro.

The $1 and $n macros can be

The $j macro should be in RFC821 format.

considered constants except under terribly unusual circumstances.

The $0 macro

consists of a list of characters which will be considered tokens and which will
separate tokens when doing parsing.

For example, if “r” were in the $0 macro, then

the input “address” would be scanned as three tokens: “add,” “r,” and “ess.”

Finally, the $q macro specifies how an address should appear in a message when it
is defaulted. For example, on our system these definitions are:

De$j Sendmail $v ready at $b
DnMAILER-DAEMON

DIlFrom $g $d
Do.:% @"=/

Dq$g$?x ($x)$.

Dj$H.$D

An acceptable alternative for the $q macro is “$?7x$x $.<$g>".

These correspond to

the following two formats:
eric@Berkeley (Eric Allman)
Eric Allman <eric@Berkeley>

Some macros are defined by sendmail for interpolation into argv’s for mailers

MHMg<c

=550

OO0 oD

or for other contexts.

2-42

These macros are:

The origination date in Arpanet format
The current date in Arpanet format
The hop count

The date in UNIX (ctime) format
The sender (from) address
The sender address relative to the recipient
The recipient host

The queue id

Sendmail’s pid
Protocol used
Sender’s host name
A numeric representation of the current time
The recipient user
The version number of sendmail

The hostname of this site
The full name of the sender
The id of the sender’s tty
The home directory of the recipient

There are three types of dates that can be used.

The $a and $b macros are in

Arpanet format; $a is the time as extracted from the “Date:” line of the message (if

there was one), and $b is the current date and time (used for postmarks). If no
“Date:” line is found in the incoming message, $a is set to the current time also.
The $d macro is equivalent to the $a macro in UNIX (ctime) format.

Sendmail Installation and Operation Guide

2-43

The $f macro is the id of the sender as originally determined; when mailing to

a specific host the $g macro is set to the address of the sender relative to the recipient. For example, if I send to “bollard@matisse” from the machine “ucbarpa”
the $f macro will be “eric” and the $g macro will be “eric@ucbarpa.”
The $x macro is set to the full name of the sender. This can be determined in

several ways.

It can be passed as flag to sendmail.

The second choice is the value

of the “Full-name:” line in the header if it exists, and the third choice is the comment field of a “From:” line. If all of these fail, and if the message is being originated locally, the full name is looked up in the /etc/passwd file.
When sending, the $h, $u, and $z macros get set to the host, user, and home
The first two are set from the $ @ and $: part

directory (if local) of the recipient.
of the rewriting rules, respectively.

The

$p and

$t macros are used to create unique strings

“Message-Id:” field).

(e.g.,

for the

The $i macro is set to the queue id on this host; if put into

the timestamp line it can be extremely useful for tracking messages. The $y macro
is set to the id of the terminal of the sender (if known); some systems like to put
this in the Unix “From” line.

The $v macro is set to be the version number of

sendmail; this is normally put in timestamps and has been proven extremely useful
for debugging.

The $w macro is set to the name of this host if it can be deter-

mined. The $c field is set to the “hop count,” i.e., the number of times this message

has been processed. This can be determined by the —h flag on the command line or
by counting the timestamps in the message.
The $r and $s fields are set to the protocol used to communicate with sendmail and the sending hostname; these are not supported in the current version.

Conditionals can be specified using the syntax:

$27x textl $ltext2 $.
This interpolates textl if the macro $x is set, and text2 otherwise. The “else”($])
clause may be omitted.
5.2.2. Special classes
The class $=w is set to be the set of all names this host is known by.

This

can be used to delete local hostnames.

5.2.3. The left hand side
The left hand side of rewriting rules contains a pattern.

simply matched directly.

Metasyntax

is introduced

Normal words are

using a dollar

sign.

The

metasymbols are:

$*
$+

Match zero or more tokens
Match one or more tokens

$—

Match exactly one token

$=x

Match any token in class x

Match any token not in class x

If any of these match, they are assigned to the symbol $n for replacement on the
right hand side, where n is the index in the LHS. For example, if the LHS:
$—:5+

is applied to the input:

UCBARPA:eric
the rule will match, and the values passed to the RHS will be:

2-44

Sendmail Installation and Operation Guide

UCBARPA

eric

5.2.4. The right hand side
When the right hand side of a rewriting rule matches, the input is deleted and
replaced by the right hand side.

Tokens are copied directly from the RHS unless

they are begin with a dollar sign. Metasymbols are:
$n

Substitute indefinite token n from LHS

$>n

“Call” ruleset n

$#mailer Resolve to mailer

$ @host

Specify host

$:user

Specify user

The $n syntax substitutes the corresponding value from a $+, $—, $*, $=, or
$~ match on the LHS. It may be used anywhere.
The $>n syntax causes the remainder of the line to be substituted as usual
and then passed as the argument to ruleset n.

The final value of ruleset n then

becomes the substitution for this rule.
The $# syntax should only be used in ruleset zero.

It causes evaluation of the

ruleset to terminate immediately, and signals to sendmail that the address has completely resolved.

The complete syntax is:

$#mailer$ @hostP:user
This specifies the {mailer, host, user} 3-tuple necessary to direct the mailer.
mailer is local the host part may be omitted.

If the

The mailer and host must be a single

word, but the user may be multi-part.
A RHS may also be preceeded by a $ @ or a $: to control evaluation.

A $ @

prefix causes the ruleset to return with the remainder of the RHS as the value. A $:
prefix causes the rule to terminate immediately, but the ruleset to continue; this can

be used to avoid continued application of a rule.

The prefix is stripped before con-

tinuing.

The $ @ and $: prefixes may preceed a $> spec; for example:
R$+

$:$>7%1

matches anything, passes that to ruleset seven, and continues; the $: is necessary to

avoid an infinite loop.
5.2.5. Semantics of rewriting rule sets
There are five rewriting sets that have specific semantics.

These are related as

depicted by figure 2.
Ruleset three should turn the address into “canonical form.” This form should
have the basic syntax:

local-part@host-domain-spec
If no “@” sign is specified, then the host-domain-spec may be appended from the

sender address (if the C flag is set in the mailer definition corresponding to the
sending mailer).

Ruleset three is applied by sendmail before doing anything with

any address.
Ruleset zero is applied after ruleset three to addresses that are going to actu-

ally specify recipients.

{mailer, host, user} triple.

The mailer

must be defined in the mailer definitions from the configuration file.

The host is

add

-7

Avvcr/a.\ia-avui

It must resolve to a
BB

BRSO

S B

LAV

L N

Sendmail Installation and Operation Guide 2-45

domain

addition

address

-t

Rewriting

Figure
D--Sender

b resolved

Set

Semantics

S--mailer-specific

R--mailer-specific

recipient

sender

rewriting

defined into the $h macro for use in the argv expansion of the specified mailer.
Rulesets one and two are applied to all sender and recipient addresses respectively. They are applied before any specification in the mailer definition. They
must never resolve.

Ruleset four is applied to all addresses in the message. It is typically used to
translate internal to external form.

5.2.6. Mailer flags etc.

There are a number of flags that may be associated with each mailer, each
identified by a letter of the alphabet. Many of them are assigned semantics internally. These are detailed in Appendix C. Any other flags may be used freely to conditionally assign headers to messages destined for particular mailers.
5.2.7. The “error” mailer

The mailer with the special name “error” can be used to generate a user error.

The (optional) host field is a numeric exit status to be returned, and the user field is
a message to be printed. For example, the entry:

$#error$:Host unknown in this domain

on the RHS of a rule will cause the specified error to be generated if the LHS
matches. This mailer is only functional in ruleset zero.

5.3. Building a Configuration File From Scratch

Building a configuration table from scratch is an extremely difficult job. Fortunately, it is almost never necessary to do so; nearly every situation that may come up
may be resolved by changing an existing table. In any case, it is critical that you
understand what it is that you are trying to do and come up with a philosophy for the
configuration table. This section is intended to explain what the real purpose of a
configuration table is and to give you some ideas for what your philosophy might be.

"'!

pea

p—i

[

Set

(‘E

5.3.1. What you are trying to do
three major purposes. The first and simplestis to
mazl. This involves settingthe options, defining a

2-46

Sendmail Installation and Operation Guide

few critical macros, etc.

Since these are described in other places, we will not go

into more detail here.
The second purpose is to rewrite addresses in the message.

cally be done in two phases.
canonical form.

This should typi-

The first phase maps addresses in any format into a

This should be done in ruleset three.

The second phase maps this

canonical form into the syntax appropriate for the receiving mailer.
this in three subphases.

Sendmail does

Rulesets one and two are applied to all sender and reci-

pient addresses respectively.

After this, you may specify per-mailer rulesets for both

sender and recipient addresses; this allows mailer-specific customization.

Finally,

ruleset four is applied to do any default conversion to external form.
The third purpose is to map addresses into the actual set of instructions necessary to get the message delivered.

Ruleset zero must resolve to the internal form,

which is in turn used as a pointer to a mailer descriptor.

The mailer descriptor

describes the interface requirements of the mailer.
5.3.2. Philosophy
The particular philosophy you choose will depend heavily on the size and

structure of your organization.

I will present a few possible philosophies here.

One general point applies to all of these philosophies: it is almost always a mistake to try to do full name resolution.
of the

form ‘“‘user@host”

For example, if you are trying to get names

to the Arpanet,

it does not pay to route them to

“xyzvax!decvax!ucbvax!c70:user@host” since you then depend on several links not
under your control.

The best approach to this problem is to simply forward to

“xyzvaxluser@host” and let xyzvax worry about it from there.

In summary, just get

the message closer to the destination, rather than determining the full path.
9.3.2.1. Large site, many hosts — minimum information
Berkeley is an example of a large site, i.e., more than two or three hosts.

We have decided that the only reasonable philosophy in our environment is to
designate one host as the guru for our site.
of mail it receives.

It must be able to resolve any piece

The other sites should have the minimum amount of infor-

mation they can get away with.

In addition, any information they do have

should be hints rather than solid information.
For example, a typical site on our local ether network is

“monet.” Monet

has a list of known ethernet hosts; if it receives mail for any of them, it can do

direct delivery. If it receives mail for any unknown host, it just passes it directly
to “ucbvax,” our master host.

Ucbvax may determine that the host name is ille-

gal and reject the message, or may be able to do delivery.

However, it is impor-

tant to note that when a new ethernet host is added, the only host that must
have its tables updated is ucbvax; the others may be updated as convenient, but
this is not critical.

This picture is slightly muddied due to network connections that are not

actually located on ucbvax. For example, our TCP connection is currently on
“ucbarpa.” However, monet does not know about this; the information is hidden
totally between ucbvax and ucbarpa.

Mail going from monet to a TCP host is

transfered via the ethernet from monet to ucbvax, then via the ethernet from
ucbvax to ucbarpa, and then is submitted to the Arpanet. Although this involves
some extra hops, we feel this is an acceptable tradeoff.
An interesting point is that it would be possible to update monet to send
TCP mail directly to ucbarpa if the load got too high; if monet failed to note a

Sendmail Installation and Operation Guide 2-47

host as a TCP host it would go via ucbvax as before, and if monet incorrectly
sent a message to ucbarpa it would still be sent by ucbarpa to ucbvax as before.
The only problem that can occur is loops, as if ucbarpa thought that ucbvax had
the TCP connection and vice versa. For this reason, updates should always happen to the master host first.

This philosophy results as much from the need to have a single source for
the configuration files (typically built using m4(1) or some similar tool) as any
logical need. Maintaining more than three separate tables by hand is essentially
an impossible job.

5.3.2.2. Small site — complete information

A small site (two or three hosts) may find it more reasonable to have complete information at each host. This would require that each host know exactly
where each network connection is, possibly including the names of each host on
that network. As long as the site remains small and the the configuration
remains relatively static, the update problem will probably not be too great.
5.3.2.3. Single host

This is in some sense the trivial case. The only major issue is trying to
insure that you don’t have to know too much about your environment. For
example, if you have a UUCP connection you might find it useful to know about
the names of hosts connected directly to you, but this is really not necessary
since this may be determined from the syntax.

5.3.3. Relevant issues

The canonical form you use should almost certainly be as specified in the
Arpanet protocols RFC819 and RFC822. Copies of these RFC’s are included on the

sendmail tape as doc/rfc819.lpr and doc/rfc822.Ipr.

RF(C822 describes the format of the mail message itself. Sendmail follows this
RFC closely, to the extent that many of the standards described in this document
can not be changed without changing the code. In particular, the following characters have special interpretations:

<>{()”\

Any attempt to use these characters for other than their RFC822 purpose in
addresses is probably doomed to disaster.

RFC819 describes the specifics of the domain-based addressing. This is
touched on in RFC822 as well. Essentially each host is given a name which is a
right-to-left dot qualified pseudo-path from a distinguished root. The elements of
the path need not be physical hosts; the domain is logical rather than physical. For
example, at Berkeley one legal host is ‘“a.cc.berkeley.arpa”; reading from right to
left, “arpa” is a top level domain (related to, but not limited to, the physical
Arpanet), “berkeley” is both an Arpanet host and a logical domain which is actually
interpreted by a host called ucbvax (which is actually just the “major” host for this
domain), “cc” represents the Computer Center, (in this case a strictly logical entity),
and “a” is a host in the Computer Center; this particular host happens to be connected via berknet, but other hosts might be connected via one of two ethernets or
some other network.

Beware when reading RFC819 that there are a number of errors in it.

2-48

Sendmail Installation and Operation Guide

5.3.4. How to proceed

Once you have decided on a philosophy, it is worth examining the available
configuration tables to decide if any of them are close enough to steal major parts of.
Even under the worst of conditions, there is a fair amount of boiler plate that can be

collected safely.
The next step is to build ruleset three.

job.

This will be the hardest part of the

Beware of doing too much to the address in this ruleset, since anything you do

will reflect through to the message.

In particular, stripping of local domains is best

deferred, since this can leave you with addresses with no domain spec at all. Since
sendmail likes to append the sending domain to addresses with no domain, this can

change the semantics of addresses. Also try to avoid fully qualifying domains in this
ruleset.

Although technically legal, this can lead to unpleasantly and unnecessarily

long addresses reflected into messages.

The Berkeley configuration files define

ruleset nine to qualify domain names and strip local domains.

This is called from

ruleset zero to get all addresses into a cleaner form.

Once you have ruleset three finished, the other rulesets should be relatively
trivial.

If you need hints, examine the supplied configuration tables.

5.3.5. Testing the rewriting rules — the —bt flag

When you build a configuration table, you can do a certain amount of testing
using the “test mode” of sendmail. For example, you could invoke sendmail as:

sendmail —bt —Ctest.cf
which would read the configuration file “test.cf” and enter test mode.

In this mode,

you enter lines of the form:
rwset address

where rwset is the rewriting set you want to use and address is an address to apply
the set to.

Test mode shows you the steps it takes as it proceeds, finally showing

you the address it ends up with.

You may use a comma separated list of rwsets for

sequential application of rules to an input; ruleset three is always applied first.

For

example:
1,21,4 monet:bollard

first applies ruleset three to the input “monet:bollard.” Ruleset one is then applied
to the output of ruleset three, followed similarly by rulesets twenty-one and four.
If you need more detail, you can also use the “—d21” flag to turn on more

debugging. For example,
sendmail —bt —d21.99
turns on an incredible amount of information; a single word address is probably

going to print out several pages worth of information.
5.3.6. Building mailer descriptions
To add an outgoing mailer to your mail system, you will have to define the

characteristics of the mailer.
Each mailer must have an internal name.

This can be arbitrary, except that

the names “local” and “prog” must be defined.
The pathname of the mailer must be given in the P field.

If this mailer should

be accessed via an IPC connection, use the string “[IPC]” instead.

Sendmail Installation and Operation Guide 2-49

The F field defines the mailer flags. You should specify an “f” or “r” flag to

pass the name of the sender as a —f or —r flag respectively. These flags are only
passed if they were passed to sendmail, so that mailers that give errors under some
circumstances can be placated. If the mailer is not picky you can just specify “—f
$g” in the argv template. If the mailer must be called as root the “S” flag should

be given; this will not reset the userid before calling the mailer’. If this mailer is

local (i.e., will perform final delivery rather than another network hop) the “1” flag
should be given. Quote characters (backslashes and ” marks) can be stripped from
addresses if the “s” flag is specified; if this is not given they are passed through. If
the mailer is capable of sending to more than one user on the same host in a single
transaction the “m” flag should be stated. If this flag is on, then the argv template
containing $u will be repeated for each unique user on a given host. The “e” flag
will mark the mailer as being “expensive,” which will cause sendmail to defer con-

nection until a queue run®.
An unusual case is the “C” flag. This flag applies to the mailer that the message is received from, rather than the mailer being sent to; if set, the domain spec of
the sender (i.e., the “@host.domain” part) is saved and is appended to any
addresses in the message that do not already contain a domain spec. For example, a
message of the form:

From: eric@ucbarpa

To: wnj@monet, mckusick
will be modified to:
From: eric@ucbarpa

To: wnj@monet, mckusick@ucbarpa

if and only if the “C” flag is defined in the mailer corresponding to “eric@ucbarpa.”
Other flags are described in Appendix C.

The S and R fields in the mailer description are per-mailer rewriting sets to be
applied to sender and recipient addresses respectively. These are applied after the
sending domain is appended and the general rewriting sets (numbers one and two)
are applied, but before the output rewrite (ruleset four) is applied. A typical use is
to append the current domain to addresses that do not already have a domain. For
example, a header of the form:
From: eric

might be changed to be:

From: eric@ucbarpa
or

From: ucbvax!eric

depending on the domain it is being shipped into. These sets can also be used to do

special purpose output rewriting in cooperation with ruleset four.

The E field defines the string to use as an end-of-line indication. A string con-

taining only newline is the default. The usual backslash escapes (\r,\n,\f,\b) may
be used.

Finally, an argv template is given as the E field. It may have embedded
spaces. If there is no argv with a $u macro in it, sendmail will speak SMTP to the
mailer. If the pathname for this mailer is “[IPC],” the argv should be

Sendmail must be runnin
.

root for this to work.
M

)

“The “c” configuration option must be given for this to be effective.

2-50 Sendmail Installation and Operation Guide

IPC $h [ port ]
where port is the optional port number to connect to.
For example, the specifications:

Mlocal, P=/bin/mail, F=rlsm S=10, R=20, A=mail —d $u

Mether,P=[IPC],

F=meC, S=11, R=21, A=IPC $h, M=100000

specifies a mailer to do local delivery and a mailer for ethernet delivery.

The first is

called “local,” is located in the file “/bin/mail,” takes a picky —r flag, does local

delivery, quotes should be stripped from addresses, and multiple users can be
delivered at once; ruleset ten should be applied to sender addresses in the message
and ruleset twenty should be applied to recipient addresses; the argv to send to a
message will be the word “mail,” the word “—d,” and words containing the name of

the receiving user.

If a —r flag is inserted it will be between the words “mail” and
“—~d.” The second mailer is called “ether,” it should be connected to via an IPC con-

nection, it can handle multiple users at once, connections should be deferred, and
any domain from the sender address should be appended to any receiver name
without a domain; sender addresses should be processed by ruleset eleven and reci-

pient addresses by ruleset twenty-one.

passed through this mailer.

There is a 100,000 byte limit on messages

Sendmail Installation and Operation Guide 2-51

APPENDIX A

COMMAND

LINE FLAGS

Arguments must be presented with flags before addresses. The flags are:

—f addr

The sender’s machine address is addr.

This flag is ignored unless the real user

is listed as a “trusted user” or if addr contains an exclamation point (because
of certain restrictions in UUCP).
—r addr
—h cnt

- An obsolete form of —f.

Sets the “hop count” to cnt.

This represents the number of times this mes-

sage has been processed by sendmail (to the extent that it is supported by the

underlying networks).

Cnt is incremented during processing, and if it reaches
MAXHOP (currently 30) sendmail throws away the message with an error.

—Fname

Sets the full name of this user to name.
Don’t do aliasing or forwarding.

Read the header for “To:”, “Cc:”, and “Bcec:” lines, and send to everyone
listed in those lists.

The “Bcce:” line will be deleted before sending.

Any

addresses in the argument vector will be deleted from the send list.

Deliver mail (default)

Run in arpanet mode (see below)

Run as a daemon

N
= <

m
c+ 0. ®

Set operation mode to x. Operation modes are:

Speak SMTP on input side
Run in test mode
Just verify addresses, don’t collect or deliver
Initialize the alias database

Print the mail queue
Freeze the configuration file

The special processing for the ARPANET includes reading the “From:” line
from the header to find the sender, printing ARPANET style messages (pre-

ceded by three digit reply codes for compatibility with the FTP protocol
[Neigus73,

Postel74,

Postel77]),

and ending lines of error messages

with

<CRLF>.
—qtime

Try to process the queued up mail.

If the time is given, a sendmail will run
through the queue at the specified interval to deliver queued mail; otherwise,
it only runs once.

—Cfile

Use a different configuration file.

—dlevel

Set debugging level.

—ox value

Set option x to the specified value.

These options are described in Appendix

There are a number of options that may be specified as primitive flags (provided for

compatibility with delivermail).
be specified as the —s flag.

These are the e, i, m, and v options.

Also, the f option may

2-52 Sendmail Installation and Operation Guide

APPENDIX B

CONFIGURATION OPTIONS

The following options may be set using the —o flag on the command line or the O line in
the configuration file:
Afile

Use the named file as the alias file.

If no file is specified, use aliases in the

current directory.

If set, wait for an “@:@” entry to exist in the alias database before starting
up. If it does not appear in five minutes, rebuild the database.
If an outgoing mailer is marked as being expensive, don’t connect immedi-

ately. This requires that queueing be compiled in, since it will depend on a
queue run process to actually send the mail.
dx

Deliver in mode x. Legal modes are:

Deliver interactively (synchronously)

b
q

Deliver in background (asynchronously)
Just queue the message (deliver during queue run)

If set, rebuild the alias database if necessary and possible. If this option is not

set, sendmail will never rebuild the alias database unless explicitly requested
using —bi.

Dispose of errors using mode x. The values for x are:

g 5o

Print error messages (default)
No messages, just give exit status
Mail back errors

Write back errors (mail if user not logged in)
Mail back errors and give zero exit stat always

The temporary file mode, in octal. 644 and 600 are good choices.

Save Unix-style “From” lines at the front of headers.
assumed redundant and discarded.
gn

Set the default group id for mailers to run in to n.

Hfile

Specify the help file for SMTP.

Ignore dots in incoming messages.

Set the default log level to n.

Mx value

Set the macro x to value.

Normally they are

This is intended only for use from the command

line.

Send to me too, even if I am in an alias expansien.

Assume that the headers may be in old format, i.e., spaces delimit names.
This actually turns on an adaptive algorithm: if any recipient address contains

a comma, parenthesis, or angle bracket, it will be assumed that commas
already exist. If this flag is not on, only commas delimit names. Headers are
always output with commas between the names.
Use the named dir as the queue directory.

Sendmail Installation and Operation Guide 2-53

Sfile

Log statistics in the named file.

Be super-safe when running things, i.e., always instantiate the queue file, even
if you are going to attempt immediate delivery. Sendmail always instantiates
the queue file before returning control the the client under any circumstances.
Ttime

tS,D

Set the queue timeout to time. After this interval, messages that have not
been successfully sent will be returned to the sender.

Set the local timezone name to S for standard time and D for daylight time;

this is only used under version six.

Set the default userid for mailers to n.

mailer definition will run as this user.
Run in verbose mode.

Mailers without the S flag in the

2-54 Sendmail Installation and Operation Guide

APPENDIX C

MAILER FLAGS

The following flags may be set in the mailer description.
f

The mailer wants a —f from flag, but only if this is a network forward operation (i.e., the
mailer will give an error if the executing user does not have special permissions).

Same as f, but sends a —r flag.

Don’t reset the userid before calling the mailer. This would be used in a secure environment where sendmail ran as root.

This could be used to avoid forged addresses.

This

flag is suppressed if given from an “unsafe” environment (e.g, a user’s mail.cf file).
Do not insert a UNIX-style “From” line on the front of the message.
This mailer is local (i.e., final delivery will be performed).

Strip quote characters off of the address before calling the mailer.
This mailer can send to multiple users on the same host in one transaction.

When a $u

macro occurs in the argv part of the mailer definition, that field will be repeated as neces-

- -

sary for all qualifying users.
This mailer wants a “From:” header line.
This mailer wants a “Date:” header line.
This mailer wants a “Message-Id:” header line.
This mailer wants a “Full-Name:” header line.
This mailer wants a “Return-Path:” line.

Upper case should be preserved in user names for this mailer.

=E- S

Upper case should be preserved in host names for this mailer.
This is an Arpanet-compatible mailer, and all appropriate modes should be set.
This mailer wants Unix-style “From” lines with the ugly UUCP-style “remote from

<host>" on the end.
This mailer is expensive to connect to, so try to avoid connecting normally; any necessary

connection will occur during a queue run.
This mailer want to use the hidden dot algorithm as specified in RFC821; basically, any
line beginning with a dot will have an extra dot prepended (to be stripped at the other

end).

This insures that lines in the message containing a dot will not terminate the mes-

sage prematurely.
Limit the line lengths as specified in RFC821.

Use the return-path in the SMTP “MAIL FROM:” command rather than just the return

address; although this is required in RFC821, many hosts do not process return paths
properly.

This mailer will be speaking SMTP to another sendmail — as such it can use special protocol features.

This option is not required (i.e., if this option is omitted the transmission

will still operate successfully, although perhaps not as efficiently as possible).

If mail is received from a mailer with this flag set, any addresses in the header that do
not have an at sign (“@”’) after being rewritten by ruleset three will have the “@domain”
clause from the sender tacked on.
@ AR
AR N

B A

A B

This allows mail with headers of the form:
R

RBANS VY

o B B

A A

AER%WECANA

S A

YAAR

Sendmail Installation and Operation Guide 2-55

From: usera@hosta

To: userb@hostb, userc
to be rewritten as:
From: usera@hosta
To: userb@hostb, userc@hosta

automatically.

2-56 Sendmail Installation and Operation Guide

APPENDIX D

OTHER CONFIGURATION

There are some configuration changes that can be made by recompiling sendmail. These
are located in three places:

md/config.m4

These contain operating-system dependent descriptions. They are interpolated into the Makefiles in the src and aux directories. This includes information about what version of UNIX you are running, what libraries you have to
include, etc.

src/conf.h

Configuration parameters that may be tweaked by the installer are included in
conf.h.

src/conf.c

Some special routines and a few variables may be defined in conf.c. For the
most part these are selected from the settings in conf.h.

Parameters in md/config.m4
m4CONFIG

The following compilation flags may be defined in the
md/config.m4 to define the environment in which you are operating.

Vo6

macro in

If set, this will compile a version 6 system, with 8-bit user id’s, single character
tty id’s, etc.

VMUNIX

If set, you will be assumed to have a Berkeley 4BSD or 4.1BSD, including the
vfork (2) system call, special types defined in <sys/types.h> (e.g, u char), etc.

If none of these flags are set, a version 7 system is assumed.

You will also have to specify what libraries to link with sendmail in the m4LIBS macro.
Most notably, you will have to include if you are running a 4.1BSD system.
Parameters in src/conf.h

Parameters and compilation options are defined in conf.h. Most of these need not normally be tweaked; common parameters are all in sendmail.cf. However, the sizes of certain
primitive vectors, etc., are included in this file. The numbers following the parameters are
their default value.

MAXLINE [256] The maximum line length of any input line. If message lines exceed this
length they will still be processed correctly; however, header lines,
configuration file lines, alias lines, etc., must fit within this limit.

MAXNAME [128] The maximum length of any name, such as a host or a user name.
MAXFIELD [2500]

The maximum total length of any header field, including continuation
lines.

MAXPV [40]
MAXHOP [30]

The maximum number of parameters to any mailer.

This limits the

number of recipients that may be passed in one transaction.

When a message has been processed more than this number of times, send-

mail rejects the message on the assumption that there has been an aliasing
loop. This can be determined from the —h flag or by counting the number
of trace fields (i.e, “Received:” lines) in the message header.
A

B Ot

W/ AV

G
s WA

BB LB

S B

-G

VL T

Vee

S=es

Sendmail Installation and Operation Guide 2-57

MAXATOM [100] The maximum number of atoms (tokens) in a single address. For example,
the address “eric@Berkeley” is three atoms.
MAXMAILERS [25]
The maximum number of mailers that may be defined in the configuration
file.

MAXRWSETS [30]

The maximum number of rewriting sets that may be defined.
MAXPRIORITIES [25]
The maximum number of values for the “Precedence:” field that may be
defined (using the P line in sendmail.cf).

MAXTRUST [30] The maximum number of trusted users that may be defined (using the T
line in sendmail.cf).

A number of other compilation options exist.

These specify whether or not specific code

should be compiled in.

DBM

If set, the “DBM” package in UNIX is used (see DBM(3X) in [UNIX80]). If
not set, a much less efficient algorithm for processing aliases is used.

DEBUG

If set, debugging information is compiled in.

To actually get the debugging

output, the —d flag must be used.

LOG

If set, the syslog routine in use at some sites is used. This makes an informational log record for each message processed, and makes a higher priority log
record for internal system errors.

QUEUE

This flag should be set to compile in the queueing code. If this is not set,
mailers must accept the mail immediately or it will be returned to the sender.

SMTP

If set, the code to handle user and server SMTP will be compiled in. This is
only necessary if your machine has some mailer that speaks SMTP.

DAEMON

If set, code to run a daemon is compiled in. This code is for 4.2BSD if the
NVMUNIX flag is specified; otherwise, 4.1a BSD code is used. Beware however that there are bugs in the 4.1a code that make it impossible for sendmail to work correctly under heavy load.

UGLYUUCP

If you have a UUCP host adjacent to you which is not running a reasonable
version of rmail, you will have to set this flag to include the “remote from

sysname” info on the from line. Otherwise, UUCP gets confused about where

the mail came from.

NOTUNIX

If you are using a non-UNIX mail format, you can set this flag to turn off special processing of UNIX-style “From ” lines.

Configuration in src/conf.c

Not all header semantics are defined in the configuration file. Header lines that should
only be included by certain mailers (as well as other more obscure semantics) must be
specified in the HdrInfo table in conf.c. This table contains the header name (which should
be in all lower case) and a set of header control flags (described below), The flags are:
H ACHECK

H EOH

Normally when the check is made to see if a header line is compatible with a
mailer, sendmail will not delete an existing line. If this flag is set, sendmail
will delete even existing header lines. That is, if this bit is set and the mailer
does not have flag bits set that intersect with the required mailer flags in the
header definition in sendmail.cf, the header line is always deleted.
If this header field is set, treat it like a blank line, i.e., it will signal the end of
the header and the beginning of the message text.

2-58 Sendmail Installation and Operation Guide

H FORCE

Add this header entry even if one existed in the message before. If a header
entry does not have this bit set, sendmail will not add another header line if a
header line of this name already existed. This would normally be used to
stamp the message by everyone who handled it.

H TRACE

If set, this is a timestamp (trace) field.

If the number of trace fields in a mes-

sage exceeds a preset amount the message is returned on the assumption that
it has an aliasing loop.

H RCPT

If set, this field contains recipient addresses. This is used by the —t flag to
determine who to send to when it is collecting recipients from the message.

H FROM

This flag indicates that this field specifies a sender.

The order of these fields

in the HdrInfo table specifies sendmail’s preference for which field to return
error messages to.

Let’s look at a sample HdrInfo specification:
struct hdrinfo

{

HdrInfo[] =

/* originator fields, most to least significant */
"resent-sender”,
H FROM,
”resent-from”,
H FROM,
”sender”,
H FROM,
”from”,
H FROM,
”full-name”,
H ACHECK,
/* destination fields */
”t0”,
H RCPT,
”resent-to”,
H RCPT,
”cc”,
H RCPT,
/* message identification and control */
”message”,
H EOH,
“text”,
H EOH,
/* trace fields */

"received”,

H TRACEH FORCE,

NULL,

I
This structure indicates that the “To:”, “Resent-To:”, and “Cc:” fields all specify recipient
addresses. Any ‘“Full-Name:” field will be deleted unless the required mailer flag (indicated in
the configuration file) is specified. The ‘“Message:” and “Text:” fields will terminate the
header; these are specified in new protocols [NBS80] or used by random dissenters around the
network world.

The “Received:” field will always be added, and can be used to trace mes-

sages.

There are a number of important points here.

First, header fields are not added

automatically just because they are in the HdrInfo structure; they must be specified in the
configuration file in order to be added to the message. Any header fields mentioned in the
configuration file but not mentioned in the HdrInfo structure have default processing performed; that is, they are added unless they were in the message already. Second, the HdrInfo
structure only specifies cliched processing; certain headers are processed specially by ad hoc
code regardless of the status specified in HdrInfo. For example, the “Sender:” and “From:”
fields are always scanned on ARPANET mail to determine the sender; this is used to perform

the “return to sender” function. The “From:” and ‘“Full-Name:” fields are used to determine

the full name of the sender if possible; this is stored in the macro $x and used in a number of
ways.

Sendmail Installation and Operation Guide 2-59

The file conf.c also contains the specification of ARPANET reply codes.

There are four

classifications these fall into:
char
char
char
char

Arpa Info[] =
Arpa TSyserr[] =
Arpa PSyserr[] =
Arpa Usrerr[] =

”050”;
”455”;
7554”;
”554”;

/* arbitrary info */
/* some (transient) system error */
/* some (transient) system error */
/* some (fatal) user error */

The class Arpa Info is for any information
that is not required by the protocol, such as for-

warding information.

Arpa T'Syserr and Arpa PSyserr is printed by the syserr routine.

TSyserr is printed out for transient errors, whereas PSyserr is printed for permanent errors;

the distinction is made based on the value of errno.

Finally, Arpa Usrerr is the result of a

user error and is generated by the usrerr routine; these are generated when the user has

specified something wrong, and hence the error is permanent, i.e., it will not work simply by
resubmitting the request.
If it is necessary to restrict mail through a relay, the checkcompat routine can be

modified.

This routine is called for every recipient address.

It can return TRUE to indicate

that the address is acceptable and mail processing will continue, or it can return FALSE to
reject the recipient.

If it returns false, it is up to checkcompat to print an error message

(using usrerr) saying why the message is rejected. For example, checkcompat could read:
bool

checkcompat(to)
register ADDRESS *to;
if (MsgSize > 50000 && to->q mailer != LocalMailer)
usrerr(”Message too large for non-local delivery”);

NoReturn = TRUE;
return (FALSE);

}

1}'eturn (TRUE);

This would reject messages greater than 50000 bytes unless they were local.

The NoReturn

flag can be sent to supress the return of the actual body of the message in the error return.
The actual use of this routine is highly dependent on the implementation, and use should be

limited.

2-60 Sendmail Installation and Operation Guide

APPENDIX E

SUMMARY OF SUPPORT FILES

This is a summary of the support files that sendmail creates or generates.

/usr/lib/sendmail

The binary of sendmail.

/usr/bin/newaliases

A link to /usr/lib/sendmail; causes the alias database to be rebuilt. Running
this program is completely equivalent to giving sendmail the —bi flag.

/usr/bin/mailq Prints a listing of the mail queue. This program is equivalent to using the
—bp flag to sendmail.

/usr/lib/sendmail.cf

The configuration file, in textual form.

/usr/lib/sendmail.fc
The configuration file represented as a memory image.
/usr/lib/sendmail.hf
The SMTP help file.

/usr/lib/sendmail.st
A statistics file; need not be present.

/usr/lib/aliases The textual version of the alias file.
/usr/lib/aliases. {pag,dir}
The alias file in dbm (3) format.

/etc/syslog

The program to do logging.

/etc/syslog.conf The configuration file for syslog.

/etc/syslog.pid Contains the process id of the currently running syslog.
/usr/spool/mqueue
The directory in which the mail queue and temporary files reside.

/usr/spool/mqueue/qf*

Control (queue) files for messages.

/usr/spool/mqueue/df*
Data files.

/usr/spool/mqueue/lf*
Lock files

/usr/spool/mqueue/tf*

Temporary versions of the gf files, used during queue file rebuild.

/usr/spool/mqueue/nf*

A file used when creating a unique id.

/usr/spool/mqueue/xf*

A transcript of the current session.

Introduction 3-1

PART 3: COMMUNICATIONS

The three articles in this part cover a range of communications topics, from general background information to detailed descriptions of program structures and protocols. They

describe the interprocess communication software (this can be either interactive or batch) and

sendmail, an internetwork mail server.

Ftp, telnet, and the r-command set are three other networking software utilities available on
the ULTRIX-32 system but not mentioned in these articles. See the end of this introduction
for a brief description of each.

Interprocess Communication

The first two articles describe the socket software, a set of system calls (new with the 4.2BSD
distribution) used for interprocess communication. The communicating processes can be running on the same computer or on separate computers linked by the DARPA standard communication protocols.

Interprocess communication requires each process to set up one of three types of socket:

Stream socket
Datagram socket

Raw socket

Communication is bidirectional, reliable, sequenced, and unduplicated.
= Communication is bidirectional but not promised to be reliable,
sequenced, or unduplicated.

Communication is possible through access to underlying protocols.

“A 4.2BSD Interprocess Communication Primer” gives the format for each socket-related call
and explains how to coordinate the calls to establish a connection and send and receive messages:

Create a socket

Bind a name to a socket

(Connect - initiate a connection

Listen for a connect request

Accept a connect request

Write a message

Read a message

Send a message

Receive a message

Sendto - send a datagram message

Recvfrom - receive a datagram message

(lose a connection

3-2 Introduction

These commands are listed individually in the ULTRIX-32 Programmer’s Manual. The article also tells how to use: a library of routines that manipulate addresses, server and client
calls, and connectionless servers. And information on a variety of advanced topics is available
for sophisticated users.

The second article, “4.2BSD Networking Implementation Notes,” describes the internal structure of the interprocess communication software. This information should be useful to
engineers who are developing new communication protocols and network utilities. The article
explains:

Support for multiple protocol families and addressing styles

Structures for internal address representation

Memory management for network functions

Internal layering

Protocol interfaces

(Gateways

Routing tables

Use of raw sockets for direct access to low level protocols

Buffering issues

Handling out-of-band data
o

Use of trailer protocols

You can also find a description of the user interface to the interprocess communication
software in Section 2.3 of the “4.2BSD System Manual” (in Volume II of this set). Prefer the
more recent “4.2BSD Interprocess Communication Primer” when you find discrepancies.
Sendmail

The article by Allman, “Sendmail - An Internetwork Mail Router,” offers good background
information for people who install and maintain the sendmail utility. For actual instructions
on installation, see the ‘“Sendmail Installation and Operation Guide” in Part 2 of this volume.

Sendmail acts like a post office, enabling different networking systems to route mail between
them. For example, people using the ARPANET and others using the ETHERNET can send
mail to each other, and sendmail will cooperate with the network software at each end to
make sure that the messages get through. The sendmail functions are transparent to people
sending the messages; each sender or receiver needs to deal only with the interface to the local
network used on his or her computer system.

A reading of this article is prerequisite to an understanding of the “Sendmail Installation and
Operation Guide.”

Standard Networking Utilities
Three other networking systems are available on ULTRIX-32:

ftp

File transfer program (a user interface to the ARPANET)

telnet

Remote login protocol

r-commands

New networking software layered on sockets

You can find information on these utilities in the ULTRIX-32 Programmer’s Manual. Ftp
and telnet are utilities that prompt you for commands when you run them. Users must give
appropriate passwords when accessing information on remote systems. The command descriptions listed under ftp and telnet are comprehensive.

The r-commands, like the interprocess communication commands, are listed individually in
the ULTRIX-32 Programmer’s Manual, because you must call each one from the shell:

Introduction 3-3

remd

Connect, then execute a command.

rcp

Remote file copy

rdump

File system dump across a network

rexec

Remote execute

rlogin

Remote login

rmt

Remote magnetic tape dump

rrestore

Restore a system from a file system dump across a network

rshell

Remote shell; provide remote execution facilities

ruptime

Show how long a remote system has been up

rwho

Show who is on a remote system

However, the r-command software requires trust between system users, because remote access
using the r-commands does not require use of passwords.

Interprocess Communication Primer 3-5

A 4.2BSD Interprocess Communication Primer
Samuel J. Leffler
Robert S. Fabry
William N. Joy

Computer Systems Research Group

Department of Electrical Engineering and Computer Science
University of California, Berkeley
Berkeley, California 94720
(415) 642-7780

1. INTRODUCTION

One of the most important parts of 4.2BSD is the interprocess communication facilities.
These facilities are the result of more than two years of discussion and research. The facilities
provided in 4.2BSD incorporate many of the ideas from current research, while trying to
maintain the UNIX* philosophy of simplicity and conciseness. It is hoped that the interprocess communication facilities included in 4.2BSD will establish a standard for UNIX. From
the response to the design, it appears many organizations carrying out work with UNIX are
adopting it.

UNIX has previously been very weak in the area of interprocess communication. Prior
4.2BSD facilities, the only standard mechanism which allowed two processes to comthe
to
municate were pipes (the mpx files which were part of Version 7 were experimental). Unfor-

tunately, pipes are very restrictive in that the two communicating processes must be related
through a common ancestor. Further, the semantics of pipes makes them almost impossible
to maintain in a distributed environment.

Earlier attempts at extending the ipc facilities of UNIX have met with mixed reaction.
The majority of the problems have been related to the fact these facilities have been tied to
the UNIX file system; either through naming, or implementation. Consequently, the ipc facilities provided in 4.2BSD have been designed as a totally independent subsystem. The 4.2BSD
ipc allows processes to rendezvous in many ways. Processes may rendezvous through a UNIX
file system-like name space (a space where all names are path names) as well as through a network name space. In fact, new name spaces may be added at a future time with only minor
changes visible to users. Further, the communication facilities have been extended to
included more than the simple byte stream provided by a pipe-like entity. These extensions
have resulted in a completely new part of the system which users will need time to familiarize
themselves with. It is likely that as more use is made of these facilities they will be refined;
only time will tell.

The remainder of this document is organized in four sections. Section 2 introduces the
new system calls and the basic model of communication. Section 3 describes some of the supporting library routines users may find useful in constructing distributed applications. Section
4 is concerned with the client/server model used in developing applications and includes
examples of the two major types of servers. Section 5 delves into advanced topics which
sophisticated users are likely to encounter when using the ipc facilities.

* UNIX is a Trademark of Bell Laboratories.

3-6 Interprocess Communication Primer
2. BASICS
The basic building block for communication is the socket.
communication to which a name may be bound.
more associated processes.

A socket is an endpoint of

Each socket in use has a type and one or

Sockets exist within communication domains.

A communication

domain is an abstraction introduced to bundle common properties of processes communicating
through sockets.

One such property is the scheme used to name sockets.

For example, in the

UNIX communication domain sockets are named with UNIX path names; e.g. a socket may

be named “/dev/foo”.

Sockets normally exchange data only with sockets in the same domain

(it may be possible to cross domain boundaries, but only if some translation process is performed).

The 4.2BSD ipc supports two separate communication domains: the UNIX domain,

and the Internet domain is used by processes which communicate using the the DARPA standard communication protocols. The underlying communication facilities provided by these

domains have a significant influence on the internal system implementation as well as the
interface to socket facilities available to a user.

An example of the latter is that a socket

“operating” in the UNIX domain sees a subset of the possible error conditions which are pos-

sible when operating in the Internet domain.
2.1.

Socket types

Sockets are typed according to the communication properties visible to a user. Processes
are presumed to communicate only between sockets of the same type, although there is nothing that prevents communication between sockets of different types should the underlying

communication protocols support this.
Three types of sockets currently are available to a user.

A stream socket provides for

the bidirectional, reliable, sequenced, and unduplicated flow of data without record boun-

daries.

Aside from the bidirectionality of data flow, a pair of connected stream sockets pro-

vides an interface nearly identical to that of pipes*.
A datagram socket supports bidirectional flow of data which is not promised to be
sequenced, reliable, or unduplicated. That is, a process receiving messages on a datagram
socket may find messages duplicated, and, possibly, in an order different from the order in
which it was sent. An important characteristic of a datagram socket is that record boundaries
in data are preserved.

Datagram sockets closely model the facilities found in many contem-

porary packet switched networks such as the Ethernet.
A raw socket provides users access to the underlying communication protocols which

support socket abstractions.

These sockets are normally datagram oriented, though their

exact characteristics are dependent on the interface provided by the protocol.

Raw sockets

are not intended for the general user; they have been provided mainly for those interested in

developing new communication protocols, or for gaining access to some of the more esoteric
facilities of an existing protocol.

The use of raw sockets is considered in section 5.

Two potential socket types which have interesting properties are the sequenced packet

socket and the reliably delivered message socket.

A sequenced packet socket is identical to a

stream socket with the exception that record boundaries are preserved.

This interface is very

similar to that provided by the Xerox NS Sequenced Packet protocol.

The reliably delivered

message socket has similar properties to a datagram socket, but with reliable delivery.

While

these two socket types have been loosely defined, they are currently unimplemented in
4.2BSD.

As such, in this document we will concern ourselves only with the three socket types

for which support exists.

* In the UNIX domain, in fact, the semantics are identical and, as one might expect, pipes have been imple-

mented internally as simply a pair of connected stream sockets.

Interprocess Communication Primer 3-7
2.2.

Socket creation

To create a socket the socket system call is used:

s = socket(domain, type, protocol);
This call requests that the system create a socket in the specified domain and of the specified
type. A particular protocol may also be requested. If the protocol is left unspecified (a value
of 0), the system will select an appropriate protocol from those protocols which comprise the
communication domain and which may be used to support the requested socket type. The
user is returned a descriptor (a small integer number) which may be used in later system calls
which operate on sockets. The domain is specified as one of the manifest constants defined in
the file <sys/socket.h>. For the UNIX domain the constant is AF_UNIX*; for the Internet
domain AF INET. The socket types are also defined in this file and one of SOCK_STREAM,
SOCK DGRAM, or SOCK RAW must be specified. To create a stream socket in the Internet
domain the following call might be used:
s = socket(AF_INET, SOCK_STREAM, 0);

This call would result in a stream socket being created with the TCP protocol providing the
underlying communication support. To create a datagram socket for on-machine use a sample
call might be:
s = socket(AF_UNIX, SOCK DGRAM, 0);

To obtain a particular protocol one selects the protocol number, as defined within the
communication domain. For the Internet domain the available protocols are defined in
<netinet/in.h> or, better yet, one may use one of the library routines discussed in section 3,
such as getprotobyname:

ttinclude <sys/types.h>
ttinclude <sys/socket.h>
ttinclude <netinet/in.h>
#tinclude <netdb.h>
pp = getprotobyname(”tcp”);
s = socket(AF_INET, SOCK STREAM, pp->p proto);
There are several reasons a socket call may fail. Aside from the rare occurrence of lack

of memory (ENOBUFS), a socket request may fail due to a request for an unknown protocol
(EPROTONOSUPPORT), or a request for a type of socket for which there is no supporting
protocol (EPROTOTYPE).
2.3.

Binding names

A socket is created without a name.

Until a name is bound to a socket, processes have

no way to reference it and, consequently, no messages may be received on it. The bind call is
used to assign a name to a socket:
bind(s, name, namelen);

The bound name is a variable length byte string which is interpreted by the supporting

protocol(s). Its interpretation may vary from communication domain to communication
domain (this is one of the properties which comprise the “domain”). In the UNIX domain
names are path names while in the Internet domain names contain an Internet address and

port number. If one wanted to bind the name “/dev/foo” to a UNIX domain socket, the following would be used:
* The manifest constants are named AF whatever as they indicate the “address format” to use in interpreting names.

3-8 Interprocess Communication Primer

bind(s, ”/dev/foo”, sizeof (”/dev/foo”) — 1);
(Note how the null byte in the name is not counted as part of the name.) In binding an Internet address things become more complicated. The actual call is simple,

#tinclude <sys/types.h>
ttinclude <netinet/in.h>
struct sockaddr_in sin;
bind(s, &sin, sizeof (sin));
but the selection of what to place in the address sin requires some discussion.

We will come

back to the problem of formulating Internet addresses in section 3 when the library routines
used in name resolution are discussed.
2.4. Connection establishment

With a bound socket it is possible to rendezvous with an unrelated process. This operation is usually asymmetric with one process a “client” and the other a “server”.

The client

requests services from the server by initiating a ‘“connection” to the server’s socket.

server, when willing to offer its advertised services, passively “listens” on its socket.
client side the connect call is used to initiate a connection.

The

On the

Using the UNIX domain, this

might appear as,

connect(s, "server-name”, sizeof ("server-name”));
while in the Internet domain,

struct sockaddr_in server;
connect(s, &server, sizeof (server));
If the client process’s socket is unbound at the time of the connect call, the system will

automatically select and bind a name to the socket; c.f. section 5.4. An error is returned when
the connection was unsuccessful (any name automatically bound by the system, however,
remains).

Otherwise, the socket is associated with the server and data transfer may begin.

Many errors can be returned when a connection attempt fails.

The most common are:

ETIMEDOUT

After failing to establish a connection for a period of time, the system decided there was
no point in retrying the connection attempt any more.

This usually occurs because the

destination host is down, or because problems in the network resulted in transmissions
being lost.
ECONNREFUSED
The host refused service for some reason.

When connecting to a host running 4.2BSD

this is usually due to a server process not being present at the requested name.
ENETDOWN or EHOSTDOWN
These operational errors are returned based on status information delivered to the client

host by the underlying communication services.
ENETUNREACH or EHOSTUNREACH
These operational errors can occur either because the network or host is unknown (no

route to the network or host is present), or because of status information returned by
intermediate gateways or switching nodes.

Many times the status returned is not

sufficient to distinguish a network being down from a host being down.

In these cases

the system is conservative and indicates the entire network is unreachable.
For the server to receive a client’s connection it must perform two steps after binding its

socket. The first is to indicate a willingness to listen for incoming connection requests:

Interprocess Communication Primer 3-9
listen(s, 5);

The second parameter to the listen call specifies the maximum number of outstanding connections which may be queued awaiting acceptance by the server process. Should a connection be
requested while the queue is full, the connection will not be refused, but rather the individual
messages which comprise the request will be ignored. This gives a harried server time to
make room in its pending connection queue while the client retries the connection request.
Had the connection been returned with the ECONNREFUSED error, the client would be
unable to tell if the server was up or not. As it is now it is still possible to get the
ETIMEDOUT error back, though this is unlikely. The backlog figure supplied with the listen
call is limited by the system to a maximum of 5 pending connections on any one queue. This
avoids the problem of processes hogging system resources by setting an infinite backlog, then
ignoring all connection requests.

With a socket marked as listening, a server may accept a connection:
fromlen = sizeof (from);

snew = accept(s, &from, &fromlen);

A new descriptor is returned on receipt of a connection (along with a new socket). If the
server wishes to find out who its client is, it may supply a buffer for the client socket’s name.
The value-result parameter fromlen is initialized by the server to indicate how much space is
associated with from, then modified on return to reflect the true size of the name. If the

client’s name is not of interest, the second parameter may be zero.
Accept normally blocks. That is, the call to accept will not return until a connection is

available or the system call is interrupted by a signal to the process. Further, there is no way
for a process to indicate it will accept connections from only a specific individual, or individuals. It is up to the user process to consider who the connection is from and close down the

connection if it does not wish to speak to the process. If the server process wants to accept

connections on more than one socket, or not block on the accept call there are alternatives;

they will be considered in section 5.

2.5. Data transfer

data may begin to flow. To send and receive data there
With a connection established,
are a number of possible calls. With the peer entity at each end of a connection anchored, a
user can send or receive a message without specifying the peer. As one might expect, in this
case, then the normal read and write system calls are useable,
write(s, buf, sizeof (buf));
read(s, buf, sizeof (buf));

In addition to read and write, the new calls send and recv may be used:
send (s, buf, sizeof (buf), flags);
recv(s, buf, sizeof (buf), flags);

While send and recv are virtually identical to read and write, the extra flags argument is
important. The flags may be specified as a non-zero value if one or more of the following is
required:

SOF OOB
SOF PREVIEW
SOF DONTROUTE

send/receive out of band data
look at data without reading
send data without routing packets

Out of band data is a notion specific to stream sockets, and one which we will not immediately
consider. The option to have data sent without routing applied to the outgoing packets is
currently used only by the routing table management process, and is unlikely to be of interest
to the casual user. The ability to preview data is, however, of interest. When

3-10 Interprocess Communication Primer

SOE_PREVIEW is specified with a recv call, any data present is returned to the user, but
treated as still “unread”.

That is, the next read or recv call applied to the socket will return

the data previously previewed.
2.6.

Discarding sockets

Once a socket is no longer of interest, it may be discarded by applying a close to the

descriptor,

close(s);
If data is associated with a socket which promises reliable delivery (e.g. a stream socket) when

a close takes place, the system will continue to attempt to transfer the data. However, after a
fairly long period of time, if the data is still undelivered, it will be discarded.

Should a user

have no use for any pending data, it may perform a shutdown on the socket prior to closing it.
This call is of the form:

shutdown(s, how);
where how is 0 if the user is no longer interested in reading data, 1 if no more data will be

sent, or 2 if no data is to be sent or received.

Applying shutdown to a socket causes any data

queued to be immediately discarded.
2.7. Connectionless sockets
To this point we have been concerned mostly with sockets which follow a connection

oriented model.

However, there is also support for connectionless interactions typical of the

datagram facilities found in contemporary packet switched networks.

vides a symmetric interface to data exchange.

A datagram socket pro-

While processes are still likely to be client and

server, there is no requirement for connection establishment.

Instead, each message includes

the destination address.
Datagram sockets are created as before, and each should have a name bound to it in

order that the recipient of a message may identify the sender. To send data, the sendto primitive 1s used,

sendto(s, buf, buflen, flags, &to, tolen);
The s, buf, buflen, and flags parameters are used as before. The to and tolen values are used

to indicate the intended recipient of the message.

When using an unreliable datagram inter-

face, it is unlikely any errors will be reported to the sender.

Where information is present

locally to recognize a message which may never be delivered (for instance when a network is
unreachable), the call will return —1 and the global value errno will contain an error number.
To receive messages on an unconnected datagram socket, the recvfrom primitive is pro-

vided:
recvfrom(s, buf, buflen, flags, &from, &fromlen);

Once again, the fromlen parameter is handled in a value-result fashion, initially containing the
size of the from buffer.
In addition to the two calls mentioned above, datagram sockets may also use the connect

call to associate a socket with a specific address. In this case, any data sent on the socket will
automatically be addressed to the connected peer, and only data received from that peer will

be delivered to the user.
multi-casting).

Only one connected address is permitted for each socket (i.e. no

Connect requests on datagram sockets return immediately, as this simply

results in the system recording the peer’s address (as compared to a stream socket where a
connect request initiates establishment of an end to end connection).

tant details of datagram sockets are described in section 5.

Other of the less impor-

Interprocess Communication Primer 3-11

2.8. Input/Output multiplexing

One last facility often used in developing applications is the ability to multiplex i/o

requests among multiple sockets and/or files. This is done using the select call:
select(nfds, &readfds, &writefds, &execptfds, &timeout);

Select takes as arguments three bit masks, one for the set of file descriptors for which the
caller wishes to be able to read data on, one for those descriptors to which data is to be written, and one for which exceptional conditions are pending. Bit masks are created by or-ing
bits of the form “1 << fd”. That is, a descriptor fd is selected if a 1 is present in the fd’th bit
of the mask. The parameter nfds specifies the range of file descriptors (i.e. one plus the value
of the largest descriptor) specified in a mask.

A timeout value may be specified if the selection is not to last more than a predetermined period of time. If timeout is set to 0, the selection takes the form of a poll, returning
immediately. If the last parameter is a null pointer, the selection will block indefinitelyTM*.
Select normally returns the number of file descriptors selected. If the select call returns due
to the timeout expiring, then a value of —1 is returned along with the error number EINTR.
Select provides a synchronous multiplexing scheme. Asynchronous notification of output

completion, input availability, and exceptional conditions is possible through use of the SIGIO
and SIGURG signals described in section 5.

* To be more specific, a return takes place only when a descriptor is selectable, or when a signal is received
by the caller, interrupting the system call.

3-12 Interprocess Communication Primer
3. NETWORK LIBRARY ROUTINES
The discussion in section 2 indicated the possible need to locate and construct network

addresses when using the interprocess communication facilities in a distributed environment.
To aid in this task a number of routines have been added to the standard C run-time library.
In this section we will consider the new routines provided to manipulate network addresses.
While the 4.2BSD networking facilities support only the DARPA standard Internet protocols,
these routines have been designed with flexibility in mind. As more communication protocols
become available, we hope the same user interface will be maintained in accessing networkrelated address data bases.

The only difference should be the values returned to the user.

Since these values are normally supplied the system, users should not need to be directly

aware of the communication protocol and/or naming conventions in use.
Locating a service on a remote host requires many levels of mapping before client and

server may communicate. A service is assigned a name which is intended for human consump-

tion; e.g. “the login server on host monet”.

This name, and the name of the peer host, must

then be translated into network addresses which are not necessarily suitable for human consumption. Finally, the address must then used in locating a physical location and route to the
service. The specifics of these three mappings is likely to vary between network architectures.
For instance, it is desirable for a network to not require hosts be named in such a way that

their physical location is known by the client host.

Instead, underlying services in the net-

work may discover the actual location of the host at the time a client host wishes to communi-

cate.

This ability to have hosts named in a location independent manner may induce over-

head in connection establishment, as a discovery process must take place, but allows a host to

be physically mobile without requiring it to notify its clientele of its current location.
Standard routines are provided for: mapping host names to network addresses, network

names to network numbers, protocol names to protocol numbers, and service names to port
numbers and the appropriate protocol to use in communicating with the server process. The
file <netdb.h> must be included when using any of these routines.
3.1. Host names

A host name to address mapping is represented by the hostent structure:

struct

hostent {

char
char

*h_name;
**haliases;

/* official name of host */
/* alias list */

int

h addrtype;

int

h length;

/* host address type */
/* length of address */

char

*h addr;

/* address */

I
The official name of the host and its public aliases are returned, along with a variable length

address and address type. The routine gethostbyname(3N) takes a host name and returns a
hostent structure, while the routine gethostbyaddr(3N) maps host addresses into a hostent
structure. It is possible for a host to have many addresses, all having the same name.
Gethostybyname returns the first matching entry in the data base file /etc/hosts; if this is
unsuitable, the lower level routine gethostent(3N) may be used. For example, to obtain a hostent structure for a host on a particular network the following routine might be used (for simplicity, only Internet addresses are considered):

Interprocess Communication Primer 3-13

ttinclude <sys/types.h>
#tinclude <sys/socket.h>
#tinclude <netinet/in.h>
#tinclude <netdb.h>
struct hostent *

gethostbynameandnet(name, net)
char *name;
int net;

sethostent(0);

while ((hp = gethostent()) != NULL) {

if (hp->h addrtype != AFINET)
continue;

if (strcmp(name, hp->h name)) {
for (cp = hp->h aliases; cp && *cp '= NULL; cp++)
if (strcmp(name, *cp) == 0)
goto found;
continue;

}

found:

if (in netof(*(struct in addr *)hp->h addr)) == net)
break;

endhostent(0);
return (hp);

}
(in netof(3N) is a standard routine which returns the network portion of an Internet address.)
3.2. Network names

As for host names, routines for mapping network names to numbers, and back, are provided. These routines return a netent structure:
/*

* Assumption here is that a network number
* fits in 32 bits -- probably a poor one.
*/
struct

netent {

char
char
int
int

*n name;
**n aliases;
n addrtype;
n net;

/* official name of net */
/* alias list */
/* net address type */
/* network # */

b
The routines getnetbyname(3N), getnetbynumber(3N), and getnetent(3N) are the network
counterparts to the host routines described above.
3.3. Protocol names

For protocols the protoent structure defines the protocol-name mapping used with the
routines getprotobyname(3N), getprotobynumber(3N), and getprotoent(3N):

3-14 Interprocess Communication Primer
struct

protoent {

char

*p name;

/* official protocol name */

int

p_proto;

/* protocol # */

char

**p_aliases;

/* alias list */

%
3.4.

Service names

Information regarding services is a bit more complicated. A service is expected to reside
at a specific “port” and employ a particular communication protocol.

This view is consistent
Further, a ser-

with the Internet domain, but inconsistent with other network architectures.

vice may reside on multiple ports or support multiple protocols.

If either of these occurs, the

higher level library routines will have to be bypassed in favor of homegrown routines similar
in spirit to the “gethostbynameandnet” routine described above.

A service mapping is

described by the servent structure,

struct

servent {
char

*s name;

/* official service name */

int

s port;

/* port # */

char

**g aliases;

*s proto;

/* alias list */

/* protocol to use */

b
The routine getservbyname(3N) maps service names to a servent structure by specifying a service name and, optionally, a qualifying protocol. Thus the call
sp = getservbyname(”telnet”, (char *)0);
returns the service specification for a telnet server using any protocol, while the call

sp = getservbyname(”telnet”, "tcp”);
returns only that telnet server which uses the TCP protocol.

The routines getservbyport(3N)

and getservent(3N) are also provided. The getservbyport routine has an interface similar to
that provided by getservbyname; an optional protocol name may be specified to qualify lookups.

3.5. Miscellaneous

With the support routines described above, an application program should rarely have to

deal directly with addresses. This allows services to be developed as much as possible in a
network independent fashion. It is clear, however, that purging all network dependencies is
very difficult. So long as the user is required to supply network addresses when naming services and sockets there will always some network dependency in a program. For example, the
normal code included in client programs, such as the remote login program, is of the form
shown in Figure 1.

(This example will be considered in more detail in section 4.)

If we wanted to make the remote login program independent of the Internet protocols

and addressing scheme we would be forced to add a layer of routines which masked the network dependent aspects from the mainstream login code. For the current facilities available
in the system this does not appear to be worthwhile. Perhaps when the system is adapted to

different network architectures the utilities will be reorganized more cleanly.
Aside from the address-related data base routines, there are several other routines avail-

able in the run-time library which are of interest to users. These are intended mostly to simplify manipulation of names and addresses. Table 1 summarizes the routines for manipulating
variable length byte strings and handling byte swapping of network addresses and values.
The byte swapping routines are provided because the operating system expects addresses

to be supplied in network order.

On a VAX, or machine with similar architecture, this is

Interprocess Communication Primer 3-15

ttinclude <sys/types.h>
#tinclude <sys/socket.h>

#include <netinet/in.h>
#tinclude <stdio.h>

#include <netdb.h>
main(argc, argv)
char *argv[];

{

struct sockaddr in sin;
struct servent *sp;
struct hostent *hp;
int s;

sp = getservbyname(”login”, "tcp”);
if (sp == NULL) {
fprintf(stderr, rlogin: tcp/login: unknown servicew”);
exit(1);

hp = gethostbyname(argv[1]);
if (hp == NULL) {
fprintf(stderr, "rlogin: %s: unknown hostw”, argv[1]);
exit(2);

bzero((char *)&sin, sizeof (sin));
becopy(hp->h addr, (char *)&sin.sin addr, hp->h length);
sin.sin family = hp->h addrtype;
sin.sin port = sp->s port;

s = socket(AF INET, SOCK_STREAM, 0);
if (s <0) {
perror("rlogin: socket”);
exit(3);

j
if (connect(s, (char *)&sin, sizeof (sin)) < 0) {
AW
0
31
»
perror("rlogin:
connect”);

exit(5);

Figure 1. Remote login client code.

Call

Synopsis

bemp(sl, s2, n) | compare byte-strings; 0 if same, not 0 otherwise
becopy(sl, s2, n) | copy n bytes from sl to s2
bzero(base, n)
zero-fill n bytes starting at base
htonl(val)
convert 32-bit quantity from host to network byte order
htons(val)
convert 16-bit quantity from host to network byte order
ntohl(val)
convert 32-bit quantity from network to host byte order
ntohs(val)
convert 16-bit quantity from network to host byte order |
Table 1. C run-time routines.

3-16 Interprocess Communication Primer

usually reversed. Consequently, programs are sometimes required to byte swap quantities.
The library routines which return network addresses provide them in network order so that
they may simply be copied into the structures provided to the system. This implies users
should encounter the byte swapping problem only when interpreting network addresses. For
example, if an Internet port is to be printed out the following code would be required:
printf("port number %dwn”, ntohs(sp->s port));

On machines other than the VAX these routines are defined as null macros.

Interprocess Communication Primer 3-17

4. CLIENT/SERVER MODEL

The most commonly used paradigm in constructing distributed applications 1is the

client/server model. In this scheme client applications request services from a server process.

This implies an asymmetry in establishing communication between the client and server
which has been examined in section 2. In this section we will look more closely at the interactions between client and server, and consider some of the problems in developing client and
|

server applications.

Client and server require a well known set of conventions before service may be rendered
(and accepted). This set of conventions comprises a protocol which must be implemented at
both ends of a connection. Depending on the situation, the protocol may be symmetric or
asymmetric. In a symmetric protocol, either side may play the master or slave roles. In an
asymmetric protocol, one side is immutably recognized as the master, with the other the slave.
An example of a symmetric protocol is the TELNET protocol used in the Internet for remote
terminal emulation. An example of an asymmetric protocol is the Internet file transfer protocol, FTP. No matter whether the specific protocol used in obtaining a service is symmetric or
asymmetric, when accessing a service there is a “client process” and a “server process”. We
will first consider the properties of server processes, then client processes.

A server process normally listens at a well know address for service requests. Alternative
schemes which use a service server may be used to eliminate a flock of server processes clogging the system while remaining dormant most of the time. The Xerox Courier protocol uses
the latter scheme. When using Courier, a Courier client process contacts a Courier server at
the remote host and identifies the service it requires. The Courier server process then creates
the appropriate server process based on a data base and “splices” the client and server
together, voiding its part in the transaction. This scheme is attractive in that the Courier

server process may provide a single contact point for all services, as well as carrying out the
initial steps in authentication. However, while this is an attractive possibility for standardizing access to services, it does introduce a certain amount of overhead due to the intermediate

process involved. Implementations which provide this type of service within the system can

minimize the cost of client server rendezvous. The portal notion described in the “4.2BSD
System Manual” embodies many of the ideas found in Courter, with the rendezvous mechanism implemented internal to the system.
4.1. Servers

In 4.9BSD most servers are accessed at well known Internet addresses or UNIX domain
names. When a server is started at boot time it advertises it services by listening at a well
know location. For example, the remote login server’s main loop is of the form shown in Figure 2.

The first step taken by the server is look up its service definition:
sp = getservbyname(”login”, "tcp”);
if (sp == NULL) {

fprintf(stderr, "rlogind: tcp/login: unknown servicew”);
exit(1);

This definition is used in later portions of the code to define the Internet port at which 1t
listens for service requests (indicated by a connection).

Step two is to disassociate the server from the controlling terminal of its invoker. This

is important as the server will likely not want to receive signals delivered to the process group
of the controlling terminal.

Once a server has established a pristine environment, it creates a socket and begins
accepting service requests. The bind call is required to insure the server listens at its

3-18 Interprocess Communication Primer

main(argce, argv)
int argc;

char **argv;
int f;

struct sockaddr in from;
struct servent *sp;

sp = getservbyname("login”, tcp”);
if (sp == NULL) {
fprintf(stderr, "rlogind: tcp/login: unknown service\n”);
exit(1);

}

#ifndef DEBUG

<<disassociate server from controlling terminal>>

Hendif
sin.sin_port = sp->s port;
f = socket(AF_INET, SOCK_STREAM, 0);
if (bind(f, (caddr_t)&sin, sizeof (sin)) < 0) {

}
listen(f, 5);

for (;;) {
int g, len = sizeof (from);
g = accept(f, &from, &len);

if (g < 0) {

if (errno '= EINTR)
perror(rlogind: accept”);

continue;

}

if (fork() == 0) {

close(f);
doit(g, &from);

close(g);

}
Figure 2.

Remote login server.

expected location. The main body of the loop is fairly simple:

Interprocess Communication Primer 3-19

for (;;) {

int g, len = sizeof (from);
g = accept(f, &from, &len);
if (g <0) {
if (errno != EINTR)
perror("rlogind: accept”);
continue;

}

if (fork() == 0) {
close(f);
doit(g, &from);

close(g);

}
An accept call blocks the server until a client requests service. This call could return a failure
status if the call is interrupted by a signal such as SIGCHLD (to be discussed in section 5).
Therefore, the return value from accept is checked to insure a connection has actually been
established. With a connection in hand, the server then forks a child process and invokes the
main body of the remote login protocol processing. Note how the socket used by the parent
for queueing connection requests is closed in the child, while the socket created as a result of
the accept is closed in the parent. The address of the client is also handed the doit routine
because it requires it in authenticating clients.
4.2. Clients

The client side of the remote login service was shown earlier in Figure 1. One can see
the separate, asymmetric roles of the client and server clearly in the code. The server is a

passive entity, listening for client connections, while the client process is an active entity, ini-

tiating a connection when invoked.

Let us consider more closely the steps taken by the client remote login process. As in
the server process the first step is to locate the service definition for a remote login:
sp = getservbyname(”login”, "tcp”);
if (sp == NULL) {
fprintf(stderr, "rlogin: tcp/login: unknown servicen”);
exit(1);

J
Next the destination host is looked up with a gethostbyname call:
hp = gethostbyname(argv([1]);
if (hp == NULL) {

fprintf(stderr, rlogin: %s: unknown host\”, argv([1]);
exit(2);

With this accomplished, all that is required is to establish a connection to the server at the
requested host and start up the remote login protocol. The address buffer is cleared, then
filled in with the Internet address of the foreign host and the port number at which the login
process resides:

bzero((char *)&sin, sizeof (sin));

beopy(hp->h_addr, (char *)sin.sin_addr, hp->h length);
sin.sinfamily = hp->h _addrtype;
sin.sin port = sp->sport,

3-20 Interprocess Communication Primer
A socket is created, and a connection initiated.

s = socket(hp->h_addrtype, SOCK_STREAM, 0);
if (s < 0) {
perror(”rlogin: socket”);

exit(3);

j
“if (connect(s, (char *)&sin, sizeof (sin)) < 0) {
perror(“rlogin: connect”);

exit(4);

)
The details of the remote login protocol will not be considered here.
4.3.

Connectionless servers

While connection-based services are the norm, some services are based on the use of

datagram sockets. One, in particular, is the “rwho” service which provides users with status
information for hosts connected to a local area network.

This service, while predicated on the

ability to broadcast information to all hosts connected to a particular network, is of interest as
an example usage of datagram sockets.
A user on any machine running the rwho server may find out the current status of a
machine with the ruptime(1) program.

The output generated is illustrated in Figure 3.

arpa

9:45,

5 users, load

1.15,

1.39,

1.31

cad

2+12:04,

8 users, load

4.67,

5.13,

4.59

calder

10:10,

0 users, load

0.27,

0.15,

0.14

dali

2+406:28,

9 users, load

1.04,

1.20,

1.65

degas

254+09:48,

O users, load

1.49,

143,

1.41

ear

5+00:05,

O users, load

1.51,

1.54,

1.56

ernie

down

0:24

esvax

down

17:04

ingres

down

0:26

kim

3+09:16,

8 users, load

2.03,

2.46,

3.11

matisse

3+06:18,

0 users, load

0.03,

0.05

medea

~up

3+09:39,

2 users, load

0.35,

0.37,

0.50

merlin

down

19+15:37

miro

1407:20,

7 users, load

4.59,

3.28,

2.12

monet

1+400:43,

2 users, load

0.22,

0.09,

0.07

down

16:09

statvax

2+15:57,

3 users, load

1.52,

1.81,

1.86

ucbvax

9:34,

2 users, load

6.08,

5.16,

3.28

Figure 3. ruptime output.

Status information for each host is periodically broadcast by rwho server processes on
each machine.

The same server process also receives the status information and uses it to

update a database.
each host.

This database is then interpreted to generate the status information for

Servers operate autonomously, coupled only by the local network and its broadcast

capabilities.
The rwho server, in a simplified form, is pictured in Figure 4.

tasks performed by the server.

broadcast by other hosts on the network.
gram.

There are two separate

The first task is to act as a receiver of status information
This job is carried out in the main loop of the pro-

Packets received at the rwho port are interrogated to insure they’ve been sent by

another rwho server process, then are time stamped with their arrival time and used to update

Interprocess Communication Primer 3-21

a file indicating the status of the host. When a host has not been heard from for an extended
period of time, the database interpretation routines assume the host is down and indicate such
on the status reports. This algorithm is prone to error as a server may be down while a host is
actually up, but serves our current needs.
main()

{
sp = getservbyname("who”, "udp”);

net = getnetbyname(”localnet”);
sin.sin_addr = inet_makeaddr(INADDR
_ANY, net);
sin.sin_port = sp->s port;

s = socket(AF_INET, SOCK_DGRAM, 0);
bind(s, &sin, sizeof (sin));

sigset(SIGALRM, onalrm);
onalrm();

for (;;) {

struct whod wd,;

int cc, whod, len = sizeof (from);

cc = recvfrom(s, (char *)&wd, sizeof (struct whod), 0, &from, &len);
if (cc <= 0) {

if (cc < 0 && errno != EINTR)
perror("rwhod: recv”);
continue;

if (from.sin_port != sp->s port) {
fprintf(stderr, rwhod: %d: bad from portw”,
ntohs(from.sin port));
continue;

J
if ('verify(wd.wd hostname)) {
fprintf(stderr, rwhod: malformed host name from %x3%”,
ntohl(from.sin addr.s addr));
continue;

}

(void) sprintf(path, ”%s/whod.
%s”, RWHODIR, wd.wd hostname);

whod = open(path, FWRONLYIFCREATEIFTRUNCATE, 0666);
(void) time(&wd.wd recvtime);

(void) write(whod, (char *)&wd, cc);
(void) close(whod);

Figure 4. rwho server.

The second task performed by the server is to supply information regarding the status of
its host. This involves periodically acquiring system status information, packaging it up in a
message and broadcasting it on the local network for other rwho servers to hear. The supply
function is triggered by a timer and runs off a signal. Locating the system status information
is somewhat involved, but uninteresting. Deciding where to transmit the resultant packet

3-22 Interprocess Communication Primer
does, however, indicates some problems with the current protocol.

Status information is broadcast on the local network. For networks which do not support the notion of broadcast another scheme must be used to simulate or replace broadcasting.
One possibility is to enumerate the known neighbors (based on the status received). This,
unfortunately, requires some bootstrapping information, as a server started up on a quiet network will have no known neighbors and thus never receive, or send, any status information.
This is the identical problem faced by the routing table management process in propagating
routing status information. The standard solution, unsatisfactory as it may be, is to inform
one or more servers of known neighbors and request that they always communicate with these
neighbors. If each server has at least one neighbor supplied it, status information may then
propagate through a neighbor to hosts which are not (possibly) directly neighbors. If the
server is able to support networks which provide a broadcast capability, as well as those which
do not, then networks with an arbitrary topology may share status information*.
The second problem with the current scheme is that the rwho process services only a single local network, and this network is found by reading a file.

It is important that software
operating in a distributed environment not have any site-dependent information compiled into
it. This would require a separate copy of the server at each host and make maintenance a
severe headache. 4.2BSD attempts to isolate host-specific information from applications by

providing system calls which return the necessary informationt. Unfortunately, no straight-

forward mechanism currently exists for finding the collection of networks to which a host is
directly connected. Thus the rwho server performs a lookup in a file to find its local network.
A better, though still unsatisfactory, scheme used by the routing process is to interrogate the
system data structures to locate those directly connected networks. A mechanism to acquire
this information from the system would be a useful addition.

* One must, however, be concerned about “loops”. That is, if a host is connected to multiple networks, it
will receive status information from itself. This can lead to an endless, wasteful, exchange of information.
T An example of such a system call is the gethostname(2) call which returns the host’s “official” name.

Interprocess Communication Primer 3-23

5. ADVANCED TOPICS

A number of facilities have yet to be discussed. For most users of the ipc the mechanisms already described will suffice in constructing distributed applications. However, others
will find need to utilize some of the features which we consider in this section.
5.1. Out of band data

The stream socket abstraction includes the notion of “out of band” data. Out of band
data is a logically independent transmission channel associated with each pair of connected
stream sockets. Out of band data is delivered to the user independently of normal data along
with the SIGURG signal. In addition to the information passed, a logical mark is placed in
the data stream to indicate the point at which the out of band data was sent. The remote
login and remote shell applications use this facility to propagate signals from between client
and server processes. When a signal is expected to flush any pending output from the remote
process(es), all data up to the mark in the data stream is discarded.
The stream abstraction defines that the out of band data facilities must support the reliable delivery of at least one out of band message at a time. This message may contain at least
one byte of data, and at least one message may be pending delivery to the user at any one
time. For communications protocols which support only in-band signaling (i.e. the urgent
data is delivered in sequence with the normal data) the system extracts the data from the normal data stream and stores it separately. This allows users to choose between receiving the
urgent data in order and receiving it out of sequence without having to buffer all the intervening data.

To send an out of band message the SOE_OOB flag is supplied to a send or sendto calls,
while to receive out of band data SOF OOB should be indicated when performing a recvfrom
or recv call. To find out if the read pointer is currently pointing at the mark in the data
stream, the SIOCATMARK ioctl is provided:
ioctl(s, SIOCATMARK, &yes);

If yes is a 1 on return, the next read will return data after the mark. Otherwise (assuming out
of band data has arrived), the next read will provide data sent by the client prior to transmission of the out of band signal. The routine used in the remote login process to flush output
on receipt of an interrupt or quit signal is shown in Figure 5.
5.2. Signals and process groups

Due to the existence of the SIGURG and SIGIO signals each socket has an associated
process group (just as is done for terminals). This process group is initialized to the process
group of its creator, but may be redefined at a later time with the SIOCSPGRP ioctl:

3-24 Interprocess Communication Primer
oob()

{

int out = 1+1;

char waste[BUFSIZ], mark;

signal(SIGURG, oob);

/* flush local terminal input and output */
ioctl(1, TIOCFLUSH, (char *)&out);

for (;;) {

if (ioctl(rem, SIOCATMARK, &mark) < 0) {
perror(”ioctl”);
break;

}

if (mark)

break;
(void) read(rem, waste, sizeof (waste));

}

recv(rem, &mark, 1, SOF_OOB);

Figure 5. Flushing terminal i/o on receipt of out of band data.
ioctl(s, SIOCSPGRP, &pgrp);

A similar ioctl, SIOCGPGRP, is available for determining the current process group of a
socket.

5.3. Pseudo terminals

Many programs will not function properly without a terminal for standard input and
output. Since a socket is not a terminal, it is often necessary to have a process communicating
over the network do so through a pseudo terminal. A pseudo terminal is actually a pair of
devices, master and slave, which allow a process to serve as an active agent in communication
between processes and users. Data written on the slave side of a pseudo terminal is supplied
as input to a process reading from the master side. Data written on the master side is given
the slave as input. In this way, the process manipulating the master side of the pseudo terminal has control over the information read and written on the slave side. The remote login
server uses pseudo terminals for remote login sessions. A user logging in to a machine across
the network is provided a shell with a slave pseudo terminal as standard input, output, and
error. The server process then handles the communication between the programs invoked by
the remote shell and the user’s local client process. When a user sends an interrupt or quit
signal to a process executing on a remote machine, the client login program traps the signal,
sends an out of band message to the server process who then uses the signal number, sent as
the data value in the out of band message, to perform a killpg(2) on the appropriate process
group.

5.4. Internet address binding

Binding addresses to sockets in the Internet domain can be fairly complex. Communicating processes are bound by an association. An association is composed of local and foreign
addresses, and local and foreign ports. Port numbers are allocated out of separate spaces, one
for each Internet protocol. Associations are always unique. That is, there may never be
duplicate <protocol, local address, local port, foreign address, foreign port> tuples.
The bind system call allows a process to specify half of an association, <local address,
local port>, while the connect and accept primitives are used to complete a socket’s

Interprocess Communication Primer 3-25

association. Since the association is created in two steps the association uniqueness requirement indicated above could be violated unless care is taken. Further, it is unrealistic to
expect user programs to always know proper values to use for the local address and local port
since a host may reside on multiple networks and the set of allocated port numbers is not

directly accessible to a user.

To simplify local address binding the notion of a “wildcard” address has been provided.

When an address is specified as INADDRANY (a manifest constant defined in
<netinet/in.h>), the system interprets the address as “any valid address”. For example, to
bind a specific port number to a socket, but leave the local address unspecified, the following
code might be used:
ttinclude <sys/types.h>
ttinclude <netinet/in.h>
struct sockaddr in sin;

s = socket(AF_INET, SOCK_STREAM, 0);
sin.sin family = AF INET;
sin.sin addr.s addr = INADDR_ANY;
sin.sin port = MYPORT;
bind(s, (char *)&sin, sizeof (sin));

Sockets with wildcarded local addresses may receive messages directed to the specified port
number, and addressed to any of the possible addresses assigned a host. For example, if a
host is on a networks 46 and 10 and a socket is bound as above, then an accept call is performed, the process will be able to accept connection requests which arrive either from network 46 or network 10.

In a similar fashion, a local port may be left unspecified (specified as zero), in which case
the system will select an appropriate port number for it. For example:
sin.sin addr.s addr = MYADDRESS;
sin.sin port = 0;

bind(s, (char *)&sin, sizeof (sin));

The system selects the port number based on two criteria. The first is that ports numbered O
through 1023 are reserved for privileged users (i.e. the super user). The second is that the
port number is not currently bound to some other socket. In order to find a free port number
in the privileged range the following code is used by the remote shell server:

3-26 Interprocess Communication Primer
struct sockaddr_in sin;
lport = IPPORT_RESERVED - 1;

sin.sin addr.s addr = INADDR_ANY;

for (;;) {

sin.sin port = htons((u short)lport);
if (bind(s, (caddr t)&sin, sizeof (sin)) >= 0)
break;
if (errno != EADDRINUSE && errno != EADDRNOTAVAIL) {
perror(”socket”);
break;

}

lport--;

if (Iport == IPPORT RESERVED/2) {
fprintf(stderr, "socket: All ports in use\”);

break;

)
The restriction on allocating ports was done to allow processes executing in a ‘“secure’

environment to perform authentication based on the originating address and port number.
In certain cases the algorithm used by the system in selecting port numbers is unsuitable
for an application.

This is due to associations being created in a two step process.

For exam-

ple, the Internet file transfer protocol, FTP, specifies that data connections must always originate from the same local port.

different foreign ports.

However, duplicate associations are avoided by connecting to

In this situation the system would disallow binding the same local

address and port number to a socket if a previous data connection’s socket were around.

override the default port selection algorithm then an option call must be performed prior to
address binding:

setsockopt(s, SOL_SOCKET,

SQ REUSEADDR, (char *)0, 0);

bind(s, (char *)&sin, sizeof (sin));

With the above call, local addresses may be bound which are already in use.

This does not

violate the uniqueness requirement as the system still checks at connect time to be sure any

other sockets with the same local address and port do not have the same foreign address and
port (if an association already exists, the error EADDRINUSE is returned).
Local address binding by the system is currently done somewhat haphazardly when a
host is on multiple networks. Logically, one would expect the system to bind the local address

associated with the network through which a peer was communicating.

For instance, if the

local host is connected to networks 46 and 10 and the foreign host is on network 32, and

traffic from network 32 were arriving via network 10, the local address to be bound would be
the host’s address on network 10, not network 46.

This unfortunately, is not always the case.

For reasons too complicated to discuss here, the local address bound may be appear to be

chosen at random.

This property of local address binding will normally be invisible to users

unless the foreign host does not understand how to reach the address selected*.

* For example, if network 46 were unknown to the host on network 32, and the local address were bound to
that located on network 46, then even though a route between the two hosts existed through network 10, a
connection would fail.

Interprocess Communication Primer 3-27

5.5. Broadcasting and datagram sockets
By using a datagram socket it is possible to send broadcast packets on many networks

supported by the system (the network itself must support the notion of broadcasting; the system provides no broadcast simulation in software).

Broadcast messages can place a high load

on a network since they force every host on the network to service them.

Consequently, the

ability to send broadcast packets has been limited to the super user.
To send a broadcast message, an Internet datagram socket should be created:

s = socket(AF_INET, SOCK_DGRAM, 0);

and at least a port number should be bound to the socket:
sin.sin_family = AFINET;

sin.sin_addr.s addr = INADDR_ANY;
sin.sin_port = MYPORT,;
bind(s, (char *)&sin, sizeof (sin));

Then the message should be addressed as:
dst.sinfai“uly = Ar INET
dst.sin_addr.s addr= INADDR ANY;
dst.sin_port = DESTPORT;

and, finally, a sendto call may be used:
sendto(s, buf, buflen, 0, &dst, sizeof (dst));
Received broadcast messages contain the senders address and port (datagram sockets are

anchored before a message is allowed to go out).
5.6. Signals
Two new signals have been added to the system which may be usedin conjunction with

the

1nterprocess

communication

facilities.

The

SIGURG

signal

associated

with

the

existence of an ‘““urgent condition”. The SIGIO signal is used with “interrupt driven i/o” (not
presently implemented). SIGURG is currently supplied a process when out of band data is
present at a socket.

If multiple sockets have out of band data awaiting delivery, a select call

may be used to determine those sockets with such data.
An old signal which is useful when constructing server processes is SIGCHLD. This sig-

nal is delivered to a process when any children processes have changed state.
servers use the signal to “reap” child processes after exiting.

server loop shown in Figure 2 may be augmented as follows:

Normally

For example, the remote login

3-28 Interprocess Communication Primer
int reaper();

sigset(SIGCHLD, reaper);
listen(f, 10);

for (5;) {

int g, len = sizeof (from);

g = accept(f, &from, &len, 0);

if (g < 0) {

if (errno '= EINTR)
perror(”rlogind: accept”);

continue;

J
#include <wait.h>
reaper()

{

union wait status;

while (wait3(&status, WNOHANG, 0) > 0)
b

}
If the parent server process fails to reap its children, a large number of “zombie”
processes may be created.

Networking Implementation Notes 3-29

4.2BSD Networking Implementation Notes
Revised July, 1983
Samuel J. Leffler, William N. Joy, Robert S. Fabry
Computer Systems Research Group
Computer Science Division

Department of Electrical Engineering and Computer Science
University of California, Berkeley
Berkeley, CA 94720
(415) 642-7780

1. Introduction
This report describes the internal structure of facilities added to the 4.2BSD version of
the UNIX operating system for the VAX. The system facilities provide a uniform user interface to networking within UNIX. In addition, the implementation introduces a structure for
network communications which may be used by system implementors in adding new networking facilities. The internal structure is not visible to the user, rather it is intended to aid
implementors of communication protocols and network services by providing a framework
which promotes code sharing and minimizes implementation effort.

The reader is expected to be familiar with the C programming language and system
interface, as described in the 4.2BSD System Manual [Joy82a]. Basic understanding of network communication concepts is assumed; where required any additional ideas are introduced.
The remainder of this document provides a description of the system internals, avoiding,
when possible, those portions which are utilized only by the interprocess communication facilities.

UNIX is a trademark of Bell Laboratories.

DEC, VAX, DECnet, and UNIBUS are trademarks of Digital Equipment Corporation.

3-30 Networking Implementation Notes

Overview
If we consider the International Standards Organization’s (ISO) Open System Intercon-

nection (OSI) model of network communication [ISO81] [Zimmermann80], the networking
facilities described here correspond to a portion of the session layer (layer 3) and all of the
transport and network layers (layers 2 and 1, respectively).
The network layer provides possibly imperfect data transport services with minimal

addressing structure.

Addressing at this level is normally host to host, with implicit or expli-

cit routing optionally supported by the communicating agents.

At the transport layer the notions of reliable transfer, data sequencing, flow control, and
service addressing are normally included.
nowledgement of data delivered.

of the data.

Reliability is usually managed by explicit ack-

Failure to acknowledge a transfer results in retransmission

Sequencing may be handled by tagging each message handed to the network

layer by a sequence number and maintaining state at the endpoints of communication to utilize received sequence numbers in reordering data which arrives out of order.

The session layer facilities may provide forms of addressing which are mapped into formats required by the transport layer, service authentication and client authentication, etc.

Various systems also provide services such as data encryption and address and protocol translation.

The following sections begin by describing some of the common data structures and utility routines, then examine the internal layering.

are considered.

The contents of each layer and its interface

Certain of the interfaces are protocol implementation specific. For these cases

examples have been drawn from the Internet [Cerf78] protocol family.

Later sections cover

routing issues, the design of the raw socket interface and other miscellaneous topics.

Networking Implementation Notes

3-31

3. Goals
The networking system was designed with the goal of supporting multiple protocol families and addressing styles. This required information to be “hidden” in common data structures which could be manipulated by all the pieces of the system, but which required interpretation only by the protocols which “controlled” it. The system described here attempts to
minimize the use of shared data structures to those kept by a suite of protocols (a protocol
family), and those used for rendezvous between “synchronous” and ‘“‘asynchronous” portions
of the system (e.g. queues of data packets are filled at interrupt time and emptied based on
user requests).

A major goal of the system was to provide a framework within which new protocols and
hardware could be easily be supported. To this end, a great deal of effort has been extended
to create utility routines which hide many of the more complex and/or hardware dependent
chores of networking. Later sections describe the utility routines and the underlying data
structures they manipulate.

3-32 Networking Implementation Notes

4. Internal address representation
Common to all portions of the system are two data structures. These structures are used
to represent addresses and various data objects. Addresses, internally are described by the
sockaddr structure,

struct sockaddr {

short safamily; /* data format identifier */
/* address */
char sa_data[14];

¥
All addresses belong to one or more address families which define their format and interpretation. The safamily field indicates which address family the address belongs to, the sa data
field contains the actual data value. The size of the data field, 14 bytes, was selected based on
a study of current address formats*.

* Later versions of the system support variable length addresses.

Networking Implementation Notes 3-33

5. Memory management
A single mechanism is used for data storage: memory buffers, or mbuf’s. An mbuf is a
structure of the form:

struct mbuf {

struct

u_long

mbuf *m next;

m_off;

short m len;
short m_type;

u_char
struct

m_dat[MLEN];

/* next buffer in chain */

/* offset of data */

/* amount of data in this mbuf */
/* mbuf type (accounting) */

mbuf *m_act;

/* data storage */

/* link in higher-level mbuf list */

¥
The m next field is used to chain mbufs together on linked lists, while the m_act field allows
lists of mbufs to be accumulated. By convention, the mbufs common to a single object (for
example, a packet) are chained together with the mnext field, while groups of objects are
linked via the m_act field (possibly when in a queue).

Each mbuf has a small data area for storing information, mdat. The m_len field indicates the amount of data, while the m_off field is an offset to the beginning of the data from
the base of the mbuf. Thus, for example, the macro mtod, which converts a pointer to an
mbuf to a pointer to the data stored in the mbuf, has the form

#define

mtod(x,t) ((t)((int)(x) + (x)->m off))

(note the t parameter, a C type cast, is used to cast the resultant pointer for proper assignment).

In addition to storing data directly in the mbuf’s data area, data of page size may be also
be stored in a separate area of memory. The mbuf utility routines maintain a pool of pages
for this purpose and manipulate a private page map for such pages. The virtual addresses of
these data pages precede those of mbufs, so when pages of data are separated from an mbuf,
the mbuf data offset is a negative value. An array of reference counts on pages is also maintained so that copies of pages may be made without core to core copying (copies are created
simply by duplicating the relevant page table entries in the data page map and incrementing
the associated reference counts for the pages). Separate data pages are currently used only

when copying data from a user process into the kernel, and when bringing data in at the
hardware level. Routines which manipulate mbufs are not normally aware if data is stored
directly in the mbuf data array, or if it is kept in separate pages.
The following utility routines are available for manipulating mbuf chains:
m = m copy(mO0, off, len);

The m_copy routine create a copy of all, or part, of a list of the mbufs in m0. Len bytes
of data, starting off bytes from the front of the chain, are copied. Where possible, reference counts on pages are used instead of core to core copies. The original mbuf chain
must have at least off + len bytes of data. If len is specified as M COPYALL, all the
data present, offset as before, is copied.
m_cat(m, n);

The mbuf chain, n, is appended to the end of m.

Where possible, compaction is per-

formed.

madj(m, diff);

The mbuf chain, m is adjusted in size by diff bytes. If diff is non-negative, diff bytes are
shaved off the front of the mbuf chain. If diff is negative, the alteration is performed
from back to front. No space is reclaimed in this operation, alterations are accomplished
by changing the m_len and m_off fields of mbufs.
m = m_pullup(mO, size);

After a successful call to m_pullup, the mbuf at the head cof the returned list, m, is

3-34 Networking Implementation Notes
guaranteed to have at least size bytes of data in contiguous memory (allowing access via

a pointer, obtained using the mtod macro). If the original data was less than size bytes
long, len was greater than the size of an mbuf data area (112 bytes), or required
resources were unavailable, m is 0 and the original mbuf chain is deallocated.
This routine is particularly useful when verifying packet header lengths on reception.

For example, if a packet is received and only 8 of the necessary 16 bytes required for a

valid packet header are present at the head of the list of mbufs representing the packet,
the remaining 8 bytes may be “pulled up” with a single mpullup call.

the invalid packet will have been discarded.

If the call fails

By insuring mbufs always reside on 128 byte boundaries it is possible to always locate

the mbuf associated with a data area by masking off the low bits of the virtual address.

This

allows modules to store data structures in mbufs and pass them around without concern for
locating the original mbuf when it comes time to free the structure.

The dtom macro is used

to convert a pointer into an mbuf’s data area to a pointer to the mbuf,

t#define

dtom(x)

((struct mbuf *)((int)x & “(MSIZE-1)))

Mbufs are used for dynamically allocated data structures such as sockets, as well as
memory allocated for packets.

Statistics are maintained on mbuf usage and can be viewed by

users using the netstat(1) program.

Networking Implementation Notes 3-35

6. Internal layering

The internal structure of the network system is divided into three layers. These layers
correspond to the services provided by the socket abstraction, those provided by the communication protocols, and those provided by the hardware interfaces. The communication protocols are normally layered into two or more individual cooperating layers, though they are collectively viewed in the system as one layer providing services supportive of the appropriate
socket abstraction.

The following sections describe the properties of each layer in the system and the interfaces each must conform to.

6.1. Socket layer

The socket layer deals with the interprocess communications facilities provided by the
system. A socket is a bidirectional endpoint of communication which is “typed” by the
semantics of communication it supports. The system calls described in the 4.2BSD System

Manual are used to manipulate sockets.

A socket consists of the following data structure:
struct socket {

short
short
short
short
caddr_t
struct
struct
struct
short
struct
short
short
struct
struct
short
ushort
short
short

so_type;
so_options;
so_linger;
so_state;
sQ_pcb;
protosw *so proto;
socket *so head;
socket *so_qO0;
so_qOlen;
socket *so _q;
so_qlen;
so_qlimit;
sockbuf sosnd;
sockbuf so rev;
so_timeo;
SO_error;
so_oobmark;
SO_pgrp;

/* generic type */
/* from socket call */
/* time to linger while closing */
/* internal state flags */
/* protocol control block */
/* protocol handle */
/* back pointer to accept socket */
/* queue of partial connections */
/* partials on sog0 */
/* queue of incoming connections */
/* number of connections on so q */
/* max number queued connections */
/* send queue */
/* receive queue */
/* connection timeout */
/* error affecting connection */
/* chars to oob mark */
/* pgrp for signals */

¥
Each socket contains two data queues, sq_rcv and sosnd, and a pointer to routines which
provide supporting services. The type of the socket, so_type is defined at socket creation time
and used in selecting those services which are appropriate to support it. The supporting protocol is selected at socket creation time and recorded in the socket data structure for later use.
Protocols are defined by a table of procedures, the protosw structure, which will be described
in detail later. A pointer to a protocol specific data structure, the “protocol control blockTM is
also present in the socket structure. Protocols control this data structure and it normally
includes a back pointer to the parent socket structure(s) to allow easy lookup when returning
information to a user (for example, placing an error number in the so_error field). The other
entries in the socket structure are used in queueing connection requests, validating user
requests, storing socket characteristics (e.g. options supplied at the time a socket is created),
and maintaining a socket’s state.

Processes “rendezvous at a socket” in many instances. For instance, when a process
wishes to extract data from a socket’s receive queue and it is empty, or lacks sufficient data to
satisfy the request, the process blocks, supplying the address of the receive queue as an “wait
channel’ to be used in notification. When data arrives for the process and is placed in the

3-36 Networking Implementation Notes
socket’s queue, the blocked process is identified by the fact it is waiting “on the queue”.
6.1.1. Socket state

A socket’s state is defined from the following:

ttdefine

SS_NOFDREF

ttdefine
#define

SS_ISCONNECTED
SSISCONNECTING

0x001
0x002
0x004

ttdefine
ttdefine

SS_ISDISCONNECTING 0x008
SSCANTSENDMORE
0x010

/* in process of disconnecting */
/* can’t send more data to peer */

#define
t#tdefine
ttdefine

SS CANTRCVMORE
SSCONNAWAITING
SSRCVATMARK

0x020
0x040
0x080

/* can’t receive more data from peer */
/* connections awaiting acceptance */
/* at mark on input */

#define

SS PRIV

0x100

/* privileged */

#define
#define

SS NBIO
SS ASYNC

0x200
0x400

/* non-blocking ops */
/* async i/o notify */

/* no file table ref any more */
/* socket connected to a peer */
/* in process of connecting to peer */

The state of a socket is manipulated both by the protocols and the user (through system
calls).

When a socket is created the state is defined based on the type of input/output the

user wishes to perform.
await resources.

‘“Non-blocking” I/O

implies a process should never be blocked to

Instead, any call which would block returns prematurely with the error

EWOULDBLOCK (the service request may be partially fulfilled, e.g. a request for more data

than is present).

If a process requested “asynchronous’” notification of events related to the socket the
SIGIO signal is posted to the process.

An event is a change in the socket’s state, examples of

such occurances are: space becoming available in the send queue, new data available in the
receive queue, connection establishment or disestablishment, etc.

A socket may be marked “priviledged” if it was created by the super-user.

Only

priviledged sockets may send broadcast packets, or bind addresses in priviledged portions of

an address space.
6.1.2.

Socket data queues

A socket’s data queue contains a pointer to the data stored in the queue and other
entries related
to the management of the data. The following structure defines a data queue:

struct sockbuf {

short
short
short

sh_cc;
sh_hiwat;
sb mbent;

/* actual chars in buffer */
/* max actual char count */
/* chars of mbufs used */

short

sb mbmax;

/* max chars of mbufs to use */

short
short

sb lowat;
sh_timeo;

/* low water mark */
/* timeout */

struct
struct

mbuf *sb mb;
proc *sb sel;

/* the mbuf chain */
/* process selecting read/write */

short

sh flags;

/* flags, see below */

b
Data is stored in a queue as a chain of mbufs.

The actual count of characters as well as

high and low water marks are used by the protocols in controlling the flow of data.

The

socket routines cooperate in implementing the flow control policy by blocking a process when
it requests to send data and the high water mark has been reached, or when it requests to
receive data and less than the low water mark is present (assuming non-blocking I/0 has not

been specified).

Networking Implementation Notes 3-37

When a socket is created, the supporting protocol “reserves” space for the send and
receive queues of the socket. The actual storage associated with a socket queue may fluctuate
during a socket’s lifetime, but is assumed this reservation will always allow a protocol to

acquire enough memory to satisfy the high water marks.
The timeout and select values are manipulated by the socket routines in implementing

various portions of the interprocess communications facilities and will not be described here.
A socket queue has a number of flags used in synchronizing access to the data and in
acquiring resources;

#tdefine
#define
#define
ttdefine
t#tdefine

SBLOCK
SB WANT
SB WAIT
SB SEL
SB COLL

0x01
0x02
0x04
0x08
0x10

/* lock on data queue (so rcv only) */
/* someone is waiting to lock */
/* someone is waiting for data/space */
/* buffer is selected */
/* collision selecting */

The last two flags are manipulated by the system in implementing the select mechanism.
6.1.3. Socket connection queueing

In dealing with connection oriented sockets (e.g. SOCK_ STREAM) the two sides are considered distinct. One side is termed active, and generates connection requests. The other side
is called passive and accepts connection requests.

From the passive side, a socket is created with the option SOACCEPTCONN specified,
creating two queues of sockets: sog0 for connections in progress and sog for connections
already made and awaiting user acceptance. As a protocol is preparing incoming connections,
it creates a socket structure queued on sog0 by calling the routine sonewconn(). When the
g, making it available
connection is established, the socket structure is then transfered to so
for an accept.

If an SO ACCEPTCONN socket is closed with sockets on either sog0 or sog, these

sockets are dropped.

6.2. Protocol layer(s)

Protocols are described by a set of entry points and certain socket visible characteristics,

some of which are used in deciding which socket type(s) they may support.
An entry in the “protocol switch” table exists for each protocol module configured into

the system. It has the following form:

3-38 Networking Implementation Notes

struct protosw {

short

pr type;

short

prfamily;

/* socket type used for */

/* protocol family */

short

prprotocol;

/* protocol number */

short

prflags;

/* socket visible attributes */

int

(*pr input)();

/* input to protocol (from below) */

int

(*proutput)();

/* output to protocol (from above) */

int

(*pr_ctloutput)();

/* control output (from above) */

int

(*pr_usrreq)();

/* user request */

int

(*pr_init)();

/* initialization routine */

int

(*pr_fasttimo)();

/* fast timeout (200ms) */

int

(*prslowtimo)();

/* slow timeout (500ms) */

int

(*pr_drain)();

/* flush any excess space possible */

/* protocol-protocol hooks */

int

(*plf:ctlinput)();

/* control input (from below) */

/* user-protocol hook */

/* utility hooks */

‘A protocol is called through the pr init entry before any other.

Thereafter it is called

every 200 milliseconds through the pr fasttimo entry and every 500 milliseconds through the
prslowtimo for timer based actions.

The system will call the pr_drain entry if it is low on

space and this should throw away any non-critical data.
Protocols pass data between themselves as chains of mbufs using the prinput and

proutput routines.

(towards

the

pr ctloutput.

Prinput passes data up (towards the user) and pr_output passes it down

network);

control

information

passes

and

down

pr_ctlinput

and

The protocol is responsible for the space occupied by any the arguments to

these entries and must dispose of it.
The pr userreq routine interfaces protocols to the socket code and is described below.
The pr flags field is constructed from the following values:

ttdefine

##define

PR ADDR

PR ATOMIC

0x01

0x02

/* addresses given with messages */

/* exchange atomic messages only */

#tdefine
#define
#define

PRCONNREQUIRED
PRWANTRCVD
PRRIGHTS

0x04
0x08
0x10

/* connection required by protocol */
/* want PRU RCVD calls */
/* passes capabilities */

Protocols which are connection-based specify the

PRCONNREQUIRED flag so that the

socket routines will never attempt to send data before a connection has been established.
the

PRWANTRCVD flag is set, the socket routines will notfiy the protocol when the user has

remcved data from the socket’s receive queue.

This allows the protocol to implement ack-

nowledgement on user receipt, and also update windowing information based on the amount

of space available in the receive queue.

The PR ADDR field indicates any data placed in the

socket’s receive queue will be preceded by the address of the sender.

The

PRATOMIC flag

specifies each user request to send data must be performed in a single protocol send request;
it 1s the protocol’s responsibility to maintain record boundaries on data to be sent.
PRRIGHTS flag indicates the protocol supports the passing of capabilities;

The

this is currently

used only the protocols in the UNIX protocol family.
When a socket is created, the socket routines scan the protocol table looking for an
appropriate protocol to support the type of socket being created.

one of the possible socket types (e.g.

The pr type field contains

SOCK_ STREAM), while the prfamily field indicates

which protocol family the protocol belongs to.

The pr
protocol field contains the protocol

number of the protocol, normally a well known value.

Networking Implementation Notes 3-39

6.3. Network-interface layer

Each network-interface configured into a system defines a path through which packets
may be sent and received. Normally a hardware device is associated with this interface,
though there is no requirement for this (for example, all systems have a software “loopback”
interface used for debugging and performance analysis). In addition to manipulating the
hardware device, an interface module is responsible for encapsulation and deencapsulation of
any low level header information required to deliver a message to it’s destination. The selec-

tion of which interface to use in delivering packets is a routing decision carried out at a higher
level than the network-interface layer. Each interface normally identifies itself at boot time to
the routing module so that it may be selected for packet delivery.
An interface is defined by the following structure,

struct ifnet {

char
short
short
int

short

short
int
struct

*if name;
ifunit;
if mtu;
if net;

/* name, e.g. “‘en” or “lo” */
/* sub-unit for lower level driver */
/* maximum transmission unit */
/* network number of interface */

iftimer;
ifhost[2];
sockaddr ifaddr;

/* time ’til ifwatchdog called */
/* local net host number */
/* address of interface */

ifflags;

/* up/down, broadcast, etc. */

union {

struct
struct

sockaddr ifubroadaddr;
sockaddr ifudstaddr;

struct
int
int
int
int
int
int
int
int
int
int

ifqueue if_snd;
(*if_init)();
(*ifoutput)();
(*if ioctl)();
(*if reset)();
(*ifwatchdog)();
ifipackets;
if ierrors;
ifopackets;
if oerrors;
if collisions;

/* output queue */
/* init routine */
/* output routine */
/* ioctl routine */
/* bus reset routine */
/* timer routine */
/* packets received on interface */
/* input errors on interface */
/* packets sent on interface */
/* output errors on interface */
/* collisions on csma interfaces */

struct

ifnet *ifnext;

} if ifuy;

¥
Each interface has a send queue and routines used for initialization, ifinit, and output,

ifoutput. If the interface resides on a system bus, the routine ifreset will be called after a

bus reset has been performed. An interface may also specify a timer routine, ifwatchdog,
which should be called every iftimer seconds (if non-zero).

The state of an interface and certain characteristics are stored in the if lags field. The
following values are possible:

#define
#define
ttdefine
#define
#tdefine
t#tdefine
#define
ttdefine

IFF UP
IFFBROADCAST
IFF DEBUG
IFF ROUTE
IFFPOINTOPOINT
IFF NOTRAILERS
IFF RUNNING
IFF NOARP

0x1
0x2
0x4
0x8
0x10
0x20
0x40
0x80

/* interface is up */
/* broadcast address valid */
/* turn on debugging */
/* routing entry installed */
/* interface is point-to-point link */
/* avoid use of trailers */
/* resources allocated */
/* no address resolution protocol */

If the interface is connected to a network which supports transmission of broadcast packets,

3-40 Networking Implementation Notes

the IFFBROADCAST flag will be set and the ifbroadaddr field will contain the address to be
used in sending or accepting a broadcast packet.
point hardware link (for example,

If the interface is associated with a point to

a DEC DMR-11), the IFFPOINTOPOINT flag will be set

and ifdstaddr will contain the address of the host on the other side of the connection. These
addresses and the local address of the interface, if
addr, are used in filtering incoming packets.
The interface sets IFF RUNNING after it has allocated system resources and posted an initial

read on the device it manages.

This state bit is used to avoid multiple allocation requests

when an interface’s address is changed.

The IFF_NOTRAILERS flag indicates the interface

should refrain from using a trailer encapsulation on outgoing packets; trailer protocols are

described in section 14.

The IFF_NOARP flag indicates the interface should not use an

“address resolution protocol” in mapping internetwork addresses to local network addresses.
The information stored in an ifnet structure for point to point communication devices is

not currently used by the system internally.

Rather, it is used by the user level routing pro-

cess in determining host network connections and in initially devising routes (refer to chapter
10 for more information).
Various statistics are also stored in the interface structure.

These may be viewed by

users using the netstat(1l) program.

The interface address and flags may be set with the SIOCSIFADDR and SIOCSIFFLAGS ioctls.

SIOCSIFADDR is used to initially define each interface’s address; SIOGSIF-

FLAGS can be used to mark an interface down and perform site-specific configuration.
6.3.1. UNIBUS interfaces
All hardware related interfaces currently reside on the UNIBUS.

Consequently a com-

mon set of utility routines for dealing with the UNIBUS has been developed.

Each UNIBUS

interface utilizes a structure of the following form:

struct

ifuba {
short
ifu_uban;
short
ifu_hlen;
struct
uba_regs *ifu_uba;
struct ifrw {
caddr_t
ifrw addr;
int
ifrw bdp;
int
ifrw_info;
int
ifrw_proto;

struct

/* uba number */
/* local net header length */
/* uba regs, in vm */
/* virt addr of header */
/* unibus bdp */
/* value from ubaalloc */
/* map register prototype */

pte *ifrw_mr;/* base of map registers */

}ifur, ifu
w;

struct
short

pte ifuwmap[IF MAXNUBAMR];/* base pages for output */
ifu xswapd;
/* mask of clusters swapped */

ifu flags;

/* used during uballoc’s */

struct

mbuf *ifu xtofree;

/* pages being dma’d out */

short

¥
The ifuba structure describes UNIBUS resources held by an interface.
map registers are held for datagram data, starting at ifrmr.

IF NUBAMR

UNIBUS map register ifr_mr[—1]

maps the local network header ending on a page boundary. UNIBUS data paths are reserved
for read and for write, given by ifrbdp.

The prototype of the map registers for read and for

write 1s saved in ifr proto.

When write transfers are not full pages on page boundaries the data is just copied into
the pages mapped on the UNIBUS and the transfer is started.

If a write transfer is of a (1024

byte) page size and on a page boundary, UNIBUS page table entries are swapped to reference
the pages, and then the initial pages are remapped from ifywmap when the transfer completes.

Networking Implementation Notes 3-41

When read transfers give whole pages of data to be input, page frames are allocated from
a network page list and traded with the pages already containing the data, mapping the allocated pages to replace the input pages for the next UNIBUS data input.
The following utility routines are available for use in writing network interface drivers,
all use the ifuba structure described above.
ifubainit(ifu, uban, hlen, nmr);

ifubainit allocates resources on UNIBUS adaptor uban and stores the resultant information in the ifuba structure pointed to by ifu. Itis called only at boot time or after a
UNIBUS reset. Two data paths (buffered or unbuffered, depending on the ifuflags field)
are allocated, one for reading and one for writing. The nmr parameter indicates the
number of UNIBUS mapping registers required to map a maximal sized packet onto the
UNIBUS, while hlen specifies the size of a local network header, if any, which should be
mapped separately from the data (see the description of trailer protocols in chapter 14).
Sufficient UNIBUS mapping registers and pages of memory are allocated to initialize the
input data path for an initial read. For the output data path, mapping registers and
pages of memory are also allocated and mapped onto the UNIBUS. The pages associ-

ated with the output data path are heldin reserve in the event a write requires copying
non-page-aligned data (see if wubaput below). If ifubainit is called with resources
already allocated, they will be used instead of allocating new ones (this normally occurs

after a UNIBUS reset). A 1 is returned when allocation and initialization is successful, O
otherwise.

m = ifrubaget(ifu, totlen, off0);

ifrubaget pulls read data off an interface. totlen specifies the length of data to be
obtained, not counting the local network header. If off0 is non-zero, it indicates a byte
offset to a trailing local network header which should be copied into a separate mbuf and
prepended to the front of the resultant mbuf chain. When page sized units of data are
present and are page-aligned, the previously mapped data pages are remapped into the
mbufs and swapped with fresh pages; thus avoiding any copying. A 0 return value indicates a failure to allocate resources.

ifwubaput(ifu, m);

ifwubaput maps a chain of mbufs onto a network interface in preparation for output.
The chain includes any local network header, which is copied so that it resides in the
mapped and aligned I/O space. Any other mbufs which contained non-page sized data
portions are also copied to the I/O space. Pages mapped from a previous output operation (no longer needed) are unmapped and returned to the network page pool.

3-42

Networking Implementation Notes

7. Socket/protocol interface
The interface between the socket routines and the communication protocols is through
the pr usrreq routine defined in the protocol switch table.

The following requests to a proto-

col module are possible:

t#tdefine

PRU ATTACH 0

t#tdefine

PRUDETACH 1

/* detach protocol */

#tdefine
t#tdefine

PRU BIND
PRU_LISTEN

/* bind socket to address */
/* listen for connection */

2
3

/* attach protocol */

ttdefine

PRU_CONNECT

#tdefine
#tdefine

PRU ACCEPT 5
/* accept connection from peer */
PRUDISCONNECT 6
/* disconnect from peer */

#tdefine
#tdefine
#tdefine

PRU SHUTDOWN
PRURCVD
8

#tdefine
ttdefine
t#tdefine

PRU ABORT
10
PRU_CONTROL
PRU SENSE
12

/* abort (fast DISCONNECT, DETATCH) */
11
/* control operations on protocol */
/* return status into m */

#tdefine
ttdefine

PRU RCVOOB 13
PRU_SENDOOB

/* retrieve out of band data */
14
/* send out of band data */

PRUSEND

/* establish connection to peer */

7
/* won’t send any more data */
/* have taken data; more room now */
/* send this data */

#define

PRUSOCKADDR

/* fetch socket’s address */

ttdefine

PRU PEERADDR

/* fetch peer’s address */

#define
PRU CONNECT2
17
/* connect two sockets */
/* begin for protocols internal use */

#tdefine

PRUFASTTIMO

/* 200ms timeout */

#define

PRUSLOWTIMO

#tdefine
#define

PRU PROTORCV
PRUPROTOSEND

19
20

/* 500ms timeout */
/* receive from below */

/* send to below */

A call on the user request routine is of the form,
error = (*protosw[].pr usrreq)(up, req, m, addr, rights);
int error; struct socket *up; int req; struct mbuf *m, *rights; caddr_t addr;

The mbuf chain, m, and the address are optional parameters.

The rights parameter is an

optional pointer to an mbuf chain containing user specified capabilities (see the sendmsg and
recomsg system calls).

The protocol is responsible.for disposal of both mbuf chains.

A non-

zero return value gives a UNIX error number which should be passed to higher level software.
The following paragraphs describe each of the requests possible.
PRU ATTACH

When a protocol is bound to a socket (with the socket system call) the protocol module
is called with this request.
resources necessary.

It is the responsibility of the protocol module to allocate any

The “attach” request will always precede any of the other requests,

and should not occur more than once.
PRU DETACH
This is the antithesis of the attach request, and is used at the time a socket is deleted.
The protocol module may deallocate any resources assigned to the socket.
PRU BIND

When a socket is initially created it has no address bound to it. This request indicates
an address should be bound to an existing socket.

The protocol module must verify the

requested address is valid and available for use.

PRU LISTEN
The “listen” request indicates the user wishes to listen for incoming connection requests
on the associated socket.

The protocol module should perform any state changes needed

to carry out this request (if possible).

A “listen” request always precedes a request to

Networking Implementation Notes

3-43

accept a connection.
PRU CONNECT

The “connect” request indicates the user wants to a establish an association.
parameter supplied describes the peer to be connected to.
request may vary depending on the protocol.

The addr

The effect of a connect

Virtual circuit protocols, such as TCP

[Postel80b], use this request to initiate establishment of a TCP connection.

Datagram

protocols, such as UDP [Postel79], simply record the peer’s address in a private data
structure and use it to tag all outgoing packets.

There are no restrictions on how many

times a connect request may be used after an attach.

If a protocol supports the notion

of multi-casting, it is possible to use multiple connects to establish a multi-cast group.
Alternatively, an association may be broken by a

PRU DISCONNECT request, and a

new association created with a subsequent connect request; all without destroying and
creating a new socket.

PRU ACCEPT
Following a successful

PRU LISTEN request and the arrival of one or more connections,

this request is made to indicate the user has accepted the first connection on the queue
of pending connections.

The protocol module should fill in the supplied address buffer

with the address of the connected party.
PRUDISCONNECT

Eliminate an association created with a

PRU CONNECT request.

PRUSHUTDOWN

This call is used to indicate no more data will be sent and/or received (the addr parameter indicates the direction of the shutdown, as encoded in the soshutdown system call).
The protocol may, at its discretion, deallocate any data structures related to the shut-

down.
PRU RCVD

This request is made only if the protocol entry in the protocol switch table includes the
PRWANTRCVD flag.

When a user removes data from the receive queue this request

will be sent to the protocol module. It may be used to trigger acknowledgements, refresh
windowing information, initiate data transfer, etc.
PRU SEND

Each user request to send data is translated into one or more

PRU SEND

requests (a

protocol may indicate a single user send request must be translated into a single
PRU SEND request by specifying the PRATOMIC flag in its protocol description).
The data to be sent is presented to the protocol as a list of mbufs and an address is,

optionally, supplied in the addr parameter.

The protocol is responsible for preserving

the data in the socket’s send queue if it is not able to send it immediately, or if it may
need it at some later time (e.g. for retransmission).
PRU ABORT
This request indicates an abnormal termination of service.

The protocol should delete

any existing association(s).
PRU CONTROL
The “control” request is generated when a user performs a UNIX ioct! system call on a
socket (and the ioctl is not intercepted by the socket routines).

It allows protocol-

specific operations to be provided outside the scope of the common socket interface.
The addr parameter contains a pointer to a static kernel data area where relevant information may be obtained or returned.

The m parameter contains the actual ioct! request

code (note the non-standard calling convention).
PRU SENSE
The “sense” request is generated when the user makes an fstat system call on a socket;
it requests status of the associated socket. There currently is no common format for the

status returned. Information which might be returned includes per-connection statistics,

3-44 Networking Implementation Notes

protocol state, resources currently in use by the connection, the optimal transfer size for
the connection (based on windowing information and maximum packet size). The addr
parameter contains a pointer to a static kernel data area where the status buffer should
be placed.

PRU RCVOOB

Any “out-of-band” data presently available is to be returned. An mbuf is passed in to
the protocol module and the protocol should either place data in the mbuf or attach new
mbufs to the one supplied if there is insufficient space in the single mbuf.

PRU SENDOOB

Like PRU SEND, but for out-of-band data.

PRUSOCKADDR

The local address of the socket is returned, if any is currently bound to the it. The
address format (protocol specific) is returned in the addr parameter.

PRU PEERADDR

The address of the peer to which the socket is connected is returned. The socket must
be in a SS ISCONNECTED state for this request to be made to the protocol. The
address format (protocol specific) is returned in the addr parameter.

PRU CONNECT2

The protocol module is supplied two sockets and requested to establish a connection
between the two without binding any addresses, if possible. This call is used in implementing the system call.

The following requests are used internally by the protocol modules and are never generated by the socket routines. In certain instances, they are handed to the prusrreq routine
solely for convenience in tracing a protocol’s operation (e.g. PRU SLOWTIMO).
PRU FASTTIMO

A “fast timeout” has occured. This request is made when a timeout occurs in the
protocol’s pr fastimo routine. The addr parameter indicates which timer expired.

PRU SLOWTIMO

A “slow timeout” has occured. This request is made when a timeout occurs in the
protocol’s pr_slowtimo routine. The addr parameter indicates which timer expired.

PRU PROTORCV

This request is used in the protocol-protocol interface, not by the routines. It requests
reception of data destined for the protocol and not the user. No protocols currently use
this facility.

PRUPROTOSEND

This request allows a protocol to send data destined for another protocol module, not a
user. The details of how data is marked “addressed to protocol” instead of “addressed
to user” are left to the protocol modules. No protocols currently use this facility.

Networking Implementation Notes 3-45

8. Protocol/protocol interface
The interface between protocol modules is through the prusrreq, pr_input, proutput,
prctlinput, and pr_ctloutput routines. The calling conventions for all but the pr usrreq rou-

tine are expected to be specific to the protocol modules and are not guaranteed to be consistent across protocol families. We will examine the conventions used for some of the Internet protocols in this section as an example.
8.1. pr output

The Internet protocol UDP uses the convention,

error = udpoutput(inp, m);
int error; struct inpcb *inp; struct mbuf *m;

where the inp, “internet protocol control block”, passed between modules conveys per connection state information, and the mbuf chain contains the data to be sent. UDP performs consistency checks, appends its header, calculates a checksum, etc. before passing the packet on
to the IP module:

error = ipoutput(m, opt, ro, allowbroadcast);

int error; struct mbuf *m, *opt; struct route *ro; int allowbroadcast;

The call to IP’s output routine is more complicated than that for UDP, as befits the
additional work the IP module must do. The m parameter is the data to be sent, and the opt
parameter is an optional list of IP options which should be placed in the IP packet header.
The ro parameter is is used in making routing decisions (and passing them back to the caller).
The final parameter, allowbroadcast is a flag indicating if the user is allowed to transmit a
broadcast packet. This may be inconsequential if the underlying hardware does not support
|
the notion of broadcasting.

All output routines return 0 on success and a UNIX error number if a failure occured

which could be immediately detected (no buffer space available, no route to destination, etc.).
8.2. pr_input

Both UDP and TCP use the following calling convention,
(void) (*protosw[].pr_input)(m);
struct mbuf *m;

Each mbuf list passed is a single packet to be processed by the protocol module.
The IP input routine is a VAX software interrupt level routine, and so is not called with
any parameters. It instead communicates with network interfaces through a queue, ipintrq,
which is identical in structure to the queues used by the network interfaces for storing packets
awaiting transmission.

8.3. pr_ctlinput

This routine is used to convey “control” information to a protocol module (i.e. information which might be passed to the user, but is not data). This routine, and the pr_ctloutput
routine, have not been extensively developed, and thus suffer from a “clumsiness” that can
only be improved as more demands are placed on it.

The common calling convention for this routine is,

(void) (*protosw[].pr_ctlinput)(req, info);
int req; caddr_t info;

The req parameter is one of the following,

3-46 Networking Implementation Notes

PRC_IFDOWN
PRCROUTEDEAD
PRC QUENCH
PRC HOSTDEAD
PRCHOSTUNREACH

#define

PRC

t#tdefine
t#tdefine

PRCUNREACH HOST
PRC UNREACH PROTOCOL 10

#define PRC:UNREACH PORT

/* no route to host */
/* dst says bad protocol */

#define

PRC MSGSIZE

/* message size forced drop */

/* net routing redirect */

t#tdefine

=
JIO0
0
©

UNREACH NET

—=O

#define
#tdefine
#tdefine
#define
ttdefine

PRCREDIRECT NET

#tdefine
#tdefine

PRCREDIRECT HOST
PRCTIMXCEED INTRANS

t#tdefine

PRC_TIMXCEED REASS

#define

PRCPARAMPROB

/* interface transition */
/* select new route if possible */
/* some said to slow down */
/* normally from IMP */

/* ditto */

/* no route to network */

/* bad port # */

14
17

/* host routing redirect */
/* packet lifetime expired in transit */

/* lifetime expired on reass q */

/* header incorrect */

while the info parameter is a “catchall” value which is request dependent.

Many of the

requests have obviously been derived from ICMP (the Internet Control Message Protocol),

and from error messages defined in the 1822 host/IMP convention [BBN78].

Mapping tables

exist to convert control requests to UNIX error codes which are delivered to a user.
8.4. pr_ctloutput
This routine is not currently used by any protocol modules.

Networking Implementation Notes

3-47

9. Protocol/network-interface interface
The lowest layer in the set of protocols which comprise a protocol family must interface
itself to one or more network interfaces in order to transmit and receive packets. It is

assumed that any routing decisions have been made before handing a packet to a network
interface, in fact this is absolutely necessary in order to locate any interface at all (unless, of
course, one uses a single ‘“hardwired” interface). There are two cases to be concerned with,
transmission of a packet, and receipt of a packet; each will be considered separately.
9.1. Packet transmission

Assuming a protocol has a handle on an interface, ifp, a (struct ifnet *), it transmits a
fully formatted packet with the following call,

error = (*ifp->ifoutput)(ifp, m, dst)
int error; struct ifnet *ifp; struct mbuf *m; struct sockaddr *dst;

The output routine for the network interface transmits the packet m to the dst address, or
returns an error indication (a UNIX error number). In reality transmission may not be
immediate, or successful; normally the output routine simply queues the packet on its send
queue and primes an interrupt driven routine to actually transmit the packet. For unreliable
mediums, such as the Ethernet, “successful” transmission simply means the packet has been
placed on the cable without a collision. On the other hand, an 1822 interface guarantees
proper delivery or an error indication for each message transmitted. The model employed in
the networking system attaches no promises of delivery to the packets handed to a network
interface, and thus corresponds more closely to the Ethernet. Errors returned by the output
routine are normally trivial in nature (no buffer space, address format not handled, etc.).
9.2. Packet reception

Each protocol family must have one or more “lowest level” protocols. These protocols
deal with internetwork addressing and are responsible for the delivery of incoming packets to
the proper protocol processing modules. In the PUP model [Boggs78] these protocols are
termed Level 1 protocols, in the ISO model, network layer protocols. In our system each such
protocol module has an input packet queue assigned to it. Incoming packets received by a
network interface are queued up for the protocol module and a VAX software interrupt is
posted to initiate processing.

Three macros are available for queueing and dequeueing packets,
IF ENQUEUE(ifq, m)

This places the packet m at the tail of the queue ifg.
IF DEQUEUEK(ifq, m)

This places a pointer to the packet at the head of queue ifg in m. A zero value will be
returned in m if the queue is empty.

IF PREPEND(ifq, m)

This places the packet m at the head of the queue ifq.
Each queue has a maximum length associated with it as a simple form of congestion con-

trol. The macro IFQFULL(ifq) returns 1 if the queue is filled, in which case the macro
IF DROP(ifq) should be used to bump a count of the number of packets dropped and the
offending packet dropped. For example, the following code fragment is commonly found in a
network interface’s input routine,

if IF QFULL(inq)) {

“IFDROP(inq);
m freem(m);

} else
IF ENQUEUE(ing, m);

3-48 Networking Implementation Notes

10. Gateways and routing issues
The system has been designed with the expectation that it will be used in an internetwork environment. The “canonical” environment was envisioned to be a collection of local
area networks connected at one or more points through hosts with multiple network interfaces
(one on each local area network), and possibly a connection to a long haul network (for example, the ARPANET). In such an environment, issues of gatewaying and packet routing
become very important. Certain of these issues, such as congestion control, have been handled
in a simplistic manner or specifically not addressed. Instead, where possible, the network sys-

tem attempts to provide simple mechanisms upon which more involved policies may be imple-

mented. As some of these problems become better understood, the solutions developed will be
incorporated into the system.

This section will describe the facilities provided for packet routing.

The simplistic

mechanisms provided for congestion control are described in chapter 12.
10.1. Routing tables

The network system maintains a set of routing tables for selecting a network interface to
use in delivering a packet to its destination. These tables are of the form:
struct rtentry {

u long
struct
struct
short
short

rt_hash;
/* hash key for lookups */
sockaddr rtdst;
/* destination net or host */
sockaddr rtgateway;/* forwarding agent */
rtflags;
/* see below */
rtrefent;
/* no. of references to structure */

struct

ifnet *rt ifp;

u long

rt_use;

/* packets sent using route */

/* interface to give packet to */

b
The routing information is organized in two separate tables, one for routes to a host and
one for routes to a network. The distinction between hosts and networks is necessary so that
a single mechanism may be used for both broadcast and multi-drop type networks, and also
for networks built from point-to-point links (e.g DECnet [DEC80]).
Each table is organized as a hashed set of linked lists. Two 32-bit hash values are calculated by routines defined for each address family; one based on the destination being a host,

and one assuming the target is the network portion of the address. Each hash value is used to
locate a hash chain to search (by taking the value modulo the hash table size) and the entire
32-bit value is then used as a key in scanning the list of routes. Lookups are applied first to
the routing table for hosts, then to the routing table for networks. If both lookups fail, a final
lookup is made for a “wildcard” route (by convention, network 0). By doing this, routes to a
specific host on a network may be present as well as routes to the network. This also allows a
“fall back” network route to be defined to an “smart” gateway which may then perform more
intelligent routing.

Each routing table entry contains a destination (who’s at the other end of the route), a
gateway to send the packet to, and various flags which indicate the route’s status and type
(host or network). A count of the number of packets sent using the route is kept for use in
deciding between multiple routes to the same destination (see below), and a count of “held
references” to the dynamically allocated structure is maintained to insure memory reclamation
occurs only when the route is not in use. Finally a pointer to the a network interface is kept;
packets sent using the route should be handed to this interface.
Routes are typed in two ways: either as host or network, and as “direct” or “indirect”.
The host/network distinction determines how to compare the rtdst field during lookup. If
the route is to a network, only a packet’s destination network is compared to the r¢dst entry
4
A i
Tf +1
3
stored
in thn
the +alhl
table. If
the route is
to a host, the addresses must m

Networking Implementation Notes 3-49

The distinction between “direct” and “indirect” routes indicates whether the destination
is directly connected to the source. This is needed when performing local network encapsulation. If a packet is destined for a peer at a host or network which is not directly connected to
the source, the internetwork packet header will indicate the address of the eventual destination, while the local network header will indicate the address of the intervening gateway.
Should the destination be directly connected, these addresses are likely to be identical, or a
mapping between the two exists. The RTF_GATEWAY flag indicates the route is to an
“indirect” gateway agent and the local network header should be filled in from the rtgateway
field instead of rt_dst, or from the internetwork destination address.

It is assumed multiple routes to the same destination will not be present unless they are
deemed equal in cost (the current routing policy process never installs multiple routes to the
same destination). However, should multiple routes to the same destination exist, a request
for a route will return the “least used” route based on the total number of packets sent along

this route. This can result in a “ping-pong” effect (alternate packets taking alternate routes),
unless protocols “hold onto” routes until they no longer find them useful; either because the
destination has changed, or because the route is lossy.

Routing redirect control messages are used to dynamically modify existing routing table
entries as well as dynamically create new routing table entries. On hosts where exhaustive
routing information is too expensive to maintain (e.g. work stations), the combination of wildcard routing entries and routing redirect messages can be used to provide a simple routing
management scheme without the use of a higher level policy process. Statistics are kept by the
routing table routines on the use of routing redirect messages and their affect on the routing
tables. These statistics may be viewed using

Status information other than routing redirect control messages may be used in the
future, but at present they are ignored. Likewise, more intelligent “metrics” may be used to
describe routes in the future, possibly based on bandwidth and monetary costs.
10.2. Routing table interface

A protocol accesses the routing tables through three routines, one to allocate a route, one

to free a route, and one to process a routing redirect control message. The routine rtalloc performs route allocation; it is called with a pointer to the following structure,
struct route {
struct
struct

rtentry *ro_rt;

sockaddr ro_dst;

The route returned is assumed “held” by the caller until disposed of with an rtfree call. Protocols which implement virtual circuits, such as TCP, hold onto routes for the duration of the
circuit’s lifetime, while connection-less protocols, such as UDP, currently allocate and free
routes on each transmission.

The routine rtredirect is called to process a routing redirect control message. It is called
with a destination address and the new gateway to that destination. If a non-wildcard route
exists to the destination, the gateway entry in the route is modified to point at the new gateway supplied. Otherwise, a new routing table entry is inserted reflecting the information supplied. Routes to interfaces and routes to gateways which are not directly accesible from the
host are ignored.

10.3. User level routing policies

Routing policies implemented in user processes manipulate the kernel routing tables
through two ioctl calls. The commands SIOCADDRT and SIOCDELRT add and delete routing entries, respectively; the tables are read through the /dev/kmem device. The decision to
place policy decisions in a user process implies routing table updates may lag a bit behind the
identification of new routes, or the failure of existing routes, but this period of instability is

3-50 Networking Implementation Notes
normally very small with proper implementation of the routing process.

Advisory information,

such as ICMP error messages and IMP diagnostic messages, may be read from raw sockets

(described in the next section).
One routing policy process has already been implemented.

The system standard “rout-

ing daemon” uses a variant of the Xerox NS Routing Information Protocol [Xerox82] to maintain up to date routing tables in our local environment.

Interaction with other existing rout-

ing protocols, such as the Internet GGP (Gateway-Gateway Protocol), may be accomplished

using a similar process.

Networking Implementation Notes

3-51

11. Raw sockets
A raw socket is a mechanism which allows users direct access to a lower level protocol.
Raw sockets are intended for knowledgeable processes which wish to take advantage of some
protocol feature not directly accessible through the normal interface, or for the development
of new protocols built atop existing lower level protocols. For example, a new version of TCP
might be developed at the user level by utilizing a raw IP socket for delivery of packets. The
raw IP socket interface attempts to provide an identical interface to the one a protocol would
have if it were resident in the kernel.

The raw socket support is built around a generic raw socket interface, and (possibly)

augmented by protocol-specific processing routines. This section will describe the core of the
raw socket interface.

11.1. Control blocks

Every raw socket has a protocol control block of the following form,
struct rawcb {

struct

rawcb *rcb_next;

struct

rawch *rchprev;

struct
struct
struct
caddrt

socket *rcb socket;
sockaddr rcb faddr;
sockaddr rcb laddr;
rcbpcb;

short

rcbflags;

/* doubly linked list */
/* back pointer to socket */
/* destination address */
/* socket’s address */
/* protocol specific stuff */

¥
All the control blocks are kept on a doubly linked list for performing lookups during packet
dispatch. Associations may be recorded in the control block and used by the output routine
in preparing packets for transmission. The addresses are also used to filter packets on input;
this will be described in more detail shortly. If any protocol specific information is required,
it may be attached to the control block using the rcb_pcb field.
A raw socket interface is datagram oriented.

That is, each send or receive on the socket

requires a destination address. This address may be supplied by the user or stored in the con-

trol block and automatically installed in the outgoing packet by the output routine. Since it is
not possible to determine whether an address is present or not in the control block, two flags,
RAW LADDR and RAW _FADDR, indicate if a local and foreign address are present. Another
flag, RAW_DONTROUTE, indicates if routing should be performed on outgoing packets. If it
is, a route is expected to be allocated for each ‘“new” destination address. That is, the first

time a packet is transmitted a route is determined, and thereafter each time the destination

address stored in rch route differs from rch faddr, or rcb_route.rqrt is zero, the old route is
discarded and a new one allocated.
11.2. Input processing

Input packets are “éssigned” to raw sockets based on a simple pattern matching scheme.
Each network interface or protocol gives packets to the raw input routine with the call:
raw input(m, proto, src, dst)

struct mbuf *m; struct sockproto *proto, struct sockaddr *src, *dst;
The data packet then has a generic header prepended to it of the form
struct raw header {
struct
sockproto raw _proto;
struct
struct

sockaddr raw_dst;
sockaddr rawsrc;

3-52 Networking Implementation Notes

and it is placed in a packet queue for the ‘“raw input protocol” module.

Packets taken from

this queue are copied into any raw sockets that match the header according to the following
rules,

1)
2)

The protocol family of the socket and header agree.
If the protocol number in the socket is non-zero, then it agrees with that found in the
packet header.

If a local address is defined for the socket, the address format of the local address is the
same as the destination address’s and the two addresses agree bit for bit.

The rules of 3) are applied to the socket’s foreign address and the packet’s source

address.
A basic assumption is that addresses present in the control block and packet header (as con-

structed by the network interface and any raw input protocol module) are in a canonical form
which may be “block compared”.
11.3. Output processing
On output the raw prusrreq routine passes the packet and raw control block to the raw
protocol output routine for any processing required before it is delivered to the appropriate
network interface.

socket interface.

The output routine is normally the only code required to implement a raw

Networking Implementation Notes

3-53

12. Buffering and congestion control
One of the major factors in the performance of a protocol is the buffering policy used.
Lack of a proper buffering policy can force packets to be dropped, cause falsified windowing
information to be emitted by protocols, fragment host memory, degrade the overall host performance, etc. Due to problems such as these, most systems allocate a fixed pool of memory
to the networking system and impose a policy optimized for “normal” network operation.
The networking system developed for UNIX is little different in this respect. At boot
time a fixed amount of memory is allocated by the networking system. At later times more

system memory may be requested as the need arises, but at no time is memory ever returned
to the system. It is possible to garbage collect memory from the network, but difficult. In
order to perform this garbage collection properly, some portion of the network will have to be
“turned off”’ as data structures are updated. The interval over which this occurs must kept
small compared to the average inter-packet arrival time, or too much traffic may be lost,
impacting other hosts on the network, as well as increasing load on the interconnecting mediums. In our environment we have not experienced a need for such compaction, and thus have
left the problem unresolved.

The mbuf structure was introduced in chapter 5. In this section a brief description will
be given of the allocation mechanisms, and policies used by the protocols in performing connection level buffering.

12.1. Memory management

The basic memory allocation routines place no restrictions on the amount of space which
may be allocated. Any request made is filled until the system memory allocator starts refusing to allocate additional memory. When the current quota of memory is insufficient to
satisfy an mbuf allocation request, the allocator requests enough new pages from the system to
satisfy the current request only. All memory owned by the network is described by a private
page table used in remapping pages to be logically contiguous as the need arises. In addition,
an array of reference counts parallels the page table and is used when multiple copies of a
page are present.

Mbufs are 128 byte structures, 8 fitting in a 1Kbyte page of memory. When data is
placed in mbufs, if possible, it is copied or remapped into logically contiguous pages of
memory from the network page pool. Data smaller than the size of a page is copied into one
or more 112 byte mbuf data areas.

12.2. Protocol buffering policies

Protocols reserve fixed amounts of buffering for send and receive queues at socket creation time. These amounts define the high and low water marks used by the socket routines in
deciding when to block and unblock a process. The reservation of space does not currently
result in any action by the memory management routines, though it is clear if one imposed an
upper bound on the total amount of physical memory allocated to the network, reserving
memory would become important.

Protocols which provide connection level flow control do this based on the amount of
space in the associated socket queues. That is, send windows are calculated based on the
amount of free space in the socket’s receive queue, while receive windows are adjusted based
on the amount of data awaiting transmission in the send queue. Care has been taken to avoid
the “silly window syndrome” described in [Clark82] at both the sending and receiving ends.

12.3. Queue limiting
Incoming packets from the network are always received unless memory allocation fails.
However, each Level 1 protocol input queue has an upper bound on the queue’s length, and
any packets exceeding that bound are discarded. It is possible for a host to be overwhelmed
by excessive network traffic (for instance a host acting as a gateway from a high bandwidth
b

VIR
I

3-54 Networking Implementation Notes

network to a low bandwidth network). As a “defensive’” mechanism the queue limits may be
adjusted to throttle network traffic load on a host. Consider a host willing to devote some
percentage of its machine to handling network traffic. If the cost of handling an incoming
packet can be calculated so that an acceptable “packet handling rate” can be determined,
then input queue lengths may be dynamically adjusted based on a host’s network load and the
number of packets awaiting processing. Obviously, discarding packets is not a satisfactory

solution to a problem such as this (simply dropping packets is likely to increase the load on a
network); the queue lengths were incorporated mainly as a safeguard mechanism.
12.4. Packet forwarding

When packets can not be forwarded because of memory limitations, the system generates
a “source quench” message. In addition, any other problems encountered during packet forwarding are also reflected back to the senderin the form of ICMP packets. This helps hosts
avoid unneeded retransmissions.
Broadcast packets are never forwarded due to possible dire consequences.

In an early

stage of network development, broadcast packets were forwarded and a “routing loop”
resulted in network saturation and every host on the network crashing.

Networking Implementation Notes

3-55

13. Out of band data
Out of band data is a facility peculiar to the stream socket abstraction defined.
agreement appears to exist as to what its semantics should be.

Little

TCP defines the notion of

“urgent data” as in-line, while the NBS protocols [Burruss81] and numerous others provide a
fully independent logical transmission channel along which out of band data is to be sent.

addition, the amount of the data which may be sent as an out of band message varies from
protocol to protocol; everything from 1 bit to 16 bytes or more.

A stream socket’s notion of out of band data has been defined as the lowest reasonable

common denominator (at least reasonable in our minds); clearly this is subject to debate.

Out

of band data is expected to be transmitted out of the normal sequencing and flow control constraints of the data stream.

A minimum of 1 byte of out of band data and one outstanding

out of band message are expected to be supported by the protocol supporting a stream socket.
It is a protocols perogative to support larger sized messages, or more than one outstanding out

of band message at a time.
Out of band data is maintained by the protocol and usually not stored in the socket’s
send queue.

The

PRUSENDOOB and PRU_RCVOOB requests to the pr_usrreq routine are

used in sending and receiving data.

3-56 Networking Implementation Notes

14. Trailer protocols
Core to core copies can be expensive. Consequently, a great deal of effort was spent in
minimizing such operations. The VAX architecture provides virtual memory hardware organized in page units. To cut down on copy operations, data is kept in page sized units on page-

aligned boundaries whenever possible. This allows data to be moved in memory simply by

remapping the page instead of copying. The mbuf and network interface routines perform
page table manipulations where needed, hiding the complexities of the VAX virtual memory
hardware from higher level code.

Data enters the system in two ways: from the user, or from the network (hardware interface). When data is copied from the user’s address space into the system it is deposited in
pages (if sufficient data is present to fill an entire page). This encourages the user to transmit
information in messages which are a multiple of the system page size.

Unfortunately, performing a similar operation when taking data from the network is very
difficult. Consider the format of an incoming packet. A packet usually contains a local network header followed by one or more headers used by the high level protocols. Finally, the
data, if any, follows these headers. Since the header information may be variable length,
DMA’ing the eventual data for the user into a page aligned area of memory is impossible
without a priori knowledge of the format (e.g. supporting only a single protocol header format).

To allow variable length header information to be present and still ensure page alignment of data, a special local network encapsulation may be used. This encapsulation, termed
a trailer protocol, places the variable length header information after the data. A fixed size
local network header is then prepended to the resultant packet. The local network header contains the size of the data portion, and a new trailer protocol header, inserted before the variable length information, contains the size of the variable length header information. The following trailer protocol header is used to store information regarding the variable length protocol header:

struct {

short protocol;
shortlength;

/* original protocol no. */
/* length of trailer */

¥
The processing of the trailer protocol is very simple. On output, the local network
header indicates a trailer encapsulation is being used. The protocol identifier also includes an
indication of the number of data pages present (before the trailer protocol header). The
trailer protccol header is initialized to contain the actual protocol and variable length header
size, and appended to the data along with the variable length header information.
On input, the interface routines identify the trailer encapsulation by the protocol type

stored in the local network header, then calculate the number of pages of data to find the
beginning of the trailer. The trailing information is copied into a separate mbuf and linked to
the front of the resultant packet.

Clearly, trailer protocols require cooperation between source and destination. In addition, they are normally cost effective only when sizable packets are used. The current scheme
works because the local network encapsulation header is a fixed size, allowing DMA operations
to be performed at a known offset from the first data page being received. Should the local
network header be variable length this scheme fails.

Statistics collected indicate as much as 200Kb/s can be gained by using a trailer protocol
with 1Kbyte packets. The average size of the variable length header was 40 bytes (the size of
a minimal TCP/IP packet header). If hardware supports larger sized packets, even greater
gains may be realized.

Networking Implementation Notes 3-57

Acknowledgements
The internal structure of the system is patterned after the Xerox PUP architecture
[Boggs79], while in certain places the Internet protocol family has had a great deal of
influence in the design. The use of software interrupts for process invocation is based on similar facilities found in the VMS operating system. Many of the ideas related to protocol modularity, memory management, and network interfaces are based on Rob Gurwitz’s TCP/IP
implementation for the 4.1BSD version of UNIX on the VAX [Gurwitz81]. Greg Chesson
explained his use of trailer encapsulations in Datakit, instigating their use in our system.

References
[Boggs79]

Boggs, D. R., J. F. Shoch, E. A. Taft, and R. M. Metcalfe; PUP: An
Internetwork Architecture. Report CSL-79-10. XEROX Palo Alto
Research Center, July 1979.

[BBN78]

Bolt Beranek and Newman; Specification for the Interconnection of
Host and IMP. BBN Technical Report 1822. May 1978.

[Cerf78]

Cerf, V. G.; The Catenet Model for Internetworking. Internet Working

[Clark82]

Group, IEN 48. July 1978.

Clark, D. D.; Window and Acknowledgement Strategy in TCP. Internet
Working Group, IEN Draft Clark-2. March 1982.

[DEC80]

Digital Equipment Corporation; DECnet DIGITAL Network Architecture — General Description. Order No. AA-K179A-TK. October 1980.

[Gurwitz81]

Gurwitz, R. F.; VAX-UNIX Networking Support Project — Implementation Description. Internetwork Working Group, IEN 168. January
1981.

[ISO81]

International Organization for Standardization. ISO Open Systems
Interconnection — Basic Reference Model. ISO/TC 97/SC 16 N 719.
August 1981.

[Joy82a]

Joy, W.; Cooper, E.; Fabry, R.; Leffler, S.; and McKusick, M.; 4.2BSD
System Manual. Computer Systems Research Group, Technical Report
5. University of California, Berkeley. Draft of September 1, 1982.

[Postel79]

Postel, J., ed. DOD Standard User Datagram Protocol. Internet Working Group, IEN 88.

May 1979.

[Postel80a]

Postel, J., ed.

[Postel80b]

Postel, J., ed. DOD Standard Transmission Control Protocol. Internet

DOD Standard Internet Protocol.

Internet Working

Group, IEN 128. January 1980.

Working Group, IEN 129. January 1980.
[Xerox81]

Xerox Corporation. Internet Transport Protocols.
Integration Standard 028112. December 1981.

[Zimmermann80]

Zimmermann, H. OSI Reference Model — The ISO Model of Architecture for Open Systems Interconnection. IEEE Transactions on Communications. Com-28(4); 425-432. April 1980.

Xerox

System

Sendmail 3-59

SENDMAIL — An Internetwork Mail Router
Eric Allmant
Britton-Lee, Inc.

1919 Addison Street, Suite 105.
Berkeley, California 94704.

ABSTRACT

Routing mail through a heterogenous internet presents many new problems.

the worst of these is that of address mapping.
an ad hoc basis.

Among

Historically, this has been handled on

However, this approach has become unmanageable as internets

grow.

Sendmail acts a unified "post office” to which all mail can be submitted. Address interpretation is controlled by a production system, which can parse both domain-based
addressing and old-style ad hoc addresses. The production system is powerful
enough to rewrite addresses in the message header to conform to the standards of a

number of common target networks, including old (NCP/RFC733) Arpanet, new
(TCP/RFC822) Arpanet, UUCP, and Phonenet. Sendmail also implements an SMTP
server, message queueing, and aliasing.

Sendmail implements a general internetwork mail routing facility, featuring aliasing and
forwarding, automatic routing to network gateways, and flexible configuration.
In a simple network, each node has an address, and resources can be identified with a
host-resource pair; in particular, the mail system can refer to users using a host-username
pair. Host names and numbers have to be administered by a central authority, but usernames

can be assigned locally to each host.

In an internet, multiple networks with different characterstics and managements must
In particular, the syntax and semantics of resource identification change. Certain special cases can be handled trivially by ad hoc techniques, such as providing network
communicate.

names that appear local to hosts on other networks, as with the Ethernet at Xerox PARC.
the general case is extremely complex. For example, some networks require point-

However,

to-point routing, which simplifies the database update problem since only adjacent hosts must

be entered into the system tables, while others use end-to-end addressing.

Some networks use
a left-associative syntax and others use a right-associative syntax, causing ambiguity in mixed

addresses.

Internet standards seek to eliminate these problems. Initially, these proposed expanding
the address pairs to address triples, consisting of {network, host, resource} triples. Network

numbers must be universally agreed upon, and hosts can be assigned locally on each network.
The user-level presentation was quickly expanded to address domains, comprised of a local
resource identification and a hierarchical domain specification with a common static root. The

domain technique separates the issue of physical versus logical addressing. For example, an
address of the form “eric@a.cc.berkeley.arpa” describes only the logical organization of the

TA considerable part of this work was done while under the employ of the INGRES Project at the University of

California at Berkeley.

3-60 Sendmail

address space.
Sendmail is intended to help bridge the gap between the totally ad hoc world of networks that know nothing of each other and the clean, tightly-coupled world of unique network
numbers.

It can accept old arbitrary address syntaxes, resolving ambiguities using heuristics

specified by the system administrator, as well as domain-based addressing.

It helps guide the

conversion of message formats between disparate networks. In short, sendmail is designed to

assist a graceful transition to consistent internetwork addressing schemes.

Section 1 discusses the design goals for sendmail.
basic functions of the system.

Section 2 gives an overview of the

In section 3, details of usage are discussed.

Section 4 compares

sendmail to other internet mail routers, and an evaluation of sendmail is given in section 5,
including future plans.

1. DESIGN GOALS

Design goals for sendmail include:
(1)

Compatibility with the existing mail programs, including Bell version 6 mail, Bell version 7 mail [UNIX83], Berkeley Mail [Shoens79], BerkNet mail [Schmidt79], and
hopefully UUCP mail [Nowitz78a, Nowitz78b].

ARPANET mail [Crocker77a, Pos-

tel77] was also required.
(2)

Reliability, in the sense of guaranteeing that every message is correctly delivered or

at least brought to the attention of a human for correct disposal; no message should
ever be completely lost.

This goal was considered essential because of the emphasis

on mail in our environment.

It has turned out to be one of the hardest goals to

satisfy, especially in the face of the many anomalous message formats produced by
various ARPANET sites.

For example, certain sites generate improperly formated

addresses, occasionally causing error-message loops.

Some hosts use blanks in names,

causing problems with UNIX mail programs that assume that an address is one word.
The semantics of some fields are interpreted slightly differently by different sites.

summary, the obscure features of the ARPANET mail protocol really are used and

are difficult to support, but must be supported.

(3)

Existing software to do actual delivery should be used whenever possible.

This goal

derives as much from political and practical considerations as technical.
(4)

Easy expansion to fairly complex environments, including multiple connections to a
single network type (such as with multiple UUCP or Ether nets [Metcalfe76]).

This

goal requires consideration of the contents of an address as well as its syntax in order
to determine which gateway to use.

For example, the ARPANET is bringing up the

TCP protocol to replace the old NCP protocol.

No host at Berkeley runs both TCP

and NCP, so it is necessary to look at the ARPANET host name to determine
whether to route mail to an NCP gateway or a TCP gateway.

(5)

Configuration should not be compiled into the code.

A single compiled program

should be able to run as is at any site (barring such basic changes as the CPU type or
the operating system).
in real life.

We have found this seemingly unimportant goal to be critical

Besides the simple problems that occur when any program gets recom-

piled in a different environment, many sites like to “fiddle” with anything that they

will be recompiling anyway.

(6)

Sendmail must be able to let various groups maintain their own mailing lists, and let
individuals specify their own forwarding, without modifying the system alias file.

(7)

Each user should be able to specify which mailer to execute to process mail being
Anlivrarad

U.CU.VULUu

far

lUl

him

FALEALR,

Mhia

llllD

foatiive

LcabuLC

allAwx

allUVVD

1anra

uDClD

rhn

VVllU

arn

[¢ B W

110iNag
uDllls

annnialinna

DPC\/la

J.LUU

maillare

LILQALETELEDS

+hat

LiiAu

use a different format to build their environment without changing the system, and

Sendmail 3-61

facilitates specialized functions (such as returning an “I am on vacation” message).

(8)

Network traffic should be minimized by batching addresses to a single host where
possible, without assistance from the user.
These goals motivated the architecture illustrated in figure 1.

a mail generating and sending program.

The user interacts with

When the mail is created, the generator calls

sendmail, which routes the message to the correct mailer(s).

Since some of the senders

may be network servers and some of the mailers may be network clients, sendmail may be
used as an internet mail gateway.

. OVERVIEW
2.1. System Organization
Sendmail neither interfaces with the user nor does actual mail delivery.

Rather,

it collects a message generated by a user interface program (UIP) such as Berkeley
Mail, MS [Crocker77b], or MH [Borden79], edits the message as required by the destination network, and calls appropriate mailers to do mail delivery or queueing for net-

work transmission!.
cost.

This discipline allows the insertion of new mailers at minimum

In this sense sendmail resembles the Message Processing Module (MPM) of

[Postel79b].

2.2. Interfaces to the Outside World
There are three ways sendmail can communicate with the outside world, both in
receiving and in sending mail.

These are using the conventional UNIX argument

s.ender1

sender2

sender3

sendmail

maller1

maller2

maller3

Figure 1 — Sendmail System Structure.

e i

R
I

‘except when mailing to a file, when sendmail does the delivery directly.

3-62

Sendmail

vector/return status, speaking SMTP over a pair of UNIX pipes, and speaking SMTP
over an interprocess(or) channel.
2.2.1. Argument vector/exit status

This technique is the standard UNIX method for communicating with the process. A list of recipients is sent in the argument vector, and the message body is
sent on the standard input. Anything that the mailer prints is simply collected and
sent back to the sender if there were any problems. The exit status from the mailer
is collected after the message is sent, and a diagnostic is printed if appropriate.
2.2.2.

SMTP over pipes

The SMTP protocol [Postel82] can be used to run an interactive lock-step
interface with the mailer. A subprocess is still created, but no recipient addresses
are passed to the mailer via the argument list. Instead, they are passed one at a
time in commands sent to the processes standard input. Anything appearing on the
standard output must be a reply code in a special format.
2.2.3. SMTP over an IPC connection

This technique is similar to the previous technique, except that it uses a
4.2BSD IPC channel [UNIX83]. This method is exceptionally flexible in that the
mailer need not reside on the same machine. It is normally used to connect to a
sendmail process on another machine.
2.3. Operational Description

When a sender wants to send a message, it issues a request to sendmail using one
of the three methods described above. Sendmail operates in two distinct phases. In
the first phase, it collects and stores the message. In the second phase, message
delivery occurs. If there were errors during processing during the second phase, sendmail creates and returns a new message describing the error and/or returns an status
code telling what went wrong.

2.3.1. Argument processing and address parsing

If sendmail is called using one of the two subprocess techniques, the arguments are first scanned and option specifications are processed. Recipient addresses
are then collected, either from the command line or from the SMTP RCPT com-

mand, and a list of recipients is created. Aliases are expanded at this step, including
mailing lists. As much validation as possible of the addresses is done at this step:
syntax is checked, and local addresses are verified, but detailed checking of host
names and addresses is deferred until delivery. Forwarding is also performed as the
local addresses are verified.

Sendmail appends each address to the recipient list after parsing. When a
name is aliased or forwarded, the old name is retained in the list, and a flag is set
that tells the delivery phase to ignore this recipient. This list is kept free from
duplicates, preventing alias loops and duplicate messages deliverd to the same recipient, as might occur if a person is in two groups.
2.3.2. Message collection

Sendmail then collects the message. The message should have a header at the
beginning. No formatting requirements are imposed on the message except that
they must be lines of text (i.e., binary data is not allowed).
UJ.‘-\/J

BB GBS
B

o B B B

s a

s ot s Ty

ST W

The header is parsed
——
e LT R RS
B

and stored in memory, and the body of the message is saved in a temporary file.

Sendmail 3-63

simplify

the

program

interface,

the

message

collected

even

if no

addresses were valid. The message will be returned with an error.
2.3.3. Message delivery
For each unique mailer and host in the recipient list, sendmail calls the

appropriate mailer.

Kach mailer invocation sends to all users receiving the message

on one host. Mailers that only accept one recipient at a time are handled properly.
The message is sent to the mailer using one of the same three interfaces used

to submit a message to sendmail.
tomized header.

Each copy of the message is prepended by a cus-

The mailer status code is caught and checked, and a suitable error

message given as appropriate.

The exit code must conform to a system standard or

a generic message (“Service unavailable”) is given.
2.3.4. Queueing for retransmission

If the mailer returned an status that indicated that it might be able to handle
the mail later, sendmail will queue the mail and try again later.
2.3.5.

Return to sender

If errors occur during processing, sendmail returns the message to the sender

for retransmission.

The letter can be mailed back or written in the file “dead.letter”

in the sender’s home directory®.
2.4. Message Header Editing

Certain editing of the message header occurs automatically.
inserted under control of the configuration file.

Header lines can be

Some lines can be merged; for example,

a “From:” line and a “Full-name:” line can be merged under certain circumstances.
2.5. Configuration File

Almost all configuration information is read at runtime from an ASCII file, encoding macro definitions (defining the value of macros used internally), header declarations

(telling sendmail the format of header lines that it will process specially, i.e., lines that
it will add or reformat), mailer definitions (giving information such as the location and

characteristics of each mailer), and address rewriting rules (a limited production system

to rewrite addresses which is used to parse and rewrite the addresses).
To improve performance when reading the configuration file, a memory image can

be provided. This provides a “compiled” form of the configuration file.
3. USAGE AND IMPLEMENTATION
3.1. Arguments
Arguments may be flags and addresses.

Flags set various processing options.

Fol-

lowing flag arguments, address arguments may be given, unless we are running in

SMTP mode. Addresses follow the syntax in RFC822 [Crocker82] for ARPANET
address formats. In brief, the format is:
(1)

Anything in parentheses is thrown away (as a comment).

2Obviously, if the site giving the error is not the originating site, the only reasonable option is to mail back to
the sender. Also, there are many more error disposition options, but they only effect the error message — the “return
to sender” function is always handled in one of these two ways.

3-64 Sendmail

(2)

Anything in angle brackets (“<>”) is preferred over anything else.
implements the ARPANET standard that addresses of the form

This rule

user name <machine-address>

will send to the electronic “machine-address’ rather than the human “user name.”

(3)

Double quotes ( ” ) quote phrases; backslashes quote characters. Backslashes are
more powerful in that they will cause otherwise equivalent phrases to compare
differently — for example, user and ’user” are equivalent, but »ser is different
from either of them.

Parentheses, angle brackets, and double quotes must be properly balanced and

nested. The rewriting rules control remaining parsing®.
3.2. Mail to Files and Programs

Files and programs are legitimate message recipients. Files provide archival
storage of messages, useful for project administration and history. Programs are useful
as recipients in a variety of situations, for example, to maintain a public repository of
systems messages (such as the Berkeley msgs program, or the MARS system [Sattley78]).

Any address passing through the initial parsing algorithm as a local address (i.e,
not appearing to be a valid address for another mailer) is scanned for two special cases.
If prefixed by a vertical bar (“”) the rest of the address is processed as a shell command. If the user name begins with a slash mark (““/”’) the name is used as a file name,
instead of a login name.

Files that have setuid or setgid bits set but no execute bits set have those bits
honored if sendmail is running as root.

3.3. Aliasing, Forwarding, Inclusion

Sendmail reroutes mail three ways. Aliasing applies system wide. Forwarding
allows each user to reroute incoming mail destined for that account. Inclusion directs
sendmail to read a file for a list of addresses, and is normally used in conjunction with
aliasing.

3.3.1. Aliasing

Aliasing maps names to address lists using a system-wide file. This file is
indexed to speed access. Only names that parse as local are allowed as aliases; this
guarantees a unique key (since there are no nicknames for the local host).
3.3.2. Forwarding

After aliasing, recipients that are local and valid are checked for the existence
of a “.forward” file in their home directory. If it exists, the message is not sent to
that user, but rather to the list of users in that file. Often this list will contain only
one address, and the feature will be used for network mail forwarding.
Forwarding also permits a user to specify a private incoming mailer.
example, forwarding to:

”|/usr/local/newmail myname”
will use a different incoming mailer.

RS a SR

Disclaimer: Some special processing is done after rewriting local

.
SN

[

PUSN. I S

(SRR

For

Sendmail 3-65

3.3.3. Inclusion

Inclusion is specified in RFC 733 [Crocker77a] syntax:
:Include: pathname

An address of this form reads the file specified by pathname and sends to all users
listed in that file.

The intent is not to support direct use of this feature, but rather to use this as
a subset of aliasing. For example, an alias of the form:
project: :include:/usr/project/userlist

is a method of letting a project maintain a mailing list without interaction with the
system administration, even if the alias file is protected.
It is not necessary to rebuild the index on the alias database when a :include:
list is changed.

3.4. Message Collection

Once all recipient addresses are parsed and verified, the message is collected. The
message comes in two parts: a message header and a message body, separated by a
blank line.

The header is formatted as a series of lines of the form
field-name: field-value

Field-value can be split across lines by starting the following lines with a space or a tab.
Some header fields have special internal meaning, and have appropriate special processing. Other headers are simply passed through. Some header fields may be added
automatically, such as time stamps.

The body is a series of text lines. It is completely uninterpreted and untouched,
except that lines beginning with a dot have the dot doubled when transmitted over an
SMTP channel. This extra dot is stripped by the receiver.
3.5. Message Delivery

The send queue is ordered by receiving host before transmission to implement
message batching. Each address is marked as it is sent so rescanning the list is safe.
An argument list is built as the scan proceeds. Mail to files is detected during the scan
of the send list. The interface to the mailer is performed using one of the techniques
described in section 2.2.

After a connection is established, sendmail makes the per-mailer changes to the
header and sends the result to the mailer. If any mail is rejected by the mailer, a flag is
set to invoke the return-to-sender function after all delivery completes.
3.6. Queued Messages

If the mailer returns a “temporary failure” exit status, the message is queued. A
control file is used to describe the recipients to be sent to and various other parameters.
This control file is formatted as a series of lines, each describing a sender, a recipient,
the time of submission, or some other salient parameter of the message. The header of
the message is stored in the control file, so that the associated data file in the queue is
just the temporary file that was originally collected.
3.7. Configuration
1
Configuration is controlled primarily
by a configuration file read at startup. SendN

NS BB AL

Tl B

matil should not need to be recomplied except

3-66

Sendmail

(1)

To change operating systems (V6, V7/32V, 4BSD).

(2)

To remove or insert the DBM (UNIX database) library.

(3)

To change ARPANET reply codes.

(4)

To add headers fields requiring special processing.

Adding mailers or changing parsing (i.e., rewriting) or routing information does not
require recompilation.
If the mail is being sent by a local user, and the file

“.mailcf” exists in the

sender’s home directory, that file is read as a configuration file after the system
configuration file.
The

The primary use of this feature is to add header lines.

configuration

file

encodes

macro

definitions,

header

definitions,

mailer

definitions, rewriting rules, and options.
3.7.1.

Macros

Macros can be used in three ways.

Certain macros transmit unstructured tex-

tual information into the mail system, such as the name sendmail will use to identify itself in error messages.

Other macros transmit information from sendmail to

the configuration file for use in creating other fields (such as argument vectors to
mailers); e.g., the name of the sender, and the host and user of the recipient.

Other

macros are unused internally, and can be used as shorthand in the configuration file.

3.7.2. Header declarations
Header declarations inform sendmail of the format of known header lines.
Knowledge of a few header lines is built into sendmail, such as the “From:” and
“Date:” lines.

Most configured headers will be automatically inserted in the outgoing message
if they don’t exist in the incoming message.

Certain headers are suppressed by some

mailers.

3.7.3. Mailer declarations

Mailer declarations tell sendmail of the various mailers available to it.

The

definition specifies the internal name of the mailer, the pathname of the program to
call, some flags associated with the mailer, and an argument vector to be used on the

call; this vector is macro-expanded before use.
3.7.4. Address rewriting rules
The heart of address parsing in sendmail is a set of rewriting rules.

These are

an ordered list of pattern-replacement rules, (somewhat like a production system,
except that order is critical), which are applied to each address.

The address is

rewritten textually until it is either rewritten into a sp/ecial canonical form (i.e., a

(mailer, host, user)

3-tuple, such as

{arpanet, usc-isif, postel}

address “postel@usc-isif”’), or it falls off the end.

representing the

When a pattern matches, the rule

is reapplied until it fails.
The configuration file also supports the editing of addresses into different for-

mats. For example, an address of the form:

ucsfcgl!tef
might be mapped into:
tef@ucsfcgl. UUCP
to conform to the domain syntax.

Translations can also be done in the other

Sendmail 3-67

direction.

3.7.5. Option setting

There are several options that can be set from the configuration file. These
include the pathnames of various support files, timeouts, default modes, etc.
4. COMPARISON WITH OTHER MAILERS
4.1. Delivermail

Sendmail is an outgrowth of delivermail. The primary differences are:

(1)

Configuration information is not compiled in. This change simplifies many of the
problems of moving to other machines. It also allows easy debugging of new
mailers.

(2)

Address parsing is more flexible. For example, delivermail only supported one
gateway to any network, whereas sendmail can be sensitive to host names and
reroute to different gateways.

(3)

Forwarding and :include: features eliminate the requirement that the system alias
file be writable by any user (or that an update program be written, or that the
system administration make all changes).

(4)

Sendmail supports message batching across networks when a message is being
sent to multiple recipients.

(5)

A mail queue is provided in sendmail. Mail that cannot be delivered immediately
but can potentially be delivered later is stored in this queue for a later retry. The
queue also provides a buffer against system crashes; after the message has been
collected it may be reliably redelivered even if the system crashes during the initial delivery.

(6)

Sendmail uses the networking support provided by 4.2BSD to provide a direct
interface networks such as the ARPANET and/or Ethernet using SMTP (the Simple Mail Transfer Protocol) over a TCP/IP connection.

4,2, MMDF

MMDF [Crocker79] spans a wider problem set than sendmail. For example, the
domain of MMDF includes a “phone network’ mailer, whereas sendmail calls on preexisting mailers in most cases.

MMDF and sendmail both support aliasing, customized mailers, message batching, automatic forwarding to gateways, queueing, and retransmission. MMDF supports
two-stage timeout, which sendmail does not support.

The configuration for MMDF is compiled into the code®.
Since MMDF does not consider backwards compatibility as a design goal, the
address parsing is simpler but much less flexible.

It is somewhat harder to integrate a new channel® into MMDF. In particular,
MMDF must know the location and format of host tables for all channels, and the
channel must speak a special protocol. This allows MMDF to do additional verification
(such as verifying host names) at submission time.
‘Dynamic configuration tables are currently being considered for MMDF; allowing the installer to select either
compiled or dynamic tables.

The MMDF equivalent of a sendmail “mailer.”

3-68 Sendmail

MMDF strictly separates the submission and delivery phases.

Although sendmail

has the concept of each of these stages, they are integrated into one program, whereas

in MMDF they are split into two programs.
4.3.

Message Processing Module
The Message Processing Module (MPM) discussed by Postel [Postel79b] matches

sendmail closely in terms of its basic architecture.

However, like MMDF, the MPM

includes the network interface software as part of its domain.
MPM also postulates a duplex channel to the receiver, as does MMDF, thus

allowing simpler handling of errors by the mailer than is possible in sendmail.

When a

message queued by sendmail is sent, any errors must be returned to the sender by the

mailer itself.

Both MPM and MMDF mailers can return an immediate error response,

and a single error processor can create an appropriate response.
MPM prefers passing the message as a structured object, with type-length-value

tuples®.

Such a convention requires a much higher degree of cooperation between

mailers than is required by sendmail.

MPM also assumes a universally agreed upon

internet name space (with each address in the form of a net-host-user tuple), which

sendmail does not.

5. EVALUATIONS AND FUTURE PLANS
Sendmail is designed to work in a nonhomogeneous environment.

Every attempt is

made to avoid imposing unnecessary constraints on the underlying mailers.

driven much of the design.

This goal has

One of the major problems has been the lack of a uniform

address space, as postulated in [Postel79a] and [Postel79b].
A nonuniform address space implies that a path will be specified in all addresses,
either explicitly (as part of the address) or implicitly (as with implied forwarding to gateways).

This restriction has the unpleasant effect of making replying to messages exceed-

ingly difficult, since there is no one “address” for any person, but only a way to get there
from wherever you are.
Interfacing to mail programs that were not initially intended to be applied in an
internet environment has been amazingly successful, and has reduced the job to a manage-

able task.
Sendmail has knowledge of a few difficult environments

built in.

It generates

ARPANET FTP/SMTP compatible error messages (prepended with three-digit numbers
[Neigus73, Postel74, Postel82]) as necessary, optionally generates UNIX-style “From” lines
on the front of messages for some mailers, and knows how to parse the same lines on input.
Also, error handling has an option customized for BerkNet.
The decision to avoid doing any type of delivery where possible (even, or perhaps
especially, local delivery) has turned out to be a good idea.

Even with local delivery, there

are issues of the location of the mailbox, the format of the mailbox, the locking protocol
used, etc., that are best decided by other programs.

One surprisingly major annoyance in

many internet mailers is that the location and format of local mail is built in.

seems to be that local mail is so common that it should be efficient.

The feeling

This feeling is not

born out by our experience; on the contrary, the location and format of mailboxes seems to
vary widely from system to system.

The ability to automatically generate a response to incoming mail (by forwarding
mail to a program) seems useful (“I am on vacation until late August....”) but can create

problems such as forwarding loops (two people on vacation whose programs send notes
*This is similar to the NBS standard.

Sendmail 3-69

back and forth, for instance) if these programs are not well written. A program could be
written to do standard tasks correctly, but this would solve the general case.
It might be desirable to implement some form of load limiting.

I am unaware of any

mail system that addresses this problem, nor am I aware of any reasonable solution at this
time.

The configuration file is currently practically inscrutable; considerable convenience

could be realized with a higher-level format.
It seems clear that common protocols will be changing soon to accommodate changing
requirements and environments.

These changes will include modifications to the message

header (e.g., [NBS80]) or to the body of the message itself (such as for multimedia mes-

sages [Postel80]).

Experience indicates that these changes should be relatively trivial to

integrate into the existing system.

In tightly coupled environments, it would be nice to have a name server such as

Grapvine [Birrell82] integrated into the mail system.

This would allow a site such as

“Berkeley” to appear as a single host, rather than as a collection of hosts, and would allow

people to move transparently among machines without having to change their addresses.

Such a facility would require an automatically updated database and some method of
Ideally this would be effective even without all hosts being under a single management. However, it is not clear whether this feature should be integrated into
resolving conflicts.

the aliasing facility or should be considered a ‘“value added” feature outside sendmail
itself.

As a more interesting case, the CSNET name server [Solomon81] provides an facility
that goes beyond a single tightly-coupled environment.
exist outside of sendmail however.

Such a facility would normally

ACKNOWLEDGEMENTS

Thanks are due to Kurt Shoens for his continual cheerful assistance and good advice,
Bill Joy for pointing me in the correct direction (over and over), and Mark Horton for more

advice, prodding, and many of the good ideas. Kurt and Eric Schmidt are to be credited for
using delivermail as a server for their programs (Mail and BerkNet respectively) before any
sane person should have, and making the necessary modifications promptly and happily. Eric
gave me considerable advice about the perils of network software which saved me an unknown
amount of work and grief. Mark did the original implementation of the DBM version of aliasing, installed the VFORK code, wrote the current version of rmail, and was the person who
really convinced me to put the work into delivermail to turn it into sendmail. Kurt deserves
accolades for using sendmail when I was myself afraid to take the risk; how a person can continue to be so enthusiastic in the face of so much bitter reality is beyond me.
Kurt, Mark, Kirk McKusick, Marvin Solomon, and many others have reviewed this
paper, giving considerable useful advice.

Special thanks are reserved for Mike Stonebraker at Berkeley and Bob Epstein at
Britton-Lee, who both knowingly allowed me to put so much work into this project when there

were so many other things I really should have been working on.

3-70 Sendmail

REFERENCES
[Birrell82]

Birrell, A. D., Levin, R., Needham, R. M., and Schroeder, M. D., “Grapevine: An Exercise in Distributed Computing.” In Comm. A.C.M. 25,
4, April 82.

[Borden79]

Borden, S., Gaines, R. S., and Shapiro, N. Z., The MH Message Handling

System:

Users’

Manual.

R-2367-PAF.

October 1979.
[Crocker77a]

Rand

Corporation.

Crocker, D. H., Vittal, J. J., Pogran, K. T., and Henderson, D. A. Jr.,
Standard for the Format of ARPA Network Text Messages.

RFC 733,

NIC 41952. In [Feinler78]. November 1977.
[Crocker77b]

Crocker, D. H., Framework and Functions of the MS Personal Message System.
fornia.

[Crocker79]

R-2134-ARPA, Rand Corporation, Santa Monica, Cali-

1977.

Crocker, D. H., Szurkowski, E. S., and Farber, D. J., An Internetwork
Memo Distribution Facility

— MMDF.

6th Data Communication

Symposium, Asilomar. November 1979.
[Crocker82]

Crocker, D. H., Standard for the Format of Arpa Internet Text Messages.

RFC 822.

Network Information Center, SRI International,

Menlo Park, California. August 1982.
[Metcalfe76]

Metcalfe, R., and Boggs, D., “Ethernet: Distributed Packet Switching
for Local Computer Networks”, Communications of the ACM 19, 1.

July 1976.
[Feinler78]

Feinler, E., and Postel, J.

(eds.), ARPANET Protocol Handbook.

NIC 7104, Network Information Center, SRI International, Menlo
Park, California.

[NBS80]

1978.

National Bureau of Standards, Specification of a Draft Message For-

mat Standard. Report No. ICST/CBOS 80-2. October 1980.
[Neigus73]

Neigus, N., File Transfer Protocol for the ARPA Network.

RFC 542,

NIC 17759. In [Feinler78]. August, 1973.
[Nowitz78a]

Nowitz, D. A., and Lesk, M. E.,
tems.

Bell Laboratories.

A Dial-Up Network of UNIX Sys-

In UNIX Programmer’s Manual, Seventh

Edition, Volume 2. August, 1978.

[Nowitz78b]

Nowitz, D. A., Uucp Implementation Description.

Bell Laboratories.

In UNIX Programmer’s Manual, Seventh Edition, Volume 2.

October,

1978.

[Postel74]

Postel, J., and Neigus, N., Revised FTP Reply Codes.

RFC 640, NIC

30843. In [Feinler78]. June, 1974.
[Postel77]

Postel, J., Mail Protocol.

NIC 29588. In [Feinler78].

[Postel79a]

Postel, J., Internet Message Protocol.

November 1977.

RFC 753, IEN 85.

Network

Information Center, SRI International, Menlo Park, California. March
1979.

[Postel79b]

Postel, J. B., An Internetwork Message Structure.
the

Sixth Data

November 1979.
LW

Communications

Symposium,

In Proceedings of

IEEE.

New York.

Sendmail 3-71

[Postel80]

Postel, J. B., A Structured Format for Transmission of Multi-Media

Documents.

RFC 767.

Network Information Center, SRI Interna-

tional, Menlo Park, California. August 1980.
[Postel82]

Postel, J. B., Simple Mail Transfer Protocol.

RFC788).

Network Information Center,

RFC821 (obsoleting

SRI International, Menlo

Park, California. August 1982.

[Schmidt79]

Schmidt, E., An Introduction to the Berkeley Network.
California, Berkeley California.

[Shoens79]

Shoens, K., Mail Reference Manual.
ley.

University of

1979.
University of California, Berke-

In UNIX Programmer’s Manual, Seventh Edition, Volume 2C.

December 1979.

[Sluizer81]

Sluizer, S., and Postel, J. B., Mail Transfer Protocol.

RFC 780.

Net-

work Information Center, SRI International, Menlo Park, California.
May 1981.

[Solomon81]

Solomon, M., Landweber, L., and Neuhengen, D., “The Design of the
CSNET Name Server.” CS-DN-2, University of Wisconsin, Madison.
November 1981.

[Su82]

Su, Zaw-Sing, and Postel, Jon, The Domain Naming Convention for
Internet User Applications.

RFC819.

Network Information Center,

SRI International, Menlo Park, California. August 1982.
[UNIX83]

The UNIX Programmer’s Manual, Seventh Edition, Virtual VAX-11
Version, Volume 1.

Bell Laboratories, modified by the University of

California, Berkeley, California.

March, 1983.

Introduction 4-1

PART 4: SECURITY CONSIDERATIONS

Security on the ULTRIX-32 system is flexible and reasonably comprehensive. These two articles describe a number of measures you can take to make your installation moderately secure.
The first article in this part, “On the Security of UNIX,” explains the major features and
weaknesses of ULTRIX-32 system security. The second article, “Password Security: A Case
History,” tells how the password facility used on the ULTRIX-32 system was developed.

Protection Against Crashes and Unauthorized Access

Unrestricted use of disk space on the ULTRIX-32 system may cripple or stop the operating
system, and Ritchie indicates in “On the Security of UNIX” that the software cannot be protected from this type of abuse. However, the ULTRIX-32 system does include the quota utility, which enables the system manager to control the use of resources by limiting the number
of blocks and the number of files available to each user. See “Disk Quotas in a UNIX
Environment” in Part 2 of this volume.

The Ritchie article explains the functions of the file protection bits, the user identification
number (UID), and the user-group identification number (GID); these functions allow users to
control access to their files.

In addition, Ritchie outlines the ULTRIX-32 system schemes for:
e

Data encryption

Password security

Precautions concerning the superuser account, set-UID programs, mail, and the
mount command

Password Security Development

“Password Security: A Case History,” by Morris and Thompson, outlines the objectives of the
password system on the ULTRIX-32 system:

To protect the system against unauthorized users

To prevent logged-in users from performing unauthorized functions

To minimize inconvenience to legitimate users

The case history describes the early, rejected password schemes and their weaknesses, and it
shows how they evolved to the current scheme. You may find the article useful as well as
interesting, because it offers well-researched precautions to users and tested recommendations
to system managers.

UNIX Security 4-3

On the Security of UNIX
Dennis M. Ritchie

Recently there has been much interest in the security aspects of operating systems and

software. At issue is the ability to prevent undesired disclosure of information, destruction of
information, and harm to the functioning of the system.

This paper discusses the degree of

security which can be provided under the UNIXt system and offers a number of hints on how
to improve security.
The first fact to face is that UNIX was not developed with security, in any realistic sense,
in mind; this fact alone guarantees a vast number of holes.

(Actually the same statement can

be made with respect to most systems.) The area of security in which UNIX is theoretically
weakest is in protecting against crashing or at least crippling the operation of the system. The
problem here is not mainly in uncritical acceptance of bad parameters to system calls— there
may be bugs in this area, but none are known— but rather in lack of checks for excessive con-

sumption of resources.

Most notably, there is no limit on the amount of disk storage used,

either in total space allocated or in the number of files or directories.

Here is a particularly

ghastly shell sequence guaranteed to stop the system:

while : ; do
mkdir x

cd x
done
Either a panic will occur because all the i-nodes on the device are used up, or all the disk

blocks will be consumed, thus preventing anyone from writing files on the device.
In this version of the system, users are prevented from creating more than a set number

of processes simultaneously, so unless users are in collusion it is unlikely that any one can stop
the system altogether.

However, creation of 20 or so CPU or disk-bound jobs leaves few

resources available for others.

Also, if many large jobs are run simultaneously, swap space

may run out, causing a panic.

It should be evident that excessive consumption of disk space, files, swap space, and
processes can easily occur accidentally in malfunctioning programs as well as at command
level.
fix.

In fact UNIX is essentially defenseless against this kind of abuse, nor is there any easy

The best that can be said is that it is generally fairly easy to detect what has happened

when disaster strikes, to identify the user responsible, and take appropriate action. In prac-

tice, we have found that difficulties in this area are rather rare, but we have not been faced
with malicious users, and enjoy a fairly generous supply of resources which have served to
cushion us against accidental overconsumption.
The picture is considerably brighter in the area of protection of information from unau-

thorized perusal and destruction.

Here the degree of security seems

(almost) adequate

theoretically, and the problems lie more in the necessity for care in the actual use of the system.

Each UNIX file has associated with it eleven bits of protection information together with

a user identification number and a user-group identification number (UID and GID).

Nine of

the protection bits are used to specify independently permission to read, to write, and to
T UNIX is a trademark of Bell Laboratories.

4-4 UNIX Security

execute the file to the user himself, to members of the user’s group, and to all other users.
Each process generated by or for a user has associated with it an effective UID and a real
UID, and an effective and real GID. When an attempt is made to access the file for reading,
writing, or execution, the user process’s effective UID is compared against the file’s UID; if a

match is obtained, access is granted provided the read, write, or execute bit respectively for
the user himself is present. If the UID for the file and for the process fail to match, but the
GID’s do match, the group bits are used; if the GID’s do not match, the bits for other users
are tested. The last two bits of each file’s protection information, called the set-UID and setGID bits, are used only when the file is executed as a program. If, in this case, the set-UID
bit is on for the file, the effective UID for the process is changed to the UID associated with
the file; the change persists until the process terminates or until the UID changed again by
another execution of a set-UID file. Similarly the effective group ID of a process is changed to
the GID associated with a file when that file is executed and has the set-GID bit set. The real
UID and GID of a process do. not change when any file is executed, but only as the result of a
privileged system call.

The basic notion of the set-UID and set-GID bits is that one may write a program which
is executable by others and which maintains files accessible to others only by that program.
The classical example is the game-playing program which maintains records of the scores of
its players. The program itself has to read and write the score file, but no one but the game’s
sponsor can be allowed unrestricted access to the file lest they manipulate the game to their
own advantage. The solution is to turn on the set-UID bit of the game program. When, and
only when, it is invoked by players of the game, it may update the score file but ordinary programs executed by others cannot access the score.

There are a number of special cases involved in determining access permissions. Since
executing a directory as a program is a meaningless operation, the execute-permission bit, for
directories, is taken instead to mean permission to search the directory for a given file during
the scanning of a path name; thus if a directory has execute permission but no read permission for a given user, he may access files with known names in the directory, but may not read
(list) the entire contents of the directory. Write permission on a directory is interpreted to
mean that the user may create and delete files in that directory; it is impossible for any user
to write directly into any directory.

Another, and from the point of view of security, much more serious special case is that
there is a “super user” who is able to read any file and write any non-directory. The superuser is also able to change the protection mode and the owner UID and GID of any file and to
invoke privileged system calls. It must be recognized that the mere notion of a super-user is a

theoretical, and usually practical, blemish on any protection scheme.

The first necessity for a secure system is of course arranging that all files and directories
have the proper protection modes. Traditionally, UNIX software has been exceedingly permissive in this regard; essentially all commands create files readable and writable by everyone. In
the current version, this policy may be easily adjusted to suit the needs of the installation or
the individual user. Associated with each process and its descendants is a mask, which is in
effect and -ed with the mode of every file and directory created by that process. In this way,
users can arrange that, by default, all their files are no more accessible than they wish. The
standard mask, set by login, allows all permissions to the user himself and to his group, but
disallows writing by others.

To maintain both data privacy and data integrity, it is necessary, and largely sufficient,
to make one’s files inaccessible to others. The lack of sufficiency could follow from the
existence of set-UID programs created by the user and the possibility of total breach of system security in one of the wavs discussed below (or one of the ways not discussed below). For
greater protection, an encryption scheme is available. Since the editor is able to create
encrypted documents, and the crypt command can be used to pipe such documents into the
other text-processing programs, the length of time during which cleartext versions need be
available is strictly limited. The encryption scheme used is not one of the strongest known,

UNIX Security 4-5

but it is judged adequate, in the sense that cryptanalysis is likely to require considerably more
effort than more direct methods of reading the encrypted files. For example, a user who
stores data that he regards as truly secret should be aware that he is implicitly trusting the
system administrator not to install a version of the crypt command that stores every typed
password in a file.

Needless to say, the system administrators must be at least as careful as their most
demanding user to place the correct protection mode on the files under their control. In particular, it is necessary that special files be protected from writing, and probably reading, by
ordinary users when they store sensitive files belonging to other users. It is easy to write programs that examine and change files by accessing the device on which the files live.
On the issue of password security, UNIX is probably better than most systems.

Pass-

words are stored in an encrypted form which, in the absence of serious attention from specialists in the field, appears reasonably secure, provided its limitations are understood. In the
current version, it is based on a slightly defective version of the Federal DES; it is purposely
defective so that easily-available hardware is useless for attempts at exhaustive key-search.
Since both the encryption algorithm and the encrypted passwords are available, exhaustive

enumeration of potential passwords is still feasible up to a point. We have observed that
users choose passwords that are easy to guess: they are short, or from a limited alphabet, or in
a dictionary. Passwords should be at least six characters long and randomly chosen from an
alphabet which includes digits and special characters.
Of course there also exist feasible non-cryptanalytic ways of finding out passwords. For
example: write a program which types out “login:” on the typewriter and copies whatever is
typed to a file of your own. Then invoke the command and go away until the victim arrives.
The set-UID (set-GID) notion must be used carefully if any security is to be maintained.
The first thing to keep in mind is that a writable set-UID file can have another program

copied onto it. For example, if the super-user (su) command is writable, anyone can copy the
shell onto it and get a password-free version of su. A more subtle problem can come from setUID programs which are not sufficiently careful of what is fed into them. To take an obsolete
example, the previous version of the mail command was set-UID and owned by the superuser. This version sent mail to the recipient’s own directory. The notion was that one should
be able to send mail to anyone even if they want to protect their directories from writing.
The trouble was that mail was rather dumb: anyone could mail someone else’s private file to
himself.

Much more serious is the following scenario: make a file with a line like one in the

password file which allows one to log in as the super-user. Then make a link named “.mail” to

the password file in some writable directory on the same device as the password file (say
/tmp). Finally mail the bogus login line to /tmp/.mail; You can then login as the super-user,
clean up the incriminating evidence, and have your will.
The fact that users can mount their own disks and tapes as file systems can be another
way of gaining super-user status. Once a disk pack is mounted, the system believes what is on
it. Thus one can take a blank disk pack, put on it anything desired, and mount it. There are

obvious and unfortunate consequences. For example: a mounted disk with garbage on it will
crash the system; one of the files on the mounted disk can easily be a password-free version of
su; other files can be unprotected entries for special files. The only easy fix for this problem
is to forbid the use of mount to unprivileged users. A partial solution, not so restrictive,
would be to have the mount command examine the special file for bad data, set-UID programs owned by others, and accessible special files, and balk at unprivileged invokers.

Password Security 4-7

Password Security: A Case History
Robert Morris
Ken Thompson

Bell Laboratories

Murray Hill, New Jersey 07974

INTRODUCTION

Password security on the UNIXT time-sharing system [1] is provided by a collection of
programs whose elaborate and strange design is the outgrowth of many years of experience

with earlier versions. To help develop a secure system, we have had a continuing competition
to devise new ways to attack the security of the system (the bad guy) and, at the same time,
to devise new techniques to resist the new attacks (the good guy). This competition has been
in the same vein as the competition of long standing between manufacturers of armor plate
and those of armor-piercing shells. For this reason, the description that follows will trace the
history of the password system rather than simply presenting the program in its current state.
In this way, the reasons for the design will be made clearer, as the design cannot be understood without also understanding the potential attacks.
An underlying goal has been to provide password security at minimal inconvenience to

the users of the system. For example, those who want to run a completely open system
without passwords, or to have passwords only at the option of the individual users, are able to
do so, while those who require all of their users to have passwords gain a high degree of security against penetration of the system by unauthorized users.
The password system must be able not only to prevent any access to the system by
unauthorized users (i.e. prevent them from logging in at all), but it must also prevent users
who are already logged in from doing things that they are not authorized to do. The so called
“super-user’” password, for example, is especially critical because the super-user has all sorts
of permissions and has essentially unlimited access to all system resources.

Password security is of course only one component of overall system security, but it is an
essential component. Experience has shown that attempts to penetrate remote-access systems
have been astonishingly sophisticated.

Remote-access systems are peculiarly vulnerable to penetration by outsiders as there are
threats at the remote terminal, along the communications link, as well as at the computer
itself. Although the security of a password encryption algorithm is an interesting intellectual

and mathematical problem, it is only one tiny facet of a very large problem. In practice, physical security of the computer, communications security of the communications link, and physical control of the computer itself loom as far more important issues. Perhaps most important of all is control over the actions of ex-employees, since they are not under any direct control and they may have intimate knowledge about the system, its resources, and methods of
access. Good system security involves realistic evaluation of the risks not only of deliberate
attacks but also of casual unauthorized access and accidental disclosure.

t UNIX is a trademark of Bell Laboratories.

4-8 Password Security
PROLOGUE

The UNIX system was first implemented with a password file that contained the actual
passwords of all the users, and for that reason the password file had to be heavily protected
against being either read or written. Although historically, this had been the technique used
for remote-access systems, it was completely unsatisfactory for several reasons.
The technique is excessively vulnerable to lapses in security.

Temporary loss of protec-

tion can occur when the password file is being edited or otherwise modified. There is no way
to prevent the making of copies by privileged users. Experience with several earlier remoteaccess systems showed that such lapses occur with frightening frequency. Perhaps the most
memorable such occasion occurred in the early 60’s when a system administrator on the CTSS
system at MIT was editing the password file and another system administrator was editing the
daily message that is printed on everyone’s terminal on login. Due to a software design error,
the temporary editor files of the two users were interchanged and thus, for a time, the password file was printed on every terminal when it was logged in.
Once such a lapse in security has been discovered, everyone’s password must be changed,
usually simultaneously, at a considerable administrative cost. This is not a great matter, but
far more serious is the high probability of such lapses going unnoticed by the system administrators.

Security against unauthorized disclosure of the passwords was, in the last analysis,
impossible with this system because, for example, if the contents of the file system are put on
to magnetic tape for backup, as they must be, then anyone who has physical access to the tape
can read anything on it with no restriction.

Many programs must get information of various kinds about the users of the system, and

these programs in general should have no special permission to read the password file. The
information which should have been in the password file actually was distributed (or replicated) into a number of files, all of which had to be updated whenever a user was added to or
dropped from the system.
THE FIRST SCHEME

The obvious solution is to arrange that the passwords not appear in the system at all,
and it is not difficult to decide that this can be done by encrypting each user’s password, putting only the encrypted form in the password file, and throwing away his original password
(the one that he typed in). When the user later tries to log in to the system, the password
that he types is encrypted and compared with the encrypted version in the password file. If
the two match, his login attempt is accepted. Such a scheme was first described in [3, p.91ff.].
It also seemed advisable to devise a system in which neither the password file nor the pass-

word program itself needed to be protected against being read by anyone.
All that was needed to implement these ideas was to find a means of encryption that was

very difficult to invert, even when the encryption program is available.

Most of the standard

encryption methods used (in the past) for encryption of messages are rather easy to invert. A
convenient and rather good encryption program happened to exist on the system at the time;
it simulated the M-209 cipher machine [4] used by the U.S. Army during World War II. It
turned out that the M-209 program was usable, but with a given key, the ciphers produced by
this program are trivial to invert. It is a much more difficult matter to find out the key given
the cleartext input and the enciphered output of the program. Therefore, the password was
used not as the text to be encrypted but as the key, and a constant was encrypted using this
key. The encrypted result was entered into the password file.
ATTACKS ON THE FIRST APPROACH

Suppos‘e that the bad guy has available the text of the password encryption program and
the complete password file.
disposal.

Suppose also that he has substantial computing capacity at his

Password Security 4-9

One obvious approach to penetrating the password mechanism is to attempt to find a
general method of inverting the encryption algorithm. Very possibly this can be done, but few
successful results have come to light, despite substantial efforts extending over a period of
more than five years. The results have not proved to be very useful in penetrating systems.
Another approach to penetration is simply to keep trying potential passwords until one

succeeds; this is a general cryptanalytic approach called key search. Human beings being
what they are, there is a strong tendency for people to choose relatively short and simple passwords that they can remember. Given free choice, most people will choose their passwords
from a restricted character set (e.g. all lower-case letters), and will often choose words or
names. This human habit makes the key search job a great deal easier.

The critical factor involved in key search is the amount of time needed to encrypt a
potential password and to check the result against an entry in the password file. The running
time to encrypt one trial password and check the result turned out to be approximately 1.25

milliseconds on a PDP-11/70 when the encryption algorithm was recoded for maximum speed.
It is takes essentially no more time to test the encrypted trial password against all the passwords in an entire password file, or for that matter, against any collection of encrypted pass-

words, perhaps collected from many installations.

If we want to check all passwords of length n that consist entirely of lower-case letters,
the number of such passwords is 26n. If we suppose that the password consists of printable
characters only, then the number of possible passwords is somewhat less than 957. (The stan-

dard system “character erase” and “line kill” characters are, for example, not prime candidates.) We can immediately estimate the running time of a program that will test every password of a given length with all of its characters chosen from some set of characters. The following table gives estimates of the running time required on a PDP-11/70 to test all possible
character strings of length n chosen from various sets of characters: namely, all lower-case
letters, all lower-case letters plus digits, all alphanumeric characters, all 95 printable ASCII
characters, and finally all 128 ASCII characters.

26 lower-case
letters

36 lower-case letters
and digits

1
2
3

30 msec.
800 msec.
22 sec.

40 msec.
2 sec.
58 sec.

4 hrs.

21 hrs.

10 min.

62 alphanumeric
characters

95 printable
characters

all 128 ASCII
characters

80 msec.
5 sec.
5 min.

120 msec.
11 sec.
17 min.

160 msec.
20 sec.
43 min.

35 min.

5 hrs.

318 hrs.

28 hrs.

93 hrs.

107 hrs.

One has to conclude that it is no great matter for someone with access to a PDP-11 to test all
lower-case alphabetic strings up to length five and, given access to the machine for, say,
several weekends, to test all such strings up to six characters in length. By using such a program against a collection of actual encrypted passwords, a substantial fraction of all the passwords will be found.

Another profitable approach for the bad guy is to use the word list from a dictionary or
to use a list of names. For example, a large commercial dictionary contains typicallly about
950,000 words; these words can be checked in about five minutes. Again, a noticeable fraction
of any collection of passwords will be found. Improvements and extensions will be (and have
been) found by a determined bad guy. Some “good” things to try are:
-

The dictionary with the words spelled backwards.

A list of first names (best obtained from some mailing list). Last names, street names,

The above with initial upper-case letters.

and city names also work well.

4-10 Password Security
-

All valid license plate numbers in your state.

(This takes about five hours in New Jer-

sey.)
-

Room numbers, social security numbers, telephone numbers, and the like.
The authors have conducted experiments to try to determine typical users’ habits in the

choice of passwords when no constraint is put on their choice.
except to the bad guy.

The results were disappointing,

In a collection of 3,289 passwords gathered from many users over a

long period of time;

15 were a single ASCII character;
72 were strings of two ASCII characters;
464 were strings of three ASCII characters;
477 were string of four alphamerics;

706 were five letters, all upper-case or all lower-case;
605 were six letters, all lower-case.
An additional 492 passwords appeared in various available dictionaries, name lists, and the
like.

A total of 2,831, or 86% of this sample of passwords fell into one of these classes.
There was, of course, considerable overlap between the dictionary results and the charac-

ter string searches.

The dictionary search alone, which required only five minutes to run, pro-

duced about one third of the passwords.

Users could be urged (or forced) to use either longer passwords or passwords chosen
from a larger character set, or the system could itself choose passwords for the users.

AN ANECDOTE
An entertaining and instructive example is the attempt made at one installation to force
users to use less predictable passwords.
system supplied them.

The users did not choose their own passwords; the

The supplied passwords were eight characters long and were taken

from the character set consisting of lower-case letters and digits.

pseudo-random number generator with only 215 starting values.

They were generated by a

The time required to search

(again on a PDP-11/70) through all character strings of length 8 from a 36-character alphabet
is 112 years.

Unfortunately, only 215 of them need be looked at, because that is the number of possi-

ble outputs of the random number generator. The bad guy did, in fact, generate and test each
of these strings and found every one of the system-generated passwords using a total of only
about one minute of machine time.
IMPROVEMENTS TO THE FIRST APPROACH
1.

Slower Encryption
Obviously, the first algorithm used was far too fast.

The announcement of the DES

encryption algorithm [2] by the National Bureau of Standards was timely and fortunate.

The

DES is, by design, hard to invert, but equally valuable is the fact that it is extremely slow
when implemented in software.

The DES was implemented and used in the following way:

The first eight characters of the user’s password are used as a key for the DES; then the algorithm is used to encrypt a constant.

Although this constant is zero at the moment, it is easily

accessible and can be made installation-dependent.

Then the DES algorithm is iterated 25

times and the resulting 64 bits are repacked to become a string of 11 printable characters.
2.

Less Predictable Passwords

The password entry program was modified so as to urge the user to use more obscure

passwords.

If the user enters an alphabetic password (all upper-case or all lower-case) shorter

than six characters, or a password from a larger character set shorter than five characters,

Password Security

then the program asks him to enter a longer password.

4-11

This further reduces the efficacy of

key search.

These improvements make it exceedingly difficult to find any individual password. The
user is warned of the risks and if he cooperates, he is very safe indeed. On the other hand, he
is not prevented from using his spouse’s name if he wants to.
3. Salted Passwords
The key search technique is still likely to turn up a few passwords when it is used on a
large collection of passwords, and it seemed wise to make this task as difficult as possible. To
this end, when a password is first entered, the password program obtains a 12-bit random
number (by reading the real-time clock) and appends this to the password typed in by the
user. The concatenated string is encrypted and both the 12-bit random quantity (called the
salt) and the 64-bit result of the encryption are entered into the password file.

When the user later logs in to the system, the 12-bit quantity is extracted from the password file and appended to the typed password. The encrypted result is required, as before, to
be the same as the remaining 64 bits in the password file. This modification does not increase
the task of finding any individual password, starting from scratch, but now the work of testing
a given character string against a large collection of encrypted passwords has been multiplied
by 4096 (212). The reason for this is that there are 4096 encrypted versions of each password
and one of them has been picked more or less at random by the system.
With this modification, it is likely that the bad guy can spend days of computer time
trying to find a password on a system with hundreds of passwords, and find none at all. More
important is the fact that it becomes impractical to prepare an encrypted dictionary in
advance. Such an encrypted dictionary could be used to crack new passwords in milliseconds
when they appear.

There is a (not inadvertent) side effect of this modification. It becomes nearly impossible to find out whether a person with passwords on two or more systems has used the same
password on all of them, unless you already know that.
4. The Threat of the DES Chip

Chips to perform the DES encryption are already commercially available and they are
very fast. The use of such a chip speeds up the process of password hunting by three orders
of magnitude. To avert this possibility, one of the internal tables of the DES algorithm (in
particular, the so-called E-table) is changed in a way that depends on the 12-bit random
number. The E-table is inseparably wired into the DES chip, so that the commercial chip
cannot be used. Obviously, the bad guy could have his own chip designed and built, but the
cost would be unthinkable.
5. A Subtle Point

To login successfully on the UNIX system, it is necessary after dialing in to type a valid
user name, and then the correct password for that user name. It is poor design to write the
login command in such a way that it tells an interloper when he has typed in a invalid user
name. The response to an invalid name should be identical to that for a valid name.

When the slow encryption algorithm was first implemented, the encryption was done
only if the user name was valid, because otherwise there was no encrypted password to compare with the supplied password. The result was that the response was delayed by about
one-half second if the name was valid, but was immediate if invalid. The bad guy could find
out whether a particular user name was valid. The routine was modified to do the encryption
in either case.

4-12 Password Security

CONCLUSIONS
On the issue of password security, UNIX is probably better than most systems.

The use

of encrypted passwords appears reasonably secure in the absence of serious attention of
experts in the field.
It is also worth some effort to conceal even the encrypted passwords.

Some UNIX sys-

tems have instituted what is called an “external security code” that must be typed when dialing into the system, but before logging in.

If this code is changed periodically, then someone

with an old password will likely be prevented from using it.
Whenever any security procedure is instituted that attempts to deny access to unauthor-

ized persons, it is wise to keep a record of both successful and unsuccessful attempts to get at
the secured resource.

Just as an out-of-hours visitor to a computer center normally must not

only identify himself, but a record is usually also kept of his entry.

Just so, it is a wise pre-

caution to make and keep a record of all attempts to log into a remote-access time-sharing
system, and certainly all unsuccessful attempts.
Bad guys fall on a spectrum whose one end is someone with ordinary access to a system

and whose goal is to find out a particular password (usually that of the super-user) and, at the
other end, someone who wishes to collect as much password information as possible from as
many systems as possible.

Most of the work reported here serves to frustrate the latter type;

our experience indicates that the former type of bad guy never was very successful.
We recognize that a time-sharing system must operate in a hostile environment.

We did

not attempt to hide the security aspects of the operating system, thereby playing the cus-

tomary make-believe game in which weaknesses of the system are not discussed no matter
how apparent.

Rather we advertised the password algorithm and invited attack in the belief

that this approach would minimize future trouble. The approach has been successful.
References

[1]

Ritchie, D.M. and Thompson, K.

The UNIX Time-Sharing System.

Comm. ACM 17

Proposed Federal Information Processing Data Encryption Standard.

Federal Register

(July 1974), pp. 365-375.
[2]

(40FR12134), March 17, 1975

[3]

Wilkes, M. V.

Time-Sharing Computer Systems. American Elsevier, New York, (1968).

[4]

U.S. Patent Number 2,089,603.

Introduction 5-1

PART 5: SUPPORTING DOCUMENTS

Of the seven articles in this part, only the first is current and directly applicable to the
“Changes to the Kernel in 4.2BSD,” by LefHler, describes the differences

ULTRIX-32 system.

between the 4.1BSD and 4.2BSD software distributions.
4.2BSD.

The ULTRIX-32 system is based on

Be sure to use this article if you are converting an existing UNIX 4.1BSD system to

ULTRIX-32. The information provided covers:
Revising local software so that it is compatible with 4.2BSD

New directory organization
-

Machine-independent code

Machine-dependent code

New file names

Changes to old files
Changes to stand-alone and bootstrap utilities
The six remaining articles in this part are included for historical reasons. They provide background information, some of which you may find useful in installing or tuning your ULTRIX-

32 system. However, much of the information in these articles is obsolete. Note in particular
that the articles on uucp do not refer to the current implementation. Check the ULTRIX-32
Installation Manual and the ULTRIX-32 System Management Guide for installation and
maintenance procedures.

Changes to the Kernel 5-3

Changes to the Kernel in 4.2BSD
July 25, 1983
Samuel J. Leffler

Computer Systems Research Group
Department of Electrical Engineering and Computer Science
University of California, Berkeley
Berkeley, California 94720
(415) 642-7780

This document summarizes the changes to the kernel between the September 1981

4.1BSD release and the July 1983 4.2BSD distribution. The information is presented in both

overall terms (e.g. organizational changes), and as specific comments about individual files.
See the source code itself for more details.
The system has undergone too many changes to detail everything.

Instead the major

areas of change will pointed out, followed by a brief description of the contents of files present

in the 4.1BSD release. Where important changes and/or bug fixes were applied they are
The networking support is not discussed in this document, refer to “4.2BSD Networking Implementation Notes” for a discussion of the internal structure of the network facildescribed.

ities.

Major changes include:
e

organizational changes to isolate VAX specific portions of the system

changes to support the new file system organization

changes to support the new interprocess communication facilities

changes for the new networking support; in particular, the DARPA standard Internet protocols TCP, UDP, IP, and ICMP, and the network interface drivers which provide
hardware support
e

changes for the new signal facilities

changes for the new time and interval timer facilities

changes to eliminate references to global variables; in particular, the global variables
u.u base, u.u offset, u.u segflg, and u.u count have been almost completely replaced by uio
structures which are passed by reference; the u.u error variable has not been completely
purged from low level portions of the system, but is in many places now returned as a
function value; the uio changes were necessitated by the new scatter-gather i/o facilities

changes for the new disk quota facilities

changes for more flexible configuration of the disk space used for paging and swapping

1. Carrying over local software

With the massive changes made to the system, both in organization and in content, it
may take some time to understand how to carry over local software. The majority of this
document is devoted to describing the contents of each important source file in the system. If
you have local software other than device drivers to incorporate in the system you should first
read this document completely, then study the source code to more fully understand the
changes as they affect you.

5-4 Changes to the Kernel
Locally written device drivers will need to be converted to work in the new system.

The

changes required of device drivers are:

The calling convention for the driver ioctl routine has changed.

Any data copied in or

out of the system is now done at the highest level inside ioctl(). The third parameter to
the driver roct! routine is a data buffer passed by reference. Values to be returned by a
driver must be copied into the associated buffer from which the system then copies them
into the user address space.

The read, write, and ioct! entry points in device drivers must return O or an error code

The read and write entry points should no longer reference global variables out of the

from <errno.h>.

user area. A new uio parameter is passed to these routines which should, in turn, be
passed to the physio() routine if the driver supports raw i/o.

Disk drivers which are to support swapping/paging must have a new routine which

returns the size, in sectors, of a disk partition.

This value is used in calculating the size

of swapping/paging areas at boot time.
Code which previously used the iomove, passc, or cpass routines will have to be modified

to use the new uiomove, ureadc, and uwritec routines.

The new routines all use a uio
structure to communicate the i/o base, offset, count, and segflag values previously passed
globally in the user area.
|
Include files have been rearranged and new ones have been created.

Common machine-

dependent files such as mtpr.h, pte.h, reg.h, and psl.h are no longer in the “h” directory;
see below under organizational changes.
The handling of UNIBUS resets has changed. The reset routine should no longer deallo-

cate UNIBUS resources allocated to pending i/o requests (this is done in the ubareset
routine).

For most drivers this means the reset routine simply needs to invalidate any

ub info values stored in local data structures to insure new UNIBUS resources will be

allocated the next time the “device start” routine is entered.
2. Organizational changes
The directory organization and file names are very different from 4.1BSD.

The new

directory layout breaks machine-specific and network-specific portions of the system out into
separate directories.
machine, e.g. vax.

A new file, machine is a symbolic link to a directory for the target

This allows a single set of sources to be shared between multiple machine

types (by including header files as “../machine/file”’). The directory naming conventions, as
they relate to the network support, are intended to allow expansion in supporting multiple
“protocol families”.

The following directories comprise the system sources for the VAX:

/sys/h

machine independent include files

/sys/sys

machine independent system source files

/sys/conf
/sys/net
/sys/netinet
/sys/netimp
/sys/netpup

site configuration files and basic templates
network independent, but network related code
DARPA Internet code
IMP support code
PUP-1 support code

/sys/vax

VAX specific mainline code

/sys/vaxif
/sys/vaxmba

VAX network interface code
|
VAX MASSBUS device drivers and related code

/sys/vaxuba

VAX UNIBUS device drivers and related code

Files indicated as machine independent are shared among 4.2BSD systems running on

the VAX and Motorola 68010. Files indicated as machine dependent are located in directories indicative of the machine on which they are used; the 4.2BSD release from Berkeley
L
v

Wt e

i N

o o

e s

Changes to the Kernel 5-5

contains support only for the VAX. Files marked network independent form the “core” of
the networking subsystem, and are shared among all network software; the 4.2BSD release
from Berkeley contains complete support only for the DARPA Internet protocols IP, TCP,
UDP, and ICMP.

3. Bug fixes and changes

This section contains a brief description of each file which is not part of the network
subsystem, and also indicates important changes and bug fixes applied to the source code contained in the file.

3.1. /sys/h

Files residing here are intended to be machine independent. Consequently, the header
files for device drivers which were present in this directory in 4.1BSD have been moved to

other directories; e.8. /sys/vaxuba. Many files which had been duplicated in /usr/include are

now present onlyiin /sys/h. Further, the 4.1BSD /usr/include/sys directoryis now normally a

symbolic link to this directory. By having only a single copy of these files the° ‘multiple
update” problem no longer occurs. (It is still possible to have /usr/include/sys be a copy of
the /sys/h for sites where it is not feasible to allow the general user community access to the
system source code.)

The following files are new to /sys/h in 4.2BSD:

domain.h

describes the internal structure of a communications domain; part of the new
ipc facilities

errno.h

had previously been only in /usr/include; the file /usr/include/errno.h is now a

fs.h

replaces the old filsys.h description of the file system organization
describes various data structures used in profiling the kernel; see gprof(1) for

gprof.h

symbolic link to this file

details

kernel.h

mbuf.h

mman.h

nami.h

is an offshoot of systm.h and param.h; contains constants and definitions

related to the logical UNIX “kernel”

describes the memory managment support used mostly by the network; see
“4,9BSD Networking Implementation Notes” for more information
contains definitions for planned changes to the memory management facilities
(not implemented in 4.2BSD)

defines various structures and manifest constants used in conjunctions with

the namei routine (part of this file reflects future plans for changes to name:
rather than current use)

protosw.h

contains a description of the protocol switch table and related manifest constants and data structures use in communicating with routines located in the
table

quota.h

resource.h
socket.h
socketvar.h

ttychars.h

contains definitions related to the new disk quota facilities
contains definitions used in the getrusage, getrlimit, and getpriority system
calls (among others)

contains user-visible definitions related to the new socket ipc facilities
contains implementation definitions for the socket ipc facilities

contains definitions related to tty character handling; in particular, manifest
constants for the system standard erase, kill, interrupt, quit, etc. characters

are stored here (all the appropriate user programs use these manifest

definitions)

5-6 Changes to the Kernel

ttydev.h

contains definitions related to hardware specific portions of tty handling (such
as baud rates); to be expanded in the future

uio.h

contains definitions for users wishing to use the new scatter-gather i/o facilities; also contains the kernel uio structure used in implementing scattergather i/o

un.h

contains user-visible definitions related to the “unix” ipc domain

unpcb.h

contains the definition of the protocol control block used in the “unix” ipc
domain

walit.h

contains definitions used in the wait and wait3(2) system calls; previously in
/usr/include/wait.h

The following files have undergone significant change:
buf.h

reflects the changes made to the buffer cache for the new file system organiza-

tion — buffers are variable sized with pages allocated to buffers on demand
from a pool of pages dedicated to the buffer cache; one new structure member
has been added to eliminate overloading of a commonly unreferenced struc-

ture member; a new flag

B CALL, when set, causes the function b iodone to
be called when i/o completes on a buffer (this is used to wakeup the pageout
daemon); macros have been added for manipulating the buffer queues, these
replace the previous subroutines used to insert and delete buffers from the

queues

conf.h

reflects changes made in the handling of swap space and changes made for the
new select (2) system call; the block device table has a new member, d psize,
which returns the size of a disk partition, in sectors, given a major/minor
value; the character device table has a new member, d select, which is passed
a dev t value and an FREAD (FWRITE) flag and returns 1 when data may be
read (written), and O otherwise; the swdevt structure now includes the size, in
sectors, of a swap partition

dir.h

completely

different

since

See also specific commands

pages with a macro, GEN

reference list, PGM 2-398

5-T5K

Command (Mail)

specifying in text file, GEN 5-6

See also specific commands

starting, GEN 5-35

reference list, GEN 2-28 to 2-33,

text formatting commands for
double columns, GEN 5-15E,
5-35

Comma character (ed)
compared with semicolon, GEN
3-45

COMMA operator (C compiler)
defined, PGM 2-66
Command (Bourne shell)

2-39T

Command (make)

defined, PGM 3-16
Command (nroff)

description, GEN 5-22 to 5-25
Command (nroff/troff)
See also specific commands
reference list, GEN 5-51
Command (vi)

See also specific commands

grammar, GEN 4-26

case and, GEN 3-59

grouping, GEN 4-14

Command (C shell)
See also Program

See also specific commands
defined, GEN 4-64
reference list, GEN 4-63 to 4-74

regenerating, SYS 5-118
repeating, GEN 4-41 to 4-43,
4-51E

substituting output for, GEN
4-61E

suspending temporarily, GEN
4-36

terminating, GEN 4-35 to 4-38
typing, GEN 2-4

within quotation marks, GEN
4-60

Command (DC)

See also specific commands

ex 3.5 changes and, GEN 3-103
for file manipulation, GEN 3-71T

preceding counts and, GEN 3-70
Command file
description, GEN 1-29
Command line

running two programs with one,
GEN 2-11

Command line flag (Mail)

See Flag (Mail)
Command mode (ex)
defined, GEN 3-85
Command name
defined, GEN 4-64

Command procedure
See Shell procedure
Command substitution

See also Modifier (C shell)
defined, GEN 4-65

for human use

Index-11

Command-list

Configuration file (Cont.)

specifying multiple bootable

defined, GEN 4-8
grouping commands, GEN 4-14

images, SYS 5-80
syntax, SYS 5-79 to 5-83

Comment (awk)

VAX-11/780 sample, SYS 5-84 to

defined, PGM 3-9

Comment (BC)

5-87

convention, GEN 2-49, 2-50

connect system call
datagram sockets and, SYS 3-10

Comment (ex)

errors, SYS 3-8

description, GEN 3-86

establishing connection between

Comment (nroff/troff)

sockets, SYS 1-10

specifying, GEN 5-67

initiating connection, SYS 3-8E

Communication domain
defined, SYS 3-6

Connect time accounting
summarizing, SYS 5-56

Component
defined, GEN 4-65

Connection

Compound statement (BC)

accepting, SYS 3-9K
receiving, SYS 3-8 to 3-9

forming, GEN 2-54

Computer-aided instruction

Constant (BC)

See CAI scripts

defined, GEN 2-50

comsat program

Context search (ed)

4.2BSD improvement, SYS 1-19
CON operator (C compiler)
defined, PGM 2-66

backslash character and, GEN
3-43

defined, GEN 3-35
methods, GEN 3-30 to 3-31

Conditional

See if/endif commands

question mark character and,
GEN 3-43

conf.c file

4.2BSD improvement, SYS 5-14

repeating a search, GEN 3-31

installing device driver and, SYS

reverse order and, GEN 3-31

slashes and, GEN 3-39

5-119

Context search (edit)

conf.h file
4.2BSD improvement, SYS 5-6

d command and, GEN 3-16

4.2BSD improvement, SYS 1-19

delete command and, GEN 3-16C
move command and, GEN 3-15

adding nonstandard system

repeating, GEN 3-20E

facilities, SYS 5-96

reversing, GEN 3-20

config program

defined, SYS 5-73

description, SYS 5-73 to 5-105
device defaults, SYS 5-99 to 5-100
files generated by, SYS 5-76
modifying system code, SYS 5-88
modifying system configuration,

SYS 5-76
prerequisite information, SYS

specifying options items, SYS

defined, PGM 3-9

Control character (C shell)
defined, GEN 4-65

Control character (nroff/troff)
commands and, GEN 5-56
Control character (vi)
in text file, GEN 3-61

5-75

Configuration clause
description, SYS 5-80
Configuration file
SYS 5-76

creating, SYS 5-76
grammar, SYS 5-97 to 5-98

specifying devices, SYS 5-81

Index-12

defined, GEN 4-65
continue statement (awk)

changing, GEN 5-67

5-74

profiled systems and, SYS 5-78

contents,

s command and, GEN 3-20
continue command (C shell)

Control statement (BC), GEN
2-47E

description, GEN 2-47 to 2-48

Cooper, E., & others
4.2BSD System Manual, PGM
4-15 to 4-52

CSPACE operator (C compiler)

copy command (C shell)
See cp command (C shell)

defined, PGM 2-64
css network driver

copy command (edit)

See co command (edit)

4.2BSD improvement, SYS 1-15

copy command (ex)

ctags

4.2BSD improvement, SYS 1-5

See co command (ex)
copy command (Mail)

ctime library

See also save command (Mail)
description, GEN 2-29

4.2BSD improvement, SYS 1-14
CTRL-B

using, GEN 2-23E

defined, GEN 3-75

description, GEN 3-56

copy program

loading, SYS 5-24E

CTRL-C

mini-root file system and, SYS

ULTRIX-32 and, GEN 2-1
CTRL-D

5-24

Core dump file