Digital PDFs

AA-L669A-TC

April 1983

177 pages

Original

7.3MB

Document:	RSX-11M/M-PLUS RMS-11 User’s Guide
Order Number:	AA-L669A-TC
Revision:	0
Pages:	177
Original Filename:	http://bitsavers.org/pdf/dec/pdp11/rsx11m_s/RSX11M_V4.1_Apr83/7_RMS11/AA-L669A-TC_rms11ug_198304.pdf

OCR Text

RSX-11 M/M-PLUS
RMS-11 User's Guide
Order No. AA-L669A-TC

April 1983
This manual provides information on file and task design using
RMS-11. The information includes design considerations for writing
application programs in both MACR0-11 and high-level languages.

SUPERSESSION/UPDATE INFORMATION:

This revised document
supersedes the RMS-11
User's Guide {Order No.
AA-D538A-TC).

OPERATING SYSTEM AND VERSION:

RSX-11 M Version 4.1,
RSX-11 M-PLUS Version 2.1

SOFTWARE VERSION:

RMS-11 Version 2.0

digital equipment corporation · maynard, massachusetts

First Printing, April 1983
The information in this document is subject to change without notice
and should not be construed as a commitment by Digital Equipment
Corporation. Digital Equipment Corporation assumes no responsibility
for any errors that may appear in this document.
The software described in this document is furnished under a license
and may be used or copied only in accordance with the terms of such
license.
No responsibility is assumed for the use or reliability of software on
equipment that is not supplied by Digital Equipment Corporation or its
affiliated companies.
Copyright

@ 1983 by Digital Equipment Corporation
All Rights Reserved.
Printed in U.S.A.

The postpaid READER'S COMMENTS form on the last page of this document
requests the user's critical evaluation to assist in preparing future
documentation.
The following are trademarks of Digital Equipment Corporation:
DIBOL
EduSystem
IAS
MAS SB US
PDP
PDT
RSTS

DEC

DEC/CMS
DEC/MMS
f"'\'Clrt..,...,....,.i...

LI .WVLl'C' '-

DECsystem-10
DECSYSTEM-20
DEC US
DECwriter

RSX
UNIBUS
VAX
VMS

mnmnomo
ZK2168

HOW TO ORDER ADDITIONAL DOCUMENTATION
In Continental USA and Puerto Rico call

800-258-1710

In New Hampshire, Alaska, and Hawaii call
In Canada call

603-884-6660

613-234-7726 (Ottawa-Hull)
800-267-6146 (all other Canadian)

DIRECT MAIL ORDERS (USA & PUERTO RICO)*
Digital Equipment Corporation
P.O. Box CS2008
Nashua, New Hampshire 03061

DIRECT MAIL ORDERS (CANADA)
Digital Equipment of Canada Ltd.
940 Belfast Road
Ottawa, Ontario K1 G 4C2
Attn: A&SG Business Manager

DIRECT MAIL ORDERS (INTERNATIONAL)
Digital Equipment Corporation
A&SG Business Manager
c/o Digital's local subsidiary or
approved distributor

*Any prepaid order from Puerto Rico must be placed
with the iucai Digital subsidiary (809-754-7575)
Internal orders should be placed through the Software Distribution Center (SDC). Digital Equipment
Corporation, Northboro, Massachusetts 01532

CONTENTS

Page
PREFACE
MANUAL OBJECTIVES
• • • •
INTENDED AUDIENCE • • • •
STRUCTURE OF THIS DOCUMENT • • • • • • • •
ASSOCIATED DOCUMENTS • •
• • • •
CONVENTIONS USED IN THIS DOCUMENT
SUMMARY OF TECHNICAL CHANGES • • • • • • • • • • • • • • • • • •
CHAPTER l
1.1
1.1.1
1.1.2
1.1.3
1.1.4
1.1.5
1.2
1. 2 .1
1. 2. 2
1. 2. 3
1. 2. 4
1.3
1. 3 .1
1. 3. 2
1. 3. 3
1.4
1.5

CHAPTER 2

ix
ix
ix

x
• • x
xiii

RMS-11 CONCEPTS AND PROCESSING ENVIRONMENT
CONCEPTS OF DATA ORGANIZATION AND ACCESS • e • • • 1-1
Records
• • • •
• • • • • • • • 1-1
Files
• • • • • • • • 1-2
Access • •
• • • • • • •
• • • • 1-7
Processing •
• • • • • • • • • • • • 1-10
File Maintenance • • • • • • • • • • • •
1-10
RMS-11 IMPLEMENTATION OF DATA ORGANIZATION AND
ACCESS • • • • • • • • •
• • • • • • • 1-11
RMS-11 Record Formats
• • • • • • • 1-11
RMS-11 File Organizations
1-11
RMS-11 Record Access Modes
1-12
RMS-11 Utilities • • • • • •
1-12
RMS-11 PROCESSING ENVIRONMENT
1-12
RMS-11 Task Structure
• • • •
1-13
RMS-11 Record Processing •
1-15
RMS-11 File Processing • •
1-16
FILE ATTRIBUTES
• • • • •
1-18
PROCESSING BY BLOCK ACCESS •
1-20
APPLICATION DESIGN

2.1
WHEN TO DESIGN • •
DESIGN CONSIDERATIONS
2.2
2.2.1
Speed
• • • • • • • •
2.2.2
Space • • • •
n.::1+-::1
~+-nr.::1n~
2.2.2.1
---- -----J....... .
2.2.2.2
Task Size
•••••
Buffer Sizes •
• •••
2.2.2.3
Shared Access
• • • . • • •
2.2.3
Bucket Locking •
• •••
2.2.3.l
Sharing among Access Streams •
2.2.3.2
Programming Considerations •
2.2.3.3
Ease of Design • • • • • • • •
2.2.4
DESIGN PROCESS • • • • • • • • •
2.3
SELECTING A FILE ORGANIZATION •
2.4
2.4.1
Record Formats • • • •
• • •
2.4.1.1
Fixed-Length Format
Variable-Length Format •
2.4.1.2

iii

• 2-2
• • • • • • • • • 2-3
. • • • • • • . 2-3
• • 2-4
. 2-5
• • • • 2-5
• • • •
• 2-5
• • 2-5
• • • • • • • 2-8
• • • • • • • • • 2-9
• • • • 2-10
• • • • • • 2-10
2-11
• • •
2-11
2-15
• • • • • • • • 2-15
2-15

CONTENTS
2.4.1.3
2. 4. L 4
2.4.1.5
2.4.2
CHAPTER 3

Variable-with-Fixed-Control Format •

2-16
2-16
2-17
2-17

Stream Format

Undefined Format
I/O Techniques •
SEQUENTIAL FILE APPLICATIONS

3.1
FILE STRUCTURE •
• •
3.2
RECORD SIZE
• • • •
• • • • • •
FILE DESIGN
3.3
• • • •
• • • •
3.3.1
Data Storage Medium
• • • • • • • •
File Allocation
3.3.2
• • • •
• •
3.3.2.1
Initial Allocation •
• • • • • • •
3.3.2.2
Default Extension Quantity •
• •
3.3.3
Contiguity • • • • • • • •
3.4
ACCESS SHARING • • • • • • • • • • • • •
3.4.1
Record Access to Sequential Files
•••• •
Block Access to Sequential Files •
3.4.2
•
3.5
RECORD AND FILE PROCESSING OF SEQUENTIAL FILES
3.5.1
Record and Stream Operations •
3.5.1.1
CONNECT
• • • • • • • • •
• • • • •
3.5.1.2
DISCONNECT
• • • •
3.5.1.3
FIND •
FLUSH
3.5.1.4
•
3.5.1.5
GET
• • • • • • • • • • • • • • •
3.5.1.6
PUT
• • • • • • • •
• • • •
3.5.1.7
REWIND • • • • • •
• • • •
3.5.1.8
TRUNCATE • • • • • • • • • • • • • •
3.5.1.9
UPDATE • •
• • • • • • • • • •
3.5.2
Record Transfer Modes
Move Mode
3.5.2.1
3.5.2.2
Locate Mode
• • • • • • •
• • • • •
3.5.3
I/O Techniques • • • • • • • • • •
• • •
3.5.3.1
Asynchronous Record Operations • • • • • •
3.5.3.2
Deferred Write • • • • • •
3.5.3.3
Multiple Buffers • • • • • • •
3.5.3.4
Multiple Access Streams
••••
3.5.3.5
Multiblock Count • • • • • • •
3.5.4
File and Directory Operations
• • • • • • •
CHAPTER 4

• • 3-1
• • 3-2

• • 3- 3
•
•
•
•
•
•
•
•
•
•
•
•
•
•

•
•
•
•
•
•
•
•
•
•
•
•
•
•
•

3-3
3-4
3-4
3-4
3-5
3-5
3-6
3-6
3-6
3-7
3- 7
3-7
3-7
3-9
3-9
3-10
3-11
3-12
3-12
3-13
3-13
3-14
3-14
3-14
3-14
3-14
3-15
3-15
3-15

•

'"'2:-..L

•

RELATIVE FILE APPLICATIONS

4.1
FILE STRUCTURE •
RECORD SIZE
4.2
• •
FILE DESIGN
4.3
• • • • • • • • • • • • •
4.3.l
Bucket Size
• • • •
• • • •
4.3.2
File Allocation
• •
4.3.2.1
Initial Allocation • •
• • • •
4.3.2.2
Default Extension Quantity •
• •
4.3.3
Contiguity • • • • • • • •
4.3.4
Maximum Record Number
4.4
ACCESS SHARING • • • • • • • • • • • •
4.4.1
Record Access to Relative Files
4.4.2
Block Access to Relative Files • •
4.5
RECORD AND FILE PROCESSING OF RELATIVE
4.5.1
Record and Stream Operations •
4.5.1.1
CONNECT
• • • • • •
4.5.1.2
DELETE • •
4..5,L3
DISCONNECT , ,
4.5.1.4
FIND •
4.5.1.5
FLUSH
• • •
4.5.1.6
GET
• • • • •
4.5.1.7
PUT
• • • •
• •
REWIND •
4.5.1.8
• •
iv

•

• • • •
• • • •
•
• •
•
•
. • • •
•
•
• • • •
•

FILES

• • •
• • •
• • •

• • 4- 2
• • 4-2
• • 4-2
• 4-3
• • 4-3
• • 4-4
• • 4-4
• • 4-5
• • 4-6
• • 4-6
• • 4-6
• • • 4-6
• • 4-7
• • • 4-7
• • • 4-7
, 4-8
• • 4-8
• • 4-10
4-10
• • 4-11
• • 4-12

CONTENTS
4.5.1.9
4.5.2
4.5.2.1
4.5.2.2
4.5.3
4.5.3.1
4.5.3.2
4.5.3.3
4.5.3.4
4.5.4
CHAPTER 5

UPDATE • • • • • • • •
Record Transfer Modes
Move Mode
• • • •
Locate Mode
• . • • •
• • • •
I/0 Techniques • • • •
• • • • • •
Asynchronous Record Operations •
• • • • •
Deferred Write • • • • • • • • • • • •
Multiple Buffers • • • • • • • •
Multiple Access Streams
••••
File and Directory Operations

4-12
4-12
4-12
4-13
4-14
4-14
4-14
4-15
4-15
4-15

INDEXED FILE STRUCTURE AND ACCESS

PHYSICAL FILE STRUCTURE
• • 5-2
5.1
CONCEPTUAL FILE STRUCTURE
• 5-4
5.2
Data • • • • • . • • • • •
• • 5-5
5.2.1
Level 0 of the Primary Index •
• • • • • • • 5-5
5.2.1.1
Level 0 of an Alternate Index
• • • • 5-5
5.2.1.2
Indexes
• • • • • • • • • • • •
• • • • 5-6
5.2.2
Random Access Using the RMS-11 Indexed File
5.2.3
Structure
• • • • • • • • • • • • • • • • • • • 5-7
Why this Structure?
• • • • • • • • •
• • 5-8
5.2.4
PROCEDURES FOR PERFORMING RANDOM RECORD OPERATIONS 5-9
5.3
Writing a Record • • • • • • •
5-10
5.3.l
Simplest Case
• • • • • • •
5-10
5.3.1.1
Bucket Splitting • • • • •
• • • • •
5-11
5.3.1.2
Incremental Reorganization
• • • • • 5-12
5.3.1.3
Getting and/or Finding a Record
5-13
5.3.2
Updating a Record
• • • • • • • • • • • •
5-14
5.3.3
Deleting a Record
• • • • • • •
5-15
5.3.4
PROCEDURES FOR PERFORMING SEQUENTIAL RECORD
5~4
OPERATIONS • • • • • • • • • • • • •
5-16
I/O COST OF PERFORMING RECORD OPERATIONS • •
5-17
5.5
CHAPTER 6
6.1
6.2
6.2.1
6.2.2
6.2.2.1
6.2.2.2
6.2.2.3
6.2.2.4
6.2.2.5
6.2.2.6
6.2.3
6.2.4
6.2.5
6.2.5.1
6.2.5.2
6.2.5.3
6.3
6.4
6.5
6.5.1
6.5.2
6.5.3
6.6
6.6.1
6.6.2
6.7
6.7.1

INDEXED FILE DESIGN
RECORD SIZE
• • • • • • • • • • • • • • • • • 6-1
KEY SELECTION
• • • • • • • • 6-2
Number of Keys • • • • • • • •
• • • • • • 6-2
Key Data Types • • • • • • • • • •
• • • • 6-3
String Type
• • • • • •
• • • • 6-3
Two-Byte Signed Integer Type • • • • • • • • • • 6-4
Four-Byte Signed Integer Type
• • • • • • 6-4
Two-Byte Unsigned Binary Type • • • • • • • • • 6-5
Four-Byte Unsigned Binary Type •
• • • • • 6-5
Packed Decimal Type
• • • • • 6-6
Key Size • • • • • • • • • • •
• • • • • • • 6-6
Position of Key in Record
• • • • •
• 6-7
Key Characteristics
• • • • • • • • • 6-8
Duplicates • • . • • • • • • • •
• • • • • 6-8
Changes
• • • • • • • • • • • • • • • • • • • • 6-9
Null Key • •
• • • • •
6-10
AREAS
• • • • • • • • • •
• • • • . • • • • 6-10
PLACEMENT CONTROL
• • • • • • •
6-13
BUCKET SIZE
6-15
Bucket Size for Primary Index
6-16
Bucket Sizes for Alternate Indexes •
6-19
Program Syntax • • • • • •
• • • • • 6-21
FILE ALLOCATION • • • • • •
• • • • • • • • 6-22
Initial Allocation • • • • • •
6-22
Default Extension Quantity •
• • • • 6-26
POPULATION TECHNIQUES
• • • • •
6-26
Ascending Order by Primary Key •
6-27

CONTENTS

6.7.2
r

..,

'"'

0 . 1 • .G • .1.

6.7.2.2
CHAPTER 7
7.1

7 .1.1
7 .1. 2
7.2
7.2.1
7.2.2
7.2.3
7.2.4
7.2.5
7.2.6
7.2.7
7.2.8
7.2.9
7.3
7.3.1
7.3.2
7.4
7.4.1
7.4.2
7.4.3
7.4.4
7.4.5
7.5
CHAPTER 8

Random Insertions after File Population
Bucket Fill Size
Mass Insertion • • • • • • •

6-28
6-28
6-29

RECORD AND FILE PROCESSING OF INDEXED FILES
ACCESS SHARING • • • • • •
Record Access to Indexed Files • • • • • •
Block Access to Indexed Files
••••••
RECORD AND STREAM OPERATIONS
• • • •
CONNECT
.
DELETE •
DISCONNECT
FIND • •
FLUSH
GET • • • • •
PUT
. . • •
REWIND • • • • •
UPDATE •
RECORD TRANSFER MODES
• • •
Move Mode
• • • • • •
Locate Mode
• . • •
• • • • •
I/O TECHNIQUES • • • • •
• • • •
•
Asynchronous Record Operations • • • • • •
Deferred Write • • • • • •
• •••
Multiple Buffers • • • • • • • • • • • • • •
Multiple Access Streams
• • • •
Sequentially Reading Write-Shared Files
FILE AND DIRECTORY OPERATIONS
• • • • • • •

• • 7-1
7-1
7-1
• • 7-2
• • 7-2
7-3
7-3
• • 7-3
7-5
• 7-5
. • 7-5
• 7-6
• 7-6
• • 7-6

• 7-7
• 7-7
• • 7-8
7-8
• 7-8
• • 7-9
7-10
7-10
7-10

TASK BUILDING AND COMMON OPTIMIZATION TECHNIQUES

8.1
TASK BUILDING WITH RMS-11 ROUTINES • • • • • • •
Disk-Resident Overlays • • • • •
8 .1.1
ODL Files
•••••••••••••••
8.1.1.1
•
Memory-Resident Overlays • • • • •
8 .1. 2
Task Building against the RMS-11 Resident
8.1.2.1
Library
• • • • • • • • • •
• • • •
8.1.2.2
Using RMS-11 Operations from within Your Own
Resident Library • • • • • • • • • • . • •
Deciding Between Types of Overlays • • •
8.1.2.3
•
PROGRAM DEVELOPMENT
• • • • • • • • • • •
8.2
8.2.i
Flow of Operations Should Reflect RMS-11 Coda
Structure
• • • • • • • • • •
Task Builder Considerations
8.2.2
VIRTUAL-TO-LOGICAL-BLOCK MAPPING
8.3
8.3.1
Retrieval Pointers on Disk • •
Retrieval Pointers in Memory • •
8.3.2
8.3.3
Optimizing Window Turning
8.4
OTHER OPTIMIZATIONS • • • • •
8.4.1
Allocating More Resources to the Task
Disk Usage • • • • • • • • •
8.4.2
APPENDIX A
A.l
A.1.1
A. l. 2
A. l. 3
A.1.4
A. l . 5
A.2
A.2.1

• 8-1
8-3
• 8-5
• 8-6
• 8-6
• 8-8
• 8-9
• 8-9
8-10
8-10
8-11
8-11
8-11
8-12
8-13
8-13
8-13

FILE SPECIFICATION PARSING
STANDARD FILE SPECIFICATION SYNTAX •
• • • • . A-1
Device • • •
• • A-1
, , • • A= l
Directory
Name • •
• A-2
Type • • • • • • • •
• • A-3
Version
••••
• • A-3
ANSI MAGNETIC TAPE FILE SP.ECIFICATION SYNTAX • • • A-4
Dev ice • • . • •
• A-4
vi

CONTENTS
A.2.2
A.2.3
A.2.4
A.3
APPENDIX B
B.l
B.2
B.3

Directory
• • • •
Quoted String
••••
Version
• • • •
GENERATION OF A FULL FILE SPECIFICATION

•
•
•
•

A-4
A-5
A-5
A-5

REMOTE FILE AND RECORD ACCESS VIA DECNET
REMOTE NODE SPECIFICATION • • • •
REMOTE ACCESS ENVIRONMENTS • • • •
REMOTE ACCESS POOL CONSIDERATIONS

• • • B-2
B-3
• B-3

INDEX

FIGURES
FIGURE

1-1
1-2
1-3
1-4
1-5
1-6
1-7
1-8
1-9
2-1
2-2
2-3
2-4
3-1
4-1
5-1
5-2
5-3
5-4
5-5
5-6
6-1
6-2
6-3
6-4
7-1
8-1
8-2

Record Formats • • • • • • • • • • • • •
1-2
Files
• • • • • • • • • • •
1-3
Sequential File Organization • •
• • • •
• 1-4
Relative File Organization •
• • • • • • • 1-5
Indexed File Organization
• • • • •
• 1-6
Indexed File Example • • • • •
1-6
Record Access Modes
• 1-8
RMS-11 Task Structure • • • •
1-14
Records Spanning Blocks
1-17
Time Factors in an I/0 Operation • • • • • •
• 2-4
System Protection Concepts • •
• 2-6
Bucket Locking Example • • • •
• 2-9
Count Field on Disk and Tape • •
2-15
RMS-11 Task Structure • • • •
3-13
RMS-11 Task Structure • • • •
4-13
Indexed File with and without Areas
• • • • • • • 5-3
Formatted Bucket • •
• • • • • • • • • • . • 5-4
Index as a Pyramid •
• • • • • •
• • • 5-4
Format for Secondary Index Data Record
• • 5-6
Example of a Primary Index
• • • • •
• • • 5-7
Search Time Curves •
• • • •
5-9
Single-Area Indexed File •
• • • •
6-11
Example of Single-Area Indexed File
6-11
Two-Area Indexed File
• • • •
• • • • •
6-12
Example of Multi-Area Indexed File • • • • •
6-13
RMS-11 Task Structure
• • • • •
• 7-7
Source-to-Task Sequence • • • •
• • • • • • • 8-2
RMS-11 Tasks • • • • • • •
• • • 8-4

TABLES
TABLE

1-1
2-1
2-2
3-1
3-2
4-1
5-1
6-1

Record Formats and File Organizations
1-20
File Organization Characteristics and
Capabilities • • • • • • • • • • • • • •
2-12
File Organization Advantages and Disadvantages • 2-13
End-of-Block Indicators
• • • • • • • • • • • • • 3-2
Sequential File Data Sizes (in bytes)
• • • • • • 3-3
Relative File Data Sizes (in bytes)
• • • • • • • 4-2
I/O Cost of Performing Record Operations • •
5-18
Key Data Types • • • • • • • • • • • • • •
• • 6-6

vii

PREFACE

MANUAL OBJECTIVES
This document is a guide to using RMS-11 capabilities and operations
in file and task design for application programs written in either
MACR0-11 or high-level languages.

INTENDED AUDIENCE
This document is intended for application programmers who want to
achieve optimal performance with new applications they are writing or
with existing applications.
NOTE
Only MACR0-11 programmers can use the
full
set
of
RMS-11
capabilities.
Subsets of
these
capabilities
are
available
to
high-level
language
programmers.
See
your
high-level
language documentation to determine:
•

What RMS-11 facilities you can use in
your high-level language

•

The syntax for using these facilities

STRUCTURE OF THIS DOCUMENT
This manual contains eight chapters and two appendixes:
1, RMS-11 Concepts and
Processing
Environment,
• Chapter
introduces the concepts of data organization and access and

the RMS-11 implementation of these concepts.
•

Chapter 2, Application Design, presents general considerations
that apply to application design and information that will
help the application designer select a file organization.

•

Chapter 3, Sequential File Applications, discusses
file structure, design, and processing.

•

Chapter 4, Relative File Applications, discusses relative file
structure, design, and processing.

sequential

PREFACE
•

Chapters 5, Indexed File Structure and Access, 6, Indexed File
Design, and 7, Record and File Processing of Indexed Files,
discuss indexed file structure, design, and processing.

•

Chapter 8, Task Building and Common Optimization Techniques,
describes techniques that can be used to optimize application
programs that use RMS-11, regardless of the file organization
selected.

•

Appendix A, File Specification Parsing,
handling of file specifications.

•

Appendix B, Remote File and Record Access via DECnet, briefly
describes the remote access environment and remote file
specification syntax.

documents

RMS-ll's

ASSOCIATED DOCUMENTS
In addition to this user's guide,
contains the following manuals.

the

RMS-11

documentation

set

RSX-llM/M~PLUS

RMS~ll:
An Introduction presents the major concepts of
RMS-11,
introduces the~RMS-11 operations, and defines key terms
required for understanding RMS-11 capabilities and functions.
You
should read the introduction before proceeding to other manuals in the
RMS-11 documentation set.

The RSX-llM/M-PLUS RMS-11 Macro Programmer's Guide is a reference
document for MACR0-11 programmers that describes the macros and
symbols that make up the interface between a MACR0-11 program or
subprogram and the RMS-11 operation routines.
The RSX~llM/M-PLUS RMS-11 Utilities manual is both a user and a
reference document for all users, both programmers and nonprogrammers.
It describes the RMS-11 utilities that are available for creating and
maintaining RMS-11 files.
In addition, the Mini-Reference Insert includes an easy-reference
guide for users who are familiar with RMS-11 and its documentation.
It summarizes the RMS-11 utilities and error codes.

CONVENTIONS USED IN THIS DOCUMENT
Convention

Meaning

UPPERCASE

Uppercase words and letters, used in format examples,
indicate that you should type the word or letter
exactly as shown.

lowercase

Lowercase words and letters, used in format examples,
indicate that you are to substitute a word or value
of your choice.

quotation marks

The term "quotation marks" refers to double quotation
marks ( ") •

apostrophes

The term "apostrophe" refers to
mark (' ) •

single

quotation

PREFACE
[

]

Square brackets indicate that the
optional.
A horizontal ellipsis indicates
item(s)
can be repeated one
example:

enclosed

item

that the preceding
or more times. For

file-spec[,file-spec ••. ]
A vertical ellipsis indicates that not all of
statements in an example or figure are shown.
TKB>//

the

In examples of commands you enter
and
system
responses, all output lines and prompting characters
that the system prints or displays are shown in black
letters.
All the lines you type are shown in red
letters.

Unless otherwise noted, all numeric values are represented in
notation.
Unless otherwise specified, you terminate
RETURN key.

commands

decimal

pressing

the

SUMMARY OF TECHNICAL CHANGES

RMS-11 Version 2.0 supports random access to fixed-format disk
sequential files and sequential block access to disk files of any
format and organization.
The RMS-11 Version 2.0 resident libraries are task independent.
This
means that once a program is linked with this library, the library can
be rebuilt or replaced without requiring that the task linked to it be
rebuilt.
RMS-11 Version 2.0 contains no library equivalent to the RMS SEQ
memory-resident library included with RMS-11 Version 1.8. The RMS RES
resident library or the disk-resident ODL files can be used to obtain
equivalent functionality and performance.
New versions of the RMS-11 Version 1.8 ODL files are provided.
These
ODL files are:
RMSllS.ODL,
RMSllX.ODL, RMS12X.ODL, and RMSll.ODL.
The Version 1.8 ODL files will still work with Version 2.0, but the
new versions will be more efficient. RMS-11 Vl.8 ODL structures other
than RMSllS.ODL, RMSllX.ODL, and RMS12X.ODL may not work correctly
with the RMS-11 V2.0 code; when in doubt, verify them by comparison
with the V2.0 RMSll.ODL file.
In addition, two new ODL files are
provided with Version 2.0: RMS12S.ODL and DAPllX.ODL.
Files with stream and VFC records can now be created on unit-record
devices
to avoid the need for special-case code in copy-type
operations.
•

For VFC files, the record header is thrown
unless the file is a "print format" file.

away

output

•

For stream files, if none of the 3 carriage control bits is
set
(print file format, carriage control, or FORTRAN carriage
control), and if the last character is not a linefeed,
formfeed,
or
vertical tab,
the carriage-return/linefeed
(CR/LF) is appended at the end of the record.

•

For stream files, if either the carriage control or FORTRAN
carriage
control attribute is set, and if the last 2
characters of the record are CR/LF, the trailing CR/LF is
stripped off and then definition of the carriage control
attribute (CR or FTN) is applied.

For similar ease-of-copying reasons, RMS-11 now allows creation of
relative and indexed files for output to nondisk devices (for magtape,
however, the record format must be variable length or fixed length).
The RMS-11 File Design Utility (RMSDES) is a new utility that allows
you to design and create files interactively.
It is fully documented
in the RSX-llM/M~PLUS RMS-11 Utilities manual.
RMS-11 Version 2.0 supports five new directory operations:
$ENTER,
$PARSE, $REMOVE, $RENAME, and $SEARCH. These operations are fully
documented in the RSX-llM/M-PLUS RMS-11 Macro Programmer's Guide.

xiii

SUMMARY OF TECHNICAL CHANGES
RMS-11 Version 2.0 supports a new wildcard file specification facility
and a new print-record output handling format.
These are also fully
documented in the macro programmer's guide.
User-provided interlocks allow a special, limited form of sequential
file sharing among a group of accessors that includes at most one
read/write accessor and any number of read-only accessors.
If suitable DECnet facilities exist on your system and on the target
system, RMS-11 Version 2.0 will allow file and record access to files
on remote network nodes, if those nodes include an RMS-11-based file
access listener (FAL) •
For magtape, RMS-11 now allows fixed-format records to be less than 18
bytes.
Files with stream or VFC records can now be created on unit-record
devices.
In addition, RMS-11 now allows the creation of relative and
indexed files for output to nondisk devices, although they will be
treated as sequential files.
<CTRL/Z> and <ESC> are no longer recognized as record terminators for
stream files,
and <CTRL/Z> is no longer recognized as a file
terminator for stream files.
RMS-11 Version 2.0 pads stream files with null characters, to the high
block of the file (not just to the end of the current block).
The memory-resident library RMSRES can be clustered
resident library that supports clustering.

with

any

other

On RSX-llM-PLUS systems that include hardware support
mode, RMSRES can also be used in supervisor mode.

for

supervisor

On RSX-llM systems, an optional subset library, which contains support
for sequential and relative files only, is available.
NOTE
All new RMS-11 features
are
fully
accessible only to MACR0-11 prograrn.~ers.
See
your
high-level
language
documentation for supported features.

xiv

CHAPTER 1
RMS-11 CONCEPTS AND PROCESSING ENVIRONMENT

Your business, whether commercial, scientific, governmental,
or
educational, relies on data.
That data indicates the current state of
your business and helps you control the future of the business.
Therefore, you want fast, efficient access to the right data when you
need it.
You are familiar with dealing with data on paper and know that records
of transactions and reports on your business's activities can occupy a
very large number of file folders.
You also know that finding exactly
the data you need can be a time-consuming process.
Computer hardware, however, with its speed and mass data storage
capabilities, provides the means for fast, efficient access to data.
Computer software provides the means for translating the data from the
format you use to a format the computer system can handle -- and back
again.
RMS-11 is such a translater between you and your system. This chapter
introduces RMS-11 in terms of general concepts of data organization
and access, which apply regardless of whether data is stored on paper
or
within a computer's memorye
It then discusses the RMS-11
implementation of data organization and access, and the RMS-11 data
processing environment.

1.1

CONCEPTS OF DATA ORGANIZATION AND ACCESS

This section examines the general concepts of data organization, using
images from the noncomputer environment you may be most familiar with.

1.1.1

Records

When data is stored on paper, it is recorded in groups of items whose
form is repeated throughout the data.
Each group of items is called a
record. Within each record are the specific items of data you are
_,, ...l..llit:::
,__
information on an employee
concerned with.
For example, Q.L.L
constitutes a personnel record;
all the information on a stock item
constitutes an inventory record.
On paper, a record can be a form; different types of records require
different forms.
Some forms are always the same length;
their
information does not expand with time or use.
For example, a product
information form does not vary in size.
If the facts about a product
change, you fill out a new form.
If a new product is added, you also
fill out a new form.

1-1

RMS-11 CONCEPTS AND PROCESSING ENVIRONMENT
Other forms vary in length with time and use, continuing on to new
pages as they grow.
For example, an employee with the company for 10
years has more data in his or her personnel record than a new
employee.
Other forms might use a combination of these two formats.
For
example,
a record of service on a piece of equipment might begin with
control information describing the specific piece of equipment
(name,
model number, date of installation, and so on) and continue on to new
pages documenting the service performed on it.
Figure 1-1 illustrates various record formats.

PRODUCT RECORD

SALARY HISTORY

E.C.O. HISTORY

- ----.....
----~

-_......._. ..........

END

ZK-1170-82

Figure 1-1: Record Formats

1.1.2

Files

When data is stored on paper records,
it is usually gathered into
files and stored physically in filing cabinets, organized by related
records. For example, all employee records might be stored in one
file and placed in one drawer of the filing cabinet.
A file not only keeps related data in one place,
that data from other, unrelated data.

also

segregates

As data grows,
the file and storage requirements become
more
complicated,
and the number of filing cabinets multiplies.
Then, the
files acquire names or numbers, the drawers acquire signs indicating
the
contents
of
the
drawers
and
who
may use them, and
cross-referencing systems are introduced to help locate data.
These
identifying characteristics and restrictions upon who may read or
alter specific files can be called attributes.

1-2

RMS-11 CONCEPTS AND PROCESSING ENVIRONMENT

Figure 1-2 illustrates data storage using filing cabinets.

ZK-1167-82

Figure 1-2: Files
In general, the person who uses a
file establishes a method of
organizing the records within it.
This method reflects the file's use
and dictates what information is needed and how much time is required
to locate a record within the file.
There are several typical methods for organizing records in a
file,
depending on how the records are used.
If you generally use all the
records in a file whenever you open it (that is, you have little or no
need to locate individual records in the file) and the order of the
records is not important, then you can organize
the
records

sequentially:
•

The records assume the physical sequence
in which they are
inserted
into the file (that is, records are appended to the
file) •

•

No empty spaces are left in the sequence of records, where
records could be inserted later.
Each record, except the
first, has a record before it;
each record, except the last,
has a record following it.

1-3

RMS-11 CONCEPTS AND PROCESSING ENVIRONMENT
Employee payroll records, for example, might be kept in a sequential
file.
Because all the records must be accessed every time the payroll
is done, sequential file organization would allow easy access to the
records.
The overhead and maintenance for sequential files
is minimal.
To
insert a record into the file, you simply put it after the last record
already there. Figure 1-3 illustrates sequential file organization.

ZK-1168-82

Figure 1-3: Sequential File Organization
For more access flexibility than sequential files, if you want to be
able to locate individual records easily, you can set up a series of
file folders and number them in sequence from first folder to last.
Each folder is the same size;
it holds only one record, but it can be
empty. Thus, you do not have to look sequentially through the records
to locate the one you want (although you can if you want to access all
the records) • You use the numbers on the folders to locate or insert
records;
each record will be numbered relative to the beginning of
the file.
The numbers can relate to some numbering system meaningful
to your business:
for example, order numbers or part numbers.
Figure 1-4 illustrates relative file organization.

1-4

RMS-11 CONCEPTS AND PROCESSING ENVIRONMENT

ZK-1171-82

Figure 1-4: Relative File Organization
If you have a large file and most of the time you want to be able to
locate individual records, you may want to index your files.
Indexing
is useful when you want to be able to use several kinds of information
to locate records.
For example, in an employee file, you may want to
use last-name information to obtain a report on all employees, and
job-designation information to obtain a report on all clerical
employees.
When you open an indexed file drawer, you find records filed with
numbered tabs separating them. At the front of the drawer is a set of
small card files, containing groups of cards separated by dividers.
The cards in each of these small card files are an index to the
records at the back of the file.
To insert a record in the file,
you
find the data item marked "key" on the record, and using that
informationf consult the appropriate innPx to determine where the
record should be inserted.
Figure 1-5 illustrates indexed file
organization.
To find a record in an indexed file, you look for the specific key
information in the appropriate key file and use that information to
locate the record.
For example,
if you want the record of a
transaction with the Q,R,&S Company, you open the indexed file drawer
for transactions, which contains data records filed at the back and
indexes at the front.
Figure 1-6 illustrates this example.

1-5

RMS-11 CONCEPTS AND PROCESSING ENVIRONMENT

SEQUENTIAL
ACCESS

ZK-1169-82

Figure 1-5: Indexed File Organization

ZK-1175-82

Figure 1-6: Indexed File Example

1-6

RMS-11 CONCEPTS AND PROCESSING ENVIRONMENT

You know that company name is the primary key for records in the file
and that index 0 indexes the primary keys-.~The first record in index
O is the root, which lists selected primary key values, that is, the
company names,
in alphabetical order. Not all company names appear
here:
instead, a small subset of names, distributed fairly evenly
across the full set of names,
is used as the highest level of
indexing. By selecting one name; you establish the region of the file
(range of names) that interests you.
You look down the list until you find a name that either matches
Q,R,&S or occurs after this name in the alphabet.
You find Rhesus,
Inc with the number 3 alongside it.
You put the root record back in the file and go to the first divider
and the third index record behind it.
Again, the set of names here is
incomplete: only a small set of names distributed fairly evenly
across the range covered by the highest level index entry exists.
This provides an intermediate level of indexing, and further limits
the range of names in which you are interested.
Rhesus, Inc is the last entry on this card, but you scan the list and
find the name "Queeg Co, which is the first entry at or after Q,R,&S in
alphabetical sequence. The entry for Queeg Co has the number 7
alongside it.
So you reach into the data records at the back of the drawer to tab
number 7.
You search sequentially through the records behind this tab
until you find the record of the Q,R,&S transaction.
For another example, using the same transaction file, suppose you want
to find a record but all you know is its transaction number.
Fortunately, the second alternate key for the file
is transaction
number.
Index 2 indexes the second alternate keys (recall from the
previous example, that the indexes are numbered starting with primary
index 0). You look at the root record in index 2 and move through the
index as you did in the previous example until you find a card listing
the transaction number you are looking for.
Next to the number is the
code 7/5.
So you reach into the data records at the back of the drawer to tab
number 7 and count back to the fifth record behind the tab.
You find
that the transaction you are looking for was made with the Q,R,&S
Company.
Here, only one level of indexing -- the root record -- was used.
If
many records exist in the file, another intermediate level would also
be used, as it was in index O.
Use of intermediate index levels
allows the number of entries you must scan in each level to be small;
regardless of the total number of records in the file.

1.1.3

Access

Once you have records organized in a file, you
them in two ways:

can

get,

access,

•

You can search all the records one after the other.
called sequential access.

This

•

You can use an identifier to
This is called random access.

record.

locate

individual

Note that access means not only retrieving a record from
putting a record into the file as well.

1-7

file

but

RMS-11 CONCEPTS AND PROCESSING ENVIRONMENT
Figure 1-7 illustrates the random and sequential access modes.

ZK-1172-82

Figure 1-7: Record Access Modes

Sequential Access
For sequential access, you pick a point in the file and access the
records beyond that point one at a time. At times, the starting point
is the beginning of the file because you want to look at, or access,
each record in the file.
Other times, you may begin midway through
the file.
To read each record, you take it out of the file, marking the position
of the record you just removed with a card or some other marker so
that you know:
•

Where to put the record back into the file

•

Where the next record is

To insert records sequentially, you reach into the drawer to the place
where you want the records to go and mark the position of that place.
Often, the point at which you will insert the new records will be the
end of the file.
At other times, it may be midway through the file.

1-8

RMS-11 CONCEPTS AND PROCESSING ENVIRONMENT
You insert the records by taking the first record from the stack of
new records and slipping it into position in the file.
You then mark
the position after the record you just inserted and add the next
record in that position. You continue in this manner until all the
new records are inserted.

Note that in both retrieving and inserting records you move through
the records consecutively.
Each record is retrieved or inserted with
respect to the record accessed right before it.

Random Access
For random access, you determine the location of the record you want
on the basis of some identifier,
rather than on the basis of the
record's position within the file.
If, for example, you have a list
of locations of records in the file, you can reach into the file to a
record's exact location. Each record selection is independent of the
previously accessed record and of the next record to be accessed.
The record identifier can be a number, as for relative files,
or it
can be a key, as for
indexed files.
Or, the identifier can be a
physical location within the file drawer;
for example, you could
place each record in a numbered slot within the file drawer and use
the slot number to access the records in the file.
The slot number
would be the address of the record.
This type of random access could
be used with any type of file organization.
Often, you will want to switch the mode of access you use.
You may
want to use random access to find the first record in a series and
then use sequential access to retrieve all the records in that series.
For example, if your employee records are grouped by department codes
within the file, you can use a specific department code as the
identifier to randomly access the first record with that department
code and then switch to sequential access to consecutively read all
the records with that code.

Context
In either type of access, sequential or random,
the marking of
position in the file is important.
This is called context:
the
position of the record you are accessing is the current record, and
the position of the record that follows it is the next record.

Access Control
One advantage of the segregation of data provided by files
is
controlled access.
Some files, such as budget or payroll, should be
available to only a small group of authorized people.
Other files,
such as inventory or transaction files, may be used by larger groups
of people. And some files, such as the telephone directory, must be
accessible to everyone.
Files allow you to control who can use what data.
You can lock the
filing cabinet that contains the payroll data and give keys to
yourself and the payroll manager only.
And you can distribute
telephone directories to every employee.
In addition, within a file, you can further control how the data can
be used within the group of authorized users.
Some users may be
allowed to write new data in the file or to modify existing data,
while others may be allowed only to read the data.

1-9

RMS-11 CONCEPTS AND PROCESSING ENVIRONMENT
1.1.4

Processing

Once you locate, either sequentially or randomly, a record's position
within a file, you will probably want to do something with the record
that belongs there.
Record operations fall generally into the
following categories:
•

Verify that the record exists in the right location

•

Read the record;

•

Insert a record in the position that you have located

•

Revise the contents of the record;
its data contents

•

Remove the record from the file

1.1.5

that is, examine its data contents

that is,

modify

some

File Maintenance

Once you establish files and their records and begin using them
regularly, you will want to be able to maintain them to ensure both
the protection of the data within them and their continued usability.
Typically, maintenance might include the following activities.
•

The data in a file is valuable or you would not keep it.
You
should have duplicates of your records in some other place in
case something happens to the originals, Therefore, you need
the ability to back up files.

•

If something does happen to your original data, you
able to obtain, or restore, the duplicate records.

•

You need the ability to list, or display,
their names and other attributes.

•

Files often grow very large and their usage can change over
time.
Therefore, you may want to change a file's organization
from sequential to indexed; or you may want to reload a file
that has grown very large to use space more efficiently.
Conversely, usage and file size might decrease and you may
want to make a
file simpler.
It is also possible that the
information in one file is suitable for another application.
In all these cases, you would want to be able to convert a
file
into a new one, perhaps changing some
attributes
(including organization} to make it more usable.

•

You want to be able
require.

•

Creating an indexed file and putting records into it can be
complicated and time-consuming.
You would want a procedure
indexed file loading -- that would produce an optimal
indexed
file quickly and efficiently.

design

1-10

and

create

your

must

files,

files

that

be
with

you

RMS-11 CONCEPTS AND PROCESSING ENVIRONMENT
1.2

RMS-11 IMPLEMENTATION OF DATA ORGANIZATION AND ACCESS

RMS-11 provides file structure capabilities that allow you to organize
your data within a computer's memory using the same concepts that were
described in Section 1.1 for paper records in filing cabinets.
The following sections briefly present
capabilities.
For more details,
see
Introduction.

1.2.l

the

RMS-11

file

RSX-llM/M-PLUS

structure

RMS-11:

RMS-11 Record Formats

RMS-11 supports the following record formats that allow you to
the size of your data records:

define

•

Fixed length -- Every record in the file is the same size.

•

Variable length
Records
in the file are of different
lengths, up to a maximum size that you can optionally specifya

with fixed control -- Records in the file are of
• Variable
different
lengths, up to a maximum size that you can
optionally specify, and in addition,
area precedes the data.

fixed-length

control

a continuous stream of ASCII
special termina·tor character or

•

Stream -- Records consist of
characters delimited by a
sequence of characters.

•

Undefined -- Records in a file may have no record format or
may be in a
format different from the four standard RMS-11
formats.

RMS-ll's support of stream and undefined
limited support for non-RMS-11 files.

1.2.2

record

formats

provides

RMS-11 File Organizations

RMS-11 supports three file organizations:
•

Sequential -- Records are arranged within the
order in which they were written into the file.

•

Relative -- Records are stored in the file
in cells, or
fixed-length units of storage, one record per cell. lfhe-cells
are numbered sequentially.
These numbers, called relative
record numbers, are identifiers for the records.

•

Indexed -- Records are arranged in the file in ascending order
by key.
A key is a data field within the record that RMS-11
uses as an identifier to access the record.
An indexed file
must have one primary key and may optionally have other
alternate keys.

1-11

file

the

RMS-11 CONCEPTS AND PROCESSING ENVIRONMENT
1.2.3

RMS-11 Record Access Modes

RMS-11 provides three record access modes for storing
records in files:

retrieves

records

Sequential
RMS-11
stores
sequentially, one after another.

•

Random by key -- RMS-11 uses either a key
(for an indexed
file)
or a relative record number (for a relative file or for
a disk sequential file with fixed-length format records) as an
identifier to gain direct access to an individual record in
the file.

•

Random by record file address (RFA) -- RMS-11 uses the RFA as
an identifier to gain direct access to an individual record in
the file.
The RFA is a unique identifier that RMS-11
establishes for every record that it writes into a disk file.

RMS-11 Utilities

RMS-11 provides utility programs that can help you
record maintenance:

1.3

retrieving

•

1.2.4

and

perform

file

and

RMSBCK -- The RMS-11 File Back-Up Utility transfers the
contents of an RMS-11 file to another file, which may be on
another device, to maintain the file should the original file
be lost or damaged.

•

RMSRST -- The RMS-11 File Restoration Utility transfers files
that were backed up using RMSBCK back to you so your programs
can access them.

•

RMSDSP -- The RMS-11 File Display Utility produces a concise
description of any RMS-11 file, including back-up files.

•

RMSCNV -- The RMS-11 File Conversion Utility reads records
from an RMS-11 file of any organization and loads them into
another RMS-11 file of any organization.

•

RMSDES -- The RMS-11 File Design Utility allows you to
and create sequential, relative, and indexed files.

•

RMSIFL -- The RMS-11 Indexed File Load Utility reads records
from an RMS-11 file of any organization and loads them into an
indexed file.

design

RMS-11 PROCESSING ENVIRONMENT

The RMS-11 software routines organize data on
your
computer,
implementing the concepts discussed in the previous sections, and
provide the interface between your application programs and the
computer system.
Your computer system consists of layers of hardware and software:
@

The ha~dware devices -- magnetic tapes and disks -the data.

1-12

store

RMS-11 CONCEPTS AND PROCESSING ENVIRONMENT
•

The operating system software
file control processor,
device drivers -- controls the hardware to maintain files.

•

RMS-11 software controls the internal structure of
described in Section 1.2).

files

(as

•

Your application program makes use of these hardware
software facilities to process data records and files.

and

1.3.l

RMS-11 Task Structure

You use the RMS-11 software routines by combining them with a
you have written in a language that implements RMS-11.

program

NOTE
Only MACR0-11 programmers can use the
full
set
of
RMS-11
capabilities.
Subsets of
these
capabilities
are
available
to
high-level
language
programmers.
See
your
high-level
language documentation to determine:
•

Which RMS-11 facilities you
in your high-level language

•

The syntax for using these facilities

Once you write your program, you convert
either a compiler or an assembler.

can

use

object

code,

using

To combine your object code with the RMS-11 routines, you use the task
builder, which converts object code (modules) to an executable form
called a task.
In the process, the task builder not only combines
different object modules, but may also arrange the task so that some
executable modules overlay each other when the task is run.
You can combine RMS-11 routines with your object code in either of the
following ways:
routines

•

In the task itself, with
nonoverlaid
disk-resident overlay structure

•

In memory-resident overlays, a form apart from your task

between
these
techniques
is
that
The
primary
difference
memory-resident overlays can be shared among programs. Nonoverlaid
and disk-resident overlaid routines cannot be shared; each accessing
In addition,
program must have its own copy of such routines.
memory-resident overlays eliminate the I/0 operations needed to bring
disk-resident overlays from disk: thereby making your tasks run
significantly faster.
In either case, your task takes a logical form in which program code
exists in one part of the task and the RMS-11 routines run in another
part. When your program performs an RMS-11 operation, it sets up the
necessary parameters and data and calls the appropriate RMS-11
routine. Control jumps to that part of the task, the routine runs to
completion,
and
control returns to your program.
Figure 1-8
illustrates this logical structure.

1-13

RMS-11 CONCEPTS AND PROCESSING ENVIRONMENT

r;:;-------- - - - - - - - ,
I SIZE DEPENDS ON:

I• NUMBER OF FILES OPENED SIMULTANEOUSLY I
1 • BUCKET SIZES
I
S L'. NUMBER_9F__RECr-D~CCE~ "!:~MS _ _

USER BUFFER

--==:l

'
1/0
BUFFERS

VIRTUAL
MEMORY

RMS-11

PROGRAM

INTERNAL
CONTROL
STRUCTURES

fS1zEDEPENDs

ON:- -L ~

I• RMS-11 FUNCTIONS USED I
~ ~V~R~~~U~TURE_:1~E~
ZK-1174-82

Figure 1-8: RMS-11 Task Structure

Also part of the task are storage
three forms:

structures,

which

generally

take

User buffers -- These buffers are used to pass data records
• between
your program and RMS-11.
They are available to your

......

program and the data
in them can be manipulated,
changed, used for calculations, and so on.

read,

I/O buffers -- Fer each file your

RMS-11

program

has

open,

normally requires at least one internal I/O buffer.
All data
going to or coming from disk is stored in--an I/0 buffer as
follows:
RMS-11 requests the file control processor to move block(s)
from a disk file into this buffer to satisfy your program's
requirements.
Each request normally specifies the same
number of blocks, called an I/O unit.
The size of the I/O
unit depends on the file organization, file design,
and
settings at access time (such as multiblock count).
RMS-11 moves records between the I/O buffer and the user
buffer.
Your program can also directly access a record
within the I/O buffer in certain restricted circumstances.
:

Control structures
RMS-11 control structures,
called
control blocks, are used to communicate between your program
and the RMS-11 routines and with each other.
Some are
accessible to your program;
others are for RMS-11 internal
use only.

1-14

RMS-11 CONCEPTS AND PROCESSING ENVIRONMENT
1.3.2

RMS-11 Record Processing

The RMS-11 stream and record operations are the interface between your
program and the data records your program requires.
Before your program can access records in a file,
open and an access stream must be initiated.

the

file

must

NOTE
Most high-level languages do not support
access streams at the user level. They
use the RMS-11 access stream facilities
to
implement their own file access
techniques.
An access stream is a path to the file's data records;
record
operations are performed via that stream, one operation at a time.
RMS-11 keeps track of the stream's position, or context, in a file, in
terms of current record and next record.
The stream's position
changes at the completion of an operation.
Chapters 3, 4, and 7
discuss
context for record operations with the different file
organizations.
The stream operations control the stream associated with a file.
are:

They

•

CONNECT -- initiates an access stream.

•

DISCONNECT -- terminates a stream.

•

FLUSH -- writes the currrent contents of I/O
file.

•

FREE -- releases control of the record or block most
accessed (and locked) by the stream.

•

REWIND -- resets the stream context to the first record in the
file.

•

WAIT -- suspends processing until an outstanding
operation is completed.

The record operations process records within a file.

buffers

the

recently

asynchronous

They are:

•

FIND -- reads a record from a file to an I/O buffer
the current-record context to that record.

•

GET -- reads a record from a file to an I/O buffer and then to
a user buffer, and sets the current-record context to that
record.

•

PUT -- writes a new record from a user buffer to an I/O buffer
and then to a file.

•

UPDATE -- transfers a modified record from a user buffer to an
I/O buffer and then to a file, overwriting the previous copy
of the record in the file.

•

DELETE -- removes
indexed file.

•

TRUNCATE -- effectively deletes all records in a sequential
file from the current record to the logical end-of-file.

existing

1-15

record

from

and

relative

sets

RMS-11 CONCEPTS AND PROCESSING ENVIRONMENT
For the FIND, GET, and PUT operations, your program specifies the
recora access mode -- sequential, random by RFA (FIND and GET only),
or random by key -- which determines which record is the target of the
operation.
See RSX~llM/M-PLUS RMS-11: An Introduction, Chapter 4,
for a more
detailed introduction to record processing. Chapters 3, 4, and 7 of
this user's guide describe specifically how the record operations work
depending on the file organization selected and (for FIND, GET, and
PUT) the access mode specified.

1.3.3

RMS-11 File Processing

RMS-11 must manipulate the contents of files so that it can process
records.
However,
RMS-11 does not directly perform the actual file
manipulation, and the flow of data, control, and overlay segments that
the file manipulation entails.
RMS-11 issues requests to the file
control processor to perform the actual I/O and other operations on
the files.
Thus,
the file control processor's internal operation,
while invisible to RMS-11, can affect your program's performance.
The file control processor is not concerned with the data records in a
file.
It knows only virtual and logical block numbers, directories
and other information, and the disk drivers involved.
Therefore,
RMS-11 can direct file manipulation as long as it makes the proper
requests to the file control processor. To do so, RMS-11 maintains
the following structures, or I/O units:
•

Blocks -- The I/O unit for sequential files is the block.
You
can adjust the block count for each record access stream so
that more than one block can be moved during each I/O
operation.
In addition, you must decide whether records can cross block
boundaries.
When records can cross block boundaries, RMS-11
can pack them with optimal density in the file because a
record can be stored in one or more blocks. This is called
block spanning. Figure 1-9 illustrates block spanning.
When records are restricted by block boundaries, each record
must be no more than 512 bytes (one block) long, and unused
bytes may be left at the end of each block.

•

Buckets -- The I/O unit for relative and indexed files is the
bucket.
A bucket consists of one or more blocks that RMS-11
treats as a unit.
Records can cross block boundaries but they
cannot cross bucket boundaries.
Bucket size is a file
attribute that you specify when you create the file.
Buckets are an RMS-11 concept, so when RMS-11 initiates an
operation for a relative or indexed file, it requests the file
control processor to move a bucket by specifying the virtual
block number for the first block in the bucket and the size of
the bucket in bytes.
Note that buckets are fixed within the
file;
once created, buckets contain the same virtual blocks
at all times.

1-16

RMS-11 CONCEPTS AND PROCESSING ENVIRONMENT

r--BLOCK~BLOCK~BLOCK~BLOCK~BLOCK~BLOCK--i

II II II ! II II ! II II ii Ii II II ! Ii II i
I

11~1·1~1

THAN 512
BYTES

I
C.

VARIAB~E-L~NGTH ~
RECORDS

I I

'=7

RECORDS

ZK-1173-82

Figure 1-9: Records Spanning Blocks

You can also direct RMS-11 to request the file control processor to
place a file on a disk at a specific location. This is called
placement control and can improve performance by taking advantage of,
for example, tracks and cylinders.
RMS-11 provides access sharing;
that is, your program can control who
can gain concurrent access to the data in a file and what type of
operations they can perform on the data. See Section 2.2.3 for more
information on access sharing.
The RMS-11 directory and file operations perform the file-level
functions.
The directory operations affect file specification entries
in directories (not the contents of the files).
They are:
•

ENTER -- places a disk file specification in a directory.

REMOVE

deletes a disk file specification from a directory.

RENAME
new one.

replaces an existing disk file specification with

PARSE
program.

returns

•

SEARCH -- examines one or more directories for a specified
file and returns the file specification and location.

file

specification

1-17

information

your

RMS-11 CONCEPTS AND PROCESSING ENVIRONMENT
NOTE
Most high-level languages do not support
the
directory operations.
See your
high-level language documentation.
The file operations provide access to files.

They are:

•

CREATE -- creates a new file with the attributes
and opens it for processing.

you

specify

•

OPEN -- makes an existing file available for processing.

terminates access to a file.

ERASE
deletes a file and removes its
• one
is specified.

directory

entry,

• EXTEND -- increases the allocated size of an open file.
-- returns file information about an open file to your
• DISPLAY
program.
See RSX~llM/M-PLUS RMS-11: An Introduction, Chapter 4,
for a more
detailed introduction to file processing. Chapters 3, 4, and 7 of
this user's guide describe specifically how the file operations work
depending on the file organization selected.

1.4

FILE ATTRIBUTES

When you create an RMS-11 file, either through a program
(using the
CREATE file operation routine)
or by using the RMSDES utility, you
must specify the following information:
•

Medium -- Disk or magnetic tape.
You can also create files on
unit-record devices,
such as line printers and terminals.
Note that relative and indexed files are restricted to disk
devices.

•

File specification -- The name you assign to a file enables
RMS-11 to find the file later. Use the file specification
conventions specific to your operating system.

•

Protection -- RMS-11 allows you to assign a protection code to
a file when you create it. Use the protection codes specific
to your operating system.

•

File organization -- Sequential, relative, or indexed.

•

Record format -- Fixed length, variable length,
or undefined.

•

Record size
For fixed-length records, the size is the same
for every record
in the file.
For variable-length records,
the size is the maximum length any record can be.

VFC,

stream,

For VFC records, there are two size specifications:
(1)
the
fixed
length of the control area, and (2) the maximum length
of the variable data area.
RMS-11 also keeps the length of the longest record actually
stored in a sequential file for variable-length and VFC
records.

1-18

RMS-11 CONCEPTS AND PROCESSING ENVIRONMENT

•

Block spanning (sequential files)
block boundaries.

-- Whether records can cross

•

Bucket size (relative and indexed
blocks in each bucket.

files)

•

Maximum record number (relative files) -- The
of records that the file will contain.

•

Keys (indexed files) -- ~he number of keys;
the position and
size of each key;
the data type for each key; and other key
characteristics.

The

number

maximum

number

Record-output handling
You can specify three
(mutually
• exclusive)
types of handling for records being written
directly to a
specify any:

unit-record

device,

although

you

need

not

Carriage control -- The device driver inserts a linefeed
character as a pref ix to each record and a carriage-return
character as a suffix to each record before passing it to
the device.
FORTRAN -- The device interprets the first byte
record as a FORTRAN forms control character.

each

Print file format (VFC records with a fixed header size of
2 bytes) -- RMS-11 interprets the first byte of the
header as a pref ix for the record and the second byte as a
suffix for the record.

O or

•

File allocation -- You must specify two quantities:
Initial allocation -- the size of the file in
it is created.

blocks

Default extension quantity -- the number of blocks to
added to a file when RMS-11 automatically extends it.

when
be

Whether the disk space initially allocated to
• Contiguity
the file is to be allocated in continuous, adjacent logical
blocks.
•

Placement control -located on the disk.

Where

the

file

physically

During the file creation process, RMS-11 stores this information,
called the file attributes, in the file directory and, for relative
and indexed files, in the first blocks, or prologue, of the file as
well.
After creation, for the life of the file,
RMS-11 gets information
about a file from the file itself. This offers several advantages:
•

Most file attributes do not change.

•

You can design your RMS-11 files offline.
No
program
accessing the files need specify attributes (except those that
may be required by high-level languages) , because RMS-11
requires only a file specification from a program to open a
file.

•

You can open an RMS-11 file with its file specification only.
After that, RMS-11 enables you to read the file attributes.
You can write a program or use the RMSDSP utility to display
those attributes.
1-19

RMS-11 CONCEPTS AND PROCESSING ENVIRONMENT
Note that some of the attributes are interdependent;
that is,
the
selection of one attribute directly affects;
or restricts: other
attribute options. File organization, record format, and medium are
all
interdependent.
For example, if you select magnetic tape medium,
you must use sequential file organization.
And if you select VFC
records, you cannot use indexed file organization and you must use a
disk device.
Table 1-1
lists
interdependencies.

the

format

record

and

organization

file

Table 1-1: Record Formats and File Organizations
File
Organization

Fixed

Record Format:
Variable VFC Stream

Undefined

Sequential:
Mag tape
Disk

Yes
Yes

No
Yes

Relative

Yes

Indexed

Yes

Chapters 3 through 7 discuss your file design options in detail,
depending on your selection of file organization. Chapter 2 provides
information to help you make that selection.

1.5

PROCESSING BY BLOCK ACCESS

Your program can bypass RMS-11 record processing
RMS-11 file in a mode called block access.

and

process

any

Your program can read or write blocks in a file either sequentially or
(on disk only)
randomly by virtual block number (VBN). But your
program must be able to interpret the contents of those blocks.
,... _ _

oee

T"\l*'t"-7

.,.,,.,.,1,.,.

nT'r.,~

ftOA-i~~1~-r~ui.:>

block access and
Programmer's Guide
processing.

'n'a.•t"'I

•

1'1"Jo.:>- ~ ~.

An Introduction

f cr

introduction

processing.
See the RSX-llM/M-PLUS RMS-11 Macro
for detailed information on block access----and

1-20

CHAPTER 2
APPLICATION DESIGN

When you write an application program, you want that program to input
data, process it, store it, update it if necessary, and at intervals
output it in the p~oper formats.
You want all this to happen simply, quickly, and accurately.
You must
therefore take the time to design your application by carefully
considering RMS-11 file structure and file and record processing
capabilities.
Important RMS-11 considerations are data storage
medium, record format, file organization, access mode, allocation,
overlays, and so on.
If you do not consider RMS-11 capabilities when you design your
application, you may not get the best peformance possible from your
application because of the defaults that will be applied automatically
to your files (see Section 2.1).
Example:
The first
time one user created a
file,
she used a
high-level language program and took all
the defaults.
Then she
loaded records into the file;
the process was quite lengthy.
However, when she re-examined the file and re-created it applying some
RMS-11 design considerations,
the record insertion process went 10
times as fast.
Example:
Some users, accustomed to programming with BASIC-PLUS record
I/O,
learned that RMS-11 uses 15 bytes of control data in each bucket
and 7 bytes of control data for each fixed-length record in an indexed
file
(see Chapter 6).
Then, because they were accustomed to working
with whole blocks, they set up single-block buckets
(512 bytes)
and
subtracted RMS-11 overhead (22 bytes) to come up with a record size of
490 bytes.
But when they used those files, the users were alarmed to see them
grow at high rates.
They had not read that RMS-11 preserves its fast
sequential and alternate key access during random insertions by moving
records
and
leaving
behind 7-byte pointers
(see Chapter 5) •
Therefore, when one of those 490-byte records was movedf
it left
behind 7 bytes, which meant that no other record fit into that bucket.
Soon the file was filled with practically empty buckets that could not
be used because the designers did not allow for the full implications
of RMS-11 structure.
If you develop an application with a high-level language, you probably
will not worry about RMS-11.
You will accept the language's concept
of design, if any.
It is possible, however,
that the defaults the
language uses
in its interface with RMS-11 are not well suited for
your application.

2-1

APPLICATION DESIGN
This chapter presents general design considerations that apply to all
aesigns and information that will help you make the first
important design decision:
selection of a file organization.

application

2.1

WHEN TO DESIGN

There are two times to design an application:
1.

Before you write the application, especially if you have:
•

Large file(s)

•

Many users simultaneously accessing the file(s)

•

A high level of activity
(many records read,
updated, or deleted in a given time period)

After you write the application, if you are
its performance.

not

written,

happy

with

Often, poor performance results from default values
that
are
inappropriate
for
your
application.
You can frequently find
improvements by studying the nature and source of the defaults and how
they affect the structure of your application and your file.
Basically, defaults have three sources:
1.

Source language compilers
In many instances, source language compilers such as COBOL-81
or
BASIC-PLUS-2 supply default values for
RMS-11 file
attributes and/or facilities.
Example:
RMS-11 does not calculate an optimal bucket size
for
indexed files.
Rather,
the program creating the file
must specify a bucket size. When that program is the product
of a compiler, the bucket size can be explicitly specified in
the source code or it can be implicitly set by the compiler,
using a default value.

RMS-11
The interface between the RMS-11 routines and your program
has the same structure in all tasks, regardless of their
source (PDP-11 COBOL-81, RPG, MACR0-11,
and so on).
This
interface consists of control blocks (see the RSX-llM/M-PLUS
RMS-11 Macro
Programmer's
Guide
for
details).
The
informatron-- provided
by your program in these blocks
effectively controls RMS-11,
causing it to create, open,
access,
and close files.
However, when explicit information
is not provided, RMS-11 uses its default values.

Operating system
RMS-11 acts as an intermediary between your task and
the
operating
system.
As such,
RMS-11 can supply control
information for system functions such as protection codes.
However,
if RMS-11 supplies no control data. the system uses
its defaults.

2-2

APPLICATION DESIGN
2.2

DESIGN CONSIDERATIONS

When you design your application, you
four design considerations:

are

concerned

Speed
You want to maximize
programs process data.

Space
You want to minimize the room for the data
task on disk and the memory the task takes to run.

Shared access
accessible to
necessary.

the

speed

with

the

You want your
people using

Ease of design
You do not want to
necessary writing the application.

with

which

the

and

the

pr1mar1.4.y

data to be exactly
the computer system
spend

time

as
as
than

Remember, the importance of design is proportional to the complexity
of the file organization.
That is, design is least important for
applications using sequential
files
and
most
important
for
applications using indexed files.

2.2.1

Speed

You can make many performance (speed) decisions before you have to
consider anything else.
Therefore,
the first criterion to apply
throughout the design process is minimize I/O time.
The mechanics of the mass storage devices on your system consume most
of the time for any RMS-11 operation.
The memory-resident routines
that prepare the data for I/O or process it afterwards are very much
faster (one to three orders of magnitude).
An application's entire environment affects I/0 time:
•

File structure -- A variety
time, including:

file

attributes

affect

I/O

bucket size (for a relative or indexed file)
number of keys (for an indexed file)
number of duplicate key values (for an indexed file)
initial file allocation
default extension quantity
•

File size -- The number of records in the file affects the I/0
operations required to scan a file sequentially or follow an
index.

•

Program -- Your program affects
I/O time by requiring
I/O
operations for
file operations
(OPEN, CLOSE,
and so on),
record operations (GET; PUTr and so on), and overlays.

•

RMS-11
The RMS-11
routines
can
be
structured
disk-resident overlays or as memory-resident overlays.

•

File control processor
Besides requiring overlay segments
from disk,
the file control processor can also request I/O
operations required to map virtual blocks of the file
to
logical blocks on the storage device.

2-3

APPLICATION DESIGN
•

Device hardware -- The storage device that contains the task
and data files is the primary contributor to the length of an
I/O operation.
The type of device chosen
{moving-head,
fixed-head,
and so on) and the demands on it {amount of I/O
activity for that device within the system) are crucial to I/O
performance.

Figure 2-1 illustrates this environment.

TI§@
TIME
DEVICE

ZK-1163-82

Figure 2-1: Time Factors in an I/O Operation

2.2.2

Space

RMS-11 requires space for three reasons:
1.

To store data in a file

To store the RMS-11 routines either (a) on disk when they are
not in use, or {b) in memory when they are being executed

To buffer data in memory while the task runs

2-4

APPLICATION DESIGN
2.2.2.1 Data Storage - The space RMS-11 requires to store data is
proportional to the organization of the file, and to the processing
capabilities of that organization:
Sequential file organization -- RMS-11 adds to the size of
• your
data an empty byte,
if necessary, to align each
fixed-length,
variable-length,
or
VFC
record
on
an
even-numbered
byte
boundary.
When
the file contains
variable-length records, RMS-11 also prefixes a count field to
each record.
file organization -- RMS-11 constructs a series of
• Relative
record storage ceiis based on the length of the records.
The
cells are 1 byte longer than the fixed size of fixed-length
records or 3 bytes longer than the maximum size specified for
variable-length records.
•

RMS-11 adds to your data:

Indexed file organization

An index for each defined key.
15 bytes of formatting information for each bucket.
A 7-byte header for each record.
A count field for each variable-length record.
Other overhead of varying lengths for records RMS-11
during file activity and for deleted records.
You should keep the size of records to the minimum required
application.

for

moves
your

2.2.2.2 Task Size - The space RMS-11 routines occupy in a task
depends on the method you use to link the routines with your program.
See Section 8.1 for more details.

2.2.2.3 Buffer Sizes - You can vary the size of the I/O buffers
RMS-11 uses to store data in memory.
Generally, the larger the
buffers, the faster the task processes data.
See Section 3.5.3,
Section 4.5.3, or Section 7.4 for the file organization(s) you are
interested in.

2.2.3

Shared Access

Shared access revolves around the question: Who is allowed to read
from or write to a file? The answer involves your operating system's
protection codes: your access
declaration,
and
your
sharing
declaration.
System Protection Codes: Before you can access an RMS-11 file, you
must log into your computer system using an account number that will
allow you the kinds of access you need when your access request is
validated against the file's protection codes.

2-5

APPLICATION DESIGN
Ope~ating systems allow you to assign a protectiQn code to
each file
when it is created. This code describes concentric circies of users
who are allowed different levels of access to that file.
See your
operating system documentation for specific protection conventions.

Figure 2-2 illustrates the system protection concepts.

READ ACCESS

WRITE ACCESS

EXTEND ACCESS

DELETE ACCESS

RSX-11 M/M-PLUS
ZK-1166-82

Figure 2-2: System Protection Concepts

Access Declarations: Your program must declare the types of access
you need by spec1tying the record or block operations it intends to
perform on the file, as follows:
•

Read-only access is granted if your program specifies
only FIND/GET or READ operations can be performed.

that

No PUT, UPDATE, DELETE, TRUNCATE, or WRITE operations will be
allowed, nor will any other operation which would modify the
file (an EXTEND operation, for example, will not be allowed
for read-only access) •
•

Read/write access is granted if your program specifies that
PUT, UPDATE, DELETE, TRUNCATE, or WRITE operations can be
performed. FIND/GET and READ operations will also be allowed,
as will EXTEND operations.
Note that, in addition to any access deciaration, a CREATE
operation always forces read/write access so that the newly
created file can be populated (using PUT operations for recora
access or WRITE operations for block access) •

2-6

APPLICATION DESIGN
Sharing Declarations: Your sharing declaration specifies the types of
access to the file that your program is willing to allow to other
programs that request access to the file at the same time that your
program is accessing it. These declarations can be:
•

No sharing -- You do not want any other program to access
file.

the

A no-sharing specification in your
sharing
declaration
overrides any other sharing specification you may also have
included, and guarantees that no other program will have
concurrent read/write access to the file.
That is, no other
program will be able to modify the file.
Note, however, that it is not possible to guarantee
concurrent read-only accessors will be denied access.

that

•

Read-only sharing -- You are willing to allow
read-only access to the file.

•

Read/write sharing -- You are willing to allow other programs
read/write, as well as read-only, access to the file.

•

Sharing with user-provided interlocks (sequential files only)
This specifies a special form of sharing among a group of
programs that includes any number of read-only accessors and
at most one read/write accessor.

other

programs

User-provided interlocks offer a limited form of access
sharing of sequential files.
If the file organization is
sequential, this specification in your sharing declaration
overrides any other sharing specification (except no sharing).
For any other file organization, this specification
is
ignored.
NOTE
High-level languages may use slightly
different terms to designate the access
and sharing declarations, and may not
provide equivalents for all the sharing
options. See your high-level language
documentation.
Once the operating system's protection checks are passed, RMS-11 and
the operating system cooperate to determine whether the type of access
you request (your access declaration) and the type of sharing you
permit
(your sharing declaration)
are consistent with any other
current accessors of the file.
If no other tasks have accessed the file at the time that your program
requests access, your access request must only pass the system
protection checks to be granted. However, if one or more programs
already have access to the file, RMS-11 and the operating system will
use the access and sharing declarations of those programs along with
those of your program to determine whether your program will be
allowed concurrent access.
No-sharing and read-only declarations are processed as described above
for files of all organizations and access method (block or record).
In other cases, however, RMS-11 and the operating system interpret the
access and sharing declarations in the manner best suited to the
file's organization and the access method, as described in Section 3.4
for; sequential files, Section 4.4 for relative files, and Section 7.1
for indexed files.

2-7

APPLICATION DESIGN

NOTE
As noted, file sharing is a cooperative
effort between RMS-11 and the operating
system.
The
RMS-11
processing
algorithms
depend upon the detailed
nature of this cooperation.
If you
access a file concurrently with multiple
programs, some of which use RMS-11 and
some of which do not, the results may be
unpredictable.

2.2.3.1 Bucket Locking - Any time a record is updated, accessing
programs must be assured that the data written to the file is current
until the record is re-accessed and the record updated again.
If no control is placed on access, two or more programs could access
the same record, one after the other, and update it, one after the
other.
Only the last update would remain in the file.
Access sharing
could thus impair data integrity.
To ensure data integrity, RMS-11 uses bucket locking for a relative or
indexed file when the file is open for write-shared access.
From that
point, RMS-11 requests the operating system to lock each bucket read
from disk until RMS-11 explicitly releases the bucket.
After a GET,
FIND, or mass-insert PUT operation, only the bucket containing the
data record remains locked.
(See Chapter 7 for information on mass
insertion.) While that bucket is locked, no other program can access
it~

RMS-11 requests the operating system to unlock such a bucket when
of the following occurs:

one

•

The GET, FIND, or PUT operation fails.

•

The GET or FIND operation succeeds
declared read-only access to the file.

•

The program initiates another record operation that accesses a
different bucket.

the

program

has

After the bucket is unlocked, other programs can access it.
Example:
Programs A and B are write-sharing a file named RMSREL.DAT.
Both try to update relative record number 12. However, program B
initiates the prerequisite GET operation first,
locking the bucket
containing the record.
The operating system keeps program A from
accessing that bucket while program B uses it.
After program B
updates record 12, RMS-11 unlocks the bucket and the operating system
allows program A to get record 12
(including program B's updated
data).
Figure 2-3 illustrates this example.
Bucket locking incurs costs:
The operating system administers bucket
locking.
It establishes, for each file, a list of virtual blocks that
are locked. The system must scan this list every time RMS-11 performs
an I/O operation and then either permit the operation or return an
error~
In addition to this lock-list overhead, extra instructions are
executed to lock and unlock the buckets.

2-8

APPLICATION DESIGN

PROGRAM

PROGRAM B

DO NEXT
RECORD

PROCESS
RECORD

Figure 2-3: Bucket Locking Example

2.2.3.2

Sharing among Access

Streams

addition

the

bucket

locking used when programs allow sharing, RMS-11 provides its own
version of bucket locking when a program accesses a
file
for
write-type operations.
This locking allows multiple streams to share
the file.
RMS-11 bucket locking works the same way as the locking
provided by the operating system, except that the locks can be
encountered only by different access streams within the same program.
The overhead for RMS-11 bucket locking is small.

2-9

APPLICATION DESIGN
2.2.3.3 Programming Considerations - For the greatest flexibility at
run time, you should assume that access to any record by your program
can be denied because the bucket containing the record is locked.
RMS-11 returns the error code ER$RLK when the bucket is locked by
another access stream in the same or in another program.
Therefore, you should use the following techniques
RMS-11 programs that involve shared access:
•

when

you

write

Never keep a bucket locked longer than necessary.
You should
follow any successful GET or FIND operation with another
record operation of any type as soon as possible. The second
operation
unlocks
the
bucket locked by the read-type
operation.
Alternatively, you can release the bucket explicitly with a
FREE operation.
A FREE operation releases only the bucket
locked by the access stream associated with the operation.

If your program detects an ER$RLK error
(or its high-level
• language
equivalent),
its error processing depends on the
number of access streams active on the file:
Single stream
Set up a loop that
stalls,
then
re-initiates the record operation until RMS-11 indicates a
successful completion.
Multiple streams -- Do not set up a loop that continuously
re-initiates the record operation.
You should either (a)
continue processing on the other streams, attempting the
record operation on the locked-out stream periodically, or
(b) release the buckets locked by all other streams, then
re-initiate
the
record
operation that failed.
Any
GET-UPDATE or FIND-UPDATE sequences interrupted on the
other streams must be restarted, because the release of a
bucket destroys the record context.

2.2.4

Ease of Design

When you design and write your application, you should
yourself and the person who will maintain the application.
following design guidelines in mind:
•

consider
Keep the

Keep things simple. You can apply this criterion to the whole
development process,
from program flowcharts to the record
layouts to file organization and design.
Example: From sequential through indexed,
the RMS-11 file
organizations offer increasing capabilities, but they are also
increasingly complex. Choose the organization that supplies
enough capabilities, but no more. For instance, if you want
to randomly access a file by a single key only, you might use
a relative file and a hashing algorithm instead of an indexed
file.

•

Apply optimizations one by one until you reach a satisfactory
level of performance. Generally, further improvements are not
necessary.

2-10

APPLICATION DESIGN
Example:
The optimization of performance of applications
using
indexed files can be involved, but you do not have to
use every technique discussed in this manual.
You should only
satisfy current performance requirements.
For instance, when
an application program needed optimization, the indexed file
beina read was made contiauous (see Chaoter 6) and the RMS-11
overlay structure was changed (s~e Chapter 8) =· Execution time
dropped
from 16 minutes to 8.5. Since this performance was
adequate, no further optimizations were considered.
Some optimizations apply to one type of record operation, but
not to others.
Determine whether an optimization will benefit
your processing before you implement it.

2.3

DESIGN PROCESS

The first step in the design process is the selection of the file
organization.
Section 2.4 presents information to help you make this
selection.
Once you have selected a file
chapter(s):
Sequential
Relative
Indexed

organization,

the

appropriate

Chapter 3
Chapter 4
Chapters 5, 6, 7

Each chapter discusses file structure
(physical and conceptual)
as
well as design considerations.
Indexed files are the most complex to
design because of their power and flexibility.
After you read the file organization chapter(s), go to Chapter 8, Task
Building and Common Optimization Techniques.
Finally, apply the design considerations described in these chapters.
Write your application;
create and populate the files, using the
RMS-11 utilities when they are useful;
use the programs and files
in
a
simulated environment while you evaluate performance.
You may ha~e
to return to this manual, changing your design and/or combining
attributes
and
RMS-11 facilities
in different ways,
until
the
application runs to your satisfaction.
Good design is important to the success of your RMS-11 application.

2.4

SELECTING A FILE ORGANIZATION

Table 2-1 lists
important features of each file organization
sequential,
relative,
and indexed -- to help you decide which one(s)
you need.
Table 2-2 points out advantages and disadvantages of each
organization.
The sections that follow the tables provide information about two of
the features of file organization -- record format and I/0 techniques
-- to help you select a file organization.

2-11

APPLICATION DESIGN
Table 2-1: File Organization Characteri?tics and Capabilities
Characteristics
and
Capabilities

Sequential

Relative

Indexed

Medium
Disk
Magnetic Tape
Unit Record

Yes
Yes
Yes

Yes
No
No

Record Formats
Fixed-length
Variable-length
VFC (disk only)
Stream (disk only)
Undefined (disk only)

Yes
Yes
Yes
Yes
Yes

Yes
Yes
Yes
No
No

Yes
Yes
No
No
No

Overhead per Record

None

1 byte

7 bytes

Access Modes
Sequential
Random
RFA access (disk only)

Yes
Yesl
Yes

Yes
Yes
Yes

Record Operations
CONNECT
DELETE
DISCONNECT
FIND
FLUSH
FREE
GET
REWIND
TRUNCATE
UPDATE (disk only)
PUT

Yes
No
Yes
Yes
Yes
No
Yes
Yes
Yes
Yes
Yes

Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
No
Yes
Yes

1 or more
blocks

Bucket

Normal mode
of operation
Yes
No

Selectable

No
No

Yes
No

Access Sharing2

Read-only

Read/write

Other Features

Block-spanni ng records

I/0 Unit
I/O Techniques
l.Jeferred write
Multiblock count
Multiple access
streams
Multiple buffers
Mass insertion

Bucket size Bucket size
Yes
Yes

Maximum record
number

Yes
Yes
Read/write
Areas

1. For fixed-format disk sequential files only.
2. See exceptions in Section 2.2.3, and in Sections 3.4, 4.4,
7 .1.

2-12

and

APPLICATION DESIGN
Table 2-2: File Organization Advantages and Disadvantages
Organization

Advantages

Disadvantages

Sequential

Simplest organization.

To get a record, most
high-level languages
must access all records
before it (no access by
RFA or by key) .2

Optimal use of disk and
memory:
• minimum overhead on
disk
• block spanning

You can add records only
at end of file.2

Optimal if application
accesses all records on
each run, except if file
must be write-shared~

Interactive process is
awkward: operator must
wait as a program searches
for a record.2

Most versatile in record
formats:

Certain compiled programs
cannot access a record
already passed without
closing and re-opening
file (REWIND is not
available).

• exchange data with
non RMS-11 systems
• compatible with
RSX-llM/M-PLUS
FCS filesl
• compatible with ANSI
magnetic tape format
• compatible with
RSTS/E stream filesl

You can delete records
only at end of file; use
TRUNCATE record operation.
Sharing normall~ restricted
to multiple readers.

Most versatile in storage
media; file is portable.
Random by key (RRN)
record access available
on fixed-format disk
sequential files.
Relative

Restricted to disk.

Random access in all
languages.

File contains a cell
for each cell number
between 1 and last
record in file; data may
not be stored densely.

Allows deletions.
Allows random GET and
PUT operations.

l.RMS-11 can read these file structures and return a record to your program.
However, differences in data storage techniques among programming languages
can keep the program from properly interpreting the contents of that record.
2. These restrictions do not exist for disk sequential files with fixed-length
record format;
records in such files can be stored and retrieved using random
by key access, depending on your high-level language capabilities.

(Continued on next page)

2-13

APPLICATION DESIGN
Table 2-2 (Cont.): File Organization Advantages and Disadvantages
Organization

Advantages

Disadvantages

Relative
(Cont.)

Optimal if application
accesses all records on
each run and file must
be write-shared.

Program must know relative record number or
RFA of record before it
can randomly access the
data; no generic access
as in indexed file organization.

Random and sequential
access with low overhead.
Can be write-shared.

Interactive access can be
awkward if you do not
access records by relative
number.
You can insert records
only into unused record
cells, but you can update
existing records.
RMS-11 does not allow
duplicate relative record
numbers.

Indexed

Most flexible random
access:

Highest overhead on
disk and in memory.

• by any one of multiple keys or RFA
• key access by generic
or approximate value
• you access records by
record contents

Restricted to disk.
Least simple programming.

Duplicate key values
possible.
Automatic sort of records by primary and
alternate keys; available during sequential
access.
Record location is
transparent to user.
Can be write-shared.
Potential range of key
values not physically
present as in relative
file organization.
variety of data formats
for keys.

2-14

APPLICATION DESIGN
2.4.1

Record Formats

RMS-11 supports all of the record formats described in the following
sections for sequential files, but restricts relative and indexed file
organizations (see Table 2-1).

2.4.1.1 Fixed-Length Format - Records in the file are the same size,
which is a file attribute. The fixed-length record format requires no
RMS-11 overhead.

RMS-11 limits fixed block-spanning records to 32,765 bytes, while
minimum valid record is 1 byte of data.

the

2.4.1.2 variable-Length Format - Records in the file can be any
length, up to a maximum of 32,763 bytes for block-spanning records.
This file attribute is user-settable and optional. For each record,
RMS-11 maintains a count field specifying the number of data bytes in
the record.
The size of this field depends on the storage medium for
the file.

•

On disk, the count field is a 2-byte binary
not include the 2 bytes for the field.

•

On ANSI magnetic tape, the count
decimal count that does include
field.

count

that

does

field
is a 4-character
the 4 characters for the

Figure 2-4 illustrates the count field for each medium.

ILE~~~H I

DATA RECORD

ON DISK

. 1~1.~ -

,_./ ", ...... ___ .....-

-------- --

--------------

......... /
ZK-1162-82

Figure 2-4: Count Field on Disk and Tape

Choose the variable-length record format if:
•

The data truly varies in length, because the format
length field to each record's size.

adds

the

•

You are designing a new application
require records to change length.

uses

may

2-15

where

future

APPLICATION DESIGN
NOTE
Changing a record's size during
an
UPDATE operation is restricted by file
organization.
See
Sections
3.5.1,
4.5.1, and 7.2 for more information on
using the UPDATE operation with the
specific file organizations.
RMS-11 limits variable-length block-spanning records on disk to 32,763
bytes because of the count field.
RMS-11 allows records to reach this
maximum only in sequential files;
other file organizations place
further restrictions on record size. The minimum valid record is 2
bytes of zeroes, representing a null record.

2.4.1.3 variable-with-Fixed-Control Format - A VFC record consists of
two areas:
•

A fixed-length control area from 1 to 255
length is maintained as a file attribute.

bytes

long;

the

•

A variable-length area that can vary in length from zero bytes
to the maximum record size stored as a file attribute.

For each record, RMS-11 maintains a count field specifying the number
of data bytes in the record including fixed and variable areas. The
size of this field is a 2-byte binary count that does not include the
2 bytes for the field.
RMS-11 limits VFC block-spanning records to 32,763 bytes because of
the count field.
The minimum valid record is 3 bytes: the length
field plus the minimum fixed area of 1 byte.
The maximum variable
area is the difference between 32,763 and the length of the fixed
area.

2.4.1.4 Stream Format - A stream record consists of a series of
contiguous bytes.
RMS-11 detects the end of a stream record only by
the presence of one of the following terminators:
Form feed (014 octal)
Line feed (012 octal)
Vertical tab (013 octal)
RMS-11 limits stream format to disk sequential files.
In addition,
the format causes the most CPU overhead because RMS-11 must examine
each record character by character for the terminator.
During record operations, RMS-11 processes stream records as follows:
•

For FIND and GET operations, RMS-11 scans the stream of bytes,
removing leading NULL (000) characters and searching for the
first occurrence of one of the terminators.
If it finds a
form feed, vertical tab, or line feed, RMS-11 includes the
terminator character with the record and considers the record
complete.

2-16

APPLICATION DESIGN
If it finds a carriage return,
RMS-11 checks the character
following the carriage return.
If the next character is a
line feed, RMS-11 discards both characters
(carriage return
and line feed) and considers the record complete. Otherwise,
RMS-11 includes the carriage-return character in the record
and resumes its search for a terminator.
During a GET operation, RMS-11 moves each character included
in the record into the user buffer as it scans the stream of
bytes.
RMS-11 does not move any data into the user buffer
during a FIND operation.
•

For PUT and UPDATE operations, RMS-11 checks the
last
character of the record in the user buffer.
If it finds a
line feed, vertical tab, or form feed, RMS-11 moves the record
as it is to the I/O buffer.
If it does not find one of these
terminators, RMS-11 moves the record to the I/O buffer and
adds a carriage-return/line-feed pair to the end of the
record.

2.4.1.5 Undefined Format - The undefined format means that RMS-11
reads only blocks, not records~
Your program must interpret the
contents of each block.

2.4.2

I/O Techniques

RMS-11 supports the following I/O techniques so
performance of record operations:

you

can

adjust

the

•

Asynchronous
record
operations
When
operating
asynchronously, your program may regain control before the
operation is completed;
that is, the program will continue
processing while the operation is being performed.
This may
improve processing time.

•

Multiple access streams -- A stream can handle only one record
at a time, but you can connect more than one access stream to
a relative or indexed file if you want to:
Process more than one record in
asynchronous record operations.

file

time

Maintain more than one context during the processing
file.
Each stream represents an independent,
sequence of record operations.

concurrently

with
of

active

•

Deferred write -- Normally, every write-type record operation
~u
a relative or indexed file resui~s in a physical I/O
operation.
However, you can sometimes have RMS-11 defer this
write function until the I/O buffer is full or must be used
for another bucket. Deferred write is the normal mode of I/O
for sequential files.

•

Multiblock count -- You can open a disk sequential file so
that RMS-11 reads or writes more than one block of the file
into the I/O buffer at a time. This capability speeds file
processing,
though the buffer gets bigger.
For relative and
indexed files, you achieve a similar effect by increasing
bucket sizes.

2-17

APPLICATION DESIGN
•

Multiple buff-ers
You can allo-cate I/O buffers for
a
relative or indexed file beyond RMS-ll's minimum requirements:
one for relative;
two for
indexed.
If the file
is not
accessed
for
read/write sharing, RMS-11 uses the buffers to
save in memory, or cache, buckets from the file, so that they
do not have to be read again from disk if needed.
For indexed files, RMS-11 caches the root buckets from indexes
that are used, saving one I/O operation on every random record
operation.
However,
for
relative files,
RMS-11 makes no
distinction between buckets, saving them until it has to use
the buffer.

•

Mass insertion -- Specified before the insertion of a series
of records already sorted in ascending order by primary key,
this mode enables RMS-11 to store the records
tightly and
quickly in the file.
Records can be mass inserted only at the
logical end of an indexed file.
Mass insertion significantly
improves performance for single-key indexed files.
However,
with each additional key defined for the file, the percentage
improvement is smaller~

2-18

CHAPTER 3
SEQUENTIAL FILE APPLICATIONS

This chapter discusses sequential file structure,
design,
and
processing.
Sequential file design consists generally of determining
the specific attributes, including record size and format,
that will
allow you to store, retrieve, and process your data efficiently within
the sequential file structure.
Your task design, along with your file
design, will determine your record and file processing options,
including record access modes.

3.1

FILE STRUCTURE

Physical Structure
Sequential files carry almost no
RMS-11
overhead.
The operating system's file management software stores
attributes in the file directory.
RMS-11 stores data
records
beginning with virtual block number (VBN) 1.
If records cross block boundaries (span blocks) , RMS-11 packs
• records
into the file end-to-end, allowing for control
information and padding.
•

If you do not allow records to span blocks, RMS-11 packs
records
into each block, allowing for control information and
padding.
NOTE
You will waste space in your file
both of the following are true:
allow

•

You do not
blocks.

span

•

Your records do not exactly fit
a block.

into

To be compatible with other

records

file management systems, RMS-11 flags
space that is not used at the end of each block. When you allow
records to span blocks, the only unused space starts after the last
record in the file.
Table 3-1 lists the end-of-block indicators.

3-1

SEQUENTIAL FILE APPLICATIONS
Table 3-1: End-of-Block IndiGators
Medium

Record Format

End-of-Block Indicator

Disk

All but stream

-1 in word following last valid byte

Disk

Stream

nulls (000)

to end of file

Magtape

All

circumflex

(A)

to end of block

For disk sequential files,
RMS-11 uses the end-of-file attribute,
stored in the file directory, to determine where the valid data in a
file ends.
This attribute includes a VBN and a byte offset within
this block.
The virtual block containing the logical end-of-file may
not be the last block allocated to the file.
RMS-11 reads the end-of-file attribute with the other file attributes
when it opens a file.
RMS-11 also updates the end-of-file in the file
directory when it closes the file if the end-of-file changed while the
file was open.
The end-of-file changes if records were added to the
end of the file or if the file was truncated.
Conceptual Structure -- In most cases, RMS-11 stores records
in the
sequence that programs write them, one after the other from the first
record in the file to the last.
For these files,
RMS-11 can only
access the records sequentially or, for disk files, randomly by record
file address (RFA).
The exception to this structure is the case of disk sequential files
with fixed-length record format.
In this case, RMS-11 stores records
in a series of fixed-length cells;
this is similar to relative file
organization
(see Chapter 4).
The cell size is the size of the
fixed-length record.
Only one record can be put into a cell, and
RMS-11 assumes that each cell contains a record.
RMS-11 numbers the
cells consecutively from 1 to n, where n indicates the last cell
in
the file.
A cell number indicates the location of the cell relative
to the beginning of the file, and is associated with the record as a
relative record number (RRN).
RMS-11 can access records
in a
fixed-format disk sequential
sequentially, randomly by RFA, or randomly by key (RRN).

file

NOTE
RMS-11 does not initialize the cells in
a fixed-format disk sequential file, nor
does it "know" whether a cell contains a
valid record.
Your application program
must maintain this information.

3.2

RECORD SIZE

Records in disk sequential files are word aligned, which means that
RMS-11 adds a pad byte to the end of any record with an odd number of
bytes~
RMS-11 uses
this
convention
to
maintain
structural
compatibility with FCS-11 sequential files.
You can define a sequential file so that RMS-11 writes records across
the boundaries between blocks.
Such a sequential file is optimally
dense;
all bytes within its allocated space are used, except at the
end of the file where no data has been written.
3-2

SEQUENTIAL FILE APPLICATIONS
Table 3-2 shows the maximum data size for records in a sequential
file.
They are adjusted for RMS-11 restrictions and overhead.
Table 3-2: Sequential File Data Sizes (in bytes)
Maximum Size
Format

With Block
Spanning

Without Block
Spanning

Data Size Calculation

Fixed

32,766

512

Your data + MOD(DS/2)1

Variable

32,765

510

Your data+ 2 + MOD(DS/2)1

VFC

32,765

509

Fixed + variable + 2 + MOD(DS/2)1

Stream

None

5112

Data+ terminator(s)

l.MOD(DS/2) is the remainder after the size of
bytes is divided by 2:

your

data

(DS)

MOD(DS/2)

0 if the data size is an even number of bytes.

MOD (DS/2)

1 if the data size is an odd number of bytes.

For VFC, DS = fixed + variable
2. Assuming a 1-byte terminator character;
however,
if
the
terminator is CR-LF, then the maximum length without block-spanning
records is 510 bytes. Note that these figures do not include the
terminator characters.

3.3

FILE DESIGN

For sequential files, the primary design considerations are:
•

Record format (see Section 2.4.1
RMS-11 record formats)

for

description

the

• Data storage medium
• File allocation
• Contiguity
3.3.l
Sequential files can be accessed on both disk and magnetic tape. When
you select the medium for your file,
you should consider the
following:
•

Speed of access -- How long can each record
Tape is significantly slower than disk.

•

Frequency of use -- How often do you use the file? If you use
it once a month, a quarter, and so on, you could store the
file on tape and save your disk for more immediate purposes.

3-3

operation

take?

SEQUENTIAL FILE APPLICATIONS
•

3.3.2

Transportability -- Do you need to use the file on different
operating systems?
RSTS/E disk structure is not compatible
with !AS, RSX-llM/M-PLUS, or VAX/VMS, and vice-versa.
If you
need to use the file across these systems, you should consider
using a magnetic tape file.

File Allocation

Disk file allocation involves two quantities:
•

Initial allocation quantity -- the number of
to a file when you create it.

blocks

assigned

•

Default extension quantity -- the number of blocks added to
file each time RMS-11 automatically extends it.

3.3.2.1 Initial Allocation - Even with sequential files, where a file
extension requires only an allocation of blocks by the operating
system, total allocation of the file when you create it is much more
efficient.
You calculate the allocation
records as follows:
ALQ

(ALQ),

blocks,

for

block-spanning

(NRF*RSZ)/512

where:
ALQ

is the allocation quantity in blocks

NRF

is the largest number of records that will be in the file at
time

RSZ

is the size of the record in bytes

For variable-length or VFC records, use the average
RSZ, including 2 bytes for the count field.

record

one

size

for

word

For fixed-length records, use the actual record size for RSZ.
Be sure to round RSZ up to
alignment.

multiple

account

This allocation can be done by RMSDES or by your application
depending on the capabilities of your high-level language.

program,

3.3.2.2 Default Extension Quantity - If the file cannot be totally
allocated at creation time, you should establish a reasonable default
extension quantity (DEQ) to minimize the number of (and the time spent
on)
file extensions. Even if the file is totally allocated when you
create it, you should establish a reasonable DEQ in case the file gets
bigger than planned.
The time required for each file extension is
significant, involving:
•

A call to the file control processor

•

Possible I/O operations
routines into memory

3-4

bring

file

control

processor

SEQUE~TIAL

FILE APPLICATIONS

•

I/O operations to read and change file directory information

•

I/O operations to read and change the disk free-block bit map

A good basis for calculation is the number of records added to the
file in a given period of time, such as a day;
use the formula for
allocation quantity in Section 3.3.2.1.
The DEQ can be set by RMSDES or by your application program, depending
on the capabilities of your high-level language.
If you do not
specify a DEQ, it will default to zero whether you create the file
with RMSDES or a high-level language. This means that RMS-11 will
extend the fiie according to the operating system default for file
extensions.
Example: You are inserting 1000 50-byte fixed-length records into a
sequential file.
Records do not span blocks;
therefore, each block
contains 10 records. The file is currently full;
that is, no more
records can be added without an extension.

3.3.3

•

If DEQ is zero, RMS-11 extends the file according to operating
system defaults, which are typically only a few blocks.
Therefore, in this example, if the system default is 5 blocks,
RMS-11 extends the file 20 times.

•

If DEQ is 1, RMS-11 extends the file for every tenth
operation after the first, for a total of 100 extensions.

•

If DEQ is 25, RMS-11 extends the file 4 times.

•

If DEQ is 100 or more, RMS-11 extends the file only once.

PUT

Contiguity

Contiguity can significantly affect performance.
Therefore, you
should consider contiguity for a disk sequential file to minimize the
time spent on each I/0 operation.
If the blocks in a file are not contiguous, they may be on different
parts of the disk, and thus require significant head movement to
access the file contents.
Physical contiguity, however, ensures that the file is stored on one
track or, at worst, adjacent tracks. Because the disk can read a
track without moving the heads, file contiguity reduces head movement.
This assumes that no other software is accessing the disk at the same
time.
Contiguity also enhances virtual-to-logical-block mapping (see Chapter
8) •

To ensure that the blocks in the file are physically contiguous,
allocate the whole file when you create it (see Section 3.3o2.l) and
specify that the allocation be performed contiguously.

3.4

ACCESS SHARING

Access sharing can be specified for disk sequential files, as
described in the following sections. See Section 2.2.3 for general
information on shared access.

3-5

SEQUENTIAL FILE APPLICATIONS
3.4.1

Record Access to Sequential Files

Because of their
internal structure,
record-structured sequential
files are not read/write sharable in the manner of relative and
indexed files.
Thus, a read/write sharing declaration for such a file
is converted internally to a read-only sharing declaration before the
file is processed.
As a result, multiple read-only accessors who
have
specified
no-sharing, read-only sharing, or read/write sharing can access such a
file concurrently as long as no read/write accessor is present;
or a
single accessor who has specified no-sharing, read-only sharing, or
read/write sharing can access such a file as long as no other accessor
of any kind is present. Other combinations are rejected:
the access
and sharing declarations are incompatible.
Limited sequential file sharing is possible, however, in the case of a
single read/write accessor in combination with multiple read-only
accessors, when the application programs involved (rather than RMS-11)
can take responsibility for any interlocking required.
In this case,
the read-only accessors must specify sharing with
user-provided interlocks to gain access;
the sharing declaration of
the single read/write accessor is immaterial.
Each read-only accessor
cannot read beyond the logical end-of-file mark that existed at the
time that accessor opened the file,
and must
recognize
that
inconsistent data may be returned if the single read/write accessor
modifies data within the accessible portion of the file.

3.4.2

Block Access to Sequential Files

Sequential files can be read/write shared using block access, but for
those
accessors who specify read/write sharing, automatic file
extensions will not occur and the logical end-of-file mark in the file
header will be neither respected nor updated.
(Again, this is because
of the internal structure of sequential files.)
Such read/write
sharing uses the operating system's block-locking facilities
to
coordinate shared access.
Sequential files can also be shared in a noninterlocked manner, with
user-provided interlocks.
Because of operating system restrictions,
the single read/write accessor must specify no-sharing or sharing with
user-provided
interlocks, and multiple read-only accessors must
specify sharing with user-provided interlocks.
These restrictions
also prohibit concurrent access to the file by read/write-sharing
accessors or an accessor who specified read-only
sharing
and
read/write access.
When no write accessor is present, sequential files can be shared
among multiple read-only accessors who have specified no sharing or
read-only sharing.

3.5

RECORD AND FILE PROCESSING OF SEQUENTIAL FILES

The

record

and
file
processing
capabilities
described
in
RMS-11:
An Introduction are available for sequential
tiles.
This section discusses the operations and their implementation
and restrictions with sequential files.
RSX-llM/M~PLUS

3-6

SEQUENTIAL FILE APPLICATIONS
3.5.1

Record and Stream Operations

The following record
sequential files:

and

stream

operations

can

performed

CONNECT
DISCONNECT
FIND
FLUSH
GET
PUT
REWIND
TRUNCATE
UPDATE
In all record operations, RMS-11 establishes the current
context
(if any) and the next record context (if applicable).
record operation fails,
RMS-11 normally sets the current
context to none and does not change the next record context.

record
If any

record

NOTE
For more information on the RMS-11 error
codes
referred cu iu ~ne following
sections, see the RSX-llM/M-PLUS RMS-11
Macro Programmer's Guide.

3.5.1.1 CONNECT - A CONNECT operation affects the record context
the access stream as follows:

for

•

Current record -- There is no current record.
Any
requiring a current record fails at this point.

•

Next record -- If you did not specify that you were going to
append records to the file,
the next record is the first
record in the file.
If you did specify that you were going to append
the file, the next record is the end-of-file.

operation

records

3.5.1.2 DISCONNECT - A DISCONNECT operation destroys the current
record context for the access stream. You cannot resume this context
by reconnecting the stream.

3.5.1.3
RMS-11:
1.

FIND - To perform a FIND

operation

sequential

Determines the location of the record in the
to the specified record access mode:

file

according

•

In sequential-access mode, location is
next record pointer.

•

In key-access mode, location is determined
by
the
specified relative record number and match criterion.
(This access mode is available for
fixed-format disk
sequential files only.)

3-7

indicated

file,

the

SEQUENTIAL FILE APPLICATIONS
In RFA-access mode,
location is determined
by
the
• specified
(This access mode is available for disk
RFA.
files only.)
2.

Reads the block containing the record, or the first part of
the record if the record spans blocks, from disk into the
task's I/O buffer, if it is not already in memory.
The block
may be in memory if the block was required by a previous
operation.

For disk files, returns the RFA to the program, but does
transfer the record to the program's user buffer.

Returns the RRN for fixed-format disk sequential files.

If no valid record exists in
depends on the access mode:

the

location

specified,

the

not

response

•

In sequential-access mode, the error code is ER$EOF, meaning
that no record was located because there are no more records
in the file.

•

In RFA-access mode, the error code is either ER$RFA,
if the
RFA had an invalid format, or ER$EOF, if the RFA specified a
location beyond the end of the file.

•

In key-access mode for fixed-format disk sequential files, the
error code is ER$KEY, if the key value had an invalid format,
or ER$EOF, if the key value specified a location beyond the
end of the file.

A FIND operation affects the record context for the access
follows:
•

stream

For a sequential-access FIND operation:
Current record is set to value of the record found,
that
is, the next record before the FIND operation started.
Example:
You have connected a stream to a sequential file
without specifying that records will be appended to the
file (see Section 3.5.1.1). There is no current record,
but the next record is the first record in the file.
If

you execute a sequential FIND operation, the current record
is set to the first record in the file.

Next record is set to the Iecord

virtually

following.

the

current record.
Example:
From the previous example, the next record is the
second record in the file.
•

For an RFA-access or key-access FIND operation:
Current record is set to the record found,
record identified by the RFA or RRN.
Next record is unchanged.

3-8

that

is,

the

SEQUENTIAL FILE APPLICATIONS
Example:
In the preceding example, you performed
a
sequential-access
FIND operation after connecting the
stream to the file.
You now execute a FIND by RFA.
The
current record is set to the record specified, but the next
record is not changed.
Therefore, when you perform another
sequential FIND operation, the current record is set to the
second record in the file, not to the record following the
one found by RFA.
You use a FIND operation instead of a GET operation for two reasons:
1.

It is faster because the record is not moved to the user
buffer. Although the time required to move a record from one
part of memory to another is very short, do not expend it
unnecessarily.

It does not change the next record in RFA or key access mode.
This
convention
allows
you to branch off sequential
processing for updating or deleting, and keep your place in
the file.

You can use a FIND operation in the following ways:
•

To skip records
in sequential
successive FIND operations.

•

To establish a random starting point using RFA or key access
mode.
You could then initiate successive GET operations,
where the first operation gets the record found by RFA or by
RRN.

•

To establish a
operation.

•

To determine whether a record cell specified by RRN exists
a file (for fixed-format disk sequential files only).

current

record

access

for

mode

UPDATE

initiating

TRUNCATE

3.5.1.4 FLUSH - A FLUSH operation does not affect the record
for the access stream.

3.5.1.5
RMS-11:
1.

GET - To perform

GET

operation

•

context

sequential

Determines the location of the record in the
to the specified access mode:

file

file,

according

In sequential-access mode, location is
indicated by the
next
record pointer,
if the get operation was not
immediately preceded by a successful FIND operation, or
the current record pointer set by an immediately preceding
successful FIND operation.

Location is determined by the specified relative record
• number
and
match
criterion
in
key-access
mode
(fixed-format disk sequential files only).
•
2.

Location is determined by the specified RFA in
mode (disk files only) •

RFA-access

Reads the block containing the record, or the first part of
the record if the record spans blocks, from disk into the
task's I/0 buffer, if the block is not already in memory.

3-9

SEQUENTIAL FILE APPLICATIONS
Example: Your records are 50 bytes long.
When you read
sequentially through the filef RMS-11 must request a disk I/O
operation for every tenth GET operation that your program
executes.
3.

For disk files, returns the RFA to the program and moves the
record from the I/O buffer to the specified user buffer in
the program unless the program is operating in locate record
transfer mode
(see Section 3.5.2}.
If the buffer does not
contain the entire record, RMS-11 reads more blocks into the
I/O buffer and assembles the record in the program's user
buffer, regardless of record transfer mode.

Returns the RRN for fixed-format disk sequential files.

If no valid record exists in
depends on the access mode:

the

location

specified,

the

response

•

In sequential-access mode, the error code is ER$EOF, meaning
that no record was located because there are no more records
in the file.

•

In RFA-access mode, the error code is either ER$RFA,
if the
RFA had an invalid format, or ER$EOF, if the RFA specified a
location beyond the end of the file.

•

A GET operation affects the current
stream as follows:

record

context

for

the

access

• Current record is set to the record read.
Next record is set
• current
record.

the

record

virtually

following

the

Example: You have connected a stream to a sequential file without
specifying that records will be appended to the file (see Section
3.5.1.l). There is no current record, but the next record is the
first record in the file.
If you execute a sequential-access GET
operation; the current record is set to the first record in the file
and the next record is the second record in the file.

3.5.1.6
RMS-11:

PUT - To perform

PUT

operation

sequential

file,

Determines whether the specified access mode is allowed.
Sequential-access mode must be specified unless the file is a
fixed-format disk file;
in that case, key-access mode is
allowed.
RMS-11 returns the error code ER$RAC if an illegal
access mode is specified.

Determines the destination of the record
according to the specified access mode:
•

the

file

In sequential-access mode, the next
record
pointer
indicates the destination.
The destination must be the
end-of-file;
if it is not, RMS-11 returns the error code
ER$NEF.

3-10

SEQUENTIAL FILE APPLICATIONS

Your program gets to the end of a sequential file by:
Specifying that records will be appended to the file
when the program connects the record access stream to
the file (see Section 3.5.1.1).
Initiating sequential FIND and/or GET operations
RMS-11 returns an ER$EOF error
•

until

In key-access mode, the specified relative record number
indicates the destination.
Note that RMS-11 does not
check the validity of the designated RRN:
if
the
destination block is beyond the current end-of-file,
RMS-11 will extend the file to the destination block.

Reads the destination block in the file into the I/O buffer,
if the block is not already in memory.
The block may be in
memory if it was required by a previous operation.

Moves the record from the
buffer.

Writes the I/O buffer to disk only if the buffer is full.
If
there is no room for the block(s) in the file, RMS-11 extends
the file (see Section 3.3.2) and then writes the buffer to
disk.

For disk files, returns the RFA to the program.

Returns the RRN for fixed-format disk sequential files.

user

buffer

the

task's

I/O

A PUT operation affects the context for the access stream as follows:
•

•

For a sequential-access PUT operation:
Current record -- None. Any operation requiring a
record fails at this point.

current

Next record -- End-of-file.
A sequential
operation fails with error code ER$EOF.

FIND

GET

For a key-access PUT operation:
Current record -- None. Any operation requiring a
record fails at this point.

current

Next record -- Unchanged.

3.5.1.7 REWIND - A REWIND operation affects the
the access stream as follows:

context

requiring

•

Current record -- None. Any
record fails at this point.

•

Next record -- Set to the first record in the file.

3-11

operation

record

for

current

SEQUENTIAL FILE APPLICATIONS
3.5.1.8 TRUNCATE - A TRUNCATE operation declares end-of-file at the
the operation
position
of
the current record.
In doing so,
effectively deletes the current record and all records in the
sequential file following that record.
The TRUNCATE operation requires a valid current record.
It therefore
should follow a successful GET or FIND operation;
otherwise, RMS-11
returns the error code ER$CUR.
A TRUNCATE operation affects the context
follows:
•

Current record -- None. Any
record fails at this point.

•

Next record -- End-of-file.

for

operation

After a TRUNCATE operation, you can immediately
file using PUT operations.

the

access

requiring

add

stream
a

records

current

the

NOTE
The TRUNCATE operation does not reduce
the
actual
allocated
size
of
a
sequential file on a disk:
it merely
specifies
a new logical end-of-file
mark.

3.5.1.9 UPDATE - In an UPDATE operation, RMS-11 moves the specified
record from the task's user buffer to the I/O buffer, replacing the
current record set by a previous GET or FIND operation.
However,
RMS-11 does not immediately write the buffer to the file.
RMS-11
requests the file control processor to write the changed buffer over
its original location on the disk only when the buffer must be
replaced in memory by another operation.
Example: You get a record by RFA and update it.
Then, you get
another record by RFA.
RMS-11 writes the buffer containinq the first
record you updated only when it must replace the data in the buffer to
satisfy the second GET operation.
UPDATE operations have the following restrictions:
•

The operation is valid only on disk sequential files.
If you
attempt it on magnetic tape files or unit record devices,
RMS-11 returns the error code ER$IOP.

•

The operation requires a valid current record.
It therefore
should follow a successful GET or FIND operation; otherwise,
RMS-11 returns the error ER$CUR.

The size of the record cannot change during an UPDATE
• operation.
If it changes,
RMS-11 returns the error code
ER$RSZ.
•

You cannot update stream records.
returns the error code ER$RFMc

If you attempt

it,

RMS-11

None of these errors affects the original record in the file on disk.

3-12

SEQUENTIAL FILE APPLICATIONS
An UPDATE operation affects the
follows:

context

•

Current record -- None. Any
record fails at this point.

•

Next record -- Unchanged.

3.5.2

for

the

operation

access

stream

requiring

current

Record Transfer Modes

You can manipulate records either ~u the I/O buffer or in your
program's user buffer.
Each of these options is called a record
transfer mode. You can change record transfer mode at run time, even
between record operations.
Figure 3-1 shows the I/O and user buffers.

fS•ZEDEPENDS ON:- - - - - - - -i
I• NUMBER OF FILES OPENED SIMULTANEOUSLY i
I • BUCKET SiZES
I

L NUMBE~~REC~D~CCE~ ~~AMS -

USER BUFFERS

_=}_ I

1/0
BUFFERS
VIRTUAL
MEMORY

PROGRAM

RMS-11

INTERNAL
CONTROL
STRUCTURES

----- -- --L ~

[S1zE DEPENDS ON:

l•

RMS-11 FUNCTIONS USED

~ ~_v~RLA~~U~TURE_:J~E~

ZK-1174-82

Figure 3-1: RMS-11 Task Structure

3.5.2.1 Move Mode - Move mode requires that
between the user and I/O buffers:

each

record

copied

•

On GET operations, RMS-11 moves the record from the I/O buffer
to the user buffer before returning control to your program.

•

On PUT and UPDATE operations, your program assembles the
record to be written into the file in the user buffer. During
the operation, RMS-11 moves the data into the I/O buffer
before updating the file.

Move mode is the default record transfer
languages and all file organizations.

3-13

mode

for

all

programming

SEQUENTIAL FILE APPLICATIONS
3.5.2.2

Locate Mode - Locate mode enables your program to manipulate
in the I/0 buffer, eliminating the data transfers between it
and the user buffer. However, when you specify locate mode, RMS-11
uses it only when such usage does not compromise data integrity.
Otherwise, RMS-11 uses move mode. Therefore, your program must still
contain a user buffer.

reCOids

Example:
RMS-11 uses move mode instead of locate
span buffers in a sequential file.

mode

when

records

Example:
RMS-11 uses move mode instead of locate mode if you opened
the file
indicating that you were going to perform UPDATE operations
on it.
RMS-ll's use of move mode instead of locate mode is transparent to
your program as long as you use RMS-11 facilities to access the record
data.
For sequential files, your program can both performs both GET and PUT
operations in locate mode.
See your high-level language documentation
to determine whether the language supports locate mode and,
if it
does, what the programming techniques are.

3.5.3

I/O Techniques

You can use the following techniques to
record operations.

improve

the

performance

3.5.3.1 Asynchronous Record Operations - Within each access stream,
your program can perform any record operation either synchronously or
asynchronously.
In synchronous operations, RMS-11 returns control to
your program after the operation ends, either successfully or with an
error.
When you execute an asynchronous operation, RMS-11 may return control
to your program before the operation is complete.
The program
continues processing while the physical transfer of data between disk
and memory is carried out.
However, you must not initiate another
record operation on that stream until the first operation ~uu~;
otherwise, RMS-11 returns the error code ER$ACT.
See your high-level
language documentation for asynchronous techniques.

3.5.3.2 Deferred Write - The normal mode of operation for sequential
files
is similar to operations using deferred write with the other
file organizations (see Chapters 4 and 7). Using this technique for
sequential files does not change or improve performance.

3.5.3.3 Multiple Buffers - The
available to sequential files.

multiple

3-14

buffer

capability

not

SEQUENTIAL FILE APPLICATIONS
3.5.3.4 Multiple Access Streams - RMS-11 allows each program to use
only one stream on a sequential file because sequential files are not
formatted to permit simple and economical sharing (see Section 3.4).

3.5.3.5 Multiblock Count - Your task can be set up so that more than
one block from a disk sequential file is read or written at one time.
This multiple-block I/O can improve processing because it tends to
reduce the number of physical I/O operations.
However, it also
increases the size of the task, on a one-for-one basis; that is, for
each increment of the multiblock count (MBC), the I/O buffer in the
task grows by 512 bytes.
An MBC greater than 1 is therefore useful for
including file population.

sequential

processing,

Example:
You are using
50-byte
records.
During
sequential
processing, if the MBC is 1, RMS-11 requests a disk I/O operation for
every tenth record operation your program executes, whether the
operations are GET or PUT operations.
If you set MBC to 5 for
instance, RMS-11 requests a physical I/O operation for every 50 record
operations.

3.5.4

File and Directory Operations

The following file
sequential files:

and

directory

operations

can

performed

CLOSE
CREATE
DISPLAY
ENTER
ERASE
EXTEND
OPEN
PARSE
REMOVE
RENAME
SEARCH
See your high-level language documentation for a
support provided.

3-15

description

the

CHAPTER 4
RELATIVE FILE APPLICATIONS

This chapter discusses relative
file
structure,
design,
and
processing.
Relative file design consists generally of determining
the specific attributes, including record size and format,
that will
allow you to store, retrieve, and process your data efficently within
the relative file structure. Your task design, along with your file
design, will determine your record and file processing options,
including record access modes.

4.1

FILE STRUCTURE

Physical Structure -- Relative files contain at least one block of
RMS-11 information known as the prologue. The operating system's file
management software stores attributes in the file directory.
RMS-11
stores the prologue in VBN 1 -- unless the bucket size is 2, 4, or 8
blocks.
In that case, RMS-11 makes the prologue equal to 1 bucket in
size. Data records begin in the block following the prologue.
RMS-11 allocates relative files
in bucket increments.
The first
bucket begins with the first data block. To support deleted record
control, RMS-11 initializes each bucket (sets all bits to 0)
when it
allocates the blocks to the file.
The fixed-length cells are set up in each bucket starting with byte 0
and packed end-to-end, byte-aligned, until no more cells can fit in
the bucket
(no padding necessary) •
Cells cannot
span
bucket
boundaries, although they can cross block boundaries within multiblock
buckets. The first byte of each cell is used by RMS-11 to provide
deleted record control.
Conceptual Structure
RMS-11 stores records in a series
of
fixed-size cells.
Only one record can be put into a cell, but all
cells do not have to contain records. The cell size is based on the
length you specify as the maximum for any record in the file.
RMS-11
numbers the cells consecutively from 1 to n, where n indicates the
last cell in the file.
A cell number relates its location to the
beginning of the file and is associated with the record in the cell,
if any, as a relative record number.
RMS-11 can access records in a relative file either sequentially
randomly, both by relative record number (key) and by RFA.

4-1

RELATIVE FILE APPLICATIONS
4.2

RECORD SIZE

RMS-11 calculates the number of bytes in each record cell in the
(CL) of a relative record cell as follows:
CL

file

l+RFO+DS+FSZ

where:
1

is a byte for RMS-11 overhead

RFO

is bytes for record
variable or VFC

FSZ

is the fixed control size for VFC format;

is bytes of data

format

overhead:

For variable-length or VFC record format, DS
size set for the file.

for

fixed;

for

O for other formats

the

maximum

record

Table 4-1 shows the maximum data sizes for records in a relative file.
These are the sizes of your data;
they are adjusted for RMS-11
restrictions and overhead.
Table 4-1: Relative File Data Sizes (in bytes)

4.3

Format

Maximum Size

Record Cell Size Calculation

Fixed
Variable
VFC

16,383
16,381
16,381

Data size + 1
Maximum record size + 3
Fixed + variable + 3

FILE DESIGN

For relative files, the primary design considerations are:
•

Record format (See Section 2.4.1
RMS-11 record formats)

•

Buck~t

•

File allocation

•

Contiguity

•

Maximum record number

4.3.1

for

description

the

size

Bucket Size

Buckets are the I/O units for relative files.
Their size is therefore
critical to the space required by a task and the speed with which the
task performs. Sequential access, especially, benefits when there are
multiple records per bucket. There is, of course, a trade-off:
the
larger the bucket size, the larger the task, but the faster the task
reads data sequentially:
•

Each block added to the bucket size increases the task size by
512 bytes for each access stream.

4-2

RELATIVE FILE APPLICATIONS
•

The speed of an RMS-11 operation is closely proportional to
the number of I/O operations involved. RMS-11 requests an I/O
operation each time it requires a new bucket to locate a
record.
Therefore, tne more record cells in a bucket, the
fewer I/0 operations RMS-11 needs to read a file sequentially.
However, write sharing a relative file counteracts this
optimization if your program has read-only access to the file.
RMS-11 reads a bucket from disk during each GET operation
even if the next record is in the bucket in memory -- because
the bucket is not locked after each GET operation and a
writing program may have changed the bucket since the record
was last read.

Bucket size can be set by RMSDES or by your application
depending on the capabilities of your high-level language.

4.3.2

program

File Allocation

File allocation involves two quantities:
•

Initial allocation quantity -- The number of
to a file when you create it

blocks

assigned

•

Default extension quantity -- The number of blocks added to
file each time RMS-11 automatically extends it

4.3.2.1 Initial Allocation - Total allocation of a file when you
create
it is the most efficient technique regardless of file
organization, but with relative files initial allocation becomes most
critical.
Each allocation, whether at creation time or during an
extension, requires RMS-11 to initialize the new buckets by setting
all bits to zero. You can avoid time-consuming file extensions during
normal processing by totally allocating the file when you create it or
by explicitly extending the file when it is not being used for
processing.
You calculate the allocation (ALQ), in blocks, as follows:
ALQ

PLG+(NRF/NRBKT)*BKS

where:
PLG

is equal to 1 block or to BKS if BKS is 2, 4, or 8

NRF

is equal to the maximum record number (MRN) or to the number of
records that will be written into the file

BKS

is the bucket size in blocks

NRBKT

is the number of records in a bucket

4-3

RELATIVE FILE APPLICATIONS
You calculate NRBKT as follows:
NRBKT

= (512*BKS)/(RSZ+RFO)

where:
RSZ

is the size of the record in bytes:
•

Data size for fixed-length records

•

Maximum record length for variable-length records

•

Size of the fixed-length control area plus the maximum size of
the variable-length area for VFC records

RFO

is the record format overhead:
e

RFO

1 byte for fixed-length records

RFO

3 bytes for variable-length and VFC records

This allocation can be done during file creation by RMSDES or by your
application program, depending on the capabilities of your high-level
language.
The allocation can also be done by using a PUT operation to write the
"last record" into the file first;
that is, the record whose relative
record number is equal to the maximum record number
(MRN).
Before
RMS-11 can write this record, it must allocate all record cells from 1
to MRN and initialize the new blocks. After the PUT operation, the
relative file will be completely allocated.

4.3.2.2 Default Extension Quantity - If the file cannot be totally
allocated at creation time, you should establish a reasonable default
extension quantity (DEQ) to minimize the number of (and the time spent
on)
file extensions.
Even if the file is totally allocated when you
create it, you should establish a reasonable DEQ in case the file must
become bigger than planned.

A good basis for calculation is the number of records that

are added
to the end of the file in a given time period, such as a day;
use the
formula for allocation quantity in Section 4.3.2.1.

The DEQ for the file can be set by RMSDES or by your application
program, depending on the capabilities of your high-level language.
If you do not specify a DEQ, it defaults to zero.
RMS-11 responds to
a DEQ of zero by requesting 4 times the bucket size in blocks from the
file control processor each time it automatically extends the file.

4.3.3

Contiguity

Contiguity can significantly affect performance.
Therefore, you
should consider contiguity for a relative file to minimize the time
spent on each I/O operation.
If the blocks in a file are not
contiguous, they may be on different parts of the disk and thus
require significant head movement to access the file contents.

4-4

RELATIVE FILE APPLICATIONS
Physical contiguity, however, ensures that the file
is stored on a
single track or, at worst, adjacent tracks.
Because the disk can read
an entire track without moving the heads, file contiguity reduces head
movement.
This assumes that no other software is accessing the disk
at the same time.
Contiguity also enhances virtual-to-logical-block
in Chapter 8) •

mapping

(discussed

To ensure that the blocks in the file are physically contiguous,
allocate the whole file when you create it (see Section 4.3.2.1).

4.3.4

Maximum Record Number

The MRN associated with a relative file limits the size of the file.
RMS-11 will not put a record into a file with a relative record number
greater than the assigned MRN.
However, if an MRN is not set
(that
is, MRN is zero),
RMS-11 only checks whether the record number is
greater than zero before attempting to store a record
in a
relative
file.
MRN determines the maximum useful size of a
file because RMS-11
a11ocates a record cell for each record between relative record number
1 and the highest relative record number used.
You can explicitly
make the file
larger than this maximum, but RMS-11 will not use the
space. The actual size can be smaller than the size that would be set
if a record with the MRN were written into the file.
You can calculate the file size
(FSZ)
in blocks from
relative record number actually present in the file:
FSZ

the

largest

PLG+l+((LRN-l)/((BKS*512)/(RSZ+RFO)))

where:
PLG

is the size of
otherwise, 1

the

prologue:

BKS

LRN

is the largest RRN actually present in the file

BKS

is the bucket size in blocks

RSZ

is the size of the record in bytes:

• Data size for fixed-length records
• Maximum record length for variable-length records
of the fixed-length control area plus the maximum size of
• Size
the variable-length area for VFC records
RFO

is the record format overhead:
•

RFO

1 byte for fixed-length records

RFO

3 bytes for variable-length and VFC records

MRN can be set by RMSDES or by your application program, depending
the capabilities of your high-level language.

4-5

RELATIVE FILE APPLICATIONS
4.4

ACCESS SHARING

Access sharing can be specified for relative files as described in the
following sections.
See Section 2.2.3 for general information on
shared access.

4.4.1

Record Access to Relative Files

Relative files allow fully interlocked read/write sharing, dependent
upon the compatibility of the access and sharing declarations of
multiple accessors, as follows:
•

If you have requested read/write access, your request will be
denied unless all other accessors have allowed read/write
sharing.
(Otherwise, your read/write access request will
conflict with the sharing declaration of at least one other
accessor.)

•

If you have not permitted read/write sharing, your request for
read/write access will be denied if any other read/write
accessor is present.
(In this case, the read/write accessor
does not meet the requirements of your sharing declaration.)

4.4.2

Block Access to Relative Files

Because block access bypasses the record structure and interlocking
algorithms used with relative files, read/write sharing cannot be
permitted. Any read/write sharing declaration is converted internally
to read-only before the file is processed
(this is similar to
record-accessed sequential files).
Thus, multiple read-only accessors
(regardless of their sharing
declarations)
can share relative files concurrently using block
access, as long as no read/write record accessor is
present.
Read-only block accessors can share files with read-only record
accessors.
In addition, a single read/write accessor can access a
relative file using block access (regardless of sharing declaration)
as long as no other accessor of any kind is present.
Other combinations are rejected:
-

Cl.Lit::

4.5

.! -

-- -

.L .!

,_ ,

the access and sharing

declarations

.lll\.,;Vlllf:JCl l. .LU..ltt:: •

RECORD AND FILE PROCESSING OF RELATIVE FILES

The
record
and
file
processing
capabilities
described
in
RSX-llM/M-PLUS RMS-11:
An Introduction are available for relative
files.
This section discusses the operations and their implementation
and restrictions with relative files.

4-6

RELATIVE FILE APPLICATIONS
4.5.1

Record and Stream Operations

The following record and stream
relative file:

operations

can

performed

CONNECT
DELETE
DISCONNECT
FIND
FLUSH
GET
PUT
REWIND
UPDATE
In all record operations, RMS-11 establishes the current
context
(if any)
and next record context (if applicable).
record operation fails,
RMS-11 normally sets the current
context to none and does not change the next record context.

record
If any
record

NOTE
For more information on the RMS-11 error
codes
referred to in tne ro11owing
sections, see the RSX-llM/M-PLUS RMS-11
Macro Programmer's Guide.

4.5.1.1 CONNECT - A CONNECT operation affects
context for the access stream as follows:

the

current

record

•

Current record -- There is no current record.
Any
requiring a current record fails at this point.

operation

•

Next record -- The next record is the first record cell in the
file.

4.5.1.2 DELETE - In a DELETE operation, RMS-11 flags the current
record cell to indicate that it contains a deleted record.
RMS-11
does this by setting the RMS-11 control byte in the cell to a certain
value.
Then,
RMS-11 writes the bucket over its original location pn
the disk, unless you have specified deferred write
(see Section
4.5.3.2).
A DELETE operation requires a valid current record.
Therefore, a
DELETE operation should follow a successful GET or FIND operation;
otherwise, RMS-11 returns the error code ER$CUR. This error does not
affect the original record in the file on disk.
A DELETE operation affects the current record context for
stream as follows:
•

Current record -- None. Any
record fails at this point.

•

Next record -- Unchanged.

4-7

operation

requiring

the

access

current

RELATIVE FILE APPLICATIONS
4.5.1.3 DISCONNECT - A DISCONNECT operation destroys the current
record context for the access stream.
You cannot resume this context
by reconnecting the stream.

4.5.1.4
RMS-11:
1.

FIND - To perform

FIND

operation

Determines the location of the record in the
to the specified access mode:

relative
file

file,

according

•

In sequential-access mode, location is
next record pointer.

the

•

In key-access mode, location is determined
by
specified relative record number and match criterion.

the

•

In RFA-access
specified RFA.

the

mode,

location

indicated

determined

Reads the bucket containing the indicated cell from disk into
the task's I/O buffer,
if the bucket is not already in
memory. The bucket may be in memory if it was required by a
previous operation.

Returns the RFA and the RRN to the program, but
transfer the record to the program's user buffer.
If the cell is empty or contains a
response depends on the access mode:

deleted

does

not

record,

the

•

In sequential-access mode, RMS-11 repeats steps 1 through
3, moving through cells until the MRN is exceeded (ER$MRN)
or the end of the file is reached (ER$EOF) •

•

In key-access mode, RMS-11
specified match criterion:
On an equal
ER$RNF.

match,

RMS-11

reacts

according

returns

the

error

code

On a greater-than or greater-than-or-equal
match,
RMS-11 internally adds 1 to the relative record number
and repeats steps l through 3, until either the MRN is
exceeded
(ER$MRN)
or the end of the file is reached
(ER$RNF) •
•

In RFA-access mode, RMS-11 returns the
code:
ER$RNF
No valid record
specified location.

has

appropriate

ever

ER$DEL -- The control byte in the cell
the record in it was deleted.

existed

the

indicates

that

A FIND operation affects the record context for the access
follows:
•

error

stream

For a sequential-access FIND operation:
Current record is set to the relative record number of the
record found,
that is, the next record before the FIND
operation started.

4-8

RELATIVE FILE APPLICATIONS
Example: You have connected a stream to a relative file.
There is no current record, but the next record is the
first
record
in
the
file.
If
you
execute
a
sequential-access FIND operation, the current record is set
to the first record in the file.
Next record is set to a relative record number 1 higher
than the relative record number for the current record.
Example: From the previous example, the next record is the
second record cell in the file.
•

For a key-access or RFA-access FIND operation:
Current record is set to the record found,
that is,
the
record identified by the relative record number or RFA.
Next record is unchanged.
Example:
In the preceding examples, you performed a
sequential-access
FIND operation after connecting the
stream to the file.
You now execute an RFA-access .FIND
operation.
The current record is set to the record
specified, but the next record is not changed.
Therefore,
when you perform another sequential-access FIND, the search
will begin in the second record cell in the file,
not in
the cell following the one found by RFA.

You use a FIND operation instead of a GET operation for two reasons:
1.

It is faster because the record is not moved to the user
buffer. Although the time required to move a record from one
part of memory to another is very short,
there is no use
expending it if you do not need to.

It does not change the next record in key-access mode or
RFA-access mode.
This allows you to branch off sequential
processing for purposes of updating or deleting records, and
keep your place.

You can use a FIND operation in the following ways:
•

To skip records in sequential-access
successive FIND operations.

•

To establish a random starting point for sequential processing
using RFA-access mode.
You could then initiate successive GET
operations, where the first operation gets the record found by
RFA.

•

To establish
operation.

•

To determine the existence of
access mode.

current

record

4-9

for
record

mode

DELETE
by

using

initiating

UPDATE

or
a

random

RELATIVE FILE APPLICATIONS
4.5.1.5 FLUSH - A FLUSH operation does not affect the record
for the access stream.

4.5.1.6
1.

context

GET - To perform a GET operation on a relative file, RMS-11:
Determines the location of the record in the
to the specified access mode:

file

according

•

In sequential-access mode, location is indicated by:
(a)
the next record pointer,
if the GET operation was not
immediately preceded by a successful FIND operation;
or
(b}
the current record pointer set by an immediately
preceding FIND operation.

•

In key-access mode,
location is determined
by
specified relative record number and match criterion.

•

In RFA-access mode, location is
RFA.

determined

the

specified

Reads the bucket containing the indicated cell from disk into
the task's I/O buffer,
if the bucket is not already in
memory.
The bucket may be in memory if it was required by a
previous operation.
Example:
Your fixed-length records are 50 bytes long;
bucket size is 2 blocks. When you read sequentially through
the file, RMS-11 must request a disk I/O operation every
twentieth GET operation that your program executes.
NOTE
If you have opened a relative file with read-only
access and read/write sharing declarations, each GET
operation causes an I/O operation.

Returns the RFA and the RRN to the program and moves the
record from the I/O buffer to the specified user buffer in
the program -- unless the program is operating in locate
record transfer mode (see Section 4.5.2.2).
1L ~ne cell is
empty or contains a
response depends on the access mode:

deleted

record,

the

•

In sequential-access mode, RMS-11 repeats steps 1 through
3, moving through cells until the MRN is exceeded (ER$MRN)
or the end of the file is reached (ER$EOF).

•

In key-access mode, RMS-11
specified match criterion:
On an equal
ER$RNF.

match,

RMS-11

reacts

according

returns

the

error

code

On a greater-than or greater-than-or-equal
match,
RMS-11 internally adds 1 to the relative record number
and repeats steps 1 through 3, until either the MRN is
exceeded
(ER$MRN)
or the end of the file is reached
( ER$RNF) •

4-10

RELATIVE FILE APPLICATIONS
•

In RFA-access mode, RMS-11 returns the
code:
ER$RNF
No valid record
specified location.

has

appropriate

ever

existed

ER$DEL -- The control byte in the cell
the record in it was deleted.
A GET operation affects the record context for the
follows:

the

indicates

that

access

•

Current record is set to the relative
record read.

•

Next record is set to a relative record number 1
the relative record number for current record.

4.5.1.7
1.

record

error

stream

number

the

higher

than

PUT - To perform a PUT operation on a relative file, RMS-11:
Determines the destination of the record
according to the specified access mode:
•

In sequential-access mode,
indicates the destination.

the

•

In key-access mode, the specified relative
indicates the destination.

file

the

pointer

record
record

number

Determines whether the bucket containing the indicated cell
is in the file.
If it is, RMS-11 goes to the next step.
If
it is not, RMS-11 extends the file until it has enough blocks
for all buckets up to and including the required one. Then,
RMS-11 initializes all newly allocated buckets.

Reads the bucket containing the indicated cell from disk into
the task's I/O buffer, if the bucket is not already in
memory. The bucket may be in memory if it was required by a
previous operation.

Checks the indicated cell:
if it already contains an
existing, valid record, RMS-11 returns error code ER$REX;
otherwise, RMS-11 goes to the next step.
Note that in some cases, you may be
existing, valid record in a cell.
language documentation.

able to update an
See your high-level

Moves the record from the user buffer in the program
task's I/O buffer.

Returns the RFA and the RRN to the program=

Writes the I/O buffer to disk, unless
deferred write (see Section 4.5.3.2).

A PUT operation affects the record context for the
follows:
•

you

have

access

the

specified
stream

For a sequential-access PUT operation:
Current record -- None. Any operation requiring a
record fails at this point.

4-11

current

RELATIVE FILE APPLICATIONS
Next record -- The cell with a
relative record number 1
higher
than the relative record number of the record just
inserted.
•

For a key-access PUT operation:
Current record -- None.
Any operation requiring a
record fails at this point.

current

Next record -- Unchanged.

4.5.1.8 REWIND - A REWIND operation sets the context of the access
stream to the beginning of the relative file.
In doing so, it affects
the record context for the stream as follows:
•

Current record -- None.
Any
record fails at this point.

operation

requiring

current

•

Next record -- Set to the first record cell in the file.

4.5.l.9 UPDATE - In an UPDATE operation, RMS-11 moves the specified
record from the task's user buffer to the I/O buffer, replacing the
current record set by a previous GET or FIND operation.
Then,
RMS-11
writes the bucket over its original location on the disk, unless you
have specified deferred write (see Section 4.5.3.2).
An UPDATE operation requires a valid current record.
Therefore,
an
UPDATE operation should follow a successful GET or FIND operation;
otherwise, RMS-11 returns the error code ER$CUR.
This error does not
affect the original record in the file on disk.
An UPDATE operation affects the current record context for the
stream as follows:
•

Current record -- None.
Any operation
record will fail at this point.

•

Next Record -- Unchanged.

4.5.2

requiring

access
current

Record Transfer Modes

You can manipulate records either
in the I/O buffer or
in your
program's user buffer.
Each of these options is called a record
transfer mode.
You can change record transfer mode at run time,
even
between record operations.
Figure 4-1 illustrates the RMS-11 task
structure.

4.5.2.l Move Mode - Move mode requires that
between the user and I/O buffers:

each

record

copied

•

On GET operations, RMS-11 moves the record from the I/O buffer
to the user buffer before returning control to your program.

On PUT and UPDATE operations, your program assembles the
record to be written into the file in the user buffer and,
during the operations, RMS-11 moves
the data
into the
I/O
buffer before updating the file.

4-12

RELATIVE FILE APPLICATIONS
Move mode is the default record transfer
languages and all file organizations.

mode

for

all

programming

4.5.2.2 Locate Mode - Locate mode enables your program to manipulate
records in the I/O buffer, eliminating the data transfers between it
and the user buffer. However, when you specify locate mode, RMS-11
uses it only when such usage does not compromise data integrity.
Otherwise, RMS-11 uses move mode.
Therefore, your program must still
contain a user buffer.

r;:::-------- - - - - - - - ,
I SIZE DEPENDS ON:

I• NUMBER OF FILES OPENED SIMUL T_ANEOUSLY I
I

j • BUCKET SIZES

~ NUMBE~"._R~rD~C~E~ ~~MS _ _ J

USER BUFFERS

=?
_

I I
VIRTUAL
MEMORY

I
l/O
BUFFERS

PROGRAM

RMS-11

INTERNAL

CONTROL
I STRUCTURES

fSizEDEPENDs oN:-

l•

RMS-11 FUNCTIONS USED

~ ~V~R~~~U~T~E~~E~

ZK-1174-82

Figure 4-1: RMS-11 Task Structure

Example: RMS-11 uses move mode instead of locate mode when a relative_
file is shared.
Example: RMS-11 uses move mode instead of locate mode if you opened a
file indicating you were going to perform UPDATE operations on it.
RMS-ll's use of move mode instead of locate mode is transparent to
your program as long as you use RMS-11 facilities to access the record
data.
For relative files, your program can only perform GET operations in
locate mode.
See your high-level language documentation to determine
whether the language supports locate mode and, if it does, what the
exact programming techniques are.

4-13

RELATIVE FILE APPLICATIONS
4.5.3

I/O Techniques

You can use the following techniques to
record operations.

improve

the

performance

4.5.3.1 Asynchronous Record Operations - Within each access stream,
your program can perform any record operation either synchronously or
asynchronously.
In synchronous operations, RMS-11 returns control to
your program after the operation ends, either successfully or with an
error.
When you execute an asynchronous operation, RMS-11 may return control
to your program before the operation is complete.
The program
continues processing while the physical transfer of data between disk
and memory is carried out. However, you must not initiate another
record operation on that stream until the first operation ends;
otherwise, RMS-11 returns the error code ER$ACT. See your high-level
language documentation for asynchronous techniques.

4.5.3.2 Deferred Write - Normally, each write-type record operation
(DELETE, UPDATE, and PUT) results in a bucket being written to disk.
This convention emphasizes data integrity:
you know that when a
write-type operation has ended successfully, the file reflects that
operation.
However, you can improve the performance of sequential write-type
operations by using deferred write. Basically, deferred write directs
RMS-11 to write a bucket to disk only when RMS-11 must use the I/O
buffer for some other purpose.
NOTE
Deferred write, although not illegal, is
essentially invalidated while a relative
file is being shared by multiple tasks
or streams.
In that environment, every
write-type operation results in an I/O
operation so that:
•

The bucket locked by the previous GET
or
FIND
(for UPDATE and DELETE
operations) or by the PUT operation
can be released.

•

The new data is available
other tasks or streams.

the

Therefore, if you perform sequential write-type operations on a
nonshared relative file, deferred write improves performance. RMS-11
writes out the buffer only when it must read another bucket to
complete an operation.
Example: Your records are 304 bytes long and the bucket size
blocks.
During sequential write-type operations, deferred
causes I/O operations per bucket to drop from 5 to 1.
Deferred write offers little or no benefit to random
operations or read-type operations in any access mode.

4-14

is 3
write

write-type

RELATIVE FILE APPLICATIONS
4.5.3.3 Multiple Buffers - When you open a relative file,
RMS-11
normally allocates 1 bucket-sized I/O buffer in your task's address
space. RMS-11 uses this buffer during record operations.
However,
you can direct RMS-11 to allocate more than the one buffer.
RMS-11 uses any extra buffers to keep, or cache, buckets in memory.
When a record operation requires that--ablicket be read from disk,
RMS-11 checks its cache first.
RMS-11 does not perform an I/O
operation if both of the following are true:
•

The requested bucket is already in memory.

•

That bucket is still valid, that is, the file
and/or the bucket has been kept locked.

not

shared

You do not benefit from multiple buffers during sequential operations.
You can improve performance with multiple buffers during random
operations only if your program accesses the same buckets often.

4.5.3.4 Multiple Access Streams - RMS-11 allows each program
multiple streams on a relative file.

4.5.4

use

performed

File and Directory Operations

The following file
relative files:

and

directory

operations

can

CLOSE
CREATE
DISPLAY
ENTER
ERASE
EXTEND
OPEN
PARSE
REMOVE
RENAME
SEARCH
See your high-level language documentation for a
support provided.

4-15

description

the

CHAPTER 5
INDEXED FILE STRUCTURE AND ACCESS

DIGITAL designed the RMS-11 indexed file organization to
following goals:

achieve

the

•

Content-addressable record access -- Each record in the file
can be located on the basis of the values in designated
portions of the data, called key fields.

•

Uniform random access time -- Each record in the file can be
located with approximately the same number of I/0 operations,
regardless of when it was added to the file.

•

Alternate key capabilities (comply with ANSI COBOL Level 2) -Each record
in the file can be located via more than one key
field.

•

Very good performance on sequential access by primary key -- A
program can sequentially read a reasonably designed indexed
file by primary key almost as fast as it can sequentially read
a sequential file.

•

Good performance on sequential access by alternate keys
Each record in the series can be accessed with (typically) one
to three I/O operations.

•

Unique record address for the life of the file (data base key
concept)
A record
in a file can be located via a unique
identifier
(record file address)
established by the PUT
operation.
The record may be deleted, but its unique
identifier is never reused.

Preserve the state of processing despite a system failure
• Normally,
each logical write operation results in a physical
transfer of data from memory to disk.
Therefore, the file
reflects each record inserted.
However, you can override this
mode with deferred write in some cases.
More importantly, RMS-11 performs record operations so
the following are true:

that

both

•

File corruption is avoided or minimized even if a
failure occurs during a write-type record operation.

•

Even if some
accessed.

corruption

exists,

user

data

NOTE
You should still reorganize your file if
the
system
fails during write-type
processing on an RMS-11 indexed file.
5-1

can

system

still

INDEXED FILE STRUCTURE AND ACCESS
5.1

PHYSICAL FILE STRUCTURE

On disk, an indexed file consists of three kinds of blocks:
•

Prologue -- RMS-11 information about the
attributes and key and area descriptions

file,

including

•

Index -- Index records for primary and alternate keys pointing
the way to a data record

•

Data

Your data records and index data records

The prologue contains information about the keys and areas of the
file.
RMS-11 allocates at least one block for the key descriptors and
at least one block for the area descriptors. RMS-11 uses more blocks
as needed. Size calculations are discussed in Section 6.6.1.
Areas are portions of an indexed file that are treated independently
for initial allocation, extensions, placement, and bucket sizes. Like
subfiles, but invisible to the operating system, areas allow you to
divide indexed files logically into separate units for each index and
for the data records to improve performance; see Section 6.3 for more
information on areas.
In addition, RMS-11 extends the prologue to an integral multiple of
the area 0 bucket size,
if the area 0 bucket size is 2, 4, or 8
blocks. See Section 6.5 for more information on bucket sizes.
The location of the index and data blocks is up to you:
•

If the file is a single area, RMS-11 allocates data and index
blocks in buckets as it needs them;
they are therefore
interspersed throughout the file.

•

If the index and data are set up in separate areas, RMS-11
allocates each type of bucket from the appropriate area;
the
index is therefore set apart physically from the data portion
of the file.

Figure 5-1 illustrates an indexed file both with and without areas.
RMS-11 formats buckets in an indexed file as it requires them for
record storage.
The RMS-11 control bytes are set to their initial
values:
•

14 bytes, beginning with byte O of the bucket
control information.

contain

•

The last byte of the last block duplicates the first
the bucket for checking I/O completion.

bucket
byte

RMS-11 packs index or data records, including record format overhead,
into each bucket, beginning with byte 14, end-to-end and byte-aligned.
Figure 5-2 shows the RMS-11 bucket format.

5-2

INDEXED FILE STRUCTURE AND ACCESS

RECORDS

ALTERNATE

WITHOUT AREAS

WITH AREAS

Pl = PRIMARY INDEX
DR= DATA RECORDS
Al= ALTERNATE INDEX
ZK-1165-82

Figure 5-1: Indexed File with and without Areas

5-3

INDEXED FILE STRUCTURE AND ACCESS

~-----BUCKET BOUNDARIES----

RECORDS

CONTROL FLAGS
LEVEL IN INDEX
- - - - VBN OF NEXT BUCKET IN LEVEL
- - - - - - R E C O R D IDENTIFIER INFORMATION
- - - - - - - - POINTER TO FREE SPACE IN BUCKET
- - - - - - - - - - B U C K E T ADDRESS SAMPLE
- - - - - - - - - - - - A R E A CONTAINING THIS BUCKET
- - - - - - - - - - - - - C H E C K BYTE EQUALS LAST B Y T E - - - -

ZK-1160-82

Figure 5-2: Formatted Bucket

5.2

CONCEPTUAL FILE STRUCTURE

No matter how it is laid out physically,
the indexed file
is
conceptually a prologue plus a group of indexes, one per key.
Each
index consists of horizontal chains of buckets called levels.
Figure
5-3 illustrates this structure as a pyramid.

LEVEL 2

LEVEL 1

LEVEL 0

Figure 5-3: Index as a Pyramid

5-4

INDEXED FILE STRUCTURE AND ACCESS
The lowest level of an index is level O.
The level number is
incremented for each successive (and smaller) level, that is, level 1,
level 2, and so on. The highest level in an index is a single bucket
called the root;
this bucket is the entry point to the index for
random accesses using this key.
Each index has at least two levels (0
and 1) •
The depth of an index is equal to the level number of the root.
An
index-depth relates to the time needed to randomly access any record
in the file via that index.

5.2.1

Data

Level 0 of each index is called the data level;
it consists of data
buckets.
In the primary index, level 0 contains buckets of your data
records.
In the alternate indexes, level 0 buckets contain pointers
to your data records.

5.2.1.l Level 0 of the Primary Index - RMS-11 physically orders data
records by ascending primary key value along the bucket chain.
The
records having the lowest primary key value reside in the first bucket
of the level and the records with the highest primary key values
reside in the last bucket. RMS•ll preserves this order regardless of
the insertion sequence of the· records.
Each bucket in level 0 shares the following properties:
•

The last data record in a bucket has an equal
value than any other record in the bucket.

higher

key

•

The last data record in a bucket has a lower key
the first record in the next bucket in the chain.

value

than

Each bucket thus has a high-key value, located in the last record of
the bucket.
This concept is the core of RMS-11 index file structure.
NOTE
RMS-11 places records with duplicate key
values next to each other on a first-in,
first-out
(FIFO)
basis.
If
these
duplicate records cannot fit in the same
bucket, RMS-11 stores the overflow in a
continuation
bucket.
Continuation
buckets are extensions of
level
0
buckets and,
as such, are not indexed.
This extension storage preserves the
high-key concept.

5.2.1.2 Level 0 of an Alternate Index - Level O, the data level, of
an alternate index· contains secondary index data records (SIDRs).
A
SIDR consists of two elements:
1.

An alternate key value from a data record stored in the
primary data level.
The SIDRs in the data level of each
alternate index are stored in ascending order by this key
value.

5-5

INDEXED FILE STRUCTURE AND ACCESS
2.

One or more pointers to data records in the primary data
level. ~Ultfple pointers cCCur when· you allow duplicates for
the alternate key and records with duplicate values for
the
key actually exist in the file.

Figure 5-4 illustrates the SIDR format.

A. DUPLICATES ALLOWED
8

DUPLICATE
6~E
COUNT
ARRAY
SIDR IDENTIFIER
CONTROL

B. NO DUPLICATES ALLOWED

POINTER

KEY

ARRAY~

VBN

I I I j2£

DATA RECORDjD
FLAG BYTE
POINTER TO PRIMARY
LEVEL 0

ZK-1152-82

Figure 5-4: Format for Secondary Index Data Record

5.2.2

Indexes

Levels 1 and
consist of
guide RMS-11
0. An index

above in an index are called the index levels;
they
index buckets.
Index buckets contain index records that
through the levels to the data records -rn-primary level
record consists of two elements:

The high-key value of a bucket in the next lower level in the
index.
Because RMS-11 arranges these values in ascending
sequence, there is a high-key value for index buckets also.
However,
the last high-key value in the last index bucket of
a level is set to the highest possible key value, rather than
the highest key value in the file.
The associated pointer
references the last bucket in the next lower level.

A pointer to the bucket associated with the high-key value.

Example:
The buckets in level 1 of the primary index contain the
high-key values of the data buckets in level O.
Then, level 2
contains the high-key values from level 1 and so on. Figure 5-5 shows
an example of a primary index.
In other words, each bucket on a given level
is represented by an
index record in the next higher level. Thus, the number of buckets
required on each successive level decreases exponentially until
the
root bucket is reached.
Example:
e

If an index bucket can hold 10 index records, then:
If level O contains 2000 data buckets:
Level 1 contains 200 index buckets
Level 2 contains 20 index buckets

5-6

INDEXED FILE STRUCTURE AND ACCESS
Level 3 contains 2 index buckets
Level 4 contains 1 index bucket

• i f level 0 contains 10,000 data buckets:
Level
Level
Level
Level

1 contains
2 contains
3 contains
4 contains

1000 index buckets
100 index buckets
10 index buckets
1 index bucket

LEVEL 3
(ROOT)

LEVEL 2

LEVEL 1

LEVEL 0

MAXIMUM KEY VALUE

IABLE ELM AVE. 3621
I

I JONES, MAIN ST. 1.

ISMITH,HOLT RD,589,INS I

IYOOS,FIRST ST.,97821
ZK-1151-82

Figure 5-5: Example of a Primary Index

5.2.3

Random Access Using the RMS-11 Indexed File Structure

The following steps show how RMS-11 uses the indexed file structure to
execute a random access operation. These steps constitute a process
called "follow the index."
1.

RMS-11 examines memory-resident index descriptors to find the
location of the root for the specified index.
Note that the
root can be cached (see Section 7.4.3), eliminating the
I/O
operation to read the root in the next step.

RMS-11 reads the root and scans for the first value greater
than or equal to the key value specified when the operation
was initiated.
If all else fails, the search will find
the
highest possible key value in the last index record.

RMS-11 reads the bucket indicated by the pointer associated
with the selected key value and scans for the first key value
greater than or equal to the value specified.
RMS-11 repeats
this step through the levels until level 0 is reached.

5-7

INDEXED FILE STRUCTURE AND ACCESS
Example:

Refer to Figure 5-5 for this example.

The specified primary key value is "YOOS."
RMS-11 determines the VBNs of the root bucket from the memory-resident
index descriptors and requests the file control processor to read
those blocks into an I/O buffer.
RMS-11 scans the index records
in
the root.
The first key value equal to or greater than "YOOS" is the
maximum key value in the last record.
RMS-11 uses the bucket pointer in this index record to request another
I/O operation. The file control processor reads the specified blocks
into the I/0 buffer, and RMS-11 scans them looking for a key value
equal to or greater than "YOOS." Again, it finds no qualifying key
value until the last record in the bucket, which contains the maximum
key value. This index record points to a level 1 bucket.
Upon RMS-ll's request, the file control processor brings the indicated
bucket into memory.
RMS-11 searches the bucket, terminating with the
last record in the bucket, which contains the maximum key value.
The file control processor reads
RMS-ll's request.

the

indicated

level

bucket

Why this Structure?

5.2.4

Mechanical data storage devices make I/O operations the slowest part
of file processing.
Ideally, a file is read into memory when it is
opened and maintained there, without additional I/0 operations, until
the file is closed. Some very small files allow this approach and are
handled most efficiently by your own search techniques rather than by
RMS-ll's indexing facilities.
However, most indexed files are very much larger than the memory
available for data buffering. Such files are therefore partitioned
into pieces that can be read to memory.
RMS-11 calls these pieces
buckets.
By definition, one I/O operation is required to access one
bucket.
If no index to the data exists, a task must scan sequentially through
the buckets of a file to find a specific record.
Such a search, on
the average, accesses half the buckets in the file.
Figure 5-6 plots
the time curves fer various
You can optimize nonindexed access by:
•

Ordering the records by a key value

•

Using a binary search technique

Then, the number of accesses required to find a record approaches log2
of the total number of buckets (see Figure 5-6). This better, but
still mediocre, speed is realized on one of perhaps many keys.
The RMS-11 indexed structure uses buckets so that your programs can
handle files more efficiently.
In most cases, RMS-11 uses n+l I/O
operations to locate a record by primary key, where n is the depth of
the file's primary index.

5-8

INDEXED FILE STRUCTURE AND ACCESS
In a small file, this technique is not appreciably faster than a
sequential scan. However, given typical key sizes, a primary index of
depth 3 can represent from 1,000 to 125,000 buckets of data records,
using only single-block buckets.
Normally, four disk accesses are
needed to get any record by primary key value.

Example:

You want to search 50 buckets of data for records with
specific primary key values.
The average number of buckets you read
during each search depends on your search technique (see Figure 5-6):
25.5

5.3

buckets for a nonindexed search of unsorted records
buckets for a binary search of records sorted by primary
key
buckets for an RMS-11 indexed search

PROCEDURES FOR PERFORMING RANDOM RECORD OPERATIONS

The procedures for performing random record operations on indexed
files depend on the circumstances for the individual operation, the
file's design, and whether alternate indexes must be updated.

RMS-11 PRIMARY INDEX SEARCH,
10 RECORDS PER INDEX BUCKET•
BINARY SEARCH FOR PRIMARY KEY,
RECORDS ORDERED BY PRIMARY KEY

NUMBER OF
PRIMARY DATA
BUCKETS

NUMBER OF BUCKETS READ IN SEARCH

*LINE SHIFTS TO LEFT AND BREAK RIGHT MOVES UP
AS NUMBER OF INDEX RECORDS PER BUCKET GOES UP
ASSUMING OPTIMAL PACKING
'
ZK-1159-82

Figure 5-6: Search Time Curves
5-9

INDEXED FILE STRUCTURE AND ACCESS
5.3.1

Writing a Record

When your program initiates a PUT operation, RMS-11 moves the data
from the task to the proper bucket in level 0 of the primary index and
updates all indexes involved with the record.
This process can be
simple,
requiring minimal
I/O operations.
It can also be complex,
requiring more procedures and data transfers. The complexity depends
on whether there is enough room for the new record in its data bucket.

5.3.1.1 Simplest Case - In the simplest PUT operation, RMS-11 finds
room in the target data bucket to insert the record.
To execute the
operation, RMS-11 performs the following steps:
1.

Determines the value
record.

the

primary

key

field

from

Follows the primary index to the proper level 0 bucket.

Reads the level 0 bucket and sequentially scans for the first
record with a primary key value greater than the specified
value.
RMS-11 then establishes a position before that
record, or after the last existing record in the bucket if:
•

The key values are equal.

•

The first record in the next data bucket has a higher
value.

the

key

Compresses deleted records.
RMS-11 can reuse bytes in a
deleted record depending on the record format and whether you
allow duplicates in the primary key field.
Section 6.2.5
discusses reusing space from deleted records.

Determines whether the record to be inserted
bucket (in this simplest case, it does).

No primary
Inserts the record at the established position.
index buckets are updated since no high-key value has
changed.

If there are alternate keys, updates those indexes, using the
following sequence of steps for each one:

fits

the

•

Follows the alternate index to the proper level 0 bucket.

•

Reads the level 0 bucket and sequentially
key value specified in the record:

scans

for

the

If a value higher than the one specified is found,
inserts a SIDR for the record before the SIDR for the
higher value.
If a match is found, determines whether duplicates
allowed for the alternate key:

are

If duplicates are allowed, RMS-11 follows the duplicate
pointer array in the SIDR to the end, then inserts a
pointer to the newly inserted record.
This procedure

preserves

the

first-in,

first-out convention.

the last alternate key,
RMS-11
completion code to the program.

5-10

returns

After

successful

INDEXED FILE STRUCTURE AND ACCESS
If duplicates are not allowed, RMS-11 returns to

level

O of the primary index, flags the newly inserted record
as deleted, logically removing it; and returns an error
code to the program.
Example:

Refer to Figure 5-5 for this example.

RMS-11 examines the record in the user buffer of the record access
stream initiating the PUT operation. The value in the primary key
field is "JACKSON. 11 RMS-11 locates the primary root and requests the
file control processor to read the bucket into the I/O buffer
associated with the stream. When that I/0 operation completes, RMS-11
scans the bucket, looking for a key value equal to or greater than
11
JACKSON. 11 It finds "JONES. 11
RMS-11 requests the bucket indicated by the pointer in the 11 JONES 11
index record.
When RMS-11 scans this level 2 bucket, it finds that
"JONES 11 again ends the search. Following the pointer in this index
record, RMS-11 requests another bucket.
Its search of the level 1
bucket ends in another 11 JONES" index record.
RMS-11 requests the level 0 bucket indicated by this last index
record.
It finds that a data record with a primary key value of
"JONES" is the only occupant of the bucket.
There are no deleted
records to compress, so RMS-11 writes the 11 JACKSON" record before the
"JONES" data record, moving the "JACKSON" record down in the bucket.
There are no alternate keys.
code to the program.

RMS-11 returns a

successful

completion

5.3.1.2 Bucket Splitting - If there is not enough room in the target
data bucket for the record,
RMS-11 allocates a new bucket and
reorganizes the records in the old one between the two buckets.
This
procedure is called bucket splitting.
Bucket splitting is identical with the simplest case (Section 5.3.1.1)
to step 5 where RMS-11 determines whether the new record fits in the
bucket. When there is not enough room, RMS-11 does the following:
1.

Reads the appropriate area descriptor from the file prologue.
If enough blocks for a bucket are allocated for the area,
RMS-11 formats the blocks into a bucket and updates the area
descriptor to reflect the new bucket. Otherwise, RMS-11
requests the operating system to allocate enough blocks, and
then formats them into a bucket and updates the area
descriptor.

Splits the target bucket at the point where the record should
be inserted.
RMS-11 moves the records in the high portion of
the bucket into the new bucket; these records have primary
lei::.u
1 ni::.c:: hi
nhi::.Y t-h;::in
·--.z. u;::a
. . _____
... _J··-_. . - . . . t-h/'\c::o
_. . . . ., __ /'\.f
-- +-ho
-··- nor.J
. . . - ........ -"'-"'" . . "-&•
Yt:>,...l'\YA

5-11

INDEXED FILE STRUCTURE AND ACCESS
NOTE
When RMS-11 moves a record between buckets, it marks
the
record's
original
location with a
record
reference vector (RRV) • An RRV is a copy of the
record's
header
(both contain 7 bytes).
RRVs
preserve alternate key and RFA access, holding the
original location of the record and pointing to its
current location.
Only one RRV is created for
a
record:
if the record moves again, RMS-11 updates
the RRV with the record's new location.
Since the original location of a record
is filled,
either with the record or a pointer to that record,
RMS-11 does not have to update alternate indexes
every time a record moves.
This convention means one
extra I/O operation may be needed to find or get a
record via an alternate key, but it prevents a
complex and costly index update for each bucket
split.
3.

Inserts the data record in the original target bucket.
If
the record will not fit,
RMS-11
inserts it into the new
bucket.
If the record will not fit there either, RMS-11 will
create another bucket (see step 1) and put the record there.

Updates
the
bucket (s).

Returns to the primary root bucket and follows the index to
the level 1 index bucket that points to the data bucket that
split.

Inserts index record(s) for the new data bucket(s).
If the
index bucket splits, RMS-11 uses a procedure similar to this
to move the index records and update the next higher level of
the index. Splitting can occur all the way to the root where
a new root is created and the file prologue updated.

If there are alternate keys, RMS-11 updates those indexes as
descr1oed
in step 7 of the simplest case (Section 5.3.1.1).
Bucket splitting can occur in alternate indexes also.

level

bucket

chain

include

the

new

5.3.1.3 Incremental Reorganization - The process of inserting each
data record where it belongs in level 0 and updating the indexes when
RMS-11 inserts the record is called incremental reorganization of the
file.
Incremental reorganization has the following advantages:
•

It eliminates reorganization periods where special software
incorporates overflow areas into the main file and that file
is not available for processing

•

It ensures equal access time to old and new records

•

It enables performance on sequential access by primary key to
approach the speed of sequential access to a sequential file

5-12

INDEXED FILE STRUCTURE AND ACCESS
This process has its costs: additional I/O operations occur when a
bucket splits.
But with good file design and file loading, bucket
splitting (and the time for each bucket split) is minimal. Chapter 6
discusses these considerations in detail.

5.3.2

Getting and/or Finding a Record

To execute a key-access GET or FIND
following steps:
1.

operation,

RMS-11

performs

the

Determines from ~ne instruction initiating the operation
following criteria:

the

•

Key of reference, indicating which index to search
which key field within the data record to examine

and

•

Value to find

•

Match criterion (equal to, greater than, or both)

•

Number of characters to match

Follows the index to the proper level 0 bucket.

Reads the level 0 bucket, sequentially scanning for the first
record with a value in the specified key field that matches
the specified value according to the match criterion.
This
search can continue into other buckets:
•

If no such record is found, RMS-11 returns an error code.

•

If such a record is found, RMS-11:
Determines which index has been read:
If it is the primary index, RMS-11
step.

goes

the

If it is an alternate index, the record located is a
SIDR.
RMS-11 follows the SIDR pointer to the primary
level 0 data record.
For a GET operation only, moves the record to the user
buffer associated with the access stream performing the
operation.
Sets the current context for the stream performing this
operation.
The effect of each record operation on
context is described in Section 7.2.
Returns a successful completion code.

Example:

Refer to Figure 5-5 for this example.

RMS-11 determines the key (and index) of reference and the value to
find ;from the instruction initiating the operation.
In this case,
they are the primary key (key 0) and "ABI."

5-13

INDEXED FILE STRUCTURE AND ACCESS
RMS-11 locates the primary root and requests the file control
processor to reaa ~ne bucket into an I/O buffer.
RMS-11 sequentially
scans the root for an index record whose key value is equal to or
greater than "ABI." It finds "ABRAM."
RMS-11 requests the bucket indicated by the pointer in the "ABRAM"
index record.
When RMS-11 searches this level 2 bucket, it finds an
index record containing the key value "ABNER." Following the pointer
in this index record, RMS-11 requests another bucket.
The search of
the level 1 bucket ends in the key value "ABLE."
RMS-11 requests the level 0 bucket indicated by this last index
record.
RMS-11 changes its search criteria to that specified in the
initiating instruction:
it looks for a record where the first 3 bytes
of the primary key field equal "ABI." Since the only record in the
bucket contains "ABLE" in its primary key field, RMS-11 cannot satisfy
the search requirements.
It returns a "record not found" error code
to the program.

5.3.3

Updating a Record

RMS-11 requires an UPDATE operation to be preceded by a GET or FIND
operation, although some high-level languages hide this prerequisite.
To execute an UPDATE operation, RMS-11 performs the following steps:
1.

Locates the key fields of the revised record in the user
buffer associated with the access stream performing the
operation. RMS-11 compares those key values with the values
in the current record:
•

If the primary key value changed, RMS-11 returns an
code.

•

If an alternate key value changed, RMS-11
you allowed changes for that key:

checks

error
whether

If not, RMS-11 returns an error code.
If so, RMS-11 continues processing.
2.

For each alternate key where the key value changed, RMS-11
performs the following steps to delete the pre-update value
from the alternate index:
•

Reads the data bucket containing the
that bucket is not in memory.

current

•

Saves the pre-update alternate key value from the
record.

current

•

Follows the index to the level 0 bucket that
contain the SIDR for the pre-update key value.

should

•

Reads the level 0 bucket and sequentially
pre-update key value:

scans

If a value higher than the one specified
RMS-11 goes to the next alternate index.

5-14

record,

for
is

the

found,

INDEXED FILE STRUCTURE AND ACCESS
Example:
RMS-11 scans a bucket, searching for a
pre-update key value of "D." It finds a record with a
key value of "E." Since "E" is greater than "D," RMS-11
ends the search and this step in the procedure.
If a match is found, RMS-11 scans the duplicate pointer
array in the SIDR to the entry for the record being
updated and flags it as deleted.
NOTE
To allow keys to change, RMS-11 requires that you
also allow duplicates.
Therefore, if you allow
alternate key values to change, there is
a
duplicate pointer array in the SIDR for each key
value.
However, you should
refer
to
your
high-level language documentation for specific
information on your compiler's implementation of
this capability.
3.

Reads the data bucket containing the current record, if that
bucket is not in memory. RMS-11 replaces the current record
in the I/O buffer with the updated version in the user
buffer.

Writes the bucket to the file.

For each alternate key where the key value changed, RMS-11
performs the following steps to insert the post-update value
in the alternate index:
•

Reads the data bucket containing the
that bucket is not in memory.

current

record,

•

Follows the index to the level 0
contain the post-update key value.

bucket

that

•

Reads the level 0 bucket and sequentially
post-update key value:

scans

should
for

the

If a value higher than the one specified is found,
RMS-11 inserts a SIDR for the new record before the
SIDR for the higher value.
If a match is found,
RMS-11 follows the duplicate
pointer array in the SIDR to the end, then inserts a
pointer to the new record.
After the last alternate key is updated, RMS-11 returns
successful completion code to the program.

5.3.4

Deleting a Record

RMS-11 requires a DELETE operation to be preceded by a GET or FIND
operation, although some high-level languages hide this prerequisite.

5-15

INDEXED FILE STRUCTURE AND ACCESS
To execute a DELETE operation, RMS-11 performs the following steps:
1.

If there are alternate keys, RMS-11 updates those indexes
follows, using the same sequence of steps for each:

•

Reads the data bucket containing the
that bucket is not in memory.

•

Follows the index to the level 0 bucket that should
contain the SIDR for the key value in the deleted record.

•

Reads the level 0 bucket and sequentially
specified key value:

current

record,

scans

for

If a value higher than the one specified is found,
goes to the next alternate index, if any.
If a match is found, RMS-11 determines
allowed duplicates:

whether

the

RMS-11

you

have

If so, RMS-11 follows the duplicate pointer array in
the SIDR to the entry for the record being deleted and
flags it as deleted.
If not, RMS-11 deletes the SIDR.
2.

Reads the data bucket containing the current record, if
bucket is not in memory.

that

Changes the flag byte in the header of the current record
indicate that the current record is deleted.

Writes the bucket to the file.

If the record has moved, reads the level 0 bucket containing
the RRV.
RMS-11 changes the flag byte in the RRV to indicate
that the record is deleted.

Writes the bucket to the file.

RMS-11 does not compress a deleted record until it needs space to
insert another user data record into the bucket (see Section 5.3.1).
RMS-11 does not compress deleted RRVs.
NOTE
RMS-11 does not modify or reduce any
index structure or allocation during a
DELETE operation.

5.4

PROCEDURES FOR PERFORMING SEQUENTIAL RECORD OPERATIONS

Your program can use sequential access mode to perform
record operations:
FIND
GET
PUT

5-16

the

following

INDEXED FILE STRUCTURE AND ACCESS
During sequential-access GET and FIND operations, RMS-11 does not
usually read an index to locate the specified record.
Instead, RMS-11
uses the record context for the stream performing the operation to
identify the proper data bucket.
For FIND operations, RMS-11 uses the next record pointer to identify
the target bucket.
For GET operations, RMS-11 uses the next record
pointer, if the GET operation was not immediately preceded by a
successful FIND operation.
The current record pointer is used if the
GET operation was immediately preceded by a successful FIND operation.
Next, RMS-11 requests the file control processor to move the target
bucket into the I/O buffer, if that bucket is not in memory.
If it
has requested a SIDR bucket, RMS-11 then follows the appropriate
pointer to the user data record.
During sequential-access PUT operations, RMS-11 compares the primary
key value of the specified record with the primary key value of the
last record written:

5.5

•

If the specified record's primary key value is equal to or
greater than the last record's primary key value, RMS-11
performs a key-access PUT operation
(described in Section
5.3.1).

•

If the specified record's primary key value is less than the
last reco£d's primary key value, RMS-11 returns an error code
to the program.

I/O COST OF PERFORMING RECORD OPERATIONS

Table 5-1 provides simple algorithms for predicting the number of
operations any RMS-11 record operation requires:

I/O

•

The value n = index depth of the indicated key;
do not necessarily have the same depth.

•

Algorithms do not include I/O operations caused by program or
RMS-11
overlays, operating system overhead, or by file
extensions (see Chapter 8).

5-17

all

indexes

INDEXED FILE STRUCTURE AND ACCESS
Table 5-1: I/O Cost of Performing Record Operations

Record Operation

Primary
Key

Each
Alternate
Key

Key-access GET or FIND
Record in original location
RRV in original location (record moved)

n+l
n+l

n+2
n+3

Sequential-access GET or FIND

0-2 a

PUT
Simplest case
Split in data level
Bucket split up entire index

n+2
2n+6 c
(n**2+1ln+20)/2 d

n+2
2n+6 c
(n**2+1ln+20) /2 d

UPDATE e
Alternate key value did not change
Alternate key value changed

1
1

2(n+2)f

DELETE
Record in original location
RRV in original location (record moved)

1
3

aBreaks down to:

0 or 1
0 or 1

I/O to position to current record
I/O to locate next record

bBreaks down to:

0 or l
0 or 1
1 or 2

I/0 to position to SIDR for current record
I/O to locate SIDR fo~ next record
I/Os to retrieve user data record

cBreaks down to:
n+2
n+l
3

I/Os to read and write the old bucket
I/Os to read and write the level 1 index bucket
I/Os to write the new bucket and update the area
descriptor in the prologue

dBreaks down to:
(n+2)+(n+l)+n+(n-l)+ ••• +3+2
I/Os to return to the primary root and read and
write updated buckets from level 0 to the root
3(n+l)
5

I/Os for each bucket split (see footnote c)
I/Os to create the new root

evalues assume record
splitting.

length

does

not

change

and

cause

bucket

fn+2 if either the old or new key value does not belong in the
index;
for example, the field contains the null key value
defined for the key, or a variable-length record does not
contain the whole key field.
gValue is different if one of the following is true:
•

You specified the "fast delete" option
(available in MACR0-11
only) when you initiated the DELETE operation. Then, RMS-11 does
not update alternate indexes in which duplicate keys are allowed.

RMS-11 has to scan a long duplicate array into one or more
• continuation
buckets. Then, one I/O operation is needed for each
additional bucket.

5-18

CHAPTER 6
INDEXED FILE DESIGN

Indexed file design ranges from the basic elements of your application
(record definition and key selection) to the structure of the file and
to the methods used to put the data into the file.
This range
includes:

6.1

Record size

Key selection

Areas

Placement control

Bucket size

Allocation

Population techniques

RECORD SIZE

You can use only fixed- and variable-length records in RMS-11
files.
RMS-11 calculates length (RL), in bytes, as follows:
RL

indexed

7+RFO+DS

where:
7

is bytes for RMS-11 record header

RFO

is bytes for record format overhead:

is bytes of data

0 for fixed, 2 for variable

Set your record size to reflect application requirements;
do not
adjust it to fit bucket size. For instance, if you are using 1-block
buckets, you should not, if you can avoid it, set a record length so
the records just fit into the buckets:
512
-15

bytes in a block
bytes of indexed file overhead per bucket

497
- 7

bytes left for records
bytes for the record header

490

bytes left for the data and record format overhead

6-1

INDEXED FILE DESIGN

This calculation seems ideal at first.
However, when the record moves
during a bucKeL split or RMS-11 deletes the record, and some RMS-11
overhead is left in the bucket, a normal data record cannot fit:
the
bucket is essentially useless, with up to 490 bytes of unused space.
If your application requires 490-byte records, you should use them,
keeping the preceding limitation in mind and, perhaps, choosing a
different bucket size.
NOTE
Records in an indexed file cannot span
buckets and bucket sizes are limited by
the operating system to 32
blocks.
Therefore,
the maximum record size,
including overhead, is 16,369 bytes.

6.2

KEY SELECTION

A file's keys can take up significant space in an indexed file and can
have a significant effect on the number of I/O operations needed to
access the file.
During key selection, you should consider the
following:

• Number of keys
• Key data type
• Key size
• Position of key in record
• Key characteristics
6.2.1

Number of Keys

You can specify from 1 to 255 keys for an indexed file:
•

One primary key that RMS-11 requires for every indexed file

•

254 alternate keys

There are overhead costs in key specification: For each key specified
in an indexed file, RMS-11 builds an index.
Since RMS-11 requires a
primary key, you must accept that key's index overhead, but you should
consider the cost before specifying an alternate key for the file:
•

RMS-11 updates alternate indexes when your program:
Puts a new record into the file
Updates a record in the file and the alternate
change

key

values

Deletes an existing record
The time required for this update relates to the number of I/O
operations needed to follow each alternate index from the root
to level O, to change or insert the SIDR, and to rewrite the
bucket.
RMS-11 can require additional time if one or mo~e
buckets in the index split.
6-2

INDEXED FILE DESIGN
•

An index takes room in the file.
You can estimate
space for an alternate index (see Section 6.6.1).

the

disk

Whether the cost of each alternate key is bearable depends on your
application.
If the primary purpose of the application is to write,
update, or delete records, each alternate key will noticeably burden
the operations;
therefore, the number of alternate keys should be
kept to the minimum.
Rarely used alternate access paths call for a
separate program that sorts by the desired nonkey field and then
processes the data.
However, if the primary purpose of your application is to get records
from the file, then alternate keys do not burden processing.
In fact,
alternate keys give flexibility to information retrieval.
However,
the cost of the extra keys is borne on those few occasions when
records are added to the file.

6.2.2

Key Data Types

Each key in an indexed file can be one of the following data types:
String
2-byte
4-byte
2-byte
4-byte
Packed

signed integer
signed integer
unsigned binary
unsigned binary
decimal

6.2.2.l String Type - RMS-11 interprets each character of the key in
a byte by its binary contents.
Permissible values are not limited to
valid ASCII codes.
Example:

The key value "RMS-11" is represented as follows:

0
0

MOST SIGNIFICANT BYTE = R
=M

= 1

1
ZK-1191-82

The first (lowest-addressed) byte of the key is the most significant
byte of a string key for collating purposes. RMS-11 compares primary
keys byte-by-byte, first-to-last, when it determines where the record
should be placed in the file.
The maximum key value is all bits in each byte set to 1 (octal 377).
There is a cost in the number of bytes specified as the key length.
For example, if you specify a key length of 12, each representation of
the key in the data record and in the index takes 12 bytes.
6-3

INDEXED FILE DESIGN
6.2.2.2

Two-Byte Signed Integer Type - Each key requires
RMS-11 interprets the data in the following format:

bytes;

LEAST SIGNIFICANT BYTE

I:::::::

MOST SIGNIFICANT BYTE

SIGN BIT
ZK-1192-82

NOTE
The least significant byte of an integer
or binary key is the byte with the
lowest address.
Significance increases
with address. Within a byte, the lowest
significant
bit
is
bit
O,
and
significance increases with position.
See your PDP-11 Processor Handbook.
A 2-byte signed integer
through +32,767.

can

represent

the

decimal

values

-32,768

Maximum key value is +32,767.
The cost in key size is 2 bytes per representation.

6.2.2.3

Four-Byte Signed Integer Type - Each key requires
RMS-11 interprets the data in the following format:

,...

...,

LEAST SIGNIFICANT BYTE

II : : : : : : :

MOST SIGNIFICANT BYTE

SIGN BIT
ZK-1193-82

6-4

bytes;

INDEXED FILE DESIGN
A 4-byte signed
integer
can
represent
-2,147,483,648 through +2,147,483,647.

the

decimal

values

Maximum key value is +2,147,483,647.
The cost in key size is 4 bytes per representation.

6.2.2.4 Two-Byte Unsigned Binary Type - Each key requires
RMS-11 interprets the data in the following format:
7

bytes;

0
LEAST SIGNIFICANT BYTE

:: ::: ::

MOST SIGNIFICANT BYTE
ZK-1194-82

A 2-byte unsigned binary value can

represent

the

decimal

values

through +65,535.
Maximum key value is 65,535.
The cost in key size is 2 bytes per representation.

6.2.2.5 Four-Byte Unsigned Binary Type - Each key requires
RMS-11 interprets the data in the following format:

bytes;

LEAST SIGNIFICANT BYTE

MOST SIGNIFICANT BYTE
ZK-1195-82

A 4-byte unsigned binary value can

represent

the

through +4,294,967,295.
Maximum key value is 4,294,967,295.
The cost in key size is 4 bytes per representation.

6-5

decimal

values

INDEXED FILE DESIGN
6.2.2.6

Packed Decimal Type - RMS-11 recognizes 2 decimal digits of
in eacu byte except
the last.
The key format takes the
following form:

-cue

Key

0
01

A+ 1

SIGN

A+ N - 1
ZK-1196-82

where:
A

is an address:

A, A+l, •••

are increasing {byte) addresses.

Dl-DI

are decimal digits: Dl is the most significant digit and DI is
the least significant digit.

SIGN

has a value of 10 through 15: + is represented by
14, or 15; and - is represented by an 11 or 13.

is the length of the key in bytes (maximum of 16)

is the length of the digit string, an odd number in
of 1 through 31, where I
2N - 1

10,

12,

the

range

their

Maximum key value is 99 in each byte with the sign positive.

6.2.3

Key Size

Keys for indexed files have length restrictions
data types.
Table 6~1 lists these restrictions.

according

Table 6-1: Key Data Types
Data Type

Length (bytes)

String
15-bit signed integer
31-bit signed integer
16-bit unsigned binary
32-bit unsigned binary
packed decimal

1-255
2
4
2
4

1-16

The cost of each key's size is borne in the data record and
in
index:
RMS-11 stores an entire key value in each index record.

6-6

the

INDEXED FILE DESIGN
Position of Key in Record

6.2.4

You can locate any key anywhere in the record:
•

Alternate keys can precede the primary key.

•

Keys can overlap each other.
Note, however, that COBOL-81, in
keeping with the ANSI standard, does not permit more than one
key to start at the same position. The standard calls this
leftmost correspondencee

You benefit from careful placement of keys within the record:
•

Deleting a record -- When you allow duplicates in the primary
key of variable-length records, RMS-11 compresses a deleted
record by removing all data except:
The record header
Enough of the record to contain the primary key
Therefore, you can optimize DELETE operations if you place the
primary key at the beginning of the record. The closer the
key is to the beginning of the record and the shorter the key,
the fewer overhead bytes remain in the file.
However, if you have fixed-length records or do not allow
primary key duplicates, the position of that key in the record
is not significant. See Section 6.2.5.1 for more information
on duplicates.

a record
You can optimize PUT operations for
• Writing
variable-length records, by placing alternate keys at the end
of the record. Then, if no valid data is present in an
you can shorten the record to exclude
alternate key field,
that field, thus reducing the record space in the data level
as well as eliminating a reference to that record in the
alternate index.
You can segment string keys;
all other key data types must be
contiguous bytes. You can specify up to eight segments in one string
key, each segment with its own length;
the total of the lengths
cannot exceed 255 bytes.
Note that some high-level languages do not
make this capability available;
see your
high-level
language
documentation.
RMS-11 concatenates the segments you specify before performing any
operations requiring a value for the key.
RMS-11 defines a segment by
byte position within the record and length in bytes.
Therefore, the
key segments you define with either a MACR0-11 program or RMSDES do
not have to align with the data fields you define within the records:
RMS-11 has no knowledge of the form of such files.
Examole:
You have an inventory application with a master product
file.
Within the product records, you have fields for vendor number,
vendor's part code, and your part number, among others.
You can
define the following keys for the file regardless of the placement of
the fields.
Primary key =
Alternate key
Alternate key
Alternate key

vendor number + vendor part code
1
vendor number + your part number
2
vendor number
3
your part number

6-7

INDEXED FILE DESIGN
E'or cost, see the preceding considerations_about the placement of keys
within a record, knowing that a key consists of aii segments.

6.2.5

Key Characteristics

Key characteristics include:
•

Duplicates

•

Changes

•

Null key

Characteristics are restricted according to key number:
Characteristics

Primary Key (0)

Alternate Keys (l+)

Duplicates
Changes
Null key

Allowed
Not allowed
Not allowed

Allowed
Allowed
Allowed

The combination of changes and duplicates is also
number:

restricted

Combination

Primary Key (0)

Alternate Keys (1+)

CHG+DUP
CHG+NODUP
NOCHG+DUP
NOCHG+NODUP

Error
Error
Allowed
Allowed

Allowed
Error
Allowed
Allowed

key

NOTE
COBOL-81
allows
the
CHG+NODUP
combination
for alternate keys.
To
enable this option,
the COBOL-81 OTS
uses a hidden FIND operation to check on
duplicates each time an alternate key
value changes on an UPDATE operation
(REWRITE in COBOL-81).

6.2.5.1 Duplicates - If duplicates are allowed for a key, more than
one record can have the same value in that key field.
The overhead
costs are:
•

File space -- Duplicates have little effect on space usage as
long as records are not frequently updated with changing key
values or deleted.
If anything, records with duplicate key
values
are
stored
more efficiently than records with
nonduplicate values:
fewer
index records are required to
cover data records with duplicate primary keys.
In alternate indexes, one SIDR with one representation of the
key value is needed to cover multiple data records with the
same value in the key field.

•

Writing a record -- RMS-11 stores records with duplicate key
values for first-in, first-out access. Writing (and updating)
records containing duplicate key values takes more time as the
number of duplicates builds up.

6-8

INDEXED FILE DESIGN
A PUT operation can fail because duplicates are not allowed
for one of the keys.
If this is the primary key, RMS-11 has
wasted little time since it has performed only the I/O
operations to find the previous record with that value in the
key field.
However, if you allowed no duplicates in one of the
keys, RMS-11:

•

alternate

Updates the primary index, including the data level.

Updates the preceding alternate indexes.

Discovers that it cannot insert the record
record already exists with that key value.

Reverses the actions it has taken, removing all updates
from the indexes it has already rewritten.
Entries made
in SIDR duplicate arrays are flagged as deleted and not
compressed out of existence.
However, RMS-11 cannot
reverse bucket splits.

Returns an error code.

because

Deleting a record -- If you do not allow duplicate values for
the primary key,
RMS-11 compresses a deleted record to a
2-byte indicator when it performs a
DELETE
operation.
However,
if you allow duplicate values for the primary key,
RMS-11 keeps enough of the record to contain the entire
primary key:
1.

If the format is fixed, the entire record remains
file.

the

If the format is variable, enough of a record
place to hold the entire primary key.

remains

•

If you do not allow duplicate values for an alternate key,
RMS-11 removes the SIDR when it deletes the data record.
However, if duplicates are allowed, the pointer remains in the
SIDR array with the delete flag set.

•

Updating a record -- If you allow duplicate values for the
primary key, the length of a variable record cannot be changed
during an UPDATE operation.
In addition, updating records
containing duplicate key values takes more time as the number
of duplicates builds up.
Finally, the SIDR pointers for
deleted records are flagged as deleted, but not removed from
the duplicate array.

•

Summary -- Duplicates are not costly for write-type operations
unless there are too many of them.
Pick a key field that
minimizes duplicates.
Example: Fields where there are only two choices for entries,
such as sex, are not good candidates for key fields.

6.2.5.2 Changes - The value of a primary key field cannot change
during an UPDATE operation; however, you can allow the value in any
alternate key field to change if you are willing to allow duplicate
values in that key.

6-9

INDEXED FILE DESIGN
During any UPDATE operation, RMS-11 checks the characteristics of all
keys and compares the new key values (in the record about to be
rewritten) with the old values: if you do not allow changes in a key
field, but changes have been made, RMS-11 immediately terminates the
UPDATE operation with an error code.
Cost: If an alternate key value changes during an UPDATE operation,
RMS-11 must trace the old SIDR and delete it, then insert the new one,
starting with the root of the index for both processes. If the data
does not change, however, RMS-11 does not update the alternate index.

6.2.5.3 Null Key - You can specify the null key characteristic for
any alternate key.
If RMS-11 finds that an alternate key field is
filled with the null key value specified for that key, it does not
insert an entry into the index for the record being written.
Zero is the null key value for the numeric key data types (integers,
binaries, and packed decimal). The null key character for string keys
can be any octal value (000 through 377) including an ASCII character:
if all bytes in the key field contain this value, the key is
considered null.
Cost: The use of a null key value can reduce the disk space that an
alternate index occupies, but it also precludes accessing those
records not entered in the index via that alternate key.

6.3

AREAS

You should divide an indexed file into areas.
the file that RMS-11 treats as an entity for:
•

Initial allocation

•

Extensions

•

Bucket size

•

Placement on disk

An area is a portion of

Areas allow you to gather logical elements of the file into groups of
continuous ranges of VBNs. These VBNs can be mapped onto a contiguous
set of logical blocks on disk. This tight sequence of VBNs is lost
when RMS-11 extends an area.
NOTE
Unless you completely allocate each area
when you create the indexed file, the
division of the file into areas may not
improve performance.
Areas can be set up for:
•

Primary index level 0 (the data records)

•

Primary index level

•

Primary index levels 2 and greater (the rest of the index)

{the lowest index level)

6-10

INDEXED FILE DESIGN
•

Alternate index level 0 (SIDRs)

•

Alternate index level 1 {the lowest index level)

•

Alternate index levels 2 and greater (the rest of the index)

Dividing a file into areas primarily saves I/O time. As explained in
Section 5.1, in a single-area file, RMS-11 intersperses index and data
buckets:
index buckets are scattered among the data buckets.
During
each random record access, RMS-11 consults the appropriate index
descriptor in memory and then directs (through the operating system)
the disk head to read the root and levels 2 and greater, level 1, then
the appropriate level 0 bucket. These buckets can be anywhere in the
file, and the disk head can travel large distances several times to
complete one access operation. Figure 6-1 shows an indexed file with
one area. Figure 6-2 shows an example of a single-area indexed file.

BUCKET
NUMBER

~~~~~~~~-.--~~.-------t

R()()T

DATA

PRIMARYI
INDEX

DATA

l p~-l~~RYI

DATA

I DATA

DATA

INDEX

I··.

ZK-1153-82

Figure 6-1: Single-Area Indexed File

ROOT

VBN 17933

LEVEL 2

VBN 305

LEVEL 1

LEVEL 0

VBN 14

VBN 20433

VBN = VIRTUAL BLOCK NUMBER
ZK-1158-82

Figure 6-2: Example of Single-Area Indexed File

6-11

INDEXED FILE DESIGN
To randomly access a specific record in the file illustrated in Figure
6-2, RMS-11 makes the following I/O requests:
1.

Read VBN 17933

Read VBN 305

Read VBN 14

Read VBN 20433

You can now realize how much the device has to move its read
service one random access operation.

head

A multiarea file, on the other hand, can have all index buckets
allocated contiguously
(if enough blocks were initially allocated):
all index information is available in one physical part of the disk.
RMS-11 can then traverse an index with little or no head movement
until it reads the indicated data bucket.
In addition, a sequential
read of the file moves the head mechanism smoothly through the
physically contiguous area assigned to the primary index level 0.
Figure 6-3 shows an indexed file with two areas.

BUCKET
NUMBER

N-1

DATA

.--~----.~~--.-~~--.--~~-.---

ROOT
PRIMARY PRIMARY
OF
PRIMARY
INDEX
INDEX PRIMARY
INDEX
I INDEX

AREA 0

'DATA

DATA

AREA 1
ZK-1154-82

Figure 6-3: Two-Area Indexed File
To refine your file even more, place the lowest level of each index
(level 1) in an area separate from the rest of the index (levels 2 and
greater).
Figure 6-4 shows an example of a multiarea indexed file.
To randomly access a specific record in the file illustrated in Figure
6-4, RMS-11 makes the following I/O requests:
1.

Read VBN 418

Read VBN 423

Read VBN 1537

Read VBN 14703

You can now realize how much the proper use of areas reduces disk head
movement during a random access operation.

6-12

INDEXED FILE DESIGN

ROOT

LEVEL 21

VBN 423

AREA 0
AREA 1

LEVEL 1

VBN 1537

AREA i

AREA 2

LEVEL 0

VBN

147031

VBN = VIRTUAL BLOCK NUMBER
ZK-1161-82

Figure 6-4: Example of Multi-Area Indexed File
When you specify and preallocate multiple areas, RMS-11 arranges them
in order in the file:
area 0 (including the file prologue) in the
first virtual blocks of the file, then area 1, and so on.
If you
specify contiguity for the entire file,
this control over the
distribution of structural elements of the file is propagated from the
virtual block sequence to the logical block sequence on the disk.
Contiguity is very important to performance.
contiguity, Section 8.3.

6.4

For more information

PLACEMENT CONTROL

Placement control enables you to specify the location on a disk for a
file or the areas of a
file.
You use placement control for the
following reasons:
•

To start a file or area at the first block of a track or
cylinder so that the file or area can reside in one or more
contiguous tracks or cylinders. This effort minimizes head
movement during file access.

6-13

INDEXED FILE DESIGN
•

To place the files used by a single application together on a
disk.
This effort reduces I/O time by minimizing head
movement among the files.
Example: You want to run a general ledger application that
uses several files (an accounts file, a transaction file, and
so on). The application consists of several tasks.
So, you
start with an initialized disk and copy the tasks onto it.
Then, you create (and populate) your data files, placing them
near the tasks.
This effort reduces the distance the disk head moves to
service
I/0
operations required by an RMS-11 program:
disk-resident overlays (discussed in Chapter 8) and data file
accesses.
Note, however, you gain more improvement if you eliminate head
contention by placing the individual files on separate disks.

You calculate track and cylinder starting block numbers as follows:
1.

Read the documentation that came with your disk drive.
and write down the following numbers:
•

Number of surfaces on a volume (or pack or disk)

•

Number of tracks on a surface

•

Number of sectors in a track

Find

NOTE
On most DIGITAL disk drives, a sector equates to a
logical block.
For example, the following decimal
numbers apply to an RP06 only:
Number of cylinders per disk = 815
Number of tracks per cylinder = 19
Number of sectors per track = 22
2.

Establish the starting logical block number
(LBN)
for each
track
on
the disk by writing down the multiples of
sectors-per-track. Since LBNs start with O, tracks start at
multiples of track length.
Example: From the RP06 specifications, the first 10 tracks
start at LBNs:
O, 22, 44, 66, 88, 110, 132, 154, 176, 198.

Multiply sectors-per-track by tracks-per-cylinder to get
sectors-per-cylinder.
Establish the starting LBN for each
cylinder on the disk by writing down the multiples of
sectors-per-cylinder.
Example: For an RP06, the first 10 cylinders start at the
following LBNs:
O, 418, 836, 1254, 1672, 2090, 2508, 2926,
3344, 3762.

After you decide where on the disk you want to place your file, you
create the file using RMS-11 placement control.
In the process, you
place area 0 (which will position the whole file,
if the file is
contiguous) at the location you calculated.
If you are using a high-level language, you can specify place~ent
control by using RMSDES.
If you are programming in MACR0-11, you can
specify placement control through the use of allocation XABs.
6-14

INDEXED FILE DESIGN
6.5

BUCKET SIZE

Buckets are the units of access for indexed files.
Bucket size is
critical to the virtual address space required by a task and to the
speed with which a task performs. There is, of course, a trade-off:
the larger a bucket,
the larger the task, but the faster it reads
data:
•

The speed of an RMS-11 operation is closely proportional to
the number of I/O operations involved.
For indexed files, the
number of data transfers during a random retrieval operation
is approximately equal to the depth of the index (in most
cases, one more than the depth).
That number includes only
the I/O operations directly related to the record operation;
other data transfers can be required to service the operation,
including overlays and system overhead (discussed in Chapter
8) •

Therefore, the larger the buckets, the shallower the index,
and the faster the random retrieval operation. Without other
considerations, you should pick the largest possible bucket.
The maximum bucket size allowed is 32 blocks.
•

The larger the bucket, the more records it can contain,
sequential access can require fewer I/O operations.

and

However, there are other considerations.
RMS-11 requires two I/O
buffers, each the size of the largest bucket, when it connects a
record access stream to an indexed file.
By making bucket size
smaller, you reduce the size of the buffers your task requires.
Depending on the record operations your program requires, that virtual
address space may be better used in overlay structure optimization
(discussed in Section 8.2).
Therefore, you should set bucket size to some lower value that still
allows good performance; a reasonable goal is an index depth of 2 or
3 (root at level 2 or 3), although very large files can require four
levels of index, in addition to the data level (level 0).
Each area can have its own bucket size, but normally
the maximum size for all buckets:
•

You should consider more than the size
(plus the 7-byte header)
when you
bucket size:

you

should

use

of your data record
calculate primary data

Records that move from one bucket to another leave a 7-byte
pointer.
Deleted records leave from 2 bytes to enough
primary key to the whole record.

hold

the

Therefore, you should consider the predominant activity in the
file:
If you intend to populate the file and then only read from
it, you do not consider activity overhead.
You must
populate the file with records in ascencting order by
primary key value (discussed in Section 6.7).
If you intend to populate the file and then insert and/or
delete a lot of records, you should allow for those
activities in your bucket size calculations.

6-15

INDEXED FILE DESIGN
6.5.l

Bucket Size for Primary Index

You can calculate bucket sizes in two steps.
Step 1:
Calculate the following quantities for different bucket sizes
3, and so on) :
NIRBK

((512*BKS)-15)/(PKL+BPL)

(Equation la)

NDRBK

((512*BKS)-15-AO)/(RSZ+RFO)

(Equation lb)

(1,

where:
NIRBK

is the number of index records per level l+ index buckets

NDRBK

is the number of data records per level 0 bucket

BKS

is the bucket size as number of blocks

PKL

is the primary key length in bytes

BPL

is the bucket pointer length:

RSZ

RFO

BPL

is 3 for pointers to the first 65,535 blocks
file

BPL

is 4 for pointers to
65,536 and (2**24)-1

the

blocks

numbered

between

BPL

is 5 for pointers to
2**24 and (2**32)-1

the

blocks

numbered

between

the

is the size of the record:
•

data size for fixed-length records

•

average record length for variable-length records

is the record format overhead:
RFO

is 7 bytes for fixed-length records

RFO

is 9 bytes for variable-length records

is activity overhead.
If any noticeable number of bucket
splits will occur (due to random record insertions or UPDATE
operations that increase record sizes), specify a value of
at least 7
(more if bucket splits will be common).
If
insertion and
deletion
activity
will
occur
often,
significantly larger values of AO may be desirable, as well
as occasional file reorganizations to reclaim space and
improve access performance.

6-16

INDEXED FILE DESIGN
When you load a file using RMS-11 bucket fill factors, you preallocate
space in each bucket for future activity.
In such a case, Equation la
becomes:
NIRBK

(FF-15)/(PKL+BPL)

and Equation lb becomes:
NDRBK

(FF-15)/(RSZ+RFO)

where:
FF

is the appropriate fill factor in bytes and has been adjusted to
leave extra space in each bucket to accommodate future activity
overhead.

Step 2:
Select bucket size for
equation is true:
NIRBK**n

;:::

data

NRF/NDRBK

and

index

areas

where

the

following

(Equation 2)

where:
NRF

is the number of data records in the file

is the depth of the index

This equation portrays the exponential relationship between the number
of data records in a file and the depth of its index.
You use the
values for NIRBK and NDRBK you calculated in step 1.
a.

Set up a grid (see the example after step 2e).

For each value of NIRBK, calculate the left side of
2, for n = 2, 3, and for very large files, 4.

For each value of NDRBK, calculate the right side of Equation
2.

Where the equation is true, that is, the left side is greater
than or equal to the right side, you have a valid combination
of bucket sizes. The bucket size used to calculate the left
side may be equal to the size used to calculate the right
side, but it does not have to be.

Equation

NOTE
You gain no advantage using different index and data
bucket sizes.
RMS-11 requires two I/O buffers, both
the size of the largest bucket defined for the file.
In fact, PDP-11 COBOL users must not choose different
index and data bucket sizes.
e.

Select one of the valid combinations
application's requirements.

6-17

according

your

INDEXED FILE DESIGN
NOTE
•

Equation 2 is true only for files where records
are inserted in order by ascending value of the
primary key.
See also Section 6.7.

Bucket size is a step function of index depth •
• Therefore,
intermediate bucket sizes generally
waste address space.
For example, given a file where:
BKS
BKS

4 --- index depth of 3
8 --- index depth of 2

then bucket sizes of 5, 6, and 7 blocks would not
normally be used, although you might choose a bucket
size of 5 blocks if future file activity seemed
likely to increase the index depth otherwise.
Example: For a file containing 50,000 200-byte fixed-length records
with a 15-byte primary key, use the equations in steps 1 and 2 to fill
in the following grid:
•

Calculate values for NIRBK using Equation la and bucket sizes
1 through 6. Drop the remainder; use only the integer part
of the result.

•

Calculate values for NDRBK using Equation lb and bucket sizes
1 through 6. Drop the remainder; use only the integer part
of the result.

•

Calculate the number of data buckets in level O (NRF/NDRBK)
corresponding to bucket sizes 1 through 6. Round the result
up to the nearest integer.

Calculate NIRBK**2 for the values of NIRBK corresponding to
• bucket
sizes 1 through 6.
Round the result up to the nearest
integer.
Calculate NIRBK**3 for the values of NIRBK corresponding to
• bucket
sizes 1 through 6.
Round the result up to the nearest
integer.

BKS

NIRBK

112

141

169

NDRBK

NRF/NDRBK

25000

12500

7143

5556

4167

3572

NIRBK**3

19683

175616

592704

1014049

2803220

4826810

NIRBK**2

729

3136

7056

12544

19881

28561

6-18

INDEXED FILE DESIGN
•

To determine the combinations of bucket sizes where Equation 2
is true, compare the values in the NIRBK**3 row one at a time
to each of the values in the NRF/NDRBK row.
Where the
NIRBK**3 value is greater than or equal to NRF/NDRBK, a valid
bucket size combination exists.

Example: The first NIRBK**3 value is 19683.
This is less
than 25000, the first NRF/NDRBK value, but it is greater than
12500, the second NRF/NDRBK value.
Therefore,
index bucket
size of 1 (from NIRBK**3 row) and data bucket size of 2 (from
the NRF/NDRBK row) is a valid combination.
•

Compare the values in the NIRBK**2 row one at a time to each
of the values in the NRF/NDRBK row. Where the NIRBK**2 value
is greater than or equal to NRF/NDRBK, a valid bucket size
combination exists.

Example: The first NIRBK**2 value is 729. This is too small
to use, as is the second value in the row. However, the third
value is 7056. This is less than 25000 (the first NRF/NDRBK
value)
as well as the next two values, but greater than 5556,
the fourth NRF/NDRBK value. Therefore, index bucket size of 3
(from NIRBK**2 row)
and data bucket size of 4 (from the
NRF/NDRBK row) is a valid combination.
As a result of the comparisons in steps 6 and 7 above, Equation
true in the following cases:

NIRBK**3
DBKS

IBKS

IOB (bytes)

2
1

2048
2048

DBKS

IBKS

IOB (bytes)

1
2
4

6
4

6144
4096
4096

2
NIRBK**2

where:
DBKS

is the data bucket size from the NRF/NDRBK row

IBKS

is the index bucket size from the NIRBK**n rows

IOB

is the maximum I/O space required by the largest bucket size
the pair

The choice of bucket size pairs depends on what you need to optimize
most in the aoolication:
task size or access timea After vou choose.
make data and index bucket sizes equal to the larger size selected.

--- -

6.5.2

... .....

..s.

•

Bucket Sizes for Alternate Indexes

The selection of bucket sizes for alternate indexes follows
procedure as that of primary key bucket sizes.

6-19

the

same

INDEXED FILE DESIGN
Step 1:
The records-per-bucket equations for alternate indexes are:
NIRBK

= ((512*BKS)-15)/(AKL+BPL)

and
NDRBK

((512*BKS)-15)/(AKL+(DBPL*DF)+4+DO)

where:
AKL

is the alternate key length in bytes

DBPL

is the data bucket pointer length:

DBPL

is 4 for pointers to the first 65,535 blocks in the file

DBPL

is 5 for pointers to the blocks
and (2**24)-1

DBPL

is 6 for pointers to the blocks numbered between 2**24 and
(2**32)-1

numbered

between

65,536

is the duplicate factor:
DF

is 1 if you allow no duplicates

is the average number of records with the
for any key value present in the file

same

key

values

NOTE
The DF factor does not compensate enough if DF is
greater than the number of data records that fit in
a bucket. RMS-11 must then use continuation buckets
to store the records with duplicate values.
DO

is the duplicate overhead:
DO

is 0 if you allow no duplicates

is 4 if you allow duplicates

No record movement or space/deletion overhead occurs in index buckets.
Step 2:
RMS-11 cannot load buckets in alternate indexes as efficiently as in
the primary index because alternate key values inevitably fall in
random order {unless you use the RMSIFL utility described in the
RSX~llM/M-PLUS
RMS-11 Utilities manual). The ideal values resulting
from the equations in Section 6.5.1 must be reduced by a packing
efficiency factor, unless RMSIFL is used to load the file.
Studies have shown that the packing efficiency factor for alternate
keys is normally about 0.5.
However, this factor applies only to the
lower levels of the index and to the data level, and not to the root.
The packing efficiency of any index's root is always 1.

6-20

INDEXED FILE DESIGN
Therefore, the index depth equation for alternate indexes is:
(PF**n) * (NIRBK**n)

NRF/NDRBK

where:
PF

is the packing efficiency factor.

Example: Using the file in the primary key example and adding a
10-byte first alternate key,
allowing no duplicates, the following
grid can be filled in (NRF=50,000 since there is one SIDR per data
record) :
BKS

NIRBK

117

156

195

235

NDRBK

113

142

170

NRF/NDRBK

1811

892

592

443

354

295

0.125*NIRBK**3

6859

57067

200202

474552

926860

1622240

0.250*NIRBK**2

361

1483

3423

6084

9507

i3807

for

alternate

indexes

The index depth
following cases:

equation

true

the

NIRBK**3
DBKS

IBKS

!OB (bytes)

1024

DBKS

IBKS

IOB (bytes)

1
2

3
2

3072
2048

NIRBK**2

Do not choose a bucket size smaller than that selected for the primary
index (Section 6.5.1).

6.5.3

Program Syntax

RMS-11 requires bucket size as a whole number of blocks.
However,
some high-level language compilers require or allow you to specify the
bucket size in number of records. This syntax can lead to a different
number of records per bucket than you are counting on.
Example: A BASIC-PLUS-2 program contains the following clause
OPEN statement that creates an indexed file:
BUCKETSIZE 5%

6-21

INDEXED FILE DESIGN
The record format is fixed;
record length is 100 bytes.
makes the following calculation:
100
+ 7

bytes for the data
bytes for the record header

107
x 5

bytes for each record
records specified in a bucket

535
+15

bytes for the records in a bucket
bytes for the bucket overhead

550

bytes required to be in the bucket

The compiler

A bucket must be a whole number of blocks long, so the compiler rounds
the bucket size to 2 blocks and passes that to RMS-11 to create the
file.
However, 2 blocks contain 1024 bytes;
that leaves 1009 bytes for
record storage after the bucket overhead is subtracted.
Since each
record is 107 bytes long, the buckets that were originally supposed to
contain only 5 records now can contain 9 (1009/107).
Bucket size can be set by RMSDES or by your application
depending on the capabilities of your high-level language.

6.6

program,

FILE ALLOCATION

RMS-11 requests the file control processor to allocate
file at three different points in the file's life:

blocks

complete

•

When the file is created

•

When RMS-11 must dynamically extend the file
operation

•

When you explicitly instruct RMS-11 to extend the file

The allocation of blocks to a file takes time, mainly I/O time as the
operating system performs its function.
If RMS-11 has to request an
allocation every time it requires a new bucket, this time can be a
significant factor in an application's performance, especially during
file population.
You can help optimize performance by minimizing allocation overhead in
the following areas:

6.6.1

•

Initial allocation

•

Default extension quantity

Initial Allocation

Total allocation of an
efficient.

indexed

file

6-22

when

you

create

most

INDEXED FILE DESIGN
The total allocation for a file is the sum of the prologue and the
allocations for the different indexes that make up the file; an
index's allocation is the sum of the allocations for all levels in the
index.
You should start with the primary level 0 and "build" each
level of each index on paper, as shown in the following steps.
1.

Calculate the number of buckets in level 0 (NBK@O):
NBK@O

= NRF/NDRBK

where:
NRF

is the total number of records in the file

NDRBK

is the number of data records in a bucket in level 0
(see Section 6.5.1 for the method of determining this
value)
NOTE

The method described in Section 6.5.1 assumes that
you will put records into the file in order by
ascending primary key value. However, if you will be
loading the file in random primary key value order,
you should divide the NDRBK value obtained using the
method described in Section 6.5.1 by 2. You will
need twice as many data buckets.
2.

Calculate the number of buckets in level 1 (NBK@l):
NBK@l

= NBK@O/NIRBK

where:
NBK@O

is the number of buckets in
Step 1)

level

NIRBK

is the number of index records per bucket in the index
(see Section 6.5.1 for the method of determining this
value)

(calculated

NOTE
The method described in Section 6.5.l assumes that
you will put records into the file in order by
ascending primary key value. However, if you will be
loading the file in random primary key value order,
you should divide the NIRBK value obtained using the
method described in Section 6.5.1 by 2 for every
index level but the root. You will need twice as
many index buckets.
3.

Calculate the number of buckets in level 2:
NBK@2 = NBK@l/NIRBK

6-23

INDEXED FILE DESIGN
4.

Continue this sequence of calculations until
root level, that is~
NBK@n

you

reach

the

= 1 = NBK@(n-1)/NIRBK

where:

NBK@n

is the number of buckets in the root, which
definition

is the index depth

Calculate the allocation in blocks for each level:
AQ@O
AQ@l

NBK@O
NBK@l

AQ@n

IBKS

* DBKS
* IBKS

where:

AQ@n

is the allocation quantity in blocks for level n (0 for
level O, 1 for level 1, and so on)

DBKS

is the data bucket size in blocks

IBKS

is the index bucket size

Calculate the allocation for each alternate index as shown in
Steps 1 through 5;
see Section 6.5.2, for equations.
NOTE
Alternate indexes are normally populated in random
key value order.
Therefore, you should divide the
NDRBK and NIRBK values obtained using the method
described in Section 6.5.2 by 2 except for the root
level.

The total allocation quantity for the file (ALQ) is the
of the index allocation quantities plus the prologue:

sum

ALQ = PLG + AQPK + AQAKl + ••• + AQAKn
where:
n

is the last alternate key defined for the file

The prologue of an indexed file can be from 2 to 84 blocks long.
The
size is the sum of the key descriptor blocks and the area descriptor
blocks:
•

VBN 1 describes the primary key (and contains other
information).

•

Each key descriptor block covers up to 5 alternate keys.

•

Each area descriptor block covers up to 8

6-24

areas~

attribute

INDEXED FILE DESIGN
Finally, RMS-11 extends the prologue to an integral multiple of bucket
size if the criteria described in Section 6.5 are met.

Example: Given an indexed file of 100,000 fixed-length user data
records with the following attributes, calculate a reasonable initial
allocation size in blocks:
Data size = 200 bytes
Primary key

20-byte string;

no duplicates allowed

Alternate key = 8-byte packed decimal;

no duplicates allowed

Data bucket size = indexed bucket size = 3 blocks
Calculate the primary index first:
1.

AO = 0, so
NDRBK

((512*3)-15)/(200+7)

7 data records per bucket

NBK@O

NRF/NDRBK = 100000/7

14,286 buckets in level 0

NIRBK

((512*3)-15)/(20+3) = 66 index records per bucket

NBK@l

NBK@O/NIRBK

14286/66

NBK@2

NBK@l/NIRBK

217/66 = 4 buckets in level 2

217 buckets in level 1

NOTE
If the number of buckets in the level under the root
is very much less than the number of index records
that fit in a bucket, you may be able to use a
smaller bucket size without increasing the index
depth.
4.

NBK@3 = NBK@2/NIRBK = 4/66 = 1 bucket in level 3' the root

AQ@O

NBK@O*DBKS

14286*3 = 42,858 blocks in level 0

AQ@l

NBK@l*IBKS

217*3

AQ@2

NBK@2* IBKS

4*3

12 blocks in level 2

AQ@3

NBK@3*IBKS

1*3

3 blocks in level 3

AQPK

43,521 blocks in the primary index

Now calculate the alternate index

648 blocks in level 1

(DF=l, DO=O):

NDRBK = ((512*3)-15)/(8+(4*1)+4)
= 89 data records per bucket
NBK@O = NRF/NDRBK = 100000/89
level 0

1124*2

2,248

buckets

The doubling compensates for a packing efficiency of 0.5.

6-25

INDEXED FILE DESIGN
2.

NIRBK

((512*3)-15)/(8+3)

138 index records per bucket

NBK@l

NBK@O/NIRBK

17*2

34 buckets in level 1

NBK@2

NBK@l/NIRBK

1 bucket in level 2, the root

AQ@O

NBK@O*BKS

2248*3 = 6,744 blocks in level 0

AQ@l

NBK@l*BKS

34*3

102 blocks in level 1

AQ@2

NBK@2*IBKS

= 1*3

3 blocks in level 2

AQAK

6,849 blocks in the alternate index

Finally:

= 3 + 43,518 + 6,849, or

ALQ

PLG + AQPK + AQAKl

ALQ

50,370 blocks for the whole file

This allocation can be done by RMSDES or by your application
depending on the capabilities of your high-level language.

6.6.2

program,

Default Extension Quantity

If the file cannot be totally allocated at creation time, you should
establish a reasonable default extension quantity (DEQ) to minimize
the number of (and the time spent on) file extensions.
Even if the
file is totally allocated when it is created, you should establish a
reasonable DEQ in case the file gets bigger than planned.
A good basis for calculation is the number of records that are added
to the file in a given period of time, such as a day; use the formula
for allocation quantity in Section 6.6.1.
The DEQ should equal a
multiple of the bucket size.
If you do not specify a DEQ, it defaults to zero whether you create
the file with RMSDES or a high-level language.
RMS-11 responds to a
DEQ of zero by requesting 4 times the bucket size in blocks from the
file control processor each time it automatically extends the file.
The DEQ for the file can be set by RMSDES or by your application
program, depending on the capabilities of your high-level language.

6.7

POPULATION TECHNIQUES

File population entails a large burst of records written into the file
after it has been created and before it is made available for normal
processing. You can populate a file with the RMSIFL or RMSCNV utility
programs,
or
with
an
application program, depending on the
capabilities of your high-level language.
The aim of populating an RMS-11 indexed file is to avoid bucket splits
and record movement during the population and during later use of the
file.
The techniques to achieve this goal are:
•

Inserting records in ascending order by primary key

•

Use of fill numbers

6-26

INDEXED FILE DESIGN
6.7.1

Ascending Order by Primary Key

The best way to populate an indexed file is to insert the records in
ascending primary key value order.
You do not need to insert the
records all at once. This technique:
•

Minimizes population time

•

Avoids the creation of RRV records, allowing RMS-11 to fill
buckets with data records and thereby find records with the
least access time.

Contrast this technique with records loaded in descending order by
primary key value.
In that case, you introduce the packing efficiency
factor p to the primary key equations. Normally, p is 1, when you
insert records in ascending order and the factor drops out of the
equation, as shown here:
NIRBK**n
But when p

NRF/NDRBK

< 1, the equation becomes:

(p**n) (NIRBK**n)

NRF /NDRBK

Since p is a fraction, the introduction of this factor reduces the
left side of the equation, at times dramatically, thereby potentially
increasing:
•

The index depth needed to cover
records

specific

number

data

•

Frequency of bucket splitting (an important factor in the time
required to populate an indexed file)

As mentioned in Section 6.5.2, alternate indexes are a prime example
of packing inefficiency, a situation avoided only with the RMSIFL
utility. The best general approximation for p in the case of
alternate indexes is 0.5, the value used in Section 6.5.2.
You can populate a file with records in ascending order by primary key
as follows:

Use the RMSIFL utility.

This utility:

Sorts your input file into ascending order by the output
file's primary key, if the file is not already sorted that
way
Transfers the records from the input
file

file

the

output

RMSIFL uses techniques not available to you to further improve
the population of an indexed file.
•

Use the RMSCNV utility,
(/MA) switch.

specifying

the

mass-insertion

•

Write a MACR0-11 program to populate the file and specify:

mode

In the FAB, deferred write when you open the file
In the RAB, when you connect to the
mode and sequential access mode

6-27

file:

mass-insertion

INDEXED FILE DESIGN
See the RSX-llM/M-PLUS RMS-11
more information.

Macro

Programmer's

Guide

for

Be sure to sort your input records into ascending order by the
indexed file's primary key before you run the program.

6.7.2

Random Insertions after File Population

If you will be inserting records into an indexed file after it
populated, you should consider ways to optimize these operations:

•

If the new records to be inserted span the full
range
primary key values, you should use a bucket fill size.

•

If the inserted records are sorted into ascending order by
primary key value and added at the logical end-of-file, you
should use mass-insertion mode.

6.7.2.1 Bucket Fill Size - You can optimize for evenly distributed
random insertions by leaving free space in buckets during the initial
population of the file.
To do this, you specify a bucket fill size as
a set amount of bytes for each area in your file.
Normally, RMS-11
ignores this number, but you can direct RMS-11 to obey it:
RMS-11
then fills each bucket in the file to the level specified by the
number.
Example:
Your bucket size is 2 blocks; you set the bucket fill
size
to 768 bytes.
When you tell RMS-11 to honor the fill size, it only
uses 768 out of 1024 bytes in each bucket -- the buckets are logically
three-quarters size.
You use the bucket fill size when you populate a file to improve its
performance during normal operations:
if free space is available in
every bucket in the file, any record randomly inserted~into the file
is likely to fit without causing a bucket split.
The size of the bucket fill size depends on:
•

The amount of insertion activity you expect.
Allow room (including record header) for the number of records
you
will add to each bucket during normal operations.
Occasional insertions might not warrant the use of bucket fill
sizes, whereas heavy insertion can require room for multiple
additional records in each bucket to optimize, but not
eliminate, bucket splitting activity.

•

The type of bucket (data or index)

involved.

Because of the difference in record sizes and frequency of
insertion,
data and
index buckets should normally have
different bucket fill sizes.
Example:
The file contains 240-byte fixed-length
records with a
primary key field 24 bytes long.
To optimize random insertions, the
fill size for data buckets should therefore be at most: bucket length
minus bucket overhead
(i5)
minus record length (240) minus record
overhead (7).
This number leaves room for one data record.
This same bucket fill size for index buckets leaves room for
9 index
records.
A more reasonable bucket fill size for index buckets is:
bucket length minus bucket overhead minus 2 times 27 bytes.
This
6-28

INDEXED FILE DESIGN
number leaves room for 2 index records, where:
the primary key length
(24) plus the bucket pointer length (3) equals the index record length
( 2 7) •

See Section 6.5.2 for a more complete discussion.
NOTE
RMS-11 ignores a bucket fill size of
less than 50 percent of the bucket
length and uses the 50 percent figure.
The bucket fill size for a file can be set by RMSDES or by your
application program, depending on the capabilities of your high-level
language.

6.7.2.2 Mass Insertion - You use mass-insertion mode when you have
series of records to add to an indexed file and:
•

You have sorted the records into ascending order by the file's
primary key.

The lowest key value in the records is greater than the
• highest
key value in the file;
that is, the records will be
inserted at the logical end-of-file.
While the mass-insertion bit is on, RMS-11 performs
normally (see Section 5.3.1) except that it:

PUT

operation

•

Does not unlock the primary level O data bucket

•

Keeps a pointer to the primary level 1 bucket that pointed
the proper level 0 bucket

These extra steps enable RMS-11 to:
•

Write the next record without following the primary index
the mass insertion bit is still on).

(if

•

Rapidly split the primary level 0 bucket when it is full:
since RMS-11 has a pointer to the primary level 1 bucket that
will contain the index record for the new bucket,
it can
update that bucket without following the index.

By using these techniques, RMS-11 can extend the primary level 0
bucket by bucket, packing records into the buckets in the order they
are written. As each bucket becomes full, RMS-11 creates a new one,
beginning with the next record inserted, and notes its existence in
the primary level 1 index bucket.
NOTE
Mass insertion significantly improves
performance
for
single-key
indexed
files.
The percentage of improvement
lessens with each additional key defined
in the file.
You can enhance mass insertion performance
(see Section 7.4).

6-29

using

deferred

write

CHAPTER 7
RECORD AND FILE PROCESSING OF INDEXED FILES

The
record
and
file
processing
capabilities
described
in
RSX-llM/M-PLUS RMS~ll:
An Introduction are available for indexed
files.
This chapter discusses the operations and their implementation
and restrictions with indexed files.

7.1

ACCESS SHARING

Access sharing can be specified for indexed files as described in the
following sections.
See Section 2.2.3 for general information on
shared access.

7.1.l

Record Access to Indexed Files

Indexed files allow fully interlocked read/write sharing, dependent
upon the compatibility of the access and sharing declarations of
multiple accessors, as follows:

7.1.2

•

Block Access to Indexed Files

Because block access bypasses the record structure
algorithms used with indexed files,
read/write
,,....,,....,..,..'""'.;~ ..... -...::1
.t:'CJ..lll.L 1..1..CU.

" - .....

nuy

---...:11 •• -.:.a...-

.&..COU/ W.L. .L l..C

-t....--.:--

;::)lJQ.L. .Lll'j

...:11 _ _ ,

_ _ _ .... .: _ _

U'Cl.,;.1.Q.L.Q~.LVU

to read-only before the file is
record-accessed sequential files).

processed

,: _
.L;::)

and interlocking
sharing cannot be
..__..:!

,:_.._ ____ ,, .....

similar to

_______
1.,;VllV'C.L. ~ICU

(this

.Lll~'C.L.lJQ.1..J.Y

Thus, multiple read-only accessors
(regardless of their sharing
declarations) can share indexed files concurrently using block access,
as long as no read/write record accessor is present. Read-only block
accessors can share files with read-only record accessors.
In
addition, a single read/write accessor can access an indexed file
using block access (regardless of sharing declaration) as long as no
other accessor of any kind is present.

7-1

RECORD AND FILE PROCESSING OF INDEXED FILES
Other combinations are rejected:
are incompatible.

7.2

the access and sharing

declarations

RECORD AND STREAM OPERATIONS

The following record and stream operations can be performed on indexed
files.
See also the discussions of read- and write-type record
operations in Chapter 5.
CONNECT
DELETE
DISCONNECT
FIND
FLUSH
GET
PUT
REWIND
UPDATE
In all record operations, RMS-11 establishes the current record
(if
any) and next record (if applicable) context.
If any record operation
fails, RMS-11 normally sets the current record to none and does not
change the next record.
NOTE
For more information on the RMS-11 error
codes
referred to in the following
sections, see the RSX-llM/M-PLUS RMS-11
Macro Programmer 1· s Guide.

7.2.1

CONNECT

A CONNECT operation affects the
follows:

conte~t

for

the

access

•

Current record -- There is no current record.
Any
requiring a current record fails at this point.

•

Next record
according to
reference.

strea~

operation

The next record is the first record in the file
the collating sequence of the specified key of

Example:
In an indexed file with multiple
record
varies by the key specified in
initiating the CONNECT operation:

keys, the next
the instruction

If the primary key is specified, the next record is the
first record in primary level O, the one with the lowest
primary key value in the file.
If an alternate key is specified, the next record is
indicated by the first SIDR in the alternate index's level
O; the record itself can be located anywhere in the
primary level O.

7-2

RECORD AND FILE PROCESSING OF INDEXED FILES
7.2.2

DELETE

In a DELETE operation: RMS-11 flags the header of the current record
to indicate that it is a deleted record. The prerequisite GET or FIND
operation brought the bucket containing the record into the I/O
buffer.
Then, RMS-11 writes the bucket over its original location on the disk,
unless you specified deferred write (see Section 7.4.2).
A DELETE operation affects
follows:

the

context

•

Current record -- None. Any
record fails at this point.

•

Next record -- Unchanged.

for

the

stream

requiring

current

A DISCONNECT operation destroys the context for the access
You cannot resume this context by reconnecting the stream.

stream.

7.2.3

operation

access

DISCONNECT

7.2.4

FIND

FIND
Section 5.3.2 describes how RMS-11 performs a key-access
RMS-11
performs
a
operation.
Section
5.4
describes
how
sequential-access FIND operation.
If the record does not exist or has been deleted,
error code depending on the access mode:

RMS-11

returns

•

In sequential-access mode, the error code is ER$EOF.

•

In key-access mode, the error code is ER$RNF.

•

In RFA-access mode, the error code is:

ER$RFA -- no valid record has ever existed at the specified
location.
ER$DEL -- the record header indicates that the
deleted.

record

was

A FIND operation affects the context for the access stream as follows:
•

For a sequential-access FIND operation:
Current record -- Is set to the value of the record found.
Example: You have connected a stream to an indexed file,
specifying 0 as the key of reference. There is no current
record, but the next record is the first record in primary
level
O.
If
you execute a sequential-access FIND
operation, the current record is set to this record.
Next record -- Is set to the record logically following the
current record in the index of reference.

7-3

RECORD AND FILE PROCESSING OF INDEXED FILES
NOTE
RMS-11 enacts this logical sequence
actually accesses the next record:

only

record,

when

reading

RMS-11 locates the current
bucket if necessary.

RMS-11 locates the record logically following
the current record, reading another bucket if
necessary.

If the indexed file is shared, the actual record in
the next record position can change between the
operation that accesses the current record and the
one that finds the next record.

Example: From the previous example, the next record is the
record in the file with the next higher primary key value.
•

For a key-access or RFA-access FIND operation:
Current record -- Is set to the record found, that is,
record identified by the RFA.

the

Next record -- Unchanged.

Example:
In
the
previous
examples,
you
did
a
sequential-access
FIND operation after connecting the
stream to the file.- You now execute an RFA-access FIND
operation.
The current record is set to the record
specified, but the next record is not changed.
Therefore,
if you do another sequential-access FIND operation, the
current record will be set to the second record in primary
level O, not the record following the one found by RFA.
You use a FTNn operation instead of a

~RT

operation for two reasons:

It is faster because the record is not moved to the user
buffer. Although the time required to move a record from one
part of memory to another is very short, do not expend it if
you do not need to.

It does not change the next record in key-access mode or
RFA-access mode.
This convention allows you to branch off
sequential processing for the purpose of updating or deleting
records, and keep your place.

You can use a FIND operation in the following ways:
•

To skip records in sequential
successive FIND operations.

access

•

To establish
operation.

for

•

To determine the existence of a
mode.

current

record

7-4

record

mode
a

DELETE
by

using

initiating
or

UPDATE

key-access

RECORD AND FILE PROCESSING OF INDEXED FILES
7.2.5

FLUSH

A FLUSH operation does not affect the context for the access stream.

7.2.6

GET

Section 5.3.2 describes how RMS-11 performs a
key-access
GET
operation.
Section
5.4
describes
how
RMS-11
performs
a
sequential-access GET operation.
If the record does not exist or has been deleted,
error code depending on the access mode:

RMS-11

returns

•

In sequential-access mode, the error code is ER$EOF.

•

In key-access mode, the error code is ER$RNF.

•

In RFA-access mode, the error code is:

ER$RFA -- No valid record has ever existed at the specified
location.
ER$DEL -- The record header indicates that the
deleted.

record

was

A GET operation affects the context for the access stream as follows:

Is set to the value of the record read.

•

Current record

•

Next record -- Is set to the record logically
current record in the index of reference.

following

the

NOTE
RMS-11 enacts this logical sequence
actually accesses the next record:
current

only

record,

when

reading

RMS-11 locates the
bucket if necessary.

RMS-11 locates the record logically following the
current
record,
reading
another
bucket if
necessary.

If the indexed file is shared, the actual record
in
the next record position can change between the
operation that accesses the current record and the one
that tinds the next record.

7.2.7

PUT

Section 5.3.l decribes how RMS-11 performs a key-access PUT operation.

7-5

RECORD AND FILE PROCESSING OF INDEXED FILES
A PUT operation affects the context for the access stream as follows:
•

For a sequential-access PUT operation:
Current record -- None.
Any operation requiring a
record fails at this point.

The record retrieved by a
GET operation at this point is

Next record
Undefined.
sequential-access FIND or
not specified.
•

current

For a key-access PUT operation:
Current record -- None.
Any operation requiring a
record fails at this point.

current

Next record -- Unchanged.

7.2.8

REWIND

A REWIND operation sets the context of the access stream to a logical
beginning of the indexed file.
In doing so, the operation affects the
context for the stream as follows:
•

Current record
None.
Any
record fails at this point.

•

Next record -- is set to the first
record
according to the specified key of referencee

7.2.9

operation

requiring

current
the

file

UPDATE

In an UPDATE operation, RMS-11 moves the specified record from the
task's user buffer to the I/O buffer, replacing the current record set
by the previous GET or FIND operation.
Then, RMS-11 writes the bucket
over
its original location on the disk.
Section 5.3.3 describes the
UPDATE operation in detail.
An UPDATE operation requires a valid current record.
Therefore, an
UPDATE operation should follow a successful GET or FIND operation;
otherwise, RMS-11 returns the error code ER$CUR.
This error does not
affect the original record in the file on disk.
An UPDATE operation affects the
folows:

7.3

context

•

Current record -- None.
Any
record fails at this point.

•

Next record -- Unchanged.

for

operation

the

access

requiring

stream
a

current

RECORD TRANSFER MODES

You can manipulate records either in the
I/O buffer or in your
program's user buffer.
Each of these options is called a record
transfer mode.
You can change record transfer mode at run time, even
between record operations.
Figure 7-1 illustrates the RMS-11 task
structure.

7-6

RECORD AND FILE PROCESSING OF INDEXED FILES

r;;---------------,
I SIZE DEPENDS ON:

I• NUMBER OF FILES OPENED SIMULTANEOUSLY I
1•

BUCKET SIZES

L NUMBE~:_RECr-D~CCE~ ~~AMS USER BUFFERS-===::!

'
!/O
BUFFERS

VIRTUAL
MEMORY

PROGRAM

RMS-11

INTERNAL
CONTROL
STRUCTURES

.---- -- - _ J_----,
i SIZE DEPENDS ON:

I• RMS-11 FUNCTIONS USED l
~ ~V~R~~~U~TURE~~E~

ZK-1174-82

Figure 7-1: RMS-11 Task Structure

7.3.1

Move Mode

Move mode is the default record transfer
languages and all file organizations.

7.3.2

mode

for

all

programming

•

On GET operations, RMS-11 moves the record from the I/O buffer
to the user buffer before returning control to your program.

•

On PUT and UPDATE operations, your program assembles the
record to be written into the file in the user buffer, and
during the operation, RMS-11 moves the data into the I/O
buffer before updating the file.

Locate Mode

Locate mode enables your program to manipulate records in the I/O
buffer, eliminating the data transfers between it and the user buffer.
However, when you specify locate mode, RMS-11 uses it only when such
usage does not compromise data integrity. Otherwise, RMS-11 uses move
mode. Therefore, your program must still contain a user buffer.
Example: RMS-11 uses move mode instead of locate mode when an indexed
file is shared.

7-7

RECORD AND FILE PROCESSING OF INDEXED FILES
Example:
RMS-11 uses move mode instead of locate mode if you opened
the file indicating that you were going to perform UPDATE operations
on it.
RMS-ll's use of move mode instead of locate mode is transparent to
your program as long as you use RMS-11 facilities to access the record
data.
For indexed files, your program can only perform GET operations in
locate mode.
See your high-level language documentation to determine
whether the language supports locate mode and, if it does, what the
exact programming techniques are.

7.4

I/O TECHNIQUES

You can use the following techniques to
record operations.

7.4.1

improve

the

performance

Asynchronous Record Operations

Within each record access stream, your program can perform any record
operation either synchronously or asynchronously.
In synchronous
operations, RMS-11 returns control to your program after the operation
completes, either successfully or with an error.
When you execute an asynchronous operation, RMS-11 may return control
to your program before the operation is complete.
The program
continues processing while the physical transfer of data between disk
and memory is carried out. However, you must not initiate another
record operation on that stream until the first operation ends;
otherwise, RMS-11 returns the error code ER$ACT.
See your high-level
language documentation for asynchronous techniques.

7.4.2

Deferred Write

Normally, each write-type record operation (DELETE, PUT, and UPDATE)
results in a bucket being written to disk.
This convention emphasizes
data integrity: you know that when a write-type operation ends
successfully, the file reflects that operation.
However, you can improve the performance of mass-insert sequential (by
primary key)
PUT or DELETE operations by using deferred write.
Basically, deferred write directs RMS-11 to write a bucket out to disk
only when RMS-11 must use the I/O buffer for some other purpose.
Example:
Your records are 114 bytes long and the bucket size is 2
blocks. During sequential write-type operations, deferred write could
cause I/O operations per bucket to drop from 9 to 1.
Deferred write offers little or no benefit to
operations or read-type operations of any mode.

7-8

random

write-type

RECORD AND FILE PROCESSING OF INDEXED FILES
NOTE
Deferred write should only be used with
mass-insert PUT operations.
Although
not
illegal,
deferred
write
is
essentially invalidated while an indexed
file is shared by multiple tasks
except when you are also using mass
insertion
mode.
In
the
non-mass-insertion,
write-shared
environment, every write-type operation
results in an I/O operation so that:

7.4.3

•

The bucket locked by the prerequisite
GET or FIND
(for UPDATE and DELETE
operations) or by the PUT operation
can be released.

•

The new data is available
other tasks or streams.

the

Multiple Buffers

When you open an indexed file,
RMS-11 normally sets
up
two
bucket-sized I/O buffers in your task's address space. RMS-11 uses
both buffers for record operations. However, you can direct RMS-11 to
use more than the two buffers.
RMS-11 uses any extra buffers to keep, or cache, index root buckets if
either of the following is true:
•

The file is shared only by tasks with read-only access.

•

The file is not shared.

Therefore, only keys
RMS-11 caches the roots as it uses them.
specified or implicit in record operations have their index root
buckets cached:
•

During normal PUT operations, RMS-11 typically accesses all
indexes in a file.
You benefit from root caching only when
the number of extra buffers equals or exceeds the number of
indexes.

•

During mass-insertion mode PUT operations, one extra buffer
provides some benefit, regardless of sharing and number of
indexes.
If the file is not being shared, you benefit from
root caching only when you provide one more extra buffer than
indexes.

•

During GET operations, RMS-11 accesses one index
(associated
with the key of reference).
You benefit from root caching
when you provide an extra buffer for each different key you
reference.

•

During UPDATE and DELETE operations, RMS-11 accesses the
alternate indexes where a SIDR must be inserted or deleted.
You benefit from root caching when you provide an extra buffer
for each alternate index affected.

7-9

RECORD AND FILE PROCESSING OF INDEXED FILES
While root caching saves one disk read per index accessed, you may be
able to employ the address space used for the extra buffers more
profitably to optimize RMS-11 overlays (see Chapter 8).

7.4.4

Multiple Access Streams

RMS-11 allows each program to use multiple streams on an indexed file.

7.4.S

Sequentially Reading Write-Shared Files

If your task is trying to read sequentially by primary key an indexed
file that is write-shared, you can improve performance by specifying
write-access as well.
Example:
Include in your BASIC-PLUS-2
ACCESS MODIFY and ALLOW MODIFY.

OPEN

statement

the

clauses

When there is a possibility that your
task will update a
record
(established when it opened the file), RMS-11 locks the bucket when
your task gets a record and holds the bucket in the task's I/O buffer.
If your task then gets records sequentially, RMS-11 finds them in
memory. When a record in a different bucket is specified, RMS-11
unlocks the previous bucket and repeats the procedure with the new
one.
However, if your task opens a file in a read-only and write-sharing
mode,
RMS-11 does not retain the lock on the buckets read;
RMS-11
reaccesses the file for each subsequent GET operation, although it
does not start at the root and go down the index again.

7.5

FILE AND DIRECTORY OPERATIONS

The following file
indexed files:

and

directory

operations

can

performed

CLOSE
CREATE
DISPLAY
ENTER
ERASE
EXTEND
OPEN
PARSE
REMOVE
RENAME
SEARCH
See your high-level language documentation for a
support provided.

7-10

description

the

CHAPTER 8
TASK BUILDING AND COMMON OPTIMIZATION TECHNIQUES

Chapter 2 introduced four application design considerations.
Two of
those
design considerations, sharing and ease of design, were
discussed there.
The others, speed and space, were the underlying
concepts for the file and task design discussions in Chapters 4
through 7.
They are also the prime considerations for the use of the
techniques discussed in this chapter.
You can optimize the speed of and the space used by
by:
•

•

8.1

Improving the structure of each task.

your

application

This includes:

The method of combining your program with
(discussed in Section 8.1)

RMS-11

routines

Program development, including the sequence
(discussed in Section 8.2)

operations

Using all features of the environment in which the task runs.
Especially important is optimizing virtual-to-logical-block
mapping (discussed in Section 8.3), but there are other
factors as well (discussed in Section 8.4).

TASK BUILDING WITH RMS-11 ROUTINES

The software routines that perform the RMS-11 functions are distinct
from your programming language. These routines must be combined with
your program as follows:
1.

A compiler or the assembler converts your program to object
code.
In the process, the RMS-11 routines that your program
uses are listed as unresolved global references.

The task builder combines object modules into an executable
task.
It resolves the RMS-11 global references with the
RMS-11 routines in either:
An object module library named RMSLIB.OLB
An RMS-11 resident library
You must decide whether RMS-11 is to be overlaid
or
nonoverlaid when combined with your program to form a task.
This section should guide your choice.

When the task builder is finished, your task is ready to run.

8-1

TASK BUILDING AND COMMON OPTIMIZATION TECHNIQUES
Figure 8-1 illustrates this sequence, from souree
code to executable task.

program

obje~t

SOURCE
PROGRAM

COMPILER
or
ASSEMBLER

OBJECT
CODE

TASK
BUILDER

ZK-1198-82

Figure 8-1: Source-to-Task Sequence
The RMS-11 routines that become part of your task can be overlaid or
nonoverlaid.
Overlays are task segments that can run independently;
therefore, they do not have to be available to the task at the same
time and can share address space.
When a segment is needed, the
operating system makes it available, replacing (overlaying) a segment
no longer being used.
By interchanging its parts, a task can run even
though it is too large to be executed as one piece.
Nonoverlaid RMS-11:
For synchronous operations,
the task builder
concatenates the RMS-11 routines with your program, that is, without
overlays, if you add the following term to the command line:
,LB: [l,l]RMSLIB/LB
For asynchronous operations, use the following term:
,LB: [l,l]RMSLIB/LB:ROEXEC:RORSET:ROWATB,RMSLIB/LB
The task builder extracts from RMSLIB.OLB only those routines required
by ;your program.
These routines contribute from 8KB to 44KB to the
task size.
Note that if other portions of your task are overlaid, you
8-2

TASK BUILDING AND COMMON OPTIMIZATIO.N TECHNIQUES

can use nonoverlaid RMS-11 only if all references to RMS-11 take place
in the root segment of your task.
Overlaid RMS-11:
If the sum of your program plus RMS-11 code is
oreater than 64KB,
there is not enough user address space for your
task to run without overlaysc
NOTE
Although you can overlay segments of
your program, this section is devoted to
the best use
of
RMS-11
overlays.
Therefore, all references to "overlays"
mean "overlays in RMS-11 routines."
Overlays can take one of two forms:
1.

Disk-resident overlays
The overlay segments are part of the task image, and they
remain on disk until
they are needed. When a routine is
required, the operating system reads the overlay segment
containing that routine into the task's address space,
replacing a segment no longer needed.
Section
8 .1.1
discusses disk-resident overlays.

Memory-resident overlays
The overlay segments are part of a
task
image maintained
separately
in memory.
When a
routine is needed,
the
operating system maps the segment into the task's address
space with two of its active page registers (APRs) • Section
8.1.2 discusses memory-resident overlays.

Figure 8-2 illustrates nonoverlaid and
memory resident) task structure.

8.1.1

overlaid

(disk

resident

and

Disk-Resident Overlays

One disk-resident overlay can address others, which can address
others, and so on.
This chain of calls defines the overlay structure
of a task.
You describe this structure in a
file with overlay
description language (ODL) statements (described in your task builder
manual).
You must generate an ODL file for each overlaid task and supply it to
the task builder.
However, you do not normally create ODL statements
for the RMS-11 portion of your task, but instead refer to the RMS-11
ODL files provided on your system.
The RMS-11 installation process
provides overlay descriptions in two forms:
•

A series of standard ODL files describing disk-resident RMS-11
overlay structures that---require differing amounts of task
address space.
The larger structures may run faster;
you
should use the best one for your application.

•

A prototype ODL file you can modify, making overlay segments
larger if there-rs-room in your address space, or eliminating
them if your program does not use those functions.

8-3

TASK BUILDING AND COMMON OPTIMIZATION TECHNIQUES
The installation process places -these files- in acc-0-unt [l,.l]
on
logical device LB:.
If you are using asynchronous RMS-11 operations,
you must select special RMS-11 modules, as indicated in these files.

A. NONOVERLAID RMS-11

PROGRAM

RMS-11

BUFFERS ==--..J

I•---...
F:~BM ----•-ti
TO

44KB
B. RMS-11 IN DISK-RESIDENT OVERLAYS

t='

PROGRAM

en
~
a:

---------10KB----~·~'

BUFFERS

(CAN BE LARGER WiTH
FEWER OVERLAYS)
C. RMS-11 IN MEMORY-RESIDENT OVERLAYS
APR n+1
SHARED

I
PROGRAM

APR n
COMMON
SHARED
SEGMENT
OF
RMS RES

OVERLAID
SEGMENTS
OF
RMS RES

-==::J j

BUFFERS
UNSHARED RMS - 11

ZK-1199-82

Figure 8-2: RMS-11 Tasks

8-4

TASK BUILDING AND COMMON OPTIMIZATION TECHNIQUES
Each high-level language has its method of generating the ODL file for
your program and referencing the RMS-11 ODL files. They normally
generate the following hierarchy of files:
•

program-name.CMD
You supply this indirect file to the task builder.
The file
contains the appropriate command lines(s) for the task builder
and references a primary ODL file.

•

program-name.ODL
This primary ODL file determines the general structure of the
task and references secondary ODL files, including RMS-11 ODL
files, such as a standard file or your modification of the
prototype file.

See your high-level language documentation for more details.
If you are a MACR0-11 programmer, however, you must write your own ODL
file. Make sure the file contains the following terms, if you want to
use RMS-11 disk-resident overlays:
•

The factor names RMSROT and RMSALL in the
.ROOT statement.
RMSROT represents a set of concatenated modules that perform
functions common to multiple RMS-11 operations.
You must
concatenate RMSROT with your program's root so that it is
memory-resident while the task runs.

•

An indirect reference to an RMS-11 ODL file, either a standard
file or your customized version of the prototype, in the form:
@file-name
This RMS-11 ODL file resolves the
RMSALL.
For example:
.ROOT
USRSEG:
.FCTR
@LB : [ 1 , 1 ] RMS 11 X
.END

references

RMSROT

and

USRROT-RMSROT-USRSEG,RMSALL
(USR1,USR2,USR3)

8.1.1.1 ODL Files - DIGITAL provides the following standard ODL
files.
Do not change these files; make a copy in your own directory
if you want to modify one for your own use.
RMSllS.ODL

Structured to add about 6.5KB to the task size, this file
provides only sequential and relative file organization
routines in 11 overlay segments.
RMSllS.ODL is designed to use minimal virtual address
_........... c , file
space for the support provided. Because of +-hi
PUT,
operation performance and performance where GET,
and/or
UPDATE
operations
on
sequential files are
intermixed will be slower than when using other ODL
structures.

RMSllX.ODL

Structured to add about lOKB to the task size, this
provides
sequential,
relative,
and
indexed
organization routines in 35 overlay segments.

8-5

file
file

TASK BUILDING AND COMMON OPTIMIZATION TECHNIQUES

RMSllX .. ODL is_designed to use minimal virtual address
space
for
the support provided.
Because of this,
performance for record operations on indexed files will
usually be slower than when using the RMS12X or DAPllX ODL
structures.
RMS12X.ODL

Structured to add about 12KB to the task size, this
provides
sequential,
relative,
and
indexed
organization routines in 13 overlay segments.

file
file

RMS12X.ODL is designed to offer a good compromise between
performance for record operations on indexed files and use
of task virtual address space.
DAPllX.ODL

Structured to add about 14KB to the task size, this file
provides sequential, relative,
indexed, and (on systems
with the required DECnet support) remote access facilities
in 16 overlay segments.
DAPllX.ODL is designed to use minimal virtual address
space
for the support provided.
For local access,
however, it uses the efficient structure contained in
RMS12X.ODL.

RMS12S.ODL

Structured to add about 9KB to the task size, this file
provides only sequential and relative file organization
routines in 5 overlay segments.
RMS12S.ODL is designed to offer a good compromise between
performance and use of task virtual address space.

8.1.2

Memory-Resident Overlays

The RMS-11 resident libraries contain RMS-11 routines in re-entrant
executable code.
Tasks that use RMS-11 can be built with global
references resolved in the resident library RMSRES, if this library is
present in your system.
While it is executing one of these tasks, the operating system uses
two of the task APRs to map references from the task to the resident
library. Therefore, any time the task requires an RMS-11 routine, the
operating system changes the APRs to point to the segments of the
resident library that contain the routines for the operation.
This mapping is called memory-resident overlaying.
Because the
overlay segments are in memory, the operating system does not perform
an I/O operation to provide the routines
(as
it
does
with
disk-resident overlays).

8.1.2.l Task Building against the RMS-11 Resident Library - You build
tasks, directing the task builder to resolve global references with a
library, with one of the following sequences of commands:
TKB>command-string
TKB>/
ENTER OPTIONS:
TKB>LIBR=RMSRES:RO
TKB>//

8-6

TASK BUILDING AND COMMON OPTIMIZATION TECHNIQUES
or, to use RMSRES in a cooperating cluster of libraries that share the
same set of task APRs:
TKB>Command-string
TKB>/
ENTER OPTIONS:
TKB>CLSTR=LIB1,LIB2,RMSRES/RO
TKB>//
NOTE
Not all libraries can be clustered with
RMSRES.
See your high-level language,
FCS-11, or other
documentation
for
details. Clustering RMSRES may decrease
performance for some applications.
See your task builder manual for a description of the command string.
You must also do one of the following:
e

Specify LB:[l,l]RMSRLX.ODL as the RMS-11
name in your primary ODL file.

secondary

•

Merge the contents of RMSRLX.ODL into your
into your task builder command string.

own

ODL

ODL
file

file
or

If your system provides the required DECnet support and the RMS-11
library DAPRES, and you want to use the RMS-11 remote access
facilities, include the entry DAPRES in the task builder CLSTR option
(after
the
RMSRES
entry),
as
described
above,
and
use
LB: [l,l]DAPRLX.ODL instead of RMSRLX.ODL.
If you are using asynchronous RMS-11 operations, you must select
special RMS-11 modules, as indicated in RMSRLX.ODL or DAPRLX.ODL.
On RSX-llM-PLUS systems that support supervisor mode, you may instead
choose to use RMSRES as a supervisor-mode library. Because this
configuration uses two otherwise idle supervisor-mode APRs to map most
of the RMS-11 code, the impact of the RMS-11 code on your user-mode
virtual address space is reduced to the absolute minimum;
there also
may be slight performance advantages over the clustered RMS-11
configuration.
To use RMSRES as a supervisor-mode library, use the following sequence
of commands:
TKB>command-string
TKB>/
ENTER OPTIONS:
TKB>RESSUP=LB:[3,54]RMSRES/SV:O
TKB>//
See your task builder manual for a description of the command string.
You must also include the modules
LB: [l,l]RMSLIB/LB:ROEXSY:ROAUTS:ROIMPA
in the root of your task, using either the task builder command string
or an ODL file.

8-7

TASK BUILDING AND COMMON OPTIMIZATION TECHNIQUES
In addition, you should include the module
LB: [l,l]RMSLIB/LB:RMSSYM
if your task requires global definitions of the user-visible RMS-11
symbols.
If your task uses asynchronous operations, replace the
module ROEXSY above with ROEXEC.
To include remote access (OAP)
support, replace the module ROAUTS above with ROAULS, and include
DAPRES as a LIBR or CLSTR option in the task builder command sequence.
If you are using DAPRES and the RMS-11 asynchronous facilities
together, you must also include
LB:[l,l]SYSLIB/LB:AUTOT
in the task.
If you are using resident libraries, a BPT trap will be generated and
RO will contain the value 175744 (the error code ER$LIB). This can
happen if not all segments of the library are installed or if the
version numbers of one or more segments do not match the root sgement,
the RMSDAP code, or the task itself.
See your system manager to
properly install the library.

8.1.2.2 Using RMS-11 Operations from within Your Own Resident Library
- You can invoke RMS-11 operations from within a resident library if
you task build that library to include the module RORMSC from
RMSLIB.OLB and to exclude the following symbols using the task builder
GBLXCL option:
.SAVR1,$RMENT,$RMREM,$RMSEA,$RMERA,$RMOPE,
$RMPAR,$RMCRE,$RMREN,$RMDSP,$RMEXT,$RMCLO,
$RMCON,$RMDIS,$RMGET,$RMPUT,$RMUPD,$RMDEL,
$RMFIN,$RMTRU,$RMFRE,$RMREL,$RMFLU,$RMRWI,
$RMNXT,$RMSPA,$RMREA,$RMWRI,$RMWAI
Such a resident library may be clustered with the RMS-11 resident
libraries only if it contains ubsolutely ~c pure or impure dat~ (such
as, RMS-11 structures and call parameter blocks, RMS-11 internal
structures and buffers, file specifications, key or record buffers,
and so on) that RMS-11 needs during its processing. If such a library
is a default member of the cluster and has a non-null root segment, it
must not contain an RMS-11 get-space routine or the completion routine
for any asynchronous RMS-11 operation in its root segment.
When you build tasks that use your library, you include
resident library support as described in Section 8.1.2.1.

RMS-11

If instead you want to use RMS-11 disk-resident overlays, you can do
so as described in Section 8.1.1, and must also include ORG$
statements in the task's root segment appropriate to the needs of your
library.
To use nonoverlaid RMS-11 routines, you must tailor and
assemble your own copy of the source module LB:[l,l]RORMSl.MAC,
include it in the root of your task, and build nonoverlaid RMS-11 as
described in Section 8.1.

8-8

TASK BUILDING AND COMMON OPTIMIZATION TECHNIQUES
NOTE
In RSX-llM-PLUS systems that support
supervisor mode: under no circumstances
can RMS-11 operations be used in a
supervisor-mode resident library.

8.1.2.3 Deciding Between Types of Overlays - You should normally use
the RMS-11 resident libraries whenever possible, for the following
reasons:
Program execution speed will typically be faster than with
• disk-resident
overlaid RMS-11, and nearly as fast as with
nonoverlaid RMS-11.
•

Virtual address space required in your program will usually be
less than with nonoverlaid RMS-11, and may be less than with
disk-resident overlaid RMS-11, if you are able to cluster
RMS-11 with other libraries or, on RSX-llM-PLUS systems that
support supervisor mode, use RMSRES as a supervisor~mode
library.

tasks will build significantly faster and take up
• Your
significantly less space on disk than with other RMS-11
configurations.
•

when

You will usually not need to rebuild your tasks
enhancements or corrections to RMS-11 are issued.

Because the RMS-11 resident libraries can be shared among
• multiple
programs, using them will often result in more
efficient use of the system's physical memory.
Reasons that you might not use RMS-11 resident libraries include:

8.2

•

Your system manager has not included them in your system,
perhaps because very little system memory is available and
RMS-11 is seldom used.

•

You require indexed file organization support but your RSX-llM
system manager has chosen the optional version of RMSRES for
RSX-llM that does not include this support.

•

The special virtual address requirements of your task
permit the use of RMS-11 resident libraries.

not

PROGRAM DEVELOPMENT

You should consider performance while you are writing

application

program~

•

Your program's flow of operations can either cooperate with or
fight against the RMS-11 code structure.

•

Task-building consumes a significant portion of
resources. Minimize that time when you can.

8-9

your

machine

TASK BUILDING AND COMMON OPTIMIZATION TECHNIQUES
8.2.l

Flow of Operations Should_Reflect RMS-ll Code Structure

The overlay process causes a significant portion of the I/0 performed
for
a
program with disk-resident overlays.
Using RMSRES,
an
RSX-llM-PLUS system may be forced to perform I/O paging operations to
access RMS-11 code segments if physical memory is in short supply.
You should structure the task to maximize the time each segment stays
in memory and thus minimize the number of I/O operations. You do this
by placing similar RMS-11 operations together in your program.
This
process also makes you aware of the nature of the operations your
program is performing:
•

File-related operations
File-related operations are generally
required
at
the
beginning and end of processing.
Therefore, they are fairly
easy to group.

Example:

Open all files that the program uses and set up
record access streams at the beginning of the program.

all

Example:

the

Disconnect record access streams and close all
files at one time, probably at the end of the program.
NOTE
Most high-level
languages
automatically
perform
CONNECT and DISCONNECT operations during the execution
of file open and close statements.

•

Record operations
The primary overlay or (on RSX-llM-PLUS systems) paging burden
of your task comes from record operations. However, the
nature of processing often dictates the placement of record
operations in your program.
Therefore, the type and sequence
of these operations direct your optimization of the ODL files
(see Section 8.1.1).

Example:

If your task uses GET operations to read records
from a sequential file, and then uses PUT operations to write
records to an indexed file, you could reduce the number of
overlays required for those specific operations.

Example:

If your task uses GET operations to read records
from an indexed file and UPDATE operations to modify the
records, you should optimize those operations.

Whenever possible, perform operations on only one type of file
organization at a time.

8.2.2

Task Builder Considerations

The task builder contructs a task and ensures that the taskis
overlays,
if any, work properly.
To do this, the task builder must
know the task's overlay structure if you use disk-resident overlays:
you supply this information by means of an ODL file.

8-10

TASK BUILDING AND COMMON OPTIMIZATION TECHNIQUES
To reduce the time that the task builder needs to build your task, you
can reduce the number of overlays in the task (see Section 8.1.1).
Each overlay adds time to task building because it requires that a
symbol table be built and then resolved.
NOTE
If you use memory-resident
overlays
(resident
library} , you reduce task
builder overhead needed
to
process
overlay segments.
You can also reduce task building time by not requesting a map.
If
you really need a map for debugging, specify a short one (the
default) •
Note that the use of overlaid I- and
task building time.

8.3

D-space

segments

can

increase

VIRTUAL-TO-LOGICAL-BLOCK MAPPING

wnen RMS-il issues a data transfer request, it specifies a starting
virtual block number (VBN) and the size of the request in bytes to the
operating system.
The system maps the VBN onto a logical block number
(LBN)
that it must use to find the block on disk.
To do this, the
system uses a set of retrieval pointers, called a window, to the file.
The operating system creates a window in its part of memory by reading
the first set of pointers from disk when a task opens a
file.
These
pointers specify blocks on disk, and from the structure and content of
the pointers for a file, the system equates virtual blocks to logical
blocks.

8.3.1

Retrieval Pointers on Disk

The file directory contains the retrieval pointers for
representation depends on your operating system.

file.

The

The file control processor stores retrieval pointers in a file header,
using enough file headers to cover the file.
A file header can
contain up to 102 pointers. Each pointer consists of:
•

The number of blocks the pointer maps

•

The LBN where the group of blocks starts

The largest group of blocks that can be covered by one pointer is 256
blocks.
Therefore, one file header can map a maximum of 26,112
logical blocks.

8.3.2

Retrieval Pointers in Memory

The operating system keeps one window in memory for each file.
If
that window does not contain the retrieval pointer that covers the
virtual block requested by RMS-11, the system must bring more pointers
into memory in a process called window turning.

8-11

TASK BUILDING AND COMMON OPTIMIZATION TECHNIQUES
Window turning normally requires an I/O operation.
The operating
system builds file control block {FCBs) in memory when a task opens a
file.
An FCB contains information about one file header, including
the range of virtual blocks covered by the header's retrieval
po~nters.
Whenever the system has to turn a window, it consults the
FCBs for the file to determine which file header contains the
appropriate retrieval pointer. The file control processor then reads
that block from disk, requiring only one I/O operation {unless the
software needs one or possibly two overlays) •
Example: An evaluation of one application revealed that window
turning during record operations accounted for nearly 30 percent of
the I/O operations.

8.3.3

Optimizing Window Turning

When you reduce window turning, you improve performance.
You can reduce the I/O operations associated with window turning by
increasing a window size, maximizing contiguity, using areas in
indexed files, or increasing the size of the FllACP.
Increase Window Size: Use one of the following methods to increase
the number of retrieval pointers the system keeps in memory for the
file:
•

Initialize the disk volume that will contain the RMS-11 file
with a window size greater than the default of seven pointers
per window. See your system documentation for initialization
procedures.

the volume containing the RMS-11 file using the /WIN
• Mount
switch to specify a window size greater than the volume
default. See your system documentation
procedures.
•

for

volume

mounting

Use a MACR0-11 subroutine that sets the RTV field in the FAB
for
the
file.
See
the
RSX-llM/M-PLUS RMS-11 Macro
Programmer's Guide.

In each of these methods, you can specify a number of window pointers,
as follows:
•

If you specify a -1, the system tries to make the window large
enough to map the entire file,
using up to 81 retrieval
pointers.

•

If you specify a positive number of pointers, the system uses
that number, up to a maximum of 127 pointers in each window.

The initialization and mount methods apply to all files on a disk.
These methods cause the system to use more executive memory than when
you set window size for an individual file in a program.
Maximize Contiguity: To maximize contiguity, you should make the file
contiguous or,
if that is not possible, reduce the number of extents
in 'the file, making each extent as large as possible.

8-12

TASK BUILDING AND COMMON OPTIMIZATION TECHNIQUES
One retrieval pointer in memory can map up to 65,536 logical blocks.
When the file control processor reads a pointer from the header, the
software determines whether the extent mapped by the pointer is
logically contiguous with the extent covered by the preceding pointer
in the window.
If it is, the file control processor adds the extent
size to the size field of the pointer in the window, then it reads the
next pointer.
If the two extents are not contiguous, it adds the new
pointer to the window. This compaction extends across file headers.
In this way, any file can be mapped with a default window if the
is sufficiently contiguous.
Use Areas in Indexed Files:
and reduce window turning.

Areas localize successive block

file

requests

Increase the Size of the FllACP: The large version of the FllACP does
not require overlays of its own routines to perform window turning,
whereas the small version does.

8.4

OTHER OPTIMIZATIONS

You can improve the environment in which your RMS-11 task runs by:
•

Allocating more resources to the task

•

Improving disk usage

8.4.1

Allocating More Resources to the Task

You can improve the performance of a task by giving it more of the
system to use, more CPU time, more memory, and so on. You take those
resources away from other jobs, unless the system is not used to
capacity.
The techniques for allocating system resources vary by operating
system. Each of the following techniques affects system throughput by
changing the number of I/O operations your task requires to complete
its work.
•

Priorities

•

Checkpointing

•

Round-robin scheduling

•

Swapping

8.4.2

Disk Usage

You should consider the devices that store your data and task images
when you are optimizing the performance of an application.
Efforts at
improving disk usage often result in significant increases
in
performance.
•

Use the fastest disk drives available because the physical I/O
operation causes the most significant portion of I/O time.

8-13

TASK BUILDING AND COMMON OPTIMIZATION TECHNIQUES
•

Minimize I/O request overhead:
Reduce
I/O
necessary.

request

queues,

using

private

packs

Use a disk that has exclusive use of its disk controller.
If other disks must share the controller, a disk driver
that supports "overlapped seeks" is desirable.
•

If your system has multiple disk drives which are not heavily
used by other people, spread an application's files, including
disk-resident overlaid tasks, across the devices.
Thus, while
a
job runs, one disk device does not access more than one
file.
You can also put data files on a disk device other than
the one containing a disk-resident overlaid task image.
If you are using the resident library, and not overlays, you
do not consider the task file,
unless your code or your
language run-time facilities are overlaid.

•

Combine free blocks on a disk into one contiguous group using
the DSC utility.
By eliminating fragmentation, you are
increasing the chances that file extents are contiguous, even
if they are not requested that way.
The more contiguous the
file, the fewer the disk head moves required to access it.

8-14

APPENDIX A
FILE SPECIFICATION PARSING

A.l

STANDARD FILE SPECIFICATION SYNTAX

A full file specification consists of the following elements,
order listed:

the

device
directory
name
type
version

A.1.1

Device

The device element of the file specification names the device on which
the file resides.
For unit-record devices, such as terminals and
lineprinters, this is the only significant element in the file
specification.
The device specification consists of
2
alphabetic
characters
specifying the device name,
followed by 0 to 3 octal numeric
characters specifying the device unit number, followed by a colon (:).
If you use lowercase characters to specify the device name, RMS-11
will convert them to uppercase befo~e passing them to the operating
system. The device unit number must not exceed 377 octal;
if no unit
number is given, RMS-11 will specify unit number O.
Note that RSX-llM/M-PLUS systems allow definition of logical device
names that conform to the above description.
RMS-11 processes such
logical device names as well as physical device names.
For example:
db2:

and DB02:

Are equivalent

SY:

and syOO:

Are equivalent

A.1.2