Digital PDFs

AA-L669A-TC

April 1983

182 pages

Original

7.4MB

Document:	RSX-11M/M-PLUS RMS-11 User’s Guide
Order Number:	AA-L669A-TC
Revision:	0
Pages:	182
Original Filename:	http://bitsavers.org/pdf/dec/pdp11/rsx11m_plus/RSX11Mplus_V3.0_Jul85/AA-L669A-TC_RSX-11M-PLUS_V2.1_RMS-11_Users_Guide_Apr83.pdf

OCR Text

RSX-11 M/M-PLUS
RMS-11 User's Guide
Order No. AA-L669A-TC

April 1983
This manual provides information on file and task design using
RMS-11. The information includes design considerations for writing
application programs in both MACRO-11 and high-level languages.

SUPERSESSION/UPDATE INFORMATION:

This revised document
supersedes the RMS-11
User's Guide (Order No.
AA-D538A-TC).

OPERATING SYSTEM AND VERSION:

RSX-11 M Version 4.1,
RSX-11 M-PLUS Version 2.1

SOFTWARE VERSION:

RMS-11 Version 2.0

digital equipment corporation · maynard, massachusetts

First Printing, April 1983
The information in this document is subject to change without notice
and should not be construed as a commitment by Digital Equipment
Corporation. Digital Equipment Corporation assumes no responsibility
for any errors that may appear in this document.
The software described in this document is furnished under a license
and may be used or copied only in accordance with the terms of such
license.
No responsibility is assumed for the use or reliability of software on
equipment that is not supplied by Digital Equipment Corporation or its
affiliated companies.
Copyright

® 1983 by Digital Equipment Corporation
All Rights Reserved.
Printed in U.S.A.

The postpaid READER'S C,)MMENTS form on the last page of this document
requests the user's critical evaluation to assist in preparing future
documentation.
The following are trademarks of Digital Equipment Corporation:
DEC
DEC/CMS
DEC/MMS
DECnet
DECsystem-1O
DECSYSTEM-20
DECUS
DECwriter

DIBOL
EduSystem
lAS
MASSBUS
PDP
PDT
RSTS

RSX
UNIBUS
VAX
VMS
VT

~DmDDmD
ZK2l68

HOW TO ORDER ADDITIONAL DOCUMENTATION
In Continental USA and Puerto Rico call

800-258-1710

In New Hampshire, Alaska, and Hawaii call
In Canada call

603-884-6660

613-234-7726 (Ottawa-HUll)
800-267 -6146 (all other Canadian)

DIRECT MAIL ORDERS (USA & PUERTO RICO)*
Digital Equipment Corporation
P.O. Box CS2008
Nashua, New Hampshire 03061

DIRECT MAIL ORDERS (CANADA)
Digital Equipment of Canada Ltd.
940 Belfast Road
Ottawa, Ontario K1 G 4C2
Attn: A&SG Business Manager

DIRECT MAIL ORDERS (INTERNATIONAL)
Digital Equipment Corporation
A&SG Business Manager
c/o Digital's local subsidiary or
approved distributor

* Any prepaid order from Puerto Rico must be placed

with the local Digital subsidiary (809-754-7575)
Internal orders should be placed through the Software Distribution Center (SDC), Digital Equipment
Corporation, Northboro, Massachusetts 01532

CONTENTS

Page
PREFACE
MANUAL OBJECTIVES • • • •
INTENDED AUDIENCE • • • • •
STRUCTURE OF THIS DOCUMENT • • • • • • • • •
ASSOCIATED DOCUMENTS • •
CONVENTIONS USED IN THIS DOCUMENT

ix
ix
ix
•

•

SUMMARY OF TECHNICAL CHANGES. • • • • • • • • • • • • • • • • •
CHAPTER 1
1.1
1.1.1
1.1.2
1.1.3
1.1.4
1.1.5
1.2
1.2.1
1.2.2
1.2.3
1.2.4
1.3
1.3.1
1.3.2
1.3.3
1.4
1.5
CHAPTER 2
2.1
2.2
2.2.1
2.2.2
2.2.2.1
2.2.2.2
2.2.2.3
2.2.3
2.2.3.1
2.2.3.2
2.2.3.3
2.2.4
2.3
2.4
2.4.1
2.4.1.1
2.4.1.2

•

• x
• x

xiii

RMS-11 CONCEPTS AND PROCESSING ENVIRONMENT
CONCEPTS OF DATA ORGANIZATION AND ACCESS • • • • • 1-1
Records
• • • • • ••
••••
• • • • • 1-1
Files
. • • • • •
• 1-2
Access • • •
• • • •
• • • • • • • • • 1-7
Processing •
••••• • • •
• • • • 1-10
File Maintenance •
• • • • • •
1-10
RMS-11 IMPLEMENTATION OF DATA ORGANIZATION AND
ACCESS • • • • • • • • •
1-11
• • • • • • • • • 1-11
RMS-11 Record Formats
• • • • • • • • • • 1-11
RMS-11 File Organizations
RMS-11 Record Access Modes •
1-12
RMS-11 utilities • • • • • •
1-12
RMS-11 PROCESSING ENVIRONMENT
1-12
RMS-11 Task Structure • • • •
1-13
RMS-11 Record Processing •
1-15
RMS-11 File Processing • •
1-16
FILE ATTRIBUTES • • • • • • • • •
1-18
PROCESSING BY BLOCK ACCESS • •
1-20
APPLICATION DESIGN
WHEN TO DESIGN • •
•••••••
• • • • • 2-2
DESIGN CONSIDERATIONS • • • • • • •
• • • 2-3
Speed
•••.••••
• • • • • • • • 2-3
Space • • • •
• • 2-4
Data Storage • • • • • • • •
•••••
• 2-5
Task Size
• • • •
• • 2-5
Buffer Sizes •
• • • •
• • • • • • • 2-5
Shared Access
• • • •
2-5
Bucket Locking •
• • • • •
• • 2-8
Sharing among Access Streams •
• • 2-9
Programming Considerations •
2-10
Ease of Design • • • • • • • •
2-10
DESIGN PROCESS • • • • • • • • • • • • • •
2-11
SELECTING A FILE ORGANIZATION
• • • • • ••
2-11
Record Formats • • • • • • • •
•• • •
2-15
Fixed-Length Format • • • • • • • • • • • • • 2-15
Variable-Length Format •
2-15
iii

CONTENTS

2.4.1.3
2.4.1.4
2.4.1.5
2.4.2
CHAPTER 3
3.1
3.2
3.3
3.3.1
3.3.2
3.3.2.1
3.3.2.2
3.3.3
3.4
3.4.1
3.4.2
3.5
3.5.1
3.5.1.1
3.5.1.2
3.5.1.3
3.5.1.4
3.5.1.5
3.5.1.6
3.5.1.7
3.5.1.8
3.5.1.9
3.5.2
3.5.2.1
3.5.2.2
3.5.3
3.5.3.1
3.5.3.2
3.5.3.3
3.5.3.4
3.5.3.5
3.5.4
CHAPTER 4
4.1
4.2
4.3
4.3.1
4.3.2
4.3.2.1
4.3.2.2
4.3.3
4.3.4
4.4
4.4.1
4.4.2
4.5
4.5.1
4.5.1.1
4.5.1.2
4.5.1.3
4.5.1.4
4.5.1.5
4.5.1.6
4.5.1.7
4.5.1.8

Variable-with-Fixed-Control Format •
Stream Format
Undefined Format •
I/O Techniques •

2-16
2-16
2-17
2-17

SEQUENTIAL FILE APPLICATIONS
FILE STRUCTURE • • • • •
. • •.
• • • 3-1
RECORD SIZE
. • • • .
3-2
FILE DESIGN
. • • • • • • • • 3-3
Data Storage Medium
. • . • •
• 3-3
File Allocation
. • • • • • • • • . • • • 3-4
Initial Allocation •
. • • • • • . • . • • • 3-4
Default Extension Quantity.
. • • . • • • 3-4
Contiguity • • • . • • • •
•••.
3-5
ACCESS SHARING • • • • • • •
.•• •
• • 3-5
Record Access to Sequential Files
• • • • • • • 3-6
Block Access to Sequential Files • •
• • • 3-6
RECORD AND FILE PROCESSING OF SEQUENTIAL FILES • • 3-6
Record and Stream Operations •
• • 3-7
CONNECT
. • • •
• • • .
3-7
DISCONNECT
•.••
• . • • •
• 3-7
FIND • • • • • •
• • • 3-7
FLUSH
• • • 3-9
GET
• • • • • • • •
• • • • • . • • • 3-9
PUT
• • • • •
• • • •
3-10
REWIND • • • • • . • • • • •
3-11
TRUNCATE •
. . • •
• • • •
3-12
UPDATE • .
3-12
Record Transfer Modes
•••• • . • • • • ••
3-13
Move Mode
• • • •
• • • • 3-13
Locate Mode
• • • • • •
• • • • • 3-14
I/O Techniques • • • • •
• • • • • • • • 3-14
Asynchronous Record Operations .
3-14
Deferred Write • • • • •
. • • • • ••
3-14
Multiple Buffers . • • • • • •
• • • • • • 3-14
Multiple Access Streams
• • • • . • • • •
3-15
Multiblock Count • • • • • . •
3-15
File and Directory Operations
3-15
RELATIVE FILE APPLICATIONS
FILE STRUCTURE •
••••••
• • 4-1
RECORD SIZE
• . • • • 4-2
FILE DESIGN • • • • • • . • • • •
4-2
Bucket Size
• • • •
• • • • 4-2
Fi~e Allocation
• • • •
• 4-3
Initial Allocation • • • • • •
• • • • • 4-3
Default Extension Quantity •
• • • 4-4
Contiguity. • • • • • • • •
• • • • 4-4
Maximum Record Number
• • • • • • • • 4-5
ACCESS SHARING • • • • • • • •
. • • • • • • 4-6
Record Access to Relative Files
• • • • . • • • 4-6
Block Access to Relative Files • •
• • 4-6
RECORD AND FILE PROCESSING OF RELATIVE FILES • • • 4-6
Record and Stream Operations •
• • • • • • • 4-7
CONNECT
• • • 4-7
DELETE • • •
••••.•••
4-7
DISCONNECT .
••••••••
• 4-8
FIND • •
• . ••
• • • 4-8
FLUSH
• • • • •
4-10
GET . • . • •
. • • • 4-10
PUT
• • .•
•••••••••
4-11
REWIND • . • • •
4-12
iv

CONTENTS
4.5.1.9
4.5.2
4.5.2.1
4.5.2.2
4.5.3
4.5.3.1
4.5.3.2
4.5.3.3
4.5.3.4
4.5.4
CHAPTER 5

UPDATE • • • • • • • •
Record Transfer Modes
Move Mode
• • • • • • • • • •
Locate Mode • • • • • • • • •
I/O Techniques • • • • •
••••
Asynchronous Record Operations • • • • • • • •
Deferred write • • • • • •
• •••••
Multiple Buffers • • • • • • • • • • • • •
Multiple Access Streams
••••
File and Directory Operations
•••••

4-12
4-12
4-12
4-13
4-14
4-14
4-14
4-15
4-15
4-15

INDEXED FILE STRUCTURE AND ACCESS

5.1
PHYSICAL FILE STRUCTURE • • • •
• • 5-2
CONCEPTUAL FILE STRUCTURE
• • • • 5-4
5.2
5.2.1
Data • • • • • • • • • •
• • 5-5
5.2.1.1
Level 0 of the Primary Index •
• • ••
• 5-5
Level 0 of an Alternate Index
• • • • • • • • • 5-5
5.2.1.2
Indexes
••••••••••••
• • • 5-6
5.2.2
Random Access Using the RMS-ll Indexed File
5.2.3
Structure • • • • • • • • • • • • • • • • • • • 5-7
Why this Structure? • • • • • • • • • • • • • • 5-8
5.2.4
PROCEDURES FOR PERFORMING RANDOM RECORD OPERATIONS 5-9
5.3
5.3.1
Writing a Record • • • • • • • • • • • •
5-10
5.3.1.1
Simplest Case • • • • • • • • •
5-10
5.3.1.2
Bucket Splitting • • • • • • • •
• • • • 5-11
Incremental Reorganization • • •
• • • • • 5-12
5.3.1.3
Getting and/or Finding a Record
••••
5-13
5.3.2
Updating a Record
••••••••
5-14
5.3.3
5.3.4
Deleting a Record
•••••••
5-15
5.4
PROCEDURES FOR PERFORMING SEQUENTIAL RECORD
OPERATIONS • • • • • • • • • • • • •
5-16
I/O COST OF PERFORMING RECORD OPERATIONS. •
5-17
5.5
CHAPTER 6
6.1
6.2
6.2.1
6.2.2
6.2.2.1
6.2.2.2
6.2.2.3
6.2.2.4
6.2.2.5
6.2.2.6
6.2.3
6.2.4
6.2.5
6.2.5.1
6.2.5.2
6.2.5.3
6.3
6.4
6.5
6.5.1
6.5.2
6.5.3
6.6
6.6.1
6.6.2
6.7
6.7.1

INDEXED FILE DESIGN
RECORD SIZE
. . . .
KEY SELECTION
• • • •
Number of Keys •
. . .
Key Data Types • • • • •
String Type
• • ••
•••••
Two-Byte Signed Integer Type •
Four-Byte Signed Integer Type . .
Two-Byte unsigned Bina~y Type
Four-Byte Unsigned Binary Type • •
Packed Decimal Type
Key Size • • • • • • • • •
·
position of Key in Record
•
Key Characteristics
• • • • • • ·
Duplicates •
• •••
·
Changes
• • • • • • • • . . . . .
Null Key • • • • •

. 6-1

• • • • • • • 6- 2
.
·

.
.

. 6-2
. 6- 3

• • ••

• 6-3

·
.

. . . .
. . . .

. 6-4
. 6-4

.
.

• • 6-5
• 6-5
• 6-6

• • •
.

• • ••
.

. 6- 8
. 6- 8
. 6-9

PLACEMENT CONTROL
• • • • •
BUCKET SIZE
Bucket Size for Primary Index • • • • • • • •
Bucket Sizes for Alternate Indexes •
Program Syntax • • • • • •
FILE ALLOCATION • • • • • •
• • • •
Initial Allocation • • • • • • • • •
Default Extension Quantity •
• • • •
POPULATION TECHNIQUES • • • •
Ascending Order by Primary Key •

6-10
6-10
6-13
6-15
6-16
6-19
6-21
6-22
6-22
6-26
6-26
6-27

AREAS

•

.
.
.

. 6-6

• 6-7

•

CONTENTS

6.7.2
6.7.2.1
6.7.2.2
CHAPTER 7
7.1
7.1.1
7.1.2
7.2
7.2.1
7.2.2
7 •2 •3

7.2.4
7.2.5
7.2.6
7.2.7
7.2.8
7.2.9
7.3
7.3.1
7.3.2
7.4
7.4.1
7.4.2
7.4.3
7.4.4
7.4.5
7.5
CHAPTER 8

Random Insertions after File Population
Bucket Fill Size
••••
Mass Insertion • • • • . • .

6-28
6-28
6-29

RECORD AND FILE PROCESSING OF INDEXED FILES
ACCESS SHARING • . . . . •
Record Access to Indexed Files • •
Block Access to Indexed Files
RECORD AND STREAM OPERATIONS
CONNECT
. . • •
DELETE •
• . • • • .
DISCONNECT •
. • • • .
FIND • • •
FLUSH
••••
GET . • • . • • • •
PUT
• • • • • • •
REWIND .
• • ••
UPDATE
.•••
• • • .
RECORD TRANSFER MODES
• • • •
Move Mode
• • . • • • • • • •
Locate Mode
• • • •
• • • • • •
I/O TECHNIQUES • • • • •
Asynchronous Record Operations • . . • • .
Deferred Write . • • • . •
. • •
Multiple Buffers . • . . • • •
• • • •
Multiple Access Streams • • • .
Sequentially Reading Write-Shared Files
FILE AND DIRECTORY OPERATIONS
• • . • • • •

•
·
.
·
·

• . 7-1
7-1
• • 7-1
7-2
• • 7-2
7-3
• • 7-3
• • 7-3
• • 7-5
• 7-5

• • • 7- 5
• 7- 6

. • • 7-6
•

•

• 7- 6

• • • 7-7
•

•

7- 7

· • • 7-8
• . • 7- 8
• . • 7-8
• • • 7- 9

7-10
7-10
7-10

TASK BUILDING AND COMMON OPTIMIZATION TECHNIQUES

8.1
TASK BUILDING WITH RMS-ll ROUTINES . . • • • · • • 8-1
8.1.1
Disk-Resident Overlays • • . • • • • • • .
• 8-3
8.1.1.1
ODL Files
••..••••••..
8- 5
8.1.2
Memory-Resident Overlays • • • • . . • • •
8-6
8.1.2.1
Task Building against the RMS-ll Resident
Library • • • • • • • • . •
•• • •
• 8-6
8.1.2.2
Using RMS-ll Operations from within Your Own
Resident Library • • • • • • • • • • • • • • • • 8-8
8.1.2.3
Deciding Between Types of Overlays • • •
• • 8-9
8.2
PROGRAM DEVELOPMENT • • • . • • • • . . •
• • 8-9
8.2.1
Flow of Operations Should Reflect RMS-ll Code
Structure • • • . • • • • • .
8-10
8.2.2
Task Builder Considerations
• • .•••.
8-10
8.3
VIRTUAL-TO-LOGICAL-BLOCK MAPPING • • • • • •
8-11
8.3.1
Retrieval Pointers on Disk • • • • • • • •
8-11
8.3.2
Retrieval Pointers in Memory •
8-11
8.3.3
Optimizing Window Turning
• • • • • • • • • • 8-12
8.4
OTHER OPTIMIZATIONS • • . • • •
8-13
8.4.1
Allocating More Resources to the Task
8-13
8.4.2
Disk Usage • • . • • . • • •
.•.•
8-13
APPENDIX A
A.l
A.l.l
A.l.2
A.l.3
A.l.4
A.l.5
A.2
A.2.l

FILE SPECIFICATION PARSING
STANDARD FILE SPECIFICATION SYNTAX •
Device • • •
Directory
.
Name • .
Type • • • •
Version
ANSI MAGNETIC TAPE FILE SPECIFICATION SYNTAX
Dev ice • . . • .
vi

A-I
• • • A-I
. . .' A-I
• • • A-2
• • • A-3
A-3
.
A-4
• A-4

CONTENTS

A.2.2
A.2.3
A.2.4
A.3
APPENDIX B
B.1
B.2
B.3

Directory
• • • • A-4
Quoted String
• • • • • • • • • • ••
• A-5
Version
• • • •
• • • • • • • • A-5
GENERATION OF A FULL FILE SPECIFICATION
• • • A-5
REMOTE FILE AND RECORD ACCESS VIA DECNET
REMOTE NODE SPECIFICATION • • • •
REMOTE ACCESS ENVIRONMENTS • • • •
REMOTE ACCESS POOL CONSIDERATIONS

• B-2
• B-3
• B-3

INDEX

FIGURES
FIGU:RE

1-1
1-2
1-3
1-4
1-5
1-6
1-7
1-8
1-9
2-1
2-2
2-3
2-4
3-1
4-1
5-1
5-2
5-3
5-4
5-5
5-6
6-1
6-2
6-3
6-4
7-1
8-1
8-2

Record Formats • • • • • • • •
• • 1-2
Fi 1es
•••••••••••
•••••
• 1-3
Sequential File Organization • • • • •
• 1-4
Relative File Organization •
• • • 1-5
Indexed File Organization • • • • • • • • • • • • 1-6
Indexed File Example • • • • • • • • • • •
• • 1-6
Record Access Modes
• • 1-8
RMS-ll Task Structure • • • • • • • • •
1-14
Records Spanning Blocks
• • • • 1-17
Time Factors in an I/O Operation •
• • 2-4
System Protection Concepts • •
• 2-6
Bucket Locking Example • • • •
••••
• 2-9
Count Field on Disk and Tape
••••
2-15
RMS-11 Task Structure • • • • • • •
3-13
RMS-1l Task Structure • • • • • • • • • • • • • 4-13
Indexed Fi le wi th and wi thout Areas
• • • • • • • 5-3
Forma t ted Bucket .' •
• • • • • •
• • 5-4
Index as a Pyramid •
• • • • • •
• • • 5-4
Format for Secondary Index Data Record
• • 5-6
Example of a Primary Index • • • • • • • • • • • • 5-7
Search Time Curves •
• • • •
• • 5-9
Sing Ie-Area Indexed Fi le •
••••
6-11
Example of Single-Area Indexed File
6-11
Two-~rea Indexed File
• • • •
• • • • • 6-12
Example of Multi-Area Indexed File •
6-13
RMS-l1 Task Structure • • • • • • • •
• 7-7
Source-to-Task Sequence
• • 8-2
RMS-l1 Tasks • • • • • • • • • • • • • • • • • • • 8-4

TABLES
TABLE

1-1
2-1
2-2
3-1
3-2
4-1
5-1
6-1

Record Formats and File Organizations
File Organization Characteristics and
Capabilities • • • • • • • • • • • • • •
File Organization Advantages and Disadvantages
End-of-B1ock Indicators
• • • • • • •
Sequential File Data Sizes (in bytes)
•
Relative File Data Sizes (in bytes)
I/O Cost of Performing Record Operations •
Key Data Types • • • • • • • • • • • • • • • •

vii

1-20
2-12
2-13
• 3-2
• • 3-3
• • 4-2
5-18
• • 6-6
•

PREFACE

MANUAL OBJECTIVES
This document is a guide to using RMS-ll capabilities and operations
in file and task design for application programs written in either
MACRO-ll or high-level languages.

INTENDED AUDIENCE
This document is intended for application programmers who want to
achieve optimal performance with new applications they are writing or
with existing applications.
NOTE
Only MACRO-ll programmers can use the
full
set
of
RMS-ll
capabilities.
Subsets of
these
capabilities
are
available
to
high-level
language
programmers.
See
your
high-level
language documentation to determine:
•

What RMS-ll facilities you can use in
your high-level language

•

The syntax for using these facilities

STRUCTURE OF THIS DOCUMENT
This manual contains eight chapters and two appendixes:
•

Chapter 1, RMS-ll Concepts and
Processing
Environment,
introduces the concepts of data organization and access and
the RMS-ll implementation of these concepts.

•

Chapter 2, Application Design, presents general considerations
that apply to application design and information that will
help the application designer select a file organization.

•

Chapter 3, Sequential File Applications, discusses
file structure, design, and processing.

•

Chapter 4, Relative File Applications, discusses relative file
structure, design, and processing.

sequential

PREFACE
•

Chapters 5, Indexed File Structure and Access, 6, Indexed File
Design, and 7, Record and File Processing of Indexed Files,
discuss indexed file structure, design, and processing.

•

Chapter 8, Task Building and Common Optimization Techniques,
describes techniques that can be used to optimize application
programs that use RMS-II, regardless of the file organization
selected.

•

Appendix A, File Specification Parsing,
handling of file specifications.

•

Appendix B, Remote File and Record Access via DECnet, briefly
describes the remote access environment and remote file
specification syntax.

documents

RMS-Il's

ASSOCIATED DOCUMENTS
In addition to this user's guide,
contains the following manuals.

the

RMS-II

documentation

set

RSX-IIM/M-PLUS RMS-ll: An Introduction presents the major concepts of
RMS-II,
lntroduces the--RMS-II operations, and defines key terms
required for understanding RMS-II capabilities and functions.
You
should read the introduction before proceeding to other manuals in the
RMS-II documentation set.
The RSX-IIM/M-PLUS RMS-II Macro Programmer's Guide is a reference
document for MACRO-II programmers that descrlbes the macros and
symbols that make up the interface between a MACRO-II program or
subprogram and the RMS-II operation routines.
ThE~ RSX-IIM/M-PLUS RMS-II Utilities
manual
is both a user and a
reference document for all users, both programmers and nonprogrammers.
It describes the RMS-II utilities that are available for creating and
maintaining RMS-II files.

In addition, the Mini-Reference Insert includes an easy-reference
guide for users who are famlliar with RMS-II and its documentation.
It summarizes the RMS-II utilities and error codes.

CONVENTIONS USED IN THIS DOCUMENT
Convention

Meaning

UPPERCASE

Uppercase words and letters, used in format examples,
indicate that you should type the word or letter
exactly as shown.

lowercase

Lowercase words and letters, used in format examples,
indicate that you are to substitute a word or value
of your choice.

quotation marks

The term "quotation marks" refers to double quotation
marks (").

apostrophes

The term "apostrophe" refers to
mark (').

single

quotation

PREFACE
[

]

Square brackets indicate that the
optional.
A horizontal ellipsis indicates
item(s)
can be repeated one
example:

enclosed

item

that the preceding
or more times. For

file-spec[,file-spec ••• ]
A vertical ellipsis indicates that not all of
statements in an example or figure are shown.
TKB>//

the

In examples of commands you enter
and
system
responses, all output lines and prompting characters
that the system prints or displays are shown in black
letters.
All the lines you type are shown in red
letters.

Unless otherwise noted, all numeric values are represented in
notation.
Unless otherwise specified, you terminate
RETURN key.

commands

decimal

pressing

the

SUMMARY OF TECHNICAL CHANGES

RMS-ll Version 2.0 supports random access to fixed-format disk
sequential files and sequential block access to disk files of any
format and organization.
The RMS-ll Version 2.0 resident libraries are task independent.
This
means that once a program is linked with this library, the library can
be rebuilt or replaced without requiring that the task linked to it be
rebuilt.
RMS-ll Version 2.0 contains no library equivalent to the RMSSEQ
memory-resident library included with RMS-ll Version 1.8. The RMSRES
resident library or the disk-resident ODL files can be used to obtain
equivalent functionality and performance.
New versions of the RMS-ll Version 1.8 ODL files are provided.
These
ODL files are:
RMSllS.ODL,
RMSIIX.ODL, RMS12X.ODL, and RMSll.ODL.
The Version 1.8 ODL files will still work with Version 2.0, but the
new versions will be more efficient.
RMS-ll Vl.8 ODL structures other
than RMSllS.ODL, RMSIIX.ODL, and RMS12X.ODL may not work correctly
with the RMS-ll V2.0 code; when in doubt, verify them by comparison
with the V2.0 RMSll.ODL file.
In addition, two new ODL files are
provided with Version 2.0: RMS12S.0DL and DAPIlX.ODL.
Files with stream and VFC records can now be created on unit-record
devices
to avoid the need for special-case code in copy-type
operations.
•

For VFC files, the record header is thrown
unless the file is a "print format" file.

away

output

•

For stream files, if none of the 3 carriage control bits is
set
(print file format, carriage control, or FORTRAN carriage
control), and if the last character is not a linefeed,
formfeed,
or
vertical tab,
the carriage-return/linefeed
(CR/LF) is appended at the end of the record.

•

For stream files, if either the carriage control or FORTRAN
carriage
control attribute is set, and if the last 2
characters of the record are CR/LF, the trailing CR/LF is
stripped off and then definition of the carriage control
attribute (CR or FTN) is applied.

For isimilar ease-of-copying reasons, RMS-ll now allows creation of
relative and indexed files for output to nondisk devices (for magtape,
however, the record format must be variable length or fixed length).
The lRMS-ll File Design utility (RMSDES) is a new utility that allows
you to design and create files interactively.
It is fully documented
in the RSX-IIM/M~PLUS RMS-ll Utilities manual.
RMS-ll Version 2.0 supports five new directory operations:
$ENTER,
$PARSE, $REMOVE, $RENAME, and $SEARCH. These operations are fully
documented in the RSX-IIM/M-PLUS RMS-ll Macro Programmer's Guide.

xiii

SUMMARY OF TECHNICAL CHANGES
RMS-ll Version 2.0 supports a new wildcard file specification facility
and a new print-record output handling format.
These are also fully
documented in the macro programmer's guide.
User-provided interlocks allow a special, limited form of sequential
file sharing among a group of accessors that includes at most one
read/write accessor and any number of read-only accessors.
If suitable DECnet facilities exist on your system and on the target
system, RMS-ll Version 2.0 will allow file and record access to files
on remote network nodes, if those nodes include an RMS-ll-based file
access listener (FAL).
For magtape, RMS-ll now allows fixed-format records to be less than 18
bytes.
Files with stream or VFC records can now be created on unit-record
devices.
In addition, RMS-ll now allows the creation of relative and
indexed files for output to nondisk devices, although they will be
treated as sequential files.
<CTRL/Z> and <ESC> are no longer recognized as record terminators for
stream files,
and <CTRL/Z> is no longer recognized as a file
terminator for stream files.
RMS-ll Version 2.0 pads stream files with null characters, to the high
block of the file (not just to the end of the current block).
The memory-resident library RMSRES can be clustered
resident library that supports clustering.

with

any

other

On RSX-llM-PLUS systems that include hardware support
mode, RMSRES can also be used in supervisor mode.

for

supervisor

On RSX-llM systems, an optional subset library, which contains support
for sequential and relative files only, is available.
NOTE
All new RMS-ll features
are
fully
accessible only to MACRO-Il programmers.
See
your
high-level
language
documentation for supported features.

xiv

CHAPTER 1
RMS-ll CONCEPTS AND PROCESSING ENVIRONMENT

Your business, whether commercial, scientific, governmental,
or
educational, relies on data. That data indicates the current state of
your business and helps you control the future of the business.
TherE~fore,
you want fast, efficient access to the right data when you
need it.
You are familiar with dealing with data on paper and know that records
of transactions and reports on your business's activities can occupy a
very large number of file folders.
You also know that finding exactly
the data you need can be a time-consuming process.
Computer hardware, however, with its speed and mass data storage
capabilities, provides the means for fast, efficient access to data.
Computer software provides the means for translating the data from the
format you use to a format the computer system can handle -- and back
again.
RMS-ll is such a translater between you and your system. This chapter
introduces RMS-ll In terms of general concepts of data organization
and access, which apply regardless of whether data is stored on paper
or
within a computer's memory.
It then discusses the RMS-ll
implementation of data organization and access, and the RMS-ll data
processing environment.

1.1

CONCEPTS OF DATA ORGANIZATION AND ACCESS

This section examines the general concepts of data organization, using
images from the noncomputer environment you may be most familiar with.

1.1.1

Records

When data is stored on paper, it is recorded in groups of items whose
form is repeated throughout the data. Each group of items is called a
recol:d. Within each record are the specific items of data you are
concerned with.
For example, all the information on an employee
constitutes a personnel record; all the information on a stock item
constitutes an inventory record.
On paper, a record can be a form; different types of records require
different forms.
Some forms are always the same length;
their
information does not expand with time or use. For example, a product
information form does not vary in size.
If the facts about a product
change, you fill out a new form.
If a new product is added, you also
fill out a new form.

1-1

RMS-11 CONCEPTS AND PROCESSING ENVIRONMENT
Other forms vary in length with time and use, continuing on to new
pages as they grow. For example, an employee with the company for 10
years has more data in his or her personnel record than a new
employee.
Other forms might use a combination of these two formats.
For
example, a record of service on a piece of equipment might begin with
control information describing the specific piece of equipment
(name,
model number, date of installation, and so on) and continue on to new
pages documenting the service performed on it.
Figure 1-1 illustrates various record formats.

PRODUCT RECORD

SALARY HISTORY

(MAY CONT.)
_
(MAY CONT.)
•• ,
(MAY CONT.) .•
-

I
I
I
I

I
I

(MAY CONT.).
(MAY CONTI
(MAY CONT.) •
(MAY GuN .)
ZK-1170-82

Figure 1-1: Record Formats

1.1.2

Files

When data is stored on paper records,
it is usually gathered into
files and stored physically in filing cabinets, organized by related
records. For example, all employee records might be stored in one
file and placed in one drawer of the filing cabinet.
A file not only keeps related data in one place,
that data from other, unrelated data.

also

segregates

As data grows,
the file and storage requirements become
more
complicated, and the number of filing cabinets multiplies. Then, the
files acquire names or numbers, the drawers acquire signs indicating
the
contents
of
the
drawers
and
who
may use them, and
cross-referencing systems are introduced to help locate data.
These
identifying characteristics and restrictions upon who may read or
alter specific files can be called attributes.

1-2

RMS-ll CONCEPTS AND PROCESSING ENVIRONMENT
Figure 1-2 illustrates data storage using filing cabinets.

ZK-1167-82

Figure 1-2: Files
In general, the person who uses a
file establishes a method of
organizing the records within it.
This method reflects the file's use
and dictates what information is needed and how much time is required
to locate a record within the file.
There are several typical methods for organizing records
in a
file,
depending on how the records are used.
If you generally use all the
records in a file whenever you open it (that is, you have little or no
need to locate individual records in the file) and the order of the
records
is not
important, then you can organize
the
records
sequentially:
•

The records assume the physical sequence in which they are
inserted
into the file (that is, records are appended to the
file) •

•

No empty spaces are left in the sequence of records, where
records could be inserted later.
Each record, except the
first, has a record before it;
each record, except the last,
has a record following it.

1-3

RMS-ll CONCEPTS AND PROCESSING ENVIRONMENT
Employee payroll records, for example, might be kept in a sequential
file.
Because all the records must be accessed every time the payroll
is done, sequential file organization would allow easy access to the
records.
The overhead and maintenance for sequential files
is minimal.
To
insert a record into the file, you simply put it after the last record
already there.
Figure 1-3 illustrates sequential file organization.

ZK-1168-82

Figure 1-3: Sequential File Organization
For more access flexibility than sequential files, if you want to be
able to locate individual records easily, you can set up a series of
file folders and number them in sequence from first folder
to last.
Each folder is the same size~
it holds only one record, but it can be
empty. Thus, you do not have to look sequentially through the records
to locate the one you want (although you can if you want to access all
the records).
You use the numbers on the folders to locate or insert
records;
each record will be numbered relative to the beginning of
the file.
The numbers can relate to some numbering system meaningful
to your business:
for example, order numbers or part numbers.
Figure 1-4 illustrates relative file organization.

1-4

RMS-ll CONCEPTS AND PROCESSING ENVIRONMENT

ZK-1171-82

Figure 1-4: Relative File Organization
If you have a large file and most of the time you want to be able to
locate individual records, you may want to index your files.
Indexing
is useful when you want to be able to use several kinds of information
to locate records.
For example, in an employee file, you may want to
use last-name information to obtain a report on all employees, and
job-designation information to obtain a report on all clerical
employees.
When you open an indexed file drawer, you find records filed with
numbered tabs separating them. At the front of the drawer is a set of
small card files, containing groups of cards separated by dividers.
The cards in each of these small card files are an index to the
records at the back of the file.
To insert a record in the file,
you
find the data item marked "key" on the record, and using that
information, consult the appropriate index to determine where the
record should be inserted.
Figure 1-5 illustrates indexed file
organization.
To find a record in an indexed file, you look for
the specific key
information in the appropriate key file and use that information to
locate the record.
For example,
if you want the record of a
transaction with the Q,R,&S Company, you open the indexed file drawer
for transactions, which contains data records filed at the back and
indexes at the front.
Figure 1-6 illustrates this example.

1-5

RMS-11 CONCEPTS AND PROCESSING ENVIRONMENT

SEQUENTIAL
ACCESS

BY
KEY a

ZK-1169-82

Figure 1-5: Indexed File Organization

ZK-1175-82

Figure 1-6: Indexed File Example

1-6

RMS-ll CONCEPTS AND PROCESSING ENVIRONMENT

You know that company name is the primary key for records in the file
and that index 0 indexes the primary keys-.--The first record in index
o is the root, which lists selected primary key values, that is, the
company names,
in alphabetical order. Not all company names appear
here:
instead, a small subset of names, distributed fairly evenly
across the full set of names,
is used as the highest level of
indexing. By selecting one name, you establish the region of the file
(range of names) that interests you.
You look down the list until you find a name that either matches
Q,R,&S or occurs after this name in the alphabet. You find Rhesus,
Inc with the number 3 alongside it.
You put the root record back in the file and go to the first divider
and the third index record behind it. Again, the set of names here is
incomplete: only a small set of names distributed fairly evenly
across the range covered by the highest level index entry exists.
This provides an intermediate level of indexing, and further limits
the range of names in which you are interested.
Rhesus, Inc is the last entry on this card, but you scan the list and
find the name Queeg Co, which is the first entry at or after Q,R,&S in
alphabetical sequence. The entry for Queeg Co has the number 7
alongside it.
So you reach into the data records at the back of the drawer to tab
number 7. You search sequentially through the records behind this tab
until you find the record of the Q,R,&S transaction.
For another example, using the same transaction file, suppose you want
to find a record but all you know is its transaction number.
Fortunately, the second alternate key for the file is transaction
number.
Index 2 indexes the second alternate keys (recall from the
previous example, that the indexes are numbered starting with primary
index 0). You look at the root record in index 2 and move through the
index as you did in the previous example until you find a card listing
the transaction number you are looking for.
Next to the number is the
code 7/5.
So you reach into the data records at the back of the drawer to tab
number 7 and count back to the fifth record behind the tab.
You find
that the transaction you are looking for was made with the Q,R,&S
Company.
Here, only one level of indexing -- the root record -- was used.
If
many records exist in the file, another intermediate level would also
be used, as it was in index O.
Use of intermediate index levels
allows the number o~ entries you must scan in each level to be small,
regardless of the total number of records in the file.

Access

1.1.3

Once you have records organized in a file, you
them in two ways:

can

get,

access,

•

You can search all the records one after the other.
called sequential access.

This

•

You can use an identifier to
This is called random access.

record.

locate

individual

Note that access means not only retrieving a record from
putting a record into the file as well.

1-7

file

but

RMS-ll CONCEPTS AND PROCESSING ENVIRONMENT
Figure 1-7 illustrates the random and sequential access modes.

ZK-1172-82

Figure 1-7: Record Access Modes

Sequential Access
For sequential access, you pick a point in the file and access the
records beyond that point one at a time. At times, the starting point
is the beginning of the file because you want to look at, or access~
each record
in the file.
Other times, you may begin midway through
the file.
To read each record, you take it out of the file, marking the position
of the record you just removed with a card or some other marker so
that you know:
•

Where to put the record back into the file

•

Where the next record is

To insert records sequentially, you reach into the drawer to the place
where you want the records to go and mark the position of that place.
Often, the point at which you will insert the new records will be the
end of the file.
At other times, it may be midway through the file.

1-8

RMS-ll CONCEPTS AND PROCESSING ENVIRONMENT
You insert the records by taking the first record from the stack of
new records and slipping it into position in the file.
You then mark
the position after the record you just inserted and add the next
record in that position. You continue in this manner until all the
new records are inserted.
Note that in both retrieving and inserting records you move through
the records consecutively.
Each record is retrieved or inserted with
respect to the record accessed right before it.
Random Access
For random access, you determine the location of the record you want
on the basis of some identifier,
rather than on the basis of the
record's position within the file.
If, for example, you have a list
of locations of records in the file, you can reach into the file to a
record's exact location. Each record selection is independent of the
previously accessed record and of the next record to be accessed.
The record identifier can be a number, as for relative files,
or
it
can be a key, as for
indexed files.
Or, the identifier can be a
physical location within the file drawer:
for example, you could
place each record
in a numbered slot within the file drawer and use
the slot number to access the records in the file.
The slot number
would be the address of the record.
This type of random access could
be used with any type of file organization.
Often, you will want to switch the mode of access you use.
You may
want to use random access to find the first record in a series and
then use sequential access to retrieve all the records in that series.
For example, if your employee records are grouped by department codes
within the file, you can use a specific department code as the
identifier to randomly access the first record with that department
code and then switch to sequential access to consecutively read all
the records with that code.
Context
In either type of access, sequential or random, the marking of
position in the file is important.
This is called context:
the
position of the record you are accessing is the current record, and
the position of the record that follows it is the next record.
Access Control
One advantage of the segregation of data provided by files
is
controlled access.
Some files, such as budget or payroll, should be
available to only a small group of authorized people.
Other files,
such as inventory or transaction files, may be used by larger groups
of people. And some files, such as the telephone directory, must be
accessible to everyone.
Files allow you to control who can use what data. You can lock the
filing cabinet that contains the payroll data and give keys to
yourself and the payroll manager only.
And you can distribute
telephone directories to every employee.
In addition, within a file, you can further control how the data can
be used within the group of authorized users.
Some users may be
allowed to write new data in the file or to modify existing data,
while others may be allowed only to read the data.

1-9

RMS-11 CONCEPTS AND PROCESSING ENVIRONMENT
1.1.4

Processing

Once you locate, either sequentially or randomly, a record's position
within a file, you will probably want to do something with the record
that belongs there.
Record operations fall generally into the
following categories:
•

Verify that the record exists in the right location

•

Read the record;

•

Insert a record in the position that you have located

•

Revise the contents of the record;
its data contents

•

Remove the record from the file

1.1.5

that is, examine its data contents

that is,

modify

some

File Maintenance

Once you establish files and their records and begin using them
regularly, you will want to be able to maintain them to ensure both
the protection of the data within them and their continued usability.
Typically, maintenance might include the following activities.
•

The data in a file is valuable or you would not keep it.
You
should have duplicates of your records in some other place in
case something happens to the originals. Therefore, you need
the ability to back up files.

•

If something does happen to your original data, you
able to obtain, or restore, the duplicate records.

•

You need the ability to list, or display,
their names and other attributes.

•

Files often grow very large and their usage can change over
time.
Therefore, you may want to change a file's organization
from sequential to indexed; or you may want to reload a file
that has grown very large to use space more efficiently.
Conversely, usage and file size might decrease and you may
want to make a file simpler.
It is also possible that the
information in one file is suitable for another application.
In all these cases, you would want to be able to convert a
file into a new one, perhaps changing some
attributes
(including organization) to make it more usable.

•

You want to be able
require.

•

Creating an indexed file and putting records into it can be
complicated and time-consuming.
You would want a procedure
indexed file loading -- that would produce an optimal
indexed
file quickly and efficiently.

design

1-10

and

create

your

must

files,

files

that

be
with

you

RMS-11 CONCEPTS AND PROCESSING ENVIRONMENT
1.2

RMS-11 IMPLEMENTATION OF DATA ORGANIZATION AND ACCESS

RMS-11 provides file structure capabilities that allow you to organize
your data within a computer's memory using the same concepts that were
described in Section 1.1 for paper records in filing cabinets.
The following sections briefly present
capabilities.
For more details, see
Introduction.

1.2.1

the RMS-ll file structure
RSX-IIM/M-PLUS RMS-ll:
An

RMS-1l Record Formats

RMS-ll supports the following record formats that allow you to
the size of your data records:

define

•

Fixed length -- Every record in the file is the same size.

•

Variable length
Records in the file are of different
lengths, up to a maximum size that you can optionally specify.

with fixed control -- Records in the file are of
• variable
different
lengths, up to a maximum size that you can
optionally specify, and in addition,
area precedes the data.

fixed-length

control

a continuous stream of ASCII
special terminator character or

•

Stream -- Records consist of
characters delimited by a
sequence of characters.

•

Undefined -- Records in a file may have no record format or
may be in a format different from the four standard RMS-ll
formats.

RMS-ll's support of stream and undefined
limited support for non-RMS-ll files.

1.2.2

record

formats

provides

RMS-11 File Organizations

RMS-ll supports three file organizations:
•

Sequential -- Records are arranged within the
order in which they were written into the file.

file

the

-- Records are stored in the file in cells, or
• Relative
fixed-length units of storage, one record per cell. The cells
are numbered sequentially.
These numbers, called
record numbers, are identifiers for the records.
•

relative

Indexed -- Records are arranged in the file in ascending order
by key.
A key is a data field within the record that RMS-ll
uses as an identifier to access the record. An indexed file
must have one primary key and may optionally have other
alternate keys.

1-11

RMS-ll CONCEPTS AND PROCESSING ENVIRONMENT

1.,2.3

RMS-ll Record Access Modes

RMS-ll provides three record access modes for storing
records in files:
and

retrieving

retrieves

records

•

Sequential
RMS-ll
stores
sequentially, one after another.

•

Random by key -- RMS-ll uses either a key
(for an
indexed
file)
or a relative record number (for a relative file or for
a disk sequential file with fixed-length format records) as an
identifier to gain direct access to an individual record in
the file.

•

Random by record file address (RFA) -- RMS-ll uses the RFA as
an identifier to gain direct access to an individual record in
the file.
The RFA is a unique identifier that RMS-ll
establishes for every record that it writes into a disk file.

1.2.4

RMS-ll utilities

RMS-ll provides utility programs that can help you
record maintenance:

1.3

and

perform

file

and

•

RMSBCK -- The RMS-ll File Back-Up utility transfers
the
contents of an RMS-ll file to another file, which may be on
another device, to maintain the file should the original
file
be lost or damaged.

•

RMSRST -- The RMS-ll File Restoration utility transfers
files
that were backed up using RMSBCK back to you so your programs
can access them.

•

RMSDSP -- The RMS-ll File Display utility produces a concise
description of any RMS-ll file, including back-up files.

•

RMSCNV -- The RMS-ll File Conversion utility reads records
from an RMS-ll file of any organization and loads them into
another RMS-ll file of any organization.

•

RMSDES -- The RMS-ll File Design Utility allows you to
and create sequential, relative, and indexed files.

•

RMSIFL -- The RMS-ll Indexed File Load utility reads records
from an RMS-ll file of any organization and loads them into an
indexed file.

design

RMS-ll PROCESSING ENVIRONMENT

The RMS-ll software routines organize data on
your
computer,
implementing the concepts discussed
in the previous sections, and
provide the
interface between your application programs and
the
computer system.
Your computer system consists of layers of hardware and software:
•

The hardware devices -- magnetic tapes and disks -the data.

1-12

store

RMS-ll CONCEPTS AND PROCESSING ENVIRONMENT
•

The operating system software
file control processor,
device drivers -- controls the hardware to maintain files.

•

RMS-l1 software controls the internal structure of
described in Section 1.2).

files

(as

•

Your application program makes use of these hardware
software facilities to process data records and files.

and

1.3.1

RMS-ll Task Structure

You use the RMS-ll software routines by combining them with a
you have written in a language that implements RMS-ll.

program

NOTE
Only MACRO-II programmers can use the
full
set
of
RMS-ll
capabilities.
Subsets of
these
capabilities
are
available
to
high-level
language
programmers.
See
your
high-level
language documentation to determine:
•

Which RMS-ll facilities you
in your high-level language

•

The syntax for using these facilities

Once you write your program, you convert
either a compiler or an assembler.

can

use

object

code,

using

To combine your object code with the RMS-ll routines, you use the task
builder, which converts object code (modules) to an executable form
called a task.
In the process, the task builder not only combines
different object modules, but may also arrange the task so that some
executable modules overlay each other when the task is run.
You can combine RMS-ll routines with your object code in either of the
following ways:
routines

•

In the task itself, with
nonoverlaid
disk-resident overlay structure

•

In memory-resident overlays, a form apart from your task

between
these
techniques
is
that
The
primary
difference
memory-resident overlays can be shared among programs. Nonoverlaid
and disk-resident overlaid routines cannot be shared; each accessing
program must have its own copy of such routines.
In addition,
memory-resident overlays eliminate the I/O operations needed to bring
thereby making your tasks run
disk-resident overlays from disk,
significantly faster.
In either case, your task takes a logical form in which program code
exists in one part of the task and the RMS-ll routines run ln another
part. When your program performs an RMS-ll operation, it sets up the
necessary parameters and data and calls the appropriate RMS-ll
routine. Control jumps to that part of the task, the routine ~uns to
completion,
and
control returns to your program.
Figure 1-8
illustrates this logical structure.

1-13

RMS-ll CONCEPTS AND PROCESSING ENVIRONMENT

rc:;-------- - - - - - - - ,
I SIZE DEPENDS ON:

I- NUMBER OF FILES OPENED SIMULTANEOUSLY I

I- BUCKET SIZES
I
L NUMBE~~RECOrRD~CCE~ ~~AMS _ _

USER BUFFERS

(

I/O
BUFFERS
VIRTUAL
MEMORY

PROGRAM

RMS-11

INTERNAL
CONTROL
STRUCTURES

fSlZEDEPENDS

00:- ~- ~

~ - RMS-11 FUNCTIONS USED ~
~ ~V~LA~~U~TURE~~~
ZK-1174-82

Figure 1-8: RMS-ll Task Structure
Also part of the task are storage
three forms:

structures,

which

generally

take

•

User buffers -- These buffers are used to pass data records
between your program and RMS-ll_
They are available to your
program and the data in them can be manipulated, read,
changed, used for calculations, and so on.

•

I/O buffers -- For each fi le your ,program has open, RMS-ll
normally requires at least one internal I/O buffer. All data
going to or coming from disk is stored in--an I/O buffer as
follows:
RMS-ll requests the file control processor to move block(s)
from a disk file into this buffer to satisfy your program's
requirements. Each request normally specifies the same
number of blocks, called an I/O unit.
The size of the I/O
unit depends on the file organization,
file design,
and
settings at access time (such as multiblock count).
RMS-ll moves records between the I/O buffer and the user
buffer.
Your program can also directly access a record
within the I/O buffer in certain restricted circumstances.

•

Control structures
RMS-ll control structures,
called
control blocks, are used to communicate between your program
and the RMS-ll routines and with each other.
Some are
accessible to your program;
others are for RMS-ll internal
use only.

1-14

RMS-11 CONCEPTS AND PROCESSING ENVIRONMENT
1.3.2

RMS-11 Record Processing

The RMS-II stream and record operations are the interface between your
program and the data records your program requires.
Before your program can access records in a file,
open and an access stream must be initiated.

the

file

must

NOTE
Most high-level languages do not support
access streams at the user level. .They
use the RMS-11 access stream facilities
to
implement their own file access
techniques.
An access stream is a path to the file's data records~
record
operations are performed via that stream, one operation at a time.
RMS-ll keeps track of the stream's position, or context, in a file, in
terms of current record and next record.
The stream's position
changes at the completion of an operation.
Chapters 3, 4, and 7
discuss
context for record operations with the different file
organizations.
The stream operations control the stream associated with a file.
are:

They

•

CONNECT -- initiates an access stream.

•

DISCONNECT -- terminates a stream.

•

FLUSH -- writes the currrent contents of I/O
file.

•

FREE -- releases control of the record or block most
accessed (and locked) by the stream.

•

REWIND -- resets the stream context to the first record in the
file.

•

WAIT -- suspends processing until an outstanding
operation is completed.

The record operations process records within a file.

buffers

the

recently

asynchronous

They are:

•

FIND -- reads a record from a file to an I/O buffer
the current-record context to that record.

•

GET -- reads a record from a file to an I/O buffer and then to
a user buffer, and sets the current-record context to that
record.

•

PUT -- writes a new record from a user buffer to an I/O buffer
and then to a file.

•

UPDATE -- transfers a modified record from a user buffer to an
I/O buffer and then to a file, overwriting the previous copy
of the record in the file.

•

DELETE -- removes
i nd e x ed f i 1 e •

•

TRUNCATE -- effectively deletes all records in a sequential
file from the current record to the logical end-of-file.

existing

1-15

record

from

and

relative

sets

RMS-ll CONCEPTS AND PROCESSING ENVIRONMENT
For the FIND, GET, and PUT operations, your program specifies the
record access mode -- sequential, random by RFA (FIND and GET only) ,
or random by key -- which determines which record is the target of the
operation.
See RSX-llM/M-PLUS RMS-ll: An Introduction, Chapter 4,
for a more
IntroductIon to record processing. Chapters 3, 4, and 7 of
this user's guide describe specifically how the record operations work
depending on the file organization selected and (for FIND, GET, and
PUT) the access mode specified.
dE~taIled

1.3.3

RMS-ll File Processing

RMS-ll must manipulate the contents of files so that it can process
records.
However,
RMS-ll does not directly perform the actual file
manipulation, and the flow of data, control, and overlay segments that
the file manipulation entails.
RMS-11 issues requests to the file
control processor to perform the actual I/O and other operations on
the files.
Thus,
the file control processor's internal operation,
while invisible to RMS-ll, can affect your program's performance.
The file control processor is not concerned with the data records in a
file.
It knows only virtual and logical block numbers, directories
and other information, a~ the disk drivers
involved.
Therefore,
RMS-ll can direct file manipulation as long as it makes the proper
requests to the file control processor.
To do so,
RMS-ll maintains
the following structures, or I/O units:
•

Blocks -- The I/O unit for sequential files is the block~
You
can adjust the block count for each record access stream so
that more than one block can be moved during each I/O
operation.
In addition, you must decide whether records can cross block
boundaries.
When records can cross block boundaries, RMS-ll
can pack them with optimal density in the file because a
record can be stored in one or more blocks. This is called
block spanning. Figure 1-9 illustrates block spanning.
When records are restricted by block boundaries, each record
must be no more than 512 bytes (one block) long, and unused
bytes may be left at the end of each block.

•

Buckets -- The I/O unit for relative and indexed files is the
bucket.
A bucket consists of one or more blocks that RMS-ll
treats as a unit.
Records can cross block boundaries but they
cannot cross bucket boundaries.
Bucket size is a file
attribute that you specify when you create the file.
Buckets are an RMS-ll concept, so when RMS-ll
initiates an
operation for a relative or indexed file, it requests the file
control processor to move a bucket by specifying the virtual
block number for the first block in the bucket and the size of
the bucket in bytes.
Note that buckets are fixed within the
file;
once created, buckets contain the same virtual blocks
at all times.

1-16

RMS-II CONCEPTS AND PROCESSING ENVIRONMENT

I
I
I

I
I

+
~ RECORDS

A. RECORDS LESS
THAN 512
BYTES

II
I

I
I
I

I
I

B. RECORDS GREATER
THAN 512
BYTES

ZK-1173-82

Figure 1-9: Records Spanning Blocks
You can also direct RMS-II to request the file control processor to
place a file on a disk at a specific location. This is called
placernent control and can improve performance by taking advantage of,
for example, tracks and cylinders.
RMS-ll provides access sharing;
that is, your program can control who
can gain concurrent access to the data in a file and what type of
operations they can perform on the data. See Section 2.2.3 for more
information on access sharing.
The RMS-ll directory and file operations perform the file-level
functions.
The directory operations affect file specification entries
in directories (not the contents of the files).
They are:
•

ENTER -- places a disk file specification in a directory.

•

REMOVE

deletes a disk file specification from a directory.

RENAME
new one.

replaces an existing disk file specification with

•

PARSE
program.

returns

•

SEARCH -- examines one or more directories for a specified
file and returns the file specification and location.

file

specification

1-17

information

your

RMS-ll CONCEPTS AND PROCESSING ENVIRONMENT
NOTE
Most high-level languages do not support
the
directory operations.
See your
high-level language documentation.
The file operations provide access to files.

They are:

•

CREATE -- creates a new file with the attributes
and opens it for processing.

you

specify

•

OPEN -- makes an existing file available for processing.

•

ERASE
deletes a file and removes its
one is specified.

terminates access to a file.
directory

entry,

• EXTEND -- increases the allocated size of an open file.
-- returns file information about an open file to your
• DISPLAY
program.
See RSX-IIM/M-PLUS RMS-ll:
An Introduction, Chapter 4,
for a more
detailed introduction to fIle processing. Chapters 3, 4, and 7 of
this user's guide describe specifically how the file operations work
depending on the file organization selected.

1.4

FILE ATTRIBUTES

When you create an RMS-ll file, either through a program
(using the
CREATE file operation routine)
or by using the RMSDES utility, you
must specify the following information:
•

Medium -- Disk or magnetic tape.
You can also create files on
unit-record devices, such as
line printers and terminals.
Note that relative and indexed files are restricted to disk
devices.

•

File specification -- The name you assign to a file enables
RMS-ll to find the file later.
Use the file specification
conventions specific to your operating system.

•

Protection -- RMS-ll allows you to assign a protection code to
a file when you create it.
Use the protection codes specific
to your operating system.

•

File organization -- Sequential, relative, or indexed.

•

Record format -- Fixed length, variable length,
or undefined.

•

Record size
For fixed-length records, the size is the same
for every record
in the file.
For variable-length records,
the size is the maximum length any record can be.

VFC,

stream,

For VFC records, there are two size specifications:
(1)
the
fixed
length of the control area, and (2) the maximum length
of the variable data area.
RMS-ll also keeps the length of the longest record actually
stored in a sequential file for variable-length and VFC
records.

1-18

RMS-ll CONCEPTS AND PROCESSING ENVIRONMENT
•

Block spanning (sequential files)
block boundaries.

-- Whether records can cross

•

Bucket size (relative and indexed
blocks in each bucket.

files)

•

Maximum record number (relative files) -- The
of records that the file will contain.

•

Keys (indexed files) -- ~he number of keys;
the position and
size of each key;
the data type for each key;
and other key
characteristics.

The

number

maximum

number

Record-output handling
• exclusive)
types of
directly to a
specify any:

You can specify three
(mutually
handling for
records being written
unit-record device, although you need not

Carriage control -- The device driver
inserts a
linefeed
character as a prefix to each record and a carriage-return
character as a suffix to each record before passing it to
the device.
FORTRAN -- The device interprets the first byte
record as a FORTRAN forms control character.

each

Print file format (VFC records with a fixed header size of
2 bytes) -- RMS-ll interprets the first byte of the
header as a prefix for the record and the second byte as a
suffix for the record.

o or

•

File allocation -- You must specify two quantities:
Initial allocation -- the size of the file in
it is created.

blocks

Default extension quantity -- the number of blocks to
added to a file when RMS-ll automatically extends it.

when
be

Whether the disk space initially allocated to
• Contiguity
is to be allocated in continuous, adjacent logical
the file
blocks.
•

Placement control -located on the disk.

Where

the

file

physically

During the file creation process,
RMS-ll stores this
information,
called the file attributes, in the file directory and, for relative
and indexed files, in the first blocks, or prologue, of the file as
well.
After creation, for the life of the file,
RMS-ll gets infor~ation
about a file from the file itself.
This offers several advantages:
•

Most file attributes do not change.

•

You can design your RMS-ll files offline.
No
program
accessing the files need specify attributes (except those that
may be required by high-level languages), because RMS-ll
requires only a file specification from a program to open a
file.

•

You can open an RMS-ll file with its file specification only.
After that,
RMS-ll enables you to read the file attributes.
You can write a program or use the RMSDSP utility to display
those attributes.
1-19

RMS-ll CONCEPTS AND PROCESSING ENVIRONMENT
Note that some of the attributes are interdependent;
that
is,
the
selection of one attribute directly affects,
or restricts, other
attribute options.
File organization, record format, and medium are
all
interdependent.
For example, if you select magnetic tape medium,
you must use sequential file organization.
And
if you select VFC
records, you cannot use indexed file organization and you must use a
disk device.
Table 1-1
lists
interdependencies.

the

record

format

and

organization

file

Table 1-1: Record Formats and File Organizations
File
Organization

Fixed

Record Format:
Variable VFC Stream

Undefined

Sequential:
Magtape
Disk

Yes
Yes

No
Yes

Relative

Yes

Indexed

Yes

Chapters 3 through 7 discuss your file design options in detail,
depending on your selection of file organization. Chapter 2 provides
information to help you make that selection.

1.5

PROCESSING BY BLOC~ ACCESS

Your program can bypass RMS-ll record processing
RMS-ll file in a mode called block access.

and

process

any

Your program can read or write blocks in a file either sequentially or
(on disk only)
randomly by virtual block number (VBN).
But your
program must be able to interpret the contents of those blocks.
See RSX-IIM/M-PLUS RMS-ll:
An Introduction for
an introduction to
block access and processing.
See the RSX-IIM/M-PLUS RMS-ll Macro
Programmer's Guide for detailed
information on block access----and"
processing. -----

1-20

CHAPTER 2
APPLICATION DESIGN

When you write an application program, you want that program to input
data, process it, store it, update it if necessary, and at intervals
output it in the proper formats.
You want all this to happen simply, quickly, and accurately. You must
therefore take the time to design your application by carefully
considering RMS-ll file structure and file and record processlng
capabilities.
Important RMS-ll considerations are data storage
medium, record format, file organization, access mode, allocation,
overlays, and so on.
If you do not consider RMS-ll capabilities when you design your
application, you may not get the best peformance possible from your
application because of the defaults that will be applied automatically
to your files (see Section 2.1).
Example: The first time one user created a file, she used a
high-level language program and took all the defaults. Then she
loaded records into the file;
the process was quite lengthy.
However, when she re-examined the file and re-created it applying some
RMS-ll design considerations, the record insertion process went 10
times as fast.
Example: Some users, accustomed to programming with BASIC-PLUS record
I/O, learned that RMS-ll uses 15 bytes of control data in each bucket
and 7 bytes of control data for each fixed-length record in an indexed
file
(see Chapter 6). Then, because they were accustomed to working
with whole blocks, they set up single-block buckets
(512 bytes)
and
subtr:acted RMS-ll overhead (22 bytes) to come up with a record size of
490 bytes.
But when they used those files, the users were alarmed to see them
grow at high rates. They had not read that RMS-ll preserves its fast
sequential and alternate key access during random insertions by moving
records
and
leaving
behind 7-byte pointers
(see Chapter 5).
Therefore, when one of those 490-byte records was moved,
it left
behind 7 bytes, which meant that no other record fit into that bucket.
Soon the file was filled with practically empty buckets that could not
be used because the designers did not allow for the full implications
of RMS-ll structure.
If you develop an application with a high-level language, you probably
will not worry about RMS-ll. You will accept the language's concept
of design, if any.
It is possible, however, that the defaults the
language uses in its interface with RMS-ll are not well suited for
your application.

2-1

APPLICATION DESIGN
This chapter presents general design considerations that apply to all
application designs and information that will help you make the first
important design decision:
selection of a file organization.

2.1

WHEN TO DESIGN

There are two times to design an application:
1.

Before you write the application, especially if you have:
•

Large file(s)

•

Many users simultaneously accessing the file(s)

•

A high level of activity
(many records read,
updated, or deleted in a given time period)

After you write the application, if you are
its performance.

not

written,

happy

with

Often, poor performance results from default values
that
are
inappropriate
for
your
application.
You can frequently find
improvements by studying the nature and source of the defaults and how
they affect the structure of your application and your file.
Basically, defaults have three sources:

Source language compilers
In many instances, source language compilers such as COBOL-Sl
or
BASIC-PLUS-2 supply default values for
RMS-ll file
attributes and/or facilities.
Example:
RMS-ll does not calculate an optimal bucket size
for
indexed files.
Rather,
the program creating the file
must specify a bucket size. When that program is the product
of a compiler, the bucket size can be explicitly specified in
the source code or it can be implicitly set by the compiler,
using a default value.

RMS-ll
The interface between the RMS-ll routines and your program
has the same structure in all tasks, regardless of their
source (PDP-II COBOL-Sl, RPG, MACRO-II,
and so on).
This
interface consists of control blocks (see the RSX-llM/M-PLUS
RMS-ll Macro
Programmer's
Guide
for
details).
The
informat~ provided
by your--program in these blocks
effectively controls RMS-ll,
causing it to create, open,
access,
and close files.
However, when explicit information
is not provided, RMS-ll uses its default values.

Operating system
RMS-ll acts as an intermediary between your task and the
operating
system.
As such,
RMS-ll can supply control
information for system functions such as protection codes.
However,
if RMS-ll supplies no control data, the system uses
its defaults.

2-2

APPLICATION DESIGN
2.2

DESIGN CONSIDERATIONS

When you design your application, you
four design considerations:

are

concerned

with

which

the

and

the

Speed -- You want to maximize
programs process data.

-- .
.)

Space -- You want to minimize the room for the data
task on disk and the memory the task takes to run.

:3 •

Shared access
accessible to
necessary.

You want your
people using

Ease of design
You do not want to
necessary writing the application.

speed

with

the

primarily

data to be exactly
the computer system
spend

time

as
as
than

Remember, the importance of design is proportional to the complexity
of the file organization.
That is, design is least important for
applications using sequential
files
and
most
important
for
applications using indexed files.

2.2.1

Speed

You can make many performance (speed) decisions before you have to
consider anything else.
Therefore, the first criterion to apply
throu9hout the design process is minimize I/O time.
The mechanics of the mass storage devices on your system consume most
of the time for any RMS-ll operation. The memory-resident routines
that prepare the data for I/O or process it afterwards are very much
faster (one to three orders of magnitude) •
An application's entire environment affects I/O time:
•

File structure -- A variety
time, including:

file

attributes

affect

I/O

bucket size (for a relative or indexed file)
number of keys (for an indexed file)
number of duplicate key values (for an indexed file)
initial file allocation
default extension quantity
•

File size -- The number of records in the file affects the I/O
operations required to scan a file sequentially or follow an
index.

•

Program -- Your program affects I/O time by requiring I/O
operations for file operations
(OPEN, CLOSE, and so on),
record operations (GET, PUT, and so on), and overlays.

•

RMS-ll
The RMS-ll routines
can
be
structured
disk-resident overlays or as memory-resident overlays.

•

File control processor
Besides requiring overlay segments
from disk, the file control processor can also request I/O
operations required to map virtual blocks of the file to
logical blocks on the storage device.

2-3

APPLICATION DESIGN
•

Device hardware -- The storage device that contains the task
and data files is the primary contributor to the length of an
I/O operation.
The type of device chosen
(moving-head,
fixed-head,
and so on) and the demands on it (amount of I/O
activity for that device within the system) are crucial to I/O
performance.

Figure 2-1 illustrates this environment.

TIg@
TIME
DATA
DEVICE

ZK-1163-82

Figure 2-1: Time Factors in an I/O Operation

2.2.2

Space

RMS-ll requires space for three reasons:
1.

To store data in a file

To store the RMS-ll routines either (a) on disk when they are
not in use, or (b) in memory when they are being executed

To buffer data in memory while the task runs

2-4

APPLICATION DESIGN
2.2.2.1 Data Storage - The space RMS-11 requires to store data is
proportional to the organization of the file, and to the processing
capabilities of that organization:
Sequential file organization -- RMS-11 adds to the size of
• your
data an empty byte,
if necessary, to align each
fixed-length,
variable-length,
or
VFC
record
on
an
even-numbered
byte
boundary.
When
the file contains
variable-length records, RMS-11 also prefixes a count field to
each record.
file organization -- RMS-11 constructs a series of
• Relative
record storage cells based on the length of the records. The
cells are 1 byte longer than the fixed size of fixed-length
records or 3 bytes longer than the maximum size specified for
variable-length records.
•

RMS-ll adds to your data:

Indexed file organization

An index for each defined key.
15 bytes of formatting information for each bucket.
A 7-byte header for each record.
A count field for each variable-length record.
Other overhead of varying lengths for records RMS-11
during file activity and for deleted records.
You should keep the size of records to the minimum required
application.

for

moves
your

2.2.2.2 Task Size - The space RMS-11 routines occupy in a task
depends on the method you use to link the routines with your program.
See Section 8.1 for more details.

2.2.2.3 Buffer Sizes - You can vary the size of the I/O buffers
RMS-ll uses to store data in memory.
Generally, the larger the
buffers, the faster the task processes data.
See Section 3.5.3,
Section 4.5.3, or Section 7.4 for the file organization(s) you are
interested in.

2.2.3

Shared Access

SharE~d

access revolves around the question: Who is allowed to read
from or write to a file? The answer involves your operating system's
protection codes, your access
declaration,
and
your
sharing
declaration.
System Protection Codes: Before you can access an RMS-ll file, you
must log into your computer system using an account number that will
al101fl you the kinds of access you need when your access request is
validated against the file's protection codes.

2-5

APPLICATION DESIGN
Operating systems allow you to assign a protection code to each file
when it is created. This code describes concentric circles af users
who are allowed different levels of access to that file.
See your
operating system documentation for specific protection conventions.
Figure 2-2 illustrates the system protection concepts.

READ ACCESS

WRITE ACCESS

EXTEND ACCESS

DELETE ACCESS

RSX-11 M/M-PLUS
ZK-1166-82

Figure 2-2: System Protection Concepts

Access Declarations: Your program must declare the types of access
you need by specifying the record or block operations it intends to
perform on the file, as follows:
•

Read-only access is granted if your program specifies
only FIND/GET or READ operations can be performed.

that

No PUT, UPDATE, DELETE, TRUNCATE, or WRITE operations will be
allowed, nor will any other operation which would modify the
file (an EXTEND operation, for example, will not be allowed
for read-only access).
•

Read/write access is granted if your program specifies that
PUT, UPDATE, DELETE, TRUNCATE, or WRITE operations can be
performed. FIND/GET and READ operations will also be allowed,
as will EXTEND operations.
Note that, in addition to any access declaration, a CREATE
operation always forces
read/write access so that the newly
created file can be populated (using PUT operations for recora
access or WRITE operations for block access).

2-6

APPLICATION DESIGN
Sharing Declarations: Your sharing declaration specifies the types of
access to the file that your program is willing to allow to other
programs that request access to the file at the same time that your
program is accessing it. These declarations can be:
•

No sharing -- You do not want any other program to access
file.

the

A no-sharing specification in your
sharing
declaration
overrides any other sharing specification you may also have
included, and guarantees that no other program will have
concurrent read/write access to the file.
That is, no other
program will be able to modify the file.
Note, however, that it is not possible< to guarantee
concurrent read-only accessors will be denied access.
other

that

•

Read-only sharing -- You are willing to allow
read-only access to the file.

programs

•

Read/write sharing -- You are willing to allow other programs
read/write, as well as read-only, access to the file.

•

Sharing with user-provided interlocks (sequential files only)
This specifies a special form of sharing among a group of
programs that includes any number of read-only accessors and
at most one read/write accessor.
User-provided interlocks offer a limited form of access
sharing of sequential files.
If the file organization is
sequential, this specification in your sharing declaration
overrides any other sharing specification (except no sharing) •
For any other file organization, this specification
is
ignored.
NOTE
High-level languages may use slightly
different terms to designate the access
and sharing declarations, and may not
provide equivalents for all the sharing
options. See your high-level language
documentation.

Once the operating system's protection checks are passed, RMS-11 and
the operating system cooperate to determine whether the type of access
you request (your access declaration) and the type of sharing you
permit
(your sharing declaration)
are consistent with any other
current accessors of the file.
If no other tasks have accessed the file at the time that your program
requests access, your access request must only pass the system
protection checks to be granted. However, if one or more programs
already have access to the file, RMS-II and the operating system will
use the access and sharing declarations of those programs along with
those of your program to determine whether your program will be
allowed concurrent access.
No-sharing and read-only declarations are processed as described above
for files of all organizations and access method (block or record).
In other cases, however, RMS-11 and the operating system interpret the
access and sharing declarations in the manner best suited to the
file's organization and the access method, as described in Section 3.4
forj sequential files, Section 4.4 for relative files, and Section 7.1
for indexed files.
2-7

APPLICATION DESIGN
NOTE
As noted, file sharing is a cooperative
effort between RMS-ll and the operating
system.
The
RMS-ll
processing
algorithms
depend upon the detailed
nature of this cooperation.
If you
access a file concurrently with multiple
programs, some of which use RMS-ll and
some of which do not, the results may be
unpredictable.

2.2.3.1 Bucket Locking - Any time a record is updated, accessing
programs must be assured that the data written to the file is current
until the record is re-accessed and the record updated again.
If no control is placed on access, two or more programs could access
the same record, one after the other, and update it, one after the
other. Only the last update would remain in the file.
Access sharing
could thus impair data integrity.
To ensure data integrity, RMS-ll uses bucket locking for a relative or
indexed file when the file is open for write-shared access.
From that
point, RMS-ll requests the operating system to lock each bucket read
from disk until RMS-ll explicitly releases the bucket.
After a GET,
FIND, or mass-insert PUT operation, only the bucket containing the
data record remains locked.
(See Chapter 7 for information on mass
insertion.) While that bucket is locked, no other program can access
it.
RMS-ll requests the operating system to unlock such a bucket when
of the following occurs:

one

•

The GET, FIND, or PUT operation fails.

•

The GET or FIND operation succeeds
declared read-only access to the file.

•

The program initiates another record operation that accesses a
different bucket.

the

program

has

After the bucket is unlocked, other programs can access it.
Example:
Programs A and B are write-sharing a file named RMSREL.DAT.
Both try to update relative record number 12. However, program B
initiates the prerequisite GET operation first,
locking the bucket
containing the record.
The operating system keeps program A from
accessing that bucket while program B uses it.
After program B
updates record 12, RMS-ll unlocks the bucket and the operating system
allows program A to get record 12
(including program B's updated
data).
Figure 2-3 illustrates this example.
Bucket locking incurs costs:
The operating system administers bucket
locking.
It establishes, for each file, a list of virtual blocks that
are locked.
The system must scan this list every time RMS-ll performs
an I/O operation and then either permit the operation or return an
error.
In addition to this lock-list overhead, extra instructions are
executed to lock and unlock the buckets.

2-8

APPLICATION DESIGN

RELATIVE
RECORD
NUMBER 12

PROGRAM B
UPDATE
RECORD

PROGRAM B
PROCESS
RECORD

Figure 2-3: Bucket Locking Example

2.2.3.2 Sharing among Access Streams - In addition to the bucket
locking used when programs allow sharing, RMS-il provides its own
version of bucket locking when a program accesses a file for
write·-type operations. This locking allows multiple streams to share
the file.
RMS-II bucket locking works the same way as the locking
provided by the operating system, except that the locks can be
encountered only by different access streams within the same program.
The overhead for RMS-II bucket locking is small.

2-9

APPLICATION DESIGN
2.2.3.3 Programming Considerations - For the greatest flexibility at
run time, you should assume that access to any record by your program
can be denied because the bucket containing the record is locked.
RMS-ll returns the error code ER$RLK when the bucket is locked by
another access stream in the same or in another program.
Therefore, you should use the following techniques
RMS-ll programs that involve shared access:

when

you

write

keep a bucket locked longer than necessary.
You should
• Never
follow any successful GET or FIND operation with another
record operation of any type as soon as possible. The second
operation
the
unlocks
bucket locked by the read-type
operation.
Alternatively, you can release the bucket explicitly with a
A FREE operation releases only the bucket
FREE operation.
locked by the access stream associated with the operation.
If your program detects an ER$RLK error
(or its high-level
• language
equivalent),
its error processing depends on the
number of access streams active on the file:
Single stream
Set up a loop that
stalls,
then
re-initiates the record operation until RMS-ll indicates a
successful completion.
Multiple streams -- Do not set up a loop that continuously
re-initiates the record operation.
You ~hould either (a)
continue processing on the other streams, attempting the
record operation on the locked-out stream periodically, or
(b) release the buckets locked by all other streams, then
re-initiate
the
record
operation that failed.
Any
GET-UPDATE or FIND-UPDATE sequences interrupted on the
other streams must be restarted, because the release of a
bucket destroys the record context.

2.2.4

Ease of Design

When you design and write your application, you should
yourself and the person who will maintain the application.
following design guidelines in mind:
•

consider
Keep the

Keep things simple. You can apply this criterion to the whole
development process,
from program flowcharts to the record
layouts to file organization and design.
Example:
From sequential through indexed,
the RMS-ll file
organizations offer increasing capabilities, but they are also
increasingly complex. Choose the organization that supplies
enough capabilities, but no more. For instance, if you want
to randomly access a file by a single key only, you might use
a relative file and a hashing algorithm instead of an indexed
file.

•

Apply optimizations one by one until you reach a satisfactory
level of performance. Generally, further improvements are not
necessary.

2-10

APPLICATION DESIGN
Example: The optimization of performance of applications
using indexed files can be involved, but you do not have to
use every technique discussed in this manual. You should only
satisfy current performance requirements. For instance, when
an application program needed optimization, the indexed file
being read was made contiguous (see Chapter 6) and the RMS-ll
overlay structure was changed (see Chapter 8). Execution time
dropped from 16 minutes to 8.5. Since this performance was
adequate, no further optimizations were considered.
Some optimizations apply to one type of record operation, but
not to others. Determine whether an optimization will benefit
your processing before you implement it.

2.3

DESIGN PROCESS

The first step in the design process is the selection of the file
organization.
Section 2.4 presents information to help you make this
selection.
Once you have selected a file
chapter(s) :
Sequential
Relative
Indexed

organization,

the

appropriate

Chapter 3
Chapter 4
Chapters 5, 6, 7

Each chapter discusses file structure
(physical and conceptual)
as
well as design considerations.
Indexed files are the most complex to
design because of their power and flexibility.
Aftel: you read the file organization chapter(s), go to Chapter 8, Task
Building and Common Optimization Techniques.
Finally, apply the design considerations described in these chapters.
Write your application~
create and populate the files, using the
RMS-ll utilities when they are useful~ use the programs and files
In
a simulated environment while you evaluate performance. You may have
to return to this manual, changing your design and/or combining
attributes
and RMS-ll facilities
in different ways, until the
application runs to your satisfaction.
Good design is important to the success of your RMS-ll application.

2.4

SELECTING A FILE ORGANIZATION

Table! 2-1 lists important features of each file organization
sequential, relative, and indexed -- to help you decide which one(s)
you need. Table 2-2 points out advantages and disadvantages of each
organization.
The sections that follow the tables provide information about two of
the features of file organization -- record format and I/O techniques
-- to help you select a file organization.

2-11

APPLICATION DESIGN
Table 2-1: File Organization Characteristics and Capabilities
Characteristics
and
Capabilities

Sequential

Relative

Indexed

Medium
Disk
Magnetic Tape
unit Record

Yes
Yes
Yes

Yes
No
No

Record Formats
Fixed-length
Variable-length
VFC (disk only)
Stream (disk only)
Undefined (disk only)

Yes
Yes
Yes
Yes
Yes

Yes
Yes
Yes
No
No

Yes
Yes
No
No
No

Overhead per Record

None

1 byte

7 bytes

Access Modes
Sequential
Random
RFA access (disk only)

Yes
Yes 1
Yes

Yes
Yes
Yes

Record Operations
CONNECT
DELETE
DISCONNECT
FIND
FLUSH
FREE
GET
REWIND
TRUNCATE
UPDATE (disk only)
PUT

Yes
No
Yes
Yes
Yes
No
Yes
Yes
Yes
Yes
Yes

Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
No
Yes
Yes

1 or more
blocks

Bucket

Normal mode
of operation
Yes
No

Selectable

I/O Unit
I/O Techniques
Deferred write
Multiblock count
Multiple access
streams
Multiple buffers
Mass insertion

Bucket size Bucket size
Yes
Yes

No
No

Yes
No

Access Sharing 2

Read-only

Read/write

Other Features

Block-spanning records

Maximum record
number

Yes
Yes
Read/write
Areas

For fixed-format disk sequential files only.

2. See exceptions in Section 2.2.3, and in Sections 3.4, 4.4,
7 • 1.

2-12

and

APPLICATION DESIGN
Table 2-2: File Organization Advantages and Disadvantages
Organization

Advantages

Disadvantages

Sequential

Simplest organization.

To get a record, most
high-level languages
must access all records
before it (no access by
RFA or by key).2

Optimal use of disk and
memory:
• minimum overhead on
disk
• block spanning

You can add records only
at end of file. 2

Optimal if application
accesses all records on
each run, except if file
must be write-shared.

Interactive process is
awkward: operator must
wait as a program searches
for a record.2

Most versatile in record
formats:

Certain compiled programs
cannot access a record
already passed without
closing and re-opening
file (REWIND is not
available) •

• exchange data with
non RMS-II systems
• compatible with
RSX-IIM/M-PLUS
FCS files l
• compatible with ANSI
magnetic tape format
• compatible with
RSTS/E stream files l

You can delete records
only at end of file; use
TRUNCATE record operation.
Sharing normall~ restricted
to multiple readers.

Most versatile in storage
media; file is portable.
Random by key (RRN)
record access available
on fixed-format disk
sequential files.
Relative

Random access in all
languages.

Restricted to disk.
File contains a cell
for each cell number
between I and last
record in file; data may
not be stored densely.

Allows deletions.
Allows random GET and
PUT operations.

l.RMS-11 can read these file structures and return a record to your program.
However, differences in data storage techniques among programming languages
can keep the program from properly interpreting the contents of that record.
2. ~rhese restrictions do not exist for disk sequential files with fixed-length
record format;
records in such files can be stored and retrieved using random
by key access, depending on your high-level language capabilities.

(Continued on next page)

2-13

APPLICATION DESIGN
Table 2-2 (Cont.): File Organization Advantages and Disadvantages
Organization

Advantages

Disadvantages

Relative
(Cont.)

Optimal if application
accesses all records on
each run and file must
be write-shared.

Program must know relative record number or
RFA of record before it
can randomly access the
data; no generic access
as in indexed file organization.

Random and sequential
access with low overhead.
Can be write-shared.

Interactive access can be
awkward if you do not
access records by relative
number.
You can insert records
only into unused record
cells, but you can update
existing records.
RMS-ll does not allow
duplicate relative record
numbers.

Indexed

Most flexible random
access:

Highest overhead on
disk and in memory.

• by anyone of multiple keys or RFA
• key access by generic
or approximate value
• you access records by
record contents

Restricted to disk.
Least simple programming.

Duplicate key values
possible.
Automatic sort of records by primary and
alternate keys; available during sequential
access.
Record location is
transparent to user.
Can be write-shared.
Potential range of key
values not physically
present as in relative
file organization.
variety of data formats
for keys.

2-14

APPLICATION DESIGN
2.4.1

Record Formats

RMS-ll supports all of the record formats described in the following
sections for sequential files, but restricts relative and indexed file
organizations (see Table 2-1).

2.4.1.1 Fixed-Length Format - Records in the file are the same size,
which is a file attribute. The fixed-length record format requires no
RMS-ll overhead.
RMS-ll limits fixed block-spanning records to 32,765 bytes, while
minimum valid record is 1 byte of data.

the

2.4.1.2 Variable-Length Format - Records in the file can be any
length, up to a maximum of 32,763 bytes for block-spanning records.
This file attribute is user-settable and optional. For each record,
RMS-ll maintains a count field specifying the number of data bytes in
the record. The size of this field depends on the storage medium for
the file.
•

On disk, the count field is a 2-byte binary
not include the 2 bytes for the field.

•

On ANSI magnetic tape, the count
decimal count that does include
field.

count

that

does

field is a 4-character
the 4 characters for the

Figure 2-4 illustrates the count field for each medium.

LENGTH

I ll~\
l

DATA RECORD

',-/ ~',---..----

----- --------------------

ION MAGTAPE

.........

/
ZK-1162-82

Figure 2-4: Count Field on Disk and Tape
Choose the variable-length record format if:
•

The data truly varies in length, because the format
length field to each record's size.

adds

the

•

You are designing a new application
require records to change length.

uses

may

2-15

where

future

APPLICATION DESIGN
NOTE
Changing a record's size during
an
UPDATE operation is restricted by file
organization.
See
Sections
3.5.1,
4.5.1, and 7.2 for more information on
using the UPDATE operation with the
specific file organizations.
RMS-ll limits variable-length block-spanning records on disk to 32,763
bytes because of the count field.
RMS-ll allows records to reach this
maximum only in sequential files;
other file organizations place
further
restrictions on record size. The minimum valid record is 2
bytes of zeroes, representing a null record.

2.4.1.3 Variable-with-Fixed-Contro1 Format - A VFC record consists of
two areas:
•

A fixed-length control area from 1 to 255
length is maintained as a file attribute.

bytes

long;

the

•

A variable-length area that can vary in length from zero bytes
to the maximum record size stored as a file attribute.

For each record, RMS-ll maintains a count field specifying the number
of data bytes in the record including fixed and variable areas.
The
size of this field is a 2-byte binary count that does not include the
2 bytes for the field.
RMS-ll limits VFC block-spanning records to 32,763 bytes because of
the count field.
The minimum valid record is 3 bytes:
the length
field plus the minimum fixed area of 1 byte.
The maXImum variable
area is the difference between 32,763 and the length of the fixed
area.

2.4.1.4 Stream Format - A stream record consists of a series of
contiguous bytes.
RMS-l1 detects the end of a stream record only by
the presence of one of the following terminators:
Form feed (014 octal)
Line feed (012 octal)
vertical tab (013 octal)
RMS-ll limits stream format to disk sequential files.
In addition,
the format causes the most CPU overhead because RMS-l1 must examine
each record character by character for the terminator.
During record operations, RMS-ll processes stream records as follows:
•

For FIND and GET operations, RMS-l1 scans the stream of bytes,
removi~g
leading NULL (000) characters and searching for the
first occurrence of one of the terminators.
If it finds a
form feed,
vertical tab,
or line feed, RMS-ll includes the
terminator character with the record and considers the record
complete.

2-16

APPLICATION DESIGN
If it finds a carriage return,
RMS-ll checks the character
following the carriage return.
If the next character is a
line feed, RMS-ll discards both characters
(carriage return
and line feed) and considers the record complete. Otherwise,
RMS-ll includes the carriage-return character in the record
and resumes its search for a terminator.
During a GET operation, RMS-ll moves each character included
in the record into the user buffer as it scans the stream of
bytes.
RMS-ll does not move any data into the user buffer
during a FIND operation.
•

For PUT and UPDATE operations, RMS-ll checks the
last
character of the record in the user buffer.
If it finds a
line feed, vertical tab, or form feed, RMS-ll moves the record
as it is to the I/O buffer.
If it does not find one of these
terminators, RMS-ll moves the record to the I/O buffer and
adds a carriage-return/line-feed pair to the end of the
record.

2.4.1.5 Undefined Format - The undefined format means that RMS-ll
reads only blocks, not records.
Your program must interpret the
contents of each block.

2.4.2

I/O Techniques

RMS-ll supports the following I/O techniques so
performance of record operations:

you

can

adjust

the

•

Asynchronous
record
operations
When
operating
asynchronously, your program may regain control before the
operation is completed;
that is, the program will continue
processing while the operation is being performed.
This may
improve processing time.

•

Multiple access streams -- A stream can handle only one record
at a time, but you can connect more than one access stream to
a relative or indexed file if you want to:
Process more than one record in
asynchronous record operations.

file

time

Maintain more than one context during the processing
file.
Each stream represents an independent,
sequence of record operations.

concurrently

with
of

active

•

Deferred write -- Normally, every write-type record operation
to a relative or indexed file results in a physical I/O
operation. However, you can sometimes have RMS-ll defer this
write function until the I/O buffer is full or must be used
for another bucket. Deferred write is the normal mode of I/O
for sequential files.

•

Multiblock count -- You can open a disk sequential file so
that RMS-ll reads or writes more than one block of the file
into the I/O buffer at a time. This capability speeds file
processing, though the buffer gets bigger. For relative and
indexed files, you achieve a similar effect by increasing
bucket sizes.

2-17

APPLICATION DESIGN
•

Multiple buffers
You can allocate
I/O buffers
for
a
relative or indexed file beyond RMS-ll's minimum requirements:
one for relative;
two for
indexed.
If the file
is not
accessed
for read/write sharing, RMS-ll uses the buffers to
save in memory, or cache, buckets from the file, so that they
do not have to be read again from disk if needed.
For indexed files, RMS-ll caches the root buckets from indexes
that are used, saving one I/O operation on every random record
operation.
However,
for
relative files,
RMS-ll makes no
distinction between buckets, saving them until it has to use
the buffer.

•

Mass insertion -- Specified before the insertion of a series
of records already sorted in ascending order by primary key,
this mode enables RMS-ll to store the records tightly and
quickly in the file.
Records can be mass inserted only at the
logical end of an indexed file.
Mass insertion significantly
improves performance for single-key indexed files.
However,
with each additional key defined for the file, the percentage
improvement is smaller.

2-18

CHAPTER 3
SEQUENTIAL FILE APPLICATIONS

This

chapter

discusses sequential file structure,
design,
and
Sequential file design consists generally of determining
the specific attributes, including record size and format,
that will
allo1~ you to store, retrieve, and process your data efficiently within
the sequential file structure. Your task design, along with your file
desiqn, will determine your record and file processing options,
including record access modes.
procE~ssing.

3.1

FILE STRUCTURE

Physical Structure
Sequential files carry almost no
RMS-11
overhead.
The operating system's file management software stores
attributes in the file directory.
RMS-1l stores data
records
beginning with virtual block number (VBN) 1.
If records cross block boundaries (span blocks), RMS-1l packs
• records
into the file end-to-end, allowing for control
information and padding.
•

If you do not allow records to span blocks, RMS-1l packs
records into each block, allowing for control information and
padding.
NOTE
You will waste space in your file
both of the following are true:
allow

records

•

You do not
blocks.

span

•

Your records do not exactly fit
a block.

into

To be compatible with other file management systems, RMS-ll flags
spacE~
that is not used at the end of each block. When you allow
records to span blocks, the only unused space starts after the last
record in the file.
Table 3-1 lists the end-of-block indicators.

3-1

SEQUENTIAL FILE APPLICATIONS
Table 3-1: End-of-Block Indicators
Medium

Record Format

End-of-Block Indicator

Disk

All but stream

-1 in word following last valid byte

Disk

Stream

nulls (000) to end of file

Magtape

All

circumflex (A)

to end of block

For disk sequential files,
RMS-il uses the end-of-file attribute,
stored in the file directory, to determine where the valid data in a
file ends.
This attribute includes a VBN and a byte offset within
this block. The virtual block containing the logical end-of-file may
not be the last block allocated to the file.
RMS-II reads the end-of-file attribute with the other file attributes
when it opens a file.
RMS-II also updates the end-of-file in the file
directory when it closes the file if the end-of-file changed while the
file was open.
The end-of-file changes if records were added to the
end of the file or if the file was truncated.
conceptual Structure -- In most cases, RMS-II sto~es records in the
sequence that programs write them, one after the other from the first
record in the file to the last. For these files,
RMS-II can only
access the records sequentially or, for disk files, randomly by record
file address (RFA).
The exception to this structure is the case of disk sequential files
with fixed-length record format.
In this case, RMS-II stores records
in a series of fixed-length cells;
this is similar to relative file
organization
(see Chapter 4).
The cell size is the size of the
fixed-length record. Only one record can be put into a cell, and
RMS-II assumes that each cell contains a record.
RMS-II numbers the
cells consecutively from I to n, where n indicates the last cell
in
the file.
A cell number indicates the location of the cell relative
to the beginning of the file, and is associated with the record as a
relative record number (RRN).
RMS-II can access records in a fixed-format 6isk sequential
sequentially, randomly by RFA, or randomly by key (RRN).

file

NOTE
RMS-II does not initialize the cells in
a fixed-format disk sequential file, nor
does it "know" whether a cell contains a
valid record. Your applicatioh program
must maintain this information.

3.2

RECORD SIZE

Records in disk sequential files are word aligned, which means that
RMS-II adds a pad byte to the end of any record with an odd number of
bytes.
RMS-II uses
this
convention
to
maintain
structural
compatibility with FCS-II sequential files.
You can define a sequential file so that RMS-II writes records across
the boundaries between blocks.
Such a sequential file is optimally
dense;
all bytes within its allocated space are used, except at the
end of the file where no data has been written.
3-2

SEQUENTIAL FILE APPLICATIONS
Table 3-2 shows the maximum data size for records in a sequential
file.
They are adjusted for RMS-ll restrictions and overhead.
Table 3-2: Sequential File Data Sizes (in bytes)
Maximum Size
F ormat

With Block
Spanning

Data Size Calculation

without Block
Spanning

Fixed

32,766

512

Your data + MOD(DS/2)1

VaIiable

32,765

510

Your data + 2 + MOD(DS/2)1

VFC

32,765

509

Fixed + variable + 2 + MOD(DS/2)1

Stream

None

511 2

Data + terminator(s)

1.MOD(DS/2) is the remainder after the size of
bytes is divided by 2:

your

data

(OS)

•

MOD(DS/2)

o if the data size is an even number of bytes.

•

MOD (DS/2)

1 if the data size is an odd number of bytes.

For VFC, DS

= fixed + variable

2. Assuming a I-byte terminator character;
however,
if
the
terminator is CR-LF, then the maximum length without block-spanning
records is 510 bytes. Note that these figures do not include the
terminator characters.

3.3

FILE DESIGN

For sequential files, the primary design considerations are:
•

Record format (see Section 2.4.1
RMS-ll record formats)

•

Data storage medium

•

File allocation

•

Contiguity

3.3.1

for

description

the

Data Storage Medium

Sequential files can be accessed on both disk and magnetic tape. When
you select the medium for your file, you should consider the
following:
•

Speed of access -- How long can each record
Tape is significantly slower than disk.

•

Frequency of use -- How often do you use the file? If you use
it once a month, a quarter, and so on, you could store the
file on tape and save your disk for more immediate purposes.

3-3

operation

take?

SEQUENTIAL FILE APPLICATIONS
Transportability -- Do you need to use the file on different
operating systems?
RSTS/E disk structure is not compatible
with lAS, RSX-IIM/M-PLUS, or VAX/VMS, and vice-versa.
If you
need to use the file across these systems, you should consider
using a magnetic tape file.

•

3.3.2

File Allocation

Disk file allocation involves two quantities:
•

Initial allocation quantity -- the number of
to a file when you create it.

blocks

assigned

•

Default extension quantity -- the number of blocks added to
file each time RMS-ll automatically extends it.

3.3.2.1 Initial Allocation - Even with sequential files, where a file
extension requires only, an allocation of blocks by the operating
system, total allocation of the file when you create it is much more
efficient.
You calculate the allocation
records as follows:
ALQ

(ALQ) ,

blocks,

for

block-spanning

(NRF*RSZ)/512

where:
ALQ

is the allocation quantity in blocks

NRF

is the largest number of records that will be in the file at
time

RSZ

is the size of the record in bytes

For variable-length or VFC records, use the average
RSZ, including 2 bytes for the count field.

record

one

size

for

word

For fixed-length records, use the actual record size for RSZ.
Be sure to round RSZ up to
alignment.

mUltiple

account

This allocation can be done by RMSDES or by your application
depending on the capabilities of your high-level language.

program,

3.3.2.2 Default Extension Quantity - If the file cannot be totally
allocated at creation time, you should establish a reasonable default
extension quantity (DEQ) to minimize the number of (and the time spent
on)
file extensions.
Even if the file is totally allocated when you
create it, you should establish a reasonable DEQ in case the file gets
bigger than planned.
The time required for each file extension is
significant, involving:
•

A call to the file control processor

•

possible I/O operations
routines into memory

3-4

bring

file

control

processor

SEQUE~TIAL

PILE APPLICATIONS

•

I/O operations to read and change file directory information

•

I/O operations to read and change the disk free-block bit map

A good basis for calculation is the number of records added to the
file in a given period of time, such as a day; use the formula for
allocation quantity in Section 3.3.2.1.
The DEQ can be set by RMSDES or by your application program, depending
on the capabilities of your high-level language.
If you do not
specify a DEQ, it will default to zero whether you create the file
with RMSDES or a high-level language. This means that RMS-ll will
extend the file according to the operating system default for file
extensions.
Example: You are inserting 1000 50-byte fixed-length records into a
sequential file.
Records do not span blocks;
therefore, each block
contains 10 records. The file is currently full;
that is, no more
records can be added without an extension.

3.3.3

If DEQ is zero, RMS-ll extends the file according to operating
system defaults, which are typically only a few blocks.
Therefore, in this example, if the system default is 5 blocks,
RMS-ll extends the file 20 times.

•

If DEQ is 1, RMS-ll extends the file for every tenth
operation after the first, for a total of 100 extensions.

•

If DEQ is 25, RMS-ll extends the file 4 times.

•

If DEQ is 100 or more, RMS-ll extends the file only once.

PUT

Contiguity

Contiguity can significantly affect performance.
Therefore, you
should consider contiguity for a disk sequential file to minimize the
time spent on each I/O operation.
If thE~ blocks in a file are not contiguous, they may be on different
parts of the disk, and thus require significant head movement to
access the file contents.
Physical contiguity, however, ensures that the file is stored on one
track or, at worst, adjacent tracks. Because the disk can read a
track without moving the heads, file contiguity reduces head movement.
This assumes that no other software is accessing the disk at the same
time.
Conti9uity also enhances virtual-to-logical-block mapping (see Chapter
8) •

To ensure that the blocks in the file are physically contiguous,
allocate the whole file when you create it (see Section 3.3.2.1) and
specify that the allocation be performed contiguously.

3.4

ACCESS SHARING

Access sharing can be specified for disk sequential files,
as
described in the following sections. See Section 2.2.3 for general
information on shared access.

3-5

SEQUENTIAL FILE APPLICATIONS
3.4.1

Record Access to Sequential Files

Because of their internal structure, record-structured sequential
files are not read/write sharable in the manner of relative and
indexed files.
Thus, a read/write sharing declaration for such a file
is converted internally to a read-only sharing declaration before the
file is processed.
As a result, multiple read-only accessors who
have
specified
no-sharing, read-only sharing, or read/write sharing can access such a
file concurrently as long as no read/write accessor is present; or a
single accessor who has specified no-sharing, read-only sharing, or
read/write sharing can access such a file as long as no other accessor
of any kind is present. Other combinations are rejected:
the access
and sharing declarations are incompatible.
Limited sequential file sharing is possible, however, in the case of a
single read/write accessor in combination with multiple read-only
accessors, when the application programs involved (rather than RMS-ll)
can take responsibility for any interlocking required.
In this case,
the read-only accessors must specify sharing with
user-provided
interlocks to gain access;
the sharing declaration of
the single read/write accessor IS immaterial.
Each read-only accessor
cannot read beyond the logical end-of-file mark that existed at the
time that accessor opened the file,
and must
recognize
that
inconsistent data may be returned if the single read/write accessor
modifies data within the accessible portion of the file.

3.4.2

Block Access to Sequential Files

Sequential files can be read/write shared using block access, but for
those
accessors who specify read/write sharing, automatic file
extensions will not occur and the logical ,end-of-file mark in the file
header will be neither respected nor updated.
(Again, this is because
of the internal structure of sequential files.)
Such read/write
sharing uses the operating system's block-locking facilities to
coordinate shared access.
Sequential files can also be shared in a noninterlocked manner, with
user-provided interlocks.
Because of operating system restrictions,
the single read/write accessor must specify no-sharing or sharing with
user-provided
interlocks, and multiple read-only accessors must
specify sharing with user-provided interlocks.
These restrictions
also prohibit concurrent access to the file by read/write-sharing
accessors or an accessor who specified read-only
sharing
and
read/write access.
When no write accessor is present, sequential files can be shared
among multiple read-only accessors who have specified no sharing or
read-only sharing.

3.5

RECORD AND FILE PROCESSING OF SEQUENTIAL FILES

The
record
and
file
processing
capabilities
described
in
RSX-llM/M-PLUS RMS-ll:
An Introduction are available for sequential
flIes.
This sectlon discusses the operations and their implementation
and restrictions with sequential files.

3-6

SEQUENTIAL FILE APPLICATIONS
3.5.1

Record and Stream Operations

The following record
sequential files:

and

stream

operations

can

performed

CONNECT
DISCONNECT
FIND
FLUSH
GET
PUT
REWIND
TRUNCATE
UPDATE
In all record operations, RMS-ll establishes the current
context
(if any) and the next record context (if applicable).
record operation fails,
RMS-ll normally sets the current
context to none and does not change the next record context.

record
If any
record

NOTE
For more information on the RMS-ll error
codes
referred to in the following
sections, see the RSX-llM/M-PLUS RMS-1l
Macro Programmer's Guide.

3.5.1.1 CONNECT - A CONNECT operation affects the record context
the access stream as follows:

for

•

Current record -- There is no current record.
Any
requiring a current record fails at this point.

•

Next record -- If you did not specify that you were going to
append records to the file,
the next record is the first
record in the file.
If youcdid specify that you were going to append
the file, the next record is the end-of-file.

operation

records

3.5.1.2 DISCONNECT - A DISCONNECT operation destroys the current
record context for the access stream. You cannot resume this context
by reconnecting the stream.

3.5.1.3
RMS-ll:
1.

FIND - To perform a FIND

operation

~ile,

sequential

Determines the location of the record in the
to the specified record access mode:

file

•

In sequential-access mode, location is
next record pointer.

•

In key-access mode, location is determined
by
the
specified relative record number and match criterion.
(This access mode is available for fixed-format disk
sequential files only.)

3-7

indicated

according
by

the

SEQUENTIAL FILE APPLICATIONS
In RFA-access mode,
location is determined
by
the
• specified
RFA.
(This access mode is available for disk
files only.)
2.

Reads the block containing the record, or the first part of
the record
if the record spans blocks, from disk into the
task's I/O buffer, if it is not already in memory.
The block
may be in memory if the block was required by a previous
operation.

For disk files, returns the RFA to the program, but does
transfer the record to the program's user buffer.

Returns the RRN for fixed-format disk sequential files.

If no vali9 record exists in
depends on the access mode:

the

location

specified,

the

not

response

•

In sequential-access mode, the error code is ER$EOF, meaning
that no record was located because there are no more records
in the file.

•

In RFA-access mode, the error code is either ER$RFA,
if the
RFA had an invalid format, or ER$EOF, if the RFA specified a
location beyond the end of the file.

•

In key-access mode for fixed-format disk sequential files, the
error code is ER$KEY, if the key value had an invalid format,
or ER$EOF, if the key value specified a
location beyond the
end of the file.

A FIND operation affects the record context for the access
follows:
•

stream

For a sequential-access FIND operation:
Current record is set to value of the record found,
that
is, the next record before the FIND operation started.
Example:
You have connected a stream to a sequential file
without specifying that records will be appended to the
file (see Sec~ion 3.5.1.1). There is no current record,
but the next record is the first record in the file.
If
you execute a sequential FIND operation, the current record
is set to the first record in the file.
Next record is set to the record
current record.

virtually

following

the

Example:
From the previous example, the next record is the
second record in the file.
•

For an RFA-access or key-access FIND operation:
Current record is set to the record found,
record identified by the RFA or RRN.
Next record is unchanged.

3-8

that

is,

the

SEQUENTIAL FILE APPLICATIONS
Example:
In the preceding example, you performed
a
sequential-access
FIND operation after connecting the
stream to the file.
You now execute a FIND by RFA.
The
current record is set to the record specified, but the next
record is not changed. Therefore, when you perform another
sequential FIND operation, the current record is set to the
second record in the file, not to the record following the
one found by RFA.
You use a FIND operation instead of a GET operation for two reasons:
1.

It is faster because the record is not moved to the user
buffer. Although the time required to move a record from one
part of memory to another is very short, do not expend it
unnecessarily.

It does not change the next record in RFA or key access mode.
This
convention
allows
you to branch off sequential
processing for updating or deleting, and keep your place in
the file.

You can use a FIND operation in the following ways:
•

To skip records in sequential
successive FIND operations.

•

To establish a random starting point using RFA or key access
mode.
You could then initiate successive GET operations,
where the first operation gets the record found by RFA or by
RRN.

•

To establish a
operation.

•

To determine whether a record cell specified by RRN exists
a file (for fixed-format disk sequential files only).

current

record

access

for

mode

UPDATE

initiating

TRUNCATE

FLUSH - A FLUSH operation does not affect the record
for the access stream.

context

3.5.1.4

3.5.1.5

GET - To perform

GET

operation

sequential

file,

RMS-11:
1.

Determines the loc~tion of the record in the
to the specified access mode:
•

file

according

In sequential-access mode, location is indicated by the
next
record pointer,
if the get operation was not
immediately preceded by a successful FIND operation, or
the current record pointer set by an immediately preceding
successful FIND operation.

Location is determined by the specified relative record
• number
and
match
criterion
in
key-access
mode
(fixed-format disk sequential files only).
•
2.

Location is determined by the specified RFA in
mode (disk files only).

RFA-access

Reads the block containing the record, or the first part of
the record if the record spans blocks, from disk into the
task's I/O buffer, if the block is not already in memory.
3-9

SEQUENTIAL FILE APPLICATIONS
Example:
Your records are 50 bytes long.
When you read
sequentially through the file, RMS-ll must request a disk I/O
operation for every tenth GET operation that your program
executes.
3.

For disk files, returns the RFA to the program and moves the
record from the I/O buffer to the specified user buffer in
the program unless the program is operating in locate record
transfer mode
(see Section 3.5.2).
If the buffer does not
contain the entire record, RMS-ll reads more blocks into the
I/O buffer and assembles the record in the program's user
buffer, regardless of record transfer mode.

Returns the RRN for fixed-format disk sequential files.

If no valid record exists in
depends on the access mode:

the

location

specified,

the

response

•

In sequential-access mode, the error code is ER$EOF, meaning
that no record was located because there are no more records
in the file.

•

In RFA-access mode, the error code is either ER$RFA,
if the
RFA had an invalid format, or ER$EOF, if the RFA specified a
location beyond the end of the file.

•

In key-access mode for fixed-format disk sequential files, the
error code is ER$KEY, if the key value had an invalid format,
or ER$EOF, if the key value ,specified a
location beyond the
end of the file.

A GET operation affects the current
stream as follows:

record

for

context

• Current record is set to the record read.
Next record is set to the record virtually
• current
record.

the

access

following

the

Example:
You have connected a stream to a sequential file without
specifying that records will be appended to the file (see Section
3.5.1.1). There is no current record, but the next record
is the
first
record
in the file.
If you execute a sequential-access GET
operation, the current record is set to the first record in the file
and the next record is the second record in the file.

3.5.1.6
RMS-ll:

PUT - To perform

PUT

operation

sequential

file,

Determines whether the specified access mode is allowed.
Sequential-access mode must be specified unless the file is a
fixed-format disk file;
in that case, key-access mode is
allowed.
RMS-ll returns the error code ER$RAC if an illegal
access mode is specified.

Determines the destination of the record
according to the specified access mode:
•

the

file

In sequential-access mode,
the next
record
pointer
indicates the destination.
The destination must be the
end-of-fiLe;
if it is not, RMS-ll returns the error cods
ER$NEF.

3-10

SEQUENTIAL FILE APPLICATIONS
Your program gets to the end of a sequential file by:
Specifying that records will be appended to the file
when the program connects the record access stream to
the file (see Section 3.5.1.1).
Initiating sequential FIND and/or GET operations
RMS-ll returns an ER$EOF error code.
•

until

In key-access mode, the specified relative record number
indicates the destination.
Note that RMS-ll does not
check the validity of the designated RRN:
if
the
destination block is beyond the current end-of-file,
RMS-ll will extend the file to the destination block.

Reads the destination block in the file into the I/O buffer,
if the block is not already in memory.
The block may be in
memory if it was required by a previous operation.

Moves the record from the
buffer.

Writes the I/O buffer to disk only if the buffer is full.
If
there is no room for the block(s) in the file, RMS-ll extends
the file (see Section 3.3.2) and then writes the buffer to
disk.

For disk files, returns the RFA to the program.

Returns the RRN for fixed-format disk sequential files.

user

buffer

the

task's

I/O

A PUT operation affects the context for the access stream as follows:
•

•

For a sequential-access PUT operation:
Current record -- None. Any operation requiring a
record fails at this point.

current

Next record -- End-of-file.
A sequential
operation fails with error code ER$EOF.

FIND

GET

For a key-access PUT operation:
Current record -- None. Any operation requiring a
record fails at this point.

current

Next record -- Unchanged.

3.5.1.7 REWIND - A REWIND operation affects the
the access stream as follows:

context

requiring

•

Current record -- None. Any
record fails at this point.

•

Next record -- Set to the first record in the file.

3-11

operation

record

for

current

SEQUENTIAL FILE APPLICATIONS
3.5.1.8 TRUNCATE - A TRUNCATE operation declares end-of-file at the
position
of
the current record.
In doing so,
the operation
effectively deletes the current record and all records in the
sequential file following that record.
The TRUNCATE operation requires a valid current record.
It therefore
should follow a successful GET or FIND operation; otherwise, RMS-ll
returns the error code ER$CUR.

A TRUNCATE operation affects the context

for

the

access

stream

follows:
•

Current record -- None. Any
record fails at this point.

•

Next record -- End-of-file.

operation

After a TRUNCATE operation, you can immediately
file using PUT operations.

requiring

add

records

current

the

NOTE
The TRUNCATE operation does not reduce
the
actual
allocated
size
of
a
sequential file on a disk:
it merely
specifies
a new logical end-of-file
mark.

3.5.1.9 UPDATE - In an UPDATE operation, RMS-ll moves the specified
record from the task's user buffer to the I/O buffer, replacing the
current record set by a previous GET or FIND operation.
However,
RMS-11 does not immediately write the buffer to the file.
RMS-ll
requests the file control processor to write the changed buffer over
its original location on the disk only when the buffer must be
replaced in memory by another operation.
Example: You get a record by RFA and update it.
Then, you get
another record by RFA.
RMS-11 writes the buffer containing the first
record you updated only when it must replace the data in the buffer to
satisfy the second GET operation.
UPDATE operations have the following restrictions:
•

The operation is valid only on disk sequential files.
If you
attempt it on magnetic tape files or unit record devices,
RMS-ll returns the error code ER$IOP.

•

The operation requires a valid current record.
It therefore
should follow a successful GET or FIND operation; otherwise,
RMS-ll returns the error ER$CUR.

The size of the record cannot change during an UPDATE
• operation.
RMS-ll returns the error code
If it changes,
ER$RSZ.
•

You cannot update stream records.
returns the error code ER$RFM.

If you attempt

it,

RMS-ll

None of these errors affects the original record in the file on disk.

3-12

SEQUENTIAL FILE APPLICATIONS
An UPDATE operation affects the
follows:

context

•

Current record -- None. Any
record fails at this point.

•

Next record -- Unchanged.

3.5.2

for

the

operation

access

stream

requiring

current

Record Transfer Modes

You can manipulate records either in the I/O buffer or in your
program's user buffer.
Each of these options is called a record
transfer mode. You can change record transfer mode at run time, even
between record operations.
Figure 3-1 shows the I/O and user buffers.

r.;-------- - - - - - - - ,
I SIZE DEPENDS ON:

I· NUMBER OF FILES OPENED SIMULTANEOUSLY:

I. BUCKET SIZES
L NUMBE~~RECOrRD~CCE~ ~~AMS _ _
USER BUF FE S
R -...

1/0
BUFFERS

VIRTUAL
MEMORY

PROGRAM

RMS-11

INTERNAL
CONTROL
STRUCTURES

-------~-

islZE DEPENDS ON:

~ • RMS-11 FUNCTIONS USED I
~ ~V~RLA~~U~TURE~~E~

ZK-1174-82

Figure 3-1: RMS-11 Task Structure

3.5.2.1 Move Mode - Move mode requires that
between the user and I/O buffers:

each

record

copied

•

On GET operations, RMS-11 moves the record from the I/O buffer
to the user buffer before returning control to your program.

•

On PUT and UPDATE operations, your program assembles the
record to be written into the file in the user buffer. During
the operation, RMS-11 moves the data into the I/O buffer
before updating the file.

Move :mode is the default record transfer
languages and all file organizations.

3-13

mode

for

all

programming

SEQUENTIAL FILE APPLICATIONS
3.5.2.2 Locate Mode - Locate mode enables your program to manipulate
records in the I/O buffer, eliminating the data transfers between it
and the user buffer. However, when you specify locate mode, RMS-ll
uses it only when such usage does not compromise data integrity.
Otherwise, RMS-ll uses move mode. Therefore, your program must still
contain a user buffer.
Example:
RMS-ll uses move mode instead of locate
span buffers in a sequential file.

mode

when

records

Example: RMS-ll uses move mode instead of locate mode if you opened
the file indicating that you were going to perform UPDATE operations
on it.
RMS-ll's use of move mode instead of locate mode is transparent to
your program as long as you use RMS-ll facilities to access the record
data.
For sequential files, your program can both performs both GET and PUT
operations in locate mode. See your high-level language documentation
to determine whether the language supports locate mode and,
if it
does, what the programming techniques are.

3.5.3

I/O Techniques

You can use the following techniques to
record operations.

improve

the

performance

3.5.3.1 Asynchronous Record Operations - Within each access stream,
your program can perform any record operation either synchronously or
asynchronously.
In synchronous operations, RMS-ll returns control to
your program after the operation ends, either successfully or with an
error.
When you execute an asynchronous operation, RMS-ll may return control
to your program before the operation is complete.
The program
continues processing while the physical transfer of data between disk
and memory is carried out. However, you must not initiate another
record operation on that stream until the first operation ends;
otherwise, RMS-ll returns the error code ER$ACT. See your high-level
language documentation for asynchronous techniques.

3.5.3.2 Deferred Write - The normal mode of operation for sequential
files
is similar to operations using deferred write with the other
file organizations (see Chapters 4 and 7). Using this technique for
sequential files does not change or improve performance.

3.5.3.3 Multiple Buffers - The
available to sequential files.

multiple

3-14

buffer

capability

not

SEQUENTIAL FILE APPLICATIONS
3.5.3.4 Multiple Access Streams - RMS-ll allows each program to use
only one stream on a sequential file because sequential files are not
formatted to permit simple and economical sharing (see Section 3.4).

3.5.3.5 Multiblock Count - Your task can be set up so that more than
one block from a disk sequential file is read or written at one time.
This multiple-block I/O can improve processing because it tends to
reduc,e the number of physical I/O operations.
However, it also
increases the size of the task, on a one-for-one basis; that is, for
each increment of the multiblock count (MBC) , the I/O buffer in the
task grows by 512 bytes.
An MBC greater than 1 is therefore useful for
including file population.

sequential

processing,

Example:
You are using
50-byte
records.
During
sequential
processing, if the MBC is 1, RMS-ll requests a disk I/O operation for
every tenth record operation your program executes, whether the
operations are GET or PUT operations.
If you set MBC to 5 for
instance, RMS-ll requests a physical I/O operation for every 50 record
operations.

3.5.4

File and Directory Operations

The following file
sequential files:

and

directory

operations

can

CLOSE
CREATE
DISPLAY
ENTER
ERASE
EXTEND
OPEN
PARSE
REMOVE
RENAME
SEARCH
See your high-level language documentation for a
support provided.

4-15

description

the

CHAPTER 5
INDEXED FILE STRUCTURE AND ACCESS

DIGITAL designed the RMS-ll indexed file organization to
following goals:

achieve

the

•

Content-addressable record access -- Each record in the file
can be located on the basis of the values in designated
portions of the data, called key fields.

•

Uniform random access time -- Each record in the file can be
located with approximately the same number of I/O operations,
regardless of when it was added to the file.

•

Alternate key capabilities (comply with ANSI COBOL Level 2) -Each record
in the file can be located via more than one key
field.

•

Very good performance on sequential access by primary key -- A
program can sequentially read a reasonably designed indexed
file by primary key almost as fast as it can sequentially read
a sequential file.

•

Good performance on sequential access by alternate keys
Each record in the series can be accessed with (typically) one
to three I/O operations.

•

Unique record address for the life of the file (data base key
concept)
A record
in a file can be located via a unique
identifier
(record file address)
established by the PUT
operation.
The record may be deleted, but its unique
identifier is never reused.

Preserve the state of processing despite a system failure
• Normally,
each logical write operation results in a physical
transfer of data from memory to disk.
Therefore,
the file
reflects each record inserted.
However, you can override this
mode with deferred write in some cases.
More importantly, RMS-ll performs record operations so
the following are true:

that

both

•

File corruption is avoided or minimized even if a
failure occurs during a write-type record operation.

•

Even if some
accessed.

corruption

exists,

user

data

NOTE
You should still reorganize your file if
the
system
fails during write-type
processing on an RMS-ll indexed file.
5-1

can

system

still

INDEXED FILE STRUCTURE AND ACCESS
5.1

PHYSICAL FILE STRUCTURE

On disk, an indexed file consists of three kinds of blocks:
•

Prologue -- RMS-11 information about the
attributes and key and area descriptions

file,

including

•

Index -- Index records for primary and alternate keys pointing
the way to a data record

•

Data

Your data records and index data records

The prologue contains information about the keys and areas of the
file.
RMS-1l allocates at least one block for the key descriptors and
at least one block for the area descriptors.
RMS-ll uses more blocks
as needed.
Size calculations are discussed in Section 6.6.1.
Areas are portions of an indexed file that are treated independently
ror-rnitial allocation, extensions, placement, and bucket sizes.
Like
subfiles, but invisible to the operating system, areas allow you to
divide indexed files logically into separate units for each index and
for the data records to improve performance;
see Section 6.3 for more
information on areas.
In addition, RMS-11 extends the prologue to an integral multiple of
the area 0 bucket size,
if the area 0 bucket size is 2, 4, or 8
blocks. See Section 6.5 for more information on bucket sizes.
The location of the index and data blocks is up to you:
•

If the file is a single area, RMS-11 allocates data and
index
blocks in buckets as it needs them;
they are therefore
interspersed throughout the file.

•

If the index and data are set up in separate areas,
RMS-1l
allocates each type of bucket from the appropriate area;
the
index is therefore set apart physically from the data portion
of the file.

Figure 5-1 illustrates an indexed file both with and without areas.
RMS-11 formats buckets in an indexed file as
it requires them for
record storage.
The RMS-11 control bytes are set to their initial
values:
•

14 bytes, beginning with byte 0 of the bucket
control information.

•

The last byte of the last block duplicates the first
the bucket for checking I/O completion.

contain

bucket
byte

RMS-l1 packs index or data records, including record format overhead,
into each bucket, beginning with byte 14, end-to-end and byte-aligned.
Figure 5-2 shows the RMS-11 bucket format.

5-2

INDEXED FILE STRUCTURE AND ACCESS

ALTERNATE

WITHOUT AREAS

WITH AREAS

PI = PRIMARY INDEX
DR = DATA RECORDS
AI = ALTERNATE INDEX
ZK-1165-82

Figure 5-1: Indexed File with and without Areas

5-3

INDEXED FILE STRUCTURE AND ACCESS

BUCKET BOUNDARIES

~
RECO RDS

II
I

•

CONTROL FLAGS
LEVEL IN INDEX
VBN OF NEXT BUCKET IN LEVEL
RECORD IDENTIFIER IN FORMATION
POINTER TO FREE SP ACE IN BUCKET
BUCKET ADDRESS SA MPLE
AREA CONTAINING THI S BUCKET
CHECK BYTE EQUALS LAST BYTE-----/

ZK-1160-82

Figure 5-2: Formatted Bucket

5.2

CONCEPTUAL FILE STRUCTURE

No matter how it is laid out physically,
the
indexed file
is
conceptually a prologue plus a group of indexes, one per key.
Each
index consists of horizontal chains of buckets called levels.
Figure
5-3 illustrates this structure as a pyramid.

LEVEL 2

LEVEL 1

LEVEL 0

BUCKET

Figure 5-3: Index as a Pyramid

5-4

INDEXED FILE STRUCTURE AND ACCESS
The lowest level of an index is level O.
The level number is
incremented for each successive (and smaller) level, that is, levell,
level 2, and so on. The highest level in an index is a single bucket
called the root;
this bucket is the entry point to the index for
random accesses using this key.
Each index has at least two levels (0
and 1).
The depth of an index is equal to the level number of the root.
An
inde:~pth
relates to the time needed to randomly access any record
in the file via that index.

5.2.1

Data

Level 0 of each index is called the data level;
it consists of data
buckets.
In the primary index, level 0 contains buckets of your data
reco:rds.
In the alternate indexes, level 0 buckets contain pointers
to your data records.

5.2.1.1 Level 0 of the Primary Index - RMS-ll physically orders data
records by ascending primary key value along the bucket chain.
The
records having the lowest primary key value reside in the first bucket
of the level and the records with the highest primary key values
reside in the last bucket. RMS-ll preserves this order regardless of
the insertion sequence of the· records.
Each bucket in level 0 shares the following properties:
•

The last data record in a bucket has an equal
value than any other record in the bucket.

higher

key

•

The last data record in a bucket has a lower key
the first record in the next bucket in the chain.

value

than

Each bucket thus has a high-key value, located in the last record of
the bucket. This concept is the core of RMS-ll index file structure.
NOTE
RMS-ll places records with duplicate key
values next to each other on a first-in,
first-out
(FIFO)
basis.
If
these
duplicate records cannot fit in the same
bucket, RMS-ll stores the overflow in a
continuation
bucket.
Continuation
buckets are extensions of
level
0
buckets and, as such, are not indexed.
This extension storage preserves the
high-key concept.

5.2.1.2 Level 0 of an Alternate Index - Level 0, the data level, of
an alternate index' contains secondary index data records (SIDRs). A
SIDR consists of two elements:
1.

An alternate key value from a data record stored in the
primary data level.
The SIDRs in the data level of each
alternate index are stored in ascending order by this key
value.

5-5

INDEXED FILE STRUCTURE AND ACCESS
2.

One or more pointers to data records
in the primary data
level.
Multiple pointers occur when you allow duplicates for
the alternate key and records with duplicate values for
the
key actually exist in the file.

Figure 5-4 illustrates the SIDR format.

A. DUPLICATES ALLOWED

POINTER ARRAY

~
B. NO DUPLICATES ALLOWED

VBN

[

[. [

71£

DATA RECORDjD
FLAG BYTE
POINTER TO PRIMARY
LEVEL 0

r-----,---.-,------~

ZK-1152-82

Figure 5-4: Format for Secondary Index Data Record

5.2.2

Indexes

Levels 1 and
consist of
guide RMS-ll
O. An index

above in an index are called the
index levels;
they
index buckets.
Index buckets contain index records that
through the levels to the data records rn--primary level
record consists of two elements:

The high-key value of a bucket in the next lower level in the
index.
Because RMS-ll arranges these values in ascending
sequence, there is a high-key value for index buckets also.
However,
the last high-key value in the last index bucket of
a level is set to the highest possible key value, rather than
the highest key value in the file.
The associated pointer
references the last bucket in the next lower level.

A pointer to the bucket associated with the high-key value.

Example:
The buckets in level 1 of the primary index contain the
high-key values of the data buckets
in level
O. Then, level 2
contains the high-key values from level 1 and so on.
Figure 5-5 shows
an example of a primary index.
In other words, each bucket on a given level
is
represented by an
index record
in the next higher level.
Thus, the number of buckets
required on each successive level decreases exponentially until
the
root bucket is reached.
Example:
•

If an index bucket can hold 10 index records, then:
If level 0 contains 2000 data buckets:
Level 1 contains 200 index buckets
Level 2 contains 20 index buckets

5-6

INDEXED FILE STRUCTURE AND ACCESS
Level 3 contains 2 index buckets
Level 4 contains 1 index bucket

• If level 0 contains 10,000 data buckets:
Level
Level
Level
Level

1 contains
2 contains
3 contains
4 contains

1000 index buckets
100 index buckets
10 index buckets
1 index bucket

MAXIMUM KEY VALUE

LEVEL 3
(ROOT)

LEVEL 2

LEVEL 1

LEVEL a

MAXIMUM KEY VALUE

1Y005t lR5T 5T.,97821

t..===~===~~

ZK-1151-82

Figure 5-5: Example of a primary Index

5.2.3

Random Access Using the RMS-ll Indexed File Structure

The following steps show how RMS-ll uses the indexed file structure to
execute a random access operation. These steps constitute a process
called "follow the index."
1.

RMS-ll examines memory-resident index descriptors to find the
location of the root for the specified index. Note that the
root can be cached (see Section 7.4.3), eliminating the I/O
operation to read the root in the next step.

RMS-ll reads the root and scans for the first value greater
than or equal to the key value specified when the operation
was initiated.
If all else fails, the search will find the
highest possible key value in the last index record.

RMS-ll reads the bucket indicated by the pointer associated
with the selected key value and scans for the first key value
greater than or equal to the value specified. RMS-ll repeats
this step through the levels until level 0 is reached.

5-7

INDEXED FILE STRUCTURE AND ACCESS
Example:

Refer to Figure 5-5 for this example.

The specified primary key value is "YOOS."
RMS-ll determines the VBNs of the root bucket from the memory-resident
index descriptors and requests the file control processor to read
those blocks into an I/O buffer.
RMS-ll scans the
index records
in
the root.
The first key value equal to or greater than "YOOS" is the
maximum key value in the last record.
RMS-ll uses the bucket pointer in this index record to request another
I/O operation. The file control processor reads the specified blocks
into the I/O buffer, and RMS-ll scans them looking for a key value
equal
to or greater than "YOOS." Again, it finds no qualifying key
value until the last record in the bucket, which contains the maximum
key value.
This index record points to a level 1 bucket.
Upon RMS-ll's request, the file control processor brings the indicated
bucket into memory.
RMS-ll searches the bucket, terminating with the
last record in the bucket, which contains the maximum key value.
The file control processor reads
RMS-ll's request.

5.2.4

the

indicated

level

bucket

Why this Structure?

Mechanical data storage devices make I/O operations the slowest part
of file processing.
Ideally, a file is read into memory when it is
opened and maintained there, without additional I/O operations, until
the file is closed.
Some very small files allow this approach and are
handled most efficiently by your own search techniques rather than by
RMS-ll's indexing facilities.
However, most indexed files are very much larger than the memory
available for data buffering.
Such files are therefore partitioned
into pieces that can be read to memory.
RMS-ll calls these pieces
buckets.
By definition, one I/O operation is required to access one
bucket.
If no index to the data exists, a task must scan sequentially through
the buckets of a file to find a specific record.
Such a search, on
the average, accesses half the buckets in the file.
Figure 5-6 plots
the time curves for various file searching techniques.
You can optimize nonindexed access by:
•

Ordering the records by a key value

•

Using a binary search technique

Then, the number of accesses required to find a record approaches 10g2
of the total number of buckets (see Figure 5-6). This better, but
still mediocre, speed is realized on one of perhaps many keys.
ThE= RMS-ll indexed structure uses buckets so that your programs can
handle files more efficiently.
In most cases, RMS-ll uses n+l I/O
operations to locate a record by primary key, where n is the depth of
the file's primary index.

5-8

INDEXED FILE STRUCTURE AND ACCESS
In a :small file, this technique is not appreciably faster
than a
sequential scan. However, given typical key sizes, a primary index of
depth 3 can represent from 1,000 to 125,000 buckets of data records,
using only single-block buckets.
Normally, four disk accesses are
needed to get any record by primary key value.
Example: You want to search 50 buckets of data for
records with
specific primary key values.
The average number of buckets you read
during each search depends on your search technique (see Figure 5-6):
25.5

5.3

buckets for a nonindexed search of unsorted records
buckets for a binary search of records sorted by primary
key
buckets for an RMS-11 indexed search

PROCEDURES FOR PERFORMING RANDOM RECORD OPERATIONS

The procedures for performing random record operations on indexed
files depend on the circumstances for the individual operation, the
file's design, and whether alternate indexes must be updated.

RMS-11 PRIMARY INDEX SEARCH,
10 RECORDS PER INDEX BUCKET'
BINARY SEARCH FOR PRIMARY KEY,
RECORDS ORDERED BY PRIMARY KEY

125

100

NUMBER OF
PRIMARY DATA
BUCKETS

O~~~~~~~~~~~~~~~~-+-L~~~

NUMBER OF BUCKETS READ IN SEARCH

"LINE SHIFTS TO LEFT AND BREAK RIGHT MOVES UP
AS NUMBER OF INDEX RECORDS PER BUCKET GOES UP
ASSUMING OPTIMAL PACKING
'
ZK-1159-82

Figure 5-6: Search Time Curves
5-9

INDEXED FILE STRUCTURE AND ACCESS
5.3.1

Writing a Record

When your program initiates a PUT operation, RMS-ll moves the data
from the task to the proper bucket in level 0 of the primary index and
updates all indexes involved with the record.
This process can be
simple,
requiring minimal
I/O operations.
It can also be complex,
requiring more procedures and data transfers. The complexity depends
on whether there is enough room for the new record in its data bucket.

5.3.1.1 Simplest Case - In the simplest PUT operation, RMS-ll finds
room in the target data bucket to insert the record.
To execute the
operation, RMS-ll performs the following steps:
1.

Determines the value
record.

the

primary

key

field

from

Follows the primary index to the proper level 0 bucket.

Reads the level 0 bucket and sequentially scans for the first
record with a primary key value greater than the specified
value.
RMS-ll then establishes a position before that
record, or after the last existing record in the bucket if:
•

The key values are equal.

•

The first record in the next data bucket has a higher
value.

the

key

Compresses deleted records.
RMS-ll can reuse bytes in a
deleted record depending on the record format and whether you
allow duplicates in the primary key field.
Section 6.2.5
discusses reusing space from deleted records.

Determines whether the record to be inserted
bucket (in this simplest case, it does).

No primary
Inserts the record at the established position.
index buckets are updated since no high-key value has
changed.

If there are alternate keys, updates those indexes, using the
following sequence of steps for each one:

fits

the

•

Follows the alternate index to the proper level 0 bucket.

•

Reads the level 0 bucket and sequentially
key value specified in the record:

scans

for

the

If a value higher than the one specified is found,
inserts a SIDR for the record before the SIDR for the
higher value.
If a match is found, determines whether duplicates
allowed for the alternate key:

are

If duplicates are allowed, RMS-ll follows the duplicate
pointer array in the SIDR to the end, then inserts a
pointer to the newly inserted record.
This procedure
preserves the first-in,
first-out convention. After
the last alternate key,
RMS-ll returns a successful
completion code to the program.

5-10

INDEXED FILE STRUCTURE AND ACCESS
If duplicates are not allowed, RMS-ll returns to

level

o of the primary index, flags the newly inserted record

as deleted, logically removing it, and returns an error
code to the program.
Example:

Refer to Figure 5-5 for this example.

RMS-ll examines the record in the user buffer of the record access
stream initiating the PUT operation. The value in the primary key
field is "JACKSON." RMS-ll locates the primary root and requests the
file control processor to read the bucket into the I/O buffer
associated with the stream. When that I/O operation completes, RMS-ll
scans the bucket, looking for a key value equal to or greater than
"JACKSON." It finds "JONES."
RMS-ll requests the bucket indicated by the pointer in the "JONES"
index record.
When RMS-ll scans this level 2 bucket, it finds that
"JONES" again ends the search. Following the pointer in this
index
record,
RMS-ll requests another bucket.
Its search of the level 1
bucket ends in another "JONES" index record.
RMS-ll requests the level 0 bucket indicated by this last index
record.
It finds that a data record with a primary key value of
"JONES" is the only occupant of the bucket.
There are no deleted
records to compress, so RMS-ll writes the "JACKSON" record before the
"JONES" data record, moving the "JACKSON" record down in the bucket.
There are no alternate keys.
code to the program.

RMS-11 returns a

successful

completion

5.3.1.2 Bucket Splitting - If there is not enough room in the target
data bucket for the record,
RMS-ll allocates a new bucket and
reorganizes the records in the old one between the two buckets.
This
procedure is called bucket splitting.
Bucket splitting is identical with the simplest case (Section 5.3.1.1)
to step 5 where RMS-11 determines whether the new record fits in the
bucket. When there is not enough room, RMS-ll does the following:
1.

Reads the appropriate area descriptor from the file prologue.
If enough blocks for a bucket are allocated for the area,
RMS-11 formats the blocks into a bucket and updates the area
descriptor to reflect the new bucket. Otherwise, RMS-ll
requests the operating system to allocate enough blocks, and
then formats them into a bucket and updates the area
descriptor.

Splits the target bucket at the point where the record should
be inserted.
RMS-11 moves the records in the high portion of
the bucket into the new bucket;
these records have primary
key values higher than those of the new record.

5-11

INDEXED FILE STRUCTURE AND ACCESS
NOTE
When RMS-ll moves a record between buckets, it marks
the
record's
original
location with a record
reference vector (RRV).
An RRV is a copy of the
record's
header
(both contain 7 bytes).
RRVs
preserve alternate key and RFA access, holding the
original location of the record and pointing to its
current location.
Only one RRV is created for a
record:
if the record moves again, RMS-ll updates
the RRV with the record's new location.
Since the original location of a record
is filled,
either with the record or a pointer to that record,
RMS-ll does not have to update alternate indexes
every time a record moves.
This convention means one
extra I/O operation may be needed to find or get a
record via an alternate key, but it prevents a
complex and costly index update for each bucket
split.
3.

Inserts the data record in the original target bucket.
If
the record will not fit,
RMS-ll inserts it into the new
bucket.
If the record will not fit there either, RMS-Il will
create another bucket (see step 1) and put the record there.

Updates the
bucket(s).

Returns to the primary root bucket and follows the index to
the level 1 index bucket that points to the data bucket that
split.

Inserts index record(s) for the new data bucket(s).
If the
index bucket splits, RMS-Il uses a procedure similar to this
to move the index records and update the next higher level of
the index.
Splitting can occur all the way to the root where
a new root is created and the file prologue updated.

If there are alternate keys, RMS-Il updates those indexes as
described
in step 7 of the simplest case (Section 5.3.1.1).
Bucket splitting can occur in alternate indexes also.

level

bucket

chain

include

the

new

5.3.1.3 Incremental Reorganization - The process of inserting each
data record where it belongs in level 0 and updating the indexes when
RMS-ll inserts the record is called incremental reorganization of the
file.
Incremental reorganization has the following advantages:
•

It eliminates reorganization periods where special software
incorporates overflow areas into the main file and that file
is not available for processing

•

It ensures equal access time to old and new records

•

It enables performance on sequential access by primary key to
approach the speed of sequential access to a sequential file

5-12

INDEXED FILE STRUCTURE AND ACCESS
This process has its costs:
additional I/O operations occur when a
bucket splits.
But with good file design and file loading, bucket
splitting (and the time for each bucket split) is minimal.
Chapter 6
discusses these considerations in detail.

5.3.2

Getting and/or Finding a Record

To execute a key-access GET or FIND
following steps:
1.

operation,

RMS-ll

performs

the

Determines from the instruction initiating the operation
following criteria:

the

•

Key of reference, indicating which index to search
which key field within the data record to examine

and

•

Value to find

•

Match criterion (equal to, greater than, or both)

•

Number of characters to match

Follows the index to the proper level 0 bucket.

Reads the level 0 bucket, sequentially scanning for the first
record with a value in the specified key field that matches
the specified value according to the match criterion.
This
search can continue into other buckets:
•

If no such record is found, RMS-ll returns an error code.

•

If such a record is found, RMS-ll:
Determines which index has been read:
If it is the primary index, RMS-ll
step.

goes

the

If it is an alternate index, the record located
is a
SIDR.
RMS-ll follows the SIDR pointer to the primary
level 0 data record.
For a GET operation only, moves the record to the user
buffer associated with the access stream performing the
operation.
Sets the current context for the stream performing this
operation.
The effect of each record operation on
context is described in Section 7.2.
Returns a successful completion code.
Example:

Refer to Figure 5-5 for this example.

RMS-ll determines the key (and index) of reference and the value to
find ,from the
instruction initiating the operation.
In this case,
they are the primary key (key 0) and "ABI."

5-13

INDEXED FILE STRUCTURE AND ACCESS
RMS-ll locates the primary root and requests the file control
processor to read the bucket into an I/O buffer.
RMS-ll sequentially
scans the root for an index record whose key value is equal to or
greater than "ABI." It finds "ABRAM."
RMS-ll requests the bucket indicated by the pointer in the "ABRAM"
index record.
When RMS-ll searches this level 2 bucket, it finds an
index record containing the key value "ABNER." Following the pointer
in this index record, RMS-ll requests another bucket. The search of
the level 1 bucket ends in the key value "ABLE."
RMS-ll requests the level 0 bucket indicated by this last index
record.
RMS-ll changes its search criteria to that specified in the
initiating instruction:
it looks for a record where the first 3 bytes
of the primary key field equal "ABI." Since the only record in the
bucket contains "ABLE" in its primary key field, RMS-ll cannot satisfy
the search requirements.
It returns a "record not found" error code
to the program.

5.3.3

Updating a Record

RMS-ll requires an UPDATE operation to be preceded by a GET or FIND
operation, although some high-level languages hide this prerequisite.
To execute an UPDATE operation, RMS-ll performs the following steps:
1.

Locates the key fields of the revised record in the user
buffer associated with the access stream performing the
operation. RMS-ll compares those key values with the values
in the current record:
•

If the primary key value changed, RMS-ll returns an
code.

•

If an alternate key value changed, RMS-ll
you allowed changes for that key:

checks

error
whether

If not, RMS-ll returns an error code.
If so, RMS-ll continues processing.
2.

For each alternate key where the key value changed, RMS-ll
performs the following steps to delete the pre-update value
from the alternate index:
•

Reads the data bucket containing the
that bucket is not in memory.

current

•

Saves the pre-update alternate key value from the
record.

current

•

Follows the index to the level 0 bucket that
contain the SIDR for the pre-update key value.

should

•

Reads the level 0 bucket and sequentially
pre-update key value:

scans

If a value higher than the one specified
RMS-ll goes to the next alternate index.

5-14

record,

for
is

the

found,

INDEXED FILE STRUCTURE AND ACCESS
Example:
RMS-ll scans a bucket,
searching for a
pre-update key value of "D." It finds a record with a
key value of "E." Since "E" is greater than "0," RMS-ll
ends the search and this step in the procedure.
If a match is found, RMS-ll scans the duplicate pointer
array in the SIDR to the entry for the record being
updated and flags it as deleted.
NOTE
To allow keys to change, RMS-ll requires that you
also allow duplicates.
Therefore, if you allow
alternate key values to change,
there is
a
duplicate pointer array in the SIOR for each key
value.
However, you should
refer
to
your
high-level language documentation for specific
information on your compiler's implementation of
this capability.
3.

Reads the data bucket containing the current record, if that
bucket is not in memory.
RMS-ll replaces the current record
in the I/O buffer with the updated version in the user
buffer.

Writes the bucket to the file.

For each alternate key where the key value changed,
RMS-ll
performs the following steps to insert the post-update value
in the alternate index:
•

Reads the data bucket containing the
that bucket is not in memory.

current

record,

•

Follows the index to the level 0
contain the post-update key value.

bucket

that

•

Reads the level 0 bucket and sequentially
post-update key value:

scans

should
for

the

If a value higher than the one specified is found,
RMS-ll inserts a SIOR for the new record before the
SIDR for the higher value.
If a match is found,
RMS-ll follows the duplicate
pointer array in the SIDR to the end, then inserts a
pointer to the new record.
After the last alternate key is updated, RMS-ll returns
successful completion code to the program.

5.3.4

Deleting a Record

RMS-ll requires a DELETE operation to be preceded by a GET or FIND
operation, although some high-level languages hide this prerequisite.

5-15

INDEXED FILE STRUCTURE AND ACCESS
To execute a DELETE operation, RMS-ll performs the following steps:
1.

If there are alternate keys, RMS-ll updates those indexes
follows, using the same sequence of steps for each:

•

Reads the data bucket containing the
that bucket is not in memory.

current

record,

Follows the index to the level 0 bucket that should
• contain
the SIDR for the key value in the deleted record.
Reads the level 0 bucket and sequentially
specified key value:

•

scans

for

If a value higher than the one specified is found,
goes to the next alternate index, if any.
If a match is found, RMS-ll determines
allowed duplicates:

whether

the

RMS-ll

you

have

If so, RMS-ll follows the duplicate pointer array in
the SIDR to the entry for the record being deleted and
flags it as deleted.
If not, RMS-ll deletes the SIDR.

Reads the data bucket containing the current record, if
bucket is not in memory.

that

Changes the flag byte in the header of the current record
indicate that the current record is deleted.

Writes the bucket to the file.

If the record has moved, reads the level 0 bucket containing
the RRV.
RMS-ll changes the flag byte in the RRV to indicate
that the record is deleted.

Writes the bucket to the file.

RMS-ll does not compress a deleted record until
it needs space to
insert another user data record into the bucket (see Section 5.3.1).
RMS-ll does not compress deleted RRVs.
NOTE
RMS-ll does not modify or reduce any
index structure or allocation during a
DELETE operation.

5.4

PROCEDURES FOR PERFORMING SEQUENTIAL RECORD OPERATIONS

Your program can use sequential access mode to perform
record operations:
FIND
GET
PUT

5-16

the

following

INDEXED FILE STRUCTURE AND ACCESS
During sequential-access GET and FIND operations, RMS-ll does not
usually read an index to locate the specified record.
Instead, RMS-ll
uses the record context for the stream performing the operation to
identify the proper data bucket.
For FIND operations, RMS-ll uses the next record pointer to identify
the target bucket.
For GET operations, RMS-ll uses the next record
pointer, if the GET operation was not immediately preceded by a
successful FIND operation. The current record pointer is used if the
GET operation was immediately preceded by a successful FIND operation.
Next, RMS-ll requests the file control processor to move the target
bucket into the I/O buffer, if that bucket is not in memory.
If it
has requested a SIDR bucket, RMS-ll then follows the appropriate
pointer to the user data record.
During sequential-access PUT operations, RMS-ll compares the primary
key value of the specified record with the primary key value of the
last record written:

5.5

•

If the specified record's primary key value is equal to or
greater than the last record's primary key value, RMS-ll
performs a key-access PUT operation
(described in Section
5.3.1) •

•

If the specified record's primary key value is less than the
last record's primary key value, RMS-ll returns an error code
to the program.

I/O COST OF PERFORMING RECORD OPERATIONS

Table 5-1 provides simple algorithms for predicting the number of
operations any RMS-ll record operation requires:
key~

all

I/O

indexes

•

The value n = index depth of the indicated
do not necessarily have the same depth.

•

Algorithms do not include I/O operations caused by program or
RMS-ll
overlays, operating system overhead, or by file
extensions (see Chapter 8).

5-17

INDEXED FILE STRUCTURE AND ACCESS
Table 5-1: I/O Cost of Performing Record Operations

Record Operation

Primary
Key

Each
Alternate
Key

Key-access GET or FIND
Record in original location
RRV in original location (record moved)

n+l
n+l

n+2
n+3

Sequential-access GET or FIND

0-2 a

PUT
Simplest case
Split in data level
Bucket split up entire index

n+2
2n+6 c
(n** 2+lln+20) /2 d

UPDATE e
Alternate key value did not change
Alternate key value changed

I
I

DELETE
Record in original location
RRV in original location (record moved)

n+2
2n+6 c
(n**2+lln+20) /2 d

o
2(n+2)f

aBreaks down to:

o or 1
o or 1

I/O to position to current record
I/O to locate next record

bBreaks down to:

o or I
o or I
1 or 2

I/O to position to SIDR for current record
I/O to locate SIDR fo~ next record
l/Os to retrieve user data record

cBreaks down to:
n+2
n+l
3

l/Os to read and write the old bucket
l/Os to read and write the level 1 index bucket
l/Os to write the new bucket and update the area
descriptor in the prologue

dBreaks down to:
(n+2)+(n+I)+n+(n-l)+ .•• +3+2
l/Os to return to the primary root and read and
write updated buckets from level 0 to the root
3(n+l)
5

l/Os for each bucket split (see footnote c)
l/Os to create the new root

eYalues assume record
splitting.

length

does

not

change

and

cause

bucket

fn+2 if either the old or new key value does not belong in the
index;
for example,
the field contains the null key value
defined for the key, or a variable-length record does not
contain the whole key field.
gYalue is different if one of the following is true:
•

You specified the "fast delete" option
(available in MACRO-II
only) when you initiated the DELETE operation. Then, RMS-ll does
not update alternate indexes in which duplicate keys are allowed.

RMS-ll has to scan a long duplicate array into one or more
• continuation
buckets. Then, one I/O operation is needed for each
additional bucket.

5-18

CHAPTER 6
INDEXED FILE DESIGN

Indexed file design ranges from the basic elements of your application
(record definition and key selection) to the structure of the file and
to the methods used to put the data into the file.
This range
includes:

6.1

Record si ze

Key selection

Areas

Placement control

Bucket size

Allocation

Population techniques

RECORD SIZE

You can use only fixed- and variable-length records in RMS-ll
files.
RMS-ll calculates length (RL), in bytes, as follows:
RL

indexed

= 7+RFO+DS

where:
7

is bytes for RMS-1l record header

RFO

is bytes for record format overhead:

is bytes of data

0 for fixed, 2 for variable

Set your record size to reflect application requirements;
do not
adjust it to fit bucket size. For instance, if you are using I-block
buckets, you should not, if you can avoid it, set a record length so
the records just fit into the buckets:
512
-15

bytes in a block
bytes of indexed file overhead per bucket

497
- 7

bytes left for records
bytes for the record header

490

bytes left for the data and record format overhead

6-1

INDEXED FILE DESIGN
This calculation seems ideal at first.
However, when the record moves
during a bucket split or RMS-ll deletes the record, and some RMS-ll
overhead is left in the bucket, a normal data record cannot fit:
the
bucket is essentially useless, with up to 490 bytes of unused space.
If your application requires 490-byte records, you should use them,
keeping the preceding limitation in mind and, perhaps, choosing a
different bucket size.
NOTE
Records in an indexed file cannot span
buckets and bucket sizes are limited by
the operating system to 32
blocks.
Therefore,
the maximum record size,
including overhead, is 16,369 bytes.

6.2

KEY SELECTION

A file's keys can take up significant space in an indexed file and can
have a significant effect on the number of I/O operations needed to
access the file.
During key selection, you should consider the
following:
•

Number of keys

•

Key data type

•

Key size

•

position of key in record

•

Key characteristics

Number of Keys

6.2.1

You can specify from 1 to 255 keys for an indexed file:
•

One primary key that RMS-11 requires for every indexed file

•

254 alternate keys

There are overhead costs in key specification: For each key specified
in an indexed file, RMS-ll builds an index.
Since RMS-11 requires a
primary key, you must accept that key's index overhead, but you should
consider the cost before specifying an alternate key for the file:
•

RMS-ll updates alternate indexes when your program:
Puts a new record into the file
Updates a record in the file and the alternate
change

key

values

Deletes an existing record
The time required for this update relates to the number of I/O
operations needed to follow each alternate index from the root
to level 0, to change or insert the SIDR, and to rewrite the
bucket.
RMS-ll can require additional time if one or more
buckets in the index split.
6-2

INDEXED FILE DESIGN
•

An index takes room in the file.
You can estimate
space for an alternate index (see Section 6.6.1).

the

disk

Whether the cost of each alternate key is bearable depends on your
application.
If the primary purpose of the application is to write,
update, or delete records, each alternate key will noticeably burden
the operations;
therefore, the number of alternate keys should be
kept to the minimum. Rarely used alternate access paths call for a
separate program that sorts by the desired nonkey field and then
processes the data.
However, if the primary purpose of your application is to get records
In fact,
from the file, then alternate keys do not burden processing.
alternate keys give flexibility to information retrieval.
However,
the cost of the extra keys is borne on those few occasions when
records are added to the file.

6.2.2

Key Data Types

Each key in an indexed file can be one of the following data types:
String
2-byte signed integer
4-byte signed integer
2-byte unsigned binary
4-byte unsigned binary
Packed decimal

6.2.2.1 String Type - RMS-11 interprets each character of the key in
a byte by its binary contents.
Permissible values are not limited to
valid ASCII codes.
Example:

The key value "RMS-11" is represented as follows:

0.1.0.1.0.0.1.0

MOST SIGNIFICANT BYTE = R

0.1.0.0.1.1.0.1

0.1.0.1.0.0.1.1

0.0.1.0.1.1.0.1
0.0.1.1.0.0.0.1

°.°.

1 .1.0.0.0.1

ZK-1191-82

The first (lowest-addressed) byte of the key is the most significant
byte of a string key for collating purposes. RMS-11 compares primary
keys byte-by-byte, first-to-last, when it determines where the record
should be placed in the file.
The maximum key value is all bits in each byte set to 1 (octal 377).
There is a cost in the number of bytes specified as the key length.
For example, if you specify a key length of 12, each representation of
the key in the data record and in the index takes 12 bytes.
6-3

INDEXED FILE DESIGN
6.2.2.2 Two-Byte Signed Integer Type - Each key requires
RMS-ll interprets the data in the following format:

bytes;

LEAST SIGNIFICANT BYTE

I:::::::

MOST SIGNIFICANT BYTE

SIGN BIT
ZK-1192-82

NOTE
The least significant byte of an integer
or binary key is the byte with the
lowest address.
Significance increases
with address. Within a byte, the lowest
significant
bit
is
bit
0,
and
significance increases with position.
See your PDP-II Processor Handbook.
A 2-byte signed integer
through +32,767.

can

represent

the

decimal

values

-32,768

Maximum key value is +32,767.
The cost in key size is 2 bytes per representation.

6.2.2.3 Four-Byte Signed Integer Type - Each key requires
RMS-II interprets the data in the following format:

7
1

LEAST SIGNIFICANT BYTE

MOST SIGNIFICANT BYTE

SIGN BIT
ZK-1193-82

6-4

bytes;

INDEXED FILE DESIGN
A 4-byte signed
integer
can
represent
-2,147,483,648 through +2,147,483,647.

the

decimal

values

Maximum key value is +2,147,483,647.
The cost in key size is 4 bytes per representation.

6.2.2.4 Two-Byte Unsigned Binary Type - Each key requires
RMS-ll interprets the data in the following format:

bytes;

LEAST SIGNIFICANT BYTE

:: :::::

MOST SIGNIFICANT BYTE
ZK-1194-82

A 2-byte unsigned binary value can
through +65,535.

represent

the

decimal

values

Maximum key value is 65,535.
The cost in key size is 2 bytes per representation.

6.2.2.5 Four-Byte Unsigned Binary Type - Each key requires
RMS-ll interprets the data in the following format:

bytes;

7
1

-'-

-.l

LEAST SIGNIFICANT BYTE

MOST SIGNIFICANT BYTE
ZK-1195-82

4-byte unsigned binary value can
through +4,294,967,295.

represent

the

Maximum key value is 4,294,967,295.
The cost in key size is 4 bytes per representation.

6-5

decimal

values

INDEXED FILE DESIGN
6.2.2.6 Packed Decimal Type - RMS-ll recognizes 2 decimal digits of
the key in each byte except the last.
The key format takes the
following form:

~___D_1____~____D_2____~1 A
D3

D4. A + 1

SIGN
ZK-1196-82

where:
A

is an address:

A, A+l, •••

are increasing (byte) addresses.

Dl-DI

are decimal digits: Dl is the most significant digit and Dl is
the least significant digit.

SIGN

has a value of 10 through 15: + is represented by
14, or 15; and
is represented by an 11 or 13.

is the length of the key in bytes (maximum of 16)

is the length of the digit string, an odd number in
of 1 through 31, where I
2N
1

10,

12,

the

range

their

Maximum key value is 99 in each byte with the sign positive.

6.2.3

Key Size

Keys for indexed files have length restrictions
data types. Table 6-1 lists these restrictions.

according

Table 6-1: Key Data Type$
Data Type

Length (bytes)

String
IS-bit signed integer
31-bit signed integer
16-bit unsigned binary
32-bit unsigned binary
packed decimal

1-255
2
4
2
4

1-16

The cost of each key's size is borne in the data record and in
index:
RMS-ll stores an entire key value in each index record.

6-6

the

INDEXED FILE DESIGN
position of Key in Record

6.2.4

You can locate any key anywhere in the record:
•

Alternate keys can precede the primary key.

•

Keys can overlap each other. Note, however, that COBOL-81, in
keeping with the ANSI standard, does not permit more than one
key to start at the same position. The standard calls this
leftmost correspondence.

You benefit from careful placement of keys within the record:
•

Deleting a record -- When you allow duplicates in the primary
key of variable-length records, RMS-ll compresses a deleted
record by removing all data except:
The record header
Enough of the record to contain the primary key
Therefore, you can optimize DELETE operations if you place the
primary key at the beginning of the record. The closer the
key is to the beginning of the record and the shorter the key,
the fewer overhead bytes remain in the file.
However, if you have fixed-length records or do not allow
primary key duplicates, the position of that key in the record
is not significant. See Section 6.2.5.1 for more information
on duplicates.

•

Writing a record
You can optimize PUT operations for
variable-length records, by placing alternate keys at the end
of the record. Then, if no valid data 1S present in an
alternate key field, you can shorten the record to exclude
that field, thus reducing the record space in the data level
as well as eliminating a reference to that record in the
alternate index.

You can segment string keys;
all other key data types must be
contiguous bytes. You can specify up to eight segments in one string
key, each segment with its own length;
the total of the lengths
cannot exceed 255 bytes. Note that some high-level languages do not
make this capability available;
see your
high-level
language
documentation.
RMS-ll concatenates the segments you specify before performing any
operations requiring a value for the key. RMS-ll defines a segment by
byte position within the record and length in bytes.
Therefore, the
key segments you define with either a MACRO-II program or RMSDES do
not have to align with the data fields you define within the records:
RMS-ll has no knowledge of the form of such files.
Example: You have an inventory application with a master product
file.
Within the product records, you have fields for vendor number,
vendor's part code, and your part number, among others.
You can
define the following keys for the file regardless of the placement of
the fields.
Primary key = vendor number + vendor part code
Alternate key 1
vendor number + your part number
Alternate key 2
vendor number
Alternate key 3
your part number

6-7

INDEXED FILE DESIGN
For cost, see the preceding considerations about the placement of keys
within a record, knowing that a key consists of all segments.

6.2.5

Key Characteristics

Key characteristics include:
•

Duplicates

•

Changes

•

Null key

Characteristics are restricted according to key number:
Characteristics

Primary Key (0)

Alternate Keys (1+)

Duplicates
Changes
Null key

Allowed
Not allowed
Not allowed

Allowed
Allowed
Allowed

The combination of changes and duplicates is also
number:

restricted

Combination

Primary Key (0)

Alternate Keys (1+)

CBG+DUP
CHG+NODUP
NOCHG+DUP
NOCHG+NODUP

Error
Error
Allowed
Allowed

Allowed
Error
Allowed
Allowed

key

NOTE
COBOL-8l
allows
the
CHG+NODUP
combination
for alternate keys.
To
enable this option, the COBOL-8l OTS
uses a hidden FIND operation to check on
duplicates each time an alternate key
value changes on an UPDATE operation
(REWRITE in COBOL-8l).

6.2.5.1 Duplicate~ - If duplicates are allowed for a key, more than
one record can have the same value in that key field.
The overhead
costs are:
space -- Duplicates have little effect on space usage as
• File
long as records are not frequently updated with changing key
values or del~ted.
If anything, records with duplicate key
values
are
stored
more efficiently than records with
nonduplicate values:
fewer
index records are required to
cover data records with duplicate primary keys.
In alternate indexes, one SIDR with one representation of the
key value is needed to cover multiple data records with the
same value in the key field.
•

Writing a record -- RMS-ll stores records with duplicate key
values for first-in, first-out access. Writing (and updating)
records containing duplicate key values takes more time as the
number of duplicates builds up.

6-8

INDEXED FILE DESIGN
A PUT operation can fail because duplicates are not allowed
for one of the keys.
If this is the primary key, RMS-ll has
wasted little time since it has performed only the I/O
operations to find the previous record with that value in the
key field.
However, if you allowed no duplicates in one of the
keys, RMS-ll:

alternate

Updates the primary index, including the data level.

Updates the preceding alternate indexes.

Discovers that it cannot insert the record
record already exists with that key value.

Reverses the actions it has taken, removing all updates
from the indexes it has already rewritten.
Entries made
in SIDR duplicate arrays are flagged as deleted and not
compressed out of existence.
However, RMS-ll cannot
reverse bucket splits.

Returns an error code.

because

a record -- If you do not allow duplicate values for
• Deleting
the primary key, RMS-ll compresses a deleted record to a
operation.
2-byte indicator when it performs a
DELETE
However,
if you allow duplicate values for the primary key,
RMS-ll keeps enough of the record to contain the entire
primary key:
1.

If the format is fixed, the entire record remains
file.

the

If the format is variable, enough of a record
place to hold the entire primary key.

remains

•

If you do not allow duplicate values for an alternate key,
RMS-ll removes the SIDR when it deletes the data record.
However, if duplicates are allowed, the pointer remains in the
SIDR array with the delete flag set.

•

Updating a record -- If you allow duplicate values for the
primary key, the length of a variable record cannot be changed
during an UPDATE operation.
In addition, updating records
containing duplicate key values takes more time as the number
of duplicates builds up.
Finally, the SIDR pointers for
deleted records are flagged as deleted, but not removed from
the duplicate array.

•

Summary -- Duplicates are not costly for write-type operations
unless there are too many of them.
pick a key field that
minimizes duplicates.
Example: Fields where there are only two choices for entries,
such as sex, are not good candidates for key fields.

6.2.5.2 Changes - The value of a primary key field cannot change
during an UPDATE operation; however, you can allow the value In any
alternate key field to change if you are willing to allow duplicate
values in that key.

6-9

INDEXED FILE DESIGN
During any UPDATE operation, RMS-ll checks the characteristics of all
keys and compares the new key values
(in the record about to be
rewritten) with the old values:
if you do not allow changes in a key
field,
but changes have been made, RMS-ll immediately terminates the
UPDATE operation with an error code.
Cost:
If an alternate key value changes during an UPDATE operation,
RMS-ll must trace the old SIDR and delete it, then insert the new one,
starting with the root of the index for both processes.
If the data
does not change, however, RMS-ll does not update the alternate index.

6.2.5.3 Null Key - You can specify the null key characteristic for
any alternate key.
If RMS-ll finds that an alternate key field is
filled with the null key value specified for that key,
it does not
insert an entry into the index for the record being written.
Zero is the null key value for the numeric key data types
(integers,
binaries, and packed decimal).
The null key character for string keys
can be any octal value (000 through 377) including an ASCII character:
if all bytes in the key field contain this value, the key is
considered null.
Cost: The use of a null key value can reduce the disk space that an
alternate index occupies, but it also precludes accessing those
records not entered in the index via that alternate key.

6.3

AREAS

You should divide an indexed file into areas.
the file that RMS-ll treats as an entity for:
•

Initial allocation

•

Extensions

•

Bucket size

•

Placement on disk

An area is a portion of

Areas allow you to gather logical elements of the file into groups of
continuous ranges of VBNs.
These VBNs can be mapped onto a contiguous
set of logical blocks on disk.
This tight sequence of VBNs is lost
when RMS-ll extends an area.
NOTE
Unless you completely allocate each area
when you create the indexed file, the
division of the file into areas may not
improve performance.
Areas can be set up for:
•

Primary index level 0 (the data records)

•

Primary index level 1 (the lowest index level)

•

Primary index levels 2 and greater

6-10

(the rest of the index)

INDEXED FILE DESIGN
•

Alternate index level 0 (SIDRs)

•

Alternate index level 1 (the lowest index level)

•

Alternate index levels 2 and greater

(the rest of the index)

Dividing a file into areas primarily saves I/O time. As explained
in
Section 5.1, in a single-area file, RMS-ll intersperses index and data
buckets:
index buckets are scattered among the data buckets.
During
each random record access,
RMS-ll consults the appropriate index
descriptor in memory and then directs (through the operating system)
the disk head to read the root and levels 2 and greater, levell, then
the appropriate level 0 bucket. These buckets can be anywhere in the
file,
and the disk head can travel large distances several times to
complete one access operation. Figure 6-1 shows an indexed file with
one area.
Figure 6-2 shows an example of a single-area indexed file.

BUCKET
NUMBER~__~____~~__~~~__~~~_K__~__- ,____~__
N__~_

ROOT
DATA

PRIMARY
INDEX

DATA

PRIMARY
INDEX

DATA

•••

ZK-1153-82

Figure 6-1: Single-Area Indexed File

ROOT
VBN 17933

~
LEVE L2

VBN 305

/'
LEVE L 1

VBN 14

~
LEVE LO

VBN 20433

VBN = VIRTUAL BLOCK NUMBER
ZK-1158-82

Figare 6-2: Example of Single-Area Indexed File

6-11

INDEXED FILE DESIGN
To randomly access a specific record in the file illustrated in Figure
6-2, RMS-l1 makes the following I/O requests:
1.

Read VBN 17933

Read VBN 305

Read VBN 14

Read VBN 20433

You can now realize how much the device has to move its read
service one random access operation.

head

A multiarea file, on the
other hand, can have all
index buckets
allocated contiguously
(if enough blocks were initially allocated):
all index information is available in one physical part of the disk.
RMS-11 can then traverse an
index with little or no head movement
until it reads the indicated data bucket.
In addition, a sequential
read of the file moves the head mechanism smoothly through the
physically contiguous area assigned to the primary index level
O.
Figure 6-3 shows an indexed file with two areas.

BUCKET
NUMBER

r-------r-----r----.---4-,-----t r----r-_ _,.-_ _,.--_N_-1_r---_N-----.,
ROOT
PRIMARY PRIMARY
OF
PRIMARY
INDEX
INDEX
PRIMARY
INDEX
INDEX
~

DATA

__________________-J' _________________
~I

AREA 0

AREA 1
ZK-1154-82

Figure 6-3: Two-Area Indexed File
To refine your file even more, place the lowest level of each index
(level 1) in an area separate from the rest of the index (levels 2 and
greater) •
Figure 6-4 shows an example of a multiarea indexed file.
To randomly access a specific record in the file illustrated in Figure
6-4, RMS-ll makes the following I/O requests:
I

Read VBN 418

Read VBN 423

Read VBN 1537

Read VBN 14703

You can now realize how much the proper use of areas reduces disk head
movement during a random access operation.

6-12

INDEXED FILE DESIGN

VBN 418

ROO T

\
VBN 423

LEV EL 2

_/AREAO

AREA 1

VBN 1537

LEV EL 1

fAR~1 AREA 2

LEV EL 0

VBN 14703

VBN = VIRTUAL BLOCK NUMBER
ZK-1161-82

Figure 6-4: Example of Multi-Area Indexed File
When you specify and preallocate multiple areas, RMS-ll arranges them
in order in the file:
area 0 (including the file prologue) in the
first virtual blocks of the file, then area 1, and so on.
If you
specify contiguity for the entire file, 'this control over the
distribution of structural elements of the file is propagated from the
virtual block sequence to the logical block sequence on the disk.
Contiguity is very important to performance.
contiguity, Section 8.3.

6.4

For more information

PLACEMENT CONTROL

Placement control enables you to specify the location on a disk for a
file or the areas of a file.
You use placement control for the
following reasons:
•

To start a file or area at the first block of a track or
cylinder so that the file or area can reside in one or more
contiguous tracks or cylinders. This effort minimizes head
movement during file access.

6-13

INDEXED FILE DESIGN
•

To place the files used by a single application together on a
disk.
This effort reduces I/O time by minimizing head
movement among the files.
Example:
You want to run a general ledger application that
uses several files (an accounts file, a transaction file, and
so on).
The application consists of several tasks.
So, you
start with an initialized disk and copy the tasks onto it.
Then, you create (and populate) your data files, placing them
near the tasks.
This effort reduces the distance the disk head mov~s to
service
I/O
operations required by an RMS-l1 program:
disk-resident overlays (discussed in Chapter 8) and data file
accesses.
Note, however, you gain more improvement if you eliminate head
contention by placing the individual files on separate disks.

You calculate track and cylinder starting block numbers as follows:
1.

Read the documentation that came with your disk drive.
and write down the following numbers:
•

Number of surfaces on a volume (or pack or disk)

•

Number of tracks on a surface

•

Number of sectors in a track

Find

NOTE
On most DIGITAL disk drives, a sector equates to a
logical block.
For example, the following decimal
numbers apply to an RP06 only:
Number of cylinders per disk = 815
Number of tracks per cylinder = 19
Number of sectors per track = 22
2.

Establish the starting logical block number
(LBN)
for each
track
on
the disk by writing down the multiples of
sectors-per-track. Since LBNs start with 0, tracks start at
multiples of track length.
Example: From the RP06 specifications, the first 10 tracks
start at LBNs:
0, 22, 44, 66, 88, 110, 132, 154, 176, 198.

Multiply sectors-per-track by tracks-per-cylinder to get
sectors-per-cylinder.
Establish the starting LBN for each
cylinder on the disk by writing down the multiples of
sectors-per-cylinder.
Example: For an RP06, the first 10 cylinders start at the
following LBNs:
0, 418, 836, 1254, 1672, 2090, 2508, 2926,
3344, 3762.

After you decide where on the disk you want to place your file, you
create the file using RMS-11 placement control.
In the process, you
place area
(which will position the whole file,
if the file is
contiguous) at the location you calculated.

If you are using a high-level language, you can specify place~ent
control by using RMSDES.
If you are programming in MACRO-II, you can
specify placement cont~ol through the use of allocation XABs.
6-14

INDEXED FILE DESIGN
6.5

BUCKET SIZE

Buckets are the units of access for indexed files.
Bucket size is
critical to the virtual address space required by a task and to the
speed with which a task performs. There is, of course, a trade-off:
the larger a bucket, the larger the task, but the faster it reads
data:
•

The speed of an RMS-ll operation is closely proportional to
the number of I/O operations involved. For indexed files, the
number of data transfers during a random retrieval operation
is approximately equal to the depth of the index (in most
cases, one more than the depth). That number includes only
the I/O operations directly related to the record operation;
other data transfers can be required to service the operation,
including overlays and system overhead (discussed in Chapter
8) •

Therefore, the larger the buckets, the shallower the index,
and the faster the random retrieval operation. Without other
considerations, you should pick the largest possible bucket.
The maximum bucket size allowed is 32 blocks.
•

The larger the bucket, the more records it can contain,
sequential access can require fewer I/O operations.

and

However, there are other considerations.
RMS-ll requires two I/O
buffers, each the size of the largest bucket, when it connects a
record access stream to an indexed file.
By making bucket size
smaller, you reduce the size of the buffers your task requires.
Depending on the record operations your program requires, that virtual
address space may be better used in overlay structure optimization
(discussed in Section 8.2).
Therefore, you should set bucket size to some lower value that still
allows good performance; a reasonable goal is an index depth of 2 or
3 (root at level 2 or 3), although very large files can require four
levels of index, in addition to the data level (level 0).
Each area can have its own bucket size, but normally
the maximum size for all buckets:
•

You should consider more than the size
(plus the 7-byte header)
when you
bucket size:

you

should

use

of your data record
calculate primary data

Records that move from one bucket to another leave a 7-byte
pointer.
Deleted records leave from 2 bytes to enough
primary key to the whole record.

hold

the

Therefore, you should consider the predominant activity in the
file:
If you intend to populate the file and then only read from
it, you do not consider activity overhead.
You must
populate the file with records In ascenning order by
primary key value (discussed in Section 6.7).
If you intend to populate the file and then insert and/or
delete a lot of records, you should allow for those
activities in your bucket size calculations.

6-15

INDEXED FILE DESIGN
6.5.1

Bucket Size for primary Index

You can calculate bucket sizes in two steps.
Step 1:
Calculate the following quantities for different bucket sizes
3, and so on):
NIRBK

((512*BKS)-15)/(PKL+BPL)

(Equation la)

NDRBK

((512*BKS)-15-AO)/(RSZ+RFO)

(Equation 1b)

(1,

where:
NIRBK

is the number of index records per level 1+ index buckets

NDRBK

is the number of data records per level 0 bucket

BKS

is the bucket size as number of blocks

PKL

is the primary key length in bytes

BPL

is the bucket pointer length:

RSZ

RFO

BPL

is 3 for pointers to the first 65,535 blocks
file

the

BPL

is 4 for pointers to
65,536 and (2**24)-1

the

blocks

numbered

between

BPL

is 5 for pointers to
2**24 and (2**32)-1

the

blocks

numbered

between

is the size of the record:
•

data size for fixed-length records

•

average record length for variable-length records

is the record format overhead:
RFO

is 7 bytes for fixed-length records

RFO

is 9 bytes for variable-length records

is activity overhead.
If any noticeable number of bucket
splits will occur (due to random record insertions or UPDATE
operations that increase record sizes), specify a value of
at least 7
(more if bucket splits will be common).
If
insertion and
deletion
activity
will
occur
often,
significantly larger values of AO may be desirable, as well
as occasional file reorganizations to reclaim space and
improve access performance.

6-16

INDEXED FILE DESIGN
When you load a file using RMS-ll bucket fill factors, you preallocate
space in each bucket for future activity.
In such a case, Equation la
becomes:
NIRBK

(FF-15)/(PKL+BPL)

and Equation Ib becomes:
NDRBK

(FF-15)/(RSZ+RFO)

where:
FF

is the appropriate fill factor in bytes and has been adjusted to
l,eave extra space in each bucket to accommodate future acti vi ty
overhead.

Step :2:
Select bucket size for
equation is true:
NIRBK**n

data

NRF/NDRBK

and

index

areas

where

the

following

(Equation 2)

where:
NRF

is the number of data records in the file

is the depth of the index

This equation portrays the exponential relationship between the number
of data records ln a file and the depth of its index.
You use the
values for NIRBK and NDRBK you calculated in step 1.
a.

Set up a grid (see the example after step 2e).

For each value of NIRBK, calculate the left side of
2, for n = 2, 3, and for very large files, 4.

For each value of NDRBK, calculate the right side of Equation
2.

Where the equation is true, that is, the left side is greater
than or equal to the right side, you have a valid combination
of bucket sizes. The bucket size used to calculate the left
side may be equal to the size used to calculate the right
side, but it does not have to be.

Equation

NOTE
You gain no advantage using different index and data
bucket sizes. RMS-ll requires two I/O buffers, both
the size of the largest bucket defined for the file.
In fact, PDP-II COBOL users must not choose different
index and data bucket sizes.
e.

Select one of the valid combinations
application's requirements.

6-17

according

your

INDEXED FILE DESIGN
NOTE
•

Equation 2 is true only for files where records
are inserted in order by ascending value of the
primary key.
See also Section 6.7.

size is a step function of index depth •
• Bucket
Therefore,
intermediate bucket sizes generally
waste address space.
For example, given a file where:
BKS
BKS

4 ~ index depth of 3
8 ~ index depth of 2

then bucket sizes of 5, 6, and 7 blocks would not
normally be used, although you might choose a bucket
size of 5 blocks if future file activity seemed
likely to increase the index depth otherwise.
Example: For a file containing 50,000 200-byte fixed-length records
with a 15-byte primary key, use the equations in steps 1 and 2 to fill
in the following grid:
•

Calculate values for NIRBK using Equation la and bucket sizes
1 through 6.
Drop the remainder; use only the integer part
of the result.

•

Calculate values for NDRBK using Equation Ib and bucket sizes
1 through 6. Drop the remainder;
use only the integer part
of the result.

•

Calculate the number of data buckets in level 0
(NRF/NDRBK)
corresponding to bucket sizes 1 through 6.
Round the result
up to the nearest integer.

Calculate NIRBK**2 for the values of NIRBK corresponding to
• bucket
sizes 1 through 6.
Round the result up to the nearest
integer.
Calculate NIRBK**3 for the values of NIRBK corresponding to
• bucket
sizes 1 through 6.
Round the result up to the nearest
integer.

BKS

123

NIRBK

112

141

169

NDRBK

NRF/NDRBK

25000

12500

7143

5556

4167

3572

NIRBK**3

19683

175616

592704

1014049

2803220

4826810

NIRBK**2

729

3136

7056

12544

19881

28561

6-18

INDEXED FILE DESIGN
•

To determine the combinations of bucket sizes where Equation 2
is true, compare the values in the NIRBK**3 row one at a time
to each of the values in the NRF/NDRBK row.
Where the
NIRBK**3 value is greater than or equal to NRF/NDRBK, a valid
bucket size combination exists.
Example: The first NIRBK**3 value is 19683.
This is less
than 25000, the first NRF/NDRBK value, but it is greater than
12500, the second NRF/NDRBK value.
Therefore, index bucket
size of 1 (from NIRBK**3 row) and data bucket size of 2 (from
the NRF/NDRBK row) is a valid combination.

•

Compare the values in the NIRBK**2 row one at a time to each
of the values in the NRF/NDRBK row. Where the NIRBK**2 value
is greater than or equal to NRF/NDRBK, a valid bucket size
combination exists.
Example: The first NIRBK**2 value is 729. This is too small
to use, as is the second value in the row.
However, the third
value is 7056. This is less than 25000 (the first NRF/NDRBK
value)
as well as the next two values, but greater than 5556,
the fourth NRF/NDRBK value. Therefore, index bucket size of 3
(from NIRBK**2 row)
and data bucket size of 4 (from the
NRF/NDRBK row) is a valid combination.

As a result of the comparisons in steps 6 and 7 above, Equation
true in the following cases:

NIRBK**3
DBKS

IBKS

lOB (bytes)

2048
2048

OBKS

IBKS

lOB (bytes)

4
3

6144
4096
4096

NIRBK**2

where:
DBKS

is the data bucket size from the NRF/NDRBK row

IBKS

is the index bucket size from the NIRBK**n rows

lOB

is the maximum I/O space required by the largest bucket size
the pair

The choice of bucket size pairs depends on what you need to optimize
most in the application:
task size or access time. After you choose,
make data and index bucket sizes equal to the larger size selected.

6.5.2

Bucket Sizes for Alternate Indexes

The selection of bucket sizes for alternate indexes follows
procedure as that of primary key bucket sizes.

6-19

the

same

INDEXED FILE DESIGN
Step 1:
The records-per-bucket equations for alternate indexes are:
NIRBK

= ((512*BKS)-15)/(AKL+BPL)

and
NDRBK

((512*BKS)-15)/(AKL+(DBPL*DF)+4+DO)

where:
AKL

is the alternate key length in bytes

DBPL

is the data bucket pointer length:

DBPL

is 4 for pointers to the first 65,535 blocks in the file

DBPL

is 5 for pointers to the blocks
and (2**24)-1

DBPL

is 6 for pointers to the blocks numbered between 2**24 and
(2**32)-1

numbered

between

65,536

is the duplicate factor:
OF

is I if you allow no duplicates

is the average number of records with the
for any key value present in the file

same

key

values

NOTE
The OF factor does not compensate enough if OF is
greater than the number of data records that fit in
a bucket. RMS-Il must then use continuation buckets
to store the records with duplicate values.
DO

is the duplicate overhead:
DO

is 0 if you allow no duplicates

is 4 if you allow duplicates

No record movement or space/deletion overhead occurs in index buckets.
Step 2:
RMS-ll cannot load buckets in alternate indexes as efficiently as in
the primary index because alternate key values inevitably fall in
random order (unless you use the RMSIFL utility described in the
RSX-IIM/M-PLUS RMS-ll utilities manual).
The ideal values resulting
from the equations in Section 6.5.1 must be reduced by a packing
efficiency factor, unless RMSIFL is used to load the file.
Studies have shown that the packing efficiency factor for alternate
keys is normally about 0.5. However, this factor applies only to the
lower levels of the index and to the data level, and not to the root.
The packing efficiency of any index's root is always 1.

6-20

INDEXED FILE DESIGN
Therefore, the index depth equation for alternate indexes is:
(PF**n) * (NIRBK**n)

NRF/NDRBK

where:
PF

is the packing efficiency factor.

Example: Using the file in the primary key example and adding a
10-byte first alternate key, allowing no duplicates, the following
grid can be filled in (NRF=50,000 since there is one SIDR per data
record) :
BKS

NIRBK

117

156

195

235

NDRBK

113

142

170

NRF/NDRBK

1811

892

592

443

354

295

0.125*NIRBK**3

6859

57067

200202

474552

926860

1622240

0.250*NIRBK**2

361

1483

3423

6084

9507

13807

The index depth
following cases:

equation

for

alternate

indexes

true

the

NIRBK,**3
DBKS

IBKS

lOB (bytes)

1024

DBKS

IBKS

lOB (bytes)

1
2

3
2

3072
2048

NIRBK**2

Do not choose a bucket size smaller than that selected for the primary
index (Section 6.5.1).

6.5.3

Program Syntax

RMS-ll requires bucket size as a whole number of blocks.
However,
some high-level language compilers require or allow you to specify the
bucket size in number of records. This syntax can lead to a different
number of records per bucket than you are counting on.
Example: A BASIC-PLUS-2 program contains the following clause
OPEN statement that creates an indexed file:
BUCKETSIZE 5%

6-21

INDEXED FILE DESIGN
The record format is fixed;
record length is 100 bytes.
makes the following calculation:
100
+ 7

bytes for the data
bytes for the record header

107
x 5

bytes for each record
records specified in a bucket

535
+15

bytes for the records in a bucket
bytes for the bucket overhead

550

bytes required to be in the bucket

The compiler

A bucket must be a whole number of blocks long, so the compiler rounds
the bucket size to 2 blocks and passes that to RMS-ll to create the
file.
However, 2 blocks contain 1024 bytes;
that leaves 1009 bytes for
record storage after the bucket overhead is subtracted. Since each
record is 107 bytes long, the buckets that were originally supposed to
contain only 5 records now can contain 9 (1009/107).
Bucket size can be set by RMSDES or by your application
depending on the capabilities of your high-level language.

6.6

program,

FILE ALLOCATION

RMS-ll requests the file control processor to allocate
file at three different points in the file's life:

blocks

complete

•

When the file is created

•

When RMS-ll must dynamically extend the file
operation

•

When you explicitly instruct RMS-ll to extend the file

The allocation of blocks to a file takes time, mainly I/O time as the
operating system performs its function.
If RMS-ll has to request an
allocation every time it requires a new bucket, this time can be a
significant factor in an application's performance, especially during
file population.
You can help optimize performance by minimizing allocation overhead in
the following areas:

6.6.1

•

Initial allocation

•

Default extension quantity

Initial Allocation

Total allocation of an
efficient.

indexed

file

6-22

when

you

create

most

INDEXED FILE DESIGN
The total allocation for a file is the sum of the prologue and the
allocations for
the different indexes that make up the file;
an
index's allocation is the sum of the allocations for all levels in the
index.
You should start with the primary level 0 and "build" each
level of each index on paper, as shown in the following steps.
1.

Calculate the number of buckets in level 0 (NBK@O):
NBK@O = NRF/NDRBK
where:
NRF

is the total number of records in the file

NDRBK

is the number of data records in a bucket in level 0
(see Section 6.5.1 for the method of determining this
value)
NOTE

The method described in Section 6.5.1 assumes that
you will put records into the file in order by
ascending primary key value.
However, if you will be
loading the file in random primary key value order,
you should divide the NDRBK value obtained using the
method described in Section 6.5.1 by 2.
You will
need twice as many data buckets.
2.

Calculate the number of buckets in level 1 (NBK@l):
NBK@l

= NBK@O/NIRBK

where:
NBK@O

is the number of buckets in
Step 1)

level

(calculated

NIRBK

is the number of index records per bucket in the index
(see Section 6.5.1 for the method of determining this
value)
NOTE

The method described in Section 6.5.1 assumes that
you will put records into the file in order by
ascending primary key value.
However, if you will be
loading the file in random primary key value ord~r,
you should divide the NIRBK value obtained using the
method described in Section 6.5.1 by 2 for every
index level but the root.
You will need twice as
many index buckets.
3.

Calculate the number of buckets in level 2:
NBK@2 = NBK@l/NIRBK

6-23

INDEXED FILE DESIGN
4.

Continue this sequence of calculations until
root level, that is:
NBK@n

you

reach

the

= 1 = NBK@(n-l)/NIRBK

where:

NBK@n

is the number of buckets in the root, which
definition

is the index depth

Calculate the allocation in blocks for each level:
AQ@O
AQ@l

NBK@O
NBK@l

AQ@n

IBKS

* DBKS
* IBKS

where:

AQ@n

is the allocation quantity in blocks for level n
level 0, 1 for level I, and so on)

DBKS

is the data bucket size in blocks

IBKS

is the index bucket size

rO for

Calculate the allocation for each alternate index as shown in
Steps 1 through 5; see Section 6.5.2, for equations.
NOTE
Alternate indexes are normally populated in random
key value order.
Therefore, you should divide the
NDRBK and NIRBK values obtained using the method
described in Section 6.5.2 by 2 except for the root
level.

The total allocation quantity for the file (ALQ) is the
of the index allocation quantities plus the prologue:

sum

ALQ = PLG + AQPK + AQAKI + ••• + AQAKn
where:
n

is the last alternate key defined for the file

The prologue of an indexed file can be from 2 to 84 blocks long.
The
size is the sum of the key descriptor blocks and the area descriptor
blocks:
•

VBN 1 describes the primary key (and contains other
information) •

•

Each key descriptor block covers up to 5 alternate keys.

•

Each area descriptor block covers up to 8 areas.

6-24

attribute

INDEXED FILE DESIGN
Finally, RMS-ll extends the prologue to an integral multiple of bucket
size if the criteria described in Section 6.5 are met.
Example: Given an indexed file of 100,000 fixed-length user data
records with the following attributes, calculate a reasonable initial
allocation size in blocks:
Data size = 200 bytes
Primary key

20-byte string;

no duplicates allowed

Alternate key = 8-byte packed decimal;

no duplicates allowed

Data bucket size = indexed bucket size = 3 blocks
Calculate the primary index first:
1.

= 0, so

NDRBK

((512*3)-15)/(200+7)

7 data records per bucket

NBK@O

NRF/NDRBK = 100000/7

14,286 buckets in level 0

NIRBK

((512*3)-15)/(20+3) = 66 index records per bucket

NBK@l

NBK@O/NIRBK

14286/66

NBK@2

NBK@l/NIRBK

217/66 = 4 buckets in level 2

217 buckets in level 1

NOTE
If the number of buckets in the level under the root
is very much less than the number of index records
that fit in a bucket, you may be able to use a
smaller bucket size without increasing the index
depth.
4.

NBK@3 = NBK@2/NIRBK = 4/66 = 1 bucket in level 3, the root

AQ@O

NBK@O*DBKS

14286*3 = 42,858 blocks in level 0

AQ@l

NBK@l*IBKS

217*3

AQ@2

NBK@2*IBKS

4*3

12 blocks in level 2

AQ@3

NBK@3*IBKS

1*3

3 blocks in level 3

AQPK

43,521 blocks in the primary index

648 blocks in level 1

Now calculate the alternate index (DF=l, DO=O):
1.

NDRBK = ((512*3)-15)/(8+(4*1)+4)
= 89 data records per bucket
NBK@O = NRF/NDRBK = 100000/89
level 0

1124*2

2,248

buckets

The doubling compensates for a packing efficiency of 0.5.

6-25

INDEXED FILE DESIGN
2.

NIRBK

«512*3)-15)/(8+3)

138 index records per bucket

NBK@l

NBK@O/NIRBK

17*2

34 buckets in level 1

NBK@2

NBK@l/NIRBK

1 bucket in level 2, the root

AQ@O

NBK@O*BKS

2248*3 = 6,744 blocks in level 0

AQ@l

NBK@l*BKS

34*3

102 blocks in level 1

AQ@2

NBK@2*IBKS

= 1*3

3 blocks in level 2

AQAK

6,849 blocks in the alternate index

Finally:
ALQ

PLG + AQPK + AQAKl = 3 + 43,518 + 6,849, or

ALQ

50,370 blocks for the whole file

This allocation can be done by RMSDES or by your application
depending on the capabilities of your high-level language.

6.6.2

program,

Default Extension Quantity

If the file cannot be totally allocated at creation time, you should
establish a reasonable default extension quantity (DEQ) to minimize
the number of (and the time spent on) file extensions.
Even if the
file is totally allocated when it is created, you should establish a
reasonable DEQ in case the file gets bigger than planned.
A good basis for calculation is the number of records that are added
to the file in a given period of time, such as a day; use the formula
for allocation quantity in Section 6.6.1.
The DEQ should equal a
multiple of the bucket size.
If you do not specify a DEQ, it defaults to zero
the file with RMSDES or a high-level language.
DEQ of zero by requesting 4 times the bucket size
file control processor each time it automatically

whether you create
RMS-ll responds to a
in blocks from the
extends the file.

The DEQ for the file can be set by RMSDES or by your application
program, depending on the capabilities of your high-level language.

6.7

POPULATION TECHNIQUES

File population entails a large burst of records written into the file
after it has been created and before it is made available for normal
processing. You can populate a file with the RMSIFL or RMSCNV utility
programs,
or
with
an
application program, depending on the
capabilities of your high-level language.
The aim of populating an RMS-ll indexed file is to avoid bucket splits
and record movement during the population and during later use of the
file.
The techniques to achieve this goal are:
•

Inserting records in ascending order by primary key

•

Use of fill numbers

6-26

INDEXED FILE DESIGN
6.7.1

Ascending Order by Primary Key

The best way to populate an indexed file is to insert the records in
ascending primary key value order.
You do not need to insert the
records all at once. This technique:
•

Minimizes population time

•

Avoids the creation of RRV records, allowing RMS-ll to fill
buckets with data records and thereby find records with the
least access time.

Contrast this technique with records loaded in descending order by
primary key value.
In that case, you introduce the packing efficiency
factor p to the primary key equations. Normally, p is 1, when you
insert records in ascending order and the factor drops out of the
equation, as shown here:
NIRBK**n
But when p

NRF/NDRBK

< 1, the equation becomes:

(p**n) (NIRBK**n)

NRF/NDRBK

Since p is a fraction, the introduction of this factor reduces the
left side of the equation, at times dramatically, thereby potentially
increasing:
•

The index depth needed to cover
records

specific

number

data

•

Frequency of bucket splitting (an important factor in the time
required to populate an indexed file)

As mentioned in Section 6.5.2, alternate indexes are a prime example
of packing inefficiency, a situation avoided only with the RMSIFL
utility. The best general approximation for p in the case of
alternate indexes is 0.5, the value used in Section 6.5.2.
You can populate a file with records in ascending order by primary key
as follows:
•

Use the RMSIFL utility.

This utility:

Sorts your input file into ascending order by the output
file's primary key, if the file is not already sorted that
way
Transfers the records from the input
file

file

the

output

RMSIFL uses techniques not available to you to further improve
the population of an indexed file.
•

Use the RMSCNV utility,
(/MA) switch.

specifying

the

mass-insertion

•

Write a MACRO-II program to populate the file and specify:

mode

In the FAB, deferred write when you open the file
In the RAB, when you connect to the
mode and sequential access mode

6-27

file:

mass-insertion

INDEXED FILE DESIGN
See the RSX-llM/M-PLUS RMS-ll
more information.

Macro

Programmer's

Guide

for

Be sure to sort your input records into ascending order by the
indexed file's primary key before you run the program.

6.7.2

Random Insertions after File Population

If you will be inserting records into an indexed file after it
populated, you should consider ways to optimize these operations:

•

If the new records to be inserted span the full
range
primary key values, you should use a bucket fill size.

•

If the inserted record~ are sorted into ascending order by
primary key value and added at the logical end-of-file, you
should use mass-insertion mode.

6.7.2.1 Bucket Fill Size - You can optimize for evenly distributed
random insertions by leaving free space in buckets during the initial
population of the file.
To do this, you specify a bucket fill size as
a set amount of bytes for each area in your file.
Normally, RMS-ll
ignores this number, but you can direct RMS-ll to obey it:
RMS-ll
then fills each bucket in the file to the level specified by the
number.
Example: Your bucket size is 2 blocks; you set the bucket fill size
to 768 bytes.
When you tell RMS-ll to honor the fill size, it only
uses 768 out of 1024 bytes in each bucket -~ the buckets are logically
three-quarters size.
You use the bucket fill size when you populate a file to improve its
performance during normal operations:
if free space is available in
every bucket in the file, any record randomly inserted into the file
is likely to fit without causing a bucket split.
The size of the bucket fill size depends on:
•

The amount of insertion activity you expect.
Allow room (including record header) for the number of records
you
will add to each bucket during normal operations.
Occasional insertions might not warrant the use of bucket fill
sizes, whereas heavy insertion can require room for multiple
additional records in each bucket to optimize, but not
eliminate, bucket splitting activity.

•

The type of bucket (data or index)

involved.

Because of the difference in record sizes and frequency of
insertion,
data and index buckets should normally have
different bucket fill sizes.
Example: The file contains 240-byte fixed-length records with a
To optimize random insertions, the
primary key field 24 bytes long.
fill size for data buckets should therefore be at most: bucket length
(15) minus record length (240) minus record
minus bucket overhead
overhead (7). This number leaves room for one data record.
This same bucket fill size for index buckets leaves room for 9 index
records.
A more reasonable bucket fill size for index buckets is:
bucket length minus bucket overhead minus 2 times 27 bytes.
This
6-28

INDEXED FILE DESIGN
number leaves room for 2 index records, where:
the primary key length
(24) plus the bucket pointer length (3) equals the index record length
(27) •
See Section 6.5.2 for a more complete discussion.
NOTE
RMS-ll ignores a bucket fill size of
less than 50 percent of the bucket
length and uses the 50 percent figure.
The bucket fill size for a file can be set by RMSDES or by your
application program, depending on the capabilities of your high-level
language.

6.7.:2.2 Mass Insertion - You use mass-insertion mode when you have
series of records to add to an indexed file and:

•

You have sorted the records into ascending order by the file's
primary key.

The lowest key value in the records is greater than the
• highest
key value in the file;
that is, the records will be
inserted at the logical end-of-file.
Whil,e the mass-insertion bit is on, RMS-ll performs
normally (see Section 5.3.1) except that it:

PUT

operation

•

Does not unlock the primary level 0 data bucket

•

Keeps a pointer to the primary level 1 bucket that pointed
the proper level 0 bucket

Thes1e extra steps enable RMS-ll to:
•

write the next record without following the primary index
the mass insertion bit is still on).

(if

•

Rapidly split the primary level 0 bucket when it is full:
since RMS-ll has a pointer to the primary level 1 bucket that
will contain the index record for the new bucket,
it can
update that bucket without following the index.

By using these techniques, RMS-ll can extend the primary level 0
bucket by bucket, packing records into the buckets in the order they
are written. As each bucket becomes full, RMS-ll creates a new one,
beginning with the next record inserted, and notes its existence in
the primary level 1 index bucket.
NOTE
Mass insertion significantly improves
performance
for
single-key
indexed
files.
The percentage of improvement
lessens with each additional key defined
in the file.
You can enhance mass insertion performance
(see Section 7.4).

6-29

using

deferred

write

CHAPTER 7
RECORD AND FILE PROCESSING OF INDEXED FILES

The
record
and
file
processing
capabilities
described
in
RSX-llM/M-PLUS RMS~ll:
An Introduction are available for indexed
files. ThlS chapter dlscusses the operatlons and their implementation
and restrictions with indexed files.

7.1

ACCESS SHARING

Access sharing can be specified for indexed files as described in the
following sections.
See Section 2.2.3 for general information on
shared access.

Record Access to Indexed Files

7.1.1.

Indexed files allow fully interlocked read/write sharing, dependent
upon the compatibility of the access and sharing declarations of
multiple accessors, as follows:

7.1.2

•

Block Access to Indexed Files

Because block access bypasses the record structure and interlocking
algorithms used with indexed files,
read/write sharing cannot be
permitted. Any read/write sharing declaration is converted internally
to read-only before the file is processed
(this is similar to
record-accessed sequential files).
Thus,. multiple read-only accessors
(regardless of their sharing
declarations) can share indexed files concurrently using block access,
as long as no read/write record accessor is present. Read-only block
accessors can share files with read-only record accessors.
In
addition, a single read/write accessor can access an indexed file
using block access (regardless of sharing declaration) as long as no
other accessor of any kind is present.

7-1

RECORD AND FILE PROCESSING OF INDEXED FILES
Other combinations are rejected:
are incompatible.

7.2

the access and sharing

declarations

RECORD AND STREAM OPERATIONS

The following record and stream operations can be performed on indexed
files.
See also the discussions of read- and write-type record
operations in Chapter 5.
CONNECT
DELETE
DISCONNECT
FIND
FLUSH
GET
PUT
REWIND
UPDATE
In all record operations, RMS-ll establishes the current record
(if
any) and next record (if applicable) context.
If any record operation
fails, RMS-ll normally sets the current record to none and does not
change the next record.
NOTE
For more information on the RMS-ll error
codes
referred to in the following
sections, see the RSX-IIM/M-PLUS RMS-ll
Macro Programmer's Guide.

7.2.1

CONNECT

A CONNECT operation affects the
follows:

context

for

the

access

•

Current record -- There is no current record.
Any
requiring a current record fails at this point.

•

Next record
according to
reference.

stream

operation

The next record is the first record in the file
the collating sequence of the specified key of

Example:
In an indexed file with multiple
record
varies by the key specified in
initiating the CONNECT operation:

keys, the next
the instruction

If the primary key is specified, the next record is the
first record in primary level 0, the one with the lowest
primary key value in the file.
If an alternate key is specified, the next record is
indicated by the first SIDR in the alternate index's level
0;
the record itself can be located anywhere in the
primary level O.

7-2

RECORD AND FILE PROCESSING OF INDEXED FILES
7.2.2

DELETE

In a DELETE operation, RMS-ll flags the header of the current record
to indicate that it is a deleted record. The prerequisite GET or FIND
operation brought the bucket containing the record into the I/O
buffer.
Then, RMS-ll writes the bucket over its original location on the disk,
unless you specified deferred write (see Section 7.4.2).
A DE[,ETE operation affects
follows:

the

context

•

Current record -- None. Any
record fails at this point.

•

Next record -- Unchanged.

for

the

stream

requiring

current

A DISCONNECT operation destroys the context for the access
You cannot resume this context by reconnecting the stream.

stream.

7.2.3

operation

access

DISCONNECT

7.2.4

FIND

Section 5.3.2 describes how RMS-ll performs a key-access
FIND
RMS-ll
performs
a
operation.
Section
5.4
describes
how
sequential-access FIND operation.
If the record does not exist or has been deleted,
error code depending on the access mode:

RMS-ll

returns

•

In sequential-access mode, the error code is ER$EOF.

•

In key-access mode, the error code is ER$RNF.

•

In RFA-access mode, the error code is:

ER$RFA -- no valid record has ever existed at the specified
location.
ER$DEL -- the record header indicates that the
deleted.

record

was

A FIND operation affects the context for the access stream as follows:
•

For a sequential-access FIND operation:
Current record -- Is set to the value of the record found.
Example: You have connected a stream to an indexed file,
specifying 0 as the key of reference. There is no current
record, but the next record is the first record in primary
level
O.
If
you execute a sequential-access FIND
operation, the current record is set to this record.
Next record -- Is set to the record logically following the
current record in the index of reference.

7-3

RECORD AND FILE PROCESSING OF INDEXED FILES
NOTE
RMS-ll enacts this logical sequence
actually accesses the next record:

only

record,

when

reading

RMS-ll locates the current
bucket if necessary.

RMS-ll locates the record logically following
the current record, reading another bucket if
necessary.

If the indexed file is shared, the actual record in
the next record position can change between the
operation that accesses the current record and the
one that finds the next record.
Example:
From the previous example, the next record is the
record in the file with the next higher primary key value.
•

For a key-access or RFA-access FIND operation:
Current record -- Is set to the record found, that is,
record identified by the RFA.

the

Next record -- Unchanged.
Example:
In
the
previous
examples,
you
did
a
sequential-access
FIND operation after connecting the
stream to the file.
You now execute an RFA-access FIND
operation.
The current record is set to the record
specified, but the next record is not changed.
Therefore,
if you do another sequential-access FIND operation, the
current record will be set to the second record in primary
level 0, not the record following the one found by RFA.
You use a FIND operation instead of a GET operation for two reasons:
1.

It is faster because the record is not moved to the user
buffer. Although the time required to move a record from one
part of memory to another is very short, do not expend it if
you do not need to.

It does not change the next record in key-access mode or
RFA-access mode.
This convention allows you to branch off
sequential processing for the purpose of updating or deleting
records, and keep your place.

You can use a FIND operation in the following ways:
•

To skip records in sequential
successive FIND operations.

access

•

To establish
operation.

for

•

To determine the existence of a
mode.

current

record

7-4

record

mode
a

DELETE
by

using

initiating
or

UPDATE

key-access

RECORD AND FILE PROCESSING OF INDEXED FILES
7.2.5

FLUSH

A FLUSH operation does not affect the context for the access stream.

7.2.6

GET

GET
Section 5.3.2 describes how RMS-ll performs a
key-access
a
RMS-ll
performs
operation.
Section
5.4
describes
how
sequential-access GET operation.
If the record does not exist or has been deleted,
error code depending on the access mode:

RMS-ll

returns

•

In sequential-access mode, the error code is ER$EOF.

•

In key-access mode, the error code is ER$RNF.

•

In RFA-access mode, the error code is:

ER$RFA -- No valid record has ever existed at the specified
location.
ER$DEL -- The record header indicates that the
deleted.

record

was

A GET operation affects the context for the access stream as follows:
Is set to the value of the record read.

•

Current record

•

Next record -- Is set :0 the record
logically
current record in the index of reference.

following

the

NOTE
RMS-ll enacts this
logical sequence
actually accesses the next record:
current

only

record,

when

reading

RMS-ll locates the
bucket if necessary.

the
RMS-ll locates the record logically following
current
record,
reading
another
bucket if
necessary.

If the indexed file is shared, the actual
record
in
the next record position can change between the
operation that accesses the current record and the one
that tinds the next record.

7.2.7

PUT

Section 5.3.1 decribes how RMS-ll performs a key-access PUT operation.

7-5

RECORD AND FILE PROCESSING OF INDEXED FILES
A PUT operation affects the context for the access stream as follows:
•

For a sequential-access PUT operation:
Current record -- None.
Any operation requiring a
record fails at this point.
Next record
Undefined.
sequential-access FIND or
not specified.

•

current

The record retrieved by a
GET operation at this point is

For a key-access PUT operation:
Current record -- None. Any operation requiring a
record fails at this point.

current

Next record -- Unchanged.

7.2.8

REWIND

A REWIND operation sets the context of the access stream to a logical
beginning of the indexed file.
In doing so, the operation affects the
context for the stream as follows:
•

Current record
None. Any
reco~d fails at this point.

•

Next record -- is set to the first record
according to the specified key of reference.

7.2.9

operation

requiring

current
the

file

UPDATE

In an UPDATE operation, RMS-ll moves the specified record from the
task's user buffer to the I/O buffer, replacing the current record set
by the previous GET or FIND operation. Then, RMS-ll writes the bucket
over its original location on the disk.
Section 5.3.3 describes the
UPDATE operation in detail.
An UPDATE operation requires a valid current record.
Therefore, an
UPDATE operation should follow a successful GET or FIND operation;
otherwise, RMS-ll returns the error code ER$CUR.
This error does not
affect the original record in the file on disk.
An UPDATE operation affects the
folows:

7.3

context

•

Current record -- None. Any
record fails at this point.

•

Next record -- Unchanged.

for

operation

the

access

requiring

stream
a

current

RECORD TRANSFER MODES

You can manipulate records either in the I/O buffer or in your
program's user buffer.
Each of these options is called a record
transfer mode.
You can change record transfer mode at run time, even
between record operations.
Figure 7-1 illustrates the RMS-ll task
st:ructure.

7-6

RECORD AND FILE PROCESSING OF INDEXED FILES

fSlZE DEPENDS 00:- - - - - - - - i
I· NUMBER OF FILES OPENED SIMULTANEOUSLY:

I. BUCKET SIZES
L NUMBE~~RECOrRD~CCE~ ~~AMS _ _

USER B UFFERS

--.

{

I/O
BUFFERS
VIRTUAL
MEMORY

PROGRAM

RMS-11

INTERNAL
CONTROL
STRUCTURES

ON:- ~- ~
~ • RMS-11 FUNCTIONS USED ~

fSlZEDEPENDS

~ ~'y~RLA '~_:)"~::'~U~TURE~~E~
ZK-1174-82

Figure 7-1: RMS-11 Task Structure

7.3.1.

Move Mode

Move mode is the default record transfer
languages and all file organizations.

7.3.2

mode

for

all

programming

•

On GET operations, RMS-11 moves the record from the I/O buffer
to the user buffer before returning control to your program.

•

On PUT and UPDATE operations, your program assembles the
record to be written into the file in the user buffer, and
during the operation, RMS-II moves the data into the I/O
buffer before updating the file.

Locate Mode

Locate mode enables your program to manipulate records in the I/O
buffer, eliminating the data transfers between it and the user buffer.
However, when you specify locate mode, RMS-l1 uses it only when such
usage does not compromise data integrity. Otherwise, RMS-ll uses move
mode. Therefore, your program must still contain a user buffer.
Example: RMS-II uses move mode instead of locate mode when an indexed
file is shared.

7-7

RECORD AND FILE PROCESSING OF INDEXED FILES
Example:
RMS-ll uses move mode instead of locate mode if you opened
indicating that you were going to perform UPDATE operations
the file
on it.
RMS-ll's use of move mode instead of locate mode is transparent to
your program as long as you use RMS-ll facilities to access the record
data.
For indexed files, your program can only perform GET operations in
locate mode.
See your high-level language documentation to determine
whether the language supports locate mode and, if it does, what the
exact programming techniques are.

7.4

I/O TECHNIQUES

You can use the following techniques to
record operations.

7.4.1

improve

the

performance

Asynchronous Record Operations

within each record access stream, your program can perform any record
operation either synchronously or asynchronously.
In synchronous
operations, RMS-ll returns control to your program after the operation
completes, either successfully or with an error.
When you execute an asynchronous operation, RMS-ll may return control
to your program before the operation is complete.
The program
continues processing while the physical transfer of data between disk
and memory is carried out.
However, you must not initiate another
record operation on that stream until the first operation ends;
otherwise, RMS-ll returns the error code ER$ACT.
See your high-level
language documentation for asynchronous techniques.

7.4.2

Deferred Write

Normally, each write-type record operation (DELETE, PUT, and UPDATE)
results in a bucket being written to disk.
This convention emphasizes
data integrity: you know that when a write-type operation ends
successfully, the file reflects that operation.
However, you can improve the performance of mass-insert sequential (by
primary key)
PUT or DELETE operations by using defer~ed write.
Basically, deferred write directs RMS-ll to write a bucket out to disk
only when RMS-ll must use the I/O buffer for some other purpose.
Example:
Your records are 114 bytes long and the bucket size is 2
blocks. During sequential write-type operations, deferred write could
cause I/O operations per bucket to drop from 9 to 1.
Deferred write offers little or no benefit to
operations or read-type operations of any mode.

7-8

random

write-type

RECORD AND FILE PROCESSING OF INDEXED FILES
NOTE
Deferred write should only be used with
mass-insert PUT operations.
Although
not
illegal,
deferred
write
is
essentially invalidated while an indexed
file is shared by multiple tasks
except when you are also using mass
insertion
mode.
In
the
non-mass-insertion,
write-shared
environment, every write-type operation
results in an I/O operation so that:

7.4.3

•

The bucket locked by the prerequisite
GET or FIND
(for UPDATE and DELETE
operations) or by the PUT operation
can be released.

•

The new data is available
other tasks or streams.

the

Multiple Buffers

When you open an indexed file,
RMS-ll normally sets
up
two
bucket-sized I/O buffers in your task's address space. RMS-ll uses
both buffers for record operations. However, you can direct RMS-ll to
use more than the two buffers.
RMS-ll uses any extra buffers to keep, or cache, index root buckets if
either. of the following is true:
•

The file is shared only by tasks with read-only access.

•

The file is not shared.

RMS-ll caches the roots as it uses them.
Therefore, only keys
specified or implicit in record operations have their index root
buckets cached:
•

During normal PUT operations, RMS-ll typically accesses all
indexes in a file.
You benefit from root caching only when
the number of extra buffers equals or exceeds the number of
indexes.

•

During mass-insertion mode PUT operations, one extra buffer
provides some benefit, regardless of sharing and number of
indexes.
If the file is not being shared, you benefit from
root caching only when you provide one more extra buffer than
indexes.

•

During GET operations, RMS-ll accesses one index
(associated
with the key of reference).
You benefit from root caching
when you provide an extra buffer for each different key you
reference.

•

During UPDATE and DELETE operations, RMS-ll accesses the
alternate indexes where a SIDR must be inserted or deleted.
You benefit from root caching when you provide an extra buffer
for each alternate index affected.

7-9

RECORD AND FILE PROCESSING OF INDEXED FILES
While root caching saves one disk read per index accessed, you may be
able to employ the address space used for the extra buffers more
profitably to optimize RMS-ll overlays (see Chapter 8).

7.4~4

Multiple Access Streams

RMS-ll allows each program to use multiple streams on an indexed file.

7.4.5

Sequentially Reading Write-Shared Files

If your task is trying to read sequentially by primary key an indexed
file
that is write-shared, you can improve performance by specifying
write-access as well.
Example:
Include in your BASIC-PLUS-2
ACCESS MODIFY and ALLOW MODIFY.

OPEN

statement

the

clauses

task will update a
record
When there is a possibility that your
(established when it opened the file), RMS-ll locks the bucket when
your task gets a record and holds the bucket in the task's I/O buffer.
If your task then gets records sequentially, RMS-ll finds them in
memory. When a record in a different bucket is specified,
RMS-ll
unlocks the previous bucket and repeats the procedure with the new
one.
However, if your task opens a file in a read-only and write-sharing
mode,
RMS-ll does not retain the lock on the buckets read;
RMS-ll
reaccesses the file for each subsequent GET operation, although it
does not start at the root and go down the index again.

7.5

FILE AND DIRECTORY OPERATIONS

The following file
indexed files:

and