Digital PDFs
Documents
Guest
Register
Log In
AA-D113A-TE
March 1980
141 pages
Original
5.9MB
view
download
Document:
VAX-11 SORT User's Guide
Order Number:
AA-D113A-TE
Revision:
000
Pages:
141
Original Filename:
OCR Text
VAX-11 SORT User's Guide Order No. AA-D113A-TE January 1979 This manual describes how to use the VAX-11 native mode SORT utility. The manual is Intended for all users. VAX-11 SORT User's Guide Order No. AA-D113A-TE SUPERSESSION/UPDATE INFORMATION: This Is a new document for this release. OPERATING SYSTEM AND VERSION: VAX/VMX V01.5 SOFTWARE VERSION: VAX/VMS V01.5 To order additional copies of this document, contact the Software Distribution Center, Digital Equipment Corporation, Maynard, Massachusetts 01754 digital equipment corporation · maynard, massachusetts First Printing, January 1979 The information in this document is subject to change without notice and should not be construed as a commitment by Digital Equipment Corporation. Digital Equipment Corporation assumes no responsibility for any errors that may appear in this document. The softwar·e described in this document is furnished under a license, and may only be used or copied it in accordance with the terms of such license. No responsibility is assumed for the use or reliability of software on equipment that is not supplied by DIGITAL or its affiliated companies. Copyright© 1979 Digital Equipment Corporation The postage-paid READER'S COMMENTS form on the last page of this document requests your critical evaluation to assist us in preparing future documentation. The following are trademarks of Digital Equipment Corporation: DEC DECnet DECsystem-10 DECSYSTEM-20 DECtape DECUS DIBOL DIGITAL FOCAL IAS MASSBUS PDP RSX UNIBUS VAX VMS 7/80-14 Contents Page Preface u Chapter 1 Introduction 1.1 1.2 1.3 1.4 Sort Types . . . . . Input and Output . . Statistics . . . . . . . Functions Supported. . 1-1 1-2 1-2 1-2 Chapter 2 Running SORT In Interactive and Batch Mode 2.1 2.2 2.3 2.4 2.5 2.6 2.7 The SORT Command . Interactive Sessions . '• 2-1 2-5 2.2.1 2.2.2 2.2.3 2.2.4 Batch Sessions. The SORT Command Description 2-6 2-7 2-12 2-12 2-11· 2-17 2.4.1 2.4.2 2.4.3 2-17 2-20 2-21 A Sample Sort. Selecting the Sort Type SORT Statistics . Samples. Command Name Qualifiers. Input-File-Specification Qualifiers Output-File-Specification Qualifiers. Specification File 2-23 2.5.1 2.5.2 2-24 2-25 Specification File Records Specification File Record Formats Setting up the Keys . Setting up the Work Files 2-31 2-35 Chapter 3 Calling SORT from User Programs 3.1 3.2 3.3 3.4 3.5 3.6 3.7 File 1/0 Interface . . . . . . Record 1/0 Interface . . . . . Programming Considerations . 3-1 3-2 3-2 3.3.1 3-4 Key Comparisons .. Subroutines (Parameters, Definitions, and Valid Re~urns) 3-5 3.4.1 3.4.2 3.4.3 3.4.4 3.4.5 3.4.6 3-6 3-9 3-11 3-12 3-13 3-14 3-15 3-17 3-20 SOR$INIT_SORT. . . . SOR$PASS_FILES . . . SOR$RELEASE_REC. SOR$SORT_MERGE .. SOR$RETURN-REC . . SOR$END_SORT . . . . Sample MACRO Program . . . . . Sample COBOL-74/VAX Program . . Sample FORTRAN IV PLUS Program iii Chapter 4 Error Conditions 4.1 4.2 4.3 Command Interpreter Error Messages. SORT ]~rror Messages . RMS E:rror Codes . . . . . . .4-2 4-2 4-8 Chapter 5 Improving SORT Efficiency 5.1 Functional Description. . . . 5.1.1 Sorting Processes . . 5.1.1.1 5.1.1.2 5.1.1.3 5.1.1.4 5-3 5-3 5-5 5-6 5-7 Internal Organization Buffer Allocation and Work Areas. Dynamic Memory Usage . . 1/0 Considerations . . . . . . . . 5-10 5-10 5-11 Tuning Procedure . . . . . . . . . . . . 5-11 5.2.1 5.2.2 5-11 5-14 5.1.2 5.1.3 5.1.4 5.1.5 5.2 Record Sort . Tag Sort . . Address Sort . . Index Sort . 5-1 5-3 User Performance Considerations . . System Manager Performance Considerations . Glossary Appendix A Octal/Hexadeclmal/Declmal Conversion Appendix B Character Set ASCII Collatlng Sequence Appendix C Data Types Appendix D Data Structures and Basic Concepts Index Figures 2-1 2-2 2-3 2-4 2-5 2-6 2-7 2-8 2-9 iv SORT's Four Sorting Processes. File Organization 1/0 Flow . . . . . Sample Sort Statistics Printout. . Interactive Session Sample #1 Interactive Session Sample #2 . . SORT Specification Form . . . Setting Up the Keys . . . . . . . Recognizing Data Types and Signed Numbers . . Specifying Work Files . . . . . . . . . . . . . 2-8 . 2-10 . 2-12 . 2-13' . 2-15 . 2-25 2-32 . 2-33 2-36 3-1 5-1 5-6 Subroutine Set Summary . . . . . . . . . . . . . . . . VAX-11 SORT 1\.rchitecture, Main Functional Components Sample Record Types . . . . . . . . . Sample Address Sort Output File. . Sample Index Sort Output File . . . VAX-11 SORT Operating Phases . . . SORT Dynamic Memory Usage. . . 5-10 2-1 2-2 2-3 5-1 SORT Command Summary . . . . . . . . . . File 1/0 Considerations. . . . . . . . . . . . . Fixed Position SORT Specification Summary . . Sorted Output File. . . . . . . . . . . . . . . . 2-11 . 2-28 . 5-5 5-2 5-3 5-4 5-5 . 3-3 . 5-2 . 5-4 . 5-6 . 5-7 . 5-9 Tables . 2-3 Commercial Engineering Publications typeset this manual using DIGITAL's TMS-11 System. 784all u Preface Intended Audience This manual is written for the full range of VAXNMS system users, from beginners to the most advanced level: system operator, applications programmer, system manager, or software developer. Emphasis is on the how to use information, and detailed descriptions of SORT internals are kept to a minimum. You can use SORT as an interactive utility (Chapter 2), or as a set of subroutines, callable from VAX-11 programming languages (Chapter 3). New users or those with simple sort requirements, can learn how to use SORT by reading Chapters 1and2. To use SORT efficiently or for more than simple sorts, read also Chapters 3 and 5. vii Structure of this Document Chapter 1 introduces the VAX-11 SORT program and describes its environment, features, and requirements, and explains user requisites. Chapter 2 explains how the SORT command is used to run VAX-11 SORT interactively or in batch mode. · Chapter 3 explains how to call SORT routines from user programs, and describes how to use subroutine parameters. Chapter 4 provides complete lists of SORT error messages and recovery procedures. Chapter 5 provides information and programming techniques for improving VAX-11 SORT efficiency. The Glossary defines terms used in this manual. Appendixes A, B and C consist of helpful programming aids such as: code conversion charts, character sets, and data types used by VAX-11 SORT. Appendix D summarizes basic concepts. Finally, page references to key terms appear in the index. Associated Documents The following documents are relevant to VAX-11 SORT users: • PDP-11 SORT Reference Manual • VAXNMS Primer • VAXNMS Summary Description • VAXNMS Command Language User's Guide • VAXNMS System Messages and Recovery Procedures Manual • Introduction to VAX-11 Record Management Service • VAX-11 Record Management Services User's Guide • VAX-11 Record Management Services Reference Manual • VAX-11 Software Installation Guide • VAXNMS System Services Reference Manual • VAX-11 Common Run-Time Procedure Library Reference Manual • VAX-11/780 Architecture Handbook • VAX-11/780 Processor Handbook • VAX-11/780 Software Handbook • VAX-11/780 Technical Summary um Symbology You will encounter the following symbols, colors, and special graphics in this manual. dollar sign The system prompt; indicates that the VAXNMS com$ mand interpreter is ready for command input. The next $ prompt indicates successful completion of command processing, and the system's readiness to accept another command. In addition, the $ must appear in the first character position of a command to be executed in an indirect command file. Return ml Indicates RETURN or ESC key entry required. Pressing this key after entering a full command line ends the command input and begins processing. When using the prompted command format, ~ or is required after each command segment. Square Brackets m [] Used in manual text to indicate qualifiers; not entered. Used in command syntax to indicate enclosed portion is optional. Braces Used in manual text to indicate input options where n Used in this manual text to indicate variable data input (typically some number value); not entered. n one in the vertical list must be selected; not entered. Underscore Indicates an entered underscore character. Hyphen Indicates line continuation. Comma Commas are entered to separate listed subqualifiers. Circumflex Represents the CTRL key on many terminals. Normally entered simultaneously with the .alphabetic character that immediately follows. For example, AC is the same as CTRL/C. " Uppercase Letters ABC Indicates command inputs that must be entered as shown. Lowercase Letters abc Used in text to describe the command syntax; not entered. Red print Indicates characters you type at the terminal. All system printouts appear in black print. Shading Used to highlight that portion of an example that is being described in text. ix Chapter 1 Introduction VAX-11 SORT rearranges and reformats records in any VAX-11 record man"'. agement service (VAX-11 RMS) file organization. SORT consists of two functional parts: a control program called the utility, and a callable subroutine package. The utility can be used in an interactive terminal session or in batch mode using the VAXNMS DIGITAL command language (DCL) SORT command. The callable subroutines are invoked by the SORT utility. Users can write control programs in most VAX-11 languages using these callable subroutines. You can invoke SORT interactively by entering a SORT command with qualifiers and input/output parameters. The command specifies one of four sort types and the sorting keys. During program execution, SORT indicates all errors. At the completion of each session, SORT prints a statistical summary. 1.1 Sort Types The four sort types (or sorting processes) are: • Record Sort • Tag Sort • Address Sort • Index Sort Record Sort produces a reordered data file sorted by specified key fields (that is, entire records are reordered). This sort uses any VAXNMS input device and can process any valid VAX-11 RMS format. Record, a relatively slow sort, is the default process. Tag Sort produces the same kind of output file as record sort by sorting only the record keys. Tag sort then randomly reaccesses the input file to create a resequenced output file according to those record keys. This method conserves temporary storage, but can only accept input files residing on disk. 1-1 Addre11 Sort produces an address file. That is, a reordered address file, on disk only, of record's file addresses (RFAs). The address file, sorted by record keys, can be used by programs as an index to read the original file in the desired sequence. This is the fastest of the four sorting processes. Index Sort produces an address file containing the key field of each data record and a pointer (RFA) to its location in the input file. The address file can be used by programs to randomly access data from the original file in the desired sequence. Like address sort, this is a high-speed process. For more information on sort types, see Chapter 2. 1.2 l·nput and Output As input, SORT accepts sequential, relative or indexed-sequential data files containing records of fixed, variable, or variable with fixed-length control (VFC) format. Character, binary, or decimal data types, and files from disk, magnetic tape, card reader or terminals are accepted. As output, SORT produces sequential, relative or indexed-sequential data files. These files can be of fixed, variable or VFC format and output to disk, magnetic tape, printer or terminal. In addition, SORT outputs address files (on disk only) for sequential access by programs. 1.3 Statistics SORT prints statistics at the end of each session. These statistics include: • Elapsed execution time • Number of records read, sorted, and output • The longest record length For more information on statistics and how they can be useful, see Section 2.2.3 and Chapter 5. 1.4 Functions Supported by VAX-11 SORT 1. Sort types: record, tag, address, index. 2. File organizations as input and output: sequential, relative, indexedsequential. All VAX-11 RMS file types are supported. 3. Record format for input and output: fixed, variable, and VFC. All VAX-11 RMS record formats are supported. 4. All VAXNMS devices are supported for input and output. 5. Multivolume support as provided by VAX-11 RMS. 1-2 Introduction 6. Callable subroutine package. VAX-11 programming languages producing native mode code are supported. Included are: VAX-11 COBOL-74 VAX-11 FORTRAN IV-PLUS VAX-11 MACRO VAX-11 BLISS 7. Controlled by command string or specification file. 8. Free field and fixed position specification file formats. 9. Data Types: • Character data is ASCII representation • Binary data is VAX representation • Packed decimal data is VAX representation • Zoned data is VAX representation • Decimal data supports: - leading separate sign leading overpunched sign trailing separate sign trailing overpunched sign 10. Ascending/descending output based on each key field. 11. Output file blocking and allocation size. 12. Sort statistics provided at completion. 13. ASCII collating sequence for character keys. 14. RSX SORT-11 utility option. Introduction 1-3 Chapter 2 Running SORT in Interactive and Batch Mode This Chapter explains how to use the SORT command to sort files interactively or in batch mode. 2.1 The SORT Command The SORT command consists of three parts: the command name, the input file specification parameter, and the output file specification parameter, in that order. Each part must be separated by one or more spaces or tabs, and is invoked by terminating with ~ when the command is entered as a continuous command string. This section describes how sorts are performed using the SORT command without the specification file qualifier. The specification file is a more sophisticated method of controlling SORT, and therefore is described later in Section 2.5. The specification file is a command qualifier and should not be confused with the file specifications for the input and output files. Format: $ ' ' 40 RT C u a 1 if i er s ~~n Put - f i 1 e - s Pe c if i cation C u a 1 i f i e rs}.. ci ci '\: u t Put - f i 1 e - s Pe c i f i c at i on Cci u a 1 i f i e r s J/([IT) O Command Name (SORT) I • SORT is the command name· that invokes the VAX-11 SORT utility. Command name qualifiers specify the sort process, describe the sorting key(s), specify the number of work files, indicate the specification file if a sort other than a standard sort is to be performed, and finally indicate whether the VAX-11 SORT utility or the RSX SORT-11 utility is to be invoked. 2-1 9 Input Fiie Specification Parameter This parameter specifies the physical location of the input file (see Appendix D for additional file specification information). Input file qualifiers define the input file attributes such as record format and file size. e Output F'lle Specification Parameter This parameter specifies the physical location of the sorted output file (see Appendix D for additional file specification information). Output file qualifiers define the output file attributes such as record format, record size, block size, file organization, allocation quantity, contiguous allocation, overlay existing file, and bucket size. The VAXNMS command interpreter will prompt you for input and output file specifications if they are not entered in the first command string. The following is an example of prompted format: $.._J...sw.o....R....,r..1.....K ... 'E ••'._'= .............................,.......=..,. , SIZE= 80) (Bfil) .....i---command with qualifiers $_Fi 1 e : R 1 0 0 SQ (8If) input-file-specification $ _ 0 u t Put : TEST • TM P (8(f) output-file-specification The following example shows how the SORT command is structured: SORT Command ;, rcommand Name f$ - '. * Command Parameters . . . . ""j'. '~ET Ke1wort input- f.l 1 e - s P e [ u t - f i. 1 e-sPec ·i=i Command Qualifiers (/keyword eyword/keyword) File Qualiliersj (/keyword/keyword/keyword) ,.___ _ _ _ _ Subquallflers J (=n) or (=(keywordfeyword)) Values (=n) Notes: 1. keywords may be truncated and are unique. 2. n indicates variable data input (typically some number value). Table 2-1 summarizes all the SORT command qualifiers, subqualifiers, and input values. The complete details on qualifiers and input values are discussed in Section 2.4. 2-2 Running SORT in Interactive and Batch Mode Table 2-1: SORT Command Summary Notation Used: • Underlined upper-case characters indicates the minimum entry required. • Brackets [ ] indicate enclosed portion is optional. • If several enclosed words are listed vertically, only one may be used. • Qualifiers, subqualifiers and values that must be specified are shown without brackets. • Braces I I indicate a selection must be made from the vertical list. • Defaults are shown in bold type. Subqualifiers and Values Command Qualifiers Notes $SORT RECORDl~ r'ROCESS~ [~gRESS INDEX /KEY= /KEY: is not required if specified in a specification file. ( [NUMBER=[l-101] ,POSITION=ll-163831 1-255 for CHARACTER data type} ,SIZE= l, 2, or 4 for BINARY data type { 1-31 for DECIMAL data type CHARACTER ] .~IN ARY ,iONED [ ,D.E.QIMAL ,PACKED_DECIMAL 1 TRAILING_SIGN is default if data type is DECIMAL. ,LEADING_SIGN [ ,TRAILING_SIG~ 1 ,,SEPARATE_SIGN [ ,OVERPUNCHED_SIGNj OVERPUNCHED_SIGN data type is DECIMAL. is default if 1 ,ASCENDING) [ ,DESCENDING )j [!WORK_FILES=[0,2-101] [!SPECIFICATION [=file-specification[) SYS$INPUT is default file name. [!RSX11] VAX-11 SORT is default. /RSXll requires PDP-11 SORT command switches. Refer to the PDP-11 SORT Reference Manual. (continued on next page) Running SORT in Interactive and Batch Mode 2-3 Table 2-1: SORT Command Summary (continued) Input File Qualifiers Subqualifiers and Values Notes input-file-specification [!.EORMAT= See Appendix D for file-specifications. DAT is default file-type. (RECORD_-8IZE=[l-16383LJ GEILE_SIZE= (1-4294967295]) Output File Qualifiers RECORD_SIZE specified. J Subqualifiers and Values output-file-specification [!.EORMAT] = is not normally FILE_SIZE is not normally specified. Notes See Appendix D for file specifications. The default output file type is the same as input file type. FIXED=[l-16383] ] (VARIABLE:=[l-16383] (CONTROLLED=[l -16383] ~ FIXED record format is default if sort process is index or address. GSIZE=[l-25fil] Used for VFC records only. Default value is 2 if CONTROLLED is specified and SIZE is not. (,BLOCK_SIZE= (18-327671 )] For magnetic tape files only. /SEQUENTIAL ] /RELATIVE [ /INDEXED_SEQUENTIAL Default is the input file organization if sort process is record or tag, otherwise /SEQUENTIAL is default. If /INDEXED_SEQUENTIAL is specified, /OVERLAY must be specified. [I,ALLOCATION=(1-4294967295 J] Required if /CONTIGUOUS is specified. The default value is determined by the number of records sorted. [!QONTIGUOUS] /NOCONTIGUOUS is default. /CONTIGUOUS is invalid /ALLOCATION is not specified. if [!Q.YERLA Y] /NOOVERLA Y is default. /OVERLAY is required if /INDEXED SEQUENTIAL output file organization is specified. [/fil.ICKET_SIZE=[l-321] Default value is the same as the input file value if the input and output file organizations are the same, otherwise default is 1. 2-4 Running SORT in Interactive and Batch Mode 2.2 Interactive Sessions To invoke SORT in interactive mode simply enter the s·ORT command. Any errors in the command are immediately reported at your terminal (see Chapter 4, Error Conditions). At the end of a successful run, SORT prints the statistics message (see Section 2.2.3). SORT accepts two kinds of command formats: a keyboard-oriented command string containing all the command qualifiers (excluding /SPECIFICATION), or a keyboard-oriented command string containing the /SPECIFICATION qualifier pointing to a specification file containing the command qualifiers. For example: $ SORT/KEY=<POSITION=l ,SIZE=10) inPut-file-sPecification outp1.1t-f i le-specification ffiJ or: $ SORT/SPECIFICATION=file-specification inPut-file-sPecification output-file-specification l8@ The use of the specification file is the more involved method and therefore explained in Section 2.5. In order to specify a sorting sequence, you must select key fields within the data itself. Remember, SORT reorders the entire file. The information provided in Section 2.6 can help you to set up the key fields (keys). You can extract key information from a file and store it in a reordered format for future use in accessing data in your original file in the order of your reordered file. In addition, the contents of your sorted file can be entire records, key fields with record pointers, or record indices relative to the position of each record within the file. Your intentions for the sorted output file usage, together with input and output file organizations, determine what sort process to use. The information provided in Section 2.2.2 can help you to choose the correct sorting process. Because SORT is designed to process all RMS file organizations, you also must consider how to direct the sorting process you have chosen, so that your output file organization will be usable on your peripheral device. The information provided in Section 2.2.2 and Table 2-2 compares file organizations and sorting processes. If your sorting task requires more than two work files, Section 2.7 can help you to set up additional work files. Most sorts will normally use the default number of work files. Finally, you must specify input and output file specifications. Appendix D reviews the standard VAX/VMS file-specification information, and file specification qualifiers are summarized in Table 2-1, and described in detail in Section 2.4. Running SORT in Interactive and Batch Mode 2-5 2.2.1 A Sample Sort Users can invoke the SORT command by simply providing the required key position and size for a single key and the file name of a single input file located on the user's default disk. The format of the command is: $ SORT/KEY=<POSITION=Cl-18383] tSIZE=C1-255J) inPut-file-sPecification output-file-specification IBEf) This means: • A record sorting process is performed on the specified input file • The input file key data type must be character • The input file must reside on the user's default disk • All the records in the input file are reordered in the output file in ascending alphabetic order • Input file type DAT is assigned, and output file type DAT is assigned • SORT assigns two work files for temporary storage • Output file organization is the same as the input file organization • Output file record format is the same as input records format • Output file bucket size is the same as input file bucket size • SORT statistics are printed at the terminal that executed the sort An abbreviated representation of the preceding command example is: $ SORT/K=<P0=1 tSI=80) INPUT OUTPUT CB:IT) Description: If you specify the key position and size, and character data type by default; this sort reads the single input file specified (on the user's default disk), sets up two work files on the user's default disk, and performs a record sort. This process creates an output file named OUTPUT.DAT having the same file organization as the input file. All the records in INPUT.DAT are reordered in ascending alphabetic order in the output file. The alphabetic order is determined by the contents of the 80-character key field (SI=80) starting in position one (PO=l) of each record. NOTE: A quick test can be run at your terminal by using SYS$0UTPUT as the output-file-specification. This technique displays the sorted output file before the sort statistics are printed. 2-6 Running SORT in Interactive and Batch Mode Finally, upon completion of the run the sort statistics are printed at the terminal that executed the sort. 2.2.2 Selecting the Sort Type SORT offers a choice of four processes: record, tag, address, and index. You specify the sort process by using the proper qualifier in the command or in the specification file code. Each process has its particular input requirements, processing methods, device requirements, and resultant output files. SORT provides four sorting techniques: • RECORD (/PROCESS=RECORD, or SORTR if specification file) Record sort produces a reordered data file sorted by _specified key fields (that is, entire records are reordered). This sort can be used on any acceptable input device, and can process any valid VAX-11 RMS format. Record, a relatively slow sort, is the default process. • TAG (/PROCESS=TAG, or SORTT if specification file) Tag sort produces the same kind of reordered data file as record sort by sorting only the record keys. This method conserves temporary storage, but can only accept input files residing on disk. Tag sort is faster than record sort, if the key size is much smaller than the record size and the file size is small so that the reaccessing process is short. • ADDRESS (/PROCESS=ADDRESS, or SORTA if specification file) Address sort produces an address file without reordering the input file. That is, a reordered address file (on disk only) of record's file addresses (RF As). The address file, sorted by record keys, can later be used as an index* to read the original file in the desired sequence. Any number of address files may be created for the same data base. A customer master file, for instance, may be referenced by either customer-number index or sales-territory index for different reports. This is the fastest of the four sorting processes. • INDEX (/PROCESS=INDEX, or SORTI if specification file) Index sort produces an address file containing the key field of each data record and a pointer (RFA) to its location in the input file. The address file can be used by programs to randomly access data from the original file in the desired sequence. Like Address sort, this is a high speed process. Figure 2-1 summarizes these options to help you determine which process is best for your sorting application. Chapter 5 provides additional information regarding sorting processes where performance considerations are important. * Not indexed by VAX-11 RMS. Running SORT in Interactive and Batch Mode 2-7 Figure 2-1: SORT's Four Sorting Processes Input Data File [~ }·~\.... default process. (Slowest Process) (Dlsk,Magtape,Cards,Termlnal) ~ of Reordered Output Data Fiie Rec.ords Entire Records. Sorted by Keys. D_ __ --~~---~-~~- Temporary Storage (Work Files) Disk only, 2-10 Files. C _ __ B A--(Disk,ANSI Magtape, Printer, Terminal) (Faster than Record sort if Key Size is smaller than record and file size is small.) Uses less temporary storage than Record sort. Input Data File A_ __ (Fastest Process) Uses minimum temporary storage. 8 _ __ c ___ D (Disk Only) Output Address Fiie of Reordered Record's Fiie Address (RFA) Records (fixed 6-byte records). RFA in binary RFA in binary RFA in binary RFA in binar (Disk Only) For !ater use to access the original File when this particular sequence is desired. Temporary Storage (Work Flies) INDEX Sort Disk only, 2-10 Flies. Output Address Fiie of Record's Fiie Address (RFA) Records (fixed 6-byte records) plus Key Fields. RF in nary + ey RFA in binary + Key _ _ _ _ _.....,.RFA in binary + Key (A fast process if Key Size is less than Record.) F-MK-00018-00 2-8 Running SORT in Interactive and Batch Mode RFA in binary + Key (Disk or ANSI Magtape) For later use to randomly access the original File in the desired sequence. Fiie 1/0 Co11slderatlons Input and output file organizations are another important factor in determining which eort type to use. Figure 2-2 shows how the 1/0 flows through SORT, and Table 2-2 list all possible 1/0 combinations and shows the default output file organb:ations. Inputs to v,,X-11 SORT can be files of sequential, relative, or indexed organization containing records of fixed, variable, or VFC format from disk, magnetic tape, card reader, or terminals. Input parameters to the sort program are either provided by RMS after processing the input file header records, or specified in the command in the form of input-file-specification qualifiers (that is, /FORMAT ... ). Outputs frc1m VAX-11 SORT are files of records reordered by key fields and are created in sequential, relative, or indexed organization. 'rhese files may contain record types of fixed, variable, or VFC format. Output files can be written to disk, m.ignetic tape, printer, or terminals. Sorted output address files of 6-byte RF As in binary coded records are output to disk only for sequential access by programs. These output address files are intended {iJr software use as indices into input files, and cannot be output to printers or terminals without further processing. Output parameters to the sort program are specified in the command in the form of output-file-specification qualifiers (that is, /FORMAT ... ). Running SORT in Interactive and Batch Mode 2-9 Figure 2-2: File Organization 1/0 Flow INPUT OUTPUT SEQUENTIAL DATA FILE SEQUENTIAL DATA FILE SORT SEQUENTIAL ADDRESS FILE RELATIVE DATA FILE RELATIVE DATA FILE INDEXED-SEQUENTIAL DATA FILE SEQUENTIAL ADDRESS 'FILE INDEXED DATA FILE SEQUENTIAL ADDRESS FILE NOTES: 1. RECORD & TAG produce reordered data files of the same organization as input by default. 2. INDEX produces reordered address files of RFA's plus keys in sequential file organization. 3. ADDRESS produces reordered address files of RFA's only in sequential file organization. F-MK-00019-00 2-10 Running SORT in Interactive and Batch Mode Table 2-2: File 1/0 Considerations Type of Input File Sequential Data File Relative Data File IndexedSequential Data File Sort Process Output File Organization Specified Results Record Sequential Relative Indexed-Seq Tag Sequential } Relative Indexed-Seq Same as for record. Address Sequential } Relative Indexed-Seq Sequential address file of RFAs. Index Sequential } Relative Indexed-Seq Sequential address file of RFAs with keys. Record Sequential Relative Indexed-Seq Tag Sequential Relative Indexed-Seq Address Sequential Relative Indexed-Seq Index Sequential Relative Indexed-Seq Record Sequential Relative Indexed-Seq Tag Sequential Relative Indexed-Seq Reordered sequential data file. Reordered file of data records. Populates (overlays) an already existing IndexedSequentiai output file with reordered data records. Same as above for each process. Same as above for each process. Sequential--~ Address Relative Indexed-Seq ~____, Index Sequential Relative Indexed-Seq ....,.,.., .. _ Note: The default output file organization is shown in italic type. Running SORT in Interactive and Batch Mode 2-11 . 2.2.3 SORT Statistics Statistics are automatically printed at the completion of each sort session. These consist of: elapsed execution time, the number of records read, sorted, and output; the longest record length; the multiblock count used and the multibuffer count used for input and output; the merge order; the number of merge passes; the working set size used; the number of initial runs; and the virtual memory used for the sort tree. In addition, SORT statistics include statistics kept by VAX/VMS for the number of buffered and direct 1/0 operations, CPU time, and the number of page faults. Figure 2-3 illustrates a typical SORT statistics printout of a single sequential input file (filename RlOOSQ.DAT) that is 10,000 records in length, and each record is 80 characters long. The sorting is done on an 80 character key starting at position 1 of each record. The output filename is TEST.TMP and is output in the same format as the input file by default. The command string that caused the sample printout in Figure 2-3 was: SORT/KEY=CPOS=1,SIZE=80) C8fll $_File: R100SQ (8fll $_01.ttP1.tt:TEST.TMP ~ $ The statistics can be used to help tune the parameters you specify for a specific sort, such as the best working set quota size to use (see Chapter 5, Section 5.2.2.5). Figure 2-3: Sample Sort Statistics Printout SORT STATISTICS: RECORDS READ: 10000 RECORDS SORTED: 10000 RECORDS OUTPUT: 10000 MM{ I MUM WORK I NG SET USED: 128 0 VIRTUAL MEMORY ADDED: 2380328 DIRECT IO COUNT: 227 BUFFERED IO COUNT: 23 PAGE FAULTS: 15598 ELAPSED TIME: 00: 02: 28. 97 C) LONGEST RECORD LENGTH: 80 INPUT MULTI BLOCK COUNT: 11 OUTPUT MULTI BLOCK COUNT: 20 INPUT MULTI BUFFER COUNT: 2 OUTPUT MULTI BUFFER COUNT: 2 NUMBER OF INITIAL RUNS: 38 ORDER OF THE MERGE: 7 NUMBER OF MERGE PASSES: 2 CPU TI ME: 5055 8 $ Notes: O Maximum working set used is in blocks. fJ Virtual memory added is in bytes. C) Elapsed time is the total sort run time from start to end in hrs: min: sec. 1/100secs. 8 CPU time is the data processing time less 1/0 time in 1/1 OOsecs. (that is, 5055 is 50 seconds and 55/100th's seconds.) 2.2.4 Samples Figure 2-4 shows a step-by-step session for an interactive sort on a single key. Figure 2-5 shows how an interactive sort would appear when sorting on two keys. 2-12 Running SORT in Interactive and Batch Mode Figure 2-4: Interactive Session Sample #1 Step 1: Observe the Input file you want to sort to determine where the key fields are located and their size. To sort this Input file named BOATS.LST In alphabetic order by manufacturer you must specify a single key field starting at character position 2 and having a key field size of 1O characters. MANUFACTURER MODEL RIG LENGTH NORTHERN CHALLENGER OLYMPIC EASTWARD AMERICAN LINDSEY WINDPOWER CAPE DOf~Y VENTURE SALT AMERICAN HUNTER TANZER ALBIN GRAMPIAN CHRIS-CRAF ISLANDER COLUMBIA 37 al ADIJENTURE HO 28-MS 39 IMPULSE TYPHOON 222 19 28 27 28 BALLAD KETCH KETCH KETCH M/S M/S M/S SLOOP SLOOP SLOOP SLOOP SLOOP SLOOP SLOOP SLOOP SLOOP SLOOP SLOOP SLOOP 37 Z-3a CARIBBEAN 36 41 a1 az 2a 28 39 16 18 zz 25 28 27 28 30 3a 35 36 41 WEIGHT BEAM PRICE 1a1000 281700 2a12so 7 1000 5t500 1a15oo 650 1 t 900 21000 21600 a1000 G t500 6t800 71276 11 t800 18t000 13t450 20 t 700 $5(11000 $511228 $80 I 500 $15 1900 $18t895 $351800 $3t500 $4t285 $3t56ll $Gt580 $91895 $1a1sss $17t500 $27t500 $28t675 $37t850 $31 t730 $ll8 tll90 11 13 13 09 08 12 07 08 07 07 08 08 10 10 10 11 11 11 7 /KEY =(POSITION=2,SIZE= 10) Step 2: Enter the following SORT command to sort the input file named BOATS. LST and create an output flle named BOATS.ALB: $ SORT/KEY=<POS=2tSIZE=10) BOATS+LST BOATS.ALB (5£!) Running SORT in Interactive and Batch Mode 2-13 Step 3: Observe this printout when SORT has completed. SORT STATISTICS: RECORDS READ: 18 RECORDS SORTED: 18 RECORDS OUTPUT: 18 MAXIMUM WORKING SET USED: 200 VIRTUAL MEMORY ADDED: 404982 DIRECT IO COUNT: 2 BUFFERED I 0 COUNT: 17 PAGE FAUL TS: 146 ELAPSED TIME: OO:OO:Oi.87 LONGEST RECORD LENGTH: 57 INPUT MULTI BLOCK COUNT: 20 OUTPUT MULTI BLOCK COUNT: 32 INPUT MULTI BUFFER COUNT: 2 OUTPUT MULTI BUFFER COUNT: 2 NUMBER OF INITIAL RUNS: 0 ORDER OF THE MERGE: 0 NUMBER OF MERGE PASSES: 0 CPU TIME: 54 $ Step 4: Examine your newly sorted output file named BOATS.ALB. Notice that the records are now in alphabetical order. 2-14 MANUFACTURER MODEL RIG LENGTH ALBIN AMER I CAN AMERICAN CAPE DORY CHALL ENGEi~ CHRIS-CRAF COLUMBIA EASTWARD GRAMPIAN HUNTER ISLANDER LI ND SEY NORTHERN OLYMPIC SALT TANZER VENTURE WIND POWER BALLAD 26 26-MS TYPHOON 41 CARIBBEAN ll 1 HO 2-34 27 38 38 37 ADVENTURE 18 28 222 IMPULSE SLOOP SLOOP M/S SLOOP KETCH SLOOP SLOOP M/S SLOOP SLOOP SLOOP MIS KETCH KETCH SLOOP SLOOP SLOOP SLOOP 30 26 26 18 ll 1 35 41 2l! 3l! 27 36 38 37 42 25 28 22 18 Running SORT in Interactive and Batch Mode WEIGHT BEAM 7t27G t.!1000 5,500 1 ,900 261700 181000 20,700 71000 111800 8t500 131l!50 1l! t500 1l!1000 2llt250 21600 6, 800 21000 650 10 08 08 06 13 11 11 08 10 09 11 12 11 13 07 1(I 07 07 Pl~ I CE $27 1500 $81885 $181885 $41285 $51 1228 $371850 $l!8 1ll80 $15t800 $2B1G75 $ ltJ 1888 $31 ,730 $351800 $501000 $801500 $81580 $171500 $3 15Gl! $31500 Figure 2-5: · Interactive Session Sample #2 Step 1: Observe the Input flle you want to sort to determine where the key fields are located and their size. To sort this Input file named BOATS.LST In ASCII alphanumeric order first by beam, and then by price, you must specify two keys. The first key (or primary key) field starts at character position 47 and has a size of 2. The second key starts at character position 51 and has a size of 7. MANUFACTURER MODEL RIG LENGTH NORTHERN CHALLENGER OLYMPIC EASTWARD AMERICAN LINDSEY WINDPOWER CAPE DORY VENTURE SALT AMERICAN HUNTER TANZER ALBIN GRAMPIAN CHRIS-CRAF ISLANDER COLUMBIA 37 ll 1 ADlJENTURE HO 28-MS 39 IMPULSE TYPHOON 222 19 28 27 28 BALLAD 2-3ll CARIBBEAN 36 ll1 KETCH KETCH KETCH M/S MIS MIS SLOOP SLOOP SLOOP SLOOP SLOOP SLOOP SLOOP SLOOP SLOOP SLOOP SLOOP SLOOP 37 ll 1 ll2 2ll 28 39 18 18 22 ·25 28 27 28 30' 3ll 35 38 ll1 WEIGHT BEAM PRICE 1llt000 2Gt700 2llt250 7 , 000 5t500 1llt500 850 1 t800 ZtOOO 2 t GOO lltOOO Gt500 Gt800 7t278 11 tBOO 18t000 13tll50 20t700 $50t000 $51 t228 $80t500 $15,900 $18t895 $35t800 $3t500 $llt285 $3t5Gll $6t580 $8t885 $1llt889 $17t500 $27t500 $29t875 $37t850 $31 t730 $ll8tll90 11 13 13 09 08 12 07 OB 07 07 08 09 10 10 10 11 11 11 /KEY=(POS=47,~2) T /KEY=(P0S=51,SIZE=7) Step 2: Enter the following SORT command to sort the input file named BOATS.LST and create an output file named BOATS.BEM: $ SORT /KEY= ( POS=tn 1SIZE=2) /KEY= ( POS=51 t!3I:ZE=7) BOATS+LST BOATS+BEM (Bffl Running SORT in Interactive and Batch Mode 2-15 Step 3: Observe this printout when SORT has completed. SORT STATISTICS: RECORDS READ: 18 RECORDS SORTED: 18 RECORDS OUTPUT: 18 MAXIMUM WORKING SET USED: 200 VIRTUAL MEMORY ADDED: 202240 DIRECT ID COUNT: 2 BUFFERED IO COUNT: 17 PAGE FAUL TS: 142 ELAPSED TIME: 00:00:01.85 LONGEST RECORD LENGTH: 57 INPUT MULTI BLOCK COUNT: 20 OUTPUT MULTI BLOCK COUNT: 17 INPUT MULTI BUFFER COUNT: 2 OUTPUT MULTI BUFFER COUNT: 2 NUMBER OF INITIAL RUNS: 0 ORDER OF THE MERGE: 0 NUMBER OF MERGE PASSES: 0 CPU TIME: as $ Step 4: Examine your newly sorted output file named BOATS.BEM. Notice that the records are now In order first by beam width, and second by price. MANUFACTURER MODEL RIG LENGTH CAPE DORY WINO POWER VENTURE SALT AMERICAN AMERICAN HUNTER EASTWARD TANZER ALBIN GRAMPIAN ISLANDER CHRIS-CRAF TYPHOON IMPULSE 222 18 26 28-MS 27 HO 28 BALLAD 2-3ll 36 CARIBBEAN ll1 37 38 41 ADVENTURE SLOOP SLOOP SLOOP SLOOP SLOOP M/S SLOOP MIS SLOOP SLOOP SLOOP SLOOP SLOOP SLOOP KETCH M/S KETCH KETCH 18 18 22 25 26 26 27 24 28 30 3ll 38 35 ll1 37 39 ll1 ll2 COLUM~IA NORTHERN LINDSEY CHALLENGER OLYMPIC 2-16 Running SORT in Interactive and Batch Mode WEIGHT BEAM PRICE 1 ,900 850 2,000 2,900 $4,285 $3,500 $3,584 $8,580 $8,885 $18,885 $14,999 $15,800 $17 ,500 $27,SOO $29,675 $31 ,730 $37,850 $ll8,ll90 $50t000 $35,800 $51 ,220 $8(1,500 a,ooo 5,500 9,500 7,000 s,soo 1,218 11 , 800 13,450 10,000 20,700 1a,ooo 1a,500 28,700 za,250 08 07 07 07 08 08 09 09 10 10 10 11 11 11 11 12 13 13 2.3 Batch Sessions To run the same sort as shown in Figure 2-4 using batch mode, perform the following steps: Step 1: Create a command file named BOATSl.COM as follows: $ $ $ PRINT BOATS.LST SORT/KEY=<POS=2tSIZE=10) BOATS.LST BOATS.ALB PRINT BOATS.ALB Step 2: Enter this command: $ SUBMIT BOATS1.COM filj) Observe this response: Job n entered on queue SYS$6ATCH Step 3: Observe that the input file, output file, sort statistics, and batch statistics are all printed on the system printer. 2.4 The SORT Command Description NOTE: Review the SYMBOLOGY in the front of this manual before continuing. Format: $ SORT[qualifiersJ inPut-file-sPecification[qualifiersJ outPut-file-sPecification[qualifiersJ ~ 2.4.1 Command Name Quallflers Abbreviated Example: $ s 0 Rrl/Jp::~::a::e::fi::~::s::;::ri::z::k:iiNt:!B\fJ%M\\~fdBJiliX:x~::li.~~ltt~il1S\:~::~:;·~~:i::D1m1::ttN[\f:i;"rt1 inPut-file-sPecification[qualifiersJ outPut-file-sPecification[qualifiersJ /PROCESS= ~ r~g::J ~NDEX ~ Indicates the type of sort to be performed. /PROCESS=RECORD is the default. Running SORT in Interactive and Batch Mode 2-17 /KEY= This qualifier must be specified unless defined in a specification file.· It defines a sorting key, and may appear several times in a single command string in order to specify several sort keys (up to 10). NOTE: ThE~ /KEY subqualifiers group must be enclosed in parentheses. ( (NUMBER=n] n specifies the precedence of the sort key being defined, where 1 is the primary sort key, 2 is the secondary sort key, and so on. If this option is not specified on the first /KEY qualifier, NUMBER=! is assumed. If this option is not specified on subsequent /KEY qualifiers, the default NUMBER value is the NUMBER value of the previous key plus 1. Legal values are 1 - 10. ,POSITION=n n specifies the position of the key within each record, where the first character of the record is 1. This subqualifier input must be specified. ,SIZE=n n specifies the length of the sort key in either characters, bytes, or digits, depending on the key field data type. This subqualifier input must be specified. If the sort key data type is CHARACTER, key size must be less than or equal to 255 characters. If the data type is binary, key size must be 1, 2, or 4 bytes. If the data type is any of the decimal types, key size must be less than or equal to 31 digits. The total of all key field sizes must be less than or equal to 255 bytes. See Section 2.6 for additional key size information. ,CHARACTER ,BINARY ,ZONED ,DECIMAL ,PACKED_DECIMAL ThiB subqualifier indicates the type of data appearing in the sort key field. See Section 2.6 for data type descriptions. CHARACTER is the default. 2-18 Running SORT in Interactive and Batch Mode l f,LEADING_SIGN LTRAILING_SIGNJ This subqualifier indicates whether the sign of a decimal data type key appears at the beginning or end of the key. If the key data type is DECIMAL and this option is not specifed, TRAILING_SIGN is the default. See Section 2.6 for key descriptions. OVERPUNCHED_SIGNJ [,SEPARATE_SIGN This subqualifier indicates whether the sign of a decimal data type key is superimposed on the decimal value or is separate from the decimal value. If the key data type is DECIMAL and this option is not specified, OVERPUNCHED_SIGN is the default. See Section 2.6 for key descriptions. f,ASCENDING 6DESCENDING J Indicates whether the key is to be sorted into ascending or descending order. ASCENDING is the default value. [ /WORK_FILES=n] n specifies the number of temporary work files to be used during the sort. Values of 0, or from 2 to 10 may be used. Default value is 2. 0 specifies no work files because data will fit in real memory. See Section 2. 7 for additional information. J] [ /SPECIFICATION[=file-specification Specifies the name of a file which contains SORT specification statements. If this qualifier is not specified, a standard sort is performed. See Section 2.5 and Appendix D for additional information. SYS$INPUT is the default value. [ /RSXll] Indicates that SORT-11 (/RSXll) is to be invoked. The SORT-11 command format and switches are not described in this manual. Refer to the PDP-11 SORT Reference Manual when using the /RSXll qualifier. VAX-11 SORT is the default value. Running SORT in Interactive and Batch Mode 2-19 NOTE: Only the minimal unique abbreviated form of qualifier and parameter inputs are required, but all four character abbreviations are accepted (for example, enter SPE= for spgc1FICATION=SYS$INPUT). An abbreviated example is: $ SORT/KEY=CNUM=1,POS=12tSIZE=2,QECI)/SPE= inPut-file-sPecification[qualifiersl outPut-file-sPecification[qualifiersl ~ The actual example including defaults is: $SORT/PROCESS=RECORD/KEY=< NUMBER= 1 ,posITION=12tSIZE=2,QECIMALtTRAILING_SIGN,OVERPUNCHED_SIGN,ASCENDING)/WORK_FILES=2/SPECIFICATION=SYS$INPUT inPut-file-sPecification[qualifiersJ outPut-file-sPecification[c:iualifiersl (Bf[) 2.4.2 Input-Fiie-Specification Quallflers Defines input file attributes. Format: $ SORTCc:iualifiersJ inPut-file-sPecificationUWSiO.!~iMAJhi:::~}R!t;:O:iJJ~:(l:fil§:jJ:j(g:E::#,fo@mt:::tJL§kH~tUG)i#A\i}]J outPut-file-sPecification[c:iualifiers] ~ NOTE: If the input file name does not contain a file type, the default file type becomes DAT. If only one FORMAT subqualifier is specified, the parentheses () can be omitted. [ /FORMAT=(RECORD_SIZE=n J This input should be used only to override the record size input normally · retrieved from RMS. Omitting RECORD_SIZE indicates that the file record format is to be obtained from the file header or label. n specifies the longest record length (LRL) in bytes. The LRL input is optional, but should be specified if the input file is not on disk or is inaccurate. The longest record length allowed is 16,383 bytes (not including control bytes). For additional information on determining the LRL, refer to the $FAB MRS parameter in the VAX-11 Record Management Services Reference Manual. Note, stream format is not supported because VAX-11 RMS does not support it. 2-20 Running SORT in Interactive and Batch Mode ~FILE_SIZE=n)] This input should only be used to supply the file size normally provided by RMS when the input file is not on disk. This input is used to determine the size of the work files based on input file size. n specifies the input file size in blocks. Default is 1000 if file size cannot be obtained from RMS and is not specified by the user. Maximum file size is 4,294,967 ,295 blocks. 2.4.3 Output-File-Specification Quallflers Defines output file attributes. Format: $ SORT[qualifiersJ inPut-file-sPecification[qualifiersJ outPut-file-sPecification- !~lllllillllWPdDllli~\~l~'l'~'~~~IBllMlllllllllllll NOTE: If the output file name does not contain a file type, the output file type becomes the same as the input file type. If only one FORMAT option is specified, parentheses() may be omitted. [/FORMAT=] ~ (FIXED=n l (VARIABLE=n (CONTROLLED=n Indicates the output file record format. n specifies the longest record length (LRL) of the output records in bytes, and is optional. The longest record length allowed is 16,383 bytes (less any control bytes). Default is input file record format if record or tag sort, and FIXED if index or address sort. For additional information on determining the LRL, refer to the $FAB MRS parameter in the VAX-11 Record Management Services Reference Manual. [,SIZE=n] This input applies to CONTROLLED records only. That is, variable with fixed-length controlled (VFC) records. n specifies the size in bytes of the fixed portion of controlled records. Maximum fixed control area size is 255 bytes. If CONTROLLED is specified, and SIZE is not, default is two bytes. Running SORT in Interactive and Batch Mode 2-21 GBLOCILSIZE=n )] This input applies to magnetic tape files only. n specifies the block length in bytes of the output file. Default value is the block size of the input tape file, or that which was established at tape mounting time. Block length must be in the range of 18 to 65,535 bytes. NOTE: To ensure for correct data interchange with other DIGITAL systems, you should specify a block size less than or equal to 512 bytes. To ensure compatibilty with most non-DIGITAL systems, the block size should be less than or equal to 2048 bytes. l /SEQUENTIAL /RELATIVE [ /INDEXED-SEQUENTIAL Indicates the organization of the output file. If /INDEXED_SEQUENTIAL is specified, the output file must already exist and must be empty; therefore, /OVERLAY must be specified. Default is the input file organization if a record or tag sort (/PROCESS= RECORD or /PROCESS=TAG) is performed. Otherwise, /SEQUENTIAL is default. [ /ALLOCATION=n] n specifies the number of 512-byte blocks of disk space to be allocated for the output file. The default value is whatever the output requires based on the number of records sorted. Blocks allocated must be in the range of 1 to 4,294,967, 295. [/CONTIGUOUS] Indicates contiguous allocation of blocks for output file. Default is /NOCONTIGUOUS. This qualifier is invalid if /ALLOCATION is not specified, or if /ALLOCATION value is insufficient for total output and the file must be extended. [/OVERLAY] /OVERLAY indicates that an existing file which has the same name as the output file should be overwritten with the SORT output. /OVERLAY requires that the existing file must be empty. Default is /NOOVERLAY. [ /BUCKET._SIZE=n] n specifies the RMS bucket size (that is, the number of 512-byte blocks per bucket) for the output file. If the output file has the same organization as the input file, the default value is the same as input file bucket size. The maximum number of blocks per bucket is 32. If the output file organization is different from the input file organization, the default value is 1. 2-22 Running SORT in Interactive and Batch Mode 2.5 Specification Fiie Use of the /SPECIFICATION qualifier in the SORT command allows SORT to be controlled by SORT specification statements. These statements are contained in the header record and field records of a specification file, and provide a means for expanding the range of sorting features. Having sort processes controlled by specification files enables dynamic program control of specification file statements, and therefore dynamic control of subsequent sort processes using the same specification file modified. Also, specification file libraries can be maintained for often-used sorts. The command string for a typical standard sort using a specification . file would look like this: $ SORT/SPECIFICATION[=specification file] input-file-specification output-file-specification ~~ There are eeveral methods of entering the /SPECIFICATION qualifier. If you allow the predetermined specification file statements to control the sort, SORT will run automatically with no further operator prompts. Example: $ SORT/SPECIFICATION[=file-sPecification of the Predetermined specification file] inPut-file-sPecification output-file-specification CB~ However, if you use the default specification file (that is, =SYS$INPUT, and providing your terminal is set to be the input device), SORT will prompt you for the specification file values. Example: $ SORT/SPECIFICATION inPut-file-sPecification OUtPUt-file-SPecification (BTI) PLEASE ENTER SPECIFICATION FILE RECORDS. _enter the specification file header record 1.iali.tf?~5 IBID -enter the specification file field record values for the 1st ~\e'/ field CBtf) _enter the specification file t he l a '::i t K e '/ f i e 1d field record values far (QIB~I] Running SORT in Interactive and Batch Mode 2-23 If you allow SORT to prompt you for the input and output files as well as the specification file values, then the SORT command will be input in the following sequence: SORT/SPECIFICATION ~ _file: input-file-specification ~ _OUTPUT: output-file-specification ~ PLEASE ENTER SPECIFICATION FILE RECORDS. -enter the specification file header record 1.1alues ~ -enter the specification file field record values for the 1st ~\e)' field~ $ -enter the specification file field record values for the last ~\e)' field@~ 2.5.1 Specification Fiie Records The specification file records can have either of two formats; fixed position field format (SORT-11), and logical position field format (VAX--11 SORT). NOTE: Since omit/include and alternate collating sequences are not supported, ALTSEQ records and record type records are invalid and cause errors. Only header and field specification records are processed. Fixed Position Fleld Format (SORT-11) In order to allow ease of conversion from SORT-11 V2 use to VAX-11 SORT, the existing fixed-position-fields format of SORT-11 is accepted. Free Fleld Format (VAX-11 SORT) To allow some flexibility for new users, fields may be separated by commas, and records may be variable length up to 132 characters. Blanks are ignored unless they are embedded within a field, such as 1 00, in which case an error is generated. Continuation lines are supported as in DCL. The individual fields, their length, meaning and order are identical to SORT-11. Comments in this format are placed at the end of the line by placing an exclamation point ( ! ) immediately before the comment. The format for a VAX-11 SORT header record would look like: Pase nu1r1bertline nu1r1ber,H,sort t)'Pettotal ~\e)' field size, sort ins order ,col la.tins sec:iuence ,output ~\e)' trecord lensth ! c o lrlfrl e n t 2-24 Running SORT in Interactive and Batch Mode A specific example would be: 1 t1 tHt t10tAt t}-{t132 !header for ordert KeY field size 10. record sort ascend ins - 2.5.2 Specification Fiie Record Formats Two record types and two record formats exist for specification files. The two record types are header records and field specification records. Each record type may contain either fixed position fields to support SORT-11 compatible files, or free fields for VAX-11 SORT. A DIGITAL SORT specification form is available for use when setting up fixed position fields (see Figure 2-6). 2.5.2.1 For SORT-11 Type Flies (fixed position flelds) - Figure 2-6: SORT Specification Form I SORT SPECIFICATIONS D11e _ __ Paye HEADER SPECIFICATION Programmer-------·---- :c ! >- ,_ SORTR SORTT SORTA SORTI P•og•am ldent1ficat1011 _ -· _____ --·· _. ---.-------------- SORTR,T Mode of Processin Lone o l [J I] !~ ~6. .1]_ 78 7! l!O 2 ITT To111 Leng1h of Key Fields 2 j NOT USED Comments (Program 1ft'cntif1ca11onl Comments L1 FIELD SPECIFICATION .-------...---..---------- ·--- .----.--........... ----·--------~ Forced Lme u. ~~ ~za Field Location ~ ~~i ~ ! §1--Fr-om·~-To----1~ ! 8 I-----, Field Nilme Comments I Running SORT in Interactive and Batch Mode 2-25 The format of each type of SORT-11 specification record is fixed. The SORT specification form is based on card columns, as shown in Figure 2-6. The following entries are common to both types of specification file lines. Column Notes Entry 1-2 Page number Required only when different types of records are to be described. A separate page, numbered in ascending sequence, should be used for each record type and its corresponding Field Spe~ifications. Only the first page has a header specification. 3-5 Line number Specifies line sequence. If column 5 is blank, 0 is assumed. Thus a digit entry in this column can be used to identify later line insertions. 6 Specification Type H for Header, or F for Field In the following material, unless otherwise stated, these criteria apply: • Numeric data is decimal. • Either leading zeroes or leading blanks are acceptable in right-justified entries. • All field position definition records begin at column 1. Table 2-3 summarizes all fixed position SORT specification entries. Header Record: The first record in a specification file must be the header. The header tells the SORT program what kind of sorting process to use, key field size, sorting order, and output record size. Format: 2-26 Field Position Function Legal Values 1-2 Page number Any number or blanks 3-5 Line number Any number or blanks 6 Header record ID H 7-12 Sorting process SORT R,l,A, T or blanks 13-17 Total key field size Any number or blanks 18 Sorting order A, D, or blank 19-25 (not used) Blanks (anything-ignored) 26 (not used) Blank (anything-ignored) 27 (not used) Blank (anything-ignored) 28 (not used) Anything-ignored 29-32 Output record length Any number or blanks 33-132 (not used by SORT, may be used for comments) Anything-ignored Running SORT in Interactive and Batch Mode Notes and Comments on Header Specification Entries: Columns Explanations and Legal Entries 7-12 Type of SORT (must be left-justified) Legal values: SORTR or blanks - Record sort SORTT - Tag sort SORTA - Address sort SORT! - Index sort 13-17 Total of all key field sizes Legal values: 1-255 Must be equal to the total size in bytes of the largest record key on the file and right-justified. 18 Normal sort order sequence Legal values: A or blank - ascending D - descending This field may be qualified by N or 0 entered in column 7 of the field specification. 29-32 Output Record Length (for SORTR and SORTT only) Legal values: A decimal number (right-justified) equal to the number of bytes in the largest output record. To determine this number, add the sizes of the key fields in the field specifications for the largest record in the file. If neither SORTT nor SORTR are to be run at this time, an entry in this field is not needed. Field Specification Records: The field specification records follow the header record and specify key fields (up to ten). NOTE: Data fields are not supported since each entire record in a file is written to the output file for SORTR and SORTT. For SORTA or SORTI, output files contain only pointers and possibly some restricted-format key data. Format: :Field Position Function Legal Values 1-5 Page/Line number See Header Specification 6 Field record ID F 7 Key field order Nor 0 8 Key field type B,C,D,I,J,K,P,Z 9-12 First byte of field Any number or blanks 13-16 Last byte of field Any number or blanks 17-19 (not used) Anything-ignored 20-80 (not used - available for comments) Anything-ignored Running SORT in Interactive and Batch Mode 2-27 Notes and Comments on Field Specification Entries: Columns 7 Explanations and Legal Entries Key Field Order - specifies keys and their -sort sequence (this entry can satisfy the column 18 entry for the Header Specification). Legal values: N - normal sort sequence ' 0 - opposite sort sequence 8 Key field data type codes: B - Binary (two's complement binary) C - Character (8-bit ASCII coded alphanumeric characters). This is the default data type. D - Decimal d~ta with sign trailing and overpunched. I- Same as D, but with the sign leading and separate, so that the first byte of the field is a + or -. 1J - Same as I, but with the sign trailing and separate. K - Same as D, but with the sign leading and overpunched P - Packed-decimal format. Z - Zoned ASCII format. 9-12 Field location (location of the first byte of a multi-byte key field). Legal values: A decimal number (right-justified) specifying the first byte of a key field. Blanks can be used to specify a one-byte key field. 13-16 Field location (location of the last, or only, byte of a key field). Legal values: A decimal number (right-justified) specifying the last byte, or the only byte, in a key field. Table 2-3: Fixed Position SORT Specification Summary Header Specifications --- Entry Column Explanation 6 H Header specification 7-12 SORTR SORTT SORTA SORTI Record sort Tag sort Address sort Index sort 13-17 1-255 Decimal number specifying the total length of all key fields listed in the Field Specifications (must be the maximum for SORTR). A or blank, Sort processing sequence: ascending or descending. 18 D 29-32 Decimal number (SORTR or SORTT only) 1-16,383. This entry specifies the number of bytes for the largest record. (continued on next page) 2-28 Running SORT in Interactive and Batch Mode Table 2-3: Fixed Position SORT Specification Summary (continued) Field Specifications Explanation Entry Column 6 F Field specification 7 N Normal - Key field sequenced as indicated in column 18 of Header Specification. 0 Opposite - Key field sequenced opposite to column 18 of Header Specification. c Character type data (8-bit ASCII alphanumeric data in key field). z Zoned ASCII. D Digit - use digit value or convert to binary for FORTRAN IV numbers. I Same as D, but with sign leading and separate (that is, the first byte of the field is a + or - ). J Same as D, but with sign trailing and separate (that is, the last byte of the field is + or - ) . K Same as D, but with sign leading overpunched (that is, the sign is superimposed on the first byte of the field). p Packed-decimal data type. B Binary data type - the key field is in two's complement binary notation. 9-12 Decimal number 1-16,383 Location of the first byte of the key field. 13-16 Decimal number 1-16,383 Location of the last (or only) byte in the key field. 17-19 (not used) All values are ignored. 20-80 Anything Comments. 8 Sample Fixed Position Specification Fiie: The following sample shows a header record and field record. Together they specify an index sort process on a character key of four bytes starting in position 10 of the record, and the output file is to be sorted in descending order. c 0 11.llllrl numbers Header Record Field Record 1234587 11 HSORTI FOC00100013 33 48 HEADER I NOE)-{ SORT ALL DEFAULT FIELD SPEC ALPHA KEY NOTE: This sample specification file performs the same sorting process as the sample shown for free field position format. Running SORT in Interactive and Batch Mode 2-29 2.5.2.2 For VAX-11 SORT (free fields, that Is flelds separated by commas): - Free fields are formatted in the same sequence as the fixed position fields described previously; however, instead of identifying fields with column numbers, commas are used. If you wish to enter blanks or use the default value, . you must follow the entry with a comma. Header records and field specification records are used in the same manner here as they are for the fixed position field records described previously, and the same explanations and legal entries also apply (see Table 2-3). Header Records: Field Position Function 1 Page number Any number, or blank, or comma, 2 Line number Any number, or blank, or comma, 3 Header record ID H, 4 Sorting process SORT R,I,A, T, or blank, or comma 5 Total key field size Any number, or blank, or comma, 6 Sorting order A, D, or blank, or comma, 7 Collating sequence Anything, or comma, 8 Output key Anything, or comma, 9 Record length Any number, or blank, or comma, 10 Comment ! anything, or blank Legal Values Fleld Specification Records: Field Position Function 1 Page number See Header 2 Line number See Header 3 Field record ID F, 4 Key sorting order N, 0, or blank, or comma, 5 Key data type B,C,D,I,J,K,P,Z, or blank, or comma, 6 start position of key Any number, or blank, or comma, 7 ending position of key Any number, or blank, or comma, Anything-ignored, or comma, 8 9 2-30 Legal Values comment Running SORT in Interactive and Batch Mode ! Anything-ignored Sample Free Fleld Position Specification Fiie: The following sample shows a header record and field record. Together they specify an index sort process on a character key of four· bytes starting in position 10 of the record, and the output file is to be sorted in descending order. Header Record t t H t SORT I t t , t , ! HEADER INDEH SORT ALL DEFAULT E}<CEPT TYPE t1F101C110t13tt!FIELD SPEC ALPHA KEY 4 BYTES POS 1 (I OPPOSITE ORDER Field Record NOTE: This sample specification file performs the same sorting process as the sample shown for fixed position format . . 2.6 Setting Up the Keys When entering the SORT command qualifier /KEY, you must specify /KEY=(subqualifiers and values) for each key field by which the records are to be sorted. The total size of all key fields must be less than or equal to 255 bytes, and the maximum number of key fields allowed is ten. Before entering the key subqualifiers and values into the SORT command string, perform the following steps (1through6). See Section 2.4.1 for /KEY= specification information. Figure 2-7 provides a flowchart for quick reference when setting up keys. Step 1 Each sort key is assigned a precedence number. You may choose to use either the default system or the key numbering system. First decide what your key fields will be and in what order you want them sorted. Make notes of their sequence for use when assigning the precedence number to each. Note, if you intend to enter the sort keys into the command string in the order of precedence you have chosen for your sorting operation, then the NUMBER= subqualifier is not necessary. Instead, the default feature will automatically assign the first key entered as key number 1 and each subsequent key the next higher number. K NUMB:a~p,le: 18 .no7required n ~:'3'. Q-MK-00020-00 2nd. Key 3rd. Key 4th.Key $ SORT/KEY=()1K_E_Y-==(-)/KEY=( )/KEY==(· ) (default) 2nd.Key Primary .Key 4th. Key 3rd. Key Running SORT in Interactive and Batch Mode 2-31 Figure 2-"/: Setting Up the Keys Make sure to enter keys Into the SORT command string In the correct order or .precedence If default , key numbering ll' used. NO (default STEP 1 ASSIGN KEY NUMBERS Make notes of key number assignment, starting position, and key field size for each key. CHARACTER (default) BINARY SPECIFY BINARY YES SPECIFY ZONED ~ STEP 5 SPECIFY START POS and SIZE for each key ~ STEP 8 SPECIFY ASCEND* or DESCEND ORDER STEP3 SPECIFY LEADING or TRAILING SIGN* TEP4 SPECIFY OVERPUNCHED* or SEPARATE SIGN NOTE: Data type determines the unit of key field size. NOTE: Total size of all key fields must be le11 than or equal to 255 bytes. *Indicates default parameter. Enter these key specifications for each key Into the SORT command string. 2-32 Running SORT in Interactive and Batch Mode F·MK-00021·00 Step 2 Before you can enter key size in Step 5, you must first determine what the data type of each key field is. Figure 2-8 shows a summary of the data types supported. See Appendix C for specific descriptions of the date types. Example: For decimal data type, specify DECIMAL in the command string. For example:$ SORT/KEY=(POS=l,SIZE=lO,DECIMAL). Figure 2-8: Recognizing Data Types and Signed Numbers 1. CHARACTER - 2. BINARY - 8-b It ASCII coded alphanumeric characters. (for example, 01010101) Packed-Decimal String POSITIVE NUMBER Odd# Even# 54321+ II 54321A NEGATIVE NUMBER Odd# Even# 4321+ Is 04321A 54321- Is 54321D TT._______ 4321- Is 04321D Last byte In string .___ _ _ _ _ _ First byte In string 3.. DECIMAL - 8-blt ASCII coded decimal digits 0-9 ~•ding Numeric String . (Moat significant byte ls llgnod 12345) + SIGNED NUMBER +or -12345 UNSIGNED NUMBER 12346 Separate Numeric Format Overpunched Format + 12346 Is A2345 +12345 ~ex 28 or 2~ ~ -12346 -12346 II J2345 I ~ ~ Tralllng Numeric String (Least significant digit la algned 64321) + SIGNED NUMBER 54321+ or - UNSIGNED NUMBER 54321 Separate Numeric Format Zoned Numeric Format Overpunched Format 64321+ 54321 + Is 54321 54321 + Is 5432A 54321- Is 5432g 54321-11 5432J I /Hex 28 or 2d\ 54321- ~ ~ ~ ~ ~ Running SORT in Interactive Batch Mode 2-33 Step 3 If data type is decimal, specify the position of the sign. If data type is not decimal, proceed to Step 5. Examples: 1. For +12345 or -12345, specify the optional keyword LEADING_SIGN in the command string. For example: $ SORT/KEY=<POS=1 tSIZE=10tDECIMALt LEAD) 2. For trailing sign numbers (that is, 12345+ or 12345-), the optional key- word TRAILING_SIGN is not required (TRAILING_SIGN is default). Step 4 For decimal data types, specify if the sign is overpunched (superimposed) or separate from the decimal value. Examples: 1. For separate sign (that is, +12345 or -12345), specify the optional keyword SEPAHATE_SIGN in the command string. For example: $ SORT/KEY=<POS=1 tSIZE=10 tDECIMAL tLEAD tSEPA) 2. For overpunched sign (that is, 5432A or 5432J), the optional keyword OVERPUNCHED_SIGN is not required (OVERPUNCHED_SIGN is default). Step 5 You must specify the starting position (first character in the key field) and the size for each key. The first character of the record is 1. Example: (in this example, key NUMBER= is default, data type is CHARACTER). $ SORT/KEY =(POS== 1,SIZE=10 )/KEY =(POS=22,SIZE=8 )/KEY =(POS=42,SIZE=6· )/KEY=-(POS=:84,SIZE=18·) 80-~:::r r i I~I ::: ~: := [ ~ 1 10 Character/ ~ POBitione 10-ch. key 79 80 ........,,.,,,,48 8-ch. 7-ch. 16-ch. key key key 29 42 64 ~ 22 Q..MK-OOOU-00 2-34 Running SORT in Interactive and Batch Mode NOTE: Key field size can represent either the number of bytes, or digits depending on the data type. The chart below describes which unit of key field size to use: Data Type Size Indicates Character Number of characters (bytes) must be less than or equal to 255. Binary Number of bytes must be either 1, 2, or 4. Decimal 1 byte for decimal values in the range of -128 to 127. 2 bytes for decimal values in the range of --32, 768 to 32,767. 4 bytes for decimal values in the range of-2,147,483,648 to 2,147,483,647. Number of digits in the string must be less than or equal to 31. The number of bytes in the key field for character, binary, zoned numeric, and overpunched is identical to the number of characters or digits. The size of leading separate or trailing separate fields is equal to the number of digits plus one. The size of packed-decimal fields is equal to (number of digits/2) + 1. See Appendix C for additional information. Step 6 Specify for each key, whether that key is to be sorted into ascending or descending order. Ascending order is default. Example: To sort the first key in ascending order and the second key in descending order, enter the key parameters into the command string as follows: $ SORT/KEY=(POS=11SIZE=10)/KEY=(POS=221SIZE=81DESCENDING) Now that you have performed Steps 1through6 to assemble specifications for each key, you are ready to _enter these key specifications into the SORT command string. 2. 7 Setting Up the Work Files SORT automatically assigns two work files to your SYS$DISK device if you choose to use the default. The size of these two work files (SORTWORKO and SORTWORKl) is determined by SORT from the size of your input file. (Note; if no assignment is done, work files are created on SYS$DISK). Running SORT in Interactive Batch Mode 2-35 To assign your work files to a device other than the deviee your directory is on, type: $ASSIGN (device>: SORTWORKO $ASSIGN (device>: SORTWORKl $ASSIGN (device>: SORTWORK8 Example: $ ASSIGN 063: SORT WORK 1 Figure 2-9 illustrates how logical names are assigned to physical devices. Figure 2-9: Specifying W~rk Files. WORK FILE # Logical Name 1 2 3 4 5 6 SORTWORKO SORTWORK1 SORTWORK2 SORTWORK3 SORTWORK4 SORTWORK5 SORTWORK6 SORTWORK7 SORTWORK8 SORTWORK9 7 8 9 10 A specific physical device code is assigned to a specific logical name using the ASSIGN, or DEFINE commands. Physical Device C o d e s - - - - - - - - - - - DB: RP04, RP05, RP06 Disk RK06 Disk OM: Example: ASSIGN E ORKO Unit 0 Default is AO Controller A - - - - - - RP06 Disk 2-36 } Running SORT in Interactive and Batch Mode Chapter 3 Calllng SORT from User Programs You can use SORT as a set of callable subroutines from your programming language. There are two functional interfaces to choose from; the file 1/0 interface and the record 1/0 interface. Both 1/0 interfaces share the same set of six subroutines, and the same calls are used from all languages. This SORT subroutine package consists of six external function calls. Each call causes a phase of the SORT program to be performed, and returns a status (32-bit) value indicating either success or the failure type of the phase. Calls and associated parameters conform to the VAX-11 standard calling interface. The calls are: Subroutine Name Function 1. SOR$INIT_SORT Initialize scratch files, work area, sorting parameters 2. SOR$PASS_FILES Pass a file name to SORT 3. SOR$RELEASE-REC Pass a record to SORT 4. SOR$SORT_MERGE Initiate sorting and intermediate merging of records 5. SOR$RETURN-REC Initiate final merge pass and receive output record from SORT 6. SOR$END_SORT Allow clean up of files and work area to complete the sort operation 3.1 Fiie 1/0 Interface The file 1/0 interface enables you to specify an input file and an output file to SORT. SORT then reads the data from the input file and sorts it into the output file. 3-1 For the file VO interface, use the following four calls in the order listed: Function Call 1. SOR$PASS__FILES Pass file specifications 2. SOR$INIT__SQRT Initialize work areas 3. SOR$SORT._MERGE Sort records 4. SOR$END__SORT Clean up work areas 3.2 Record 1/0 Interface The record 1/0 interface enables you to pass individual data records to SORT. SORT orders them, then returns each record in correct order, individually. For the record VO interface, use the following five calls in the order listed: Function Call 1. SOR$1NIT__SQRT Intialize work areas 2. SOR$RELEASE_REC Pass an input record 3. SOR$SORT_MERGE Sort records 4. SOR$RETURN_REC Receive a sorted output record 5. SOR$END__SORT Clean up work areas NOTE: Calls 2 and 4 are each repeated for as many times as there are records to be sorted. 3.3 Programming Considerations Any program can use either SORT subroutine package interface, providing the languagEi used produces VAX-11 native mode code and supports the following features. • 32-bit integers • Longword addresses • Call by string descriptors • Call by reference • Either CALLS or CALLG (that is, VAX/VMS standard calling sequence) • External function calls (each SORT subroutine returns a 32-bit status code) 3-2 Calling SORT from User Programs Additional information regarding the VAX/VMS calling standards can be found in Appendix C of the VAX-11 Common Run-Time Procedure Library Reference Manual. SORT follows the Mod\Ilar Procedure Standards and uses the common Run-Time Library routines to allocate memory and event flags. However, SORT is not re-entrant. The SORT subroutines are a part of the standard VMS library, therefore to use the package a user only has to code the appropriate calls into his program, compile or assemble and link. During the linking process the appropriate SORT routines will automatically be linked with the user's program. Figure 3-1 summarizes the callable subroutine set. Figure 3-1: Subroutine Set Summary NOTE:. Use the subroutine calls In the order shown. Call Function 1. SOR$PASS_FILES Open the Input file and create the output file. 2. SOR$1NIT_SORT Set up the key comparison buffer and validate key Information. Get memory for sorting Initial phase, Input and output buffers, and set up to read Input. Create work files and lnltlallze the sort. 3. SOR$RELEASLREC Get record from user and bulld key. 4. SOR$SORT_MERGE Insert record by key into sort tree. If sort tree Is full, continue; If not, get another record. Output records to work files as a number of strings of sorted records. Output and Input until no more records and all records are output. Read In strings from work file and merge them until there are ten or less left in work files. 5. SOR$RETURN_REC Set up to output records to user. Do final merge pass to output records (not work files) to user. 6. SOR$ENO_SORT Return memory, close output and Input files, and delete work files. Calling SORT from User Programs 3-3 3.3.1 Key Comparisons Both Interfaces When using either interface, you have the choice of allowing SORT to do key comparison to determine the correct order of any two records, or of writing a routine of your own that SORT can call to do the key comparisons. The advantage of writing your own routine is that you may know a great deal more about the nature of the key data and therefore write a routine specifically tailored to that particular data. Because SORT does not know anything about the key data in advance of receiving it, SORT's key comparison routine must be general in order to handle all types of data. A routine tailored to a particular data type or set can therefore be much·more efficient, both in space and performance. If you want to use the SORT key comparison routine, you must provide the key definitions in the SOR$INIT_SORT call. Or, if you want to use your key comparison routine with SORT, you must pass the address of your routine's entry point (with parameters) to SORT in the SOR$INIT_SORT call. See Section 3.4.3, Definitions, for details. Users can write a program that uses any key data type. For debugging purposes, it should be noted that the key comparison routine may not necessarily be called each time a call is made to SORT. This situation can occur with the following calls: SOR$RELEASE-REC SOR$SORT_MERGE SOR$RETURN-REC Record 1/0 Interface Only For record 1/0 interface, you must set up the key data area before passing the record to SORT if SORT is to do the comparisons. The key field must be set up with each key physically next to the one before it, in order of precedence from left to right. For example: If the key definitions looked like this: Key 1 - Character, Ascending, Pos 1, Size 4. Key 2 - Binary, Descending, Pos 15, Size 2. Key 3 - Packed, Ascending, Pos 30, Size 4. Then, the key area in the user's program should look like this: ,A I 0.aj-3,6 c 3\4 D, keyl k/y2 key3 SORT will handle ascending/descending considerations as long as SORT is doing the key comparisons. The user does not have to modify the key data in any way. 3-4 Calling SORT from User Programs In addition the entire key area must physically preceed and be adjacent to the record. For example: (KEYAREA) ABCD45634D (RECORD) ABCDEFGHIJKLMN45 ....... . When passing the record to SORT, the record descriptor must describe the entire string including the key area. Therefore, the length of the string is (total key length plus record length) and the address is the address of the first byte of the key area. For record 1/0 the only valid key types are 1, 2, and 4; character, binary, and packed decimal. However, the instruction set provides a set of decimal instructions that allow conversion from all of the other decimal formats to packed. Therefore, when you build the key area from your record data you can convert the other decimal types to packed, and by doing so, sort on any of the nine valid key data types that the file 1/0 interface accepts. When SORT returns the record it will strip off the key data. The length returned will be the length of the record alone and the first byte of the output buffer will contain the first byte of the record, not the key. If you are passing the address of your own key comparison routine to SORT and you do not wish to set up the key field :preceding the record, you may specify a 0 value as the total key size in the call to SOR$INIT_SORT. You then pass just the record to SORT. When SORT calls your key comparison routine the addresses of the two keys will be the ,iddresses of the first byte of each record. 3.4 Subroutines (Parameters, Definitions, and Valld Returns) Each call requires several user supplied parameters. Parameters, parameter definitions, and valid returns are provided in the following paragraphs for each call. Both symbolic and hexadecimal values are provided for the returned messages as an aid when debugging. All user program value parameters must be passed to SORT using "call by reference" (that is, the address of the value in the user's data area is passed to the SORT routine, not the value itself). All file specifications and records are passed to SORT using string descriptors. A descriptor is a 2-longword structure of format. For example: flags word ] length. of string address of string = 2 words = 1 longword = 1 longword The address of the descriptor is passed to SORT. To omit an optional parameter, either leave it null or pass a 0 address in the argument list; do not pass the address of a data item with a 0 value. In general the meaning of a parameter and its legal values are identical to the equivalent parameter in the command line to the utility. Calling SORT from User Programs 3-5 3.4.1 SOR$1NIT_SORT Function: Initialize scratch files, work area, and sorting parameters. Parameters: Each of the following parameters is numbered to match its definition which follows. 1. Key buffer address 0 2. Longest record length (LRL) • 3. File size• 4. Number of work files 5. Sort type e 6. Total key size • 7. Comparison routine address• Notes: o Mandatory for the file 1/0 interface and for the record 1/0 interface only if SORT is to do the key compares. 8 Mandatory for the record 1/0 interface. e Needed for the record 1/0 interface and input from unit record or magnetic tape devices in order for SORT to be efficient, but is not required. 8 Valid only for the file 1/0 interface. e Mandatory only if parameter 1 is not present and the user program is to do the key compares. Definitions: 1. Set up the key buffer in your user data area. The key buffer describes the definition of the keys to be sorted on, and has the following format: key type key order start position length one word = 1-9 for file 1/0 interface, and 1, 2, or 4 for record 1/0 interface one word = 0 or 1 one word = 1 to (max record size) one word = 1-255 (depends on key type) Up to ten of these blocks can be specified in the order of key precedence. 3-6 Calling SORT from User Programs The key buffer must be preceded by a word specifying the number of keys specified in the following blocks. For example: Key 1 Key 2 2 number of keys 1 0 10 40 key type (character) key order (ascending) start position in record length of key 4 0 60 10 key type (packed-decimal) key order (ascending) start position in record length of key in number of digits Key Types: 1 =Character 2 =Binary ·a= Zoned 4 = Packed-decimal 5 =not used 6 = Decimal leading overpunched 7 = Decimal leading separate 8 = Decimal trailing overpunched 9 = Decimal trailing separate Key Order: 0 = Ascending 1 = Descending When passing the address of the key buffer, pass the address of the word with the number of keys. 2. Longest record length (LRL) is a decimal number (one word in length) indicating the longest record length in bytes not including key size. 3. File size (one longword in length) is the value for the input file size in blocks. 4. Number of work files (one byte in length) is the value of 2 - 10 or 0. 5. Sort type (one byte in length) is the value of 1 - 4 as listed: 1 = Record sort 2 =Tag sort 3 = Index sort 4 = Address sort 6. Total key size (one byte in length) is the value of 1 - 255. 7. Address of the user generated key comparison routine. You have the option of performing your own key comparisons, and not supplying a key definition to SORT. SORT calls your routine at the specified address, and with the following parameters: 1) address of key 1 2) address of key 2 SORT expects the following return value: -1 if key 1 is less than key 2. 0 if key 1 is equal to key 2. 1 if key 1 is greater than key 2. NOTE Keys must not be modified in any way. Calling SORT from User Programs 3-7 Valid Returns: Symbolic Hex Value Meanin1 SOR$-SORT__ON 1C802C A sort is already in progress or this call is in the wrong sequence. SOR$_MISS_.KEY 1C8004 No key definition specified. SOR$-13AD_TYPE 1C806C An invalid sort process was specified. SOR$_BAD-1.JRL 1C8084 An invalid LRL was specified. SOR$_LRLMISS 1C8074 No LRL was specified and is required. SOR$_BAD_FILE 1C808C An invalid file size. SOR$_WORK_DEV 1C800C Work file deviee not random access device or not local node. SOR$_VM_FAIL 1C801C SORT failed to get needed virtual memory. SOR$_WS_FAIL 1C8024 SORT failed to get needed working set size. SOR$_NUM_.KEY 1C803C Invalid number of keys specified (must be 1-10). SOR$_KEY_LEN 1C80AC Invalid key length specified. SS$_NORMAL All RMS error codes 3-8 Calling SORT from User Programs 1 Success See Chapter 4. 3.4.2 SOR$PASS_FILES Pass a file specification to SORT. Function: Parameters: Each of the following parameters is numbered to match its definition which follows. ]· 1. Input file descriptor 2. Output file descriptor 3. Output file organizaton 4. Output file record format 5. Output file bucket size • 6. Output file block size 7. Output file maximum record size 8. Output file allocation 9. Output file file options Notes: All output file parameters are specified as for VAX-11 RMS. O These parameters are mandatory. 8 ;These parameters are optional. ( Definitions: 1. Input file descriptor is the string descriptor for the string in ASCII of the input file specification. 2. Output file descriptor is the string descriptor for the string in ASCII of the output file specification. 3. Value of output file organization (one byte in length): FAB$C_SEQ FAB$C-REL FAB$C-1DX 4. Value of record format for output (one byte in length): FAB$C_FIX FAB$C_VAR FAB$C_VFC Calling SORT from User Programs 3-9 5. Value for bucket size (one byte in length) is 1 - 32. 6. Value for block size (one word in length) is 18 - 32,767. 7. Value for maximum record size (one word in length) is 1 - 16,383. 8. Value for output file allocation (one longword in length) is 1 to the maximum RMS file size. 9. Value for output file file options (one longword in length) is: see the $FAB FOP parameters in the VAX-11 Record Management Services Reference Manual. Valld Returns: Symbolic SS$_NORMAL 1 I Success SOR$_SORT_ON 1C802C A sort is already in progress or this call is in the wrong sequence. SOR$_VAR__FIX 1C8064 Cannot change variable records to fixed records. SOR$-1NCONSIS 1C805C Inconsistent data for file. SOR$_0PENIN 1C109C Cannot open input file. SOR$_0PENOUT 1ClOA4 Cannot open output file. All RMS error codes 3-10 Meaning Hex Value Calling SORT from User Programs See Chapter 4. 3.4.3 SOR$RELEASE_REC Function: Pass a record to SORT. Parameters: Each of the following parameters is numbered to match its definition which follows .. 1. Record descriptor Notes: Parameter 1 is mandatory. Definitions: 1. Record descriptor is the address of the descriptor for the key and record being input to SORT. The length of the record must include the total key length plus the total record length. Also, the key field must physically immediately precede and adjoin the record, and the descriptor must point to the beginning of the key. Valld Returns: Symbolic SS$_NORMAL Meaning Hex Value 1 Success SOR$_SQRT_ON 1C802C A sort is already in progress or this call is in the wrong sequence. SOR$__BAD_LRL 1C8084 Record length is longer than LRL specified. SOR$__BAD_ADR 1C8094 Invalid descriptor address passed. SOR$_KEY_LEN 1C80AC Invalid key length specified. SOR$_EXTEND 1C80A4 Failed to extend work file. SOR$_MAP 1C809C Internal sort map error. SOR$_NO_WRK 1C8014 Cannot do sort in memory, need work files. Calling SORT from User Programs 3-11 3.4.4 SOA$SOAT_MEAGE Function: Initiate sorting and intermediate merging of records. Parameter1: None. Valld Returns: Symbolic SS$_NORMAL 3-12 Meaning Hex Value 1 Success SOR$_SOR'f'_QN 1C802C A sort is already in progress or this call is in the wrong sequence. SOR$_EXTE:ND 1C80A4 Failed to extend work file. SOR$_NQ_WRK 1C8014 Cannot do sort in memory, need work files. SOR$_MAP 1C809C Internal sort map error. SOR$_READERR 1ClOB4 Cannot read a specified input file record. SOR$_WRIT.EERR 1ClOD4 Cannot write a specified output file record. SOR$_BADFIELD lClOlC Bad data in key field. Calling SORT from User Programs 3.4.5 SORSRETURN_REC Function: Initiate final merge pass and receive output record from SORT. Parameters: Each of the following parameters is numbered to match its definition which follows. 1. Record descriptor 8 2. Record size Notes: 8 This parameter is mandatory. Definitions: 1. Record descriptor for the output area that SORT is to place the output record into. 2. The location (one word in length) in which SORT is to place the actual size of the record returned. Valld Returns: Symbolic Hex Value Meaning SOR$_MAP 1C809C Internal sort map error. SOR.$-EXTEND 1C80A4 Failed to extend work file. SS$_NORMAL 1 Success, a record has been returned. SS$-ENDOFFILE 870 Success, no more records to return. Calling SORT from User Programs 3-13 3.4.6 SOR$END_SORT Function: Allow clean up of files and work area to complete the sort opera- tion. Parameters: None Definitions: None Valld Returns: Symbolic SS$_NORMAL SOR$_CLEAN_UP 3-14 Meaning Hex Value 1 1C80B4 Calling SORT from User Programs Success Failed to delete work files and reinitialize work areas and data areas. 3.5 Sample MACRO Program .TITLE TESTSUB .IDENT x01.01 THIS IS A SAMPLE MACRO PROGRAM WHICH CALLS THE SORT SUBROUTINE PACKAGE. THERE IS AN EXAMPLE USING EACH INTERFACE. ' DATA AREA ; .ASCII /R010SQ.DAT/ .ASCII /TEST.TMP/ .BLKB 2 IN_FAB: .BLKB 80 IN_RAB: .BLKB 68 OUT_FAB: .BLKB 80 .BLKB 68 OUT-RAB: • LONG 10 FILEIN: .ADDRESS FILENAMEIN FILEOUT: .LONG 8 +ADDRESS FILENAMEOUT .WORD 1 KEYBUF: .WORD 1 KEYTYPE: .WORD 0 KEYORD: .WORD 1 KEYPOS: • WORD 10 KEYSIZ: .WORD 80 INLRL: • LONG 500, WRKFILE: .BYTE ll NUMWRK: .BYTE 2 TAGSRT: .BLKB 2 .BLKB 10 KEYAREA: .BLKB 80 RECORDBUF: .LONG 80 RECDESC: .ADDRESS KEYAREA FILENAMEIN: FILENAMEOUT: ;INPUT FILENAME ;QUTPUT FI LE NAME ;RMS DATA BLOCKS ;INPUT FILE NAME DESCRIPTOR ;ouTPUT FILE NAME DESCRIPTOR ;KEY DEFINITION BUFFER ;INPUT RECORD LONGEST LENGTH ;woRK FILE SIZE ;NUMBER OF WORK FILES ;TAG SORT ;KEY BUFFER ;RECORD BUFFER ;RECORD DESCRIPTOR FIRST THE FILE I/O INTERFACE. DO A TAG SORT ON THE FILE 'R010SQ.DAT' INTO THE FILE 'TEST.TMP' USING ll WORK FILES. KEY IS CHARACTERt 10 BYTES LONGt STARTING POSITION 1 • • EXTRN SOR$PASS-FILEStSOR$INIT_SORTtSOR$SORT_MERGEtSOR$END-SORTtSOR$RELEASE_RECtSOR$RETURN_REC FILEID:: .ENTRY PUSHAB PUSHAB CALLS BLBC PUSH AB PUSHAB FILEOUT FILEIN #2tSOR$PASS_FILES ROt2$ TAGSRT NUMWRK ;SAIJE REGISTERS ;DEFAULT ALL OUTPUT OPTIONS ;pusH FILENAME DESCRIPTOR ADDRESS ;PASS FILENAMES TO SORT ;TEST FOR ERROR ;PUSH SORT TYPE ;pusH NUMBER OF WORK FILES Calling SORT from User Programs 3-15 CLRQ PUSHAB CALLS BLBC -<SP> KEYBUF •5tSOR$1NIT-SORT ROt2$ CALLS BLBC CALLS BLBC •OtSOR$SORT-MERGE ROt2$ •OtSOR$ENO_SORT ROt2$ DEFAULT LRL ANO WORM FILE SIZE PUSH KEY BUFFER ADDRESS INITIALIZE THE SORT TEST FOR ERROR LET SORT DO COMPARES START SORTING TEST FOR ERROR 00 CLEAN UP TEST FOR ERROR NOW TRY THE RECORD I/O INTERFACE. RECORDS ARE 80 BYTES LONG, KEY IS CHARACTER, 10 BYTES LONGt STARTING IN POSITION 1. WORK FILE SIZE IS 500 BLOCKS. CALLS BLBC •OtOPEN-INPUT ROt2$ PUSHAB PUSHAB PUSHAB CALLS BLBC MOVZWL CALLS BLBC MOVC3 WRKFILE INLRL KEYBUF •3 tSORUNIT-SORT ROt2$ •1000 tl~G #OtGELRECORD ROt2$ •lOtRECORDBUFtKEYAREA PUSHAB CALLS BLBC SOBGTR RECDESC •1 tSOR$RELEASE_REC R0t2$ RS, 1$ CALLS 2$: BLBC 3$: PUSHAB PUSHAB CALLS CMPL BEQL BLBC CALLS BRB 4$: CALLS BLBC CALLS MOVL RET 6$: CLRL RET •OtSOR$SORT-MERGE ROtG$ 1$: ;soRT DOES COMPARES ;NO MORE RECORDS TO GIVE INLRL RECDESC •2,SOR$RETURN_REC ROtSSLENDOFFILE 4$ ROt8$ •O, PUT .. RECORD 3$ •OtSOR$ENO_SORT ROtG$ #OtCLOSE_FILE •1 tRO ;pusH RECORD SIZE LOCATION ;pusH RECORD DESCRIPTOR ;GET RECORD BACK ;GOTTEN ALL RECORDS ;YES ;ERROR ;puT RECORD INTO OUTPUT RO ;INDICATE FAILURE .END 3-16 ;oPEN USER INPUT ANO OUTPUT FILE ;TEST FOR ERROR ;DEFAULT SORT TYPE ANO WORK FILES ;pusH WORK FILE SIZE ;pusH LRL ;pusH KEY BUFFER ADDRESS ;INITIALIZE THE SORT ;TEST FOR ERROR ;sET UP LOOP INDEX ;GET RECORD FROM MY FILE ;TEST FOR ERROR ;SET UP KEY IN KEY BUFFER ;soRT DOES COMPARES ;pusH RECORD DESCRIPTOR ;GIVE RECORD TO SORT ;TEST FOR ERROR Calling SORT from User Programs ;FINISH UP ;TEST FOR ERROR ;CLOSE UP FILES ;INDICATE SUCCESS 3.6 Sample COBOL-74NAX Program IDENTIFICATION DIVISION. PROGRAM-IO. TSTSORT. * * THIS IS A SAMPLE COBOL-74/VAX PROGRAM THAT CALLS THE NATIVE * SORT SUBROUTINE PACKAGE USING THE RECORD I/O INTERFACE. IT * REQUESTS A RECORD SORT USING A 5 BYTE CHARACTER KEY. * ENVIRONMENT DIVISION. INPUT-OUTPUT SECTION. FILE-CONTROL. SELECT FILE-IN ASSIGN TO 11 SY 11 • SELECT FILE-OUT ASSIGN TO "SY". DATA DIVISION. * * ASSIGN FILE DEVICES * ANO NAMES ANO DEFINE INPUT AND OUTPUT RECORD AREAS. * FILE SECTION. FD 01 FD 01 FILE-IN VALUE OF ID IS 11 SORTIN.DAT 11 LABEL RECORDS ARE STANDARD. IN-REC. 05 IN-1 PIC X<9> • 05 IN-2 PIC X<5>. 05 IN-3 PIC X<S>. FILE-OUT VALUE OF ID IS 11 SORTOU.DAT 11 LABEL RECORDS ARE STANDARD. OUT-REC PIC X<20>. ** SET UP DATA FOR SORT SUBROUTINE PARAMETERS. *WORKING-STORAGE SECTION. 77 77 ENO-OF-FILE-SW 88 END-OF-FILE SHOW-STAT PIC X VALUE 11 0 11 • VALUE 1". PIC 9(9) • II ** LONGEST RECORD LENGTH, WORK FILE SIZE * AND RETURN STATUS VALUES. *77 LRL PIC 99 VALUE 20 COMP. 77 01 FILE-SIZ SORT-STATUS 88 SS-NORMAL 88 SS-ENDOFFILE PIC 9(8) VALUE 1 COMP. PIC S9(8) COMP VALUE O. VALUE 1. VALUE 2160. ** KEY BUFFER INDICATING ONE 5 BYTE CHARACTER KEY STARTING IN * POSITION 10 OF EACH RECORD, ASCENDING ORDER. *01 KEY-BUFFER. 05 05 05 05 05 KEY-NUMBER KEY-TYPE KEY-ORDER KEY-START KEY-LENGTH PIC 9(4) PIC 9(4) PIC 9(4) PIC 9(4) PIC 9(4) VALUE 1 COMP. COMP VALUE 1 • IJALUE o. COMP COMP VALUE 10. COMP VALUE 5. ** AREA FOR KEY AND RECORD. * Calling SORT from User Programs 3-17 01 WK-REC-ALL. 05 WK-KEYl PIC X<5>. 05 WK-REC. 10 WK-1 PIC X<S>. 10 WK-2 PIC X<S>. 10 WK-3 PIC X<G>. PROCEDURE DIVISION. MAIN-LOGIC• ** OPEN THE INPUT AND OUTPUT FILES. THEN INITIALIZE THE SORT *SPECIFYING THE KEY DEFINITION, THE LRL AND WORK FILE SIZE. * OPEN INPUT FILE-IN OUTPUT FILE-OUT. CALL "SOR$INIT_SORT" USING KEY-BUFFER LRL FILE-SIZ GIVING SORT-STATUS. IF NOT SS-NORMAL MOVE SORT-STATUS TO SHOW-STAT DISPLAY "FAILURE DURING SOR$INIT, STATUS WAS "SHOW-STAT PERFORM ABORT-JOB. ** READ RECORDS FROM FILE * EXTRACT THE KEY AND THEN HAND EACH TO SORT. * PERFORM RELEASE-RECS UNTIL END-OF-FILE. ** END OF FILE CALL SORT TO FINISH SORTING RECORDS. * CALL SOR$SORT_MERGE" GIVING SORT-STATUS. 11 IF NOT SS-NORMAL MOVE SORT-STATUS TO SHOW-STAT DISPLAY "FAILURE DURING SOR$MERGE, STATUS WAS " SHOW-STAT PERFORM ABORT-JOB. MOVE "0" TO END-OF-FILE-SW. ** REQUEST RECORDS BACK FROM SORT UNTIL ALL RECEIVED. * PERFORM RETURN-RECS UNTIL END-OF-FILE. ** CALL SORT TO CLEAN UP WORK AREAS. * CALL "SOR$END_SORT" GIVING SORT-STATUS. IF NOT SS-NORMAL MOVE SORT-STATUS TO SHOW-STAT DISPLAY "FAILURE DURING SOR$END, STATUS WAS PERFORM ABORT-JOB. 11 SHOW-STAT ** CLOSE FILES. * CLOSE FILE-IN FILE-OUT. STOP RUN. ** READ RECORDS AND BUILD KEY. * RELEASE-RECS. READ FILE-IN AT END MOVE 1 TO END-OF-FILE-SW. IF NOT END-OF-FILE MOVE IN-REC TO WK-REC MOVE IN-2 TO WK-KEYl CALL "SOR$RELEASE_REC USING BY DESCRIPTOR WK-REC-ALL GIVING SORT-STATUS. 11 11 11 3-18 Calling SORT from User Programs IF NOT SS-NORMAL MOVE SORT-STATUS TO SHOW-STAT DISPLAY "FAILURE DURING SOR$RELEASEt STATUS WAS " SHOW-STAT PERFORM ABORT-JOB. ** RECEIVE RECORDS AND WRITE THEM OUT. *RETURN-RECS. CALL "SOR$RETURN_REC" USING BY DESCRIPTOR WK-REC BY REFERENCE LRL GIVING SORT-STATUS. IF SS-ENDOFFILE MOVE "1" TO END-OF-FILE-SW. IF NOT END-OF-FILE MOVE SPACES TO OUT-REC MOVE WK-REC TO OUT-REC WRITE OUT-REC• ABORT-JOB. DISPLAY "ABNORMAL END OF JOB"• CLOSE FILE-IN FILE-OUT. STOP RUN. Calling SORT from User Programs 3-19 3. 7 Sample FORTRAN IV PLUS Program c c PROGRAM CALLSORT C C C C THIS IS A SAMPLE FORTRAN IV PLUS PROGRAM THAT CALLS THE NATIVE SORT SUBROUTINE PACKAGE USING THE FILE I/O INTERFACE. THIS PROGRAM REQUESTS AN INDEX SORT OF FILE 'R010SQ.DAT' INTO THE FILE 'TEST.TMP'. THE KEY IS AN 80 BYTE CHARACTER ASCENDING KEY STARTING IN POSITION ONE OF EACH RECORD. C DEFINE EXTERNAL FUNCTIONS AND DATA c c c c c C C C c c CHARACTER*lO INPUTNAME CHARACTER*B OUTPUTNAME INTEGER*Z KEYBUF<5> INTEGER*Z NUMWRK INTEGER*Z ISRTTYP INTEGER*4 SOR$PASS_FILES INTEGER*4 SOR$INIT_SORT INTEGER*4 SOR$SORT-MERGE INTEGER*4 SOR$END-SORT INTEGER*4 !STATUS !INPUT FILE NAME !OUTPUT FILE NAME !KEY DEFINITION BUFFER !NUMBER OF WORK FILES !SORT PROCESS !SORT FUNCTION NAMES !STORAGE FOR SORT FUNCTION VALUE INITIALIZE DATA - FIRST THE FILENAMES THEN THE KEY BUFFER FOR ONE 80 BYTE CHARACTER KEY STARTING POSITION 1' 3 WORK FILES AND AN INDEX SORT PROCESS DATA INPUTNAME,OUTPUTNAME/'R010SQ.DAT' ,'TEST.TMP'/ DATA KEYBUF,NUMWRK,ISRTTYP/1 ,1 ,0,1 ,90,3,3/ C CALL THE SORT EACH CALL IS A FUNCTION C PASS SORT THE FILENAMES c c c c C c c C c c C c 10 3-20 IBTATUS = SOR$PASS_FILES<INPUTNAME,OUTPUTNAME> IF <.NOT. !STATUS> GOTO 10 INITIALIZE WORK AREAS AND KEYS !STATUS= SOR$INIT_SORTCKEYBUF,,,NUMWRKtISRTTYP> IF <.NOT. !STATUS> GOTO 10 SORT THE RECORDS !STATUS = SOR$SORT_MERGE< > IF <•NOT. I STATUS) GOTO 10 CLEAN UP WORK AREAS AND FILES !STATUS= SOR$END-SORT<> IF <.NOT. !STATUS> GOTO 10 STOP 'SORT SUCCESSFUL' STOP 'SORT UNSUCCESSFUL' END Calling SORT from User Programs Chapter 4 Error Conditions You can encounter error conditions at three operating levels: first with the VAXNMS DCL command interpreter, next with the SORT error messages, and last with VAX-11 RMS messages. SORT handles two basic types of errors; fatal and warning. Fatal errors (severity level F) cause SORT to halt processing; warning errors (severity level W) cause a warning message to be output and allow sort processing to pro'." ceed. Errors in both these categories are grouped into three classes: • Errors caused by 1/0 or other system failures. • Errors caused by misinformation passed to SORT as a parameter of a subroutine call. • Errors caused by invalid data in a key field. For the SORT utility, errors of all types and classes are signaled to the system; this signal causes a message to be output. Execution is either stopped or continued based on the severity of the error. Execution can be resumed only if the severity level is W (that is, code = 0). In summary, only invalid data errors and a few RMS errors cause warning error messages. System or 1/0 failures and bad subroutine parameters are fatal. For additional information regarding error condition handling, refer to the VAX/VMS System Services Reference Manual. 4-1 4.1 Command Interpreter Error Messages In interactive mode, when you enter a command line incorrectly, the command interpreter issues a descriptive error message telling you what was wrong. For example, if you specify more than one parameter for a command that accepts a single parameter, you receive the message: %DCL-W-MAXPARAMt maximum Parameter count exceeded You must then retype the command line. Other error messages may occur during execution of a command. These messages can indicate such errors as a nonexistent file or a conflict in qualifiers. Not all messages from the system indicate errors; other messages are informative, or merely warn you of a particular condition. The VAXNMS system messages have the general format: %XXX-L-CODEt text t L t. Descriptive comment. - - - - - - - Shorthand code for the message text. , - - - - - - - Severity Level: S =Success W =Warning E =Error F =Fatal Mnemonic for the operating system program issuing the message. Example: %SORT-W-CLOSEOUTt error ification) closin~ output CoutPut file-spec- Because these messages are descriptive, you can usually understand what you need to do differently when you issue the command again. But, if you do not, the VAX/VMS Messages and Recovery Procedures Manual lists all the possible command interpreter error messages and describes what you can do to correct a command interpreter error. 4.2 SORT Error Messages The following VAX-11 SORT error messages are listed in alphabetic order. All SORT error messages have the same format as command interpreter messages, that is: %SORT-<seueritY leuel>-<code) tCtext). 4-2 Error Conditions The following descriptions of error messages observe the following conventions: • Only the (code), (text) part of the message is shown in the following list. • (filespec) ind~cates a DB1:[153,101TEST.TMP;3 file specification. For example: • (number) indicates the user entered numeric value. • LRL means the longest record length (specified in bytes). 6AD,....ADR t inl.1alid descriptor address specified. You passed the subroutine package an address for a descriptor, and the descriptor was invalid format. Character string descriptors in VAX consist of two longwords. The first word of the first longword contains the character string length in bytes. The second longword contains the address of the· string. User Action: See Section 3.4 and check character string format. 6ADFIELDt (filesPect or field text that is inl.1alid) field invalid at (nur11ber>. Bad data in key field or command. In this message, (number) indicates the record number of the record containing bad data in hex. User Action: Check key field data type and the starting positions and lengths (See Section 2.6, Setting Up the Keys). 6AD_FILEt file size invalid. You specified a negative file size or a zero file size. File size must be greater than zero. User Action: Specify a file size greater than zero. 6AD_KEYt invalid ~\e~· specification. Either the key field size, position, data type, or order is incorrect within the key definition. Positions start at one and cannot be greater than the maximum record size. Size must be less than or equal to 255 for character data, 1, 2, or 4 for binary data, and less than or equal to 31 for decimal. User Action: See Section 2.6, Setting Up the Keys, and check the command string key specifications. Error Conditions 4-3 BAD-LENt outPut record lensth less than 18 bYtes for Mastape. Magnetic tape requires record lengths to be at least 18 bytes and no greater than 4096 bytes. User Action: See Section 2.4.3, and check your output file block size parameters. BAD-LRL t inPut file (filespec). ,Record size sreater than specified LRL. In reading the input file, SORT encountered a record longer than the specified LRL. The record will be truncated to the LRL and sorted. User Action: ·Re-execute SORT with a larger LRL. BAO_SPECt invalid specification file record. FIELD: (record specification>. An incorrect field was specified in the specification file record. (record specification) indicates bad record contents. User Action: See descriptions of specification file record formats (Section 2.5.2) and change field specifications. BAD_TYPEt invalid sort Process. You passed the subroutine package a sort type code of less than 1 or greater than 4 if file 1/0 or not equal to 1 if record 1/0, or an invalid key word in command /PROCESS. Legal values are 1-4 for file 1/0, nothing for record 1/0, and RECORD, TAG, INDEX, or ADDRESS for command /PROCESS parameter. User Action: Specify a different sorting process. CLEAN_LJp, failed to reinitialize 1,,iork area and files.· SORT was unable to deallocate the extra virtual memory, deassign work file channels, or readjust working set size. For the SORT utility, this is a warning of little importance. For the SORT subroutine packages, this could mean a failure to be able to recall SORT from the same program until it has exited. This is an internal error. User Action: Exit from the user program before re-executing SORT. CLOSEINt error closins (filesPec) as inPut. An error occurred closing an inprtt file. This message is usually accompanied by an RMS message indicating the reason for the failure. User Action: Take corrective action based on the associated message. 4-4 Error Conditions CLOSEOUT terror closins (filespec) as output. An error occurred closing an output file. This message is usually accompanied by an RMS message indicating the reason for the failure. User Action: Take corrective action based on the accompanying message. EXTENDt failed to extend i...1orK file. SORT failed to extend a user's temporary work file. Either the device is full, or the user does not have extend privilege. User Action: See Section 2. 7 and reassign work files to a different device with more space, and make sure you have extend privilege on that directory. INCONSISt inconsistent data in file (filesPec). If you specified /OVERLAY plus other· output file qualifiers, SORT will verify that the information in the existing file matches the information you provided. If it does not, this error message is reported. Unless you specifically want a verification, /OVERLAY should be used without other qualifiers. User Action: Check the command string output file qualifiers (See Section 2.4.3). IND-OVRt fier. indexed sequential outPut requires overlay quali- You specified indexed output file organization and did not specify /OVERLAY. User Action: You must create the indexed file first with RMS DEFINE utility (or other). The primary key of the file should be the same as the sort key for efficiency but is not required to be. Then you must specify /OVERLAY in the SORT command string. KE'1'-LENt •te}' Cn1.1mber>. lensth in\.1alid. Ke}' tHlf11ber (n1.u11ber)t size The key size is incorrect for the data type, or the total key size is greater than 255. User action: See Section 2.6, Setting Up the Keys, and specify correct key field size. Size must be less than or equal to 255 for character data, 1, 2, or 4 for binary data, and less than or equal to 31 for decimal. Also, only ascending or descending order is allowed. Error Conditions 4-5 LRL_MISS t LRL rr1ust be specified• If record 1/0 interface subroutine package is selected, the longest record length (LRL) must be passed to SORT in the call. User Action: See Section 3.4, and specify LRL. MAP, failed to rr1aP 1,,1orR file. This is an internal SORT failure. User Action: Verify that the system parameter "maximum process sections" has been set up at 10. If it has, then report this failure to a specialist. Otherwise, set that system parameter to 10. MISS_KEY t H.eY specification frlissins. SORT did not find any key definition in either the command line or specification file, or in the parameters to the subroutine package. User Action: You must input at least one key definition in one of these three areas. NO_WRK t need 1,,1or•:. files cannot do SORT in rr1err1orY. You specified /WORK-FILES=O indicating the data would fit in memory, but the data was too large. User Action: Either increase the working set quota, or allow SORT to use two or more work files. NUM_KEYt too man>' •:.e>'S specified. Up to ten key definitions are allowed. Either too many have been specified, or the NUMBER value is wrong. User Action: See Section 2.6, Setting Up the Keys, and check your command string key field specifications. ONE_IN, onh· one inPut file allo1,,1ed. SORT will take only one input file at a time. User Action: You can concatenate files of the same organization and record format using COPY, and then sort. OPENINt error oPeninS' (filesPec> as inPut An input file cannot be opened. This message is usually accompanied by an RMS message indicating the reason for the failure. User Action: Take corrective action based on the associated message. 4-6 Error Conditions OPENOUTt error oPenins (filesPec> as output An output file cannot be opened. This message is usually accompanied by an RMS message indicating the reason for the failure. User Action: Take corrective action based on the associated message. READERRt error readins (filesPec> An input file record specified cannot be read. This message is usually accompanied by an RMS message indicating the reason for the failure. User Action: Take corrective action based on the associated message. SORT_ONt sort alread}' in Prosress. You tried to call the SORT subroutine package with calls in the wrong order, or to recall it before it finished running the previous sort. User Action: Reorder the subroutine calls and then re-execute SORT. VAR-FIXt lenath. cannot chanse variable lenSth records into fixed You specified variable length input records and requested fixed length output. User Action:Output records must be variable or controlled in this case. t.JM_FAILt failed to Set rec:iuired virtual 1rle1r1orY (nurriber). SORT could not get the amount of virtual memory required for the sort. (number) indicates the number of bytes needed. User Action: If the SORT utility is being run, decrease the working set quota; if either SORT subroutine package is being run, either decrease the quota or return some memory to the system inside the user's program before calling SORT. WORK_DEl.Jt 1,..1orH file (filespec) det.iice specified not rando1r1 access or not local. Work files must be specified for random access devices that are local to the CPU the sort is being performed on (that is, not on node in a network). Random access devices are disk devices. User Action: See Section 2.7, Setting Up the Work Files, and specify the correct device. Error Conditions 4-7 WRITEERRt error 1A1ritins (filespec> An output file record cannot be written. This message is usually accompanied by an RMS message indicating the reason for the failure. User Action: Take corrective action based on the associated message. ws_FAILt failed to Set rec:iuired IAfOrKins set space (n•.lfrlber). SORT could not set the required amount of real memory space. A minimum 75 page working set is needed. (number) indicates number of pages available. User Action: Increase the working set quota. 4.3 VAX-11 RMS Error Codes Listed below, in alphabetic order, are the VAX-11 RMS completion status codes. This list includes both symbolic and hexadecimal codes for error messages and success messages. These RMS codes are returned to your program by the operating system. All VAX-11 RMS error messages have the same format as command interpreter messages, that is: 'X.RMS-<se,.ieritY level>-( code) tCtext). For additional information refer to the VAX-11 Record Management Services Reference Manual. Valld Returns for Error Messages: 4-8 Symbolic Hex Value RMS$_ACC 0001C002 File access error. RMS$_ACT 0001825A File activity precludes operation. RMS$_AID 000183F4 Bad area identification number field in allocation XAB. RMS$_ALN 000183FC Invalid alignment boundary type in allocation XAB. RMS$_ALQ 00018404 Incorrect allocation quantity in allocation XAB; the value either exceeds the maximum allowed, or is equal to zero for the extend service. RMS$_ANI 0001840C Records in a magnetic tape file are not ANSI D format. RMS$_AOP 00018414 Invalid allocation option in allocation XAB. RMS$_ATR OOOlCOCC Read error on file header. RMS$_ATW 0001COD4 Write error on file header. RMS$_BKS 0001841C Invalid bucket size in FAB. Error Conditions Meaning Meaning Symbolic Hex Value RMS$_BKZ 00018424 Invalid bucket size in the allocation XAB for relative file. RMS$_BLN 0001842C Invalid value in block length field. RMS$_BOF 00018198 File is already at beginning of the file (backspace operation). RMS$_BUG_J)DI 0001843C Invalid default directory. Internal VAX-11 RMS error; no recovery possible - contact a software specialist. RMS$_CCR 00018494 Cannot connect RAB (only one record stream permitted for sequential files). RMS$_CD A 0001COE4 Cannot deliver AST. RMS$_CHN OOOlCOEC Channel assignment failure. RMS$_COD 000184AC Invalid type code in XAB. RMS$_CRE OOOlCOOA File create error. RMS$_CUR 000184B4 No current record; operation not immediately preceded by a successful get or find service. RMS$_DAC 0001C012 File deaccess error during a close service. RMS$_DEL 00018262 Record accessed by RF A record access mode has been deleted. RMS$_DEV 000184C4 Bad device or inappropriate device type for operation. RMS$_J)IR 000184CC Error in directory name. RMS$_J)ME 000184D4 Dynamic memory exhausted; occurs only if the related 1/0 segment in the control region is full and the file is either a direct access process permanent file, or the user has disallowed the use of the program region for 1/0 buffers to VAX-11 RMS. RMS$_J)NA 000184DC Error detected in the default file specification string. RMS$_J)NF 0001826A Directory not found. RMS$_J)NR 00018272 Device not ready. RMS$_]) PE 0001C03A Device positioning error; applies only to magnetic tape. RMS$_J)VI 000184F4 Invalid device identification in NAM block. RMS$_ENT OOOlCOlA Error during file enter service. RMS$_ENV 00018724 Environment error; the code necessary to support the file organization or facility was not selected at system generation. RMS$_EOF 0001827A End of file. RMS$_ESA 000184FC Invalid expanded string area in NAM block. ·RMS$_ESL 00018714 Invalid expanded string length in NAM block. RMS$_ESS 00018504 Expanded string area too short. RMS$_EXP 000182C2 File expiration date not yet reached. RMS$_EXT 0001C022 File extend error. RMS$_FAB 0001850C Invalid FAB; block indentifier field incorrect. Error Conditions 4-9 Symbolic Hex Value Meaning RMS$_FAC 00018514 Operation not allowed by the value set in the file access field of the FAB. RMS$_FEX 00018282 File already exists. RMS$_FLK 0001828A File is locked and therefore not available. RMS$_FNA 00018524 Invalid file specification string address in FAB. RMS$_FND 0001C02A Files-11 find function failed. RMS$_FNF 00018292 File not found. RMS$_FNM 0001852C Syntax error in file name. RMS$_FQP 0001853C Invalid file processing options. RMS$_FSZ 00018534 Invalid fixed control area size in FAB (equal to 1 for print files). RMS$_FUL 00018544 Device full; cannot create or extend file. RMS$_IFA 0001C124 Illegal file attributes; file header corrupted. RMS$_IFI 00018564 Invalid internal file identifier in FAB; must be zero. RMS$_IMX 0001856C More than one XAB of the same type is present for the file. RMS$_IOP 00018574 Illegal operation attempted: 1. block I/0 when not block I/0 access. 2. record I/O when block I/0 access. 3. rewind of process permanent file. 4. inappropriate device type or file organization. RMS$_IRC 0001857C Illegal record in sequential file; invalid count field. RMS$_ISI 00018584 Invalid internal stream identifier in RAB. RMS$_KBF 0001858C Invalid key buffer address; not in access limits. RMS$_KEY 00018594 Invalid record key for random operation to a relative file. RMS$_KSZ 000185A4 Key size not equal to 4 (relative file). RMS$-1NE 000185BC Logical name error; resulted in duplicates. RMS$_MBC 00018734 Invalid multi-block count; must not be greater than 127. 4-10 RMS$_MKD 0001C032 Files-11 ACP could not mark file for deletion. RMS$_MRN 000185CC Illegal value for maximum record number. RMS$_MRS 00018504 Illegal value for maximum record size. RMS$_NAM 000185DC Invalid NAM block. RMS$_NEF 000185E4 Attempt to use the put service to a sequential file when not positioned to end of file. RMS$_NMF 000182CA No more files for a search operation. RMS$_NQD 000185F4 Node name error. RMS$_0RG 0001860C Illegal file organization. R.MS$_FBF 00018614 Invalid prompt buffer address. Error Conditions Symbolic Hex Value Meaning RMS$_pLG 0001861C Error in file prologue; file is corrupted. RMS$_pLV 0001872C Prologue version unsupported. RMS$_pRV 0001829A Privilege violation; access denied. RMS$_QUO 00018634 Error in quoted string. RMS$_RAB 0001863C Not a valid RAB; block identifier field incorrect. RMS$_RAC 00018644 Illegal value in record access mode field of RAB. RMS$_RAT 0001864C Record attributes invalid in FAB. RMS$_RBF 00018654 Invalid record address. RMS$_RER 0001COF4 File read error. RMS$_REX 000182A2 Record already exists; in a random access mode operation to a relative file, a record was found in the target record cell. RMS$_RFA 0001865C Invalid record's file address contained in RAB. RMS$_RFM 00018664 Illegal record format. RMS$_RHB 0001866C Invalid record header buffer. RMS$_RLF 00018674 Invalid related file. RMS$_RLK 000182AA Record locked by another task. RMS$_RMV OOOlCOFC Files-11 remove function failed. RMS$_RNF. 000182B2 Record not found. RMS$_RNL 000181AO Record not locked. RMS$_RPL 0001Cl04 Error while reading prologue. RMS$_RSA 0001868C Record stream active; an attempt was made to issue a record operation request in an asynchronous environment to a record stream that has a request outstanding. RMS$_RSL 0001873C Resultant string length field of NAM block invalid. RMS$_RSS 00018694 Resultant string area size field of NAM block is too small. RMS$_RST 0001869C Invalid resultant string area. RMS$_RSZ 000186A4 Illegal record size. RMS$_RTB 000181A8 Record too large for user buffer. RMS$_SHR 000186B4 Invalid value in the file sharing field of FAB. RMS$_SQO 000186C4 Operation not sequential. RMS$_SYN 000186D4 Syntax error in file specification. RMS$_SYS OOOlClOC Error in system QIO directive. RMS$_TMO 000181BO Time-out period expired. RMS$_TYP 000186E4 Error in file type. RMS$_UBF 000186EC Invalid user record area address. RMS$_USZ 000186F4 Invalid user record area size. RMS$_VER 000186FC Error in version number. Error Conditions 4-11 Symbolic Hex Value Meaning RMS$_WER 0001C114 File processor write error. RMS$_WLK 000182BA Device is not write-locked. RMS$_WPL OOOlCllC Error while writing prologue. RMS$_WSF 0001871C Working set full. RMS$-XAB 0001870C Not a valid XAB. Valld Returns for Success Messages: 4-12 RMS$_CONTROLC 00010651 Operation completed under Control C. RMS$_CON'I'ROLO 00010609 Operation completed under Control 0. RMS$_CONTROL Y 00010611 Operation completed under Control Y. RMS$_CREATED 00010619 File was created; not opened; used in conjunction with the CIF option. RMS$-KFF 00018031 Known file found. RMS$_NORMAL 00010001 Operation successful (synonym for RMS$_SUC). RMS$_0K-ALK 00018039 Record already locked. RMS$_0K-1>EL 00018041 Deleted record accessed correctly. RMS$_0K_RLK 00018021 Record locked but read anyway; locked set RLK bit in ROP field. RMS$_01L.RNF 00018049 Non-existent record accessed correctly. RMS$____FENDING 00018009 Asynchronous operation not yet completed. RMS$_SUC 00010001 Operation successful (synonym for RMS$_NORMAL). RMS$_SUPERSEDE 00010631 Created file superseded an existing version. Error Conditions Chapter 5 Improving SORT Efficiency Users who have special sorting requirements such as very large files, storage media contraints, and processing time restrictions can modify SORT's behavior for optimum performance. Your ability to improve SORT's performance depends on your understanding of SORT's operational characteristics described in this chapter. This chapter discusses: • How the SORT program functions in each phase of operation, and what sequence of events occur during a sort run • How a user can improve SORT's efficiency through the use of tuning procedures 5.1 Functional Description The SORT program consists of two basic parts: a control program called the utility and a callable subroutine package (see Figure 5-1). The utility directs the overall processing. The callable subroutine package serves as a collection of subroutines that the utility uses during its processing. You can write your own control program to take advantage of SORT's callable subroutines (see Chapter 3). There are eight phases of operation in the SORT utility. These are described in more detail in Section 5.1.2. A sort run breaks down into three tasks. First, SORT reads the command string and the specification file, if present, decodes them, and then stores the qualifier values and parameters. Any errors in the command string or specification file are reported at this point. 5-1 Figure 5-1: VAX-11 SORT Architecture, Main Functional Components CALLABLE SUBROUTINE PACKAGE UTILILTY I 1/191_ __,A'-'----.. \ MAIN LINE RMS FILE 1/0 COMMAND INTERPRETER INTERFACE VAX-11 SORT COMMAND DCL COMMAND USER F-MK-00024-00 5-2 Improving SORT Efficiency Second, SORT begins the pre-sort operation. The control program calls routines to open and read the input file and establish -the keys. Then the SORT subroutine package is called to begin the initial sorting process. At this point, the amount of available internal storage space becomes important to the efficiency of the sort. If that space is not sufficient to hold all the records, SORT builds strings of sorted records· and transfers them to work files on temporary storage devices (disk). The SORT program normally provides for a default of two work files. A qualifier in the command string can increase the number of work files used. Third, SORT rebuilds the intermediate work files into a merged file. If the process is tag sort, another subroutine reads the records in the proper sequence. The records are then written in the output file. If there· are no work files to merge because main memory was sufficient to hold all the records, the sorted records are written directly into the output file. After the last record is written, the control program cleans up the work files and exits; SORT is then ready to accept another job. 5.1.1 Sorting Processes All four sorting processes can sort records of fixed or variable length, VFC, or any valid VAX-11 RMS. Stream format is not supported. The size of the records on a fixed-length format file is determined when the file is created. The first word of a variable-length format record contains the size of the record in bytes. This first word is used by the file system and is transparent to SORT. 5.1.1.1 Record Sort - Record sort outputs all data records in a specified sorted sequence. Each record is kept intact throughout the entire sorting process. Since this process moves the whole record, it is relatively slow and may require considerable· main memory or external storage work space for large files. 5.1.1.2 Tag Sort - Tag sort produces the same kind of output file as record sort, but it only handles record pointers and key fields. Since this process moves a smaller amount of data than record sort, it may perform a faster sort than record sort. The input file must be randomly re-accessed to create the entire output file, which may be a lengthy process for large files. Input Data Flies A record is usually divided into several logical areas called data fields. The data in each field may or may not be relevant to SORT. Each field may be interpreted as a record identifier, key data, or general data related to the Improving SORT Efficiency 5-3 logical content of the record and not relevant to the sorting process. SORT uses record identifiers to distinguish the various types of records in a file. SORT uses the key fields in each record to reorder an input file. Any other data field in a record may be retained in the output file or ignored by SORT. Figure 5-2 shows three different types of input records, each with a different format. The record identifiers· are the letters in position 1: S means sales record, 0 means order record, and R means restock record. In this case, the keys chosen for sorting the sales record types are the "item number code" in positions 2 to 7, and the "number of items sold" in positions 8to13. The "total amount of sale" is an example of a data field not relevant to the sorting process. If you request a sort in ascending order on the sales records as shown in Figure 5-2, the sort is based on the item number code first and then on the number of each item sold within that item number. In order of decreasing significance, the keys are: 1. Item number 2. Number of items sold Figure 5-2: Sample Record Types RECORD IDENTIFIER KEY FIELD KEY FIELD DATA FIELD SALES RECORD '--~........-~--"--~-.---''--'--.....-......_~--~......._~~-v---~--~-v~~~-----~~ NUMBER OF ITEMS SOLO ITEM NUMBER CODE UNIT CODE TOTAL AMOUNT OF SALE PRICE PER UNIT CUSTOMER NO. CODE DATE OF SALE ORDER RECORD '-------...----·--··-____,.__--~-· OE SCRIPTION OF ITEM -......,----____..._ ITEM NO. CODE PRICE OF ITEM NUMBER OF ITEMS ORDERED PURCHASE ORDER NO. RESTOCK RECORD ITEM NO. CODE UN IT CODE COST PER UNIT MINIMUM STOCK QUANTITY PRESENT AMT. OF STOCK REORDER NUMBER 11-1520 5-4 Improving SORT Efficiency Output Data Flies The output file contains all sales records in the order shown in Table 5-1. Table 5-1: Sorted Output File Minor Key: Quantity Major Key: Item Number ~----i Lowest quantity Lowest item no. Next higher quantity Highest quantity Lowest item no. ~-----------Next higher item no. ----------------! Lowest quantity Next higher quantity Next higher item no. Highest item no. ______________ .,. Highest quantity -------------~ Lowest quantity Next higher quantity Highest item no. Highest quantity 5.1.1.3 Address Sort - Address sort produces address files, which consist of record's file addresses (RFAs), beginning at 1, and written in binary words. These files can be used as a special index file to access randomly the data in the original file. It is possible to maintain only one data file, but several different index files as needed. Like tag sort, this process uses the minimum amount of data necessary in the sorting process. Once the input phase is completed, the input file is not read again. This means that address sort is the fastest sorting method of the four SORT types. NOTE: Do not transfer an address index file to a device that cannot handle binary data, such as a printer or terminal. Improving SORT Efficiency 5-5 The address sort produces an output file consisting of record indices. Each record index occupies one 6-byte record in the output file. Assume that you are sorting a file consisting of. six records using the address sort process. If the sequence of record indices corresponding to the sorted records is 5,1, 6,3,4,2 then the output file can be represented as shown in Figure 5-3. Figure 5-3: Sample Address Sort Output File •.. - Record's Fiie Address (RFA) ·- Block Number RECORD NUMBER LOW (2 bytes 16 bits) HIGH (2 bytes 16 bits) BYTE-IN-BLOCK (2 bytes 16 bits) 1 000001 000000 000162 2 000001 000000 000000 3 000001 000000 000236 4 000001 000000 000042 5 000001 000000 000132 6 000001 000000 I Number of Blocks per File .__..... . 000026 ·-""""" I Number of Contiguous Bytes per Block Note: Byte and Block numbers are shown here in hexadecimal, they are written to the actual output file in binary. 5.1.1.4 Index Sort - Index sort produces an address file consisting of records file addresses (RF As) in binary, and key fields in original form. This makes it slightly slower than address sort. During processing this sort handles only the RF As and two forms of the key fields. One form is used for sorting and the other is left as it was in the original data. Index sort p:coduces an output file consisting of record indices plus keys in original form. Each record in the output file consists of a 6-byte record index plus the key field. NOTE: Do not transfer an index sort output file to a device that cannot handle binary data, such as a printer or terminal. 5-6 Improving SORT Efficiency Assume that you are sorting a file consisting of six records using Index sort process, and you are using a key size of four characters (bytes). The sequence of record indices corresponding to the sorted record, is 5,1,6,3,4,2 as shown in Figure 5-4. Figure 5-4: Sainple Index Sort Output File Record's Fiie Address 1RFA1 Block Number RECORD NUMBER LOW (2 bytes 18 bits) HIGH (2 bytes 16 bits) BYTE-IN-BLOCK (2 bytes 16 bits) 1 000001 000000 000162 A B c D 2 000001 000000 000000 A B c D 3 000001 000000 000236 A B c D 4 000001 000000 000042 A B c D 5 000001 000000 000132 A B c D 6 000001 000000 000026 A B c D KEY IN ORIGINAL FORM Notes: 1. ABCD represents the sorting keys In original format. 2. Byte and Block numbers are shown here In hexadecimal, they are written to the actual output file In binary. 5.1.2 Internal Organization SORT operates in eight phases (phase 0 through 7). Figure 5-5 summarizes these phases. Phase Function 0 Decode command line and specification file 1 Initialize SORT 2 Get records 3 Sort records 4 Initialize merge 5 lVlerge records 6 Output records 7 End SORT Improving SORT Efficiency 5-7 The VAXNMS command interpreter calls the SORT utility at its main entry point and Phase 0 is initiated. The initial process statistics are acquired from the system and stored in a table. SORT calls the command interpreter to parse and validate the command line. Then SORT validates this information and stores it in various tables and buffers. If a specification file is present it is opened, the records are read, the information validated and stored in various tables and buffers, and the file is closed. Any errors up to this point are reported by signaling the command interpreter. SORT opens the input file, and creates the output file. SORT begins Phase .1. The sorting process is initialized by filling in the key comparison information, allocating the space needed for input and output buffers and the sort tree, creating the work files and initializing the sort tree. Phase 2 begins the sort proper. SORT either reads records from the input file, or receives them from the caller. At Phase 3, SORT builds the key from the record and inserts each record into the tree by key. This process repeats until the sort tree is full or there are no more records. SORT then outputs the records to the work files as a variable number of strings each of which is a set of sorted records. Each time a record is output from the tree a new one is input until there are no more records. The rest of the records in the tree are output and that ends the initial sorting phase (phases 1 through 3). Phase 4 starts SORT's internal merging operation. The memory is redivided at this point for the merge phase into one to ten input buffers and one output buffer, depending on the number of initial strings. A different string is read into each input buffer and the records are merged together into one string and output to a work file. This process is repeated until the total number of strings is less than ten. Phase 5 performs the final merge pass and outputs the remaining string of records, which is the final sorted file, to either the output file or the caller. Phase 6 closes the input and output files, closes and deletes the work files, and returns the me·mory. Phase 7 acquires the final statistics and prints them, then exits SORT back to the VAXNMS command interpreter. Notes: 1. Phases 0 and 7 are part of the utility only. 2. The last part of phase 0 (opening the input file and creating the output file) and phase 6 are used only by the file 1/0 subroutine package and the utility. 3. Phases 1through5 are used by the utility and both the file 1/0 and record 1/0 subroutine packages. 5-8 Improving SORT Efficiency 4. Errors during phases 1 through 6 are signaled to the VAXNMS command interpreter if the utility is running, or returned as a status code to the caller if the subroutine package is running. 5. All signaled errors produce messages at the command interpreter level (see Chapter 4). Figure 5-5: VAX-11 SORT Operating Phases PHASE 0 DCL SORT Command (via VAX/VMS command Interpreter). Entry point to SORT. Get Initial process statistics. Call command line processor to decode command line. If It was specified, call specification file decoder. Open the Input file and create the output flle. PHASE 1 Set up the key comparison buffer and validate key Information. Get memory for sorting Initial phase, Input and· output buffers, and set up to read Input. Create work flies and lnltlallze the sort. PHASE 2 Read or get record from user and build key. PHASE 3 Insert record by key Into sort tree. Is sort tree full? (YES, continue /NO, go back to phase 2) Output records to work flies as a number of strings of sorted records. Output and Input until end of file and all records are output. PHASE 4 Read In strings from work file and merge them until there are 10 or fewer strings left In the work flies. PHASE 5 Set up to output records to user or output file. Do final merge pass outputting records to user files, not work files. PHASE 6 Delete work files, return memory, close output and Input flies. PHASE 7 Print statistics and exit. Improving SORT Efficiency 5-9 5.1.3 Buff'er Allocation and Work Areas The SORT utility and subroutine package are initially linked with a minimum of space allocated. When SORT is initialized, the work area manager assigns as much virtual memory as the process needs, and adjusts the working set size to the process maximum. This allows SORT to minimize page faults during the sorting, and maximize the order of the merge. At the end of the sort operation the limits are restored to the size they were at the entry to SORT, returning the additional virtual memory to the system. SORT requires a minimum of 75 pages of memory for the working set. The VAXNMS memory management system service, create and map sec_tion, allows a user to specify that a particular span of virtual addresses in the program should be read from and written into a particular set of virtual blocks within a file on disk, when referenced or paged-out by the entry of another page. The actual 1/0 to and from the disk is all handled by the pager. SORT maps the individual virtual addresses representing the work area onto specific blocks within a work file. When a particular buffer is then referenced within the work area, the pager automatically brings the correct blocks from the work file into real memory, and writes the existing blocks back to the correct place in the file. 5.1.4 Dynamic Memory Usage Figure 5-6 shows the total address space used by SORT during each of its eight phases of operation. Figure 5-6: SORT Dynamic Memory Usage A= User Limit. B= User Quota that· must be greater than or equal to 75 pag1~s of real memory. TAS= Total Address Space. VM= Virtual Memory. WS= Working Set. 'r-------, Unused Portion of Code and Data. TAS=WS+(=B) "~ VM(=3•B=3•WS) Code and Data (fixed) Phases_.. ~ I I I I : I I I I I r I VM=3•WS(=B) lvM= 1Buffers(=3•WS=B)I I J. I Tree=1/2WS Input Buffer= 1/4WS Input Buffer #1 ~Same as Phases 1,2,3 T Portion of Code and Data needed for Phases 1-3 WS=B Code and Data 1,2,3 4,5 0 ~ __ _B~f~r___ I 1 I Input Buffer I I #10 I ,-- - - r----1 ~ Output Buffer WS=A TAS=A r--------1 Output 1 I WS=B Code and Data (fixed) j WS=A TAS=A 6,7 I ._1 1--------~-----~~~~~---SORTRUN-------------------------~.. H-MK-00025-00 5-10 Improving SORT Efficiency 5.1.5 1/0 Considerations The input and output files 1/0 and the specification file 1/0 are all performed under the control of VAX-11 RMS record 1/0 facilities. Multi-buffering is used together with read ahead on unit record devices to optimize the 1/0 operations. The work files are processed by the VAXNMS memory management system service, create and map section. Various devices can be used for input and output files. Figure 2-1 shows which devices are allowed for each of the four sorting techniques. Use Figure 2-1 to match the sorting process with the devices that best suit your processing environment. Data may be stored in binary, ASCII, decimal, packed or zoned. 5.2 Tuning Procedure All generalized sorts consider several factors such as: the memory environment (large, small, virtual memory capability or not); the 1/0 devices to be used for work files and their characteristics (speed, arm movement, seek time, public or private units); type of files and data most likely to be sorted (large or small files, large or small records and keys, random or ordered partially, characters or numbers). The algorithm must be very good for the cases occurring most often, and reasonable on all other cases. VAX-11 SORT is designed for an environment of: fairly large files 1 virtual memory capability, random access disk devices, public and private, larger random character data files, medium size records and keys. There are three components of a sort that account for the majority of the processing time: • The number of key comparisons per record per sort. • The number of merge passes needed to complete the sort. • The amount of time spent waiting for and/or doing 1/0 to work files. 5.2.1 User Performance Considerations This section discusses how you can determine the most efficient values for the following SORT performance parameters. • Working set quota • Work file devices • Number of work files • Type of sort (process) • File size Improving SORT Efficiency 5-11 • Record size • Key size • System load • System process parameters 5.2.1.1 Working Set Quota - For SORT to work efficiently the most important parameter is the working set quota (or size) the user decides to choose. The optimum working set quota is the smallest one for which the data can be sorted in memory, that is with 0 merge passes. To compute the appropriate working set quota size, perform the following procedure: Step 1. For any sort, take the size of the key fields added together in bytes. Step 2. Then add 20. Step 3. If the sorting process is record sort, add the number of bytes in the longest record; otherwise add 6. Then multiply by the number of records in the file. This is the total amount of data you have in bytes. Step 4. Divide that number by 512 to get the amount in blocks. Step 5. Multiply by 2 to get the size of the working set quota you should start with. For most larger files the number computed will.be much to large to actually use as a quota. In such cases, the largest reasonable size based on the system load and scheduling considerations is the correct size to use. An individual user's authorized quota is generally the largest reasonable size for the particular system. For example: To sort a 1000 record file with 80-byte records and a total key field size of 80 bytes using the record process, compute the following: 1000 X (80+20+80) = 180,000 bytes of data 180,000/512 = 352 blocks of data 352 X 2 = 704 block working set quota Answer: start with a working set quota of 700. However, if the same type of a file contained 40,000 records, the total amount of data would be 14,063 blocks. For most systems a quota of 28,000 blocks (pages), or even 14,063 is unacceptable. Here the largest reasonable quota should be used; for example 1024 pages. 5-12 Improving SORT Efficiency 5.2.1.2 Work Fiie Devices ~ Another important parameter is where the work files are placed. The fuller the disk and the more activity on the disk containing the work files, the less efficient SORT will be. The optimum configuration would be to have each work file, and the input and output file all on separate empty disks which are only being used by SORT during the sorting process. However, this is seldom possible, so the next best configuration is to place work files on available disks having the lowest activity. See Section 2.7, Setting up the Work Files. 5.2.1.3 Number of Work Flies - Because SORT does not depend on the number of work files used to determine the order of the merge like SORT-11, the advantage of using more than the default number of work files is limited. There are two reasons for using more than the default of two work files: 1) to spread the work files between more than two disks and/or 2) to have each individual file be a smaller size in order to fit onto a smaller or fuller disk. If you are using three or four disks, it will help the sort performance to use three or four work files, one on each disk as discussed above. For example, if you have a 100,000 block file to sort, using two work files would create two 150,000 block files. But, using four work files would create four 75,000 block files that could be placed on disks with less free space. 5.2.1.4 Type of Sort - Although the type of sort used is often dictated more by functionality. required than performance there are significant differences between the sorts. Address sort is the fastest and uses the least temporary disk space. Index sort is only slightly slower than address sort but uses more tempory storage. Tag sort uses the same temporary storage as address sort, but is significantly slower. For large records with small keys it is faster than record sort in smaller memory sizes if the file is not large. Record sort uses a larger amount of temporary storage and is the slowest. 5.2.1.5 Using SORT's Statistics -Analyze the sort statistics (Section 2.2.3) to determine how to improve the sort's performance. The number of records in, out, sorted if not all equal indicates that there were input or output errors, or that there are null records in the file (that is, the number of records read was greater than the number sorted or the number output). This condition can also be caused by some records containing invalid data in the key fields (if less than ten records are in error SORT will continue, otherwise SORT will stop executing). Longest record length value is obtained from either RMS or the user and can be used to make sure the RMS value is correct. Improving SORT Efficiency 5-13 The multl block and buffer counts indicate the amount of 1/0 optimization on the input and output file. The larger the working set quota the more optimization possible. No optimization would show all these counts as 1. This should not occur unless the file is huge compared to the working set quota. If it does, raise the quota if possible. The order of the merge is the number, less one, of merge buffers that the working set is divided into for the merge phase. Number of merge passes and the number of lnltlal runs shows you how close the data is to fitting in memory. The higher these numbers are, particularly the number of passes, the longer SORT takes and the further away the working set size is from containing the data. Virtual memory added is the amount of virtual memory SORT used for the data. Elapsed time is the total wall clock time in hours, minutes, seconds, and 1/100 seconds from start to end for the sort run. The total of the two 1/0 counts are the number of disk hits to get and write data and these will be higher if the multi block and buffer counts are lower. The lower the better. CPU time is the time spent actually processing data minus all 1/0 time. The closer to the elapsed time the better optimization you are seeing in 1/0. Page faults are also a good indication of how well the data did or did not fit into memory. The higher the number of page faults, the less efficient the sort is. 5.2.2 System Manager Performance Considerations The system manager can determine the following SORT performance parameter values based on the overall system usage: number of users, types of process most commonly run, and the amount of real memory available. • System per process working set quota {WSMAX) • System per process virtual page count (VIRTUALPAGECNT) • System per process section count (PROCSECTCNT) • System modified page writer cluster factor (MPW_ WRTCLUSTER) The values recommended are based solely on sort considerations; it is up to the system manager to integrate other system considerations with these in determining the appropriate final values. 5-14 Improving SORT Efficiency 5.2.2.1 Working Set Quota -The maximum for this value should be set to the largest size any sort job would ever require. For very large files, working sets of 500 to 1000 pages are not at all unreasonable, provided the system has enough physical memory to accommodate them. Individual maximums, to prevent users from monopolizing real memory, can be set on a per user basis by using the authorization file. For information on how to determine an appropriate working set for a particular sort job see Section 5.2.1.1. The general rule is, the smaller the working set, relative to the files to be sorted, the slower the sort. 5.2.2.2 · Vlrtual Page Count - For this parameter the current value as well as the maximum value should be set to a minimum of 3 to 4 times the value of the working set quota maximum. When SORT initially starts executing it will request 2 and 1/2 to 3 times the working set quota of virtual memory from the system. If this value is too low SORT will be unable to run in certain cases. 5.2.2.3 Process Section Count - For working set quota maximums of 500 or less this parameter may stay at a minimum level. However, for working set quotas greater than 500 to 1000 a current value of 10 or greater is necessary. If this parameter is set too low, SORT will be unable to run in larger working sets due to internal mapping failures. The value should be increased as the working set quota maximum increases. 5.2.2.4 Modified Page Writer Cluster Factor -The value of this parameter will never cause SORT to fail, however it can cause a large difference in performance. For any larger sorts (that is, using working sets of 250 pages or greater) the larger this parameter, the better. Values of 64 and up are not too large. Be sure to adjust MPW_HILIM and MPW_LOLIMIT·accordingly. For more information refer to the SYSGEN procedures in the VAX-11 Software Instal- lation Guide. Improving SORT Efficiency 5-15 Glossary Alphanumeric Characters The entire set of 128 ASCII characters (see Appendix B). ASCII Character Set The set of 128 eight-bit American Standard Code for Information Interchange characters (see Appendix B). Batch A mode of processing in which all commands to be executed by the operating system and, optionally, data to be used as input to the commands are placed in a file or punched onto cards and submitted to the system for execution. BLISS A high-level system implementation programming language. VAX-11 SORT is written in BLISS. Block The smallest addressable unit of data that the specified device can transfer in an 1/0 operation (512 contiguous bytes for most disk devices). Bucket See File Bucket. Buffer A temporary data storage area in a process address space used when performing input or output operations. Byte The smallest addressable unit of information; eight bits. For example, an ASCII character requires a single byte (see Appendix C for further definitions). Call The operation of invoking a procedure. Caller The procedure that invoked this procedure by a Call. At the time of procedure invocation, the invoking procedure is said to be the caller, and the invoked procedure is the callee. Contrast with User. Glossary-1 Character The smallest addressable unit of usable data (byte). It is also a single letter, numeral, punctuation mark, or other symbol (such as $ or %) , and is represented within the computer as a unique combination of bits. Typically, a character code consists of eight bits. Character String Descriptor A quadword data structure used for describing character data (strings). The first word of the quadword contains the length of the character string. The second word can contain type information. The remaining· longword contains the address of the string. CPU The Central Processor Unit portion of a computer system. Collating Sequence The order into which characters are sorted based upon numeric values assigned to each. Command An instruction, generally an English word, typed by the user at a terminal or included in a command. file, which requests the software monitoring a terminal or reading a command file to perform some well-defined activity. For example, typing the SORT command request the system to invoke the SORT utility. Command Fiie A file containing command strings. See also Command Procedure. Command Interpreter Procedure-based system code that executes in supervisor mode in the context of a process to receive, syntax check, and parse commands typed by the user at a terminal or submitted in a command file. Command Parameter The positional operand of a command delimited by spaces, such as a file specification, option, or constant. Command Procedure A file containing commands and data that the command interpreter can accept in lieu of the user typing the commands individually on a terminal. Glossary-2 Command String A line (or set of continued lines), normally terminated by typing the carriage return key, containing a command and, optionally, information modifying the command. A complete command string consists of a command, its qualifiers, if any, and its parameters (file specifications, for example), if any, and their qualifiers, if any. Compatlblllty Mode A mode of execution that enables the central processor to execute non-privileged PDP-U instructions. The operating system supports compatibility mode execution by providing an RSX-UM programming environment for an RSX-UM task image. The operating system compatibility mode procedures reside in control region of the process executing a compatibility mode image. The procedures intercept calls to the RSX-UM executive and convert them to the appropriate operating system functions. Contiguous Blocks Physically adjacent and/or consecutively numbered blocks of data. Data Fiie Record A record containing user data. Data Structure Any table, list, array, queue, or tree whose format and access conventions are well defined for reference by one or more images. Data Type In general, the way in which bits are grouped and interpreted. In reference to the processor instructions, the data type of an operand identifies the size of the operand and the significance of the bits in the operand. Operand data types include: byte, word, longword, and quad-word integer, floating and double floating, character string, packed decimal string, and variable-length bit field (see Appendix C). DCL Digital Command Language (DCL) is a set of English- like statements that a user types to initiate and control system operations. Default An assumed value supplied to the system when a command qualifier does not specifically override the normal command function; fields in a file specification that the system fills in when the specification is not complete. Descriptor See Character String Descriptor. Glossary-3 Device The general name :for any physical terminus or link connected to the processor that is capable of receiving, storing, or transmitting data. Card readers, line printers, and terminals are examples of record-oriented devices. Magnetic tape devices and disk devices are examples of mass storage devices. Terminal line interfaces and interprocessor links are examples .of communications devices. Directory A file used to locate files on a volume that contains a list of file names (including extension and version number) and their unique internal identifications. Directory Name The field in a file specification that identifies the directory file in which a file is listed. The directory name is enclosed in brackets ( [] or <>). Fie Id A logically distinguishable area within a record. Usually a logical unit of data. Fiie A logically related collection of data on a volume such as disk or magnetic tape. A file can be referenced by a name assigned by the user. A file normally consists of one or more logical records. Fiie Bucket Within the RMS Relative File organization, a bucket is a storage structure of one to 32 blocks of data. Fiie Header A block in the index file describing a file on a FILES-11 disk structure. The file header identifies the locations of the file's extents. There is a file header for every file on the disk. Fiie Organization The particular file structure used to record the data constituting a file on a mass storage medium. RMS file organizations are: Sequential, Relative, and Indexed. Fiie Prologue The first block in a relative or indexed file which contains header information for the file. File Specification A unique name for a file on a mass storage medium. It identifies the node, the device, the directory name, the file name, and the version number under which a file is stored (see Appendix D for additional information). Glossary-4 Fiie Structure The way in which the blocks forming a file are distributed on a disk or magnetic tape to provide a physical accessing technique suitable for the way the data in the file is processed. Fiie System A method of recording, cataloging, and accessing files on a volume. Fiie Type The field in a file specification that is preceded by a period or dot(.) and consists of a zero-to three-character type identification. By convention, the type identifies a generic class of files that have the same use or characteristics, such as ASCII text files, binary object files, etc. Flles-11 The standard physical disk structure used by VAX-11 RMS. Flies pee Fi.le Specification that uniquely identifies a file by physical location (see Appendix D). Fiie, Input See Input File. Fiie, Output See Output File. Fiie, Work See Work File. Fixed Control Area An area associated with a variable length record available for controlling or assisting record access operations. Typical uses include line numbers and printer format control information. Fixed Position Fleld An area associated with character position (or column numbers). Used in SORT-11 Specification Files. Fixed Length Record Format A file format in which all records have the same length. Glossary-5 Format The arrangement of any record or file; the order in which fields reside in a record. Free Fields Logically positioned fields separated by commas. Contrast with fixed position fields. Home Block A block in the index file that contains the volume identification, such as volume label and protection. Image A file consisting of procedures and data that have been bound together by the linker. There are three types of images: Executable, Shareable, and System. Indexed Fiie Organization A file organization in which a file contains records and a primary key index (and optionally one or more alternate key indices) used to process the records sequentially by index or randomly by index. Index Fiie The file on a Files-11 volume that contains the access information for all files on the volume and enables the operating system to identify and access the volume. Index Fiie Bit Map A table in the index file of a Files-11 volume that indicates which file headers are in use. Index Fiie Record A record of file system data that is invisible to the user. Input File The file containing the records you wish to sort. Key, Key Fie Id The data field containing the values chosen from a record to control the sort (see section 2.6). Key, Major The most important field in the total key. If you were sorting a list by department, salary and name, department would be the major key. Glossary-6 Key, Minor The least significant field in the total key. In the preceding example name is the minor key. Library A collection of commonly used files. Line In this document, a line generally refers to a line in the SORT specifications form or a record in the specification file. Logical Name A user-specified name for any portion or all of a file specification. For example, the logical name INPUT can be assigned to a terminal devic_e from which a program reads data entered by a user. Logical name assignments are maintained in logical name tables for each process, each group, and the system. A logical name can be created and assigned a value permanently or dynamically. Longword Four contiguous bytes (32 bits) starting on an addressable byte boundary (see Appendix C). LRL Longest Record Length (LRL) specified in bytes. Merge A process by which two or more ordered groups of records are put together record-byrecord into a single identically ordered group. Native Mode The processor's execution mode, in which the programmed instructions are interpreted as byte-aligned, variable length instructions that operate on byte, word, longword, quadword integ·er, floating and double floating, character string, packed decimal, and variable length bit field data. The instruction execution mode other than compatibility mode. Node An individual computer system in a network. Output Fiie The file created by running SORT. The output file may be either a data file or an address file. Glossary-7 Page 1). A set of 512 contiguous byte locations used as the unit of memory mapping and protection. 2). The data between the beginning of file and a page mark~r, between two markers, or between a marker and the end of a file. Page Fault An exception generated by a reference to a page which is not mapped into a working set. Pager A set of kernel mode procedures that executes as the result of a page fault. The pager makes the page for which the fault occurred available in physical memory so that the image can continue execution. The pager and the image activator provide part of the operating system's memory management functions. Paging The action of bringing pages of an executing process into physical memory when referenced. When a process executes, all of its pages are said to reside in virtual memory. Only the actively used pages, however, need to reside in physical memory. In this system, a process is paged only when it references more pages than it is allowed to have in its working set. When a process refers to a page not in its working set, a page fault occurs. This causes the operating system's pager to read in the referenced page fault if it is on disk (and optionally, other related pages depending on a cluster factor), replacing the least recently faulted pages as needed. This system only pages a process against itself. Packed Decimal A method for compact storage of numeric data; two digits are stored in each 8-bit byte and the sign resides in the last byte of the low-order digit. Parse To break down into individual parts from a whole. Physical Memory The memory modules contained within the CPU. Also called main memory. Procedure A routine that follows the VAX-11 calling sequence standard. A procedure may return values via the argument list and/or the standard value return registers. Contrast with routine. Process The basic entity scheduled by the system software that provides the context in which an image executes. A process consists of an address space and both hardware and software context. Glossary-8 Program A program is the basic entity that is executed by the processor. Each program consists of a set of procedures and its execution represents a distinct activity that is potentially concurrent with others in the system. Prologue See File Prologue. Quadword Eight contiguous bytes (64 bits) starting on an addressable byte boundary. See Appendix C. Quallfler A portion of a command string that modifies a command verb or command parameter by selecting one of several options. A qualifier, if present, follows the command verb or parameter to which it applies and is in the format: "/qualifier=option". For example, in the command string "PRINT filename/ COPIES=3", the COPIES qualifier indicates that the user wants three copies of a given file printed. Random acces~ by record's flle address The retrieval of a record by its unique address, which is provided to the program by RMS. The· method of access can be used to randomly access a sequentially organized file containing variable length records. Random access by relative record number The retrieval or storagE~ of a record by specifying its position relative to the beginning of the file. Real Memory See Physical Memory. Record The unit of information in a file; a group of related fields treated as a logical unit. Record Cell A fixed length area in a relatively organized file that is used to contain one record. Record Management Services (RMS) A set of system procedures in the operating system that are called by programs to p'rocess files and records within files. RMS allows programs to issue GET and PUT requests at the record level (record 1/0) as well as read and write blocks (block 1/0). RMS is an integral part of the system software. RMS procedures run in executive mode. Glossary-9 Record-oriented Device A device such as a terminal, line printer, or card reader, on which the largest unit of data that a program can access is the device's physical record. Record's Fiie Address (RFA) The unique address of a record in a file that allows records to be accessed randomly regardless of file organization. Record, Data Fiie See Data File Record. Record, Field Specification See Section 2.5.2 Record, Header See Section 2 .5 .2 Record, Index Fiie See Index File Record. Relative Fiie Organization A file organization in which the file contains fixed length record cells. Each cell is assigned a consecutive number that represents its position relative to the beginning of the file. Records within each cell can be the same length or smaller than the cell. Relative file organization permits sequential record access, random record access by record number, and random record access by record's file address. RMS See Record Management Services (RMS). Routine A sequence of instructions that performs a well defined action. It may have multiple entry points. For example, the SIN routine has SIN and COS entry points. A routine that follows the VAX-11 calling sequence standard is termed a procedure. Sequential Fiie Organization A file organization in which records appear in the order in which they were originally written. The records can be fixed length or variable length. Although one does not speak of record cells with sequentially organized files, for purposes of comparison with relatively organized files one can say that the record itself is the same as its record cell, and its record number is the same as its relative cell number. Sequential file organization permits sequential record access and random record access by record's file address. Sequential file organization with fixed length records also permits random access by relative record number. Glossary-IO SORT Utlllty A processing program that can be used to sort records by keys into a prescribed sequence. To segregate items into groups according to some definite rules. Sort Tree ,J A data structure used to keep order of records by sort. Subroutine A procedure that does not return a known value in the value registers. If values are returned, they are returned via the argument list. By convention, the function "value" is unpredictable. System Device The device on which the Executive resides. Terminal The general name for those peripheral devices that have keyboards and video screen or printers. Under program control, a terminal enables people to type commands and data on a keyboard and receive messages on a video screen or printer. Examples of terminals are the LA3H DECwriter (hard-copy terminal) and the VT52 video display terminal (soft-copy terminal). Unit Record Device See Record-oriented Device. User The person who is directly using the computer, either via terminal or batch input. Contrast with Caller. Variable-length Record A record format in which records need not be the same length. Variable with fixed-length control (VFC) record A record format in which records of variable length contain an additional fixed-length control area. The control area may be used to contain file line numbers and/or print format control characters. Virtual Address A 32-bit integer identifying a byte "location" in virtual address space. The memory management hardware translates a virtual address to a physical address. The te'rm virtual address may allso refer to the address used to identify a virtual block on a mass storage device. Glossary-11 Virtual Memory The set of storage locations in physical memory and on disk that are referred to by virtual addresses. From the programmer's viewpoint, the secondary storage locations appear to be locations in physical memory. The size of virtual memory in any system depends on the amount of physical memory available and the amount of disk storage used for non-resident virtual memory. Volume A mass storage medium such as a disk pack or reel of magnetic tape. Wiid card The asterisk character when used as a substitute parameter in file specification indicates "all" for a given field. Word Two contiguous bytes (16 bits) starting on an addressable byte boundary (see Appendix C). Work Fiie A collection of sortE~d records created during the processing cycle and released after the sort is finished. (Sometimes called Scratch Files.) Working Set The set of pages in process address space to which an executing process can refer without incurring a page fault. The working set must be resident in memory for the process to execute. The remaining pages of that process, if any, are either in memory and not in the process working set or they are on secondary storage. Zoned Numeric Format A specific ASCII coded decimal data type where the number sign and the least significant digit are combined into a single hexadecimal code (see Appendix C). Glossary-12 Appendix A Octal/Hexadecimal/Decimal Conversion A.1 Octal/Declmal Conversion To convert a number from octal to decimal, locate in each column of the table the decimal equivalent for the octal digit in that position. Add the decimal equivalents to obtain the decimal number. To convert a decimal number to octal: 1. locate the largest decimal value in the table that will fit into the decimal number to be converted, 2. note its octal equivalent and column position, 3. find the decimal remainder. Repeat the process on each remainder. When the remainder is 0, all digits will have been generated. 82 83 84 85 81 go 0 0 0 0 0 0 0 1 32,768 4,096 512 64 8 1 2 65,536 8,192 1,024 128 16 2 3 98,304 12,228 1,536 192 24 3 4 31,072 16,384 2,048 256 32 4 5 163,840 20,480 2,560 320 40 5 6 16£1,608 24,576 3,072 384 48 6 7 22£1,376 28,672 3,584 448 56 7 A-1 A.2 Powers of 2 and 16 Powers of 16 Powers of 2 2**n n 16**n n 256 512 1024 2048 4096 8192 16384 32768 65536 131072 262144 524288 1048576 2094304 4194304 8388608 16777216 8 9 10 1 16 256 4096 65536 1048576 16777216 268435456 4294967296 68719476736 1099511627776 17592186044416 281474976710656 4503599627370496 72057594037927936 1152921504606846976 0 1 2 3 4 11 12 13 14 15 16 17 18 19 20 21 22 23 24 5 6 7 8 9 10 11 12 13 14 15 A.3 Hexadecimal to Decimal Conversion For each integer position of the hexadecimal value, locate the corresponding column integer and record its decimal equivalent in the conversion table A.5. Add the decimal equivalent to obtain the decimal value. Example: D0500ADO Cl G > = ?< 10) 00000000 500000 AOO DO = = = = 3,499,sso,920 5,zaz,sso 2 , 580 208 D0500ADO = 3,a9a,9oa,s7s A.4 Decimal to Hexadecimal Conversion 1. Locate in the conversion table A.5 the largest decimal value that does not exceed the decimal number to be converted. 2. Record the hexadecimal equivalent followed by the number of zeros (0) that corresponds to the integer column minus one. " 3. Subtract the table decimal value from the decimal number to be converted. 4. Repeat steps 1-3 until the subtraction balance equals zero (0). Add the hexadecimal equivalents to obtain the hexadecimal value. A-2 Octal/Hexadecimal/Decimal Conversion Example: = = = = = = = 22t4GG ( 1 0 ) 20t480 1 t792 192 2 ------ 22t4GG ? ( 1 G> 5000 700 22t4GG -20t480 2 1 t98G - 1 t 792 co 57C2 19£1 192 2 2 0 A.5 Hexadecimal Integer Columns 8 DEC HEX 0 1 2 3 4 5 6 7 8 9 0 268,435,456 536,870,912 805,306,368 1,073,741,824 1,342, 177 ,280 1,610,612, 736 1,879,048,192 2,147,483,643 2,415,919, 104 2,684,354,560 2,952, 790,016 3,221,225,472 3,489,660,928 3, 758,096,384 4,026,531,840 0 1 2 3 4 5 6 7 8 9 A B c D E F \ 6 7 HEX A B c D E F v DEC 5 DEC 0 0 1,048,576 2,097,152 3,145,728 4,194,304 5,242,880 6,291,456 7,340,032 8,338,608 9,437,184 10,485,760 11,534,336 12,582,912 13,631,488 14,680,064 15,728,640 0 16,777,216 33,554,432 50,331,648 67,108,864 83,886,080 100,663,296 117 ,440,512 134, 217' 728 150,994,944 167,772,160 184,549,376 201,326,592 218, 103,808 234,881,024 251,658,240 I 2 3 4 5 6 7 8 9 A B c D E F \ 0 2 3 4 5 6 7 8 9 A B c D E F HEX 0 65,536 lill,072 196,608 262, 144 :m,680 ::193,216 458,752 fi24,288 fi89,824 . 6fifi,360 720,896 786,432 851,968 917,504 983,040 v 0 1 2 3 4 5 6 7 8 9 A B c D E F I \ I \ BYTE BYTE \ v- \ WORD 3 4 HEX 1 DEC HEX DEC 2 HEX DEC HEX 0 1 2 3 4 5 6 7 8 9 0 256 512 768 1,024 1,280 1,536 1,792 2,048 2,304 2,560 2,916 3,072 3,328 3,584 3,840 0 1 2 3 4 5 6 7 8 9 0 4,096 8,192 12,288 16,384 20,480 24,576 28,672 32,768 36,864 40,960 45,056 49,152 53,248 57,344 61,440 A B c D E F v I BYTE A B c D E F ' 0 16 32 48 64 80 96 112 128 144 160 176 192 208 224 240 0 2 3 4 5 6 7 8 9 A B c D E F v 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 I BYTE I v WORD =v 1 DEC HEX DEC I LONGWORD Octal/Hexadecimal/Decimal Conversion A-3 Appendix B The ASCII Chara.cter Set Collating Sequence ASCII Character Hexadecimal Number ASCII 8--Bit Octal Decimal ASCII Character Hexadecimal Number ASCII 8-Bit Octal Decimal NUL 00 000 0 FS lC 034 28 SOH 01 001 1 GS 1D 035 29 STX 02 002 2 RS lE 036 30 ETX 03 003 3 us lF 037 31 EOT 04 004 4 SP 20 040 32 ENQ 05 005 5 ! 21 041 33 ACK 06 006 6 II 22 042 34 BEL 07 007 7 # 23 043 35 BS 08 010 8 $ 24 044 36 HT 09 011 9 % 25 045 37 LF OA 012 10 & 26 046 38 VT OB 013 11 27 047 39 FF oc 014 12 ( 28 050 40 CR OD 015 13 ) 29 051 41 so OE 016 14 * 2A 052 42 SI OF 017 15 + 2B 053 43 DLE 10 020 16 2C 054 44 DCl 11 021 17 2D 055 45 DC2 12 022 18 2E 056 46 DC3 13 023 29 I 2F 057 47 DC4 14 024 20 0 30 060 48 NAK 15 025 21 1 31 061 49 SYN 16 026 22 2 32 062 50 ETB 17 027 23 3 33 063 51 CAN 18 030 24 4 34 064 52 EM 19 031 25 5 35 065 53 SUB lA 032 26 6 36 066 54 ESC 1B 033 27 7 37 067 55 ' - (continued on next page) B-1 ·~-..tt.<'<·,jl;;;;o',, ,,~' Hexadecimal Number ASCII 8-Bit Octal Decimal \ 5C 134 92 57 1 50 135 93 072 58 /\ 5E 136 94 3B 073 59 - 5F 137 95 < 3C 074 60 60 140 96 = 30 075 61 a 61 141 97 > 3E 076 62 b 62 142 98 ? 3F 077 63 c 63 143 99 @ 40 100 64 d 64 144 100 A 41 101 65 e 65 145 101 B 42 102 66 f 66 146 102 c 43 103 67 g 67 147 103 0 44 104 68 h 68 150 104 E 45 105 69 i 69 151 105 F 46 106 70 j 6A 152 106 G 47 107 71 k 6B 153 107 H 48 110 72 I 6C 154 108 I 49 111 73 m 60 155 109 J 4A 112 74 n 6E 156 110 K 4B 113 75 0 6F 157 111 L 4C 114 76 p 70 160 112 M 40 115 77 q 71 161 113 N 4E 116 78 r 72 162 114 0 4F 117 79 s 73 163 115 p 50 120 80 t 74 164 116 Q 51 121 81 u 75 165 117 R 52 122 82 v 76 166 118 s 53 123 83 w 77 167 119 T 54 124 84 x 78 170 120 u 55 125 85 y 79 171 121 v 56 126 86 z 7A 172 122 w 57 127 87 I 7B 173 123 x 58 130 88 I 7C 174 124 y 59 131 89 I 7D 175 125 z 5A 132 90 - 7E 176 126 [ 5B -· 133 91 DEL 7F 177 127 Hexadecimal Number ASCII 8-Bit Octal Decimal' 8 38 070 56 9 39 071 : 3A j ASCII Character B-2 ASCII Character l Characters Set ASCII Collating Sequence Appendix C Data Types The data type refers to the way in which bits are grouped and interpreted. In reference to the processor instructions, the data type of an operand identifies the size of the eiperand and the significance of the bits in the operand. Data types applicable to SORT and its associated VAXNMS programs are: separated into threE! classes; character, binary, and decimal. These classes can be subdivided into data types of different sizes and formats such as; byte, word, longword, quad.word, floating, double floating, character string, packed decimal string, and variable-length bit field. C.1 Byte A byte is 8 contiguous bits starting on an addressable byte boundary. The bits are numbered from the right 0 through 7: 7 0 c__] :A A byte is specified by its address A. When interpreted arithmetically, a byte is a twos complement integer with bits of increasing significance going 0 through 6 and bit 7 the sign bit. The value of the integer is in the range -128 through 127. For the purposes of addition, subtraction, and comparison, VAX-11 instructions also provide direct support for the interpretation of a byte as an unsigned integer with bits of increasing significance going 0 through 7. The value of the unsigned integer is in the range 0 through 255. C.2 Word A word is 2 contiguous bytes starting on an arbitrary byte boundary. The bits are numbered from the right 0 through 15: 1 5 [, _ __ 0 :A A word is specified by its address A, the address of the byte containing bit 0. When interpreted arithmetically, a word is a twos complement integer with bits of increasing significance going 0 through 14 and bit 15 the sign bit. The value of the integer is in the range -32, 768 through 32, 767. For the purposes of C-1 addition, subtraction and comparison, VAX-11 instructions also provide direct support for the interpretation of a word as an unsigned integer with bits of increasing significance going 0 through 15. The value of the unsigned integer is in the range 0 through 65,535. C.3 Longword A longword is 4 contiguous bytes starting on an arbitrary byte boundary. The bits are numbered from the right 0 through 31: 3 0 '-----------·--~ :A A longword is specified by its address A, the address of the type containing bit 0. When interpreted arithmetically, a longword is a twos complement integer with bits of increasing significance going 0 through 30 and bit 31 the sign bit. The value of the integer is in the range -2,147,483,648 through 2,147,483,647. For the purposes of addition, subtraction, and comparison, VAX-11 instructions also provide direct support for the interpretation of a longword as an unsigned integer with bits of increasing significance going 0 through 31. The value of the unsigned integer is in the range 0 through 4,294,967,295. Note that the longword format is different from the longword format defined by the PDP--11 FP-11. In that format, bits of increasing significance go from 16 through 31 and 0 through 14. Bit 15 is the sign bit. Most DIGITAL software and in particular PDP-11 FORTRAN and COBOL use the VAX-11 longword format. C.4 Quadword A quadword is 8 contiguous bytes starting on an arbitrary byte boundary. The bits are numbered from the right 0 through 63: 3 0 E 6 3 ~ :A :A+4 3 2 A quadword is specified by its address A, the address of the byte containing bit 0. When interpreted arithmetically, a quadword is a twos complement integer with bits of increasing significance going 0 through 62 and bit 63 the sign bit. The value of the integer is in the range -2**63 to 2**63--1. The quadword data type is not fully supported by VAX-11 instructions. C-2 Data Types C.5 Floating A floating datum is 4 contiguous bytes starting on an arbitrary byte boundary. The bits are labelled from the right 0 through 31. 1 1 1 6 5 4 3 fraction Isl 7 6 exp 0 L_ -~raction :A ] A floating datum is specified by its address A, the address of the byte containing bit 0. The form of a floating datum is sign magnitude with bit 15 the sign ·bit, bits 14:7 an excess 128 binary exponent, and bits 6:0 and 31:16 a normalized 24--bit fraction with the redundant most significant fraction bit not represented. Within the fraction, bits of increasing significance go from 16 through 31and0 through 6. The 8-bit exponent field encodes the values 0 through 255. An exponent value of 0 together with a sign bit ofO, is taken to indicate that the floating datum has a value of 0. Exponent values of 1 through 255 indicate true binary exponents of -127 through +127. An exponent value of 0, together with a sign bit of 1, is taken as reserved. Floating point instructions processing a reserved operand take a reserved operand fault (See Chapter 4 and 6). The value of a floating datum is in the approximate range .29*10**-38 through 1. 7*10**38. The precision of a floating datum is approximately one part in 2**23, that is, typically 7 decimal digits. C.6 Double Floating A double floating datum is 8 contiguous bytes starting on an arbitrary byte boundary. The bits are labelled from the right 0 through 63: 3 1 1 1 6 5 4 fraction sJ fraction 6 3 exp 7 6 0 l ~ fraction 4 4 8 7 :A :A+4 3 2 A double floating datum is specified by its address A, the address of the byte containing bit 0. The form of a double floating datum is identical to a floating datum except for an additional 32 low significance fraction bits. Within the fraction, bits of increasing significance go 48 through 63, 32 through 47, 16 through 31, and 0 through 6. The exponent conventions, and approximate range of values is the same for double floating as floating. The precision of a double floating datum is approximately one part in 2**55, that is, typically 16 decimal digits. Data Types C-3 C. 7 Variable Length Bit Fleld A variable bit field is 0 to 32 contiguous bits located arbitrarily with respect to byte boundaries. A variable bit field is specified by 3 attributes: the address A of a byte, a bit position P which is the starting location of the field with respect to bit 0 of the byte at A, and a size S of the field. The specification of a bit field is indicated by the following where the field is the shaded area. P+S P+S-1 P P-1 0 _=1.:A S-1 0 The position is in the range -2**31 through 2**31-1 and is conveniently viewed as a signed 29-bit offset and a 3-bit bit-within-byte (BWB) field: 3 3 2 0 --~~------~~~b-y-te_of-fs_e_t___________,____~____I_. __:~~ The sign extended 29-bit by~e offset is added to the address A and the resulting address specifies the byte in which the field begins. The 3-bit bit-withinbyte field encodes the starting position (0 through 7)· of the field within that byte. The VAX-11 field instructions provide direct support for the interpretation of a field as a signed or unsigned integer. When interpreted as a signed integer, it is twos complement with bits of increasing significance going 0 through S-2; bit S-1 is the sign bit. When interpreted as an unsigned integer, bits of increasing significance go from 0 to S-1. A field of size 0 has a value identically equal to 0. A variable bit field may be contained in 1to5 bytes. From a memory management point of view, only the minimum number of bytes necessary to contain the field is actually referenced. C-4 Data Types C.8 Character String A character string is a contiguous sequence of bytes in memory. A character string is specified by 2 attributes: the address A of the first byte of the string, and the length L of the string in bytes. Thus the format of a character string is: 0 7 :A :A+L-1 7 0 The address of a string specifies the first character of a string. Thus 11 XYZ 11 is represented: "X" :A "Y" :A+1 "Z" :A+2 The length L of a string is in the range 0 through 65,535. C.9 Tralllng Numeric String A trailing numeric string is a contiguous sequence of bytes in memory. The string is specified by 2 attributes: the address A of the first byte (most significant digit) of the string, and the length L of the string in bytes. All bytes of a trailing numeric string, except the least significant digit byte, must contain an ASCII decimal digit character (0-9). The representation for the high order digits is: digit decimal hex 0 48 49 50 51 52 53 54 55 56 57 30 31 32 33 34 35 36 37 38 39 1 2 3 4 5 6 7 8 9 ASCII character 0 1 2 3 4 5 6 7 8 9 Data Types C-5 The highest addressed byte of a trailing numeric string represents an encoding of both the least significant digit and the sign of the numeric string. The VAX numeric string instructions support any encoding; however, there are 3 preferred encodings used by DIGITAL software. Thes~ are (1) unsigned numeric in which there is no sign and the least significant digit contains an ASCII decimal digit character, (2) zoned numeric, and (3) overpunched numeric. Because the overpunch format has been used by compilers of many manufacturers over many years, and because various card encodings are used, several variations in overpunch format have evolved. Typically, these alternate forms are accepted on input. The valid representations of the digit and sign in each of the later two formats is: Representation of Least Significant Digit and Sign Zoned Numeric Format digit 0 1 2 3 decimal 48 49 50 51 52 53 54 55 56 57 112 113 114 115 116 117 118 119 120 121 4 5 6 7 8 9 -0 -1 -2 -3 -4 -5 -6 -7 -8 -9 hex 30 31 32 33 34 35 36 37 38 39 70 71 72 73 74 75 76 ASCII char 0 1 2 3 4 5 6 7 8 9 p q r s t u v 77 w 78 79 x y ~ Overpunch Format decimal 123 65 66 67 68 69 70 71 72 73 125 74 75 76 ASCII char norm alt. hex 7B 41 42 43 44 45 46 47 48 49 7D 4A 4B 4C 4D 4E 4F 50 51 52 77 78 79 80 81 82 [? I A B a b c d e f g h i c D E F G H I I ]!: J K L M N 0 j k 1 m n 0 p p q r Q R The length L of a trailing numeric string must be in the range 0 to 31 (0 to 31 digits). The value of a 0 length string is identically 0. The address A of the string specifies the byte of the string containing the most significant digit. Digits of decreasing significance are assigned to increasing addresses. Thus "123" is represented: Zoned Format or Unsigned 4 3 7 C-6. Data Types 0 Overpunch Format 4 3 7 0 3 1 :A 3 1 :A 3 2 :A+1 3 2 :A+1 3 3 :A+2 4 3 :A+2 and "-123" is represented: Overpunch Format Zoned Format 43 7 43 7 0 0 3 1 :A 3 1 3 2 :A+1 3 2 7 3 :A+2 4 c :A :A+2 C.10 Leading Separate Numeric String A leading separate numeric string is a contiguous sequence of bytes in memory. A leading separate numeric string is specified by 2 attributes: the address A of the first byte (containing the sign character), and a length L, which is the length of the string in digits and NOT the length of the string in bytes. The number of bytes in a leading separate numeric string is L+l. The sign of a separate leading numeric string is stored in a separate byte. Valid sign bytes are: Sign decimal hex ASCII character + + 43 32 45 2B 20 2D <blank> + The preferred representation for 11 + 11 is ASCII "+". All subsequent bytes contain an ASCII digit character: digit decimal hex ASCII character 0 1 2 3 48 49 50 51 52 53 54 55 56 57 30 31 32 33 34 35 36 37 38 39 0 4~ e:· d {; 7 8 H 1 2 3 4 5 6 7 8 9 The length L of a leading separate numeric string must be in the range 0 to 31 (0 to 31 digits). The value of a 0 length string is identically 0. Data Types C-7 The address A of the string specifies the byte of the string containing the sign. Digits of decreasing significance are assigned to bytes of increasing addresses. Thus "+123" is: · 4 3 7 0 2 B :A 3 1 :A+1 3 2 :A+2 3 3 :A+3 and "-123" is: 0 43 7 2 D 3 1 3 2 3 3 :A . C.11 Packed Decimal String A packed decimal string is a contiguous sequence of bytes in memory. A packed decimal string is specified by 2 attributes: the address A of the first byte of the string and a length L which is the number of digits in the string and NOT the length of the string in bytes. The bytes of a packed decimal string are divided into 2 4-bit fields (nibbles) which must contain decimal digits except the low nibble (bits 3:0) of the last (highest addressed) byte which must contain a sign. The representation for the digits and sign is: C-8 Data Types digit or sign decimal hex 0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 9 9 9 + 10,12,14, or 15 11 .or 13 A,C,E, or F B, or D The preferred sign representation is 12 for 11 + 11 and 13 for 11 - 11 • The length L is the number of digits in the packed decimal string (not counting the sign) and must be in th~~ range 0 through 31. When the number of digits is odd, the digits and' the sign fit in L/2 (integer part only) + 1 bytes. When the number of digits is even, it is required that an extra 11 0" digit appear in the high nibble (bits 7:4) of the first byte of the string. Again the length in bytes of the string is L/2 + 1. The address A of the string specifies the byte of the string containing the most significant digit in its high nibble. Digits of decreasing significance are assigned to increasing byte addresses and from high nibble to low nibble within a byte. Thus 11 +123" has length 3 and is represented: 7 4 3 0 CGJ:A ~:A+1 and 11 -12 11 has length 2 and is represented: 7 4 3 0 0 1 :A 2 13 :A+1 Data Types · C-9 Appendix D Data Structures and Basic Concepts This Appendix. provides beginning users with additional information regarding the following topics: • VAX-11 RMS data files, records, and structures • File specification parameters • Programming languages D.1 Data Flies, Records, and Structures D.1.1 Data Hierarchy Four level data hierarchy (character, field, record, file) is shown in Figure D.1: Figure D.1: Data Hierarchy m I I character I Adams, Jotii!]J field .,,,,.., ,..., } $175 l . . --- . . ---= Adams r638-36-100& / record -- - A file is a collection of related information. For example, a file might contain a company's personnel information (employee names, addresses, and job titles). Within this file, the information is divided into records. All the information on a single employee might constitute a single record. Each record in the personnel file would be subdivided into discrete pieces of information known as fields. By specifying key fields (Keys) in a particular order, you can sort entire records into any order. Using VAX-11 SORT you can retrieve records in ascending or descending order by ordered key fields (that is, create sorted data files); and you can create sorted address files for random record retrieval by user programs. D-1 D.1.2 Record Types VAX-11 SORT processes three diff~rent types of records: Fixed, Variable, and Variable with fixed-length control (VFC). Figure D.2 summarizes these record types. Figure D.2: Record Types (fixed, variable, VFC) FIXED LENGTH 32-BYTE RECORDS 24 bytes 8 bytes ·------ \..__ _ _ RECORD 1 * - - - - ' \----RECORD 2* _ __./'-.~--RECORD 3* - - *NOTE: VAX-11 RMS considers all 32 bytes to be used, even though they may not contain useful Information In the eyes of the user. VARIABLE LENGTH RECORDS ON MAGTAPE 4-Byte Count Fields I 8 lbytes I 16 bytes I 2-Byte Count Fields i:~4 bytes . .........-------~~~--....... '----./ '----1'RECORD 1 RECORD 2 VARIABLE LENGTH RECORDS ON DIS~· RECORD 3 I ~ ....__b_y_te_s__.,....!. w--___ \..._/\ RECORD 1 RECORD 2 I\ I RECORD 3 VARIABLE WITH FIXED-LENGTH CONTROL RECORD (VFC) Count field Fixed control area portion -===i '""----RECORD 1 _ _ _ _ _/ F-MK-00026-00 D-2 Data Structures and Basic Concepts D.1.3 VAX-11 RMS Fiie Organizations 1. Sequential Flies (see Figure D.3) Sequential files may contain the follow- ing record types: a. Fixed LEmgth Records b. Variable Length Records c. Variable with Fixed-length Control (VFC) Records The order of the records in a sequential file is determined by the order in which the records were originally written to the file. The first record in the file is the first record read; the second record next, and so on.Sequential files are the only files permitted for magnetic tape and unit record devices. They are also permitted for disk. 2. Relative Flies - Records may be any type (that is, fixed, variable, or VFC) as long as the maximum record length is specified. Each record is numbered 1 ton relative to the beginning of file (as shown in Figure D.4). A relative file consists of record areas (cells) that are identified by relative record numbers. The first record area in the file is record number 1, the second is 2, and so on. Empty or null records are permitted. Relative files can reside only on disk. Relative file considerations: • Most efficient random access in terms of speed and storage space overhead. • Addresses of records (relative record numbers) must be known to process file randomly. • Requires storage space to contain all record positions from record number one to highest record number stored in file. • Records can span blocks, but cannot span buckets. • Can be write shared. 3. Indexed FHes - Contain one or more indices, as well as data records. Records can be of any type (that is, fixed, variable, or VFC) as long as the maximum :record length is specified. To retrieve information, you ask for the proper record by primary or alternate key. The system looks up the key in the appropriate index and retrieves the record using the record pointer associated with the key. Indexed files can reside only on disk. The location of records in the indexed file organization is transparent to the program. RMS completely controls the placement of records in an indexed fih~. The presence of keys in the records of the file governs this placement. Data Structures and Basic Concepts D-3 A key is a field present in every record of an indexed file. The location and length of this field are identical in all records. When creating an indexed file, the user decides which field or fields in the file's records are to be a key. Selecting such fields indicates to RMS that the contents (that is, key value) of those- fields in any particular record written to the file can be used by a program to identify that record for subsequent retrieval. At least one key must be defined for an indexed file: the primary key. Optionally, additional keys or alternate keys can be defined. An alternate key value can also be used as a means of identifying a record for retrieval. As programs write into an indexed file, RMS builds a tree-structured table _known as an index. An index consists of a series of entries containing a key value copied from a record that a program wrote into the file. Stored with each key value is a pointer to the location in the file of the record from which the value was copied. RMS builds and maintains a separate index for each key defined for the file. Each index is stored in the file. Thus, every indexed file contains at least one index, the primary key index. Figure D.5 shows an RMS indexed file organization with a primary key. When alternate keys are defined, RMS builds and stores an additional index for each alternate key. Index file considerations: • Multi-key indexed sequential capability. • Most flexible in terms of how a record is accessed. • A record is addressed by the contents of a field in the record (the key field). • Records can be retrieved sequentially in a collated order by key field. • Requires the most storage overhead (that is, the RMS index tree structure). • Index records consist of block numbers, byte-in-block numbers and key. • Can be write shared. All VAX-11 RMS files have two additional blocks in the directory. These additional blocks contain information relating the type of RMS file and the record length. D-4 Data Structures and Basic Concepts Figure D.3: Sequential Files END OF FILE~--- RECORD RECORD RECORD RECORD RECORD RECORD RECORD RECORD Q-MK-00027-00 Figure D.4: Relative Files CELL NO. L1 2 3 4 999 5 1000 RECORD 999 I~.--------------------------BUCKET*------------------~--~-~1 . *A bucket is a storage structure of 1 to 32 blocks. Q-M K-00028-00 Figure D.5: Indexed Files KEY DEFINITION ~--PRIMARY INDEX (EMPLOYEE NAME)~ L ABLE ABLE ELM AV 24379] • • • •• •• JONES •• •• JONES MAIN ST 19724 SMITH SMITH HOLT RD "------~---------------------DATA RECORDS----------------~---------- Q-M K-00029-00 Data Structures and Basic Concepts D-5 D.2 Input and Output File Specification An input or output file specification uniquely identifies a file by indicating its physical location and a directory in which it is cataloged, as well as providing a unique filename for that particular file within the directory. However, it is not necessary to supply the physical location and directory for the file, since the system uses the defaults set up during the log-in procedure when these components are omitted from a file specification. This section is only a summary. For a full description of defaults, wild cards, logical names, and subdirectories, see the VAX/VMS Command Language User's Guide. The format of a file specification representing a physical file or device is: node-naMe::deuice-naMe:[directoryJfilenaMe.file-tYPe;file-uersion Node-name:: The individual computer system (or node) name within a network consists of 1-6 alphanumeric characters. Example: BOSTON:: Device-name: The device name consists of three components: device type [controller] [unit-number]: The maximum length of the device type and controller specification is 15 characters. The maximum unit number is 65535. The default value for controller is A, and the default value for unit is 0. Physical device names are: Mnemonic CR DB DM DR DX LP MB MT NET TT XM Device Card Reader RP04, RP05, RP06 Disk RK06 Disk RM03 Disk Floppy Disk Line Printer Mailbox TE16 Magnetic Tape Network Communication Device Interactive Terminal DMC-11 Example: DB: is actually device name DBAO: by default. [directory] The directory name or names must be inclosed in either square brackets ([]) or angle brackets ( <>). A directory without a directory name (for example, [ ]) is not valid. The directory types are: • A 1- to 9-alphanumeric character string • A two-part number in the format of a user identification code (UIC) D-6 Data Structures and Basic Concepts • As subdirectories, in the format of name.name.name where each name can consist of up to 9 alphanumberic characters; each name represents a diretory level. Filename. The file name is limited to nine ASCII characters. File type The file type is limited to three ASCII characters. Some commonly used file types are: File type B2S CMD COM COB DAT* DIF DIR DMP EXE FOR LIB LIS LOG LST MAC MAP MAR MLB OBJ ODL OLB OPT STB TSK ;File-version Contents Input source file for the PDP-11 BASIC-PLUS-2NAX compiler Compatibility mode indirect command file Command procedure file to be executed with the @ (Execute Procedure) command, or to be submitted for batch execution with the SUBMIT command Input file containing source statements for the PDP-11 COBOL-74NAX compiler Input or output data file Output listing created by the DIFFERENCES command Directory file · Output listing created by the DUMP command Executable program image Input file containing source statements for the VAX-11 FORTRAN-IV-PLUS compiler Library file Listing file created by a language compiler or assembler; default input file type for PRINT and TYPE commands Batch job output file Compatibility mode listing file MACR0-11 source file Memory allocation map created by the linker VAX -11 MACRO source file Macro library Object file created by a language compiler or assembler Overlay description file · Object module library Options file for input to the LINK command Symbol table file created by the linker Compatibility mode task image The file version number is automatically updated by the system each time the file is changed. Commands may optionally use a period to delimit the file version number, but the documentation will use a semicolon. * indicates default file type for input files. Default file type for output files is whatever the input file typu is. Data Structures and Basic Concepts D-7 D.3 Programming Languages Supported The following compilers produce native mode programs that can use VAX-11 SORT: • VAX-11 FORTRAN IV-PLUS • VAX-11 MACRO • VAX-11 BLISS • VAX-11 COBOL-74 VAX-11 FORTRAN IV-PLUS FORTRAN IV-PLUS is an especially complete version of the leading language for scientific and engineering computation. It is a high-performance superset of the American National Standard lnstitute's (ANSI) 1966 FORTRAN. It also implements many of the anticipated features of the forthcoming ANSI standard. FORTRAN IV-PLUS supports character data types, an IF-THEN-ELSE statement, long variable names, and the standard CALL facility for calling system services. The FORTRAN IV-PLUS compiler first optimizes user source code, then translates it to take advantage of the VAX-11 instruction set, which can compile whole FORTRAN IV-PLUS statements into single instructions. An interactive symbolic debugger allows source-level debugging of FORTRAN IV-PLUS programs. VAX-11 MACRO The VAX-11 MACRO assembly language allows the programmer to write 32-bit machine language instructions for special efficiency. The symbolic debugger can also be used with VAX-11 MACRO. VAX-11 BLISS BLISS is a medium level language designed for building system software; such as compilers, real-time processors, and utilities. VAX-11 COBOL-74 The VAX-11 COBOL-74 language is based on the 1974 ANSI standard. D-8 Data Structures and Basic Concepts Index Address files, 2:...8, 2-11 Address sort, 1-1, 2-8, 5-5 sample output, 5-6 selection of, 2-7 ASCII character set collating sequence, B-1 Batch sessions, 2-17 BLISS, D-8 Bucket, D-5 Buffer allocation and work areas, 5-10 Byte, C-1 Call by reference, 3-6 Calling SORT subroutines, 3-1 Character, D-1 Character string, C-5 COBOL-74, D-8 Command, 1-1, 2-2 description, 2-17 summary, 2-3 Command description, command name qualifiers, 2-18 input-file-specification qualifiers, 2-20 output-file-specification qualifiers, 2-21 Command format, continuous command string, 2-1 using system prompts, 2-1 Command qualifier, /KEY, 2-18 /PROCESS, 2-17 /RSXll, 2-19 /SPECIFICATION, 2-19 /WORK_FILES, 2-19 Command structure, command name, 2-1 command parameters, 2-2 command qualifiers, 2-2 file qualifiers, 2-2 input-file-specification, 2-2 keywords, 2-2 output-file-specification, 2-2 subqualifiers, 2-2 values, 2-2 Command summary, 2-3 CPU time, 5-14 Data files, 2-11 input, 5-3 output, 5-4 Data hierarchy, D-1 Data types, C-1 byte, C-1 character string, C-5 double floating, C-3 floating, C-3 leading separate numeric string, C-7 longword, C-2 packed decimal string, C-8 quadword, C-2 trailing numeric string, C-5 variable length bit field, C-4 word, C-1 Decimal to hexadecimal conversion A-2 Decimal/Octal conversion, A-1 ' Descriptor, 3-5 Device name, D-6 DIGITAL command language (DCL) 1-1 Directory, D-6 ' Double floating datum, C-3 Dynamic memory usage, 5-10 Efficiency, 5-1 Elapsed time, 5-14 Error conditions, fatal, 4-1 warning, 4-1 Error messages, 4-1 SORT, 4-2 VAX-11 RMS, 4-8 VAXNMS DCL command interpreter, 4-2 Fields, D-1 File, I/0 considerations, 2-9 1/0 interface, 3-1 name, D-7 size, 3-6 type, D-7 version, D-7 File organizations, 1/0 flow, 2-10 indexed, D-3 indexed-sequential data files, 2-11 relative, D-3 relative data files, 2-11 Sequential, D-3 sequential address file, 2-11 sequential data files, 2-11 File specification format, D-6 Index-1 Files, D-1 Fixed length record sample, D-2 Floating datum, C-3 FORTRAN IV PLUS, D-8 Functionality, 1-2 Hexadecimal integer columns, A-3 Hexadecimal to decimal conversion, A-2 I/0 considerations, 2-8, 2-9, 5-11 1/0 counts, 5-14 Index sort, 1-1, 2-8, 5-6 sample output, 5-7 selection of, 2-7 Indexed files sample, D-5 Indexed-sequential data files, 2-11 Input and output, 1-2 Input and output file specificatiori, D.:...6 Input buffer, 5-10 Input data files, 5-3 Input file, descriptor, 3-9 Input file format specifications, file size, 2-21 record size, 2-20 Input file qualifiers, /FORMAT, 2-20 Input record, sample, 5-4 types, 5-4 Inputs to VAX-11 SORT, 2-·8, 2-9 Interactive mode, 2-1 Interactive samples, 2-12 Interactive sessions, 2-5 sample -#1, 2-13 sample -#2, 2-15 Invoking SORT, 2-5 Key, 2-31 Key area, 3-5 Key buffer, address, 3-6 specifications, 3-6 Key comparison routine address, 3-6, 3-7 Key comparisons, SORT's routine, 3-4 user's routine, 3-4 Key specifications, ascending/descending, 2-19 data type, 2-18 leading/trailing sign, 2-19 number, 2-18 overpunched/separate sign, 2-19 position, 2-18 size, 2-18 Index-2 Keys, ascending or descending order, 2-35 assigning a precedence number, 2-31 binary, 2-33 character, 2-33 data type, 2-32 decimal, 2-33 multiple, 2-34 NUMBER=n, 2-31 POSITION=n, 2-34 quick reference flowchart, 2-31 setting up, 2-31 signed numbers, 2-33 size restrictions, 2-35 SIZE=n, 2-34 Keywords, 2-2 Languages, D-8 Leading separate numeric string, C-7 Longest record length, 3-6, 5-13 Longword, C-2 MACRO, D-8 Mapping, 5-10 Multi-block and buffer counts, 5-14 Node name, D-6 Number of, work files, 3-6 initial runs, 5-14 merge passes, 5-14 Octal/Decimal conversion, A-1 Order of the merge, 5-14 Output buffer, 5-10 Output data files, 5-4 sample, 5-5 Output file, allocation, 3-9 bucket size, 3-9 descriptor, 3-9 file options, 3-9 maximum record size, 3-9 organization, 3-9 record format, 3-9 Output file format specifications, block size, 2-22 record size, 2-21 record type, 2-21 Output file qualifiers, /ALLOCATION, 2-22 /BUCKET_SIZE, 2-22 /CONTIGUOUS, 2-22 /FORMAT, 2-21 /INDEXED_SEQUENTIAL, 2-22 Output file qualifiers (Cont.) /OVERLAY, 2-22 /RELATIVE, 2-22 /SEQUENTIAL, 2-22 Output record placement, 3-13 Outputs from VAX-11 SORT, 2-9 Overpunch format, C-6 Packed decimal string, C-8 Page faults, 5-10, 5-14 Paged-out, 5-10 Pager, 5-10 Parameters, 2-2 Phases of operation, 5-7 Powers of 2 and 16, A-2 Programming languages supported, BLISS, D-8 COBOL-74, D-8 FORTRAN IV PLUS, D-8 MACRO, D-8 Quadword, C-2 Qualifiers, 2-2, 2-3 Real memory, 5-10 Record descriptor, 3-11, 3-13 Record I/0 interface, 3-2 Record sort, 1-1, 2-8, 5-3 selection of, 2-7 Record types, Fixed, D-2 Variable, D-2 Variable with fixed-length control (VFC), D-2 Records, D-1 Re la ti ve data files, 2-11 Relative files sample, D-5 RMS, completion status codes, 4-8 data files, D-1 data structures, D-1 file organizations, D-3 records, D-1 Running SORT, 2-1 Sample programs, COBOL-74NAX, 3-17 FORTRAN IV PLUS, 3-20 MACRO, 3-15 Sequential, address files, 2-11 data files, 2-11 files sample, D-5 SORT, architecture, 5-2 definition of, 1-1 SORT (Cont.) functional description, 5-1 internal organization, 5-7 Sort, improving efficiency, 5-1 sample, 2-6 tree, 5-10 tuning, 5-1 type, 3-6 types, 1-1 Sort types, address, 1-1, 2-7, 5-6 index, 1-1, 2-7,5-6 record, 1-1, 2-7, 5-3 selection of, 2-7 specification of, 2-17 tag, 1-1, 2-7, 5-3 SORT's four sorting processes, 2-8 Sorting processes, 2-8 functional descriptions, 5-3 See sort types, 1-1 Sorting sequence, 2-5 Specification file, 2-23 fixed position field, format (SORT-11), 2-24 field record specfications, 2-27 header record specifications, 2-26 sample, 2-29 summary, 2-28 free field, format (VAX-11 SORT), 2-24 field record specifications, 2-30 header record specifications, 2-30 sample, 2-31 methods of entering, 2-23 record formats for fixed position fields, 2-25 record formats for SORT-11 type files, 2-25 records, 2-24 VAX-11 SORT, 2-30 Specification form, 2-25 Statistics, 1-2 example of, 2-12 String descriptors, 3-6 Subqualifiers, 2-2, 2-3 Subroutines, 3-1 calls, 3-1 interfaces, 3-1 package, 1-1 parameters, definitions, and valid returns, 3-5 programming considerations, 3-2 set summary, 3-3 SOR$END_SORT, 3-14 SOR$INIT_SORT, 3-6 SOR$PASS_FILES, 3-9 Index-3 Subroutines (Cont.) SOR$RELEASE-REC, 3-11 SOR$RETURN-REC, 3-13 SOR$SORT_MERGE, 3-12 Tag sort, 1-1, 2-8, 5-3 selection of, 2-7 Temporary storage, 2-8 Total key size, 3-6 Trailing numeric string, C-5 Tuning procedure, 5-11 system manager performance considerations, 5-14 user performance considerations, 5-11 Tuning procedure parameters, modified page writer clustel' factor, 5-15 number of work files, 5-13 process section count, 5-15 type of sort, 5-13 using SORT'S statistics, 5-13 virtual page count, 5-15 work file devices, 5-13 working set quota, 5-12, 5-15 Index-4 Utility, 1-1 Values, 2-2, 2-3 Variable length bit field, C-4 Variable length record samples, D-2 VAX-11 SORT, definition of, 1-1 VAXNMS calling standards, 3-2 VFC records, D-2 sample, D-2 Virtual memory, 5-10 Virtual memory added, 5-14 VMS library, 3-2 Word, C-1 Work files, 2-8 default, 2-35 logical name of, 2-35 physical device codes, 2-35 setting up, 2-35 Working set, 5-10 Zoned numeric format, C-6 READER'S COMMENTS VAX-11 SORT User's Gulde Order No. AA-D113A-TE NOTE: This form is for document comments only. DIGITAL will use comments submitted on this form at the company's discretion. If you require a written reply and are eligible to receive one under Software Performance Report (SPR) service, submit your comments on an SPR form. Did you find this manual understandable, usable, and well-organized? Please make suggestions for improvement. Did you find errors in this manual? If so, specify the error and the page number. Please indicate the type of user/reader that you most nearly represent. D D D D D D Name Assembly language programmer Higher-level language programmer Occasional programmer (experienced) User with little programming experience Student programmer Other (please s p e c i f y ) - - - - - - - - - - - - - - - - - - - - - - - - - - - Date----------------- Organization---------------~------------------ Street------------------------------------C i t Y - - - - - - - - - - - - - - - - - - State------ Zip Code _ _ _ _ _ _ __ or Country - - Do Not Tear - Fold Here and Tape - - - - - - - - - - - - - 111111 POSTAGE WILL BE PAID BY ADDRESSEE RT/C SOFTWARE PUBLICATIONS TW/A14 DIGITAL EQUIPMENT CORPORATION 1925 ANDOVER STREET TEWKSBURY, MASSACHUSETTS Do Not Tear- Fold Here 01876 - No Postage Necessary if Mai led in thE United States
Home
Privacy and Data
Site structure and layout ©2025 Majenko Technologies