If you want to cancel a command before it has completed, press Ctrl/C. If you press Ctrl/C during a wildcard transaction that updates the library, VSI Digital Test Manager completes the current transaction, but does not continue.

When you enter a VSI Digital Test Manager command from DCL and then press Ctrl/C during execution of the command, VSI Digital Test Manager returns control to DCL level. If you enter the command from a subsystem prompt level, VSI Digital Test Manager retains control as indicated by the DTM> prompt.

You can obtain VSI Digital Test Manager Help in a DECwindows environment by pulling down the Help menu.

If a help frame is not defined for an artifact, DTM displays a “could not find frame” error.

You can display files and view information on collections, tests, groups, and variables through views. The views are the DECwindows equivalent of the character-cell interface SHOW commands.

VSI Digital Test Manager displays the collection view when you invoke the product.

Figure 2.3 shows a sample test view.

You can obtain more detail by double clicking on an item (test, group,collection, variable, or library) inside a view. If an item is expanded fully, double clicking collapses the item to the previous level of information.

Most VSI Digital Test Manager commands have a corresponding menu path in the DECwindows interface; however, there is not a complete one-to-one correspondence.

There is no corresponding DECwindows action for the VSI Digital Test Manager SPAWN and ATTACH commands, because DECwindows enables you to create another process without having to spawn out of a process.

There is no corresponding DECwindows action for the VSI Digital Test Manager SHOW HISTORY command. History records can only be accessed from the command-line interface.

Figure 2.5 shows how to create a DECwindows test. The test is for the DECwindows clock application. The command to run the clock application is specified in the Command field of the Create DECwindows Test dialog box.

Figure 2.6 shows a sample DECwindows Record dialog box and how to invoke it. You can use the Record dialog box to specify parameters or test files associated with the application being recorded. If you click on the test name before invoking the Record dialog box, the test name automatically appears in the Test field.

When you record the DECwindows test, ensure that the conditions of the test at its start are the same as at its end. For example, creating a solid background, making icons of all applications, and placing the icons in a consistent order all help to ensure that start and end conditions are equal. See Section 3.5.2 for more information about recording DECwindows tests. See the VSI DECset for OpenVMS Test Manager Reference Manual for more information about the RECORD command and its qualifiers.

When you click OK in the Record window, a DECterm window appears and a message lets you know that the recording has started.

At the end of the recording process, press Return to remove the displayed DECterm.

DECterm Window Configuration

For the DECwindows interface to VSI Digital Test Manager, DECterm windows are created during RECORD, PLAY and RUN operations. You can control the position and size of these DECterm windows.

Figure 2.7 shows how to create a collection. In the example, the CLOCK_TEST test is the only test in the collection. Tests can only be run in the context of a collection.

Figure 2.8 shows an expanded view of the newly created collection and the test it contains.

When the collection process has finished, press Return to remove the displayed DECterm.

In the Collection view, expand the appropriate collection, expand the test to show the benchmark, result, and difference files, then double click on the file you want to display. VSI Digital Test Manager displays the Screen Editor with the selected file (as shown in Figure 2.10).

VSI Digital Test Manager automatically creates the first benchmark files from the recorded session's output for both DECwindows and terminal interactive tests. For noninteractive tests, however, create the first benchmark file from a result file.

Use the result file to update the benchmark file when results do not match the previous benchmark file, but are the expected results for a test.

Several factors can cause a test to fail with undesirable results. For example, if mail is received during the sample session described in this chapter, the icons in the DECwindows Mail facility can become altered. To ensure that areas of a DECwindows application that you have no interest in do not affect a test-result comparison, you can create areas called masks on benchmark images using the VSI Digital Test Manager Screen Editor. Areas that are masked are not compared when VSI Digital Test Manager compares the results of a test execution against the benchmark image.

Generally, areas are masked on a benchmark image after recording the test and before executing the test to mask out run-dependent image data and maximize the chances for successful comparison status for the test. However, you can create masked areas on a benchmark image at any time.

To access the Screen Editor, click on the DECwindows test, then pull down the Testing menu and choose the Mask... menu item. The benchmark image is automatically loaded into the Screen Editor.

Figure 2.12 shows a mask (vertical stripes in the rectangle below the date) being defined for the CLOCK_TEST benchmark image.

Masked areas do not become part of the benchmark image. VSI Digital Test Manager stores the coordinates of the masked areas in a mask file, but you determine whether to compare the image with or without the mask. If you create masked areas on a benchmark file, the masks are used by default. If you do not want previously created mask areas to be applied to a result comparison, you must explicitly specify ignoring masks on the Create Collection or Compare dialog box. You can delete a mask by double clicking on a defined mask.

The VSI Digital Test Manager DECwindows Screen Editor does not purge benchmark files when new versions are created by modifying the masks defined for a benchmark file. You must manually purge the older versions. This applies even if the benchmark files are located in a VSI Digital Test Manager library.

You can also move a mask by placing the pointer on the mask you want to move, pressing and holding MB3, and releasing MB3 when you move the mask to the new area.

To save a mask file, choose Save Mask from the File menu. The Screen Editor saves the mask file in the same location as the test benchmark file. All mask files have the extension .MXK.

The global mask filter is useful when tests fail because of a difference that affects several screens (e.g., if an unexpected window is present in the test results ). By eliminating this area from the screen comparisons, other reasons for unsuccessful comparisons are more easily detected.

The global mask filter is not included when masks are saved, so it does not affect collection comparisons involving the test.

To dismiss the Screen Editor without saving any defined masks, choose Quit from the File menu. The file is not updated.

To Exit the Screen editor and save any defined masks, choose Exit from the File menu.

$ CREATE/DIRECTORY [.DTMLIB]
$ DTM
DTM> CREATE LIBRARY [.DTMLIB] "New Digital Test Manager library"
%DTM-S-CREATED, Digital Test Manager library DUA0:[USER01.DTMLIB] created
DTM>

The phrase enclosed in quotation marks (" ") following the library specification is a remark that you associate with the library you are creating. VSI Digital Test Manager prompts you for a remark if you do not include one on the command line, but a null remark string is permitted.

VSI Digital Test Manager creates a subdirectory in the library for each collection you create. Do not create subdirectories or files in a directory containing the VSI Digital Test Manager library or any of its subdirectories. Do not set the default directory as a VSI Digital Test Manager library or any of its subdirectories.

If you put files in these subdirectories, they are deleted along with the collection files when you instruct VSI Digital Test Manager to delete the collections.

When you invoke VSI Digital Test Manager, you must explicitly specify the library you want to use to store files. You do this by selecting an existing library using the SET LIBRARY command.

$ DTM SET LIBRARY [.DTMLIB]
%DTM-S-LIBIS, Digital Test Manager library is DUA0:[USER01.DTMLIB]

After you select a library, all VSI Digital Test Manager commands you enter refer to that library until you select another library, create another library, or logout.

Setting a Benchmark Directory

DTM> SET BENCHMARK_DIRECTORY DUA0:[USER01.BMK]
_Remark: “New default benchmark directory”
%DTM-S-NEWDEF, DUA0:[USER01.BMK] is the new default collection benchmark
directory
DTM>

See Chapter 7 for information about storing files outside the VSI Digital Test Manager library, such as in a Code Management System (CMS) library.

Setting a Template Directory

DTM> SET TEMPLATE_DIRECTORY DUA0:[USER01.TMPL]
_Remark: “New default template directory”
%DTM-S-NEWDEF, DUA0:[USER01.TMPL] is the new default
collection template directory
DTM>

DTM> SHOW LIBRARY
Your VSI Digital Test Manager library is DUA0:[USER01.DTMLIB]

DTM> SHOW ALL

Description of VSI Digital Test Manager Library DUA0:[USER01.DTMLIB]


Default template  directory: DUA0:[USER01.TEMPLATES] ""
Default benchmark directory: DUA0:[USER01.BENCHMARKS] ""
Default collection prologue: None Specified
Default collection epilogue: None Specified
Number of collections:       20
Number of test descriptions: 152
Number of groups:            18
Number of variables:         9

If one of these entities does not exist for the library, the message NONESPECIFIED is displayed. You can also place the library summary information into a file by specifying the /OUTPUT qualifier.

When you create a new library, VSI Digital Test Manager automatically creates a history for that library. Whenever you issue a command that alters the library, VSI Digital Test Manager enters that command and its associated remark into the history.

You cannot access these history records from the DECwindows interface. However, you can access them via the SHOW HISTORY command from the VSI Digital Test Manager command line, the DCL command line, or the VSI Digital Test Manager callable interface.

DTM> SHOW HISTORY

History in Digital Test Manager Library DUA0:[USER01.DTMLIB]

25-JAN-1998 12:03:54 USER01 CREATE LIBRARY DUA0:[USER01.DTMLIB]
                            "Test library"
25-JAN-1998 12:04:12 USER01 CREATE TEST_DESCRIPTION MAIL_TEST/INTERACTIVE

“Going to record a MAIL test”
25-JAN-1998 12:05:32 USER01 RECORD MAIL_TEST
                     "Recording MAIL on the terminal"
   .
   .
   .
DTM>

DTM> SHOW HISTORY MAIL_TEST/TRANSACTION=RECORD/USER=USER01

History in Digital Test Manager Library DUA0:[USER01.DTMLIB]

25-JAN-1998 12:05:32 USER01 RECORD MAIL_TEST "Record MAIL on the terminal"
   .
   .
   .
DTM>

DTM> REMARK "End of Version 3.3 Testing"
%DTM-S-REMARK, remark added to history file
DTM>

14-MAY-1998 13:01:32 USER01 REMARK "End of Version 3.3 Testing"

1-JUN-1998 15:03:55 USER01 REMARK "Deleting the old information"

DTM> DELETE HISTORY "Deleting all the old information"
Confirm DELETE HISTORY/BEFORE=14-Nov-1998 [Y/N] (N): Y
%DTM-S-HISTDEL, 323 history records deleted
DTM>

DTM> DELETE HISTORY/BEFORE=1-JUN-1998 "Deleting the old information"
Confirm DELETE HISTORY/BEFORE=1-Jun-1998 [Y/N] (N): Y
%DTM-S-HISTDEL, 150 history records deleted
DTM>

You create a test description by using the CREATE TEST_DESCRIPTION command with qualifiers to specify the test description fields.

DTM> CREATE TEST_DESCRIPTION MAIL_TEST /INTERACTIVE
_Remark: Going to record a MAIL test
%DTM-I-DEFAULTED, benchmark file name defaulted to MAIL_TEST.BMK
%DTM-I-DEFAULTED, template file name defaulted to MAIL_TEST.SESSION
%DTM-S-CREATED, test description MAIL_TEST created
DTM> RECORD MAIL_TEST “Recording MAIL on the terminal”
%DTM-I-BEGIN, your interactive test session is now beginning...
Type Ctrl/P twice to terminate the session.


$

The VSI Digital Test Manager Create Test dialog box in the DECwindows interface does not allow you to specify variables where the variable value contains commas. For example, you cannot define a variable X with a value of “A,B,C.” If you require this ability, use the CREATE TEST_DESCRIPTION command in the VSI Digital Test Manager DCL interface.

DTM> SHOW TEST_DESCRIPTION MAIL_TEST

Test descriptions in VSI Digital Test Manager Library DUA0:[USER01.DTMLIB]

MAIL_TEST          "MAIL command test"

  Template        = MAIL_TEST.COM
  Benchmark       = MAIL_TEST.BMK
  Prologue        = None Specified
  Epilogue        = None Specified

You can display more than one test by specifying test names or group names, or you can use wildcards.

You can also use qualifiers to display all the test description information or a portion of it; the default qualifier is /INTERMEDIATE.

Use the COPY TEST_DESCRIPTION command to create an exact or modified copy of a test description in the VSI Digital Test Manager library. This command enables you to create a series of similar test descriptions without repeatedly entering the CREATE TEST_DESCRIPTION command. You can modify some of the field values for each new test description.

Although you can specify new values for some of the fields of a new test description, you must either copy a test description with its filter, variable,and group field values intact, or eliminate the field values altogether; you cannot modify these values when you copy the test description. You can eliminate these field values with the COPY TEST_DESCRIPTION negating qualifiers. (For example, /NOFILTER is a negating qualifier that disassociates filters from the new test description.)

DTM> CREATE TEST_DESCRIPTION MAIL_TEST_NONINT -
_DTM> /TEMPLATE=MAIL_NONINT.COM/NONINTERACTIVE -
_DTM> /PROLOGUE=NOBROADCAST.COM/EPILOGUE=BROADCAST.COM
_Remark: Creating a MAIL test
%DTM-I-DEFAULTED, benchmark file name defaulted to MAIL_TEST_NONINT.BMK
%DTM-I-DEFAULTED, template file name defaulted to MAIL_TEST_NONINT.COM
%DTM-S-CREATED, test description MAIL_TEST_NONINT created
DTM>
DTM> COPY TEST_DESCRIPTION MAIL_TEST_NONINT SHOW_ALL_TEST -
_DTM> /TEMPLATE=SHOW_ALL_NONINT.COM
_Remark: Copied margin test into SHOW_ALL_TEST with new template file
%DTM-I-DEFAULTED, benchmark file name defaulted to SHOW_ALL_TEST.BMK
%DTM-I-COPIED, test description MAIL_TEST_NONINT copied
-DTM-S-CREATED, test description SHOW_ALL_TEST created
DTM>
DTM> COPY TEST_DESCRIPTION MAIL_TEST_NONINT SEND_MAIL_TEST -
_DTM> /TEMPLATE=MAIL_TEMPLATE.COM
_Remark: Copied margin test into SEND_MAIL_TEST with new template file
%DTM-I-DEFAULTED, benchmark file name defaulted to SEND_MAIL_TEST.BMK
%DTM-I-COPIED, test description MAIL_TEST_NONINT copied
-DTM-S-CREATED, test description SEND_MAIL_TEST created
DTM>

The test description generated by the CREATE TEST_DESCRIPTION command sets up the prologue and epilogue files. Subsequent COPY TEST_DESCRIPTION commands create new tests using the first test description, MAIL_TEST_NONINT. By default, the prologue and epilogue files that are associated with MAIL_TEST_NONINT are copied into the SHOW_ALL_TEST and SEND_MAIL_TEST test descriptions. However, the MAIL_TEST_NONINT, SHOW_ALL_TEST, and SEND_MAIL_TEST test descriptions have different template files.

If you do not specify any qualifiers, the value for the test description fields are copied from the existing test description to the new test description.

Use the MODIFY TEST_DESCRIPTION command to modify a test description to include or exclude a prologue file, an epilogue file, filters, variables, or other test description attributes.

DTM> MODIFY TEST_DESCRIPTION MAIL_TEST/EPILOGUE=NODCL_BROADCAST.COM
_Remark: Using a different epilogue file
%DTM-S-MODIFIED, test_description MAIL_TEST modified
DTM>

If you use a negating qualifier, VSI Digital Test Manager either removes the value of the qualifier, or reverts the value to its default value.

DTM> MODIFY TEST_DESCRIPTION MAIL_TEST /NOPROLOGUE
_Remark: Deleting the prologue
%DTM-S-MODIFIED, test_description MAIL_TEST modified
DTM>

Only the test description fields you specify with the MODIFY TEST_DESCRIPTION command are affected when you modify a test description.

VSI Digital Test Manager ignores negating qualifiers specified for fields for which no value was assigned. For example, the /NOPROLOGUE qualifier is ignored if no prologue file had been previously assigned.

The VSI Digital Test Manager Modify Test dialog box in the DECwindows interface does not allow you to specify variables where the variable value contains commas. For example, you cannot define a variable X with a value of “A,B,C.” If you require this ability, use the MODIFY TEST_DESCRIPTION command in the VSI Digital Test Manager DCL interface.

Use the DELETE TEST_DESCRIPTION command to delete a test description from the library. If the test description has a benchmark file that resides in the library, the benchmark file is also deleted. If the benchmark file is outside the library, it is unaffected.

The DELETE TEST_DESCRIPTION command does not delete any template, prologue, or epilogue file associated with the test.

The DELETE TEST_DESCRIPTION command has no effect on any result files or difference files that were produced for this test during a collection run. VSI Digital Test Manager deletes result and difference files with the collection rather than with the test description, because these files are associated with the collection.

DTM> DELETE TEST_DESCRIPTION SHOW_ALL_TEST
_Remark: Deleting the revised Test for sending mail
Confirm deletion of test_description SHOW_ALL_TEST [Y/N] (N): Y
%DTM-S-DELETED, test_description SHOW_ALL_TEST deleted
DTM>

You cannot delete a test description while it is a member of any group. Use the REMOVE TEST_DESCRIPTION command to remove a test description from all groups to which it belongs. Then use the DELETE TEST_DESCRIPTION command to delete the test description.

The template file for a noninteractive test can be the test itself. For example, the test in Example 3.2 can be executed by itself. If the template file is the test itself, you do not need to create a new template file to execute your test.

$!    *** MAIL TEMPLATE ***
$!       MAIL_TEMPLATE.COM
$!
$ SET BROADCAST=NONE
$ @SEND_MAIL_TEST
$ SET BROADCAST=ALL

DTM> RECORD MAIL_TEST " "
%DTM-I-BEGIN, your interactive test session is now beginning...
Type CTRL/P twice to terminate the session.

See Chapter 8 for complete information about session files for interactive terminal tests.

The template file for an interactive terminal test is the recorded terminal session file that VSI Digital Test Manager produces when you use the RECORD command. An interactive terminal test requires user input from a terminal keyboard. For example, an interactive terminal test can be a forms program that requests information from you.

VSI Digital Test Manager subsequently uses the recorded session file to supply data to applications in future test runs to ensure that the applications you are testing have not regressed. You can have VSI Digital Test Manager execute the interactive session file on the screen or in batch mode. Either way, VSI Digital Test Manager supplies the input data that was recorded.

If your interactive terminal session needs a screen size larger than 24 lines by 132 columns, set the display device to the largest size it requires during recording before you begin the recording session.

When recording interactive terminal tests from the DCL interface, VSI Digital Test Manager sets the terminal characteristics to “Application Keypad.” If the test you are recording requires “Numeric Keypad,” specify the DCL command SET TERM /NOAPP at the start of test recording.

DTM> RECORD MAIL_TEST/TERMINATION_CHARACTER=^D “”
Type Ctrl/D twice to terminate the session.
   .
   .
   .

DTM> RECORD MAIL_TEST/TERMINATION_CHARACTER=12
“”
Type Ctrl/L twice to terminate the session.
   .
   .
   .

DTM> RECORD CLOCK_TEST " "
%DTM-I-XTRAPVERSION, Display SDCW01::0 is running XTrap V0.3-0
%DTM-I-RECORDING, Recording to file DISK:[USER]CLOCK_TEST.SESSION at line 0

Before recording a DECwindows test, ensure that the testing environment has been initialized so conditions remain the same at the start of the test and at the end. This enables you to play several DECwindows session files without interruption.

For example, iconizing all windows and ordering the icons at the start of a test enables you to reproduce the start condition of a test. It also enables you to mask out small portions of the workstation screen using the Screen Editor. See Section 2.2.8 for information about masking portions of the workstation screen.

See Chapter 2 for a sample, interactive recording session using DECwindows.

DTM> RECORD test-name/KEYSYM=F7
DTM> RECORD test-name/KEYSYM=a
DTM> RECORD test-name/KEYSYM="comma"

DTM> PLAY MAIL_TEST.SESSION/INTERACTIVE
%DTM-I-BEGIN, your interactive test session is now beginning
   .
   .
   .
%DTM-S-CONCLUDED, your interactive test session has concludedDTM>

A session file is executed as if it were being run on the same type of display device on which it was recorded. If the display device characteristics differ from those for the recording display device, the output might not appear as you expect.

DTM> PLAY CLOCK_TEST.SESSION/DECWINDOWS
   .
   .
   .
DTM>

If the test corresponding to the session file has a DCL command associated with it, specify the DCL command with the /COMMAND qualifier.

By default, VSI Digital Test Manager does not execute the test at the same speed at which it was recorded; therefore, time-dependent screens marked for comparison usually do not match, and the comparison of the test results are usually unsuccessful. For example, you cannot test the OpenVMS Phone Utility because your input as the person initiating the call depends on the person you are calling to answer your call. These two, time-dependent events cannot be consistently duplicated.

Other examples of timing-dependent applications are those submitted as batch jobs or executed as subprocesses. You cannot test timing-dependent applications unless the application is a submitted batch job with the DCL SYNCHRONIZE command. Using the SYNCHRONIZE command, you have the choice of either waiting for the batch job or subprocess to finish, or performing other operations with the application. The Language-Sensitive Editor (LSE) COMPILE command is an example of this type of timing-dependent application.

By default, VSI Digital Test Manager plays back tests at a rate faster than the speed at which they were recorded. Regulation of playback speed is governed solely by the rate at which the application reads input from a terminal device.

Conversely, some applications are more than simple input-processing loops,and might not play back correctly if input is sent to them faster than they can read it. In this sense, these types of applications are time dependent, for which VSI Digital Test Manager does not guarantee accurate playback.

However, to accommodate testing of time-dependent applications, VSI Digital Test Manager provides a /REALTIME qualifier that you can specify with the RECORD, PLAY, or CREATE COLLECTION commands. When you specify the /REALTIME qualifier, all input time delays that occur during an interactive recording session are preserved during playback.

When specified with the RECORD command, the /REALTIME qualifier forces time delays to be recorded as WAIT records in the session file. When used with the PLAY and CREATE COLLECTION command, the /REALTIME qualifier forces recorded timing information to be interpreted as input time delays and test synchronization (based on application read detection) is disabled. This enables VSI Digital Test Manager to send input to the application at a rate determined only by the speed at which the test was recorded.

Use the /REALTIME qualifier sparingly because it slows down the execution of your tests. Although time delays are preserved during playback, VSI Digital Test Manager cannot absolutely guarantee accurate playback of time-dependent applications. For such applications, the readiness of the application to accept input can be highly dependent on the current system performance and load. When executing such tests, it is recommended that you run these tests on a system where processing demands are strictly regulated.

While recording a test, do not press Ctrl/C or Ctrl/Y except at a point where the application being tested is expecting input. Pressing Ctrl/C or Ctrl/Y at a point other than where the application is expecting input terminates the application at a random point that generally cannot be duplicated when you run the test. For example, if you press Ctrl/C or Ctrl/Y while output is being displayed during testing of the DCL DIRECTORY command,you create a screen that cannot be consistently duplicated; the comparison of the test results is usually unsuccessful.

You can use VSI Digital Test Manager to test only those applications that accept input after prompting for it. You cannot test programs such as the OpenVMS Monitor Utility (MONITOR) with VSI Digital Test Manager. MONITOR displays screen after screen of continuously changing statistical information about the system. After you invoke MONITOR, it does not prompt you for input; it displays information until you terminate it by pressing Ctrl/Z. The termination occurs in a way that cannot be consistently duplicated; thus, the comparison of the test results usually is unsuccessful.

If VSI Digital Test Manager does not recognize the device type of the terminal you are using to record the interactive terminal session, it defaults to a VT100-compatible terminal. Chapter 8 describes the significance of device type and terminal characteristics when recording an interactive terminal session.

You can compare tests that contain graphics mode (sixel and ReGIS) commands. For tests where screen comparison is specified, VSI Digital Test Manager ignores graphics-mode commands. Only differences in the text mode sections of the test are found.

Screen comparison is performed at the completion of the previous operation. If the previous operation causes information to scroll off the screen, only the information visible at the completion of the operation is captured and compared. To ensure that scrolling occurs page by page, use the TYPE/PAGE command.

DTM> SUBMIT MAIL_COLL/NOTIFY/LOG_FILE=[]/QUEUE=SYS$LARGE
%DTM-S-SUBMITTED, collection MAIL_COLL submitted
-DTM-I-TEXT, Job MAIL_COLL (queue SYS$LARGE entry 1000) started on
     SYS$LARGE

You can submit a collection to a batch queue more than once. However, if you attempt to resubmit a collection that you have executed and not reviewed, VSI Digital Test Manager prompts you to confirm that you want to resubmit the collection without reviewing it. To eliminate the confirmation prompt when resubmitting a collection without reviewing it, specify the SUBMIT command with the /NOCONFIRM qualifier.

You can execute a collection interactively using the RUN command. The collection you execute can contain any combination of noninteractive, interactive terminal, and DECwindows tests.

If you specify a test name rather than an existing collection name as the parameter for the RUN command, VSI Digital Test Manager prompts you for automatic collection creation. Collections created this way contain only the specified test and have the same collection name as the test name.

The RUN command displays the output of each test on the screen. If you want to see messages from the prologue or epilogue file, specify the /LOG_FILE qualifier.

DTM> RUN SEND_MAIL_COLL "running the send mail test"
Starting SEND_MAIL_TEST test run...

Your personal name is "Digital Test Manager - Project Q"

Performing post-run cleanup with comparison...

%DTM-I-NEWTEST, test SEND_MAIL_TEST is a New test
%DTM-S-COMPARED, collection SEND_MAIL_COLL compared
DTM>

Section 4.6 describes how to compare the results of the execution of tests in a collection with the benchmark files of each test in a collection.

Use the STOP command to terminate a collection executing in batch; this command stops execution of the collection and cleans up the VSI Digital Test Manager library.

Press Ctrl/C (rather than Ctrl/Y) to terminate a collection running interactively.

If you stop an executing collection with a command other than the STOP command or Ctrl/C, or if the system crashes while a collection is executing, errors will occur and you will not be able to review the collection. Pressing Ctrl/C or typing the STOP command to terminate a collection run enables VSI Digital Test Manager to restore its library to a consistent state and to perform necessary post-run cleanup after the collection run stops. See Chapter 7 for instructions on how to recover the library, review the executed tests,and rerun the tests that did not execute.

DTM> STOP MSGTEST /CONFIRM “Stopping collection run”
Confirm stop of collection MSGTEST [Y/N] (N): Y
%DTM-S-STOPPED, collection MSGTEST stopped
DTM>

Result descriptions contain information about test output and comparison status. In the same way that a test description contains information about test files, VSI Digital Test Manager creates a result description that summarizes information about the test results (see Section 5.1.1.1).

DTM_REVIEW> SHOW MAIL_TEST
Result Description MAIL_TEST         Comparison Status :  Successful

   Benchmark File  MAIL_TEST.BMK
   Result file does not exist
   Difference file does not exist

DTM_REVIEW>

This section provides an overview of the Review subsystem and shows how to use it to locate test results in a terminal environment. The Review subsystem exists only in a terminal environment; DECwindows tests cannot be reviewed in a terminal environment. You review DECwindows test results using views in the DECwindows environment (see Chapter 2).

DTM> REVIEW MSGTEST
Collection MSGTEST with 4 tests was created on 20-MAY-1998 10:46:34 by the command:
        CREATE COLLECTION MSGTEST MSGTEST/GROUP "Creating collection
                                                             MSGTEST"
        Last Review Date = 23-MAY-1998 10:04:45
        Success count = 0
        Unsuccessful count = 0
        New test count = 4
        Updated test count = 0
        Comparisons aborted = 0
        Test not run count = 0

Result Description MESSAGE_1    Comparison Status : New Test

DTM_REVIEW>

DTM_REVIEW> SELECT INFOMSGTEST

DTM_REVIEW> SHOW/UNSUCCESSFUL
Result Description MSGTEST   Comparison Status :  Unsuccessful

    Benchmark File is DUA0:[USER01.DTMLIB]MSGTEST.BMK
    Result file is present
    Difference file is present

Result Description INFOMSGTEST   Comparison Status :  Unsuccessful

    Benchmark File is DUA0:[USER01.DTMLIB]INFOMSGTEST.BMK
    Result file is present
    Difference file is present

DTM_REVIEW>

5.2.1.5. Using the Review Subsystem Keypads

The Review subsystem has several default keypads that you can use to issue commands in the command-line interface. The keypads are intended for use in the terminal environment. You can redefine these keypads to create custom keypads. Chapter 6 shows how to redefine the keypad keys for any of the default keypads. This section describes the following default keypads:

• Review subsystem

• SHOW/BENCHMARK and SHOW/RESULT

• SHOW/DIFFERENCES

In the following illustrations, the white area of a key indicates that you can issue the command by pressing only the key. The shaded area of a key indicates that you can issue the command by pressing PF1, then the shaded key.

For example, to display a benchmark file using the Review subsystem keypad (Figure 5.1), press KP9. To print a benchmark file, press PF1, then KP9.

Note

The number keys on the keyboard give you the same movement as the numbers on the keypad.

Within the Review subsystem, Ctrl/H displays HELP, Ctrl/W refreshes the screen, and Ctrl/Z terminates display of the result,benchmark, or difference file and returns control to the Review subsystem.

Figure 5.1 shows the default Review subsystem keypad, which has most of the keypad keys defined.

Table 5.4 describes the key definitions shown in Figure 5.1.

Key Sequence	Key Definition
KP0	Displays the next test in a collection.
PF1-KP0	Displays the previous test in a collection.
KP1	Displays the next unsuccessful test in a collection.
PF1-KP1	Displays the previous unsuccessful test in a collection.
KP2	Displays the next new test in a collection.
PF1-KP2	Displays the previous new test in a collection.
KP3	Displays the next updated test in a collection.
PF1-KP3	Displays the previous updated test in a collection.
KP4	Places the currently selected files for printing on the print queue immediately.
KP5	Marks the test description and inserts into a group.
KP6	Updates the current test in a collection.
KP7	Issues the SHOW command with the /DIFFERENCES qualifier and displays the differences.
PF1-KP7	Prints the difference file.
KP8	Issues the SHOW/RESULTS command and displays the result file.
PF1-KP8	Prints the result file.
KP9	Issues the SHOW/BENCHMARK command and displays the benchmark file.
PF1-KP9	Prints the benchmark file.
period (KP.)	Displays the first test.
PF1-period (KP.)	Displays the last test.
comma (KP,)	Issues the SPAWN command, causing you to temporarily exit from VSI Digital Test Manager.
PF1-comma (KP,)	Issues the ATTACH command to attach you to the parent process (return to the VSI Digital Test Manager Review subsystem).
Dash (KP-)	Displays the collection summary.
PF1-PF2	Displays help for review subsystem.
PF2	Displays review subsystem Keypad.

Figure 5.2 shows the default SHOW/BENCHMARK, SHOW/RESULT, and DISPLAY/BENCHMARK keypad. This keypad is enabled under the following circumstances:

• When you press either KP8 or KP9 on the default Review subsystem keypad (see Figure 5.1)

• When you issue the SHOW command with the /BENCHMARK or /RESULT qualifier from the Review subsystem command line

• When you specify the DISPLAY/BENCHMARK command from the DTM> command line

Table 5.5 describes the key definitions shown in Figure 5.2.

Key Sequence	Key Definition
KP0	Displays the next screen.
KP1	Displays the previous screen.
KP5	Displays the first screen in the file.
KP7	Toggles the screen number display. If the screen number display is on the screen, pressing KP7 removes it. If the screen number display is not on the screen, pressing KP7 displays it.
PF2	Displays review subsystem Keypad.

Figure 5.3 shows the SHOW/DIFFERENCES keypad. When you press KP7 on the Review keypad, or issue the SHOW command with the /DIFFERENCES qualifier from the Review subsystem command line, the default Review subsystem SHOW/DIFFERENCES keypad becomes available.

Table 5.6 describes the key definitions shown in Figure 5.3.

Key Sequence	Key Definition
KP0	Displays the next screen.
KP1	Displays the previous screen.
KP3	Shifts split-screen mode. If the top (or bottom) half of the result and benchmark files is displayed, pressing KP3 displays the other half of the two screens. If the right (or left) half of the result and benchmark screens is displayed, pressing KP3 displays the other half of the two screens.
KP4	Changes highlighting of differences for screens that have not been displayed. Differences are highlighted in bold reverse video. You can change this so differences are underlined. Pressing KP4 changes highlighting for the screens that have not been displayed. After a screen is displayed, you cannot change the way its differences are highlighted.
KP5	Displays the first screen in the file.
KP7	Toggles the screen number. If the screen number is on the screen, pressing KP7 removes it. If the screen number is not displayed,pressing KP7 displays it.
KP8	Toggles split-screen mode. If you are in full-screen mode,pressing KP8 puts you in split-screen mode. If you are in split-screen mode,pressing KP8 switches you between horizontal split-screen mode and vertical,split-screen mode.
KP9	Toggles full-screen mode. If you are in split-screen mode,pressing KP9 puts you in full-screen mode. If you are in full-screen mode,pressing KP9 switches you between displaying full screens from the result and benchmark files.
PF2	Displays review subsystem Keypad.

This section describes how to display test results in the terminal environment. See Chapter 2 for information about displaying tests in a DECwindows environment.

The Review subsystem SHOW command displays information about result descriptions and displays the output files associated with result descriptions to the display device (SYS$OUTPUT).

You can specify a result description expression with the SHOW command, which identifies the result descriptions about which information is displayed. If you do not specify a result description, VSI Digital Test Manager displays information about the current result description.

If you do not include an output-file qualifier, VSI Digital Test Manager displays information about the specified result descriptions rather than an output file.

DTM_REVIEW> SHOW/BENCHMARK
Benchmark File DUA0:[USER01.DTMLIB]MSGTEST.BMK For Result Description
MSGTEST
   .
   .
   .
DTM_REVIEW> 

When you enter a SHOW command with an output file qualifier for an interactive terminal test, VSI Digital Test Manager displays the result or benchmark files screen by screen, record by record, or character by character (depending on the comparison type).

For a comparison using the /SCREENS qualifier, the differences are displayed in split-screen mode; the top half of a result file screen is displayed on the top half of the screen, and the top half of the corresponding benchmark screen is displayed on the bottom half of the screen. You can also view the differences in full-screen mode.

When you enter the SHOW command with the /RESULT or /BENCHMARK qualifiers, an initial banner screen (Screen 0) is displayed. Subsequent screens are numbered for easy referencing. Press RETURN to access the next screen.

When you display a result or benchmark file, the specified file is displayed screen by screen. The screen number, initially on the top right corner of the screen, shows the number of the screen and type of file currently displayed.

To view other screens in the result and benchmark files, you must use the default keypad for the SHOW/RESULT and SHOW/BENCHMARK screens. Section 5.2.1.5 describes the SHOW/RESULT or SHOW/BENCHMARK keypad keys and their associated functions.

When you enter the SHOW command with the /DIFFERENCE qualifier, an initial banner screen (Screen 0) is displayed, as shown in Figure 5.4.

When you display a difference file, you are in horizontal split-screen mode by default; the terminal screen is divided horizontally into two windows. The top window displays the top half of a screen from the result file, and the bottom window displays the top half of the corresponding screen from the benchmark file.

When you display a difference file for a failed, interactive terminal test from its DECwindows interface, VSI Digital Test Manager attempts to create a 49-line DECterm screen. This is too large for some monitors. If you cannot examine the full contents of an interactive terminal test's difference file from the DECwindows interface, you will need to use the DCL interface.

To view other screens in the result or benchmark file, you must use the default keypad for the SHOW/DIFFERENCES screen. Section 5.2.1.5describes the SHOW/DIFFERENCES keypad keys and their associated functions.

The PRINT command does not queue the selected files for printing until after you exit from the Review subsystem with the EXIT command, or by pressing Ctrl/Z. If you leave the Review subsystem by pressing Ctrl/C, or entering the EXIT/NOPRINT/NOINSERT command, the selected files are not printed.

If you do not include an output file qualifier, VSI Digital Test Manager prints the result file associated with the current result description.

You might want to update an existing benchmark file if you have changed an application in a way that changes the expected results for a test.

DTM_REVIEW> SELECT MSGTEST
Result Description MSGTEST    Comparison Status :  Unsuccessful

DTM_REVIEW> UPDATE
%DTM_I_UPDATED, the benchmark for test MSGTEST has been updated

DTM_REVIEW>

The result file is renamed as the benchmark file and the reference to the former benchmark file is removed. If the former benchmark file is in the library, it is deleted.

When you update a benchmark file that is stored in a Code Management System (CMS) library, VSI Digital Test Manager reserves the existing benchmark element in the CMS library and replaces it with the result file from the current test run. If you specified a CMS generation for the existing benchmark file with the CREATECOLLECTION/CLASS command, VSI Digital Test Manager inserts the updated benchmark file as the new generation in the specified CMS class.

If you update the benchmark file for an interactive terminal test, the file containing benchmark screens is also updated. If you update the benchmark file for a DECwindows test, any masks in the existing benchmark are transferred to the new benchmark.

This section shows you how to create a benchmark file for a new test in the Review Subsystem.

DTM> REVIEW MAIL_COLL 
Collection MAIL_COLL with 1 test was created on 31-OCT-1998 07:19:16
by the command:
        CREATE COLLECTION MAIL_COLL MAIL_TEST "Creating the MAIL test
         collection"
        Last Review Status = not previously reviewed
        Success count = 0
        Unsuccessful count = 0
        New test count = 1
        Updated test count = 0
        Comparisons aborted = 0
        Test not run count = 0

Result Description MAIL_TEST       Comparison status : New test

DTM_REVIEW> SHOW/RESULT 
  .
  .
  .
DTM_REVIEW> UPDATE 
%DTM-I-UPDATED, the benchmark file for test MAIL_TEST has been updated
DTM_REVIEW> SHOW/SUMMARY 
Collection MAIL_COLL with 1 test was created on 31-OCT-1998 07:19:16
by the command:
        CREATE COLLECTION MAIL_COLL MAIL_TEST "Creating the MAIL test
         collection"
        Last Review Date = 02-NOV-1998 08:19:20
        Success count = 0
        Unsuccessful count = 0
        New test count = 0
        Updated test count = 1
        Comparisons aborted = 0
        Test not run count = 0

DTM_REVIEW> PRINT/BENCHMARK 
%DTM-S-PRINT,  file DUA0:[USER01.DTMLIB]MAIL_TEST.BMK of test MAIL_TEST
selected for printing
DTM_REVIEW> EXIT 
%DTM-S-PRINTQD, print job has been sent to the print queue
-DTM-I-TEXT, Job MAIL_TEST(queue SYS$PRINT, entry 710)started on SYS$PRINT
%DTM-EXIT, leaving Review subsystem
DTM>

Issuing the STOP command is the recommended way to terminate a collection executing in batch mode. Pressing Ctrl/C is the recommended way to stop a collection that is running interactively. Taking either of these actions causes VSI Digital Test Manager to clean up the VSI Digital Test Manager library, after which you must enter the COMPARE command to compare the partially run collection before reviewing it.

If you do not use the STOP command or press Ctrl/C to stop a collection,errors might occur and you will not be able to compare or review the collection. Before using the collection, you must enter the VERIFY/RECOVER command to cleanup the library, correct the errors, and mark the tests that did not run.

Reviewing a partially run collection is especially important if the collection is large. Following the instructions described, prepare the partially run collection for review. Then, from the Review subsystem, examine the tests that ran and use the INSERT/NOT_RUN command to create a group containing all the tests that did not run. This group can then be included in a collection, executed, and reviewed. Creating a group while reviewing a collection is described in Chapter 6.

In a DECwindows environment, you specify test prologue and test epilogue files in text entry fields on the Create Test... or Modify Test... dialog box.

Test prologue and epilogue files are associated with a specific test description and are executed whenever the test is executed.

A prologue file can set up an environment for the test template file. For example, a test prologue file can define local variables, retrieve elements from a CMS library (using the FETCH command), or set default values.

An epilogue file can clean up the environment after the template file is run. Test epilogue files can also remove run-dependent data from a test's result file. By running a test once and checking for run-dependent data in the result file, you can determine the need to filter run-dependent data with an epilogue file. See Section 6.3.3.3 and Section 6.4 for more information.

You can disassociate a prologue or epilogue file from a test description by specifying the MODIFY TEST_DESCRIPTION command with the /NOPROLOGUE or /NOEPILOGUE qualifier. This does not delete the prologue or epilogue file.

Example 1.1 shows the disabling and enabling of broadcast messages as part of a recorded test. However, it is possible that broadcast messages can appear in the recording before the SET BROADCAST=NONE command is entered, or after the SETBROADCAST=ALL command is entered and before the test recording is terminated. By placing these commands into test prologue and epilogue files, you eliminate that possibility. The following sections show how to place these commands into test prologue and epilogue files.

$!  DTMNOBROADCAST.COM
$!
$!  Disable broadcast messages before recording.
$!
$ SET BROADCAST=NONE$!

DTM> MODIFY TEST_DESCRIPTION MAIL_TEST -
_DTM> /PROLOGUE=DUA0:[USER01.PROLIB]DTMNOBROADCAST.COM
_Remark: Adding the prologue to disable broadcast messages
%DTM-S-MODIFIED, test description MAIL_TEST modified
DTM>

$!  RESETBROADCAST.COM
$!
$!  Re-enable broadcast messages after recording.
$!
$ SET BROADCAST=ALL$!

DTM> MODIFY TEST_DESCRIPTION MAIL_TEST -
_DTM> /EPILOGUE=DUA0:[USER01.EPILIB]RESETBROADCAST.COM
_Remark: Adding the epilogue to re-enable broadcast messages
%DTM-S-MODIFIED, test description MAIL_TEST modified
DTM>

When you associate prologue and epilogue files with a library using the SETPROLOGUE and SET EPILOGUE commands, those files become the default prologue and epilogue files for any collections you create. The default prologue and epilogue files are invoked whenever you execute one of those collections.

To cancel the default files, specify the SET NOPROLOGUE and SET NOEPILOGUE commands.

To associate different prologue and epilogue files when creating subsequent collections, specify new files with the CREATE COLLECTION command and the /PROLOGUE and /EPILOGUE qualifiers.

In a DECwindows environment, you specify collection prologue and collection epilogue files in text entry fields on the Create Collection... dialog box.

To create a collection that does not use the default collection prologue and epilogue files, specify the CREATE COLLECTION command with the /NOPROLOGUE and /NOEPILOGUE qualifiers.

$!              –-- COLLECTION_PROLOGUE.COM –--
$! Collection prologue file for running the Collector in batch mode
$!
$  SET VERIFY$
!$  TRANSLIT:==$'F$LOGICAL("TRANSLIT")'
$!
$! Test DTM variable to determine whether or not to run PCA prologue
$!
$  IF USE_PCA .EQS. "FALSE" THEN EXIT
$!
$! Define the Collector initialization file
$!
$  DEFINE PCAC$INIT USE_PCA_INIT_FILE
$!
$! End of collection prologue file

DTM> SET PROLOGUE DUA0:[USER01.DTM_CMSLIB]COLLECTION_PROLOGUE.COM

In this example, the prologue file exists in the CMS library.

$!              –--  COLLECTION_EPILOGUE.COM  –--
$! Collection epilogue file for mailing test results to USER87,
$! upon completion of the test run.
$!
$ DTM SHOW COLLECTION 'DTM$COLLECTION_NAME'/FULL-
$ /OUTPUT='DTM$COLLECTION_NAME'.REPORT
$!
$ MAIL 'DTM$COLLECTION_NAME'.REPORT/SUBJECT="Collection summary" USER87
$!
$! End of collection epilogue file

DTM> SET EPILOGUE DUA0:[USER01.DTM_CMSLIB]COLLECTION_EPILOGUE.COM

In this example, the epilogue file exists in the CMS library.

After you create a group name with the CREATE GROUP command, you can insert one or more tests by listing the test names with the INSERT TEST_DESCRIPTION command. You can use wildcards when you specify test names to be inserted into a group. Tests are associated together by group name only; they are not relocated or copied.

DTM> CREATE GROUP BOUNDARIES "Creating Group BOUNDARIES" 
%DTM-S-CREATED, group BOUNDARIES created
DTM> CREATE GROUP LMARGIN "Creating Group LMARGIN"
%DTM-S-CREATED, group LMARGIN created
DTM> CREATE GROUP RMARGIN "Creating Group RMARGIN"
%DTM-S-CREATED, group RMARGIN created
DTM> CREATE GROUP MARGINS "Creating Group MARGINS"
%DTM-S-CREATED, group MARGINS created

DTM> INSERT TEST_DESCRIPTION LMTEST1,LMTEST2,LMTEST3 LMARGIN 
_Remark: Grouping the left margin tests
%DTM-I-INSERTED, test description LMTEST1 inserted into group LMARGIN
%DTM-I-INSERTED, test description LMTEST2 inserted into group LMARGIN
%DTM-I-INSERTED, test description LMTEST3 inserted into group LMARGIN
%DTM-S-INSERTIONS, 3 insertions completed

DTM> INSERT TEST_DESCRIPTION RMTEST1,RMTEST2,RMTEST3,RMTEST4 RMARGIN 
_Remark: Grouping the right margin tests
%DTM-I-INSERTED, test description RMTEST1 inserted into group RMARGIN
%DTM-I-INSERTED, test description RMTEST2 inserted into group RMARGIN
%DTM-I-INSERTED, test description RMTEST3 inserted into group RMARGIN
%DTM-I-INSERTED, test description RMTEST4 inserted into group RMARGIN
%DTM-S-INSERTIONS, 4 insertions completed

DTM> INSERT GROUP LMARGIN,RMARGIN MARGINS 
_Remark: Grouping the margin tests together under MARGINS
%DTM-I-INSERTED, group LMARGIN inserted into group MARGINS
%DTM-I-INSERTED, group RMARGIN inserted into group MARGINS
%DTM-S-INSERTIONS, 2 insertions completed

DTM> INSERT GROUP MARGINS BOUNDARIES 
_Remark: Grouping the margin groups into a boundaries group
%DTM-I-INSERTED, group MARGINS inserted into group BOUNDARIES DTM>

When you insert a group into another group, you create a group hierarchy. Figure 6.1 shows the BOUNDARIES groups hierarchy.

Use the REMOVE TEST_DESCRIPTION command to disassociate tests from a group. Use the REMOVE GROUP command to disassociate groups from a group. These commands reverse the actions of the INSERT TEST_DESCRIPTION and INSERT GROUP commands. The remove commands do not delete any tests or groups.

DTM> REMOVE TEST_DESCRIPTION LMTEST2 LMARGIN
_Remark: Removing test 2 from LMARGIN
Confirm removal of test description LMTEST2 from group LMARGIN [Y/N] (N): Y
%DTM-I-REMOVED, test description LMTEST2 removed from group LMARGIN
DTM>

DTM> REMOVE GROUP RMARGIN MARGINS "Removing RMARGIN from MARGINS"
Confirm removal of group RMARGIN from group MARGINS [Y/N] (N): Y
%DTM-I-REMOVED, group RMARGIN removed from group MARGINS
DTM>

When you create a group, VSI Digital Test Manager continues to associate that name with a group, even if the group no longer contains tests. You can delete the associated group name by specifying the DELETE GROUP command.

DTM> REMOVE TEST_DESCRIPTION * RMARGIN
_Remark: Preparing to delete group RMARGIN
Confirm removal of test description RMTEST1 from group RMARGIN [Y/N] (N): Y
%DTM-I-REMOVED, test description RMTEST1 removed from group RMARGIN
   .
   .
   .
Confirm removal of test description RMTEST4 from group RMARGIN [Y/N] (N): Y
%DTM-I-REMOVED, test description RMTEST4 removed from group RMARGIN
%DTM-S-REMOVALS, 4 removals completed
DTM> DELETE GROUP RMARGIN “Deleting the RMARGIN group”
Confirm deletion of group RMARGIN [Y/N] (N): Y
%DTM-S-DELETED, group RMARGIN deleted
DTM>

You can modify one or more variable characteristics and delete variables.

DTM> MODIFY VARIABLE INPUT_FILE/VALUE=INPUT.RNO
_Remark: "Replacing value of INPUT_FILE with INPUT.RNO"
%DTM-S-MODIFIED, variable INPUT_FILE modified.

The variable expression parameter can be a variable name, a wildcard character, a wildcard in combination with a variable name, or a list of these separated by commas.

DTM> DELETE VARIABLE INPUT_FILE "Deleting the INPUT_FILE variable"
Confirm deletion of variable INPUT_FILE [Y/N] (N): Y
%DTM-S-DELETED, variable INPUT_FILE deleted.

Most tests use a variable's default value. However, certain tests might require special handling and variable values. For example, you might want to use one template file to run several tests. You can do this by using a variable in the template file to override the variable's value for each test description.

DTM> CREATE VARIABLE TEMPLDIR DUA0:[USER01.TMP]/GLOBAL
_Remark: Template Directory
%DTM-S-CREATED, logical variable TEMPLDIR created
DTM> MODIFY TEST_DESCRIPTION MECHANIX -
_DTM> /VARIABLE=(TEMPLDIR=DUA0:[USER01.PROJECT.TMP])
_Remark: "Change variable value when used in MECHANIX"
%DTM-S-MODIFIED, test description MECHANIX modified
DTM>

DTM> CREATE COLLECTION KEYTESTS * -
_DTM> /VARIABLE=(TEMPLDIR="DUA0:[USER01.TEST.TMP]")-
_Remark: New template directory for this collection

VSI Digital Test Manager supplies built-in variables that you can use in template, prologue, and epilogue files. The following sections describe these built-in variables and provide examples of how to use each one.

$!          –--  MAIL_TEMPLATE.COM  –--
$! TEMPLATE file for MAIL message sending commands
$!
$ @'dtm$test_name'.COM
$!
$! This new template file can be used with any test whose test
$! name is the same as that of the input file.

$! MEM.FIL -- Eliminate any "Memory Used:"
$!            messages from .RES files.
$ EDIT/EDT DTM$RESULT
c;32767('Memory Used:' dl) ex
EXIT
$ PURGE DTM$RESULT

Alternatively, you can define VSI Digital Test Manager variables and have the logicals automatically defined when recording tests or running collections. The following sections describe these variables and provide examples.

DTM> CREATE VARIABLE /LOGICAL DTM$DELAY_TIMEOUT "0 00:00:15.0" -
_DTM> "MY TIMEOUT"
DTM> MODIFY TEST MAIL /VARIABLE=DTM$DELAY_TIMEOUT "NEW TIMEOUT" 

DTM> CREATE VARIABLE /LOGICAL /GLOBAL DTM$OMIT_PRINTABLE_SCREENS "1"

$ DEFINE DTM$OMIT_PRINTABLE_SCREENS "1"
$ DTM COMPARE collection_name

DTM> CREATE VARIABLE /LOGICAL DTM$DATE_FILTER_MIN_YEAR "1998" -
_DTM> "FIRST YEAR IN FILTER RANGE"
DTM> MODIFY TEST MAIL /VARIABLE=DTM$DATE_FILTER_MIN_YEAR "1998" -
_DTM> "NEW FILTER RANGE START DATE"

DTM> CREATE VARIABLE /LOGICAL DTM$DATE_FILTER_MAX_YEAR "1998" -
_DTM> "LAST DATE IN FILTER RANGE"
DTM> MODIFY TEST MAIL /VARIABLE=DTM$DATE_FILTER_MAX_YEAR "1998" -
_DTM> "NEW FILTER RANGE END DATE"

DTM> CREATE VARIABLE /LOGICAL DTM$DATE_FILTER_STRING "DATE" -
_DTM> "STRING USED TO REPLACE DATES"
DTM> MODIFY TEST MAIL /VARIABLE=DTM$DATE_FILTER_STRING -
_DTM> "NEW DATE FILTER STRING"

DTM> CREATE VARIABLE /LOGICAL DTM$DATE_FILTER_FIRST "YEAR"
DTM> MODIFY TEST MAIL /VARIABLE=DTM$DATE_FILTER_FIRST

DTM> CREATE VARIABLE /LOGICAL DTM$DATE_MASK_MIN_YEAR "1998" -
_DTM> "FIRST YEAR IN MASK RANGE"
DTM> MODIFY TEST MAIL /VARIABLE=DTM$DATE_MASK_MIN_YEAR "1998" -
_DTM> "NEW MASK RANGE START DATE"

DTM> CREATE VARIABLE /LOGICAL DTM$DATE_MASK_MAX_YEAR "1998" -
_DTM> "LAST DATE IN MASK RANGE"
DTM>
MODIFY TEST MAIL /VARIABLE=DTM$DATE_MASK_MAX_YEAR "1998" -
_DTM> "NEW MASK RANGE END DATE"

    Transaction complete at 4.34
    Transaction complete at 11.47

Use the CREATE TEST_DESCRIPTION or MODIFY TEST_DESCRIPTION command with the /FILTER qualifier to associate filters with a specific test description.

When the test is executed, VSI Digital Test Manager filters the result file (after the epilogue file is run).

DTM> MODIFY TEST_DESCRIPTION MAIL_TEST/NOFILTER=DATE
_Remark: Disabling the DATE filter

The SHOW TEST_DESCRIPTION with the /FULL or /FILTER qualifier lists the filters associated with a specific test description.

For interactive terminal test results and benchmark screens comparison, VSI Digital Test Manager enables you to mask, or ignore, particular data that varies from one test run to the next. Masking is performed in the comparison phase of a collection run when interactive test screens are being compared.

If you specify more than one keyword, separate the keywords with commas and enclose the list in parentheses. If you specify only one keyword, omit the parentheses. Masking is performed in the order of the keywords shown above.

The user defined filter facility provides a mechanism whereby filters written as DEC Text Processing Utility (DECTPU) programs are automatically executed when tests are run. These filters are referred to as user filters.

This enables users to solve easily many filtering problems, often with a single-line program, while allowing full access to the facilities of DECTPU to solve more complex cases.

global_replace( identifier + ':[' , 'DEVICE:[' )
Usage:
DTM> FILTER/USER_FILTER=DEVICE.TPU TEST.DAT

Alternatively, filters can be developed using the TPU pattern style feature of Language-Sensitive Editor for OpenVMS.

DTM> CREATE VARIABLE /LOGICAL DTM$UF_DEVICE
"DISK1:[TEST.FILTERS]DEVICE.TPU"

DTM> MODIFY TEST /VARIABLE=DTM$UF_DEVICE

When the test is run, as part of a collection, the filter is applied.

FILTER
     /USER_FILTER
          /USER_FILTER=filename
          /USER_FILTER=(filename[,...])
          /NOUSER_FILTER (D)

DTM> MODIFY TEST CHECK_DEVICES /VARIABLE=DTM$UF_DEVICE=" "

DTM> RECORD /FILTER /VARIABLES TERMINAL_TEST1

PROCEDURE global_replace ( pattern_to_replace,
                              replacement_string;
                              search_mode,
                              evaluate_replacement,
                              convert_linefeeds)DESCRIPTION:
             Replace all occurrences of a given pattern with a
             given string in the buffer "filter_buffer".

global_replace ( 'UDISK' + number, 'DISK_NAME')

global_replace ( ("U"|null) + "DISK" + number, "DISK_NAME")

    37-NOV-0999 as a valid date.
    day := any(" 123") + digit;
    month := "JAN" | "FEB" | "MAR" | "APR" |
             "MAY" | "JUN" | "JUL" | "AUG" |
             "SEP" | "OCT" | "NOV" | "DEC";
             year := any(digits,4);
             date := day + "-" + month + "-" + year;
    global_replace( date, "dd-mmm-yyyy");

global_replace( LINE_BEGIN + LINE_END, '');

global_replace( LINE_BEGIN + (white_space|null) +  LINE_END, '');

day := any(" 123") + digit;
          month := "JAN" | "FEB" | "MAR" | "APR" |
                   "MAY" | "JUN" | "JUL" | "AUG" |
                   "SEP" | "OCT" | "NOV" | "DEC";
                   year := any(digits,4);
                   date := (day + "-"@day_part) + month +
                            ("-" + year@year_part);
global_replace( date, 'str(day_part) + "mmm" + str(year_part)',,ON);

global_replace (number+LINE_END, "x"+lf,,,ON)

global_replace (number+(LINE_END@sep),'"x"+STR(sep,lf)',,ON,ON)

global_replace ( "/*" + UNANCHOR + "*/", "/* Deleted */")

EDIT( filter_buffer, UPPER, OFF)

POSITION (BEGINNING_OF (filter_buffer));
                  LOOP
                    found_range := SEARCH_QUIETLY (number, FORWARD);
                    EXITIF found_range = 0;
                    POSITION (END_OF(found_range));
                    MOVE_HORIZONTAL(1);
                    value := INT(STR(found_range));
                    IF (value>350) AND (value<570)
                      THEN
                        COPY_TEXT ("XXX");
                        ERASE (found_range);
                    ENDIF;
                  ENDLOOP;

You create command files with a text editor and invoke them from the VSI Digital Test Manager subsystem level, from the Review subsystem level, or from within another command file. If you include only a file name with the @file-specification command, VSI Digital Test Manager assumes a file type of .COM.

If you specify a command but omit a required parameter, VSI Digital Test Manager prompts you for the missing parameter.

When you invoke VSI Digital Test Manager from a DCL command procedure, be sure to supply all required command parameters. If you omit a required parameter for which you would be prompted if you entered the command interactively, DCL reads the next line in the command file as the missing parameter rather than as a separate command. The second command is lost.

Use the DCLDEFINE command to define an initialization command file. VSI Digital Test Manager provides the OpenVMS logical name DTM$INIT that you define to identify a command file that you want VSI Digital Test Manager to execute each time you invoke it. The command file specified by DTM$INIT is not executed when you invoke VSI Digital Test Manager using the DTM/DECWINDOWS command.

$ DEFINE DTM$INIT SAMPLE_STARTUP.COM

!Initialization file to set library and define keys
!
!Establish the library! SET LIBRARY DUA0:[USER01.DTMLIB]
!
!Define keypad keys!
 DEFINE/KEY KP3/IF_STATE=DTM         "SHOW COLLECTION */FULL"/TERMINATE
 DEFINE/KEY PF4/IF_STATE=GOLD_DTM    "SET LIBRARY DUA0:[USER01.LIB_A]"/TERMINATE
 DEFINE/KEY KP1/IF_STATE=REVIEW      "NEXT/SUCCESSFUL"/TERMINATE
 DEFINE/KEY KP1/IF_STATE=GOLD_REVIEW "BACK/SUCCESSFUL"/TERMINATE

$ DTM/NOINIT

Use the SET BENCHMARK_DIRECTORY and SET TEMPLATE_DIRECTORY commands to establish default benchmark and template directories for the current VSI Digital Test Manager library. VSI Digital Test Manager processes files faster when you use default benchmark and template directories rather than specifying a directory with each file name.

If you do not specify default benchmark and template directories, VSI Digital Test Manager uses your current default directory (SYS$DISK:[]) for template files and the VSI Digital Test Manager library (DTM$LIB) for benchmark files.

In a DECwindows environment, default benchmark and template directories are specified on the Create Library or Modify Library dialog box. The Create Library dialog box is shown in Chapter 2.

After you create or modify a test description, you can override the default directories for the current VSI Digital Test Manager library (if defaults exist) by using the CREATE COLLECTION command with the /BENCHMARK_DIRECTORY and /TEMPLATE_DIRECTORY qualifiers. The specified directories are used for the benchmark and template files for the collection being created.

To remove a default benchmark or template directory without replacing it, enter the SET NOBENCHMARK_DIRECTORY or the SET NOTEMPLATE_DIRECTORY command. These commands return the benchmark directory to DTM$LIB and the template directory to SYS$DISK:[].

Every file has a user identification code protection associated with it. UIC protection is determined by an owner UIC and a protection code. UIC protection controls access to directories and files. When you attempt to gain access to a directory or file, the system checks for existing ACLs; it then checks the UIC protection code.

See VSI OpenVMS DCL Dictionary for more information about UIC protection.

$ SET PROTECTION=(S:RWE,O:RWE,G:RWE,W) [PROJ]TESTING.DIR

$ SET PROTECTION=(S:RWED,O:RWED,G:RW,W) [PROJ.TESTING]00DTM.CON

If you enable read-only access to a library directory or to the library control file, users cannot make changes to the contents of the library, execute tests, review tests, or perform comparisons. If you do not enable write access to the collection control files, users cannot review collections. Therefore, you should enable group read and write access to these files.

You can restrict a person's access to library files by using a UIC protection or ACL. If you do not use ACLs, all users of a particular VSI Digital Test Manager library must be in the same user group. If you want to define more selective protection (where various individuals in the user group have differing access), you can use ACLs for the library directory and its files and subdirectories.

The following sections summarize the procedures you can use to define the access to a VSI Digital Test Manager library. For more information, see the VSI OpenVMS DCL Dictionary and the VSI OpenVMS Guide to System Security.

An ACL consists of access control entries (ACEs) that grant or deny access by specific users to a library directory file or library file. ACLs, used with library directory files, enable you to define access to an entire library. ACL sused with library files enable you to establish specific control over access to library contents. Generally, OpenVMS ACLs are used in conjunction with the standard UIC-based protection as a way to fine-tune protection.

See the VSI OpenVMS DCL Dictionary for more information on these commands. See the VSI OpenVMS Guide to System Security for more information on using ACLs and ACEs.

IDENTIFIER=TESTGRP,OPTIONS=DEFAULT,S:RWED,O:RWED,G:RWED

IDENTIFIER=TESTGRP,OPTIONS=DEFAULT,ACCESS=READ+WRITE+DELETE

Because both timing and system load can affect the performance of the system and the application being tested, two or more session files recorded independently might result in completely different, yet valid, session files. This occurs even though you typed exactly the same keystrokes when recording both terminal sessions. Thus, performing a source comparison on two session files yields no useful information.

Because most OpenVMS terminal drivers are full duplex, when dealing with session files you should consider input and output to be asynchronous events. You can be typing input while the system is simultaneously writing output. As a result,input and output can appear mixed in a session file.

  .
  .
  .
Ia
Oa
Ib
Ob
Ic
Oc
Id
Od
Ie
Oe
If
Of
  .
  .
  .

  .
  .
  .
Iab
Oa
Ic
Obc
Ide
Ode
If
Of
  .
  .
  .

Although you typed the same input, ABCDEF, both times and the system echoed back the same letters, the two session files are different. Session files recorded under different circumstances will vary, but all session files are equally valid.

The session file will be further varied if the program outputs something to the screen other than what you type. For example, the program might output X whenever you type A, or it might output nothing at all. Asynchronous events, for example, entering Ctrl/T or a broadcast write that occurs while the system is echoing input, will change the session file.

The terminal driver might also arbitrarily place text in a buffer, or split it into groups before generating output. Even though a program might have issued a six-character QIO to output ABCDEF, the terminal driver might output the text as two groups, ABCD followed by EF; or as six separate characters; or as a single six-character string. The terminal driver might even output the text appended to some previously entered text that has not yet been output. Each elementary operation of the terminal driver results in a separate record being written to the session file. Thus, each record in the session file describes a single terminal-driver operation.

VSI Digital Test Manager also writes additional timing and control information to the session file.

8.1.2.1. Record Structure of Session Files

The record structure of a session file is extremely important. You must not change it except as described in this section.

Terminal Characteristics Block

The first record in a session file is a 12-byte (12-character) information block called a terminal characteristics block. This block of information describes the type of terminal on which the terminal session was recorded and the characteristics of that terminal. The terminal characteristics block is described in the OpenVMS I/O User's Reference Manual: Part 1 and is shown in Figure 8.1.

The 12 bytes in the record are stored low order to high order as three longwords. The record conforms to the structure returned by a sense mode terminal QIO with a P2 parameter of 12. The first byte has the value DC$_TERM.

Note

You must not change the terminal characteristics block in any way. Any change to this record can invalidate the entire file. Do not add bytes to this record or delete bytes from it. This record must contain exactly 12 bytes.

Subsequent Records

All subsequent records in the session file begin with a record type, which is a one-byte indicator that describes the contents of the record. The record type is a number, usually represented by its corresponding ASCII character. For example, the decimal number 66 is referred to by the letter B. Table 8.1 shows the formats for all possible record types.

Record Type	Meaning	Description
B	BEGIN_COMPARE	Restarts automatic screen compare terminated by a previous E record. When an input point is reached, VSI Digital Test Manager automatically marks the current screen for comparison with the corresponding screen in the benchmark file.
C	COMPARE_SCREEN	Marks the current screen for comparison, even though automatic screen comparison is turned off.
E	END_COMPARE	Terminates automatic screen compare. When an input point is reached, VSI Digital Test Manager automatically marks the current screen for comparison with the corresponding screen in the benchmark file.
I	INPUT	Contains characters you input, that is, characters you type at the terminal. This record usually contains only one character. These characters do not automatically echo to the screen unless the terminal is set in half-duplex mode. This input is usually echoed in a subsequent O record.
O	OUTPUT	Labels the record as containing characters output by the system and displayed on the terminal.
T	TIMING	Contains a standard OpenVMS delta time specification. This time interval represents the clock time elapsed between the previous I (INPUT) record and the next I record.
W	WAIT	Contains a standard OpenVMS delta time specification in the following format: dddd hh::mm:ss.cc. It produces a pause of the specified length in the input stream when the terminal session is played back. A sample time interval of 10 seconds appears as follows: {WAIT} 0000 00:00:10.00
!	BEGIN_COMMENT	Contains a comment. This record is ignored.
0	Null or OUTPUT	This record type is supported for compatibility,though its use is not recommended. VSI Digital Test Manager interprets this as an O record.
1	Ctrl/A or INPUT	This record type is supported for compatibility, though its use is not recommended. VSI Digital Test Manager interprets this as an I record.

The T and W (TIMING and WAIT) records contain standard OpenVMS delta time specifications. A sample time interval of 2.3 seconds appears in a T record as follows:

T0 :00:02.3

It appears in a W record as follows:

W0 :00:02.3

In T records, the time interval represents the elapsed clock time between the previous and next I records. VSI Digital Test Manager does not normally record timing intervals of less than one second.

When VSI Digital Test Manager replays a terminal session, T records cause a time delay on the input stream when VSI Digital Test Manager replays the session at the speed at which it was recorded.

In W records, this time interval causes a pause in the input stream of the specified duration.

DTM> EXTRACT
_session file: TEST1.SESSION
_input file: TEST1.INP

%DTM-S-EXTRACTED, input file DUA0:[USER01.DTMLIB]TEST1.INP created
DTM>

SHOW DEFAULT{<CR>}

The input file TEST1.INP contains the text entered during the terminal session (the DCL command SHOW DEFAULT) and the special string {<CR>}, which terminates the entered text. Note that output supplied by the system is not included in the input file. You can edit this input file and then use it to generate a new session file.

All nonprinting characters and recording functions found in session files are replaced in input files by special strings, which have the following format: {special-string}

Special strings are textual representations for nonprinting characters and recording functions. They are enclosed within braces ({}).

DTM> EXTRACT session-file-specification [input-file-specification]/INTERACTIVE

The session file specification is the file specification of the session file from which VSI Digital Test Manager extracts an input file. If you specify a filename for the session file without specifying a file type, the file type defaults to .SESSION.

The input file specification is the file specification for the input file being created. If you do not specify an input file specification, the file specification defaults to session-file-name.INP. If you specify an input filename without specifying a file type, the file type defaults to .INP.

You can store both files in CMS libraries. The EXTRACT command can fetch the session file from a CMS library and place the input file in the CMS library.

If you subsequently record a session file from an input file stored in a CMS library, you must first fetch the input file from the CMS library by issuing the appropriate CMS commands.

The EXTRACT command takes the /[NO]LOG and /TERMINATION_CHARACTER qualifiers. The /TERMINATION_CHARACTER qualifier specifies the termination character that VSI Digital Test Manager is to use when translating the recording functions in the session file to special strings in the input file. If the interactive terminal session you are recording does not use the default termination character (Ctrl/P), you need not specify a different termination character. If the interactive terminal session you are recording uses Ctrl/P for its own purposes, you must specify a different termination character.

DTM> EXTRACT TEST1.SESSION TEST1_A.INP

%DTM-S-EXTRACTED, input file DUA0:[USER01.DTMLIB]TEST1_A.INP created
DTM>

You can create an input file using any text editor. See Section 8.2.2.2 for information on using special strings in input files.

{BEGIN_COMMENT}
! This is a sample input file. When used, it calls up EDT
! to create a file, enters some text into the buffer, and
! moves around that text, finally quitting without saving
! the file.
{END_COMMENT}
edit/edt sample.tmp{<CR>}
change{<CR>}
{BEGIN_COMMENT}
! Enter the text into the buffer.
{END_COMMENT}
This is the first line of the file.{<CR>}
This is the
second line of the file.{<CR>}
{UP_ARROW}{UP}{KP2}
{SPACE}This is more of the first line.
{Ctrl/Z}
quit{<CR>}

The RECORD command with the /INPUT qualifier specifies that VSI Digital Test Manager record a new session file by initiating an interactive terminal session and taking input from the specified input file. If you do not also specify the /APPEND qualifier, VSI Digital Test Manager terminates the interactive terminal session when the input file is exhausted. If you include both the/INPUT and /APPEND qualifiers, VSI Digital Test Manager leaves the terminal in record mode when the input file is exhausted. You can then continue the terminal session interactively and terminate it by entering the termination character(Ctrl/P) twice.

DTM> RECORD/INPUT=TEST2.INP
_Remark: Recording TEST2.SESSION from TEST2.INP
%DTM-I-DEFAULTED, benchmark file name defaulted to TEST2.BMK
%DTM-I-DEFAULTED, template file name defaulted to TEST2.SESSION
%DTM-I-BEGIN, your interactive test session is now beginning...


$ show default
  DUA0:[USER01.DTMLIB]

^P

%DTM-I-BMK_SAVED, benchmark has been saved in file
 DUA0:[USER01.DTMLIB]TEST2.BMK;1
%DTM-S-RECORDED, test TEST2 has been successfully recorded in
 DUA0:[USER01.DTMLIB]TEST2.SESSION
DTM>

DTM> RECORD/INPUT=SAMPLE_TEST.INP
_Remark: Recording a sample session file
%DTM-I-DEFAULTED, benchmark file name defaulted to SAMPLE_TEST.BMK
%DTM-I-DEFAULTED, template file name defaulted to SAMPLE_TEST.SESSION
%DTM-I-BEGIN, your interactive test session is now beginning...
Type Ctrl/P twice to terminate the session.


$ show default
  DUA0:[USER01.DTMLIB]
$ show time

$ ^P^P

^P

%DTM-I-BMK_SAVED, benchmark has been saved in file
 DUA0:[USER01.DTMLIB]SAMPLE_TEST.BMK;1
%DTM-S-RECORDED, test SAMPLE_TEST has been successfully recorded in
 DUA0:[USER01.DTMLIB]SAMPLE_TEST.SESSION
DTM>

When you enter the INSERT recording function Ctrl/P-I during an active recording session, it specifies that VSI Digital Test Manager is to take input from the specified input file. When you enter the INSERT recording function, VSI Digital Test Manager prompts you for the file specification for a single input file. VSI Digital Test Manager then takes input from the specified input file and returns control to the terminal when the input file is exhausted.

DTM> RECORD/INPUT=SAMPTEST.INP
_test name: SAMPTEST
_Remark: Recording SAMPTEST.SESSION from SAMPTEST.INP
%DTM-I-DEFAULTED, benchmark file name defaulted to SAMPTEST.BMK
%DTM-I-DEFAULTED, template file name defaulted to SAMPTEST.SESSION
%DTM-I-BEGIN, your interactive test session is now beginning...
Type Ctrl/P twice to terminate the session.


   .
   .
   .



^PI

_Input file: SAMPTEST2.INP

   .
   .
   .


^P^P

^P

%DTM-I-BMK_SAVED, benchmark has been saved in file
 DUA0:[USER01.DTMLIB]SAMPTEST.BMK;1
%DTM-S-RECORDED, test SAMPTEST has been successfully recorded in
 DUA0:[USER01.DTMLIB]SAMPTEST.SESSION
DTM>

During a terminal session, you can read input from multiple input files sequentially. The input files cannot be nested. VSI Digital Test Manager ignores any INSERT recording functions in an input file.

The process of creating an input file is not terminal specific. When you extract an input file from a session file, all control codes and other nonprinting characters are translated to special strings, regardless of whether they have meaning to the terminal you are using to perform the translation.

The process of creating a session file from an input file is terminal specific. When you record a session file, VSI Digital Test Manager translates all special strings back to control codes and other nonprinting characters based on the terminal characteristics for the recording terminal. If VSI Digital Test Manager encounters a special string that it cannot translate for the recording terminal, the braces are stripped off the special string and the characters representing the untranslated special string are printed in the session file. They are also displayed on the terminal screen along with an error message.

Anything you type on a terminal while input is being taken from an input file will have no immediate effect on the terminal session. All or part of what you type might be stored as type-ahead and might appear when the input file is exhausted and control is returned to the terminal.