Troubleshooting

This appendix describes troubleshooting for ProSort-related issues.

1. Memory Usage

If the size of memory used for sorting is specified, a sorting method depends on the size of the memory and an input file to sort.

There are two sorting methods as follows:

  • Internal Sorting (1-pass sorting)

    Loads an entire input file to memory and then sorts the file. Used when the specified memory size is greater than the input file size.

  • External Sorting (2-pass sorting)

    Loads a part of an input file to memory and saves the interim sorting result to a temporary file. Used when the specified memory size is less than the input file size.

The memory size can be specified in a script or the $PROSORT_SID.tip file. If the size is specified in both the script and the file, the value specified in the file has priority. The specified memory size does not mean physical memory usage, and it is used only to determine a sorting method. The actual memory usage includes not only the record storage but also the storage for sorting keys, etc. However, if the following is configured, actual physical memory usage cannot exceed the specified memory size.

USE_STRICT_MEMORY=Y

1.1. Memory Size Specification

  • Issue

    External sorting decreases performance.

  • Solution

    The specified memory size should be more than 80% of the input file size. It is recommended to set the memory size to 64 to 256 MB for external sorting.

1.2. Available System Memory

  • Issue

    If the specified memory size exceeds the available physical system memory size, an issue occurs. If the available system memory is insufficient, the operating system uses disks for swap-in and swap-out of virtual memory. This causes thrashing.

  • Solution

    The specified memory size must be less than the available physical system memory size.

1.3. Number of Concurrent Jobs

  • Issue

    If the number of concurrent jobs for external sorting is greater than the number of physical disks, an issue occurs.

  • Solution

    The number of concurrent jobs for external sorting must not be greater than the number of physical disks and must be set appropriately according to the rate of disk input and output.

1.4. Memory Usage for Jobs

  • Issue

    If the total size of memory allocated to each job exceeds the available physical system memory size, an issue occurs.

  • Solution

    Check the available physical system memory size, and set the memory size to the following value in the script or $PROSORT_SID.tip.

    A value less than (the available physical system memory size) / (the number of jobs)

1.5. Required Disk Space

  • Issue

    If a result or temporary file cannot be written to a disk because a disk space is insufficient, an issue occurs.

  • Solution

    For external sorting, interim results are saved to a temporary file. Since the temporary file size is the same as the input file size, a disk must have available space that is at least 2 times of the input file size.