Installation and Uninstallation

This chapter describes how to install, start, and uninstall ProSort.

1. Installation and Startup

ProSort is started by executing the executable file without a separate installation process.

1.1. Startup

The following are the steps for starting ProSort.

  1. Decompress the compressed binary file (.gz) in the desired directory.

    The following directory structure is created.

    prosort
   |----- bin
   |       |----- prosort
   |       |----- psapi
   |       |----- psctl
   |       |----- pserr
   |----- config
   |       |----- gen_tip.sh
   |       |----- prosort_tip.template
   |       |----- revision
   |       |----- variant
   |----- docs
   |----- include
   |       |----- prosort.h
   |       |----- prosort_rc.h
   |----- lib
   |       |----- libprosort.a
   |       |----- libprosort.so
   +----- license
   +----- tests

    The following describes each directory.

    bin

    Contains the ProSort execution file and the following utilities.

    Utility Description

    prosort

    Sorts, merges, transforms, and operates data.

    psapi

    ProSort API used to sort, merge, transform, and operate data.

    psctl

    Creates a data file that meets specific conditions.

    pserr

    Shows detailed descriptions of error codes.
    config

    Contains the configuration file used to execute ProSort.

    docs

    Contains ProSort guides.

    include

    Contains header files used to create applications with ProSort APIs.

    lib

    Contains library files used to create applications with ProSort APIs.

    license

    Contains the ProSort license file (license.xml), which can be read with any text editor.

    tests

    Contains sample ProSort scripts and programs.

  2. Execute prosort in the prosort/bin directory.

1.2. Setting Parameters

To use ProSort, set parameters in the $PROSORT_SID.tip file.

Basic parameters are generated automatically in $PROSORT_SID.tip if executing the gen_tip.sh script file in the config directory.

The following is a sample file.

<$PROSORT_SID.tip>

SORT_THREAD_COUNT=2
SORT_READ_BLK_SIZE=4194304
SORT_WRITE_BLK_SIZE=65536
MERGE_READ_BLK_SIZE=2097152
USE_AIO_READ=Y
USE_AIO_WRITE=Y
MAXIMIZE_MEM_RUN_SIZE=Y
USE_STRICT_MEMORY=Y

For more information, refer to Parameters.

2. Verifying Startup and Operation

Verify that ProSort starts up and operates correctly by using the following ProSort commands and API functions.

2.1. ProSort Command

The prosort command can be used in systems where ProSort is installed. Before using the command, environment variables must be set appropriately.

The usage of the prosort command is as follows:

prosort [-h] [-v] [-s] [-j] [script file]
Option Description

-h

Shows brief help for prosort command.

-v

Shows ProSort version.

-s

Shows the following status information after executing the script file.

  • Total Infile Size: input file size

  • Total Allocated Data Block Size: total size of data blocks that save all records

  • Total Allocated Key Block Size: total size of key blocks that save sorting key fields

  • Total Allocated Key Store Size: total size of key stores that save key fields' prefixes

  • Maximum Allocated Memory Size: maximum memory usage

  • Total Disk Read Size: size of data read from disks

  • Total Disk Write Size: size of data written to disks

  • Current Data Block Size: unprocessed data block size

  • Current Key Block Size: unprocessed key block size

  • Current Key Store Size: unprocessed key store size

-j

Shows profile information after executing the script file.

Since profile information varies depending on the product version, this guide does not explain it.

script file

Script file directory path and name. If omitted, a standard input script file is used.

2.2. ProSort API Functions

ProSort provides the following C API functions. All functions return a value of 0 upon success, and return a negative value upon failure.

Function Description

prosort_run_script

Performs all operations in a script.

prosort_setup

Initializes the data required to execute ProSort.

prosort_setup_with_userexit

Has the same input/output parameters as prosort_setup.

prosort_release_record

Inputs data by record for a sort operation.

prosort_release_end

Ends data input for a sort operation.

prosort_return_record

Gets an operation result by record.

prosort_end

Returns all resources used for operations.

prosort_get_error_number

Gets an error number.

prosort_get_error_message

Gets an error message.

2.2.1. prosort_run_script

Performs all operations in a script.

  • Prototype

    int prosort_run_script(const char *const script);

  • Parameter

    Parameter Description

    script

    ProSort script used for operations.

  • Example

    #include <stdio.h>
#include <stdlib.h>
#include "prosort.h"

int
main(int argc, char **argv)
{
    int rc;
    char *script =
        "DEFREC FIXED,SIZE=256 "
        "MEMORY 500M "
        "INFILE = (x1,100M, x2, 100M, x3, 100M, x4, 100M) "
        "SORT FIELDS=(1,13,AC,A) "
        "OUTFIL FNAMES = xresult ";

    rc = prosort_run_script (script);
    if (rc < 0) {
        fprintf(stderr,"Error[%d]:\n", rc);
        fprintf(stderr,"%s\n", ps_err_code_to_cause(rc));
        fprintf(stderr,"%s\n", ps_err_code_to_action(rc));

        exit (rc);
    }

    return 0;
}

2.2.2. prosort_setup

Initializes the data required to execute ProSort. If prosort_run_script is used, prosort_setup does not need to be called.

  • Prototype

    typedef int (*callback_func_t)(char **data_buf_addr,
                               int *data_buf_length);

int prosort_setup (const char *prosort_script,
                   callback_func_t *data_fetch_callback_funcs,
                   int data_fetch_callback_func_cnt,
                   void **ptr_prosort_context);

  • Parameter

    Parameter Description

    *prosort_script

    Pointer to a script used by ProSort APIs.

    *data_fetch_callback_ funcs

    Pointer to a callback function or its array.

    For merge or copy operations, the necessary data is inputted through a callback function. For merge operations, the data can be inputted through multiple callback functions.

    data_fetch_callback_ func_cnt

    Number of callback functions.

    **ptr_prosort_context

    Pointer to context information required to perform operations. The created context can be applied to another API for operations.

  • Example

    #include <stdio.h>
#include <stdlib.h>
#include "prosort.h"

static void
print_err_msg(int ec)
{
    fprintf(stderr,"Error[%d]:\n", ec);
    fprintf(stderr,"%s\n", ps_err_code_to_cause(ec));
    fprintf(stderr,"%s\n", ps_err_code_to_action(ec));
    fprintf(stderr,"%s\n", ps_err_get_last_msg());
}

/* 16-byte record */
#define REC_LEN 16

#define RAND (char) (rand() % 24) + 'A'
int
main(int argc, char **argv)
{
    int   i      = 0;
    int   j      = 0;
    int   rc     = 0;

    void *ps_ctx;

    char buf[REC_LEN];
    const char *script = "MEMORY 100M\n"
                         "DATASIZE 100M\n"
                         "DEFREC FIXED, SIZE=16\n"
                         "SORT FIELDS = (1,8,CH,A)\n";

    /* script setting */
    rc = prosort_setup (script, NULL, 0, &ps_ctx);

    if (rc < 0) {
        print_err_msg (rc);
        exit (rc);
    }

    /* record input */
    for (i = 0; i < 100; i++) {

        for (j = 0; j < REC_LEN - 1; j++)
            buf[j] = RAND;
        buf[j] = 0;

        rc = prosort_release_record (ps_ctx, buf, REC_LEN);
        if (rc < 0) {
            print_err_msg (rc);
            exit (rc);
        }
    }
    /* record input completed */
    prosort_release_end (ps_ctx);

    do {
        char outbuf[REC_LEN];
        char *outbufs[1];
        unsigned int len = REC_LEN;

        outbufs[0] = outbuf;

        rc = prosort_return_record (ps_ctx, outbufs, &len);
        if (rc < 0) {
            print_err_msg (rc);
            exit (rc);
        }
        if (len == 0)
            break;
        printf ("%s\n", outbuf);

    } while (1);

    prosort_end (ps_ctx);
    return 0;
}

2.2.3. prosort_setup_with_userexit

Has the same input/output parameters as prosort_setup except for func_e15, func_e32, func_e35, and func_e61.

  • Prototype

    typedef int (*user_exit_func_t)(const void *rec_addr,
                                const int rec_len,
                                int file_no,
                                void **ret_rec_addr,
                                int *ret_rec_len);

int prosort_setup_with_userexit(const char *prosort_script,
                                callback_func_t *data_fetch_callback_funcs,
                                int data_fetch_callback_func_cnt,
                                user_exit_func_t func_e15,
                                user_exit_func_t func_e32,
                                user_exit_func_t func_e35,
                                user_exit_func_t func_e61,
                                void **ptr_prosort_context);

  • Parameter

    Parameter Description

    *prosort_script

    Pointer to a script used by ProSort APIs.

    *data_fetch_callback_ funcs

    Pointer to a callback function or its array.

    For merge or copy operations, the necessary data is inputted through a callback function. For merge operations, the data can be inputted through multiple callback functions.

    data_fetch_callback_ func_cnt

    Number of callback functions.

    func_e15

    User Exit Function E15.

    func_e32

    User Exit Function E32.

    func_e35

    User Exit Function E35.

    func_e61

    User Exit Function E61.

    **ptr_prosort_context

    Pointer to the context information required to perform operations. The created context can be applied to another API for operations.

  • Example

    int func_e15(const void *rec_addr,
             const int rec_len,
             int file_no,
             void **ret_rec_addr,
             int *ret_rec_len)
{
    ...
}

int main(int argc, char **argv)
{
    int rc;
    void *ps_ctx;
    const char *script =
        "DEFREC FIXED,SIZE=150 \n"
        "DATASIZE 2M \n"
        "MEMORY 512M \n"
        "WORKSPACE = (./) \n"
        "SORT FIELDS=(1,2,A),FORMAT=BI \n"
        "INCLUDE COND=(1,2,CH,EQ,C'12',OR,1,2,CH,EQ,C'22',OR,1,2,CH,EQ,C'32') \n"
        "OUTFIL FNAMES=OUT1 \n";

    rc = prosort_setup_with_userexit(script, NULL, 0, func_e15, NULL,
                                     NULL, NULL, &ps_ctx);
    ...
}

2.2.4. prosort_release_record

Inputs data by record for sort operations.

  • Prototype

    int prosort_release_record(void *prosort_context,
                         const char *record,
                         const unsigned int record_length);

  • Parameter

    Parameter Description

    *prosort_context

    Pointer to a context created in prosort_setup.

    *record

    Pointer to a record.

    record_length

    Record length.

  • Example

    #include <stdio.h>
#include <stdlib.h>
#include "prosort.h"

static void
print_err_msg(int ec)
{
    fprintf(stderr,"Error[%d]:\n", ec);
    fprintf(stderr,"%s\n", ps_err_code_to_cause(ec));
    fprintf(stderr,"%s\n", ps_err_code_to_action(ec));
    fprintf(stderr,"%s\n", ps_err_get_last_msg());
}

/* 16-byte record */
#define REC_LEN 16

#define RAND (char) (rand() % 24) + 'A'
int
main(int argc, char **argv)
{
    int   i      = 0;
    int   j      = 0;
    int   rc     = 0;

    void *ps_ctx;

    char buf[REC_LEN];
    const char *script = "MEMORY 100M\n"
                         "DATASIZE 100M\n"
                         "DEFREC FIXED, SIZE=16\n"
                         "SORT FIELDS = (1,8,CH,A)\n";

    /* script setting */
    rc = prosort_setup (script, NULL, 0, &ps_ctx);

    if (rc < 0) {
        print_err_msg (rc);
        exit (rc);
    }

    /* record input */
    for (i = 0; i < 100; i++) {

        for (j = 0; j < REC_LEN - 1; j++)
            buf[j] = RAND;
        buf[j] = 0;

        rc = prosort_release_record (ps_ctx, buf, REC_LEN);
        if (rc < 0) {
            print_err_msg (rc);
            exit (rc);
        }
    }
    /* record input completed */
    prosort_release_end (ps_ctx);

    do {
        char outbuf[REC_LEN];
        char *outbufs[1];
        unsigned int len = REC_LEN;

        outbufs[0] = outbuf;

        rc = prosort_return_record (ps_ctx, outbufs, &len);
        if (rc < 0) {
            print_err_msg (rc);
            exit (rc);
        }
        if (len == 0)
            break;
        printf ("%s\n", outbuf);

    } while (1);

    prosort_end (ps_ctx);
    return 0;
}

2.2.5. prosort_release_end

Ends data inputs for sort operations.

  • Prototype

    int prosort_release_end(void *prosort_context);

  • Parameter

    Parameter Description

    *prosort_context

    Pointer to a context created in prosort_setup.

  • Example

    #include <stdio.h>
#include <stdlib.h>
#include "prosort.h"

static void
print_err_msg(int ec)
{
    fprintf(stderr,"Error[%d]:\n", ec);
    fprintf(stderr,"%s\n", ps_err_code_to_cause(ec));
    fprintf(stderr,"%s\n", ps_err_code_to_action(ec));
    fprintf(stderr,"%s\n", ps_err_get_last_msg());
}

/* 16-byte record */
#define REC_LEN 16

#define RAND (char) (rand() % 24) + 'A'
int
main(int argc, char **argv)
{
    int   i      = 0;
    int   j      = 0;
    int   rc     = 0;

    void *ps_ctx;

    char buf[REC_LEN];
    const char *script = "MEMORY 100M\n"
                         "DATASIZE 100M\n"
                         "DEFREC FIXED, SIZE=16\n"
                         "SORT FIELDS = (1,8,CH,A)\n";

    /* script setting */
    rc = prosort_setup (script, NULL, 0, &ps_ctx);

    if (rc < 0) {
        print_err_msg (rc);
        exit (rc);
    }

    /* record input */
    for (i = 0; i < 100; i++) {

        for (j = 0; j < REC_LEN - 1; j++)
            buf[j] = RAND;
        buf[j] = 0;

        rc = prosort_release_record (ps_ctx, buf, REC_LEN);
        if (rc < 0) {
            print_err_msg (rc);
            exit (rc);
        }
    }
    /* record input completed */
    prosort_release_end (ps_ctx);

    do {
        char outbuf[REC_LEN];
        char *outbufs[1];
        unsigned int len = REC_LEN;

        outbufs[0] = outbuf;

        rc = prosort_return_record (ps_ctx, outbufs, &len);
        if (rc < 0) {
            print_err_msg (rc);
            exit (rc);
        }
        if (len == 0)
            break;
        printf ("%s\n", outbuf);

    } while (1);

    prosort_end (ps_ctx);
    return 0;
}

2.2.6. prosort_return_record

Gets an operation result by record. Since there can be multiple outputs because of OUTFIL, the result is obtained by using a buffer array. The array[0] is SORTOUT (output without an OUTFIL execution), and OUTFIL execution results are outputted in the order as described in a script.

  • Prototype

    int prosort_return_record(void *prosort_context,
                        char **output_buffer,
                        unsigned int *output_buffer_length);

  • Parameter

    Parameter Description

    *prosort_context

    Pointer to a context created in prosort_setup.

    **output_buffer

    Pointer to the array of addresses of buffers to get output records.

    *output_buffer_length

    Pointer to the array to save output record length.

  • Example

    #include <stdio.h>
#include <stdlib.h>
#include "prosort.h"

static void
print_err_msg(int ec)
{
    fprintf(stderr,"Error[%d]:\n", ec);
    fprintf(stderr,"%s\n", ps_err_code_to_cause(ec));
    fprintf(stderr,"%s\n", ps_err_code_to_action(ec));
    fprintf(stderr,"%s\n", ps_err_get_last_msg());
}

/* 16-byte record */
#define REC_LEN 16

#define RAND (char) (rand() % 24) + 'A'
int
main(int argc, char **argv)
{
    int   i      = 0;
    int   j      = 0;
    int   rc     = 0;

    void *ps_ctx;

    char buf[REC_LEN];
    const char *script = "MEMORY 100M\n"
                         "DATASIZE 100M\n"
                         "DEFREC FIXED, SIZE=16\n"
                         "SORT FIELDS = (1,8,CH,A)\n";

    /* script setting */
    rc = prosort_setup (script, NULL, 0, &ps_ctx);

    if (rc < 0) {
        print_err_msg (rc);
        exit (rc);
    }

    /* record input */
    for (i = 0; i < 100; i++) {

        for (j = 0; j < REC_LEN - 1; j++)
            buf[j] = RAND;
        buf[j] = 0;

        rc = prosort_release_record (ps_ctx, buf, REC_LEN);
        if (rc < 0) {
            print_err_msg (rc);
            exit (rc);
        }
    }
    /* record input completed */
    prosort_release_end (ps_ctx);

    do {
        char outbuf[REC_LEN];
        char *outbufs[1];
        unsigned int len = REC_LEN;

        outbufs[0] = outbuf;

        rc = prosort_return_record (ps_ctx, outbufs, &len);
        if (rc < 0) {
            print_err_msg (rc);
            exit (rc);
        }
        if (len == 0)
            break;
        printf ("%s\n", outbuf);

    } while (1);

    prosort_end (ps_ctx);
    return 0;
}

2.2.7. prosort_end

Returns all resources used for operations.

  • Prototype

    int prosort_end(void *prosort_context);

  • Parameter

    Parameter Description

    *prosort_context

    Pointer to a context created in prosort_setup.

  • Example

    #include <stdio.h>
#include <stdlib.h>
#include "prosort.h"

static void
print_err_msg(int ec)
{
    fprintf(stderr,"Error[%d]:\n", ec);
    fprintf(stderr,"%s\n", ps_err_code_to_cause(ec));
    fprintf(stderr,"%s\n", ps_err_code_to_action(ec));
    fprintf(stderr,"%s\n", ps_err_get_last_msg());
}

/* 16-byte record */
#define REC_LEN 16

#define RAND (char) (rand() % 24) + 'A'
int
main(int argc, char **argv)
{
    int   i      = 0;
    int   j      = 0;
    int   rc     = 0;

    void *ps_ctx;

    char buf[REC_LEN];
    const char *script = "MEMORY 100M\n"
                         "DATASIZE 100M\n"
                         "DEFREC FIXED, SIZE=16\n"
                         "SORT FIELDS = (1,8,CH,A)\n";

    /* script setting */
    rc = prosort_setup (script, NULL, 0, &ps_ctx);

    if (rc < 0) {
        print_err_msg (rc);
        exit (rc);
    }

    /* record input */
    for (i = 0; i < 100; i++) {

        for (j = 0; j < REC_LEN - 1; j++)
            buf[j] = RAND;
        buf[j] = 0;

        rc = prosort_release_record (ps_ctx, buf, REC_LEN);
        if (rc < 0) {
            print_err_msg (rc);
            exit (rc);
        }
    }
    /* record input completed */
    prosort_release_end (ps_ctx);

    do {
        char outbuf[REC_LEN];
        char *outbufs[1];
        unsigned int len = REC_LEN;

        outbufs[0] = outbuf;

        rc = prosort_return_record (ps_ctx, outbufs, &len);
        if (rc < 0) {
            print_err_msg (rc);
            exit (rc);
        }
        if (len == 0)
            break;
        printf ("%s\n", outbuf);

    } while (1);

    prosort_end (ps_ctx);
    return 0;
}

2.2.8. prosort_get_error_number

Gets an error number.

  • Prototype

    int prosort_get_error_number(void *prosort_context);

  • Parameter

    Parameter Description

    *prosort_context

    Pointer to a context created in prosort_setup.

  • Example

    int main(int argc, char **argv)
{
    int rc;
    void *ps_ctx;
    const char *script =
        "DEFREC FIXED,SIZE=150 \n"
        "DATASIZE 2M \n"
        "MEMORY 512M \n"
        "WORKSPACE = (./) \n"
        "SORT FIELDS=(1,2,A),FORMAT=BI \n"
        "INCLUDE COND=(1,2,CH,EQ,C'12',OR,1,2,CH,EQ,C'22',OR,1,2,CH,EQ,C'32') \n"
        "OUTFIL FNAMES=OUT1 \n";

    rc = prosort_setup(script, NULL, 0, &ps_ctx);
    if(rc < 0) {
        fprintf(stderr, "Error number is %d\n", prosort_get_error_number(ps_ctx));
    }
    ...
}

2.2.9. prosort_get_error_message

Gets an error message.

  • Prototype

    int prosort_get_error_message(void *prosort_context,
                            char *error_message_buffer,
                            unsigned int *error_message_length);

  • Parameter

    Parameter Description

    *prosort_context

    Pointer to a context created in prosort_setup.

    *error_message_buffer

    Pointer to the buffer for getting an error message.

    *error_message_length

    Pointer to the error message length.

  • Example

    #define ERR_MSG_MAXLEN 1024
int main(int argc, char **argv)
{
    int rc;
    void *ps_ctx;
    char *err_msg;
    unsigned int err_msg_len;
    const char *script =
        "DEFREC FIXED,SIZE=150 \n"
        "DATASIZE 2M \n"
        "MEMORY 512M \n"
        "WORKSPACE = (./) \n"
        "SORT FIELDS=(1,2,A),FORMAT=BI \n"
        "INCLUDE COND=(1,2,CH,EQ,C'12',OR,1,2,CH,EQ,C'22',OR,1,2,CH,EQ,C'32') \n"
        "OUTFIL FNAMES=OUT1 \n";
    err_msg = (char *)malloc(sizeof(char) * ERR_MSG_MAXLEN);
    err_msg_len = ERR_MSG_MAXLEN;

    rc = prosort_setup(script, NULL, 0, &ps_ctx);
    if(rc < 0) {
        prosort_get_error_message(ps_ctx, err_msg, &err_msg_len);
        fprintf(stderr, "Error message is %s\n", err_msg);
    }
    ...
}

3. Uninstallation

Remove the ProSort directory by using the following OS command.

$ rm –rf prosort