WebtoB 5 Fix#2

This chapter describes the changes and additions in functionality and bug patches in WebtoB 5 Fix#2.

1. New Functions

1.1. Scale-out Detection and Graceful Connection Admin Commands

Added function to detect and process scaling in and out of WebtoB servers in the cloud environment.

You can use 'wsadmin > cfg -d' command to check the connection information.

Monitoring Settings Added to *DOMAIN

$>wsadmin
--- Welcome to WebtoB Admin (Type "quit" to leave) ---
$$1 qpsx2 (wsadm) [2017-09-11T13:27:46]: cfg -d
    DOMAIN: Name = webtob1,
         DomainId = 0,
         MaxSvc = 512,
         NHthChkTime(nct) = 30,
         CloudFriendWebServers(cfws) = 1,
         CloudFriendJsvServers(cfjs) = 1,
         CloudConnectionBalance(ccb) = 50,
         CloudDasAddress = "192.168.1.1:9736"

1.2. WebtoB Graceful Shut Down Implementation (wsdown -G)

Added 'wsdown -G' command to wait until all client FDs of an HTH are closed before shutting down the HTH. FDs are forcibly closed when timeout (seconds) expires.

wsdown -G (timeout_sec) Command

$>wsdown -G 10

WSDOWN for node(qpsx1) is starting:
        WSDOWN: HTL downed: Fri Jul 20 04:14:04 2018
        WSDOWN: HTH downed: Fri Jul 20 04:14:04 2018
        WSDOWN: WSM downed: Fri Jul 20 04:14:04 2018
        WSDOWN: Graceful Down

1.3. Session Redistribution During JEUS Scale Out

WebtoB maintains the session count for each connected JEUS and redistributes the sessions when JEUS scales out. It uses session ID hashing on sticky sessions and distributes them evenly across JEUS servers, and then configures them as sticky sessions again.

1.4. WSM and JEUS DAS Communication Protocol for the Cloud

Added communication protocol between WSM and DAS to allow auto scale event notification between WebtoB and JEUS in the cloud.

Configure JEUS DAS IP (Method 1) and port or set them in the environment variable (Method 2).
- Method 1)
  
  *DOMAIN.CloudDasAddress="Das.host.com:9736"
- Method 2)
  
  *DOMAIN.CloudDasAddress="$(DASURL)"
You can use the *DOMAIN.CloudServiceGroupId setting to identify WebtoB by each cloud service.
*DOMAIN.CloudDasAddress and CloudServiceGroupId
```
*DOMAIN
webtob1      CloudDasAddress = "192.168.1.1:9736",    #"$DASURL", $ENV
             CloudServiceGroupId = "group1"
```
Related issue: IMS-149295

1.5. SSL.DHParameter

Added this setting to support a 2048bit Diffie-Hellman group.

You can create a 2048 bit dhparams.pem file and set its path to the *SSL.DHParameter setting.
Creating a 2048 bit dhparams.pem File
```
$>wbssl dhparam -out dhparams.pem 2048
```
*SSL.DHParameter Setting
```
*SSL
ssl      ...
            DHParameter = = "$(WEBTOBDIR)/ssl/dhparams.pem"
         ...
```
Related issue: IMS-123759

1.6. ExcludeAllowHeaderOnError As NODE.Options

Added this option to exclude "Allow" from the response header for 400 level error responses to avoid security vulnerabilities.
ExcludeAllowHeaderOnError As NODE.Options
```
*NODE
webtob      WEBTOBDIR = "/home/webtob5",
            SHMKEY = 54800,
            Options = "ExcludeAllowHeaderOnError"
```
Related issue: IMS-131070

1.7. AUTHENT.ACCESS

Added this setting to specify a specific IP bandwidth in the *AUTHENT section.

*AUTHENT.AccessName Setting

*AUTHENT
auth1           Type = Basic,
                UserFile = "/home/server/webtob/auth/auth1.pass",
                AccessName = access1

*ACCESS
access1         Order = "allow,deny",
                Allow = "192.168.0.0/255.255.0.0"

*SVRGROUP
htmlg           SVRTYPE = HTML, AuthentName = auth1
jsvg            SVRTYPE = JSV

Related issue: IMS-137456

1.8. SVRGROUP.UserAgentRegExp

Added this setting to the *SVRGROUP section to route requests of the specified user agent to the server group for processing.

*SVRGROUP.UserAgentRegExp Setting

*SVRGROUP
jsvg           SVRTYPE = JSV, UserAgentRegExp = "(FireFox|Opera|Chrome|Safari)"
jsvg2          SVRTYPE = JSV, UserAgentRegExp = "Whale"

*SERVER
MyGroup        SVGNAME = jsvg, MinProc = 5, MaxProc = 10
MyGroup2       SVGNAME = jsvg2, MinProc = 5, MaxProc = 10

Related issue: IMS-136851, IMS-139916

1.9. DynamicServerAddress As REVERSE_PROXY.Options

Added this option to allow DNS resolution attempts when the ServerAddress set in the Reverse_Proxy section is modified dynamically.

DynamicServerAddress As *REVERSE_PROXY.Options

*REVERSE_PROXY
jsp_url
               ReverseProxyGroupName="jeus_rpg",
               ServerAddress = "192.168.1.1:5580",
               Options = "DynamicServerAddress"

Related issue: IMS-138268

1.10. JEUS WebSocket Selector

Added selector support to use WebSocket between WebtoB and JEUS without using reverse proxy.

To use WebSocket, you can use the WSProc setting in the SERVER section to obtain a WebSocket connection.
*SERVER.WSProc Setting
```
*SERVER
MyGroup        SVGNAME = jsvg, MinProc = 10, MaxProc = 20, WSProc = 1
```
This function is supported in JEUS 8 Fix#2 and later versions.

1.11. SSL.SSLServerCipherPref

This is similar to Apache SSLHonorCipherOrder or nginx ssl_prefer_server_ciphers. Added this setting to use the preferences defined in RequiredCipher.

*SSL.SSLServerCipherPref Setting

*SSL
ssl        CertificateFile = "$(WEBTOBDIR)/ssl/server.crt",
           CertificateKeyFile = "$(WEBTOBDIR)/ssl/server.key",
           RequiredCiphers = "ECDHE-RSA-AES128-GCM-SHA256:AES128-SHA256",
           PassPhraseDialog="exec:$(WEBTOBDIR)/ssl/pass.sh",
           SSLServerCipherPref = Y

Related issue: IMS-143514

1.12. Trial License

Added a trial license with 5 user limit for use with JEUS.
You can use the trial license with the WebtoB 5 Fix#2 Installer.

1.13. NODE.UpperDirRestrict

Added this setting to restrict access to the parent directory in addition to the existing method of using URL Rewrite. (Default value: N)

If UpperDirRestrict is set to Y and the request URI contains a '/', the server returns a "403 Forbidden" response.
*NODE.UpperDirRestrict Setting
```
*NODE
webtob      WEBTOBDIR = "/home/webtob5",
            SHMKEY = 54800,
            UpperDirRestrict = Y 
```
Related issue: IMS-125134

1.14. *REVERSE_PROXY(GROUP).AcccessName

Added this setting to control access to a specific IP bandwidth in the *REVERSE_PROXY section.

*REVERSEPROXY(GROUP).AccessName Setting

*REVERSE_PROXY
jsp_url
               ReverseProxyGroupName="jeus_rpg",
               ServerAddress = "192.168.1.1:5580",
               AccessName = access1

*ACCESS
access1         Order = "allow,deny",
                Allow = "192.168.0.0/255.255.0.0"

Related issue: IMS-149202

2. Updated Functions

2.1. Added "%A" Field to Access Log Format

You can use the "%A" field to specify a server IP.

"%A" Field in Access Log Format

*LOGGING
log1        Format = "%h %A %l %u %t \"%r\" %s %b",
            FileName = "C:/TmaxSoft/WebtoB5/log/access.log",
            Option = "sync"

Related issue: IMS-121596

2.2. Added AllServers Option to SERVER.Options

Added this setting to allow FILTERS process to perform filtering on HTMLS (HTML) requests whereas only JEUS (JSV) requests were processed by FILTERS before.

AllServers As *SERVER(filters).Options

*FILTER
sm_filter     RealPath = "/home/webtob5/config/filter/wbSmISAPI.so"

*SVRGROUP
filterg  SVRTYPE=filter, filter="sm_filter"

*SERVER
filters SVGNAME="filterg", Minproc=1, Maxproc=40, Options="AllServers"

Related issue: IMS-121825

2.3. Considering Threshold when Loading Balancing Requests from JEUS

WebtoB uses RR method to distribute requests from JEUS. Now it first routes requests to MSs with the number of running threads below its threshold limit.

Added threshold Column to the Result of wsadmin > st -j

$>wsadmin
--- Welcome to WebtoB Admin (Type "quit" to leave) ---

$$13 qpsx1 (wsadm) [2017-05-25T10:50:23]: st -j
HTH 0(18696): RDY
   --------------------------------------------------------------------------------
   svr_name  jengineno  threshold   cons       reqs      count      avg   jengineid
   --------------------------------------------------------------------------------
   MyGroup         0          -        10          0          0     0.0000  amV1c19kb21haW4vc2VydmVyMQ==(jeus_domain/server1)

Related issue: IMS-129440, IMS-131069

2.4. Kill Unresponsive FILTERS Process

HTH now kills an unresponsive FILTERS process that has timed out after client has been disconnected.
Related issue: IMS-136392

2.5. Added NODE.CheckUrlJsvExcept

It *NODE.CheckUrlJsvExcept is set to Y, CheckURL setting is disabled on requests sent to JEUS.

*NODE.CheckUrlJsvExcept Setting

*NODE
webtob          WEBTOBDIR = "/home/webtob5",
                SHMKEY = 54800,
                CheckURL = Y,
                CheckURLTo = "euc-kr",
                CheckURLFrom = "utf-8",
                CheckUrlJsvExcept = Y

Related issue: IMS-139327

2.6. Removed 2GB Limit for PUT Method Requests

The Content-Length limit of 2GB (INT_MAX) has been removed for PUT method requests.

2.7. Added Dual SSL Authentication

WebtoB now supports dual SSL certificates (ecdhe-rsa and ecdhe-ecdsa).

You can specify multiple certificates as *SSL.CertificateFile and CertificateKeyFile pairs separated by a comma (,).
*SSL.CertificateFile and CertificateKeyFile Settings
```
*SSL
ssl1            CertificateFile = "$(WEBTOBDIR)/ssl/server.crt,$(WEBTOBDIR)/ssl/server-ecc.crt",
                CertificateKeyFile = "$(WEBTOBDIR)/ssl/server.key, $(WEBTOBDIR)/ssl/server-ecc.key"
```
Related issue: IMS-152131

2.8. Added DNS Resolution Settings in REVERSE_PROXY

Added settings to handle delays during DNS resolution when *REVERSE_PROXY.Options is set to "DynamicServerAddress".

*ReverseProxy.ConnectRetryCount sets the DNS resolution retry count when the TCP connection to the server fails, and *ReverseProxy.ConnectTimeout sets the TCP connection timeout for reconnecting.
*ReverseProxy.ConnectRetryCount and ConnectTimeout Settings
```
*REVERSE_PROXY
jsp_url
               ReverseProxyGroupName="jeus_rpg",
               ServerAddress = "192.168.1.1:5580",
               Options = "DynamicServerAddress",
               ConnectRetryCount = 10,
               ConnectTimeout = 5,
```
Related issue: IMS-162352

2.9. Added *NODE.CheckPingTimeoutStatus

Added this setting to override the default status code when *SERVER.RequestLevelPing fails.

You can set *NODE.CheckPingTimeoutStatus to 512 to override the default error code (503) for a failed RequestLevelPing.
*NODE.CheckPingTimeoutStatus Setting
```
*NODE
webtob          WEBTOBDIR = "/home/webtob5",
                SHMKEY = 54800,
                CheckPingTimeoutStatus = 512

*SERVER
MyGroup         SVGNAME = jsvg, MinProc = 30, MaxProc = 30, RequestLevelPing=Y
```
Related issue: IMS-162786

2.10. Added TCPGW Monitoring Command

You can use 'wsadmin > st -tcpgw' to display the current connection information in addition to 'wsadmin > ci'.

wsadmin > st -tcpgw Command

$>wsadmin
--- Welcome to WebtoB Admin (Type "quit" to leave) ---

$$10 tmaxsoft (wsadm) [2018-07-20T20:43:29]: st -tcpgw s*

-------------------------------------------------------------------------
 hth (tcpgwi)tcpgwname count avg cons remote_ipaddr:port
-------------------------------------------------------------------------
   0 ( 1/ 0) sw_jeus2 0 0.0000 0 192.168.1.14:18088
   0 ( 1/ 1) sw_jeus2 0 0.0000 0 192.168.1.14:28088
   0 ( 1/ 2) sw_jeus2 0 0.0000 0 192.168.1.14:38088
   0 ( 1/ 3) sw_jeus2 0 0.0000 0 192.168.1.14:48088
   0 ( 1/ 4) sw_jeus2 0 0.0000 0 192.168.1.14:58088

Related issue: IMS-167446

3. Changed Functions

3.1. WBSSL Version Upgrade

WBSSL version has been changed from WBSSL 2.3.0_B0 to WBSSL 2.3.1_B2.
```
$>wbssl version
WBSSL 2.3.1 B2 30 Mar 2018
```

3.2. Changed Reverse Proxy Option Processing Method

Changed how WebtoB responds to a request for an unsupported method in *NODE(*VHOST).Method when using reverse proxy.
Related issue: IMS-120109

3.3. Responding with ERRORDOCUMENT URL for a 405 Response

When the same method request comes in after the previous method request is redirected to 307, the second request is now processed as a GET request. If the request URL is same as the ERRORDOCUMENT URL, the ERRORDOCUMENT URL is returned instead of 405.
Related issue: IMS-120109

3.4. Changed the Maximum Length of *Node.NodeName from 32 to 128

The length limit is increased to 128 to accommodate using a hostname on cloud VM as the node name.
Related issue: IMS-124789

3.5. Deleting Stat Data After a Cloud Scale In

When using *DOMAIN.CloudDasAddress and all connections between an MS and WebtoB are closed (cloud scaled in), its jengineid(no) is removed from the stats returned by 'wsadmin> st -j'.
Related issue: IMS-124997

3.6. Changed the Maximum Length of JENGINEID from 64 to 128

The length limit is increased to 128 to accommodate encrypted server names.
Related issue: IMS-132832

3.7. Changed the Maximum Length of REVERSE_PROXY.ServerAddress from 48 to 96

The length limit is increased to 96 to prevent wscfl failure when a long ELB domain name is set to *REVERSEPROXY.ServerAddress.
Related issue: IMS-137224

3.8. Changed Default Password Encryption Method for wsmkpw

Changed the default encryption method to MD5 with no character length limit when creating a password using wsmkpw. The previous default encryption method, CRYPT, was also changed to only accept up to 8 characters instead of truncating passwords that exceeds the limit.
Related issue: IMS-150340

3.9. Prevent Incrementing aq_count for RequestLevelPing Request

RequestLevelPing requests are no longer counted in aq_count so that the aq_count returned by 'wsadmin > st -s' includes only requests that are actually queued.
Related issue: IMS-156689

3.10. Closing Server Connection for Invalid WJP Header Message Length

To prevent HTH from restarting unexpectedly, WebtoB now closes the server connection when a request header from JEUS contains an invalid message length.
Related issue: IMS-153787

3.11. Gracefully Dropping Invalid Messages from JEUS

WebtoB now gracefully drops invalid messages before closing the connection.
Related issue: IMS-117370

3.12. Changed webtob.pid File Permission to 640

The webtob.pid file created when starting WebtoB is now created with 640 instead of 666 permission when the file is set to IPCPERM 0777.
Related issue: IMS-162827

4. Bug Patches

4.1. Fix for the Failure to Process an SSL Pending Pipelined Request

Fixed the issue of the buffer being freed after a processing a request when the next request is a pending pipelined request from an SSL client connection.

4.2. Fix for JSV Connection Abnormally Terminating During Request Processing

Fixed the issue of pipelined SSL bug where HTH shuts down when JSV connection is abnormally terminated during SSL processing for a JEUS request.

4.3. Fix for Abnormal Termination of JSV Connection After WebtoB Scale Out

Fixed the issue of existing JSV connection being abnormally terminated after a WebtoB scale out in the cloud.
Related issue: IMS-124997

4.4. Fix for a Core Dump or HTH Shutting Down While Using Reverse Proxy

Fixed the issue of the reverse proxy server connection not being cleared when a request error returns a 400 error. This happens because a large message body that is sent via reverse proxy has chunked data that is not properly terminated by CRLF.
Related issue: IMS-129710

4.5. Fix for TCPGW SSL Bug

Fixed the issue of incorrect processing of *SSL setting in *TCPGW.
Related issue: IMS-132851

4.6. Fix for Memory Leak Caused by Response Header Filter

Fixed the issue of a memory leak caused by using the response header filter for a POST request with ISAPI.
Related issue: IMS-133609

4.7. Fix for Chunked Data Processing Bug with Reverse Proxy

Fixed the issue of broken chunk offset when flow control is needed. This happens when processing a large sized chunked message via reverse proxy and the chunks are coming in faster than being sent to the backend server.
Related issue: IMS-136341

4.8. Fix for Excluding HEADERS When Using URLRewrite

Fixed the issue of *HEADERS setting being ignored when redirecting via URLRewrite.
Related issue: IMS-138490

4.9. Fix for Invalid Input Processing Bug for wsdown

Fixed the issue of wsdown being terminated without an error message when an invalid input, such as Y or N, is entered.

4.10. Fix for Bug when Response Header Uses LF(\n) as Newline Character

Fixed the issue so that response header can be processed using CRLF(\r\n) or LF(\n) as a newline character.
Related issue: IMS-139916

4.11. Fix for Server Request to a Downed Server via TCPGW

Fixed the validity checking logic in socket (fd) when connection to *TCPGW.ServerAddress fails or times out.
Related issue: IMS-138818

4.12. Fix for Compiling Invalid Environment File

Fixed the issue of wscfl normally compiling an environment file with an invalid input, such as *NODE.Port = 0.
Related issue: IMS-140277

4.13. Fix for Reverse Proxy Failure when No Default JSV Server is Set

Fixed the issue of reverse proxy failing and responding with 503 error when a request is sent to a vhost with no default JSV server set.
Related issue: IMS-140087

4.14. Fix for Unwanted Output of FD Size

Fixed the issue of hth -v and wsboot -v displaying the FD size as 8192.
Related issue: IMS-153001

4.15. Fix for Access Log FD Not Closing when AccessLogThread is Enabled

Fixed the issue of access log FD not closing when there are no requests for an HTH and *HTH_THREAD.AccessLogThread is set to Y.
Related issue: IMS-143449

4.16. Fix for 0 Code Recorded in Access Log when Parsing LF(\n) from Response Header

Fixed the issue of an error that occurs when parsing the newline character, LF(\n), from the response header. This causes the server to fail to parse the response code from the response status line and to record a 0 in the access log.
Related issue: IMS-143445

4.17. Fix for Connection Closing During POST Request Redirection

Fixed the issue so that the filter connection is maintained after a 302 redirect to process a POST request body. WebtoB now receives and dumps the request body, and closes only the client connection while maintaining the filter connection.
Related issue: IMS-143843

4.18. Fix for WebSocket.send() Bug

Fixed the issue of websocket flag being reset after WebSocket upgrade via reverse proxy in WebtoB 5.
Related issue: IMS-148638

4.19. Fix for HTH Shutting Down when Using RequestLevelPing

Fixed the issue of HTH shutting down unexpectedly after reconnecting to JEUS to process the previous request when using RequestLevelPing. This happens when WebtoB closes the client and JSV connections because no pong is received for a ping request.
Related issue: IMS-148767

4.20. Fix for SSL Initialize Failure when SSL is Set Only in *TCPGW

Fixed the issue of failing to initialize SSL when it is only set in *TCPGW. After client completes SSL handshake, it now connects to the backend server.

4.21. Fix for Bug that Occurs when Using URLRewrite and ReverseProxy

Fixed the issue of 504 response code that occurs when using URLRewrite and ReverseProxy together due to a buffer management bug in rewriting the HTTP header.
Related issue: IMS-150379

4.22. Fix for Using StdEnvVars as *SSL.Options with ReverseProxyGroup

Fixed the issue of stdEnvVars setting not working when using ReverseProxyGroup. This is a side effect from the fix for reverse proxy failure when no default JSV server is set (4.13).
Related issue: IMS-150532

4.23. Fix for Parsing '$' Incorrectly in Option Value

Fixed the issue of parsing '$' only as the start of an environment variable so that '$' can be parsed correctly when it is used within RegExp or SetHostHeader value.
Related issue: IMS-152421

4.24. Fix for HTH Shutting Down During JSV Compression

Fixed the issue of HTH shutting down unexpectedly while executing JSV compression.
Related issue: IMS-153164

4.25. Fix for FILTERS Process Shutting Down During Startup

Fixed the issue of FILTERS process shutting down unexpectedly during WebtoB startup. This only happens in WebtoB 5.
Related issue: IMS-153986

4.26. Fix for HTH Shutting Down While Processing Indexname Page

Fixed the issue of core dump caused by an undeleted buffer pointer from the previous request remaining. This happens when an SSL request is processed by appending a slash (/) after setting Indexname="index.php", and then sending the same request via NonSSL.
Related issue: IMS-152091

4.27. Fix for 'wsadmin > restat' Not Resetting HTMLS Count

Fixed the issue of 'wsadmin > restat -a' not resetting the htmls count, which is displayed via 'wsadmin > st -s', and the thread information.
Related issue: IMS-156670

4.28. Fix for HTH Shutting Down When JEUS Attempts to Reconnect Continuously

Fixed the issue of a bug caused by creating too many connection objects than allocated in shared memory leading to an index overflow.
Related issue: IMS-155395

4.29. Fix for patchinfo Error

Added fixes from WebtoB 4.1.9.1 that are missing in WebtoB 5, and corrected spelling errors in patchinfo.
Related issue: IMS-160414

4.30. Fix for Bug that Occurs when Using URLRewrite for a Vhost set in REVERSE_PROXY

Fixed the issue so that the reverse proxy setting is used instead of URI matching when URLRewirte enabled vhost is set in *ReverseProxy. This is a side effect from the fix for IMS-140087.
Related issue: IMS-147271

4.31. Fix for EXPIRES Setting Not Working w/o SVRGROUP Setting

Fixed the issue so that no HTML server group needs to be set in *SVRGROUP to use the *EXPIRES setting.
Related issue: IMS-164680