XMLSERVICE Connections¶
Sometimes a real conversation helps …¶
Customer wants to know how to remain stateless, but get the benefit of being able to audit the use of RPG subprocedure invocations via XMLToolkit without toggling back and forth (different QSQSRVR jobs).
Stateless may use multiple QSQSRVR jobs (toggling), this is definition of ‘stateless’ in PHP fastcgi environment (working correct).
Will ToolkitService object using persistent db2 connections and “stateless => true” help single QSQ job?
Toolkit “stateless => true” via db2 transport using db2_connect or db2_pconnect makes no difference, fastcgi implies unpredictable QSQSRVR job will be used (command line slightly more predictable over web).
Why ‘unpredictable’ QSQ?
It’s all web PHP fastcgi ‘random’ worker selection, NOTHING to do with ibm_db2 ‘connection’ persistent or non-persistent, toolkit connection is irrelevant. Apache web site using fastcgi routes “randomly” to any available PHP worker job php-cgi, therefore ‘random’ php-cgi worker using db2_(p)connect<>QSQSRVR will also appear ‘random’. To wit, you end up different QSQSRVR jobs (toggle back and forth). Technically, fastcgi protocol, PHP workers poll a single fastcgi socket waiting to take some work, ZS on i at /www/zendsvr6/logs/fcgi-njjjineh.sock (sock name random). As each php worker strips work off fastcgi socket (1st come == 1st do), the worker is busy communicating on ‘private web socket’ running script until finished (no longer waiting). Combination of natural web worker selection (browser +/- KeepAlive), and fastcgi socket poll for work, results in ‘random’ appearance QSQ job toolkit usage, DB2 connection just came along for ride in back seat of the 1950 PHP fastcgi roadster.
Help me be one-and-only-one job?
The only way to assure private connection back to same XTOOLKIT job is ‘internalKey’=>’/tmp/XTOOLKIT_job_1’ (‘ipc’=/tmp/XTOOLKIT_job_1’).
XMLSERVICE connections discussed this page¶
- Stateless – clean running come/go, “full open/close,” start/stop connection to be used by any requester. Must set LIBL during each request
- State full – State retained between requests: LIBL, transactions, etc., “private” connection used by one requester/task for a long period of time (like 5250)
- State full – hybrid “private/persistent” connection shared by many requesters, but keep open PGM, files, etc.
- State full – reservation hybrid “private/persistent” connection exclusively held for a period each requesters, but returned back to pool for re-use (rare use)
The jobs involved in “connections”:
I. Stateless
(job 1) (job 2) (job 3)
Apache FastCGI DB2 (stateless)
------- --------------- ---------------------
browser/client ->thread-->socket->php-cgi
--->$ctl="*here"; -->QSQSRVR+XMLSERVICE
shut down after
each request
("stateless")
II. State full
(job 2) (job 3) (job 4)
FastCGI QSQ (proxy) XMLSERVICE (state full)
--------------- --------------------- ------------------------
-->socket->php-cgi
--->$ctl="*sbmjob";
--->$ipc="/tmp/sally"; -->QSQSRVR+XMLSERVICE -->XMLSERVICE
alive until stopped
("state full")
Jobs originates::
(job 1) Apache picks any thread (1st level routing)
(job 2) FastCGI all "non-busy" worker php-cgi wait on unix domain socket /www/zend2/logs/fcgi-hmjadgek.sock (2nd level routing)
(job 3) php-cgi - database connections odbc, ibm_db2, pdo_ibm (3rd level routing)
db2_pconnect() attach to pooled/persistent QSQ (matching profile) but leaves connection open on exit
db2_connect() acquires a unused pre-start QSQ (or starts one) then attaches to QSQ (profile), and returns to unused pool on exit
(job 3) XMLSERVICE -- Stateless -- run inside QSQ job and clean-up after each request (3rd level routing)
$ctl = "*here";
(job 4) XMLSERVICE -- State full -- run in separate job that any QSQ job can call using IPC (4th level routing)
$ctl = "*sbmjob";
$ipc = "/tmp/sally";
Drivers involved in conection:
400 server start-up Common usage Big picture Comments
STRTCPSVR SERVER(*HTTP)
port 80 or 10088 or … (REST/HTTP interface)
1-tier - XMLCGI CLIENT <==> IBM HTTP Server <==> XMLCGI (CLI server mode) <==> QSQxxxx <==> IBM i Resources XMLCGI + CLI
PASE library (no start needed)
no port
1-tier - PHP ibm_db2+pdo_ibm+odbc (Native PASE CLI libdb400.a driver) CLIENT (PASE libdb400.a driver) <==> QSQxxx <==> IBM i Resources Native PASE CLI libdb400.a driver (IBM Rochester)
STRHOSTSVR SERVER(*DATABASE)
port 8471 (database)
2-tier - PHP odbc interface (IBM i Client Access odbc driver interface) CLIENT (Client Access drivers) <==> QZDAxxxx <==> IBM i Resources Client Access odbc-based drivers (IBM Rochester)
STRTCPSVR SERVER(*DDM)
port 446 (DDM/DRDA)
2-tier - PHP ibm_db2+pdo_ibm (DB2 Connect driver interface) CLIENT (DB2 Connect drivers) <==> QRWxxxx <==> IBM i Resources DB2 CLI DRDA-based DB2 Connect drivers (IBM Toronto)
1) Stateless – no LIBL, come/go¶
These connections are traditional web requests “full open/close” clean running start/stop connection to be used by any requester.
- Stateless
$ctl = "*here";
(1) (2) (3)
Apache FastCGI DB2 (server mode)
------- --------------- ---------------------
browser/client -->thread--socket->php-cgi--->QSQSRVR(profile fred)
XMLSERVICE (fred) <--shut down after each request
--->QSQSRVR(profile sally)
XMLSERVICE (sally) <--shut down after each request
--->QSQSRVR(profile john)
XMLSERVICE (john) <--shut down after each request
Example new Toolkit (stateless):
if ($i5persistentconnect) $conn = db2_pconnect($database,$user,$password);
else $conn = db2_connect($database,$user,$password);
try { $ToolkitServiceObj = ToolkitService::getInstance($conn); }
catch (Exception $e) { die($e->getMessage()); }
$ToolkitServiceObj->CLCommand("CHGLIBL LIBL(FREDFLIN WILMAFLIN) CURLIB(FREDFLIN)");
Stateless: If you choose $ctl=’*here’, you will run in the calling process DB2 connection (QSQSRVR job). When XMLSERVICE completes your XML script it will shut down to nothing, considered stateless and holds zero state on return.
- In general you will run slower in stateless mode (CW default / Toolkit default), because XMLSERVICE has to keep starting things over and over and over again, but perhaps not an issue if you have CPU to burn.*
- The is no semaphore locking or shared memory ipc when running as stateless (*here), because only one sally client/server is a pair, but of course there may be many sally client/server pairs on the same machine.*
- There is no “memory” of the LIBL in stateless, so it must be set EVERY time before use.*
2) State full – LIBL, transactions, etc.¶
These connections are traditional 5250-like “private” connection used by one requester/task for a long period of time.
State full (most RPG programs)¶
$ctl = "*sbmjob";
$ipc = "/tmp/sally";
-- or --
$ipc = "/tmp/john";
(1) (2) (3) (4)
Apache FastCGI DB2 (server mode) XMLSERVICE
------- --------------- --------------------- ----------
-->thread--socket->php-cgi--->QSQSRVR(profile sally)---.->XMLSERVICE (sally) <--alive until stopped (or idle timemout)
--->QSQSRVR(profile john)--. |
-->thread--socket->php-cgi--->QSQSRVR(profile fred) | |
--->QSQSRVR(profile sally)---.
--->QSQSRVR(profile fred) |
--->QSQSRVR(profile john)--.--->XMLSERVICE (john) <--alive until stopped (or idle timemout)
Example new Toolkit (state full):
$internalKey = '/tmp/packers';
if ($i5persistentconnect) $conn = db2_pconnect($database,$user,$password);
else $conn = db2_connect($database,$user,$password);
try { $ToolkitServiceObj = ToolkitService::getInstance($conn); }
catch (Exception $e) { die($e->getMessage()); }
$ToolkitServiceObj->setToolkitServiceParams(array(
'InternalKey'=>$internalKey, // *** RIGHT HERE internalKey/IPC
// *** run state full
// use SBMJOB command run in new job
// PHP can call again, again, again
// with /tmp/packers and get
// same job every time
// same library list (*LIBL)
// same PGMs with open files, etc.
// exactly like 5250 sign-on screen
'plug'=>"iPLUG32K")); // max size data i/o (iPLUG4K,32K,65K.512K,1M,5M,10M,15M)
// state full - MUST do this ONCE ONLY after start/sbmjob of XMLSERVICE job
// then forget about it (unless you choose to change libl)
$ToolkitServiceObj->CLCommand("CHGLIBL LIBL(FREDFLIN WILMAFLIN) CURLIB(FREDFLIN)");
/* Do not use the disconnect() function for "state full" connection */
/* NEVER EVER USE THIS ... $ToolkitServiceObj->disconnect(); */
/* Why? *immed kill of job, not nice or sync, just kill */
/* Use idle timeout for "state full" / "private" connections */
State full: If you choose $ctl="\*sbmjob" + $ipc="/tmp/packers", you will run in a separate job past
the calling DB2 connection (child job of QSQSRVR). This $ctl/$ipc combination will allow you to return to the
same XMLSERVICE job from any connection to the machine, therefore considered “state full” and any called program
can keep open files, transactions, etc. (just like a real RPG 5250 program does mate).
- $ipc=’/tmp/anything’ can be any unique/accessible directory you want to route you back to same XMLSERVICE job (*sbmjob), but usually anchored in /tmp directory because xmlservice will try to create it if missing.
- Technically $ipc=”/tmp/packers” is a unique IFS machine location in posix function ftok(‘/tmp/packers’) which presents a unique numeric key representing /tmp/packers that is used for XMLSERVICE shared memory and semaphores creation/attach (XMLSERVICE uses shared memory/semaphores for communication).
- Shared memory + semaphore locking is only required for state full connections ($ctl=”*sbmjob” + $ipc=”/tmp/packers”), where each sally XMLSERVICE semaphore “front door lock” will allow only one sally client to chat with a XMLSERVICE job, the other sally requesters will wait until they are invited to chat (just like the dentist office).
- Security is managed through IFS shared memory / semaphores access control just like any other IFS file, so once profile sally owns an active XMLSERVICE ctl+ipc then no other profile can attach to the active XMLSERVICE job … well … except for high authority profiles like *SECOFR (of course).
- With version 1.6.6 state full XMLSERVICE connections are ended via configurable idle timeout $ctl .= ” *idle(3000)”, you may keep the jobs alive forever using $ctl .= ” *idle(0)” to match the original version behavior. There are other options for client wait $clt .= ” *wait(30)” and waiting for called program to return $ctl .= ” *call(200)” and various actions that can be taken for each wait/timer (busy,kill,etc.).
- In this example we have been using one sally client/server $ipc=”/tmp/packers”, you can of course have many different sally client/server ($ipc=”/tmp/packers”, $ipc=”/tmp/vikings”, $ipc=”/tmp/bears”, etc.) and each of these sally ipcs may have many clients chatting with each sally server ipc … sort of a sally work load balancing dream situation where we can clone a new sally ipc server for each task at hand.
- You only need to set the LIBL once in state full (unless you want to change LIBL for some reason).
Toolkit State full (with IPC) - avoid start up/shut down XMLSERVICE (NOT toolkit default)
- avoid using toolkit disconnect ($ctl=”*immed”) to leave XMLSERVICE up and running (will timeout shut down if idle for 1/2 hour)
- choose a the minimum plug size need for the task to avoid send/receive extra blanks
- TURN DEBUG and LOGS off in toolkit to avoid IFS file write (takes forever in computer timings)
- db2_pconnect() - persistent or “shared” connection with toolkit avoids acquire/release QSQ jobs (NOT toolkit default)
XMLSERVICE adopt authority issues¶
When using ctl+ipc “state full” jobs is a generally bad idea to “adopt authority” as originating profile sally will lose all access … and … in fact ipc may become unreachable causing a orphan XMLSERVICE (client is still sally, ipc is still sally’s, but adopt xmlservice server becomes fred).
Two choices:
- If you MUST “adopt authority” do it in a stateless job (*here), where full connection processing may undo “left over switch profile” potential damage on the way out of XMLSERVICE script. This option should always work.
- Be very careful to return back to sally profile EACH TIME leaving xmlservice sending data back to the waiting sally client … sort of good manners talk to sally client as sally server (adopted fred can speak/do only when asked, then go away)
Note: We are thinking about forcing “switch back to originating profile on the way back” within XMLSERVICE code, but have not yet understood what that means to PHP wrappers like CW, so the mission is at the moment in your called program and/or PHP wrapper/user code.
3) State full – hybrid “private/persistent” connection¶
These connections are hybrid “private/persistent” connection shared by many requesters, but keep open PGM, files, etc.
Worried about too many IPC’s/XMLSERVICE jobs??
The following gives you a hybrid “private/persistent” connection
- most all the benefits for called RPG (state full, open files, etc.)
- but only $maxpool XMLSERVICE jobs
Try this simple technique for pooled IPC’s/XMLSERVICE jobs $internalKey = '/tmp/packers'.rand(1,$maxpool)
- IF your application set can tolerate multi-client shared access to a pool of persistent/private/semi-stateless connections the random technique should work well.
- However, if you need your client make a exclusive reservation see the next topic
State full – hybrid “private/persistent” connection
$maxpool = 3;
// -- PHP raw ---
$ctl = "*sbmjob";
$ipc = "/tmp/sally".rand(1,$maxpool);
// -- or PHP toolkit --
$internalKey = '/tmp/sally'.rand(1,$maxpool)
(1) (2) (3) (4)
Apache FastCGI DB2 (server mode) XMLSERVICE
------- --------------- --------------------- ----------
-->thread--socket->php-cgi--->QSQSRVR(profile sally)--.
-->thread--socket->php-cgi--->QSQSRVR(profile sally)--|
: |
-->thread--socket->php-cgi--->QSQSRVR(profile sally)--|->XMLSERVICE (/tmp/sally1) <--alive until stopped (or idle timemout)
-->thread--socket->php-cgi--->QSQSRVR(profile sally)--|->XMLSERVICE (/tmp/sally2) <--alive until stopped (or idle timemout)
-->thread--socket->php-cgi--->QSQSRVR(profile sally)--|->XMLSERVICE (/tmp/sally3) <--alive until stopped (or idle timemout)
: |
-->thread--socket->php-cgi--->QSQSRVR(profile sally)--|
-->thread--socket->php-cgi--->QSQSRVR(profile sally)--.
3 XMSLERVICE jobs handle work for all sally clients using the site
Example new Toolkit (hybrid “private/persistent” connection):
$maxpool = 40; // 40 jobs good enough to handle my machine needs
if ($i5persistentconnect) $conn = db2_pconnect($database,$user,$password);
else $conn = db2_connect($database,$user,$password);
try { $ToolkitServiceObj = ToolkitService::getInstance($conn); }
catch (Exception $e) { die($e->getMessage()); }
$internalKey = '/tmp/packers'.rand(1,$maxpool);
$ToolkitServiceObj->setToolkitServiceParams(array(
'InternalKey'=>$internalKey, // *** RIGHT HERE internalKey/IPC $maxpool jobs for service
'plug'=>"iPLUG32K")); // max size data i/o (iPLUG4K,32K,65K.512K,1M,5M,10M,15M)
/* Do not use the disconnect() function for "state full" connection */
/* NEVER EVER USE THIS ... $ToolkitServiceObj->disconnect(); */
/* Why? *immed kill of job, not nice or sync, just kill */
/* Use idle timeout for "state full" / "private" connections */
So simple, why does it work???
- Works much same as Apache FastCGI PHP jobs (even using random), because $maxpool “child XMLSERVICE workers” can be increased to match machine workload (tinker-trial-error) … just like Apache threads … just like PHP children … all the same
- Most web requests are sub-second, so even on routing collision by random it is a short wait.
Could i dedicate different pools to different tasks ???
- Yes, a bag full of really low effort work
(/tmp/packers1-40, /tmp/vikings1-40).
- Yes, a bag full of really low effort work
Could i dedicate different user ids to different pools as well as tasks???
- Yes, a bag full of really low effort work
(/tmp/packers1-40, /tmp/vikings1-40).
- Yes, a bag full of really low effort work
Can i idle timeout unused XMLSERVICE jobs ??? + Yes of course, toolkit.ini setting or specify manually. + NEVER EVER USE THIS …
$ToolkitServiceObj->disconnect();Should i use persistent connections??
- db2_pconnect – Yes of course, it will save the time “attaching” a QSQSRVR job
- db2_connect – However, you can use same technique with full open/close (yes it does work, try it)
Can i prestart jobs?
- Yes, but they will start on web demand which i think is much better (just like Apache)
SBMJOB CMD(CALL PGM(ZENDSVR/XMLSERVICE) PARM('/tmp/packers1')) JOBD(ZENDSVR/ZSVR_JOBD) USER(PACKERS) SBMJOB CMD(CALL PGM(ZENDSVR/XMLSERVICE) PARM('/tmp/vikings1')) JOBD(ZENDSVR/ZSVR_JOBD) USER(VIKINGS)
4) State full – reservation hybrid “private/persistent” connection¶
These connections are hybrid “private/persistent” connection exclusively held for a period of time by each requesters, but returned back to pool for re-use.
If your client needs to start/use/stop a reservation hybrid “private/persistent” connection, use the appropriate keyword in your XML sent to XMLSERVICE to gain exclusive rights to the hybrid “private/persistent” connection.
<start>unique-user-key</start> – acquire exclusive IPC if available
<use>unique-user-key</use> – must appear XML every request job held forever until see <stop>
<stop>unique-user-key</stop> – release IPC for any other use
Errors:
<use>no-match-user-key</use> – non-matching key results in error almost instantly (no wait)
busy response (1301060): <error> <errnoxml>1301060</errnoxml> <xmlerrmsg>IPC owner busy</xmlerrmsg> </error>
thoughtful setting server idle timeout can control unwanted reservation hangs due to careless users or errors ** $ctl .= ” *idle(60)” **
hybrid “private/persistent” connection with reservation
$maxpool = 3;
// -- PHP raw ---
$ctl = "*sbmjob";
$ipc = "/tmp/sally".rand(1,$maxpool);
// -- or PHP toolkit (not available yet -- Alan) --
(1) (2) (3) (4)
Apache FastCGI DB2 (server mode) XMLSERVICE
------- --------------- --------------------- ----------
-->thread--socket->php-cgi--->QSQSRVR(profile sally)--.
-->thread--socket->php-cgi--->QSQSRVR(profile sally)--|
: |
-->thread--socket->php-cgi--->QSQSRVR(profile sally)--|->XMLSERVICE (/tmp/sally1) <--alive until stopped (or idle timemout)
<start>unique-user-key</start> <--exclusive reservation until stopped
<use>unique-user-key</use>
<stop>unique-user-key</stop>
-->thread--socket->php-cgi--->QSQSRVR(profile sally)--|->XMLSERVICE (/tmp/sally2) <--alive until stopped (or idle timemout)
<start>unique-user-key</start> <--exclusive reservation until stopped
<use>unique-user-key</use>
<stop>unique-user-key</stop>
-->thread--socket->php-cgi--->QSQSRVR(profile sally)--|->XMLSERVICE (/tmp/sally3) <--alive until stopped (or idle timemout)
<start>unique-user-key</start> <--exclusive reservation until stopped
<use>unique-user-key</use>
<stop>unique-user-key</stop>
: |
-->thread--socket->php-cgi--->QSQSRVR(profile sally)--|
-->thread--socket->php-cgi--->QSQSRVR(profile sally)--.
3 XMSLERVICE jobs handle work for all sally clients using the site However, reservation locks exclusive use until reservation is stopped.
Example new Toolkit (hybrid “private/persistent” connection with reservation):
--- unfortunately reservation is not available in PHP wrapper yet (Alan) ---
--- raw xml pseudo code version of what happens follows start/use/stop ---
-- no time out --
$ctl .= " *idle(0)"
-- request 1 --
<?xml version="1.0"?>
<script>
<start>unique-user-key</start>
</script>
-- request 2 (two minutes later) --
<?xml version="1.0"?>
<script>
<use>unique-user-key</use>
<cmd exec='rexx'>RTVJOBA USRLIBL(?)</cmd>
</script>
-- request 3 (1/2 hour later) --
<?xml version="1.0"?>
<script>
<use>unique-user-key</use>
<pgm name='ZZCALL'>
<parm>
<data type='1A'>a</data>
</parm>
<return>
<data type='10i0'>0</data>
</return>
</pgm>
</script>
-- request n (2 hours later) --
<?xml version="1.0"?>
<script>
<stop>unique-user-key</stop>
</script>