ipc.q

Inter-Process Communication Library

This library provides connection management and tracking when communicating with other kdb+ processes.

.ipc.connect / .ipc.connectWait / .ipc.connectWithTimeout

These functions accept a 'host/port' type target and attempts to connect with varying connection timeouts:

• .ipc.connect: Will use a connection timeout of .ipc.cfg.defaultConnectTimeout (5 seconds by default)
• .ipc.connectWait: Will wait indefinitely
• .ipc.connectWithTimeout: Will wait the specified number of milliseconds

.ipc.connectWithHp

This function accepts a separate 'host' and 'port' argument and builds the 'host/port' type before calling .ipc.connect.

.ipc.oneShot / .ipc.oneShotWait / .ipc.oneShotWithTimeout

These function accepts a 'host/port' type target and sends the specified query via a one-shot connection:

• .ipc.oneShot: Will use a connection timeout of .ipc.cfg.defaultConnectTimeout (5 seconds by default)
• .ipc.oneShotWait: Will wait indefinitely to connect
• .ipc.oneShotWithTimeout: Will wait the specified number of milliseconds to connect

.ipc.getHandlesFor

This function looks for any existing connections for the specified host/port via hostPortHash (after normalisation via .type.ensureHostPortSymbol) in .ipc.outbound and returns the list of handles that match.

Example

q) .ipc.connect 12345
2023.11.08 09:34:56.770 INFO  pid-558 jas 0 Attempting to connect to :unix://12345 (timeout 5000 ms)
2023.11.08 09:34:56.771 INFO  pid-558 jas 0 Successfully connected to :unix://12345 on handle 4
4i

q) .ipc.getHandlesFor 12345
,4i

Outbound Connection Tracking

By using the included .ipc.connect* and .ipc.disconnect functions, all successful connections will be logged in the .ipc.outbound table.

Example

q) .ipc.connect 12345
2023.11.08 09:34:56.770 INFO  pid-558 jas 0 Attempting to connect to :unix://12345 (timeout 5000 ms)
2023.11.08 09:34:56.771 INFO  pid-558 jas 0 Successfully connected to :unix://12345 on handle 4
4i

q) .ipc.outbound
handle| targetHostPort connectTime                   hostPortHash
------| ---------------------------------------------------------------------------------------
4     | :unix://12345  2023.11.08D09:34:56.771135500 0x33b76acd1fad540fd7bf2ed7e5d562b4c1e96f0f

Note that targetHostPort will not match the hostPort value that is passed into the function as it can be modified by the function itself (by converting localhost connections to UNIX domain sockets and/or obfuscating the password).

hostPortHash is a SHA-1 hash of the parameter, after normalisation via .type.ensureHostPortSymbol that can be used for comparison (it's a hash so the password can be included for comparison, but not visible).

Inbound Connection Tracking

Inbound connection tracking is optional and can be configured by changing the .ipc.cfg.enableInboundConnTracking boolean.

If enabled, the Event Management library will be loaded to allow it to listen to inbound port and websocket open / close events.

All inbound connections will be stored in .ipc.inbound.

Example

Notifying listeners of event [ Event: websocket.open ] [ Args: 496i ]
New inbound websocket connection on handle 496 [ IP Address: 127.0.0.1 ] [ User: unknown ]
Notifying listeners of event [ Event: port.open ] [ Args: 492i ]
New inbound kdb connection on handle 492 [ IP Address: 127.0.0.1 ] [ User: jasra_000 ]

q).ipc.inbound
handle| sourceIp  user      connectTime                   connectType
------| -------------------------------------------------------------
496   | 127.0.0.1 unknown   2017.03.31D10:35:51.783433000 websocket
492   | 127.0.0.1 jasra_000 2017.03.31D10:35:55.109387000 kdb

Notifying listeners of event [ Event: port.close ] [ Args: 296i ]
Inbound connection on handle 296 closed [ IP Address: 127.0.0.1 ] [ User: jasra_000 ]

q) .ipc.inbound
handle| sourceIp user connectTime
------| -------------------------

By default, if a password is specified in the host/port, the password component will be switched to asterisks for logging and for adding into .ipc.outbound.

This behaviour can be changed via .ipc.cfg.logPasswordsDuringConnect.

Unix Domain Sockets

All the .ipc.connect* functions support the Unix Domain Socket format (:unix://*port*) but they do not check if the Operating System supports them before calling hopen.

As of kdb-common-1.4.0-2022.06.29, all 'localhost' connections will be automatically converted from TCP/IP to Unix Domain Sockets on supported Operating Systems. This is configured via .ipc.cfg.forceUnixDomainSocketsForLocalhost.

The following addresses are defined as 'localhost' (in .ipc.localhostAddresses):

• localhost
• 127.0.0.1

The following are evaluated during library initialisation:

• .z.h
• .Q.host .z.a
• .convert.ipOctalToSymbol .z.a
• .convert.ipOctalToSymbol .Q.addr .z.h

Any calls with a target whose hostname matches one of these will be automatically converted to Unix Domain Socket before the hopen call.

Example

q).ipc.connect each (12346; `::12346; `:127.0.0.1:12346; `:localhost:12346)
2022.06.29 17:15:23.489 DEBUG pid-343 jas 0 Host/port translated to Unix Domain Socket [ Original: ::12346 ] [ Now: :unix://12346 ]
2022.06.29 17:15:23.489 INFO  pid-343 jas 0 Attempting to connect to :unix://12346 (timeout 5000 ms)
2022.06.29 17:15:23.489 INFO  pid-343 jas 0 Successfully connected to :unix://12346 on handle 6
2022.06.29 17:15:23.489 DEBUG pid-343 jas 0 Host/port translated to Unix Domain Socket [ Original: ::12346 ] [ Now: :unix://12346 ]
2022.06.29 17:15:23.489 INFO  pid-343 jas 0 Attempting to connect to :unix://12346 (timeout 5000 ms)
2022.06.29 17:15:23.489 INFO  pid-343 jas 0 Successfully connected to :unix://12346 on handle 7
2022.06.29 17:15:23.489 DEBUG pid-343 jas 0 Host/port translated to Unix Domain Socket [ Original: :127.0.0.1:12346 ] [ Now: :unix://12346 ]
2022.06.29 17:15:23.489 INFO  pid-343 jas 0 Attempting to connect to :unix://12346 (timeout 5000 ms)
2022.06.29 17:15:23.490 INFO  pid-343 jas 0 Successfully connected to :unix://12346 on handle 8
2022.06.29 17:15:23.490 DEBUG pid-343 jas 0 Host/port translated to Unix Domain Socket [ Original: :localhost:12346 ] [ Now: :unix://12346 ]
2022.06.29 17:15:23.490 INFO  pid-343 jas 0 Attempting to connect to :unix://12346 (timeout 5000 ms)
2022.06.29 17:15:23.490 INFO  pid-343 jas 0 Successfully connected to :unix://12346 on handle 9
6 7 8 9i