A Practical Handbook for System Administrators

Daniel Horsley


Contents

Preface

Part I - Getting Oriented

  1. What Za is (and is not)
  2. Running Za
  3. The REPL

Part II - Language Fundamentals

  1. Lexical structure
  2. Literals and constants
  3. Variables and assignment
  4. Typed declarations with var

Part III - Data Types and Structures

  1. Scalar types
  2. Arrays and multi-dimensional data
  3. Maps
  4. Maps as sets
  5. Structs and anonymous structs
  6. Struct-associated functions and self
  7. Enums

Part IV - Expressions and Control Flow

  1. Expressions and operator precedence
  2. Operators Overview
  3. Conditionals
  4. Loops
  5. Case statements

Part V - Functional Data Processing

  1. Expression strings, #, and $idx
  2. Map and filter operators (->, ?>)
  3. Searching arrays (find, where)

Part VI - Functions, Modules, and Composition

  1. Functions (def … end)
  2. Modules and namespaces

Part VII - Errors, Debugging, and Safety

  1. Error handling philosophy
  2. Exceptions (try … catch … then … endtry)
  3. Enhanced error handling (trap, error_*)
  4. Debugger (how to use it)
  5. Profiler (how to interpret output)
  6. Security controls (permit)

Part VIII - Concurrency

  1. Async execution (async, await)

Part IX - Output and Presentation

  1. Program output (print, println)
  2. Inspection (pp) and tables (table)
  3. Array display controls (array_format, array_colours)
  4. ANSI colour/style macros

Part X - Standard Library Overview

  1. Library categories and discovery (help, func)
  2. Category tour (representative idioms)
  3. Category samples

Part XI - Sysadmin Cookbook

  1. CLI data ingestion
  2. Disk and filesystem checks
  3. Process and service inspection
  4. Network diagnostics
  5. Parallel host probing
  6. Drift detection and set-based reasoning

Part XII - Logging

  1. Logging Overview
  2. Logging Configuration
  3. Logging Architecture

Part XIII - Testing

  1. Testing Overview
  2. Test Blocks
  3. Test Behaviours

Appendices

A. Appendix A - Operator Reference

B. Appendix B - Keywords Summary

C. Appendix C - Built-in Constants

D. Appendix D - Standard Library Categories

E. Appendix E - Worked Example Script

F. Appendix F - C Library Imports (FFI) (v1.2.2+)

Preface

About this book

Za is an interpreted language aimed primarily at:

Za is stable enough to run in production, but it is not designed as a framework language or a long-running service runtime. Its strengths are readability and portability.

Who this book is for

The intended readership is Linux system administrators, from novice to expert. Existing programmers should also be able to use this book as a reliable syntax and behaviour reference.

Version coverage

This text is written for Za 1.2.2.


Part I — Getting Oriented

1. What Za is (and is not)

Za is a scripting language for people who need to maintain and monitor systems. It can be used for glue scripts, test rigs, ad-hoc reporting, monitoring and many small tasks that are in the scope of system administrators, SREs, developers and others who are regularly expected to probe issues and generate state information.

It prioritises:

Za does not attempt to replace a general-purpose language for large services and should be used cautiously in production environments.

2. Running Za

Za runs scripts and also provides interactive tooling.

Za is available for Linux-, BSD- and Windows-variants. The same features are available across all platforms where allowed by the OS.

The interpreter also has a REPL with interactive help features.

3. The REPL

The REPL is a workflow tool designed for prototyping, data exploration and system inspection.

You can use it as a shell, if desired, but this is not the intent of this interactive mode.

3.1 REPL Overview

The REPL provides several key capabilities that make it particularly effective for system administration tasks:

3.2 Getting Started

Starting the REPL is simple:

$ za

Upon startup, you’ll see the default prompt:

>>

Customizing Your Environment

A useful REPL feature is the startup script. Create a file at ~/.zarc to automatically configure your session. Here’s a simplified one:


# Set a basic prompt
prompt=">> "

# Define helpful macros
macro +ll `ls -la`
macro +df `df -h`

# Enable command fallback - this allows execution of shell commands
_=permit("cmdfallback",true)

The full example startup script in the Za repository demonstrates advanced features including:

Display welcome message

println "Za REPL ready. Type 'help' for assistance."

Now when you start za, your environment is automatically configured with your preferred prompt, modules, and helper macros.

3.3 Discovery and Help

Za provides built-in help and discovery mechanisms. Use the command-line help to see available options:

$ za -h

For discovering available functions and modules, examine the standard library source files and examples in the eg/ directory. The language documentation and examples provide comprehensive coverage of available capabilities.

3.4 Interactive Features

Command History

The REPL maintains a persistent command history across sessions:

# History is automatically saved to ~/.za_history
# Navigate with up/down arrows
# Search history with ctrl-r (reverse search)
>> ctrl-r
(search): `ls`:
# Type to search, use arrows to navigate results

Line Editing

Interactive mode supports UTF-8 characters. Navigation keystrokes should be familiar from similar systems:

Standard arrow navigation works as expected

ctrl-a  # Beginning of line
ctrl-e  # End of line
ctrl-u  # Delete to beginning
ctrl-k  # Delete to end
ctrl-c  # interrupt (interrupt session)
ctrl-d  # end-of-input (end session)
ctrl-z  # suspend REPL to background

3.5 REPL Macros

Macros provide powerful command shortcuts, particularly useful for repetitive system administration tasks.

Basic Macro Definition

# Simple text replacement
macro +ps `ps aux | grep -v grep`

# Use in commands
>> #ps
# Expands to: ps aux | grep -v grep

Parameterized Macros

# Macro with parameters
macro +ls(path) `ls -la {path}`

# Usage
>> #ls("/etc")
>> #ls("~")  # Home directory expansion works

Varargs and Advanced Features

# Variable arguments with ...
macro +addall(base, ...) `$base + $1 + $2 + $3`

# Nested macro expansion
macro +complex `#addall(10, #ls("."), "extra")`

# Debug macro expansion - Shows expanded code before execution
>> #macro_name!

Macro Management

# List all macros
macro

# Remove specific macro
macro -ls

# Remove all macros
macro -

# Verbose operations (when logging enabled, shows confirmation messages)
macro ! +test `echo "debug"`
# Output: Macro 'test' defined

3.6 REPL Workflow Examples

The REPL is ideal for prototyping and interactive data exploration. Test your commands interactively before incorporating them into scripts.

# Filter system data for interesting entries
>> high_usage = disk_usage() ?> `#.usage_percent > 50`

# Examine results
>> println high_usage.pp
[
  {
    "available": 65881,
    "mounted_path": "/sys/firmware/efi/efivars",
    "path": "efivarfs",
    "size": 151464,
    "usage_percent": 56.50385570168489,
    "used": 85583
  }
]

# Create a quick report
>> foreach item in high_usage
>>     println "ALERT: {=item.path} at {=item.mounted_path} is {=item.usage_percent}"
>> endfor
ALERT: /dev/sda1 at /boot is 92%
ALERT: /dev/sda2 at / is 87%

System Inspection Workflow

# Quick system overview
>> println sys_resources().pp
{
  "CPUCount": 20,
  "LoadAverage": [
    0.59,
    0.32,
    0.28
  ],
  "MemoryTotal": 32889544704,
  "MemoryUsed": 3189137408,
  "MemoryFree": 25005395968,
  "MemoryCached": 4691128320,
  "SwapTotal": 4294963200,
  "SwapUsed": 0,
  "SwapFree": 4294963200,
  "Uptime": 12977.04
}

# Network interface check
>> println pp(net_devices() ?> `#.name ~ "wlan"`)
[
  {
    "device_type": "1",
    "duplex": "",
    "enabled": true,
    "gateway": "192.168.1.1",
    "ip_addresses": [
      "192.168.1.16",
      "fe80::869e:56ff:fe34:d39d"
    ],
    "link_speed": "",
    "mac_address": "84:9e:56:34:d3:9d",
    "name": "wlan0",
    "operstate": "up"
  }
]

These patterns demonstrate how the REPL enables iterative development with immediate feedback, making it ideal for the exploratory nature of system administration work.


Part II — Language Fundamentals

4. Lexical structure

5. Literals and constants

5.1 Strings

Za supports double-quoted strings and backtick strings. Both may span multiple source lines and may contain interpolation.

Backticks are especially useful when you want to avoid escaping double quotes inside expression strings:

expr = `#.replace("%","").as_int > 80`

You can also escape characters such as quotes/backticks within string literals. Similarly, control codes such as line feed, tab and others may be expressed using escape sequences, in the usual manner for C-like sprintf formatting.

Za also supports interpolation forms such as {...} and {=...} (interpolation can be enabled/disabled via policy controls).

5.2 Numeric literals

Numeric literal type is determined by suffix and decimal point:

Integer base prefixes are supported:

5.3 Built-in constants

Za provides only:

6. Variables and assignment

6.1 Implicit creation

Most variables are created by assignment:

x = 10
name = "db1"

With implicit creation, there is no fixed type for the variable, but using it in invalid combinations with operators will report run-time errors.

In the event that you wish to catch these type errors earlier, you can also declare a variable using the VAR statement: e.g.

var x int = 10

Using VAR in this way will:

To re-type that variable locally you would first have to UNSET the variable.

6.2 Auto-vivification (assignment-driven)

Auto-vivification is an assignment feature: assigning through an access path creates intermediate containers as needed. This enables concise construction of nested structures without pre-allocation boilerplate.

6.3 Global mutation is explicit (@)

To modify a global variable from inside a function, use @. Example pattern:

def q()
    @a = true
end

Without @, assignment targets local scope.

7. Typed declarations with var

Za is dynamically typed by default. var is used when you want explicit intent:

7.1 Scalars and structs

var z int
var user struct_user
var cow,pig,sheep animal

Namespaced struct types are supported:

var x ea::type_struct

7.2 Fixed-size arrays (usual pattern)

Fixed-size arrays use:

var arr [1000] int

Multi-dimensional fixed arrays are also supported:

var grid [2][3]int

7.3 Multi-dimensional dynamic arrays

var matrix [][]int
var cube [][][]string

Dynamic arrays will be resized on demand on out-of-bounds assignment.


Part III — Data Types and Structures

8. Scalar types

Za provides the scalar types used most often in operational scripting:

9. Arrays and multi-dimensional data

Arrays may be dynamic or fixed-size. Nested arrays represent multi-dimensional data. The array library provides 23 functions for creation, manipulation, searching, and analysis.

9.1 Array Creation Functions

Create arrays with specific patterns and dimensions:

# zeros - create zero-filled arrays
z1d = zeros(5)              # [0, 0, 0, 0, 0]
z2d = zeros(2, 3)           # [[0,0,0], [0,0,0]]
z3d = zeros(2, 2, 2)        # 2x2x2 cube of zeros

# ones - create one-filled arrays
o1d = ones(4)               # [1, 1, 1, 1]
o2d = ones(3, 2)            # [[1,1], [1,1], [1,1]]

# identity - create identity matrix
i3 = identity(3)            # [[1,0,0], [0,1,0], [0,0,1]]

# reshape - change array dimensions
flat = [1, 2, 3, 4, 5, 6]
mat = reshape(flat, [2, 3]) # [[1,2,3], [4,5,6]]

# flatten - convert multi-dim to 1D
nested = [[1, 2], [3, 4], [5, 6]]
flat = flatten(nested)      # [1, 2, 3, 4, 5, 6]

9.2 Array Search and Selection

Find elements and extract values based on conditions:

# argmax/argmin - find index of max/min value
arr = [1, 5, 3, 9, 2]
max_idx = argmax(arr)       # 3 (index of value 9)
min_idx = argmin(arr)       # 0 (index of value 1)

# find - locate indices of elements matching a condition
nums = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
find(nums, "#>5")           # [5, 6, 7, 8, 9] - indices where value > 5
find(nums, "#%2==0")        # [1, 3, 5, 7, 9] - indices of even values
find(nums, 5)               # [4] - index where value equals 5

# find with map data
people = [
    map(.name "Alice", .age 25, .salary 50000),
    map(.name "Bob", .age 30, .salary 60000)
]
find(people, "#.age>28")    # [1] - indices where age > 28

# where - conditional selection (returns values, not indices)
where(nums, "#>5")          # [6, 7, 8, 9, 10] - values > 5
where(nums, "#%2==0", 99, 0) # Replace even with 99, odd with 0

9.3 Array Combination Functions

Join and stack arrays together:

# concatenate - join arrays along existing axis
a = [1, 2, 3]
b = [4, 5, 6]
concatenate(a, b)           # [1, 2, 3, 4, 5, 6] - default

mat1 = [[1, 2], [3, 4]]
mat2 = [[5, 6], [7, 8]]
concatenate(mat1, mat2, 1)  # [[1,2,5,6], [3,4,7,8]] - horizontal

# stack - create new axis
stack(a, b)                 # [[1,2,3], [4,5,6]] - vertical
stack(a, b, 1)              # [[1,4], [2,5], [3,6]] - horizontal

# squeeze - remove singleton dimensions
single_row = [[1, 2, 3]]    # 1x3 matrix
squeeze(single_row)         # [1, 2, 3] - becomes 1D

9.4 Statistical Operations

Compute aggregates and statistics across arrays:

# Basic statistics (flatten all dimensions)
data = [1, 2, 3, 4, 5]
mean(data)                  # 3.0
std(data)                   # ~1.414
variance(data)              # 2.0
median(data)                # 3
prod(data)                  # 120

# Axis-aware operations
matrix = [[1, 2, 3],
          [4, 5, 6],
          [7, 8, 9]]

mean(matrix, 0)             # [4, 5, 6] - column means
mean(matrix, 1)             # [2, 5, 8] - row means
sum(matrix, 0)              # [12, 15, 18] - column sums
sum(matrix, 1)              # [6, 15, 24] - row sums

# keepdims parameter preserves dimensions
mean(matrix, 0, true)       # [[4, 5, 6]] - keeps 2D shape

9.5 Linear Algebra Operations

Matrix decomposition and analysis:

# Matrix properties
mat = [[1, 2], [3, 4]]
trace(mat)                  # 5 (sum of diagonal: 1 + 4)
det(mat)                    # -2.0 (determinant)
rank(mat)                   # 2 (number of independent rows/cols)

# identity - create identity matrix
i3 = identity(3)            # [[1,0,0], [0,1,0], [0,0,1]]

# inverse - compute matrix inverse
inv = inverse(mat)          # [[-2, 1], [1.5, -0.5]]

# Use det_big and inverse_big for higher precision
det_big(mat)                # -2 (arbitrary precision)
inv_big = inverse_big(mat)  # High precision inverse

# Example with 3x3 matrix
mat3 = [[2, 0, 0],
        [0, 3, 0],
        [0, 0, 4]]
det(mat3)                   # 24 (2 * 3 * 4)
trace(mat3)                 # 9 (2 + 3 + 4)

See Section 22 for detailed condition syntax in find() and where(). See Appendix A for the complete list of all array functions.

10. Maps

Map literals use a dotted-key form:

m = map(.host "localhost", .port 5432)

Maps are used for:

11. Maps as sets

A map can represent a set using its keys. Set algebra operators apply to maps, which may be hierarchical or flat:

Predicate functions for relationships:

Use operators when you want a resulting set, and predicate functions when you want a relationship test.

12. Structs and anonymous structs

12.1 Defining structs

struct person
    name string [ = "default_value" ]
    age  int    [ = default_value ]
endstruct

Field names are normalised (you do not need to capitalise struct field names).

You may declare a variable of a struct type using

var variable_name struct_name

The variable will receive any default values you set during struct definition (or zero-like defaults if no defaults were provided).

You can manipulate fields of the struct through dotted access, e.g.:

var p person
p.name = "Billy"
p.age  = 42

12.2 Struct literals

Struct instances may also be created using constructor-style syntax and field initialisers (as shown in examples):

p1 = person("Alice",30)
p2 = person(.name "Bob", .age 21)

12.3 Anonymous structs

Anonymous structs are created with anon(...):

x = anon(.device device, .usage usage)

Use them when you want record-like data without declaring a named struct type.

13. Struct-associated functions and self

Structs may contain function definitions, supporting a lightweight method-like scheme.

Inside a struct-associated function, self refers to the current instance, and fields may be read/updated via self.field.

14. Enums

Za has an enum statement for defining enums at global or module scope. There is one predefined enum: ex, containing default exception categories.

An enum is defined like this:

enum enum_name ( enum_name_1 [ = value1 ] [ , ... , enum_name_N [ = valueN ] ] )

Setting a value with = sets the current auto-incrementing value. This means that you should always set a value for non-integer enum values.


Part IV — Expressions and Control Flow

15. Expressions and precedence

Za defines operator precedence in the interpreter. Notable points:

16. Operators (overview)

Comprehensive operator coverage for system administration tasks.

Za provides a rich set of operators that cover everything from basic arithmetic to advanced string manipulation and file operations. These operators are designed to make common system administration tasks concise and readable.

16.1 Arithmetic Operators

The standard arithmetic operators work with both integers and floating-point numbers:

# Basic arithmetic
result = 10 + 5        # 15
difference = 20 - 8    # 12
product = 6 * 7        # 42
quotient = 15 / 3      # 5.0
remainder = 17 % 5     # 2
power = 2 ** 8         # 256

# Floating-point operations
pi_approx = 22 / 7     # 3.142857...
area = 3.14159 * radius ** 2

# System administration examples
cpu_cores = 4
total_threads = cpu_cores * 2  # Hyperthreading
memory_gb = 16
memory_mb = memory_gb * 1024    # Convert to MB
disk_usage_percent = as_float(used_space / total_space) * 100

16.2 Comparison Operators

Comparison operators return boolean values and are essential for conditional logic:

# Numeric comparisons
if cpu_usage > 80.0
    alert("High CPU usage")
endif

if memory_available < 1024  # Less than 1GB
    alert("Low memory")
endif

# String comparisons
if hostname == "web-server-01"
    role = "web"
endif

if version != "latest"
    update_available = true
endif

# System administration examples
if disk_usage_percent >= 90
    cleanup_old_logs()
endif

if response_time <= 100  # milliseconds
    service_status = "good"
endif

if load_average >= cpu_cores
    scale_horizontal()
endif

16.3 Boolean Operators

Za provides both word-based and symbol-based boolean operators:

# Word-based operators (more readable)
if user_exists and password_valid
    grant_access()
endif

on backup_failed or disk_full do send_alert()

if not service_running
    start_service()
endif

# Symbol-based operators (concise)
user = get_env("USER")
pass = get_env("PASSWORD")
debug = get_env("DEBUG")
verbose = get_env("VERBOSE")

if user != "" && pass != ""
    authenticate()
endif

if debug == "true" || verbose == "true"
    enable_logging()
endif

service_active = get_env("SERVICE_ACTIVE")
if service_active != "true"
    restart_service()
endif

# System administration examples
if file_exists(config) and permissions_ok(config)
    load_config(config)
endif

if morning_hours or weekend
    backup_mode = "full"
else
    backup_mode = "incremental"
endif

16.4 Bitwise Operators

Bitwise operators are useful for working with file permissions, network masks, and flags:

# File permission manipulation
read_write = 0o666
executable = read_write | 0o111  # Add execute permission

# Permission checking
if file_mode & 0o111  # Check if executable
    file_type = "executable"
endif

# Network operations
network_mask = 0xFFFFFF00
network_part = ip_address & network_mask

# Flag operations
backup_flags = 0
backup_flags = backup_flags | 0x01  # Enable compression
backup_flags = backup_flags | 0x02  # Enable encryption

if backup_flags & 0x01
    use_compression = true
endif

16.5 Set Operators on Maps

Set operators work on maps to combine, intersect, and subtract key-value pairs:

# Configuration merging
default_config = map(.port 8080, .timeout 30, .debug false)
user_config = map(.timeout 60, .debug true)
final_config = default_config | user_config
# Result: {"port": 8080, "timeout": 60, "debug": true}

# Finding common configuration
required_keys = map(.host "localhost", .port 8080)
provided_keys = map(.host "localhost", .port 8080, .ssl true)
common = required_keys & provided_keys
# Result: {"host": "localhost", "port": 8080}

# Removing unwanted settings
base_config = map(.user "admin", .pass "secret", .host "db")
sanitized = base_config - map(.pass "secret")
# Result: {"user": "admin", "host": "db"}

# System administration examples
server_defaults = map(.cpu_limit "2", .memory "4G", .disk "20G")
override_config = map(.memory "8G", .disk "50G")
final_config = server_defaults | override_config

16.6 Range Operator

The range operator .. creates numeric ranges useful for loops and indexing:

# Basic ranges
numbers = 1..10        # [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

# System administration examples
#  - many of these can be done in alternate ways

# Port checking using shell commands (you would normally use a lib call for this)
open_ports=[]
foreach port in 8000..8005
    port_check = ${nc -z localhost {port} 2>&1}
    # Empty output means port is open
    on port_check.len == 0 do open_ports=concat(open_ports,port)
endfor

# Process multiple log files
foreach day in 1..31
    log_file = "/var/log/app-{day}.log"
    on is_file(log_file) do process_log(log_file)
endfor

# Generate server names
foreach i in 1..5
    server_name = "web-server-{i}"
    create_server(server_name)
endfor

16.7 Regex Match Operators

Za provides several regex operators for pattern matching:

# Case-sensitive match (for this you would normally just do file_type = $pe filename)
if filename ~ "\\.log$"
    file_type = "log"
endif

# Case-insensitive match
if hostname ~i "^web"
    server_role = "web"
endif

# System administration examples
if log_line ~ "ERROR"
    error_count += 1
endif

if user_agent ~i "bot|crawler|spider"
    traffic_type = "automated"
endif

16.8 Path Unary Operators

Path operators provide convenient file system path manipulation:

filepath = "/home/user/documents/report.txt"

# $pa - absolute path
path = $pa filepath        # "/home/user/documents"

# $pp - parent path
parent = $pp filepath          # "/home/user"

# $pb - Base name (filename)
basename = $pb filepath       # "report.txt"

# $pn - Name without extension
name = $pn filepath           # "report"

# $pe - Extension
extension = $pe filepath      # "txt"

16.9 String Transform Operators

String operators provide common text transformations:

text = "Hello World"

# $uc - Uppercase
upper = $uc text              # "HELLO WORLD"

# $lc - Lowercase
lower = $lc text              # "hello world"

# $st - String trim (both ends)
trimmed = $st "   spaced   "      # "spaced"
# $lt and $rt also exist for left trim and right trim

16.10 File I/O Operators

File operators make reading and writing files concise:

# $in - Read file contents
config_content = $in "/etc/app/config.json"
log_data = $in "/var/log/system.log"

# $out - Write to file
"New config content" $out "/tmp/new_config.json"
log_entry $out "/var/log/app.log"

# System administration examples
# Read configuration
db_config = json_decode($in "/etc/database/config.json")

# Write backup
current_config $out "/backup/config-{=date()}.json"

# Process configuration files - you could also use glob() for this
foreach config_file in dir("/etc/app",".*.conf")
    content = $in config_file.name
    processed = process_config(content)
    processed $out "/tmp/processed_{=$pb config_file}"
endfor

16.11 Shell Execution Operators

Shell operators enable system command execution:

# | - Pipe to shell (discard output)
| mkdir -p /tmp/backup

# Capture output
files =| ls -la /var/log
disk_usage =| df -h /

# Process monitoring - there are other ways of doing this
if ${pgrep mysqld} == ""
    | systemctl start mysql
endif

# Log analysis
error_count = ${grep -c ERROR /var/log/app.log} . as_int
if error_count > 100
    send_alert("Too many errors in logs")
endif

These operators provide a comprehensive toolkit for system administration tasks, making Za scripts concise, readable, and powerful for managing complex system operations.

16.1 Numeric Clamping with [start:end]

Za provides a clamping operator for constraining numeric values within specified ranges. This is particularly useful for data validation, normalization, and ensuring values stay within acceptable bounds.

Clamping Syntax

result = value[start:end]

The expression returns:

Type Inference

Partial Clamping

Either bound may be omitted:

# Only upper bound
percentage = score[:100]

# Only lower bound
temperature = [-10:]

# No clamping (full range)
normal_range = [0:100]

Practical Examples

# Basic clamping
sensor_reading = 127
normalized = sensor_reading[0:255]  # Clamps to 0-255 range
percentage = 85[0:100]              # Stays at 85
overflow = 300[0:255]               # Clamped to 255

# Type handling
float_value = 4.7[3.0:5.0]          # Result: 4.7 (within range)
int_clamped = 5.7[3:5]              # Result: 5 (int, clamped to upper bound)

# Configuration bounds
cpu_usage = current_cpu[0f:100f]  # Normalize CPU percentage
memory_usage = current_mem[0f:1f] # Normalize to 0-100% range

Use Cases for System Administration

# Network latency normalization
latency_ms = ping_time[0:5000]      # Clamp to reasonable network range

# File size validation
file_size_mb = file_size[0:10240]   # Max 10GB in MB

# Process priority adjustment
priority = nice_level[-20:19]       # Valid nice range

# Temperature monitoring - not sure why you would enforce this, but...
temp_celsius = sensor_temp[-40:125] # Operating range for server room

This clamping syntax reuses Za’s range-like brackets [:] but provides distinct behaviour for numeric types compared to array slicing.

17. Conditionals

Block conditional:

if condition
    # action
[ else
    # action ]
endif

Single-statement guard:

on condition1 do break
on condition2 do println "ok"
# etc

on … do executes exactly one statement when condition is true.

18. Loops

Counted loop:

for i=0 to 10
    println i
endfor

-or-

# c-like for construct - each term is optional
for i=0, i<=10, i++
    println i
endfor

Container iteration:

foreach item in items
    println item
endfor

Loop control:

break [ construct_type ]
break if condition
continue
continue if condition

19. Case statements

The CASE construct is written as a switch-like variant. It also allows for pattern matching:

case [expression]
[is expression_value
    # action
    ]
[has condition_expression
    # action
    ]
[contains "regex_match"
    # action
    ]
    .
    .
[or
    # default action
    ]
endcase

Part V — Functional Data Processing

20. Expression strings, #, and $idx

Za uses expression strings in several places:

Substitution phrases:

Use backticks for clarity when the expression contains quotes.

21. Map and filter operators

Filter:

bad = rows ?> `#.UsePercent.replace("%","").as_int > 80`

Map:

names = users -> `#.name`

22. Searching arrays (find, where)

Both functions use the same expression engine for consistent condition syntax. find() returns indices matching a condition, while where() returns the matching values.

Basic usage:

# find - returns indices of matching elements
nums = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
find(nums, "#>5")           # [5, 6, 7, 8, 9] - indices where value > 5
find(nums, "#%2==0")        # [1, 3, 5, 7, 9] - indices of even values

# where - returns the values themselves
where(nums, "#>5")          # [6, 7, 8, 9, 10] - values > 5
where(nums, "#%2==0")       # [2, 4, 6, 8, 10] - even values

Condition syntax:

Advanced examples:

# Numeric comparisons
find(nums, "#>5")              # Greater than
find(nums, "#>=5 and #<=8")    # Range

# Index-based conditions
find(nums, "$idx>5")           # Elements at index > 5
find(nums, "$idx%2==0")        # Elements at even indices

# With map data
rows = [map(.MountedOn "/"), map(.MountedOn "/boot")]
idx = rows.find(`#.MountedOn=="/"`)     # [0]
sel = rows.where(`#.MountedOn~"^/b"`)   # [map(...)] - /boot match

# Nested map field access
employees = [
    map(.name "Alice", .dept map(.name "Eng", .budget 100000)),
    map(.name "Bob", .dept map(.name "Sales", .budget 75000))
]
find(employees, "#.dept.budget>80000")  # [0]

# Replacement with where
where(nums, "#>5", 99, 0)    # Replace >5 with 99, else 0

See Section 9 for comprehensive array library function examples.


Part VI — Functions, Modules, and Composition

23. Functions (def … end)

def f(x)
    return x*2
end

return may be used without a value. return may also return multiple values (comma separated expressions).

The return values may also be unpacked on return:

def f(a,b,c)
    return b,c,a
end

def g(a,b,c)
    return [b,c,a]
end

b,c,a=f(1,2,3)
b,c,a=g(4,5,6)
# or
vals=g(4,5,6) # vals=[5,6,4]

24. Modules and namespaces

Import a module:

module "cron"
module "util" as u

Namespaced types and values are referenced with :::

var x u::struct_example

The USE statement

This statement is used to indicate the order in which namespaces are processed by the interpreter.

Syntax:

    USE -           # empties the use_chain internal array
    USE + name      # adds name to the use_chain namespace list (if unique)
    USE - name      # removes name from use_chain namespace list
    USE ^ name      # places namespace name at the top of the use_chain namespace list (pushes rest down)
                    # new name: inserts, existing name: moves
    USE PUSH        # push current chain on chain stack
    USE POP         # pop chain from top of chain stack
                    # push and pop would be used to completely isolate namespacing in a module.

The current namespace is always either main:: or the module name/alias. If you want to use a different namespace then you need to create a new file and import it with MODULE.

The use_chain array will be consulted, when not empty, on function calls, enum references and struct references for matches ahead of the default behaviour, if no explicit name is supplied:

1. explicit namespace (name::)
2. use_chain match
3. current namespace (no :: ref), then
4. main:: (global namespace)

Example:

    # global ns / main program
    MODULE "modpath/time" AS tm

    tm::string_date()       # call function string_date in module time with explicit alias 'tm'
    string_date()           # tries to call (non-existant) function string_date in main:: namespace (current namespace)
                            # which should error as undefined.

    USE +tm

    string_date()           # check if string_date() exists in tm namespace and call it if found.

                            # if not found (even though this one would be) then try to call it in current namespace (main::)
                            #  which should error as undefined.

                            # whenever there are conflicting names then the first match takes precedence.
                            #  i.e.
                            # explicit name > use_chain > current namespace > main

Bundling (v.1.2.1+)

There is an experimental facility for packaging the interpreter with a za script and it’s module dependencies into a single executable.

If the -x argument is used, this bundling will be triggered.

That is:

za -x [ -n test_bundle_name ] script_name

On execution, the bundled version will unpack to a uniquely named directory in /tmp/. The execution path should still be where the bundle was called from, not the extracted directory.

N.B.: due to this, you should avoid the use of the execpath() call inside bundled scripts.

If the -n argument is not provided, the default bundled file is named exec.za.

Example:

# build the bundle
> za -x -n factest eg/fac
Rewriting MODULE statements in module: eg/syntax_checks/modules/math.mod
RewriteModuleContent called for scriptDir=eg, modulePath=eg/syntax_checks/modules/math.mod
- Found 0 MODULE statements in module eg/syntax_checks/modules/math.mod
Discovered 1 modules
Module: syntax_checks/modules/math.mod -> syntax_checks/modules/math.mod
RewriteScript called with 1 modules
- Rewriting module: syntax_checks/modules/math.mod -> syntax_checks/modules/math.mod
- Replaced 1 instances of MODULE "syntax_checks/modules/math.mod" with "./syntax_checks/modules/math.mod"
- zaData size = 6229228, bundleData size = 4096
- tarStart = 6229228, tarLength = 4096
writing magic bytes: [90 65 66 85 78 68 76 69]
Bundle created: factest (6233348 bytes)

# execute the bundle with arguments
> ./factest 5
120

The aim of this bundling process is to keep dependencies together and allow for greater portability. There may still be architectural issues with bundles as they copy your local za version into the bundle.


Part VII — Errors, Debugging, and Safety

25. Error handling philosophy

Exceptions exist, but are not intended as a primary, idiomatic error-handling scheme. You should prefer explicit return values and structured results where available, using exceptions when there is no better alternative or when you choose that style deliberately.

Important Note: Avoid using try...catch blocks for routine error handling. Reserve exceptions for truly exceptional circumstances that cannot be handled through normal return value patterns. Overuse of exceptions makes code harder to read and maintain.

26. Exceptions (try … catch … then … endtry)

A try block may capture outer variables explicitly with uses:

try uses captured_var [,...,captured_var] [throws string_category|ex_enum_category]
    # action
catch [err] [is category_expr | in list_expr | contains regex]
    # action
endtry

Catches may use predicates (example pattern):

catch err is "invalid"
    println "invalid:", err

then is the cleanup/finally section and runs regardless of whether an exception occurred.

27. Enhanced error handling (trap, error_*)

Za supports registering an error trap and introspecting error context inside the handler using error_* functions. Use this to produce better diagnostics (message, source location, source context, call stack, locals/globals) than the default handler when needed.

28. Debugger

Za includes an interactive debugger that supports step execution, expression inspection, tracing, and runtime introspection. The debugger is designed for both development debugging and operational troubleshooting.

28.1 Enabling the Debugger

Debugging can be enabled in two ways:

Disable it using: za debug off

28.2 Debugger Activation

When enabled, the debugger can pause execution at:

⚠️ The debugger does not automatically trigger on unhandled errors or exceptions.

28.3 Setting Breakpoints

Use breakpoints to pause execution at specific points:

def complex_calculation(x, y)
    debug break  # Pause execution here
    result = x * y + sqrt(x + y)
    debug break  # Or here to inspect intermediate result
    return result
end

28.4 Debugger Prompt and Commands

When paused, Za shows an interactive prompt:

[scope main : line 0012 : idx 0008] debug> _

Essential Debugger Commands

Command Description
c, continue Resume execution
s, step Step into next statement or function
n, next Step to next statement in current function
l, list Show current statement tokens
v, vars Dump local variables
p <var>, print <var> Print value of a variable
bt, where Show call chain backtrace
w <var>, watch <var> Add variable to watch list
e <expr>, eval <expr> Evaluate expression in current scope

Advanced Debugger Commands

Command Description
ctx Set line context size for list
mvars Dump module/global variables
gvars Dump system/global variables
sf, showf Show all defined functions
ss, shows Show all defined structs
b, breakpoints List all breakpoints
b+, ba Add a breakpoint interactively
b-, br Remove a breakpoint
d, dis Disassemble current statement tokens
uw <var>, unwatch <var> Remove variable from watch list
wl, watchlist Show all watched variables
fn, file Show current file name
ton, traceon Enable line-by-line execution trace
toff, traceoff Disable execution trace
fs, functionspace Show current debug entrypoint and functionspace
cls Clear debugger screen
q, quit, exit Exit interpreter completely
h, help Show this help message

28.5 Debugger Scope and Behaviour

28.6 Practical Debugging Examples

Debugging Function Calls

def calculate_discount(price, category)
    debug break  # Inspect inputs
    if category == "premium"
        discount = 0.20
    else
        discount = 0.10
    endif
    debug break  # Check calculation logic
    final_price = price * (1 - discount)
    debug break  # Verify result
    return final_price
end

# Usage
price = calculate_discount(100.0, "premium")
# Debugger pauses at each debug break for inspection

Debugging Data Processing

# Complex data transformation
data = table(${ps aux}, map(.parse_only true))
cpu_intensive = data ?> `#[2].as_float > 0.5`

debug break  # Inspect filtering results

# Process high-CPU processes
foreach proc in cpu_intensive
    debug break  # Examine each process
    println "Killing process, name : ", proc[10]
    | kill -9 {=proc[1]}
endfor

Debugging System Integration

# Network service debugging
def check_service(host, port)
    debug break  # Check connection parameters
    result = tcp_ping(host, port, 5000)  # 5 second timeout
    debug break  # Examine connection result

    if not result.okay
        log error: "Cannot reach", host, ":", port
        return false
    endif

    debug break  # Verify success path
    return true
end

# Test with debugger
reachable = check_service("db.example.com", 5432)

29. Profiler

Za includes a built-in function-level profiler that records execution times for each function and optionally provides detailed call-chain breakdowns.

29.1 Enabling the Profiler

Enable profiling using either:

29.2 Profiler Behaviour

When enabled:

Profiling incurs minimal overhead but is best used for performance debugging and analysis.

29.3 Viewing and Interpreting Results

After script execution completes, a summary is printed to standard output:

Profile Summary

main:
  parse: 360.182µs
  enum_names: 15.696µs
  execution time: 13.345043ms

  main > x11:
    execution time: 57.649µs

x11::fg:
  enum_names: 287.404µs
  eval: 717.135µs
  execution time: 3.801211ms

Interpreting Output

29.4 Performance Analysis Patterns

Finding Bottlenecks

# Profile this script
za -P data_processing.za

# Sample output shows:
data_processing:
  load_data: 2.3s
  parse_records: 8.7s
  validate_data: 15.2s
  transform_data: 45.6s  # <-- Bottleneck
  save_results: 1.1s

#### Analyzing Recursive Performance

**Note:** The following example is illustrative only - actual profiling output may differ.

```za
# Recursive function with profiling
def factorial(n)
    on n <= 1 do return 1
    return n * factorial(n - 1)
end

# Profile to analyze recursion depth
za -P recursive_test.za
# Look for performance patterns in actual output

29.5 Profiler Best Practices

30. Security controls (permit)

permit() controls runtime capabilities such as allowing shell execution, eval, interpolation, macros, and strictness for uninitialised variables.

Comprehensive security best practices for Za scripts.

If you are going to use Za in situations it was not designed for, then work through the checklist below:

30.1 Security Checklist

Use this checklist for security review:

Input Validation:

File Operations:

Command Execution:

Network Security:

Logging and Monitoring:

Code Integrity:

By following these security practices and using permit() to control runtime capabilities, you can write Za scripts that are robust, maintainable, and secure against common vulnerabilities. It does not mean that you should though.

Handle errors without exposing sensitive information:

30.2 Runtime Security with permit()

The permit() function controls runtime capabilities such as allowing shell execution, eval, interpolation, macros, and strictness for uninitialised variables. For hardened scripts, disable what you don’t need and re-enable only deliberately.

# Disable all potentially dangerous features
permit("shell", false)
permit("eval", false)
permit("interpol", false)
permit("macro", false)

# Enable strict variable checking
permit("uninit", false)

# Later, selectively re-enable what's needed
permit("shell", true)
permit("sanitisation", false)

30.3 Input Validation and Sanitization

Always validate and sanitize external inputs to prevent injection attacks and data corruption.

31. Common async patterns

Parallel execution patterns for system administration.

Za’s async capabilities enable efficient parallel processing of multiple hosts, services, or data sources.

31.1 Fan-out/Fan-in Host Probing

# Define async check function
def check_host(host_id)
    pause rand(500)  # Simulate network delay
    return host_id % 3 == 0 ? "up" : "down"
end

# Fan-out: check multiple hosts in parallel
hosts = [1, 2, 3, 4, 5, 6]
var handles map

for x = 0 to len(hosts)-1
    async handles check_host(hosts[x]) x  # Use index as key
endfor

# Fan-in: collect all results
results = await(ref handles, true)
for e = 0 to len(hosts)-1
    println "Host {=hosts[e]} -> {=results[e]}"
endfor

# Filter up hosts
up_hosts = hosts ?> `results[#] == "up"`
println "Up hosts:", up_hosts

31.2 Parallel Service Checks

# Service status checking
def check_service(service_name)
    In real usage: status = ${systemctl is-active {service_name}}
    return service_name ~ "nginx|mysql" ? "running" : "stopped"
end

# Check multiple services in parallel
services = ["nginx", "mysql", "redis", "postgresql"]
var service_handles map

for x = 0 to len(services)-1
    async service_handles check_service(services[x]) x  # Use index as key
endfor

service_results = await(ref service_handles, true)
for e = 0 to len(services)-1
    println "Service {=services[e]} -> {=service_results[e]}"
endfor

# Filter running services
running_services = services ?> `service_results[#] == "running"`
println "Running services:", running_services

31.3 Parallel Data Processing

# Process multiple data items in parallel
def process_data(item)
    pause rand(500)  # Simulate processing time
    return item * 2  # Simple transformation
end

# Process array in parallel
data = [10, 20, 30, 40, 50]
var data_handles map

for x = 0 to len(data)-1
    async data_handles process_data(data[x]) x  # Use index as key
endfor

processed_results = await(ref data_handles, true)
for e = 0 to len(data)-1
    println "Input: {=data[e]} -> Output: {=processed_results[e]}"
endfor

# Filter results
high_values = processed_results ?> `# > 50`
println "High values:", high_values

31.4 Isolated Failure Handling

# Network checks with isolated failures
def check_network(host)
    pause rand(500)  # Simulate network timeout
    # Simulate different failure modes
    case host
    is "8.8.8.8"
        return "up"
    is "1.1.1.1"
        return "timeout"
    or
        return "down"
    endcase
end

# Check multiple network hosts
network_hosts = ["8.8.8.8", "1.1.1.1", "gateway.company.com"]
var net_handles map

for x = 0 to len(network_hosts)-1
    async net_handles check_network(network_hosts[x]) x  # Use index as key
endfor

network_results = await(ref net_handles, true)
for e = 0 to len(network_hosts)-1
    println "Host {=network_hosts[e]} -> {=network_results[e]}"
endfor

# Process results even if some failed
up_hosts = network_hosts ?> `network_results[#] == "up"`
problem_hosts = network_hosts ?> `network_results[#] != "up"`

println "Up hosts:", up_hosts
println "Problem hosts:", problem_hosts

31.5 Partial Completion Collection

# Database connectivity with partial success handling
def check_database(db_name)
    pause rand(800)  # Simulate connection attempt
    # Simulate different connection outcomes
    case db_name
    is "primary"
        return "connected"
    is "replica"
        return "connected"
    is "cache"
        return "timeout"
    or
        return "error"
    endcase
end

# Database configurations
databases = ["primary", "replica", "cache", "backup"]

# Check all databases in parallel
var db_handles map
for x = 0 to len(databases)-1
    async db_handles check_database(databases[x]) x  # Use index as key
endfor

db_results = await(ref db_handles, true)
for e = 0 to len(databases)-1
    println "Database {=databases[e]} -> {=db_results[e]}"
endfor

# Count successful connections
connected_dbs = databases ?> `db_results[#] == "connected"`
total_dbs = len(databases)

println "Connected: {=len(connected_dbs)}/{total_dbs} databases"

if len(connected_dbs) < total_dbs
    failed_dbs = databases ?> `db_results[#] != "connected"`
    println "Failed databases:", failed_dbs
endif

These async patterns enable efficient parallel processing while maintaining robust error handling and partial result collection.


Part IX — Output and Presentation

32. Program output (print, println)

Use print/println for ordinary output and examples not related to logging.

33. Inspection (pp) and tables (table)

UFCS calling forms are equivalent mechanisms for the same call:

table() is used heavily for formatting record-like data and for importing CLI output when parsing is enabled via options.

34. Array display controls (array_format, array_colours)

These are configuration functions, not formatters:

35. ANSI colour/style macros

Za strings supports inline style macros like:

println "[#bold][#1]ERROR[#-] message"

The ANSI macro handling can be enabled/disabled via ansi(true|false) and startup flags.

A full list of supported style macros can be found with:

help colour

Part X — Standard Library Overview

36. Library categories and discovery

Za groups standard library calls into categories. For further information about library calls use:

HELP [statement_name|function_name]
funcs("partial-function-name|category-name")

37. Category tour (representative idioms)

Comprehensive examples from Za’s standard library categories.

Za’s standard library is organized into functional categories that provide ready-to-use tools for common system administration tasks. This section showcases representative idioms from each category to demonstrate practical usage patterns.

38. Category samples

38.1 String Operations

String manipulation is fundamental for processing logs, configuration files, and user input:

# Basic string operations
text = "  System Log Entry  "
trimmed = text.trim                      # "System Log Entry"
upper = text.upper                       # "  SYSTEM LOG ENTRY  "
lower = text.lower                       # "  system log entry  "

# String splitting and joining
log_line = "2023-12-01 10:30:15 ERROR: Database connection failed"
parts = log_line.split(" ")              # ["2023-12-01", "10:30:15", "ERROR:", "Database", "connection", "failed"]
timestamp = parts[0] + " " + parts[1]    # "2023-12-01 10:30:15"
message = parts[3:].join(" ")            # "Database connection failed"

# Pattern matching and replacement
config_line = "port=8080"
if config_line ~ "port"
    port_value = config_line.split("=")[1].trim
    port_num = int(port_value)
endif

# Regular expressions for log parsing
log_entry = "192.168.1.100 - - [01/Dec/2023:10:30:15] \"GET /api/users HTTP/1.1\" 200 1234"
ip_match = log_entry.match("^(\\d+\\.\\d+\\.\\d+\\.\\d+)")
if ip_match
    client_ip = ip_match[0]
endif

# String formatting for reports
report = "Server: {0}, CPU: {1}%, Memory: {2}%".format(hostname, cpu_usage, memory_usage)

38.2 List and Array Operations

Lists and arrays are essential for managing collections of servers, files, or data points:

# Creating and manipulating lists
servers = ["web-01", "web-02", "db-01", "cache-01"]
web_servers = servers ?> `has_start("web-")`     # ["web-01", "web-02"]

# List transformations
port_numbers = [80, 443, 8080, 3000]
secure_ports = port_numbers -> `as_int(#+1)`              # [81, 444, 8081, 3001]

# List aggregation
response_times = [120, 85, 200, 95, 150]
avg_response = response_times.sum / response_times.len    # Average response time
max_response = response_times.max                         # 200
min_response = response_times.min                         # 85

# Array operations for numeric data
cpu_readings = [45.2, 67.8, 89.1, 34.5, 78.9]
high_cpu_periods = cpu_readings ?> `#>80`                 # [89.1]

# Multi-dimensional arrays for metrics
hourly_metrics = [
    [10, 15, 12, 8],
    [20, 25, 18, 22],
    [30, 35, 28, 32]
] # Hour 0-3, 4-7, 8-11
morning_avg = hourly_metrics[1].sum / 4                # Average for hours 4-7

38.3 Map Operations

Maps are perfect for configuration management, key-value stores, and structured data:

# Configuration management
server_config = map(
    .host "localhost",
    .port 8080,
    .ssl true,
    .timeout 30
)

# Accessing and modifying
if server_config.ssl is bool and server_config.ssl
    protocol = "https"
else
    protocol = "http"
endif

# Merging configurations
default_config = map(.timeout 60, .retries 3, .debug false)
user_config = map(.timeout 120, .debug true)
final_config = default_config.merge(user_config)
# Result: {"timeout": 120, "retries": 3, "debug": true}

# Map operations for system inventory
server_info = map(
    .web-01 map(.cpu 4, .memory 8, .disk 100),
    .web-02 map(.cpu 4, .memory 8, .disk 100),
    .db-01  map(.cpu 8, .memory 32,.disk 500)
)

# Extract specific information
total_memory = (server_info . values -> `#.memory`) . sum  # 48 GB
high_cpu_servers = server_info ?> `#.cpu>4`                # {"db-01": {...}}

# Dynamic map building - not real functions!
for server in server_list
    cpu = get_cpu_usage(server)
    memory = get_memory_usage(server)
    metrics[server] = map(.cpu cpu, .memory memory)
endfor

38.4 Type Conversion

Conversion between data types is crucial for data processing and validation:

# String to number conversion
port_str = "8080"
port_num = port_str.as_int                     # 8080
memory_str = "4.5 GB"
memory_gb = as_float(memory_str.split(" ")[0]) # 4.5

# Number to string conversion
cpu_percent = 75.5
cpu_str = as_string(cpu_percent)               # "75.5"
status_code = 200
status_str = as_string(status_code)            # "200"

# JSON conversion
config_map = map(.host "localhost, .port 8080)
config_json = config_map.pp                 # '{"host":"localhost","port":8080}'
parsed_config = json_decode(config_json)    # Back to map

# Base64 encoding/decoding - don't do this
secret_data = "user:password"
encoded = secret_data.base64e        # "dXNlcjpwYXNzd29yZA=="
decoded = encoded.base64d            # "user:password"

38.5 File Operations

File operations are essential for configuration management, log processing, and data persistence:

# Reading files
config_content = read_file("/etc/app/config.json")

# Writing files
backup_content = "Backup created at " + date()
write_file("/backup/config-{=date()}.bak", backup_content)

# File existence and properties
if is_file("/etc/app/config")
    stat = stat("/etc/app/config")
    size_mb = stat.size / (1024f * 1024)
    modified = stat.modtime
endif

# Directory operations
config_files = dir("/etc/app",".*.conf")  # List all .conf files
foreach config_file in config_files
    if config_file.size > 0
        process_config(config_file.name)
    endif
endfor

38.6 OS and System Operations

System operations enable interaction with the operating system for monitoring and control:

# Environment variables
user = get_env("USER") or "unknown"
home_dir = get_env("HOME") or "/tmp"
path_list = get_env("PATH").split(":")

# System information
hostname = hostname()
os_type = os()
pid = pid()

# Directory operations
if not is_dir(file)
    | mkdir "{file}"
    | chmod 755 {file}
endif

cd("/var/log")
current_dir = cwd()

# Process management
current_pid = pid()
parent_pid = ppid()
process_info = ps_info(current_pid)

# User and group information
current_user = user()
current_uid = user_info(current_user).UID
current_gid = user_info(current_user).GID

38.7 Network Operations

Network operations provide tools for connectivity testing, HTTP requests, and network monitoring:

# HTTP requests
response = web_get("https://api.example.com/status")
if response.code == 200
    status_data = json_decode(response.result)
endif

# POST request with data
data = map(.message "Server backup completed")
headers = map(.Content-Type "application/json")
response = web_raw_send("POST", https://hooks.slack.com/webhook", headers, data)

# Network connectivity testing
if icmp_ping("8.8.8.8")
    internet_available = true
endif

# Port checking
if port_scan("localhost", [80], 2) . 80
    web_server_running = true
endif

# DNS resolution
ip_address = dns_resolve("example.com")
if ip_address.records.len>0
    println "example.com resolves to: " + ip_address.records[0]
endif

# Network interface information
for interface in net_interfaces_detailed()
    if interface.up and not interface.name == "lo"
        println "Interface: ", interface.name, " IP: ", interface.ips
    endif
endfor

38.8 Database Operations

Database operations enable interaction with various database systems for data storage and retrieval:


set_env("ZA_DB_ENGINE","sqlite3")

try
    h=db_init(execpath()+"/files/test.db")
    res=h.db_query("select * from users",map(.format "map"))
    h.db_close
    # first 30
    println res[:30].table(
        map(
            .border_style "unicode",
            .colours map(
                .header     fgrgb(200,100,0),
                .data       fgrgb(10,100,200)
            ),
            .column_order ["id","name","email"]
        )
    )
endtry

38.9 YAML and Configuration Operations

YAML operations are essential for managing configuration files in modern applications.

Basic Usage

Parse YAML:

yaml_str = "name: John\nage: 30\ncity: New York"
data = yaml_parse(yaml_str)

Marshal to YAML:

data["name"] = "Alice"
data["age"] = 28
yaml_output = yaml_marshal(data)

Parse nested structures:

yaml3 = "person:\n  name: Jane\n  age: 25\n  hobbies:\n    - reading\n    - swimming"
result3 = yaml_parse(yaml3)

Parse lists:

yaml2 = "- apple\n- banana\n- orange"
result2 = yaml_parse(yaml2)

Get nested values

host = yaml_get(data, "server.host")           # returns "localhost"
debug = yaml_get(data, "server.config.debug")  # returns true
port1 = yaml_get(data, "server.ports[0]")      # returns 8080
port2 = yaml_get(data, "server.ports[1]")      # returns 8081

Update existing values

data = yaml_set(data, "server.host", "example.com")
data = yaml_set(data, "server.config.debug", false)
data = yaml_set(data, "server.ports[0]", 9090)

Add new values

data = yaml_set(data, "server.timeout", 30)

Remove specific values

data = yaml_delete(data, "server.config.debug")
data = yaml_delete(data, "server.ports[1]")  # removes second port

Please see za_tests/test_yaml.za for a larger example set.

38.10 Archive Operations (ZIP)

Archive operations are useful for backup, deployment, and file distribution.

Create ZIP:

files = ["test1.txt", "test2.txt"]
result = zip_create("test_archive.zip", files)

List contents:

contents = zip_list("test_archive.zip")

Extract all files:

result = zip_extract("test_archive.zip", extract_dir)

Extract specific files:

result = zip_extract_file("test_archive.zip", ["files","to","extract"], "single_extract_dir")

Add files:

result = zip_add("test_archive.zip", ["files","to","add])

Remove files:

result = zip_remove("test_archive.zip", ["test2.txt"])

Please see za_tests/test_zip.za for a larger example set.

38.11 Regular Expressions (PCRE)

Regular expressions provide powerful pattern matching for text processing. The reg_* library calls use a PCRE library implementation instead of the builtin regular expression engine. Due to this, these calls are only available on static linux builds of Za.

Searching:

# Tests if string contains regex match:
reg_match(string, regex)

Filtering:

# Returns array of [start_pos, end_pos] match positions:
reg_filter(string, regex[, count])

Replacement:

# Replaces regex matches with replacement string:
reg_replace(var, regex, replacement[, int_flags])

38.12 Checksum Operations

Checksum operations are essential for file integrity verification and security.

# Returns MD5 checksum of input string:
md5sum(string)

 Returns SHA1 checksum of input string
sha1sum(string)

# Returns SHA224 checksum of input string
sha224sum(string)

# Returns SHA256 checksum of input string:
sha256sum(string)

# Returns struct with .sum and .err for S3 ETag comparison:
s3sum(filename[, blocksize])

S3 ETag Functionality

The s3sum function specifically calculates checksums compatible with Amazon S3 ETags, including multipart upload format (hash-parts) for files larger than the blocksize.

38.13 TUI (Terminal User Interface)

Create TUI objects and style:

# Create TUI options map
tui_obj = tui_new()
# Create style with custom borders and colours
style = tui_new_style()

Text display with box:

tui_obj["Action"] = "text"
tui_obj["Content"] = "Hello, World!"
tui_obj["Row"] = 5
tui_obj["Col"] = 10
tui_obj["Width"] = 30
tui_obj["Height"] = 5
tui_obj["Border"] = true
tui(tui_obj, style)

Interactive menu:

tui_obj["Action"] = "menu"
tui_obj["Title"] = "Choose an option:"
tui_obj["Options"] = ["Option 1", "Option 2", "Exit"]
tui_obj["Row"] = 10
tui_obj["Col"] = 20
result = tui(tui_obj, style)

Progress bar:

tui_obj["Action"] = "progress"
tui_obj["Title"] = "Processing..."
tui_obj["Value"] = 0.75  # 75% complete
tui_obj["Row"] = 15
tui_obj["Col"] = 5
tui_obj["Width"] = 40
tui_progress(tui_obj, style)

Text editor:

edited_text = editor("Initial content", 80, 24, "Edit Document")

Table display:

tui_obj["Action"] = "table"
tui_obj["Data"] = "Name,Age,City\nJohn,30,NYC\nJane,25,LA"
tui_obj["Format"] = "csv"
tui_obj["Headers"] = true
tui_table(tui_obj, style)

Screen buffer switching:

tui_screen(0)  # Switch to primary screen
tui_screen(1)  # Switch to secondary screen

The TUI system uses maps to configure display properties like position (Row, Col), size (Width, Height), content (Content, Data), and styling (Border, colours)

38.14 Notification Operations

Za provides 7 builtin file system notification library functions.

Watcher Management Functions

# Create new watcher, returns [watcher, error_code]
# - Error codes: 0=success, 1=create_watcher_failed, 2=file_path_failure
ev_watch(filepath_string)

# Dispose of watcher object
ev_watch_close(watcher)

# Check if watcher is still available
ev_exists(watcher)

Path Management Functions

# Add a path to existing watcher
ev_watch_add(watcher, filepath_string)

# Remove a path from watcher
ev_watch_remove(watcher, filepath_string)

Event Handling Functions

# Sample events from watcher, returns notify_event or nil
ev_event(watcher)

# Test event type, returns filename or nil
ev_mask(notify_event, str_event_type)

Event Types

Supported event types for ev_mask:

38.15 Error Handling and Logging

Robust error handling and logging are essential for reliable system administration:

# Structured error handling
try
    config = load_configuration("/etc/app/config.json")
    validate_config(config)
    apply_config(config)
catch config_error
    log error: "Configuration error: " + config_error.message
catch validation_error
    log error: "Validation failed: " + validation_error.message
    rollback_config()
catch system_error
    log critical: "System error: " + system_error.message
    emergency_shutdown()
finally
    cleanup_temp_files()
endtry

# Custom exception types
exreg("ConfigError", "error")
exreg("NetworkError", "warning")
exreg("DatabaseError", "critical")

# Logging with different levels
log debug: "Starting configuration process"
log info: "Loading configuration from " + config_file
log warning: "Using default values for missing settings"
log error: "Failed to connect to database"
log critical: "System out of memory"

These representative idioms demonstrate the flexibility of Za’s standard library categories for system administration tasks. Each category provides specialized tools that can be combined to create comprehensive automation solutions.

38.16 INI Configuration File Operations

INI files provide simple configuration management for applications and services. The INI library, where possible, preserves comments, blank lines, and formatting while reading and writing configuration files.

Reading and Writing

Basic read and write operations:

# Read INI file
config = ini_read("/etc/app/config.ini")

# Modify configuration and write back
config.ini_write("/etc/app/config.ini")

Section Management

Add, insert, and delete sections:

# Append new section at end
config = ini_new_section(config, "logging")

# Insert section at specific position (1-indexed, 0=prepend)
config = ini_insert_section(config, "cache", 2)

# Delete section by name
config = ini_delete_section(config, "deprecated_section")

Adding Configuration Entries (direct/manual)

Create section data with metadata, comments, and values:

# Section metadata (required)
section_meta["type"] = "metadata"
section_value["section_order"] = 1
section_meta["value"] = section_value

# Data entries
entry1["type"] = "data"
entry1["key"] = "host"
entry1["value"] = "localhost"

entry2["type"] = "data"
entry2["key"] = "port"
entry2["value"] = 8080

entry3["type"] = "data"
entry3["key"] = "debug"
entry3["value"] = true

# Assemble section
section_data = [section_meta, comment, entry1, entry2, entry3]
config["database"] = section_data

Key/Value Manipulation

You may also use helper calls for simplified access to configuration entries:

# Add or update keys
config = ini_add_key(config, "database", "host", "localhost")
config = ini_set_key(config, "database", "port", 5432)

# Get key values
host = ini_get_key(config, "database", "host")  # returns "localhost"
port = ini_get_key(config, "database", "port")  # returns 5432

# Delete keys
config = ini_delete_key(config, "database", "legacy_field")

These functions provide a more intuitive interface for common configuration management tasks compared to manual map manipulation.

Key Inspection and Listing

Check for key existence and list all keys in a section:

# Check if key exists
has_host = ini_has_key(config, "database", "host")     # returns true
has_ssl = ini_has_key(config, "database", "ssl_mode") # returns false

# List all keys in a section
db_keys = ini_list_keys(config, "database")  # returns ["host", "port", "username"]
log_keys = ini_list_keys(config, "logging")  # returns ["level", "file", "format"]

These inspection functions are useful for validation, migration scripts, and conditional configuration updates.

Section Operations

Retrieve and replace entire sections:

# Get complete section data
db_section = ini_get_section(config, "database")
# Returns: [metadata, entry1, entry2, ...]

# Replace entire section
new_logging = [
    map(.type "metadata", .value map(.section_order 3)),
    map(.type "data", .key "level", .value "DEBUG"),
    map(.type "data", .key "file", .value "/var/log/app.log")
]
config = ini_set_section(config, "logging", new_logging)

Section operations are useful for bulk updates, template-based configuration, and migrating between different configuration formats.

Adding Keys with Comments

Add keys with inline comments for documentation:

# Add key with explanatory comment
config = ini_add_key_with_comment(config, "database", "max_connections",
    100, "# Maximum concurrent database connections")

config = ini_add_key_with_comment(config, "logging", "rotation",
    "daily", "# Rotate logs daily to manage disk space")

Comments are preserved when writing the configuration and help maintain documentation within the configuration file itself.

Metadata Updates

Update section metadata for ordering and organization:

# Update section order for consistent file layout
config = ini_meta_update(config)

# After operations that modify sections, call ini_meta_update
# to ensure section_order metadata is consistent

The ini_meta_update function automatically updates section ordering metadata after structural changes, ensuring consistent output formatting when the configuration is written to file.

Global Section Operations

Access and modify global entries (before any section headers):

# Get global section entries
global_entries = ini_get_global(config)

# Set global section entries
new_global = [
    map(.type "comment", .comment "# Global settings"),
    map(.type "data", .key "timeout", .value 30)
]
config = ini_set_global(config, new_global)

Array Formatting

Control array output format:

# CSV format: value1,value2,value3
entry["format"] = "csv"
entry["value"] = [1, 2, 3]

# Za format: ["value1","value2","value3"]
entry["format"] = "za"
entry["value"] = ["a", "b", "c"]

Preserving Formatting

The library automatically attempts to preserve blank lines between sections and maintains original formatting:

# Blank lines are preserved as "space" entries
blank_line["type"] = "space"
section_data = [metadata, entry1, blank_line, entry2]

# When writing, sections are separated by blank lines
config.ini_write("/etc/app/config.ini")
# Output includes blank line separators between sections

Example: Complete Configuration Update

# Load configuration
config = ini_read("/etc/myapp/config.ini")

# Add new logging section
log_meta["type"] = "metadata"
log_meta["value"] = map(.section_order 4)

log_entry1["type"] = "data"
log_entry1["key"] = "level"
log_entry1["value"] = "INFO"

log_entry2["type"] = "data"
log_entry2["key"] = "file"
log_entry2["value"] = "/var/log/myapp.log"

logging_section = [log_meta, log_entry1, log_entry2]
config["logging"] = logging_section

# Update database section
config["database"]["port"] = 5432

# Remove deprecated section
config = ini_delete_section(config, "legacy")

# Write updated configuration (preserves formatting and adds blank lines)
config.ini_write("/etc/myapp/config.ini")

See eg/initest for a complete example of INI manipulation.


Part XI — Sysadmin Cookbook

39. CLI data ingestion

Use table() to turn columnar CLI output into structured data, avoiding fragile string slicing.

t = table(| "df -h", map(.parse_only true))
println t.pp

40. Disk and filesystem checks

t = disk_usage()
bad = t ?> `#.usage_percent > 90`
foreach r in bad
    println r.path, r.mounted_path, r.usage
endfor

41. Process and service inspection

Use system/process library calls where available; otherwise, ingest CLI output via table() where possible and operate structurally.

41.1 Process Monitoring

# Get process list from /proc filesystem
proc_dirs = dir("/proc") ?> `#.name ~ "^[0-9]+$"` -> `#.name`
println "Found processes:", len(proc_dirs)

# Read process information
if len(proc_dirs) > 0
    first_pid = proc_dirs[0]
    stat_file = "/proc/" + first_pid + "/stat"
    if is_file(stat_file)
        stat_content = $in stat_file
        parts = split(stat_content, " ")
        if len(parts) > 1
            println "PID:", first_pid, "Process:", parts[1]
        endif
    endif
endif

# Filter processes by criteria
test_pids = proc_dirs[0:10]  # First 10 processes
filtered_pids = test_pids ?> `int(#) > 100`
println "High PIDs:", filtered_pids

41.2 Service Status via CLI

# Parse service status using table()
service_output = ${systemctl list-units --type=service --state=running}
services = table(service_output, map(.parse_only true))

# Filter services by name
web_services = services ?> `#.0 ~ "nginx|apache|httpd"`
println "Web services:", web_services

# Check specific service status
nginx_status = ${systemctl is-active nginx}
if $st nginx_status == "active"
    println "Nginx is running"
else
    println "Nginx is not running"
endif

42. Network diagnostics

Za provides network helpers for common tasks (reachability, DNS, port checks). Prefer structured results over parsing external tool output.

42.1 Basic Network Testing

# Test connectivity using ping
ping_result = ${ping -c 1 8.8.8.8}
if ping_result ~ "1 received"
    println "Internet connectivity OK"
else
    println "Internet connectivity failed"
endif

# DNS resolution test
dns_result = ${nslookup google.com}
if dns_result ~ "Address:"
    println "DNS resolution working"
else
    println "DNS resolution failed"
endif

42.2 Port Checking

# Check if ports are open using netcat
def check_port(host, port)
    result = ${nc -z {host} {port} 2>&1}
    return result.len() == 0  # Empty output means port is open
end

# Test multiple ports
ports_to_check = [80, 443, 22, 3306]
for port in ports_to_check
    if check_port("localhost", port)
        println "Port", port, "is open"
    else
        println "Port", port, "is closed"
    endif
endfor

43. Parallel host probing

Use async fan-out and deterministic collection:

def check(h)
    return icmp_ping(h, 2)
end

var handles map
foreach h in hosts
    async handles check(h) h
endfor
res = await(ref handles, true)
println res.pp

44. Drift detection and set-based reasoning

Represent key sets as maps and use set operators/predicates:

changed = before ^ after
on changed.len > 0 do println changed.pp

Part XII - Logging

45. Logging Overview

Za provides a unified logging system designed for operational monitoring and debugging. The system supports both application logging and web access logging through a single, coherent infrastructure that handles background processing, rotation, and multiple output formats.

45.1 Logging Philosophy

The logging system is built around several key principles:

The basic logging configuration provides:

46. Logging Configuration

46.1 Format Control

Za supports both plain text and JSON logging formats:

# Enable JSON formatting for structured logs
logging json on
logging subject "WEBMON"

# Add custom fields to all JSON entries
logging json fields +service "web-monitor"
logging json fields +version "1.2.1"

# Use plain text for human-readable logs
logging json off

JSON logging provides structured data that’s easier to parse and analyze:

// Plain text output
2023-10-15 14:23:11 [WEBMON] Service started on 15 Oct 23 14:23:11 +0000

// JSON output
{"timestamp":"2023-10-15T14:23:11Z","level":"INFO","subject":"WEBMON","message":"Service started on 15 Oct 23 14:23:11 +0000","service":"web-monitor","version":"1.2.1"}

46.2 Web Access Logging

For scripts that use Za’s built-in web server, you can enable separate access logging:

# Enable web access logging
logging web enable

# Set custom access log location
logging accessfile "/var/log/za_access.log"

# Configure web-specific settings
logging web enable
log "Web server started on port: ", port

Web access logs capture HTTP requests, response codes, and client information with automatic status code categorization (3xx=WARNING, 4xx/5xx=ERROR).

46.3 Rotation and Resource Management

Configure log rotation to manage disk space:

# Rotate when files reach 10MB
logging rotate size 10485760

# Keep 5 rotated files
logging rotate count 5

# Set memory reserve for critical logs (1MB)
logging reserve 1048576

The rotation system automatically:

46.4 Performance and Monitoring

Monitor logging system performance:

# Check logging system status
logging status

# View detailed statistics
stats = logging_stats()
println "Queue usage: ", stats.queue_usage, "%"
println "Processed: ", stats.total_processed, " entries"

The background queue system provides:

47. Logging Architecture

Za’s logging system provides a comprehensive infrastructure for both application events and web access logging. The system is designed around non-blocking operations, unified processing, and graceful resource management to ensure reliable logging without impacting script performance.

47.1 Logging Statement Types

The logging system supports several categories of statements for different logging needs. Application Logging uses the primary log statement which writes to both log file and console by default, with support for level-specific logging.

Basic Control statements enable and disable logging, configure output paths, and control console echo behaviour.

Format Control allows switching between plain text and JSON formats, managing custom fields for structured logs.

Web Access Logging provides separate controls for HTTP request logging with configurable file locations and automatic status code categorization.

Advanced Features include subject prefixes, automatic error logging, queue management, rotation control, and memory reservation.

Application Logging:

log "message",x,y,z             # Primary logging statement
log level: "message",x,y,z      # Level-specific logging

Basic Control:

logging on [filepath]           # Enable main logging, optionally set log file path
logging off                     # Disable main logging
logging status                  # Display comprehensive logging configuration and statistics
logging quiet                   # Suppress console output from log statements
logging loud                    # Enable console output from log statements (default)

Format Control:

logging json on                     # Enable JSON format for all logs
logging json off                    # Use plain text format (default)
logging json fields +field value    # Add custom field to JSON logs
logging json fields -field          # Remove specific field from JSON logs
logging json fields -               # Clear all custom fields
logging json fields push            # Save current fields to stack
logging json fields pop             # Restore fields from stack

Web Access Logging:

logging web enable                  # Enable web access logging
logging web disable                 # Disable web access logging
logging accessfile <path>           # Set web access log file location (default: ./za_access.log)

Advanced Features:

logging subject <text>              # Set prefix for all log entries
logging error on/off                # Enable/disable automatic error logging
logging queue size <number>         # Set background processing queue size (default: 60)
logging rotate size <bytes>         # Set log rotation file size threshold
logging rotate count <number>       # Set number of rotated files to keep
logging reserve <bytes>             # Set emergency memory reserve for logging under pressure

47.2 Infrastructure Design

The logging architecture uses a unified architecture where both application code and web server code feed into a common background queue that processes entries through shared formatting and rotation pipelines.

Key Components include background queue processing with configurable size and overflow handling, dual destinations for main logs and web access logs, and format management supporting both plain text and JSON with custom field capabilities.

Format Management handles automatic timestamps and subject prefix handling while allowing custom field manipulation for structured logs. Log Rotation provides size-based rotation for both main and web access logs with configurable file count retention and automatic cleanup of old rotated files.

Memory Management includes an emergency memory reserve system and priority-based queue management that favours errors over normal logs under pressure. Error integration automatically logs Za interpreter errors with enhanced context and HTTP status code tracking for web access logs.

47.3 Performance Characteristics

The logging system is optimized for minimal performance impact through non-blocking I/O for all logging operations, ensuring script execution never waits on log writes.

The system implements memory-aware request dropping for web access logs under memory pressure, with automatic warnings when queue capacity is exceeded.

Statistics tracking provides comprehensive monitoring of logging system performance and health. Cross-platform path validation and security ensures safe file operations with appropriate permission checks and path sanitization across different operating systems.

This design ensures that logging operations provide comprehensive coverage of application events while maintaining high performance and reliability.


Part XIII - Testing

48. Testing Overview

Za provides a built-in testing framework designed for both development verification and operational validation. The testing system supports assertions, documentation integration, and flexible execution modes that make it suitable for everything from unit tests to operational checks.

48.1 Testing Philosophy

The testing framework follows these principles:

48.2 Test Execution Modes

# Run all tests
za -t script

# Run specific test groups
za -t -G "database" script

# Run with custom output file
za -t -o "test_results.txt" script

# Override group assertion failure action]
za -t -O "fail|continue" script

48.3 Test Structure

Tests use a simple structure:

test "test_name" GROUP "group_name" [ASSERT FAIL|CONTINUE]
    # Test setup code
    assert condition [, custom_message ]
    # Additional assertions
    doc "This test verifies that..."
    # Test execution code
endtest

The optional assertion mode controls test behaviour:

49. Test Blocks

49.1 Basic Test Structure

test "integer_addition" GROUP "math_basics"
    # Test basic arithmetic
    result = 2 + 3
    assert result == 5, "2 + 3 should equal 5"

    # Test with different values
    assert (10 + 15) == 25, "10 + 15 should equal 25"
    doc "Verifies basic integer addition operations"
endtest

49.2 Error Handling Tests

test "file_error_handling" GROUP "io_operations" ASSERT CONTINUE
    # Test file not found error
    try
        content = read_file("/nonexistent/file.txt")
    catch err
        println "error type : ",err.pp
    endtry

    # Test permission error handling
    try
        write_file("/root/protected.txt", "test")
    catch err
        println "error type : ",err.pp
    endtry

    doc "Tests file operation error detection and categorization"
endtest

49.3 Function Return Value Tests

test "function_returns" GROUP "function_validation"
    # Test multiple return values
    def compute_stats(a, b)
        sum = a + b
        diff = a - b
        return sum, diff
    end

    result_sum, result_diff = compute_stats(10, 3)
    assert result_sum == 13
    assert result_diff == 7

    # Test single return value unpacking
    values = compute_stats(5, 2)
    assert values == [7, 3]

    doc "Validates function return value handling"
endtest

49.4 Data Structure Tests

test "map_operations" GROUP "data_structures"
    # Test map creation and access
    config = map(.host "localhost", .port 5432, .ssl true)
    assert config.host == "localhost"
    assert config.port == 5432
    assert config.ssl == true

    # Test map as set operations
    set_a = map(.a 1, .b 2, .c 3)
    set_b = map(.b 2, .c 3, .d 4)

    intersection = set_a & set_b
    assert intersection.len == 2
    assert intersection.c == 2
    assert intersection.b == 2

    doc "Tests map literal syntax and set operations"
endtest

49.5 Integration and System Tests

test "system_integration" GROUP "integration"
    # Test system call integration
    result =| "echo 'test output'"
    assert result.okay
    assert result.out ~ "test output"

    # Test table parsing
    df_data = table(${echo 'Filesystem 1K-blocks Used Available Use% Mounted on'},
                   map(.parse_only true)
    )
    assert df_data.len == 1
    assert df_data[0].Filesystem == "Filesystem"

    doc "Validates integration with system commands and data parsing"
endtest

50. Test Behaviours

50.1 Test Organization

Group tests logically by functionality:

test "user_auth_valid" GROUP "authentication"
test "user_auth_invalid" GROUP "authentication"
test "user_permission_check" GROUP "authorization"
test "database_connection" GROUP "database"
test "database_query" GROUP "database"

This organization allows:

50.2 Error Handling in Tests

The ASSERT ERROR syntax handles function call failures gracefully:

test "robust_function_calls" GROUP "error_handling"
    # This continues execution even if connect() fails
    assert error connect("invalid_host")
    doc "Tests error handling with ASSERT ERROR syntax"
endtest

50.3 Documentation Integration

Use doc statements to provide test context:

test "complex_business_logic" GROUP "business_rules"
    # Setup complex scenario
    customer = create_test_customer()
    order = process_order(customer, test_items)

    # Document the test purpose
    doc "Verifies that order processing correctly applies business rules:

         1. Customer discount applied correctly
         2. Tax calculations accurate
         3. Inventory updated appropriately"

    # Assertions for each rule
    assert order.discount_applied
    assert order.tax_amount > 0
    assert inventory_updated(order.items)
endtest

Expanded DOC statement usage

The DOC statement is also used to generate HEREDOC content in both normal execution and test modes. Some example use cases below:

Basic DOC with VAR clause

doc var myvar "Hello World"
println myvar

DOC with GEN clause (default delimiter)

doc gen
These lines should be
captured in test mode.
Yes?

DOC with GEN and custom DELIM clause

doc gen delim TERMINAL
These [#2]delimited[#-] lines should
be captured in test mode.
TERMINAL

DOC with custom DELIM and VAR clauses

doc delim END var multiline
This is a multi-line
string with "quotes"
END
print multiline

DOC with GEN clause and variable interpolation

doc gen
These lines should be
captured in test mode.
abc value is {=abc}
Yes?

Key Features Demonstrated


Appendix A — Operator reference

Appendix B — Keywords Summary

if else endif for foreach endfor case is has contains or endcase while endwhile def end return try catch then endtry struct endstruct enum module use namespace test endtest assert doc async var pause | debug continue break exit print println log logging cls at pane input prompt on do

Appendix C — Built-in constants

true, false, nil, NaN

Appendix D — Standard Library Categories

Za’s standard library is implemented in the interpreter source as a set of built-in calls. These library calls do not require any module imports.

This appendix lists the calls by category. For each category, all function names are listed, followed by a short “commonly used” section.

This appendix intentionally does not repeat full per-function documentation, because Za can generate function reference pages automatically and the REPL supports help and func(...) lookups.

array

Functions (23):

argmax, argmin, concatenate, det, det_big, find, flatten, identity, inverse, inverse_big, mean, median, ones, prod, rank, reshape, squeeze, stack, std, trace, variance, where, zeros

Commonly used (from examples/tests):

conversion

Functions (35):

as_bigf, as_bigi, as_bool, as_float, as_int, as_int64, as_string, as_uint, asc, base64d, base64e, btoi, byte, char, dtoo, explain, f2n, is_number, itob, json_decode, json_format, json_query, kind, m2s, maxfloat, maxint, maxuint, md2ansi, otod, pp, read_struct, s2m, table, to_typed, write_struct

Commonly used (from examples/tests):

cron

Functions (4):

cron_next, cron_parse, cron_validate, quartz_to_cron

Commonly used (from examples/tests):

date

Functions (17):

date, date_human, epoch_nano_time, epoch_time, format_date, format_time, now, time_diff, time_dom, time_dow, time_hours, time_minutes, time_month, time_nanos, time_seconds, time_year, time_zone, time_zone_offset

Commonly used (from examples/tests):

db

Functions (3):

db_close, db_init, db_query

Commonly used (from examples/tests):

error

Functions (15):

error_call_chain, error_call_stack, error_default_handler, error_emergency_exit, error_extend, error_filename, error_global_variables, error_local_variables, error_message, error_source_context, error_source_line_numbers, error_source_location, error_style, log_exception, log_exception_with_stack

Commonly used (from examples/tests):

file (unix)

Functions (17):

fclose, feof, fflush, file_mode, file_size, flock, fopen, fread, fseek, ftell, fwrite, is_dir, is_file, perms, read_file, stat, write_file

Commonly used (from examples/tests):

file (windows)

Functions (17):

fclose, feof, fflush, file_mode, file_size, flock, fopen, fread, fseek, ftell, fwrite, is_dir, is_file, perms, read_file, stat, write_file

Commonly used (from examples/tests):

html

Functions (22):

wa, wbody, wdiv, wh1, wh2, wh3, wh4, wh5, whead, wimg, wli, wlink, wol, wp, wpage, wtable, wtbody, wtd, wth, wthead, wtr, wul

Commonly used (from examples/tests):

image

Functions (22):

svg_circle, svg_def, svg_def_end, svg_desc, svg_ellipse, svg_end, svg_grid, svg_group, svg_group_end, svg_image, svg_line, svg_link, svg_link_end, svg_plot, svg_polygon, svg_polyline, svg_rect, svg_roundrect, svg_square, svg_start, svg_text, svg_title

Commonly used (from examples/tests):

internal

Functions (105):

ansi, argc, argv, array_colours, array_format, ast, await, bash_versinfo, bash_version, capture_shell, clear_line, clktck, cmd_version, conclear, conread, conset, conwrite, coproc, cursoroff, cursoron, cursorx, difference, dinfo, dump, dup, echo, enum_all, enum_names, eval, exception_strictness, exec, execpath, expect, exreg, feed, format_stack_trace, func_categories, func_descriptions, func_inputs, func_outputs, funcref, funcs, gdump, get_col, get_cores, get_mem, get_row, has_colour, has_shell, has_term, home, hostname, interpol, interpolate, intersect, is_disjoint, is_subset, is_superset, key, keypress, lang, last, last_err, len, local, log_queue_status, logging_stats, mdump, merge, os, pane_c, pane_h, pane_r, pane_w, panic, permit, pid, powershell_version, ppid, release_id, release_name, release_version, rlen, set_depth, shell_pid, sizeof, suppress_prompt, symmetric_difference, system, sysvar, term, term_h, term_w, thisfunc, thisref, tokens, trap, unmap, user, utf8supported, varbind, wininfo, winterm, zainfo, zsh_version

Commonly used (from examples/tests):

list

Functions (35):

alltrue, anytrue, append, append_to, avg, col, concat, empty, eqlen, esplit, fieldsort, head, insert, list_bigf, list_bigi, list_bool, list_fill, list_float, list_int, list_int64, list_string, max, min, msplit, peek, pop, push_front, remove, scan_left, sort, ssort, sum, tail, uniq, zip

Commonly used (from examples/tests):

math

Functions (38):

abs, acos, acosh, asin, asinh, atan, atanh, cos, cosh, deg2rad, dot, e, floor, ibase, ln, ln10, ln2, log10, log2, logn, matmul, numcomma, phi, pi, pow, prec, rad2deg, rand, randf, round, seed, sin, sinh, tan, tanh, transpose, ubin8, uhex32

Commonly used (from examples/tests):

network

Functions (31):

dns_resolve, has_privileges, http_benchmark, http_headers, icmp_ping, icmp_traceroute, net_interfaces_detailed, netstat, netstat_established, netstat_interface, netstat_listen, netstat_process, netstat_protocol, netstat_protocol_info, netstat_protocols, network_stats, open_files, port_scan, ssl_cert_install_help, ssl_cert_validate, tcp_available, tcp_client, tcp_close, tcp_ping, tcp_receive, tcp_send, tcp_server, tcp_server_accept, tcp_server_stop, tcp_traceroute, traceroute

Commonly used (from examples/tests):

notify

Functions (7):

ev_event, ev_exists, ev_mask, ev_watch, ev_watch_add, ev_watch_close, ev_watch_remove

Commonly used (from examples/tests):

os

Functions (37):

can_read, can_write, cd, chroot, copy, cwd, delete, dir, env, fileabs, filebase, get_env, glob, group_add, group_del, group_info, group_list, group_membership, group_mod, groupname, is_device, is_pipe, is_setgid, is_setuid, is_socket, is_sticky, is_symlink, parent, rename, set_env, umask, user_add, user_del, user_info, user_list, user_mod, username

Commonly used (from examples/tests):

package

Functions (5):

install, is_installed, service, uninstall, vcmp

Commonly used (from examples/tests):

pcre

Functions (3):

reg_filter, reg_match, reg_replace

Commonly used (from examples/tests):

smtp

Functions (13):

email_add_header, email_base64_decode, email_base64_encode, email_extract_addresses, email_get_attachments, email_get_body, email_parse_headers, email_process_template, email_remove_header, email_validate, smtp_send, smtp_send_with_attachments, smtp_send_with_auth

Commonly used (from examples/tests):

string

Functions (56):

addansi, bg256, bgrgb, ccformat, clean, collapse, count, fg256, fgrgb, field, fields, filter, format, get_value, grep, gsub, has_end, has_start, inset, is_utf8, join, keys, levdist, line_add, line_add_after, line_add_before, line_delete, line_filter, line_head, line_match, line_replace, line_tail, lines, literal, log_sanitise, lower, match, next_match, pad, pos, replace, reverse, rvalid, sanitisation, sgrep, split, stripansi, stripcc, stripquotes, strpos, substr, tr, trim, upper, values, wrap, wrap_text

Commonly used (from examples/tests):

sum

Functions (5):

md5sum, s3sum, sha1sum, sha224sum, sha256sum

Commonly used: (no occurrences found in eg/ or za_tests/ for this category in the uploaded tree)

system

Functions (23):

cpu_info, debug_cpu_files, dio, disk_usage, gw_address, gw_info, gw_interface, iodiff, mem_info, mount_info, net_devices, nio, ps_info, ps_list, ps_map, ps_tree, resource_usage, sys_load, sys_resources, top_cpu, top_dio, top_mem, top_nio

Commonly used (from examples/tests):

tui

Functions (16):

editor, tui, tui_box, tui_clear, tui_input, tui_menu, selector, tui_new, tui_new_style, tui_pager, tui_progress, tui_progress_reset, tui_radio, tui_screen, tui_table, tui_template, tui_text

Commonly used (from examples/tests):

web

Functions (28):

download, html_escape, html_unescape, net_interfaces, web_cache_cleanup_interval, web_cache_enable, web_cache_max_age, web_cache_max_memory, web_cache_max_size, web_cache_purge, web_cache_stats, web_custom, web_display, web_download, web_get, web_gzip_enable, web_head, web_max_clients, web_post, web_raw_send, web_serve_decode, web_serve_log, web_serve_log_throttle, web_serve_path, web_serve_start, web_serve_stop, web_serve_up, web_template

Commonly used (from examples/tests):

yaml

Functions (5):

yaml_delete, yaml_get, yaml_marshal, yaml_parse, yaml_set

Commonly used (from examples/tests):

zip

Functions (7):

zip_add, zip_create, zip_create_from_dir, zip_extract, zip_extract_file, zip_list, zip_remove

Commonly used (from examples/tests):

Appendix E — Worked Example Script

This appendix is an annotated walkthrough of the shipped example script eg/mon. All claims below are grounded in the script itself, with line references.

E.1 What eg/mon is

The file identifies itself as a test/diagnostic script: it describes itself as a “Test script for za” and says it displays “key system resource information” in a summary view (lines 3–9).

E.2 Core drawing primitives

Before it gathers any system information, the script defines a small set of terminal-drawing helpers.

clear(lstart,lend,column) clears a range of lines by calling clear_line in a counted loop (lines 15–19).

header(t) draws a coloured underline across the pane width, then prints a title at the top (lines 21–28). You can see it iterating from 0 to pane_w()-2 and printing a coloured underscore (lines 23–25).

Vertical and horizontal bars

The script implements its own bar widgets using block characters.

E.3 Small utility helpers

Several short helpers support later panes:

E.4 Built-in unit test embedded in the script

eg/mon includes a small unit test section:

E.5 Environment pane (showEnv)

showEnv() selects the envs pane, redraws its line colour, prints a header, clears a region, and prints system facts such as hostname, user, OS, locale, and distribution information (lines 131–149).

A small OS-specific clause is shown via case os() where it prints the bash version only for Linux (lines 145–148).

E.6 Files/inodes pane (showFiles)

showFiles() is guarded to avoid Windows terminal mode: it runs only when !winterm() (lines 155–185).

It reads Linux kernel counters directly from /proc using $in (lines 160–162), then parses them using string helpers such as field, tr, and as_float (lines 171–183).

If either /proc/sys/fs/file-nr or /proc/sys/fs/inode-nr could not be read, it returns early (line 163).

E.7 Memory pane (showMem)

showMem() shows two important things:

First, it gathers memory information from built-in calls: it calls mem_info() (line 216) and sys_resources() (line 252), then pulls named fields such as MemoryTotal, MemoryFree, MemoryCached, SwapFree, and SwapTotal (lines 253–259).

Second, it optionally computes “slab” usage details when it has privileges (access) (lines 210–212, 290–313). It iterates mem_detailed.Slab and derives MB sizes from object counts and sizes (lines 224–228), then sorts via fieldsort (line 237).

For display, it uses mdisplay(...) which draws a bar and prints a compact size using smallprint (lines 189–199, 270–281).

It also displays Za’s own memory usage via get_mem().alloc and get_mem().system (lines 283–287).

E.8 Process pane (showProcs)

showProcs(ct, uptime) calls ps_list(map(.include_cmdline true)) to obtain process info (line 324).

It builds proc_list as a map keyed by PID, storing a small array of derived metrics including computed CPU percentages (lines 326–352).

To present the “top” entries, it serialises rows into a string (shellout), sorts with fieldsort(..., "n", true) and takes lines (lines(":17")) before applying uniq (lines 358–365).

The display loop uses fields(p," ") and the resulting F[...] fields for formatted printing. It supports filtering with a regex against proc_filter (lines 374–387).

E.9 CPU pane (showCpu)

showCpu(...) pulls per-core usage from cpu_info().Usage["cores"] (line 424) and uses prev/diff maps to compute deltas over time (lines 427–451).

If the script is still in its initial warm-up period (sample_start_in-->0), it prints a message and returns the updated counter (lines 453–457).

Core names are sorted alphanumerically (cpuinfo.keys.sort(map(.alphanumeric true)), line 460).

The pane can show: - detailed per-activity counters (lines 472–481), - a coloured bar row built from repeated activity glyphs (lines 483–497), - and an optional totals column (lines 499–506).

E.10 Disk pane (showHdd)

showHdd() uses disk_usage() directly (line 521). It then filters out unwanted devices using regex conditions under a case os() block and continues early for skipped entries (lines 534–542).

It assembles a sortable list of anonymous structs containing usage_pct from the usage_percent field and the mounted path (lines 543–552).

It sorts with ssort(sorted_disks, "usage_pct", false) (line 555) and prints a limited number of rows (lines 557–582).

Sizes are formatted using the local hobbitsize(...) helper (lines 399–404) and strings are truncated with shorten(...) (lines 126–129, 576–579).

E.11 Network pane (showNet)

showNet(...) selects the network pane, prints the chosen interface and its IP, then scans nio() for that interface (lines 586–598).

It keeps previous byte counters (prev_rbytes, prev_tbytes) and maintains history lists rblist and tblist by shifting and appending new deltas (lines 610–619).

The charts are plotted by mapping the history list through an expression string using -> and then converting to integers with .list_int (lines 628–629). The expression clamps values with [0f:100f] inside the string.

Finally it prints averaged RX/TX rates using humansize(...) scaled by the sampling timeout (lines 635–641).

E.12 Pane layout (redef_layout)

redef_layout(cpu_count,pfilter) defines the pane grid using repeated pane define calls (lines 647–659).

It also initialises the network history lists to a fixed length (net_sample_count = 40, lines 660–668) and draws an origin line for the network chart using a UTF‑8 glyph when supported (lines 672–681).

E.13 Main loop and key handling

The script exits early if there is no output channel (term_h()==-1, line 690).

It sets up timing (key_timeout=1000, lines 695–697), determines the gateway interface (gw_interface(), line 701), and detects privilege level (access=has_privileges(), line 716).

It computes CPU tick rate via clktck() and aborts if unavailable (lines 706–709).

Panes are created by calling redef_layout(cpu_count,pfilter) after discovering core count (get_cores(), lines 721–728).

Inside the main while !quit loop (line 752), it redraws on window changes, captures uptime (sys_resources().Uptime, line 765), and calls the pane renderers (lines 768–774).

The bottom status line prints a human date (date_human(...)) and frame time based on epoch_nano_time() deltas (lines 783–795).

User input is handled with keypress(key_timeout) and a case char(k) dispatch (lines 798–834). It supports changing interface (i), process filter (f), timeout (t), toggling CPU sections (D, B, T), help (h), quit (q), and redraw on Ctrl‑L (k==12) (lines 800–834).

On exit it restores terminal state and turns the cursor back on (lines 841–844).

Appendix F — C Library Imports (FFI) (v1.2.2+)

F.1 Introduction and Use Cases

Za’s Foreign Function Interface (FFI) is an experimental feature that enables direct calling of C library functions from Za scripts. This allows you to leverage the vast ecosystem of existing C libraries for specialized tasks.

When to use FFI:

When NOT to use FFI:

Platform support: FFI is fully supported on Unix/Linux systems via libffi. Windows support is limited. The feature requires CGO and is disabled in no-FFI builds.

F.2 Loading Libraries with MODULE

C libraries are loaded dynamically using the MODULE keyword with an alias:

MODULE "/path/to/library.so" AS alias_name [AUTO]
USE +alias_name

The MODULE statement loads the shared library and makes its symbols available. The USE +alias statement appends the alias to the namespace lookup chain. (use USE ^alias to push the namespace to the head of the search list instead if required).

Common system libraries:

# Math library
module "/usr/lib/libm.so.6" as m
use +m

# C standard library
module "/usr/lib/libc.so.6" as c
use +c

# libcurl for HTTP operations
module "/usr/lib/libcurl.so.4" as curl
use +curl

# ncurses for terminal UI
module "/usr/lib/libncursesw.so.6" as nc
use +nc

# GLib utility library
module "/usr/lib/libglib-2.0.so.0" as glib
use +glib

# JSON-C for JSON parsing
module "/usr/lib/libjson-c.so.5" as json
use +json

# zlib compression
module "/usr/lib/libz.so.1" as z
use +z

# libgd for graphics
module "/usr/lib/libgd.so.3" as gd
use +gd

Finding library paths: System libraries are typically in /usr/lib, /usr/lib64, or /lib. Use ldconfig -p or find /usr/lib -name "lib*.so*" to locate specific libraries.

F.3 Automatic Header Parsing with AUTO

The AUTO clause enables automatic discovery and parsing of C header files to extract constants, enums, and function signatures, eliminating the need to manually declare #define values and LIB function signatures.

Performance Note: AUTO imports can be slow on large header file sets (10-30+ seconds for complex libraries like OpenGL or X11). A progress bar displays import progress. Use explicit header paths and specify only needed headers to minimize parsing time. For scripts where startup speed is critical, manual LIB declarations are much faster than AUTO, though you lose automatic constant extraction, function discovery, and typedef resolution—requiring you to manually declare each function signature and constant needed. Set ZA_NO_PROGRESS=1 to disable progress display.

F.3.1 Basic Usage

Auto-discovery (searches standard include paths):

module "libm.so.6" as m auto
use +m

# M_PI, M_E, and other math constants now available
println M_PI  # 3.141592653589793

Explicit header path:

module "libc.so.6" as c auto "/usr/include/limits.h"
use +c

println INT_MAX  # 2147483647

Multiple headers:

module "libgl.so" as gl auto "gl.h" "glext.h"
use +gl

F.3.2 What Gets Parsed

The AUTO clause parses C headers and extracts:

  1. Integer constants from #define:

  2. Float constants from #define:

  3. String constants from #define:

  4. Character literals from #define (converted to numeric values):

  5. Enum definitions (single-line or multiline):

    enum Status {
        OK = 0,
        ERROR = -1,
        PENDING = 100
    };
  6. Function signatures (auto-generates LIB declarations):

    size_t strlen(const char *s);
    void *malloc(size_t size);
    int printf(const char *format, ...);
    BGD_DECLARE(gdImagePtr) gdImageCreateTrueColor(int sx, int sy);
    __declspec(dllexport) void exported_function(void);
  7. Constants referencing earlier constants:

    #define BASE_SIZE 1024
    #define BUFFER_SIZE (BASE_SIZE * 2)      // Works! = 2048
    #define LARGE_BUFFER (BUFFER_SIZE * 4)   // Works! = 8192
  8. Typedef declarations:

    typedef unsigned int uint32_t;
    typedef struct Point { int x, y; } Point;
    typedef Point* PointPtr;
  9. Preprocessor conditionals (platform-aware parsing):

    #ifdef __linux__
      #define BUFFER_SIZE 2048
      int linux_only_func(void);
    #endif
    
    #ifndef __WINDOWS__
      #define UNIX_FLAG 1
    #endif
    
    #if defined(__LP64__) && VERSION > 1
      typedef unsigned long size_t;  // 64-bit
    #elif VERSION == 1
      typedef unsigned int size_t;   // 32-bit legacy
    #else
      typedef unsigned short size_t;  // 16-bit
    #endif
  10. Struct definitions (automatically registered as Za types):

   typedef struct {
       int x;
       int y;
   } Point;

   typedef struct {
       uint8_t rgb[3];
       char name[32];
   } Color;
  1. Numeric type coercion (comparison operators):

  2. wchar_t support (platform-aware implicit type alias):

    // C header
    int get_wchar_size(void);
    wchar_t wchar_echo(wchar_t c);
    void wchar_fill_array(wchar_t *arr, int len);
    # Za code - wchar_t automatically mapped
    module "./libtest.so" as tw auto "./test.h"
    use +tw
    
    size = get_wchar_size()    # Returns 4 on Linux, 2 on Windows
    result = wchar_echo(65)    # wchar_t as value → uint16/uint32
    wchar_fill_array(ptr, 5)   # wchar_t* as pointer → CPointer

    How it works:

    Compatible with typedefs:

    typedef wchar_t WCHAR;
    typedef WCHAR* LPWSTR;

    Already handled by existing typedef resolution (item 8 above).

    Use cases:

  3. Function pointer typedefs (v1.2.2+):

typedef int (*compare_t)(const void*, const void*);
typedef void (*callback_t)(int signal);
typedef double (*math_func_t)(double);
  1. Function pointer fields in structs and unions (v1.2.2+):
typedef struct {
    int (*callback)(void*);
    void (*cleanup)(void);
} Handler;

typedef union {
    int (*compare)(int, int);
    void (*action)(void);
} FuncUnion;

Not parsed (automatically skipped - not errors):

F.3.3 Accessing Constants

Constants are accessible via the USE chain or namespace qualification:

module "libpng.so" as png auto
use +png

# Via USE chain (unqualified)
color_type = PNG_COLOR_TYPE_RGB

# Via namespace (qualified)
alpha = png::PNG_COLOR_TYPE_RGBA

F.3.4 Header Auto-Discovery

When no explicit path is provided, AUTO searches for headers in standard locations:

Search paths (in order):

  1. /usr/include/<header>
  2. /usr/local/include/<header>
  3. /usr/include/<arch>-linux-gnu/<header> (e.g., x86_64-linux-gnu)
  4. /usr/include/<libname>/<header> (e.g., curl/curl.h)

Header name derivation:

F.3.5 Checking if Constants Exist

Use the defined() function to check if a constant is available:

module "libm.so.6" as m auto
use +m

if defined("M_PI")
    println "Pi is available: " + as_string(M_PI)
else
    println "M_PI not found"
endif

F.3.6 Auto-Discovered Function Signatures

AUTO automatically parses function prototypes from headers and generates LIB declarations, eliminating manual function signature declarations.

Without AUTO:

module "libc.so.6" as c
use +c

# Must manually declare every function
lib c::strlen(s:string) -> int64
lib c::strcmp(s1:string, s2:string) -> int
lib c::malloc(size:int64) -> pointer
lib c::free(ptr:pointer) -> void

len = strlen("hello")  # Requires manual LIB declaration above

With AUTO:

module "libgd.so" as gd auto
use +gd

# Functions automatically discovered from headers - no LIB needed!
image = gdImageCreateTrueColor(400, 300)  # Works automatically
if not c_ptr_is_null(image)
    gdImageDestroy(image)                  # Works automatically
endif

(Note: Explicit header paths may be needed for libraries where the header filename doesn’t match the alias. See F.3.4 for details.)

What gets auto-discovered:

What gets skipped:

Override mechanism:

Explicit LIB declarations override auto-discovered signatures:

module "libc.so.6" as c auto
use +c

# AUTO discovers strlen, but we want to override the return type
lib c::strlen(s:string) -> int   # Override: use int instead of int64

len = strlen("hello")  # Uses our override signature

F.3.7 Declaration Macros

Many C libraries use declaration macros to wrap function signatures with platform-specific attributes (calling conventions, visibility, export decorators, etc.). Za’s AUTO clause automatically expands these macros to correctly identify the actual return types.

Examples of declaration macros:

Library Macro Definition Used For
libgd BGD_DECLARE(type) Expands to platform-specific export qualifiers Export control on different platforms
Windows __declspec(dllexport) Windows DLL export decorator DLL symbol visibility
GCC/POSIX __attribute__((visibility(...))) Visibility control Symbol visibility management
POSIX WINAPI, __stdcall Calling conventions Function calling convention specifiers

Problem without macro expansion:

// Header file (libgd.h)
#define BGD_DECLARE(type) BGD_EXPORT_DATA_PROT type BGD_STDCALL
BGD_DECLARE(gdImagePtr) gdImageCreateTrueColor(int sx, int sy);

Without expansion, Za would see this as:

gdImageCreateTrueColor(...) -> void    # WRONG: No visible return type!

Solution with AUTO (macro expansion enabled):

Za automatically expands the BGD_DECLARE macro, then correctly parses:

gdImageCreateTrueColor(...) -> void*<gdImagePtr*>   # CORRECT: Pointer return type identified

How it works:

  1. AUTO scans for macro calls in function declarations (e.g., MACRO_NAME(...) function_name(...))
  2. Looks up the macro definition from the parsed header
  3. Substitutes parameters (e.g., typegdImagePtr)
  4. Recursively expands any nested macros
  5. Parses the expanded declaration to extract the real return type

Real-world example:

module "libgd.so" as gd auto
use +gd

# AUTO automatically handles BGD_DECLARE(gdImagePtr) expansion
# No manual LIB declaration needed - return type is correctly identified as pointer
image = gdImageCreateTrueColor(400, 300)

if c_ptr_is_null(image)
    panic("Failed to create image")
endif

Limitations:

F.3.8 Error Handling

If AUTO cannot find or parse headers:

Warning: failed to parse headers for png: header file not found
  Searched: /usr/include/png.h, /usr/local/include/png.h, ...
  Hint: Specify explicit path: module "libpng.so" as png auto "/path/to/png.h"

F.3.9 Complete Example

# Load libgd with automatic header parsing
# Header auto-discovery finds /usr/include/gd.h
module "libgd.so" as gd auto
use +gd

# No LIB declarations needed - AUTO discovers function signatures!
# Constants and functions automatically available from gd.h

def create_image(width:int, height:int)
    # gdImageCreateTrueColor is auto-discovered function (no LIB needed)
    # Return type correctly identified as gdImagePtr despite BGD_DECLARE wrapper
    image = gdImageCreateTrueColor(width, height)

    if c_ptr_is_null(image)
        panic("Failed to create image")
    endif

    return image
end

def cleanup_image(image)
    # gdImageDestroy is also auto-discovered
    gdImageDestroy(image)
end

# Usage
img = create_image(640, 480)
println "Image created: 640x480"
cleanup_image(img)
println "Image destroyed"

F.3.10 Benefits Over Manual Constants and LIB Declarations

Before AUTO (manual constants and LIB declarations):

module "libgd.so" as gd
use +gd

# Must manually declare every constant
GD_TRUE = 1
GD_FALSE = 0

# Must manually declare every function signature
lib gd::gdImageCreateTrueColor(sx:int, sy:int) -> pointer
lib gd::gdImageDestroy(im:pointer) -> void
lib gd::gdImageColorAllocate(im:pointer, r:int, g:int, b:int) -> int
lib gd::gdImageFill(im:pointer, x:int, y:int, col:int) -> void
# ... dozens more ...

After AUTO (everything automatic):

module "libgd.so" as gd auto
use +gd

# All constants and functions automatically available!
# GD_TRUE, GD_FALSE, etc. auto-discovered
# gdImageCreateTrueColor, gdImageDestroy, etc. auto-discovered
image = gdImageCreateTrueColor(400, 300)
gdImageDestroy(image)
# ... etc - no manual declarations needed!

Summary of benefits:

F.4 Declaring Function Signatures with LIB

C functions require explicit type declarations using the LIB keyword. This tells Za how to marshal arguments and return values.

Syntax:

lib namespace::function_name(param1:type1, param2:type2, ...) -> return_type

Supported types:

Integer types:

Floating-point types:

Other types:

Array types (v1.2.2+):

Examples covering all type combinations:

# Math functions (double parameters and returns)
lib m::sqrt(x:double) -> double
lib m::pow(x:double, y:double) -> double
lib m::sin(x:double) -> double
lib m::floor(x:double) -> double

# String functions (string and pointer types)
lib c::strlen(s:string) -> pointer         # Returns size_t as pointer
lib c::strcmp(s1:string, s2:string) -> int
lib c::strcpy(dest:pointer, src:string) -> pointer
lib c::strcat(dest:pointer, src:string) -> pointer

# Memory management (pointer and int)
lib c::malloc(size:int) -> pointer
lib c::calloc(nmemb:int, size:int) -> pointer
lib c::free(ptr:pointer) -> void           # void return (no value)
lib c::memcpy(dest:pointer, src:pointer, n:int) -> pointer

# File operations (string, pointer returns)
lib c::fopen(filename:string, mode:string) -> pointer
lib c::fclose(stream:pointer) -> int
lib c::fwrite(ptr:pointer, size:int, nmemb:int, stream:pointer) -> int

# Environment access (string return)
lib c::getenv(name:string) -> string

# GLib functions (various return types)
lib glib::g_strdup(str:string) -> string
lib glib::g_malloc(n_bytes:int) -> pointer
lib glib::g_free(mem:pointer) -> void
lib glib::g_random_int() -> int
lib glib::g_random_double() -> double
lib glib::g_strcmp0(str1:string, str2:string) -> int

# JSON-C functions (pointer-heavy)
lib json::json_object_new_object() -> pointer
lib json::json_object_new_string(s:string) -> pointer
lib json::json_object_new_int(i:int) -> pointer
lib json::json_object_object_add(obj:pointer, key:string, val:pointer) -> void
lib json::json_object_to_json_string(obj:pointer) -> string
lib json::json_object_put(obj:pointer) -> void

# Graphics (libgd)
lib gd::gdImageCreate(width:int, height:int) -> pointer
lib gd::gdImageColorAllocate(im:pointer, r:int, g:int, b:int) -> int
lib gd::gdImageDestroy(im:pointer) -> void

# Array parameters (v1.2.2+) - automatic conversion to C pointers
lib arr::sum_int_array(data:[]int, len:int) -> int
lib arr::average_float_array(values:[]float64, len:int) -> double
lib arr::double_int_array(data:mut []int, len:int) -> void
lib arr::fill_sequence(arr:mut []int, len:int, start:int) -> void

Variadic functions (experimental):

# Functions with variable arguments
lib c::printf(fmt:string, ...args) -> int

Optional return types: Functions that perform side effects without returning meaningful values use -> void.

F.4.4 Arrays as Function Parameters (v1.2.2+)

Za automatically converts Za arrays to C pointers when passing them to C functions. This eliminates the need for verbose manual memory allocation patterns like c_alloc() and c_set_byte().

Basic Usage

Read-only arrays:

Za arrays are automatically converted to C pointers and passed to C functions without any extra work:

module "libarray.so" as arr auto "array.h"
use +arr

# Arrays are automatically converted to C pointers
data = [1, 2, 3, 4, 5]
sum = arr::sum_int_array(data, 5)  # Automatically converts to int*
println "Sum: {sum}"  # 15

# Float arrays
values = [1.5, 2.5, 3.5]
avg = arr::average_float_array(values, 3)  # Automatically converts to double*
println "Average: {avg}"  # 2.5

Mutable arrays (C function modifies in place):

Use the mut keyword to allow C functions to modify array contents:

# Mutable array - C function can modify values
buffer = [10, 20, 30]
arr::double_int_array(mut buffer, 3)  # Values are doubled in place

println buffer[0]  # 20
println buffer[1]  # 40
println buffer[2]  # 60

Supported Array Types

All numeric types are supported:

# Create arrays of any supported type
int_array = [1, 2, 3, 4, 5]
float_array = [1.5, 2.5, 3.5]
byte_array = [0xFF, 0xFE, 0xFD]
uint64_array = [1000000000000, 2000000000000]

# Pass to C functions
sum = lib::process_ints(int_array, 5)
doubled = lib::scale_floats(float_array, 3, 2.0)
checksum = lib::compute_checksum(byte_array, 3)

# String arrays are also supported
args = ["program", "--option", "value"]
count = lib::count_args(args, 3)

How It Works

  1. Memory allocation: Za automatically allocates C memory for the array
  2. Data copying: Array elements are copied to the C buffer
  3. Pointer passing: The pointer is passed to the C function
  4. Memory cleanup: Allocated memory is automatically freed after the call
  5. Mutable updates: If mut is used, modifications are copied back to the Za array

Example: Building pixel data

Before (verbose):

row = c_alloc(ROW_SIZE)
for x = 0 to WIDTH - 1
    offset = x * 3
    c_set_byte(row, offset, r)
    c_set_byte(row, offset + 1, g)
    c_set_byte(row, offset + 2, b)
endfor
png_write_row(png_ptr, row)
c_free(row)

After (automatic):

row = []
for x = 0 to WIDTH - 1
    row = append(row, r)
    row = append(row, g)
    row = append(row, b)
endfor
png_write_row(png_ptr, row)  # Automatic allocation and cleanup!

Edge Cases

Empty arrays: Handled correctly (passed as null pointers)

empty = []
result = lib::sum_array(empty, 0)  # C receives NULL pointer and count=0

Large arrays: No practical size limit (limited only by available memory)

large = [1..100000]  # Create array with 100,000 elements
result = lib::process(large, 100000)  # Automatically allocated and managed

Benefits

F.4.5 Passing Structs by Reference with mut

The mut keyword allows passing VAR-declared structs by reference to C functions, enabling C code to modify struct fields in place. This is the preferred method for “out parameters” with AUTO-registered structs (available in v1.2.2+).

Basic Usage

module "libexample.so" as ex auto "example.h"
use +ex

# Declare struct variable (type from AUTO)
var result ex::ResultStruct

# Initialize fields
result.status = 0
result.value = 0

# Pass by reference - C function modifies fields
ex::compute(10, 20, mut result)

# Access modified fields directly
println "Status:", result.status
println "Value:", result.value

How It Works

  1. Declaration: var name namespace::Type creates a struct variable from AUTO-parsed C header
  2. Field Access: Use dot notation (e.g., result.status, result.value)
  3. Pass by Reference: Use mut prefix when calling C functions that take pointer parameters
  4. Automatic Updates: C modifications are immediately visible in Za without unmarshaling

Comparison with Manual Marshaling

With mut (recommended for AUTO structs):

var color x11::XColor
color.red = 255 * 256
color.green = 128 * 256
color.blue = 0
XAllocColor(display, colormap, mut color)
# color.pixel is now set by C

Manual marshaling (for non-AUTO structs):

color_ptr = c_alloc_struct("XColor")
# ... set fields via marshaling ...
XAllocColor(display, colormap, color_ptr)
color = c_unmarshal_struct(color_ptr, "XColor")
c_free_struct(color_ptr)

When to Use mut

Use mut when:

Use manual marshaling when:

X11 Color Allocation Example

module "/usr/lib/libX11.so.6" as x11 auto "/usr/include/X11/Xlib.h"
use +x11

def alloc_colour(r, g, b)
    # Declare XColor struct from AUTO header
    var color x11::XColor

    # Set RGB values (X11 uses 16-bit: 0-65535)
    color.pixel = 0
    color.red = r * 256
    color.green = g * 256
    color.blue = b * 256
    color.flags = 0
    color.pad = 0

    # XAllocColor modifies color.pixel
    status = XAllocColor(display, colormap, mut color)

    if status == 0
        return 0  # Allocation failed
    endif

    # Return the allocated pixel value
    return color.pixel
end

# Usage
red = alloc_colour(255, 0, 0)
green = alloc_colour(0, 255, 0)
blue = alloc_colour(0, 0, 255)

Requirements

Technical Details

When you use mut structname:

  1. Za passes a pointer to the Go struct’s memory
  2. C function writes directly to that memory
  3. No marshaling/unmarshaling overhead
  4. Changes are immediately visible in Za

This is more efficient than manual marshaling which requires:

F.4.6 Global Variable Fallback for mut Parameters (v1.2.2+)

When using the mut keyword to pass a variable to a C function, Za now checks both local and global scopes for the variable. If a local variable with the given name doesn’t exist, Za will look for a global variable with the same identifier.

How It Works

Variable scope resolution for mut parameters:

  1. First: Look for local variable in current function scope
  2. Second: Look for global variable with same identifier
  3. Error: If neither exists, report undefined variable

Local variables take precedence over globals, maintaining backward compatibility.

Basic Example

# Global variable
global_buffer = [10, 20, 30]

def modify_global()
    # Pass global variable to C function using mut
    arr::double_int_array(mut global_buffer, 3)
    # global_buffer is now [20, 40, 60]
    println global_buffer[0]  # 20
end

modify_global()

# Can also use global in main scope
arr::double_int_array(mut global_buffer, 3)  # Works here too

Local Variable Precedence

Local variables shadow globals:

global_data = [1, 2, 3]

def process()
    # Local variable shadows global
    var local_data = [10, 20, 30]

    # This modifies the local variable, not the global
    arr::double_int_array(mut local_data, 3)
    println local_data[0]     # 20
end

process()
println global_data[0]  # Still 1 (unchanged)

Thread Safety

Global variable updates via mut parameters are protected with locks to ensure thread-safe access in concurrent scenarios.

When to Use Global mut Parameters

Good use cases:

Avoid when:

F.4.7 Output Parameter Declaration: var name mut (v1.2.2+)

The var name mut syntax declares variables with unknown types that will be determined by the result of FFI calls. This is useful for receiving opaque pointers and complex types from C functions without pre-declaring their types.

Basic Usage

# Declare a variable with unknown type (will be determined by FFI call)
var buffer mut

# Pass to C function as output parameter - type is determined from result
arr::fill_sequence(mut buffer, 3, 100)

# Variable now has proper type (array in this case) from FFI call result
println buffer[0]  # 100
println buffer[1]  # 101
println buffer[2]  # 102

# After FFI assignment, variable can be used normally
buffer[0] = 200
println buffer[0]  # 200

When to Use var name mut

This syntax is specifically for declaring output parameters that will receive their type and initial value from C functions. Use it when:

  1. Opaque pointers from C - You receive a pointer to an unknown struct type:
var font mut
# FreeType function fills the font pointer - type unknown until call
arr::create_font(mut font, "/path/to/font.ttf")
# font now has proper type from C function result
  1. Complex C types - You need to receive struct/union types without pre-declaring them:
var result mut
# C function writes struct data to result parameter
my_lib::get_struct_data(mut result)
# result gains proper type automatically from C data
  1. When type is truly unknown - C function returns data with type determined at runtime:
var data mut
# C function determines what data structure to create
lib::fetch_data(mut data, some_config)
# data now typed based on what C returned

How Type Determination Works

When a mut-declared variable is used as an output parameter (mut varname):

  1. Before FFI call: Type is unknown (koutparam), value is nil
  2. During FFI call: C function writes to the variable
  3. After FFI call: Za inspects the returned value and automatically sets:

Type Mapping on Return

Supported return types are automatically determined:

C Return Type Za Type IKind
int int kint
int[] []int ksint
float64 float kfloat
float64[] []float ksfloat
Struct/pointer any kany
Map data map kmap
All other types (mapped accordingly) (appropriate kind)

Comparison: var name mut vs Pre-Declaration

With var name mut (for FFI output parameters with unknown type):

var result mut
# Pass as output parameter to C function
lib::get_result(mut result)
# result now has proper type determined by C function

With pre-declared type (when type is known):

var result []int  # Must know type upfront
result = []  # Initialize with empty array
lib::fill_result(mut result)  # C function fills it

The var name mut syntax is specifically for cases where the type is truly unknown at declaration time and will be determined by the FFI call.

Key Behavior

Examples

Receiving array from C function as output parameter:

var filled_array mut
# Pass to C function which will fill it
lib::fill_sequence(mut filled_array, 5, 100)
# filled_array is now typed as []int with values [100, 101, 102, 103, 104]
println filled_array[0]  # 100

Global output parameter:

var global_result mut

def fetch_from_c()
    # C function writes to global output parameter
    c_library::get_data(mut global_result)
    # global_result now contains C data with proper type
    println global_result
end

fetch_from_c()

Using output parameters in function:

# Multiple variables with unknown types from different C calls
var config, data mut

lib::load_config(mut config, "config.ini")
lib::load_data(mut data, "data.bin")

# Both are now properly typed based on what C functions returned
if config
    println "Config loaded"
endif

Advantages Over Manual Marshaling

Aspect var mut Manual Marshaling
Syntax Simple: var x mut Complex: c_alloc() + setup
Type declaration Automatic Manual via c_unmarshal_struct()
Memory management Automatic Manual: c_alloc() + c_free()
Readability Clean Verbose
Performance Slightly faster Overhead of allocation/deallocation

F.5 Calling C Functions

Once declared with LIB, C functions are called using namespace::function(args) syntax. The namespace qualifier is optional but strongly recommended to avoid conflicts with Za’s built-in functions.

Without namespace qualifier: Za performs lookup in this order:

  1. USE chain (if USE +alias was called)
  2. Current namespace
  3. main:: (global namespace)

With namespace qualifier: Za calls the explicitly specified version:

c::strlen(str)     # Always calls libc version
strlen(str)        # May call Za built-in or libc, depending on USE chain

Best practice: Always use explicit namespace qualifiers for C library functions (see F.11.10 for common conflicts).

Examples

use +c
# Simple calls with return values
result = m::sqrt(16.0)           # result = 4.0
power = m::pow(2.0, 8.0)         # power = 256.0
rounded = m::floor(3.7)          # rounded = 3.0

# String operations
len = c_ptr_to_int(c::strlen("Hello"))  # len = 5
cmp = c::strcmp("abc", "xyz")           # cmp < 0

# Memory allocation and use
buffer = malloc(1024)         # Allocate 1KB
memset(buffer, 0, 1024)       # Zero the buffer
free(buffer)                  # Free when done

# File operations (with namespace qualifiers to avoid clash with za's fopen/fclose)
# Important: Many C standard library functions share names with Za's built-in functions.
# For example, Za has its own `fopen()` and `fclose()` functions for file handling that return
# different types than the C versions. Always use explicit namespace qualifiers (`c::`) when
# calling C library functions to avoid accidentally calling Za's built-in version.
# See section F.11.10 for details on namespace conflicts.
fp = c::fopen("/tmp/test.txt", "w")
if !c_ptr_is_null(fp)
    # Write to file
    c::fclose(fp)
endif

# Environment variables (with namespace to avoid confusion with builtin env functions)
home = c::getenv("HOME")
path = c::getenv("PATH")

Type conversions: Za automatically converts between Za types and C types. Strings become char*, integers map to appropriate C integer types, and floats/doubles are preserved. Pointers remain opaque.

F.5.2 Working with Opaque Pointers as int64 Addresses

When FFI functions return opaque pointers (like FT_Library from FreeType or png_structp from libpng), Za represents them as int64 values rather than CPointerValue objects. This is necessary because opaque types cannot be materialized as concrete Za objects.

The memory access functions (c_get_uint32, c_get_int32, etc.) require a CPointerValue as the first parameter, which means you cannot directly use them with these opaque pointer values. To solve this, Za provides a parallel set of memory access functions that work directly with int64 addresses.

The Problem and Solution

The Problem:

# Opaque pointer returned as int64 (from FreeType library)
library_handle = FT_Init_FreeType(mut ft_library)  # Returns int64
face_handle = FT_New_Face(library_handle, font_path, 0, mut face)  # Returns int64

# Calculate struct field offset
glyph_slot_ptr = face_handle + 16  # Valid arithmetic, still int64

# But this fails - c_get_uint32 expects a CPointerValue, not int64:
bitmap_width = c_get_uint32(glyph_slot_ptr, 4)  # ERROR!

The Solution: Use the *_at_addr functions that accept int64 addresses directly:

# Same calculation as above
glyph_slot_ptr = face_handle + 16

# Now this works - c_get_uint32_at_addr accepts int64:
bitmap_width = c_get_uint32_at_addr(glyph_slot_ptr, 4)  # OK!

Available *_at_addr Functions

All 21 *_at_addr functions follow the same pattern and work with opaque pointers (int64 addresses):

Read functions - signature: function_name(address:int64, offset:int) -> type - c_get_byte_at_addr(address, offset) → int - c_get_uint16_at_addr(address, offset) → int - c_get_int16_at_addr(address, offset) → int - c_get_uint32_at_addr(address, offset) → int - c_get_int32_at_addr(address, offset) → int - c_get_uint64_at_addr(address, offset) → uint - c_get_int64_at_addr(address, offset) → int - c_get_float_at_addr(address, offset) → float - c_get_double_at_addr(address, offset) → float

Write functions - signature: function_name(address:int64, offset:int, value:type) - c_set_byte_at_addr(address, offset, value) - writes byte - c_set_uint16_at_addr(address, offset, value) - writes uint16 - c_set_int16_at_addr(address, offset, value) - writes int16 - c_set_uint32_at_addr(address, offset, value) - writes uint32 - c_set_int32_at_addr(address, offset, value) - writes int32 - c_set_uint64_at_addr(address, offset, value) - writes uint64 - c_set_int64_at_addr(address, offset, value) - writes int64 - c_set_float_at_addr(address, offset, value) - writes 32-bit float - c_set_double_at_addr(address, offset, value) - writes 64-bit double

All functions: - Accept an int64 address (from opaque pointers) - Accept an int offset (byte offset within the structure) - Return the same types as their CPointerValue counterparts (see F.6) - Perform address validation (return 0 or no-op if address is null) - Are fully compatible with C shared libraries (glibc, OpenGL, SDL, FreeType, libpng, etc.)

When to Use

Real-World Example: Accessing Nested Struct Fields

This example demonstrates reading nested struct fields through opaque pointers in FreeType:

module "libfreetype.so.6" as ft auto
use +ft

# Initialize FreeType (returns opaque FT_Library handle as int64)
library_ptr = c_alloc(8)
ft_error = FT_Init_FreeType(library_ptr)
ft_library = c_get_int64(library_ptr, 0)  # Extract int64 from buffer
c_free(library_ptr)

# Load font (returns opaque FT_Face handle as int64)
face_ptr = c_alloc(8)
ft_error = FT_New_Face(ft_library, "/usr/share/fonts/TTF/font.ttf", 0, face_ptr)
ft_face = c_get_int64(face_ptr, 0)
c_free(face_ptr)

# Load a glyph for character 'A'
FT_Load_Char(ft_face, 65, 4)  # 4 = FT_LOAD_RENDER

# Navigate opaque struct to get bitmap dimensions
# FT_Face->glyph offset is 16 bytes
glyph_slot_ptr = ft_face + 16

# FT_GlyphSlot->bitmap is at offset 216 bytes
bitmap_ptr = glyph_slot_ptr + 216

# Read bitmap metadata using *_at_addr functions
bitmap_height = c_get_uint32_at_addr(bitmap_ptr, 0)      # rows field
bitmap_width = c_get_uint32_at_addr(bitmap_ptr, 4)       # width field
bitmap_pitch = c_get_int32_at_addr(bitmap_ptr, 8)        # pitch field
bitmap_buffer = c_get_int64_at_addr(bitmap_ptr, 16)      # buffer field (void*)

# Read pixel data from the bitmap buffer
for row = 0 to bitmap_height - 1
    for col = 0 to bitmap_width - 1
        offset = row * bitmap_pitch + col
        pixel = c_get_byte_at_addr(bitmap_buffer, offset)
        if pixel > 0
            println "Pixel ({col}, {row}): {pixel}"
        endif
    endfor
endfor

Why This Is Necessary

  1. Opaque Types: Many C libraries use opaque pointers that have no C definition available. Za cannot create CPointerValue wrappers for types it doesn’t know.

  2. Void Pointers: Return types like void* or library-specific types like png_structp are represented as int64 for safety and consistency.

  3. Memory Navigation: Once you have an opaque pointer, you need to perform arithmetic to navigate to nested struct fields, read values at known byte offsets, and pass calculated addresses to other functions.

  4. Direct Field Access: C libraries provide accessor macros or documentation specifying byte offsets. The *_at_addr functions let you use this information directly without creating intermediate C structures.

Common Access Patterns

Pattern 1: Nested struct access

# Get address of nested struct field
nested_address = opaque_ptr + offset_to_nested_struct

# Read field from nested struct
value = c_get_uint32_at_addr(nested_address, field_offset)

Pattern 2: Array element access

# Array of int32 at opaque_address, get element 5
element_5 = c_get_int32_at_addr(opaque_address, 5 * 4)  # 4 bytes per int32

Pattern 3: Chain of pointers

# Read pointer field from opaque struct
pointer_field = c_get_int64_at_addr(opaque_address, pointer_offset)

# Use the pointer field as a new address
value = c_get_uint32_at_addr(pointer_field, value_offset)

See Also

F.6 Helper Functions for FFI

Za provides built-in helper functions to work with C pointers and memory. This section covers functions that work with CPointerValue objects (returned from c_alloc() and other standard operations).

Note: If you’re working with opaque pointers (represented as int64 values from C library calls like FreeType or libpng), see F.5.2 “Working with Opaque Pointers as int64 Addresses” for the parallel *_at_addr functions (c_get_uint32_at_addr, c_set_int32_at_addr, etc.) that work with int64 addresses instead.

c_ptr_is_null(ptr) -> bool

Checks if a C pointer is null.

fp = c::fopen("/nonexistent", "r")
if c_ptr_is_null(fp)
    println "Failed to open file"
else
    c::fclose(fp)
endif

c_ptr_to_int(ptr) -> int

Converts a C pointer to an integer. Essential for size_t return values.

# strlen returns size_t as a pointer
len = c_ptr_to_int(c::strlen("Hello, World!"))  # len = 13
println "Length: {len}"

c_alloc(size:int) -> pointer

Allocates a zero-initialized byte buffer (similar to calloc).

buffer = c_alloc(256)
# Use buffer with C functions
c_set_byte(buffer, 0, 65)  # Set first byte to 'A'
# ...
c_free(buffer)

c_free(ptr)

Frees memory allocated by c_alloc().

buf = c_alloc(1000)
# ... use buffer ...
c_free(buf)

c_null() -> pointer

Returns a null C pointer.

null_ptr = c_null()
assert c_ptr_is_null(null_ptr), "Should be null"

c_fopen(path:string, mode:string) -> pointer

Opens a file and returns a FILE* pointer. Modes: "r", "w", "a", "rb", "wb", etc.

fp = c_fopen("/tmp/output.txt", "w")
if !c_ptr_is_null(fp)
    # Use with fwrite, fprintf, etc.
    c_fclose(fp)
endif

c_fclose(file_ptr) -> int

Closes a FILE* pointer. Returns 0 on success.

result = c_fclose(fp)
assert result == 0, "Failed to close file"

c_set_byte(ptr:pointer, offset:int, value:int)

Sets a byte at a specific offset in a buffer.

buf = c_alloc(256)
c_set_byte(buf, 0, 72)   # 'H'
c_set_byte(buf, 1, 105)  # 'i'
c_set_byte(buf, 2, 0)    # Null terminator

c_get_byte(ptr:pointer, offset:int) -> int

Reads a byte at a specific offset in a buffer.

buf = c_alloc(256)
c_set_byte(buf, 0, 65)
value = c_get_byte(buf, 0)  # Returns 65
println value  # 65

c_get_uint16(ptr:pointer, offset:int) -> int

Reads a 16-bit unsigned integer at a specific byte offset. Useful for reading 16-bit integers from serialized data or C structures.

buf = c_alloc(256)
c_set_uint16(buf, 10, 40000)
val = c_get_uint16(buf, 10)  # 40000

c_get_uint32(ptr:pointer, offset:int) -> int

Reads a 32-bit unsigned integer at a specific byte offset.

# Pack multiple values into buffer
header = c_alloc(16)
c_set_uint32(header, 0, 0x12345678)    # Magic number
c_set_uint16(header, 4, 2)             # Version
c_set_uint16(header, 6, 1024)          # Size

magic = c_get_uint32(header, 0)  # 0x12345678
size = c_get_uint16(header, 6)   # 1024

c_get_int16(ptr:pointer, offset:int) -> int

Reads a 16-bit signed integer at a specific byte offset.

buf = c_alloc(32)
c_set_int16(buf, 0, -1000)
val = c_get_int16(buf, 0)  # -1000

c_get_int32(ptr:pointer, offset:int) -> int

Reads a 32-bit signed integer at a specific byte offset.

coords = c_alloc(12)  # 3 x 4-byte integers
c_set_int32(coords, 0, -500)
c_set_int32(coords, 4, 1000)
c_set_int32(coords, 8, 250)

x = c_get_int32(coords, 0)   # -500
y = c_get_int32(coords, 4)   # 1000
z = c_get_int32(coords, 8)   # 250

c_get_uint64(ptr:pointer, offset:int) -> uint

Reads a 64-bit unsigned integer at a specific byte offset.

buf = c_alloc(16)
c_set_uint64(buf, 0, 9223372036854775807)
val = c_get_uint64(buf, 0)  # Large number

c_get_int64(ptr:pointer, offset:int) -> int

Reads a 64-bit signed integer at a specific byte offset.

buf = c_alloc(16)
c_set_int64(buf, 0, -9223372036854775808)
val = c_get_int64(buf, 0)  # Minimum int64

c_set_uint16(ptr:pointer, offset:int, value:int)

Writes a 16-bit unsigned integer at a specific offset in a buffer.

buf = c_alloc(256)
c_set_uint16(buf, 0, 65535)   # Write max uint16
value = c_get_uint16(buf, 0)  # Read back: 65535

c_set_int16(ptr:pointer, offset:int, value:int)

Writes a 16-bit signed integer at a specific offset in a buffer.

buf = c_alloc(256)
c_set_int16(buf, 0, -32768)  # Write min int16
value = c_get_int16(buf, 0)  # Read back: -32768

c_set_uint32(ptr:pointer, offset:int, value:int)

Writes a 32-bit unsigned integer at a specific offset in a buffer.

buf = c_alloc(256)
c_set_uint32(buf, 4, 4294967295)   # Write max uint32
value = c_get_uint32(buf, 4)       # Read back

c_set_int32(ptr:pointer, offset:int, value:int)

Writes a 32-bit signed integer at a specific offset in a buffer.

buf = c_alloc(256)
c_set_int32(buf, 4, -2147483648)  # Write min int32
value = c_get_int32(buf, 4)       # Read back

c_set_uint64(ptr:pointer, offset:int, value:int)

Writes a 64-bit unsigned integer at a specific offset in a buffer.

buf = c_alloc(256)
c_set_uint64(buf, 8, 18446744073709551615)  # Write max uint64
value = c_get_uint64(buf, 8)                # Read back

c_set_int64(ptr:pointer, offset:int, value:int)

Writes a 64-bit signed integer at a specific offset in a buffer.

buf = c_alloc(256)
c_set_int64(buf, 8, -9223372036854775808)  # Write min int64
value = c_get_int64(buf, 8)                # Read back

c_get_float(ptr:pointer, offset:int) -> float

Reads a 32-bit floating-point number at a specific byte offset. Float values are stored in IEEE 754 format and are limited to about 6-7 significant digits.

buf = c_alloc(256)
c_set_float(buf, 0, 3.14159)      # Write float
value = c_get_float(buf, 0)       # Read back: ~3.14159

c_set_float(ptr:pointer, offset:int, value:float)

Writes a 32-bit floating-point number at a specific offset in a buffer.

# Store physics constants
constants = c_alloc(12)
c_set_float(constants, 0, 9.81)    # Gravity
c_set_float(constants, 4, 299792458.0)  # Speed of light
c_set_float(constants, 8, 6.626e-34)    # Planck's constant

g = c_get_float(constants, 0)

c_get_double(ptr:pointer, offset:int) -> float

Reads a 64-bit floating-point number (double) at a specific byte offset. Doubles provide higher precision (about 15-16 significant digits) than floats and should be used for scientific calculations.

buf = c_alloc(256)
c_set_double(buf, 0, 3.141592653589793)
value = c_get_double(buf, 0)  # Read back with full precision

c_set_double(ptr:pointer, offset:int, value:float)

Writes a 64-bit floating-point number (double) at a specific offset in a buffer.

# Store high-precision coordinates
matrix = c_alloc(32)  # 4 doubles
c_set_double(matrix, 0, 1.0000000000000001)
c_set_double(matrix, 8, 2.7182818284590452)
c_set_double(matrix, 16, 3.1415926535897932)
c_set_double(matrix, 24, 1.4142135623730951)

val = c_get_double(matrix, 8)  # Read back with precision

Struct Allocation and Marshaling Functions

c_alloc_struct(struct_type_name:string) -> pointer

Allocates C memory for a struct type defined in Za. Returns a pointer to the allocated memory.

struct Point
    x int
    y int
endstruct

ptr = c_alloc_struct("Point")
# ptr now points to C memory sized for the Point struct

Important: The struct must be defined with the struct keyword before calling this function.

c_free_struct(ptr:pointer)

Frees C struct memory allocated by c_alloc_struct().

ptr = c_alloc_struct("Point")
# ... use the pointer ...
c_free_struct(ptr)

c_unmarshal_struct(ptr:pointer, struct_type_name:string) -> struct

Reads C memory and converts it to a Za struct. Note: For AUTO-registered structs, use the mut keyword approach instead (see F.2), which is simpler and automatic.

Legacy usage (for manually-defined structs without AUTO):

struct FileStat
    st_mode int
    st_size int
endstruct

# Allocate memory for C to fill
statbuf = c_alloc_struct("FileStat")

# Call C function with pointer
LIB c::stat(pathname:string, statbuf:pointer) -> int
c::stat("/etc/passwd", statbuf)

# Unmarshal C data back to Za struct
stats = c_unmarshal_struct(statbuf, "FileStat")
println "File size:", stats.st_size

c_free_struct(statbuf)

Function Pointer Operations (v1.2.2+)

Za provides functions for working with C function pointers, enabling callbacks from C libraries like qsort comparators, signal handlers, and event-driven libraries.

c_as_function_ptr(pointer, signature:string) -> CFunctionPointer

Converts a C function pointer to a callable CFunctionPointer object.

Signature format: "param1,param2,...->return_type"

# Get function pointer from C library
add_ptr_raw = mylib::get_add_function()  # Returns CPointer

# Convert to callable function pointer
add_ptr = c_as_function_ptr(add_ptr_raw, "int,int->int")

# Now callable with c_call_function_ptr

Supported types: All standard C types (int, float, double, pointer, string, struct types, etc.)

c_call_function_ptr(func_ptr:CFunctionPointer, …args) -> any

Calls a C function pointer with the provided arguments.

# Create function pointer
add_ptr = c_as_function_ptr(raw_ptr, "int,int->int")

# Call it
result = c_call_function_ptr(add_ptr, 10, 20)
println result  # 30

Complete example:

module "./libtest.so" as test auto "./test.h"
use +test

def main()
    # Get function pointer from C library
    add_ptr_raw = test::get_add_ptr()

    # Convert to callable function pointer
    add_ptr = c_as_function_ptr(add_ptr_raw, "int,int->int")

    # Call the function pointer
    result = c_call_function_ptr(add_ptr, 10, 20)
    println "10 + 20 = " + result  # 30

    # Works with different signatures
    compare_ptr = c_as_function_ptr(test::get_compare_ptr(), "int,int->int")
    cmp_result = c_call_function_ptr(compare_ptr, 5, 10)  # -1, 0, or 1
end

Error handling: - Returns error if pointer is null - Validates argument count against signature - Checks type compatibility through libffi

Use with AUTO-parsed function pointer typedefs: When AUTO parses typedef declarations like typedef int (*compare_t)(const void*, const void*), the type information is registered and can be used with these functions.

Opaque Pointer Functions (int64 Addresses)

The functions described in F.5.2 “Working with Opaque Pointers as int64 Addresses” (*_at_addr variants) provide parallel access patterns for the basic functions above, but work with opaque pointers (represented as int64) instead of CPointerValue objects. See F.5.2 for detailed examples and guidance on when to use *_at_addr versus the standard c_get_* / c_set_* functions.

Example: Reading wchar_t array values

# On Linux, wchar_t is 4 bytes
detected_size = get_wchar_size()  # Returns 4
arr_ptr = c_alloc(detected_size * 5)
wchar_fill_array(arr_ptr, 5)

# Read back array elements
for i = 0 to 4
    if detected_size == 2
        val = c_get_uint16(arr_ptr, i * 2)
    else
        val = c_get_uint32(arr_ptr, i * 4)
    endif
    println "Value {i}: {val}"
endfor
c_free(arr_ptr)

F.7 Discovering Functions with help plugin

Za’s help system provides runtime introspection of loaded C libraries.

List all loaded libraries

help plugin

Output:

Loaded C libraries:
  m -> /usr/lib/libm.so.6 (45 symbols)
  c -> /usr/lib/libc.so.6 (200 symbols)
  json -> /usr/lib/libjson-c.so.5 (89 symbols)

Use 'help plugin <library_name>' for detailed information.

Show library functions

help plugin m

Output:

C Library: m
Symbols found: 45

Functions:
  acos(x:double) -> double
  asin(x:double) -> double
  atan(x:double) -> double
  ceil(x:double) -> double
  cos(x:double) -> double
  floor(x:double) -> double
  log(x:double) -> double
  pow(x:double, y:double) -> double
  sin(x:double) -> double
  sqrt(x:double) -> double
  tan(x:double) -> double
  ...

Search for a function

help plugin find strlen

Output:

Searching for function: strlen

Found in 1 library:
  • c

Looking up function signature...

Found in man page:
  size_t strlen(const char *s);

Suggested LIB declaration:
  lib c::strlen(s:string) -> int

For more details:
  • Run: man 3 strlen
  • Online: https://man7.org/linux/man-pages/man3/strlen.3.html

Namespaced search:

help plugin find c::strlen
help plugin find glib::g_malloc

Man page integration: Za automatically looks up function signatures from system man pages (section 3) and suggests appropriate LIB declarations with C-to-Za type mappings.

View AUTO-discovered macros

When using the AUTO clause to import C headers, help plugin <library> displays discovered macros with status indicators:

module "libc.so.6" as pth auto "/usr/include/pthread.h"
help plugin pth

Output:

C Library: pth
Path: /usr/include/pthread.h
Symbols found: 89

Discovered Macros:
  ✓ CLOCKS_PER_SEC = ((__clock_t) 1000000)
  ○ PTHREAD_CANCEL_ASYNCHRONOUS = PTHREAD_CANCEL_ASYNCHRONOUS
  ○ PTHREAD_MUTEX_INITIALIZER = { { __PTHREAD_MUTEX_INITIALIZER (PTHREAD_MUTEX_TIMED_NP) } }
  ? pthread_cleanup_push = do {
      __pthread_unwind_buf_t __cancel_buf;
      void (*__cancel_routine) (void *) = (routine);
      ...
  ? pthread_cleanup_pop = do { } while (0);
      } while (0);
      __pthread_unregister_cancel (&__cancel_buf);
      ...

Functions:
  pthread_create(thread:pointer, attr:pointer, start_routine:pointer, arg:pointer) -> int
  pthread_join(thread:uint64, retval:pointer) -> int
  ...

Macro status indicators:

Symbol Color Meaning
Green Successfully evaluated - value available as constant
Gray Skipped - filtered out (references system macros, contains type keywords, etc.)
Red Evaluation failed - attempted but encountered error
? Gray Unknown - function-like macro (not evaluated, displayed for reference only)

Notes:

F.8 Type Mapping Reference

Integer types:

C Type Za Type Range Auto-converts from Za types
int, int32_t int -2³¹ to 2³¹-1 int, uint, int64, uint64
unsigned int, uint32_t uint 0 to 2³²-1 int, uint, int64, uint64
int8_t int8 -128 to 127 int, int64, uint, uint64, uint8
uint8_t uint8 / byte 0 to 255 uint8 (native), int, uint, int64
short, int16_t int16 -32,768 to 32,767 int, int64, uint, uint64, uint8
unsigned short, uint16_t uint16 0 to 65,535 int, uint, int64, uint64, uint8
long long, int64_t, off_t int64 -2⁶³ to 2⁶³-1 int, int64, uint, uint64
unsigned long long, uint64_t uint64 0 to 2⁶⁴-1 uint64 (native), int, int64, uint
intptr_t, ptrdiff_t int64 Pointer-sized int, int64, uint, uint64
uintptr_t uint64 Pointer-sized uint64, int, int64, uint
size_t uint64 Unsigned pointer-sized uint64, int, uint
ssize_t int Signed pointer-sized int, int64, uint, uint64

Floating-point types:

C Type Za Type Notes
float float 32-bit single precision
double double 64-bit double precision (preferred)
long double longdouble 80-bit extended precision (may lose precision)

Other types:

C Type Za Type Notes
void (none) Return type only
void* pointer Generic pointer
char* string Null-terminated strings
bool, _Bool bool Boolean type
struct, union pointer or struct<name> Opaque pointer OR marshaled struct (see F.9.1, F.9.2)

Array types (v1.2.2+):

Za Type C Equivalent Notes
[]int int* Dynamic array of integers, automatically converted to pointer
[]float64 double* Dynamic array of doubles, automatically converted to pointer
[]uint8 uint8_t*, unsigned char* Byte array, useful for binary data and buffers
[]int64 long long*, int64_t* Dynamic array of 64-bit integers
[]uint unsigned int* Dynamic array of unsigned integers
[]uint16 uint16_t*, unsigned short* Dynamic array of 16-bit unsigned integers
[]uint64 uint64_t*, unsigned long long* Dynamic array of 64-bit unsigned integers
[]int8 int8_t*, signed char* Dynamic array of 8-bit signed integers
[]int16 short*, int16_t* Dynamic array of 16-bit signed integers
[]bool unsigned char* Dynamic array of booleans (0/1), useful for flags
[]string char** Dynamic array of string pointers

Array behavior:

String handling: Za strings are automatically converted to null-terminated char* when passed to C. C strings returned as string type are copied into Za memory.

Pointer considerations: Pointers to structures are opaque in Za. You cannot access struct fields directly—pass pointers to C functions that know the layout.

Struct marshaling: Za supports marshaling structs between Za and C memory. Use struct<typename> in LIB declarations for automatic marshaling of input parameters. For “out parameters” where C fills the struct, use manual marshaling with c_alloc_struct(), c_unmarshal_struct(), and c_free_struct().

F.9 Complete Working Examples

F.9.1 Math Operations (libm)

module "/usr/lib/libm.so.6" as m
use +m

LIB m::sqrt(x:double) -> double
LIB m::pow(x:double, y:double) -> double
LIB m::sin(x:double) -> double
LIB m::cos(x:double) -> double
LIB m::floor(x:double) -> double
LIB m::ceil(x:double) -> double

# Calculate hypotenuse
a = 3.0
b = 4.0
c = m::sqrt(m::pow(a, 2.0) + m::pow(b, 2.0))
println "Hypotenuse: {c}"  # 5.0

# Trigonometry (angles in radians)
angle = 3.14159 / 4.0  # 45 degrees
println "sin(45°) = ",m::sin(angle)  # 0.707...
println "cos(45°) = ",m::cos(angle)  # 0.707...

# Rounding
println "floor(3.7) = ",m::floor(3.7)  # 3.0
println "ceil(3.2) = ",m::ceil(3.2)    # 4.0

F.9.2 String Manipulation (libc)

module "/usr/lib/libc.so.6" as c
use +c

LIB c::strlen(s:string) -> pointer
LIB c::strcmp(s1:string, s2:string) -> int
LIB c::strdup(s:string) -> pointer

# String length
len = c_ptr_to_int(c::strlen("Hello, World!"))
println "Length: {len}"  # 13

# String comparison
cmp = c::strcmp("apple", "banana")
if cmp < 0
    println "apple comes before banana"
endif

cmp2 = c::strcmp("test", "test")
println "Equal strings: {cmp2}"  # 0

F.9.3 Memory Management (libc)

module "/usr/lib/libc.so.6" as c
use +c

LIB c::malloc(size:int) -> pointer
LIB c::memset(ptr:pointer, value:int, num:int) -> pointer
LIB c::free(ptr:pointer) -> void

# Allocate buffer
buffer = malloc(1000)
if c_ptr_is_null(buffer)
    println "Memory allocation failed!"
    exit 1
endif

# Initialize memory
memset(buffer, 42, 1000)  # Fill with value 42

# Use buffer...

# Free when done
c::free(buffer)
println "Memory freed"

F.9.4 JSON Processing (libjson-c)

module "/usr/lib/libjson-c.so.5" as json
use +json

LIB json::json_object_new_object() -> pointer
LIB json::json_object_new_string(s:string) -> pointer
LIB json::json_object_new_int(i:int) -> pointer
LIB json::json_object_new_double(d:double) -> pointer
LIB json::json_object_object_add(obj:pointer, key:string, val:pointer) -> void
LIB json::json_object_to_json_string(obj:pointer) -> string
LIB json::json_object_put(obj:pointer) -> void

# Create JSON object
person = json_object_new_object()

# Add fields
json_object_object_add(person, "name", json_object_new_string("Alice"))
json_object_object_add(person, "age", json_object_new_int(30))
json_object_object_add(person, "salary", json_object_new_double(75000.50))

# Convert to JSON string
json_str = json_object_to_json_string(person)
println json_str
# Output: {"name":"Alice","age":30,"salary":75000.5}

# Cleanup
json_object_put(person)

F.9.5 Array Processing (v1.2.2+)

Arrays are automatically converted to C pointers when passed to C functions, eliminating manual memory management:

module "libarray.so" as arr auto "array.h"
use +arr

# Read-only arrays - automatically converted to C pointers
data = [1, 2, 3, 4, 5]
sum = arr::sum_int_array(data, 5)
println "Sum: {sum}"  # 15

# Float arrays
values = [1.5, 2.5, 3.5]
avg = arr::average_float_array(values, 3)
println "Average: {avg}"  # 2.5

# Byte arrays (useful for image data, checksums, etc.)
bytes = [0xFF, 0xFE, 0xFD, 0xFC]
checksum = arr::compute_checksum(bytes, 4)
println "Checksum: {checksum}"

# Mutable arrays - C function modifies values in place
buffer = [10, 20, 30]
arr::double_int_array(mut buffer, 3)
println "After doubling: {buffer[0]}, {buffer[1]}, {buffer[2]}"  # 20, 40, 60

# Build array dynamically
row = []
for x = 0 to 2
    row = append(row, x * 100)
    row = append(row, x * 50)
    row = append(row, x * 25)
endfor
result = arr::process_rgb_row(row, 9)  # 3 pixels × 3 components
println "Processed: {result}"

# String arrays
args = ["program", "--verbose", "--output", "file.txt"]
arg_count = arr::count_args(args, 4)
println "Args: {arg_count}"

F.9.6 Graphics (libgd)

module "/usr/lib/libgd.so.3" as gd
use +gd

LIB gd::gdImageCreate(width:int, height:int) -> pointer
LIB gd::gdImageColorAllocate(im:pointer, r:int, g:int, b:int) -> int
LIB gd::gdImageLine(im:pointer, x1:int, y1:int, x2:int, y2:int, color:int) -> void
LIB gd::gdImageRectangle(im:pointer, x1:int, y1:int, x2:int, y2:int, color:int) -> void
LIB gd::gdImageJpeg(im:pointer, out:pointer, quality:int) -> void
LIB gd::gdImageDestroy(im:pointer) -> void

# Create image
im = gdImageCreate(200, 150)

# Allocate colors
white = gdImageColorAllocate(im, 255, 255, 255)
red = gdImageColorAllocate(im, 255, 0, 0)
blue = gdImageColorAllocate(im, 0, 0, 255)

# Draw shapes
gdImageLine(im, 10, 10, 190, 140, red)
gdImageRectangle(im, 50, 50, 150, 100, blue)

# Save to file
fp = c_fopen("/tmp/output.jpg", "wb")
if !c_ptr_is_null(fp)
    gdImageJpeg(im, fp, 90)  # Quality: 90
    c_fclose(fp)
    println "Image saved to /tmp/output.jpg"
endif

# Cleanup
gdImageDestroy(im)

F.9.7 Terminal UI (ncurses)

module "/usr/lib/libncursesw.so.6" as nc
use +nc

LIB nc::initscr() -> pointer
LIB nc::printw(fmt:string) -> int
LIB nc::mvaddch(y:int, x:int, ch:int) -> int
LIB nc::mvprintw(y:int, x:int, fmt:string) -> int
LIB nc::refresh() -> int
LIB nc::getch() -> int
LIB nc::endwin() -> int

# Initialize screen
stdscr = nc::initscr()

# Print text
nc::printw("Hello from Za FFI!\n")
nc::mvprintw(5, 10, "Press any key to exit...")

# Draw characters
nc::mvaddch(10, 20, 42)   # '*'
nc::mvaddch(10, 21, 42)
nc::mvaddch(10, 22, 42)

# Refresh and wait
nc::refresh()
nc::getch()

# Cleanup
nc::endwin()

F.9.8 X11 Window (Xlib) - Complete AUTO Example

This example demonstrates Za’s comprehensive FFI/AUTO capabilities including unions, structs, and auto-discovered constants:

#!/usr/bin/za

# Import X11 with auto header parsing
module "/usr/lib/libX11.so.6" as x11 auto "/usr/include/X11/Xlib.h"
use +x11

WIDTH = 800
HEIGHT = 600

# Open connection to X server
display = XOpenDisplay(c_null())
if c_ptr_is_null(display)
    println "Error: Cannot open X display"
    exit 1
endif

screen = XDefaultScreen(display)
root = XRootWindow(display, screen)
black = XBlackPixel(display, screen)
white = XWhitePixel(display, screen)

# Create window
window = XCreateSimpleWindow(display, root, 0, 0, WIDTH, HEIGHT, 1, black, white)
XStoreName(display, window, "Za FFI X11 Window")

# Select events (using auto-generated constants)
event_mask = x11::ExposureMask | x11::KeyPressMask | x11::ButtonPressMask
XSelectInput(display, window, event_mask)

# Show window
XMapWindow(display, window)
XFlush(display)

println "Window created! Press Ctrl+C in window to close"

# Event loop with XEvent union
event_ptr = c_alloc_struct("XEvent")
running = true

while running
    XNextEvent(display, event_ptr)
    event = c_unmarshal_struct(event_ptr, "XEvent")

    event_type = event["type"]

    if event_type == x11::KeyPress
        key_event = event["xkey"]
        state = key_event["state"]
        keycode = key_event["keycode"]

        println "Key pressed: keycode=" + as_string(keycode)

        # Check for Ctrl+C (Control key + 'c' key)
        if (state & x11::ControlMask) != 0 and keycode == 54
            println "Ctrl+C detected - exiting"
            running = false
        endif
    endif
endwhile

# Cleanup
c_free(event_ptr)
XDestroyWindow(display, window)
XCloseDisplay(display)
println "Done!"

Key features demonstrated:

Full code: eg/ffi-x11-window

F.10 Advanced Topics

F.10.1 Opaque Structure Pointers

C structures are handled as opaque pointers. Za doesn’t need to know the internal layout:

module "/usr/lib/libglib-2.0.so.0" as glib
use +glib

LIB glib::g_list_append(list:pointer, data:pointer) -> pointer
LIB glib::g_list_length(list:pointer) -> int
LIB glib::g_list_free(list:pointer) -> void

# GList is a struct, but we treat it as an opaque pointer
list = c_null()
list = glib::g_list_append(list, "Item 1")
list = glib::g_list_append(list, "Item 2")
list = glib::g_list_append(list, "Item 3")

length = glib::g_list_length(list)
println "List length: {length}"  # 3

glib::g_list_free(list)

F.10.2 Struct Marshaling Between Za and C

Za supports bidirectional struct marshaling for passing data between Za and C libraries.

Three Ways to Use Structs with C

Note: For AUTO-registered structs, consider using the mut keyword pattern (see F.4.5) for cleaner syntax with out parameters.

  1. Opaque pointers (pointer): When C library manages the struct internally
  2. Marshaled structs: VAR-declared AUTO structs passed by reference (recommended for out parameters)
  3. Manual marshaling (struct<Name>): When you define the struct in Za and pass data to/from C

Defining Structs

Use Za’s existing struct keyword:

struct Point
    x int
    y int
endstruct

Automatic Marshaling (Input Parameters)

Use struct<typename> in LIB declarations for automatic marshaling:

module "libexample.so" as ex
use +ex

struct Point
    x int
    y int
endstruct

# C function: void draw_point(Point pt)
LIB ex::draw_point(pt:struct<Point>) -> void

# Create Za struct and call - automatic marshaling
point = Point(.x 100, .y 200)
ex::draw_point(point)

Za automatically:

  1. Allocates C memory for the struct
  2. Marshals Za struct fields to C memory layout
  3. Passes pointer to C function
  4. Frees C memory after the call

When C functions fill struct data using AUTO-registered structs, use the mut keyword for clean, automatic handling:

module "libc.so.6" as c auto "/usr/include/sys/stat.h"
use +c

def get_file_info(filepath)
    # Declare struct variable from AUTO-parsed C header
    var info c::stat

    # Call C function that fills the struct
    # The mut keyword passes by reference, allowing C to modify fields
    result = c::stat(filepath, mut info)

    if result != 0
        return null
    endif

    # Fields are directly accessible - no unmarshaling needed
    return info
end

# Use it
info = get_file_info("/etc/passwd")
if info != null
    println "Size: {info.st_size} bytes"
    println "Owner: UID {info.st_uid}"
    println "Mode: {info.st_mode}"
endif

How it works:

  1. AUTO clause: Loads struct definitions from C headers automatically
  2. var declaration: Creates a struct variable using the C type directly (c::stat)
  3. mut keyword: Passes the struct by reference, so C can modify fields in-place
  4. Direct access: Fields are accessible with dot notation - no marshaling required
  5. Automatic cleanup: Memory is managed automatically

Supported Field Types

Za Field Type C Equivalent Notes
int int, long, int32_t Signed integer
float float Single precision
double double Double precision
bool _Bool, int Converted to 0/1
string char* Allocated as C string
pointer void* Raw pointer value
[int] int[N] Fixed-size arrays (v1.2.2+)

Supported (v1.2.2+): Fixed-size arrays in structs, union types, struct-by-value parameters/returns, nested structs/unions, function pointer fields

Struct-by-Value Support (v1.2.2+)

Za fully supports C functions that pass or return structs by value (not pointers). This is handled automatically when using the AUTO clause or when struct definitions are registered with the FFI system.

Return by value:

module "libexample.so" as ex auto "example.h"
use +ex

# C function: Color make_color(uint8_t r, uint8_t g, uint8_t b, uint8_t a)
# Automatically discovered and callable
color = ex::make_color(255, 128, 64, 200)

# Access fields directly
println "R: {color.rgb[0]}, G: {color.rgb[1]}, B: {color.rgb[2]}"
println "Alpha: {color.alpha}"

Pass by value:

# C function: int get_average_rgb(Color c)
avg = ex::get_average_rgb(color)
println "Average RGB: {avg}"

Fixed-size arrays in structs:

# C struct: typedef struct { int id; int values[5]; int count; } DataBlock;
block = ex::make_data_block(42, 10, 20, 30, 40, 50)

# Access array elements
println "ID: {block.id}"
println "First value: {block.values[0]}"
println "Third value: {block.values[2]}"
println "Count: {block.count}"

# Pass back to C
sum = ex::sum_data_block(block)
println "Sum: {sum}"  # Prints: Sum: 150

Key features:

  1. Automatic marshaling: Za creates proper libffi type descriptors for structs
  2. Array expansion: Fixed-size arrays like uint8_t rgb[3] work correctly
  3. Round-trip fidelity: Data is preserved when passing structs C → Za → C
  4. Type safety: Validates struct definitions exist before calling C functions
  5. Performance: Struct type descriptors are cached and reused

Notes:

Union Support (v1.2.2+)

Za supports C unions through the AUTO clause. Unions are instantiated using Za’s map() constructor with a single active field:

Creating unions in Za:

# C union: typedef union { int int_value; float float_value; } Number;

# Instantiate with one active field
num1 = map(.int_value 42)
num2 = map(.float_value 3.14)

# Pass to C function
result = process_number(num1)  # C receives union with int_value active

Receiving unions from C:

# C function returns union by value
color = make_color_union(255, 128, 64)

# Access union fields as map keys
println "R: {color["rgb"][0]}"
println "Alpha: {color["alpha"]}"

Why one field? In C, a union is a special data structure where all fields share the same memory location. Only one field can hold valid data at a time - setting one field overwrites all others. Za enforces this by requiring map literals with exactly one field when creating unions.

Union marshaling rules:

  1. Creating unions: Use map() with exactly ONE field (the active union member)
  2. Multi-field error: map(.field1 value1, .field2 value2) fails - unions require exactly one active field
  3. Empty map error: map() with no fields fails - unions require exactly one field
  4. Invalid field error: Field name must match a union member defined in C header

Key features:

See working examples in:

Best Practices

  1. Match C layouts: Ensure Za struct field order and types match the C definition
  2. Free memory: Always call c_free_struct() on manually allocated structs
  3. Use automatic when possible: For input parameters, struct<typename> handles cleanup
  4. Manual for out parameters: When C fills the struct, use alloc → call → unmarshal → free pattern
  5. Platform differences: Struct layouts can vary by platform - test on target systems

Complete Examples

See working examples in:

When to Use Each Mode

Use opaque pointers (pointer) when:

Use marshaled structs (struct<Name>) when:

Use manual marshaling when:

F.10.3 Complex Data Structures

Working with complex C data structures via pointers:

# JSON arrays
module "/usr/lib/libjson-c.so.5" as json
use +json

LIB json::json_object_new_array() -> pointer
LIB json::json_object_array_add(arr:pointer, val:pointer) -> int
LIB json::json_object_array_length(arr:pointer) -> int

arr = json::json_object_new_array()
json::json_object_array_add(arr, json::json_object_new_int(10))
json::json_object_array_add(arr, json::json_object_new_int(20))
json::json_object_array_add(arr, json::json_object_new_int(30))

length = json::json_object_array_length(arr)
println "Array length: {length}"  # 3

json_str = json::json_object_to_json_string(arr)
println json_str  # [10,20,30]

json::json_object_put(arr)

F.10.4 Multi-Library Integration

Combining multiple C libraries in one script:

# Load multiple libraries
module "libm.so.6" as m
module "/usr/lib/libc.so.6" as c
module "/usr/lib/libglib-2.0.so.0" as glib
use +m
use +c
use +glib

# Declare functions from different libraries
LIB m::sqrt(x:double) -> double
LIB c::malloc(size:int) -> pointer
LIB c::free(ptr:pointer) -> void
LIB glib::g_random_double() -> double

# Use functions together
random_val = glib::g_random_double() * 100.0
sqrt_val = m::sqrt(random_val)

buffer = c::malloc(1024)
# ... use buffer ...
c::free(buffer)

F.10.5 Error Handling Strategies

Check return values and null pointers:

# File operations with error checking
fp = c_fopen("/tmp/test.dat", "w")
if c_ptr_is_null(fp)
    println "[ERROR] Failed to open file"
    exit 1
endif

# Write data
bytes_written = c::fwrite(buffer, 1, 100, fp)
if bytes_written != 100
    println "[ERROR] Write failed: expected 100, wrote {bytes_written}"
    c_fclose(fp)
    exit 1
endif

# Close and check
result = c_fclose(fp)
if result != 0
    println "[ERROR] Failed to close file properly"
    exit 1
endif

println "File operations completed successfully"

F.10.6 Memory Management Patterns

Pattern 1: Allocate-Use-Free

buffer = c::malloc(1024)
if !c_ptr_is_null(buffer)
    # Use buffer
    c::memset(buffer, 0, 1024)
    # ... operations ...
    c::free(buffer)
endif

Pattern 2: Resource wrapper with cleanup

def safe_file_operation(filename, data)
    fp = c_fopen(filename, "w")
    if c_ptr_is_null(fp)
        return false
    endif

    # Use file
    c::fwrite(data, 1, len(data), fp)

    # Always cleanup
    c_fclose(fp)
    return true
end

Pattern 3: Library-managed memory

# Some libraries manage their own memory
obj = json::json_object_new_object()
# ... use object ...
json::json_object_put(obj)  # Library's cleanup function

F.11 Callback Functions

Za supports passing Za functions to C as callbacks (function pointers). This enables integration with C APIs that require callbacks such as qsort_r(), pthread_create(), signal handlers, and event-driven libraries.

F.11.1 How Callbacks Work

Za uses a trampoline pattern with cgo.Handle to safely pass Za functions through C:

  1. Register a Za function with c_register_callback()
  2. Receive a callback object with .trampoline and .handle fields
  3. Pass these to the C function (trampoline = function pointer, handle = context)
  4. When C calls back, the trampoline invokes your Za function
  5. Cleanup with c_unregister_callback() when done

F.11.2 Supported Callback Signatures

Za supports callback signatures via libffi closures! The system automatically generates closures for any signature.

Core Signatures:

Signature C Type Use Case
ptr,ptr->int int (*)(void*, void*, void*) qsort_r comparators
int,int->int int (*)(int, int, void*) Integer comparators
ptr->ptr void* (*)(void*) pthread_create threads
int,ptr,ptr->void void (*)(int, siginfo_t*, void*) sigaction handlers
int->void void (*)(int) Simple signal handlers
double->double double (*)(double, void*) Math transformations
float->float float (*)(float, void*) Single-precision math
double,double->double double (*)(double, double, void*) Binary math ops
ptr,ptr,ptr->int int (*)(void*, void*, void*, void*) 3-argument comparators
void->void void (*)(void*) Simple callbacks
ptr->void void (*)(void*, void*) Cleanup/destructors
ptr,ptr->void void (*)(void*, void*, void*) Iteration callbacks
ptr,int->void void (*)(void*, int, void*) Buffer processors
ptr,int->int int (*)(void*, int, void*) Buffer validators
ptr,ptr->bool int (*)(void*, void*, void*) Predicate filters
int->int int (*)(int, void*) Hash functions
string->void void (*)(char*, void*) Logging callbacks
string->int int (*)(char*, void*) String validators
int,int->void void (*)(int, int, void*) Progress callbacks

Any signature can be used - Za automatically generates the appropriate closure at runtime. Example custom signatures:

# Triple integer comparator
c_register_callback("my_func", "int,int,int->int")

# Quad pointer callback
c_register_callback("my_func", "ptr,ptr,ptr,ptr->ptr")

# Mixed types
c_register_callback("my_func", "int64,float,double->uint32")

Supported Types: int8, uint8, int16, uint16, int32/int, uint32/uint, int64, uint64, float, double, ptr/pointer, string, bool, void

Note: Most callbacks require C APIs that support a context parameter (also called user_data, thunk, or arg). This is where the .handle is passed.

F.11.3 Example: qsort_r with Custom Comparator

module "libc.so.6" as c
use +c

# qsort_r takes a comparator function and context parameter
LIB c::qsort_r(base:pointer, nmemb:int, size:int, compar:pointer, arg:pointer) -> void

# Define a Za comparator function
def compare_ints(a_ptr, b_ptr)
    # Extract int values from pointers (requires c_ptr_read functions)
    a = c_ptr_read_int(a_ptr, 0)
    b = c_ptr_read_int(b_ptr, 0)

    if a < b
        return -1
    endif
    if a > b
        return 1
    endif
    return 0
end

# Create array and get pointer
arr = [5, 2, 8, 1, 9, 3]
arr_ptr = get_array_pointer(arr)  # Implementation-specific

# Register callback
cb = c_register_callback("compare_ints", "ptr,ptr->int")

# Sort using Za comparator
c::qsort_r(arr_ptr, 6, 4, cb.trampoline, cb.handle)

# Cleanup
c_unregister_callback(cb)

# arr is now sorted

F.11.4 Example: Thread Creation

module "libpthread.so.0" as pthread auto "/usr/include/pthread.h"
use +pthread

# Define thread function
def worker_thread(arg)
    thread_id = c_ptr_to_int(arg)
    println "Thread {thread_id}: Starting work"

    for i = 1 to 5
        println "Thread {thread_id}: Work iteration {i}"
        pause(100)
    endfor

    println "Thread {thread_id}: Work completed"
    return c_null()
end

# Register callback
cb = c_register_callback("worker_thread", "ptr->ptr")

# Create thread - allocate memory for pthread_t handle
VAR thread_handle pointer
VAR thread_arg pointer

thread_handle = c_alloc(8)     # pthread_t storage
thread_arg = c_alloc(8)         # Thread argument
c_set_byte(thread_arg, 0, 42)   # Pass ID = 42

result = pthread::pthread_create(thread_handle, c_null(), cb.trampoline, thread_arg)

if result == 0
    # pthread_join needs the pthread_t VALUE, not pointer
    thread_id = c_get_uint64(thread_handle, 0)
    pthread::pthread_join(thread_id, c_null())
    c_unregister_callback(cb)
else
    println "Failed to create thread"
    c_unregister_callback(cb)
endif

# Cleanup
c_free(thread_handle)
c_free(thread_arg)

F.11.5 Custom Callback Signatures

Za automatically generates closures for custom callback signatures at runtime. This enables integration with any C API that uses callbacks.

Example: Custom 3-Parameter Callback

module "libcustom.so" as custom
use +custom

# Hypothetical C API that takes a callback with 3 integers
LIB custom::process_data(callback:pointer, context:pointer) -> int

def my_processor(a, b, c)
    println "Processing: " + as_string(a) + ", " + as_string(b) + ", " + as_string(c)
    return a + b + c
end

# Register with custom signature
cb = c_register_callback("main::my_processor", "int,int,int->int")

# Use it - Za automatically generates the closure
result = custom::process_data(cb.trampoline, cb.handle)

c_unregister_callback(cb)

Example: Mixed Type Callback

# Define callback with mixed types
def mixed_callback(int_val, float_val, ptr_val)
    println "Got int: " + as_string(int_val)
    println "Got float: " + as_string(float_val)
    println "Got pointer: " + sf("%p", ptr_val)
    return 42  # Return int
end

# Any combination of supported types works
cb = c_register_callback("main::mixed_callback", "int32,float,ptr->int64")

# Za creates the appropriate closure automatically

Type Mapping Reference

Za Signature Type C Type Size
int8 int8_t 1 byte
uint8 / byte uint8_t 1 byte
int16 int16_t 2 bytes
uint16 uint16_t 2 bytes
int32 / int int32_t / int 4 bytes
uint32 / uint uint32_t / unsigned int 4 bytes
int64 int64_t 8 bytes
uint64 uint64_t 8 bytes
float float 4 bytes
double double 8 bytes
ptr / pointer void* Platform size
string char* Platform size
bool int (0/1) 4 bytes
void void -

F.11.6 Callback Lifecycle

Registration:

cb = c_register_callback(function_name, signature)
# Returns: map with .trampoline and .handle fields

Usage:

# Pass to C function that accepts function pointer + context
c::some_function(cb.trampoline, other_args, cb.handle)

Cleanup:

c_unregister_callback(cb)
# Must be called when C will no longer call the callback

F.11.7 Thread Safety

Important: Callbacks are serialized with a mutex to protect Za’s interpreter. This means:

F.11.8 Limitations

  1. Context parameter required: Most callbacks need C APIs with a context/user_data parameter. Simple callbacks without context (like old signal()) have limited support.

  2. No closure capture: Callback functions cannot capture variables from enclosing scope. They only access their parameters and global variables.

  3. Manual cleanup required: You must call c_unregister_callback() or memory will leak. Unregister before the callback might be called again.

  4. Function must exist: The Za function must remain defined while the callback is registered. Don’t redefine or delete callback functions while in use.

  5. Error handling: Callback errors cannot propagate to C. Errors return safe default values (0, nil) to C.

  6. Limitations: While any signature can be registered, callbacks currently do not support:

  7. Performance consideration: Callback generation adds a small runtime marshaling cost (~1-2μs per call). This is negligible for most use cases.

F.11.9 Best Practices

  1. Always cleanup: Use try/catch blocks to ensure callbacks are unregistered:

    cb = c_register_callback("my_func", "ptr,ptr->int")
    try
        # Use callback
    catch e
        c_unregister_callback(cb)
        throw e
    endtry
    c_unregister_callback(cb)
  2. Check return values: Many C APIs return error codes. Check them before assuming success.

  3. Simple callback logic: Keep callback functions simple. Complex logic increases risk of errors that can’t be reported to C.

  4. Document context usage: When using callbacks with C APIs, document which parameter is the context.

  5. Test thread safety: If C library uses threads, test that callbacks work correctly with concurrent invocations.

F.11.10 Choosing File Operation Functions

Za provides three different ways to open and close files. Understanding when to use each is important for correct behavior.

The Three Variants

Variant Type Returns Use Case
fopen() / fclose() Za built-in pfile (Za file handle) Za file operations (read_file, write_file, etc.)
c_fopen() / c_fclose() Za helper CPointer (FILE*) Convenient wrapper for C file operations
c::fopen() / c::fclose() FFI direct pointer / int Direct C library calls via FFI

When to Use Each

1. Za Built-in: fopen() / fclose()

Use for Za’s file operations. Returns a pfile type.

# Za file operations
fp = fopen("/tmp/data.txt", "w")
fwrite(fp, "Hello, World!")
fclose(fp)

2. Za Helper: c_fopen() / c_fclose()

Convenient wrapper for C FILE* operations. Returns CPointer type with proper type tag.

# Using C library functions with Za helper
fp = c_fopen("/tmp/output.jpg", "wb")
if !c_ptr_is_null(fp)
    gdImageJpeg(image, fp, 90)  # Pass to C library function
    c_fclose(fp)
endif

3. FFI Direct: c::fopen() / c::fclose()

Direct FFI call to libc. Returns raw pointer / int error code.

module "libc.so.6" as c
use +c

LIB c::fopen(filename:string, mode:string) -> pointer
LIB c::fclose(stream:pointer) -> int

fp = c::fopen("/tmp/test.txt", "w")
result = c::fclose(fp)
assert result == 0, "fclose failed"

Common Pitfall: Wrong Function Called

Problem - Namespace collision:

module "libc.so.6" as c
use +c

LIB c::fopen(filename:string, mode:string) -> pointer
LIB c::fclose(stream:pointer) -> int

# WITHOUT namespace qualifier - calls Za's built-in fopen!
fp = fopen("/tmp/file", "w")      # ❌ Returns pfile, not FILE*
result = fclose(fp)                # ❌ Returns void, not int

Solution - Use explicit namespace:

# WITH namespace qualifier - calls C library version
fp = c::fopen("/tmp/file", "w")   # ✓ Returns FILE* pointer
result = c::fclose(fp)             # ✓ Returns int error code
assert result == 0, "fclose failed"

Best Practices

  1. Use explicit namespace qualifiers for FFI calls: c::fopen(), not fopen()
  2. Prefer Za helpers (c_fopen) over direct FFI for most C library usage
  3. Check return types - if function returns unexpected type, you’re calling wrong version
  4. Consult help - use help find fopen to see all available versions

Other Functions with Multiple Variants

Function Za Built-in C Library Behavior Difference
getenv Za environment vars C getenv Different variable sources
system Za system calls C system() Different error handling

See section 24 (“Modules and namespaces”) for namespace resolution rules.

F.12 Limitations and Best Practices

Limitations:

  1. Manual memory management: Unlike Za’s garbage collection, C memory must be manually freed with free() or library-specific cleanup functions.

  2. Preprocessor macros: C #define constants are not symbols in the .so file, but can be extracted using the AUTO clause (see F.3). For simple cases without headers, define them in Za:

    # With AUTO clause (recommended):
    module "libc.so.6" as c auto
    use +c
    # EOF, NULL, etc. now available from stdio.h
    
    # Without AUTO (manual definition):
    NULL = c_null()
    EOF = -1
  3. Platform-specific behavior: Some libraries behave differently across platforms. Test thoroughly.

  4. Struct field access: Za now supports struct marshaling for passing data between Za and C:

  5. Size_t special handling: Functions returning size_t require c_ptr_to_int() conversion.

Best Practices:

  1. Prefer Za standard library: If Za has built-in functionality, use it instead of FFI.

  2. Check for null pointers: Always verify pointers before use:

    ptr = c::malloc(1000)
    if c_ptr_is_null(ptr)
        println "Allocation failed"
        exit 1
    endif
  3. Free all allocated memory: Track allocations and ensure cleanup:

    buf = c::malloc(1000)
    try
        # Use buffer
    catch e
        c::free(buf)
        throw e
    endtry
    c::free(buf)
  4. Document C library versions: Different library versions may have incompatible ABIs.

  5. Test edge cases: C libraries often have undefined behavior on invalid inputs. Test boundary conditions.

  6. Prefer mut for AUTO structs: When using AUTO-registered struct types, use var + mut pattern instead of manual marshaling for cleaner code and better performance. See section F.4.5 for details.

  7. Use help plugin for discovery: Leverage help plugin find to get correct signatures from man pages.

  8. Wrap C operations in Za functions: Create higher-level abstractions:

    
    def json_create_person(name, age)
        obj = json::json_object_new_object()
        json::json_object_object_add(obj, "name",
            json::json_object_new_string(name))
        json::json_object_object_add(obj, "age",
            json::json_object_new_int(age))
        return obj
    end
  9. When in doubt, use shell commands: If FFI becomes complex, consider using Za’s shell integration instead.

  10. Keep local copies for distribution: When distributing FFI-capable scripts, include local copies of both .so and .h files in the same directory as your script:

# Use relative paths for portability
module "./mylib.so" as mylib auto "./mylib.h"
use +mylib

Benefits:

F.13 Platform Support and Build Requirements

Unix/Linux:

Windows:

No-FFI Builds:

Build flags:

//go:build !windows && !noffi && cgo

FFI requires:

Checking for FFI support: If FFI is not available, MODULE statements will produce an error message indicating the feature is not supported in this build.

Fallback strategies: If FFI is unavailable:


This appendix documents Za’s experimental FFI feature. The API may change in future versions. Test thoroughly and refer to test files in za_tests/test_ffi*.za for additional examples.