neuralogic.nn package

Subpackages

Submodules

neuralogic.nn.init module

class Constant(value: float = 0.1)[source]

Bases: Initializer

Initializes learnable parameters with the value.

Parameters:: value (float) – Value to fill weights with. Default: 0.1

get_settings() → Dict[str, Any][source]

class Glorot(scale: float = 2)[source]

Bases: Initializer

Initializes learnable parameters with samples from a uniform distribution (from the interval [-scale / 2, scale / 2]) using the Glorot method.

Parameters:: scale (float) – Scale of a uniform distribution interval [-scale / 2, scale / 2]. Default: 2

get_settings() → Dict[str, Any][source]

is_simple() → bool[source]

class He(scale: float = 2)[source]

Bases: Initializer

Initializes learnable parameters with samples from a uniform distribution (from the interval [-scale / 2, scale / 2]) using the He method.

Parameters:: scale (float) – Scale of a uniform distribution interval [-scale / 2, scale / 2]. Default: 2

get_settings() → Dict[str, Any][source]

is_simple() → bool[source]

class Initializer[source]

Bases: object

get_settings() → Dict[str, Any][source]

is_simple() → bool[source]

class InitializerNames[source]

Bases: object

CONSTANT = 'CONSTANT'

GLOROT = 'GLOROT'

HE = 'HE'

LONGTAIL = 'LONGTAIL'

NORMAL = 'NORMAL'

TORCH = 'TORCH'

UNIFORM = 'UNIFORM'

class Longtail[source]

Bases: Initializer

Initializes learnable parameters with random samples from a long tail distribution

class Normal[source]

Bases: Initializer

Initializes learnable parameters with random samples from a normal (Gaussian) distribution

class Torch[source]

Bases: Initializer

Initializes learnable parameters uniformly from [-1 / sqrt(fan_in), 1 / sqrt(fan_in)], which is what every layer PyTorch ships draws from.

torch.nn.Linear uses kaiming_uniform_(a=sqrt(5)), whose bound works out at exactly 1 / sqrt(fan_in), and torch.nn.RNN, LSTM and GRU draw every weight from U(+-1 / sqrt(hidden_size)), which is the same for their square recurrent weight. Matching it means a model written here and the same model written there also start from the same place.

A matrix takes its fan_in from its columns, which are the inputs it consumes. A vector keeps only one of its two declared dimensions, so a weight declared (1, n) consumes all n and is drawn from 1 / sqrt(n), while one declared (n, 1) - or as a plain (n) - consumes one and keeps the full [-1, 1], exactly as torch.nn.Linear(1, n) does. A scalar likewise keeps [-1, 1].

So only the weights whose fan-in is both unambiguous and large differ from Uniform at all, which are the ones a wide template saturates on.

Not the default, and not for anything it fails at. Glorot is, because sqrt(6/(fan_in+fan_out)) comes to sqrt(3/fan_in) for a square weight, the constant that actually keeps a uniformly drawn layer’s output variance equal to its input’s - where 1/sqrt(fan_in) is sqrt(3) under it, 0.125 against 0.2165 on a 64x64. Torch’s own source points at pytorch issue 57109 about that. Reach for this one when a model here has to begin from the same distribution as the same model written in torch; the activation correction applies to either.

Unlike Uniform, Glorot and He, this takes no scale - the whole point is that the spread follows the shape of the weight rather than a number chosen in advance.

is_simple() → bool[source]

class Uniform(scale: float = 2)[source]

Bases: Initializer

Initializes learnable parameters with random uniformly distributed samples from the interval [-scale / 2, scale / 2].

Parameters:: scale (float) – Scale of the distribution interval [-scale / 2, scale / 2]. Default: 2

get_settings() → Dict[str, Any][source]

neuralogic.nn.loss module

class CrossEntropy(with_logits: bool = True, reduction: str = 'mean')[source]

Bases: ErrorFunction

Cross Entropy loss function. Suitable for classification tasks.

class ErrorFunction(reduction: str = 'mean')[source]

Bases: object

Base class for error (loss) functions in the neural network.

Every one of them takes a reduction, as in torch, and it means the same thing: how a batch’s per-query errors are reduced into the single quantity being descended, and therefore what the gradient is. "mean" divides by the total element count and "sum" does not divide at all.

The divisor generalises torch’s N x C to the two things a relational engine has and torch does not - a query’s importance, and a target whose width can differ between queries - as the sum of importance times width over the batch. With unit importances and equal widths that is exactly N x C.

Note the per-query values that validate() and train() hand back are not reduced across the batch: they are the un-aggregated errors, each summed over its own components, which is torch’s reduction="none". The reduction governs the gradient and any aggregate over the batch.

class ErrorFunctionNames[source]

Bases: object

CROSSENTROPY = 'CROSSENTROPY'

MSE = 'SQUARED_DIFF'

SOFTENTROPY = 'SOFTENTROPY'

class MSE(reduction: str = 'mean')[source]

Bases: ErrorFunction

Mean Squared Error (SQUARED_DIFF) loss function. Suitable for regression tasks.

Parameters:: reduction (str) – "mean" or "sum", as in torch, and it changes the gradient. Default: "mean".

class SoftEntropy(reduction: str = 'mean')[source]

Bases: ErrorFunction

Soft Entropy loss function. Similar to Cross Entropy but usually applied with a soft layer at the end.

Parameters:: reduction (str) – "mean" or "sum", as in torch, and it changes the gradient. Default: "mean".

neuralogic.nn.torch_function module

Module contents

Neural network building blocks — modules, losses, initializers, and optimizers.

All primary symbols are importable directly from neuralogic.nn:

from neuralogic.nn import GCNConv, LSTM, MSE, Uniform, Adam

class APPNPConv(output_name: str, feature_name: str, edge_name: str, k: int, alpha: float, activation: TransformationFunction = <neuralogic.core.constructs.function.function.TransformationFunction object>, aggregation: AggregationFunction = <neuralogic.core.constructs.function.function.AggregationFunction object>, add_self_loops: bool | None = None, normalize: bool = True)[source]

Bases: Module

Approximate Personalized Propagation of Neural Predictions layer from “Predict then Propagate: Graph Neural Networks meet Personalized PageRank”. Which can be expressed as:

\[\mathbf{x}^{0}_i = \mathbf{x}_i\]

\[\mathbf{x}^{k}_i = \alpha \cdot \mathbf{x}^0_i + (1 - \alpha) \cdot {agg}_{j \in \mathcal{N}(i)}(\mathbf{x}^{k - 1}_j)\]

\[\mathbf{x}^{\prime}_i = act(\mathbf{x}^{K}_i)\]

Where act is an activation function and agg aggregation function.

The first part of the second equation that is “\(\alpha \cdot \mathbf{x}^0_i\)” is expressed in the logic form as:

R.<output_name>__<k>(V.I) <= R.<feature_name>(V.I)[<alpha>].fixed()

The second part of the second equation that is “\((1 - \alpha) \cdot {agg}_{j \in \mathcal{N}(i)}(\mathbf{x}^{k - 1}_j)\)” is expressed as:

R.<output_name>__<k>(V.I) <= (R.<output_name>__<k-1>(V.J)[1 - <alpha>].fixed(), R.<edge_name>(V.J, V.I))

Examples

The whole computation of this module (parametrized as APPNPConv("h1", "h0", "_edge", 3, 0.1, Transformation.SIGMOID)) is as follows:

metadata = Metadata(transformation=Transformation.IDENTITY, aggregation=Aggregation.SUM)

(R.h1__1(V.I) <= R.h0(V.I)[0.1].fixed()) | metadata
(R.h1__1(V.I) <= (R.h0(V.J)[0.9].fixed(), R._edge(V.J, V.I))) | metadata
R.h1__1/1 [Transformation.IDENTITY]

(R.h1__2(V.I) <= <0.1> R.h0(V.I)) | metadata
(R.h1__2(V.I) <= (<0.9> R.h1__1(V.J), R._edge(V.J, V.I))) | metadata
R.h1__2/1 [Transformation.IDENTITY]

(R.h1(V.I) <= <0.1> R.h0(V.I)) | metadata
(R.h1(V.I) <= (<0.9> R.h1__2(V.J), R._edge(V.J, V.I))) | metadata
R.h1 / 1 [Transformation.SIGMOID]

Parameters:

output_name (str) – Output (head) predicate name of the module.
feature_name (str) – Feature predicate name to get features from.
edge_name (str) – Edge predicate name to use for neighborhood relations.
k (int) – Number of iterations
alpha (float) – Teleport probability
activation (TransformationFunction) – Activation function of the output. Default: Transformation.IDENTITY
aggregation (AggregationFunction) – Aggregation function of nodes’ neighbors. Default: Aggregation.SUM

class Adam(lr: float = 0.001, betas: tuple[float, float] = (0.9, 0.999), eps: float = 1e-08, lr_decay: LRDecay | None = None, weight_decay: float = 0.0)[source]

Bases: Optimizer

Adam optimizer. It implements the Adaptive Moment Estimation (Adam) algorithm.

property betas: tuple[float, float]

property eps: float

initialize() → Any[source]

Initializes the Java representation of the Adam optimizer.

Returns:: The Java optimizer object.
Return type:: Any

class Attention(embed_dim: int, output_name: str, query_name: str, key_name: str, value_name: str, mask_name: str | None = None, arity: int = 1)[source]

Bases: Module

A single-head attention module based on “Attention Is All You Need”.

Parameters:

embed_dim (int) – The number of expected features.
output_name (str) – Output (head) predicate name of the module.
query_name (str) – The name of the queries predicate.
key_name (str) – The name of the keys predicate.
value_name (str) – The name of the values predicate.
mask_name (str, optional) – The name of the input mask predicate. Default: None
arity (int) – Arity of the input and output predicates. Default: 1

class AvgPooling(output_name: str, input_name: str, input_arity: int = 1)[source]

Bases: Pooling

Apply average pooling over the input specified by the input_name and the input arity parameters. Can be expressed as:

\[h = \frac{1}{|N|}\sum_{i_{0}, .., i_{n} \in N} x_{(i_{0}, .., i_{n})}\]

Where \(N\) is a set of tuples of length \(n\) (specified by the input arity parameter) that are valid arguments for the input predicate.

This module extends the generic pooling Pooling.

Examples

The whole computation of this module (parametrized as AvgPooling("h1", "h0")) is as follows:

(R.h1 <= R.h0(V.X0)) | [Aggregation.AVG, Transformation.IDENTITY]
R.h1 / 0 | [Transformation.IDENTITY]

Parameters:

output_name (str) – Output (head) predicate name of the module.
input_name (str) – Input name.
input_arity (int) – Arity of the input predicate input_name. Default: 1

class CheckpointCallback(directory: str | Path, filename: str = 'best.pkl')[source]

Bases: TrainerCallback

Save the model whenever validation loss improves.

Parameters:

directory (str or Path) – Directory to save checkpoints into (created if missing).
filename (str) – Filename for the checkpoint file (default "best.pkl").

on_epoch_end(trainer: Trainer, epoch: int, logs: dict[str, Any]) → None[source]

Called after every epoch.

Parameters:

trainer (Trainer) – The trainer instance (access trainer.model, etc.).
epoch (int) – 0-indexed epoch number that just finished.
logs (dict) – Dictionary with keys "train_loss", "val_loss" (if available), "lr", and per-metric keys like "train_accuracy", "val_mae", etc.

on_train_begin(trainer: Trainer) → None[source]: Called once before the first epoch.

property path: Path

class Constant(value: float = 0.1)[source]

Bases: Initializer

Initializes learnable parameters with the value.

Parameters:: value (float) – Value to fill weights with. Default: 0.1

get_settings() → Dict[str, Any][source]

class CrossEntropy(with_logits: bool = True, reduction: str = 'mean')[source]

Bases: ErrorFunction

Cross Entropy loss function. Suitable for classification tasks.

class EarlyStoppingCallback(patience: int, min_delta: float = 0.0)[source]

Bases: TrainerCallback

Stop training when validation loss stops improving.

Parameters:

patience (int) – Number of epochs with no improvement after which training stops.
min_delta (float) – Minimum absolute change to qualify as an improvement.

on_epoch_end(trainer: Trainer, epoch: int, logs: dict[str, Any]) → None[source]

Called after every epoch.

Parameters:

trainer (Trainer) – The trainer instance (access trainer.model, etc.).
epoch (int) – 0-indexed epoch number that just finished.
logs (dict) – Dictionary with keys "train_loss", "val_loss" (if available), "lr", and per-metric keys like "train_accuracy", "val_mae", etc.

on_train_begin(trainer: Trainer) → None[source]: Called once before the first epoch.

class ErrorFunction(reduction: str = 'mean')[source]

Bases: object

Base class for error (loss) functions in the neural network.

Every one of them takes a reduction, as in torch, and it means the same thing: how a batch’s per-query errors are reduced into the single quantity being descended, and therefore what the gradient is. "mean" divides by the total element count and "sum" does not divide at all.

The divisor generalises torch’s N x C to the two things a relational engine has and torch does not - a query’s importance, and a target whose width can differ between queries - as the sum of importance times width over the batch. With unit importances and equal widths that is exactly N x C.

Note the per-query values that validate() and train() hand back are not reduced across the batch: they are the un-aggregated errors, each summed over its own components, which is torch’s reduction="none". The reduction governs the gradient and any aggregate over the batch.

class GATv2Conv(in_channels: int, out_channels: int, output_name: str, feature_name: str, edge_name: str, share_weights: bool = False, activation: TransformationFunction = <neuralogic.core.constructs.function.function.TransformationFunction object>, add_self_loops: bool = True, negative_slope: float = 0.2)[source]

Bases: Module

GATv2 layer from “How Attentive are Graph Attention Networks?”.

\[\alpha_{ij} = \mathrm{softmax}_{j \in \mathcal{N}(i) \cup \{i\}} \left( \mathbf{a}^{\top} \mathrm{LeakyReLU} \left( \mathbf{W}_{l} \mathbf{x}_{i} + \mathbf{W}_{r} \mathbf{x}_{j} \right) \right)\]

\[\mathbf{x}^{\prime}_{i} = act \left( \sum_{j \in \mathcal{N}(i) \cup \{i\}} \alpha_{ij} \mathbf{W}_{r} \mathbf{x}_{j} \right)\]

The score is one number per edge, so \(\mathbf{a}\) is a 1 x out_channels weight and it sits on the body of the rule that normalizes it, not on the head: a head weight is applied by the atom neuron after the aggregation, and the softmax aggregation casts every input it is given to a scalar. The normalization is Aggregation.SOFTMAX with agg_terms, which groups the groundings by the terms it is not given - so over the neighbours J for each target I, while staying one value per edge. That is the same shape Attention uses.

The edge appears as a hidden atom in both rules, and only to restrict the grounding: without it the score would ground for every pair of nodes and the softmax would normalize over the whole graph rather than over a neighbourhood. PyG’s GATv2 takes edge features through a projection rather than an edge weight, so there is nothing for the edge’s own value to mean here.

The slope defaults to PyG’s 0.2, not to the backend’s own 0.01: this module exists to be PyG’s layer, and quietly using a different slope made every comparison bend PyG instead. It is a per-rule slope now, so negative_slope says what it says.

Parameters:

in_channels (int) – Input feature size.
out_channels (int) – Output feature size.
output_name (str) – Output (head) predicate name of the module.
feature_name (str) – Feature predicate name to get features from.
edge_name (str) – Edge predicate name to use for neighborhood relations.
share_weights (bool) – Share weights in attention. Default: False
activation (TransformationFunction) – Activation function of the output. Default: Transformation.IDENTITY
add_self_loops (bool) – Let a node attend to itself, as PyG does. Default: True
negative_slope (float) – The LeakyReLU slope of the attention score. Default: 0.2, as in PyG.

class GCNConv(in_channels: int, out_channels: int, output_name: str, feature_name: str, edge_name: str, activation: TransformationFunction = <neuralogic.core.constructs.function.function.TransformationFunction object>, aggregation: AggregationFunction = <neuralogic.core.constructs.function.function.AggregationFunction object>, add_self_loops: bool | None = None, normalize: bool = True)[source]

Bases: Module

Graph Convolutional layer from “Semi-supervised Classification with Graph Convolutional Networks”.

\[\mathbf{X}^{\prime} = act \left( \mathbf{\hat{D}}^{-1/2} \mathbf{\hat{A}} \mathbf{\hat{D}}^{-1/2} \mathbf{X} \mathbf{W} \right)\]

where \(\mathbf{\hat{A}} = \mathbf{A} + \mathbf{I}\). The body combines by product, so an edge’s value is PyG’s edge_weight: it scales the message it carries and it enters the degree. Note __edge_count is a weighted **in-**degree despite its name - the two rules sum over V.X in the source position, and they sum the edge values rather than counting the edges, both of which is what gcn_norm does. Neither is visible on an undirected graph given both directions at 1.0, where in-degree is out-degree and summing ones is counting them, which is why the name survived being either.

Parameters:

in_channels (int) – Input feature size.
out_channels (int) – Output feature size.
output_name (str) – Output (head) predicate name of the module.
feature_name (str) – Feature predicate name to get features from.
edge_name (str) – Edge predicate name to use for neighborhood relations.
activation (TransformationFunction) – Activation function of the output. Default: Transformation.IDENTITY
aggregation (AggregationFunction) – Aggregation function of nodes’ neighbors. Default: Aggregation.SUM
add_self_loops (bool | None) – Add self loops if either set to True or None (if normalize is True). Default: None
normalize (bool) – Add normalization. Default : True

class GENConv(in_channels: int, out_channels: int, output_name: str, feature_name: str, edge_name: str, aggregation: AggregationFunction = <neuralogic.core.constructs.function.softmax.SoftmaxAggregation object>, num_layers: int = 2, expansion: int = 2, eps: float = 1e-07, train_eps: bool = False, edge_dim: int | None = None)[source]

Bases: Module

GENConv layer from “DeeperGCN: All You Need to Train Deeper GCNs”.

Warning

This module cannot be evaluated on a graph whose features are vectors. Building a dataset for it throws ClassCastException: VectorValue cannot be cast to ScalarValue, so it computes nothing - it is not divergent from PyG, it is unusable. Every other GNN module here is checked against torch_geometric in tests/alignment/test_gnn_alignment.py; this one cannot be.

The cause is not a defect and lifting it would not be a fix. Aggregation.SOFTMAX normalises over groundings, so it wants one number per grounding - a softmax over a set of scalars, which is its only reading, and the implementation is built that way throughout: an n x n Jacobian and a scalar handed back to each input. The __gen_feat_agg rule below asks it to softmax vector-valued groundings instead.

What the module is written against is PyG’s SoftmaxAggregation, which is a different operator: softmax(x, index, dim=-2) then sum(x * alpha), i.e. one independent softmax per feature channel, weighting the neighbours separately for each. That is a per-channel softmax-weighted mean rather than attention. Offering it would mean adding an element-wise softmax aggregation as its own function - a decision about what the library should provide, not a bug to close.

GATv2Conv shows the shape that does work within the restriction: project the per-edge score to one number before the aggregation, with the weight on the rule’s body rather than its head.

Parameters:

in_channels (int) – Input feature size.
out_channels (int) – Output feature size.
output_name (str) – Output (head) predicate name of the module.
feature_name (str) – Feature predicate name to get features from.
edge_name (str) – Edge predicate name to use for neighborhood relations.
aggregation (AggregationFunction) – The aggregation function. Default: Aggregation.SOFTMAX
num_layers (int) – The number of MLP layers. Default: 2
expansion (int) – The expansion factor of hidden channels in MLP. Default: 2
eps (float) – \(\epsilon\)-value. Default: 0.0
train_eps (bool) – Is eps trainable parameter. Default: false
edge_dim (int | None) – Dimension of edge features (None is projection to in_channels is not needed). Default: None

class GINConv(in_channels: int, out_channels: int, output_name: str, feature_name: str, edge_name: str, activation: TransformationFunction = <neuralogic.core.constructs.function.function.TransformationFunction object>, aggregation: AggregationFunction = <neuralogic.core.constructs.function.function.AggregationFunction object>)[source]

Bases: Module

Implements the Graph Isomorphism Network (GIN) convolution layer. GIN is a powerful GNN layer that can distinguish between different graph structures.

class GINEConv(in_channels: int, feature_name: str, edge_name: str, nn_name: str, eps: float = 0.0, train_eps: bool = False, edge_dim: int | None = None)[source]

Bases: Module

GINEConv layer from “Strategies for Pre-training Graph Neural Networks”.

Parameters:

in_channels (int) – Input feature size.
out_channels (int) – Output feature size.
feature_name (str) – Feature predicate name to get features from.
edge_name (str) – Edge predicate name to use for neighborhood relations.
nn_name (str) – Neural network predicate name.
eps (float) – \(\epsilon\)-value. Default: 0.0
train_eps (bool) – Is eps trainable parameter. Default: false
edge_dim (int | None) – Dimension of edge features (None is projection to in_channels is not needed). Default: None

class GRU(input_size: int, hidden_size: int, output_name: str, input_name: str, hidden_0_name: str, arity: int = 1, next_name: str = '_next__positive')[source]

Bases: Module

One-layer Gated Recurrent Unit (GRU) module which is computed as:

\[\begin{split}r_t = \sigma(\mathbf{W}_{xr} \mathbf{x}_t + \mathbf{W}_{hr} \mathbf{h}_{t-1}) \\\end{split}\]

\[\begin{split}z_t = \sigma(\mathbf{W}_{xz} \mathbf{x}_t + \mathbf{W}_{hz} \mathbf{h}_{t-1}) \\\end{split}\]

\[\begin{split}n_t = \tanh(\mathbf{W}_{xn} \mathbf{x}_t + r_t \odot (\mathbf{W}_{hn} \mathbf{h}_{t-1})) \\\end{split}\]

\[h_t = (1 - z_t) \odot n_t + z_t \odot h_{t-1}\]

where \(t \in (1, sequence\_length + 1)\) is a time step. In the template, the \(t\) is referred to as V.T, and \(t - 1\) is referred to as V.Z. This module expresses the first equation as:

(R.<output_name>__r(<...terms>, V.T) <= (
    R.<input_name>(<...terms>, V.T)[<hidden_size>, <input_size>],
    R.<hidden_input_name>(<...terms>, V.Z)[<hidden_size>, <hidden_size>],
    R.<next_name>(V.Z, V.T),
)) | [Transformation.SIGMOID]

R.<output_name>__r / <arity> + 1 | [Transformation.IDENTITY]

The second equation is expressed in the same way, except for a different head predicate name. The third equation is split into three rules. The first two computes the element-wise product - \(r_t * (\mathbf{W}_{hn} \mathbf{h}_{t-1})\).

(R.<output_name>__n_helper_weighted(<...terms>, V.T) <= (
    R.<hidden_input_name>(<...terms>, V.Z)[<hidden_size>, <hidden_size>], R.<next_name>(V.Z, V.T),
)) | [Transformation.IDENTITY],

R.<output_name>__n_helper_weighted / (<arity> + 1) | [Transformation.IDENTITY],

(R.<output_name>__n_helper(<...terms>, V.T) <= (
    R.<output_name>__r(<..terms>, V.T), R.<>__n_helper_weighted(<...terms>, V.T)
)) | [Transformation.IDENTITY, Combination.ELPRODUCT],

R.<output_name>__n_helper / (<arity> + 1) | [Transformation.IDENTITY],

The third one computes the sum and applies the \(tanh\) activation function.

(R.<output_name>__n(<...terms>, V.T) <= (
    R.<input_name>(<...terms>, V.T)[<hidden_size>, <input_size>],
    R.<output_name>__n_helper(<...terms>, V.T)
)) | [Transformation.TANH]
R.<output_name>__n / (<arity> + 1) | [Transformation.IDENTITY],

The last equation is computed via three rules. The first two rules computes element-wise products. That is:

(R.<output_name>__left(<...terms>, V.T) <= (
    R.<output_name>__z(<...terms>, V.T), R.<output_name>__n(<...terms>, V.T)
)) | [Transformation.IDENTITY, Combination.ELPRODUCT]

(R.<output_name>__right(<...terms>, V.T) <= (
    R.<output_name>__z(<...terms>, V.T), R.<hidden_input_name>(<...terms>, V.Z), R.<next_name>(V.Z, V.T),,
)) | [Transformation.IDENTITY, Combination.ELPRODUCT]

R.<output_name>__left / <arity> + 1 | [Transformation.IDENTITY]
R.<output_name>__right / <arity> + 1 | [Transformation.IDENTITY]

The last output rule sums up the element-wise products.

(R.<output_name>(<...terms>, V.T) <= (
    R.<output_name>__left(<...terms>, V.T), R.<output_name>__right(<...terms>, V.T)
)) | [Transformation.IDENTITY]
R.<output_name> / <arity> + 1 | [Transformation.IDENTITY],

Additionally, we define a rule for the “stop condition”, that is:

(R.<output_name>(<...terms>, 0) <= R.<hidden_0_name>(<...terms>)) | [Transformation.IDENTITY]

Parameters:

input_size (int) – Input feature size.
hidden_size (int) – Output and hidden feature size.
output_name (str) – Output (head) predicate name of the module.
input_name (str) – Input feature predicate name to get features from.
hidden_0_name (str) – Predicate name to get initial hidden state from.
arity (int) – Arity of the input and output predicate. Default: 1

class Glorot(scale: float = 2)[source]

Bases: Initializer

Initializes learnable parameters with samples from a uniform distribution (from the interval [-scale / 2, scale / 2]) using the Glorot method.

Parameters:: scale (float) – Scale of a uniform distribution interval [-scale / 2, scale / 2]. Default: 2

get_settings() → Dict[str, Any][source]

is_simple() → bool[source]

class He(scale: float = 2)[source]

Bases: Initializer

Initializes learnable parameters with samples from a uniform distribution (from the interval [-scale / 2, scale / 2]) using the He method.

Parameters:: scale (float) – Scale of a uniform distribution interval [-scale / 2, scale / 2]. Default: 2

get_settings() → Dict[str, Any][source]

is_simple() → bool[source]

class Initializer[source]

Bases: object

get_settings() → Dict[str, Any][source]

is_simple() → bool[source]

class LSTM(input_size: int, hidden_size: int, output_name: str, input_name: str, hidden_0_name: str, cell_state_0_name: str, arity: int = 1)[source]

Bases: Module

One-layer Long Short-Term Memory (LSTM) RNN module which is computed as:

\[i_t = \sigma(\mathbf{W}_{xi} \mathbf{x}_t + \mathbf{W}_{hi} \mathbf{h}_{t-1})\]

\[f_t = \sigma(\mathbf{W}_{xf} \mathbf{x}_t + \mathbf{W}_{hf} \mathbf{h}_{t-1})\]

\[o_t = \sigma(\mathbf{W}_{xo} \mathbf{x}_t + \mathbf{W}_{ho} \mathbf{h}_{t-1})\]

\[\begin{split}g_t = \tanh(\mathbf{W}_{xg} \mathbf{x}_t + \mathbf{W}_{hg} \mathbf{h}_{t-1}) \\\end{split}\]

\[c_t = f_t \odot c_{t-1} + i_t \odot g_t\]

\[h_t = o_t \odot \tanh(c_t)\]

Parameters:

input_size (int) – Input feature size.
hidden_size (int) – Output and hidden feature size.
output_name (str) – Output (head) predicate name of the module.
input_name (str) – Input feature predicate name to get features from.
hidden_0_name (str) – Predicate name to get initial hidden state from.
cell_state_0_name (str) – Predicate name to get initial cell state from.
arity (int) – Arity of the input and output predicate. Default: 1

class Linear(in_channels: int, out_channels: int, output_name: str, input_name: str, activation: TransformationFunction = <neuralogic.core.constructs.function.function.TransformationFunction object>, arity: int = 1)[source]

Bases: Module

Apply linear transformation on the input. Can be expressed as:

\[h_{i_0, .., i_{n}} = W \cdot x_{i_0, .., i_{n}}\]

Where \(x\) is the input, \(W \in R^{(out\_channels \times in\_channels)}\) is a learnable parameter, and \(n\) is the arity of the input and output.

It is also possible to attach non-linearity via the activation parameter and compute:

\[h_{i_0, .., i_{n}} = act(W \cdot x_{i_0, .., i_{n}})\]

Example

The whole computation of this module (parametrized as Linear(1, 2, "h1", "h0")) is as follows:

(R.h1(V.X0)[2, 1] <= R.h0(V.X0)) | [Transformation.IDENTITY]
R.h1 / 1 | [Transformation.IDENTITY]

Module parametrized as Linear(1, 2, "h1", "h0", Transformation.SIGMOID, 2) translates into:

(R.h1(V.X0, V.X1)[2, 1] <= R.h0(V.X0, V.X1)) | [Transformation.IDENTITY]
R.h1 / 2 | [Transformation.SIGMOID]

Parameters:

in_channels (int) – Input feature size.
out_channels (int) – Output feature size.
output_name (str) – Output (head) predicate name of the module.
input_name (str) – Input name.
activation (TransformationFunction) – Activation function of the output. Default: Transformation.IDENTITY
arity (int) – Arity of the input and output predicate. Default: 1

class Longtail[source]

Bases: Initializer

Initializes learnable parameters with random samples from a long tail distribution

class MAGNNLinear(in_channels: int, out_channels: int, output_name: str, feature_name: str, relation_name: str, type_name: str | None, meta_paths: ~typing.List[str], activation: ~neuralogic.core.constructs.function.function.TransformationFunction = <neuralogic.core.constructs.function.function.TransformationFunction object>, aggregation: ~neuralogic.core.constructs.function.function.AggregationFunction = <neuralogic.core.constructs.function.function.AggregationFunction object>)[source]

Bases: MAGNNMean

Intra-metapath Aggregation module with Linear encoder from “MAGNN: Metapath Aggregated Graph Neural Network for Heterogeneous Graph Embedding”. Which can be expressed as:

\[\mathbf{h}_{P(v,u)} = \mathbf{W}_p \cdot MEAN(\{\mathbf{x}_t | \forall t \in P(v,u) \})\]

\[\mathbf{h}^P_{v} = act(\sum_{u \in N^P_v} \mathbf{h}_{P(v,u)})\]

Where act is an activation function, \(P(v,u)\) is a single metapath instance, \(N^P_{v}\) is set of metapath-based neighbors.

Parameters:

in_channels (int) – Input feature size.
out_channels (int) – Output feature size.
output_name (str) – Output (head) predicate name of the module.
feature_name (str) – Feature predicate name to get features from.
relation_name (str) – Relation predicate name for connectivity checks between entities.
type_name (Optional[str]) – Metapath type predicate name. If none, meta_paths will be used instead.
meta_paths (List[str]) – Name of types forming a single metapath.
activation (TransformationFunction) – Activation function of the output. Default: Transformation.SIGMOID
aggregation (AggregationFunction) – Aggregation function of the output. Default: Aggreagtion.SUM

class MAGNNMean(output_name: str, feature_name: str, relation_name: str, type_name: str | None, meta_paths: ~typing.List[str], activation: ~neuralogic.core.constructs.function.function.TransformationFunction = <neuralogic.core.constructs.function.function.TransformationFunction object>, aggregation: ~neuralogic.core.constructs.function.function.AggregationFunction = <neuralogic.core.constructs.function.function.AggregationFunction object>)[source]

Bases: Module

Intra-metapath Aggregation module with Mean encoder from “MAGNN: Metapath Aggregated Graph Neural Network for Heterogeneous Graph Embedding”. Which can be expressed as:

\[\mathbf{h}_{P(v,u)} = MEAN(\{\mathbf{x}_t | \forall t \in P(v,u) \})\]

\[\mathbf{h}^P_{v} = act(\sum_{u \in N^P_v} \mathbf{h}_{P(v,u)})\]

Where act is an activation function, \(P(v,u)\) is a single metapath instance, \(N^P_{v}\) is set of metapath-based neighbors.

Parameters:

output_name (str) – Output (head) predicate name of the module.
feature_name (str) – Feature predicate name to get features from.
relation_name (str) – Relation predicate name for connectivity checks between entities.
type_name (Optional[str]) – Metapath type predicate name. If none, meta_paths will be used instead.
meta_paths (List[str]) – Name of types forming a single metapath.
activation (TransformationFunction) – Activation function of the output. Default: Transformation.SIGMOID
aggregation (AggregationFunction) – Aggregation function of the output. Default: Aggreagtion.SUM

class MLP(units: ~typing.List[int], output_name: str, input_name: str, activation: ~neuralogic.core.constructs.function.function.TransformationFunction | ~typing.List[~neuralogic.core.constructs.function.function.TransformationFunction] = <neuralogic.core.constructs.function.function.TransformationFunction object>, arity: int = 1)[source]

Bases: Module

Parameters:

units (List[int]) – List of layer sizes.
output_name (str) – Output (head) predicate name of the module.
input_name (str) – Input name.
activation (Union[TransformationFunction, List[TransformationFunction]]) – Activation function of all layers or list of activations for each layer. Default: Transformation.RELU
arity (int) – Default: -1

class MSE(reduction: str = 'mean')[source]

Bases: ErrorFunction

Mean Squared Error (SQUARED_DIFF) loss function. Suitable for regression tasks.

Parameters:: reduction (str) – "mean" or "sum", as in torch, and it changes the gradient. Default: "mean".

class MaxPooling(output_name: str, input_name: str, input_arity: int = 1)[source]

Bases: Pooling

Apply max pooling over the input specified by the input_name and the input arity parameters. Can be expressed as:

\[h = max_{i_{0}, .., i_{n} \in N}(x_{(i_{0}, .., i_{n})})\]

Where \(N\) is a set of tuples of length \(n\) (specified by the input arity parameter) that are valid arguments for the input predicate.

This module extends the generic pooling Pooling.

Examples

The whole computation of this module (parametrized as MaxPooling("h1", "h0")) is as follows:

(R.h1 <= R.h0(V.X0)) | [Aggregation.MAX, Transformation.IDENTITY]
R.h1 / 0 | [Transformation.IDENTITY]

Parameters:

output_name (str) – Output (head) predicate name of the module.
input_name (str) – Input name.
input_arity (int) – Arity of the input predicate input_name. Default: 1

class MetaConv(in_channels: int, out_channels: int, output_name: str, feature_name: str, role_name: str | None, roles: ~typing.List[str], activation: ~neuralogic.core.constructs.function.function.TransformationFunction = <neuralogic.core.constructs.function.function.TransformationFunction object>, aggregation: ~neuralogic.core.constructs.function.function.AggregationFunction = <neuralogic.core.constructs.function.function.AggregationFunction object>)[source]

Bases: Module

Metagraph Convolutional Unit layer.

From Meta-GNN: metagraph neural network for semi-supervised learning in attributed heterogeneous information networks. Which can be expressed as:

\[\mathbf{x}^{\prime}_i = act(\mathbf{W_0} \cdot \mathbf{x}_i + {agg}_{j \in \mathcal{N}_r(i)} \sum_{k \in \mathcal{K}} (\mathbf{W_k} \cdot \mathbf{x}_j))\]

Where act is an activation function, agg aggregation function (by default average), \(W_0\) is a learnable root parameter and \(W_k\) is a learnable parameter for each role.

Parameters:

in_channels (int) – Input feature size.
out_channels (int) – Output feature size.
output_name (str) – Output (head) predicate name of the module.
feature_name (str) – Feature predicate name to get features from.
role_name (str, optional) – Role predicate name to use for role relations. When None, elements from roles are used instead.
roles (List[str]) – List of relations’ names.
activation (TransformationFunction, optional) – Activation function of the output. Default: Transformation.SIGMOID.
aggregation (AggregationFunction, optional) – Aggregation function of nodes’ neighbors. Default: Aggregation.AVG.

class Metric(value)[source]

Bases: str, Enum

Enum of available metric names.

Inherits str so members can be used directly where a string is expected:

>>> Metric.ACCURACY == "accuracy"
True
>>> trainer.fit(..., metrics=[Metric.ACCURACY, Metric.F1_MACRO])

ACCURACY = 'accuracy'

AUC_PR = 'auc_pr'

AUC_ROC = 'auc_roc'

BEST_ACCURACY = 'best_accuracy'

BEST_THRESHOLD = 'best_threshold'

F1_MACRO = 'f1_macro'

MAE = 'mae'

MSE = 'mse'

PRECISION_MACRO = 'precision_macro'

R2 = 'r2'

RECALL_MACRO = 'recall_macro'

RMSE = 'rmse'

SATURATION = 'saturation'

class Module[source]: Bases: ABC

class MultiheadAttention(embed_dim: int, num_heads: int, output_name: str, query_name: str, key_name: str, value_name: str, vdim: int | None = None, kdim: int | None = None, mask_name: str | None = None, arity: int = 1)[source]

Bases: Module

A multi-head attention module based on “Attention Is All You Need”.

Parameters:

embed_dim (int) – The number of expected features.
num_heads (int) – The number of heads.
output_name (str) – Output (head) predicate name of the module.
query_name (str) – The name of the queries predicate.
key_name (str) – The name of the keys predicate.
value_name (str) – The name of the values predicate.
vdim (Optional[int]) – Total number of features for values.
kdim (Optional[int]) – Total number of features for keys.
mask_name (str, optional) – The name of the input mask predicate. Default: None
arity (int) – Arity of the input and output predicates. Default: 1

class Normal[source]

Bases: Initializer

Initializes learnable parameters with random samples from a normal (Gaussian) distribution

class Optimizer(lr: float, lr_decay: LRDecay | None = None, weight_decay: float = 0.0)[source]

Bases: object

Base class for all optimizers. Optimizers are used to update the weights of the neural network during training.

get_lr_decay() → Any | None[source]

Initializes and returns the learning rate decay object.

Returns:: The Java learning rate decay object, or None if no decay is set.
Return type:: Any

initialize() → Any[source]

Initializes the Java representation of the optimizer.

Returns:: The Java optimizer object.
Return type:: Any

property lr: float

Returns the current learning rate.

Returns:: Current learning rate.
Return type:: float

name() → str[source]

property weight_decay: float: The L2 penalty added to every gradient before the step.

class Pooling(output_name: str, input_name: str, aggregation: AggregationFunction, input_arity: int = 1)[source]

Bases: Module

Apply generic pooling over the input specified by the input_name and the input arity parameters. Can be expressed as:

\[h = agg_{i_{0}, .., i_{n} \in N}(x_{(i_{0}, .., i_{n})})\]

Where \(N\) is a set of tuples of length \(n\) (specified by the input arity parameter) that are valid arguments for the input predicate.

For example, a classic pooling over graph nodes represented by relations of arity 1 (node id) would be calculated as:

\[h = agg_{i \in N}(x_{(i)})\]

Here \(N\) refers to a set of all node ids. Lifting the restriction of the input arity via the input_arity parameter allows for pooling not only nodes but also edges (input_arity=2) and other objects (hyperedges etc.)

Examples

The whole computation of this module (parametrized as Pooling("h1", "h0", Aggregation.AVG)) is as follows:

(R.h1 <= R.h0(V.X0)) | [Aggregation.AVG, Transformation.IDENTITY]
R.h1 / 0 | [Transformation.IDENTITY]

Module parametrized as Pooling("h1", "h0", Aggregation.MAX, 2) translates into:

(R.h1 <= R.h0(V.X0, V.X1)) | [Aggregation.MAX, Transformation.IDENTITY]
R.h1 / 0 | [Transformation.IDENTITY]

Parameters:

output_name (str) – Output (head) predicate name of the module.
input_name (str) – Input name.
aggregation (AggregationFunction) – Aggregation function.
input_arity (int) – Arity of the input predicate input_name. Default: 1

class PositionalEncoding(embed_dim: int, max_len: int, output_name: str, input_name: str, arity: int = 1, learnable: bool = False)[source]

Bases: Module

Implements positional encoding for logic sequences. It generates a set of rules that add fixed or learnable positional embeddings to the input features.

class ProgressCallback(epochs: int)[source]

Bases: TrainerCallback

Show a tqdm progress bar during training.

Parameters:: epochs (int) – Total number of epochs (used for progress bar length).

on_epoch_end(trainer: Trainer, epoch: int, logs: dict[str, Any]) → None[source]

Called after every epoch.

Parameters:

trainer (Trainer) – The trainer instance (access trainer.model, etc.).
epoch (int) – 0-indexed epoch number that just finished.
logs (dict) – Dictionary with keys "train_loss", "val_loss" (if available), "lr", and per-metric keys like "train_accuracy", "val_mae", etc.

on_train_begin(trainer: Trainer) → None[source]: Called once before the first epoch.

on_train_end(trainer: Trainer) → None[source]: Called once after training finishes (or early-stops).

class RGCNConv(in_channels: int, out_channels: int, output_name: str, feature_name: str, edge_name: str | None, relations: ~typing.List[str], activation: ~neuralogic.core.constructs.function.function.TransformationFunction = <neuralogic.core.constructs.function.function.TransformationFunction object>, aggregation: ~neuralogic.core.constructs.function.function.AggregationFunction = <neuralogic.core.constructs.function.function.AggregationFunction object>)[source]

Bases: Module

Relational Graph Convolutional layer from Modeling Relational Data with Graph Convolutional Networks. Which can be expressed as:

\[\mathbf{x}^{\prime}_i = act(\mathbf{W_0} \cdot \mathbf{x}_i + \sum_{r \in \mathcal{R}} {agg}_{j \in \mathcal{N}_r(i)}(\mathbf{W_r} \cdot \mathbf{x}_j))\]

Where act is an activation function, agg aggregation function (by default average), \(W_0\) is a learnable root parameter and \(W_r\) is a learnable parameter for each relation.

The first part of the equation that is “\(\mathbf{W_0} \cdot \mathbf{x}_i\)” can be expressed in the logic form as:

R.<output_name>(V.I) <= R.<feature_name>(V.I)[<W0>]

Another part of the equation that is “\({agg}_{j \in \mathcal{N}_r(i)}(\mathbf{W_r} \cdot \mathbf{x}_j)\)” can be expressed as:

R.<output_name>(V.I) <= (R.<feature_name>(V.J)[<Wr>], R.<edge_name>(V.J, relation, V.I))

where “relation” is a constant name, or as:

R.<output_name>(V.I) <= (R.<feature_name>(V.J)[<Wr>], R.<relation>(V.J, V.I))

The outer summation, together with summing it with the first part, is handled by aggregation of all rules with the same head (and substitution).

Examples

The whole computation of this module (parametrized as RGCNConv(1, 2, "h1", "h0", "_edge", ["sibling", "parent"])) is as follows:

metadata = Metadata(activation=Transformation.IDENTITY, aggregation=Aggregation.AVG)

(R.h1(V.I) <= R.h0(V.I)[2, 1]) | metadata
(R.h1(V.I) <= (R.h0(V.J)[2, 1], R._edge(V.J, sibling, V.I))) | metadata
(R.h1(V.I) <= (R.h0(V.J)[2, 1], R._edge(V.J, parent, V.I))) | metadata
R.h1 / 1 [Transformation.IDENTITY]

Module parametrized as RGCNConv(1, 2, "h1", "h0", None, ["sibling", "parent"]) translates into:

metadata = Metadata(activation=Transformation.IDENTITY, aggregation=Aggregation.AVG)

(R.h1(V.I) <= R.h0(V.I)[2, 1]) | metadata
(R.h1(V.I) <= (R.h0(V.J)[2, 1], R.sibling(V.J, V.I))) | metadata
(R.h1(V.I) <= (R.h0(V.J)[2, 1], R.parent(V.J, V.I))) | metadata
R.h1 / 1 [Transformation.IDENTITY]

Parameters:

in_channels (int) – Input feature size.
out_channels (int) – Output feature size.
output_name (str) – Output (head) predicate name of the module.
feature_name (str) – Feature predicate name to get features from.
edge_name (str | None) – Edge predicate name to use for neighborhood relations. When None, elements from relations are used instead.
relations (List[str]) – List of relations’ names
activation (TransformationFunction) – Activation function of the output. Default: Transformation.IDENTITY
aggregation (AggregationFunction) – Aggregation function of nodes’ neighbors. Default: Aggregation.SUM

class RNN(input_size: int, hidden_size: int, output_name: str, input_name: str, hidden_0_name: str, activation: TransformationFunction = <neuralogic.core.constructs.function.function.TransformationFunction object>, arity: int = 1)[source]

Bases: Module

One-layer Recurrent Neural Network (RNN) module which is computed as:

\[h_t = act(\mathbf{W}_{ih} \mathbf{x}_t + \mathbf{W}_{hh} \mathbf{h}_{t-1})\]

where \(t \in (1, sequence\_length + 1)\) is a time step. In the template, the \(t\) is referred to as V.T, and \(t - 1\) is referred to as V.Z. This module expresses the first equation as:

(R.<output_name>(<...terms>, V.T) <= (
    R.<input_name>(<...terms>, V.T)[<hidden_size>, <input_size>],
    R.<hidden_input_name>(<...terms>, V.Z)[<hidden_size>, <hidden_size>],
    R.special.next(V.Z, V.T),
)) | [<activation>]

R.<output_name> / <arity> + 1 | [Transformation.IDENTITY]

Additionally, we define a rule for the “stop condition”, that is:

(R.<output_name>(<...terms>, 0) <= R.<hidden_0_name>(<...terms>)) | [Transformation.IDENTITY]

Parameters:

input_size (int) – Input feature size.
hidden_size (int) – Output and hidden feature size.
output_name (str) – Output (head) predicate name of the module.
input_name (str) – Input feature predicate name to get features from.
hidden_0_name (str) – Predicate name to get initial hidden state from.
activation (TransformationFunction) – Activation function. Default: Transformation.TANH
arity (int) – Arity of the input and output predicate. Default: 1

class ResGatedGraphConv(in_channels: int, out_channels: int, output_name: str, feature_name: str, edge_name: str, gating_activation: TransformationFunction = <neuralogic.core.constructs.function.function.TransformationFunction object>, activation: TransformationFunction = <neuralogic.core.constructs.function.function.TransformationFunction object>, aggregation: AggregationFunction = <neuralogic.core.constructs.function.function.AggregationFunction object>)[source]

Bases: Module

Residual Gated Graph Convolutional layer from “Residual Gated Graph ConvNets”. Which can be expressed as:

\[\mathbf{x}^{\prime}_i = act(\mathbf{W}_1 \mathbf{x}_i + {agg}_{j \in \mathcal{N}(i)}(\eta_{i,j} \odot \mathbf{W}_2 \mathbf{x}_j))\]

\[\mathbf{\eta}_{i,j} = gating\_act(\mathbf{W}_3 \mathbf{x}_i + \mathbf{W}_4 \mathbf{x}_j)\]

Where act is an activation function, agg aggregation function, gating_act is a gating activation function and \(W_n\) are learnable parameters. This equation is translated into the logic form as:

(R.<output_name>__gate(V.I, V.J) <= (R.<feature_name>(V.I)[<W>], R.<feature_name>(V.J)[<W>])) | [Transformation.IDENTITY]
R.<output_name>__gate / 2 | [<activation>]

(R.<output_name>(V.I) <= R.<feature_name>(V.I)[<W>]) | [Transformation.IDENTITY]
(R.<output_name>(V.I) <= (
    R.<output_name>__gate(V.I, V.J), R.<feature_name>(V.J)[<W>], R.<edge_name>(V.J, V.I))
) | Metadata(activation="elementproduct-identity", aggregation=<aggregation>)

R.<output_name> / 1 | [<activation>]

Examples

The whole computation of this module (parametrized as ResGatedGraphConv(1, 2, "h1", "h0", "_edge")) is as follows:

metadata = Metadata(activation="elementproduct-identity", aggregation=Aggregation.SUM)

(R.h1__gate(V.I, V.J) <= (R.h0(V.I)[2, 1], R.h0(V.J)[2, 1])) | [Transformation.IDENTITY]
R.h1__gate / 2 | [Transformation.SIGMOID]

(R.h1(V.I) <= R.h0(V.I)[2, 1]) | [Transformation.IDENTITY]
(R.h1(V.I) <= (R.h1__gate(V.I, V.J), R.h0(V.J)[2, 1], R._edge(V.J, V.I))) | metadata
R.h1 / 1 | [Transformation.IDENTITY]

Parameters:

in_channels (int) – Input feature size.
out_channels (int) – Output feature size.
output_name (str) – Output (head) predicate name of the module.
feature_name (str) – Feature predicate name to get features from.
edge_name (str) – Edge predicate name to use for neighborhood relations.
gating_activation (TransformationFunction) – Gating activation function. Default: Transformation.SIGMOID
activation (TransformationFunction) – Activation function of the output. Default: Transformation.IDENTITY
aggregation (AggregationFunction) – Aggregation function of nodes’ neighbors. Default: Aggregation.SUM

class RvNN(input_size: int, output_name: str, input_name: str, parent_map_name: str, max_children: int = 2, activation: TransformationFunction = <neuralogic.core.constructs.function.function.TransformationFunction object>, aggregation: AggregationFunction = <neuralogic.core.constructs.function.function.AggregationFunction object>, arity: int = 1)[source]

Bases: Module

Recursive Neural Network (RvNN) module which is computed as:

\[\mathbf{h}_i = act(agg_{j \in \mathcal{Ch(i)}}(\mathbf{W_{id(j)}} \mathbf{h}_j))\]

Where \(act\) is an activation function, \(agg\) aggregation function and \(\mathbf{W}\)’s are learnable parameters. \(\mathcal{Ch(i)}\) represents the ordered list of children of node \(i\). The \(id(j)\) function maps node \(j\) to its index (position) in its parent’s children list.

Parameters:

input_size (int) – Input feature size.
output_name (str) – Output (head) predicate name of the module.
input_name (str) – Input feature predicate name to get leaf features from.
parent_map_name (str) – Name of the predicate to get mapping from parent to children
max_children (int) – Maximum number of children (specify which <max_children>-ary tree will be considered). Default: 2
activation (TransformationFunction) – Activation function of all layers. Default: Transformation.TANH
aggregation (AggregationFunction) – Aggregation function of a layer. Default: Aggregation.SUM
arity (int) – Arity of the input and output predicate (doesn’t include the node id term). Default: 1

class SAGEConv(in_channels: int, out_channels: int, output_name: str, feature_name: str, edge_name: str, activation: TransformationFunction = <neuralogic.core.constructs.function.function.TransformationFunction object>, aggregation: AggregationFunction = <neuralogic.core.constructs.function.function.AggregationFunction object>)[source]

Bases: Module

GraphSAGE layer from “Inductive Representation Learning on Large Graphs”. Which can be expressed as:

\[\mathbf{x}^{\prime}_i = act(\mathbf{W}_1 \mathbf{x}_i + \mathbf{W}_2 \cdot {agg}_{j \in \mathcal{N}(i)}(\mathbf{x}_j)))\]

Where act is an activation function, agg aggregation function and W’s are learnable parameters. This equation is translated into the logic form as:

(R.<output_name>(V.I)[<W1>] <= (R.<feature_name>(V.J), R.<edge_name>(V.J, V.I))) | [<aggregation>, Transformation.IDENTITY]
(R.<output_name>(V.I)[<W2>] <= R.<feature_name>(V.I)) | [Transformation.IDENTITY]
R.<output_name> / 1 | [<activation>]

Parameters:

in_channels (int) – Input feature size.
out_channels (int) – Output feature size.
output_name (str) – Output (head) predicate name of the module.
feature_name (str) – Feature predicate name to get features from.
edge_name (str) – Edge predicate name to use for neighborhood relations.
activation (TransformationFunction) – Activation function of the output. Default: Transformation.IDENTITY
aggregation (AggregationFunction) – Aggregation function of nodes’ neighbors. Default: Aggregation.AVG

class SGConv(in_channels: int, out_channels: int, output_name: str, feature_name: str, edge_name: str, k: int = 1, activation: TransformationFunction = <neuralogic.core.constructs.function.function.TransformationFunction object>, aggregation: AggregationFunction = <neuralogic.core.constructs.function.function.AggregationFunction object>, add_self_loops: bool | None = None, normalize: bool = True)[source]

Bases: Module

Simple Graph Convolutional layer from “Simplifying Graph Convolutional Networks”. Which can be expressed as:

\[\mathbf{X}^{\prime} = act \left( {\left( \mathbf{\hat{D}}^{-1/2} \mathbf{\hat{A}} \mathbf{\hat{D}}^{-1/2} \right)}^k \mathbf{X} \mathbf{W} \right)\]

where \(\mathbf{\hat{A}} = \mathbf{A} + \mathbf{I}\), act is an activation function and W is a learnable parameter. Each of the k hops walks one edge and carries one normalization factor, so the equation is translated into the logic form as:

R.<output_name>__edge(V.I, V.I)[1.0].fixed()
R.<output_name>__edge(V.I, V.J) <= R.<edge_name>(V.I, V.J)

(R.<output_name>__edge_count(V.I, V.J) <= R.<output_name>__edge(V.X, V.J)) | [Aggregation.SUM]
(R.<output_name>__edge_count(V.I, V.J) <= R.<output_name>__edge(V.X, V.I)) | [Aggregation.SUM]
R.<output_name>__edge_count / 2 | [Combination.PRODUCT, Transformation.INVERSE]

(R.<output_name>(V.I<0>)[<W>] <= (
    R.<feature_name>(V.I<k>),
    R.<output_name>__edge(V.I<1>, V.I<0>), Transformation.SQRT(R.<output_name>__edge_count(V.I<1>, V.I<0>)),
    ...,
    R.<output_name>__edge(V.I<k>, V.I<k-1>), Transformation.SQRT(R.<output_name>__edge_count(V.I<k>, V.I<k-1>)),
)) | [<aggregation>, Combination.PRODUCT]

R.<output_name> / 1 | [<activation>]

The body combines by product, which is what makes the normalization factors scale the features rather than shift them - and it is also why the edge is an ordinary valued atom rather than a hidden one: an edge’s value is then PyG’s edge_weight, and at the natural 1.0 it is the identity of a product, so the plain spelling of a graph costs nothing.

Note __edge_count is a weighted **in-**degree despite its name: the two rules sum over V.X in the source position, and they sum the edge values rather than counting the edges. Both are what gcn_norm does, and both are invisible on an undirected graph given both directions at 1.0 - where in-degree is out-degree and summing ones is counting them - which is why the name survived being either.

Examples

Module parametrized as SGConv(2, 3, "h1", "h0", "_edge", 1) translates into:

R.h1__edge(V.I, V.I)[1.0].fixed()
R.h1__edge(V.I, V.J) <= R._edge(V.I, V.J)
(R.h1__edge_count(V.I, V.J) <= R.h1__edge(V.X, V.J)) | [Aggregation.SUM]
(R.h1__edge_count(V.I, V.J) <= R.h1__edge(V.X, V.I)) | [Aggregation.SUM]
R.h1__edge_count / 2 | [Combination.PRODUCT, Transformation.INVERSE]
(R.h1(V.I0)[3, 2] <= (R.h0(V.I1), R.h1__edge(V.I1, V.I0), Transformation.SQRT(R.h1__edge_count(V.I1, V.I0)))) | [Aggregation.SUM, Combination.PRODUCT]
R.h1 / 1 | [Transformation.IDENTITY]

Setting normalize=False and add_self_loops=False drops both groups of extra rules and leaves the plain walk-and-sum this module used to be - which is not what PyG computes.

Parameters:

in_channels (int) – Input feature size.
out_channels (int) – Output feature size.
output_name (str) – Output (head) predicate name of the module.
feature_name (str) – Feature predicate name to get features from.
edge_name (str) – Edge predicate name to use for neighborhood relations.
k (int) – Number of hops. Default: 1
activation (TransformationFunction) – Activation function of the output. Default: Transformation.IDENTITY
aggregation (AggregationFunction) – Aggregation function of nodes’ neighbors. Default: Aggregation.SUM
add_self_loops (bool | None) – Add self loops if either set to True or None (if normalize is True). Default: None
normalize (bool) – Add symmetric normalization. Default: True

class SGD(lr: float = 0.1, lr_decay: LRDecay | None = None, weight_decay: float = 0.0)[source]

Bases: Optimizer

Stochastic Gradient Descent (SGD) optimizer.

initialize() → Any[source]

Initializes the Java representation of the SGD optimizer.

Returns:: The Java optimizer object.
Return type:: Any

class SoftEntropy(reduction: str = 'mean')[source]

Bases: ErrorFunction

Soft Entropy loss function. Similar to Cross Entropy but usually applied with a soft layer at the end.

Parameters:: reduction (str) – "mean" or "sum", as in torch, and it changes the gradient. Default: "mean".

class SumPooling(output_name: str, input_name: str, input_arity: int = 1)[source]

Bases: Pooling

Apply sum pooling over the input specified by the input_name and the input arity parameters. Can be expressed as:

\[h = \sum_{i_{0}, .., i_{n} \in N} x_{(i_{0}, .., i_{n})}\]

Where \(N\) is a set of tuples of length \(n\) (specified by the input arity parameter) that are valid arguments for the input predicate.

This module extends the generic pooling Pooling.

Examples

The whole computation of this module (parametrized as SumPooling("h1", "h0")) is as follows:

(R.h1 <= R.h0(V.X0)) | [Aggregation.SUM, Transformation.IDENTITY]
R.h1 / 0 | [Transformation.IDENTITY]

Parameters:

output_name (str) – Output (head) predicate name of the module.
input_name (str) – Input name.
input_arity (int) – Arity of the input predicate input_name. Default: 1

class TAGConv(in_channels: int, out_channels: int, output_name: str, feature_name: str, edge_name: str, k: int = 2, activation: TransformationFunction = <neuralogic.core.constructs.function.function.TransformationFunction object>, aggregation: AggregationFunction = <neuralogic.core.constructs.function.function.AggregationFunction object>, normalize: bool = True)[source]

Bases: Module

Topology Adaptive Graph Convolutional layer from “Topology Adaptive Graph Convolutional Networks”. Which can be expressed as:

\[\mathbf{X}^{\prime} = act \left( \sum_{k=0}^K {\left( \mathbf{D}^{-1/2} \mathbf{A} \mathbf{D}^{-1/2} \right)}^k \mathbf{X} \mathbf{W}_k \right)\]

Where act is an activation function and Wk are learnable parameters, one per hop. Note this normalization is not the one GCNConv and SGConv use: there are no self-loops, so the degree is the plain degree rather than one more than it, matching PyG’s add_self_loops=False. The equation is translated into the logic form as:

(R.<output_name>__edge_count(V.I, V.J) <= R.<edge_name>(V.X, V.J)) | [Aggregation.SUM]
(R.<output_name>__edge_count(V.I, V.J) <= R.<edge_name>(V.X, V.I)) | [Aggregation.SUM]
R.<output_name>__edge_count / 2 | [Combination.PRODUCT, Transformation.INVERSE]

(R.<output_name>(V.I0)[<W0>] <= R.<feature_name>(V.I0)) | [<aggregation>, Combination.PRODUCT]
(R.<output_name>(V.I0)[<W1>] <= (R.<feature_name>(V.I1), R.<edge_name>(V.I1, V.I0), Transformation.SQRT(R.<output_name>__edge_count(V.I1, V.I0)))) | [<aggregation>, Combination.PRODUCT]
...
R.<output_name> / 1 | [<activation>]

Examples

Module parametrized as TAGConv(1, 2, "h1", "h0", "_edge", 1) translates into:

(R.h1__edge_count(V.I, V.J) <= R._edge(V.X, V.J)) | [Aggregation.SUM]
(R.h1__edge_count(V.I, V.J) <= R._edge(V.X, V.I)) | [Aggregation.SUM]
R.h1__edge_count / 2 | [Combination.PRODUCT, Transformation.INVERSE]
(R.h1(V.I0)[2, 1] <= R.h0(V.I0)) | [Aggregation.SUM, Combination.PRODUCT]
(R.h1(V.I0)[2, 1] <= (R.h0(V.I1), R._edge(V.I1, V.I0), Transformation.SQRT(R.h1__edge_count(V.I1, V.I0)))) | [Aggregation.SUM, Combination.PRODUCT]
R.h1 / 1 | [Transformation.IDENTITY]

Setting normalize=False drops the normalization rules and leaves the plain walk-and-sum this module used to be - which is not what PyG computes. An edge’s value is PyG’s edge_weight: it scales the message it carries and enters the degree, which despite the __edge_count name is a weighted **in-**degree, as in SGConv.

One difference from PyG that normalizing without self-loops leaves: a node of degree zero has no normalization atom at all, so its hop rules do not ground, where PyG makes that factor zero instead - agreeing on the value and not on whether the head exists.

Parameters:

in_channels (int) – Input feature size.
out_channels (int) – Output feature size.
output_name (str) – Output (head) predicate name of the module.
feature_name (str) – Feature predicate name to get features from.
edge_name (str) – Edge predicate name to use for neighborhood relations.
k (int) – Number of hops. Default: 2
activation (TransformationFunction) – Activation function of the output. Default: Transformation.IDENTITY
aggregation (AggregationFunction) – Aggregation function of nodes’ neighbors. Default: Aggregation.SUM
normalize (bool) – Add symmetric normalization. No self-loops are added either way, matching PyG. Default: True

class Trainer(module: NeuralModule)[source]

Bases: object

Run the training loop.

Parameters:

train_dataset – Training data. Raw Dataset objects are built automatically; pass a BuiltDataset to skip repeated grounding.
val_dataset – Optional validation data. When provided, validation loss (and any requested metrics) are computed after every epoch. Early stopping and checkpointing depend on validation loss.
epochs (int) – Number of epochs to train. Default 1.
batch_size (int) – Batch size when building raw datasets. Default 1.
early_stopping_patience (int or None) – Stop after this many epochs without validation-loss improvement. Requires val_dataset. Default None (no early stopping).
min_delta (float) – Minimum absolute change in validation loss to count as improvement. Default 0.0.
checkpoint_dir (str, Path, or None) – Directory to save the best model (by validation loss). A file named best.pkl is written on every improvement. Default None (no checkpointing).
metrics (Sequence[str or Metric] or None) – Extra metrics to compute, e.g. [Metric.ACCURACY] or ["mae", "r2"]. Loss is always tracked. The deep classification metrics auc_roc, auc_pr, best_accuracy, best_threshold and the saturation ratio are computed by the Java training strategy. Default None.
silent (bool) – If True, suppress the tqdm progress bar. Default False.
callbacks (Sequence[TrainerCallback] or None) – Additional callbacks to invoke. Built-in callbacks (early stopping, checkpoint, progress) are appended automatically based on the other arguments.

Returns:

Losses and metrics for every epoch.

Return type:

TrainerHistory

test(dataset: Dataset | GroundedDataset | BuiltDataset | StaticGraphDataset, *, batch_size: int = 1) → list[source]

Evaluate the model on a dataset (no weight updates).

Parameters:

dataset – Test data. Raw Dataset objects are built automatically.
batch_size (int) – Batch size when building raw datasets. Default 1.

Returns:

Model outputs for every sample.

Return type:

list

class TrainerCallback[source]

Bases: object

Base class for training callbacks.

Override any of the hooks. The trainer calls them in the order they were passed to Trainer.fit().

on_epoch_end(trainer: Trainer, epoch: int, logs: dict[str, Any]) → None[source]

Called after every epoch.

Parameters:

trainer (Trainer) – The trainer instance (access trainer.model, etc.).
epoch (int) – 0-indexed epoch number that just finished.
logs (dict) – Dictionary with keys "train_loss", "val_loss" (if available), "lr", and per-metric keys like "train_accuracy", "val_mae", etc.

on_train_begin(trainer: Trainer) → None[source]: Called once before the first epoch.

on_train_end(trainer: Trainer) → None[source]: Called once after training finishes (or early-stops).

class TrainerHistory(train_losses: list[float] = <factory>, val_losses: list[float] = <factory>, train_metrics: dict[str, list[float]]=<factory>, val_metrics: dict[str, list[float]]=<factory>, learning_rates: list[float] = <factory>, best_epoch: int = 0, best_val_loss: float = inf, stopped_early: bool = False)[source]

Bases: object

Training history collected during a Trainer.fit() run.

train_losses

Mean training loss per epoch.

Type:: list[float]

val_losses

Mean validation loss per epoch (empty if no validation set).

Type:: list[float]

train_metrics

Per-epoch extra metrics on the training set (each key maps to a list of epoch-level means).

Type:: dict[str, list[float]]

val_metrics

Per-epoch extra metrics on the validation set.

Type:: dict[str, list[float]]

learning_rates

Learning rate at each epoch.

Type:: list[float]

best_epoch

Epoch (0-indexed) that achieved the lowest validation loss.

Type:: int

best_val_loss

Lowest validation loss observed.

Type:: float

stopped_early

True if early stopping fired.

Type:: bool

best_epoch: int = 0

best_val_loss: float = inf

learning_rates: list[float]

stopped_early: bool = False

train_losses: list[float]

train_metrics: dict[str, list[float]]

val_losses: list[float]

val_metrics: dict[str, list[float]]

class Transformer(input_dim: int, num_heads: int, dim_feedforward: int, output_name: str, src_name: str, tgt_name: str, src_mask_name: str | None = None, tgt_mask_name: str | None = None, memory_mask_name: str | None = None, arity: int = 1)[source]

Bases: Module

A transformer module based on “Attention Is All You Need”.

Parameters:

input_dim (int) – The number of expected features.
num_heads (int) – The number of heads in the multi-head attention module.
dim_feedforward (int) – The dimension of the feedforward network.
output_name (str) – Output (head) predicate name of the module.
src_name (str) – The name of the predicate of the input to the encoder.
tgt_name (str) – The name of the predicate of the input to the decoder.
src_mask_name (str, optional) – The name of the predicate of the encoder input mask. Default: None
tgt_mask_name (str, optional) – The name of the predicate of the decoder input mask. Default: None
memory_mask_name (str, optional) – The name of the predicate of the encoder output mask. Default: None
arity (int) – Arity of the input and output predicate. Default: 1

class TransformerDecoder(input_dim: int, num_heads: int, dim_feedforward: int, output_name: str, input_name: str, encoder_name: str, mask_name: str | None = None, memory_mask_name: str | None = None, arity: int = 1)[source]

Bases: Module

A transformer decoder module based on “Attention Is All You Need”.

Parameters:

input_dim (int) – The number of expected features.
num_heads (int) – The number of heads in the multi-head attention module.
dim_feedforward (int) – The dimension of the feedforward network.
output_name (str) – Output (head) predicate name of the module.
input_name (str) – The name of the predicate of the input sequence.
input_name – The name of the input encoder.
mask_name (str, optional) – The name of the predicate of the decoder input sequence mask. Default: None
memory_mask_name (str, optional) – The name of the predicate of the encoder output mask. Default: None
arity (int) – Arity of the input and output predicate. Default: 1

class TransformerEncoder(input_dim: int, num_heads: int, dim_feedforward: int, output_name: str, input_name: str, mask_name: str | None = None, arity: int = 1)[source]

Bases: EncoderBlock

A transformer encoder module based on “Attention Is All You Need”.

Parameters:

input_dim (int) – The number of expected features.
num_heads (int) – The number of heads in the multi-head attention module.
dim_feedforward (int) – The dimension of the feedforward network.
output_name (str) – Output (head) predicate name of the module.
input_name (str) – The name of the predicate of the input sequence.
mask_name (str, optional) – The name of the predicate of the input sequence mask. Default: None
arity (int) – Arity of the input and output predicate. Default: 1

class Uniform(scale: float = 2)[source]

Bases: Initializer

Initializes learnable parameters with random uniformly distributed samples from the interval [-scale / 2, scale / 2].

Parameters:: scale (float) – Scale of the distribution interval [-scale / 2, scale / 2]. Default: 2

get_settings() → Dict[str, Any][source]

compute_metrics(targets: list, outputs: list, names: Sequence[str | Metric]) → dict[str, float][source]

Compute named metrics over a batch of (target, output) pairs.

Each metric receives the full batch and returns a single float.

Parameters:

targets (list) – Per-sample target values (floats, lists, or 2D lists).
outputs (list) – Per-sample output values (same shapes as targets).
names (Sequence[str or Metric]) – Metric names to compute, e.g. ["accuracy"] or [Metric.MAE, Metric.R2].

Returns:

Mapping from metric name to its value across the batch.

Return type:

dict[str, float]