Skip to content

Activations

concrete_fhe_toolkit.ml.activations

Activation helpers for encrypted ML circuits.

relu 

relu(value: Any) -> Any

Return max(0, value).

Useful for introducing non-linearity in neural networks while preserving positive values exactly.

Parameters:

Name Type Description Default
value Any

The encrypted value to process or validate.

 required

Returns:

Name Type Description
Any Any

The transformed value where negative inputs are set to 0.
Example 
from concrete_fhe_toolkit.ml.activations import relu

# Inside an FHE circuit (e.g. after a linear layer)
# result = relu(encrypted_tensor_element)
Source code in src/concrete_fhe_toolkit/ml/activations.py 
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
def relu(value: Any) -> Any:
    """Return max(0, value).

    Useful for introducing non-linearity in neural networks while preserving 
    positive values exactly.

    Args:
        value (Any): The encrypted value to process or validate.

    Returns:
        Any: The transformed value where negative inputs are set to 0.

    Example:
        ```python
        from concrete_fhe_toolkit.ml.activations import relu

        # Inside an FHE circuit (e.g. after a linear layer)
        # result = relu(encrypted_tensor_element)
        ```
    """
    return maximum(0, value)

leaky_relu 

leaky_relu(value: Any, alpha: float = 0.01) -> Any

Return value when positive, otherwise alpha * value truncated toward zero.

alpha must be the reciprocal of a positive integer (for example 0.01, 0.1, or 0.5) because the slope is applied with integer division.

Unlike standard ReLU, this avoids "dead neurons" by allowing a small, non-zero gradient when the unit is not active.

Parameters:

Name Type Description Default
value Any

The encrypted value to process or validate.

 required
alpha float

The slope for negative inputs. Defaults to 0.01.

 0.01

Returns:

Name Type Description
Any Any

The modified value after applying Leaky ReLU.
Example 
from concrete_fhe_toolkit.ml.activations import leaky_relu

# Inside an FHE circuit
# result = leaky_relu(encrypted_val, alpha=0.1)
Source code in src/concrete_fhe_toolkit/ml/activations.py 
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
def leaky_relu(value: Any, alpha: float = 0.01) -> Any:
    """Return value when positive, otherwise alpha * value truncated toward zero.

    ``alpha`` must be the reciprocal of a positive integer (for example 0.01,
    0.1, or 0.5) because the slope is applied with integer division.

    Unlike standard ReLU, this avoids "dead neurons" by allowing a small,
    non-zero gradient when the unit is not active.

    Args:
        value (Any): The encrypted value to process or validate.
        alpha (float): The slope for negative inputs. Defaults to 0.01.

    Returns:
        Any: The modified value after applying Leaky ReLU.

    Example:
        ```python
        from concrete_fhe_toolkit.ml.activations import leaky_relu

        # Inside an FHE circuit
        # result = leaky_relu(encrypted_val, alpha=0.1)
        ```
    """
    if alpha <= 0 or alpha > 1:
        raise ValueError("alpha must be in (0, 1]")
    divisor = round(1 / alpha)
    if abs(1 / alpha - divisor) > 1e-9:
        raise ValueError("alpha must be the reciprocal of a positive integer")
    # -((-value) // divisor) truncates toward zero, so small negative inputs
    # scale toward 0 instead of sticking at -1.
    scaled = -((-value) // divisor)
    return maximum(scaled, value)

unit_step 

unit_step(value: Any) -> Any

Return the Heaviside step of a number: 0 when value < 0, otherwise 1.

Often used in simple perceptrons or binary classification layers.

Parameters:

Name Type Description Default
value Any

The encrypted value to process or validate.

 required

Returns:

Name Type Description
Any Any

1 if value >= 0 else 0.
Example 
from concrete_fhe_toolkit.ml.activations import unit_step

# Inside an FHE circuit
# result = unit_step(encrypted_val)
Source code in src/concrete_fhe_toolkit/ml/activations.py 
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
def unit_step(value: Any) -> Any:
    """Return the Heaviside step of a number: 0 when value < 0, otherwise 1.

    Often used in simple perceptrons or binary classification layers.

    Args:
        value (Any): The encrypted value to process or validate.

    Returns:
        Any: 1 if value >= 0 else 0.

    Example:
        ```python
        from concrete_fhe_toolkit.ml.activations import unit_step

        # Inside an FHE circuit
        # result = unit_step(encrypted_val)
        ```
    """
    return greater_equal(value, 0)

threshold_activation 

threshold_activation(value: Any, threshold: Any) -> Any

Return 1 when value >= threshold, otherwise 0.

Useful for custom decision boundaries in threshold-based models.

Parameters:

Name Type Description Default
value Any

The encrypted value to process or validate.

 required
threshold Any

The threshold to compare against.

 required

Returns:

Name Type Description
Any Any

1 if value >= threshold else 0.
Example 
from concrete_fhe_toolkit.ml.activations import threshold_activation

# Inside an FHE circuit
# result = threshold_activation(encrypted_val, threshold=5)
Source code in src/concrete_fhe_toolkit/ml/activations.py 
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
def threshold_activation(value: Any, threshold: Any) -> Any:
    """Return 1 when value >= threshold, otherwise 0.

    Useful for custom decision boundaries in threshold-based models.

    Args:
        value (Any): The encrypted value to process or validate.
        threshold (Any): The threshold to compare against.

    Returns:
        Any: 1 if value >= threshold else 0.

    Example:
        ```python
        from concrete_fhe_toolkit.ml.activations import threshold_activation

        # Inside an FHE circuit
        # result = threshold_activation(encrypted_val, threshold=5)
        ```
    """
    return greater_equal(value, threshold)

make_softmax 

make_softmax(min_input: int = -127, max_input: int = 127, *, input_scale: int = 10, output_scale: int = 100, probability_scale: int = 100) -> Callable[[List[Any]], List[Any]]

Create a scaled softmax function for a list of encrypted scores.

This function uses an exponential approximation and floor division to calculate probabilities as integer percentages. The output is scaled by probability_scale.

Parameters:

Name Type Description Default
min_input int

The minimum expected value in the input domain.

 -127
max_input int

The maximum expected value in the input domain.

 127
input_scale int

The fixed-point scaling factor of the input.

 10
output_scale int

The desired fixed-point scaling factor for the output.

 100
probability_scale int

Scale for the final probability percentages. Defaults to 100.

 100

Returns:

Type Description
Callable[[List[Any]], List[Any]]

Callable[[List[Any]], List[Any]]: A function that applies scaled softmax to a list of scores.
Example 
from concrete_fhe_toolkit.ml.activations import make_softmax

# Create a softmax function for inputs scaled by 10
softmax = make_softmax(min_input=-50, max_input=50, input_scale=10)

# Inside an FHE circuit
# enc_probs = softmax(enc_scores)
Source code in src/concrete_fhe_toolkit/ml/activations.py 
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
def make_softmax(
    min_input: int = -127, 
    max_input: int = 127,
    *,
    input_scale: int = 10,
    output_scale: int = 100,
    probability_scale: int = 100,
) -> Callable[[List[Any]], List[Any]]:
    """Create a scaled softmax function for a list of encrypted scores.

    This function uses an exponential approximation and floor division to calculate
    probabilities as integer percentages. The output is scaled by `probability_scale`.

    Args:
        min_input (int): The minimum expected value in the input domain.
        max_input (int): The maximum expected value in the input domain.
        input_scale (int): The fixed-point scaling factor of the input.
        output_scale (int): The desired fixed-point scaling factor for the output.
        probability_scale (int): Scale for the final probability percentages. Defaults to 100.

    Returns:
        Callable[[List[Any]], List[Any]]: A function that applies scaled softmax to a list of scores.

    Example:
        ```python
        from concrete_fhe_toolkit.ml.activations import make_softmax

        # Create a softmax function for inputs scaled by 10
        softmax = make_softmax(min_input=-50, max_input=50, input_scale=10)

        # Inside an FHE circuit
        # enc_probs = softmax(enc_scores)
        ```
    """

    warnings.warn(
        "Softmax requires >16-bit TLUs for encrypted division and will likely fail to compile "
        "in the current version of Concrete. Consider using `argmax` or client-side Softmax instead.",
        UserWarning, stacklevel=2
    )

    exp_func = make_exp(min_input - max_input, 0, input_scale=input_scale, output_scale=output_scale)
    div_func = make_floor_divide(zero_result=0)

    def softmax(values: List[Any]) -> List[Any]:
        """Apply softmax to a list of values.

        Args:
            values (List[Any]): List of values to apply softmax on.

        Returns:
            List[Any]: Softmax probabilities.
        """
        total: Any = 0
        exp_values = []
        shifted_values = np.subtract(values, np.max(_ensure_tensor(values)))

        for value in shifted_values:
            # make_exp already handles dividing by input_scale and multiplying by output_scale internally
            exp_value = exp_func(value)
            total = total + exp_value
            exp_values.append(exp_value)

        probabilities = []
        for value in exp_values:
            # We multiply by probability_scale before dividing so the output is an integer percentage
            probabilities.append(div_func(value * probability_scale, total))  

        return _ensure_tensor(probabilities)

    return softmax

client_softmax 

client_softmax(scores: Union[ndarray, List[Any]]) -> np.ndarray

Calculate mathematically perfect Softmax on the client side after decryption.

This function avoids FHE 16-bit limitations by performing the Softmax division and scaling on the cleartext outputs decrypted by the client.

Parameters:

Name Type Description Default
scores Union[ndarray, List[Any]]

The decrypted scores from the FHE model.

 required

Returns:

Type Description
ndarray

np.ndarray: The array of softmax probabilities.
Example 
from concrete_fhe_toolkit.ml.activations import client_softmax

# After decrypting the raw scores (logits) from the FHE model
decrypted_scores = [12.4, -5.1, 14.8, 0.2]
probabilities = client_softmax(decrypted_scores)
Source code in src/concrete_fhe_toolkit/ml/activations.py 
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
@client_side_helper
def client_softmax(scores: Union[np.ndarray, List[Any]]) -> np.ndarray:
    """Calculate mathematically perfect Softmax on the client side after decryption.

    This function avoids FHE 16-bit limitations by performing the Softmax division 
    and scaling on the cleartext outputs decrypted by the client.

    Args:
        scores (Union[np.ndarray, List[Any]]): The decrypted scores from the FHE model.

    Returns:
        np.ndarray: The array of softmax probabilities.

    Example:
        ```python
        from concrete_fhe_toolkit.ml.activations import client_softmax

        # After decrypting the raw scores (logits) from the FHE model
        decrypted_scores = [12.4, -5.1, 14.8, 0.2]
        probabilities = client_softmax(decrypted_scores)
        ```
    """
    tensor = np.array(scores, dtype=np.float64)
    shifted_tensor = tensor - np.max(tensor) # Max-trick
    exp_values = np.exp(shifted_tensor)

    probabilities = (exp_values / np.sum(exp_values)) * 100
    return probabilities

compile_relu 

compile_relu(min_value: int = -15, max_value: int = 15, *, configuration: Optional[Configuration] = None) -> fhe.Circuit

Compile encrypted ReLU.

Compiles the ReLU function into a concrete-python FHE circuit.

Parameters:

Name Type Description Default
min_value int

The lower bound for the encrypted inputs, used to dimension the FHE circuit.

 -15
max_value int

The upper bound for the encrypted inputs, used to dimension the FHE circuit.

 15
configuration Optional[Configuration]

Optional FHE compiler configuration object.

 None

Returns:

Type Description
Circuit

fhe.Circuit: Compiled ReLU circuit.
Example 
from concrete_fhe_toolkit.ml.activations import compile_relu

circuit = compile_relu(min_value=-10, max_value=10)
# circuit.encrypt_run_decrypt(-5) -> 0
Source code in src/concrete_fhe_toolkit/ml/activations.py 
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
def compile_relu(
    min_value: int = -15,
    max_value: int = 15,
    *,
    configuration: Optional[fhe.Configuration] = None,
) -> fhe.Circuit:
    """Compile encrypted ReLU.

    Compiles the ReLU function into a concrete-python FHE circuit.

    Args:
        min_value (int): The lower bound for the encrypted inputs, used to dimension the FHE circuit.
        max_value (int): The upper bound for the encrypted inputs, used to dimension the FHE circuit.
        configuration (Optional[fhe.Configuration]): Optional FHE compiler configuration object.

    Returns:
        fhe.Circuit: Compiled ReLU circuit.

    Example:
        ```python
        from concrete_fhe_toolkit.ml.activations import compile_relu

        circuit = compile_relu(min_value=-10, max_value=10)
        # circuit.encrypt_run_decrypt(-5) -> 0
        ```
    """
    minimum, maximum = validate_bounds(min_value, max_value)
    return compile_function(
        relu,
        {"value": "encrypted"},
        [minimum, maximum],
        configuration,
    )

compile_leaky_relu 

compile_leaky_relu(min_value: int = -15, max_value: int = 15, *, alpha: float = 0.01, configuration: Optional[Configuration] = None) -> fhe.Circuit

Compile encrypted leaky ReLU.

Compiles the Leaky ReLU function into an FHE circuit, keeping a non-zero slope for negative inputs.

Parameters:

Name Type Description Default
min_value int

The lower bound for the encrypted inputs, used to dimension the FHE circuit.

 -15
max_value int

The upper bound for the encrypted inputs, used to dimension the FHE circuit.

 15
alpha float

Slope for negative values. Defaults to 0.01.

 0.01
configuration Optional[Configuration]

Optional FHE compiler configuration object.

 None

Returns:

Type Description
Circuit

fhe.Circuit: Compiled Leaky ReLU circuit.
Example 
from concrete_fhe_toolkit.ml.activations import compile_leaky_relu

circuit = compile_leaky_relu(min_value=-10, max_value=10, alpha=0.1)
# circuit.encrypt_run_decrypt(-10) -> -1
Source code in src/concrete_fhe_toolkit/ml/activations.py 
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
def compile_leaky_relu(
    min_value: int = -15,
    max_value: int = 15,
    *,
    alpha: float = 0.01,
    configuration: Optional[fhe.Configuration] = None,
) -> fhe.Circuit:
    """Compile encrypted leaky ReLU.

    Compiles the Leaky ReLU function into an FHE circuit, keeping a non-zero
    slope for negative inputs.

    Args:
        min_value (int): The lower bound for the encrypted inputs, used to dimension the FHE circuit.
        max_value (int): The upper bound for the encrypted inputs, used to dimension the FHE circuit.
        alpha (float): Slope for negative values. Defaults to 0.01.
        configuration (Optional[fhe.Configuration]): Optional FHE compiler configuration object.

    Returns:
        fhe.Circuit: Compiled Leaky ReLU circuit.

    Example:
        ```python
        from concrete_fhe_toolkit.ml.activations import compile_leaky_relu

        circuit = compile_leaky_relu(min_value=-10, max_value=10, alpha=0.1)
        # circuit.encrypt_run_decrypt(-10) -> -1
        ```
    """
    minimum, maximum = validate_bounds(min_value, max_value)

    def bound_leaky_relu(value: Any) -> Any:
        """Apply leaky ReLU with the bound alpha.

        Args:
            value (Any): The encrypted value to process or validate.

        Returns:
            Any: The modified value.
        """
        return leaky_relu(value, alpha)

    return compile_function(
        bound_leaky_relu,
        {"value": "encrypted"},
        [minimum, maximum],
        configuration,
    )

compile_unit_step 

compile_unit_step(min_value: int = -15, max_value: int = 15, *, configuration: Optional[Configuration] = None) -> fhe.Circuit

Compile encrypted unit step.

Compiles the Heaviside step function into an FHE circuit.

Parameters:

Name Type Description Default
min_value int

The lower bound for the encrypted inputs, used to dimension the FHE circuit.

 -15
max_value int

The upper bound for the encrypted inputs, used to dimension the FHE circuit.

 15
configuration Optional[Configuration]

Optional FHE compiler configuration object.

 None

Returns:

Type Description
Circuit

fhe.Circuit: Compiled Unit Step circuit.
Example 
from concrete_fhe_toolkit.ml.activations import compile_unit_step

circuit = compile_unit_step(min_value=-10, max_value=10)
# circuit.encrypt_run_decrypt(5) -> 1
# circuit.encrypt_run_decrypt(-3) -> 0
Source code in src/concrete_fhe_toolkit/ml/activations.py 
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
def compile_unit_step(
    min_value: int = -15,
    max_value: int = 15,
    *,
    configuration: Optional[fhe.Configuration] = None,
) -> fhe.Circuit:
    """Compile encrypted unit step.

    Compiles the Heaviside step function into an FHE circuit.

    Args:
        min_value (int): The lower bound for the encrypted inputs, used to dimension the FHE circuit.
        max_value (int): The upper bound for the encrypted inputs, used to dimension the FHE circuit.
        configuration (Optional[fhe.Configuration]): Optional FHE compiler configuration object.

    Returns:
        fhe.Circuit: Compiled Unit Step circuit.

    Example:
        ```python
        from concrete_fhe_toolkit.ml.activations import compile_unit_step

        circuit = compile_unit_step(min_value=-10, max_value=10)
        # circuit.encrypt_run_decrypt(5) -> 1
        # circuit.encrypt_run_decrypt(-3) -> 0
        ```
    """
    minimum, maximum = validate_bounds(min_value, max_value)
    return compile_function(
        unit_step,
        {"value": "encrypted"},
        [minimum, maximum],
        configuration,
    )

compile_threshold_activation 

compile_threshold_activation(min_value: int = -15, max_value: int = 15, *, configuration: Optional[Configuration] = None) -> fhe.Circuit

Compile encrypted threshold activation over two encrypted inputs.

Parameters:

Name Type Description Default
min_value int

The lower bound for the encrypted inputs, used to dimension the FHE circuit.

 -15
max_value int

The upper bound for the encrypted inputs, used to dimension the FHE circuit.

 15
configuration Optional[Configuration]

Optional FHE compiler configuration object.

 None

Returns:

Type Description
Circuit

fhe.Circuit: Compiled Threshold Activation circuit.
Example 
from concrete_fhe_toolkit.ml.activations import compile_threshold_activation

circuit = compile_threshold_activation(min_value=-10, max_value=10)
# Evaluates (value >= threshold) fully obliviously
Source code in src/concrete_fhe_toolkit/ml/activations.py 
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
def compile_threshold_activation(
    min_value: int = -15,
    max_value: int = 15,
    *,
    configuration: Optional[fhe.Configuration] = None,
) -> fhe.Circuit:
    """Compile encrypted threshold activation over two encrypted inputs.

    Args:
        min_value (int): The lower bound for the encrypted inputs, used to dimension the FHE circuit.
        max_value (int): The upper bound for the encrypted inputs, used to dimension the FHE circuit.
        configuration (Optional[fhe.Configuration]): Optional FHE compiler configuration object.

    Returns:
        fhe.Circuit: Compiled Threshold Activation circuit.

    Example:
        ```python
        from concrete_fhe_toolkit.ml.activations import compile_threshold_activation

        circuit = compile_threshold_activation(min_value=-10, max_value=10)
        # Evaluates (value >= threshold) fully obliviously
        ```
    """
    minimum, maximum = validate_bounds(min_value, max_value)
    return compile_function(
        threshold_activation,
        {"value": "encrypted", "threshold": "encrypted"},
        [
            (minimum, minimum),
            (minimum, maximum),
            (maximum, minimum),
            (maximum, maximum),
        ],
        configuration,
    )

compile_softmax 

compile_softmax(size: int, min_value: int = -127, max_value: int = 127, *, configuration: Optional[Configuration] = None) -> fhe.Circuit

Compile an FHE circuit for the softmax function over an array of fixed size.

Since FHE circuits require fixed dimensions, the size of the input array must be specified at compile time.

Parameters:

Name Type Description Default
size int

The fixed size of the input array, must be known at compile time.

 required
min_value int

The lower bound for the encrypted inputs, used to dimension the FHE circuit.

 -127
max_value int

The upper bound for the encrypted inputs, used to dimension the FHE circuit.

 127
configuration Optional[Configuration]

Optional FHE compiler configuration object.

 None

Returns:

Type Description
Circuit

fhe.Circuit: Compiled Softmax circuit.
Example 
from concrete_fhe_toolkit.ml.activations import compile_softmax

# Compile softmax for a 3-class model
circuit = compile_softmax(size=3, min_value=-50, max_value=50)

# Run it (inputs should be scaled!)
# probs = circuit.encrypt_run_decrypt([20, -10, 5])
Source code in src/concrete_fhe_toolkit/ml/activations.py 
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
def compile_softmax(
    size: int,
    min_value: int = -127,
    max_value: int = 127,
    *,
    configuration: Optional[fhe.Configuration] = None,
) -> fhe.Circuit:
    """Compile an FHE circuit for the softmax function over an array of fixed size.

    Since FHE circuits require fixed dimensions, the `size` of the input array 
    must be specified at compile time.

    Args:
        size (int): The fixed size of the input array, must be known at compile time.
        min_value (int): The lower bound for the encrypted inputs, used to dimension the FHE circuit.
        max_value (int): The upper bound for the encrypted inputs, used to dimension the FHE circuit.
        configuration (Optional[fhe.Configuration]): Optional FHE compiler configuration object.

    Returns:
        fhe.Circuit: Compiled Softmax circuit.

    Example:
        ```python
        from concrete_fhe_toolkit.ml.activations import compile_softmax

        # Compile softmax for a 3-class model
        circuit = compile_softmax(size=3, min_value=-50, max_value=50)

        # Run it (inputs should be scaled!)
        # probs = circuit.encrypt_run_decrypt([20, -10, 5])
        ```
    """

    warnings.warn(
        "compile_softmax requires >16-bit TLUs and will likely fail to compile "
        "in the current version of Concrete. Kept for future 32-bit TLU compatibility.",
        UserWarning, stacklevel=2
    )

    function = make_softmax(min_value,max_value, input_scale=1, output_scale=1, probability_scale=1)
    minimum, maximum = validate_bounds(min_value, max_value)
    return compile_function(
        function,
        {"values": "encrypted"},
        [
            [minimum] * size,
            [maximum] * size,
            [0] * size
        ],
        configuration,
    )