AttributeError: 'float' object has no attribute 'dtype'

Question

When I try to use a custom activation function in keras (2.2.5), I create a new activation function gelu. add it in activations.py :

from . import backend as K    
import numpy as np
def gelu(x):
    return 0.5 * x * (1 + K.tanh(K.sqrt(2 / np.pi) * (x + 0.044715 * K.pow(x, 3))))

use it in my main.py :

cnn1 = Conv1D(filters=256, kernel_size=2, strides=1, padding="same")(gru_output)
cnn1 = Activation('gelu')(cnn1)

But I get a error when working it:

File "C:/Users/user/Desktop/my/my_main.py", line 253, in <module>
    cnn1 = Activation('gelu')(cnn1)
  File "C:\Users\user\Desktop/my\venv\lib\site-packages\keras\engine\base_layer.py", line 451, in __call__
    output = self.call(inputs, **kwargs)
  File "C:\Users\user\Desktop\my\venv\lib\site-packages\keras\layers\core.py", line 300, in call
    return self.activation(inputs)
  File "C:\Users\user\Desktop\my\venv\lib\site-packages\keras\activations.py", line 16, in gelu
    return 0.5 * x * (1 + K.tanh(K.sqrt(2 / np.pi) * (x + 0.044715 * K.pow(x, 3))))
  File "C:\Users\user\Desktop\my\venv\lib\site-packages\keras\backend\tensorflow_backend.py", line 1675, in sqrt
    zero = _to_tensor(0., x.dtype.base_dtype)
AttributeError: 'float' object has no attribute 'dtype'

So what should i do?

Answer 1

Replace K.sqrt() with tf.math.sqrt()