regularizer causes "ValueError: Shapes must be equal rank"

Question

When trying to run

import numpy as np
import keras

X = np.ones((100,20))
Y1 = np.ones((100,5))
Y2 = np.ones((100,4))

Input_1= keras.layers.Input(shape=X.shape[1])

x = keras.layers.Dense(100)(Input_1)
x = keras.layers.Dense(100)(x)

out1 = keras.layers.Dense(5, kernel_regularizer='l1')(x)
out2 = keras.layers.Dense(4)(x)

model = keras.models.Model(inputs=Input_1, outputs=[out1,out2])
model.compile(loss = 'mse', loss_weights=np.arange(2))

model.fit(X, [Y1, Y2], epochs=2)

I get

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:805 train_function * return step_function(self, iterator) /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:795 step_function ** outputs = model.distribute_strategy.run(run_step, args=(data,)) /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:1259 run return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs) /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:2730 call_for_each_replica return self._call_for_each_replica(fn, args, kwargs) /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:3417 _call_for_each_replica return fn(*args, **kwargs) /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:788 run_step ** outputs = model.train_step(data) /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:756 train_step y, y_pred, sample_weight, regularization_losses=self.losses) /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/compile_utils.py:236 call total_loss_metric_value = math_ops.add_n(loss_metric_values) /usr/local/lib/python3.6/dist-packages/tensorflow/python/util/dispatch.py:201 wrapper return target(*args, **kwargs) /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/math_ops.py:3572 add_n return gen_math_ops.add_n(inputs, name=name) /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/gen_math_ops.py:419 add_n "AddN", inputs=inputs, name=name) /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/op_def_library.py:750 _apply_op_helper attrs=attr_protos, op_def=op_def) /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/func_graph.py:592 _create_op_internal compute_device) /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py:3536 _create_op_internal op_def=op_def) /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py:2016 init control_input_ops, op_def) /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py:1856 _create_c_op raise ValueError(str(e))

ValueError: Shapes must be equal rank, but are 1 and 0
  From merging shape 1 with other shapes. for '{{node AddN}} = AddN[N=3, T=DT_FLOAT](mul_2, mul_5, dense_199/kernel/Regularizer/mul)' with input shapes: [2], [2], [].

The error disappears if I omit the regularizer.

Answer 1

I found that loss_weights has to be a list, not an array.

import numpy as np
import keras

X = np.ones((100,20))
Y1 = np.ones((100,5))
Y2 = np.ones((100,4))

Input_1= keras.layers.Input(shape=X.shape[1])

x = keras.layers.Dense(100)(Input_1)
x = keras.layers.Dense(100)(x)

out1 = keras.layers.Dense(5, kernel_regularizer='l1')(x)
out2 = keras.layers.Dense(4)(x)

model = keras.models.Model(inputs=Input_1, outputs=[out1,out2])
model.compile(loss = 'mse', loss_weights=list(np.arange(2)))

model.fit(X, [Y1, Y2], epochs=2)

Answer 2

I was facing the same issue. changing the reduction of loss function from none to auto worked like a charm.

tf.keras.losses.MeanSquaredError(reduction=tf.keras.losses.Reduction.AUTO)