Keras Incoherent behaviour in Model.compile between loss & metrics for dict output Models

Hey!

I believe there is a bug in keras 3 related to metrics in Model.compile. At the very least there is a significant difference from keras 2. I have an example below about this below, which I tested with keras 2.15 & 3.11.3:

import numpy as np
import keras

print('keras version:', keras.__version__)

X = np.random.rand(100, 32)
Y1 = np.random.rand(100, 10)
Y2 = np.random.rand(100, 10)

def create_model():
    x = keras.Input(shape=(32,))
    y1 = keras.layers.Dense(10)(x)
    y2 = keras.layers.Dense(10)(x)
    return keras.Model(inputs = x, outputs = {'a': y1, 'b': y2})

model = create_model()
model.compile(optimizer = 'adam', loss = {'a': 'bce', 'b': 'mse'})
model.train_on_batch(X, {'a': Y1, 'b': Y2})
print('Successful training not using metrics\n')

try:
    model = create_model()
    model.compile(optimizer = 'adam', loss = {'a': 'bce', 'b': 'mse'}, metrics = {'a': ['mae', 'acc'], 'b': 'mse'})
    model.train_on_batch(X, {'a': Y1, 'b': Y2})
    print('Successful training using metrics\n')  # passes in keras 2.15
except ValueError as ex:
    print('Unsuccessful training with metrics')  # fails in keras 3.11.3
    print(ex)

model = create_model()
model.compile(optimizer = 'adam', loss = {'a': 'bce', 'b': 'mse'}, loss_weights = {'a': 1., 'b': 0.5})
model.train_on_batch(X, {'a': Y1, 'b': Y2})
print('Successful training using loss_weights\n')

x = keras.Input(shape=(32,))
y1 = keras.layers.Dense(10, name = 'a')(x)
y2 = keras.layers.Dense(10, name = 'b')(x)
model = keras.Model(inputs = x, outputs = {'a': y1, 'b': y2})
model.compile(optimizer = 'adam', loss = {'a': 'bce', 'b': 'mse'}, metrics = {'a': ['mae', 'acc'], 'b': 'mse'})
model.train_on_batch(X, {'a': Y1, 'b': Y2})
print('Successful even with metrics')

As we see in the example, in keras 3 the names in the dict output of the Model are only assumed for the loss (and loss_weights), but not for metrics. This does not seem like intended behaviour.

If we give the layer names then it works in keras 3, as discussed #21259 but that is not a fix that works in all codebases.

I also created this message about this.

I have not looked at the keras code yet, but plan to, and if I can, I'll create a PR.

Comment From: dhantule

Hi @miguelteixeiragomes, thanks for reporting this.

I've tested your code with Keras 3.11.3, and I can confirm the behavior you described in this gist. When using a dict for model outputs like {'a': y1, 'b': y2}, lossand loss_weights work correctly even if y1 and y2 lack names. However, metrics raises a ValueError unless output layers are explicitly named to match the dict keys. Naming the layers(Dense(..., name='a')) resolves the issue. We'll look into this and update you.