Keras: Using The Sum Of The First And Second Derivatives Of A Model As Final Output

Suppose that I created this model using Keras: model = Sequential() model.add(Dense(32, activation='tanh', input_dim=1)) model.add(Dense(10, activation='tanh')) model.add(Dense(1,

Solution 1:

You can calculate gradient in Tensorflow with tf.gradients and in Keras with K.gradients:

first = K.gradients(model.outputs, model.inputs)
second = K.gradients(first, model.inputs)

Here is the code that calculates the gradient in your model:

from tensorflow.python.keras import Model, Inputfrom tensorflow.python.keras import backend as K
from tensorflow.python.keras.layers import Dense, Lambda

def deriative(inps):
    i, o = inps[0], inps[1]
    grad1 = K.gradients(o, i)[0]
    grad2 = K.gradients(grad1, i)[0]
    return K.concatenate([grad1, grad2])


inps = Input(shape=(1,))
dense1 = Dense(32, activation='tanh')(inps)
dense2 = Dense(10, activation='tanh')(dense1)
dense3 = Dense(1, activation='linear')(dense2)

output = Lambda(deriative)([inps, dense3])

new_model = Model(inputs=inps, outputs=output)
new_model.compile('adam', 'mse')
print(new_model.summary())