By simply replace loss = tf.keras.losses.Huber() or with your choice of delta.
Note here is Huber Loss behaves better than MSE and MAE
Is the Huber loss function supposed to perform better than Mean Squared Error as my notebooks are showing different results
As you can see from the screens, MSE actually performed better than Huber.
Great try @gopalakrishna-r! It really depends on the data you are trying to fit. In general, huber loss is less sensitive to outliers in data than MSE.