Tensorflow 2 version of nn.CrossEntropyLoss and nn.NLLLoss - 컴퓨터

Cross Entropy

Torch 1.2
loss = nn.CrossEntropyLoss()
#y_pred: [batch, classes, length], logit, float
#y_true: [batch, length], index, int
loss(y_pred, y_true)

Tensorflow 2.1
loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
#y_pred: [batch, length, classes], logit, float
#y_true: [batch, length], index, int
loss(y_true, y_pred)

Negative Log-Likelihood

Torch 1.2
loss = nn.NLLLoss()
#y_pred: [batch, classes, length], logit, float
#y_true: [batch, length], index, int
loss(y_pred, y_true)

Tensorflow 2.1
#y_pred: [batch, length, classes], logit, float
#y_true: [batch, length], index, int
mask=tf.one_hot(y_true,depth=tf.shape(y_pred)[-1])
-tf.reduce_sum(y_pred * mask) / tf.cast(tf.reduce_prod(tf.shape(y_true)), y_pred.dtype)

Tensorflow 2 version of unfold in Torch - 컴퓨터

def unfold(tensor, kernel_size, dilation=1, padding=0, stride=1):
  """
      Tensorflow 2 version of unfold in Torch (1.2)
      Thanks to tf.image.extract_patches,
      we just need to reshape, pad, and transpose before and after the operation.

      tensor: [b, channel, width, height], float
      kernel_size: [], int
      dilation: [], int
      padding: [], int
      stride: [], int
      * The four scalars above are broadcast to width and height
  """
  if dilation != 1:
    print("WARNING!!: dilation != 1 might work not as intended.")

  b, c, _, _ = tf.shape(tensor)
  # tensor: from [b, channel, width, height]
  #           to [b, channel, width, channel]
  tensor = tf.transpose(tensor, [0, 2, 3, 1])
  # tensor: from [b, channel, width, channel]
  #           to [b, channel + padding * 2, width + padding * 2, channel]
  tensor = tf.keras.layers.ZeroPadding2D(padding=padding)(tensor)
  # this implementation is tf.image.extract_patches
  kernel_size = [1, kernel_size, kernel_size, 1]
  stride = [1, stride, stride, 1]
  dilation = [1, dilation, dilation, 1]
  tensor = tf.image.extract_patches(images=tensor, sizes=kernel_size,
                                    strides=stride, rates=dilation,
                                    padding='VALID')
  # it needs to be refactored
  w = tf.shape(tensor)[1]
  h = tf.shape(tensor)[2]
  tensor = tf.reshape(tensor, [b, w, h, -1, c])
  tensor = tf.transpose(tensor, [0, 1, 2, 4, 3])
  tensor = tf.reshape(tensor, [b, w * h, -1])
  tensor = tf.transpose(tensor, [0, 2, 1])

  return tensor

pytorch 1.2.0, pillow version - 컴퓨터

if you see "ImportError: cannot import name 'PILLOW_VERSION'" during running PyTorch 1.2.0 scripts then downgrade pillow to 6.2.1.

ref: https://github.com/python-pillow/Pillow/issues/4130