sleepy

thinking

calender

그럴수만 있다면 얼마나 좋을까. - 일상

'엄마는 할머니되지마~나계속 이렇게 있을게~'- 큰딸

오늘은 일찍 퇴근해야겠다.

written time : 2020-05-20 13:52:04.0

Tensorflow 2 version of nn.CrossEntropyLoss and nn.NLLLoss - 컴퓨터

Cross Entropy

Torch 1.2
loss = nn.CrossEntropyLoss()
#y_pred: [batch, classes, length], logit, float
#y_true: [batch, length], index, int
loss(y_pred, y_true)

Tensorflow 2.1
loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
#y_pred: [batch, length, classes], logit, float
#y_true: [batch, length], index, int
loss(y_true, y_pred)

Negative Log-Likelihood

Torch 1.2
loss = nn.NLLLoss()
#y_pred: [batch, classes, length], logit, float
#y_true: [batch, length], index, int
loss(y_pred, y_true)

Tensorflow 2.1
#y_pred: [batch, length, classes], logit, float
#y_true: [batch, length], index, int
mask=tf.one_hot(y_true,depth=tf.shape(y_pred)[-1])
-tf.reduce_sum(y_pred * mask) / tf.cast(tf.reduce_prod(tf.shape(y_true)), y_pred.dtype)

written time : 2020-05-03 15:49:25.0

Tensorflow 2 version of unfold in Torch - 컴퓨터

def unfold(tensor, kernel_size, dilation=1, padding=0, stride=1):
"""
Tensorflow 2 version of unfold in Torch (1.2)
Thanks to tf.image.extract_patches,
we just need to reshape, pad, and transpose before and after the operation.

tensor: [b, channel, width, height], float
kernel_size: [], int
dilation: [], int
padding: [], int
stride: [], int
* The four scalars above are broadcast to width and height
"""
if dilation != 1:
print("WARNING!!: dilation != 1 might work not as intended.")

b, c, _, _ = tf.shape(tensor)
# tensor: from [b, channel, width, height]
#           to [b, channel, width, channel]
tensor = tf.transpose(tensor, [0, 2, 3, 1])
# tensor: from [b, channel, width, channel]
#           to [b, channel + padding * 2, width + padding * 2, channel]
tensor = tf.keras.layers.ZeroPadding2D(padding=padding)(tensor)
# this implementation is tf.image.extract_patches
kernel_size = [1, kernel_size, kernel_size, 1]
stride = [1, stride, stride, 1]
dilation = [1, dilation, dilation, 1]
tensor = tf.image.extract_patches(images=tensor, sizes=kernel_size,
strides=stride, rates=dilation,
padding='VALID')
# it needs to be refactored
w = tf.shape(tensor)[1]
h = tf.shape(tensor)[2]
tensor = tf.reshape(tensor, [b, w, h, -1, c])
tensor = tf.transpose(tensor, [0, 1, 2, 4, 3])
tensor = tf.reshape(tensor, [b, w * h, -1])
tensor = tf.transpose(tensor, [0, 2, 1])

return tensor

written time : 2020-05-03 13:35:37.0
...  10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 |  ...