Learning notes for Pytorch （1）

mac2022-06-30 26

torch.tensor

1. construction

construct from list/numpy data: Can be a list, tuple, NumPy ndarray, scalar, and other types torch.tensor([[1., -1.], [1., -1.]]) >>>tensor([[ 1.0000, -1.0000], [ 1.0000, -1.0000]]) torch.tensor(np.array([[1, 2, 3], [4, 5, 6]])) torch.as_tensor(data, dtype=None) # do not need copy tensor.requires_grad_() torch.zeros([2, 4], dtype=torch.int32) >>>tensor([[ 0, 0, 0, 0], [ 0, 0, 0, 0]], dtype=torch.int32) cuda0 = torch.device('cuda:0') torch.ones([2, 4], dtype=torch.float64, device=cuda0) torch.linspace(0, 1, steps=5) >>>tensor([ 0.0000, 0.2500, 0.5000, 0.7500, 1.0000]) # 线性间距向量与均匀分布相区别 random number

torch.normal(mean, std, size, out=None) Returns a tensor of random numbers drawn from separate normal distributions（离散正态分布） whose mean and standard deviation are given.

The shapes of mean and std don’t need to match, but the total number of elements in each tensor need to be the same

>>> torch.normal(mean=torch.arange(1., 11.), std=torch.arange(1, 0, -0.1)) tensor([ 1.0425, 3.5672, 2.7969, 4.2925, 4.7229, 6.2134, 8.0505, 8.1408, 9.0563, 10.0566]) >>> torch.normal(mean=0.5, std=torch.arange(1., 6.)) tensor([-1.2793, -1.0732, -2.0687, 5.1177, -1.2303]) >>> torch.normal(2, 3, size=(1, 4)) tensor([[-1.3987, -1.9544, 3.6048, 0.7909]]) # mean: the mean for all distributions # std (float) – the standard deviation for all distributions torch.rand(size, out=None, dtype=None) Returns a tensor filled with random numbers from a uniform distribution（均匀分布/标准分布） on the interval [0, 1) >>> torch.rand(4) tensor([ 0.5204, 0.2503, 0.3525, 0.5673]) >>> torch.rand(2, 3) tensor([[ 0.8237, 0.5781, 0.6879], [ 0.3816, 0.7249, 0.0998]])

torch.rand_like(input, dtype=None) Returns a tensor with the same size as input that is filled with random numbers from a uniform distribution on the interval [0, 1) .

torch.rand_like(input) is equivalent to torch.rand(input.size(), dtype=input.dtype, layout=input.layout, device=input.device)

torch.randint(low=0, high, size, out=None, dtype=None） Returns a tensor filled with random integers generated uniformly between [low, high)

>>> torch.randint(3, 5, (3,)) tensor([4, 3, 4]) >>> torch.randint(10, (2, 2)) tensor([[0, 2], [5, 5]]) torch.randn(*size, out=None, dtype=None) Returns a tensor filled with random numbers from a normal distribution（标准正态分布） with mean 0 and variance 1 (also called the standard normal distribution).

$out_i \backsim \Nu (0, 1)$

>>> torch.randn(4) tensor([-2.1436, 0.9966, 2.3426, -0.6366]) >>> torch.randn(2, 3) tensor([[ 1.5954, 2.8929, -1.0923], [ 1.1719, -0.4709, -0.1996]])

2. access

x = torch.tensor([[1, 2, 3], [4, 5, 6]]) print(x[1][2]) >>>tensor(6) # not scalar, also tensor print(x[1][2].item()) >>>6 # scalar x[0][1] = 8 print(x) >>>tensor([[ 1, 8, 3], [ 4, 5, 6]])

3. common function

inplace：

a = b.abs() # a is different from bb.abs_() # inplace version, change b

Next all function are actually torch.func(), the output is all tensor, can written as a.func() or torch.func(a)

函数

Pointwise Ops

元素级运算

abs() 绝对值 acos() asin() atan()

torch.ceil 向上取整

torch.floor 向下取整

torch.round() 取整(四舍五入)

torch.clamp torch.clamp(input, min, max)

>>> a = torch.randn(4) >>> a tensor([-1.7120, 0.1734, -0.0478, -0.0922]) >>> torch.clamp(a, min=-0.5, max=0.5) tensor([-0.5000, 0.1734, -0.0478, -0.0922]) >>> torch.clamp(a, min=0.5) tensor([ 0.5000, 0.5000, 2.1593, 0.5000])

torch.digamma(input) 对数导数 torch.exp() 指数函数

torch.expm1() torch.log() 对数函数，以e为底

torch.log10() 对数函数，以10为底

torch.pow(self, exponent) 幂运算

torch.sqrt(input) 开方函数

torch.frac(input) 计算小数部分

>>> torch.frac(torch.tensor([1, 2.5, -3.2])) tensor([ 0.0000, 0.5000, -0.2000])

torch.reciprocal(input) 倒数 torch.sign() 取符号函数

>>> a = torch.tensor([0.7, -1.2, 0., 2.3]) >>> a tensor([ 0.7000, -1.2000, 0.0000, 2.3000]) >>> torch.sign(a) tensor([ 1., -1., 0., 1.])

torch.sigmoid

四则运算

torch.add(tensor, scalar) 广播机制，加

torch.div 除 torch.div(input, other, out=None)

>>> a = torch.randn(5) >>> a tensor([ 0.3810, 1.2774, -0.2972, -0.3719, 0.4637]) >>> torch.div(a, 0.5) tensor([ 0.7620, 2.5548, -0.5944, -0.7439, 0.9275]) >>> a = torch.randn(4, 4) >>> a tensor([[-0.3711, -1.9353, -0.4605, -0.2917], [ 0.1815, -1.0111, 0.9805, -1.5923], [ 0.1062, 1.4581, 0.7759, -1.2344], [-0.1830, -0.0313, 1.1908, -1.4757]]) >>> b = torch.randn(4) >>> b tensor([ 0.8032, 0.2930, -0.8113, -0.2308]) >>> torch.div(a, b) tensor([[-0.4620, -6.6051, 0.5676, 1.2637], [ 0.2260, -3.4507, -1.2086, 6.8988], [ 0.1322, 4.9764, -0.9564, 5.3480], [-0.2278, -0.1068, -1.4678, 6.3936]])

torch.fmod(input, other) 除法余数

>>> torch.fmod(torch.tensor([-3., -2, -1, 1, 2, 3]), 2) tensor([-1., -0., -1., 1., 0., 1.]) >>> torch.fmod(torch.tensor([1., 2, 3, 4, 5]), 1.5) tensor([ 1.0000, 0.5000, 0.0000, 1.0000, 0.5000])

torch.mul() torch.mul(input, other) 同torch.div()

#用法1 other为常数 >>> a = torch.randn(3) >>> a tensor([ 0.2015, -0.4255, 2.6087]) >>> torch.mul(a, 100) tensor([ 20.1494, -42.5491, 260.8663]) #用法2 other为矩阵 >>> a = torch.randn(4, 1) >>> a tensor([[ 1.1207], [-0.3137], [ 0.0700], [ 0.8378]]) >>> b = torch.randn(1, 4) >>> b tensor([[ 0.5146, 0.1216, -0.5244, 2.2382]]) >>> torch.mul(a, b) tensor([[ 0.5767, 0.1363, -0.5877, 2.5083], [-0.1614, -0.0382, 0.1645, -0.7021], [ 0.0360, 0.0085, -0.0367, 0.1567], [ 0.4312, 0.1019, -0.4394, 1.8753]])

矩阵乘积

matmul(input, other, out=None) Matrix product of two tensors 向量点积 / 矩阵乘法 >>> # vector x vector 向量点积 >>> tensor1 = torch.randn(3) # tensor([-0.5563, 1.0305, 1.9447]) >>> tensor2 = torch.randn(3) # tensor([-1.3812e+00, 1.0959e-03, 8.9040e-01]) >>> torch.matmul(tensor1, tensor2).size() torch.Size([]) # tensor(2.5011) >>> # matrix x vector >>> tensor1 = torch.randn(3, 4) >>> tensor2 = torch.randn(4) >>> torch.matmul(tensor1, tensor2).size() torch.Size([3]) >>> # matrix x matrix >>> a = torch.randn(3, 2) >>> b = torch.randn(2, 3) >>> print(torch.matmul(a, b).size()) torch.Size([3, 3])

mul 与 matmul的区别

a = torch.tensor([[1., 2., 3.], [2., 3., 4.], [3., 4., 5.]]) b = torch.ones(3, 3) c = torch.mul(a, b) print(c) # tensor([[1., 2., 3.], # [2., 3., 4.], # [3., 4., 5.]]) c=torch.matmul(a, b) print(c) # tensor([[ 6., 6., 6.], # [ 9., 9., 9.], # [12., 12., 12.]])

mm(input, mat2, out=None) Performs a matrix multiplication of the matrices input and mat2. This function does not broadcast. For broadcasting matrix products, see torch.matmul()

torch.cross(input, other, dim=-1) 向量叉积

a = torch.tensor([[1, 2], [2, 3], [3, 4]]) b = torch.tensor([[1, 0], [0, 1], [1, 1]]) print(torch.cross(a, b)) >> tensor([[ 2, -1], [ 2, -2], [-2, 2]]) print(torch.cross(a, b, dim=1)) >> dimension 1 does not have size 3 apply_(callable) 将函数应用到各元素 >>> a = torch.randn(4, 4) >>> a tensor([[ 1.3398, 0.2663, -0.2686, 0.2450], [-0.7401, -0.8805, -0.3402, -1.1936], [ 0.4907, -1.3948, -1.0691, -0.3132], [-1.6092, 0.5419, -0.2993, 0.3195]]) >>> torch.argmax(a, dim=1) tensor([ 0, 2, 0, 1]) # 每行元素的最大值索引，结果为一维向量

Reduction Ops

均为两种用法 torch.argmax(input)

>>> a = torch.randn(4, 4) >>> a tensor([[ 1.3398, 0.2663, -0.2686, 0.2450], [-0.7401, -0.8805, -0.3402, -1.1936], [ 0.4907, -1.3948, -1.0691, -0.3132], [-1.6092, 0.5419, -0.2993, 0.3195]]) >>> torch.argmax(a) # 整个矩阵中的最大值 tensor(0)

torch.argmax(input, dim, keepdim=False)

a = torch.randn(4, 4) >> tensor([[ 4.0322e-01, -1.4349e-01, 4.2135e-01, 4.8610e-01], [ 2.2702e+00, 1.5530e-03, -2.2095e+00, -7.1159e-01], [ 1.7592e+00, -4.4188e-01, -2.5789e-01, -1.0545e+00], [ 1.5922e+00, -1.1223e-01, 1.7855e+00, -5.7305e-01]]) b = torch.argmax(a, dim=1, keepdim=True) # dim:维度 keepdim:是否保持原维度 >> tensor([[3], [0], [0], [2]]) argmax(input, dim=None, keepdim=None) input中最大值的索引, 这一类的函数还有： argminmax min mean median(中位数) dim 默认：展平，全部元素最大值，dim=1 每行元素中最大值dim=0 每列… keepdim keepdim=True 保持原维度keepdim=False 结果会降维

这一类函数有:

torch.argmax() torch.argmin() torch.mean() # 均值 torch.median() # 中位数 torch.prod() # 乘积 torch.sum() # 和 torch.logsumexp()

其中,logsumexp计算如下：

不减维的函数

# 不减维的函数 torch.cumprod() # 累积 torch.cumsum() # 累和 a = torch.ones(4, 4) >> tensor([[1., 1., 1., 1.], [1., 1., 1., 1.], [1., 1., 1., 1.], [1., 1., 1., 1.]]) b = torch.cumsum(a, dim=1) >> tensor([[1., 2., 3., 4.], [1., 2., 3., 4.], [1., 2., 3., 4.], [1., 2., 3., 4.]])

距离-范数(p-norm) torch.dist

# torch.dist(input, other, p) a = torch.tensor([1., 1., 1.]) b = torch.tensor([2., 2., 3.]) print(torch.dist(a, b, 1)) >> tensor(3.) # 1范数 1+1+2 print(torch.dist(a, b, 2)) >> tensor(2.4495) # 2-范数 sqrt(1+1+2^2)

标准差(standard-deviation) torch.std() torch.std_mean()

""" torch.std(input, dim, keepdim=False, unbiased=True, out=None) -> tensor 标准差 unbiased (bool) – whether to use the unbiased estimation or not """ """ torch.std_mean() 标准差均值 torch.std_mean(input, unbiased=True) -> (Tensor, Tensor) """ """ torch.var(input, unbiased=True) → Tensor 方差 """ """ torch.var_mean() """

不重复值 torch.unique

""" torch.unique(input, sorted=True, return_inverse=False, return_counts=False, dim=None) """ >>> output = torch.unique(torch.tensor([1, 3, 2, 3], dtype=torch.long)) >>> output tensor([ 2, 3, 1]) >>> output, inverse_indices = torch.unique( torch.tensor([1, 3, 2, 3], dtype=torch.long), sorted=True, return_inverse=True) >>> output tensor([ 1, 2, 3]) >>> inverse_indices tensor([ 0, 2, 1, 2]) >>> output, inverse_indices = torch.unique( torch.tensor([[1, 3], [2, 3]], dtype=torch.long), sorted=True, return_inverse=True) >>> output tensor([ 1, 2, 3]) >>> inverse_indices tensor([[ 0, 2], [ 1, 2]])

Comparison Ops

Other Ops

bincount(input, weights=None, minlength) 计数 weights 权重与input的size相同，计数权重 minlength 指定最短长度 input = torch.randint(0, 8, (5,), dtype=torch.int64) >>> weights = torch.linspace(0, 1, steps=5) >>> input, weights (tensor([4, 3, 6, 3, 4]), tensor([ 0.0000, 0.2500, 0.5000, 0.7500, 1.0000]) >>> torch.bincount(input) tensor([0, 0, 0, 2, 2, 0, 1]) # 在3和4的位置均记了两次 torch.cat((tensor1, tensor2), dim) dim (int, optional) – tensor连接的维度 0-行连接 1-列连接 >>> x = torch.randn(2, 3) >>> x tensor([[ 0.6580, -1.0969, -0.4614], [-0.1034, -0.5790, 0.1497]]) >>> torch.cat((x, x, x), 0) tensor([[ 0.6580, -1.0969, -0.4614], [-0.1034, -0.5790, 0.1497], [ 0.6580, -1.0969, -0.4614], [-0.1034, -0.5790, 0.1497], [ 0.6580, -1.0969, -0.4614], [-0.1034, -0.5790, 0.1497]]) >>> torch.cat((x, x, x), 1) tensor([[ 0.6580, -1.0969, -0.4614, 0.6580, -1.0969, -0.4614, 0.6580, -1.0969, -0.4614], [-0.1034, -0.5790, 0.1497, -0.1034, -0.5790, 0.1497, -0.1034, -0.5790, 0.1497]])

-torch.tensor.scatter(dim, index, src) * dim 0-在行上分配，1-在列上分配，行不变 * index：索引 https://blog.csdn.net/qq_39004117/article/details/95665418

input = torch.randn(2, 4) output = torch.zeros(2, 5) index = torch.tensor([[3, 1, 2, 0], [1, 2, 0, 3]]) output = output.scatter(1, index, input) print(input) print(output) >> tensor([[-0.2558, -1.8930, -0.7831, 0.6100], [ 0.3246, 2.1289, 0.5887, 1.5588]]) tensor([[ 0.6100, -1.8930, -0.7831, -0.2558, 0.0000], [ 0.5887, 0.3246, 2.1289, 1.5588, 0.0000]])

多用于one-hot

index = torch.tensor([[1], [2], [0], [3]]) onehot = torch.zeros(4, 4) onehot.scatter_(1, index.view(-1, 1), 1) print(onehot) >> tensor([[0., 1., 0., 0.], [0., 0., 1., 0.], [1., 0., 0., 0.], [0., 0., 0., 1.]])

torch.combinations(input, r=2, with_replacement=False)

>>> a = [1, 2, 3] >>> list(itertools.combinations(a, r=2)) [(1, 2), (1, 3), (2, 3)] >>> list(itertools.combinations(a, r=3)) [(1, 2, 3)] >>> list(itertools.combinations_with_replacement(a, r=2)) [(1, 1), (1, 2), (1, 3), (2, 2), (2, 3), (3, 3)] >>> tensor_a = torch.tensor(a) >>> torch.combinations(tensor_a) tensor([[1, 2], [1, 3], [2, 3]]) >>> torch.combinations(tensor_a, r=3) tensor([[1, 2, 3]]) >>> torch.combinations(tensor_a, with_replacement=True) tensor([[1, 1], [1, 2], [1, 3], [2, 2], [2, 3], [3, 3]])

torch.diag(input, diagonal=0)

torch.combinations(input, r=2, with_replacement=False)

>>> a = torch.randn(3) >>> a tensor([ 0.5950,-0.0872, 2.3298]) >>> torch.diag(a) tensor([[ 0.5950, 0.0000, 0.0000], [ 0.0000,-0.0872, 0.0000], [ 0.0000, 0.0000, 2.3298]]) >>> torch.diag(a, 1) tensor([[ 0.0000, 0.5950, 0.0000, 0.0000], [ 0.0000, 0.0000,-0.0872, 0.0000], [ 0.0000, 0.0000, 0.0000, 2.3298], [ 0.0000, 0.0000, 0.0000, 0.0000]]) >>> a = torch.randn(3, 3) >>> a tensor([[-0.4264, 0.0255,-0.1064], [ 0.8795,-0.2429, 0.1374], [ 0.1029,-0.6482,-1.6300]]) >>> torch.diag(a, 0) tensor([-0.4264,-0.2429,-1.6300]) >>> torch.diag(a, 1) tensor([ 0.0255, 0.1374])

torch.flatten(input, start_dim=0, end_dim=-1)

>>> t = torch.tensor([[[1, 2], [3, 4]], [[5, 6], [7, 8]]]) >>> torch.flatten(t) tensor([1, 2, 3, 4, 5, 6, 7, 8]) >>> torch.flatten(t, start_dim=1) tensor([[1, 2, 3, 4], [5, 6, 7, 8]])

最新回复(0)