1 Introduction
1.1 Instance discrimination (样本判别)
制定了一种划分正样本和负样本的规则
1.2 InfoNCE Loss
1.3 Momentum
动量在数学上可以理解为是一种指数移动平均(Exponential Moving Average)
\(m\)为动量系数,目的是为了 \(Y_t\) 不仅仅依赖于当前时刻的输入 \(X_t\)
- \(Y_t\): 当前时刻的输出
- \(Y_{t-1}\): 上一时刻的输出
- \(X_t\): 当前时刻的输入
\(m\)越大,对当前时刻的输入 \(X_t\) 的依赖越小
1.4 Momentum Contrast (MoCo)
2 Related Work
the relationships between Supervised Learning and Unsupervised Learning/Self-supervised Learning