在学习cs229-notes11 时遇到了这样一个等式

\[\nabla_W|W| = |W|(W^{-1})^T \]

于是找到了这个 UCI 的“矩阵菜谱”。其中列举了与矩阵有关的公式，在此上传供以后查阅。

Credit to: Max Welling's CS 273B: Kernel-Based Learning, 2005

(可见这是一个比较古早的研究kernel的课程)

Gaussian discriminant analysis - lemon的文章 - 知乎 https://zhuanlan.zhihu.com/p/22940577

Normal Equation如何实现一步求解最优参数及其对比梯度下降的特点是什么？ - 深度碎片的回答 - 知乎 https://www.zhihu.com/question/273799498/answer/370173526

Part I Linear Regression

Try to train the hypothesis function \(h\).

Letting \(x_0=1\) gives the error term. \[h(x) = \sum_{i=0}^d{\theta_ix_i}=\theta^T x\]

Cost function w.r.t parameter \(\theta\):

\[J(\theta) = \frac12\sum_{i=1}^{n}{(h_\theta(x^{(i)}) - y^{(i)})^2}\]

Some common random variables

• Discrete random variables
  • Bernoulli(p)
  • Binomial(n, p)
  • Geometric(p)
  • Poisson(\(\lambda\)): \(p(x) = e^{-\lambda}\frac{\lambda^x}{x!}\), non-negative integers
• Continuous random variables
  • Uniform(a,b)
  • Exponential(\(\lambda\)): \(f(x) = \lambda e^{-\lambda x}, x\geq 0\); \(F(x) = e^{-\lambda x}, x\geq 0\)
  • Normal(\(\mu\), \(\sigma^2\))

Comment: some simulation methods 1. Inverse CDF technique : \(X = F^{-1}(U), U\sim unif(0,1)\) 2. Box Muller method for generating Gaussian

Two random variables

Expectation & Covariance

Link

CS229: Review of Linear Algebra

Operations and properties

Norms

Euclidean or \(l_2\) norm;

题目描述

给你一个括号字符串 s ，它只包含字符 '(' 和 ')' 。一个括号字符串被称为平衡的当它满足：

任何左括号 '(' 必须对应两个连续的右括号 '))' 。 左括号 '(' 必须在对应的连续两个右括号 '))' 之前。 比方说 "())"， "())(())))" 和 "(())())))" 都是平衡的， ")()"， "()))" 和 "(()))" 都是不平衡的。

你可以在任意位置插入字符 '(' 和 ')' 使字符串平衡。

请你返回让 s 平衡的最少插入次数。

Some hints on Midterm

Problem1: PSD: Random feature decomposition to prove

Problem2: Find a random feature map of the product of two kernels, angular might be negative

Data Mining Midterm

Hua Yao(UNI: hy2632)

Problem 1: Anisotropic Gaussian Kernels (50 points)

Given: \[K(x,y) = (2\pi)^{-\frac{d}{2}}(\det(\Sigma))^{-\frac{1}{2}}\exp(-\frac{1}{2}(x-y)^{\top}\Sigma^{-1}(x-y)), \] \(\Sigma \in \mathbb{R}^{d\times d}\) is positive definite symmetric.

Show that K does not need to be an RBF kernel (10 points)

Performers: variant of transformer

Random feature for different kernels Softmax: triangnometric, positive Orthogonal features construction: different ways(Givens, Hadamard, regular(GM), or even more), different renormalizations

concentration, computing variance of certain feature map, Cherbychev, concentration results

attention: ..., transformer, MLP, resnet, ...

Markov's inequality

Monte Carlo Approximation techniques

Evolutionary Strategies, 策略梯度

Policy optimization can be done through gradient ascend:

\[\nabla_\theta \mathbb{E}_{\epsilon\sim\mathcal{N}(0, I)} F(\theta + \sigma \epsilon) = \frac{1}{\sigma}\mathbb{E}_{\epsilon\sim\mathcal{N}(0, I)} [ \epsilon F(\theta + \sigma\epsilon) ]\]

Proof:

概述

Managerial Negotiation 第五周的案例，强调了在谈判前要发现双方共同利益、maximize value creation。在本案例情境中，谈判轮数增加会导致双方可变成本增加，且谈判核心-\(w\)值的增加会导致双方总福利的降低。在设计本模拟工具时也考虑了两种情况：

• 君子条约: 在罢工（即造成双方成本增加）开始前确定\(w\)在 \(0.5-0.52\) 范围内，对两者较为公平且最大化总价值
• 消耗战(War of Attrition): 通过模拟发现作为Kunzler(希望w小)，利用前两轮压价、第三轮结束通常能获得最好结果，但该结果仍比君子条约差；作为Arnold(希望w大)，第六轮后结果比君子条约差，但在此之前有较大的议价空间。没有达成君子条约默契并消耗至第六轮，最终结果\(w\)通常在0.6左右，Arnold小赚而Kunzler较亏

针对消耗战情况，又设计了一个函数find_Case来针对当前已知信息（轮数，己方出价，对方出价），找出一个模拟情况，从而对未来局势有所判断。

1

def find_Case(round, w_K, w_A)

Transformers

"Attention is all you need"

Attention Block

a sequence of feature vectors \(X = [x_1, ... x_L]^T \in \mathbb{R}^{L\times d}\) are fed into an attention block.

\(W_Q, W_K, W_V\)