Learning Entity and Relation Embeddings for Knowledge Graph Completion阅读笔记

TransR embeds entities and relations in distinct entity space and relation space, and learns embeddings via translation between projected entities.CTransR models internal complicated correlations within each relation type.

# Problem Statement

In fact, an entity may have multiple aspects and various relaitons may focus on different aspects of entities, which makes a common space insurficient for modeling.

# Contribution

• propose a TransR model which models entities and relations in distinct spaces
• CTransR models internal complicated correlations within each relation type.
• experiment on benchmark datasets of WordNet and Freebase and gain consistent improvements compared to state-of-the-art models

# Future work

• Existing models including TransR consider each relational fact separately.
• relation transitive
• explore a unified embedding model of both text side and knowledge graph
• modeling internal correlations within each relation type

# TransR 1. for each triple$(h, r, t)$, entities embeddings are set as $\mathbf{h}, \mathbf{t} \in \mathbb{R}^{k}$ and relation embedding is set as $\mathbf{r} \in \mathbb{R}^{d}$, $k \neq d$

2. for each relation $r$, set a projection matrix $\mathbf{M}_{r} \in\mathbb{R}^{k \times d}$

• projects entities from entity space to relation space
3. projected vectors of entities as

4. score function:

# Cluster-based TransR (CTransR)

### why propose CTransR

TransE, TransH and TransR, learn a unique vector for each relation, which may be under-representative to fit all entity pairs under this relation, because these relations are usually rather diverse.

### basic idea

• incorporate the idea of piecewise linear regression Ritzema and others 1994
• segment input instances into several groups

### process

1. for a specific relation r, all entity pairs (h, t) in the training data are clustered into multiple groups, and entity pairs in each group are expected to exhibit similar r relation.

• All entity pairs (h, t) are represented with their vector offsets (h − t) for clustering, where h and t are obtained with TransE.
2. learn a separate relation vector $r_c$for each cluster and matrix $M_r$ for each relation, respectively

3. projected vectors of entities as $\mathbf{h}_{r, c}=\mathbf{h} \mathbf{M}_{r} \text { and } \mathbf{t}_{r, c}=\mathbf{t} \mathbf{M}_{r}$

4. sorce fuction

the later item aims to ensure cluster-specific relation vector rcnot too far away from the original relation vector r

# dataset

Dataset #Rel #Ent #Train #Valid # Test
WN18 18 40,943 141,442 5,000 5,000
FB15K 1,345 14,951 483,142 50,000 59,071
WN11 11 38,696 112,581 2,609 10,544
FB13 13 75,043 316,232 5,908 23,733
FB40K 1,336 39528 370,648 67,946 96,678

# Experiment

### Triple classification

Moreover, the “bern” sampling technique improves the performance of TransE, TransH and TransR on all three data sets.

bern采样方法需要掌握。

### Relation Extraction from Text

上一篇 DocRED A Large-Scale Document-Level Relation Extraction Dataset阅读笔记

2019-07-01 Neural Relation Extraction with Selective Attention over Instances阅读笔记

2019-06-26
目录