Continual representation learning for node classification in power-law graphs

The recent advent of node embedding techniques enabled a more efficient application of machine learning techniques on graphs. These techniques allow each node of a network to be encoded into an arbitrary low-dimensional vector representation, which can be exploited by statistical learning models. However, the main limitation of these approaches is that the embedding task is solved as an optimization problem on a static snapshot of the graph. In a real scenario, temporal dynamics should be considered with some consequences: new nodes might join the network and get a representation of only these new ones. As a consequence, a new training step over the entire graph is required. Even more, training models with static approaches can have resource-intensive requirements, especially when dealing with large networks. In light of this, a continual feature learning that builds on top of previously already learned knowledge (previous partial embedding of the network) and well-known properties can be a solution to address both limitations efficiently in real scenarios. Our approach is suitable for graphs whose degree distribution is described by a power-law function that is a common property of real systems. This research work presents three main scientific contributions: (a) a continual feature learning meta-algorithm for node embedding, which exploits properties of power-law distribution and spaces alignment techniques; It is suitable with any traditional node embedding techniques that relies on embedding spaces (b) we demonstrate empirically, by performing node labeling tasks, that a lightweight solution to encode new nodes, based on limited knowledge of the embedding of the network hub-nodes, can provide comparable or better performances, with respect to static approaches. (c) Finally, we experimented our algorithm in the temporal graphs domain and we achieved better results in node classification compared with other state of the art techniques.