Highlight

Automatic Acquisition of Hyponyms from Large Text Corpora

Introduces lexico-syntactic patterns (Hearst patterns) to automatically extract hyponymy relations from unrestricted text.

This paper describes a method for automatically acquiring the hyponymy lexical relation from unrestricted text, motivated by avoiding pre-encoded knowledge and broad applicability across genres. It identifies lexico-syntactic patterns that are easily recognizable, frequent, cross-genre, and reliably indicate hyponymy, and describes how to discover them, suggesting other lexical relations could be acquired similarly. A subset of the algorithm is implemented, and its results augment and critique a large hand-built thesaurus, with extensions suggested for areas like information retrieval.

Based on: Automatic Acquisition of Hyponyms from Large Text Corpora · International Conference on Computational Linguistics

Curated by Aramai Editorial

Read summary →
Highlight

Knowledge Graph Embedding by Translating on Hyperplanes

Proposes TransH, a knowledge graph embedding that models each relation as a hyperplane with a translation to capture complex mapping types.

The paper embeds knowledge graphs of entities and relations into a continuous vector space. The efficient TransE handles reflexive and one-to-many, many-to-one, and many-to-many relations poorly, while richer models fix this but sacrifice efficiency. TransH represents each relation as a hyperplane with a translation on it, preserving these mapping properties at nearly TransE's complexity, plus a sampling trick that cuts false-negative labels. On WordNet and Freebase—link prediction, triplet classification, fact extraction—it significantly improves accuracy over TransE.

Based on: Knowledge Graph Embedding by Translating on Hyperplanes · AAAI Conference on Artificial Intelligence

Curated by Aramai Editorial

Read summary →
Highlight

DeBERTa: Decoding-enhanced BERT with Disentangled Attention

Proposes DeBERTa, improving BERT/RoBERTa with disentangled attention over content and position plus an enhanced mask decoder.

DeBERTa (Decoding-enhanced BERT with disentangled attention) improves BERT and RoBERTa via two techniques. Disentangled attention represents each word with separate content and position vectors, computing attention using disentangled matrices over contents and relative positions. An enhanced mask decoder replaces the output softmax to predict masked tokens in pretraining. Trained on half the data, DeBERTa beats RoBERTa-Large on MNLI (+0.9%), SQuAD v2.0 (+2.3%), and RACE (+3.6%). Code and models are released publicly.

Based on: DeBERTa: Decoding-enhanced BERT with Disentangled Attention · International Conference on Learning Representations

Curated by Aramai Editorial

Read summary →
Highlight

Multi-column deep neural networks for image classification

Presents multi-column deep convolutional networks that reach near-human image classification accuracy on MNIST and surpass humans on traffic signs.

Traditional computer vision and machine learning cannot match humans at recognizing handwritten digits or traffic signs. They build wide, deep, biologically plausible networks whose convolutional winner-take-all neurons use small receptive fields, creating many sparsely connected layers with only winner neurons trained. Several deep columns become experts on differently preprocessed inputs, and their averaged predictions, trained quickly on GPUs, achieve the first near-human MNIST accuracy, beat humans twofold on traffic signs, and improve many image benchmarks.

Based on: Multi-column deep neural networks for image classification · 2012 IEEE Conference on Computer Vision and Pattern Recognition

Curated by Aramai Editorial

Read summary →
Highlight

Deep Neural Networks for YouTube Recommendations

Describes YouTube's deep learning recommendation system split into deep candidate generation and deep ranking stages.

This paper describes YouTube's large-scale, sophisticated industrial recommendation system and the substantial performance gains brought by deep learning. Following the classic two-stage information retrieval structure, it details a deep candidate generation model and then a separate deep ranking model. The authors also share practical lessons and insights from designing, iterating on, and maintaining a massive recommendation system with enormous user-facing impact.

Based on: Deep Neural Networks for YouTube Recommendations · ACM Conference on Recommender Systems

Curated by Aramai Editorial

Read summary →
Highlight

Quantum Mechanics Helps in Searching for a Needle in a Haystack

Introduces a quantum search algorithm that finds an item in an unsorted database of N entries using only about sqrt(N) accesses.

Quantum mechanics can accelerate search over unsorted data. Finding a target in a randomly ordered N-entry phone directory with 50% probability requires any classical algorithm at least 0.5N database accesses. Because quantum systems can occupy a superposition of states, they examine many entries at once; by tuning the phases of operations, correct computations reinforce while others interfere randomly. This yields the answer in only about O(sqrt(N)) database accesses.

Based on: Quantum Mechanics Helps in Searching for a Needle in a Haystack · arXiv

Curated by Aramai Editorial

Read summary →
Highlight

Face recognition by elastic bunch graph matching

Presents a face recognition system representing faces as labeled Gabor-wavelet graphs matched elastically against a database.

This work presents a system for recognizing human faces from single images against a large database holding one image per person. Faces are represented as labeled graphs based on a Gabor wavelet transform, and graphs for new faces are extracted via an elastic graph matching process, then compared with a simple similarity function. It extends prior work in three ways: phase information enables accurate node positioning, object-adapted graphs handle large rotations in depth, and image graph extraction relies on a novel data structure, the bunch graph, built from a small set of sample graphs.

Based on: Face recognition by elastic bunch graph matching · Proceedings of International Conference on Image Processing

Curated by Aramai Editorial

Read summary →
Highlight

Recommender Systems

Surveys recommender systems, their types, problems, and future directions from over 1,000 papers published 2011 to early 2020.

This paper provides an overview of recommender systems (RS), covering their types, common problems, and future scope. Its main aim is to identify research trends in RS, drawing on more than 1,000 research papers published by ACM, IEEE, Elsevier, and Springer from 2011 through the first quarter of 2020. The review surfaces several interesting findings intended to help current and future researchers assess and set their research roadmaps. It also envisions the future of recommender systems and potential new research directions.

Based on: Recommender Systems · Wirtschaftsinf.

Curated by Aramai Editorial

Read summary →
Highlight

Self-Refine: Iterative Refinement with Self-Feedback

Introduces Self-Refine, a training-free method where one LLM iteratively improves its own outputs via self-generated feedback.

Self-Refine improves LLM outputs through iterative self-feedback and refinement, mirroring how humans revise their writing. A single LLM generates an initial output, then provides feedback on it and uses that feedback to refine itself, requiring no supervised data, extra training, or reinforcement learning. Evaluated across 7 diverse tasks on GPT-3.5, ChatGPT, and GPT-4, Self-Refine outputs are preferred by humans and automatic metrics, improving performance by about 20% absolute on average. The work shows even top LLMs like GPT-4 can be improved at test time.

Based on: Self-Refine: Iterative Refinement with Self-Feedback · Neural Information Processing Systems

Curated by Aramai Editorial

Read summary →
Highlight

Holistically-Nested Edge Detection

Introduces HED, a deep learning edge detector using fully convolutional and deeply-supervised nets for image-to-image boundary prediction.

HED (holistically-nested edge detection) tackles edge detection with holistic image-to-image prediction and multi-scale, multi-level feature learning. It uses fully convolutional networks combined with deeply-supervised nets, learning rich hierarchical representations guided by deep supervision on side responses to resolve boundary ambiguity. HED advances state-of-the-art results on BSDS500 (ODS F-score 0.790) and NYU Depth (0.746) while running at 0.4s per image, far faster than prior CNN-based methods. It also shows encouraging results on Multicue and PASCAL-Context benchmarks.

Based on: Holistically-Nested Edge Detection · International Journal of Computer Vision

Curated by Aramai Editorial

Read summary →
Highlight

Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback

Applies preference modeling and RLHF to finetune language models into helpful and harmless assistants, using iterated online updates from human feedback.

The authors apply preference modeling and reinforcement learning from human feedback (RLHF) to finetune language models into helpful and harmless assistants. This alignment training improves almost all NLP evaluations and stays compatible with skills like Python coding and summarization. They explore an iterated online mode where preference models and RL policies are updated weekly with fresh human feedback. Studying robustness, they find a roughly linear relation between RL reward and the square root of the KL divergence between the policy and its initialization.

Based on: Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback · arXiv.org

Curated by Aramai Editorial

Read summary →
Highlight

KEGG for taxonomy-based analysis of pathways and genomes

Updates KEGG with taxonomy-based pathway and genome analysis, a new genome browser, improved KO annotation, and Brite taxonomy mapping.

KEGG is a manually curated database integrating biological objects across systems, genomic, chemical, and health information, each identified by a KEGG identifier (kid). The pathway viewer, Brite hierarchy viewer, and a new genome browser each launch via URL paths. An improved KO (KEGG Orthology) annotation procedure adds more eukaryotic genomes for better taxonomic representation. New taxonomy files classify organisms and viruses, and Brite-based taxonomy mapping in the new KEGG Mapper suite reveals how functional and physical gene links are conserved across organisms.

Based on: KEGG for taxonomy-based analysis of pathways and genomes · Nucleic Acids Res.

Curated by Aramai Editorial

Read summary →