bookBlog

Zanzibar is an authorization system developed at Google and used by more than 1500 client services, including Calendar, Cloud, Drive, Maps, Photos, and Youtube

FoundationDB is an open source key-value store and one of the first systems to combine the flexibility and scalability of NoSQL architectures with the power of ACID transactions.

An LSM-Tree is at the heart of storage engines that are based on the principle of merging and compacting sorted files. Such storage engines are typically called LSM storage engines.

Many databases internally use a log which is an append-only sequence of records. These are log-structured databases. Hash indexes are used to index key-value data, and may be used to power log-structured databases.

A data-intensive application is one for which raw CPU power is rarely a limiting factor — it has bigger concerns over the amount of data, complexity of data and the speed at which the data is changing.

Information Retrieval (IR) is the science of searching for material in a collection of resources based on an information need. The most common form of an IR system is a text retrieval system that allows users to retrieve textual data in the form of a sorted list of documents using natural language queries.

ZooKeeper is a service that allows distributed processes to coordinate with each other using a shared name space of data registers. It exposes a wait-free interface and an event-driven mechanism to provide a simple and high-performance kernel for building distributed applications. Originally developed at Yahoo!, it is now under the care of the Apache Software Foundation.

This post is a summary of the paper: Epidemic algorithms for replicated database maintenance, Demers, et al., 1988 Introduction Published in the late 80’s, this paper lays out early ideas on gossip based replication algorithms and focuses on eventual consistency, in contrast to the traditional ACID model. The idea of eventual consistency or BASE is well-known these days, however, in the late 80’s, it was one of the most important novel ideas.

This post is a summary of the paper: Web-Scale Responsive Visual Search at Bing, Hu et al., 2018 Introduction Visual search is an interesting research area. A visual search system ranks a list of visually similar images when presented with a query image. As in all search systems, latency and relevance of returned results are key metrics to evaluate such a system.

Background C++ is an old language. The excitement that C++ generated in the 90’s had faded by 2010, probably because of the emergence of several newer languages like Java and also because the Standards Committee had released only a few enhancements in the decade. And then came C++11. The specification for C++11 was approved in 2011 (hence, the name).

Peloton is Uber’s cluster scheduler that is capable of co-scheduling mixed types of workloads such as batch, stateless and stateful jobs in a single cluster for better resource utilization. Designed for scaling to millions of containers and tens of thousands of nodes, it features advanced resource management capabilities like elastic resource sharing, hierarchical max-min fairness, workload preemption to name a few.

This is a well-known paper. It was published in 1998 and it describes Google as a prototype of a large-scale search engine. According to the authors, this paper was the first in-depth public description of a large-scale web search engine. The authors describe how to build a practical large-scale search system that exploits information present in hypertext, thus producing better search results than existing search systems.