One way to reduce the impact of data movement on real-time analytic solutions is to eliminate or significantly reduce the amount of data that is being moved. If the amount of data that needs to be moved in the example from our previous blog can be reduced by 95% (from 1PB to 50TB), some significant things happen. The 32-server cluster can now load the entire data set in roughly 26 seconds. Alternatively, the 32-server cluster [...]
About Vladimir AlvesThis author has not yet filled in any details.
So far Vladimir Alves has created 9 blog entries.
Of course, every memory-based analytics application workflow must start by moving data from storage or from I/O streams to server memory. For most meaningful problem sets, this requires that the dataset either be broken into pieces and loaded into memory sequentially. It takes a server with a 32-bit PCIe Gen4 bus can load a memory complex of 6TB (the largest size typically found in today’s servers) a little under 100 seconds. While that sounds slow, [...]
Believe it or not, business intelligence is an extremely old field of study. It was first mentioned in 1865 regarding a banker who utilized information about the outcome of battles to make strategic business decisions. Hans Peter Luhn of IBM published one of the earliest articles about business intelligence in 1958. Forrester Research defines business intelligence as “a set of methodologies, processes, architectures, and technologies that transform raw data into meaningful and useful information used [...]
In a sense, we are all spoiled in the technology world. Our environment is one of constant progress and change, and you can always say “if last year was great, next year will be even better!”, or “if you don’t like today’s hardware/software/solutions, wait until tomorrow”. It’s easy to become jaded at how quickly our technology is evolving, and to downplay any specific milestones as just being part of the overall advancement of technology. However, there [...]
If you read our blog from Tuesday, you know that NGD Systems was named as a "Cool Vendor" in storage technology by Gartner this month. As the Chief Technology Officer (CTO) for NGD Systems, this is especially meaningful to me. After all, this validates our technology and shows it is about cool technology, where NGD Systems is first and foremost focused. In our case, that technology is the paradigm shift in infrastructure called Computational Storage, and [...]
While BLAST represents a great bioscience use case for computational storage, it is by far not the only great bioscience use case. There are a variety of bioscience problems that are “parallel in nature” where massive data movement is a significant problem, and for which Computational Storage could have value. Radiomics, a field of medical study whose goal is to extract useful features from large numbers of medical images to improve diagnostics, is a perfect example [...]
One of the more important tools in biological computation today is known as the Basic Local Alignment Search Tool, or BLAST. The purpose of BLAST, which was developed by the National Institutes of Health (NIH), is to take a genetic protein sequence and compare it against a database of sequences. The output of BLAST is a list of sequences that are identical or similar to the query sequence. It does this by looking at “short words” [...]
Are biosciences and computation for bioscience still hot topics? In the early 2000s, the promise of bioscience was everywhere around us, and much of it was powered by advances in computation and big data. The sequencing of the human genome in 2002 is but one example of how computation, enabled by significant reductions in the cost of DNA sequencing, has accelerated bioscience. Yet, this trend seemingly gave way (or at least press time) to other computationally-accelerated [...]
Vladimir Alves, CTO (July 12, 2018) - Good news! Our Catalina-2 intelligent storage devices have been enabled to natively support containers, enabling even more applications to run with near-data processing. Our R&D team has been working on providing support for containers, starting with Docker. Containerized applications can now run seamlessly on the Catalina-2 NVMe SSD enabling near-data computing. We are confident that combining In-Situ Processing, Docker containerization all on a 16TB SSD provides a solid platform, [...]