Engineering faculty awarded NSF funding to improve computing system memory

Dorin Patru and Linlin Chen, faculty-researchers at Rochester Institute of Technology, received a grant from the National Science Foundation to upgrade functions of programmable memory. The research partners at RIT, along with colleagues from University of Rochester (UR), will develop new algorithms to improve the internal computing memory system to enable scalable and more robust performance.

Today’s computing systems depend on internal memory hierarchies—the complex network of processing, computation, and storage elements that allow computers to perform effectively. With several memory system levels that allow for access to stored information, researchers and system developers are continually working to better integrate these levels to improve access to information, particularly for computationally intensive simulations and applications.

“When it comes to applications that are computationally intensive, we need to execute faster,” said Patru, associate professor of electrical engineering in RIT’s Kate Gleason College of Engineering. “We are talking about applications which in a ‘slow’ computer may take a few weeks, or a few days, to complete these intensive computations, but we need to speed it up to a few hours or a few minutes. It is what the industry at large has been trying to do ever since its inception.”

Examples of intensive computations can be found in the simulations run to assess applications such as chemical reactions for vaccines and other drug research, or generating mathematical models to manage Big Data.

Many of these computationally intensive applications can be parallelized, meaning they can be separated to be computed in tandem, increasing throughput, or the units of information a system can process in a set time frame.

Patru and Linlin Chen, associate professor in RIT’s School of Mathematical Sciences in the College of Science, received a three-year, $228,000 grant for “Collaborative Research SHF Small: Programmable Hierarchical Caches – Design, Programming and Prototyping.” It is part of a larger research initiative led by Chen Ding, professor, UR Computer Science Department.

“Our methodologies not only improve those, but even more, they can improve the execution of sequential programs, those that cannot be parallelized today. They are the most difficult to tackle,” Patru said. “Hardware and software are often disconnected. We are trying to better integrate software and hardware together, for better performance.”

Ding agreed. “Since computer cost and speed depend on memory hierarchy, programmable designs can overcome the current limit in scalability and power efficiency. Beyond its technical content, the project advances teaching in the science of computer memory and strives to increase the diversity among participants in this area of research and development.”

Modern caches use reactive algorithms to determine which data to keep in the cache or to replace. The advantage is that these algorithms use run-time information. The disadvantage is that due to the limited availability of resources, tracking can only occur over a short period of the recent past.

The UR team developed programmable caches using lease-based algorithms. They are prescriptive and use compile-time information about program and data structures to determine how long data is stored in the cache. The advantage is that these algorithms can be applied at compile-time with little additional resource overhead at run-time.

The collaboration between the universities and co-investigators started more than three years ago, and several RIT graduate students work with Patru, including Ian Prechtl ’20 (electrical engineering) and current student Matthew Gould, who will graduate this December with a BS/MS degree in electrical engineering. The students were involved in the design, simulation, and implementation of a FPGA, a RISCV-based system with a reconfigurable cache.

“Our work ensures that the data we have in our cache, the memory, is the most relevant. We are trying to optimize computations, reduce errors, and speed up execution time,” said Gould. “With our algorithms, we are trying to bring in and keep the data that will be needed by the CPU most of the time.”

As demand for more computing power for more complex applications increases, through this new work, data can be better stored, accessed, and used more efficiently and reliably. The algorithms can eventually be integrated into new computer system designs and prototypes that can retain software portability, ensure matching performance to current automatic solutions by default, and provide precisely defined cache performance properties.